Scraping shows "not yet data"

How to?
#1

Describe the problem : scraping goes well, though "refresh button" says "not yet data"... And there is no data in the CSV file exported... How could I collect the companies directory ?

Url: http://www.journaldunet.com/web-tech/fonds/index/investisseurs

Sitemap:
{"_id":"vcfrancelist","startUrl":["http://www.journaldunet.com/web-tech/fonds/index/investisseurs"],"selectors":[{"id":"startinglist","type":"SelectorLink","parentSelectors":["_root"],"selector":"div.odGlossary a","multiple":true,"delay":0},{"id":"perletterlist","type":"SelectorLink","parentSelectors":["startinglist"],"selector":"div.odGlossary a","multiple":true,"delay":0},{"id":"companies link","type":"SelectorLink","parentSelectors":["perletterlist"],"selector":"ul.bloc a","multiple":true,"delay":0},{"id":"company name","type":"SelectorText","parentSelectors":["companies link"],"selector":"h1","multiple":false,"regex":"","delay":0},{"id":"board members","type":"SelectorText","parentSelectors":["companies link"],"selector":"div.hidden:nth-of-type(7) p","multiple":false,"regex":"","delay":0}]}

@iconoclast ? I could see your answer to a similar problem, but not exactly the same...

#2

Hey,

Is there any particular reason you have selected the top URLs twice? As that might be the reason your sitemap was not working.

Try this;

{"_id":"vcfrancelist","startUrl":["http://www.journaldunet.com/web-tech/fonds/index/investisseurs"],"selectors":[{"id":"perletterlist","type":"SelectorLink","parentSelectors":["_root"],"selector":"div.odGlossary a","multiple":true,"delay":0},{"id":"companies link","type":"SelectorLink","parentSelectors":["perletterlist"],"selector":"ul.bloc a","multiple":true,"delay":0},{"id":"company name","type":"SelectorText","parentSelectors":["companies link"],"selector":"h1","multiple":false,"regex":"","delay":0},{"id":"board members","type":"SelectorText","parentSelectors":["companies link"],"selector":"div.hidden:contains('Dirigeant(s)')","multiple":false,"regex":"","delay":0}]}

P.S. not quite sure if you where trying to select the board director list of the company, but I have fixed that, otherwise the selector will select anything that is in the 7th position of the page and unfortunately, it varies and you will not get the desired information.

#3

thank you @webber as it works now :slightly_smiling_face:
I'm not a developper, and it's my first time using web scraping, which remains quite possible for people like me

#4

I could get only 60 line out of about 250. Shall I give more time for each page ? Like 5000 ms ?
@webber

#5

I could only see 60 lines all together, do you have a link to the section, that has the 250?

#6

ok, let me check this out. They announce 250 companies, but they may have only 60 lines fulfilled...