Marvimundo scraping

[johndoe]

Hello everyone,
I have an issue with a scraping task as I don't succeed in scraping the landing page properly. I perfectly scrape 69 stores out of 70, but the first one sucks.
The idea would be 1) check if there is content like name, address.. 2) click on link to go to the next store, but I don't succeed in implementing this.

Any idea to help me please ?

Url: http://www.marvimundoperfumerias.com/perfumerias/result.php?t=1

Sitemap:
{"startUrl":"http://www.marvimundoperfumerias.com/perfumerias/result.php?t=1","selectors":[{"parentSelectors":["_root"],"type":"SelectorLink","multiple":true,"id":"link","selector":"dt:nth-of-type(n+2) a","delay":""},{"parentSelectors":["link"],"type":"SelectorText","multiple":false,"id":"name","selector":"h3","regex":"","delay":""},{"parentSelectors":["link"],"type":"SelectorText","multiple":false,"id":"address","selector":"p.contenido","regex":"","delay":""},{"parentSelectors":["link"],"type":"SelectorText","multiple":false,"id":"zip_code","selector":"p.contenido","regex":"([1-9][0-9]|0?[1-9])[0-9]{3}","delay":""},{"parentSelectors":["_root"],"type":"SelectorText","multiple":false,"id":"name2","selector":"h3","regex":"","delay":""},{"parentSelectors":["_root"],"type":"SelectorText","multiple":false,"id":"address2","selector":"p.contenido","regex":"","delay":""},{"parentSelectors":["_root"],"type":"SelectorText","multiple":false,"id":"zipcode2","selector":"p.contenido","regex":"([1-9][0-9]|0?[1-9])[0-9]{3}","delay":""}],"_id":"marvimundo"}

[johndoe]

@iconoclast any idea please ?

[iconoclast]

Hi!

The list of perfume shops contains 69 of them, not 70. It's the text that says it, yet links lead only to 69.

I've simplified your sitemap:

{"_id":"marvimundo2","startUrl":["http://www.marvimundoperfumerias.com/perfumerias/result.php?t=1"],"selectors":[{"id":"link","type":"SelectorLink","selector":"dt a","parentSelectors":["_root"],"multiple":true,"delay":"1000"},{"id":"name","type":"SelectorText","selector":"h3","parentSelectors":["group"],"multiple":false,"regex":"","delay":""},{"id":"address","type":"SelectorText","selector":"p.contenido","parentSelectors":["group"],"multiple":false,"regex":"","delay":""},{"id":"zip_code","type":"SelectorText","selector":"p.contenido","parentSelectors":["group"],"multiple":false,"regex":"\\d{5}","delay":""},{"id":"group","type":"SelectorElement","selector":"div.agua","parentSelectors":["link"],"multiple":true,"delay":0}]}

[johndoe]

Thank's @iconoclast yep i know this, because the last shop can't be reached through clicking. You reach it when you land on the main page. This is the first item at the top.
I hadn't any issue scraping the 69 others

[iconoclast]

Well, if you want the 1st page too, the only solution that will work is creating a sitemap that contains just links to shops, otherwise you'll have 1st page present in resulting data for all shops.

[johndoe]

@iconoclast I'm not sure you get my point.
We have 2 kinds of formatting right there:

1. One with links for 69 out of 70 shops. No problem with this.
2. Another formatting for the first shop you can see when you land on the page with all the shops which is located Calle reyes católicos in Alicante.

How can I also scrape this one in Alicante please ?

[iconoclast]

@johndoe I got your point
Fortmatting is the same across all perfume shops. I ment, that in order to get all 70, you have to make a sitemap with URLs of all shops in it, that means you have to add URL of each shop in sitemap metadata. It can be easily done if you have a macro for text editor to prepare URLs for importing into WebScraper.

Otherwise it's easier to add the very first one manually, unless there is a button on a website that leads to the first shop.

UPD: here's the full sitemap with all shops that will result in all 70 being scraped.

{"_id":"marvimundo3","startUrl":["http://www.marvimundoperfumerias.com/perfumerias/result.php?e=55&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=26&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=2&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=62&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=25&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=24&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=20&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=12&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=77&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=81&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=83&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=85&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=63&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=64&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=90&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=43&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=65&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=41&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=67&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=87&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=21&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=17&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=60&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=33&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=91&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=32&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=29&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=44&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=38&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=28&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=52&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=19&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=76&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=66&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=59&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=57&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=15&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=61&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=22&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=35&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=58&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=10&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=13&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=86&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=9&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=82&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=14&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=47&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=75&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=69&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=74&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=49&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=34&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=11&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=18&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=51&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=40&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=68&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=78&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=92&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=84&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=46&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=56&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=88&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=27&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=79&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=54&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=37&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?e=36&p=&s=0&l=&f=0","http://www.marvimundoperfumerias.com/perfumerias/result.php?t=1"],"selectors":[{"id":"Shop","type":"SelectorText","selector":"h3","parentSelectors":["_root"],"multiple":false,"regex":"","delay":0},{"id":"Address","type":"SelectorText","selector":"p.contenido","parentSelectors":["_root"],"multiple":false,"regex":"","delay":0},{"id":"ZIP","type":"SelectorText","selector":"p.contenido","parentSelectors":["_root"],"multiple":false,"regex":"\\d{5}","delay":0}]}