Popup link navigation

How to?
#1

Hi,

I'm trying to wrap my head around whats going on here.

I'm building a scraper to work on a e-commer site where start URL i set to a category page.

Then I want want the scraper to open up a popup link and navigate to the subcategory pages and start gathering productURLs.

Howverer when I run my scraper it stops after opening up the popup. It never navigates to the sub-categories.

Pliz help me!

Url: https://www.weekday.com/en_sek/men/categories/accessories.html

Sitemap:
{"_id":"weekday3","startUrl":["https://www.weekday.com/en_sek/men/categories/accessories.html"],"selectors":[{"id":"subcategorydropdown","type":"SelectorPopupLink","parentSelectors":["_root"],"selector":"div.filters-content div.o-filter:nth-of-type(1) label.a-label","multiple":false,"delay":0},{"id":"subcatlink","type":"SelectorLink","parentSelectors":["subcategorydropdown"],"selector":"li.a-option:nth-of-type(n+2) a.a-link","multiple":true,"delay":0},{"id":"productlink","type":"SelectorLink","parentSelectors":["subcatlink"],"selector":"div.small-tiles div.o-product > a.a-link, div.o-category-listing > div.o-product > a.a-link","multiple":true,"delay":0}]}

#2

Hi!

First of all you shouldn't have used Popup link selector, as when you click on 'Show items' button it will draw a list of categories, not a new window/popup that this selector was designer for.

It can be done in a few ways. Since list of items can be directly accessed, you can add Link selector and assign all category links to it using this selector (it will skip 'All' category):

ul.options li:not(:first-child) a

image
image.png1038×846 110 KB

Here's corrected sitemap:

{"_id":"weekday3","startUrl":["https://www.weekday.com/en_gbp/men/categories/accessories.html"],"selectors":[{"id":"subcategorydropdown","type":"SelectorLink","selector":"ul.options li:not(:first-child) a","parentSelectors":["_root"],"multiple":true,"delay":"2000"},{"id":"productlink","type":"SelectorLink","selector":"div.small-tiles div.o-product > a.a-link, div.o-category-listing > div.o-product > a.a-link","parentSelectors":["subcategorydropdown"],"multiple":true,"delay":0}]}
#3

Thank you for your help, and your quick reply!