Hi,
I'm trying to scrape off www.real valueiga.com/shop (remove the space) and while I've created Link selectors for every sub-level of the menu, it never goes go to the lowest level.
The thing is, the site already displays items on the 2nd level menu, even if this said menu has a 3rd or 4th level below. In the spreadsheet, products will naturally be displayed at this 2nd level and not at the 3rd or 4th level.
My sitemap :
{"_id":"test_iga","startUrl":["https://www.realvalueiga.com/shop/"],"selectors":[{"id":"Click L1 categories","type":"SelectorLink","parentSelectors":["_root"],"selector":".fp-has-children a[data-id='21891483']","multiple":true,"delay":0},{"id":"Click L2 categories","type":"SelectorLink","parentSelectors":["Click L1 categories"],"selector":"a[data-depth='1']","multiple":true,"delay":0},{"id":"Click L3 categories","type":"SelectorLink","parentSelectors":["Click L2 categories"],"selector":"a[data-depth='2']","multiple":true,"delay":0},{"id":"Click L4 categories","type":"SelectorLink","parentSelectors":["Click L3 categories"],"selector":".fp-nav-3 a","multiple":true,"delay":0},{"id":"Item wrapper","type":"SelectorElement","parentSelectors":["Click L1 categories","Click L2 categories","Click L3 categories","Click L4 categories"],"selector":"ul.fp-result-list","multiple":false,"delay":0},{"id":"Item","type":"SelectorLink","parentSelectors":["Item wrapper"],"selector":".fp-item-name a","multiple":true,"delay":0},{"id":"Name","type":"SelectorText","parentSelectors":["Item"],"selector":"h1","multiple":false,"regex":"","delay":0}]}
P.S. You need to max up the default delay of 2000ms as the pages take long to load. I do 9000ms.
Thank you if you have any idea.