I'm trying to scrap data from a website that has product information.
It shows 50+ products per page and pagination button contains prev. next bottons as well
I've done two methods and both are not working properly.
-
Containing all pagination selector ('li.pagination-box__item a' in this website) will get all pages in first 1~10 pages but when it move to next button, it somehow recognize prev as 10th so it goes back and end up collecting data
-
After 1st method failed I've tried :nth-of-type(n+2) (and also ive tried+3) it gets me as far as 20th page but it won't go next where there are more pages after next button. (I'm assuming that after 1~10 page there was next button selected so the program won't go to next page after 11~20 <--I'm assuming this because I've tried to hard type all pages by number but prev. button was selected as well)
plz help me on this I'm struggling to collect data from this website for days now.
Url: PC견적 : 샵다나와
Sitemap:
{"_id":"DANAWA_NEW","startUrl":["PC견적 : 샵다나와 a","type":"SelectorPagination"},{"id":"ELEMENT_WRAPER","multiple":true,"parentSelectors":["PAGINATION"],"selector":"tr","type":"SelectorElement"},{"id":"PRODUCT","multiple":false,"parentSelectors":["ELEMENT_WRAPER"],"regex":"","selector":".subject a","type":"SelectorText"},{"id":"PRICE","multiple":false,"parentSelectors":["ELEMENT_WRAPER"],"regex":"","selector":"span.prod_price","type":"SelectorText"},{"id":"DETAIL","multiple":false,"parentSelectors":["ELEMENT_WRAPER"],"regex":"","selector":"a.spec","type":"SelectorText"}]}