Interrupted scraping

#1

Scraping stops on the second page. How to properly build pagination on this site ?

{"_id":"rossmann_cialo","startUrl":["https://www.rossmann.pl/kategoria/pielegnacja-i-higiena/cialo,13093"],"selectors":[{"id":"strony","parentSelectors":["_root","strony"],"paginationType":"auto","type":"SelectorPagination","selector":"[data-testid=\"pagination-next-page\"]:not([aria-disabled=\"true\"])"},{"id":"produkty","parentSelectors":["strony"],"type":"SelectorElement","selector":"[style*=\"grid-template-columns\"] > div","multiple":true},{"id":"nazwa","parentSelectors":["produkty"],"type":"SelectorText","selector":"div.DescriptionSection-module_brand--YWvzf","multiple":false,"regex":""},{"id":"nazwa_","parentSelectors":["produkty"],"type":"SelectorText","selector":"div.DescriptionSection-module_caption--EuQOh","multiple":false,"regex":""},{"id":"nazwa__","parentSelectors":["produkty"],"type":"SelectorText","selector":"div.DescriptionSection-module_unit--ASwix","multiple":false,"regex":""},{"id":"status","parentSelectors":["produkty"],"type":"SelectorText","selector":"span.ProductTileBadge-module_productTileBadge--BepfZ","multiple":false,"regex":""},{"id":"chwilowo_niedostepny","parentSelectors":["produkty"],"type":"SelectorText","selector":"div.AvailabilitySection-module_record--6s-zJ:nth-of-type(1)","multiple":false,"regex":""},{"id":"cena_aktualna","parentSelectors":["produkty"],"type":"SelectorText","selector":"div.PriceSection-module_priceBox--Z-c6z","multiple":false,"regex":""},{"id":"cena_bez_apki","parentSelectors":["produkty"],"type":"SelectorText","selector":"div.RecentlyIntroducedPrice-module_text--kKjD6","multiple":false,"regex":""},{"id":"cena_30dni","parentSelectors":["produkty"],"type":"SelectorText","selector":"div.LowestPriceIn30Days-module_text--hJ38g","multiple":false,"regex":""},{"id":"cena_regularna","parentSelectors":["produkty"],"type":"SelectorText","selector":"div.PriceSection-module_priceBox--Z-c6z","multiple":false,"regex":""},{"id":"link","parentSelectors":["produkty"],"type":"SelectorLink","selector":".ImageSection-module_imageSection--2Hl82 a","multiple":false,"linkType":"linkFromHref"}]}

Thank you for reply.

#2

You followed wrong way. No pagination is needed. You can regulate the number of pages to be scraped by changing start URL inside brackets: Page=[1-10]... this will check from page1 to page 10... no problem at all..
Also, you used not proper selectors, should use universal ones.

Check it out:

{"_id":"rossmann_cialo","startUrl":["https://www.rossmann.pl/kategoria/pielegnacja-i-higiena/cialo,13093?Page=[1-10]"],"selectors":[{"id":"produkty","multiple":true,"parentSelectors":["_root"],"selector":"[style*=\"grid-template-columns\"] > div","type":"SelectorElement"},{"id":"nazwa","multiple":false,"parentSelectors":["produkty"],"regex":"","selector":"div[class*=\"DescriptionSection-module_brand\"]","type":"SelectorText"},{"id":"nazwa_","multiple":false,"parentSelectors":["produkty"],"regex":"","selector":"div[class*=\"DescriptionSection-module_caption\"]","type":"SelectorText"},{"id":"nazwa__","multiple":false,"parentSelectors":["produkty"],"regex":"","selector":"div[class*=\"DescriptionSection-module_unit\"]","type":"SelectorText"},{"id":"status","multiple":false,"parentSelectors":["produkty"],"regex":"","selector":"span[class*=\"ProductTileBadge\"]","type":"SelectorText"},{"id":"chwilowo_niedostepny","multiple":false,"parentSelectors":["produkty"],"regex":"","selector":"div[class*=\"AvailabilitySection-module_text\"]:contains(\"online\")","type":"SelectorText"},{"id":"cena_aktualna","multiple":false,"parentSelectors":["produkty"],"regex":"","selector":"span[class*=\"PriceSection-module_price-\"]","type":"SelectorText"},{"id":"cena_bez_apki","multiple":false,"parentSelectors":["produkty"],"regex":"","selector":"span[class*=\"RecentlyIntroducedPrice-module_price\"]","type":"SelectorText"},{"id":"cena_30dni","multiple":false,"parentSelectors":["produkty"],"regex":"","selector":"span[class*=\"LowestPriceIn29Days\"]","type":"SelectorText"},{"id":"link","linkType":"linkFromHref","multiple":false,"parentSelectors":["produkty"],"selector":"[class*=\"ImageSection\"] a","type":"SelectorLink"}]}