Hello,
I'm trying to extract the data of all movies and the review data available in Amazon.
Url: https://www.amazon.com/s?i=instant-video&qid=1723972396&ref=sr_pg_1
Problem : The scraping happens for 2 pages and then repeats again and stops. It doesn't move to other pages. I tried including pagination also but wasn't working and that is why went with [0-n] in the URL type.
The reviews of each movie here are also in pagination which is working fine.
Sitemap:
{"id":"Movie_database","startUrl":["https://www.amazon.com/s?i=instant-video&page=2&qid=1723226415&ref=sr_pg[0-400]"],"selectors":[{"id":"Movie_link","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":".a-size-mini a","type":"SelectorLink"},{"id":"Review_count","multiple":false,"parentSelectors":["Movie_link"],"regex":"","selector":"span._1Y_bt3","type":"SelectorText"},{"id":"Release_year","multiple":false,"parentSelectors":["Movie_link"],"regex":"","selector":"span[aria-label='Released 2024']","type":"SelectorText"},{"id":"Review_link","linkType":"linkFromHref","multiple":false,"parentSelectors":["Movie_link"],"selector":"a.dg8uVH","type":"SelectorLink"},{"id":"Review_pagination","paginationType":"auto","parentSelectors":["Review_link","Review_pagination"],"selector":".a-last a","type":"SelectorPagination"},{"id":"Review_element","multiple":true,"parentSelectors":["Review_pagination"],"selector":".a-row div.a-section.celwidget","type":"SelectorElement"},{"id":"Review_title","multiple":false,"parentSelectors":["Review_element"],"regex":"","selector":".a-size-base span:nth-of-type(2)","type":"SelectorText"},{"id":"Review_detail","multiple":false,"parentSelectors":["Review_element"],"regex":"","selector":".review-text span","type":"SelectorText"},{"id":"Review_date","multiple":false,"parentSelectors":["Review_element"],"regex":"","selector":"span.a-color-secondary","type":"SelectorText"},{"id":"Reviewer_name","multiple":false,"parentSelectors":["Review_element"],"regex":"","selector":"span.a-profile-name","type":"SelectorText"},{"id":"Review_rating","multiple":false,"parentSelectors":["Review_element"],"regex":"","selector":"span.a-icon-alt","type":"SelectorText"}]}