ElementClick & Pagination

Hi, complete newb here trying to scrape mobile phone price data and struggling to get the pagination to work. The sitemap code below doesn't include any attempt at pagination, because I've got the "see more results" clicks to work and I just can't figure where to start with pagination as well. Thanks so much.

https://www.envirofone.com/en-gb/buy/apple/search?type=MPHONE

Sitemap:
{"_id":"envirofone_sales_apple_v1","startUrl":["Refurbished Apple iPhones & iPads | Buy Cheap Second Hand Apple Devices | Envirofone Shop a","multiple":true,"delay":0},{"id":"product_wrapper","type":"SelectorElementClick","parentSelectors":["model"],"selector":"div#InStockProducts","multiple":true,"delay":2000,"clickElementSelector":"#see-more-link > span","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"model_description","type":"SelectorText","parentSelectors":["model"],"selector":"h1[itemprop='name']","multiple":false,"regex":"","delay":0},{"id":"description_1","type":"SelectorText","parentSelectors":["product_wrapper"],"selector":"div.row:nth-of-type(1) div.col-xs-8","multiple":false,"regex":"","delay":0},{"id":"description_2","type":"SelectorText","parentSelectors":["product_wrapper"],"selector":"div.row:nth-of-type(2) div.col-xs-8","multiple":false,"regex":"","delay":0},{"id":"description_3","type":"SelectorText","parentSelectors":["product_wrapper"],"selector":"div.row:nth-of-type(3) div.col-xs-8","multiple":false,"regex":"","delay":0},{"id":"description_4","type":"SelectorText","parentSelectors":["product_wrapper"],"selector":"div:nth-of-type(4) div.col-xs-8","multiple":false,"regex":"","delay":0},{"id":"full_price","type":"SelectorText","parentSelectors":["product_wrapper"],"selector":"div.price-strike-prod","multiple":false,"regex":"","delay":0},{"id":"discounted_price","type":"SelectorText","parentSelectors":["product_wrapper"],"selector":"div.price-prod","multiple":false,"regex":"","delay":0}]}

Hi @fin_dg

Couldn't open your sitemap due to invalid JSON. When pasting your sitemap code, mark it and use preformatted text option. Otherwise when i try to copy and paste it, some error appears.

Anyway to paginate through the pages element-click should work:

{"_id":"envirofone","startUrl":["https://www.envirofone.com/en-gb/buy/apple/search?type=MPHONE"],"selectors":[{"id":"wrappper-pagination","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.block-grid-item","multiple":true,"delay":2000,"clickElementSelector":"li.next a","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"title","type":"SelectorText","parentSelectors":["wrappper-pagination"],"selector":"div.sr-product-name span+span","multiple":false,"regex":"","delay":0}]}

1 Like

Hey @viesturs
Thanks so much for the tips and the sitemap. I've now managed to get this working the way I wanted. I was able to get the ElementClick pagination to work or the ElementClick on the product pages to work but not both. However after starting with your sitemap I've got it. Really appreciate the help. Thank you.

1 Like