Hello!
I am trying to scrape a cell phone provider's website for phone details. Each phone model is displayed in an element, and by clicking different memory size variants within this wrapper, different pricing information is displayed. I'm trying to scrape the pricing information for each memory variant of each phone (along with title, term details, and ideally the memory size that the element click selector is choosing).
My sitemap seems to work okay, but is missing some phone / memory variants. I used the "Element Click" selector to pick the phone's wrapper, then the click selector to pick each memory variant, and then have text selectors to pull the relevant data.
Also, more phones become visible when scrolling down, so I also need to figure out how to use the element scroll and also the element click selector together to first load all phones, then grab the data.
TIA!
Url: https://www.rogers.com/web/totes/wireless/choose-phone
Sitemap:
{"_id":"rogers","startUrl":["https://www.rogers.com/web/totes/wireless/choose-phone"],"selectors":[{"id":"phone_price","type":"SelectorText","parentSelectors":["size"],"selector":"span.ng-scope span.ng-binding","multiple":false,"regex":"","delay":0},{"id":"phone_term","type":"SelectorText","parentSelectors":["size"],"selector":"div.desc-font span.ng-scope","multiple":false,"regex":"","delay":0},{"id":"size","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.wrapper-box","multiple":true,"delay":0,"clickElementSelector":"div.color-memory-section.colors-4 div.memory-box","clickType":"clickOnce","discardInitialElements":true,"clickElementUniquenessType":"uniqueText"},{"id":"phone_title","type":"SelectorText","parentSelectors":["size"],"selector":"div.col-xs-12 div.col-xs-12 span.ng-binding","multiple":false,"regex":"","delay":0},{"id":"phone_size","type":"SelectorText","parentSelectors":["size"],"selector":"div.memory-box.selected span.ng-binding","multiple":false,"regex":"","delay":0}]}