Website: https://www.environdec.com/library
I am trying to scrape each Product Category's results from the dropdown filter (I have an element click selector and then click the Search button to receive the results. (This is where I am having trouble).
Once I have the results it will scrape each product page linked in the results for specific information (I think this part is ok).
Additional question - after results load there is a count of how many results from the product category were loaded. Minimally, I am ok scraping from those result loaded. But, this signals to me that perhaps there are more results somewhere else, a longer listing. How might I find if that is the case? No big deal if this is not doable.
Sitemap:
{"_id":"epdlibrary","startUrl":["https://www.environdec.com/library"],"selectors":[{"id":"resultcontainer","type":"SelectorLink","parentSelectors":["_root"],"selector":"#__next > main > div.library__ResultContainer-sc-1r54bpz-1.kAmJKe > a","multiple":true,"delay":0},{"id":"name","type":"SelectorText","parentSelectors":["resultcontainer"],"selector":"h1","multiple":false,"regex":"","delay":0},{"id":"image","type":"SelectorImage","parentSelectors":["resultcontainer"],"selector":".epd__ProductImages-sc-9l6euf-0 img","multiple":false,"delay":0},{"id":"proddescription","type":"SelectorText","parentSelectors":["resultcontainer"],"selector":".epd__Summary-sc-9l6euf-3 p","multiple":false,"regex":"","delay":0},{"id":"epddoc","type":"SelectorLink","parentSelectors":["resultcontainer"],"selector":"a.epd__Document-sc-9l6euf-9:nth-of-type(1)","multiple":false,"delay":0},{"id":"registrationnumber","type":"SelectorText","parentSelectors":["resultcontainer"],"selector":"div.epd__PropertyValue-sc-9l6euf-8:nth-of-type(2)","multiple":false,"regex":"","delay":0},{"id":"registrationexpiration","type":"SelectorText","parentSelectors":["resultcontainer"],"selector":"div:nth-of-type(8)","multiple":false,"regex":"","delay":0},{"id":"vendorname","type":"SelectorText","parentSelectors":["resultcontainer"],"selector":"div:nth-of-type(13)","multiple":false,"regex":"","delay":0},{"id":"vendorwebsite","type":"SelectorLink","parentSelectors":["resultcontainer"],"selector":"div:nth-of-type(19) a","multiple":false,"delay":0},{"id":"qrcode","type":"SelectorImage","parentSelectors":["resultcontainer"],"selector":"img.epd__QRCode-sc-9l6euf-10","multiple":false,"delay":0},{"id":"category","type":"SelectorText","parentSelectors":["resultcontainer"],"selector":".EpdCard__ResultCard-sc-1fobu4b-0 h4","multiple":true,"regex":"","delay":0},{"id":"dropdownprodcats","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"body","multiple":true,"delay":2000,"clickElementSelector":"div.css-1hwfws3 -ws-data-preview-element","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"searchbutton","type":"SelectorElementClick","parentSelectors":["dropdownprodcats"],"selector":"div.library__OptionContent-sc-1r54bpz-6 dsoFrd > button","multiple":false,"delay":2000,"clickElementSelector":"div.library__OptionContent-sc-1r54bpz-6 dsoFrd > button","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueHTMLText"}]}
Any guidance is appreciated. I have looked through so many posts trying to figure this out that at this point I am completely confused what to ultimately look for, period.