Describe the problem.
URL: BING Internet Tools
I watched some tutorials but I don't understand how I can set up a scraper that clicks every selectbox option one by one and scrapes the datatable on the screen
Any pros here who can help me with a solution?
Hi,
You can use the sitemap below as a template. Note that a selector will have to be created for each data point. Please click the 'Show data' button manually the first time the scraper opens the pop-up window to load the first selection.
{"_id":"bingpower-de2","startUrl":["https://apps.bingpower.de/einstelldaten_en.aspx"],"selectors":[{"id":"dropdown-select","multiple":true,"parentSelectors":["pagination"],"selector":"_parent_","type":"SelectorElement"},{"id":"selected-option","multiple":false,"parentSelectors":["dropdown-select"],"regex":"","selector":"p.header3","type":"SelectorText"},{"id":"replacement for","multiple":false,"parentSelectors":["dropdown-select"],"regex":"","selector":"td:contains('replacement for') +","type":"SelectorText"},{"id":"replaced by","multiple":false,"parentSelectors":["dropdown-select"],"regex":"","selector":"td:contains('replaced by') +","type":"SelectorText"},{"id":"similar to","multiple":false,"parentSelectors":["dropdown-select"],"regex":"","selector":"td:contains('similar to') +","type":"SelectorText"},{"id":"engine model_1","multiple":false,"parentSelectors":["dropdown-select"],"regex":"","selector":"td:contains('engine model') +","type":"SelectorText"},{"id":"engine model_2","multiple":false,"parentSelectors":["dropdown-select"],"regex":"","selector":"tr:contains('engine model') + tr td:nth-of-type(2)","type":"SelectorText"},{"id":"engine model_3","multiple":false,"parentSelectors":["dropdown-select"],"regex":"","selector":"tr:contains('engine model') ++ tr td:nth-of-type(2)","type":"SelectorText"},{"id":"pagination","paginationType":"clickMore","parentSelectors":["_root","pagination"],"selector":"body:not(:has(select option:nth-of-type(10):selected)) select option:selected +","type":"SelectorPagination"}]}
The following part is added to the pagination selector for testing purposes to stop the scraping after 10 clicks. It can be removed once you are ready to launch the full scraping job:
This works great, thank you very much!
