Click Selector with Dropdown

How to?
#1

I am trying to pull the information from the drop-down menu for each state. I am doing this with the click selector but seem to be having trouble getting the scrapper to go down the list of states to pull data from.

Also, Arizona is the first state that has all the categories I am pulling form (no how to for Alaska). Not sure if this will cause an issue.

Url: http://benefitcorp.net/policymakers/state-by-state-status?state=0

Sitemap:
{"_id":"laws_pbc","startUrl":["http://benefitcorp.net/policymakers/state-by-state-status?state=0"],"selectors":[{"id":"dropdown","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.choose-state a.chosen-single, div.view","multiple":true,"delay":0,"clickElementSelector":"div.choose-state a.chosen-single div","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"state","type":"SelectorText","parentSelectors":["dropdown"],"selector":"div.state-individual.alaska h3","multiple":true,"regex":"","delay":0},{"id":"status","type":"SelectorText","parentSelectors":["dropdown"],"selector":"div.state-individual.alaska > div:nth-of-type(1)","multiple":true,"regex":"","delay":0},{"id":"effective date","type":"SelectorText","parentSelectors":["dropdown"],"selector":"div.state-individual.alaska div:nth-of-type(2)","multiple":true,"regex":"","delay":0},{"id":"legislation","type":"SelectorText","parentSelectors":["dropdown"],"selector":"div.state-individual.alaska div:nth-of-type(3)","multiple":true,"regex":"","delay":0},{"id":"leg-link","type":"SelectorLink","parentSelectors":["dropdown"],"selector":"div.state-individual.alaska a","multiple":true,"delay":0},{"id":"sposnops","type":"SelectorText","parentSelectors":["dropdown"],"selector":"div.state-individual.alaska div:nth-of-type(4)","multiple":true,"regex":"","delay":0},{"id":"key-supporters","type":"SelectorText","parentSelectors":["dropdown"],"selector":"div.state-individual.alaska div:nth-of-type(5)","multiple":true,"regex":"","delay":0},{"id":"how-to-state","type":"SelectorLink","parentSelectors":["dropdown"],"selector":"div.state-individual.arizona a.state-anchor","multiple":true,"delay":0}]}

Drop Down To Get Data From All Clothes Sizes
Selecting drop down option and making page change
#2

Hi!

You don't need to pull any information out of drop-down, as it's all stored within a page, drop-down only changes visibility of a particular State according to your choice.

Your sitemap:
{"_id":"laws_pbc","startUrl":["http://benefitcorp.net/policymakers/state-by-state-status?state=0"],"selectors":[{"id":"state","type":"SelectorText","selector":"h3","parentSelectors":["group"],"multiple":false,"regex":"","delay":0},{"id":"State","type":"SelectorText","selector":"div.state-individual h3","parentSelectors":["state"],"multiple":false,"regex":"","delay":0},{"id":"benefit","type":"SelectorText","selector":"div.state-individual h4","parentSelectors":["state"],"multiple":false,"regex":"","delay":0},{"id":"Benefit","type":"SelectorText","selector":"h4","parentSelectors":["group"],"multiple":false,"regex":"","delay":0},{"id":"group","type":"SelectorElement","selector":"div.state-individual","parentSelectors":["_root"],"multiple":true,"delay":0},{"id":"Status","type":"SelectorText","selector":"div:nth-of-type(1)","parentSelectors":["group"],"multiple":false,"regex":"","delay":0},{"id":"Legislation","type":"SelectorText","selector":"div.item-list ul","parentSelectors":["group"],"multiple":false,"regex":"","delay":0},{"id":"Sponsors","type":"SelectorText","selector":"div:nth-of-type(4)","parentSelectors":["group"],"multiple":false,"regex":"","delay":0},{"id":"Key Supporters","type":"SelectorText","selector":"div:nth-of-type(5)","parentSelectors":["group"],"multiple":false,"regex":"","delay":0},{"id":"How to","type":"SelectorLink","selector":"a.state-anchor","parentSelectors":["group"],"multiple":false,"delay":0}]}

#3

Hi iconoclast

Can you please also help with below using dropdown which actually changes the price as well?

If it is possible for you to do little video on it as many people suffering from this and this will be great help.

Thanks in advance.

#4

Hi there!

In your case you can set up an Element Click selector to click through options that can be chosen from a drop down, and pick updated price and selected value too.

Try this one:
{"_id":"shinemirrors","startUrl":["https://www.shinemirrors.com.au/products/ablaze-clear-float-round-bathroom-mirror?variant=18475448961"],"selectors":[{"id":"prices","type":"SelectorElementClick","selector":"div.content.container","parentSelectors":["_root"],"multiple":true,"delay":"1000","clickElementSelector":"select.single-option-selector option","clickType":"clickOnce","discardInitialElements":true,"clickElementUniquenessType":"uniqueText"},{"id":"price","type":"SelectorText","selector":"span.money","parentSelectors":["prices"],"multiple":false,"regex":"","delay":0},{"id":"chosen_size","type":"SelectorText","selector":":selected","parentSelectors":["prices"],"multiple":false,"regex":"","delay":0}]}

#5

Hello, i have problem with scraping product site.

I tried both solution from here but it not working. I need to scrape two parameters "Dostępność: ... " and "Kod produktu: ... "

My Sitemap:

{"_id":"ls222","startUrl":["https://ls2.sklep.pl/pl/p/KASK-MOTOCYKLOWY-KASK-LS2-FF353-RAPID-WHITE%2C-model-2018-/595"],"selectors":[{"id":"prices","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div#box_productfull.box","multiple":true,"delay":"1000","clickElementSelector":"div.option_select select","clickType":"clickOnce","discardInitialElements":true,"clickElementUniquenessType":"uniqueText"},{"id":"rozmiar","type":"SelectorText","parentSelectors":["prices"],"selector":":selected","multiple":false,"regex":"","delay":0},{"id":"stan","type":"SelectorText","parentSelectors":["prices"],"selector":"div.availability div.row div.row span.second","multiple":false,"regex":"","delay":0},{"id":"dostepnosc","type":"SelectorText","parentSelectors":["prices"],"selector":"div.availability div.row div.row span.second","multiple":false,"regex":"","delay":0}]}

#6

Hi,
I'm trying to go through all the options in the drop down list, but it doesn't work.
Any idea what I am doing wrong?

Best regrards

{"_id":"gestrata-as","startUrl":["https://www.gestrata.at/wir-ueber-uns/asphalt-mischanlagen"],"selectors":[{"id":"liste","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div#gestratamembers","multiple":true,"delay":2000,"clickElementSelector":"select.profSelector option","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"data","type":"SelectorLink","parentSelectors":["liste"],"selector":"","multiple":true,"delay":0},{"id":"bereich","type":"SelectorText","parentSelectors":["data"],"selector":"div.memberDataField:nth-of-type(1) a","multiple":false,"regex":"","delay":0},{"id":"adresse","type":"SelectorText","parentSelectors":["data"],"selector":"div.fieldValueBlock","multiple":false,"regex":"","delay":0},{"id":"tel1","type":"SelectorText","parentSelectors":["data"],"selector":"div:nth-of-type(3) a.fieldValue","multiple":false,"regex":"","delay":0},{"id":"fax","type":"SelectorText","parentSelectors":["data"],"selector":"font","multiple":false,"regex":"","delay":0},{"id":"mail","type":"SelectorText","parentSelectors":["data"],"selector":".link-mailto a","multiple":false,"regex":"","delay":0},{"id":"mobil","type":"SelectorText","parentSelectors":["data"],"selector":"div:nth-of-type(4) a","multiple":false,"regex":"","delay":0}]}