Pls Help_ Scraping multiple seasons from dropdown menu

Hello everybody,
I'm stuck scraping the data for total shots from previous seasons.
The current season is not important. That's why I excluded them with

select#seasons option:not(:contains(2019/2020))

The source for the element click is a dropdown box.

When I start scraping, only one season (2018/2019) is selected. I need the last 5 seasons. In addition, the table is not extracted.

Can someone help me with this?

Thanks a lot :slight_smile:

Url: https://www.whoscored.com/Regions/81/Tournaments/3/Seasons/7872/Germany-Bundesliga

Sitemap:
{"_id":"whoscored_totalshots_his","startUrl":["https://www.whoscored.com/Regions/81/Tournaments/3/Seasons/7872/Germany-Bundesliga"],"selectors":[{"id":"click seasons","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.layout-content-2col-left","multiple":true,"delay":"2100","clickElementSelector":"select#seasons option:not(:contains(2019/2020))","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"Team Stats","type":"SelectorPopupLink","parentSelectors":["click seasons"],"selector":".with-single-level li:nth-of-type(3) a","multiple":false,"delay":0},{"id":"click detailed","type":"SelectorElementClick","parentSelectors":["Team Stats"],"selector":"#stage-team-stats","multiple":false,"delay":"2000","clickElementSelector":"div.option-group li a:contains('Detailed')","clickType":"clickOnce","discardInitialElements":"discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"click total","type":"SelectorElementClick","parentSelectors":["click detailed"],"selector":"#statistics-team-table-detailed","multiple":false,"delay":"2000","clickElementSelector":"select#statsAccumulationType option:contains('Total')","clickType":"clickOnce","discardInitialElements":"discard-when-click-element-exists","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"Total Table","type":"SelectorTable","parentSelectors":["click total"],"selector":"table","multiple":false,"columns":[{"header":"R","name":"Placing","extract":true},{"header":"Team","name":"Team","extract":true},{"header":"Total","name":"Total","extract":true},{"header":"OutOfBox","name":"OutOfBox","extract":true},{"header":"SixYardBox","name":"SixYardBox","extract":true},{"header":"PenaltyArea","name":"PenaltyArea","extract":true},{"header":"Rating","name":"Rating","extract":false}],"delay":0,"tableDataRowSelector":"#top-team-stats-summary-content tr","tableHeaderRowSelector":"thead tr"}]}

On this site, the URL changes with each season so it is actually easier to just extract the URLs for each season first, instead of navigating the dropdown. You can get the URLs with this sitemap:

{"_id":"forum-whoscored-get-seasons","startUrl":["https://www.whoscored.com/Regions/81/Tournaments/3/Seasons/6392/Germany-Bundesliga"],"selectors":[{"id":"Season URL","type":"SelectorElementAttribute","parentSelectors":["_root"],"selector":"div.main-content-column select#seasons option:not(\":contains('2020')\")","multiple":true,"extractAttribute":"value" }]}

Then you just prefix the whoscored domain so that the URLs all look like:
https://www.whoscored.com/Regions/81/Tournaments/3/Seasons/7405/Germany-Bundesliga

This is easy to do, and can even be done with Notepad with search n replace.

After that, you create a single sitemap which uses all the URLs.