Cannot scrape the search results

[Askier]

Is it possible to scrape the search result of this website?
Url: https://www.axs.com/category/music

I tried to scrape the search results but upon running the scraper and a new window pops up, the search results doesn't appear.

Here's the Sitemap:

{"_id":"Events--AXS","startUrl":["https://www.axs.com/category/music"],"selectors":[{"id":"Event Name","parentSelectors":["Events Link"],"type":"SelectorText","selector":"h1","multiple":false,"regex":"","multipleType":"singleColumn","version":2},{"id":"Event Date","parentSelectors":["Events Link"],"type":"SelectorText","selector":"span.Div-sc-15aqupl time","multiple":false,"regex":"","multipleType":"singleColumn","version":2},{"id":"Event City and State","parentSelectors":["Events Link"],"type":"SelectorText","selector":".cAoggN span","multiple":false,"regex":"","multipleType":"singleColumn","version":2},{"id":"Pagination","parentSelectors":["_root","Pagination"],"paginationType":"auto","type":"SelectorPagination","selector":"button.AXSButton-sc-t4i2qj a"},{"id":"Events Link","parentSelectors":["_root","Pagination","Events Link"],"type":"SelectorLink","selector":"div[data-testid=\"EventItemCard\"]","multiple":true,"version":2,"linkType":"linkFromRedirect"}]}

[JanAp]

Hi,

Use this sitemap:

{"_id":"Events--AXS","startUrl":["https://www.axs.com/category/music"],"selectors":[{"id":"Event Name","multiple":false,"multipleType":"singleColumn","parentSelectors":["Events Link"],"regex":"","selector":"h1","type":"SelectorText","version":2},{"id":"Event Date","multiple":false,"multipleType":"singleColumn","parentSelectors":["Events Link"],"regex":"","selector":"span.Div-sc-15aqupl time","type":"SelectorText","version":2},{"id":"Event City and State","multiple":false,"multipleType":"singleColumn","parentSelectors":["Events Link"],"regex":"","selector":".cAoggN span","type":"SelectorText","version":2},{"id":"Pagination","paginationType":"auto","parentSelectors":["_root","Pagination"],"selector":"button.AXSButton-sc-t4i2qj a","type":"SelectorPagination"},{"id":"Events Link","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root","Pagination","Events Link"],"selector":"a[data-testid='CTAButton']","type":"SelectorLink","version":2}]}

Start the scraping with a longer page-load delay, then execute the search manually in the pop-up window.

Hope this helps!

[Askier]

Thank you very much @JanAp for correcting my sitemap in order to work.

I have several websites that I have scraped, but this one I failed to scrape even a single data.
I'm thinking to use a Website State setup, but I don't know how to do since it doesn't have a clear documentation about it.

[Askier]

The only disadvantage of having a longer page-load delay is it takes a lot of seconds to proceed to another page to scrape which it takes a longer time to finished scraping all the data.
If you lower the page-load delay, you didn't finish typing in search box, the pop-up page will disappear.

I hope this app will add another feature that the user will have an option to pause/start the scraper to scrape while the user still typing in a search bar pop-up window and waiting for the results.

[JanAp]

Typically, this would be possible to set up via Website State Setup; however, this website is an edge case because it displays an overlay that is preventing the automated selection of the location.