I have trouble scraping data when I enter multiple URLs in the sitemap metadata (with the + sign).
It starts and only scrapes the last url and does not continue to the previous one in the list.
Please, do you have any idea what to change?
Hello, Thank you very much for message
Of course, here you are.
It completely scrapes data (addresses of ATM machines) from the first URL, but then it loops with "load more" and does not continue to the following URL.
{"_id":"ceskasporitelna_bankomaty","startUrl":["https://www.csas.cz/cs/pobocky-a-bankomaty#/vyhledavani?type=ATM&flags=ATM&address=jihočeský%20kraj&tab=LIST","https://www.csas.cz/cs/pobocky-a-bankomaty#/vyhledavani?type=ATM&flags=ATM&address=liberecký%20kraj&tab=LIST"],"selectors":[{"id":"loadmore","paginationType":"auto","parentSelectors":["_root","loadmore"],"selector":"a.csas-click--big","type":"SelectorPagination"},{"delay":0,"id":"atm_link","multiple":true,"parentSelectors":["loadmore"],"selector":"h3 a","type":"SelectorLink"},{"delay":0,"id":"name","multiple":false,"parentSelectors":["atm_link"],"regex":"","selector":"h1","type":"SelectorText"},{"delay":0,"id":"address","multiple":false,"parentSelectors":["atm_link"],"regex":"","selector":".csas-branch-header-address strong","type":"SelectorText"},{"delay":0,"id":"services","multiple":false,"parentSelectors":["atm_link"],"regex":"","selector":"ul.cs-grid","type":"SelectorText"},{"delay":0,"id":"atm_number","multiple":false,"parentSelectors":["atm_link"],"regex":"","selector":".csas-mb-md--tablet span.ng-binding","type":"SelectorText"}]}
So I found out what the problem was.
The problem was the javascript of the scraped page, which caused, that even though I entered different URLs, the page still loaded the first one in the list.
The solution was to put a question mark before the # sign in each URL and a different letter each time. So I forced the site to always load the URL including the search query...