I'm trying to define the to be scraped pages using sitemap.xml which contains a subset of the product pages available on a website. There are few hundred products and the list is dynamic so I think using start urls is not possible/feasible.
I do not have access to the target website so I must store the modified sitemap.xml on my own site. The problem is that I get "No Data Extracted" when I point the sitemap.xml location to my own website. If I point it to the original websites sitemap.xml, I get the data but of course containing all the thousands of products/urls.
I have tried also with the exactly same sitemap.xml just copied to my own site so I know it is not the sitemap.xml file itself causing the issue. I have also tried this with my own website with a sitemap.xml containing links to my own website and it works fine.
It seems like the Sitemap.xml Links selector prevents loading a custom sitemap.xml from another website than the scraped one. This seems unnecessary and could perhaps be changed?
Any help is well appreciated.