The problem
I scrap an url like : www.mywebsite.com/
I extract a list of [products ID] datas like : 123456, 123457, 123458
I want to scrap a second url like : mywebsite.com is available for purchase - Sedo.com
toi scrap all the products pages.
And i don't see how do add a data from one sitemap, to a second sitemap as variable.
An idea?
Regards.
You can generate your product links in Excel, publish them on pastelink.net as a start URL and start scraping as a regular page.... No problems at all...
Thx for the answer...
Euhhh... This is not the more directly way...
I think it isn't so difficult for you to store each created selector (not the datas) with ID_selector rattached to his sitemap_id.
.
For example :
Sitemap_1 : id_selector_1, id_selector_2, id_selector_3
Sitemap_2 : id_selector_10, id_selector_11, id_selector_12
Sitemap 3 : [start_url]/[sitemap_1[id_selector_2]]/[sitemap_3[id_selector_11]]
I understand that it is not a ready to go developpment, but you have these datas.
A lot of website have a homepage with a list of something we want to scrap.
And theses things generally genertating new urls we want scrap to have details for example.
So, dependancies between differents starting urls (so sitemaps data) is a recurent need.
i think you should consider these future to give a real power to your solution versus your competitors.
Your product is great and your team too.
But proposing to generate an excel file for copying data on a website to scrap it, seems not a webscraping professionnal solution (paid solution :))
But thanks for earing your clients and for always answer them.
I'm not an employee of Webscraper... Just a usual user as you ))
So, Many thanks for answer