I tried to select "link" to get the links of all sub-pages, but the webscraper needs to open each sub-page to get the link, which leads to very low efficiency, because my main webpage has 400 pages, if each main webpage is The links to the sub-pages all need to be opened one by one to get it, it is too slow, I need a faster method, please help.
https://www.amazon.de/s?k=iphone+11+h%C3%BClle&i=electronics&__mk_de_DE=%C3%85M%C3%85%C5%BD%C3%95%C3%91&crid=129VFVTAKRFFM&sprefix=iPhone+11%2Celectronics%2C399&ref=nb_sb_ss_ts-doa-p_2_9
This is the link I attached. It is a search result of a shopping page. Please check it out.
The result is four hundred pages in total. I need to get the link addresses of all the products on each page, so that I can extract information from the addresses. I know that I can get it by selecting the crawl link, but this leads to the need for every link to crawl Open a product. This is too slow, I just need to get the link, I hope there is a faster way.
If you just use type: Link without any child selectors (no other data scrapers within it), it will just get the title and URL, and it will not click thru to the other pages. E.g.
{"_id":"amazon-only-links","startUrl":["https://www.amazon.com/"],"selectors":[{"delay":0,"id":"Current Links","multiple":true,"parentSelectors":["_root"],"selector":"a","type":"SelectorLink"}]}