Scrape data on popup pages - Aliexpress

How to?
#1

Hi there!
First I looked on the search and couldn't find an answer for what I'm looking

I want to scrap details of the products pages on aliexpress.com
Example page:
vs890 - Buy vs890 with free shipping on AliExpress

first I scrool down
second I select the products.

when I try to open each page to select what I want, it opens in a new tab.

my sitemap is this:

{"_id":"vs890","startUrl":["https://www.aliexpress.com/wholesale?trafficChannel=main&d=y&CatId=0&SearchText=vs890&ltype=wholesale&SortType=price_asc&page=[1-10]"],"selectors":[{"id":"link","type":"SelectorLink","parentSelectors":["Scroller"],"selector":"a.item-title","multiple":true,"delay":0},{"id":"Scroller","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"ul.list-items li.list-item","multiple":true,"delay":"250"},{"id":"test","type":"SelectorPopupLink","parentSelectors":["link"],"selector":"a[title='Octopus Tripod LS990 Car-Phone-Holder Flexible LG for G2 G3 Mini G4 D802 Vs890/D850/D855/Ls990']","multiple":false,"delay":0},{"id":"2units","type":"SelectorElementClick","parentSelectors":["test"],"selector":".next-after button","multiple":false,"delay":2000,"clickElementSelector":"i.next-icon-add.next-xs","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"title","type":"SelectorText","parentSelectors":["test"],"selector":"h1","multiple":false,"regex":"","delay":0},{"id":"price","type":"SelectorText","parentSelectors":["test"],"selector":"span[itemprop='price']","multiple":false,"regex":"","delay":0},{"id":"ship","type":"SelectorText","parentSelectors":["test"],"selector":"div.shipping-hba-title","multiple":false,"regex":"","delay":0}]}

Thank you for all help.

#2

Hello @nandobarreto

I think pop-up is not the case here, i would just stick to link and use element click for pagination.
Like this:

{"_id":"vs890","startUrl":["https://www.aliexpress.com/wholesale?trafficChannel=main&d=y&CatId=0&SearchText=vs890&ltype=wholesale&SortType=price_asc&page"],"selectors":[{"id":"link","type":"SelectorLink","parentSelectors":["wrapper-scroll"],"selector":"a.item-title","multiple":true,"delay":0},{"id":"wrapper-scroll","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"li.list-item","multiple":true,"delay":2000},{"id":"title","type":"SelectorText","parentSelectors":["wrapper"],"selector":"h1","multiple":false,"regex":"","delay":0},{"id":"price","type":"SelectorText","parentSelectors":["wrapper"],"selector":"span[itemprop='price']","multiple":false,"regex":"","delay":0},{"id":"wrapper","type":"SelectorElement","parentSelectors":["link"],"selector":"body:has(div[itemprop=\"offers\"])","multiple":true,"delay":0},{"id":"pagination-click","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"li.list-item","multiple":true,"delay":2000,"clickElementSelector":"button.next-next","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"}]}

1 Like
#3

Thank you so much for your reply!

I tried to improve few things but I'm still having few problems:
1- the page scroll down only scrolls on the first page, I tried to solve this deleting your pagination click and using the pagination on the URL

aliexpress com/wholesale?trafficChannel=main&d=y&CatId=0&SearchText=vs890&ltype=wholesale&SortType=price_asc&page=[1-4]

It worked but hurt my performance a lot (like 5 times slower, cant understand why)

2- some pages like this: https://www.aliexpress.com/item/1005002080294510.html

I need to select the country and the product.
I made an element click and it selects and work, but if there is only 1 option it deselects and I cant get the price of shipment. example: https://www.aliexpress.com/item/4000386139879.html

There is an option to select if is not selected?

Here is what I got so far:

{"_id":"vs8903","startUrl":["vs890 - Buy vs890 with free shipping on AliExpress"], "selectors":[{"id":"link","type":"SelectorLink","parentSelectors":["wrapper-scroll"],"selector":"a.item-title","multiple":true,"delay":0},{"id":"wrapper-scroll","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"li.list-item","multiple":true,"delay":"200"},{"id":"Title","type":"SelectorText","parentSelectors":["wrapper"],"selector":"h1","multiple":false,"regex":"","delay":0},{"id":"wrapper","type":"SelectorElement","parentSelectors":["link"],"selector":"body:has(div[itemprop="offers"])","multiple":true,"delay":0},{"id":"more1","type":"SelectorElementClick","parentSelectors":["wrapper"],"selector":"i.next-icon-add.next-xs","multiple":false,"delay":"200","clickElementSelector":"i.next-icon-add.next-xs","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"color","type":"SelectorElementClick","parentSelectors":["wrapper"],"selector":".sku-property-image img","multiple":true,"delay":"200","clickElementSelector":".sku-property-image img","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"from","type":"SelectorElementClick","parentSelectors":["wrapper"],"selector":"div.sku-property-text","multiple":true,"delay":"200","clickElementSelector":"li:nth-of-type(1) div.sku-property-text","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"frete","type":"SelectorLink","parentSelectors":["wrapper"],"selector":"span.bold","multiple":false,"delay":0},{"id":"preço","type":"SelectorText","parentSelectors":["wrapper"],"selector":"span[itemprop='price']","multiple":false,"regex":"","delay":0},{"id":"foto","type":"SelectorImage","parentSelectors":["wrapper"],"selector":"img.magnifier-image","multiple":false,"delay":0}]}

Sorry, idk how to not change the url on my code put a space after the url also

3- I'm running web scraper on a incognito tab. Aliexpress shows more results this way, but even the code being called via incognito tab it opens a non incognito tab to execute. There is any way to run incognito?