Help me to scape with pagination

#1

i made a sitemap to scrape traders data from bingx website and its work fine .. but its only scrape the first page . i need all pages ... pagination not work

{"_id":"test","startUrl":["https://bingx.com/en/CopyTrading/?from=5&exchangeId=0&sort=accEarningRatio"],"selectors":[{"id":"title","parentSelectors":["_root"],"type":"SelectorText","selector":".futures-card[data-v-6b55157c] div.text","multiple":true,"regex":""},{"id":"click","parentSelectors":["_root"],"type":"SelectorElementClick","clickActionType":"real","clickElementSelector":"div[data-v-6b55157c]:nth-of-type(1) div.text","clickElementUniquenessType":"uniqueText","clickType":"clickOnce","delay":2000,"discardInitialElements":"do-not-discard","multiple":true,"selector":"div[data-v-6b55157c]:nth-of-type(1) div.text"},{"id":"data","parentSelectors":["_root"],"type":"SelectorText","selector":"div#data-overview","multiple":false,"regex":""},{"id":"page","parentSelectors":["click","page"],"paginationType":"auto","type":"SelectorPagination","selector":"div.page-cell:nth-of-type(n+4)"},{"id":"pagination","parentSelectors":["_root","pagination"],"paginationType":"auto","type":"SelectorPagination","selector":"div.jump-cell_next"}]}

image
image899×285 19.4 KB

#2

How URL changes if you select page 3 for example...?

#3

it's not changed .. load with javascript when you change the page

#4

Hi,

Because of the JS pagination, it will only be possible to scrape the trader cards data, but not click through the trader profiles.

{"_id":"test","startUrl":["https://bingx.com/en/CopyTrading/?from=5&exchangeId=0&sort=accEarningRatio"],"selectors":[{"id":"title","multiple":true,"parentSelectors":["_root","pagination"],"regex":"","selector":".futures-card[data-v-6b55157c] div.text","type":"SelectorText"},{"id":"pagination","paginationType":"clickMore","parentSelectors":["_root","pagination"],"selector":"div.jump-cell_next","type":"SelectorPagination"}]}
#5

thank you for helping .. but i need more details for each trader profile page .. not only the name .. how can do this ?

#6

Unfortunately, it will only be possible to scrape the details from the listing view.

#7

but at my first code i got details for each profile .. but only the first page

#8

Yes, one page works, but for the pagination to work, the scraper has to be able to gather the profile URLs. If both are JS generated and not hardcoded in the HTML, it is not possible at the moment.