Need some help pls :’( :’( :’(

How to?
#1

Hello :handshake:

How to scrape this page with the pagination next button.

Does anyone have an idea? (I'm a beginner sorry if this question seems ridiculous :sob: )

Thanks to those who can help me :smile:

Url: https://aeromexico.com/en-us/book/options?itinerary=NYC_MEX_2021-10-29&leg=1&travelers=A1_C0_I0_PH0_PC0

Sitemap:

{"_id":"travel","startUrl":["https://aeromexico.com/en-us/book/options?itinerary=NYC_MEX_2021-10-29&leg=1&travelers=A1_C0_I0_PH0_PC0"],"selectors":[{"id":"day","type":"SelectorText","parentSelectors":["_root"],"selector":".is-currentDate .FlightOptionsCalendar-dayDateButtonDate span","multiple":false,"regex":"","delay":0},{"id":"next","type":"SelectorPopupLink","parentSelectors":["_root"],"selector":"button.Icon--arrowRight","multiple":true,"delay":0}]}

#2

@George Hi, in this case you should use element-click selector to use pagination.

{"_id":"travel","startUrl":["https://aeromexico.com/en-us/book/options?itinerary=NYC_MEX_2021-10-29&leg=1&travelers=A1_C0_I0_PH0_PC0"],"selectors":[{"id":"next","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"body:has(span:contains(\"Departing\"))","multiple":true,"delay":2000,"clickElementSelector":"button.Icon--arrowRight","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"Calendar","type":"SelectorText","parentSelectors":["next"],"selector":"button.is-currentDate","multiple":false,"regex":"","delay":0}]}

#3

Excellent, but I want you to continue with the other days after clicking the "next" button, because it stops and does not work anymore, it closes, could you help me please :sob:

#4

@George Can you explain the idea behind this? Do you want to scrape every day and price?

#5

Yes, I try to scrape the days and prices so that it continues with the next day and so on.

But when it scrapes the data it does not continue with the other days and I was able to create a sinemap that the pagination works but does not scrape any data.

I've been wasted a long time, could you please help me?

{"_id":"one","startUrl":["https://aeromexico.com/en-us/book/options?itinerary=NYC_MEX_2021-10-29&leg=1&travelers=A1_C0_I0_PH0_PC0"],"selectors":[{"id":"day","type":"SelectorText","parentSelectors":["_root","pagination"],"selector":".is-currentDate .FlightOptionsCalendar-dayDateButtonDate span","multiple":true,"regex":"","delay":0},{"id":"pagination","type":"SelectorElementClick","parentSelectors":["_root","pagination"],"selector":"button.Icon--arrowRight","multiple":false,"delay":"500","clickElementSelector":"li:nth-of-type(4) button","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"}]}

#6

Hi @George

Set a long page delay, like 6000(ms). Remember to wait till the end and that the results will come out in pseudo-random order.

{"_id":"one","startUrl":["https://aeromexico.com/en-us/book/options?itinerary=NYC_MEX_2021-10-29&leg=1&travelers=A1_C0_I0_PH0_PC0"],"selectors":[{"id":"day","type":"SelectorText","parentSelectors":["pagination"],"selector":"span.FlightOptionsCalendar-dayDateButtonDate","multiple":false,"regex":"","delay":0},{"id":"pagination","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"li[class*=\"FlightOptionsCalendar-day\"]","multiple":true,"delay":"6000","clickElementSelector":"button.Icon--arrowRight","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"},{"id":"price","type":"SelectorText","parentSelectors":["pagination"],"selector":"span.FlightOptionsCalendar-dayDateButtonTotal","multiple":false,"regex":"","delay":0}]}