Can't scrap pagination (JS)

How to?
#1

hello,

I try to scrape a website, but I don't know how to do with the pagination (JS) as you can see below:

19

Ex: https://www.eterritoire.fr/evenements/france/visites-et-circuits

Can someone help me?

Thanks,

D

#2

Hello, if you would use link selector for pagination, it should be working fine.

#3

Thanks for the reply, but I dont know how to configure the link selector with JS.
Can you help me?
Thanks,

David

#4

.pagination a:contains("Suivant")

#5

Hello,

Thanks for the help, I have this:
{"_id":"eterritoire","startUrl":["https://www.eterritoire.fr/evenements/france/visites-et-circuits"],"selectors":[{"id":"link1","type":"SelectorLink","parentSelectors":["_root"],"selector":"a[itemprop='url']","multiple":true,"delay":0},{"id":"pagination","type":"SelectorElementClick","parentSelectors":["_root","pagination"],"selector":".pagination a:contains("Suivant")","multiple":true,"delay":"50","clickElementSelector":".pagination a:contains("Suivant")","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"}]}

it doesnt' seem to work :cry:

#6

it is invalid sitemap.

#7

{"_id":"eterritoire","startUrl":["https://www.eterritoire.fr/evenements/france/visites-et-circuits"],"selectors":[{"id":"link1","type":"SelectorLink","parentSelectors":["_root"],"selector":"a[itemprop='url']","multiple":true,"delay":0},{"id":"pagination","type":"SelectorElementClick","parentSelectors":["_root","pagination"],"selector":".pagination a:contains("Suivant")","multiple":true,"delay":"50","clickElementSelector":".pagination a:contains("Suivant")","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"}]}

#8

{"_id":"agenda","startUrl":["https://www.eterritoire.fr/evenements/france/visites-et-circuits"],"selectors":[{"id":"link1","type":"SelectorLink","parentSelectors":["pagination"],"selector":"a[itemprop='url']","multiple":true,"delay":0},{"id":"pagination","type":"SelectorElementClick","parentSelectors":["_root"],"selector":".lstline li[itemscope]","multiple":true,"delay":"1000","clickElementSelector":".pagination a:contains("Suivant")","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueCSSSelector"}]}

#9

Thanks but I have invalid JSON

#10

{
"_id": "agenda",
"startUrl": ["https://www.eterritoire.fr/evenements/france/visites-et-circuits"],
"selectors": [{
"id": "link1",
"type": "SelectorLink",
"parentSelectors": ["pagination"],
"selector": "a[itemprop='url']",
"multiple": true,
"delay": 0
}, {
"id": "pagination",
"type": "SelectorElementClick",
"parentSelectors": ["_root"],
"selector": ".lstline li[itemscope]",
"multiple": true,
"delay": "1000",
"clickElementSelector": ".pagination a:contains("Suivant")",
"clickType": "clickOnce",
"discardInitialElements": "do-not-discard",
"clickElementUniquenessType": "uniqueCSSSelector"
}]
}

#11

Thanks for the help, but still invalid JSON... Sorry

#12

I cannot post it anymore, you should fix it by yourself at https://jsonlint.com/

#13

Hello,

It works, thanks for your help!
The only issue is that the scrap stops after page 4, is it normal ?

D

#14

Site url does seem to change with each new page, e.g. page 5 is https://www.eterritoire.fr/evenements/france/visites-et-circuits/5
so you don't really need a paginator and you can use the Specify multiple urls with ranges method.

#15

Thanks for the help!