Hi
I am facing an issue that it only scrapes the first page and not moving to the next one.
Can anyone help me with this?
Hi
I am facing an issue that it only scrapes the first page and not moving to the next one.
Can anyone help me with this?
@Saravanan_Kumar Hello, it seems that there's some sort of a trigger that happens after a scroll is performed - after you scroll down the elements above are replaced, therefore, when scraping I'd suggest zooming the page as far as you can.
Thank you, I made it work by adding selector as suggested below
and the site map is
{
"_id": "hl7",
"startUrl": [
"Caristix HL7-Definition V2"
],
"selectors": [
{
"id": "container",
"type": "SelectorElementScroll",
"parentSelectors": [
"_root"
],
"selector": ".cdk-virtual-scroll-content-wrapper mat-nav-list",
"multiple": true,
"delay": 2000,
"scrollElementSelector": "cdk-virtual-scroll-viewport"
},
{
"id": "name",
"type": "SelectorLink",
"parentSelectors": [
"container"
],
"selector": "a",
"multiple": true,
"delay": 0
}
]
}
@Saravanan_Kumar Yes, the inner scroll works, but, unfortunately it is not loading every available item.
yes Viesturs, It scrolling a bit fast so as per your solution I zoomed out as much as the browser could and its doing the expected job. its great to have a person like you to follow-up on technical issues like this. I really appreciate this.
Thanks a lot
Saravana