Website with Load More Button stops after 1 page

How to?
#1

i want to extract data from this page which is structured like this:

  • listing page
    • link
      • new page (exhibitor)
        - name
        - email
  • load more button

The problem is that it stops after the first page load, i think i am doing something wrong tried element click and pagination elements. Also tried wrapping it in an element and then the link.

Nothing seems to work - please help!

https://exhibitors.bauma.de/de/aussteller-warengruppen/aussteller-marken

Sitemap:
{"_id":"Bauma","startUrl":["Exhibitors & brand names a","type":"SelectorLink"},{"id":"name","multiple":false,"parentSelectors":["link"],"regex":"","selector":".contentblock_firma h1","type":"SelectorText"}]}

#2

Hi, it looks like the Load more button is active only after a scroll to the bottom. See the below sitemap for reference. I have also added an element qty limitation to the load more button, otherwise it could run into a timeout. You can increase the number by editing the load-more selector: nth-of-type(100)

{"_id":"bauma","startUrl":["https://exhibitors.bauma.de/de/aussteller-warengruppen/aussteller-marken"],"selectors":[{"delay":2000,"elementLimit":500,"id":"scroll","multiple":true,"parentSelectors":["_root","load-more"],"selector":"body","type":"SelectorElementScroll"},{"clickActionType":"real","clickElementSelector":"[class=\"lazymore\"]","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":2000,"discardInitialElements":"do-not-discard","id":"load-more","multiple":true,"parentSelectors":["_root"],"selector":"body:not(:has(.hittype_Aussteller:nth-of-type(100)))","type":"SelectorElementClick"},{"id":"element","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":".treffer-titel a","type":"SelectorLink"}]}
1 Like
#3

Oh thanks that's super nice! But can i now also give the element > link > text elements?
so it clicks on every page and scrapes the available data from there?

#4

Yes, those will have to be child elements of the link selector.

#5

Yeah just trying it.
I enhanced the nth-of-type(100) to (500) but i need at least 2000 i think. You mentioned timeout, what can i do? With 500 i am only coming till letter C

#6

If the record count is too high, there is not much you can do besides creating a start URL for each letter