Unable to Scrape Dynamic Exhibitor List from Ungerboeck Site – Pagination and Links Not Working Body

How to?
#1

Hi team,

I'm trying to scrape exhibitor data (specifically website links) from this Ungerboeck-based site:

:link: Show Map

I created a sitemap using a pagination selector for the “Next” button and tried to collect the exhibitor links from div.booth-title a, and then grab the website URL from the exhibitor detail page.

This method works fine on other sites with a similar structure, but on this one:

  • Pagination doesn’t trigger new pages in the scraper
  • The links aren’t being followed
  • No data is being collected

{"_id":"hallerickson","startUrl":["[Momentus":{"id":"Exhibitors","paginationType":"auto","parentSelectors":["_root","Exhibitors"],"selector":"div.booth-title","type":"SelectorPagination"},{"id":"Weblink","linkType":"linkFromHref","multiple":false,"parentSelectors":["Exhibitors"],"selector":".contact-item a","type":"SelectorLink"}]}

I suspect it might be due to JavaScript rendering or how the site loads data via AJAX. Is there any known workaround to scrape such Ungerboeck portals with Web Scraper?

Would really appreciate any guidance or sample sitemaps.

Thanks in advance!

#2

Hi,

You can try this setup, be sure that pop-ups are not blocked by the browser:

{"_id":"ungerboeck","startUrl":["https://hallerickson.ungerboeck.com/prod/app85.cshtml?aat=6f63746d6d6f51456b52726e6f6a2b30763143586e6e4c53727951396664386933796b41346979483363773d"],"selectors":[{"clickActionType":"real","clickElementSelector":"#exhibitorList li.au-target:not(.results-count):nth-of-type(-n+10)","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":2000,"discardInitialElements":"discard-when-click-element-exists","id":"listing","multiple":true,"parentSelectors":["_root"],"selector":".modal-content","type":"SelectorElementClick"},{"id":"link","linkType":"linkFromRedirect","multiple":false,"parentSelectors":["listing"],"selector":".contact-item a","type":"SelectorLink"},{"clickActionType":"real","clickElementSelector":".modal-header button","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickOnce","delay":2000,"discardInitialElements":"do-not-discard","id":"close","multiple":true,"parentSelectors":["listing"],"selector":"_parent_","type":"SelectorElementClick"}]}

I have limited the clicks for testing purposes:

image
image718×145 14.4 KB

The highlighted part can be removed to scrape the full list.