How to scrape data that only appears when screen is showing that section

How to?
#1

I am trying to scrape data from a website that only loads a section of the website when the user if 'looking' at that section of the webpage. While I am able to use the "Element scroll down" feature of https://webscraper.io/, scrolling all the way to the bottom of the page does not load the section of data I am trying to scrape.

If the data is not properly loaded, I only get 2 of the links I am trying to scrape. When fully loaded (after 'looking' at the section), then webscraper is able to pick up the 4-5 links I want.

I have tried using element click to scroll through the options (it is a carousel), and have also tried using the element selector, but both are only able to pick up the basic 2 links.

Please let me know what I can do to properly load/scrape this kind of data!!

#2

Hard to diagnose if you don't provide the website or sitemap. Couple of things you can try:

  1. For either Element scroll down or Element click, make sure you specify a delay of at least 500 to 1000, so that the server has enough time to load that section of the page. For some websites, you may require even longer delays.
  2. Zoom out your browser view to 33% before you start scraping. This will cause the website to display more info on your browser. Don't worry about the text being too small to read; the scraper does not care.