Navigating a Dynamic Site

How to?
#1

Describe the problem.
I want to go into each NBA game being played today and grab all the bet options and odds on the Same Game Parlay tab. But before I get into the complexity of odds, I created this simple tester sitemap to go into each game and grab the team names and game times. I can't even get that to work. :slight_smile: It is a dynamic site with complex CSS.
Problem 1. With the games being played today (1/7), the selector doesn't "see" the last game being played today (Hawks/Lakers).
Problem 2. Even with the multiple games the selector does see, it doesn't go into each game.

Url: FanDuel Sportsbook

Sitemap:
{"_id":"nba-crawler","startUrl":["FanDuel Sportsbook a.aq","type":"SelectorLink"},{"delay":0,"id":"game-teams","multiple":false,"parentSelectors":["game-link"],"regex":"","selector":"h3","type":"SelectorText"},{"delay":0,"id":"game-time","multiple":false,"parentSelectors":["game-link"],"regex":"","selector":"span.ae.fr","type":"SelectorText"}]}

Any advice to get the links to work?
And foresee any challenges with my next step of grabbing all the bet options and odds on the Same Game Parlay tab?

#2

@medelste Hello, as it seems the targeted website uses generated classes which most likely are re-generated on a regular basis. In order to circumvent this, you would have to create most of the selectors manually.

Example:

{"_id":"sportsbook-fanduel-com","startUrl":["https://sportsbook.fanduel.com/navigation/nba"],"selectors":[{"delay":2000,"id":"wrapper","multiple":true,"parentSelectors":["_root"],"selector":"div#root > div > div > div > div > div > div > div > div > div > div > div > div > div:has(title)","type":"SelectorElementScroll"},{"delay":0,"id":"home-team","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":"a[style=\"cursor: pointer;\"]:nth(0) > div > div:nth(0) span:nth(0)","type":"SelectorText"},{"delay":0,"id":"away-team","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":"a[style=\"cursor: pointer;\"]:nth(0) > div > div:nth(2) span:nth(0)","type":"SelectorText"},{"delay":0,"id":"home-team-score","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":"a[style=\"cursor: pointer;\"]:nth(0) > div > div:nth(0) span:nth(1)","type":"SelectorText"},{"delay":0,"id":"away-team-score","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":"a[style=\"cursor: pointer;\"]:nth(0) > div > div:nth(2) span:nth(1)","type":"SelectorText"},{"delay":0,"extractAttribute":"","id":"home-team-spread","parentSelectors":["wrapper"],"selector":"div[role=\"button\"]:nth(0) span","type":"SelectorGroup"},{"delay":0,"extractAttribute":"","id":"away-team-spread","parentSelectors":["wrapper"],"selector":"div[role=\"button\"]:nth(3) span","type":"SelectorGroup"},{"delay":0,"extractAttribute":"","id":"home-team-money","parentSelectors":["wrapper"],"selector":"div[role=\"button\"]:nth(1) span","type":"SelectorGroup"},{"delay":0,"extractAttribute":"","id":"away-team-money","parentSelectors":["wrapper"],"selector":"div[role=\"button\"]:nth(4) span","type":"SelectorGroup"},{"delay":0,"extractAttribute":"","id":"home-team-total","parentSelectors":["wrapper"],"selector":"div[role=\"button\"]:nth(2) span","type":"SelectorGroup"},{"delay":0,"extractAttribute":"","id":"away-team-total","parentSelectors":["wrapper"],"selector":"div[role=\"button\"]:nth(5) span","type":"SelectorGroup"}]}

#3

@ViestursWS Thank you so much! You've given me hope that this can be done. Yes the targeted website does use classes that are re-generated on a regular basis.

I'd happy to create the selectors manually but I don't know how to pick up from where you've left off. I've looked at the selectors that you've created and then I've inspected the corresponding webpage's CSS and looking at them side-by-side ... I don't see how you were able to do what you did - the two look completely unrelated and disconnected to my untrained-in-CSS eye.

Is there some sort of tutorial or video that can give me the understanding how to convert what I'm seeing in the CSS and translate it to the selector syntax so I can create the selector manually?

In addition to what you've demonstrated (scraping text off a complex webpage), I need to create and follow links into each NBA game, choose a new tab, and expand accordions before I can scrape the text and move to the next NBA game.

Any guidance or education would be greatly appreciated.