Scrapping dynamic <div> table with accordion style row reveal

#1

I need help with scraping a dynamic table made out of . Only 16 rows are shown in the DOM. The table is made out of left side (each row initially contains chevron button that opens few more rows) and right side (when you open the row these columns also populate new data). It is an accordion table, which make it more complicated.

Here is the problem:
-Left side and right side aren't connected, it seems like two separate child sections (when I want to extract the full row it doesn't select it, only the row in each side)
-When I try to first open each row and go through the whole table to the end(which opens 300+ new rows) and then go up to the top of the table, lots of things break and I don't get correct data (or no data at all)

If someone could explain to me in details the ideas around this problem I would appreciate it.

The table is all the way down at the bottom of the page:

#2

Hi,

Given the mechanics of the website, I recommend scraping the name data and the rest of the data in separate sitemaps and combining the data output. Please see the reference sitemaps below:

{"_id":"artemisanalytics-com_name","startUrl":["https://app.artemisanalytics.com/stablecoins"],"selectors":[{"elementLimit":0,"id":"name_wrapper","multiple":true,"parentSelectors":["_root"],"scroll":true,"selector":".MuiDataGrid-pinnedColumns div.MuiDataGrid-cell--withRenderer","type":"SelectorElement"},{"id":"name","multiple":false,"multipleType":"singleColumn","parentSelectors":["name_wrapper"],"regex":"","selector":"div.MuiDataGrid-treeDataGroupingCell","type":"SelectorText","version":2}]}

{"_id":"artemisanalytics-com_data","startUrl":["https://app.artemisanalytics.com/stablecoins"],"selectors":[{"elementLimit":0,"id":"row_wrapper","multiple":true,"parentSelectors":["_root"],"scroll":true,"selector":".MuiDataGrid-virtualScrollerRenderZone div.MuiDataGrid-row","type":"SelectorElement"},{"extractAttribute":"alt","id":"Chains","multipleType":"singleColumnWithSeparator","parentSelectors":["row_wrapper"],"selector":"div[data-field='child_keys'] img","type":"SelectorElementAttribute","version":2},{"id":"Pegged Currency","multiple":false,"multipleType":"singleColumn","parentSelectors":["row_wrapper"],"regex":"","selector":"div[data-field='peggedCurrency']","type":"SelectorText","version":2},{"id":"Supply","multiple":false,"multipleType":"singleColumn","parentSelectors":["row_wrapper"],"regex":"","selector":"[data-field='stablecoin_supply'] div","type":"SelectorText","version":2},{"id":"1M Change","multiple":false,"multipleType":"singleColumn","parentSelectors":["row_wrapper"],"regex":"","selector":"[data-field=\"stablecoin_supply_pct_chg\"]","type":"SelectorText","version":2}]}

Hope it helps!

#3

Thank you for your feedback! I have another problem. For clarification I will start with steps I need to take:

  1. Each row on left side has a button inside "MuiDataGrid-treeDataGroupingCellToggle" div, which I click with click selector
    a. when clicked it changes the aria-label from "see children" to "hide children"
    b. new rows that appear don't have a button element inside that class
  2. I read the data from that row which contains the button, and data from newly opened rows (which open after clicking the button)
  3. I repeat that step for every row
  4. I come all the way to the bottom of the page
  5. I want to scroll up to read data from right side of the table where all the metrics are

Here is the problem:

  1. Not every row data is taken when parsing the left side
  2. Now that all the rows are opened I need to also extract data from right side. My idea was to scroll all the way up, but I couldn't make that to work (maybe you can't do it in this app). Because if I make a new sitemap it will reload the page and close all the rows (or this can be avoided?)