'Data preview' shows data, but getting no data when exporting to .csv

How to?
#1

Here is my simple sitemap. Could someone please take a look? I'm eventually trying to extend this to look at previous and future dates, but looking to get the current date working first. Thanks

{"_id":"dividends","startUrl":["https://www.nasdaq.com/dividend-stocks/dividend-calendar.aspx"],"selectors":[{"id":"dividends","type":"SelectorTable","parentSelectors":["_root"],"selector":"table.DividendCalendar","multiple":true,"columns":[{"header":"Company (Symbol)","name":"Company (Symbol)","extract":true},{"header":"Ex-Dividend Date","name":"Ex-Dividend Date","extract":true},{"header":"Dividend","name":"Dividend","extract":true},{"header":"Indicated Annual Dividend","name":"Indicated Annual Dividend","extract":true},{"header":"Record Date","name":"Record Date","extract":true},{"header":"Announcement Date","name":"Announcement Date","extract":true},{"header":"Payment Date","name":"Payment Date","extract":true}],"delay":0,"tableDataRowSelector":"tbody tr","tableHeaderRowSelector":"thead tr"}]}

#2

The table selector seems not to work properly. You can use the element selector to build a table yourself:

{"_id":"dividends-2","startUrl":["https://www.nasdaq.com/dividend-stocks/dividend-calendar.aspx"],"selectors":[{"id":"table-element","type":"SelectorElement","parentSelectors":["_root"],"selector":"table.DividendCalendar tbody tr","multiple":true,"delay":0},{"id":"company","type":"SelectorText","parentSelectors":["table-element"],"selector":"td:nth-of-type(1)","multiple":false,"regex":"","delay":0},{"id":"ex-dividend-date","type":"SelectorText","parentSelectors":["table-element"],"selector":"td:nth-of-type(2)","multiple":false,"regex":"","delay":0},{"id":"dividend","type":"SelectorText","parentSelectors":["table-element"],"selector":"td:nth-of-type(3)","multiple":false,"regex":"","delay":0},{"id":"indicated-annual-dividend","type":"SelectorText","parentSelectors":["table-element"],"selector":"td:nth-of-type(4)","multiple":false,"regex":"","delay":0},{"id":"record-date","type":"SelectorText","parentSelectors":["table-element"],"selector":"td:nth-of-type(5)","multiple":false,"regex":"","delay":0},{"id":"announcment-date","type":"SelectorText","parentSelectors":["table-element"],"selector":"td:nth-of-type(6)","multiple":false,"regex":"","delay":0},{"id":"payment-date","type":"SelectorText","parentSelectors":["table-element"],"selector":"td:nth-of-type(7)","multiple":false,"regex":"","delay":0}]}

1 Like
#3

Thanks for your help!!!

I am a newbie to this so apologies in advance for the follow on.

But do you know how I would be able to scrape the current table/date, the table for 10 days back and also the table 20 days forward in one scrape? There are hyper links at the top of the table, but the url doesn't change.

#4

The URL does change as you click on the different dates. You can add all of them into your metadata as start URLs and grab the date from the top to identify the easier like this:

{"_id":"dividends-2","startUrl":["https://www.nasdaq.com/dividend-stocks/dividend-calendar.aspx"],"selectors":[{"id":"table-element","type":"SelectorElement","parentSelectors":["_root"],"selector":"table.DividendCalendar tbody tr","multiple":true,"delay":0},{"id":"company","type":"SelectorText","parentSelectors":["table-element"],"selector":"td:nth-of-type(1)","multiple":false,"regex":"","delay":0},{"id":"ex-dividend-date","type":"SelectorText","parentSelectors":["table-element"],"selector":"td:nth-of-type(2)","multiple":false,"regex":"","delay":0},{"id":"dividend","type":"SelectorText","parentSelectors":["table-element"],"selector":"td:nth-of-type(3)","multiple":false,"regex":"","delay":0},{"id":"indicated-annual-dividend","type":"SelectorText","parentSelectors":["table-element"],"selector":"td:nth-of-type(4)","multiple":false,"regex":"","delay":0},{"id":"record-date","type":"SelectorText","parentSelectors":["table-element"],"selector":"td:nth-of-type(5)","multiple":false,"regex":"","delay":0},{"id":"announcment-date","type":"SelectorText","parentSelectors":["table-element"],"selector":"td:nth-of-type(6)","multiple":false,"regex":"","delay":0},{"id":"payment-date","type":"SelectorText","parentSelectors":["table-element"],"selector":"td:nth-of-type(7)","multiple":false,"regex":"","delay":0},{"id":"date","type":"SelectorText","parentSelectors":["_root"],"selector":"#earnings-announcements h2","multiple":false,"regex":"","delay":0}]}

1 Like
#5

I do see how the URL changes, and so see how I could create multiple sitemaps for multiple dates.

That said, is it possible to have this all in one job?

And would there be a way to make the dates relative to the initial/start/base date (in this case, June 19th) so that the specific URL can be generated instead of passed?

Thanks again, really appreciate it. And apologies for my green-ness.

#6

Or actually, I do see how I can add multiple start URLs to the single sitemap metadata.

I guess my question then is about the relative dates. Assuming thats a tall order?

#7

And sorry to pile on - so thankful for the help - but I tried doing the same for: https://www.nasdaq.com/earnings/earnings-calendar.aspx
And coming up with nothing. Tried to follow the logic from dividends. Maybe I need to educate myself more on this... sorry.
If there is a good resource for a beginner on how to accomplish this let me know. I looked through forum and some docs, but I couldn't seem to find anything about this specific case.