Click and links don't work on pagination

How to?
#1

Url: https://tijorifinance.com/filter/?q=market%20cap%20>%200&qb=&fields=&checksum=NaN

On this page, I can get the pagination clicks to work but not the links which will take me to the data pages within the element. Any ideas will be very helpful. Thanks in advance.

Sitemap:
Pagination

{"_id":"tijori","startUrl":["https://tijorifinance.com/filter/?q=market%20cap%20>%200&qb=&fields=&checksum=NaN"],"selectors":[{"id":"pagination","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"td.nameclass","multiple":true,"delay":2000,"clickElementSelector":"span a.paginate_button:nth-of-type(n+2)","clickType":"clickOnce","discardInitialElements":"discard-when-click-element-exists","clickElementUniquenessType":"uniqueText"}]}

Data Extraction on the pages
{"_id":"tijor_2","startUrl":["https://tijorifinance.com/company/madras-fertilizers-limited"],"selectors":[{"id":"date_1","type":"SelectorText","parentSelectors":["_root"],"selector":"#p_and_lsmyTable th.col0","multiple":false,"regex":"","delay":0},{"id":"data_1","type":"SelectorText","parentSelectors":["_root"],"selector":"#p_and_lsmyTable tr:nth-of-type(1) td.col0","multiple":false,"regex":"","delay":0},{"id":"date_1_1","type":"SelectorText","parentSelectors":["_root"],"selector":"tr.row11:nth-of-type(18) td.col0","multiple":false,"regex":"","delay":0}]}

#2

A bit tricky cos this site makes heavy use of javascript. Modify the sitemap below as needed. Tested with Page load delay: 6500

{"_id":"forum-tijorifinance","startUrl":["https://tijorifinance.com/filter/?q=market%20cap%20%3E%200&qb=&fields=&checksum=NaN"],"selectors":[{"id":"Results wrapper","type":"SelectorElement","parentSelectors":["_root","Paginator"],"selector":"table.dataTable tbody","multiple":false },{"id":"Click names","type":"SelectorPopupLink","parentSelectors":["Results wrapper"],"selector":"div.companyname","multiple":true },{"id":"Company","type":"SelectorText","parentSelectors":["Click names"],"selector":"div.companyname","multiple":false,"regex":"" },{"id":"Market capital","type":"SelectorText","parentSelectors":["Click names"],"selector":"div#topmcap","multiple":false,"regex":"" },{"id":"P/E","type":"SelectorText","parentSelectors":["Click names"],"selector":"div.pe","multiple":false,"regex":"" },{"id":"Paginator","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.resultsblock","multiple":true,"delay":"2500","clickElementSelector":"span a.paginate_button:nth-of-type(n+2)","clickType":"clickOnce","discardInitialElements":"discard","clickElementUniquenessType":"uniqueText"}]}

1 Like
#3

Thank you for the code. I just tried it with 6500 ms delay and while it does the pagination part, the click to company page still doesn't work. It just loops through all the pages and there is no data extracted.

Edit: Removing the pagination part ensures that the links are clicked and I go to the page and get the data. Now need to get the pagination to click part work in tandem. Need to somehow get the looping back to the results page working.

#4

Did you wait for the whole process to finish? The paginating must complete first (takes a long time) then it will start clicking company names.

#5

Ah. I wasn't aware. My mistake. Though I finally copy pasted python code from stackoverflow to get this going. Really appreciate all the help.