Hello everyone,
Im sorry if my english is bad, it's not my native langage.
For my work I'm trying to extract data from this web page ( Schedule-Search) with pagination, i followed the tutorial (at 4 minutes) : https://youtu.be/x8bZmUrJBl0?t=241
I could copy and paste all the pages in excel file, but i want an automatic method.
So it seem I can extract all the first column (Class No) pretty easily but it seems it add me some "null data" in the CSV. I get more than 7000 entries, and you can see on the website there is "only" 4626. it's not really a problem if i can just delete them. Here what i did to have this :
So in my root i first use a
Element click with
Selector : td (select all entries in both Column)
click Selector : span a:nth-of-type(n+2) (I dont use div.dataTables_paginate)
I check "multiple"; never discard; 2000ms delay
Then In the Element click, i Add a selector : link with selector "a". I don't really understand what this does, but i need it to have my data collected, and i cant chose something else than "a" (a = QS 1)
here what i Got :
Not bad, but now i need the "caption column" and I dont understand how I can do that, except create 2 element click with in the first, select "td.sorting_1" and in the other : td:nth-of-type(2). But even with that, data will not be in the good order.
At the end, what I want is a CSV file with ALL the entries "Class No" in order on column A, and ALL the entries "caption" in the same order in Column B. Something like that :
I know the solution must be simple, but it's the first time i use this tool and I would be very grateful if you could help me.
{"_id":"nlm","startUrl":["https://classification.nlm.nih.gov/schedules/search"],"selectors":[{"id":"element click","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"td","multiple":true,"delay":2000,"clickElementSelector":"span a:nth-of-type(n+2)","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"link","type":"SelectorLink","parentSelectors":["element click"],"selector":"a","multiple":false,"delay":0}]}
EDIT: It seems i Can use the selector "table". I can extract the data for the first page, now i need to link that with element click for all the pages. Work in progress
EDIT : I did it !!!
{"_id":"nlm6","startUrl":["https://classification.nlm.nih.gov/schedules/search"],"selectors":[{"id":"table","type":"SelectorTable","parentSelectors":["pagination"],"selector":"table","multiple":true,"columns":[{"header":"Class No","name":"Class No","extract":true},{"header":"Caption","name":"Caption","extract":true}],"delay":0,"tableDataRowSelector":"tbody tr","tableHeaderRowSelector":"thead tr"},{"id":"pagination","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.container-fluid:nth-of-type(2)","multiple":true,"delay":2000,"clickElementSelector":"span a:nth-of-type(n+2)","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"}]}
First i needed a Selectorclick, and then, IN the selectorclick, i can have a selectortable