Trouble Getting Element Selector to Comply

How to?
#1

Describe the problem: when I created an element parent selector within a pagination selector, all that was returned was null, but when I don't have the element, I receive the information, but not in an organized row.

Url: Word Study Tool - Numen - The Latin Lexicon - An Online Latin Dictionary - A Dictionary of the Latin Language

Sitemap:
{"_id":"latinlexicon","startUrl":["https://latinlexicon.org/word_study_tool.php"],"selectors":[{"id":"next","parentSelectors":["_root","next"],"paginationType":"auto","selector":"div#advance_container","type":"SelectorPagination"},{"id":"flashcardtitle","parentSelectors":["flashcardelement"],"type":"SelectorText","selector":"div.flash_card_title","multiple":false,"delay":0,"regex":""},{"id":"pos","parentSelectors":["flashcardelement"],"type":"SelectorText","selector":"div.flash_card_info:nth-of-type(n+3)","multiple":false,"delay":0,"regex":""},{"id":"flashcardelement","parentSelectors":["next"],"type":"SelectorElementAttribute","selector":"div#lookup_div_element","multiple":true,"delay":0,"extractAttribute":""},{"id":"ol1","parentSelectors":["flashcardelement"],"type":"SelectorText","selector":"li:nth-of-type(1)","multiple":false,"delay":0,"regex":""},{"id":"ol2","parentSelectors":["flashcardelement"],"type":"SelectorText","selector":"li:nth-of-type(2)","multiple":false,"delay":0,"regex":""},{"id":"ol3","parentSelectors":["flashcardelement"],"type":"SelectorText","selector":"li:nth-of-type(3)","multiple":false,"delay":0,"regex":""},{"id":"ol4","parentSelectors":["flashcardelement"],"type":"SelectorText","selector":"li:nth-of-type(4)","multiple":false,"delay":0,"regex":""},{"id":"ol5","parentSelectors":["flashcardelement"],"type":"SelectorText","selector":"li:nth-of-type(5)","multiple":false,"delay":0,"regex":""},{"id":"ol6","parentSelectors":["flashcardelement"],"type":"SelectorText","selector":"li:nth-of-type(6)","multiple":false,"delay":0,"regex":""},{"id":"ol7","parentSelectors":["flashcardelement"],"type":"SelectorText","selector":"li:nth-of-type(7)","multiple":false,"delay":0,"regex":""},{"id":"ol8","parentSelectors":["flashcardelement"],"type":"SelectorText","selector":"li:nth-of-type(8)","multiple":false,"delay":0,"regex":""},{"id":"ol9","parentSelectors":["flashcardelement"],"type":"SelectorText","selector":"li:nth-of-type(9)","multiple":false,"delay":0,"regex":""}]}

Any help would be much appreciated, if I am being stupid, please tell me.

#2

@rembrandt Hi. After having a look at your sitemap, it doesn't appear that the start URL would have any of the selectors you have applied. Is it the correct start URL?

example
example1755×1182 176 KB

#3

Yes it is the correct start URL, but the URL doesn't change after searching up a word.

And so, the only way I know how to get the right pagination is by pasting the words and clicking enter once the page for scraping has loaded, and before Webscraper stops and scrapes nothing.

My apologies, I submitted the wrong sitemap, here is the right one:

{"_id":"latinlexicon","startUrl":["https://latinlexicon.org/word_study_tool.php"],"selectors":[{"id":"next","paginationType":"clickMore","parentSelectors":["_root","next"],"selector":"div#advance_container","type":"SelectorPagination"},{"delay":0,"id":"title","multiple":true,"parentSelectors":["next"],"regex":"","selector":"div:nth-of-type(4) div.flash_card_title","type":"SelectorText"},{"delay":0,"id":"info","multiple":true,"parentSelectors":["next"],"regex":"","selector":"div:nth-of-type(4) div.flash_card_info","type":"SelectorText"},{"delay":0,"id":"ol1","multiple":true,"parentSelectors":["next"],"regex":"","selector":"div:nth-of-type(5) li:nth-of-type(1)","type":"SelectorText"},{"delay":0,"id":"ol2","multiple":true,"parentSelectors":["next"],"regex":"","selector":"li:nth-of-type(2)","type":"SelectorText"},{"delay":0,"id":"ol3","multiple":true,"parentSelectors":["next"],"regex":"","selector":"li:nth-of-type(3)","type":"SelectorText"},{"delay":0,"id":"ol4","multiple":true,"parentSelectors":["next"],"regex":"","selector":"li:nth-of-type(4)","type":"SelectorText"},{"delay":0,"id":"ol5","multiple":true,"parentSelectors":["next"],"regex":"","selector":"li:nth-of-type(5)","type":"SelectorText"},{"delay":0,"id":"ol6","multiple":true,"parentSelectors":["next"],"regex":"","selector":"li:nth-of-type(6)","type":"SelectorText"}]}

I would like the header of the Latin word, the info underneath it (the last line of info, not the ones before) and the translations all in one row, but when I use an element selector, I cannot choose the right text selector.

To test, paste a group of Latin words like these:
adulescens
advenio
adversus
aedificium
aedifico

As a side note, this is not that important, but when scraping it stops once it encounters a definition is has already gone through.

Many thanks,
-R.

#4

For this site, you can access the word lookups directly with this type of URL. The layout would look different though:

https://latinlexicon.org/ajax/lookup_entry_morphology.php?p1=adulescens
https://latinlexicon.org/ajax/lookup_entry_morphology.php?p1=adversus 
https://latinlexicon.org/ajax/lookup_entry_morphology.php?p1=incognito

So you just need to build a list of starturls for scraping. Also see:

#5

Thank you so much for this, it solved two problems at once.
However, I am not quite sure on how to efficiently add all of the list of starturls for scraping, do you have any further advice? Any help would be much appreciated.
Sorry for the bother.

R.

#6

@rembrandt Hi. You can use a custom approach which can be seen in the following video - 2019 - UnTECH # 17 - Scraping Multiple URL's with Webscraper.io - YouTube

#7

Thank you for all the help - it succeeded!
I am sorry to reply so late, I just felt that you should know.

- R.