Job posting collector

#1

I was trying to use this to collect the results from several job search sites like Monster and Indeed into one list, but no data is extracted even though it seems that I should be getting some type of result.

For example:

{"_id":"indeed","startUrl":["https://www.indeed.com/jobs?q=technology&l=CT"],"selectors":[{"id":"results","type":"SelectorElement","parentSelectors":["_root"],"selector":"td#resultsCol","multiple":true,"delay":0},{"id":"title","type":"SelectorText","parentSelectors":["results"],"selector":"div.jobsearch-SerpJobCard.vjs-highlight a.jobtitle","multiple":false,"regex":"","delay":0},{"id":"description","type":"SelectorText","parentSelectors":["results"],"selector":"div.jobsearch-SerpJobCard.vjs-highlight span.summary","multiple":false,"regex":"","delay":0}]}

#2

I don't think you can make one recipe for multiple sites as each site uses a unique layout of elements. Your current sitemap doesn't work because your element selectors aren't correct.

Quick a dirty, this will get you name, title, location

{"_id":"indeed","startUrl":["https://www.indeed.com/jobs?q=technology&l=CT"],"selectors":[{"id":"Pag","type":"SelectorLink","parentSelectors":["_root","Pag"],"selector":"a:contains("Next")","multiple":false,"delay":0},{"id":"Element","type":"SelectorElement","parentSelectors":["_root","Pag"],"selector":".result","multiple":true,"delay":0},{"id":"Job","type":"SelectorText","parentSelectors":["Element"],"selector":"a.turnstileLink","multiple":false,"regex":"","delay":0},{"id":"Company","type":"SelectorText","parentSelectors":["Element"],"selector":".company","multiple":false,"regex":"","delay":0},{"id":"Location","type":"SelectorText","parentSelectors":["Element"],"selector":"span.location","multiple":false,"regex":"","delay":0},{"id":"Date","type":"SelectorText","parentSelectors":["Element"],"selector":"span.date","multiple":false,"regex":"","delay":0}]}

#3

Oh right I can only do one site at a time so maybe it is not so useful. I tried putting in the code in the sitemap json import but it says it is invalid. Is that what I do to try the code?

#4

Hmm.. Try this

{"_id":"indeed","startUrl":["https://www.indeed.com/jobs?q=technology&l=CT"],"selectors":[{"id":"Pag","type":"SelectorLink","parentSelectors":["_root","Pag"],"selector":"a:contains(\"Next\")","multiple":false,"delay":0},{"id":"Element","type":"SelectorElement","parentSelectors":["_root","Pag"],"selector":".result","multiple":true,"delay":0},{"id":"Job","type":"SelectorText","parentSelectors":["Element"],"selector":"a.turnstileLink","multiple":false,"regex":"","delay":0},{"id":"Company","type":"SelectorText","parentSelectors":["Element"],"selector":".company","multiple":false,"regex":"","delay":0},{"id":"Location","type":"SelectorText","parentSelectors":["Element"],"selector":"span.location","multiple":false,"regex":"","delay":0},{"id":"Date","type":"SelectorText","parentSelectors":["Element"],"selector":"span.date","multiple":false,"regex":"","delay":0}]}
#5

Hi!

I tried using the code you posted, but instead of company name as text I also want to get the company link from indeed. I tried to change text to link for Company, but it doesn't return anything:

The code is:

{"_id":"works","startUrl":["https://nl.indeed.com/jobs?q=engineer&l=Nederland&lang=en&start=[0-200:10]"],"selectors":[{"id":"Element","type":"SelectorElement","parentSelectors":["_root"],"selector":".result","multiple":true,"delay":0},{"id":"Job","type":"SelectorLink","parentSelectors":["Element"],"selector":"a.turnstileLink","multiple":false,"delay":0},{"id":"Location","type":"SelectorText","parentSelectors":["Element"],"selector":"span.location","multiple":false,"regex":"","delay":0},{"id":"Date","type":"SelectorText","parentSelectors":["Element"],"selector":"span.date","multiple":false,"regex":"","delay":0},{"id":"Company","type":"SelectorLink","parentSelectors":["Element"],"selector":".company","multiple":false,"delay":0},{"id":"Summary","type":"SelectorText","parentSelectors":["Element"],"selector":"ul","multiple":false,"regex":"","delay":0}]}

Thanks in advance for your help!