How to scrape elements in new window?

How to?
#1

I don't understand how to scrape information wich includes in new blank.
Site Avito shows an ads with shortened text. Full information open in new blank. When I use "link", scraper just opened it and do nothing else to scrape text in new window.

https://www.avito.ru/samara/vakansii?q=с+ежедневной+оплатой+подработка

Sitemap:
{"_id":"avitovideo","startUrl":["https://www.avito.ru/samara/vakansii?q=с+ежедневной+оплатой+подработка"],"selectors":[{"id":"link","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"#i2076906681 div.iva-item-content-m2FiN:nth-of-type(1)","multiple":true,"delay":2000,"clickElementSelector":"#i2076906681 h3","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"block","type":"SelectorElement","parentSelectors":["_root"],"selector":"div.l-content","multiple":false,"delay":0},{"id":"title","type":"SelectorText","parentSelectors":["block"],"selector":"span.title-info-title-text","multiple":false,"regex":"","delay":0},{"id":"description","type":"SelectorText","parentSelectors":["block"],"selector":"div.item-view-block:nth-of-type(3)","multiple":false,"regex":"","delay":0}]}

https://www.avito.ru/samara/vakansii?q=с+ежедневной+оплатой+подработка

#2

Hello @servolk56

You can't proceed further with element click because the link changes, you have to use link selector here.

See my example:

{"_id":"avitovideo","startUrl":["https://www.avito.ru/samara/vakansii?p=1&q=с+ежедневной+оплатой+подработка","https://www.avito.ru/samara/vakansii?p=2&q=с+ежедневной+оплатой+подработка","https://www.avito.ru/samara/vakansii?p=3&q=с+ежедневной+оплатой+подработка","https://www.avito.ru/samara/vakansii?p=4&q=с+ежедневной+оплатой+подработка","https://www.avito.ru/samara/vakansii?p=5&q=с+ежедневной+оплатой+подработка","https://www.avito.ru/samara/vakansii?p=6&q=с+ежедневной+оплатой+подработка"],"selectors":[{"id":"block","type":"SelectorElement","parentSelectors":["_root"],"selector":"[data-marker=\"item\"]","multiple":true,"delay":0},{"id":"title","type":"SelectorText","parentSelectors":["block"],"selector":"h3","multiple":false,"regex":"","delay":0},{"id":"description","type":"SelectorText","parentSelectors":["card"],"selector":"div[itemprop='description']","multiple":false,"regex":"","delay":0},{"id":"link","type":"SelectorLink","parentSelectors":["block"],"selector":"a[data-marker=\"item-title\"]","multiple":false,"delay":0},{"id":"card","type":"SelectorElement","parentSelectors":["link"],"selector":"body:has(h1.title-info-title)","multiple":true,"delay":0}]}

Hope it helps.

#3

Excellent!!! Thank you very much!
It's fucking works!))

But, after some pages, site has blocked my IP with capcha.
Does anybody knows, how to cheat this?

#4

@servolk56 You need a proxy. Try out Cloud Scraper with proxy enabled: Web Scraper - Pricing

When Proxy is enabled for a scraping job:

  • The IP address of the scraper will be rotated at an interval
  • The scraper will automatically rotate its IP address when a page containing CAPTCHA or a page containing elements that signal that the access is denied due to the scraper being blocked is encountered.