Extract URL from popup link redirect

How to?
#1

It's unclear to me how to get the final url on a redirect using WebScraper browser extension. This question has been asked a handful of times, and i do not see a clear answer posted. The site I'm scraping has links on the page, which I can extract the URL for, but that link is in front of at least one redirect. I am after the final URL of the redirect(s). I tried changing the type to "popup link" but that doesn't do it. Any and all help is much appreciated!

I'm trying to grab the final URL of the "See Deal" link.

Sitemap:
{id:"sitemap code"}

#2

Not really possible, as the final link you are after does not existing on this page.

The only way of getting the correct data would be by following the redirect link and getting the correct URL from the amazon page.

Something along the lines of this:

{"_id":"slickdeals-see-more","startUrl":["https://slickdeals.net/f/13782314-philips-sonicare-essence-rechargeable-electric-toothbrush-19-95-fs-w-prime?src=frontpage"],"selectors":[{"id":"see-more-url","type":"SelectorLink","parentSelectors":["_root"],"selector":"a#buyNow","multiple":true,"delay":0},{"id":"final-url","type":"SelectorText","parentSelectors":["see-more-url"],"selector":"script[type=\"a-state\"][data-a-state]:contains(\"shareWithFriendsPrompt\")","multiple":true,"regex":"(?<=url\\\":\\\")[^\\\"]+","delay":0}]}

1 Like
#3

This works great! Why not use the popup link selector though? How are these different?

#4

The redirect url is actually buried within the source, but you can't get at it with the regular WS element picker. Also, it is URL-encoded, so you would need post-processing to decipher it. Try:

Type: Element attribute
Selector: div[id='originalPost'] a
Attribute name: href

Redirect url is the part after &u2=, towards the end.

#5

Awesome find! I'll have to check the docs on post processing. Thanks for the alternate method. More efficient too.