Scraping lists of interests that changes daily

How to?
#1

Hi,

I am trying to scrape the information on banks interestes rates on savings accounts. The position of the listed banks changes which means that my current setup only work for the rents before they change the listing. Can I get any advice on how I can change my selectors to follow the correct interest with the correct bank. I am trying to select all of the banks in the list.

{"_id":"tremanader","startUrl":["https://www.konsumenternas.se/konsumentstod/jamforelser/sparande--pension/eget-sparande/jamfor-sparkonton---rantor/","https://www.compricer.se/sparande/?srbindtime=3months"],"selectors":[{"id":"Alla 3m","parentSelectors":["_root"],"type":"SelectorText","selector":"tr:nth-of-type(1) div.text--shark-50, tr:nth-of-type(n+2) a.text--bold, tr:nth-of-type(n+2) td:nth-of-type(3) .d-flex div","multiple":true,"regex":""},{"id":"Aros Kapital ","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(24) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"Bluestep ","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(49) span","multiple":false,"regex":""},{"id":"Borgo","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(12) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"Brocc Finance","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(16) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"Collector","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(5) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"Danske Bank","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(25) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"Erik Penser Bank","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(4) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"Froda","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(32) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"Handelsbanken","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(46) span","multiple":false,"regex":""},{"id":"JAK","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(11) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"Marginalen Bank","parentSelectors":["_root"],"type":"SelectorText","selector":"div.js-saving-rate-row:nth-of-type(2) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"MedMera Bank","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(14) div.tw-relative","multiple":false,"regex":""},{"id":"Moank","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(37) span","multiple":false,"regex":""},{"id":"Multitude Bank","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(21) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"Nordea","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(41) span","multiple":false,"regex":""},{"id":"Nordiska","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(5) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"Northmill Bank","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(31) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"Qliro","parentSelectors":["_root"],"type":"SelectorText","selector":"div.js-saving-rate-row:nth-of-type(1) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"SBAB","parentSelectors":["_root"],"type":"SelectorText","selector":"div.js-saving-rate-row:nth-of-type(1) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"SEB","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(42) span","multiple":false,"regex":""},{"id":"Sevenday Bank","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(9) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"Skandiabanken","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(18) span.tw-whitespace-nowrap","multiple":false,"regex":""},{"id":"Swedbank","parentSelectors":["_root"],"type":"SelectorText","selector":"div:nth-of-type(43) span","multiple":false,"regex":""}]}
{id:"sitemap code"}

#2

Error 1
Error 12529×831 131 KB

As you can see it is listing the wrong element since the bank has changed place in the list

#3

Hi,

Instead of :nth-of-type(), you can use :has() or :contains() jQuery selectors. See example below:

.js-saving-rate-row:has([alt="SBAB Bank"]) div.tw-relative

image

#4

Wow,

You just saved me hours of work! Thank you so much JanAp!

#5

image
image2332×988 86.2 KB

Hopefully last question.

How would you go about these two? It only registers one of them (the one with 4,00%)

#6

If you need both, check the Multiple box. Hope it helps!

#7

Hi again JanAp,

I've been trying to scrape the interests from this web page but I just end up with blanks in the scraping. What am I doing wrong in my set up:

{"_id":"Ovriga","startUrl":["https://www.konsumenternas.se/konsumentstod/jamforelser/sparande--pension/eget-sparande/jamfor-sparkonton---rantor/","https://www.ekobanken.se/sv/hem/privat/vara-konton"],"selectors":[{"id":"Alla","multiple":true,"parentSelectors":["_root"],"regex":"","selector":"tr:nth-of-type(n+13) a.text--bold, tr:nth-of-type(n+13) .d-flex div:nth-of-type(1)","type":"SelectorText"},{"id":"Ekobanken rör","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"tr:nth-of-type(6):contains('Sparkonto') td:nth-of-type(2)","type":"SelectorText"},{"id":"Ekobanken rör 2","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"tr:contains('Ekobanken Spar (ränta beräknas från första kronan vid saldo överstigande 10.000 kr)') td:nth-of-type(2)","type":"SelectorText"},{"id":"Ekobanken 3 mån","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"tr:contains('Sparbrev med uppsägningstid 1-9 mån (förtida uttag 0,1 %, min 100 kr)') td:nth-of-type(2)","type":"SelectorText"},{"id":"Ekobanken 1 år ","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"td.onefourth","type":"SelectorText"},{"id":"Ekobanken 2 år ","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"tr:contains('Bundet konto med fast ränta 2 år, årlig ränteutbetalning') td:nth-of-type(2)","type":"SelectorText"},{"id":"Ekobanken 5 år","multiple":false,"parentSelectors":["_root"],"regex":"","selector":"tr:contains('Bundet konto med fast ränta 5 år, årlig ränteutbetalning') td:nth-of-type(2)","type":"SelectorText"}]}

image
image2549×1196 194 KB

#8

URL: https://www.ekobanken.se/sv/hem/privat/aktuella-rantor

#9

Hi,

Every website has a different structure, thus the previous sitemap will only work for the https://www.konsumenternas.se domain. A new sitemap has to be set up for https://www.ekobanken.se