Need bit of help with scraping and formatting in the correct output

How to?
#1

Hi,

I'm successfully scraped the webpage and the child pages. But the release dates isn't in the same row as the release. How can I fix this so it's in the same row?

Schermafbeelding 2025-01-02 110005
Schermafbeelding 2025-01-02 1100052136×1005 433 KB

Thanks!

Url: Nieuw op Videoland in week 1 2025: nieuwe films en series - Nieuw Deze Week

Sitemap:
{"_id":"NieuwDezeWeek-Videoland","startUrl":["https://nieuwdezeweek.nl/videoland/page/[1-2]"],"selectors":[{"id":"media-cards","parentSelectors":["_root"],"type":"SelectorLink","selector":"a.movie__title","multiple":true,"linkType":"linkFromHref"},{"id":"videoland-link","parentSelectors":["media-cards"],"type":"SelectorLink","selector":"a.btn-md:nth-of-type(1)","multiple":false,"linkType":"linkFromHref"},{"id":"ReleaseDates","parentSelectors":["_root"],"type":"SelectorText","selector":"a.btn-sm","multiple":true,"regex":""}]}

#2

It is easy to fix in Excel...

#3

Hi, you have to create a wrapper element and place the data selectors inside the wrapper:

{"_id":"NieuwDezeWeek-Videoland","startUrl":["https://nieuwdezeweek.nl/videoland/page/[1-2]"],"selectors":[{"id":"media-cards","linkType":"linkFromHref","multiple":false,"parentSelectors":["wrapper"],"selector":"a.movie__title","type":"SelectorLink"},{"id":"videoland-link","linkType":"linkFromHref","multiple":false,"parentSelectors":["media-cards"],"selector":"a.btn-md:nth-of-type(1)","type":"SelectorLink"},{"id":"ReleaseDates","multiple":false,"parentSelectors":["wrapper"],"regex":"","selector":"a.btn-sm","type":"SelectorText"},{"id":"wrapper","multiple":true,"parentSelectors":["_root"],"selector":".movie","type":"SelectorElement"}]}
#4

Thanks that worked.

One more question:
I have title set to link so it can follow the sub pages. Is there an option to hide the link of the title in the output like this?

Schermafbeelding 2025-01-02 132724
Schermafbeelding 2025-01-02 1327241977×290 106 KB

{"_id":"NieuwDezeWeek-Videoland","startUrl":["https://nieuwdezeweek.nl/videoland/"],"selectors":[{"id":"Titels","linkType":"linkFromHref","multiple":false,"parentSelectors":["wrapper_for_Titels_ReleaseDates"],"selector":"a.movie__title","type":"SelectorLink"},{"id":"ReleaseDate","multiple":false,"parentSelectors":["wrapper_for_Titels_ReleaseDates"],"regex":"","selector":"a.btn","type":"SelectorText"},{"id":"wrapper_for_Titels_ReleaseDates","multiple":true,"parentSelectors":["_root"],"selector":"div.movie:nth-of-type(n+3)","type":"SelectorElement"},{"id":"Genre","multiple":false,"parentSelectors":["wrapper_for_Titels_ReleaseDates"],"regex":"","selector":"p.movie__option","type":"SelectorText"},{"id":"Speeltijd","multiple":false,"parentSelectors":["wrapper_for_Titels_ReleaseDates"],"regex":"","selector":"p.movie__time","type":"SelectorText"},{"extractAttribute":"href","id":"Link Videolad","multiple":false,"parentSelectors":["Titels"],"selector":"a.btn-md:nth-of-type(1)","type":"SelectorElementAttribute"}]}

#5

Why not just remove/hide the column in the output file?

#6

Yes I can do that, but thought if there is an simple option to hide it while scraping, it would be better.
If it's not simple I will just hide it in excel.

#7

If it is required to open the link, then it is not possible to avoid the href column.

#8

Thanks.

I've encoutered that the scraper is skipping movies subpages when scraping.
On every page there are 12 movies/series. When I monitor the scraper it goes good for the first few pages but after that it randomly skipping subbpages. You can see in the output that on some pages it is less then 12.

i tried setting request interval and page load to 3000, but that didn't help.

#9

The scraper will visit every unique URL only once. There are series with different episodes listed but with the same URL, thus they will be scraped only after the first occurrence.

#10

I really need them to be included in the list. Is there an option to include them, even if they are already scraped once?

#11

you can do it in 2 steps.... at first, you can collect ALL links on each page.... Secondly - you can gather all information from each movie's page... In Excel you can combine this data with your first step... easy )