"Data preview" shows data, but scrape returns empty cells

How to?
#1

Hi, not sure where to categorize this, but I've been trying to scrape image urls with this script and I do get results when "previewing" data, but when I go to "full" scrape cells are empty. Any ideas?

{"_id":"brickset-2017-images","startUrl":["https://brickset.com/sets/year-2018/theme-Star-Wars","https://brickset.com/sets/year-2018/theme-City","https://brickset.com/sets/year-2018/theme-BrickHeadz","https://brickset.com/sets/year-2018/theme-Classic","https://brickset.com/sets/year-2018/theme-Creator","https://brickset.com/sets/year-2018/theme-DC-Comics-Super-Heroes","https://brickset.com/sets/year-2018/theme-Disney","https://brickset.com/sets/year-2018/theme-Duplo","https://brickset.com/sets/year-2018/theme-Elves","https://brickset.com/sets/year-2018/theme-Friends","https://brickset.com/sets/year-2018/theme-Juniors","https://brickset.com/sets/year-2018/theme-Marvel-Super-Heroes","https://brickset.com/sets/year-2018/theme-Minecraft","https://brickset.com/sets/year-2018/theme-Ninjago","https://brickset.com/sets/year-2018/theme-Star-Wars","https://brickset.com/sets/year-2018/theme-Technic","https://brickset.com/sets/year-2018/theme-The-LEGO-Batman-Movie"],"selectors":[{"id":"proizvod","type":"SelectorElementClick","selector":"section.main","parentSelectors":["krekre"],"multiple":false,"delay":"2000","clickElementSelector":"a:contains('More images')","clickType":"clickOnce","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"fotka1-src","type":"SelectorImage","selector":"li:nth-of-type(1) img","parentSelectors":["proizvod"],"multiple":false,"delay":"200"},{"id":"fotka2-src","type":"SelectorImage","selector":"li:nth-of-type(2) img","parentSelectors":["proizvod"],"multiple":false,"delay":""},{"id":"fotka3-src","type":"SelectorImage","selector":"li:nth-of-type(3) img","parentSelectors":["proizvod"],"multiple":false,"delay":""},{"id":"fotka4-src","type":"SelectorImage","selector":"li:nth-of-type(4) img","parentSelectors":["proizvod"],"multiple":false,"delay":""},{"id":"fotka5-src","type":"SelectorImage","selector":"li:nth-of-type(5) img","parentSelectors":["proizvod"],"multiple":false,"delay":""},{"id":"fotka6-src","type":"SelectorImage","selector":"li:nth-of-type(6) img","parentSelectors":["proizvod"],"multiple":false,"delay":""},{"id":"fotka7-src","type":"SelectorImage","selector":"li:nth-of-type(7) img","parentSelectors":["proizvod"],"multiple":false,"delay":""},{"id":"fotka8-src","type":"SelectorImage","selector":"li:nth-of-type(8) img","parentSelectors":["proizvod"],"multiple":false,"delay":""},{"id":"fotka9-src","type":"SelectorImage","selector":"li:nth-of-type(9) img","parentSelectors":["proizvod"],"multiple":false,"delay":""},{"id":"fotka10-src","type":"SelectorImage","selector":"li:nth-of-type(10) img","parentSelectors":["proizvod"],"multiple":false,"delay":""},{"id":"fotka11-src","type":"SelectorImage","selector":"li:nth-of-type(11) img","parentSelectors":["proizvod"],"multiple":false,"delay":""},{"id":"fotka12-src","type":"SelectorImage","selector":"li:nth-of-type(12) img","parentSelectors":["proizvod"],"multiple":false,"delay":""},{"id":"set number","type":"SelectorText","selector":"dt:contains('Set number')+dd","parentSelectors":["proizvod"],"multiple":false,"regex":"","delay":""},{"id":"krekre","type":"SelectorLink","selector":"div.meta h1 a","parentSelectors":["_root"],"multiple":true,"delay":""}]}

#2

Your sitemap doesn't go to the image page. You are missing the part when scraper goes to item page it has to go to "Parts" page to access the images.

#3

I'm puzzled... This exact script used to work nicely a while ago and I don't see any updates to the site...
I will try few more things... Thanks!

#4

Actually, when I am looking at the scraper opening the pages in the Scrape mode - everything looks good. Pages are opened properly, including tabs, but no data (in some fields) is shown in the final export

#5

image
image.png1901×230 11.5 KB

Please see the screenshot... All fields are working fine, except for the image urls... At the same time, when I verify the scrape on the page, I get proper results:

image
image.jpg877×727 157 KB

#6

Can you post a URL to this page?

#7

Can you be a little more precise please? This scrape used to work properly... I might be missing something.

#8

What URL do you need exactly?
This is an example page I'm trying to scrape:
https://brickset.com/sets/year-2018/theme-Duplo
Just image URLS from the "more images tab": https://brickset.com/sets/10859-1/My-First-Ladybird

#9

Check Discard initial elements that are available before click on your element click selector and it should work fine.

#10

Ah, yes it does work now. Thanks so much. Was this changed with an update or something? I didn't use this before.

#11

By looking at the configuration of the sitemap it is not possible that the sitemap worked before the update. Sitemap probably didn't work anymore because the site have been changed.

#12

Thanks much. Really appreciate your help. :grinning:

#13

I'm having this problem too. I have one scraper working, but another shows data in Preview but "No Data Scraped Yet" and nothing in the downloaded data file except column headers.
This answer, "Check Discard initial elements that are available before click on your element click selector " looked like the right answer, but I can't see that option Discard Initial Elements anywhere. Where should I see it?

1 Like
#14

So do I , where is the 'Check Discard initial elements'? :joy:

#15

It is a checkbox for element click selector.

#16

Got it. Thank you~:grin: