Right data are in the data preview, but are missing after scraping

#1

Hello,

my following sitemap doesn't work anymore:

{"_id":"gelbeseiten-grillgeraete","startUrl":["https://www.gelbeseiten.de/Suche/Grillgeräte%20und%20-zubehör/Bundesweit"],"selectors":[{"id":"card","parentSelectors":["_root"],"type":"SelectorElementClick","clickElementSelector":"a.mod-LoadMore--button","clickElementUniquenessType":"uniqueText","clickType":"clickMore","delay":2000,"discardInitialElements":"do-not-discard","multiple":true,"selector":"article"},{"id":"name","parentSelectors":["card"],"type":"SelectorText","selector":"h2","multiple":false,"delay":0,"regex":""},{"id":"adresse","parentSelectors":["card"],"type":"SelectorText","selector":"p[data-wipe-name='Adresse']","multiple":false,"delay":0,"regex":""},{"id":"telefonnummer","parentSelectors":["card"],"type":"SelectorText","selector":"p.mod-AdresseKompakt__phoneNumber","multiple":false,"delay":0,"regex":""},{"id":"email","parentSelectors":["card"],"type":"SelectorLink","selector":"a.contains-icon-email","multiple":false,"delay":0},{"id":"website","parentSelectors":["card"],"type":"SelectorLink","selector":"a.contains-icon-homepage","multiple":false,"delay":0}]}

The data preview shows me the right data, but after scraping most of the data are missing.

data-preview
data-preview3200×656 139 KB

results
results1920×394 66.7 KB

Can anyone help me please?

#2

@smafox Hi, before scraping try to zoom out the page as far as you can and add an additional ''Element scrolldown'' selector.

Example:

{"_id":"gelbeseiten-grillgeraete","startUrl":["https://www.gelbeseiten.de/Suche/Grillger%C3%A4te%20und%20-zubeh%C3%B6r/Bundesweit"],"selectors":[{"clickElementSelector":"a.mod-LoadMore--button","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickMore","delay":2000,"discardInitialElements":"do-not-discard","id":"card","multiple":true,"parentSelectors":["_root"],"selector":"article","type":"SelectorElementClick"},{"delay":0,"id":"name","multiple":false,"parentSelectors":["scroll-down"],"regex":"","selector":"h2","type":"SelectorText"},{"delay":0,"id":"adresse","multiple":false,"parentSelectors":["scroll-down"],"regex":"","selector":"p[data-wipe-name='Adresse']","type":"SelectorText"},{"delay":0,"id":"telefonnummer","multiple":false,"parentSelectors":["scroll-down"],"regex":"","selector":"p.mod-AdresseKompakt__phoneNumber","type":"SelectorText"},{"delay":0,"id":"email","multiple":false,"parentSelectors":["scroll-down"],"selector":"a.contains-icon-email","type":"SelectorLink"},{"delay":0,"id":"website","multiple":false,"parentSelectors":["scroll-down"],"selector":"a.contains-icon-homepage","type":"SelectorLink"},{"delay":2000,"id":"scroll-down","multiple":true,"parentSelectors":["_root"],"selector":"article","type":"SelectorElementScroll"}]}

#3

Hi viesturs, thank you for your sitemap. It works well.

But if there are more than 1000 Elements, it doesn't work anymore.

For example:

{"_id":"gelbeseiten-new","startUrl":["https://www.gelbeseiten.de/Suche/Gartenmöbel/Bundesweit"],"selectors":[{"id":"card","parentSelectors":["_root"],"type":"SelectorElementClick","clickElementSelector":"a.mod-LoadMore--button","clickElementUniquenessType":"uniqueCSSSelector","clickType":"clickMore","delay":5000,"discardInitialElements":"do-not-discard","multiple":true,"selector":"article"},{"id":"name","parentSelectors":["scroll-down"],"type":"SelectorText","selector":"h2","multiple":false,"delay":0,"regex":""},{"id":"adresse","parentSelectors":["scroll-down"],"type":"SelectorText","selector":"p[data-wipe-name='Adresse']","multiple":false,"delay":0,"regex":""},{"id":"website","parentSelectors":["scroll-down"],"type":"SelectorLink","selector":"a.contains-icon-homepage","multiple":false,"delay":0},{"id":"scroll-down","parentSelectors":["_root"],"type":"SelectorElementScroll","selector":"article","multiple":true,"delay":5000}]}

Do you have a solution for this?