Pagination issue regarding Wine store

How to?
#1

Hi there!

I have been reading this forum and using Web Scraper for some time! Thank you for all your work & time, this tool is amazing!

I have a problem regarding the scraping of a wine store. I don't know why it only scrapes 2 pages (60 products) instead of the entire database. I tried with href pagination but it won't work...

Can you take a look at it?

Thank you!

Url: https://www.bodeboca.com/vino/tinto

Sitemap:
{"_id":"tinto2","startUrl":["https://www.bodeboca.com/vino/tinto"],"selectors":[{"id":"product_element","type":"SelectorElementClick","selector":"div.wineblock-info-wrapper","parentSelectors":["_root"],"multiple":true,"delay":"4000","clickElementSelector":"li.pager-next a.active","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"Link","type":"SelectorLink","selector":"h2 a.bb-empathy-processed","parentSelectors":["product_element"],"multiple":false,"delay":0},{"id":"title","type":"SelectorText","selector":"h1.product-title","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"bodega","type":"SelectorText","selector":"div.field-item a","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"marca","type":"SelectorText","selector":"div.field.field-field-marca div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"denominacionorigen","type":"SelectorText","selector":"div.field-item span.doyicon","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"tipo","type":"SelectorText","selector":"div.field.field-type-term div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"anada","type":"SelectorText","selector":"div.field.field-field-vintage div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"grado","type":"SelectorText","selector":"div.field.field-type-number-float div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"produccion","type":"SelectorText","selector":"div.field.field-field-prod-vino div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"variedad","type":"SelectorText","selector":"div.field.field-field-variedad div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"cata_nariz","type":"SelectorText","selector":"div.field.field-field-nariz div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"cata_boca","type":"SelectorText","selector":"div.field.field-field-boca div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"temperatura_servicio","type":"SelectorText","selector":"div.field.field-field-temperatura div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"vinedo_desc","type":"SelectorText","selector":"div.field.field-field-direccion-viniedo div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"vinedo_edad","type":"SelectorText","selector":"div.field.field-field-plantacion div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"vinedo_suelo","type":"SelectorText","selector":"div.field.field-field-suelo div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"vinedo_vendimia","type":"SelectorText","selector":"div.field.field-field-vendimia div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"vinedo_vinificacion","type":"SelectorText","selector":"div.field.field-field-vinificacion div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"vinedo_envejecimiento","type":"SelectorText","selector":"div.field.field-field-envejecimiento div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"vinedo_embotellado","type":"SelectorText","selector":"div.field.field-field-embotellado div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"precio","type":"SelectorText","selector":"div.node-section-wrapper div.node span.uc-price","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"formato","type":"SelectorText","selector":"div.node-section-wrapper div.node span.formato","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"descripcion","type":"SelectorText","selector":"div.body p","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0}]}

#2

Hello,
the difficulty for this kind of site is to put the right parameter for the click selector.
With:
"li.pager-next a.active" you can retrieve 60 records
"a.bb-filter-pagination-registered" you can extract 100 records
At this point, I can not help you anymore because my knowledge of HTML is weak.

#3

Hi chefas,

I have tried anything I could think of and nothing is giving a good result...

Is anybody here who can help me a little?

Thanks!

#4

Hello,
I tested scratching with a different selection of wines to reduce the number of pages to 5 pages.
I managed to scrape 130 bottles.
So continue to test if this solution is good for you.
Here are the sitemaps:

{"_id":"test_bodeboca","startUrl":["https://www.bodeboca.com/vino/generoso"],"selectors":[{"id":"product_element","type":"SelectorElementClick","selector":"div.wineblock-info-wrapper","parentSelectors":["_root"],"multiple":true,"delay":"2000","clickElementSelector":"li.pager-next a.bb-filter-pagination-registered","clickType":"clickMore","discardInitialElements":true,"clickElementUniquenessType":"uniqueText"},{"id":"Link","type":"SelectorLink","selector":"a.verficha","parentSelectors":["product_element"],"multiple":false,"delay":"0"},{"id":"title","type":"SelectorText","selector":"h1.product-title","parentSelectors":["Link"],"multiple":false,"regex":"","delay":"3000"},{"id":"bodega","type":"SelectorText","selector":"div.field-item a","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"marca","type":"SelectorText","selector":"div.field.field-field-marca div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"denominacionorigen","type":"SelectorText","selector":"div.field-item span.doyicon","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"tipo","type":"SelectorText","selector":"div.field.field-type-term div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"anada","type":"SelectorText","selector":"div.field.field-field-vintage div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"grado","type":"SelectorText","selector":"div.field.field-type-number-float div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"produccion","type":"SelectorText","selector":"div.field.field-field-prod-vino div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"variedad","type":"SelectorText","selector":"div.field.field-field-variedad div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"cata_nariz","type":"SelectorText","selector":"div.field.field-field-nariz div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"cata_boca","type":"SelectorText","selector":"div.field.field-field-boca div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"temperatura_servicio","type":"SelectorText","selector":"div.field.field-field-temperatura div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"vinedo_desc","type":"SelectorText","selector":"div.field.field-field-direccion-viniedo div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"vinedo_edad","type":"SelectorText","selector":"div.field.field-field-plantacion div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"vinedo_suelo","type":"SelectorText","selector":"div.field.field-field-suelo div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"vinedo_vendimia","type":"SelectorText","selector":"div.field.field-field-vendimia div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"vinedo_vinificacion","type":"SelectorText","selector":"div.field.field-field-vinificacion div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"vinedo_envejecimiento","type":"SelectorText","selector":"div.field.field-field-envejecimiento div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"vinedo_embotellado","type":"SelectorText","selector":"div.field.field-field-embotellado div.field-item","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"precio","type":"SelectorText","selector":"div.node-section-wrapper div.node span.uc-price","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"formato","type":"SelectorText","selector":"div.node-section-wrapper div.node span.formato","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0},{"id":"descripcion","type":"SelectorText","selector":"div.body p","parentSelectors":["Link"],"multiple":false,"regex":"","delay":0}]}