Scraping Not Working for All Products

How to?
#1

Hello Friends.

I am trying to scrape information for each product. However some of the products data is not captured.

Url: Search Results – House&Hold

Sitemap:

{"_id":"Bernhardt_Dining_FromHousehold","startUrl":["https://www.houseandhold.com/pages/search-results-page?q=bernhardt%20dining"],"selectors":[{"id":"Pages","parentSelectors":["_root","Pages"],"paginationType":"auto","type":"SelectorPagination","selector":".snize-pagination a"},{"id":"Plist","parentSelectors":["Pages"],"type":"SelectorLink","selector":"a.snize-view-link","multiple":true,"linkType":"linkFromHref"},{"id":"Pname","parentSelectors":["Plist"],"type":"SelectorText","selector":"h1","multiple":false,"regex":""},{"id":"MSRP","parentSelectors":["Plist"],"type":"SelectorText","selector":".compare-price span","multiple":false,"regex":""},{"id":"Brand","parentSelectors":["Plist"],"type":"SelectorText","selector":".metaitemwrap a","multiple":false,"regex":""},{"id":"SKU","parentSelectors":["Plist"],"type":"SelectorText","selector":"span#varaint_sku","multiple":false,"regex":""},{"id":"Size","parentSelectors":["Plist"],"type":"SelectorText","selector":"ul:nth-of-type(4) li","multiple":false,"regex":""},{"id":"Material","parentSelectors":["Plist"],"type":"SelectorText","selector":"ul:nth-of-type(5) li","multiple":false,"regex":""}]}
#2

Hi,

  1. The start URL is not loading any items. Can you please explain how you executed the query?
  2. Please provide a specific example URL where data was not scraped as expected.
#3

URL is loading properly at my end.

There were total 70 Products listed on the site. However I am getting data for only 60 products. Some products are not getting captured. It is difficult to find which ones are missing.

#4

Is it the same as this query: Search: 66 results found for "bernhardt dining" – House&Hold

#5

Actually I am getting "Showing 70 results for "bernhardt dining"

#6

Are you by any chance logged in?

#7

Not Logged In

image
image1881×689 82.2 KB