It's been an hour now and nothing has been scrapped yet

How to?
#1

**Web Scraper version1.96.16:
**Chrome version138.0.7204.159:
**OSMacOs Sonoma version 14.7.1:

**Link to the site you were scrapinghttps://enginesdirect.us.com:

Sitemap{"_id":"engine_direct","startUrl":["https://enginesdirect.us.com"],"selectors":[{"id":"category_link","parentSelectors":["_root"],"type":"SelectorLink","selector":".product-categories > li > a","multiple":true,"linkType":"linkFromHref"},{"id":"subcategory_link","parentSelectors":["category_link"],"type":"SelectorLink","selector":".children a","multiple":true,"linkType":"linkFromHref"},{"id":"product_link","parentSelectors":["subcategory_link"],"type":"SelectorLink","selector":".woocommerce-loop-product__title a","multiple":true,"linkType":"linkFromHref"},{"id":"name","parentSelectors":["product_link"],"type":"SelectorText","selector":"h1.product_title","multiple":false,"regex":""},{"id":"price","parentSelectors":["product_link"],"type":"SelectorText","selector":"p bdi","multiple":false,"regex":""},{"id":"Short_description","parentSelectors":["product_link"],"type":"SelectorText","selector":".entry-summary-inner li, .woocommerce-product-details__short-description p","multiple":false,"regex":""},{"id":"description","parentSelectors":["product_link"],"type":"SelectorText","selector":"div.woocommerce-Tabs-panel--description","multiple":false,"regex":""},{"id":"image","parentSelectors":["product_link"],"type":"SelectorImage","selector":"img.wp-post-image","multiple":false}]}

(Please make the sitemap as minimal as possible so it’s easier to replicate the bug. You can export the sitemap by opening it and choosing “Export Sitemap” in the dropdown menu):

{id:"engine_direct"}

*Log Information

Screenshot 2025-07-26 at 2.55.36 PM
Screenshot 2025-07-26 at 2.55.36 PM1918×484 108 KB

  • (Screen Record):

Tutorial on how to submit a video bug report: How to submit a video bug report

#2

Hi,

There are a couple of issues with your sitemap.

  1. The category_link selector does not match any elements on the start URL https://enginesdirect.us.com/
    I assume the start URL should be https://enginesdirect.us.com/shop/

  2. I would recommend replacing the subcategory_link selector with a pagination selector to go over all product pages.

Here is a reference: {"_id":"engine_direct","startUrl":["https://enginesdirect.us.com/shop/"],"selectors":[{"id":"category_link","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":".product-categories > li > a","type":"SelectorLink"},{"id":"pagination","paginationType":"auto","parentSelectors":["category_link","pagination"],"selector":"a.next","type":"SelectorPagination"},{"id":"product_link","linkType":"linkFromHref","multiple":true,"parentSelectors":["pagination"],"selector":".woocommerce-loop-product__title a","type":"SelectorLink"},{"id":"name","multiple":false,"parentSelectors":["product_link"],"regex":"","selector":"h1.product_title","type":"SelectorText"},{"id":"price","multiple":false,"parentSelectors":["product_link"],"regex":"","selector":"p bdi","type":"SelectorText"},{"id":"Short_description","multiple":false,"parentSelectors":["product_link"],"regex":"","selector":".entry-summary-inner li, .woocommerce-product-details__short-description p","type":"SelectorText"},{"id":"description","multiple":false,"parentSelectors":["product_link"],"regex":"","selector":"div.woocommerce-Tabs-panel--description","type":"SelectorText"},{"id":"image","multiple":false,"parentSelectors":["product_link"],"selector":"img.wp-post-image","type":"SelectorImage"}]}