Only selects first description (of 50) on category page (child selector under element)

Hi @ViestursWS :slight_smile:

Web Scraper version: 1.29.60
Chrome version: 113.0.5672.129 (Official Build) (64-bit)
OS: Windows 10 Pro 64-bit

Sitemap:
{"_id":"BBCW-Search-All-Brands-In-stock-Page","startUrl":["https://www.bbcw.com/?target=search&mode=search&including=all&categoryId[0]=252&categoryId[1]=no_category&categoryId[2]=no_category&categoryId[3]=no_category"],"selectors":[{"id":"Pagination","parentSelectors":["_root","Pagination"],"paginationType":"auto","selector":".list-pager-bottom .selected span, .list-pager-bottom .next-page a","type":"SelectorPagination"},{"id":"Product Wrappers","parentSelectors":["_root","Pagination"],"type":"SelectorElement","selector":"div.product","multiple":true},{"id":"Title Link","parentSelectors":["Product Wrappers"],"type":"SelectorLink","selector":"a.fn","multiple":false},{"id":"SKU","parentSelectors":["Product Wrappers"],"type":"SelectorText","selector":".sku div.bbcw-value","multiple":false,"regex":""},{"id":"Status","parentSelectors":["Product Wrappers"],"type":"SelectorText","selector":".status div.bbcw-value","multiple":false,"regex":""},{"id":"Manufacturer","parentSelectors":["Product Wrappers"],"type":"SelectorLink","selector":".bbcw-value a","multiple":false},{"id":"Item Type","parentSelectors":["Product Wrappers"],"type":"SelectorText","selector":"div.bbcw-product-info-item:nth-of-type(3) div.bbcw-value","multiple":false,"regex":""},{"id":"Price","parentSelectors":["Product Wrappers"],"type":"SelectorText","selector":"span.price","multiple":false,"regex":""},{"id":"Min Quantity","parentSelectors":["Product Wrappers"],"type":"SelectorText","selector":"span.pricing-notation","multiple":false,"regex":""},{"id":"Description","parentSelectors":["Product Wrappers"],"type":"SelectorText","selector":"p","multiple":false,"regex":""}]}

Only selects first description (of 50) on category/search URL (child selector, under element).

I originally set this up as a child of my pagination selector which allowed me to select multiple "descriptions" before the tool auto selected the rest (50 in total). This game me .productid-66731 p, li:nth-of-type(n+2) div.description.

However I soon realized I needed to use the Element selector under pagination to get everything under the same line in excel when exported.

All my other child selectors worked fine when past under the element selector, but this "description one" showed zero results...

So I re did the selection for it again now under the element selector. This only allowed me to select one description on the page since I was now selecting within the highlighted element. This game me solely "p"

The result is that even thou the Element selector is set to "multiple" and the "Description" selector, a long with all the other child selectors are not, all the child selectors bring back all data with the exception of this "description" one that only brings back the first description...


I also tried doing just the description directly under my pagination selector, but then it omits any descriptions that happen to be identical, making it impossible to sync up the data in excel afterwords.

Image 01:

Image 02:
Scraper 02

Error Message:
Download the React DevTools for a better development experience: React Developer Tools – React
vendors.js:5770 You are currently using minified code outside of NODE_ENV === "production". This means that you are running a slower development build of Redux. You can use loose-envify (GitHub - zertosh/loose-envify: Like envify but much faster) for browserify or setting mode to production in webpack (https://webpack.js.org/concepts/mode/) to ensure you have the correct code for your production build.

Thanks a million!

Regards
Jayen

@JustAnotherUser Hello, this happens because some of the product cards do not have paragraphs. To extract the product description you can use the following 'Text' selector: .description.product-description

1 Like

Thank you ever so much for this! Its totally awesome that your tool allows us to manually adjust the hook from the page source code, the other scrapper tools I tried had no way to do this :slight_smile:

I also love you can test and see the result just by hitting the preview bottom right next to your text selector.

I just played with it a bit, I see the format needed now and that I could have put div in-front of the ".", but clearly its not necessary.

I was just thinking, it would be great if in the preview windows it added up the total cells it finds data for above each column.

That way if you know how many are on the page you can quickly see if any aren't being picked up!

Also assuming some aren't being picked up:

When looking for these cases with the element preview option, it may work for the element selector it self (product wrappers), but for child selectors under one of these, the preview is only going to show you the first product wrapper and the child within it. You can't see all the others down the page.

Could this not be changed so all were highlighted on the page allowing for quick and easy localization of the cases where the selectors weren't working?

Thanks

Have an awesome day :smiley:

@JustAnotherUser Thank you for your suggestion. We would appreciate it if you could share the concept in more detail by also showing this with the help of illustrations here: Feature Requests - Web Scraper