It's there a better way to scraper?

How to?
#1

Hi,
Hope you can help me if you know how to solve it.
Url: www.mollyjewelryus.com

1.I want to scrape products from 14 different gemstone categories. However, when I hover over “Gemstones,” — ruby, peridot, sapphire, opal, citrine, sunstone, and tanzanite — can not be selected,

Do you know why this happens and how I can make all of them selectable?
Since I couldn’t select all the categories, I tried selecting them a second time, but I’m not sure if my method is correct.

I’ve included my sitemap at the end — please take a look.

834fd0652e72d174a2aa661ee695015c
834fd0652e72d174a2aa661ee695015c1896×866 280 KB

2.When I select one of the gemstones and open its ring collection page, some of the pages have pagination while others don’t.
In this case, should I select the pagination first?
Pls help me after you check my sitemap!
Maybe my whole logic is wrong.

1bab11b6dd0c41aa8da93cea523b4c73
1bab11b6dd0c41aa8da93cea523b4c731908×642 201 KB

Uploading: 3d5151ae-e913-48a7-b0b1-4d98f1185886.png...

Sitemap: {"_id":"mollygemstones","startUrl":["https://mollyjewelryus.com/"],"selectors":[{"id":"7choose","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":".e-lazyloaded .elementor-element-bcc1373 a","type":"SelectorLink"},{"id":"another7","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":".e-lazyloaded .elementor-element-6aacadd .elementor-icon-list-item a","type":"SelectorLink"},{"id":"differentpage","paginationType":"auto","parentSelectors":["7choose","differentpage"],"selector":"a.next","type":"SelectorPagination"},{"id":"eachproduct link","linkType":"linkFromHref","multiple":true,"parentSelectors":["7choose"],"selector":".c-product-grid__title-wrap a:nth-of-type(1)","type":"SelectorLink"},{"id":"titile","multiple":false,"multipleType":"singleColumn","parentSelectors":["eachproduct link"],"regex":"","selector":"h1","type":"SelectorText"},{"id":"price","multiple":false,"multipleType":"singleColumn","parentSelectors":["eachproduct link"],"regex":"","selector":"p ins bdi","type":"SelectorText"},{"id":"image1","multiple":false,"multipleType":"singleColumn","parentSelectors":["eachproduct link"],"selector":".active img.c-product__slider-img","type":"SelectorImage"},{"id":"image2","multiple":false,"multipleType":"singleColumn","parentSelectors":["eachproduct link"],"selector":".active img.c-product__slider-img","type":"SelectorImage"},{"id":"pagination","paginationType":"auto","parentSelectors":["another7","pagination"],"selector":"a.next","type":"SelectorPagination"},{"id":"eachlink","linkType":"linkFromHref","multiple":true,"parentSelectors":["another7"],"selector":".c-product-grid__title-wrap a:nth-of-type(1)","type":"SelectorLink"},{"id":"title","multiple":false,"multipleType":"singleColumn","parentSelectors":["eachlink"],"regex":"","selector":"h1","type":"SelectorText"},{"id":"prices","multiple":false,"multipleType":"singleColumn","parentSelectors":["eachlink"],"regex":"","selector":"p bdi","type":"SelectorText"},{"id":"image11","multiple":false,"multipleType":"singleColumn","parentSelectors":["eachlink"],"selector":".active img.c-product__slider-img","type":"SelectorImage"},{"id":"image22","multiple":false,"multipleType":"singleColumn","parentSelectors":["eachlink"],"selector":".active img.c-product__slider-img","type":"SelectorImage"}]}

#2

image

maybe it is better to select SHOP ALL GEMSTONE RINGS and get only 1 URL to srape...
Afterwards,there is no problem to follow all pages and scrape what you need.... No problems at all...

#3

Hi,don
Thanks for your help.
Yeah, I notice there is a button "shop all gemstone rings".
The reason I want to select each gemstone individually is so that, after scraping, the data I download to Excel can be organized according to the gemstone name. This will be very helpful for my later analysis.

Also, if I can understand how to solve this issue, I’ll know what to do the next time I encounter a similar problem.

if you know how to fix it, pls let me know.

Thanks again~

#4

you can easily extract gemstone name from the title:

image
image1122×422 53.3 KB

#5

Thanks for your reply again~
Yes, I understand the title is including the gemstone name.

But it's annoying that some titles are including 2 or 3 gemstone name.

That's sad.

Mic

#6

Thanks for your reply again~
Yes, I understand the title is including the gemstone name.

But it's annoying that some titles are including 2 or 3 gemstone name.

That's sad.

Mic