Element limit function in Element scroll down

How to?
#1

Dear Sir,

Respectfully, I want to limit the scroll down of a page to limited elements. I have tried several times to use the "element limit" option in "Element scroll down" but it does not work. For example, I have set a 50 elements limit but more elements are scrolled! It takes too much time for me to scrape data that has been repeated before. I have checked your forum for similar questions but they were for more than 3 years ago and did not work for me. I would appreciate to let me know how to solve this problem.

Furthermore please:

1- Shall I select several elements in "Element scroll down"?
2- How can I de-select an entire element among several elements that are selected? (I am not meaning parent and child element selection)

Thank you for your help and consultation.

Url: خرید و فروش و قیمت هیوندای سانتافه امروز

Sitemap:
{"_id":"Santafeix45-main","startUrl":["https://bama.ir/car/hyundai-santafeix45"],"selectors":[{"delay":1000,"elementLimit":50,"id":"scroll","multiple":true,"parentSelectors":["_root"],"selector":"a.bama-ad","type":"SelectorElementScroll"},{"id":"santafe-link","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":"a.bama-ad","type":"SelectorLink"},{"id":"name","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"h1.bama-ad-detail-title__title","type":"SelectorText"},{"id":"man-year","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"span.bama-ad-detail-title__subtitle:nth-of-type(1)","type":"SelectorText"},{"id":"engine","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"span.bama-ad-detail-title__subtitle:nth-of-type(3)","type":"SelectorText"},{"id":"date","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"span.bama-ad-detail-title__ad-time","type":"SelectorText"},{"id":"city","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"span.address-text","type":"SelectorText"},{"id":"price","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"span.bama-ad-detail-price__price-text","type":"SelectorText"},{"id":"seller","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":".dealer-banner__title-holder span","type":"SelectorText"},{"id":"seller-adress","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"span.dealer-banner__sub-title","type":"SelectorText"},{"id":"kilometer","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"div.bama-vehicle-detail-with-icon__detail-holder:nth-of-type(1) p","type":"SelectorText"},{"id":"gearbox","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"div.bama-vehicle-detail-with-icon__detail-holder:nth-of-type(3) p","type":"SelectorText"},{"id":"body","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"div.bama-vehicle-detail-with-icon__detail-holder:nth-of-type(4) p","type":"SelectorText"},{"id":"exteriorcolor","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"div.bama-vehicle-detail-with-icon__detail-holder:nth-of-type(5) p","type":"SelectorText"},{"id":"interiortrim","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"div:nth-of-type(6) p","type":"SelectorText"},{"id":"engine-disp","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"div.bama-vehicle-detail-with-link__row:nth-of-type(1) span.bama-vehicle-detail-with-link__row-text","type":"SelectorText"},{"id":"engine-type","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"div.bama-vehicle-detail-with-link__row:nth-of-type(2) span.bama-vehicle-detail-with-link__row-text","type":"SelectorText"},{"id":"description","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":".desc p","type":"SelectorText"},{"id":"image01","multiple":false,"parentSelectors":["santafe-link"],"selector":".is-visible img","type":"SelectorImage"},{"clickActionType":"real","clickElementSelector":".bama-call-to-seller__button span","clickElementUniquenessType":"uniqueText","clickType":"clickOnce","delay":2000,"discardInitialElements":"do-not-discard","id":"seller-phones","multiple":false,"parentSelectors":["santafe-link"],"selector":"button.bama-call-to-seller__button","type":"SelectorElementClick"},{"id":"phone1","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"a:nth-of-type(1) span.bama-call-to-seller__number-text","type":"SelectorText"},{"id":"phone2","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"a:nth-of-type(2) span.bama-call-to-seller__number-text","type":"SelectorText"},{"id":"phone3","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"a:nth-of-type(3) span.bama-call-to-seller__number-text","type":"SelectorText"},{"id":"phone4","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"a:nth-of-type(4) span.bama-call-to-seller__number-text","type":"SelectorText"},{"id":"phone5","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"a:nth-of-type(5) span.bama-call-to-seller__number-text","type":"SelectorText"},{"id":"phone6","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"a:nth-of-type(6) span.bama-call-to-seller__number-text","type":"SelectorText"},{"id":"phone7","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"a:nth-of-type(7) span.bama-call-to-seller__number-text","type":"SelectorText"}]}

#2

Your sitemap structure looks Ok. Just change your scroll selector to use :nth-of-type, e.g.
div.bama-ad-holder:nth-of-type(-n+50)
You can change the 50 to whatever number you need. Also change the Element limit to match:

image
image794×322 11.6 KB

#3

Thank you very much for answering my question so quickly and explaining it so clearly.
I copied and pasted your suggested code in the same way and made other settings as well. This time, by setting the number of elements equal to 25, the scraper output was 65 elements. It seems that the number of elements for scrolling down is not very accurate.

Also, I would be grateful if you could guide me on how to select the elements that I asked in the previous question. That is, does it affect the results or not?

Many thanks again

#4

Dear leemeng,
I appreciate your kindness indeed. Unfortunately, my trouble is still not solved. I admire your consultation in advance.

Best regards,

#5

Hi, you can try this sitemap for reference:

{"_id":"Santafeix45-main","startUrl":["https://bama.ir/car/hyundai-santafeix45"],"selectors":[{"delay":1000,"elementLimit":50,"id":"scroll","multiple":true,"parentSelectors":["_root"],"selector":"body:not(:has([class=\"bama-ad-holder\"]:nth-of-type(60))) a.bama-ad","type":"SelectorElementScroll"},{"id":"santafe-link","linkType":"linkFromHref","multiple":true,"parentSelectors":["_root"],"selector":"a.bama-ad","type":"SelectorLink"},{"id":"name","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"h1.bama-ad-detail-title__title","type":"SelectorText"},{"id":"man-year","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"span.bama-ad-detail-title__subtitle:nth-of-type(1)","type":"SelectorText"},{"id":"engine","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"span.bama-ad-detail-title__subtitle:nth-of-type(3)","type":"SelectorText"},{"id":"date","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"span.bama-ad-detail-title__ad-time","type":"SelectorText"},{"id":"city","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"span.address-text","type":"SelectorText"},{"id":"price","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"span.bama-ad-detail-price__price-text","type":"SelectorText"},{"id":"seller","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":".dealer-banner__title-holder span","type":"SelectorText"},{"id":"seller-adress","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"span.dealer-banner__sub-title","type":"SelectorText"},{"id":"kilometer","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"div.bama-vehicle-detail-with-icon__detail-holder:nth-of-type(1) p","type":"SelectorText"},{"id":"gearbox","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"div.bama-vehicle-detail-with-icon__detail-holder:nth-of-type(3) p","type":"SelectorText"},{"id":"body","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"div.bama-vehicle-detail-with-icon__detail-holder:nth-of-type(4) p","type":"SelectorText"},{"id":"exteriorcolor","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"div.bama-vehicle-detail-with-icon__detail-holder:nth-of-type(5) p","type":"SelectorText"},{"id":"interiortrim","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"div:nth-of-type(6) p","type":"SelectorText"},{"id":"engine-disp","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"div.bama-vehicle-detail-with-link__row:nth-of-type(1) span.bama-vehicle-detail-with-link__row-text","type":"SelectorText"},{"id":"engine-type","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"div.bama-vehicle-detail-with-link__row:nth-of-type(2) span.bama-vehicle-detail-with-link__row-text","type":"SelectorText"},{"id":"description","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":".desc p","type":"SelectorText"},{"id":"image01","multiple":false,"parentSelectors":["santafe-link"],"selector":".is-visible img","type":"SelectorImage"},{"clickActionType":"real","clickElementSelector":".bama-call-to-seller__button span","clickElementUniquenessType":"uniqueText","clickType":"clickOnce","delay":2000,"discardInitialElements":"do-not-discard","id":"seller-phones","multiple":false,"parentSelectors":["santafe-link"],"selector":"button.bama-call-to-seller__button","type":"SelectorElementClick"},{"id":"phone1","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"a:nth-of-type(1) span.bama-call-to-seller__number-text","type":"SelectorText"},{"id":"phone2","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"a:nth-of-type(2) span.bama-call-to-seller__number-text","type":"SelectorText"},{"id":"phone3","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"a:nth-of-type(3) span.bama-call-to-seller__number-text","type":"SelectorText"},{"id":"phone4","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"a:nth-of-type(4) span.bama-call-to-seller__number-text","type":"SelectorText"},{"id":"phone5","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"a:nth-of-type(5) span.bama-call-to-seller__number-text","type":"SelectorText"},{"id":"phone6","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"a:nth-of-type(6) span.bama-call-to-seller__number-text","type":"SelectorText"},{"id":"phone7","multiple":false,"parentSelectors":["santafe-link"],"regex":"","selector":"a:nth-of-type(7) span.bama-call-to-seller__number-text","type":"SelectorText"}]}