Issue with Load More button on caranddriver.com/reviews

How to?
#1

Hi, I started using Webscraper recently and have been loving it but there is a simple task I am totally stuck at. I was wondering if any of you can help me out.

Basically, I am having trouble with a load more button on the following URL (Car and Driver Reviews page). I'm using the Element-Click selector as the parent one but it just doesn't seem to load more than the first 20 elements. Been stuck at this for days now, would highly appreciate any help with this. The complete sitemap I am using is below for reference.

Url: https://www.caranddriver.com/reviews/

Sitemap:
{"_id":"car_and_driver_reviews","startUrl":["https://www.caranddriver.com/reviews/"],"selectors":[{"id":"car_review_links","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.full-item:nth-of-type(n+4)","multiple":true,"delay":"2000","clickElementSelector":"button.load-more","clickType":"clickMore","discardInitialElements":"discard-when-click-element-exists","clickElementUniquenessType":"uniqueText"},{"id":"click_link","type":"SelectorLink","parentSelectors":["car_review_links"],"selector":"a.full-item-title","multiple":false,"delay":0},{"id":"car_name","type":"SelectorText","parentSelectors":["click_link"],"selector":"p:nth-of-type(2) strong","multiple":false,"regex":"","delay":0},{"id":"vehicle_type","type":"SelectorText","parentSelectors":["click_link"],"selector":".specs-content p:nth-of-type(3)","multiple":false,"regex":"","delay":0},{"id":"price","type":"SelectorText","parentSelectors":["click_link"],"selector":".specs-content p:nth-of-type(4)","multiple":false,"regex":"","delay":0},{"id":"engine_type","type":"SelectorText","parentSelectors":["click_link"],"selector":".specs-content p:nth-of-type(5)","multiple":false,"regex":"","delay":0},{"id":"transmission","type":"SelectorText","parentSelectors":["click_link"],"selector":".specs-content p:nth-of-type(6)","multiple":false,"regex":"","delay":0},{"id":"chassis","type":"SelectorText","parentSelectors":["click_link"],"selector":".specs-content p:nth-of-type(7)","multiple":false,"regex":"","delay":0},{"id":"dimensions","type":"SelectorText","parentSelectors":["click_link"],"selector":".specs-content p:nth-of-type(8)","multiple":false,"regex":"","delay":0},{"id":"cd_testresults","type":"SelectorText","parentSelectors":["click_link"],"selector":".specs-content p:nth-of-type(9)","multiple":false,"regex":"","delay":0},{"id":"cd_fuel_economy","type":"SelectorText","parentSelectors":["click_link"],"selector":".specs-content p:nth-of-type(10)","multiple":false,"regex":"","delay":0},{"id":"epa_fueleconomy","type":"SelectorText","parentSelectors":["click_link"],"selector":".specs-content p:nth-of-type(11)","multiple":false,"regex":"","delay":0}]}

#2

Not as simple as it seems. This site is tricky 'cos you need to keep scrolling down before the Load More appears.There's a kludge-y way of doing this which involves using separate scrollers and separate clicks for Load More (not the typical method for using these). You can try the sitemap below which will click Load More 3 times, yielding 80 results. Each click reveals 20 more results.

To get more results you'll need to edit the sitemap and add more scroller + Load More combos before the row wrappers. If you load the sitemap in JSONLint, it will be much easier to edit. You'll notice there are currently 3 repeating blocks of Separate Scrollerx and Click Load Morex. You can add more blocks below them, as needed. You'll also need to edit the nth selector for Separate Scrollerx, e.g. div.full-item:nth-of-type(n+41)

Sitemap:
{"_id":"forum-caranddriver3","startUrl":["https://www.caranddriver.com/reviews/"],"selectors":[{"id":"Separate Scroller1","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"div.full-item:nth-of-type(n+2)","multiple":true,"delay":"1500"},{"id":"Click Load More1","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"span.icon-chevrondown","multiple":true,"delay":"3500","clickElementSelector":"span.icon-chevrondown","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueHTML"},{"id":"Separate Scroller2","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"div.full-item:nth-of-type(n+21)","multiple":true,"delay":"1500"},{"id":"Click Load More2","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"span.icon-chevrondown","multiple":true,"delay":"3500","clickElementSelector":"span.icon-chevrondown","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueHTML"},{"id":"Separate Scroller3","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"div.full-item:nth-of-type(n+41)","multiple":true,"delay":"1500"},{"id":"Click Load More3","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"span.icon-chevrondown","multiple":true,"delay":"3500","clickElementSelector":"span.icon-chevrondown","clickType":"clickMore","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueHTML"},{"id":"Row wrappers","type":"SelectorElement","parentSelectors":["_root"],"selector":"div.full-item:nth-of-type(n+1)","multiple":true,"delay":0},{"id":"Headline","type":"SelectorLink","parentSelectors":["Row wrappers"],"selector":"a[class^='full-item-title'] ","multiple":false,"delay":0},{"id":"Date","type":"SelectorText","parentSelectors":["Row wrappers"],"selector":"div.publish-date","multiple":false,"regex":"","delay":0}]}