Scrape Output Limited to 24 Entries

How to?
#1

Sitemap:

{"_id":"matrix-nwmls-mo","startUrl":["https://www.matrix.nwmls.com/Matrix/Public/Portal.aspx?L=1&k=3916969XP9ZD&p=AE-1459101-152#1"],"selectors":[{"id":"List","type":"SelectorElementClick","parentSelectors":["List View"],"selector":"div.row.d-marginBottom--4","multiple":true,"delay":"1000","clickElementSelector":"div.mtx-pageMore.j-resultsPageNext","clickType":"clickMore","discardInitialElements":true,"clickElementUniquenessType":"uniqueCSSSelector"},{"id":"List View","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.col-xxs-12","multiple":true,"delay":"","clickElementSelector":"a.mtx-btn-link.mtx-icon-viewList","clickType":"clickOnce","discardInitialElements":true,"clickElementUniquenessType":"uniqueText"},{"id":"Address","type":"SelectorText","parentSelectors":["List"],"selector":"div.col-sm-12.d-text","multiple":false,"regex":"","delay":"1000"},{"id":"City, Sate, ZIP","type":"SelectorText","parentSelectors":["List"],"selector":"div.col-sm-12.d-textSoft","multiple":false,"regex":"","delay":"1000"}]}

I can't seem to figure out why my scrapes only return 24 entries. The .csv file size is less than 8KB. It only shows 24 entries in the queue also. I have js "view more results" button which I can see is being 'clicked' to display all the entries but it doesn't seem to scrape them all. There should be 102 entries.
Thanks
BGMS

#2

Here is the Fix

in short:

Element Click Selector (for load more situation) does two things. It defines each row (selector) of data you want and it identified the button to click. You need to set it up (Click more elements) and not pagination. I set it to "Unique CSS" but i'm not entirely sure why I chose that or if the others would have worked. @iconoclast can perhaps add an explanation there.

Both of our selectors were wrong also.. I fixed that.

Note: I have in List mode and not gallery mode which would change all the selectors.

Once you have that set, you make all other selectors as a child of that. You'll notice only the first property is able to be selected which is how you know it's set up correctly. Whatever you choose will be replicated across everything Else.

As it's set right now you're getting price and location but just add more selectors to get additional info.

{"_id":"forum-broke-click-more","startUrl":["https://www.matrix.nwmls.com/Matrix/Public/Portal.aspx?L=1&k=3916969XP9ZD&p=AE-1459101-152#1"],"selectors":[{"id":"List View","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.row.d-paddingTop--6","multiple":true,"delay":"3000","clickElementSelector":"div.mtx-pageMore.j-resultsPageNext a.btn","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueCSSSelector"},{"id":"Address","type":"SelectorText","parentSelectors":["List View"],"selector":"div.col-sm-12.d-text","multiple":false,"regex":"","delay":""},{"id":"City, Sate, ZIP","type":"SelectorText","parentSelectors":["List View"],"selector":"div.col-sm-12.d-textSoft","multiple":false,"regex":"","delay":""}]}