Ajax paginagion

How to?
#1

hi!
is there a way to scrape this page where the main url doesn't change?
thanks!

Url: https://www.bbt.com/locator/state.page?stateSelected=Texas,TX

Sitemap:
{"_id":"bbt","startUrl":["https://www.bbt.com/locator/state.page?stateSelected=Texas,TX"],"selectors":[{"id":"locations","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.row a.btn--locator-result","multiple":true,"delay":0,"clickElementSelector":"button#next_button.btn","clickType":"clickOnce","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"city","type":"SelectorText","parentSelectors":["locations"],"selector":"div.main-parent.main-parent--branch-locator h2","multiple":false,"regex":"","delay":0},{"id":"box","type":"SelectorElementClick","parentSelectors":["locations"],"selector":"div.container.main-parent div.content-block","multiple":true,"delay":0,"clickElementSelector":"button#next_button.btn","clickType":"clickOnce","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"branch","type":"SelectorText","parentSelectors":["box"],"selector":"a.smooth-scroll","multiple":false,"regex":"","delay":0},{"id":"address","type":"SelectorText","parentSelectors":["box"],"selector":"div.card__block p","multiple":false,"regex":"","delay":0},{"id":"phone","type":"SelectorText","parentSelectors":["box"],"selector":"div.card__block.card__footer a:nth-of-type(1)","multiple":false,"regex":"","delay":0}]}

#2

Here is the general idea. Although this is an element select within an element select which will only work on the firefox version (Currently) . However a few points

1 - Use Element Click Select, set to "click more" and multiple checked. The Click Selector = #next_button and Selector = div.container.main-parent div.column--4

2 - all the data points you want to scrape go within that elementCLickSelect (As a child) and you don't need to click multiple .

{"_id":"forum-fix-ajax-pag","startUrl":["https://www.bbt.com/locator/state.page?stateSelected=Texas,TX"],"selectors":[{"id":"locations","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.container.main-parent div.column--4","multiple":true,"delay":0,"clickElementSelector":"#next_button","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"city","type":"SelectorText","parentSelectors":["locations"],"selector":"span.result-desc","multiple":false,"regex":"","delay":0},{"id":"box","type":"SelectorLink","parentSelectors":["locations"],"selector":"a.btn--locator-result","multiple":false,"delay":0},{"id":"branch","type":"SelectorElementClick","parentSelectors":["box"],"selector":"div.container.main-parent div.content-block","multiple":true,"delay":0,"clickElementSelector":"#next_button","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueCSSSelector"},{"id":"Adress","type":"SelectorText","parentSelectors":["branch"],"selector":"div.card__block p","multiple":false,"regex":"^\\d+\\s[A-z]+\\s[A-z]+","delay":0},{"id":"City State","type":"SelectorText","parentSelectors":["branch"],"selector":"div.card__block p","multiple":false,"regex":"(?=\\n)[A-Z\\s]+,\\s\\w{2}","delay":0},{"id":"Phone","type":"SelectorText","parentSelectors":["branch"],"selector":"div.card__block.card__footer a:nth-of-type(1)","multiple":false,"regex":"","delay":0}]}```
#3

hey @bretfeig
thanks a lot! it is working like charm!
:rocket:
:smiley:

#4

So,thank you a lot! But how can we deal with those web who don't contain all the element in one certain url page? For example, I tried to scrape this web: http://mzj.beijing.gov.cn/wssbweb/wssb/dc/searchOrgList.do?action=searchOrgList, even though I set the "click more",it still could't work. Can you please help to deal with that? Thank you very much!
Here are my sitemap:
{"_id":"beijingcharity","startUrl":["http://mzj.beijing.gov.cn/wssbweb/wssb/dc/searchOrgList.do?action=searchOrgList"],"selectors":[{"id":"changepage","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"table:nth-of-type(3)","multiple":true,"delay":0,"clickElementSelector":"a:nth-of-type(3)","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"url","type":"SelectorLink","parentSelectors":["changepage"],"selector":"tr:nth-of-type(n+2) a","multiple":true,"delay":0},{"id":"name","type":"SelectorText","parentSelectors":["url"],"selector":"table:nth-of-type(8) tr:nth-of-type(1) td","multiple":true,"regex":"","delay":0}]}