If page is loaded by a selector link, text selectors in page do not recognize parent

Bug Reports
#1

emphasized text

image
image783×260 7.31 KB

To access pages with data, I need to use a selector "element click" (MO_page_Free). However in the accessed pages, I am not able to setup the "text selectors" because they do not recognise the parent selector "element click".
I tested and, if I use a simple link in MO_Page_Free, the Text selectors would recognise their parent and would retrieve the data. However, being a "onclick js event" that triggers the loading of the page, setting MO_page_Free as link will not navigate to pages with data.

#2

Code:

{

"_id": "proper_on_the_works",

"startUrl": [

"https://www.maitredoeuvre.com/annuaire/villes-morbihan.php"

],

"selectors": [

{

  "id": "villeLink",

  "type": "SelectorLink",

  "parentSelectors": [

    "_root"

  ],

  "selector": "li:nth-of-type(n+2) a",

  "multiple": true,

  "delay": 0

},

{

  "id": "MO_page_Free",

  "type": "SelectorElementClick",

  "parentSelectors": [

    "villeLink"

  ],

  "selector": ".freeBG",

  "multiple": true,

  "delay": 2000,

  "clickElementSelector": ".freeBG img",

  "clickType": "clickOnce",

  "discardInitialElements": "do-not-discard",

  "clickElementUniquenessType": "uniqueText"

},

{

  "id": "name_MO",

  "type": "SelectorText",

  "parentSelectors": [

    "MO_page_Free"

  ],

  "selector": "h1",

  "multiple": false,

  "regex": "",

  "delay": 0

},

{

  "id": "AgenceLocal_MO",

  "type": "SelectorText",

  "parentSelectors": [

    "MO_page_Free"

  ],

  "selector": ".agence_infos h3",

  "multiple": true,

  "regex": "",

  "delay": 0

},

{

  "id": "AgenceTel_MO",

  "type": "SelectorElementAttribute",

  "parentSelectors": [

    "MO_page_Free"

  ],

  "selector": "#agences-proches .agence_action a",

  "multiple": true,

  "extractAttribute": "data-tel",

  "delay": 0

},

{

  "id": "AgenceDir_MO",

  "type": "SelectorText",

  "parentSelectors": [

    "MO_page_Free"

  ],

  "selector": ".agence_infos p",

  "multiple": true,

  "regex": "",

  "delay": 0

},

{

  "id": "AgencePostalCode_MO",

  "type": "SelectorText",

  "parentSelectors": [

    "MO_page_Free"

  ],

  "selector": ".agence_infos p strong",

  "multiple": true,

  "regex": "",

  "delay": 0

},

{

  "id": "pages",

  "type": "SelectorElementClick",

  "parentSelectors": [

    "villeLink"

  ],

  "selector": "div.villes",

  "multiple": true,

  "delay": 2000,

  "clickElementSelector": ".villes a",

  "clickType": "clickOnce",

  "discardInitialElements": "do-not-discard",

  "clickElementUniquenessType": "uniqueText"

}

]

}

#3

@labregowski Hello, unfortunately, you can't access the desired data by using an element click selector, because the page is reloaded and a new link is generated - instead, you should use the link selector.

Test results using Web Scraper Cloud:

maitredoeuvre.com
maitredoeuvre.com2322×1184 289 KB

For the pagination - div#pagination span + a
For the agency links - div#agences a

More information:

https://webscraper.io/documentation/selectors/link-selector
https://webscraper.io/how-to-video/link-button-pagination

Hope it helps!