Unable to scrape after update!

#24

again, i will note that this approach has been functional in web scraper for over six months. as i understand it, the selector input field only provides a context in which the downstream selectors operate. the downstream elements have always previously operated correctly within the 'all' context.

i will try to implement another sitemap design that addresses the limitations that have been introduced to the web scraper application as of the most recent release. if i can't figure something out, i will need to find another scraper solution that supports our use case

#25

I've simplified this further, to highlight exactly where I'm experiencing the issue. In the selector graph below, if I delete the CONTACTS_LINK node and its children (OFFICE, REGION), Web Scraper successfully returns scrape results for the RFP_ID node and its children.

However, whenever I attempt to add back the CONTACTS_LINK and children to the sitemap, I get a null result set when I scrape. No data is returned for any field in the graph.

For the "Selector" input field for CONTACTS_LINK, I've tried to explicitly define it (tr:nth-of-type(2)), which which was the approach that worked prior to the current Web Scraper release. I've also tried to scope it using "parent" and "*". None of the approaches I've tried have been successful...all result in a null result set.

Capture
Capture.JPG878×813 49.6 KB

#26

I'm sorry the forum did mess up the parent notation. Use _parent_ instead of parent.

#28

i had realized that from looking at the documentation. i have already used _parent_ as you indicate

#29

I'm having similar problems with one my sitemaps, using Chrome 74.0.3729.157 and Web Scraper 0.4.1.
I set up this sitemap to scrape a set of elements off a page. After that, it visits the page each element links to and scrapes info. From that psprices.com page it follow the link to playstation.com and scrapes info.

If the psprices.com page has a link to a base game for the item, it visits the base game's page and does the same as above.

I tested it on another profile with Chrome 74.0.3729.157 and Web Scraper 0.3.8 and it works as intended. With Web Scraper 0.4.1 I'm getting less info -- it's not going to the non-base-games' playstation.com page.

Here's the sitemap (with a bunch of selectors removed to hopefully make it simpler to troubleshoot):

{"_id":"z_psprices_test","startUrl":["https://psprices.com/region-us/discounts/?platform=PS4&page=1"],"selectors":[{"id":"itemLink","type":"SelectorLink","parentSelectors":["itemElement"],"selector":"a.content__game_card__cover","multiple":false,"delay":0},{"id":"psnLink","type":"SelectorLink","parentSelectors":["itemLink"],"selector":"div.col-md-9 a.content__game__btn","multiple":false,"delay":0},{"id":"releaseDate","type":"SelectorText","parentSelectors":["itemLink"],"selector":"div#price_history > div:nth-of-type(2)","multiple":false,"regex":"","delay":0},{"id":"baseGame","type":"SelectorLink","parentSelectors":["itemLink"],"selector":"div.content__game__title__parent a","multiple":false,"delay":0},{"id":"baseGameLink","type":"SelectorLink","parentSelectors":["baseGame"],"selector":"div.col-md-9 a.content__game__btn","multiple":false,"delay":0},{"id":"baseGameReleaseDate","type":"SelectorText","parentSelectors":["baseGame"],"selector":"div#price_history > div:nth-of-type(2)","multiple":false,"regex":"","delay":0},{"id":"psnRelease","type":"SelectorText","parentSelectors":["psnLink","baseGameLink"],"selector":"span.provider-info__list-item:nth-of-type(2)","multiple":false,"regex":"","delay":0},{"id":"itemElement","type":"SelectorElement","parentSelectors":["_root"],"selector":"div.col-md-2:nth-of-type(n+7)","multiple":true,"delay":0}]}

#30

It seems that nested links are broken in general. I have the same problem and performance as described above in a different project: element or text element selection at the level of the nested link fails completely.