Help create first selector for reading an array result

How to?
#1

Not sure yet if this tool can help me..

I have 300 URLs I have to scan to get ID's from some javascript created array results, so I can move on to next part of crawl... The id's are obtained from some inner workings, and not available to be crawled to be obtained... but if they are provided I can then compile data gained here to get more javascript results gets the final call to get rest of the data i need..

So I provide a initial set of 300 URLs and create a sitemap for it
http://domain.com/api/search/product=2999998246
http://domain.com/api/search/product=4919288383

These URLs get back a page with an array, in which case I can't figure out how to get a selector on it..
I've tried text, and element, just nothing seems to be able to understand I need this text result array selected..

If i can get a selector on this page result, from there i can regex what i need {"Id":(.*?), but nothing seems to get me past the selector i try to make to parse this result, I need to pull the id out to get this array text selected so it can be parsed..

Here is the result of my sitemap provided url
http://domain.com/api/search/product=2999998246

Here is the result, i need to get a selector on this and capture it and then use regex {"Id":(.*?),"
to get the id, but again, can't get this data to be selected

Any help would be much appreciated!!

[{"Item":{"Id":2999998246,"PlotId":0,"MfrID":"Cat","GTIN":"923932799583","Description":"Vacuum Hose","packQuantity":0,"MixedLot":null,"Availabile":0,"Weight":0.0,"Height":0.0,"Width":0.0,"Depth":0.0},"Quantity":0}]

This array is called by the site using javascript, i managed to put together the url to get the info needed, there is no html or anything on the result.. the array above is the whole page returned by the sitemap url

#2

one more note is that.. like when I go to create the selector... its not changing the page and pulling the data for me to select anything.. i can select data from the web page already loaded.. but if I load this array into chrome, then create sitemap, then try to select, nothing happens either... i not getting any green selection... however the chrome web developer too easily see's this array as an element.. so i think maybe crawler bugged and cant handle a simple array result?!?!?

#3

Make a text or HTML selector like this: _parent_ .

#4

Thank you for response, I don't see how this is possible to make such a selector?!?
It never gets any data, so the selector parent is made, but it has nothing to parse...

i have sitemap then
a selector productid parent_ SelectorText no _root

the _ is there site is removing

it does nothing

well, it actually works when i select get data, but the problem is the crawler doesnt start and do anything...

#5

_parent_ selects everything from _root so it should return json even if there is no HTML.

#6

Hello, Actually got it working today... sometimes a fresh reboot helps :smiley:

One thing though, I don't see no documentation on..

my regex is
{"Id":(.*?),"C

in which case, the data result is {"Id":55555,"C is there any way i can clean that up? obviously my result is $1 not sure even if i made it more specific to digits how to get rid the rest of the regex

#7

Currently web scraper doesn't support match groups for regex.