Please help my sanity!

How to?
#1

I have been trying to scrape the details from prime distribution for the past week and I am getting nowhere fast. I have scraped other sites in the past however this particular one is a doozy. I want to paginate the record labels, A-Z. Select the label names as links, and click the link to scrape the EP information (name and track details). Please help as my already thinning hair cant take anymore.

Url: http://www.primedirectdist.co.uk/labels.htm

Sitemap:
{"_id":"primedistlabels","startUrl":["http://www.primedirectdist.co.uk/labels.htm"],"selectors":[]}

Sorry for the lack of sitemap. I thought would be best for someone who can get around this to start from scratch.

#2

Site is loading extremely slowly on my end, esp the label pages. WS probably won't work if the pages can't finish loading.

Anyway, you don't really need a paginator for this page because all the labels are already shown up front (same as clicking ALL). So you can just use Element click to get at the label pages.

#3

Hi Lee,

Thanks for the reply. I looked at what you said and did an element click. I have slightly changed the sitemap to go to artists. The scrape creates a list of the artists, GREAT! I want the scraper to go one level deeper and scrape the information of the artists tracks (this would be clicking the artists name, which then reveals the ep's and tracks). I have created a link as a parent of the artist and then for the scrape to collect the data of the label. Nothing is being scraped, see the below for the exported site map. Can you help with this?

I hope this makes sense.

Thanks

{"_id":"primeartist","startUrl":["http://www.primedirectdist.co.uk/artists.htm"],"selectors":[{"id":"click_pagination","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"#artistsMenuResults div","multiple":true,"delay":"500","clickElementSelector":".aToZ div.mlabel:nth-of-type(n+2)","clickType":"clickOnce","discardInitialElements":"do-not-discard","clickElementUniquenessType":"uniqueText"},{"id":"links","type":"SelectorLink","parentSelectors":["click_pagination"],"selector":"parent","multiple":true,"delay":0},{"id":"label_name","type":"SelectorText","parentSelectors":["links"],"selector":"a.label","multiple":false,"regex":"","delay":0}]}