HELP with Scraper pulling data into two separate line

[ngkaris]

Describe the problem. I am trying to have the scraper pull both the sellers in amazon listing and their prices, right now it is pulling both but they are going into separate lines. When it pulls into excel i would like to have "Seller Price 1" and "identity" in the same line. basically right now it is pulling them separately, which helps but it would be a lot easier to have them in the same line. Please help.

Url: https://www.amazon.com/s/ref=nb_sb_noss_2?url=search-alias%3Daps&field-keywords=Ram+Mounts+

Sitemap:
{"_id":"rammountwithprice","startUrl":["https://www.amazon.com/s/ref=nb_sb_noss_2?url=search-alias%3Daps&field-keywords=Ram+Mounts+"],"selectors":[{"id":"Link","type":"SelectorLink","parentSelectors":["_root","detail page"],"selector":"div.a-fixed-left-grid-col div.a-row:nth-of-type(1) div.a-row a.a-link-normal","multiple":true,"delay":0},{"id":"Title","type":"SelectorText","parentSelectors":["Link"],"selector":"h1.a-size-large span.a-size-large","multiple":false,"regex":"","delay":0},{"id":"Buy Box Price","type":"SelectorText","parentSelectors":["Link"],"selector":"div.a-section div.feature span.a-size-medium.a-color-price","multiple":false,"regex":"","delay":0},{"id":"Reviews","type":"SelectorText","parentSelectors":["Link"],"selector":"div.centerColAlign span.a-declarative span.a-size-base","multiple":false,"regex":"","delay":0},{"id":"ASIN","type":"SelectorText","parentSelectors":["Link"],"selector":"tr:contains('ASIN') td.a-size-base","multiple":false,"regex":"","delay":0},{"id":"Part Number","type":"SelectorText","parentSelectors":["Link"],"selector":"tr:contains('Item model number') td.a-size-base","multiple":false,"regex":"","delay":0},{"id":"detail page","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"li.s-result-item:nth-of-type(n+4) div.s-item-container","multiple":true,"delay":0,"clickElementSelector":"span#pagnNextString","clickType":"clickMore","discardInitialElements":true,"clickElementUniquenessType":"uniqueHTMLText"},{"id":"Seller PRICE","type":"SelectorLink","parentSelectors":["Link"],"selector":"div#olp_feature_div.feature a","multiple":false,"delay":0},{"id":"Seller Price 1","type":"SelectorText","parentSelectors":["Seller PRICE"],"selector":"span.a-size-large","multiple":true,"regex":"","delay":"1000"},{"id":"Identity","type":"SelectorText","parentSelectors":["Seller PRICE"],"selector":"span.a-size-medium a","multiple":true,"regex":"","delay":"2000"}]}

[bretfeig]

Hi There -

Firstly, Amazon is very tricky as they actively deploy anti-scraping measures that include changing elements and structures of pages (dynamic loading) and a host of other things.

That being said, I fixed your map to grab the first 250+ elements for you. You might be able to increase the delays and get more, I don't have time to fully look into it.

There were a plethora of errors in your map. It looks like you tried to use element select pagination with recursive, which has never worked for me.

This was my approach.

Step 1 - Use element click selector to define a: Each Box you want to scrape B: the click selector (next page arrow (not the larger rectangle that precedes the arrow) - set that to "click more", Unique CSS and tick the multiple box.

(that's both your pagination and what's going to align everything in columns)

Step 2- Create a child to that element click selector. This time use Link-selector and choose the name of the product. This will tell it to go into the product page.

NOTE if only the FIRST BOX is available to be selected, you've followed the first step correctly. If you can select any product, go back to step 1

Side Notes You'll see I added two additional text selectors that grab Name./Price from the search screen. This is unnecessary if you plan on grabbing that the product page.**

Step 3 - create a child to your new link selector (now you're in Root > Element Click Selector > Link Selector > New Elements "Product Page"
** You'll need to click on one of the product link to load the product page before continuing. I doesn't matter which one, you just need ANY product page **

Here you will use all your text selectors to

AZN, Price, Description etc.

{"_id":"rammountwithprice","startUrl":["https://www.amazon.com/s/ref=nb_sb_noss_2?url=search-alias%3Daps&field-keywords=Ram+Mounts+"],"selectors":[{"id":"detail page","type":"SelectorLink","parentSelectors":["Element Select"],"selector":"div.a-row.a-spacing-small a.a-link-normal","multiple":false,"delay":0},{"id":"Element Select","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.a-fixed-left-grid-inner","multiple":true,"delay":"4000","clickElementSelector":"a.pagnNext span.srSprite","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueCSSSelector"},{"id":"Product  Name","type":"SelectorText","parentSelectors":["Element Select"],"selector":"h2.a-size-medium","multiple":false,"regex":"","delay":0},{"id":"Price","type":"SelectorText","parentSelectors":["Element Select"],"selector":"span.sx-price","multiple":false,"regex":"","delay":0},{"id":"Customer Reviews","type":"SelectorText","parentSelectors":["detail page"],"selector":"div.centerColAlign span.a-declarative span.a-size-base","multiple":false,"regex":"","delay":0},{"id":"Job Description","type":"SelectorText","parentSelectors":["detail page"],"selector":"div.a-row p","multiple":false,"regex":"","delay":0},{"id":"ASIN","type":"SelectorText","parentSelectors":["detail page"],"selector":"tr:contains('ASIN') td.a-size-base","multiple":false,"regex":"","delay":0}]}

[ngkaris]

Bretfeig,

Thank you for the help it is almost exactly where i want it. basically the only thing missing is the offer page, (check the screen shots attached) i want it to click into this page and take all of the seller names and their prices. This will help me immediately see who is selling the product for what price. before i had it set up and it would pull them but instead of being in the same line they would pull into separate lines and in order one after the other.

attached is what is being pulled into a CSV and you can see how they are pulling prices and sellers but the price and seller are different lines. dont worry about the line that has buy box price, the "Seller Price" and "identity are what i need in the same line if possible

[bretfeig]

Looks doable. How do you envision it looking? [seller one] [Seller one price] [seller two] [seller two price].

Or would you like it all grouped together in 1 sell for each?

[ngkaris]

i would like in one row [link] [Title] [review count] [ASIN] [Manufacturer Part Number] [amount of sellers] [individual seller price] [individual seller name]

i am close to getting it but its just the individual seller price and seller name go into two different lines.

[bretfeig]

The second part is confusing me.

Since each product has 5-8 different sellers and matching prices, how do you see that being displayed on one line?

[bretfeig]

We could expand this to all sellers

[ngkaris]

That looks pretty good, and what i mean is each line has the seller and their price. so say you have 10 sellers in the same ASIN, each line would be a different seller and their price until it goes to the next Page/ asin. Then it starts displaying that ASIN, the seller and price side by side. thank you for the help by the way.

[bretfeig]

Ok - I think I understand what you mean. Let me give it one last try

{"_id":"rammountwithprice","startUrl":["https://www.amazon.com/s/ref=nb_sb_noss_2?url=search-alias%3Daps&field-keywords=Ram+Mounts+"],"selectors":[{"id":"detail page","type":"SelectorLink","parentSelectors":["Element Select"],"selector":"div.a-row.a-spacing-small a.a-link-normal","multiple":false,"delay":0},{"id":"Element Select","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"div.a-fixed-left-grid-inner","multiple":true,"delay":"3000","clickElementSelector":"a.pagnNext:contains(\"Next\")","clickType":"clickMore","discardInitialElements":false,"clickElementUniquenessType":"uniqueCSSSelector"},{"id":"Product","type":"SelectorText","parentSelectors":["detail page"],"selector":"h1.a-size-large span.a-size-large","multiple":false,"regex":"","delay":0},{"id":"Price","type":"SelectorText","parentSelectors":["detail page"],"selector":"td.a-span12 span.a-size-medium","multiple":false,"regex":"","delay":0},{"id":"Asin","type":"SelectorText","parentSelectors":["detail page"],"selector":"tr:contains('ASIN') td.a-size-base","multiple":false,"regex":"","delay":0},{"id":"model number","type":"SelectorText","parentSelectors":["detail page"],"selector":"tr:contains('Item model number') td.a-size-base","multiple":false,"regex":"","delay":0},{"id":"Sellers","type":"SelectorLink","parentSelectors":["detail page"],"selector":"div.centerColAlign div.feature div.a-section.a-spacing-top-small span a","multiple":false,"delay":0},{"id":"Seller-select","type":"SelectorElement","parentSelectors":["Sellers"],"selector":"div.a-section div.a-section div.a-row:nth-of-type(n+2)","multiple":true,"delay":0},{"id":"Seller-Alt-Name","type":"SelectorText","parentSelectors":["Seller-select"],"selector":"h3.a-spacing-none","multiple":false,"regex":"","delay":0},{"id":"Seller-Alt-Price","type":"SelectorText","parentSelectors":["Seller-select"],"selector":"span.a-size-large","multiple":false,"regex":"","delay":0},{"id":"Condition","type":"SelectorText","parentSelectors":["Seller-select"],"selector":"div.a-section span.a-size-medium","multiple":false,"regex":"","delay":0},{"id":"Seller Rating","type":"SelectorText","parentSelectors":["Seller-select"],"selector":"p.a-spacing-small a","multiple":false,"regex":"","delay":0},{"id":"seller-link-to-page","type":"SelectorElementAttribute","parentSelectors":["Seller-select"],"selector":"span.a-size-medium a","multiple":false,"extractAttribute":"href","delay":0}]}

Looks like this (Partial run)

Temp-Example

[ngkaris]

wow that is pretty much perfect, i would like the amount of reviews where the rating is but it doesnt need to be perfect. this is amazing thank you for the help.

[ngkaris]

The amount of reviews of the product if thats not too difficult.

[ngkaris]

after trying myself it is pulling pretty much all of the sponsored listings which i would normally skip, and then it is coming back saying null for seller name, seller price, condition, seller rating, and link to page