I am trying to scrape an eCommerce website but i can't manage to get the categories from the breadcrumbs at all.
<div class="breadcrumb"><a href="https://www.emanualonline.com/" class="">Home</a><span class="raquo">»</span><a href="https://www.emanualonline.com/Education.htm" class="">Education</a><span class="raquo">»</span><a href="https://www.emanualonline.com/Education/Books.htm" class="">Books</a><span class="raquo">»</span><a href="https://www.emanualonline.com/Education/Books/Language-and-Grammar.htm" class="">Language and Grammar</a><span class="raquo">»</span>!Te Toca!: A New Communicative Spanish Course - , Richard Pym</div>
Sitemap:
{"_id":"all-products-pagination","startUrl":["https://www.emanualonline.com/allprods.php?page=1"],"selectors":[{"id":"product-link","type":"SelectorLink","parentSelectors":["_root"],"selector":"[data-align='left'] a","multiple":true,"delay":0},{"id":"Product Name","type":"SelectorText","parentSelectors":["product-link"],"selector":"h1","multiple":false,"regex":"","delay":0},{"id":"Product Price","type":"SelectorText","parentSelectors":["product-link"],"selector":"span.productPriceCurrent","multiple":false,"regex":"[0-9]+\.[0-9]+","delay":0},{"id":"Product Description","type":"SelectorHTML","parentSelectors":["product-link"],"selector":".tabs_content td","multiple":false,"regex":"","delay":0},{"id":"Product Image","type":"SelectorImage","parentSelectors":["product-link"],"selector":".popupGall img","multiple":false,"delay":0},{"id":"Product Category","type":"SelectorHTML","parentSelectors":["product-link"],"selector":".breadcrumb a","multiple":true,"regex":"","delay":0}]}
Another thing i am having troubles with is for examle if you click on the category on the left on Cars then for example Audi and for some models there is also pagination, while for some there is not. How can i account for that so that if there is pagination make sure it checks that too