I would post a sitemap but I've had this exact problem on multiple very different site scrapes that share nothing in common. So Here is the structure that throws me.
I am scraping down several levels on a site
Origin Page > Multiple Sub Event Pages > Multiple Event Detail Pages > Final Page With Link & Image
I am able to successfully scrape a site down through Origin Page > Sub Event Pages > Event Detail Pages My problem occurs when I attempt to follow a link from the Event Details Page to the Final Page With Link & Image. The resulting scrape EITHER only scrapes the final page once for the first entry and none of the others or if I alter the sitemap parent structure the result is an alternating record pattern.
Any thoughts?