Scraping FB Group Member info

How to?
#1

Trying to build a scraper for FB member info,

I can pull the FB url and name of the person but I'm having difficulty mapping the "About" and "Contact Info" sections....any advice?

{"_id":"fb_member_scraper","startUrl":["https://www.facebook.com/groups/SourcingAlchemy/members/"],"selectors":[{"id":"profile","type":"SelectorLink","parentSelectors":["_root"],"selector":"div#groupsMemberSection_recently_joined._21m- div.fbProfileBrowserList.fbProfileBrowserListContainer > ul.uiList div._60ri a, div.fbProfileBrowserList div.fbProfileBrowserList div._60ri a","multiple":true,"delay":0},{"id":"name","type":"SelectorText","parentSelectors":["profile"],"selector":"div._3ioa","multiple":false,"regex":"","delay":0},{"id":"link","type":"SelectorLink","parentSelectors":["profile"],"selector":"a._2nlw","multiple":false,"delay":0},{"id":"About","type":"SelectorLink","parentSelectors":["profile"],"selector":"a._6-6._6-7","multiple":false,"delay":0},{"id":"overview","type":"SelectorLink","parentSelectors":["About"],"selector":"a.5pwr.47","multiple":false,"delay":0},{"id":"Current Role","type":"SelectorText","parentSelectors":["overview"],"selector":"li#u_v_5 div._c24","multiple":false,"regex":"","delay":0},{"id":"Lives in","type":"SelectorText","parentSelectors":["overview"],"selector":"li#u_v_1 div._c24 a.profileLink","multiple":false,"regex":"","delay":0}]}

#2

Hmm.. I've gotten part of the way. I got element scroll down working and mapped element click to the about tab. Form there I can get another element click to cycle through the sub catagories within the about tab. @iconoclast - your thoughts?

{"_id":"fb_member_scraper","startUrl":["https://www.facebook.com/groups/SourcingAlchemy/members/"],"selectors":[{"id":"Scroll Down","type":"SelectorElementScroll","parentSelectors":["_root"],"selector":"._gse","multiple":true,"delay":0},{"id":"Profile Link","type":"SelectorLink","parentSelectors":["Scroll Down"],"selector":"._60ri a","multiple":false,"delay":0},{"id":"About Tab ","type":"SelectorElementClick","parentSelectors":["Profile Link"],"selector":"div#pagelet_timeline_medley_about._5h60","multiple":false,"delay":0,"clickElementSelector":"li:nth-of-type(2) a._6-6","clickType":"clickOnce","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"Work Education","type":"SelectorElementClick","parentSelectors":["About Tab "],"selector":"#u_0_2e","multiple":false,"delay":"2000","clickElementSelector":"._5pws:eq(1)","clickType":"clickOnce","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"Current Company","type":"SelectorText","parentSelectors":["Work Education"],"selector":"li#u_0_2j._43c8 div._2lzr","multiple":false,"regex":"","delay":0},{"id":"Title","type":"SelectorText","parentSelectors":["Work Education"],"selector":"li#u_0_2j._43c8 div._173e","multiple":false,"regex":"","delay":0},{"id":"Places lived","type":"SelectorElementClick","parentSelectors":["About Tab "],"selector":"div._4ms4","multiple":false,"delay":"2000","clickElementSelector":"a._5pwr._Interaction__ProfileSectionPlaces span._5pws","clickType":"clickOnce","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"Current City","type":"SelectorText","parentSelectors":["Places lived"],"selector":"span._2iel","multiple":false,"regex":"","delay":0},{"id":"Contact","type":"SelectorElementClick","parentSelectors":["About Tab "],"selector":"#u_0_2e","multiple":false,"delay":"2000","clickElementSelector":"a._5pwr._Interaction__ProfileSectionContactBasic span._5pws","clickType":"clickOnce","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"Phone","type":"SelectorText","parentSelectors":["Contact"],"selector":"#u_0_2h","multiple":false,"regex":"","delay":0},{"id":"Facebook","type":"SelectorText","parentSelectors":["Contact"],"selector":"._39g6","multiple":false,"regex":"","delay":0},{"id":"Social Links","type":"SelectorText","parentSelectors":["Contact"],"selector":"._509-","multiple":false,"regex":"","delay":0}]}

#3

That's pretty slick Bret, I definitely need to mess with the scroll and element click more. Let me me know if you find a solution on how to map the work and education information, my goal is to extract multiple groups one sitemap....but it's hard to find time this week to really dive in hard to this stuff

#4

I think I'm little late -- facebook page says Sorry, this content isn't available right now

#5

How about this - How do I scroll through each subheader and scrape the elements

The first Click Selector chooses the "About Tab"

Each Box on the left, changes the elements and fields on the right. There doesn't seem to be any consistancy but I wish to scrape each box on the same line, as an additional field. How do I do hhis

image

{"_id":"fb_member_scraper","startUrl":["https://www.facebook.com/groups/SourcingAlchemy/members/"],"selectors":[{"id":"Click Selector","type":"SelectorElementClick","parentSelectors":["_root"],"selector":"ul.uiList div._5aj7","multiple":false,"delay":0,"clickElementSelector":"li:nth-of-type(2) a._6-6","clickType":"clickOnce","discardInitialElements":false,"clickElementUniquenessType":"uniqueText"},{"id":"Element-Click-Left-Hand","type":"SelectorLink","parentSelectors":["Click Selector"],"selector":"li.47-:nth-of-type(n+2) a._5pwr","multiple":true,"delay":0},{"id":"Work-Education-Company","type":"SelectorText","parentSelectors":["Element-Click-Left-Hand"],"selector":"li#u_2l_6._43c8 div._2lzr a","multiple":false,"regex":"","delay":0},{"id":"Work-Title","type":"SelectorText","parentSelectors":["Element-Click-Left-Hand"],"selector":"li#u_2l_6._43c8 div.fsm","multiple":false,"regex":"","delay":0},{"id":"","type":"SelectorText","parentSelectors":["Element-Click-Left-Hand"],"selector":"li#current_city._3pw9 span._2iel a","multiple":false,"regex":"","delay":0}]}

#6

I've had a similar issue with other scraping tools, but at long as you can pull some info (profile links and URLs for example), if may be worth cleaning the csv data.

Is this parsing the data? I'm not on my desktop lol

#7

It's pretty hard to figure things out without access to the original source :slight_smile:
I guess you have to add buttons to the left into Element Click selector (the Click selector), and what appears to the right have to be assigned into actual selector. I'd assign whole 'About' section into selector, so no data will be missed.

#8

Hi,

did you find the answer? I am also looking to do that.

#9

Do you know how it can be done