How Can I do This? Grab data from a page and insert into spreadsheet?

[Hawker]

Hello all. Whats a good way to grab some data from a web page and then have it inserted into a spreadsheet for me?

Specifically what I want to grab is some data from a Facebook group page.

Specifically what I need to grab is the group name, status & number of current users.

I have a list of around 1500 Facebook groups in a really big niche and currently doing it manually which is taking forever.

It might behove me to find something I can use to get that and do that for me.

Not just for this but in the future too and for updating my current lists.

What is something I can use to do that with?

Thanks!

This is what I need it to do in steps.

1. Visit the group URL and check if the group still exists.
2. If does not then skip and move to next group URL.
3. If group exists, grab the group name, group status, amount of users.
4. Insert that into a spreadsheet or save a report of some sort.

How can I do that?

Is iMacros good for doing something like that?

 

[wms]

Hello all. Whats a good way to grab some data from a web page and then have it inserted into a spreadsheet for me?

Specifically what I want to grab is some data from a Facebook group page.

Specifically what I need to grab is the group name, status & number of current users.

What purposes and why do you want to grab data from FB groups?

To add into an auto software and doing some jobs for you?

I have a list of around 1500 Facebook groups in a really big niche and currently doing it manually which is taking forever.

It might behove me to find something I can use to get that and do that for me.

Not just for this but in the future too and for updating my current lists.

What is something I can use to do that with?

Thanks!

Of course, if you do this manually, you're so tired with the job "copy and paste" :crazy:

Is iMacros good for doing something like that?

How iMacros could open an URL and grap data from different URLs?, its impossible for iMacros, iMacros should run on a web page and do tasks you prepared for it.

This is what I need it to do in steps.

1. Visit the group URL and check if the group still exists.
2. If does not then skip and move to next group URL.
3. If group exists, grab the group name, group status, amount of users.
4. Insert that into a spreadsheet or save a report of some sort.

How can I do that?

You can do that if you know PHP and using file_get_contents or cURL in PHP but I don't know Facebook have blocked file_get_contents function or still opening it.

This is what I need it to do in steps.

1. Visit the group URL and check if the group still exists.

Creating a small script to get all Facebook groups from your keyword or your conditions.

After got all groups, using a function to get content, detect tags by using PHP regex to filter content in Facebook groups so you can get group name, status & number of current users, urls..etc.

4. Insert that into a spreadsheet or save a report of some sort.

Of course you can use PHP to create and push data into an Exel file or insert data into database instead.

Ok I only show ways to do, not go into details so you can have directions to do OR you can use other ways.

Hope it helps!

 

[Hawker]

What purposes and why do you want to grab data from FB groups?

To add into an auto software and doing some jobs for you?

Yes something like that!

Of course, if you do this manually, you're so tired with the job "copy and paste" :crazy:

Tell me about it! It's a very tedious, repetitive task, not hard but just meh!

How iMacros could open an URL and grap data from different URLs?, its impossible for iMacros, iMacros should run on a web page and do tasks you prepared for it.

Hmm, well I asked a while ago on iMacros forums something similar and they said its possible and gave some examples saying you could use a URL_List.txt and have it check that. Although there was some stipulation about how it would actually open each new URL. Last member did provide what they think should work but I'm a bit of an iMacro noob to work it out LOL

[URLnofo]http://forum.imacros.net/viewtopic.php?f=2&t=25434&p=66614#p66614[/URLnofo]

You can do that if you know PHP and using file_get_contents or cURL in PHP but I don't know Facebook have blocked file_get_contents function or still opening it.

Creating a small script to get all Facebook groups from your keyword or your conditions.

After got all groups, using a function to get content, detect tags by using PHP regex to filter content in Facebook groups so you can get group name, status & number of current users, urls..etc.

Of course you can use PHP to create and push data into an Exel file or insert data into database instead.

I figured something could be done in PHP and would probably be pretty easy for a PHP expert. Unfortunately that is not me LOL.

Ok I only show ways to do, not go into details so you can have directions to do OR you can use other ways.

Hope it helps!

Yes you have been very wise Hoang! You have made me realize that I have some choices to me.

1. Learn iMacros and test test test until I crack it!
   This option could prove long and complicated and I could probably complete the list manually myself before I ever reach that level.
2. Learn PHP and create my own web script that can do it.
   Again this option could take years, tons of reading, many hours spent learning PHP which is handy but very long and hard and could do it manually myself before I ever reach the level of building a PHP script for it.
3. Pay someone to write me a PHP script (or something else) that can do it.
   This option would be quicker and could probably get someone to do it for $20 or so. Will probably hire someone on SEOClerks.
4. Just pay someone to do it for me.
   But that would mean they would have access to it so comes with some risk they might sell it on / abuse.
5. Stop being so lazy and just do it myself.
   Probably my best real solution.

I think my only real option is the last one.

It's just one of them things!

For me at least.

 

[Hawker]

252 down.

1248 to go...

:help:

 

[Joe B. Ray]

This type of web data extraction requires a custom code. It's not that difficult to write, we do it all the time, but it still cost some money. I don't know if you have a budget for this.

 

[PHP]

You need to create first an APP that gives you some secret keys or API credentials.

Then you can use any application that can supports API.

But i'm not sure what you are looking for to achive will allow fB.

They have some certain rules for API's and what you can extract and from where you want to extract and save it in a txt file, excel...etc...

 

[EpicGlobalWeb]

Yes you need a Facebook developer account and then make an app with it. They will give you API access and Then it's a matter of saving scraped contents to a file or a database, trimming the file for whitespace, etc and then serving up the documents.

It is not necessarily difficult to do this but you will end up using php to some capacity as well as the curl library.

There are a bunch of hacky ways to get around the file get co tents block.

 

[Hawker]

Thanks for the tips guys!

I finally managed to do it! Only took about 2 weeks! LOL

You can see what I wanted it for here.

Cheers!

 

[Joe B. Ray]

Joe B. Ray

You should have let me do the code for you for, then it would have taken you 5 minutes to do this.

 

[Hawker]

Hawker

Well how could I have done that Joe if I didn't know you was willing and able to do it for me until now?

How exactly would you do it? Like what would you use?

Drop me a PM about more details on how you would do it mate might still need it in the future cheers!

 

[Joe B. Ray]

Joe B. Ray

I actually sent you a PM when I first read your thread, on 03-21-2016, over a moth ago. I said in my PM that I can do this for you for $100. But, I never received a reply. Maybe somehow you missed my PM.

BTW: Since then, I posted a "Special Offer" for web data extraction here: https://www.webmastersun.com/threads/13144-Web-Scraping-Script
You can read some details about what I can do in that post. The price can also be lower depending on the project.

If you click on my signature, you can visit my website and find more info about data engineering and web scraping.

 

[WebmasterPhil]

Have you heard of WebHarvy? (short for Web Harvester, I believe). It's pretty cool and it cost about $100 I think. I believe there's a trial version as well.

I've used it before there's a small learning curve, but I was able to catalog data retrieved from the Yellow Pages and Classified Ads. Their tech support is excellent too. Go check out their tutorial videos on YouTube.