The og:image of the website has been changed recently. The site contains more than 100 pages, each containing their individual og:image. How can i ask or force facebook to re-scrape all the pages, so that the image gets updated? Using the facebook debugger tool will be too tedious task. Until facebook re-scrape's the site, i won't be able to submit collection for the app.
6 Answers
You can force a re-scrape via API, as described here: https://developers.facebook.com/docs/opengraph/using-objects/#update:
POST /?id={object-instance-id or object-url}&scrape=true
(But if you don’t have a real list of the affected URLs, this is kinda moot. Then you only can wait until it happens automatically, I guess.)
-
This is great. The response includes the refreshed OG data as well, so you can check it yourself while scraping. Apr 18, 2014 at 21:04
-
This is the true answer and should be the chosen one. shame the other one is, since it's not even an answer at all.– vsyncJul 2, 2014 at 10:01
-
2This is definitely the correct answer. POSTing /?id=<url>&scrape=true to graph.facebook.com will force a re-scrape of the og tags for hte url. One must remember that this must be a POST request, not a GET request, otherwise you just get the number of shares. And if you add another parameter called callback=<somecallbackfunction>, you can do this via an AJAX JSONP request within the browser, and no need to tie up your server. Aug 5, 2014 at 21:29
-
Hi , can this method be used to update dynamically created meta tags using javascript? ref : stackoverflow.com/questions/40868686/…– HaswinAug 3, 2017 at 9:21
-
@HaswinVidanage the answer to that question over there ... well, already answers that.– CBroeAug 3, 2017 at 9:23
You have two alternatives here in your situation
As per ysrb's answer loop through list of your URLs with the Open Graph Debugger tool
Or wait patiently for 30 days until Facebook re-scrapes your Pages as it says in documentation here
Why and when does Facebook scrape my website?
Facebook needs to scrape links shared to Facebook to know which to link preview information to show on Facebook.com or on Facebook for iOS and Android. This happens every 30 days to ensure the properties are up to date. The linked page is also scraped when the URL is entered into the Debugger Tool.
Facebook observes cache headers on your URLs - it will look at "Expires" and "Cache-Control" in order of preference. However, even if you specify a longer time, Facebook will scrape your page every 30 days.
-
3@ChrisHoughton - developers.facebook.com/docs/plugins/faqs/#scraperinfo (as of 2015.04.08) Apr 8, 2015 at 19:15
-
This is no longer correct. There is an API endpoing that Doug and others explain in another answer. Feb 25, 2016 at 20:15
-
my pages were scraped last time in 2012, and now im doing all of them manually :(– AzizJan 31, 2017 at 18:14
$config = array(
"appId" => 'APP_ID',
"secret" => 'APP_SECRET');
$fb = new Facebook($config);
$r=$fb->api('/','POST',array(
'id'=>PAGE_URL,
'scrape'=>'true'
));
Be aware that Facebook requires the parameters be passed via POST
. Facebook just ignores GET
requests.
Here is the code in C#.
HttpWebRequest httpRequest = (HttpWebRequest)WebRequest.Create(
"https://graph.facebook.com/?id="
+ HttpUtility.UrlEncode("http://www.example.com/index.html")
+ "&scrape=true");
httpRequest.Method = "POST";
using (HttpWebResponse httpResponse = (HttpWebResponse)httpRequest.GetResponse())
{
using (Stream responsestream = httpResponse.GetResponseStream())
{
if (responsestream != null)
{
using (StreamReader bodyreader = new StreamReader(responsestream))
{
string fbResp = bodyreader.ReadToEnd();
}
}
}
}
curl --insecure "https://graph.facebook.com/?id=[YOUR-URL-TO-SCRAPE]&scrape=true"
curl should be ssl capable (as it is https://graph...)
-
I got this to work, thanks a lot. - You need to be sure to set the user agent, or else FB will decide not to honor the scrape request, however. Mar 26, 2016 at 6:31
You can try loop from the list of url and do
curl "http://developers.facebook.com/tools/debug/og/object?q=$url"
-
Ok. The exact list of url is unknown. Approx there are 310 urls as of now. Manually specifying each of them for the loop will be time consuming and we may miss some url due to humman error May 3, 2013 at 5:44
-
You must have some kind of sitemap.xml or something. Even if it's static files, you can list it and create the url– ysrbMay 3, 2013 at 7:02
-
3
-
-
3@ysrb I mean I tried it, and it doesn't work, probably because you can't access that URL without being logged into facebook. The data doesn't get scrapped by facebook with this method.– GTCraisSep 15, 2014 at 10:11