0

So I had asked a question a couple of months ago with regards to this issue I was facing with Facebook unable to fetch the required Open Graph tags from a couple of pages on my website presumably because of the heavy content on the page.

URL where I'm facing this issue:

https://dietbros.com/fat-burning/the-ultimate-fat-burning-foods-list/ Although the page is very lengthy (~50,000 words) and has around 130 images, it's not more than 10mb because the images have been highly optimized.

Unfortunately, I haven't received any responses to the question and I'm wondering if there are any alternate solutions to the issue. Facebook's docs state that: "Optimizing Metadata

You can optimize content by delivering only Open Graph meta tags to the crawler and only the content itself to regular users. Alternatively, you can choose to point the crawler to a separate page used only for metadata with ."

They also state: "The URL where your content is hosted should contain the required Open Graph tags."

1) How can I achieve this when Facebook times out before picking up the tags at all or does it get redirected before the time out? And if this is possible, how do I go about doing this (duplicate the post minus the content?)?

2) Are there any alternative solutions to this issue where I can somehow get the Facebook crawler to pick up the OG tags?

I'd be super grateful if I could get any help with this as social media traffic is arguably the most important source of traffic for us considering the kind of content we cover and the niche we're in. Would love any sort of help that would point us in the right direction.

5
  • Pointing the crawler to URL B from A likely won’t help much, if it has trouble reading A because of the amount of data it outputs or the time it takes to assemble. Outputting the OG meta data only, and no content, when your page is requested by the FB scraper, is the best option here. The scraper can be recognized by the User-Agent header it sends. So you can either implement that in your CMS’ business logic directly. Or you could rewrite requests by the scraper internally, so that it reads the data from a different source (such as a static, minimal HTML document) and circumvents your CMS.
    – CBroe
    May 3, 2017 at 14:33
  • @CBroe, thank you so much for taking the time to get back to me. I've read at least 100 other answers with people facing similar issues and I've seen you help so many people get out of a hole; really appreciate the good work, unfortunately, the existing answers haven't been applicable to me. Getting to the point you made, if I duplicate the post -> strip down all the text and image content (except the one featured image) -> add the meta data, tags, make it no-index -> then directing it from URL A to B would help? Would the scraper be redirected before the timeout though or how do I do this?
    – Diet Bros
    May 4, 2017 at 6:05
  • You would not be externally redirecting from A to B, but only internally. Assuming you are using Apache/.htaccess, you would first check the User-Agent header using a RewriteCond - and if the requesting client is the FB scraper, you internally redirect from /article.htm (the full article) to /article.meta.htm (only the OG meta data). So from the outside, the URL stays the same - you are just serving different content, depending on who is requesting it.
    – CBroe
    May 4, 2017 at 8:15
  • @CBroe You should post that as an answer, complete with a couple of code examples :)
    – anon
    May 4, 2017 at 15:44
  • @CBroe, thanks for that! Any chance you could direct me as to how I can go about doing this? Any links I could look at or if you could guide me? I have only a couple of pages I will need to do this for, since the rest of the content pages aren't more than a fifth (10,000 words) in size - at best.
    – Diet Bros
    May 5, 2017 at 6:03

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.