Since this morning, my website has started failing with 403 error codes when anybody attempts to share a link to my website.
When I use the Facebook debugger tool, it is suggesting 'This response code could be due to a robots.txt block. Please allowlist facebookexternalhit on your sites robots.txt config to utilize Facebook scraping'
I have a whitelist override rule setup in Cloudflare and show the Facebook bot coming to the page every time I attempt to update the last scraped time and receiving a '200 OK' response from a Cached Page (meaning that Cloudflare is sending Facebook a Cached page of the link).
But for some reason, Facebook is saying 'The document returned no data.'
All of my 403 pages for the website return data (an HTML page saying that you re blocked or error etc). Basically, nothing should return blank.
Is this happening for anybody else? I have tried debugging it all morning and I am sure that Facebook bot is correctly accessing my website (receiving a 200 OK from a Cached page that other users are also receiving fine) - yet it's showing these errors.
Where can I begin to fix this or how?
I have updated my robots.txt file to specifically allow the Facebook bot (and Google bot and Twitter).