I run a website where aspiring writers post their writing. Naturally, some of this writing can be of a 'sensitive' nature. Sometimes it contains a lot of swear words, sometimes it's malicious, sometimes it's suicidal. It's not the norm, but it exists.
Of course, we want our users to feel free to post what content they like, but the issue arises with advertisers. Namely, Google Adsense, which routinely sends us warnings about a piece of content that contains too much swearing or is defamatory. The latest is a piece someone wrote about cutting themselves which Google has deemed too 'tragic'.
I've generally been able to write scripts too look through the text and detect a lot of swear words or racist terms or anti-gay rhetoric, etc. But I can't for the live of me figure out how to detect 'tragic' text.
So, the question is 2 fold.
- Is there some sort of PHP class/function/API that is good at detecting inappropriate content?
- Any idea on how to automatically detect suicidal or self-harming content?