Kuro5hin.org: technology and culture, from the trenches
create account | help/FAQ | contact | links | search | IRC | site news
[ Everything | Diaries | Technology | Science | Culture | Politics | Media | News | Internet | Op-Ed | Fiction | Meta | MLP ]
We need your support: buy an ad | premium membership

Smut Filter Blocks All But Smut

By fluffy grue in News
Wed Jun 21, 2000 at 04:30:34 PM EST
Tags: Internet (all tags)

According to Wired News, a supposedly "intelligent" neural-net-based pornography-blocking filter has been found to be not at all intelligent. It seems to simply block 95% of images, regardless of content, and use a few other futile heuristics (such as "does the filename contain 'sex'?") in a lame attempt to improve its accuracy.

I think this finally hammers home the point to journalists that filtering software can't be perfect, and automatic filtering software is just as dangerous (in different ways) than filters setup by humans. According to the Wired report, some of the images claimed to be obscene were some very sexy-looking tomatoes, a stud in heat, a deity going down on an entire population, and a Wired news staff meeting (though knowing those guys, it probably was obscene). Images which were passed by the filter (sorry, no links :) were things such as "explicit images of oral sex, anal sex, group sex, masturbation, and ejaculation."

Of course, the guys' excuses are pathetic. "It was working before, you just caught us at a bad time. It used to work perfectly, we swear! We'll have it working again within a month!" They didn't have any backups of any properly-working algorithms. Of course.

This quote just sums it up pretty well:

PC Magazine, which in April 2000 gave BAIR a five-star rating and an editor's choice award, did not immediately respond to a request for comment. Ironically, BAIR frequently blocks ads on the websites of PC Magazine, Wired News, CNN.com, and other news services.


Voxel dot net
o Managed Hosting
o VoxCAST Content Delivery
o Raw Infrastructure


Related Links
o Wired News
o very sexy-looking tomatoes
o a stud in heat
o going down
o Also by fluffy grue

Display: Sort:
Smut Filter Blocks All But Smut | 20 comments (11 topical, 9 editorial, 0 hidden)
Always remember Dilbert... (4.00 / 2) (#8)
by PresJPolk on Wed Jun 21, 2000 at 02:07:44 PM EST

Dilbert: I'm inventing a new technlolgy to prevent kids from seeing smut on the internet.
Dogbert: So, you're pitting your intelligence against the collective sex drive of all the teenagers who own computers?
Dilbert: What is your point?
Dogbert: Did you know that if you put a little hat on a snowball it can last a long time in hell?

Dilbert: Matt [a kid], your job is to text my new invention that blocks kids from seeing dirty pictures on the internet.
Dilbert turns to Alice: His youthful curiosity is no match for my technical brilliance.
[Matt's eyeballs get really big as he looks at the screen]
Dilbert: I hope that wasn't the sound of eyeballs getting really big

Re: Always remember Dilbert... (3.00 / 2) (#13)
by Anonymous Hero on Wed Jun 21, 2000 at 09:22:48 PM EST

Lol :-P

Dilbert rocks.


[ Parent ]
This is horrible! (3.00 / 2) (#11)
by Moebius on Wed Jun 21, 2000 at 08:00:46 PM EST

Now all they have to do is add a not gate and it'll work perfectly!

Try it yourself (4.50 / 2) (#12)
by Moebius on Wed Jun 21, 2000 at 08:06:31 PM EST

It appears they have a downloadable 14 day trail version of BAIR available on their page. I look forward to putting this through some paces of my own...

Re: Try it yourself (none / 0) (#18)
by 31: on Fri Jun 23, 2000 at 12:50:28 PM EST

hmmm... any link to a win2000 version? It seems to want service pack 4... my esteame of them is dwindling even faster now, how friggen hard is it to check which version of NT it's running on?
(and it just ruined my day, all the buildings at my university are flesh-toned to some degree, and with a digital camera with slightly messed up color, I just want it to call one of the lecture halls obscene)

[ Parent ]
Hmm (none / 0) (#14)
by Anonymous Hero on Thu Jun 22, 2000 at 07:43:39 AM EST

Surely a much better filter would be one that didn't show any images on pages with meta tags that contain "bad" words?

Image based filters will not work well for a long time. If you don't want a kid looking at naughty sites, why block just the images? Surely you should block the sites as well, and then you can bring in other knowledge, such as the URL, the content on the site, or whatever (20 baner ads on top of each other?).

I can't see the use of a technology that seems to ban 90% more than it should, and still get it wrong. This is just a rip-off, and should be passed to the criminal authorities as such.

the core of the article: (none / 0) (#15)
by Anonymous 242 on Thu Jun 22, 2000 at 11:44:41 AM EST

I highly encourage anyone who hasn't actually read the article to go back and do so. Aside from the claims of the manufacturere seeming to be outright fraudelent, the reported brought up a good point from a CMU scientist.

Dave Touretzky, a senior research scientist in the computer science department at Carnegie Mellon University, doubts Exotrope's claims.

"How do you tell the difference between a woman in a bikini in a sailboat which is not racy and a naked woman in a sailboat?" Touretzky asks. "The only difference is a couple of nipples and a patch of pubic hair. You're not going to be able to find that with a neural network."

"If they don't disclose the training data, there's no way to figure out what's going on," Touretzky says. "But anyone who knows anything about neural networks knows there's no way it can do what they're claiming."

I'm not exactly a proponent of pr0n, but it seems to me that the whole idea of a neural network judging whether or not images are 'obscene' or 'adult' in nature is laughable. You can't even get two randomly selected people to agree on obscenity (and pornography) actually is.

You can't do it only on skin tones. Adult cartoons will slip right on past. Many innocent pictures will get canned. I could be wrong, but to give this even a shot at working it seems to be that you would need a big-blue class machine to realtime pattern recognition based on hundreds of thousands, if not millions of patterns.

Don't trust optical computing blindly! (none / 0) (#16)
by Mantra on Thu Jun 22, 2000 at 02:47:49 PM EST

This product reminds me of a story I heard about in the 80's when DARPA was trying to create a similar neural net-like recognition package to identify tanks in the field with the obvious intention of using it to search for camoflaged enemy tanks without human eyes. The system seemed to work great, becoming more and more accurate as more learning sets were fed in. Almost too good.

Someone wisely decided to see if the system would work in other combat environments beside the NATO-theatre from which the training sets had been based. Viola, when they tried desert environments the match rate dropped to near zero even when all there was was the tank on bare ground!

They went back to the original training set and discovered that it didn't matter if a tank was present in the training set - substituting images of the same NATO background scenes without tanks always matched also. The neural net was matching the foliage that was repeatedly being used in the staged learning set scenes - they accidently were always camoflaging the tanks in exactly the same foliage so the system seemed to match tanks but was only matching foliage.

Like others have said, programs can't tell porn from non-porn, and when you use neural nets it's likely you won't be able to tell what the program is actually matching on either.

Funny as hell, but not surprising... (none / 0) (#17)
by Anonymous Hero on Fri Jun 23, 2000 at 03:57:28 AM EST

From the PC-magazine review:

"If the software finds pornographic images, they're removed from the page before it reaches your browser, and the server initiates further analysis that may result in the page being added to the blacklist. The filtering and analysis don't slow down your Internet access noticeably, though, because they're performed on Exotrope's bank of Cray supercomputers rather than on your CPU"

This is a clear hint on what is going on: URLs are sent to exotrope that match them with their URL database (+ 'sex' in filename stuff). If the URL is not found, then (one or more) another Censorware (probably third-party) is probably used to match it and add it to the censored database. If the third-party censorware don't know, the the 'neural network' fires and returns a random match.

Clearly, if sent a random sample of internet sites, it will probably get a good match (at least not worse than the underlying censorware).

But wired sent unknown URLs, and the thing was doomed.

Stay tuned for the next version: it will probably have an extra layer, indexed by the MD5 of every know pornographic/non-pornographic image (or something slighly more fuzzy). Expect all images copied from photo.net of mccullagh.org being accepted.

It will still fails on 'new' pictures (taking your figital camera and making new shots). But exotrope will be happy selling this shitware to people, while handling your surf habits to intereseted third parties...



This is good! (none / 0) (#19)
by Anonymous Hero on Fri Jun 23, 2000 at 07:01:05 PM EST

If a smut filter blocks all but smut, can't we filter what it filters? E.g., for each image downloaded, check if it's blocked by the smut filter, if it is, let it through! If not, then the image really is smut so it should be blocked. What does everyone else think?

Re: This is good! (1.00 / 1) (#20)
by Anonymous Hero on Fri Jun 23, 2000 at 09:30:48 PM EST

the thing is that it doesn't really block everything that isn't smut.
you sir are a babboon

[ Parent ]
Smut Filter Blocks All But Smut | 20 comments (11 topical, 9 editorial, 0 hidden)
Display: Sort:


All trademarks and copyrights on this page are owned by their respective companies. The Rest 2000 - Present Kuro5hin.org Inc.
See our legalese page for copyright policies. Please also read our Privacy Policy.
Kuro5hin.org is powered by Free Software, including Apache, Perl, and Linux, The Scoop Engine that runs this site is freely available, under the terms of the GPL.
Need some help? Email help@kuro5hin.org.
My heart's the long stairs.

Powered by Scoop create account | help/FAQ | mission | links | search | IRC | YOU choose the stories!