I've thought about this issue before (and proof-of-concepted a similar system, see http://www.blahedo.org/botblock/), and came to similar conclusions, but there's an important difference:
A crucial part of making this a successful anti-spam system is that it is a moving target. Every user of the system must be able to write their own questions. If that happens, the spammer's task is intractable. But if there is a central site serving these, it will be worth the spammers' while to just hardcode the patterns and write a little bit of logic to parse and answer them.
Now, there's a fair bit of interesting UI design in the question of "how do I get a non-programmer to write what is in essence a very small program". My proof of concept used some cute Perl-isms to basically construct a mini-language that was restricted enough that an inexperienced programmer could "script kiddie" their way through it, and I think this is the right general direction, but you'd need a fair amount of work to really make it accessible to the masses.
(Other crucial points that he gets right: it must be text based; it must have questions that hinge on natural language understanding but not be otherwise difficult; and it must have questions that are really question templates each of which can generate infinite numbers of question instances.)
I've often wanted my own text-based CAPTCHA for a video game website I run. I'd ask things like "What is the name of the purple weapon?" or "How many shields do you start with?" People who actually play the game could nail questions like that, while bots would be up a creek.
Then he would be a douchebag... and probably a huge moron, too.
Who creates a bot specifically to overcome the CAPTCHA on a forum for a 15 year old video game with very little traffic? We're not really a significant target; I only have to ban about one spambot per week. There's a tremendously low ROI from spambots on our forum, I can't imagine it'd be worth anyone's time to even attempt to incorporate it into their CAPTCHA-breaking bot.
A crucial part of making this a successful anti-spam system is that it is a moving target. Every user of the system must be able to write their own questions. If that happens, the spammer's task is intractable. But if there is a central site serving these, it will be worth the spammers' while to just hardcode the patterns and write a little bit of logic to parse and answer them.
Now, there's a fair bit of interesting UI design in the question of "how do I get a non-programmer to write what is in essence a very small program". My proof of concept used some cute Perl-isms to basically construct a mini-language that was restricted enough that an inexperienced programmer could "script kiddie" their way through it, and I think this is the right general direction, but you'd need a fair amount of work to really make it accessible to the masses.
(Other crucial points that he gets right: it must be text based; it must have questions that hinge on natural language understanding but not be otherwise difficult; and it must have questions that are really question templates each of which can generate infinite numbers of question instances.)