I know it's 2 years old, but it's fun to look back and see where we were, not too long ago.
His #1 need is "Outstanding Unicode support", and he's switching to Python for this. Just today there was a blog post that made the rounds, "The Truth About Unicode In Python" (http://www.cmlenz.net/archives/2008/07/the-truth-about-unico...) that gripes about Unicode in Python!
One of the gripes about Python was that its internal representation is UCS-2 by default. In comparison, SBCL uses 21 bits internally, and his Python example works fine in SBCL (on the Mac, no less):
* (characterp #\MUSICAL_SYMBOL_G_CLEF)
T
* (char-name #\MUSICAL_SYMBOL_G_CLEF)
"MUSICAL_SYMBOL_G_CLEF"
I wonder if he had to make the choice again on a fresh project today which he would choose. Things change fast.
(But note that CL implementations also don't provide clever sorting or any kind of regexps at all. SBCL and Allegro do get case conversions right -- at least for Latin characters with accents. I'm too ignorant to test more exotic characters.)
He could have just said "I only use stuff well-supported on OSX". This isn't about Lisp, really. He'll face this dilemma "X vs OSX" over and over and over again, where X can be anything, from latest vim goodies to some arbitrary libraries.
I was in the same situation and I simply ditched OSX: it just can't rival Linux as an ultimate programmers' world exploration vehicle. Ironically, it was Python and some Python libraries that drove me to Linux.
I have a Mac too, of course, as an ultimate Safari+Photoshop machine.
Vim works allright, but getting the latest version with all your favorite plugins won't be as easy as "apt-get and fly". When Leopard came out I was left without gvim for some time.
I actually just decided to start learning Closure. I was really impressed by this introduction: http://clojure.blip.tv/file/982823 and http://clojure.blip.tv/file/982957. I've tried to get into Lisp a number of times (largely because of the arguments/enthusiasm expressed for it in Hackers & Painters), but have not found an implementation that seemed worth pouring large amounts of time and energy into. My first impression with Clojure is that it could become the first "mainstream" Lisp.
Uh, what? SBCL + SLIME works fine on OS X. In fact, the canonical SLIME video uses that exact combination.
I haven't use CLISP on a Mac... but I'm sure it works fine.
I also think it's interesting that he describes Python as having "outstanding" Unicode support. There's an article on the front page right now about how poor Python's Unicode support is. (There's some support, but it's not outstanding. Merely acceptable.)
Might be considered outstanding when compared to no support at all. You're right though, Unicode in Python could be much better, but on the other hand I've never had Python's Unicode support get in my way.
Well, I don't know. Popularity definitely gets you libraries, but to a large extent, the runtimes are VERY daunting even to interested people. The author raises some valid issues with CL implementations. I'd also add concurrency/threading, for example. Part of the problem is that the standard (in my opinion) standardises some of the "wrong" things. In part, that's just down to its age, but also just the whole "design by committee" problem. So if there are important omissions in the standard, you get a load of implementations that technically comply, but in effect are incompatible with one another for any real work.
Most of the popular newer languages have a "benevolent dictator" (or not so benevolent in some cases), which means there's a reference implementation which almost everyone uses. Okay, maybe you lose out a bit on runtime quality, but I think the current trend towards building on top of the JVM, etc. will continue and alleviate that particular problem.
I also tried to use Common Lisp, I really did. I liked the core, but in the end it was the sheer volume of little problems that drove me away. Luckily, I've found a lovely compromise in Clojure.
Well, libraries may be superficial, but I few times now I've started in LISP, needed something that I just can't find, can't build myself (I'm not that good) and switched to Python because I needed to get stuff done. I am slowly but surely assembling all the LISP pieces I need (CLSQL being the latest piece of the puzzle). But the truth is you can install Python, be doing something useful within a day and writing production code within a week. LISP requires much more upfront investment before you (well, me) can be productive in it. That's not a criticism of LISP itself, but the lack of a "batteries included" distribution.
OK, here's one example. I have an app that receives email, processes it, and sends it back. I need to get all its attachments, do work on them and build a new message. In Python parsing an email is:
import email
msg = email.message_from_string(sys.stdin.read())
How easy is that? The subject is msg["Subject"]. Iterating over the MIME parts is 2 lines of code. I could build all the scaffolding very quickly, and concentrate on the problems I needed to solve. Maybe this is just as easy in LISP - if you already know where to get all the bits, and I don't, and I'm not good enough to write a production-grade RFC822 handler in a day.
I like LISP a lot and am trying to use it where I can, but at the end of the day, I have to get stuff out the door as quickly as I can...
I think more than anything else, this is probably the largest problem with Lisp adoption. Maybe Clojure will have better luck in this space because of its ability to drop into the JVM to get libraries.
Java's email library sucks. You need to sumbit to the JavaMail way of looking at email to do the simplest thing with it. First you need to understand that JavaMail is 'cross-platform' in the sense that it's for any hypothetical messaging system, not just email. So it has layers and layers of lame abstractions. Lots of configuration needed to do the simplest things.
That's one of the worst examples, but my overall feeling is that dynamic languages on the JVM aren't worth it. You're probably better off with an independent language environment.
I suspect the biggest problem with Lisp adoption may just be that people find s-expressions terrifyingly incomprehensible. Though there's always some reason the median programmer doesn't want to use Lisp, the reason changes. In the 1980s people said Lisp was too slow. Now it's libraries. This kind of variety is the signature of reasons people offer when they're avoiding something out of fear or laziness.
I started writing something similar: I think the real reason most people don't like Lisp is because most people don't like to think in s-expressions. (Then I thought nah, nobody's going to like that, so I deleted it and just left the bit about libraries - foolish move!)
It's not only fear or laziness; I know two very good programmers who have tried Lisp and don't like it. Interestingly, both are Smalltalkers, though one was of the original generation that migrated to Java.
I wonder if it's that some people like thinking in trees and some don't. Human language has barely any nesting. Putting one parenthetical phrase inside another is already rare and that's just two levels. Parenthetical nesting is so difficult for humans to process that there's even a well-known hypnotic induction based on it (that essentially creates a stack overflow in a person's brain).
One of the guys I'm talking about has a programming style that's markedly linear (though not procedural). He likes chaining expressions together almost like noun phrases in English. For example, where in Lisp you'd write
(min y (max x 0))
he might write:
x.atLeast(0).atMost(y)
It's a trivial example, but enough to imagine more complex ones. The difference is dramatic if you think of a tree with many levels and the comparable unwound form. Of course, not all trees can be unwound in this way.
The converse is true, though: all chains of this form could be converted to s-expressions, so it's easy to imagine embedding this style into Lisp with a macro. That's the sort of thing that gets talked about naively, but never catches on. I think it would be a losing proposition. You'd lose Lisp's regularity, which is one of its greatest advantages. And the resulting programs would still have plenty of s-expressions, so you'd end up driving away the same group of people after all.
Most programmers I've known learn whatever languages they're taught in school and then learn new languages are required by their jobs. So even if a "median programmer" is terrified by Lisp, if they work somewhere that uses Lisp and have to learn it, then they will. I introduced Python to my workplace simply by using it on a project for which it was the best choice, which forced some of my co-workers to later start learning it.
Thus, the lack of a batteries-included Lisp might well be more significant than a fear of s-expressions. Hopefully Arc will one day reach the point where I can sneak it into my workplace as I have done for Python. At that point the fear of s-expressions will hopefully be irrelevant.
This kind of variety is the signature of reasons people offer when they're avoiding something out of fear or laziness.
Heh, there is no fear or laziness here. I am a 32-year-old who grew up with Pascal, learned FORTRAN in college, wasted a few years working in Java, and have done Python for a few years. All the time I am moving up the scale towards more powerful languages. I started learning LISP this year, and so long as I stay within the LISP runtime I'm a happy bunny. At the boundary between LISP and the outside world, tho', I keep hitting obstacles that Python breezes through without a pause.
The lack of libraries is a significant problem, certainly to anyone who wants to use LISP in an existing environment, no matter how motivated they are to do so, given the constraints that a good-enough language already exists and the expectation that integrating the code with everything else is essentially a solved problem.
this is probably the largest problem with Lisp adoption
Sometimes it's not clear if that's really the problem, or if it's more the perception (and the fact that the perception gets repeated a lot) that's the problem. Hence my question earlier in the thread.
So, it would take W hours to get Lisp working well. He will spend X hours writing and maintaining his python over the next Y years. And if he used Lisp instead, after the initial W hour investment, he would save Z hours on less maintenance and coding time thanks to Lisp's power.
Is Z greater than W, or less than W? That's the critical question for him.
How can he decide which approach is best without estimating the hour values of each variable?
Edit: of course there's some other issues. maybe he'd spend 80 hours on python maintenance vs 100 writing lisp unicode support. the numbers are made up, but the point is that maintenance probably isn't much fun, so one might choose the lisp path even if it takes a bit more time. besides fun, one might learn more that way.
Why are you automatically assuming that the maintenance costs of a Lisp app are lower than of a Python app?
Lisp as a concept is very powerful. The current dialects, their implementations, respective libraries and package management systems on the other hand have often gotten in my way. For example take a look at http://article.gmane.org/gmane.lisp.cclan.general/807
More powerful language -> shorter, DRYer code -> less maintenance.
(Plus a few other reasons. See PG's essays, PG's Lisp books, and SICP if you don't know the other reasons.)
If you don't believe this, don't use Lisp. The guy writing the article does seem to believe it. He likes and respects Lisp. So he ought to analyze in the way I said.
So it's lack of faith that is his problem? But yeah I agree with you, the more powerful the language, the shorter code you can write.
Though python is a pretty powerful language in it's own right, it's got a repl console and sweet little list comprehensions and more, so is the difference really that great between python and lisp? Python is not assembler after all. To know that I suppose I should have to learn lisp myself, which I haven't had the time to do yet. Surrounded by java zealots at work as I am though, and that clojure seems to be the best working non-statically typed language on the JVM I just might.
The issue with utf-8 seems to be solved by now from reading the comments around here. But there are always problems working with utf-8 with libraries written by people that have never needed to work on non-ascii text, the code has simply not been tested thoroughly on the problem of working with utf-8 text so while it might be production quality regarding to ASCII there are weird bugs that pop out of the woodwork when working with musical notes.
On a related note: I've written a few programs in ruby where I've really run into trouble not having decent unicode support in the language. With iconv you can easily make it work if you are working with characters that map to an 8-bit character set. But if you are working with characters that map to several european languages at once, and need the euro symbol, you're in the cold. You can have utf-8 strings in ruby yes, but if you pry them apart, or make changes to them, they loose their utf-8:ness and look like shit. This is an issue that should really be solved at language level, if you don't want to have roughly similar the same abstraction level dealing with strings as you have in C. I love ruby, but I hate working with utf-8 strings in it just now.
> unicode is an issue that should really be solved at language level
with lisp, you don't need anything to be built in "at the language level" because you can modify the language much better than with python or ruby. many "built-in" lisp features, that would be language-level in other languages, are just built-in macros, and you could have written them yourself exactly the same. for example, loops are done this way.
What you say is true, of course, but at the same time it's true of any Turing-complete languages, and therefore, meaningless. I'm not a bad engineer because I don't have time to start every project from the bare metal. Runtime and libraries make a huge difference in the real world. Sure I'd love to spend the time really diving deep and making LISP into the perfect DSL for me, but the truth is, I can just use Python and go.
His #1 need is "Outstanding Unicode support", and he's switching to Python for this. Just today there was a blog post that made the rounds, "The Truth About Unicode In Python" (http://www.cmlenz.net/archives/2008/07/the-truth-about-unico...) that gripes about Unicode in Python!
One of the gripes about Python was that its internal representation is UCS-2 by default. In comparison, SBCL uses 21 bits internally, and his Python example works fine in SBCL (on the Mac, no less):
I wonder if he had to make the choice again on a fresh project today which he would choose. Things change fast.