@codahale, was this email from you or someone else in your company?
Judging from your blog post a year ago at http://eng.yammer.com/blog/tag/scala, at one point you thought Scala was great. As someone who still thinks Scala is great, I would like to know how long it took you to change your mind and what triggered you to reach this realization. When it comes down to it, I've never developed Scala as part of a team or worked on a project more than a few thousand lines of code, so I definitely lack the experience you probably have with the language and runtime.
As to the email itself, I find the complaints about performance most uninsightful, and I say this as someone who has written CPU-intensive NLP code in Scala. Anyone who writes performant code to run on the JVM knows that the only way to get C-like performance in Java is to, it turns out, write C-like code in Java!
That means operating only on primitives and arrays of primitives when in tight loops. It means avoiding allocations as much as possible. It even means avoiding generics like the plague, including java.util.HashMap. In my NLP code, for instance, I got at least a 100x speed up by switching from HashMaps with string keys to arrays by manually interning each string into an incremented number. One library I used, called Mallet, relied on GNU trove to have hash maps without boxing of primitives, because for high performance, even java.util.HashMap is too slow!
This means that, if you want amazing performance in Scala, you have to write C-like code in Scala as well. I'd expect anyone who needs top-notch performance to know this kind of thing already. Writing performant code in any high-level language in our day and age, even C++, still requires you to understand how your abstractions are actually getting implemented underneath the hood. Scala doesn't work miracles here or everyone would probably be using it already.
Just ask the people who run JGit about performant JVM code. (http://marc.info/?l=git&m=124111702609723&w=2) They ran into a ton of cases where they couldn't write the equivalent C-code in Java and performance suffered as a consequence.
However, the points about the build toolchain, backwards binary compatibility, and to some extent, initial learning complexity are the ones that I think are most damning against Scala. I hope Typesafe has a good response to this or other companies step up to share how they worked around these deficiencies successfully. Otherwise, I feel like this could be potentially very damaging to Scala's reputation, and maybe even deservedly so.
I've found Scala to be a fabulous language for writing > 90% of my app(s). The overhead of Scala's built-in collections, for loops, property accessors/mutators, closures, etc is simply unimportant outside of critical hotspots and provides so much in terms of brevity and expressiveness.
Yes, Scala's lack of looping constructs (break, continue, a real indexed for loop) means that you're better off writing performance-critical things like crypto or protocol parsing code in Java (or if it really matters, C called via JNI). But once you're concerned about performance at that level, every language requires you to carefully consider what your code is really generating, what your allocation patterns are, etc.
Scala is a much more expressive language compared to Java, and the Scala standard library is excellent. Completely dropping Scala for Java when you're building large, complex systems seems like chopping your foot off because your small toe hurts. Use Java or C for the % of code that needs it and Scala for the rest!
The complaints were not primarily about performance. The bigger issue seemed to me the difficulty of finding people experienced in Scala and teaching it to those who are not, (due to the complexity of the language), and the difficulty of producing maintainable code (due to frequent changes in the language and a lack of standard practices).
Expressiveness alone does not make a language suited for building large, complex systems.
Part of the original complaint was the issues introduced by using a combination of Java and Scala rather than just one or the other (with Scala-only being impractical due to the need to use Java-based libraries).
"2. Don't ever use scala.collection.mutable. Replacing a scala.collection.mutable.HashMap with a java.util.HashMap in a wrapper produced an order-of-magnitude performance benefit for one of these loops. Again, this led to some heinous code as any of its methods which took a Builder or CanBuildFrom would immediately land us with a mutable.HashMap. (We ended up using explicit external iterators and a while-loop, too.)"
Overall Scala<->Java interop is pretty seamless, and aside from some annoying things (for loop becomes foreach with a closure, break & continue are done with exceptions, etc) it's quite easy to treat Scala as a more concise Java with very similar performance.
Rather than using a Scala collections wrapper for the java.util.HashMap he should have simply used the Java HashMap as-is. Yes you must do hashMap.get("key") instead of Scala's shortcut hashMap("key"), but so what? You're talking performance-critical code already, wouldn't you want to avoid the wrapper?
"avoiding generics like the plague, including java.util.HashMap"
With generics being a compile time concept, you probably meant "avoiding collections like the plague"
Judging from your blog post a year ago at http://eng.yammer.com/blog/tag/scala, at one point you thought Scala was great. As someone who still thinks Scala is great, I would like to know how long it took you to change your mind and what triggered you to reach this realization. When it comes down to it, I've never developed Scala as part of a team or worked on a project more than a few thousand lines of code, so I definitely lack the experience you probably have with the language and runtime.
As to the email itself, I find the complaints about performance most uninsightful, and I say this as someone who has written CPU-intensive NLP code in Scala. Anyone who writes performant code to run on the JVM knows that the only way to get C-like performance in Java is to, it turns out, write C-like code in Java!
That means operating only on primitives and arrays of primitives when in tight loops. It means avoiding allocations as much as possible. It even means avoiding generics like the plague, including java.util.HashMap. In my NLP code, for instance, I got at least a 100x speed up by switching from HashMaps with string keys to arrays by manually interning each string into an incremented number. One library I used, called Mallet, relied on GNU trove to have hash maps without boxing of primitives, because for high performance, even java.util.HashMap is too slow!
This means that, if you want amazing performance in Scala, you have to write C-like code in Scala as well. I'd expect anyone who needs top-notch performance to know this kind of thing already. Writing performant code in any high-level language in our day and age, even C++, still requires you to understand how your abstractions are actually getting implemented underneath the hood. Scala doesn't work miracles here or everyone would probably be using it already.
Just ask the people who run JGit about performant JVM code. (http://marc.info/?l=git&m=124111702609723&w=2) They ran into a ton of cases where they couldn't write the equivalent C-code in Java and performance suffered as a consequence.
However, the points about the build toolchain, backwards binary compatibility, and to some extent, initial learning complexity are the ones that I think are most damning against Scala. I hope Typesafe has a good response to this or other companies step up to share how they worked around these deficiencies successfully. Otherwise, I feel like this could be potentially very damaging to Scala's reputation, and maybe even deservedly so.