Because setting up a TCP connection is costly in terms of roundtrips/latency. With SSL inside even more.

Also when you have only 1 connection you might better utilize the bandwidth you have because of TCP's slow-start algo.

I wonder if the better plan would be to insert another layer between TCP and HTTP (or any app layer protocol) that provides for multiple streams between the same points.

(If that's exactly what SPDY is, never mind.)

That's basically what SPDY is. It has two halves, a framing layer (which is generic), and a definition of how to embed HTTP within that framing layer. In theory, you could use the framing layer for other purposes, but we designed it tailor-made for http.

For the original question - HTTP can only send one request at a time. SPDY can send many all at the same time, avoiding lots of round trips. As a side effect, it sends fewer bytes and packets too. look for the IETF slides for the latest data.