Thats not so easy to answer. It starts to matter a lot when you get into very high performance architectures (think disruptor, anything requiring lots of mechanical sympathy, highly contended memory access etc.), but usually you're waiting for the database or the network anyway.

Supposing you meant memory bloat compared to C, increased developer productivity is almost always more important. Things like memory access patterns can be important when you are interested in optimizing hot loops, but not generally.