Context switching is pretty cheap, but it’s hard to beat free. Eventually the overhead of OS context switching is probably going to become non-negligible in most cases. I think up to a point the answer to when it matters is “that depends on how deep your pockets are.” You can keep scaling vertically and horizontally forever with more resources, obviously, but I don’t think it would take too long for the memory and CPU efficiency of async to start to win out, at least for runtimes with proper async.
Besides that, it’s pretty hard to decide you want to go async later on. In languages where async is not omnipresent, you usually get some form of explicit continuations; async/await sugar if you’re lucky. But you can always accidentally call something not async and hang, or just spend a lot of CPU time with no ability to preempt.
I think Erlang, Go and probably a couple of others basically stand alone in this regard; everything is just async by default. It’s a shame that it requires a decent amount of runtime baggage to accomplish this.
I also think people forget that Robinhood was made with Python, Reddit used Python, Github/Gitlab use Ruby. Stripe AKA payments that need to be processed fast within SLA's use Ruby.
Ruby and Python can only ever have one thread running at a time due to global interpreter lock. Though to keep in mind when I/O happens on a thread it can be parked and another thread can be picked up to continue processing. But in essence one thread is running at a time and these languages built billion dollar businesses.
Just scale horizontally computers are cheap is the message I get and remember to keep it simple.
But, it’s only efficient in convenient, shared-nothing type architectures. If you are dealing with longer lived, more stateful stuff (like a game server perhaps) it’s not going to be so easy.
Of course, it’s easy for fancy VC funded stuff to defer optimizations later. If you are working solo you are probably going to want to, counter intuitively maybe, spend more time thinking about how to keep things optimal and simple. There’s probably startups out there worth real money whose entire stack could run on a $20/mo DigitalOcean box if it were better optimized, but actually runs on a $1000/mo Amazon or GCP account. Of course there is a lot to be said about that (obviously, the latter set up should, if done well, be a lot more resilient and prepared to deal with scale) but it does go a long way to demonstrate why someone might care about efficient async; the scale of money you can burn when you are bootstrapping or doing solo hobby projects is a lot different from well-funded startup or established business...
Everything has tradeoffs. I do believe Go wins in context switching, but the GC latency and CPU usage has definitely caused some problems. The Go developers have done an impressive job with GC latency and performance, but it’s hard to be cheaper than free, so sometimes Rust is a better option, for example.
Python and Ruby are fine for embarrassingly parallel problems, like serving content from well-sharded users.
I'd be very surprised if Robinhood's trade matching engine is done in Python. I'd bet they use an off-the-shelf engine in C++/Java or another language with good multi-threading support.
> computers are cheap is the message I get and remember to keep it simple.
Fully agree! Getting to product-market fit is probably easier with python and ruby.
Besides that, it’s pretty hard to decide you want to go async later on. In languages where async is not omnipresent, you usually get some form of explicit continuations; async/await sugar if you’re lucky. But you can always accidentally call something not async and hang, or just spend a lot of CPU time with no ability to preempt.
I think Erlang, Go and probably a couple of others basically stand alone in this regard; everything is just async by default. It’s a shame that it requires a decent amount of runtime baggage to accomplish this.