I tried that in the past and found it extremely unreliable as a production environment. Documentation was also non-existent and I'd have to manually handle clusters, setup my own observability and log stack, etc. Any cloud provider these days gives you all that out of the box for K8s, so I'm, not sure the time one would invest on Swarm really makes sense?
It’s not really black and white as you describe it, there’s a huge spectrum between “do it all yourself” and “go all in on someone’s setup.”
Different cloud services have different levels of difficulty in migrating in or out of them.
You can also mix levels of abstraction for different layers of your product.
For example, you can host something on a bare EC2 instance fronted by nginx (let’s encrypt for certs) with an RDS database and that’s going to be far more portable to someone else’s cloud than deploying to Lambda behind an ALB using AWS certificate manager.
You still didn’t “do it all yourself” because RDS still took a solid chunk of your work away even though you did nginx and your deployment to EC2 on your own.
In the case of RDS it’s one of the more trivial services to move to another provider or move to bare metal since you’re just running a standard database and all your app needs is a connection string.
(I’m not claiming this is a real architecture that makes sense, just an example of how different layers can be chosen to be managed or unmanaged).
Unless you start using a non-portable RDS feature. Or you set up RDS auth using the preferred AWS method which is non-portable. Or you come to depend on a performance feature of RDS hosting. Or..
1. Rare, and easy to avoid, even if using Aurora. Anything without "Aurora" in the name is a plain database engine with no Amazon customization (e.g., PostgreSQL).
2. Not correct, IAM authentication is not the preferred connection method, and it has a performance limit of 200 connections per second. It's intended for access by humans and not by your applications. In my experience I've never seen any organization set it up. The other authentication methods are not AWS specific (Kerberos/password auth). Easy to avoid.
3. Most performance features of RDS have some kind of non-AWS equivalent. AWS isn't reinventing the wheel as a database host.
This startup is probably just a web server though. Let’s not pretend that multi cloud doesn’t have administration cost, network latency when using one cloud your data can literally be on the same server rack as your compute if you need it to be at best or at worse be in the same data center.
That also begs the question if you need the cheapest costs possible, wouldn’t you be better off at a colo and have your processing and data in the same data center and not have to spend extra on cloud costs?
