Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

What if you also have a week between marking/moving a file to start deletion, and the final removal?

If a backup and a GC race then a file can get both referenced and marked at the same time, but then a future GC will see the references and put the file back into a normal state. Assume other operations can still find the file while it's marked.

Are there benefits to CAS for this situation other than resolving faster?



Sure, you could probably use that kind of delay. I have personally seen GC systems that take more than a week to mark which files need to be deleted but this is admittedly unlikely (the system in question was massive).

> Are there benefits to CAS for this situation other than resolving faster?

I think this kind of thing comes up a lot, where you’d find it convenient to have a CAS update for your file. Like, maybe you should be using a database, but you’re already using S3 and having one or two CAS operation would mean that you can stick with S3.

Sometimes, the alternative is a little ugly. Like, “I’m going to create a DynamoDB table, and it’s only going to contain one row.”

What I’d really love, even more, is to have some kind of distributed lock service on AWS. Something like Zookeeper or Etcd as a SaaS product, where it’s cheap just to get a couple distributed locks. Feels like a gap in cloud offerings to me, but I can understand why it’s missing.


You can use S3 for this, no? (Admittedly it's clunkier than a service with SDKs.)

LIST, GET, and PUT are strongly consistent, the file name is the lock name, write the owner id and expiry timestamp in the file, and periodically extend the lock expiry (heartbeat). If an other process finds an expired lock delete the file.


Oh I'm sure there's lots of systems where CAS is very useful.

It's just that a backup tends to have mostly immutable files sitting around, so it becomes more niche. It's awkward to do a lock but you don't need a lot of locking.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: