This is what I was recalling, this method gives you a clever way to do it using the file itself as the key:
> “Convergent encryption solves this problem in a very clever way:
“The way to make sure that every unique user with the same file ends up with an encrypted version of that file that is also identical is to ensure they use the same key.
However, you can’t share keys between users, because that defeats the entire point; you need a common reference point between users that is unknown to anyone but those users.
“The answer is to use the file itself: the system creates a hash of the file’s content, and that hash (a long string of characters derived from a known algorithm) is the key that is used to encrypt said file.
“If every iCloud user uses this technique — and given that Apple implements the system, they do — then every iCloud user with the same file will produce the same encrypted file, given that they are using the same key (which is derived from the file itself); that means that Apple only needs to store one version of that file even as it makes said file available to everyone who “uploaded” it (in truth, because iCloud integration goes down to the device, the file is probably never actually uploaded at all — Apple just includes a reference to the file that already exists on its servers, thus saving a huge amount of money on both storage costs and bandwidth).
“There is one huge flaw in convergent encryption, however, called “confirmation of file”: if you know the original file you by definition can identify the encrypted version of that file (because the key is derived from the file itself). When it comes to CSAM, though, this flaw is a feature: because Apple uses convergent encryption for its end-to-end encryption it can by definition do server-side scanning of files and exploit the “confirmation of file” flaw to confirm if CSAM exists, and, by extension, who “uploaded” it. Apple’s extremely low rates of CSAM reporting suggest that the company is not currently pursuing this approach, but it is the most obvious way to scan for CSAM given it has abandoned its on-device plan.”
That makes me happy, because 12 years ago here on HN I posted a comment [1] outlining how a Dropbox-like service could be implemented that stored user files encrypted, with the service not having the keys, yet allow for full deduplication when different users were storing the same file, while still supporting the normal Dropbox sharing features.
The file encryption part was based on using a hash of the file as the key.
It's always nice to later find out that one's quick amateur idea turns out to be an independent rediscovery of something legit. Now that I've learned it is called "convergent encryption" Googling tells me it it goes back to 1995 and a Stac patent.
This still suffers the same problems as the original proposal. Specifically, Apple could still be pressured or forced by governments to check for non-CSAM images. And using cryptographic hashing means they can’t detect altered files, while using perspective hashing leaves them open to false positives.
That’s not what’s commonly understood to be a modern cipher.
It would be trivial for a government to make a list of undesired messages/images and find everyone that has forwarded it.
Bit, yes. Though on a moderately large file it would be easy to brute force all one-bit modifications, and then the effort grows exponentially (basically) in the number of bits flipped, so you’ll want to do more than a few.
> “Convergent encryption solves this problem in a very clever way:
“The way to make sure that every unique user with the same file ends up with an encrypted version of that file that is also identical is to ensure they use the same key. However, you can’t share keys between users, because that defeats the entire point; you need a common reference point between users that is unknown to anyone but those users.
“The answer is to use the file itself: the system creates a hash of the file’s content, and that hash (a long string of characters derived from a known algorithm) is the key that is used to encrypt said file.
“If every iCloud user uses this technique — and given that Apple implements the system, they do — then every iCloud user with the same file will produce the same encrypted file, given that they are using the same key (which is derived from the file itself); that means that Apple only needs to store one version of that file even as it makes said file available to everyone who “uploaded” it (in truth, because iCloud integration goes down to the device, the file is probably never actually uploaded at all — Apple just includes a reference to the file that already exists on its servers, thus saving a huge amount of money on both storage costs and bandwidth).
“There is one huge flaw in convergent encryption, however, called “confirmation of file”: if you know the original file you by definition can identify the encrypted version of that file (because the key is derived from the file itself). When it comes to CSAM, though, this flaw is a feature: because Apple uses convergent encryption for its end-to-end encryption it can by definition do server-side scanning of files and exploit the “confirmation of file” flaw to confirm if CSAM exists, and, by extension, who “uploaded” it. Apple’s extremely low rates of CSAM reporting suggest that the company is not currently pursuing this approach, but it is the most obvious way to scan for CSAM given it has abandoned its on-device plan.”
https://stratechery.com/2022/apple-icloud-encryption-csam-sc...