Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
The Moral Failure of Computer Scientists (theatlantic.com)
94 points by ___ab___ on Dec 12, 2015 | hide | past | favorite | 46 comments


Yes, cryptographers should take on the surveillance state.

So should developers, and entrepreneurs, and politicians, and the common citizen that thinks AES is some foreign sports conference or IND-CCA is a new trade agreement.

I'm growing tired of watching journos try to point the fingers of "encryption is good, says cryptographers" and "encryption is bad, says this guy in a federal law enforcement position" and completely miss the greater point that unless everyone takes on the surveillance state, everyone will lose. The intelligence community is made up of humans. Some of them are there idealistically; others are there solely for power, just like in any megapolitical organization.

Should <x> take on the surveillance state is a shit clickbait title. It can always be reduced to:

Should you take on the surveillance state?

If you care at all about a semblance of privacy and the expression of individual ideas, the answer is always yes. Of course, there's a whole lot less to write about there to generate ad revenue.

Any approach in which a single subculture, whether it is thought leaders or soccer moms, tries to enact meaningful change in a system that at least pretends to be a representative democracy, will not be enough to reach a critical mass to actually do something. Cultures correct negative behavior through consistent reinforcement of a norm. Until people want privacy as a norm, and fight for privacy as a norm, the flames fueling a surveillance state are simply being retarded, not extinguished.


" tries to enact meaningful change in a system that at least pretends to be a representative democracy, will not be enough to reach a critical mass to actually do something. Cultures correct negative behavior through consistent reinforcement of a norm. Until people want privacy as a norm, and fight for privacy as a norm, the flames fueling a surveillance state are simply being retarded, not extinguished."

That's my exact point. The laws need to change to reduce what they can do, punish offenses, and optionally encourage the better approaches. To get that, a huge amount of people have to lean on Congress. That will only happen if they start valuing their privacy or at least have common sense that scumbags + unlimited power/knowledge + immunity is a bad idea. Cryptographers, past speaking out, can't solve that problem and hence are just irritating to opponents who continue to win while laws enable them to.


It's hard to take on the surveillance state when you have to make a living and all the money and jobs are in surveillance and surveillance based business models.


I think there is a distinction to be made between surveillance and data, and I do not necessarily think you need to go full cypherpunk to make a meaningful difference. The cat is well out of the bag on data collection and analytics and it will be that way forever.

However, people working within institutions that contain sensitive data can help affect positive change from the inside as well as the outside. Be a voice for the security of user data within your organization, and do what you can to meaningfully contribute to methods that increase user anonymity and business models that do not require being excessively intrusive into the personal details of others. There is something you can do everywhere, and it is far better to have people privacy-minded even inside of the classical "surveillance" companies than having these companies run recklessly without a voice for the user. In fact, it may even have more impact than yet another frontend for yet another OTR implementation.


I hear you, but whether or not something is difficult is orthogonal to whether or not it's morally right. You don't get off the hook because doing the right thing is hard.


I wish this article were more particular about what cryptologists are doing wrong? What exactly could they have done instead? Without that info, it just seems like pointing a finger at cryptologists and saying, “This is your job! Do something!”


Rogaway wrote an companion essay for his invited talk that is rather accessible: http://web.cs.ucdavis.edu/~rogaway/papers/moral-fn.pdf


Modern academic cryptographers have veered off into a land of purely theoretical constructs. Now I love the cool theory we have in crypto, but it's sad that despite all this cool theory, very little of it becomes applicable in building practical privacy-preserving systems.

Rogaway, is a famed academic cryptographer, student of Silvio Micali, who basically founded modern academic cryptography. He is highly respected within the crypto community, so his words hold a lot of weight, and my interactions with crypto professors indicates that they agree with what he says.


That's probably not a fair criticism. This is reporting about an extremely famous cryptographer pointing his finger at other cryptographers.

Here's more about it:

https://news.ycombinator.com/item?id=10657540


Thanks for that pointer. I agree with GP's criticism, though, at least the way the article is written. I was trying to understand it through the lens of the nuclear analogy and failing drastically, because whereas those scientists were developing PRO-nuclear technology, these ones are (a priori, to first order) developing ANTI-surveillance technology.


The nuclear analogy is a little odd, given that one of the reasons we don't have even more nuclear weapons / more states with nuclear weapons / nuclear weapons out in the wild is not technological... but because of international treaties and enforcement.


Rogaway's paper starts with an extended discussion of the role of scientists in the nuclear arms race.


In short: no. This is a political problem that must be solved by laws that people push for. People have been supporting surveillance state or apathetic. Hence, it's winning and their combo of police power + secrecy + immunity is stronger than crypto.


Strongly disagree.

Legal protection provides a recourse after everything happens. Technological measures don't let it happen in the first place. Or, well, to be more correct - make it significantly harder to happen.

Consider: we can send all our email as non-enveloped postcards and rely on the laws that our correspondence privacy is protected. But for some reason we don't. Why we still send send out our Internet correspondence completely unprotected is beyond me.

It is important that we have laws, so we can get a legal recourse if something goes wrong. But it's extremely naive to think that no one would violate those laws just because they are in place.

Even more, I believe that technological measures must come first. Because if a law comes first, the public relaxes, thinks they're safe now, and few bother about actually putting a lock on the door.


"Legal protection provides a recourse after everything happens. Technological measures don't let it happen in the first place. Or, well, to be more correct - make it significantly harder to happen."

It actually prevents many things when the law is clear. Your email example misses the entire point. So, let's use it to illustrate the point. I create an encryption system to protect email. It gets large uptake to point NSA and FBI are pissed by it. With current laws, they will feel free to:

1. Hit me with a FISA warrant to order a backdoor or key leak.

2. Hit me with court order to do the same.

3. Parallel construct some dirt on me.

4. Use NSA TAO or TAREX to smash my systems for their benefit.

5. Use FBI to raid my stuff or seize my property.

6. Have me audited by SEC or IRS depending on my company structure.

We've seen stuff like this happen to leakers, supporters of Wikileaks, companies resisting subversion, etc. You can build all the tech in the world but it's not that helpful if legal system is set up to destroy the user or developer easily. Those laws need to be rolled back. Only the people can do that. They don't give a shit enough to act. So, it's a political problem rather than technical one.

Feel free to continue to deploy and use tech to protect yourself. Just know the bigger problem is what's enabling their surveillance dragnet and police state problem in first place. The things that can get you with or without crypto. The things that have to go away to maintain democracy.


Ah, sorry, I see your point now. I suppose I got it wrong when I replied to your comment. Yes, I fully agree with you here on the point that the laws that allow this are wrong and they must be rolled back. Those are legal issues and they must be fixed as such.

I must make it clear that I stand that both legal and technological measures are necessary and are equally important. And I believe that neither would work well without the other one.

Current mass surveillance relies on lack of technical measures that protect from one. So, I believe that if everyone and their dog encrypts their correspondence in a secure manner, it would cause much greater hit on mass surveillance programs than any lawmaking could do. Please note I don't say that lawmaking is not necessary here. On the contrary, it is equally important to prevent TLAs from even trying to break technological measures and hold them responsible for their actions.


"I must make it clear that I stand that both legal and technological measures are necessary and are equally important. And I believe that neither would work well without the other one."

100% agree. The overall solution will combine technological methods and legal reforms. We continue developing and implementing what technical solutions we can for privacy and security in general. Just have to never fool ourselves on what it will take to stop the huge internal threat.


Pointers to (seemingly) frivolous prosecutions (3) and pointers to anything resembling 4 or 6 would make them a lot more interesting.

Without just a little bit of evidence, they are like saying the NSA will shoot your dog.

My naive, facile reading suggests that systems like Signal, Pond and Tor tend to be more effective at actually securing communications, so it would be especially interesting to hear about the jackboots kicking them.


The technique used for No 3 is here and other documents suggest they work with many agencies rather than just DEA:

http://www.huffingtonpost.com/peter-van-buren/parallel-const...

Number 4 we're not going to get examples of: TAO & TAREX operate in a bubble. There are two known efforts to do this sort of thing but tactics are unknown. One is BULLRUN:

http://securityaffairs.co/wordpress/17577/intelligence/nsa-b...

Additionally, the ECI leaks mention that the "FBI compels" firms to "SIGINT-enable" their stuff. This means the FBI has some way of coercing companies to backdoor things. The specifics were left out. That they've been doing it for years with no details public indicate even talking about it must be a crime. Like the other stuff.

IRS, SEC, whoever being used against people is a tactic with a long history. My bookmarks aren't giving me a link right now. I do recall Nacchio of Qwest claiming government came after him for being only ones not helping NSA. A quick Google had Binney saying IRS and NSA worked tight together albeit with speculation on Tea Party rather than obstacles to SIGINT as target.

http://www.wnd.com/2014/07/whistleblower-irs-in-cahoots-with...

And FBI raiding and seizing opponents stuff is well-known, happening to most leakers, too. Civil forfeiture is another weapon with a long history at FBI and DEA especially. Some journalists during Bush-Cheney Administration ended up on Do Not Fly list. Tor project people like Applebaum get harrassed at borders. So on and so forth. Many methods they can use without ever doing time for the abuse.

What they will do to you depends on who you are, what you're doing, what dirt they have on you, your resources, and so on. The uncertainty is one of their most powerful weapons. Never know when hammer will drop on you or how hard.


Specific examples are more interesting than raising the specter. You've doubled down on raising the specter.


Think I could've done better? Just file some FOIA requests and lawsuits on the subject requesting all specific examples of cooperation between NSA and other LEO's plus list of all TAO and BULLRUN activities against Americans. Bet you'll have less than I posted.

Btw, a manual for DEA of using NSA's evidence isn't a specter: means it's ongoing.


Ongoing retaliation against people working on crypto?


Part of my overall claims here is that the police state aspect of our government only kicks in on priority targets. Average person or project in crypto doesn't matter. A good case would be solid protection that applies to high-value target. Might support those like Wikileaks, Snowden, or terrorist cell that happens to use a specific product.

So, we saw LEO's and payment processors largely kill Wikileaks by cutting its donations off. Wikileaks supporter and Tor evangelist Jacob Appelbaum does plenty OPSEC to avoid problems they aim at him. Lavabit, Snowden's email provider, was shutdown after receiving a secret order to compromise all its users and lie to them about it (see records if you doubt that part). Surespot allegedly pulled its warrant canary after ISIS used its tech. Apple and other companies doing end-to-end messaging are getting hit hundreds of millions at a time via Virnetx: a shell company for patents from NSA, CIA, and SAIC.

Seems to be a number of actions and reactions against anyone that becomes a problem. Most never see it. Hence, would doubt it's a concern. That they do enforcement part with "legitimate" organizations and courts makes that more so. That's be beauty of the modern, Dual State: invisible to most means odds of questioning or getting rid of it are lower than prior surveillance and police state models.


You said I create an encryption system to protect email. It gets large uptake to point NSA and FBI are pissed by it. With current laws, they will feel free to

Lavabit shut down because they got a court order! Related to Snowden (who clearly violated the letter of the law, he isn't just some rabble rouser that they decided to hassle). It would also be more interesting if Lavabit was forced to shut down because it did not have the ability to access users communications, rather than the refusal of Levison of provide the FBI with access (a capability he did have).

You directly state that Appelbaum manages to avoid problems (remember where I started? How would he just avoid them shooting his dog?).


"Lavabit shut down because they got a court order!"

Love how you word it in a way that throws it under the rug. You could similarly say all of America deserves to be under surveillance because they got a [FISA] court order for certain parameters. Most court orders involve invading the privacy of the accused or some people connected. Lavabit was ordered to compromise all users and to lie about it. The only reason we know the order exists is because the owner fought for that. Before that, people like you would've argued that individual accounts info are turned over per warrants or court orders but not all users. And certainly not secret, black boxes added to companies' networks to do whatever the government wanted with that data.

"It would also be more interesting if Lavabit was forced to shut down because it did not have the ability to access users communications, rather than the refusal of Levison of provide the FBI with access (a capability he did have)."

Hence the warrant canaries, including what I gave you. Unfortunately, they're threatened with prison if they give details up to that they've been compelled to compromise their users. "End-to-end," but proprietary, schemes can be compromised with the update mechanism. It's all we have because that's what kind of pseudo-police state we live in.

"How would he just avoid them shooting his dog?"

You have no arguments against my claims so you make up other ones I never said? People's dogs getting shot is something that happens in my city but I never said it would happen to U.S. government opponents. I've said they would use whatever legal pressure they could against whoever mattered. This includes FBI FISA warrants, NSL's, and court orders. This might also include harassment via other agencies. There's precedent for these things. They don't shoot peoples' dogs and they rarely assassinate them if that's your next move.

Going to prison with what dirt NSA can give FBI is already bad enough for most. Lifelong surveillance, risk, and/or harassment is the common reaction. Financial troubles for businesses. Most just go along with whatever they're doing.


If you're going to use the envelope analogy, also consider that we don't put our mail in envelopes to prevent the police from intercepting it; we put them in envelopes to prevent access by all of the people handling the mail between the sender and intended recipient. A cop with a warrant can rightly get access to a person's mail in transit. The envelope also isn't particularly difficult to get around - we don't secure our mail through technical measures but instead by putting stiff legal penalties on tampering with it.


Wrong. The surveillance state does not follow laws, and it has all the dirt it could ever want on politicians. Laws are not a solution here. The first rule of security is "don't trust the network". Computer scientists and developers who build systems that require users to trust the network are uniquely culpable. This is an unpopular opinion here because many of us would like to continue collecting hefty paychecks while washing our hands of any responsibility for our actions.


You're way off. The people and politicians knew to get a grip on this decades ago when they discovered all the abuses of CIA, etc. They had two main choices:

1. Create accountability mechanisms a la GAO working alongside these organizations ensuring they follow the law and imprison offenders.

2. Create a court that approves most of what they do, never imprisons offenders, and operates in secret.

America went with No 2. Further, most of those caught red-handed didn't do time. Americans also didn't push hard for reform with their votes. Intelligence and oversight fought back and forth but effective immunity let their corruption and power expand over time. It went into overdrive post-9/11 where people not only didn't do crap: they encouraged giving secrecy, vast power, and criminal immunity to the very groups that failed pre-9/11.

So, this didn't happen in a vacuum and isn't today. American's apathy and frankly ignorance is what gave scumbags a series of blank checks with immunity. Americans didn't do anything learning about Iraq WMD's, 2008 frauds, Snowden leaks, and so on. Largely nothing but griping. Meanwhile, in Iceland, they straight up overthrew their dirty government after 2008 abuses and passed new laws protecting their citizens. Exactly what Americans have to do.

Let's say they don't. Then, Congress continues passing police state style legislation, secret agencies bribe our ISP's/whoever, fabs eventually get compromised, dissidents are harassed via many means, opponents with dirt are jailed via parallel construction, patent system will be used against tech companies trying to solve it, and so on. Damn near pointless to try to technologically solve a problem that a country's citizens and politicians are creating and expanding with laws that can attack users of the tech.

All this shit is Americans' fought. Their common sense should've told them giving God-like knowledge and power to already-dirty groups was stupid. Doing it with secrecy and immunity was stupider. Not doing anything post abuses was foolish. My money is on them still being fools aiding surveillance state 5 years from now. They have to wise up and remove the internal threats' legal authority before technical solutions have a chance.


This comment doesn't make a whole lot of sense to me.

What's all this talk about America? We've been talking about China the whole time. You're seriously blaming the Chinese people for their heavy-handed government? By focusing their work in directions that harm freedom, computer scientists and developers make it easier for the Chinese government to use surveillance to hold onto power.


I'm talking about the surveillance state in America. My points apply to surveillance states in most democracies, though. China is a rather extreme situation. Yet, the points still apply: their people at the mercy of a corrupt government means the government can use laws and money poured into harmful tech to continue to hold them down. The solution, even there, will be aimed by the people straight at the government.


>You're seriously blaming the Chinese people for their heavy-handed government?

Yes.


>American's apathy and frankly ignorance is what gave scumbags a series of blank checks with immunity. Americans didn't do anything learning about Iraq WMD's, 2008 frauds, Snowden leaks, and so on. Largely nothing but griping.

Hold on, I think you're giving the average American way too much credit here. My mom has at various times said that the police state is a good thing, the FBI should be able to view anybody's data for any reason, and groups the state names as terrorists should be denied freedom of speech.


Yeah, there are plenty of those... an even bigger problem.


The legal aspect was taken care of by the Bill of Rights. The govt is trying to erode them and so now we need to "fight" against it.


I'm not with 'nickpsecurity on this issue, but what you just said is not a valid argument.

Cryptography enables individuals to override the interests of any element of the state, no matter how compelling those interests are.

Any way you read the Fourth Amendment, even the wrong way that implies that the government requires a warrant for any conceivable search, there remains a pathway through which the state can compel the production of data. Congress may have to enact a law to authorize the compulsion, and a judge may need to sign off on the warrant, but at the end of the day, the government has the authority to compel production.

I think cryptography is mostly orthogonal to the Fourth Amendment, but to the extent it isn't, its main implication is that it thwarts the Fourth Amendment.


First, the Constitution as a whole is "addressed" to the government, not the people. That is, the Constitution tells the government what it may do. In the case of the IVth, it tells the government search powers are limited to those allowed by the Constitution.

If the Constitution is amended to say "Government officials may levitate" it does not confer the power to make gravity illegal.

Similarly, if encryption can build an uncrackable "safe" for your documents - moreover one that can be made invisible and deniable - government search powers are as limited by mathematics as they are by gravity. The only difference is that government officials don't actually expect to levitate.

That means when the people invent something that thwarts government power, there is nothing in the Constitution that says anything about that. Even less does it say "No, the people can't have that."


My position is that possession and use of cryptography won't protect individuals against government's illegitimate actions if citizens let that governments rogue agencies continually amass more power, surveillance, and control over IT.

If you're against that position, please tell me why crypto alone (not active democracy) is all one needs to be safe against state abuse in a surveillance state that targets crypto users. I'm sure any activists aiding Chinese and North Korean dissidents will appreciate your tips, too. They, like I, have been under the same delusion that TLA's legal power matters and must be dealt with.


It wasn't: that was only the beginning. Democracy, like security, is a process rather than a thing you do once. So, people have to be "eternally vigilant" fighting off advances against their established rights in legislature and in court rulings. People can sit on the sidelines of corruption and have a democracy. That's what's happening now. It's why we're pretty far from the Bill of Rights in practice but still have enough to reform with.

People just got to put it to use.


Whoah: this title sucks all the oxygen out of what is a very interesting article centering on an interview with Phil Rogaway, one of the world's great cryptographers.

The title should instead be the one The Atlantic chose, which closely mirrors Rogaway's recent paper: "The Moral Failure of Computer Scientists".

Submitters to HN are meant to use the original title from the article where possible, unless that title is so bad it detracts from the article.


Sorry, our mistake. Fixed.


J'ACCUSE! Please, sir, read the HN guidelines, which I've helpfully linked here:

https://news.ycombinator.com/newsguidelines.html

Please don't editorialize titles. We ban accounts that do that, so please don't do that.

We've asked you this before!


At last, we've found the counter-example to Betteridge's Law of Headlines.


There are many counter-examples. The important part isn’t that the answer is “no”, as the Law states, but that the story is tired enough to need a question in its headline in the first place. Andrew Marr’s explanation focusses especially on this aspect of the Law: “A headline with a question mark at the end means, in the vast majority of cases, that the story is tendentious or over-sold. It is often a scare story, or an attempt to elevate some run-of-the-mill piece of reporting into a national controversy and, preferably, a national panic. To a busy journalist hunting for real information a question mark means ‘don’t bother reading this bit’.” He makes a few assumptions, but I think the overall takeaway of “question mark equals clickbait” rings true.


the title has been changed to include a ? when posting here it seems, (or they changed it on theatlantic.com). Nevertheless your comment about ? equalling clickbait really made me think.

Edit: ok I just realized that the summary at the top was used as the title when posting here. I don't feel there is anything wrong with this.


Agreed.


There's a corollary to Betteridge's Law: when someone posts an article with a question mark in the headline to Hacker News, someone will inevitably post a comment to the article that mentions Betteridge's Law.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: