Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
From Radio to Porn, British Spies Track Web Users’ Online Identities (theintercept.com)
450 points by jakobdabo on Sept 25, 2015 | hide | past | favorite | 236 comments


Ladies and gentlemen. This is how you do blackmail, corporate espionage, and sway politicians on a truly industrial scale.

Want to push a bill through the House of Commons to curb GCHQ's powers? Think again Mr. Politician. Someone might just leak your sexual deviances to the newspaper.


The pr0n gets you blackmail material, but the social networks give you relationship maps.

Leaning on a politician is an obvious tactic, but it's somewhat heavy-handed. If the politician were to fight back, there could be expensive blowback. Worse, it's a single (or nearly single) point of failure. The bill could pass regardless of the actions of that one politician. A talented blackmailer should only want to directly lean on someone as a last resort.

There is much better tactic: find the people who may organize other people against your interests. If you take out the potential future leaders, journalists, and educators - aka, the people that will organize the writing of that bill - then you prevent the problem before it starts. When the FBI was busting up the hippies, this was known as COINTELPRO. It's laughable to think the program ended with the Church Committee - like "total information awareness", the program was simply split up into various existing programs.

A politician could be idealistic and fight back against blackmail. With social network information, a counter intelligence program can avoid this risk by leaning on the target's friends or family instead.


It reminds me of the post about using metadata to find Paul Revere: http://kieranhealy.org/blog/archives/2013/06/09/using-metada...

Last-second blackmail isn't really the best way to defeat a political movement. Compromising the central figures in organizing the movement -- not necessarily "taking them out", but potentially just reducing their involvement or creating some alienation -- when the movement is still young is the way to really stop it.


I was thinking of how to use it to de-cloak anonymous sources for journalists. Many of them are going to use anonymous email addresses, potentially VPNs/tor to further protect that but that's not the weak part.

Before you choose to share information, you're going to find out who will actually use it effectively which implies research. Then once you share the information, you're going to pay attention to what they write with it. In both cases, since you're not doing the actual leak then, you're unlikely to take any countermeasures, let alone effective ones.

For example, odds are Snowden researched Greenwald before he released anything to him. He probably read a ton of his pieces, looked at his wikipedia, and dug into his professional, etc profiles. And then after the leak, he probably continued to check Greenwald's regularly keeping an eye out for the new pieces. Even if we didn't know who Snowden was, a program like this would uncloak* him relatively quickly.

* Or at least narrow down the suspects.


All it takes is metadata.

The "digital exhaust" we leave behind is extremely revealing. I usually use COTRAVELER as my example of how just a few data points (without any PII) can reveal a LOT about your social interactions, but browser history is another powerful data source.

I believe this is why Dan Geer, in his incredible talk last year[1], suggested that "privacy" should now be defined as "the effective capacity to misrepresent yourself". To avoid exactly the type of analysis, you need the capability to misrepresent yourself, and you need to regularly be in the habit of use use that capability.

    Misrepresentation is using disinformation to frustrate data fusion
    on the part of whomever it is that is watching you.  Some of it can
    be low-tech, such as misrepresentation by paying your therapist in
    cash under an assumed name.  Misrepresentation means arming yourself
    not at Walmart but in living rooms.  Misrepresentation means swapping
    affinity cards at random with like-minded folks.  Misrepresentation
    means keeping an inventory of misconfigured webservers to proxy
    through.  Misrepresentation means putting a motor-generator between
    you and the Smart Grid.  Misrepresentation means using Tor for no
    reason at all.  Misrepresentation means hiding in plain sight when
    there is nowhere else to hide.  Misrepresentation means having not
    one digital identity that you cherish, burnish, and protect, but
    having as many as you can. 
 
[1] https://www.youtube.com/watch?v=nT-TGvYOBpI http://geer.tinho.net/geer.blackhat.6viii14.txt


And thus Angela Merkel has been spied on since her earliest political days.


I think we're due for a second sexual revolution, one where we once and for all settle our relationship between individual, society, and sex, where the only reason you should be concerned with someone else's consensual behavior is if:

1: You're banging them

2: You want to bang them

3: It's your kid and they're underage.


Isn't that just a statement of contemporary popular standards? They're not rules set in stone. The definitions of "underage" and "consent" are not nearly as clear cut as you might hope. Lots of people (in couples) have sex as a kind of trade for something they want. Is that consensual? What if it's a trade for something really really important to them? What if it's a trade to avoid being verbally abused by your partner? What if it's for money you need to survive?

Regarding age of consent, what if a mature 15yr old seduces a naive and vulnerable 21yr old and causes lasting emotional harm? How is that "none of my business" but the reverse isn't?

What if a 16 year old from England travels to America and has sex with a 20 year old? Is that OK (UK law) or not (US law)?

What you call a revolution sounds more like the way it already is for many people, and is full of arbitrary and changeable definitions that unfairly discriminate against harmless activity while also allowing harmful activity.


I find Japan's model of consent ages intriguing. I believe the standard age is 18, but exceptions can be made with parental consent down to 13. Not that it should be 13, but I do think it's rather absurd to create legal issues for, say, a 16-year old and a 19-year old who have been dating for three years,


This is one thing the EU has yet to work up the courage for, "harmonizing" the age of consent, which varies from 13 to 16 in member countries.


Why should the EU harmonize such things? I'm not necessarily disagreeing but it would be interesting to see the reasoning.


Because harmonizing things is what it does, it is bound to make the attempt at some point. It will be very interesting see how member countries react...


> Isn't that just a statement of contemporary popular standards?

No. Some sexual acts are still seemed shameful by society.


I think this is far different from a sexual revolution. Through history, our notions of acceptable sexual behaviour changed quite dramatically, so the sexual revolution isn't that unique.

But all through history, human beings had some notion of a private space. The catholic confession booth is the only counter example i can think of, and even there it was just between you and you priest which was trained specifically for that role, and you had some control of what you did expose there.

And at least some psychologists ,when asked about this issue, believe that a right for privacy is critical for the psychological well being of people, not something we can give away without consequences.


> But all through history, human beings had some notion of a private space.

This CBC series, The Origins of the Modern Public, would dispute that idea (specifically - and I don't know if this is what you are talking about - the positive sense of privacy). Luckily, a nice HNer ripped the mp3s. https://news.ycombinator.com/item?id=2984068

Here's the specific episode (8) on privacy

http://www.cbc.ca/player/Radio/Ideas/Full+Episodes/2010/ID/1...


How about a political revolution, where legislative power would rest on every citizen in the country thus making the aforementioned tactic impossible (if such system ever came to exist I think we should call it a "Democracy").


That is an absolutely horrible idea. Why? Because proper legislation requires expert knowledge. It is simply impossible for every citizen to know enough to be able to be an effective legislator That's why we elect representatives.


So instead we hope that the results of a popularity contest are experts in crafting 'proper legislation'? No, representatives are theoretically there to guard against majority tyranny - it has nothing to do with building the most efficient legislation generating machine. This design is reflected in the whole checks and balances thing. I do agree with you about it being a horrible idea though, maybe we should reconsider the wisdom in granting some small portion of our freedom to every other voter. 42% of Americans believe in ghosts... let that sink in.


>No, representatives are theoretically there to guard against majority tyranny

This is a classic case of the just world fallacy.

Historically, representative democracy with one man one vote exists because that was the result of the last compromise between the underclasses and elites trying to keep them from getting out of hand.

It isn't any more theoretically reasonable than the compromise before that (no women voting), or the one before that (no blacks) or the one before that (nobody votes who doesn't own property) or the one before that (nobody votes).


> This is a classic case of the just world fallacy.

Who is experiencing the bias here, the founders of the representative democracy or me for pointing to the political theory? I assume you're talking about some system that isn't the US government, because otherwise your recollection of history is way off.


That depends on whether you are endorsing it or not.

"Theoretically", we had kings because of the divine right of kings. Theoretically the American expansion was justified by manifest destiny. Theoretically women were not allowed to vote because "90% did not want it" or because it would "place the government under petticoat rule".

Theoretically, today, we have representatives in government because the people aren't smart enough to represent themselves.

Every system of power and domination creates a justification for its own existence and it's nearly always bullshit.


> That depends on whether you are endorsing it or not.

You might want to reserve the use of the word "fallacy" for cases where an argument's logic does not follow. Ironically, use the word otherwise often betrays one's own cognitive biases.

As far as my endorsement, I don't think I could state my position any more clearly than I already have: "...maybe we should reconsider the wisdom in granting some small portion of our freedom to every other voter. 42% of Americans believe in ghosts..."


> You might want to reserve the use of the word "fallacy" for cases where an argument's logic does not follow.

You have yet to make an argument that "representatives are theoretically there to guard against majority tyranny." If you had, it wouldn't be a just world fallacy.


> You have yet to make an argument that...

Correct, I have no interest in arguing that position. Any highschool graduate should immediately recognize "majority tyranny" and the political theory that it is connected to. If I had argued the virtue of representative democracy, it might have been biased by the just world fallacy, but I haven't - so it wasn't.



I have been thinking about sortition for a while now.


As have I, with some level of favourability. It avoids the problems of campaign finance and low voter turnout, though not of tampering once in office. It adds the additional complication of legislators or executives who face little or no consequence for their actions. Not sure how it would work out ultimately.


If it works out properly, such legislators will be few enough that it will not be a major problem.


Switzerland disproves your argument.


I can think of an exception to every rule.

Also, Switzerland does not vote on every law with a referendum.


Hmm... representatives... like the guy with two kitchens representing the homeless.


Democracy is slow and ill-suited for getting shit done.


Bureaucracy is slow and ill-suited for getting shit done.

The question is whether democracy needs bureaucracy, and if so, how much.

Obviously, democracy requires certain decisions to be made according to certain processes, which take time.

So you can't do away with all bureaucracy.

The documents aren't going to push themselves. :)


In ancient Athens, they managed it.


Ancient Athens, the home of democracy, was a small community.


Tell that to everyone who clicked on a tabloid headline about David Cameron and a pig's head.


Cameron is still in power.

Considering what his government gets up to, it's completely clear he has no conscience at all.


GCHQ and MI5 have been up to this sort of thing for decades.

I don't expect it'll ever change, as by the time someone's made it to high office, then the agencies will have collected enough material to keep them quiet and on side.


I'm finding humor in the fact that number 3 has an AND and not an OR in it.


Out of curiousity, why?

If your kid isn't underage, then presumably they are capable of making rational decisions about with whom and to what they consent. And at that point it's none of your damn business.


I think Lawtonfogle meant the opposite. If the minor isn't your kid, but you're aware of it, you should still make it your business.


I think he was pointing out the bug in the rule of (underage, not your kid).


Yeah I meant "your kid and underage" as your business in a personal sense. I assumed underage being the law's business was implied.

I like your choice of the word "bug." That's why I come to HN.


In a legal jurisdiction where the average person has ability to influence the law, the areas of personal business and legal business have significant overlap.


So if someone else is having sex with their underage daughter, I shouldn't care?


Consider the case where the individual is underage, isn't your kid, and the one(s) whose kid they are doesn't care.


I don't care if two OTHER 15 yr olds bang each other. If some punk 15yr old is banging mine, I care.


I'd correct #3 to *they're underage.


Who's really the weirdo. The guy watching pr0n or the guys watching the guy watching pr0n?


It is done because of data that it brings. Not because of deviances of House of Commons members. Moreover, in western countries like UK it is quite difficult to be classified as deviant.


"Someone might just leak your sexual deviances to the newspaper."

Within the services, this is colloquially called the 'sluts & nuts' file.


You don't blackmail them, you proactively undermine their credibility by just leaking anything that might discredit them.


Selectors accept wildcards: https://firstlook.org/wp-uploads/sites/1/2015/09/mutant-brot...

I wonder what this query returns: _@_

or: _._._._

And the justification textbox has a default width that accommodates about 7 words.

At one point in the article it says the domain of a logged website is considered metadata but the path (full URL) is considered content. However, this screenshot shows a logged HTTP GET including its full path: https://firstlook.org/wp-uploads/sites/1/2015/09/cryptome.pn...

"When compared to surveillance rules in place in the U.S., GCHQ notes in one document that the U.K. has 'a light oversight regime.'" ... and UK is one of the US's Five Eyes partners, so the US has a light oversight regime, in effect.

At least the GCHQ shows some honest self-awareness in naming their surveillance apps. Karma Police. For a minute here, it seems like the intelligence agencies have lost themselves. http://genius.com/Radiohead-karma-police-lyrics ... I particularly like the last comment on the page analyzing the song, which begins, "This song is about the projection of guilt onto the Other", and concludes: "Prisoners, soldiers, soldiers' victims; these are all sacrifices, onto which we project our guilt, so that we can be saved."


I've never been so ashamed to be British.

These last few years have really started to wear me down. The indiscriminate mass-surveillance, the savaging of our public services by an obviously corrupt government, the blatent hyprocrisy of our foreign policy, the repeated and unfettered fraud of the City of London financial sector that has yet to see anyone prosecuted or any institution meaningfully punished.

I really hate this place.


Given how much British politicians seem determined to make life shitty for anyone making less than six figures, you'd think they'd at least let people spend what idle time they have high on drugs instead of questioning a system that seems intent on shredding the middle class.


Plus we ( the UK ) have now jumped onboard with remote drone assassination of UK citizens abroad. Even though there hasn't been a death penalty on the statute books since 1998.


Every country in the Five Eyes is doing this, and many if not most who have the money to fund this kind of thing are doing it.

Move to Greece. They can't afford surveillance.


CSEC (Canada) is up to the same shenanigans, approx zero politicians are talking about reigning in the surveillance state in any meaningful way during the election right now so it will keep on rolling. I often talk to people who have no idea who Snowden is, nor have they ever heard of CSEC.


You should read more history. We have been awful for 100s of years. Just in the 20thC we destabilised the Middle East, exploded bombs in pubs in Eire to drum up discontent, jailed the innocent, protected the powerful, destroyed our own manufacturing in favour of financial services, exported asset stripping to the US.....

The list could go on.

Your shame is proportional to your knowledge, the level of reprehensible action is a constant.


Just get out. However this ends it's not a happy ending.

The UK establishment are nuts.


Get out? To where?!


Well, there's Canada..... Where the Harper government is in the process of eliminating all historical archives and scientific libraries...

Australia? Hmm, wasn't there something about data retention? And maybe Western Australia trying to change the laws so protestors would have to prove that they weren't about to do something illegal when they were arrested?

New Zealand? Baaah.


Don't forget the moves towards Internet censorship in Australia too.

Aside from lots of sheep, what's wrong with New Zealand?


Here's the PM of NZ on the GCSB bill that allowed sweeping powers for the NZ spy agency (under his control) [1].

"All they can do is protect you, so it's against malware or a virus. ... On your computer at home you almost certainly have Norton Antivirus...that is exactly what that is, at a much higher level."

And the good people of NZ decided they didn't mind. Because. I don't know. I give up. It's like the world is determined to make a mess of itself.

[1] http://www.3news.co.nz/tvshows/campbelllive/john-key-defends... (from 8:00 in the video)



Along with the items already noted, it's also pretty small and I'm not sure how many immigrants they accept though I think there's something about Commonwealth nations and immigration that might help.


Germany? How ironic would it be if people from the UK ended up fleeing to Germany?


Land prices are far lower there. Working people fought and now their kids are neo-serfs in the UK.


Austria. They still fight fascism.


Israel maybe?


The American establishment isn't much better.


Britain has a history of far worse things. I find the lack of awareness of Britains brutal past in the UK to be terribly disturbing. Morally it has one of the most repulsive histories of countries on earth yet all we hear from politicians is about British values as if they are some kind of goal to look up to. The superiority complex prevalent in British political discourse has no foundation yet is rolled out time and time again.


Perhaps I'm older than you but it's been the same in my mind since the 1980s at least. My father informs me earlier.

At least we have the joys of privatized services now (I'm serious). They are an order of magnitude better than the public railway, gas and electricity infrastructure we had. Controversially perhaps, I've had much better service from the NHS since they moved to a service model.

I'm not ashamed of my identity at all.


>They are an order of magnitude better than the public railway

Order of magnitude more expensive, maybe. Better? No.


The first 15 years of my life in the 1970s and 1980s was spent going in and out of London to various hospitals on trains.

No. Absolutely not. It's much better now. Orders of magnitude better. Every train was late. There were engineering works overruns every Monday and weekend service was so bad you might as well walk.

And no not more expensive. People couldn't afford the train then and regularly took the inferior busses. If you went long distance, it was coach.

Either way you were doomed to a face full of diesel fumes from a Paxman or a Leyland and spend the entire day with your nose running black snot. Incidentally this is what killed my father; he was a bus driver.


  I've never been so ashamed to be British.
The "illustrious" colonial and expansionist history of the British empire wasn't enough to tip the scales for you before?


Why should we be ashamed of something that was done before we where born, should the Germans still be ashamed for world war two, should US citizens still be ashamed for what the US Army did, should the Italians be ashamed for what the Romans did, the French be ashamed for what Napoleon did?.

I accept our history, I'm not ashamed of it, it's history and I wasn't there.

However I am ashamed to be British for the things we currently do.


Yep. Why do you think they're tracking porn habits, if not for blackmail material?

There's no legitimate reason why a government needs to know anything about citizens sexual preferences or habits, but such information is perfect for shaming or threatening. I will note that the collection and analysis of this sort of information far surpasses anything envisioned in dystopian fiction or actually practiced by the likes of Stasi/KGB during the worst of times.

EDIT: there were actually a bunch of documents published with this article. I'll check those out then re-comment.


It can be useful for blackmailing prominent figures of adversarial foreign regimes, though to be fair Britain's spies will have to sift through their own Parliament first.


I love the way this article cites loop-holes. These guys do whatever they like. They have no mandate and they do it anyway with the blessing of politicians who then pretend to be concerned.

For me as a Brit the UK establishment is the biggest threat to people in the West. I really hope the UK has a big financial crash and the resulting chaos leads to a full regime change.

The UK really scares me a lot more than Isis.


The trouble is that any regime change could be worse and the full tools of oppression are already built...


Could be worse, could be a lot better.

We are in diminishing returns for how much worse our government could be without going outright "pick up that can" authoritarian.

The list of scandals, corruption, privacy invasions and out right shittiness just goes on and on.

To echo the GP, I worry alot more about what the politicians are up to than I do terrorists.


With all honesty, if you're a programmer or a hacker working for GCHQ (I'm certain there are some of you around here) you should really bury your head in shame. No excuses, you are simply disgusting.


I completely disagree. GCHQ, NSA, CIA, et al do serve a legitimate purpose. The world is not all roses and sunshine, there are rogue nation-states and rogue actors, and they do want to harm the public. It is the duty of the government to protect its citizens from attackers. These agencies exist to protect the public by identifying threats, and preventing attacks.

I know you are upset about the spying on private citizens for deceptive purposes, but lets not throw out the good with the bad. Yes, the agencies are overzealous and have overstepped their mandate, but that does not mean every single person working for them is an evil person out to get you. Real people work there, and nearly all of them probably have nothing to do with this. Reform, not harassing innocent bystanders, should be the aim here.


The question is - is reform even possible in organizations like these that have repeatedly broken the law with impunity? Is oversight possible at all when those who are supposed to have the power of oversight are unable to do so because these organizations are actively and intentionally hiding what they are doing?

When what they've done, effectively, is surveil everyone including the very people who are supposed to be able to exercise oversight and enact reforms how can oversight succeed?

I for one think the corruption is simply too deep to root out while keeping these orgs intact. They need to be torn down and rebuilt with clear missions consistent with our constitution (in the us), strict boundaries, and oversight. Reform is insurmountable when they still hold this wealth of illegally gathered surveillance data and retain their existing command structure.

The data must be destroyed. These orgs must be broken up and rebuilt so they are fit for purpose. Criminal, not overzealous is the word for what they've done. That in no way makes every person working there a criminal and I don't think demonizing the rank and file is productive or appropriate but we cannot keep watering down the severity of the problem for the sake of their feelings either.


The data must be destroyed

History shows that the only way this is likely to happen is when the revolutionaries storm the building, such as the collapse of East Germany and the Stasi, or the Libyan security files blowing around in the street after the airstrikes.


You aren't wrong. Frankly I'm not sure anything I said is reasonably possible or how to make it happen.


Anything of interest was long moved to be used later. Don't kid yourself. Often times to control the so called revolutionaries.


> the data must be destroyed

A single 1To hdd may contain 10Ko of information on every UK citizen. So while we clean, we'd better check every single pocket of every employee...


Indeed. Complete destruction of the data is probably impossible. This is exactly why this kind of vacuum everything collection is so dangerous. There's no data access protection in the world bullet proof enough to justify the risk of storing it.


Thinking about it, at least with key escrow the private keys can be stored in HSMs.


It needs to be viewed as akin to nuclear waste


Here's the thing though. The reason there is even a debate about this is because the agencies serve a legitimate purpose that allows this to confuse our sense of morality. Without that rider, these methods would be roundly condemned in any other organization. A programmer working on unjustifiable programs should feel gross when they do something immoral.

Let's be clear here. This is the first time in the history of the world that nearly all ordinary communications by ordinary citizens, globally, are aggressively being monitored. The impact hasn't yet been fully felt by the global populace, but the power of these tools to rig the system and suppress dissent is very large and there is currently no effective check on these powers that is subject to public scrutiny.


Without reaching the Godwin point, actually it is not. The Soviet Union or East Germany were spying on their citizens on a similar scale, at least in respect of the technology available at that time.


The categorisation of "threat" is what we have a problem with. There was a scandal a while ago about the embedded deep cover agents in environmental groups: one had been there so long that he'd married and had a child with one of the people he was spying on. Further back there is the fairly terrible record of Special Branch et al in Northern Ireland.

Then there's the whole business of supplying intelligence to support the policy of invading Iraq, and the mysterious death of Dr. David Kelly. The Iraq invasion made us considerably less safe at the cost of about a million (non-coalition) lives.


GCHQ is a relic of the Cold War desperate to find a reason to continue existing.


What most people forget when they are sitting at home being scared about "rogue nation-states" from across the globe, or worry more about dying by the hand of people with dangerously fluffy beards instead of losing a foot to diabetes, is that complete security simply doesn't exist.

But what would we do if we wanted to provide total security? The naive and most popular approach is to give total control to the government. Not secure yet? More control then. Technology is the most promising way to achieve total control yet, and it's actually growing much faster than any global threat.

Anytime I read an article about some spying technology, I can imagine those developers looking at the existing infrastructure and internet technologies, thinking "we can totally do something to control this and that as well". That will actually be the "office culture" in places like the NSA. I know that's how I would think if I was sure my organization was serving a legitimate purpose. My colleagues would agree, and we'd high-five each other about redirecting yet another stream of information into our system. And whenever anyone has any doubts, group-dynamics or -think will take care of that. And if that's still not enough, maybe you need deeper philosophical help? https://boingboing.net/2015/08/11/the-failed-writer-who-beca...

With all those leaked documents, I've not even once heard about any limits. None. There's no "we'd like to know and control much more, but obviously we'll stop at that". If you gave the CIA the ultimate device that could provide them with all the information about every single human being on the planet and would also allow them to kill any individual instantly, they would immediately start using that. They don't want any oversight, they don't want anyone to control them, they don't want to have to respect anyone's privacy. They think they have a certain mandate or duty, and those things are by nature standing in their way of fulfilling it.

These developers absolutely should hang their heads in shame. Because every last one of them should understand that their organizations won't stop for ethical reasons, or deny themselves certain powers which should not be concentrated in the hands of a few people, no matter how well they think they mean. Being a "real person", not being out to get the average joe (but still spying on him), and generally meaning well - that doesn't make you any less guilty. At best it means you're a naive idiot who trusts authority too much, at worst you don't care and just want to play with your cool toys.

The world definitely isn't all roses and sunshine. Just because you think you're working for your country, making sure those who promise to protect it get all the help they could ever wish for and more, that doesn't mean the things you create will actually be used for "good". All it means is that someone further up now has more power than before, and they will use it the way everyone uses their power - to further their own interests. You simply hope your interests and theirs are the same and along the lines of "sticking it to those terrorists", but if history has shown anything, it's that power corrupts. I have zero doubt that those systems will be used against all of us eventually.


And what I don't understand is that people seem to be afraid of the bearbed terrorist who will strike the country once every few years, perhaps killing a couple of persons, but not by the local thugs who will kill, rape or injure a dozen persons in their neighbourhood every week.


Let's be honest. Secret services were always useful thugs. Using methods that would land them in jail in any other organisation. These guys make a career of spying, stealing, blackmailing, corrupting, sometimes killing. It's fair to argue that in certain circumstances they are useful and that a State should have the ability to commit these crimes. But let's not make the employees of these agencies anything else than what they are: hired thugs. And what I find worrying is that these crimes that were meant to be committed on an exceptional basis are now commited on a large scale against their own population.


The mafia also does lots of good and helpful things in the community.

Does that means mobsters also shouldn't be reviled?


"but that does not mean every single person working for them is an evil person out to get you."

I agree. Those engineers are just following orders. They bear no personal responsibility at all...


I think he meant that not everyone in these organizations is working on unsavory projects like this one, not that those who do are free from blame.


Lieutenant General David Morrison of the Australian Army was giving a strong message[1] against against sexism when he said,

    "The standard you walk past, is the standard you accept."
People working at GCHQ (or the NSA) may not be working specifically on an unsavory project, but they are still choosing to support the organization. Also, how do they know the real nature of their project? In a compartmentalized ("need to know") environment, it can be hard or impossible to know the real purpose of any particular task.

Jacob Appelbaum gave a very interesting talk[2] about a month ago where he specifically addressed[3] this problem (among other topics - the entire talk is worth watching). He even gets a question from someone from the NSA[4] who was in the audience, who also asked about this problem.

[1] https://www.youtube.com/watch?v=QaqpoeVgr8U

[2] https://www.youtube.com/watch?v=n9Xw3z-8oP4

[3] https://www.youtube.com/watch?v=n9Xw3z-8oP4#t=3160

[4] https://www.youtube.com/watch?v=n9Xw3z-8oP4#t=4028


> In a compartmentalized ("need to know") environment, it can be hard or impossible to know the real purpose of any particular task.

There are definitely a lot of areas that wouldn't suffer from this problem. I'd be surprised if a cryptographer couldn't distinguish between working on civilian signals intelligence vs military (since the first will be using published ciphers).

Any how far does the scope of assisting the organization go? If you work for the FBI in a non-surveillance capacity, are you still part of the machine since the FBI and NSA cooperate on counter-terrorism? Should everybody at the FBI with a conscience quit because some in the organization have taken advantage of overextended surveillance powers?


People who worked for the SS in random roles like radio operator are still being prosecuted for murder. The precedent set after 1945 is that working in an organisation that breaks basic human rights is a crime against humanity in and of itself virtually irrespective of your role in the organisation.

Also it's not like the UK has conscription into GCHQ.


Then why don't they focus on that alone? If you worked for an organization that saved 10 babies a month, but used 90 women for sex trade, would you continue to work there because "you're helping save babies", or would you be disgusted with that organization?

I know my example is a little extreme, but it's kind of what GCHQ and NSA do, too. They use their powers maybe 10% of the time to look for terrorists and whatnot, and then the other 90% to get data on everyone, including people in power that they can blackmail and influence. Don't think this is just a theory. There have been articles in the UK press about MI5 for instance covering up child abuse by politicians because they wanted to blackmail them with that. Enough said?


Not everyone views the world the same way. Some people would willingly trade their privacy for safety ("myself included") and would not feel "ashamed".

However, my appraisal of "safety" markedly differs (like most people's appraisals of things) and I am more concerned with an overzealous government than a foreign attacker. I would trade my google search history for a bullet-proof vest if I was in a war-zone. However, I am not, and I don't think giving it to the government makes me safer.

However, people view the risk to the country and themselves differently and they also put markedly different values on their own privacy[0]. Also, this article should come as absolutely NO surprise to anyone in the HN community (or even anyone with internet).

edit: [0] which they use as a framework to assess how others value their own privacy AND the value they themselves put on others privacy.


The problem here of course is you're not just trading your privacy for your feeling of security, it's that you are trading MY privacy for your sense of security, and doing so without my permission and against my direct wishes.


Look, I am pretty security conscious and while I am not going to list off my environment configuration, and it isn't perfect, I am sure I take privacy and security much more seriously (technologically) than most Americans.

The point I was sort of facetiously making here was that the people who implement these things don't think about it the way we do. They probably don't feel ashamed and do it out of a misplaced sense of patriotism, power or coercion(if they got caught doing wire crimes).

More granularly, if you believe you are protecting America from a 9/11 scale attack I could certainly see trading away some of your fellow countrymen's privacy.

These aren't my personal views on privacy or security I was making the point that some engineers won't be ashamed implementing this sort of stuff. I am pro freedom and privacy to be clear, and, I wouldn't make the above trade.


Cheers, you're selling the rest of us down the river.


>There have been articles in the UK press about MI5 for instance covering up child abuse by politicians because they wanted to blackmail them with that.

Citation?

I've seen articles like this, http://news.sky.com/story/1523821/child-abuse-pm-tells-polic..., but none that give the reason for the coverup as being enabling MI5 to blackmail people.



  there are rogue nation-states and rogue actors
Funny you're saying this in the same sentence about those agencies and you're totally missing how ironic that is.


It's sad that some people are downvoting the parent post. It makes a valid point in a reasonable way.

There certainly are problems with the excessive government surveillance -- it is also the duty of a government in any civilised, democratic nation to protect its citizens from itself, after all.

However, it seems unconstructive to go all gung-ho against people who are probably mostly just trying to do some good from what is inevitably sometimes an awkward position. It's like arguing we'd be better without a police force because sometimes bad people manage to become police officers and sometimes their colleagues can't just flick a switch and shut them off, or arguing that we shouldn't invest in military forces because it gives the government the power to kill people from afar and in any case diplomacy is preferable to violence. Those things may all be true, but that doesn't mean we'd really be better off if the organisations with a degree of corruption or inadequate safeguards were eliminated rather than fixed.


At least so far as police forces are concerned, they can't really be reformed. The police is fundamentally a paramilitary unit that serves the interests of a state, not private citizens. You can make the case that such an evil is preferable to abolishing it entirely, but ultimately the odds are stacked against it achieving any positive equilibrium, only trying to keep it as contained as possible.


> The police is fundamentally a paramilitary unit that serves the interests of a state, not private citizens.

This is not intrinsically the case; though modern general-purpose police organizations tend to be paramilitary (however, specialized regulatory law-enforcement bodies with police powers often aren't paramilitary in structure and operation the way most general police agencies working for the same government are.)

So, I don't think this is a situation that is fundamentally not subject to reform, though it might require rethinking fairly fundamentally the role of law enforcement, and redistributing much of what is now the role of general law enforcement agencies into domain specific agencies that are not defined by being enforcement agencies despite including enforcement functions within their domain as part of their brief.


At least so far as police forces are concerned, they can't really be reformed.

That seems pessimistic.

For one thing, any organisation made of real people is in reality affected by the views of those people. Even if there are some bad spies/cops/soldiers out there, a lot of people who get into that line of work do do it for the right reasons and because they genuinely want to make the world a safer place. If very bad things start happening, none of these organisations is going to universally side with a government that is doing those things.

For another thing, any organisation that requires funding is subject to influence by whoever funds it. As long as someone has to pay PC Smith's salary and there is enough civilisation left for essentials to require money to buy them, PC Smith is going to be accountable on some level because he or she literally can't afford not to be.

For a third thing, any organisation that enjoys special privileges that most people do not have, for example greater resources provided by the state or powers to do certain things legally when they would otherwise be illegal, is always under threat of having their special status revoked. The natural order is that everyone has the same rights and powers and resources, and the only ways that change are either through democratic acceptability as with policing by consent or through an actual dictatorship/police state scenario. And for reasons 1 and 2 above, it is highly unlikely we would ever actually reach the latter, which means the only way the police and security services get to keep doing what they're doing (and the only way the politicians get to keep their jobs) is by maintaining sufficient public support and trust. As we've seen in recent elections, there isn't a lot of that going around lately, so it seems unlikely that anyone wants to rock the boat unnecessarily.

Finally, it's worth remembering that we're talking about Britain here, which means the police very much isn't a paramilitary force in the way that many other countries have. Neither our citizens nor our police routinely carry firearms, for example, and while there are certainly specialist units in the police that do have that kind of equipment, they don't have anything like the numbers or equipment to put down a serious popular revolt. At the end of the day, there are still only 150,000 or so people working for the police, out of a population of well over 60,000,000.

This is why I think we're still a very, very long way from the kind of brutal dictatorships and police states that some people in the world still tragically live under today, and why I think despite the flaws of our various government organisations and the need to be mindful of their power and restrict it appropriately we are still a long way from any point of no return in terms of the balance of power between the state and its citizens. We've just let paranoia after a few high profile but actually relatively minor incidents tip the balance too far towards authoritarian tendencies in recent years -- which is, after all, why it's called "terrorism". At some point, we as a society are going to realise that this just plays into the bad guys' hands and stop putting up with it, and the balance will tip back again.


> If very bad things start happening, none of these organisations is going to universally side with a government that is doing those things.

If anything history has taught us that this isn't true. Oragnisations like the police and army of any dictatorhip get swept up in it and become part of it.


Oragnisations like the police and army of any dictatorhip get swept up in it and become part of it.

But we aren't starting from a position of dictatorship in the West, nor do we have things like an army formed primarily from conscripts or a police force whose members live in fear of some secret police force hurting their families if they don't comply.


Godwin. That is all.


[deleted]


Either they are doing well but not publicizing, or that the threat is highly exaggerated and that they are really doing nothing.


Or, probably, somewhere in between. It seems likely both that the government publicity around the terrorism theme is vastly exaggerating the true threat but also that some level of danger really does exist and these agencies really do prevent some attacks from taking place.


The question is whether the true risk level is acceptable? If it is then we don't need any of the surveillance (which in and of itself is a massive risk to our democracy). The other question is whether the surveillance actually reduces the terrorist risk?


Exactly. These are the kinds of questions we should be asking.

If those with access to the relevant information were willing to give reasonably realistic and honest assessments of the risks involved, then everyone else could make their own judgement about what levels of risk they were willing to accept and our governments could act accordingly.

Of course, the problem is that with such obvious asymmetry of information and power, and with the biased perspective that often seems to result from working in an environment where your job is to deal with the worst of humanity every day, it's rather unlikely that we'll actually get an honest and realistic assessment of the situation and the true risks involved (both from the bad guys and from any unintended consequences of the measures that are supposed to protect us from those bad guys).


We elect our representative (partly) for this reason. One of the advantages of a parliamentary democracy is that members of the parliament could in theory be cleared for full access to information and then could in theory provide the needed oversight. One of the issues for me about the way that GCHQ in the UK and the NSA in the US operate is that this democratic oversight has either been absent or has been ignored. Furthermore the massive data collection programs that these agencies run will allow them to manipulate politicians (which I assume is already done). Also the oversight measures that have been proposed so far are in the UK are totally laughable.


A fat paycheck and job security can do wonders in solving those pesky cognitive dissonance issues.


there's no fat pay cheques at GCHQ, they're well known for paying 2-3 times less than the private sector: http://www.gizmodo.co.uk/2015/05/gchqs-government-sancitoned...

it really makes you wonder what sort of people their roles appeal to...


I actually know a few people who work there. All well educated (ex-Cambridge), but the one thing they have in common is that they are very religious. I'd be interested to know about the prevalence of strong religious beliefs in their staff in comparison to people with similar levels of education in the general population.


were they Mathematical Tripos graduates out of interest?

almost all of the mathmos I knew really thought GCHQ would be an amazing place to work, and about 50% ended up there, with the other 50% at quant funds (more or less correlated with religion).

this was of course prior to the Snowden disclosures, I'm not sure it's such a hot place to work these days...


Yes mostly, with those who weren't coming from Physics/NatSci.


OK, that's weird, I thought my sample was just biased. Know several who work there, all are very religious - one was a priest briefly after university before going to the dark side.

I suppose the only way you can do this work is by being delusional, they likely seek the quality out.


Utah? The NSA hire Mormons because they do as they are told and believe in stupid things. So you might be onto something.

I grew up in the shadow of the doughnut (it was sheds then) and they recruited 'second 11' folks from my school, the ones that didn't get A's. The buy in is that they can pay a mortgage, brings some more kids into the world and not have to leave Gloucestershire. They are all kept in little boxes so none know the big picture, except they believe they are protecting the world from Osama/kiddie porn.

Although no hard evidence, I believe the doughnut people also have everyone's school reports too. Imagine the lengths or procedures in place to do that. Yep, the spy on kids.


You're actually onto something that's very insightful indeed.

Religious zealots fit perfectly the psychological profile of the typical alphabet agency stooge.

They're dogmatic, conformists, traditionalists/conventionalists, authoritarians by nature (more specifically RWA[0]), malleable to chauvinistic rhetoric and propaganda, averse and hostile to opposition's rhetoric and dissenters.

That explains why these folks are attracted to these positions and why those spy agencies seek them aggressively and make no mistake some of them are very bright and innovative but unfortunately their brain is severely compartmentalized and the one chamber holding the religious and nationalist stuff is extremely deficient in terms of mental capacity and reasoning.

[0]: https://en.wikipedia.org/wiki/Right-wing_authoritarianism


A job for life and a final salary pension? You can't get that in the private sector at all.


That's true. Truer than people would think. The funny thing is, I've seen the very people who were passionately against these things join these organizations, because they pay well. I don't entirely fault them -- gotta put food on the plate.


I don't excuse them in the least, there is no one working a programming job that can't afford to "put food on the table" -- independent of their employer.


> there is no one working a programming job that can't afford to "put food on the table" -- independent of their employer.

Okay, see, you are just wrong there. I personally know about 5 people who are crazy good programmers (systems programming and all) who have been unable to find jobs. They say they have trouble finding employment because of their age... time commitments (can only put in 40 hours, need to take care of kids), etc. It absolutely is the fact that some of these people take these jobs because they pay competitively and meet with their requirements. I know this because I personally know some of them and they told me exactly this.


I'm sorry, but while I can believe that they may have some hard choices to make (moving somewhere else, accepting a pay cut, having to go back to school, taking out debt), it doesn't excuse the damage they're doing to society. People like the ones you're describing do have options, they may just not be ones they like as much.


Tell those people they are contributing to the downfall of modern, civilized society, by participating in a secret state within the state, over which no average citizen has any control. They are creating a neo-feudalistic society which, in just a few generations, may well be the motivation for severe violent uprising and revolt from within. No society that ever perpetuated a "surveillance class" ever survived long. Instead of more repressive technology, give them a dose of history.


And yet here in the US where there is similar, the NSA is viewed as a refuge for the "best and brightest" and "most patriotic". People nod approvingly when they learn someone works at the NSA, even in light of the disclosures.

Who has the courage to shame these people to their face for collaborating in our oppression? It's easy to wax poetic and spit fire on the internet, but it's much harder to actually shun someone standing in front of you.


Really? In my circles, anyone working for Palantir (much less the NSA itself) is considered disgusting.


It helps that British geeks love James Bond and Alan Turing, and at GCHQ they get to play at being both - although you'd have to hope some of them have wondered if this time round they might not be the unequivocal good guys.

Also, if you get a reputation for hiring the brightest and best, you also get people who've spent so long being told they're the brightest and best that they may have little concern for whatever lesser beings get spied on, shamed, discredited, disenfranchised during their fun intellectual challenges.

Right, I'm on a list somewhere now! Hi echelon.


Aren't they more like glorified filing clerks?


GCHQ have a brainwashing program that is designed to keep their staff in line with the status quo. If you so much as step into an interview, you have joined their cult.

And it IS a cult, make no doubt about it!


I'd imagine many working on this won't tell you their real job. For example when you apply for MI5 stuff in the UK you cannot tell people your actual job.

God knows how many there are doing these jobs.


I don't personally know anyone who respects the NSA like you described.


I think that really depends on your circles. I can't think of anyone I know who trusts the NSA or looks at an employee of the NSA as a role model by default.


I speak only for myself, but I recently met someone who works for the NSA, and I told him off like I've never told anyone off before. It was my goal to do my best to break him as a person as much as I can with words.

People who work for these agencies deserve no quarter, and they serve no useful place in a free and open society. I don't view myself as a 'mean' person, and I probably wasn't particularly effective when abusing this guy, but I like to think it's the thought that counts.


> It was my goal to do my best to break him as a person as much as I can with words.

Really? Wow. I am concerned at the lack of human decency, the unwillingness to at least conduct a civil dialogue about his point of view. "Breaking" a person is really quite manipulative, harsh, and implies their existing person and personality are 'wrong', and I am concerned that is even considered as a valid action. Even the devil deserves his fair day in court.


Why should he have a civil dialogue with someone who's working 40 hours a week to exfiltrate his personal private communications without cause? Do you sit car thieves down and explain how not having transportation for a month really hurt your feelings?


> fair day in court.

FISA? In my dealings with sociopaths, or merely the deluded, nothing sets the record straight like a public calling out on their bullshit.

The softly softly approach with these enablers doesn't work quick enough, unless you meet them regularly. You want the experience to stick.


Wow. With that approach, I fear that a lot of innocents will be hurt on your path to 'justice'. Whatever happened to 'It is better that ten guilty persons escape than that one innocent suffer'?


Why are you conflating a traditional principle of jurisprudence with the right to free speech? These are entirely separate topics.

The point was to publicly challenge the people that work for the surveillance-industrial-complex. That is simply free speech. The other party has the right to yell back if the desire, or to simply leave[1]l.

    All tyranny needs to gain a foothold is for people of good conscience to remain silent.
I fear that a lot of innocents will be hurt because of your support that you give by choosing to let these problems pass without challenge. Why should you expect anything to change when you withhold the necessary feedback necessary to correct bad behavior?

There is a difference between crude or vulgar name-calling/insults and challenging someone with language that is frank, strong, and maybe a bit rude. One is just trying to make people angry and the other is seeking to throw a bucket of cold water onto someone's naive or harmful point of view.[2]

[1] Free speech doesn't guarantee you an audience.

[2] Failure to understand this distinction is why some people perceive Linus Torvalds as being rude and "foul mouthed".


They changed it to "It is better that an entire country lose its right to privacy and security than... well, anything. Seriously, this shit is awesome, look at what my stupid neighbors are talking about on the phone"


An individual is not a court, and the courts are not holding the people responsible well, responsible.

I don't understand why someone would choose to say that the NSA or their workers are innocent, as we have mountains of evidence to their obvious malfeasance.

There is 0 path to justice in holding our government accountable, so welcome to the impotent rage of the masses.


A tongue lashing and a lashing are different. Is your day in court actual or metaphorical? Pick one instead of playing word games.

You are mistaken if you think people should only be taken to task within a court hearing.

In any case these guys are never going to court.


Doesn't it really depend on what the ten were guilty of?

Jaywalking, sure. Homicide, not so much.


Punishing murderers doesn't actually solve any problems unless you suspect them of continuing to do it. And punishing innocent people creates massive problems.


Anyone who promotes the idea that total surveillance over the entire population as a valid means of keeping order deserves not one iota of human decency - for they provide none at all to their fellows.


That you think so is very concerning, everyone deserves human decency. Your views sounds dangerously close to zealotry, the same sort of views that have harmed so many in the past...


It is the surveillance zealots you should be worried about, not those who would disallow the conditions necessary for them to continue to deprive us all of our rights.

Total Surveillance is a Crime Against Humanity.


Well if the threat is actually that severe maybe you shouldn't rule out physical violence? And who will enforce this oppression, the cops so we should hate them. And who will finance this future crime, the banks so we should hate anyone that works there, even the tellers. Embrace or deflect, no good comes from hate.


I don't support physical violence. But I do support shunning those who support the creation of the Total Surveillance State.


[deleted]


"I work for the NSA." Doesn't give me any useful information about their duties or role in the organization, but I know where they work. I imagine cover stories are more important when dealing with human intelligence assets (spies undercover).


I concur. I once knew someone who worked for GCHQ, making hardware devices. Once it was revealed to me the extent to which he was involved in producing these heinous devices, I lost all respect for him - especially when his authoritarian side came out and he justified his continued involvement by implying that the lives of my children were at stake because "secret reasons I can't tell you about".

NO! THIS IS NOT HOW A FREE SOCIETY WORKS!

The corruption of government begins with its secrets. A truly free people keep no secrets.

If you continue to justify this corruption, you are encouraging the downfall of Western society. We did not attain the heights we have through secrecy and class warfare disguised as security theatre.

We must discourage involvement in these organizations to the same degree that we discourage teenagers from going on jihad. It is truly the same degree of bigotry and intolerance which allows such machinations to persist in our society - on the one hand, violent extremism. On the other hand: covert extremism.

My former GCHQ-supporting associate recently left, to start his own hardware company in an unrelated (non-surveillance-state) field. As much as I appreciate his design and skill at producing appealing devices (synthesizers), I strongly boycott his business. We must punish this "secret clearance class" of society with utter disdain, disrespect, shame and discouragement. We must not let future generations rise to assume that this totalitarian/authoritarian control system is the norm - it may be too late, but nevertheless, civil discouragement must be perpetuated in light of the total failure of our supposed democratic institutions to protect us from this covert violence.


Wouldn't it be fantastic if there were a coup in the CIA/NSA/GCHQ, etc. of all tech people who just started deleting data on a massive scale surreptitiously. When the dept. heads were like, "WTF happened to the numbers on all the porn viewers...they just kinda shrugged their shoulders and said...'dunno..guess people got bored with porn'"...

Replace porn with '*' and now we're really cooking...


Meh, it's fairly inevitable. Systems are only getting bigger and more competent. This kind of stuff requires resources and some solid thinking about algorithms and data structures. No doubt they could publish some amazing papers. But I don't suspect they have revolutionary breakthroughs ala the NSA with crypto.

I've written a small-scale system for VoIP calls, to archive and index all network traffic - it's amazingly useful for debugging. Several years back, on a single quad-core machine with 1 disk, I was able to handle 5TB (several billion messages) of signalling data per day - indexing, archiving, searching. I'm nothing amazing. (I went to SF thinking I could somehow turn this indexing system into amazing profit. First guy I showed says "oh yeah I recognize this, go get <some intro db textbook>". Oops, so much for breakthrough ideas.)

A team of me-equivalents (with some that _have_ gone to school) and a nice budget could end up designing something like these spy systems. They aren't the Manhattan Project requiring new knowledge no one in the world has. I'm no 3-sigma intellect. There are dozens of millions of people more intelligent than me. Complaining about it and shaming engineers will not work against such critical projects.


> But I don't suspect they have revolutionary breakthroughs ala the NSA with crypto.

I wouldn't be so sure - GCHQ recruits heavily from Cambridge and Oxford, and I once met a Mathematics PhD who told me in the UK his two main options for employment were academia or the service. They have a lot of crypto talent there and they're not working on the big data systems talked about here.


Yes I only meant in context of this program which seems so reprehensible. I don't think anyone has anything against them doing crypto.


> But I don't suspect they have revolutionary breakthroughs ala the NSA with crypto

Besides inventing public key cryptography?


Sorry for poorly made comment. I meant to say that creating a data collection system won't require such breakthroughs. The GCHQ obviously has tons of amazingly bright people. I'm only talking about a data-collection system, and that it's not too hard, given the resources, to create.


Privacy issues aside - after all, these programmes have scant interest in the average citizen - these look like fascinating projects to work on from a technical point of view, compared to most software development jobs. And it's surely a more ethical pursuit than, say, writing control software for missile targeting systems.


It's a pity they have a hiring booth at jQuery UK, which is otherwise a great conference.


Really? Do you really expect intelligence organizations not to do this? This is what we pay them to do. International relations is anarchy. We shouldn't be blaming GCHQ we should be lauding Snowden while building surveillance hardened systems.


"This is what we pay them to do". Who do you mean by we? Sure, we pay tax, but I don't remember ticking the "I'd like to sign up to pay for GCHQ to monitor people porn habits" box on my tax return.

If the government were building a device to blow up the moon, would saying "that's what we pay them to do" (due to us paying tax) be a valid excuse?


This is a very naive viewpoint. States around the world are going to tap communication cables. It is not blowing up the moon, which is a silly comparison. States rely on information to make decisions, and outside of their borders with not restrain themselves. We can ask them to change but they won't. We're better off just encrypting our systems.


Really? You expect plumbers not to flood tower blocks and drown everyone in them? Plumbing is what they do!!1


Yes, I actually do expect them not to be the kind of morons who could decide it's smart to hoover, aggregate, and store a toxic mountain of data which massively undermines personal and state security.

There's a reason for external oversight and governance - you don't need malice in a large organisation to get a really shitty outcome, just a bunch of cogs in the machine each chasing local optima.


It does seem rather unwise to collect data if they can't protect it.

Didn't the US government organisation responsible for security checking millions of employees and presumably storing all kinds of sensitive information about them in the process just have a huge leak?

I have a sense of Schadenfreude about that particular incident from the point of view of a government collecting too much data and itself becoming the victim, but when you look at it on the level of the individuals involved and presumably also their family and friends in some cases, it's pretty hard to see any real upside to that kind of result.


Still somebody makes a decision to contribute to these systems. Most of those people think they do the right thing I believe. It is important to show that this kind of surveillance is clearly wrong and painfully shameful to be part of.


It sucks that it has come to this. I am afraid to do certain types of searches. The other day, I wanted to learn more about "Azan" -- the Muslim morning call to prayer that is often broadcast over loudspeakers from a mosque. I find the topic interesting and started poking around, but it landed me on several websites that made me start to feel uncomfortable that I was going to incite some trigger. I'm betting most people in my neighborhood aren't searching for this information, and I don't really want to be flagged. Pre-Snowden, I wouldn't have given it a second thought and thought it ridiculously paranoid. Now, I am very careful how I use the computer when it is connected to the wider network...and I think this is really sad and Orwellian.


I have two takeaways from this piece:

1: We should never assume any online activity isn't being tracked by someone.

2: The above doesn't give the government a pass on due process when citizens choose to avoid tracking by encryption or otherwise.


Except where they have a warrant and reach their dirty little fingers into certificate authorities. Unless you're doing key exchange yourself I would assume nothing is truly private.


They don't need warrants. They'll just hack in like they did with Belgacom.

And even if they don't, they won't get a warrant. They'll get a certificate from the relevant politician that'll authorise them to do it.


"HERE WAS A SIMPLE AIM at the heart of the top-secret program: Record the website browsing habits of “every visible user on the Internet.”

Jesus fucking christ.


Heh, not trying to be pendantic because I actually thought that first sentence didn't make much sense.. there is a graphic T on the left, so it actually read "There was a simple aim"


Slightly off topic, but recently watched Citizenfour - great documentary about Snowden and why he chose to reveal all of this information. It has real footage of Snowden himself right before the leaks, communicating via PGP with journalists, etc.

Even though I already knew most of the stuff, rehashing it all over again makes it so much more impactful. Overall, highly recommended for anyone who cares about our world now and in the future.

P.S. The government's idea that by somehow ingesting all of this data and effectively spying on everyone collectively will somehow protect us the helpless citizens sounds pretty bonkers (to use an UK phrase) to me. It's all just wasteful money spending as usual.

Stop the scaremongering.


This is just the beginning - the problem we are facing are the digital personal assistants. They are like servants - but their loyalty is first to the corporation that runs the server not to the user. This is where the really rich data will come from. And the data will be useful in all kinds of criminal investigations - there will be more and more legitimate cases and there will be no will to limit it.

https://medium.com/the-wtf-economy/we-ve-got-this-whole-unic... - see how useful the assistants can be - people will use them


The solution is for everyone to mess with their signal-to-noise ratio. Make alts. Tell lies. Use VPNs. Encrypt trivial conversations. Use steganography for real secrets. If they like the job security, give them their damn job security by giving them nothing else.


Alternately, you can take the attitude that you will practice radical transparency in your online and offline preferences and habits, so there's nothing to potentially blackmail you with because you live your life unashamed.

I mean, personally my initial reaction when I first heard about the early Snowden leaks two years ago was "welp, hope the NSA enjoyed reading all that Tony Stark/Bruce Banner fanfic as much as I did."


If this the case for you then please walk down the street naked whilst reciting your banking password... I doubt you want to do that. Everyone has something to hide.


I wonder if Radiohead would be disgusted to find that their song title had been appropriated for this kind of alarming surveillance.


They approve...


"Black Hole contains data collected by GCHQ as part of bulk “unselected” surveillance, meaning it is not focused on particular “selected” targets and instead includes troves of data indiscriminately swept up about ordinary people’s online activities. Between August 2007 and March 2009, GCHQ documents say that Black Hole was used to store more than 1.1 trillion “events” — a term the agency uses to refer to metadata records — with about 10 billion new entries added every day. "

I wonder if this "black hole" is backed by a huge HDFS cluster? When I was working in government consulting a few years back I used to alway see job posting at Fort Meade for Hadoop experts, and I know the government is (or was) a customer of Cloudera as of 2009. Incidentally, Amazon created a separate cloud service like EC2 for government data a few years back also I believe.

I assume the NSA must be utilizing Hadoop, HDFS, Impala and/or Facebook Presto - or have a system they built internally but never released that can processing and store data at the same order of magnitude.

If these documents are all true and we assume they are currently collecting more, not less data, then they were in 2012, then they are probably storing more data than Facebook/google on a daily basis

EDIT - i guess the might not be storing as much data as Facebook or google because this sounds like mostly text, no images or video. It still must be a shitload of data though! I also realize that GCHQ is not the same thing as the NSA, but I assume they are doing similar things with similar size data sets

Thoughts?


This presentation (GCHQ) specifically notes use of HDFS and Hadoop to process data: https://theintercept.com/document/2015/09/25/gchq-analytic-c...


thank you!


Can you imagine if the GCHQ had a breach and all the dirt on all everyone they spied on suddenly became public?


You won't have to imagine – if NSA and GCHQ continue on their current course, it'll eventually happen. It's only a matter of time.


But Snowden is the criminal.

So do the brits have all my porn records or only the porn I watched in the last 60 days? The article didn't make it clear.


This is also how you decloak users from VPN, first you profile their online behaviour without vpn, track cookies and graph site visits. Then when people use vpn you still see and track their identity.


Worst part of all this? It's not a feature on any of the major british news outlets. Go look for yourself.


Let's make a list of the UK[0] press and see who reports. Correct as of 6pm GMT Sept. 26th. The story is just breaking but only the Daily Mail and Daily Mirror have reported it so far. I'll probably be on a nice list now having made all those searches : ) Advice on how to search The Sun's website welcome.

  - The Sun[1], can't find a search box
  - Daily Mail[2], yes!
  - Daily Mirror[3], yes!
  - Evening Standard[4], no
  - Daily Telegraph[5], no
  - Daily Express[6], no
  - Daily Star[7], no
  - The Times[8], no
  - i / The Independent[9], no
  - Financial Times[10], no
  - Daily Record[11], no
  - The Guardian[12], no
[0] https://en.wikipedia.org/wiki/List_of_newspapers_in_the_Unit...

[1] uh [2] http://www.dailymail.co.uk/home/search.html?sel=site&searchP... [3] http://www.mirror.co.uk/search/simple.do?destinationSectionI... [4] http://www.standard.co.uk/search/site/gchq [5] http://www.telegraph.co.uk/search/?queryText=gchq&sort=recen... [6] http://www.express.co.uk/search/gchq [7] http://www.dailystar.co.uk/search/gchq [8] http://www.thetimes.co.uk/tto/public/sitesearch.do?querystri... [9] http://www.independent.co.uk/search/site/gchq [10] http://search.ft.com/search?queryText=gchq [11] http://www.dailyrecord.co.uk/search/simple.do?destinationSec... [12] http://www.theguardian.com/uk/gchq


Well sites like YouPorn, RedTube and PornTube etc, despite being ridiculously popular, haven't deployed any kind of TLS. They don't care about your privacy. It's low hanging fruit... far too tempting for the likes of GCHQ, with their capabilities, to just ignore.

If you don't want people spying on your porn habits then boycott sites like this. And if you're running a porn site, consider going HTTPS only, turning off access_logs (or purging them regularly), and providing a Tor hidden service.


>Separate programs were built to keep tabs on “suspicious” Google searches and usage of Google Maps.

I always figured virtual geo-fencing of online maps usage was probably a thing.


It would be nice if web browsers had an option to "only send cookies on HTTPS".

Also: "MEMORY HOLE"? Someone seriously approved that as a code name?


Already available, secure cookies: https://www.owasp.org/index.php/SecureFlag


I am aware of that, and it's not what I'm talking about. That has to be set by the site to be effective.

I'm suggesting a global setting in the browser that means it won't send any cookies to any plain HTTP site, regardless of what the site says.

If enough people enabled an option like this, sites would have to move to HTTPS if they wanted to reliably use cookies, which doesn't seem problematic.


That's a setting for webservers to set on cookies.


Does the fact that GCHQ have been able to scrape cookies from Hotmail, YouTube, Facebook, Reddit, WordPress, Amazon, CNN, BBC, Channel 4 indicate that https encryption has been broken?

In many cases of the above sites, authenticated requests (ie those with cookies) are made via https and so they would not have otherwise had access to the plaintext version to extract the identifying usernames/etc.


Do all of those sites only send cookies over HTTPS (or did they at the time the GCHQ document was written)? I know, for example, that the Amazon homepage is sent in the clear - is that sufficient to ID the user?

My gut reaction is that HTTPS isn't generally broken (though is probably susceptible to implementation flaws and targeted attacks to e.g. steal a priv. key) or we'd have seen more direct evidence in the leaked documents. That said, given the volume of documents that remain unpublished (or even unleaked), and the time that has passed since they were written, it's difficult to say for sure.


The trouble with https is that if you control the certificate authorities you can easily beat https. I don't see how the NSA doesn't control at least some certificate authorities and they work closely with GCHQ (even paying the UK money to fund some of these programs).


How many of those sites actually run over HTTPS? Reddit doesn't by default, neither do WordPress installs, CNN the BBC or Channel 4.


The comparison with the Stasi is usually perceived as a Godwin point, but I think it is now a very reasonable comparison. I am amazed and honestly surprised how easily the secret services turned against their own population in the world's oldest democracy. It happens slowly and like the boiling frog we won't realise we are in a totalitarian regime until we're there.


> The comparison with the Stasi is usually perceived as a Godwin point

I don't see how, the Stasi was Communist, not Nazi. Perhaps you're thinking about the Gestapo.


There's a huge economic incentive in having this data available - predict markets, sell or provide predictions to businesses, understand what your citizens are thinking at any point in time... Facebook, twitter, et. al. are all sitting on gold mines as well. Ignoring the privacy concerns (which are substantial) this would be an amazing set of data to work with. I'm not surprised there are people willing to do this.

An interesting article [1] was published in Nature a few days ago. I haven't read the entire paper yet but the abstract claims, "group-level ability to produce complex innovations is maximized when social information is easy to acquire and when individuals are organized into large and partially connected populations." Imagine the innovations and industries that would be created if all the information governments and corporations are sitting on were freely available.

I hope there will eventually be a broad 'free data' movement. If Google and Facebook are secretly pushing propoganda that privacy is dead (as stated in some conspiracy theory I recently saw on HN), and if these campaigns work, then the public will eventually not mind having their data open and available. But they should mind that it's all locked behind closed doors. Data is power! Do we want these massive corporations or governments to have so much power over the economy? Should they hinder social and technological progress by keeping their data unavailable? I have no answers but I think these problems will grow in time.

[1] http://www.nature.com/articles/ncomms9398.epdf?shared_access...


You bring up two very interesting and powerful phenomena: "siren servers" and "social physics".

The first term is used by Jaron Lanier to describe computer systems that, like "sirens" of the sea, lure everyone into their beautiful environs, collect all of the data on them, and statistically model all of this data in ways that effectively master the nature of this individual - her beliefs, her abilities, her emotions - until ultimately that individual is made both completely vulnerable and completely powerless. Lanier's book is intentionally provocative and controversial, while rather intelligent and compassionate in its spirit, and I recommend checking it out: http://www.amazon.com/Who-Owns-Future-Jaron-Lanier/dp/145165....

The second term comes from a talk at Google in 2014 called "Social Physics: How Good Ideas Spread" (https://www.youtube.com/watch?v=HMBl0ttu-Ow) by MIT Professor Sandy Pentland. He describes research supporting the recent Nature publication you link to on social and cultural progress, which concludes that our connectivity - free and fast flow of data on networks, people on roads, goods and services across nations, etc. - is our greatest economic and social virtue. His statistical modeling results really strongly show - "like a law" - that the more connected we are, the wealthier we are as a society.

It's really interesting to consider how we could concentrate on using all of the data and models to make deep positive impacts on our society, while respecting our human dignity. To quote Pope Francis's address to the U.N. today: "Integral human development and the full exercise of human dignity cannot be imposed. They must be built up and allowed to unfold for each individual, for every family, in communion with others, and in a right relationship with all those areas in which human social life develops – friends, communities, towns and cities, schools, businesses and unions, provinces, nations, etc."


This has to be the nail on the coffin. They're spying on our porn. Those are personal details about myself I would never share with anybody and wouldn't want anybody to know. Fine, keep an image of my penis, but knowing details about my porn usage?


Well, most people share more with Google (search text) that with their closest friends. And most of them are just fine with that. Until of course they are in the spotlight. Since that almost never happens, we have this privacy apathy going on.


I have one question. Any idea how they can extract cookie data from encrypted connections?


These agencies have all the incentive in the world to "encourage" popular websites (bbc.co.uk perhaps) to make their cookies particularly detailed and leaky.


Absolutely disgusting.


Absolutely disgusting.


When comparing GCHQ or NSA, who is the spookiest of them all? Which one of them is sucking in more information and who does more with that info ? my impression is that there is less institutional oversight over spooks in the UK than in the US, is that true ?


Snowden claimed that the UK's surveillance operations were a lot more severe than the US, basically due to a much weaker oversight framework.

The US might have a secret court with secret judges but all it takes for surveillance to be signed off on in the UK is the signature of a minister in the currently governing party.


Just to clarify, these are all currently metadata, right? Eg, this stuff is, in theory, not linked to any individual people without a warrant/authority?


They mention mass tracking of cookies from sources like google, yahoo, reddit, youtube, etc. - those might technically count as "metadata", but they are used to track individuals.


I'd be fascinated as to how that's justified legally. Mass tracking has been accepted by the high courts, but the moment you start tracking individual users, you need some sort of warrant or RIPA authority.


>I'd be fascinated as to how that's justified legally.

They don't feel the need to justify it legally, especially when oversight is sitting down with the head of GCHQ over dinner and asking if there's anything dodgy going on.

>Mass tracking has been accepted by the high courts

By the Investigatory Powers Tribunal, maybe - but that's not a real court and it's barely part of the court system at all.


Not to mention that it can either be 'mass' or 'tracking' not both, at least not with the implied meaning that 'no individual is tracked'.

Tracking is by definition of an identifiable group or individual. If you can't be sure that the track you are looking at refers to the same individual throughout its length then you are not tracking, at least not successfully. Conversely if you are tracking you cannot reasonably also claim that you don't know who you are tracking except in the trivial sense that you, perhaps, don't know their name or some other attribute; although you could look it up.


No. Read the article.


Seems like I should turn On open access for my wifi router to pollute these results by running am open hotspot.


So Tor?

Tails?


Absolutely disgusting.


Sickening.


People jumped down my throat, here on HN, whenever I brought up this very fact over the last couple of years so I find it interesting to see it reach the top. I guess we have to rotate targets more slowly over time.


I'm done writing stuff online.

Good luck to y'all.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: