Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Bayes' theorem strikes again. Even if the odds of a false positive are only 1 in 1000, if 1 in a million are criminals then out of 1000 flagged people only 1 might be a criminal.


...which makes an 81% false positive rate remarkably good. You've massively whittled down your list of potential subjects and made the job of humans immeasurably easier. An 81% false positive rate would be nightmarishly awful if this system served as judge, jury and executioner, but it's incredibly useful as a means of improving the signal-to-noise ratio of surveillance data.


That would be true if wiggling out of the net of justice when you are innocent were not so expensive. There are a lot of poor people who can't afford the system you speak of.


I wonder how this 81% false positive rate compares to the false positive rate of traditional techniques for developing lists of suspects.


I would love to know this as well. It sounds high but we have no baseline to compare it against - for all we know it could be a massive improvement.


But we're going into this story already knowing that 81% were deemed innocent: already wiggled out.


So how many additional people are actually innocent but still considered suspects?


Hard to guess. This would have to be people who still look like a dead-ringer for a criminal even to a human who compares their face to a database photo, and who are not able to supply ID to prove that it's just coincidental resemblance.

Police have always relied on facial recognition, just not facial recognition carried out by machines. We accept imperfect facial recognition from machines if it is faster, and leaves less work for humans to deal with.


all of them are suspects, you have just not been caught yet


The world is not America. Your criminal justice system might be a Kafkaesque nightmare, but ours is mostly OK.


Both the USA and UK provide legal aid for poor people even if the procedures are different. If you cannot afford a solicitor, the government will help you pay for one, but they probably won't be the best. In both countries rich people get better legal representation than poor people.

Don't make the UK out to be some paradise (legal or otherwise), it struggles with plenty of its own problems.


The UK doesn't have cash bail, so nobody is stuck in pre-trial detention simply because they can't afford to buy their freedom. It doesn't have elected prosecutors, judges or sheriffs. It incarcerates less than a quarter as many people per capita. It doesn't have three-strike laws and uses mandatory minimum sentences only for an extremely limited number of serious offences.

The UK is far from perfect, but most of the criticisms of the American criminal justice system simply don't apply.


We have at least recently ended the appalling practice of non-time-limited police bail.

It still takes 2-3 years to get to trial for serious offences during which time your name and face are plastered all over the papers (at least locally).

Mounting a successful defence can cost 10s to 100s of thousands of pounds.

Fighting off charges of which you are innocent is not fun. If we assume that the UK's system is good enough to just increase the arrest rate and it will all come out in the wash this would destroy many people's lives.


Aren’t people far more likely to be a victim of crime in the UK?


It's very difficult to compare like-for-like, because we have different laws and compile crime statistics differently.

The intentional homicide rate is substantially lower in the UK (1.2 vs 5.35 per 100,000), as is our rate of killings by law enforcement (0.2 vs 30.4 per 10,000,000). The UK may have slightly more property crime (12.2% vs 10% of population victimised), but it's not clear if that comparison is statistically valid.

https://en.wikipedia.org/wiki/List_of_countries_by_intention...

https://en.wikipedia.org/wiki/List_of_killings_by_law_enforc...

https://www.nationmaster.com/country-info/stats/Crime/Proper...


From petty crime, I suspect, yes. Why, because even when criminals are convicted they spend very short periods of time in prison (if at all). And, on release, they have much less chance of integrating into normal society than previously and fall into a pattern of reoffending, going to prison, reoffending, and so on.

In the UK, it seems, almost everyone has a tale about when they were burgled, when they were mugged or when their car was broken into.

Is this so in other countries, I think not?


I have never been burgled, mugged or had my car broken into. The only crime that I worry about on a regular basis is pickpocketing but this is similar to other European countries.


It doesn't seem to be true (some silly news articles notwithstanding) but these things are surprisingly difficult to compare properly.


It depends on what you mean by people.

Are you a WASP? I guess you're safe in US.

Are you black, latin american, native american, foreigner? I'm quite sure you are safer outside of the US.


Let's agree on that just for the sake of argument. Systems change, but once settled facial recognition will not go away. You might find yourself engulfed in hell before you know it.


Just like now.


There were big cuts to legal aid in the UK under the Coalition.


You really need to read up on changes to legal aid in the UK.


I'm well aware of the cuts to legal aid and have lobbied against them. It is worth noting that the changes to legal aid have primarily affected civil law, with the effects on criminal law being primarily limited to means-testing for high earners and an 8.25% reduction in overall fees paid to legal practitioners.

Although it is clear that the cuts have significantly impacted family, immigration and welfare benefits cases, it is not clear that they have affected the fairness of the criminal legal system.


Why should an innocent richer person have to use their life savings defending a charge of which they are innocent? (Of course the poorer person would be found guilty and sent to prison so the system is even more unfair for them.)


They've definitely affected criminal law. The secret barrister's book describes it quite well.


8.25% is significant cut. I wonder if that is inflation adjusted?


Just wait.

If yours isn't already as bad as ours, then just wait.


I'm sure that's comforting to believe, but it just isn't true. America's criminal justice system is uniquely dysfunctional in a multitude of ways, best exemplified by your extraordinarily high incarceration rate. There is no law of nature that dictates that all criminal justice systems inexorably deteriorate into tyranny, it's just a specific and fixable set of policy failures.

The iniquities of pre-trial detention and cash bail are fixable. Racially biased arrest, conviction and sentencing rates are fixable. Needlessly punitive sentences and an over-reliance on incarceration are fixable. Miscarriages of justice due to pseudoscientific forensic evidence and under-funded public defenders are fixable.


Except for the people who are falsely accused and may be hassled in the process.

Around 15 years ago I would often schedule last minute flights while traveling for work. When I did, I almost certainly would receive an enhanced screening from TSA. I was the false positive, and when traveling on tight schedules, I was not amused at the frequency in which I was considered suspect (in contrast to the treatment from the airlines, who treated me like royalty). Of course, TSAs true positive rate is 0%.


Sorta. Mathematically yes. Socially, I dunno. When you have a test determines you to be 21% likely to be guilty but the people judging you see "99.99% accurate test" statistical innumeracy is dangerous. The best fix is to use the useful info in the smart way, but "the smart way" means fixing education problems we're already trying and failing to fix.


> which makes an 81% false positive rate remarkably good.

Friendly reminder that our _entire_ legal system was arranged by the Founders so that innocent people wouldn’t be incarcerated even if that means some guilty people walk free.

Your reasoning flips the most important feature of our criminal justice system on its head.

I know most people on HN haven’t dealt with the criminal justice system but being wrongly charged and, if you’re lucky, beating the charges is an incredibly awful experience. Just being charged will follow you for the rest of your life.


It is incredibly good nonetheless

Remember this is used by police to find suspects, not to replace the judge.

It means that just looking at a picture, 2 suspects every ten are successfully recognized

Compared to the system we have now, were to find who's guilty police have to go trough a list of suspects were all of them except one (or a few) is innocent, 2 every 10 is very good.

We're not talking about considering people guilty until proven innocent, but helping police to investigate in the right direction.

In UK almost a quarter of murder cases are still unsolved.

https://www.ibtimes.co.uk/almost-quarter-all-murders-are-sti...


It doesn't really work that way. Or, at least, you need some more information.

There are two big assumptions you haven't made explicit. One, that the FN rate is low. If it isn't you haven't necessarily improved your SNR at all, you're just spent resources looking in the wrong place. Two, that the cost of a FP is 0. The latter obviously isn't true, so you have to weight the positive and negative impacts across all scenarios before you can tell if this is a net win.

Think of this in a different context; say I have a new cancer screening protocol, with a low FN but also a high FP rate. So everyone we mark as positive gets a follow up biopsy. You say "great, we catch x% more cancers now than the old way, good win". Until someone points out that biopsies have a y% rate of catastrophic infection resulting in death. Depending how these rates balance out, we may be killing more people that we are saving....


Thank you. This is the principle that always gets overlooked anytime I or my company sees these articles. No one should ever be suggesting that these be a sole means of identification. There should always be a human in the loop.

I've yet to meet a customer who's looking at this technology as a way to make decisions. In every case it's there to help make their analysts' jobs easier, by sifting through the chaff and presenting them with the data that makes the best use of their skills.

Note: I work for AWS.


An 81% false positive rate would be nightmarishly awful if this system served as judge, jury and executioner

This is a highly black-and-white way of thinking about the implications. "Heck, it's only a problem if they get executed."


We should be careful before we unnecessarily increase the number of interactions between police and the general public.

For example, roadside drug test accuracy rate is abysmal but innocent people are being put in jail because uninformed LEO either don't know or don't care baout the rate. If you can't afford to fight the false charge, your life/record/job/etc is screwed.

Also, these false positives tend to have a conveniently racist default twist in our country.

"59% of the Houston defendants in drug arrests involving false positives were black.", I expect the number that didn't fight the charge and were wrongfully convicted is quite high as well.

https://www.nytimes.com/2016/07/10/magazine/how-a-2-roadside...


>We should be careful before we unnecessarily increase the number of interactions between police and the general public

I'd say a society is more successful the more they strive to minimize law enforcement contact.


Assuming 0% false negatives. I am not sure such an assumption is backed by evidence.


If you're arrested and then a few hours later de-arrested that will have an impact on your life.

It means you have to explain the arrest on any enhanced DBS you need.

It means you probably longer qualify for visa free travel to the US and will need to ask the US embassy if you have to get a visa or not.


…assuming there aren't any false negatives as well


An acolyte of the conspiracy...

Seriously though, for anyone interested in understanding this interesting statistical principle, check out the "Intuitive Explanation of Bayes Theorem" [1]

[1]: https://betterexplained.com/articles/an-intuitive-and-short-...


Indeed. And considering the hundreds of millions[1] of people who pass through london’s stations each year, that would means hundreds of thousands of false positives.

[1] https://en.m.wikipedia.org/wiki/List_of_busiest_London_Under...


^ this. Reasons why I teach basic Bayes to my first-year law students. If at least one of them becomes a competent defense lawyer...


You might find "reckoning with risk" by here giggerenzer interesting.

He presents useful tools for everyday people to understand this, mostly by converting percentages to real numbers and drawing probability trees.


Yep. The lack of understanding of false positives truly never ceases to amaze.


I feel like saying “its correct 99.9%” of the time should only said for real sample sets of input data.


This might be weighted with a very high risk coefficient


Conviction by human court has a success rate.


First thing that came to mind


"if 1 in a million are criminals"?

NYC population is 8.6 million in 2017, so there are only 8~9 criminals in NYC? Wow.

Come on, you should give a much better estimate.


For each individual crime there probably is only one or a small number of perpetrators. You can't just say... hey we found 1000 (or even 5) of you face matched for a single crime (assault) which you almost certainly (or even probably) didn't commit, but after a search or warrant some of you appear to have committed a different other crime (drug possession, tax evasion, public urination, evading arrest, immigration status) so go to jail and wait for court.

Once police have the right to search, enter, confiscate. and arrest probable cause is out the window.


I don't follow. Do they match faces to crimes or match faces to faces of criminals?


They match faces out of a crowd for criminals, but to be a criminal you have to have committed a specific crime... usually that means a single person. For any one crime, they are looking for a single person and that's worst case for false positive recognition. For a large sample set 50k-1000k, they are lucky they don't match any one person to multiple criminals (birthday paradox) ... that would be obviously disqualifying.

I suppose it could be worse and they just take an aggregate face of all (accused?) criminals or for particular crimes (white collar?) and match against that... hey your skin was brown, or hey you didn't shave... you appear Male.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: