Supply generates its own demand; by making captioning even cheaper, it can increase the demand for transcription services and people to check it over. There are a lot of podcasts and YT videos that could benefit from transcriptions but it's too expensive now.
We have to hardwire architectures because we can't learn architectures yet, or to put it another way, backpropagation doesn't yet work on hyperparameters as well as on parameters. Hyperparameters should be learnable as in theory there's nothing special about them (it's models all the way down!) - a hyperparameter is merely a parameter we don't yet know how to learn - and this has been demonstrated: http://jmlr.org/proceedings/papers/v37/maclaurin15.pdf "Gradient-based Hyperparameter Optimization through Reversible Learning"
"Tuning hyperparameters of learning algorithms is hard because gradients are usually unavailable. We compute exact gradients of cross-validation performance with respect to all hyperparameters by chaining derivatives backwards through the entire training procedure. These gradients allow us to optimize thousands of hyperparameters, including step-size and momentum schedules, weight initialization distributions, richly parameterized regularization schemes, and neural network architectures. We compute hyperparameter gradients by exactly reversing the dynamics of stochastic gradient descent with momentum."
But it's not feasible yet. Once it is, you can imagine collapsing the whole neural net zoo: you merely specify the input/output type/dimension and then it starts gradient-ascent over all the possible models as tweaked by internal hyperparameters.
It could also reflect what is most distinguishable. Which is easier for a NN to confidently distinguish: black pubic hair on black skin, or black pubic hair on white skin? Darker nipples on black skin, or darker nipples on white skin? etc You're doing gradient ascent on confidence of classification, not simply trying to find a plausible input, but the maximal input. There's no reason to expect this to be racially unbiased as there are simple objective reasons that higher contrast would be useful. (Similarly, I would not be surprised if a face recognition NN worked better on Europeans rather than Chinese, for the prima facie reason that they have more variable facial features and other aspects like multiple hair colors other than black.)
And since Yahoo needs to detect porn of all races and there's plenty of black porn out there, it would be odd if their porn detector had such a huge gaping hole in it.
> I would not be surprised if a face recognition NN worked better on Europeans rather than Chinese, for the prima facie reason that they have more variable facial features and other aspects like multiple hair colors other than black
To people who are used to them. i.e. to you.
I'm Irish, and can tell many Irish accents apart. But I've had people from England not being able to hear the difference between Irish accents, thinking they all just sounded Irish.
I've literally been in a group of Europeans in Africa, and Africans mixing up several of the women there, because they were all relatively tall, slim build, with longish straight brown hair.
> Similarly, I would not be surprised if a face recognition NN worked better on Europeans rather than Chinese, for the prima facie reason that they have more variable facial features and other aspects like multiple hair colors other than black
Hair color, yes, but facial feature variability? I think your perception of more variability in European faces than Chinese ones says more about where/how you grew up than about the actual variability. To someone who grew up in an Asian monoculture, without exposure to media with European faces, all Europeans would look alike too.
Err... I'm not so sure about this. I understand that as you grow up in one community, your brain networks/neurons specialize for distinguishing differences in facial features in that community more than in others. While I haven't seen enough chinese to argue, they have lesser facial feature variations, I can imagine how the algorithm defines facial features would be biased more towards the caucasian/European faces. So from the POV of the "facial feature detection" algorithm, European faces will have more variation in facial features than Chinese faces.
Context: I grew up in southern india and spent most of my early years there, but travelled out to the northern India in the mid twenties. I can now say, I can see facial feature variations in the NE India folks(these folks have facial similarity with chinese) I see around now (I currently live in the south)
>To someone who grew up in an Asian monoculture, without exposure to media with European faces, all Europeans would look alike too.
I'm instead sure about this because of a funny incident that happened to my mother just a few years ago. She was standing near a pizzeria when a chinese waitress stepped out and said (translating from italian, sorry if I make some mistakes):
waitress: "Take away?"
my mother: "What?"
waitress: "Take away?"
my mother:"No, I haven't ordered anything"
waitress: "You Italians! You look all alike!"
It's funny how racial bias always comes out in discussion about NN. It all depends on the training data and that's all there is to it. Yet somehow, some people tend to blame the models or the programmers for not correcting this "bias" that is not "supposed" to be there.
If the output of your fancy new NN classifies black people as gorillas it's absolutely the programmers' fault. It is useless for a large swathe of humanity, it creates a PR nightmare, etc.
It depends on how you define "programmers' fault". In this case it would more a case of needing more training data to correct the mistake, which may or may not be the programmer's fault.
Of course it can be the programmer's fault. Programming NNs is not just about picking an unbiased sample set of data, you need to help the NN know what it's looking for.
As I've explained in https://news.ycombinator.com/item?id=12759089, different races have different sets of facial features which are used for facial recognition (because they vary more). Now, when programming the NN, what if you only told it to look for the set of facial features you thought was important? It's very easy to let unconscious biases seep into your code.
> different races have different sets of facial features which are used for facial recognition
I thought that «race» is a social construct not a scientific one and thus can't be relied on as a deciding factor in a scientific experiment or endeavor like face detection.
Actually, no, _some_ humans in certain places / cultures currently perceive and classify race.
The Roman empire spanned swathes of North Africa and had major centres there and traded with subsaharan Africa. And traded with the 'orient' all the way to China.
Their writings don't make racial classifications. They don't even remark much on skin colour or other features.
Visual features were known, but not considered that significant. And later in the empire people from all over the world became Romans via citizenship and what we now call "race" never factored into it.
Yeah, Humans perceive and classify race (accurately or inaccurately) and that's why it can't be a reliable indicator or cornerstone for any reliable scientific system esp. when taking into account the "fluid" or "conflicting" nature of some of the prevailing definitions of races out there.
"The cross-race effect (sometimes called cross-race bias, other-race bias or own-race bias) refers to the tendency to more easily recognize members of one's own race."
This premise is not well grounded in facts as evidence can show that some people esp. racially unaware, uneducated or desensitized don't fall into this category. To support their claim, they cite a US-only study which can't be sufficient to extrapolate for all humanity as a universal fact since we all know that US society is divided along racial line and it's deeply ingrained into people's minds from childhood.
It's like their caste system and with a highly cultural not scientific element to it.
> (Similarly, I would not be surprised if a face recognition NN worked better on Europeans rather than Chinese, for the prima facie reason that they have more variable facial features and other aspects like multiple hair colors other than black.)
You might be surprised to learn that to Chinese people, westerners all look more or less the same. To Africans, white people look more or less the same. Our biological hardware is great at distinguishing minute differences in things (especially people) we are very familiar with, while we automatically start generalizing about things we are less familiar with.
For comparison: a metal fan may be able to distinguish a dozen distinctly different styles of Norwegian Black Metal, while to outsiders it's all loud white noise made by people dressed like undead clowns.
> You might be surprised to learn that to Chinese people, westerners all look more or less the same.
Yes, I would definitely be surprised if Chinese people thought a pale-skinned Irish woman with shocking red hair looked more or less the same as an olive skinned mediterranean brunette.
The fact of the matter is, the greatly reduced range of hair and skin color in the Chinese is a legitimate reason to think they look more "the same" than a European/Westerner. There is undisputably greater diversity in appearance in Europe than China.
The borders of China aren't exactly clear in the image you posted, but here's what I'm seeing:
Europe ranges from 1-14
"Western" ranges from 1-23
China ranges from 12-17
Am I interpreting the image incorrectly? Doesn't it confirm my assertion that China has a smaller range of skin tone?
Edit: The definition of "Western" appears to be very vague, but for reference I referenced at the rough vague shape of the United States to determine the 1-23 figure.
The way I read it almost all of Europe is in the 1-12 range with the exception of Spain. China is 12 to 17 or 12 to 20 depending on how you read the borders. I would eliminate North America from consideration unless we are talking about aboriginals, since it's a melting pot. (That is to say, it goes without saying that it has more variety.)
But my point is only to say that your statement of "greatly reduced range of .. skin color" is a rather exaggerated statement. (For hair color I suppose you might be right ;)
Hard to say due to the lack of precision in the map in the lighter range, but if you think about how incredibly large China is, it stands to reason that your statement might be missing the mark a bit. Especially if you consider other Asian countries. (Which maybe you should, if comparing with "Europe" or "North America"..)
You don't get it. The importance you place on hair and skin color in telling people apart isn't following a set of objective, mathematical criteria, it's a set of priorities based on what you're used to. It is entirely possible someone from a different background thinks of hair- and skin color as irrelevant details, much like you would with a brown and white cow vs a black and white one.
> I would not be surprised if a face recognition NN worked better on Europeans rather than Chinese, for the prima facie reason that they have more variable facial features and other aspects like multiple hair colors other than black
This is false. Each individual has a preferred set of facial features that are used to distinguish faces, and this set is usually dependent on our race and/or where we grow up. People of some races look alike to us because they have lesser variability in this set, but may have more variability in a different set of facial features that our mind is not used to looking for.
A face recognition NN would be biased this way only if the training set was biased, or if it was only taught one set of features.
Given a 1 in 10 ratio of blacks in the USA, I expect a similar ratio in porn. It might be different in South America. The algorithmic preference for maximal differences from another comment sounds like a good argument, anyhow.
I know. I think I met him when Xanadu was being developed. (Xanadu was a pre-WWW technology for storing and distributing hypertext documents. Everything is pay per view in Xanadu. The Xanadu crowd were mostly fanatical libertarians. Everything is a market, with micropayments for everything.)
Nelson's vision was something like a wiki, where everybody paid to read, and anyone could edit or fork the document. You got paid for how much of your stuff was read. Here's part of his original paper.[1] Towards the end of that paper, you can see him describing something like Github, with all the branching stuff, but with a better UI and intended for text documents, not code. This was in 1974. He was way ahead of his time.
He tried to architect a system to do this, and it was insanely complex. It had strong internal consistency requirements, so it wouldn't scale out or parallelize well. It had explicit links all over the place, which was how people thought about databases back then. They were combining the application logic and the database architecture, which resulted in a horrid mess. Today we know to decouple those. Github is built on top of a key/value store. Wikis are built on an SQL database. Works fine.
Eventually, in 2014, there was a working demo of Xanadu. Here's a view of religion in Xanadu format.[2]
What sort of performance can be expected compared to running in the terminal? How large NNs will this scale to in practice? I see a 50-layer resnet is mentioned; but not 1000-layers?
On these demos, I'm getting several seconds on the imagenet inception v3 recognition on an i7 macbook pro (nvidia gpu), on both gpu and cpu modes.
I've built tensorflow for android, running inceptionv3 trained on imagenet and it's much faster, running just on mobile CPU pretty much realtime, around 5fps. On a desktop CPU/GPU it's obviously even faster
You mean with tensorflow or theano as the backend? They have all kinds of optimizations that isn't possible to replicate here yet. There is certainly room for optimization! Also, 1000-layer resnets should theoretically be possible, but probably isn't that practical. Lots of exciting work happening in searching for more efficient architectures.
