Hacker Newsnew | past | comments | ask | show | jobs | submit | john111's commentslogin

You say you monitor the whole web. How is that possible?


I doubt such thing, considering especially some services may require an account in order to read messages, and some are unknown (for various reasons). Also, not all of the communications is done by web some are done using IRC, NNTP, etc.


I think the "web" here is used to refer to the World Wide Web, which I believe is limited to `text/html` [1] — i.e. excludes IRC, NNTP, etc., which would probably just be classified as "internet communication" (as you mention).

It would probably be better if the service were more explicit about what it scanned — after all, 'the web' means very different things to different people — but I think it's safe to say that "scrapable" html served over http(s) is the indended meaning.

[1] https://en.wikipedia.org/wiki/World_Wide_Web


Yes, I did expect they meant HTML served over HTTP(S), although not all documents are HTML and not all internet communications are "the web", and even among those that are HTML served over HTTP(S) and available on the internet, many might not be found so easily. It does need to be mentioned better, because "the entire web" still seems not specific enough; even if it is only HTML over HTTP(S), exactly which documents are found? (I also think it strange they say they monitor "the entire web plus social networks". Most (maybe all?) social networks are web based, and even if they don't, they don't specifically say which ones.) (Still, also note that IRC logs are sometimes available over HTTP (in which case presumably they are still going to be found), but sometimes they are plain text and not HTML. Plain text documents are used for other reasons too.)


Even if it’s just http(s) requests that’s a lot of data to find & crawl. The bandwidth costs are probably insane.


I have a background in scraping from prior projects over the last decade.

Bandwidth is not a concern for projects like this at a lot of hosting/VPS providers.


Data ingress is usually free, which really cuts down on costs when scraping. If you can do everything in-memory, it's surprisingly cheap. The important bit is being respectful of robots.txt files and not overloading small sites with too many requests.


>Your decisions only matter in the case that you're an actual human

Why do you say that decisions matter if you're a human? At least, why would decisions matter more as a human than if you're a Boltzmann brain?


I'm not using "matter" in a very deep way here. As a human, your decisions matter to yourself in the sense that you're going to experience the consequences of them. If you choose to eat an inedible object, then you're going to have a bad time. But if a Boltzmann brain chooses to eat an inedible object, it does not matter because the Boltzmann brain can't execute that decision in reality and is going to stop existing in a moment.


in the sense that we could be "brains in boxes" like in the matrix, one could argue that this reality we experience could happen to a boltzmann brain as well. similar to a dream, in which the dream reality is a result of feeding all the senses with data from within.


An important distinction is that a boltzman brain only "works" for an instant before the harsh environment of space destroys it. There's simply no time for a whole dream matrix reality to play out.


Another Boltzmann brain can pop up, which contains a short term memory of being previous Boltzmann brain. Sensory inputs aren't constrained though. Most probable sequence of such brains should feel like noise all over your senses, I think.


Another Boltzmann brain can pop up with the result of whatever decision the previous brain could have made, the actual decision it made is irrelevant.


Why? The time passed for the Boltzmann Brain won’t necessarily be tied to its real frame of existence i think, it may simulate any timeframe imaginable, depending on its starting conditions.


In fact the simulation of any timeframe would be constrained by physical laws.

For example for a Boltzmann Brain with a similar chemical composition as ours the speed at which chemical reactions occur would limit the speed of experience.

For a Boltzmann Brain that simulates ours with a very different physical substrate the speed of light would still be a limit.


What's the point? How is typing foo::bar better than typing foo_bar?

It seems like you want the language to be more complicated for no benefit. Why not use C++ at that point?


> What's the point? How is typing foo::bar better than typing foo_bar?

You're missing the whole point of namespaces. The goal is not to replace foo_bar with foo::bar. The whole point is that within a scope you can type bar instead of foo::bar, or bar instead of foo::baz::qux::bar, because you might have multiple identifiers that might share a name albeit they are expected to be distinct symbols.

https://en.wikipedia.org/wiki/Namespace


> What's the point?

using namespace kind::ofa::long_name;

or

using kind::ofa::long_name::bar;

is the point.


I really wish we had better definitions for "smart", "cognition", and "intelligence." With our current definitions, all comparisons are purely subjective. Maybe crows are the "smartest" today. Tomorrow it'll be dolphins, parrots, elephants, pigs, or cephalopods. Who knows?


I mean, there are pretty objective things to test for.

Object permanence.

How many things can they keep in working memory?

How many steps can they work through linearly to an objective?

How many dependent elements of a puzzle can be branched off for a single step?

Can they understand symbols as being representative of objects? Of verbs? Adjectives?

Can they understand counting, grouping?

Can they extrapolate solutions from one puzzle to another? How dissimilar can they appear for them to still recognize patterns?

Are they creative? Are they social?


One overriding question may be: Do they need to?

There are things that other animals can do which humans can't, such as an octopus communicating with a grouper fish, two entirely different species, by changing the color and patterns of their skin, to corral and trap a prey (as seen on a segment of BBC's Blue Planet.)

Are humans "dumber" if we use that objective criteria to compare intelligence?


> Are humans "dumber" if we use that objective criteria to compare intelligence?

We have advanced communication with our domesticated animals, especially dogs. Also have some communication with non domesticated animals like primates, dolphins, etc..


Can they abelinize a simplicial resolvent and show that it is an integer cohomology of a given group?


>To get an RSS feed of a single channel’s uploads, paste the channel URL into your RSS reader.

How does this work? How does the feed-reader know to grab the RSS XML instead of the page HTML, if they have the same URL?


The RSS reader just looks for the,

   <link rel="alternate" type="application/rss+xml" href="yourfeed.rss" /> 
tag(s) in the head of your document. So for example, my channel:

https://www.youtube.com/user/leaddadaist

Here's the tag in the source,

    <link rel="alternate" type="application/rss+xml" title="RSS" href="https://www.youtube.com/feeds/videos.xml?channel_id=UCnQRp2dhO1SIp9--CSPmYpQ">


In Firefox, there is an option to customise the toolbar by adding a '(RSS) Subscribe' button to it. This becomes active when Firefox detects that a web page has an accompanying RSS feed. Clicking on it either brings you to the RSS feed directly or give you subscription options if multiple RSS/Atom feeds are found for the web page.


Glad to see the author here! Just curious, how did you know this was submitted?


I got an email about the post, and asked the person where they found it.

I also browse HN quite frequently so would have been likely to see it anyway :)


Can you link to an example? I know that 4x4x4 is thing, but 3x3x4 does seem impossible.



I found this video really interesting. I knew that the current US system, first past the post, has a lot of problems from watching CPG Grey's [1] videos on it. I didn't know that the alternative systems, like ranked voting, had all these other weird problems.

It's depressing, really.

[1] https://www.youtube.com/watch?v=s7tWHJfhiyo


While it's true that every voting system has paradoxes, I don't feel that this makes every system equally bad.

The US has what I would consider a big systemic problem, which is that the first-past-the-post system leads to spoiler effects, and the result is a two party system. When I've talked to some people about this, the response I got was "well all voting systems have problems so we can't fix it without introducing new problems".

But the monotonicity paradox for elimination voting doesn't seem quite as serious. It seems to only be likely to come up when the two major choices are close anyway. If all voting systems are evil, it's the lesser evil.

If the US could implement elimination voting, we could remove a big problem (the two party system) and replace it with a smaller one: an occasional wrong choice between the two major parties. But this can happen anyway, for other reasons, eg one candidate wins the popular vote and the other wins the electoral college.

I'm aware that the Democratic and Republican parties benefit from the two party system, so they might not want this, but it seems to me that this is what voters should want.


I'd like to add that changing the "front-end" of voting (having people rank choices rather than pick-one) enables a whole slew of potential better-algorithms, because the input data is fundamentally better.


This might interest you if you think that elimination voting solves the spoiler effect: https://www.youtube.com/watch?v=JtKAScORevQ


What I think would solve some issues is to have a last option on the ballot that says, "None of the above" and if that wins, then the current president if they reached their term limit, steps down, and the vice president takes over for the next term. IE pretend the head of state has passed away and move everyone up in succession.

Basically it removes the "lesser" of two evils from the equation. Yes one could argue, the party in power could stay in power if every year the party abstains from voting but I don't think that would be a likely outcome because, lets face it. The VP probably wouldn't be a leader everyone likes come the next election cycle. They could also abstain from running in the next election if they choose.

EDIT: Also puts a lot of emphasis on selecting a good VP for either party since they could possibly succeed after 2 terms.


I would prefer a "none of the above" option that is not actually implicitly choosing a specific person.

If "none of the above" wins, then the sensible thing to do would be to re-start the elections process from the primaries, wherein any person who was a candidate in the previous iteration(s) is ineligible to run again. Mid-November to January is more than enough time to re-do the entire election cycle.


This makes sense to me, excepting the timeline perhaps, only I'd also modify it to mean only that whichever candidates were the finalists in the previous iteration are ineligible to run again. In other words, my party may have picked a candidate that the nation as a whole rejected, but it does not mean that another candidate who ran in my party's primary would be rejected by the nation as a whole. Otherwise I think you may be throwing the baby out with the bathwater.


The public story of the primary races is that they select for the most desirable candidate of those running. If the electorate decides that the "best" of those candidates is not suitable, isn't it reasonable to assume that none of the rest could beat the "none of these" option, either? Do you really want to force vote after vote of people saying "no" before trying an entirely new crop of candidates?

You're not throwing the baby out with the bathwater. You're throwing out the unidentifiable goo, the dirt clod, the rotting fish, and the floating turd. The "none of these" result is a clear indicator that there is no baby worth saving in the entire tub full of bathwater.


If we had open primaries everywhere, I'd be more inclined to agree with you here. As someone who lives in a state with closed primaries, the way it looks to me is that the candidate selected as "most desirable" by a party's base is not at all necessarily the one who will be "most desirable" for the country as a whole--especially when combined with the much lower percentages of people who come out to vote in the primary elections, hence my hesitation to toss 'em all out. The ones most likely to succeed are the ones who pander the most to their hardcore base, to the distress/distaste of much of the rest of the population, especially those outside the party in question.

I am however on the record elsewhere as saying that a necessary part of the path to improving Washington is, in fact, to throw them all out, so I won't fight you too hard on this!


You start to run into issues if you don't allow the same candidates to run again. Issues being that you are suppressing the right to run for office. Now, if the party made the candidate ineligible then that's a different story. Also no clue why I was downvoted without a reason.


Issues being that you are suppressing the right to run for office.

Their ability to run for office isn't being suppressed, they ran for office the first time and lost. They could possibly run again at the end of the next term. This is no more suppressing a right to run for office than the 22nd amendment suppresses a right to run for office.


You run the risk of the exact same outcome running every body again.


We have that risk now, yet it doesn't happen.


Personally, I would rather choose a random sample of all politicians within the country to put into power. Over time, this should tend towards a compromise for everyone.

Failing that, each party having a number of seats proportional to the number of votes they received is not that bad of a system.


Can someone explain how the frequency domain stuff works? I've never really understood that, and the article just waves it away with saying it's like converting from binary to hex.


It's a bad analogy. Binary and hex are just different formats for representing the same number. Spatial domain and frequency domain are different views of a complex data set. In the spatial domain, you are looking at the intensity of different points of the image. In the frequency domain, you are looking at the frequencies of intensity changes in patterns in the image.

A good way to develop an intuition for the fourier space is to look at simple images and their DFT transforms: http://web.cs.wpi.edu/~emmanuel/courses/cs545/S14/slides/lec... (3/4 of the way through the slide deck).

This analysis of a "bell pepper" image and its transform is also helpful: https://books.google.com/books?id=6TOUgytafmQC&pg=PA116&lpg=....

As for why you want to do this: throwing away bits in the spatial domain eliminates distinctions between similar intensities, making things look blocky. In the frequency domain, however, you can throw away high-frequency information, which tends to soften patterns like the speaker grills in the MBP image that the human eye isn't that sensitive to to begin with.


> Spatial domain and frequency domain are different views of a complex data set.

Or in this case, a real data set.


The search keyword to learn about it is Fourier Transform: https://en.wikipedia.org/wiki/Fourier_transform

Along with the Wikipedia article and the obvious Internet search, there's a lot of good stuff that has been on HN: https://hn.algolia.com/?query=fourier%20transform&sort=byPop...


Basically we can represent any signal as an infinite sum of sinusoids. If you know about Taylor expansion of a function, then you know that the first order term is the most important, then the second and so on. Same principle with the sinusoids. So if we remove the sinusoids with very high frequency we remove the terms with least information.


Image and video codecs don't actually use the fourier transform as presented in the article, they use the DCT. Check out the example section on Wikipedia: https://en.wikipedia.org/wiki/Discrete_cosine_transform

The JPEG article also has a very good, step by step example of the DCT, followed by quantization and entropy coding: https://en.wikipedia.org/wiki/JPEG


In the most basic terms, not even talking about frequency, the mechanics of this is that one series of numbers (pixel values, audio samples, etc.) is replaced, according to some recipe or another, with a different series of numbers from which the original can be recovered (using a similar "inverse" recipe). The benefit of doing this comes from the discovery that this new series has more redundancy in it and can be compressed more efficiently than the original, and even if some of the data are thrown away at this point, the purpose of which is to make compression even more effective, the original can still be recovered with high fidelity.


It's the Fourier transform basically. There were even some past links on HN that explained it nicely so you might check those first

(Though for images it's in 2D, not 1D which is more commonly done)


For everyone shocked at the $200k/mo, there is a lot of previous discussion here [1].

Their staff cost about $200k/person, and their total budget for 2017 is 2.9M!

[1] https://news.ycombinator.com/item?id=12541383


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: