The problem is that the internet we remember from when google was good no longer exists. Blogs are dead. Personal websites are dead. Noncommercial, informational or niche interest websites are dead. Search sucks primarily because there’s nothing worth searching for anymore.
Yes, we can most of us name real websites that we still read & rely on. When did those sites start publishing? How many of the creator-controlled, non-commercial websites/blogs you read began less than say 5 years ago? I bet the number rounds down to 0.
Google search sure as hell played their part in creating this world, but fixing search isn’t going to bring back an internet worth searching.
I can't believe 'There's no good content anymore.' There are more people; there's more connectivity; there are more digital cameras.
IMHO, the thing that changed was the signal:noise ratio.
When Google started, it was, what? 1:10? 1:100? Interesting pages : crap.
Now it's... 1:10,000? 1:1,000,000?
That turns a challenging problem into a possibly impossible one.
Granted, Google bears culpability in this, because at some point they realized: we can make as much or more money if we optimize for ad delivery than user search intent. And once they showed those cards with a wink, content optimized to this for its own monetization.
A certain amount of people probably would have taken the time to learn how to do something like host a web page on shared hosting or even github pages, but don't because it's both easier to upload to a platform, and you get infinitely more reach - sometimes facilitated by the platform (e.g. TikTok often promotes <10 view videos to gauge them, and will skyrocket a post from an otherwise new account if enough viewers interact with the video).
However, if these platforms exist, there's no telling how many multitudes more people wouldn't even attempt to put stuff online. Life is busy and spending even an hour setting up an online presence on the "open web" is too much for most people.
Yep. Most of the non-developers probably couldn't set up a HTML page. And the developers can also be too busy.
Many years ago, I wrote my own homepage in PHP. With every version of PHP it needed to be fixed again, so after a few years I gave up. I tried to install some existing software, but you need to update that regularly, too often. So now I am happy that there is Substack I can write on (despite complaining about its functionality all the time), because otherwise I wouldn't have a blog.
This is a feedback loop though. It is a reflection of the moderation wars in some ways: Google’s main job now is to try to keep you away from malicious sites as much as find good ones.
So they prioritize ‘known good’ sites (big platforms, major companies, etc.) over unknown, individual sites. Then, since it’s now harder to be seen as your individual site (and less work), this pushes people to move to higher ranked platforms. This increases the ratio of noise (the ratio of spam/malicious sites increases compared to good ones) making it worthwhile to skew more towards results from platforms, etc.
Platforms also have the benefit in that it’s a lot harder for a YouTube video to be malicious, giving them more freedom to take chances recommending unknown content. It still could be hate mail, but at least it’s not going to ransomware you.
I kind of hope some walled gardens pop up and close off other parts of the internet.
I still reflect when I go back on tours of 70s / 80s / 90s media, and while I get some of the "oldies tour" is also kind of a "greatest hits" effect, but there seemed to be a lot more creativity, especially wacky creativity, in the past.
Some of that is callous mass market media calculations by "big media": multi-hundred million dollar movies/games aren't going to be off the beaten path.
What is strange is the ARMIES of "content producers" and all they do is produce things in basically the same way.
So the "SEO" optimization seems to clip the wings of all grassroots content production. Google should have enabled the long tail. It ... sort of ... has, but man its hard to find it drowned out by all the mediocre crap out there.
People don't seem to be isolated from the internet and let some path of creativity percolate for years or even decades like what used to happen. Internet monoculture really seems to be destroying interesting creativity more than it delivers novel creativity from around the world.
"Comorbidity" with this is the death of patience and attention for wacky stuff, people click away too quickly.
About the only actual evidence I can deliver is the oppressive nature of the cinema sequel. For some reason, I just mark the first Matrix movie (1999) as the death of creativity in Hollywood.
However, of course, that may be the point in the 18-40 demo that I aged out of the great popular culture recycling machine. I think a lot of that 18-40 demographic number comes from the fact that by the time you hit 40, you've seen all the tricks, and the repackaging of the tricks/tropes just seems cheap compared to the "original" experience you had with the trick/trope.
I also noticed that starting about 3-6 months ago, most of the things I Google have actual results for maybe the first 1.5 pages, and the rest are websites with URL’s like poltgabdismrvdusvetw.xyz with an obviously scraped/gpt description that will try to get you to give their latest cryptominer or cryptolocker a try. The ones that claim you have a virus, or worse - try to get you to buy McAfee.
oh, do you have an example for a search phrase? What I see is that google images results are pretty bad: stocks/pinterest first and just a couple of pages of the results (even for searches that should return millions of matches).
It's just more consolidated into hubs that are locked down in comparison to the earlier version. It was like 100,000 countries all with their own language and culture and people's and now it's like a handful of huge countries and a few smaller but the dropoff is quick. We organized the Internet the way we organized the world. Some Conways law thing going on I guess.
The content creators got robbed of any of their value. Reddit, Facebook etc all sucked up and monetized the content and gave creators nothing in return except a bit of limited exposure.
One issue for me (maybe it's my ADHD) is that there is almost TOO much content. I can't decide what to read. I have so many tabs of blog posts and such that look interesting that I never get around to. There is plenty of interesting content. More than ever before.
Until youtube or google’s algo decides to kill your chanel without much recourse or get nearly 0 visits because the algo expects some kind of engaging formula. These abuses are enough to warrant not ‘posting content’ (I highly dislike this term) on their platforms at all.
But at the same time I understand that a large number of people choose convenience over managing their own hosting.
Youtube is different from the web. You can't just be informative you have to be informative and entertaining.
There are lots of channels that manage that, sometimes it's fun to watch a knowledgable person explain something or review a product purely out of entertainment even if you will never buy it.
If people are informative an audience will find them, if they're entertaining and audience will subscribe to them.
Whatever downsides youtube has for a creator is vastly outmatched by the upsides for most creators.
I would still speculate that the content that gets published will depend on the environment it's being published in.
A smaller, tighter knit community will encourage the sharing of more intimate details. If, however, all the action is happening on larger stages that pull people's attention out of the phpbb's and into the tiktok reels, those conversations no longer happen, even if the volume of communication has increased 1000fold
To offer a prominent counter-example, consider the success of Substack, Ghost, Medium, and other blogs-and-newsletter platforms. Blogs are thriving for quality writers (especially those with unique experiences), as evidenced by the ability of certain writers (including some I subscribe to) to make a full-time living through these platforms.
Outside of blogs, I personally know people in non-technology fields who have their own personal website, to make it easier for people to contact them and view their portfolio of work. They made their through page builders to avoid the need to learn how to program, and they personally found some collaboration opportunities due to their personal websites. Searching for someone's name and possibly finding their personal website still has value.
Though I do admit that there are far more heavily commercialized or clickbait articles nowadays (such as padded-out listicle-style articles that bury the important information, which may also have questionable accuracy), I personally believe there are still good websites out there to find and learn from, which I've discovered through personal recommendations or searches on niche topics of interest.
A major source of well-written personal websites for me have been those that focus on the use of specialized software (such as Anki and SuperMemo) in spaced repetition: I've learned a lot from excellent recently-published, non-commercial blogs found directly through Google searches. I've also found similar very high-quality and non-commercial personal websites and blogs when searching about other specialized topics, such as workflows to use the Vim text editor to create technical documents with LaTeX typesetting.
I have a blog, like the early 2000’s! I’m also working on a book, hopefully I’ll have it done by the end of this year. The blog is called supermemoadventures. I’ve been using SuperMemo for 17 years, every single day. If you have any questions, feel free to ask here.
Cool! Yeah, at a certain point I was looking for help in my flashcard routine but realized not many people have used a program like SuperMemo for a few years in a row every single day, and I thought it would be a good idea to write down my experiences and observations.
I really enjoyed physicist Michael Nielsen's "Using spaced repetition systems to see through a piece of mathematics," as the post lets the reader learn about the thought process behind a researcher who is highly accomplished in a mathematics-heavy field, in both academia and industry (2019): https://cognitivemedium.com/srs-mathematics
I plan to re-read the post as I gain a better understanding of higher-level mathematics over time, but from my first reads, I could relate and better put to words the idea that using spaced repetition to study technical concepts (though I personally always make sure it's not the main method of study, versus practice exercises) can help you better grasp connections between concepts over time.
A statement from the author's article that I still think about from time to time is: "I can’t emphasize enough the value of finding multiple different ways of thinking about the “same” mathematical ideas."
---
From a completely different angle, Nicky Case's "How to remember anything forever-ish" is a fun introductory post for giving an overview of what spaced repetition is, in case a friend or colleague is curious about the study method (2018): https://ncase.me/remember/
---
For a much older article that contains the fundamentals of making good elements/items/flashcards, Piotr Wozniak's "20 rules of knowledge formulation" is an article that I've referred to time and time again. I treat the rules more as guidelines, as I personally don't use images or cloze deletion, but I've improved my retention and understanding of topics by avoiding questions that ask for lists, and by always trying to understand a topic before making a flashcard about the subject (though this can be flexible to a certain extent, depending on the subject): https://supermemo.guru/wiki/20_rules_of_knowledge_formulatio...
---
As a last one that focuses more on the technical aspects of using the SuperMemo article specifically, I applied the information from this article published in 2010 to create a special Google Sheet to make it easier to import spreadsheets into SuperMemo: https://thesupermemoblog.wordpress.com/2010/05/15/exporting-...
Specifically, to import dozens of vocabulary flashcards at once, I open a spreadsheet file (aka workbook) and begin with a main spreadsheet with the English in column A and the translation in column B. A second spreadsheet then contains a copy of the first spreadsheet's information, but appending "Q: " to the column A entries and "A: " to the column B entries.
I then export this as a .csv file, replace the extension to .txt, and then use Vim (though Sublime Text or another text editor equally works well), to replace each instance of `,A` (that is, each instance of a comma that separates each item) with `\rA` (that is, a line break), to fit the import formatting required by SuperMemo.
This process helps a lot for picking up new phrases from newspaper articles and storing them in my records, and then importing them into my spaced repetition software for review without too much typing. I would guess that Anki's import method is easier and more straightforward, but I still do prefer SuperMemo despite the complexity (and lack of mobile access for the version I use), as I prefer the item scheduling. My impression is that in the long run, I have fewer reviews with the same amount of retention, which anecdotally seems to be true so far.
Although what you say is mostly true, I refuse to live in a world where the open web is dead. I run my own blog that does attract a near-zero amount of daily visitors(except of course the countless robots).
Because what is the alternative? Use the sterile platforms provided by Silicon Valley mega Corps? Fuck that.
I will continue self publishing my half baked thoughts on my personal website like it’s 1998. link is in my profile
Works flawlessly with my reader/feed-combination that I call Really Social Sites. RSS needs a new name that winks at the social opportunities, I think. Thanks!
P.S. Space content, always nice to read!
Blogs are dead but YouTube is thriving. I can find endless high quality content on basically any topic. Even compared to 10 years ago it’s so much better today.
I don’t think there has ever been a better time to learn on the internet than today.
Depends what kind of stuff you are trying to learn. If it's tech related, then you probably want text docs. Most of the stuff I want to learn about is physical like cooking or crafts which make video content vastly superior.
It's also not realistically possible to auto generate spam video content like you can for blog posts.
> It's also not realistically possible to auto generate spam video content like you can for blog posts.
It definitely is. If you look for videos just a bit longer then 10 minutes (something to do with YouTube recommendation algorithm) you will notice how much unnatural en spammy conversation and “information” is being told (backed by meme or stock images or just a talking head) before getting to the point of the video (which often is barely 1min of content). Sure it takes more time to actually speak the text and edit the video. But it really feels like generated garbage to fill time which can be easily churned out. Now with the dawn of AI I can only image this getting worse and worse.
Ugh I hate the "obviously padding time with rambling to meet X minutes long" where X = some arbitrary number the YouTube algorithm is prioritizing this quarter. Some things are worth a deep dive with pitfalls laid out in a methodical way. Most things are not.
> X = some arbitrary number the YouTube algorithm is prioritizing
This thing about video length is mostly a myth. "The algorithm" juggles many variables but I've seen no evidence content length is one of them.
There was a very common pattern of rambling to meet the 10m mark because this was a threshold to add a midroll ad to your video and thus make more money.
So your point still applies, it's just purely out of creator greed and not chasing youtube's heuristics.
If the algorithm preferred long videos, you'd be better off with one 10m video as opposed to two 5m videos as it would be pushed harder. Whereas if it was agnostic to video length, the latter would be OK.
Regardless, you're correct that watch time may be a motivating factor in making them stretch content longer than it needs to be though. I don't think they get paid for watch time (it's all about ad impressions or CPMs) but it would positively impact their channel statistics.
Ah you're right. There was a change a while ago that did seem to favour creators that made longer videos over shorter ones, but it's not as cut and dry as watch time = more money
Because not all content creators pad their videos. They often provide links to documentation. You can easily skip through the video and you have the option to watch them on 2x speed which is what I often do.
I wouldn't say blogs are dead. There are still some good ones(swyx.io, simonwillison.net, eugeneyan.com, etc).
There are also a lot of good content in the form of podcasts and newletters(basically blogs with regular update).
I think naturally practitioners/experts have better insider insight about the industry than full time content creators but they can't dedicate as much time to create content. Lots of youtube channels just try to cater to beginners, abuse clickbait titles, recycle content from other sources, fill the videos with low effort memes.
Videos as a medium are also incentivized to reach certain lengths so they can inject ads into otherwise boring and mundane topics.
As an example, try to look up on YouTube how to replace your car fob’s battery (doesn’t matter what brand) and you will absolutely encounter an 8-10 minute video offered up for what’s probably a 15 second value and something the manufacturer could have provided in a decent 3 panel illustration hosted on a static .html page, but won’t.
>for what’s probably a 15 second value and something the manufacturer could have provided in a decent 3 panel illustration hosted on a static .html page, but won’t.
Right, because you're not supposed to do that yourself. You're supposed to go to your dealership and pay them $50 to change the $0.20 battery in 15 seconds. Do you really think you're qualified to do such a difficult operation by yourself?
I actually got into an argument on Nextdoor once because someone was promoting a battery store that charged "only" $20 for this "service", and felt it was a "good value" and got mad that I was pointing out that you can just buy the battery by itself for less than a dollar and do it yourself.
yes! try to learn anything related to home maintenance and it feels like youtube is your only option. and you are stuck trying to scrub through a 20 minute video to find the 10 second slice on info you’re looking for.
The issue I see with this is while there are myriads of blog hosting services, an equal number of vps providers if you want to have more control, and you can even host your blog yourself in your basement if you are so inclined, there is only one Youtube grabbing much of the money and leaving peanuts for the actual creators.
What happened is that people didn’t want to pay for high quality content. So sites need to rely on ads. Delivering more ads is not aligned with user interests. If we lived in a world where people paid for content, the incentives would have been better aligned.
> What changed is that people want to make money with the web
People always wanted to make money on the web, but the sites of people that didn't still appeared in search results. Now search result positioning depends on the amount of ads you're prepared to host. So, the people who are optimizing for revenue are listed way above the people who aren't optimizing at all and are just creating.
There are times when I've searched for a tumblr page literally by the exact and totally unique name and still not found it on the 3rd page of results, all of which were unrelated garbage.
They've decided that they'd rather piss you off than lose money sending traffic to sites that won't pay to play. It's toxic.
That is true. The initial wave was driven by the sheer excitement of being able to “publish”. I do think that the ad supported model is still responsible for what happened next. Ads encourage optimizing for eyeballs and traffic vs quality. And that’s exactly what happened. The good stuff was buried by the ad optimized garbage everyone is complaining about.
Btw, I’m not dogmatically against ads. I believe they serve a purpose and are necessary. Just pointing out the side effects.
25 years ago, people were happy to post their own personal webpage on Tripod for free and look at how many hits their hit counter got, and get thank-you emails from people who liked whatever they posted.
Now everyone wants to get paid for whatever they produce.
I am not sure about “Blogs are dead. Personal websites are dead.” because I find a wide range of interesting blog articles and personal web sites from people I follow on Mastodon and X.
I broadly agree with your points, and yes we can fight back by spending most of our consumption time on personal web sites and blogs. Long form reading feels better than doom scrolling.
This may feel correct, but it's easy to prove that this is not the primary reason search sucks because searching the commercial internet also sucks. By your logic, finding tech companies relevant to your search should be easy since they by and large still put their general info websites on their own domain. Nope. You still get mostly SEO junk.
If you're still not convinced just look at Youtube search. There's no excuse for that to be terrible, yet it is.
I spend a lot of time on Urbit and there are plenty of blogs and nothing is commercial. They're even working on an Urbit search engine now. Not trying to promote Urbit, just saying that whether the internet is like this depends on how you use it.
Using Google to access the web now, is like using the Yahoo portal to access the internet in the early 2000s.
Google itself is reluctant to reward new sites. They throttle every single site and increase the number of visitors they send you in extremely small increments. They'll do things like shuffle the pages that get traffic on your site without increasing the "daily cap" that you're predisposed to. And when asked about this, their answer is that "no, we don't do this", but all you need to verify this is to create a new site and see it for yourself.
But this is great news for the conglomerates that have been around since early 2010s that can now enjoy being always #1 because Google itself has locked those sites in as "reputable" and "trustworthy". You can only imagine how those sites exploit this for personal gain.
I've been running a low-ish volume site since 2014. It gets a YoY increase in traffic of like 20%. We didn't really do much SEO, at least not consistently enough that it warrants a consistent 20% YoY increase.
Then last year I made a English version of that website. Traffic has been very very slow initially, but slowly and steadily increasing. I suspect if we don't do anything to it for a couple years traffic will just keep increasing just like the original.
I generally have better things to do than to jump onto the cargo cult that is SEO, but the GP's description of how Google throttles web sites is exactly what I observe anecdotally, and is exactly how I hypothesized what Google is actually doing (without doing any SEO research).
Of course, there's not going to be any conclusive "proof" of such claims unless they admit it themselves, or unless you start an antitrust lawsuit against Google.
I definitely agree it's not a good idea to route all traffic to a new site that hasn't "proven" itself through some test of time, but that's more evidence that they're probably doing this throttling.
> Search sucks primarily because there’s nothing worth searching for anymore.
I thought the same until recently. Google doesn't appear to be indexing/prioritizing anything that's old, infrequently updated, or noncompliant with AMP, even if it's an authoritative source.
People blame SEO gaming but I'd sooner accuse them of alignment tweaking (all old/outdated beliefs are purged; my queries were religious and controversial in nature). For some topics, there is still an entire other internet out there they pretend doesn't exist...that starts on page 1 of fucking Yandex.
Not advocating for Yandex specifically (I don't trust them) but...try other options (Kagi, Bing, etc.).
The internet points to useful things, those things are still there. It's just that Google search increasingly prioritises recent over relevant, because recent pays the bills.
Blogs are far from dead. Mass appeal is maybe dead. Niches are alive and well. OSR ttrpg blogs paper wargame blogs (fewer) but still more content than I can keep up with.
And niches have always been the strength, the value of the open web. That you and the 14 other people on the planet with same obscure interest can connect.
A lot of what I liked about the fun quirkiness of the old web can be found on TikTok. Huge variety of things I would not have encountered on my own. Recently: footwear suggestions for the Appalachian trail, an absolutely massive ceramic raku firing, 1x1 meter 3D printer, calisthenics tips, clips of the Twilight Zone, handsfree bareback horseriding, independing one-woman Daily Show-style political commentary, synthesizer tutorials, Chinese underground mall walkthrough (psyops propaganda?), glacial pool swimming, etc. All that with no unskippable ads. There have been some enshittification changes: TikTok live and TikTok shop, but overall, it's still great.
Google was supposed to solve the info overload problem that growing networks produce.
Instead they focused on growing the network and making content creation and broadcast of content free.
So today less than .5% of content produced is consumed according to the UN report on the attention economy.
Whats the fix? Content production needs to be scaled down. A better system is required that decides Who gets to broadcast, how much and when. Bandwith Advertising consumes within such a system must have a cap. Now there is no cap. Ads can be shown on everywhere and all the time. "Scaling things up" should be a political/social decision not a business/technical decision.
The focus must be brought back to solving info overload.
Content targeted / display ads would fix this. By design, they pay sites a premium for quality content, unlike user targeting which pays for quality eyeballs.
If user targeting were banned, we’d be back to economic incentives for content creators that are similar to pre-internet media and publishing.
In particular, you’d be paid for having a consistent measurable audience, which means the money would go to distribution channels that had high quality editors and curation. Today’s ad netwoks funnel that money to clickbait farms.
Only partly cuz Content is easy to copy and duplicate and rebroadcast. All of which the platforms have made free and easy. That has to be addressed.
Cuz One journo surfaces info. 6000 journos copy it and broadcast it and all expect to paid.
The solutions have to built around content reduction cuz right now we are over loaded with millions of people creating nothing new and expecting to be rewarded.
Not true. The 6000 copycat “journalists” aren’t going to be able to find quality advertisers for their chatgpt garbage mill / copyright infringement site.
Today, they get paid the same as the original journalists, specifically because ad auctions are designed to transfer all wealth to the middlemen, and therefore commoditized content creation and distribution.
You can't fix search no more than you can fix social meda because search and social media has grown so big that they are politically important and hence state property.
Once search and social media reached critical mass, the state got involved. The state was probably always involved but they took control when it got 'too big to be independent'. When google and social media showed that they can overthrow governments and affect elections, the party was over.
Even if there was an internet worth searching, google would not showing us that. Google will be showing us what the state wants.
'Beyond a critical point within a finite space, freedom diminishes as numbers increase.' - Dune
In a noncommercial context, "search" and "advertising" are mutually exclusive.
An internet where all ordering of information is done via secret algorithms that are constantly being tweaked behind the scenes, optimised to prioritise popularity and other metrics useful to advertising.
Online advertising services companies formed from high traffic websites calling themselves "tech" companies.
In 1997, the founders announced Google in a paper where they vowed to offer a search engine that could be studied in the "academic realm".
But we soon learned Google lacks integrity. Say one thing, do another. Fast forward. Outside of Google, who actually studies the Google search engine. People without any university education, or at least no need for it, calling themselves "SEO consultants".
When Google publishes academic papers about their internal operations is this done to push search engine technology into the academic realm. No. It's purely commercial, a recruiting tactic; Google is "showing off".[1]
High traffic websites incorporated as so-called "tech" companies have sought to bury noncommercial websites beneath all the advertising-driven garbage and/or to discourage web exploration and development outside their walled gardens.
However, it's still possible to use the internet for noncommercial purposes. High web traffic or market cap does not give any single website the authority to define how the internet can be used.
Arguably it's easier to use the internet today than it ever has been.
The web is not the internet. And the internet is not a handful of gigantic websites calling themselves "tech" companies. In practice, the internet is reliant on non-binding cooperation. As such, no one can own the internet, and no one can control it.
One can try. And that's what large so-called "tech" companies have done. It may have worked in the past and it may continue to work, for now.
But things can change, they will change, and Google could become just like AltaVista.
Contrary to most stories people tell about the mid 90's, I actually liked AltaVista better than Google. When someone first showed me Google I was not interested and I did not switch immediately. I liked comprehensive searching, combing through large numbers of results. Google seemed antithetical to that, focusing only on the "top" results.
One difference between then and today is that no one back then was patiently and desperately waiting for something better to come along, publishing articles such as "The End of the AltaVistaverse". There are people today who really want to get past Google and on to something better. They have been waiting for years. Progress has really stallled.
Yes, we can most of us name real websites that we still read & rely on. When did those sites start publishing? How many of the creator-controlled, non-commercial websites/blogs you read began less than say 5 years ago? I bet the number rounds down to 0.
Google search sure as hell played their part in creating this world, but fixing search isn’t going to bring back an internet worth searching.