> The issue is solely that OG Mercurial was written in Python.
Are we back to "programming language X is slow" assertions? I thought those had died long ago.
Better algorithms win over 'better' programming languages every single time. Git is really simple and efficient. You could reimplement it in Python and I doubt it would see any significant slowness. Heck, git was originally implemented as a handful of low level binaries stitched together with shell scripts.
Every time I've rewritten something from Python into Java, Scala, or Rust it has gotten around ~30x faster. Plus, now I can multithread too for even more speedups.
Python is absurdly slow - every method call is a string dict lookup (slots are way underused), everything is all dicts all the time, the bytecode doesn't specialize at all to observed types, it is a uniquely horrible slow language.
I love it, but python is almost uniquely a slow language.
Algorithms matter, but if you have good algorithms, or you're already linear time and just have a ton of data, rewriting something from a single-threaded Python program to a multithreaded rust program I've seen 500x speedups, where the algorithms were not improved at all.
It's the difference between a program running overnight vs. in 30 seconds. And if there are problems, the iteration speed from that is huge.
To be fair, Python as implement today is horribly slow. You could leave the language the same but apply all the tricks and heroic efforts they used to make JavaScript fast. The language would be the same, but the implementations would be faster.
Of course, in practice the available implementations are very much part of the language and its ecosystems; especially for a language like Python which is so defined by its dominant implementation of CPython.
Fair! I guess I didn't mean language as such, but as used.
But a lot of the monkey-patching kind of things and dynamism of python also means a lot of those sorts of things have to be re-checked often for correctness, so it does take a ton of optimizations off the table. (Of course, those are rare corner cases, so compilers like pypy have been able to optimize for the "happy case" and have a slow fall-back path - but pypy had a ton of incompatibility issues and now seems to be dying).
Python has a JIT compiling version in GraalPy. If you have pure Python it works well. The problem is, a lot of Python code is just callouts to C++ ML libs these days and the Python/C interop boundary just assumes you're using CPython and requires other runtimes to emulate it.
You don't even need to go all V8, you could just build something like LuaJIT and get most of the way there. LuaJIT is like 10k LOCs and V8 is 3M LOC.
The real reason is that it is a deliberate choice by the CPython project to prefer extensibility and maintainability to performance. The result is that python is a much more hackable language, with much better C interop than V8 or JVM.
I've rewritten a python tool in go, 1:1. And that turned something that was so slow that it was basically a toy, into something so fast that it became not just usable, but an essential asset.
Later on I also changed some of the algorithms to faster ones, but their impact was much lower than the language change.
I don’t know if people think this way anymore, but Python gained traction to some degree as a prototyping language. Verify the logic and structures, then implement the costly bits or performance sensitive bits in a more expense-to-produce more performant language.
Which is only to say: that rewrite away from python story can also work to show python doing its job. Risk reduction, scaffolding, MVP validation.
> git was originally implemented as a handful of low level binaries stitched together with shell scripts.
A bunch of low level binaries stitched together with shell scripts is a lot faster than python, so not really sure what the point of this comparison is.
Python is an extremely versatile language, but if what you're doing is computing hashes and diffs, and generally doing entirely CPU-bound work, then it's objectively the wrong tool, unless you can delegate that to a fast, native kernel, in which case you're not actually using Python anymore.
Well, you can and people do use Python to stitch together low level C code. In that sense, you could go the early git approach, but use Python instead of shell as the glue.
Their point was that by offloading the bottlenecks to C, you've essentially conceded that Python isn't fast enough for them, which was the original point made above
Python is by far the slowest programming language, an order of magnitude slower than other languages
One of the reason mercurial lost the dvcs battle is because of its performance - even the mercurial folks admitted that was at least in part because of python
You barely have to try to have Python be noticeably slow. It's the only language I have ever used where I was even aware that a programming language could be slow.
> Are we back to "programming language X is slow" assertions? thought those had died long ago.
Yes we are? The slow paths of mercurial have been rewritten in C (and more recently in Rust) and improved the perf story substantially, without taking away from the wild modularity and extensibility hg always had.
> You could reimplement it in Python and I doubt it would see any significant slowness
I doubt it wouldn't be significantly slower. I can't disprove it's possible to do this but it's totally possible for you to prove your claim, so I'd argue that the ball is in your court.
You must belong to the club of folks who use hashmaps to store 100 objects. It's amazing how much we've brainwashed folks to focus on algorithms and lose sight of how to actually properly optimize code. Being aware of how your code interacts with cache is incredibly important. There are many cases of using slower algorithms to do work faster purely because it's more hardware friendly.
The reason that some more modern tools, like jj, really blow git out of the water in terms of performance is because they make good choices, such as doing a lot of transformations entirely in memory rather than via the filesystem. It's also because it's written in a language that can execute efficiently. Luckily, it's clear that modern tools like jj are heavily inspired by mercurial so we're not doomed to the ux and performance git binds us with.
> You must belong to the club of folks who use hashmaps to store 100 objects.
Apparently I belong to the same club -- when I'm writing AWK scripts. (Arrays are hashmaps in a trenchcoat there.) Using hashmaps is not necessarily an indictment you apparently think it is, if the access pattern fits the problem and other constraints are not in play.
> It's amazing how much we've brainwashed folks to focus on algorithms and lose sight of how to actually properly optimize code. Being aware of how your code interacts with cache is incredibly important.
By the time you start worrying about cache locality you have left general algorithmic concerns far behind. Yes, it's important to recognize the problem, but for most programs, most of the time, that kind of problem simply doesn't appear.
It also doesn't pay to be dogmatic about rules, which is probably the core of your complaint, although unstated. You need to know them, and then you need to know when to break them.
Most code most people work on isn't about algorithms at all. The most straightforward algorithm will do. Maybe put some clever data structure somewhere in the core.But for the vast majority of code, there isn't any clear algorithmic improvement, and even if there was, it wouldn't make a difference for the typically small workloads that most pieces of code are processing.
I'll take it back a little bit, because there _is_ in fact a lot of algorithmically inefficient code out there, which slows down everything a lot. But after getting the most obvious algorithmic problems out of the way -- even a log-n algorithm isn't much of an improvement to a linear scan, if n < 1000. It's much more important to get that 100+x speedup by implementing the algorithm in a straightforward and cache friendly way.
My core complaint is that folks repeat best practices without understanding them. It's simple to provide API semantics that appear like a map without resorting to using hashmap. I fear python style development has warped people's perception for the sake of simplifying the lives of developers. And all users end up suffering as a result.
They died because everyone knows that Python is infact very very slow. And that’s just totally fine for a vast number of glue operations.
It’s amusing you call Git fast. It’s notoriously problematic for large repos such that virtually every BigTech company has made a custom rewrite at some point or another!
Now that is interesting too, because git is very fast for all I have ever done. It may not scale to Google monorepo size, it would ve the wrong tool for that. But if you are talking Linux kernel source scale, it asolutely, is fast enough even for that.
For everything I've ever done, git was practically instant (except network IO of course). It's one of the fastest and most reliable tools I know. If it isn't fast for you, chances are you are on a slow Windows filesysrem additionally impeded by a Virus scanner.
The fact that Git has an extremely strong preference for storing full and complete history on every machine is a major annoyance! “Except for network IO” is not a valid excuse imho. Cloning the Linux kernel should take only a few seconds. It does not. This is slow and bad.
The mere fact that Git is unable to handle large binary files makes it an unusable tool for literally every project I have ever worked on in my entire career.
Takes 21 seconds on my work laptop, indeed a corporate Windows laptop with antivirus installed. Majority of that time is simply network I/O. The cloned repository is 276 MB large.
Actually checking the kernel out takes 90 seconds. This amounts to creating 99195 individual files, totaling 2 GB of data. Expect this to be ~10 times faster on a Linux file system.
—-depth=1 is a hack and breaks assorted things. It’s irritating. No I can’t tell you what random rakes I’ve stepped on in the past because of this. Yes they still exist.
If you’d like to argue that version control should be centralized, shallow, and sparse by default then I agree.
> If you’d like to argue that version control should be centralized, shallow, and sparse by default then I agree.
I get your sentiment, but I know how working with e.g. SVN feels. Just doing "svn log" was a pain when I had to do it. The "distributed" aspect of DVCS doesn't prevent you from keeping central what you need central. E.g. you can have github or your own hosting server that your team is exchanging through.
The main point of distributed is speed and self-sufficiency which is a huge plus. E.g. occasional network outages and general lack of bandwidth are still a thing in 2026 (and remain so to some extent for the foreseeable future).
Now, could git improve and allow some things to be staged/tiered/transparently cached better? Probably, and that's where some things like LFS come in. I don't have a large amount of experience in this field though, because what I work with is adequately served by the out-of-the-box git experience.
Then just do git pull --unshallow whenever you see fit. I normally don't do --depth 1 because cloning repositories is rarely my bottleneck. Just saying that when you need a relatively fast clone time, you can have it.
Git LFS is a gross hack that results in pain and suffering. Effectively all games use Perforce because Git and GitLFS suck too much. It’s a necessary evil.
> This is fun to read but any such galactic intelligence would probably recognize that its predecessor were meat
That does not follow at all. It's _likely_ that life elsewhere would be carbon-based since carbon is so useful and common. It is not a requirement. Silicon has been proposed as a replacement. While not as flexible as carbon, it's pretty close. Silicon-based lifeforms wouldn't be "organic" at all. Even if we just stick to carbon, there are many organic compounds (and lifeforms) that aren't anything close to what we would consider 'meat'.
We are working with N=1. Until we find more lifeforms elsewhere, we can't assume anything beyond basic physics and chemistry. RNA isn't a given. A lifeforms probably needs something that will pass along instructions to their offspring (in whatever form they take). It doesn't have to be RNA.
For a fictional description of a lifeforms that doesn't have RNA, DNA or anything remotely similar, I like to point out Blindsight, by Peter Watts. https://www.rifters.com/real/Blindsight.htm
Consciousness might have actually started today at 7am and, before that, we were all automatons without subjective experience of the world, just going through the motions.
You might say that's impossible, because yesterday you were conscious and you know that, but you can't prove it to anyone.
Epistemologically, this is not a problem that can be solved with "give it time".
I am in a regular company that has fortunately a tech department for in-house software and this is the absolute opposite; we're currently trying to convince leadership that using tokens is part of the game for LLM adoption.
I know at least of a major LATAM company which has dashboards to see AI usage per employee and they will call your attention if you don't use it enough.
> Seems like a big issue is I'm guessing insistence on having this be a solo operation for cultural reasons.
I had to do some stool collection and it took every ounce of willpower and a N95 mask to prevent me from vomiting everywhere. And that was my poop. I think it's more than cultural, there's a strong visceral reaction.
On the other hand, I can pickup my dog's poop no problem.
Having an repulsion for shit is a healthy adaptation. But it seems that for some people they're much more sensitive.
Similarly, it's probably useful for a primitive person to vomit on sight of a familiar person vomiting, collective protection. Definitely a trait to find out before going to space!
The one I've never got is how so many people faint or become I'll when they see blood. Always seemed like a massive maladaptive that should create even more risk in a presumably dangerous situation. If a tiger attacks me in the night and the guy next to me faints because I'm getting eaten, we'll both end up dying.
It seems maladaptive. I faint (sometimes) at the sight of my own blood, and must look away when nurses draw it. I also get queasy when even talking about blood or reading about it. I can't think of any good reason this would be helpful; in fact keeping my cool would be advantageous.
Rival tribe comes and kills Lug and Glug. You faint at seeing the bloodshed. They assume you died. They leave. You live and pass on your fainting genes.
Alternatively it could just be an overshoot of the behavior to recognize that you bleeding is a dangerous situation. These behaviors probably follow some gaussian distribution in their potential "effect" among the population and fainters are on a long tail of that distribution.
> Microsoft is the go to solution for every government agency, FEDRAMP / CMMC environments, etc.
I've been involved with FEDRAMP initiatives in the past. That doesn't mean as much as you'd think. Some really atrocious systems have been FEDRAMP certified. Maybe when you go all the way to FEDRAMP High there could be some better guardrails; I doubt it.
Microsoft has just been entrenched in the government, that's all. They have the necessary contacts and consultants to make it happen.
> Thinking that the solution is a full reset is not necessarily wrong but it's a bit of a red flag.
The author does mention rewriting subsystem by subsystem while keeping the functionality intact, adding a proper messaging layer, until the remaining systems are just a shell of what they once were. That sounds reasonable.
Thanks. That was exactly the plan. Full rewrites are extremely risky (see the 2nd System syndrome) as people wrongly assume they will redo everything and also add everything everyone always wanted, and fix all dept, and do it in a fraction of the time, which is delusional and almost always fail. Stepwise modernization is a proven technique.
As someone who had worked adjacent to the functionally-same components (and much more) at your biggest competitor, you have my sympathy.
Running 167 agents in the accelerator? My gawd that would never fly at my previous company. I'd get dragged out in front of a bunch of senior principals/distinguished and drawn and quartered.
And 300k manual interventions per year? If that happened on the monitoring side , many people (including me) would have gotten fired. Our deployment process might be hack-ish, but none of it involved a dedicated 'digital escort' team.
I too have gotten laid off recently from said company after similar situation. Just take a breath, relax, and realize that there's life outside. Go learn some new LLM/AI stuff. The stuff from the last few months are incredible.
We are all going to lose our jobs to LLM soon anyway.
> I've been involved with FEDRAMP initiatives in the past. That doesn't mean as much as you'd think. Some really atrocious systems have been FEDRAMP certified. Maybe when you go all the way to FEDRAMP High there could be some better guardrails; I doubt it.
I never said otherwise. I said that Microsoft services are the defacto tools for FEDRAMP. I never implied that those environments are some super high standard of safety. But obviously if the tools used for every government environment are fundamentally unsafe, that's a massive national security problem.
> Microsoft has just been entrenched in the government, that's all.
Yes, this is what I was saying.
> The author does mention rewriting subsystem by subsystem while keeping the functionality intact, adding a proper messaging layer, until the remaining systems are just a shell of what they once were. That sounds reasonable.
It sounds reasonable, it's just hard to say without more insight. We're getting one side of things.
Well, part 3 at least explains something I've observed; the platform is incredibly unstable. The same calls, with the same parameters, will often randomly fail with HTTP 400 errors, only to succeed later(hopefully without involving support). That made provisioning with terraform a nightmare.
I won't even dive too much into all the braindead decisions. Mixing SKUs often isn't allowed if some components are 'premium' and others are not, and not everything is compatible with all instances. In AWS, if I have any EBS volume I can attach it to any instance, even if it is not optimal. There's no faffing about "premium SKUs". You won't lose internet connectivity because you attached a private load balancer to an instance. Etc...
At my company, I've told folks that are trying to estimate projects on Azure to take whatever time they spent on AWS or GCP and multiply by 5, and that's the Azure estimate. A POC may take a similar amount of time as any other cloud, but not all of the Azure footguns will show themselves until you scale up.
> $93 billion over 13 years doesn't feel like a great deal
So, around 7 billion a year?
We are at around half of the total Artemis cost just one month after the Iran invasion. One week of this war finances one year of the Artemis program. Do you think that's a better deal?
Compared to the military spending, that doesn't even register. Maybe you should be mad about that.
Are we back to "programming language X is slow" assertions? I thought those had died long ago.
Better algorithms win over 'better' programming languages every single time. Git is really simple and efficient. You could reimplement it in Python and I doubt it would see any significant slowness. Heck, git was originally implemented as a handful of low level binaries stitched together with shell scripts.
reply