This is exactly what I came to point out. Varnish was pushed in a very arrogant manner insulting other software that was just plain better. While they were trash talking squid and pretending the kernel could just magically know how to do everything perfectly for any arbitrary userland workload, we were consistently getting better performance from squid.
> No, it isn't. That's very much the point. They have spent all this time realizing that doing it wrong is bad and rewriting it to actually work.
Troll much? Nope. That is not accurate at all. Varnish, with it's current storage engines manages to power sites such as Wikipedia and NYT. This announcement was not about that, it was about enabling caching of really large datasets. Datasets that run in the hundreds of terabytes.
And we did tens of migrations from Squid -> Varnish. Usually it involved scaling down the number of servers to 1/6 whilst seeing a massive decrease in response time. So it kind of surprising that your findings are completely different and somewhat disappointing that you cannot back them up.
I posted them back in 2008 or so when it was relevant. I got the standard "our propaganda benchmarks where we deliberately misconfigured squid show otherwise and that's all that matters" response followed by some irrelevant ranting about how everything that isn't freebsd or linux shouldn't exist.
I am not talking about mmap() vs write() here. That is just ordinary propaganda. There is a lot of powerful ideas at work there. The two that I know about:
That is neither new, nor interesting "engineering". Lots of software that needs significant customization uses the host programming language for configuration.
How is "we used an existing data structure and deliberately misrepresent this as some amazing discovery" an amazing feat of engineering? This supports the notion that varnish's biggest achievement is an amazing feat of marketing.
Regarding the .so configuration: can you show other examples? I find this technique fascinating, as it combines speed, expressiveness and actually ease of use.
About the alternative heap implementation: can you point out who else has described this aggregated-heap data structure? I don't even know if it has any name, but sure as hell it is a good idea.
