For all the talk of "resume driven development" I think most developers - unless they are playing around with side projects and non-essential experimental stuff - prefer to do the simplest thing that will get the job done and keep the site maintainable, and the reason they resort to more complex and advanced frontend solutions is due to onerous requirements from stakeholders for more and more intricate and complex UI. It's another question entirely whether this more intricate and complex UI actually delivers business value or provides a better experience for the end users, but that question is above the developers' pay grade.
I suspect a little of both. Developers do like to play with cool new technologies, and product managers do like to see how spiffy they can make their products.
Having said that, though, I do not think it's a universal truth that all web sites must expand to fill all available functionality, which seems to be implicit in a lot of arguments here and in the highest-rated rebuttal to me.
Is this really a problem with web developers? Or is it with product managers who demand SPA like behavior?