What makes this stuff possible now though? New global illumination algorithms? Better scanning techniques? Or mostly better hardware?
I don't know how it compares to other games to be honest. If it's a big jump though, I'd be interested to know what's being done different and why it's being done now.
The big thing people keep not understanding is how big a deal the the PS5's SSD infrastructure is. This engine is based around the idea of being able to grab (very large) assets from the disk only milliseconds before they're needed. In current video games designed to be playable with a HDD (spinning disk drive), they need to make sure that if it's possible for a player to look at an asset in the next 30 seconds, it needs to be already loaded in to the VRAM (the ram of the video card).
The PS5 (and XSX, to a lesser degree) put a ton of work into making is so that data can be moved from storage on the disk into the GPU as fast as possible. It does this through having shared ram between the CPU and GPU, having a fast interconnect and having dedicated hardware so that data from the SSD can be decompressed without ever going through the CPU.
On a PC with PCIe 4.0, you can basically move the same amount of textures to the GPU in the same amount of time as these new consoles, but there's going to be way more latency because data has to first be loaded into the CPU's RAM, then decompressed by the CPU then go through a bumch of software layers to ensure compatibility and sent through the motherboard to the GPU's VRAM and only then can the GPU use the texture to draw something on screen.
The PS5 and XSX built a highway between the storage and GPU that doesn't need the CPU, or compatibility software to be involved at all. That opens the door to a ton of new graphical techniques that used to just not be possible because of VRAM limitations.
The 8K cinematic assets that Epic used in the demo are between 1 and 2MB in size. So I suspect it's more to do with PS5's SSD which has a bandwidth of more than 5GB/s and 16GB of GDDR6 RAM, which allows more textures to be streamed from the hard drive and held in the RAM.
PS5's GPU supports raytracing, which Epic said they didn't use in the demo. So the dynamic lighting must be something new they've developed for UE5.
I don't know how it compares to other games to be honest. If it's a big jump though, I'd be interested to know what's being done different and why it's being done now.