There's no reason an ISP can't overcommit a /8 network. There's no particular reason they need to promise that you can reach the IP of another Rogers subscriber. One "instance" of 10/8 per region or household or whatever would work just fine.

It would maybe work, but it would also vastly complicate everything. Debugging and logging, setup, maintenance would all be rigged to support that address reuse.

If you have dynamic allocation of customer IPs, you have to store that data anyway.

Yes, but you need a lot more gateways, jumphosts and general indirection to deal with the different network segments. And you need to customize a lot if stuff, because you can't just log "it came from 10.12.13.14", you need to log the network segment and gateway it came from as well. Non-flat address spaces are hell.

That said, the real solution is IPv6

> but you need a lot more gateways, jumphosts and general indirection to deal with the different network segments

That hardware is already there. You don't think they share one giant switch between 16M addresses, do you?

That hardware isn't already there. They won't share a switch, but they will share routers. IP routing with overlapping networks doesn't really work, so you have to get creative with e.g. DNAT or proxies. Both of which consumes additional resources, even if the routers can do it, you will have to buy bigger licenses, processors will have more load, etc.

Yes, in a simple routing setup, the limiting factor is how many ways you can divide up your address space hierarchically, not some giant 2^N number.

10../8 could be split into three hierarchies each with 2^8 entries:

• 10.X../16: sites (e.g. global offices)

• 10.X.Y../24: on site vlans / individual buildings, typically a broadcast network (although switches use MACs to limit actual broadcasting.)

• 10.X.Y.Z/32: individual hosts

It’s not an enormous amount of space, really, hence IPAM. You could divide it on non-8-bit boundaries. Oof.

With 128-bit IPv6, each ISP has a /32, each client a /48, and each broadcast domain a /64, leaving a remaining 64 bits for clients to just randomly make up addresses as they wish.

That’s still 16-bits of address space to work with when creating networks (just as you have 16-bits for the X.Y in 10.X.Y../24) but all addresses are globally routable and each network can support essentially infinite hosts and without needing DHCP, instead of just 254.

> each broadcast domain a /64, leaving a remaining 64 bits for clients to just randomly make up addresses as they wish.

What's the use of this especially when the prefix is dynamic? Two major cellular ISPs in India hand out dynamic prefix IPs. Are they doing it wrong? Are ISPs supposed to hand out static mac bound IPs for ipv6?

> It’s not an enormous amount of space, really, hence IPAM. You could divide it on non-8-bit boundaries. Oof.

Why not? I do this all the time.

FWIW, you can easily use all but two of the addresses in a /8 if you set your subnet mask to 255.0.0.0. MIT used to run their public IPs this way. The entire campus was one huge switched network. You could get any IP in their /8 and take it anywhere on campus.

Thinking about the amount of ARP alone in that subnet (let alone other types of broadcast traffic) is horrifying...

"smart" switches which don't allow broadcast and direct ARP packets only to the correct destinations can fix this.

Many colleges ran this way in the 90's. I am familiar with several that had flat /16's, no subnetting.

At least it would be bounded by the number of students and staff, ie around 10k / 2^14.

Forgive me, I'm far from wearing a network hat and still struggle to wrap my head around things after 15 years. I don't know if this will make sense.

But seeing as the ISP uses NAT anyway, is there any way to further route the private ranges behind private ranges? I don't know if this is what 'Double NAT' is, I tried searching online to see if this would work or if it would cause all sorts of issues. I'm not too familiar with ISP Natting as my home ISP has always assigned a public IP address.

I'm no ISP network engineer, but at a guess I'd probably look at splitting up customers into some sort of logical grouping (say per state or something) where they all sit behind the same CGNAT infra anyway, and give each of those their own 10/8.

Kludges like that are possible but it puts you into an even deeper circle of hell. The solution is probably something like 464XLAT.

Yes, you can keep NATting the same address space as many times as you want. As long as you have proper network boundaries there's nothing preventing, say, your ISP from using 10/8 for the country, then each province having a router that NATs 10/8 up to your gateway, which then NATs 10/8 for your home network.

But the further you go into the NAT layers the worse performance you'll see, because each NAT adds some latency overhead and more places where things can go wrong.

Definitely. I've seen tethering implementations that put the tethered device on its own NAT, behind that of whatever network the host device is connected to.

Would you use a single NAT for a whole country? I always assumed ISP network infrastructures were regional. Not the least because if you put 10 millions IPs behind a single public IP, you will quickly run out of your 65000 ports.

You can use more connections than 65k, since connections are identified by srchost, srcport, desthost, and destport. You are restricted to 65k NATted connections to a single server's* web site, though.

*Assuming the server is using a single IP.

Good point, but still, there aren’t that many google, instagram or windows update IPs. I can easily imagine more than 1% of the 10 millions people connecting to google simultaneously.

I'm not an expert in this field, but I did come across the carrier-grade NAT range on a very odd project:

https://en.wikipedia.org/wiki/Carrier-grade_NAT

I can't recall, does TCP require that src:srcport->dest:destport pairs be unique, or is there another way to distinguish connections (sequence numberd maybe?)? I guess there are other IP protocols like UDP though...