There's no DDoS prevention tool in play here (aside from that being Fastly's business).
Arista's default of having the TTL as part of the hashing function to calculate packet path means that any TCP connection that does not have a consistent TTL when it reaches this (source) ISP's routers can potentially be bucketed onto multiple different egress paths (transit or peering connections) onto different upstream SP's.
Because BGP works on the concept of AS Numbers rather than IP's (and because Fastly uses Anycast, which announces the same set of IP's from multiple POP's), it's possible that packets that leave the ISP network over different transit connections end up at different Fastly POP's, as each transit provider or peer will have a different route to Fastly's AS.
Fastly does not share TCP session information between POP's as I imagine at that scale it's massively prohibitive to do so.
The TCP RST happens because there is no established TCP session at the secondary POP which receives the TLS handshake packet, and because TTL is part of the hash function it will always end up following a different path (and therefore hitting a different datacenter, if routing stays consistent) if the TTL is reduced by 1.
It's generally deemed 'acceptable' in Anycast terms to cause a TCP RST when traffic switches from one POP to another, as this usually happens rarely, in response to changes in the path between two AS numbers.
For things like websites, browsers will usually just retry the connection, and as the path has changed and stays consistent, it starts to work and all you notice is a slight delay.
It doesn't work in this situation because the second packet with the reduced TTL is consistently routed to an 'incorrect' datacenter and breaks the TCP session every time, effectively dropping service to 0.
>It's generally deemed 'acceptable' in Anycast terms to cause a TCP RST when traffic switches from one POP to another...
>For things like websites, browsers will usually just retry the connection
This is not the case. Every single TCP RST you send on an active http connection will lead to an http request failing in a way which causes application brokenness. No browser auto-retries in that case. The user always has to hit refresh to fix it.
At least with the TCP RST it'll fail fast. What's annoying is that some providers decide that you aren't worthy of the RST; then your browser has to wait for like 3 minutes until the timeout is reached — really annoying.
Arista's default of having the TTL as part of the hashing function to calculate packet path means that any TCP connection that does not have a consistent TTL when it reaches this (source) ISP's routers can potentially be bucketed onto multiple different egress paths (transit or peering connections) onto different upstream SP's.
Because BGP works on the concept of AS Numbers rather than IP's (and because Fastly uses Anycast, which announces the same set of IP's from multiple POP's), it's possible that packets that leave the ISP network over different transit connections end up at different Fastly POP's, as each transit provider or peer will have a different route to Fastly's AS.
Fastly does not share TCP session information between POP's as I imagine at that scale it's massively prohibitive to do so.
The TCP RST happens because there is no established TCP session at the secondary POP which receives the TLS handshake packet, and because TTL is part of the hash function it will always end up following a different path (and therefore hitting a different datacenter, if routing stays consistent) if the TTL is reduced by 1.
It's generally deemed 'acceptable' in Anycast terms to cause a TCP RST when traffic switches from one POP to another, as this usually happens rarely, in response to changes in the path between two AS numbers.
For things like websites, browsers will usually just retry the connection, and as the path has changed and stays consistent, it starts to work and all you notice is a slight delay.
It doesn't work in this situation because the second packet with the reduced TTL is consistently routed to an 'incorrect' datacenter and breaks the TCP session every time, effectively dropping service to 0.