Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> Do you update the cluster state more than 1000 times per second? Curious what operation needs that.

Ooo, ooo, I know this one! It's for clusters with more than approximately 300 Nodes, as their status polling will actually knock over the primary etcd cluster. That's why kube-apiserver introduced this fun parameter: https://kubernetes.io/docs/reference/command-line-tools-refe... and the "but why" of https://kubernetes.io/docs/setup/best-practices/cluster-larg...

I've only personally gotten clusters up to about 550 so I'm sure the next scaling wall is hiding between that and the 5000 limit they advertise



> their status polling will actually knock over the primary etcd cluster

Polling the status sounds like a read-only operation, why would it trigger an fsync?


kubelet constantly checks-in to report "I am alive, and here is the state of affairs" so that kube-apiserver can make informed decisions about whether a Pod needs attention to align with its desired state

So, I don't this second know what the formal reconciliation loop is called for that Node->apiserver handshake but it is a polling operation in that the connection isn't left open all the time, and it is a status-reporting operation. So that's how I ended up calling it "status polling." It is a write operation because whichever etcd is assigned to track the current state of the Events needs to be mutated to record the current state of the Events

It actually could be that even a smaller sized cluster could get into this same situation if there were a bazillion tiny Pods (e.g. not directly correlated with the physical size of the cluster) but since the limits say one cannot have more than 110 Pods per Node, I'm guessing the Event magnification is easier to see with physically wider clusters


I see, that makes sense. Thanks for the details!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: