contrariwise, i have never come across a case where having tests for private functions/methods has hindered refactoring or bugfixing. that feels more like undue coupling than undue testing to me. if i have to alter or delete swaths of code i just delete the associated tests as well, and write new tests for the new implementation.
perhaps relatedly, i like to build my code from the bottom up, and make sure each layer is solid so that i can use it to construct the next layer. there is often not a public api at all until i am well into the project, so for me that would involve writing fair bit of code with no tests at all simply because it was all private code.
i do use the "public methods on private helper objects" pattern a lot, but there is a fair mix of times where it's the best way to write the code and times where it's just to keep the test framework happy.
perhaps relatedly, i like to build my code from the bottom up, and make sure each layer is solid so that i can use it to construct the next layer. there is often not a public api at all until i am well into the project, so for me that would involve writing fair bit of code with no tests at all simply because it was all private code.
i do use the "public methods on private helper objects" pattern a lot, but there is a fair mix of times where it's the best way to write the code and times where it's just to keep the test framework happy.