The telemetry gets bad rep, but its impact is almost non-existent. At least for the actual telemetry.
There's some functionality crap, like Bing-search in Start menu, which slows user down, because it makes a web request. But that's an actual feature. Telemetry is just sent in the background, whenever CPU is not busy.
There was a lot of contention when Cortana was being integrated into the start menu directly. They have very temperamental performance tests for first paint on the start menu open. Cortana caused their 250ms (or something to that number) first paint time limit to exceed to over 5 seconds for first paint. After enough grumbling, they eventually decoupled Cortana from the start menu directly. Thankfully.
The telemetry gets bad rep, but its impact is almost non-existent. At least for the actual telemetry. There's some functionality crap, like Bing-search in Start menu, which slows user down, because it makes a web request. But that's an actual feature. Telemetry is just sent in the background, whenever CPU is not busy.