As a musician and drummer for many years... I am putting myself in the "natural rhythm" camp - I never struggled with keeping time, it came easily and I didn't have to work too hard. Now, actually learning to play a drum set, actually getting the limb separation down, took years and years of practice before I could feel really comfortable.
As far as the rest of the rhythm section -- it's the same for any member of the band. You either have it, or you don't. I played with several bass players who didn't have it, and I felt like as the drummer I was working twice as hard as playing with a bass player that did "have it". I knew a bass player who practiced for hours with a metronome, and he could never play an upbeat in the right place. That's something that comes at an intuitive level.
As far as the rest of the rhythm section -- it's the same for any member of the band. You either have it, or you don't. I played with several bass players who didn't have it, and I felt like as the drummer I was working twice as hard as playing with a bass player that did "have it". I knew a bass player who practiced for hours with a metronome, and he could never play an upbeat in the right place. That's something that comes at an intuitive level.