The whole "CIA agent" probably comes from "special agent" which is the title for US police investigators, who sometimes do work undercover, mostly on domestic policing matters. The CIA does have special agents, but it's mostly a desk job, and they are definitely not the clandestine operatives of the pop culture idea. (Most US federal agencies have special agents - even NASA has a little Office of the Inspector General.)
Random aside, but ATF field personnel used to be 'Inspectors' and we had a pretty good working relationship during annual inspections and so on.
Sometime in the Post-9/11 era they transitioned to 'Investigators' and the majority of them got a big stick up their rear ends and it has become a trying, adversarial relationship every time they come out.
