Logo was always regarded as a "kids language" by association with schools and turtles. It was very much a niche even in the mid 1980s when I learned it. Of course it was in reality a fairly sophisticated LISP relative, but programming languages are driven by fashion rather than facts.
Logo was designed to be a children’s language, so the direction of causality between it being regarded as a “kid’s language” and its association with schools and turtles goes the other way!
It used a turtle because Wally Feurzeig, Cynthia Solomon, and Seymour Papert thought this was a good device for teaching about geometry, and Logo was associated with schools (and Lego!!!) because they were trying to change education.
Of course, you are also right in that once it has momentum along these lines, people tend to put it in a little box marked “children’s language” and not build on it in a more general-purpose language direction.
A few folks have tried, but like squeak and pharos and so many other dialects of Lisp and Smalltalk, it has never gained serious traction outside of its original purpose.