It also makes more sense when you realize that 1) pomme in older French meant fruit generally, not apples specifically, and 2) sweet potatoes were introduced to Europe well before white potatoes were. So "earth fruit" seems fitting.