It's more that, as I understand it, "fish" isn't a coherent phylogenetic category so much as a convention-based descriptive grouping of certain characteristics. I don't think of whales as fish, no. But an exclusion based on eg tail fin orientation or lack of gills is based on convention rather than strict taxonomic practices. So if someone wants to weigh the characteristics differently and include whales in the term fish I would at least hear them out.
As the saying goes, there’s no such thing as a fish - that is, there is no reasonable definition that both strongly selects for fish was, while rejecting non-fish.
I know it’s not the point, but how does a flounder get categorised in this system? Are they vertical or horizontal?
When they swim it usually has their tail horizontal.
Hmm, my wife likes to swim, I better start calling her my fish.
Whales need to take breaths on the surface, unlike fish, which can stay underwater basically indefinitely. Without that difference, whaling would be a lot more complicated, if not outright impossible, especially in the 19th century - you just can't catch a huge solitary animal in an ocean on purpose if it doesn't have to surface from time to time. Or only with sheer luck.
Even the ancient whalers were aware of the fact that whales need to breathe, and thus, at the very least, could say "well, these are some rather special fish, you know".
You're also of the opinion that whales are fish?