In non-programming math contexts, it's pretty standard for Z to point up. I have no idea why, as it seems to me like the typical way it works in graphics programming is the more obvious extension of 2D plotting into 3D.
As far as I know, in the CAD world Z is typically up/down. It makes sense if you look at the scene from above, like an architect a floor plan - world-space X and Y map to screen-space X and Y, and Z is then the depth. In a 3D virtual world, on the other hand, the perspective is usually of a person embedded in the world, in which case "Y is up" is more intuitive.
Sort of, kind of, but it's a stretch. "Up and down" on a page are not oriented according to gravity, but according to the top and bottom of the page. If you say "Well, we'll make the z-axis vertical because vertical is perpendicular to the plane of the desk, and that's depth", and then you put the z-axis toward the top of the page, you're using two completely different definitions of "vertical" in the same sentence.
I always use Z for up/down even though I'm a programmer. For some reason I just find it easier to reason about. In fairness, my first forays into 3D were extending a 2D top-down game with an "altitude".