Antimatter's reputation for being incredibly difficult to store comes from the fact that it's produced as individual particles. A superconducting antimatter hockey puck would be much easier to store than a cloud of antiprotons of the same mass.

And yeah, you'll need a way to build gamma ray mirrors before antimatter reactions will push you in any direction (the energy comes flying out isotropically and we can't presently do anything to stop or direct it), but we can cross that parsec when we come to it. :-)

Presumably antimatter would be your energy source, but not your propellant. The gamma rays from a small number of annihilations would heat up a much larger amount of normal matter.

At least for the first generation, you likely also wouldn't be using antimatter as the main source of energy, but rather as a method of initiating some other reaction. For example where in a conventional fission reaction you get a relatively clean split of a nucleus into two halves plus a few extra neutrons to drive a chain reaction, an antiproton will blast apart such a nucleus like a billiard break, allowing fission reactions with much less than a conventional critical mass. A quick burst of positrons hitting the surface of some lithium deuteride would be able to replace a fission primary and make a pure-fusion explosion. Either of these options could be used as either incredibly low-mass nukes for an orion drive or as a light weight reactor for a more conventional nuclear propulsion method. While about 600 times less energy dense than pure antimatter, you're still talking 10 million times better energy density than our best current rocket fuels, while using several orders of magnitude less antimatter.

I would be so nervous to be in the middle of space with an hockey puck of antimatter.

I guess I'd be just as nervous in space with thousands of tonnes of explosive propellant. Spaceflight always operates on the very edge of what's possible, not of what's safe.

Gamma ray mirrors sound like they’d be extremely useful for nuclear power too.

Superconductors pretty notoriously need to be kept cold, which adds another difficulty of cooling the antimatter without touching it

If it was surrounded by a cold mass that it could radiate photons to across a vacuum, its equilibrium temperature would be that of its container.

For now. By the time anyone could even possibly create enough antimatter to matter (heh), critical temperatures should be much higher. The record is broken fairly commonly.

I think that's what the dilithium crystals are for. ;D