Yeah that would be cool. Even better if it could be done over UDP, so the log messages are fire-and-forget from the client side. Even BETTER if there was a standardized way for all programs regardless of implementation language to write to said socket/log.
Yeah, they thought that up 30 years ago. It's called syslog.
There are some issues with syslog. Fortunately an extension is ready: http://www.graylog2.org/about/gelf - can be structured and is not limited in size anymore.
Tbh, I haven't seen any standard syslog daemon supporting structured messages - except for apple's syslogd and syslog-ng. Others mostly cut the message to 1024 or so bytes and each one handles new-lines in a different way. So far it seems noone is sending messages that way and noone is expecting them. If a new protocol is the way to push structured logging forwards, I'm all for it. It doesn't mean those two protocols cannot coexist. It doesn't mean a typical logging pipeline cannot include syslog->gelf converter which adds annotations / parses the message.
It also seems that the structured messages handling assumes the message fits in a packet and can be processed dropped when convenient depending on its size. That's not something I expect from a logging system. Gelf adds chunking / compression just in case.
rsyslog also supports structured RFC5424 messages. Others will support it in time.
I agree it's not the most perfect format, but it's usable, and already much better than freeform text messages if you intend to do programmatic processing.
Yeah, they thought that up 30 years ago. It's called syslog.