> Personally, I would have the program run tput(1) as a subprocess and capture (and save) the correct escape sequences from the pipe, for later re-use, but I could imagine how, in some special cases, that might not be practical.
That's not a bad suggestion generally speaking. I've seen a fair amount of shell scripts that do this too.
In this specific case, they're running DNS lookups in parallel to speed the tool up, so I think forking tput wouldn't be preferable.
But for tools that are more forgiving for latency (and specifically targeting POSIX too), that's definitely an option.
> You originally wrote “in reality you can get away with hardcoding vt100 sequences for most things.” But now you want to argue that you were always talking only about this specific use case?
Fair point.
> (Also, it seems that for this specific use case, only one escape sequence would be needed: “cursor_up”/“cuu1”.)
I was thinking cursor down too, but actually you're right, \n would do that job better for a multitude of reasons.
Edit: thinking about this more, there’s a reverse carriage return ASCII character so you don’t actually need any ANSI escape sequences at all.
> The same was said about the x86 instruction set, and it did look completely entrenched for many years. But then it wasn’t. And old ideas and concepts about what can be done with terminals, like MGR, may yet rise again. I see things like sixels only as symptoms of a pent-up demand for this. And people won’t want to go through ANSI X3.64 sequences to render their remote applications, so non-ANSI terminal protocols are likely to emerge again.
Professionally speaking:
I can't see that happening. More systems are implementing vt100 support, not less. And escape sequences are "easy" to implement and "good enough" that most people have a quick moan about them but then quickly move on to getting whatever job they want done, done.
Plus the decline of x86 was driven by $$$ (like most changes in IT). Apple wanting to own more of the pipeline. Datacentres wanting to reduce infrastructure and operational costs. And ARM was already as old and established as x86 so it wasn't like trying to create a new standard (how many other architectures have fallen by the wayside?). There isn't anything like that happening for terminals right now.
However, personally speaking:
I genuinely hope you're right. The status quo sucks. Just so long as whatever new that comes along is a complete redesign. In-band meta sequences are just horrible.
But I think realistically, those that don't like the status quo with terminals either use web technologies or Microsoft RPCs instead. And those that like the terminal have already learned to live with its many warts.
Just like UTF-8 unified ASCII with extended character sets, maybe ANSI X3.64 can be unified with something more, incorporating features from MGR and/or sixels.
> there’s a reverse carriage return ASCII character
Which one? And if it works, why does ”tput cuu1” not output it, instead of ^[M or ^[[A?
> Just like UTF-8 unified ASCII with extended character sets, maybe ANSI X3.64 can be unified with something more, incorporating features from MGR and/or sixels.
Not another character set please. Terminals are better off UTF8 and we are better off keeping control sequences out of character sets. It made sense in the 60s and 70s but makes zero sense these days. Plus Unicode has enough footguns as it is without introducing control sequences.
In my opinion the only good way to advance beyond escape sequences is to do away entirely with in-band control codes. Control codes should be on a separate channel. This is basically the only thing HTML gets right: separating content from formatting and control sequences.
Having that separation in the terminal would also allow for less hacks in pipe detection. To elaborate, `isatty()`, the C function that's used to detect if a fd is a pipe or TTY, basically just performs a terminal-specific I/O control operation to see if the file descriptor can handle terminal-specific controls, and if it can't, then assumes the fd is not a TTY. Its a complete hack in my opinion. A hack that works and has worked for a great many years. But a hack non-the-less.
Also having a separate data and control channel allows you to incorporate more meta data about the data channel. I actually use this trick in my $SHELL, Murex [0]. It's a typed shell (like Powershell but far less verbose and far more ergonomic to use) however it works perfectly fine with standard POSIX pipes because the type annotations are sent over a different channel. So you have the best of both worlds: raw byte streams for anything classical, but also rich type metadata for anything that understands Murex pipes. While introducing zero boilerplate code for anyone who uses the shell.
> > there’s a reverse carriage return ASCII character
Which one? And if it works, why does ”tput cuu1” not output it, instead of ^[M or ^[[A?
You're right. I was thinking of ^[M [1] (C1 control sequence) and getting confused with C0 sequences (ie ASCII characters). Wasn't helped by me misremembering that modern terminal emulators tend to treat multiple different ASCII characters as LF, such as VT (vertical tab) and FF (form feed) [2]
Oh no, I was merely using character sets as an analogy. What I was envisoning was a new control scheme, backward compatible with ANSI X3.64, but not merely a bundle of extensions either.
> Control codes should be on a separate channel.
I’m not sure about that. Two separate channels results in synchronization problems (which is why the parallel port went away and why USB and all modern protocols are serial). However, I would not be adverse to having the channel transferring richer data than plain bytes.
> Oh no, I was merely using character sets as an analogy. What I was envisoning was a new control scheme, backward compatible with ANSI X3.64, but not merely a bundle of extensions either.
I'm currently working on getting
- full vt100 compatibility (it's mostly there, just failing a couple of tests in vttest which I believe are related to cursor save and restore)
- most of xterm
- plus a few original escape codes to demonstrate it's concept
I plan to add Tektronix 4014 support
and a few other never seen before Terminal features that I haven't yet decided how best to implement.
I'm also going to leverage the fact that I already have a relatively mature $SHELL and tie in some specific support between the shell and the terminal emulator. If just to demonstrate the terminal's capabilities.
It's still pretty alpha but I'd definitely welcome any feedback on the project's mission.
> I’m not sure about that. Two separate channels results in synchronization problems (which is why the parallel port went away and why USB and all modern protocols are serial).
Synchronizing electrical signals is a very different problem to synchronizing data streams. There's plenty of protocols out there that solve the second problem already: TCP/IP (order of packets), video codecs (audio / video
synchronization), and so on.
They key isn't to have the zero control sequences in the content stream. It's to reduce those control sequences to just being tags or markers, and shifting the meta data out to the data stream.
An example of this is HTML vs CSS. HTML is the content, CSS is the formatting. Granted they're not streamed, but it's a visual demonstration for how separate concerns can be divided but still connected.
This type of approach would also solve the existing synchronization issues with stdout and stderr. Though if we're already redesigning the terminal in backwards incompatible ways, I'd do away with stderr entirely and replace it dedicated control sequences and first class error handling.
None of this would ever happen but a guy can dream.... (though this is why I started the mxtty project: to see just how far I could push terminal capabilities in a backwards compatible way)
That's not a bad suggestion generally speaking. I've seen a fair amount of shell scripts that do this too.
In this specific case, they're running DNS lookups in parallel to speed the tool up, so I think forking tput wouldn't be preferable.
But for tools that are more forgiving for latency (and specifically targeting POSIX too), that's definitely an option.
> You originally wrote “in reality you can get away with hardcoding vt100 sequences for most things.” But now you want to argue that you were always talking only about this specific use case?
Fair point.
> (Also, it seems that for this specific use case, only one escape sequence would be needed: “cursor_up”/“cuu1”.)
I was thinking cursor down too, but actually you're right, \n would do that job better for a multitude of reasons.
Edit: thinking about this more, there’s a reverse carriage return ASCII character so you don’t actually need any ANSI escape sequences at all.
> The same was said about the x86 instruction set, and it did look completely entrenched for many years. But then it wasn’t. And old ideas and concepts about what can be done with terminals, like MGR, may yet rise again. I see things like sixels only as symptoms of a pent-up demand for this. And people won’t want to go through ANSI X3.64 sequences to render their remote applications, so non-ANSI terminal protocols are likely to emerge again.
Professionally speaking:
I can't see that happening. More systems are implementing vt100 support, not less. And escape sequences are "easy" to implement and "good enough" that most people have a quick moan about them but then quickly move on to getting whatever job they want done, done.
Plus the decline of x86 was driven by $$$ (like most changes in IT). Apple wanting to own more of the pipeline. Datacentres wanting to reduce infrastructure and operational costs. And ARM was already as old and established as x86 so it wasn't like trying to create a new standard (how many other architectures have fallen by the wayside?). There isn't anything like that happening for terminals right now.
However, personally speaking:
I genuinely hope you're right. The status quo sucks. Just so long as whatever new that comes along is a complete redesign. In-band meta sequences are just horrible.
But I think realistically, those that don't like the status quo with terminals either use web technologies or Microsoft RPCs instead. And those that like the terminal have already learned to live with its many warts.