There's no escaping it: an exploration of ANSI codes

Posted by ankitg12 5 days ago

Comments

Comment by skissane 2 days ago

As an aside, I'd always been told that the actual ANSI X3.64 standard wasn't available unless you paid $$$ for it, and that's why people referred to ECMA-48 instead. Recently, I discovered that it has been available for free all this time from the US government, who republished it as FIPS Pub 86: https://nvlpubs.nist.gov/nistpubs/Legacy/FIPS/fipspub86.pdf

Comment by daneel_w 2 days ago

I am not sure if it's part of the ANSI standard, but the AmigaDOS shell supported a set of ANSI codes that provided single line stepping - not a whole text line/row, but a single raster line - allowing for pretty advanced graphics rendering by overlaying rows of text shifted by just one or two pixels vertically. It was a tad fancier than the very common ASCII art, being used in the same venues and always a "treat" to come across, though not as common due to the size and additional time needed to render.

Comment by mspreij 5 days ago

https://en.wikipedia.org/wiki/ANSI_escape_code goes a bit more in-depth.

Comment by trumpdong 5 days ago

I was expecting something more... informative?

Comment by kps 2 days ago

Start with ECMA-48 https://ecma-international.org/publications-and-standards/st...

Then find DEC and XTerm extensions https://invisible-island.net/xterm/ctlseqs/ctlseqs.html

Comment by JdeBP 2 days ago

I recommend starting with ECMA-35. The actual structure of escape sequences is explained there. Otherwise it gets lost that it's the ESC [ that is the entire escape sequence, an alternative form of CSI, and that this actually is one part of an entire mechanism of escape sequences with intermediate and final bytes. It's a control sequence that CSI then introduces.

Comment by kevin_thibedeau 2 days ago

The important part is that the escape prefix is just an alternate way to represent each of the 32 C1 control characters with a pair of 7-bit characters.

Comment by kevin_thibedeau 2 days ago

> Even fancier terminal UIs, like Vim and htop, extend the ANSI codes for cursor positioning and screen manipulation

These aren't an extension. They're either part of ECMA-48 like the color codes or legacy from VT52/VT100 that has become de facto standard.

Comment by kstrauser 2 days ago

I don’t think that’s right. Ghostty has its own TERM=xterm-ghostty value, which surely it wouldn’t need if it supported nothing more than ECMA-48/VT100.

Comment by Chu4eeno 2 days ago

That's usually only necessary for more questionable things that was niche for a reason, like file transfer, images, audio, and rediscovering the root cause of old security issues like this all over again: https://web.archive.org/web/20030302210517/http://www.digita...

Comment by kevin_thibedeau 1 day ago

Cursor movement isn't an extension. Xterm extensions don't deal with in terminal rendering.

Comment by Lerc 2 days ago

I have never quite found a full and comprehensive catalogue of escape sequences. I think the last time I needed a list, I found a developer of a terminal app(might have been kitty?) had a page with what they had found.

This isn't no much a specification as a collection of variously supported codes.

Some have been deliberately killed off (like setting the window title to the string returned from a commandline string). An escape code so powerful that it gives text files shell access.

Comment by shakna 2 days ago

Or, a collection of specifications.

C0 in ISO 6429

C1 in ECMA-48

Though, the big one is C1. OS commands and CSI (graphics, cursors, etc.) are both defined in it.

And hyperlinks have sorta default to ESC]8 but its not in any of the standards. Ta to Gnome and VTE for making it semi-supported everywhere.

The Linux manpage for ANSI escapes is pretty comprehensive as a catalogue. [1] If you're looking for what you can actually use, and what has been deprecated, man should be the first stop.

[1] https://man7.org/linux/man-pages/man4/console_codes.4.html

Comment by skissane 2 days ago

> The Linux manpage for ANSI escapes is pretty comprehensive as a catalogue. [1] If you're looking for what you can actually use, and what has been deprecated, man should be the first stop.

The Linux console_codes(4) man page is only what escape sequences are supported by the Linux VT console, which you probably aren't using. You are more likely using a GUI terminal emulator under Wayland or X, in which case you need to look at the docs for that terminal emulator.

Most terminal emulators have more comprehensive support than the VT console does, so if the VT console supports it, a recent terminal emulator probably will too. The exception is "Linux Console Private CSI Sequences", which I doubt terminal emulators would support, although some of them could be supported.

Comment by shakna 2 days ago

> Most terminal emulators have more comprehensive support than the VT console does, so if the VT console supports it, a recent terminal emulator probably will too. The exception is "Linux Console Private CSI Sequences", which I doubt terminal emulators would support, although some of them could be supported.

Huh? Most of them have less. And none of them will have documented the OS-specific escapes, because that's kinda the role of console_codes.

xterm-specific things, which supports more than most other emulators, and a lot more than VT, is documented in... console_codes. Sure, they have their own documentation, too.

But console_codes is absolutely not only what is supported by the Linux VT console.

Comment by skissane 2 days ago

But this isn't true. console_codes is only what the Linux VT console supports.

For an example of what it omits: Alternate Screen Buffer support (DECSET 1049). This is common used by full screen apps (e.g. vi, less) to save/restore the contents of the command line screen. xterm supports it; most popular terminal emulators support it; the console_codes man page does not mention it. The Linux VT console did not support it until Linux 7.1 (see https://github.com/torvalds/linux/commit/23743ba64709a9c137c...) and the console_codes man page still hasn't been updated to mention the kernel now supports it.

> And none of them will have documented the OS-specific escapes,

These aren't really "OS-specific". The Linux console defines private sequences of the form CSI Ps ; Ps ] – those aren't specific to Linux, they are specific to the Linux console – you can't assume most Linux terminal emulators support them. They also aren't technically valid per ECMA-48 – the ] terminator was already reserved for the "START DIRECTED STRING" (SDS), which was intended for use with Bidi (e.g. Arabic, Hebrew), although I'm not sure if any terminal supported it; and also, when preceded by a space, the "SELECT ALTERNATIVE PRESENTATION VARIANTS" (SAPV) function, which also relates to Bidi, but also to Indic scripts; few vendors ever implemented either function, but IBM aixterm does partially implement SAPV.

Most terminal emulators (including xterm) instead use OSC (Operating System Command) to implement custom functionality, which would have been a more defensible choice from a standards-compliance perspective, but this feature was added in the 1990s, by developers who likely didn't have a good understanding of ECMA-48. Despite OSC's name, on almost all platforms it isn't implemented by the operating system, it is implemented by the terminal emulator, and so different terminal emulators on the same OS will implement different OSC sequences. (The only exception I am personally aware of is RMX, which uses OSC to do the moral equivalent of the termios IOCTLs under Unix-like systems, and so on RMX an OSC sent by an application is actually interpreted by the tty driver as a command to change its config.)

Comment by shakna 1 day ago

I'm pretty sure all the references to X11 systems, aren't supported by the VT, that ain't X.

    ESC [ ? 1000 h
                X11 Mouse Reporting (default off): Set reporting mode to 2
                (or reset to 0) —see below—.

And as I _can_ see a DECSET section, the manpage is referencing it.

    DEC Private Mode (DECSET/DECRST) sequences

        These are not described in ECMA-48.  We list the Set Mode
        sequences; the Reset Mode sequences are obtained by replacing the
        final 'h' by 'l'.

And as there's whole sections about sequences _not_ implemented by the Linux console, it does not only contain what the Linux VT console support. Which is the opposite of your first statement.

    Escape sequences

        VT100 console sequences not implemented on the Linux console:

        ESC N       SS2   Single shift 2.  (Select G2 character set for
                            the next character only.)

Which means either we're looking at a different version of the manpage, you haven't looked before throwing things, or you just really like to argue.

Comment by skissane 1 day ago

> I'm pretty sure all the references to X11 systems, aren't supported by the VT, that ain't X.

This isn't true. The Linux VT console supports both X10 and X11 mouse reporting–except the Linux kernel source code calls X11 mouse reporting "VT200" instead, but it is the same thing. It relies on a user space component (traditionally the gpm daemon) to talk to the mouse driver to get the actual mouse position, and then feed that into the VT device using the TIOCLINUX ioctl (subcode TIOCL_SETSEL, mode TIOCL_SELMOUSEREPORT)

> And as I _can_ see a DECSET section, the manpage is referencing it.

Yes, it describes DECSET. But it doesn't describe many of the DECSET codes, including 1049 and 2004 which the Linux VT console supports – see https://github.com/torvalds/linux/blob/master/drivers/tty/vt... – and the many more which xterm supports – https://invisible-island.net/xterm/ctlseqs/ctlseqs.html – other terminal emulators frequently implement many (but not necessarily all) of the DECSET codes xterm does.

> And as there's whole sections about sequences _not_ implemented by the Linux console, it does not only contain what the Linux VT console support. Which is the opposite of your first statement.

It briefly mentions a few features which xterm provide that Linux VT console doesn't. It makes no attempt to provide an exhaustive summary of all the features xterm has which are lacking in the Linux VT console, since that's the job of the xterm documentation

Comment by dang 1 day ago

Can you please make your substantive points without swipes? This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html.

Your comment would be just fine without that last sentence.

Comment by noelwelsh 5 days ago

This would be far better without the slop and just the widget with a little bit of explanatory text.