A shell colon does nothing. Use it anyway
Posted by olexsmir 2 days ago
Comments
Comment by garethrowlands 1 day ago
For example, what does `$foo` mean in shell syntax? In any reasonable language (perl or powershell, for example, or python if you drop the `$`), it's an expression that evaluates to whatever value's inside that variable. In shell, `$foo` isn't an expression in that sense, and what it does depends on what's inside it via a variety of string substitution rules.
This is the main reason we have arcane articles like this.
That said, nice article.
Comment by bityard 1 day ago
Old doesn't imply arcane. It instead can (and does, in this case) mean that it won out over the course of decades against other less worthy alternatives.
Although you can write "programs" in sh, the shell syntax was never meant to be anything like a systems or application programming language. It was meant to efficiently automate systems tasks. It is kind of like the fork lift of computing. It does it's job very well, is bad at doing anything else, yet no other kind of vehicle can do it's job nearly as efficiently.
Comparing the shell to C, Go, Rust, Python, or JavaScript is crazypants. It has a different job than those, so of course it will look and feel different!
Comment by garethrowlands 1 day ago
> Old doesn't imply arcane.
Indeed so. For example, awk is about the same age but doesn't suffer the same problems.
> the shell syntax was never meant to be anything like a systems or application programming language
A programmable shell is not only like a programming language, it is a programming language. Systems tasks have some specialised requirements but, overall, scripting is programming. There is nothing in the problem domain that requires the posix string-substitution problems.
If fish or oil had existed back then, they would likely have won, since they are better.
Comment by dahart 1 day ago
This is a logical statement that is technically true while not matching real-world experience. Scripting as a type of programming is different from application development programming, and scripting needs differences in features and interface to be convenient and comfortable.
> There is nothing in the problem domain that requires the posix string-substitution problems.
What do you mean? Environment variables and command line execution both separately lead to string substitutions. The fish shell uses string substitutions.
You emphasized ‘requires’ which again might make your statement technically true while still missing the experiential forest for the logical trees. You can use x86 assembly for your shell if you want, there’s nothing that “requires” posix. The reason string substitutions exist is because they’re useful and convenient; the alternatives are verbose and klunky.
Comment by rmwaite 20 hours ago
Comment by andrewl 1 day ago
Comment by tyre 1 day ago
Whitespace sensitivity is horrendous (can’t autoformat, no multi-line lambdas, bonkers inline list comprehensions) but here we are.
Comment by cogman10 1 day ago
Shell was designed to launch processes easily. Unix philosophy pushed to make every function an application. And that's ultimately what shell "syntax" is.
There's shockingly little syntax in the shell. A lot of what we think of as "shell syntax" is actually full blown (standard) programs which the shell is launching. My favorite of which is `[`.
What makes shell unique vs all other programming languages is it puts launching processes first. It has no friction there because if it did, it wouldn't work.
This is also what I think is gross about shell. Every other scripting language has at least some friction when launching other processes and that's usually a good thing IMO.
Comment by lynndotpy 1 day ago
Comment by tejtm 1 day ago
Pray tell which one of the myriad is the true one?
And do not worry, no one would be rash enough to bring in external dependencies it is just too darn inconvenient.
I should probably drink my coffee before posting, be less grumpy.
Comment by lynndotpy 1 day ago
Of the languages listed in GP, only Python and JavaScript are scripting languages with REPLs like Bash. Of those two, Python is far more serviceable for what one uses bash for, even in embedded applications (with MicroPython).
Perl, Awk, Ruby, and Lua are also scripting languages one can use in the course of OS scripting. (Your distro probably uses some combination of these, including Python.) Of these, at least Ruby can also be used as a shell.
> Pray tell which one of the myriad is the true one?
You've basically got it, that's just `which python`
I use Python for longer scripts where script maintainability is a priority and external libraries are unnecessary, and as a general shell when I also want my shell to be a calculator, especially when computing on a smartphone (with an appropriate pythonrc).
The key thing is that I already know Python and I find Python's warts more palatable than those on Bash.
Comment by andrehacker 1 day ago
What I appreciate about the traditional shell languages is their remarkable stability. Shell scripts written in 2000, or even earlier, are often still able to run today with little or no modification. By contrast, Python applications frequently require recreating a historical runtime environment, including older language versions and dependencies, many of which have accumulated significant security vulnerabilities over time.
Interestingly, apart from the various shell languages, Perl is probably one of the strongest alternatives in this regard. The Perl community has placed a high value on backward compatibility, allowing older code to continue functioning while the language itself remains actively maintained and up to date.
Comment by lynndotpy 23 hours ago
> Comparing the shell to C, Go, Rust, Python, or JavaScript is crazypants. It has a different job than those, so of course it will look and feel different!
The point was just a small one, that Python is not like the others. C, Go, and Rust do not come with REPLs. Node is plausible to use as an OS shell but I've never heard of anyone doing that. Python is the only one of those I've used as a shell or have heard of others using as a shell.
Again, I'm not advocating for Python as a good general-purpose shell. I'm only making the small claim that C, Go, Rust, and (to a smaller extent) JavaScript cannot be be used like bash. Python could be used as a shell.
Comment by amiga386 1 day ago
Python is a full-on programming language. Its REPL is a REPL for that language. Interacting with the OS and filesystem is a niche, tucked away in a corner of the language.
It has has none of the ergonomic affordances needed to be the user interface for interacting with files and processes.
Here are some simple commands:
ls
cd foo
ls
rm bar
cd ../baz
rm bar
echo >readme.txt baz directory
curl -s http://example.com >download
Let's try doing that with python: python
>>> ls
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'ls' is not defined
>>> import system
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'system'
>>> import os
>>> os.dir.list()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'os' has no attribute 'dir'
>>> help(os)
>>> os.listdir()
['foo', 'bar', 'baz']
>>> cd foo
File "<stdin>", line 1
cd foo
^
SyntaxError: invalid syntax
>>> chdir("foo")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'chdir' is not defined
>>> os.chdir("foo")
>>> os.listdir()
['bar']
>>> os.delete("bar")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'os' has no attribute 'delete'
>>> os.remove("bar")
>>> os.chdir("../baz")
>>> os.remove("bar")
>>> f = open("readme.txt", "w")
>>> f.write("baz directory")
13
>>> f.close()
>>> import process
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'process'
>>> import subprocess
>>> subprocess.run("curl", "-s", "http://example.com/")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.9/subprocess.py", line 505, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/lib/python3.9/subprocess.py", line 778, in __init__
raise TypeError("bufsize must be an integer")
TypeError: bufsize must be an integer
>>> subprocess.run(["curl", "-s", "http://example.com/"])
...
CompletedProcess(args=['curl', '-s', 'http://example.com/'], returncode=0)
>>> subprocess.run(args=["curl", "-s", "http://example.com/"], capture_output=True)
CompletedProcess(args=['curl', '-s', 'http://example.com/'], returncode=0, stdout=b'...\n', stderr=b'')
>>> p = subprocess.run(args=["curl", "-s", "http://example.com/"], capture_output=True)
>>> fh = open("download", "w")
>>> fh.write(p.stdout)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: write() argument must be str, not bytes
>>> fh = open("download", "wb")
>>> fh.write(p.stdout)
559
>>> fh.close()Comment by lynndotpy 1 day ago
Comment by amiga386 1 day ago
I would be ready for the window if I had to type
import subprocess; p = subprocess.run(args=["curl", "-s", "http://example.com/"], capture_output=True); fh = open("output", "wb"); fh.write(p.stdout); fh.close()
rather than curl -s http://example.com/ >output
There are certainly attempts to create a shell that's powered by Python and allows the Python language into it, but one could not "largely replace bash with the Python REPL". Only people who didn't want a shell in the first place would think that reasonable.Comment by lynndotpy 23 hours ago
The point I am making is that, of the listed languages, Python is far more comparable to Bash than the others. Suppose you had no choice but to use one of the following executables in place of /usr/bin/bash
- go - gcc - rustc - node - python
You would choose Python, of course. Plausibly node, but certainly not any of the other three.
Comment by amiga386 22 hours ago
I didn't read the list of languages given by the earlier poster as specific list, I read them as "<list of random programming languages that aren't shells>". And sure, a scripting language is closer to a shell than a compiled language, but I wouldn't say either Python or Node treat shell commands as first-class citizens, as fellow scripting languages Perl and Ruby do... and even the I wouldn't use Ruby or Perl REPLs as shells, because they're still actually programming languages, not text-based user interfaces to your OS.
Perl: $x = `ls -l`;
Ruby: x = `ls -l`
Python: import subprocess; x = subprocess.run(args=["ls", "-l"], capture_output=True).stdout
Node: let x = (await require('util').promisify(require('child_process').exec)("ls -l")).stdoutComment by rlpb 21 hours ago
This is handy for quick scripting as a natural extension of a command line interface. A counterexample might be an equivalent script using the Python subprocess module. For a simple script, the required extra quoting overhead is not worth the trouble. For something complicated it’s essential and a shell script is no longer suitable. For something in the middle, a carefully crafted shell script (eg. one that passes shellcheck) may be the most suitable solution depending on the situation.
Both cases have their place. It’s about using the right tool for the job.
Comment by dahart 1 day ago
I don’t agree with this assumption that shell should use mechanisms we accept in “regular” programming languages, whatever that means. And why not the other way around? Why do we accept mechanisms in programming that we wouldn’t accept in a shell? There are layers of assumption in your assumption.
I also disagree with lumping scripting and programming together. Those are two different activities, and shell is better at scripting than programming, and it’s better at scripting than programming languages.
Try using python as your shell, and you’ll find out that python is fundamentally bad as a shell REPL. The shell is necessarily different from python or C++ or <your_favorite_programming_language>, starting with the ability to run command lines by typing them, with zero extra syntax. That is something regular programming languages can’t do, and it leads directly to the need for string substitution rules.
If you want a shell made out of a good programming language… make one! People have been thinking about this for decades and nobody has come up with a good one, because shell languages like bash are better at shelling than programming languages.
Comment by chrisweekly 1 day ago
Comment by dahart 1 day ago
Edit I’m reading about it, here are some notes to myself:
Sounds like there are 2 modes, subprocess & python
Env vars are still string substitution, and there are many rules and string literal prefixes.
You will want the prompt toolkit (ptk). It’s an additional install. (Non-default installs, btw, are why I stopped using zsh. I loved zsh, but had to move to bash.)
Regexes use backticks
@.imp for inline imports… good idea
BTW I googled for xonsh complaints and got what I expected; awkward & klunky interactive editing, special prefix syntax, some missing features, system python dependency issues, high CPU usage, and lack of portability. People seem to suggest that xonsh is better for scripting than interactive usage.
Comment by layer8 1 day ago
Comment by garethrowlands 1 day ago
Comment by assimpleaspossi 1 day ago
Posted from my phone since my system came crashing down.
Comment by garethrowlands 1 day ago
Comment by assimpleaspossi 1 day ago
Comment by amelius 1 day ago
Bash et al are great for command line use, but produce a situation of really bad engineering hygiene when used in scripts.
Do not use. Stay away. Bash et al considered harmful. Red flags on job interview when referenced.
Comment by ndsipa_pomu 1 day ago
Possibly the most important tip is to use ShellCheck (https://www.shellcheck.net/) as a linter on your scripts and work through the various warnings to eliminate them. For the rare times that ShellCheck is overly cautious, you can remove a specific warning with a comment before the offending line such as
# shellcheck disable=SC1091
Also, take time to look through Greg's Wiki as it is possibly the best Bash resource that discusses the various pitfalls: https://mywiki.wooledge.org/BashPitfallsAs a Bash script writer, I disagree with your "red flag" interpretation as that'll mean that you'll end up with poorly written Bash scripts along with a hotch-potch of different tools to achieve the functionality (e.g. different versions of Python required for older scripts and newer ones). Knowing Bash is also very useful for writing Dockerfiles.
Comment by amiga386 1 day ago
However, the only one I already knew...
if some-command; then
: # command required
else
echo "command failed"
fi
I used to do that until I learned of if ! some-command; then
echo "command failed"
fi
It's in the POSIX standard so it's not just a bashism: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...> If the pipeline does not begin with the "!" reserved word, the exit status shall be the exit status of the last command specified in the pipeline. Otherwise, the exit status shall be the logical NOT of the exit status of the last command
Comment by mcc1ane 1 day ago
Comment by tzot 1 day ago
I am asking because I have never needed preserving the `$?` outside of an `if`, nor have I ever seen this in somebody else's scripts, and I'm curious.
Comment by mananaysiempre 1 day ago
Comment by teddyh 1 day ago
Example:
foo(){
: "This is a docstring for the foo() function"
bar --verbose | baz --quiet
}
(Repost of <https://news.ycombinator.com/item?id=29152308>)Comment by kevincox 2 days ago
: "${1:?missing argument, aborting!}"
I wouldn't use this because I would want to give $1 a name for the rest of the script, so I would assign. But it can be a nice way to give a clear error for missing required environment variables.Many of the others (like truncating files) are probably more clearly written with dedicated commands, but may come in useful if you are going to extreme lengths to avoid dependencies outside of the shell.
Comment by refp 1 day ago
You are very much correct and I 100% agree with you, I have updated the first example to include a snippet where a proper env-var is used to show of the automatic diagnostic.
Thanks for your feedback, much much appreciated!
Comment by wolletd 1 day ago
I usually go with something like this:
set -eu
function eusage() {
echo "Usage: $0 <input-file> <output-file>" >&2
echo "Error: $@" >&2
exit 1
}
infile=${1:-}; shift || eusage "Missing input filename"
outfile=${1:-}; shift || eusage "Missing output filename"
The `${1:-}` in this case evaluates to an empty string if `$1` is not set, but `shift` fails when there is no argument to remove.Comment by yjftsjthsd-h 1 day ago
> Why use the null-command when I could do VAR=${VAR:-default-value}?
and points out it's one less thing to typo, but that assumes the name is the same; I like e.g.
TARGETFILE="${1:?need input file}"
OPTIONALVAL="${2:-defaultvalue}"Comment by wpm 18 hours ago
Comment by akoboldfrying 1 day ago
Comment by Brian_K_White 1 day ago
: ${DEBUG:=false}
debug is not only false, but garanteed to be set to something so that elsewhere the places that use it don't need extra syntax and checking in case it's empty, and yet you can just set it from the parent environment to override without touching the script or adding a commandline args parser.
And do me, reading this is almost as easy on the brain as just plain DEBUG=true.
Even if you aren't familiar with the extra syntax, like you're someone else in the future who needs to look at it, the two important words pop out, and probably don't mean NOT because whatever those extra bits mean, there's no !. So you get the idea well enough & cruise on.
...[edit] yep it's in the article. but one thing isn't, exactly
condition && {
then-cmd # might exit > 0
: # ensure this block ends true
} || {
else-cmd
}
In other words, maybe you should have figured out some other way to write the whole construct, but IF you want to write the if/else this way maybe for just readability or organization, and you want the else-block to only hinge on the initial condition and not also on whatever the then-block might do, then you need a safety-true at the end of the then-block.Comment by notpushkin 1 day ago
Comment by fphilipe 1 day ago
I have an alias[1] for that which I call a quick interactive rebase:
riq = -c sequence.editor=: rebase --interactive
[1]: https://github.com/fphilipe/dotfiles/blob/94f2ff70bade070694...Comment by teddyh 1 day ago
Comment by jeremyjh 1 day ago
Comment by teddyh 1 day ago
Comment by refp 1 day ago
Comment by __del__ 1 day ago
Comment by dominiwe 1 day ago
Comment by c0l0 1 day ago
Comment by archargelod 1 day ago
[ -z "$1" ] && { echo "missing argument, aborting." 1>&2; exit 1 }Comment by refp 1 day ago
https://refp.se/articles/your-shell-and-the-magic-colon#why-...
Comment by notnmeyer 1 day ago
different strokes of course.
Comment by pimlottc 20 hours ago
Comment by tzot 1 day ago
Comment by archargelod 1 day ago
Comment by xorcist 1 day ago
[ -z "$1" ] && echo "fail" >&2 && exit 1Comment by layer8 1 day ago
Comment by gpvos 1 day ago
1. Everyone wants the colon.
2. Larry gets the colon.Comment by search_facility 1 day ago
Switched from bash to plain python scripts for shell stuff everywhere several years ago, and never looked back into bash zoo anymore. Stable syntax across Win/Mac/Linux, no bash/zsh/msys2 obscure differences, normal errors and Clause writes scaffolds quick and flawless anyway
Comment by wpm 18 hours ago
I'd never call Python "portable" though if you have outside dependencies.
Comment by wodenokoto 1 day ago
Calling all the command line tools is cumbersome, the Python cloud APIs are verbose compared to the cli tools and often have very different concepts for how they interact with services.
But I agree that there is a need for a better bash and zsh is nice but not the upgrade that is needed
Comment by rgrau 1 day ago
Some of the examples here are interesting, but they show parameter substitution more than colon itself: https://tldp.org/LDP/abs/html/parameter-substitution.html
In small scopes, I tend to inline the `:?` validation inside the arg of the command. `echo "${1:? first param required}"`
Another usecase is to use colon in the body of a while loop, while doing work in the condition of the loop.
while rlwrap -o -S'>> ' tr a-z A-Z ; do :; done
Gives you the "do X while it succeeds. stop when it returns non-0" semantics.I've also written about this and other bash tricks over the years in https://github.com/kidd/scripting-field-guide/blob/master/bo.... You might like them :)
Comment by dtj1123 1 day ago
The extent to which the script author gets to feel smart and efficient is exactly the extent to which the future reader of the script gets to feel like an idiot.
Comment by rgrau 1 day ago
Usually, those scripts are written by people who didn't learn the "weird tricks", and use global variables (they didn't learn that `local` creates local variables), for example.
In bash, many times the "dumb way" takes 10 times more lines, and it's probably buggier than the "smart tricky way". So, in the end it's your choice what side you pick, and when to stop. Or, use a "real programming language" (I hate when people say this LOL)
After a while, you get a bit of stockholm syndrome, and you're just fine with bash as an orchestration language, and lean into unix principles. I accept its limitations, and I think it keeps me on my toes as it repels bloat.
idk if you were expecting an answer, but bash lives in this liminal space in the tower of languages that is worth a thought.
Comment by garethrowlands 1 day ago
Comment by zaptheimpaler 1 day ago
Comment by jghn 1 day ago
Comment by AlecSchueler 1 day ago
Comment by jghn 1 day ago
What I've found is I can get get the frontier models to generate bash scripts, perl one liners, etc that do exactly what I need at roughly the same quality as any other code it generates.
Comment by AlecSchueler 1 day ago
I'm a shell scripter at heart, the arcane stuff has always been a delight for me, so I've driven them to do some pretty complex stuff where previously I would have "copped out" and used python. I'd say the general adage of them being at the level of a very talent junior holds true.
Comment by Arshad-Talpur 1 day ago
Comment by nullsanity 1 day ago
Comment by nomel 1 day ago
It's the only language terrible enough to make the default behavior ignore undefined variables, commands, and execution errors, and happily continue executing whatever was produced by me smashing my hands on the keyboard, until the end of the file, while returning an exit code of 0, claiming complete success.
Comment by shric 1 day ago
Perl (needs use strict)
Ancient VB/VBA/VB script
Original PHP (no idea about modern)
PowerShell
Old Windows/DOS batch
Comment by _bernd 1 day ago
No. It does not. Just write good scripts and catch errors and handle them. That's the same more less with every language. And bash can be written in a way that it is sane and readable and maintainable. Just because you have seen a lot of junk in the language does not make the language bad per se. Sure there are a lot of languages "features" which are more then questionable but I want to rise again the point that it has its place and can be used in a good way.
Comment by nomel 3 hours ago
I first learned about shellcehck when I worked with someone that claimed they could write flawless bash scripts, then blamed me when their script broke because there was a space in a folder name. I ran his scripts through, resulting in him immediately adopting it, while he enjoyed eating crow.
Comment by _bernd 2 hours ago
Comment by nomel 1 hour ago
But, you'll never convince me that "ignore errors by default" is sane. ;)
Comment by valleyer 1 day ago
Comment by Gigachad 1 day ago
Comment by garethrowlands 1 day ago
Comment by axus 1 day ago
Comment by genidoi 1 day ago
Comment by csydas 1 day ago
a common tell of ai generated powershell is a script that has dedicated functions to check types, often via several methods, and happily prints the output of the checks to shell. i do not get why it does this but it often adds dozens of lines that really serve no purpose but to make the shell output look fancy
Comment by genidoi 1 day ago
Comment by garethrowlands 1 day ago
...
I have analysed your idiomatic one-liner and propose adding the following 30 lines into the application Yes/Yes and always accept such idiocy/No
ESCAPE ESCAPE
/clear
I want to update my powershell skill so that it writes idiomatic pipelines and does not introduce idioms from other languages.
[PASTES overeagerness wording from Claude prompting best practices]
Comment by Grimburger 1 day ago
I have to put in every claude.md that the shell is zsh. It's reached the point of annoyance I might just go back to bash.
Comment by kergonath 1 day ago
You can use zsh as the default shell and still write your scripts for bash. Actually, it’s an advice I saw more than a couple of times. Otherwise you need to translate the bash-isms when you get bits of code from random places on the Internet.
Just put the right shebang (or ask Claude to do it). What’s the problem?
Comment by delta_p_delta_x 1 day ago
But maybe that's because I'm the sucker, and since PowerShell is more verbose it costs me more tokens than the terseness of Unix shells. Oh well.
Comment by garethrowlands 1 day ago
Comment by srcoder 1 day ago
Comment by IshKebab 1 day ago
The knowledge that Bash is awful and should be avoided as much as possible is surprisingly and disappointingly rare.
Comment by cozzyd 1 day ago
Comment by antonvs 21 hours ago
Comment by wpm 18 hours ago
Sometimes I feel like the people on HN repeating the trope "uhhh if you need ____ go learn a real language!" are just bad at writing shell scripts.
Comment by amelius 1 day ago
(...)
> especially in the age of LLMs
Funny way of putting it.
But I think you are right still :)
Comment by ocd 1 day ago
After a while, you probably know more commands and utilities than you know what to do with, and you'll forget they exist when you need them. In order to not waste time looking for a program for a particular and infrequent purpose, I create "do nothing" aliases like `: alias f3probe` so I can realize I just forgot I already have something for it. Nice predictable pattern to grep with `^: alias` to look through all of these.
Comment by IdiotSavage 1 day ago
param(
[parameter(mandatory)] $name
)
"Hello, $name!"
And then: $ ./script.ps1 Dave
Hello, Dave!
$ ./script.ps1 -name Dave
Hello, Dave!
$ ./script.ps1
cmdlet script.ps1 at command pipeline position 1
Supply values for the following parameters:
name: <cursor here>
Or non interactively: $ pwsh -nonint ./script.ps1
script.ps1: Cannot process command because of one or more missing mandatory parameters: name.Comment by delta_p_delta_x 1 day ago
And in my opinion, the most slept-on: the fact it runs on the CLR and direct access to .NET objects and types which means access to P/Invoke and thence the Windows API. One can write business logic in the fast language and write a nice CLI wrapper around that in the natural shell language, and not worry about painful FFI unlike everyone else trying to fit Python or Bash into whatever world they're using.
The typical counter to this will be: PowerShell is verbose, PowerShell used `curl` as an alias to Invoke-WebRequest instead of the Real Thing™. Neither are real arguments.
Comment by embedding-shape 1 day ago
Comment by refp 1 day ago
I wrote my first real `.ps1` the other day for auto-installing all dependencies needed to run a `gitea`-runner on windows for windows builds; powershell - felt like the lover I never had. And the documentation in readable comments at the top that just.. generates usable docs? Damn, damn, damn.
There is a part of me that low-key wanna try that as my daily driver for a week or two. But with that said, I'm a zsh vi-mode guy - always have been, always will be.. but I'd happily take powershell on a romantic getaway every once in a while!
Comment by rezonant 1 day ago
Comment by crabbone 1 day ago
Another important feature of a tool like this is the ability to tolerate errors: I can't imagine a Linux today that would be able to even boot if the shell was extra pedantic about errors. A lot of mostly irrelevant things routinely fail on boot and during normal operation. Stamping them all out is an arduous... well, basically, an impossible task for practical purposes where releases are expected to come on time, where users may manipulate configuration in gazzilions of unpredictable ways.
PowerShell is just another language in the same box with Python, Perl, Ruby and many like that. It's not a good language, if you decided to reach for that box. Probably not the worst either.
System shell, however, isn't meant for writing entire applications. Writing applications with elaborate command-line interface should be left to languages that can properly address this problem. PowerShell is trying to be there, but it doesn't hold a candle to its "older brothers" who can, indeed, design a very robust command-line interface, often using a dedicated library for it.
PowerShell appeals to the novice crowd who are very enthusiastic about automatic checks in their code: the benefits are on the surface, the downsides are difficult to assess. This is in line with other Microsoft software products / languages which target novice programmers by implementing as many as possible of the highly-advertised features without regard to the overall usefulness of the product (think about C# or MS Office suit etc.)
Comment by garethrowlands 1 day ago
Programmable shells are, in fact, for programming. That posix shell syntax makes it impractical for meaningfully large scripts is something you just accept. And, realistically, it is something you have to accept.
Minimalism isn't why posix shell syntax is bad. For example, awk and jq are minimal but don't make the same mistakes.
Powershell was never designed to be your system shell (Windows doesn't really rely on one) but the main reason it can't be the shell is startup time, which rules it out of wrapper scripts. Its having lots of features might contribute to that but it's not the problem per se (being built on .net is the reason - that and startup time not really mattering on windows). Its lacking the syntax problems of posix shells is very much not a problem. Powershell's error handling is just fine, by the way.
The startup time of even pwsh is too slow to be usable in many scenarios.
PowerShell's not going to be the system shell on your *nix box but it wasn't designed to be. But your posix shell isn't the system shell because it's better. It's pretty much the ultimate expression of worse is better.
That's why there's an industry of alternatives without the faults, such as oil, elvish, fish, nushell. And why awk is so popular.
Comment by crabbone 12 hours ago
For instance, consider that Unix Shell only has one data-type: strings. I believe this was a deliberate decision by its creators (or a stroke of luck). Compare this to languages s.a. original JavaScript with a handful of built-in types, or, even worse, modern JavaScript with user-defined types. Even though the language tries hard to supply default "solutions" to type mismatch errors, it doesn't work well, and sometimes, at all. Unix Shell is inherently incapable of type errors.
While in statically checked programs, or even dynamically checked ones with robust debugging tools types could prevent certain kinds of programming errors, in the mostly interactive programs types make no sense. There simply isn't enough code to generate the kinds of problems types are supposed to solve.
> Minimalism isn't why posix shell syntax is bad.
I never said that.
> Powershell was never designed to be your system shell
Parent wants to replace Linux system shell with PowerShell, that was the reason I replied in the way I did.
Comment by IdiotSavage 1 day ago
You seem to be implying PowerShell aborts on any error. That's not the case:
https://learn.microsoft.com/en-us/powershell/module/microsof...
Comment by crabbone 12 hours ago
Comment by ndsipa_pomu 1 day ago
Comment by kunley 1 day ago
Comment by jiveturkey 2 days ago
- it creates the file if it does not exist, not merely truncate. as a tutorial kind of blog post this incomplete description matters IMO.
- it would work the same without the colon (similar for default variable assignment examples). we generally strive not to have "extra" things, like useless use of cat.
- educationally it's useful to demonstrate that redirection, like parameter expansion, works before the command executes (the null command in this case), but the article doesn't explain that at all!
otherwise i <3 this article. some uses of colon i had never thought of or seen before. like file truncation, not sure i'd use them but it was cool to see them.
Comment by refp 1 day ago
I agree with you, and for what it's worth the truncation snippet is very much tongue-in-cheek much like `( : >> output ) && echo "is writable"`.
I wasn't expecting anyone to actually use these in Prod, rather I aimed to show what can be done with a command designed to.. do nothing (crazy).
Happy you enjoyed the article, and thank you!
Comment by unixhero 5 hours ago
Comment by 1vuio0pswjnm7 1 day ago
>file1 >file2
versus : >file1 >file2
I've done quick and dirty, interactive truncation like the former for many years, no colon. But I would not use it in scriptsAccording to https://www.in-ulm.de/~mascheck/bourne/ SVR4 (1989) had a bug when using this method in a for or while loop and this bug showed up in a SunOS 5 variant, too
Apparently, early in the shell's evolution, : was used as a comment marker before # was added
Single quotes could be used to prevent undesired behaviour
: echo output 1>&2
: `echo output 1>&2`
: '`echo output 1>&2`'
System III (1981) also had a bug when using : as a substitute for true if false;then :;fi
returned 1 instead of 0Comment by olexsmir 1 day ago
: "${DOTFILES_PATH:=$HOME/.dotfiles}"
Which will use $DOTFILES_PATH value if it's set, otherwise it's going to be $HOME/.dotfilesComment by darkoob12 1 day ago
The other day there was a blog post about learning SIMD. I think in future "programmers" will just nag about the speed of the program and the coding assistant will eventually introduce SIMD to the source code.
It is a little sad but we have to go with the current if we want to survive.
Comment by garethrowlands 1 day ago
Comment by user3939382 1 day ago
Comment by tim-- 1 day ago
Comment by jagadaga 1 day ago
Comment by kazinator 1 day ago
The subshell execution parentheses and the colon are superfluous here, just:
< dataset.json && echo YES
Redirections do not require a colon command to hang off of, and there is no need to fork a subshell to execute such a command.> ( : >> result.json ) && echo YES # is result.json writable?
As a go-to idiom for a writability test, it gives me pause. If the file didn't exist, we created a zero-length one. That might be okay if we are going to write to it anyway as the next action.
If we are testing because we intend to overwrite it, why not just "> result.json" (which is by itself an idiom for truncating a file to zero length).
When would we every do this? Maybe before some command which takes the file name as a destination file argument rather than using output redirection, and which performs a lengthy computation before trying to open the file for writing. We can catch the permission error early.
I don't think I've ever coded such a test; normally you just do the operation that writes to the file and let that fail.
In POSIX C, there is a function access() for doing these kinds of tests. But it has a special purpose: it is meant to be used by a setuid root process to perform a permission test as if it were the real user/group (the one which elevated privilege to root). I.e. it's not can we do this operation, but should we do this operation (would we still be allowed, if we dropped privileges back to the original user).
Comment by refp 1 day ago
zsh% echo "hello world" > data
zsh% < data && echo "READABLE" # <- will print the contents of data
hello world
READABLE
zsh% : < data && echo "READABLE" # <- this will not
READABLE
So, if you want something that "everyone can use" without going into details about the difference between commonly used shells.. you'd use the null-command.---
and given that we use the null-command, it _WILL_ behave different with or without subshell.. and all you need is `bash --posix` to prove it:
% cat subshell.sh
#!/bin/bash --posix
( : < missing.json ); echo AFTER # <- will echo AFTER
% ./subshell.sh
./subshell.sh: line 2: missing.json: No such file or directory
AFTER
% cat no-subshell.sh
#!/bin/bash --posix
: < missing.json; echo AFTER # <- this will not
% ./no-subshell.sh
./no-subshell.sh: line 2: missing.json: No such file or directory
% : ^- apparently.. there is a difference
The output above is not truncated, `no-subshell.sh` will stop executing due to the broken read.---
One should never trust things just because they are written, but that also applies to comments on HN. Originally when I read your message I actually thought I made a mistake, I was very close to writing an apology comment and adding a note to the blog post, but not close enough - I had to test it again.
I'm thankful for the watchful eyes and scrutiny when reading things online, that's good - keep it up, but your message is factually wrong - on so many levels.
Comment by kazinator 18 hours ago
This is required by POSIX in the section "Consequences of Shell Errors". If a redirection error occurs in a special built-in command, a non-interactive shell is required to exit.
The bare redirection without : does not have this problem:
< missing.json; echo AFTER
Therefore, in a POSIX-conforming shell, we do not need to wrap it in a subshell.> but your message is factually wrong - on so many levels.
How many? Can you count the levels? I didn't know that in zsh "< file" dumps to standard output, which is suppressed by :. If I used Zsh, I would know that sort of thing.
Comment by brabel 1 day ago
One of the reasons I stopped writing.
Comment by refp 1 day ago
Fast-forward to seeing that comment climb up the ranks, posted by a person who has 270x my karma, who seemingly gets upvotes by just.. writing things? no proof? no rationale? nothing?
Yeah, feels bad. I'm super happy so many are enjoying the article, and this situation can't take too much away from that, but man.. discouraging for sure.
Comment by kergonath 1 day ago
> Yeah, feels bad. I'm super happy so many are enjoying the article, and this situation can't take too much away from that, but man.. discouraging for sure.
Again, don’t worry too much about that. The story got upvoted to the front page and lots of people will read it, that’s what matters. Not that a misguided post got a bunch of upvotes.
Comment by kazinator 1 day ago
Or, you might not even write that something yourself; you just understand why it is that way and leave it alone in the refactoring that you are doing; then you get a review comment like "you can get rid of this from here", and you have to respond "no you can't, because ...".
Comment by kazinator 1 day ago
zsh% < data > /dev/null && echo READABLE # <- fixes it, sorta
Maybe zsh still reads the entire file and copies it to /dev/null, which would be a severe performance problem for large files.The colon by itself, without the subprocess, also prevents the dumping to stdout:
zsh% : < data && echo READABLE
However, with the subprocess parentheses, we can redirect the nonexistence diagnostic to /dev/null, which I don't see mentioned or exemplified in the article: zsh% : < nonexistent 2> /dev/null && echo READABLE
zsh: no such file or directory: nonexistent
zsh% (: < nonexistent) 2> /dev/null && echo READABLE
The normal way of testing whether something exists and is readable that you would actually use in production script looks more like this: $ test -r file && echo YES
$ [ -r file ] && echo YES
so we are talking about obfuscated coding.I would say that "if you want something everyone can use", the -r test would be the first candidate.
BTW, why not bring up the strawman of tcsh?
tcsh% < nonexistent && echo READABLE
Invalid null command.
tcsh% : < nonexistent && echo READABLE
nonexistent: No such file or directory.
Yes, if we want an obfuscated readability test which works literally in any shell that is currently still in deployment in systems that permit new scripts to be installed and run, it looks like do need that colon.However, fish doesn't like &&:
fish> < nonexistent && echo yes
fish: Expected a command, but instead found a redirection
fish> : < nonexistent && echo yes
fish: Unsupported use of '&&'. In fish, please use 'COMMAND; and COMMAND'.
: < nonexistent && echo yes
^
Does it support test -r? fish> test -r nonexistent
fish> test -r nonexistent; and echo yes
fish> test -r /etc/hosts; and echo yes
yes
Not parentheses though: fish> ( test -r /etc/hosts ); and echo yes
fish: Illegal command name “( test -r /etc/hosts )”
We clearly have to restrict the idea of what "everyone can use" to POSIX-like shells; there is no getting around shell differences absolutely, other than for perhaps trivial command invocations.Comment by refp 1 day ago
https://refp.se/articles/your-shell-and-the-magic-colon#why-...
I am not asking anyone to put this in production, it is a tongue-in-cheek/cute way of explaining/showing what the null-command _can_ do, not what it _must_ do.
No, any distro that stopped shipping entities mentioned to use only null-command variants.. well, I'd save whatever your end goal is to a meeting with them.
Just enjoy the article for what it is — a piece of trivia.
Best Regards, Filip Roséen - refp
Comment by kazinator 1 day ago
To me, it's "cool" that you can just do "< test-file && echo exists", on a shell whose developers haven't decided to imbue a whole bunch of new requirements into redirection. It's completely legit to note that the existence test is not coming from the : command, but from redirections, and redirections can be set up without a command. The : is only there to work around shell quirks.
For decades, I used "> file" to truncate files to zero length, never with a colon command; the example makes it look as if the colon is required in order for the command line to be valid and for the effect to take place.
BTW I noticed that zsh not only does the cat-like dumping to stdout in non-interactive mode, but it also does the redirection to the pager!!! So if you drop that test into a non-interactive script, and it finds an existing file, if that script is run in a terminal session, it will pause for input.
Comment by luciana1u 1 day ago
Comment by mandus 1 day ago
while :; do
<do stuff>
sleep n
done
Maybe to mainstream to make the cut!?Comment by ndsipa_pomu 1 day ago
Strangely though, I much prefer using the builtin "printf" to replace "echo" and "date", though the "date" usage is harder to read.
Comment by lucideer 1 day ago
One-liners are a cool little artifact of early shell culture & are sometimes still useful today if they're short to avoid the readability problems of `/` when copy pasting a quick shell command to run, but they have no place in scripts.
None of this seems useful to me.
> if you are like me and prefer less typing (gotta go fast)
Yeah, no.
Comment by brabel 1 day ago
Comment by lucideer 1 day ago
I want my personal local utility/productivity scripts to be readable: quick to write & quick to modify on the fly. Brevity doesn't help here - wpm optimises for natural language typing & that translates better to idiomatic logical block structures than to symbol-heavy one-liners.
I also want the same for the small bash snippets in my CI jobs - this is a particular example where brevity is actively bad: this encourages folk to inline their bash snippets in yaml (no syntax highlighting & unlintable) when they should be packaged in script files in CI directories.
Comment by brabel 14 hours ago
Comment by __del__ 1 day ago
Comment by teytra 1 day ago
https://www.in-ulm.de/~mascheck/bourne/PWB/goto.1.html
DESCRIPTION
Goto is allowed only when the Shell is taking commands from
a file. The file is searched from the beginning for a line
beginning with `:' followed by one or more spaces followed
by the label. If such a line is found, the goto command
returns. Since the read pointer in the command file points
to the line after the label, the effect is to cause the
Shell to transfer to the labelled line.Comment by mqus 1 day ago
Comment by Linux-Fan 1 day ago
<https://pubs.opengroup.org/onlinepubs/9799919799/utilities/t...>
The most important differences seem to be:
* `:` is a builtin vs. `true` is an utility * passing arguments to `:` is safe but for `true` its not hence `:` is the right thing to use here.
Comment by ButlerianJihad 16 hours ago
The key difference is that : is required to be builtin. There are several good reasons for this. For example, syntax: the shell can single out this special character syntactically before searching $PATH.
Also, ":" is a disallowed or problematic character for certain filesystem types. If your shell attempted to omit this builtin, it could not always rely on an external command file by this name.
Lastly, people keep bringing up fish, but it is not a POSIX shell, so its similarities in syntax and operation are coincidental.
Comment by hidroto 1 day ago
Comment by orphereus 1 day ago
We have some large bash scripts in my company, ~10,000 LOC spread across multiple files, all sourcing each other and what not. It is truly hard to read bash, which means it is truly hard to maintain bash, which means that when the one person knowing the bash scripts in your company goes away, you're in for some "fun".
My point is, these quirks are not useful, except for some bash enthusiasts.
Comment by wpm 18 hours ago
I spend about 50% of time working time in the shell. I didn't find anything in TFA unreadable.
Comment by garethrowlands 1 day ago
Your company likely started using bash before python was ubiquitous.
Comment by m2f2 1 day ago
while : ; do case "$1" in "") break;; -f|-foo) shift; whatever;; *) usage; exit 1;; esac done
For this... instead
if something; then
true
else
echo ERROR
exit 1
fi
Using : would be too much here.For anything else including json etc. I usually go to duckdb. Awesome support, single file install, readable, easy to maintain.
Powershell on Linux or Unix? Just another huge dependency if you manage 1000s of machines, and good luck finding a Linux gal/guy wanting or able to touch pwsh without chemical grade gloves.
Comment by garethrowlands 1 day ago
Comment by jeffrallen 1 day ago
Also, I'll never use it.
Because a language feature that needs marketing is against readability, among those in my target audience who have not yet read the marketing.
I need my shell scripts to be long enough to explain to my audience exactly what they are doing.
Comment by kps 1 day ago
: stuff;
Then you can copy/paste entire lines of commands.(Yes, this assumes you don't put naughty fragile stuff in your prompt. Buy you're smart enough not to do that.)
Comment by PunchyHamster 1 day ago
I'd reject the pull request. Bash is already bad as programming language (the goodness of language for long code is inversely proportional to how nice it is for shell one-liners), this is just turning "bad" into "line noise"
If your bash script takes more than one screen, rewrite it in Python, hell, rewrite it in Perl, even that's better
Comment by Affric 1 day ago
Comment by garethrowlands 1 day ago
> the goodness of language for long code is inversely proportional to how nice it is for shell one-liners
Well, except for this bit. Oil or fish and awk are perfectly fine for one liners but don't repeat the mistakes of posix shell syntax.
Comment by normie3000 1 day ago
if x then :; else something; fi
over if ! x; then something; fi
Really? Colon is the appendix of the shell.Comment by bodyfour 1 day ago
Probably not an issue for most people in 2026 -- you have to back pretty far for it to be missing. Technically, though, "if x; then :; else" is more portable.
Comment by yjftsjthsd-h 1 day ago
if x then :; else something; fiComment by normie3000 1 day ago
Comment by refp 1 day ago
---
It is a (contrived) example of usage where a command is required and `:` can fill in the blanks. There are certainly scenarios where negating an expression becomes harder than doing the "dumb" way, and I for one has written code where `:` can be used as a placeholder meaning "fill this in later" or equivalent.
With that said, 100% agree with you that in actual "production" code - there is always a cleaner way.
Comment by croes 1 day ago
Comment by greatgib 1 day ago
And now I see this article. So I guess that it is a construct suddenly popularized by llm.
Comment by kunley 1 day ago
Btw, can the sequence :? be called "reverse Elvis" ?
Comment by Hamuko 1 day ago
Comment by Brian_K_White 1 day ago
It's almost but not quite in the article. And you don't necessarily always want this. It depends if you want else-cmds to run only when condition fails, or when either condition or then-cmds fails.
You could write the word true instead of :, coincidentally showing that : never really was a no-op in the first place. It's so not-no-op that there is even an entire external executable /bin/true to do the same job.
condition && {
then-cmds # might exit > 0
: # ensure this block ends true
} || {
else-cmds
}Comment by caruasdo 1 day ago
Comment by sgarland 1 day ago
:(){ :|:& };:
Comment by nekusar 1 day ago
This is one of those clever things, similar to people using perl5 use trinary expressions. Like, are you TRYING to make this obtuse and hard to read?
Comment by garethrowlands 1 day ago
my $x = if ($cond) { 1 } else { 2 }; # Syntax error
Because of this, Perl introduces a second syntax to plug the gap: my $x = $cond ? 1 : 2;
That expression is neither obtuse nor hard to read.Something like the code below is hard to read:
my $fee =
$is_member
? ($is_student ? $student_member_fee : $member_fee)
: ($is_student ? $student_fee : $standard_fee);
It's equivalent to the following code that uses `if`: my $fee;
if ($is_member) {
if ($is_student) {
$fee = $student_member_fee;
}
else {
$fee = $member_fee;
}
}
else {
if ($is_student) {
$fee = $student_fee;
}
else {
$fee = $standard_fee;
}
}
But ideally you would want to write this: my $fee =
if ($is_member) {
if ($is_student) {
$student_member_fee
}
else {
$member_fee
}
}
else {
if ($is_student) {
$student_fee
}
else {
$standard_fee
}
};
Though, of course, perl5 doesn't allow that.Comment by nekusar 1 day ago
I was complaining of the "cleverness" of them, when working or analyzing a codebase with people swapping from if/else loops to trinaries.
I prefer clarity and a bit more verbosity than a per5-ism. If that means a 5 line loop that I can easily follow, then so be it. 1 liners that effectively say "lookie at me im a l33t developer" are a very bad code smell, and make maintenance harder each cycle of more cleverness.
Comment by SoftTalker 1 day ago
And every language has its idioms that are incomprehensible until you learn them. This isn't clever code so much as shorthand for common things.
Comment by fenestella 1 day ago
Comment by vladsiu 1 day ago
Comment by draw_down 1 day ago
Comment by dfasifsaf 1 day ago
Comment by shevy-java 1 day ago
Shell scripts simply suck for many reason. They are ugly, verbose, convoluted, outright stupid too such as argument passing into functions. Then there is straight up retarded stuff such as case/esac. Whoever came up with that was clearly an incompetent language designer.
Comment by garethrowlands 1 day ago
Comment by mondainx 1 day ago
Comment by Sleaker 1 day ago