Bash:Lesser Known Bits

This is one that is supported by most if not all of the other shells out there, but it is still super useful and I don’t see it used often. A one-liner loop is effectively a very short (one line in fact) script used to perform a small number of operations (it gets confusing if you do too many) in bulk. A good example here is with a server environment of any size greater than I’d say two. I frequently need to check lots of servers for something, be it the existence of a file, the status of a file in comparison with a local copy (diffs), bulk modifying remote files using sed, etc.

Recently though, I needed to verify the installed version of sudo specifically on a list of about 50 servers. I sent the list of servers to a text file, one server per line, and did the following and had my answer within about 30 seconds (it takes a few hundred milliseconds for ssh connections to establish on our atrociou…​er…​awesome network).

Presto! A big list of sudo versions across the entire environment.

This one is really great. Some commands require one or more file paths to do what they need to do. A good example is diff. The diff command requires two file path parameters: file a and file b. What if you want to diff the outputs of two remote files though? Using process substitution, we can cat out a remote file using the typical command, ssh user@server cat /etc/something, and have the output go to a local temp file for the life of the command calling it so we have something to work on. For example…​

What we have here is a diff of the local /etc/something file and the remote /etc/something. The ssh connection string is encapsulated in a <(). This is the process substitution. This doesn’t just work with remote files though. Say for instance you wanted to diff the contents of a directory on a local system and a remote system. Here’s how you’d do that.

Here we used process substitution to write the output of ls -l /var/log to a temp file, then write the output of the same command run on another system over ssh to yet another temp file, then we use diff as usual to show us what is different. If you really wanted to get crazy, you could throw this into a bash one-liner loop and run the diff on multiple systems.

Brace expansion is really neat and I think super handy. This is the one I don’t have a lot of use for though. It gets used about once every few scripts or about once or twice a month. Brace expansion is effectively on-the-fly array loops for commands. For a simple example, say you wanted to create three directories: dev, test, and prod. To create these without brace expansion, you’d have to run mkdir three times. With brace expansion, you can do this

That’s cool, but what’s REALLY cool is that you can use this with nested directories. Say for isntance we are creating a small (and poorly designed) dev environment. Inside of each we want the directories bin, etc, lib, var (we’re just making 'em up now). Here’s how you’d do that in one command

That is the equivelant of <syntaxhighlight lang="bash"> mkdir dev/bin mkdir dev/etc mkdir dev/lib mkdir dev/var mkdir test/bin mkdir test/etc mkdir test/lib mkdir test/var mkdir prod/bin mkdir prod/etc mkdir prod/lib mkdir prod/var </syntaxhighlight>

Another application for this is if you want to cat out a big list of specific files without catting out the entire directory (I did this one earlier this morning actually). Say you have 20 files called list.<num> (0-19) and you want to cat out numbers 1-9. Now, there are a lot of ways to do this of course, but this is how you can do it with brace expansion.

…​or even shorter…​

Those are the equivelant of

How’s that for time saving.

Category:Bash Category:Shells Category:Linux