miscellaneous utilities quick-ref

misc.txt

+Miscellaneous Utilities
+=======================
+
+`man`
+-----
+
+This is one command you need to know. It gives you access to the
+manual pages ("man pages" or "manpages", for short) for programs.
+When you don't know what a program does, or what options it has,
+`man program` should be the first thing you try.
+
+
+`kill` and `killall`
+--------------------
+
+When a program is out of control, or if it's running in the background,
+you will probably need to fall back on the `kill` command to terminate
+it. This takes one or more process IDs (PIDs) as arguments, and
+optionally the signal to use. SIGTERM is the default, and is usually
+what you want, though sometimes you want SIGKILL:
+
+    kill 1234     # kill PID 1234 with TERM signal
+    kill -9 1234  # kill PID 1234 with KILL signal
+
+Note that the TERM signal can be caught by the process being killed,
+allowing it to clean up after itself. The KILL signal cannot be
+caught, and causes the process to terminate immediately.
+
+The `killall` program matches command names, rather than PIDs. It
+is potentially error-prone, but sometimes very useful.
+
+
+`true` and `false`
+------------------
+
+These are very useful in scripts. `true` exits with status 0, and
+does nothing else. `false` exits with a non-0 status (often -1),
+and does nothing else.  These can be used as nops, or to create
+infinite loops:
+
+    while true
+    do
+        # ...
+    done
+
+    until false
+    do
+        # ...
+    done
+
+
+`yes`
+-----
+
+This program is similar to the file `/dev/zero`, in that it will
+keep providing output as long as you read it. Rather than producing
+nulls, it produces an infinite stream of lines containing the
+character "y". This can be useful for scripting with tools that
+require confirmation.
+
+
+`seq`
+-----
+
+This produces a sequence of numbers, optionally with a starting
+point and increment. Compare the following:
+
+    seq 5
+    seq 1 5
+    seq 1 2 5
+    seq 5 1
+    seq 5 -2 1
+
+See the manpage for other options, including more complex formatting.
+
+This is useful in scripts to provide a loop over indices:
+
+    for a in $(seq 0 5)
+    do
+        # ...
+    done
+
+
+`tar`
+-----
+
+On Unix systems, `tar` (Tape Archive) is used more frequently than
+`zip`, so it's worth learning to use.
+
+| *Command*              | *Meaning*                                     |
+| ---------              | ---------                                     |
+| `tar cf foo.tar foo/`  | Create a tar file named `foo.tar` from `foo/` |
+| `tar zcf foo.tgz foo/` | As above, but the file will be gzipped        |
+| `tar jcf foo.tbz foo/` | As above, but the file will be bzipped        |
+| `tar xf foo.tar`       | Extract the contacts of `foo.tar`             |
+|                        | Also works on gzipped and bzipped files       |
+| `tar tf foo.tar`       | Read the table of contents of `foo.tar`       |
+
+There are many other options, but these will get you far.
+
+
+`cut`
+-----
+
+This is a workhorse for splitting lines of text.
+
+    cut -d, -f2 foo.csv      # get column 2 from a comma-separated list
+    cut -d, -f2,4-7 foo.csv  # get columns 2, 4, 5, 6, and 7
+    ifconfig | grep flags | cut -d\< -f2 | cut -d\> -f1
+
+
+`awk`
+-----
+
+cut is somewhat limited, so a more powerful tool is frequently
+useful.  awk has a full programming language, but you'll typically
+only need a few pieces of it.
+
+By default, awk splits on whitespace, but you can change this with
+the `-F` option, which takes a regex, rather than a single character.
+A typical invocation would look like:
+
+    awk '{ print $1,$3 }' foo.txt
+
+to print columns 1 and 3 from foo.txt.
+
+You can also do math in awk, which makes it a useful supplement to
+bash's integer math. For example:
+
+    total=$(echo ${total} ${s} | awk '{ print $1 + $2 }')
+
+This allows us to sum potentially floating-point numbers. We could
+also do this by assigning values to variables:
+
+    total=$(echo | awk -v a=${total} b=${s} '{print a + b }')
+
+We still have to pass a file to awk, because it's expecting to
+operate on a file. Fortunately, echo is fairly light-weight.
+
+Here's an example from a script that updates a single column in a
+CSV, re-sums the values, and dumps the results. It also strips off
+a trailing comma, using another utility called `sed` (see the
+manpage).
+
+    echo $LINE | awk -v s=${score} -F\, '{
+            $5=s
+            for (i=3; i<=7; i++) SUM+=$i;
+            for (i=1; i<=NF; i++){
+                    if(i == 2) $i=SUM
+                    printf "%s,",$i
+            }
+            print ""
+    }' | sed 's/,$//g'
+
+This overwrites one of the input fields in the line
+
+            $5=s
+
+The first time we add to the variable `SUM`, it's initialized to
+0.  The `printf` command works pretty much the same as in C.
+