Loops

Almost every useful script is a loop. Do this to every file. Retry until it works. Read the list and act on each line. The shell gives you three loop forms, and two of them cover nearly everything.

for walks a list of words

for FILE in *.log
do
  echo "Compressing $FILE"
  gzip "$FILE"
done

for takes a list of words — not a range, not a collection, just words separated by whitespace — and runs the body once per word with the variable set to it.

That list can come from anywhere the shell can produce words: a glob, a literal list, a variable, the output of a command.

The semicolon form is the same loop written on one line — do needs a separator before it exactly as then did.

The empty-glob trap

This is the one thing to know about for before you write a real one:

for FILE in *.log
do
  rm "$FILE"
done

In a directory with no .log files, a POSIX shell leaves the unmatched pattern alone. The loop runs once, with FILE set to the literal string *.log, and rm fails complaining about a file named *.log.

With rm that's merely noisy. With something that creates or moves files, you get a real file called *.log on disk, and a * in a filename is a genuinely unpleasant thing to clean up.

The fix is to check:

for FILE in *.log
do
  [ -e "$FILE" ] || continue
  rm "$FILE"
done

while repeats until a command fails

Where for walks a known list, while repeats as long as a command keeps succeeding:

COUNT=1
while [ "$COUNT" -le 5 ]
do
  echo "Attempt $COUNT"
  COUNT=$((COUNT + 1))
done

$(( ... )) is arithmetic expansion — the shell's built-in integer maths. It is the portable way to count; the older expr runs an external program for every single increment.

The condition is a command, same as with if. Which makes retry loops read almost like English:

until curl -sf http://localhost:8080/health
do
  echo "Waiting for the service..."
  sleep 2
done
echo "Up."

until is while with the sense inverted: it loops while the command fails. Both exist because sometimes one of them reads better than negating the other.

Reading a file line by line

This is the pattern worth committing to memory, because the obvious alternative is wrong:

while read -r LINE
do
  echo "Line: $LINE"
done < servers.txt

The redirect goes on done, feeding the whole loop from the file. read consumes one line per iteration and returns non-zero at end of file, which is what stops the loop.

-r stops read from interpreting backslashes in the data. There is essentially never a reason to omit it.

break and continue

break leaves the loop; continue skips to the next iteration. Both are what turn a loop into something with a shape:

for HOST in web-01 web-02 web-03
do
  ping -c1 -W1 "$HOST" >/dev/null 2>&1 || continue
  echo "$HOST is up"
done

A loop that does something real

Putting the lesson together — compress yesterday's logs, skipping anything already compressed:

#!/bin/sh
LOGDIR=/var/log/myapp

for FILE in "$LOGDIR"/*.log
do
  [ -e "$FILE" ] || continue          # empty-glob guard
  [ -s "$FILE" ] || continue          # skip empty files

  echo "Compressing $FILE"
  if gzip "$FILE"; then
    echo "  ok"
  else
    echo "  FAILED: $FILE" >&2
  fi
done

Note >&2 on the failure line — that sends it to standard error, so a human watching sees the problem while a pipeline collecting the normal output stays clean.

Next up: case — the readable way to branch on a value, and how scripts dispatch on their first argument.