Lesson 14 · Shell scripting

Shell scripting: automate it once, run it forever

Bash scripts that are safe in production: variables and quoting, if and case, loops, functions and exit codes, strict mode, debugging, and running on a schedule.

45 min read · intermediate · hands-on

By the end of this lesson you'll be able to

0 of 6 complete

Write a script with a shebang, run it, and pass it arguments
Use variables, defaults and quoting without the classic word-splitting bugs
Branch with if, [[ ]] and case, and loop over files, lines and numbers
Write functions, return exit codes, and stop on errors with set -euo pipefail
Debug a script with bash -x and check it with ShellCheck
Schedule a script with cron or a systemd timer

Your first script

A script is the commands you'd type, saved in a file. If you've run the same five commands twice, they belong in a script.

#!/usr/bin/env bash
# hello.sh: say hello to a name, or to the user running it
name=${1:-$USER}
echo "Hello, $name. Today is $(date +%A)."

$ chmod +x hello.sh               # make it executable (once)
$ ./hello.sh                      # ./ because the current folder isn't in $PATH
Hello, rajeev. Today is Tuesday.
$ ./hello.sh team
Hello, team. Today is Tuesday.

chmod +x hello.sh

chmodchange permissions

The first line, the shebang, says which program runs the file. #!/usr/bin/env bash finds bash wherever it's installed. Use bash, not sh, when you use bash features such as [[ ]] and arrays: on Ubuntu sh is dash, which doesn't have them.

The anatomy of a production script. The shebang line. set -euo pipefail, strict mode. Arguments with defaults and a usage message. Small functions such as log and die. Checks before doing anything. The work itself. Exit codes: 0 for success, non-zero for failure, which cron, systemd and CI read. On the right, what each part protects against. — The same shape works for almost every ops script: strict mode, inputs, checks, work, a clear exit code.© Diagram: DevOps Academy

Variables, arguments and quoting

$ app=myapp                       # no spaces around =
$ echo "$app is in /srv/$app"
myapp is in /srv/myapp
$ release=$(date +%Y%m%d)         # the output of a command
$ count=$((3 + 4))                # arithmetic
$ echo "${app}_v2"                # braces where the name would run on
myapp_v2

echo "$app is in /srv/$app"

echoprint text

In a script	Means
`$1`, `$2` …	arguments
`$#`	how many arguments
`"$@"`	all arguments, each kept whole
`$?`	exit code of the last command
`$$`	this script's PID
`${1:-7}`	`$1`, or `7` if it's empty
`${1:?usage: …}`	`$1`, or stop with that message

Quote every variable: "$file", not $file. Without quotes, bash splits the value on spaces and expands *:

$ x="a b"
$ [ $x = "a b" ]
bash: [: too many arguments
$ [ "$x" = "a b" ] && echo match
match

[ "$x" = "a b" ] && echo match

&&run the next command only if this one worked
echoprint text

Single quotes keep everything literal ('$HOME' prints $HOME); double quotes still expand variables and $( ).

Conditionals: if, test and case

if runs a command and checks its exit code. [[ … ]] is a command that tests things:

if [[ -f /etc/myapp.env ]]; then
  echo "config found"
elif [[ -d /etc/myapp ]]; then
  echo "folder but no config"
else
  echo "not installed" >&2
  exit 1
fi

if grep -q "^deploy:" /etc/passwd; then echo "deploy user exists"; fi
[[ $(id -u) -eq 0 ]] || { echo "run as root" >&2; exit 1; }

Test	True when
`-f file` / `-d dir` / `-e path`	it's a file / a folder / exists
`-r` `-w` `-x file`	readable / writable / executable
`-z "$s"` / `-n "$s"`	string is empty / not empty
`"$a" == "$b"` / `!=`	strings equal / differ
`$n -eq 5` `-ne` `-lt` `-le` `-gt` `-ge`	number comparisons
`"$s" =~ ^v[0-9]+`	matches a regular expression

case is the clean way to handle a fixed set of words, such as subcommands:

case "$1" in
  start|up) systemctl start myapp ;;
  stop)     systemctl stop myapp ;;
  *)        echo "usage: $0 start|stop" >&2; exit 2 ;;
esac

Loops

for f in /etc/*.conf; do                  # over files: a glob, never $(ls)
  echo "$f: $(wc -l < "$f") lines"
done

for host in web1 web2 web3; do            # over a list
  ssh "$host" uptime
done

for i in {1..5}; do echo "try $i"; done   # over numbers

while read -r user shell; do              # over lines of a file or command
  echo "$user uses $shell"
done < <(awk -F: '$7 ~ /bash/ {print $1, $7}' /etc/passwd)

until curl -fs http://localhost:8080/health > /dev/null; do   # retry until it works
  sleep 2
done

read -r keeps backslashes as they are; always use it. break leaves a loop and continue skips to the next round.

Functions and exit codes

log() { echo "$(date +%T) $*" >&2; }      # messages go to stderr, data to stdout
die() { log "ERROR: $*"; exit 1; }

is_up() {
  local url=$1                            # local: doesn't leak out of the function
  curl -fs -m 5 "$url" > /dev/null        # the function returns curl's exit code
}

if is_up http://localhost:8080/health; then log "healthy"; else die "app is down"; fi

Every command, function and script ends with an exit code: 0 for success, 1–255 for failure. That number is what if, &&, ||, cron, systemd and CI pipelines read. A script that fails but exits 0 lies to all of them, so end with exit 1 (or die) when something went wrong.

Strict mode

Put this on the second line of every script:

set -euo pipefail

Flag	Without it
`-e`	a failed command is ignored and the script continues
`-u`	a typo such as `$BACKUP_DRI` silently becomes empty: `rm -rf "$BACKUP_DRI"/*`
`-o pipefail`	`false \| true` counts as success, hiding the failure on the left

When a failure is expected, say so: grep -q x file || true, or test it in an if. One -e trap: ((count++)) returns failure when count is 0, so write count=$((count + 1)).

Debugging a script

$ bash -n deploy.sh               # syntax check only, runs nothing
$ bash -x deploy.sh               # print every command as it runs
+ n=3
+ echo 'n is 3'
n is 3

Each + line is a command after its variables were filled in, so you see exactly what ran. Turn it on for one section with set -x … set +x. Before you ship a script, run ShellCheck (sudo apt install shellcheck, then shellcheck deploy.sh): it catches missing quotes and most of the bugs in this lesson.

Running scripts on a schedule

cron runs commands at set times. Edit your table with crontab -e:

# minute hour day-of-month month day-of-week  command
30 2 * * *     /opt/scripts/backup.sh /srv/app 7 >> /var/log/backup.log 2>&1
*/5 * * * *    /opt/scripts/health.sh
0 9 * * 1      /opt/scripts/weekly-report.sh

That's 02:30 every day, every 5 minutes, and 09:00 on Mondays. Cron runs with a tiny $PATH and no terminal, so use full paths and send the output to a log, or it's lost. On systemd machines a timer (from the Services with systemd lesson) does the same job with logs in journalctl and a record of failures.

Mini mission

A backup script you can trust

Write backup.sh: it takes a folder and how many backups to keep, makes a dated .tar.gz in ~/backups, deletes the oldest ones beyond that number, refuses bad input with a clear message, and exits non-zero on any failure.

Input: Any folder, such as a small siteOutput: ~/backups/<name>-<date>.tar.gz

Reveal one safe solution

#!/usr/bin/env bash
set -euo pipefail

src=${1:?usage: backup.sh <folder> [keep]}
keep=${2:-7}
dest="$HOME/backups"
stamp=$(date +%Y%m%d-%H%M%S)

log() { echo "$(date +%T) $*" >&2; }
die() { log "ERROR: $*"; exit 1; }

[[ -d $src ]] || die "$src is not a folder"
mkdir -p "$dest"

name=$(basename "$src")
archive="$dest/$name-$stamp.tar.gz"
tar -czf "$archive" -C "$(dirname "$src")" "$name"
log "created $archive ($(du -h "$archive" | cut -f1))"

count=0
for old in $(ls -1t "$dest/$name"-*.tar.gz | tail -n +$((keep + 1))); do
  rm -- "$old"
  count=$((count + 1))
done
log "removed $count old backup(s), keeping $keep"

$ chmod +x backup.sh
$ ./backup.sh site 2              # run it three times
05:35:09 created /home/rajeev/backups/site-20261013-053509.tar.gz (4.0K)
05:35:09 removed 1 old backup(s), keeping 2
$ ls ~/backups | wc -l
2
$ ./backup.sh nope; echo "exit $?"
05:35:09 ERROR: nope is not a folder
exit 1

chmod +x backup.sh

chmodchange permissions

ls ~/backups | wc -l

lslist files
|pipe: send output to the next command
wccount lines, words and bytes
-lcount lines only

Archive names have no spaces, so the $(ls …) loop is safe here; for arbitrary file names use find … -print0 with while read -r -d ''. To run it nightly: crontab -e and add 30 2 * * * /home/rajeev/backup.sh /srv/site 7 >> /home/rajeev/backup.log 2>&1.

Try it yourself

0 of 6 steps done

Run each task in your own account or terminal, then tick it off.

Write a three-line script with #!/usr/bin/env bash, make it executable with chmod +x, and run it with ./.
Run x="a b"; [ $x = "a b" ] and read the error, then run it again with "$x".
Print $? after true, after false, and after grep -q root /etc/passwd.
Loop over every .conf file in /etc and print its name and line count.
Run any script with bash -x and read the + lines.
Run crontab -l, then read the fields of a cron line in this lesson.

Knowledge check

Quick check

Question 1 of 5

What does $? hold?

Go deeper

Official reading for this lesson

Or get every quiz answer right and it completes itself.

Lesson 14 · Shell scripting

Shell scripting: automate it once, run it forever

Bash scripts that are safe in production: variables and quoting, if and case, loops, functions and exit codes, strict mode, debugging, and running on a schedule.

45 min read · intermediate · hands-on

By the end of this lesson you'll be able to

0 of 6 complete

Write a script with a shebang, run it, and pass it arguments
Use variables, defaults and quoting without the classic word-splitting bugs
Branch with if, [[ ]] and case, and loop over files, lines and numbers
Write functions, return exit codes, and stop on errors with set -euo pipefail
Debug a script with bash -x and check it with ShellCheck
Schedule a script with cron or a systemd timer

Your first script

A script is the commands you'd type, saved in a file. If you've run the same five commands twice, they belong in a script.

#!/usr/bin/env bash
# hello.sh: say hello to a name, or to the user running it
name=${1:-$USER}
echo "Hello, $name. Today is $(date +%A)."

$ chmod +x hello.sh               # make it executable (once)
$ ./hello.sh                      # ./ because the current folder isn't in $PATH
Hello, rajeev. Today is Tuesday.
$ ./hello.sh team
Hello, team. Today is Tuesday.

chmod +x hello.sh

chmodchange permissions

Variables, arguments and quoting

$ app=myapp                       # no spaces around =
$ echo "$app is in /srv/$app"
myapp is in /srv/myapp
$ release=$(date +%Y%m%d)         # the output of a command
$ count=$((3 + 4))                # arithmetic
$ echo "${app}_v2"                # braces where the name would run on
myapp_v2

echo "$app is in /srv/$app"

echoprint text

In a script	Means
`$1`, `$2` …	arguments
`$#`	how many arguments
`"$@"`	all arguments, each kept whole
`$?`	exit code of the last command
`$$`	this script's PID
`${1:-7}`	`$1`, or `7` if it's empty
`${1:?usage: …}`	`$1`, or stop with that message

Quote every variable: "$file", not $file. Without quotes, bash splits the value on spaces and expands *:

$ x="a b"
$ [ $x = "a b" ]
bash: [: too many arguments
$ [ "$x" = "a b" ] && echo match
match

[ "$x" = "a b" ] && echo match

&&run the next command only if this one worked
echoprint text

Single quotes keep everything literal ('$HOME' prints $HOME); double quotes still expand variables and $( ).

Conditionals: if, test and case

if runs a command and checks its exit code. [[ … ]] is a command that tests things:

if [[ -f /etc/myapp.env ]]; then
  echo "config found"
elif [[ -d /etc/myapp ]]; then
  echo "folder but no config"
else
  echo "not installed" >&2
  exit 1
fi

if grep -q "^deploy:" /etc/passwd; then echo "deploy user exists"; fi
[[ $(id -u) -eq 0 ]] || { echo "run as root" >&2; exit 1; }

Test	True when
`-f file` / `-d dir` / `-e path`	it's a file / a folder / exists
`-r` `-w` `-x file`	readable / writable / executable
`-z "$s"` / `-n "$s"`	string is empty / not empty
`"$a" == "$b"` / `!=`	strings equal / differ
`$n -eq 5` `-ne` `-lt` `-le` `-gt` `-ge`	number comparisons
`"$s" =~ ^v[0-9]+`	matches a regular expression

case is the clean way to handle a fixed set of words, such as subcommands:

case "$1" in
  start|up) systemctl start myapp ;;
  stop)     systemctl stop myapp ;;
  *)        echo "usage: $0 start|stop" >&2; exit 2 ;;
esac

Loops

for f in /etc/*.conf; do                  # over files: a glob, never $(ls)
  echo "$f: $(wc -l < "$f") lines"
done

for host in web1 web2 web3; do            # over a list
  ssh "$host" uptime
done

for i in {1..5}; do echo "try $i"; done   # over numbers

while read -r user shell; do              # over lines of a file or command
  echo "$user uses $shell"
done < <(awk -F: '$7 ~ /bash/ {print $1, $7}' /etc/passwd)

until curl -fs http://localhost:8080/health > /dev/null; do   # retry until it works
  sleep 2
done

read -r keeps backslashes as they are; always use it. break leaves a loop and continue skips to the next round.

Functions and exit codes

log() { echo "$(date +%T) $*" >&2; }      # messages go to stderr, data to stdout
die() { log "ERROR: $*"; exit 1; }

is_up() {
  local url=$1                            # local: doesn't leak out of the function
  curl -fs -m 5 "$url" > /dev/null        # the function returns curl's exit code
}

if is_up http://localhost:8080/health; then log "healthy"; else die "app is down"; fi

Strict mode

Put this on the second line of every script:

set -euo pipefail

Flag	Without it
`-e`	a failed command is ignored and the script continues
`-u`	a typo such as `$BACKUP_DRI` silently becomes empty: `rm -rf "$BACKUP_DRI"/*`
`-o pipefail`	`false \| true` counts as success, hiding the failure on the left

When a failure is expected, say so: grep -q x file || true, or test it in an if. One -e trap: ((count++)) returns failure when count is 0, so write count=$((count + 1)).

Debugging a script

$ bash -n deploy.sh               # syntax check only, runs nothing
$ bash -x deploy.sh               # print every command as it runs
+ n=3
+ echo 'n is 3'
n is 3

Running scripts on a schedule

cron runs commands at set times. Edit your table with crontab -e:

# minute hour day-of-month month day-of-week  command
30 2 * * *     /opt/scripts/backup.sh /srv/app 7 >> /var/log/backup.log 2>&1
*/5 * * * *    /opt/scripts/health.sh
0 9 * * 1      /opt/scripts/weekly-report.sh

Mini mission

A backup script you can trust

Input: Any folder, such as a small siteOutput: ~/backups/<name>-<date>.tar.gz

Reveal one safe solution

#!/usr/bin/env bash
set -euo pipefail

src=${1:?usage: backup.sh <folder> [keep]}
keep=${2:-7}
dest="$HOME/backups"
stamp=$(date +%Y%m%d-%H%M%S)

log() { echo "$(date +%T) $*" >&2; }
die() { log "ERROR: $*"; exit 1; }

[[ -d $src ]] || die "$src is not a folder"
mkdir -p "$dest"

name=$(basename "$src")
archive="$dest/$name-$stamp.tar.gz"
tar -czf "$archive" -C "$(dirname "$src")" "$name"
log "created $archive ($(du -h "$archive" | cut -f1))"

count=0
for old in $(ls -1t "$dest/$name"-*.tar.gz | tail -n +$((keep + 1))); do
  rm -- "$old"
  count=$((count + 1))
done
log "removed $count old backup(s), keeping $keep"

$ chmod +x backup.sh
$ ./backup.sh site 2              # run it three times
05:35:09 created /home/rajeev/backups/site-20261013-053509.tar.gz (4.0K)
05:35:09 removed 1 old backup(s), keeping 2
$ ls ~/backups | wc -l
2
$ ./backup.sh nope; echo "exit $?"
05:35:09 ERROR: nope is not a folder
exit 1

chmod +x backup.sh

chmodchange permissions

ls ~/backups | wc -l

lslist files
|pipe: send output to the next command
wccount lines, words and bytes
-lcount lines only

Try it yourself

0 of 6 steps done

Run each task in your own account or terminal, then tick it off.

Write a three-line script with #!/usr/bin/env bash, make it executable with chmod +x, and run it with ./.
Run x="a b"; [ $x = "a b" ] and read the error, then run it again with "$x".
Print $? after true, after false, and after grep -q root /etc/passwd.
Loop over every .conf file in /etc and print its name and line count.
Run any script with bash -x and read the + lines.
Run crontab -l, then read the fields of a cron line in this lesson.

Knowledge check

Quick check

Question 1 of 5

What does $? hold?

Go deeper

Official reading for this lesson

Or get every quiz answer right and it completes itself.