Services with systemd: run, restart, read the logs
Start, stop, enable and inspect services with systemctl, find out why one failed with journalctl, and turn your own app into a service that starts at boot and restarts when it crashes.
By the end of this lesson you'll be able to
0 of 6 completeWhat systemd does
On almost every modern Linux server, systemd is PID 1: the first process, which starts and
looks after everything else. Each thing it manages is a unit, described by a small text file.
The ones you'll use most are services (nginx.service, ssh.service), timers (scheduled
jobs) and targets (groups of units, like "everything needed for a multi-user server").
You talk to systemd with systemctl, and read what services printed with journalctl.
Reading systemctl status
$ systemctl status nginx ● nginx.service - A high performance web server and a reverse proxy server Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; preset: enabled) Active: active (running) since Tue 2026-10-13 09:12:04 IST; 2h 9min ago Docs: man:nginx(8) Main PID: 812 (nginx) Tasks: 3 (limit: 4532) Memory: 5.1M (peak: 6.0M) CGroup: /system.slice/nginx.service ├─812 "nginx: master process /usr/sbin/nginx" └─813 "nginx: worker process" Oct 13 09:12:04 web-01 systemd[1]: Started nginx.service - A high performance web server…
systemctl status nginxsystemctlcontrol systemd services
| Line | Tells you |
|---|---|
| Loaded | which unit file is used, and enabled (starts at boot) or disabled |
| Active | active (running), inactive (dead) (stopped), or failed, plus since when |
| Main PID | the process systemd watches; the CGroup lists all its processes |
| last lines | the newest log lines from this service |
The dot at the start is green when running and red when failed. systemctl is-active nginx and
systemctl is-enabled nginx answer with one word, which is handy in scripts.
Start, stop, enable and reload
$ sudo systemctl start nginx # run it now $ sudo systemctl stop nginx # stop it now $ sudo systemctl restart nginx # stop, then start: a short outage $ sudo systemctl reload nginx # re-read the config without stopping (if supported) $ sudo systemctl enable nginx # start at every boot (doesn't start it now) $ sudo systemctl enable --now nginx # both: now and at every boot $ sudo systemctl disable nginx # don't start at boot $ systemctl --failed # every unit that has failed $ systemctl list-units --type=service --state=running
sudo systemctl start nginxsudorun the next command as rootsystemctlcontrol systemd services
Enabled and running are independent. A service can run without being enabled (gone after a reboot) or be enabled while stopped (starts at the next boot). Check both after every change.
Prefer reload when the service supports it: nginx and sshd re-read their config and keep
existing connections. restart drops them. And always validate first (nginx -t, sshd -t), as
in the Editing files lesson.
Why did it fail? journalctl
$ systemctl status myapp # the last ~10 lines, often enough $ sudo journalctl -u myapp -n 50 # the last 50 lines from this unit $ sudo journalctl -u myapp -f # follow live while you restart it $ sudo journalctl -u myapp --since "10 min ago" $ sudo journalctl -u myapp -b # only since the last boot $ sudo journalctl -p err -b # errors from everything since boot $ sudo journalctl -u myapp -o cat # just the messages, no timestamps
systemctl status myappsystemctlcontrol systemd services
sudo journalctl -u myapp -n 50sudorun the next command as rootjournalctlread the systemd journal (logs)-uone service-nhow many lines
sudo journalctl -u myapp -f-ffollow new entries
sudo journalctl -u myapp -b-bsince this boot
sudo journalctl -p err -b-pminimum priority, like err
The usual reasons a service fails, and what the journal says:
| Journal says | Usually means |
|---|---|
status=203/EXEC | the ExecStart path is wrong or the file isn't executable |
status=217/USER | the User= account doesn't exist |
Address already in use | another process is on that port: sudo ss -tlnp |
Permission denied on a file | the service's user can't read it: check owner and mode |
Start request repeated too quickly | it crashed several times in a row and systemd gave up: fix the cause, then systemctl reset-failed myapp |
On many servers the journal is kept in memory and lost at reboot. To keep it, make sure
/var/log/journal/ exists (sudo mkdir -p /var/log/journal && sudo systemctl restart systemd-journald). journalctl --disk-usage and sudo journalctl --vacuum-time=14d keep its size
in check.
Your own service: a unit file
Running your app with nohup or inside tmux works until the server reboots or the app crashes.
A unit file fixes both:
# /etc/systemd/system/myapp.service [Unit] Description=My web app After=network-online.target Wants=network-online.target [Service] User=myapp WorkingDirectory=/srv/myapp EnvironmentFile=/etc/myapp/myapp.env ExecStart=/usr/bin/python3 -m http.server 8080 Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target
$ sudo systemctl daemon-reload # after creating or editing any unit file $ sudo systemctl enable --now myapp $ systemctl status myapp
sudo systemctl daemon-reloadsudorun the next command as rootsystemctlcontrol systemd services
The details that matter:
User=runs it as the service account from the Users and groups lesson, never as root.ExecStart=needs the full path to the program. systemd doesn't use your shell'sPATH, and doesn't run a shell: writeExecStart=/bin/bash -c '…'if you need pipes or&&.EnvironmentFile=keeps settings and secrets out of the unit file (chmod 600it).Restart=on-failurebrings the app back after a crash or a kill, but not after you stop it.WantedBy=multi-user.targetis whatenablehooks into: "start this with the normal server".
Changing a packaged service: overrides
Never edit the files in /usr/lib/systemd/system/: they belong to the package and are replaced on
the next upgrade. Add an override instead:
$ sudo systemctl edit nginx
sudo systemctl edit nginxsudorun the next command as rootsystemctlcontrol systemd services
# opens /etc/systemd/system/nginx.service.d/override.conf [Service] LimitNOFILE=65535 Restart=on-failure
$ systemctl cat nginx # the package's file plus your override, as systemd sees them $ sudo systemctl restart nginx
systemctl cat nginxsystemctlcontrol systemd services
sudo systemctl restart nginxsudorun the next command as root
systemctl edit runs daemon-reload for you. Files in /etc/systemd/system/ always win over
/usr/lib/systemd/system/.
Scheduled jobs: timers and cron
crontab -e is still the quickest way to run something on a schedule:
# m h dom mon dow command 30 2 * * * /srv/scripts/backup.sh >> /var/log/backup.log 2>&1
systemd timers do the same job with more control: logs in the journal, no overlap if the last
run hasn't finished, and catch-up runs after downtime (Persistent=true). A timer starts a
service of the same name:
# /etc/systemd/system/backup.timer (runs backup.service) [Timer] OnCalendar=*-*-* 02:30:00 Persistent=true [Install] WantedBy=timers.target
$ sudo systemctl enable --now backup.timer $ systemctl list-timers # every timer, with its next and last run
sudo systemctl enable --now backup.timersudorun the next command as rootsystemctlcontrol systemd services
Make an app survive crashes and reboots
A small web app is started by hand with nohup and dies every time the server reboots. Turn it into a service that runs as its own user, starts at boot, and restarts by itself if it crashes. Then prove the restart works.
/etc/systemd/system/myapp.serviceReveal one safe solution
$ sudo useradd --system --no-create-home --shell /usr/sbin/nologin myapp $ sudo mkdir -p /srv/myapp && echo "hello from myapp" | sudo tee /srv/myapp/index.html > /dev/null $ sudo tee /etc/systemd/system/myapp.service > /dev/null <<'EOF' [Unit] Description=My web app After=network-online.target [Service] User=myapp WorkingDirectory=/srv/myapp ExecStart=/usr/bin/python3 -m http.server 8080 Restart=on-failure RestartSec=2 [Install] WantedBy=multi-user.target EOF $ sudo systemctl daemon-reload && sudo systemctl enable --now myapp $ curl -s localhost:8080 hello from myapp $ sudo kill -9 "$(systemctl show -p MainPID --value myapp)" # simulate a crash $ sleep 3; systemctl show -p NRestarts --value myapp 1 $ curl -s localhost:8080 hello from myapp
sudo useradd --system --no-create-home --shell /usr/sbin/nologin myappsudorun the next command as root
sudo mkdir -p /srv/myapp && echo "hello from myapp" | sudo tee /srv/myapp/index.html > /dev/nullmkdirmake a directory-pcreate missing parents, no error if it exists&&run the next command only if this one workedechoprint text|pipe: send output to the next commandteewrite input to a file and pass it on>write output to a file, replacing it
sudo systemctl daemon-reload && sudo systemctl enable --now myappsystemctlcontrol systemd services
curl -s localhost:8080curlmake an HTTP request-ssilent: no progress bar
sudo kill -9 "$(systemctl show -p MainPID --value myapp)"killsend a signal to a process
NRestarts=1 is systemd telling you it noticed the crash and brought the app back, with nobody
paged. enable means the same happens after a reboot. Clean up with sudo systemctl disable --now myapp.
Try it yourself
0 of 6 steps doneRun each task in your own account or terminal, then tick it off.
Quick check
You installed nginx and ran sudo systemctl start nginx. After a reboot it isn't running. Why?
Official reading for this lesson
Or get every quiz answer right and it completes itself.