DevOps.Academy
CoursesCurriculumPricing

    Loading…

    ↑ ↓ move · ↵ open · Ctrl K anywhere

    ◔My learningSign in
    DevOps.Academy

    Free, hands-on DevOps lessons. Made with care in India.

    Learn

    All coursesLinux roadmapCurriculumMy learning

    Academy

    PricingCertificates

    Legal

    PrivacyTermsContact

    devops.rajeev.pro

    DockerLesson 6 of 14
    Your progress · 0%
    On this page
    1. The container forgets
    2. Named volumes
    3. Bind mounts
    4. Backups: dump, restore, prove it
    ↖ Course roadmap
    Lesson 6 · Data persistence

    Keeping data: volumes, bind mounts and backups

    A container's files die with it. Keep what matters in named volumes, share your code with bind mounts, use tmpfs for scratch space, and back up and restore a database volume, proving each one works.

    35 min read · beginner · hands-on

    By the end of this lesson you'll be able to

    0 of 6 complete

    The container forgets

    Lesson 2 showed that a container writes into its own thin layer, and that layer is deleted with the container. For a web server that's fine. For a database it's a disaster:

    $ docker run -d --name db -e POSTGRES_PASSWORD=devpass postgres:17-alpine
    $ docker exec db psql -h 127.0.0.1 -U postgres -c "create table notes(t text); insert into notes values ('keep me');"
    $ docker rm -f db && docker run -d --name db -e POSTGRES_PASSWORD=devpass postgres:17-alpine
    $ docker exec db psql -h 127.0.0.1 -U postgres -c "select t from notes"
    ERROR:  relation "notes" does not exist
    
    docker run -d --name db -e POSTGRES_PASSWORD=devpass postgres:17-alpine
    • dockerbuild and run containers
    • runstart a new container
    • -drun in the background
    • --name dbname the container
    • -e POSTGRES_PASSWORD=devpassset an environment variable inside
    • postgres:17-alpinethe image to start
    docker exec db psql -h 127.0.0.1 -U postgres -c "create table notes(t text); insert into notes values ('keep me');"
    • execrun a command in a running container
    • dbthe container
    • psql -h 127.0.0.1 -U postgres -c "create table notes(t text); insert into notes values ('keep me');"the command to run inside it
    docker rm -f db && docker run -d --name db -e POSTGRES_PASSWORD=devpass postgres:17-alpine
    • rmdelete a container
    • -fstop it first if running
    • &&run the next command only if this one worked
    docker exec db psql -h 127.0.0.1 -U postgres -c "select t from notes"
    • psql -h 127.0.0.1 -U postgres -c "select t from notes"the command to run inside it

    The fix is to keep data outside the container, in storage that outlives it:

    The same commands twice. Only the second keeps the data, because it lives in a volume, not in the container.© Animation: DevOps Academy

    The same commands twice. Only the second keeps the data, because it lives in a volume, not in the container.© Animation: DevOps Academy
    TypeLivesUse for
    Named volume -v pgdata:/pathDocker's storage, managed by Dockerdatabase data, anything to keep
    Bind mount -v "$PWD"/site:/pathan exact folder on the hostyour code during development, config files
    tmpfs --tmpfs /pathmemory onlyscratch and sensitive temporary files

    Named volumes

    $ docker volume create pgdata
    $ docker run -d --name db -e POSTGRES_PASSWORD=devpass -v pgdata:/var/lib/postgresql/data postgres:17-alpine
    $ docker exec db psql -h 127.0.0.1 -U postgres -c "create table notes(t text); insert into notes values ('keep me');"
    $ docker rm -f db
    $ docker run -d --name db -e POSTGRES_PASSWORD=devpass -v pgdata:/var/lib/postgresql/data postgres:17-alpine
    $ docker exec db psql -h 127.0.0.1 -U postgres -tAc "select t from notes"
    keep me
    $ docker volume ls --filter name=pgdata
    DRIVER    VOLUME NAME
    local     pgdata
    $ docker volume inspect pgdata --format '{{.Mountpoint}}'
    /var/lib/docker/volumes/pgdata/_data
    
    docker volume create pgdata
    • dockerbuild and run containers
    • volumemanage volumes
    • createcreate one
    docker run -d --name db -e POSTGRES_PASSWORD=devpass -v pgdata:/var/lib/postgresql/data postgres:17-alpine
    • runstart a new container
    • -drun in the background
    • --name dbname the container
    • -e POSTGRES_PASSWORD=devpassset an environment variable inside
    • -v pgdata:/var/lib/postgresql/datamount a volume or folder, source:path-inside
    • postgres:17-alpinethe image to start
    docker exec db psql -h 127.0.0.1 -U postgres -c "create table notes(t text); insert into notes values ('keep me');"
    • execrun a command in a running container
    • dbthe container
    • psql -h 127.0.0.1 -U postgres -c "create table notes(t text); insert into notes values ('keep me');"the command to run inside it
    docker rm -f db
    • rmdelete a container
    • -fstop it first if running
    docker exec db psql -h 127.0.0.1 -U postgres -tAc "select t from notes"
    • psql -h 127.0.0.1 -U postgres -tAc "select t from notes"the command to run inside it
    docker volume ls --filter name=pgdata
    • lslist them
    • --filter name=pgdatashow only matching items
    docker volume inspect pgdata --format '{{.Mountpoint}}'
    • inspectshow details as JSON
    • --format '{{.Mountpoint}}'print only these fields (Go template)

    The path inside the container (/var/lib/postgresql/data) comes from the image's documentation. Get it wrong and the data quietly lands in the container layer again.

    Anonymous volumes pile up

    The postgres image declares its data folder as a volume:

    $ docker image inspect postgres:17-alpine --format '{{json .Config.Volumes}}'
    {"/var/lib/postgresql/data":{}}
    
    docker image inspect postgres:17-alpine --format '{{json .Config.Volumes}}'
    • dockerbuild and run containers
    • imagemanage images
    • inspectshow details as JSON
    • --format '{{json .Config.Volumes}}'print only these fields (Go template)

    So every run without -v for that path creates a new anonymous volume, named by a 64-character ID. After this lesson's experiments, docker volume ls here listed ten of them. They hold old data, use disk, and survive docker rm unless you add -v to it (docker rm -v) or run the container with --rm. Lesson 12 cleans them up.

    Bind mounts

    A bind mount maps an exact host folder into the container. Edit on your machine, see it inside at once:

    $ mkdir -p site && echo '<h1>v1</h1>' > site/index.html
    $ docker run -d --name web -p 8080:80 -v "$PWD/site":/usr/share/nginx/html:ro nginx:1.30-alpine
    $ curl -s localhost:8080
    <h1>v1</h1>
    $ echo '<h1>v2, edited on the host</h1>' > site/index.html
    $ curl -s localhost:8080
    <h1>v2, edited on the host</h1>
    $ docker exec web sh -c 'echo x > /usr/share/nginx/html/hack.html'
    sh: can't create /usr/share/nginx/html/hack.html: Read-only file system
    
    mkdir -p site && echo '<h1>v1</h1>' > site/index.html
    • mkdirmake a directory
    • -pcreate missing parents, no error if it exists
    • &&run the next command only if this one worked
    • echoprint text
    • >write output to a file, replacing it
    docker run -d --name web -p 8080:80 -v "$PWD/site":/usr/share/nginx/html:ro nginx:1.30-alpine
    • dockerbuild and run containers
    • runstart a new container
    • -drun in the background
    • --name webname the container
    • -p 8080:80publish a port, host:container
    • -v "$PWD/site":/usr/share/nginx/html:romount a volume or folder, source:path-inside
    • nginx:1.30-alpinethe image to start
    curl -s localhost:8080
    • curlmake an HTTP request
    • -ssilent: no progress bar
    docker exec web sh -c 'echo x > /usr/share/nginx/html/hack.html'
    • execrun a command in a running container
    • webthe container
    • sh -c 'echo x > /usr/share/nginx/html/hack.html'the command to run inside it

    :ro makes it read-only inside: the container can serve the files but never change them. Use it unless the container must write. Bind mounts are the heart of a good development setup (lesson 13), but they tie a container to one host's folder layout and file permissions: files a container writes into a bind mount are owned by the container's user ID, often root.

    --mount is the longer, explicit syntax, clearer in scripts:

    $ docker run --mount type=bind,src="$PWD/site",dst=/usr/share/nginx/html,readonly nginx:1.30-alpine
    $ docker run --mount type=volume,src=pgdata,dst=/var/lib/postgresql/data …
    
    docker run --mount type=bind,src="$PWD/site",dst=/usr/share/nginx/html,readonly nginx:1.30-alpine
    • dockerbuild and run containers
    • runstart a new container
    • --mount type=bind,src="$PWD/site",dst=/usr/share/nginx/html,readonlymount a volume or folder, long form
    • nginx:1.30-alpinethe image to start
    docker run --mount type=volume,src=pgdata,dst=/var/lib/postgresql/data …
    • --mount type=volume,src=pgdata,dst=/var/lib/postgresql/datamount a volume or folder, long form
    • …the image to start

    tmpfs: memory only

    $ docker run --rm --tmpfs /scratch alpine:3.22 sh -c 'echo hi > /scratch/a && df -h /scratch | tail -1'
    tmpfs                     1.8G      4.0K      1.8G   0% /scratch
    
    docker run --rm --tmpfs /scratch alpine:3.22 sh -c 'echo hi > /scratch/a && df -h /scratch | tail -1'
    • dockerbuild and run containers
    • runstart a new container
    • --rmdelete the container when it exits
    • --tmpfs /scratcha folder kept in memory only
    • alpine:3.22the image to start
    • sh -c 'echo hi > /scratch/a && df -h /scratch | tail -1'the command run inside, instead of the image’s default

    Nothing written there ever reaches the disk, and it vanishes when the container stops: good for temporary files that may contain secrets.

    Backups: dump, restore, prove it

    A volume is still just storage on one disk. Back it up, and prove the backup by restoring it:

    $ docker exec db pg_dump -h 127.0.0.1 -U postgres --clean --if-exists postgres > backup.sql
    $ docker run -d --name db3 -e POSTGRES_PASSWORD=devpass -v pgrestore:/var/lib/postgresql/data postgres:17-alpine
    $ docker exec -i db3 psql -q -h 127.0.0.1 -U postgres < backup.sql > /dev/null
    $ docker exec db3 psql -h 127.0.0.1 -U postgres -tAc "select t from notes"
    keep me
    
    docker exec db pg_dump -h 127.0.0.1 -U postgres --clean --if-exists postgres > backup.sql
    • dockerbuild and run containers
    • execrun a command in a running container
    • dbthe container
    • pg_dump -h 127.0.0.1 -U postgres --clean --if-exists postgresthe command to run inside it
    • >write output to a file, replacing it
    • backup.sqlthe command to run inside it
    docker run -d --name db3 -e POSTGRES_PASSWORD=devpass -v pgrestore:/var/lib/postgresql/data postgres:17-alpine
    • runstart a new container
    • -drun in the background
    • --name db3name the container
    • -e POSTGRES_PASSWORD=devpassset an environment variable inside
    • -v pgrestore:/var/lib/postgresql/datamount a volume or folder, source:path-inside
    • postgres:17-alpinethe image to start
    docker exec -i db3 psql -q -h 127.0.0.1 -U postgres < backup.sql > /dev/null
    • -ikeep standard input open
    • db3the container
    • psql -q -h 127.0.0.1 -U postgresthe command to run inside it
    • <read input from a file
    • /dev/nullthe command to run inside it
    docker exec db3 psql -h 127.0.0.1 -U postgres -tAc "select t from notes"
    • psql -h 127.0.0.1 -U postgres -tAc "select t from notes"the command to run inside it

    For any volume, a tar archive works, with the container stopped so files aren't changing mid-copy:

    $ docker stop db
    $ docker run --rm -v pgdata:/data:ro -v "$PWD":/backup alpine:3.22 tar czf /backup/pgdata.tgz -C /data .
    
    docker stop db
    • dockerbuild and run containers
    • stopstop a container (SIGTERM, then SIGKILL)
    • dbthe container
    docker run --rm -v pgdata:/data:ro -v "$PWD":/backup alpine:3.22 tar czf /backup/pgdata.tgz -C /data .
    • runstart a new container
    • --rmdelete the container when it exits
    • -v pgdata:/data:romount a volume or folder, source:path-inside
    • -v "$PWD":/backupmount a volume or folder, source:path-inside
    • alpine:3.22the image to start
    • tar czf /backup/pgdata.tgz -C /data .the command run inside, instead of the image’s default

    The helper container mounts the volume read-only and your folder as /backup, writes the archive and is removed.

    A backup you haven't restored isn't a backup

    Prefer the database's own tool (pg_dump, mysqldump) over copying files: it gives a consistent snapshot while the database runs. Store backups somewhere else (another machine, object storage), and restore one into a fresh volume on a schedule, checking the data, so you know it works before the day you need it.

    ⌘
    Mini mission

    Keep a database across containers, and prove your backup

    Run Postgres with a named volume, add a row, replace the container, and show the row survived. Then back up with pg_dump, restore into a brand-new volume and container, and query the restored data. Finally, remove what you created.

    Input: DockerOutput: 'keep me' read back after a container replacement and after a restore
    Reveal one safe solution
    $ docker volume create pgdata
    $ docker run -d --name db -e POSTGRES_PASSWORD=devpass -v pgdata:/var/lib/postgresql/data postgres:17-alpine
    $ until docker exec db pg_isready -h 127.0.0.1 -U postgres >/dev/null; do sleep 1; done
    $ docker exec db psql -h 127.0.0.1 -U postgres -c "create table notes(t text); insert into notes values ('keep me');"
    $ docker rm -f db
    $ docker run -d --name db -e POSTGRES_PASSWORD=devpass -v pgdata:/var/lib/postgresql/data postgres:17-alpine
    $ until docker exec db pg_isready -h 127.0.0.1 -U postgres >/dev/null; do sleep 1; done
    $ docker exec db psql -h 127.0.0.1 -U postgres -tAc "select t from notes"
    keep me
    $ docker exec db pg_dump -h 127.0.0.1 -U postgres --clean --if-exists postgres > backup.sql
    $ docker run -d --name restored -e POSTGRES_PASSWORD=devpass -v pgrestore:/var/lib/postgresql/data postgres:17-alpine
    $ until docker exec restored pg_isready -h 127.0.0.1 -U postgres >/dev/null; do sleep 1; done
    $ docker exec -i restored psql -q -h 127.0.0.1 -U postgres < backup.sql > /dev/null
    $ docker exec restored psql -h 127.0.0.1 -U postgres -tAc "select t from notes"
    keep me
    $ docker rm -f db restored && docker volume rm pgdata pgrestore
    
    docker volume create pgdata
    • dockerbuild and run containers
    • volumemanage volumes
    • createcreate one
    docker run -d --name db -e POSTGRES_PASSWORD=devpass -v pgdata:/var/lib/postgresql/data postgres:17-alpine
    • runstart a new container
    • -drun in the background
    • --name dbname the container
    • -e POSTGRES_PASSWORD=devpassset an environment variable inside
    • -v pgdata:/var/lib/postgresql/datamount a volume or folder, source:path-inside
    • postgres:17-alpinethe image to start
    until docker exec db pg_isready -h 127.0.0.1 -U postgres >/dev/null; do sleep 1; done
    • dostart of what the loop runs each time
    • doneend of the loop
    docker exec db psql -h 127.0.0.1 -U postgres -c "create table notes(t text); insert into notes values ('keep me');"
    • execrun a command in a running container
    • dbthe container
    • psql -h 127.0.0.1 -U postgres -c "create table notes(t text); insert into notes values ('keep me');"the command to run inside it
    docker rm -f db
    • rmdelete a container
    • -fstop it first if running
    docker exec db psql -h 127.0.0.1 -U postgres -tAc "select t from notes"
    • psql -h 127.0.0.1 -U postgres -tAc "select t from notes"the command to run inside it
    docker exec db pg_dump -h 127.0.0.1 -U postgres --clean --if-exists postgres > backup.sql
    • pg_dump -h 127.0.0.1 -U postgres --clean --if-exists postgresthe command to run inside it
    • >write output to a file, replacing it
    • backup.sqlthe command to run inside it
    docker run -d --name restored -e POSTGRES_PASSWORD=devpass -v pgrestore:/var/lib/postgresql/data postgres:17-alpine
    • --name restoredname the container
    • -v pgrestore:/var/lib/postgresql/datamount a volume or folder, source:path-inside
    docker exec -i restored psql -q -h 127.0.0.1 -U postgres < backup.sql > /dev/null
    • -ikeep standard input open
    • restoredthe container
    • psql -q -h 127.0.0.1 -U postgresthe command to run inside it
    • <read input from a file
    • /dev/nullthe command to run inside it
    docker rm -f db restored && docker volume rm pgdata pgrestore
    • &&run the next command only if this one worked
    • rmdelete one

    Two separate proofs: the data outlived a container, and the backup file alone rebuilt it in a new database. That second one is the habit that saves you.

    Try it yourself

    0 of 6 steps done

    Run each task in your own account or terminal, then tick it off.

    Knowledge check

    Quick check

    Question 1 of 5

    A Postgres container without a volume is removed and recreated. What happens to the data?

    Go deeper

    Official reading for this lesson

    • DocsDocker Docs: volumesDocker Docs ↗
    • DocsDocker Docs: bind mountsDocker Docs ↗
    • DocsDocker Docs: tmpfs mountsDocker Docs ↗
    • DocsPostgreSQL: pg_dumpPostgreSQL ↗

    Or get every quiz answer right and it completes itself.

    ← PreviousOther people's images: databases, tools and runtimesNext lesson →Building images: Dockerfiles, the layer cache and multi-stage builds