Simple PostgreSQL Backup Agent

Dockerized cron job to backup PostgreSQL database or multiple databases on different hosts. It’s based on Alpine docker image, so the image size is less than 11 Mb. The script can be also used without docker and docker compose or as a base for your own dockerized cron jobs. My general recommendation is to run docker container on your backup host to provide a kind of isolation from the management partition.

The script or “agent” does the following:

  • Reads content of /config/passfile to get pg_dump connection parameters
  • Verifies if the backup can be done by executing a dry run for each db
  • If the dry run is completed and plain format set, produces plain-text sql script and compresses it with gzip
  • If the dry run succeeds and custom format set, outputs a custom backup archive (more flexible and by default)
  • Cleans up the storage folder. Files older than 30 days are deleted
  • Redirects all cron job statuses to stdout
  • Keeps backup files under ./psql/backups/{hostname}/{dbname}/ on your host
  • Default settings: twice a day at 8:30 and 20:30 UTC; custom format; clean backups older than 30 days

Current limitations:

  • no encryption for specific databases (in to-do list)
  • no handling of wildcars in passfile (in to-do list)

Content

  • Dockerfile – describes docker image
  • docker-compose.yml – docker compose file to build and run agent service
  • /config/cronfile – cron job schedule settings
  • /config/passfile – PostgreSQL .pgpass actually
  • /config/psql_backup.sh – the script itself

Usage guide

  • check out the passfile and provide your own connection parameters
  • verify the cron job settings in the /config/cronfile
  • change make_backup function argument to set format output (plain/custom)
  • update cleaner function argument at the bottom of the script if necessary
  • edit dockerfile/docker-compose.yml or script itself if necessary
  • run docker compose build
  • run docker compose up -d
  • check out the stoud of the container to get the job’s status
  • TO RESTORE: use psql (if plain set) or pg_restore command (if custom format set)

Dockerfile

FROM alpine:3.16.2
LABEL AUTHOR="Roman Levchenko"
LABEL WEBSITE="rlevchenko.com"
RUN mkdir /etc/periodic/custom \
    && mkdir -p /backup/config \ 
    && touch /var/log/cron.log \
    && apk --no-cache add \
    postgresql14-client=14.5-r0 \
    bash=5.1.16-r2
COPY /config/cronfile /etc/crontabs/root
COPY /config/psql_backup.sh /etc/periodic/custom/backup
COPY ["/config/psql_backup.sh","/config/passfile","/backup/config/"]
RUN chmod 755 /etc/periodic/custom/backup \
    && chmod 0600 /backup/config/passfile
CMD ["-f","-l","8", "-L", "/dev/stdout"]
ENTRYPOINT ["crond"]

Script (excerpt)

# Clean old backup files
function cleaner()
{
set -o pipefail -e
	if [[ -n $(find $BACKUP_DIR \( -name "*.sql.gz" -o -name "*.custom" \) -type f -mtime +"$1") ]]; 
	then
		echo -e "\n${GREEN}[INFO]${OFF} ${BOLD}There are backup files older than $1 days. Cleaning up the following files:${OFF}"
		find $BACKUP_DIR \(-name "*.sql.gz" -o -name "*.custom" \) -print -type f -mtime +"$1" -exec rm {} \;
	else 
		echo -e "\n${GREEN}[INFO]${OFF} ${BOLD}There are no backup files older than $1 days. \nHave a nice day!${OFF}"
	fi
set +o pipefail +e
}

Result

Sample Output (w/error and success messages):

2021 Performance Review

First of all, happy new year to everyone. I hope this year will be fun-filled, awesome, and wish you a good health, to overcome any challenges you might face. The 2021 year was pretty tough to me: survived Covid; worked with a few international projects (AWS/Azure), which took a long time to finish/proceed. That’s why it’s been a long time since my last blog post.

In this post, I’d like to highlight shortly my 2021 year results.

Passed Linux Foundation Exams – Kubernetes and Linux
Linux Foundation Kubernetes Administrator :: Linux System Administrator
Renewed Azure certifications
Azure Solutions Architect Expert :: Azure DevOps Engineer Expert :: Azure Administrator Associate
Did technical reviews (some are still in progress)
manning books - kubernetes in action, kubernetes on windows, core kubernetes
Kubernetes in Action 2nd Edition :: Kubernetes on Windows :: Core Kubernetes

And usual activities such as blog posts, GitHub commits/issues and etc.

%d bloggers like this: