Simple Duplicity S3 Script

Duplicity is a command line tool that performs encrypted incremental backups.  It supports a variety of file locations, including Amazon S3, and comes pre-installed on several Linux distributions.  Ubuntu’s backup application, Deja Dup, is a frontend for Duplicity that adds some convenient right-click functionality to the file explorer.  This works well on the home directory to quickly restore previous versions of individual files, which is usually all I need.  My only complaint is enabling the Amazon S3 location required extra steps, I would expect all locations to be available by default.

For my larger data, I scripted Duplicity with a few options.  The most significant of which was throttling the upload speed with Trickle, or the initial full backup choked the machine for days.

#!/bin/bash

export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=

function BACKUP {
    SRC=$1
    DST=$2
    LOG=$3

    logger -t backup-script "Started backup of $SRC"

    trickle -s -u 1000 -d 100 \
        duplicity \
            --log-file $LOG \
            --encrypt-key BF68AB40 \
            $SRC \
            $DST

    logger -t backup-script "Backup finished. ($LOG)"
}

BACKUP "/media/folder1/" "s3+http://my_backup_bucket/folder1" "/var/log/backup1.log"
BACKUP "/media/folder2/" "s3+http://my_backup_bucket/folder2" "/var/log/backup2.log"

Symlinking the script to /etc/cron.daily worked after ironing out the kinks, like no dots in the symlink name.  Testing cron or running it manually helped debug the issues:

sudo run-parts --test /etc/cron.daily
sudo run-parts /etc/cron.daily

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s