Difference between revisions of "Backups"

From "PTTLink Wiki"
Jump to navigation Jump to search
(Created page with "=Server Backups= AllStarLink, Inc uses a per server backup method based on borg. = Installing Borg on Ubuntu 16 = the default package that ships with ubuntu 16.04 LTS is b...")
 
Tag: visualeditor
 
(8 intermediate revisions by the same user not shown)
Line 9: Line 9:
 
== Install nessary packages==
 
== Install nessary packages==
  
apt-get install libacl1-dev python3-dev libssl-dev gcc g++
+
apt-get install libacl1-dev python3-dev libssl-dev gcc g++ python3-llfuse
  
 
== install pip ==
 
== install pip ==
  
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
+
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python3 get-pip.py
+
python3 get-pip.py
  
 
== Install Borg ==
 
== Install Borg ==
pip3 install borgbackup
+
pip3 install borgbackup
  
 
== Test for proper version ==
 
== Test for proper version ==
root@db-ord:~# borg -V
+
root:~# borg -V
borg 1.1.7
+
borg 1.1.7
  
which borg
+
which borg
/usr/local/bin/borg
+
/usr/local/bin/borg
  
 
== Install the scripts ==
 
== Install the scripts ==
 
This will install a new ssh key for root that matches the Rsync.net account and place the borg.inc program in the /root/ directory.
 
This will install a new ssh key for root that matches the Rsync.net account and place the borg.inc program in the /root/ directory.
  
cd /
+
cd /
tar -xvf /root/borg-root-config.tar  
+
tar -xvf /root/borg-root-config.tar  
  
 
= Edit the borg script=  
 
= Edit the borg script=  
Line 36: Line 36:
 
On the Db servers we add a command to dump the database to /var/mysql-backup-current.sql.bz2
 
On the Db servers we add a command to dump the database to /var/mysql-backup-current.sql.bz2
  
= Initalize the repo=
+
= Initialize the repo=
  
 
Copy the export lines to your shell and run then do  
 
Copy the export lines to your shell and run then do  
  
echo $BORG_REPO
+
echo $BORG_REPO
ASLUSER@host.rsync.net:borg/ASL/db-ord
+
ASLUSER@host.rsync.net:borg/ASL/$HOST
  
 
The borg/ASL/db-ord needs to be created on the server
 
The borg/ASL/db-ord needs to be created on the server
ssh -t ASLUSER@host.rsync.net mkdir -p borg/ASL/db-ord
+
ssh -t ASLUSER@host.rsync.net mkdir -p borg/ASL/$HOST
  
 
now we need to init the repo at that location
 
now we need to init the repo at that location
 +
borg init -e keyfile-blake2 -p -v
  
borg init -e keyfile-blake2 -p -v
 
  
 +
= export the key =
  
== export the key ==
+
borg key export --paper
  
borg key export --paper
+
Email this output GPG encrypted to the admin team members.
  
email this output GPG encrypted to the admin team members.
+
'''If we lose this key, there is no way to restore the backup.''' This means if the server dies, we need the paper key record and the passphrase.
 
 
If we lose this key, there is no way to restore the backup.  This means if the server dies, we need the paper key record and the passphrase.
 
  
 
= Do the first backup =  
 
= Do the first backup =  
  
/root/borg.inc
+
/root/borg.inc
  
 
Check that it's succeed  
 
Check that it's succeed  
Line 67: Line 66:
 
= move borg.inc to cron =
 
= move borg.inc to cron =
  
mv /root/borg.inc /etc/cron.daily/
+
mv /root/borg.inc /srv/borg.sh
 +
 
 +
= edit crontab =
 +
 
 +
edit /etc/crontab to run daily at 8am UTC
 +
 
 +
#borg backup
 +
30 8 * * *      root    /srv/borg.sh
 +
 
 +
Reload the crontab file
 +
 
 +
service cron reload
 +
 
 +
= Mounting and restoring =
 +
Borg makes it easy to mount a backup on the server using "borgfs".
 +
 
 +
If this is done on a different server or during a restore operation on new servers, the key files need to be imported from the paper key.  Selected admin users have the paper keys backed up in encrypted email.
 +
 
 +
For example on a our db-fnt server:
 +
 
 +
Take the first 5 lines from the /srv/borg.sh file and export them on the cli:
 +
 
 +
export BORG_REMOTE_PATH=/usr/local/bin/borg1/borg1
 +
export HOST=
 +
export BORG_REPO=
 +
export BORG_PASSPHRASE=
 +
export BORG=/usr/local/bin/borg
 +
 
 +
Note if doing this on a different server, BORG_REPO= must be the server you intend to restore from.
 +
 
 +
#import the paper key
 +
borg key import --paper $BORG_REPO
 +
 
 +
#make a directory to mount the backups on:
 +
mkdir /mnt/backups
 +
 
 +
#now mount the borg repo
 +
borgfs $BORG_REPO /mnt/backups/
 +
 
 +
The files will now be in /mnt/backups:
 +
 
 +
ls  /mnt/backups/
 +
db-fnt.allstarlink.org-2018-10-31_08:30  db-fnt.allstarlink.org-2019-02-28_08:30
 +
db-fnt.allstarlink.org-2018-11-30_08:30  db-fnt.allstarlink.org-2019-03-02_08:30
 +
db-fnt.allstarlink.org-2018-12-31_08:30  db-fnt.allstarlink.org-2019-03-03_08:30
 +
db-fnt.allstarlink.org-2019-01-31_08:30  db-fnt.allstarlink.org-2019-03-04_08:30
 +
db-fnt.allstarlink.org-2019-02-03_08:30  db-fnt.allstarlink.org-2019-03-05_08:30
 +
db-fnt.allstarlink.org-2019-02-10_08:30  db-fnt.allstarlink.org-2019-03-06_08:30
 +
db-fnt.allstarlink.org-2019-02-17_08:30  db-fnt.allstarlink.org-2019-03-07_08:30
 +
db-fnt.allstarlink.org-2019-02-24_08:30  db-fnt.allstarlink.org-2019-03-08_08:30
 +
 
 +
You can now go into any of these directories and restore files as of the date for the respective directory using normal UNIX utilities.
 +
 
 +
[[Category: Infrastructure]]

Latest revision as of 04:46, 23 April 2020

Server Backups

AllStarLink, Inc uses a per server backup method based on borg.

Installing Borg on Ubuntu 16

the default package that ships with ubuntu 16.04 LTS is borg 1.0 based and we use the latest 1.1 version as it has several security fixes.

Install nessary packages

apt-get install libacl1-dev python3-dev libssl-dev gcc g++ python3-llfuse

install pip

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python3 get-pip.py

Install Borg

pip3 install borgbackup

Test for proper version

root:~# borg -V
borg 1.1.7
which borg
/usr/local/bin/borg

Install the scripts

This will install a new ssh key for root that matches the Rsync.net account and place the borg.inc program in the /root/ directory.

cd /
tar -xvf /root/borg-root-config.tar 

Edit the borg script

On the Db servers we add a command to dump the database to /var/mysql-backup-current.sql.bz2

Initialize the repo

Copy the export lines to your shell and run then do

echo $BORG_REPO
ASLUSER@host.rsync.net:borg/ASL/$HOST

The borg/ASL/db-ord needs to be created on the server

ssh -t ASLUSER@host.rsync.net mkdir -p borg/ASL/$HOST

now we need to init the repo at that location

borg init -e keyfile-blake2 -p -v


export the key

borg key export --paper

Email this output GPG encrypted to the admin team members.

If we lose this key, there is no way to restore the backup. This means if the server dies, we need the paper key record and the passphrase.

Do the first backup

/root/borg.inc

Check that it's succeed

move borg.inc to cron

mv /root/borg.inc /srv/borg.sh

edit crontab

edit /etc/crontab to run daily at 8am UTC

#borg backup
30 8 * * *      root    /srv/borg.sh

Reload the crontab file

service cron reload

Mounting and restoring

Borg makes it easy to mount a backup on the server using "borgfs".

If this is done on a different server or during a restore operation on new servers, the key files need to be imported from the paper key. Selected admin users have the paper keys backed up in encrypted email.

For example on a our db-fnt server:

Take the first 5 lines from the /srv/borg.sh file and export them on the cli:

export BORG_REMOTE_PATH=/usr/local/bin/borg1/borg1
export HOST=
export BORG_REPO=
export BORG_PASSPHRASE=
export BORG=/usr/local/bin/borg

Note if doing this on a different server, BORG_REPO= must be the server you intend to restore from.

#import the paper key 
borg key import --paper $BORG_REPO
 
#make a directory to mount the backups on:
mkdir /mnt/backups
  
#now mount the borg repo
borgfs $BORG_REPO /mnt/backups/

The files will now be in /mnt/backups:

ls  /mnt/backups/
db-fnt.allstarlink.org-2018-10-31_08:30  db-fnt.allstarlink.org-2019-02-28_08:30
db-fnt.allstarlink.org-2018-11-30_08:30  db-fnt.allstarlink.org-2019-03-02_08:30
db-fnt.allstarlink.org-2018-12-31_08:30  db-fnt.allstarlink.org-2019-03-03_08:30
db-fnt.allstarlink.org-2019-01-31_08:30  db-fnt.allstarlink.org-2019-03-04_08:30
db-fnt.allstarlink.org-2019-02-03_08:30  db-fnt.allstarlink.org-2019-03-05_08:30
db-fnt.allstarlink.org-2019-02-10_08:30  db-fnt.allstarlink.org-2019-03-06_08:30
db-fnt.allstarlink.org-2019-02-17_08:30  db-fnt.allstarlink.org-2019-03-07_08:30
db-fnt.allstarlink.org-2019-02-24_08:30  db-fnt.allstarlink.org-2019-03-08_08:30

You can now go into any of these directories and restore files as of the date for the respective directory using normal UNIX utilities.