Backups - Revision history

Bryan: /* Initialize the repo */

2020-04-23T04:46:56Z

Initialize the repo

Bryan: /* Install nessary packages */

2019-10-17T20:35:25Z

Install nessary packages

Bryan at 05:17, 9 March 2019

2019-03-09T05:17:03Z

Bryan at 00:09, 7 October 2018

2018-10-07T00:09:19Z

Bryan: /* edit crontab */

2018-10-06T21:31:05Z

edit crontab

Bryan: /* edit crontab */

2018-10-06T21:30:45Z

edit crontab

Bryan: /* edit crontab */

2018-10-06T21:29:56Z

edit crontab

Bryan: /* Do the first backup */

2018-10-06T21:27:28Z

Do the first backup

Bryan at 02:06, 27 September 2018

2018-09-27T02:06:21Z

Bryan: Created page with "=Server Backups= AllStarLink, Inc uses a per server backup method based on borg. = Installing Borg on Ubuntu 16 = the default package that ships with ubuntu 16.04 LTS is b..."

2018-09-27T01:23:35Z

Created page with "=Server Backups= AllStarLink, Inc uses a per server backup method based on borg. = Installing Borg on Ubuntu 16 = the default package that ships with ubuntu 16.04 LTS is b..."

New page

=Server Backups=

AllStarLink, Inc uses a per server backup method based on borg.

= Installing Borg on Ubuntu 16 =

the default package that ships with ubuntu 16.04 LTS is borg 1.0 based and we use the latest 1.1 version as it has several security fixes.

== Install nessary packages==

apt-get install libacl1-dev python3-dev libssl-dev gcc g++

== install pip ==

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python3 get-pip.py

== Install Borg ==
pip3 install borgbackup

== Test for proper version ==
root@db-ord:~# borg -V
borg 1.1.7

which borg
/usr/local/bin/borg

== Install the scripts ==
This will install a new ssh key for root that matches the Rsync.net account and place the borg.inc program in the /root/ directory.

cd /
tar -xvf /root/borg-root-config.tar

= Edit the borg script=

On the Db servers we add a command to dump the database to /var/mysql-backup-current.sql.bz2

= Initalize the repo=

Copy the export lines to your shell and run then do

echo $BORG_REPO
ASLUSER@host.rsync.net:borg/ASL/db-ord

The borg/ASL/db-ord needs to be created on the server
ssh -t ASLUSER@host.rsync.net mkdir -p borg/ASL/db-ord

now we need to init the repo at that location

borg init -e keyfile-blake2 -p -v

== export the key ==

borg key export --paper

email this output GPG encrypted to the admin team members.

If we lose this key, there is no way to restore the backup. This means if the server dies, we need the paper key record and the passphrase.

= Do the first backup =

/root/borg.inc

Check that it's succeed

= move borg.inc to cron =

mv /root/borg.inc /etc/cron.daily/

← Older revision		Revision as of 05:17, 9 March 2019
Line 78:		Line 78:

	service cron reload		service cron reload
		+
		+	= Mounting and restoring =
		+	Borg makes it easy to mount a backup on the server using "borgfs".
		+
		+	If this is done on a different server or during a restore operation on new servers, the key files need to be imported from the paper key. Selected admin users have the paper keys backed up in encrypted email.
		+
		+	For example on a our db-fnt server:
		+
		+	Take the first 5 lines from the /srv/borg.sh file and export them on the cli:
		+
		+	export BORG_REMOTE_PATH=/usr/local/bin/borg1/borg1
		+	export HOST=
		+	export BORG_REPO=
		+	export BORG_PASSPHRASE=
		+	export BORG=/usr/local/bin/borg
		+
		+	Note if doing this on a different server, BORG_REPO= must be the server you intend to restore from.
		+
		+	#import the paper key
		+	borg key import --paper $BORG_REPO
		+
		+	#make a directory to mount the backups on:
		+	mkdir /mnt/backups
		+
		+	#now mount the borg repo
		+	borgfs $BORG_REPO /mnt/backups/
		+
		+	The files will now be in /mnt/backups:
		+
		+	ls /mnt/backups/
		+	db-fnt.allstarlink.org-2018-10-31_08:30 db-fnt.allstarlink.org-2019-02-28_08:30
		+	db-fnt.allstarlink.org-2018-11-30_08:30 db-fnt.allstarlink.org-2019-03-02_08:30
		+	db-fnt.allstarlink.org-2018-12-31_08:30 db-fnt.allstarlink.org-2019-03-03_08:30
		+	db-fnt.allstarlink.org-2019-01-31_08:30 db-fnt.allstarlink.org-2019-03-04_08:30
		+	db-fnt.allstarlink.org-2019-02-03_08:30 db-fnt.allstarlink.org-2019-03-05_08:30
		+	db-fnt.allstarlink.org-2019-02-10_08:30 db-fnt.allstarlink.org-2019-03-06_08:30
		+	db-fnt.allstarlink.org-2019-02-17_08:30 db-fnt.allstarlink.org-2019-03-07_08:30
		+	db-fnt.allstarlink.org-2019-02-24_08:30 db-fnt.allstarlink.org-2019-03-08_08:30
		+
		+	You can now go into any of these directories and restore files as of the date for the respective directory using normal UNIX utilities.
		+
		+
		+
		+
	[[Category: Infrastructure]]		[[Category: Infrastructure]]

← Older revision		Revision as of 00:09, 7 October 2018
Line 78:		Line 78:

	service cron reload		service cron reload
		+	[[Category: Infrastructure]]

← Older revision		Revision as of 21:31, 6 October 2018
Line 75:		Line 75:
	30 8 * * * root /srv/borg.sh		30 8 * * * root /srv/borg.sh

		+	Reload the crontab file

	service cron reload		service cron reload

← Older revision		Revision as of 21:30, 6 October 2018
Line 74:		Line 74:
	#borg backup		#borg backup
	30 8 * * * root /srv/borg.sh		30 8 * * * root /srv/borg.sh
		+
		+
		+	service cron reload

← Older revision		Revision as of 21:29, 6 October 2018
Line 71:		Line 71:

	edit /etc/crontab to run daily at 8am UTC		edit /etc/crontab to run daily at 8am UTC
		+
		+	#borg backup
		+	30 8 * * * root /srv/borg.sh

@@ Line 41: / Line 41: @@
   echo $BORG_REPO
-  ASLUSER@host.rsync.net:borg/ASL/db-ord
+  ASLUSER@host.rsync.net:borg/ASL/$HOST
 The borg/ASL/db-ord needs to be created on the server
-  ssh -t ASLUSER@host.rsync.net mkdir -p borg/ASL/db-ord
+  ssh -t ASLUSER@host.rsync.net mkdir -p borg/ASL/$HOST
 now we need to init the repo at that location

@@ Line 9: / Line 9: @@
 == Install nessary packages==
-    apt-get install libacl1-dev python3-dev libssl-dev gcc g++
+ apt-get install libacl1-dev python3-dev libssl-dev gcc g++ python3-llfuse
 == install pip ==
-    curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
+ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
-    python3 get-pip.py
+ python3 get-pip.py
 == Install Borg ==
-    pip3 install borgbackup
+ pip3 install borgbackup
 == Test for proper version ==
-    root:~# borg -V
+ root:~# borg -V
-    borg 1.1.7
+ borg 1.1.7
-    which borg
+ which borg
-    /usr/local/bin/borg
+ /usr/local/bin/borg
 == Install the scripts ==
 This will install a new ssh key for root that matches the Rsync.net account and place the borg.inc program in the /root/ directory.
-    cd /
+ cd /
-    tar -xvf /root/borg-root-config.tar
+ tar -xvf /root/borg-root-config.tar
 = Edit the borg script=
@@ Line 40: / Line 40: @@
 Copy the export lines to your shell and run then do
-    echo $BORG_REPO
+ echo $BORG_REPO
-    ASLUSER@host.rsync.net:borg/ASL/db-ord
+ ASLUSER@host.rsync.net:borg/ASL/db-ord
 The borg/ASL/db-ord needs to be created on the server
-    ssh -t ASLUSER@host.rsync.net mkdir -p borg/ASL/db-ord
+ ssh -t ASLUSER@host.rsync.net mkdir -p borg/ASL/db-ord
 now we need to init the repo at that location
-    borg init -e keyfile-blake2 -p -v
+ borg init -e keyfile-blake2 -p -v
 = export the key =
-    borg key export --paper
+ borg key export --paper
 Email this output GPG encrypted to the admin team members.
@@ Line 60: / Line 60: @@
 = Do the first backup =
-    /root/borg.inc
+ /root/borg.inc
 Check that it's succeed
@@ Line 66: / Line 66: @@
 = move borg.inc to cron =
-    mv /root/borg.inc /srv/borg.sh
+ mv /root/borg.inc /srv/borg.sh
 = edit crontab =
@@ Line 72: / Line 72: @@
 edit /etc/crontab to run daily at 8am UTC
-    #borg backup
+ #borg backup
 8 * * *      root    /srv/borg.sh
 Reload the crontab file
-     service cron reload
+ service cron reload
 = Mounting and restoring =
@@ Line 88: / Line 88: @@
 Take the first 5 lines from the /srv/borg.sh file and export them on the cli:
-    export BORG_REMOTE_PATH=/usr/local/bin/borg1/borg1
+ export BORG_REMOTE_PATH=/usr/local/bin/borg1/borg1
-    export HOST=
+ export HOST=
-    export BORG_REPO=
+ export BORG_REPO=
-    export BORG_PASSPHRASE=
+ export BORG_PASSPHRASE=
-    export BORG=/usr/local/bin/borg
+ export BORG=/usr/local/bin/borg
 Note if doing this on a different server, BORG_REPO= must be the server you intend to restore from.
-   #import the paper key
+ #import the paper key
-   borg key import --paper $BORG_REPO
+ borg key import --paper $BORG_REPO
-   #make a directory to mount the backups on:
+ #now mount the borg repo
-   mkdir /mnt/backups
+ borgfs $BORG_REPO /mnt/backups/
 The files will now be in /mnt/backups:
-    ls  /mnt/backups/
+ ls  /mnt/backups/
-    db-fnt.allstarlink.org-2018-10-31_08:30  db-fnt.allstarlink.org-2019-02-28_08:30
+ db-fnt.allstarlink.org-2018-10-31_08:30  db-fnt.allstarlink.org-2019-02-28_08:30
-    db-fnt.allstarlink.org-2018-11-30_08:30  db-fnt.allstarlink.org-2019-03-02_08:30
+ db-fnt.allstarlink.org-2018-11-30_08:30  db-fnt.allstarlink.org-2019-03-02_08:30
-    db-fnt.allstarlink.org-2018-12-31_08:30  db-fnt.allstarlink.org-2019-03-03_08:30
+ db-fnt.allstarlink.org-2018-12-31_08:30  db-fnt.allstarlink.org-2019-03-03_08:30
-    db-fnt.allstarlink.org-2019-01-31_08:30  db-fnt.allstarlink.org-2019-03-04_08:30
+ db-fnt.allstarlink.org-2019-01-31_08:30  db-fnt.allstarlink.org-2019-03-04_08:30
-    db-fnt.allstarlink.org-2019-02-03_08:30  db-fnt.allstarlink.org-2019-03-05_08:30
+ db-fnt.allstarlink.org-2019-02-03_08:30  db-fnt.allstarlink.org-2019-03-05_08:30
-    db-fnt.allstarlink.org-2019-02-10_08:30  db-fnt.allstarlink.org-2019-03-06_08:30
+ db-fnt.allstarlink.org-2019-02-10_08:30  db-fnt.allstarlink.org-2019-03-06_08:30
-    db-fnt.allstarlink.org-2019-02-17_08:30  db-fnt.allstarlink.org-2019-03-07_08:30
+ db-fnt.allstarlink.org-2019-02-17_08:30  db-fnt.allstarlink.org-2019-03-07_08:30
-    db-fnt.allstarlink.org-2019-02-24_08:30  db-fnt.allstarlink.org-2019-03-08_08:30
+ db-fnt.allstarlink.org-2019-02-24_08:30  db-fnt.allstarlink.org-2019-03-08_08:30
 You can now go into any of these directories and restore files as of the date for the respective directory using normal UNIX utilities.
 [[Category: Infrastructure]]