You surely have experienced a lot of pain when handling your backups coming from the various servers you have online. A very important thing to remember is, in fact, to have backups of your live web servers, so if things go crazy, you have the "last resource".
Plus, backups should be kept for at least some days, so that if you find out a web server breach or something you haven't noticed before, you can roll back the situation to the backup of N-days before.
If you're running cPanel, getting automated backups is actually pretty easy and straightforward. With cPanel you can even have a default backup retention period of N days, and retain that backups directly on the main system. But, that retention period is relative to the HDD of the main server you're running on. And, another negative fact is, if your main cPanel server that keeps your 10-days old backups fails or gets destroyed by the sys admin?
This is why, in my opinion, the best thing to do is have an external backup server where you send, via FTP, every type of backup from the webservers you are currently running.
Having an external backup server also enables you to have more than one server to be backed-up. Personally, I run a backup server that accepts backups from 3 different cPanel servers sources.
To help your backup server, and to help you setting everything up quick, I've coded a little script, that can be cron-jobbed and run via PHP CLI, that reads the backup destination directory ( the one into which you make your remote cPanel FTP send into ) and organizes the files contained into it by day, purging the backups that are too old to be retained.
If you're looking at guides on how to turn on automated cPanel backup and FTP remote send, there are plenty of guides and walkthroughs. Google is you friend.
Let's have a look at the code:
<?php
/*
|--------------------------------------------------------------------------
| PHP / CGI Automated "Backup File Rotation" script
|--------------------------------------------------------------------------
|
| Version: 1.1.0
| Author: Maurizio Fonte
| Author URL: https://www.mauriziofonte.it
| Description:
| This script ( that needs to be run via CLI e.g /usr/bin/php -q this_script.php ) serves as an automated backup rotation organization.
| In fact, it takes all subdirectories under the main BACKUP_DIR folder and copies them to another location in which these backup files will be stored and rotated.
| For example, consider these directories and their contents:
|
| BACKUP_DIR = /root/backup
|
| /root/backup
| |-> folder_a
| | -> folder_a_backup_file_1.tar.gz
| | -> folder_a_backup_file_2.tar.gz
| |-> folder_b
| | -> folder_b_backup_file_1.tar.gz
| | -> folder_b_backup_file_2.tar.gz
| | -> folder_b_backup_file_3.tar.gz
| |-> folder_c
| | -> folder_c_backup_file_1.tar.gz
|
| For every execution day, the script will read the contents of the /root/backup folder, and re-organize the files under the directory /back_rot/
| The script will create this rotation structure:
|
| BACKUP_ROTATION_DIR = /back_rot/
|
| /back_rot/
| |-> 2016-01-01
| |-> folder_a
| | -> folder_a_backup_file_1.tar.gz
| | -> folder_a_backup_file_2.tar.gz
| |-> folder_b
| | -> folder_b_backup_file_1.tar.gz
| | -> folder_b_backup_file_2.tar.gz
| | -> folder_b_backup_file_3.tar.gz
| |-> folder_c
| | -> folder_c_backup_file_1.tar.gz
| |-> 2016-01-02
| ...
| ...
|
| After a backup file / folder in the BACKUP_ROTATION_DIR becomes "too old to survive" ( after BACKUP_RETENTION_PERIOD days ), the folder will be automatically purged.
|
| Please consider that, in order for this automated backup rotator to work *properly*, these conditions have to be met:
| 1) Each day, some other script ( from a remote server, for example ) has to fill its own directory on BACKUP_DIR ( in the example provided, "folder_a", "folder_b", "folder_c" )
| 2) This script needs to be run once per day ( nothing harmful will happen if you run this more than once per day, though. Simply, it will see there are no actions to do ).
| 3) This script is meant to be executed via CLI
*/
@ini_set('max_execution_time', 300);
@ini_set('memory_limit', '256M');
date_default_timezone_set ( 'Europe/Rome' );
define ( 'HOME_DIR', rtrim ( dirname ( __FILE__ ), '/' ) . '/' );
define ( 'LOGS_DIR', 'logs/' );
define ( 'BACKUP_DIR', '/BACKUPS/' );
define ( 'BACKUP_ROTATION_DIR', '/BACKUPS_ROTATION/' );
define ( 'BACKUP_RETENTION_PERIOD', 20 );
define ( 'EXCLUDE_BACKUP_FILENAME', 'exclude_this_backup_filename.tar.gz' );
define ( 'EXCLUDE_BACKUP_FOLDER', 'exclude_this_directory_name_while_parsing' );
ob_start ();
out ( '#############################################################################################' );
out ( '############# #############' );
out ( '############# Automated PHP/CGI BACKUP FILE ROTATION ALGORITHM v1.1.0 #############' );
out ( '############# Copyright(c) 2016 Maurizio Fonte - mauriziofonte.it #############' );
out ( '############# #############' );
out ( '#############################################################################################' );
out ( '############### ' . date ( 'Y-m-d H:i:s' ) . ' ###############' );
out ( '#############################################################################################' );
// fase 0: se non esiste la cartella BACKUP_DIR, non possiamo fare ovviamente nulla...
if ( ! is_dir ( BACKUP_DIR ) ) closenow ( true, 'This magic script can't do anything as long BACKUP_DIR does not exists ... actual BACKUP_DIR is "' . BACKUP_DIR . '"' );
// fase 1: controllo che la directory dei backup giornalieri e dei backup rotation sia leggibile e scrivibile e mi pre-carico i suoi contenuti in un semplice array di file
out ( ' ** Starting "' . BACKUP_DIR . '" recursive directory iterator...' );
$backup_dir_contents = Array ();
$objects = new RecursiveIteratorIterator ( new RecursiveDirectoryIterator ( BACKUP_DIR ), RecursiveIteratorIterator::SELF_FIRST );
if ( $objects ) {
foreach ( $objects as $name => $object ){
if ( $name !== '.' && $name !== '..' ) {
$name = realpath ( $name );
if ( ! in_array ( $name, $backup_dir_contents ) && $name !== '/' && $name !== rtrim ( BACKUP_DIR, '/' ) ) $backup_dir_contents[] = $name;
}
}
}
// fase 1a: controllo che abbiamo realmente qualcosa da fare dentro $backup_dir_contents e creo le directory di destinazione, se non esistono
if ( count ( $backup_dir_contents ) == 0 ) closenow ( true, 'Nothing found on "' . BACKUP_DIR . '" that can be eligible to a copy/paste! Is that directory empty?' );
if ( ! is_dir ( BACKUP_ROTATION_DIR . '_placeholder' ) ) @mkdir ( BACKUP_ROTATION_DIR . '_placeholder', 0755 );
if ( ! is_dir ( BACKUP_ROTATION_DIR . '_placeholder' ) ) closenow ( true, 'Backup rotation folder ( ' . BACKUP_ROTATION_DIR . ' ) is not write-able ...' );
@touch ( BACKUP_ROTATION_DIR . 'test.txt' );
if ( ! is_file ( BACKUP_ROTATION_DIR . 'test.txt' ) ) closenow ( true, 'Backup rotation folder ( ' . BACKUP_ROTATION_DIR . ' ) is not write-able ...' );
@unlink ( BACKUP_ROTATION_DIR . 'test.txt' );
// fase 2: ciclo di ricognizione del file tree della directory SORGENTE dei backup
out ( ' ** Starting "' . BACKUP_DIR . '" folder accounts+files recognition...' );
$files_found = 0;
$accounts_found = 0;
$backup_tree = Array ();
$remember_backup_account_roots = Array ();
foreach ( $backup_dir_contents as $i => $fullpath ) {
$stripped_path = str_replace ( BACKUP_DIR, '', $fullpath );
$chunks = explode ( '/', $stripped_path );
if ( count ( $chunks ) == 1 ) {
if ( ! array_key_exists ( $chunks[0], $backup_tree ) && $chunks[0] != EXCLUDE_BACKUP_FOLDER ) {
$backup_tree[$chunks[0]] = Array ();
$remember_backup_account_roots[] = $fullpath;
$accounts_found++;
continue;
}
}
else if ( count ( $chunks ) == 2 ) {
if ( ! array_key_exists ( $chunks[0], $backup_tree ) && $chunks[0] != EXCLUDE_BACKUP_FOLDER ) {
$backup_tree[$chunks[0]] = Array ();
$remember_backup_account_roots[] = $fullpath;
$accounts_found++;
}
if ( is_file ( BACKUP_DIR . $chunks[0] . '/' . $chunks[1] ) && $chunks[1] != EXCLUDE_BACKUP_FILENAME ) {
$mtime = filemtime ( BACKUP_DIR . $chunks[0] . '/' . $chunks[1] );
$backup_tree[$chunks[0]][bdate($mtime)][] = BACKUP_DIR . $chunks[0] . '/' . $chunks[1];
$files_found++;
}
else continue;
}
else if ( count ( $chunks ) == 3 ) {
if ( ! array_key_exists ( $chunks[0], $backup_tree ) && $chunks[0] != EXCLUDE_BACKUP_FOLDER ) {
$backup_tree[$chunks[0]] = Array ();
$remember_backup_account_roots[] = $fullpath;
$accounts_found++;
}
if ( is_file ( BACKUP_DIR . $chunks[0] . '/' . $chunks[1] . '/' . $chunks[2] ) && $chunks[2] != EXCLUDE_BACKUP_FILENAME ) {
$mtime = filemtime ( BACKUP_DIR . $chunks[0] . '/' . $chunks[1] . '/' . $chunks[2] );
$backup_tree[$chunks[0]][bdate($mtime)][] = BACKUP_DIR . $chunks[0] . '/' . $chunks[1] . '/' . $chunks[2];
$files_found++;
}
else continue;
}
}
out ( ' // Done working on "' . BACKUP_DIR . '": found ' . $accounts_found . ' eligible ACCOUNTS and ' . $files_found . ' eligible FILES to be copied into backup rotation directory' );
// fase 3: per ogni elemento nell'array $backup_tree controllo nella directory dei backup rotation se ho quella data e quell'account "root" salvato
out ( ' ** Starting "' . BACKUP_ROTATION_DIR . '" smart copy...' );
foreach ( $backup_tree as $account_name => $date_to_filename ) {
foreach ( $date_to_filename as $date_ymd => $filenames ) {
foreach ( $filenames as $i => $filename ) {
// prima di tutto, se non esiste ancora, creo la directory con la data come "livello zero"
if ( ! is_dir ( BACKUP_ROTATION_DIR . $date_ymd ) ) {
out ( ' --> Going to create a folder: ' . BACKUP_ROTATION_DIR . $date_ymd );
@mkdir ( BACKUP_ROTATION_DIR . $date_ymd, 0755 );
}
if ( ! is_dir ( BACKUP_ROTATION_DIR . $date_ymd ) ) closenow ( true, 'Backup rotation folder ( ' . BACKUP_ROTATION_DIR . ' ) is not write-able ...' );
// poi, se non esiste ancora, creo la directory dell'account
if ( ! is_dir ( BACKUP_ROTATION_DIR . $date_ymd . '/' . $account_name ) ) {
out ( ' --> Going to create a folder: ' . BACKUP_ROTATION_DIR . $date_ymd . '/' . $account_name );
@mkdir ( BACKUP_ROTATION_DIR . $date_ymd . '/' . $account_name, 0755 );
}
if ( ! is_dir ( BACKUP_ROTATION_DIR . $date_ymd . '/' . $account_name ) ) closenow ( true, 'Backup rotation folder ( ' . BACKUP_ROTATION_DIR . ' ) is not write-able ...' );
// in ultimo, se non esiste il file, lo copio
if ( ! is_file ( BACKUP_ROTATION_DIR . $date_ymd . '/' . $account_name . '/' . $filename ) ) {
out ( ' --> Going to copy a backup file: ' . $filename );
$file_basename = pathinfo ( $filename, PATHINFO_BASENAME );
if ( copy ( $filename, BACKUP_ROTATION_DIR . $date_ymd . '/' . $account_name . '/' . $file_basename ) ) {
@unlink ( $filename );
}
else closenow ( true, 'Closing backup rotation now... the file copy of "' . $filename . '" to "' . BACKUP_ROTATION_DIR . $date_ymd . '/' . $account_name . '/' . $file_basename . '" failed with no apparent reason...' );
}
}
}
}
// fase 4: creo l'albero delle cartelle/file sulla directory di rotazione backup
out ( ' ** Starting "' . BACKUP_ROTATION_DIR . '" automated erase of backup files greater than ' . BACKUP_RETENTION_PERIOD . ' days old' );
$backuprotation_dir_contents = Array ();
$objects = new RecursiveIteratorIterator ( new RecursiveDirectoryIterator ( BACKUP_ROTATION_DIR ), RecursiveIteratorIterator::SELF_FIRST );
if ( $objects ) {
foreach ( $objects as $name => $object ){
if ( $name !== '.' && $name !== '..' ) {
$name = realpath ( $name );
if ( ! in_array ( $name, $backuprotation_dir_contents ) && $name !== '/' && $name !== rtrim ( BACKUP_ROTATION_DIR, '/' ) ) $backuprotation_dir_contents[] = $name;
}
}
}
// fase 5: cancello tutti i file, sulla directory di backup, che sono più anziani della BACKUP_RETENTION_PERIOD ( ovvero, i backup troppo vecchi )
$today_tstamp = strtotime ( date ( 'Y' ) . '-' . date ( 'm' ) . '-' . date ( 'd' ) . ' 00:00:00' );
$backup_retention_deadline_tstamp = strtotime ( '-' . BACKUP_RETENTION_PERIOD . ' days', $today_tstamp );
foreach ( $backuprotation_dir_contents as $i => $fullpath ) {
// sicuramente il primo chunk è relativo alla data, per come è costruito l'albero di copia/incolla
$stripped_path = str_replace ( BACKUP_ROTATION_DIR, '', $fullpath );
$chunks = explode ( '/', $stripped_path );
if ( count ( $chunks ) == 1 && $chunks[0] !== '_placeholder' ) {
if ( preg_match ( '/^[0-9]{4}-[0-9]{2}-[0-9]{2}$/', $chunks[0], $match ) ) {
$date_of_folder_tstamp = strtotime ( $chunks[0] . ' 00:00:00' );
if ( $date_of_folder_tstamp < $backup_retention_deadline_tstamp ) {
out ( ' --> folder ' . $chunks[0] . ' needs to be purged, it's more than ' . BACKUP_RETENTION_PERIOD . ' days old...' );
emptyDirectory ( BACKUP_ROTATION_DIR . $chunks[0], true );
}
else out ( ' --> folder ' . $chunks[0] . ' can be left on place... it's not time for its death ... for now ...' );
}
else closenow ( true, 'Damnit... folder ' . $fullpath . ' failed the preg_match check on the first chunk ( ' . $chunks[0] . ' )' );
}
}
// fase 5: per ogni "$remember_backup_account_roots" devo svuotare la cartella perchè è già stata processata
out ( ' ** Starting "' . BACKUP_DIR . '" automated erase of backup files that have been already copied in this session...' );
foreach ( $remember_backup_account_roots as $i => $folder ) {
emptyDirectory ( $folder );
}
// fatto!
out ( '' );
out ( ' yayyy!! everything done!' );
out ( '' );
out ( ' Copyright (c) Maurizio Fonte 2016 - https://www.mauriziofonte.it' );
closenow ();
function bdate ( $tstamp ) {
return date ( 'Y-m-d', $tstamp );
}
function emptyDirectory ( $directory, $remove_parent = false ) {
$files = new RecursiveIteratorIterator( new RecursiveDirectoryIterator ( $directory, RecursiveDirectoryIterator::SKIP_DOTS ), RecursiveIteratorIterator::CHILD_FIRST );
if ( $files ) {
foreach ( $files as $fileinfo ) {
$todo = ( $fileinfo -> isDir ( ) ) ? 'rmdir' : 'unlink';
$todo ( $fileinfo -> getRealPath ( ) );
}
if ( $remove_parent ) rmdir ( $directory );
}
}
function closenow ( $error = false, $error_string = null ) {
if ( ! is_dir ( HOME_DIR . LOGS_DIR ) ) mkdir ( HOME_DIR . LOGS_DIR, 0755 );
if ( ! is_dir ( HOME_DIR . LOGS_DIR . date ( 'Y-m' ) . '/' ) ) mkdir ( HOME_DIR . LOGS_DIR . date ( 'Y-m' ) . '/', 0755 );
if ( $error && ! empty ( $error_string ) ) {
out ( '!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!' );
out ( '!! program detected an error and is going to be killed NOW !!' );
out ( '!!' );
out ( '!! ERROR_REASON = ' . $error_string );
}
$out = ob_get_clean ();
file_put_contents ( HOME_DIR . LOGS_DIR . date ( 'Y-m' ) . '/' . date ( 'Y-m-d-H-i-s' ) . '.txt', $out );
exit ();
}
function out ( $string ) {
echo $string . chr(10);
}
?>
The code is pretty straightforward. If your backups fall every day into /BACKUPS/name_of_server, then you make this script "read" the contents of the /BACKUP/ directory. It will take care of moving the backups into "BACKUP_ROTATION_DIR" and will make sure that backup files that are older than "BACKUP_RETENTION_PERIOD" will get deleted at the right day.
To make it run correctly, place this script into its own directory, because it will automatically create a "logs" folder for you. If things go bad, go check the script verbose output into the right log file. Then, make it run via a new cronjob, and make sure that the user that actually runs the PHP script has the necessary permissions to read/write into the backup rotation folder.