Home > Backup, Unix > Remote Web Site Incremental Backup

Remote Web Site Incremental Backup

December 17th, 2008 Leave a comment Go to comments

Problem: You have a web site on a server but can’t connect to it with ssh but only FTP and you want to backup it daily but the website is huge, too huge to make a complete backup daily

My solution is to put a php script on the webserver that list files with a modification date in the last X seconds.
Then within the following shell script I get the result of the php page then I retreive listed file using FTP.

This is, I think, the most generic solution. I see an other one:

  • It would be to create a tar archive in the php script, then download the .tar directly. This as 2 problems: It’s less secure unless you can apply an htaccess file on the .tar file to put an auth, you may not be allowed to execute binary on the remote server to do the tar.
  • You may also use FTPFS, it’s nice, but require that the host making the backup can use ftpfs 🙂

This is the source of my SH script

#! /bin/sh

BK_YEAR="`date +%Y`"
BK_MONTH="`date +%b`"
BK_DAY="`date +%d`"

for file in `lynx -auth=login:password -source http://www.domain.com/admtool/last-modified.php`
do ADL="`echo $file | sed "s|/path/to/www.domain.com/root/document/or/the/ftp/chroot/path/||"`"
mkdir -p "/some/path/backup/domain.com/$BK_YEAR/$BK_MONTH/$BK_DAY/web-page/`dirname $ADL`"
wget -q --output-document="/some/path/backup/domain.com/$BK_YEAR/$BK_MONTH/$BK_DAY/web-page/$ADL" -r "ftp://login:pass@ftp.domain.com/$ADL"
done

tar -czf "/some/path/backup/domain.com/$BK_YEAR/$BK_MONTH/$BK_DAY/web-page.tgz" "/some/path/backup/domain.com/$BK_YEAR/$BK_MONTH/$BK_DAY/web-page"
rm -r "/some/path/wd1a/backup/domain.com/$BK_YEAR/$BK_MONTH/$BK_DAY/web-page"

This is the source of my PHP script

<?php 

$path = '.';
$time = 86400;      // will print all file modified in the last 86400 seconds

$curtime = time();

GetFileList($path);

function GetFileList($path) {
$curtime = time();
        $handle=opendir($path);
                while($file = readdir($handle)) {
                        if ($file=='.' || $file=='..') continue;
                        if (is_dir($path . '/' . $file)) {
                                GetFileList($path . '/' . $file);
                        }
                        else {
                                if (($curtime - filemtime($path . '/' . $file)) < 86400) {
                                        echo $path . '/' . $file . "n";
                                }
                        }
                }
        closedir($handle);
}

?>
Categories: Backup, Unix Tags: , ,
  1. No comments yet.
  1. No trackbacks yet.