sync – My Thoughts

git cli

To backup my arduino sketches I will use a remote git repository hosted on my own server. So lets start

git init – initialize a new local repository
git clone @: – check out/clone from a remote reporitory

The local repo is devided into a working copy, the index as interim stage und the head which is a link to the last commit.

git add – add a file to local repository
git commit -m ““
git push origin master – sync local repository with remote repo

If the local repo was not cloned but should be linked to a remote repository use:

git remote add origin

To sync the local repository use:

git pull – fetch remote changes and merge them with the local files
git add – inform git about solved merge problems after solving

Summary:

git init

git add .

git commit -am “”

git remote add origin @:

git push -u origin master

WebFrontEnd for rsnapshot based backup system

As already discussed in: rsnapshot backup solution I am using rsnapshot as my main backup solutions for a variety of servers.
But I am lazy and often forget about the correct settings, commands, etc. For this reason I started building up some web front ends for several common daily tasks. Just to remember what to do and how and of course to have some fun with programming.
And here is another part of this series: The Rsnapshot Web Front End:

The complete webpage is enclosed into one file – this was my first internal order, because I wanted something that may be installed very easily. There is also a file structure as follows:
main folder:

default – the base for new config files
pid – the process id file
logs – the folder with log files
rsnapshot – the folder holding the config files
backup.php – the web front end

The default file is cloned to get the base for an new config file and as such is the master. All files in folder rsnapshot are shown in a matrix to access the action.

The web page itself is self explanatory and allows to:

view the config and logfiles,
create and delete a config file/server entry
start a backup process and
edit the config file

Any new entry made in the config file is checked against a regular expression already within the backup.php file to avoid misconfigurations. The editing is further devided into normal and expert mode whereas normal mode is only showing the most important options and expert mode is showing all options.

The styling/design of the page is very poor and might be improved later on. Also the security of the page is lacking some attention and might change.

But now to the code:

<?php ############# HEAD  ############################
echo "<!DOCTYPE html>\n";
echo "<html>\n<head>\n <meta charset='utf-8'><meta name='viewport' content='width=device-width, initial-scale=1.0'>\n";
############# style sheets ########################
echo "<title>Backup Admin Page</title>\n</head>\n<body>\n\n";
echo "<header><h1>Backup Admin - Main Page</h1>";
echo "<h4>created to configure my own backup system based on rsnapshot</h4></header>\n";
echo "<style>.red { color: red;} .blue {color: blue;} .hide {display: none;} .comment { width: 100%;}</style>";
############# Configuration ##########################
$config['configfile'] = "rsnapshot/";# folder for config files
$config['logfile'] = "log/";# folder for logfiles
$config['command'] = "ls -l";# command to be executed to start backup
##################################### no editing below here ###########################
$PHP_SELF = htmlspecialchars($_SERVER['PHP_SELF']); # use one file only which will be reload with new parameters all the time
$general_menu = array( "Cancel", "NewHost");# general menue options not associated with host/server
$menue_items = array("ShowConfigfile", "StartBackup", "ShowLogfile", "EditConfigfile", "Delete");# menue items for each host/server
$config['columns'] = count($menue_items);# the number of columns in total
$config['values'] = array("snapshot_root", "interval", "backup", "no_create_root", "cmd_cp", "cmd_rm",
		"cmd_rsync", "cmd_ssh", "cmd_logger", "cmd_du", "verbose", "loglevel", "rsync_long_args", "ssh_args", 
		"du_args", "logfile", "lockfile", "config_version");
$config["inputtype"]["interval"] = array("weekly","hourly","daily","monthly");
$config["inputtype"]["verbose"] = array("1","2","3","4","5");
$config["inputtype"]["loglevel"] = array("1","2","3","4","5");
##################################### function definitions ########################################
####### clear html variables from malicious code
function clean_html($variable){
	return trim(htmlspecialchars($variable));
}
###### debug output of POST
function debug_output(){
	echo "<pre><br><h1>Hier kommt der schamass!</h1>";
	print_r($_POST);
	echo "</pre>"; return ;
}
###### parse contents of configfile and store everything in two arrays
function parse_file_contents($filename){
	global $inputvalues; # one matrix for all values already filled with defaults
	if ($data = file_get_contents($filename)){
		$contents = explode("\n", $data);$comment = "";
		foreach ($contents as $line) { # parse each line of the input file seperately
			$line = trim($line); $value = ""; # and override default values with config file values
			if (strlen($line) && substr($line, 0, 1) == '#') # a comment line
				$comment .= "\n".$line; 
			elseif (strlen($line)){ # non empty line found
				$key = trim(strtok($line, " \t")); # name of the specific option is first parameter
				while ($inputpart = strtok(" \t")) $value = $value.trim($inputpart)."\t"; # rest of the line is combined as value
				if ($key == "exclude") # special treatement for excluded directories
					$inputvalues["exclude"][] = trim($value);
				else # normal value will override default values in array
					$inputvalues["values"][$key] = trim($value);
				if ($comment != "") {# non enpty comment will be stored
					$inputvalues['comment'][$key] = $comment; 
					$comment = "";
				}
			}
		}
		$return = "File: ".$filename." successfully paresed!";
	}
	else # could not get file contents via function so die
		$return = "Could not open file: ".$filename;
	return $return;
}
####### save config file
function save_conf_file($inputvalues, $filename){
	global $config;
	$output = "";
	foreach ($config['values'] as $key){# iterate over all config file parameters
		if (isset($inputvalues['comment'][$key])) $output .= $inputvalues['comment'][$key]."\n";
		if (isset($inputvalues['values'][$key]))  $output .= $key."\t".$inputvalues['values'][$key]."\n";
	}
	if (isset($inputvalues['comment']['exclude'])) $output .= $inputvalues['comment']['exclude']."\n";
	foreach ($inputvalues['exclude'] as $key => $value)
		if ($inputvalues['exclude'][$key] != "") $output .= "exclude"."\t".$value."\n";
	
	/*
	if ($handle = fopen($filename, "w")){ # open configfile for write
		fwrite($handle, $output);
		fclose($handle);
	}
	
	else 
		$return = "Could not open file: ".$filename;
		*/
	return "<pre>".$output."</pre>";
#	return $return;
}
####### edit values using defaults or values from config file
function edit_conf_file($inputvalues, $config, $mode){
	$class = "show";
	$return = "<table border='1' width='100%'>\n<tr><td colspan='4' align='right'>";
	$return .= "<input type='submit' name='action[Expert]' value='".$mode."'></td></tr>\n";
	foreach ($config['values'] as $key){
		if ($key == "no_create_root" && $mode == "Normal") $class = "hide";# show other entries only in expert mnode
		if (isset($inputvalues['comment'][$key])){# any comments for the forthcoming option?
			$rows = substr_count($inputvalues['comment'][$key], "\n") + 2;
			$return .= "<tr class='".$class."'><td colspan='3'><textarea cols='100%' rows='".$rows;
			$return .= "' name='inputvalues[comment][".$key."]'> ".$inputvalues['comment'][$key];
			$return .= " </textarea></td></tr>\n";
		}# and now the entry
		$return .= "<tr class='".$class."'><td>".$key."</td>";
		if (isset($config['inputtype'][$key])) {# no simple input field but a select box
			$return .= "<td><select name='inputvalues[values][".$key."]' size='1'>";
			foreach ($config['inputtype'][$key] as $value){
				$return .= "<option";
				if(isset($inputvalues['values'][$key]) && strtok($inputvalues['values'][$key], " ") == $value ) $return .= " selected ";
				$return .= ">".$value."</option>\n";
			}
			$return .= "</select></td>";
		} else {# simple input field
			$return .= "<td><input type='input' name='inputvalues[values][".$key."]' value='";
			if (isset($inputvalues['values'][$key])) $return .= $inputvalues['values'][$key]; 
			$return .= "'></td>";
		}
		$return .= "<td></td></tr>\n";
	}################# Exclude directories ##################
	$return .= "<tr class='show'><td colspan='4'>Exclude Directories:</td></tr>\n";
	if (isset($inputvalues['comment']['exclude'])){
		$rows = substr_count($inputvalues['comment']['exclude'], "\n") + 2;
		$return .= "<tr class='show'><td colspan='3'><textarea cols='100%' rows='".$rows;
		$return .= "' name='inputvalues[comment][exclude]'> ".$inputvalues['comment']['exclude'];
		$return .= " </textarea></td></tr>\n";
	}
	if (isset($inputvalues['exclude'])){
		foreach ($inputvalues['exclude'] as $subkey => $value){# show excludes as checkbox list
			if ($value != "") {# only if there is an entry and not en empty line
				$return .= "<tr class='show'><td colspan='2'>".$value."</td><td><input type='checkbox' name='inputvalues[exclude]";
				$return .= "[".$subkey."]' value='".$value."' checked></td></tr>\n";
			}
		}
	} 
	for ($i=0; $i < 5; $i++)# add 5 extra rows for exclude input
		$return .= "<tr class='show'><td>Add exclude folder</td><td colspan='2'><input type='input' value='' name='inputvalues[exclude][]'></td></tr>\n";
	$return .= "<tr class='show'><td align='center' colspan='2'><input type='submit' value='SaveConfigfile' ";
	$return .= "name='action[SaveConfigfile]'></td><td align='center' colspan='2'>";
	$return .= "<input type='submit' name='action[Cancel]' value='Cancel'></td></tr>\n</table>\n";
	return $return;
}
####### function generate menu
function generate_menue($menue_items, $configfile){
	$return = "<tr><thead align='center'>";
	foreach ($menue_items as $key) # table headers
			$return .= "<th>".$key."</th>";
	$return .= "</thead>\n<tbody align='center'>\n";
	foreach (scandir($configfile) as $file){# iterate over complete directory contents
		if (substr($file, 0, 1) != ".") { # dotfiles are ignored
			$file = htmlspecialchars($file);$return .= "<tr>";
			foreach ($menue_items as $key) {
				$return .= "<td><input type='submit' value='  ".$file."  ' name='action[".$key."]'></td>";
			}
			$return .= "</tr>\n";
		}
	}
	return $return;}
######################################################################
############################## MAIN ##################################
######################################################################
echo "<body><form method='POST' action='".$PHP_SELF."'>\n";# page header with headline and table headers
echo "<table border=1 cellspacing='10'>\n";# everything will be bound to one table
echo generate_menue($menue_items, $config['configfile']);
$cols = round($config['columns'] / 2);$cols_remain = $config['columns'] - $cols;
echo "<tr><td colspan='".$cols."'><input type='submit' name='action[Cancel]' value='Cancel'></td>";
echo "<td colspan='".$cols_remain."'><input type='submit' name='action[NewHost]' value='NewHost'></td></tr>";
######################### Main Menue #####################################
if (isset($_POST['expertMode']) && $_POST['expertMode'] == "Expert") $mode = "Expert";
else $mode = "Normal";
if (isset($_POST['action'])) {
	$menue_entry = array_keys($_POST['action'])[0];# action[<menue_entry>] -> <host>/<menue_entry>
	$host = clean_html($_POST['action'][$menue_entry]);
	switch ($menue_entry){
		case "EditConfigfile":###########################################
			if (file_exists($config['configfile'].$host)) {
				$return = "<h3>Edit (".$host.")</h3>";# show headline
				$return .= parse_file_contents($config['configfile'].$host);
				$return .= "<input type='hidden' name='hostname' value='".$host."'>";			
				$return .= "<pre class='blue'>".edit_conf_file($inputvalues, $config, $mode)."</pre>";
			} else {
				$return = "<h3>Error (".$host.")</h3>";# show headline
				$return .= "<pre class='red'>File: ".$config['configfile'].$host." does not exist!</pre>\n";# show error message
			}
			break;
		case "Expert":###################################################
			if (isset($_POST['hostname'])){
				$return = "<h3>Edit (".$_POST['hostname'].")</h3>";# show headline
				if ($host == "Expert") $mode = "Normal";
				else $mode = "Expert";
				$return .= "<input type='hidden' name='hostname' value='".$_POST['hostname']."'>";			
				$return .= "<pre class='blue'>".edit_conf_file($_POST['inputvalues'], $config, $mode)."</pre>";
			} else {
				$return = "<h3>Error (".$host.")</h3>";# show headline
				$return .= "<pre class='red'>Error! No host defined!</pre>\n";# show error message
			}
			break;
		case "SaveConfigfile":############################################
			if (isset($_POST['inputvalues'])){
				$return = "<h3>Save Configfile (".$host.")</h3>";# show headline
				$return .= "<pre class='blue'>".save_conf_file($_POST['inputvalues'], $config['configfile'].$host)."</pre>";# write file contents
			} else {
				$return = "<h3>Error (".$host.")</h3>";# show headline
				$return .= "<pre class='red'>Error! No host defined!</pre>\n";# show error message
			}
			break;
		case "ShowConfigfile":############################################
			if (file_exists($config['configfile'].$host)){# check if file exists in folder
				$return = "<h3>Show Configfile (".$host.")</h3>";# show headline
				$return .= "<pre class='blue'>".file_get_contents($config['configfile'].$host)."</pre>";# get file contents
			} else { 
				$return = "<h3>Error (".$host.")</h3>";# show headline
				$return .= "<pre class='red'>File: ".$config['configfile'].$host." does not exist!</pre>";# show error
			}
			break;
		case "StartBackup":###############################################
			$befehl = $config['command']." ".$config['configfile'].$host." 2>&1";# create backup command
			if (file_exists($config['configfile'].$host)){# check if file exists
				$return = "<h3>Start Backup (".$host.")</h3>";
				$return .= "<pre class='blue'>Execute Command: (".$befehl.")</pre>";
				$return .= exec($befehl, $output, $return_val);# execute backup command
#				$return .= $output." ".$return_val;
			} else {
				$return = "<h3>Error (".$host.")</h3>";# show headline
				$return .= "<pre class='red'>File: ".$config['configfile'].$host." does not exist!</pre>";# show error message
			}
			break;
		case "ShowLogfile":##############################################
			if (file_exists($config['logfile'].$host)){ # check if file exists
				$return = "<h3>Show Logfile (".$host.")</h3>";# show headline
				$return .= "<pre class='blue'>".file_get_contents($config['logfile'].$host)."</pre>";# get file contents
			} else { 
				$return = "<h3>Error (".$host.")</h3>";# show headline
				$return .= "<pre class='red'>File: ".$config['logfile'].$host." does not exist!</pre>";# show error message
			}
			break;
		case "Delete":###################################################
			$return = "<h3>Delete (".$host.")</h3>";# show headline
			$return .= "<div class='red'>You really want to delete:"; 
			$return .= "<input type='submit' name='action[ReallyDelete]' value='".$host."'></div>";
			break;
		case "ReallyDelete":#############################################
			$return = "<h3>Delete (".$host.")</h3>";# show headline
			if (file_exists($config['configfile'].$host)){
				if (unlink($config['configfile'].$host)){
						unlink($config['configfile'].$host);
						$return .= "<pre class='blue'>Host: ".$host." successfully deleted!</pre>";
				}
				else 
					$return .= "<pre class='red'>An error occured while deleting Host: ".$host."</pre>";
			}
			else 
				$return .= "<pre class='red'>File: ".$config['configfile'].$host." does not exist!</pre>\n";# show error message
			break;
		case "NewHost":##################################################
			$return = "<h3>Create (New Host)</h3>";# show headline
			$return .= "<div class='blue'>Please name new host entry:";
			$return .= "<input type='input' name='hostname' value='New Host'>";
			$return .= "<input type='submit' name='action[ReallyNewHost]' value='Create'></div>";
			break;
		case "ReallyNewHost":############################################
			$return = "<h3>Create (".$_POST['hostname'].")</h3>";# show headline
			if (file_exists($config['configfile'].$_POST['hostname'])) 
				$return .= "<pre class='red'>Host: <em>".$_POST['hostname']."</em> already exists!</pre>";
			else{
				if (copy('default', $config['configfile'].$_POST['hostname'])) {
					$return .= "<pre class='red'>New Host: <em>".$_POST['hostname']."</em> created!</pre>";
					$return .= parse_file_contents($config['configfile'].$_POST['hostname']);
					$return .= "<input type='hidden' name='hostname' value='".$_POST['hostname']."'>";
					$return .= "<pre class='blue'>".edit_conf_file($inputvalues, $config, $mode)."</pre>";
				}
				else 
					$return .= "<pre class='red'>Error while copying!</pre>";
			}
			break;
		default:#########################################################
			$return = "<h3>Error (".$menue_entry.")</h3>";# show headline
			$return .= "<pre class='red'>Please choose one of the options above!</pre>";
			break;
	}
}
elseif (isset($_POST['Comment'])){# comment button pressed
	$menue_entry = clean_html(array_keys($_POST['Comment'])[0]);# Comment[<config file option>][<line no>] -> <+/->
	$line = clean_html(array_keys($_POST['Comment'][$menue_entry])[0]);
		if (isset($_POST['hostname'])){
			$return = "<h3>Edit (".$_POST['hostname'].")</h3>";# show headline
			$return .= "<input type='hidden' name='hostname' value='".$_POST['hostname']."'>";
			$inputvalues = $_POST['inputvalues']; unset($inputvalues['comment'][$menue_entry][$line]);# delete specific line from variable			
			$return .= "<pre class='blue'>".edit_conf_file($inputvalues, $config, $mode)."</pre>";
		}
		else 
			$return = "<pre class='red'>Error! No host defined!</pre>\n";# show error message
}
else $return = "";
$return .= "<input type='hidden' name='expertMode' value='".$mode."'>";
################# Main Menue End ########################################
echo "<tr><td colspan='".$config['columns']."' align='left'>".$return."</td></tr>";
echo "</tr>\n</tbody></table>\n";
echo "<pre>";print_r($config);echo "</pre>";
echo "<pre> ";print_r($inputvalues);echo "</pre>";
debug_output();
############################# END #################################################################
echo "<br><br>";
echo "</form>\n<footer id='footer'>My personal page hosted on my own server &copy; olkn</footer></body></HTML>\n";
########### END ###################################?>

rsnapshot backup solution

In order to backup my systems I have choosen rsnapshot because it stores all files as normal files on the hard disk using links between different backup dates to reduce the file size of the complete backup. This enables me to check directly in the file system for old files. Normally I do not restore a complete backup but use to check for specific files only.
To simplify things further on I found a description to use rsync on the client machines and a server certificate, so that no interaction with the client is needed. The server may connect using his certificate and start a local rsync server on the client which handles the backup itself. And now to the steps required to setup the system on the client:

CLIENT
First we need rsync installed:

aptitude install rsync

then we need to create a special backup user:

useradd backupuser -c "Backup" -m -u 4210

Within the home folder of the new user we need to store the server certificate as .ssh/authorized_keys

mkdir .ssh
scp /home/backupuser/.ssh/authorized_keys

Now we need a special script which starts the rsync programm on the client listening for incoming requests:

scp /home/backupuser/

Finally some tweaking for the permissions:

chown backupuser:backupuser
chmod 755

We also need to allow the new user to execute rsync, so we adjust sudo for this by editing /etc/sudoers:

Defaults:backupuser !requiretty Defaults:backupuser !visiblepw backupuser ALL:NOPASSWD: /usr/bin/rsync

Thats it!
I also wrote a simple script to show a list of available rsnapshot config files stored on the backup server and start the backup for this script meaning backing up a specific system.
Here are the relevant scripts:
First the wrapper script which simply logs the current date and starts rsync using the command line options from rsnapshot remote config:

#!/bin/sh

/usr/bin/logger "BACKUP: Start backup at `/bin/date` "; /usr/bin/sudo /usr/bin/rsync "$@"; /usr/bin/logger "BACKUP: Backup finalized at `/bin/date` ";

And here is the console script to start the backup:

#!/bin/sh # DIALOG="/usr/bin/dialog" RSNAPSHOT="/usr/bin/rsnapshot" CONFIGDIR="" BACKUPDIR="" CONFIGFILES=""


##### Check if Backupdisk is connected and correctly mounted ################

check_for_backup_dir() {

	if [ ! -e $BACKUPDIR/ps2 ]

	 then

		if [ $# -ne 0 ]

		then

			$DIALOG --clear --title "rsnapshot config" --msgbox "The backup disk is not connected to the machine!" 10 52

		else

			echo "The backup disk is not connected to the machine!"

		fi

	 exit

	 fi

}
##### main ######

# choose dialog interface or command line parameter

if [ $# -ne 0 ]

then

	if [ $# -gt 1 ]

	then

		echo "Usage: backup.sh or backup.sh "

		echo "       too many arguments on command line"

		exit

	fi

	check_for_backup_dir

	# command parameter available so no dialog interface used

	if [ ! -e $CONFIGDIR/$1 ]

	then

		echo "No config file for system $1"

		exit

	else

		COMMAND="$RSNAPSHOT -q -c $CONFIGDIR/$1 weekly"

		$COMMAND

	fi

else

	check_for_backup_dir dialog

	# no parameter available on command line so using dialog for choosing configuration

	for file in $CONFIGDIR/*

	do # create one variable including all available configs

	   CONFIGFILES="$CONFIGFILES $file ! "

	done

	TEMPFILE=`tempfile 2>/dev/null` || TEMPFILE=/tmp/test$$

	trap "rm -f $TEMPFILE" 0 1 2 5 15

	$DIALOG --clear --title "rsnapshot config" --menu "Choose rsnapshot configuration:" 18 50 10 $CONFIGFILES 2> $TEMPFILE

	RUECKGABE=$?

	case $RUECKGABE in

	  0)

		AUSWAHL=`cat $TEMPFILE`

		COMMAND="$RSNAPSHOT -q -c $AUSWAHL weekly"  ### execute backup command

		$COMMAND

		;;

	  1)

  		echo "Backup canceled!"

		;;

	  255)

 	 	echo "ESC pressed!"

		;;

	esac
	trap "rm -f $TEMPFILE" 0 1 2 5 15

fi # clean up and exit after all work is done exit

The rsnapshot config files always look like the original ones delivered with rsnapshot except for the ssh section which starts the remote wrapper script shown above:

################################################# # rsnapshot.conf - rsnapshot configuration file # ################################################# # # # PLEASE BE AWARE OF THE FOLLOWING RULES: # # # # This file requires tabs between elements # # # # Directories require a trailing slash: # # right: /home/ # # wrong: /home # # # ################################################# # ####################### # CONFIG FILE VERSION # #######################


config_version	1.2
###########################

# SNAPSHOT ROOT DIRECTORY #

###########################
# All snapshots will be stored under this root directory.

snapshot_root	
# If no_create_root is enabled, rsnapshot will not automatically create the

# snapshot_root directory. This is particularly useful if you are backing

# up to removable media, such as a FireWire drive.

#

no_create_root	1
#################################

# EXTERNAL PROGRAM DEPENDENCIES #

#################################
# LINUX USERS:   Be sure to uncomment "cmd_cp". This gives you extra features.

# EVERYONE ELSE: Leave "cmd_cp" commented out for compatibility.

#

# See the README file or the man page for more details.

#

cmd_cp		/bin/cp
# uncomment this to use the rm program instead of the built-in perl routine.

cmd_rm		/bin/rm
# rsync must be enabled for anything to work.

cmd_rsync	/usr/bin/rsync
# Uncomment this to enable remote ssh backups over rsync.

cmd_ssh		/usr/bin/ssh
# Comment this out to disable syslog support.

cmd_logger	/usr/bin/logger
# Uncomment this to specify a path to "du" for disk usage checks.

cmd_du		/usr/bin/du
#########################################

#           BACKUP INTERVALS            #

# Must be unique and in ascending order #

# i.e. hourly, daily, weekly, etc.      #

#########################################
# The interval names (hourly, daily, ...) are just names and have no influence

# on the length of the interval. The numbers set the number of snapshots to

# keep for each interval (hourly.0, hourly.1, ...).

# The length of the interval is set by the time between two executions of

# rsnapshot , this is normally done via cron.

# Feel free to adapt the names, and the sample cron file under /etc/cron.d/rsnapshot

# to your needs. The only requirement is that the intervals must be listed

# in ascending order. To activate just uncomment the entries.
interval	weekly	5
############################################

#              GLOBAL OPTIONS              #

# All are optional, with sensible defaults #

############################################
# If your version of rsync supports --link-dest, consider enable this.

# This is the best way to support special files (FIFOs, etc) cross-platform.

# The default is 0 (off).

# In Debian GNU cp is available which is superior to link_dest, so it should be

# commented out (disabled).

#

#link_dest	0
# Verbose level, 1 through 5.

# 1    Quiet           Print fatal errors only

# 2    Default         Print errors and warnings only

# 3    Verbose         Show equivalent shell commands being executed

# 4    Extra Verbose   Show extra verbose information

# 5    Debug mode      More than you care to know

#

verbose		2
# Same as "verbose" above, but controls the amount of data sent to the

# logfile, if one is being used. The default is 3.

loglevel	3
# If you enable this, data will be written to the file you specify. The

# amount of data written is controlled by the "loglevel" parameter.

logfile		
# The include and exclude parameters, if enabled, simply get passed directly

# to rsync. If you have multiple include/exclude patterns, put each one on a

# seperate line. Please look up the --include and --exclude options in the

# rsync man page for more details.

# 
exclude		/dev

exclude		/lost+found

exclude		/media

exclude		/mnt

exclude		/proc

exclude		/run

exclude		/sys

exclude		/tmp
# The include_file and exclude_file parameters, if enabled, simply get

# passed directly to rsync. Please look up the --include-from and

# --exclude-from options in the rsync man page for more details.

#

#include_file	/path/to/include/file

#exclude_file	/path/to/exclude/file
# Default rsync args. All rsync commands have at least these options set.

#

rsync_long_args		-v --numeric-ids --relative -ev --rsync-path=/home/backupuser/rsync-wrapper.sh
# ssh has no args passed by default, but you can specify some here.

#

ssh_args	-i 
# Default arguments for the "du" program (for disk space reporting).

# The GNU version of "du" is preferred. See the man page for more details.

#

du_args	-csh
# If this is enabled, rsync won't span filesystem partitions within a

# backup point. This essentially passes the -x option to rsync.

# The default is 0 (off).

#

#one_fs		0
# If enabled, rsnapshot will write a lockfile to prevent two instances

# from running simultaneously (and messing up the snapshot_root).

# If you enable this, make sure the lockfile directory is not world

# writable. Otherwise anyone can prevent the program from running.

#

lockfile	/path-to-pid-file
###############################

### BACKUP POINTS / SCRIPTS ###

###############################

# LOCALHOST backup @:/ root/

SERVER
We have already seen the backup script that I use to keep an overview on all my systems and choosing a specific system to backup. For this to run of course we need rsnapshot, rsync and dialog.

aptitude install rsync rsnapshot dialog

The a dedicated user is required which I will call backupserver:

useradd backupserver -c "Backup user on server" -m

For this user we need to create a certificate for ssh:

ssh-keygen

Follow the instructions and do not give a password because we would need to showup this password all the times taking a backup.
Now we are able to spread the public part of the certificate to all client machines and we are done.
Finally I installed screen and used that as the default shell for the backupserver user in order to close – detache – the current connection to my headless backup server while the backup is running in back.

aptitude install screen

I also add the following to /etc/screenrc which gives me a normal bash in debian when using screen:

shell /bin/bash

Just for my own insufficiency:

Use the correct ownership and rights for the backup folder in order to allow the backupserver user to write the backup in the folder and always use this user when performing backups. Do not use the root user for that because it will destroy the rights structure in the backup folder and does not function with the certificate created earlier.

picture viewer

This is my own version of a picture archive and view web interface. On the corresponding server the pictures are stored in folders for year, month and day. There is also the possibility to create specific albums with pictures as well as storing pictures in a folder to reprint/copy.

<?php ############# HEAD  ############################
echo "<!DOCTYPE html>\n";
echo "<html>\n<head>\n <meta charset='utf-8'><meta name='viewport' content='width=device-width, initial-scale=1.0'>\n";
############# style sheets ########################
echo "<link rel='stylesheet' type='text/css' href='stylesheet.css'>";
echo "<title>PictureViewer</title>\n</head>\n<body>\n\n";
echo "<header><h1>My Thoughts</h1><h4>sniplets worth to be remebered</h4></header>\n";
############# Configuration ##########################
$PHP_SELF = htmlspecialchars($_SERVER['PHP_SELF']); # use one file only which will be reload with new parameters all the time
$pathes = array("pictures" => "BackupFromMobile",
				"reprint" => "BackupFromMobile/reprint",
				"albums" => "BackupFromMobile/albums");
$thumb_params = array("width" => 300, "height" => 300, "row" => 5, "big" => 500);# thumbnail parameters
$folder_to_hide = array("pictures", "icons", "thumbnails", "bin");# some folders should not be showed normally
$file_extensions = array( "image" => array(".jpg", ".jpeg", ".png", ".gif"),
						  "audio" => array(".mp3", ".ogg", ".wav"),
						  "ebook" => array(".epub", ".pdf", ".rtx"),
						  "video" => array(".avi", ".3gp", ".flv"));# allowed extensions for files
$rotation_select_box="<option selected value='none'>none</option>
		<option value='0'>0&deg;</option>
		<option value='90'>+90&deg;</option>
		<option value='180'>+180&deg;</option>
		<option value='270'>-90&deg;</option>";# select box for image rotation in 90 degree grade
####### clear html variables from malicious code
function clean_html($variable){
	return trim(htmlspecialchars($variable));
}
###### debug output of POSTå
function debug_output(){
	echo "<pre><br><h1>Hier kommt der schamass!</h1>";
	print_r($_POST);
	echo "</pre>"; return ;
}
############# create thumbnails in subfolder ######
function check_thumbnail($path, $filename){
	global $thumb_params;
	if (!file_exists($path."thumbnails")){#create thumbnail folder
		if (!mkdir($path."thumbnails")) die ("Unable to create folder (".$path."thumbnails)!");
	}
	$thumb = new Imagick(realpath($path."/".$filename));# Imagick needs absolute path for reference
	$orientation = $thumb -> getimageorientation();# check for image orientation
	# check if rotation was selected
	if (!empty($orientation)){
		switch ($orientation){# rotate image to get it upright
			case imagick::ORIENTATION_BOTTOMRIGHT:
				$thumb->rotateimage("#000000", 180);
				break;
			case imagick::ORIENTATION_RIGHTTOP:
				$thumb->rotateimage("#000000", 90);
				break;
			case imagick::ORIENTATION_LEFTBOTTOM:
				$thumb->rotateimage("#000000", -90);
				break;
			case imagick::ORIENTATION_UNDEFINED:
				$thumb->rotateimage("#000000", 90);
				break;
		}
		$thumb->setImageOrientation(imagick::ORIENTATION_TOPLEFT);# set exif data
	}# resize image but keep ratio
	$thumb->setResourceLimit( Imagick::RESOURCETYPE_MEMORY, 2 );
	$thumb->thumbnailImage($thumb_params["width"], $thumb_params["height"], true);
	$thumb->writeImage(realpath($path)."/thumbnails/".$filename);
	return ; }
############# show side navigation #######################
function explore_directories($current_path, $target_path, $expert_mode){
	global $file_extensions, $folder_to_hide;
	echo "<ul class='navigation'>\n";
	foreach (scandir($current_path, 0) as $element){
		if (substr($element, 0, 1) != "." && !in_array($element, $folder_to_hide)){
			if (is_file($current_path."/".$element)){
				echo "<input type='hidden' name='elements[]' value='".$current_path."/".$element."'>";
				foreach ($file_extensions as $key => $value){# extensions are grouped in multidimensional array
					if (in_array(strtolower(strrchr($element, ".")), $file_extensions[$key])){# only some file extensions will be shown
						echo "<li class='actual_entry'><label><img src='icons/file.png'>";
						echo "<input class='hidden' type='submit' name='folder[".$key."]' value='".$current_path."/".$element."'>".$element."</label></li>\n";
					}
				}
			}
			else { #subfolder
				echo "<li class='navigation'><label><img src='icons/dir.png'>";
				echo "<input class='hidden' type='submit' name='folder[folder]' value='".$current_path."/".$element."/test'>".$element."</label></li>\n";
			}
			if (strncmp($current_path."/".$element, $target_path, strlen($current_path."/".$element)) == 0){
				explore_directories($current_path."/".$element, $target_path, $expert_mode);
			}
		} elseif ($element == "reprint" && $expert_mode == "normal"){# show reprint folder contents if in expert mode
			echo "<li class='navigation'><label><img src='icons/dir.png'>";
			echo "<input class='hidden' type='submit' name='folder[folder]' value='".$current_path."/".$element."/test'>".$element."</label></li>\n";
		}
	}
	echo "</ul>\n";
	return ; }
############ show thumbnails #############################
function show_thumbnails($path, $number_of_rows, $expert_mode){
	global $file_extensions,$rotation_select_box, $pathes;
	$i=0;
	foreach (scandir($path) as $file){# iterate over complete directory contents
		if (substr($file,0,1) != "." && in_array(strtolower(strrchr($file, ".")), $file_extensions["image"])){# only allowed extension will be shown
			$file = htmlspecialchars($file); $i++;# clean filename
			if (!file_exists($path."/thumbnails/".$file)){
				check_thumbnail($path."/", $file);
			}
			echo "<div class='gallery'><label><input class='hidden' type='submit' name='folder[image]' value='".$path."/".$file."'>";
			echo "<img src='".$path."/thumbnails/".$file."' alt='".$file."'></label>\n";
			if ($expert_mode == "normal"){# show checkbox for reprint
				if (strncmp($path, $pathes['reprint'], strlen($pathes['reprint'])) == 0 ) {
					echo "<p class='delete'><input type='checkbox' name='delete".$i."' value='".$path."/".$file."'> delete from reprint list </p>\n";
				}
				else {
					echo "<p class='reprint'><input type='checkbox' name='reprint".$i."' value='".$path."/".$file."'> reprint <span> rotate <select name='rotate".$i."'>";
					echo $rotation_select_box."</select></span><input type='hidden' name='rotation".$i."' value='".$path."/".$file."'></p>\n";# we need the filename beside the rotation value
					if (strncmp($path, $pathes['albums'], strlen($pathes['albums'])) == 0)
						echo "<p class='delete'><input type='checkbox' name='delete".$i."' value='".$path."/".$file."'> delete from album </p>\n";
					else
						echo "<p class='album'><input type='checkbox' name='album".$i."' value='".$path."/".$file."'> add to album choosen above</p>\n";
				}
			}
			echo "</div>\n";
		}
	}
	return ;}
############# rotate pictures from list in given direction ##############
function rotatepictures($rotation, $files){
	global $thumb_params; $i=0;
	$path = dirname($files[0]);
	foreach ($files as $value) {
		$thumb = new Imagick(realpath($value));# Imagick needs absolute path for reference
		switch ($rotation[$i]){# rotate image to get it upright
			case "0";
				$thumb->rotateimage("#000000", 0);
				break;
			case "90";
				$thumb->rotateimage("#000000", 90);
				break;
			case "180":
				$thumb->rotateimage("#000000", 180);
				break;
			case "270":
				$thumb->rotateimage("#000000", -90);
				break;
		}
		$thumb->setImageOrientation(imagick::ORIENTATION_TOPLEFT);# set exif data
		$thumb->setResourceLimit( Imagick::RESOURCETYPE_MEMORY, 2 );# resize image but keep ratio
		$thumb->thumbnailImage($thumb_params["width"], $thumb_params["height"], true);
		$thumb->writeImage(realpath($path)."/thumbnails/".basename($value));
		$i++; # take a counter with us to get desired orientation
	}
	return "done";}
############# copy pics for reprint ###################
function copypictures($files, $folder){
	if (!file_exists($folder)){
		if (!mkdir($folder)) die("Could not create folder:".$folder);
	}
	foreach ($files as $value) {
		if (!copy($value, $folder."/".basename($value) ))
				die("Could not copy file".$value." to ".$folder."/".basename($value));
		}
	return "done";}
############# delete pictures from album ##############
function delpictures($files){
	foreach ($files as $value) {
		if (!unlink($value))
			die("Could not delete file".$value);
	}
	return "done";}
######################################################################
############################## MAIN ##################################
######################################################################
echo "<body><form method='POST' action='$PHP_SELF'>\n";# page header with headline and table headers
$filelist=array(); $albumlist=array();$dellist=array();$rotatelist=array();$rotationnamelist=array();# list of files for reprint
foreach (array_diff_key($_POST, array("change" => "", "action" => "", "folder" => "", "album" => "")) as $key => $value){
	if (substr($key, 0, 7) == "reprint")# files for reprint are numbered as reprint1..9
		array_push($filelist, $value);# generate array list of files to reprint
	elseif (substr($key, 0, 5) == "album")# files for album are numbered as album1..9
		array_push($albumlist, $value);# generate array list of files for actual album
	elseif (substr($key, 0, 6) == "delete")# files to delete from album
		array_push($dellist, $value);
	elseif (substr($key, 0, 6) == "rotate" && $_POST[$key] != "none"){# files to rotate
		array_push($rotatelist, $value);# store value for rotation
		$index_for_rotation="rotation".substr($key, 6);
		array_push($rotationnamelist, $_POST[$index_for_rotation]);# get image name to rotate
	}
}
# get default values for action, target folder and expert view mode
# $key = last action choosen, e.g. play video, show pictures
# $target_folder = the last folder that was opened
# $expert_mode = status of expert view
if (isset($_POST['actual_action']) && $_POST['actual_action'] != "")
	$key = clean_html($_POST['actual_action']);
else 
	$key = "folder";# default action is showing pictures in folder
if (isset($_POST['folder']) && $_POST['folder'] != ""){
	foreach ($_POST[folder] as $key => $value)# generate target folder variable from POST
		$target_path = clean_html(dirname($value));# if a new folder icon was clicked update target folder
	$key = clean_html(array_keys($_POST['folder'])[0]);$value = clean_html($_POST['folder'][$key]);
}elseif (isset($_POST['actual_folder']) && $_POST['actual_folder'] != ""){
	$target_path = clean_html($_POST['actual_folder']);$value = clean_html($_POST['actual_contents']);
} else 
	$target_path = $pathes['pictures'];
# check the actual status of expert view and toggle if requested by user button press
if (isset($_POST['change']) && $_POST['change'] == "expert")# button is pressed, so we need to toggle
	$expert_mode = "normal";# change value as user triggered the button to change
elseif (isset($_POST['change']) && $_POST['change'] == "normal")
	$expert_mode = "expert";# keep the old stage if set
elseif (isset($_POST['expert_mode']) && $_POST['expert_mode'] == "normal")
	$expert_mode = "normal";
else 
	$expert_mode = "expert";
echo "<input type='hidden' name='expert_mode' value='".$expert_mode."'><input type='hidden' name='actual_folder' value='".$target_path."'>";
echo "<input type='hidden' name='actual_action' value='".$key."'><input type='hidden' name='actual_contents' value='".$value."'>";
############## show navigation on top ###########################################
echo "<div class='navi'><input type='submit' name='change' value='".$expert_mode."'>";
# manage access to albums in special folder
if ($expert_mode == "normal"){# select album from list if in expert mode
	echo "<select name='album'>\n<option value='new'>create new entry</option>\n";
	foreach ( glob($pathes['albums']."/*", GLOB_ONLYDIR) as $directory){
		echo "<option ";
		if (isset($_POST['album']) && $_POST['album'] == $directory)
			echo " selected ";
			echo "value='".$directory."'>".basename($directory)."</option>\n";
	}
	echo "</select>\n<input type='submit' name='action' value='action'>\n";
}
if (isset($_POST['action'])){# action button pressed
	echo "<span class='response'>";
	if (isset($_POST['new_album']) && !empty($_POST['new_album'])){
		if (!file_exists($pathes['albums']."/".$_POST['new_album']))
			if (!mkdir($pathes['albums']."/".$_POST['new_album'])) die("Could not create folder:".$_POST['new_album']);
	}
	else {
		if (isset($_POST['album']) && $_POST['album'] == "new"){
			echo "Create new album: <input type='text' size='20' name='new_album'>\n";
		}
		elseif (!empty($albumlist))# copy pictures to album
			echo copypictures($albumlist, $_POST['album']);
		if (!empty($filelist)) # copy pictures for reprint
			echo copypictures($filelist, $pathes['reprint']);
		if (!empty($dellist)) # delete entries from album
			echo delpictures($dellist);
		if (!empty($rotatelist))# rotate pictures in given direction
			echo rotatepictures($rotatelist, $rotationnamelist);
}
	echo "</span>\n";
}
explore_directories($pathes['pictures'], $target_path, $expert_mode);# show the navigation menue
echo "</div>\n";
############### page content to the right ##################################
echo "<section class='contents'>";
switch ($key){
	case "audio":
		echo "<".$key." controls height='240' width='320'>\n<source src='".$value."' type='".$key."/".strtolower(substr(strrchr($value, "."),1))."'>\n";
		echo "No support for file: ".$value." in your Browser!</".$key.">\n";
		break;
	case "video":
		echo "<".$key." controls height='240' width='320'>\n<source src='".$value."' type='".$key."/".strtolower(substr(strrchr($value, "."),1))."'>\n";
		echo "No support for file: ".$value." in your Browser!</".$key.">\n";
		break;
	case "ebook":
		echo "<br>Read ebook: ".$value."<br>";
		break;
	case "image":
		echo "<div class='gallery'>\n";
		if (isset($_POST['elements']) && $_POST['elements'] != "") {
			$number_of_pictures = count($_POST['elements'])-1;# index is running from 0 up
			for ($i=0; $i<=$number_of_pictures; $i++){# iterate over complete image list
				if ($_POST['elements'][$i] == $value){ # find the number of the current image in list
					if ($i == 0) $last = $_POST['elements'][$number_of_pictures];# we are at position 0 already
					else $last = $_POST['elements'][$i-1];
					if ($i == $number_of_pictures) $next = $_POST['elements'][0];# we are at the end already
					else $next = $_POST['elements'][$i+1];
					echo "<div class='arrow-left'><button type='submit' name='folder[image]' value='".$last."'>";
					echo "<img src='icons/transparent.png' style='border:none;'></button></div>";
					echo "<a href='".$value."'> <img src='".$value."' width='".$thumb_params["big"]."' alt='".$value."'></a>";
					echo "<div class='arrow-right'><button type='submit' name='folder[image]' value='".$next."'>";
					echo "<img src='icons/transparent.png' style='border:none;'></button></div>";
					break;
				}
			}
		}
		if ($expert_mode == "normal"){# show checkbox for reprint
			if (explode("/", $value)[0] == $pathes['reprint']) {
				echo "<p class='delete'><input type='checkbox' name='delete".$i."' value='".$value."'> delete from reprint list </p>\n";
			}
			else {
				echo "<p class='reprint'><input type='checkbox' name='reprint".$i."' value='".$value."'> reprint <span> rotate <select name='rotate".$i."'>";
				echo $rotation_select_box."</select></span><input type='hidden' name='rotation".$i."' value='".$value."'></p>\n";# we need the filename beside the rotation value
				if (explode("/", $value)[0] == $pathes['albums'])
					echo "<p class='delete'><input type='checkbox' name='delete".$i."' value='".$value."'> delete from album </p>\n";
				else
					echo "<p class='album'><input type='checkbox' name='album".$i."' value='".$value."'> add to album choosen above</p>\n";
			}
		}
		echo "</div>";
		break;
	case "folder":
		show_thumbnails($target_path, $thumb_params['row'], $expert_mode);# show thumbs
		break;
	default:
		echo "<br>File type not supported!<br>";
		break;
}
echo "</section>\n";
############# cleanup and end ########################################		
debug_output();
echo "</form>\n<footer id='footer'>My personal page hosted on my own server &copy; olkn</footer></body></HTML>\n";
########### END ###################################?>

@CHARSET "ISO-8859-1";
body {
    background-color: #2C2C29;
   	font-family:"Times New Roman", Times, serif;
    font-size: 13px;
    color: white;
}
.arrow-right {
    position: absolute;
	width: 20%;
    height: 100%;
    right: -50px;
    top: 0;
}
.arrow-left {
    position: absolute;
	width: 20%;
    height: 100%;
    left: -50px;
    top: 0;
}
.arrow-left:hover {
	background: transparent url(icons/arrow-left.png) no-repeat left center;
}
.arrow-right:hover {
	background: transparent url(icons/arrow-right.png) no-repeat right center;
}
.hidden {
	display: none;
}
.response {
	padding: 0 0 0 50px;
}
.navigation {/* navigation divs are nested on the left */
	padding: 0px 0px 0px 12px;
	text-shadow: 4px 3px 0 #444444;
}
ul {
    list-style-type: none;
    padding: 0px 20px 0px 0px;
}
li {
	padding: 0px 0px 0px 12px;
}
.navi { /* show text shadow */
	float: left;
	text-shadow: 4px 3px 0 #444444;
}
a { /* some stylings for links */
	text-decoration:none;
}
a:link {
    color: dimgray;
}
a:visited {
    color: darkgray;
}
a:hover {
    color: royalblue;
}
a:active {
    color: silver;
}
p.reprint { /* caption for reprint */
	background: teal;
	margin: 5px;
	text-align: left;
}
p.reprint span { 
	float:right;
	padding: 0 5px 0 0;
}
p.album { /* caption for add to album */
	background: royalblue;
	margin: 5px;
	text-align: left;
}
p.delete { /* caption for pic delete */
	background: darkred;
	margin: 5px;
	text-align: left;
}
.gallery { /* show nice white border for pics */
	position: relative;
	display: inline-block;
	padding: 5px 5px 20px 5px;
	margin: 20px;
	background: white;
}
.gallery img {
	margin:2px;
	border:1px solid #000000;
  	height:auto;
}
.gallery img:hover { /* blurr image when mouse is over */
	opacity:0.7;
	filter:alpha(opacity=70); 
}
img.arrow {
	border: none;
	margin: 100% 0;
}
button {
	height: 100%;
	width: 130px;
	background: transparent;
	border: none;
}
footer { /* footer in new line */
	clear: left;
}

rsync setup for client backup

I am using rsync for keeping my photos on the mobile phone in sync with my server. The complete system is growing and growing and the actual state is able to sync photos, books, music and movies. And I also wrote a very limited simple interface to show the contents on the web after syncing. Photos are stored in the server in a date-oriented form. The php script to handle the files on the server side may be found here.
The app is just installed from google play (https://play.google.com/store/apps/details?id=eu.kowalczuk.rsync4android&hl=en). Follow the instructions to generate a key and copy the public key to the server ~/.ssh/authorized_keys – possibly you need to add the key to already stored keys in your authorized_keys file

The setup in android should state the following options:
rsync -vHrltD --exclude=thumbnails --chmod=ug+rwx,o-rwx --perms -e "ssh -y -p 22 -i "mobile@:BackupFromMobile/pictures
for the other folders like:

pictures

ebooks

music

movies

the same settings apply.

Tine 2.0 Setup

To enable SMTP and IMAP support in Tine 2.0 the following points must be followed:

enable system account in the setup (Systemkonto verwenden)
the login name and password must be identical in Tine as well as on the email server

The setup of Tine 2.0 itself is straight forward and following the general installation guide.

foxmarks sync with own server

Just add your own server in the foxmarks configuration under expert settings and please remember to define a file rather than a directory in order for foxmarks to synchronize with.

The first shot may fail so please synchronize your bookmarks twice or more times seperate by a firefox reboot.