Historically, Fedora fulfilled its promise of durability by choosing transparent forms of persistence (e.g. human-readable XML) and using them in ways that systems outside the repository could readily penetrate if needed. Transparency in support of durability is as valid a principle as ever, but there is a weakness to it: transparent forms of persistence are not performant. What's more, many users didn't particularly care for that principle, but they were still stuck paying the performance costs associated with it. So in Fedora 4, we shifted responsibility for transparent persistence away from the core repository software. If you'd like to maintain some simple, human-readable form of your repository, that's fine, but you need to support that with an integration around the core. The form of persistence used by the core repository component itself is not meant to be manipulated directly by a human except in the most unusual situations, it's meant instead for use by the software to provide speedy service at the repository's API. You might compare this to the use of database software. You don't expect to directly manipulate database indexes, and if you are concerned for the durability of your data in the database, you take backups in a transparent format and use _those_ to ensure durability.

An analogy: you may expect your bank to provide downloadable images for any checks you write, but you don't expect them to use those images to run their accounting software. 

If a POST body specifying a writeable directory (local to Fedora 4 server) is not included in the request, the backup will be written to the system temp directory.

POST /rest/fcr:backup

> optional POST body

Note: Restoring a backup replaces the repository content with the contents of the backup, so any data in the repository will be lost.

POST /rest/fcr:restore

> with POST body

#!/bin/bash

# Location of the fcrepo home directory
FCREPO_HOME=/var/lib/tomcat7/fcrepo4-data
# Destination Directory
BACKUP_TO=/home/vagrant/backup
# Backup exact (without the structural changes introduces by Python verfication script).
# (Additional temporary storage is used, if true)
BACKUP_EXACT=true
# Max Backup Attempts on failure
ATTEMPTS=10

echo `date`" $0: fcrepo home dir: $FCREPO_HOME"
echo `date`" $0: backup dir: $BACKUP_TO"
echo `date`" $0: max attempts on failure: $ATTEMPTS"

if [ ! -d $BACKUP_TO ]; then
  mkdir $BACKUP_TO
fi

LEVELDB_DIR=fcrepo.ispn.repo.cache
DATA_DIR=dataFedoraRepository
backup_succeeded=false
attempts=$ATTEMPTS

echo `date`" $0: Backing up leveldb."
while [ $attempts -gt 0 ]; do
	MANIFEST_FILE=`ls $FCREPO_HOME/$LEVELDB_DIR/$DATA_DIR/MANIFEST-*`
  MANIFEST_MD5=`md5sum $MANIFEST_FILE`
  rm -rf $BACKUP_TO/$LEVELDB_DIR"-tmp"
  cp -r $FCREPO_HOME/$LEVELDB_DIR $BACKUP_TO/$LEVELDB_DIR"-tmp"
  copy_success=$?
  MANIFEST_FILE_POST_BACKUP=`ls $FCREPO_HOME/$LEVELDB_DIR/$DATA_DIR/MANIFEST-*`
  MANIFEST_MD5_POST_BACKUP=`md5sum $MANIFEST_FILE_POST_BACKUP`
  if [ "$MANIFEST_MD5" = "$MANIFEST_MD5_POST_BACKUP" ] && [ "$copy_success" = "0" ]; then
  	backup_succeeded=true
  	break;
  fi
  attempts=$((attempts - 1))
  echo `date`" $0: leveldb manifest changed during backup process! $attempts attempts remaining."
done

if [ "$backup_succeeded" = false ]; then
	echo `date`" $0: Failed to backup with a consistent leveldb manifest!"
else
	echo `date`" $0: Backup created and verified leveldb manifest consistency!"
fi

if [ "$BACKUP_EXACT" = true ]; then
	rm -rf $BACKUP_TO/$LEVELDB_DIR"-unchanged"
	cp -r $BACKUP_TO/$LEVELDB_DIR"-tmp" $BACKUP_TO/$LEVELDB_DIR"-unchanged"
fi

backup_repaired=false
# Verify and repair using python script
python verify_leveldb.py $BACKUP_TO/$LEVELDB_DIR"-tmp"/$DATA_DIR
if [ "$?" != "0" ]; then
	echo `date`" $0: Discovered backup corruption! Attempting to repair!"
	python repair_leveldb.py $BACKUP_TO/$LEVELDB_DIR"-tmp"/$DATA_DIR
	if [ "$?" != "0" ]; then
		echo `date`" $0: Backup repair failed!"
	else
		python verify_leveldb.py $BACKUP_TO/$LEVELDB_DIR"-tmp"/$DATA_DIR
		if [ "$?" != "0" ]; then
			echo `date`" $0: Backup repair failed!"
		else
			echo `date`" $0: Backup repair succeeded!"
			backup_repaired=true
			backup_succeeded=true
		fi
	fi
else
	echo `date`" $0: Backup passed corruption verification!"
fi

if [ "$backup_succeeded" = true ]; then
	rm -rf $BACKUP_TO/$LEVELDB_DIR
	if [ "$BACKUP_EXACT" = true ] && [ "$backup_repaired" = false ]; then
		mv $BACKUP_TO/$LEVELDB_DIR"-unchanged" $BACKUP_TO/$LEVELDB_DIR
	else
		mv $BACKUP_TO/$LEVELDB_DIR"-tmp" $BACKUP_TO/$LEVELDB_DIR
	fi
fi

#!/usr/bin/python
import sys
import leveldb
from datetime import datetime
import subprocess
if (len(sys.argv) != 2) :
  print 'Usage:'
  print 'python verify_leveldb.py /path/to/leveldb/cache/'
  sys.exit(1)
dbpath = sys.argv[1]
try:
  startTime = datetime.now()
  print sys.argv[0], ": Inspecting db: ", dbpath
  db = leveldb.LevelDB(dbpath,  create_if_missing=False, paranoid_checks=True)
  it = db.RangeIter(None, None, True, True, True)
  records = 0;
  for key, value in it:
    records = records + 1
  print sys.argv[0], ": Backup verfiication successful!"
  print sys.argv[0], ": Total records: ", records
  print sys.argv[0], ": Time taken:", datetime.now() - startTime
except:
  print sys.argv[0], ": Backup verfication failed!"
  print sys.argv[0], ": Unexpected error:", sys.exc_info()[0]
  raise
  exit(1)
# Rename ldb files to sst (Python API uses ldb extension, whereas infinispan expects sst)
script = 'for f in ' + dbpath + '/*.ldb; do mv $f "${f%.ldb}.sst"; done'
subprocess.call(['/bin/bash', '-c', script])

#!/usr/bin/python
import leveldb
import sys
import os
import subprocess

if (len(sys.argv) != 2) :
  print 'Usage:'
  print 'python repair_leveldb.py /path/to/leveldb/cache'
  sys.exit(1)

path = sys.argv[1]
if not os.path.isdir(path) :
  print sys.argv[0], ": Directory %s does not exist!" % path
  sys.exit()
 
# Use the leveldb module to repair the files
leveldb.RepairDB(path)

# Rename ldb files to sst (Python API uses ldb extension, whereas infinispan expects sst)
script = 'for f in ' + path + '/*.ldb; do mv $f "${f%.ldb}.sst"; done'
subprocess.call(['/bin/bash', '-c', script])