Federation

Filesystem Federation

Filesystem federation maps a node in the repository to a directory on disk.  This allows files on disk to be served and updated by Fedora 4 as though they were in the repository.  Filesystem federation avoids having to transfer files using HTTP – and with larger file sizes (or with larger numbers of files being processed), this can improve performance significantly.  If you are ingesting a large number of multi-gigabyte files, we recommend you consider filesystem federation.

Another use for filesystem federation is interoperability with another system.  If you have files on disk managed by another application or workflow, you can use filesystem federation to serve them with Fedora 4 without having to ingest them using the REST API or create another copy of the files. 

Configuration

An example filesystem federation configuration to include in your Modeshape repository.json :

"externalSources" : {
    "federated-directory" : {
        "classname" : "org.modeshape.connector.filesystem.FileSystemConnector",
        "directoryPath" : "/path/to/files",
        "projections" : [ "default:/federated => /" ],
        "contentBasedSha1" : "false",
        "readOnly" : true,
        "extraPropertiesStorage" : "none"
   }
}

• directoryPath base directory for all files shared with the repository.

• projections lists one or more mappings from the repository to the filesystem.  The format is "{workspace}:{repository path} => {filesystem path}".

• contentBasedSha1 controls how internal identifiers are computed for files.  By default (contentBaseSha1 = true), Modeshape computes the SHA-1 checksum of a file's content every time the file is accessed.  For small files this creates a modest overhead.  For large files, however, this dramatically reduces performance, since generating the checksum can take several seconds per gigabyte of data.  For this reason, we recommend setting contentBasedSha1 to false when serving files larger than 100MB.

• readOnly controls whether the contents of the filesbase directory for all files shared with the repository.

• extraPropertiesStorage sets the format for storing "extra" properties (properties that can't be set using filesystem attributes).  Recommended values are "json" for the current JSON properties format, or "none" for disabling extra properties.
  

Fedora 3 Federation

As discussed in Fedora 3 to 4 Upgrade, another use for federation is to include content from a Fedora 3 repository in Fedora 4.  This could be used as a temporary approach for migrating Fedora 3 content to Fedora 4.  Or it could be used on an ongoing basis, for example to interoperate with applications that require Fedora 3.

Configuration

A sample Fedora 3 connector configuration to include in your Modeshape repository.json :

"externalSources" : {
    "fedora3" : {
        "classname" : "org.fcrepo.connector.fedora3.Fedora3FederationConnector",
        "fedoraUrl" : "http://localhost:${servlet.port}/fedora",
        "projections" : [ "default:/f3 => /" ],
        "username" : "fedoraAdmin",
        "password" : "fedoraAdmin",
        "organizer" : {
            "classname" : "org.fcrepo.connector.fedora3.organizers.GroupingOrganizer",
            "maxContainerSize": 10
        }
    }
}

• fedoraUrl contains the URL of the Fedora 3 repository.
• projections maps a path of the Fedora 4 repository to the Fedora 3 repository.  The format is "{workspace}:{Fedora 4 path} => {Fedora 3 path}".
  
• username and password are the credentials to use to connect to Fedora 3.
• organizer specifies a mapping from the flat Fedora 3 structure to the Fedora 4 structure.  Because Fedora 4 performance can be degraded if there are too many children of a single node, we recommend setting maxContainerSize to 1000 or less.

Further Reading

Modeshape Federation Documentation:

• Federation Overview

• FileSystemConnector documentation and configuration

Custom Connector References:

• Modeshape Connector reference
• FileSystemConnector JavaDoc reference
• Sample Connector Project