Versions Compared

Old Version 48

changes.mady.by.user Yinlin Chen

Saved on

compared with

New Version Current

changes.mady.by.user David Wilcox

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Excerpt

To support horizontal scalability use cases as well as geographic distribution, Fedora 4 can be configured as a cluster of application servers.

Overview

Note

This feature is still undergoing development.

Warning
titleIssues

The are still some issues present in Fedora 4 which may lead to partial ingests, due to synchronization timeouts. This can be partialy mitigated by increasing the replTimeout property in infinispan.xml.

Configuration 

Fedora 4 is built in top of the JCR implementation Modeshape. Modeshape uses Infinispan as a distributed datastore, which in turn uses the Messaging Toolkit JGroups to transfer state between nodes.

...

Deploy cluster using the UDP Multicast protocol for node discovery and the TCP protocol for replication

A couple of few configuration options have to be set in order to have Fedora 4 work as a cluster on a local machine:

...

  1. Build the fcrepo war file or download the prebuilt fcrepo war file
    1. Build the War file as described on this page
    2. Fetch the WAR file from the download page
  2. Get Tomcat

    1. Download Tomcat 7 and unpack it (Tomcat 7.0.50 and unpack itis used in this example)

      Code Block
      #> wget http://mirror.synyx.de/apache/tomcat/tomcat-7/v7.0.50/bin/apache-tomcat-7.0.50.tar.gz
#> tar -zxvf apache-tomcat-7.0.50.tar.gz 
#> mv apache-tomcat-7.0.50 tomcat7
  3. Put the WAR file into tomcat's webapp directory or create a symbolic link

    1. Copy the fcrepo-webapp-VERSION.war file

      Code Block
      #> cp fcrepo-webapp-VERSION.war tomcat7/webapps/fcrepo.war
  4. Set the send/recv buffer sizes if neccessary

    1. Use the following commands to set the buffer size.  To persist these settings between reboots, also put them in /etc/sysctl.conf.

      Code Block
      #> sysctl net.core.rmem_max=26214400 
#> sysctl net.core.wmem_max=5242880
  5. Start instances

    1. Using a custom configuration by pointing Fedora 4 to custom configuration files:

      Code Block
      #> CATALINA_OPTS="-Xmx1024m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true -Djgroups.udp.mcast_addr=239.42.42.42 -Dfcrepo.modeshape.configuration=file:///path/to/repository.json -Dfcrepo.ispn.jgroups.configuration=/path/to/jgroups-fcrepo-tcp.xml  -Dfcrepo.infinispan.cache_configuration=/path/to/infinispan.xml" bin/catalina.sh run

...

  1. Build the fcrepo war file or download the prebuilt fcrepo war file
    1. Build the War file as described on this page
    2. Fetch the WAR file from the download page
  2. Get Tomcat

    1. Download Tomcat 7 and unpack it (Tomcat 7.0.50 and unpack itis used in this example)

      Code Block
      #> wget http://mirror.synyx.de/apache/tomcat/tomcat-7/v7.0.50/bin/apache-tomcat-7.0.50.tar.gz
#> tar -zxvf apache-tomcat-7.0.50.tar.gz 
#> mv apache-tomcat-7.0.50 tomcat7
  3. Put the WAR file into tomcat's webapp directory or create a symbolic link

    1. Copy the fcrepo-webapp-VERSION.war file

      Code Block
      #> cp fcrepo-webapp-VERSION.war tomcat7/webapps/fcrepo.war
  4. Setup the cluster configuration (Optional)
    1. This github project contains a sample configuration for a cluster in distributed mode.
    2. Change the configuration as required. Description is available at the JGroups, Infinispan and Modeshape documentations
    3. Make sure to point Fedora 4 to the configuration files by updating the file $TOMCAT_HOME/bin/setenv.sh (create if necessary) using the properties fcrepo.modeshape.configuration,  fcrepo.ispn.jgroups.configuration and fcrepo.infinispan.cache_configuration

       

  5. Set the send/recv buffer sizes if neccessary

    1. Use the following commands to set the buffer size

      Code Block
      #> sysctl net.core.rmem_max=5242880 
#> sysctl net.core.wmem_max=5242880
  6. Start instance

    1. Using the default clustered configuration (Replication mode):

      Code Block
      #> CATALINA_OPTS="-Xmx1024m -XX:MaxPermSize=256m -Dfcrepo.modeshape.configuration=config/clustered/repository.json -Djava.net.preferIPv4Stack=true -Djgroups.udp.mcast_addr=239.42.42.42" bin/catalina.sh run

    2. Using a custom configuration by pointing Fedora 4 to custom configuration files:

      Code Block
      #> CATALINA_OPTS="-Xmx1024m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true -Djgroups.udp.mcast_addr=239.42.42.42 -Dfcrepo.modeshape.configuration=file:///path/to/repository.json -Dfcrepo.ispn.jgroups.configuration=/path/to/jgroups-fedora-udp.xml  -Dfcrepo.infinispan.cache_configuration=/path/to/infinispan.xml" bin/catalina.sh run

...

  1. Build the fcrepo war file or download the prebuilt fcrepo war file
    1. Build the War file as described on this page
    2. Fetch the WAR file from the download page
  2. Get Tomcat

    1. Download Tomcat 7 and unpack it (Tomcat 7.0.50 and unpack itis used in this example)

      Code Block
      #> wget http://mirror.synyx.de/apache/tomcat/tomcat-7/v7.0.50/bin/apache-tomcat-7.0.50.tar.gz
#> tar -zxvf apache-tomcat-7.0.50.tar.gz 
#> mv apache-tomcat-7.0.50 tomcat7-8080
  3. Put the WAR file into tomcat's webapp directory or create a symbolic link

    1. Copy the fcrepo-webapp-VERSION.war file

      Code Block
      #> cp fcrepo-webapp-VERSION.war tomcat7-8080/webapps/fcrepo.war
  4. Get the Infinispan XML configuration file

    1. Download from github and put it into tomcat7-8080

      Code Block
      #> wget -O infinispan.xml https://gist.github.com/fasseg/8646707/raw
#> mv infinispan.xml tomcat7-8080/
  5. Get the Modeshape JSON configuration file

    1. Download from Github and put into tomcat7-8080

      Code Block
      #> wget -O repository.json https://gist.github.com/fasseg/8646727/raw
#> mv repository.json tomcat7-8080/
  6. Get the JGroups UDP Transport configuration file

    1. Download from Github and put into tomcat7-8080

      Code Block
      #> wget -O jgroups-fedora-udp.xml https://gist.github.com/fasseg/8646743/raw
#> mv jgroups-fedora-udp.xml tomcat7-8080/
  7. Set the send/recv buffer sizes if neccessary

    1. Use the following commands to set the buffer size

      Code Block
      #> sysctl net.core.rmem_max=5242880 
#> sysctl net.core.wmem_max=5242880
  8. Copy the whole tomcat folder in order to create a second instance
    1. Code Block
      #> cp -R tomcat7-8080/ tomcat7-8081
  9. Change the connector ports of the second Tomcat instance to 8081, 8006 and 8010
    1. Code Block
      #> sed -i 's/8080/8081/g' tomcat7-8081/conf/server.xml
#> sed -i 's/8005/8006/g' tomcat7-8081/conf/server.xml
#> sed -i 's/8009/8010/g' tomcat7-8081/conf/server.xml
  10. Start the first instance
    1. Code Block
      #> CATALINA_OPTS="-Xmx1024m -XX:MaxPermSize=256m -Dfcrepo.modeshape.configuration=file:///path/to/repository.json -Djava.net.preferIPv4Stack=true -Dfcrepo.ispn.jgroups.configuration=/path/to/jgroups-fedora-udp.xml -Djgroups.udp.mcast_addr=239.42.42.42 -Dfcrepo.infinispan.cache_configuration=/path/to/infinispan.xml" tomcat7-8080/bin/catalina.sh run
  11. Start the second instance
    1. Code Block
      #> CATALINA_OPTS="-Xmx1024m -XX:MaxPermSize=256m -Dfcrepo.modeshape.configuration=file:///path/to/repository.json -Djava.net.preferIPv4Stack=true -Dfcrepo.ispn.jgroups.configuration=/path/to/jgroups-fedora-udp.xml -Djgroups.udp.mcast_addr=239.42.42.42 -Dfcrepo.infinispan.cache_configuration=/path/to/infinispan.xml -Djms.port=61617 -Dstomp.port=61614" tomcat7-8081/bin/catalina.sh run
  12. Check that the instances are reachable and that the clustersize is '2':
    1. Code Block
      #> wget http://localhost:8080/fcrepo/rest
#> wget http://localhost:8081/fcrepo/rest
    2. Navigate to http://localhost:8080/fcrepo/rest

...