Contribute to the DSpace Development Fund

The newly established DSpace Development Fund supports the development of new features prioritized by DSpace Governance. For a list of planned features see the fund wiki page.

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 47 Next »

This document is intended to be kept up to date by the DSpace Release Managers.  It details the steps necessary to perform snapshot and official releases of DSpace and supporting Modules.

Table of Contents

Useful Sonatype Links

For lack of a better place at this time, here's some useful pages on Sonatype which detail the Sonatype Maven Release Process:

Release Numbering Convention

major.minor.subminor (e.g. 1.6.0)

  • Increment subminor for bug-fix only releases (database, translations and core APIs stable)
  • Increment minor for feature changes (database schema, translations, APIs may change, though we'll try and keep API changes as small as possible, and will always provide an upgrade/migration path)

The one exception is that the Language Packs (dspace-api-lang and dspace-xmlui-lang) use the numbering convention major.minor.subminor.sequence-number (e.g. 1.6.0.0). This allows us to release new versions of the language packs more frequently than normal DSpace releases.

Prerequisites

Verify Release Privileges

To perform a release, you must have all of the following:

  1. Write access to the DSpace subversion repository hosted at http://scm.dspace.org/svn/repo; This requires an administrator to add you to the svn repository permissions via the TRAC user group.
  2. Write access to the org.dspace groupId in the snapshot and staging repositories hosted at oss.sonatype.org.  If you don't already have this, you will need to:
  3. You must generate and publish your own personal Code Signing Key (required by Sonatype). Here are two sites that give hints on how to do that:
  4. Write access to projects.dspace.org (which is where the DSpace Projects website is hosted)
    • Ask the previous release manager (or someone else with access) to be given access to this server.
    • Include your proposed username
    • Include a local SSH Public Key (which will be used for authentication).

Setting up a new person with proper access to projects.dspace.org

Here's notes from Mark Diggory on how to give someone else access to the OSUOSL server (projects.dspace.org):

  1. adduser -m <user-name>
  2. Add authorized key to ~/.ssh/authorizedkeys (make sure permissions are correct on file)
  3. Add user to /etc/groups
    maven:x:4002:peterdietz,esm,mdiggory,tdonohue,ben,kim,apache
  4. Make sure umask 002 is in the users ~/.bash_profile so maven permissions are written correctly.

Update Maven settings.xml

DSpace's root pom.xml already has the correct staging and snapshot repositories listed in the OSS parents distributionManagement section.  In order to deploy, you will need to add your Sonatype OSS username and password to your local ~/.m2/settings.xml file:

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
                      http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <servers>
    <!--Login info for Sonatype SnapShot repository-->
    <server>
      <id>sonatype-nexus-snapshots</id>
      <username>YourSonatypeUsername</username>
      <password>YourSonatypePassword</password>
    </server>
    <!--Login info for Sonatype Staging/Release repository-->
    <server>
      <id>sonatype-nexus-staging</id>
      <username>YourSonatypeUsername</username>
      <password>YourSonatypePassword</password>
    </server>
    <!--Login info for DSpace Projects (projects.dspace.org) website @OSUOSL-->
    <server>
       <id>website</id>
       <username>YourOSUOSLUserName</username>
       <privateKey>[Full-path-to]\.ssh\id_rsa</privateKey>
       <passphrase>YourKeyPassphrase (if necessary)</passphrase>
    </server>
  </servers>
</settings>

If you don't yet have a ~/.m2/settings.xml file, you should create one, and copy the full contents above (obviously make sure to put in your username and password).

Avoid Using Maven 2.1.0 or 2.2.0

Make sure you're using a recent version of Maven. As of this writing, the latest was 2.2.1, and it worked fine. In particular, avoid version 2.2.0: it has a serious bug that affects deployments: MNG-4235. Maven 2.1.0 also is problematic, as it produces improper GPG signatures.

For more information see the Pre-Requisities section of theSonatype Maven Repository Usage Guide

Making a Snapshot Release (e.g. 'dspace-x.y.z-SNAPSHOT')

One Step Process

From a clean, up-to-date copy of trunk, run the following command:

  • mvn clean javadoc:jar source:jar deploy

You will have to enter in your GPG passphrase (which you established when you created your Code Signing Key).

The snapshot will be immediately available in the public Sonatype repository: http://oss.sonatype.org/content/groups/public

Making an Official Release (e.g. 'dspace-x.y.z' or 'dspace-x.y.z-rc1')

For More Information

These same steps are also covered in the Sonatype Maven Repository Usage Guide

Release DSpace Language Packs (I18N)

If possible, you should release the DSpace Language Packs (i18n modules) before the main release. Oftentimes, it's best to release these one or two days prior to the DSpace release (that way they've already propagated out to the Maven repository).

At the moment the i18n modules are maintained in SVN under the modules folder:

http://scm.dspace.org/svn/repo/modules/

There are currently two i18n modules you will need to release:

  • dspace-api-lang
  • dspace-xmlui-lang

Version Numbering Convention for Language Packs

Note that the version numbering convention for Language Packs is always the same as the current DSpace release, with an additional .<sequence-number> (e.g. the i18n modules for the 1.6.0 release are numbered as follows: 1.6.0.0, 1.6.0.1, etc.)

For each module, perform the full release steps that follow. To save space, the steps are only listed for one of the modules (but don't forget to run it for both language packs):

  1. Checkout Language Pack Module: svn co https://scm.dspace.org/svn/repo/modules/dspace-api-lang/trunk dspace-api-lang
  2. Do a Dry Run: mvn release:prepare -DdryRun=true
  3. Tag and Increment Version: mvn release:prepare -Dresume=false
    • Make sure to assign a version number of the format: major.minor.subminor.sequence-number (e.g. 1.6.0.0, 1.6.0.1, etc for 1.6.0 releases of language packs)
  4. Deploy Artifacts to Staging in Sonatype: mvn release:perform
  5. Verify and Release Staged Artifacts in Sonatype (see instructions at link)

Final Commits & Preparation

Double Check Language Packs' Version Ranges

Once the Language Packs are released, you may need to modify the DSpace Parent pom.xml (http://scm.dspace.org/svn/repo/dspace/trunk/pom.xml) to reference the new version of the Language Packs. This should be similar to the following:

If possible, you'd only want to commit this after the i18n modules are available in the Maven Repository. But, if you are in a rush, you can commit this change earlier (though be warned that this will break the build process for anyone who hasn't manually installed the i18n modules to his/her local ~/.m2/ directory).

In the main pom.xml, provide the proper version range for each language pack. In the below example, we are saying to use any language pack version which is at least version 1.6.0.0, but is less than version 1.7.0.0:

        <dependency>
             <groupId>org.dspace</groupId>
             <artifactId>dspace-api-lang</artifactId>
             <version>[1.6.0.0,1.7.0.0)</version>
        </dependency>
        <dependency>
             <groupId>org.dspace</groupId>
             <artifactId>dspace-xmlui-lang</artifactId>
             <version>[1.6.0.0,1.7.0.0)</version>
             <type>war</type>
        </dependency>

Ensure HTML & PDF versions of Wiki Documentation is in SVN

Make sure the latest Wiki-based Documentation has been exported as HTML and PDF and committed to [dspace-src]/dspace/docs/ in SVN.

Notes about Generating PDF & HTML

  • When generating the PDF document, use the "Scroll Wiki Exporter" option. Make sure to select the checkbox next to the following options on the "Formatting Tab":
    • Ignore toc macros - ignores all Table of Contents on individual pages
    • Ignore children macros - ignores any child page listings on individual pages
  • When generating to HTML document, just use the normal HTML export (Browse -> Advanced) option. Make sure to deselect the Homepage, as this is not officially part of the DSpace Documentation.

Watch for conflicts when updating HTML Documentation

It's often easier to completely remove all current HTML content before committing the updated HTML content. Because of the HTML directory structure exported by Confluence, it can be very difficult to determine which files need removing (especially old attachments). So, you may want to completely remove the old HTML directory, before committing the updated content. An example follows:

  1. Delete old HTML docs: svn delete http://scm.dspace.org/svn/repo/dspace/trunk/dspace/docs/html
  2. Update your checked-out copy of Trunk: svn up
  3. Locally, recreate the [dspace-source]/dspace/docs/html/ in your checked out copy of Trunk

  4. Copy new HTML docs to [dspace-source]/dspace/docs/html/, so that the index.html file is at html/index.html

  5. Commit new docs back to SVN

Double Check Contents of all README (and similar) files in SVN

Make sure that the contents of all README, LICENSE, NOTICE files are up-to-date in SVN. Most of these files exist in one of two main locations:

  1. [dspace-src]
    • includes LICENSE, NOTICE and README files
  2. [dspace-src]/dspace/
    • includes CHANGES, KNOWN_BUGS, LICENSE, NOTICE and README files

If anything is out-of-date, make sure to update it and commit the proper changes before continuing.

Checkout Trunk or Branch to Release

Checkout a copy of the to-be-release version either from a branch or trunk.

svn co https://scm.dspace.org/svn/repo/dspace/trunk dspace-release

Do a Dry Run

This step is not required, but performs a useful sanity check without committing any changes. From your clean, up-to-date copy of trunk, run the following command:

mvn release:prepare -DdryRun=true

You will have to enter in your GPG passphrase (which you established when you created your Code Signing Key).

If you notice an issue or an error occurs, you can re-run the Dry Run using the following command:

  • mvn release:prepare -DdryRun=true -Dresume=false

Tag and Increment Version

This step will set the version declared in the project's pom.xml files, commit the changes to trunk, tag the release, and finally, check in another trunk change that increments the next development version (e.g. x.y.z-SNAPSHOT) in the pom.xml files.

mvn release:prepare -Dresume=false

(Optionally, you may also include the parameters -Dusername=YourSVNUsername -Dpassword=YourSVNPassword at the end of the above command, though I've not found these to be necessary)

The above command will ask you three basic questions. Here are sample answers for DSpace 1.6.0:

 "What is the release version for: XXX" 1.6.0
 "What is SCM release tag or label for: XXXX" dspace-1.6.0
 "What is the new development version for: XXXX" 1.6.1-SNAPSHOT

You will also have to enter in your GPG passphrase (which you established when you created your Code Signing Key).

If you receive a project dependency error

This may fail to compile part way through the process, complaining that an internal project dependency is not met. If this occurs, don't worry. Just run the following:

  1. mvn install
  2. mvn release:prepare

Backing out of changes

If backing out of this step is needed for any reason, the following will restore the subversion repository and your working copy to the state it was previously in:

  1. mvn release:rollback
  2. svn rm https://scm.dspace.org/svn/repo/dspace/tags/dspace-x.y

Deploy Artifacts to Staging in Sonatype

This step will sign, checksum, and push all release artifacts (including javadocs and sources) to the Sonatype staging repository (http://oss.sonatype.org/). Run the following:

mvn release:perform

You should be prompted by Maven to specify your GPG passphrase (which you established when you created your Code Signing Key). If you run into any issues, it's possible to specify your GPG key and passphrase as arguments to the above command (e.g. -Darguments="-Dgpg.keyname=YourKeyId -Dgpg.passphrase=YourKeyPassword")

If you need to re-deploy

If any errors or problems occur during the deploy, you can re-run mvn release:perform safely after fixing those issues (re-running it will just overwrite existing staged contents).

Verify and Release Staged Artifacts in Sonatype

For screenshots and more details on this step, visit the Sonatype Repository Usage Guide's section on Releasing your artifacts

  1. Login to http://oss.sonatype.org/
  2. Click "Staging Repositories" in the left column, then select the checkbox next to the staged repository on the right. The contents of it will open up at the bottom of the page.
  3. With the staged repository still selected, click the "Close" button at the top.
  4. Download and test that the artifacts in staging are exactly as they should be once deployed to central.
  5. If everything looks good, select the repository and select "Release". The artifacts should be synced to Maven central (http://repo2.maven.org/maven2/org/dspace/) within an hour.

If you need to revert back

If anything is incorrect, select the staged repository and select "Drop". After the problem is resolved, you can re-deploy the artifacts to staging and verify them again. To re-deploy an already-tagged release:
mvn release:perform -Dtag=dspace-x.y -DconnectionUrl=scm:svn:https://scm.dspace.org/svn/repo/dspace -Darguments="-Dgpg.keyname=YourKeyId -Dgpg.passphrase=YourKeyPassword"

Create the Distribution Zips for SourceForge

The previous actions will have checked out the release tag into the target directory under [dspace-src]/target/checkout. Navigate to that directory and execute the creation of the distributions using the following command. This will create two zip, bzip and gzipped files in the target directory. One set of files is the "binary" release, and the other set is the source release.

(Note: If you've already removed the target/checkout directory, you can also checkout a fresh copy of the newly tagged version and run this command from the DSpace parent directory.)

 localhost$ cd target/checkout
 localhost$ mvn package -Pdistributions
 [INFO] Scanning for projects...
 [INFO] ----------------------------------------------------------------------------
 [INFO] Building DSpace Parent Project
 [INFO]    task-segment: [package]
 [INFO] ----------------------------------------------------------------------------
 [INFO] [site:attach-descriptor]
 [INFO] [assembly:single -(execution. default-)]
 [INFO] Building zip: .../target/dspace-1.5.0-beta1-release.zip
 [INFO] Building tar: .../target/dspace-1.5.0-beta1-release.tar.gz
 [INFO] Building tar: .../target/dspace-1.5.0-beta1-release.tar.bz2
 [INFO] Building zip: .../target/dspace-1.5.0-beta1-src-release.zip
 [INFO] Building tar: .../target/dspace-1.5.0-beta1-src-release.tar.gz
 [INFO] Building tar: .../target/dspace-1.5.0-beta1-src-release.tar.bz2
 [INFO] ------------------------------------------------------------------------
 [INFO] BUILD SUCCESSFUL
 [INFO] ------------------------------------------------------------------------

Upload to SourceForge

Upload both the source and binary releases to Sourceforge.net. You can either upload them via the web interface, or copy them over via scp or other command line tools. If you want to copy the files from command line, follow the directions found here: http://apps.sourceforge.net/trac/sourceforge/wiki/File%20management%20service

Using scp to copy to the "DSpace Stable" file directory: (example for 1.6.0 final)
(NOTE: You will need to first create the 'DSpace Stable/1.6.0' directory via SourceForge's Admin UI)

 scp <files> <username>,dspace@frs.sourceforge.net:"/home/frs/project/d/ds/dspace/DSpace\\ Stable/1.6.0/"

Using scp to copy to the "DSpace Release Candidate" file directory: (example for 1.6.0-rc2)
(NOTE: You will need to first create the 'DSpace Release Candidate/1.6.0-rc2' directory via SourceForge's Admin UI)

 scp <files> <username>,dspace@frs.sourceforge.net:"/home/frs/project/d/ds/dspace/DSpace\\ Release\\ Candidate/1.6.0-rc2/"

After the Release is Finished

Don't Announce Until Maven Packages Have Propagated

You must wait for all the packages to be available at http://repo2.maven.org/maven2/org/dspace/ before you announce the release. Until the DSpace packages are available in the Maven repository, no one else will be able to build DSpace using Maven.

  • Add the new contributors to the list: DSpaceContributors
  • Coordinate Announcements with DuraSpace Staff:
    • Announcement on dspace.org, duraspace.org, twitter
    • Ensure that the Latest Release page on dspace.org is updated.
      • Plus, ask dspace.org admins to upload latest documentation in PDF/HTML format
    • Announce on all DSpace mailing lists
    • Link announcement on Home of DSpace Wiki, change any version numbers listed on that page.
  • Update Wiki pages, particularly these pages which refer to the Current and Next Releases:
  • For major releases, create a new branch in SVN for any upcoming sub-minor releases:
    • E.g., after 1.6.0 release, we created a 1.6.x branch for the subsequent 1.6.1 and 1.6.2 releases
  • Updates to JIRA:
    • Move any uncompleted issues to the next DSpace version tag in JIRA.
    • Ask a JIRA Administrator to close out the release in JIRA (this will ensure no new issues can be added to that release).

Possible Errors you may Encounter

"Could not find model file" error (with language packs)

If you encounter one of these errors when building/packaging DSpace:

FATAL ERROR: "Reason: Could not find the model file '../dspace-xmlui-lang'. for project unknown"

OR

FATAL ERROR: "Reason: Could not find the model file '../dspace-api-lang'. for project unknown"

This is a known bug in Maven. The problem is that you likely have a 'dspace-xmlui-lang' or 'dspace-api-lang' folder at the same level as your [dspace-source] parent folder. Essentially, Maven located them and tried to add them into the build process (which it shouldn't have). The fix is to completely delete the 'dspace-xmlui-lang' and 'dspace-api-lang' folders, and try to rebuild DSpace.

  • No labels