Table of Contents
Step by step of how to perform a Release via Maven |
This document is intended to be kept up to date by the DSpace Release Team. It details the steps necessary to perform snapshot and official releases of DSpace and supporting Modules. |
Table of Contents |
For lack of a better place at this time, here's some useful pages on Sonatype which detail the Sonatype Maven Release Process: |
As of 2012 (starting with DSpace 3.0), DSpace has moved to a new release numbering scheme/format. Release numbers will now only consist of two numbers.
Release Numbering Scheme: [major].[minor] (e.g. 3.0, 3.1, 3.2, 4.0)
For more information see DSpace Release Numbering Scheme
The one exception is that the Language Packs ( |
To perform a release, you must have all of the following:
hkp://pgp.mit.edu, as this is the Key Server Sonatype uses for verification:gpg --keyserver hkp://pgp.mit.edu --send-keys [yourKeyID][yourKeyId] can be found by running the following command and copying the alpha-numeric string after the "/" on the "pub" line
gpg --list-keys
DSpace's root pom.xml already has the correct staging and snapshot repositories listed in the OSS parent's '<distributionManagement>' section. In order to deploy, you will need to add your Sonatype OSS username and password to your local ~/.m2/settings.xml file. For example:
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
http://maven.apache.org/xsd/settings-1.0.0.xsd">
<servers>
<!--Login info for Sonatype SnapShot repository-->
<server>
<id>ossrh</id>
<username>YourSonatypeUsername</username>
<password>YourSonatypePassword</password>
</server>
</servers>
</settings>
|
If you don't yet have a ~/.m2/settings.xml file, you should create one, and copy the full contents above (obviously make sure to put in your username and password).
During a release you have to type the passwords of your GPG and SSH keys very often. Every DSpace module produces several files, all have to be signed and transfered to Sonatype. GPG and SSH agents help you to avoid typing passwords again and again. To use an ssh-agent just start it, export the required environment variables and add your ssh-key. To use a gpg agent start it, export the required environment variables and add the following to your ~/.m2/settings.xml:
<settings>
...
<profiles>
<profile>
<id>gpg-profile</id>
<properties>
<gpg.useagent>true</gpg.useagent>
<!-- If you have gpg2 instead, you can tell Maven to use it instead of 'gpg' by uncommenting the following -->
<!--<gpg.executable>gpg2</gpg.executable>-->
</properties>
</profile>
</profiles>
...
</settings> |
On each maven run it may ask you once about your password which is a big improvement.
For DSpace 7.x, you must use Java 11.
Use Maven 3 or above
For more information see the Prerequisites section of the Sonatype Maven Repository Usage Guide
For DSpace 7.x, you must use Maven 3.5.4 or above. It's necessary to regenerate the LICENSES_THIRD_PARTY file (see notes below)
If you normally use a MAVEN_OPTS environment variable on the machine you're using to cut the release, be sure to unset it, with this command:
unset MAVEN_OPTS |
It's highly unlikely the configuration in your MAVEN_OPTS will be useful for the release. It's highly likely to cause problems. Better to be safe.
For a new module (or a major modification), sometimes it can be useful to release a SNAPSHOT version to Maven. That way you can test this SNAPSHOT version in a local DSpace build (or in a Docker build) before you do the official release.
From a clean, up-to-date copy of master/branch, run the following command:
mvn clean javadoc:jar source:jar deployYou will have to enter in your GPG passphrase (which you established when you created your Code Signing Key).
The snapshot will be immediately available in the public Sonatype snapshot repository: https://oss.sonatype.org/content/repositories/snapshots
(NOTE: it's not possible to list the contents of the Snapshot repository, but our DSpace Parent POM references it as a source... so anything released to the Snapshot repository can be immediately tested/used by the DSpace codebase.)
These same steps are also covered in the Sonatype Maven Repository Usage Guide |
On skimming these instructions, this might look like a small thing, but it is not. It is a big information management task. Ask for help in wrangling/verifying the license information, and, if at all possible, DO NOT leave this job for release day. |
Make sure that the contents of all README, LICENSE, LICENSES_THIRD_PARTY, NOTICE files are up-to-date in GitHub. These files reside in [dspace-src]. If anything is out-of-date, make sure to update it and commit the proper changes before continuing.
If this release includes fixes to any draft security advisories, make sure to request CVEs from GitHub as early as you can. As of 2022, GitHub's policies currently say they will respond within 3 working days. Since the announcement of the release requires the CVEs, you'll want to make sure that you have the CVEs assigned as early as reasonably possible.
(Keep in mind requesting CVEs does NOT make the security advisories public. They will not become public until you publish them. We recommend not publishing the security advisories until the release is already completed and the announcement is about to go out.)
The latest version (v2.0.0) of license-maven-plugin requires Maven v3.5.4 or above to run these commands. |
The "LICENSES_THIRD_PARTY" file is now autogenerated via a Maven command (using the codehaus license-maven-plugin). Simply run the following from your local source directory to re-generate this file:
# Install latest version of all dependencies in local cache # (Only necessary if you haven't run this recently) mvn -U clean install # Regenerate LICENSES_THIRD_PARTY file mvn verify -Dthird.party.licenses=true |
On completion, a new, updated version of the LICENSES_THIRD_PARTY file will be written to your source directory. Please double check this file or "git diff" it to see if the changes look reasonable. Here are some things to especially be on the lookout for:
src/main/license/LICENSES_THIRD_PARTY.properties file which corrects all "UNKNOWN" licenses. Finally, rerun the command above to regenerate the new LICENSES_THIRD_PARTY based on this update.<licenseMerges> tag of this plugin: https://github.com/DSpace/DSpace/blob/master/pom.xml#L406If the file was updated, commit it.
Before performing an official release, you should see if the DSpace Language Packs (i18n modules) need an updated release. The easiest way to check if they need to be released it by checking to see if any commits have occurred since the previous release (see below for links). Please note that you can release these I18N Modules on the same day as the main DSpace release. The DSpace parent pom.xml is now configured to also check Sonatype's Release Repository for any Maven artifacts (so you do NOT need to wait for the I18N modules to appear in Maven Central)
At the moment the i18n modules are maintained in two separate GitHub projects. There are currently two i18n modules you will need to release:
dspace-api-lang - Check if any new commits have occurred on 'dspace-api-lang' since the last release. (Also required for 7.x and above!)dspace-xmlui-lang - Check if any new commits have occurred on 'dspace-xmlui-lang' since the last release. (For v6.x or below only)Note that the version numbering convention for Language Packs is always the same as the current DSpace release, with an additional |
Language pack updates are not back-ported. If you are making a security release for an older branch of DSpace, there will be no language pack commits to release. Continue with Final Commits & Preparation, below. |
For each module, perform the full release steps that follow. To save space, the steps are only listed for one of the modules (but don't forget to run it for both language packs):
git clone git@github.com:DSpace/dspace-api-lang.git dspace-api-lang
cd dspace-api-lang
git checkout main
NOTE: always release language packs from the main branch -- we do not use a maintenance branch for language packs.
mvn release:prepare -DdryRun=truemvn release:prepare -Dresume=false [major].[minor].[sequence-number] (e.g. 5.0.0, 5.0.1, etc for 5.0 releases of language packs)mvn release:performOnce both Language Packs have been released, you can immediately perform the DSpace release. You do not need to wait for them to appear in Maven Central, as our DSpace parent pom.xml will find them in Sonatype's Release Repository immediately.
NOTE: if you're skimming these instructions, you may be tempted to think you've already handled this step, because you have already released new language packs, as detailed above. If you think so, you probably have NOT yet completed the steps below. The steps below tell DSpace what version of language packs to use. The language packs you've released following the steps above won't ever get used if you don't do the steps below. This is an easy thing to miss. Don't. Just check, to be sure. |
Once the Language Packs are released, you will probably need to modify the DSpace root pom.xml (https://github.com/DSpace/DSpace/blob/master/pom.xml) to reference the new version of the Language Packs. This should be similar to the following:
If possible, you'd only want to commit this after the i18n modules are available in the Maven Repository. But, if you are in a rush, you can commit this change earlier (though be warned that this will break the build process for anyone who hasn't manually installed the i18n modules to his/her local ~/.m2/ directory).
In the main pom.xml, provide the proper version range for each language pack. In the below example, we are saying to use any language pack version which is at least version 7.0.0, but is less than version 8.0.0:
<!-- DSpace Localization Packages --> <dependency> <groupId>org.dspace</groupId> <artifactId>dspace-api-lang</artifactId> <version>[7.0.0,8.0.0)</version> </dependency> |
Hopefully, you've already been talking with others about getting Documentation updated! 
You should also double check that the following "main pages" are updated in the Documentation:
History- Make sure the online History for this latest Release is included. This section just links to all the tickets/PRs closed under the release "milestone" in GitHub.
Obviously, this is just a brief reminder of important areas of Documentation which always require updates. There's surely other areas, like Configuration section, which will require some updates for your release. |
If you are performing multiple releases at once (e.g. backporting security or bug fixes), it is IMPORTANT to tag your releases chronologically. For example, the backported fixes to 3.x should be tagged BEFORE 4.x which should be tagged BEFORE 5.x. The reason for this is that the timestamp of the tag determines the ORDERING of the releases in GitHub. So, in order for the 5.x release to appear after the backported releases, it needs to be released LAST. The last tagged release will become the "Latest Release" in GitHub.
Just including a little warning about this up front. The following optional modules need to be specified with every mvn command below. We will make an effort to keep this list up to date, but you should verify it before you cut a new release. Things change. Forgetting an optional module means you'll have to cut another release.
mvn {target} -Drelease
# NOTE: for DSpace 7.x, you MUST use the "-Drelease" flag in all commands. It will automatically release all modules. |
Checkout a fresh copy of the to-be-released version either from a branch or main. For example:
git clone git@github.com:DSpace/DSpace.git dspace-release cd dspace-release git checkout main |
Note: do not just re-use an old working copy of the DSpace Main branch, for obvious reasons, you don't want your own work in progress sneaking into the release. It's also important to use the SSH repository path as noted above (NOT the https URL), otherwise you will be prompted for your GitHub credentials during the release process. More than once. Save yourself some time, be sure to use the SSH path.
Note: if you are doing a maintenance release, you will need to check out the maintenance branch, and not the main branch. In this case, the example above would instead read:
git clone git@github.com:DSpace/DSpace.git dspace-release cd dspace-release git checkout dspace-7_x (or whatever the current maintenance branch might be named) |
This step is not required, but performs a useful sanity check without committing any changes. From your clean, up-to-date copy of master/branch, run the following command (from [dspace-src]):
# For DSpace 7.x or above (the "-Drelease" flag is required and it selects all modules to release) mvn release:prepare -DdryRun=true -Drelease |
You will have to enter in your GPG passphrase (which you established when you created your Code Signing Key).
If GPG is not working, you can "prime" the GPG agent by signing something like this:
|
If you notice an issue or an error occurs, you can re-run the Dry Run using the following command:
You can also clean up any of the release files that the Dry Run created, and just re-run it.
|
This step will set the version declared in the project's pom.xml files, commit the changes to master/branch, tag the release, and finally, check in the master/branch change that increments the next development version (e.g. x.y-SNAPSHOT) in the pom.xml files. Run the following (from [dspace-src]):
# For DSpace 7.x or above (the "-Drelease" flag is required and it selects all modules to release) mvn release:prepare -Dresume=false -Drelease |
(Optionally, you may also include the parameters -Dusername=YourGitHubUsername -Dpassword=YourGitHubPassword at the end of the above command, though I've not found these to be necessary)
The above command will ask you three basic questions. Here are sample answers for DSpace 3.0:
|
You will also have to enter in your GPG passphrase (which you established when you created your Code Signing Key).
As the release process scrolls by, you likely will see a LOT of "WARNING" messages. Don't worry, these should be just Javadocs warnings, and can be safely ignored. Just be patient, and see if it all succeeds in the end. We know it's nerve-wracking, but it will all be OK. |
Assuming everything worked right, you should see ALL the following changes in GitHub:
The results from Maven look similar to this. (Don't worry about the "SKIPPED" messages, those are normal, as the actual release process just runs from the "DSpace Parent Project")
[INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] DSpace Parent Project ............................. SUCCESS [2.320s] [INFO] DSpace Services Framework :: API and Implementation SKIPPED [INFO] DSpace Kernel :: API and Implementation ........... SKIPPED [INFO] DSpace Addon Modules .............................. SKIPPED [INFO] DSpace Kernel :: Additions and Local Customizations SKIPPED [INFO] DSpace XML-UI (Manakin) ........................... SKIPPED [INFO] DSpace XML-UI (Manakin) :: Local Customizations ... SKIPPED [INFO] DSpace LNI ........................................ SKIPPED [INFO] DSpace LNI :: Local Customizations ................ SKIPPED [INFO] DSpace JSP-UI ..................................... SKIPPED [INFO] DSpace JSP-UI :: Local Customizations ............. SKIPPED [INFO] DSpace SWORD ...................................... SKIPPED [INFO] DSpace SWORD :: Local Customizations .............. SKIPPED [INFO] DSpace SWORD v2 ................................... SKIPPED [INFO] DSpace SWORD v2 :: Local Customizations ........... SKIPPED [INFO] DSpace SOLR :: Local Customizations ............... SKIPPED [INFO] DSpace OAI 2.0 .................................... SKIPPED [INFO] DSpace OAI 2.0 :: Local Customizations ............ SKIPPED [INFO] DSpace Assembly and Configuration ................. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ |
The
|
If backing out of this step is needed for any reason, the following will restore the github repository and your working copy to the state it was previously in:
|
This step will sign, checksum, and push all release artifacts (including javadocs and sources) to the Sonatype staging repository (http://oss.sonatype.org/). Run the following (from [dspace-src]):
# For DSpace 7.x or above (the "-Drelease" flag is required and it selects all modules to release) mvn release:perform -Drelease |
You should be prompted by Maven to specify your GPG passphrase (which you established when you created your Code Signing Key). If you run into any issues, it's possible to specify your GPG key and passphrase as arguments to the above command (e.g. -Darguments="-Dgpg.keyname=YourKeyId -Dgpg.passphrase=YourKeyPassword")
In case the upload to sonatype seems to be stalled, be patient. Maven will wait for a timeout and automatically retry the upload. |
If any errors or problems occur during the deploy, you can re-run the same |
If you run into issues, or need to perform the
|
For screenshots and more details on this step, visit the Sonatype Repository Usage Guide's section on Releasing your artifacts |
Ensure that the artifacts in staging are exactly as they should be once deployed to Maven Central. Here's a few things to watch out for...
Download one (or more) of the POMs, and make sure the <version> tag is correct (e.g. 6.0 and not a SNAPSHOT version or similar)
Compare it against a past release in Maven Central (https://search.maven.org/search?q=org.dspace), making sure it has the same JARs or WARs, etc
You can also verify the checksums of one or more of the JARs/WARs in Sonatype versus those that were installed into your .m2 directory. They should be the same.
If anything is incorrect, select the staged repository and select "Drop". After the problem is resolved, you can re-deploy the artifacts to staging and verify them again. To re-deploy an already-tagged release: |
If everything looks good, select the repository and select "Release". The artifacts should be synced to Maven central (https://search.maven.org/search?q=org.dspace) within 2 hours.
Once you select "Release", there is no way to "undo" the release. If any major issues are found, you'll have to increment the version number and perform a new bug-fix release. |
Only required for DSpace 7.x and above. In 6.x and below, the UIs are in the same repository as the backend |
Before running the "yarn version" command, you will need to tell your local "yarn" to NOT create a git tag. We'll tag this release ourselves:
# This only needs to be done once. You can check your settings via "yarn config list" yarn config set version-git-tag false |
First, increment the release in our package.json. Node.js / NPM / Yarn requires that release tags all be valid semantic versioning (https://semver.org/).
So, our "dspace-angular" release numbering looks slightly different than the backend release numbering. It's MAJOR.MINOR.PATCH
Increment the version by running (NOTE: This will immediately apply a git commit to update the "version" in package.json). In the below example, the current version is "7.4.0-next", and we've updated it to be "7.4.0" in preparation for the "dspace-7.4" tagged release.
git checkout dspace-7_x (or main) yarn version ... info Current version: 7.6.1-next question New version: 7.6.1 ... (EXAMPLE for 8.0) info Current version: 8.0.0-next question New version: 8.0.0 |
Commit the change to package.json
git commit -a -m "Update version tag for release" git push upstream dspace-7_x (or main) |
You'll need to run the "version" command a second time to update package.json for our next release. In the below example, the current version is "7.4.0" and we've updated the version to be "7.5.0-next" for the next release.
yarn version ... info Current version: 7.6.1 question New version: 7.6.2-next ... (EXAMPLE for 8.0) info Current version: 8.0.0 question New version: 8.1.0-next |
Commit the change to package.json
git commit -a -m "Update version tag for development of next release" git push upstream dspace-7_x (or main) |
Export the latest Wiki-based Documentation as PDF.
See this DSpace documentation management guide: How To Export Downloadable Docs from Wiki |
This step is now AUTOMATED via our 'docker' GitHub Action. However, you should double check DockerHub to ensure that newly tagged Docker images were auto-created, especially for:
If the tag doesn't appear in DockerHub (it may take a few hours!), then you can check our GitHub Actions for possible failures, and/or build & tag the images manually as described below. |
Images can be built & pushed from command-line to DockerHub. Again, use the same tag name (e.g. "dspace-7.0") as above.
Tim & Kim currently have Push access. Request it from one of them if you don't have it yet.
cd <DSPACE-SRC> --checkout tag-- docker build -t dspace/dspace:<tag> . docker push dspace/dspace:<tag> cd <ANGULAR-SRC> --checkout tag-- docker build -t dspace/dspace-angular:<tag> . docker push dspace/dspace-angular:<tag> |
You must wait for all the packages to be available at https://search.maven.org/search?q=org.dspace before you announce the release. Until the DSpace packages are available in the Maven repository, no one else will be able to build DSpace using Maven. |
For bugfix releases
# This example is to find the list of contributors to 6.3 # It lists contributors to 6.x branch since the 6.2 tag git shortlog -ns dspace-6_x ^dspace-6.2 |
After a new major release (e.g. 8.0), we need to create a maintenance branch for any bug-fix releases. This allows the "main" branch to hold commits for the next major release, while bug fixes get applied to the maintenance branch.
Creating a maintenance branch must be done for the backend and frontend separately.
The easiest way to create a new branch is by using the release:branch command. This command uses the same params as "release:prepare" (see above examples), but will create a new branch instead of a new tag. For example:
# Start from current main (latest code) git checkout main git pull upstream main # The examples below assume that "main" currently has a POM version of "[majorversion].1-SNAPSHOT" # DRY RUN: Create a branch named "dspace-7_x" from main # This will copy the existing POM version tags to the "dspace-7_x" branch. # It will ask you what the next version is, and update to that version in the "main" branch. mvn release:branch -DdryRun=true -Drelease -DbranchName=dspace-7_x # If everything looks good, run it for real. This will immediately create the branch in GitHub, # and then ask you again what new version to update the POMs on "main" to mvn release:branch -Dresume=false -Drelease -DbranchName=dspace-7_x # When selecting the next version for main, make sure it's a SNAPSHOT of the next major version (e.g. 8.0-SNAPSHOT) |
Double check that the POM versions are now correct in both the "main" and maintenance branches. If something is wrong, you should be able to use release:update-versions
git checkout [branch-to-correct] mvn release:update-versions -Drelease |
Now, check the "version" at the top of the package.json file in both branches. Ensure the maintenance branch version looks like "[major-version].1.0-next". Ensure the "main" branch looks like "[next-major-version].0.0-next". If either one is incorrect, then fix it using "yarn version". For example:
git checkout main yarn version ... info Current version: 8.1.0-next question New version: 9.0.0-next |
Then, commit your changes.
git commit -a -m "Update version tag for development of next major release" git push upstream main |
If you encounter one of these errors when building/packaging DSpace:
FATAL ERROR: "Reason: Could not find the model file '../dspace-xmlui-lang'. for project unknown"
OR
FATAL ERROR: "Reason: Could not find the model file '../dspace-api-lang'. for project unknown"
This is a known bug in Maven. The problem is that you likely have a 'dspace-xmlui-lang' or 'dspace-api-lang' folder at the same level as your [dspace-source] parent folder. Essentially, Maven located them and tried to add them into the build process (which it shouldn't have). The fix is to completely delete the "dspace-xmlui-lang" and "dspace-api-lang" folders, and try to rebuild DSpace.
With a straight face assure the next Release Coordinator that "Maven is easy" and there is nothing to be afraid of, then put your feet up and open a beer.