Versions Compared

Old Version 62

changes.mady.by.user Ivan Masár

Saved on

compared with

New Version Current

changes.mady.by.user Tim Donohue

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Version 5.0

Infowarning

Support for DSpace 5 .0 is still in the planning stages and will be released sometime in late (likely December) 2014

 

Table of Contents:

Table of Contents
maxLevel3
minLevel2
outlinetrue
stylenone

 

New features in DSpace 5.0 (WIP)

ended on January 1, 2023.  See Support for DSpace 5 and 6 is ending in 2023


Tip
titleDSpace 5.0 was officially released to the public on January 16, 2015.

DSpace 5.0 can be downloaded immediately from:

See the DSpace 5.x Release Notes for more information.


Info
titleWhere are the Release Notes?

DSpace 5.x Release Notes have been incorporated into the DSpace 5.x documentation wiki. The content of that page, and this one, is similar, but it's not a perfect duplicate. This page is for coordinating and planning DSpace Release 5.0, and for communicating this status information to the community. This Release Status page is a historical document, with much of the same material.


Table of Contents:

Table of Contents
maxLevel3
minLevel2
outlinetrue
stylenone


New features in DSpace 5.0

XMLUI new features

  • Excerpt


    DSpace 5.0 ships with a number of new features. Certain features are automatically enabled by default while others require deliberate activation. 
    The following non-exhaustive list contains the major new features in 5.0
    Image Added

    Easier Upgrading to 5.x from ANY previous DSpace version (1.x.x, 3.x or 4.x).

    • Your underlying DSpace database now upgrades itself automatically when you first run a newer version of DSpace (see DS-2167, by Tim Donohue with support/feedback from 5.0 Release Team). 
    • Solr/Lucene indexes now upgrade automatically during the "ant update" step of the Upgrade process (see DS-2297 by Tim Donohue, with support/feedback from Ivan Masár and 5.0 Release Team).


    Image Added

    Perform Batch Imports from the User Interface (in both XMLUI and JSPUI)

    Image Added

    Image Added

    Image Added

    Image Added

    Image Added

    XMLUI new features

    Excerpt
    DSpace 5.0 ships with a number of new features. Certain features are automatically enabled by default while others require deliberate activation. 
    The following non-exhaustive list contains the major new features in 5.0 that are enabled by default:
    Image Removed

    Easier Upgrading to 5.x from any previous DSpace version. Your underlying DSpace database now upgrades itself automatically.

    Jira
    serverDuraSpace JIRA
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    keyDS-2167
    by Tim Donohue

     

    Image Removed

    XMLUI and JSPUI new features

    Jira
    serverDuraSpace JIRA
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    keyDS-1641
    by Peter Dietz with the support of Ohio State University Libraries and Longsight

    Jira
    serverDuraSpace JIRA
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    keyDS-2177
    by the Greek National Documentation Centre/EKT

    Image Removed

    JSPUI new features

    • Jira
      serverDuraSpace JIRA
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
      keyDS-1994
      by Pascal-Nicolas Becker with the support of TU Berlin
    • Jira
      serverDuraSpace JIRA
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
      keyDS-2162
      by Greek National Documentation Centre/EKT

    Image Removed

    Image Removed

    Jira
    serverDuraSpace JIRA
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    keyDS-2053
    by Kevin Van de Velde and Bert Vanderhallen with the support of @mire (previously available only in JSPUI)
    Jira
    serverDuraSpace JIRA
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    keyDS-635
    by Peter Dietz
  • Jira
    serverDuraSpace JIRA
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    keyDS-1596
    by Tim Donohue with the support of DuraSpace
    • Jira
    serverDuraSpace JIRA
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    keyDS-682

     

    Image Removed

    Image Removed

    New CRUD endpoints for REST web service API

    Jira
    serverDuraSpace JIRA
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    keyDS-2168
    by the Czech Technical University in Prague

    Image Removed
    Jira
    serverDuraSpace JIRA
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    keyDS-2061
    by     Pascal-Nicolas BeckerImage Removed

    BTE: batch import from various bibliographic formats

    Jira
    serverDuraSpace JIRA
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    keyDS-2183
    Update to Biblio-Transformation-Engine 0.9.3.5 (changelog)

    Jira
    serverDuraSpace JIRA
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    keyDS-1933

    Kindly contributed by the Greek National Documentation Centre/EKT

    Image Removed


    Image Added

    JSPUI new features

    Image Added

    Image Added

    REST API new features

    Image Added

    RDF Interface to support Linked (Open) Data (NEW)

    • DSpace can now provide its content as Linked (Open) Data via a new RDF interface (provided as an "rdf" webapp), see DS-2061 by Pascal -Nicolas Becker
    Image Added

    OAI-PMH interface enhancements / bug fixes

    • OpenAIRE v3 compliance (operators

    Jira
    serverDuraSpace JIRA
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    keyDS-1649
    by João Melo 

    OpenAIRE v3 compliance (operators
    • over filters)
    • OAI respects item READ rights
    • /oai displays the list of available contexts; contexts have descriptions

    Image Removed

    Solr libraries were upgraded to version 4.10.2 (JSPUI, XMLUI and OAI modules)

    Jira
    serverDuraSpace JIRA
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    keyDS-2253
    by Roeland Dillen with the support of @mire

      The following list contains features that are included in the DSpace 5.0 release, but need to be enabled manually.
    Review the documentation for these features carefully, especially if you are upgrading from an older version of DSpace.

    Image Removed

    Image Removed

    Image Removed

    XMLUI new features

    • Jira
      serverDuraSpace JIRA
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
      keyDS-2052
      by @mire
    • Jira
      serverDuraSpace JIRA
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
      keyDS-2049
      by @mire
    • Jira
      serverDuraSpace JIRA
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
      keyDS-2175
      by Peter Dietz with the support of Ohio State University Libraries and Longsight 
    • Jira
      serverDuraSpace JIRA
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
      keyDS-2108
      by Robin Taylor
    • Jira
      serverDuraSpace JIRA
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
      keyDS-2088
      by Robin Taylor
    Image RemovedDOI Support

    See DS-1649 by João Melo 

    Image Added

    Enhanced Thumbnail Quality (disabled by default)

    See DS-2105 by Terry Brady with the support of Georgetown University

    Image Added

    Bug fixes / improvements to Biblio-Transformation-Engine (BTE)

    Kindly contributed by the Greek National Documentation Centre/EKT

    Image Added

    Enhancements to DOI Support (disabled by default)

    • Enhanced EZID IdentifierProvider Metadata Mapping via XSLT, see DS-2119 by Mohamed Mohideen Abdul Rasheed


    Image Added

    Apache Solr libraries were upgraded for all interfaces (JSPUI, XMLUI, and OAI)

    See DS-2253 by Roeland Dillen with the support of @mire

    Image Added

    Add a place for third-party JARs / plugins to be "found" by DSpace (disabled by default)

    • DSpace will now look for JARs / plugins in the locations specified by "plugin.classpath" value specified in dspace.cfg.

    See DS-2107 by Mark H. Wood with the support of  IUPUI University Library

    Image Added

    All objects now have metadata support

    • All DSpace objects (Communities, Collections, Items, EPeople, Groups) now have metadata, and most now use the default "dc" (Dublin Core) metadata schema.
      • NOTE: The only exception is EPeople metadata, which is stored in a new "eperson" metadata schema.
    • The User Interfaces don't yet take advantage of this enhancement in DSpace 5.0. Instead, this is an internal restructuring of data within DSpace. In the future, this provides the potential to create more enhanced metadata (or even more configurable metadata) on all objects

    See DS-1582 by Mark H. Wood with the support of  IUPUI University Library and Kevin Van de Velde with the support of @mire


    Release TODO list

    •  Check documentation for all new features, add links to them the list of new features above. Incomplete list of missing docs:
      • Jira
        serverDuraSpace JIRA
        serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
        keyDS-

    ...

      • 1641
      • Jira
        serverDuraSpace JIRA
        serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
        keyDS-

    ...

      • 2168
      • Jira
        serverDuraSpace JIRA
        serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
        keyDS-

    ...

      • 2175
      • Jira
        server
      • DuraSpace JIRA
        serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
        keyDS-

    ...

    • All DSpaceObject-inheriting classes now have metadata. The UIs don't yet take advantage of them in this release. But, you will notice a new "eperson" metadata schema for EPeople metadata. All other DSpaceObjects now use the default "dc" metadata schema to store object information.

     

    Release TODO list

    ...

    1. Seems more difficult than anticipated. There is no longer any way to easily get the Index version, as "getCurrentVersion()" has been removed (see below).
      1. updated for current version below

    ...

      • 2183
    •  Current list of release blockers: 
      Jira
      serverDuraSpace JIRA
      columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
      maximumIssues20
      jqlQueryproject = DS AND priority = Blocker AND fixVersion = 5.0 AND statusCategory != Complete
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    •  
    • Jira
      serverDuraSpace JIRA
      columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
      serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
      keyDS-2249

    ...

    •  Seems like we have 2 options:
      1. put Tomcat-specific code to JSPUI's web.xml
      2. just document

    ...

      1. this Tomcat 7-

    ...

      1. specific warning
        On DevMtg 2014-11-05 we decided to document for RC1 and decide before RC2
    •  Update the list of contributors on the Release Notes in Documentation (see "TBA" in text)
    •  Draft up announcement for mailing lists
    •  Update DSpace.org

    Automatic Solr upgrade

    ...

    Note

    This has now been done in

    Jira
    serverDuraSpace JIRA
    serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5
    keyDS-2297
    . This section is kept for archival purposes only.


    Findings

    Solr is distributed on Maven Central as a

    Findings

    Solr is distributed on Maven Central as a .war file, which includes the Lucene .jars. No Lucene version relevant to DSpace is available from Maven Central.

    ...

    DSpace 1.6 was the first version to use Solr for statistics. In order to upgrade to DSpace 5, we need to upgrade the Solr index format in two steps. The second step is easy, upgrading from DSpace 3 (Solr 3.5) to DSpace 5 means just running the Solr 4.10.4 2 optimize command, which can be run even concurrently when the UIs are running. However, Solr 4.4 doesn't understand the legacy index formats used in Solr before 3.5. Therefore, in the first step, we need to use Solr 3.5 or Lucene 3.5 to upgrade from the older index versions to 3.5.

    ...

    We need to avoid the conflict of having both Solr 3.5 and 4.10.4 2 in the DSpace classpath. Therefore I suggest to run the first step before the Solr 4.4 10.2 webapp is started as part of DSpace 5.

    To do that, we need to:

    1. detect the index Detect the oldest segment version using getCurrentVersion(index_dir)NOTE: Looks like the 4.4.0 version of IndexReader no longer contains this getCurrentVersion() function. It was deprecated in 3.6.0 and removed thereafter
    2. Here's a new getVersion() method
      3. In case it turns out to be useful, here's how CheckIndex determines the versions on a per-segment basis: http://grepcode.com/file/repo1.maven.org/maven2/org.apache. SegmentInfo.getVersion() of Lucene 4.10
      1. General logic is in this area of the CheckIndex script: https://github.com/apache/lucene-solr/blob/lucene_solr_4_10/lucene/core/src/java/org/apache/lucene/index/CheckIndex.java#L426
    3. If that oldest segment version is < 3.5 go to 3. Otherwise go to 5.
    4. get lucene-core-3.5.0.jar from Maven Central
    5. run the IndexUpgrader class of Lucene 3.5 (or  optimize())
    6. start up DSpace in order to start up Solr 4
    7. run the  IndexUpgrader  class of Lucene 4.10.2


    The manual steps would be:

    Code Block
    wget "http://search.maven.org/remotecontent?filepath=org/apache/lucene/lucene-core/

    ...

    3.

    ...

    5.0

    ...

    /lucene

    ...

    -core-3.5.0.jar" -O lucene-core-3.5.0.jar

    ...

    The manual steps would be:

    Code Block
    wget "http://search.maven.org/remotecontent?filepath=org/apache/lucene/lucene-core/3.5.0/lucene-core-3.5.0.jar" -O lucene-core-3.5.0.jar
# check index version, see table below:
# check index version, see table below:
java -cp lucene-core-3.5.0.jar org.apache.lucene.index.CheckIndex /dspace/solr/statistics/data/index/
java -cp lucene-core-3.5.0.jar org.apache.lucene.index.CheckIndex  /dspace/solr/statistics/data/index/
java -cp lucene-core-3.5.0.jar org.apache.lucene.index.CheckIndex  /dspace/solr/search/data/index/
# upgrade index version:
java -cp lucene-core-3.5.0.jar org.apache.lucene.index.IndexUpgrader /dspace/solr/statistics/data/index/
java -cp lucene-core-3.5.0.jar org.apache.lucene.index.IndexUpgrader /dspace/solr/search/data/index/
# check index version again, should be "version=3.5 format=FORMAT_3_1 [Lucene 3.1+]"

     

    DSpace/Solr/Lucene versions and compatibility

    ...

    (before optimize)

    versions=[4.4.0 .. 4.10.2] format= 

    ...

    LUCENE_35

    ...

     index version again, should be "version=3.5 format=FORMAT_3_1 [Lucene

    ...

     3.1+]"


    DSpace/Solr/Lucene versions and compatibility

    DSpaceSolrLuceneUsed Lucene index versionversion reported by CheckIndexSupported Lucene index versions
    5.04.10.14.10.24.10

    (before optimize)

    versions=[4.4.0 .. 4.10.2] format= 


    4.04.4.04.44.4version=4.4.0 format=
    3.03.5.03.5

    LUCENE_35

    		version=3.5 format=FORMAT_3_1 [Lucene 3.1+]LUCENE_20, LUCENE_21, LUCENE_22, LUCENE_23, LUCENE_24, LUCENE_29, LUCENE_30, LUCENE_31, LUCENE_32, LUCENE_33, LUCENE_34, LUCENE_35
    1.83.3.03.3???version=3.3 format=FORMAT_3_1 [Lucene 3.1+]
    1.71.4.12.9.3 (2.9.3 951790 - 2010-06-06 01:30:55)???version=FORMAT_DIAGNOSTICS
    [Lucene 2.9]
    1.61.3.02.4-dev (2.4-dev 691741 - 2008-09-03 15:25:16)???  

    Luke

    https://code.google.com/p/luke/wiki/Compatibility

    Java version

    Minimum version of JVM required to run Luke 1.0.1 and earlier is 1.5.

    Minimum version of JVM required to run Luke 3.x and 4.x is 1.6.

    Lucene Java indexes

    Luke 1.0.1 should be able to open indexes built using the following versions of Lucene-Java:

    • 3.0.x
    • 2.x (any version)

    Starting with version 4.x Luke cannot open indexes created with Lucene 3.0.0 or earlier.

    Starting with version 3.1.0 Luke releases will use the same numbering as Lucene releases, to avoid confusion.

    Luke

    https://code.google.com/p/luke/wiki/Compatibility

    Java version

    Minimum version of JVM required to run Luke 1.0.1 and earlier is 1.5.

    Minimum version of JVM required to run Luke 3.x and 4.x is 1.6.

    Lucene Java indexes

    Luke 1.0.1 should be able to open indexes built using the following versions of Lucene-Java:

    • 3.0.x
    • 2.x (any version)

    Starting with version 4.x Luke cannot open indexes created with Lucene 3.0.0 or earlier.

    Starting with version 3.1.0 Luke releases will use the same numbering as Lucene releases, to avoid confusion.

    Possible JIRA tickets to resolve (automated list)

    This is the list of JIRA tickets ("New Features" and Improvements) currently scheduled for 5.0.  There is no guarantee they will all be completed in time for 5.0 release, and tickets may be dropped/added to the list at any time. But, if you'd like to volunteer to help with one, please let us know by attaching a comment to the ticket in question.

    JiraserverDuraSpace JIRAcolumnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolutionmaximumIssues20jqlQueryproject = DS AND issuetype in ("New Feature", Improvement) AND resolution = Unresolved AND fixVersion = "5.0" ORDER BY key ASC serverIdc815ca92-fd23-34c2-8fe3-956808caf8c5

    Changes

    The following changes have already been made to the codebase and will be released in 5.0. Please note that the below listing is dynamically generated, so it will be changing as we continue to add new features, improvements and bug fixes to the 5.0 release.

    ...

    • Peter Dietz (Longsight)
    • Hardy Pottinger (U of Missouri)
    • Ivan Masár
    • Mark H. Wood (Indiana University)
    • Robin Taylor (University of Edinburgh)
    • Pascal-Nicolas Becker (Technische Universität Berlin)
    • Andrea Schweer (Library Consortium of New Zealand)

    All Release Team membership information is as of November 4, 2014. Please volunteer (by emailing Tim Donohue), if you are interested in joining the team!

    ...

    Your contributions are welcome now! Code and documentation need not be finished, so long as it is working and we can all see what it is for. Time is set aside for fixing, polishing, and integration. We have some general Code Contribution Guidelines available, but you are also welcome to ask questions on the dspace-devel mailing list.

    DateMilestoneWhat it means
    October 06Deadline for feature pull requestsIf you wish to contribute features to DSpace 5.0, you must submit a pull request by this date.
    October 08Weekly developers' meeting devoted to review of feature pull requests for 5.0The entire hour's meeting will be used to discuss proposed features submitted by the deadline.
    October 15Weekly developers' meeting devoted to review of feature pull requests for 5.0The entire hour's meeting will be used to discuss proposed features submitted by the deadline.
    October 31Feature freezeDSpace 5.0 is considered feature-complete on this date.  Only bugfixes will be pulled between this date and final release.
    November 6Release Candidate 1 taggedA DSpace 5.0 Release Candidate will be available for wider testing.
    November 10Release Candidate 2 tagged5.0-rc2 was released to address missing optional artifacts from 5.0-rc1 (Mirage2 and LNI)
    November 10-21TestathonIntensive public testing of the 5.0 Release Candidate is invited.  The Release Team will focus on getting problems resolved.
    December
    1
    19Release Candidate 3 taggedAn updated DSpace 5.0 Release Candidate will be available for wider testing.
    Mid December
    January 15DSpace 5.0 is publicly releasedDSpace 5.0 is released for download and general use.

    ...


    Release Process needs to proceed according to the following Maven release process: Release Procedure