Release Notes

 
Welcome to Release 6.3, a bug-fix release for the DSpace 6.x platform. Any previous version of DSpace may be upgraded to DSpace 6 directly. For more information, please see Upgrading DSpace.

6.3 Release Notes

DSpace 6.3 is a bug fix release to resolve several issues located in previous 6.x releases. As it only provides only bug fixes, DSpace 6.3 should constitute an easy upgrade from DSpace 6.x for most users. No database changes should be necessary when upgrading from DSpace 6.x to 6.3. One configuration addition (orcid.api.url property) has been made to the default dspace.cfg to support the new ORCID API v2, for ORCID Authority Control users.

JSPUI security fixes include

• [HIGH SEVERITY] A user can inject malicious Javascript into the names of EPeople or Groups. This is most severe in sites which allow anyone to register for a new account. (https://jira.duraspace.org/browse/DS-3866 - requires a JIRA account to access.) 

  • Reported by Julio Brafman

• [MEDIUM SEVERITY] Any user was able to export metadata to CSV format if they knew the correct JSPUI path/parameters. Additionally, the exported CSV included metadata fields which are flagged as hidden in configuration. (https://jira.duraspace.org/browse/DS-3840 - requires a JIRA account to access.) 

  • Reported by Eike Kleiner (ZHAW, Zurich University of Applied Sciences)

Major bug fixes include:

• Update DSpace ORCID Integration to use ORCID API v2 (instead of now obsolete ORCID v1): DS-3447
• Update DSpace Statistics to use GeoIP API v2 (instead of now discontinued GeoIP API v1): DS-3832
• Database specific fixes
  • Oracle database migration fix. Configurable Workflow migration threw errors: DS-3788
  • PostgreSQL JDBC driver upgraded to latest version (to allow for full compatibility with PostgreSQL v10): DS-3854
  • Fix issue where DSpace wasn't starting if it used a database connection pool supplied through JNDI: DS-3434
• Bitstream deletion issues ("dspace cleanup" command)
  • Fixed issues where Bitstreams were not being flagged for deletion when an Item was deleted: DS-3729
  • Fixed issues where Bitstreams were not being removed from assetstore even when flagged as deleted: DS-3627 and DS-3461
    • Note: This issue was limited to 6.0, 6.1 or 6.2, and specifically occurred when Item Level Versioning was NOT enabled (which is the default setting) or when Item Level Versioning was first enabled on DSpace version 6.0, 6.1 or 6.2
  • Fixed issues where Bitstreams were removed from all versions of an Item (resulting in inaccurate versioning) when deleted from the latest version of an Item: DS-3627
    • Note: This issue was limited to 6.0, 6.1 or 6.2, and specifically ONLY occurred when Item Level Versioning was first enabled on DSpace version 4.x or 5.x (and that old versioning data had since been migrated to 6.x).
• Other API-level fixes (affecting all UIs)
  
  • Fixed issues where Item Level Versioning accidentally duplicated both metadata (DS-3703) and bitstreams (DS-3702)
  • Fixed issue where Shibboleth authentication plugin appeared to login when no password provided: DS-3662
  • Fixed issues with Solr Search Query escaping in both UIs: DS-3507
  • Update last modified timestamp (on Item) when a new bitstream is added: DS-3734
  • Ensure ImageMagick thumbnails respect the orientation of original file: DS-3839
  • Fix MediaFilter "too many open files" issues (where it forgot to close an input stream): DS-3700
  • Fix PubMed Import submission step (StartSubmissionLookupStep) to use updated URL of PubMed API:  DS-3933
  • Cleanup EHCache configuration: DS-3694 and DS-3710
• JSPUI fixes
  • Fixed issues with authority control popup: DS-3404
  • Fixed issues with pausing HTML5 uploads: DS-3865
• XMLUI fixes
  • Fixed Mirage v2 build issues caused by Bower Registry URL change: DS-3936
  • Fixed performance issues for Items with 100+ bitstreams: DS-3883
  • Fixed occasional Hibernate LazyInitializationException when completing submissions: DS-3775
  • Fixed Unicode character issues in metadata: DS-3733
  • Fix issue where search results lose Community/Collection context when sorting: DS-3835
  • Fixed bitstream download issues which could leave AWS connections open when using S3 storage backend: DS-3870
  • Update Mirage to use recommended MathJax inline delimiters (DS-3087) and to use new CDN location (DS-3560)
• OAI-PMH Fixes
  • Ensure OAI-PMH updates harvestable items when an item is made private (DS-3707) or an embargo expires (DS-3715)
  • Fixed Unicode character issues in metadata: DS-3733 and DS-3556
  • Fix content type of OAI-PMH response: DS-3889
  • Enhanced "oai import" command to report on items that cause indexing issues: DS-3852
• REST API fixes and minor improvements
  • Fixed issue where REST API was no longer able to return JSON responses: DS-3903
  • Fixed update bitstream data returning 500 response: DS-3511
  • Improvements to REST Based Quality Control Reports:
    • Enable login via Shibboleth: DS-3811
    • Add bitstream field data to item listing: DS-3704
    • Fix bug filtering for bitstream permissions: DS-3713
    • Improve ability to find withdrawn items: DS-3714

For more information, see the Changes section in the DuraSpace Wiki.

6.3 Acknowledgments

The 6.3 release was led by Kim Shepherd

The following individuals provided tests, code or bug fixes or review to the 6.3 release: Saiful Amin, Pascal-Nicolas Becker, Ben Bosman, Terry Brady, Per Broman, Jacob Brown, James Creel, Tom Desair, Tim Donohue, Stefan Fritzsche, Hendrik Geßner, Werner Greßhoff, Marsa Haoua, Iris Hausmann, Chris Herron, Lotte Hofstede, Eike Kleiner, Ivan Masár, Dinesh Mendhe, Philip Münch, Sébastien Nadeau, Miika Nurminen, Alan Orth, Hardy Pottinger, Jakub Řihák, J. Savell, Christian Scheible, Kim Shepherd, Ilja Sidoroff, S. Solim, Eduardo Speroni, Alexander Sulfrian, Jonas Van Goolen, Philip Vissenaekens, Martin Walk, Andrew Wood, Mark Wood

6.2 Release Notes

DSpace 6.2 is a bug fix release to resolve several issues located in previous 6.x releases. As it only provides only bug fixes, DSpace 6.2 should constitute an easy upgrade from DSpace 6.x for most users. No database changes or additional configuration changes should be necessary when upgrading from DSpace 6.x to 6.2.
 

Major bug fixes include:

• Important bug fixes to DSpace database connection/caching management
  
  • Indexing: Items failed to be reindexed after metadata changes were made: DS-3660
  • Batch processing: XMLUI Batch Import Failure: DS-3648
  • Group creation: Database migrations sometimes failed to create initial groups (Administrator and Anonymous) in fresh_installs: DS-3659
  • Cache management: DOIOrganiser CLI does not change the database anymore: DS-3656
  • Cache Management: Database changes of consumers aren't persisted anymore: DS-3680
  • Cache Management: Slow batch operations due to Hibernate caching: DS-3286
• General bug fixes (to all UIs):
  
  • Pre-6.x Bitstream statistics were not being displayed: DS-3602
  • ImageMagick PDF thumbnail should always create sRGB JPEG files: DS-3517
  • ImageMagick PDF Processing Degraded with Changes in 5.7 release: DS-3661

6.2 Acknowledgments

The 6.2 release was led by the DSpace Committers.

The following individuals provided code or bug fixes or review to the 6.2 release:  Pascal-Nicolas Becker, Terry Brady, Tom Desair, Tim Donohue, jawadmakki, Steve Michaels, Sébastien Nadeau, Alan Orth, Hardy Pottinger, Adan Roman, Sven Soliman, Alexander Sulfrian, Chris Wilper, Mark Wood.

6.1 Release Notes 

DSpace 6.1 is a security & bug fix release to resolve several issues located in previous 6.x releases. As it only provides bug/security fixes, DSpace 6.1 should constitute an easy upgrade from DSpace 6.x for most users. No database changes or additional configuration changes should be necessary when upgrading from DSpace 6.x to 6.1.
 

Major bug fixes include:

• Security fixes for both JSPUI and XMLUI:

  • [HIGH SEVERITY] Basic (Traditional) Workflow approval process is vulnerable to unauthorized manipulations.(https://jira.duraspace.org/browse/DS-3647 - requires a JIRA account to access.) 
    • Discovered by Pascal Becker (The Library Code / TU Berlin).
  • [LOW SEVERITY] DSpace failed to check if policies had valid dates when checking access permissions.(https://jira.duraspace.org/browse/DS-3619 - requires a JIRA account to access.) 
    • Discovered by Pascal Becker (The Library Code / TU Berlin).
• Security fixes for REST API:
  • [HIGH SEVERITY] A user with submit permissions can bypass workflow approvals by depositing via REST API.(https://jira.duraspace.org/browse/DS-3281 - requires a JIRA account to access.) 
    • Discovered by Emilio Lorenzo.
  • [LOW SEVERITY] The "find-by-metadata" path publicly exposes metadata from access-restricted items.(https://jira.duraspace.org/browse/DS-3628 - requires a JIRA account to access.) 
    • Reported by Bram Luyten (Atmire).
• General bug fixes (to all UIs):
  • Performance improvements at API layer: DS-3558, DS-3552
  • Submitters (who are not Admins) could not remove bitstreams from their in progress submission: DS-3446
  • Full text searching was only possible in the first bitstream (file): DS-2952
  • Configurable Workflow was throwing "Authorization is Denied" errors: DS-3367
  • IP Authorization range restrictions were not working properly: DS-3463
  • Item Versioning was not saving properly: DS-3381
  • Improve the text of database migration errors: DS-3571
  • Improve cache management for command line processes:  DS-3579
  • Resolve CSV line break issue in bulk edit: DS-3245
  • Resolve error with null referrer to feedback page: DS-3601
  • Support all UTF-8 characters in configuration files: DS-3568
  • Fix update-handle-prefix script to no longer update handle suffix: DS-3632
• XMLUI bug fixes:
  • /handleresolver path was no longer working: DS-3366
  • Display a restricted image thumbnail for access restricted bitstreams: DS-2789
  • Fix broken images when running Mirage 2 on Jetty: DS-3289
  • Archived submissions were being displayed chronologically instead of reverse chronologically: DS-3334
  • On Move Item page, the list of Collections was sorted by Collection name, instead of being first grouped by Community: DS-3336
  • ORCID / Authority Lookup button was no longer working in Mirage 2: DS-3387
  • Improve error message when user attempts to update an e-mail address to an existing address: DS-3584
  • Allow localization of input-forms.xml with XMLUI: DS-3598
  • Fix error when uploading large files (>2GB) via a web browser: DS-2359
  • Various other minor bug fixes
• JSPUI bug fixes
  • READ access rights not being respected on Collection homepage: DS-3441
  • Fix issue where database connections were being kept open on some JSPUI pages: DS-3582
• Oracle support bug fixes:
  • Oracle migrations took forever because of missing indexes: DS-3378
  • Community and Collection handles were not properly migrated between 5.x and 6.x: DS-3409
• OAI-PMH bug fixes:
  • DIM crosswalks repeated authority information: DS-2947
• REST API bug fixes:
  • Support for Shibboleth added: DS-3108
• Solr Statistics fixes:
  • Item Statistics displayed UUID instead of file name: DS-3164
  • Sharding statistics corrupted some fields and was unstable: DS-3436, DS-3457, DS-3458
• AIP Backup and Restore fixes:
  • Failed AIP imports left files in assetstore: DS-2227
  • Could not restore items from AIP if embargo lift date was in the past: hDS-3348
  • Replication Task Suite plugin was not working with 6.0: DS-3389

Minor improvements include:

• SEO improvement: Add configurable support for whitelisting specific file formats for Google Scholar citation_pdf_url tag: DS-3127
• Add support for *.docx files (newer MS Word) to indexing process (via media filters). See DS-1140
• Add ability to multi-select options in XMLUI's My Submission page. See DS-3448
• Filter labels were missing in XMLUI's search screen. See DS-3573
• Minor improvements to logging and error reporting.

In addition, this release fixes a variety of minor bugs in the 6.x releases. For more information, see the Changes in 6.x section.

6.1 Acknowledgments 

The 6.1 release was led by the DSpace Committers.

The following individuals provided code or bug fixes to the 6.1 release: Pascal-Nicolas Becker (pnbecker), Andrew Bennet (AndrewBennet), Andrea Bollini (abollini), Terry Brady (terrywbrady), Per Broman (pbroman), Samuel Cambien (samuelcambien),  Yana De Pauw, Tom Desair (tomdesair), Peter Dietz (peterdietz), Roeland Dillen, Tim Donohue (tdonohue), edusperoni, Frederic-Atmire, Generalelektrix, Claudia Juergen (cjuergen), Bram Luyten (bram-atmire),  Enrique Martínez Zúñiga ( enrique),  Ivan Masar (helix84),  Miika Nurminen ( minurmin), Alan Orth (alanorth), Andrea Pascarelli (lap82), Hardy Pottinger (hardyoyo),  Toni Prieto ( toniprieto).  Christian Scheible (christian-scheible), Andrea Schweer (aschweer),  Kim Shepherd (kshepherd),  Alexander Sulfrian ( AlexanderS), Jonas Van Goolen (jonas-atmire), Philip Vissenaekens (PhilipVis), and Mark Wood (mwoodiupui).

6.0 Release Notes

The following is a list of the new features included for the 6.x platform (not an exhaustive list):

DSpace 6.0 ships with a number of new features. Certain features are automatically enabled by default while others require deliberate activation.  The following non-exhaustive list contains the major new features in 6.0
	Major Java API refactor, supporting UUIDs and Hibernate. The DSpace Java API has been refactored significantly to make it more modular, and make it easier to achieve future RoadMap modularity goals. For more detailed information, see DSpace Service based api or DS-2701. This feature was contributed by Kevin Van de Velde of @mire, with support/help from the DSpace Committers. Hibernate allows us more stability in our database layer (decreases the likelihood of database specific bugs), and potentially allows us to support additional database platforms in the future UUIDs provide all objects with stable, globally unique identifiers (instead of existing incremental, non-unique database identifiers). This simplifies the management of identifiers in our object model. See also DS-1782. The Java API itself is now split into three "layers" which are all now initialized via Spring Service layer, which contains DSpace business logic Database Access Object (DAO) layer, which contains specific Hibernate database queries to locate/find individual objects Database Object layer, which provides access to specific database columns, object properties via Hibernate Most sites will not notice this major API refactor, as the upgrade is seamless. However, if you've performed major (Java-level) customizations, you may need to refactor your own customizations to use this newly refactored API. Some examples are on the DSpace Service based api page.
	Enhanced (Reloadable) Configuration System. See Configuration Reference, Enhanced Configuration Scheme and DS-2654. This feature was contributed by Tim Donohue of DuraSpace. Provides easier management of local configurations via a new local.cfg file. Any configurations (from *.cfg files) can be overridden in DSpace by simply copying them into your local.cfg and changing the value. See Configuration Reference Many configurations now automatically reload as soon as you save your local.cfg file. This means you don't need to restart Tomcat every time you need to change a configuration. Please note: Unfortunately, at this time, some configurations do still get cached in the XMLUI or JSPUI (or similar). So, while many will reload, not all of them yet support this feature.
	Enhanced file (bitstream) storage plugins, including support for Amazon S3 file storage. This feature was contributed by Peter Dietz of Longsight. Configuring the Bitstream Store (also see https://github.com/DSpace/DSpace/pull/1159 and DS-1117)
	Configurable site healthcheck (i.e. repository status) reports via email. This feature was contributed by Jozef Misutka of Lindat. Healthcheck reports (also see DS-2659).
	XMLUI new features New framework for metadata import from external sources (including an out-of-the-box plugin supporting PubMed, and hopefully more coming soon). This concept was previously only supported in JSPUI. Screencast example: http://screencast.com/t/QBImSGbAUZ.  These features were contributed by @mire. The PubMed metadata import plugin was also contributed by the Netherlands Cancer Institute. Framework to support metadata import from external sources: DS-2876 PubMed metadata import plugin: DS-2880 Extensible administrative control panel, allowing institutions to override or customize the tabs in the XMLUI administrative "control panel" (see also DS-2625). This feature was contributed by Jozef Misutka of Lindat. Export of XMLUI search results to CSV for batch metadata editing. This feature was previously only supported in JSPUI. See also DS-1262. This feature was contributed by William Welling of Texas A&M
	REST API new features REST Based Quality Control Reports for collection manager. See also DS-2583. This feature was contributed by Terry Brady of Georgetown University See Metadata Query Report on demo.dspace.org See Collection Filter Report on demo.dspace.org Support for all authentication methods via REST (e.g. Shibboleth, LDAP, etc). See also DS-2898. New /registries endpoint which provides access to the metadata registries via REST. See also DS-2539. This feature was contributed by Terry Brady of Georgetown University. REST API now provides a version via /status endpoint (inherits the version of DSpace API). See DS-2619. This feature was contributed by Ivan Masár.
	Search/Discovery enhancements All searches now default to boolean AND. See DS-2809. This enhancement was contributed by Andrea Schweer. New "Has File" facet, which allows you to easily filter by items that have one or more files. See DS-2648. This enhancement was contributed by Christian Scheible. Full text indexing of Excel spreadsheets, so that they are now searchable. See DS-2629. This enhancement was contributed by Ed Goulet. Right-to-left text in PDFs is now indexed for searching. See DS-1187. This enhancement was contributed by Saiful Amin
	Other enhancements: PDFBox was upgraded to version 2.0 (DS-3035). A new PDFBox Thumbnail generator was also added and enabled by default (DS-3052). These features were contributed by Ivan Masár. OAI-PMH was upgraded for compliance with the OpenAIRE 3.0 guidelines for literature repositories. This enhancement was contributed by Pedro Príncipe.
	Features Removed or Replaced in 6.0 The build.properties configuration file has been replaced by an enhanced local.cfg configuration file. The new local.cfg allows you to easily override any configuration (from dspace.cfg or modules/*.cfg files) by simply copying it into your local.cfg and specifying a new value. It also provides enhanced configuration options as detailed in the Configuration Reference documentation. The old build.properties file is no longer used nor supported. WARNING: As part of adding this new configuration scheme, many of the configuration settings in DSpace (primarily those in modules/*.cfg files) had to be renamed or prepended with the name of the module. This means that 5.x (or below) configurations are no longer guaranteed to be compatible with 6.x. If possible, we recommend starting with fresh configs (see below), and moving all your locally customized settings into the new local.cfg file. The PDF Citation Cover Page configuration file has been renamed (from disseminate-citation.cfg to citation-page.cfg). See this feature's documentation for more details. The legacy search engine (based on Apache Lucene) and legacy Browse system (based on database tables) have been removed from DSpace 6.0 or above. Instead, DSpace now only uses Discovery (based on Apache Solr) for all Search/Browse capabilities. See DS-2160. The legacy browse system was removed by Kevin Van de Velde. The legacy search system was removed by Tim Donohue. The DSpace Lightweight Networking Interface (LNI), supporting WebDAV / SOAP / RPC API, has been removed from DSpace 6.0 or above. We recommend using REST or SWORD (v1 or v2) as a replacement. However, if you still require it, the old (unmaintained) LNI codebase is still available at https://github.com/DSpace/dspace-lni. This change was contributed by Robin Taylor of University of Edinburgh. Support for SRB (Storage Resource Broker) file storage has been removed from DSpace 6.0 or above.  As it was unmaintained (and seemingly unused) for many years, this feature was removed along with its configurations. As a replacement, a new file storage plugin system was added, featuring a traditional local file storage option (default) and an Amazon S3 file storage option (see Storage Layer documentation, especially Configuring the Bitstream Store). For more information on the removal of SRB support, also see DS-3055. This change was contributed by Peter Dietz of Longsight. The user groups Administrator and Anonymous cannot be renamed or deleted.  If you had renamed them, they will be renamed back to the stock names during the upgrade. DSpace is now dependent on these specific names due to internal changes. This change was contributed by Mark Wood of IUPUI. XPDF PDF Thumbnail generation has been removed. Please use the ImageMagick or PDFBox thumbnail generators instead. See DS-2159. This change was contributed by Mark Wood of IUPUI. The default strategy to create new handles for versioned Items has changed. If you have Item Level Versioning enabled and you have versioned Items in your DSpace installation, you may want to change the configuration to continue using the mechanism to create handles as it was in DSpace 4 and 5. You can find more informations here: Item Level Versioning#IdentifierServiceOverride. Elasticsearch Usage Statistics feature is deprecated in the 6.0 release. While they still function, and you can continue to use them, in a future DSpace release they will be removed. If you are looking for a usage statistics option, we recommend instead using the default SOLR Statistics engine and/or DSpace's integration with Google Analytics. See DS-2897 for more information.

A full list of all changes / bug fixes in 6.x is available in the Changes in 6.x section.

6.0 Acknowledgments

The following individuals have contributed directly to this release of DSpace:  Tim Donohue, Mark H. Wood, Pascal-Nicolas Becker, Kevin Van de Velde, Ivan Masár, Hardy Pottinger, Terry Brady, Andrea Schweer, Philip Vissenaekens, Peter Dietz, Jonas Van Goolen, Tom Desair, Dylan MEEUS, Luigi Andrea Pascarelli, William Welling, Christian Scheible, Andrea Bollini, Aleksander Kotyński-Buryła, Ondřej Košarko, Jozef Mišutka, Chris Wilper, Ilja Sidoroff, Roeland Dillen, Bram Luyten, Marsa Haoua, Claudia Jürgen, Kim Shepherd, Art Lowel, Ivo Prajer, Petr Karel, Mini Pillai, Facundo Gabriel Adorno, Luiz Claudio Santos, Robin Taylor, Tim Van den Langenbergh, Arnaud de Bossoreille, Bill Tantzen, Tiago Guimarães, Oriol Olivé Comadira, Àlex Magaz Graça, Anne Lawrence, Brad Dewar, Bruno Nocera Zanette, David Baker, Ed Goulet, Mateusz Neumann, Monika Mevenkamp, Pablo Buenaposada, Patricio Marrone, Petya Kohts, Eike Kleiner, Antoine Snyers, Bjorn Jaspers, Chris Herron, Dan Scott, David Cook, Davor Cubranic, José Carvalho, Jozsef Marton, Juan Manuel Catá, Panagiotis Koutsourakis, Pantelis Karamolegkos, Pedro Príncipe, Philippe Gray, Rodrigo Prado de Jesus, RomanticCat, Saiful Amin, junwei1229, Keith Gilbertson, Nicolas Schwab, Pablo Buenaposada, Michael Marttila, samuel, tmtvl , and others who reviewed and commented on their work.  Many of these could not do this work without the support (release time and financial) of their associated institutions. We offer thanks to those institutions for supporting their staff to take time to contribute to the DSpace project.

A big thank you also goes out to the DSpace Community Advisory Team (DCAT), who helped the developers to prioritize and plan out several of the new features that made it into this release. The current DCAT members include: Augustine Gitonga, Bram Luyten, Bharat Chaudhari, Claire Bundy, Dibyendra Hyoju, Elin Stangeland, Felicity A Dykas, Iryna Kuchma, James Evans, Jim Ottaviani, Kate Dohe, Kathleen Schweitzberger, Leonie Hayes, Lilly Li, Maureen Walsh, Pauline Ward, Roger Weaver, Sarah Molloy, Sarah Potvin, Steve Van Tuyl, Terry Brady, Yan Han and Valorie Hollister.

We apologize to any contributor accidentally left off this list. DSpace has such a large, active development community that we sometimes lose track of all our contributors. Our ongoing list of all known people/institutions that have contributed to DSpace software can be found on our DSpace Contributors page. Acknowledgments to those left off will be made in future releases.

Want to see your name appear in our list of contributors? All you have to do is report an issue, fix a bug, improve our documentation or help us determine the necessary requirements for a new feature! Visit our Issue Tracker to report a bug, or join dspace-devel mailing list to take part in development work. If you'd like to help improve our current documentation, please get in touch with one of our Committers with your ideas. You don't even need to be a developer! Repository managers can also get involved by volunteering to join the DSpace Community Advisory Team and helping our developers to plan new features.

The Release Team consisted of:

• Tim Donohue (DuraSpace)
• Kevin Van de Velde (@mire)
  
• Pascal-Nicolas Becker (Technische Universität Berlin)

Additional thanks to Tim Donohue from DuraSpace for keeping all of us focused on the work at hand, for calming us when we got excited, and for the general support for the DSpace project.