This documentation space is deprecated. Please make all updates to DuraCloud documentation on the live DuraCloud documentation space.

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 52 Next »

Release 3.0.0

Released: April 22, 2014

The DuraCloud 3.0.0 release is the first step towards allowing DuraCloud to be run in a more distributed and efficient manner. Towards that end, the services framework has been removed in favor of handling the same tasks (primarily duplication between providers and bit integrity checking) through a scalable processing engine which is run outside of the DuraCloud instances themselves. This provides for better performance on the instances and better performance for the services, as well as less effort for DuraCloud users.

The primary features of release 3.0.0 are:

  • Users of DuraCloud no longer need to manage services. The service framework has been replaced by a back-end task processing system that leverages the massively parallel processing capabilities of the cloud.
  • Library updates, to keep the primary components of DuraCloud software up-to-date. Examples: Upgraded from Maven 2 build system to Maven 3. Upgraded the Spring Framework version from 2.5 to 4.0.2.
  • The auditing framework's use of ActiveMQ have been removed in favor of a more pluggable architecture using an external queuing system. Audit event logging is now being handled by the same task processing framework that is managing duplication and bit integrity checks.
  • Media streaming capabilities updated to continue to work just as before, but without the need for the Media Streamer service.

Release 2.4.0

Released: September 20, 2013

The primary features of release 2.4.0 are:

  • New drag and drop capability for file upload
    • DuraCloud now provides the option to upload files using the familiar file selector or with simple drag and drop.
      • This new functionality is enabled using HTML5, which removes the need for a java browser plugin.
  • New installers for the Sync Tool
    • The Sync Tool can now be installed using a platform-specific installer on Windows, Mac, and Linux. 
    • The installers provide a simple graphical installation flow and are consistent with the usual installation process on each operating system.
    • The installers provide the option to start up the Sync Tool on system restart, to ensure that the Sync Tool will continue to run even if you need to reboot your computer.
    • The installers add links and shortcuts which make it easy to find the Sync Tool when you need to see progress or change settings.
  • New features for the Sync Tool
    • The Sync Tool now provides 3 options for handling files which have changed locally: 
      1. Overwrite the file (the default, and the only option available in previous versions)
      2. Skip the file (ensures that there are no changes made to any existing files in DuraCloud)
      3. Rename original (rename the old copy of this file first, then add the new file to DuraCloud)
    • The Sync Tool now allows the DuraCloud password to be provided via a system property or via a prompt, both of which remove the need to include a password as part of the command line call
  • New features for the Retrieval Tool
    • The Retrieval Tool can now retrieve a list of the files in a space
    • The Retrieval Tool can now retrieve a specific set of files from a space, as well as being able to retrieval all files in a space
    • The Retrieval Tool now allows the DuraCloud password to be provided via a system property or via a prompt, both of which remove the need to include a password as part of the command line call

To see the full list of changes, or for more details about specific changes in release 2.4.0, see the JIRA issue tracker.

Release 2.3.1

Released: March 28, 2013

Release 2.3.1 is a bug fix release which focused on resolving bugs which resulted in errors when uploading files via the Sync Tool.

For more details about specific changes in release 2.3.1, see the JIRA issue tracker.

Release 2.3.0

Released: March 1, 2013

The primary features of release 2.3.0 are:

  • A new graphical Sync Tool
    • The Sync Tool can now be run using either a command line or a graphical user interface
      • The new graphical interface provides a setup wizard and an administration console with a monitoring display.
      • The familiar command line interface continues to provide access to all sync features, some of which are not yet available via the graphical interface.
      • Regardless of which interface is used, the underlying sync functionality remains the same.
    • The Sync Tool now captures and preserves the time stamps of files being transferred to DuraCloud, and the Retrieval Tool now re-instates those time stamps when files are retrieved.
    • The Sync Tool's command line interface now supports an exclude list, which can be used to specify files and directories that should not be pushed to DuraCloud.
  • Amazon Glacier integration
    • Amazon Glacier is now available as a secondary storage provider in DuraCloud

For more details about specific changes in release 2.3.0, see the JIRA issue tracker.

Important Known Issues:

  • Java Browser Security Vulnerabilities
    • The DuraCloud Upload Tool, which is deployed as a Java browser applet to assist with file uploads through the DuraCloud web UI, requires the Java browser plugin be installed.
    • Due to known security vulnerabilities in Java browser plugins, it is recommended that users upgrade their local version of Java to the latest available.
    • If you prefer to not use a Java browser plugin, or if the plugin is not functioning properly, we recommend using the newly updated Sync Tool, which is available from the "Get Sync Tool" button in the DuraCloud web UI, or from the DuraCloud downloads page.

A detailed list of known issues in release 2.3.0 may be found found here.

Release 2.2.0

Released: November 14, 2012

The primary features of release 2.2.0 are:

  • Java 7 support
    • The DuraCloud code base now requires Java 7 to build and run. This update is necessary both because Java 6 is reaching end of life, and to provide access to new features in Java 7.
    • All client-side Java tools now require Java 7 to run, this includes the Sync Tool, the Retrieval Tool, and the Upload Tool. Previous versions of these tools will continue to work with Java 6.
    • The file upload capability built into the DuraCloud UI (the Upload Tool in applet form) will now require Java browser plugins be updated to Java 7. Simply installing Java 7 for your platform should install the appropriate browser plugins.
  • A long list of UI tweaks and improvements including better graph and report display and labeling, improved multi-select support, and a variety of other small updates.

For more details about specific changes in release 2.2.0, see the JIRA issue tracker

Release 2.1.1

Released: September 28, 2012

The primary features of release 2.1.1 are:

  • Duplicate on Change service updates
    • Duplication settings for newly created spaces can now be specified using a default setting. Any new spaces created will be configured using the default duplication settings.
  • Space counting
    • The space counting feature in the UI now captures the final count on a space, and will only re-run the count when asked.

For more details about specific changes in release 2.1.1, see the JIRA issue tracker.

Release 2.1.0

Released: August 9, 2012

The primary features of release 2.1.0 are:

  • Duplicate on Change service updates
    • Duplication is now configured at the space level, allowing for much finer grained selection of which content will be automatically copied to a secondary provider, and which provider that will be.
  • SDSC Storage Provider
    • The SDSC storage provider connection was tested and improved, in close cooperation with SDSC personnel, to ensure its readiness for production status.
  • Security
    • The DuraCloud security configuration has been extended to only allow Administrative users to perform service execution.

For more details about specific changes in release 2.1.0, see the JIRA issue tracker.

Release 2.0.0

Released: April 17, 2012

The primary features of release 2.0.0 are:

  • Integrated History Reports
    • Reports illustrating both current and historical views of the content stored in DuraCloud have now been integrated directly into the display for each individual space.
  • Automated Service Execution
    • Bit Integrity Checks
      • Bit Integrity checking is now an automated function. Each content item in each space in each storage provider will be checked regularly to verify that it has not changed since it arrived in DuraCloud. No user intervention is required for this to occur. As each space is checked, the display will update to indicate that the bit integrity of the contained content has been verified.
    • Media Serving
      • Streaming content from DuraCloud now requires only a single button click. Each space now provides the option to turn on or off streaming, with no need to configure, deploy, or re-deploy a service.
  • Audit Logging
    • Logging is now being generated and captured which tracks the events occurring within DuraCloud. This provides a permanent record of when content is added, updated, or removed.
  • Manifest Generation
    • Building on the work of the audit log, a content manifest can now be requested for any space within DuraCloud. This manifest, which can be in multiple formats, describes the content that resides in the space the moment that the manifest is requested.

For more details about specific changes in release 2.0.0, see the JIRA issue tracker.

Release 1.3.1

Released: Jan 20, 2012

The primary features of release 1.3.1 are:

  • Improved service reporting
    • The information within service reports can now be viewed in tabular form directly from within the service details area.
    • Items which are considered error cases that occur as part of service execution are now included in an independent error report that is available for viewing upon service completion.
  • Simplified public access
    • Setting the contents of a space to be publicly viewable is now accomplished by simply granting read access to the "public" group.
  • CloudSync service
    • The latest version of CloudSync, a utility for managing the movement of content between DuraCloud and a Fedora repository, can now be run as a service within DuraCloud

For more details about specific changes in release 1.3.1, see the JIRA issue tracker.

Release 1.3

Released: Dec 14, 2011

The primary features of release 1.3 are:

  • Improved access control
    • Administrators can now define access control lists for each space, indicating which users and groups have read or write access to the content within that space.
    • Users now see only the spaces in their spaces listing which they have access to view. This includes all Open spaces, which remain available for public read access.
    • Users now only see options to perform add, edit, or delete actions in spaces where they have write permissions.
  • Content copy across providers
    • Files stored in DuraCloud can now be easily copied individually to another storage provider via both the REST API and the web interface.
  • Email notification on service completion
    • After a service in DuraCloud completes, an email is now sent to the user who launched the service, notifying them that the service has completed, and providing details about the results of the service.

For more details about specific changes in release 1.3, see the JIRA issue tracker.

Release 1.2

Released: Oct 30, 2011

The primary features of release 1.2 are:

  • Upload Tool
    • Provides a graphical method for transferring large numbers of files to DuraCloud.
    • Allows for the selection of both files and folders for transfer, and presents a visual indication of the transfer progress.
    • Can be run from the DuraCloud UI by choosing the "Add Many Items" button when a space is selected.
    • Can also be download and run locally.
  • SDSC Storage Provider
    • An initial beta release of the new storage provider integration for connecting to the SDSC Cloud storage system.

For more details about specific changes in release 1.2, see the JIRA issue tracker.

Release 1.1

Released: Sept 30, 2011

The primary features of release 1.1 are:

  • Media Streamer service
    • Service now allows content from multiple spaces to be streamed.
    • Service now recognizes when new content is added to spaces which are being streamed and starts streaming on those files.
  • Duplicate on Change service
    • Service updated to provide greater assurance of file transfer and produce an output report which details the actions of the service and the results of those actions.
  • Content item copy and rename capability
    • New copy operation is able to copy, move, and rename content items within a space and between spaces.
  • Stitch Tool
    • New utility for large files which were "chunked" (split into multiple small files) when placed into storage. This new utility will recombine all of the pieces of a file to re-produce the original file in local storage.
    • Stitch capabilities incorporated into the Retrieval Tool, allowing any content which has been "chunked" to be reconstituted on retrieval.
  • UI updates
    • Improvements to the user interface in order to provide simpler and more useful feedback for actions being performed throughout the application.
  • Properties
    • Use of the term 'properties' has replaced 'metadata' to describe the name/value pairs which can be associated with spaces and content. This update helps to clarify the purpose and capability of this attached information.
  • Local service repositories
    • Service repositories can now reside within the same storage container as is used by DuraStore, allowing for simpler configuration of stand-alone DuraCloud instances.
  • Initialization endpoint
    • New REST API endpoint (/init) for application initialization.
  • Tools tab
    • New tools tab on the dashboard provides convenient links for downloading DuraCloud client tools.

For more details about specific changes in release 1.1, see the JIRA issue tracker.

Release 1.0

Released: July 29, 2011

The primary features of release 1.0 are:

  • Storage Reporting
    • A new feature which provides detailed information about the amount of data you have stored in DuraCloud, as well as the kinds of data you have stored in DuraCloud. Reports are generated automatically and the information can be viewed using the DuraCloud dashboard or downloaded for processing using new REST API methods.
  • Service Reporting
    • A new feature which provides detailed information about services which are currently running and which have run previously in your DuraCloud account. Reports are generated automatically as services are run and the information can be viewed using the DuraCloud dashboard or downloaded for processing using new REST API methods.
  • Service Dependencies
    • Any service can now define a dependency on another service.
    • "System" services, which need to be installed prior to other services being deployed, are now installed on-the-fly only when needed.
  • Improved service feedback
    • The information provided by running services is now more complete and more consistent with other DuraCloud services.
  • Improved character set support
    • Content IDs can now consist of any characters which can be properly encoded using UTF-8 (with the exception of "reserved" characters mentioned here)
  • Image Viewer URL stability
    • The URLs for images made available by the Image Server service will now stay consistent across restarts of the service and restarts of the DuraCloud instance.

For more details about specific changes in release 1.0, see the JIRA issue tracker.

Release 0.9

Released: April 27, 2011

The primary features of release 0.9 are:

  • The Duplicate on Ingest service is now the Duplicate on Change service.
    • This service now supports all of the same on-ingest features as before, but it now also performs duplication of all update and delete actions as well. This allows the primary and secondary cloud stores to be kept completely in sync.
  • The Bulk Bit Integrity Service has been improved.
    • This service has been updated and verified to properly handle spaces with up to 1 million items
    • The second step of the MD5 verification, which used to run locally on the instance, has been moved to hadoop, allowing the service to complete much more quickly for large data sets.
  • User management functions have been removed, as they are now performed by the DuraCloud Management Console.
    • As a convenience, administrators are still able to see the list of users and their roles within the DuraCloud Administrator UI.
  • Service outputs have been made more consistent.
    • All DuraCloud services which produce an output file now store that file in the x-service-out space.
    • Services which produce log files store those logs in the x-service-work space.
    • The names of the output files have been made more consistent, making it simpler to determine which files correspond to which service deployment.
  • Password security has been improved.
    • All passwords used within DuraCloud are now immediately pushed through a hashing function before being are stored, so that no user passwords are transferred or stored as clear text.
  • A ServiceClient is now available, to compliment the StoreClient and make it easier to make direct API calls to manipulate DuraCloud services.

For more details about specific changes in release 0.9, see the JIRA issue tracker.

Release 0.8

Released: Jan 26, 2011

The primary features of release 0.8 are:

  • Simplified services
    • The listing of services has been better organized, to make finding the service you would like to run simpler.
    • All services now require you to set fewer options, simplifying the deployment process.
    • Bulk services (Image Transformer - Bulk, Bit Integrity Checker - Bulk, and Duplicate on Demand) now provide a standard configuration mode which handles the setting of server type, and number of servers used to perform the job, so that you no longer have to make those choices.
    • The output location for services has been set to the x-service-out space, which removes the need to set this value for each service, and provides a standard location to look for service output reports.
    • The work location for services has been set to the x-service-work space, which removes the need to set this value for each service, and provides a standard location to look for service logs and other run time artifacts.
  • More reliable services
    • Several bugs which have caused services to fail have been resolved.
  • Sync Tool command line flags now match those offered by the Retrieval Tool.
  • UI updates which provide better visual cues for which storage provider is in use.
  • A host of bug fixes and small tweaks

For more details about specific changes in release 0.8, see the JIRA issue tracker.

Release 0.7

Released: Oct 28, 2010

The primary features of release 0.7 are:

  • A new Retrieval Tool, a companion to the existing Sync Tool, which is a command-line tool for the retrieving content from DuraCloud spaces.
  • A new Bulk Bit Integrity Checker service, which can be run over content stored in Amazon to create a listing of checksums calculated for each file. This new service pairs well with the Bit Integrity Checker service (previously known as the Fixity Service), allowing the heavy processing to be handled in parallel using Hadoop on an Amazon EC2 cluster, and the simpler checks and comparisons to be handled by the DuraCloud instance.
  • A new Duplicate on Demand service, which can be used to copy files from the primary Amazon store into another storage provider. This service pairs well with the Duplicate on Upload service (previously known as the Replication Service) by performing the large up-front copy using Hadoop on an Amazon EC2 cluster, then allowing Duplicate on Upload to watch for and add new files as they are uploaded.
  • Integration of a new storage provider: Microsoft Windows Azure.

For more details about specific changes in release 0.7, see the JIRA issue tracker.

Note that there have been issues discovered during testing of the Bulk Image Transformer (included in release 0.6 as the Bulk Image Conversion Service). If you choose to run this service, it is recommended that the size of images being used be kept under 100MB. The likelihood of success appears to increase with server size, and number of servers being set to 3 or more is recommended. If you do run this service, please note the data set and configuration and make us aware of the outcome.

Release 0.6

Released: Sept 03, 2010

The primary features of release 0.6 are:

  • Addition of a new Fixity Service, which allows for bit integrity checking on content stored within DuraCloud. This service has many options to fit various usage needs. For more information, see the Fixity Service page.
  • Addition of a new Bulk Image Conversion Service, which, like the Image Conversion Service, allows for converting images into other formats. This new service, however, makes use of Hadoop in the background to run the conversion using multiple servers, allowing for much higher overall throughput.
  • An updated handling of space metadata so that spaces with a large number of content items will not cause slow response times. Now spaces with more than 1000 items will initially show a value of 1000+ as the number of items in the space. DurAdmin, the administrative interface, will then calculate the total number of items on the fly.
  • The Sync Tool has a new option (-e) which will cause the tool to exit once it has completed syncing rather than continually monitoring for changes. This makes it easier for administrators to include the Sync Tool in scripts which run daily or weekly to ensure all local content is moved to DuraCloud.
  • DurAdmin now provides a way to delete groups of content items and spaces in one step.
  • A host of bug fixes and small tweaks

For more details about specific changes in release 0.6, see the JIRA issue tracker.

Release 0.5

Released: July 28, 2010

The primary feature of release 0.5 is the addition of a completely new administrative user interface. This UI, called DurAdmin like its predecessor, provides for easy access to the primary features of DuraCloud.

For more details about specific changes in release 0.5, see the JIRA issue tracker.

Release 0.4

Released 0.4.1: June 30, 2010
  • This build release is the first publicly available
  • It primarily provides clean-up of projects and tests
Released 0.4: June 21, 2010

The primary features added in release 0.4 of DuraCloud were:

  • Media Streaming Service
    • Provides a way to enable streaming for video and audio files as well as providing an example media player.
  • Logging moved to SLF4J over Logback
    • Provides greater consistency in log output and greater flexibility in log configuration

For more details about specific changes in release 0.4, see the JIRA issue tracker.

Release 0.3

Released: May 17, 2010

The primary features in the third pilot release of DuraCloud are:

  • Security
    • All DuraCloud applications now require authentication prior to performing write activities
    • Read activities on 'closed' spaces also require authentication, but 'open' spaces allow anonymous read access
  • Sync Tool
    • Provides a command line utility for keeping DuraCloud content synchronized with the local file system

Other improvements in the 0.3 release:

  • Image Conversion Service
    • Adds an option to convert images to the (web standard) sRGB color space
    • Adds the capability to perform multiple conversions at once (providing the compute capacity is available) and provides more frequent activity feedback through the continual writing of the conversion output file
  • DuraStore
    • Adds an option for users to provide MD5 checksum when adding content. This disables the in-transfer MD5 computation (providing improved performance) and compares the final MD5 computed by the storage provider with the user provided MD5.

For more details about specific changes in release 0.3, see the JIRA issue tracker.

Release 0.2

Released: Feb 19, 2010

The second pilot release of DuraCloud focused on providing access to services which can be run over content, as well as improvements to the storage foundation provided by the first release.

Services available as of release 0.2:

  • J2K service - serves J2K images, provides a J2K image viewer
  • Image Conversion service - converts image files from one format to another
  • Replication service - replicates content stored in one provider to another upon content upload
  • Web Application Utility service - infrastructure service required by J2K service (allows for deployment of web applications)
  • ImageMagick service - infrastructure service required by Image Conversion service (provides access to ImageMagick utilities)

Service functions available as of release 0.2:

  • Services may be deployed with configuration
  • Available and deployed services may be listed
  • Deployed service configuration may be viewed and updated
  • Deployed service properties may be viewed
  • Deployed services may be undeployed and redeployed

New storage functions available as of release 0.2:

  • Space content may be listed in chunks with an optional prefix filter
  • Space and content metadata may be edited via the UI
  • Space and content metadata tags may be added/removed via the UI

For more details about specific changes in release 0.2, see the JIRA issue tracker. Note that while most items included in the release are listed in the tracker, we migrated to using JIRA while working on release 0.2, so issues completed prior to the migration are not included.

Release 0.1

Released: Nov 2, 2009

The first pilot release of DuraCloud laid the foundation for storage across underlying providers.

Through either the web-ui or via direct REST calls

  • underlying providers may be listed
  • spaces may be created/deleted
  • content may be uploaded/downloaded/deleted
  • metadata may be viewed
  • metadata may be modified
    • modification is fully supported through the REST API
    • modification is partially supported through the web-ui
  • No labels

All content on the LYRASIS Wiki is licensed under the CC BY (Attribution) license, unless otherwise noted.