Release 2.3.0

Released: Mar 1, 2013

The primary features of release 2.3.0 are:

• A new graphical Sync Tool
  
  • A graphical management application was added to an improved Sync Tool that supports a setup wizard and a administration console with a monitoring display.
  • The command line tool is also included in the download for users, especially on servers, who prefer it.  The underlying sync functionality is the same in both packages.
  • The new Sync Tool, like all client-side Java tools, now require Java 7 to run.
• A long list of UI tweaks and improvements including better graph and report display and labeling, improved multi-select support, and a variety of other small updates.

Important Known Issues:

• Java Browser Security Vulnerabilities
  
  • A number of Java browser-based security exploits surfaced that, while not an issue for DuraCloud servers, could make user's desktops vulnerable through their browsers.
  • The DuraCloud upload tool, usually used as a Java browser applet, requires the appropriate browser plug-ins be installed.
  • Please read this section for details about protecting your desktops, or better, install the new and improved Sync Tool. 

For more details about specific changes in release 2.3.0, see the JIRA issue tracker

Release 2.2.0

Released: November 14, 2012

The primary features of release 2.2.0 are:

• Java 7 support
  
  • The DuraCloud code base now requires Java 7 to build and run. This update is necessary both because Java 6 is reaching end of life, and to provide access to new features in Java 7.
  • All client-side Java tools now require Java 7 to run, this includes the Sync Tool, the Retrieval Tool, and the Upload Tool. Previous versions of these tools will continue to work with Java 6.
  • The file upload capability built into the DuraCloud UI (the Upload Tool in applet form) will now require Java browser plugins be updated to Java 7. Simply installing Java 7 for your platform should install the appropriate browser plugins.
• A long list of UI tweaks and improvements including better graph and report display and labeling, improved multi-select support, and a variety of other small updates.

For more details about specific changes in release 2.2.0, see the JIRA issue tracker

Release 2.1.1

Released: September 28, 2012

The primary features of release 2.1.1 are:

• Duplicate on Change service updates
  • Duplication settings for newly created spaces can now be specified using a default setting. Any new spaces created will be configured using the default duplication settings.
• Space counting
  • The space counting feature in the UI now captures the final count on a space, and will only re-run the count when asked.

For more details about specific changes in release 2.1.1, see the JIRA issue tracker.

Release 2.1.0

Released: August 9, 2012

The primary features of release 2.1.0 are:

• Duplicate on Change service updates
  • Duplication is now configured at the space level, allowing for much finer grained selection of which content will be automatically copied to a secondary provider, and which provider that will be.
• SDSC Storage Provider
  • The SDSC storage provider connection was tested and improved, in close cooperation with SDSC personnel, to ensure its readiness for production status.
• Security
  • The DuraCloud security configuration has been extended to only allow Administrative users to perform service execution.

For more details about specific changes in release 2.1.0, see the JIRA issue tracker.

Release 2.0.0

Released: April 17, 2012

The primary features of release 2.0.0 are:

• Integrated History Reports
  • Reports illustrating both current and historical views of the content stored in DuraCloud have now been integrated directly into the display for each individual space.
• Automated Service Execution
  • Bit Integrity Checks
    • Bit Integrity checking is now an automated function. Each content item in each space in each storage provider will be checked regularly to verify that it has not changed since it arrived in DuraCloud. No user intervention is required for this to occur. As each space is checked, the display will update to indicate that the bit integrity of the contained content has been verified.
  • Media Serving
    • Streaming content from DuraCloud now requires only a single button click. Each space now provides the option to turn on or off streaming, with no need to configure, deploy, or re-deploy a service.
• Audit Logging
  • Logging is now being generated and captured which tracks the events occurring within DuraCloud. This provides a permanent record of when content is added, updated, or removed.
• Manifest Generation
  • Building on the work of the audit log, a content manifest can now be requested for any space within DuraCloud. This manifest, which can be in multiple formats, describes the content that resides in the space the moment that the manifest is requested.

For more details about specific changes in release 2.0.0, see the JIRA issue tracker.

Release 1.3.1

Released: Jan 20, 2012

The primary features of release 1.3.1 are:

• Improved service reporting
  • The information within service reports can now be viewed in tabular form directly from within the service details area.
  • Items which are considered error cases that occur as part of service execution are now included in an independent error report that is available for viewing upon service completion.
• Simplified public access
  • Setting the contents of a space to be publicly viewable is now accomplished by simply granting read access to the "public" group.
• CloudSync service
  • The latest version of CloudSync, a utility for managing the movement of content between DuraCloud and a Fedora repository, can now be run as a service within DuraCloud

For more details about specific changes in release 1.3.1, see the JIRA issue tracker.

Release 1.3

Released: Dec 14, 2011

The primary features of release 1.3 are:

• Improved access control
  • Administrators can now define access control lists for each space, indicating which users and groups have read or write access to the content within that space.
  • Users now see only the spaces in their spaces listing which they have access to view. This includes all Open spaces, which remain available for public read access.
  • Users now only see options to perform add, edit, or delete actions in spaces where they have write permissions.
• Content copy across providers
  • Files stored in DuraCloud can now be easily copied individually to another storage provider via both the REST API and the web interface.
• Email notification on service completion
  • After a service in DuraCloud completes, an email is now sent to the user who launched the service, notifying them that the service has completed, and providing details about the results of the service.

For more details about specific changes in release 1.3, see the JIRA issue tracker.

Release 1.2

Released: Oct 30, 2011

The primary features of release 1.2 are:

• Upload Tool
  • Provides a graphical method for transferring large numbers of files to DuraCloud.
  • Allows for the selection of both files and folders for transfer, and presents a visual indication of the transfer progress.
  • Can be run from the DuraCloud UI by choosing the "Add Many Items" button when a space is selected.
  • Can also be download and run locally.
• SDSC Storage Provider
  • An initial beta release of the new storage provider integration for connecting to the SDSC Cloud storage system.

For more details about specific changes in release 1.2, see the JIRA issue tracker.

Release 1.1

Released: Sept 30, 2011

The primary features of release 1.1 are:

• Media Streamer service
  • Service now allows content from multiple spaces to be streamed.
  • Service now recognizes when new content is added to spaces which are being streamed and starts streaming on those files.
• Duplicate on Change service
  • Service updated to provide greater assurance of file transfer and produce an output report which details the actions of the service and the results of those actions.
• Content item copy and rename capability
  • New copy operation is able to copy, move, and rename content items within a space and between spaces.
• Stitch Tool
  • New utility for large files which were "chunked" (split into multiple small files) when placed into storage. This new utility will recombine all of the pieces of a file to re-produce the original file in local storage.
  • Stitch capabilities incorporated into the Retrieval Tool, allowing any content which has been "chunked" to be reconstituted on retrieval.
• UI updates
  • Improvements to the user interface in order to provide simpler and more useful feedback for actions being performed throughout the application.
• Properties
  • Use of the term 'properties' has replaced 'metadata' to describe the name/value pairs which can be associated with spaces and content. This update helps to clarify the purpose and capability of this attached information.
• Local service repositories
  • Service repositories can now reside within the same storage container as is used by DuraStore, allowing for simpler configuration of stand-alone DuraCloud instances.
• Initialization endpoint
  • New REST API endpoint (/init) for application initialization.
• Tools tab
  • New tools tab on the dashboard provides convenient links for downloading DuraCloud client tools.

For more details about specific changes in release 1.1, see the JIRA issue tracker.

Release 1.0

Released: July 29, 2011

The primary features of release 1.0 are:

• Storage Reporting
  • A new feature which provides detailed information about the amount of data you have stored in DuraCloud, as well as the kinds of data you have stored in DuraCloud. Reports are generated automatically and the information can be viewed using the DuraCloud dashboard or downloaded for processing using new REST API methods.
• Service Reporting
  • A new feature which provides detailed information about services which are currently running and which have run previously in your DuraCloud account. Reports are generated automatically as services are run and the information can be viewed using the DuraCloud dashboard or downloaded for processing using new REST API methods.
• Service Dependencies
  • Any service can now define a dependency on another service.
  • "System" services, which need to be installed prior to other services being deployed, are now installed on-the-fly only when needed.
• Improved service feedback
  • The information provided by running services is now more complete and more consistent with other DuraCloud services.
• Improved character set support
  • Content IDs can now consist of any characters which can be properly encoded using UTF-8 (with the exception of "reserved" characters mentioned here)
• Image Viewer URL stability
  • The URLs for images made available by the Image Server service will now stay consistent across restarts of the service and restarts of the DuraCloud instance.

For more details about specific changes in release 1.0, see the JIRA issue tracker.

Release 0.9

Released: April 27, 2011

The primary features of release 0.9 are:

• The Duplicate on Ingest service is now the Duplicate on Change service.
  • This service now supports all of the same on-ingest features as before, but it now also performs duplication of all update and delete actions as well. This allows the primary and secondary cloud stores to be kept completely in sync.
• The Bulk Bit Integrity Service has been improved.
  • This service has been updated and verified to properly handle spaces with up to 1 million items
  • The second step of the MD5 verification, which used to run locally on the instance, has been moved to hadoop, allowing the service to complete much more quickly for large data sets.
• User management functions have been removed, as they are now performed by the DuraCloud Management Console.
  • As a convenience, administrators are still able to see the list of users and their roles within the DuraCloud Administrator UI.
• Service outputs have been made more consistent.
  • All DuraCloud services which produce an output file now store that file in the x-service-out space.
  • Services which produce log files store those logs in the x-service-work space.
  • The names of the output files have been made more consistent, making it simpler to determine which files correspond to which service deployment.
• Password security has been improved.
  • All passwords used within DuraCloud are now immediately pushed through a hashing function before being are stored, so that no user passwords are transferred or stored as clear text.
• A ServiceClient is now available, to compliment the StoreClient and make it easier to make direct API calls to manipulate DuraCloud services.

For more details about specific changes in release 0.9, see the JIRA issue tracker.

Release 0.8

Released: Jan 26, 2011

The primary features of release 0.8 are:

• Simplified services
  • The listing of services has been better organized, to make finding the service you would like to run simpler.
  • All services now require you to set fewer options, simplifying the deployment process.
  • Bulk services (Image Transformer - Bulk, Bit Integrity Checker - Bulk, and Duplicate on Demand) now provide a standard configuration mode which handles the setting of server type, and number of servers used to perform the job, so that you no longer have to make those choices.
  • The output location for services has been set to the x-service-out space, which removes the need to set this value for each service, and provides a standard location to look for service output reports.
  • The work location for services has been set to the x-service-work space, which removes the need to set this value for each service, and provides a standard location to look for service logs and other run time artifacts.
• More reliable services
  • Several bugs which have caused services to fail have been resolved.
• Sync Tool command line flags now match those offered by the Retrieval Tool.
• UI updates which provide better visual cues for which storage provider is in use.
• A host of bug fixes and small tweaks

For more details about specific changes in release 0.8, see the JIRA issue tracker.

Release 0.7

Released: Oct 28, 2010

The primary features of release 0.7 are:

• A new Retrieval Tool, a companion to the existing Sync Tool, which is a command-line tool for the retrieving content from DuraCloud spaces.
• A new Bulk Bit Integrity Checker service, which can be run over content stored in Amazon to create a listing of checksums calculated for each file. This new service pairs well with the Bit Integrity Checker service (previously known as the Fixity Service), allowing the heavy processing to be handled in parallel using Hadoop on an Amazon EC2 cluster, and the simpler checks and comparisons to be handled by the DuraCloud instance.
• A new Duplicate on Demand service, which can be used to copy files from the primary Amazon store into another storage provider. This service pairs well with the Duplicate on Upload service (previously known as the Replication Service) by performing the large up-front copy using Hadoop on an Amazon EC2 cluster, then allowing Duplicate on Upload to watch for and add new files as they are uploaded.
• Integration of a new storage provider: Microsoft Windows Azure.

For more details about specific changes in release 0.7, see the JIRA issue tracker.

Note that there have been issues discovered during testing of the Bulk Image Transformer (included in release 0.6 as the Bulk Image Conversion Service). If you choose to run this service, it is recommended that the size of images being used be kept under 100MB. The likelihood of success appears to increase with server size, and number of servers being set to 3 or more is recommended. If you do run this service, please note the data set and configuration and make us aware of the outcome.

Release 0.6

Released: Sept 03, 2010

The primary features of release 0.6 are:

• Addition of a new Fixity Service, which allows for bit integrity checking on content stored within DuraCloud. This service has many options to fit various usage needs. For more information, see the Fixity Service page.
• Addition of a new Bulk Image Conversion Service, which, like the Image Conversion Service, allows for converting images into other formats. This new service, however, makes use of Hadoop in the background to run the conversion using multiple servers, allowing for much higher overall throughput.
• An updated handling of space metadata so that spaces with a large number of content items will not cause slow response times. Now spaces with more than 1000 items will initially show a value of 1000+ as the number of items in the space. DurAdmin, the administrative interface, will then calculate the total number of items on the fly.
• The Sync Tool has a new option (-e) which will cause the tool to exit once it has completed syncing rather than continually monitoring for changes. This makes it easier for administrators to include the Sync Tool in scripts which run daily or weekly to ensure all local content is moved to DuraCloud.
• DurAdmin now provides a way to delete groups of content items and spaces in one step.
• A host of bug fixes and small tweaks

For more details about specific changes in release 0.6, see the JIRA issue tracker.

Release 0.5

Released: July 28, 2010

The primary feature of release 0.5 is the addition of a completely new administrative user interface. This UI, called DurAdmin like its predecessor, provides for easy access to the primary features of DuraCloud.

For more details about specific changes in release 0.5, see the JIRA issue tracker.

Release 0.4

Released 0.4.1: June 30, 2010

• This build release is the first publicly available
• It primarily provides clean-up of projects and tests

Released 0.4: June 21, 2010

The primary features added in release 0.4 of DuraCloud were:

• Media Streaming Service
  • Provides a way to enable streaming for video and audio files as well as providing an example media player.
• Logging moved to SLF4J over Logback
  • Provides greater consistency in log output and greater flexibility in log configuration

For more details about specific changes in release 0.4, see the JIRA issue tracker.

Release 0.3

Released: May 17, 2010

The primary features in the third pilot release of DuraCloud are:

• Security
  • All DuraCloud applications now require authentication prior to performing write activities
  • Read activities on 'closed' spaces also require authentication, but 'open' spaces allow anonymous read access
• Sync Tool
  • Provides a command line utility for keeping DuraCloud content synchronized with the local file system

Other improvements in the 0.3 release:

• Image Conversion Service
  • Adds an option to convert images to the (web standard) sRGB color space
  • Adds the capability to perform multiple conversions at once (providing the compute capacity is available) and provides more frequent activity feedback through the continual writing of the conversion output file
• DuraStore
  • Adds an option for users to provide MD5 checksum when adding content. This disables the in-transfer MD5 computation (providing improved performance) and compares the final MD5 computed by the storage provider with the user provided MD5.

For more details about specific changes in release 0.3, see the JIRA issue tracker.

Release 0.2

Released: Feb 19, 2010

The second pilot release of DuraCloud focused on providing access to services which can be run over content, as well as improvements to the storage foundation provided by the first release.

Services available as of release 0.2:

• J2K service - serves J2K images, provides a J2K image viewer
• Image Conversion service - converts image files from one format to another
• Replication service - replicates content stored in one provider to another upon content upload
• Web Application Utility service - infrastructure service required by J2K service (allows for deployment of web applications)
• ImageMagick service - infrastructure service required by Image Conversion service (provides access to ImageMagick utilities)

Service functions available as of release 0.2:

• Services may be deployed with configuration
• Available and deployed services may be listed
• Deployed service configuration may be viewed and updated
• Deployed service properties may be viewed
• Deployed services may be undeployed and redeployed

New storage functions available as of release 0.2:

• Space content may be listed in chunks with an optional prefix filter
• Space and content metadata may be edited via the UI
• Space and content metadata tags may be added/removed via the UI

For more details about specific changes in release 0.2, see the JIRA issue tracker. Note that while most items included in the release are listed in the tracker, we migrated to using JIRA while working on release 0.2, so issues completed prior to the migration are not included.

Release 0.1

Released: Nov 2, 2009

The first pilot release of DuraCloud laid the foundation for storage across underlying providers.

Through either the web-ui or via direct REST calls

• underlying providers may be listed
• spaces may be created/deleted
• content may be uploaded/downloaded/deleted
• metadata may be viewed
• metadata may be modified
  • modification is fully supported through the REST API
  • modification is partially supported through the web-ui