We are initiating work to create a more robust embargo feature than the one that currently exists in DSpace. We've contracted with @mire to do the heavy lifting in terms of code work needed to meet our specific needs here at the University of Michigan. That work will start soon, so we'd like to invite others to tweak the feature set if there's something we've missed.

Business Requirements

Without going into detail about the technical implementation, the basic feature set we are looking at, which will build on the current implementation includes the ability to provide

1. Full embargo: no access to either the metadata or bitstream of an item until embargo period ends.  Items do not appear in search or browse results, even if a match exists; though the metadata and bitstreams are stored, it's as if the item and its bitstreams had never been deposited, or indexed.
2. Partial embargo: metadata visible, some/all bitstreams inaccessible. In this case, items appear in browse or search indexes so searchers and browsers know an item and its bitstreams exist. However, they can't download/view bitstreams. (This is the behavior of the current implementation.)
3. Group access to embargoed items: embargoed items (full or partial) are viewable and all contents are completely accessible to members of select groups.  This will address needs often expressed by archivists, who need a dark archive of materials in DSpace

So, when an item is under embargo:

• Whether the metadata page and all metadata fields are still visible to the world (and through OAI-PMH) should be configurable at the Item level.

• Whatever is viewable by an anonymous searcher and browser becomes part of the general search and browse indexes. Everything else is not.

• None of an item's bitstreams, including full text indexed by the system, are readable to the world under either a full or a partial embargo. Collection Administrators or pre-defined groups can bypass that protection.

• Any new bitstreams added to an item inherits that item's embargo settings.

• Administrators can change embargo status: either add an embargo to a currently open item, or release an item early.

• The presence of an embargo and its expiration date is visible in the administrative UI view of an item. Whether to display this to an end-user/non-administrator searcher or browser could be an option.

That's the basic idea, and I hope it's enough to give you a sense of where we're headed. The new embargo solution will be implemented for both XMLUI and JSPUI.

Again, we welcome comments on this and will do our best to incorporate your suggestions.

Technical Requirements

Associated JIRA Tasks

DS-908 Embargo Overhaul: Utilize ResourcePolicy Start and Stop datestamps for enforcing embargo in DSpace

This approach is currently implemented in IDEALS and is based on existing Tapir embargo work that was completed there.

The adoption of this approach would allow for Embargo to be applied at either the Item level or individual Bitstream levels as a series of ResourcePolicies that use start/stop datestamps currently on the ResourcePolicy object.

Benefits:

• Does not require executing a cronjob to adjust the state of the Item
• Enforcement of embargo is stateless driven off the current date and the defined embargo time period.
• Resource Policies and Item State do not change over time
• Allows a Resource Policy to be a first class DSpace Domain Model citizen and that would include expanding it to have  having name and description fields available to define the reason for a resource policy being set.

Establishment of a RESTRICT Action that would be enforced by the AuthorizationManager to allow for "Explicit Definition" of the Embargo Policy on Annonymous Users.

Example Use Cases:

Example 1 (Individual Bitstream Restriction)

Example 2 (Item Access Restriction)

The previous example would enforce Embargo and Access rights "Explicitly" and "Clearly" in the Policies attached to the Bitstream and/or Item. The AuthorizationManager may need minor enhancement to address "inheritance" of ResourcePolicies assigned on parent Items. It may be advisable to use such inheritance to enforce "DEFAULT_XXX" policies rather than copying them into place on each and every Bitstream/Bundle and Item created, this will reduce the "bloat" of ResourcePolicies currently in effect in the existing system.

And important benefit of these changes to ResourcePolicies and the underlying AuthorizationManager framework are that they can then be used to encode the explicit technical or administrative metadata sections into the AIP or METS manifests concerning the Policies that are in effect on the Item and its contents. Adjustments to the DSpace SIP Profile to capture enforcement of embargo details by consumers of those tools would be more clearly expressed and machine automatable than dumping it intot he metadata. Achieving Machine actionability means that Ingest Packagers and services that rely on them can define a more concrete business logic to be maintained.

As we evolve the Metadata capabilities to support system/tech/admin/descriptive metadata sections for all parts of the item, we can consider that the ResourcePolices will inform the production of metadata about the embargo state of the Item being exposed in OAI / SWORD /METS packagers and so-on. But for now, we really need to set a standard that actual Resource Policies be the mechanism that enforces the access rules/policies within the system and not some metadata field set in the item metadata description.

Somewhat a concern is how other areas of DSpace treat ResourcePolicies rather bluntly. Recommend that ResourcePolicies should be managed in central manner (such as ResourcePolicyService: or "ResorcePolicyManager") such that the manner in which policies are enforced or allowed to be edited does NOT cause emergent conflicting behavior across different parts of the system such as those described within DS-906 and DS-525.

According the DS-525, the issue of embargoed items is documented as a warning in our Documentation: https://wiki.duraspace.org/display/DSDOC/System+Administration#SystemAdministration-Movingitems

I consider this documentation insufficient as a solution to the problem of embargo permissions getting overridden in the mapping. A more appropriate solution would show to the user the exact changes that would happen to the item and allow them to decide which policies should be enforce/changed on the item.

Restricting Discovery of Embargoed Items in Search and Browse

Item will need to be restricted from search and browse, the Tapir solution for restricting Item from viewing in Search and Browse is to set Resource Policies on the Item that limit access to it, during the embargo period, an an "EmbargoLifter" command is responsible for removing these embargo settings fromt he Item to lift the embargo

Restricting viewing of Embargoed Items

Resource Policies are currently set into place on DSpace Bitstreams during the desired period defined in the (usually dc.date.embargountil

Problem: Lifting the embargo removes any record of the original embargo being in effect/

Solution: Use of the provided "Start" and "End dates on the Resource Policies would allow the creation of an Embargo Period that could be preserved with the Item and would not need physical alteration to release the Item from Embargo.

DSpace Items Private 

The AuthorizationManager already supports the enforcement of timeframes in ResourcePolicies. I would like to propose that we expand ResourcePolicy in the following manner:

Migration from existing Embargo solutions

Metadata Concerning the setting of an embargo term and lift date may still be recoreded in the metadata. ALterations may include:

1. Not relying on date described in metadata to manage access rights
2. Exposure of the embargo details for any item in OAI/METS, XMLUI/METS or JSPUI Item views by evaluation of the ResourcePolices that are in force.