DSpace 2015-18 Strategic Plan - Technology

(Very Rough) Summary

Over the last few years, the Steering Group along with various strategic working groups have validated the following vision statements which describe the goals of the DSpace open source product:

1. DSpace will focus on the fundamentals of the modern "Institutional Repository" use case. We are striving to meet the IR needs of the next 5-10 years.
2. DSpace will be "lean", with agility and flexibility as primary goals.
3. DSpace will include a "core" set of functionality that can be "extended" (think plugins) or have "hooks" (integration points) to complimentary services/tools
4. DSpace will be designed in such a way that it can be easily/quickly configured to integrate with new & future tools/services in the larger digital scholarship "ecosystem"
5. DSpace will support low-cost, hosted solutions and deployments (by featuring an easy, "just works" setup)

These five statements lead us to a proposed "value proposition" for the DSpace platform (DRAFT)

DSpace is a turn-key web application for maintaining, disseminating and promoting the digital resources or digital output of an organization and its members.

Additional notes with regards to strategic direction of the software platform / technology:

• We do NOT plan to rewrite DSpace from scratch, for the following reasons:
  • We have a highly active (and global) development community on the existing platform. We are averaging 50+ contributors in recent major releases. We also have a very active and healthy set of service providers.
  • A complete rewrite would require significant funding / centralized resources, neither of which are currently available. There also seem to be few (if any) grant opportunities to rebuild existing, established platforms.
  • A complete rewrite is very risky in the open source world. While in some cases it can succeed, it also can run the risk of fragmenting or fracturing a user community or developer community.
• We ARE aiming for a potentially substantial leap forward in user experience / web user interface.
  • We've heard the feedback that neither of the two UIs (JSPUI or XMLUI) provides an optimal user or administrative experience. So, a UI rewrite or major refactoring would be "on the table".

Based on this proposed value proposition and vision statements, the Steering Group recommends the following actions corresponding to each goal:

Goal 1: DSpace will focus on the fundamentals of the modern "Institutional Repository" use case.

Assigned: Tim Donohue

In November 2002, DSpace was initially announced as an out-of-the-box "institutional repository software platform" (see DSpace 1.0 release announcement). While that basic goal has not changed, the common needs and use cases of an "institutional repository" have changed significantly in the last decade or so.  Therefore, this goal is oriented towards striving to retain DSpace's niche while revitalizing it to meet current and future use cases associated with the modern repository platform.

• Action 1A: Verify and validate the needs of a "modern institutional repository". This is instrumental in formalizing the value proposition of DSpace.
  
  • This activity has already begun part of the 2014 Vision Survey and Use Case gathering by DCAT. However, the gathered use cases are still being analyzed and prioritized.
• Action 1B: Community Survey (Do we want another, post OR15 survey of the community based on Strategic Plan, Tech Roadmap & Use Case gathering?)
• Action 1C: Clarify and improve the DSpace value proposition. Enhance our message and goals of the project going forward.  The DSpace Steering Group is working towards establishment of a new "DSpace Marketing Working Group" to help with this effort
  
  • NOTE: This is not really a "Technology" activity, but more of a "Sustainability" activity.

Goal 2: DSpace will be "lean", with agility and flexibility as primary goals

Assigned: Tim Donohue

Since its initial release in 2002, numerous features, configurations and options have been added to the DSpace codebase in an ongoing effort to keep up with the changing needs of its user base. While many of these changes have helped us to achieve new use cases, in some instances they have also complicated the codebase and made setup and upgrades more complex. Therefore, this goal is oriented towards cleaning up (and simplifying) the codebase and its configuration options, while also working towards avoiding duplication (of code and development efforts). We feel DSpace can be a "leaner" platform, which will allow the codebase to better adapt to the needs of the future and simplify its maintenance, setup and upgrade processes.

• Action 2A: Avoid duplicative functionality. To be "lean", the DSpace technology platform should avoid duplicative functionality except where necessary to meet use cases or achieve "flexibility" goals.  Where unnecessary duplicative functionality already exists, the technology team should choose a "best option" solution, or propose building a new solution when a "best option" does not exist.
  
  • Examples of unnecessary duplicative functionality, which should be resolved:
    
    • maintaining multiple User Interfaces (JSPUI vs. XMLUI)
    • maintaining multiple search/browse systems (Solr vs. Lucene)
    • maintaining multiple built-in statistical engines (Solr vs Elastic Search)
  • Examples of necessary duplicative functionality. Each of these duplicative functions would be considered in line with use cases or supportive of our goal for "flexibility" (of data import/export/storage).
    
    • supporting multiple database storage backends (PostgreSQL, Oracle, etc.)
    • supporting multiple interfaces for deposit (SWORD, REST, etc.)
    • supporting multiple interfaces for export (OAI-PMH, REST, RDF, etc.)
    • supporting multiple external identifier schemes (Handle system, DOI, etc.)
• Action 2B: Enhance communication and collaboration. In order to avoid duplicative effort / projects in the future, we need to enhance communication and collaboration between various developers, institutions and service providers, and communicate a common vision / roadmap for the platform.
  
  • As a part of this action, we will be developing and maintaining a Technical Roadmap for the DSpace software platform, detailing high priority development projects which can be "claimed" or shared by groups of developers, institutions or service providers
  • At the same time, we should recognize that ad-hoc or competitive work can also be beneficial. We should continue to encourage ad-hoc contributions, and even competitive contributions, but make it clear that we will actively discourage further unnecessary duplicative functionality to be added to the platform (see Action 2A for examples).
  • Develop a basic User Interface "style / layout guide". In order to ensure a consistent user experience across all pages/features within the User Interface, we should provide basic guidelines for layout and styling of common page elements, etc. Examples may include basic guidelines for how errors/warnings/notices should be displayed, what class(es) to use for types of buttons, etc.

Goal 3: DSpace will include a "core" set of functionality that can be "extended" (think plugins) or have "hooks" (integration points) to complimentary services/tools

Assigned: Richard Rodgers

[summary of goal]  There will obviously be limitations to what DSpace can and should do, so we need to have ways to support plugins/addons/extensions to that core functionality. Not all users of DSpace will need to achieve the same set of Use Cases, so we will need to define which are "core" and which would be better implemented as plugins/extensions (either centrally supported or third-party supported).

• Action 3A: Identify minimum set of functionality/features for 'IR-core' and refactor codebase to provide this. This core may not be functional as-is, since it may require plugins that aren't extensions (e.g. authN)
• Action 3B: Define a family of specifications/interfaces/tooling for 'implementation' plugins (like authN, storage, persistent ID services, etc).
• Action 3C: Define a family of specifications/interfaces for functional extensions to 'IR-core' ( working title: 'modules'), and refactor existing bundled code to conform to new model (if appropriate/cost-effective). 
• Action 3D: Provide infrastructure/tools for a module registry, where users can discover, and install modular extensions. Likely include both modules maintained by committers and community contributions.
• Action 3E: Devise a flexible but rigorous system of versioning all components (core, module, etc) where compatibility requirements can be checked/enforced by the build/deploy tools.
• Action 3F: Define and expose new interfaces (in 'IR-core' and possibly modules) to allow local customized code to run: 'integration points'.
• Action 3G: (Highly dependent on UI architectural work) Provide a user-discoverable registry/library of user interface templates (working title: 'themes'), that can be installed and adapted for local use. 

Goal 4: DSpace will be designed in such a way that it can be easily/quickly configured to integrate with new & future tools/services in the larger digital scholarship "ecosystem"

Assigned: Bram Luyten

To provide easy and out of the box integration aspects with external services in the following areas:

• Action 4A: Ingest of complete metadata records (items) from external services, with or without files.

• Action 4B: Consume external authority control sources to enrich specific metadata fields 

• Action 4C: Expose metadata and content to external services

• Action 4D: Expose usage data to external services

• Action 4E: Integrate with external Authentication and Single Sign on services

• Action 4F: Integrate with external services providing identifiers (handle, datacite, ...)
• Action 4G: Integrate with external storage and backup services (Duracloud, Amazon Glacier/S3, Arkivum, ...)

To integrate with parallel projects and initiatives (fedora, hydra, islandora) we first need to pin down the use cases of what those integrations will bring to DSpace, or what these will bring to the other platforms. They currently do not fit immediately in any of these five areas.

More examples are in the comment below, but those are probably already too detailed for this page and should end up in the technology document.

Goal 5: DSpace will support low-cost, hosted solutions and deployments (by featuring an easy, "just works" setup)

Assigned: Stuart Lewis, Robin Taylor, Claire Knowles

DSpace should be easy to install without requiring Java development expertise, to configure without requiring server access, and to monitor from within the application. Basic configuration options, including the look and feel and selecting themes should be accessible from within the DSpace online administration area.

• Action 5A: Investigate download, packaging, and installation tools for Java web applications to make it easier to build a working system; What do comparator systems use?
• Action 5B: Examine options for lightweight installation with setup tasks such as database population undertaken via the web interface upon first use, and creation of initial admin account;
• Action 5C: Move configuration of basic theme configuration options (colours, logo) into administrative interface;
• Action 5D: Move most configuration into the database, so that it can be updated via the administration screens;
• Action 5E: Document options for scaling and load-balancing DSpace;
• Action 5F: Consider developing a Simple Asset Store as the default asset store that uses a more logical naming structure for human traversal;
• Action 5G: DSpace should have an understanding of its requirements and dependencies and report on issues (e.g. "solr webapp not accessible / working");
• Action 5H: Ensure no data is stored in transient technologies (for example solr statistics do not have persistent storage and can be easily lost or corrupted);
• Action 5I: Create alerting tool for new upgrades to alert administrators within the admin user interface;
• Action 5J: Investigate options to assist with upgrades (for example highlighting changes from core code or configurations