Child pages
  • DSpace Spring Services Tutorial
Skip to end of metadata
Go to start of metadata

DSpace Service Manager Tutorial

Introduction

The objectives of this tutorial are to provide the general DSpace Developer with an understanding of what the DSpace Service Manager is and what we are attempting to attain through its usage and some basic software development practices and design principles.  The tutorial will break these practices and the overall development platform down into developer centric terms that should, if successful, give the developer a reference for how to best design their code.  Goals of the Service Manager are to assist in separating the dependencies of individual functional areas of DSpace on one another, by eliminating significant dependencies on other parts of the codebase and the prevalence of "StaticManager" Classes.

The DSpace Services Framework is a back-porting of the DSpace 2.0 Development Group's work in creating a reasonable and simple "Core Services" layer for DSpace. The Services Framework provides a means for application developers to both lookup and register their own "services" or JAVA objects that can be referred to by the application.

What are services?

Answer: services are a generic term for the business actions that provide functionality that will complete a specific task in the application.

In DSpace Services are conceptually similar to OSGi Services , where an addon library (a OSGi Bundle) delivers a singleton instance of a class as a service for other application code to utilize.  In OSGi the Service often has a Java Interface class and constitutes a "Contract" for the application.  

From a Design Standpoint, The Service Manager is a Service Locator Pattern. This shift represents a "best practice" for new DSpace architecture and the implementation of extensions to the DSpace application. DSpace Services are best described as a "Registry" of Services that are delivered to the application for use by the use of a Spring Application Context. The original (DSpace 2.0 ) core services are the main services that make up a DSpace Service Manager system. These include services for the application "Configuration", "Transactional Context", "Requests" and user "Session",  "Persistence" things like user and permissions management and storage and caching. These services can be used by any developer writing DS2 plugins (e.g. statistics), providers (e.g. authentication), or user interfaces (e.g. JSPUI).

What is OSGi?

An OSGi service is a java object instance, registered into an OSGi framework with a set of properties. Any java object can be registered as a service, but typically it implements a well-known interface. Considerable has evolved in both the Spring and OSGi Communities.

What is Spring?

Spring is an Inversion of Control (IoC) container that utilizes Dependency Injection (a fancy way of just saying that Spring creates and hands the JAVA objects you would normally have had to create in your code).

Example of Not Using Spring

As a brief example Here is a class that does not use Dependency Injection, it calls "new SomeOtherClass()" directly in its constructor.

Example Using Spring

Here is an example of a better coding practice, where we give up the responsibility for the creation of the class and allow the IoC/DI container to be responsible for its creation.

This immediately opens the door for "SomeOtherClass" to be changed out with new/other subclasses of "SomeOtherClass".  In Spring, the definition "example.xml" file might look like this:

Where, when the ServiceManager is started, two beans are instantiated by Spring, and one is used in the constructor argument of the other. The simplest instantiation of a Spring Container might look like the following.

The Case For Spring

This tutorial focuses on adoption of Spring as a best practice for many aspects of DSpace development, from Core Library definition and instantiation to Application Developer implementation of customizations and addons.

  • Spring focuses around providing a way to manage your business objects. (DSpace currently lacks this capability).
  • Spring is both comprehensive and modular. Spring has a layered architecture, you can choose to use just about any part of it in isolation.
  • It is easy to introduce Spring incrementally into existing projects. (The Latest DSpace WebMVC, REST and XMLUI development efforts already leverage Spring WebMVC in the application tier).
  • Spring is designed from the ground up to help you write code that's easy to test. Spring is an ideal framework for test driven projects. (DSpace has only just introduced a JUnit Test Suite, which does not leverage Spring in its solution. However, the DSpace Service Manager already delivers a testing suite leverages Spring to support testing configuration).
  • Spring is an increasingly important integration technology, its role recognized by several large vendors. By utilizing Spring, DSpace will be able to incrementally improve its architecture to be more robust, and more "enterprise grade".

The Service Manager

The ServiceManager provides the DSpace Application with the above Spring ApplicationContext so that the developer  does not need to be responsible for its creation when developing against DSpace. Thus, to extend the pervious example, the DSpace class and its underlying Service Manager can be utilized to get at any object that has been instantiated as a service bean by core or addon application code.

The DSpace Service Manager implementation manages the entire lifecycle of a running DSpace application and provides access to services by Applications that may be executing external to this "kernel" of DSpace Services. Via Spring and loading of individual dspace.cfg properties the ServiceManager manages the configuration of those services, (either through providing those properties to the Spring Application Context where they can be injected in Spring definition xml files and/or "annotations" or by exposing those properties via the injection of the DSpace ConfigurationService.

DSpace ConfigurationService

The ServiceManagerSystem abstraction allows the DSpace ServiceManager to use different systems to manage its services. The current implementation is Spring Framework based. The original design intent of the Service Manager was to support more than one IoC/DI Solution, however, as work has progressed with DSpace, it has become clear that there are trade offs to consider in its usage.

  1. Spring Injection, when used properly, means we do not reallyneed to make "lookup calls" to a central "ServiceManager" or "DSpace" object to acquire the service beans we our own code to work with.
  2. Fewer lookups mean less centralized dependencies
  3. Fewer Centralized Dependencies means fewer bottlenecks in source Code Dependency Management.
  4. Fewer Bottlenecks means greater modularity and encapsulation, less need to carry around all the source code when overriding and customizing "dspace".
  5. More use of binary distributions means greater ease in upgrading DSpace.

It is clear that Spring has become quite dominant a solution in DSpace with the adoption of Apache Cocoon 2.2 for the Manakin XMLUI and Spring MVC for the Freemarker Prototype Webapplication currently under Development in the community.

This leads us to our first best practice:

Icon

Best Practice 1: If possible, do use Dependency Injection to acquire your services. This will be of benefit down the road if we make changes to the service manager architecture.

In creating classes that work within the system, the ServiceManager will inject those services you need for you. 

Where in our spring configuration we would register this Service via the following configuration:

Spring AutoWiring looks for other bean of our specific type elsewhere in our configuration and injects them into our service. This is the basic mechanism whereby Addon Modules can reuse existing services or even services provided by other third party modules without having to explicitly depend on any specific implementation of those services.

The DSpace Application Lifecycle

The life cycle of the container and the services therein is controlled by the Web-application context or the DSpace CLI ScriptLauncher main executable in which the servlet has been deployed. The Lifecycle assures that when the Service managers (and specifically Spring in this case) are initialized, the required core and addon services are wired and made available to your application.

The ServiceManager Request Cycle

The ServiceManager Request Cycle is similar to the Webapplication Request Cycle.  In the case of the Service Request, There is an incoming "Request" object with the state of the a call from the client, this Request is then enterpreted by the Container and mapped to a specific Servlet, which executes to completion, On Completion, the Serlvet Generates a

When a request is made by either the Webapplication or the CLI initialization, then the Request Lifecycle is engaged:

Basic Usage

To use the Framework you must begin by instantiating and starting a DSpaceKernel. The kernel will give you references to the ServiceManager and the ConfigurationService. The ServiceManager can be used to get references to other services and to register services which are not part of the core set. For standalone applications, access to the kernel is provided via the Kernel Manager and the DSpace object which will locate the kernel object and allow it to be used.

The DSpace launcher ([dspace]/bin/dspace) initializes a kernel before dispatching to the selected command.

The Service Manager Interface

Core Services

Configuration Service

ConfigurationService contributed to DSpace 1.7.1 (Service Manager Version 2.0.3) And maintains Parity with the existing DSpace ConfigurationManager in supporting "dspace.cfg" and modular "config/modules/[module].cfg" configuration.

The ConfigurationService controls the external and internal configuration of DSpace 2. It reads Properties files when the kernel starts up and merges them with any dynamic configuration data which is available from the services. This service allows settings to be updated as the system is running, and also defines listeners which allow services to know when their configuration settings have changed and take action if desired. It is the central point to access and manage all the configuration settings in DSpace.

Manages the configuration of the DSpace ServiceManager system. Can be used to manage configuration for any Service Bean within the ServiceManager

Acquiring the Configuration Service

The ConfigurationService API

Benefits over the Legacy DSpace ConfigurationManager

Type Casting and Array Parsing
  • Type Casting: Common Configuration Interface supports type casting of configuration values of the type required by the caller.
  • Array Parsing: As part of this type casting, the Configuration Service will split comma separated values for you when you request the property as type "Array"

  

Modular Default Configuration

[addon.jar]/config/[service].cfg 

Any service can provide sane defaults in a java properties configuration file. These properties will be able to be looked up directly using a prefix as syntax.

Example of Usage:

Modularization of Configuration Not Bound to API signature.

[dspace]/config/module/[prefix].cfg

Any service can provide overrides in the DSpace home configuration directory sane defaults in a java properties configuration file. These properties will be able to be looked up directly using a prefix as syntax.

Example of Usage:

In DSpace 1.7.0 enhanced capabilities were added to the ConfigurationManager to support the separation of of properties into individual files. The name of these files is utilized as a "prefix" to isolate properties that are defined across separate files from colliding.

Example of Usage:

Icon

Use commas for lists of values, use lookups (If you end up thinking you want to create maps in your properties, your doing it in the wrong place look instead at Spring Configuration and objectifying your configuration)

Icon

Objectifying Configuration... If you Configuration is too complex, then it probably should be an Object Model

Request Service

A request is an atomic transaction in the system. It is likely to be an HTTP request in many cases but it does not have to be. This service provides DSpace with a way to manage atomic transactions so that when a request comes in which requires multiple things to happen they can either all succeed or all fail without each service attempting to manage this independently.

In a nutshell this simply allows identification of the current request and the ability to discover if it succeeded or failed when it ends. Nothing in the system will enforce usage of the service, but we encourage developers who are interacting with the system to make use of this service so they know if the request they are participating in with has succeeded or failed and can take appropriate actions.

The DSpace Session Service

The Session represents a user's session (login session) in the system. Can hold some additional attributes as needed, but the underlying implementation may limit the number and size of attributes to ensure session replication is not impacted negatively. A DSpace session is like an HttpSession (and generally is actually one) so this service is here to allow developers to find information about the current session and to access information in it. The session identifies the current user (if authenticated) so it also serves as a way to track user sessions. Since we use HttpSession directly it is easy to mirror sessions across multiple servers in order to allow for no-interruption failover for users when servers go offline.

Icon

Do not pass Http Request or Session Objects in your code. Use Dependency Injection to make the RequestService, SessionService and Configuration Service Available in your Service Classes. Or use ServiceManager lookups if your work is out of scope of the ServiceManager.

DSpace Context Service COMING SOON

The DSpace Context Service is part of the DSpace Domain Model refactoring work and provides an easy means for any Service Bean to gain access to a DSpace Context object that is in scope for the current user request cycle. This Context will be managed by the ServiceManager RequestService and represents a means to maintain a "Transactional Envelope" for attaining "Atomic" changes to DSpace (Add Item, Update Item, Edit Metadata, etc).

DSpace Legacy DataSource Service COMING SOON

Similar to the Context Service, The DSpace Legacy DataSource Service is part of the Domain Model refactoring work and bring the preexisting DSpace DataSource instantiated within the the DSpace DatabaseManager into the Spring Application Context. The exposure of the DataSource will enable Service Beans in the DSpace ServiceManager to utilize popular tools for ORM such as Hibernate, JPA2, ibatis, Spring Templates, or your own custom persistence support to be used when developing your Services for DSpace.

Test Driven Development

Test-driven development (TDD) is a software development process that relies on the repetition of a very short development cycle: first the developer writes a failing automated test case that defines a desired improvement or new function, then produces code to pass that test and finally refactors the new code to acceptable standards. Kent Beck, who is credited with having developed or 'rediscovered' the technique, stated in 2003 that TDD encourages simple designs and inspires confidence.[http://en.wikipedia.org/wiki/Test-driven_development]

http://en.wikipedia.org/wiki/Test-driven_development#cite_note-Beck-0

We want to clarify that that the Testing Framework in the DSpace Services Module Predated the actual JUnit testing support that was added to dspace-api.  Testing is a very beneficial practice where the developer writes small java based test of the code they are going to produce.  The 

Test-driven development is related to the test-first programming concepts of extreme programming, begun in 1999,[2] but more recently has created more general interest in its own right.[3]

Using the Service Manager Testing Framework

DSpaceAbstractRequestTest

This is an abstract class which makes it easier to test execution of your service within a DSpace "Request Cycle" and includes an automatic request wrapper around every test method which will start and end a request, the default behavior is to end the request with a failure which causes a rollback and reverts the storage to the previous values

Other DSpace Resources on Spring and the Services Manager

Further Reading and Resources:

  • No labels

14 Comments

  1. I find that the ease with which Spring can be introduced to existing work depends on how you do it. Spring really wants to create your top-level objects and their main dependencies. If you try to sneak Spring in from the bottom up, you keep running into classes that Spring wants to instantiate when you'd rather leave that to later. It seems to work much more easily from the top down. Bringing Spring in seems to require an overall commitment to IoC and causes rework all up and down your control paths.

    There's a school of thought which says that the presence of classes with all static members indicates a problem. Maybe so. If so, we have this problem big-time and will need to re-think a lot of our code organization. I don't say we shouldn't.

  2. I find myself wondering why we need a ConfigurationService. Shouldn't we just ask the appropriate objects for the things they know? Why should configuration be an entity? Isn't it just properties of things which naturally are entities?

    For example: dspace.name, dspace.url, dspace.hostname and the like are about the overall DSpace instance or, looking at it from the code's POV, the Site. So shouldn't we just inject these into the Site, and if some code wants to know these things, ask the Site for them? There are a handful of classes which define the static structure of DSpace and it seems to me that they will tend to be natural repositories for configuration of their aspects of DSpace's operation.

    That doesn't mean that we are tied down to particular implementations of the "static structure" types. Spring can create a Site from a particular implementation, configure it, create and configure a DatabaseManager (and possibly wire it into Site), etc. at startup.

    It doesn't make sense to me to drink the IoC koolaid and then inject all of the simple-type dependencies into one God-object, to be called up later by the entities which logically should own them.

    1. MarkW,

      If we are going to want to create "runtime" vs "application" configuration capabilities, then that is where the ConfigurationService offers the benefit.

      Now, in the original DSpace 2.0 ConfigurationService,  there was an option to offer injection of basic primitive values into the Spring objects.  We found this to be a bit too overly "custom",  Ideally we want to keep the runtime property configuration separate from the fixed application configuration.

      I Do imagine sometime eventually where Spring Scopes might be utilized for application vs runtime.  But at the time that we designed this, we were attempting to maintain our own complete Domain Model for DSpace functionalities, Only using the features that were more common across IoC containers.  Now that we do focus more on Spring, I can see an opportunity to rely more on its featureset and ignore using others for now.

      Finally, I will add that the dialog that Graham and I shared about switching from the ServiceManager "managing" the Spring Application Context lifecycle, to one where its WebMVC that manages it is an important direction that we want to take the ServiceManager in. We really want to be "conventional" in our Spring usage and we want the power that WebMVC provides off the shelf for features like request, session, context "scopes", authentication/authorization and so on.

      1. I'm guessing that "runtime" here means site configuration while "application" means code assembly? That's a good point. Perhaps we don't really want to use Spring for site configuration, regardless how attractive its built-in XML parsing and object construction may be.

        Or maybe we need to split some of our entities into e.g. DatabaseModel (which gets configured by the site and holds particulars of the database connection) and DatabaseController (which gets configured at assembly time, is not easy to get at directly, and provides the behavior). You'd have to give specific name attributes to your Models so that they can be injected into the Controllers when you can't easily modify the Controllers' instantiations.

    2. That doesn't mean that we are tied down to particular implementations of the "static structure" types. Spring can create a Site from a particular implementation, configure it, create and configure a DatabaseManager (and possibly wire it into Site), etc. at startup.

      I believe we really want to utilize Spring Configuration for DataSources and database connectivity.  This will always really be "application" configuration, not runtime.

      See for example:

      http://scm.dspace.org/svn/repo/modules/dspace-database/trunk/impl/src/main/resources/spring/spring-dspace-core-services.xml

      Where we inject the DSpaceDataSource configured in DatabaseManager and reuse it directly in Spring tools for Database support. this will open the door to features such as:

      http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/jdbc.html

      http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/dao.html

      http://static.springsource.org/spring/docs/3.0.x/spring-framework-reference/html/spring-data-tier.html

      Peter Deitz, are you listening???

      1. Hmm, well, we already have DataSource injection, in a sense. DatabaseManager.initialize() tries to get a DataSource from JNDI and only looks at dspace.cfg if that fails. Tomcat contains a JNDI provider and either Server or Context can easily be configured to provide the DataSource you specify. I wrote a lightweight "ephemeral" JNDI provider (that I need to publish) which could do the same for commandline context.

        Could you be a little more explicit about the meaning of "application" vs. "runtime", please?

        1. Yes, and we can borrow that up into the Service Manager Spring config using a facotry method on DSpaceManaget.getDataSource()...  so, yea, thats fine for the moment....

          Application = Hardcoded Spring Configuration put into place before the application is started.

          Runtime = Configuration that is loaded either from properties files or database (we hope) that can, in theory be changed while the application is running.  If application code is respectful and looks these values up via the Configuration Service or Configuration Manager exactly when they are needed and does not store them locally during static initialization, in theory all these configuration value may be mutable at runtime.

          1. Proceeding tangentially, I think there are three people we need to consider. There's the person who administers a DSpace instance, who will be interested in runtime reconfiguration. There's the person who installs DSpace, who will be interested in static configuration (and who may not be the same as the first person, and may have different access to the host). And there's the person who creates DSpace and may want to pull the pieces together through external configuration. The second person wants installation configuration to be readily accessible to someone with access to the filesystem. The third person wants application-assembly configuration to be packed in archive containers (JAR, WAR) and not easily accessible, because when you are tweaking it generally this is because you are creating, destroying, and rearranging code. So this splits "application" configuration into install-time and assembly-time.

              1. person who administers a DSpace instance : Should get to do it in user interface or via configuration files, ideally the former and its state maintained in a db to survive upgrades without heavy developer lifting required (IE, they don't want to have to "restart" DSpace to change submission forms or manage enabled themes, or add metadata fields or enable/diable workflow steps
              2. person who installs DSpace : just wants to run something like "yum install dspace; yum install dspace-addon;" and not have to worry about anything else. (IE package and deploy a DSpace with sane defaults and good documentation for system admin config options.)
              3. person who creates DSpace : just wants to be able to test aggregations and various configurations from their IDE and in JUnit or continues integration (IE package these N addons and DSpace and launch with sane defaults)
  3. Apologies for going off at a tangent. In the section entitled Service Manager there is a code example...Example example =new DSpace().getSingletonService("my-example", Example.class);

    I don't think this is correct in that the actual method has a signature of...

    getSingletonService(java.lang.Class<T> type)

    Could someone that has a better understanding of the code than I possibly correct this.

    Cheers.

  4. A thing which I find difficult to do in Spring is to accumulate configuration of a single object across different files. So, for example, if we have a PluginManager, all the plugins will have to be wired into it in one place; you can't easily drop in a plugin with its own self-contained configuration including defining it to the PluginManager.

    One way to do this is to reverse the wiring: wire the PluginManager into each plugin and have said plugin call a registration method of the injected manager in order to be managed.

    1. Maybe I'm misunderstanding, but this is what the existing ServiceManager does, it aggregates all the addon spring configurations under one Application Context to produce one service endpoint to access them. All the services are not wired in one place, auto-wiring is utilized to inject services into one another when necessary, and this can support auto-wiring of collections of services or just selected individual services.

      As a tangent, the older way we are interacting directly witht eh service manager to add in UsageLogListeneres int he webapplications is not what I'm talking about, to understand what I'm talking about you need to be looking in the src/main/resources/spring of the individual addons and the dspace/config/spring directories to see where these configuration files are utilized to define the services that will be available.

      1. Ah, my lack of Spring experience may be showing. I have a deep and abiding suspicion of things like autowiring, and tend to overlook it. But if for example we have PluginManager.setPlugins(Map<Plugin>) then it looks like we could code <bean id='PluginManager'...autowire='byType'> and have lots of <bean id='FooPlugin'...> scattered wherever the framework will look for configuration files and it will wire them all into that Map. Convenient but still a little spooky.

        1. Yes, but I wouldn't do it "statically" from outside PluginManager, I would treat PluginManager like the way we treat "new DSpace()" as a "Helper" interface, just for getting to the beans that are "plugins" in the service Manager application context.

          in Pluginmanager we would do something like...

          In this manner, it would be stateless.  All the plugin definitions in the dspace.cfg would move into dspace/config/spring/... xml files or placed in similar files in src/main/resources/spring in separate addons.

          This means that we will be able to deploy plugins into DSpace simply by including the addon jar on the classpath, with no configuration required.

          Ideally I would percieve dspace.cfg configuration as subtractive.  Where you can place properites that signal if a plugin is enabled or not, is a consume is enabled or not, but does not define what class or bean is provider of the functionality.

          plugin.foo.enabled=false
          consumer.foo.enabled=false

          and this was a feature that Arron Zeckoski implemented in the original service manager...