...

This document describes the technical aspects of the testing integrated into Dspace. In it we describe the tools used as well as how to use them and the solutions applied to some issues found during development. It's intended to serve as a reference for the community so more test cases can be created.

Issues Found

During implementation we found several issues, which are described in this section, along the solution implemented to work around them. 

...

To achieve this we mock DatabaseManager and we replace the connector to point to our in-memory database. In this class we also initialise the replica with the proper data.

Structure

There is a base class called "AbstractUnitTest". This class contains a series of mocks and references which are necessary to run the tests in DSpace, like mocks of the DatabaseManager object. All Unit Tests should inherit this class, located under the package "org.dspace" in the test folder of DSpace-test.

About the implementation, several objects only offer a hidden constructor and a factory method to create an instance of the object. This means we effectively have to create them using available factory methods. Other specifics have been commented above, like:

...

Due to the Dspace Maven structure discussed in previous sections, all the tests belonging to any module (dspace-api, dspace-xmlui-api, etc) must be stored in the module dspace-test. This module enables us to apply common configuration, required by all tests, in a single area thus avoiding duplication of code. Related to this point is the requirement for Dspace to run using a database and a certain file system structure. We have created a base class that initializes this structure via a in-memory database (using H2) and a temporary copy of the required file system.

The described base class is called "AbstractUnitTest". This class contains a series of mocks and references which are necessary to run the tests in DSpace, like mocks of the DatabaseManager object. All Unit Tests should inherit this class, located under the package "org.dspace" in the test folder of dspace-test. There is an exception with classes that originally inherit DSpaceObject, its tests should inherit AbstractDSpaceObjectTest class.

Several mocks are used in the tests. The more relevant ones are:

• MockDatabaseManager: a mock of the database manager that launches H2 instead of PostgreSQL/Oracle and creates the basic structure of tables for DSpace in memory
• MockBrowseCreateDAOOracle: due to the strong link between DSpace and the databases, there are some classes that have specific implementations if we are using Oracle or PostgreSQL, like this one. In this case we've had to create a mock class that overrides the functionality of MockBrowseCreateDAOOracle so we are able to run the Browse related tests.

You may need to create new mocks to be able to test certain areas of code. Creation of the Mock goes beyond the scope of this document, but you can see the mentioned classes as an example. BAsically it consists on adding annotations to a copy of the existing class to indicate a method is a mock of the original implementation and modifying the code as required for our tests.

Limitations

The solution to copy the file system is not a very elegant one, so we appreciate any insight that can help us to replicate the required files appropriately.

The fact that we load the tests configuration from a dspace-test.cfg file means we are only testing the classes against a specific set of configurations. We probably would like to have tests that runs with multiple settings for the specific piece of code we are testing. This will require some extra classes to modify the configuration system and the way this is accessed by DSpace.

How to build new tests

To build a new Unit Test, create the corresponding class in the project dspace-test, under the test folder, in the package where the original class belongs. Tests for all the projects (dspce-api, dspace-jspui-api, etc) are stored in this project, to avoid duplication of code. Name the class following the format <OriginalClass>Test.java.

There are some common imports and structure, you can use the following code as a template:

package org.dspace.content;//Add DSpace licensing here at the top!
package org.dspace.content;

import java.sql.SQLException;
import org.dspace.core.Context;
import org.junit.*;
import static org.junit.Assert.* ;
import static org.hamcrest.CoreMatchers.*;
import mockit.*;
import org.apache.log4j.Logger;
import org.dspace.core.Constants;

/**
 * Unit Tests for class <OriginalClass>Test
 * @author you name
 */
public class <OriginalClass>Test extends AbstractUnitTest
{

    /** log4j category */
    private static final Logger log = Logger.getLogger(<OriginalClass>Test.class);

    /**
     * <OriginalClass> instance for the tests
     */
    private <OriginalClass> c;

    /**
     * This method will be run before every test as per @Before. It will
     * initialize resources required for the tests.
     *
     * Other methods can be annotated with @Before here or in subclasses
     * but no execution order is guaranteed
     */
    @Before
    @Override
    public void init()
    {
        super.init();
        try
        {
            //we have to create a new community in the database
            context.turnOffAuthorisationSystem();
            this.c = <OriginalClass>.create(null, context);


            //we need to commit the changes so we don't block the table for testing
            context.restoreAuthSystemState();
            context.commit();
        }
        catch (AuthorizeException ex)
        {
            log.error("Authorization Error in init", ex);
            fail("Authorization Error in init");
        }
        catch (SQLException ex)
        {
            log.error("SQL Error in init", ex);
            fail("SQL Error in init");
        }
    }

    /**
     * This method will be run after every test as per @After. It will
     * clean resources initialized by the @Before methods.
     *
     * Other methods can be annotated with @After here or in subclasses
     * but no execution order is guaranteed
     */
    @After
    @Override
    public void destroy()
    {
        c = null;
        super.destroy();
    }

    /**
     * Test of XXXX method, of class <OriginalClass>
     */
    @Test
    public void testXXXX() throws Exception
    {
        int id = c.getID();
        <OriginalClass> found =  <OriginalClass>.find(context, id);
        assertThat("testXXXX 0", found, notNullValue());
        assertThat("testXXXX 1", found.getID(), equalTo(id));
        assertThat("testXXXX 2", found.getName(), equalTo(""));
    }

   [... more tests ...]
}

The sample code contains common imports for the tests and common structure (init and destroy methods as well as the log). You should add any initialization required for the test in the init method, and free the resources in the destroy method. 

The sample test shows the usage of the assertThat clause. This clause (more information in JUnit help) allows you to check for condition that, if not true, will cause the test to fail. We name every condition via a simple schema (method name plus an integer indicating order) as the first parameter. This allows you to identify which specific assert if failing whenever a test returns an error.

Please be aware methods init and destroy will run once per test, which means that if you create a new instance every time you run init, you may end up with several instances in the database. This can be confusing when implementing tests, specially when using methods like findAll.

If you want to add code that it's executed once per test class, edit the parent AbstractUnitTest and its methods initOnce and destroyOnce. Be aware these methods contain code used to recreate the structure needed to run DSpace tests, so be careful when adding or removing code there. Our suggestion is to add code at the end of initOnce and at the beginning of destroyOnce, to minimize the risk of interferences between components.

Be aware that tests of classes that extend DSpaceObject should extend AbstractDSpaceObjectTest instead due to some extra methods and requirements implemented in there.

How to run the tests

The tests can be activated using the commands

mvn package -Dmaven.test.skip=false  //builds DSpace and runs tests

  or
mvn test -Dmaven.test.skip=false     //just runs the tests

or by changing the property "activeByDefault" at the profile (skiptests) in the main pom.xml file, at the root of the project and then running

mvn package  //builds DSpace and runs tests
  or
mvn test     //just runs the tests

...

Integration Tests

These tests work at the API level and test the interaction of components within the system. Some examples, placing an item into a collection, or creating a new metadata schema and adding some fields. Primarily these tests operate at the API level ignoring the interface components above it.

...

The integration tests also make use of ContiPerf to evaluate the performance of the system. We believe it doesn't make sense to add this layer to the unit tests, as they are tested in isolation and we care about performance not on individual calls but on certain tasks that can only be emulated by integration testing.

Structure

Integration tests use the same structure as Unit tests. A class has been created, called AbstractIntegrationTest, that inherits from AbstractUnitTest. This provides the integration tests with the same temporal file system and in-memory database as the unit tests. The class AbstractIntegrationTest is created just in case we may need some extra scaffolding for these tests. All integration tests should inherit from it to both distinguish themselves from unit tests and in case we require specific changes for them.

...

• Integrating with a Quality Management tool like Sonar
• Integrating with a Continuous Integration tools 
• Adding Unit and Integration tests for the remaining classes
• Extending Functional tests
• Creating a "Code quality" release, where priority is not new functionalities but stability and quality of code

Thanks

This page has been created with help from Stuart Lewis, Scott Phillips and Gareth Waller. I want to thank them all for their comments. Some information has been taken from Wikipedia to make the text more complete. I'm to blame for errors in the text.

...