Overview

Please review the upgrade notes carefully

Upgrading to VIVO 1.10.0 from a previous version of VIVO requires you to unload the data in your triple store and reload it, using tools provided. See Upgrading VIVO. You will also need to consider the impact that the dependency changes may have on your customizations and integrations. Preserving your customizations and integrations can be done using a Maven custom installer. See Preserving Customizations During Build

There are no ontology changes, and data produced by 1.10.0 is compatible with 1.6 - 1.9 (and vice versa). It is not required to upgrade to this release prior to subsequent releases. Please see Upgrading VIVO for more details.

What's New

Data Distribution API

The Data Distribution API uses configuration files you specify to create custom APIs of your design. You can specify the data to be returned, the addresses to be used, API parameters and formats of output. The Data Distribution API can be used to provide data for web sites, prepare reports, and much more. See Data Distribution API

Triple Pattern Fragments (TPF)

TPF presents a lightweight means of obtaining triples from a linked data application as a web service, with very low overhead, and high reliability (it only pattern matches for triples, there are no arbitrary complex queries, so individual requests can not have a high impact on the server). See Triple Pattern Fragments

Bootstrap Theme - "Tenderfoot"

Tenderfoot is a new, responsive theme for VIVO. Based on the work by Symplectic, it uses Bootstrap 3 to provide a view that scales better to different sizes of devices.

Argon 2i Password Hashing

VIVO now hashes passwords using Argon-2i replacing MD5. See Configuration Reference.

Branding and Theming Improvements

It is no longer necessary to modify or create a new theme if all you want to do is apply some local CSS and/or message customisations to your installation. By applying local customisations on top of a theme, it allows for the possibility to swap compatible themes whilst retaining your site definitions, and easier upgrades in the future, where you don't need to merge changes to templates in a theme.

For theme developers, the remaining JSP pages now render their body and are wrapped by the Freemarker page structure, so you don't need to maintain a separate JSP page structure, and ensure that the structure matches that in Freemarker. This allows more flexibility in the theme structure, without adversely affecting the JSP pages.

Multi-Lingual Improvements

The message lookups have been extended, so that application wide VIVO messages are distinct from Theme messages (and distinct from Vitro messages). It also allows for an additional "local" messages bundle, which overrides the theme, VIVO and Vitro layers.

Language packs can now be added to VIVO through the dependency mechanism, although you will still need to edit your runtime.properties to enable the languages in your UI.

ORCiD API v2

This release includes an updated ORCiD integration that can use the ORCiD v2 API. Note that ORCiD are planning to shut down the v1.x API endpoints.

Note that the configuration options have been changed, and you will need to update your runtime.properties.

The required properties are:

orcid.clientId = 0000-0000-0000-000X
orcid.clientPassword = 00000000-0000-0000-0000-000000000000
orcid.webappBaseUrl = http://localhost:8080/vivo
orcid.externalIdCommonName = VIVO Cornell Identifier
orcid.apiVersion = 2.0
orcid.api = sandbox

orcid.apiVersion is simply the version value (e.g. 1.2, 2.0), and orcid.api is just "release" (for the production API), and "sandbox" for the sandbox.

Direct2Experts Endpoint

For any sites wishing to participate in the Direct2Experts federated site - http://direct2experts.org/ - VIVO now includes the necessary endpoints. See Direct2Experts API for a description of the VIVO implementation in 1.10, and information regarding participation in this federated search site.

RDF 1.1 support

VIVO has now been updated to use Jena 3, which brings full RDF 1.1 support. As a result, internally all literals are treated as having a datatype - those with a language tag are rdf:langString, and any other are xsd:string, and the triple stores need to be reloaded to ensure that they have consistent internal representations. Only applications that directly open the triple stores using Jena libraries, or have SPARQL that explicitly references datatypes will be affected. Please see the upgrade notes for how to perform the migration and evaluate any impact.

In particular, if you are using VIVO Harvester, you will need to use a VIVO Harvester 2.x version.

Dependency Convergence and Vulnerability Elimination

Along with Jena 3 and Bootstrap, all the dependencies have been reevaluated and upgraded to ensure convergence.

In the back end, multiple conflicting versions of dependencies have been eliminated (e.g. httpclient, OSGi bundles), and all code now works with consistent versions of dependencies. Multiple JSON parsers have been removed, and all code now uses only Jackson.

Dependencies with known vulnerabilities, as determined by the Maven dependency-check plugin have been upgraded where a newer version exists.

In all, 38 dependencies have been removed, although 22 have been added due to unbundling the OSGi dependencies. Only 11 dependencies retrain the same version as VIVO 1.9.

In the front end, jQuery has been updated to support Bootstrap. D3.js has been upgraded to v4. All jQuery plugins have been updated to work with the updated jQuery.

One javascript library used to format the index page has been removed due to being GPL licensed, and replaced with an MIT licensed equivalent.

Beyond simple theming changes (e.g. logos, colours, text), local customisations may need to be upgraded for the dependency changes.

Vocabulary Services

AGROVOC has been updated to use a new API.

UMLS has been transitioned to the NIH service. In order to use this, you need to obtain an application key from NIH (free registration).

Reasoning Improvements

If SameAs reasoning is enabled (by default, this is disabled), the reasoner will now generate the correct vitro:mostSpecificType.

List View Query Improvements

List view configuration files can now include <precise-subquery></precisde-subquery> elements. Due to the way SDB works, OPTIONAL clauses are slow because they are evaluated independently, and then joined with the rest of the query restrictions. <precise-subquery> allows you to duplicate the external restrictions that the clause will be joined with. This results in much more efficient SQL queries.

By including the restrictons inside a <precise-subquery> element, we can eliminate the CONSTRUCT that would otherwise be used, making it easier to maintain the SELECT query.

For triple stores other than SDB, the <precise-subquery> element is filtered out, as it is often unnecessary to include these extra restrictions, and in some cases may hurt performance. However, the filtering can always be enabled or disabled globally via the runtime.properties.

Performance Improvements

Search results render faster if they contain Person results. Indexing time has also been improved.

Updating data via the UI is faster.

Graph URIs are now cached for triple stores using a Jena implementation - this is a significant difference for TDB triple stores.

Full handling of TDB type conversions, preventing isomorphic test failures that resulted in reloading filegraph on restart.

Servlet 3.0

VIVO already required the use of a Servlet 3.0 compatible version of Tomcat. The web.xml has now been upgraded to take advantage of the servlet 3.0 spec, which allows developers to use annotations for servlet configuration. If you have a customisation that adds a new servlet, you can enable it without modifying the web.xml file.

Java 9

The Maven projects and code have had updates to work with the latest Java JDK 9. Note that Java 9 is a very recent release, and has not been extensively tested with VIVO. Also, Java 9 removes support for endorsed dirs, so you need to use a compatible version of Tomcat.

Additional Error Checking During Builds

The Maven projects now integrate Google's Error Prone - http://errorprone.info/ - tool into the compilation to detect serious errors in the Java code. Any customisations and contributions will now automatically be checked, preventing many serious errors from entering the code base.

Testing Framework

The Selenium IDE tests have been updated to use specific named selectors, rather than positions. Additional attributes (domain and range for faux properties) have been added to the UI to allow for this.

This allows the tests to be run against both the old (wilma) and new (tenderfoot) themes, and will make the tests more robust in the event of future ontology changes.

Note that Selenium IDE no longer works with the current versions of Firefox. Whilst we can currently run the test suite using a Java project and WebDriver, we will need to consider how these tests can be maintained in the future.

Issues Resolved

Sub-task

[VIVO-1476] - Export and provide link of Jena 2 VIVO dataset for testing

Bug

[VIVO-855] - RDF export not working correctly
[VIVO-1060] - Add DumpRestoreController to Vitro web.xml
[VIVO-1310] - pom.xml implies that VIVO develop branch can still run under Java 1.7
[VIVO-1311] - Remove or replace the UMLS concept source
[VIVO-1324] - Security vulnerabilties in 1.9.x release
[VIVO-1342] - Ampersands (and possibly other characters?) not escaped in GraphML export of collaborator network graph
[VIVO-1394] - AboxRecomputer does not generate correct mostSpecificType for equivalent classes
[VIVO-1404] - Some SPARQL queries trigger Chrome XSS Auditor
[VIVO-1435] - Compliance with ORCID style guidelines
[VIVO-1461] - Assertions are ascribed to Vcard, but terms are not in the Vcard ontology
[VIVO-1465] - Jena2tools NullPointerException on read
[VIVO-1479] - Set version in pom.xml to 1.10.0-SNAPSHOT
[VIVO-1483] - VIVO hangs on startup after upgrading using jena3tools
[VIVO-1497] - Overview can not be manually edited
[VIVO-1517] - Jenatools expects package names instead of namespaces

New Feature

[VIVO-1252] - Incorporate Cornell's DataDistributor API into core Vitro.
[VIVO-1312] - Implement Linked Data Fragments
[VIVO-1335] - Create Bootstrap Theme

Task

[VIVO-812] - Automate the process of adjusting documentation to the release
[VIVO-1316] - Place external lookup base architecture in Vitro layer
[VIVO-1486] - Update mysql-connector-java dependency in jenatools to 8.0.11
[VIVO-1520] - Deploy jenatools 1.1.2 to Maven Central and increment version in Vitro pom.xml

Improvement

[VIVO-1063] - Update included jQuery library
[VIVO-1246] - Improve the ConfigurationBeanLoader
[VIVO-1247] - Remove duplicate code, based on improvements in ConfigurationBeanLoader
[VIVO-1248] - Add functionality to the edu.cornell.mannlib.vitro.webapp.utils.sparql package
[VIVO-1260] - Make http.createCacheHeaders true by default
[VIVO-1270] - Update Jena to latest release
[VIVO-1272] - Inject JSP content to Freemarker, instead of having secondary layout
[VIVO-1273] - Update DOI URL Schema to follow recomendations from Crossref
[VIVO-1290] - Improve Multi-Lingual Support
[VIVO-1294] - Language values (all.properties) should not be part of theme
[VIVO-1307] - Remove dom4j from the project
[VIVO-1309] - Update DWR to more recent version
[VIVO-1317] - Reduce the number of JSON libraries in the dependencies
[VIVO-1318] - In the ORCID client code, use Jackson library to handle JSON data
[VIVO-1319] - Remove dependency on sourceforge.net JSON parser.
[VIVO-1367] - AGROVOC external service not working
[VIVO-1373] - VIVO 1.10 Beta 2 does not support FROM and FROM NAMED queries
[VIVO-1375] - Upgrade to Servlet 3.0 spec and annotations
[VIVO-1376] - Add smoke test to ensure that there are no XSD:Strings in SDB
[VIVO-1377] - Add output format selection to jenatools
[VIVO-1381] - Upgrade Solr to 4.10.4
[VIVO-1382] - Update JFact dependency
[VIVO-1383] - Update pooling libraries
[VIVO-1384] - Use commons-lang3 throughout
[VIVO-1385] - Replace unmaintained CSV parser with commons-csv
[VIVO-1386] - Make consistent use of HttpClient 4.5, remove conflicting dependencies
[VIVO-1387] - Update all dependencies with known vulnerabilities to latest versions
[VIVO-1393] - Replace isotope jQuery plugin
[VIVO-1397] - Improve performance and reliability of search indexing
[VIVO-1400] - Have optional "precise subquery" elements in list views for triple stores that perform better with more selective queries
[VIVO-1403] - Improve update performance
[VIVO-1405] - Defeat browser cacheing for new versions of JavaScript and CSS files.
[VIVO-1406] - Visualisations in a multi-lingual release
[VIVO-1408] - ConifigurationBeanLoader should recognize RDF namespaces in java: URIs
[VIVO-1438] - Some text on Forms are made of different strings associated following english syntax
[VIVO-1447] - Organize the ontology files. Produce a vivo.owl
[VIVO-1448] - Move password encryption from MD5 to a salted hash
[VIVO-1458] - Update to Jena 3.6
[VIVO-1463] - Update ontologies.owl
[VIVO-1470] - Improve cross-platform support in build
[VIVO-1478] - MySQL driver throws InvalidConnectionAttributeException
[VIVO-1487] - Improve VIVO home page text -- replace scientists with scholars
[VIVO-1488] - Read runtime.properties from {vivo-home}/config
[VIVO-1490] - Missing '.' in aboutMapOfScience_en_US.ftl

Test

[VIVO-1410] - Release the next version of VIVO and Vitro
[VIVO-1496] - Add OpenVIVO and UF data to sample-data

Code Task

[VIVO-1523] - Enable retrieval of SNAPSHOT dependencies

Documentation

[VIVO-31] - Improve documentation of Google Analytics
[VIVO-34] - Make it easy to do "next", "previous" and "up" links in a Confluence page
[VIVO-242] - Establish Wiki versioning process
[VIVO-772] - Write "Preparing for Production"
[VIVO-813] - Improve document details
[VIVO-917] - Create a confluence macro to show sections numbers for multi-page documents
[VIVO-1274] - Improve documentation process for next release
[VIVO-1334] - Create a recommendation for the use of string and langString
[VIVO-1351] - Write "TPF Endpoint" for Tech doc
[VIVO-1352] - Complete "Using VIVO"
[VIVO-1464] - Identify and separate candidate ontologies for removal
[VIVO-1473] - Write "VIVO for Data Analysts"
[VIVO-1485] - Add jenatools documentation to the tech wiki
[VIVO-1498] - Update Deploying Additional Ontologies
[VIVO-1531] - Write Direct2Experts API
[VIVO-1534] - Document steps for using UMLS vocabulary service

Release Managers

Ralph O'Flinn, University of Alabama, Birmingham

Andrew Woods, DuraSpace

Contributors

Qazi Azim Ijaz Ahmad, TIB Hannover

Sabih Ali, Digital Science

Martin Barber, TIB Hannover

Jim Blake, Cornell

Mike Conlon, University of Florida

Don Elsborg, University of Colorda

Kitio Fofack, Université du Québec à Montréal

Benjamin Gross, UNAVCO / Clarivate Analytics

Christian Hauschke, TIB Hannover

Violeta Ilik, Columbia University

Huda Khan, Cornell

Ted Lawless, Clarivate Analytics / Brown University

Jacob Levernier, University of Pennsylvania

Brian Lowe, Ontocale

Jose Luis Martin, UC3M

Steve McKay, Plum Analytics

Javed Muhammed, Cornell University