Summary

This document presents the process required to develop a new language to be integrated into VIVO and Vitro. It is intended for any developer who wishes to develop a new integrated language. The development process requires a good knowledge of the VIVO internal structure and its internal working mechanisms. This document describes the development cycle of new languages as well as the components required for this development.

The project of adding a new language requires diverse and varied skills. Therefore, it requires the involvement of a multidisciplinary team composed of software developers, linguists and ontologists. It is therefore necessary to allow a few days or even a few weeks, depending on the computer skills of the team, to complete all the iterations presented in this guide. All team members must be able to master the development cycle presented below, as well as the IT actions associated with each stage of the cycle. It will therefore be necessary to offer adequate IT support to the non-IT personnel working on the translation of the texts contained in the various files. To simplify this work, we strongly recommend the use of UQAM-DEV R.3 : An Interoperable UQAM Development Environment for VIVO (UQAM-DEV) which has been specially designed to carry out this type of project. However, in order to preserve the generality of the presented solution, we have chosen to present the method in a Ubuntu Linux development context.

To illustrate the process of integrating new languages, we will give examples of how the French Canadian linguistic context is integrated into VIVO.

Methodoligical overview

The method developed to integrate a new linguistic context into VIVO is an Agile one that involves a development cycle punctuated by evolutionary iterations whose purpose is the introduction of new language into VIVO. Source code control aspects are deliberately not dealt with in this methodology in order to leave each team free to deal with this subject according to the governance rules specific to the institution leading the project.

Development Cycle

The figure below shows the five generic steps in the cycle of integrating a new language into VIVO that are performed at each iteration.

Vivo's internationalization process is a five-step cycle:

Step 1) At the beginning of the cycle, the first step is to compile, deploy, and run the Vivo instance on the local computer.
Step 2) The second step is to test and identify problems related to internationalization.
Step 3) If everything works properly this ends the development cycle.
- When a problem is identified it is necessary to search the code to find the problem.
- This involves doing a lot of pattern managing search activities.
Step 4) In fourth place, comes the stage of correcting the problem.
- Once the problem has been identified, the professional must edit the concerned file to make correction.
Step 5) In step five certain databases need to be initialized depending on the type of correction made.
- Then back to step one of recompiling and running vivo.

Development iteration

The development of VIVO's internationalization is an iterative process that requires several iterations before full internationalization is achieved. Each iteration runs through the development cycle we discussed earlier. In the following section we will present the different iterations by demonstrating the internationalization of the French Canadian. Each iteration will contain a section describing the purpose of the iteration, a section demonstrating each stage of the development cycle, and in conclusion a summary explaining the highlights of the iteration.

Iteration 1: Preparing and initializing the development environment

Goal:

This itération consists in setting up the software elements necessary to run an VIVO instance on a local host and prepare directories to undertake the task of internationalizing VIVO's content.

At the end of this iteration all software and data components will be in place to start the process of translating the files into the target language. French Canadian will be used as the demonstration language.

Item A) VIVO's software infrastructure installation

A.1 Linux configuration

Assume the configuration of the following environment variables

Example for Linux

export GIT_HOME=/opt/tomcat/GIT
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export SOLR=/opt/solr
export TOMCAT_HOME=/opt/tomcat
export VIVO_HOME=/opt/tomcat/vivo_i18n/home

First it is necessary to install Java, Maven, GIT, and SOLR. If necessary, you will find instructions at the following link: Starting with basic installation: Java-Maven-Solr-GIT

Secondly, it is essential to install the Apache Tomcat application server. You will find the necessary instructions for its installation at the following link: Installing Tomcat 8.5 for Vivo-1.11.0-i18n

A.2 UQAM-DEV

An alternative VIVO deployment management solution is also available. You can see it here: UQAM-DEV R.3 : An Interoperable UQAM Development Environment for VIVO (UQAM-DEV)

Item B) Configuring the source code workspace and extracting the code

Step

Action

1) Creating the GIT directory

Whether under Windows or on Unix it is necessary to create a space in which it is possible to work. As proposals we suggest creating a directory called under the Tomcat root directory. In the case of users and Unix we suggest to carry out the operations under the tomcat username. This precaution facilitates the management of credentials associated with development under Linux.

sudo -i -u tomcat # sudo in Linux - not on Windows
mkdir $TOMCAT_HOME/GIT # $TOMCAT_HOME is predefined at the installation of Tomcat
cd $TOMCAT_HOME/GIT

2) Retrieving VIVO the code

To VIVO internationalization, it is necessary to extract the four following GitHub repo:

git clone https://github.com/vivo-project/Vitro.git Vitro -b branch_name_to_be_coming
git clone https://github.com/vivo-project/VIVO.git VIVO -b branch_name_to_be_coming
git clone https://github.com/vivo-project/VIVO-languages VIVO-languages -b branch_name_to_be_coming
git clone https://github.com/vivo-project/Vitro-languages Vitro-languages -b branch_name_to_be_coming

3) Retrieving UQAM Vivo-installer-i18n

The installer makes it easy to manage the compilation and deployment of the VIVO instance.

git clone https://github.com/UQAM-SB/Vivo-installer-i18n

Item C) Configuring the VIVO search accelerator (SOLR)

Step	Action
1) Configuring SOLR	follow the configuration steps available at this link: Configure and Start Solr
2) Check vivocore in Solr	sudo systemctl start solr http://localhost:8983/solr/#/vivocore_i18n/core-overview

Item D) Preparing VIVO for first start

Step	Action
1) Create VIVO home	sudo -i -u tomcat # for Linux user mkdir -p $VIVO_HOME cp -r $TOMCAT_HOME/GIT/vivo-uqam-i18n/vivo-project-i18n-config/vivo_i18n/home/* $VIVO_HOME/home/
2) Edit runtime.properties file Make sure that attribute properties fit with current installation	sudo -u tomcat vi $VIVO_HOME/config/runtime.properties runtime.properties Vitro.defaultNamespace = http://<your-server-ip-address>:8080/vivo_i18n/individual/ ... rootUser.emailAddress = YOUR_ADRESS rootUser.passwordChangeRequired = false rootUser.password = YOUR-PASSWORD ... vitro.local.solr.url = http://<server-ip-address>:8983/solr/vivocore_i18n ... RDFService.languageFilter = true languages.selectableLocales = en_US, de_DE
3) Edit applicationSetup.n3 for TDB data persistence	applicationSetup.n3 # UNCOMMENT :tdbContentTripleSource a vitroWebapp:triplesource.impl.tdb.ContentTripleSourceTDB , vitroWebapp:modules.tripleSource.ContentTripleSource ; # May be an absolute path, or relative to the Vitro home directory. :hasTdbDirectory "tdbContentModels" . :sparqlContentTripleSource a vitroWebapp:triplesource.impl.sparql.ContentTripleSourceSPARQL , vitroWebapp:modules.tripleSource.ContentTripleSource ; # The URI of the SPARQL endpoint for your triple-store. :hasEndpointURI "http://localhost:3030/vivo/query" ; # The URI to use for SPARQL UPDATE calls against your triple-store. :hasUpdateEndpointURI "http://localhost:3030/vivo/update" . COMMENT #:sdbContentTripleSource # a vitroWebapp:triplesource.impl.sdb.ContentTripleSourceSDB , # vitroWebapp:modules.tripleSource.ContentTripleSource . #REPLACE :application a vitroWebapp:application.ApplicationImpl , vitroWebapp:modules.Application ; :hasSearchEngine :instrumentedSearchEngineWrapper ; :hasSearchIndexer :basicSearchIndexer ; :hasImageProcessor :iioImageProcessor ; :hasFileStorage :ptiFileStorage ; :hasContentTripleSource :tdbContentTripleSource ; # :hasContentTripleSource :sdbContentTripleSource ; # :hasConfigurationTripleSource :tdbConfigurationTripleSource ; :hasConfigurationTripleSource :tdbConfigurationTripleSource ; :hasTBoxReasonerModule :jfactTBoxReasonerModule .
4) Edit and configure VIVO installation setting file Check that the file contains the following code:	cd $GIT_HOME/Vivo-installer-i18n sudo -u tomcat cp example_settings_i18n_linux.xml settings_i18n_linux.xml sudo -u tomcat vi settings_i18n_linux.xml settings_i18n_uqam.xml <settings xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd"> <profiles> <profile> <id>defaults</id> <properties> <app-name>vivo_i18n</app-name> <vivo-dir>/opt/tomcat/vivo_i18n/home</vivo-dir> <tomcat-dir>/opt/tomcat</tomcat-dir> <default-theme>wilma</default-theme> </properties> </profile> </profiles> <activeProfiles> <activeProfile>defaults</activeProfile> </activeProfiles> </settings>

Item E) Compiling VIVO

This activity consists in making a first compilation of VIVO i18n.

Step

Action

1) check the state of the Git directory before starting the compilation

make sure that all directories are present and are all at the same hierarchical level in Git directories

Command

cd $GIT_HOME
ls -al

Result

drwxrwxr-x ... Vitro
drwxrwxr-x  ... Vitro-languages
drwxrwxr-x  ... VIVO
drwxrwxr-x  ... Vivo-installer-i18n
drwxrwxr-x  ... VIVO-languages

2) start compiling

Before starting the compilation make sure you have correctly configured the settings_i18n_linux.xml file as described above. The -DskipTests option is optional.

Assurez-vous que le fichier war est installé dans le répertoire approprié de tomcat

Command

cd Vivo-installer-i18n
mvn -s settings_i18n_linux.xml -DskipTests=true install

At the end you should see a compilation end looking like this:

Result

[INFO] 
[INFO] --- maven-dependency-plugin:2.10:unpack (install) @ vivo-installer-vivo ---
[INFO] Configured Artifact: org.vivoweb:vivo-installer-vivo:1.11.2-SNAPSHOT:war
[INFO] Unpacking /opt/tomcat/GIT/Vivo-installer-i18n/webapp/target/vivo.war to /opt/tomcat/webapps/vivo_i18n with includes "" and excludes ""
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for VIVO Installer for i18n 1.11.2-SNAPSHOT:
[INFO] 
[INFO] Vitro Home ......................................... SUCCESS [ 2.409 s]
[INFO] VIVO Home .......................................... SUCCESS [ 1.453 s]
[INFO] VIVO Languages for Home en_US ...................... SUCCESS [ 0.340 s]
[INFO] VIVO Languages for Home de_DE ...................... SUCCESS [ 0.036 s]
[INFO] VIVO Languages for Home es ......................... SUCCESS [ 0.034 s]
[INFO] VIVO Languages for Home pt_BR ...................... SUCCESS [ 0.027 s]
[INFO] VIVO Languages for Home en_CA ...................... SUCCESS [ 0.024 s]
[INFO] VIVO Languages for Home CORE ....................... SUCCESS [ 0.105 s]
[INFO] VIVO Installer for i18n ............................ SUCCESS [ 0.151 s]
[INFO] Vitro Install Home for i18n ........................ SUCCESS [ 1.382 s]
[INFO] Vitro Dependencies ................................. SUCCESS [ 0.447 s]
[INFO] Vitro API .......................................... SUCCESS [ 1.295 s]
[INFO] Vitro Web App ...................................... SUCCESS [ 0.882 s]
[INFO] Vitro Languages for Web App en_US .................. SUCCESS [ 0.313 s]
[INFO] Vitro Languages for Web App de_DE .................. SUCCESS [ 0.064 s]
[INFO] Vitro Languages for Web App es ..................... SUCCESS [ 0.042 s]
[INFO] Vitro Languages for Web App pt_BR .................. SUCCESS [ 0.051 s]
[INFO] Vitro Languages for Web App en_CA .................. SUCCESS [ 0.099 s]
[INFO] Vitro Languages for Web App CORE ................... SUCCESS [ 0.212 s]
[INFO] VIVO API ........................................... SUCCESS [ 0.379 s]
[INFO] VIVO Web App ....................................... SUCCESS [ 1.146 s]
[INFO] VIVO Languages for Web App en_US ................... SUCCESS [ 0.027 s]
[INFO] VIVO Languages for Web App de_DE ................... SUCCESS [ 0.104 s]
[INFO] VIVO Languages for Web App es ...................... SUCCESS [ 0.039 s]
[INFO] VIVO Languages for Web App pt_BR ................... SUCCESS [ 0.034 s]
[INFO] VIVO Languages for Web App en_CA ................... SUCCESS [ 0.026 s]
[INFO] VIVO Languages for Web App CORE .................... SUCCESS [ 0.085 s]
[INFO] VIVO WebApp Install dependencies for i18n .......... SUCCESS [ 0.002 s]
[INFO] VIVO Install Web App for i18n ...................... SUCCESS [ 4.094 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15.851 s
[INFO] Finished at: 2020-09-23T14:50:52-07:00
[INFO] ------------------------------------------------------------------------

Item F) Preparing sample data

This activity is the first step in internationalization. It involves creating an internationalized test data file in the target language that will be used to assess the quality of the internationalization of the different VIVO modules.

Step

Action

1) access the sample file

Extract the sample file from GitHub

git clone https://github.com/vivo-project/sample-data.git  -b branch_name_to_be_coming

2) build a translated file in the target language

This step consists of copying language files from the directory is in the target language and translating the content into the new language and associating the appropriate linguistic tag to it.

Notes: Make sure your file is in UTF-8 format.

cd $GIT_HOME/sample-data/i18n
cp sample-data-en_US.ttl sample-data-fr_CA.ttl
edit sample-data-fr_CA.ttl

#- sample change

Change from

sample-data:n1246
rdfs:label "ENC 1114 -- Introduction to Rhetoric"@en-US ;
.

Change to

sample-data:n1246
rdfs:label "ENC 1114 -- Intruction à la Rhétorique"@fr-CA ;
.

3) Install the files so that they are processed each time VIVO is started.

Change to

cp sample-data.ttl sample-data-en_US.ttl sample-data-fr_CA.ttl $VIVO_HOME/rdf/abox/filegraph

Item G) Starting VIVO

Step

Action

1) Start TOMCAT

After making the appropriate configurations move to the TOMCAT executables directory and launch the application.

cd $TOMCAT_HOME/bin
./catalina.sh jpda start

2) Monitoring the execution

after running the "taill -f" command you can do ^C to stop the Monitoring

cd $TOMCAT_HOME/log
tail - f 


.createCacheHeaders=true, languages.selectableLocales=en_US, de_DE, orcid.api=sandbox, orcid.apiVersion=2.0, orcid.clientId=APP-MWPTQ7Z850AY2GCH, orcid.clientPassword=5eeb8f1e-05fd-4aa3-af3f-3bf12911f683, orcid.externalIdCommonName=Universitï¿½ du Quï¿½bec ï¿½ Montrï¿½al, orcid.webappBaseUrl=http://locahost:8080/vivo/, proxy.eligibleTypeList=http://xmlns.com/foaf/0.1/Person, http://xmlns.com/foaf/0.1/Organization, rootUser.emailAddress=vivo@uqam.ca, rootUser.password=Vivo2435...., rootUser.passwordChangeRequired=false, rp.multiple=config, selfEditing.idMatchingProperty=http://localhost:8080/ns#networkId, , vitro.local.solr.url=http://localhost:8983/solr/vivocore_i18n}]
2020-09-23 15:32:27,278 INFO [RevisionInfoBean] Revision info [build date: 20-09-23 22 h 18, level info: [[VIVO, 1.11.2-SNAPSHOT, d3e9606]]]
2020-09-23 15:32:27,321 WARN [StartupStatus] From FreemarkerEmailFactory$Setup: Invalid SMTP host: 'smtp.mydomain.edu': host name is not recognized - Email functions are disabled.
2020-09-23 15:32:27,383 INFO [RDFFilesLoader] Loading rdf/display/everytime/dataGetterLabels.n3
2020-09-23 15:32:27,384 INFO [RDFFilesLoader] Loading rdf/display/everytime/displayModelListViews.rdf
2020-09-23 15:32:27,386 INFO [RDFFilesLoader] Loading rdf/display/everytime/homePageDataGetters.n3
2020-09-23 15:32:27,388 INFO [RDFFilesLoader] Loading rdf/display/everytime/localeSelectionGUI.n3
2020-09-23 15:32:27,389 INFO [RDFFilesLoader] Loading rdf/display/everytime/n3ModelChangePreprocessors.n3
2020-09-23 15:32:27,390 INFO [RDFFilesLoader] Loading rdf/display/everytime/orcidInterfaceDataGetters.n3
2020-09-23 15:32:27,392 INFO [RDFFilesLoader] Loading rdf/display/everytime/pageList.n3
2020-09-23 15:32:27,393 INFO [RDFFilesLoader] Loading rdf/display/everytime/permissions.n3
2020-09-23 15:32:27,399 INFO [RDFFilesLoader] Loading rdf/display/everytime/searchIndexerConfigurationVitro.n3
2020-09-23 15:32:27,404 INFO [RDFFilesLoader] Loading rdf/display/everytime/searchIndexerConfigurationVivo.n3
2020-09-23 15:32:27,410 INFO [RDFFilesLoader] Loading rdf/display/everytime/vitroSearchProhibited.n3
2020-09-23 15:32:27,411 INFO [RDFFilesLoader] Loading rdf/display/everytime/vivoConceptDataGetters.n3
2020-09-23 15:32:27,414 INFO [RDFFilesLoader] Loading rdf/display/everytime/vivoListViewConfig.rdf
2020-09-23 15:32:27,426 INFO [RDFFilesLoader] Loading rdf/display/everytime/vivoOrganizationDataGetters.n3
2020-09-23 15:32:27,431 INFO [RDFFilesLoader] Loading rdf/display/everytime/vivoQrCodeDataGetter.n3
2020-09-23 15:32:27,434 INFO [RDFFilesLoader] Loading rdf/display/everytime/vivoSearchProhibited.n3
2020-09-23 15:32:27,440 INFO [RDFFilesLoader] Loading rdf/displayTbox/everytime/displayTBOX.n3
2020-09-23 15:32:27,450 INFO [RDFFilesLoader] Loading rdf/displayDisplay/everytime/displayDisplay.n3
2020-09-23 15:32:27,453 INFO [RDFFilesLoader] Loading rdf/auth/everytime/permission_config.n3
2020-09-23 15:32:27,464 INFO [RDFFilesLoader] Loading rdf/tbox/everytime/initialTBoxAnnotations_de_DE.nt
2020-09-23 15:32:27,490 INFO [RDFFilesLoader] Loading rdf/tbox/everytime/initialTBoxAnnotations_en_US.nt
2020-09-23 15:32:28,738 INFO [FileGraphSetup] Read 12 abox file graphs
2020-09-23 15:32:29,173 INFO [FileGraphSetup] Read 6 tbox file graphs

3) Observing the execution

After logging in to VIVO and passing the smocktest you should see a presentation similar to this one. If you don't observe people in the faculties make sure that the example files are copied to the right place as indicated above or that Solr has been started

.

Connect to http://localhost:8080/vivo_i18n

Iteration summary

The objective of this iteration was to set up the software environment necessary for the integration of the new language. At this point an instance of TOMCAT, Solr, as well as the deployment of the en_US version of VIVO and an example file are fully operational and executable.

Iteration 1: Create the directory structure of the target language context

Goal

The objective of this iteration is to set up the directory structure to host the files needed to integrate the new linguistic context in VIVO. At the end of the iterations, the execution of VIVO will allow the texts to be presented according to the chosen linguistic context.

Step 1) Compile and deploye a new/corrected instance of VIVO and start it

At this stage it is necessary to stop the execution of TOMCAT by running the following commands

cd $TOMCAT_HOME/bin
./catalina.sh jpda stop

Page tree

How a Developer Add New Languages

Summary

Methodoligical overview

Development Cycle

Development iteration

Iteration 1: Preparing and initializing the development environment

Goal:

Item A) VIVO's software infrastructure installation

A.1 Linux configuration

A.2 UQAM-DEV

Item B) Configuring the source code workspace and extracting the code

1) Creating the GIT directory

2) Retrieving VIVO the code

3) Retrieving UQAM Vivo-installer-i18n

Item C) Configuring the VIVO search accelerator (SOLR)

1) Configuring SOLR

2) Check vivocore in Solr

Item D) Preparing VIVO for first start

1) Create VIVO home

2) Edit runtime.properties file

3) Edit applicationSetup.n3 for TDB data persistence

4) Edit and configure VIVO installation setting file

Item E) Compiling VIVO

1) check the state of the Git directory before starting the compilation

2) start compiling

Item F) Preparing sample data

1) access the sample file

2) build a translated file in the target language

Item G) Starting VIVO

1) Start TOMCAT

2) Monitoring the execution

Iteration summary

Iteration 1: Create the directory structure of the target language context

Goal

Step 1) Compile and deploye a new/corrected instance of VIVO and start it

Step 2) Running VIVO for testing and/or finding an i18n problem

Step 3) Searching for the problematic file in the VIVO source code

Step 4) Editing the problematic file

Step 5) If necessary reset the various VIVO's indexes and databases

Iteration summary

Iteration 2:

Goal

Step 1) Compile and deploye a new/corrected instance of VIVO and start it

Step 2) Running VIVO for testing and/or finding an i18n problem

Step 3) Searching for the problematic file in the VIVO source code

Step 4) Editing the problematic file

Step 5) If necessary reset the various VIVO's indexes and databases

Iteration summary