This page contains information about how demo.dspace.org server is setup/configured. This demo.dspace.org server is managed jointly by the DSpace Committer Team. Any Committer can request server access. If major issues occur or something needs to be installed requiring root access, contact Tim Donohue or "sysadmin AT duraspace DOT org" |
Here's an overview of how everything is setup on the 'demo.dspace.org' server:
~/tomcat
~/dspace-src
~/dspace
~/bin/
includes various useful scriptsOnly a DuraSpace employee can do the following:
Contact sysadmin@duraspace.org or Tim Donohue if you need any of these tasks performed.
This is how you provide a DSpace Committer with command-line access to this server.
~/.ssh/authorized_keys
fileNOTE: Please add a comment regarding who's key this is, so that it makes it easier to clean up later on. For example:
# Tim Donohue's SSH Key ssh-rsa .... |
They should now be able to connect as follows:
ssh dspace@demo.dspace.org |
To ensure we are consistently updating DSpace in the same manner, please perform the following steps when updating any configuration
or making any customization to DSpace.
(If you have updates/suggestions, please let us know – we can change these processes, but we just need to make sure we are all consistently following the same general steps)
The ~/dspace-src/
folder is a Git clone of the DSpace-demo GitHub Repository: https://github.com/DSpace-Labs/demo.dspace.org
In this local Git repository, we are running off of a branch named "demo". You can see all the branches by running
git branch |
Changes that you wish to keep should be committed to this "demo" branch.
At any one time, you can compare this 'demo' branch to any version of DSpace. For example, to compare 'demo' to DSpace 5.5 run:
cd ~/dspace-src git checkout demo git diff dspace-5.5 |
WARNING: If you make direct config edits to ~/dspace/config/ you can expect that they may be overwritten in future (unless you also copy them to ~/dspace-src/dspace/config/) |
If you are upgrading to the next stable version of DSpace, you can use git merge
to help you merge all changes.
For Example:
cd ~/dspace-src # Pull down all latest changes git checkout master git pull # Merge them into our "demo" branch git checkout demo git merge dspace-6.0 (or 'git merge master') |
NOTE: You should make sure to pay close attention to whether any Conflicts occurred. If so, you will need to resolve them manually.
Resolving Conflicts: Here are some hints on how to resolve / manage conflicts encountered during a merge:
If there were a lot of conflicts and you just want to accept the "master" or tagged version (and overwrite any local changes), you can use:
git checkout --theirs [full-path-to-file] git add [full-path-to-file] |
If you need to completely delete a file that caused conflict, just use:
git rm [full-path-to-file] |
If you need to abort an in-process merge that had conflicts, just run:
git merge --abort |
cd ~/dspace-src # Build DSpace using Mirage 2 theme mvn -U clean package -Dmirage2.on=true |
WARNING: this overwrites existing configs in ~/dspace/config/
sudo service tomcat7 stop cd ~/dspace-src/dspace/target/dspace-installer/ ant update sudo service tomcat7 start |
Also make sure your changes made it to ~/dspace/
(and that you didn't remove previous settings, especially configs)
An easy way to double check config changes is to do a 'diff' of the latest dspace.cfg with the most recent '.old' one.
Assuming your changes are already over in ~/dspace-src/
this is easy...
cd ~/dspace-src/ git commit [file] # OR, to commit all changed files git commit -a # Push those changes up to GitHub! git push origin demo |
In May/June 2015, we ran into several scenarios where users were logging in as a demo Admin account and promptly changing the email address associated with that account. In order to avoid this, it is HIGHLY recommended to disable editing of email addresses on demo.dspace.org.
Here's how it's done:
In Mirage2, the following jQuery can be added to the ~/dspace-src/dspace-xmlui-mirage2/src/main/webapp/xsl/core/page-structure.xsl:
<xsl:template name="buildHead"> <head> ... <!-- CUSTOM FOR DEMO.DSPACE.ORG: Don't allow EPerson Emails to be edited, so no one can change default admin acct emails. --> <script type="text/javascript"> jQuery(function() { // Change label for email field in "Edit E-Person" jQuery("label[for='aspect_administrative_eperson_EditEPersonForm_field_email_address']").text("Email Address (editing is disabled on demo.dspace.org)"); // Make email field in "Edit E-Person" READ-ONLY jQuery("#aspect_administrative_eperson_EditEPersonForm_field_email_address").prop("readonly", true); }); </script> </head> </xsl:template> |
In JSPUI, the following jQuery can be added to the ~/dspace-src/dspace-jspui/src/main/webapp/layout/header-submission.jsp:
<head> ... <!-- CUSTOM FOR DEMO.DSPACE.ORG: Don't allow EPerson Emails to be edited, so no one can change default admin acct emails. --> <script type="text/javascript"> jQuery(function() { // Change label for email field in "Edit E-Person" jQuery("label[for='temail']").text("Email (editing disabled on demo.dspace.org):"); // Make email field in "Edit E-Person" READ-ONLY jQuery("#temail").prop("readonly", true); }); </script> </head> |
~/dspace/webapps/
(configured in Tomcat's context fragments in ~/tomcat/conf/Catalina/localhost/
)~/tomcat/webapps/ROOT/index.html
~/tomcat/webapps/javadocs/
[dspace-source]
):mvn javadoc:aggregate
mvn install javadoc:aggregate && rm -rf ~/.m2/repository/org/dspace
instead.[dspace-source]/target/site/apidocs
. Upload it to dspace@demo.dspace.org:/home/dspace/tomcat/webapps/javadocs/[dspace-major-version]/
.sudo service postgresql start sudo service tomcat7 start ~/dspace/bin/start-handle-server sudo service tomcat7 stop sudo service postgresql stop |
crontab -l |
However, here's a brief overview of a few of the more important Cron jobs.
This is controlled by the ~/bin/reset-dspace-content
script (source code in GitHub)
This is a BASH script that essentially does the following:
~/tmp/data-backup
(This backup is performed just in case something goes wrong and we~/AIP-restore
into DSpace as default content (This also autocreates the demo EPeople and Groups)~/AIP-restore/reset-dspace-content.log
The set of demo AIPs are all stored in the ~/AIP-restore/
directory.
To update these AIPs, you must use the DSpace AIP Backup & Restore tools as described at:
AIP Backup and Restore
You can regenerate / update these AIPs by doing the following:
Download the existing AIPs from this directory, e.g.
scp dspace@demo.dspace.org:~/AIP-restore/* . |
Use the downloaded AIPs to "restore" content to your local server's empty DSpace, e.g.
\[dspace\]/bin/dspace packager -r -a -f -t AIP -e [admin-email] -i 10673/0 /full/path/to/SITE@10673-0.zip |
Export a fresh set of AIPs, by performing a full SITE export e.g.
\[dspace\]/bin/dspace packager -d -a -t AIP -e [admin-email] -i 10673/0 -o includeBundles=ORIGINAL,LICENSE -o passwords=true SITE@10673-0.zip |
Upload those newly updated AIPs to demo.dspace.org, e.g.
scp . dspace@demo.dspace.org:~/AIP-restore/ |
Since the "News" sections are editable via the JSPUI, there is a cron job that automatically resets them each night.
It's a rather simple cron job that just copies the original "news-*" files from the ~/dspace-src/
directory:
05 0 * * * cp $HOME/dspace-src/dspace/config/news-* $HOME/dspace/config/ > /dev/null |
Since people have been known to change our demo user passwords on this demo.dspace.org server, we now reset them to the default password every hour.
This functionality is just a simple set of SQL UPDATE commands that are run via the ~/bin/reset-demo-passwords
script.
The kompewter IRC bot is on the server at ~/kompewter
.
It's source code is managed in GitHub at https://github.com/DSpace-Labs/kompewter
To start kompewter just run:
cd ~/kompewter nohup ./jenni > kompewter.log & |
(NOTE: The "nohup" command ensures that kompewter will keep running even after you log off the server.)
As we now have a DSpace Slack setup, this bot integrates our DSpace Slack with IRC (per the below configuration). It allows messages to be sent from Slack to IRC and vice versa.
Currently, this installation is NOT automated via Puppet (That should be changed at some point)
We are using this tool: https://github.com/ekmartin/slack-irc
Installation is rather simple:
# Ensure we have NPM & Node # NOTE: "nodejs-legacy" ensures the 'node' command maps to 'nodejs' sudo apt-get install npm nodejs nodejs-legacy # Install slack-irc tool sudo npm install -g slack-irc # Create a folder where we can store its config, etc. mkdir ~/slack-irc |
Per the documentation at https://github.com/ekmartin/slack-irc , we just need a valid JSON config file to configure this bot.
Here's the current config:
[ { "nickname": "DSpaceSlackBot", "server": "irc.freenode.net", "token": "xoxb-147848164820-lkHcW1gt1C01X4kxx3EKtQR4", "channelMapping": { "#dev-mtg": "#duraspace", "#irc": "#dspace" } } ] |
This configuration ensures messages on #duraspace IRC are also on the Slack #dev-mtg channel (and vice versa). It also ensures messages on #dspace IRC are also on the Slack #irc channel (and vice versa)
To start the slack-irc bot just run:
cd ~/slack-irc nohup slack-irc --config config.json > slack-irc.log & |
(NOTE: The "nohup" command ensures that slack-irc will keep running even after you log off the server.)
Full instructions available at: http://www.yourkit.com/docs/95/help/profiling_j2ee_remote.jsp |
In order to locate potential memory issues in DSpace, we've installed YourKit on demo.dspace.org at ~/yjp/
.
It can be accessed remotely so that we can perform various Java profiling tasks.
On your local computer:
TODO: add to puppet scripts (install package, pull configuration from S3, create cron file)
First-time installation will validate domain ownership and generate a private key. Any subsequent certificate requests will reuse the private key. The /etc/letsencrypt
directory should be backed up in private S3 storage (TODO).
The certificate is issued for 3 months. The script that checks for renewals needed is running twice a day on a random minute from /etc/cron.d/certbot
.
sudo apt-get install python-letsencrypt-apache # register and request first certificate, but do not change Apache configuration (we'll do it manually) sudo letsencrypt --apache certonly Enter email address (used for urgent notices and lost key recovery) sysadmin@duraspace.org Which names would you like to activate HTTPS for? [*] demo.dspace.org IMPORTANT NOTES: - If you lose your account credentials, you can recover through e-mails sent to sysadmin@duraspace.org. - Congratulations! Your certificate and chain have been saved at /etc/letsencrypt/live/demo.dspace.org/fullchain.pem. Your cert will expire on 2017-01-04. To obtain a new version of the certificate in the future, simply run Let's Encrypt again. - Your account credentials have been saved in your Let's Encrypt configuration directory at /etc/letsencrypt. You should make a secure backup of this folder now. This configuration directory will also contain certificates and private keys obtained by Let's Encrypt so making regular backups of this folder is ideal. # replace self-signed certificates with Let's Encrypt certificates sudo vim /etc/apache2/sites-enabled/25-ssl-demo.dspace.org.conf ## SSL directives SSLEngine on # SSLCertificateFile "/etc/ssl/certs/ssl-cert-snakeoil.pem" # SSLCertificateKeyFile "/etc/ssl/private/ssl-cert-snakeoil.key" # SSLCACertificatePath "/etc/ssl/certs" SSLCertificateFile /etc/letsencrypt/live/demo.dspace.org/cert.pem SSLCertificateKeyFile /etc/letsencrypt/live/demo.dspace.org/privkey.pem SSLCACertificateFile /etc/letsencrypt/live/demo.dspace.org/fullchain.pem # test renewal (dry run) sudo letsencrypt renew --dry-run --agree-tos # set up renewal from cron sudo vim /etc/cron.d/certbot # /etc/cron.d/certbot: crontab entries for the certbot package # # Upstream recommends attempting renewal twice a day # # Eventually, this will be an opportunity to validate certificates # haven't been revoked, etc. Renewal will only occur if expiration # is within 30 days. SHELL=/bin/sh PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin 0 */12 * * * root test -x /usr/bin/letsencrypt && perl -e 'sleep int(rand(3600))' && letsencrypt -n renew --agree-tos |