Date: Fri, 29 Mar 2024 11:03:22 -0400 (EDT) Message-ID: <1139491809.217.1711724602018@lyrasis1-roc-mp1> Subject: Exported From Confluence MIME-Version: 1.0 Content-Type: multipart/related; boundary="----=_Part_216_1014298846.1711724602018" ------=_Part_216_1014298846.1711724602018 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Location: file:///C:/exported.html
This page contains information about how demo6.dspace.org server is setu= p/configured. This demo6.dspace.org server is managed jointly by the DSpace= Committer Team. Any Committer can request server access.
If major issues occur or something needs to be installed requiring root = access, contact Tim Donohue
This page only applies to demo6.dspace.org, which is currently running D= Space 6. Information about the DSpace 7 demo sites can be found at Updating DSpace 7 De= mo Sites
Here's an overview of how everything is setup on the 'demo6.dspace.org' = server:
~/tomcat
~/dspace-src
~/dspace
~/bin/
includes various useful scripts=
Only a LYRASIS employee can do the following:
Contact Tim Donohue if you need a= ny of these tasks performed.
This is how you provide a DSpace Committer with command-line access to t= his server.
~/.ssh/=
authorized_keys
file=20
NOTE: Please add a comment regarding who's key this is, so that it m= akes it easier to clean up later on. For example:
# Tim D= onohue's SSH Key ssh-rsa ....
They should now be able to connect as follows:
ssh dsp= ace@demo6.dspace.org
To ensure we are consistently updating DSpace in the same manner, please=
perform the following steps when updating any configuration
or making any customization to DSpace.
(If you have updates/suggestions, please let us know =E2=80=93 we can ch= ange these processes, but we just need to make sure we are all consistently= following the same general steps)
The ~/dspace-src/
folder is a Git clone of the DSpace-demo =
GitHub Repository: https://github.com/DSpace-Labs/=
demo.dspace.org
In this local Git repository, we are running off of a branch nam= ed "demo". You can see all the branches by running
git bra= nch
Changes that you wish to keep should be committed to this "demo"= branch.
At any one time, you can compare this 'demo' branch to any version of DS= pace. For example, to compare 'demo' to DSpace 5.5 run:
cd ~/ds= pace-src git checkout demo git diff dspace-5.5
WARNING: If you make direct config edits to ~/dspace/config/ you can exp=
ect that they may be overwritten in future (unless you also copy them to ~/=
dspace-src/dspace/config/)
You have been warned! Again, if your changes don't make it to ~/dspace-src=
/ then THEY WILL BE LOST during the next update!
If you are upgrading to the next stable version of DSpace, you can use <= code>git merge to help you merge all changes.
For Example:
cd ~/ds= pace-src # Fetch new branches (eg. minor releases) git fetch --all # Pull down all latest changes git checkout master git pull # Merge them into our "demo" branch git checkout demo git merge dspace-6.0 (or 'git merge master')
NOTE: You should make sure to pay close attention to whether any Conflic= ts occurred. If so, you will need to resolve them manually.
Resolving Conflicts: Here are some hints on how to reso= lve / manage conflicts encountered during a merge:
If there were a lot of conflicts and you just want to accept the "ma= ster" or tagged version (and overwrite any local changes), you can use:
git che= ckout --theirs [full-path-to-file] git add [full-path-to-file]
If you need to completely delete a file that caused conflict, just u= se:
git rm = [full-path-to-file]
If you need to abort an in-process merge that had conflicts, just ru= n:
git mer= ge --abort
cd ~/ds= pace-src # Build DSpace using Mirage 2 theme mvn -U clean package -Dmirage2.on=3Dtrue
These contain the DSpace version number, and should match what we are ru= nning! See Managing Website Content below.
WARNING: this overwrites existing configs in ~/dspace/config/
sudo se= rvice tomcat7 stop cd ~/dspace-src/dspace/target/dspace-installer/ ant update sudo service tomcat7 start
Also make sure your changes made it to ~/dspace/
(and that =
you didn't remove previous settings, especially configs)
An easy way to double check config changes is to do a 'diff' of the late= st dspace.cfg with the most recent '.old' one.
Assuming your changes are already over in ~/dspace-src/
thi=
s is easy...
cd ~/ds= pace-src/ git commit [file] # OR, to commit all changed files git commit -a # Push those changes up to GitHub! git push origin demo
In May/June 2015, we ran into several scenarios where users were logging= in as a demo Admin account and promptly changing the email address associa= ted with that account. In order to avoid this, it is HIGHLY recommend= ed to disable editing of email addresses on demo6.dspace.org.
Here's how it's done:
In Mirage2, the following jQuery can be added to the ~/= dspace-src/dspace-xmlui-mirage2/src/main/webapp/xsl/core/page-structure.xsl= :
<xsl= :template name=3D"buildHead"> <head> ... <!-- CUSTOM FOR : Don't allow EPerson Emails to be edited, so no one = can change default admin acct emails. --> <script type=3D"text/javascript"> jQuery(function() { // Change label for email field in "Edit E-Person" jQuery("label[for=3D'aspect_administrative_eperson_EditEPersonFo= rm_field_email_address']").text("Email Address (editing is disabled on )"); // Make email field in "Edit E-Person" READ-ONLY jQuery("#aspect_administrative_eperson_EditEPersonForm_field_ema= il_address").prop("readonly", true); }); </script> </head> </xsl:template>
In JSPUI, the following jQuery can be added to the ~/dspace-sr= c/dspace-jspui/src/main/webapp/layout/header-submission.jsp:
<hea= d> ... <!-- CUSTOM FOR DEMO6.DSPACE.ORG: Don't allow EPerson Emails to be ed= ited, so no one can change default admin acct emails. --> <script type=3D"text/javascript"> jQuery(function() { // Change label for email field in "Edit E-Person" jQuery("label[for=3D'temail']").text("Email (editing disabled on= demo6.dspace.org):"); // Make email field in "Edit E-Person" READ-ONLY jQuery("#temail").prop("readonly", true); }); </script> </head>
~/dspace/webapps/
(configured =
in Tomcat's context fragments in ~/tomcat/conf/Catalina/localhost/)
~/tomcat/webapps/ROOT/index.html
~/tomcat/webapp=
s/javadocs/
[dspace-source]
):mvn javadoc:aggregate
mvn install javadoc:aggregate && =
rm -rf ~/.m2/repository/org/dspace
instead.[dspace-source]/target/site/apid=
ocs
. Upload it to dspace@demo6.dspace.org:/home/dspace/to=
mcat/webapps/javadocs/[dspace-major-version]/
.The 'dspace' user can easily start/stop PostgreSQL and Tomcat using the = corresponding service scripts:
sudo se= rvice postgresql start sudo service tomcat7 start ~/dspace/bin/start-handle-server sudo service tomcat7 stop sudo service postgresql stop
~/dspace/log/
~/tomcat/logs/
/var/lib/postgresql/9.5/main/pg_log/
/var/log/apache2/
# Login= to 'dspace' database as dspace (password: dspace) psql dspace # Login to 'dspace' database as Postgres Admin (no password needed) psql -h localhost -U postgres dspace
Obviously, you can get the latest information on the existing Cron jobs = by logging into the demo6.dspace.org server and running:
crontab= -l
However, here's a brief overview of a few of the more important Cron job= s.
EVERY SATURDAY NIGHT (currently at 23:59 UTC), all exis=
ting DSpace content is automatically REMOVED and reset to the AIPs located =
at ~/AIP-restore/
This is controlled by the ~/bin/reset-dspace-content
script=
(source code in GitHub)
This is a BASH script that essentially does the following:
~/tmp/data-backup
=
(This backup is performed just in case something goes wrong and we~/AIP-restore
into DSpace as default co=
ntent (This also autocreates the demo EPeople and Groups)=20
~/AIP-restore/r=
eset-dspace-content.log
The set of demo AIPs are all stored in the ~/AIP-restore/
d=
irectory.
To update these AIPs, you must use the DSpace AIP Backup & Restore t=
ools as described at:
AIP Backup and Restore
You can regenerate / update these AIPs by doing the following:
Download the existing AIPs from this directory, e.g.
scp dsp= ace@demo6.dspace.org:~/AIP-restore/* .
Use the downloaded AIPs to "restore" content to your local server's = empty DSpace, e.g.
[dspace= ]/bin/dspace packager -r -a -f -t AIP -e [admin-email] -i 10673/0 /full/pat= h/to/SITE@10673-0.zip
Export a fresh set of AIPs, by performing a full SITE export e.g.
[dspace= ]/bin/dspace packager -d -a -t AIP -e [admin-email] -i 10673/0 -o includeBu= ndles=3DORIGINAL,LICENSE -o passwords=3Dtrue SITE@10673-0.zip
Upload those newly updated AIPs to demo6.dspace.org, e.g.
scp . d= space@demo6.dspace.org:~/AIP-restore/
At this time, sharing AIPs is not automated. It's also not current=
ly possible to share them from the default ~/AIP-restore/
=
location, so this is a bit of a "temporary hack" that needs fixing in the =
future.
First, copy all the AIPs to a shareable=
location. Below, we chose /usr/share/dspace/AIP-restore=
code> folder:
# Creat= e share location sudo mkdir -p /usr/share/dspace/AIP-restore # Manually copy all existing AIPs over there (TODO: This should be automate= d or synced in future) cd /usr/share/dspace/AIP-restore/ sudo cp ~dspace/AIP-restore/* . sudo chown -R dspace:dspace /usr/share/dspace/AIP-restore/ # Add DSpace to www-data user group (to give Apache read access) sudo usermod -a -G www-data dspace # Give Apache group rights on directory sudo chgrp www-data /usr/share/dspace/AIP-restore/ sudo chmod g+rxs /usr/share/dspace/AIP-restore/
Next, update the Apache configuration f= or demo6.dspace.org to provide access to that shareable location:
sudo na= no /etc/apache2/sites-available/25-demo6.dspace.org.conf ## ADD THE FOLLOWING INTO THAT FILE (inside the <VirtualHost>) <VirtualHost *:80> ... # Define path /aip to point at shareable AIP-restore location Alias "/aip" "/usr/share/dspace/AIP-restore" <Directory "/usr/share/dspace/AIP-restore"> # Allow viewing file listing Options Indexes # Don't allow access to README, logs or parent link (..) IndexIgnore README* *.log .. # Allow access to all Order allow,deny Allow from all </Directory> # Don't proxy /aip paths to Tomcat ProxyPass /aip ! ... </VirtualHost>
Reload Apache and test it out:
sudo se= rvice apache2 reload
Assuming everything works, here's a&nbs=
p;wget
command that can be used to download the AIPs to a loca=
l computer
# This = recursively downloads all files (except index.html file) into an "aip" dire= ctory wget -r -np -nH -R "index.html*" --execute=3D"robots=3Doff" http://demo6.ds= pace.org/aip/
Since the "News" sections are editable via the JSPUI, there is a cron jo= b that automatically resets them each night.
It's a rather simple cron job that just copies the original "news-*" fil=
es from the ~/dspace-src/
directory:
05 0 * = * * cp $HOME/dspace-src/dspace/config/news-* $HOME/dspace/config/ > /dev= /null
Since people have been known to change our demo user passwords on this d= emo6.dspace.org server, we now reset them to the default password every hou= r.
This functionality is just a simple set of SQL UPDATE commands that are =
run via the ~/bin/reset-demo-passwords
script.
No longer used/enabled. We've left IRC entirely for Slack.
The kompewter IRC bot is on the server at ~/kompewter
.
It's source code is managed in GitHub at https://gith= ub.com/DSpace-Labs/kompewter
To start kompewter just run:
cd ~/ko= mpewter nohup ./jenni > kompewter.log &
(NOTE: The "nohup" command ensures that kompewter will keep running even= after you log off the server.)
No longer used/enabled. We've left IRC entirely for Slack.
As we now have a DSpace Slack setup, this bot integrates our DSpace Slac= k with IRC (per the below configuration). It allows messages to be sent fro= m Slack to IRC and vice versa.
Currently, this installation is NOT automated via Puppet (That shoul= d be changed at some point)
We are using this tool: https://github.com/ekmartin= /slack-irc
Installation is rather simple:
# Ensur= e we have NPM & Node # NOTE: "nodejs-legacy" ensures the 'node' command maps to 'nodejs' sudo apt-get install npm nodejs nodejs-legacy # Install slack-irc tool sudo npm install -g slack-irc # Create a folder where we can store its config, etc. mkdir ~/slack-irc
Per the documentation at https://github.com/ekmarti= n/slack-irc , we just need a valid JSON conf= ig file to configure this bot.
Here's the current config (save it to ~/slack-irc/config.json=
)
[ { "nickname": "DSpaceSlackBot", "server": "irc.freenode.net", "token": "xoxb-147848164820-lkHcW1gt1C01X4kxx3EKtQR4", "channelMapping": { "#dev-mtg": "#duraspace", "#irc": "#dspace" }, "ircOptions": { "port": 6697, "sasl": true, "secure": true, "selfSigned": true, "certExpired": true, "nick": "DSpaceSlackBot", "userName": "DSpaceSlackBot", "password": "[Ask Tim Donohue for it]" }, "ircStatusNotices": { "join": false, "leave": true } } ]
This configuration ensures messages on #duraspace IRC are also on the Slack #dev-mtg channel (and vice versa). It also= ensures messages on #dspace IRC are also on the Slack= #irc channel (and vice versa). Finally, it also authenticates as= the registered "DSpaceSlackBot" account with Freenode, which ensures the a= ccount is trusted (i.e. won't be blocked). This account is managed by= Tim Donohue, so contact him for mor= e info.
To start the slack-irc bot just run:
cd ~/sl= ack-irc nohup slack-irc --config config.json > slack-irc.log &
(NOTE: The "nohup" command ensures that slack-irc will keep running even= after you log off the server.)
Remote Profiling Using YourKit
Full instructions available at: = http://www.yourkit.com/docs/95/help/profiling_j2ee_remote.jsp
In order to locate potential memory issues in DSpace, we've installed Y=
ourKit on demo6.dspace.org at ~/yjp/
.
It can be accessed remotely so that we can perform various Java profilin= g tasks.
On your local computer:
TODO: add to puppet scripts (install package, pull configuration from S3= , create cron file)
First-time installation will validate domain ownership and generate a pr=
ivate key. Any subsequent certificate requests will reuse the private key. =
The /etc/letsencrypt
directory should be backed up in private =
S3 storage (TODO).
The certificate is issued for 3 months. The script that checks for=
renewals needed is running twice a day on a random minute from =
/etc/cron.d/certbot
.
# Lates= t install instructions available at: https://certbot.eff.org/lets-encrypt sudo apt-get update sudo apt-get install software-properties-common sudo add-apt-repository universe sudo add-apt-repository ppa:certbot/certbot sudo apt-get update sudo apt-get install certbot python-certbot-apache # register and request first certificate, but do not change Apache configur= ation (we'll do it manually) sudo letsencrypt --apache certonly Enter email address (used for urgent notices and lost key recovery) sysadmin@duraspace.org Which names would you like to activate HTTPS for? [*] demo6.dspace.org IMPORTANT NOTES: - If you lose your account credentials, you can recover through e-mails sent to sysadmin@duraspace.org. - Congratulations! Your certificate and chain have been saved at /etc/letsencrypt/live/demo6.dspace.org/fullchain.pem. Your cert will expire on 2017-01-04. To obtain a new version of the certificate in the future, simply run Let's Encrypt again. - Your account credentials have been saved in your Let's Encrypt configuration directory at /etc/letsencrypt. You should make a secure backup of this folder now. This configuration directory will also contain certificates and private keys obtained by Let's Encrypt so making regular backups of this folder is ideal. # replace self-signed certificates with Let's Encrypt certificates sudo vim /etc/apache2/sites-enabled/25-ssl-demo6.dspace.org.conf ## SSL directives SSLEngine on # SSLCertificateFile "/etc/ssl/certs/ssl-cert-snakeoil.pem" # SSLCertificateKeyFile "/etc/ssl/private/ssl-cert-snakeoil.key" # SSLCACertificatePath "/etc/ssl/certs" SSLCertificateFile /etc/letsencrypt/live/demo6.dspace.org/cert.pem SSLCertificateKeyFile /etc/letsencrypt/live/demo6.dspace.org/privkey.pem SSLCACertificateFile /etc/letsencrypt/live/demo6.dspace.org/fullchain.pe= m # test renewal (dry run) sudo letsencrypt renew --dry-run --agree-tos # set up renewal from cron sudo vim /etc/cron.d/certbot # /etc/cron.d/certbot: crontab entries for the certbot package # # Upstream recommends attempting renewal twice a day # # Eventually, this will be an opportunity to validate certificates # haven't been revoked, etc. Renewal will only occur if expiration # is within 30 days. SHELL=3D/bin/sh PATH=3D/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin 0 */12 * * * root test -x /usr/bin/letsencrypt && perl -e 'sleep in= t(rand(3600))' && letsencrypt -n renew --agree-tos
The logs of demo can be consulted in a webUI through https://papertrailapp.com/systems/demo/events. Ask on #dev fo= r the credentials if you want to have a look.
Installation of this viewer required a SyslogAppender appender to be add= ed to /dspace/config/log4j.properties