Testing Tickets
- RC-1
- blocker -
- non-blocker -
- non-blocker -
- RC-2
- blocker -
External Projects
Hydra (instructions)
Project | Tested by | Success? RC-1 | Success RC-2 | Notes |
---|---|---|---|---|
ActiveFedora | ||||
Hyrax | ||||
Sufia | Jennifer Smith | Tested 7.3-stable branch | ||
Valkyrie | ||||
Avalon 6.0 |
Islandora
Project | Tested by | Success? RC-1 | Success? RC-2 | Notes |
---|---|---|---|---|
CLAW |
API-X
Project | Tested by | Success? RC-1 | Success? RC-2 | Notes |
---|---|---|---|---|
fcrepo-api-x-integration | ||||
fcrepo-api-x-demo (Docker) |
Testing Plan
git clone https://github.com/fcrepo4/fcrepo4 cd fcrepo4 git checkout 4.7.5-RC
Sanity Builds
Scripts
https://github.com/awoods/fcrepo-build-scripts
Project | Command | Platform | Tested By | RC 1 | RC 2 | Notes |
---|---|---|---|---|---|---|
fcrepo4 | mvn clean install | linux | ||||
fcrepo4 | mvn clean install | mac | Frequent warning: "Unable to update victims database! Your CVE records might be out of date." | |||
fcrepo4 | mvn clean install | windows | I wasn't able to perform a sanity build in Windows 10 due to some integration tests in fcrepo-kernel-modeshape failing. A few of them were failing because the last modified date for binaries was not updating all of the time (tests were flapping). Danny Bernstein: NB - apparently this has been an ongoing issue. Apparently Windows 10 on hyper-v works, but is failing/flapping on direct windows install. Aaron Birkland and Yinlin Chen worked on resolving this a while back but did not crack it. | |||
fcrepo-module-auth-rbacl | mvn clean install | linux | ||||
fcrepo-module-auth-rbacl | mvn clean install | mac | Frequent warning: "Unable to update victims database! Your CVE records might be out of date." | |||
fcrepo-module-auth-rbacl | mvn clean install | windows | Aaron Birkland | |||
fcrepo-module-auth-xacml | mvn clean install | mac | ??? | No 4.7.5-RC branch or tag | ||
fcrepo-module-auth-xacml | mvn clean install | windows | ||||
fcrepo-module-auth-webac | mvn clean install | linux | ||||
fcrepo-module-auth-webac | mvn clean install | mac | Frequent warning: "Unable to update victims database! Your CVE records might be out of date." | |||
fcrepo-module-auth-webac | mvn clean install | windows | Aaron Birkland | |||
fcrepo-mint | mvn clean install | linux | ||||
fcrepo-mint | mvn clean install | mac | Danny Bernstein (rc-2) | Frequent warning: "Unable to update victims database! Your CVE records might be out of date." | ||
fcrepo-mint | mvn clean install | windows | Aaron Birkland | |||
fcrepo-audit | mvn clean install | linux | ||||
fcrepo-audit | mvn clean install | mac | Danny Bernstein (rc-2) | Frequent warning: "Unable to update victims database! Your CVE records might be out of date." | ||
fcrepo-audit | mvn clean install | windows | Aaron Birkland | |||
fcrepo-webapp-plus | mvn clean install | linux | ||||
fcrepo-webapp-plus | mvn clean install | mac Danny Bernstein (rc-2) | Frequent warning: "Unable to update victims database! Your CVE records might be out of date." | |||
fcrepo-webapp-plus | mvn clean install | windows | Aaron Birkland | |||
fcrepo-webapp-plus | mvn clean install -Pwebac | linux |
| |||
fcrepo-webapp-plus | mvn clean install -Pwebac | mac Danny Bernstein (rc-2) | Frequent warning: "Unable to update victims database! Your CVE records might be out of date." | |||
fcrepo-webapp-plus | mvn clean install -Pwebac | windows | Aaron Birkland |
Note (18 January 2018): The victims database warnings are due to the victi.ms site currently returning a 503 error. Jared Whiklo has reported this issue upstream: https://github.com/victims/victims-web/issues/155
One-Click Run
cd fcrepo-webapp; mvn clean install -Pone-click
Command | Platform | Tested By | RC-1 | RC-2 | Notes |
---|---|---|---|---|---|
java -jar fcrepo-webapp-<version>-SNAPSHOT-jetty-console.jar | Linux | Jared Whiklo (RC 2) | |||
java -jar fcrepo-webapp-<version>-SNAPSHOT-jetty-console.jar | Mac | ||||
java -jar fcrepo-webapp-<version>-SNAPSHOT-jetty-console.jar | Windows |
Manual Tests
All of the below should take place in the HTML UI and non-vagrant tests should run against fcrepo-webapp-plus.
- Create nested containers
- Create binary resources
- Run fixity on binary
- Update Properties: Perform SPARQL-Update on container
- Update Properties: Perform SPARQL-Update on binary
- Delete container
- Delete binary
- Use transactions
- Create versions
- View versions
- Rollback versions
Database Tests
With Tomcat7 deployment, run above manual tests with alternate backend databases (Configuring JDBC Object Store)
Database | Platform | Tested by | Success RC1? | Success RC2? | Notes |
---|---|---|---|---|---|
MySQL | osx | Danny Bernstein: Restoring v1 after creating v2 does not seem to work: Steps to reproduce: 1. Create a binary with fileA.jpg 2. Create version v1. 3. Update binary with fileB.jpg 4. Create version v2. 5. Click on versions link and click on v1 link. 6. Click "Revert to this Version" 7. Return to binary resource metadata page. Notice the file associated with the version fileB.jpg rather than the expected fileA.jpg. It does not appear that the version was reverted. Joshua Westgard: Hey Danny Bernstein, when I tested the versioning I was just making changes to metadata not binaries and didn't notice any problems; after seeing your note, however, I tried with a binary and did notice that the browser cached the previous version, so after restoring it was still displaying the previous data, but upon refreshing the resource page the restored version was visible as expected. Any chance this is the same problem you were seeing? Joshua Westgard: could be, though I thought I did a hard refresh. I'll try it again to conf hirm. Joshua Westgard: I must not have done a hard refresh. So yes after a hard refresh it worked. I wonder if it should be a logged as a bug nevertheless. Ideally any refresh would not be required at all. A Command-R refresh doesn't refresh the page. Only Command-Shift-R. It seems like it could be a source of confusion. Perhaps a JIRA is warranted. Here it is: | |||
PostgreSQL | osx | Joshua Westgard | config successful; testing in progress | ||
PostgreSQL | linux | Kevin Ford | Tests were performed against fcrepo-webapp-4.7.5-RC not fcrepo-webapp-plus-4.7.5-RC. | ||
MySQL5.6 | linux | Kevin Ford | Tests were performed against fcrepo-webapp-4.7.5-RC not fcrepo-webapp-plus-4.7.5-RC. | ||
PostgreSQL | linux | Kevin Ford | Tests were performed against fcrepo-webapp-plus-4.7.5-RC. | ||
MySQL5.6 | linux | Kevin Ford | Tests were performed against fcrepo-webapp-plus-4.7.5-RC. |
fcr:backup/fcr:restore Functionality
These tests are designed to ensure the proper function of the 'fcr:backup/fcr:restore' features by testing them against various Fedora configurations. The validity of the 'restore' can only be determined by crawling the repository and verifying the successful retrieval of the repository's content.
If the anticipated Fedora release is not backwards compatible with the previous version of Fedora, then the "From Fedora Version" should be the previous version. Otherwise, it is sufficient to test the fcr:backup/fcr:restore functionality using the same version.
See: RESTful HTTP API - Backup and Restore
# Backup curl -X POST localhost:8080/rest/fcr:backup # Restore curl -X POST -d "/path/to/backup/directory" localhost:8080/rest/fcr:restore
Resources
- These python scripts - fcrepo-testing - can be used to load RDF content and binary content to a Fedora repository and verify the integrity of the loaded resources. Output from the load process can be used to verify the integrity of a 'restored' repository. See the README for more info.
- This script can be used to walk your repository, failing if a non-success response is encountered.
Tested by | Platform | Container (Tomcat/Jetty) | Database Backend | From Fedora | To Fedora | Number of RDF Resources | Number of Binaries | Size of Backup (du -h .) | Success RC1? | Success RC2? | Notes |
---|---|---|---|---|---|---|---|---|---|---|---|
Linux | Jetty | File-simple | 4.7.5 | 4.7.5 | 100 | ||||||
Linux | Tomcat | Postgres | 4.7.1 | 4.7.5-RC | 6400 | 6400 | 6.7 G | Performed a GET test on all resources in 4.7.5 RC1 before /fcr:restore, thereby mimicking an 'in place upgrade.' All succeeded. RC2 test used backup of 471 from RC1 test. | |||
Linux | Tomcat | Postgres | 4.7.3 | 4.7.5-RC | 6400 | 6400 | 6.7 G | Performed a GET test on all resources in 4.7.5 RC1 before /fcr:restore, thereby mimicking an 'in place upgrade.' All succeeded. RC2 test used backup of 473 from RC1 test. | |||
Linux | Tomcat | Postgres | 4.7.4 | 4.7.5-RC | 6400 | 6400 | 6.7 G | Performed a GET test on all resources in 4.7.5 RC1 before /fcr:restore, thereby mimicking an 'in place upgrade.' All succeeded. RC2 test used backup of 474 from RC1 test. | |||
Linux | Tomcat | Mysql | 4.7.4 | 4.7.5-RC-1 | 79456 | 11969 | 96G | Performed an fcr:export from 4.7.4. Performed an fcr:import to 4.7.5-RC-1. Performed a reindex using camel (to walk all RDF resources). I did not test every binary. | |||
Linux | Tomcat | Mysql 5.6 | 4.7.5 | 4.7.5-RC-1 | 6400 | 6400 | 6.7 G | Performed a GET test on all resources in 4.7.5 RC1 before /fcr:restore, as verification. All succeeded. For RC2 test, the back up that was restored in this test was actually created from a postgres backend repo. | |||
Linux | Tomcat | Postgres | 4.7.5 | 4.7.5-RC | 6400 | 6400 | 6.7 G | Performed a GET test on all resources in 4.7.5 RC2 before /fcr:restore, as verification. All succeeded. For RC1 test, the back up that was restored in this test was actually created from a mysql backend repo. |
NB: "Success" is measured not by receiving a "204 No Content" message after the 'fcr:restore' command, but by performing a GET on every resource in the repository and receiving "200 OK" messages.
Vagrant Tests
vagrant destroy vagrant up
Test steps | Tested By | Success RC2? | Notes |
---|---|---|---|
FEDORA_AUTH=true | auth working audit disabled triplestore is staying in sync solr triplestore reindexing | ||
FEDORA_AUTH=false | auth (disabled as expected) audit disabled triplestore is staying in sync solr triplestore reindexing | ||
FEDORA_AUTH=true | Triple Store Auth Reindexing Solr (after a "vagrant box update" and "vagrant destroy" it seems to work now).
The above turns out not to be an issue. Updates to solr are collected and then execute in batch operations. I just needed to wait a few seconds. Audit events: with this caveat I am seeing this stacktrace in the karaf logs upon adding a resource. Not sure if it is a problem. Audit events are being logged properly as far as I can tell in /audit and fuseki is also receiving the audit triples. sudo tail -f /opt/karaf/data/log/karaf.log: Stacktrace I confirmed that this behavior exists in in the 4.7.4 | ||
FEDORA_AUTH=false |
Manual Tests
Same as above, plus:
(for reference: https://docs.google.com/presentation/d/1aU-qRVmU0lB18ywepk2AGYEmRe-HfIwhUaofHgathGQ/edit#slide=id.g11caa1fd99_0_0)
- Verify audit events are in triplestore
- Verify resources are in triplestore
- Verify resources are in Solr
- Verify authorization works for the two auth-enabled configurations
Verify reindexing to triplestore works
vagrant ssh sudo service tomcat7 stop sudo rm -rf /etc/fuseki/databases/test_data/* sudo service tomcat7 start curl -XPOST localhost:9080/reindexing/ -H"Content-Type: application/json" -d '["activemq:queue:triplestore.reindex"]'
Backwards Compatibility Tests
- Start 4.7.4 one-click
- Load sample datasets via /fcr:restore
- Run test scripts on 4.7.4
- Stop 4.7.4
- Start RC one-click
- Run test scripts on RC
- ReStart RC
- Run test scripts on RC
Tested by | Success RC1 | Success RC2 | Notes |
---|---|---|---|
Did not run camel_toolbox_tests or authz_tests. | |||
Resources
[1] Testing scripts
4 Comments
Ben Pennell
I wasn't able to perform a sanity build in Windows 10 due to some integration tests in fcrepo-kernel-modeshape failing. A few of them were failing because the last modified date for binaries was not updating all of the time (tests were flapping).
Kevin Ford
I have a 'reset' script I run between backup/restore tests. Among other things, it deletes the data from the respective databases. When I ran the MySQL test for RC5, I saw an odd message during the 'reset' phase that caused me to look deeper. The message was a complaint about the mysql database not existing when I went to DROP it.
At least for me, it seems RC2 will not respect a user-specified repository.json file. When I specify a postgres-specific repository JSON file or a mysql-specific repository JSON file, RC2 **silently** uses the file-simple repository.json file and you see this in the log:
INFO 16:09:50.593 (ModeShapeRepositoryFactoryBean) Using repo config (classpath): file:/opt/lake/tomcat7-47/work/Catalina/localhost/fcrepo/WEB-INF/classes/config/file-simple/repository.json
(By silent I mean there is no ERROR or NOTICE that this is happening, i.e. that your specified repository.json file is not being used and another is.)
Ffcrepo 475 RC2 uses the file-simple/repository.json, which I can confirm as it grows larger when loading data. If I switch back to RC1 (and the *only* thing I alter when I do this is the target WAR file), *my* postgres-specific repository JSON file is respected, which I can confirm as you can see the number of rows in the postgres database table grow. However, when I switch to RC1 and try to use my mysql-specific repository JSON file, it is not respected. Instead, the file-simple repository increases in size as data loads. (My mysql-specific repository json config works for Fcrepo 4.7.4 and, again, the only change I made was the target WAR file.)
To sum up:
Postgres repository JSON:
RC1 Respected; RC2 Not Respected (uses file-simple)
Mysql repository JSON:
RC1 Not Respected (uses file-simple); RC2 Not Respected (uses file-simple)
Is there a change I needed to make for 475? (If so, was it publicized? And apologies if I should have seen it.)
Has anyone else observed this or can confirm it?
Michael Durbin is your mysql test still laying about? Can you confirm the data went into Mysql and did not use the file-simple modeshape repository?
I'm happy to open a bug report about this, but this might be expected behavior and I missed a memo. I'd also like someone else to confirm so we can rule out something about my config settings.
I'll be on the call 25 January and can answer questions if anything above is unclear.
Joshua Westgard
FWIW, I can confirm that I also saw the correct behavior for both mySQL and Postgres when testing RC1. I have not tested the others yet.
Kevin Ford
I can confirm it happens with RC2. Will post Bug report soon.