Test Environment
MacOS 10.11.6
Composition of Resources
Used data from https://github.com/fcrepo4-labs/fcrepo-sample-dataset/tree/master/additional_datasets
- lubm
- 10kIndirectContainers
- manually created (in UI) simple object w/ uploaded file
Installation
- Install postgres 9.6.1 via homebrew:
brew install postgres gem install fcrepo_wrapperCreate config file .fcrepo_wrapper:
port: 8984 enable_jms: false
- Clone https://github.com/fcrepo4-labs/fcrepo-import-export and build 0.1.1
viamvn clean install Clone https://github.com/fcrepo4-labs/fcrepo-sample-dataset and extract:
cd fcrepo-sample-dataset/additional_datasets/ tar xzvf lubm/LUBM_02.tar.gz tar xzvf 10kIndirectContainers/10kIndirectContainers.tar.gz
Initialization
Start postgres:
pg_ctl -D db/postgres startfcrepo_wrapper cleanStart Fedora 4.7.0:
fcrepo_wrapperConfirm Fedora up/empty at: http://localhost:8984/rest/
curl -X POST -d "/full/path/to/fcrepo-sample-dataset/additional_datasets/LUBM_02/" "http://localhost:8984/rest/fcr:restore"
Restore extracted reference dataset(s):
curl -X POST -d "/full/path/to/fcrepo-sample-dataset/additional_datasets/LUBM_02/" "http://localhost:8984/rest/fcr:restore" curl -X POST -d "/full/path/to/fcrepo-sample-dataset/additional_datasets/10kIndirectContainers/" "http://localhost:8984/rest/fcr:restore"
Confirm 100 "objects" and 10000 "memberCollection" loaded at: http://localhost:8984/rest/
From "objects" node in UI, create an "albums" container. Add one binary to it, uploading an image file.
Test 0 - Export partial: albums
Directory (11360) will vary.
java -jar target/fcrepo-import-export-0.1.1-SNAPSHOT.jar --mode export --resource http://localhost:8984/rest/11360/objects/albums --dir /Users/atz/repos/import_export_test/ --binaries java -jar target/fcrepo-import-export-0.1.1-SNAPSHOT.jar -x .json -l application/ld+json --mode export --resource http://localhost:8984/rest/11360/objects/albums --dir /Users/atz/repos/import_export_test/ --binaries
Test 0 Conclusions
+ Repeated (successful) exports do not alter target dir after first run
- + If export is repeated after file is deleted, the export does replace the file, both binary and text.
- + If export is repeated after file is corrupted, the export does restore the file, both binary and text.
Test 1 - Full export/import/export
Export:
java -jar target/fcrepo-import-export-0.1.1-SNAPSHOT.jar --mode export --resource http://localhost:8984/rest/ --dir /Users/atz/repos/import_export_test/ --binaries
Deleting everything at http://127.0.0.1:8984/rest/11360 fails:
WARN 16:23:22.499 (arjuna) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a000004:e85c:588f9f9e:a9 in state RUN WARN 16:31:53.376 (arjuna) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a000004:e85c:588f9f9e:a9 in state SCHEDULE_CANCEL WARN 16:31:57.952 (arjuna) ARJUNA012095: Abort of action id 0:ffff0a000004:e85c:588f9f9e:a9 invoked while multiple threads active within it. WARN 16:32:08.391 (arjuna) ARJUNA012108: CheckedAction::check - atomic action 0:ffff0a000004:e85c:588f9f9e:a9 aborting with 1 threads active! WARN 16:32:08.391 (arjuna) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff0a000004:e85c:588f9f9e:a9 in state CANCEL_INTERRUPTED WARN 16:32:46.082 (arjuna) ARJUNA012120: TransactionReaper::check worker Thread[Transaction Reaper Worker 0,5,main] not responding to interrupt when cancelling TX 0:ffff0a000004:e85c:588f9f9e:a9 -- worker marked as zombie and TX scheduled for mark-as-rollback WARN 16:32:59.021 (arjuna) ARJUNA012110: TransactionReaper::check successfuly marked TX 0:ffff0a000004:e85c:588f9f9e:a9 as rollback only WARN 16:33:25.371 (arjuna) ARJUNA012113: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] missed interrupt when cancelling TX 0:ffff0a000004:e85c:588f9f9e:a9 -- exiting as zombie (zombie count decremented to 0)
Then, manually called kill -9 on fedora java process. Cleaned out installation and restarted Fedora.
Import:
java -jar target/fcrepo-import-export-0.1.1-SNAPSHOT.jar --mode import --resource http://localhost:8984/rest/ --dir /Users/atz/repos/import_export_test/ --binaries --auditLog
Finishes successfully with log message in importexport_audit_20170130T170006.log:
... 2017-01-30 17:10:11,021 Finished import... 20006 resources imported
But simultaneously, Fedora crashes again. Looks like a normal shutdown but was unprompted (no "Fedora is shutting down." message):
[Thread-0] INFO org.simplericity.jettyconsole.DefaultJettyManager - Shutting down Fedora Repository Deployable Web Application 4.7.0..
[Thread-0] INFO org.eclipse.jetty.server.ServerConnector - Stopped ServerConnector@1f39269d{HTTP/1.1}{0.0.0.0:8984}
[Thread-0] INFO / - Closing Spring root WebApplicationContext
[Thread-0] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped o.s.j.JettyConsoleWebappContext@1e643faf{/,file:/private/var/folders/6q/13dm9dcn1qx_gw9x66lhl7jc0000gq/T/fcrepo-webapp-4.7.0-jetty-console.jar_8984/webapp/,UNAVAILABLE}{/private/var/folders/6q/13dm9dcn1qx_gw9x66lhl7jc0000gq/T/fcrepo-webapp-4.7.0-jetty-console/fcrepo-webapp-4.7.0-jetty-console.jar}
[Thread-0] INFO org.simplericity.jettyconsole.DefaultJettyManager - Shutdown of Fedora Repository Deployable Web Application 4.7.0 complete.
After restart, appears empty. So import failed. On second identical attempt, succeeded with the same "Finished import... 20006 resources imported" message.
Export (Test 0) again:
java -jar target/fcrepo-import-export-0.1.1-SNAPSHOT.jar --mode export --resource http://localhost:8984/rest/11360/objects/albums --dir /Users/atz/repos/import_export_test/ --binaries
Diff of jsonld files:
--- a/rest/11360/objects/albums.jsonld
+++ b/rest/11360/objects/albums.jsonld
@@ -1,9 +1,9 @@
[ {
"@id" : "http://localhost:8984/rest/11360/objects/albums",
- "@type" : [ "http://fedora.info/definitions/v4/repository#Container", "http://fedora.info/definitions/v4/repository#Resource", "http://www.w3.org/ns/ldp#RDFSource", "http://www.w3.org/ns/ldp#Container" ],
+ "@type" : [ "http://fedora.info/definitions/v4/repository#Container", "http://www.w3.org/ns/ldp#Container", "http://www.w3.org/ns/ldp#RDFSource", "http://fedora.info/definitions/v4/repository#Resource" ],
"http://fedora.info/definitions/v4/repository#created" : [ {
"@type" : "http://www.w3.org/2001/XMLSchema#dateTime",
- "@value" : "2017-01-30T22:23:50.14Z"
+ "@value" : "2017-01-31T01:37:09.859Z"
} ],
"http://fedora.info/definitions/v4/repository#createdBy" : [ {
"@value" : "bypassAdmin"
@@ -13,7 +13,7 @@
} ],
"http://fedora.info/definitions/v4/repository#lastModified" : [ {
"@type" : "http://www.w3.org/2001/XMLSchema#dateTime",
- "@value" : "2017-01-30T22:24:05.691Z"
+ "@value" : "2017-01-31T01:38:32.169Z"
} ],
"http://fedora.info/definitions/v4/repository#lastModifiedBy" : [ {
"@value" : "bypassAdmin"
diff --git a/rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee/fcr%3Ametadata.jsonld b/rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee/fcr%3Ametadata.jsonld
index 029aa6b..01e1490 100644
--- a/rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee/fcr%3Ametadata.jsonld
+++ b/rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee/fcr%3Ametadata.jsonld
@@ -1,9 +1,9 @@
[ {
"@id" : "http://localhost:8984/rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee",
- "@type" : [ "http://fedora.info/definitions/v4/repository#Binary", "http://fedora.info/definitions/v4/repository#Resource", "http://www.w3.org/ns/ldp#NonRDFSource" ],
+ "@type" : [ "http://www.w3.org/ns/ldp#NonRDFSource", "http://fedora.info/definitions/v4/repository#Binary", "http://fedora.info/definitions/v4/repository#Resource" ],
"http://fedora.info/definitions/v4/repository#created" : [ {
"@type" : "http://www.w3.org/2001/XMLSchema#dateTime",
- "@value" : "2017-01-30T22:24:05.694Z"
+ "@value" : "2017-01-31T01:38:32.179Z"
} ],
"http://fedora.info/definitions/v4/repository#createdBy" : [ {
"@value" : "bypassAdmin"
@@ -16,7 +16,7 @@
} ],
"http://fedora.info/definitions/v4/repository#lastModified" : [ {
"@type" : "http://www.w3.org/2001/XMLSchema#dateTime",
- "@value" : "2017-01-30T22:24:05.694Z"
+ "@value" : "2017-01-31T01:38:32.179Z"
} ],
"http://fedora.info/definitions/v4/repository#lastModifiedBy" : [ {
"@value" : "bypassAdmin"
So the salient differences for JSON+LD are datestamps and the order of @type elements.
But for Turtle, the diff is semantically distinct. The datestamps differ in the same way, but the objects have acquired new "ldp:Container , ldp:RDFSource" and "ldp:NonRDFSource" values, respectively.
Diff (condensed):
--- rest/11360/objects/albums.ttl -<http://localhost:8984/rest/11360/objects/albums> a fedora:Container , fedora:Resource ; +<http://localhost:8984/rest/11360/objects/albums> a fedora:Container , ldp:Container , ldp:RDFSource , fedora:Resource ; ... --- rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee/fcr%3Ametadata.ttl -<http://localhost:8984/rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee> a fedora:Binary , fedora:Resource ; +<http://localhost:8984/rest/11360/objects/albums/0c/7c/4e/29/0c7c4e29-5bc0-4005-95a6-bacd103fddee> a ldp:NonRDFSource , fedora:Binary , fedora:Resource ;
Test 1 Conclusions
Fedora unresponsive during large (failing) delete operation in UI. This is probably a known problem. - Fedora responsive during large import operation, but then crashes and fails the whole load.
- Round trip adds semantic information.
Test 2 - Another Trip
Import again, without deleting.
java -jar target/fcrepo-import-export-0.1.1-SNAPSHOT.jar --mode import --resource http://localhost:8984/rest/ --dir /Users/atz/repos/import_export_test/ --binaries --auditLog
Same log message: "Finished import... 20006 resources imported"
Repeat Test 0:
- Only timestamps change, semantic fields stabilized.
Repeat Test 1:
- TTL diffs unreadable because of extreme line-length, but the element http://localhost:8984/rest/11360/memberCollection still has the same 10,000 PCDM:hasMember objects.
Diff after TTL re-export includes:
20004 files changed, 60098 insertions(+), 60098 deletions(-) rewrite rest/11360/memberCollection.ttl (85%) rewrite rest/11360/memberCollection/members.ttl (82%) rewrite rest/11360/objects.ttl (85%)
The same semantic differences are evident per file, e.g.:
-<http://localhost:8984/rest/11360/objects/47/2e/63/66/472e6366-627c-42ac-b8fd-f131b48bf3fc> a fedora:Container , fedora:Resource ; +<http://localhost:8984/rest/11360/objects/47/2e/63/66/472e6366-627c-42ac-b8fd-f131b48bf3fc> a fedora:Container , ldp:Container , ldp:RDFSource , fedora:Resource ;
During execution the following errors were thrown in fedora's terminal:
ERROR 18:29:01.923 (FedoraInvalidNamespaceExceptionMapper) NamespaceExceptionMapper caught an exception: Prefix fcr has not been registered ERROR 18:36:22.781 (velocity) VelocimacroProxy.render() : exception VM = #triples() org.eclipse.jetty.io.EofException: null at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:192) ~[jetty-io-9.3.1.v20150714.jar:9.2.3.v20140905] at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:337) ~[jetty-io-9.3.1.v20150714.jar:9.2.3.v20140905] at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:128) ~[jetty-io-9.3.1.v20150714.jar:9.2.3.v20140905] at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:646) ~[jetty-server-9.3.1.v20150714.jar:9.2.3.v20140905] at org.eclipse.jetty.util.IteratingCallback.processIterations(IteratingCallback.java:233) ~[jetty-util-9.3.1.v20150714.jar:9.2.3.v20140905] at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:193) ~[jetty-util-9.3.1.v20150714.jar:9.2.3.v20140905] at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:457) [jetty-server-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:769) [jetty-server-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:802) [jetty-server-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:139) ~[jetty-server-9.3.1.v20150714.jar:9.2.3.v20140905] at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:132) ~[jetty-server-9.3.1.v20150714.jar:9.2.3.v20140905] at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:347) ~[jetty-server-9.3.1.v20150714.jar:9.2.3.v20140905] at org.glassfish.jersey.servlet.internal.ResponseWriter$NonCloseableOutputStreamWrapper.write(ResponseWriter.java:325) ~[jersey-container-servlet-core-2.22.2.jar:na] at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:229) ~[jersey-common-2.22.2.jar:na] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:299) ~[jersey-common-2.22.2.jar:na] at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) ~[na:1.8.0_31] at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282) ~[na:1.8.0_31] at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) ~[na:1.8.0_31] at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:135) ~[na:1.8.0_31] at java.io.OutputStreamWriter.write(OutputStreamWriter.java:220) ~[na:1.8.0_31] at java.io.Writer.write(Writer.java:157) ~[na:1.8.0_31] at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:491) ~[velocity-1.7.jar:1.7] at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72) ~[velocity-1.7.jar:1.7] at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:87) ~[velocity-1.7.jar:1.7] at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72) ~[velocity-1.7.jar:1.7] at org.apache.velocity.runtime.directive.Foreach.render(Foreach.java:420) ~[velocity-1.7.jar:1.7] at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:207) [velocity-1.7.jar:1.7] at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72) ~[velocity-1.7.jar:1.7] at org.apache.velocity.runtime.directive.VelocimacroProxy.render(VelocimacroProxy.java:216) ~[velocity-1.7.jar:1.7] at org.apache.velocity.runtime.directive.RuntimeMacro.render(RuntimeMacro.java:311) [velocity-1.7.jar:1.7] at org.apache.velocity.runtime.directive.RuntimeMacro.render(RuntimeMacro.java:230) [velocity-1.7.jar:1.7] at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:207) [velocity-1.7.jar:1.7] at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:342) [velocity-1.7.jar:1.7] at org.apache.velocity.Template.merge(Template.java:356) [velocity-1.7.jar:1.7] at org.apache.velocity.Template.merge(Template.java:260) [velocity-1.7.jar:1.7] at org.fcrepo.http.api.responses.StreamingBaseHtmlProvider.writeTo(StreamingBaseHtmlProvider.java:171) [fcrepo-http-api-4.7.0.jar:4.7.0] at org.fcrepo.http.api.responses.StreamingBaseHtmlProvider.writeTo(StreamingBaseHtmlProvider.java:76) [fcrepo-http-api-4.7.0.jar:4.7.0] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:265) [jersey-common-2.22.2.jar:na] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:250) [jersey-common-2.22.2.jar:na] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) [jersey-common-2.22.2.jar:na] at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:106) [jersey-server-2.22.2.jar:na] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) [jersey-common-2.22.2.jar:na] at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:86) [jersey-server-2.22.2.jar:na] at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) [jersey-common-2.22.2.jar:na] at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1130) [jersey-common-2.22.2.jar:na] at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:711) [jersey-server-2.22.2.jar:na] at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:444) [jersey-server-2.22.2.jar:na] at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:434) [jersey-server-2.22.2.jar:na] at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:329) [jersey-server-2.22.2.jar:na] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [jersey-common-2.22.2.jar:na] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [jersey-common-2.22.2.jar:na] at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [jersey-common-2.22.2.jar:na] at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [jersey-common-2.22.2.jar:na] at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [jersey-common-2.22.2.jar:na] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [jersey-common-2.22.2.jar:na] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [jersey-server-2.22.2.jar:na] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [jersey-server-2.22.2.jar:na] at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) [jersey-container-servlet-core-2.22.2.jar:na] at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) [jersey-container-servlet-core-2.22.2.jar:na] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) [jersey-container-servlet-core-2.22.2.jar:na] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) [jersey-container-servlet-core-2.22.2.jar:na] at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) [jersey-container-servlet-core-2.22.2.jar:na] at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769) [jetty-servlet-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) [jetty-servlet-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) [jetty-server-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) [jetty-security-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) [jetty-server-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125) [jetty-server-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) [jetty-servlet-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) [jetty-server-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059) [jetty-server-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [jetty-server-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) [jetty-server-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) [jetty-server-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) [jetty-server-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.server.Server.handle(Server.java:497) [jetty-server-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) [jetty-server-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248) [jetty-server-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) [jetty-io-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610) [jetty-util-9.2.3.v20140905.jar:9.2.3.v20140905] at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539) [jetty-util-9.2.3.v20140905.jar:9.2.3.v20140905] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_31] Caused by: java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) ~[na:1.8.0_31] at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) ~[na:1.8.0_31] at sun.nio.ch.IOUtil.write(IOUtil.java:148) ~[na:1.8.0_31] at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:503) ~[na:1.8.0_31] at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:172) ~[jetty-io-9.3.1.v20150714.jar:9.2.3.v20140905] ... 81 common frames omitted ERROR 18:36:22.783 (velocity) Exception in macro #triples called at /views/resource.vsl[line 47, column 21] Jan 30, 2017 6:36:22 PM org.glassfish.jersey.server.ServerRuntime$Responder writeResponse SEVERE: An I/O error has occurred while writing a response message entity to the container output stream. org.glassfish.jersey.server.internal.process.MappableException: org.apache.velocity.exception.VelocityException: VelocimacroProxy.render() : exception VM = #triples() at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:92) at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) at org.glassfish.jersey.message.internal.MessageBodyFactory.writeTo(MessageBodyFactory.java:1130) at org.glassfish.jersey.server.ServerRuntime$Responder.writeResponse(ServerRuntime.java:711) at org.glassfish.jersey.server.ServerRuntime$Responder.processResponse(ServerRuntime.java:444) at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:434) at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:329) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) at org.glassfish.jersey.internal.Errors.process(Errors.java:315) at org.glassfish.jersey.internal.Errors.process(Errors.java:297) at org.glassfish.jersey.internal.Errors.process(Errors.java:267) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:497) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.velocity.exception.VelocityException: VelocimacroProxy.render() : exception VM = #triples() at org.apache.velocity.runtime.directive.VelocimacroProxy.render(VelocimacroProxy.java:228) at org.apache.velocity.runtime.directive.RuntimeMacro.render(RuntimeMacro.java:311) at org.apache.velocity.runtime.directive.RuntimeMacro.render(RuntimeMacro.java:230) at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:207) at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:342) at org.apache.velocity.Template.merge(Template.java:356) at org.apache.velocity.Template.merge(Template.java:260) at org.fcrepo.http.api.responses.StreamingBaseHtmlProvider.writeTo(StreamingBaseHtmlProvider.java:171) at org.fcrepo.http.api.responses.StreamingBaseHtmlProvider.writeTo(StreamingBaseHtmlProvider.java:76) at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.invokeWriteTo(WriterInterceptorExecutor.java:265) at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$TerminalWriterInterceptor.aroundWriteTo(WriterInterceptorExecutor.java:250) at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) at org.glassfish.jersey.server.internal.JsonWithPaddingInterceptor.aroundWriteTo(JsonWithPaddingInterceptor.java:106) at org.glassfish.jersey.message.internal.WriterInterceptorExecutor.proceed(WriterInterceptorExecutor.java:162) at org.glassfish.jersey.server.internal.MappableExceptionWrapperInterceptor.aroundWriteTo(MappableExceptionWrapperInterceptor.java:86) ... 39 more Caused by: org.eclipse.jetty.io.EofException at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:192) at org.eclipse.jetty.io.WriteFlusher.write(WriteFlusher.java:337) at org.eclipse.jetty.io.AbstractEndPoint.write(AbstractEndPoint.java:128) at org.eclipse.jetty.server.HttpConnection$SendCallback.process(HttpConnection.java:646) at org.eclipse.jetty.util.IteratingCallback.processIterations(IteratingCallback.java:233) at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:193) at org.eclipse.jetty.server.HttpConnection.send(HttpConnection.java:457) at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:769) at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:802) at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:139) at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:132) at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:347) at org.glassfish.jersey.servlet.internal.ResponseWriter$NonCloseableOutputStreamWrapper.write(ResponseWriter.java:325) at org.glassfish.jersey.message.internal.CommittingOutputStream.write(CommittingOutputStream.java:229) at org.glassfish.jersey.message.internal.WriterInterceptorExecutor$UnCloseableOutputStream.write(WriterInterceptorExecutor.java:299) at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:135) at java.io.OutputStreamWriter.write(OutputStreamWriter.java:220) at java.io.Writer.write(Writer.java:157) at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:491) at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72) at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:87) at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72) at org.apache.velocity.runtime.directive.Foreach.render(Foreach.java:420) at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:207) at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72) at org.apache.velocity.runtime.directive.VelocimacroProxy.render(VelocimacroProxy.java:216) ... 53 more Caused by: java.io.IOException: Broken pipe at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) at sun.nio.ch.IOUtil.write(IOUtil.java:148) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:503) at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:172) ... 81 more
Test 2 Conclusions
- Still some work to do to make round-trips possible, unless the representational differences are considered semantically identical because of the definition of fedora:Container, for example. It seems unnecessary to have the behavior diverge between turtle and json+ld, though.
- Datestamps still vary, a known issue, if I understand correctly.
- Unclear what effect the exceptions had on data, if any.
Comments:
- Generating a config implicitly and then using it on subsequent operations means that those who expect their command-line options to apply only for that command will be surprised.
- Generating a config to /var means that on certain OS, the config will be lost upon system restart. So those who rely on the persistence of previous command line options will also be surprised.
- Generating a config on first execution means that order of operation between otherwise independent calls matters (say, 3 calls, each exporting a different object with different options).
- I would recommend an explicit --set-config or --write-config option to eliminate these 3 categories of surprise.
