...
| Excerpt |
|---|
Large files can be uploaded via the REST API, or projected into the repository using filesystem federation. Transfer times for uploading to the repository via the REST API are about the same as copying using NFS, and moderately faster than using SCP. Uploading via the REST API to a federated filesystem works, but is significantly slower. |
Java command-line options and System properties can be used
-Xmx2048m– maximum memory Java can use-Dfcrepo.home=/path/to/data– set the directory for permanent data-Djava.io.tmpdir=/path/to/tmpdir– set the directory for temp files
...
| title | OutOfMemoryException when ingesting large files |
|---|
Because of https://issues.jboss.org/browse/MODE-2103, ingesting files larger than Java heap space with certain Infinispan configuraitons (e.g. LevelDB) creates an OutOfMemoryError. The following TestCase can be used to reproduce the issue: https://github.com/futures/large-files-test
Workaround: Use Infinispan file storage with a large heap, e.g.:
...
- . Data uploaded to a federated filesystem via the REST API is written to a temp file in this directory before being moved to the federated filesystem, so this directory should have enough free space for the largest files you will upload.
Ingesting Large Files via the REST API
...