The following numbers were gathered on an AWS cluster
Node network I/O performance
When using netcat to transfer a 5 gb file from one VM to another VM, the resulting throughput was
Node hdd performance
Root partition
ubuntu@ip-10-73-143-203:~$ sync;time sudo bash -c "(dd if=/dev/zero of=bf bs=8k count=500000; sync)"
500000+0 records in
500000+0 records out
4096000000 bytes (4.1 GB) copied, 513.869 s, 8.0 MB/s
Partition at /mnt
ubuntu@ip-10-73-143-203:~$ sync;time sudo bash -c "(dd if=/dev/zero of=/mnt/bf bs=8k count=500000; sync)"
500000+0 records in
500000+0 records out
4096000000 bytes (4.1 GB) copied, 34.3706 s, 119 MB/s
We have to make sure to use the /mnt partition since it writes much faster than the root one
CPU
Following is the output of 'cat /proc/cpuinfo' on one VM
Load balancing
Results
Test Utility
BenchTool: https://github.com/futures/benchtool
A. Ingest bench using fcrepo3
Created 100 objects with one datastream of 50mb size
Size: 100 * 52428800 bytes = 5gb
B. Ingest bench using single node fcrepo4 with single thread and clustered config
Created 100 objects with one datastream of 50mb size
Size: 100 * 52428800 bytes = 5gb
C. Ingest bench using single node fcrepo4 with single thread and minimal config
Created 100 objects with one datastream of 50mb size
Size: 100 * 52428800 bytes = 5gb
D. Ingest bench using seven nodes fcrepo4 with seven threads and clustered config
Created 100 objects with one datastream of 50mb size
Size: 100 * 52428800 bytes = 5gb
E. Ingest bench using seven nodes fcrepo4 with seven threads and clustered config, no indexing
Created 100 objects with one datastream of 50mb size
Size: 100 * 52428800 bytes = 5gb
F. Ingest bench using seven nodes fcrepo4 with eleven threads and clustered config, no indexing
Created 100 objects with one datastream of 50mb size
Size: 100 * 52428800 bytes = 5gb
