| Title (goal) | Leverage Cloud Storage |
|---|---|
| Primary Actor | APTrust Consortium |
| Scope | System Black-box |
| Level | Organization |
| Story | As a consortium, APTrust is building an aggregation repository utilizing cloud compute and storage to take advantage of economies of scale. Fedora currently makes a synchronous storage assumption for writes and rendered tests showing severe degradation of performance when using cloud storage (.5 hours on local disk vs. half day on cloud storage). If this problem could be resolved, APTrust could move to an architecture where Fedora could become the entry point for ingest and allow leveraging the full features Fedora would offer in such an architecture. |
Overview
Content Tools
All content on the LYRASIS Wiki is licensed under the CC BY (Attribution) license
, unless otherwise noted.
1 Comment
Brad Spry
I utilize Fedora 3.5 and my datastreamStore is on cloud storage using YAS3FS, which can log file operations and reveal read/write block sizes.
I see read block sizes at 2k and write block sizes of 4k.
To enable a system to be tuned and optimized for any given cloud storage, Fedora would need user configurable block sizes.
For example, a stock FUSE-based filesystem has a max write block size limit of 128k. Knowing the limits, systems administrators just need the ability to adjust Fedora's file ops block size settings.