Below are the notes I used for the Akubra update in today's architecture meeting.

What's Akubra About?

  • A standard Java interface for reading/writing files, but at a different level of abstraction than a filesystem
  • Transactional by design (but implementations may ignore transaction semantics)
  • Exploring web-based exposure
  • From the Akubra wiki: Requirements and Goals

Note: The Akubra wiki is hosted at http://topazproject.org/akubra

Anyone is welcome to sign up to the dev and general mailing lists.

Filesystem vs. BlobStore

Common filesystems:

  • Have directories
  • Can provide system metadata about files (e.g. size, modified date)
  • Allow partial reads and writes of files

An Akubra BlobStore:

  • Has a collection of URI-addressable bitstreams (no "directories")
  • Only provides the size of each file -- is not concerned with other system metadata (yet?).
  • May allow partial reads (InputStreams can skip()...)
  • Does not allow partial writes

Java API

This is in flux.  We are currently testing the design with a simple filesystem implementation.

Blob (A finite, readable bitstream)
BlobStore (For getting connections)
BlobStoreConnection (For CRUD operations) 

 Transactions

  Level of support varies per-implementation (some can "fake it")
  Why: To execute a mixed set of CRUD operations of several files as one atomic unit of work.
  Observation: We can build a transactional blob store on top of a non-transactional one...with the help of a DB.

Example non-transactional BlobStore: FSBlobStore (see FSBlobStoreConnection)

Higher-level BlobStore TBD:

  • Uses FSBlobStore to persist data
  • Uses database to support transactions (via id mapping)

Other possible storage Plug-Ins:

  • S3 (anything based on current LLStore should be easy to port over)
  • ZFS (already transactional, does not need layering)
  • Centera (content-addressible...ids not available till content is written)
  • Sam/QFS (hierarchical storage implies graceful handling of delays...not a use case we've factored in yet)

Web-based exposure?

  • Opens up use of akubra-java impls to other (remotely-running) programs
  • Allows an akubra impl that's a client to remote akubra instance
  • Lots of interesting possibilities!
  • No labels