Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. In DuraCloud, content is added by the user into a Snapshot Storage Provider space. This is a staging area that is backed by S3.
  2. The user selects a button in the space to create snapshot and enters snapshot metadata
  3. The DuraCloud UI calls the storage provider snapshot task indicating space to snapshot
  4. The snapshot task creates snapshot properties file and stores it in snapshot space
    1. When the snapshot properties file is added, the space is transitioned to read-only
    2. Snapshot properties file includes the depositor ID as well as details about the snapshot (account ID, space ID, date, etc)
  5. The snapshot task calls to the bridge application to indicate that a snapshot needs to be taken, providing DuraCloud host/port/space.
  6. The bridge application adds an entry to the snapshot db table with the details of the snapshot action
  7. The bridge application connects to DuraCloud and copies all content from DuraCloud space to bridge storage
    1. During transfer, content properties are captured in a file
    2. During transfer, each content item is added to content db table (with snapshot id)
  8. The bridge application creates two manifest files (md5 and sha256) for the content and verifies all content was transferred correctly
  9. The bridge application sends a notification (email) to Chronopolis that a snapshot is ready
    1. Chronopolis Intake service polls for new snapshots to process
  10. Chronopolis Intake service uses the content in bridge storage to construct a bag for deposit
    1. The Intake service validates content against the manifest written by the bridge application
    2. The Intake service creates the necessary bag files (bagit, bag-info, etc) that are included in the bag
    3. If the content contained in the snapshot is larger than 250 GB, multiple bags are created
  11. Chronopolis Ingest service performs replication to the appropriate Chronopolis nodes
    1. Ace Tokens are created for other Chronopolis nodesThe Ingest service creates replication requests to the selected replication nodes.
    2. rsync is used between Chronopolis nodes to transfer content
  12. Chronopolis makes a call to the bridge application to indicate that content has been successfully copied to preservation storage
    1. Intake service checks for existing snapshots to see if they could be completed. 
    2. Intake service calls bridge to update history as each node replication completes
  13. The bridge application deletes the directory in bridge storage used for the snapshot
  14. The bridge application makes a call to a task in the DuraCloud Snapshot Storage Provider to indicate that it is now time to clean up the snapshot content
  15. The cleanup task sets a policy on the underlying S3 bucket which causes the content to be removed within 24 hours
  16. The bridge application watches the snapshot space, and when it becomes empty, calls the snapshot complete task, which clears the S3 bucket policy
  17. The bridge application notifies the user who requested the snapshot that it has been completed

...