Introduction

One of the questions we often get at DuraCloud is: How can I know how long it will take for me to upload all of my content to DuraCloud? To help answer that question, we've developed a very simple command-line tool called SyncRate. Don't let the "command-line" nature of this tool scare you, we'll walk you through the steps you need to follow below. Keep in mind that there many variables which can affect transfer speeds to DuraCloud. Your internet upload rate (which can be tested very easily) is the obvious first place to look, but things like the size of files being transferred, the media they are stored on, the local network speed, the amount of other traffic on the network, the routing paths used, and many other things can affect the numbers you see.

The SyncRate tool is based on the DuraCloud SyncTool, but is simplified to just perform a transfer of a small number of files, and report the transfer rate. The SyncRate tool allows you to play with a few variables, most notably thread count, to help determine what settings might yield the best results for you when using the SyncTool.

SyncRate

To run the SyncRate tool, you need two files, both of which you will need to download:

1. The SyncRate driver jar file
2. The SyncRate configuration file

The driver jar file is the executable that you will run to perform the test. The configuration file tells SyncRate how the test should be run.

Remember where you store these files when you download them (it is simplest if you place them together in their own directory), you'll need that information later.

To run this tool, you will also need access to a DuraCloud space. If you are not yet a DuraCloud subscriber, you can use a test account. If you are a subscriber with administrative access to an Enterprise account, you can create your own space to use for these tests. If you are using a Preservation or Preservation Plus account, you can either use one of your existing spaces (some test content will be stored there), or submit a support ticket to have a temporary space created for this purpose.

Configuration

Open the syncrate-config.properites file in a text editor (Notepad, Wordpad, TextEdit, or whichever editor you prefer is fine.) You will see a set of configuration parameters which you will need to set for your test to run.

• host=[host]

  • The host value should point to your DuraCloud instance. You should replace "[host]" with something like "myaccount.duracloud.org".

• spaceId=[spaceId]

  • The name of the space where test content will be stored as part of each test. You should replace "[spaceId]" with the name of your test space.

• user=[user]

  • The username you use to log in to DuraCloud. You should replace "[user]" with your username.

• pass=[password]

  • The password you use to log in to DuraCloud. You should replace "[password]" with your password.

• numfiles=[numfiles]

  • The number of files to use in the test. The larger the number of files, the longer the test will take, but the more accurate the result. It is generally recommended to use numbers between 20 and 40. You should replace "[numfiles]" with the number you select.

• filesize=[filesize-in-MB]

  • The size of each test file in Megabytes (MB). Try to estimate the average size of file in your dataset and use that number for this value. You should replace "[filesize-in-MB]" with your typical file size number.

• threads=[numthreads]

  • The number of processing threads that are run at the same time to move your content up to DuraCloud. The default value is 3, but the transfer rate generally improves with the number of threads until either the upload bandwidth is consumed, or the resources of the machine (the CPU and memory) are at full capacity. You should replace "[numthreads]" with your preferred thread count.
  • This number is a good one to experiment with to try to determine the optimal number of threads for moving your content.

• upload=false

  • Determines if the results for each test are also transferred to DuraCloud when the test is complete. The default is "false", which means the results will just be printed to the screen. Set this value to "true" if you'd like to share your results via DuraCloud (note that this will take a few extra minutes.)

Running

To run the SyncRate test, following these steps will allow your test to run

1. The first step is to open a terminal or command prompt. For instructions on doing this in Windows, click here. If you are using a Mac, you will need to open the "Terminal" application, which can be found under /Applications/Utilities.
2. Next navigate to the folder where you saved the SyncRate driver jar and configuration file. Do this by typing "cd", followed by a space, followed by the directory name where you stored these files. For example, on a Windows system, you might type something like "cd C:\Users\bob\Desktop\syncrate". On a Mac, you might type something like "cd /home/bob/syncrate". After you type this command, hit Enter.
3. Now to actually run the SyncRate tool, type: "java -jar syncrate-2.2.2-driver.jar test1 syncrate-config.properties" and hit Enter.
   
   1. The SyncRate command requires two parameters. In this case, we're using the values "test1" and "syncrate-config.properties".
      
      1. The first parameter ("test1" in this example) is the name of the current run. Each run of the SyncRate tool must have a unique name. If the same name is used twice, the transfer rates for the second run will be inaccurate.
      2. Assuming that you have not change the name of the configuration file (and it's in the same directory as the driver jar) you shouldn't need to change the second parameter, though feel free to change the values in the configuration file to run further tests.

That's it. Once the SyncRate tool completes, it will print out an upload rate.

Troubleshooting

• I get something like "java: command not found" when trying to run the SyncRate driver.
  • You need to have Java (version 7 or above) installed to run the SyncRate tool. You can download Java from here.