Page History

...

DSpace has a set of command line tools for importing and exporting items in batches, using the DSpace Simple Archive Format. Apart from the offered functionality, these tools serve as a prime an example for users who aim to implement their own item importer.

...

The basic concept behind the DSpace 's Simple Archive Format is to create an archive, which is directory full of items, with a a directory containing one subdirectory per item. Each item directory contains a file for the item's descriptive metadata, and the files that make up the item.

...

The contents file simply enumerates, one file per line, the bitstream file names. See the following example:

file_1.doc
        file_2.pdf
        license

Please notice that the license is optional, and if you wish to have one included, you can place the file in the .../item_001/ directory, for example.

...

1. Create a separate file for the other schema named metadata_[prefix].xml, where the [prefix] is replaced with the schema's prefix.
2. Inside the xml file use the dame same Dublin Core syntax, but on the <dublin_core> element include the attribute schema=[prefix].

3. Here is an example for ETD metadata, which would be in the file metadata_etd.xml:

   Code Block
   <?xml version="1.0" encoding="UTF-8"?> <dublin_core schema="etd"> <dcvalue element="degree" qualifier="department">Computer Science</dcvalue> <dcvalue element="degree" qualifier="level">Masters</dcvalue> <dcvalue element="degree" qualifier="grantor">Texas>Michigan AInstitute &of M<Technology</dcvalue> </dublin_core>

   
   

Importing Items

Before running the item importer over items previously exported from a DSpace instance, please first refer to Transferring Items Between DSpace Instances.

‡ These are mutually exclusive.

...

Replacing existing items is relatively easy. Remember that mapfile you were supposed to savesaved above? Now you will use it. The command (in short form):

...

You are able to unimport or delete items provided you have the mapfile. Remember that mapfile you were supposed to savesaved above? The command is (in short form):

...

• Resume. If, during importing, you have an error and the import is aborted, you can use the --resume (-R) flag that you can try to resume the import where you left off after you fix the error.
  
  

• Importing (per item) into multiple collections Specifying the owning collection on a per-item basis from the command line administration tool

  If you omit the -c flag, which normally is otherwise mandatory, the ItemImporter searches for a file named "collections" in each item directory. This file should contain a list of collections, one per line, specified either by their handle, or by their internal db id. The ItemImporter then will put the item in each of the specified collections. The owning collection is the collection specified in the first line of the collections file.

  If both the -c flag is specified and the collections file exists in the item directory, the ItemImporter will ignore the collections file and will put the item in the collection specified on the command line.

  Since the collections file can differ between item directories, you have this gives you more fine-grained control of the process of batch adding items to collections.

• Importing with BTE

  The DSpaceOutputGenerator, that which writes the metadata in into the DSpace Simple Archive Format, has been updated to produce the collections file, if a metadata field named "collections" (reserved word) exists in the original metadata. This is mainly applicable to the CSV input format which is more flexible, but could also be implemented with a Modifier that adds the "collections" field to each Record in the BTE pipeline.

  Important note: an entry with key the "collections" key should be in the output map that is used by the DSpaceOutputGenerator.
  More info in Importing Items via basic bibliographic formats (Endnote, BibTex, RIS, TSV, CSV) and online services (OAI, arXiv, PubMed, CrossRef, CiNii).

UI Batch Import (JSPUI)

Batch import can also take place via the Administrator’s Administrator's UI. The steps to follow are:

A. Prepare the data

1. Records, that is, Items, i.e. the metadata and their bitstreams, must be in the Simple Archive Format describer earlier in this chapter. Thus, for each record item there must be a separate folder directory that contains the corresponding files of the specific recorditem.
2. Moreover, in each record folderitem directory, there can be another file that describes the collection or the collections that this record item will be added to. The name of this file must be “collections” "collections" and it is optional. It has the following format:
   
   
   Each line contains the handle of the collection. The collection in the first line is the owning collection while the rest are the other collection the item should belong to.
3. Compress the record folders in item directories into a zip file. Be careful, Please note that you need to zip the whole list of record folders and actual item directories and not just the folder directory that has within it the record folderscontains the item directories. Thus, the final zip file must have inside it the records folder straight throughdirectly contain the item directories.
4. Place the zip file in a public domain URL, like Dropbox or Google Drive or wherever you have access to do so. Since such a zip file can be very big in size, the batch import UI needs the URL to download it for a public location rather than just upload it and get a timeout exception
   
   

B. Import the records items via the UI

1. Login as an administrator
2. Find the menu on the top right of the page, and select the “Administer” "Administer" option
   
   
   
   
3. Select the option “Batch Import” "Batch Import" option from the "Content" drop down menu named “Content” on  on the top of the page
   
   
   
   

4. Fill in the form that appears as follows:

• Field #1: select the type of the input data that you want to batch import. Be sure to select “Simple "Simple Archive Format” Format" in this drop down menu
• Field #2: Copy/Paste the public URL where the zip file mentioned earlier is located
• Filed #3: Select the owning collection of the records items you are importing. This field is optional meaning that if you leave it empty, you are supposed to include per item collection information (via the “collections” "collections" file mentioned before) in the Simple Archive Format
• Field #4: Select the other collections the record item will belong to. You can select more than one collection by just holding down the Ctrl key on your keyboard.  If you select the owning collection in this multiselect input control, it will be ignored at the very end.

...

1) If you select an owning collection from this form, then , the “collections” "collections" file that may be included in the records will item will be ignored.

2) If you do not specify an owning collection, and for some items , there exist no “collections” file in the record folderno "collections" file exists in the item directory, then the item will not be imported in DSpace

...

Finally, when you submit the form you will receive a message for the background execution of the process informing you that the import process is being executed in the background (since it may take long). At the end, you will receive a success or failure email (to the email address of your DSpace account) informing you for of the status of the import.

...

C. View past batch imports (that have be done via the UI)

1. Login

2. Visit “My DSpace” "My DSpace" page
   
   

3. In On the new next page, you can see the history of batch imports that have be done in the past. For each import, the following information is available:
   
   The status of the batch import (success or failure)
   The number of records that were finally importeditems that the user tried to import
   The number of items that were actually imported

   records that the user tried to import

    

   Moreover, the user can take the following actions:

   Download the map file that was produced during the import. In this file, someone can see the records that were imported and This file contains a list of items that were imported with the corresponding handle that was given assigned to them by DSpace.

   Delete the imported recordsitems. Everything that was imported will be deleted (and including the history folder directory in the “import” directory in the DSpace installation folder"[dspace]/import" directory)

   In case of failure, the user can “Resume” "Resume" the import. He is transferred in The user is taken to the upload form again, but the system knows recognizes the initial import (and the map file) in order to resume the old import. There is a red label in the form that informs the user for the “Resume” form.

...

1. . There is a red label in the form that informs the user about the "Resume" form.

Image Added

UI Batch Import (XMLUI)

A SimpleArchiveFormat package can be imported by an administrator in XMLUI. The SimpleArchiveFormat package needs to be compressed into a ZIP file, and it be will be uploaded to XMLUI through the browser. DSpace will then process that ZIP, and ingest items into DSpace. A stable network connection is recommended, as your browser will need to upload a potentially large ZIP file, and then wait while DSpace processes that ZIP file. 

While logged in as an administrator, click on Batch Import (ZIP):

Image Added

Then, choose the owning collection from the collection dropdown, and browse to the ZIP file on your computer that has the SimpleArchiveFormat ZIP file.

Image Added

If successful, you will get a green message with a list of handles that were imported. It is what is considered the "map file".

Image Added

If an error occurred, you will get a red error message with the issue:

Image Added

Exporting Items

The item exporter can export a single item or a collection of items, and creates a DSpace simple archive according to in the aforementioned format for each exported item to be exported. The items are exported in a sequential order in which they are retrieved from the database. As a consequence, the sequence numbers of the item subdirectories (item_000, item_001) are not related to DSpace handle or item id'sids.

Exporting a Collection

To export a collection's items you type at the CLIThe CLI command to export the items of a collection:

[dspace]/bin/dspace export --type=COLLECTION --id=collIDcollectionID_or_handle --dest=dest_dir/path/to/destination --number=seq_num

Short form:

[dspace]/bin/dspace export -t COLLECTION -i [CollID or Handle]collectionID_or_handle -d /path/to/destination -n Someseq_numbernum

Exporting a Single Item

The keyword COLLECTION means that you intend to export an entire collection. The ID can either be the database ID or the handle. The exporter will begin numbering the simple archives with the sequence number that you supply. To export a single item use the keyword ITEM and give the item ID as an argument:

[dspace]/bin/dspace export --type=ITEM --id=itemID_or_handle --dest=dest_dir/path/to/destination --number=seq_num

Short form:

[dspace]/bin/dspace export -t ITEM -i [itemID or Handle]_or_handle -d /path/to/destination -n someseq_numbernum

Each exported item will have an additional file in its directory, named '"handle'". This will contain the handle that was assigned to the item, and this file will be read by the importer so that items exported and then imported to another machine will retain the item's original handle.

...