v4.9.0

Changes

• Search for matching Component by original_filename (ignoring extension) if unable to find by local_id when building batch from DatastreamUpload .
• Permit inclusion of any admin metadata field in MetadataFileUpload uploaded file. (Non-user-editable admin metadata fields in the uploaded file continue to be ignored in subsequent processing.)
• Expand list of admin metadata fields that can be exported in CollectionsController#export_metadata.
• Send notification on ExportFilesJob failure.
• Add collection ID and title to batches created by DPC Folder Ingest, Standard Ingest, Nested Folder Ingest, and Datastream Upload.
• Re-add Collection Info tab to Collections.
• Add config option to allow Nested Folder Ingest to ignore dot-files.
• Add optional metadata file to Nested Folder Ingest.
• Replace 'Versions' tab with 'Files' tab on Component page. The 'Files' tab lists the current versions of content and derivative datastreams and provides a mechanism for authorized users to upload a new version of each datastream.

ArchivesSpace Integration

• Add message column to ArchivesSpace::CreateDigitalObjects CSV output.
• Permit custom headers to be included in ArchivesSpace::Client requests.
• Recover from Errno::ECONNREFUSED in ArchivesSpace::CreateDigitalObjects, skipping remaining records.
• Add local_id to list of exported fields.

Upgrade Notes

• Copy config/structure_component_type.yml.sample to config/structure_component_type.yml and edit as needed to desired configuration.
• To address deprecation warning, in config/environments/*.rb files, change config.serve_static_assets key to config.serve_static_files.
• Run database migrations (rake db:migrate) to pick up the addition of columns to the batches table.
• Edit config/nested_folder_ingest.yml to add :exclude_dot_files: true to the :scanner: section:

:scanner:
  :exclude:
    - Thumbs.db
  :exclude_dot_files: true

• Edit config/nested_folder_ingest.yml to add a :metadata: section:

:metadata:
  :csv:
    :encoding: UTF-8
    :headers: true
    :col_sep: "\t"
  :location: /metadata/file/directory/

v4.8.5

Changes

• Fixes triggering of collection contents reindexing by keying off
  changed datastreams rather than changed attributes.

v4.8.4

Changes

• Upgrade ddr-models and ddr-batch.

v4.8.3

Changes

• Fix dul_hydra:characterize_files Rake task.

v4.8.2

Changes

• Fix broken FileCharacterization service.

v4.8.1

Changes

• Refactor HasContentBehavior#upload to eliminate exception thrown by call to notify_update.

v4.8.0

Changes

Batch File Export

• Authorized users can now export master files in batch by inputting identifiers (PIDs, ARKs, and/or DPC identifiers) for Components, Items, and/or Collections.

Datastream Upload

• Authorized users can now upload caption files, intermediate files, streamable media files, and thumbnail files to existing Component objects through the staff UI. The Component objects must have local_id's and the uploaded files must have file names that exactly match the corresponding Component's local_id, ignoring the file extension.

Default Structural Metadata

• In the absence of provided structural metadata, default structural metadata is created when a Collection, Item, or Component object is created.
• For Collections and Items that do not have provided structural metadata, default structural metadata is recalculated whenever a child object is added or removed.

Nested Folder Ingest

• Nested Folder Ingest facilitates the ingest of content for which the location of the content file in the source directory structure needs to be preserved.
• Nested Folder Ingest currently supports only those cases in which there is a one-to-one correspondence between Components and Items; i.e., each content file results in the creation of a Component and a Item to contain that Component (and only that Component).
• The folder path from the base ingest point to the content file is stored in the nested_path attribute of the Item that contains the corresponding Component. Note that the nested_path attribute does not contain the absolute file path to the content file but rather the relative file path, relative to the base ingest folder. This is done with the intention of preserving only the relevant portion of the folder path and not incidental portions such as the storage location mount point.

Other

• Component sidebar has link to download caption files and intermediate files if present.
• Default multi-value separator for metadata export and import from ";" to "|".
• Optional preview banner added to distinguish non-production environments.
• "Folder Ingest" is now labeled "DPC Folder Ingest" for clarity.

Upgrade Notes

• Add PREVIEW_BANNER_MSG to config/local_env.yml to activate preview message. Cf. #1883 for more information.
• Datastream Upload
  • Copy config/datastream_upload.yml.sample to config/datastream_upload.yml and edit config/datastream_upload.yml to contain appropriate values for caption, intermediateFile, streamableMedia, and thumbnail.
  • Add desired number of workers for datastream_upload queue to config/resque-pool.yml.
• Nested Folder Ingest
  • Copy config/nested_folder_ingest.yml.sample to config/nested_folder_ingest.yml and edit config/nested_folder_ingest to contain appropriate values.
  • Add desired number of workers fornested_folder_ingest queue to config/resque-pool.yml.
• Batch File Export
  • Configure two environment variable:
    • EXPORT_FILES_STORE - Path to root directory for exported file storage
    • EXPORT_FILES_BASE_URL - URL path for serving export packages (include trailing slash).
  • Add Resque pool workers to queue export.
  • Configure Apache to serve export files:

Alias {EXPORT_FILES_BASE_URL w/o trailing slash} {EXPORT_FILES_STORE}
<Directory {EXPORT_FILES_STORE}>
    PassengerEnabled Off
    Options FollowSymLinks
    # Authz ...
</Directory>

v4.7.0

Changes

Streamable Media

• Adds underlying support for adding streamable files to existing Component objects.
• Adds link to streamable media to Component show view.

Rights

• Interface changes to accompany changes in use of license and rights fields.

ArchivesSpace Integration

• Exports information needed to create ArchivesSpace Digital Objects.
• Creates ArchivesSpace Digital Objects from the exported information.

Other

• Downloads controller now uses send_file for external content.
• Adds support for transient (non-committed) jobs.
• Adds Admin Set selector to Collection creation form.
• Adds affiliation and aleph_id attributes to admin metadata form.
• Removes (long-broken) Export Set functionality.
• Removes administrative dashboard.
• Standard ingest and Dublin Core CSV metadata profile config files are now committed (not ignored). config/metadata_file_profiles/Dublin_Core_CSV.yml and config/standard_ingest.yml.

Upgrade notes

• Configure the ActiveJob adapter in the production environment.
• Add local environment settings ASPACE_BACKEND_URL, ASPACE_USER, ASPACE_PASSWORD for ArchivesSpace integration.
• Allocate one or more workers to aspace queue in Resque pool config.

v4.6.0

Changes

• Ddr::Jobs migrated from ddr-models.

Updates

• Upgrades Ruby to version 2.3.1
• Upgrades Rails to version 4.2.7

Upgrade Notes

• To run Rails commands, you may have to use script/rails from the project root, rather than simply rails.
• Update config/resque_pool.yml to set number of workers for structure queue.
• Run rake db:migrate.
• Set HOST_NAME in config/local_env.yml for use in programmatically creating intra-dul-hydra links (e.g., in emails).
• To accommodate the inclusion of DPC targets and intermediate files in Standard Ingest, add the following entries to the :scanner section of config/standard_ingest.yml (with the appropriate values, if they are different from those given here):

    :targets: dpc_targets
    :intermediate_files: intermediate_files

• Immediately after deployment, run the convert_struct_metadata.rb script. Run without any options, it will convert all existing structural metadata to the new format. Add the --collection [COLLECTION PID] option to convert the existing structural metadata for the objects in a specific collection.
• Following the script above, run the set_default_structure.rb script with the --model MODEL option. This will set default structure for all objects of that model. The script needs to be run for Collection, Item, and Component (but not for Attachment and Target). To restrict the script to published or unpublished objects of the specified model, use the --publication_status [PUBLICATION_STATUS] option.

v4.5.0

Changes

• Upgrades ddr-models to v2.6 and ddr-batch to v1.2.
• Simple Ingest is now called Standard Ingest.
• Standard Ingest validation examines included metadata file to insure that all lines in file will be used (to guard against inadvertent loss of metadata due to certain types of errors in the metadata file).
• DOI field added to AdminMetadata tab and edit form.
• Adds dul_hydra:permanent_id:assign[limit] rake task.
• Incorporates several changes related to batches:
  • BatchesController enqueues Ddr::Batch::BatchDeletionJob in response to destroy action.
  • Batches index display is now paginated and no longer uses separate Pending and Already Run tabs.
  • All users are permitted to view all batches and all batch objects.
  • Adds "My Batches" view and filter to Batches index display.

Upgrade Notes

• Run rake db:migrate to apply new ddr-batch migrations.
• Reindex the objects affected by the new facet fields added in ddr-models -- cf. https://duldev.atlassian.net/browse/DDR-312 for more information on affected fields.
• In config/local_eny.yml, change SIMPLE_INGEST_BASE_PATH: to STANDARD_INGEST_BASE_PATH:.
• Update config/resque_pool.yml to change simple_ingest to standard_ingest if workers are assigned to the simple_ingest queue.
• Rename config/simple_ingest.yml to config/standard_ingest.yml.
• Remove config/log4r_batch_processor.yml file. It is no longer used.
• If desired, edit local configuration files to set Ddr::Models.auto_assign_permanent_id and/or Ddr::Models.auto_update_permanent_id to override default value of false in each case.
• If desired, add a BATCHES_PER_PAGE entry to config/local_eny.ml to override the default number of batches shown per page in the Batches index display. The default value is 10.
• If desired, edit local configuration files to set Ddr::Batch.processor_logging_level to override the default value of Logger::INFO.