Ingest

DEFINITION

A digital repository that conforms to ISO 14721 has the capability to systematically Ingest (receive and accept) permanent electronic records from records producing units in the form of Submission Information Packages (SIPs), move them to a staging area where virus checks and content and format validations are performed, transform electronic records into designated preservation formats as appropriate, extract metadata from SIPs and write it to Preservation Description Information (PDI), create Archival Information Packages (AIPs), and transfer the AIPs to the repository’s storage function. This process is considered the minimal workflow for transfer records into a digital repository for long-term preservation and access.

The Archives/RM unit does not currently accession or ingest electronic records.

Move Up to Level 1:

Educate staff on the technical requirements of a trustworthy digital repository through professional training, discussion sessions, and engagement with other archival institutions. Review electronic records transfer guidance and protocols used by peer institutions. Work with agency records managers to identify a pilot project and test producer-to-archive transfer processes. Establish a stable storage area in preparation for accessioning and ingesting electronic records.

The Archives/RM unit ingests any available electronic records from records producing units in any format but assumes only the responsibility to keep the bit stream alive.

Move Up to Level 2:

Preserve at least two copies of electronic records that agencies have transferred. Develop and maintain an inventory of records, repositories, and digital storage media locations. Collaborate with agency IT staff to develop a working profile for Submission Information Packages (SIPs) and Archival Information Packages (AIPs). Develop guidance for agencies on how to prepare electronic records for transfer.

The digital repository ingests partially conforming ISO 14721 SIPs. Partially conforming ISO 14721 SIPs are ingested and held in a staging area while virus checks, and format validations are manually executed. Partially conforming ISO 14721 AIPs are manually transferred from a staging area to archival storage.

Move Up to Level 3:

The Archives has sufficient manual or semi-automated tools to ingest electronic records that meet SIP requirements. Virus checks are performed, and file formats are validated and inventoried. A hash digest is generated for each record, and the records and their metadata are moved to archival storage. Store three copies of each asset in a minimum of two different geographic locations.

The Archives/RM staff collaborates with records producing units to gather information about upcoming records transfers and uses it to build a business case for automated tools for trustworthy digital repositories.

Fully conforming ISO 14721 SIPs are ingested and checked for virus checks and format validations with semi-automated tools. Fully conforming ISO 14721 AIPs are produced and transferred to archival storage through semi-automated tools.

Move Up to Level 4:

Moving to automated processes for routine integrity and fixity checks reduces errors, saves time, and ensures consistency as the volume and complexity of ingest activity increases. SIPs conform to the ISO 14721 standard and are processed using increasingly automated tools and workflows. AIPs conform to the ISO 14721 standard and are produced and transferred to storage using increasingly automated tools and workflows.

Fully conforming ISO 14721 SIPs are ingested and automatically checked for virus checks and format validations. Fully conforming ISO 14721 AIPs are automatically produced and transferred to archival storage.

Sustain Level 4:

The Archives monitors evolving standards and good practices in the digital preservation community. The Archives monitors its digital repository to ensure conformance and efficient, sustainable operations.

HELPFUL HINTS

Minimal technical capabilities to ingest permanent electronic records into a trustworthy digital repository include bit-level preservation actions (such as creating checksums to verify the integrity of content over time), generating preservation and access copies as appropriate, and utilizing different types of storage media in geographically distributed locations. Without funding for an integrated digital preservation system, the Archives can begin to perform basic processes with manual operations and packaging approaches that draw from prevailing standards. The enterprise can follow a path to ‘fully compliant’ submission and archival information packages over time with increasingly robust tools.

Reaching an intermediate level (and above) of ingest capabilities to handle the volume and diversity of permanent electronic records that will have to be transferred from agencies to the Archives for preservation will take time, strategies, and significant technical infrastructure and resources.

REMINDER

The CoSA Digital Preservation Capability self-assessment and Level Up Roadmap are based on the Digital Preservation Capability Maturity Model (DPCMM) which has at its foundation the assumption that archival institutions seeking to establish trusted digital repositories for permanent electronic government records will conform their operations to the specifications of ISO 14721 and ISO 16363, the de facto standards for the global digital preservation community.

DPC Components

Preservica
National Archives