Storage Management
Your work’s not done after you’ve moved your files into storage — they need active management. Use a Storage Log like the example below to document where files are stored, when they were moved to storage, how often you will check the storage (audit schedule), and how often you will need to update the storage (hardware replacement schedule), if applicable.
Storage information to document for ongoing management:
- Network Attached Storage/local server: Find out what the backup protocols are for your server. If it is networked, someone is managing it and likely has a schedule of backups they follow. How often? To what media?
- Cloud storage: Who in your organization has access? Where are the username, password, or other needed authentication stored? What is the viability of this service provider over the next 1-3 years?
- External hard drives or other removable media: When was it purchased? What brand? Where is the physical drive stored? When will it need to be replaced? (external hard drives should be replaced every 3-5 years).
Sample storage log:
| Storage format and location | Date implemented | Audit schedule | Hardware replacement schedule | |
|---|---|---|---|---|
| Location 1 | 1 TB Western Digital hard drive in director’s office | Jan. 2020 | Every six months (Jan. and July each year) | Every three years. Next replacement date: January 2023 |
| Location 2 | Carbonite Safe installed on curator’s desktop computer | Jan. 2020 | Every six months (Jan. and July each year) | N/A |
| Location 3 | Dedicated folder on the server managed by public library. | June 2021 | Annually | Refer to partnership agreement with library. |
Regardless of which storage solutions you’re using, you’ll need to check or audit them regularly to make sure they’re still working and still meeting your needs. Pull down samples of your files on a regular schedule — at least once a year — to make sure that your system can still open and read them. Run a fixity check (see below) on all files. If any files have changed, replace the corrupted file with a version you know is still good from one of your two other copies stored elsewhere.
Checking File Integrity (Fixity)
The term fixity is used to describe the stability of a digital object. The goal of digital archival storage is that your files remain unchanged over time. The challenge is that digital files can degrade or change, and those changes are often invisible to the human eye. Whenever a digital collection is moved, processed, or altered, things can go wrong. Your network connection drops out while you’re moving files, a disk gets full and subsequent data copied there is lost, a software bug or crash leads to unexpected results, or human error leads to unintentional deletions or changes.
A simple way to catch some of these kinds of potential errors is to keep an eye on your total file counts and sizes. For example, if you’re copying a folder from an external hard drive to a cloud storage location, check the total number of files and total folder size before and after the move to make sure nothing got dropped along the way.
The most reliable way to tell if your digital files have changed is by using software tools to generate and monitor checksums. A checksum is a mathematical algorithm run on a file and its resulting value. You can think of this as a digital fingerprint. If a file has remained authentic and stable, with no changes, its fingerprint (checksum) will also stay the same. If a file becomes corrupted, degraded, or is otherwise changed in some way, its fingerprint (checksum) will change.
There are software tools available to perform these fixity checks. A checksum-monitoring utility may be built into your collections management system. Or you can use a free or low-cost checksum utility such as Fixity Pro, MD5 Summer, or FastSum.
Checksums do not prevent file corruption or degradation from happening, but they let you know there’s a problem so you can address it. If a file is discovered to be altered, you can replace it with an unaltered copy from one of your other storage locations. If lots of files have changed, that’s a symptom of a bigger problem – you may need to update your storage media or revisit your file transfer procedures.
When to check file fixity (by comparing file counts, total file size, and/or checksums):
- When files are first created or acquired
- Before files are moved to a new location
- After files are moved to a new location
- On a regular schedule, i.e. every three, six, or twelve months
Illustration of unchanged and changed checksums. Adapted from POWRR Professional Development Institutes for Digital Preservation slides developed in partnership with the Digital Preservation Coalition.
Digital Preservation
Digital preservation is not only about storing and maintaining multiple copies of digital files. It’s the set of ongoing activities, plans, and resource commitments that will help keep your digital content available and usable in the coming months, years, and decades. You do not need to adopt a digital preservation plan or policy in order to successfully store and maintain your content, but as your digital collections grow, undertaking digital preservation planning will help your organization protect past investments and plan for the future.
Digital preservation is often described as a three-legged stool. One leg of the stool is technology – that’s the hardware and software you use for storing and managing your content. Another leg is the organization – this leg includes the policies, procedures, and documentation put in place to make digital preservation a part of your organization’s regular operations. The third leg of the stool is resources – the budget allocations, staff skills development, and other institutional commitments necessary to sustain digital preservation activities. All three legs of the stool need to be equally strong and stable in order to stand up.
As your digital program matures, consider these foundational resources to get started with digital preservation planning at your organization:
- “Digital Preservation Peer Assessment.” Northeast Document Conservation Center, 2019
- “Getting Started,” Digital Preservation Handbook, Digital Preservation Coalition
- Digital POWRR webinar series, 2016
Resources and Tools
- “How-To Guide: Fixity.” Outagamie Waupaca Library System, 2018.
- What is Fixity, and When Should I be Checking It? National Digital Stewardship Alliance, 2014.
- Boyd, Doug. “The Checksum and the Preservation of Digital Oral History.” . 2012.
The Digital Readiness Toolkit and related resources were made possible in part by grants from the National Historical Publications and Records Commission.
