A checksum is a compact value computed from file contents. If even one byte changes, a strong cryptographic checksum will almost certainly change too. This makes checksums useful for detecting accidental corruption, incomplete transfers, or confusion between files with the same name.
For research data, the more useful term is often fixity: confidence that a sequence of bytes has not changed since a known digest was calculated. Fixity is a narrow property, and that narrowness is its strength. A checksum can answer “are these bytes the same?” It cannot answer every question about whether the data are correct.
What a checksum does and does not establish
A cryptographic hash function reads a byte stream and produces a fixed-length digest. SHA-256, for example, produces 256 bits commonly written as 64 hexadecimal characters. The NIST Secure Hash Standard specifies SHA-256 and related algorithms.
If independently calculated SHA-256 values match, that is strong evidence that the compared byte streams are identical. If they differ, at least one byte stream or calculation context differs. Line-ending conversion, decompression, metadata embedded in a file, or a single changed base can all produce a different digest.
A matching checksum does not prove that:
- The file came from the claimed instrument or collaborator.
- It was assigned to the correct biological sample.
- The sequence is biologically valid or contamination-free.
- The file format is safe to parse.
- The checksum value itself has not been replaced.
- The file has never been disclosed to an unauthorized person.
- A derived result was produced from that file.
Those claims require other evidence: authenticated transfer, sample identifiers, chain-of-custody procedures, format validation, access controls, and explicit links between analysis inputs and outputs.
Build a realistic failure model
Teams should choose checksum controls around concrete failures.
Transfer truncation or corruption. A network interruption, damaged medium, or faulty copy produces different bytes at the destination. Comparing a source-provided digest with one calculated after receipt detects the mismatch.
Wrong-file selection. Two directories contain sample_42.ab1, but only one is the intended submission. A digest distinguishes their contents, although it does not tell you which one is scientifically correct.
Silent modification. An editor normalizes a FASTA file, a script rewrites headers, or a pipeline replaces an output in place. A changed digest reveals that the object is no longer byte-for-byte identical.
Storage loss or decay. An object becomes unreadable or is restored incorrectly. Scheduled fixity checks and restore verification can detect a problem, provided a trusted prior digest still exists.
Malicious substitution. An attacker replaces both a file and its checksum in the same writable location. A colocated manifest offers little protection. Detecting this scenario requires a separately protected or authenticated manifest, signed release information, access logging, or another trust anchor.
Time-of-check/time-of-use gaps. A file is hashed at receipt, then changed before analysis. The receipt checksum remains correct for the old bytes but says nothing about what the analysis process opened. Critical workflows may need verification immediately before use or content-addressed, versioned inputs.
Define the byte object precisely
Checksums apply to bytes, not to an abstract sequence. The same bases can have different digests when represented with different line lengths, headers, encodings, compression formats, or newline conventions. Conversely, a biologically incorrect file can have a perfectly stable digest.
An SOP should state what is hashed:
- The exact file as received, before parsing or normalization.
- A compressed archive or each member after extraction, or both.
- Generated reports and analysis outputs where fixed release artifacts matter.
- A canonical manifest when a dataset consists of many objects.
Preserve the original file checksum even when a normalized derivative is useful. Give the derivative its own identifier and checksum, and record the transformation that connects them.
Use checksums at explicit checkpoints
Calculating a digest once creates a reference value. Integrity assurance comes from later comparison.
- At creation or instrument export: calculate a digest as close to the source as operations permit.
- At receipt: compare with the sender’s value through an authenticated channel. If no source digest exists, record that the receipt value establishes a new baseline rather than proving transfer integrity.
- After upload: verify the stored object against the digest calculated from the incoming stream, especially when storage and application writes are separate operations.
- Before analysis: confirm that the intended input still matches the recorded baseline when the risk warrants it.
- During preservation: recompute at a risk-based interval and investigate mismatches or unreadable objects.
- After restore or migration: compare every restored object or a documented statistically justified sample, according to policy.
- At publication or handoff: ship a manifest with stable relative names, byte sizes, algorithms, and digests.
The BagIt specification, RFC 8493 is an established approach for packaging payload files with manifests and validating transfer completeness.
Make manifests operational
A useful fixity manifest contains more than a list of hashes. For each object, record:
- Stable object or file identifier
- Relative path or storage key, with rules for renaming
- Byte size
- Hash algorithm and digest
- Relevant version or release identifier
- Verification date, result, and responsible process
Protect the manifest according to the threat model. Keeping it in the same broadly writable folder as the data can still catch accidents, but it is weak against deliberate replacement. A repository database with restricted write access, a signed manifest, or a separately controlled preservation record provides a stronger reference.
Questions for a checksum SOP
- Which algorithm is approved for new fixity records, and how are legacy algorithms handled?
- Is the exact received file retained, even when normalized derivatives are created?
- Who supplies the first trusted checksum, and through which channel?
- Does upload verification compare the stored object or only hash the incoming stream?
- At which later checkpoints is the digest recomputed?
- What happens after a mismatch: quarantine, retry, source comparison, incident review, or re-acquisition?
- Can an operator alter both objects and their checksum records?
- How are manifests versioned, protected, backed up, and restored?
- How are file identifiers connected to samples, runs, and reports?
How GeneFlow records file fixity
During upload, GeneFlow reads the incoming file in chunks and calculates SHA-256 before saving the upload through its configured storage backend. The resulting file record retains the 64-character checksum along with original name, declared kind, byte size, project, uploader, storage key, and processing status. Storage keys include the project identifier and a generated UUID, reducing dependence on a possibly duplicated original filename.
Processing status moves through states such as processing, ready, or error, so a stored object is not automatically presented as successfully parsed. Samples created from supported files link back to the source file, providing a route from a result-bearing sample to its upload record.
This is an ingestion provenance control, not a guarantee of ongoing integrity. The current file record does not by itself show a second hash calculated after storage, periodic fixity checks, object versioning, a signed manifest, or immutable retention. Deployment operators should add verification, backup, storage logging, and recovery procedures appropriate to their risks.
Implementation checklist
- Standardize SHA-256 or another institutionally approved modern algorithm for new records.
- Hash exact received bytes before any normalization.
- Store algorithm, digest, byte size, object identifier, and calculation date together.
- Obtain source-side digests where possible and compare after receipt.
- Give each derivative a new identifier and digest.
- Protect checksum records separately enough for the stated threat model.
- Recompute digests after storage migration and restore.
- Quarantine mismatches; never silently replace the baseline with a new value.
- Link verified file identifiers to analysis inputs and released reports.
- Test the mismatch-response procedure with a deliberately altered test object.
Checksums are most valuable when teams resist asking them to prove too much. Use them as precise evidence about bytes, connect that evidence to identity and provenance controls, and verify it at the moments when data can change.