Cryptographic hash functions explained
LinkA developer downloads a 4.7 GB operating-system image from a mirror. The file looks normal and has the expected size, but neither fact tells the developer whether its contents are correct. A failed download may have damaged a few bytes, or somebody may have changed the file before it reached the mirror. The developer needs a way to compare the downloaded contents with the publisher’s original. Comparing every byte directly would settle the question, but only after obtaining a second trusted copy of the entire 4.7 GB file. That rather defeats the point of using the mirror.
Before distributing the file, the publisher runs an algorithm called SHA-256 over its contents. The algorithm reads the file and returns a short sequence of 64 characters called a digest or hash value. The publisher places that digest on its official website. After downloading the large file, the developer runs the same SHA-256 algorithm locally and gets another 64-character digest. The file does not need to be uploaded anywhere, and the developer does not need the publisher’s full copy. Only the two short results need to be compared.
This comparison works because SHA-256 is deterministic, which means that the same input always produces the same output. If the two digests differ, at least one byte in the downloaded file differs from the publisher’s version. If they match, the developer has strong evidence that the file contents match. The expected digest still needs a trustworthy route. If both the file and the displayed digest came from the same compromised mirror, an attacker could replace both and make the comparison succeed.
SHA-256 is a cryptographic hash function. A hash function accepts any sequence of bytes and reduces it to a result with a fixed size. The cryptographic part means that the function is designed for situations where an attacker may deliberately try to fool the comparison. A secure function makes it unfeasibly expensive to work backwards from a digest or to construct different inputs with a useful matching digest. It cannot make such matches mathematically impossible. There are unlimited possible inputs but only a fixed number of digests, so some different inputs must share a result. The rest of this article follows the calculation step by step before returning to what those security limits mean in practice.
How does a cryptographic hash function work?
LinkThe first step is to turn the input into bytes. A byte is a number between 0 and 255, stored in memory as eight bits. Files, images, and network messages already consist of bytes. Text must be converted with a character encoding such as UTF-8. In UTF-8, the visible letter A becomes the single byte 65, while other characters may need several bytes. From this point onwards, the hash function does not understand letters, pixels, file names, or any other human meaning. It only receives an ordered sequence of byte values. Capitalisation, line endings, character encoding, and an invisible trailing space all change that sequence. If one byte differs, the function is hashing a different message even when a person considers the two versions equivalent.
The next thing to understand is the size of the result. SHA-256 always returns 256 bits, which is 32 bytes, whether its input is the three bytes for CAT or every byte in a 4.7 GB disk image. People commonly write those 32 bytes in hexadecimal notation. Each byte becomes two hexadecimal characters, so the displayed digest contains 64 characters. The fixed size is useful because two large files can later be represented by two short values, but it also tells us that information has been discarded. A digest is not an encoding, because an encoding preserves enough information for a decoder to reverse it. It is not encrypted data either. Encryption uses a key and has a matching decryption operation. A cryptographic hash has no decryption key, and its 32-byte result cannot contain a recoverable copy of every possible input.
SHA-256 cannot mix a multi-gigabyte file in one enormous operation, so the third step is to divide the byte sequence into blocks. Before doing that, it adds padding according to exact rules. This padding is part of the calculation, not whitespace added to the file. SHA-256 appends one bit with value 1, enough zero bits to reach a required position, and a 64-bit value that records the original message length. The padded sequence then fits into complete 512-bit blocks. For example, abc occupies 24 bits in UTF-8. Adding the required padding turns it into one complete 512-bit block. Recording the original length lets the algorithm distinguish messages whose bytes might otherwise end in an ambiguous pattern.
The fourth step is where each block changes the state in memory. SHA-256 starts with eight specified 32-bit state values. It takes the first 512-bit block, derives working values from it, and performs 64 rounds of bit operations. The result of those rounds is folded back into the eight state values. The algorithm then repeats the same process with the next block, using the updated state rather than returning to its initial values. You can picture the state as eight rows of 32 switches. Each processed block rearranges those switches, and the arrangement left by one block becomes the starting point for the next. After the last block, SHA-256 joins the eight final state values to make the 256-bit digest.
This block-by-block process explains how the developer can hash a 4.7 GB file without holding all of it in memory. A streaming implementation keeps the eight-word state and a small buffer for a block that is not yet complete. It reads a file chunk, fills the buffer, mixes every complete block into the state, and then discards those processed bytes. Only the changed state moves forwards. Once all bytes have arrived, the implementation adds the padding, processes the last block or blocks, and returns the final state as the digest. The state is a summary of the calculation, not a compressed archive. It does not retain enough information to recreate the blocks that have already been discarded.
Following a small hash by hand
LinkSHA-256 needs many carefully designed operations, so a complete manual run would hide this movement of data under pages of arithmetic. We can follow the same broad pattern with a deliberately weak teaching function called TinyMix-8. It has one eight-bit state instead of eight 32-bit values, treats every byte as one block, and returns one byte instead of 32. TinyMix-8 starts with state 101. For each input byte, it first rotates the state’s eight bits one place to the left, returning the bit that falls off the left to the right side. It then combines the rotated state with the input byte using exclusive OR. This bit operation returns 1 when the two compared bits differ and 0 when they match.
Hash the text CAT. UTF-8 represents its letters with the byte values 67, 65, and 84. In memory, the working state changes as follows:
initial state = 101 = 01100101
rotate left by one bit = 202 = 11001010
combine with C, byte 67 = 137 = 10001001
rotate state 137 left by one bit = 19 = 00010011
combine with A, byte 65 = 82 = 01010010
rotate state 82 left by one bit = 164 = 10100100
combine with T, byte 84 = 240 = 11110000
The final one-byte digest is 240, displayed as hexadecimal f0. Follow what remained in memory after every line. The first input byte changed the state from 101 to 137, then the algorithm moved on. It did not keep the letter C inside a spare part of the digest. The second byte changed 137 to 82, and the third changed 82 to 240. Once the final byte had been processed, only that final eight-bit state was returned.
TinyMix-8 is not safe for security work. It has only 256 possible results and its mixing rule is easy to analyse, so an attacker can find collisions or work backwards with little effort. It also spreads changes poorly. Replacing CAT with BAT changes one bit in the first input byte, but the output merely changes from f0 to fc. Only two output bits differ. A real cryptographic design aims for an avalanche effect, where a small input change tends to alter many output bits in an unpredictable-looking pattern. That appearance alone is not a security proof. An insecure function can show a dramatic avalanche effect in a few examples, which is why cryptographers describe security as specific tasks that an attacker should be unable to complete.
What do the security properties mean?
LinkStart with preimage resistance. Someone gives the attacker a digest but not the message that produced it. The attacker wins if they find any input that produces that digest. They do not have to recover the original input, which is why casually calling a hash “irreversible” hides part of the problem. For an ideal hash with an n-bit output, a general search needs about 2n attempts. That large number only helps when the possible inputs are also hard to search. If an application hashes a four-digit PIN with SHA-256, an attacker can hash all 10,000 possible PINs and compare the results. The strength of SHA-256 does not make a small set of guesses any larger.
Second-preimage resistance changes the starting point. This time the attacker receives one particular message as well as its digest. They win by finding a different message with that same digest. Imagine a signed contract whose digest has already been recorded. An attacker would like to replace that particular contract without changing the recorded value, and they do not get to choose the original after beginning the attack. For an ideal n-bit hash, the general search again takes about 2n attempts.
Collision resistance gives the attacker freedom to choose both messages. They can keep creating pairs until any two different messages share a digest. The birthday effect makes this search much cheaper than targeting one fixed digest. For an ideal n-bit hash, a general collision search takes roughly 2n / 2 attempts. SHA-256 has a 256-bit output, but its ideal collision strength is therefore about 128 bits. These three tasks sound similar, yet success against one does not automatically solve the others. In 2017, the SHAttered research produced two visibly different PDF files with the same SHA-1 digest. That was a practical collision. It did not make every SHA-1 digest reversible, but it broke the collision resistance on which systems depended. SHA-1 should not be chosen for new designs that need that property, and MD5 is unsuitable for it as well.
How does a cryptographic hash function work in code?
LinkThe TinyMix-8 pseudocode below matches the CAT walkthrough. It uses word-based operators so each memory change remains visible.
; Teaching function only. TinyMix-8 is not safe for security work.
(define initialState 101)
(define tinyMix8 (function messageBytes)
; Memory begins with one eight-bit working state.
(define workingState initialState)
(forEach messageByte messageBytes
; Before this step, workingState summarises all earlier bytes.
(define rotatedState (rotateLeftEightBits workingState 1))
(set workingState (exclusiveOr rotatedState messageByte))
; The state has now absorbed the current byte.
; The caller may discard that byte when it is no longer needed.)
; The final state is the fixed one-byte digest.
(return workingState)))
(define messageBytes (utf8Encode "CAT"))
; messageBytes contains [67, 65, 84].
(define digest (tinyMix8 messageBytes))
; digest contains 240, displayed in hexadecimal as "f0".
Production code should not replace TinyMix-8 with a handwritten version of SHA-256. Cryptographic algorithms are easy to implement incorrectly, and subtle errors may still produce plausible-looking output. Application code should use a maintained cryptographic library. A file-hashing function then has a much simpler shape:
(define sha256File (function filePath)
; The library creates the fixed-size SHA-256 state and block buffer.
(define hashState (createSha256State))
(forEach fileChunk (readFileInChunks filePath)
; The library buffers partial blocks.
; It mixes complete blocks into the state, then can discard them.
(updateHashState hashState fileChunk))
; Finalisation adds the required padding and returns 32 digest bytes.
(return (finalizeHashState hashState))))
The file chunks in this example exist to limit memory use. They do not change the digest. A correct streaming call and a one-shot call over the same bytes produce the same SHA-256 result because the algorithm sees the same ordered sequence of bytes.
Security-sensitive code may also need a library’s constant-time comparison function. An ordinary comparison can stop at the first differing byte. Its running time may then reveal how much of a secret value matched. A constant-time function avoids making the comparison time depend on the position of the first difference. Whether this matters depends on what is being compared and what an attacker can measure, but writing a custom comparison is rarely the right answer.
When are cryptographic hashes used?
LinkThe download at the start of this article is an example of file-integrity checking. A published digest lets people detect accidental corruption or deliberate modification when they obtain the expected value through a trusted path. Package managers, software download pages, and backup systems use this pattern. Calculating the digest still requires reading every input byte once. The saving comes after that calculation, when systems can retain and compare short digests instead of keeping two complete files available. Content-addressed systems build on the same idea by deriving an object’s identifier from its content. Equal bytes produce the same identifier, while changed bytes produce a different one with overwhelming probability. Git traditionally identifies objects using SHA-1 and has documented a transition design for SHA-256. The object name identifies content, but cannot prove who created or approved it.
Digital signatures add the proof of origin that a plain digest lacks. A public-key signature system commonly hashes a message as part of signing it, turning an arbitrarily long message into a fixed-size digest that the signature algorithm can process in a defined way. Collision resistance matters because an attacker should not be able to prepare a harmless document and a malicious document with the same digest, obtain a signature for the harmless one, and then attach it to the malicious one. The hash ties a compact value to the message bytes. The private signing key supplies authenticity. Anyone can calculate a plain digest, so hashing a document without a signing key proves nothing about its author.
HMAC handles a related situation in which two systems already share a secret. It combines that secret key with a cryptographic hash through defined inner and outer calculations. The sender calculates an HMAC for the message, and the receiver repeats the calculation with the same secret. A matching result shows that the message has not changed and that its sender knew the secret. A homemade expression such as hash(secret + message) is not a safe substitute. The order, encoding, hash construction, and message boundaries create traps that are easy to miss, so application code should use a library implementation of HMAC or another established message-authentication algorithm.
Password storage needs a different kind of function again. Plain SHA-256 is designed to be fast, which lets an attacker with a stolen password database test guesses quickly on their own hardware. Adding a salt does not fix that speed. Password-hashing schemes such as Argon2id accept the password, a unique salt, and cost settings, then deliberately consume time and memory to raise the cost of every guess. The salt is not secret and can be stored beside the password hash. It makes identical passwords produce different stored values and prevents one precomputed table from working across every account. Cost settings should follow current guidance for the application’s platform and threat model because suitable values change as hardware improves.
Trade-offs and limits
LinkA cryptographic hash reduces any amount of input to a short value that is easy to store and compare. A streaming implementation needs only bounded working memory, and a secure algorithm makes useful reversal and collision attacks infeasible. The cost of that compact result is lost information. A digest cannot restore the message, hide it, or identify its author. Hashing a large file also requires reading every byte, so calculation time grows with the input length. The bit mixing costs more than the simpler hash functions used by in-memory hash tables, where fast lookup and an even spread across buckets matter more than making reversal infeasible.
The chosen algorithm must provide the property that the application needs. SHA-1’s demonstrated collision does not mean that every old algorithm fails in every possible use, just as a newer name does not prove that an algorithm is safe. Ask whether the algorithm still provides preimage, second-preimage, or collision resistance at the required strength. For a new system, use current platform guidance and maintained libraries instead of choosing an algorithm from memory.
Why collisions must exist
LinkA 256-bit digest has 2256 possible values. That number is enormous, but finite. A hash function accepts messages of many lengths, so it has more possible inputs than outputs. If every possible digest has already been assigned to one message, the next distinct message must reuse one of them. This is the pigeonhole principle applied to digests. TinyMix-8 makes the limit visible because its one-byte result has only 256 possible values. Hashing 257 distinct messages guarantees at least one collision, no matter how well the function spreads its results. Random-looking outputs tend to collide sooner because of the birthday effect. The point of a secure 256-bit hash is that its output space makes a deliberate search impractical, not that collisions have disappeared.
A subtle trap: length extension
LinkSHA-256 belongs to a family of hash constructions where the final digest exposes enough of the internal state to permit a length-extension attack. Suppose an application tries to authenticate a message by calculating SHA256(secret || message), where || means that the two byte sequences are placed next to each other. An attacker knows the message and digest but not the secret. With a suitable guess for the secret’s length, they can reconstruct the padding that SHA-256 placed after the original input. They then use the published digest as the state from which hashing continues, add chosen bytes, and calculate a valid digest for the longer input. The secret never appears in the attacker’s memory. Its effect is already present in the exposed state.
The attack does not reveal the secret or find a collision. It succeeds because the application combined the secret and message in an unsafe way. HMAC avoids this problem with a defined inner and outer construction. It is also worth remembering that other secure hash functions do not all process memory like SHA-256. SHA-3 is based on Keccak and uses a sponge construction, which absorbs input into part of a larger state through repeated permutations and then takes output from that state. SHA-256 and SHA-3 both accept arbitrary-length input and return a digest of a defined size, but their internal steps differ.
Recap and takeaway
LinkA cryptographic hash function reads bytes, divides them into blocks, and mixes each block into a fixed-size state before returning that state as a digest. The same bytes produce the same result, while a small input change should spread throughout the output. Security depends on how hard it is to find an input for a digest, replace one chosen message with another, or construct any colliding pair. If only one idea sticks, remember that a digest is a fingerprint, not a lock or a signature. It can show that bytes changed when the expected digest is trustworthy. Use HMAC or another message-authentication algorithm when a shared secret must authenticate a message, a digital-signature algorithm for public proof of origin, and a password-specific function such as Argon2id for password storage.
Further reading
Link- Hash tables explained
- NIST: FIPS 180-4, Secure Hash Standard
- NIST: Hash functions project
- NIST: FIPS 202, SHA-3 Standard
- RFC Editor: RFC 2104, HMAC
- RFC Editor: RFC 9106, Argon2
- Git: Hash function transition