The Hash Function
Continuing the Google Cybersecurity Certificate, today, I discuss the Hashing, an integral part of the Cryptography. The topics covered in this writeup are:
Hash Function
Evolution of hash functions
Creating hash values
1- Hash Function:
A hash function is a one-way mathematical function that outputs a fixed-size digest that cannot be reversed.
They are used to convert information into a unique hash value that can be used to confirm the integrity of the file.
Hash functions are designed around something called the avalanche effect. If you take a 500-page book and change a single comma , to a period ., it will change the hash values.
Data Integrity: “Makes sure that data has not been altered or tampered with in transit or storage.”
By finding the hash values of the files and comparing them with those of potentially malicious files, data integrity can be ensured.
How tools like VirusTotal help in identifying viruses using the hash values:
Once you find a suspicious file on your computer, you can run tools like sha256sum in the terminal to generate its unique hash value. You can search that hash on VirusTotal. By comparing your file's hash against VirusTotal's database of known hashes, you can quickly verify whether the file is malicious.
2- Evolution of hash functions:
One of the earliest hash functions is the Message Digest 5 or MD5. Developed by Professor Ronald Rivest, it ensured that the file sent over the network was the same file that was being received by the destination device.
Since computers only understand binary code (0s and 1s), any file, whether it’s an email or a document, when it passes through the MD5, it converts it in such a way that it outputs a 128-bit string.
To make this 128-bit raw output easier to read, the algorithm splits the binary string into 4-bit groups. Each 4-bit group translates to a single hexadecimal character (0–9 and a–f). This turns the 128-bit binary output into 32-character hexadecimal string like 5d41402abc4b2a76b9719d911017c592.
Hash Collisions and the limitations of MD5:
Hash collisions happen when the outputs generated in the hash table are the same and not unique anymore.
MD5 produces a 128-bit binary hash. Because 128/4 = 32, it renders as 32 hexadecimal characters.
Since a core purpose of hashing is to validate the integrity of the file, two files having the same hash can be used to impersonate and is a security risk.
The new algorithms that are collision-resistant:
Secure Hash Algorithms (SHAs) produce longer, more secure hash values to face modern attacks. The collision-resistant SHA algorithms and their digest bit lengths include:
- SHA-224 (224 bits)
- SHA-256 (256 bits)
- SHA-384 (384 bits)
- SHA-512 (512 bits)
- SHA-1 (SHA-160) is cryptographically broken and was retired by NIST.
How passwords are stored in the database using Hash Values:
In a database, usernames and passwords are mapped together. When a user logs in, the password they type is compared to the one stored against their username. The passwords are stored as hashes rather than plain text. This authenticates the user while making it significantly harder for attackers to recover plaintext passwords, even if the database is breached.
Get Tehreem’s stories in your inbox
Join Medium for free to get updates from this writer.
Since hashes cannot be reversed, this adds another layer of security to user credentials.
The Rainbow Tables:
Rainbow tables are used by hackers and attackers to compare the password hashes with the hashes in the table. They are tables of plaintext-to-hash mappings used to reverse hashes quickly.
Salting the Hash Function:
To secure the hashed password even further, salting is used.
A salt is a random string of characters that’s added to data before it’s hashed.
For example, a database containing passwords might have several hashed entries for the password “password.” If those passwords were all salted, each entry would be completely different. Even if the Rainbow Table was used, the actual password would be harder to get.
3- Creating hash values:
In this lab, I perform the process of generating hashes for files with little content differences to demonstrate the avalanche effect and then compare the files.
Task 1. Generate hashes for files:
In the directory hashing , the ls command shows two files file1.txt and file2.txt . Upon using the cat command to display the contents of both files, only difference we see here is the capital T and small t.
Now we use the sha256sum command to generate hashes for both files and check the differences.
Task 2. Compare hashes:
Here I used the cmp command to compare file1.txt and file2.txt directly. The output differ: byte 4, line 1 showed the exact character where the files differ by content: byte 4, which is the capital 'T' in 'There' vs the lowercase 't' in 'there'.
So, in this lab, I have covered:
- Computing hashes using
sha256sum - Comparing hashes using the
cmpcommand
