Say you want to send an important file, how do you ensure that the receiver will get the exact file without any changes ?
Well, the most common approach is use hashing, which translates the file information into fixed sized values, and you can compare these hash values with the original hash values to ensure the file authenticity.
In this guide, we’ll use the most common hashing algorithms that comes pre-installed with any Linux distribution. Below are the hashing algorithms we gonna use in this tutorial:
What is Hashing
Hashing is an algorithm used to calculate a fixed-size bit string value from a file. The hash value is a summary of the original data. The file contains a block of data, and hashing transforms this data into a shorter fixed-length key or value representing the original string. A hash is a unidirectional process, and you can’t work backward to get back the original data.
The file hashing is done on the web servers by the web administrators, who then publish the hash results. The users download the files and then apply the same hash method. The two files are identical, and there is no file corruption if the checksum of the original file and the downloaded file are the same.
What is a Checksum
A checksum is a value used to verify the file integrity in data transfer. Verifying a checksum helps to ensure that there is no corruption or manipulation in the downloaded file. Hashing algorithms like MD5, SHA256, SHA512 hashes are commonly used for checksum verification. Checksums only verify the file integrity. It does not provide any encryption or confidentiality.
There are many hashing algorithms, but in this guide, we will be using some of most common types of hashing used for file integrity and checksums in Linux.
SHA256 algorithm generated a unique, fixed-size 256-bit hash. The
sha256sum command is used to check file integrity and authenticity. It computes and checks SHA-256 encrypted message digest on Unix-like operating systems. You can use this command in binary and text mode, and both of these modes generate the same SHA-256 hash on Linux.
$ sha256sum [OPTIONS] /path/to/file
The following table shows the options of this command:
|-c, –check||Read and check SHA256 sums from the FILEs.|
|-b, –binary||Read in binary mode.|
|-t, –text||Read in a text mode.|
|–tag||Create a BSD-style checksum.|
Generate Hashes for Files
As an example, let’s create a simple text file:
$ echo "Hello World" > data.txt
The following command gives the SHA256 checksum of the
data.txt file in the current directory:
$ sha256sum data.txt
Verify File Integrity
To verify file integrity of your downloaded file, you can use the
--check parameter in
sha256sum command. Let’s first create the checksum file with the previous command:
$ sha256sum data.txt > checksum
Now in the real world, this
checksum file need to be downloaded along with the actual file, an example of this is when installing Ubuntu image, you can download the SHA256 checksum files along with the image to verify whether the file you have downloaded in your machine is the exact same as theirs.
$ sha256sum --check checksum
This will look for files listed in that checksum file, and calculates the hash for each of them and compare, here is the output:
Another alternative is to use OpenSSL library:
$ openssl sha256 data.txt d2a84f4b8b650937ec8f73cd8be2c74add5a911ba64df27458ed8229da804a26 data.txt
Let’s add a character to the file and run the checksum again:
$ echo "!" >> data.txt $ sha256sum --check checksum data.txt: FAILED sha256sum: WARNING: 1 computed checksum did NOT match
If you wish to hash your password with SHA-256, a good tool is
mkpasswd command, if it isn’t installed in your machine yet, make sure you install
$ sudo apt-get install whois
After that, as an example, let’s hash
$ mkpasswd -m sha-256 "password" -S '12234567'
-S is the optional parameter for the salt. You can always use
mkpasswd --help or
man mkpasswd for details.
SHA-512 is the hashing algorithm to perform hash functions on data given to it. It is the strongest among all SHA-2 hashes and generates a unique, fixed-size 512-bit hash. The
sha512sum command behaves the same as sha256sum and it computes and checks the SHA512 message digest.
Generate and Verify Hashes
It is the same process as
sha256sum, you basically change
Below is OpenSSL command for SHA-512:
$ openssl sha512 data.txt SHA512(data.txt)= e1c112ff908febc3b98b1693a6cd3564eaf8e5e6ca629d084d9f0eba99247cacdd72e369ff8941397c2807409ff66be64be908da17ad7b8a49a2a26c0e8086aa
mkpasswd supports several algorithms and SHA-512 is one of them, this time, we use
sha-512 as method type instead of
sha-256, and it behaves the same way too:
$ mkpasswd -m sha-512 password -S '12234567' $6$12234567$T3yqNqxLLLLmY/.JfFhtzQhHOi7uyPg92V0gHMORUoV5jCodNpYo7v9QewZLdb2yM4AHNM5zVdBFA/Yzv8Cq9/
MD5 is a message-digest algorithm that implements a cryptographic hash function used for message integrity checks. This algorithm creates a unique, fixed-size 128-bit cryptographic hash from data input. The major difference between the Windows and Linux security features is salt strings. The salt string randomly generates a value that is stored with every password. It allows additional security for every password within the system.
Generate and Verify Hashes
md5sum command handles that for us:
$ md5sum data.txt e59ff97941044f85df5297e1c302d260 data.txt
$ openssl md5 data.txt MD5(data.txt)= e59ff97941044f85df5297e1c302d260
BLAKE2 is a cryptographic hash function faster than SHA256, SHA512, and MD5. It has two different types, BLAKE2b, and BLAKE2s. BLAKE2b is optimized for 64-bit platforms, and BLAKE2s is optimized for 8 to 16-bit platforms.
Generate and Verify Hashes
The command is the following:
$ b2sum data.txt 01cc021b7aaf1ed89304c61cc48c455b2a4e500812aa9b6ff4256372f9882a6fd4df0291f70f9883268a978e615e540aab12b3f7e0b4b32914e515665aa05edb data.txt
BLAKE2s with OpenSSL:
$ openssl blake2s256 data.txt BLAKE2s256(data.txt)= cd0b3b1832c2460e30bc924ebba398f5bf6447478c2532995d4657707340c09c
$ openssl blake2b512 data.txt BLAKE2b512(data.txt)= 01cc021b7aaf1ed89304c61cc48c455b2a4e500812aa9b6ff4256372f9882a6fd4df0291f70f9883268a978e615e540aab12b3f7e0b4b32914e515665aa05edb
SHA-256 and SHA-512 are in the same family of SHA-2, there are also SHA-224 and SHA-384 functions, you can use
sha384sum normally as well.
SHA-2 is the successor of SHA-1 which is now obsolete and easy to break, the motivation of SHA-2 is to generate longer hashes which leads to higher security than SHA-1.
BLAKE2 hashes are faster than SHA-1, SHA-2, SHA-3 and even MD5 as previously mentioned, and it is even more secure than SHA-2, it is widely used in software including OpenSSL, Noise protocol, and much more.
Even though this tutorial walked you through using MD5, but you shouldn’t use it nowadays, as it is pretty obsolete now, and you should never use it, as it isn’t collision resistant.
Hashing technology is useful to verify if the files are copied correctly between two resources or not. It is amazing how easily you can verify the authenticity of a file with just one command. However, IT is a fast-changing industry, and the security risks associated with technology are also increasing. To keep the security standards high, you must follow the advanced and latest hashing algorithms for security purposes.