Checklist 44: A look at encryption, past and present.
This week we take a look at the recent history and current state of encryption.
For as long as we’ve been able to communicate, we’ve wanted to keep some of our communications secret. That’s made encryption important throughout history – never more so than today.
It is no longer something limited to governments and top secret documents, either; we use encryption every day, even if we don’t notice it, and we can even go one step further now. We can personally encrypt our own data and all our communications if we want—if we’re willing to sacrifice some small convenience.
Should you? And how does it work? Those are the types of questions we’ll answer on this week’s episode of The Checklist. Encryption in all its many forms can be a tough beast to understand. We’ll be breaking down some of its history and the ways you encounter it on the web. So without further ado, let’s get started by looking at where digital encryption came from and where it’s headed now.
- Why do we need encryption and when do we use it?
- Early encryption methods and what we use today.
- Encryption on the web.
- What happens when encryption breaks?
- Encrypting your data and communications.
Why do we need encryption and when do we use it?
As we mentioned earlier, encryption is not new. We’ve been figuring out ways to hide messages and information for strategic or personal purposes for centuries. In the past, we relied mostly on ciphers — substituting letters for one another via an often-complex process. While that might have been fine for Roman soldiers sending written messages back to their commanders, computers make cracking most old ciphers a breeze. We needed to develop new ways to deal with the challenges created by easy access to computational power. Why, though?
The answer is simple yet important. As computers grew from being machines used by hobbyists to public tools of incredible importance, businesses incorporated them into their operations more too. The development of networks and the emergence of the Internet means that we don’t just store information locally, it’s often stored somewhere else, somewhere we don’t control. Because of that, someone else might be able to access it instead of us. For the most part, early encryption efforts and standards were the sole domain of governments. No one wants to transmit Top Secret information in the clear. Over time, though, the public began to seize on its importance too.
Encryption is the answer for everything from protecting private files from prying eyes to safeguarding financial transactions conducted over a network. With computer systems holding so much valuable information, it was a natural next step between the 70s and 90s to develop ways to protect data.
With the rapid development of viruses and the explosion of malware that began in the late 80s and early 90s, encryption became even more important. Revelations about governments snooping on activities by individual Internet users, coupled with demands to reveal information, led to some developments we’ll discuss.
We’ll go into how some of this works today, but it’s worth knowing that we already encounter encryption daily in a few ways. Do you manage your banking online? You’re communicating with the bank using encryption. Are you blasting out messages to your friends from Facebook Messenger or WhatsApp? What about just visiting a site like Google or Amazon? You’re encountering encryption there, too. For others, it’s less passive and more active: encrypting emails, hard drives, and more.
In just about every case, you’re dealing with information you don’t want anyone else to see. Behind the scenes, a massive number of mathematical calculations goes on to make all that possible. There are keys, handshakes, and all kinds of other jargon to explain — but the good news is that for the most part it “just works” to provide us with peace of mind. With the many things we use our phones and computers for today, it’s a good thing, too! It wasn’t always that way; it took some time to reach this point.
Early encryption methods and what we use today.
Okay, so it’s one thing to understand why need encryption and where we find it — but where did it all start? We didn’t jump straight to the fortress-like 256-bit encryption commonly employed to protect sensitive data today. It was a long evolution, spurred forward by both public and private developments. It’s taken the efforts of many bright professionals, mathematicians, and scientists — plus a good number of amateur cryptographers — to bring us to today’s standards for encryption.
Speaking of standards, that is a good place to start when tracing some of the first developments in digital encryption. Obviously, this is a subject that could fill several books, so we can’t cover everything in today’s episode, but there is a ton of reading material out there for those curious. That said, let’s zero in our first look at the development of an early government standard for encrypting sensitive data, known as DES or Data Encryption Standard. It made its first appearance in the 1970s at the hands of IBM.
Before we discuss the generalities of how DES works, let’s sort out some terminology. DES is what is known as symmetric encryption, and it uses “keys” to help unlock data encoded by someone else. In a symmetric encryption algorithm like the DES, the same “key” is used to both encrypt and decrypt a given piece of data. This key could be a password or some other identifying information.
If the key remains well protected, the only way to break through something like DES is to undertake a brute force attack. There’s a lot more that goes into symmetric key cryptography — otherwise, you would rightly think it might be easy to break. It’s interesting to learn about DES, but it gets heavy with the jargon fast. Even so, you don’t need a degree to understand how these systems work!
Okay, back to DES, which we know is symmetric encryption. Why’s it so important? Because it was the first major standard for encryption, it set the stage for a lot of how we practice digital encryption today. Even at that time, more than a few people were skeptical about its design, with stories of NSA involvement in perhaps making the algorithm weaker than it could be. Ultimately its design ensured it would not stay in active use in serious applications beyond the end of the 90s.
DES uses a key length of only 56 bits — compared to today’s standards which often rely on 256 bits or more. This short length meant that, as computers became faster, executing a successful brute force attack on a DES-encrypted file became far easier. By 1999, a DES key could be brute forced in under a day. Though there is a variation that uses three encryption passes to improve security, it’s still not in common use anymore.
By the mid-2000s, we had a replacement to really lock down our data. Called AES, or the Advanced Encryption Standard, it took the DES algorithms and refined them to an immense degree. More than that, AES did away with the tiny 56-bit key and replaced with the ability to create variable key lengths. AES supports keys that are 128-bit, 192-bit, and even a whopping 256 bits. Brute forcing such a key would take an almost uncountable number of years even with some of today’s strongest hardware on your side. Though there are some theoretical problems, AES is unbreakable when we use it correctly.
Encryption based on AES, especially AES-256, is extraordinarily common today and the foundation for much of our secure communications. However, there are other ways to communicate securely, although they require a different approach. Besides symmetric encryption, we also have asymmetric methods too. If you’ve ever heard of “Pretty Good Privacy”, typically just called PGP, it was a major innovator in the asymmetric encryption space. Let’s break down what that means.
There are two keys in asymmetric encryption: a public one and a private one. You might, for example, share your public key with friends. You use this key to encrypt a message. (How we know your key is valid is another can of worms, but there are solutions in place.) On the other end, a key known only to the recipient — the private key — decrypts the message. The most common implementation of this methodology is RSA, and it uses keys between 1024 and 2056 bits. Asymmetric encryption is the technology behind much of the encryption on the web.
Encryption on the web.
With that in mind, let’s turn our attention now to the more practical side of encryption, rather than just focusing on the methodologies used to make it work. We use encryption in a lot of ways, and not only to encrypt our hard drives or to exchange messages directly with another person. Cryptography is very important across the Internet, too.
Don’t forget: there are plenty of hackers out there who would love to spy on all the information passing back and forth between you and the servers you interact with on the web. That data might include your passwords, credit card details, and more. Meanwhile, websites need a way to verify the authenticity of your passwords — and they can’t just store them in clear text, either. All these functions rely on encryption or a related process.
What related processes? Let’s start by talking about hashing. Though it is not technically a type of encryption itself, it’s so closely related we must mention it. We covered some info on hashes back in our episode about authentication. For a full rundown on how hashing works, we suggest you check it out! However, for those of you who missed it, we can explain it in quick and simple terms.
We take some piece of data, like your password, and run it through an algorithm that performs a large number of complex mathematical operations to spit out a final result, called the hash or hash value. That hash value is unique to the data being hashed. So if the data changed, so would the hash. A website can then store only the hash of your password. Later, when you go to log in to the website later, it creates of hash of the password you entered – and if it matches the hash value stored in their system, the password must be a match, and you’re in!
Hashing is also used to digitally “sign” things, which is a guarantee of authenticity. Signing is important for everything from software distribution to sharing sensitive documents. If the hash value does not match, you know it is not a legitimate copy — something has been added or altered. Hackers will go out of their way to try and circumvent this protection method. Every day, you’re probably encountering hashing without even knowing about it!
Sometimes, it’s possible to break these hashing functions, though. This is called a collision: a piece of data that can produce a hash value identical to an entirely different piece of data. The more collisions possible in a hash function, the less secure it is; we’ll touch on that more when we talk about the broken MD5 and SHA-1 functions today.
You use encryption daily as you browse the web, too. We often recommend you always check to make sure you are on a “secure connection” when logging in to a site or making a purchase. The easiest way to tell, of course, is to look for “HTTPS” or a lock icon in your browser bar. HTTPS is short for “HyperText Transfer Protocol” (the language of the web) over SSL (encryption). However, as encryption continues to evolve, SSL connections are migrating to Transport Layer Security connections, or TLS.
TLS is a multi-step process that allows your computer and a website, or other applications, to talk to each other while verifying the authenticity of the data exchanged. TLS works through a combination of hashing and public key encryption, like the kind we just explained. By exchanging keys and verifying one another, it is much more difficult for a hacker to execute a “man in the middle” attack.” After the initial exchange of information, you create a secure communications channel to the website. A new, unique, and one-time public key encrypts all the data transmitted between you and the site so no one can access it.
You may even be sending messages to your friends securely without knowing. We’ll dive into this deeper into a moment, but many apps — like Facebook’s Messenger — now employ what we call “end to end” encryption. From the moment data leaves your devices, it’s already been scrambled and remains that way throughout its transmission. In an era when digital snooping is on the rise, and we often hear concerns about government surveillance, end to end encryption is the future of personal cryptography. These three examples are just a few of the ways we encounter encryption, but when you start to pay attention, you’ll begin to see them in practice everywhere.
What happens when encryption breaks?
Wherever there is a lock, there will be a thief who wants to figure out how to defeat it and find out what it secures. The same goes for encryption. The breaking of DES and its eventual replacement in favor of the AES system is a good example of how the methods we use today won’t always work as well tomorrow. What we might consider “uncrackable” could turn out to be very vulnerable — once technology advances to the point where it can handle the complexity, at least.
For the most part, we can trust encryption to keep our data safe. Sometimes, though, we need to adjust rapidly to changes in the industry, especially when we find weaknesses in an encryption method. We’ve also become painfully aware over the past few years how hackers can use encryption again us through the deployment of things like ransomware. Thinking on that, what are some of the ways encryption has failed us and what do we need to consider when choosing solutions for ourselves?
Let’s start by going back to the topic of hashing algorithms. We mentioned that some hash methods have vulnerabilities called collisions. Over the years, we’ve had to discard more than one hashing algorithm due to numerous collisions. Detecting vulnerabilities is an important part of using encryption: once we find weaknesses, it’s important to fix or move away from those standards as quickly as possible. Otherwise, we risk attacks. Such was the case with the MD5 hashing algorithm.
Vulnerabilities in MD5 were known as early as 1996, but it wasn’t until 2004 that a working collision attack was published. It wasn’t long after that where MD5 was no longer recommended for serious security applications by anyone; computational power was now cheap enough to make it a trivial matter to generate an MD5 collision. Combined with how quickly MD5 hashes compute, it is simply no longer ideal for either hashing passwords or for generating security certificates.
Thus, we migrated to other hashing algorithms that do not yet have feasible attack methods, like the SHA-1 algorithm. That’s not to say MD5 is completely useless now — but for serious applications, it’s not worth the risk. Of course, earlier this year, the first major SHA-1 collision was detected — something predicted for several years. Don’t worry, though: we have newer, stronger standards, like SHA-256, to take its place as we go forward. In fact, many applications which use SHA have implemented SHA-256 simply because it was available already and was a stronger algorithm than SHA-1.
The dimise of MD5 caused plenty of problems. Websites still using MD5-hashed passwords have sometimes suffered some serious breaches. Besides that, though, we haven’t had many major security problems due to broken encryption. Most services migrate well before anyone functionally breaks an existing encryption standard. More often, we need to worry about encryption weaponized by malware authors.
When ransomware hits your machine, it rapidly encrypts your files so the authors can demand payment from you. Many people ask if there is anything you can do after ransomware has infected a machine, but the unfortunate answer is almost always “no”. Why? These programs typically rely on AES-256 to conduct their encryption. As we know from our earlier discussion, AES-256 is functionally unbreakable today. So once your drive is encrypted by ransomware, you’re locked out. And without the key, you won’t get your data back. It won’t matter if you’ve encrypted your own files, either — ransomware can just encrypt them again.
This threat is why it is so important to keep your systems updated and to use malware protection at all times. It also emphasizes the need to have recent backups of your system! As one of the biggest problems on the web today, and in the wake of WannaCry, it is worth reminding our listeners again just how important it is to pay attention to these issues. While Mac ransomware is still relatively rare, we can be sure that someone out there is working to figure out a way to make more!
Encrypting your data and communications.
Earlier, we talked about some of the ways we encounter encryption in our daily lives. More websites default to using HTTPS than ever before, in large part as a response to the changing malware threats we face today. You don’t have much of a choice in whether you use these things; it’s a basic practice on the web. However, that doesn’t mean you can’t go the next step and start employing encryption in other areas of your life as well. Though you might not be a journalist trying to protect your sources from exposure, even the average computer user can gain some peace of mind from trying out these technologies.
Encrypting your data, the stuff that resides on your hard drive, is a popular step to take these days. Not only do you secure any personally identifiable information from prying eyes, but you can also ensure that you’re the only person with legitimate access to your backup files. For Mac users, Apple makes it easy to deploy strong encryption on your files through FileVault. We’ve got another episode of The Checklist waiting for you in our archives where we cover everything about using FileVault effectively — check it out and see if it’s the right choice for you. Don’t forget to remember your encryption password, though. Much like ransomware, without that password, you cannot access those files!
It is easier than ever to encrypt your communications, too. In the past, we had to rely primarily on systems like PGP or the instant messaging encryption protocol known as Off the Record Messaging. Today, you don’t need to install an additional plugin or go through a complex setup process. We mentioned Facebook Messenger as being one app that employs end to end encryption — but of course, how do we know what Facebook does with our messages internally?
That’s always one risk to relying on a third party for handling your encryption, but there are other good apps out there that you can use with greater confidence. WhatsApp and Signal are two, though WhatsApp has certainly had some stumbles in its forays into E2E encryption. Signal prides itself on providing a very secure platform for users to communicate with encryption on a mobile platform. We’ve discussed the pros and cons of several these apps in the past as well; if you’re interested in employing them yourself, we encourage you to do plenty of homework first.
You can also still use asymmetric encryption to communicate securely over email. However, employing these methods can require an increased amount of setup. What those methods are and which personal encryption solutions are best for a given application can vary. Something for a future episode, perhaps!
Well, we’re a long way from the relatively care-free days of early Computing, when the old joke was that computer security meant locking the door at night. Now, protecting our data and our personal information from thieves and hackers requires cryptographic protection. Luckily, we are in the midst of many advances in the field, and more software authors adopt rigorous encryption methods every day. Though its political future might be uncertain, one thing we do know for sure: it’s the best way to achieve privacy for your data.
As always, we thank you for joining us for another weekly edition of The Checklist. We encourage you to learn more about the various encryption methods that are available — it’s only going to become more important over time!
