For those not in the know, or catching any of the news stories that are popping up today in mainstream media, we are in the midst of dealing with a very serious vulnerability that has been discovered in the foundation of secure data transmission on the internet. While many of the news stories out there are filled with some ridiculous hyperbole, it would be dangerous to understate the criticality of what was discovered.

SSL (Secure Sockets Layer) is a protocol for letting your computer and other systems communicate across the internet with negotiated encryption (so people can’t snoop on your passwords and other sensitive transmitted information), and authentication (so you have a way of knowing that when you’re filling in information at your bank’s website it actually is going to your bank’s website). Anytime you’re at a website with “https” in the URL, or that little lock icon in your address bar, your communications are protected by this protocol and code running in your browser and on the server you’re communicating with works on encrypting and decrypting the information flying through the tubes. The SSL protocol was initially developed by our old friends at Netscape in the early 1990s, and is what makes e-commerce and a good portion of our modern economy and communications possible.

The Heartbleed Bug lets any attacker send a somewhat-carefully crafted message to a web server running this SSL code and get back arbitrary contents of the memory within that server. This is, sadly, not an uncommon type of bug (as anyone who has ever programmed will recognize the horror and commonality of array bounds-checking problems and buffer overflow problem). On a web server, however, some things that get returned from memory when it is poked with this attack include:

• The web server’s secret key – This is the key that’s used to actually encrypt all traffic. If you are running a secure website and were vulnerable to this bug, in my opinion, you should assume that your key has been compromised and generate a new key and certificate for encrypting future traffic. Fortunately, due to the “authentication” part of the SSL protocol, in order to take advantage of having a server private key and certificate, you’d have to launch a “man in the middle” attack — which takes a bit more work and often involves actually penetrating the network of your victim and/or hijacking internet DNS service for your victim. Still, this is a very bad thing to leak.
• Sensitive Information – Usernames, passwords, things filled out in forms and submitted to the website by other customers at the time the attack is launched will be present in the server memory in plaintext and can be retrieved. It’s not a bad idea to change your passwords regularly on websites anyways, but this bug might provoke you to go and do it right now
• Session Cookies – Many secure websites keep track of which users are logged in and which aren’t by sharing a little bit of data with you known as a “cookie.” It’s pretty much a magic number that your browser can present to the website to say “hey it’s me again.” The web server will then look it up in the database to say “oh yeah, you logged in successfully a few hours ago, you’re still good.” This is how you can go to websites like facebook repeatedly and not have to enter your password over and over again. Other users’ session cookies will be present in the server memory in plaintext and can be retrieved by this attack. This is called “sidejacking” and is (in my opinion) the most frightening aspect of this bug. This blog has a more detailed example of using this vulnerability to do a sidejacking, and confirms that this is possible on at least one “fairly popular website”

This bug was disclosed in what we call a “responsible” manner. The researchers that were supposedly first to discover it did not release it to the public, but went directly to the OpenSSL project and, in turn, large stakeholders were notified several weeks ago. It can be assumed that sites like Google, Facebook, Akamai (which is good because they actually terminate a good portion of the web’s SSL — including TripAdvisor’s), and hosting providers like CloudFlare have already repaired the vulnerability before yesterday. Sadly, it appears that the publication of the vulnerability on April 7th was earlier than hoped. Linux distribution providers (Debian, CentOS, Redhat, Ubuntu) who provide the OpenSSL code packages that people like me actually have to get to install on our web servers, were not providing a fix in some cases until late in the evening on the 7th — well after exploit code was in the wild. Furthermore, while I trust the researchers listed as the discoverers of this bug, I can not (nor should anyone) be 100% certain that someone else hadn’t already discovered this problem and has been attacking websites with it for several months stealing private keys and sensitive information and credentials. So while it’s comforting that responsible disclosure and fast action on the part of the people that run the web sites you visit every day (people like me) have potentially mitigated the problem, the consequences of this vulnerability are (as you can see in the list above) far reaching and somewhat frightening.

“So as a regular person, how worried should I be?”This is a common question a lot of people have been asking in the past day or two. I can’t pretend to understand your own risk and paranoia level, but I will attempt to convey how I feel. This is not a reason to stop trusting the little lock icon in your browser or the “https” in the url. Bugs happen, sometimes information is leaked, and then they get fixed. Any damage done by this has already been done and there’s no reason to yank out your ethernet cables and delete your facebook and twitter accounts. What you should do (and should be doing already) are some common sense web security techniques. If there’s a bright side to this bug, it’s that this may increase everyone’s awareness and get people do to the following:

• Change your passwords: This is a no-brainer. If anyone gets your account information (through this vulnerability or any other means), it’s useless if you change your passwords. I do this every few months.
• Don’t use the same passwords on multiple sites: This is a common problem. Here at TripAdvisor the only thing your password protects is a bunch of travel reviews. You may think “oh whatever, big deal.” But research (and anecdotal evidence) shows that many people use the same exact password and username on many sites. The same username and password a user uses on TripAdvisor may very well be their gmail password, or the password for their online banking, or facebook or twitter. Websites get hacked all the time (none that I’m responsible for, of course, LOL [yes, I just typed LOL]) — sometimes without the public even knowing about it. So be smart. Even I don’t use a unique password for every website, but I have a set of four or five that I use for different classes of sites (social media password, email password, financial services password, shell login password, etc.).
• Pick a good password: People have been saying this forever, but I will say it again. Quick story: when I was at UIUC running the campus Email and UNIX shell/file sharing services, we first ran a password cracker against our users’ accounts. The way that these “brute force” attacks work is that an attacker will attempt login using dictionary words, names and other things. The most common password, by far, was actually password. Among the top 5 were also fuckyou, ncc1701, various people’s names (obviously people choose their girlfriend/boyfriend/mother/father’s names for passwords), and in several dozen cases people actually used their usernames as their passwords. These days many websites will prevent you from using a weak password. So don’t be dumb. Pick a good password. It should not be dictionary-word based. Even replacing numbers with letters is easily decoded by brute-force attackers, so don’t think you’re fooling anyone. Don’t use anyone’s name in your password either. And don’t even use a combination of dictionary words, names, and l33t-sp34k numbering. The brute-force password crackers are at least as smart as you and have a lot more time and computing power.

So as a website operator or systems engineer what should I do? You should act immediately if you have not already. If you run your own web server, upgrade your OpenSSL package right this goddamn minute. Also, since the library is loaded in memory at service-start time you will need to restart your web server or any other service relying on the flawed library. To be safe, just reboot after you upgrade the package. There also might be code that was built statically-linked to the flawed library. In that case you’ll have to recompile and re-install it. Run common vulnerability scanners like nessus (or other tools available) against everything you have running. If you have a website that’s hosted elsewhere, contact your hosting provider immediately. Make sure they are patched and no longer vulnerable. Also, replace your SSL key and certificate. Some will say that this step is overly paranoid, and your hosting provider might even give you shit for insisting that they generate a new key and certificate for you. As I stated above, while these researchers responsibly disclosed this bug, the possibility that this was out in the wild before can not be dismissed.

Timeline:

• December 2011: Bug is introduced into the hearbeat function of the OpenSSL library
• March 14th 2012: OpenSSL v1.0.1 released into the wild with the bug
• March? 2014: Bug is discovered by some combination of Neel Mehta at Google Security and Matti Kamunen, Antti Karljalainen and Riku Hietamäki from Codenomicon and reported to the OpenSSL project.
• >March-April 2014: NCSC-FI and OpenSSL work to notify some subset of stakeholders ahead of time of the vunerability, apparently with a patch and a workaround
• April 7th 2014: News breaks of the vulnerability and the NCSC-FI team needs to go public with it so the rest of the world can fix their web servers