We Warned Media Temple About The Need to Keep Plesk Up to Date in 2010

Ars Technica has been running a series of articles about the recent hack of several US Federal Trade Commission websites hosted at Media Temple. The latest reporting indicates that “critical vulnerability in some versions of Parallels’ Plesk Panel control panel software appears to have been key to the recent penetration”. A patch for the vulnerability was released in September, though the article mentions that Parallels didn’t send email alerting customers to the critical nature of the vulnerability until February. Media Temple’s failure to keep Plesk up to date goes back years and we unsuccessfully tried to get them to address the issue back in July of 2010.

Media Temple and Parallels are both part of the Hosting Security Forum, which is supposed to “share critical security information in order to protect the integrity of a customer’s data, their web presence, and online availability”. That organization’s website is running an outdated version of WordPress, 3.2.1, which might be a good indication as to the group’s level of dedication to security. Media Temple, as well as Dreamhost and other members of that organization, is also running outdated software on their website.

Warning Media Temple

Back in July 2010, during a period of hacks that were targeting Media Temple customers (but that Media Temple claimed was not due to their security failings), Media Temple  made some long needed security improvements and asked for people to contact them if they were missing any security measures.

While cleaning up a hacked website running on their Dedicated-Virtual service we noticed that Media Temple was using a nearly two year old version of Plesk, which also meant that the other software that comes with Plesk was also two years old. We contacted Media Temple alert them to the need to keep that and other software running on their systems up to date and at the time we hoped that they would quickly resolve the issue as they were publicly claiming to want to improve their security.

Media Temple’s response was that there hadn’t been a known vulnerability in Plesk since 2007 and therefore they were secure. We don’t why they felt they didn’t need to keep their up to date just because there had not been known vulnerabilities, but in any case it wasn’t the whole truth. While the version of Plesk didn’t have any known vulnerabilities other software that came with that version of Plesk did have known vulnerabilities. We specifically brought to their attention there were apparently security issues in at least the versions of ProFTPd, Ruby, and phpMyAdmin that came with it. We never received any response after we brought that up. Overall, their response seemed to be more focused on creating the impression that they cared about security then about actually making sure they and their customers were secure.

During the email exchange they claimed they had recently put in place a policy that “requires us to patch any software with security flaws within 30 days of a patch being made available.  For the most critical issues, such as a kernel exploit, we will patch immediately. ” We certainly would describe the vulnerability in Plesk as being a critical issue, but for some reason they didn’t feel the need to apply the patch immediately or even in the 30 day window.

Blame the Customer

When the issue of the hack was first surfacing Media Temple was quick to blame their customer for the hack and criticize them for running outdated software. While it’s true that many hacks are due to issues which the customer is responsible for and many of those are due to outdated software, it is irresponsible to claim that it was the customer’s fault without actual evidence to support that, especially to do that publicly. For Media Temple’s to do this is worse as during our email exchange they excused not keeping Plesk up to date on there not being known vulnerabilities, so they certainly should understand that just running outdated software doesn’t mean that it is vulnerable. To be criticizing a customer for running outdated when they don’t keep the software they are responsible for up to date makes the response appalling.

Unfortunately, Media Temple’s response to this incident isn’t out of line with the usual response that customer’ with hacking issues receive when contacting their web host about a hacking issue.

Media Temple Runs Outdated Web Software

Based on the rest of Media Temple’s actions it isn’t surprising that they fail to keep software running on their website up to date (while criticizing others for doing the same).

Media Temple Blog WordPress VersionTheir blog is running WordPress 3.3. The latest version, 3.3.1, was release nearly two months ago and included a “fix for a cross-site scripting vulnerability that affected version 3.3“.

Media Temple Community Wiki MediaWiki VersionThe Media Temple Community Wiki is running a version of MediaWiki, 1.16.x, that hasn’t been supported for nearly three months. They also failed to apply the last three updates, all of which included security fixes, for 1.16.x. The oldest update they failed to apply, 1.16.3, was release over ten months ago.

You can get alerts for outdated web software, like the one the ones in the screenshots above, with Meta Generator Version Check extension (available for Firefox and Chrome).

Dreamhost Shouldn’t Be Giving Others Security Advice

We don’t believe, as others do, that web host’s have a responsibility to clean up their customer’s website if it’s are hacked, unless it was caused by something on the host’s end. They do have responsibility to properly handle the security of their own systems, which as of just a few weeks ago Dreamhost didn’t do in a basic way. We also think they have a responsibility to not be giving bad security advice to their customers or the public at large, as Dreamhost does. It does a serious disservice to their customers, including the potential that customer’s hacked websites could remain vulnerable to being re-hacked because of their bad advice. The bad security advice, which certainly isn’t limited to web hosts, also makes it harder for people that deal with security issues, and take security seriously, to do their job. We have to spend to much time explaining to clients that the information they are seeing and being told is in most cases wrong. What makes Dreamhost worth focusing on is that in addition to the bad advice they give, they don’t bother following the good advice they give others.

In this post will focus mostly on comments they made in a recent post on their blog and some communication between them and one of their customers, whose website we recently cleaned.

From the blog post:

Based on the sites we have cleaned up already, these attacks have almost universally been due to insecure website software running on the site in question. You could have the best passwords in the world, but if the apps you’ve installed on your server have any security vulnerabilities or aren’t kept up to date, attackers can still find their way in.

Based on the websites we clean up and other data we collect the claim that attacks are “almost universally been due to insecure website software running on the site in question” is completely false. Vulnerabilities in web software are one of the top two things, but the other, compromised FTP credentials, has nothing to do with the software running on the website. From what we see, which one of issues is the biggest changes from time to time. For example, we might see a spike in web software attacks after a new vulnerability is discovered. There are other things that are lesser but not insignificant causes of attacks, like security vulnerabilities at web hosts.

What could explain the discrepancy between what we and Dreamhost see? It could be that they have systems in place that are effective at preventing other types of hacks. It could be that they only look for certain types of hacks, so they only find what they are looking for. Or it could be that don’t actual bother looking at the cause of attacks at all.

Determining if Outdated Web Software is the Cause of a Hack

Recently we were cleaning up a hacked website hosted at Dreamhost. From reviewing the information we had available it didn’t look the hack came through a security vulnerability in web software. Unfortunately, we couldn’t review the necessary logs to be able to determine how the hack occurred because of poor log retention at Dreamhost.

To determine if the website was hacked due to a vulnerability in web software, we would have reviewed the log of HTTP activity. That show all the requests made to the website, so it would show any exploit of software running on the website. Unfortunately, those are only stored for a very short time at Dreamhost. If Dreamhost actually wanted to determine that websites were in fact being hacked by vulnerabilities in web software they would also need to look at that as well. Except for cases where the hack is found immediately, how could they make a strong determination that the hack was due to a vulnerability in web software? The answer seems to be they don’t even try, as will get to in a little bit.

The other thing we wanted to look at was the log of FTP activity. If this is available for the time period of the hack it is easy to see whether the hack came through FTP by reviewing the log. Our client was told that the logs are only stored for two weeks and because the hack occurred before then the log wasn’t helpful. The short retention of the FTP log also means that Dreamhost couldn’t check if the hack came through FTP if it occurred more than two weeks before, which would limit their ability to know that hacks are not coming through FTP as they seem to think is the case. This also makes another claim in the post seem questionable. In regards to a recent hack of Dreamhost they stated that “An extensive investigation has revealed that no customer FTP or SSH user accounts have been maliciously accessed due to this password breach.” If you don’t keep logs longer than two weeks how could you claim to do an extensive investigation, as you can’t look at data older than two weeks once you discovered the issue?

While we couldn’t make a strong determination on the source of the hack, Dreamhost didn’t have a problem doing that. They said:

Also, “hacks” usually occur due to outdated web software, and since you’re running joomla, it’s safe to say, the hack occurred there.

There hasn’t been the kind of vulnerability in Joomla that has led to the software being hacked in a long time. That isn’t to say that something couldn’t be found, but if somebody is telling you that a hack likely occurred from that or that Joomla is not secure then they don’t know what they are talking about. From the hacked websites that we see that are running Joomla, the issues that lead to hacks are vulnerable extensions, compromised FTP credentials, a hack of the hosting provider, and recently we have had a number of clients hit by a dictionary attack campaign that is targeting Joomla websites. It clearly isn’t safe to say the hack occurred because they were running Joomla, as Dreamhost, unless they have done a thorough investigation and found evidence to back that up.

Let’s say you did believe that the cause of a hack was due to a Joomla vulnerability, there would be more work to do after that. You would need to determine if the vulnerability has been fixed in Joomla before you tell somebody “simply reinstall your software, and make sure you’re always keeping it up to date”, as Dreamhost did, because doing that would have no impact if the vulnerability hasn’t been fixed in Joomla.

If the hack is on a website was running the latest version of Joomla it would mean that the vulnerability hasn’t been fixed or more likely that it wasn’t a vulnerability in Joomla.

For a website running an older version you would need to compare what information you had about the hack to the vulnerabilities that have been fixed in subsequent versions to see if the probable vulnerability has been fixed. For example, there was recently fixed vulnerability that could make it easier for some to guess a user’s password after causing it to be reset. To check for that you would ask the client if the password stopped working recently, which would have happened if it was reset. If the logs were available, you could also check those for additional information. If you don’t find a corresponding vulnerability it would mean that the vulnerability hasn’t been fixed or more likely that it wasn’t a vulnerability in Joomla that caused the hack.

Dreamhost didn’t inquire into what version of Joomla was running when the website was hacked or doing anything else, that is another good indication that don’t know what they are doing or they just don’t care. If this customer interaction is any indication of how they handle this type of situation for other customers, it doesn’t appear they do any checking at all and instead just assume a hack must have been caused by software on the website.

Failure to Warn About Vulnerable Software

When it comes to dealing with outdated web software that is actually vulnerable to being exploited, and has been exploited, Dreamhost fails their customers. Dreamhost has tool that scans a customer’s files to find the web software in the account and warn if it is out of date. As of a couple days ago it was listing OpenX 2.8.7 as being “Up-to-date.”, despite the fact that OpenX 2.8.8 was released back in November. OpenX 2.8.7 isn’t just out of date, it contains a serious security vulnerability that was being exploited even before version 2.8.8 was released. You have wonder how they wouldn’t be aware of this if they were actually determining how websites were being hacked or just keeping track of security issues in web software.

Dreamhost Runs Outdated Web Software

In another post by the same author they recommended “upgrading website software as soon as there is an update available”, which is actually good advice. It just isn’t advice that Dreamhost follows.


Dreamhost Blog WordPress Version

If you visit the Dreamhost blog with our Meta Generator Version Check extension (available for Firefox and Chrome) you would get an alert the blog is running an outdated version of WordPress, 3.2.1. That version is not that far out of date, it’s only one major version out of date and two months out of date. (Also, contrary to recent claims by Websense there are not known vulnerabilities in 3.2.1.) But considering that Dreamhost is telling people how important it is to keep software up to date and they don’t do it themselves then it seems like Dreamhost either does not care about the security of their website or they don’t believe what they are saying.

What make this more ridiculous is that the person that wrote both of those posts, who is described as Dreamhost’s Web Application and Security administrator, will be giving a talk  titled “Security 101” at a WordPress event.  Should you be telling other people about WordPress security if you don’t even bother to take the basic step of keeping WordPress up to date?

Dreamhost Wiki MediaWiki VersionIt’s not just that blog that is running outdated software, the Dreamhost Wiki is running an outdated version of MediaWiki, 1.16.5. Support ended for that major release, 1.16.x, two and half months ago. Like the WordPress installation that is not far out of date, but for whatever reason Dreamhost doesn’t feel it is important to keep the software running on their website up to date.

Did Websense Want to Leave Websites Vulnerable to Being Exploited?

Since we discussed that Websense’s claims of a vulnerability in WordPress 3.2.1 were baseless they have updated to their post to basically admit that the infections had nothing to do with WordPress. What is more curious about the update is that they now claim that they actually originally suspected a vulnerability in WordPress plugins not WordPress 3.2.1 was leading to the infections. This doesn’t match up with the previous claims or the press coverage the claims received. The new claim does seem to be a poor attempt to explain away their previous false claims, but if they actually believed that the infections were due to plugin vulnerabilities then the advice they gave webmaster on how protect themselves would have continued to leave the vulnerable websites open to being exploited.

Let’s take a look at some key sections of the post’s update.

Not All Running WordPress 3.2.1

After obtaining access to logs and PHP files from compromised Web servers, further analysis indicates that most of the compromised Web sites were running older versions of WordPress, but they were not all running 3.2.1.

This doesn’t make much sense. The logs and PHP files related to the infection would give only basic information, if any, on what software the website is running. If you wanted to know what they were running you could get more information by looking at the websites.

As we mentioned in the previously there appears to be a serious problem with how Websense identified infected websites that lead them to only find websites running WordPress 3.2.1 to be infected. This would also bias the selection of logs and PHP files they reviewed and would have led them to continue have an over-weighted sample of websites running WordPress and WordPress 3.2.1 in their sample.

Source of the Infections

The attackers’ exact point of entry is uncertain.

Now that we have access to data from several compromised Web servers, the logs show us that, in some cases, the point of entry was compromised FTP credentials. In several instances, once attackers had access, they scanned WordPress directories and injected specific files (e.g., index.php and wp-blog-header.php) with malicious PHP code.

Somehow the point entry is unknown, but they also our say that they found the entry point in at least some cases was through compromised FTP credentials. They don’t make many mention of finding any other entry point. If the websites were infected through compromised FTP credentials, which seems fairly likely possibility, that means the infection has nothing to do with WordPress. In what seems to be an attempt to try give the impression of a link between the infection and WordPress they mentioned the attackers scanned WordPress directories and infected a WordPress specific file. We know that the infection has infected websites not running WordPress so they attacker would have scanned the directories whether they were related to WordPress or not. index.php is a generic file that exists in any websites programmed in PHP, so that specific file would be a target no matter what PHP based software is running. Why bring up WordPress other than to try to make it still seem like this is something strongly related to WordPress when it pretty clearly is general issue that would have impacted websites running any software?

Suspected Plugin Vulnerabilities?

 At first, we suspected vulnerable WordPress plugins, because a subset of analyzed sites were running vulnerable versions of the same WordPress plugins.

This claim that they originally believed that websites were exploited due to vulnerable WordPress plugins just doesn’t line up with the originally posting at all. The most relevant portion of the original post:

Based on my analysis, the site was compromised because it was running an old version of WordPress (3.2.1) that is vulnerable to publicly available exploits [1] [2].

You don’t get much clearer than that. If they believed that this was a plugin issue then it wouldn’t have mattered what version of WordPress was running as the vulnerability would exploitable as long as you are running plugin. You could be running WordPress 3.3.1, WordPress 3.2.1, or any other version and it would make any difference only if you had the plugin installed. Yet the original post repeatedly mentioned WordPress 3.2.1 and never mentioned plugins at all. The title of the post is “3-2-1 WordPress vulnerability leads to possible new exploit kit “, the list of traits for the infection mentions WordPress 3.2.1, there is a section of the post titled “What To Do If You Are Running WordPress 3.2.1”.

SC Magazine reported that a WordPress 3.2.1 vulnerability was to blame:

The attacks are taking advantage of website owners who are hosting an older — and vulnerable — version of WordPress, 3.2.1, which was updated in December but is still widely in use.

The reason for bringing up plugins now, seems to be because we pointed out the claimed vulnerabilities they cited as WordPress 3.2.1 were in fact vulnerabilities in plugins. We previously mentioned how it looks like those plugins vulnerabilities got confused to be WordPress 3.2.1 vulnerabilities (try this search).

If they believed that it was plugins then those plugin exploits they cited don’t make sense. When we looked some of the websites they listed in their sample of infected websites, none of them were running those plugins. The type of vulnerability, a blind SQL injection, which was claimed in those plugins, isn’t something that would be likely to be used in this type of hack.

Websense’s Advice

For the sake argument let’s say that they actually believed the vulnerability was due to WordPress plugins. The way to protect websites from this would be for the vulnerability in the plugins to be fixed or for the plugins to be removed. Which did Websense suggest be done? Neither, remember they never even mentioned plugins in the post. Instead their advice was:

If you have been infected, be sure to upgrade WordPress while simultaneously removing the injected code so that your Web pages aren’t simply being reinfected after being cleaned.

Upgrading WordPress would have no impact on the vulnerability in plugins, they would have been just as vulnerable as before. Even if you assume they meant that the plugins should be upgraded along with WordPress, it wouldn’t have done anything as plugins they were citing did not have updates available to fix the vulnerabilities. (We will be discussing this in more detail soon, but it is worth mentioning here that both of these plugins have been removed from the WordPress.org Plugin Directory, one of the reasons plugins get removed is for security issues that have not been fixed. You can use our new WordPress plugin, No Longer in Directory, which checks if installed plugins have been removed from the WordPress.org Plugin Directory to warn about such situations.) So why would Websense have wanted the websites to remain vulnerable? We certainly can’t think of a reason why they would want to do that. The reasonable explanation for this is that the advice was based on their belief that there was a vulnerability WordPress 3.2.1, not in plugins as they came up with later to try cover for their false claims, and upgrading WordPress would have fixed the vulnerability.

Websense’s Claim of Vulnerability in WordPress 3.2.1 Completely Baseless

Last week Websense published a post 3-2-1 WordPress vulnerability leads to possible new exploit kit written by Stephan Chenette their Principal Security Researcher, which made a the claim that there were publicly available vulnerabilities for WordPress 3.2.1 and that a malware infection was hitting only WordPress 3.2.1 based websites. Their post lead to articles in SC Magazine and PC World repeating the claims.

Previously we discussed the fact that the claimed “publicly available exploits” did not actually exist. It’s worth taking a more in depth look at that claim because if there were standards for security researchers than Websense’s Principal Security Researcher would likely have been grossly negligent in this situation.

After our previous post we were still curious with the possibility that there might be a vulnerability in WordPress 3.2.1 based on their claim that all the websites with this particular infection were running WordPress 3.2.1. We looked into this and have found that this is also false. Our finding on that also raises serious questions as to how their researcher could have come to have that conclusion.

What Stephan Chenette and Websense have done by making these baseless claims is highly inappropriate. It is a serious charge to make that software, even if it is an out of date version, has such a serious security vulnerability. To do that based on such blatantly incorrect information and with a complete lack of due diligence is inexcusable and should lead to repercussions for everyone involved in creating and spreading these falsehoods.

Publicly Available Exploits for WordPress 3.2.1?

In the post Chenette said that “Based on my analysis, the site was compromised because it was running an old version of WordPress (3.2.1) that is vulnerable to publicly available exploits [1] [2].”.

The exploits Chenette is citing come from Exploit Database. That website and other similar websites that list claimed vulnerabilities in software include submissions that have not been verified to be actually exploits before being published. Exploit Database includes vulnerability reports that are completely false, like this submission that claimed that the WordPress plugin WP Touch contained a vulnerability in a file despite the fact that the file that doesn’t even exist in the plugin. Anyone serious about security research would verify the vulnerability found on one these websites actually exists before citing it. Those reports also shouldn’t be cited by news organizations unless this has been done.

We know that Chenette didn’t verify these vulnerabilities because if he had attempted to verify the vulnerabilities he would have seen that vulnerabilities mentioned were not in WordPress 3.2.1 or WordPress at all. Instead these vulnerabilities where for WordPress plugins WP-SpamFree and UPM-POLLS. He would have also known if he had just looked at the titles of the exploit reports, which are “WP-SpamFree WordPress Spam Plugin SQL Injection Vulnerability” and “Wordpress UPM-POLLS Plugin 1.0.4 Blind SQL Injection”. What it looks he did was to do a search for WordPress 3.2.1 for that website and just grabbed those results without bothering to look at what they were. If this is level of research Websense’s Principal Security Researcher does, we would hate to see what the subordinates are doing.

There is no evidence that those plugins were exploited on the infected websites and when we checked a few of the websites that we were listed in the Websense post the plugins didn’t even appear to on the websites. In any case because the vulnerabilities are in plugins, if you had plugin installed that was exploitable you would be just a vulnerable whether you were running WordPress 3.2.1, WordPress 3.3.1, or another version.

All The Websites Running WordPress 3.2.1?

The post claimed that all the websites with this infection were running WordPress 3.2.1. If this were true it could be an indication that the infection of the websites was due to an exploit of something specific to WordPress 3.2.1, even though the claimed publicly available exploits did not exist.

As we mentioned in the previous post there are a number of possible explanations why all the infected websites would be running WordPress 3.2.1 without there being an exploit of something in WordPress 3.2.1.

The other possibility is that Websense post was incorrect in claiming all the website were running WordPress 3.2.1. In our previous post we found that for the other claim of a WordPress 3.2.1 vulnerability, by M86 Security Labs, the sample list of infected websites did not contain only websites recently running WordPress 3.2.1 as claimed. In Websense’s case we found that the sample list of 11 websites, included with their post, to have all been running WordPress 3.2.1 recently.

Because the possibility of a remotely exploitable vulnerability in WordPress 3.2.1 is such a serious issue we wanted to look further into the possibility that this infection was in fact due to an exploit of WordPress 3.2.1 as implicated by Websense’s post. The best way to check for this would be to review the HTTP log of an infected website. If an exploit of WordPress 3.2.1 had been the cause that should be discernible in the log. So far we have not cleaned up any websites with this infection, so we haven’t had a chance to do that.

If we could find websites running an earlier version of WordPress (if a website is running a new version it could have been upgraded after the exploit) or not running WordPress at all then that would mean the infection is not limited to WordPress 3.2.1 and likely point to the infection having nothing to do with an exploit of WordPress 3.2.1.

To find websites that contained this infection we looked at Google’s Safe Browsing Diagnostic data. This data includes a few example of website they have found to contain a certain infection. We started with the data for the domain listed in Websense’s post . Two of those domains were now running WordPress 3.3.1. The third doesn’t appear to run WordPress, but it was down so we couldn’t confirm if it contained the malware. We then look at the reports for three more domains involved.

Of the 11 websites that we checked, that appear to had been infected with the malware based on Google’s data, we found 2 were currently infected with the malware and were not running WordPress 3.2.1 or above. We found http://www.weightloss-blogs.org/ which was running WordPress 3.1.3 and contained the malware at the time we checked. This shows that the exploit could not be something specific to WordPress 3.2.1. We also found http://victoryaog.org/joomla/, which as you might guess from the URL is running Joomla instead of WordPress and contained the malware at the time we checked. This shows that it is not something specific to WordPress. We found more that were not running WordPress at all, but were not currently infected so we can’t be totally sure they were infected. Our results show that Websense’s claim that this is infection is specific to websites running WordPress 3.2.1 be baseless.

That finding also raises questions about their data. How could they have found 100+ websites, including the sample list of 11 websites, all running WordPress 3.2.1 but with a much sample set of only 11 we found 2 confirmed instances where this wasn’t the case? They are a few possibilities we could think of.

One possibility is that their data is not very representative of the Internet as a whole, which would make the data unreliable for research purposes and make any reports based on it unreliable as well.

Another possibility is that their system for reviewing their data is flawed or the person doing the analysis screwed up. Maybe they only looked for websites running WordPress 3.2.1, in which case obviously all the infected websites they looked at would be running WordPress 3.2.1. WordPress powers many websites, so finding 100+ that were infected wouldn’t be surprising.

The most troubling possibility, and we need to emphasize that we don’t have any evidence of this, is that their data didn’t actually show that all the website were running WordPress 3.2.1 and they intentionally created a sample list of websites that only contained websites running WordPress 3.2.1.

Claims of Vulnerability in WordPress 3.2.1 Supported by False Information

In the past few days they have been two reports of a vulnerability in WordPress 3.2.1. In looking into these we found that the claims to be supported by false information and no evidence of a vulnerability in WordPress 3.2.1 was presented. A number of news organization including SC Magazine, The Register, and PC World repeated the claims without bothering to check the information for themselves. We still would recommend that anyone running an outdated version of WordPress upgrade to version 3.3.1 (which one of the security companies making the claims hasn’t done itself) and to make sure you keep all the software on your websites up to date at all times.


The first claim is by the Principal Security Researcher at Websense. In their post it is stated that “Based on my analysis, the site was compromised because it was running an old version of WordPress (3.2.1) that is vulnerable to publicly available exploits [1] [2]”. Looking at the exploit reports they are for two WordPress plugins not for WordPress itself, so the analysis is fundamentally wrong.

If those exploits actually work (we didn’t test them) then it wouldn’t matter what version of WordPress you were running only if you were running the vulnerable version of the plugins. We checked a few of the sample websites they listed and it did not appear that they were running either of those plugins.

Their other evidence that the exploit was something related to WordPress 3.2.1 is the claim that all of the website were running that version of WordPress. If this is true (the sample list of 17 websites they provided were all recently running WordPress 3.2.1) it doesn’t necessarily mean that the hack was related to something in WordPress 3.2.1 or WordPress at all.

One likely possibility is that they are all running some other outdated software, maybe a WordPress plugin, that contain a vulnerability. Websites running an outdated version of WordPress are more likely to have not kept other software on the websites up to date as well.

WordPress and WordPress 3.2.1 run on many websites so it also could be that some other type of exploit, like compromised FTP credentials or hacked hosting providers, could be the source. We know that sometimes hackers specifically target WordPress installations when using non-WordPress related exploits, something that Websense has in the past falsely claimed as being an exploit of WordPress itself.

M86 Security Labs

The second claim comes from M86 Security Labs. In their post they state that “A few days ago, hundreds of websites, based on WordPress 3.2.1, were compromised. ” We checked what version of WordPress the sample list of 33 website they included in their post were running. We found that many of them were not running WordPress 3.2.1. Here is the breakdown:
3.3.1: 7
3.3: 2
3.2.1: 8
3.2: 1
3.1.1: 1
3.0.3: 2
3.0.1: 1

They were also 11 websites that were inaccessible at the time we checked. Of 22 we could check, less than half were running 3.2.1. Nine were running newer versions and five were running older versions. It possible that the ones running newer versions were upgraded after they were exploited, but it makes no sense that websites running older versions were running 3.2.1 at the time of the exploit. This clearly indicates that the websites were not all running WordPress 3.2.1 as claimed and that exploit is not something specific to 3.2.1. It is possible that it could be related to something that existed in 3.2.1 and below, but there is no evidence provided by them to support that.

If you visit the M86 Security Labs Blog with our Meta Generator Version Check extension (available for Firefox and Chrome) you would get an alert the blog is running an outdated version of WordPress, 3.1.3.

M86 Security Labs Blog WordPress VersionIf they really believe there is a vulnerability in outdated versions of WordPress why are they running an outdated version on their own blog?

DreamHost Does Store Non-Hashed Passwords

On Friday DreamHost reset all of their customers “FTP/shell access passwords” after they had unauthorized activity within one of their databases, the situation is discussed in blog posts on DreamHost Status blog and the The Official DreamHost Blog!. Since then there have been questions and confusion as to whether DreamHost only stores passwords in their hashed form. While we have no way of knowing if the database they detected unauthorized activity stored non-hashed password there is no question that they store non-hashed passwords in their systems. It’s fairly easy to see that DreamHost is doing this and we will show you how you can check this for yourselves at the end of the post.

The fact that they stored passwords in a non-hashed form has been discussed for many years and DreamHost has so far has decided that insuring that they were follow proper security practices by only storing password hashes wasn’t necessary for whatever reason. It’s then not all to surprising that they had this most recent security incident and the other apparent security incidents they have had over the years. For some time we have listed DreamHost in our list of web hosting providers with security issues due to them storing non-hashed passwords.

One possible reason for some of the confusion from DreamHost is that they don’t understand the difference between encryption and hashing, in which case it they probably shouldn’t be handling the security of a website, much less that of a major web host.

While discussing DreamHost’s security it is also worth bringing up the fact that both of those blogs are running an outdated version of WordPress, 3.2.1. They are also are running an outdated and now unsupported release of MediaWiki, 1.16.5, on a portion of their website (so are a number of the websites of web software). In a message that was forwarded to us while we were cleaning up a hacked website for client recently, DreamHost had told them that they should make sure to keep web software running on their website up to date. Obviously DreamHost don’t feel it is important to follow the advice they give to their customers. If you want to see when websites are running out of date version versions of WordPress, MediaWiki, and other software check out our Meta Generator Version Check web browser extension for Firefox and Chrome.

Considering DreamHost’s questionable security practices we would recommend that people avoid using their services until they have fixed these lapses in their security. We also don’t think that WordPress should be recommending them or describing them to be one of the hosts that “represent some of the best and brightest of the hosting world”.

What is Hashing?

You can think of hashing as one way encryption.  To produce a hash you run a hash function on a specified value, in this situation it would be the value being set as a password. For example, using the MD5 hashing function the hash for the password value “password” would be “5f4dcc3b5aa765d61d8327deb882cf99”.

Unlike encryption, hashes are not meant to be decryptable and ideally you wouldn’t be able to determine what the password value was if you gained access to its hash. This is why it is important to store passwords as hashes. If they are stored them in a non-hashed way someone that gains access to them could easily use the passwords to log into your account, which has happened previously after web host’s were exploited, or if you use the same password on different systems they could potentially gain access to those as well. There are a number of ways to determine the underlying value of passwords hashes, so systems using hashing for passwords need to insure they follow best practices including making sure they use salts.

So how does a system know that the correct password was entered during a login attempt if they only have the hash? The answer is that when the login attempt is made the password you enter is run through the same hash function and then compared with the stored hash of the password. If the two are they same the login attempt will succeed. If you entered the wrong password the hashes would be different and it would fail.

If passwords are only stored in hashed form there will be no way for a provider to retrieve the password from storage that for you. The only instances where they could show you the password would be when they are generating a new password for you or if they show you the password in response to you entering it.

The most common place to see that passwords are being stored in non-hashed form is on pages for handling a situation where you forgot your password. If they can show or send you the password it means the password in being stored non-hashed in their systems. With web hosts we also sometimes see that passwords are visible somewhere in the control panel for the websites.

Spotting Non-Hashed Password Storage at DreamHost

From the DreamHost’s homepage click the Panel link at the top and then click the Forgot password link. That page currently looks as follows:

DreamHost Forgot Password Page

If the password were only stored in the hashed form they wouldn’t be able to email you your password because they wouldn’t know what it was.

Does Security Really Matter to OpenX?

On December 1st OpenX finally made a public announcement on their blog about OpenX 2.8.8, which fixed a vulnerability that had already been exploited for some time before OpenX 2.8.8 was released. There post claims “If ever we find an issue, we address it quickly and communicate any updates as soon as possible.” Would anyone think a month is “as soon as possible”. What makes the length of time for the announcement even more troubling is that back on November 8 when we posted about the lack of a public announcement, and other issues, we had many visitors from OpenX visiting the blog so if they hadn’t yet thought it was important to make announcement before that they should by then.

Their post begins with the claim that “OpenX takes security seriously.” It hard to take that seriously considering that that this is third post on their blog titled Security Matters (1, 2) making the same claim and yet they have had to continually released fixes to vulnerabilities after those are already being exploited. It is understandable that software can have vulnerabilities, but when hackers are finding and exploiting them first instead of the developers finding and fixing them first it is an indication that their process for insuring the security of their code is lacking.

While there has been a fair amount of time between new vulnerabilities being exploited, and then fixed by OpenX, it is reasonable to consider that it might not be due a limited number of vulnerabilities but a lack of need to exploit more vulnerabilities. From what we have seen there seems to plenty of ad server running outdated versions of OpenX that hackers have been able to exploit well after new versions are released, so it doesn’t seem unreasonable to think that hackers might know of or could easily find more vulnerabilities in OpenX but as long there are enough ad servers running on outdated versions of OpenX to exploit there would be no need to make OpenX aware of a new vulnerability so that it can eventually be used when they run low on outdated ad servers to exploit.

It also is hard to take them seriously when there is such a public example of them not following their own advice. As part of their post they say “It’s critical to the safe maintenance and operation of any software that you not only maintain a current version of the software, but also take steps to regularly audit accounts that have access to your system.” They correctly state that it is critical to keep software up to date, but you don’t have look far to see that they don’t follow their own advice. The blog that they posted to is running WordPress 2.6.2 (if you want to see when websites are running out of date version versions of WordPress and other software check out our web browser extension for Firefox and Chrome). That version is now over three years out of date. They have failed to apply the last 16 releases that included security updates and 27 overall.

The CHANGELOG.txt file for www.openx.com indicates that it is running Drupal 6.19, which, if accurate, means the Drupal install is a year out of date and they missed a security update for that as well.

OpenX Continues Questionable Security Posture

Last Thursday OpenX released version 2.8.8 of their software. They have yet to make any sort of public announcement of the update on their blog or anywhere else that we could find. The only information given, found on the Product Updates page in the OpenX admin interface, says that:

It is highly recommended to install this update as soon as possible, because it contains a number of security fixes. The version of OpenX which you are currently using might be vulnerable to certain attacks and is probably not secure.

With a release that includes important security fixes, as this seems to be, you would expect that they would want to make sure people that use their software would be well aware of the update.

There was no information was given as to what the vulnerabilities were or what other changes were made in the new version. This is a continuing practice from OpenX as we have written about before. While it is understandable that developers would want to limit the amount of information to make it harder to for people to be able to exploit the vulnerabilities, hackers have shown that they are able to hack OpenX without this information and the information would be useful for people not looking to hack OpenX.  To repeat what we said after the last OpenX release, “[w]ithout knowing what the issue or issues that were fixed makes it hard to determine the source of a hacking, potentially leading to new vulnerabilities that are exploited in OpenX going undiagnosed in the future if the OpenX installation hacked was running an out of date version.” It also makes it hard for anyone to independently verify the vulnerabilities were fully and properly fixed in the newer version.

The larger concern we have now is that OpenX seems to continue to be releasing security fixes in response to vulnerabilities being actively exploited, commonly referred to as zero-day exploits, instead them being found beforehand during development or during subsequent security reviews. We know that with past vulnerabilities they were being exploited before updates were released. We have seen some reporting that vulnerabilities in the last version were being exploited (with the most specific report we were not able to replicate the vulnerability, but that could be because of using a different server configuration) before this version was released. This at least means that users keeping the software up to date are not safe from being hacked, which they generally are with most web software that have a good track record of finding and fixing vulnerabilities in their software before they can be exploited. It also could be an indication that OpenX is not as concerned about the security of the software as they need to be for something that is so widely deployed.

What makes there apparent lack of concern towards the security of their software more troubling is the way they used the update message for 2.8.8 as a chance to promote their hosted solutions. This is the message that followed the warning about the need to update:

OpenX also provides both free and Enterprise hosted versions of the ad server, offering significant improvements in both infrastructure and functionality. Both of these products are managed and operated by the OpenX team, including upgrades, maintenance, and security scans, freeing you and your team from handling such issues. If ad serving is mission-critical to your business, we suggest contacting our team to learn more about OpenX Enterprise. As always, please let us know of any potential security problems by emailing security@openx.org.

All the hacks of OpenX we have dealt with so far have been due to security vulnerabilities in the OpenX software and not due directly to something related to self-hosting. In many of those cases OpenX had released a update before they were hacked, so automatic upgrades provided by their hosted solutions would have helped. But unless OpenX is providing their hosted customers with a more secure version of OpenX, then the hosted customers remain as vulnerable before the fixes for the security vulnerabilities are released. The quality of their security scans should be in question as well, if vulnerabilities keep getting found and exploited before they are fixed by OpenX.

Update (November 14, 2011):

Another thing that should be noted when considering how OpenX views the importance of security is the fact that their blog is still running WordPress 2.6.2. One of the most basic and important security measure anyone running a website should be doing is making sure they keep any software running on the website up to date. The version they are currently running is now over three years out of date. Since version 2.6.2 there have been 16 releases that include security fixes that they have missed (and 26 overall releases).

VeriSign’s Bad Advice on Protecting Websites from Malware

If you do a Google search related to website malware right now you might right run across the following ad from VeriSign:

VeriSign Malware Scan
What you need to know about malware & how to protect your site

Someone interested in how to protect their website from malware might click on the ad hoping to learn about doing that. From the page the ad takes you to you could visit a page titled FAQ: Web Site Malware Scanning. One of the questions in the FAQ is “How can I protect my site from malware?”. This looks like the information their advertising was promoting.  Here is what they say:

Like most thieves, malware hackers look for easy targets—such as a Web site where malware will go undetected for as long as possible. Posting the VeriSign Trust Seal on your Web site is like posting an alarm security sign in your front window. It shows hackers that your site is scanned daily to detect malware.

There are probably many variations on what would be a good answer to this question. Verisigns answer is certainly not one of them. Not only have they given really bad advice for protecting websites, but the answer suggests a scenario that is almost never going to happen.

The scenario in the answer suggests that hackers are going to view the website before they attempt to hack it. In almost all instances that is not the case. Not only is someone not likely to view the website before attempting to hack it, but there probably will not be a person directly controlling the attempted hack. Instead, the hacking attempt is likely to be automated.

For example, someone might setup a program to go through every domain name attempting to exploit a vulnerability in an outdated version of WordPress. Because no one is viewing the website before attempting to hack it the VeriSign Trust Seal will have no impact on whether the website is hacked or not. The best that malware scanning could do in this case would be to quickly warn that the website is infected. The worst case would be the scanner not detecting the infection until it has potentially infected many visitors. What is hopefully obvious is that if you are not running an outdated version of WordPress you would not get infected in the first place.

The right way to protect your website against these types of hacks, which are done in this automated fashion, is by taking the measures we have written about here. If your website is properly secured you are very unlikely to get infected so malware scanning is of little use. If you wanted make sure that you are warned quickly if your website is ever infected you set it up so that Google will send email to an address of your choice if they ever detect malware on your website.

So would the seal have any deterring effect on someone who has decided to target your website? It is hard to say for sure, but it seems unlikely it would have any effect. If someone were looking for easy targets they wouldn’t be trying target specific websites at all. It is much more efficient for them to use untargeted methods to find easy targets. What would be more likely to happen if they were targeting you is that they would test their malware to make sure it is not detected by the scanning done by Verisign before infecting your website. In that situation letting them know it was going on would not be helpful.

Verisign is owned by a major security company, Symantec, so they should be aware of all of this, especially since they decided to run advertising promoting that they would tell “What you need to know about malware & how to protect your site”. Either they don’t know about website malware, but are offering the service any way, or they know about it and they appear to be intentionally misleading potential customers.

Increased PHP Requirement for WordPress 3.2 Not a Major Issue

With the release of WordPress 3.2 coming up shortly (we are running the release client of it on our other blog without issue) the issue of its higher version requirements for PHP and MySQL have been coming up as a possible issue. One comment that we noticed was from a self-proclaimed security researcher was making the point this would lead to more outdated WordPress installations because servers are still running versions PHP below 5.2.4, which is the new required version, will not be able to be upgraded. On that point we have actual data on what version of PHP is running on servers and, more importantly, information on why an actual security researcher would see a much bigger issue with people still running a version of PHP below 5.2.4 than not being able to upgrade WordPress.

For every client that we need access to their website’s filesystem during our work we check the PHP version as well as other software running on the server (you can check your host using a tool we have created). For hosts having particularly outdated versions of software we alert the client to the issue and we also document some cases on our page detailing hosts with security issues. Our clients host websites around the world and with host provider of all sizes so the data should be a good representation of what is exists overall. We reviewed our data for this year and we found that none of our clients had been running a version of PHP 5 below 5.2.4, the lowest we found was 5.2.6. We did have some clients that were still running PHP 4, in all those cases we were able to switch them to PHP 5, above version 5.2.4, through the host’s control panel without issue. If you are still running PHP 4 you should make the switch as soon as possible as support for PHP 4 ended on December 31, 2007 and updates for critical security issues ended on August 8, 2008.

If there are still people on hosts that are only running a version of PHP less than 5.2.4 they probably have much bigger security issue than not being able to upgrade WordPress. PHP 5.2.4 was released on August 31, 2007 and last version PHP 4 was released on August 7, 2008. So that means their host has not bothered to upgrade one of the core pieces of software on their servers for nearly three or four years. While PHP itself is not a common target of hackers other server software is. Keep software running on the servers is the most basic security measures a host should be taking, if the host is not doing that then there is good chance that are not taking care of other security measures. There are many hosts that do take the basic security measures of keeping the server software up to date, so no one should be using a host that isn’t.

We would expect that a security researcher would know that you need to keep server software up to date and that PHP 5.2.4 itself is very outdated before making the statement they did. The fact that somebody claiming to be a security researcher doesn’t know this is a great example of why website security is in such a bad place. There are many people that are involved in website security that don’t’ know even the basics, but that doesn’t seem to stop them from telling others what they should be doing. If an actual security researcher were to complain about this, you would expect them to be suggesting that WordPress and other web software raise the required PHP version even higher. There have been numerous security fixes included in versions of PHP since version 5.2.4 was released and support for PHP 5.2 ended in December of last year. PHP 5.3 includes major changes that can cause software to break so many host are holding back switching to until more software is available with a version that supports PHP 5.3, but there is no reason they could not be running the last version of 5.2, 5.2.17. 5.2.17 was released over six months ago.

WordPress 3.2 also requires at least MySQL 5. None of our clients were running something below that this year. Support for the version below that, 4.1, ended on December 31, 2009.