The Apache foundation experienced some downtime on August 28 when unauthorized access to their servers was detected. A few days ago, the Apache infrastructure team posted a very well-written post-incident report in which more details with respect to the attack are published, and an overview of the lessons that were learned from it are shared.

The report is very well written and worth reading. Some key findings:

  • "The
    use of SSH keys facilitated this attack.    " Yes, SSH is more secure than telnet (or rlogin), but it must still be hardened.

  • "The ability to run CGI scripts in any virtual host, when most of our
    websites do not need this functionality, made us unnecessarily
    vulnerable to an attack of this nature.    " Very few people are not also guilty on this one. Trim down a system's configuration to only provide the minimal amount of functionality it needs to do the job.

  • "We will re-implement measures such as IP banning after several failed logins, on all machines." Brute force attacks are still one of the most successful attack vectors. Automatic account lockout and restricting the network space from which incoming connections are allowed in the first place seriously reduce the attack surface.

  • "Because they obtained root on the CentOS machine, we are not entirely
    sure, almost all logs on the machine were destroyed. The machine ran
    many stock web applications and may of had less than secure password
    practices -- but once they got root whatever evidence of the initial
    hack was destroyed.    " Keep critical logs on a dedicated, hardened server to facilitate post-incident analysis.