Hardening SSH on your Ubuntu Server

Hardening SSH on your Ubuntu Server

Secure Shell (SSH) is an amazing cryptographic network protocol, and it undoubtedly helps secure a huge slice of today’s Internet – giving sysadmins robust remote access to their servers, but also so much much more.

I’ve been using SSH on Linux Servers for longer than I can remember. Well, not quite. I do remember those “telnet” days when nobody had a care in the world, and Edward Snowden was a kid playing video games. Or something.

I’ve learned a thing or two about hardening SSH and locking it down, including generating certificates, and using it as a remote proxy in all kinds of ways. Here is a brief brain dump that I hope you enjoy.

General SSH Configuration Tips

Configuration of SSH starts with the sshd_config file.

sudo nano /etc/ssh/sshd_config

Change the Default Port

The first is the “Port 22” default that SSH listens on; I will usually change this to a different number other than the default, or run it on multiple ports depending.

# What ports, IPs and protocols we listen for
#Port 22
Port 2222

If you don’t change the port from 22 you’ll just be subjecting yourself to endless probes and brute-force logins from everywhere. While changing to something else is not guaranteed to stop any of that, in my experience it will reduce it. A lot.

Ensure you’re using Protocol 2 (SSH2)

The second thing to check, particularly if you have an older Linux server is the version of the SSH Protocol – the line “Protocol 2” does that. Ensures you’re using SSH2 which is far superior that earlier versions of the protocol and shouldn’t be an issue today.

Protocol 2

Don’t Permit Root Login

I’m a strong advocate of always connecting to a server using my own account, and if I need superuser access I’ll “sudo” to get it. This means there’s never a valid reason ever to login as the root user.

Unfortunately, some of today’s cloud VPS hosting providers will provide you with a root login straight away. Um, no thanks.

PermitRootLogin no

Turn Off Password Authentication (if using Certificates)

If you’re using certificate based authentication (which I highly recommend – here’s some useful information on that) you’ll want to turn off any kind of password authentication, by locating and modifying the lines below to “no”.

ChallengeResponseAuthentication no
PasswordAuthentication no
UsePAM no

Restrict to Known Users

Another thing you can do, which I think a lot of people never bother doing, but might be useful, is to restrict SSH access to certain known users on the system. Even with the right password or certificate a user not in this list (if the setting is present) will not be able to login.

AllowUsers john bob alice

Using the “AllowUsers” setting could be vital if your server is hacked and somebody tries to create a backdoor user – at least this will prevent them getting any remote access, and the failed attempts may show up in your log files.

Disabling Potentially Compromised Modulus

There is some conjecture that the NSA (and others?) could intercept TLS and SSH communications by obtaining the shared private key during the “Diffie Helman” key exchange process.

The speculation about this weakness with DHE is potentially feasible. The concern is that there exists the ways and means through supercomputing power to factor the large primes commonly used for DHE.

Here’s one example news story suggesting it’s possible, which led me to researching and discovering some practical hardening tips here that I’ve refined a little further.

You’ll have to edit the moduli file:

nano /etc/ssh/moduli

Comment any lines that have a value less than 2047 shown in the 5th field on each line – and then don’t forget to restart the SSH service.

I’ve also made this SED one-liner below that you can run as root which will comment out the lines mentioned in one hit as well.

sudo cp /etc/ssh/moduli /etc/ssh/moduli.bak
sudo sed -r -e '/[[:space:]](1023|1535)[[:space:]]/ s/^/#/' -i /etc/ssh/moduli

Restarting the SSH Service

If you’re connected to a remote server via SSH already, and you’re editing the sshd_config there is a real risk that you could inadvertently cut yourself off if you get something wrong.

The biggest tip I can give you is to NEVER logout from any existing sessions until after you have verified your new SSH config is working.

systemctl restart sshd

After you have restarted SSH, try a new connection to the server and if successful everything is fine. If you have any problems, you’ll need to go back to your existing logged in session and fix up the sshd_config (check the log files for reasons why if not clear). You’ll then need to repeat and restart the SSH service until you can login with a new session.

I hope these tips help you secure your SSH server, and until next time, stay safe out there.

 

AusCERT2016 CTF – 10th Place!

I recently finished competing in the AusCERT2016 Capture the Flag (CTF) challenge which ran for 48 hours. Coming in 10th place from dozens of active participants was very rewarding!

auscert2016_ctf_scores

I entered under the team alias “InsertCoin“- partly to protect myself if I performed terribly(!) but also because I’m currently looking for new work opportunities – and the name subtly describes my current state-of-mind 🙂

Capture the Flag competitions work by having people (usually teams) compete to carry out tasks in a limited time with the goal of finding a “flag” for each challenge.

A CTF flag is a string of text that is hidden or concealed in some way – you’ll only get it by solving a problem, hacking a system, decrypting something and so forth. There are many different types of challenges, and the problems can range from ridiculously easy to virtually impossible.

As is standard for most CTF challenges, in the AusCERT2016 challenge competitors had to find flags formatted in a special way – in this case:

flag{something_here_different_for_every_challenge}

The format of the flag is really important. Flags are like passwords and have to be entered precisely into a scoring website, otherwise you don’t get the points added to your score.

But knowing what a flag looks like also means you know roughly what you’re looking for, and this can come in handy! For example, looking for the letters f, l, a, and g to make the word “flag” means you’re probably looking in the right place; and can be a time saver.

Solving the challenge: “Law”

Let me be clear – this “Law” challenge was about as easy as a CTF challenge can get – but I’m writing this up because of how I’ve seen some other people attempt to solve it; and the different approaches you could use. Some good, some not so good.

The “Law” challenge begins by downloading a given Word Document. Have a look at the document below and see if you can identify what’s happening.

Law

It’s pretty easy to see something odd with the capitalisation of the letters. Look closely at the first line and you might see something familiar if you concentrate on those capitals.

… enForced … sociaL … behAvior.G{2] …

Oh, yes.. FLAG{ ..surely the beginning of the flag we’re looking for.

At this point, there’s a burning question that every CTF competitor asks themselves:

“What’s the easiest AND fastest way I can solve this problem?”

This is always the juncture point during a CTF where my heart skips a beat, and I feel the adrenalin rush. If you’ve ever participated in a CTF you’ll know what I mean! It’s that point where the only thing that sits between you and the answer is the HOW & TIME.

How? Here’s some of the instant choices of tools/approaches that sprung to mind:

  • [1] Use a feature built-into Word, like find/replace to remove lowercase letters, or extract the capitals or similar; or
  • Extract the text from Word and do this somewhere else, probably on a Linux box…
    • [2] Write a script or something to parse the text and give me what I want; or
    • [3] Rely on some command-line-kung-fu to extract and explore what I need

To be honest, staying in Word really didn’t enter my mind. I’m sure there are features that might do what we want, and if you’ve got suggestions please do leave a comment below and tell me, but prove to me your way is faster also).

So, this story is really about the last two options only: write a script, or do kung-fu.

If you know how to do kung-fu, is there ever any other option? No.

Be patient Grasshopper, we’ll get to the Kung-Fu part, but first what really intrigued me was the day after I had solved this challenge.

While attending a local SecTalk meetup I met some other competitors who were also attempting this same CTF challenge, so I quizzed them on how they were going about solving it…

Surprisingly, one of them had written a Python script to solve this!

My. Mind. Was. Blown. A Python script to solve THIS? Knowing how I had solved it (as you’ll see below) no doubt biased my judgement, but I just felt like using Python (or in fact anything other than a single command-line) was a heavy handed, and slow.

I didn’t look at the actual Python script, but if I had attempted such a thing it would have probably looked like this…

law-python

It outputs like this:

F
L
A
G
W
H
A
T
... and continues down the screen like that.

Sure, I could import sys and use sys.stdout.write() to fix that, but using Python isn’t the focus here.

Command-Line-Kung-Fu

On the command line, this is how I solved this “Law” CTF challenge on one line:

grep -o '[A-Z]' law.txt | tr -d '\n'

That’s simply grep with the -o option to only return uppercase ‘[A-Z]’ piped through the tr command to delete -d any newlines ‘\n’

Output:

FLAGWHATWOULDAREASONABLEANDPRUDENTPERSONDO

Converting that to lowercase and inserting the { and } in the right places gave me the flag.

flag{whatwouldareasonableandprudentpersondo}

The magic of grep -o

I suspect that some people may not be aware of the -o option for grep which only returns the text that matches the given regular expression. This is great for extracting all sorts of things from files, such as domains, email addresses and all sorts of things.

And the power of tr (translate)

There’s another way of doing this also, without using grep at all, which is to only use tr which is the “translate characters” command. In this case we can start by deleting -d lowercase ‘[a-z]’ characters.

cat text | tr -d '[a-z]'

This gives one advantage of being able to see the whole file quickly, having removed only the lowercase chars, and we see the flag beginning to emerge. By repeating this process and eliminating characters one-by-one this technique can come in handy in other challenges.

prudent-using-tr

Once your brain has worked out which characters to remove, after a few iterations you get to the final result.

cat text | tr -d "\-\"\”[a-z][0-9]:'., ()\n"

Output from this is the perfect flag that just needs to be converted to lowercase:

FLAG{WHATWOULDAREASONABLEANDPRUDENTPERSONDO}

As you can see there are many different ways to solve this challenge, and each to their own. But when time is against you in these challenges you have to make sure you’re picking the fastest method possible!

Until next time, stay safe out there – and may your flags all come to you easily!

How to replace malicious PHP pages with sneaky POST data capture

How to replace malicious PHP pages with sneaky POST data capture

Whenever I’ve had the task of personally assisting someone with remediating a compromised web server, I can’t help switching into researcher mode.  I want to know how the attack has happened, and of course to stop it from happening again, but I’m always intrigued by what the attacker is trying to achieve – and it isn’t always what you’d think.

In my experience most attackers will inject a PHP file that is nothing more than a web shell that allows them to explore other directories on the server, and perhaps run a few commands here and there – sometimes leading to a full-scale breach of the server.  Most PHP shells that I have pulled apart are heavily obfuscated with the PHP code going through a series of additional steps to effectively decode itself upon execution – and sometimes they require passwords to decrypt which are only supplied by the attacker via POST form data when used.

On other occasions I have found more complicated PHP files that act as very small and powerful remote execution gateways that evaluate code submitted on the fly via POST form data – sometimes also with a decryption key or similar.  The last time I saw this was on a web server that we being used to send a spam email for each request to the malicious file.

Unfortunately, keeping standard web logs will only get you so far because they only keep the URL and URI information and not the data of the request, and while you can implement modules in Apache to log POST data such as mod_dumpio I’ve sometimes found myself short on time, so I’ve opted for a much simpler approach.

What I do is locate the malicious PHP files on the infected website and replace them completely with the script below:

<?php
// POST data capture - replaces malicious file previously found on this system
header('HTTP/1.1 500 Internal Server Error');
$date = new DateTime();
$date->setTimezone(new DateTimeZone('Australia/Melbourne'));
$fdate = $date->format('Y-m-d H:i:s'); // same format as NOW()
file_put_contents("/tmp/post.log",$fdate . "\n" . print_r($_SERVER,true) . "\n\n" . print_r($_POST,true) . "\n==========\n");
?>

This handy script will simply return a fake HTTP Status code of 500 indicating an Internal Server Error has occurred (even though really it hasn’t!).  Many attackers are used to seeing this error, since some of the very badly written scripts they write often have syntax errors.  So a 500 error is enough to keep them trying in the hope they thing they’ve done something wrong with whatever data they’re trying to inject.

But, as well as returning the fake error, this script sneakily captures any POST data submitted to the page and appends it to the /tmp/post.log file (obviously you’ll want to change this wherever you’d like).  Once you’ve started to capture some real POST data you can replay that data yourself in an isolated environment with a copy of the real malicious PHP file to see what happens.  Often the POST data will clearly reveal the decryption password you need to unlock the code for further analysis.

Yes, this is a kludge.  Yes, it’d be nice if every network was designed to capture all network traffic for later analysis (although the complexities that come into play with SSL make that troublesome), but in the environments I’m often called upon to assist in, this does the trick nicely.

Until next time. Stay safe out there.

How NOT to build a Bank ATM

How NOT to build a Bank ATM
I was standing outside my office during a heavy downpour the other day, admiring that fresh thunderstorm feeling in the air, when we had a very brief power outage…

 

Happened to glance over at a nearby Automatic Teller Machine (ATM) and to my astonishment, firstly at the fact that the ATM wasn’t power protected (pity be the person using the machine if the power goes out), as Windows XP loaded up (apparently very common in ATM’s) I watched as the the entire start-up process flashed onto the screen displaying full path references and comments in the scripts.

 

Atm-power-up

 

I have no idea what pressing any buttons at this point would have done to the machine, but one has to wonder if this isn’t a starting point for some further exploit… Mmmmmm…

 

Extra points if you can guess which bank this was.