SSH ProxyCommand & ProxyJump

SSH ProxyCommand and ProxyJump to the rescue for jumping through multiple SSH hosts. I was asked by a fellow team member how we access RDP through two Linux hosts recently. I had to think for a moment, and remembered good old ProxyCommand and ProxyJump. So I thought I would post this up as a note to self and anybody else who might find this useful.

In this scenario imagine we are on the outside of a NAT firewall (this could be across the internet of internal network), we need access to a windows host several networks deep with only Linux/ssh hosts to jump across. Similar to the below:

How do we achieve it? Well, we can actually do this several ways with different commands. One way is to SSH to each host in turn and do a local port forward. That seems a bit long winded. Another approach would be to formulate a set of SSH commands in the /etc/ssh_config file, again a bit long winded. Alternatively, a one liner more suited in this situation would see multiple SSH commands nested together with local port forwarding. There are actually also (not surprisingly) several ways we can achieve a one liner using the likes of ProxyCommand, ProxyCommand with nc and newer versions of openssh ProxyJump.

With ProxyCommand this is what our command would look like:

ssh -o "ProxyCommand=ssh -W %h:%p -o 'ProxyCommand=ssh -W %%h:%%p -L 3389:127.0.0.1:3389 root@10.0.0.46' -L 3389:127.0.0.1:3389 root@192.168.2.100" -L 3389:192.168.3.202:3389 root@192.168.3.100 -vv

This will prompt you at each host for the password, unless you use SSH key pair with no password.

There are a couple of key elements in this one liner that we will go through:

-o : This is used to specify the option you want to use, in this case proxycommand.

-W : This is important, as this redirects the std input/output back to your host securely. Two additional options after the -W are %h which is our kali host and %p is our ssh port. This is specified in each proxycommand, note in the second command the %% this is essential to escape the characters. Note -W is not to be confused with lower case -w which is for a completely different task.

-L : This is our regular local port forward, in this case at each jump host we forwarding back 3389.

Another more neat and simpler way to do this is to use the ProxyJump command which is shortened to -J.

ssh -J root@10.0.0.46,root@192.168.2.100 -L 3389:192.168.3.202:3389 root@192.168.3.100

With ProxyJump we are setting our jump hosts in line one after the other comma separated and just setting the local port forward on our final host. This tunnels the port right through from your local box through to the destination server.

All that is left to do now is to use rdesktop or xfreerdp to 127.0.0.1 on 3389 using either proxycommand or proxyjump.

Pretty cool huh.

6th February 20226th February 2022

CVE-2021-4034 “Pwnkit” Local Privilege Escalation (LPE) vulnerability

Pwnkit, this is a bad one a really bad one as the “Policy Toolkit” “Polkit” package is installed by default on almost every major Linux operating system distribution after 2009.

Pwnkit was discoved by Qualys and reported 2021-11-18, they have a good technical write up located here: https://www.qualys.com/2022/01/25/cve-2021-4034/pwnkit.txt

The Polkit package was first released in 2009 therefore the OS needs to be newer than this date, but not updated after January 2022. It essentially allows for an unprivileged user to interact with the pkexec utility to gain full local privilege escalation with relative ease. Polkit can be thought of as similar to sudo, pkexec is the front end SUID-root program, both in gui and cli form. pkexec takes a number of command line arguments like below:

The crux of the vulnerability lies within passing x amount of arguments to produce and out of bounds write, eventually overwriting a system environment variable.

Exploit:

We will use the following exploit code by Andris Raugulis located here to gain LPE. We will simply compile with gcc and run as below:

As you can see this instantly pops root privs. For info this is a recent Ubuntu server.

Fix:

Patches are being rolled out however the recommended temporary fix by Qualys is to remove the SUID bit from the pkexec binary. This can be done with a command such as the following:
chmod 0755 /usr/bin/pkexec

If you run the exploit against a patched system it simply return the pkexec help menu:

5th January 201729th April 2017

Awk – Part 1 – Printing the 5th word in a line of text to std output

AWK

To kick off ‘Project Bash’ located here I’m going to be talking about Awk. Awk is a text processing tool that can be used to manipulate text in a line in bash. There are many uses of awk so I have called this one Part 1. In Part 1 we are going to use awk to select a word in a line of text, this can be very useful if we want a specific value in a line of text like the 5th word and want to iterate that through each line in a file. Take the following example:

So we have just used Nmap to ping sweep a subnet and want to create a target IP list from the results, we send the output to a file called alive.txt. check out the below:

Awk Part 1

If we break the command down:

cat alive.txt | grep "report" | awk '{print $5}'

The first command ‘Cat alive.txt’ prints out the contents of the file alive.txt to std output (ie into bash). We then pipe the out to a second command. We then use the command ‘grep “report”‘ to find all lines with the word ‘report’ in:

Nmap scan report for 10.0.2.15

Then finally using ‘awk ‘{print $5}’ to print the 5th word in the line using the variable $5.

Changing the variable in the awk command to ‘$2’ prints the 2nd word in the line:

As you will probably agree this is powerful tool, especially when you need to clean up some out. There are multiple ways we can do the above this is just one of them. Ace!

4th December 201630th April 2017

Sed – Part 2 – Using Sed To remove text from a line in linux

How do we remove specific text from each line in a text file in Linux? In this post we cover using Sed to remove text from a line in Linux.

We previously covered in this post adding text to a line in Linux. In this post we will be specifically talking about the opposite. Using Sed to remove text from a line in linux is fairly straight forward. To remove specific characters or portion of text from a line in Linux we can use command line bash tool sed. The tool sed is used to perform basic text transformations, more info on sed can be found here. In the below example we want to specifically remove ‘BARRY\’ from a line everything within the square brackets [text to remove] is removed, so we use:

sed 's/[BARRY\]//g'

example:

cat users | grep BARRY | cut -d" " -f 2 | sed 's/[BARRY\]//g'

Hopefully you will find this useful!

5th May 201630th April 2017

Over the wire – bandit online terminal based war games delivered through SSH! Brilliant!

Over the wire – bandit – SSH War games!

I just thought I would share with you all Over the wire – bandit this is fun hacking practice game that you can play on online through SSH. It very addictive and once you start you will want to progress through the levels. Not stopping till you have finished. I started this and had to finish it. You can find it over at http://overthewire.org/wargames/ I found this through ‘The hackers playbook 2’ book of pentesting tips and tricks found here http://thehackerplaybook.com/dashboard/

Over The Wire - Bandit

Over the wire is a set of text based games, brought to you in a ‘capture the flag’ type style game where you ssh into there servers and crack the puzzle or code. The first series called ‘bandit’ is relatively straight forward with 26 levels. You start the game with a username and password to login to their servers via SSH. You can move up a level each time cracking the puzzle and obtaining the password to the next level. The bandit series is a great intro in its own right to bash as you have to use and combine tools and functions like grep, sort, base64 etc in order to crack the code. Each level finds you figuring out how to view/capture a text string which is the password to the next level. Head over to http://overthewire.org/wargames/ and check it out.

2nd March 201623rd November 2016

Information Security Defenders

We need more Information Security defenders!

As an industry we need more information security defenders to play the role of the blueteam in the Enterprise. All too often the IT Team receive a pentest report with a list of red criticals and are immediately overwhelmed with issues to fix. Whilst an organization has regular pentests it has the danger of lulling itself into a feeling of ‘doing the right thing’ however if all they are doing is getting a pentest and not re-mediating the issues then its a waste of time and money. This is a pessimistic view however it is usually the case that more vulnerabilities are released that the IT know what to do with. The IT team tend to either not have enough resource or they don’t have the skills to fix the issues, usually the first being being the case. Penetration testing needs to be more than just a test, it needs to be a more proactive engagement with after test help and support for the enterprise should they need it.

Its easy to criticise the IT department. As Information Security Professionals, we are doped daily with the next new exploit or zero day from the various blogs and podcasts that we listen to. This isn’t a small IT departments main concern or top of their list of to do tasks when they arrive at work, (although it could be argued it should be, with a little more security awareness training). The IT team tend to prioritise keeping those critical business apps or switches up and running (I’ve been that person).

As Information Security Professionals we need to be more involved in fixing and explaining the risks and supporting organisations so that they are better equipped to defend themselves.

The next new shiny security toy in the enterprise may not necessarily be the answer, the existing tools already installed around us could be better used and more creatively used to defend the enterprise. Getting the latest IDS or implementing the next new NextGen Firewall can tend to be a distraction to getting down to what needs to be done i.e. Getting that MS08-067 patched on that business critical server that’s been there for years or tightening up the existing firewall. Hardening existing systems with group policy, tuning WSUS or SCCM to deploy timely updates and utilising windows logging to alert when a domain admin group is modified for example will generally go far further, and are free.

Hopefully sites like this will help contribute to securing enterprises, and assisting system administrators to better defend their existing environments. Enterprises don’t necessarily need to spend huge amounts of money on new technology, the answer could be in being more creative with what you already have.

20th February 201623rd November 2016

Lets Encrypt!

Lets Encrypt! Free Certificate!

Lets Encrypt

I thought I would run through the process of obtaining a certificate for a web server from Lets Encrypt the open certificate authority by the Internet Security Research Group (ISRG) , for free!

The Lets Encrypt initiative has been setup to simplify the process of obtaining a certificate and make it more accessible for people to use encryption on their websites you can read more about it here: https://letsencrypt.org/. Certificates are free and valid for 90 days and need to be renewed for continuation of the service. (which can be automated). Those who have been through the process of provisioning certificates will understand the involved process to get it working, this is a breath of fresh air in comparison.

Sounds great so what do we need to do? Well the below example runs through the process for an already created standard HTTP apache debian based system.

OK So Lets Encrypt!

First we need to obtain the Lets Encrypt files. We will pull the files across from the Official GitHub repository. We’ll do this through git. This ensures that we have the latest version of the Lets Encrypt files. We will drop it under the /opt folder.

If you don’t have git installed… install it with:

apt-get install git

Then pull across the Lets Encrypt folder with:

git clone https://github.com/letsencrypt/letsencrypt /opt/letsencrypt

(If you want to update in the future run ‘git pull’ in the same folder.)

Move into the lets encrypt folder:

cd /opt/letsencrypt

We then need to install the Lets Encrypt client through the ‘letsencrypt-auto’ command, when this is run it will essentially pull down all the related dependencies from your OS repositories and update the client through either apt-get or yum for example. From there on you can run either ‘letsencrypt’ or ‘letsencrypt-auto’ command for new certs of renewals.

Next we will obtain our certificate and bundle for our test domain testdomain.co.uk and subdomain www.testdomain.co.uk with the letsencrypt-auto command. This is the recommended method from the Lets Encrypt website.

There are a number of plugins that can be used with letsencrypt run command, and a number of command line parameters you can pass, these are all designed to help ease the process along. We are going to be running the Apache plugin and so will pass it the ‘–apache’ command. This plugin is designed to be used with Apache (funnily enough!) which automates the process of obtaining and installing the certificate with Apache2.4 on Debian based systems. This effectively sorts all the certificate configuration within Apache and then restarts the service.

./letsencrypt-auto --apache -d testdomain.co.uk -d www.testdomain.co.uk

After this is executed you should see the updates scroll through pulling content from your repositories, after a while you will presented with the following screen asking you to confirm your email. This is used as a reminder for renewal:

You can automate this process passing the –email parameter at the command line if you wish. So ‘–email admin@testdomain.co.uk’

You will next be asked to agree to the terms of service:

Again you can automate this by passing the –agree-tos in the command line.

Next you will be asked where you want to access your site through http and https or just through https.

You will then be presented with the ‘Congratulations!’ screen saying you have successfully configured your certificate and enabled https:

Renewal

To renew your certificate run:

letsencrypt-auto renew

This will renew all your certificates with all previously used parameters for certificates that are due to expire within 30 days. passing the -d parameter will renew per domain.

--force-renewal

Allows you to renew before 30 days.

You can also run:

letsencrypt-auto renew --rsa-key-size 4096

This will renew your certificate with a key size of 4096 bit.

You can also rerun the existing command you ran earlier in which case you will be prompted that you have already run the command and to either re-run the install or renew your certificates.

All that is left to do is to automate the renewal with a cron job, one for another post.

Amazing that’s it! This makes obtaining a certificate for a website very accessible to people due to the way the scripts/plugins automate the apache configuration.

**Thumbs up Lets Encrypt this is great project!**

7th February 20162nd January 2017

Server Hardening: HTTP TRACE TRACK Methods Allowed – Part1 Apache

Many vulnerability scanners will often bring back HTTP TRACE TRACK Methods Allowed against Apache and Microsoft web servers of the older generation. TRACE is usually associated with Apache and TRACK for Microsoft. This has a CVSS score of 4.3 and is a relatively easy fix. Clearly the older generation operating systems should be migrated to a supported platform, both the later distributions of Ubuntu and Microsoft 2012 R2 do not allow these methods to be used. However a simple way to validate this finding is to use telnet to connect to the web server on port 80, once connected you can type something similar to the following for each method. The ‘Host’, ‘TestA’ and ‘TestB’ aren’t needed however if you use some custom text you will be sure to see it echoed back by the web server if trace is enabled.

TRACE / HTTP/1.1
Host: 192.168.0.29
TestA: Is this correct?
TestB: Are we sure?

Tap return twice to send.

Which would look something like the below as you can see the user input was returned, the web server accepting the method:

Remediation:

As I said the HTTP TRACK / TRACE issue is this is relatively straight forward to fix, simple add ‘TraceEnable off’ somewhere in your main Apache config file outside of the vhost configuration.

Once implemented retesting should reveal that the method is not allowed:

after adding 'TraceEnable off' HTTP Trace disabled on Apache — After adding ‘TraceEnable off’

30th January 20163rd March 2018

Server Hardening: Securing SSH part 2

Secure up your SSH service

This is Part two of Securing SSH in the Server Hardening Series. In this post we will continue to walk through the remaining hardening options for SSH.

In Part 1 of Securing SSH located here we discussed:

Disabling SSH Protocol 1 and using 2.
Limiting the users who can login.
Disable root login and unsing only a standard user account.
Run SSH on a different port.

In Part 2 of Securing SSH we will now cover the remaining techniques:

Use Public Private keys for Authentication
Filtering SSH with iptables
Setting strong Cryptographic Algorithms

Use Public Private keys for Authentication

As we know passwords can be subject to brute force attacks and given enough time simple passwords can be broken. Using SSH public private key pairs for authentication is a more secure method. Its a more secure method of logging in than using a password due to the computational power and time needed to decipher the public private key pair through brute force. The private key is used on the machine from which you are logging in from and the public key is used on the server or machine which you are establishing an SSH session with. In addition to using the key pair you can use a passphrase to secure the key pair. Should the key pair become compromised you have a an additional fallback and time in which to remove the public key from server it is being used on.

First we need to create the key pair, you can create either an RSA (Rivest-Shamir-Adleman) or DSA (Digital Signature Algorithm) key pair. The default key size in Ubuntu is currently 2048 however you can specify the -b parameter for a high keysize for example ‘-b 4096’. In this example we are creating a keypair for the root login, you should ideally do this for a standard user however this is just to demonstrate the procedure.

In this example we are creating the key pair on the same test ubtunu 14.04 server as the one which we are going to be using the keys to actually access. In reality there are a number of ways and platforms from which to create the key pair, whether that’s on the server or client OS.

ssh-keygen -t rsa -b 4096

After the above command you will be prompted with some further questions, what and where the keys will be stored. Press enter and move on to accept the default files name, you can specify your own file name, the files will be created in the current working directory.

You will next be prompted to enter a passphrase, you have the option to enter one here. Doing so adds a layer of complexity for anyone that has already obtained the key pair and thus further hardens your configuration. If you enter a passphrase you will need to enter it in every time you ssh to the server.

The whole process will look like this:

If we now look in the ssh folder in the home user directory for root (or where ever you chose to save the files) we will see our key pair. The private key is called id_rsa and the public key is id_rsa.pub:

At this point it would be wise to backup both sets of keys. You will need to copy off the private key for you to use from your local machine that you will be ssh’ing in from, we could do this in many ways either using winscp from a windows operating system alternatively we can view the private key and use copy and paste the contents from an existing session. Remember the folder is hidden, with the period in front of the folder name (.ssh) if your looking for it in winscp ;-).

We will need to move the public key into the ‘authorized_keys’ file on the server you are going to be using the keys for (the same server in our case), and restart SSH. The .pub file should be copied over to the correct profile you will be ssh’ing into.

cd .ssh
cat id_rsa.pub >> authorized_keys
service ssh restart

If I had created the key pair on my client machine I could use the ssh-copy-id command in order to get the public key onto the server like below:

ssh-copy-id -i .ssh/id_rsa.pub root@165.227.237.152

At this point you are ready to test the rsa key authentication method. I would strongly recommend you test this at this stage whilst you still have regular username and password ssh authentication enabled, in case you have made a mistake and can’t get back in. If you going to be testing from putty or winscp you will need to use puttygen to convert your private key file into a format that putty can read which is a ppk file. This is a straight forward task.

Once tested and you are ready to start using your key pair you will want to remove the ability for ‘root’ (and other user in this example) to login with a username and password. Only do this after you have successfully used your key pair and are comfortable with the process. In the ‘sshd_config’ amend the ‘PermitRootLogin’ to read ‘PermitRootLogin without-password’ and add a line that reads ‘PasswordAuthentication no’ like below.

Restart ssh service one final time. This will deny access via username and password.

Filtering SSH with iptables

I demonstrated the use of iptables in one of my previous posts here, so won’t be going into this again.

Setting strong cryptographic algorithms

In this particular section of the post we are going to look at how we can customize the cryptographic algorithms that SSH can use and further optimize these so we are only using the most secure ones available. To dig a little bit deeper here there are two specific issues we are going address. The first being the type of encryption mode that is being used, and the second being the use of weak MAC algorithms.

At the time of writing (as this will change) your average vulnerability scanner will detect ssh on port 22 and will try to negotiate a session with the service. In doing so it will detect the cryptographic properties that the server would like to use, in your typical out of the box setup CBC (Cipher Block Chaining) encryption mode and MD5 or 96-bit MAC (Message Authentication Code) algorithms will be configured, both of which are considered weak. The use of CBC encryption mode for SSH is currently scored as CVSS Base Score 2.6 equating generally speaking as a Low risk. We can correct this by disabling CBC encryption mode, and enabling a different type of mode such as CTR or GCM. The use of weak MAC is also scored with a CVSS Base Score of 2.6 equating again generally speaking as a Low risk. This can be correct by removing the use of MD5 and 96 bit MACs.

Amend or appending the following lines to the /etc/ssh/sshd_conf file will resolve this issue, there are other variations that you could use however this will rectify the issue:

Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128
MACs hmac-sha1,umac-64@openssh.com,hmac-ripemd160

I hope this has been informative.

16th October 201523rd November 2016

Server Hardening: Securing SSH Part 1

This will be the first of two posts in the server hardening Series where we will discuss the Secure Shell (SSH) service. I will cover the options available to us for hardening SSH. SSH is a cryptographic network protocol used for remote management over an insecure (or less secure) communication channel. Whether this is a web server, an appliance or a firewall, out on the internet or in your local subnet. For example most remote Linux based servers have SSH enabled in order for administrators to remotely manage them rather than being directly in front of the server/appliance with a monitor at the console. According to arstechnica (2015) Microsoft is also to introduce native support for SSH in 2015 – so watch this space.

There are a number of hardening techniques that we can undertake to further secure the SSH service from an out of the box typical install. These include:

Disabling SSH Protocol 1 and using Protocol 2. (Part 1)
Limit the the users who can login. (Part 1)
Disable root login and use a standard user account. (Part 1)
Run SSH on a different port to 22. (Part 1)
Use Public Private keys for Authentication. (Part 2)
Filter SSH with iptables (demonstrated in one of my previous posts here) (Part 2)
Setting strong cryptographic algorithms (Part 2)

I will be demonstrating these tasks on Ubtuntu 14.04. however the options and configurations will be very similar across the different Linux distributions.

Disabling SSH Protocol 1 and using Protocol 2

Disabling SSH Protocol 1 is done in the following file ‘/etc/ssh/sshd_config’, so using your favorite text editor nano, vi, leafpad etc (mines nano) open up the ‘sshd_config’ file and find the protocol line and ensure it has a the ‘2’ parameter next to it like the below:

# Protocol 1
Protocol 2

Limit the the users who can login

Locate the authentication section again in the file ‘/etc/ssh/sshd_config’, and add in the line ‘AllowUsers Adam Mark’ and any other usernames as needed. This should be used in conjunction with disabling the root login. This will only let ‘Adam’ and ‘Mark’ login for example.

AllowUsers Adam Mark

Disable root login and use a standard user account

Locate the authentication section and specifically the ‘PermitRootLogin yes’ and either comment the line out with a # and add a new line in or change the parameter to ‘no’.

# Authentication:
LoginGraceTime 120
# PermitRootLogin yes
PermintRootLogin no

Run SSH on a different port to 22

This can clearly be achieved in a number of different ways depending on how your infrastructure is configured. For example you could change the port on your SSH Service in the sshd_config file or adjust port forwarding/translation rules on your firewall or router. The main goal of this exercise is to obfuscate the SSH service to a potential attacker, we must be clear here this doesn’t protect the port in any way however does distract from the fact port 22 is not open for business. I say again this will not protect you against an attacker with enough intent, an attacker with enough experience will have scanned all 65k ports and verified all services on all open ports. Never the less if somebody has only scanned the top 1k ports its still an option, but should not be relied upon. I’m not going to get into the debate of ‘security by obscurity’ in this post. From the following file ‘/etc/ssh/sshd_config’, locate the ‘port 22’ line and change the number parameter to the unused port of your choice

# I have changed the default SSH port from 22 to 3333
Port 3333

I have added a comment in the line above as a reminder here to allow the next person to see what I have done for any troubleshooting purpose. You never know when you will move on and the next person needs to administer the box, this is useful to them. 😉

In most cases the SSH service will need to be restarted in order for the changes to take effect. From a terminal: (I always add a ‘-v’, optional for verbose)

Service ssh restart -v

As always I would suggest you try these out first in a test environment making one change at a time before changing your corporate machines.