OpenSSH: Proxy Connections

prerequisite concepts: prelude, basic configuration, port forwarding

Network address translation (NAT) is a very common method of providing secure access to hosts on a private network. Given the limited amount of IPv4 addresses, computer networks with relatively few, very few, and even a single public IP address are common. A typical small business customer of my consulting practice has one or more Linux servers on an office network protected by a firewall. The following is a close look at Example Industries, the theoretical owners of example.com; this customer receives support for two Linux servers, a mail server and a PBX, but only one public IP address between them. Through NAT, public services (namely mail and VoIP) on both servers are accessible via example.com. This works well for inbound mail and phone calls, which only need to access one or the other host, but SSH access is the lifeblood of remote system administration, and there's the rub-- when I enter ssh example.com I land at the mail server. SSH access to the PBX would seemingly threaten to litter my command line with unsightly extra characters, if not subsequent commands outright.

My carpals are tunneled enough, I don't want to type more than ssh mail and ssh pbx to access these servers, and while I'm at it I want to have scripted log-ins as well-- securely, not those namby-pamby no-password keys. In fact, I don't even want to have private keys on either server.

Fear not! With the power of OpenSSH, I can fix this.

Recipes for SSH proxies are like homespun cure-alls: few do much good and some are actually harmful. As indicated in my introductory rant, I have a few criteria for this sort of thing:

• No interactive passwords.


• No password-less keys.


• No private keys on servers.


• Avoid command line options.


• Demur scripts.

Previous installments in this series have intentionally focused on declarations which will help define my Example Industries SSH client configuration; here's the file so far:

# /home/garrison/.ssh/config

# Global Options
Host *
ForwardAgent yes

Host mail
HostName example.com

Host pbx
HostName example.com
LocalForward 8080 localhost:80
LocalForward 3306 localhost:3306

At this point ssh mail and ssh pbx are functionally equivalent in that both land me on the mail server, and I want forwarded access to pbx's web configuration and database but I end up with webmail and mail user database (which I can access without port forwarding). While I can certainly connect to pbx once I'm in mail, I must specify it's internal address (ssh 192.168.1.20) and the services I require will not be forwarded to my workstation. What I need is a way to tell SSH to bypass mail and connect me directly to pbx.

Rubbish. In life, particularly in technology, we often confound our challenges by mischaracterizing the requirements of a solution as I just have. It is no coincidence that my most successful customers are also the ones who make use of my ability to solve problems, rather than simply implementing solutions. Often the most challenging part of a solution is correctly stating the problem. What I really need is just for ssh pbx to connect me to pbx and forward my ports; I don't really care whether mail is bypassed or not, so long as it stays out of my way.

As it turns out, the solution I favor does not bypass the gateway host (mail) at all, but uses it as a proxy for my connection to pbx. With the addition of a ProxyCommand directive to my SSH config, I can achieve all my objectives.

The ProxyCommand directive is a subtle beast, and my early attempts to use it were unsuccessful. At the time I was doing something similar on the command line: ssh -t example.com ssh 192.168.1.20 Because I initially hoped to "do the same thing in the config file" I mistakenly assumed that ProxyCommand would allow my to connect to mail and immediately fire off a connection to pbx; LocalCommand behaves this way but doesn't allow me to accomplish what I can with ProxyCommand.
After a few unsuccessful syntax variations, I began to suspect that I might have the wrong idea about this directive.

When I eventually sorted out the correct syntax, I knew I had the wrong notion because I had no clue why one version worked and the others did not. Richard Silverman, one of the authors of the snail book was kind enough to set me straight. He explained:

ProxyCommand specifies a program which the SSH client will use to contact the remote SSH server. Instead of opening a TCP connection, it runs this program and uses its stdin/stdout as the communications channel.

I then understood that with ProxyCommand in play, SSH expects the command it executes to provide the TCP connection between mail and pbx; netcat, a phenomenally useful tool, was designed for just this sort of task:

ProxyCommand ssh example.com nc -v %h %p

Adding this directive to the Host pbx section of my config gets the whole proxy business out of my way and I can connect with just ssh pbx; tho forwarded HTTP and MySQL connections are just the beginning. I can use scp, sftp, FuseSSH, sshfs or anything built on SSH just as if pbx had a public IP. One more example:

rsync -Hav pbx:/usr/stuff backup:/archive

I often use such a command to transfer data from a machine with no public IP address to a backup server which also has no public IP and lives on another private network in a different town, state, or country. This is all done with ProxyCommand directives, over secure SSH connections, and most importantly, with no special command line syntax. What could be easier?