There are various uses for having a persistent SSH tunnel, similar to why you’d want a VPN but less complicated since network routing is not in the scope of these tunnels.
This tutorial will cover how to use the tool autossh along with a systemd service to keep it running. Examples will be for CentOS, however only package installation will be different.
- EPEL repository installed (covered below)
- Password-less ssh authentication between two machines (not covered here)
- Create persistent SSH connection from one machine to another
- This connection will include a remote port forward
- Machine A (remote) will be able to open a remote listen port on machine B (local)
There are many, however the one we are focusing on here is the ability to access a remote socket on machine A from a local port on machine B. One of the most common reasons for this would be a “poor mans VPN”.
Specifically, this guide will cover a remote machine [A], which is behind a firewall which we cannot open, and a local machine [B] which we can open the firewall on. [A] will start a ssh tunnel with [B], and open a remote listen port on [B] that becomes the normal ssh port back on [A].
[A] ssh -> [B], opens remote port 5001 which leads back to [A]’s port 22.
[A] Firewall) ~~~~~~~WAN~~~~~~~ (Firewall [B] Outbound SSH ----b.hostname.com:22------> localhost:5011 localhost:5011 on [B] now leads to localhost:22 on [A] (via tunnel) [B] Firewall) ~~~~~~~WAN~~~~~~~ (Firewall [A] Outbound SSH ----locahost:5011---------->localhost:22 (via tunnel)
Now a user on [B] can ssh to localhost:5001, and that will actually take them to [A]’s ssh socket. No firewalls need to be opened from [A].
——————– Important ——————–
The machine that initiates [A] the SSH needs to have all of these steps done, the machine on the receiving end [B] only needs an SSH key set up (which we don’t cover here).
Installation is pretty easy, you’ll need the epel-release repository if its not already installed.
$ sudo yum install epel-release $ sudo yum install autossh
There are two ways to do this, and I’m going to briefly explain both. Either most of your configuration is done via the systemd service file, or via the ssh config file. I find the ssh config file to be easier to update and I’m used to making most of my changes there so that’s how this guide will do it.
Time to create the systemd service, I’m calling mine “callhome-ssh.service”.
$ sudo vim /etc/systemd/system/callhome-ssh.service ----------------------------------------------------- [Unit] Description=Service for autossh After=network.target [Service] ExecStart=/usr/bin/autossh -M 0 -t -N b.hostname.com ExecStop=/bin/kill -HUP $MAINPID ExecReload=/bin/kill -HUP $MAINPID Restart=on-failure RestartSec=60 [Install] WantedBy=multi-user.target ----------------------------------------------------- $ sudo systemctl reload-daemon $ sudo systemctl enable callhome-ssh.service
A few things to note here:
- This tells systemd to only start this service after the network system has come up
- ExecStart=/usr/bin/autossh -M 0 -t -N b.hostname.com
- This is the command that will be executed
- -M 0 tells autossh to disable its monitoring feature (we will compensate for that)
- -t -N are passed to SSH, these stop the creation of a terminal session (we will be blocking the terminal later)
- host.com, this is your remote hostname
- ExecStop=/bin/kill -HUP $MAINPID / ExecReload=/bin/kill -HUP $MAINPID
- These tell systemd how to stop and reload the process
- This tells systemd to restart this process if it fails
- The default systemd restart time is 100ms, this tries to restart the service every 60 seconds
Now we will configure the actual SSH part. We are going to use the root user, and I am assuming that key files have already been generated for both boxes.
$ sudo su $ mkdir -p /root/.ssh $ chmod 700 /root/.ssh $ vim /root/.ssh/config ----------------------------------------------------- Host b.hostname.com HostName b.hostname.com User mfox Port 22 IdentityFile ~/.ssh/keys/b.hostname.com.key RemoteForward 5001 localhost:22 ServerAliveInterval 30 ServerAliveCountMax 3 ExitOnForwardFailure=yes
Lets explain a few things:
- The first three commands make the .ssh folder and set its permissions, even if it exists already this wont break anything
- Host b.hostname.com
- This is the host that will be referenced from the ssh command – it can match the actual hostname, or it can be a short nickname instead
- HostName b.hostname.com
- The actual hostname or IP for the remote box [B]
- User mfox
- Username for SSH to use on the remote machine [B]
- Port 22
- Port on the remote machine [B] to use for SSHing to
- IdentityFile ~/.ssh/keys/b.hostname.com.key
- This references the key file for SSH to use, not covered here
- RemoteForward 5001 localhost:22
- Tells SSH to open remote port 5001 [B], and map it to local port 22 [A]
- ServerAliveInterval 30
- This tells SSH to send a keep-alive packet over the tunnel so it dosn’t timeout
- ServerAliveCountMax 3
- This is the amount of times that the keep-alive packet is sent, if no response after three then the session is closed
- Session is closed if the port forwarding fails
I feel like these options need further context, just so you know exactly why we are doing it this way. Technically the “ServerAliveCountMax” isn’t needed because the default is 3 already, but I felt like it was good to keep the option in there in case it needed to be changed.
Other then the port settings, the last three options are crucial to keeping a steady connection. The ServerAlive options make sure that the SSH tunnel is talking back and forth. If its not talking, we want the tunnel to close ASAP so it can be re-established. If there are any errors with port forwarding, either a actual problem or perhaps another session is open already holding these ports, we also want the tunnel to close. It’s important to keep the tunnel up as long as possible, which the ServerAlive does, but also to close it as soon as possible if its not working, which is also accomplished by ServerAlive.
You should be able to test SSH now by just trying the following command:
$ ssh b.hostname.com
The prerequisite was to already have a working password-less authentication set up so this should have worked before and it should still work
Optional step – securing the remote key if its not trusted. Again this is assuming you already have the authorized_keys file configured on [B] – which is not covered here, you just need to append this to the key you’ll be using:
command="echo Key for port forwarding only",restrict,port-forwarding
Adding this to your authorized key will stop anyone trying to use the key to get regular command line access back to your machine. You should not be using this key for regular access, as it will no longer work once you add that. Be sure to test this and make sure the key doesn’t work for normal SSH sessions. Again this is not needed if you trust the remote machine where your key will sit.
At this point you should be able to start the systemd service on [A] and see if it works:
[A]$ sudo systemctl start callhome-ssh.service [A]$ sudo systemctl status callhome-ssh.service ------------- [B]$ ssh user@localhost -p 5001
[A]$ sudo journalctl -u callhome-ssh.service -----or----- [B]$ sudo tail /var/log/secure
Those two logs should give you some insight into what happened.
It seems like a lot of steps and looks confusing, but it’s really not. If you boil this down, it condenses to this:
- SSH key files created and put on each box
- autossh installation
- Systemd service file creation
- SSH config file creation
- Edit working SSH key file (optional)