Automatically reconnecting ssh sessions at wakeup

In order to keep the Sprend server running smoothly we use different kinds of monitoring and trouble alert systems.

A part of that monitoring system is me keeping an eye on what is going on at the moment. I keep three Terminal windows open as shown in the screenshot. Each is connected with ssh to the server itself.

The topmost window is running htop that shows the cpu and io load. The middle one is showing the log. The bottom window is logged in to an irc channel for team communication.

Every morning when my developer machine wakes up from sleep the ssh sessions have disconnected of course. So I go into each of the windows to reconnect to the server, start up htop, start up the log, and reattach to the irc session. Some mornings I'm just to lazy and stay disconnected. If only there was a way to reconnect automatically...

And yes, yes, yesterday was the day when I made it work. What was needed was simply a utility program that runs a script on wakeup from sleep that starts the Terminal which is set up to open up a group of three windows, each belonging to its own window profile that runs the ssh command when it opens, which logs in automatically via a public key stored on the server, and then opens a full terminal and runs the correct command in it.

Or, in a step by step instruction, do the following:

Step 1: Auto-login
Set up a public/private key so that ssh can log in without a password. I didn't know it was so simple to set up.

Step 2: Prepare three calls to ssh
Just logging in to the server is not enough. We need to run a command in a full terminal once we've logged in. Luckily that is only a flag away with ssh: -t.

To log in and start htop:

• ssh -t user@server 'htop'

To log in and show the log I'm interested in:

• ssh -t user@server 'tail -f -n500 log.txt'

To log in and reattach to a screen session with the name irssi.

• ssh -t user@server 'screen -d -r irssi'

The screen session was previously created with screen -S irssi.

Once you have made sure the commands work as expected, move on to the next step.

Step 3: Create three window profiles in Terminal
Open the Settings for Terminal. Pick your favorite window profile and make a duplicate. On the tab Shell enter the first of the ssh commands from step 2 (see screenshot). Make two more duplicates and enter the other two ssh commands.

Now quit Terminal and start it again. At least that's what I had to do to be able to use the three new window profiles.

Step 4: Create the window group

In Terminal, go to Shell / New Window and open up one window of each of your new profiles. Position the windows to your preference on the screen. If all works, this is the configuration you want to keep; three windows displaying htop, the log, and the irc channel.

In the menu Window, select Save Windows as Group. Pick a good name and check the checkbox Use window group when Terminal starts. Quit and restart Terminal to see if it works.

You will also want Terminal to start on bootup. Rightclick its icon in the dock and make sure Open at Login is checked.

Step 5: Install SleepWatcher

The last step is making sure Terminal starts when the computer wakes from sleep.

1. Download SleepWatcher.

2. Run both install packages, sleepwatcher.pkg and SleepWatcher StartupItem.pkg.

3. Open a regular Terminal window.

4. Create the .sleep text file and add in the following:

• ps axco pid,command | grep Terminal | awk '{ print $1; }' | xargs kill -9

Note that this is going to kill the Terminal process before going to sleep. It works for me but perhaps not for everyone else.

5. Create the .wakeup text file and add in the below commands. Note: The sleep will give the wifi time reconnect - you may not need it.

• sleep 3s
• open -a Terminal

6. Make the scripts executable:

• chmod +x .sleep
• chmod +x .wakeup

Done!

I hope somebody else will find the instruction useful. This problem has bugged me for some time and it was really nice to be able to put all the nuts and bolts together.

The version of Terminal is 2.1.