The situation goes as such, if you can visualize it, where you have a dedicated server hosted at an off-site location (disaster recovery, branch office, etc…), you have your primary location, and then you have a third party location somewhere in the middle — perhaps cloud hosting… From your primary location to your secondary site your transfer speeds are terrible and, if you’re using this for a DR site, it quickly becomes agonizing when off-loading backups or other important data. However, from your tertiary location you get great routing, great speeds, to your secondary location, and from your primary location to your tertiary location it’s as fast as you can go. At your DR site, you have an FTP server connected to a NAS, a SAN, or some other mass storage unit, and you need to be able to quickly and securely ship backups and other sensitive data over the line. With the routing problems from your primary site, you find yourself spending way too much time trying to figure out the best way to get the data from point A to point B rapidly. Somehow, you ponder, there must be a way to utilize the tertiary host as a hop in the trip from your primary site to your secondary site…

And, we don’t get a budget for this because of the economy, and cut-backs, and blah, blah, blah… But, management wants it done. They’re satisfied with you working late and weekends, dedicating all of your free time to ensuring that the uber-slow transfers have succeded on a daily or weekly basis — you’re not. Sound familiar? So, using the tools at your disposal, you need to setup a traffic bouncing mechanism at the third location to act as a transfer intermediary between your primary site and your DR site.

The primary transfer protocol is FTP. Presently you’re archiving, packing, and password-protecting zip files or other compressed archives to ensure that a man-in-the-middle attack doesn’t comprimise your customer’s data. Maybe you’re even using SSL encryption for the broker and passive ports to further secure your communications — fine, take all of the precautions necessary… For one reason or another you need to keep FTP as the primary protocol, so tunneling an sftp or rsync connection is out of the question. The passive port range is 10000-11000. The tertiary box is used for almost nothing, and you know that you’ll always have ports 10000-11000 available… So, the goal here is to setup the third box as a secure tunnel between your primary site (the client) and your secondary site (the server), ensuring that all of the passive data ports are encrypted, and that all of the traffic is first being routed in the order of Site1->Site3->Site2 so that you get the speed benefits.

So, we’ll need to McGuyver some SSH tunnels for port forwarding. But, there are so many ports we’ll need a script of some sort that can automatically handle the negotiating. First thing’s first, you’ll need to exchange SSH keys between Site3 and Site2 unless you want to have to type your password a whole bunch of times every time you start the traffic bouncer… Secure the keys in whatever manner you feel appropriate.

Now, the bare metal:

#!/usr/bin/perl -w

use strict;
use warnings;

my ($localip,$remoteip,$user,$ftpport,$localport,$pasvstart,$pasvend) = @ARGV;

usage() unless $localip||$remoteip||$user||$ftpport||$localport||$pasvstart||$pasvend;

my $ra_tunnels = makeSSHStrings(&constructPorts(&makeRanges($pasvstart,$pasvend)));

# Broker the FTP connection tunnel...
print "Establishing Broker tunnel...\n";
my $cmd = "ssh -f -N -L $localip:$localport:$remoteip:$ftpport $user\@$remoteip";
system($cmd);

# Establish the PASV port tunnels
print "About to create ".scalar(@$ra_tunnels)." tunnels\n";
for my $i (0 .. (scalar(@$ra_tunnels) - 1) ) {
	print "\tCreating tunnel: \#$i\n";
	my $tunnel = $$ra_tunnels[$i];
	system($tunnel);
}

print "Done. Connect to your bouncer at: $localip:$localport\n";

# Break out the SSH strings from the PASV port range
sub makeSSHStrings {
	my $ra_ports = shift;
	my @listen = @$ra_ports;

	my $str;
	foreach my $port (@listen) {
		my $conn = "ssh -f -N ".$port." $user\@$remoteip";
		push @$str,$conn;
	}
	return $str;
}

# Constructs ssh's -L option on a per-100 port
# array-entry based off of START:END, supplied
# from makeRanges()
sub constructPorts {
	my $ra_blocks = shift;
	my @blocks = @$ra_blocks;

	my $ports;
	foreach (@blocks) {
		my $start = (split(':',$_))[0];
		my $end   = (split(':',$_))[1];
		my @str;
		for my $port ($start .. $end) {
			push @str, "-L $localip:$port:$remoteip:$port";
		}
		push @$ports,join(" ",@str);
	}
	return $ports;
}

# Split the ranges into 100s and delimit a start
# and end port with a colon. This can later be
# used for easy splitting and construction.
sub makeRanges {
	my ($start,$end) = @_;

	my @hundreds=($start);
	while(1) {
		last if $start == $end;
		$start=$start+99;
		push @hundreds,$start;
		$start+=1;
		push @hundreds,$start;
	}

	my $blocks;
	my $i = 0;
	foreach (@hundreds) {
		push @$blocks,$hundreds[$i].':'.$hundreds[$i+1];
		$i+=2;
		last if $i == scalar(@hundreds) - 1;
	}
	push @$blocks,$hundreds[-1].':'.$hundreds[-1];

	return $blocks;
}

# Exit gracefully.
sub usage {
	print <<"END";
	./$0 <localip> <remoteip> <user> <ftp port> <local port> <pasv start> <pasv end>

    ---------------------------------------------------------------------------------
    localip       = the ip address you want the bouncer to bind to.
    remoteip      = your site's remote ip address.
    user          = username on your remote site
    ftp port      = the port that your remote ftp server runs on.
    local port    = the port you want to listen for ftp connections on the bouncer.
    pasv start    = beginning of allowed pasv ranges for your site.
    pasv end      = end of allowed pasv ranges for your site.
    ---------------------------------------------------------------------------------
END
	exit();
}

We’ll run this script on the tertiary server, where the remoteip is the DR site’s IP address… localip will be the IP address on the tertiary server that you want to bind to… ftp port is the broker port (usually port 21, but hopefully you changed this…) … local port will be the port that you want to listen on for the tertiary server — this is the port that you will connect your client (from your primary site) to, so make it something obscure and preferrably different from the port you’re listening on the secondary site. For the pasv start and pasv end, that’ll be specific to whatever you have configured for the passive port range on your secondary site — I used 10000-11000 for the purposes of this example. The user will be the username on the secondary box that you exchanged ssh keys with from the tertiary box.

Once you have run the script and the tunnels are established, you can connect to the tertiary box on the port that you specified … Notice the speed difference? Try xferring a file… Soak it in…

I want to stress that this really is a poor-man’s traffic bouncer setup and probably should not be used in a production environment. This is, however, a quick and dirty way to accomplish a considerable speed increase when you need it fast, and don’t have the time or the money to deal with the problem through another avenue. I hope this comes in handy for somebody; I myself have used this several times on colocated development setups where I was getting awful speeds from my house to the box, but amazing speeds from one of my colocated servers to it.

email with questions.

-dan

Leave a Reply

(required)

(required)

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

© 2013 Dan's Blog Suffusion theme by Sayontan Sinha