It’s been a very busy few weeks. Our office lease expired at the beginning of August and we were forced to move in order to get more space. We completed the move two weeks ago.

In addition to more space, we also got more bandwidth: 30 megabits, or thereabouts. Unfortunately, though, our provider (Time Warner Cable) can’t give us a single 30-megabit connection. They can, however, give us three 10 megabit connections. This is very odd considering that the installation consists of a single piece of coax feeding into a splitter to which are attached three cable modems. It sounds fishy, I know, but I have verified that we’re getting a combined 26+ megabits out of those three modems. Why I can’t get a single modem capable of 30 megabits is beyond me, but I know better than to try fighting the cable company.

In any case, we can’t really make use of three individual 10 megabit connections, so we need to bring those three WAN connections together and feed a single LAN. This won’t give us 30 megabits for a single connection, of course, but it will give us an effective 30 megabits download speed over multiple connections. That’s perfect for my Web crawler.

It turns out that multi-wan routing isn’t as simple as I thought it would be. Dual WAN routers are plentiful and cheap, and apparently work quite well. Multi (i.e. more than two) WAN routers are harder to come by, and are usually more expensive. There are some inexpensive units, but they top out at 25 megabits total throughput (upstream plus downstream). The least expensive unit that looks like it’d do we need is priced at about $2,000. And I couldn’t find any information about how well it works.

David did some research and found pfSense–an open source firewall based on m0n0wall and running on OpenBSD. pfSense is the BSD equivalent to IPCop, although pfSense has many more features, including support for multiple WAN connections. There are apparently some IPCop hacks that support multiple WAN connections, but from what I’ve read they are not highly regarded.

I won’t regale you with all I went through getting pfSense working with this configuration. It took me much longer than I thought it would, due mostly to my own ignorance. pfSense itself is incredibly easy to install and configure–much easier than IPCop–and the pfSense forums are full of helpful information and frequented by quite helpful people. In particular, I found the document about multi-wan routing to be very well done. It took me a while to understand all of what was going on there, but in the end my configuration is very similar to the configuration in that document.

A few things to note if you’re considering using pfSense for multi-WAN routing:

• pfSense needs static IP addresses if you want to do load balancing between the connections. That means you have to either get static IP addresses from your ISPs, or you have to put a router between your cable or DSL connection and the pfSense box. We elected to get static IP addresses rather than have the added complexity of three routers that serve only to turn the dynamic IP address from the ISP into a static IP address for pfSense.
• pfSense does round robin load balancing, meaning that the first request goes to WAN1, next to WAN2, etc. This is sub-optimum. A true load balancer would examine the traffic on each connection and assign requests in an attempt to balance the traffic as much as possible. For our purposes (the Web crawler), the round robin approach should work fine, but in other applications you might find that the asymmetric load balancing leaves a lot of bandwidth unused.
• Some Web sites (my email, for example, and many banking sites) expect all requests for a session to come from the same IP address. The “sticky connections” feature of pfSense will help you keep all the traffic for a particular session on the same IP address, but you might have to create special firewall rules for some sites. Creating those rules is very easy.
• You don’t need a really high-powered box to run pfSense. I’m running it on a generic box with an AMD Geode processor running at 1 GHz with 1 GB of RAM and a 100 GB hard drive. The box is not at all stressed running pfSense.

I’ve only had pfSense running in this configuration for one day, so I can’t say for sure how stable it is or how well it will work for us. IPCop stayed up for almost 80 days before we had to pull it down for the move. I can only hope that pfSense will work as well. So far, I’m quite happy with it.