Dual/multi ISP routing confusion

Hi folks. I’ve posted this problem on a (very much) older thread and it looks like it doesn’t get much attention anymore.

I’ve got 2 ISPs and one LAN. Each one has one NIC. Now, I did manage to get an acceptable load balancing using nexthop but some web applications we use don’t want to work with split traffic. So I’ve figured that I’ll have to leave the ISP’s mangling alone and route some hosts from the lan to one ISP and some to the other.

I just don’t seem to have any success…

route $lan_host_ip gw $isp2_gw

or

route $router_lan_alias_ip gw $isp2_gw

with appropriate route inside the lan host*
Still routes my host thru the default gateway.

I’m not even sure it’s possible, anymore…
Can I make one computer from the lan go thru the router towards one ISP and another computer towards the other ISP?
iptablles, ip, route?
Ideas and guidence will be greatly appreciated. I’m expecting a third ISP to arrive shortly…
Cheers,
Alex.

You can actually purchase a “Dual WAN router” to solve that problem.

I’m expecting a third ISP to arrive shortly…

Now you’ve gone and made it harder. I’ll note that Amazon seems to list Multi-WAN routers.

What makes this problem hard, is that you are probably using NAT. So the NAT translation has to keep straight which uplink to use for a particular connection.

What would that Dual WAN Router kernel have more than our openSuSE in it’s networking abilities?

But if you mentioned NAT, I will try, today, to forward some ports from the lan host to the 2nd wan… even if I’ll end up with a few megabytes of iptables rules… :smiley:

Well, it just might be that I have to write about this stuff on a forum to get my brain rolling :smiley: but i’m not there yet…

I split my LAN into two separate sub-nets on the same NIC.
192.168.1.0, the original, and made an alias on eth0 for 192.168.2.0 subnet.
After I’ve configured the firewall for that subnet, so it’d accept traffic thru the second subnet I created two routing rules with “ip” like:

 
ip route add default via $gw_romtelecom dev $it_romtelecom table romtelecom
ip rule add from 192.168.1.0/24 table romtelecom 
ip route add default via $gw_rds dev $if_rds table rds
ip rule add from 192.168.2.0/24 table rds

And It worked!
But just for one single traceroute… then I lost my ssh link and realized that the routing tables got screwed… the internet connection went down and had to reboot the server.
I was saying about that I have to write it somewhere to get my brainz straight? :smiley: Notice the “it_romtelecom” instead of the “if_romtelecom”. “if” from “interface”…
And when I executed the commands, I did not notice ip complaining…

Error: either "to" is duplicate, or "romtelecom" is a garbage.

*unintended pun

I so wonder why it worked for that split second I tracerouted google to see thru which gateway I’m routed by the server…

It could be the right track, I’m heading on. Didn’t want to have different subnets but if that’s what it takes to…

I’ve never experimented with IP masquerading in linux. Maybe the ability is there. But you would need the linux system to be directly connected to both ISP links, and handle the NAT itself to do that. You could not have the NAT handled on a separate router.

The dual WAN router is directly connected to both ISP links, and does the NAT itself. So when you make a connection from, say, 192.168.1.7 to an external site, the dual WAN router decides how to translate that to one of your public IPs, and then uses the ISP link appropriate for that public IP for all traffic for the particular connection.

Without something like that, you could have a problem of some packets being translated to one public address, and some translated to the other. And that can’t work.

I think I’ve cracked it!
It is working since I’ve removed the subnet mask from the ip rule. I don’t understand yet why as 192.168.2.0 has the same 255.255.255.0 mask as 192.168.1.0 (/24)
With the mask in place as 192.168.2.x/24, the x host can make just one ping and the connection dies.
So routing one host to one ISP is possible… thank God, Torvalds and Novell for supporting openSuSE rotfl!


ip route add default $gw2 dev $if2 table $table2
ip rule add from $host_ip table $table2

And left the default route alone, thru $gw1 on $if1, without creating a specific routing table with a specific rule.

I wonder if I could route that host within the same subnet. I bet that the pesky netmask was the culprit…

A topic often asked, and little changed…

As you’ve discovered, in general it’s very bad to configure 2 different pathways from your source to target over an IP netowrk… TCP/IP is a two-way communications protocol so the target will want to respond to the source and assumes that whatever routing it’s using on the return should work. But, if the routing information back is different then it will look different to the source and so will be rejected.

This not only happens with two different routes to the destination, it also happens if your machine has multiple actively connected NICs (any combination of wired and wireless).

The <recommended> solution which supports load balancing has been mentioned… You need a router that is configured to specially support load balancing… It maintains an internal routing table so that sessions internal machines are routed consistently through one or the other Provider. Because the sessions are consistently routed through the same provider, TCP/IP handshakes work.

The “poor man’s” solution to your setup is possible, too. It won’t support load balancing but it will provide fault tolerance (a backup route to the Internet if the primary connection fails), and it’s easy to configure… You simply configure your LAN to connect through a router (much more complicated if all aren’t configured through a common node) and configure extreme different routing priority values for each path to each ISP. The result is that all routing will be through only one primary ISP (thereby satisfying 2 way communication) and only if the connection fails then the other route is used (again, satisfying 2 way communication for all <new> sessions). Even with extremely different priority routing values you may still have a problem during extreme congestion or overload where the primary route is saturated and sessions start routing through the second ISP when the first ISP is still up… That is why this is a “poor man’s” solution that will still have issues.

HTH,
TSU

This is why you need iprout2 having Diff. route tables depending on the isp .
Iptables can leave a marker on connection tot switch the table.

Chat is big.
Except if seconde ISp line is dhcp how do i auto create that seconde route table
Seconde device is up and working but not mentioned in /etc/network/script/
Also how to add route IP and gateway in script. What script exactly suse 13.1 eth4?
As i use dhcp i do not know IP nor gateway yet.
Do know wanted route table… Feels like chicken and egg problem.
Want to have http… Services on second line eth4 Nat on eth0 other IP address
S

Knowing how is important to me.

Although I haven’t had to delve deeply into this personally (I find spending $100 for a special NIC card that does this is far easier and cheaper than my time),

If you want to do this using software, I’m pretty sure your answers are here
http://lartc.org/

Allows you to do all sorts of fancy and unusual things with networking.

Good Luck,
TSU

But, all-in-all, it really does work! :slight_smile:

I guess it had to be something like a visit to the shrink. You know… :smiley:

Here’s some of what I did:

/sbin/multirouter:


#!/bin/sh
#

conf_dir=/etc/multirouting #create that folder or somewhere else. /etc/sysconfig would do too. 
table1=ISP1   #See the /etc/iproute2/rt_tables and I think this could go away in a/the config file too. 
table2=ISP2

_log() {
echo -e "\033[31m$1\033[0m" # I like colors. 
}

echo -e "\033[32mStarting\040the\040IP\040routing\040configuration...\033[0m"  #Just playing with escapes. I need a some time off...

if [ -f $conf_dir/interfaces.conf ] # example in code pasted below. 
then source /etc/multirouting/interfaces.conf
else _log "$conf_dir/interfaces.conf missing. I quit."
fi 

if ip route show table $table2|grep default >/dev/null
then _log "routing table to $table2 already set" 
else ip route add default via $gw_ISP2 dev $if_ISP2 table ISP2 # Create the route if it doesn't exist. I made this, keeping in mind that I'll execute this script several times as I modify the client's IPs #
fi


for client in `cat $conf_dir/clients.conf`
        do if ip rule show |grep $client >/dev/null
        then _log "$client already withing routing table roule"
        else echo -ne "\03332m$client\040goes\040to\040$table2\040\0330m" 
        ip rule add from $client table $table2    ###  SO: let the IP of the machine X go thru the 2nd ISP gateway instead of default and thru 2nd ISP NIC. ###
        _log "$client went to $table2"
        fi      
done 
#### Complicating a bit the config files and how they are read, this whole thing can be scaled-up to unlimited? ISPs, LANs and clients. Anybody can be routed anywhere. 
echo "----------------------------------------------------------------"
echo -e "\03332m Routing table looks like:\0330m"
ip route show table ISP1
ip route show table ISP2
#/Alex

/etc/multirouting/interfaces.conf:

 
# Mind you that all these addresses are just examples not real IPs. 

#ISP1
ip_ISP1=1.2.3.4
net_ISP1=1.2.3.0/255.0.0.0 # this var is set just as a reminder for me and me alone.
gw_ISP1=1.2.3.1

#ISP2
ip_ISP2=20.30.40.50
net_ISP2=20.30.40.0/255.0.0.0 
gw_ISP2=20.30.40.1

#Phisical Network Interfaces
if_ISP2=eth2
if_ISP1=eth1
if_lan=eth0 # this var is set just as a reminder for me and me alone. 


The clients.conf is just a list of clients IPs with a return after each IP. (0x0a 0x0d :D)

Cheers!