Need help with Dual NICs

I have a need to run 2 NICs on a web-server and I’m running into problems. Be advised that I am VERY new to linux.

Scenario:

  • Web-server running a podcast/audio server. (LAMP)
  • 2 Subnets (192.168.2.0/24 & 192.168.168.0/24)
  • 2 Internet connections, one on each subnet with a SonicWall Firewall (TZ-170) on each.
  • TZ-170 for eth1 has a One-To-One NAT from Public IP address to the Web-server. NOTE: This works fine for an other piece of equipment that I have.
  • eth0 is set to Internal Zone (no filtering in Firewall)
  • eth1 is set to Demilitarized Zone (http,https services allowed)

Requirements:
eth0 - on 168.x network for primary in-house admin.
eth1 - on 2.x network primarily serving the website to the world, but also uploading files via the website http interface.

With both networks attached I can reach the website from both 192.168.x. networks. But not from either using Public IP. With eth0 wire disconnected I can’t get out to the web from the server. From looking through the web I have done the following tests (still with eth0 cable disconnetcted:

suse2:/home/jkofsky # tail -f /var/log/messages
Mar 18 13:02:00 suse2 kernel: [79621.136199] ll header: ff:ff:ff:ff:ff:ff:00:13:ca:a0:1a:a2:08:06
Mar 18 13:05:57 suse2 smbd[4122]: [2011/03/18 13:05:57.411918,  0] smbd/server.c:281(remove_child_pid)
Mar 18 13:05:57 suse2 smbd[4122]:   Could not find child 27034 -- ignoring
Mar 18 13:11:13 suse2 kernel: [80173.512771] martian source 192.168.2.240 from 221.1.222.162, on dev eth1
Mar 18 13:11:13 suse2 kernel: [80173.512780] ll header: 00:14:6c:2e:6b:54:00:06:b1:24:94:20:08:00
Mar 18 13:18:57 suse2 smbd[4122]: [2011/03/18 13:18:57.953535,  0] smbd/server.c:281(remove_child_pid)
Mar 18 13:18:57 suse2 smbd[4122]:   Could not find child 27055 -- ignoring
Mar 18 13:21:21 suse2 kernel: [80781.849304] martian source 192.168.2.240 from 70.167.228.41, on dev eth1
Mar 18 13:21:21 suse2 kernel: [80781.849312] ll header: 00:14:6c:2e:6b:54:00:06:b1:24:94:20:08:00
Mar 18 13:21:24 suse2 kernel: [80784.693742] martian source 192.168.2.240 from 70.167.228.41, on dev eth1
Mar 18 13:21:24 suse2 kernel: [80784.693751] ll header: 00:14:6c:2e:6b:54:00:06:b1:24:94:20:08:00
Mar 18 13:21:25 suse2 kernel: [80785.470654] martian source 192.168.2.240 from 221.1.222.162, on dev eth1
Mar 18 13:21:25 suse2 kernel: [80785.470663] ll header: 00:14:6c:2e:6b:54:00:06:b1:24:94:20:08:00
Mar 18 13:21:27 suse2 kernel: [80787.308942] martian source 192.168.2.240 from 70.167.228.41, on dev eth1
Mar 18 13:21:27 suse2 kernel: [80787.308951] ll header: 00:14:6c:2e:6b:54:00:06:b1:24:94:20:08:00
^C
suse2:/home/jkofsky # ifconfig
eth0      Link encap:Ethernet  HWaddr 00:08:02:39:54:79  
          inet addr:192.168.168.135  Bcast:192.168.168.255  Mask:255.255.255.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:208965 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13246 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:26160093 (24.9 Mb)  TX bytes:1530467 (1.4 Mb)

eth1      Link encap:Ethernet  HWaddr 00:14:6C:2E:6B:54  
          inet addr:192.168.2.240  Bcast:192.168.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:84115 errors:0 dropped:0 overruns:0 frame:0
          TX packets:677 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:24185073 (23.0 Mb)  TX bytes:192016 (187.5 Kb)
          Interrupt:16 Base address:0x4000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1245 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1245 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:157380 (153.6 Kb)  TX bytes:157380 (153.6 Kb)

suse2:/home/jkofsky # route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.168.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth0
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
0.0.0.0         192.168.168.254 0.0.0.0         UG    0      0        0 eth0
suse2:/home/jkofsky # ping 192.168.2.231 -c 2
PING 192.168.2.231 (192.168.2.231) 56(84) bytes of data.
64 bytes from 192.168.2.231: icmp_seq=1 ttl=64 time=2.05 ms
64 bytes from 192.168.2.231: icmp_seq=2 ttl=64 time=0.403 ms

--- 192.168.2.231 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.403/1.228/2.054/0.826 ms
suse2:/home/jkofsky # ping Suse2 -c 2
ping: unknown host Suse2
suse2:/home/jkofsky # arp -a
? (192.168.2.231) at 00:13:ca:a0:09:6d [ether] on eth1
? (192.168.168.254) at <incomplete> on eth0
suse2:/home/jkofsky

I uses Yast to set everything. I just believe that it is something simple that I have no idea to check :’( Thanks for any help you can provide.

p.s. eht0 & eth1 are set for DHCP addressing and both SonicWall’s DHCP Servers is set to assign a “static” IP. (192.168.x.240)

I believe your problem is both of your networks are using the same subnet mask.
Try changing the internal network subnet mask.
When you have 2 nics on the same subnet 1 will be prefered.

I would say you have a routing problem. From your description I would assume that eth1 needs a route to the gateway of the 192.168.2.0 network.
It appeaers that eth1 is using the gateway of the 192.168.168.0 network (that’s why you can hit the website internally).

Good luck,
Hiatt

I agree that I probably need to get gateways set for the different subnets.

When I was saying that I could hit the website from the two subnets, I had one computer on the 168.x network and one on the 2.x network. The subnets are completely seperate.

In the above test:


suse2:/home/jkofsky # tail -f /var/log/messages
Mar 18 13:02:00 suse2 kernel: [79621.136199] ll header: ff:ff:ff:ff:ff:ff:00:13:ca:a0:1a:a2:08:06
Mar 18 13:05:57 suse2 smbd[4122]: [2011/03/18 13:05:57.411918,  0] smbd/server.c:281(remove_child_pid)
Mar 18 13:05:57 suse2 smbd[4122]:   Could not find child 27034 -- ignoring
Mar 18 13:11:13 suse2 kernel: [80173.512771] martian source 192.168.2.240 from 221.1.222.162, on dev eth1
Mar 18 13:11:13 suse2 kernel: [80173.512780] ll header: 00:14:6c:2e:6b:54:00:06:b1:24:94:20:08:00
Mar 18 13:18:57 suse2 smbd[4122]: [2011/03/18 13:18:57.953535,  0] smbd/server.c:281(remove_child_pid)
Mar 18 13:18:57 suse2 smbd[4122]:   Could not find child 27055 -- ignoring
Mar 18 13:21:21 suse2 kernel: [80781.849304] martian source 192.168.2.240 from 70.167.228.41, on dev eth1
Mar 18 13:21:21 suse2 kernel: [80781.849312] ll header: 00:14:6c:2e:6b:54:00:06:b1:24:94:20:08:00
Mar 18 13:21:24 suse2 kernel: [80784.693742] martian source 192.168.2.240 from 70.167.228.41, on dev eth1
Mar 18 13:21:24 suse2 kernel: [80784.693751] ll header: 00:14:6c:2e:6b:54:00:06:b1:24:94:20:08:00
Mar 18 13:21:25 suse2 kernel: [80785.470654] martian source 192.168.2.240 from 221.1.222.162, on dev eth1
Mar 18 13:21:25 suse2 kernel: [80785.470663] ll header: 00:14:6c:2e:6b:54:00:06:b1:24:94:20:08:00
Mar 18 13:21:27 suse2 kernel: [80787.308942] martian source 192.168.2.240 from 70.167.228.41, on dev eth1
Mar 18 13:21:27 suse2 kernel: [80787.308951] ll header: 00:14:6c:2e:6b:54:00:06:b1:24:94:20:08:00

the martians happen when I try to browse to the Public IP from the 168.x network

A martian source is an IP address that is impossible. (See RFC1812.).

A common cause is multiple subnets on the same LAN. You should isolate the subnets using VLANs.

See Martian sources errors showing in messages log

Thanks, but the link sounded like the problem is normally associated with two NICs on the same subnet. Not my case, The two subnets for the most part physically separate as well as subnet seperate.

It appears to work by setting the gw as indicated below

suse2:/home/jkofsky # route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.168.0   192.168.168.254         255.255.255.0   U     0      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth0
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
0.0.0.0         192.168.2.254 0.0.0.0         UG    0      0        0 eth0

I can get to the web server to reply via the hostname on 168.x subnet, via Private IP on both subnets, and the PublicIP everywhere.
Thanks all.

I’ve setup similar (a multi-homed server) in which service(s) can be bound to both/more than one interface.

Ken is right that your problem is routing, and you need to do more than simply configuring both a GW the normal way and another GW for “everything”

Configured the way you describe I can still see packet losses due to the bi-directional nature of TCP/IP and your haphazard connecting to different GWs.

IMO here are some guidelines to what you’re configuring…

  • If you configure multiple GWs on a multi-homed box, you should at least set different routing priorities and make them very, very different so that only one route is used unless that one is clearly unresponsive.
  • Routes need to be precise. Any traffic that returns from a different network interface will appear to be from a different machine (although only a different interface on the same machine) so will be discarded requiring a timeout and re-request.
  • Configuring different subnets for each interface within the same Class is dangerous, if any device is configured with anything other than the proper subnet mask it’s easy to become mis-configured. It’s far safer to configure different Class addresses on each interface to be <clearly> different.
  • Don’t overlook the use of name resolution (typically DNS) to direct a client to the proper interface. So, for instance internally your DNS can point to the internal network IP whereas external DNS would point to the public IP would be then translated to the external interface’s IP. This configuration is called a “Split DNS.”

The bottom line is that you need to be <very certain> only one route to only one chosen interface is consistent for each client.

HTH,
Tony

Tony, Thanks for your reply. I am a REAL newbie in linux, could you show me a routing table to show me what it is your a talking about. My background is programming not network admin, so I need all the help I can get :smiley:

Hmmm…
A routing table on any machine is only a tiny part of what you’re dealing with.

Recommend doing a search on “split DNS” – It might turn on a light bulb in your mind.
A basic principle to keep in mind is KISS, try to make sure there is only one route to any resource, multiple routes are a bit more advanced and involve more configuration.
Configure only one GW on a machine unless you know what you’re doing.
Configure only one path to the Internet for each box unless you know what you’re doing.
If you don’t know how to configure a Split DNS, then the next best thing is to configure your services to be accessible <only> on the Public interface and send all traffic through your Internet Gateway even if it’s going to be routed back to your Network. Consistent with this idea is to physically separate your Servers which are running publicly accessible Services, don’t even have a physical link from your internal Hosts directly to these Servers.

This isn’t really a Linux issue, it’s a network architecting (design) issue that would apply to any network regardless what OS are running.

Backing up a bit,
What are you trying to do with two Internet connections? Are you trying to implement some kind of failover? If redundant Internet connectivity isn’t one of your objectives, then I’d recommend putting your Internet Servers on one connection and your main network with Users connecting through the other… And no physical network link should connect any machine connected to one Internet GW and its Hosts with anything connected to the other Internet GW. That should probably simplify things greatly…

HTH,
Tony