redundant networking ideas, suggestions

Nicholas Leippe nick at leippe.com
Thu Sep 28 15:28:07 MDT 2006


We're improving our network infrastructure and would like to implement some 
redundancy. As I go through the options, there are myriads of possible 
configurations. I am considering several and hope some of you will share some 
insight on their pros and cons.

Equipment:
- 2 managed switches, which I'll call A and B
- each server has 2, dual-port nics which I'll call n0 and n1 with n0 having 
ports e0 and e1, and n1 having ports e2 and e3.

In all of the configurations below, the physical configuration on the servers 
will be:

e0 -> A
e1 -> B
e2 -> A
e3 -> B

thus any single switch, nic, or port failure can be endured, and some double 
faults can be endured.

As for the firewall, we have two cable drops from the same edge router into 
our rack. We are planning on setting up a mirrored system with a virtual ip 
on either side using ct_sync between them, and ha linux (heartbeat) to manage 
the pair. I would think that it would be easiest to manage the network if 
each server had a single ip addr all on one subnet. Are there some cases when 
multiple subnets becomes better/easier?

configuration 1
---------------
set up a single bond0 across all 4 ports, with a single ip addr.

Depending on how the bonding driver handles activity presented to inactive 
ports, this could have some problems akin to multiple bridges w/o STP.

configuration 2
---------------
set up a single bridge across all 4 ports, with a single ip addr.

configuration 3
---------------
set up bond0 across e0 and e2, bond1 across e1, e3, and then bridge0 across 
bond0 and bond1, with a single ip addr:

         /- bond0 (e0, e2) -> A
bridge0 -
         \- bond1 (e1, e3) -> B

in this case, every machine on the network becomes a bridge--will the STP 
traffic overhead be a problem?  How fast can STP resolve changes? What's the 
typical ttl for arp cache entries?

configuration 4
---------------
set up bond0 across e0 and e2, bond1 across e1, e3, and use two different ip 
addrs on different subnets for each, and set the routing metric to favor one:
(in this case the bond is across nics, but same physical network)

ip addr 1, subnet 1 - bond0 (e0, e2) -> A
ip addr 2, subnet 2 - bond1 (e1, e3) -> B

configuration 5
---------------
set up bond0 across e0 and e1, bond1 across e2, e3, and use two different ip 
addrs on different subnets for each, and set the routing metric to favor one:
(in this case the bond is across the same nic, but different physical network)

ip addr 1, subnet 1 - bond0 (e0 -> A, e1 -> B)
ip addr 2, subnet 2 - bond1 (e2 -> A, e3 -> B)

configuration 6
---------------
set up bond0 across e0 and e3, bond1 across e1, e2, and use two different ip 
addrs on different subnets for each, and set the routing metric to favor one:
(in this case the bond is both across nics and different physical network)

ip addr 1, subnet 1 - bond0 (e0 -> A, e3 -> B)
ip addr 2, subnet 2 - bond1 (e2 -> A, e1 -> B)

In (4), (5) and (6), the firewall would need to be on both subnets, and all of 
the forwarding/filtering rules would have to be modified/doubled accordingly.

(4) or (6) was mentioned last year on the list. Details of the routing 
configuration and the two firewalls were not given iirc.

Since the switch is managed we could physically segment the lan further with 
vlans.

As is probably obvious to any full-time network admins, I'm a bit rusty on the 
details to some of these issues.  Some of this is probably obvious to some of 
you out there.

Please share ;)




More information about the PLUG mailing list