So what is this Channel Bonding all about?
Imagine you have a Multi NIC server in your environment and once you are done with the setup of this server in the network you have realized that one of the NIC cards is sitting Idle and you are not aware how you can use it?
Or you have come across the situation where you are trying to reach the 100% uptime in you server environment. When you have tried all the fail over techniques for Load Balancer, Switches, Router and you still see some possibility of failure at the server end.
Or you are looking for an In-expensive way of improving your system UP time in the server environment.
Or when you don’t want to waste 2-3 Ip’s for Two NICs in trying to achieve the failover or high availability at the server end.
Well the answer to the above? Is Channel Bonding
Channel Bonding is an Inexpensive way to Utilize you existing Idle Network Interface Card(NIC) to reap the benefits of Failover, Increasing the Data transfer speed ,using both the NIC cards to receive the data.
You can get a better mileage with your existing NIC cards with less investment of time and mney.
So what do we need to achieve this?
- Linux OS that supports Bonding module (Check if the Linux Kernel Supports bonding driver).
- Linux OS containing mii-tool or ethtool which can be used to view/change/modify the speed of NIC card (Note: mii-tool supports configuration of 10/100 NIC while ethtool can support above 100baseT).
- Two NIC (network Interface card) Cards.
- A switch that supports IEEE 802.3ad Dynamic Link Aggregation.
Well the best part is your bonding options are not limited to two Interfaces it could be ‘N’ of interfaces, you can even bond Gigabit Ethernet Cards.
For now lets look at how bonding works for Stand alone servers. The Bonding configuration/details given below is tried and tested on Red Hat Enterprise Linux AS release 4.
Verification:
Let’s first verify if the OS has the bonding module
By looking at the Kernel Config file we can verify if the bonding is enabled as a module
# grep -i bonding /boot/config-2.6.9-34.EL
CONFIG_BONDING=m (Implies bonding is configured/enabled as a module)
Let’s verify if the module is present in the system (if kernel has loaded it properly)
#modprobe –list | grep -i bonding
/lib/modules/2.6.9-34.ELsmp/kernel/drivers/net/bonding/bonding.ko
Generally you can use mii-tool to check if the network interface s are up and their status.
# mii-tool
eth0: negotiated 100baseTx-FD, link ok
eth3:negotiated 100baseTx-FD, link ok
In the above case the two NIC card’s link status is okay.
Configure Interfaces:
In order to configure the bonding lets assume ” bond0″ is the new logical bonded Interface name. This would be configured the same way as interfaces eth0& eth3.
Edit/Create the “ifcfg-bond0” config file in the following path with the following values, Note that bond0 would be acting as a Master while the other Interfaces would be the slave for this bonding channel.
Configure ifcf-bond0 as Master Interface
# vi /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
BOOTPROTO=static
ONBOOT=yes
USERCTL=no
IPADDR=x.x.x.x
NETMASK=255.255.255.0
GATEWAY=x.x.x.x
Configure the Interfaces eth0 and eth3 as Slave
Configuration for eth0
# vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=none
ONBOOT=yes
USERCTL=no
MASTER=bond0
SLAVE=yes
ETHTOOL_OPTS=”autoneg on”
Configuration for eth3
# vi /etc/sysconfig/network-scripts/ifcfg-eth3
DEVICE=eth3
BOOTPROTO=none
ONBOOT=yes
USERCTL=no
MASTER=bond0
SLAVE=yes
ETHTOOL_OPTS=”autoneg on”
Let’s try to understand some of the variables defined above in the Interface file.
BOOTPROTO=<protocol>, where <protocol> is one of the following:
none — No boot-time protocol should be used.
bootp — The BOOTP protocol should be used.
dhcp — The DHCP protocol should be used.
ETHTOOL_OPTS=<options>, where <options> are any device-specific options supported by ethtool. For example, if you wanted to force 100Mb, full duplex:
ETHTOOL_OPTS=”autoneg off speed 100 duplex full”
Note that changing speed or duplex settings almost always requires
disabling autonegotiation with the autoneg off option. This needs to be stated first, as the option entries are order dependent.
ONBOOT=<answer>, where <answer> is one of the following:
yes — This device should be activated at boot-time.
no — This device should not be activated at boot-time.
SLAVE=<bond-interface>,where <bond-interface> is one of the following:
yes — This device is controlled by the channel bonding interface specified in the MASTER directive.
no — This device is not controlled by the channel bonding interface specified in the MASTER directive.
This directive is used in conjunction with the MASTER directive.
USERCTL=<answer>, where <answer> is one of the following:
yes — Non-root users are allowed to control this device.
no — Non-root users are not allowed to control this device.
Refer
http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/ref-guide/s1-networkscripts-interfaces.html
for more details on the various variables that can be used to configure Ethernet Interfaces in Redhat.
Configure Kernel Module Configuration file:
Once the Interfaces ifcfg-bond0, ifcfg-eth0, ifcfg-eth3 are configured, configure the kernel modules configuration file to load the required bonding module while booting.
Options for the bonding driver are supplied as parameters to the bonding module at load time. Module options may be given as command line arguments to the insmod or modprobe command, but are usually specified in either the /etc/modules.conf or /etc/modprobe.conf configuration file.
# vi /etc/modprobe.conf
alias eth0 e1000
alias eth3 e1000
alias bond0 bonding
options bonding miimon=100 mode=1
The parameters miimon=100 and mode=1 are passed to the bonding driver.
miimom specifies the MII link monitoring frequency in milliseconds. This determines how often the link state of each slave is checked for link failures. A value of zero disables MII link monitoring.
mode =1,a value of “0” specifies that the bonding mode is a round robin mode providing load balancing and fault tolerance and a value of “1” specifies that only one slave in a bond is active and the interface is in a fault tolerance mode.
Once the Configuration of all the 3 interfaces ifcfg-bond0, ifcfg-eth0, ifcfg-eth3 is done restart the network services on the Linux box.
# service network restart
Now run the ifconfig -a command on the Shell to check if bonding is done and how the Interfaces are displayed. The output would be similar to the one shown below.
# ifconfig –a
bond0 Link encap:Ethernet HWaddr 00:14:4F:20:91:78
inet addr:82.132.154.49 Bcast:82.132.154.255 Mask:255.255.255.0
inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
eth0 Link encap:Ethernet HWaddr 00:14:4F:20:91:78
inet6 addr: fe80::214:4fff:fe20:9178/64 Scope:Link
UP BROADCAST NOARP SLAVE MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Base address:0x9c00 Memory:fbfe0000-fc000000
eth1 Link encap:Ethernet HWaddr 00:14:4F:20:91:79
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Base address:0x9800 Memory:fbfc0000-fbfe0000
eth3 Link encap:Ethernet HWaddr 00:14:4F:20:91:78
inet6 addr: fe80::214:4fff:fe20:9178/64 Scope:Link
UP BROADCAST NOARP SLAVE MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
Base address:0x9000 Memory:fbf80000-fbfa0000
If you observe the output above you can see that the interface “bond0″acts as a MASTER and Interfaces “eth0ð3″ act as SLAVE. Once the Channel bonding is setup run the following command to verify the Channel bonding created.
# ls -l /proc/net/bonding/
total 0
-r–r–r– 1 root root 0 Jul 21 15:02 bond0
For every channel bond created there would be one such file present in this directory.
# more /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v2.6.1 (October 29, 2004)
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth0
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:14:4f:20:91:78
Slave Interface: eth3
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:14:4f:20:91:7f
From the output of the “/proc/net/bonding/bond0″ you can see that configured bonding is fault -tolerance (active-backup) and other details.
Errors and Trouble Shooting:
Sometimes if bonding is not working after server reboot check for the following to identify as what is causing the Issue.
- Check if the /var/log/messages for error messages like
ifup: bonding device bond0 does not seem to be present, delaying initialization.
ifup: bonding device bond0 does not seem to be present, delaying initialization.
ifup: bonding device bond0 does not seem to be present, delaying initialization.
- Check if the bonding module is loaded after the system booted up
# lsmod
bonding 64180 0
md5 5697 1
- If you have manually added anything to modules.dep remove those entries and let depmod handle the proper linking of modules, the modules.dep file is there to ensure the modules are loaded, unloaded and linked in the proper order. Remove your manual entries and run your depmod -a
# depmod –a
- Now run modprobe for the required module (in this case its bond0)
# modprobe bond0
- Kernel modules are usually loaded directly by the facility that requires them, which is given correct settings in the /etc/modprobe.conf file. However, it is sometimes necessary to explicitly force the loading of a module at boot time. Red Hat Enterprise Linux checks for the existence of the /etc/rc.modules file at boot time, which contains various commands to load modules. The rc.modules should be used, and not rc.local because rc.modules is executed earlier in the boot process.
- For example, the following commands configure loading of the bond0 module at boot time (as root):
# echo modprobe bond0 >> /etc/rc.modules
# chmod +x /etc/rc.modules
A Linux admin's How to's 