Linux Basics: Create Network Bonding On CentOS 7/6.5
What is Network bonding?
Network bonding
is a method of combining (joining) two or more network interfaces
together into a single interface. It will increase the network
throughput, bandwidth and will give redundancy. If one interface is down
or unplugged, the other one will keep the network traffic up and alive.
Network bonding can be used in situations wherever you need redundancy,
fault tolerance or load balancing networks.
Linux allows us to bond multiple network interfaces into single interface using a special kernel module named bonding. The Linux bonding driver provides a method for combining multiple network interfaces into a single logical “bonded”
interface. The behaviour of the bonded interfaces depends upon the
mode; generally speaking, modes provide either hot standby or load
balancing services. Additionally, link integrity monitoring, may be
performed.
Types of Network Bonding
According the to the official documentation, here is the types of network bonding modes.
mode=0 (balance-rr)
Round-robin policy:
It the default mode. It transmits packets in sequential order from the
first available slave through the last. This mode provides load
balancing and fault tolerance.
mode=1 (active-backup)
Active-backup policy:
In this mode, only one slave in the bond is active. The other one will
become active, only when the active slave fails. The bond’s MAC address
is externally visible on only one port (network adapter) to avoid
confusing the switch. This mode provides fault tolerance.
mode=2 (balance-xor)
XOR policy:
Transmit based on [(source MAC address XOR’d with destination MAC
address) modulo slave count]. This selects the same slave for each
destination MAC address. This mode provides load balancing and fault
tolerance.
mode=3 (broadcast)
Broadcast policy: transmits everything on all slave interfaces. This mode provides fault tolerance.
mode=4 (802.3ad)
IEEE
802.3ad Dynamic link aggregation. Creates aggregation groups that share
the same speed and duplex settings. Utilizes all slaves in the active
aggregator according to the 802.3ad specification.
Prerequisites:
- Ethtool support in the base drivers for retrieving the speed and duplex of each slave.
– A switch that supports IEEE 802.3ad Dynamic link aggregation. Most
switches will require some type of configuration to enable 802.3ad mode.
mode=5 (balance-tlb)
Adaptive transmit load balancing:
channel bonding that does not require any special switch support. The
outgoing traffic is distributed according to the current load (computed
relative to the speed) on each slave. Incoming traffic is received by
the current slave. If the receiving slave fails, another slave takes
over the MAC address of the failed receiving slave.
Prerequisite:
- Ethtool support in the base drivers for retrieving the speed of each slave.
mode=6 (balance-alb)
Adaptive load balancing:
includes balance-tlb plus receive load balancing (rlb) for IPV4
traffic, and does not require any special switch support. The receive
load balancing is achieved by ARP negotiation. The bonding driver
intercepts the ARP Replies sent by the local system on their way out and
overwrites the source hardware address with the unique hardware address
of one of the slaves in the bond such that different peers use
different hardware addresses for the server.
In
this handy tutorial let us see how to setup network bonding on CentOS 7
and CentOS 6.5. Though it was tested on CentOS, it should work on RHEL
and Scientific Linux 6.x versions.
First, we will setup network bonding on CentOS 7.
1. Setting up Network Bonding on CentOS 7
I have three network interfaces in my CentOS 7 system, namely:
- enp0s3;
- enp0s8;
- enp0s9.
Let us combine two NICs (
enp0s3, and
enp0s8) and make them into one NIC named
bond0.
Configure Bond0 Interface
In CentOS 7, the bonding module is not loaded by default. Enter the following command as root user to enable it.
modprobe --first-time bonding
You can view the bonding module information using command:
modinfo bonding
Sample output:
filename: /lib/modules/3.10.0-123.el7.x86_64/kernel/drivers/net/bonding/bonding.ko
alias: rtnl-link-bond
author: Thomas Davis, tadavis@lbl.gov and many others
description: Ethernet Channel Bonding Driver, v3.7.1
version: 3.7.1
license: GPL
srcversion: E52AE00A79EA6FEFB5BF718
depends:
intree: Y
vermagic: 3.10.0-123.el7.x86_64 SMP mod_unload modversions
signer: CentOS Linux kernel signing key
sig_key: BC:83:D0:FE:70:C6:2F:AB:1C:58:B4:EB:AA:95:E3:93:61:28:FC:F4
sig_hashalgo: sha256
parm: max_bonds:Max number of bonded devices (int)
parm: tx_queues:Max number of transmit queues (default = 16) (int)
parm: num_grat_arp:Number of peer notifications to send on failover event (alias of num_unsol_na) (int)
parm: num_unsol_na:Number of peer notifications to send on failover event (alias of num_grat_arp) (int)
parm: miimon:Link check interval in milliseconds (int)
parm: updelay:Delay before considering link up, in milliseconds (int)
parm: downdelay:Delay before considering link down, in milliseconds (int)
parm: use_carrier:Use netif_carrier_ok (vs MII ioctls) in miimon; 0 for off, 1 for on (default) (int)
parm: mode:Mode of operation; 0 for balance-rr, 1 for active-backup, 2 for balance-xor, 3 for broadcast, 4 for 802.3ad, 5 for balance-tlb, 6 for balance-alb (charp)
parm: primary:Primary network device to use (charp)
parm: primary_reselect:Reselect primary slave once it comes up; 0 for always (default), 1 for only if speed of primary is better, 2 for only on active slave failure (charp)
parm: lacp_rate:LACPDU tx rate to request from 802.3ad partner; 0 for slow, 1 for fast (charp)
parm: ad_select:803.ad aggregation selection logic; 0 for stable (default), 1 for bandwidth, 2 for count (charp)
parm: min_links:Minimum number of available links before turning on carrier (int)
parm: xmit_hash_policy:balance-xor and 802.3ad hashing method; 0 for layer 2 (default), 1 for layer 3+4, 2 for layer 2+3 (charp)
parm: arp_interval:arp interval in milliseconds (int)
parm: arp_ip_target:arp targets in n.n.n.n form (array of charp)
parm: arp_validate:validate src/dst of ARP probes; 0 for none (default), 1 for active, 2 for backup, 3 for all (charp)
parm: fail_over_mac:For active-backup, do not set all slaves to the same MAC; 0 for none (default), 1 for active, 2 for follow (charp)
parm: all_slaves_active:Keep all frames received on an interfaceby setting active flag for all slaves; 0 for never (default), 1 for always. (int)
parm: resend_igmp:Number of IGMP membership reports to send on link failure (int)
Let us create a bond0 configuration file as shown below.
Go
to the directory where CentOS stores the network configuration files.
By default RHEL and its clones such as CentOS, Scientific Linux stores
the network configuration files under /etc/sysconfig/network-scripts/ directory.
Log in as root user.
Create bond0 configuration file:
vi /etc/sysconfig/network-scripts/ifcfg-bond0
Add the following lines.
DEVICE=bond0
NAME=bond0
TYPE=Bond
BONDING_MASTER=yes
IPADDR=192.168.1.150
PREFIX=24
ONBOOT=yes
BOOTPROTO=none
BONDING_OPTS="mode=1 miimon=100"
Note: Here, BONDING_OPTS describes the bonding mode. In our case, we will be configuring mode1(active-backup). Save and close file. 192.168.1.150 is bond0 IP address.
Configure Network interfaces
Now, we should modify both(enp0s3 & enp0s8) configuration files as shown below. First, let us start from enp0s3.
Edit file /etc/sysconfig/network-scripts/ifcfg-enp0s3,
vi /etc/sysconfig/network-scripts/ifcfg-enp0s3
Modify the file as shown below.
HWADDR="08:00:27:04:03:86"
TYPE="Ethernet"
BOOTPROTO="none"
DEFROUTE="yes"
PEERDNS="yes"
PEERROUTES="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="yes"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_PEERDNS="yes"
IPV6_PEERROUTES="yes"
IPV6_FAILURE_FATAL="no"
NAME="enp0s3"
UUID="a97b23f2-fa87-49de-ac9b-39661ba9c20f"
ONBOOT="yes"
MASTER=bond0
SLAVE=yes
Then, Edit file /etc/sysconfig/network-scripts/ifcfg-enp0s8,
vi /etc/sysconfig/network-scripts/ifcfg-enp0s8
Modify the file as shown below.
HWADDR=08:00:27:E7:ED:8E
TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=enp0s8
UUID=e2352c46-e1f9-41d2-98f5-af24b127b3e7
ONBOOT=yes
MASTER=bond0
SLAVE=yes
Save and close the files.
Now, activate the Network interfaces.
ifup ifcfg-enp0s3
ifup ifcfg-enp0s8
Now, enter the following command to make Network Manager aware the changes.
nmcli con reload
Restart network service to take effect the changes.
systemctl restart network
Test Network Bonding
Now enter the following command to check whether the bonding interface bond0 is up and running:
cat /proc/net/bonding/bond0
Sample output:
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: enp0s8
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 08:00:27:e7:ed:8e
Slave queue ID: 0
Slave Interface: enp0s3
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 08:00:27:04:03:86
Slave queue ID: 0
As you see in the above output, the bond0 interface is up and running and it is configured as active-backup(mode1) mode. In this mode, only one slave in the bond is active. The other one will become active, only when the active slave fails.
To view the list of network interfaces and their IP address, enter the following command:
ip addr
Sample output:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP qlen 1000
link/ether 08:00:27:e7:ed:8e brd ff:ff:ff:ff:ff:ff
3: enp0s8: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP qlen 1000
link/ether 08:00:27:e7:ed:8e brd ff:ff:ff:ff:ff:ff
4: enp0s9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:9a:aa:4a brd ff:ff:ff:ff:ff:ff
5: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 08:00:27:e7:ed:8e brd ff:ff:ff:ff:ff:ff
inet 192.168.1.150/24 brd 192.168.1.255 scope global bond0
valid_lft forever preferred_lft forever
inet6 fe80::a00:27ff:fee7:ed8e/64 scope link tentative dadfailed
valid_lft forever preferred_lft forever
That’s it.
Configure multiple IP addresses for bond0
I want to assign multiple IP addresses to bond0 interface. What should i do? Very simple, just create an alias for the bond0 interface and assign multiple IP addresses.
Let me make it more clear. Say for example we want to assign IP address 192.168.1.151 to bond0. To create an alias for bond0, copy the existing configuration file(ifcfg-bond0) to a new configuration file(ifcfg-bond0:1).
cp /etc/sysconfig/network-scripts/ifcfg-bond0 /etc/sysconfig/network-scripts/ifcfg-bond0:1
Then edit the alias file /etc/sysconfig/network-scripts/ifcfg-bond0:1,
vi /etc/sysconfig/network-scripts/ifcfg-bond0:1
Modify the device name and IP address as shown below.
DEVICE=bond0:1
NAME=bond0
TYPE=Bond
BONDING_MASTER=yes
IPADDR=192.168.1.151
PREFIX=24
ONBOOT=yes
BOOTPROTO=none
BONDING_OPTS="mode=1 miimon=100"
Here,
- bond0:1 – Device name
- 192.168.1.151 – IP address of bond0:1
Save and close the file. Restart network service to take effect the saved changes.
systemctl restart network
Now list out the network interfaces and their IP address using the command:
ip addr
Sample output:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP qlen 1000
link/ether 08:00:27:e7:ed:8e brd ff:ff:ff:ff:ff:ff
3: enp0s8: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP qlen 1000
link/ether 08:00:27:e7:ed:8e brd ff:ff:ff:ff:ff:ff
4: enp0s9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 08:00:27:9a:aa:4a brd ff:ff:ff:ff:ff:ff
5: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 08:00:27:e7:ed:8e brd ff:ff:ff:ff:ff:ff
inet 192.168.1.150/24 brd 192.168.1.255 scope global bond0
valid_lft forever preferred_lft forever
inet 192.168.1.151/24 brd 192.168.1.255 scope global secondary bond0:1
valid_lft forever preferred_lft forever
inet6 fe80::a00:27ff:fee7:ed8e/64 scope link tentative dadfailed
valid_lft forever preferred_lft forever
As you above the alias bond0:1 has been created and it’s up now.
2. Setting up Network Bonding on CentOS 6.5
We have seen Network bonding on CentOS 7. Now, we will see how to do it on CentOS 6.x.
I have three network interfaces, namely eth0, eth1 and eth2 in my CentOS 6.5 system. Let us combine two NICs (eth1 and eth2) and make them into one NIC named bond0.
Configure Bond0 Interface
First, let us create a bond0 configuration file as shown below.
Go
to the directory where CentOS stores the network configuration files.
By default RHEL and its clones such as CentOS, Scientific Linux stores
the network configuration files under /etc/sysconfig/network-scripts/ directory.
Run the following commands as root user.
Create bond0 configuration file under the above mentioned directory.
vi /etc/sysconfig/network-scripts/ifcfg-bond0
Add the following lines.
DEVICE=bond0
BOOTPROTO=none
ONBOOT=yes
IPADDR=192.168.1.200
NETWORK=192.168.1.0
NETMASK=255.255.255.0
USERCTL=no
BONDING_OPTS="mode=1 miimon=100"
Note: Here BONDING_OPTS describes the bonding mode. In our case, we will be configuring mode1(active-backup). Save and close file. 192.168.1.200 is bond0 IP address.
Next we have to load up the bond0 interface into the kernel. To do that, create a new file /etc/modprobe.d/bonding.conf,
vi /etc/modprobe.d/bonding.conf
Add the following line in it.
alias bond0 bonding
Save and close the file.
Configure Network interfaces
Now we should modify both(eth1 & eth2) configuration files as shown below. First, let us start from eth1.
Edit file /etc/sysconfig/network-scripts/ifcfg-eth1,
vi /etc/sysconfig/network-scripts/ifcfg-eth1
Modify the file as shown below.
DEVICE=eth1
MASTER=bond0
SLAVE=yes
USERCTL=no
ONBOOT=yes
BOOTPROTO=none
Then Edit file /etc/sysconfig/network-scripts/ifcfg-eth2,
# vi /etc/sysconfig/network-scripts/ifcfg-eth2
Modify the file as shown below.
DEVICE=eth2
MASTER=bond0
SLAVE=yes
USERCTL=no
ONBOOT=yes
BOOTPROTO=none
Save and close the files.
Enter the following command to load the bonding module.
modprobe bonding
Restart network service to take effect the changes.
service network restart
Test Network Bonding
Now enter the following command to check whether the bonding interface bond0 is up and running:
cat /proc/net/bonding/bond0
Sample output:
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth1
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 08:00:27:fe:6f:bf
Slave queue ID: 0
Slave Interface: eth2
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 08:00:27:34:17:c0
Slave queue ID: 0
As you see in the above output, the bond0 interface is up and running and it is configured as active-backup(mode1) mode. In this mode, only one slave in the bond is active. The other one will become active, only when the active slave fails.
To view the list of network interfaces and their IP address, enter the following command:
# ifconfig
Sample output:
bond0 Link encap:Ethernet HWaddr 08:00:27:FE:6F:BF
inet addr:192.168.1.200 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fefe:6fbf/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:379 errors:0 dropped:0 overruns:0 frame:0
TX packets:167 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:32354 (31.5 KiB) TX bytes:24078 (23.5 KiB)
eth0 Link encap:Ethernet HWaddr 08:00:27:BE:25:49
inet addr:192.168.1.101 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:febe:2549/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1402 errors:0 dropped:0 overruns:0 frame:0
TX packets:904 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:134823 (131.6 KiB) TX bytes:124938 (122.0 KiB)
eth1 Link encap:Ethernet HWaddr 08:00:27:FE:6F:BF
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:285 errors:0 dropped:0 overruns:0 frame:0
TX packets:156 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:24746 (24.1 KiB) TX bytes:22956 (22.4 KiB)
eth2 Link encap:Ethernet HWaddr 08:00:27:FE:6F:BF
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:95 errors:0 dropped:0 overruns:0 frame:0
TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7674 (7.4 KiB) TX bytes:1364 (1.3 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
As per the above output, bond0 is configured as master; eth1 and eth2 are configured as a slave.
Configure multiple IP addresses for bond0
I want to assign multiple IP addresses to bond0 interface. What should i do? Very simple, just create an alias for the bond0 interface and assign multiple IP addresses.
Let me make it more clear. Say for example we want to assign IP address 192.168.1.201 to bond0. To create an alias for bond0, copy the existing configuration file(ifcfg-bond0) to a new configuration file(ifcfg-bond0:1).
cp /etc/sysconfig/network-scripts/ifcfg-bond0 /etc/sysconfig/network-scripts/ifcfg-bond0:1
Then edit the alias file /etc/sysconfig/network-scripts/ifcfg-bond0:1,
vi /etc/sysconfig/network-scripts/ifcfg-bond0:1
Modify the device name and IP address as shown below.
DEVICE=bond0:1
BOOTPROTO=none
ONBOOT=yes
IPADDR=192.168.1.201
NETWORK=192.168.1.0
NETMASK=255.255.255.0
USERCTL=no
BONDING_OPTS="mode=1 miimon=100"
Save and close the file. Restart network service to take effect the saved changes.
service network restart
Now list out the network interfaces and their IP address using the command:
ifconfig
Sample output:
bond0 Link encap:Ethernet HWaddr 08:00:27:FE:6F:BF
inet addr:192.168.1.200 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fefe:6fbf/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:1048 errors:0 dropped:0 overruns:0 frame:0
TX packets:590 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:88622 (86.5 KiB) TX bytes:84340 (82.3 KiB)
bond0:1 Link encap:Ethernet HWaddr 08:00:27:FE:6F:BF
inet addr:192.168.1.201 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
eth0 Link encap:Ethernet HWaddr 08:00:27:BE:25:49
inet addr:192.168.1.101 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:febe:2549/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1422 errors:0 dropped:0 overruns:0 frame:0
TX packets:916 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:136317 (133.1 KiB) TX bytes:126150 (123.1 KiB)
eth1 Link encap:Ethernet HWaddr 08:00:27:FE:6F:BF
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:942 errors:0 dropped:0 overruns:0 frame:0
TX packets:581 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:80036 (78.1 KiB) TX bytes:84266 (82.2 KiB)
eth2 Link encap:Ethernet HWaddr 08:00:27:FE:6F:BF
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:111 errors:0 dropped:0 overruns:0 frame:0
TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:8916 (8.7 KiB) TX bytes:1492 (1.4 KiB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)
You should see the alias bond0:1 has been created and up.
Source & Reference Links:
