27 Nov 2016

QEMU Bridge network study -- setup

文章欢迎转载,但转载时请保留本段文字,并置于文章的顶部
Author: derek0883@gmail.com
本文原文地址:http://www.gorecursion.com/virtualization/2016/11/27/net2.html

Bridge network is much complicated than user network mode. It including virtual switch(bridge), virtual NIC(TAP/TUN) in host OS, and virtual NIC in guest OS.

Topology and setup

The following figure from this slide QEMU bridge networking

The basic concept are: 1. Using linux kernel bridge module to create a virtual switch. 2. Add one or more physical NIC card to this switch. 3. Create one or more virtual NIC deivces(TAP/TUN device in kernel). 4. Add those virtual NIC deivces to the switch. 5. Connect those virtual NIC devices to virtual NIC device of QEMU guest OS.

In order to study internal implementation of bridge network. I setup a testing environment same as above figure.

# Create a bridge name is br0 and bring it up
$ sudo ip link add name br0 type bridge
$ sudo ip link set br0 up

# Add virtual NIC name is tap0 and tap1
$ sudo ip tuntap add name tap0 mode tap
$ sudo ip tuntap add name tap1 mode tap

# Add tap0 and tap1 to bridge br0
$ sudo ip link set tap0 master br0
$ sudo ip link set tap1 master br0

# Add physical NIC eth0 to bridge br0 
$ sudo ip addr flush dev eth0
$ sudo ip link set eth0 master br0

# set br0 address 
$ sudo ip addr add 10.0.0.254/24 dev br0

Now I have setup a switch named br0 on host OS, with 3 member ports, eth0 is physical port connect to a another PC. Two virtual NIC port tap0 and tap1, not connect yet.

$ brctl show
bridge name bridge id       STP enabled	interfaces
       br0  8000.0021ccd76ee0  no       eth0	
                                        tap0
                                        tap1

$ ifconfig br0
br0       Link encap:Ethernet  HWaddr 00:21:cc:d7:6e:e0  
          inet addr:10.0.0.254  Bcast:0.0.0.0  Mask:255.255.255.0

$ sudo ethtool eth0 | grep Link
    Link detected: Yes

$ sudo ethtool tap0 | grep Link
    Link detected: no

$ sudo ethtool tap1 | grep Link
    Link detected: no

Now I start two guest OS, one connect to tap0, and another one connect to tap1. In order to further study source code, I start 1st one with gdb.

$ sudo gdb x86_64-softmmu/qemu-system-x86_64 
(gdb) r images/cirros-x86_64-disk1.img               \
    -device e1000,netdev=net0,mac=52:54:00:12:34:56, \
    -netdev tap,id=net0,ifname=tap0

$ sudo x86_64-softmmu/qemu-system-x86_64 images/cirros-x86_64-disk2.img \
    -device e1000,netdev=net0,mac=52:54:00:12:34:57, \
    -netdev tap,id=net0,ifname=tap1 -nographic

With above command line QEMU will create virtual NIC device for guest OS, and it will connect to host interface tap0 and tap1.

sudo ethtool tap0 | grep Link
    Link detected: Yes

$ sudo ethtool tap1 | grep Link
    Link detected: Yes 

Now inside guest OS, I have:

$ ifconfig eth0
eth0       Link encap:Ethernet  HWaddr 52:54:00:12:34:56 
          inet addr:10.0.0.1  Bcast:0.0.0.0  Mask:255.255.255.0

$ ifconfig eth0
eth0       Link encap:Ethernet  HWaddr 52:54:00:12:34:56 
          inet addr:10.0.0.2  Bcast:0.0.0.0  Mask:255.255.255.0

In each Guest OS, I can ping each other, can ping host OS, can ping physical PC which connected to physical port eth0.

# from guest os ping physical PC conntect to physical port eth0
$ ping 10.0.0.5
PING 10.0.0.5 (10.10.0.1): 56 data bytes
64 bytes from 10.0.0.5: seq=0 ttl=64 time=12.937 ms
64 bytes from 10.0.0.5: seq=1 ttl=64 time=82.216 ms

# from guest os ping host
$ ping 10.0.0.254
PING 10.0.0.254 (10.10.0.1): 56 data bytes
64 bytes from 10.0.0.254: seq=0 ttl=64 time=12.838 ms
64 bytes from 10.0.0.254: seq=1 ttl=64 time=81.121 ms