Bridge dropping small percent of traffic when using VLANs

Before posting something, READ the changelog, WATCH the videos, howto and provide following:
Your install is: Bare metal, ESXi, what CPU model, RAM, HD, what EVE version you have, output of the uname -a and any other info that might help us faster.

Moderator: mike

Post Reply
eRVee Moskovic
Posts: 3
Joined: Wed Jul 31, 2024 11:13 am

Bridge dropping small percent of traffic when using VLANs

Post by eRVee Moskovic » Thu Aug 01, 2024 8:22 am

Hi!

TL;DR: My issue is solved but please still read it because my solution needs manual host configuration every time I start the LAB or reboot the host.

I have a lab with part of our L2 WAN simulated by a "Network" of type "bridge". This seems to work fine, my routers (mostly IOL) can reach each other and form OSPF relations and test traffic (pings from vPCs) work.
20240801_001.png
But... :mrgreen: when pinging over the bridge I noticed some packet loss. Not much! But with some sort of interval it does drop packets.

I debugged this with tcpdumping the vnet0_1 bridge and the two vunl0_x_y interfaces on the host and I saw the ICMP echo request being sent on the vunl0_x_y of the sending router and I saw the request on the bridge, but sometimes it was not sent to the vunl0_x_y interface of the router which should have received the request. If I bypassed the bridge it all works 100%.

Now I do use VLANs over the WAN bridge. That's important! The cookbook states "The EVE Bridge interface acts like an unmanaged Switch. It supports passing along tagged dot1q packets".
I didn't want to replace the bridge with a switch because that would mean extra configuration work configuring the WAN VLANs or QinQ etc.

After hunting down the issue, a statement in https://wiki.debian.org/NetworkConfigur ... _Switching caught my eye: "By default the Linux bridge acts like a switch. This means, it remembers the MAC addresses behind a switch port and if the destination MAC address is known, data packets or only forward to the respective port - otherwise packets will be broadcasted. In some setups this is bad. For example if the bridge connects 2 trunk interfaces and the same MAC addresses may be seen from both interfaces, depending on the VLAN. As the Linux bridge does not support VLANs (dedicated MAC address tables per each VLAN), in such setups you have to disable the MAC address learning and put the bridge into a real "bridge" mode"

So I checked which host bridge was my WAN. The one with many vunl0_x_y members:

Code: Select all

root@eve-ng:~# brctl show
bridge name     bridge id               STP enabled     interfaces
[...snip...]
pnet1           8000.d0431e19ceaa       no              bond0.3080
vnet0_1         8000.1617ce59078d       no              vunl0_10_0
                                                        vunl0_25_0
                                                        vunl0_25_16
                                                        vunl0_26_0
                                                        vunl0_26_16
                                                        vunl0_32_1
                                                        vunl0_33_1
                                                        vunl0_7_0
                                                        vunl0_8_0
                                                        vunl0_8_16
                                                        vunl0_9_0
vnet0_13                8000.da25466f0ef0       no              vunl0_7_48
[...snip...]
The issued "brctl setageing vnet0_1 0" en voila! No more issues!
Now this is normally a setting you should not want to do, but in a lab with no production traffic this is fine. Although it makes troubleshooting with tcpdump or the capture option a bit worse because of the flooding of traffic to all bridge members. But this is a price I'm willing to pay for a lab that 100% passes traffic :D

The biggest bummer for me now is that this setting does not survive a lab stop/start or host reboot. So if a checkbox for let's say "support VLANs" could be added to the Network settings which in turn sets the bridge setageing to 0 that would be great!

So is this a call for help, a feature request or a call to change the cookbook statement about it supporting passing along tagged dot1q packets? I'll let you be the judge of that ;)

Yours,
Ralf.
You do not have the required permissions to view the files attached to this post.

eRVee Moskovic
Posts: 3
Joined: Wed Jul 31, 2024 11:13 am

Re: Bridge dropping small percent of traffic when using VLANs

Post by eRVee Moskovic » Thu Aug 01, 2024 11:34 am

Extra info if it matters:

Hardware : Dell PowerEdge M630 - Dual Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (16 cores, 32 thread total), 256.0 GB RAM, 280 GiB hardware RAID1 HDD with 1TB iSCSI storage on /opt, 2 NICs in bond for Management and Cloud1. 2 NICs for iSCSI multipath.

EVE-NG Running on - Bare Metal Linux (Ubuntu 20.04.6 LTS)

VT-X - Enabled (yes)

Before posting the Problem : ( Have you checked below steps Example)

My Node note starting. ( have you followed example below)

1. check the path of the image ( for windows win-XXXX): Not related to issue.

2. have you fixed permission(/opt/unetlab/wrappers/unl_wrapper -a fixpermissions): Yes but not related.

3. file name format matches against EVE-NG requirement mentioned in the site.: Yes but not related.

4. Check the logs /opt/unetlab/data/Logs/: Nothing related to the issue there.

rusty725
Posts: 360
Joined: Thu Mar 29, 2018 4:19 pm

Re: Bridge dropping small percent of traffic when using VLANs

Post by rusty725 » Fri Aug 02, 2024 12:03 am

upgrade to Eve version 6 , here's the how to https://www.eve-ng.net/index.php/1845-2 ... x-to-v6-x/

eRVee Moskovic
Posts: 3
Joined: Wed Jul 31, 2024 11:13 am

Re: Bridge dropping small percent of traffic when using VLANs

Post by eRVee Moskovic » Mon Aug 05, 2024 9:37 am

Ah, didn't see that major version upgrade. So I updated today and it is much better out of the box. I still miss an occasional ping but a lot less.

I could not find something in the changelog relating to this issue. Is it just the Ubuntu or kernel update or did you change something is the way EVE creates the bridge?

Ralf.

rusty725
Posts: 360
Joined: Thu Mar 29, 2018 4:19 pm

Re: Bridge dropping small percent of traffic when using VLANs

Post by rusty725 » Mon Aug 05, 2024 9:54 pm

there were many changes , major upgrade from ubuntu 20.04 to 22.04 and some other things.

Post Reply