Running EVE-NG in KVM

[b77miy]

Hi guys,

I've installed a Centos7 with KVM, and inside that host machine, i've created a EVE-NG VM, and started to run nodes inside. Compared with with my old setup (bare metal install), running nodes (IOSv only) seems sluggish, and even spinning one router takes 5min+, and when finally does start, Guest VM processor reports high usage (~30%), when compared with the bare metal install. I've tried to troubleshot KVM virtualization issues, but no success so far.

Any sugestions are apreciated!

My setup is as follows.
Host (centos7 with KVM):
2x Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz
128GB RAM
Lots of storage
Guest VM (EVE-NG):
10x CPU cores from the host
64GB RAM

I've checked if KVM/VT-x support is enabled, and seems the settings are right.
Checks on VM (EVE-NG):
- kvm-ok

Code: Select all

$ kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

- lscpu

Code: Select all

$ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                20
On-line CPU(s) list:   0-19
Thread(s) per core:    1
Core(s) per socket:    1
Socket(s):             20
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 85
Model name:            Intel Xeon Processor (Skylake, IBRS)
Stepping:              4
CPU MHz:               2194.848
BogoMIPS:              4389.69
Virtualization:        VT-x
Hypervisor vendor:     KVM
Virtualization type:   full
L1d cache:             32K
L1i cache:             32K
L2 cache:              4096K
NUMA node0 CPU(s):     0-19
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl eagerfpu pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch tpr_shadow vnmi flexpriority ept vpid fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 pku ospke

Checks on Host server:

Code: Select all

$ virt-host-validate
  QEMU: Checking for hardware virtualization                                 : PASS
  QEMU: Checking if device /dev/kvm exists                                   : PASS
  QEMU: Checking if device /dev/kvm is accessible                            : PASS
  QEMU: Checking if device /dev/vhost-net exists                             : PASS
  QEMU: Checking if device /dev/net/tun exists                               : PASS
  QEMU: Checking for cgroup 'memory' controller support                      : PASS
  QEMU: Checking for cgroup 'memory' controller mount-point                  : PASS
  QEMU: Checking for cgroup 'cpu' controller support                         : PASS
  QEMU: Checking for cgroup 'cpu' controller mount-point                     : PASS
  QEMU: Checking for cgroup 'cpuacct' controller support                     : PASS
  QEMU: Checking for cgroup 'cpuacct' controller mount-point                 : PASS
  QEMU: Checking for cgroup 'cpuset' controller support                      : PASS
  QEMU: Checking for cgroup 'cpuset' controller mount-point                  : PASS
  QEMU: Checking for cgroup 'devices' controller support                     : PASS
  QEMU: Checking for cgroup 'devices' controller mount-point                 : PASS
  QEMU: Checking for cgroup 'blkio' controller support                       : PASS
  QEMU: Checking for cgroup 'blkio' controller mount-point                   : PASS
  QEMU: Checking for device assignment IOMMU support                         : PASS
  QEMU: Checking if IOMMU is enabled by kernel                               : WARN (IOMMU appears to be disabled in kernel

Code: Select all

***** HOSTNAME INFORMATION *****
   Static hostname: eve
    Virtualization: kvm
  Operating System: Ubuntu 16.04.5 LTS
            Kernel: Linux 4.9.40-eve-ng-ukms-2+
      Architecture: x86-64

***** FILE SYSTEM DISK SPACE USAGE *****
Filesystem                     Size  Used Avail Use% Mounted on
udev                            32G     0   32G   0% /dev
tmpfs                          6.3G   17M  6.3G   1% /run
/dev/mapper/eve--vg-root   97G   28G   65G  31% /
tmpfs                           32G     0   32G   0% /dev/shm
tmpfs                          5.0M     0  5.0M   0% /run/lock
tmpfs                           32G     0   32G   0% /sys/fs/cgroup
/dev/vda1                      720M  172M  512M  26% /boot
tmpfs                          6.3G     0  6.3G   0% /run/user/1000

***** CPU INFORMATION  *****
model name      : Intel Xeon Processor (Skylake, IBRS)

***** FREE AND USED MEMORY *****
              total        used        free      shared  buff/cache   available
Mem:            62G        551M         61G         27M        478M         61G
Swap:          975M          0B        975M

[Uldis (UD)]

we discussed this is chat mate, it is not the best scenario you have there.

Uldis

[b77miy]

Hi UD!

Indeed, thanks for the prompt response on Live Chat.
Meanwhile, actually, I have success with my implementation, and I'll post a description shortly.

Thanks once again.

[Miljan]

Hi UD!

Indeed, thanks for the prompt response on Live Chat.
Meanwhile, actually, I have success with my implementation, and I'll post a description shortly.

Thanks once again.

Can you announce how you got on with your implementation?

[heman]

Also am curious what was done to make this work better. In my case the CPU usage seems to calm down on vIOS, ASAv and IOS-XRv but CSR1000v doesn't.

I have tried -enable-kvm in the node options as well as -cpu host with no significant change in performance

[gcastbr]

HI Mate, Yes indeed, I created this topic before the chat!