EVE-ng Cummunity edition - error vcpu0, guest rIP: 0xffffffff81f29313 disabled perfctr wrmsr

[mathewfer]

Hi Team,

Any idea what are these errors reported on EVE-ng console and in "/var/log/kern.log"?

This was on EVE-ng community edition ver "5.0.1-19-Community". EVE-ng VM runs on ESXi with the below spec.

- ESXi 6.7 - VMware vSphere 6 Enterprise Plus
- 36 CPUs x Intel(R) Xeon(R) CPU E5-2695 v4 @ 2.10GHz - Dual CPU
- 128 RAM
- 2T SSD for EVE-ng and ESXi runs on two 250G SSDs and these two SSDs are about 5 years old.

I had these errors on another store running on 2x250G SSDs before but now I setup a new EVE-ng VM - 1.5T and 120G RAM, 30 CPUs allocated and it runs on a new 2T SSD added as 3rd store few days ago but I still gets these errors.

On ESXi, I have EVE-ng and a Ubuntu Desktop with 16G RAM and 100G HD as a client for EVE-ng. That is all run on this SuperMicro X10DRL-i motherboard

LABs nodes works mostly but some Linux QUEM gets these errors in /var/log/kern.log and during this error, Linux nodes freezes - no response.

Code: Select all

Nov 24 02:41:36 tools-srv kernel: watchdog: BUG: soft lockup - CPU#2 stuck for 58s! [sshd:28626]
Nov 24 02:41:37 tools-srv kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 58s! [timer_manager:2230]
Nov 24 04:56:10 tools-srv kernel: watchdog: BUG: soft lockup - CPU#1 stuck for 31s! [scheduler:1986]
Nov 25 06:29:48 tools-srv kernel: watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/3:1:43086]

EVE-ng errors - in /var/log/kern.log

Code: Select all

Nov 22 11:19:55 EVE-ng kernel: [20905.393260] kvm [83580]: vcpu0, guest rIP: 0xffffffff81f29313 disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:19:56 EVE-ng kernel: [20906.981125] kvm [84970]: vcpu0, guest rIP: 0xffffffff81f29313 disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:20:01 EVE-ng kernel: [20911.439577] kvm [88755]: vcpu0, guest rIP: 0xffffffff81f29313 disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:20:02 EVE-ng kernel: [20912.462135] kvm [89644]: vcpu0, guest rIP: 0xffffffff81f29313 disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:20:07 EVE-ng kernel: [20918.123446] kvm [86050]: vcpu0, guest rIP: 0xffffffff810549ac disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:20:11 EVE-ng kernel: [20921.778751] kvm [83580]: vcpu0, guest rIP: 0xffffffff8257d030 disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:20:13 EVE-ng kernel: [20924.142443] kvm [84970]: vcpu0, guest rIP: 0xffffffff8257d030 disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:20:21 EVE-ng kernel: [20932.376328] kvm [82707]: vcpu0, guest rIP: 0xffffffffb6e96c64 disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:20:23 EVE-ng kernel: [20933.494919] kvm [88755]: vcpu0, guest rIP: 0xffffffff8257d030 disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:20:24 EVE-ng kernel: [20934.498634] kvm [89644]: vcpu0, guest rIP: 0xffffffff8257d030 disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:24:04 EVE-ng kernel: [21154.713194] kvm [98101]: vcpu0, guest rIP: 0xffffffff810549ac disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:24:07 EVE-ng kernel: [21157.414069] kvm [99108]: vcpu0, guest rIP: 0xffffffff810549ac disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:24:09 EVE-ng kernel: [21160.214171] kvm [100062]: vcpu0, guest rIP: 0xffffffff810549ac disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:24:10 EVE-ng kernel: [21161.037438] kvm [104972]: vcpu0, guest rIP: 0xffffffff81f29313 disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:24:11 EVE-ng kernel: [21162.139227] kvm [100973]: vcpu0, guest rIP: 0xffffffff810549ac disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:24:13 EVE-ng kernel: [21163.513058] kvm [101950]: vcpu0, guest rIP: 0xffffffff810549ac disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:24:13 EVE-ng kernel: [21163.703948] kvm [106003]: vcpu0, guest rIP: 0xffffffff81f29313 disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:24:14 EVE-ng kernel: [21165.289868] kvm [102942]: vcpu0, guest rIP: 0xffffffff810549ac disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:24:17 EVE-ng kernel: [21167.794880] kvm [104000]: vcpu0, guest rIP: 0xffffffff810549ac disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:24:28 EVE-ng kernel: [21179.009872] kvm [104972]: vcpu0, guest rIP: 0xffffffff8257d030 disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:24:31 EVE-ng kernel: [21181.526157] kvm [106003]: vcpu0, guest rIP: 0xffffffff8257d030 disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 22 11:25:04 EVE-ng kernel: [21215.288687] kvm [109512]: vcpu0, guest rIP: 0xffffffff810549ac disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 23 08:18:53 EVE-ng kernel: [96445.309587] kvm [1380054]: vcpu0, guest rIP: 0xffffffff81f29313 disabled perfctr wrmsr: 0xc2 data 0xffff
Nov 23 08:18:57 EVE-ng kernel: [96448.872231] kvm [1381309]: vcpu0, guest rIP: 0xffffffff81f29313 disabled perfctr wrmsr: 0xc2 data 0xffff

Thanks in advance any help on this.

[Uldis (UD)]

simple, your EVE VM CU cannot hold your lab nodes. Even this is just informational nmessage, but you have to bear in mind, that CPU is on edge now, it has no power

[mathewfer]

Hi UD/Team,

Thanks for the prompt reply.

I should also mentioned that this started after adding the 2nd CPU (same model). Each CPU is with 64G RAM. I believe I am not correctly setting the vCPUs etc for the EVE-ng VM. I used the VM downloaded from your download page.
I have attached here the ESXi 6.7 reported CPU COREs and the setting I have for EVE-ng.

Can I please ask your help to understand what should be the setting for EVE-ng VM for Dual CPU motherboard?

Thanks again in advance for your reply.

ESXi reported CPU - Dual CPUs


EVE-ng VM settings:

ESX-6-7-EVE-ng-VM-settings.JPG

[Uldis (UD)]

https://www.eve-ng.net/index.php/faq/

last FAQ post in this page