VMware Cloud Community
coobic
Contributor
Contributor

ESXi 5.5 PSODs (TLB Invalidate, PF Exception 14)

I`m got the several PSODs after 9 month of uptime with ESXi 5.1u1 and after updated to ESXi 5.5.

Screenshots and logs is attached.

3 VM running on host: 2xCentOS, 1xDebian (latest vmware tools) (VMXNet 3 adapters)

Latest PSOD:

2014-03-11T18:24:33.479Z cpu4:59166)User: 2886: wantCoreDump : rhttpproxy-work -enabled : 1

2014-03-11T18:24:36.264Z cpu7:32966)World: 8773: PRDA 0x418041c00000 ss 0x0 ds 0x4018 es 0x4018 fs 0x4018 gs 0x4018

2014-03-11T18:24:36.264Z cpu7:32966)World: 8775: TR 0x4020 GDT 0x4123831a1000 (0x402f) IDT 0x4180118f3000 (0xfff)

2014-03-11T18:24:36.264Z cpu7:32966)World: 8776: CR0 0x8001003d CR3 0xdda61000 CR4 0x216c

2014-03-11T18:24:36.264Z cpu7:32966)Backtrace for current CPU #7, worldID=32966, ebp=0x41238319dc90

2014-03-11T18:24:36.264Z cpu7:32966)0x41238319dc90:[0x41801188ccd9]PanicvPanicInt@vmkernel#nover+0x575 stack: 0x412300000008, 0x4123831

2014-03-11T18:24:36.264Z cpu7:32966)0x41238319dcf0:[0x41801188cf1d]Panic_NoSave@vmkernel#nover+0x49 stack: 0x41238319ddc0, 0x3, 0x0, 0x

2014-03-11T18:24:36.264Z cpu7:32966)0x41238319de80:[0x4180118a8c03]TLBDoInvalidate@vmkernel#nover+0x697 stack: 0x58, 0x0, 0x0, 0x0, 0x0

2014-03-11T18:24:36.265Z cpu7:32966)0x41238319def0:[0x418011d8a775]UserMem_CartelFlush@<None>#<None>+0xfd stack: 0x410900000000, 0x4109

2014-03-11T18:24:36.265Z cpu7:32966)0x41238319df30:[0x418011dffb74]UserMemTouchedEstimate@<None>#<None>+0x124 stack: 0x0, 0x4123831a700

2014-03-11T18:24:36.265Z cpu7:32966)0x41238319dfd0:[0x41801186138a]helpFunc@vmkernel#nover+0xab6 stack: 0x0, 0x0, 0x0, 0x0, 0x0

2014-03-11T18:24:36.265Z cpu7:32966)0x41238319dff0:[0x418011a53372]CpuSched_StartWorld@vmkernel#nover+0xfa stack: 0x0, 0x0, 0x0, 0x0, 0

2014-03-11T18:24:36.265Z cpu7:32966)←[45m←[33;1mVMware ESXi 5.5.0 [Releasebuild-1474528 x86_64]←[0m

PCPU 3 locked up. Failed to ack TLB invalidate (total of 1 locked up, PCPU(s): 3).

2014-03-11T18:24:36.265Z cpu7:32966)cr0=0x8001003d cr2=0x7f5abd0d2000 cr3=0xdda61000 cr4=0x216c

2014-03-11T18:24:36.265Z cpu7:32966)pcpu:0 world:33418 name:"helper43-0" (SH)

2014-03-11T18:24:36.265Z cpu7:32966)pcpu:1 world:32800 name:"directMapUnmap" (S)

2014-03-11T18:24:36.265Z cpu7:32966)pcpu:2 world:57443 name:"vmm0:unifi.bnav.ru" (V)

2014-03-11T18:24:36.265Z cpu7:32966)pcpu:3 world:56455 name:"vmm3:asterisk.bnav.ru" (V)

2014-03-11T18:24:36.265Z cpu7:32966)pcpu:4 world:34612 name:"vpxa-worker" (U)

2014-03-11T18:24:36.265Z cpu7:32966)pcpu:5 world:56628 name:"vmast.56625" ()

2014-03-11T18:24:36.265Z cpu7:32966)pcpu:6 world:32961 name:"memMap-6" (S)

2014-03-11T18:24:36.265Z cpu7:32966)pcpu:7 world:32966 name:"helper35-0" (SH)

2014-03-11T18:24:36.265Z cpu7:32966)@BlueScreen: PCPU 3 locked up. Failed to ack TLB invalidate (total of 1 locked up, PCPU(s): 3).

2014-03-11T18:24:36.265Z cpu7:32966)Code start: 0x418011800000 VMK uptime: 1:16:50:25.002

2014-03-11T18:24:36.265Z cpu7:32966)0x41238319dc90:[0x41801188ccd9]PanicvPanicInt@vmkernel#nover+0x575 stack: 0x412300000008

2014-03-11T18:24:36.265Z cpu7:32966)0x41238319dcf0:[0x41801188cf1d]Panic_NoSave@vmkernel#nover+0x49 stack: 0x41238319ddc0

2014-03-11T18:24:36.265Z cpu7:32966)0x41238319de80:[0x4180118a8c03]TLBDoInvalidate@vmkernel#nover+0x697 stack: 0x58

2014-03-11T18:24:36.265Z cpu7:32966)0x41238319def0:[0x418011d8a775]UserMem_CartelFlush@<None>#<None>+0xfd stack: 0x410900000000

2014-03-11T18:24:36.265Z cpu7:32966)0x41238319df30:[0x418011dffb74]UserMemTouchedEstimate@<None>#<None>+0x124 stack: 0x0

2014-03-11T18:24:36.265Z cpu7:32966)0x41238319dfd0:[0x41801186138a]helpFunc@vmkernel#nover+0xab6 stack: 0x0

2014-03-11T18:24:36.265Z cpu7:32966)0x41238319dff0:[0x418011a53372]CpuSched_StartWorld@vmkernel#nover+0xfa stack: 0x0

2014-03-11T18:24:36.266Z cpu7:32966)base fs=0x0 gs=0x418041c00000 Kgs=0x0

Has anyone same problems or ideas?

I`m try to test hardware - always ok,

I`m try to change hardware - with no luck,

I`m try to upgrade ESXi to 5.5 - with no luck.

PSOD is take place every 1-7 days.

Hardware configuration:

CPU: Intel Core i7-3770 (or i3-2xxx)

Memory: different modules (2x8Gb)

Storage controller: PCIE LSI 9650se-4LPML (or onboard AHCI) (RAID1 and standalone disks)

Motherboard: ASUStek P8Z77-V LX (or P8H61, or P8H67)

Network Card: PCIE Intel GigabitNetwork 82574L

Also changed PS.

Thank you guys!

Tags (2)
0 Kudos
3 Replies
shane019
Contributor
Contributor

I have this same issue would like to know what the fix is, have you been able to resolve your issue?

Here is the error I get.

pastedImage_0.png

0 Kudos
coobic
Contributor
Contributor

My issue is not resolved.

I`m try several things and after CPU Microcode update - 2 weeks uptime.

0 Kudos
virtualdive
VMware Employee
VMware Employee

Hi Coobic,

I had the similar problem, I upgraded the BIOS/Firmware on the Server Hardware as well as patched the host to most latest. The problem was solved and did not receive the PSOD.

Thanks,

Regards,

'V'
thevshish.blogspot.in
vExpert-2014-2021
0 Kudos