Live migration – CPU generations

Hi

We have a hyper-v cluster running 2012R2 with hosts with different CPU generations, v2 ,3.

we use cpu compability for all VM’s

we are now planing to move to 2016, + replace some of the oldest hosts.
during testing migration from hosts with:

Intel(R) Xeon(R) CPU E7- 2870  @ 2.40GHz

Intel(R) Xeon(R) CPU E7- 2870  @ 2.40GHz

Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz

Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz

Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz

to the new hosts with:

Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz

causes GEN2 VM’s to crash with BSOD a few minutes after migration to the new hosts. 

The problem is the same if i reinstall one of the old host with 2016, and migrate from this to the new hosts with new cpu.

both the old and the new host have updated CPU microcode for spectre (i have not exposed this to VM’s yet) 

i have another 2012R2 cluster where live migration works with both:

Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz

and

Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz

DMP:

*******************************************************************************
*                                                                         
   *
*                        Bugcheck Analysis                                    *
*                                                                         
   *
*******************************************************************************

DRIVER_IRQL_NOT_LESS_OR_EQUAL (d1)
An attempt was made to access a pageable (or completely invalid) address at an
interrupt request level (IRQL) that is too high.  This is usually
caused by drivers using improper addresses.
If kernel debugger is available get stack backtrace.
Arguments:
Arg1: ffffe001bb15fe58, memory referenced
Arg2: 0000000000000002, IRQL
Arg3: 0000000000000008, value 0 = read operation, 1 = write operation
Arg4: ffffe001bb15fe58, address which referenced memory

Debugging Details:
——————

DUMP_CLASS: 1

DUMP_QUALIFIER: 401

BUILD_VERSION_STRING:  6.3.9600.18969 (winblue_ltsb.180309-0600)

SYSTEM_MANUFACTURER:  Microsoft Corporation

VIRTUAL_MACHINE:  HyperV

SYSTEM_PRODUCT_NAME:  Virtual Machine

SYSTEM_SKU:  None

SYSTEM_VERSION:  Hyper-V UEFI Release v1.0

BIOS_VENDOR:  Microsoft Corporation

BIOS_VERSION:  Hyper-V UEFI Release v1.0

BIOS_DATE:  11/26/2012

BASEBOARD_MANUFACTURER:  Microsoft Corporation

BASEBOARD_PRODUCT:  Virtual Machine

BASEBOARD_VERSION:  Hyper-V UEFI Release v1.0

DUMP_TYPE:  1

BUGCHECK_P1: ffffe001bb15fe58

BUGCHECK_P2: 2

BUGCHECK_P3: 8

BUGCHECK_P4: ffffe001bb15fe58

READ_ADDRESS:  ffffe001bb15fe58 Nonpaged pool

CURRENT_IRQL:  2

FAULTING_IP: 
+0
ffffe001`bb15fe58 ??              ???

CPU_COUNT: 2

CPU_MHZ: 95b

CPU_VENDOR:  GenuineIntel

CPU_FAMILY: 6

CPU_MODEL: 2f

CPU_STEPPING: 2

CPU_MICROCODE: 6,2f,2,0 (F,M,S,R)  SIG: FFFFFFFF’00000000 (cache) FFFFFFFF’00000000 (init)

DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

BUGCHECK_STR:  AV

PROCESS_NAME:  System

ANALYSIS_SESSION_HOST:  MGT2

ANALYSIS_SESSION_TIME:  04-11-2018 14:23:37.0109

ANALYSIS_VERSION: 10.0.16299.91 amd64fre

TRAP_FRAME:  ffffd0017a7b7030 — (.trap 0xffffd0017a7b7030)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=0000000080040033 rbx=0000000000000000 rcx=fffff6fb7dbedf80
rdx=ffffd0017a7b75d0 rsi=0000000000000000 rdi=0000000000000000
rip=ffffe001bb15fe58 rsp=ffffd0017a7b71c8 rbp=ffffd0017a7b7280
 r8=0000000000000000  r9=0000000000000000 r10=7010008004002001
r11=0000000080050033 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0         nv up ei pl nz na pe nc
ffffe001`bb15fe58 ??              ???
Resetting default scope

LAST_CONTROL_TRANSFER:  from fffff803465df8a9 to fffff803465cd2a0

FAILED_INSTRUCTION_ADDRESS: 
+0
ffffe001`bb15fe58 ??              ???

STACK_TEXT:  
ffffd001`7a7b6ee8 fffff803`465df8a9 : 00000000`0000000a ffffe001`bb15fe58 00000000`00000002 00000000`00000008 : nt!KeBugCheckEx
ffffd001`7a7b6ef0 fffff803`465dc356 : 00000000`00000008 00000000`00804063 00000000`00000000 00000000`00000000 : nt!KiBugCheckDispatch+0x69
ffffd001`7a7b7030 ffffe001`bb15fe58 : ffffe001`bb64307f b3b74bde`e4453415 ffffd001`7a7b7280 ffffe001`bb638022 : nt!KiPageFault+0x516
ffffd001`7a7b71c8 ffffe001`bb64307f : b3b74bde`e4453415 ffffd001`7a7b7280 ffffe001`bb638022 00000000`00000001 : 0xffffe001`bb15fe58
ffffd001`7a7b71d0 b3b74bde`e4453415 : ffffd001`7a7b7280 ffffe001`bb638022 00000000`00000001 ffffe001`bb248820 : 0xffffe001`bb64307f
ffffd001`7a7b71d8 ffffd001`7a7b7280 : ffffe001`bb638022 00000000`00000001 ffffe001`bb248820 ffffc000`565ce900 : 0xb3b74bde`e4453415
ffffd001`7a7b71e0 ffffe001`bb638022 : 00000000`00000001 ffffe001`bb248820 ffffc000`565ce900 00000000`00000000 : 0xffffd001`7a7b7280
ffffd001`7a7b71e8 00000000`00000001 : ffffe001`bb248820 ffffc000`565ce900 00000000`00000000 ffffd001`7a7b7478 : 0xffffe001`bb638022
ffffd001`7a7b71f0 ffffe001`bb248820 : ffffc000`565ce900 00000000`00000000 ffffd001`7a7b7478 ffffc000`565ce788 : 0x1
ffffd001`7a7b71f8 ffffc000`565ce900 : 00000000`00000000 ffffd001`7a7b7478 ffffc000`565ce788 00000000`00000000 : 0xffffe001`bb248820
ffffd001`7a7b7200 00000000`00000000 : ffffd001`7a7b7478 ffffc000`565ce788 00000000`00000000 ffffe001`bc2270c0 : 0xffffc000`565ce900

THREAD_SHA1_HASH_MOD_FUNC:  bf99962f16aee8a6a536cfcc5454c0cd4db15ac9

THREAD_SHA1_HASH_MOD_FUNC_OFFSET:  1ac8ae97df21cc5e25a79e0299d3df812a04aba8

THREAD_SHA1_HASH_MOD:  2a7ca9d3ab5386d53fea7498e1d81b9c4a4c036b

FOLLOWUP_IP: 
nt!KiPageFault+516
fffff803`465dc356 440f20c0        mov     rax,cr8

FAULT_INSTR_CODE:  c0200f44

SYMBOL_STACK_INDEX:  2

SYMBOL_NAME:  nt!KiPageFault+516

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: nt

IMAGE_NAME:  ntkrnlmp.exe

DEBUG_FLR_IMAGE_TIMESTAMP:  5aa29c76

IMAGE_VERSION:  6.3.9600.18969

STACK_COMMAND:  .thread ; .cxr ; kb

BUCKET_ID_FUNC_OFFSET:  516

FAILURE_BUCKET_ID:  AV_CODE_AV_BAD_IP_nt!KiPageFault

BUCKET_ID:  AV_CODE_AV_BAD_IP_nt!KiPageFault

PRIMARY_PROBLEM_CLASS:  AV_CODE_AV_BAD_IP_nt!KiPageFault

TARGET_TIME:  2018-04-11T12:15:10.000Z

OSBUILD:  9600

OSSERVICEPACK:  18969

SERVICEPACK_NUMBER: 0

OS_REVISION: 0

SUITE_MASK:  272

PRODUCT_TYPE:  3

OSPLATFORM_TYPE:  x64

OSNAME:  Windows 8.1

OSEDITION:  Windows 8.1 Server TerminalServer SingleUserTS

OS_LOCALE:  

USER_LCID:  0

OSBUILD_TIMESTAMP:  2018-03-09 15:38:46

BUILDDATESTAMP_STR:  180309-0600

BUILDLAB_STR:  winblue_ltsb

BUILDOSVER_STR:  6.3.9600.18969

ANALYSIS_SESSION_ELAPSED_TIME:  8d2

ANALYSIS_SOURCE:  KM

FAILURE_ID_HASH_STRING:  km:av_code_av_bad_ip_nt!kipagefault

FAILURE_ID_HASH:  {73cd60cc-83fa-6b76-df08-1961c31d7403}

Followup:     MachineOwner
———

######################################################

Microsoft (R) Windows Debugger Version 10.0.16299.91 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.

Loading Dump File [X:TempMEMORY.DMP]
Kernel Bitmap Dump File: Full address space is available

Symbol search path is: srv*
Executable search path is: 
Windows 8.1 Kernel Version 9600 MP (2 procs) Free x64
Product: Server, suite: TerminalServer SingleUserTS
Built by: 9600.18969.amd64fre.winblue_ltsb.180309-0600
Machine Name:
Kernel base = 0xfffff800`b9e15000 PsLoadedModuleList = 0xfffff800`ba0e1570
Debug session time: Sun Apr 15 12:01:19.698 2018 (UTC + 2:00)
System Uptime: 0 days 1:15:10.753
Loading Kernel Symbols
………………………………………………………
………………………………………………..
Loading User Symbols

Loading unloaded module list
…….
*******************************************************************************
*                                                                         
   *
*                        Bugcheck Analysis                                    *
*                                                                         
   *
*******************************************************************************

Use !analyze -v to get detailed debugging information.

BugCheck FC, {ffffe000be9b0078, 80000000130009e3, ffffd001eadf0030, 3}

Probably caused by : ntkrnlmp.exe ( nt!MI_CHECK_KERNEL_NOEXECUTE_FAULT+64 )

Followup:     MachineOwner
———

0: kd> !analyze -v
*******************************************************************************
*                                                                         
   *
*                        Bugcheck Analysis                                    *
*                                                                         
   *
*******************************************************************************

ATTEMPTED_EXECUTE_OF_NOEXECUTE_MEMORY (fc)
An attempt was made to execute non-executable memory.  The guilty driver
is on the stack trace (and is typically the current instruction pointer).
When possible, the guilty driver’s name (Unicode string) is printed on
the bugcheck screen and saved in KiBugCheckDriver.
Arguments:
Arg1: ffffe000be9b0078, Virtual address for the attempted execute.
Arg2: 80000000130009e3, PTE contents.
Arg3: ffffd001eadf0030, (reserved)
Arg4: 0000000000000003, (reserved)

Debugging Details:
——————

DUMP_CLASS: 1

DUMP_QUALIFIER: 402

BUILD_VERSION_STRING:  9600.18969.amd64fre.winblue_ltsb.180309-0600

SYSTEM_MANUFACTURER:  Microsoft Corporation

VIRTUAL_MACHINE:  HyperV

SYSTEM_PRODUCT_NAME:  Virtual Machine

SYSTEM_SKU:  None

SYSTEM_VERSION:  Hyper-V UEFI Release v1.0

BIOS_VENDOR:  Microsoft Corporation

BIOS_VERSION:  Hyper-V UEFI Release v1.0

BIOS_DATE:  11/26/2012

BASEBOARD_MANUFACTURER:  Microsoft Corporation

BASEBOARD_PRODUCT:  Virtual Machine

BASEBOARD_VERSION:  Hyper-V UEFI Release v1.0

DUMP_TYPE:  0

BUGCHECK_P1: ffffe000be9b0078

BUGCHECK_P2: 80000000130009e3

BUGCHECK_P3: ffffd001eadf0030

BUGCHECK_P4: 3

CPU_COUNT: 2

CPU_MHZ: a22

CPU_VENDOR:  GenuineIntel

CPU_FAMILY: 6

CPU_MODEL: 55

CPU_STEPPING: 4

CPU_MICROCODE: 6,55,4,0 (F,M,S,R)  SIG: FFFFFFFF’00000000 (cache) FFFFFFFF’00000000 (init)

DEFAULT_BUCKET_ID:  WIN8_DRIVER_FAULT

BUGCHECK_STR:  0xFC

PROCESS_NAME:  System

CURRENT_IRQL:  2

ANALYSIS_SESSION_HOST:  MGT2

ANALYSIS_SESSION_TIME:  04-16-2018 08:39:52.0749

ANALYSIS_VERSION: 10.0.16299.91 amd64fre

TRAP_FRAME:  ffffd001eadf0030 — (.trap 0xffffd001eadf0030)
NOTE: The trap frame does not contain all registers.
Some register values may be zeroed or incorrect.
rax=0000000080040033 rbx=0000000000000000 rcx=fffff6fb7dbedf80
rdx=ffffd001eadf05d0 rsi=0000000000000000 rdi=0000000000000000
rip=ffffe000be9b0078 rsp=ffffd001eadf01c8 rbp=ffffd001eadf0280
 r8=0000000000000000  r9=0000000000000000 r10=7010008004002001
r11=0000000080050033 r12=0000000000000000 r13=0000000000000000
r14=0000000000000000 r15=0000000000000000
iopl=0         nv up ei pl nz na po nc
ffffe000`be9b0078 ec              in      al,dx
Resetting default scope

LAST_CONTROL_TRANSFER:  from fffff800b9f0d924 to fffff800b9f6a2a0

STACK_TEXT:  
ffffd001`eadefe38 fffff800`b9f0d924 : 00000000`000000fc ffffe000`be9b0078 80000000`130009e3 ffffd001`eadf0030 : nt!KeBugCheckEx
ffffd001`eadefe40 fffff800`ba0058e3 : ffffd001`eadefff0 ffffd001`eadefff0 ffffd001`eadf0010 ffffd001`eadf0478 : nt!MI_CHECK_KERNEL_NOEXECUTE_FAULT+0x64
ffffd001`eadefe80 fffff800`b9e7286b : 00000000`00804063 00000000`00000000 ffffd001`eadeff40 ffffd001`eadf0478 : nt!MiRaisedIrqlFault+0x167
ffffd001`eadefee0 fffff800`b9f7924b : 00000000`00000008 00000000`00804063 00000000`00000000 00000000`00000000 : nt!MmAccessFault+0x12b
ffffd001`eadf0030 ffffe000`be9b0078 : ffffe000`bee9329f b3b74bde`e4453415 ffffd001`eadf0280 ffffe000`bee88232 : nt!KiPageFault+0x40b
ffffd001`eadf01c8 ffffe000`bee9329f : b3b74bde`e4453415 ffffd001`eadf0280 ffffe000`bee88232 00000000`00000001 : 0xffffe000`be9b0078
ffffd001`eadf01d0 b3b74bde`e4453415 : ffffd001`eadf0280 ffffe000`bee88232 00000000`00000001 ffffe000`bd247200 : 0xffffe000`bee9329f
ffffd001`eadf01d8 ffffd001`eadf0280 : ffffe000`bee88232 00000000`00000001 ffffe000`bd247200 00000000`00000000 : 0xb3b74bde`e4453415
ffffd001`eadf01e0 ffffe000`bee88232 : 00000000`00000001 ffffe000`bd247200 00000000`00000000 00000000`00000000 : 0xffffd001`eadf0280
ffffd001`eadf01e8 00000000`00000001 : ffffe000`bd247200 00000000`00000000 00000000`00000000 ffffd001`eadf0478 : 0xffffe000`bee88232
ffffd001`eadf01f0 ffffe000`bd247200 : 00000000`00000000 00000000`00000000 ffffd001`eadf0478 ffffd001`eadf0a40 : 0x1
ffffd001`eadf01f8 00000000`00000000 : 00000000`00000000 ffffd001`eadf0478 ffffd001`eadf0a40 ffffe000`bd26e7e0 : 0xffffe000`bd247200

THREAD_SHA1_HASH_MOD_FUNC:  d8d3c9d3f3bd90bb79643a25c50140bd6e2bb5f7

THREAD_SHA1_HASH_MOD_FUNC_OFFSET:  c0f048902412ca9f961b8af857bbed099a1e34d1

THREAD_SHA1_HASH_MOD:  f08ac56120cad14894587db086f77ce277bfae84

FOLLOWUP_IP: 
nt!MI_CHECK_KERNEL_NOEXECUTE_FAULT+64
fffff800`b9f0d924 cc              int     3

FAULT_INSTR_CODE:  c48348cc

SYMBOL_STACK_INDEX:  1

SYMBOL_NAME:  nt!MI_CHECK_KERNEL_NOEXECUTE_FAULT+64

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: nt

IMAGE_NAME:  ntkrnlmp.exe

DEBUG_FLR_IMAGE_TIMESTAMP:  5aa29c76

STACK_COMMAND:  .thread ; .cxr ; kb

BUCKET_ID_FUNC_OFFSET:  64

FAILURE_BUCKET_ID:  0xFC_nt!MI_CHECK_KERNEL_NOEXECUTE_FAULT

BUCKET_ID:  0xFC_nt!MI_CHECK_KERNEL_NOEXECUTE_FAULT

PRIMARY_PROBLEM_CLASS:  0xFC_nt!MI_CHECK_KERNEL_NOEXECUTE_FAULT

TARGET_TIME:  2018-04-15T10:01:19.000Z

OSBUILD:  9600

OSSERVICEPACK:  0

SERVICEPACK_NUMBER: 0

OS_REVISION: 0

SUITE_MASK:  272

PRODUCT_TYPE:  3

OSPLATFORM_TYPE:  x64

OSNAME:  Windows 8.1

OSEDITION:  Windows 8.1 Server TerminalServer SingleUserTS

OS_LOCALE:  

USER_LCID:  0

OSBUILD_TIMESTAMP:  2018-03-09 15:38:46

BUILDDATESTAMP_STR:  180309-0600

BUILDLAB_STR:  winblue_ltsb

BUILDOSVER_STR:  6.3.9600.18969.amd64fre.winblue_ltsb.180309-0600

ANALYSIS_SESSION_ELAPSED_TIME:  804

ANALYSIS_SOURCE:  KM

FAILURE_ID_HASH_STRING:  km:0xfc_nt!mi_check_kernel_noexecute_fault

FAILURE_ID_HASH:  {1d0ea106-bc06-493d-fc9b-30e5321b79b0}

Followup:     MachineOwner
———

Read full post . . . .

Go Que Newsroom Categories

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 46 bytes) in /home/content/36/8658336/html/goquecom/wp-includes/wp-db.php on line 1995