Use of syscall and sysenter in VMProtect 3.1
Few days ago Xjun briefly mentioned a new feature of VMProtect 3.1 - it uses direct syscalls to check if software is running under debugger. I decided to take a quick look myself and figure out how exactly it works.I'll give you 2 targets for the research:
oppo_flash_tool.exe (https://mega.nz/#!ZgJzjQxR!cNEHMwM-jKnLVgPXf4OUupyk1DNt69FYB2rEfY-5AlA) - it was given as an example by Xjun;
asshurt.dll (https://mediafire.com/?3xyc0ugc2hxervn) - some sort of a cheat for Roblox. I don't care about the cheat itself, it just happened to have the syscall feature enabled. And it uses different syscalls than Oppo.
In addition to that, I'll provide a very simple demo executable which replicates part of the VMProtect protection, so that you don't waste time looking at obfuscated code.
As a debugger in 32-bit OS you can use anything you like. On 64-bit OS you will really need to use WinDbg - as far as I know, it's the only debugger that can handle those tricks..
32bit OS
Let's start by debugging oppo_flash_tool.exe. First, we need to get past the usual tricks like IsDebuggerPresent and CheckRemoteDebuggerPresent. If you're reading this, I'm sure you know how to do that.
Few moments later we'll arrive here:
00dcfd8e f744250000800000 test dword ptr ,8000h
00dcfd96 e9f6bdfdff jmp 00dabb91
...
00dabb91 0f84841a0b00 je 00e5d61b
Remember this conditional jump. It's taken on 32bit OSes and not taken on 64-bit OS.
Let's look at 32bit OS version first. Now VMProtect prepares to call sysenter.
00e5d61b 56 push esi
00e5d61c 85f8 test eax,edi
00e5d61e 57 push edi
00e5d61f 0facd074 shrd eax,edx,74h
00e5d623 6633d3 xor dx,bx
00e5d626 53 push ebx
00e5d627 8bd9 mov ebx,ecx
00e5d629 81fcfe567054 cmp esp,547056FEh
00e5d62f 8bd3 mov edx,ebx
00e5d631 35fb5cc775 xor eax,75c75cfbh
00e5d636 c1e202 shl edx,2
00e5d639 0fbae028 bt eax,28h
00e5d63d 8bc5 mov eax,ebp
00e5d63f 6681fbda38 cmp bx,38DAh
00e5d644 8d0410 lea eax,
00e5d647 f7c615443214 test esi,14324415h
...
Since the code is obfuscated, here comes a cleaned-up version. Please note that in other applications, different registers can be used.
;input:
; ECX = number of parameters for the syscall
; = syscall id. See https://github.com/tinysec/windows-syscall-table
; .. = params for the syscall
; .. = free space to save registers
push esi
push edi
push ebx
; save register values for later
lea eax,
mov dword ptr , eax
mov dword ptr , esp
; set up stack frame for syscall
setupParams:
mov eax, dword ptr
push eax
sub ecx, 1
jnz setupParams
; put syscall number in EAX
mov eax, dword ptr
; the actual call
call trampoline1
; restore stack and frame pointers
mov esp, dword ptr
mov ebp, dword ptr
; save result
mov dword ptr , eax
; restore registers
pop ebx
pop edi
pop esi
jmp 00de21de
trampoline1:
call trampoline2
retn
trampoline2:
mov edx, esp
sysenter
retn
// -- continue VM execution as usual --
00de21de 8b06 mov eax,dword ptr
00de21e2 8db604000000 lea esi,
00de21ea 33c3 xor eax,ebx
00de21ec 8d807cc2efb1 lea eax,
00de21f6 f7d0 not eax
00de21fd 35ee613a76 xor eax,763a61ee
00df9a28 48 dec eax
00df9a29 f8 clc
00df9a2a c1c002 rol eax,2
00df9a2e 33d8 xor ebx,eax
00df9a32 03f8 add edi,eax
00df9a34 e9a0eafbff jmp 00db84d9
00db84d9 ffe7 jmp edi
...
00ddaeba 8b542500 mov edx,dword ptr
64-bit OS
Here it is getting interesting! smile You cannot use sysenter instruction from 32-bit code in 64-bit Windows. But, as ReWolf described few years ago, one can mix x86 code with x64 code in the same process. And that's exactly what VMProtect 3.1 is doing.
Let's go back to that conditional jump and see what happens in 64-bit OS. The jump will not be taken:
00dabb91 0f84841a0b00 je 00e5d61b
00dabb97 9adf4ce8003300call 0033:00E84CDF
00dabb9e 668cd1 mov cx,ss
00dabba1 668ed1 mov ss,cx
Far call?! Last time I saw that was in 16-bit Windows era..
As explained in ReWolf's article:
Summing things up, for every process (x86 & x64) running on 64-bits Windows there are allocated two code segments:
cs = 0x23 -> x86 mode
cs = 0x33 -> x64 mode
So, as soon as you execute that call, you'll switch to a 64-bit world. WinDbg happily recognizes that, all other debuggers just go astray..
00dabb97 9adf4ce8003300call 0033:00E84CDF
00000000`00e84cdf 56 push rsi
00000000`00e84ce0 57 push rdi
00000000`00e84ce1 53 push rbx
00000000`00e84ce2 8bd9 mov ebx,ecx
00000000`00e84ce4 8bd3 mov edx,ebx
00000000`00e84ce6 33c9 xor ecx,ecx
00000000`00e84ce8 81fb04000000 cmp ebx,4
00000000`00e84cee 0f8606000000 jbe 00000000`00e84cfa
00000000`00e84cf4 8d8bfcffffff lea ecx,
00000000`00e84cfa c1e103 shl ecx,3
...
x64 code does pretty much the same thing as x86 code - sets up a stack frame, sets up registers and then executes syscall instruction. Cleaned-up and shortened version follows:
;input:
; ECX = number of parameters for the syscall
; = encoded syscall id.
; High order byte = special handling info
; Lowest 15 bits = syscall id
; .. = params for the syscall
; .. = free space to save registers
; = free space to use in specific syscalls
push rsi
push rdi
push rbx
mov ebx,ecx
mov edx,ebx
xor ecx,ecx
; calculate new stack frame pointer
cmp ebx,4
jbe @F
lea ecx,
@@:
shl ecx,3
shl edx,2
mov rax,rbp
add rax,rdx
; save registers
mov qword ptr ,rax
mov qword ptr ,rsp
; adjust RSP
sub rsp,rcx
and rsp,0FFFFFFFFFFFFFFF0h
add rsp,rcx
; useless?
mov r10d,dword ptr
shr r10d,9
; set up params for syscall
test ebx,ebx
je doneSettingParams
loopSetParams:
mov eax,dword ptr
cmp ebx,1
jne @F
mov rcx,rax
jmp nextParam
@@:
cmp ebx,2
jne @F
mov rdx,rax
jmp nextParam
@@:
cmp ebx,3
jne @F
mov r8,rax
jmp nextParam
@@:
cmp ebx,4
jne @F
mov r9,rax
jmp nextParam
@@:
push rax
nextParam:
sub ebx,1
jne loopSetParams
doneSettingParams:
; check if syscall needs special handling
mov rax,qword ptr
mov r10d,eax
shr r10d,18h
; 3 = NtQueryInformationProcess
cmp r10b,3
jne doSyscall
; fix current process pseudo-handle, if it's there
cmp ecx,0FFFFFFFFh
jne @F
movsx rcx,cl
@@:
; is this ProcessDebugObjectHandle request?
cmp edx,1Eh
jne @F
; if so, fix buffer and size for ProcessDebugObjectHandle request.
; It should be 8-bytes long.
lea r10,
mov r8,r10
mov r9d,8
@@:
jmp doSyscall
doSyscall:
and eax,7FFFh
sub rsp,20h
call trampoline
jmp processResult
trampoline:
mov r10,rcx
syscall
ret
processResult:
; check for special handling again
mov r10d,dword ptr
shr r10d,18h
; 3 = NtQueryInformationProcess
cmp r10b,3
jne returnToX86
; is this ProcessDebugObjectHandle ?
cmp dword ptr ,1Eh
jne returnToX86
; were 2 buffers the same in original call?
mov ecx,dword ptr
cmp ecx,dword ptr
je @F
; if not, copy returned DebugObjectHandle back to original buffer
mov r10d,dword ptr
mov dword ptr ,r10d
@@:
jmp returnToX86
returnToX86:
nop
mov rsp,qword ptr
mov rbp,qword ptr
mov dword ptr ,eax
pop rbx
pop rdi
pop rsi
retf
You'll notice that x64 version is slightly more complex due to the way parameters are passed (registers vs. stack). It also includes a special treatment for 8 special edge cases - it will modify syscall parameters to adjust buffers and pointer sizes to satisfy requirements for 64-bit code.
NOTE - to keep code simple, I only showed the part which deals with NtQueryInformationProcess but other cases are similar.
As you can see, return back from x64 to the x86 world is a simple retf instruction. x86 code continues right where it left off:
00000000`00e7edf1 cb retf
...
00dabb9e 668cd1 mov cx,ss
00dabba1 668ed1 mov ss,cx
00dabba4 8b16 mov edx,dword ptr
00dabba6 3bf8 cmp edi,eax
00dabba8 80fbf2 cmp bl,0F2h
00dabbab 81c604000000 add esi,4
00dabbb1 6685c8 test ax,cx
00dabbb4 33d3 xor edx,ebx
00dabbb6 f5 cmc
00dabbb7 8d927cc2efb1 lea edx,
00dabbbd 6681ff8e5b cmp di,5B8Eh
00dabbc2 f7d2 not edx
00dabbc4 80fbf5 cmp bl,0F5h
00dabbc7 81f2ee613a76 xor edx,763a61ee
00dabbcd e9479f0500 jmp 00e05b19
00e05b19 4a dec edx
00e05b1a f8 clc
00e05b1b c1c202 rol edx,2
00e05b1e 33da xor ebx,edx
00e05b20 6681fff273 cmp di,73F2h
00e05b25 03fa add edi,edx
00e05b27 ffe7 jmp edi
...
00ddaeba 8b542500 mov edx,dword ptr
Instruction at address 0x00ddaeba is the same for both x86 and x64 OS-es and VM continues as usual.
Different protection modes and syscalls
I provided you with 2 real-world test executables. Oppo seems to be simpler and use just 3 syscalls:
NtQueryInformationProcess with ProcessDebugObjectHandle class
NtSetInformationThread with ThreadHideFromDebugger class
NtProtectVirtualMemory to set protection attributes for each section in original executable
Asshurt doesn't have antidebug trick with NtQueryInformationProcess but it uses additional syscalls for some purposes:
NtOpenFile
NtCreateSection
NtMapViewOfSection
NtQueryVirtualMemory
NtUnmapViewOfSection
NtClose
Suggested workaround
Since VMProtect is using undocumented Windows features, it somehow needs to ensure that the protection will work on each and every Windows version. That's VMProtect's biggest strength and also the biggest weakness.
Windows' syscall numbers change in each version and also between major builds. Use the wrong syscall number and you're guaranteed to receive unexpected results. So, VMProtect developers had to hardcode a table with Windows build numbers and corresponding syscall id's in the executable.
You can see the syscall numbers in the j00ru's page (slightly out of date) or in tinysec's windows kernel syscall table
To obtain Windows build number, VMProtect uses information from PEB (Process Environment Block). The method is already described in The MASM Forum, so I'll just reproduce the (ugly) code from their page:
print "Read From Process Environment Block:",13,10
ASSUME FS:Nothing
mov edx,fs: ;PEB.InheritedAddressSpace
ASSUMEFS:ERROR
mov eax, ;eax = Major Version
push eax
push edx
print ustr$(eax),'.'
pop edx
push edx
mov eax, ;eax = Minor Version
print ustr$(eax),'.'
pop edx
mov eax, ;eax = build
and eax,0FFFFh ;because win 7 collapses
print ustr$(eax),13,10,13,10
pop eax
VMProtect checks only the build number and picks the corresponding syscall number. However, if the build number is not in the internal database, it will not use direct syscall and fall back to standard protection. Bingo, problem solved - no need for ugly hacks like Xjun's SharpOD plugin!
Hint: VMProtect 3.1 doesn't support Windows 10 Creators Update (build number 15063).
Demo time
As promised, here is a download link for the test application: https://mediafire.com/?niqqbs0fqcq8n23
Note: it should support most common builds of Windows XP/7/8.1/10. Windows 2003/Vista and other rare systems are not supported!
If it shows "OK" message, you've hidden your debugger well. If it shows "Debugger detected", you have a problem.
Have fun!
kao.
EDIT: Updated download link for Oppo. Mediafire's antivirus tends to have plenty of False Positives..
From：https://lifeinhex.com/use-of-syscall-and-sysenter-in-vmprotect-3-1/ 本帖最后由 失业 于 2017-8-24 13:54 编辑
几天前，Xjun简要介绍了VMProtect 3.1的一个新功能 - 它使用直接系统调用来检查软件是否在调试器下运行。我决定自己快速看一眼，弄清楚它是如何工作的。
我会给你2个研究对象：
oppo_flash_tool.exe（https://mega.nz/#!ZgJzjQxR!cNEHMwM-jKnLVgPXf4OUupyk1DNt69FYB2rEfY-5AlA） - 以Xjun为例;
asshurt.dll（https://mediafire.com/?3xyc0ugc2hxervn） - 某些欺骗Roblox。我不在乎自己的作弊，只是刚刚启动了系统调用功能。它使用与Oppo不同的系统调用。
除此之外，我将提供一个非常简单的演示可执行文件，复制VMProtect保护的一部分，以免您浪费时间查看混淆代码。
作为32位操作系统中的调试器，您可以使用任何您喜欢的操作。在64位操作系统上，您真的需要使用WinDbg - 据我所知，这是唯一可以处理这些技巧的调试器。
32位操作系统
我们先来调试一下oppo_flash_tool.exe。首先，我们需要通过像IsDebuggerPresent和CheckRemoteDebuggerPresent这样的常规技巧。如果你正在阅读这篇文章，我相信你知道如何做到这一点。
稍后我们会到达这里：
记住这个条件跳转。它采用32位操作系统，而不是采用64位操作系统。
让我们先看看32位操作系统版本。现在VMProtect准备调用sysenter。
...
由于代码是模糊的，这里是一个清理版本。请注意，在其他应用中，可以使用不同的寄存器。
64位操作系统
这里变得有趣！微笑 64位Windows中的32位代码 不能使用sysenter指令。但是，像ReWolf几年前所描述的那样，可以在同一个过程中混合x86代码与x64代码。这正是VMProtect 3.1正在做的。
让我们回到条件跳转，看看64位操作系统会发生什么。不会采取跳跃：
远叫？上次我看到这是在16位Windows时代..
正如ReWolf的文章所述：
总结起来，对于在64位Windows上运行的每个进程（x86和x64）都分配了两个代码段：
cs = 0x23 - > x86模式
cs = 0x33 - > x64模式
所以，一旦你执行该通话，你将切换到一个64位的世界。WinDbg高兴地认识到，所有其他调试器只是误入歧途..
x64代码与x86代码完全相同 - 设置一个堆栈帧，设置寄存器，然后执行系统调用指令。清理和缩短版本如下：
你会注意到，由于传递参数（寄存器与堆栈）的方式，x64版本稍微复杂一些。它还包括对8个特殊边缘情况的特殊处理 - 它将修改系统调用参数以调整缓冲区和指针大小以满足64位代码的要求。
注意 - 为了保持代码简单，我只显示处理NtQueryInformationProcess的部分，但其他情况类似。
如你所见，从x64返回到x86世界是一个简单的retf指令。x86代码继续下去：
地址0x00ddaeba上的指令对于x86和x64操作系统都是一样的，并且VM像往常一样继续。
不同的保护模式和系统调用
我为您提供了两个真实世界的测试可执行文件。Oppo似乎更简单，只使用3个系统调用：
NtQueryInformationProcess with ProcessDebugObjectHandle类
NtSetInformationThread与ThreadHideFromDebugger类
NtProtectVirtualMemory为原始可执行文件中的每个部分设置保护属性
Asshurt没有与antidebug招NtQueryInformationProcess但它使用了一些额外的目的系统调用：
NtOpenFile
NtCreateSection
NtMapViewOfSection
NtQueryVirtualMemory
NtUnmapViewOfSection
NtClose
建议的解决方法
由于VMProtect正在使用未记录的Windows功能，所以它需要确保在每个Windows版本上保护功能。这是VMProtect最大的优势，也是最大的弱点。
Windows的系统调用数字在每个版本以及主要版本之间发生变化。使用错误的系统调用号码，您将获得意外的结果。所以，VMProtect开发人员必须用可执行文件中的Windows内部版本号和对应的系统调用ID来硬编码表。
您可以在j00ru的页面中看到系统调用号码（稍微过时）或tinysec的Windows内核系统调用表
要获取Windows版本号，VMProtect使用来自PEB（Process Environment Block）的信息。该方法已经在MASM论坛中描述过，所以我只是从他们的页面重现（丑陋的）代码：
VMProtect仅检查内部版本号并选择相应的系统调用号。但是，如果内部数据库不在内部数据库中，则不会使用直接的系统调用，而是回退到标准保护。宾果，问题解决 - 不需要像Xjun的SharpOD插件那样的丑陋的黑客！
提示：VMProtect 3.1不支持Windows 10创建者更新（版本号15063）。
演示时间
如所承诺的，这是测试应用程序的下载链接：https://mediafire.com/?niqqbs0fqcq8n23
注意：它应该支持大多数常见的Windows XP / 7 / 8.1 / 10版本。Windows 2003 / Vista等罕见系统不支持！
如果显示“确定”消息，您已经很好地隐藏了调试器。如果显示“检测到调试器”，您有问题。微笑
玩的开心！
考。 Can you speak Chinese,I don't know how to speak English. 看楼主签名，楼主也是电驴城的嘛 可怜我等26个字母都忘差不多的人士了 难道是外国人{:1_903:} 一直都用英文发帖，难怪帖总火不起来 我以为我来错地方了- Aa凛冬 发表于 2017-8-24 11:20
难道是外国人
是转帖的。 Thanks for sharing
页:
[1]
2