loader: enable the AMD CPU instruction patcher on macOS (#1014)

* loader: enable AMD CPU instruction patching on macOS

The guest instruction patcher was compiled out on macOS and runtimeLinker forced emulate_amd off,
so under Rosetta every SSE4a EXTRQ/INSERTQ took the SIGILL path (about 18 us each). Build the
patcher on all hosts and let --amd-cpu enable it on macOS like on Windows and Linux. The host
feature probe already reports no SSE4a under Rosetta, so EXTRQ becomes a native trampoline and the
remaining fallbacks keep using the x64 instruction emulator through the host exception handler.

* tests: macos_sse4a covers the guest instruction patcher on macOS

Patch EXTRQ/INSERTQ blobs in an RWX area with a trampoline page behind it, with the host features of
Rosetta (no SSE4a/RDPID/CLWB), and compare against the SIGILL emulator: EXTRQ must become a native
trampoline without SIGILL, INSERTQ stays a trap handled by the emulator, and nothing is touched
without emulate_amd.

* loader: emulate the patcher's VRSQRTPS trap and MWAITX on macOS

The AMD CPU patcher marks VRSQRTPS it cannot relocate by clearing a reserved
VEX.vvvv bit, which raises #UD on every host including Rosetta. The macOS build
left TryEmulateReciprocalSquareRoot and TryEmulateMonitorxMwaitx out, so the
guest died with an unhandled SIGILL at the first such site.

Both are plain SSE2/libc, so enable them on macOS. The Darwin context also
gains ClearUpperYmm so the VEX.128 result zeroes YMM[255:128] when the thread
has the AVX context. macos_sse4a covers the trapped VRSQRTPS and MWAITX.

* tests: macos_sse4a runs a plain VEX.128 VRSQRTPS through the patcher

The existing case started from an instruction that was already marked as a trap. The
new cases feed the ordinary encoding to PatchGuestInstructions and compare the result
with the emulated reference, both through a native trampoline and, when the instruction
cannot be relocated, through the trap mark and the signal emulator.

* loader: compact macOS instruction patcher coverage

Share the emulator dispatch chain while keeping SHA emulation limited to non-Apple hosts and preserving handler order. Remove a duplicate VRSQRTPS signal test already covered by both native and trapped patcher cases, plus blank lines left by removed platform guards.

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
This commit is contained in:
Bartłomiej Wołyńczykandnmzik authored and GitHub committed 2026-10-06 21:55:11 +04:00
1 parent 4852721005
commit 35c6e8fff6
5 files changed
+244 -43

No files matched your search

+2 -1
View File
@@ -719,8 +719,9 @@ if(APPLE AND (CMAKE_OSX_ARCHITECTURES STREQUAL "x86_64" OR
add_executable(macos_sse4a_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/macosSse4aTests.cpp"
"${KYTY_SOURCE_DIR}/loader/x64InstructionEmulator.cpp"
"${KYTY_SOURCE_DIR}/loader/guestInstructionPatcher.cpp"
)
target_link_libraries(macos_sse4a_tests common fmt::fmt Zydis::Zydis)
target_link_libraries(macos_sse4a_tests common fmt::fmt Zydis::Zydis xbyak::xbyak)
target_include_directories(macos_sse4a_tests PRIVATE ${inc_headers})
if(BUILD_TESTING)
add_test(NAME macos_sse4a COMMAND $<TARGET_FILE:macos_sse4a_tests>)