Compare commits

...
Author SHA1 Message Date
feeced81d2 window: auto-hide the cursor over the game window (#958)
* window: auto-hide the cursor over the game window

Hide the cursor when the game window is created and whenever the mouse
has been idle for two seconds; any mouse motion, button or wheel event
shows it again. Games that draw their own cursor (e.g. Blue Prince) no
longer show the host pointer on top of it.

Only real mouse input counts as activity: synthetic touch/pen mouse
events and zero-delta motion (sent by some platforms on focus or cursor
visibility changes) are ignored. The event loop only waits with a
timeout while the cursor is visible, so it still blocks indefinitely
once the cursor is hidden. The cursor is restored on shutdown.

Uses SDL_HideCursor/SDL_ShowCursor, which SDL implements for X11,
Wayland, Cocoa and Win32 and which only affect the cursor over the
emulator window.

Closes #282

* window: make cursor auto-hide optional in CLI and launcher

Co-authored-by: Anatolii Kucheruk <anatolii.kucheruk01@gmail.com>

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
Co-authored-by: Anatolii Kucheruk <anatolii.kucheruk01@gmail.com>
2026-10-03 18:15:15 +04:00
820856b301 launcher: dualsense vibration/volume setting (#973)
* pad: send DualSense speaker and haptic audio over Bluetooth

* ci: fix Opus clang-cl build and macOS memory tracker test

* docs: remove Bluetooth notes from project README

* pad: keep Bluetooth audio work off emulation timing path

* pad: smooth DualSense speaker audio on USB and Bluetooth

* tests: find available macOS guest range for page manager

* pad: keep haptic waveform when Bluetooth queue overflows

* audio: reject zero output sample frequency

* Simplify DualSense audio routing and fix Bluetooth batching

* controller: add global DualSense volume and vibration controls

* Simplify global controller settings

---------

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
2026-10-03 18:02:59 +04:00
nmzik 45f55413e0 Keep sign-in dialogs active until user dismissal through shared system presentation 2026-10-03 17:31:16 +04:00
nmzik c3330caba2 Report NP Web API memory pool statistics for active library contexts 2026-10-03 17:31:16 +04:00
nmzik 92c22d60ef Support periodic kernel timers through shared event queue deadlines 2026-10-03 17:31:15 +04:00
nmzik a0858abe35 Implement batch AIO completion polling and deletion with per-request results 2026-10-03 17:31:15 +04:00
nmzik 5766b241c3 Implement POSIX thread stack attribute import through kernel setter 2026-10-03 17:31:15 +04:00
nmzik 7f158c656e Share sampler descriptor snapshots across native variants 2026-10-03 17:31:15 +04:00
nmzik 8b0cc79acf Preserve native slices when PS5 volume texture depth grows 2026-10-03 17:31:15 +04:00
Nawid3333andClaude Opus 5.5 8cbee7f4b7 cmake: raise the bundled-dependency policy floor to 3.10 (#975)
CMake 4 warns "Compatibility with CMake < 3.10 will be removed" for every
fetched project that declares an older minimum. zydis and its bundled
zycore both declare cmake_minimum_required(VERSION 3.9), so each configure
prints two deprecation warnings. The existing floor of 3.5 does not lift
them past the threshold; 3.10 does.

Existing build directories keep the cached 3.5 until reconfigured with
-U CMAKE_POLICY_VERSION_MINIMUM or a fresh cache.

Tested on Windows 11, clang-cl 19.1.5, CMake 4.4.3: configure prints 0
deprecation warnings (2 with the old floor); virtual_memory_allocation_tests
builds and links zydis/xbyak; ctest guest_red_zone_patcher passes.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 17:24:03 +04:00
rocketrobot99andnmzik 52c8ff18d0 Update gpu_format.cpp (#1003)
* Update gpu_format.cpp

Fix SRGB render-target formats (k8Srgb, k8_8Srgb)

Corrects render_target_bytes_per_element for SRGB formats:
- k8Srgb: 1 byte (was 0)
- k8_8Srgb: 2 bytes (was 0)

These formats appear in ChannelLayout::k8 and k8_8 render-target paths.
Incorrect RT byte sizes caused TextureCache render-target validation failures
in Lucky's Tale

* Fix narrow sRGB render-target format tests

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-10-03 16:57:59 +04:00
nmzik f53e5d2283 fix windows ci (#998)
* fix windows ci

* Fix Windows SPIR-V gather opcode narrowing conversion
2026-10-03 15:05:20 +04:00
bd611bfe15 Honor explicit gather LOD with shared image mip selection
Co-authored-by: YuhiAida <21134775+YuhiAida@users.noreply.github.com>
Co-authored-by: BlackCAT <52720489+psnwd@users.noreply.github.com>
Co-authored-by: shadowbeat070 <70843316+shadowbeat070@users.noreply.github.com>
Co-authored-by: Senaxx <michelbeld@gmail.com>
Co-authored-by: Mac <me@itsmemac.com>
Co-authored-by: Ali Almohaya <AliAlmohaya@gmail.com>
2026-10-03 14:19:33 +04:00
chenxiao 0bb2bdad17 Skip clean memory tracker ranges during buffer synchronization 2026-10-03 14:19:33 +04:00
nmzik 0ec3655f1c Clear single-mip color targets reused from cached mip chains 2026-10-03 14:19:33 +04:00
nmzik 190608ae20 Publish consumed color metadata without invalidating pooled images 2026-10-03 14:19:32 +04:00
nmzik ebe401d052 Preserve image contents across unrelated buffer alias writes 2026-10-03 14:19:32 +04:00
BlackCAT 0d4f993358 Keep reused BC5 textures in their own guest layout 2026-10-03 14:19:32 +04:00
BlackCATandshadowbeat070 ef4c5baf3a Implement RDNA2 64-bit image atomic maximum
Co-authored-by: shadowbeat070 <70843316+shadowbeat070@users.noreply.github.com>
2026-10-03 14:19:32 +04:00
Hamza Shakir e6cb880adb Clamp compute LDS to the device limit and ignore unopenable gamepads 2026-10-03 04:27:38 +04:00
nmzik 8fa116eeb1 Remove redundant per-dispatch checks and buffer resets 2026-10-03 04:22:39 +04:00
nmzik e317465d8a fix linux CI (#972)
* fix linux CI

* Fix indirect image initialization in materialization tests
2026-10-02 17:20:12 +04:00
nmzik 684330c9e4 Specialize shader resources in place and fuse buffer refresh work 2026-10-02 16:16:31 +04:00
nmzik f3dc5ee4a9 Collect shader interface metadata in place without copying resources 2026-10-02 16:07:16 +04:00
nmzik 2545723065 Stub NetResolverAbort import AzqoBha7js4 with success 2026-10-02 15:49:55 +04:00
nmzik 95c89229ad Remove dead shader instruction cycles with linear liveness traversal 2026-10-02 15:49:43 +04:00
nmzik 7804eef9b6 Lower finite GPU-selected image descriptors through shared indirect bindings 2026-10-02 15:37:17 +04:00
82a2f4ad42 controller: cycle DualSense speaker volume, vibration and trigger intensity in game (#946)
* controller: cycle DualSense speaker volume, vibration and trigger intensity in game

Three new input mapping controls cycle the PS5's controller settings in
game (default keys 1, 2 and 3): speaker volume (100% -> off -> 2% -> 12%
-> 42%), vibration intensity and trigger effect intensity (strong -> off
-> weak -> medium). Each starts at full and is not saved.

Changing vibration or trigger intensity re-sends the game's last rumble
and trigger effect, so the change is felt immediately.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Clean up controller settings test captures and formatting

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
2026-10-02 15:11:34 +04:00
e1be2ead54 DualSense haptics and controller speaker sound bluethooth (#969)
* pad: send DualSense speaker and haptic audio over Bluetooth

* ci: fix Opus clang-cl build and macOS memory tracker test

* docs: remove Bluetooth notes from project README

* pad: keep Bluetooth audio work off emulation timing path

* pad: smooth DualSense speaker audio on USB and Bluetooth

* tests: find available macOS guest range for page manager

* pad: keep haptic waveform when Bluetooth queue overflows

* audio: reject zero output sample frequency

* Simplify DualSense audio routing and fix Bluetooth batching

---------

Co-authored-by: Codex <codex@openai.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
2026-10-02 14:55:00 +04:00
nomolao2-cellandnmzik 89792a3bca Improve Hades II compatibility (#964)
Fix Hades II startup and color clears

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-10-02 14:37:49 +04:00
nmzik 39e23ac92e Publish synchronous buffer downloads inline after ordered completion 2026-10-02 13:24:03 +04:00
nmzik a67a033ff8 Remove redundant tracking lock checks from the hot path 2026-10-02 13:24:03 +04:00
nmzik d1c99bbfb0 Consolidate instruction operand bounds checks 2026-10-02 13:24:03 +04:00
nmzik a9620f98f4 Query dirty region bits without constructing temporary snapshots 2026-10-02 13:24:03 +04:00
nmzik 28f29b8e6f Inline common IR operands and share storage with Phi pairs 2026-10-02 13:24:03 +04:00
nmzik 3353c1ff3b shader: pack the runtime evaluation index beside the IR opcode 2026-10-02 13:24:03 +04:00
Mr.Hakkuhandnmzik 70a475ec8c audio: fix popping and time-stretching in output (#966)
fix audio accumulated timing drift

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-10-02 13:22:38 +04:00
Louisandnmzik b3e419ff11 shader: implement 32-bit RDNA2 IMAGE_ATOMIC_CMPSWAP (#952)
* shader: implement 32-bit RDNA2 image atomic compare-and-swap

* shader: share atomic emission and correct image barriers

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-10-02 01:15:53 +04:00
nmzik 4428640b44 add one time log for BDA 2026-10-02 00:53:51 +04:00
nmzik 2b2915e271 Reuse SSA read traversal storage within each pass 2026-10-02 00:53:51 +04:00
nmzik 7101741637 Cache SPIR-V arithmetic types lazily in existing emitters 2026-10-02 00:53:51 +04:00
nmzik b86cf0a820 Reuse pipelines when inactive blend settings change 2026-10-02 00:52:49 +04:00
nmzik f29bd0fe35 Reuse whole-wave ballot and first-lane SPIR-V results 2026-10-02 00:51:15 +04:00
nmzik 64eea6b32d Reuse vertex shaders across equivalent vertex buffer layouts 2026-10-02 00:51:15 +04:00
nmzik 706eec3ca9 add tests 2026-10-02 00:51:15 +04:00
nmzik 35759e48cf shader: expose wave32 feature guards through ballot mask simplification 2026-10-02 00:51:15 +04:00
nmzik 6409be2874 shader: respect branch reachability when refreshing scalar resources 2026-10-02 00:51:15 +04:00
Senaxx 0d0b215fab shader: port 64-bit ALU and atomic opcodes 2026-10-02 00:51:15 +04:00
nmzik 11001c97f7 shader: add distinct mesh shader type 2026-10-02 00:51:15 +04:00
337f432016 shader: execute PS5 BVH intersections and local FLAT stacks
Co-authored-by: Ali Almohaya <AliAlmohaya@gmail.com>
Co-authored-by: Mac <me@itsmemac.com>
Co-authored-by: Senaxx <michelbeld@gmail.com>
Co-authored-by: Jean-Philippe <github@meunier.jp>
2026-10-02 00:51:15 +04:00
581dad7dd3 shader: implement RDNA2 FP64 equality comparisons
Co-authored-by: Senaxx <michelbeld@gmail.com>
Co-authored-by: Ali Almohaya <AliAlmohaya@gmail.com>
2026-10-02 00:51:15 +04:00
64dcbe0ea1 shader: decode the combined conditional-debug branch
Co-authored-by: Mac <me@itsmemac.com>
Co-authored-by: Senaxx <michelbeld@gmail.com>
Co-authored-by: BlackCAT <52720489+psnwd@users.noreply.github.com>
2026-10-02 00:51:15 +04:00
0ecd32299c shader: cache registered code identities instead of parsing legacy footers
Co-authored-by: Mac <me@itsmemac.com>
Co-authored-by: Senaxx <michelbeld@gmail.com>
2026-10-02 00:51:15 +04:00
nmzik c2f8e64f6d graphics: reuse texture conversion scratch buffers across uploads 2026-10-02 00:51:15 +04:00
44ba657573 renderer: validate depth swizzles by available channels
Co-authored-by: Mac <me@itsmemac.com>
Co-authored-by: Senaxx <michelbeld@gmail.com>
Co-authored-by: Ali Almohaya <AliAlmohaya@gmail.com>
2026-10-02 00:51:15 +04:00
Aryan Sharmaandnmzik 44798085bd network: handle SO_NBIO on non-Windows hosts (#954)
* network: handle SO_NBIO on non-Windows hosts

Setsockopt only translated the guest SOL_SOCKET/SO_NBIO (0x1200) option
under _WIN32; other hosts rejected it with ENOPROTOOPT. FMOD Studio's Live
Update listener sets SO_NBIO, so on Linux its socket setup failed and the
Studio async manager latched FMOD_ERR_NET_CONNECT. Every blocking Studio
call then failed: loadBankFile returned FMOD_ERR_NOTREADY for all banks,
leaving the game silent and stuck on the splash screen waiting for
RuntimeManager.HaveAllBanksLoaded (PPSA28997).

Toggle O_NONBLOCK with fcntl on non-Windows hosts.

* network: map connection errors from POSIX socket calls

ConvertHostSocketError only covered a handful of errno values on
non-Windows hosts, so anything else reached the guest as EIO. With
SO_NBIO now honoured there, a nonblocking connect() that is still in
progress returned EIO instead of EINPROGRESS (and EALREADY on retry).

Map the same connection and network errors the Windows branch already
translates.

* network: share nonblocking socket error handling

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-10-01 17:54:21 +04:00
Ewen trividicandnmzik b7a1fac898 input: correct gyro orientation drift with the accelerometer (#899)
* input: correct gyro orientation drift with the accelerometer

* input: simplify gyro drift correction and reset gravity on reconnect

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-30 22:40:53 +04:00
nmzik bc193f5272 shader: V_CMPX_LT_I16 + fix SDWA with SEXT 2026-09-30 20:49:20 +04:00
nmzik 17100ec968 shareplay: return initialized connection information 2026-09-30 19:56:25 +04:00
Senaxx 8ba2caf58a videoout: WQHD detection and VRR peg/unpeg 2026-09-30 17:45:51 +04:00
Senaxx 48b52d31b5 libs: libScePngEnc on stb_image_write 2026-09-30 17:06:41 +04:00
nmzik 840b9f5753 Fix unnecessary queue drains in predication and suspend points 2026-09-30 16:51:40 +04:00
Pyxellatorandm66401970-eng 05057c9441 launcher: configure and preview DualSense lightbar color (#912)
launcher: configure and preview controller lightbar color

Add an optional lightbar color to launcher settings and pass it to the emulator with --controller-color. Preview changes on RGB gamepads, handle controller hotplug events, and restore the saved color when settings dialogs close. Release launcher preview handles when starting a game.

Co-authored-by: m66401970-eng <m66401970@gmail.com>
2026-09-30 00:46:44 +04:00
af4edc2865 window: explain why each GPU was rejected in device selection (#910)
* window: explain why each GPU was rejected in device selection

"Could not find suitable device" gave no hint of what was missing; the
reasons only went to the log file, which users often cannot find (#489).

Device selection now records each rejection reason and the fatal error
lists every GPU with its reasons, for example:

  Could not find suitable device:
    AMD BC-250 (RADV GFX1013): fragmentShaderBarycentric is not supported

The missing-extension check also names each missing extension instead of
stopping at the first one without saying which. Device requirements and
selection are unchanged; the log keeps the same messages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* window: simplify GPU rejection reporting and capability checks

* window: identify storage-image format failures accurately

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-30 00:08:19 +04:00
Kody Schramandnmzik 9d4da48320 Linux: Strip debug symbols from launcher (#913)
* Add stripping for linux launcher build

* Fix launcher strip

* Removed garbage

* Fix launcher link option formatting

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-29 23:50:20 +04:00
nmzik 2650478d92 profiling: preserve emulator fault handlers when Tracy connects 2026-09-29 21:56:38 +04:00
nmzikandMac 18a5f04d05 graphics: retain oversized readbacks through GPU completion
Co-authored-by: Mac <me@itsmemac.com>
2026-09-29 21:56:38 +04:00
nmzikandMac da9b38af2b ampr: reuse file-read staging storage on the APR worker
Co-authored-by: Mac <me@itsmemac.com>
2026-09-29 21:56:38 +04:00
nmzikandMac c045f08904 ampr: schedule address waits without blocking other priorities
Co-authored-by: Mac <me@itsmemac.com>
2026-09-29 21:56:38 +04:00
nmzik 5bf9218b32 ampr: keep commands in one append-ordered sequence 2026-09-29 21:56:38 +04:00
nmzik f9fd400dbb memory: validate sparse uploads against the guest address-space owner 2026-09-29 21:56:38 +04:00
nmzik 650dc1e0af graphics: upload streamed texture mips from sparse reservations 2026-09-29 21:56:15 +04:00
EmK530 59a1760427 sampler: clamp mipLodBias to not be greater than maxSamplerLodBias 2026-09-29 20:06:22 +04:00
nmzikandEmK530 73615c31ff graphics: preserve logical alpha when blending reordered targets (PPSA02721)
Co-authored-by: EmK530 <admin@emk530.net>
2026-09-29 19:52:40 +04:00
nmzik 6799ecbc5b shader: add native 64-bit LDS OR with shared typed storage views 2026-09-29 19:13:33 +04:00
nmzik 35c4dca2ce shader: add V_CMPX_NLE_F16 opcode 2026-09-29 19:13:33 +04:00
nmzik 6f24b031f5 kernel: block short sleep requests instead of busy waiting 2026-09-29 16:49:09 +04:00
nmzik 0c844e4bc2 profiler: use native Tracy scopes and remove unused manual lifetime tracking 2026-09-29 13:57:32 +04:00
nmzik e57c37fa9c graphics: consume vertex semantics directly from shader metadata 2026-09-29 13:57:32 +04:00
nmzik 6d6666819a graphics: serialize pipeline cache through its renderer owner 2026-09-29 13:57:32 +04:00
nmzik 0fbeac6d3f graphics: compact shader keys and hash packed pipeline state 2026-09-29 13:57:32 +04:00
nmzik 5e4f6afc30 graphics: build color target descriptors in place 2026-09-29 13:57:32 +04:00
nmzik 37501b5a78 graphics: stream clean guest buffers without backing-store lookups 2026-09-29 13:57:32 +04:00
nmzik b43bf9f7fb graphics: store vertex attribute bindings without duplicated arrays 2026-09-29 13:57:32 +04:00
nmzik 8e42e99788 graphics: batch release labels without redundant submissions 2026-09-29 13:57:32 +04:00
João Lobo 539c0f7bf8 graphics: restore block image storage usage where the device supports it (#894) 2026-09-29 01:57:25 +04:00
GodsWill Devandnmzik 62a0222e24 tests: cover 1D-array color render targets (#377)
* tests: cover 1D-array color render targets

* tests: build shader regressions on Windows CI

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-29 01:48:56 +04:00
Saif Malikandnmzik 6f241bef49 Fix macOS aggregate test build (#536)
* tests: add Darwin memory test support

* tests: fix x86_64 aggregate test build

* tests: verify macOS memory and shader tests

* tests: validate macOS death-test setup before fork

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-29 01:43:28 +04:00
Matthew HurleyandMatthew Hurley 21d7e8e1f0 tests: run the saveexec mask cases in a populated subgroup (#876)
tests: run saveexec masks in a populated subgroup

Co-authored-by: Matthew Hurley <171001439+theantipopau@users.noreply.github.com>
2026-09-29 01:42:11 +04:00
Matthew HurleyandMatthew Hurley 49db3ea54c tests: skip rasterization cases instead of aborting the suite on limited GPUs (#875)
tests: skip rasterization cases on limited GPUs

Co-authored-by: Matthew Hurley <171001439+theantipopau@users.noreply.github.com>
2026-09-29 01:41:37 +04:00
Aspenandnmzik 4b0de1b672 Zarchive Loading (#724)
* Simplify archive loading and harden filesystem integration

* Restore directory entry name length assertion

* Keep graphics-dependent filesystem tests out of CI

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-29 00:06:44 +04:00
nmzik 8e61798bfd graphics: honor CPU ownership in image dirty-region queries 2026-09-28 22:17:43 +04:00
nmzik 6ee1bf9db0 graphics: trust image-local readback validity 2026-09-28 22:17:43 +04:00
nmzik 0471c9991c audio: implement AudioOut2 user capability queries 2026-09-28 22:17:43 +04:00
nmzik b10bd53dd3 audio: prevent ATRAC9 vibration configuration heap corruption 2026-09-28 22:17:43 +04:00
nmzik 4cf456fb77 graphics: retain buffer names across draw bindings 2026-09-28 22:17:43 +04:00
Devran Cosmo Uenalandnmzik 22ff4693c5 audio: play pad speaker ports on the DualSense speaker (#868)
* audio: play pad speaker ports on the DualSense speaker

Route the right channel to the speaker with the audio-control values the Linux hid-playstation
driver uses without headphones, and fall back to the main output while no DualSense audio device
is active. Also stop the previous DualSense's rumble when another DualSense becomes active.

* audio: harden DualSense speaker routing and fallback

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-28 20:00:58 +04:00
nmzik a107d1e333 filesystem: keep unmapped guest paths out of the host namespace 2026-09-28 19:17:03 +04:00
05f92b6422 libCes CP932 fix for Windows/macOS, fix SaveDataDialog lockups (#878)
* libs: fix CP932 iconv conversion on Windows and macOS

SDL3's SDL_SYSTEM_ICONV build option defaults off on Windows and macOS,
so SDL falls back to its own bundled iconv there, which only implements
ASCII/Latin-1/UTF/UCS. SDL_iconv_open("UTF-8", "CP932") always failed as
a result, crashing games that convert Shift-JIS strings via
sceCesMbcsStrToUtf8Str (e.g. save-data dialogs).

Add a small CesIconv shim selected at compile time: Windows converts one
character at a time via the Win32 codepage-932 APIs, replicating
SDL_iconv's chunked semantics and error codes; macOS calls real POSIX
iconv directly, since libiconv is always present there and does support
CP932, unlike SDL's fallback. Linux is untouched, since SDL already
links real system iconv by default there.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* libs: fix SaveDataDialog infinite loop on save

SaveDataDialogOpen() jumped straight from SAVE_STATUS_INITIALIZED (1) to
SAVE_STATUS_FINISHED (3), skipping SAVE_STATUS_RUNNING (2) entirely. At
least one title's save-write dialog stage only accepts a FINISHED status
after observing a RUNNING poll first; without it, the title assumed the
Open() request never took effect and re-issued it forever, hanging the
save flow (this only became visible once the CP932 crash above stopped
masking it).

Open() now starts the dialog as RUNNING. UpdateStatus() advances the
state machine, reporting RUNNING once before flipping to FINISHED for
subsequent polls; GetStatus() is untouched, staying a pure peek with no
side effects, matching the real SDK's pump-vs-peek split between the two
calls.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* Add iconv linking to macOS

* Fix missing macOS iconv link for tests

* Fix CP932 edge cases and save dialog status consistency

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-28 18:42:22 +04:00
nmzik a892097ff0 tests: fix macOS extended-memory alias regression check (#889)
tests: use a macOS-owned address for extended memory alias checks
2026-09-28 18:33:46 +04:00
nmzik c8baa7b909 shader: derive immutable resource policy once per plan 2026-09-28 18:01:49 +04:00
nmzik 34a3eb6b58 memory: recycle automatic backing through physical ownership 2026-09-28 18:01:49 +04:00
nmzik a20b64d3e5 audio: rewind compressed sampler blocks at loop boundaries 2026-09-28 18:01:49 +04:00
nmzik 2016f8e2fe gpu: address extended guest memory without widening dense cache tables 2026-09-28 18:01:49 +04:00
nmzik c91fe80d64 memory: separate extended allocations from ordinary guest mappings 2026-09-28 18:01:49 +04:00
nmzik ab2c1c4c03 Update README screenshot to UFC 5 2026-09-28 18:01:49 +04:00
gohansharma79-svgandnmzik 0a5774be7b shader: support FP64 conversions used by FighterZ (#884)
shader: support float and unsigned widening to FP64

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-28 17:39:31 +04:00
nmzik 0791f921f6 UFC 5 in-game (#883)
* shader: release descriptor dependencies

* shader: simplify shared-exit control flow

* shader: fix scalar memory loads

* shader: fix LDS counter bounds

* renderer: skip inactive pixel shaders

* shader: respect FP32 mode in comparisons

* renderer: fix sampler border colors

* shader: resolve descriptors from control flow

* shader: resolve descriptors before resource planning

* shader: handle scalar guards for image bounds

* shader: handle reduced guards for image loops

* shader: fix wave guards and material indices

* shader: preserve scalar buffer offset alignment and carry

* shader: prove masked loop bounds across iteration transitions

* shader: preserve source and IR before resource tracking failures

* shader: track material keys selected from active lanes

* shader: remove obsolete post-dominator analysis

* shader: follow bounds guards through control-flow joins
2026-09-28 16:03:05 +04:00
nmzik 421684e75f docs: add weekly updates and Discord links to README (#866)
* docs: link weekly updates from README

* docs: add Discord development channel

* docs: use Discord server invite link

* docs: refine weekly updates wording
2026-09-27 18:06:26 +04:00
Elijaandnmzik 025b39c018 Fix empty query handling in HttpUriParse (#764)
* Fix empty query handling in HttpUriParse

* Handle empty URI query consistently

* Cover empty URI query handling

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-27 16:07:03 +04:00
João Loboandnmzik 7c8583d204 audio: implement the AJM Opus decoder (#858)
* audio: implement the AJM Opus decoder

* Audio: simplify Opus decoding and preserve packet boundaries

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-27 15:53:20 +04:00
9229c04ebf samplerCache: handle reserved sampler anisotropy ratios
Co-authored-by: EmK530 <admin@emk530.net>
Co-authored-by: Hamza Shakir <60213466+hamzashakir99@users.noreply.github.com>
2026-09-27 15:46:25 +04:00
Matthew Hurleyandnmzik 1b6af301b5 tests: wire dormant VOP3 lane-read check and guard new dormant tests (#840)
Tests: enable lane-read coverage and clarify layout failures

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-27 15:37:32 +04:00
nmzik e51f252b75 AvPlayer: preserve clip timestamps across playback loops 2026-09-27 15:23:36 +04:00
nmzik 0f1ff37f23 Graphics: materialize CMASK clears before publishing expanded metadata 2026-09-27 15:05:46 +04:00
nmzik da33a1c4a3 Graphics: respect physical tile layout when resolving depth aliases 2026-09-27 15:05:46 +04:00
nmzik f53f0ca483 Graphics: preserve rendered contents across 1D and 2D texture aliases 2026-09-27 15:05:46 +04:00
nmzik 7b6bfdec64 Graphics: share canonical padded layouts for single-sample color targets 2026-09-27 15:05:46 +04:00
nmzik a88181b897 Graphics: share texture layout for linear render-target mip chains 2026-09-27 11:10:02 +04:00
nmzik 041874f663 Graphics: resolve equivalent terminal mip views without expanding backing 2026-09-27 11:04:19 +04:00
nmzik 496432bbb4 Shader: skip identity image remapping on materialization hits 2026-09-27 10:47:18 +04:00
Pcniadoandnmzik bec67b7863 Fix HITMAN 3 boot, layered presentation, and socket imports (#851)
* videoout: synchronize layered flips with premultiplied composition

* shader: support unsigned FFBH subword sources

* shader: apply cube-face result scaling

* shader: prune buffer reads proven out of bounds

* net: implement send and receive exports with shared socket handling

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-27 02:13:30 +04:00
Devran Cosmo Uenalandnmzik a22a9d9348 audio: play DualSense haptics from vibration audio ports (#843)
* audio: play DualSense haptics from vibration audio ports

Games drive the DualSense's actuators through a vibration audio port, one channel per actuator.
These ports had no device, so their samples were dropped. Over USB the DualSense is a 4-channel
audio device whose back channels feed the actuators; play the port's samples there through SDL.

The device is looked up while the port plays, so the controller can be connected at any time. The
queue keeps about the latency of the regular audio path, with the playback rate nudged by up to 3%
when the port is fed off real time.

While haptic audio plays, hold a DualSense's rumble at zero: SDL disables audio haptics whenever
rumble is non-zero.

Add pad_haptics tests for the sample conversion, device-name matching and rate correction.

* Simplify DualSense audio haptics and restore idle rumble

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-27 00:37:27 +04:00
nmzik 81f312638f graphics: encode packed UNorm buffer stores before writing skinning normals 2026-09-26 23:42:19 +04:00
nmzik f1f930e06c fix(launcher): use native macOS game folder picker 2026-09-26 22:14:09 +04:00
Devran Cosmo Uenalandnmzik e0b682e4b2 AvPlayer: fix Crash Bandicoot 4 intro playback and green video padding (#848)
* AvPlayer: advance the shared audio-master clock after final audio frame

* AvPlayer: initialize padded NV12 planes with neutral black

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-26 20:39:46 +04:00
nmzik 16b83a034c Shader: execute RDNA2 FP64 shadow arithmetic and log denorm limits
Shader: disable FP64 denorm preservation controls for now
2026-09-26 15:15:35 +04:00
cf268af839 Shader: implement RDNA2 image atomic float maximum
Shader: implement RDNA2 image atomic float minimum

Co-authored-by: foufouadi <attar.foued@gmail.com>
Co-authored-by: frans <francois.att@gmail.com>
Co-authored-by: fxpw <fxpw@outlook.com>
Co-authored-by: Matthew Hurley <171001439+theantipopau@users.noreply.github.com>
2026-09-26 15:15:35 +04:00
Mehmet Cambaz fd2e15ee73 ci: run the recompiler unit tests (#447)
Build and run the CPU-only resource_tracking, scalar_provenance, resource_materialization, and shader_vertex_metadata tests on Windows, macOS, and Linux. Each test executable is excluded from the default build, so add it to the explicit CMake targets and the matching CTest selection. Leave the Vulkan-dependent shader_recompiler_compute suite for a separate CI setup.
2026-09-26 03:00:01 +04:00
Saadiandnmzik 4e8d8c849d tests: add LeastRecentlyUsedCache regression tests (#671)
* launcher: keep primary update feed answer when the fallback request fails

When the primary feed is reachable but its tag differs from the build tag,
the result was discarded and the GitHub API fallback was consulted. If the
fallback request failed (api.github.com rate-limits unauthenticated
requests), the launcher reported an error even though the primary feed had
already returned a usable update answer with a working release URL. Keep
the primary feed answer and show it if the fallback fails.

* emulator: validate numeric CLI arguments

--screen-width, --screen-height, --gpu, and --vblank-frequency were parsed
with Common::ToInt32/ToInt64, which ignores std::from_chars errors and
returns 0, so values like "--screen-width abc" or "--gpu xyz" were
silently accepted: zero screen size, or GPU 0 selected instead of auto
(-1). Reject invalid values with an error message like the other options,
using from_chars-based helpers (ParseUint32/ParseInt32); reject zero
screen dimensions.

* tests: add LeastRecentlyUsedCache regression tests

Cover the LRU cache walk order, tick filtering in ForEachItemBelow,
early-exit boolean callbacks, Touch reordering and same-tick skips,
Free, and freed-id reuse by later inserts. Modeled on BitArrayTests.

* tests: use a tick cutoff that detects a stale newest-item tick

Collect(cache, 35) also visits c when Touch(c, 35) failed to update
its tick (30 <= 35). Use a cutoff of 30 and expect only b.

* Run LRU cache tests in CI

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-26 02:42:31 +04:00
Matthew Hurleyandnmzik 1e438136fa Build save_data_memory_tests from the kyty_tests aggregate (#798)
Build save_data_memory_tests with kyty_tests

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-26 02:39:20 +04:00
Emilandnmzik 313b60e922 Capture PCM at set time when multiple AudioOut2 ports share a buffer (#826)
* capture PCM at set time when ports share a buffer

* Snapshot size limit and open device check

* Fall back to PCM pointer on stale snapshot

* audio: own AudioOut2 PCM at attribute set time

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-26 02:19:49 +04:00
Hamza Shakir cda236dbb3 Shader recompiler: accept V_NOT_B32 SDWA partial dst with any source; allow 64 buffers
SDWA source extraction and destination insertion are independent. Cover the reported word1-to-word0 preserve form and byte-source partial destinations in compute tests. Verify 64 distinct compute buffers survive resource tracking and binding layout, and remove the obsolete CFG rejection.

(cherry picked from commit 9e74df7dc0)
2026-09-26 01:58:55 +04:00
1216fa36cf docs: fix stale README notes, document building without Qt (#812)
* docs: fix stale README notes, document building without Qt

- Use the current issue template name, Game Emulation Status Report
  (renamed in 027433f).
- Raise the documented Windows CMake minimum from 3.12 to 3.22.1, which
  the bundled SPIRV-Tools requires.
- Note that -DKYTY_BUILD_LAUNCHER=OFF builds kyty_emulator and the
  regression tests without Qt (#193).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* docs: add Qt-free Linux build commands

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-26 01:50:06 +04:00
nmzik 497166009c loader: unlink exact module imports before releasing their mapping 2026-09-25 21:15:41 +04:00
nmzik 0fb3ac5605 kernel: retain modules until their final explicit unload 2026-09-25 21:15:41 +04:00
nmzik e056d55dd5 loader: consolidate eager symbol resolution and remove dead PLT machinery 2026-09-25 21:15:41 +04:00
nmzik 1371219f5c loader: preserve resolved relocations across dynamic loads 2026-09-25 21:15:41 +04:00
nmzik cf423df624 shader: remove per-draw vertex diagnostic atomics 2026-09-25 21:15:41 +04:00
nmzik 5ce4f08364 audio: improve NGS2 stream and ATRAC9 playback 2026-09-25 18:26:08 +04:00
nmzik 7e1c2d1532 avplayer: auto-start playback without an event callback 2026-09-25 16:39:54 +04:00
nmzik 0ec43a8d3d graphics: name Vulkan images and views only on creation 2026-09-25 16:15:05 +04:00
nmzikandMac bfdbba3aa4 graphics: share EOP clock and immediate write handling
Co-authored-by: Mac <me@itsmemac.com>
2026-09-25 16:14:40 +04:00
Devran Cosmo Uenalandnmzik f7ae6e0a83 audio: support NGS2 PCM16 samplers and rear low-pass filters (#825)
* audio: support standard NGS2 PCM16 sampler playback

Extend the existing fixed-point PCM path to standard samplers, loops, and progress reporting. Apply voice events immediately and honor callback interruption during source advancement.

Reuse upstream rate and pitch handling. Add regression coverage for streaming, routing, callbacks, loops, and custom sampler controls; ngs2_sampler passes.

* audio: apply NGS2 rear low-pass filters

Handle rear low-pass FCQ controls at the output sample rate with per-channel history and channel masks. Preserve filter tails through streaming gaps and use the standard pi constant for coefficients.

Add regression coverage for frequency response, control layout, channel isolation, and starvation/refill continuity. Emulator build and both audio test suites pass.

* Preserve resampling phase for accepting voices

Only reset voice.sample_phase when there are no blocks and the voice does not accept blocks (change in src/libs/ngs2.cpp). Add TestResamplingPhaseAfterStarvation to tests to ensure sampler and custom sampler retain resampling phase and pending source advancement after starvation, and that SetupSampler still resets the phase. This prevents resampling phase loss during starvation/refill across sample rates.

* audio: correct sampler controls, lifecycle, and filter tails

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-25 15:44:58 +04:00
nmzik 3585cd8f03 shader: simplify translator dispatch to void handlers 2026-09-25 14:27:09 +04:00
nmzik ff54608569 shader: add V_CMPX_O_F32 opcode 2026-09-25 14:27:09 +04:00
nmzik 6bb68a3fd9 shader: stop repeated CFG merge gateway splitting 2026-09-25 14:27:09 +04:00
nmzik 93185953f1 shader: fuse image specialization setup with materialization 2026-09-25 14:27:09 +04:00
6d4mandnmzik 7855c980cb audio: keep blocking AudioOut writes on an even cadence (#822)
* audio: keep blocking AudioOut writes on an even cadence

The blocking queue check only enforced a minimum cushion, so the guest
could queue a burst of buffers and then be blocked for the whole cushion.
Measured on PPSA03647, intervals between guest writes alternated between
0 ms and ~44 ms instead of one 5.33 ms buffer period, while the average
rate stayed correct at 48000 frames/s. Mixers that run on a timer cannot
track that cadence and the music turns slow and warbling.

Keep the cushion as-is, but also hold writes to the buffer period, and
skip that wait when the stream is running low so it cannot underrun.

Intervals become ~5.4 ms with microsecond-level deviation and the menu
music plays correctly.

* audio: reuse output clock after SDL queue priming

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-25 12:33:01 +04:00
5a705dd15f shader: support V_CMPX_EQ_U16 (#816)
* shader: support V_CMPX_EQ_U16

Decode gfx11 VOPC 0xba and lower it as an unsigned 16-bit equality
compare with CMPX EXEC-mask semantics.

🤖 Generated with Codebuff 🤖
Co-Authored-By: Codebuff <noreply@codebuff.com>

* shader: correct CMPX EQ U16 runtime expectations

Remove the added CFG tests and correct two EXEC expectations in the compute regression case.

---------

Co-authored-by: Matthew Hurley <171001439+theantipopau@users.noreply.github.com>
Co-authored-by: Codebuff <noreply@codebuff.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-25 02:27:44 +04:00
nmzik e73324c2e0 Fix Linux fiber stack overflow with Clang 18 (#813)
Keep function-name formatting off small fiber stacks
2026-09-25 02:08:35 +04:00
Mac a20fb7f9f8 agc: trace per-frame packet parameters only with graphics debug dump
The per-frame ACB submit, indirect dispatch, reset-queue and patch-address
parameter dumps, and the flip-pending poll, are logged only with the graphics
debug dump, the way the DCB submit path already is. Otherwise they were about
7.5 MB of log a minute in the Spider-Man world.
2026-09-25 00:03:08 +04:00
nmzikandMac 1cc0ac7ed8 ajm: share control handling to fully reset gapless decoders
Co-authored-by: Mac <me@itsmemac.com>
2026-09-25 00:01:51 +04:00
nmzik 83628a436e audio: fix NGS2 playback 2026-09-24 22:34:55 +04:00
nmzik 2165c710c3 audio: share block completion 2026-09-24 22:34:55 +04:00
Saadiandnmzik b9a0bd2791 loader: wrap elapsed time in Timer::GetTime (#679)
loader: format elapsed log timestamps beyond 24 hours

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-24 22:02:19 +04:00
6d4m 96874909ab cmake: pass -Map as discrete link option on non-clang-cl builds (#806)
LINK_FLAGS is a space-delimited string, so when the build path contains
spaces the linker map option is split into bogus arguments and the final
link fails:

  clang++: error: no such file or directory:
  'pro/KytyPS5/_Build/linux/kyty_emulator_clang_lld.map'

Move -Wl,-Map=<path> out of LINK_FLAGS into target_link_options, which
CMake passes to the linker as a single argument. This completes #26 for
the WIN32 OR LINUX branch (clang/gcc builds); the clang-cl branch was
already converted there.
2026-09-24 21:23:13 +04:00
Romain HedouinandClaude Sonnet 5 d646fd92d1 graphics: hoist loop-invariant checks out of draw/attrib hot paths (#807)
* graphics: hoist loop-invariant checks out of draw/attrib hot paths

DrawIndirectMulti recomputed the index_type_and_size switch on every
sub-draw of a multi-draw-indirect call, and ShaderApplyAttribSemantics
re-checked the debug-dump config flag on every vertex attribute. Both
values are invariant across their respective loops, so compute them
once before iterating instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* graphics: cache vertex buffer descriptor size in AcquireVertexBuffers

VertexBufferDescriptorSize was computed twice per vertex-buffer slot
on every draw call: once while collecting guest ranges, and again
while rebuilding slot bindings. Both calls use identical inputs, so
cache the first pass's result and reuse it in the second.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-24 21:02:23 +04:00
nmzikandMac d72a79af2d graphics: skip compute dispatches with BVH instructions
Co-authored-by: Mac <me@itsmemac.com>
2026-09-24 20:46:04 +04:00
nmzikandMac dc9280d75d shader: add GLC/DLC buffer cache policy support
Co-authored-by: Mac <me@itsmemac.com>
2026-09-24 20:46:04 +04:00
nmzik 6a9ae4ae43 shader: add SDWA support for V_MAX_I32 2026-09-24 20:46:04 +04:00
nmzik 392f39e2f6 shader: support indirect BUFFER_LOAD_DWORDX3 2026-09-24 20:46:04 +04:00
nmzikandMac cce0cd950c videoout: add RGB10A2 PQ buffer support
Co-authored-by: Mac <me@itsmemac.com>
2026-09-24 20:46:04 +04:00
e43c242683 shader: add S_MEMREALTIME opcode
Co-authored-by: Maxime Allanic <maxime@allanic.me>
Co-authored-by: Mac <me@itsmemac.com>
2026-09-24 20:46:04 +04:00
nmzikandMac 732057b072 videoout: fix VRR status import signature
Co-authored-by: Mac <me@itsmemac.com>
2026-09-24 20:46:04 +04:00
nmzik 2aaf0c9ba3 graphics: clamp raster depth after polygon offset
Enable native Vulkan depth clamping independently of guest depth clipping
2026-09-24 14:09:02 +04:00
nmzik db7745ef41 update README 2026-09-24 12:30:27 +04:00
nmzik 02578f3c6b docs: update screenshots 2026-09-24 12:18:55 +04:00
fcf0f3b67e shader: fold the lane test after a pixel shader's entry s_wqm (#801)
Preserve live lane predicates through RDNA2 whole-quad expansion

Keep the source-live predicate when WQM expands a scalar mask, so entry masks remain known live without an instruction pre-scan. Preserve scalar mask values, SCC, half writes, and raw-source behavior.

Cover entry pixel masks, cross-half copies, and scalar/high-word sources with existing shader regression suites.

Co-authored-by: nmzik <Nmzik@mail.ru>
Co-authored-by: Douglas <douglasproducoes21@gmail.com>
2026-09-24 12:02:19 +04:00
nmzik 84d015120c Remove duplicate image scratch setup from push constant test 2026-09-24 11:13:33 +04:00
nmzikandTarkusR a6248c7c7c Convert formatted half-float buffer stores from F32
Co-authored-by: TarkusR <90155059+TarkusR@users.noreply.github.com>
2026-09-24 11:13:33 +04:00
nmzik 3bd5de646d Reuse prepared shader bindings across draws and dispatches 2026-09-24 11:13:33 +04:00
nmzik 0382d0ca58 Publish reserved guest mappings atomically after backing commit 2026-09-24 11:13:33 +04:00
nmzik a670dc84b3 Test RDNA2 wave32 ReadLane material key selection 2026-09-24 11:13:33 +04:00
nmzikandTarkusR 963a0817dc Bound uniformized image keys by guarded material mask
Co-authored-by: TarkusR <90155059+TarkusR@users.noreply.github.com>
2026-09-24 11:13:33 +04:00
nmzik 1617ee77e5 Share CFG reachability for descriptor bounds analysis 2026-09-24 11:13:33 +04:00
nmzikandTarkusR 66cc2fe26d Track depth and stencil attachment feedback across graphics stages
Co-authored-by: TarkusR <90155059+TarkusR@users.noreply.github.com>
2026-09-24 11:13:33 +04:00
nmzik 51c34d1a24 Test zero scalar shift folding on RDNA2 compute shader 2026-09-24 11:13:33 +04:00
nmzikandTarkusR 3b89e36d1e Track bounded indirect image tables from signed shader loops
Co-authored-by: TarkusR <90155059+TarkusR@users.noreply.github.com>
2026-09-24 11:13:33 +04:00
João Lobo 3dc3fd0197 shader: implement DS_INC_U32 and DS_DEC_U32 (#790) 2026-09-24 00:45:12 +04:00
Deniz ŞAHİN fc86509aac shader: support DS_WRITE_B8_D16_HI (#794)
Decode GFX10 DS opcode 0xa0 as DS_WRITE_B8_D16_HI (next to
DS_WRITE_B16_D16_HI at 0xa1) and lower it through the existing 8-bit
shared-memory write. The stored byte is bits 23:16 of the data VGPR,
selected with SDWA BYTE_2 on src1.

Add captured-instruction decode coverage (from GTA V, PPSA04264) and a
runtime regression that writes one byte of a distinctive source value
into a pre-filled LDS dword.
2026-09-24 00:44:39 +04:00
569ec04d1a Fix save metadata and timestamps
Co-authored-by: Ali Almohaya <AliAlmohaya@gmail.com>
Co-authored-by: CheesyPoofs346 <jjs777406@gmail.com>
2026-09-23 22:50:51 +04:00
nmzik 7d7267cc56 Translate NGG mesh prolog with guest wave32 width 2026-09-23 22:50:51 +04:00
nmzik 2aa0a64f63 materialize compressed video-out DCC clears before flip
PPSA27624
2026-09-23 22:50:51 +04:00
nmzik fc04db7596 NGS2: preserve ATRAC9 config word byte order 2026-09-23 22:50:51 +04:00
nmzik 1275d65bb3 Audio: split ACM, Audio3D, and NGS2 implementations 2026-09-23 22:50:51 +04:00
nmzik 3742ad9980 NGS2: warn and skip unsupported custom audio modules 2026-09-23 22:50:51 +04:00
nmzik c437de9721 Presenter: avoid lock 2026-09-23 22:50:51 +04:00
Daniil Shushakov ba579088d7 loader: refresh imports after dynamic module loads (#511)
#307: loader: refresh imports after dynamic module loads
2026-09-23 21:04:10 +04:00
Fayaz Shaikhandnmzik 019929587c Allow resizing by default and recreate swapchain on resize (#763)
* Add `stretch to window` support

* Address comment

* Allow window resizing by default and recreate swapchain on resize

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-23 17:49:38 +04:00
nmzik f68c77d785 SDL3 update (#787)
* Update to SDL3

* Add XTest development dependency for SDL3

* Preserve native entry points with SDL3

* Document SDL3 XTest dependency
2026-09-23 17:22:15 +04:00
Louisandnmzik b5ba482fc2 Implement missing kernel and startup library imports (#772)
* Implement missing kernel and startup library imports

* Remove chmod and test changes; guard vectored I/O size

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-23 15:57:42 +04:00
nmzikandDaniil Shushakov 9860c2bbf8 Associate stencil when depth is acquired
Co-authored-by: Daniil Shushakov <4shushakov@gmail.com>
2026-09-23 15:41:02 +04:00
nmzik ee3749d9b6 Cover 64 MiB download ring staging 2026-09-23 15:41:02 +04:00
nmzik 3fad77fd8b Increase GPU readback staging to 64 MiB 2026-09-23 15:41:02 +04:00
nmzik d6b2abe26f Remove redundant command buffer checks 2026-09-23 15:41:02 +04:00
nmzikandMac 79530b4c5e Read indirect dispatch counts from GPU buffers
Co-authored-by: Mac <me@itsmemac.com>
2026-09-23 15:41:02 +04:00
nmzikandDaniil Shushakov d8a4412498 Refresh stencil backing through the depth image
Co-authored-by: Daniil Shushakov <4shushakov@gmail.com>
2026-09-23 15:41:02 +04:00
nmzik 5ed5fdd36e Preserve resource level bits in texture descriptors 2026-09-23 15:41:02 +04:00
nmzik ef83ba65ad Validate texture descriptor fields 2026-09-23 15:41:02 +04:00
nmzik d30dca78be Read image descriptors in one bounded span 2026-09-23 15:41:02 +04:00
nmzik 66f1a82365 Resolve scalar image tables during resource planning 2026-09-23 15:41:02 +04:00
nmzikandTechx3 eb8a5504a3 Emit coordinates for each indirect image candidate
Co-authored-by: Techx3 <56268269+Techx3@users.noreply.github.com>
2026-09-23 15:41:02 +04:00
nmzik 8a5e5c5476 Fold buffer address identities and remove empty predicates 2026-09-23 15:41:02 +04:00
nmzik 03c55e5712 Keep shader user data inline for cached programs 2026-09-23 15:41:02 +04:00
5d1ad31c49 Resolve guarded descriptor Phis at their use edge
Co-authored-by: Techx3 <56268269+Techx3@users.noreply.github.com>
Co-authored-by: GodsWill <GodsWill@users.noreply.github.com>
2026-09-23 15:41:02 +04:00
nmzik 13fcb3a133 Refresh cached resources in one SRT walk 2026-09-23 15:41:02 +04:00
d6dac92da1 Reuse scalar evaluation storage
Co-authored-by: MehmetCambaz <mcambaz18@gmail.com>
Co-authored-by: Leclowndu93150 <xdr.a123454321@gmail.com>
2026-09-23 15:41:02 +04:00
nmzik 90471836f7 Resolve storage mip views before layout lookup 2026-09-23 15:34:44 +04:00
nmzik 8252caee04 Quiet draw and audio logs 2026-09-23 15:32:30 +04:00
crevant 8b095caefe perf(files): move directory-entry names instead of copying them (#758) 2026-09-22 12:26:06 +04:00
nmzik 8dc9b9d532 Persist save allocations for search and mount capacity queries 2026-09-22 11:38:50 +04:00
e40438f95e Decode ATRAC9 streams after complete RIFF headers
Co-authored-by: Techx3 <56268269+Techx3@users.noreply.github.com>
Co-authored-by: Ali Almohaya <17254073+Almo7aya@users.noreply.github.com>
2026-09-22 11:38:50 +04:00
nmzik a6a56c8a44 Remove obsolete partial indirect draw warnings 2026-09-22 11:38:50 +04:00
nmzik 4b074cf9e9 Avoid heap allocation when compacting materialized shader images 2026-09-22 11:38:50 +04:00
nmzikandAli Almohaya 7c9e18e73b Ignore inactive metadata address bits in sampled depth descriptors
Co-authored-by: Ali Almohaya <17254073+Almo7aya@users.noreply.github.com>
2026-09-22 11:38:50 +04:00
nmzik d2df2cad74 Resolve draw-uniform image descriptors through scalar buffer chains 2026-09-22 11:38:50 +04:00
nmzikandBipin b2a80a3a96 Resolve GPU-selected raw buffer descriptors through shared RDNA2 addressing
Co-authored-by: Bipin <87369440+bipinkrish@users.noreply.github.com>
2026-09-22 11:38:50 +04:00
nmzik cfb8f7b246 Decode unsigned byte sources for SDWA half-float conversion 2026-09-22 11:11:36 +04:00
nmzik d792dfa885 Reject AvPlayer pause and seek before playback 2026-09-22 11:11:36 +04:00
nmzik 12ff571609 Preserve AvPlayer sync mode before source creation 2026-09-22 11:11:36 +04:00
nmzik d3c182f385 Centralize attachment sample validation before acquisition 2026-09-22 11:11:36 +04:00
nmzik bf4dcdce52 Accept resolved FMASK metadata with native MSAA 2026-09-22 11:11:36 +04:00
nmzik 476204cd4b Initialize native NGG vertex launch state and wave size 2026-09-22 11:11:36 +04:00
nmzik f9e56ef8a0 Scope native barriers to shaders with shared execution 2026-09-22 11:11:36 +04:00
nmzik 61761b65f2 Honor PS5 title output-resolution detection permission 2026-09-22 11:11:36 +04:00
nmzik ba55ba5dc5 Allow storage access to PS5 cubemap faces 2026-09-21 00:49:54 +04:00
nmzik c997cdb837 Skip zero-valued indirect CX 0x24a 2026-09-21 00:49:54 +04:00
Kody Schram fe4f9423e3 Significantly Reduce Linux Release Size (#738)
Added symbol stripping for linux build, drops fs from 120 mb to 20mb
2026-09-20 21:26:24 +04:00
nmzik 81af130147 Avoid game list rescans on settings changes/Remove Space pause shortcut 2026-09-20 21:07:36 +04:00
Saadiandnmzik ada0514084 launcher: add a game list refresh action (#683)
* launcher: add a game list refresh action

Dropping new games into an already-configured game folder was only
picked up after restarting the launcher or re-saving the global
settings dialog. Add a refresh toolbar button, an F5 shortcut, and a
"Refresh game list" context-menu entry, all triggering a rescan of the
game folders.

* launcher: do not rescan the game list while a game is running

Clearing cfgs_list destroys the running ConfigurationItem: the running
state is lost and the replacement items allow editing or deleting a
game whose process is still active. Skip the rescan while any item is
running (CodeRabbit review finding for the refresh action).

* launcher: preserve game state when refreshing the list

Share one refresh action across the toolbar, keyboard shortcut, and context menu. Retain running items during rescans, update their settings for the next launch, and preserve selection, filtering, and sort order.

Commit active inline editors before rebuilding rows to avoid losing comments or accessing destroyed configuration data. Tie editor callbacks to item lifetime and guard the context-menu item reference.

Validation: launcher build and startup check passed; 32 Qt interaction checks passed.

* launcher: remove refresh from the game context menu

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-20 20:44:40 +04:00
Ilyasandnmzik d7da244475 Add automatic sccache/ccache detection to CMake and remove deprecated Ninja setup action (#734)
* remove outdated ninja setup action

* Remove deprecated ninja setup action

The seanmiddleditch/gha-setup-ninja action has been archived by its
maintainer, who explicitly states that Ninja is now pre-installed on
all GitHub-hosted runners. This removes the action from all three
build jobs (Windows, macOS, Linux) and removes ninja from the package
install lists on macOS and Linux.

* Add automatic ccache/sccache detection to CMakeLists.txt

Detects sccache (preferred on Windows/MSVC) or ccache at configure time
and sets CMAKE_C_COMPILER_LAUNCHER / CMAKE_CXX_COMPILER_LAUNCHER
automatically. Works for local builds without manual flags.

Priority: sccache > ccache (sccache handles MSVC PDBs better).

* Delete github-actions-version-research.md

* Respect user-specified CMAKE_C_COMPILER_LAUNCHER / CMAKE_CXX_COMPILER_LAUNCHER

Only auto-detect sccache/ccache if no launcher was explicitly provided
via -DCMAKE_C_COMPILER_LAUNCHER=... or -DCMAKE_CXX_COMPILER_LAUNCHER=...

* Preserve explicit compiler launcher settings per language

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-20 19:49:59 +04:00
Elijaandnmzik 549495575c shader: add S_BITCMP0_B64 and S_BITCMP1_B64 (#688)
* shader: add S_BITCMP0_B64 and S_BITCMP1_B64

* test: move S_BITCMP_B64 coverage to compute suite

* test: use the existing compute CTest entry for bit comparisons

* test: remove the dedicated S_BITCMP_B64 command-line selector

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-20 19:36:12 +04:00
Maayan080andnmzik c3cc8454b9 common: remove unused recursive file enumeration (#681)
common: don't follow reparse points in the Windows recursive file walk

SysFileFindFiles recurses into every entry carrying
FILE_ATTRIBUTE_DIRECTORY, which on Windows includes directory symlinks
and junctions. A junction pointing at one of its own ancestors makes the
walk recurse until the stack runs out.

The Linux implementation already guards against this: it uses lstat
rather than stat specifically so a symlink is never followed into a
cycle. Skip reparse-point entries on Windows for the same reason.

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-20 18:50:57 +04:00
Saadi b8ad1d8c4b common: return real Win32 cache flags from GetCacheAccessType (#673)
GetCacheAccessType returned the sys_file_cache_type_t enum values as
CreateFileW flags: SYS_FILE_CACHE_SEQUENTIAL_SCAN (2) became
FILE_ATTRIBUTE_HIDDEN and SYS_FILE_CACHE_AUTO (0) lost the normal
attributes, and no FILE_FLAG_SEQUENTIAL_SCAN hint was ever applied.
Return FILE_FLAG_SEQUENTIAL_SCAN / FILE_ATTRIBUTE_NORMAL instead,
mirroring the POSIX fadvise handling on Linux.
2026-09-20 18:47:25 +04:00
Elija cfb110e0aa shader: allow partial SDWA destinations for V_NOT_B32 (#684) 2026-09-20 18:21:40 +04:00
252 changed files with 35025 additions and 12252 deletions

No files matched your search

+18 -12
View File
@@ -43,9 +43,6 @@ jobs:
- name: Setup Visual Studio environment
uses: ilammy/msvc-dev-cmd@v1
- name: Setup Ninja
uses: seanmiddleditch/gha-setup-ninja@v5
- name: Locate vcpkg
id: vcpkg
shell: pwsh
@@ -115,12 +112,16 @@ jobs:
- name: Build
shell: cmd
run: |
cmake --build _Build/windows --target launcher audio_out2_port_tests ime_dialog_tests virtual_memory_allocation_tests --parallel
cmake --build _Build/windows --target launcher ^
audio_out2_port_tests pad_haptics_tests controller_settings_tests ime_dialog_tests lru_cache_tests virtual_memory_allocation_tests ^
shader_recompiler_compute_tests ^
resource_tracking_tests scalar_provenance_tests ^
resource_materialization_tests shader_vertex_metadata_tests archive_file_tests avplayer_file_tests --parallel
- name: Test
shell: cmd
run: |
ctest --test-dir _Build/windows --output-on-failure -R "^(audio_out2_port|ime_dialog|virtual_memory_allocation)$"
ctest --test-dir _Build/windows --output-on-failure -R "^(audio_out2_port|pad_haptics|controller_settings|ime_dialog|lru_cache|resource_tracking|scalar_provenance|resource_materialization|shader_vertex_metadata|archive_file|avplayer_file|virtual_memory_allocation)$"
- name: Install
shell: cmd
@@ -155,7 +156,7 @@ jobs:
- name: Install build dependencies
shell: bash
run: brew install glslang ninja
run: brew install glslang
- name: Install Qt 6.10.3
uses: jurplel/install-qt-action@v4
@@ -201,14 +202,17 @@ jobs:
shell: bash
run: |
cmake --build _Build/macos \
--target launcher audio_out2_port_tests ime_dialog_tests virtual_memory_allocation_tests \
--target launcher audio_out2_port_tests pad_haptics_tests controller_settings_tests ime_dialog_tests lru_cache_tests virtual_memory_allocation_tests \
memory_tracker_tests page_manager_tests shader_recompiler_compute_tests \
resource_tracking_tests scalar_provenance_tests \
resource_materialization_tests shader_vertex_metadata_tests archive_file_tests avplayer_file_tests \
--parallel
- name: Test
shell: bash
run: |
ctest --test-dir _Build/macos --output-on-failure \
-R '^(audio_out2_port|ime_dialog|virtual_memory_allocation)$'
-R '^(audio_out2_port|pad_haptics|controller_settings|ime_dialog|lru_cache|memory_tracker|page_manager|resource_tracking|scalar_provenance|resource_materialization|shader_vertex_metadata|archive_file|avplayer_file|virtual_memory_allocation)$'
- name: Install
shell: bash
@@ -317,8 +321,8 @@ jobs:
libxkbcommon-dev \
libxrandr-dev \
libxss-dev \
libxtst-dev \
lld \
ninja-build \
pkg-config \
wayland-protocols
@@ -363,7 +367,7 @@ jobs:
-DCMAKE_PREFIX_PATH="$Qt6_DIR" 2>&1 | tee _Build/configure.log
exit "${PIPESTATUS[0]}"
- name: Verify SDL2 backends
- name: Verify SDL3 backends
shell: bash
run: |
status=0
@@ -382,14 +386,16 @@ jobs:
run: |
cmake --build _Build/linux \
--target launcher page_manager_tests memory_tracker_tests \
audio_out2_port_tests ime_dialog_tests virtual_memory_allocation_tests \
audio_out2_port_tests pad_haptics_tests controller_settings_tests ime_dialog_tests lru_cache_tests virtual_memory_allocation_tests \
resource_tracking_tests scalar_provenance_tests \
resource_materialization_tests shader_vertex_metadata_tests archive_file_tests avplayer_file_tests \
--parallel
- name: Test
shell: bash
run: |
ctest --test-dir _Build/linux --output-on-failure \
-R '^(audio_out2_port|ime_dialog|page_manager|memory_tracker|virtual_memory_allocation)$'
-R '^(audio_out2_port|pad_haptics|controller_settings|ime_dialog|lru_cache|memory_tracker|page_manager|resource_tracking|scalar_provenance|resource_materialization|shader_vertex_metadata|archive_file|avplayer_file|virtual_memory_allocation)$'
- name: Install
shell: bash
+3 -3
View File
@@ -1,7 +1,7 @@
[submodule "3rdparty/SDL2"]
path = 3rdparty/SDL2
[submodule "3rdparty/SDL3"]
path = 3rdparty/SDL3
url = https://github.com/libsdl-org/SDL.git
branch = SDL2
branch = release-3.4.x
[submodule "3rdparty/magic_enum"]
path = 3rdparty/magic_enum
url = https://github.com/Neargye/magic_enum.git
+72 -3
View File
@@ -2,10 +2,31 @@ set(BUILD_SHARED_LIBS OFF CACHE BOOL "" FORCE)
add_subdirectory(cpuinfo)
# The Windows guest red-zone patcher needs a decoder/encoder and a small
# trampoline assembler. Pin the same revisions used by shadPS4's implementation.
include(FetchContent)
# The Bluetooth DualSense transports speaker audio as fixed-size Opus packets.
set(kyty_build_testing_saved "${BUILD_TESTING}")
set(BUILD_TESTING OFF)
set(OPUS_BUILD_PROGRAMS OFF CACHE BOOL "" FORCE)
set(OPUS_INSTALL_PKG_CONFIG_MODULE OFF CACHE BOOL "" FORCE)
set(OPUS_INSTALL_CMAKE_CONFIG_MODULE OFF CACHE BOOL "" FORCE)
if(MSVC AND CMAKE_C_COMPILER_ID STREQUAL "Clang")
# Opus assumes MSVC accepts SSE4.1 intrinsics without per-file flags, but
# clang-cl rejects them in its baseline x64 files. Its AVX2 dispatch also
# depends on the SSE4.1 declarations, so retain the x64 SSE2 baseline only.
set(OPUS_X86_MAY_HAVE_SSE4_1 OFF CACHE BOOL "" FORCE)
set(OPUS_X86_MAY_HAVE_AVX2 OFF CACHE BOOL "" FORCE)
endif()
FetchContent_Declare(opus
GIT_REPOSITORY https://github.com/xiph/opus.git
GIT_TAG ddbe48383984d56acd9e1ab6a090c54ca6b735a6
GIT_SHALLOW FALSE)
FetchContent_MakeAvailable(opus)
set(BUILD_TESTING "${kyty_build_testing_saved}")
install(FILES "${opus_SOURCE_DIR}/COPYING" DESTINATION "licenses/opus")
# The Windows guest red-zone patcher needs a decoder/encoder and a small
# trampoline assembler. Pin the same revisions used by shadPS4's implementation.
set(XBYAK_BUILD_TESTS OFF CACHE BOOL "" FORCE)
FetchContent_Declare(
xbyak
@@ -28,8 +49,55 @@ FetchContent_Declare(
FetchContent_MakeAvailable(xbyak zydis)
# ZArchive provides random-access reads for compressed game dumps. Build only
# the reader/writer library and its zstd dependency; the command-line tool is
# not part of KytyPS5's runtime distribution.
set(ZSTD_BUILD_PROGRAMS OFF CACHE BOOL "" FORCE)
set(ZSTD_BUILD_SHARED OFF CACHE BOOL "" FORCE)
set(ZSTD_BUILD_STATIC ON CACHE BOOL "" FORCE)
set(ZSTD_BUILD_TESTS OFF CACHE BOOL "" FORCE)
set(ZSTD_LEGACY_SUPPORT OFF CACHE BOOL "" FORCE)
set(ZSTD_MULTITHREAD_SUPPORT OFF CACHE BOOL "" FORCE)
FetchContent_Declare(
zstd
GIT_REPOSITORY https://github.com/facebook/zstd.git
GIT_TAG f8745da6ff1ad1e7bab384bd1f9d742439278e99
GIT_SHALLOW FALSE
SOURCE_SUBDIR build/cmake
)
FetchContent_Declare(
zarchive_source
GIT_REPOSITORY https://github.com/Exzap/ZArchive.git
GIT_TAG d2c717730092c7bf8cbb033b12fd4001b7c4d932
GIT_SHALLOW FALSE
PATCH_COMMAND "${GIT_EXECUTABLE}" apply --ignore-whitespace
"${CMAKE_CURRENT_SOURCE_DIR}/patches/zarchive-reader.patch"
SOURCE_SUBDIR kyty-library-only
)
FetchContent_MakeAvailable(zstd zarchive_source)
add_library(zarchive STATIC
"${zarchive_source_SOURCE_DIR}/src/zarchivereader.cpp"
"${zarchive_source_SOURCE_DIR}/src/zarchivewriter.cpp"
"${zarchive_source_SOURCE_DIR}/src/sha_256.c"
)
add_library(ZArchive::zarchive ALIAS zarchive)
target_include_directories(zarchive SYSTEM PUBLIC "${zarchive_source_SOURCE_DIR}/include")
target_include_directories(zarchive PRIVATE "${zstd_SOURCE_DIR}/lib")
target_link_libraries(zarchive PRIVATE libzstd_static)
target_compile_features(zarchive PRIVATE cxx_std_20)
if(MSVC)
target_compile_options(zarchive PRIVATE /W0)
else()
target_compile_options(zarchive PRIVATE -w)
endif()
set(SDL_SHARED OFF CACHE BOOL "" FORCE)
add_subdirectory(SDL2)
if(APPLE)
# SDL's built-in iconv fallback does not support CP932.
set(SDL_SYSTEM_ICONV ON CACHE BOOL "Use system iconv for CP932 conversion" FORCE)
endif()
add_subdirectory(SDL3)
add_library(Vulkan-Headers INTERFACE)
add_library(Vulkan::Headers ALIAS Vulkan-Headers)
@@ -120,6 +188,7 @@ endif()
set(TRACY_STATIC ON CACHE BOOL "" FORCE)
set(TRACY_ENABLE ON CACHE BOOL "" FORCE)
set(TRACY_NO_CRASH_HANDLER ON CACHE BOOL "" FORCE)
set(TRACY_ON_DEMAND ON CACHE BOOL "" FORCE)
set(TRACY_DELAYED_INIT ON CACHE BOOL "" FORCE)
set(TRACY_MANUAL_LIFETIME ON CACHE BOOL "" FORCE)
-1
Submodule 3rdparty/SDL2 deleted from 4b69833bc5.
Vendored Submodule
+1
Submodule 3rdparty/SDL3 added at fa2c02bb6e.
+100
View File
@@ -0,0 +1,100 @@
--- a/include/zarchive/zarchivecommon.h
+++ b/include/zarchive/zarchivecommon.h
@@ -256,7 +256,7 @@
bool IsWithinValidRange(uint64_t fileSize) const
{
- return (offset + size) <= fileSize;
+ return offset <= fileSize && size <= fileSize - offset;
}
};
--- a/src/zarchivereader.cpp
+++ b/src/zarchivereader.cpp
@@ -14,6 +14,7 @@
static bool _ifstream_readBytes(std::ifstream& file, uint64_t offset, void* buffer, uint32_t size)
{
+ file.clear();
file.seekg(offset, std::ios_base::beg);
file.read((char*)buffer, size);
return file.gcount() == size;
@@ -80,9 +81,27 @@
// verify file tree
if (fileTree[0].IsFile())
return nullptr; // first entry must be root directory
- auto rootName = GetName(nameTable, fileTree[0].GetNameOffset());
- if (!rootName.empty())
+ if (fileTree[0].GetNameOffset() != 0x7FFFFFFF)
return nullptr; // root node must not have a name
+ const uint64_t dataCapacity = (uint64_t)offsetRecords.size() * _ZARCHIVE::ENTRIES_PER_OFFSETRECORD * _ZARCHIVE::COMPRESSED_BLOCK_SIZE;
+ for (size_t i = 0; i < fileTree.size(); ++i)
+ {
+ auto& entry = fileTree[i];
+ if (i != 0)
+ {
+ auto name = GetName(nameTable, entry.GetNameOffset());
+ if (name.empty() || name == "." || name == ".." || name.find_first_of("/\\") != std::string_view::npos || name.find('\0') != std::string_view::npos)
+ return nullptr;
+ }
+ if (entry.IsFile())
+ {
+ if (entry.GetFileOffset() > dataCapacity || entry.GetFileSize() > dataCapacity - entry.GetFileOffset())
+ return nullptr;
+ }
+ else if (entry.directoryRecord.nodeStartIndex > fileTree.size() ||
+ entry.directoryRecord.count > fileTree.size() - entry.directoryRecord.nodeStartIndex)
+ return nullptr;
+ }
// read meta data
// todo
@@ -238,7 +257,7 @@
uint32_t stepSize = std::min<uint32_t>(remainingBytes, _ZARCHIVE::COMPRESSED_BLOCK_SIZE - blockOffset);
CacheBlock* block = GetCachedBlock(blockIdx);
if (!block)
- return 0;
+ return bytesToRead - remainingBytes;
std::memcpy(bufferU8, block->data + blockOffset, stepSize);
rawReadOffset += stepSize;
remainingBytes -= stepSize;
@@ -325,14 +344,18 @@
// determine offset and size of compressed block
auto& record = m_offsetRecords[recordIndex];
uint64_t offset = record.baseOffset;
+ if (offset > m_compressedDataSize)
+ return false;
for (uint32_t i = 0; i < recordSubIndex; i++)
{
- offset += (uint64_t)record.size[i];
- offset++;
+ const uint64_t size = (uint64_t)record.size[i] + 1;
+ if (size > m_compressedDataSize - offset)
+ return false;
+ offset += size;
}
uint32_t compressedSize = (uint32_t)record.size[recordSubIndex] + 1;
// load file data
- if ((offset + compressedSize) > m_compressedDataSize)
+ if (compressedSize > m_compressedDataSize - offset)
return false;
offset += m_compressedDataOffset;
if (compressedSize == _ZARCHIVE::COMPRESSED_BLOCK_SIZE)
@@ -350,7 +373,7 @@
// returns empty view on failure
std::string_view ZArchiveReader::GetName(const std::vector<uint8_t>& nameTable, uint32_t nameOffset)
{
- if (nameOffset == 0x7FFFFFFF || nameOffset > nameTable.size())
+ if (nameOffset == 0x7FFFFFFF || nameOffset >= nameTable.size())
return "";
// parse header
uint16_t nameLength = nameTable[nameOffset] & 0x7F;
@@ -359,7 +382,7 @@
// extended 2-byte length
if (nameOffset + 1 >= nameTable.size())
return "";
- nameLength |= ((uint16_t)nameTable[nameOffset] << 7);
+ nameLength |= ((uint16_t)nameTable[nameOffset + 1] << 7);
nameOffset += 2;
}
else
+1724
View File
File diff suppressed because it is too large. Load diff
+116 -5
View File
@@ -1,7 +1,7 @@
cmake_minimum_required(VERSION 3.12...4.3)
cmake_minimum_required(VERSION 3.18...4.3)
if(CMAKE_VERSION VERSION_GREATER_EQUAL 4.0)
set(CMAKE_POLICY_VERSION_MINIMUM 3.5 CACHE STRING "Minimum policy version for bundled dependencies")
set(CMAKE_POLICY_VERSION_MINIMUM 3.10 CACHE STRING "Minimum policy version for bundled dependencies")
endif()
get_property(KYTY_GENERATOR_IS_MULTI_CONFIG GLOBAL PROPERTY GENERATOR_IS_MULTI_CONFIG)
@@ -12,6 +12,22 @@ endif()
project(Kyty)
find_program(SCCACHE_PROGRAM sccache)
find_program(CCACHE_PROGRAM ccache)
foreach(lang IN ITEMS C CXX)
# An explicitly empty launcher disables caching for this language.
if(NOT DEFINED CMAKE_${lang}_COMPILER_LAUNCHER)
if(SCCACHE_PROGRAM)
set(CMAKE_${lang}_COMPILER_LAUNCHER "${SCCACHE_PROGRAM}")
message(STATUS "sccache enabled for ${lang}: ${SCCACHE_PROGRAM}")
elseif(CCACHE_PROGRAM)
set(CMAKE_${lang}_COMPILER_LAUNCHER "${CCACHE_PROGRAM}")
message(STATUS "ccache enabled for ${lang}: ${CCACHE_PROGRAM}")
endif()
endif()
endforeach()
set(KYTY_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/src")
set(KYTY_TESTS_DIR "${CMAKE_CURRENT_SOURCE_DIR}/tests")
set(KYTY_THIRD_PARTY_DIR "${CMAKE_CURRENT_SOURCE_DIR}/3rdparty")
@@ -285,6 +301,7 @@ set(gpu_blit_generated_dir "${PROJECT_BINARY_DIR}/gpu_blit_shaders")
set(gpu_blit_shader_sources
"${gpu_tiler_shader_dir}/gpu_blit_fs_triangle.vert"
"${gpu_tiler_shader_dir}/gpu_blit_color_to_ms_depth.frag"
"${gpu_tiler_shader_dir}/gpu_video_out_overlay.frag"
)
foreach(shader_source IN LISTS gpu_blit_shader_sources)
get_filename_component(shader_name "${shader_source}" NAME_WE)
@@ -315,7 +332,7 @@ if (KYTY_CLANG_CL)
set_source_files_properties(${kyty_emulator_src} PROPERTIES COMPILE_FLAGS "-Wno-pragma-pack -Wno-deprecated-declarations -D_TIMESPEC_DEFINED")
endif()
set(kyty_emulator_link_libraries common Vulkan::Headers spirv-tools-opt spirv-tools SDL2-static imgui xxhash FFmpeg::ffmpeg fmt::fmt nlohmann_json::nlohmann_json LibAtrac9 kyty_git_version Zydis::Zydis xbyak::xbyak)
set(kyty_emulator_link_libraries common Vulkan::Headers spirv-tools-opt spirv-tools SDL3::SDL3-static imgui xxhash FFmpeg::ffmpeg fmt::fmt nlohmann_json::nlohmann_json LibAtrac9 kyty_git_version Zydis::Zydis xbyak::xbyak opus)
# Linux system libraries.
if(LINUX)
@@ -325,7 +342,6 @@ endif()
set(inc_headers
${KYTY_SOURCE_DIR}
${KYTY_THIRD_PARTY_DIR}/SDL2/include
${KYTY_THIRD_PARTY_DIR}/VulkanMemoryAllocator/include
${KYTY_THIRD_PARTY_DIR}/SPIRV-Tools/include
${KYTY_THIRD_PARTY_DIR}/SPIRV-Headers/include
@@ -391,11 +407,26 @@ add_executable(page_manager_tests EXCLUDE_FROM_ALL
target_link_libraries(page_manager_tests common)
target_include_directories(page_manager_tests PRIVATE ${inc_headers})
add_executable(archive_file_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/ArchiveFileTests.cpp"
)
target_link_libraries(archive_file_tests common ZArchive::zarchive)
target_include_directories(archive_file_tests PRIVATE ${inc_headers})
add_executable(avplayer_file_tests EXCLUDE_FROM_ALL "${KYTY_TESTS_DIR}/AvPlayerFileTests.cpp")
target_link_libraries(avplayer_file_tests common FFmpeg::ffmpeg ZArchive::zarchive)
target_include_directories(avplayer_file_tests PRIVATE ${inc_headers})
add_executable(bit_array_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/BitArrayTests.cpp"
)
target_include_directories(bit_array_tests PRIVATE ${inc_headers})
add_executable(lru_cache_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/LruCacheTests.cpp"
)
target_include_directories(lru_cache_tests PRIVATE ${inc_headers})
add_executable(ime_dialog_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/ImeDialogTests.cpp"
"${KYTY_SOURCE_DIR}/libs/imeCommon.cpp"
@@ -450,12 +481,53 @@ add_executable(audio_out2_port_tests EXCLUDE_FROM_ALL
target_link_libraries(audio_out2_port_tests common fmt::fmt)
target_include_directories(audio_out2_port_tests PRIVATE ${inc_headers})
add_executable(audio_out_timing_tests EXCLUDE_FROM_ALL "${KYTY_TESTS_DIR}/AudioOutTimingTests.cpp")
target_link_libraries(audio_out_timing_tests common fmt::fmt SDL3::SDL3-static)
target_include_directories(audio_out_timing_tests PRIVATE ${inc_headers})
add_executable(ngs2_sampler_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/Ngs2SamplerTests.cpp"
"${KYTY_SOURCE_DIR}/loader/timer.cpp"
)
target_link_libraries(ngs2_sampler_tests common fmt::fmt LibAtrac9)
target_include_directories(ngs2_sampler_tests PRIVATE ${inc_headers})
add_executable(pad_haptics_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/PadHapticsTests.cpp"
)
target_link_libraries(pad_haptics_tests common fmt::fmt SDL3::SDL3-static opus)
target_include_directories(pad_haptics_tests PRIVATE ${inc_headers})
add_executable(controller_settings_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/ControllerSettingsTests.cpp"
)
target_link_libraries(controller_settings_tests common fmt::fmt SDL3::SDL3-static)
target_include_directories(controller_settings_tests PRIVATE ${inc_headers})
add_executable(save_data_memory_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/SaveDataMemoryTests.cpp"
)
target_link_libraries(save_data_memory_tests common fmt::fmt)
target_include_directories(save_data_memory_tests PRIVATE ${inc_headers})
add_executable(png_enc_tests EXCLUDE_FROM_ALL "${KYTY_TESTS_DIR}/PngEncTests.cpp")
target_link_libraries(png_enc_tests common fmt::fmt)
target_include_directories(png_enc_tests PRIVATE ${inc_headers})
add_executable(ces_tests EXCLUDE_FROM_ALL "${KYTY_TESTS_DIR}/CesTests.cpp")
target_link_libraries(ces_tests common fmt::fmt SDL3::SDL3-static)
target_include_directories(ces_tests PRIVATE ${inc_headers})
add_executable(save_data_dialog_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/SaveDataDialogTests.cpp"
"${KYTY_SOURCE_DIR}/libs/dialog.cpp"
)
target_link_libraries(save_data_dialog_tests common fmt::fmt)
target_include_directories(save_data_dialog_tests PRIVATE ${inc_headers})
add_kyty_full_emulator_test(http_uri_parse_tests
"${KYTY_TESTS_DIR}/HttpUriParseTests.cpp")
add_executable(event_queue_lifetime_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/EventQueueLifetimeTests.cpp"
"${KYTY_SOURCE_DIR}/kernel/eventQueue.cpp"
@@ -466,6 +538,7 @@ target_include_directories(event_queue_lifetime_tests PRIVATE ${inc_headers})
add_kyty_full_emulator_test(kernel_file_system_tests
"${KYTY_TESTS_DIR}/KernelFileSystemTests.cpp")
target_link_libraries(kernel_file_system_tests ZArchive::zarchive)
add_executable(sync_on_address_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/SyncOnAddressTests.cpp"
@@ -532,7 +605,10 @@ if(BUILD_TESTING)
add_test(NAME image_page_table COMMAND $<TARGET_FILE:image_page_table_tests>)
add_test(NAME memory_tracker COMMAND $<TARGET_FILE:memory_tracker_tests>)
add_test(NAME page_manager COMMAND $<TARGET_FILE:page_manager_tests>)
add_test(NAME archive_file COMMAND $<TARGET_FILE:archive_file_tests>)
add_test(NAME avplayer_file COMMAND $<TARGET_FILE:avplayer_file_tests>)
add_test(NAME bit_array COMMAND $<TARGET_FILE:bit_array_tests>)
add_test(NAME lru_cache COMMAND $<TARGET_FILE:lru_cache_tests>)
add_test(NAME shader_vertex_metadata COMMAND $<TARGET_FILE:shader_vertex_metadata_tests>)
add_test(NAME resource_materialization COMMAND $<TARGET_FILE:resource_materialization_tests>)
add_test(NAME resource_tracking COMMAND $<TARGET_FILE:resource_tracking_tests>)
@@ -540,7 +616,15 @@ if(BUILD_TESTING)
add_test(NAME kernel_file_system COMMAND $<TARGET_FILE:kernel_file_system_tests>)
add_test(NAME sync_on_address COMMAND $<TARGET_FILE:sync_on_address_tests>)
add_test(NAME audio_out2_port COMMAND $<TARGET_FILE:audio_out2_port_tests>)
add_test(NAME audio_out_timing COMMAND $<TARGET_FILE:audio_out_timing_tests>)
add_test(NAME ngs2_sampler COMMAND $<TARGET_FILE:ngs2_sampler_tests>)
add_test(NAME pad_haptics COMMAND $<TARGET_FILE:pad_haptics_tests>)
add_test(NAME controller_settings COMMAND $<TARGET_FILE:controller_settings_tests>)
add_test(NAME save_data_memory COMMAND $<TARGET_FILE:save_data_memory_tests>)
add_test(NAME png_enc COMMAND $<TARGET_FILE:png_enc_tests>)
add_test(NAME ces COMMAND $<TARGET_FILE:ces_tests>)
add_test(NAME save_data_dialog COMMAND $<TARGET_FILE:save_data_dialog_tests>)
add_test(NAME http_uri_parse COMMAND $<TARGET_FILE:http_uri_parse_tests>)
add_test(NAME shader_recompiler_compute COMMAND $<TARGET_FILE:shader_recompiler_compute_tests>)
add_test(NAME shader_recompiler_alignbyte
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --alignbyte-only)
@@ -577,6 +661,8 @@ if(BUILD_TESTING)
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --compute-meta-clear-only)
add_test(NAME buffer_cache_dirty_gc
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --buffer-cache-gc-only)
add_test(NAME color_render_target_1d
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --rt1d-only)
if(WIN32)
# These tests still depend on the Windows multisample-depth path.
@@ -595,7 +681,10 @@ if(BUILD_TESTING)
image_page_table_tests
memory_tracker_tests
page_manager_tests
archive_file_tests
avplayer_file_tests
bit_array_tests
lru_cache_tests
shader_vertex_metadata_tests
resource_materialization_tests
resource_tracking_tests
@@ -605,6 +694,15 @@ if(BUILD_TESTING)
shader_recompiler_compute_tests
virtual_memory_allocation_tests
audio_out2_port_tests
audio_out_timing_tests
ngs2_sampler_tests
pad_haptics_tests
controller_settings_tests
save_data_memory_tests
png_enc_tests
ces_tests
save_data_dialog_tests
http_uri_parse_tests
)
endif()
@@ -662,7 +760,16 @@ if(KYTY_CLANG_CL)
)
add_custom_command(TARGET kyty_emulator POST_BUILD COMMAND ${CMAKE_COMMAND} -E copy_if_different "${KYTY_THIRD_PARTY_DIR}/winpthread/bin/libwinpthread-1.dll" $<TARGET_FILE_DIR:kyty_emulator>/libwinpthread-1.dll)
elseif(WIN32 OR LINUX)
set_target_properties(kyty_emulator PROPERTIES LINK_FLAGS "${KYTY_LD_OPTIONS} -Wl,-Map=${KYTY_EMULATOR_MAP_LINK_PATH}")
set_target_properties(kyty_emulator PROPERTIES LINK_FLAGS "${KYTY_LD_OPTIONS}")
# LINK_FLAGS is a space-delimited string: a -Map path containing spaces would
# be split into bogus arguments, so pass it as a discrete link option instead.
target_link_options(kyty_emulator PRIVATE "-Wl,-Map=${KYTY_EMULATOR_MAP_LINK_PATH}")
endif()
if(LINUX)
target_link_options(kyty_emulator PRIVATE
$<$<CONFIG:Release>:-Wl,--strip-all>
)
endif()
if(APPLE)
@@ -680,6 +787,10 @@ endif()
install(TARGETS kyty_emulator DESTINATION .)
if(KYTY_BUILD_LAUNCHER)
add_subdirectory("${KYTY_SOURCE_DIR}/launcher" "${CMAKE_CURRENT_BINARY_DIR}/launcher")
if(LINUX)
target_link_options(launcher PRIVATE $<$<CONFIG:Release>:-Wl,--strip-all>)
endif()
endif()
if(KYTY_CLANG_CL)
install(FILES "${KYTY_THIRD_PARTY_DIR}/winpthread/bin/libwinpthread-1.dll" DESTINATION .)
+52 -30
View File
@@ -4,13 +4,17 @@
[![Build KytyPS5 (Linux)](https://img.shields.io/github/actions/workflow/status/KytyPS5/KytyPS5/build.yml?branch=main&event=push&label=Build%20KytyPS5%20%28Linux%29)](https://github.com/KytyPS5/KytyPS5/actions/workflows/build.yml)
[![Build KytyPS5 (macOS)](https://img.shields.io/github/actions/workflow/status/KytyPS5/KytyPS5/build.yml?branch=main&event=push&label=Build%20KytyPS5%20%28macOS%29)](https://github.com/KytyPS5/KytyPS5/actions/workflows/build.yml)
[![Platform](https://img.shields.io/badge/platform-Windows%20x64%20%7C%20Linux%20x64%20%7C%20macOS%20x86__64-0078D4.svg)](#system-requirements)
[![Status](https://img.shields.io/badge/status-early%20development-orange.svg)](#current-status)
[![Status](https://img.shields.io/badge/status-active%20development-orange.svg)](#current-status)
[![License](https://img.shields.io/badge/license-GPL--2.0-blue.svg)](LICENSE)
**[Weekly updates](https://github.com/KytyPS5/KytyPS5/discussions/862)** — game progress, recent fixes and ongoing development.
**[Development on Discord](https://discord.gg/UNrkMqGaBg)** — KytyPS5 development.
KytyPS5 is a free and open-source PlayStation 5 emulator written in C++ for Windows and Linux,
with experimental macOS support. It is based on a heavily modified version of
[Kyty](https://github.com/InoriRus/Kyty). The project is in an early stage of development, so
compatibility is limited and behavior may change significantly between builds.
[Kyty](https://github.com/InoriRus/Kyty). The project is in active development, and behavior
can change significantly between builds.
> [!IMPORTANT]
> KytyPS5 is not affiliated with Sony Interactive Entertainment or PlayStation. The project does
@@ -20,12 +24,12 @@ compatibility is limited and behavior may change significantly between builds.
## Current Status
KytyPS5 can boot 2D games and a selection of 3D games, including titles built with Unreal Engine
4/5, Unity, and custom engines. No external low-level emulation modules are currently required.
4/5, Unity, and custom engines. External low-level emulation modules are neither required nor
planned.
Development is focused on compatibility and boot reliability.
Development is currently focused on expanding game compatibility and improving boot reliability.
Windows is the primary platform and receives the most testing. Linux builds and runs; see
[Building on Linux](#building-on-linux).
Windows and Linux are the primary platforms and receive the most testing.
macOS support is experimental. The emulator is built for x86-64 and runs on Apple Silicon under
Rosetta 2, with Vulkan provided by MoltenVK. A small number of titles have been verified in-game
@@ -36,16 +40,16 @@ Community game test results are available in the
## Bugs and Issues
The project is in an early stage, so please be mindful when opening new issues. Expect crashes,
graphical glitches, low compatibility, and poor performance.
Compatibility, stability, and performance can vary between versions. You may encounter crashes
or graphical glitches, so please include the version you tested when reporting an issue.
## Screenshots
<table align="center">
<tr>
<td align="center">
<strong>Disgaea 6</strong><br>
<img src="docs/screenshots/ps5-01.png" width="300" alt="Disgaea 6 running in KytyPS5">
<strong>Astro Bot</strong><br>
<img src="docs/screenshots/ps5-01.png" width="300" alt="Astro Bot running in KytyPS5">
</td>
<td align="center">
<strong>Dreaming Sarah</strong><br>
@@ -64,12 +68,12 @@ graphical glitches, low compatibility, and poor performance.
</tr>
<tr>
<td align="center">
<strong>Hellboy</strong><br>
<img src="docs/screenshots/ps5-02.png" width="300" alt="Hellboy running in KytyPS5">
<strong>Demon's Souls</strong><br>
<img src="docs/screenshots/ps5-02.png" width="300" alt="Demon's Souls running in KytyPS5">
</td>
<td align="center">
<strong>Paleo Pines</strong><br>
<img src="docs/screenshots/ps5-06.png" width="300" alt="Paleo Pines running in KytyPS5">
<strong>UFC 5</strong><br>
<img src="docs/screenshots/ps5-06.png" width="300" alt="UFC 5 running in KytyPS5">
</td>
</tr>
</table>
@@ -79,7 +83,7 @@ graphical glitches, low compatibility, and poor performance.
## Contributing
Testing games and submitting detailed bug reports are useful ways to contribute. Search existing
issues first, then use the **Game Emulation Bug Report** template and attach the complete log file.
issues first, then use the **Game Emulation Status Report** template and attach the complete log file.
Code contributions should be focused, build successfully on the platforms they touch, and include
relevant tests where practical. Windows is the primary target, so a change that alters shared code
@@ -132,7 +136,7 @@ the Vulkan/SPIR-V validation rules.
### Build requirements (Windows)
- Git
- CMake 3.12 or newer
- CMake 3.22.1 or newer
- Ninja
- Visual Studio 2022 or Build Tools 2022 with the **Desktop development with C++** workload and
**C++ Clang tools for Windows** component
@@ -166,19 +170,19 @@ The finished application and its runtime dependencies will be placed in
### Building on Linux
Install the toolchain and the libraries the bundled SDL2 needs. Without the audio, Wayland and
udev development packages SDL2 quietly configures itself without those backends, and the resulting
Install the toolchain and the libraries the bundled SDL3 needs. Without the audio, Wayland and
udev development packages SDL3 quietly configures itself without those backends, and the resulting
build has no working sound and no gamepad hotplug:
```bash
sudo apt-get install --no-install-recommends \
clang lld ninja-build cmake git glslang-tools pkg-config \
libgl1-mesa-dev libx11-dev libxcursor-dev libxext-dev libxfixes-dev \
libxi-dev libxrandr-dev libxss-dev libxkbcommon-dev \
libxi-dev libxrandr-dev libxss-dev libxtst-dev libxkbcommon-dev \
libasound2-dev libpulse-dev libudev-dev libdbus-1-dev libwayland-dev wayland-protocols
```
Qt 6 (Concurrent, Network, Widgets) is also required — either the distribution packages
Qt 6 (Concurrent, Network, Widgets) is required for the launcher — either the distribution packages
(`qt6-base-dev`) or an official Qt installation.
```bash
@@ -197,6 +201,18 @@ The install step copies the Qt libraries and plugins next to the binaries, so
from the pinned [KytyPS5 FFmpeg core](https://github.com/KytyPS5/ext-ffmpeg-core)
release, including VP9 and WebM support. System FFmpeg packages are not required.
To build `kyty_emulator` and the `kyty_tests` target without Qt, use a separate build directory:
```bash
git submodule update --init --recursive
cmake -S . -B _Build/linux-no-qt -G Ninja -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
-DKYTY_BUILD_LAUNCHER=OFF
cmake --build _Build/linux-no-qt --target kyty_emulator kyty_tests --parallel
```
As on Windows, the MSVC compiler is not used; Clang is required. `cl.exe` is rejected at configure
time.
@@ -204,7 +220,7 @@ The CMake source root is the repository root.
### Building on NixOS
A development shell provides Clang, CMake, Ninja, Qt 6, the Vulkan headers, and the SDL2 backend
A development shell provides Clang, CMake, Ninja, Qt 6, the Vulkan headers, and the SDL3 backend
libraries. Enter it and configure exactly as on other Linux distributions; the shell exports
`CMAKE_PREFIX_PATH` and `QT_PLUGIN_PATH`, so the `-DCMAKE_PREFIX_PATH="$Qt6_DIR"` argument is not
needed:
@@ -220,9 +236,10 @@ cmake --build _Build/linux --target launcher --parallel
cmake --install _Build/linux --prefix _Build/linux/install
```
The configure step downloads the FFmpeg prebuilts and the `xbyak`/`zydis` sources, so it needs
network access; a fully sandboxed `nix build` would require vendoring those inputs. A Vulkan 1.3
driver must be available at runtime (on NixOS, `hardware.graphics.enable = true`).
The configure step downloads the FFmpeg prebuilts and the `xbyak`, `zydis`, `zstd`, and ZArchive
sources, so it needs network access; a fully sandboxed `nix build` would require vendoring those
inputs. A Vulkan 1.3 driver must be available at runtime (on NixOS,
`hardware.graphics.enable = true`).
### Building on macOS
@@ -316,17 +333,21 @@ open _Build/macos/install/KytyPS5.app # or double-click in Finder
```
On first launch, add one or more game folders in the global settings. The launcher searches those
folders recursively for game directories containing `eboot.bin`. Select a detected game and run it
from the game list.
folders recursively for game directories containing `eboot.bin` and ZArchive (`.zar`) game dumps
whose archive root contains `eboot.bin`. Select a detected game and run it from the game list.
ZArchive dumps are mounted read-only and streamed directly; they do not need to be extracted first.
The emulator can also be started directly with a legally obtained game directory or ELF file:
The emulator can also be started directly with a legally obtained game directory, ELF file, or
ZArchive dump:
```powershell
.\_Build\windows\install\kyty_emulator.exe --game "D:\Games\ExampleGame"
.\_Build\windows\install\kyty_emulator.exe --game "D:\Games\ExampleGame.zar"
```
```bash
./_Build/linux/install/kyty_emulator --game "/games/ExampleGame"
./_Build/linux/install/kyty_emulator --game "/games/ExampleGame.zar"
```
On macOS, the adjacent flat or app-bundled `libMoltenVK.dylib` is found automatically; no
@@ -370,5 +391,6 @@ licenses included with those components.
- [InoriRus/Kyty](https://github.com/InoriRus/Kyty) — KytyPS5 is based on a heavily modified version
of the original Kyty project.
- [shadps4-emu/shadPS4](https://github.com/shadps4-emu/shadPS4) — reference for memory-model
understanding and the AVPlayer implementation.
- [shadps4-emu/shadPS4](https://github.com/shadps4-emu/shadPS4) — reference for understanding PS4
memory behavior, GPU resource aliasing and cache coherency,
and the AVPlayer implementation.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.6 MiB

After

Width:  |  Height:  |  Size: 1.9 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.6 MiB

After

Width:  |  Height:  |  Size: 1.8 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.4 MiB

After

Width:  |  Height:  |  Size: 1.7 MiB

+3 -2
View File
@@ -1,8 +1,7 @@
{ pkgs }:
let
# Build-time dependencies. Every -dev package the bundled SDL2 probes for must
# be listed here, otherwise SDL2 quietly drops the audio/Wayland/udev backends.
# Build-time dependencies for the bundled SDL3 audio, video and input backends.
buildDeps = with pkgs; [
qt6.qtbase # Concurrent, Network, Widgets (launcher)
vulkan-headers
@@ -18,6 +17,7 @@ let
libXi
libXrandr
libXScrnSaver
libXtst
libxkbcommon
# Wayland
wayland
@@ -46,6 +46,7 @@ let
libXi
libXrandr
libXScrnSaver
libXtst
libxkbcommon
wayland
libdecor
+1
View File
@@ -26,6 +26,7 @@ target_link_libraries(common
cpuinfo
kyty_git_version
spdlog::spdlog
ZArchive::zarchive
)
get_property(inc_headers TARGET common PROPERTY INCLUDE_DIRECTORIES)
+397
View File
@@ -0,0 +1,397 @@
#include "common/archive.h"
#include "common/stringUtils.h"
#include <algorithm>
#include <condition_variable>
#include <cstring>
#include <mutex>
#include <optional>
#include <string_view>
#include <thread>
#include <unordered_map>
#include <utility>
#include <zarchive/zarchivereader.h>
// POSIX hosts and MinGW's pthread-backed libstdc++ have cancellable condition waits.
// MSVC/ClangCL and MinGW's native Win32 threading backend do not use pthread waits.
#if !defined(_WIN32) || defined(_GLIBCXX_GCC_GTHR_POSIX_H)
#include <pthread.h>
#define KYTY_ARCHIVE_PTHREAD_CANCELLATION
#endif
namespace Common {
// Backends expose immutable metadata and offset-based reads. Cursor management, path routing,
// reader lifetime and the host-stack worker are shared by every archive format.
class ArchiveReader {
public:
struct Entry {
uint64_t id;
uint64_t size;
bool is_file;
};
virtual ~ArchiveReader() = default;
virtual std::optional<Entry> Find(std::string_view member) = 0;
virtual std::vector<File::DirEntry> List(std::string_view member) = 0;
virtual uint64_t Read(uint64_t id, uint64_t offset, uint32_t size, void* data) = 0;
};
namespace {
class ZArchiveReaderBackend final: public ArchiveReader {
public:
explicit ZArchiveReaderBackend(std::unique_ptr<ZArchiveReader> reader)
: m_reader(std::move(reader)) {}
std::optional<Entry> Find(std::string_view member) override {
const auto node = m_reader->LookUp(member);
if (node == ZARCHIVE_INVALID_NODE) {
return {};
}
return Entry {node, m_reader->GetFileSize(node), m_reader->IsFile(node)};
}
std::vector<File::DirEntry> List(std::string_view member) override {
const auto node = m_reader->LookUp(member);
if (!m_reader->IsDirectory(node)) {
return {};
}
const auto count = m_reader->GetDirEntryCount(node);
if (count > (1u << 20u)) {
return {};
}
std::vector<File::DirEntry> result;
result.reserve(count);
for (uint32_t index = 0; index < count; ++index) {
ZArchiveReader::DirEntry entry {};
if (m_reader->GetDirEntry(node, index, entry)) {
result.push_back({std::string(entry.name), entry.isFile});
}
}
return result;
}
uint64_t Read(uint64_t id, uint64_t offset, uint32_t size, void* data) override {
// ZArchiveReader already serializes access to its stream and decompression cache.
return m_reader->ReadFromFile(static_cast<ZArchiveNodeHandle>(id), offset, size, data);
}
private:
std::unique_ptr<ZArchiveReader> m_reader;
};
std::shared_ptr<ArchiveReader> OpenZArchive(const std::filesystem::path& path) {
auto reader = std::unique_ptr<ZArchiveReader>(ZArchiveReader::OpenFromFile(path));
return reader ? std::make_shared<ZArchiveReaderBackend>(std::move(reader)) : nullptr;
}
struct ArchiveFormat {
std::string_view extension;
std::shared_ptr<ArchiveReader> (*open)(const std::filesystem::path&);
};
constexpr ArchiveFormat Formats[] = {{".zar", OpenZArchive}};
using NativeView = std::basic_string_view<std::filesystem::path::value_type>;
const ArchiveFormat* FindFormat(NativeView path) {
for (const auto& format: Formats) {
if (path.size() < format.extension.size()) {
continue;
}
const auto extension = path.substr(path.size() - format.extension.size());
if (std::equal(
extension.begin(), extension.end(), format.extension.begin(),
[](auto a, char b) { return (a >= 'A' && a <= 'Z' ? a + ('a' - 'A') : a) == b; })) {
return &format;
}
}
return nullptr;
}
size_t FindMarker(NativeView path) {
// Native files use this check on every operation. The usual path has no marker and allocates
// nothing.
for (auto marker = path.find('!'); marker != NativeView::npos;
marker = path.find('!', marker + 1)) {
if ((marker + 1 == path.size() || path[marker + 1] == '/' || path[marker + 1] == '\\') &&
FindFormat(path.substr(0, marker)) != nullptr) {
return marker;
}
}
return NativeView::npos;
}
struct ParsedPath {
std::filesystem::path archive;
std::string member;
};
std::optional<ParsedPath> ParsePath(const std::filesystem::path& path) {
const auto& native = path.native();
const auto marker = FindMarker(native);
if (marker == NativeView::npos) {
return {};
}
ParsedPath parsed {.archive = std::filesystem::path(native.substr(0, marker))};
auto member = PathToGenericString(std::filesystem::path(native.substr(marker + 1)));
std::replace(member.begin(), member.end(), '\\', '/');
std::string_view remaining = member;
while (!remaining.empty()) {
const auto separator = remaining.find('/');
const auto component = remaining.substr(0, separator);
remaining.remove_prefix(separator == std::string_view::npos ? remaining.size() : separator + 1);
if (component.empty() || component == ".") {
continue;
}
if (component == "..") {
if (parsed.member.empty()) {
return {};
}
const auto separator = parsed.member.rfind('/');
parsed.member.resize(separator == std::string::npos ? 0 : separator);
} else {
if (!parsed.member.empty()) {
parsed.member += '/';
}
parsed.member += component;
}
}
return parsed;
}
std::mutex g_readers_mutex;
std::unordered_map<std::filesystem::path, std::weak_ptr<ArchiveReader>> g_readers;
// Guest stacks can be only 64 KiB. Decompress on an ordinary host stack, then copy into guest
// memory on the caller so GPU-tracked writes use its fault handler. A single worker bounds host
// threads and avoids duplicating the archive's own block cache. Requests live on waiting callers.
class ArchiveIoThread {
public:
static ArchiveIoThread& Instance() {
// Guest exit can run static destructors while other guest threads are still reading.
// Keep the worker alive until process termination, as with the guest threads themselves.
static auto* instance = new ArchiveIoThread;
return *instance;
}
uint64_t Read(ArchiveReader& reader, uint64_t id, uint64_t offset, uint32_t size, void* data) {
Request request {reader, id, offset, size, data};
std::unique_lock lock(m_mutex);
if (m_last != nullptr) {
m_last->next = &request;
} else {
m_first = &request;
}
m_last = &request;
m_wake.notify_one();
request.ready.wait(lock, [&] { return request.done; });
return request.result;
}
KYTY_CLASS_NO_COPY(ArchiveIoThread);
private:
struct Request {
ArchiveReader& reader;
uint64_t id;
uint64_t offset;
uint32_t size;
void* data;
Request* next = nullptr;
uint64_t result = 0;
bool done = false;
std::condition_variable ready {};
};
ArchiveIoThread() {
std::thread([this] { Loop(); }).detach();
}
void Loop() {
for (;;) {
Request* request;
{
std::unique_lock lock(m_mutex);
m_wake.wait(lock, [this] { return m_first != nullptr; });
request = m_first;
m_first = request->next;
if (m_first == nullptr) {
m_last = nullptr;
}
}
const auto result =
request->reader.Read(request->id, request->offset, request->size, request->data);
{
std::lock_guard lock(m_mutex);
request->result = result;
request->done = true;
// Notify before releasing the mutex: the waiting caller owns the condition
// variable.
request->ready.notify_one();
}
}
}
std::mutex m_mutex;
std::condition_variable m_wake;
Request* m_first = nullptr;
Request* m_last = nullptr;
};
} // namespace
struct ArchiveFile::Private {
std::shared_ptr<ArchiveReader> reader;
ArchiveReader::Entry entry;
uint64_t position = 0;
};
ArchiveFile::ArchiveFile(std::unique_ptr<Private> p): m_p(std::move(p)) {}
ArchiveFile::~ArchiveFile() = default;
uint64_t ArchiveFile::Size() const {
return m_p->entry.size;
}
uint64_t ArchiveFile::Tell() const {
return m_p->position;
}
bool ArchiveFile::Seek(uint64_t offset) {
m_p->position = offset;
return true;
}
void ArchiveFile::Read(void* data, uint32_t size, uint32_t* bytes_read) {
#if defined(KYTY_ARCHIVE_PTHREAD_CANCELLATION)
// A pending cancellation must not destroy the waiting request or its TLS buffer while the
// worker still uses them. Restore the caller's state only after the copy is complete.
int cancellation_state = PTHREAD_CANCEL_ENABLE;
pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &cancellation_state);
#endif
constexpr uint32_t chunk_size = 1u << 20u;
thread_local std::vector<uint8_t> chunk;
uint32_t read = 0;
if (data != nullptr && size != 0 && m_p->position < Size()) {
const auto requested =
static_cast<uint32_t>(std::min<uint64_t>(size, Size() - m_p->position));
if (chunk.size() < std::min(chunk_size, requested)) {
chunk.resize(std::min(chunk_size, requested));
}
while (read < requested) {
const auto step = std::min(chunk_size, requested - read);
const auto got = ArchiveIoThread::Instance().Read(*m_p->reader, m_p->entry.id,
m_p->position, step, chunk.data());
if (got == 0 || got > step) {
break;
}
std::memcpy(static_cast<uint8_t*>(data) + read, chunk.data(), got);
read += static_cast<uint32_t>(got);
m_p->position += got;
if (got < step) {
break;
}
}
}
if (bytes_read != nullptr) {
*bytes_read = read;
}
#if defined(KYTY_ARCHIVE_PTHREAD_CANCELLATION)
pthread_setcancelstate(cancellation_state, nullptr);
#endif
}
std::filesystem::path MakeArchivePath(const std::filesystem::path& archive,
const std::filesystem::path& member) {
auto root = archive;
root += "!";
return member.empty() ? root : root / member.relative_path();
}
bool IsSupportedArchive(const std::filesystem::path& path) {
return FindFormat(path.native()) != nullptr;
}
bool IsArchivePath(const std::filesystem::path& path) {
return FindMarker(path.native()) != NativeView::npos;
}
std::filesystem::path GetArchiveHostPath(const std::filesystem::path& path) {
const auto marker = FindMarker(path.native());
return marker == NativeView::npos ? std::filesystem::path {}
: std::filesystem::path(path.native().substr(0, marker));
}
std::shared_ptr<ArchiveReader> OpenArchive(const std::filesystem::path& path) {
const auto archive = IsArchivePath(path) ? GetArchiveHostPath(path) : path;
const auto* format = FindFormat(archive.native());
if (format == nullptr) {
return {};
}
std::error_code error;
auto key = std::filesystem::absolute(archive, error);
if (error) {
return {};
}
{
std::lock_guard lock(g_readers_mutex);
if (auto it = g_readers.find(key); it != g_readers.end()) {
if (auto reader = it->second.lock()) {
return reader;
}
}
}
// Do not hold the cache mutex while opening an unrelated archive from disk.
auto reader = format->open(archive);
if (reader == nullptr) {
return {};
}
std::lock_guard lock(g_readers_mutex);
std::erase_if(g_readers, [](const auto& entry) { return entry.second.expired(); });
auto& cached = g_readers[key];
if (auto existing = cached.lock()) {
return existing;
}
cached = reader;
return reader;
}
std::optional<File::Info> GetArchiveInfo(const std::filesystem::path& path) {
const auto parsed = ParsePath(path);
if (parsed) {
if (auto reader = OpenArchive(parsed->archive)) {
if (auto entry = reader->Find(parsed->member)) {
return File::Info {entry->is_file, entry->size};
}
}
}
return {};
}
std::vector<File::DirEntry> GetArchiveDirEntries(const std::filesystem::path& path) {
const auto parsed = ParsePath(path);
if (parsed) {
if (auto reader = OpenArchive(parsed->archive)) {
return reader->List(parsed->member);
}
}
return {};
}
std::unique_ptr<ArchiveFile> OpenArchiveFile(const std::filesystem::path& path) {
auto parsed = ParsePath(path);
if (!parsed) {
return {};
}
auto reader = OpenArchive(parsed->archive);
const auto entry = reader ? reader->Find(parsed->member) : std::nullopt;
if (!entry || !entry->is_file) {
return {};
}
auto p = std::make_unique<ArchiveFile::Private>();
p->reader = std::move(reader);
p->entry = *entry;
return std::unique_ptr<ArchiveFile>(new ArchiveFile(std::move(p)));
}
} // namespace Common
+50
View File
@@ -0,0 +1,50 @@
#ifndef KYTY_COMMON_ARCHIVE_H_
#define KYTY_COMMON_ARCHIVE_H_
#include "common/file.h"
#include <cstdint>
#include <filesystem>
#include <memory>
#include <vector>
namespace Common {
class ArchiveReader;
class ArchiveFile {
public:
~ArchiveFile();
[[nodiscard]] uint64_t Size() const;
[[nodiscard]] uint64_t Tell() const;
bool Seek(uint64_t offset);
void Read(void* data, uint32_t size, uint32_t* bytes_read);
KYTY_CLASS_NO_COPY(ArchiveFile);
private:
struct Private;
explicit ArchiveFile(std::unique_ptr<Private> p);
std::unique_ptr<Private> m_p;
friend std::unique_ptr<ArchiveFile> OpenArchiveFile(const std::filesystem::path& path);
};
// Virtual paths use "archive.ext!/member" and retain native host-path encoding.
[[nodiscard]] std::filesystem::path MakeArchivePath(const std::filesystem::path& archive,
const std::filesystem::path& member = {});
[[nodiscard]] bool IsSupportedArchive(const std::filesystem::path& path);
[[nodiscard]] bool IsArchivePath(const std::filesystem::path& path);
[[nodiscard]] std::optional<File::Info> GetArchiveInfo(const std::filesystem::path& path);
[[nodiscard]] std::vector<File::DirEntry> GetArchiveDirEntries(const std::filesystem::path& path);
[[nodiscard]] std::filesystem::path GetArchiveHostPath(const std::filesystem::path& path);
[[nodiscard]] std::unique_ptr<ArchiveFile> OpenArchiveFile(const std::filesystem::path& path);
// Retain this handle while a mount or metadata scan is active to reuse the index and block cache.
// Accepts either a host archive filename or a virtual path inside the archive.
[[nodiscard]] std::shared_ptr<ArchiveReader> OpenArchive(const std::filesystem::path& path);
} // namespace Common
#endif /* KYTY_COMMON_ARCHIVE_H_ */
+17
View File
@@ -23,6 +23,7 @@ void Load(const ConfigOptions& cfg) {
EXIT_IF(g_config == nullptr);
EXIT_IF(cfg.user_name.empty() || cfg.user_name.size() > MAX_USER_NAME_LENGTH);
EXIT_IF(!IsConfiguredUserIdValid(cfg.user_id));
EXIT_IF(cfg.controller_speaker_volume > 100 || cfg.controller_vibration_intensity > 100);
*g_config = cfg;
}
@@ -47,6 +48,18 @@ const std::string& GetAudioInputDevice() {
return g_config->audio_input_device;
}
const std::optional<ControllerColor>& GetControllerColor() {
return g_config->controller_color;
}
uint32_t GetControllerSpeakerVolume() {
return g_config->controller_speaker_volume;
}
uint32_t GetControllerVibrationIntensity() {
return g_config->controller_vibration_intensity;
}
PresentMode GetPresentMode() {
return g_config->present_mode;
}
@@ -59,6 +72,10 @@ bool FullscreenEnabled() {
return g_config->fullscreen_enabled;
}
bool HideCursorEnabled() {
return g_config->hide_cursor_enabled;
}
bool VrEnabled() {
return g_config->vr_enabled;
}
+11
View File
@@ -4,7 +4,9 @@
#include "common/common.h"
#include <cstddef>
#include <array>
#include <filesystem>
#include <optional>
#include <string>
#include <vector>
@@ -26,6 +28,7 @@ enum class LogDirection { Silent, Console, File };
enum class PresentMode { Fifo, Mailbox, Immediate };
using Keymap = std::vector<std::string>;
using ControllerColor = std::array<uint8_t, 3>;
constexpr uint32_t DEFAULT_CONSOLE_LANGUAGE = 1;
constexpr uint32_t MAX_CONSOLE_LANGUAGE = 29;
@@ -44,9 +47,13 @@ struct ConfigOptions {
std::string user_name = "Kyty";
int32_t user_id = DEFAULT_USER_ID;
std::string audio_input_device;
std::optional<ControllerColor> controller_color;
uint32_t controller_speaker_volume = 100;
uint32_t controller_vibration_intensity = 100;
PresentMode present_mode = PresentMode::Mailbox;
int32_t gpu_index = -1;
bool fullscreen_enabled = false;
bool hide_cursor_enabled = false;
bool vr_enabled = false;
bool amd_cpu_enabled = false;
uint32_t vblank_frequency = 60;
@@ -81,9 +88,13 @@ uint32_t GetScreenHeight();
const std::string& GetUserName();
int32_t GetUserId();
const std::string& GetAudioInputDevice();
const std::optional<ControllerColor>& GetControllerColor();
uint32_t GetControllerSpeakerVolume();
uint32_t GetControllerVibrationIntensity();
PresentMode GetPresentMode();
int32_t GetGpuIndex();
bool FullscreenEnabled();
bool HideCursorEnabled();
bool VrEnabled();
bool AmdCpuEnabled();
uint32_t GetVblankFrequency();
+142 -63
View File
@@ -1,5 +1,6 @@
#include "common/file.h"
#include "common/archive.h"
#include "common/assert.h"
#include "common/common.h"
#include "common/platform/sysFileIO.h"
@@ -61,7 +62,8 @@ SysFileTimeStruct DateTimeToFileTimeUtc(const DateTime& date_time) {
} // namespace
struct File::FilePrivate {
sys_file_t* f;
sys_file_t* f;
std::unique_ptr<ArchiveFile> archive;
};
File::File(): m_p(std::make_unique<FilePrivate>()) {
@@ -85,11 +87,15 @@ File::File(const std::filesystem::path& name, Mode mode): m_p(std::make_unique<F
}
bool File::IsInvalid() const {
return m_p->f == nullptr;
return m_p->f == nullptr && m_p->archive == nullptr;
}
bool File::Create(const std::filesystem::path& name) {
EXIT_IF(m_p->f != nullptr);
EXIT_IF(!IsInvalid());
if (IsArchivePath(name)) {
return false;
}
m_file_name = name;
@@ -104,9 +110,16 @@ bool File::Create(const std::filesystem::path& name) {
}
bool File::Open(const std::filesystem::path& name, Mode mode) {
EXIT_IF(m_p->f != nullptr);
EXIT_IF(!IsInvalid());
m_file_name = name;
if (IsArchivePath(name)) {
if (mode != Mode::Read) {
return false;
}
m_p->archive = OpenArchiveFile(name);
return m_p->archive != nullptr;
}
switch (mode) {
case Mode::Read: m_p->f = SysFileOpenR(name); break;
@@ -124,7 +137,7 @@ bool File::Open(const std::filesystem::path& name, Mode mode) {
}
bool File::OpenInMem(void* buf, uint32_t buf_size) {
EXIT_IF(m_p->f != nullptr);
EXIT_IF(!IsInvalid());
m_p->f = SysFileOpen(static_cast<uint8_t*>(buf), buf_size);
@@ -137,7 +150,7 @@ bool File::OpenInMem(void* buf, uint32_t buf_size) {
}
bool File::CreateInMem() {
EXIT_IF(m_p->f != nullptr);
EXIT_IF(!IsInvalid());
m_p->f = SysFileCreate();
@@ -150,6 +163,7 @@ bool File::CreateInMem() {
}
void File::Close() {
m_p->archive.reset();
if (m_p->f != nullptr) {
SysFileClose(m_p->f);
m_p->f = nullptr;
@@ -157,7 +171,11 @@ void File::Close() {
}
uint64_t File::Size() const {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
return m_p->archive->Size();
}
if (m_p->f == nullptr) {
return 0;
@@ -167,41 +185,76 @@ uint64_t File::Size() const {
}
uint64_t File::Remaining() const {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
return Size() - Tell();
const auto size = Size();
const auto position = Tell();
return position < size ? size - position : 0;
}
uint64_t File::Size(const std::filesystem::path& name) {
if (IsArchivePath(name)) {
const auto info = GetArchiveInfo(name);
return info ? info->size : 0;
}
return SysFileSize(name);
}
std::optional<File::Info> File::GetInfo(const std::filesystem::path& name) {
if (IsArchivePath(name)) {
return GetArchiveInfo(name);
}
Info info {};
return SysFileGetInfo(name, &info.is_file, &info.size) ? std::optional(info) : std::nullopt;
}
bool File::Seek(uint64_t offset) {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
return m_p->archive->Seek(offset);
}
return SysFileSeek(*m_p->f, offset);
}
bool File::Truncate(uint64_t size) {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
return false;
}
return SysFileTruncate(*m_p->f, size);
}
bool File::Unlink() {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
return false;
}
return SysFileUnlink(*m_p->f, m_file_name);
}
uint64_t File::Tell() const {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
return m_p->archive->Tell();
}
return SysFileTell(*m_p->f);
}
void File::Read(void* data, uint32_t size, uint32_t* bytes_read) {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
m_p->archive->Read(data, size, bytes_read);
return;
}
if (m_p->f != nullptr) {
SysFileRead(data, size, *m_p->f, bytes_read);
@@ -209,7 +262,14 @@ void File::Read(void* data, uint32_t size, uint32_t* bytes_read) {
}
void File::Write(const void* data, uint32_t size, uint32_t* bytes_written) {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
if (bytes_written != nullptr) {
*bytes_written = 0;
}
return;
}
SysFileWrite(data, size, *m_p->f, bytes_written);
}
@@ -257,24 +317,41 @@ static std::filesystem::path WithoutTrailingSeparator(
bool File::IsDirectoryExisting(const std::filesystem::path& path) {
if (IsArchivePath(path)) {
const auto info = GetArchiveInfo(path);
return info && !info->is_file;
}
return SysFileIsDirectoryExisting(WithoutTrailingSeparator(path));
}
bool File::IsFileExisting(const std::filesystem::path& name) {
if (IsArchivePath(name)) {
const auto info = GetArchiveInfo(name);
return info && info->is_file;
}
return SysFileIsFileExisting(name);
}
bool File::CreateDirectory(const std::filesystem::path& path) {
if (IsArchivePath(path)) {
return false;
}
return SysFileCreateDirectory(
WithoutTrailingSeparator(path));
}
bool File::DeleteDirectory(const std::filesystem::path& path) {
if (IsArchivePath(path)) {
return false;
}
return SysFileDeleteDirectory(
WithoutTrailingSeparator(path));
}
bool File::CreateDirectories(const std::filesystem::path& path) {
if (IsArchivePath(path)) {
return false;
}
const auto normalized_path =
WithoutTrailingSeparator(path);
@@ -297,6 +374,9 @@ bool File::CreateDirectories(const std::filesystem::path& path) {
}
bool File::DeleteDirectories(const std::filesystem::path& path) {
if (IsArchivePath(path)) {
return false;
}
const auto normalized_path =
WithoutTrailingSeparator(path);
@@ -322,11 +402,18 @@ bool File::DeleteDirectories(const std::filesystem::path& path) {
bool File::DeleteFile(
const std::filesystem::path& name) // @suppress("Member declaration not found")
{
if (IsArchivePath(name)) {
return false;
}
return SysFileDeleteFile(name);
}
bool File::Flush() {
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
return true;
}
return SysFileFlush(*m_p->f);
}
@@ -348,6 +435,9 @@ std::vector<std::byte> File::ReadWholeBuffer() {
}
DateTime File::GetLastAccessTimeUTC(const std::filesystem::path& name) {
if (IsArchivePath(name)) {
return GetLastAccessTimeUTC(GetArchiveHostPath(name));
}
SysTimeStruct t {};
SysFileToSystemTimeUtc(SysFileGetLastAccessTimeUtc(name), t);
@@ -359,6 +449,9 @@ DateTime File::GetLastAccessTimeUTC(const std::filesystem::path& name) {
}
DateTime File::GetLastWriteTimeUTC(const std::filesystem::path& name) {
if (IsArchivePath(name)) {
return GetLastWriteTimeUTC(GetArchiveHostPath(name));
}
SysTimeStruct t {};
SysFileToSystemTimeUtc(SysFileGetLastWriteTimeUtc(name), t);
@@ -373,6 +466,10 @@ void File::GetLastAccessAndWriteTimeUTC(const std::filesystem::path& name, DateT
DateTime* write) {
EXIT_IF(access == nullptr);
EXIT_IF(write == nullptr);
if (IsArchivePath(name)) {
GetLastAccessAndWriteTimeUTC(GetArchiveHostPath(name), access, write);
return;
}
SysTimeStruct at {};
SysTimeStruct wt {};
@@ -399,7 +496,12 @@ void File::GetLastAccessAndWriteTimeUTC(DateTime* access, DateTime* write) {
EXIT_IF(access == nullptr);
EXIT_IF(write == nullptr);
EXIT_IF(m_p->f == nullptr);
EXIT_IF(IsInvalid());
if (m_p->archive != nullptr) {
GetLastAccessAndWriteTimeUTC(m_file_name, access, write);
return;
}
if (m_p->f == nullptr) {
*access = DateTime();
@@ -429,62 +531,35 @@ void File::GetLastAccessAndWriteTimeUTC(DateTime* access, DateTime* write) {
}
bool File::SetLastAccessTimeUTC(const std::filesystem::path& name, const DateTime& dt) {
if (IsArchivePath(name)) {
return false;
}
auto f = DateTimeToFileTimeUtc(dt);
return SysFileSetLastAccessTimeUtc(name, f);
}
bool File::SetLastWriteTimeUTC(const std::filesystem::path& name, const DateTime& dt) {
if (IsArchivePath(name)) {
return false;
}
auto f = DateTimeToFileTimeUtc(dt);
return SysFileSetLastWriteTimeUtc(name, f);
}
bool File::SetLastAccessAndWriteTimeUTC(const std::filesystem::path& name, const DateTime& access,
const DateTime& write) {
if (IsArchivePath(name)) {
return false;
}
auto af = DateTimeToFileTimeUtc(access);
auto wf = DateTimeToFileTimeUtc(write);
return SysFileSetLastAccessAndWriteTimeUtc(name, af, wf);
}
std::vector<File::FindInfo> File::FindFiles(const std::filesystem::path& path) {
std::vector<sys_file_find_t> files;
SysFileFindFiles(path, files);
const auto normalized_path = path.lexically_normal();
std::vector<File::FindInfo> ret;
ret.reserve(files.size());
for (const auto& f: files) {
File::FindInfo r {};
r.path_with_name = f.path_with_name;
r.rel_path_with_name = f.path_with_name.lexically_normal().lexically_relative(normalized_path);
r.size = f.size;
SysTimeStruct at {};
SysTimeStruct wt {};
SysFileToSystemTimeUtc(f.last_access_time, at);
SysFileToSystemTimeUtc(f.last_write_time, wt);
if (!at.is_invalid && !wt.is_invalid) {
r.last_access_time = DateTime(Date(at.Year, at.Month, at.Day),
Time(at.Hour, at.Minute, at.Second, at.Milliseconds));
r.last_write_time = DateTime(Date(wt.Year, wt.Month, wt.Day),
Time(wt.Hour, wt.Minute, wt.Second, wt.Milliseconds));
}
// printf("%s, %s, %" PRIu64", %s, %s\n", r.path_with_name.c_str(),
// r.rel_path_with_name.c_str(), r.size, r.last_access_time.ToString().c_str(),
// r.last_write_time.ToString().c_str());
ret.push_back(std::move(r));
}
return ret;
}
std::vector<File::DirEntry> File::GetDirEntries(const std::filesystem::path& path) {
if (IsArchivePath(path)) {
return GetArchiveDirEntries(path);
}
std::vector<sys_dir_entry_t> files;
SysFileGetDents(path, files);
@@ -492,13 +567,8 @@ std::vector<File::DirEntry> File::GetDirEntries(const std::filesystem::path& pat
std::vector<File::DirEntry> ret;
ret.reserve(files.size());
for (const auto& f: files) {
File::DirEntry r {};
r.name = f.name;
r.is_file = f.is_file;
ret.push_back(std::move(r));
for (auto& f: files) {
ret.emplace_back(std::move(f.name), f.is_file);
}
return ret;
@@ -507,12 +577,18 @@ std::vector<File::DirEntry> File::GetDirEntries(const std::filesystem::path& pat
bool File::CopyFile(const std::filesystem::path& src,
const std::filesystem::path& dst) // @suppress("Member declaration not found")
{
if (IsArchivePath(src) || IsArchivePath(dst)) {
return false;
}
return SysFileCopyFile(src, dst);
}
bool File::RenameFile(const std::filesystem::path& src,
const std::filesystem::path& dst) // @suppress("Member declaration not found")
{
if (IsArchivePath(src) || IsArchivePath(dst)) {
return false;
}
if (IsFileExisting(dst)) {
DeleteFile(dst); // @suppress("Invalid arguments")
}
@@ -521,6 +597,9 @@ bool File::RenameFile(const std::filesystem::path& src,
}
void File::RemoveReadonly(const std::filesystem::path& name) {
if (IsArchivePath(name)) {
return;
}
SysFileRemoveReadonly(name);
}
+6 -9
View File
@@ -7,6 +7,7 @@
#include <cstddef>
#include <filesystem>
#include <memory>
#include <optional>
#include <string>
#include <vector>
@@ -30,18 +31,14 @@ class File {
public:
enum class Mode { Read, Write, ReadWrite, WriteRead };
struct FindInfo {
std::filesystem::path path_with_name;
std::filesystem::path rel_path_with_name;
DateTime last_access_time;
DateTime last_write_time;
uint64_t size;
};
struct DirEntry {
std::string name;
bool is_file;
};
struct Info {
bool is_file;
uint64_t size;
};
File();
explicit File(const std::filesystem::path& name);
@@ -80,6 +77,7 @@ public:
std::vector<std::byte> ReadWholeBuffer();
static uint64_t Size(const std::filesystem::path& name);
static std::optional<Info> GetInfo(const std::filesystem::path& name);
static bool IsDirectoryExisting(const std::filesystem::path& path);
static bool IsFileExisting(const std::filesystem::path& name);
@@ -98,7 +96,6 @@ public:
static bool SetLastAccessAndWriteTimeUTC(const std::filesystem::path& name,
const DateTime& access, const DateTime& write);
static std::vector<FindInfo> FindFiles(const std::filesystem::path& path);
static std::vector<DirEntry> GetDirEntries(const std::filesystem::path& path);
static bool CopyFile(const std::filesystem::path& src, const std::filesystem::path& dst);
static bool RenameFile(const std::filesystem::path& src, const std::filesystem::path& dst);
+1 -10
View File
@@ -18,14 +18,6 @@ enum sys_file_cache_type_t {
SYS_FILE_CACHE_SEQUENTIAL_SCAN = 2 // NOLINT(readability-identifier-naming)
};
// NOLINTNEXTLINE(readability-identifier-naming)
struct sys_file_find_t {
std::filesystem::path path_with_name;
SysFileTimeStruct last_access_time;
SysFileTimeStruct last_write_time;
uint64_t size;
};
// NOLINTNEXTLINE(readability-identifier-naming)
struct sys_dir_entry_t {
std::string name;
@@ -53,6 +45,7 @@ bool SysFileTruncate(sys_file_t& f, uint64_t size); // NOLINT(google-runt
bool SysFileUnlink(sys_file_t& f,
const std::filesystem::path& name); // NOLINT(google-runtime-references)
uint64_t SysFileSize(const std::filesystem::path& file_name);
bool SysFileGetInfo(const std::filesystem::path& name, bool* is_file, uint64_t* size);
bool SysFileIsError(sys_file_t& f); // NOLINT(google-runtime-references)
bool SysFileIsDirectoryExisting(const std::filesystem::path& path);
bool SysFileIsFileExisting(const std::filesystem::path& name);
@@ -75,8 +68,6 @@ bool SysFileSetLastWriteTimeUtc(const std::filesystem::path& name, SysFileTimeSt
bool SysFileSetLastAccessAndWriteTimeUtc(const std::filesystem::path& name,
SysFileTimeStruct& access, SysFileTimeStruct& write);
// NOLINTNEXTLINE(google-runtime-references)
void SysFileFindFiles(const std::filesystem::path& path, std::vector<sys_file_find_t>& out);
// NOLINTNEXTLINE(google-runtime-references)
void SysFileGetDents(const std::filesystem::path& path, std::vector<sys_dir_entry_t>& out);
bool SysFileCopyFile(const std::filesystem::path& src, const std::filesystem::path& dst);
bool SysFileRenameFile(const std::filesystem::path& src, const std::filesystem::path& dst);
+21 -69
View File
@@ -54,7 +54,7 @@ struct sys_file_t {
#endif
static std::filesystem::path get_internal_name(const std::filesystem::path& name) {
return name.is_absolute() ? name : (std::filesystem::path(".") / name);
return name.empty() || name.is_absolute() ? name : (std::filesystem::path(".") / name);
}
// Pass access-pattern hints to the host.
@@ -297,10 +297,21 @@ uint64_t SysFileSize(sys_file_t& f) {
return 0;
}
bool SysFileGetInfo(const std::filesystem::path& name, bool* is_file, uint64_t* size) {
const auto path = get_internal_name(name);
struct stat info {};
if (stat(path.c_str(), &info) != 0) {
return false;
}
*is_file = !S_ISDIR(info.st_mode);
*size = *is_file ? static_cast<uint64_t>(info.st_size) : 0;
return true;
}
uint64_t SysFileSize(const std::filesystem::path& file_name) {
sys_file_t* f = SysFileOpenR(file_name);
uint64_t size = SysFileSize(*f);
SysFileClose(f);
bool is_file;
uint64_t size = 0;
SysFileGetInfo(file_name, &is_file, &size);
return size;
}
@@ -353,29 +364,15 @@ bool SysFileIsError(sys_file_t& f) {
}
bool SysFileIsDirectoryExisting(const std::filesystem::path& path) {
auto real_name = get_internal_name(path);
auto real_name_str = real_name.string();
struct stat s {};
if (0 != stat(real_name_str.c_str(), &s)) {
return false;
}
return S_ISDIR(s.st_mode); // NOLINT
bool is_file;
uint64_t size;
return SysFileGetInfo(path, &is_file, &size) && !is_file;
}
bool SysFileIsFileExisting(const std::filesystem::path& name) {
auto real_name = get_internal_name(name);
auto real_name_str = real_name.string();
struct stat s {};
if (0 != stat(real_name_str.c_str(), &s)) {
return false;
}
return !S_ISDIR(s.st_mode); // NOLINT
bool is_file;
uint64_t size;
return SysFileGetInfo(name, &is_file, &size) && is_file;
}
bool SysFileCreateDirectory(const std::filesystem::path& path) {
@@ -610,51 +607,6 @@ bool SysFileSetLastAccessAndWriteTimeUtc(const std::filesystem::path& name,
// }
}
// Recursively collect regular files.
void SysFileFindFiles(const std::filesystem::path& path, std::vector<sys_file_find_t>& out) {
auto real_path = get_internal_name(path);
DIR* dir = opendir(real_path.string().c_str());
if (dir == nullptr) {
return;
}
for (const dirent* entry = readdir(dir); entry != nullptr; entry = readdir(dir)) {
const std::string file_name(entry->d_name);
if (file_name == "." || file_name == "..") {
continue;
}
auto child = real_path / file_name;
struct stat s {};
// lstat, so a symlink is never followed into a cycle during the recursive walk.
if (0 != lstat(child.string().c_str(), &s)) {
continue;
}
if (S_ISDIR(s.st_mode)) {
SysFileFindFiles(child, out);
} else if (S_ISREG(s.st_mode)) {
sys_file_find_t r {};
r.path_with_name = child;
r.size = static_cast<uint64_t>(s.st_size);
r.last_access_time.is_invalid = false;
r.last_access_time.time = s.st_atime;
r.last_access_time.nanos = KYTY_STAT_ATIME_NS(s);
r.last_write_time.is_invalid = false;
r.last_write_time.time = s.st_mtime;
r.last_write_time.nanos = KYTY_STAT_MTIME_NS(s);
out.push_back(r);
}
}
closedir(dir);
}
// Keep "." and ".." to match FindFirstFileW.
void SysFileGetDents(const std::filesystem::path& path, std::vector<sys_dir_entry_t>& out) {
auto real_path = get_internal_name(path);
+27 -61
View File
@@ -57,10 +57,10 @@ static DWORD GetCacheAccessType(sys_file_cache_type_t t) {
}
if (t == SYS_FILE_CACHE_SEQUENTIAL_SCAN) {
return SYS_FILE_CACHE_SEQUENTIAL_SCAN;
return FILE_FLAG_SEQUENTIAL_SCAN;
}
return SYS_FILE_CACHE_AUTO;
return FILE_ATTRIBUTE_NORMAL;
}
void SysFileRead(void* data, uint32_t size, sys_file_t& f, uint32_t* bytes_read) {
@@ -266,19 +266,27 @@ uint64_t SysFileSize(sys_file_t& f) {
return 0;
}
uint64_t SysFileSize(const std::filesystem::path& file_name) {
LARGE_INTEGER s;
WIN32_FILE_ATTRIBUTE_DATA a;
auto wide = file_name.wstring();
if (GetFileAttributesExW(wide.c_str(), GetFileExInfoStandard, &a) == 0) {
return 0;
bool SysFileGetInfo(const std::filesystem::path& name, bool* is_file, uint64_t* size) {
const bool directory_path = !name.empty() && !name.has_filename() && name != name.root_path();
WIN32_FILE_ATTRIBUTE_DATA info {};
if (GetFileAttributesExW(directory_path ? name.parent_path().c_str() : name.c_str(),
GetFileExInfoStandard, &info) == 0) {
return false;
}
const bool file = (info.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) == 0;
if (directory_path && file) {
return false;
}
*is_file = file;
*size = *is_file ? ((static_cast<uint64_t>(info.nFileSizeHigh) << 32u) | info.nFileSizeLow) : 0;
return true;
}
s.HighPart = static_cast<LONG>(a.nFileSizeHigh);
s.LowPart = a.nFileSizeLow;
return s.QuadPart;
uint64_t SysFileSize(const std::filesystem::path& file_name) {
bool is_file;
uint64_t size = 0;
SysFileGetInfo(file_name, &is_file, &size);
return size;
}
bool SysFileTruncate(sys_file_t& f, uint64_t size) {
@@ -346,17 +354,15 @@ bool SysFileIsError(sys_file_t& f) {
}
bool SysFileIsDirectoryExisting(const std::filesystem::path& path) {
auto wide = path.wstring();
DWORD a = GetFileAttributesW(wide.c_str());
return a != INVALID_FILE_ATTRIBUTES &&
((a & static_cast<DWORD>(FILE_ATTRIBUTE_DIRECTORY)) != 0u);
bool is_file;
uint64_t size;
return SysFileGetInfo(path, &is_file, &size) && !is_file;
}
bool SysFileIsFileExisting(const std::filesystem::path& name) {
auto wide = name.wstring();
DWORD a = GetFileAttributesW(wide.c_str());
return a != INVALID_FILE_ATTRIBUTES &&
((a & static_cast<DWORD>(FILE_ATTRIBUTE_DIRECTORY)) == 0u);
bool is_file;
uint64_t size;
return SysFileGetInfo(name, &is_file, &size) && is_file;
}
bool SysFileCreateDirectory(const std::filesystem::path& path) {
@@ -476,46 +482,6 @@ bool SysFileSetLastAccessAndWriteTimeUtc(const std::filesystem::path& name,
return ok;
}
void SysFileFindFiles(const std::filesystem::path& path, std::vector<sys_file_find_t>& out) {
const auto pattern = path / L"*";
WIN32_FIND_DATAW data {};
HANDLE h = FindFirstFileW(pattern.c_str(), &data);
if (h == INVALID_HANDLE_VALUE) {
return;
}
do {
std::filesystem::path file_name(data.cFileName);
if (file_name == L"." || file_name == L"..") {
continue;
}
const auto full_path = path / file_name;
if ((data.dwFileAttributes & static_cast<DWORD>(FILE_ATTRIBUTE_DIRECTORY)) != 0u) {
SysFileFindFiles(full_path, out);
} else {
sys_file_find_t r {};
r.path_with_name = full_path;
r.size = (static_cast<uint64_t>(data.nFileSizeHigh) << 32u) +
static_cast<uint64_t>(data.nFileSizeLow);
r.last_access_time.is_invalid = false;
r.last_access_time.time = data.ftLastAccessTime;
r.last_write_time.is_invalid = false;
r.last_write_time.time = data.ftLastWriteTime;
out.push_back(std::move(r));
}
} while (FindNextFileW(h, &data) != 0);
FindClose(h);
}
void SysFileGetDents(const std::filesystem::path& path, std::vector<sys_dir_entry_t>& out) {
const auto pattern = path / L"*";
-39
View File
@@ -2,52 +2,13 @@
#include "common/emulatorConfig.h"
#include <algorithm>
#include <common/TracyProtocol.hpp>
#include <common/TracyVersion.hpp>
#include <cstdio>
#include <tracy/Tracy.hpp>
#include <vector>
namespace {
thread_local std::vector<Profiler::ScopedBlock*> g_block_stack;
void RemoveBlock(Profiler::ScopedBlock* block) {
auto block_it = std::find(g_block_stack.rbegin(), g_block_stack.rend(), block);
if (block_it != g_block_stack.rend()) {
g_block_stack.erase(std::next(block_it).base());
}
}
} // namespace
namespace Profiler {
ScopedBlock::ScopedBlock(const tracy::SourceLocationData* source_location) {
if (tracy::ProfilerAvailable()) {
m_zone.emplace(source_location, TRACY_CALLSTACK, true);
g_block_stack.push_back(this);
}
}
ScopedBlock::~ScopedBlock() {
End();
}
void ScopedBlock::End() {
if (m_zone.has_value()) {
m_zone.reset();
RemoveBlock(this);
}
}
void EndBlock() {
if (!g_block_stack.empty()) {
g_block_stack.back()->End();
}
}
void SetThreadName(const char* name) {
if (tracy::ProfilerAvailable() && name != nullptr) {
tracy::SetThreadName(name);
+4 -29
View File
@@ -4,7 +4,6 @@
#include "common/common.h"
#include <cstdint>
#include <optional>
#include <tracy/Tracy.hpp> // IWYU pragma: export
namespace profiler::colors {
@@ -23,22 +22,6 @@ inline constexpr uint32_t DeepOrangeA200 = 0xff6e40;
namespace Profiler {
class ScopedBlock {
public:
explicit ScopedBlock(const tracy::SourceLocationData* source_location);
ScopedBlock(const ScopedBlock&) = delete;
ScopedBlock& operator=(const ScopedBlock&) = delete;
ScopedBlock(ScopedBlock&&) = delete;
ScopedBlock& operator=(ScopedBlock&&) = delete;
~ScopedBlock();
void End();
private:
std::optional<tracy::ScopedZone> m_zone;
};
void EndBlock();
void SetThreadName(const char* name);
void Initialize();
@@ -66,19 +49,11 @@ struct Lifecycle {
kyty_profiler_source_location_, line) {name, TracyFunction, TracyFile, \
static_cast<uint32_t>(line), \
KYTY_PROFILER_COLOR_OR_DEFAULT(0, __VA_ARGS__)}; \
Profiler::ScopedBlock KYTY_PROFILER_CONCAT(kyty_profiler_block_, line)( \
&KYTY_PROFILER_CONCAT(kyty_profiler_source_location_, line))
tracy::ScopedZone KYTY_PROFILER_CONCAT(kyty_profiler_block_, line)( \
&KYTY_PROFILER_CONCAT(kyty_profiler_source_location_, line), \
TRACY_CALLSTACK, tracy::ProfilerAvailable())
#define KYTY_PROFILER_FUNCTION(...) KYTY_PROFILER_FUNCTION_IMPL(__LINE__ __VA_OPT__(, ) __VA_ARGS__)
#define KYTY_PROFILER_FUNCTION_IMPL(line, ...) \
static constexpr tracy::SourceLocationData KYTY_PROFILER_CONCAT( \
kyty_profiler_source_location_, line) {nullptr, TracyFunction, TracyFile, \
static_cast<uint32_t>(line), \
KYTY_PROFILER_COLOR_OR_DEFAULT(0, __VA_ARGS__)}; \
Profiler::ScopedBlock KYTY_PROFILER_CONCAT(kyty_profiler_block_, line)( \
&KYTY_PROFILER_CONCAT(kyty_profiler_source_location_, line))
#define KYTY_PROFILER_END_BLOCK Profiler::EndBlock()
#define KYTY_PROFILER_FUNCTION(...) KYTY_PROFILER_BLOCK(nullptr __VA_OPT__(, ) __VA_ARGS__)
#define KYTY_PROFILER_THREAD(name) Profiler::SetThreadName(name)
+2 -40
View File
@@ -2,7 +2,6 @@
#include "common/assert.h"
#include <algorithm>
#include <atomic>
#include <cerrno>
#include <chrono> // IWYU pragma: keep
@@ -26,8 +25,7 @@
#ifdef KYTY_WIN_CS
#include <windows.h> // IWYU pragma: keep
// IWYU pragma: no_include <winbase.h>
constexpr DWORD KYTY_CS_SPIN_COUNT = 4000;
constexpr uint64_t KYTY_SLEEP_SPIN_LIMIT_100NS = 500; // 50 us
constexpr DWORD KYTY_CS_SPIN_COUNT = 4000;
#ifndef CREATE_WAITABLE_TIMER_HIGH_RESOLUTION
#define CREATE_WAITABLE_TIMER_HIGH_RESOLUTION 0x00000002
@@ -38,30 +36,6 @@ static void SleepHighResolution100ns(uint64_t units_100ns) {
return;
}
// Keep spinning only where a kernel transition is
// likely to cost more than the requested delay; ordinary millisecond sleeps use the
// per-thread high-resolution waitable timer below.
if (units_100ns <= KYTY_SLEEP_SPIN_LIMIT_100NS) {
LARGE_INTEGER frequency {};
LARGE_INTEGER start {};
if (QueryPerformanceFrequency(&frequency) != 0 && QueryPerformanceCounter(&start) != 0 &&
frequency.QuadPart > 0) {
const auto wait_ticks =
static_cast<LONGLONG>((static_cast<long double>(units_100ns) *
static_cast<long double>(frequency.QuadPart)) /
10000000.0L);
const auto deadline = start.QuadPart + std::max<LONGLONG>(wait_ticks, 1);
LARGE_INTEGER now {};
do {
if (QueryPerformanceCounter(&now) == 0) {
break;
}
YieldProcessor();
} while (now.QuadPart < deadline);
return;
}
}
thread_local HANDLE timer = CreateWaitableTimerExW(
nullptr, nullptr, CREATE_WAITABLE_TIMER_HIGH_RESOLUTION, TIMER_ALL_ACCESS);
if (timer == nullptr) {
@@ -131,14 +105,13 @@ static SleepConditionVariableCS_func_t ResolveSleepConditionVariableCS() {
#endif
#ifdef KYTY_POSIX_HIGH_RES_SLEEP
// Spin for very short waits; use an absolute deadline for longer waits.
// An absolute deadline preserves the requested sleep across signal interruptions.
static void SleepHighResolutionNanos(uint64_t nanos) {
if (nanos == 0) {
return;
}
constexpr uint64_t NANOS_PER_SEC = 1000000000;
constexpr uint64_t SPIN_LIMIT_NS = 50000; // below this a context switch dominates
timespec deadline {};
if (clock_gettime(CLOCK_MONOTONIC, &deadline) != 0) {
@@ -150,17 +123,6 @@ static void SleepHighResolutionNanos(uint64_t nanos) {
deadline.tv_sec += static_cast<time_t>(target_nsec / NANOS_PER_SEC);
deadline.tv_nsec = static_cast<long>(target_nsec % NANOS_PER_SEC);
if (nanos <= SPIN_LIMIT_NS) {
timespec now {};
do {
if (clock_gettime(CLOCK_MONOTONIC, &now) != 0) {
return;
}
} while (now.tv_sec < deadline.tv_sec ||
(now.tv_sec == deadline.tv_sec && now.tv_nsec < deadline.tv_nsec));
return;
}
while (clock_nanosleep(CLOCK_MONOTONIC, TIMER_ABSTIME, &deadline, nullptr) == EINTR) {
}
}
+1 -10
View File
@@ -38,16 +38,7 @@ bool Timer::IsPaused() const {
// return time in milliseconds
double Timer::GetTimeMs() const {
if (m_is_paused) {
return 1000.0 * (static_cast<double>(m_PauseTime - m_StartTime)) /
static_cast<double>(m_Frequency);
}
uint64_t current_time = 0;
SysQueryPerformanceCounter(&current_time);
return 1000.0 * (static_cast<double>(current_time - m_StartTime)) /
static_cast<double>(m_Frequency);
return GetTimeS() * 1000.0;
}
// return time in seconds
@@ -62,7 +62,6 @@ public:
void BufferInit();
void BufferFlush();
void BufferFlushAndWait();
void BufferWait();
HW::Context& GetCtx() { return m_ctx; }
HW::UserConfig& GetUcfg() { return m_ucfg; }
@@ -73,6 +72,9 @@ public:
void SetIndexBufferSize(uint32_t index_buffer_size);
void SetDrawIndirectArgsBaseAddress(uint64_t draw_indirect_args_base_addr);
void SetDispatchIndirectArgsBaseAddress(uint64_t dispatch_indirect_args_base_addr);
[[nodiscard]] uint64_t GetDispatchIndirectArgsBaseAddress() const {
return m_dispatch_indirect_args_base_addr;
}
void SetNumInstances(uint32_t num_instances);
void DrawIndex(DrawIndexArgs args);
void DrawIndexOffset(uint32_t index_offset, uint32_t index_count);
@@ -99,7 +101,7 @@ public:
void TriggerEopEventAtEndOfPipe(uint32_t interrupt_context_id);
void DispatchDirect(uint32_t thread_group_x, uint32_t thread_group_y, uint32_t thread_group_z,
uint32_t mode);
void DispatchIndirect(uint32_t data_offset, uint32_t mode);
void DispatchIndirect(uint64_t args_addr, uint32_t mode);
void WaitFlipDone(uint32_t video_out_handle, uint32_t display_buffer_index);
void TriggerEvent(uint32_t event_type, uint32_t event_index, uint64_t event_address = 0);
@@ -148,7 +150,6 @@ private:
void SuspendPm4();
CommandScheduler& GetScheduler() const { return m_renderer.GetCommandScheduler(); }
CommandBuffer& CurrentBuffer() { return GetScheduler().Current(); }
void CheckBuffer() const { GetScheduler().CheckActive(); }
RenderContext& m_renderer;
HW::Context m_ctx;
@@ -140,6 +140,8 @@ constexpr auto MakeShaderDispatchTable() {
g_hw_sh_func[Pm4::COMPUTE_PGM_RSRC3] = HwShSetCsRegisters;
g_hw_sh_func[Pm4::COMPUTE_PACE_ID] = HwShSetCsRegisters;
g_hw_sh_func[Pm4::SPI_GRAPHICS_SHADER_CONTROL_PS] = HwShSetRegisters;
g_hw_sh_func[Pm4::SPI_SHADER_USER_DATA_ADDR_LO_PS] = HwShSetRegisters;
g_hw_sh_func[Pm4::SPI_SHADER_USER_DATA_ADDR_HI_PS] = HwShSetRegisters;
g_hw_sh_func[Pm4::SPI_GRAPHICS_SHADER_CONTROL_GS] = HwShSetRegisters;
g_hw_sh_func[Pm4::SPI_SHADER_USER_DATA_ADDR_LO_GS] = HwShSetRegisters;
g_hw_sh_func[Pm4::SPI_SHADER_USER_DATA_ADDR_HI_GS] = HwShSetRegisters;
@@ -1331,26 +1331,13 @@ KYTY_CP_OP_PARSER(CpOpDispatchIndirect) {
EXIT_NOT_IMPLEMENTED(cmd_id != 0xc0011600 && cmd_id != 0xc0021600);
if (cmd_id == 0xc0021600) {
struct DispatchIndirectArgs {
uint32_t thread_group_x;
uint32_t thread_group_y;
uint32_t thread_group_z;
};
auto* args = reinterpret_cast<const DispatchIndirectArgs*>(
buffer[0] | (static_cast<uint64_t>(buffer[1]) << 32u));
uint32_t mode = buffer[2];
EXIT_NOT_IMPLEMENTED(args == nullptr);
cp.DispatchDirect(args->thread_group_x, args->thread_group_y, args->thread_group_z, mode);
cp.DispatchIndirect(buffer[0] | (static_cast<uint64_t>(buffer[1]) << 32u), buffer[2]);
return 3;
}
uint32_t data_offset = buffer[0];
uint32_t mode = buffer[1];
cp.DispatchIndirect(data_offset, mode);
const auto base_addr = cp.GetDispatchIndirectArgsBaseAddress();
EXIT_NOT_IMPLEMENTED(base_addr == 0);
cp.DispatchIndirect(base_addr + buffer[0], buffer[1]);
return 2;
}
@@ -2017,6 +2004,14 @@ KYTY_CP_OP_PARSER(CpOpIndirectCxRegs) {
auto pfunc = g_hw_ctx_indirect_func[cmd_offset & (Pm4::CX_NUM - 1)];
if (pfunc == nullptr) {
if (raw_cmd_offset == 0x24au && value == 0u) {
static std::atomic_flag logged = ATOMIC_FLAG_INIT;
if (!logged.test_and_set(std::memory_order_relaxed)) {
Log::WriteToConsoleAndLog(
"\t diagnostic: ignoring indirect CX {0x24a, 0}; hardware effect unresolved\n");
}
continue;
}
EXIT("unknown cx reg at %05" PRIx32 ": 0x%" PRIx32 "\n", num_dw - dw, cmd_offset);
}
@@ -2271,7 +2266,6 @@ KYTY_CP_OP_PARSER(CpOpReleaseMem) {
const bool gl2_writeback = ((gcr_cntl & GcrGl2Writeback) != 0);
auto trigger_interrupt = [&]() {
bool queued = false;
switch (interrupt_selector) {
case 0x00:
case 0x03: break;
@@ -2279,13 +2273,10 @@ KYTY_CP_OP_PARSER(CpOpReleaseMem) {
case 0x02:
case 0x04:
cp.TriggerEopEventAtEndOfPipe(interrupt_context_id);
queued = true;
cp.BufferFlush();
break;
default: EXIT("unknown release_mem interrupt selector\n");
}
if (queued) {
cp.BufferFlush();
}
};
if (data_sel == 0 || interrupt_selector == 4) {
@@ -2323,7 +2314,9 @@ KYTY_CP_OP_PARSER(CpOpReleaseMem) {
cp.WriteAtEndOfPipe32(cache_policy, event_write_dest, eop_event_type, cache_action,
event_index, event_source, dst_gpu_addr, static_cast<uint32_t>(value),
interrupt_selector, interrupt_context_id);
cp.BufferFlush();
if (interrupt_selector == 0x01 || interrupt_selector == 0x02) {
cp.BufferFlush();
}
return 7;
}
@@ -3629,6 +3622,18 @@ void GraphicsInitJmpTablesShIndirect() {
g_hw_sh_indirect_func[Pm4::SPI_GRAPHICS_SHADER_CONTROL_PS] = [](KYTY_HW_SH_INDIRECT_ARGS) {
HwShIgnoreShaderRegister(cmd_offset, value);
};
for (uint32_t offset = Pm4::SPI_SHADER_USER_DATA_ADDR_LO_PS;
offset <= Pm4::SPI_SHADER_USER_DATA_ADDR_HI_PS; offset++) {
g_hw_sh_indirect_func[offset] = [](KYTY_HW_SH_INDIRECT_ARGS) {
cp.GetShCtx().SetPsUserDataAddress(cmd_offset - Pm4::SPI_SHADER_USER_DATA_ADDR_LO_PS,
value);
};
}
for (uint32_t offset = 4; offset <= 5; offset++) {
g_hw_sh_indirect_func[offset] = [](KYTY_HW_SH_INDIRECT_ARGS) {
cp.GetShCtx().SetPsAuxiliaryTableAddress(cmd_offset - 4, value);
};
}
g_hw_sh_indirect_func[Pm4::SPI_SHADER_PACE_ID_GS] = [](KYTY_HW_SH_INDIRECT_ARGS) {
HwShIgnoreShaderRegister(cmd_offset, value);
};
+2 -2
View File
@@ -61,8 +61,8 @@ constexpr FormatInfo kFormatInfo[] = {
{BufferFormat::k32_32_32_32UInt, 16, 0, 16, true, true},
{BufferFormat::k32_32_32_32SInt, 16, 0, 16, true, false, true},
{BufferFormat::k32_32_32_32Float, 16, 0, 16, true, false},
{BufferFormat::k8Srgb, 1, 0, 0, true, false},
{BufferFormat::k8_8Srgb, 2, 0, 0, true, false},
{BufferFormat::k8Srgb, 1, 0, 1, true, false},
{BufferFormat::k8_8Srgb, 2, 0, 2, true, false},
{BufferFormat::k8_8_8_8Srgb, 4, 0, 4, true, false},
{BufferFormat::k9_9_9_5Float, 4, 0, 0, true, false},
{BufferFormat::k5_6_5UNorm, 2, 0, 2, true, false},
+67 -213
View File
@@ -35,7 +35,6 @@ namespace Libs::Graphics {
static thread_local CommandProcessor* g_current_processor = nullptr;
static thread_local Pm4Execution* g_current_execution = nullptr;
static thread_local bool g_gpu_mutex_owned = false;
static thread_local bool g_gpu_thread = false;
static thread_local GuestGpu* g_gpu_state = nullptr;
@@ -54,27 +53,6 @@ struct DrawIndexedIndirectArgs {
uint32_t start_instance_location;
};
class GpuMutexLock final {
public:
explicit GpuMutexLock(Common::Mutex& mutex): m_mutex(mutex) {
if (g_gpu_mutex_owned) {
EXIT("recursive GPU mutex acquisition\n");
}
g_gpu_mutex_owned = true;
m_mutex.Lock();
}
~GpuMutexLock() {
if (!g_gpu_mutex_owned) {
EXIT("invalid GPU mutex release\n");
}
m_mutex.Unlock();
g_gpu_mutex_owned = false;
}
private:
Common::Mutex& m_mutex;
};
static bool GraphicsRunDebugDumpEnabled() {
return Config::GraphicsDebugDumpEnabled() &&
Config::GetPrintfDirection() != Config::LogDirection::Silent;
@@ -84,6 +62,7 @@ GuestGpu::GuestGpu(RenderContext& renderer): m_renderer(renderer) {
EXIT_NOT_IMPLEMENTED(!Common::Thread::IsMainThread());
GraphicsInitJmpTables();
m_gfx_cp = std::make_unique<CommandProcessor>(renderer, 0);
m_gfx_cp->Reset();
m_thread = std::jthread(ThreadRun, this);
}
@@ -160,20 +139,16 @@ void GuestGpu::Submit(std::span<const uint32_t> draw_commands,
if (draw_commands.empty()) {
return;
}
GpuMutexLock lock(m_submission_mutex);
Submission submission;
Submission submission;
submission.type = SubmissionType::Graphics;
submission.queue_id = 0;
submission.commands = draw_commands;
submission.constant_commands = constant_commands;
submission.reset_processor = m_graphics_done;
m_graphics_done = false;
Enqueue(std::move(submission));
}
void GuestGpu::SubmitCompute(uint32_t queue, std::span<const uint32_t> commands) {
EXIT_IF(commands.empty());
GpuMutexLock lock(m_submission_mutex);
EXIT_NOT_IMPLEMENTED(queue < ComputeQueueBase || queue >= ComputeQueueBase + ComputeQueueCount);
@@ -186,22 +161,21 @@ void GuestGpu::SubmitCompute(uint32_t queue, std::span<const uint32_t> commands)
}
void GuestGpu::SubmitFlipPreparation(uint64_t request_id) {
GpuMutexLock lock(m_submission_mutex);
Submission submission;
Submission submission;
submission.type = SubmissionType::FlipPreparation;
submission.queue_id = 0;
submission.reset_processor = m_graphics_done;
submission.flip_request_id = request_id;
m_graphics_done = false;
Enqueue(std::move(submission));
}
void GuestGpu::Done() {
GpuMutexLock lock(m_submission_mutex);
if (!IsGpuThread()) {
WaitForIdle();
}
m_graphics_done = true;
void GuestGpu::SuspendPoint() {
EXIT_IF(IsGpuThread() || CommandScheduler::InDeferredOperation());
// Do not hold a queue lock while waiting: asynchronous work may be needed to
// finish the preceding graphics frame. The first point returns immediately.
m_suspend_point_ready->acquire();
Submission submission;
submission.type = SubmissionType::SuspendPoint;
Enqueue(std::move(submission));
m_done_num++;
}
@@ -229,6 +203,9 @@ void CommandProcessor::Reset() {
m_context_state_pushed = false;
m_index_type_and_size = 0;
m_index_buffer_size = 0;
m_index_base_addr = 0;
m_num_instances = 1;
m_predicate_skip = false;
m_user_data_marker = HW::UserSgprType::Unknown;
m_draw_indirect_args_base_addr = 0;
m_dispatch_indirect_args_base_addr = 0;
@@ -268,10 +245,6 @@ void CommandProcessor::BufferFlush() {
GetScheduler().Flush();
}
void CommandProcessor::BufferFlushAndWait() {
GetScheduler().FlushAndWait();
}
void CommandProcessor::BufferWait() {
BufferInit();
GetScheduler().Finish();
@@ -459,6 +432,7 @@ void GuestGpu::Enqueue(Submission submission) {
}
void GuestGpu::WaitForIdle() {
EXIT_IF(IsGpuThread() || CommandScheduler::InDeferredOperation());
Common::LockGuard lock(m_queue_mutex);
while (m_processing || !m_commands.empty() || m_submission_count != 0) {
m_idle.Wait(&m_queue_mutex);
@@ -564,11 +538,7 @@ void GuestGpu::ThreadRun(void* data) {
bool GuestGpu::Process(Submission& submission) {
const bool first_slice = !submission.started;
auto& cp = GetProcessor(submission.queue_id);
if (first_slice && submission.reset_processor) {
cp.Reset();
}
auto& cp = GetProcessor(submission.queue_id);
if (first_slice) {
submission.started = true;
@@ -645,6 +615,13 @@ bool GuestGpu::Process(Submission& submission) {
m_renderer.RunGarbageCollector();
cp.PrepareCpuFlip(submission.flip_request_id);
break;
case SubmissionType::SuspendPoint:
cp.EmitGlobalBarrier();
m_renderer.GetCommandScheduler().DeferPriorityOperation(
[ready = m_suspend_point_ready] { ready->release(); });
cp.BufferFlush();
cp.Reset();
break;
}
return complete;
@@ -849,9 +826,7 @@ void CommandProcessor::SetPredication(uint32_t condition, uint32_t op, uint32_t
}
} break;
case 0x03:
if (wait_op != 0) {
BufferFlushAndWait();
}
// The wait selector applies only to Z-pass query readiness.
EXIT_NOT_IMPLEMENTED(address == nullptr);
value = *reinterpret_cast<const volatile uint64_t*>(address);
break;
@@ -874,13 +849,11 @@ void CommandProcessor::SetPredication(uint32_t condition, uint32_t op, uint32_t
}
void CommandProcessor::DrawIndex(DrawIndexArgs args) {
CheckBuffer();
args.index_type_and_size = m_index_type_and_size;
if (args.instance_count == 0) {
args.instance_count = m_num_instances;
}
if (args.base_vertex != 0 || args.first_instance != 0) {
if (GraphicsRunDebugDumpEnabled() && (args.base_vertex != 0 || args.first_instance != 0)) {
LOGF("\t draw indexed offsets: base_vertex = %" PRId32 ", first_instance = %" PRIu32 "\n",
args.base_vertex, args.first_instance);
}
@@ -912,17 +885,6 @@ void CommandProcessor::DrawIndirect(uint32_t data_offset, uint32_t draw_initiato
if (!indexed) {
DrawIndirectArgs args {};
std::memcpy(&args, args_addr, sizeof(args));
if (args.instance_count != 1u || args.start_vertex_location != 0u ||
args.start_instance_location != 0u) {
static std::atomic<uint32_t> log_count {0};
if (log_count.fetch_add(1) < 64) {
LOGF("\t warning: partial DrawIndirect args: vertex_count=%" PRIu32
", instance_count=%" PRIu32 ", start_vertex=%" PRIu32
", start_instance=%" PRIu32 "\n",
args.vertex_count_per_instance, args.instance_count,
args.start_vertex_location, args.start_instance_location);
}
}
m_num_instances = args.instance_count;
DrawIndexAuto({.vertex_count = args.vertex_count_per_instance,
.instance_count = args.instance_count,
@@ -934,16 +896,6 @@ void CommandProcessor::DrawIndirect(uint32_t data_offset, uint32_t draw_initiato
DrawIndexedIndirectArgs args {};
std::memcpy(&args, args_addr, sizeof(args));
if (args.base_vertex_location != 0u || args.start_instance_location != 0u) {
static std::atomic<uint32_t> log_count {0};
if (log_count.fetch_add(1) < 64) {
LOGF("\t warning: partial DrawIndexIndirect args: index_count=%" PRIu32
", instance_count=%" PRIu32 ", start_index=%" PRIu32 ", base_vertex=%" PRIu32
", start_instance=%" PRIu32 "\n",
args.index_count_per_instance, args.instance_count, args.start_index_location,
args.base_vertex_location, args.start_instance_location);
}
}
uint64_t index_size = 0;
switch (m_index_type_and_size) {
@@ -999,23 +951,22 @@ void CommandProcessor::DrawIndirectMulti(uint32_t data_offset, uint32_t max_coun
const auto args_size = indexed ? sizeof(DrawIndexedIndirectArgs) : sizeof(DrawIndirectArgs);
EXIT_NOT_IMPLEMENTED(stride_in_bytes < args_size);
uint64_t index_size = 0;
if (indexed) {
switch (m_index_type_and_size) {
case 0: index_size = 2; break;
case 1: index_size = 4; break;
case 2: index_size = 1; break;
default: EXIT("unknown index_type_and_size: %u\n", m_index_type_and_size);
}
}
for (uint32_t i = 0; i < draw_count; i++) {
const auto args_addr = m_draw_indirect_args_base_addr + data_offset +
static_cast<uint64_t>(i) * stride_in_bytes;
if (!indexed) {
auto* args = reinterpret_cast<const DrawIndirectArgs*>(args_addr);
if (args->instance_count != 1u || args->start_vertex_location != 0u ||
args->start_instance_location != 0u) {
static std::atomic<uint32_t> log_count {0};
if (log_count.fetch_add(1) < 64) {
LOGF("\t warning: partial DrawIndirectMulti args[%u]: vertex_count=%" PRIu32
", instance_count=%" PRIu32 ", start_vertex=%" PRIu32
", start_instance=%" PRIu32 "\n",
i, args->vertex_count_per_instance, args->instance_count,
args->start_vertex_location, args->start_instance_location);
}
}
m_num_instances = args->instance_count;
DrawIndexAuto({.vertex_count = args->vertex_count_per_instance,
.instance_count = args->instance_count,
@@ -1026,25 +977,6 @@ void CommandProcessor::DrawIndirectMulti(uint32_t data_offset, uint32_t max_coun
}
auto* args = reinterpret_cast<const DrawIndexedIndirectArgs*>(args_addr);
if (args->base_vertex_location != 0u || args->start_instance_location != 0u) {
static std::atomic<uint32_t> log_count {0};
if (log_count.fetch_add(1) < 64) {
LOGF("\t warning: partial DrawIndexIndirectMulti args[%u]: index_count=%" PRIu32
", instance_count=%" PRIu32 ", start_index=%" PRIu32 ", base_vertex=%" PRIu32
", start_instance=%" PRIu32 "\n",
i, args->index_count_per_instance, args->instance_count,
args->start_index_location, args->base_vertex_location,
args->start_instance_location);
}
}
uint64_t index_size = 0;
switch (m_index_type_and_size) {
case 0: index_size = 2; break;
case 1: index_size = 4; break;
case 2: index_size = 1; break;
default: EXIT("unknown index_type_and_size: %u\n", m_index_type_and_size);
}
auto* index_addr = reinterpret_cast<const void*>(
m_index_base_addr + static_cast<uint64_t>(args->start_index_location) * index_size);
@@ -1077,12 +1009,8 @@ void CommandProcessor::DispatchDirect(uint32_t thread_group_x, uint32_t thread_g
m_sh_ctx.SetCsWaveSize(Pm4::ComputeWaveSize(mode));
uint32_t frame_num = 0;
// uint32_t local_x = 1;
// uint32_t local_y = 1;
// uint32_t local_z = 1;
{
CheckBuffer();
frame_num = m_renderer.GetGpu().GetFrameNum();
if (GraphicsRunDebugDumpEnabled()) {
static std::atomic<uint32_t> log_count {0};
@@ -1102,55 +1030,23 @@ void CommandProcessor::DispatchDirect(uint32_t thread_group_x, uint32_t thread_g
}
}
const auto& cs = m_sh_ctx.GetCs().cs_regs;
// local_x = std::max(cs.num_thread_x, 1u);
// local_y = std::max(cs.num_thread_y, 1u);
// local_z = std::max(cs.num_thread_z, 1u);
m_renderer.GetRenderExecutor().DispatchDirect(m_submit_id, CurrentBuffer(), thread_group_x,
thread_group_y, thread_group_z, mode);
}
/*constexpr uint32_t DispatchInitiatorUseThreadDimensions = 1u << 5u;
auto group_count = [](uint32_t threads, uint32_t group_size) {
return (threads == 0
? 0u
: (threads + std::max(group_size, 1u) - 1u) / std::max(group_size, 1u));
};
auto groups_x = thread_group_x;
auto groups_y = thread_group_y;
auto groups_z = thread_group_z;
if ((mode & DispatchInitiatorUseThreadDimensions) != 0) {
groups_x = group_count(thread_group_x, local_x);
groups_y = group_count(thread_group_y, local_y);
groups_z = group_count(thread_group_z, local_z);
}
const uint64_t invocations =
static_cast<uint64_t>(groups_x) * groups_y * groups_z * local_x * local_y * local_z;
if (invocations != 0) {
BufferFlushAndWait();
}*/
}
void CommandProcessor::DispatchIndirect(uint32_t data_offset, uint32_t mode) {
struct DispatchIndirectArgs {
uint32_t thread_group_x;
uint32_t thread_group_y;
uint32_t thread_group_z;
};
EXIT_NOT_IMPLEMENTED(m_dispatch_indirect_args_base_addr == 0);
const auto args_addr = m_dispatch_indirect_args_base_addr + data_offset;
auto* args = reinterpret_cast<const DispatchIndirectArgs*>(args_addr);
DispatchDirect(args->thread_group_x, args->thread_group_y, args->thread_group_z, mode);
void CommandProcessor::DispatchIndirect(uint64_t args_addr, uint32_t mode) {
EXIT_NOT_IMPLEMENTED(args_addr == 0 || (args_addr & 3u) != 0);
if ((mode & Pm4::COMPUTE_DISPATCH_INITIATOR_USE_THREAD_DIMENSIONS) != 0) {
const auto* args = reinterpret_cast<const vk::DispatchIndirectCommand*>(args_addr);
DispatchDirect(args->x, args->y, args->z, mode);
return;
}
m_sh_ctx.SetCsWaveSize(Pm4::ComputeWaveSize(mode));
m_renderer.GetRenderExecutor().DispatchIndirect(m_submit_id, CurrentBuffer(), args_addr, mode);
}
void CommandProcessor::DrawIndexAuto(DrawAutoArgs args) {
CheckBuffer();
if (args.instance_count == 0) {
args.instance_count = m_num_instances;
}
@@ -1172,7 +1068,7 @@ void CommandProcessor::WriteAtEndOfPipe(uint32_t cache_policy, uint32_t event_wr
uint32_t interrupt_context_id) {
static_assert(sizeof(T) == sizeof(uint32_t) || sizeof(T) == sizeof(uint64_t));
CheckBuffer();
auto& command = CurrentBuffer();
if (GraphicsRunDebugDumpEnabled()) {
const auto bits = static_cast<unsigned>(sizeof(T) * 8u);
@@ -1203,7 +1099,7 @@ void CommandProcessor::WriteAtEndOfPipe(uint32_t cache_policy, uint32_t event_wr
case 0x03: with_interrupt = false; break;
case 0x01:
if (!IsAsyncComputeQueue()) {
Sync::TriggerEopEventAtEndOfPipe(CurrentBuffer(), m_interrupt_event_id,
Sync::TriggerEopEventAtEndOfPipe(command, m_interrupt_event_id,
interrupt_context_id);
return;
}
@@ -1220,17 +1116,17 @@ void CommandProcessor::WriteAtEndOfPipe(uint32_t cache_policy, uint32_t event_wr
if (with_interrupt) {
if (with_writeback) {
Sync::WriteAtEndOfPipeWithInterruptWriteBack32(m_submit_id, CurrentBuffer(), dst,
Sync::WriteAtEndOfPipeWithInterruptWriteBack32(m_submit_id, command, dst,
data, m_interrupt_event_id,
interrupt_context_id);
} else {
Sync::WriteAtEndOfPipeWithInterrupt32(m_submit_id, CurrentBuffer(), dst, data,
Sync::WriteAtEndOfPipeWithInterrupt32(m_submit_id, command, dst, data,
m_interrupt_event_id, interrupt_context_id);
}
} else if (with_writeback) {
Sync::WriteAtEndOfPipeWithWriteBack32(m_submit_id, CurrentBuffer(), dst, data);
Sync::WriteAtEndOfPipeWithWriteBack32(m_submit_id, command, dst, data);
} else {
Sync::WriteAtEndOfPipe32(m_submit_id, CurrentBuffer(), dst, data);
Sync::WriteAtEndOfPipe32(m_submit_id, command, dst, data);
}
};
@@ -1242,7 +1138,7 @@ void CommandProcessor::WriteAtEndOfPipe(uint32_t cache_policy, uint32_t event_wr
SynchronizeGpu();
Sync::ReadGds(*m_renderer.GetBufferCache().GetGdsBuffer(), dst, value & 0xffffu,
value >> 16u);
Sync::WriteAtEndOfPipeGds32(m_submit_id, CurrentBuffer(), dst, value & 0xffffu,
Sync::WriteAtEndOfPipeGds32(m_submit_id, command, dst, value & 0xffffu,
value >> 16u);
if (with_interrupt) {
m_renderer.TriggerInterrupt(m_interrupt_event_id, interrupt_context_id);
@@ -1255,8 +1151,9 @@ void CommandProcessor::WriteAtEndOfPipe(uint32_t cache_policy, uint32_t event_wr
}
break;
case 0x02:
case 0x04:
if constexpr (sizeof(T) == sizeof(uint32_t)) {
if (eop_event_type == 0x2f && event_index == 0x06) {
if (event_write_source == 0x02 && eop_event_type == 0x2f && event_index == 0x06) {
switch (cache_action) {
case 0x00: write32(false); return;
case 0x38: write32(true); return;
@@ -1264,6 +1161,9 @@ void CommandProcessor::WriteAtEndOfPipe(uint32_t cache_policy, uint32_t event_wr
}
}
} else {
if (event_write_source == 0x04) {
value = Sync::ReadReferenceClock();
}
auto write64 = [&](bool with_writeback) {
auto* dst = static_cast<uint64_t*>(dst_gpu_addr);
std::memcpy(dst, &value, sizeof(value));
@@ -1271,18 +1171,18 @@ void CommandProcessor::WriteAtEndOfPipe(uint32_t cache_policy, uint32_t event_wr
if (with_interrupt) {
if (with_writeback) {
Sync::WriteAtEndOfPipeWithInterruptWriteBack64(
m_submit_id, CurrentBuffer(), dst, value, m_interrupt_event_id,
m_submit_id, command, dst, value, m_interrupt_event_id,
interrupt_context_id);
} else {
Sync::WriteAtEndOfPipeWithInterrupt64(m_submit_id, CurrentBuffer(), dst,
Sync::WriteAtEndOfPipeWithInterrupt64(m_submit_id, command, dst,
value, m_interrupt_event_id,
interrupt_context_id);
}
} else if (with_writeback) {
Sync::WriteAtEndOfPipeWithWriteBack64(m_submit_id, CurrentBuffer(), dst,
Sync::WriteAtEndOfPipeWithWriteBack64(m_submit_id, command, dst,
value);
} else {
Sync::WriteAtEndOfPipe64(m_submit_id, CurrentBuffer(), dst, value);
Sync::WriteAtEndOfPipe64(m_submit_id, command, dst, value);
}
};
@@ -1320,7 +1220,7 @@ void CommandProcessor::WriteAtEndOfPipe(uint32_t cache_policy, uint32_t event_wr
case 0x14:
case 0x28:
if (((eop_event_type == 0x04 || eop_event_type == 0x28) &&
event_index == 0x05 && !with_interrupt) ||
event_index == 0x05) ||
(event_index == 0x00)) {
write64(true);
return;
@@ -1352,45 +1252,6 @@ void CommandProcessor::WriteAtEndOfPipe(uint32_t cache_policy, uint32_t event_wr
}
}
break;
case 0x04:
if constexpr (sizeof(T) == sizeof(uint64_t)) {
const auto clock = Sync::ReadReferenceClock();
auto* dst = static_cast<uint64_t*>(dst_gpu_addr);
std::memcpy(dst, &clock, sizeof(clock));
switch (cache_action) {
case 0x00:
if ((eop_event_type == 0x04 && event_index == 0x05) ||
(eop_event_type == 0x28 && event_index == 0x00)) {
if (with_interrupt) {
Sync::WriteAtEndOfPipeWithInterrupt64(
m_submit_id, CurrentBuffer(), dst, clock, m_interrupt_event_id,
interrupt_context_id);
} else {
Sync::WriteAtEndOfPipeClockCounter(m_submit_id, CurrentBuffer(),
dst, clock);
}
return;
}
break;
case 0x38:
if ((eop_event_type == 0x04 &&
(event_index == 0x00 || event_index == 0x05)) ||
(eop_event_type == 0x28 && event_index == 0x00)) {
if (with_interrupt) {
Sync::WriteAtEndOfPipeWithInterruptWriteBack64(
m_submit_id, CurrentBuffer(), dst, clock, m_interrupt_event_id,
interrupt_context_id);
} else {
Sync::WriteAtEndOfPipeClockCounterWithWriteBack(
m_submit_id, CurrentBuffer(), dst, clock);
}
return;
}
break;
default: break;
}
}
break;
default: break;
}
@@ -1420,8 +1281,6 @@ void CommandProcessor::WriteAtEndOfPipe64(uint32_t cache_policy, uint32_t event_
}
void CommandProcessor::EmitGlobalBarrier() {
CheckBuffer();
Common::LockGuard lock(m_renderer.GetMutex());
vk::MemoryBarrier2 barrier {};
@@ -1438,8 +1297,6 @@ void CommandProcessor::EmitGlobalBarrier() {
}
void CommandProcessor::TriggerEopEventAtEndOfPipe(uint32_t interrupt_context_id) {
CheckBuffer();
Sync::TriggerEopEventAtEndOfPipe(CurrentBuffer(), m_interrupt_event_id, interrupt_context_id);
}
@@ -1523,8 +1380,6 @@ void CommandProcessor::TriggerEvent(uint32_t event_type, uint32_t event_index,
}
void CommandProcessor::Flip() {
CheckBuffer();
if (GraphicsRunDebugDumpEnabled()) {
LOGF("CommandProcessor::Flip()\n");
}
@@ -1538,7 +1393,7 @@ void CommandProcessor::Flip() {
}
void CommandProcessor::Flip(void* dst_gpu_addr, uint32_t value) {
CheckBuffer();
auto& command = CurrentBuffer();
if (GraphicsRunDebugDumpEnabled()) {
LOGF("CommandProcessor::Flip()\n"
@@ -1548,7 +1403,6 @@ void CommandProcessor::Flip(void* dst_gpu_addr, uint32_t value) {
}
std::memcpy(dst_gpu_addr, &value, sizeof(value));
auto& command = CurrentBuffer();
auto request = Sync::PrepareVideoOutFlip(command, m_flip.handle, m_flip.index, m_flip.flip_mode,
m_flip.flip_arg);
Sync::WriteAtEndOfPipeWithFlip32(m_submit_id, command, static_cast<uint32_t*>(dst_gpu_addr),
@@ -1559,7 +1413,7 @@ void CommandProcessor::Flip(void* dst_gpu_addr, uint32_t value) {
void CommandProcessor::FlipWithInterrupt(uint32_t eop_event_type, uint32_t cache_action,
void* dst_gpu_addr, uint32_t value) {
CheckBuffer();
auto& command = CurrentBuffer();
if (GraphicsRunDebugDumpEnabled()) {
LOGF("CommandProcessor::FlipWithInterrupt()\n"
@@ -1574,7 +1428,6 @@ void CommandProcessor::FlipWithInterrupt(uint32_t eop_event_type, uint32_t cache
EXIT("unknown event type\n");
}
std::memcpy(dst_gpu_addr, &value, sizeof(value));
auto& command = CurrentBuffer();
auto request = Sync::PrepareVideoOutFlip(command, m_flip.handle, m_flip.index, m_flip.flip_mode,
m_flip.flip_arg);
Sync::WriteAtEndOfPipeWithInterruptWriteBackFlip32(
@@ -1584,7 +1437,7 @@ void CommandProcessor::FlipWithInterrupt(uint32_t eop_event_type, uint32_t cache
}
void CommandProcessor::PrepareCpuFlip(uint64_t request_id) {
CheckBuffer();
auto& command = CurrentBuffer();
if (g_current_processor != nullptr) {
EXIT("invalid graphics-thread CPU flip preparation\n");
}
@@ -1594,9 +1447,10 @@ void CommandProcessor::PrepareCpuFlip(uint64_t request_id) {
};
ProcessorScope processor_scope(*this);
m_renderer.GetVideoOut().PrepareFlip(request_id, CurrentBuffer());
m_renderer.GetVideoOut().PrepareFlip(request_id, command);
GetScheduler().DeferPriorityOperation(
[this, request_id] { m_renderer.GetVideoOut().CompleteFlip(request_id); });
GetScheduler().Flush();
m_renderer.GetVideoOut().CompleteFlip(request_id);
}
void CommandProcessor::SynchronizeGpu() {
+13 -10
View File
@@ -13,6 +13,7 @@
#include <deque>
#include <memory>
#include <mutex>
#include <semaphore>
#include <span>
#include <thread>
@@ -32,11 +33,14 @@ public:
void SendCommandSync(Common::UniqueFunction<void>&& command);
// Submitted command memory is borrowed and must remain valid until GPU execution completes.
void Submit(std::span<const uint32_t> draw_commands,
std::span<const uint32_t> constant_commands);
void SubmitCompute(uint32_t queue, std::span<const uint32_t> commands);
void SubmitFlipPreparation(uint64_t request_id);
void Done();
void Submit(std::span<const uint32_t> draw_commands,
std::span<const uint32_t> constant_commands);
void SubmitCompute(uint32_t queue, std::span<const uint32_t> commands);
void SubmitFlipPreparation(uint64_t request_id);
// Insert an ordered graphics drain; only a previous suspend point can block the caller.
void SuspendPoint();
// Wait for guest command processing, including all compute queues (not native GPU completion).
void WaitForIdle();
[[nodiscard]] int GetFrameNum() const;
[[nodiscard]] static bool IsGpuThread() noexcept;
@@ -48,7 +52,7 @@ private:
static constexpr uint32_t ComputeQueueBase = 0x20;
static constexpr uint32_t QueueCount = 1 + ComputeQueueCount;
enum class SubmissionType { Graphics, Compute, FlipPreparation };
enum class SubmissionType { Graphics, Compute, FlipPreparation, SuspendPoint };
struct Submission {
SubmissionType type = SubmissionType::Graphics;
@@ -57,7 +61,6 @@ private:
std::span<const uint32_t> constant_commands;
Pm4Execution command_execution;
Pm4Execution constant_execution;
bool reset_processor = false;
bool started = false;
bool command_complete = false;
bool constant_complete = false;
@@ -66,14 +69,12 @@ private:
};
void Enqueue(Submission submission);
void WaitForIdle();
void ProcessCommands();
bool Process(Submission& submission);
static void ThreadRun(void* data);
CommandProcessor& GetProcessor(uint32_t queue_id);
RenderContext& m_renderer;
Common::Mutex m_submission_mutex;
Common::Mutex m_queue_mutex;
std::mutex m_shutdown_mutex;
Common::CondVar m_work_available;
@@ -84,10 +85,12 @@ private:
uint32_t m_next_queue = 0;
uint32_t m_submission_count = 0;
bool m_processing = false;
bool m_graphics_done = true;
bool m_accepting = true;
bool m_stopping = false;
bool m_shutdown_complete = false;
// Completion callbacks can outlive GuestGpu during renderer shutdown.
std::shared_ptr<std::binary_semaphore> m_suspend_point_ready =
std::make_shared<std::binary_semaphore>(1);
std::unique_ptr<CommandProcessor> m_gfx_cp;
std::array<std::unique_ptr<CommandProcessor>, ComputeQueueCount> m_compute_cp;
+9
View File
@@ -458,6 +458,9 @@ struct PsShaderResource2 {
struct PsStageRegisters {
uint64_t data_addr = 0;
uint64_t user_data_addr = 0;
// Hades II uses PS offsets 4:5 for a byte-addressed descriptor table.
uint64_t auxiliary_table_addr = 0;
PsShaderResource1 rsrc1;
PsShaderResource2 rsrc2;
};
@@ -1017,6 +1020,12 @@ public:
void SetGsShaderResource2(const GsShaderResource2& rsrc2) { m_vs.gs_regs.rsrc2 = rsrc2; }
void SetPsShaderBase(uint64_t addr) { m_ps.ps_regs.data_addr = addr; }
void SetPsUserDataAddress(uint32_t word, uint32_t value) {
SetUserDataAddressWord(m_ps.ps_regs.user_data_addr, word, value);
}
void SetPsAuxiliaryTableAddress(uint32_t word, uint32_t value) {
SetUserDataAddressWord(m_ps.ps_regs.auxiliary_table_addr, word, value);
}
void SetPsShaderResource1(const PsShaderResource1& rsrc1) { m_ps.ps_regs.rsrc1 = rsrc1; }
void SetPsShaderResource2(const PsShaderResource2& rsrc2) { m_ps.ps_regs.rsrc2 = rsrc2; }
+3
View File
@@ -697,6 +697,8 @@ constexpr uint32_t CX_NUM = 0x3FF + 1;
/* Shader registers */
constexpr uint32_t SPI_SHADER_USER_DATA_ADDR_LO_PS = 0x2;
constexpr uint32_t SPI_SHADER_USER_DATA_ADDR_HI_PS = 0x3;
constexpr uint32_t SPI_SHADER_PACE_ID_PS = 0x6;
constexpr uint32_t SPI_GRAPHICS_SHADER_CONTROL_PS = 0x7;
constexpr uint32_t SPI_SHADER_PGM_LO_PS = 0x8;
@@ -910,6 +912,7 @@ constexpr uint32_t COMPUTE_PGM_RSRC1_WGP_MODE_MASK = 0x1;
constexpr uint32_t COMPUTE_PGM_RSRC1_FWD_PROGRESS_SHIFT = 31;
constexpr uint32_t COMPUTE_PGM_RSRC1_FWD_PROGRESS_MASK = 0x1;
constexpr uint32_t COMPUTE_DISPATCH_INITIATOR_USE_THREAD_DIMENSIONS = 1u << 5u;
constexpr uint32_t COMPUTE_DISPATCH_INITIATOR_CS_W32_EN_SHIFT = 15;
constexpr uint32_t COMPUTE_DISPATCH_INITIATOR_CS_W32_EN_MASK = 0x1;
+22 -39
View File
@@ -1213,20 +1213,12 @@ bool TileGetRenderTargetSize(uint32_t width, uint32_t height, uint32_t pitch,
return true;
}
bool TileGetDccSize(uint32_t width, uint32_t height, uint32_t slices,
uint32_t bytes_per_element, uint32_t levels, Prospero::TileMode tile,
TileSizeAlign& total_size, uint32_t num_fragments_log2) {
total_size = {};
if (width == 0 || height == 0 || slices == 0 || levels != 1 || num_fragments_log2 != 0 ||
!std::has_single_bit(bytes_per_element) || bytes_per_element > 16 ||
(tile != Prospero::TileMode::kRenderTarget && tile != Prospero::TileMode::kDepth)) {
static bool TileGetColorMetadataSize(uint32_t width, uint32_t height, uint32_t slices,
uint32_t block_width, uint32_t block_height,
TileSizeAlign& total_size) {
if (width == 0 || height == 0 || slices == 0) {
return false;
}
// Gen5 color metadata uses 4 KiB blocks, with one byte per 256 bytes of color data.
// Metadata stays thin for volume surfaces, so every depth slice has its own block raster.
const uint32_t coverage_bits = 20u - std::countr_zero(bytes_per_element);
const uint32_t block_width = 1u << ((coverage_bits + 1u) / 2u);
const uint32_t block_height = 1u << (coverage_bits / 2u);
const uint64_t blocks_x = (static_cast<uint64_t>(width) + block_width - 1u) / block_width;
const uint64_t blocks_y = (static_cast<uint64_t>(height) + block_height - 1u) / block_height;
const uint64_t blocks = blocks_x * blocks_y;
@@ -1238,36 +1230,27 @@ bool TileGetDccSize(uint32_t width, uint32_t height, uint32_t slices,
return true;
}
bool TileGetRenderTargetMipLayout(uint32_t width, uint32_t height, uint32_t pitch,
uint32_t bytes_per_element, uint32_t levels,
TileSizeAlign& total_size, TileSizeOffset* level_sizes,
TilePaddedSize* padded_size) {
bool TileGetDccSize(uint32_t width, uint32_t height, uint32_t slices,
uint32_t bytes_per_element, uint32_t levels, Prospero::TileMode tile,
TileSizeAlign& total_size, uint32_t num_fragments_log2) {
total_size = {};
if (width == 0 || height == 0 || levels == 0 || levels > 16 ||
pitch != TileGetRenderTargetPitch(width, bytes_per_element)) {
if (levels != 1 || num_fragments_log2 != 0 ||
!std::has_single_bit(bytes_per_element) || bytes_per_element > 16 ||
(tile != Prospero::TileMode::kRenderTarget && tile != Prospero::TileMode::kDepth)) {
return false;
}
uint32_t max_levels = 1;
uint32_t max_dimension = std::max(width, height);
while (max_dimension > 1) {
max_dimension >>= 1u;
max_levels++;
}
if (levels > max_levels) {
return false;
}
auto format = Prospero::BufferFormat::kInvalid;
switch (bytes_per_element) {
case 1: format = Prospero::BufferFormat::k8UNorm; break;
case 2: format = Prospero::BufferFormat::k16UNorm; break;
case 4: format = Prospero::BufferFormat::k32Float; break;
case 8: format = Prospero::BufferFormat::k16_16_16_16Float; break;
case 16: format = Prospero::BufferFormat::k32_32_32_32Float; break;
default: return false;
}
TileGetTextureSize(format, width, height, levels, Prospero::TileMode::kRenderTarget,
&total_size, level_sizes, padded_size);
return total_size.size != 0 && total_size.align == 65536;
// Gen5 DCC uses 4 KiB blocks, with one byte per 256 bytes of color data.
// Metadata stays thin for volume surfaces, so every depth slice has its own block raster.
const uint32_t coverage_bits = 20u - std::countr_zero(bytes_per_element);
return TileGetColorMetadataSize(width, height, slices, 1u << ((coverage_bits + 1u) / 2u),
1u << (coverage_bits / 2u), total_size);
}
bool TileGetCmaskSize(uint32_t width, uint32_t height, uint32_t slices, uint32_t levels,
TileSizeAlign& total_size) {
total_size = {};
// Gen5 CMASK stores four bits per 8x8 tile in 4 KiB blocks covering 1024x512 pixels.
return levels == 1 && TileGetColorMetadataSize(width, height, slices, 1024, 512, total_size);
}
void TileGetTextureSize(Prospero::BufferFormat format, uint32_t width, uint32_t height,
+2 -4
View File
@@ -126,10 +126,8 @@ bool TileGetRenderTargetSize(uint32_t width, uint32_t height, uint32_t pitch
bool TileGetDccSize(uint32_t width, uint32_t height, uint32_t slices,
uint32_t bytes_per_element, uint32_t levels, Prospero::TileMode tile,
TileSizeAlign& total_size, uint32_t num_fragments_log2 = 0);
bool TileGetRenderTargetMipLayout(uint32_t width, uint32_t height, uint32_t pitch,
uint32_t bytes_per_element, uint32_t levels,
TileSizeAlign& total_size, TileSizeOffset* level_sizes,
TilePaddedSize* padded_size);
bool TileGetCmaskSize(uint32_t width, uint32_t height, uint32_t slices, uint32_t levels,
TileSizeAlign& total_size);
void TileGetTextureSize(Prospero::BufferFormat format, uint32_t width, uint32_t height,
uint32_t levels, Prospero::TileMode tile, TileSizeAlign* total_size,
TileSizeOffset* level_sizes, TilePaddedSize* padded_size);
+2
View File
@@ -29,6 +29,8 @@ struct GraphicContext {
bool memory_budget_ext_enabled = false;
bool compute_subgroup_size_control_enabled = false;
bool sample_rate_shading_enabled = false;
bool shader_image_int64_atomics_enabled = false;
// bool fp64_denorm_preserve = false; // Temporarily disabled.
bool attachment_feedback_loop_enabled = false;
bool provoking_vertex_last_enabled = false;
bool supports_block_texel_view = false;
+8 -3
View File
@@ -3,6 +3,7 @@
#include "common/bitArray.h"
#include "common/common.h"
#include "kernel/memory.h"
#include <compare>
@@ -10,7 +11,8 @@ namespace Libs::Graphics {
constexpr uint64_t TRACKER_PAGE_SIZE = 4ull * 1024ull;
constexpr uint64_t TRACKER_REGION_SIZE = 4ull * 1024ull * 1024ull;
constexpr uint64_t TRACKER_ADDRESS_SIZE = 1ull << 40u;
constexpr uint64_t TRACKER_ADDRESS_SIZE = 1ull << 44u;
constexpr uint64_t LOWER_ADDRESS_SIZE = 1ull << 40u;
constexpr size_t TRACKER_REGION_PAGES = TRACKER_REGION_SIZE / TRACKER_PAGE_SIZE;
struct GuestRange {
@@ -19,8 +21,11 @@ struct GuestRange {
[[nodiscard]] constexpr bool Empty() const noexcept { return address == 0 && size == 0; }
[[nodiscard]] constexpr bool Valid() const noexcept {
return address != 0 && size != 0 && address < TRACKER_ADDRESS_SIZE &&
size <= TRACKER_ADDRESS_SIZE - address;
if (address == 0 || size == 0) return false;
if (address < LOWER_ADDRESS_SIZE) return size <= LOWER_ADDRESS_SIZE - address;
constexpr auto base = LibKernel::Memory::kExtendedMemoryBase;
constexpr auto length = LibKernel::Memory::kExtendedMemorySize;
return address >= base && address - base < length && size <= length - (address - base);
}
[[nodiscard]] constexpr bool ValidOrEmpty() const noexcept { return Empty() || Valid(); }
[[nodiscard]] constexpr uint64_t End() const noexcept { return address + size; }
+6 -11
View File
@@ -30,21 +30,14 @@ class TrackingSpinLock final {
public:
void lock() noexcept {
const auto thread = CurrentThread();
if (m_owner.load(std::memory_order_relaxed) == thread) {
EXIT("recursive region tracking lock\n");
}
while (m_lock.test_and_set(std::memory_order_acquire)) {
if (m_owner.load(std::memory_order_relaxed) == thread) {
EXIT("recursive region tracking lock while contended\n");
}
EXIT_NOT_IMPLEMENTED(m_owner.load(std::memory_order_relaxed) == thread);
std::atomic_signal_fence(std::memory_order_seq_cst);
}
m_owner.store(thread, std::memory_order_relaxed);
}
void unlock() noexcept {
if (m_owner.load(std::memory_order_relaxed) != CurrentThread()) {
EXIT("region tracking lock released by non-owner\n");
}
EXIT_NOT_IMPLEMENTED(m_owner.load(std::memory_order_relaxed) != CurrentThread());
m_owner.store(0, std::memory_order_relaxed);
m_lock.clear(std::memory_order_release);
}
@@ -88,8 +81,7 @@ public:
template <DirtySource source>
[[nodiscard]] bool IsModified(uint64_t offset, uint64_t size) const {
const auto [start, end] = GetPageRange(m_cpu_addr + offset, size);
const auto& bits = GetBits<source>();
return RegionBits(bits, start, end).Any();
return GetBits<source>().FirstRangeFrom(start).first < end;
}
template <DirtySource source, bool enable>
@@ -122,6 +114,9 @@ public:
void ForEachModifiedRange(uint64_t vaddr, uint64_t size, Func&& func) {
const auto [start, end] = GetPageRange(vaddr, size);
auto& bits = GetBits<source>();
if (bits.FirstRangeFrom(start).first >= end) {
return;
}
RegionBits mask(bits, start, end);
if constexpr (clear) {
bits.UnsetRange(start, end);
+40 -23
View File
@@ -55,7 +55,8 @@ template <bool insert>
void BufferCache::ChangeRegister(BufferId id) {
auto& buffer = m_slot_buffers[id];
PageTable::PageRange pages {};
EXIT_IF(!PageTable::TryGetPageRange(buffer.CpuAddress(), buffer.Size(), pages));
EXIT_IF(!(GuestRange {buffer.CpuAddress(), buffer.Size()}.Valid()) ||
!PageTable::TryGetPageRange(buffer.CpuAddress(), buffer.Size(), pages));
for (size_t page = pages.first; page < pages.last_exclusive; ++page) {
if constexpr (insert) {
m_page_table[page] = id;
@@ -64,6 +65,7 @@ void BufferCache::ChangeRegister(BufferId id) {
}
}
const auto size_pages = pages.last_exclusive - pages.first;
const auto table_offset = PageIndex(buffer.CpuAddress()) * sizeof(vk::DeviceAddress);
if constexpr (insert) {
const auto [it, inserted] = m_buffers.emplace(buffer.CpuAddress(), id);
(void)it;
@@ -75,7 +77,7 @@ void BufferCache::ChangeRegister(BufferId id) {
for (uint64_t i = 0; i < size_pages; ++i) {
addresses.push_back(buffer.BufferDeviceAddress() + (i << CACHING_PAGEBITS));
}
WriteDataBuffer(m_bda_pagetable_buffer, pages.first * sizeof(vk::DeviceAddress),
WriteDataBuffer(m_bda_pagetable_buffer, table_offset,
addresses.data(), addresses.size() * sizeof(vk::DeviceAddress));
} else {
const auto found = m_buffers.find(buffer.CpuAddress());
@@ -84,7 +86,7 @@ void BufferCache::ChangeRegister(BufferId id) {
EXIT_IF(buffer.Size() > m_total_used_memory);
m_total_used_memory -= buffer.Size();
m_lru_cache.Free(buffer.lru_id);
m_bda_pagetable_buffer.Fill(pages.first * sizeof(vk::DeviceAddress),
m_bda_pagetable_buffer.Fill(table_offset,
size_pages * sizeof(vk::DeviceAddress), 0);
buffer.is_deleted = true;
}
@@ -108,6 +110,7 @@ void BufferCache::DeleteBuffer(BufferId id) {
}
}
template <bool async>
bool BufferCache::DownloadBufferMemory(Buffer& buffer, uint64_t vaddr, uint64_t size) {
std::vector<vk::BufferCopy> copies;
uint64_t total_size = 0;
@@ -127,11 +130,16 @@ bool BufferCache::DownloadBufferMemory(Buffer& buffer, uint64_t vaddr, uint64_t
return false;
}
const auto [mapped, offset] = m_download_buffer.Map(total_size, 64);
auto [mapped, offset] = m_download_buffer.Map(total_size, 64);
std::unique_ptr<Buffer> temporary;
if (mapped == nullptr) {
EXIT("BufferCache: download exceeds 32 MiB staging buffer capacity\n");
temporary = std::make_unique<Buffer>(m_graphics, m_scheduler, MemoryUsage::Download, 0,
vk::BufferUsageFlagBits::eTransferDst, total_size);
mapped = temporary->Mapped().data();
} else {
m_download_buffer.Commit();
}
m_download_buffer.Commit();
const auto& download = temporary ? *temporary : m_download_buffer;
for (auto& copy: copies) {
copy.dstOffset += offset;
}
@@ -150,27 +158,35 @@ bool BufferCache::DownloadBufferMemory(Buffer& buffer, uint64_t vaddr, uint64_t
native.pipelineBarrier(vk::PipelineStageFlagBits::eAllCommands,
vk::PipelineStageFlagBits::eTransfer, {}, 0, nullptr, 1, &before, 0,
nullptr);
native.copyBuffer(buffer.Handle(), m_download_buffer.Handle(),
native.copyBuffer(buffer.Handle(), download.Handle(),
static_cast<uint32_t>(copies.size()), copies.data());
auto after = before;
after.srcAccessMask = vk::AccessFlagBits::eTransferWrite;
after.dstAccessMask = vk::AccessFlagBits::eHostRead;
after.buffer = m_download_buffer.Handle();
after.buffer = download.Handle();
after.offset = offset;
after.size = total_size;
native.pipelineBarrier(vk::PipelineStageFlagBits::eTransfer,
vk::PipelineStageFlagBits::eAllCommands |
vk::PipelineStageFlagBits::eHost,
{}, 0, nullptr, 1, &after, 0, nullptr);
m_scheduler.DeferPriorityOperation([this, mapped, offset, total_size, buffer_address,
copies = std::move(copies)] {
m_download_buffer.Invalidate(offset, total_size);
auto publish = [this, mapped, offset, total_size, buffer_address,
copies = std::move(copies), owner = std::move(temporary)] {
(owner ? *owner : m_download_buffer).Invalidate(offset, total_size);
for (const auto& copy: copies) {
Libs::LibKernel::Memory::WriteBacking(buffer_address + copy.srcOffset,
mapped + (copy.dstOffset - offset), copy.size);
}
});
};
if constexpr (async) {
m_scheduler.DeferPriorityOperation(std::move(publish));
} else {
const auto tick = m_scheduler.CurrentTick();
m_scheduler.Wait(tick);
m_scheduler.WaitPriorityOperations(tick);
publish();
}
return true;
}
@@ -183,7 +199,7 @@ BufferCache::BufferCache(GraphicContext& graphics, CommandScheduler& scheduler,
m_memory_tracker(page_manager),
m_staging_buffer(graphics, scheduler, MemoryUsage::Upload, 512 * MiB),
m_stream_buffer(graphics, scheduler, MemoryUsage::Stream, 64 * MiB),
m_download_buffer(graphics, scheduler, MemoryUsage::Download, 32 * MiB),
m_download_buffer(graphics, scheduler, MemoryUsage::Download, 64 * MiB),
m_device_buffer(graphics, scheduler, MemoryUsage::DeviceLocal, 128 * MiB),
m_texture_cache(texture_cache) {
std::memset(m_gds_buffer.Mapped().data(), 0, static_cast<size_t>(m_gds_buffer.Size()));
@@ -249,10 +265,7 @@ void BufferCache::ReadMemory(uint64_t vaddr, uint64_t size, bool is_write) {
const auto window_begin = std::max(Common::AlignDown(vaddr, WindowSize), buffer_begin);
const auto window_end = std::min(std::max(window_begin + WindowSize, vaddr + size), buffer_end);
if (DownloadBufferMemory(buffer, window_begin, window_end - window_begin)) {
const auto tick = m_scheduler.CurrentTick();
m_scheduler.Wait(tick);
m_scheduler.WaitPriorityOperations(tick);
if (DownloadBufferMemory<false>(buffer, window_begin, window_end - window_begin)) {
m_memory_tracker.UnmarkRegionAsGpuModified(window_begin, window_end - window_begin);
}
if (is_write) {
@@ -312,10 +325,14 @@ BufferCache::OverlapResult BufferCache::ResolveOverlaps(uint64_t vaddr, uint64_t
// Reserve space in the incoming stream's direction of growth.
// The old buffer extending left of the request predicts growth to the right, and vice versa.
if (expands_left) {
end += std::min(StreamLeapSize, PageTable::kAddressSpaceSize - end);
end += std::min(StreamLeapSize, (vaddr < LOWER_ADDRESS_SIZE ? LOWER_ADDRESS_SIZE
: LibKernel::Memory::kExtendedMemoryBase +
LibKernel::Memory::kExtendedMemorySize) - end);
}
if (expands_right) {
const auto minimum = CACHING_PAGESIZE * 2;
const auto minimum = vaddr < LOWER_ADDRESS_SIZE
? CACHING_PAGESIZE * 2
: LibKernel::Memory::kExtendedMemoryBase;
if (begin > minimum) {
begin -= std::min(StreamLeapSize, begin - minimum);
}
@@ -448,7 +465,8 @@ std::pair<Buffer*, uint64_t> BufferCache::ObtainBuffer(uint64_t vaddr, uint64_t
const auto alignment = std::max<uint64_t>(
m_graphics.physical_device_properties.limits.minUniformBufferOffsetAlignment, 1);
auto [mapped, offset] = m_stream_buffer.Map(size, alignment, false);
if (mapped != nullptr && Libs::LibKernel::Memory::TryReadBacking(vaddr, mapped, size)) {
if (mapped != nullptr) {
std::memcpy(mapped, reinterpret_cast<const void*>(vaddr), size);
m_stream_buffer.Commit();
return {&m_stream_buffer, offset};
}
@@ -484,8 +502,7 @@ std::pair<Buffer*, uint64_t> BufferCache::ObtainBufferForImage(uint64_t vaddr, u
}
auto [staging, stage_offset] = m_staging_buffer.Map(size, 16);
if (staging == nullptr || (!Libs::LibKernel::Memory::TryReadBacking(vaddr, staging, size) &&
!Libs::LibKernel::Memory::TryReadPrtBacking(vaddr, staging, size))) {
if (staging == nullptr || !Libs::LibKernel::Memory::TryReadSparseBacking(vaddr, staging, size)) {
EXIT("BufferCache: failed to read mapped guest image backing\n");
}
m_staging_buffer.Commit();
@@ -603,7 +620,7 @@ void BufferCache::RunGarbageCollector() {
return false;
}
if (dirty) {
EXIT_IF(!DownloadBufferMemory(buffer, buffer.CpuAddress(), buffer.Size()));
EXIT_NOT_IMPLEMENTED(!DownloadBufferMemory<true>(buffer, buffer.CpuAddress(), buffer.Size()));
dirty_buffers.push_back(id);
} else {
m_memory_tracker.UntrackMemory(buffer.CpuAddress(), buffer.Size());
+16 -3
View File
@@ -29,10 +29,22 @@ class BufferCache {
public:
static constexpr uint32_t CACHING_PAGEBITS = 14;
static constexpr uint64_t CACHING_PAGESIZE = uint64_t {1} << CACHING_PAGEBITS;
static constexpr uint64_t CACHING_NUMPAGES = uint64_t {1} << (40 - CACHING_PAGEBITS);
static constexpr uint64_t CACHING_NUMPAGES = (LOWER_ADDRESS_SIZE + LibKernel::Memory::kExtendedMemorySize) >> CACHING_PAGEBITS;
static constexpr uint64_t BDA_PAGETABLE_SIZE =
CACHING_NUMPAGES * sizeof(vk::DeviceAddress);
static constexpr uint64_t PageIndex(uint64_t address) {
return (address < LOWER_ADDRESS_SIZE
? address
: address - LibKernel::Memory::kExtendedMemoryBase + LOWER_ADDRESS_SIZE) >>
CACHING_PAGEBITS;
}
static constexpr uint64_t GuestAddress(uint64_t offset) {
return offset < LOWER_ADDRESS_SIZE
? offset
: offset - LOWER_ADDRESS_SIZE + LibKernel::Memory::kExtendedMemoryBase;
}
BufferCache(GraphicContext& graphics, CommandScheduler& scheduler, PageManager& page_manager,
TextureCache& texture_cache);
~BufferCache();
@@ -88,7 +100,7 @@ private:
bool has_stream_leap;
};
using PageTable = MultiLevelPageTable<BufferId, CACHING_PAGEBITS, 40, 16>;
using PageTable = MultiLevelPageTable<BufferId, CACHING_PAGEBITS, 44, 20>;
static_assert(CACHING_PAGESIZE == (uint64_t {1} << PageTable::kPageBits));
void WriteDataBuffer(Buffer& buffer, uint64_t address, const void* source, uint64_t size);
void TouchBuffer(const Buffer& buffer);
@@ -105,7 +117,8 @@ private:
[[nodiscard]] vk::Buffer UploadCopies(Buffer& buffer, std::span<vk::BufferCopy> copies,
uint64_t total_size);
[[nodiscard]] bool SynchronizeBufferFromImage(Buffer& buffer, uint64_t vaddr, uint64_t size);
// Queues backing publication; callers wait before clearing dirty pages or reusing their data.
// Synchronous downloads publish before returning; asynchronous callers wait before reuse.
template <bool async>
[[nodiscard]] bool DownloadBufferMemory(Buffer& buffer, uint64_t vaddr, uint64_t size);
GraphicContext& m_graphics;
+3 -2
View File
@@ -134,8 +134,9 @@ void FaultManager::ProcessFaultBuffer() {
const auto* faults = std::bit_cast<const uint64_t*>(mapped);
const auto count = static_cast<uint32_t>(faults[0]);
for (uint32_t index = 1; index <= count; ++index) {
fault_ranges.Add(faults[index], BufferCache::CACHING_PAGESIZE);
LOGF("Accessed non-GPU cached memory at 0x%016" PRIx64 "\n", faults[index]);
const auto address = BufferCache::GuestAddress(faults[index]);
fault_ranges.Add(address, BufferCache::CACHING_PAGESIZE);
LOGF("Accessed non-GPU cached memory at 0x%016" PRIx64 "\n", address);
}
fault_ranges.ForEach([this](uint64_t start, uint64_t end) {
EXIT_IF(end - start > std::numeric_limits<uint32_t>::max());
+17 -16
View File
@@ -14,10 +14,10 @@ SamplerCache::~SamplerCache() {
}
}
vk::Sampler SamplerCache::GetSampler(const ShaderSamplerResource& r) {
vk::Sampler SamplerCache::GetSampler(const ShaderSamplerResource& r, bool integer_border) {
Common::LockGuard lock(m_mutex);
const SamplerKey key {r.fields[0], r.fields[1], r.fields[2], r.fields[3]};
const SamplerKey key {r.fields[0], r.fields[1], r.fields[2], r.fields[3], integer_border};
if (auto iter = m_samplers.find(key); iter != m_samplers.end()) {
return iter->second;
}
@@ -55,11 +55,9 @@ vk::Sampler SamplerCache::GetSampler(const ShaderSamplerResource& r) {
case Prospero::SamplerAnisoRatio::kTwo: aniso_ratio = 2.0f; break;
case Prospero::SamplerAnisoRatio::kFour: aniso_ratio = 4.0f; break;
case Prospero::SamplerAnisoRatio::kEight: aniso_ratio = 8.0f; break;
case Prospero::SamplerAnisoRatio::kSixteen: aniso_ratio = 16.0f; break;
default:
EXIT("unknown ratio: %d dwords=%08x,%08x,%08x,%08x\n",
static_cast<int>(r.MaxAnisoRatio()), r.fields[0], r.fields[1], r.fields[2],
r.fields[3]);
case Prospero::SamplerAnisoRatio::kSixteen:
// Reserved encodings 5-7 fall back to the maximum ratio.
default: aniso_ratio = 16.0f; break;
}
}
@@ -95,23 +93,23 @@ vk::Sampler SamplerCache::GetSampler(const ShaderSamplerResource& r) {
return vk::SamplerAddressMode::eClampToBorder;
};
vk::BorderColor border = vk::BorderColor::eIntTransparentBlack;
vk::BorderColor border = integer_border ? vk::BorderColor::eIntTransparentBlack
: vk::BorderColor::eFloatTransparentBlack;
switch (static_cast<Prospero::SamplerBorderColor>(r.BorderColorType())) {
case Prospero::SamplerBorderColor::kTransBlack:
border = vk::BorderColor::eIntTransparentBlack;
break;
case Prospero::SamplerBorderColor::kTransBlack: break;
case Prospero::SamplerBorderColor::kOpaqueBlack:
border = vk::BorderColor::eIntOpaqueBlack;
border = integer_border ? vk::BorderColor::eIntOpaqueBlack
: vk::BorderColor::eFloatOpaqueBlack;
break;
case Prospero::SamplerBorderColor::kOpaqueWhite:
border = vk::BorderColor::eIntOpaqueWhite;
border = integer_border ? vk::BorderColor::eIntOpaqueWhite
: vk::BorderColor::eFloatOpaqueWhite;
break;
case Prospero::SamplerBorderColor::kFromTable:
LOGF(
"temporary: approximating table border color as transparent black, index = %" PRIu16
"\n",
r.BorderColorPtr());
border = vk::BorderColor::eIntTransparentBlack;
break;
default: EXIT("unknown border color: %d", static_cast<int>(r.BorderColorType()));
}
@@ -125,8 +123,11 @@ vk::Sampler SamplerCache::GetSampler(const ShaderSamplerResource& r) {
sampler_info.addressModeU = to_vk_address_mode(r.ClampX());
sampler_info.addressModeV = to_vk_address_mode(r.ClampY());
sampler_info.addressModeW = to_vk_address_mode(r.ClampZ());
sampler_info.mipLodBias =
static_cast<float>(static_cast<int16_t>((r.LodBias() ^ 0x2000u) - 0x2000u)) / 256.0f;
const auto max_lod_bias = m_graphics.GetPhysicalDeviceProperties().limits.maxSamplerLodBias;
sampler_info.mipLodBias = std::clamp(
static_cast<float>(static_cast<int16_t>((r.LodBias() ^ 0x2000u) - 0x2000u)) / 256.0f,
-max_lod_bias, max_lod_bias
);
sampler_info.anisotropyEnable = (aniso ? VK_TRUE : VK_FALSE);
sampler_info.maxAnisotropy = aniso_ratio;
sampler_info.compareEnable = (r.DepthCompareFunc() != 0 ? VK_TRUE : VK_FALSE);
+2 -2
View File
@@ -24,10 +24,10 @@ public:
~SamplerCache();
KYTY_CLASS_NO_COPY(SamplerCache);
vk::Sampler GetSampler(const ShaderSamplerResource& r);
vk::Sampler GetSampler(const ShaderSamplerResource& r, bool integer_border);
private:
using SamplerKey = std::array<uint32_t, 4>;
using SamplerKey = std::array<uint32_t, 5>;
struct SamplerKeyHash {
std::size_t operator()(const SamplerKey& key) const {
+164 -146
View File
@@ -33,24 +33,27 @@ namespace {
constexpr uint64_t NumFramesBeforeRemoval = 32;
[[nodiscard]] bool DecodeDccClear(const TextureCache::ImageDesc& desc, uint8_t code,
[[nodiscard]] bool DecodeColorClear(const TextureCache::ImageDesc& desc, uint8_t code,
vk::ClearColorValue& clear) {
const auto& metadata = desc.info.metadata;
const auto format = desc.view_info.format;
const bool cmask = metadata.kind == ImageMetadataKind::Cmask;
if (cmask ? code == 0 : code == 0x20) {
// Register clears belong to the color buffer; the texture pipe cannot decode them.
return desc.type == TextureCache::BindingType::RenderTarget &&
metadata.clear_register_valid &&
DecodePackedColorClear(format, metadata.clear_word, clear);
}
if (cmask) {
return false;
}
switch (code) {
case 0x00:
case 0x20:
case 0x40:
case 0x80:
case 0xc0: break;
default: return false;
}
const auto& metadata = desc.info.metadata;
const auto format = desc.view_info.format;
if (code == 0x20) {
// Clear-to-register is a color-buffer operation; the texture pipe cannot decode it.
return desc.type == TextureCache::BindingType::RenderTarget &&
metadata.dcc_clear_register_valid &&
DecodePackedColorClear(format, metadata.dcc_clear_word, clear);
}
clear = {};
if (code == 0x00) {
return true;
@@ -107,45 +110,16 @@ constexpr uint64_t NumFramesBeforeRemoval = 32;
return true;
}
[[nodiscard]] const char* BindingTypeName(TextureCache::BindingType type) {
switch (type) {
case TextureCache::BindingType::Texture: return "Texture";
case TextureCache::BindingType::Storage: return "StorageTexture";
case TextureCache::BindingType::RenderTarget: return "ColorTarget";
case TextureCache::BindingType::DepthTarget: return "DepthTarget";
case TextureCache::BindingType::VideoOut: return "VideoOut";
}
return "Image";
}
void NameImageBinding(GraphicContext& graphics, Image& image, vk::ImageView view,
TextureCache::BindingType type, const ImageViewInfo& view_info) {
const auto* role = BindingTypeName(type);
SetVulkanObjectNameF(
graphics.device, image.backing.image,
"Kyty.{}.Image[guest=0x{:016x} size=0x{:x} extent={}x{}x{} format={} mips={} layers={} "
"samples={}]",
role, image.info.data.address, image.info.data.size, image.info.extent.width,
image.info.extent.height, image.info.extent.depth,
static_cast<uint32_t>(image.info.pixel_format), image.info.resources.levels,
image.info.resources.layers, image.info.samples);
SetVulkanObjectNameF(
graphics.device, view,
"Kyty.{}.View[guest=0x{:016x} format={} aspect=0x{:x} mip={}+{} layer={}+{}]", role,
image.info.data.address, static_cast<uint32_t>(view_info.format),
static_cast<vk::ImageAspectFlags::MaskType>(view_info.aspect), view_info.base_level,
view_info.level_count, view_info.base_layer, view_info.layer_count);
}
[[nodiscard]] std::vector<vk::BufferImageCopy> BuildDepthCopies(const ImageInfo& info,
uint64_t slice_stride) {
uint64_t slice_stride,
vk::ImageAspectFlags aspect) {
std::vector<vk::BufferImageCopy> copies(info.resources.layers);
for (uint32_t layer = 0; layer < info.resources.layers; ++layer) {
auto& copy = copies[layer];
copy.bufferOffset = slice_stride * layer;
copy.bufferRowLength = info.pitch;
copy.bufferImageHeight = info.extent.height;
copy.imageSubresource = {vk::ImageAspectFlagBits::eDepth, 0, layer, 1};
copy.imageSubresource = {aspect, 0, layer, 1};
copy.imageExtent = {info.extent.width, info.extent.height, 1};
}
return copies;
@@ -222,7 +196,7 @@ bool TextureCache::SameBacking(const ImageInfo& cached, const ImageInfo& request
return false;
}
if (!ImageViewOps::FormatsCompatible(cached.pixel_format, requested.pixel_format) ||
(cached.type != requested.type && requested.extent != vk::Extent3D {1, 1, 1})) {
cached.type != requested.type) {
return false;
}
if (exact_format && cached.pixel_format != requested.pixel_format) {
@@ -248,14 +222,6 @@ TextureCache::BindingType TextureCache::UploadBinding(const Image& image) {
return image.usage.storage ? BindingType::Storage : BindingType::Texture;
}
bool TextureCache::SafeToDownload(const Image& image) {
if (!image.SafeToDownload()) {
return false;
}
const auto range = image.info.data;
return !m_buffer_cache.HasGpuDirtyBytes(range.address, range.size);
}
ImageId TextureCache::InsertImage(const ImageInfo& info) {
const auto id = m_slot_images.insert(m_graphics, m_scheduler, info);
if (!info.data.Empty()) {
@@ -617,7 +583,7 @@ bool TextureCache::CopyD16(Image& destination, Image& source) {
}
auto depth_buffer = m_tiler.GetScratchBuffer(depth_size);
auto color_buffer = m_tiler.GetScratchBuffer(color_size);
auto color_buffer = m_tiler.GetScratchBuffer(color_size, depth_buffer.buffer);
const TileManager::D16Layout promote_layout {
.width = depth.info.extent.width,
.height = depth.info.extent.height,
@@ -668,9 +634,12 @@ void TextureCache::CopyImage(ImageId destination_id, ImageId source_id) {
const bool source_depth = source.info.IsDepth();
const bool dest_depth = destination.info.IsDepth();
const bool direct_copy =
source.backing.format == destination.backing.format ||
(!source_depth && !dest_depth &&
vk::blockSize(source.backing.format) == vk::blockSize(destination.backing.format));
(source.backing.image_type == destination.backing.image_type ||
(source.backing.image_type != vk::ImageType::e1D &&
destination.backing.image_type != vk::ImageType::e1D)) &&
(source.backing.format == destination.backing.format ||
(!source_depth && !dest_depth &&
vk::blockSize(source.backing.format) == vk::blockSize(destination.backing.format)));
if (direct_copy) {
destination.CopyImage(source);
} else if (!CopyD16(destination, source)) {
@@ -704,7 +673,8 @@ void TextureCache::CopyImageMip(ImageId destination_id, ImageId source_id, uint3
ImageId TextureCache::ResolveDepthOverlap(const ImageInfo& requested, BindingType binding,
ImageId cached_id) {
auto& cached = m_slot_images[cached_id];
if (!cached.info.IsDepth() && !requested.IsDepth()) {
if ((!cached.info.IsDepth() && !requested.IsDepth()) ||
cached.info.tile_mode != requested.tile_mode) {
return {};
}
const bool stencil_match = requested.HasStencil() == cached.info.HasStencil();
@@ -718,8 +688,8 @@ ImageId TextureCache::ResolveDepthOverlap(const ImageInfo& requested, BindingTyp
requested.samples == 1 && requested.data == cached.info.data &&
requested.extent == cached.info.extent && requested.resources == cached.info.resources &&
requested.type == cached.info.type && requested.pitch == cached.info.pitch &&
requested.tile_mode == cached.info.tile_mode && !requested.HasStencil() &&
!cached.info.HasStencil() && !requested.HasMetadata() && !cached.info.HasMetadata();
!requested.HasStencil() && !cached.info.HasStencil() && !requested.HasMetadata() &&
!cached.info.HasMetadata();
// PPSA04264
const bool retain_cached_layout =
requested.samples == 1 && cached.info.samples == 1 && cached.backing.samples == 1 &&
@@ -730,7 +700,7 @@ ImageId TextureCache::ResolveDepthOverlap(const ImageInfo& requested, BindingTyp
requested.resources.layers != 0 && cached.info.resources.layers != 0 &&
requested.resources.layers < cached.info.resources.layers &&
requested.type == cached.info.type && requested.pitch == cached.info.pitch &&
requested.tile_mode == cached.info.tile_mode && requested.mip_layout[0].offset == 0 &&
requested.mip_layout[0].offset == 0 &&
cached.info.mip_layout[0].offset == 0 &&
requested.mip_layout[0].size == requested.data.size &&
cached.info.mip_layout[0].size == cached.info.data.size &&
@@ -821,13 +791,6 @@ TextureCache::OverlapResult TextureCache::ResolveOverlap(const ImageInfo& reques
if (const auto depth_id = ResolveDepthOverlap(requested, binding, cached_id)) {
return {depth_id};
}
if (requested.IsBlock() && !cached.info.IsBlock()) {
return {ExpandImage(requested, cached_id)};
}
if (requested.data.size == cached.info.data.size &&
(requested.IsVolume() || cached.info.IsVolume())) {
return {ExpandImage(requested, cached_id)};
}
// Equal pitch does not imply equal mip placement: a changed extent can move
// a level into or out of the mip tail. These are separate guest layouts.
if (requested.tile_mode != cached.info.tile_mode ||
@@ -838,13 +801,29 @@ TextureCache::OverlapResult TextureCache::ResolveOverlap(const ImageInfo& reques
}
return {merged_id};
}
if (requested.IsBlock() && !cached.info.IsBlock()) {
return {ExpandImage(requested, cached_id)};
}
// Volume depth is not an array-layer count. A larger depth can retain the
// same block-slice layout while requiring a larger native image.
if ((requested.IsVolume() || cached.info.IsVolume()) &&
(requested.data.size == cached.info.data.size ||
(requested.type == cached.info.type && requested.resources == cached.info.resources &&
requested.extent.width == cached.info.extent.width &&
requested.extent.height == cached.info.extent.height &&
requested.extent.depth > cached.info.extent.depth))) {
return {ExpandImage(requested, cached_id)};
}
// PPSA08394
// A view cannot change the native image type or grow its extent.
if (requested.data.size == cached.info.data.size &&
requested.resources == cached.info.resources && requested.type == cached.info.type &&
requested.extent.width > cached.info.extent.width &&
requested.extent.height >= cached.info.extent.height &&
requested.extent.depth >= cached.info.extent.depth &&
ImageViewOps::FormatsCompatible(cached.info.pixel_format, requested.pixel_format)) {
requested.resources == cached.info.resources &&
ImageViewOps::FormatsCompatible(cached.info.pixel_format, requested.pixel_format) &&
(requested.type != cached.info.type
? requested.extent == cached.info.extent
: requested.extent.width > cached.info.extent.width &&
requested.extent.height >= cached.info.extent.height &&
requested.extent.depth >= cached.info.extent.depth)) {
return {ExpandImage(requested, cached_id)};
}
// PS5 mip tails can expose more levels without increasing the guest allocation.
@@ -905,9 +884,7 @@ TextureCache::OverlapResult TextureCache::ResolveOverlap(const ImageInfo& reques
ImageId TextureCache::ExpandImage(const ImageInfo& info, ImageId source_id) {
RefreshCopySource(source_id);
const auto expanded_id = InsertImage(info);
auto& expanded = m_slot_images[expanded_id];
auto& source = m_slot_images[source_id];
expanded.usage = source.usage;
if (source.binding.is_bound || source.binding.is_target) {
source.binding.needs_rebind = true;
}
@@ -953,22 +930,12 @@ TextureCache::BuildTextureTransfer(const Image& image, BindingType binding,
const bool upload = direction == TransferDirection::Upload;
const bool render_target = binding == BindingType::RenderTarget;
const bool video_out = binding == BindingType::VideoOut;
auto format = info.guest_format;
uint32_t layers = info.TransferLayers();
bool volume = info.IsVolume();
bool allow_depth_tile = upload;
const char* owner = "TextureCache readback";
TextureTransfer transfer;
transfer.swap_bgra16 = info.bgra16 && (!upload || render_target || video_out);
if (render_target) {
format = ImageOps::RenderTargetTransferFormat(info.bytes_per_block);
}
if (video_out) {
allow_depth_tile = false;
} else if (render_target || binding == BindingType::Storage) {
allow_depth_tile = true;
}
transfer.swap_bgra16 = info.bgra16;
if (upload) {
if ((render_target || video_out) &&
(info.resources.layers == 0 || info.data.size % info.resources.layers != 0 ||
@@ -990,9 +957,9 @@ TextureCache::BuildTextureTransfer(const Image& image, BindingType binding,
}
}
transfer.layout = TextureCalcUploadLayout(format, info.extent.width, info.extent.height,
info.resources.levels, layers, info.tile_mode,
info.data.size, allow_depth_tile, volume, owner);
transfer.layout = TextureCalcUploadLayout(info.guest_format, info.extent.width,
info.extent.height, info.resources.levels, layers,
info.tile_mode, info.data.size, volume, owner);
transfer.regions = TextureBuildImageCopies(transfer.layout);
if (info.IsDepth()) {
for (auto& region: transfer.regions) {
@@ -1031,16 +998,17 @@ TextureCache::ImageDownload TextureCache::BuildDownload(const Image& image) cons
}
void TextureCache::UploadImage(Image& image, Buffer& source, uint64_t source_offset) {
const auto& info = image.info;
const auto binding = UploadBinding(image);
auto& destination = image.depth_id ? m_slot_images[image.depth_id] : image;
const auto binding = image.depth_id ? BindingType::DepthTarget : UploadBinding(image);
const auto upload = [&](std::vector<vk::BufferImageCopy>& copies, TileManager::Result linear) {
for (auto& copy: copies) {
copy.bufferOffset += linear.offset;
}
image.Upload(copies, linear.buffer, linear.offset, linear.size);
destination.Upload(copies, linear.buffer, linear.offset, linear.size);
};
if (binding != BindingType::DepthTarget) {
const auto& info = image.info;
auto transfer = BuildTextureTransfer(image, binding, TransferDirection::Upload);
if (!transfer.valid) {
EXIT("TextureCache: invalid texture upload: binding=%u addr=0x%016" PRIx64
@@ -1064,21 +1032,32 @@ void TextureCache::UploadImage(Image& image, Buffer& source, uint64_t source_off
return;
}
if (info.samples != 1 || image.backing.samples != 1 ||
// The stencil plane has its own row pitch and shares the native image
// with the depth plane.
auto info = destination.info;
if (image.depth_id) {
info.data = image.info.data;
info.guest_format = Prospero::BufferFormat::k8UInt;
info.bytes_per_block = 1;
if (info.IsTiled()) info.pitch = TileGetDepthPitch(info.extent.width, 1, 0);
}
if (info.samples != 1 || destination.backing.samples != 1 ||
info.resources.layers == 0 || info.data.size % info.resources.layers != 0 ||
Prospero::NumBytesPerElement(info.guest_format) != info.bytes_per_block) {
EXIT("TextureCache: invalid depth upload\n");
}
const auto layers = info.resources.layers;
const auto full_slice_size = info.data.size / layers;
auto copies = BuildDepthCopies(info, full_slice_size);
auto copies = BuildDepthCopies(info, full_slice_size, image.depth_id
? vk::ImageAspectFlagBits::eStencil
: vk::ImageAspectFlagBits::eDepth);
TileManager::Result linear {source.Handle(), source_offset, source.Size() - source_offset};
if (info.IsTiled()) {
const auto tiles = BuildDepthTiles(info);
linear =
m_tiler.Detile(source.Handle(), source_offset, info.data.size, info.data.size, tiles);
}
const auto transfer_bytes = DepthAspectTransferBytes(info.pixel_format);
const auto transfer_bytes = image.depth_id ? 1u : DepthAspectTransferBytes(info.pixel_format);
if (transfer_bytes != info.bytes_per_block) {
const uint64_t texels_per_slice = static_cast<uint64_t>(info.pitch) * info.extent.height;
EXIT_NOT_IMPLEMENTED(info.bytes_per_block != sizeof(uint16_t) ||
@@ -1086,7 +1065,7 @@ void TextureCache::UploadImage(Image& image, Buffer& source, uint64_t source_off
texels_per_slice > UINT64_MAX / transfer_bytes);
const uint64_t transfer_slice = texels_per_slice * transfer_bytes;
EXIT_NOT_IMPLEMENTED(transfer_slice > UINT64_MAX / layers);
auto promoted = m_tiler.GetScratchBuffer(transfer_slice * layers);
auto promoted = m_tiler.GetScratchBuffer(transfer_slice * layers, linear.buffer);
m_tiler.ConvertD16(
linear, promoted, TileManager::D16Direction::Promote,
info.pixel_format == vk::Format::eD32SfloatS8Uint,
@@ -1135,9 +1114,10 @@ void TextureCache::InitializeImage(ImageId id) {
}
}
void TextureCache::MaterializeDccClear(ImageId id, const ImageDesc& desc,
void TextureCache::MaterializeColorClear(ImageId id, const ImageDesc& desc,
uint32_t metadata_base_layer) {
if (desc.info.metadata.kind != ImageMetadataKind::Dcc) {
if (desc.info.metadata.kind != ImageMetadataKind::Dcc &&
desc.info.metadata.kind != ImageMetadataKind::Cmask) {
return;
}
const auto range = desc.info.metadata.range;
@@ -1145,18 +1125,18 @@ void TextureCache::MaterializeDccClear(ImageId id, const ImageDesc& desc,
std::scoped_lock lock {m_lock};
auto& image = m_slot_images[id];
image.info.metadata = desc.info.metadata;
// A native DCC allocation must not retain a reused HTile/CMask/FMask interpretation.
// Native color metadata must not retain a reused HTile/CMask/FMask clear flag.
m_surface_metas.erase(range.address);
if (range.size == 0 || desc.info.resources.levels != 1 || image.info.resources.levels != 1) {
if (range.size == 0 || desc.info.resources.levels != 1) {
return;
}
}
const auto layers = desc.info.TransferLayers();
// These one-mip surfaces use complete 4 KiB DCC metadata blocks.
// These one-mip surfaces use complete 4 KiB color metadata blocks.
constexpr uint64_t MetadataBlockSize = 0x1000;
if (!range.Valid() || range.address % MetadataBlockSize != 0 || layers == 0 ||
range.size % layers != 0 || (range.size / layers) % MetadataBlockSize != 0) {
EXIT("TextureCache: DCC slices must contain aligned 4 KiB blocks\n");
EXIT("TextureCache: color metadata slices must contain aligned 4 KiB blocks\n");
}
const auto& view = desc.view_info;
const bool volume_texture = desc.info.IsVolume() && view.type == vk::ImageViewType::e3D;
@@ -1164,7 +1144,7 @@ void TextureCache::MaterializeDccClear(ImageId id, const ImageDesc& desc,
const auto image_first = volume_texture ? 0u : view.base_layer;
const auto count = volume_texture ? desc.info.extent.depth : view.layer_count;
if (first >= layers || count > layers - first) {
EXIT("TextureCache: DCC view exceeds its native metadata slices\n");
EXIT("TextureCache: color view exceeds its native metadata slices\n");
}
// Finish native metadata writes before reading backing bytes. This can submit the scheduler,
// so discovery runs before final draw uploads and never holds the texture lock across it.
@@ -1176,15 +1156,15 @@ void TextureCache::MaterializeDccClear(ImageId id, const ImageDesc& desc,
const auto address = range.address + slice_size * (first + slice);
uint8_t code = 0;
if (!LibKernel::Memory::TryReadBacking(address, &code, sizeof(code))) {
EXIT("TextureCache: failed to read DCC metadata backing\n");
EXIT("TextureCache: failed to read color metadata backing\n");
}
vk::ClearValue clear {};
if (!DecodeDccClear(desc, code, clear.color)) {
if (!DecodeColorClear(desc, code, clear.color)) {
continue;
}
std::vector<uint8_t> bytes(slice_size);
if (!LibKernel::Memory::TryReadBacking(address, bytes.data(), bytes.size())) {
EXIT("TextureCache: failed to read DCC metadata slice\n");
EXIT("TextureCache: failed to read color metadata slice\n");
}
if (!std::all_of(bytes.begin(), bytes.end(), [code](uint8_t byte) { return byte == code; })) {
continue;
@@ -1195,15 +1175,24 @@ void TextureCache::MaterializeDccClear(ImageId id, const ImageDesc& desc,
{vk::ImageAspectFlagBits::eColor, view.base_level, view.level_count,
image_first + slice, 1}, clear);
}
// Native expanded keys own consumption. Existing buffer tracking publishes this CPU
// write to future GPU readers; FillBuffer can fault and must run outside the texture lock.
m_buffer_cache.FillBuffer(address, slice_size, UINT32_MAX, false);
// Publish the conversion's expanded keys without treating them as guest writes
// to overlapping image data. Invalidate the buffer before updating its backing.
if (desc.type != BindingType::VideoOut) {
std::fill(bytes.begin(), bytes.end(), uint8_t {0xff});
m_buffer_cache.InvalidateMemory(address, slice_size);
LibKernel::Memory::WriteBacking(address, bytes.data(), bytes.size());
}
}
}
void TextureCache::RefreshImage(ImageId id) {
TrackImage(id);
auto& image = m_slot_images[id];
if (image.depth_id &&
(m_slot_images[image.depth_id].info.metadata.stencil_compressed ||
m_slot_images[image.depth_id].info.samples != 1)) {
return;
}
TrackImage(id);
if (image.IsMaybeCpuDirty()) {
const auto hash = image.HashGuestEdges();
if (image.NeedsMaybeCpuHash()) {
@@ -1225,7 +1214,7 @@ void TextureCache::RefreshImage(ImageId id) {
InitializeImage(id);
}
void TextureCache::AssociateStencil(ImageId depth_id, GuestRange stencil) {
ImageId TextureCache::AssociateStencil(ImageId depth_id, GuestRange stencil) {
if (!stencil.Valid()) {
EXIT("TextureCache: invalid stencil association range\n");
}
@@ -1251,6 +1240,7 @@ void TextureCache::AssociateStencil(ImageId depth_id, GuestRange stencil) {
auto& record = m_slot_images[association];
TouchImage(record);
record.depth_id = depth_id;
return association;
}
ImageId TextureCache::FindImage(ImageDesc& desc, bool exact_format) {
@@ -1312,16 +1302,6 @@ ImageId TextureCache::FindImage(ImageDesc& desc, bool exact_format) {
}
}
auto& image = m_slot_images[result];
if (desc.type == BindingType::VideoOut &&
desc.info.metadata.compression != VideoOutCompression::Uncompressed) {
const bool guest_dirty = image.IsBufferModified() || image.IsCpuDirty();
const bool native_current =
(image.usage.render_target || image.IsGpuModified()) && !guest_dirty;
if (!native_current) {
EXIT("TextureCache: compressed video-out read requires clean native GPU "
"contents\n");
}
}
if (view_mip >= 0) {
desc.view_info.base_level = static_cast<uint32_t>(view_mip);
}
@@ -1331,7 +1311,19 @@ ImageId TextureCache::FindImage(ImageDesc& desc, bool exact_format) {
image.tick_accessed_last = m_scheduler.CurrentTick();
TouchImage(image);
}
MaterializeDccClear(result, desc, metadata_base_layer);
MaterializeColorClear(result, desc, metadata_base_layer);
if (desc.type == BindingType::VideoOut &&
desc.info.metadata.compression != VideoOutCompression::Uncompressed) {
std::scoped_lock lock {m_lock};
const auto& image = m_slot_images[result];
const bool guest_dirty = image.IsBufferModified() || image.IsCpuDirty();
const bool native_current =
(image.usage.render_target || image.IsGpuModified()) && !guest_dirty;
if (!native_current) {
EXIT("TextureCache: compressed video-out read requires clean native GPU "
"contents\n");
}
}
return result;
}
@@ -1356,7 +1348,7 @@ ImageId TextureCache::FindImageFromRange(uint64_t address, uint64_t size, bool e
if (ensure_valid && owner->depth_id) {
owner = m_slot_images.try_get(owner->depth_id);
}
if (owner == nullptr || (ensure_valid && !SafeToDownload(*owner))) {
if (owner == nullptr || (ensure_valid && !owner->SafeToDownload())) {
continue;
}
matches.push_back(id);
@@ -1396,6 +1388,19 @@ vk::ImageView TextureCache::FindTexture(ImageId id, const ImageDesc& desc) {
}
if (!image.info.data.Empty()) {
RefreshImage(id);
if (image.info.HasStencil() &&
desc.info.data.address >= image.info.stencil.address &&
desc.info.data.End() <= image.info.stencil.End()) {
for (const auto stencil_id:
FindImagesInRegion(image.info.stencil.address, image.info.stencil.size, false)) {
const auto* stencil = m_slot_images.try_get(stencil_id);
if (stencil != nullptr && stencil->depth_id == id &&
stencil->info.data == image.info.stencil) {
RefreshImage(stencil_id);
break;
}
}
}
}
switch (desc.type) {
case BindingType::Texture: break;
@@ -1407,9 +1412,7 @@ vk::ImageView TextureCache::FindTexture(ImageId id, const ImageDesc& desc) {
break;
default: EXIT("TextureCache: invalid texture binding\n");
}
const auto view = image.FindView(desc.view_info);
NameImageBinding(m_graphics, image, view, desc.type, desc.view_info);
return view;
return image.FindView(desc.view_info);
}
vk::ImageView TextureCache::FindRenderTarget(ImageId id, const ImageDesc& desc) {
@@ -1427,9 +1430,7 @@ vk::ImageView TextureCache::FindRenderTarget(ImageId id, const ImageDesc& desc)
RefreshImage(id);
CommitGpuWrite(image);
TrackImageDownload(id, image);
const auto view = image.FindView(desc.view_info);
NameImageBinding(m_graphics, image, view, desc.type, desc.view_info);
return view;
return image.FindView(desc.view_info);
}
vk::ImageView TextureCache::FindDepthTarget(ImageId id, const ImageDesc& desc) {
@@ -1444,20 +1445,19 @@ vk::ImageView TextureCache::FindDepthTarget(ImageId id, const ImageDesc& desc) {
TouchImage(image);
image.MarkGpuModified();
image.usage.depth_target = true;
RefreshImage(id);
image.info.stencil = desc.info.stencil;
image.info.metadata = desc.info.metadata;
if (desc.info.HasMetadata()) {
image.info.metadata = desc.info.metadata;
m_surface_metas.emplace(desc.info.metadata.range.address,
MetaDataInfo {.type = MetaDataInfo::Type::HTile,
.clear_mask = image.info.htile_clear_mask});
}
RefreshImage(id);
CommitGpuWrite(image);
if (desc.info.HasStencil()) {
AssociateStencil(id, desc.info.stencil);
RefreshImage(AssociateStencil(id, desc.info.stencil));
}
const auto view = image.FindView(desc.view_info);
NameImageBinding(m_graphics, image, view, desc.type, desc.view_info);
return view;
return image.FindView(desc.view_info);
}
void TextureCache::MarkGpuWritten(ImageId id) {
@@ -1468,11 +1468,16 @@ void TextureCache::MarkGpuWritten(ImageId id) {
}
TrackImage(id);
CommitGpuWrite(image);
if (image.info.HasStencil()) {
const auto stencil_id = AssociateStencil(id, image.info.stencil);
TrackImage(stencil_id);
CommitGpuWrite(m_slot_images[stencil_id]);
}
}
void TextureCache::CommitGpuWrite(Image& image) {
if (image.depth_id || image.backing.image == nullptr) {
EXIT("TextureCache: stencil association cannot own image contents\n");
if (!image.depth_id && image.backing.image == nullptr) {
EXIT("TextureCache: GPU writes require a native image or stencil association\n");
}
image.ClearBufferModified();
if (image.IsCpuDirty()) {
@@ -1524,7 +1529,7 @@ bool TextureCache::ClearImageFromBuffer(CommandBuffer& command, uint64_t address
auto& image = m_slot_images[selected];
vk::ClearValue clear {};
if (aspect == vk::ImageAspectFlagBits::eColor) {
if (!DecodePackedColorClear(image.info.pixel_format, packed_clear, clear.color)) {
if (!DecodeColorDwordFill(image.info.pixel_format, packed_clear, clear.color)) {
return false;
}
} else {
@@ -1556,9 +1561,10 @@ void TextureCache::ClearImage(CommandBuffer& command, ImageId id, vk::Format for
range.layerCount == 0 || range.baseArrayLayer >= layers ||
range.layerCount > layers - range.baseArrayLayer ||
(range.aspectMask & aspects) != range.aspectMask);
const bool full_image = range.aspectMask == aspects && range.baseMipLevel == 0 &&
range.levelCount == image.info.resources.levels &&
range.baseArrayLayer == 0 && range.layerCount == layers;
const bool full_subresources = range.baseMipLevel == 0 &&
range.levelCount == image.info.resources.levels &&
range.baseArrayLayer == 0 && range.layerCount == layers;
const bool full_image = range.aspectMask == aspects && full_subresources;
TrackImage(id);
if (!full_image && (image.IsBufferModified() || image.IsCpuDirty())) {
InitializeImage(id);
@@ -1566,6 +1572,15 @@ void TextureCache::ClearImage(CommandBuffer& command, ImageId id, vk::Format for
EXIT("TextureCache: image clear retained guest ownership\n");
}
}
if (image.info.HasStencil() && (range.aspectMask & vk::ImageAspectFlagBits::eStencil)) {
const auto stencil_id = AssociateStencil(id, image.info.stencil);
if (!full_subresources) {
RefreshImage(stencil_id);
} else {
TrackImage(stencil_id);
CommitGpuWrite(m_slot_images[stencil_id]);
}
}
command.EndRendering();
// Transfer clears use the backing format; aliased clears must encode through their view.
if (format != image.backing.format || (image.info.IsVolume() && !full_image)) {
@@ -1638,7 +1653,7 @@ void TextureCache::DownloadDepth(Image& image, Buffer& destination, uint64_t des
EXIT_NOT_IMPLEMENTED(transfer_slice > UINT64_MAX / layers);
const uint64_t transfer_size = transfer_slice * layers;
EXIT_NOT_IMPLEMENTED(guest_slice > full_slice_size);
auto copies = BuildDepthCopies(info, full_slice_size);
auto copies = BuildDepthCopies(info, full_slice_size, vk::ImageAspectFlagBits::eDepth);
if (transfer_bytes == info.bytes_per_block) {
if (!info.IsTiled()) {
for (auto& copy: copies) {
@@ -1660,7 +1675,7 @@ void TextureCache::DownloadDepth(Image& image, Buffer& destination, uint64_t des
auto host_linear = m_tiler.GetScratchBuffer(transfer_size);
image.Download(copies, host_linear.buffer, 0, host_linear.size);
const bool tiled = info.IsTiled();
auto guest_linear = tiled ? m_tiler.GetScratchBuffer(info.data.size)
auto guest_linear = tiled ? m_tiler.GetScratchBuffer(info.data.size, host_linear.buffer)
: TileManager::Result {destination.Handle(), destination_offset,
destination.Size() - destination_offset};
m_tiler.ConvertD16(host_linear, guest_linear, TileManager::D16Direction::Demote,
@@ -1724,7 +1739,7 @@ bool BufferCache::SynchronizeBufferFromImage(Buffer& buffer, uint64_t vaddr, uin
std::scoped_lock lock {m_texture_cache.m_lock};
auto& image = m_texture_cache.m_slot_images[selected];
// The GPU thread owns image retirement; CPU invalidation can dirty this image after lookup.
if (!m_texture_cache.SafeToDownload(image)) {
if (!image.SafeToDownload()) {
return false;
}
if (!buffer.IsInBounds(image.info.data.address, 1)) {
@@ -1787,7 +1802,7 @@ bool TextureCache::DownloadImageMemory(ImageId id) {
return false;
}
auto transfer = BuildDownload(image);
if (!transfer.valid || !SafeToDownload(image)) {
if (!transfer.valid || !image.SafeToDownload()) {
return false;
}
const auto range = image.info.data;
@@ -1831,7 +1846,7 @@ void TextureCache::InvalidateMemoryFromGPU(uint64_t address, uint64_t size) {
std::scoped_lock lock {m_lock};
for (const auto id: FindImagesInRegion(address, size, true)) {
auto& image = m_slot_images[id];
if (image.depth_id || !image.Overlaps(address, size)) {
if (image.info.data.address != address) {
continue;
}
if (image.IsGpuModified()) {
@@ -1848,7 +1863,10 @@ bool TextureCache::IsRegionGpuModified(uint64_t address, uint64_t size) {
std::scoped_lock lock {m_lock};
for (const auto id: FindImagesInRegion(address, size, false)) {
const auto& image = m_slot_images[id];
if (!image.depth_id && image.IsGpuModified()) {
// PPSA17168: S_LOAD_DWORD reads shader data at an address overlapping an old
// render target whose memory the CPU has reused. The cached image still retains
// its earlier GPU-modified flag.
if (!image.depth_id && image.IsGpuModified() && !image.IsDefinitelyCpuDirty()) {
return true;
}
}
@@ -1974,7 +1992,7 @@ void TextureCache::RunGarbageCollector() {
continue;
}
if (owner->IsGpuModified()) {
const bool safe = SafeToDownload(*owner);
const bool safe = owner->SafeToDownload();
if (safe && owner->info.IsTiled()) {
continue;
}
+4 -5
View File
@@ -91,9 +91,9 @@ private:
};
using ImageIds = InlinePageOwnerList<ImageId, 16>;
using ImagePageTable = MultiLevelPageTable<ImageIds, 20, 40, 10>;
using ImagePageTable = MultiLevelPageTable<ImageIds, 20, 44, 14>;
// Callers have validated the nonempty 40-bit range with TryGetPageRange.
// Callers have validated the nonempty 44-bit range with TryGetPageRange.
template <typename Func>
static void ForEachPage(uint64_t address, size_t size, Func&& func) {
using FuncReturn = typename std::invoke_result<Func, uint64_t>::type;
@@ -128,7 +128,6 @@ private:
[[nodiscard]] static bool SameBacking(const ImageInfo& cached, const ImageInfo& requested,
bool exact_format);
[[nodiscard]] static BindingType UploadBinding(const Image& image);
[[nodiscard]] bool SafeToDownload(const Image& image);
// Caller holds m_lock; it also serializes the per-image query epoch.
[[nodiscard]] ImageIds FindImagesInRegion(uint64_t address, uint64_t size,
@@ -139,7 +138,7 @@ private:
ImageId cached);
[[nodiscard]] ImageId ExpandImage(const ImageInfo& info, ImageId source);
void RefreshImage(ImageId id);
void MaterializeDccClear(ImageId id, const ImageDesc& desc,
void MaterializeColorClear(ImageId id, const ImageDesc& desc,
uint32_t metadata_base_layer);
void InitializeImage(ImageId id);
[[nodiscard]] TextureTransfer
@@ -157,7 +156,7 @@ private:
void RefreshCopySource(ImageId id);
[[nodiscard]] bool CopyD16(Image& destination, Image& source);
void CopyImage(ImageId destination, ImageId source);
void AssociateStencil(ImageId depth, GuestRange stencil);
[[nodiscard]] ImageId AssociateStencil(ImageId depth, GuestRange stencil);
void CopyImageMip(ImageId destination, ImageId source, uint32_t mip, uint32_t layer);
void ValidateImageDesc(const ImageDesc& desc) const;
@@ -17,11 +17,10 @@
#include <algorithm>
#include <array>
#include <atomic>
#include <bit>
namespace Libs::Graphics {
static std::atomic<uint32_t> g_render_color_log_count = 0;
static bool DccAlphaOnMsb(const HW::ColorInfo& info) {
switch (info.format) {
case Prospero::ChannelLayout::k10_10_10_2:
@@ -138,8 +137,7 @@ void RenderExecutor::ResolveRenderColorTarget(CommandBuffer& buffer, RenderColor
// Color-control state selects the color-buffer operation and logical blend operation.
// The normal copy operation is a regular color write, not an attachment clear.
// Nonlinear clear values are still stored as normalized components.
// Fast color clears are metadata driven and must be handled explicitly when
// that metadata path is implemented; render-pass load must preserve contents.
// Metadata clears are materialized during image discovery; render-pass loads preserve contents.
uint32_t width = 0;
uint32_t height = 0;
uint32_t pitch = 0;
@@ -160,9 +158,6 @@ void RenderExecutor::ResolveRenderColorTarget(CommandBuffer& buffer, RenderColor
break;
default: EXIT("unknown tile mode: %u\n", static_cast<uint32_t>(rt.attrib3.tile_mode));
}
if (!tile && levels > 1) {
EXIT("linear mipmapped render targets are unsupported\n");
}
if (samples > 1 && (!tile || levels != 1)) {
EXIT("multisampled render targets require a single-mip tiled surface\n");
}
@@ -203,8 +198,7 @@ void RenderExecutor::ResolveRenderColorTarget(CommandBuffer& buffer, RenderColor
" layer=%u/%u\n",
rt.attrib3.dimension, rt.attrib3.depth, view.base_layer, view.image_layers);
}
// PPSA28068: the linear EULA target and its sampled view must share the padded pitch.
if (!tile || volume || texture_tile) {
if (samples == 1) {
pitch = TileGetTexturePitch(transfer_format, width, rt.attrib3.tile_mode);
} else {
pitch = TileGetRenderTargetPitch(width, bytes_per_element, rt.attrib.num_fragments);
@@ -232,37 +226,26 @@ void RenderExecutor::ResolveRenderColorTarget(CommandBuffer& buffer, RenderColor
}
size = volume_layout.block_slice_size;
backing_size = volume_layout.total_size;
} else if (tile) {
} else {
TileSizeAlign layout {};
bool valid_layout = false;
if (texture_tile) {
if (samples == 1) {
TileGetTextureSize(transfer_format, width, height, levels, rt.attrib3.tile_mode,
&layout, mip_sizes, mip_padded);
valid_layout = layout.size != 0 && layout.align == texture_tile_layout.block.block_size;
valid_layout = layout.size != 0 && layout.align != 0 &&
(rt.attrib3.tile_mode != Prospero::TileMode::kRenderTarget ||
levels <= std::bit_width(std::max(width, height)));
} else {
valid_layout =
levels == 1 ? TileGetRenderTargetSize(width, height, pitch, bytes_per_element,
layout, rt.attrib.num_fragments)
: TileGetRenderTargetMipLayout(width, height, pitch, bytes_per_element,
levels, layout, mip_sizes, mip_padded);
valid_layout = TileGetRenderTargetSize(width, height, pitch, bytes_per_element,
layout, rt.attrib.num_fragments);
mip_sizes[0] = {layout.size, 0, 0, 0, 0, 0};
mip_padded[0] = {pitch, height};
}
if (!valid_layout) {
EXIT("unsupported render-target layout: %ux%u pitch=%u bytes=%u levels=%u\n", width,
height, pitch, bytes_per_element, levels);
}
size = layout.size;
EXIT_IF(size > UINT32_MAX);
if (levels == 1) {
mip_sizes[0] = {static_cast<uint32_t>(size), 0, 0, 0, 0, 0};
mip_padded[0] = {pitch, height};
}
} else {
size = static_cast<uint64_t>(pitch) * height * bytes_per_element * samples;
if (size > UINT32_MAX) {
EXIT("linear render-target slice exceeds the supported layout size\n");
}
mip_sizes[0] = {static_cast<uint32_t>(size), 0, 0, 0, 0, 0};
mip_padded[0] = {pitch, height};
}
if (size == 0 || (!volume && size > UINT64_MAX / view.image_layers)) {
EXIT("render-target memory footprint is invalid\n");
@@ -277,21 +260,7 @@ void RenderExecutor::ResolveRenderColorTarget(CommandBuffer& buffer, RenderColor
EXIT("render-target backing range is invalid\n");
}
const vk::Extent2D view_extent = {std::max(width >> rt.view.current_mip_level, 1u),
std::max(height >> rt.view.current_mip_level, 1u)};
auto decision_log_id = g_render_color_log_count.fetch_add(1);
if (decision_log_id < 128) {
LOGF("RenderColorTarget: slot=%" PRIu32 " addr=0x%010" PRIx64 " size=0x%016" PRIx64
" extent=%ux%ux%u view_mip=%u view_extent=%ux%u levels=%u pitch=%u"
" fmt=0x%08" PRIx32 " nfmt=0x%08" PRIx32 " order=0x%08" PRIx32 " samples=%u tile=%s\n",
rt_slot, rt.base.addr, backing_size, width, height, depth, rt.view.current_mip_level,
view_extent.width, view_extent.height, levels, pitch,
static_cast<uint32_t>(rt.info.format), static_cast<uint32_t>(rt.info.channel_type),
static_cast<uint32_t>(rt.info.channel_order), samples, tile ? "tiled" : "linear");
}
TextureCache::ImageDesc desc {};
auto& desc = r.desc;
desc.type = TextureCache::BindingType::RenderTarget;
desc.info.data = {rt.base.addr, backing_size};
desc.info.pixel_format = target_format.format;
@@ -304,15 +273,25 @@ void RenderExecutor::ResolveRenderColorTarget(CommandBuffer& buffer, RenderColor
desc.info.samples = samples;
desc.info.tile_mode = rt.attrib3.tile_mode;
const bool has_dcc = rt.info.dcc_compression_enable && rt.dcc_addr.addr != 0;
if (has_dcc) {
const bool has_cmask = !rt.info.dcc_compression_enable && rt.info.cmask_fast_clear_enable &&
rt.cmask.addr != 0 && samples == 1 &&
!rt.info.fmask_compression_enable &&
!rt.attrib3.write_vrs_rate_hint_to_cmask;
if (has_dcc || has_cmask) {
TileSizeAlign metadata_size {};
(void)TileGetDccSize(width, height, volume ? depth : view.image_layers, bytes_per_element,
levels, rt.attrib3.tile_mode, metadata_size, rt.attrib.num_fragments);
desc.info.metadata.kind = ImageMetadataKind::Dcc;
desc.info.metadata.range = {rt.dcc_addr.addr, metadata_size.size};
desc.info.metadata.dcc_clear_word = rt.clear_word0.word0;
desc.info.metadata.dcc_clear_register_valid = true;
desc.info.metadata.dcc_alpha_msb = DccAlphaOnMsb(rt.info);
const auto layers = volume ? depth : view.image_layers;
if (has_dcc) {
(void)TileGetDccSize(width, height, layers, bytes_per_element, levels,
rt.attrib3.tile_mode, metadata_size, rt.attrib.num_fragments);
desc.info.metadata.dcc_alpha_msb = DccAlphaOnMsb(rt.info);
} else {
(void)TileGetCmaskSize(width, height, layers, levels, metadata_size);
}
// DCC owns the clear when both planes are enabled; single-sample CMASK stays expanded.
desc.info.metadata.kind = has_dcc ? ImageMetadataKind::Dcc : ImageMetadataKind::Cmask;
desc.info.metadata.range = {has_dcc ? rt.dcc_addr.addr : rt.cmask.addr, metadata_size.size};
desc.info.metadata.clear_word = rt.clear_word0.word0;
desc.info.metadata.clear_register_valid = true;
}
for (uint32_t level = 0; level < levels; level++) {
if (volume) {
@@ -353,7 +332,6 @@ void RenderExecutor::ResolveRenderColorTarget(CommandBuffer& buffer, RenderColor
desc.view_info.layer_count = view.layer_count;
desc.view_info.usage = vk::ImageUsageFlagBits::eColorAttachment;
auto& texture_cache = m_context.GetTextureCache();
r.desc = std::move(desc);
r.guest_mip_level = rt.view.current_mip_level;
r.guest_array_layer = view.base_layer;
r.image_id = texture_cache.FindImage(r.desc, exact_format);
+2 -14
View File
@@ -209,28 +209,16 @@ static void RtCheck(const HW::RenderTarget& rt) {
}
if (rt.info.fmask_compression_enable) {
EXIT_NOT_IMPLEMENTED(rt.attrib.num_samples == 0 && rt.attrib.num_fragments == 0);
// Native MSAA stores expanded samples, independent of FMASK metadata compression.
static bool logged = false;
if (!logged) {
LOGF("RenderTarget: using native Vulkan MSAA without guest FMASK metadata, "
LOGF("RenderTarget: using expanded native Vulkan MSAA samples, "
"fmask=0x%016" PRIx64 "\n",
rt.fmask.addr);
logged = true;
}
}
EXIT_NOT_IMPLEMENTED(rt.info.fmask_data_compression_disable != false);
if (rt.info.cmask_fast_clear_enable || rt.info.dcc_compression_enable) {
static bool logged = false;
if (!logged) {
LOGF("RenderTarget: temporary: ignoring PS5 color metadata fast_clear=%s dcc=%s "
"cmask=0x%016" PRIx64 " dcc_addr=0x%016" PRIx64 "\n",
rt.info.cmask_fast_clear_enable ? "true" : "false",
rt.info.dcc_compression_enable ? "true" : "false", rt.cmask.addr,
rt.dcc_addr.addr);
logged = true;
}
}
if (rt.info.blend_bypass) {
static bool logged = false;
if (!logged) {
@@ -41,13 +41,42 @@ struct RenderDepthInfo {
[[nodiscard]] vk::ImageAspectFlags AttachmentWriteAspects() const;
};
inline vk::ImageAspectFlags DepthFeedbackAspects(vk::ImageAspectFlags draw_writes,
const ImageViewInfo& target,
const ImageViewInfo& sampled) {
if (!ImageRangeOverlaps(target.base_level, target.level_count, sampled.base_level,
sampled.level_count) ||
!ImageRangeOverlaps(target.base_layer, target.layer_count, sampled.base_layer,
sampled.layer_count)) {
return {};
}
return draw_writes & sampled.aspect;
}
inline vk::ImageAspectFlags DepthReadableAspects(vk::ImageLayout layout) {
switch (layout) {
case vk::ImageLayout::eDepthReadOnlyOptimal:
case vk::ImageLayout::eDepthReadOnlyStencilAttachmentOptimal:
return vk::ImageAspectFlagBits::eDepth;
case vk::ImageLayout::eStencilReadOnlyOptimal:
case vk::ImageLayout::eDepthAttachmentStencilReadOnlyOptimal:
return vk::ImageAspectFlagBits::eStencil;
case vk::ImageLayout::eDepthStencilReadOnlyOptimal:
case vk::ImageLayout::eAttachmentFeedbackLoopOptimalEXT:
case vk::ImageLayout::eGeneral:
return vk::ImageAspectFlagBits::eDepth | vk::ImageAspectFlagBits::eStencil;
default:
return {};
}
}
inline vk::ImageLayout depth_attachment_layout(const RenderDepthInfo& depth) {
const auto available = ImageViewOps::DepthAspectMask(depth.desc.view_info.format);
const auto writes = depth.AttachmentWriteAspects();
const bool has_depth = static_cast<bool>(available & vk::ImageAspectFlagBits::eDepth);
const bool has_stencil = static_cast<bool>(available & vk::ImageAspectFlagBits::eStencil);
// LoadOp clears are separate from guest depth writes.
const bool depth_write = depth.depth_write_enable;
// The attachment layout must permit load clears as well as draw writes.
const bool depth_write = static_cast<bool>(writes & vk::ImageAspectFlagBits::eDepth);
const bool stencil_write = static_cast<bool>(writes & vk::ImageAspectFlagBits::eStencil);
if (!has_stencil) {
return depth_write ? vk::ImageLayout::eDepthAttachmentOptimal
+36 -9
View File
@@ -1,6 +1,7 @@
#include "graphics/host_gpu/renderer/image/image.h"
#include "common/assert.h"
#include "common/logging/log.h"
#include "common/profiler.h"
#include "graphics/host_gpu/renderer/cache/streamBuffer.h"
#include "graphics/host_gpu/renderer/commandScheduler.h"
@@ -10,7 +11,9 @@
#include <algorithm>
#include <array>
#include <atomic>
#include <cstdint>
#include <fmt/format.h>
#include <xxhash.h>
namespace Libs::Graphics {
@@ -48,12 +51,29 @@ namespace {
}
[[nodiscard]] vk::ImageUsageFlags ImageUsageFlags(GraphicContext& graphics, const ImageInfo& info) {
auto usage = vk::ImageUsageFlagBits::eTransferSrc | vk::ImageUsageFlagBits::eTransferDst;
if (info.IsBlock()) {
return vk::ImageUsageFlagBits::eTransferSrc | vk::ImageUsageFlagBits::eTransferDst |
vk::ImageUsageFlagBits::eSampled;
usage |= vk::ImageUsageFlagBits::eSampled;
if (graphics.supports_block_texel_view) {
const auto storage = usage | vk::ImageUsageFlagBits::eStorage;
if (graphics.GetImageFormatProperties(info.pixel_format, HostImageType(info.type),
vk::ImageTiling::eOptimal, storage,
ImageCreateFlags(graphics, info),
nullptr) == vk::Result::eSuccess) {
usage = storage;
} else {
static std::atomic_flag warned = ATOMIC_FLAG_INIT;
if (!warned.test_and_set(std::memory_order_relaxed)) {
Log::WriteToConsoleAndLog(fmt::format(
"Warning: format {} does not support storage access; block-compressed "
"textures written by the guest will not render.\n",
vk::to_string(info.pixel_format)));
}
}
}
return usage;
}
const auto properties = graphics.GetFormatProperties(info.pixel_format);
auto usage = vk::ImageUsageFlagBits::eTransferSrc | vk::ImageUsageFlagBits::eTransferDst;
if (HasFormatFeature(properties, vk::FormatFeatureFlagBits::eSampledImage)) {
usage |= vk::ImageUsageFlagBits::eSampled;
}
@@ -320,7 +340,9 @@ void Image::CopyImage(Image& source) {
EXIT_IF(source.backing.samples != backing.samples);
m_scheduler.EndRendering();
const uint32_t levels = std::min(source.backing.mip_levels, backing.mip_levels);
const uint32_t base_depth = backing.image_type == vk::ImageType::e3D
const uint32_t base_depth = source.backing.image_type == backing.image_type
? std::min(source.backing.extent.depth, backing.extent.depth)
: backing.image_type == vk::ImageType::e3D
? backing.extent.depth
: source.backing.extent.depth;
const auto source_aspect =
@@ -627,12 +649,11 @@ void Validate(const ImageInfo& info) {
}
break;
case ImageMetadataKind::Dcc:
if (info.metadata.range.address == 0 ||
info.metadata.range.address >= TRACKER_ADDRESS_SIZE ||
(info.metadata.range.size != 0 &&
info.metadata.range.size > TRACKER_ADDRESS_SIZE - info.metadata.range.address) ||
case ImageMetadataKind::Cmask:
if (!GuestRange {info.metadata.range.address,
std::max<uint64_t>(info.metadata.range.size, 1)}.Valid() ||
info.metadata.compression == VideoOutCompression::Unsupported) {
EXIT("invalid DCC metadata\n");
EXIT("invalid color metadata\n");
}
break;
}
@@ -690,6 +711,12 @@ Image::Image(GraphicContext& graphics, CommandScheduler& scheduler, const ImageI
create.extent.width, create.extent.height, create.extent.depth,
static_cast<int>(create.format), create.arrayLayers, create.mipLevels);
}
SetVulkanObjectNameF(
graphics.device, backing.image,
"Kyty.Image[guest=0x{:016x} size=0x{:x} extent={}x{}x{} format={} mips={} layers={} samples={}]",
info.data.address, info.data.size, info.extent.width, info.extent.height, info.extent.depth,
static_cast<uint32_t>(info.pixel_format), info.resources.levels, info.resources.layers,
info.samples);
}
uint64_t Image::HashGuestEdges() const {
@@ -17,17 +17,17 @@ namespace Libs::Graphics {
enum class VideoOutCompression : uint8_t { Uncompressed, Dcc256_256_0, Dcc256_64_64, Unsupported };
enum class ImageMetadataKind : uint8_t { None, Htile, Dcc };
enum class ImageMetadataKind : uint8_t { None, Htile, Dcc, Cmask };
struct ImageMetadataInfo {
GuestRange range;
ImageMetadataKind kind = ImageMetadataKind::None;
uint32_t control = 0;
uint32_t dcc_clear_word = 0;
VideoOutCompression compression = VideoOutCompression::Uncompressed;
bool stencil_compressed = false;
bool dcc_clear_register_valid = false;
bool dcc_alpha_msb = true;
ImageMetadataKind kind = ImageMetadataKind::None;
uint32_t control = 0;
uint32_t clear_word = 0;
VideoOutCompression compression = VideoOutCompression::Uncompressed;
bool stencil_compressed = false;
bool clear_register_valid = false;
bool dcc_alpha_msb = true;
};
struct ImageSubresources {
@@ -329,7 +329,9 @@ struct VideoOutFormatPolicy {
VideoOutPixelFormatInfo info;
};
inline constexpr std::array<VideoOutFormatPolicy, 6> VIDEO_OUT_FORMAT_POLICIES {{
inline constexpr uint64_t VIDEO_OUT_PIXEL_FORMAT_R10_G10_B10_A2_BT2100_PQ = 0x8100070422000000ull;
inline constexpr std::array<VideoOutFormatPolicy, 7> VIDEO_OUT_FORMAT_POLICIES {{
{0x8000000022000000ull,
{vk::Format::eR8G8B8A8Srgb, Prospero::BufferFormat::k8_8_8_8Srgb, 4, false}},
{0x8000000000000000ull,
@@ -338,6 +340,8 @@ inline constexpr std::array<VideoOutFormatPolicy, 6> VIDEO_OUT_FORMAT_POLICIES {
{vk::Format::eA2B10G10R10UnormPack32, Prospero::BufferFormat::k10_10_10_2UNorm, 4, false}},
{0x8100000000000000ull,
{vk::Format::eA2R10G10B10UnormPack32, Prospero::BufferFormat::k10_10_10_2UNorm, 4, false}},
{VIDEO_OUT_PIXEL_FORMAT_R10_G10_B10_A2_BT2100_PQ,
{vk::Format::eA2B10G10R10UnormPack32, Prospero::BufferFormat::k10_10_10_2UNorm, 4, false}},
{0xc001000622000000ull,
{vk::Format::eR16G16B16A16Sfloat, Prospero::BufferFormat::k16_16_16_16Float, 8, false}},
{0xc001000600000000ull,
@@ -424,6 +428,20 @@ inline constexpr std::array<VideoOutFormatPolicy, 6> VIDEO_OUT_FORMAT_POLICIES {
return true;
}
// Unlike a register clear, a DWORD fill repeats the same word across the entire pixel.
// Keep this separate: the first register word alone cannot describe a 64-bit color.
[[nodiscard]] inline bool DecodeColorDwordFill(vk::Format format, uint32_t packed,
vk::ClearColorValue& clear) {
if (format == vk::Format::eR16G16B16A16Sfloat) {
if (packed != 0) {
return false;
}
clear = vk::ClearColorValue {};
return true;
}
return DecodePackedColorClear(format, packed, clear);
}
[[nodiscard]] inline bool DecodePackedStencilClear(uint32_t packed, uint8_t& clear) {
const auto value = static_cast<uint8_t>(packed);
if (packed != static_cast<uint32_t>(value) * 0x01010101u) {
@@ -358,10 +358,8 @@ vk::ImageView Image::FindView(const ImageViewInfo& view_info) {
}
vk::ImageViewUsageCreateInfo usage {};
usage.usage = image.usage;
if (!is_storage) {
usage.usage &= ~vk::ImageUsageFlagBits::eStorage;
}
usage.usage = is_storage ? vk::ImageUsageFlagBits::eStorage
: image.usage & ~vk::ImageUsageFlagBits::eStorage;
vk::ImageViewMinLodCreateInfoEXT min_lod {};
if (normalized.min_lod != 0) {
min_lod.minLod = static_cast<float>(normalized.base_level) +
@@ -391,6 +389,12 @@ vk::ImageView Image::FindView(const ImageViewInfo& view_info) {
view_info.level_count, view_info.base_layer, view_info.layer_count,
static_cast<vk::ImageUsageFlags::MaskType>(view_info.usage));
}
SetVulkanObjectNameF(
m_graphics.device, view,
"Kyty.ImageView[guest=0x{:016x} format={} aspect=0x{:x} mip={}+{} layer={}+{}]",
info.data.address, static_cast<uint32_t>(normalized.format),
static_cast<vk::ImageAspectFlags::MaskType>(normalized.aspect), normalized.base_level,
normalized.level_count, normalized.base_layer, normalized.layer_count);
views.push_back({normalized, view});
return view;
}
@@ -31,20 +31,14 @@ namespace ImageViewOps {
ImageViewOps::IsFormatDepthCompatible(view_format);
}
[[nodiscard]] inline bool IsValidImageSwizzle(uint32_t swizzle) noexcept {
[[nodiscard]] inline bool IsValidImageSwizzle(uint32_t swizzle,
uint32_t component_count = 4) noexcept {
if ((swizzle & ~0xfffu) != 0) {
return false;
}
for (uint32_t channel = 0; channel < 4; channel++) {
switch (GetDstSel(swizzle, channel)) {
case 0:
case 1:
case 4:
case 5:
case 6:
case 7: break;
default: return false;
}
const auto selector = GetDstSel(swizzle, channel);
if (selector > 1 && (selector < 4 || selector >= 4 + component_count)) return false;
}
return true;
}
@@ -81,15 +75,8 @@ SelectSampledColorView(vk::Format image_format, vk::Format view_format, uint32_t
[[nodiscard]] inline bool IsSupportedSampledDepthView(vk::Format image_format,
vk::Format view_format,
uint32_t swizzle) noexcept {
if (!IsSupportedSampledDepthFormat(image_format, view_format)) {
return false;
}
switch (swizzle) {
case DstSel(4, 4, 4, 4):
case DstSel(4, 0, 0, 0):
case DstSel(4, 0, 0, 1): return true;
default: return false;
}
return IsSupportedSampledDepthFormat(image_format, view_format) &&
IsValidImageSwizzle(swizzle, 1);
}
[[nodiscard]] inline bool
@@ -103,8 +90,11 @@ IsSupportedSampledDepthResource(const ShaderRecompiler::IR::ImageResource& resou
if (resource.numeric_class != Prospero::TextureNumericClass::Float && resource.depth_compare) {
return false;
}
return resource.mip_mode == ShaderRecompiler::IR::ImageMipMode::None && resource.read &&
!resource.written && !resource.atomic;
return ((resource.mip_mode == ShaderRecompiler::IR::ImageMipMode::None &&
resource.mip_count == 1u) ||
(resource.mip_mode == ShaderRecompiler::IR::ImageMipMode::Dynamic &&
resource.mip_count != 0u)) &&
resource.read && !resource.written && !resource.atomic;
}
inline void ValidateStorageColorView(vk::Format image_format, vk::Format view_format,
@@ -127,7 +117,7 @@ IsSupportedStorageImageResource(const ShaderRecompiler::IR::ImageResource& resou
resource.dimension == ShaderRecompiler::Decoder::ImageDimension::Dim2DArray) &&
((resource.mip_mode == ShaderRecompiler::IR::ImageMipMode::None &&
resource.mip_count == 1u) ||
(resource.mip_mode == ShaderRecompiler::IR::ImageMipMode::DynamicStorage &&
(resource.mip_mode == ShaderRecompiler::IR::ImageMipMode::Dynamic &&
resource.mip_count != 0u)) &&
resource.written &&
(!resource.atomic ||
@@ -175,8 +175,7 @@ SurfaceFormatInfo TextureGetSurfaceFormatInfo(Prospero::BufferFormat format) {
TextureUploadLayout TextureCalcUploadLayout(Prospero::BufferFormat format, uint32_t width,
uint32_t height, uint32_t levels, uint32_t depth,
Prospero::TileMode tile_mode, uint64_t upload_size,
bool allow_depth_tile, bool volume_texture,
const char* owner) {
bool volume_texture, const char* owner) {
TextureUploadLayout layout {};
layout.surface.description = {
format,
@@ -204,8 +203,7 @@ TextureUploadLayout TextureCalcUploadLayout(Prospero::BufferFormat format, uint3
static_cast<uint32_t>(format));
}
} else {
if ((tile_mode == Prospero::TileMode::kDepth && !allow_depth_tile) ||
!TileGetTiledTextureLayout(description, layout.surface)) {
if (!TileGetTiledTextureLayout(description, layout.surface)) {
EXIT("%s: unsupported typed tiled upload: fmt=%u tile=%u "
"size=%" PRIu64 " extent=%ux%u levels=%u\n",
owner, static_cast<uint32_t>(format), static_cast<uint32_t>(tile_mode),
@@ -47,8 +47,7 @@ RenderTargetFormatInfo TextureGetRenderTargetFormat(Prospero::ChannelLayout layo
TextureUploadLayout TextureCalcUploadLayout(Prospero::BufferFormat format, uint32_t width,
uint32_t height, uint32_t levels, uint32_t depth,
Prospero::TileMode tile_mode, uint64_t upload_size,
bool allow_depth_tile, bool volume_texture,
const char* owner);
bool volume_texture, const char* owner);
std::vector<vk::BufferImageCopy> TextureBuildImageCopies(const TextureUploadLayout& layout);
bool TextureBuildGpuTileInfos(uint64_t tiled_size, const std::vector<vk::BufferImageCopy>& regions,
const TextureUploadLayout& layout, uint32_t levels,
+17 -37
View File
@@ -88,30 +88,6 @@ TileManager::~TileManager() {
}
}
TileManager::Scratch TileManager::AllocateScratch(uint64_t size) {
EXIT_IF(size == 0);
vk::BufferCreateInfo create {};
create.size = size;
create.usage = vk::BufferUsageFlagBits::eStorageBuffer | vk::BufferUsageFlagBits::eTransferSrc |
vk::BufferUsageFlagBits::eTransferDst;
VmaAllocationCreateInfo allocate {};
allocate.usage = VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE;
VkBuffer buffer = VK_NULL_HANDLE;
VmaAllocation memory = nullptr;
const auto raw = static_cast<VkBufferCreateInfo>(create);
RequireVulkanSuccess(static_cast<vk::Result>(vmaCreateBuffer(
m_graphics.allocator, &raw, &allocate, &buffer, &memory, nullptr)),
"allocate TileManager scratch buffer");
return {buffer, memory, size};
}
void TileManager::DeferDestroy(Scratch scratch) {
auto allocator = m_graphics.allocator;
m_scheduler.DeferOperation(
[allocator, scratch] { vmaDestroyBuffer(allocator, scratch.buffer, scratch.allocation); });
}
void TileManager::Prepare(bool tile, uint64_t tiled_capacity, uint64_t linear_capacity,
std::span<const GpuTileInfo> infos, uint64_t source_base,
uint64_t target_base, std::vector<Dispatch>& dispatches) {
@@ -369,8 +345,7 @@ TileManager::Result TileManager::Detile(vk::Buffer tiled, uint64_t tiled_offset,
const uint64_t source_base = tiled_offset & (descriptor_alignment - 1);
std::vector<Dispatch> dispatches;
Prepare(false, tiled_capacity, linear_capacity, infos, source_base, 0, dispatches);
auto scratch = AllocateScratch(Common::AlignUp(linear_capacity, 4));
DeferDestroy(scratch);
auto scratch = GetScratchBuffer(linear_capacity, tiled);
Record(tiled, tiled_offset, tiled_capacity, scratch.buffer, 0, scratch.size, dispatches,
true);
return {scratch.buffer, 0, linear_capacity};
@@ -400,11 +375,9 @@ void TileManager::TileImage(Image& image, std::span<const vk::BufferImageCopy> r
std::max<uint64_t>(limits.minStorageBufferOffsetAlignment, 4);
const uint64_t target_base = tiled_offset & (descriptor_alignment - 1);
std::vector<Dispatch> dispatches;
// Reserve all stream parameters before creating a scheduler-lived scratch dependency:
// StreamBuffer::Map is allowed to submit the current tick when it wraps.
// Reserve stream parameters before recording the image download and conversion.
Prepare(true, tiled_capacity, linear_capacity, infos, 0, target_base, dispatches);
auto linear = AllocateScratch(Common::AlignUp(linear_capacity, 4));
DeferDestroy(linear);
auto linear = GetScratchBuffer(linear_capacity, tiled);
image.Download(regions, linear.buffer, 0, linear.size);
Result source {linear.buffer, 0, linear.size};
if (transform == ColorTransform::SwapBgra16) {
@@ -414,10 +387,19 @@ void TileManager::TileImage(Image& image, std::span<const vk::BufferImageCopy> r
dispatches, false);
}
TileManager::Result TileManager::GetScratchBuffer(uint64_t size) {
auto scratch = AllocateScratch(Common::AlignUp(size, 4));
DeferDestroy(scratch);
return {scratch.buffer, 0, scratch.size};
TileManager::Result TileManager::GetScratchBuffer(uint64_t size, vk::Buffer input) {
EXIT_IF(size == 0);
size = Common::AlignUp(size, 4);
auto& buffer = m_scratch[m_scratch[0] && m_scratch[0]->Handle() == input ? 1 : 0];
if (!buffer || buffer->Size() < size) {
if (buffer) {
m_scheduler.DeferOperation([old = std::move(buffer)]() mutable { old.reset(); });
}
buffer = std::make_unique<Buffer>(m_graphics, m_scheduler, MemoryUsage::DeviceLocal, 0,
vk::BufferUsageFlagBits::eStorageBuffer | vk::BufferUsageFlagBits::eTransferSrc |
vk::BufferUsageFlagBits::eTransferDst, size);
}
return {buffer->Handle(), 0, size};
}
TileManager::StorageBinding TileManager::BindStorage(Result buffer, uint64_t size) const {
@@ -668,9 +650,7 @@ void TileManager::SwapBgra16(Result input, Result output, uint32_t pixels) {
TileManager::Result TileManager::SwapBgra16(Result input) {
EXIT_NOT_IMPLEMENTED(input.size == 0 || input.size % 8u != 0 || input.size / 8u > UINT32_MAX);
auto output = AllocateScratch(input.size);
DeferDestroy(output);
Result result {output.buffer, 0, output.size};
auto result = GetScratchBuffer(input.size, input.buffer);
SwapBgra16(input, result, static_cast<uint32_t>(input.size / 8u));
return result;
}
+5 -10
View File
@@ -6,13 +6,14 @@
#include "graphics/host_gpu/vulkanCommon.h"
#include <array>
#include <memory>
#include <span>
#include <vector>
#include <vk_mem_alloc.h>
namespace Libs::Graphics {
class CommandScheduler;
class Buffer;
class Image;
class StreamBuffer;
struct GraphicContext;
@@ -62,7 +63,7 @@ public:
~TileManager();
KYTY_CLASS_NO_COPY(TileManager);
// The returned device-local buffer remains alive through the current scheduler tick.
// Consume scratch results before the next acquisition, or pass their buffer as input.
[[nodiscard]] Result Detile(vk::Buffer tiled, uint64_t tiled_offset, uint64_t tiled_capacity,
uint64_t linear_capacity, std::span<const GpuTileInfo> infos);
void Tile(vk::Buffer linear, uint64_t linear_offset, uint64_t linear_capacity, vk::Buffer tiled,
@@ -71,7 +72,7 @@ public:
uint64_t tiled_offset, uint64_t tiled_capacity, uint64_t linear_capacity,
std::span<const GpuTileInfo> infos,
ColorTransform transform = ColorTransform::None);
[[nodiscard]] Result GetScratchBuffer(uint64_t size);
[[nodiscard]] Result GetScratchBuffer(uint64_t size, vk::Buffer input = nullptr);
void ConvertD16(Result source, Result target, D16Direction direction, bool d32,
const D16Layout& layout);
[[nodiscard]] Result SwapBgra16(Result input);
@@ -105,23 +106,16 @@ private:
uint32_t pipeline_slot = 0;
uint64_t params_offset = 0;
};
struct Scratch {
vk::Buffer buffer = nullptr;
VmaAllocation allocation = nullptr;
uint64_t size = 0;
};
struct StorageBinding {
vk::DescriptorBufferInfo info;
uint32_t base = 0;
};
[[nodiscard]] Scratch AllocateScratch(uint64_t size);
[[nodiscard]] StorageBinding BindStorage(Result buffer, uint64_t size) const;
[[nodiscard]] static uint32_t ConversionRows(uint64_t offset, uint64_t row_stride,
uint64_t active, uint32_t remaining,
uint64_t alignment, uint64_t max_range,
uint32_t max_groups) noexcept;
void DeferDestroy(Scratch scratch);
void Prepare(bool tile, uint64_t tiled_capacity, uint64_t linear_capacity,
std::span<const GpuTileInfo> infos, uint64_t source_base, uint64_t target_base,
std::vector<Dispatch>& dispatches);
@@ -134,6 +128,7 @@ private:
GraphicContext& m_graphics;
CommandScheduler& m_scheduler;
StreamBuffer& m_stream_buffer;
std::array<std::unique_ptr<Buffer>, 2> m_scratch;
vk::DescriptorSetLayout m_descriptor_layout = nullptr;
vk::PipelineLayout m_pipeline_layout = nullptr;
std::array<vk::Pipeline, PipelineCount> m_pipelines {};
@@ -0,0 +1,58 @@
#include "graphics/host_gpu/renderer/pipeline/blendMapping.h"
#include "graphics/guest_gpu/hardwareContext.h"
#include <initializer_list>
namespace Libs::Graphics {
bool BlendFactorIsDualSource(uint8_t factor) {
return factor >= static_cast<uint8_t>(Prospero::BlendFactor::kSrc1Color) &&
factor <= static_cast<uint8_t>(Prospero::BlendFactor::kOneMinusSrc1Alpha);
}
namespace {
bool BlendFactorIsConstantColor(uint8_t factor) {
return factor == static_cast<uint8_t>(Prospero::BlendFactor::kConstantColor) ||
factor == static_cast<uint8_t>(Prospero::BlendFactor::kOneMinusConstantColor);
}
} // namespace
BlendMappingSupport ClassifyBlendMapping(const HW::BlendControl& blend,
const Prospero::ColorComponentMapping& mapping) {
// Color constants are not swizzled with the exports; scalar constant alpha is unaffected.
if (!mapping.IsIdentity() && (BlendFactorIsConstantColor(blend.color_srcblend) ||
BlendFactorIsConstantColor(blend.color_destblend))) {
return BlendMappingSupport::Unsupported;
}
if (mapping.Map(3) == 3u) {
return BlendMappingSupport::Direct;
}
// Moving alpha requires the same equation for all channels.
if (blend.separate_alpha_blend && (blend.alpha_srcblend != blend.color_srcblend ||
blend.alpha_destblend != blend.color_destblend ||
blend.alpha_comb_fcn != blend.color_comb_fcn)) {
return BlendMappingSupport::Unsupported;
}
auto support = BlendMappingSupport::Direct;
for (const auto factor: {blend.color_srcblend, blend.color_destblend}) {
if (BlendFactorIsDualSource(factor)) {
return BlendMappingSupport::Unsupported;
}
switch (static_cast<Prospero::BlendFactor>(factor)) {
case Prospero::BlendFactor::kSrcAlpha:
case Prospero::BlendFactor::kOneMinusSrcAlpha:
support = BlendMappingSupport::SourceAlpha;
break;
case Prospero::BlendFactor::kDstAlpha:
case Prospero::BlendFactor::kOneMinusDstAlpha:
case Prospero::BlendFactor::kSrcAlphaSaturate: return BlendMappingSupport::Unsupported;
default: break;
}
}
return support;
}
} // namespace Libs::Graphics
@@ -0,0 +1,27 @@
#ifndef EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_PIPELINE_BLENDMAPPING_H_
#define EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_PIPELINE_BLENDMAPPING_H_
#include <cstdint>
namespace Libs::Graphics {
namespace HW {
struct BlendControl;
}
namespace Prospero {
struct ColorComponentMapping;
}
enum class BlendMappingSupport {
Direct,
SourceAlpha, // Requires logical alpha in the second blend source.
Unsupported,
};
bool BlendFactorIsDualSource(uint8_t factor);
BlendMappingSupport ClassifyBlendMapping(const HW::BlendControl& blend,
const Prospero::ColorComponentMapping& mapping);
} // namespace Libs::Graphics
#endif // EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_PIPELINE_BLENDMAPPING_H_
@@ -17,6 +17,7 @@
#include "graphics/host_gpu/hostMemory.h"
#include "graphics/host_gpu/renderer/colorRenderTarget.h"
#include "graphics/host_gpu/renderer/debug.h"
#include "graphics/host_gpu/renderer/depthRenderTarget.h"
#include "graphics/host_gpu/renderer/image/imageView.h"
#include "graphics/host_gpu/renderer/image/textureCommon.h"
#include "graphics/host_gpu/renderer/pipeline/shaderResourceBarrier.h"
@@ -90,19 +91,6 @@ vk::DescriptorImageInfo MakeImageInfo(const TextureBinding& texture, uint32_t el
return {nullptr, view, texture.layout};
}
static const char* ShaderStageResourceName(ShaderType stage) {
switch (stage) {
case ShaderType::Vertex: return "Vertex";
case ShaderType::Mesh: return "Mesh";
case ShaderType::Local: return "Local";
case ShaderType::TessellationControl: return "Hull";
case ShaderType::TessellationEvaluation: return "Domain";
case ShaderType::Pixel: return "Pixel";
case ShaderType::Compute: return "Compute";
default: return "Unknown";
}
}
static Prospero::ImageType TextureType(const ShaderTextureResource& descriptor) {
const auto type = descriptor.Type();
return type == Prospero::ImageType::kCube ? Prospero::ImageType::kColor2DArray : type;
@@ -125,8 +113,7 @@ static bool IsMultisampledTexture(Prospero::ImageType type) {
static vk::DescriptorBufferInfo
NativeStorageBuffer(RenderContext& context, const PreparedBindings::BufferSource& source,
const ShaderRecompiler::IR::BufferResource& resource, ShaderType stage,
uint32_t slot, uint32_t& buffer_offset) {
const ShaderRecompiler::IR::BufferResource& resource, uint32_t& buffer_offset) {
buffer_offset = 0;
const auto& [address, size, id] = source;
@@ -151,16 +138,6 @@ NativeStorageBuffer(RenderContext& context, const PreparedBindings::BufferSource
if (resource.written) {
context.GetTextureCache().InvalidateMemoryFromGPU(address, size);
}
const char* access = "Read";
if (resource.written && resource.read) {
access = "ReadWrite";
} else if (resource.written) {
access = "Write";
}
SetVulkanObjectNameF(
graphics.device, result.buffer,
"Kyty.{}.StorageBuffer[slot={} guest=0x{:016x} size=0x{:x} access={} formatted={}]",
ShaderStageResourceName(stage), slot, address, size, access, resource.formatted);
return result;
}
@@ -184,18 +161,19 @@ bool IsSupportedDepthTextureEncoding(const ShaderTextureResource& descriptor, bo
}
const bool full = common && descriptor.fields[4] == field4_expected &&
descriptor.fields[5] == field5_expected;
if (!full || (descriptor.fields[6] == 0 && descriptor.fields[7] != 0) ||
if (!full ||
(descriptor.MsaaDepth() && !IsMultisampledTexture(descriptor.Type()))) {
return false;
}
if (descriptor.fields[6] == 0) {
const auto metadata_control = descriptor.fields[6] & 0x00ffffffu;
if (metadata_control == 0) {
return true;
}
constexpr uint32_t htile_control = 0x00280000u;
const uint32_t expected_control = htile_control | (descriptor.MsaaDepth() ? (1u << 10u) : 0u);
const auto metadata_addr = descriptor.MetaAddr() << 8u;
return (descriptor.fields[6] & 0x00ffffffu) == expected_control && metadata_addr != 0 &&
metadata_addr < TRACKER_ADDRESS_SIZE && (metadata_addr & 0x7fffu) == 0 &&
return metadata_control == expected_control && GuestRange {metadata_addr, 1}.Valid() &&
(metadata_addr & 0x7fffu) == 0 &&
descriptor.TileMode() == Prospero::TileMode::kDepth;
}
@@ -252,10 +230,10 @@ static bool IsSupportedStorageTextureDescriptor(const ShaderRecompiler::IR::Imag
(is_color_2d_array && descriptor.BaseArray5() <= descriptor.Depth());
const bool is_2d =
resource.dimension == ShaderRecompiler::Decoder::ImageDimension::Dim2D && valid_2d_slice;
// Storage cube coordinates address individual faces, including partial cube views.
const bool is_cube = resource.cube && descriptor.Type() == Prospero::ImageType::kCube &&
descriptor.Width5() == descriptor.Height5() &&
descriptor.BaseArray5() <= descriptor.Depth() &&
(descriptor.Depth() - descriptor.BaseArray5() + 1u) % 6u == 0;
descriptor.BaseArray5() <= descriptor.Depth();
const bool is_2d_array =
resource.dimension == ShaderRecompiler::Decoder::ImageDimension::Dim2DArray &&
((!resource.cube && is_color_2d_array && descriptor.BaseArray5() <= descriptor.Depth()) ||
@@ -333,12 +311,15 @@ void ValidateStorageTexture(const ShaderRecompiler::IR::ImageResource& resource,
const bool raw_sint_storage = format == Prospero::BufferFormat::k32SInt && uint_resource &&
resource.written && !resource.read && !resource.atomic;
const auto numeric_class = Prospero::SampledTextureNumericClass(format);
const bool raw_float_atomic = format == Prospero::BufferFormat::k32Float && uint_resource &&
resource.atomic && !resource.atomic64;
const bool format_ok =
raw_sint_storage ||
raw_sint_storage || raw_float_atomic ||
(numeric_class != Prospero::TextureNumericClass::Unsupported &&
numeric_class != Prospero::TextureNumericClass::Sint &&
uint_resource == (numeric_class == Prospero::TextureNumericClass::Uint) &&
(!resource.atomic || format == Prospero::BufferFormat::k32UInt));
(!resource.atomic || format == (resource.atomic64 ? Prospero::BufferFormat::k32_32UInt
: Prospero::BufferFormat::k32UInt)));
if (resource_ok && descriptor_ok && encoding_ok && format_ok && size != 0) {
return;
}
@@ -374,7 +355,8 @@ static TextureCache::ImageDesc NullTextureDesc(const ShaderRecompiler::IR::Image
desc.info.guest_format = Prospero::BufferFormat::k32Float;
break;
case Prospero::TextureNumericClass::Uint:
desc.info.guest_format = Prospero::BufferFormat::k32UInt;
desc.info.guest_format = resource.atomic64 ? Prospero::BufferFormat::k32_32UInt
: Prospero::BufferFormat::k32UInt;
break;
case Prospero::TextureNumericClass::Sint:
desc.info.guest_format = Prospero::BufferFormat::k32SInt;
@@ -385,10 +367,10 @@ static TextureCache::ImageDesc NullTextureDesc(const ShaderRecompiler::IR::Image
desc.info.type = Prospero::ImageType::kColor2D;
desc.info.extent = {1, 1, 1};
desc.info.resources = {1, 1};
desc.info.bytes_per_block = 4;
desc.info.bytes_per_block = Prospero::NumBytesPerElement(desc.info.guest_format);
desc.info.samples = 1;
desc.info.mip_layout[0] = {0, 0, 1, 1};
desc.view_info.format = desc.info.pixel_format;
desc.view_info.format = resource.atomic64 ? vk::Format::eR64Uint : desc.info.pixel_format;
desc.view_info.type = vk::ImageViewType::e2D;
desc.view_info.aspect = vk::ImageAspectFlagBits::eColor;
desc.view_info.usage = binding == TextureCache::BindingType::Storage
@@ -510,23 +492,57 @@ static ImageViewInfo TextureViewInfo(const ShaderRecompiler::IR::ImageResource&
return view;
}
static bool TextureViewPreservesMipLayout(const TileSurfaceDescription& description,
uint32_t view_levels) {
static bool ResolveTextureMipView(const TileSurfaceDescription& description, bool metadata,
uint32_t view_levels, uint32_t& levels, uint32_t& base_level) {
TileSurfaceLayout physical {};
TileSurfaceLayout view {};
auto view_description = description;
view_description.levels = view_levels;
return TileGetTiledTextureLayout(description, physical) &&
TileGetTiledTextureLayout(view_description, view) &&
physical.first_tail_level == view.first_tail_level &&
physical.block_slice_size == view.block_slice_size &&
physical.total_size == view.total_size &&
std::equal(std::begin(physical.mips), std::begin(physical.mips) + description.levels,
std::begin(view.mips));
view_description.levels = levels;
if (!TileGetTiledTextureLayout(description, physical) ||
!TileGetTiledTextureLayout(view_description, view)) {
return false;
}
if (physical.first_tail_level == view.first_tail_level &&
physical.block_slice_size == view.block_slice_size &&
physical.total_size == view.total_size &&
std::equal(std::begin(physical.mips), std::begin(physical.mips) + description.levels,
std::begin(view.mips))) {
return true;
}
if (metadata || ((description.layers > 1 || description.depth > 1) &&
physical.block_slice_size != view.block_slice_size)) {
return false;
}
// T# addresses the last mip. A view can select the same stored subresources
// with different mip indices; inaccessible mips need no host representation.
for (uint32_t base = 0; base + view_levels <= description.levels; ++base) {
bool matches = true;
for (uint32_t i = 0; i < view_levels; ++i) {
const auto source = base_level + i;
const auto target = base + i;
if (physical.mips[target] != view.mips[source] ||
(target >= physical.first_tail_level) != (source >= view.first_tail_level) ||
std::max(description.width >> target, 1u) != std::max(description.width >> source, 1u) ||
std::max(description.height >> target, 1u) != std::max(description.height >> source, 1u) ||
std::max(description.depth >> target, 1u) != std::max(description.depth >> source, 1u)) {
matches = false;
break;
}
}
if (matches) {
levels = description.levels;
base_level = base;
return true;
}
}
return false;
}
TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageResource& resource,
const ShaderRecompiler::IR::DescriptorValue& value) {
if (resource.atomic64 && !m_context.GetGraphics().shader_image_int64_atomics_enabled) {
EXIT("64-bit image atomics require shaderImageInt64Atomics\n");
}
auto descriptor = DecodeNativeDescriptor<ShaderTextureResource>(value);
const bool storage = resource.written;
if (storage) {
@@ -550,8 +566,14 @@ TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageR
const bool multisampled = IsMultisampledTexture(type);
const auto max_mip = resource.r128 ? last_level : descriptor.MaxMip();
const auto physical_levels = multisampled ? 1u : static_cast<uint32_t>(max_mip) + 1u;
const auto levels =
multisampled ? 1u : std::max(physical_levels, static_cast<uint32_t>(last_level) + 1u);
// IMAGE_STORE addresses BASE_LEVEL; only IMAGE_STORE_MIP selects other view mips.
const bool single_storage_mip =
storage && resource.mip_mode != ShaderRecompiler::IR::ImageMipMode::Dynamic;
const auto view_levels = multisampled || single_storage_mip
? 1u
: static_cast<uint32_t>(last_level - base_level) + 1u;
auto levels =
multisampled ? 1u : std::max(physical_levels, base_level + view_levels);
const auto tile = descriptor.TileMode();
const bool depth_tile = tile == Prospero::TileMode::kDepth;
const bool msaa_tile = depth_tile || tile == Prospero::TileMode::kRenderTarget;
@@ -574,8 +596,6 @@ TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageR
descriptor.fields[6], descriptor.fields[7]);
}
const auto samples = multisampled ? 1u << last_level : 1u;
const auto view_levels =
multisampled ? 1u : static_cast<uint32_t>(last_level - base_level) + 1u;
const auto depth = static_cast<uint32_t>(descriptor.Depth()) + 1u;
const auto format = descriptor.Format();
const auto surface_format = TextureGetSurfaceFormatInfo(format);
@@ -594,12 +614,13 @@ TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageR
type == Prospero::ImageType::kColor2DArray ||
type == Prospero::ImageType::kColor2DMsaaArray;
const auto image_layers = layered ? depth : 1u;
auto view_base = static_cast<uint32_t>(base_level);
if (levels > physical_levels) {
const TileSurfaceDescription physical {
format, tile, volume ? TileSurfaceDimension::Dim3D : TileSurfaceDimension::Dim2D,
width, height, volume ? depth : 1u, physical_levels, image_layers};
// Texture mip views take precedence over the resource count, but must keep its storage layout.
if (!TextureViewPreservesMipLayout(physical, levels)) {
if (!ResolveTextureMipView(physical, !resource.r128 && descriptor.MetaCompress(),
view_levels, levels, view_base)) {
EXIT("unsupported texture mip view changes physical layout: base=%u last=%u max=%u "
"extent=%ux%ux%u tile=%u\n",
base_level, last_level, max_mip, width, height, depth,
@@ -634,7 +655,9 @@ TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageR
pixel_format = depth_format->depth_attachment_format;
}
}
const auto storage_view_format = storage && format == Prospero::BufferFormat::k32SInt
const auto storage_view_format = resource.atomic64 ? vk::Format::eR64Uint
: storage && (resource.atomic ||
format == Prospero::BufferFormat::k32SInt)
? vk::Format::eR32Uint
: SrgbStorageViewFormat(pixel_format);
const auto view_format = storage && storage_view_format != vk::Format::eUndefined
@@ -670,6 +693,7 @@ TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageR
}
desc.view_info = TextureViewInfo(resource, descriptor, view_format, surface_format, storage,
view_levels, desc.info.resources.layers);
desc.view_info.base_level = view_base;
desc.type = storage ? TextureCache::BindingType::Storage : TextureCache::BindingType::Texture;
auto id = texture_cache.FindImage(desc, shader_conversion);
@@ -696,14 +720,15 @@ static vk::Sampler NativeSampler(RenderContext& context,
const ShaderRecompiler::IR::CompiledShaderInfo& program,
uint32_t index,
const ShaderRecompiler::IR::DescriptorValue& value) {
auto descriptor = DecodeNativeDescriptor<ShaderSamplerResource>(value);
if (!program.info.samplers[index].depth_compare) {
auto descriptor = DecodeNativeDescriptor<ShaderSamplerResource>(value);
const auto& sampler = program.info.samplers[index];
if (!sampler.depth_compare) {
descriptor.fields[0] &= ~(0x7u << 12u);
}
if (program.info.samplers[index].force_point_filtering) {
if (sampler.force_point_filtering) {
descriptor.SetPointFiltering();
}
return context.GetSamplerCache().GetSampler(descriptor);
return context.GetSamplerCache().GetSampler(descriptor, sampler.integer_border);
}
static vk::DescriptorBufferInfo NativeUpload(RenderContext& context,
@@ -744,22 +769,30 @@ void RenderExecutor::ResetBindings() {
m_bound_images.clear();
}
PreparedBindings RenderExecutor::PrepareBindings(const ShaderStageRuntime& runtime) {
void RenderExecutor::PrepareBindings(const ShaderStageRuntime& runtime,
PreparedBindings& prepared) {
KYTY_PROFILER_FUNCTION();
EXIT_IF(!runtime);
const auto& program = *runtime.program;
const auto& snapshot = runtime.resources;
PreparedBindings prepared;
const auto& snapshot = *runtime.resources;
prepared.runtime = &runtime;
prepared.images.reserve(program.info.images.size());
prepared.gds = {nullptr, 0, VK_WHOLE_SIZE};
prepared.flattened_srt = {};
prepared.shader_data_buffer = {};
prepared.images.resize(program.info.images.size());
prepared.samplers.clear();
prepared.shader_data.clear();
for (uint32_t i = 0; i < program.info.images.size(); i++) {
auto binding = ResolveTexture(program.info.images[i], snapshot.images[i]);
BindImage(binding.image_id, binding.desc.type == TextureCache::BindingType::Storage);
prepared.images.push_back(std::move(binding));
binding.mip_views.swap(prepared.images[i].mip_views);
binding.mip_views.clear();
prepared.images[i] = std::move(binding);
}
prepared.samplers.reserve(program.info.samplers.size());
for (uint32_t i = 0; i < program.info.samplers.size(); i++) {
prepared.samplers.push_back(NativeSampler(m_context, program, i, snapshot.samplers[i]));
prepared.samplers.push_back(NativeSampler(
m_context, program, i, snapshot.samplers[program.info.samplers[i].snapshot_index]));
}
prepared.shader_data.reserve(program.bindings.ShaderDataDwords());
for (const auto reg: program.bindings.user_data_registers) {
@@ -770,20 +803,24 @@ PreparedBindings RenderExecutor::PrepareBindings(const ShaderStageRuntime& runti
program.bindings, ShaderRecompiler::IR::DescriptorBindingKind::Gds) != nullptr) {
prepared.gds.buffer = m_context.GetBufferCache().GetGdsBuffer()->Handle();
}
return prepared;
}
void RenderExecutor::FindBuffers(PreparedBindings& prepared) {
KYTY_PROFILER_FUNCTION();
EXIT_IF(prepared.runtime == nullptr || !*prepared.runtime);
const auto& program = *prepared.runtime->program;
const auto& snapshot = prepared.runtime->resources;
const auto& snapshot = *prepared.runtime->resources;
auto& cache = m_context.GetBufferCache();
prepared.buffer_sources.clear();
prepared.buffer_sources.reserve(program.info.buffers.size());
for (uint32_t i = 0; i < program.info.buffers.size(); i++) {
auto descriptor = DecodeNativeDescriptor<ShaderBufferResource>(snapshot.buffers[i]);
const auto& layout = program.bindings;
if (layout.memory_offset_count == 0) {
return;
}
const auto& resources = layout.descriptors.front().resources;
prepared.buffer_sources.reserve(resources.size());
for (const auto resource: resources) {
auto descriptor = DecodeNativeDescriptor<ShaderBufferResource>(snapshot.buffers[resource]);
const auto address = descriptor.Base48();
const auto requested_size = descriptor.GetSize();
if (address == 0 || requested_size == 0) {
@@ -799,12 +836,12 @@ void RenderExecutor::RebindBuffers(PreparedBindings& prepared) {
KYTY_PROFILER_FUNCTION();
EXIT_IF(prepared.runtime == nullptr || !*prepared.runtime);
const auto& program = *prepared.runtime->program;
const auto& snapshot = prepared.runtime->resources;
const auto& snapshot = *prepared.runtime->resources;
const auto& layout = program.bindings;
EXIT_IF(prepared.buffer_sources.size() != program.info.buffers.size());
EXIT_IF(prepared.buffer_sources.size() != layout.memory_offset_count);
prepared.buffers.clear();
prepared.buffers.reserve(program.info.buffers.size());
prepared.buffers.reserve(layout.memory_offset_count);
EXIT_IF(prepared.shader_data.size() != layout.ShaderDataDwords());
std::fill(prepared.shader_data.begin() + layout.memory_offset_dword,
prepared.shader_data.end(), 0);
@@ -813,10 +850,11 @@ void RenderExecutor::RebindBuffers(PreparedBindings& prepared) {
const auto shift = (index % 4u) * 8u;
prepared.shader_data[dword] |= offset << shift;
};
for (uint32_t i = 0; i < program.info.buffers.size(); i++) {
for (uint32_t i = 0; i < layout.memory_offset_count; i++) {
const auto resource = layout.descriptors.front().resources[i];
uint32_t buffer_offset = 0;
prepared.buffers.push_back(NativeStorageBuffer(m_context, prepared.buffer_sources[i],
program.info.buffers[i], program.stage, i,
program.info.buffers[resource],
buffer_offset));
pack_memory_offset(i, buffer_offset);
}
@@ -834,7 +872,7 @@ void RenderExecutor::RebindImages(PreparedBindings& prepared) {
KYTY_PROFILER_FUNCTION();
EXIT_IF(prepared.runtime == nullptr || !*prepared.runtime);
const auto& program = *prepared.runtime->program;
const auto& snapshot = prepared.runtime->resources;
const auto& snapshot = *prepared.runtime->resources;
auto& images = prepared.images;
EXIT_IF(images.size() != program.info.images.size());
auto& texture_cache = m_context.GetTextureCache();
@@ -854,7 +892,7 @@ void RenderExecutor::RebindImages(PreparedBindings& prepared) {
auto& binding = images[i];
binding.mip_views.clear();
const auto& resource = program.info.images[i];
if (resource.mip_mode == ShaderRecompiler::IR::ImageMipMode::DynamicStorage) {
if (resource.mip_mode == ShaderRecompiler::IR::ImageMipMode::Dynamic) {
EXIT_IF(resource.mip_count == 0u ||
resource.mip_count != binding.desc.view_info.level_count);
binding.mip_views.reserve(resource.mip_count);
@@ -862,15 +900,13 @@ void RenderExecutor::RebindImages(PreparedBindings& prepared) {
auto desc = binding.desc;
desc.view_info.base_level += mip;
desc.view_info.level_count = 1;
// The shader selects the mip after applying the guest minimum LOD.
desc.view_info.min_lod = 0;
binding.mip_views.push_back(texture_cache.FindTexture(binding.image_id, desc));
}
binding.image_view = binding.mip_views.front();
} else {
auto desc = binding.desc;
if (desc.type == TextureCache::BindingType::Storage) {
desc.view_info.level_count = 1;
}
binding.image_view = texture_cache.FindTexture(binding.image_id, desc);
binding.image_view = texture_cache.FindTexture(binding.image_id, binding.desc);
}
auto& image = texture_cache.GetImage(binding.image_id);
const bool storage = binding.desc.type == TextureCache::BindingType::Storage;
@@ -945,6 +981,41 @@ void RenderExecutor::CommitBindings(CommandBuffer& buffer,
(pipeline_bind_point == vk::PipelineBindPoint::eCompute &&
shader_stage != vk::ShaderStageFlagBits::eCompute));
}
for (const auto* reader: prepared_bindings) {
const auto& reads = reader->runtime->resources->specialization_reads;
if (reads.empty()) continue;
for (const auto* writer: prepared_bindings) {
if (writer->runtime->program->has_address_writes) {
EXIT("scalar resource reads cannot be proven disjoint from shader address writes\n");
}
}
for (const auto [address, size]: reads) {
for (const auto id: m_bound_images) {
const auto* image = m_context.GetTextureCache().m_slot_images.try_get(id);
if (image == nullptr ||
(!image->binding.shader_write && !image->binding.is_target)) continue;
for (const auto written: {image->info.data, image->info.stencil,
image->info.metadata.range}) {
if (written.size != 0 && ImageRangeOverlaps(address, size,
written.address, written.size)) {
EXIT("scalar resource reads overlap an image or attachment write\n");
}
}
}
for (const auto* writer: prepared_bindings) {
const auto& program = *writer->runtime->program;
for (uint32_t i = 0; i < writer->buffer_sources.size(); ++i) {
const auto resource = program.bindings.descriptors.front().resources[i];
if (!program.info.buffers[resource].written) continue;
const auto& written = writer->buffer_sources[i];
if (written.size != 0 && ImageRangeOverlaps(address, size,
written.address, written.size)) {
EXIT("scalar resource reads overlap a shader buffer write\n");
}
}
}
}
}
m_descriptor_buffers.clear();
m_descriptor_images.clear();
m_descriptor_writes.clear();
@@ -988,19 +1059,7 @@ void RenderExecutor::CommitBindings(CommandBuffer& buffer,
std::ranges::find(image.views, binding.image_view, &CachedImageView::view);
EXIT_IF(storage || host_view == image.views.end());
const auto aspect = host_view->info.aspect;
const bool depth_feedback =
layout == vk::ImageLayout::eAttachmentFeedbackLoopOptimalEXT &&
program.stage == ShaderType::Pixel;
const bool depth_read =
depth_feedback || layout == vk::ImageLayout::eDepthReadOnlyOptimal ||
layout == vk::ImageLayout::eDepthStencilReadOnlyOptimal ||
layout == vk::ImageLayout::eDepthReadOnlyStencilAttachmentOptimal;
const bool stencil_read =
layout == vk::ImageLayout::eStencilReadOnlyOptimal ||
layout == vk::ImageLayout::eDepthStencilReadOnlyOptimal ||
layout == vk::ImageLayout::eDepthAttachmentStencilReadOnlyOptimal;
if ((aspect & vk::ImageAspectFlagBits::eDepth && !depth_read) ||
(aspect & vk::ImageAspectFlagBits::eStencil && !stencil_read)) {
if (aspect & ~DepthReadableAspects(layout)) {
EXIT("sampling a writable depth/stencil attachment aspect\n");
}
}
@@ -1044,8 +1103,8 @@ void RenderExecutor::CommitBindings(CommandBuffer& buffer,
} else {
switch (binding.kind) {
case BindingKind::Buffers:
for (const auto resource: binding.resources) {
const auto& view = descriptors.buffers.at(resource);
EXIT_IF(descriptors.buffers.size() != binding.resources.size());
for (const auto& view: descriptors.buffers) {
EXIT_IF(view.buffer == nullptr);
m_descriptor_buffers.push_back(view);
}
@@ -5,11 +5,13 @@
#include "common/file.h"
#include "common/logging/log.h"
#include "common/profiler.h"
#include "common/threads.h"
#include "graphics/guest_gpu/hardwareContext.h"
#include "graphics/host_gpu/renderer/colorRenderTarget.h"
#include "graphics/host_gpu/renderer/debug.h"
#include "graphics/host_gpu/renderer/depthRenderTarget.h"
#include "graphics/host_gpu/renderer/image/imageView.h"
#include "graphics/host_gpu/renderer/pipeline/blendMapping.h"
#include "graphics/host_gpu/renderer/render.h"
#include "graphics/host_gpu/renderer/renderContext.h"
#include "graphics/shader/recompiler/ShaderRecompiler.h"
@@ -29,7 +31,6 @@
#include <span>
#include <spirv-tools/libspirv.hpp>
#include <string_view>
#include <tuple>
#include <utility>
#include <vector>
#include <xxhash.h>
@@ -92,9 +93,9 @@ void PipelineCacheLog(fmt::format_string<Args...> format, Args&&... args) {
Log::WriteToConsoleAndLog(message);
}
bool ReadShaderGuestMemory(void*, uint64_t address, uint32_t* value) {
return value != nullptr &&
Libs::LibKernel::Memory::TryReadGpuCleanBacking(address, value, sizeof(*value));
bool ReadShaderGuestMemory(void*, uint64_t address, std::span<uint32_t> values) {
return !values.empty() &&
Libs::LibKernel::Memory::TryReadGpuCleanBacking(address, values.data(), values.size_bytes());
}
void DumpShaderSpirv(const char* stage_name, uint64_t shader_hash,
@@ -116,33 +117,22 @@ void DumpShaderSpirv(const char* stage_name, uint64_t shader_hash,
}
void DumpShaderOriginal(const char* stage_name, uint64_t shader_hash,
std::span<const uint32_t> code, const std::string& decoded_dump) {
std::span<const uint32_t> code) {
if (!Config::GraphicsDebugDumpEnabled()) {
return;
}
EXIT_IF(code.empty());
static std::atomic_int id = 0;
const auto base = Config::GetShaderLogFolder() / "original" /
fmt::format("{:04d}_new_shader_{}_{:016x}", id++, stage_name, shader_hash);
Common::File::CreateDirectories(base.parent_path());
for (const auto& [suffix, data, size]: {
std::tuple {".bin", static_cast<const void*>(code.data()), code.size_bytes()},
std::tuple {".rdna2", static_cast<const void*>(decoded_dump.data()),
decoded_dump.size()},
}) {
if (size == 0) {
continue;
}
auto path = base;
path += suffix;
Common::File file(path);
if (file.IsInvalid()) {
const auto path_text = Common::PathToString(path);
LOGF_COLOR(Log::Color::BrightRed, "Can't create file: %s\n", path_text.c_str());
} else {
file.Write(data, size);
}
const auto path = Config::GetShaderLogFolder() / "original" /
fmt::format("{:04d}_new_shader_{}_{:016x}.bin", id++, stage_name, shader_hash);
Common::File::CreateDirectories(path.parent_path());
Common::File file(path);
if (file.IsInvalid()) {
const auto path_text = Common::PathToString(path);
LOGF_COLOR(Log::Color::BrightRed, "Can't create file: %s\n", path_text.c_str());
return;
}
file.Write(code.data(), code.size_bytes());
}
bool ValidateShaderSpirv(const char* label, uint64_t shader_hash,
@@ -177,6 +167,32 @@ bool ValidateShaderSpirv(const char* label, uint64_t shader_hash,
} // namespace
std::size_t PipelineCache::GraphicsPipelineKeyHash::operator()(const GraphicsPipelineKey& key) const {
std::size_t hash = 0;
PipelineKeyHash::Mix(hash, key.rendering.color_count);
for (uint32_t i = 0; i < key.rendering.color_count; i++) {
PipelineKeyHash::Mix(hash, static_cast<uint32_t>(key.rendering.color_formats[i]));
}
PipelineKeyHash::Mix(hash, static_cast<uint32_t>(key.rendering.depth_format));
PipelineKeyHash::Mix(hash, static_cast<uint32_t>(key.rendering.stencil_format));
for (const auto id: key.vertex_shader_ids) {
PipelineKeyHash::Mix(hash, id);
}
PipelineKeyHash::Mix(hash, key.ps_shader_id);
PipelineKeyHash::Mix(hash, key.vertex_input.binding_count);
for (uint32_t i = 0; i < key.vertex_input.binding_count; i++) {
PipelineKeyHash::Mix(hash, key.vertex_input.bindings[i].stride);
PipelineKeyHash::Mix(hash, key.vertex_input.bindings[i].instance);
}
PipelineKeyHash::Mix(hash, key.vertex_input.attribute_count);
for (uint32_t i = 0; i < key.vertex_input.attribute_count; i++) {
PipelineKeyHash::Mix(hash, key.vertex_input.attributes[i].offset);
PipelineKeyHash::Mix(hash, key.vertex_input.attributes[i].binding);
}
PipelineKeyHash::Mix(hash, XXH3_64bits(&key.static_params, sizeof(key.static_params)));
return hash;
}
struct PipelineCache::ProgramCache {
struct ProgramKey {
ShaderType stage = ShaderType::Unknown;
@@ -200,8 +216,10 @@ struct PipelineCache::ProgramCache {
permutations.reserve(8);
}
ShaderRecompiler::IR::ResourcePlan resource_plan;
std::vector<Permutation> permutations;
ShaderRecompiler::IR::ResourcePlan resource_plan;
ShaderRecompiler::IR::ResourceSnapshot resources;
ShaderRecompiler::IR::ResourceSpecialization specialization;
std::vector<Permutation> permutations;
};
struct ProgramKeyHash {
@@ -215,32 +233,20 @@ struct PipelineCache::ProgramCache {
PipelineKeyHash::Mix(hash, key.code_size);
PipelineKeyHash::Mix(hash, key.static_state.size());
// Bucket same-shape static variants by source. ProgramKey equality performs the one
// exact state comparison needed on a stable hit without hashing up to 429 words first.
// exact state comparison needed on a stable hit without hashing the full state first.
return hash;
}
};
static constexpr std::size_t MaxStaticKeyWords = 13 + ShaderVertexInputInfo::RES_MAX * 13;
static constexpr std::size_t MaxStaticKeyWords = 32 + ShaderVertexInputInfo::RES_MAX * 6;
Permutation CompilePermutation(const ShaderParams& params,
Permutation CompilePermutation(const char* stage_name,
const ShaderRecompiler::CompileOptions& options,
ShaderRecompiler::TranslateResult translated,
ShaderRecompiler::IR::ResourceSpecialization specialization,
uint32_t push_data_start_dword) {
const char* stage_name = nullptr;
switch (options.stage) {
case ShaderType::Vertex: stage_name = "vs"; break;
case ShaderType::Mesh: stage_name = "ms"; break;
case ShaderType::Local: stage_name = "ls"; break;
case ShaderType::TessellationControl: stage_name = "hs"; break;
case ShaderType::TessellationEvaluation: stage_name = "ds"; break;
case ShaderType::Pixel: stage_name = "ps"; break;
case ShaderType::Compute: stage_name = "cs"; break;
default: EXIT("invalid pipeline shader stage\n");
}
auto result = ShaderRecompiler::CompileProgram(std::move(translated), options,
specialization, push_data_start_dword);
DumpShaderOriginal(stage_name, options.shader_hash, params.code, result.decoded_dump);
if (!ValidateShaderSpirv(options.dump_label, options.shader_hash, result.spirv)) {
DumpShaderSpirv(stage_name, options.shader_hash, result.spirv);
EXIT("%s failed hash=0x%016" PRIx64 ": SPIR-V validation failed\n", options.dump_label,
@@ -274,33 +280,33 @@ struct PipelineCache::ProgramCache {
stage = ShaderType::Compute;
}
const auto user_data = std::span(params.user_data).first(params.user_data_count);
lookup_key.stage = stage;
lookup_key.hash = params.hash;
lookup_key.user_data_count = static_cast<uint32_t>(params.user_data.size());
lookup_key.user_data_count = params.user_data_count;
lookup_key.code_size = static_cast<uint32_t>(params.code.size());
BuildStageStaticKey(input_info, lookup_key.static_state);
auto entry = programs.find(lookup_key);
ShaderRecompiler::IR::ResourceSnapshot resources;
ShaderRecompiler::IR::ResourceSpecialization specialization;
const ShaderRecompiler::IR::SrtRuntime runtime {
.user_data = params.user_data,
.user_data = user_data,
.shader_base = params.Base(),
.read_specialization_memory = ReadShaderGuestMemory,
};
if (entry != programs.end()) {
EXIT_IF(!ShaderRecompiler::IR::MaterializeResources(
entry->second.resource_plan, runtime, resources, specialization));
entry->second.resource_plan, runtime, entry->second.resources,
entry->second.specialization));
if (const auto permutation = std::ranges::find_if(
entry->second.permutations, [&](const Permutation& candidate) {
const auto& layout = candidate.program.bindings;
return layout.push_data_start_dword ==
ShaderRecompiler::IR::PushData::StartFor(
push_data_cursor, layout.ShaderDataDwords()) &&
candidate.specialization == specialization;
candidate.specialization == entry->second.specialization;
});
permutation != entry->second.permutations.end()) {
input_info.stage = {.program = &permutation->program,
.resources = std::move(resources)};
.resources = &entry->second.resources};
permutation->program.bindings.AdvancePushData(push_data_cursor);
return permutation->handle;
}
@@ -315,20 +321,21 @@ struct PipelineCache::ProgramCache {
stage_input.compute = &input_info;
}
const char* label = nullptr;
const char* stage_name = nullptr;
switch (stage) {
case ShaderType::Vertex: label = "ShaderRecompiler VS"; break;
case ShaderType::Mesh: label = "ShaderRecompiler MS"; break;
case ShaderType::Local: label = "ShaderRecompiler LS"; break;
case ShaderType::TessellationControl: label = "ShaderRecompiler HS"; break;
case ShaderType::TessellationEvaluation: label = "ShaderRecompiler DS"; break;
case ShaderType::Pixel: label = "ShaderRecompiler PS"; break;
case ShaderType::Compute: label = "ShaderRecompiler CS"; break;
case ShaderType::Vertex: label = "ShaderRecompiler VS"; stage_name = "vs"; break;
case ShaderType::Mesh: label = "ShaderRecompiler MS"; stage_name = "ms"; break;
case ShaderType::Local: label = "ShaderRecompiler LS"; stage_name = "ls"; break;
case ShaderType::TessellationControl: label = "ShaderRecompiler HS"; stage_name = "hs"; break;
case ShaderType::TessellationEvaluation: label = "ShaderRecompiler DS"; stage_name = "ds"; break;
case ShaderType::Pixel: label = "ShaderRecompiler PS"; stage_name = "ps"; break;
case ShaderType::Compute: label = "ShaderRecompiler CS"; stage_name = "cs"; break;
default: EXIT("invalid pipeline shader stage\n");
}
ShaderRecompiler::CompileOptions options;
options.stage = stage;
options.shader_hash = params.hash;
options.user_data = params.user_data;
options.user_data = user_data;
options.back_code = params.back_code;
options.dump_ir = Config::GetShaderLogDirection() != Config::LogDirection::Silent;
options.early_dump = options.dump_ir;
@@ -337,6 +344,7 @@ struct PipelineCache::ProgramCache {
if constexpr (std::is_same_v<InputInfo, ShaderVertexInputInfo>) {
options.user_data_base = 8;
options.wave_size = input_info.wave_size;
if (stage == ShaderType::Mesh || stage == ShaderType::TessellationControl) {
options.user_data_base = 0;
options.wave_size = stage == ShaderType::Mesh ? input_info.mesh.wave_size : 64u;
@@ -344,17 +352,19 @@ struct PipelineCache::ProgramCache {
} else {
options.wave_size = input_info.wave_size;
}
DumpShaderOriginal(stage_name, options.shader_hash, params.code);
auto translated = ShaderRecompiler::TranslateProgram(params.code, options);
if (entry == programs.end()) {
auto resource_plan = ShaderRecompiler::IR::ExtractResourcePlan(translated.program);
EXIT_IF(!ShaderRecompiler::IR::MaterializeResources(resource_plan, runtime, resources,
specialization));
entry = programs.try_emplace(lookup_key, std::move(resource_plan)).first;
entry = programs.try_emplace(lookup_key,
ShaderRecompiler::IR::ExtractResourcePlan(translated.program)).first;
EXIT_IF(!ShaderRecompiler::IR::MaterializeResources(
entry->second.resource_plan, runtime, entry->second.resources,
entry->second.specialization));
}
entry->second.permutations.push_back(CompilePermutation(
params, options, std::move(translated), std::move(specialization), push_data_cursor));
stage_name, options, std::move(translated), entry->second.specialization, push_data_cursor));
const auto& permutation = entry->second.permutations.back();
input_info.stage = {.program = &permutation.program, .resources = std::move(resources)};
input_info.stage = {.program = &permutation.program, .resources = &entry->second.resources};
permutation.program.bindings.AdvancePushData(push_data_cursor);
std::array<size_t, static_cast<size_t>(ShaderType::TessellationEvaluation) + 1> counts {};
@@ -501,7 +511,6 @@ void PipelineCache::InitializeDriverCache() {
}
void PipelineCache::Save() {
Common::LockGuard lock(m_mutex);
if (m_driver_cache == nullptr) {
return;
}
@@ -592,21 +601,30 @@ PipelineCache::GraphicsPrograms PipelineCache::GetGraphicsPrograms(
}
ShaderParams pixel_params;
if (pixel_active) {
pixel_params = PrepareProgram(pixel_regs, sh, target_export_mapping, pixel_info);
const auto& blend = context.GetBlendControl(0);
const auto is_dual_source = [](uint8_t factor) {
return factor >= static_cast<uint8_t>(Prospero::BlendFactor::kSrc1Color) &&
factor <= static_cast<uint8_t>(Prospero::BlendFactor::kOneMinusSrc1Alpha);
};
pixel_params = PrepareProgram(pixel_regs, sh, target_export_mapping, pixel_info);
const auto& blend = context.GetBlendControl(0);
pixel_info.dual_source_blending =
blend.enable && !context.GetRenderTarget(0).info.blend_bypass &&
(is_dual_source(blend.color_srcblend) || is_dual_source(blend.color_destblend) ||
(blend.separate_alpha_blend &&
(is_dual_source(blend.alpha_srcblend) || is_dual_source(blend.alpha_destblend))));
(BlendFactorIsDualSource(blend.color_srcblend) ||
BlendFactorIsDualSource(blend.color_destblend) ||
(blend.separate_alpha_blend && (BlendFactorIsDualSource(blend.alpha_srcblend) ||
BlendFactorIsDualSource(blend.alpha_destblend))));
if (pixel_info.dual_source_blending) {
// MRT1 supplies a second blend source for the same render target as MRT0.
// MRT1 supplies the second blend source for target 0.
pixel_info.target_output_mode[1] = pixel_info.target_output_mode[0];
pixel_info.target_export_mapping[1] = pixel_info.target_export_mapping[0];
} else if (blend.enable && !context.GetRenderTarget(0).info.blend_bypass &&
pixel_info.target_output_mode[0] != 0 && pixel_info.target_output_mode[0] != 7 &&
std::all_of(std::begin(pixel_info.target_output_mode) + 1,
std::end(pixel_info.target_output_mode),
[](uint8_t mode) { return mode == 0; }) &&
ClassifyBlendMapping(blend, pixel_info.target_export_mapping[0]) ==
BlendMappingSupport::SourceAlpha) {
// Preserve logical alpha when the export mapping moves it.
pixel_info.alpha_blend_source_remap = true;
pixel_info.dual_source_blending = true;
pixel_info.target_output_mode[1] = pixel_info.target_output_mode[0];
pixel_info.target_export_mapping[1] = {};
}
}
if (context.GetClipControl().clip_disable) {
@@ -623,7 +641,6 @@ PipelineCache::GraphicsPrograms PipelineCache::GetGraphicsPrograms(
static_cast<float>(std::min(limits.maxViewportDimensions[1], 16384u)) * 0.5f;
clip.enabled = true;
}
Common::LockGuard lock(m_mutex);
uint32_t push_data_cursor =
mesh_active ? ShaderRecompiler::IR::PushData::MeshDrawDwordCount : 0;
GraphicsPrograms result;
@@ -641,7 +658,19 @@ ShaderProgram PipelineCache::GetComputeProgram(const HW::ComputeShaderInfo& regs
ShaderComputeInputInfo& input_info) {
input_info.host_subgroup_size = m_graphics.SupportsComputeWave64() ? 64u : 32u;
const auto params = PrepareProgram(regs, sh, input_info);
Common::LockGuard lock(m_mutex);
// Use one effective size for the cache key, LDS declaration, and access bounds.
const auto max_lds_dwords =
m_graphics.GetPhysicalDeviceProperties().limits.maxComputeSharedMemorySize / 4u;
if (input_info.lds_size_dwords > max_lds_dwords) {
static std::atomic_bool warned = false;
if (!warned.exchange(true, std::memory_order_relaxed)) {
PipelineCacheLog("GPU warning: game compute shader requests {} bytes of LDS, but "
"the Vulkan device limit is {} bytes. Clamping LDS; rendering may "
"be incorrect.",
input_info.lds_size_dwords * 4u, max_lds_dwords * 4u);
}
}
input_info.lds_size_dwords = std::min(input_info.lds_size_dwords, max_lds_dwords);
uint32_t push_data_cursor = 0;
return m_program_cache->Get(params, input_info, push_data_cursor);
}
@@ -666,7 +695,6 @@ PipelineCache::Pipeline& PipelineCache::GetGraphicsPipeline(
EXIT_IF(ps_active && !pixel_program);
const auto color_count = static_cast<uint32_t>(colors.size());
Common::LockGuard lock(m_mutex);
auto& ctx = command.GetRegisters();
const HW::ModeControl& mc = ctx.GetModeControl();
@@ -699,14 +727,35 @@ PipelineCache::Pipeline& PipelineCache::GetGraphicsPipeline(
}
const auto& rt = ctx.GetRenderTarget(colors[i].target_slot);
const auto& bc = ctx.GetBlendControl(colors[i].target_slot);
static_params.color_srcblend[slot] = bc.color_srcblend;
static_params.color_comb_fcn[slot] = bc.color_comb_fcn;
static_params.color_destblend[slot] = bc.color_destblend;
static_params.alpha_srcblend[slot] = bc.alpha_srcblend;
static_params.alpha_comb_fcn[slot] = bc.alpha_comb_fcn;
static_params.alpha_destblend[slot] = bc.alpha_destblend;
static_params.separate_alpha_blend[slot] = bc.separate_alpha_blend;
static_params.blend_enable[slot] = bc.enable && !rt.info.blend_bypass;
const bool alpha_remap =
slot == 0 && ps_input_info != nullptr && ps_input_info->alpha_blend_source_remap;
static_params.blend_enable[slot] = bc.enable && !rt.info.blend_bypass;
if (static_params.blend_enable[slot] && !alpha_remap &&
ClassifyBlendMapping(bc, colors[i].export_mapping) != BlendMappingSupport::Direct) {
static_params.blend_enable[slot] = false;
static std::atomic_bool warned = false;
if (!warned.exchange(true, std::memory_order_relaxed)) {
Log::WriteToConsoleAndLog(fmt::format(
"Warning: blending disabled for unsupported color mapping "
"(slot={} mapping=0x{:02x} color={}/{} alpha={}/{} separate={}).\n",
slot, colors[i].export_mapping.packed, bc.color_srcblend, bc.color_destblend,
bc.alpha_srcblend, bc.alpha_destblend, bc.separate_alpha_blend ? 1 : 0));
}
}
if (alpha_remap) {
static_params.blend_alpha_source_remap = true;
}
if (static_params.blend_enable[slot]) {
static_params.color_srcblend[slot] = bc.color_srcblend;
static_params.color_comb_fcn[slot] = bc.color_comb_fcn;
static_params.color_destblend[slot] = bc.color_destblend;
static_params.separate_alpha_blend[slot] = bc.separate_alpha_blend;
if (bc.separate_alpha_blend) {
static_params.alpha_srcblend[slot] = bc.alpha_srcblend;
static_params.alpha_comb_fcn[slot] = bc.alpha_comb_fcn;
static_params.alpha_destblend[slot] = bc.alpha_destblend;
}
}
}
const bool with_depth =
depth.desc.view_info.format != vk::Format::eUndefined && static_cast<bool>(depth.image_id);
@@ -770,24 +819,20 @@ PipelineCache::Pipeline& PipelineCache::GetGraphicsPipeline(
vs_input_info.resources_num > ShaderVertexInputInfo::RES_MAX);
key.vertex_input.binding_count = static_cast<uint8_t>(vs_input_info.buffers_num);
key.vertex_input.attribute_count = static_cast<uint8_t>(vs_input_info.resources_num);
uint32_t attributes_num = 0;
for (int binding = 0; binding < vs_input_info.buffers_num; binding++) {
const auto& buffer = vs_input_info.buffers[binding];
EXIT_IF(buffer.attr_num < 0 || buffer.attr_num > ShaderVertexInputBuffer::ATTR_MAX);
attributes_num += static_cast<uint32_t>(buffer.attr_num);
EXIT_IF(attributes_num > static_cast<uint32_t>(vs_input_info.resources_num));
key.vertex_input.bindings[binding] = {.stride = buffer.stride,
.instance = buffer.fetch_index != 0};
for (int attribute = 0; attribute < buffer.attr_num; attribute++) {
const auto index = buffer.attr_indices[attribute];
EXIT_IF(index < 0 || index >= vs_input_info.resources_num);
key.vertex_input.attributes[index] = {
.offset = buffer.attr_offsets[attribute],
.binding = static_cast<uint8_t>(binding),
};
}
}
EXIT_IF(attributes_num != static_cast<uint32_t>(vs_input_info.resources_num));
for (int attribute = 0; attribute < vs_input_info.resources_num; attribute++) {
const auto binding = vs_input_info.resources_dst[attribute].buffer_index;
EXIT_IF(binding < 0 || binding >= vs_input_info.buffers_num);
key.vertex_input.attributes[attribute] = {
.offset = static_cast<uint32_t>(vs_input_info.resources[attribute].Base48() -
vs_input_info.buffers[binding].addr),
.binding = static_cast<uint8_t>(binding),
};
}
}
if (auto iter = m_graphics_pipelines.find(key); iter != m_graphics_pipelines.end()) {
@@ -826,8 +871,6 @@ PipelineCache::GetComputePipeline(const ShaderComputeInputInfo& input_info,
EXIT_IF(!compute_program);
Common::LockGuard lock(m_mutex);
if (auto iter = m_compute_pipelines.find(compute_program.id);
iter != m_compute_pipelines.end()) {
return *iter->second;
@@ -4,7 +4,6 @@
#include "common/abi.h"
#include "common/assert.h"
#include "common/common.h"
#include "common/threads.h"
#include "graphics/host_gpu/renderer/renderTarget.h"
#include "graphics/host_gpu/vulkanCommon.h"
#include "graphics/shader/shader.h"
@@ -56,6 +55,7 @@ struct PipelineStaticParameters {
uint8_t alpha_destblend[RENDER_COLOR_ATTACHMENTS_MAX] = {};
bool separate_alpha_blend[RENDER_COLOR_ATTACHMENTS_MAX] = {};
bool blend_enable[RENDER_COLOR_ATTACHMENTS_MAX] = {};
bool blend_alpha_source_remap = false;
bool operator==(const PipelineStaticParameters& other) const noexcept;
};
@@ -65,7 +65,7 @@ struct PipelineStaticParameters {
static_assert(std::is_trivially_copyable_v<PipelineStaticParameters>);
static_assert(std::is_standard_layout_v<PipelineStaticParameters>);
static_assert(alignof(PipelineStaticParameters) == 1);
static_assert(sizeof(PipelineStaticParameters) == 125);
static_assert(sizeof(PipelineStaticParameters) == 126);
struct PipelineRenderingState {
std::array<vk::Format, RENDER_COLOR_ATTACHMENTS_MAX> color_formats {};
@@ -103,6 +103,7 @@ struct ShaderProgram {
explicit operator bool() const { return id != 0 && module != nullptr; }
};
// The owning renderer serializes access, including saves while the GPU is running.
class PipelineCache {
public:
explicit PipelineCache(GraphicContext& graphics);
@@ -166,45 +167,10 @@ private:
hash ^= value + static_cast<std::size_t>(0x9e3779b97f4a7c15ull) + (hash << 6u) +
(hash >> 2u);
}
static void MixStaticParams(std::size_t& hash, const PipelineStaticParameters& params) {
const auto* bytes = reinterpret_cast<const uint8_t*>(&params);
for (std::size_t i = 0; i < sizeof(params); i++) {
Mix(hash, bytes[i]);
}
}
static void MixRendering(std::size_t& hash, const PipelineRenderingState& rendering) {
Mix(hash, rendering.color_count);
for (uint32_t i = 0; i < rendering.color_count; i++) {
Mix(hash, static_cast<uint32_t>(rendering.color_formats[i]));
}
Mix(hash, static_cast<uint32_t>(rendering.depth_format));
Mix(hash, static_cast<uint32_t>(rendering.stencil_format));
}
};
struct GraphicsPipelineKeyHash {
std::size_t operator()(const GraphicsPipelineKey& key) const {
std::size_t hash = 0;
PipelineKeyHash::MixRendering(hash, key.rendering);
for (const auto id: key.vertex_shader_ids) {
PipelineKeyHash::Mix(hash, id);
}
PipelineKeyHash::Mix(hash, key.ps_shader_id);
PipelineKeyHash::Mix(hash, key.vertex_input.binding_count);
for (uint32_t i = 0; i < key.vertex_input.binding_count; i++) {
PipelineKeyHash::Mix(hash, key.vertex_input.bindings[i].stride);
PipelineKeyHash::Mix(hash, key.vertex_input.bindings[i].instance);
}
PipelineKeyHash::Mix(hash, key.vertex_input.attribute_count);
for (uint32_t i = 0; i < key.vertex_input.attribute_count; i++) {
PipelineKeyHash::Mix(hash, key.vertex_input.attributes[i].offset);
PipelineKeyHash::Mix(hash, key.vertex_input.attributes[i].binding);
}
PipelineKeyHash::MixStaticParams(hash, key.static_params);
return hash;
}
std::size_t operator()(const GraphicsPipelineKey& key) const;
};
GraphicContext& m_graphics;
@@ -214,7 +180,6 @@ private:
std::unordered_map<GraphicsPipelineKey, std::unique_ptr<Pipeline>, GraphicsPipelineKeyHash>
m_graphics_pipelines;
std::unordered_map<uint64_t, std::unique_ptr<Pipeline>> m_compute_pipelines;
Common::Mutex m_mutex;
void InitializeDriverCache();
};
@@ -90,7 +90,7 @@ vk::BufferMemoryBarrier MakeGdsDependency(vk::Buffer buffer) {
bool HasShaderBufferWrites(const ShaderStageRuntime& runtime) {
EXIT_IF(!runtime);
const auto& program = *runtime.program;
const auto& resources = runtime.resources;
const auto& resources = *runtime.resources;
EXIT_IF(resources.buffers.size() != program.info.buffers.size());
bool has_writes = false;
for (uint32_t i = 0; i < program.info.buffers.size(); i++) {
@@ -137,14 +137,17 @@ static void GetInputFormat(const ShaderBufferResource& res, vk::Format& format,
}
}
static vk::BlendFactor GetBlendFactor(uint32_t factor) {
static vk::BlendFactor GetBlendFactor(uint32_t factor, bool remap_source_alpha) {
switch (static_cast<Prospero::BlendFactor>(factor)) {
case Prospero::BlendFactor::kZero: return vk::BlendFactor::eZero;
case Prospero::BlendFactor::kOne: return vk::BlendFactor::eOne;
case Prospero::BlendFactor::kSrcColor: return vk::BlendFactor::eSrcColor;
case Prospero::BlendFactor::kOneMinusSrcColor: return vk::BlendFactor::eOneMinusSrcColor;
case Prospero::BlendFactor::kSrcAlpha: return vk::BlendFactor::eSrcAlpha;
case Prospero::BlendFactor::kOneMinusSrcAlpha: return vk::BlendFactor::eOneMinusSrcAlpha;
case Prospero::BlendFactor::kSrcAlpha:
return remap_source_alpha ? vk::BlendFactor::eSrc1Color : vk::BlendFactor::eSrcAlpha;
case Prospero::BlendFactor::kOneMinusSrcAlpha:
return remap_source_alpha ? vk::BlendFactor::eOneMinusSrc1Color
: vk::BlendFactor::eOneMinusSrcAlpha;
case Prospero::BlendFactor::kDstAlpha: return vk::BlendFactor::eDstAlpha;
case Prospero::BlendFactor::kOneMinusDstAlpha: return vk::BlendFactor::eOneMinusDstAlpha;
case Prospero::BlendFactor::kDstColor: return vk::BlendFactor::eDstColor;
@@ -327,9 +330,6 @@ void CreatePipelineInternal(GraphicContext& graphics, PipelineCache::Pipeline& p
}
}
EXIT_NOT_IMPLEMENTED(vs_input_info.resources[index].AddTid());
EXIT_NOT_IMPLEMENTED(vs_input_info.resources[index].SwizzleEnabled());
EXIT_IF(registers_num < 1 || registers_num > 4);
}
@@ -368,6 +368,8 @@ void CreatePipelineInternal(GraphicContext& graphics, PipelineCache::Pipeline& p
// MoltenVK lacks VK_EXT_depth_clip_enable; omit the depth-clip struct on macOS and accept
// Vulkan's default depth clipping (enabled) instead of the PS5's clamp behavior.
#if !defined(__APPLE__)
// The DB clamps depth to the viewport range after polygon offset is applied.
rasterizer.depthClampEnable = VK_TRUE;
rasterizer.pNext = &clip_ext;
#endif
vk::PipelineRasterizationProvokingVertexStateCreateInfoEXT provoking_vertex {};
@@ -395,17 +397,20 @@ void CreatePipelineInternal(GraphicContext& graphics, PipelineCache::Pipeline& p
color_blend_attachment[i].colorWriteMask =
vk::ColorComponentFlags {static_params.color_mask[i]};
color_blend_attachment[i].blendEnable = static_params.blend_enable[i] ? VK_TRUE : VK_FALSE;
// Only target 0 can use the second blend source carrying logical alpha.
const bool remap_source_alpha = i == 0 && static_params.blend_alpha_source_remap;
color_blend_attachment[i].srcColorBlendFactor =
GetBlendFactor(static_params.color_srcblend[i]);
GetBlendFactor(static_params.color_srcblend[i], remap_source_alpha);
color_blend_attachment[i].dstColorBlendFactor =
GetBlendFactor(static_params.color_destblend[i]);
GetBlendFactor(static_params.color_destblend[i], remap_source_alpha);
color_blend_attachment[i].colorBlendOp = GetBlendOp(static_params.color_comb_fcn[i]);
color_blend_attachment[i].srcAlphaBlendFactor =
(static_params.separate_alpha_blend[i] ? GetBlendFactor(static_params.alpha_srcblend[i])
: color_blend_attachment[i].srcColorBlendFactor);
(static_params.separate_alpha_blend[i]
? GetBlendFactor(static_params.alpha_srcblend[i], remap_source_alpha)
: color_blend_attachment[i].srcColorBlendFactor);
color_blend_attachment[i].dstAlphaBlendFactor =
(static_params.separate_alpha_blend[i]
? GetBlendFactor(static_params.alpha_destblend[i])
? GetBlendFactor(static_params.alpha_destblend[i], remap_source_alpha)
: color_blend_attachment[i].dstColorBlendFactor);
color_blend_attachment[i].alphaBlendOp =
(static_params.separate_alpha_blend[i] ? GetBlendOp(static_params.alpha_comb_fcn[i])
+8 -2
View File
@@ -45,6 +45,7 @@ enum class CommandBufferDebugOp : uint32_t {
EopFlip,
EopWriteBackFlip,
EopOnlyFlip,
DispatchIndirect,
Unknown,
};
@@ -156,8 +157,10 @@ public:
void DispatchDirect(uint64_t submit_id, CommandBuffer& buffer, uint32_t thread_group_x,
uint32_t thread_group_y, uint32_t thread_group_z, uint32_t mode);
void DispatchIndirect(uint64_t submit_id, CommandBuffer& buffer, uint64_t args_addr,
uint32_t mode);
[[nodiscard]] PreparedBindings PrepareBindings(const ShaderStageRuntime& runtime);
void PrepareBindings(const ShaderStageRuntime& runtime, PreparedBindings& prepared);
void FindBuffers(PreparedBindings& bindings);
void RebindBuffers(PreparedBindings& bindings);
void RebindImages(PreparedBindings& bindings);
@@ -193,7 +196,8 @@ private:
bool primitive_restart_enable);
[[nodiscard]] RenderState AcquireRenderTargets(CommandBuffer& buffer, RenderColorInfo* colors,
uint32_t color_count, RenderDepthInfo& depth,
const std::optional<PreparedBindings>& pixel = std::nullopt);
vk::ImageAspectFlags& feedback_aspects,
std::span<PreparedBindings* const> stages = {});
[[nodiscard]] bool ResolveColorTargets(CommandBuffer& buffer,
uint32_t render_target_slice_offset);
void BindImage(ImageId id, bool storage);
@@ -206,6 +210,8 @@ private:
uint32_t group_y, uint32_t group_z, uint32_t mode);
RenderContext& m_context;
GraphicsBindings m_graphics_bindings;
PreparedBindings m_compute_bindings;
std::vector<ImageId> m_bound_images;
std::vector<vk::DescriptorBufferInfo> m_descriptor_buffers;
std::vector<vk::DescriptorImageInfo> m_descriptor_images;
@@ -9,6 +9,7 @@
#include "graphics/guest_gpu/gpu_defs.h"
#include "graphics/guest_gpu/graphicsRun.h"
#include "graphics/guest_gpu/hardwareContext.h"
#include "graphics/guest_gpu/pm4.h"
#include "graphics/host_gpu/graphicContext.h"
#include "graphics/host_gpu/renderer/image/imageInfo.h"
#include "graphics/host_gpu/renderer/pipeline/descriptors.h"
@@ -51,7 +52,7 @@ static bool FillSourcesDisjoint(std::span<const ShaderRecompiler::IR::Descriptor
bool RenderExecutor::TryConsumeComputeMetaClear(const ShaderComputeInputInfo& input,
const CommandBuffer& buffer) {
const auto& program = *input.stage.program;
const auto& resources = input.stage.resources;
const auto& resources = *input.stage.resources;
if (resources.buffers.size() != program.info.buffers.size()) {
EXIT("compute runtime buffer count does not match shader metadata\n");
}
@@ -85,7 +86,7 @@ bool ResolveComputeBufferFill(const ShaderComputeInputInfo& input, uint32_t grou
uint32_t group_y, uint32_t group_z, uint32_t mode,
ShaderBufferResource& resolved_descriptor, uint32_t& resolved_clear,
uint64_t& resolved_size) {
const auto& resources = input.stage.resources;
const auto& resources = *input.stage.resources;
const auto& fill = resources.uniform_fill;
if (fill.kind != ShaderRecompiler::IR::UniformFillKind::Buffer) {
return false;
@@ -128,7 +129,7 @@ bool RenderExecutor::TryConsumeComputeImageClear(const ShaderComputeInputInfo& i
CommandBuffer& command, uint32_t group_x,
uint32_t group_y, uint32_t group_z, uint32_t mode) {
const auto& program = *input.stage.program;
const auto& resources = input.stage.resources;
const auto& resources = *input.stage.resources;
const auto& fill = resources.uniform_fill;
auto& cache = command.GetContext().GetTextureCache();
if (fill.kind == ShaderRecompiler::IR::UniformFillKind::Image) {
@@ -262,16 +263,15 @@ void RenderExecutor::DispatchDirect(uint64_t submit_id, CommandBuffer& buffer,
}
const auto& program = *input_info.stage.program;
const auto& resources = input_info.stage.resources;
if (TryConsumeComputeMetaClear(input_info, buffer)) {
ResetBindings();
return;
}
if (TryConsumeComputeImageClear(input_info, buffer, thread_group_x, thread_group_y,
thread_group_z, mode)) {
const auto& resources = *input_info.stage.resources;
if (resources.specialization_reads.empty() &&
(TryConsumeComputeMetaClear(input_info, buffer) ||
TryConsumeComputeImageClear(input_info, buffer, thread_group_x, thread_group_y,
thread_group_z, mode))) {
ResetBindings();
return;
}
const bool large_workgroup =
(input_info.threads_num[0] * input_info.threads_num[1] * input_info.threads_num[2] >= 512);
const bool has_sampler = !program.info.samplers.empty();
@@ -321,7 +321,8 @@ void RenderExecutor::DispatchDirect(uint64_t submit_id, CommandBuffer& buffer,
static_cast<uint32_t>(r.TileMode()));
}
for (uint32_t i = 0; i < program.info.samplers.size(); i++) {
const auto r = DecodeNativeDescriptor<ShaderSamplerResource>(resources.samplers[i]);
const auto r = DecodeNativeDescriptor<ShaderSamplerResource>(
resources.samplers[program.info.samplers[i].snapshot_index]);
LOGF(" CS sampler[%u]: source=%u clamp=%u/%u/%u filter=%u/%u/%u mip=%u "
"lod=%u-%u bias=%d\n",
i, program.info.samplers[i].source, static_cast<uint32_t>(r.ClampX()),
@@ -361,7 +362,8 @@ void RenderExecutor::DispatchDirect(uint64_t submit_id, CommandBuffer& buffer,
buffer.EndRendering();
auto& pipeline =
m_context.GetPipelineCache().GetComputePipeline(input_info, compute_program);
auto bindings = PrepareBindings(input_info.stage);
auto& bindings = m_compute_bindings;
PrepareBindings(input_info.stage, bindings);
FindBuffers(bindings);
if (program.info.uses_dma) {
m_context.PrepareBda();
@@ -395,4 +397,61 @@ void RenderExecutor::DispatchDirect(uint64_t submit_id, CommandBuffer& buffer,
ResetBindings();
}
void RenderExecutor::DispatchIndirect(uint64_t submit_id, CommandBuffer& buffer,
uint64_t args_addr, uint32_t mode) {
EXIT_IF(buffer.IsInvalid() || args_addr == 0 || (args_addr & 3u) != 0 ||
(mode & Pm4::COMPUTE_DISPATCH_INITIATOR_USE_THREAD_DIMENSIONS) != 0);
m_context.GetCommandScheduler().PopPendingOperations();
buffer.SetDebugInfo(static_cast<uint32_t>(CommandBufferDebugOp::DispatchIndirect), submit_id,
static_cast<uint32_t>(args_addr), static_cast<uint32_t>(args_addr >> 32u),
0, mode, buffer.GetShaders().GetCs().cs_regs.data_addr);
Common::LockGuard lock(m_context.GetMutex());
const auto& cs_regs = buffer.GetShaders().GetCs();
if (cs_regs.cs_regs.data_addr == 0) {
return;
}
ShaderComputeInputInfo input_info {};
const auto compute_program = m_context.GetPipelineCache().GetComputeProgram(
cs_regs, buffer.GetRegisters().GetShaderRegisters(), input_info);
buffer.EndRendering();
auto& pipeline = m_context.GetPipelineCache().GetComputePipeline(input_info, compute_program);
auto& bindings = m_compute_bindings;
PrepareBindings(input_info.stage, bindings);
FindBuffers(bindings);
const auto& program = *input_info.stage.program;
if (program.info.uses_dma) {
m_context.PrepareBda();
}
RebindImages(bindings);
// Acquiring arguments can merge cache buffers; finalize shader bindings afterward.
const auto [args_buffer, args_offset] = m_context.GetBufferCache().ObtainBuffer(
args_addr, sizeof(vk::DispatchIndirectCommand), false);
EXIT_IF(args_buffer == nullptr || (args_offset & 3u) != 0);
RebindBuffers(bindings);
PreparedBindings* descriptor_stage = &bindings;
CommitBindings(buffer, vk::PipelineBindPoint::eCompute, pipeline,
std::span {&descriptor_stage, 1u});
const auto vk_buffer = buffer.Handle();
const bool has_storage_writes = HasShaderBufferWrites(input_info.stage) ||
std::any_of(program.info.images.begin(), program.info.images.end(), [](const auto& image) {
return image.written && image.resource_class ==
ShaderRecompiler::IR::ImageResourceClass::Storage;
});
if (has_storage_writes) {
ShaderWriteHazardBarrier(vk_buffer, vk::PipelineStageFlagBits::eComputeShader);
}
vk::MemoryBarrier barrier {};
barrier.srcAccessMask = vk::AccessFlagBits::eShaderWrite | vk::AccessFlagBits::eTransferWrite;
barrier.dstAccessMask = vk::AccessFlagBits::eIndirectCommandRead;
vk_buffer.pipelineBarrier(vk::PipelineStageFlagBits::eAllGraphics |
vk::PipelineStageFlagBits::eComputeShader |
vk::PipelineStageFlagBits::eTransfer,
vk::PipelineStageFlagBits::eDrawIndirect, {},
1, &barrier, 0, nullptr, 0, nullptr);
vk_buffer.bindPipeline(vk::PipelineBindPoint::eCompute, pipeline.pipeline);
vk_buffer.dispatchIndirect(args_buffer->Handle(), args_offset);
ShaderAccessBarrier(vk_buffer, vk::PipelineStageFlagBits::eComputeShader);
ResetBindings();
}
} // namespace Libs::Graphics
@@ -119,6 +119,10 @@ void RenderContext::UnmapMemory(uint64_t vaddr, uint64_t size) {
}
void RenderContext::PrepareBda() {
if (!m_bda_logged) {
Log::WriteToConsoleAndLog("GPU: using buffer device address (BDA) shader memory access.\n");
m_bda_logged = true;
}
std::shared_lock lock(m_mapped_ranges_mutex);
m_mapped_ranges.ForEach([this](uint64_t start, uint64_t end) {
m_buffer_cache.SynchronizeBuffersInRange(start, end - start);
@@ -81,6 +81,7 @@ private:
std::unique_ptr<GuestGpu> m_gpu;
VideoOut::VideoOutDriver* m_video_out = nullptr;
bool m_fault_process_pending = false;
bool m_bda_logged = false;
Common::Mutex m_interrupt_mutex;
std::vector<InterruptEqRegistration> m_interrupt_eqs;
+85 -111
View File
@@ -58,7 +58,7 @@ std::pair<int32_t, uint32_t> ResolveDrawOffsets(uint32_t index_offset,
EXIT_IF(!vs_input_info.stage);
const auto& program = *vs_input_info.stage.program;
const auto& resources = vs_input_info.stage.resources;
const auto& resources = *vs_input_info.stage.resources;
if (index_offset == 0 &&
program.info.vertex_offset_sgpr >= static_cast<int32_t>(program.user_data_base)) {
const auto index =
@@ -249,8 +249,8 @@ static void LogDrawInputState(const CommandBuffer& buffer, const RenderColorInfo
for (int bi = 0; bi < vs_input_info.buffers_num; bi++) {
const auto& b = vs_input_info.buffers[bi];
LOGF("DrawInputState[%u]: vb[%d] addr=0x%010" PRIx64
" stride=%u records=%u fetch_index=%u attr_num=%d\n",
log_id, bi, b.addr, b.stride, b.num_records, b.fetch_index, b.attr_num);
" stride=%u records=%u fetch_index=%u\n",
log_id, bi, b.addr, b.stride, b.num_records, b.fetch_index);
const auto* bytes = reinterpret_cast<const uint8_t*>(b.addr);
if (bytes != nullptr && b.stride != 0) {
@@ -272,11 +272,13 @@ static void LogDrawInputState(const CommandBuffer& buffer, const RenderColorInfo
raw[5], raw[6], raw[7], raw[8], flt[0], flt[1], flt[2], flt[3], flt[4], flt[5],
flt[6], flt[7], flt[8]);
for (int ai = 0; ai < b.attr_num; ai++) {
const auto res_index = b.attr_indices[ai];
const auto& r = vs_input_info.resources[res_index];
const auto& rd = vs_input_info.resources_dst[res_index];
const auto offset = b.attr_offsets[ai];
for (int ai = 0; ai < vs_input_info.resources_num; ai++) {
const auto& r = vs_input_info.resources[ai];
const auto& rd = vs_input_info.resources_dst[ai];
if (rd.buffer_index != bi) {
continue;
}
const auto offset = static_cast<uint32_t>(r.Base48() - b.addr);
if (offset + 4u <= b.stride &&
r.Format() == Prospero::BufferFormat::k8_8_8_8UNorm) {
uint32_t packed = 0;
@@ -296,14 +298,16 @@ static void LogDrawInputState(const CommandBuffer& buffer, const RenderColorInfo
}
}
for (int ai = 0; ai < b.attr_num; ai++) {
const auto res_index = b.attr_indices[ai];
const auto& r = vs_input_info.resources[res_index];
const auto& rd = vs_input_info.resources_dst[res_index];
LOGF("DrawInputState[%u]: attr[%d] res=%d offset=%u dst=v%d regs=%d fetch_index=%u "
for (int ai = 0; ai < vs_input_info.resources_num; ai++) {
const auto& r = vs_input_info.resources[ai];
const auto& rd = vs_input_info.resources_dst[ai];
if (rd.buffer_index != bi) {
continue;
}
LOGF("DrawInputState[%u]: attr[%d] offset=%u dst=v%d regs=%d fetch_index=%u "
"sharp=%08" PRIx32 " %08" PRIx32 " %08" PRIx32 " %08" PRIx32 "\n",
log_id, ai, res_index, b.attr_offsets[ai], rd.register_start, rd.registers_num,
rd.fetch_index, r.fields[0], r.fields[1], r.fields[2], r.fields[3]);
log_id, ai, static_cast<uint32_t>(r.Base48() - b.addr), rd.register_start,
rd.registers_num, rd.fetch_index, r.fields[0], r.fields[1], r.fields[2], r.fields[3]);
}
}
}
@@ -453,15 +457,16 @@ struct DrawCallInfo {
RenderState RenderExecutor::AcquireRenderTargets(CommandBuffer& buffer, RenderColorInfo* colors,
uint32_t color_count, RenderDepthInfo& depth,
const std::optional<PreparedBindings>& pixel) {
vk::ImageAspectFlags& feedback_aspects,
std::span<PreparedBindings* const> stages) {
EXIT_IF(colors == nullptr || color_count > RENDER_COLOR_ATTACHMENTS_MAX);
feedback_aspects = {};
auto& cache = m_context.GetTextureCache();
RenderState state {};
state.width = std::numeric_limits<uint32_t>::max();
state.height = std::numeric_limits<uint32_t>::max();
state.num_layers = std::numeric_limits<uint32_t>::max();
state.num_color_attachments = 0;
uint32_t attachment_samples = 0;
for (uint32_t i = 0; i < color_count; i++) {
auto& target = colors[i];
EXIT_IF(!target.image_id);
@@ -472,22 +477,7 @@ RenderState RenderExecutor::AcquireRenderTargets(CommandBuffer& buffer, RenderCo
}
const auto image_view = cache.FindRenderTarget(target.image_id, target.desc);
auto& image = cache.GetImage(target.image_id);
SetVulkanObjectNameF(m_context.GetGraphics().device, image.backing.image,
"Kyty.MRT{}.Image[guest=0x{:016x} size=0x{:x} format={}]",
target.target_slot, image.info.data.address, image.info.data.size,
static_cast<uint32_t>(image.info.pixel_format));
SetVulkanObjectNameF(m_context.GetGraphics().device, image_view,
"Kyty.MRT{}.View[guest=0x{:016x} mip={} layer={}+{}]",
target.target_slot, image.info.data.address,
target.desc.view_info.base_level, target.desc.view_info.base_layer,
target.desc.view_info.layer_count);
EXIT_IF(image.backing.samples != target.desc.info.samples || image_view == nullptr);
if (attachment_samples == 0) {
attachment_samples = target.desc.info.samples;
} else if (attachment_samples != target.desc.info.samples) {
EXIT("mixed color attachment sample counts are unsupported: %u and %u\n",
attachment_samples, target.desc.info.samples);
}
const auto& view = target.desc.view_info;
const auto layout = image.binding.is_bound ? vk::ImageLayout::eGeneral
: vk::ImageLayout::eColorAttachmentOptimal;
@@ -527,43 +517,30 @@ RenderState RenderExecutor::AcquireRenderTargets(CommandBuffer& buffer, RenderCo
EXIT("failed to consume HTile clear state\n");
}
auto& image = cache.GetImage(depth.image_id);
SetVulkanObjectNameF(m_context.GetGraphics().device, image.backing.image,
"Kyty.DepthTarget.Image[guest=0x{:016x} size=0x{:x} format={}]",
image.info.data.address, image.info.data.size,
static_cast<uint32_t>(image.info.pixel_format));
SetVulkanObjectNameF(m_context.GetGraphics().device, image_view,
"Kyty.DepthTarget.View[guest=0x{:016x} layer={}+{}]",
image.info.data.address, depth.desc.view_info.base_layer,
depth.desc.view_info.layer_count);
EXIT_IF(image_view == nullptr || image.backing.samples != depth.desc.info.samples);
if (attachment_samples == 0) {
attachment_samples = depth.desc.info.samples;
} else if (attachment_samples != depth.desc.info.samples) {
EXIT("mixed color/depth sample counts are unsupported: %u and %u\n", attachment_samples,
depth.desc.info.samples);
const auto draw_writes = depth.AttachmentWriteAspects();
vk::ImageAspectFlags sampled_aspects;
for (const auto* stage: stages) {
for (const auto& binding: stage->images) {
if (binding.image_id != depth.image_id ||
binding.desc.type != TextureCache::BindingType::Texture) continue;
const auto native =
std::ranges::find(image.views, binding.image_view, &CachedImageView::view);
EXIT_IF(native == image.views.end());
sampled_aspects |= native->info.aspect;
feedback_aspects |= DepthFeedbackAspects(draw_writes, depth.desc.view_info,
native->info);
}
}
const bool feedback = depth.depth_write_enable && pixel &&
std::ranges::any_of(pixel->images, [&](const TextureBinding& binding) {
if (binding.image_id != depth.image_id ||
binding.desc.type != TextureCache::BindingType::Texture) {
return false;
}
const auto native =
std::ranges::find(image.views, binding.image_view, &CachedImageView::view);
EXIT_IF(native == image.views.end());
const auto& sampled = native->info;
const auto& target = depth.desc.view_info;
return (sampled.aspect & vk::ImageAspectFlagBits::eDepth) &&
ImageRangeOverlaps(sampled.base_level, sampled.level_count,
target.base_level, target.level_count) &&
ImageRangeOverlaps(sampled.base_layer, sampled.layer_count,
target.base_layer, target.layer_count);
});
if (feedback && !m_context.GetGraphics().attachment_feedback_loop_enabled) {
if (feedback_aspects && !m_context.GetGraphics().attachment_feedback_loop_enabled) {
EXIT("depth attachment feedback loop is not supported by the host\n");
}
const auto layout = feedback ? vk::ImageLayout::eAttachmentFeedbackLoopOptimalEXT
: depth_attachment_layout(depth);
auto layout = depth_attachment_layout(depth);
if (sampled_aspects & ~DepthReadableAspects(layout)) {
layout = m_context.GetGraphics().attachment_feedback_loop_enabled
? vk::ImageLayout::eAttachmentFeedbackLoopOptimalEXT
: vk::ImageLayout::eGeneral;
}
// The attachment store writes even when guest depth/stencil tests do not.
const auto access = vk::AccessFlagBits2::eDepthStencilAttachmentRead |
vk::AccessFlagBits2::eDepthStencilAttachmentWrite;
@@ -592,9 +569,6 @@ RenderState RenderExecutor::AcquireRenderTargets(CommandBuffer& buffer, RenderCo
const auto& limits = buffer.GetGraphics().GetPhysicalDeviceProperties().limits;
state.width = limits.maxFramebufferWidth;
state.height = limits.maxFramebufferHeight;
} else if (attachment_samples == 0 ||
vulkan_sample_count(attachment_samples) == vk::SampleCountFlagBits {}) {
EXIT("render state has no valid attachments\n");
}
if (state.num_layers == std::numeric_limits<uint32_t>::max()) {
state.num_layers = 1;
@@ -605,12 +579,17 @@ RenderState RenderExecutor::AcquireRenderTargets(CommandBuffer& buffer, RenderCo
return state;
}
static bool DrawHasActivePixelShader(const CommandBuffer& buffer) {
const auto& ctx = buffer.GetRegisters();
const auto& sh_regs = ctx.GetShaderRegisters();
const bool has_color_output = (ctx.GetRenderTargetMask() & sh_regs.m_cbShaderMask) != 0;
return buffer.GetShaders().GetPs().ps_regs.data_addr != 0 &&
(has_color_output || PixelShaderHasDepthOrCoverageSideEffects(sh_regs));
static uint32_t DrawColorOutputMask(const HW::Context& ctx) {
const auto& sh_regs = ctx.GetShaderRegisters();
const auto write_mask = ctx.GetRenderTargetMask() & sh_regs.m_cbShaderMask;
uint32_t output_mask = 0;
for (uint32_t slot = 0; slot < RENDER_COLOR_ATTACHMENTS_MAX; slot++) {
if (sh_regs.target_output_mode[slot] != 0 &&
render_target_mask_slot(write_mask, slot) != 0) {
output_mask |= 1u << slot;
}
}
return output_mask;
}
enum class CbColorMode : uint8_t {
@@ -655,19 +634,22 @@ struct PreparedIndexBuffer {
vk::IndexType type = vk::IndexType::eUint16;
};
static uint64_t VertexBufferDescriptorSize(const ShaderVertexInputBuffer& buffer,
const ShaderVertexInputInfo& info) {
static uint64_t VertexBufferDescriptorSize(int binding, const ShaderVertexInputInfo& info) {
const auto& buffer = info.buffers[binding];
if (buffer.stride != 0 || buffer.num_records == 0) {
return static_cast<uint64_t>(buffer.stride) * buffer.num_records;
}
uint64_t size = 0;
for (int i = 0; i < buffer.attr_num; i++) {
const auto& resource = info.resources[buffer.attr_indices[i]];
for (int i = 0; i < info.resources_num; i++) {
if (info.resources_dst[i].buffer_index != binding) {
continue;
}
const auto& resource = info.resources[i];
// RDNA2 OOB_SELECT=2 only checks NumRecords != 0. A constant attribute still
// fetches its entire format; NumRecords is not a byte count in this mode.
const uint64_t extent = resource.OutOfBounds() == 2
? static_cast<uint64_t>(buffer.attr_offsets[i]) +
? resource.Base48() - buffer.addr +
ShaderRecompiler::Format::GetFormatInfo(resource.Format()).byte_size
: buffer.num_records;
size = std::max(size, extent);
@@ -699,11 +681,13 @@ static PreparedVertexBuffers AcquireVertexBuffers(CommandBuffer& b
vs_input_info.buffers_num > ShaderVertexInputInfo::RES_MAX);
// Collect the non-empty guest vertex ranges.
std::array<uint64_t, ShaderVertexInputInfo::RES_MAX> sizes {};
std::array<VertexBufferRange, ShaderVertexInputInfo::RES_MAX> ranges {};
uint32_t range_count = 0;
for (int i = 0; i < vs_input_info.buffers_num; i++) {
const auto& vertex = vs_input_info.buffers[i];
const auto size = VertexBufferDescriptorSize(vertex, vs_input_info);
const auto size = VertexBufferDescriptorSize(i, vs_input_info);
sizes[i] = size;
if (size == 0) {
continue;
}
@@ -741,9 +725,6 @@ static PreparedVertexBuffers AcquireVertexBuffers(CommandBuffer& b
Libs::LibKernel::Memory::ClampRangeSize(range.base_address, range.RequestedSize());
range.acquired_end = range.base_address + size;
range.binding = cache.ObtainBuffer(range.base_address, size, false);
SetVulkanObjectNameF(
buffer.GetContext().GetGraphics().device, range.binding.first->Handle(),
"Kyty.VertexBufferRange[guest=0x{:016x} size=0x{:x}]", range.base_address, size);
}
// Rebuild slot bindings, offsetting non-empty slots into their acquired merged range.
@@ -752,7 +733,7 @@ static PreparedVertexBuffers AcquireVertexBuffers(CommandBuffer& b
vk::Buffer null_buffer = nullptr;
for (int i = 0; i < vs_input_info.buffers_num; i++) {
const auto& vertex = vs_input_info.buffers[i];
const auto size = VertexBufferDescriptorSize(vertex, vs_input_info);
const auto size = sizes[i];
if (size == 0) {
if (null_buffer == nullptr) {
null_buffer = cache.GetBuffer(NULL_BUFFER_ID).Handle();
@@ -775,10 +756,6 @@ static PreparedVertexBuffers AcquireVertexBuffers(CommandBuffer& b
prepared.buffers[i] = range->binding.first->Handle();
prepared.offsets[i] = range->binding.second + vertex.addr - range->base_address;
prepared.sizes[i] = std::min(size, range->acquired_end - vertex.addr);
SetVulkanObjectNameF(
buffer.GetContext().GetGraphics().device, prepared.buffers[i],
"Kyty.VertexBuffer[slot={} guest=0x{:016x} size=0x{:x} stride={} records={}]", i,
vertex.addr, size, vertex.stride, vertex.num_records);
}
return prepared;
@@ -875,7 +852,7 @@ static bool ResolvePrimitiveRestart(const CommandBuffer& buffer,
}
static void RefreshShaders(CommandBuffer& buffer, const DrawCallInfo& draw,
DrawRenderState& state) {
uint32_t color_output_mask, DrawRenderState& state) {
auto& ctx = buffer.GetRegisters();
auto& sh_ctx = buffer.GetShaders();
@@ -889,7 +866,7 @@ static void RefreshShaders(CommandBuffer& buffer, const DrawCallInfo& draw,
target_export_mapping {};
for (uint32_t slot = 0; slot < RENDER_COLOR_ATTACHMENTS_MAX; slot++) {
const auto& rt = ctx.GetRenderTarget(slot);
if (rt.base.addr != 0 && render_target_mask_slot(ctx.GetRenderTargetMask(), slot) != 0) {
if ((color_output_mask & (1u << slot)) != 0 && rt.base.addr != 0) {
target_export_mapping[slot] =
TextureGetRenderTargetFormat(rt.info.format, rt.info.channel_type,
rt.info.channel_order)
@@ -908,8 +885,12 @@ static void RefreshShaders(CommandBuffer& buffer, const DrawCallInfo& draw,
bool RenderExecutor::PrepareDrawRenderState(CommandBuffer& buffer, const DrawCallInfo& draw,
uint32_t render_target_slice_offset,
DrawRenderState& state) {
state.ps_active = DrawHasActivePixelShader(buffer);
RefreshShaders(buffer, draw, state);
const auto& shader_regs = buffer.GetRegisters().GetShaderRegisters();
const auto color_output_mask = DrawColorOutputMask(buffer.GetRegisters());
state.ps_active = buffer.GetShaders().GetPs().ps_regs.data_addr != 0 &&
(color_output_mask != 0 ||
PixelShaderHasDepthOrCoverageSideEffects(shader_regs));
RefreshShaders(buffer, draw, color_output_mask, state);
uint32_t mrt_mask = 0;
if (state.ps_active) {
for (const auto& output: state.ps_input_info.stage.program->info.outputs) {
@@ -918,6 +899,7 @@ bool RenderExecutor::PrepareDrawRenderState(CommandBuffer& buffer, const DrawCal
}
}
}
mrt_mask &= color_output_mask;
if (draw.IsIndexed()) {
LogDrawPhase(draw.Name(), "ResolveRenderColorTarget");
}
@@ -955,17 +937,11 @@ static PreparedIndexBuffer PrepareIndexBuffer(CommandBuffer& buffe
auto& stream = buffer.GetContext().GetBufferCache().GetUtilityBuffer(MemoryUsage::Stream);
prepared.offset = stream.Copy(source.host_data, source.size, 16);
prepared.buffer = stream.Handle();
SetVulkanObjectNameF(buffer.GetContext().GetGraphics().device, prepared.buffer,
"Kyty.IndexBuffer[guest=transient size=0x{:x} type={}]", source.size,
static_cast<uint32_t>(source.type));
} else {
auto [buffer_ptr, offset] =
buffer.GetContext().GetBufferCache().ObtainBuffer(source.address, source.size, false);
prepared.buffer = buffer_ptr->Handle();
prepared.offset = offset;
SetVulkanObjectNameF(buffer.GetContext().GetGraphics().device, prepared.buffer,
"Kyty.IndexBuffer[guest=0x{:016x} size=0x{:x} type={}]",
source.address, source.size, static_cast<uint32_t>(source.type));
}
return prepared;
}
@@ -1056,6 +1032,7 @@ void RenderExecutor::ExecutePreparedDraw(uint64_t submit_id, CommandBuffer& buff
uint32_t mesh_groups = 0;
if (mesh_active) {
const auto& mesh = state.vertex_info[0].mesh;
EXIT_NOT_IMPLEMENTED(mesh.fast_launch && (draw.IsIndexed() || primitive_restart_enable));
static std::atomic_bool restart_warned = false;
if (primitive_restart_enable && !restart_warned.exchange(true, std::memory_order_relaxed)) {
std::printf("Warning: primitive restart is not implemented for mesh shaders; "
@@ -1089,15 +1066,16 @@ void RenderExecutor::ExecutePreparedDraw(uint64_t submit_id, CommandBuffer& buff
index_source.guest_element_size);
}
LogDrawPhase(draw.Name(), "PrepareBindings");
GraphicsBindings bindings;
auto& bindings = m_graphics_bindings;
std::array<PreparedBindings*, 4> descriptor_stages {};
uint32_t stage_count = 0;
for (uint32_t i = 0; i < vertex_stages.size(); i++) {
bindings.vertex[i] = PrepareBindings(state.vertex_info[i].stage);
PrepareBindings(state.vertex_info[i].stage, bindings.vertex[i]);
descriptor_stages[stage_count++] = &bindings.vertex[i];
}
if (state.ps_active) {
bindings.pixel.emplace(PrepareBindings(state.ps_input_info.stage));
if (!bindings.pixel) bindings.pixel.emplace();
PrepareBindings(state.ps_input_info.stage, *bindings.pixel);
descriptor_stages[stage_count++] = &*bindings.pixel;
}
const auto stages = std::span {descriptor_stages.data(), stage_count};
@@ -1109,10 +1087,6 @@ void RenderExecutor::ExecutePreparedDraw(uint64_t submit_id, CommandBuffer& buff
vertex_bindings = AcquireVertexBuffers(buffer, state.vertex_info[0]);
index_binding = PrepareIndexBuffer(buffer, index_source);
}
const auto rendering =
AcquireRenderTargets(buffer, state.color_info, state.color_count, state.depth_info,
bindings.pixel);
if (draw.IsIndexed()) {
LogDrawPhase(draw.Name(), "CreatePipeline");
}
@@ -1120,6 +1094,10 @@ void RenderExecutor::ExecutePreparedDraw(uint64_t submit_id, CommandBuffer& buff
std::span {state.color_info, state.color_count}, state.depth_info, vertex_stages, buffer,
state.ps_active ? &state.ps_input_info : nullptr, topology, primitive_restart_enable,
state.programs);
vk::ImageAspectFlags feedback_aspects;
const auto rendering =
AcquireRenderTargets(buffer, state.color_info, state.color_count, state.depth_info,
feedback_aspects, stages);
// Resource preparation above may synchronously finish and restart the scheduler. From this
// point onward, every operation targets the current command buffer and cannot touch guest
@@ -1129,7 +1107,7 @@ void RenderExecutor::ExecutePreparedDraw(uint64_t submit_id, CommandBuffer& buff
if (!mesh_active) {
CommitVertexBuffers(vk_buffer, vertex_bindings);
}
if (bindings.pixel && !draw.IsIndexed()) {
if (state.ps_active && !draw.IsIndexed()) {
SetDrawDebugPhase(buffer, submit_id, draw, 0x300u);
}
CommitBindings(buffer, vk::PipelineBindPoint::eGraphics, pipeline, stages);
@@ -1151,11 +1129,7 @@ void RenderExecutor::ExecutePreparedDraw(uint64_t submit_id, CommandBuffer& buff
SetGraphicsDynamicParams(buffer, vk_buffer, vertex_stages.back(), state.depth_info, rendering);
if (m_context.GetGraphics().attachment_feedback_loop_enabled) {
vk_buffer.setAttachmentFeedbackLoopEnableEXT(
rendering.depth_stencil_attachment.image_layout ==
vk::ImageLayout::eAttachmentFeedbackLoopOptimalEXT
? vk::ImageAspectFlags {vk::ImageAspectFlagBits::eDepth}
: vk::ImageAspectFlags {});
vk_buffer.setAttachmentFeedbackLoopEnableEXT(feedback_aspects);
}
LogDrawPhase(draw.Name(), "BeginRendering");
+1 -21
View File
@@ -112,26 +112,6 @@ void WriteAtEndOfPipe64(uint64_t submit_id, CommandBuffer& buffer, uint64_t* dst
EndOfPipeWriteSize::Qword, EndOfPipeWriteAction::Write);
}
void WriteAtEndOfPipeClockCounter(uint64_t submit_id, CommandBuffer& buffer, uint64_t* dst_gpu_addr,
uint64_t value) {
RecordEndOfPipeWrite(submit_id, buffer, reinterpret_cast<uint64_t>(dst_gpu_addr), 0,
EndOfPipeWriteSize::Qword, EndOfPipeWriteAction::Write);
LOGF_COLOR(Log::Color::BrightGreen,
"EndOfPipe Signal!!! [0x%016" PRIx64 "] <- Clock: 0x%016" PRIx64 "\n",
reinterpret_cast<uint64_t>(dst_gpu_addr), value);
}
void WriteAtEndOfPipeClockCounterWithWriteBack(uint64_t submit_id, CommandBuffer& buffer,
uint64_t* dst_gpu_addr, uint64_t value) {
RecordEndOfPipeWrite(submit_id, buffer, reinterpret_cast<uint64_t>(dst_gpu_addr), 0,
EndOfPipeWriteSize::Qword, EndOfPipeWriteAction::WriteBack);
LOGF_COLOR(Log::Color::BrightGreen,
"EndOfPipe Signal!!! [0x%016" PRIx64 "] <- Clock: 0x%016" PRIx64 "\n",
reinterpret_cast<uint64_t>(dst_gpu_addr), value);
}
void WriteAtEndOfPipeWithWriteBack64(uint64_t submit_id, CommandBuffer& buffer,
uint64_t* dst_gpu_addr, uint64_t value) {
RecordEndOfPipeWrite(submit_id, buffer, reinterpret_cast<uint64_t>(dst_gpu_addr), value,
@@ -192,7 +172,7 @@ uint64_t PrepareVideoOutFlip(CommandBuffer& buffer, int handle, int index, int f
"\n",
result, handle, index, flip_mode, flip_arg);
}
video_out.WaitForSubmitSlot();
video_out.WaitForSubmitSlot(handle);
}
}
-4
View File
@@ -25,10 +25,6 @@ void WriteAtEndOfPipe64(uint64_t submit_id, CommandBuffer& buffer, uint64_t* dst
uint64_t value);
void WriteAtEndOfPipeGds32(uint64_t submit_id, CommandBuffer& buffer, uint32_t* dst_gpu_addr,
uint32_t dw_offset, uint32_t dw_num);
void WriteAtEndOfPipeClockCounter(uint64_t submit_id, CommandBuffer& buffer, uint64_t* dst_gpu_addr,
uint64_t value);
void WriteAtEndOfPipeClockCounterWithWriteBack(uint64_t submit_id, CommandBuffer& buffer,
uint64_t* dst_gpu_addr, uint64_t value);
void WriteAtEndOfPipeWithWriteBack32(uint64_t submit_id, CommandBuffer& buffer,
uint32_t* dst_gpu_addr, uint32_t value);
void WriteAtEndOfPipeWithWriteBack64(uint64_t submit_id, CommandBuffer& buffer,
@@ -0,0 +1,15 @@
#version 450
layout(location = 0) in vec2 uv;
layout(location = 0) out vec4 color;
layout(binding = 0) uniform sampler2D source;
layout(push_constant) uniform AlphaControl {
uint premultiplied;
};
void main() {
color = texture(source, uv);
if (premultiplied == 0u) {
color.rgb *= color.a;
}
}
+10 -2
View File
@@ -4,6 +4,7 @@
#include "common/common.h"
#include <memory>
#include <span>
namespace Libs::Graphics {
@@ -15,6 +16,11 @@ struct WindowContext;
class Presenter final {
public:
struct Frame;
struct Layer {
Frame* frame;
int bus;
bool premultiplied_alpha;
};
explicit Presenter(WindowContext& window);
~Presenter();
@@ -23,11 +29,13 @@ public:
[[nodiscard]] Frame& PrepareFrame(CommandBuffer& command, const ImageInfo& info);
[[nodiscard]] Frame& PrepareBlankFrame(uint32_t width, uint32_t height, bool opaque,
CommandBuffer* producer = nullptr);
[[nodiscard]] Frame* PrepareLastFrame();
[[nodiscard]] bool PresentLastFrame();
[[nodiscard]] bool IsGuestPaused() const noexcept;
[[nodiscard]] bool NeedsSystemOverlayRefresh() const noexcept;
[[nodiscard]] RenderContext& Renderer() const noexcept;
void Present(Frame& frame, bool reuse = false);
void Present(Frame& frame);
void Present(std::span<const Layer> layers);
void ClearLayer(int bus);
void Discard(Frame& frame);
private:
+105 -94
View File
@@ -1,6 +1,7 @@
#include "graphics/presentation/systemOverlay.h"
#include "SDL.h"
#include <SDL3/SDL.h>
#include "common/assert.h"
#include "common/stringUtils.h"
#include "graphics/host_gpu/graphicContext.h"
@@ -28,9 +29,9 @@ namespace Libs::Graphics {
namespace {
namespace CoreIme = Libs::Ime;
namespace DialogIme = Libs::Dialog::ImeDialog;
namespace ErrorDialog = Libs::Dialog::ErrorDialog;
namespace CoreIme = Libs::Ime;
namespace DialogIme = Libs::Dialog::ImeDialog;
namespace SystemDialog = Libs::Dialog::SystemDialog;
namespace Ime {
@@ -103,7 +104,7 @@ bool HostQueueExternalInput(uint64_t generation, ExternalInput input) {
} // namespace Ime
enum class OverlayKind : uint8_t { None, Ime, Error };
enum class OverlayKind : uint8_t { None, Ime, Dialog };
struct OverlaySession {
OverlayKind kind = OverlayKind::None;
@@ -113,14 +114,14 @@ struct OverlaySession {
};
struct OverlaySnapshot {
OverlaySession session;
Ime::HostSnapshot ime;
ErrorDialog::HostSnapshot error;
OverlaySession session;
Ime::HostSnapshot ime;
SystemDialog::HostSnapshot dialog;
};
bool GetOverlaySnapshot(OverlaySnapshot* snapshot) {
if (ErrorDialog::GetHostSnapshot(&snapshot->error)) {
snapshot->session = {OverlayKind::Error, snapshot->error.generation};
if (SystemDialog::GetHostSnapshot(&snapshot->dialog)) {
snapshot->session = {OverlayKind::Dialog, snapshot->dialog.generation};
return true;
}
if (Ime::GetHostSnapshot(&snapshot->ime)) {
@@ -176,6 +177,7 @@ bool g_input_multiline = false;
bool g_input_lifecycle_active = false;
bool g_controller_captured = false;
OverlaySession g_session;
SDL_Window* g_input_window = nullptr;
void ClearInputEvents() {
std::scoped_lock lock(g_input_mutex);
@@ -212,7 +214,7 @@ void RetryVisibilityWakeup() {
}
SDL_Event event {};
event.type = type;
if (SDL_PushEvent(&event) > 0) {
if (SDL_PushEvent(&event)) {
g_missing_visibility_wakeups--;
}
}
@@ -232,7 +234,7 @@ void RefreshVisibility() {
bool capture_keyboard = false;
bool text_input = false;
bool multiline = false;
if (snapshot.session.kind == OverlayKind::Error) {
if (snapshot.session.kind == OverlayKind::Dialog) {
capture_controller = true;
capture_keyboard = true;
} else if (snapshot.session.kind == OverlayKind::Ime) {
@@ -252,7 +254,7 @@ void RefreshVisibility() {
std::scoped_lock input_lock(g_input_mutex);
g_visibility_updates.push_back({snapshot.session, visible, capture_controller,
capture_keyboard, text_input, multiline});
if (g_missing_visibility_wakeups != 0 || SDL_PushEvent(&event) <= 0) {
if (g_missing_visibility_wakeups != 0 || !SDL_PushEvent(&event)) {
g_missing_visibility_wakeups++;
}
}
@@ -268,16 +270,16 @@ void OnDialogVisibilityChanged(bool, uint64_t) {
uint32_t ExternalKeyStatus(SDL_Keymod modifiers, bool character_valid) {
uint32_t status = 0x00000001 | (character_valid ? 0x00000002 : 0);
if ((modifiers & KMOD_LCTRL) != 0) status |= 0x00000100;
if ((modifiers & KMOD_LSHIFT) != 0) status |= 0x00000200;
if ((modifiers & KMOD_LALT) != 0) status |= 0x00000400;
if ((modifiers & KMOD_LGUI) != 0) status |= 0x00000800;
if ((modifiers & KMOD_RCTRL) != 0) status |= 0x00001000;
if ((modifiers & KMOD_RSHIFT) != 0) status |= 0x00002000;
if ((modifiers & KMOD_RALT) != 0) status |= 0x00004000;
if ((modifiers & KMOD_RGUI) != 0) status |= 0x00008000;
if ((modifiers & KMOD_NUM) != 0) status |= 0x00010000;
if ((modifiers & KMOD_CAPS) != 0) status |= 0x00020000;
if ((modifiers & SDL_KMOD_LCTRL) != 0) status |= 0x00000100;
if ((modifiers & SDL_KMOD_LSHIFT) != 0) status |= 0x00000200;
if ((modifiers & SDL_KMOD_LALT) != 0) status |= 0x00000400;
if ((modifiers & SDL_KMOD_LGUI) != 0) status |= 0x00000800;
if ((modifiers & SDL_KMOD_RCTRL) != 0) status |= 0x00001000;
if ((modifiers & SDL_KMOD_RSHIFT) != 0) status |= 0x00002000;
if ((modifiers & SDL_KMOD_RALT) != 0) status |= 0x00004000;
if ((modifiers & SDL_KMOD_RGUI) != 0) status |= 0x00008000;
if ((modifiers & SDL_KMOD_NUM) != 0) status |= 0x00010000;
if ((modifiers & SDL_KMOD_CAPS) != 0) status |= 0x00020000;
return status;
}
@@ -392,14 +394,14 @@ float AlignmentPivot(Ime::Alignment alignment) {
ImGuiKey ControllerButtonToKey(int button) {
switch (button) {
case SDL_CONTROLLER_BUTTON_A: return ImGuiKey_GamepadFaceDown;
case SDL_CONTROLLER_BUTTON_B: return ImGuiKey_GamepadFaceRight;
case SDL_CONTROLLER_BUTTON_X: return ImGuiKey_GamepadFaceLeft;
case SDL_CONTROLLER_BUTTON_Y: return ImGuiKey_GamepadFaceUp;
case SDL_CONTROLLER_BUTTON_DPAD_LEFT: return ImGuiKey_GamepadDpadLeft;
case SDL_CONTROLLER_BUTTON_DPAD_RIGHT: return ImGuiKey_GamepadDpadRight;
case SDL_CONTROLLER_BUTTON_DPAD_UP: return ImGuiKey_GamepadDpadUp;
case SDL_CONTROLLER_BUTTON_DPAD_DOWN: return ImGuiKey_GamepadDpadDown;
case SDL_GAMEPAD_BUTTON_SOUTH: return ImGuiKey_GamepadFaceDown;
case SDL_GAMEPAD_BUTTON_EAST: return ImGuiKey_GamepadFaceRight;
case SDL_GAMEPAD_BUTTON_WEST: return ImGuiKey_GamepadFaceLeft;
case SDL_GAMEPAD_BUTTON_NORTH: return ImGuiKey_GamepadFaceUp;
case SDL_GAMEPAD_BUTTON_DPAD_LEFT: return ImGuiKey_GamepadDpadLeft;
case SDL_GAMEPAD_BUTTON_DPAD_RIGHT: return ImGuiKey_GamepadDpadRight;
case SDL_GAMEPAD_BUTTON_DPAD_UP: return ImGuiKey_GamepadDpadUp;
case SDL_GAMEPAD_BUTTON_DPAD_DOWN: return ImGuiKey_GamepadDpadDown;
default: return ImGuiKey_None;
}
}
@@ -415,9 +417,11 @@ void CheckVulkanResult(VkResult result) {
} // namespace
void InitializeSystemOverlayInput() {
void InitializeSystemOverlayInput(SDL_Window* window) {
EXIT_IF(window == nullptr);
const Uint32 type = SDL_RegisterEvents(1);
EXIT_IF(type == static_cast<Uint32>(-1));
g_input_window = window;
{
std::scoped_lock lock(g_visibility_mutex);
g_input_lifecycle_active = true;
@@ -425,14 +429,14 @@ void InitializeSystemOverlayInput() {
}
CoreIme::SetVisibilityCallback(OnCoreVisibilityChanged);
DialogIme::SetVisibilityCallback(OnDialogVisibilityChanged);
ErrorDialog::SetVisibilityCallback(RefreshVisibility);
SystemDialog::SetVisibilityCallback(RefreshVisibility);
RefreshVisibility();
}
void ShutdownSystemOverlayInput() {
CoreIme::SetVisibilityCallback(nullptr);
DialogIme::SetVisibilityCallback(nullptr);
ErrorDialog::SetVisibilityCallback(nullptr);
SystemDialog::SetVisibilityCallback(nullptr);
{
std::scoped_lock lock(g_visibility_mutex);
g_input_lifecycle_active = false;
@@ -453,17 +457,18 @@ void ShutdownSystemOverlayInput() {
g_input_controller = false;
g_input_keyboard = false;
g_input_multiline = false;
if (SDL_IsTextInputActive() == SDL_TRUE) {
SDL_StopTextInput();
if (g_input_window != nullptr && SDL_TextInputActive(g_input_window)) {
SDL_StopTextInput(g_input_window);
}
g_input_window = nullptr;
}
SystemOverlayVisualState GetSystemOverlayVisualState() noexcept {
const auto core = CoreIme::GetVisualState();
const auto dialog = DialogIme::GetVisualState();
const auto error = ErrorDialog::GetVisualState();
return {core.active || dialog.active || error.active,
core.revision + dialog.revision + error.revision};
const auto system = SystemDialog::GetVisualState();
return {core.active || dialog.active || system.active,
core.revision + dialog.revision + system.revision};
}
bool ProcessSystemOverlayInput(const SDL_Event& event) {
@@ -487,13 +492,13 @@ bool ProcessSystemOverlayInput(const SDL_Event& event) {
g_input_keyboard = update.capture_keyboard;
g_input_multiline = update.multiline;
if (update.text_input) {
SDL_StartTextInput();
} else if (SDL_IsTextInputActive() == SDL_TRUE) {
SDL_StopTextInput();
SDL_StartTextInput(g_input_window);
} else if (SDL_TextInputActive(g_input_window)) {
SDL_StopTextInput(g_input_window);
}
return true;
}
if (event.type == SDL_CONTROLLERDEVICEREMOVED) {
if (event.type == SDL_EVENT_GAMEPAD_REMOVED) {
if (g_input_active && g_input_controller) {
QueueInput({InputKind::ResetController, g_input_session, 0, 0.0f, 0.0f});
}
@@ -505,26 +510,27 @@ bool ProcessSystemOverlayInput(const SDL_Event& event) {
const auto session = g_input_session;
const uint64_t generation = session.generation;
const bool controller_event = event.type == SDL_CONTROLLERBUTTONDOWN ||
event.type == SDL_CONTROLLERBUTTONUP ||
event.type == SDL_CONTROLLERAXISMOTION;
const bool controller_event = event.type == SDL_EVENT_GAMEPAD_BUTTON_DOWN ||
event.type == SDL_EVENT_GAMEPAD_BUTTON_UP ||
event.type == SDL_EVENT_GAMEPAD_AXIS_MOTION;
if (controller_event && !g_input_controller) {
return false;
}
const bool keyboard_event = event.type == SDL_TEXTINPUT || event.type == SDL_TEXTEDITING ||
event.type == SDL_KEYDOWN || event.type == SDL_KEYUP;
const bool keyboard_event = event.type == SDL_EVENT_TEXT_INPUT || event.type == SDL_EVENT_TEXT_EDITING ||
event.type == SDL_EVENT_KEY_DOWN || event.type == SDL_EVENT_KEY_UP;
if (keyboard_event && !g_input_keyboard) {
return false;
}
if (keyboard_event && session.kind == OverlayKind::Error) {
if (event.type == SDL_KEYDOWN && event.key.repeat == 0 &&
(event.key.keysym.sym == SDLK_RETURN || event.key.keysym.sym == SDLK_KP_ENTER)) {
ErrorDialog::HostAccept(generation);
if (keyboard_event && session.kind == OverlayKind::Dialog) {
if (event.type == SDL_EVENT_KEY_DOWN && !event.key.repeat &&
(event.key.key == SDLK_RETURN || event.key.key == SDLK_KP_ENTER ||
event.key.key == SDLK_ESCAPE)) {
SystemDialog::HostClose(generation);
}
return true;
}
switch (event.type) {
case SDL_TEXTINPUT: {
case SDL_EVENT_TEXT_INPUT: {
const auto text = Utf8ToUtf16(event.text.text);
if (!text.empty()) {
auto input = MakeExternalInput(Ime::ExternalAction::Text, g_last_external_keycode,
@@ -535,26 +541,24 @@ bool ProcessSystemOverlayInput(const SDL_Event& event) {
}
return true;
}
case SDL_TEXTEDITING: return true;
case SDL_KEYDOWN: {
g_last_external_keycode = static_cast<uint16_t>(event.key.keysym.scancode);
g_last_external_status =
ExternalKeyStatus(static_cast<SDL_Keymod>(event.key.keysym.mod), false);
case SDL_EVENT_TEXT_EDITING: return true;
case SDL_EVENT_KEY_DOWN: {
g_last_external_keycode = static_cast<uint16_t>(event.key.scancode);
g_last_external_status = ExternalKeyStatus(event.key.mod, false);
auto action = Ime::ExternalAction::Text;
bool queue = true;
if (event.key.keysym.sym == SDLK_BACKSPACE) {
if (event.key.key == SDLK_BACKSPACE) {
action = Ime::ExternalAction::Backspace;
} else if (event.key.keysym.sym == SDLK_LEFT) {
} else if (event.key.key == SDLK_LEFT) {
action = Ime::ExternalAction::MoveLeft;
} else if (event.key.keysym.sym == SDLK_RIGHT) {
} else if (event.key.key == SDLK_RIGHT) {
action = Ime::ExternalAction::MoveRight;
} else if (event.key.keysym.sym == SDLK_ESCAPE) {
} else if (event.key.key == SDLK_ESCAPE) {
action = Ime::ExternalAction::Cancel;
} else if (event.key.keysym.sym == SDLK_RETURN ||
event.key.keysym.sym == SDLK_KP_ENTER) {
} else if (event.key.key == SDLK_RETURN || event.key.key == SDLK_KP_ENTER) {
action =
g_input_multiline ? Ime::ExternalAction::Newline : Ime::ExternalAction::Accept;
} else if (event.key.keysym.sym == SDLK_TAB) {
} else if (event.key.key == SDLK_TAB) {
action = Ime::ExternalAction::None;
} else {
queue = false;
@@ -566,28 +570,28 @@ bool ProcessSystemOverlayInput(const SDL_Event& event) {
}
return true;
}
case SDL_KEYUP: return true;
case SDL_CONTROLLERBUTTONDOWN:
case SDL_CONTROLLERBUTTONUP:
QueueInput({InputKind::Button, session, event.cbutton.button,
event.type == SDL_CONTROLLERBUTTONDOWN ? 1.0f : 0.0f, 0.0f});
case SDL_EVENT_KEY_UP: return true;
case SDL_EVENT_GAMEPAD_BUTTON_DOWN:
case SDL_EVENT_GAMEPAD_BUTTON_UP:
QueueInput({InputKind::Button, session, event.gbutton.button,
event.type == SDL_EVENT_GAMEPAD_BUTTON_DOWN ? 1.0f : 0.0f, 0.0f});
return true;
case SDL_CONTROLLERAXISMOTION:
QueueInput({InputKind::Axis, session, event.caxis.axis,
static_cast<float>(event.caxis.value), 0.0f});
case SDL_EVENT_GAMEPAD_AXIS_MOTION:
QueueInput({InputKind::Axis, session, event.gaxis.axis,
static_cast<float>(event.gaxis.value), 0.0f});
return true;
case SDL_MOUSEMOTION:
case SDL_EVENT_MOUSE_MOTION:
QueueInput({InputKind::MousePosition, session, 0, static_cast<float>(event.motion.x),
static_cast<float>(event.motion.y)});
return true;
case SDL_MOUSEBUTTONDOWN:
case SDL_MOUSEBUTTONUP:
case SDL_EVENT_MOUSE_BUTTON_DOWN:
case SDL_EVENT_MOUSE_BUTTON_UP:
QueueInput({InputKind::MousePosition, session, 0, static_cast<float>(event.button.x),
static_cast<float>(event.button.y)});
QueueInput({InputKind::MouseButton, session, event.button.button,
event.type == SDL_MOUSEBUTTONDOWN ? 1.0f : 0.0f, 0.0f});
event.type == SDL_EVENT_MOUSE_BUTTON_DOWN ? 1.0f : 0.0f, 0.0f});
return true;
case SDL_MOUSEWHEEL:
case SDL_EVENT_MOUSE_WHEEL:
QueueInput({InputKind::MouseWheel, session, 0, static_cast<float>(event.wheel.x),
static_cast<float>(event.wheel.y)});
return true;
@@ -681,9 +685,9 @@ struct SystemOverlay::Impl {
case InputKind::Button: {
const bool down = event.x != 0.0f;
if (session.kind == OverlayKind::Ime && down) {
if (event.id == SDL_CONTROLLER_BUTTON_B) {
if (event.id == SDL_GAMEPAD_BUTTON_EAST) {
Ime::HostCancel(session.generation);
} else if (event.id == SDL_CONTROLLER_BUTTON_Y) {
} else if (event.id == SDL_GAMEPAD_BUTTON_NORTH) {
Ime::HostBackspace(session.generation);
}
}
@@ -695,19 +699,19 @@ struct SystemOverlay::Impl {
}
case InputKind::Axis: {
const float value = event.x / (event.x < 0.0f ? 32768.0f : 32767.0f);
if (event.id == SDL_CONTROLLER_AXIS_LEFTX) {
if (event.id == SDL_GAMEPAD_AXIS_LEFTX) {
io.AddKeyAnalogEvent(ImGuiKey_GamepadLStickLeft, value < -0.25f,
std::max(-value, 0.0f));
io.AddKeyAnalogEvent(ImGuiKey_GamepadLStickRight, value > 0.25f,
std::max(value, 0.0f));
} else if (event.id == SDL_CONTROLLER_AXIS_LEFTY) {
} else if (event.id == SDL_GAMEPAD_AXIS_LEFTY) {
io.AddKeyAnalogEvent(ImGuiKey_GamepadLStickUp, value < -0.25f,
std::max(-value, 0.0f));
io.AddKeyAnalogEvent(ImGuiKey_GamepadLStickDown, value > 0.25f,
std::max(value, 0.0f));
} else if (event.id == SDL_CONTROLLER_AXIS_RIGHTX) {
} else if (event.id == SDL_GAMEPAD_AXIS_RIGHTX) {
right_stick.x = std::abs(value) > 0.2f ? value : 0.0f;
} else if (event.id == SDL_CONTROLLER_AXIS_RIGHTY) {
} else if (event.id == SDL_GAMEPAD_AXIS_RIGHTY) {
right_stick.y = std::abs(value) > 0.2f ? value : 0.0f;
}
break;
@@ -901,7 +905,7 @@ struct SystemOverlay::Impl {
ImGui::End();
}
void DrawError(const ErrorDialog::HostSnapshot& snapshot, vk::Extent2D extent) {
void DrawDialog(const SystemDialog::HostSnapshot& snapshot, vk::Extent2D extent) {
const ImVec2 display(static_cast<float>(extent.width), static_cast<float>(extent.height));
const float scale = std::max(std::min(display.x / 1280.0f, display.y / 720.0f), 0.5f);
ImGui::GetBackgroundDrawList()->AddRectFilled({0.0f, 0.0f}, display,
@@ -919,17 +923,23 @@ struct SystemOverlay::Impl {
constexpr ImGuiWindowFlags flags = ImGuiWindowFlags_NoDecoration | ImGuiWindowFlags_NoMove |
ImGuiWindowFlags_NoSavedSettings |
ImGuiWindowFlags_AlwaysAutoResize;
ImGui::Begin("##SystemError", nullptr, flags);
ImGui::TextUnformatted("Error");
const bool signin = snapshot.kind == SystemDialog::Kind::Signin;
ImGui::Begin("##SystemDialog", nullptr, flags);
ImGui::TextUnformatted(signin ? "Sign in to PlayStation Network" : "Error");
ImGui::Separator();
const auto error_code = static_cast<uint32_t>(snapshot.error_code);
ImGui::TextWrapped("%s", error_code == 0x80550006u
ImGui::TextWrapped("%s", signin ? "PlayStation Network sign-in is unavailable."
: error_code == 0x80550006u
? "You are not signed in to PlayStation Network."
: "An error has occurred.");
ImGui::TextDisabled("Error code: 0x%08X", error_code);
if (!signin) {
ImGui::TextDisabled("Error code: 0x%08X", error_code);
}
const float button_width = std::min(140.0f * scale, ImGui::GetContentRegionAvail().x);
ImGui::SetCursorPosX((ImGui::GetWindowSize().x - button_width) * 0.5f);
const bool accepted = ImGui::Button("OK", {button_width, 44.0f * scale});
const bool accepted =
ImGui::Button(signin ? "Cancel" : "OK", {button_width, 44.0f * scale}) ||
(signin && ImGui::IsKeyPressed(ImGuiKey_GamepadFaceRight, false));
if (focus_pending) {
ImGui::SetItemDefaultFocus();
focus_pending = false;
@@ -938,7 +948,7 @@ struct SystemOverlay::Impl {
ImGui::PopFont();
ImGui::PopStyleVar(2);
if (accepted) {
ErrorDialog::HostAccept(snapshot.generation);
SystemDialog::HostClose(snapshot.generation);
}
}
@@ -952,7 +962,8 @@ struct SystemOverlay::Impl {
if (session != snapshot.session) {
session = snapshot.session;
focus_pending = true;
shift = (snapshot.ime.option & Ime::OPTION_NO_AUTO_CAPITALIZE) == 0;
shift = session.kind == OverlayKind::Ime &&
(snapshot.ime.option & Ime::OPTION_NO_AUTO_CAPITALIZE) == 0;
symbol_mode = false;
panel_offset = {};
right_stick = {};
@@ -978,8 +989,8 @@ struct SystemOverlay::Impl {
ImGui::EndFrame();
return false;
}
if (snapshot.session.kind == OverlayKind::Error) {
DrawError(snapshot.error, frame_extent);
if (snapshot.session.kind == OverlayKind::Dialog) {
DrawDialog(snapshot.dialog, frame_extent);
} else {
DrawIme(snapshot.ime, frame_extent);
}
+2 -1
View File
@@ -7,6 +7,7 @@
#include <memory>
union SDL_Event;
struct SDL_Window;
namespace Libs::Graphics {
@@ -17,7 +18,7 @@ struct SystemOverlayVisualState {
uint64_t revision;
};
void InitializeSystemOverlayInput();
void InitializeSystemOverlayInput(SDL_Window* window);
void ShutdownSystemOverlayInput();
bool ProcessSystemOverlayInput(const SDL_Event& event);
SystemOverlayVisualState GetSystemOverlayVisualState() noexcept;
+252 -76
View File
@@ -20,9 +20,12 @@
#include "kernel/pthread.h"
#include "libs/errno.h"
#include "libs/libs.h"
#include "loader/systemContent.h"
#include <algorithm>
#include <array>
#include <atomic>
#include <fmt/format.h>
#include <list>
#include <thread>
#include <vector>
@@ -48,6 +51,8 @@ constexpr int VIDEO_OUT_BUS_TYPE_OVERLAY = 1;
constexpr int VIDEO_OUT_BUS_TYPE_SUB = 2;
constexpr int VIDEO_OUT_FLIP_MODE_VSYNC = 1;
constexpr int VIDEO_OUT_FLIP_MODE_VSYNC_MULTI = 4;
constexpr int VIDEO_OUT_FLIP_MODE_SLAVE = 8;
constexpr int VIDEO_OUT_FLIP_MODE_MASTER = 9;
constexpr int VIDEO_OUT_BUFFER_INDEX_BLACK = -2;
constexpr int VIDEO_OUT_BUFFER_INDEX_BLANK = -1;
constexpr int VIDEO_OUT_BUFFER_NUM_MAX = 16;
@@ -60,6 +65,7 @@ constexpr uint64_t VIDEO_OUT_REFRESH_RATE_119_88HZ = 13;
constexpr int VIDEO_OUT_BUFFER_ATTRIBUTE_CATEGORY_UNCOMPRESSED = 0;
constexpr int VIDEO_OUT_BUFFER_ATTRIBUTE_CATEGORY_COMPRESSED = 1;
constexpr uint64_t VIDEO_OUT_BUFFER_ATTRIBUTE_OPTION_STRICT_COLORIMETRY = 8;
constexpr uint64_t VIDEO_OUT_BUFFER_ATTRIBUTE_OPTION_ALPHA_PREMULTIPLIED = 32;
enum class VideoOutEventKind : uintptr_t {
Flip = VIDEO_OUT_EVENT_FLIP,
@@ -180,6 +186,9 @@ struct VideoOutConfig {
uint64_t generation = 0;
bool opened = false;
bool closing = false;
int bus = 0;
VideoOutConfig* master = nullptr;
uint32_t slave_count = 0;
int flip_rate = 0;
uint64_t output_mode = VIDEO_OUT_OUTPUT_MODE_DEFAULT;
float gamma = 1.0f;
@@ -203,7 +212,7 @@ public:
void Cancel(VideoOutConfig& cfg);
void Prepare(uint64_t request_id, Graphics::CommandBuffer& buffer);
void Complete(uint64_t request_id);
void WaitForSubmitSlot();
void WaitForSubmitSlot(VideoOutConfig& cfg);
bool Flip(uint32_t micros);
void GetFlipStatus(VideoOutConfig& cfg, VideoOutFlipStatus& out);
void Wait(VideoOutConfig& cfg, int index);
@@ -213,7 +222,9 @@ private:
struct Request {
uint64_t id;
uint64_t group;
VideoOutConfig* cfg;
VideoOutConfig* master;
uint64_t generation;
int index;
int64_t flip_arg;
@@ -221,6 +232,7 @@ private:
FlipRequestSource source;
RequestState state;
Graphics::Presenter::Frame* frame;
bool premultiplied_alpha;
};
Graphics::Presenter& m_presenter;
@@ -255,6 +267,7 @@ public:
VideoOutConfig* Get(int handle);
VideoOutConfig* Get(int handle, uint64_t& generation);
bool IsOpened(int handle);
int SetFlipMaster(int slave_handle, int master_handle);
void Init(uint32_t width, uint32_t height);
FlipQueue& GetFlipQueue() { return m_flip_queue; }
@@ -495,7 +508,8 @@ static bool IsSpecialBufferIndex(int index) {
}
static bool IsValidFlipMode(int mode) {
return mode >= VIDEO_OUT_FLIP_MODE_VSYNC && mode <= VIDEO_OUT_FLIP_MODE_VSYNC_MULTI;
return (mode >= VIDEO_OUT_FLIP_MODE_VSYNC && mode <= VIDEO_OUT_FLIP_MODE_VSYNC_MULTI) ||
mode == VIDEO_OUT_FLIP_MODE_SLAVE || mode == VIDEO_OUT_FLIP_MODE_MASTER;
}
static int ReserveFlipRequest(VideoOutDriver::Impl& driver, int handle, int index, int flip_mode,
@@ -505,13 +519,17 @@ static int ReserveFlipRequest(VideoOutDriver::Impl& driver, int handle, int inde
return VIDEO_OUT_ERROR_INVALID_HANDLE;
}
if (!IsValidFlipMode(flip_mode)) {
return VIDEO_OUT_ERROR_INVALID_VALUE;
return VIDEO_OUT_ERROR_INVALID_FLIP_MODE;
}
if (!IsValidBufferIndex(index)) {
return VIDEO_OUT_ERROR_INVALID_INDEX;
}
Common::LockGuard lock(video_out->mutex);
if ((video_out->master != nullptr) != (flip_mode == VIDEO_OUT_FLIP_MODE_SLAVE) ||
(video_out->slave_count != 0) != (flip_mode == VIDEO_OUT_FLIP_MODE_MASTER)) {
return VIDEO_OUT_ERROR_INVALID_FLIP_MODE;
}
if (video_out->closing ||
(!IsSpecialBufferIndex(index) && !video_out->buffers[index].Occupied())) {
return VIDEO_OUT_ERROR_INVALID_INDEX;
@@ -529,8 +547,8 @@ Graphics::ImageInfo BufferAttributeGroup::ImageInfo(const VideoOutBuffer& buffer
if (attribute.reserved0 != 0 || attribute.aspect_ratio != 0 || attribute.width == 0 ||
attribute.height == 0 || attribute.width > 16384 || attribute.height > 16384 ||
attribute.pitch_in_pixel != 0 ||
(attribute.option != 0 &&
attribute.option != VIDEO_OUT_BUFFER_ATTRIBUTE_OPTION_STRICT_COLORIMETRY) ||
(attribute.option & ~(VIDEO_OUT_BUFFER_ATTRIBUTE_OPTION_STRICT_COLORIMETRY |
VIDEO_OUT_BUFFER_ATTRIBUTE_OPTION_ALPHA_PREMULTIPLIED)) != 0 ||
attribute.tiling_mode != 0 || attribute.pad0 != 0 || attribute.reserved1[0] != 0 ||
attribute.reserved1[1] != 0 || attribute.reserved1[2] != 0 || buffer.data_address == 0 ||
compression == Graphics::VideoOutCompression::Unsupported) {
@@ -540,6 +558,14 @@ Graphics::ImageInfo BufferAttributeGroup::ImageInfo(const VideoOutBuffer& buffer
if (!Graphics::DecodeVideoOutPixelFormat(attribute.pixel_format, pixel_format)) {
EXIT("unsupported video-out pixel format: 0x%016" PRIx64 "\n", attribute.pixel_format);
}
if (attribute.pixel_format == Graphics::VIDEO_OUT_PIXEL_FORMAT_R10_G10_B10_A2_BT2100_PQ) {
static std::atomic_flag warned = ATOMIC_FLAG_INIT;
if (!warned.test_and_set(std::memory_order_relaxed)) {
Log::WriteToConsoleAndLog(fmt::format(
"Warning: HDR-to-SDR conversion is not implemented; displaying PQ video-out "
"pixels unchanged (format=0x{:016x}).\n", attribute.pixel_format));
}
}
const auto tile_mode = Graphics::Prospero::TileMode::kRenderTarget;
const auto pitch =
Graphics::TileGetTexturePitch(pixel_format.guest_format, attribute.width, tile_mode);
@@ -564,7 +590,12 @@ Graphics::ImageInfo BufferAttributeGroup::ImageInfo(const VideoOutBuffer& buffer
info.bgra16 = pixel_format.bgra16;
info.mip_layout[0] = {0, total.size, pitch, attribute.height};
if (compression != Graphics::VideoOutCompression::Uncompressed) {
info.metadata.range = {buffer.metadata_address, 0};
Graphics::TileSizeAlign dcc_size {};
if (!Graphics::TileGetDccSize(attribute.width, attribute.height, 1,
pixel_format.bytes_per_element, 1, tile_mode, dcc_size)) {
EXIT("invalid video-out DCC footprint\n");
}
info.metadata.range = {buffer.metadata_address, dcc_size.size};
info.metadata.kind = Graphics::ImageMetadataKind::Dcc;
info.metadata.control = attribute.dcc_control;
info.metadata.compression = compression;
@@ -600,6 +631,11 @@ VideoOutDriver::Impl::~Impl() {
m_present_thread.request_stop();
m_present_thread.join();
}
for (int handle = 1; handle < VIDEO_OUT_NUM_MAX; handle++) {
if (m_video_out_ctx[handle].master != nullptr) {
(void)Close(handle);
}
}
for (int handle = 1; handle < VIDEO_OUT_NUM_MAX; handle++) {
(void)Close(handle);
}
@@ -644,6 +680,7 @@ int VideoOutDriver::Impl::Open(int bus_type, int index) {
config.closing = false;
config.opened = true;
config.bus = bus_type;
if (++config.generation == 0) {
EXIT("video-out port generation wrapped\n");
}
@@ -670,11 +707,13 @@ bool VideoOutDriver::Impl::Close(int handle) {
VideoOutEventQueues pre_vblank_events;
VideoOutEventQueues vblank_events;
VideoOutEventQueues output_mode_events;
VideoOutConfig* master = nullptr;
{
Common::LockGuard config_lock(config.mutex);
if (config.closing) {
return false;
}
EXIT_IF(config.slave_count != 0);
config.opened = false;
config.closing = true;
if (++config.generation == 0) {
@@ -687,7 +726,9 @@ bool VideoOutDriver::Impl::Close(int handle) {
vblank_events = std::move(config.events->vblank);
output_mode_events = std::move(config.events->output_mode);
}
config.flip_rate = 0;
config.flip_rate = 0;
master = config.master;
config.master = nullptr;
for (const auto& buffer: config.buffers) {
if (buffer.Occupied() &&
@@ -704,8 +745,12 @@ bool VideoOutDriver::Impl::Close(int handle) {
}
config.vblank_cond.SignalAll();
}
if (master != nullptr) {
Common::LockGuard master_lock(master->mutex);
master->slave_count--;
}
m_flip_queue.Cancel(config);
m_presenter.ClearLayer(config.bus);
DeleteVideoOutEvents(flip_events, VideoOutEventKind::Flip);
DeleteVideoOutEvents(pre_vblank_events, VideoOutEventKind::PreVblankStart);
DeleteVideoOutEvents(vblank_events, VideoOutEventKind::Vblank);
@@ -743,6 +788,31 @@ bool VideoOutDriver::Impl::IsOpened(int handle) {
return handle > 0 && handle < VIDEO_OUT_NUM_MAX && m_video_out_ctx[handle].opened;
}
int VideoOutDriver::Impl::SetFlipMaster(int slave_handle, int master_handle) {
Common::LockGuard lock(m_mutex);
if (slave_handle <= 0 || slave_handle >= VIDEO_OUT_NUM_MAX ||
master_handle <= 0 || master_handle >= VIDEO_OUT_NUM_MAX ||
!m_video_out_ctx[slave_handle].opened || !m_video_out_ctx[master_handle].opened) {
return VIDEO_OUT_ERROR_INVALID_HANDLE;
}
auto& slave = m_video_out_ctx[slave_handle];
auto& master = m_video_out_ctx[master_handle];
if (&slave == &master) {
return VIDEO_OUT_ERROR_UNSUPPORTED_OPERATION;
}
Common::LockGuard first_lock(m_video_out_ctx[std::min(slave_handle, master_handle)].mutex);
Common::LockGuard second_lock(m_video_out_ctx[std::max(slave_handle, master_handle)].mutex);
if (slave.master != nullptr || slave.slave_count != 0 || master.master != nullptr) {
return VIDEO_OUT_ERROR_UNSUPPORTED_OPERATION;
}
if (slave.flip_status.flipPendingNum != 0 || master.flip_status.flipPendingNum != 0) {
return VIDEO_OUT_ERROR_RESOURCE_BUSY;
}
slave.master = &master;
master.slave_count++;
return OK;
}
void VideoOutDriver::Impl::VblankBegin() {
Common::LockGuard lock(m_mutex);
@@ -805,9 +875,7 @@ void VideoOutDriver::Impl::PresentThread(std::stop_token token) {
const auto period = std::max(frequency / refresh, uint64_t {1});
if (m_presenter.IsGuestPaused()) {
if (auto* frame = m_presenter.PrepareLastFrame(); frame != nullptr) {
m_presenter.Present(*frame, true);
}
(void)m_presenter.PresentLastFrame();
const auto frame_end = Common::Timer::QueryPerformanceCounter();
total_wait +=
static_cast<int64_t>(period) - static_cast<int64_t>(frame_end - frame_begin);
@@ -817,8 +885,7 @@ void VideoOutDriver::Impl::PresentThread(std::stop_token token) {
VblankBegin();
bool presented = m_flip_queue.Flip(0);
if (!presented && m_presenter.NeedsSystemOverlayRefresh()) {
if (auto* frame = m_presenter.PrepareLastFrame(); frame != nullptr) {
m_presenter.Present(*frame, true);
if (m_presenter.PresentLastFrame()) {
presented = true;
} else {
uint32_t width = 0;
@@ -861,25 +928,57 @@ bool FlipQueue::Reserve(VideoOutConfig& cfg, int index, int64_t flip_arg, FlipRe
uint64_t& request_id) {
Common::LockGuard lock(m_mutex);
if (m_requests.size() + m_cpu_requests.size() >= VIDEO_OUT_FLIP_QUEUE_CAPACITY) {
// A newer slave request replaces the one not yet captured by a master flip.
if (cfg.master != nullptr) {
for (auto* queue: {&m_requests, &m_cpu_requests}) {
for (auto it = queue->begin(); it != queue->end();) {
if (it->cfg != &cfg || it->group != 0) {
++it;
continue;
}
cfg.flip_status.flipPendingNum--;
cfg.flip_status.gcQueueNum -= it->source == FlipRequestSource::GpuEop;
if (it->state == RequestState::Ready) {
m_presenter.Discard(*it->frame);
it = queue->erase(it);
} else {
auto cancelled = it++;
m_cancelled_requests.splice(m_cancelled_requests.end(), *queue, cancelled);
}
}
}
m_done_cond_var.SignalAll();
}
if (cfg.flip_status.flipPendingNum >= VIDEO_OUT_FLIP_QUEUE_CAPACITY) {
return false;
}
auto& pending = source == FlipRequestSource::GpuEop ? m_requests : m_cpu_requests;
Request r {};
r.id = m_next_request_id++;
r.group = cfg.master == nullptr ? r.id : 0;
r.cfg = &cfg;
r.master = cfg.master;
r.generation = cfg.generation;
r.index = index;
r.flip_arg = flip_arg;
r.submit_ptc = LibKernel::KernelGetProcessTimeCounter();
r.source = source;
r.state = RequestState::Reserved;
if (cfg.slave_count != 0) {
for (auto* queue: {&m_requests, &m_cpu_requests}) {
for (auto& slave: *queue) {
if (slave.master == &cfg && slave.group == 0) {
slave.group = r.id;
}
}
}
}
pending.push_back(r);
request_id = r.id;
cfg.flip_status.flipPendingNum = static_cast<int>(m_requests.size() + m_cpu_requests.size());
cfg.flip_status.flipPendingNum++;
cfg.flip_status.submitProcessTimeCounter = r.submit_ptc;
if (source == FlipRequestSource::GpuEop) {
cfg.flip_status.gcQueueNum++;
@@ -901,7 +1000,9 @@ FlipQueue::~FlipQueue() {
void FlipQueue::Cancel(VideoOutConfig& cfg) {
std::vector<Graphics::Presenter::Frame*> frames;
m_mutex.Lock();
while (m_processing && !m_requests.empty() && m_requests.front().cfg == &cfg) {
while (m_processing && std::any_of(m_requests.begin(), m_requests.end(), [&cfg](const auto& r) {
return r.cfg == &cfg && r.state == RequestState::Presenting;
})) {
m_done_cond_var.Wait(&m_mutex);
}
for (auto* queue: {&m_requests, &m_cpu_requests}) {
@@ -975,6 +1076,7 @@ void FlipQueue::Prepare(uint64_t request_id, Graphics::CommandBuffer& buffer) {
uint32_t width = 0;
uint32_t height = 0;
bool current = false;
bool premultiplied_alpha = false;
{
Common::LockGuard lock(cfg->mutex);
current = cfg->opened && !cfg->closing && cfg->generation == generation;
@@ -996,6 +1098,9 @@ void FlipQueue::Prepare(uint64_t request_id, Graphics::CommandBuffer& buffer) {
request_id, index, surface.group_index);
}
source_info = cfg->groups[surface.group_index].ImageInfo(surface);
premultiplied_alpha =
(cfg->groups[surface.group_index].attribute.option &
VIDEO_OUT_BUFFER_ATTRIBUTE_OPTION_ALPHA_PREMULTIPLIED) != 0;
}
}
}
@@ -1035,6 +1140,7 @@ void FlipQueue::Prepare(uint64_t request_id, Graphics::CommandBuffer& buffer) {
EXIT("video-out request changed while recording, id=%" PRIu64 "\n", request_id);
}
prepared->frame = frame;
prepared->premultiplied_alpha = premultiplied_alpha;
}
void FlipQueue::Complete(uint64_t request_id) {
@@ -1066,9 +1172,12 @@ void FlipQueue::Complete(uint64_t request_id) {
}
}
void FlipQueue::WaitForSubmitSlot() {
void FlipQueue::WaitForSubmitSlot(VideoOutConfig& cfg) {
Common::LockGuard lock(m_mutex);
while (m_requests.size() + m_cpu_requests.size() >= VIDEO_OUT_FLIP_QUEUE_CAPACITY) {
auto belongs_to_port = [&cfg](const auto& request) { return request.cfg == &cfg; };
while (std::count_if(m_requests.begin(), m_requests.end(), belongs_to_port) +
std::count_if(m_cpu_requests.begin(), m_cpu_requests.end(), belongs_to_port) >=
VIDEO_OUT_FLIP_QUEUE_CAPACITY) {
if (m_requests.empty()) {
EXIT("video-out queue is saturated by CPU flips queued behind the current EOP\n");
}
@@ -1095,75 +1204,90 @@ bool FlipQueue::Flip(uint32_t micros) {
m_mutex.Lock();
if (m_requests.empty()) {
m_submit_cond_var.WaitFor(&m_mutex, micros);
if (m_requests.empty()) {
m_mutex.Unlock();
return false;
}
}
if (m_processing) {
EXIT("video-out flip queue processing is already active\n");
}
if (m_requests.front().state != RequestState::Ready) {
EXIT_IF(m_processing);
const auto owner = std::find_if(m_requests.begin(), m_requests.end(),
[](const auto& r) { return r.group == r.id; });
if (owner == m_requests.end()) {
m_mutex.Unlock();
return false;
}
m_processing = true;
auto r = m_requests.front();
m_mutex.Unlock();
r.cfg->mutex.Lock();
if (!IsFlipDueLocked(*r.cfg, r.generation)) {
r.cfg->mutex.Unlock();
Common::LockGuard queue_lock(m_mutex);
m_processing = false;
m_done_cond_var.SignalAll();
const uint64_t group = owner->id;
if (std::any_of(m_cpu_requests.begin(), m_cpu_requests.end(),
[group](const auto& r) { return r.group == group; }) ||
std::any_of(m_requests.begin(), m_requests.end(), [group](const auto& r) {
return r.group == group && r.state != RequestState::Ready;
})) {
m_mutex.Unlock();
return false;
}
m_mutex.Lock();
if (m_requests.empty() || m_requests.front().id != r.id ||
m_requests.front().state != RequestState::Ready || !m_processing) {
EXIT("video-out request changed before presentation, id=%" PRIu64 "\n", r.id);
std::array<Request, VideoOutDriver::Impl::VIDEO_OUT_BUSES.size()> requests {};
size_t count = 0;
for (auto& request: m_requests) {
if (request.group == group) {
EXIT_IF(count == requests.size());
request.state = RequestState::Presenting;
requests[count++] = request;
}
}
m_requests.front().state = RequestState::Presenting;
m_processing = true;
m_mutex.Unlock();
m_presenter.Present(*r.frame);
// Lock each port in bus order, then present and complete the whole group at one Vblank.
std::sort(requests.begin(), requests.begin() + count,
[](const auto& a, const auto& b) { return a.cfg->bus < b.cfg->bus; });
bool due = true;
std::array<Graphics::Presenter::Layer, VideoOutDriver::Impl::VIDEO_OUT_BUSES.size()> layers {};
for (size_t i = 0; i < count; i++) {
auto& r = requests[i];
r.cfg->mutex.Lock();
due &= r.cfg->opened && !r.cfg->closing && r.cfg->generation == r.generation;
if (r.id == group) {
due &= IsFlipDueLocked(*r.cfg, r.generation);
}
layers[i] = {r.frame, r.cfg->bus, r.premultiplied_alpha};
}
if (due) {
m_presenter.Present(std::span(layers.data(), count));
}
m_mutex.Lock();
if (m_requests.empty() || m_requests.front().id != r.id ||
m_requests.front().state != RequestState::Presenting) {
EXIT("video-out flip queue changed while processing its front request\n");
for (auto it = m_requests.begin(); it != m_requests.end();) {
if (it->group != group) {
++it;
continue;
}
if (!due) {
it++->state = RequestState::Ready;
continue;
}
auto& r = *it;
r.cfg->flip_status.count++;
r.cfg->flip_status.processTime = LibKernel::KernelGetProcessTime();
r.cfg->flip_status.processTimeCounter = LibKernel::KernelGetProcessTimeCounter();
r.cfg->flip_status.submitProcessTimeCounter = r.submit_ptc;
r.cfg->flip_status.flipArg = r.flip_arg;
r.cfg->flip_status.currentBuffer = r.index;
r.cfg->flip_status.flipPendingNum--;
r.cfg->flip_status.gcQueueNum -= r.source == FlipRequestSource::GpuEop;
TriggerVideoOutEvents(*r.cfg, VideoOutEventKind::Flip, reinterpret_cast<void*>(r.flip_arg));
it = m_requests.erase(it);
}
m_requests.pop_front();
r.cfg->flip_status.count++;
r.cfg->flip_status.processTime = LibKernel::KernelGetProcessTime();
r.cfg->flip_status.processTimeCounter = LibKernel::KernelGetProcessTimeCounter();
r.cfg->flip_status.submitProcessTimeCounter = r.submit_ptc;
r.cfg->flip_status.flipArg = r.flip_arg;
r.cfg->flip_status.currentBuffer = r.index;
r.cfg->flip_status.flipPendingNum = static_cast<int>(m_requests.size() + m_cpu_requests.size());
if (r.source == FlipRequestSource::GpuEop && r.cfg->flip_status.gcQueueNum > 0) {
r.cfg->flip_status.gcQueueNum--;
}
TriggerVideoOutEvents(*r.cfg, VideoOutEventKind::Flip, reinterpret_cast<void*>(r.flip_arg));
m_processing = false;
m_done_cond_var.SignalAll();
m_submit_slot_cond_var.Signal();
m_submit_slot_cond_var.SignalAll();
m_mutex.Unlock();
r.cfg->mutex.Unlock();
Graphics::RenderDocOnGuestFlip(m_presenter.Renderer());
if (Config::GraphicsDebugDumpEnabled() &&
Config::GetPrintfDirection() != Config::LogDirection::Silent) {
LOGF("Flip done: %d\n", r.index);
for (size_t i = count; i != 0; i--) {
requests[i - 1].cfg->mutex.Unlock();
}
return true;
if (due) {
Graphics::RenderDocOnGuestFlip(m_presenter.Renderer());
if (Config::GraphicsDebugDumpEnabled() &&
Config::GetPrintfDirection() != Config::LogDirection::Silent) {
LOGF("Flip done: %d\n", requests[0].index);
}
}
return due;
}
void FlipQueue::GetFlipStatus(VideoOutConfig& cfg, VideoOutFlipStatus& out) {
@@ -1172,6 +1296,18 @@ void FlipQueue::GetFlipStatus(VideoOutConfig& cfg, VideoOutFlipStatus& out) {
out = cfg.flip_status;
}
KYTY_SYSV_ABI int VideoOutSetFlipMaster(int slave_handle, int master_handle) {
PRINT_NAME();
return DriverState().SetFlipMaster(slave_handle, master_handle);
}
KYTY_SYSV_ABI void VideoOutAddBufferAttributeOption(VideoOutBufferAttribute2* attribute,
uint64_t option) {
PRINT_NAME();
EXIT_IF(attribute == nullptr);
attribute->option |= option;
}
KYTY_SYSV_ABI int VideoOutOpen(int user_id, int bus_type, int index, const void* param) {
PRINT_NAME();
@@ -1451,7 +1587,7 @@ KYTY_SYSV_ABI int VideoOutSubmitFlip(int handle, int index, int flip_mode, int64
uint64_t request_id = 0;
const int result = ReserveFlipRequest(DriverState(), handle, index, flip_mode, flip_arg,
FlipRequestSource::Cpu, request_id);
if (result == VIDEO_OUT_ERROR_INVALID_VALUE) {
if (result == VIDEO_OUT_ERROR_INVALID_FLIP_MODE) {
LOGF("\t unsupported flip_mode = %d\n", flip_mode);
}
if (result != OK) {
@@ -1488,8 +1624,10 @@ void VideoOutDriver::CompleteFlip(uint64_t request_id) {
m_impl->GetFlipQueue().Complete(request_id);
}
void VideoOutDriver::WaitForSubmitSlot() {
m_impl->GetFlipQueue().WaitForSubmitSlot();
void VideoOutDriver::WaitForSubmitSlot(int handle) {
auto* cfg = m_impl->Get(handle);
EXIT_IF(cfg == nullptr);
m_impl->GetFlipQueue().WaitForSubmitSlot(*cfg);
}
void VideoOutDriver::WaitFlipDone(int handle, int index) {
@@ -1540,7 +1678,9 @@ KYTY_SYSV_ABI int VideoOutIsFlipPending(int handle) {
VideoOutFlipStatus status {};
DriverState().GetFlipQueue().GetFlipStatus(*ctx, status);
LOGF("\t flipPendingNum = %d\n", status.flipPendingNum);
if (Config::GraphicsDebugDumpEnabled()) {
LOGF("\t flipPendingNum = %d\n", status.flipPendingNum);
}
return status.flipPendingNum;
}
@@ -1654,8 +1794,12 @@ KYTY_SYSV_ABI int VideoOutGetOutputStatus(int handle, VideoOutOutputStatus* stat
return VIDEO_OUT_ERROR_INVALID_HANDLE;
}
int32_t attribute3 = 0;
Loader::SystemContentParamSfoGetInt("ATTRIBUTE3", &attribute3);
ctx->mutex.Lock();
status->resolution = (ctx->width >= 3840 || ctx->height >= 2160 ? 2u : 1u);
// Primary output reports 4K unless param.json Video-out Info enables resolution detection.
status->resolution =
((attribute3 & 4) != 0 && ctx->width < 3840 && ctx->height < 2160 ? 1u : 2u);
status->dynamicRange = 1;
status->refreshRate =
(ctx->output_mode == VIDEO_OUT_OUTPUT_MODE_119_88HZ || Config::GetVblankFrequency() >= 119
@@ -1670,6 +1814,38 @@ KYTY_SYSV_ABI int VideoOutGetOutputStatus(int handle, VideoOutOutputStatus* stat
return OK;
}
KYTY_SYSV_ABI int VideoOutAllowOutputResolutionWqhdDetection(int handle) {
if (!DriverState().IsOpened(handle)) {
return VIDEO_OUT_ERROR_INVALID_HANDLE;
}
return OK;
}
KYTY_SYSV_ABI int VideoOutVrrPegToFixedRate(int handle, uint64_t arg1, uint64_t arg2) {
PRINT_NAME();
static std::atomic_bool logged {false};
if (!logged.exchange(true, std::memory_order_relaxed)) {
LOGF("\t handle = %d\n"
"\t arg1 = 0x%016" PRIx64 "\n"
"\t arg2 = 0x%016" PRIx64 "\n",
handle, arg1, arg2);
}
return OK;
}
KYTY_SYSV_ABI int VideoOutVrrUnpegFromFixedRate(int handle) {
PRINT_NAME();
static std::atomic_bool logged {false};
if (!logged.exchange(true, std::memory_order_relaxed)) {
LOGF("\t handle = %d\n", handle);
}
return OK;
}
static int ValidateOutputConfig(int handle, uint64_t mode, const VideoOutOutputOptions* options,
void* reserved_ptr, uint64_t reserved) {
if (!DriverState().IsOpened(handle)) {
+7 -1
View File
@@ -37,7 +37,7 @@ public:
void PrepareFlip(uint64_t request_id, Graphics::CommandBuffer& buffer);
void CompleteFlip(uint64_t request_id);
void SubmitFlipPreparation(uint64_t request_id);
void WaitForSubmitSlot();
void WaitForSubmitSlot(int handle);
void WaitFlipDone(int handle, int index);
[[nodiscard]] Impl& State() noexcept;
@@ -52,6 +52,9 @@ void VideoOutShutdown();
KYTY_SYSV_ABI int VideoOutOpen(int user_id, int bus_type, int index, const void* param);
KYTY_SYSV_ABI int VideoOutClose(int handle);
KYTY_SYSV_ABI int VideoOutSetFlipMaster(int slave_handle, int master_handle);
KYTY_SYSV_ABI void VideoOutAddBufferAttributeOption(VideoOutBufferAttribute2* attribute,
uint64_t option);
KYTY_SYSV_ABI void VideoOutSetBufferAttribute2(VideoOutBufferAttribute2* attribute,
uint64_t pixel_format, uint32_t tiling_mode,
uint32_t width, uint32_t height, uint64_t option,
@@ -87,6 +90,9 @@ KYTY_SYSV_ABI int VideoOutGetEventData(const LibKernel::EventQueue::KernelEvent*
KYTY_SYSV_ABI int VideoOutGetEventCount(const LibKernel::EventQueue::KernelEvent* ev);
KYTY_SYSV_ABI int VideoOutWaitVblank(int handle);
KYTY_SYSV_ABI int VideoOutGetOutputStatus(int handle, VideoOutOutputStatus* status);
KYTY_SYSV_ABI int VideoOutAllowOutputResolutionWqhdDetection(int handle);
KYTY_SYSV_ABI int VideoOutVrrPegToFixedRate(int handle, uint64_t arg1, uint64_t arg2);
KYTY_SYSV_ABI int VideoOutVrrUnpegFromFixedRate(int handle);
KYTY_SYSV_ABI int VideoOutInitializeOutputOptions(VideoOutOutputOptions* options);
KYTY_SYSV_ABI int VideoOutIsOutputSupported(int handle, uint64_t mode,
const VideoOutOutputOptions* options,
+132 -65
View File
@@ -1,12 +1,7 @@
#include "graphics/presentation/window/hostInput.h"
#include "SDL_error.h"
#include "SDL_events.h"
#include "SDL_keyboard.h"
#include "SDL_keycode.h"
#include "SDL_mouse.h"
#include "SDL_timer.h"
#include "SDL_video.h"
#include <SDL3/SDL.h>
#include "common/assert.h"
#include "common/emulatorConfig.h"
#include "common/logging/log.h"
@@ -30,6 +25,8 @@ struct ControlInfo {
Controller::Axis axis = Controller::Axis::AxisMax;
bool positive = false;
float touch_x = 0.0f;
Controller::Setting setting {};
};
static constexpr std::array CONTROL_INFO = {
@@ -60,6 +57,10 @@ static constexpr std::array CONTROL_INFO = {
ControlInfo {"RightStickRight", 0, Controller::Axis::RightX, true},
ControlInfo {"RightStickUp", 0, Controller::Axis::RightY, false},
ControlInfo {"RightStickDown", 0, Controller::Axis::RightY, true},
ControlInfo {.name = "SpeakerVolume", .setting = Controller::Setting::SpeakerVolume},
ControlInfo {.name = "VibrationIntensity", .setting = Controller::Setting::VibrationIntensity},
ControlInfo {.name = "TriggerEffectIntensity",
.setting = Controller::Setting::TriggerEffectIntensity},
};
constexpr std::size_t INVALID_CONTROL = CONTROL_INFO.size();
@@ -71,6 +72,7 @@ struct Binding {
};
constexpr int MOUSE_POLL_INTERVAL_MS = 33;
constexpr uint64_t CURSOR_IDLE_HIDE_MS = 2000;
constexpr std::string_view MOUSE_SENSITIVITY = "MouseSensitivity=";
struct MouseJoystickState {
@@ -80,6 +82,8 @@ struct MouseJoystickState {
};
MouseJoystickState g_mouse;
SDL_Window* g_mouse_window = nullptr;
uint64_t g_cursor_hide_at = 0;
std::size_t ControlFromName(std::string_view name) {
const auto info = std::find_if(CONTROL_INFO.begin(), CONTROL_INFO.end(),
@@ -141,9 +145,8 @@ public:
}
}
const bool reserved = binding.key == SDLK_ESCAPE || binding.key == SDLK_SPACE ||
binding.key == SDLK_F1 || binding.key == SDLK_F7 ||
binding.key == SDLK_F11;
const bool reserved = binding.key == SDLK_ESCAPE || binding.key == SDLK_F1 ||
binding.key == SDLK_F7 || binding.key == SDLK_F11;
if (binding.control == INVALID_CONTROL || reserved ||
(binding.key == SDLK_UNKNOWN && binding.mouse_button == 0)) {
EXIT("Invalid input mapping: %s\n", value.c_str());
@@ -214,14 +217,14 @@ uint32_t DefaultKeyboardButton(int key_code) {
case SDLK_LEFT: return Controller::PAD_BUTTON_LEFT;
case SDLK_DOWN: return Controller::PAD_BUTTON_DOWN;
case SDLK_RIGHT: return Controller::PAD_BUTTON_RIGHT;
case SDLK_j: return Controller::PAD_BUTTON_CROSS;
case SDLK_i: return Controller::PAD_BUTTON_TRIANGLE;
case SDLK_k: return Controller::PAD_BUTTON_SQUARE;
case SDLK_l: return Controller::PAD_BUTTON_CIRCLE;
case SDLK_q: return Controller::PAD_BUTTON_L1;
case SDLK_e: return Controller::PAD_BUTTON_R1;
case SDLK_z: return Controller::PAD_BUTTON_L2;
case SDLK_c: return Controller::PAD_BUTTON_R2;
case SDLK_J: return Controller::PAD_BUTTON_CROSS;
case SDLK_I: return Controller::PAD_BUTTON_TRIANGLE;
case SDLK_K: return Controller::PAD_BUTTON_SQUARE;
case SDLK_L: return Controller::PAD_BUTTON_CIRCLE;
case SDLK_Q: return Controller::PAD_BUTTON_L1;
case SDLK_E: return Controller::PAD_BUTTON_R1;
case SDLK_Z: return Controller::PAD_BUTTON_L2;
case SDLK_C: return Controller::PAD_BUTTON_R2;
case SDLK_LSHIFT: return Controller::PAD_BUTTON_L3;
case SDLK_LCTRL: return Controller::PAD_BUTTON_R3;
case SDLK_RETURN:
@@ -247,6 +250,12 @@ void SetStickAxis(Controller::Axis axis, bool negative, bool positive) {
Controller::SetAxis(Controller::HOST_INPUT_CONTROLLER_ID, axis, value);
}
void CycleSettingOnPress(Controller::Setting setting, bool down) {
if (down) {
Controller::CycleSetting(setting);
}
}
void SetControl(std::size_t control, bool down) {
if (control == INVALID_CONTROL) {
return;
@@ -261,6 +270,10 @@ void SetControl(std::size_t control, bool down) {
SetButton(info.button, down);
return;
}
if (info.axis == Controller::Axis::AxisMax) {
CycleSettingOnPress(info.setting, down);
return;
}
struct AxisKeys {
bool negative = false;
@@ -285,35 +298,38 @@ void DefaultKeyboardInput(int key_code, bool down) {
switch (NormalizeKey(static_cast<SDL_Keycode>(key_code))) {
case SDLK_BACKSPACE: SetTouchPad(0.25f, down); return;
case SDLK_TAB: SetTouchPad(0.75f, down); return;
case SDLK_a:
case SDLK_1: CycleSettingOnPress(Controller::Setting::SpeakerVolume, down); return;
case SDLK_2: CycleSettingOnPress(Controller::Setting::VibrationIntensity, down); return;
case SDLK_3: CycleSettingOnPress(Controller::Setting::TriggerEffectIntensity, down); return;
case SDLK_A:
left.left = down;
SetStickAxis(Controller::Axis::LeftX, left.left, left.right);
return;
case SDLK_d:
case SDLK_D:
left.right = down;
SetStickAxis(Controller::Axis::LeftX, left.left, left.right);
return;
case SDLK_w:
case SDLK_W:
left.up = down;
SetStickAxis(Controller::Axis::LeftY, left.up, left.down);
return;
case SDLK_s:
case SDLK_S:
left.down = down;
SetStickAxis(Controller::Axis::LeftY, left.up, left.down);
return;
case SDLK_f:
case SDLK_F:
right.left = down;
SetStickAxis(Controller::Axis::RightX, right.left, right.right);
return;
case SDLK_h:
case SDLK_H:
right.right = down;
SetStickAxis(Controller::Axis::RightX, right.left, right.right);
return;
case SDLK_t:
case SDLK_T:
right.up = down;
SetStickAxis(Controller::Axis::RightY, right.up, right.down);
return;
case SDLK_g:
case SDLK_G:
right.down = down;
SetStickAxis(Controller::Axis::RightY, right.up, right.down);
return;
@@ -321,11 +337,11 @@ void DefaultKeyboardInput(int key_code, bool down) {
}
}
void MouseToJoystick(int delta_x, int delta_y) {
void MouseToJoystick(float delta_x, float delta_y) {
const double distance = std::hypot(delta_x, delta_y);
const double scale =
std::clamp(distance * GetInputMap().MouseSensitivity() + 16.0, 64.0, 128.0) / distance;
const auto map_axis = [scale](int delta) {
const auto map_axis = [scale](float delta) {
return std::clamp(128 + static_cast<int>(std::lround(delta * scale)), 0, 255);
};
Controller::SetRightStick(Controller::HOST_INPUT_CONTROLLER_ID, map_axis(delta_x),
@@ -341,16 +357,69 @@ void CenterMouseStick() {
}
bool SetRelativeMouseMode(bool enabled) {
if (SDL_SetRelativeMouseMode(enabled ? SDL_TRUE : SDL_FALSE) == 0) {
if (SDL_SetWindowRelativeMouseMode(g_mouse_window, enabled)) {
return true;
}
LOGF("Mouse-to-joystick relative mode failed: %s\n", SDL_GetError());
return false;
}
int PollMouse(uint64_t now_ms) {
if (now_ms < g_mouse.next_poll) {
return static_cast<int>(g_mouse.next_poll - now_ms);
}
g_mouse.next_poll = now_ms + MOUSE_POLL_INTERVAL_MS;
float delta_x = 0;
float delta_y = 0;
SDL_GetRelativeMouseState(&delta_x, &delta_y);
if (delta_x == 0 && delta_y == 0) {
CenterMouseStick();
} else {
MouseToJoystick(delta_x, delta_y);
g_mouse.output = true;
}
return MOUSE_POLL_INTERVAL_MS;
}
bool IsCursorActivity(const SDL_Event& event) {
SDL_MouseID which;
switch (event.type) {
case SDL_EVENT_MOUSE_MOTION:
if (event.motion.xrel == 0.0f && event.motion.yrel == 0.0f) {
return false;
}
which = event.motion.which;
break;
case SDL_EVENT_MOUSE_BUTTON_DOWN:
case SDL_EVENT_MOUSE_BUTTON_UP: which = event.button.which; break;
case SDL_EVENT_MOUSE_WHEEL: which = event.wheel.which; break;
default: return false;
}
return which != SDL_TOUCH_MOUSEID && which != SDL_PEN_MOUSEID;
}
} // namespace
void HostInputInit() {
void HostInputInit(SDL_Window* window) {
GetInputMap();
g_mouse_window = window;
if (Config::HideCursorEnabled()) {
g_cursor_hide_at = SDL_GetTicks() + CURSOR_IDLE_HIDE_MS;
}
}
void HostInputShutdown() {
if (g_mouse.enabled) {
SetRelativeMouseMode(false);
CenterMouseStick();
g_mouse = {};
}
g_mouse_window = nullptr;
g_cursor_hide_at = 0;
if (Config::HideCursorEnabled()) {
SDL_ShowCursor();
}
}
void HostInputKey(int key_code, bool down) {
@@ -381,51 +450,49 @@ void HostInputToggleMouseToJoystick() {
if (!SetRelativeMouseMode(true)) {
return;
}
int ignored_x = 0;
int ignored_y = 0;
SDL_GetRelativeMouseState(&ignored_x, &ignored_y);
g_mouse.enabled = true;
g_mouse.next_poll = SDL_GetTicks64() + MOUSE_POLL_INTERVAL_MS;
g_mouse.enabled = true;
LOGF("Mouse to right stick: enabled (F7 to release)\n");
}
int PollMouse(uint64_t now_ms) {
if (now_ms < g_mouse.next_poll) {
return static_cast<int>(g_mouse.next_poll - now_ms);
}
g_mouse.next_poll = now_ms + MOUSE_POLL_INTERVAL_MS;
int delta_x = 0;
int delta_y = 0;
SDL_GetRelativeMouseState(&delta_x, &delta_y);
if (delta_x == 0 && delta_y == 0) {
CenterMouseStick();
return MOUSE_POLL_INTERVAL_MS;
}
MouseToJoystick(delta_x, delta_y);
g_mouse.output = true;
return MOUSE_POLL_INTERVAL_MS;
}
bool HostInputWaitEvent(SDL_Event* event) {
if (!g_mouse.enabled || SDL_GetKeyboardFocus() == nullptr) {
int timeout = -1;
if (!g_mouse.enabled || SDL_GetKeyboardFocus() != g_mouse_window) {
g_mouse.next_poll = 0;
CenterMouseStick();
if (SDL_WaitEvent(event) == 0) {
EXIT("%s\n", SDL_GetError());
} else {
if (g_mouse.next_poll == 0) {
SDL_GetRelativeMouseState(nullptr, nullptr);
g_mouse.next_poll = SDL_GetTicks() + MOUSE_POLL_INTERVAL_MS;
}
return true;
timeout = PollMouse(SDL_GetTicks());
}
const int timeout_ms = PollMouse(SDL_GetTicks64());
SDL_ClearError();
if (SDL_WaitEventTimeout(event, timeout_ms) != 0) {
return true;
if (g_cursor_hide_at != 0) {
const auto now_ms = SDL_GetTicks();
const int cursor_timeout =
now_ms < g_cursor_hide_at ? static_cast<int>(g_cursor_hide_at - now_ms) : 0;
timeout = timeout < 0 ? cursor_timeout : std::min(timeout, cursor_timeout);
}
if (SDL_GetError()[0] != '\0') {
EXIT("%s\n", SDL_GetError());
const bool has_event = SDL_WaitEventTimeout(event, timeout);
if (Config::HideCursorEnabled()) {
const auto now_ms = SDL_GetTicks();
if (has_event && !g_mouse.enabled && IsCursorActivity(*event) &&
SDL_GetWindowFromEvent(event) == g_mouse_window) {
SDL_ShowCursor();
g_cursor_hide_at = now_ms + CURSOR_IDLE_HIDE_MS;
} else if (g_cursor_hide_at != 0 && now_ms >= g_cursor_hide_at) {
SDL_HideCursor();
g_cursor_hide_at = 0;
}
}
return false;
if (has_event && event->type == SDL_EVENT_WINDOW_FOCUS_LOST &&
event->window.windowID == SDL_GetWindowID(g_mouse_window)) {
g_mouse.next_poll = 0;
CenterMouseStick();
}
return has_event;
}
} // namespace Libs::Graphics
+3 -1
View File
@@ -3,11 +3,13 @@
#include <cstdint>
struct SDL_Window;
union SDL_Event;
namespace Libs::Graphics {
void HostInputInit();
void HostInputInit(SDL_Window* window);
void HostInputShutdown();
void HostInputKey(int key_code, bool down);
void HostInputMouseButton(uint8_t mouse_button, bool down);
void HostInputToggleMouseToJoystick();
+366 -158
View File
@@ -4,6 +4,8 @@
#include "common/logging/log.h"
#include "common/profiler.h"
#include "common/threads.h"
#include "gpu_blit_shaders/gpu_blit_fs_triangle_spv.h"
#include "gpu_blit_shaders/gpu_video_out_overlay_spv.h"
#include "graphics/host_gpu/graphicContext.h"
#include "graphics/host_gpu/renderer/render.h"
#include "graphics/host_gpu/renderer/renderContext.h"
@@ -14,7 +16,7 @@
#include "graphics/presentation/window/windowInternal.h"
#include <algorithm>
#include <deque>
#include <array>
#include <limits>
#include <memory>
#include <vector>
@@ -25,10 +27,10 @@
namespace Libs::Graphics {
struct Presenter::Frame {
VulkanImage image;
uint64_t present_tick = 0;
bool busy = false;
bool reusing_last = false;
VulkanImage image;
vk::ImageView view = nullptr;
uint64_t present_tick = 0;
bool busy = false;
void Configure(GraphicContext& graphics, vk::Extent2D extent, vk::Format format);
void Transit(vk::CommandBuffer command, vk::ImageLayout layout, vk::AccessFlags2 access);
@@ -43,6 +45,7 @@ public:
~FramePool() {
m_scheduler.Wait(m_scheduler.CurrentTick() - 1);
for (auto& frame: m_frames) {
m_window.graphic_ctx.device.destroyImageView(frame->view, nullptr);
if (frame->image.image != nullptr) {
m_window.graphic_ctx.DeleteImage(frame->image);
}
@@ -60,6 +63,7 @@ public:
}
m_format = format;
m_frames.reserve(count);
m_free.reserve(count);
for (uint32_t i = 0; i < count; i++) {
auto frame = std::make_unique<Presenter::Frame>();
m_free.push_back(frame.get());
@@ -83,60 +87,45 @@ public:
return m_format;
}
Presenter::Frame* Acquire() {
Presenter::Frame* Acquire(vk::Extent2D extent, vk::Format format) {
m_mutex.Lock();
if (m_frames.empty()) {
EXIT("prepared-frame pool was used before swapchain initialization\n");
}
while (m_free.empty()) {
m_available.Wait(&m_mutex);
// A synchronized flip may need more frames than the swapchain has images.
// Queue capacity bounds growth; waiting here can prevent its master from recording.
if (m_free.empty()) {
auto frame = std::make_unique<Presenter::Frame>();
m_free.push_back(frame.get());
m_frames.push_back(std::move(frame));
}
auto* frame = m_free.front();
m_free.pop_front();
const auto compatible = std::ranges::find_if(m_free, [&](const auto* frame) {
return frame->image.extent.width == extent.width &&
frame->image.extent.height == extent.height && frame->image.format == format;
});
auto* frame = compatible == m_free.end() ? m_free.back() : *compatible;
if (compatible != m_free.end()) {
*compatible = m_free.back();
}
m_free.pop_back();
if (frame->busy) {
EXIT("prepared-frame pool returned an invalid frame\n");
}
if (m_last_frame == frame) {
m_last_frame = nullptr;
}
frame->busy = true;
frame->reusing_last = false;
frame->busy = true;
m_mutex.Unlock();
WaitForFrame(*frame);
m_scheduler.Wait(frame->present_tick);
return frame;
}
Presenter::Frame* AcquireLast() {
m_mutex.Lock();
auto* frame = m_last_frame;
if (frame == nullptr) {
m_mutex.Unlock();
return nullptr;
}
auto free = std::find(m_free.begin(), m_free.end(), frame);
if (free == m_free.end() || frame->busy) {
m_mutex.Unlock();
EXIT("last submitted frame is not available for reuse\n");
}
m_free.erase(free);
m_last_frame = nullptr;
frame->busy = true;
frame->reusing_last = true;
m_mutex.Unlock();
WaitForFrame(*frame);
return frame;
}
void ValidateForPresent(Presenter::Frame* frame, bool reuse) {
void ValidateForPresent(Presenter::Frame* frame) {
Common::LockGuard lock(m_mutex);
if (frame == nullptr || !frame->busy || frame->reusing_last != reuse) {
if (frame == nullptr || !frame->busy) {
EXIT("prepared frame has invalid presentation ownership\n");
}
}
void Release(Presenter::Frame* frame, bool make_last = false) {
void Release(Presenter::Frame* frame) {
if (frame == nullptr) {
EXIT("cannot release a null prepared frame\n");
}
@@ -144,26 +133,17 @@ public:
if (!frame->busy) {
EXIT("prepared frame was released twice\n");
}
frame->busy = false;
frame->reusing_last = false;
if (make_last) {
m_last_frame = frame;
}
frame->busy = false;
m_free.push_back(frame);
m_available.Signal();
}
private:
void WaitForFrame(Presenter::Frame& frame) { m_scheduler.Wait(frame.present_tick); }
WindowContext& m_window;
CommandScheduler& m_scheduler;
Common::Mutex m_mutex;
Common::CondVar m_available;
std::vector<std::unique_ptr<Presenter::Frame>> m_frames;
std::deque<Presenter::Frame*> m_free;
Presenter::Frame* m_last_frame = nullptr;
vk::Format m_format = vk::Format::eUndefined;
std::vector<Presenter::Frame*> m_free;
vk::Format m_format = vk::Format::eUndefined;
};
void Presenter::Frame::Configure(GraphicContext& graphics, vk::Extent2D extent, vk::Format format) {
@@ -171,22 +151,26 @@ void Presenter::Frame::Configure(GraphicContext& graphics, vk::Extent2D extent,
EXIT("unsupported prepared frame, extent=%ux%u format=%d\n", extent.width, extent.height,
static_cast<int>(format));
}
const auto features = graphics.GetFormatProperties(format).optimalTilingFeatures;
const auto required =
vk::FormatFeatureFlagBits::eBlitSrc | vk::FormatFeatureFlagBits::eSampledImageFilterLinear |
vk::FormatFeatureFlagBits::eTransferSrc | vk::FormatFeatureFlagBits::eTransferDst;
if ((features & required) != required) {
EXIT("prepared presentation format lacks optimal blit support: format=%d features=0x%x\n",
static_cast<int>(format), static_cast<vk::FormatFeatureFlags::MaskType>(features));
}
auto& dst = image;
const bool compatible = dst.image != nullptr && dst.extent.width == extent.width &&
dst.extent.height == extent.height && dst.format == format;
if (compatible) {
return;
}
const auto features = graphics.GetFormatProperties(format).optimalTilingFeatures;
const auto required =
vk::FormatFeatureFlagBits::eBlitSrc | vk::FormatFeatureFlagBits::eSampledImageFilterLinear |
vk::FormatFeatureFlagBits::eSampledImage | vk::FormatFeatureFlagBits::eTransferSrc |
vk::FormatFeatureFlagBits::eTransferDst;
if ((features & required) != required) {
EXIT("prepared presentation format lacks optimal blit support: format=%d features=0x%x\n",
static_cast<int>(format), static_cast<vk::FormatFeatureFlags::MaskType>(features));
}
if (dst.image != nullptr) {
graphics.device.destroyImageView(view, nullptr);
view = nullptr;
graphics.DeleteImage(dst);
}
@@ -199,7 +183,8 @@ void Presenter::Frame::Configure(GraphicContext& graphics, vk::Extent2D extent,
create.format = format;
create.tiling = vk::ImageTiling::eOptimal;
create.initialLayout = vk::ImageLayout::eUndefined;
create.usage = vk::ImageUsageFlagBits::eTransferSrc | vk::ImageUsageFlagBits::eTransferDst;
create.usage = vk::ImageUsageFlagBits::eTransferSrc | vk::ImageUsageFlagBits::eTransferDst |
vk::ImageUsageFlagBits::eSampled;
create.sharingMode = vk::SharingMode::eExclusive;
create.samples = vk::SampleCountFlagBits::e1;
if (!graphics.CreateImage(create, dst)) {
@@ -257,7 +242,6 @@ void Presenter::Frame::CopyFrom(CommandBuffer& command_buffer, Image& source) {
EXIT_IF(copy.srcSubresource.layerCount != copy.dstSubresource.layerCount);
command.copyImage(source.backing.image, vk::ImageLayout::eTransferSrcOptimal, image.image,
vk::ImageLayout::eTransferDstOptimal, copy);
Transit(command, vk::ImageLayout::eTransferSrcOptimal, vk::AccessFlagBits2::eTransferRead);
}
void Presenter::Frame::Clear(CommandBuffer& command_buffer, const vk::ClearColorValue& color) {
@@ -266,7 +250,6 @@ void Presenter::Frame::Clear(CommandBuffer& command_buffer, const vk::ClearColor
Transit(command, vk::ImageLayout::eTransferDstOptimal, vk::AccessFlagBits2::eTransferWrite);
const vk::ImageSubresourceRange range {vk::ImageAspectFlagBits::eColor, 0, 1, 0, 1};
command.clearColorImage(image.image, vk::ImageLayout::eTransferDstOptimal, &color, 1, &range);
Transit(command, vk::ImageLayout::eTransferSrcOptimal, vk::AccessFlagBits2::eTransferRead);
}
class Swapchain final {
@@ -279,11 +262,12 @@ public:
void Create();
void Recreate(bool surface_lost = false);
[[nodiscard]] bool NeedsResize() const;
[[nodiscard]] Status AcquireNextImage();
[[nodiscard]] bool PrepareSystemOverlay();
void RecordPresentCommands(CommandBuffer& command, VulkanImage& source,
bool draw_system_overlay);
uint64_t Submit(CommandScheduler& scheduler);
void RecordPresentCommands(CommandBuffer& command, Presenter::Frame* source,
const Presenter::Layer& overlay, bool draw_system_overlay);
uint64_t Submit(CommandScheduler& scheduler);
[[nodiscard]] Status Present();
[[nodiscard]] uint32_t ImageCount() const noexcept {
@@ -293,18 +277,25 @@ public:
private:
void Destroy();
void DrawOverlay(vk::CommandBuffer command, const Presenter::Layer& layer);
WindowContext& m_window;
vk::SwapchainKHR m_handle = nullptr;
vk::Format m_format = vk::Format::eUndefined;
vk::Extent2D m_extent {};
std::vector<vk::Image> m_images;
std::vector<vk::ImageView> m_image_views;
std::vector<vk::Semaphore> m_image_acquired;
std::vector<vk::Semaphore> m_render_complete;
WindowContext& m_window;
vk::SwapchainKHR m_handle = nullptr;
vk::Format m_format = vk::Format::eUndefined;
vk::Extent2D m_extent {};
// Drawable pixel size observed when this swapchain was created.
vk::Extent2D m_window_extent {};
std::vector<vk::Image> m_images;
std::vector<vk::ImageView> m_image_views;
std::vector<vk::Semaphore> m_image_acquired;
std::vector<vk::Semaphore> m_render_complete;
std::unique_ptr<SystemOverlay> m_system_overlay;
uint32_t m_image_index = static_cast<uint32_t>(-1);
uint32_t m_frame_index = 0;
vk::DescriptorSetLayout m_overlay_descriptors = nullptr;
vk::PipelineLayout m_overlay_layout = nullptr;
vk::Pipeline m_overlay_pipeline = nullptr;
vk::Sampler m_overlay_sampler = nullptr;
uint32_t m_image_index = static_cast<uint32_t>(-1);
uint32_t m_frame_index = 0;
};
struct Presenter::Impl {
@@ -343,12 +334,15 @@ struct Presenter::Impl {
cache.UpdateImage(image_id);
return image;
}
void Present();
RenderContext& renderer;
WindowContext& window;
Swapchain swapchain;
CommandScheduler present_scheduler;
FramePool frames;
Common::Mutex present_mutex;
std::array<Layer, 2> layers {};
std::atomic<uint64_t> presented_overlay_revision {0};
};
@@ -357,19 +351,25 @@ void Swapchain::Create() {
EXIT_IF(graphics.device == nullptr);
EXIT_IF(m_window.surface == nullptr);
Common::LockGuard lock(m_window.mutex);
EXIT_IF(graphics.screen_width == 0);
EXIT_IF(graphics.screen_height == 0);
const auto& surface = m_window.surface_capabilities;
// Capture the size before querying surface capabilities. If the window changes
// during creation, the next presentation will detect the newer size.
{
Common::LockGuard lock(m_window.mutex);
m_window_extent = {graphics.screen_width, graphics.screen_height};
}
EXIT_IF(m_window_extent.width == 0);
EXIT_IF(m_window_extent.height == 0);
m_window.RefreshSurfaceCapabilities();
const auto& surface = m_window.surface_capabilities;
EXIT_NOT_IMPLEMENTED(surface.formats.empty());
m_extent = surface.capabilities.currentExtent;
if (m_extent.width == std::numeric_limits<uint32_t>::max()) {
m_extent.width =
std::clamp(graphics.screen_width, surface.capabilities.minImageExtent.width,
std::clamp(m_window_extent.width, surface.capabilities.minImageExtent.width,
surface.capabilities.maxImageExtent.width);
m_extent.height =
std::clamp(graphics.screen_height, surface.capabilities.minImageExtent.height,
std::clamp(m_window_extent.height, surface.capabilities.minImageExtent.height,
surface.capabilities.maxImageExtent.height);
}
uint32_t image_count = surface.capabilities.minImageCount + 1;
@@ -432,7 +432,7 @@ void Swapchain::Create() {
LOGF("warning: requested present mode is unavailable; falling back to Fifo\n");
create_info.presentMode = vk::PresentModeKHR::eFifo;
}
create_info.clipped = VK_TRUE;
create_info.clipped = VK_TRUE;
RequireVulkanSuccess(graphics.device.createSwapchainKHR(&create_info, nullptr, &m_handle),
"vkCreateSwapchainKHR");
EXIT_IF(m_handle == nullptr);
@@ -495,6 +495,14 @@ void Swapchain::Destroy() {
if (m_system_overlay != nullptr) {
m_system_overlay->ReleaseVulkan();
}
graphics.device.destroyPipeline(m_overlay_pipeline, nullptr);
graphics.device.destroyPipelineLayout(m_overlay_layout, nullptr);
graphics.device.destroyDescriptorSetLayout(m_overlay_descriptors, nullptr);
graphics.device.destroySampler(m_overlay_sampler, nullptr);
m_overlay_pipeline = nullptr;
m_overlay_layout = nullptr;
m_overlay_descriptors = nullptr;
m_overlay_sampler = nullptr;
for (const auto semaphore: m_image_acquired) {
if (semaphore != nullptr) {
@@ -515,11 +523,12 @@ void Swapchain::Destroy() {
graphics.device.destroySwapchainKHR(m_handle, nullptr);
}
m_handle = nullptr;
m_format = vk::Format::eUndefined;
m_extent = {};
m_image_index = static_cast<uint32_t>(-1);
m_frame_index = 0;
m_handle = nullptr;
m_format = vk::Format::eUndefined;
m_extent = {};
m_window_extent = {};
m_image_index = static_cast<uint32_t>(-1);
m_frame_index = 0;
m_images.clear();
m_image_views.clear();
m_image_acquired.clear();
@@ -532,15 +541,22 @@ void Swapchain::Recreate(bool surface_lost) {
#if defined(__APPLE__)
// Surface recreation goes through SDL_Vulkan_CreateSurface, which touches the
// window's view/layer and must run on the main thread on macOS.
m_window.RunOnMainThread([this] { m_window.RecreateSurface(); });
EXIT_IF(!SDL_RunOnMainThread(
[](void* window) { static_cast<WindowContext*>(window)->RecreateSurface(); }, &m_window,
true));
#else
m_window.RecreateSurface();
#endif
}
m_window.RefreshSurfaceCapabilities();
Create();
}
bool Swapchain::NeedsResize() const {
Common::LockGuard lock(m_window.mutex);
return m_window_extent.width != m_window.graphic_ctx.screen_width ||
m_window_extent.height != m_window.graphic_ctx.screen_height;
}
Swapchain::Status Swapchain::AcquireNextImage() {
EXIT_IF(m_handle == nullptr || m_frame_index >= m_image_acquired.size());
m_image_index = static_cast<uint32_t>(-1);
@@ -574,14 +590,147 @@ bool Swapchain::PrepareSystemOverlay() {
return m_system_overlay->PrepareFrame(m_extent, m_format, ImageCount());
}
void Swapchain::RecordPresentCommands(CommandBuffer& command, VulkanImage& source,
bool draw_system_overlay) {
if (source.state.layout != vk::ImageLayout::eTransferSrcOptimal) {
EXIT("invalid prepared presentation image, vk_image=%p layout=%d\n",
static_cast<void*>(source.image), static_cast<int>(source.state.layout));
void Swapchain::DrawOverlay(vk::CommandBuffer command, const Presenter::Layer& layer) {
auto device = m_window.graphic_ctx.device;
if (m_overlay_pipeline == nullptr) {
vk::SamplerCreateInfo sampler {};
sampler.magFilter = vk::Filter::eLinear;
sampler.minFilter = vk::Filter::eLinear;
sampler.addressModeU = vk::SamplerAddressMode::eClampToEdge;
sampler.addressModeV = vk::SamplerAddressMode::eClampToEdge;
sampler.addressModeW = vk::SamplerAddressMode::eClampToEdge;
RequireVulkanSuccess(device.createSampler(&sampler, nullptr, &m_overlay_sampler),
"create video-out overlay sampler");
const vk::DescriptorSetLayoutBinding binding {0, vk::DescriptorType::eCombinedImageSampler,
1, vk::ShaderStageFlagBits::eFragment};
vk::DescriptorSetLayoutCreateInfo descriptors {};
descriptors.flags = vk::DescriptorSetLayoutCreateFlagBits::ePushDescriptorKHR;
descriptors.bindingCount = 1;
descriptors.pBindings = &binding;
RequireVulkanSuccess(
device.createDescriptorSetLayout(&descriptors, nullptr, &m_overlay_descriptors),
"create video-out overlay descriptor layout");
const vk::PushConstantRange alpha {vk::ShaderStageFlagBits::eFragment, 0, sizeof(uint32_t)};
vk::PipelineLayoutCreateInfo layout {};
layout.setLayoutCount = 1;
layout.pSetLayouts = &m_overlay_descriptors;
layout.pushConstantRangeCount = 1;
layout.pPushConstantRanges = &alpha;
RequireVulkanSuccess(device.createPipelineLayout(&layout, nullptr, &m_overlay_layout),
"create video-out overlay pipeline layout");
const auto vertex = CompileSPV(GPU_BLIT_FS_TRIANGLE_SPV, device);
const auto fragment = CompileSPV(GPU_VIDEO_OUT_OVERLAY_SPV, device);
std::array<vk::PipelineShaderStageCreateInfo, 2> stages {};
stages[0].stage = vk::ShaderStageFlagBits::eVertex;
stages[0].module = vertex;
stages[0].pName = "main";
stages[1].stage = vk::ShaderStageFlagBits::eFragment;
stages[1].module = fragment;
stages[1].pName = "main";
vk::PipelineVertexInputStateCreateInfo vertex_input {};
vk::PipelineInputAssemblyStateCreateInfo assembly {};
assembly.topology = vk::PrimitiveTopology::eTriangleList;
vk::PipelineViewportStateCreateInfo viewport {};
viewport.viewportCount = 1;
viewport.scissorCount = 1;
vk::PipelineRasterizationStateCreateInfo rasterization {};
rasterization.lineWidth = 1.0f;
vk::PipelineMultisampleStateCreateInfo multisample {};
multisample.rasterizationSamples = vk::SampleCountFlagBits::e1;
vk::PipelineColorBlendAttachmentState attachment {};
attachment.blendEnable = VK_TRUE;
attachment.srcColorBlendFactor = vk::BlendFactor::eOne;
attachment.dstColorBlendFactor = vk::BlendFactor::eOneMinusSrcAlpha;
attachment.srcAlphaBlendFactor = vk::BlendFactor::eOne;
attachment.dstAlphaBlendFactor = vk::BlendFactor::eOneMinusSrcAlpha;
attachment.colorWriteMask = vk::ColorComponentFlagBits::eR |
vk::ColorComponentFlagBits::eG |
vk::ColorComponentFlagBits::eB | vk::ColorComponentFlagBits::eA;
vk::PipelineColorBlendStateCreateInfo blend {};
blend.attachmentCount = 1;
blend.pAttachments = &attachment;
const std::array dynamic_states {vk::DynamicState::eViewport, vk::DynamicState::eScissor};
vk::PipelineDynamicStateCreateInfo dynamic {};
dynamic.dynamicStateCount = static_cast<uint32_t>(dynamic_states.size());
dynamic.pDynamicStates = dynamic_states.data();
vk::PipelineRenderingCreateInfo rendering {};
rendering.colorAttachmentCount = 1;
rendering.pColorAttachmentFormats = &m_format;
vk::GraphicsPipelineCreateInfo create {};
create.pNext = &rendering;
create.stageCount = static_cast<uint32_t>(stages.size());
create.pStages = stages.data();
create.pVertexInputState = &vertex_input;
create.pInputAssemblyState = &assembly;
create.pViewportState = &viewport;
create.pRasterizationState = &rasterization;
create.pMultisampleState = &multisample;
create.pColorBlendState = &blend;
create.pDynamicState = &dynamic;
create.layout = m_overlay_layout;
RequireVulkanSuccess(
device.createGraphicsPipelines(nullptr, 1, &create, nullptr, &m_overlay_pipeline),
"create video-out overlay pipeline");
device.destroyShaderModule(fragment, nullptr);
device.destroyShaderModule(vertex, nullptr);
}
auto& frame = *layer.frame;
if (frame.view == nullptr) {
vk::ImageViewCreateInfo view {};
view.image = frame.image.image;
view.viewType = vk::ImageViewType::e2D;
view.format = frame.image.format;
view.subresourceRange = {vk::ImageAspectFlagBits::eColor, 0, 1, 0, 1};
RequireVulkanSuccess(device.createImageView(&view, nullptr, &frame.view),
"create video-out overlay image view");
}
const vk::DescriptorImageInfo image {m_overlay_sampler, frame.view,
vk::ImageLayout::eShaderReadOnlyOptimal};
vk::WriteDescriptorSet write {};
write.dstBinding = 0;
write.descriptorCount = 1;
write.descriptorType = vk::DescriptorType::eCombinedImageSampler;
write.pImageInfo = &image;
command.pushDescriptorSetKHR(vk::PipelineBindPoint::eGraphics, m_overlay_layout, 0, 1, &write);
const uint32_t premultiplied = layer.premultiplied_alpha;
command.pushConstants(m_overlay_layout, vk::ShaderStageFlagBits::eFragment, 0,
sizeof(premultiplied), &premultiplied);
command.bindPipeline(vk::PipelineBindPoint::eGraphics, m_overlay_pipeline);
vk::RenderingAttachmentInfo attachment {};
attachment.imageView = m_image_views[m_image_index];
attachment.imageLayout = vk::ImageLayout::eColorAttachmentOptimal;
attachment.loadOp = vk::AttachmentLoadOp::eLoad;
attachment.storeOp = vk::AttachmentStoreOp::eStore;
vk::RenderingInfo rendering {};
rendering.renderArea.extent = m_extent;
rendering.layerCount = 1;
rendering.colorAttachmentCount = 1;
rendering.pColorAttachments = &attachment;
command.beginRendering(&rendering);
const vk::Viewport viewport {
0.0f, 0.0f, static_cast<float>(m_extent.width), static_cast<float>(m_extent.height),
0.0f, 1.0f};
const vk::Rect2D scissor {{0, 0}, m_extent};
command.setViewport(0, 1, &viewport);
command.setScissor(0, 1, &scissor);
command.draw(3, 1, 0, 0);
command.endRendering();
}
void Swapchain::RecordPresentCommands(CommandBuffer& command, Presenter::Frame* source,
const Presenter::Layer& overlay, bool draw_system_overlay) {
EXIT_IF(m_image_index >= m_images.size());
auto vk_command = command.Handle();
auto vk_command = command.Handle();
const bool draw_overlay = overlay.frame != nullptr;
const bool draw_attachment = draw_overlay || draw_system_overlay;
if (source != nullptr) {
source->Transit(vk_command, vk::ImageLayout::eTransferSrcOptimal,
vk::AccessFlagBits2::eTransferRead);
}
if (draw_overlay) {
overlay.frame->Transit(vk_command, vk::ImageLayout::eShaderReadOnlyOptimal,
vk::AccessFlagBits2::eShaderRead);
}
vk::ImageMemoryBarrier to_transfer {};
to_transfer.sType = vk::StructureType::eImageMemoryBarrier;
@@ -601,49 +750,68 @@ void Swapchain::RecordPresentCommands(CommandBuffer& command, VulkanImage& sourc
vk::PipelineStageFlagBits::eTransfer, vk::DependencyFlags {}, 0,
nullptr, 0, nullptr, 1, &to_transfer);
vk::ImageBlit region {};
region.srcSubresource.aspectMask = vk::ImageAspectFlagBits::eColor;
region.srcSubresource.mipLevel = 0;
region.srcSubresource.baseArrayLayer = 0;
region.srcSubresource.layerCount = 1;
region.srcOffsets[1].x = static_cast<int>(source.extent.width);
region.srcOffsets[1].y = static_cast<int>(source.extent.height);
region.srcOffsets[1].z = 1;
region.dstSubresource.aspectMask = vk::ImageAspectFlagBits::eColor;
region.dstSubresource.mipLevel = 0;
region.dstSubresource.baseArrayLayer = 0;
region.dstSubresource.layerCount = 1;
region.dstOffsets[1].x = static_cast<int>(m_extent.width);
region.dstOffsets[1].y = static_cast<int>(m_extent.height);
region.dstOffsets[1].z = 1;
vk_command.blitImage(source.image, vk::ImageLayout::eTransferSrcOptimal,
m_images[m_image_index], vk::ImageLayout::eTransferDstOptimal, 1, &region,
vk::Filter::eLinear);
if (source != nullptr) {
vk::ImageBlit region {};
region.srcSubresource.aspectMask = vk::ImageAspectFlagBits::eColor;
region.srcSubresource.mipLevel = 0;
region.srcSubresource.baseArrayLayer = 0;
region.srcSubresource.layerCount = 1;
region.srcOffsets[1].x = static_cast<int>(source->image.extent.width);
region.srcOffsets[1].y = static_cast<int>(source->image.extent.height);
region.srcOffsets[1].z = 1;
region.dstSubresource.aspectMask = vk::ImageAspectFlagBits::eColor;
region.dstSubresource.mipLevel = 0;
region.dstSubresource.baseArrayLayer = 0;
region.dstSubresource.layerCount = 1;
region.dstOffsets[1].x = static_cast<int>(m_extent.width);
region.dstOffsets[1].y = static_cast<int>(m_extent.height);
region.dstOffsets[1].z = 1;
vk_command.blitImage(source->image.image, vk::ImageLayout::eTransferSrcOptimal,
m_images[m_image_index], vk::ImageLayout::eTransferDstOptimal, 1,
&region, vk::Filter::eLinear);
} else {
const vk::ClearColorValue black {};
vk_command.clearColorImage(m_images[m_image_index], vk::ImageLayout::eTransferDstOptimal,
&black, 1, &to_transfer.subresourceRange);
}
vk::ImageMemoryBarrier to_present {};
to_present.sType = vk::StructureType::eImageMemoryBarrier;
to_present.srcAccessMask = vk::AccessFlagBits::eTransferWrite;
to_present.dstAccessMask = draw_system_overlay ? vk::AccessFlagBits::eColorAttachmentRead |
vk::AccessFlagBits::eColorAttachmentWrite
: vk::AccessFlagBits::eMemoryRead;
to_present.oldLayout = vk::ImageLayout::eTransferDstOptimal;
to_present.newLayout = draw_system_overlay ? vk::ImageLayout::eColorAttachmentOptimal
: vk::ImageLayout::ePresentSrcKHR;
to_present.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
to_present.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
to_present.image = m_images[m_image_index];
to_present.sType = vk::StructureType::eImageMemoryBarrier;
to_present.srcAccessMask = vk::AccessFlagBits::eTransferWrite;
to_present.dstAccessMask = draw_attachment ? vk::AccessFlagBits::eColorAttachmentRead |
vk::AccessFlagBits::eColorAttachmentWrite
: vk::AccessFlagBits::eMemoryRead;
to_present.oldLayout = vk::ImageLayout::eTransferDstOptimal;
to_present.newLayout = draw_attachment ? vk::ImageLayout::eColorAttachmentOptimal
: vk::ImageLayout::ePresentSrcKHR;
to_present.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
to_present.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
to_present.image = m_images[m_image_index];
to_present.subresourceRange.aspectMask = vk::ImageAspectFlagBits::eColor;
to_present.subresourceRange.baseMipLevel = 0;
to_present.subresourceRange.levelCount = 1;
to_present.subresourceRange.baseArrayLayer = 0;
to_present.subresourceRange.layerCount = 1;
vk_command.pipelineBarrier(
vk::PipelineStageFlagBits::eTransfer,
draw_system_overlay ? vk::PipelineStageFlagBits::eColorAttachmentOutput
: vk::PipelineStageFlagBits::eAllCommands,
vk::DependencyFlagBits::eByRegion, 0, nullptr, 0, nullptr, 1, &to_present);
if (draw_system_overlay) {
m_system_overlay->Record(vk_command, m_image_views[m_image_index]);
vk_command.pipelineBarrier(vk::PipelineStageFlagBits::eTransfer,
draw_attachment ? vk::PipelineStageFlagBits::eColorAttachmentOutput
: vk::PipelineStageFlagBits::eAllCommands,
vk::DependencyFlagBits::eByRegion, 0, nullptr, 0, nullptr, 1,
&to_present);
if (draw_attachment) {
if (draw_overlay) {
DrawOverlay(vk_command, overlay);
if (draw_system_overlay) {
to_present.srcAccessMask = vk::AccessFlagBits::eColorAttachmentWrite;
to_present.oldLayout = vk::ImageLayout::eColorAttachmentOptimal;
vk_command.pipelineBarrier(vk::PipelineStageFlagBits::eColorAttachmentOutput,
vk::PipelineStageFlagBits::eColorAttachmentOutput,
vk::DependencyFlagBits::eByRegion, 0, nullptr, 0,
nullptr, 1, &to_present);
}
}
if (draw_system_overlay) {
m_system_overlay->Record(vk_command, m_image_views[m_image_index]);
}
to_present.srcAccessMask = vk::AccessFlagBits::eColorAttachmentWrite;
to_present.dstAccessMask = vk::AccessFlagBits::eMemoryRead;
to_present.oldLayout = vk::ImageLayout::eColorAttachmentOptimal;
@@ -703,19 +871,18 @@ Presenter::~Presenter() = default;
Presenter::Frame& Presenter::PrepareFrame(CommandBuffer& buffer, const ImageInfo& info) {
KYTY_PROFILER_FUNCTION();
EXIT_IF(buffer.IsInvalid());
auto* frame = m_impl->frames.Acquire();
Common::LockGuard render_lock(m_impl->renderer.GetMutex());
auto& image = m_impl->ResolveSurface(info);
if (image.backing.format == vk::Format::eUndefined) {
EXIT("unsupported presentation source, image=%p\n", static_cast<const void*>(&image));
}
auto frame_format = info.pixel_format;
switch (frame_format) {
case vk::Format::eR8G8B8A8Srgb: frame_format = vk::Format::eR8G8B8A8Unorm; break;
case vk::Format::eB8G8R8A8Srgb: frame_format = vk::Format::eB8G8R8A8Unorm; break;
default: break;
}
auto* frame = m_impl->frames.Acquire({info.extent.width, info.extent.height}, frame_format);
Common::LockGuard render_lock(m_impl->renderer.GetMutex());
auto& image = m_impl->ResolveSurface(info);
if (image.backing.format == vk::Format::eUndefined) {
EXIT("unsupported presentation source, image=%p\n", static_cast<const void*>(&image));
}
frame->Configure(m_impl->window.graphic_ctx,
{image.backing.extent.width, image.backing.extent.height}, frame_format);
frame->CopyFrom(buffer, image);
@@ -726,7 +893,7 @@ Presenter::Frame& Presenter::PrepareBlankFrame(uint32_t width, uint32_t height,
CommandBuffer* producer) {
KYTY_PROFILER_FUNCTION();
auto format = m_impl->frames.GetFormat();
auto* frame = m_impl->frames.Acquire();
auto* frame = m_impl->frames.Acquire({width, height}, format);
Common::LockGuard render_lock(m_impl->renderer.GetMutex());
frame->Configure(m_impl->window.graphic_ctx, {width, height}, format);
vk::ClearColorValue clear {};
@@ -742,8 +909,13 @@ Presenter::Frame& Presenter::PrepareBlankFrame(uint32_t width, uint32_t height,
return *frame;
}
Presenter::Frame* Presenter::PrepareLastFrame() {
return m_impl->frames.AcquireLast();
bool Presenter::PresentLastFrame() {
Common::LockGuard lock(m_impl->present_mutex);
if (m_impl->layers[0].frame == nullptr && m_impl->layers[1].frame == nullptr) {
return false;
}
m_impl->Present();
return true;
}
bool Presenter::IsGuestPaused() const noexcept {
@@ -760,40 +932,76 @@ RenderContext& Presenter::Renderer() const noexcept {
return m_impl->renderer;
}
void Presenter::Present(Frame& frame, bool reuse) {
void Presenter::Present(Frame& frame) {
const Layer layer {&frame, 0, false};
Present(std::span(&layer, 1));
}
void Presenter::Present(std::span<const Layer> layers) {
Common::LockGuard lock(m_impl->present_mutex);
for (const auto& layer: layers) {
EXIT_IF(layer.bus < 0 || layer.bus >= static_cast<int>(m_impl->layers.size()));
m_impl->frames.ValidateForPresent(layer.frame);
auto& previous = m_impl->layers[layer.bus];
if (previous.frame != nullptr) {
m_impl->frames.Release(previous.frame);
}
previous = layer;
}
m_impl->Present();
}
void Presenter::ClearLayer(int bus) {
if (static_cast<size_t>(bus) >= m_impl->layers.size()) {
return;
}
Common::LockGuard lock(m_impl->present_mutex);
auto& layer = m_impl->layers[bus];
if (layer.frame != nullptr) {
m_impl->frames.Release(layer.frame);
layer = {};
}
}
void Presenter::Impl::Present() {
KYTY_PROFILER_FUNCTION();
m_impl->frames.ValidateForPresent(&frame, reuse);
const auto overlay_visual = GetSystemOverlayVisualState();
auto& swapchain = m_impl->swapchain;
// Some window systems keep presenting an old swapchain after a resize.
if (swapchain.NeedsResize()) {
RecoverSwapchain(Swapchain::Status::Recreate);
}
for (uint32_t attempt = 0; attempt < 2; attempt++) {
auto status = swapchain.AcquireNextImage();
if (status != Swapchain::Status::Success) {
m_impl->RecoverSwapchain(status);
RecoverSwapchain(status);
continue;
}
{
Common::LockGuard render_lock(m_impl->renderer.GetMutex());
auto& command = m_impl->present_scheduler.BeginCommand();
Common::LockGuard render_lock(renderer.GetMutex());
auto& command = present_scheduler.BeginCommand();
const bool draw_system_overlay =
overlay_visual.active && swapchain.PrepareSystemOverlay();
swapchain.RecordPresentCommands(command, frame.image, draw_system_overlay);
frame.present_tick = swapchain.Submit(m_impl->present_scheduler);
swapchain.RecordPresentCommands(command, layers[0].frame, layers[1],
draw_system_overlay);
const auto tick = swapchain.Submit(present_scheduler);
for (const auto& layer: layers) {
if (layer.frame != nullptr) {
layer.frame->present_tick = tick;
}
}
}
status = swapchain.Present();
if (status != Swapchain::Status::Success) {
m_impl->RecoverSwapchain(status);
RecoverSwapchain(status);
continue;
}
m_impl->presented_overlay_revision.store(overlay_visual.revision,
std::memory_order_release);
m_impl->window.UpdateTitle();
m_impl->frames.Release(&frame, true);
presented_overlay_revision.store(overlay_visual.revision, std::memory_order_release);
window.UpdateTitle();
return;
}
LOGF("Vulkan presentation retry exhausted; dropping frame\n");
m_impl->frames.Release(&frame, reuse);
}
void Presenter::Discard(Frame& frame) {
+176 -249
View File
@@ -1,20 +1,6 @@
#include "SDL.h"
#include "SDL_error.h"
#include "SDL_events.h"
#include "SDL_gamecontroller.h"
#include "SDL_hints.h"
#include "SDL_joystick.h"
#include "SDL_keyboard.h"
#include "SDL_keycode.h"
#include "SDL_mouse.h"
#include "SDL_pixels.h"
#include "SDL_rwops.h"
#include "SDL_stdinc.h"
#include "SDL_surface.h"
#include "SDL_thread.h"
#include "SDL_touch.h"
#include "SDL_video.h"
#include "SDL_vulkan.h"
#include <SDL3/SDL.h>
#include <SDL3/SDL_vulkan.h>
#include "common/assert.h"
#include "common/common.h"
#include "common/emulatorConfig.h"
@@ -151,11 +137,13 @@ static uint32_t VulkanFindQueueFamily(vk::PhysicalDevice device, vk::SurfaceKHR
return static_cast<uint32_t>(-1);
}
// On failure out_device is null and out_rejections says why each device was skipped.
// NOLINTNEXTLINE(readability-function-cognitive-complexity)
static void VulkanFindPhysicalDevice(vk::Instance instance, vk::SurfaceKHR surface,
const std::vector<const char*>& device_extensions,
SurfaceCapabilities& out_capabilities,
vk::PhysicalDevice& out_device, uint32_t& out_queue_family) {
vk::PhysicalDevice& out_device, uint32_t& out_queue_family,
std::string& out_rejections) {
EXIT_IF(instance == nullptr);
EXIT_IF(surface == nullptr);
@@ -184,14 +172,22 @@ static void VulkanFindPhysicalDevice(vk::Instance instance, vk::SurfaceKHR surfa
vk::PhysicalDeviceProperties device_properties {};
device.getProperties(&device_properties);
const auto reject = [&](const std::string& reason) {
LOGF("%s\n", reason.c_str());
out_rejections +=
skip_device ? "; " : fmt::format("\n {}: ", device_properties.deviceName.data());
out_rejections += reason;
skip_device = true;
};
LOGF("Vulkan device: %s\n", device_properties.deviceName.data());
if (device_properties.apiVersion < VULKAN_TARGET_API_VERSION) {
LOGF("Vulkan %u.%u is required, but device supports only %u.%u.%u\n",
VK_VERSION_MAJOR(VULKAN_TARGET_API_VERSION),
VK_VERSION_MINOR(VULKAN_TARGET_API_VERSION),
VK_VERSION_MAJOR(device_properties.apiVersion),
VK_VERSION_MINOR(device_properties.apiVersion),
VK_VERSION_PATCH(device_properties.apiVersion));
reject(fmt::format("Vulkan {}.{} is required, but device supports only {}.{}.{}",
VK_VERSION_MAJOR(VULKAN_TARGET_API_VERSION),
VK_VERSION_MINOR(VULKAN_TARGET_API_VERSION),
VK_VERSION_MAJOR(device_properties.apiVersion),
VK_VERSION_MINOR(device_properties.apiVersion),
VK_VERSION_PATCH(device_properties.apiVersion)));
continue;
}
@@ -217,7 +213,7 @@ static void VulkanFindPhysicalDevice(vk::Instance instance, vk::SurfaceKHR surfa
fragment_barycentric.pNext = &depth_clip_control;
features12.pNext = &fragment_barycentric;
#endif
features13.pNext = &features12;
features13.pNext = &features12;
device_features2.pNext = &features13;
device.getFeatures2(&device_features2);
@@ -226,147 +222,82 @@ static void VulkanFindPhysicalDevice(vk::Instance instance, vk::SurfaceKHR surfa
const auto queue_family = VulkanFindQueueFamily(device, surface);
if (queue_family == static_cast<uint32_t>(-1)) {
LOGF("No universal graphics, compute, and presentation queue\n");
skip_device = true;
reject("No universal graphics, compute, and presentation queue");
}
const auto check_feature = [&](vk::Bool32 supported, const char* name,
vk::Bool32 required = VK_TRUE) {
if (required == VK_TRUE && supported != VK_TRUE) {
reject(fmt::format("{} is not supported", name));
}
};
#if defined(__APPLE__)
if (color_write_ext.colorWriteEnable != VK_TRUE) {
LOGF("colorWriteEnable is not supported\n");
#if !defined(__APPLE__)
skip_device = true;
}
#else
check_feature(color_write_ext.colorWriteEnable, "colorWriteEnable");
#endif
}
if (image_view_min_lod.minLod != VK_TRUE) {
LOGF("image view minLod is not supported\n");
skip_device = true;
}
check_feature(image_view_min_lod.minLod, "image view minLod");
if (depth_clip_control.depthClipControl != VK_TRUE) {
LOGF("depthClipControl is not supported\n");
skip_device = true;
}
check_feature(depth_clip_control.depthClipControl, "depthClipControl");
#if defined(__APPLE__)
if (depth_clip_enable.depthClipEnable != VK_TRUE) {
LOGF("depthClipEnable is not supported\n");
#if !defined(__APPLE__)
skip_device = true;
}
#else
check_feature(depth_clip_enable.depthClipEnable, "depthClipEnable");
#endif
}
#if !defined(__APPLE__)
if (fragment_barycentric.fragmentShaderBarycentric != VK_TRUE) {
LOGF("fragmentShaderBarycentric is not supported\n");
skip_device = true;
}
check_feature(device_features2.features.depthClamp, "depthClamp");
check_feature(fragment_barycentric.fragmentShaderBarycentric, "fragmentShaderBarycentric");
#endif
if (required_features12.samplerMirrorClampToEdge == VK_TRUE &&
features12.samplerMirrorClampToEdge != VK_TRUE) {
LOGF("samplerMirrorClampToEdge is not supported\n");
skip_device = true;
}
if (required_features12.timelineSemaphore == VK_TRUE &&
features12.timelineSemaphore != VK_TRUE) {
LOGF("timelineSemaphore is not supported\n");
skip_device = true;
}
if (required_features12.shaderOutputLayer == VK_TRUE &&
features12.shaderOutputLayer != VK_TRUE) {
LOGF("shaderOutputLayer is not supported\n");
skip_device = true;
}
if (required_features12.shaderOutputViewportIndex == VK_TRUE &&
features12.shaderOutputViewportIndex != VK_TRUE) {
LOGF("shaderOutputViewportIndex is not supported\n");
skip_device = true;
}
if (required_features12.bufferDeviceAddress == VK_TRUE &&
features12.bufferDeviceAddress != VK_TRUE) {
LOGF("bufferDeviceAddress is not supported\n");
skip_device = true;
}
if (required_features12.shaderBufferInt64Atomics == VK_TRUE &&
features12.shaderBufferInt64Atomics != VK_TRUE) {
LOGF("shaderBufferInt64Atomics is not supported\n");
skip_device = true;
}
if (features13.robustImageAccess != VK_TRUE) {
LOGF("robustImageAccess is not supported\n");
skip_device = true;
}
if (required_features13.dynamicRendering == VK_TRUE &&
features13.dynamicRendering != VK_TRUE) {
LOGF("dynamicRendering is not supported\n");
skip_device = true;
}
if (required_features13.synchronization2 == VK_TRUE &&
features13.synchronization2 != VK_TRUE) {
LOGF("synchronization2 is not supported\n");
skip_device = true;
}
if (device_features2.features.sampleRateShading != VK_TRUE) {
LOGF("sampleRateShading is not supported\n");
skip_device = true;
}
if (device_features2.features.depthBiasClamp != VK_TRUE) {
LOGF("depthBiasClamp is not supported\n");
skip_device = true;
}
if (device_features2.features.shaderClipDistance != VK_TRUE) {
LOGF("shaderClipDistance is not supported\n");
skip_device = true;
}
if (device_features2.features.shaderCullDistance != VK_TRUE) {
LOGF("shaderCullDistance is not supported\n");
skip_device = true;
}
if (device_features2.features.largePoints != VK_TRUE) {
LOGF("largePoints is not supported\n");
skip_device = true;
}
if (device_features2.features.multiViewport != VK_TRUE) {
LOGF("multiViewport is not supported\n");
skip_device = true;
}
if (device_features2.features.fillModeNonSolid != VK_TRUE) {
LOGF("fillModeNonSolid is not supported\n");
skip_device = true;
}
if (device_features2.features.fragmentStoresAndAtomics != VK_TRUE) {
LOGF("fragmentStoresAndAtomics is not supported\n");
skip_device = true;
}
check_feature(features12.samplerMirrorClampToEdge, "samplerMirrorClampToEdge",
required_features12.samplerMirrorClampToEdge);
check_feature(features12.timelineSemaphore, "timelineSemaphore",
required_features12.timelineSemaphore);
check_feature(features12.shaderOutputLayer, "shaderOutputLayer",
required_features12.shaderOutputLayer);
check_feature(features12.shaderOutputViewportIndex, "shaderOutputViewportIndex",
required_features12.shaderOutputViewportIndex);
check_feature(features12.bufferDeviceAddress, "bufferDeviceAddress",
required_features12.bufferDeviceAddress);
check_feature(features12.shaderBufferInt64Atomics, "shaderBufferInt64Atomics",
required_features12.shaderBufferInt64Atomics);
check_feature(features13.robustImageAccess, "robustImageAccess");
check_feature(features13.dynamicRendering, "dynamicRendering",
required_features13.dynamicRendering);
check_feature(features13.synchronization2, "synchronization2",
required_features13.synchronization2);
check_feature(device_features2.features.sampleRateShading, "sampleRateShading");
check_feature(device_features2.features.depthBiasClamp, "depthBiasClamp");
check_feature(device_features2.features.shaderClipDistance, "shaderClipDistance");
check_feature(device_features2.features.shaderCullDistance, "shaderCullDistance");
check_feature(device_features2.features.largePoints, "largePoints");
check_feature(device_features2.features.multiViewport, "multiViewport");
check_feature(device_features2.features.fillModeNonSolid, "fillModeNonSolid");
check_feature(device_features2.features.fragmentStoresAndAtomics,
"fragmentStoresAndAtomics");
if (device_features2.features.samplerAnisotropy != VK_TRUE) {
LOGF("samplerAnisotropy is not supported\n");
skip_device = true;
}
if (device_features2.features.robustBufferAccess != VK_TRUE) {
LOGF("robustBufferAccess is not supported\n");
skip_device = true;
}
check_feature(device_features2.features.samplerAnisotropy, "samplerAnisotropy");
check_feature(device_features2.features.robustBufferAccess, "robustBufferAccess");
#if defined(__APPLE__)
if (device_features2.features.depthBounds != VK_TRUE) {
LOGF("depthBounds is not supported\n");
#if !defined(__APPLE__)
skip_device = true;
}
#else
check_feature(device_features2.features.depthBounds, "depthBounds");
#endif
}
if (device_features2.features.shaderStorageImageWriteWithoutFormat != VK_TRUE) {
LOGF("shaderStorageImageWriteWithoutFormat is not supported\n");
skip_device = true;
}
check_feature(device_features2.features.shaderStorageImageWriteWithoutFormat,
"shaderStorageImageWriteWithoutFormat");
if (device_features2.features.shaderImageGatherExtended != VK_TRUE) {
LOGF("shaderImageGatherExtended is not supported\n");
skip_device = true;
}
check_feature(device_features2.features.shaderImageGatherExtended,
"shaderImageGatherExtended");
if (device_features2.features.independentBlend != VK_TRUE) {
LOGF("independentBlend is not supported\n");
skip_device = true;
}
if (device_features2.features.tessellationShader != VK_TRUE) {
LOGF("tessellationShader is not supported\n");
skip_device = true;
}
check_feature(device_features2.features.independentBlend, "independentBlend");
check_feature(device_features2.features.tessellationShader, "tessellationShader");
if (!skip_device) {
auto available_extensions = EnumerateVulkan<vk::ExtensionProperties>(
@@ -378,8 +309,7 @@ static void VulkanFindPhysicalDevice(vk::Instance instance, vk::SurfaceKHR surfa
for (const char* ext: device_extensions) {
if (!HasExtension(available_extensions, ext)) {
skip_device = true;
break;
reject(fmt::format("{} is not supported", ext));
}
}
@@ -397,86 +327,46 @@ static void VulkanFindPhysicalDevice(vk::Instance instance, vk::SurfaceKHR surfa
if (!(candidate_capabilities.capabilities.supportedUsageFlags &
vk::ImageUsageFlagBits::eTransferDst)) {
LOGF("Surface cannot be destination of blit\n");
skip_device = true;
reject("Surface cannot be destination of blit");
}
}
if (!skip_device && !CheckFormat(device, vk::Format::eD32Sfloat, true,
vk::FormatFeatureFlagBits::eDepthStencilAttachment)) {
LOGF("Format vk::Format::eD32Sfloat cannot be used as depth buffer\n");
skip_device = true;
}
if (!skip_device && !CheckFormat(device, vk::Format::eD32SfloatS8Uint, true,
vk::FormatFeatureFlagBits::eDepthStencilAttachment)) {
LOGF("Format vk::Format::eD32SfloatS8Uint cannot be used as depth buffer\n");
skip_device = true;
}
if (!skip_device && !CheckFormat(device, vk::Format::eD16Unorm, true,
vk::FormatFeatureFlagBits::eDepthStencilAttachment)) {
LOGF("Format vk::Format::eD16Unorm cannot be used as depth buffer\n");
skip_device = true;
}
if (!skip_device && !CheckFormat(device, vk::Format::eBc3SrgbBlock, true,
vk::FormatFeatureFlagBits::eSampledImage |
vk::FormatFeatureFlagBits::eTransferDst)) {
LOGF("Format vk::Format::eBc3SrgbBlock cannot be used as texture\n");
skip_device = true;
}
if (!skip_device && !CheckFormat(device, vk::Format::eR8G8B8A8Srgb, true,
vk::FormatFeatureFlagBits::eSampledImage |
vk::FormatFeatureFlagBits::eTransferDst)) {
LOGF("Format vk::Format::eR8G8B8A8Srgb cannot be used as texture\n");
skip_device = true;
}
if (!skip_device && !CheckFormat(device, vk::Format::eR8Unorm, true,
vk::FormatFeatureFlagBits::eSampledImage |
vk::FormatFeatureFlagBits::eTransferDst)) {
LOGF("Format vk::Format::eR8Unorm cannot be used as texture\n");
skip_device = true;
}
if (!skip_device && !CheckFormat(device, vk::Format::eR8G8Unorm, true,
vk::FormatFeatureFlagBits::eSampledImage |
vk::FormatFeatureFlagBits::eTransferDst)) {
LOGF("Format vk::Format::eR8G8Unorm cannot be used as texture\n");
skip_device = true;
}
if (!skip_device && !CheckFormat(device, vk::Format::eR8G8B8A8Srgb, true,
vk::FormatFeatureFlagBits::eStorageImage |
vk::FormatFeatureFlagBits::eTransferDst)) {
LOGF("Format vk::Format::eR8G8B8A8Srgb cannot be used as texture\n");
if (!skip_device && !CheckFormat(device, vk::Format::eR8G8B8A8Unorm, true,
vk::FormatFeatureFlagBits::eStorageImage |
vk::FormatFeatureFlagBits::eTransferDst)) {
LOGF("Format vk::Format::eR8G8B8A8Unorm cannot be used as texture\n");
skip_device = true;
const auto check_format = [&](vk::Format format, vk::FormatFeatureFlags features,
const char* usage) {
if (!skip_device && !CheckFormat(device, format, true, features)) {
reject(fmt::format("Format vk::Format::e{} cannot be used as {}",
vk::to_string(format), usage));
}
};
const auto depth_features = vk::FormatFeatureFlagBits::eDepthStencilAttachment;
const auto texture_features =
vk::FormatFeatureFlagBits::eSampledImage | vk::FormatFeatureFlagBits::eTransferDst;
const auto storage_features =
vk::FormatFeatureFlagBits::eStorageImage | vk::FormatFeatureFlagBits::eTransferDst;
check_format(vk::Format::eD32Sfloat, depth_features, "depth buffer");
check_format(vk::Format::eD32SfloatS8Uint, depth_features, "depth buffer");
check_format(vk::Format::eD16Unorm, depth_features, "depth buffer");
check_format(vk::Format::eBc3SrgbBlock, texture_features, "texture");
check_format(vk::Format::eR8G8B8A8Srgb, texture_features, "texture");
check_format(vk::Format::eR8Unorm, texture_features, "texture");
check_format(vk::Format::eR8G8Unorm, texture_features, "texture");
if (!skip_device &&
!CheckFormat(device, vk::Format::eR8G8B8A8Srgb, true, storage_features)) {
LOGF("Format vk::Format::eR8G8B8A8Srgb cannot be used as storage image\n");
check_format(vk::Format::eR8G8B8A8Unorm, storage_features, "storage image");
}
if (!skip_device && !CheckFormat(device, vk::Format::eB8G8R8A8Srgb, true,
vk::FormatFeatureFlagBits::eStorageImage |
vk::FormatFeatureFlagBits::eTransferDst)) {
LOGF("Format vk::Format::eB8G8R8A8Srgb cannot be used as texture\n");
if (!skip_device && !CheckFormat(device, vk::Format::eB8G8R8A8Unorm, true,
vk::FormatFeatureFlagBits::eStorageImage |
vk::FormatFeatureFlagBits::eTransferDst)) {
LOGF("Format vk::Format::eB8G8R8A8Unorm cannot be used as texture\n");
skip_device = true;
}
if (!skip_device &&
!CheckFormat(device, vk::Format::eB8G8R8A8Srgb, true, storage_features)) {
LOGF("Format vk::Format::eB8G8R8A8Srgb cannot be used as storage image\n");
check_format(vk::Format::eB8G8R8A8Unorm, storage_features, "storage image");
}
if (!skip_device && device_properties.limits.maxSamplerAnisotropy < 16.0f) {
LOGF("maxSamplerAnisotropy < 16.0f");
skip_device = true;
reject("maxSamplerAnisotropy < 16.0f");
}
if (skip_device) {
@@ -530,16 +420,20 @@ static vk::Device VulkanCreateDevice(GraphicContext& graphics,
#endif
depth_clip_control.depthClipControl = VK_TRUE;
auto features12 = WindowContext::RequiredVulkan12Features();
features12.pNext = &depth_clip_control;
const bool workgroup_layout_extension =
HasExtension(device_extensions, VK_KHR_WORKGROUP_MEMORY_EXPLICIT_LAYOUT_EXTENSION_NAME);
vk::PhysicalDeviceWorkgroupMemoryExplicitLayoutFeaturesKHR supported_workgroup_layout {};
vk::PhysicalDeviceVulkan12Features supported_features12 {};
supported_features12.pNext = workgroup_layout_extension ? &supported_workgroup_layout : nullptr;
vk::PhysicalDeviceVulkan13Features supported_features13 {};
supported_features13.pNext = &supported_features12;
const auto robustness2_ext_enabled =
HasExtension(device_extensions, VK_EXT_ROBUSTNESS_2_EXTENSION_NAME);
vk::PhysicalDeviceRobustness2FeaturesEXT supported_robustness2 {};
if (robustness2_ext_enabled) {
supported_robustness2.pNext = supported_features13.pNext;
supported_features13.pNext = &supported_robustness2;
}
@@ -566,7 +460,31 @@ static vk::Device VulkanCreateDevice(GraphicContext& graphics,
provoking_vertex.pNext = supported_features2.pNext;
supported_features2.pNext = &provoking_vertex;
}
const bool image_atomic_int64_extension =
HasExtension(device_extensions, VK_EXT_SHADER_IMAGE_ATOMIC_INT64_EXTENSION_NAME);
vk::PhysicalDeviceShaderImageAtomicInt64FeaturesEXT image_atomic_int64 {};
if (image_atomic_int64_extension) {
image_atomic_int64.pNext = supported_features2.pNext;
supported_features2.pNext = &image_atomic_int64;
}
physical_device.getFeatures2(&supported_features2);
graphics.shader_image_int64_atomics_enabled = image_atomic_int64.shaderImageInt64Atomics;
auto features12 = WindowContext::RequiredVulkan12Features();
features12.shaderSharedInt64Atomics = supported_features12.shaderSharedInt64Atomics;
vk::PhysicalDeviceWorkgroupMemoryExplicitLayoutFeaturesKHR workgroup_layout {};
workgroup_layout.workgroupMemoryExplicitLayout =
supported_workgroup_layout.workgroupMemoryExplicitLayout;
workgroup_layout.pNext = &depth_clip_control;
features12.pNext = workgroup_layout_extension ? static_cast<void*>(&workgroup_layout)
: static_cast<void*>(&depth_clip_control);
if (!features12.shaderSharedInt64Atomics || !workgroup_layout.workgroupMemoryExplicitLayout) {
Log::WriteToConsoleAndLog(fmt::format(
"WARNING: Native 64-bit LDS atomics are unavailable: shaderSharedInt64Atomics={}, "
"workgroupMemoryExplicitLayout={}. Continuing with the selected Vulkan device.\n",
features12.shaderSharedInt64Atomics != VK_FALSE,
workgroup_layout.workgroupMemoryExplicitLayout != VK_FALSE));
}
graphics.mesh_shader_enabled = mesh_extension && supported_mesh.meshShader;
vk::PhysicalDeviceSubgroupSizeControlProperties subgroup_size_control {};
@@ -574,8 +492,10 @@ static vk::Device VulkanCreateDevice(GraphicContext& graphics,
vk::PhysicalDeviceVulkan11Properties properties11 {};
properties11.pNext = &subgroup_size_control;
vk::PhysicalDeviceFloatControlsProperties float_controls {};
float_controls.pNext = &properties11;
vk::PhysicalDeviceProperties2 properties2 {};
properties2.pNext = &properties11;
properties2.pNext = &float_controls;
if (graphics.mesh_shader_enabled) {
subgroup_size_control.pNext = &graphics.mesh_shader_properties;
@@ -621,6 +541,7 @@ static vk::Device VulkanCreateDevice(GraphicContext& graphics,
device_features.robustBufferAccess = VK_TRUE;
#if !defined(__APPLE__)
device_features.depthBounds = VK_TRUE; // unsupported by MoltenVK
device_features.depthClamp = VK_TRUE;
#endif
device_features.shaderStorageImageWriteWithoutFormat = VK_TRUE;
device_features.shaderImageGatherExtended = VK_TRUE;
@@ -637,6 +558,15 @@ static vk::Device VulkanCreateDevice(GraphicContext& graphics,
device_features.vertexPipelineStoresAndAtomics = VK_TRUE;
graphics.sample_rate_shading_enabled = true;
device_features.shaderInt64 = VK_TRUE;
device_features.shaderFloat64 =
supported_features2.features.shaderFloat64 &&
float_controls.shaderSignedZeroInfNanPreserveFloat64 &&
float_controls.shaderRoundingModeRTEFloat32;
// if (device_features.shaderFloat64 && !float_controls.shaderDenormPreserveFloat64) {
// Log::WriteToConsoleAndLog(
// "WARNING: Vulkan device does not guarantee FP64 denormal preservation; "
// "continuing with native FP64 arithmetic. Very small values may be flushed to zero.\n");
// }
vk::PhysicalDeviceRobustness2FeaturesEXT robustness2 {};
#if defined(__APPLE__)
@@ -683,6 +613,11 @@ static vk::Device VulkanCreateDevice(GraphicContext& graphics,
provoking_vertex.transformFeedbackPreservesProvokingVertex = VK_FALSE;
create_info.pNext = &provoking_vertex;
}
if (graphics.shader_image_int64_atomics_enabled) {
image_atomic_int64.pNext = const_cast<void*>(create_info.pNext);
image_atomic_int64.sparseImageInt64Atomics = VK_FALSE;
create_info.pNext = &image_atomic_int64;
}
create_info.pQueueCreateInfos = &queue_create_info;
create_info.queueCreateInfoCount = 1;
create_info.enabledExtensionCount = static_cast<uint32_t>(device_extensions.size());
@@ -700,25 +635,13 @@ static vk::Device VulkanCreateDevice(GraphicContext& graphics,
return device;
}
static void VulkanGetExtensions(SDL_Window* window, VulkanExtensions& r) {
EXIT_IF(window == nullptr);
static void VulkanGetExtensions(VulkanExtensions& r) {
uint32_t required_extensions_count = 0;
auto sdl_result = SDL_Vulkan_GetInstanceExtensions(window, &required_extensions_count, nullptr);
EXIT_NOT_IMPLEMENTED(sdl_result == SDL_FALSE);
const char* const* extensions = SDL_Vulkan_GetInstanceExtensions(&required_extensions_count);
EXIT_NOT_IMPLEMENTED(extensions == nullptr);
EXIT_NOT_IMPLEMENTED(required_extensions_count == 0);
r.required_extensions =
std::vector<const char*>(required_extensions_count); // @suppress("Ambiguous problem")
sdl_result = SDL_Vulkan_GetInstanceExtensions(window, &required_extensions_count,
r.required_extensions.data());
EXIT_NOT_IMPLEMENTED(sdl_result == SDL_FALSE);
EXIT_NOT_IMPLEMENTED(required_extensions_count == 0);
EXIT_NOT_IMPLEMENTED(required_extensions_count != r.required_extensions.size());
r.required_extensions.assign(extensions, extensions + required_extensions_count);
r.available_extensions =
EnumerateVulkan<vk::ExtensionProperties>( // @suppress("Ambiguous problem")
@@ -899,7 +822,7 @@ void WindowContext::CreateVulkan() {
VULKAN_HPP_DEFAULT_DISPATCHER.init(get_instance_proc_addr);
VulkanExtensions r;
VulkanGetExtensions(window, r);
VulkanGetExtensions(r);
VulkanCheckInstanceVersion();
vk::ApplicationInfo app_info {};
@@ -986,8 +909,8 @@ void WindowContext::CreateVulkan() {
}
vk::SurfaceKHR::CType native_surface = VK_NULL_HANDLE;
if (SDL_Vulkan_CreateSurface(window, static_cast<vk::Instance::CType>(graphic_ctx.instance),
&native_surface) == SDL_FALSE) {
if (!SDL_Vulkan_CreateSurface(window, static_cast<vk::Instance::CType>(graphic_ctx.instance),
nullptr, &native_surface)) {
EXIT("Could not create a Vulkan surface");
}
surface = native_surface;
@@ -1014,11 +937,13 @@ void WindowContext::CreateVulkan() {
}
#endif
std::string rejected_devices;
VulkanFindPhysicalDevice(graphic_ctx.instance, surface, device_extensions, surface_capabilities,
graphic_ctx.physical_device, graphic_ctx.queue_family);
graphic_ctx.physical_device, graphic_ctx.queue_family,
rejected_devices);
if (graphic_ctx.physical_device == nullptr) {
EXIT("Could not find suitable device");
EXIT("Could not find suitable device:%s", rejected_devices.c_str());
}
vk::PhysicalDevicePushDescriptorProperties push_descriptor_properties {};
@@ -1057,8 +982,10 @@ void WindowContext::CreateVulkan() {
graphic_ctx.memory_budget_ext_enabled = true;
}
for (const auto* extension: {VK_EXT_ROBUSTNESS_2_EXTENSION_NAME,
VK_EXT_SHADER_IMAGE_ATOMIC_INT64_EXTENSION_NAME,
VK_EXT_PROVOKING_VERTEX_EXTENSION_NAME,
VK_EXT_MESH_SHADER_EXTENSION_NAME,
VK_KHR_WORKGROUP_MEMORY_EXPLICIT_LAYOUT_EXTENSION_NAME,
VK_EXT_DEPTH_RANGE_UNRESTRICTED_EXTENSION_NAME}) {
if (HasExtension(available_extensions, extension)) {
device_extensions.push_back(extension);
@@ -1100,8 +1027,8 @@ void WindowContext::RecreateSurface() {
surface = nullptr;
}
vk::SurfaceKHR::CType native_surface = VK_NULL_HANDLE;
if (SDL_Vulkan_CreateSurface(window, static_cast<vk::Instance::CType>(graphic_ctx.instance),
&native_surface) == SDL_FALSE) {
if (!SDL_Vulkan_CreateSurface(window, static_cast<vk::Instance::CType>(graphic_ctx.instance),
nullptr, &native_surface)) {
EXIT("Could not recreate the Vulkan surface: %s\n", SDL_GetError());
}
surface = native_surface;
@@ -1137,7 +1064,7 @@ WindowContext::~WindowContext() {
SDL_DestroyWindow(window);
window = nullptr;
}
SDL_QuitSubSystem(SDL_INIT_VIDEO | SDL_INIT_GAMECONTROLLER);
SDL_QuitSubSystem(SDL_INIT_VIDEO | SDL_INIT_GAMEPAD);
}
} // namespace Libs::Graphics
+170 -238
View File
@@ -1,23 +1,7 @@
#include "graphics/presentation/window.h"
#include "SDL.h"
#include "SDL_error.h"
#include "SDL_events.h"
#include "SDL_filesystem.h"
#include "SDL_gamecontroller.h"
#include "SDL_hints.h"
#include "SDL_joystick.h"
#include "SDL_keyboard.h"
#include "SDL_keycode.h"
#include "SDL_mouse.h"
#include "SDL_pixels.h"
#include "SDL_rwops.h"
#include "SDL_stdinc.h"
#include "SDL_surface.h"
#include "SDL_thread.h"
#include "SDL_touch.h"
#include "SDL_video.h"
#include "SDL_vulkan.h"
#include <SDL3/SDL.h>
#include "common/assert.h"
#include "common/common.h"
#include "common/emulatorConfig.h"
@@ -72,39 +56,39 @@ struct EventKeyboard {
static uint32_t ControllerButtonToPadButton(int button) {
switch (button) {
case SDL_CONTROLLER_BUTTON_A: return Controller::PAD_BUTTON_CROSS;
case SDL_CONTROLLER_BUTTON_B: return Controller::PAD_BUTTON_CIRCLE;
case SDL_CONTROLLER_BUTTON_X: return Controller::PAD_BUTTON_SQUARE;
case SDL_CONTROLLER_BUTTON_Y: return Controller::PAD_BUTTON_TRIANGLE;
case SDL_CONTROLLER_BUTTON_START: return Controller::PAD_BUTTON_OPTIONS;
case SDL_CONTROLLER_BUTTON_LEFTSTICK: return Controller::PAD_BUTTON_L3;
case SDL_CONTROLLER_BUTTON_RIGHTSTICK: return Controller::PAD_BUTTON_R3;
case SDL_CONTROLLER_BUTTON_LEFTSHOULDER: return Controller::PAD_BUTTON_L1;
case SDL_CONTROLLER_BUTTON_RIGHTSHOULDER: return Controller::PAD_BUTTON_R1;
case SDL_CONTROLLER_BUTTON_DPAD_UP: return Controller::PAD_BUTTON_UP;
case SDL_CONTROLLER_BUTTON_DPAD_DOWN: return Controller::PAD_BUTTON_DOWN;
case SDL_CONTROLLER_BUTTON_DPAD_LEFT: return Controller::PAD_BUTTON_LEFT;
case SDL_CONTROLLER_BUTTON_DPAD_RIGHT: return Controller::PAD_BUTTON_RIGHT;
case SDL_CONTROLLER_BUTTON_TOUCHPAD: return Controller::PAD_BUTTON_TOUCH_PAD;
case SDL_GAMEPAD_BUTTON_SOUTH: return Controller::PAD_BUTTON_CROSS;
case SDL_GAMEPAD_BUTTON_EAST: return Controller::PAD_BUTTON_CIRCLE;
case SDL_GAMEPAD_BUTTON_WEST: return Controller::PAD_BUTTON_SQUARE;
case SDL_GAMEPAD_BUTTON_NORTH: return Controller::PAD_BUTTON_TRIANGLE;
case SDL_GAMEPAD_BUTTON_START: return Controller::PAD_BUTTON_OPTIONS;
case SDL_GAMEPAD_BUTTON_LEFT_STICK: return Controller::PAD_BUTTON_L3;
case SDL_GAMEPAD_BUTTON_RIGHT_STICK: return Controller::PAD_BUTTON_R3;
case SDL_GAMEPAD_BUTTON_LEFT_SHOULDER: return Controller::PAD_BUTTON_L1;
case SDL_GAMEPAD_BUTTON_RIGHT_SHOULDER: return Controller::PAD_BUTTON_R1;
case SDL_GAMEPAD_BUTTON_DPAD_UP: return Controller::PAD_BUTTON_UP;
case SDL_GAMEPAD_BUTTON_DPAD_DOWN: return Controller::PAD_BUTTON_DOWN;
case SDL_GAMEPAD_BUTTON_DPAD_LEFT: return Controller::PAD_BUTTON_LEFT;
case SDL_GAMEPAD_BUTTON_DPAD_RIGHT: return Controller::PAD_BUTTON_RIGHT;
case SDL_GAMEPAD_BUTTON_TOUCHPAD: return Controller::PAD_BUTTON_TOUCH_PAD;
default: return 0;
}
}
static Controller::Axis ControllerAxisFromSdl(int axis_id) {
switch (axis_id) {
case SDL_CONTROLLER_AXIS_LEFTX: return Controller::Axis::LeftX;
case SDL_CONTROLLER_AXIS_LEFTY: return Controller::Axis::LeftY;
case SDL_CONTROLLER_AXIS_RIGHTX: return Controller::Axis::RightX;
case SDL_CONTROLLER_AXIS_RIGHTY: return Controller::Axis::RightY;
case SDL_CONTROLLER_AXIS_TRIGGERLEFT: return Controller::Axis::TriggerLeft;
case SDL_CONTROLLER_AXIS_TRIGGERRIGHT: return Controller::Axis::TriggerRight;
case SDL_GAMEPAD_AXIS_LEFTX: return Controller::Axis::LeftX;
case SDL_GAMEPAD_AXIS_LEFTY: return Controller::Axis::LeftY;
case SDL_GAMEPAD_AXIS_RIGHTX: return Controller::Axis::RightX;
case SDL_GAMEPAD_AXIS_RIGHTY: return Controller::Axis::RightY;
case SDL_GAMEPAD_AXIS_LEFT_TRIGGER: return Controller::Axis::TriggerLeft;
case SDL_GAMEPAD_AXIS_RIGHT_TRIGGER: return Controller::Axis::TriggerRight;
default: return Controller::Axis::AxisMax;
}
}
static bool ControllerAxisIsTrigger(int axis_id) {
return axis_id == SDL_CONTROLLER_AXIS_TRIGGERLEFT ||
axis_id == SDL_CONTROLLER_AXIS_TRIGGERRIGHT;
return axis_id == SDL_GAMEPAD_AXIS_LEFT_TRIGGER ||
axis_id == SDL_GAMEPAD_AXIS_RIGHT_TRIGGER;
}
static int ControllerAxisValueFromSdl(int axis_id, int axis_value) {
@@ -165,22 +149,6 @@ struct EventController {
double timestamp_seconds;
};
enum class DisplayOrientation {
Unknown, /* The display orientation can't be determined */
Landscape, /* The display is in landscape mode, with the right side up, relative to portrait
mode */
LandscapeFlipped, /* The display is in landscape mode, with the left side up, relative to
portrait mode */
Portrait, /* The display is in portrait mode */
PortraitFlipped, /* The display is in portrait mode, upside down */
DisplayEventOrientation = 0xF0
};
struct EventDisplay {
DisplayOrientation orientation;
};
namespace {
std::unique_ptr<WindowContext> g_window;
@@ -213,16 +181,13 @@ static void ToggleDesktopFullscreen() {
}
const auto flags = static_cast<uint32_t>(SDL_GetWindowFlags(g_window->window));
const bool fullscreen =
(flags & static_cast<uint32_t>(SDL_WINDOW_FULLSCREEN_DESKTOP)) != 0u;
const auto mode =
fullscreen ? 0u : static_cast<uint32_t>(SDL_WINDOW_FULLSCREEN_DESKTOP);
if (SDL_SetWindowFullscreen(g_window->window, mode) != 0) {
const bool fullscreen = (flags & SDL_WINDOW_FULLSCREEN) != 0u;
if (!SDL_SetWindowFullscreen(g_window->window, !fullscreen)) {
LOGF("Toggle fullscreen failed: %s\n", SDL_GetError());
}
}
static void GameEventKeyboard(WindowLoopState& game, const EventKeyboard& key) {
static void GameEventKeyboard(const EventKeyboard& key) {
static SDL_Keycode fullscreen_key = SDLK_UNKNOWN;
#ifdef KYTY_DBG_INPUT
@@ -235,7 +200,6 @@ static void GameEventKeyboard(WindowLoopState& game, const EventKeyboard& key) {
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS || KYTY_PLATFORM == KYTY_PLATFORM_LINUX
if (key.down) {
switch (key.key_code) {
case SDLK_SPACE: SetPause(game, !game.paused.load(std::memory_order_acquire)); break;
case SDLK_F1:
if (!key.repeat) {
RenderDocRequestCapture();
@@ -248,7 +212,7 @@ static void GameEventKeyboard(WindowLoopState& game, const EventKeyboard& key) {
break;
case SDLK_RETURN:
case SDLK_KP_ENTER:
if (!key.repeat && (key.mod & KMOD_ALT) != 0) {
if (!key.repeat && (key.mod & SDL_KMOD_ALT) != 0) {
ToggleDesktopFullscreen();
fullscreen_key = key.key_code;
}
@@ -341,15 +305,21 @@ static void GameEventController([[maybe_unused]] const EventController& f) {
#endif
if (f.added) {
auto* pad = SDL_GameControllerOpen(f.id);
EXIT_NOT_IMPLEMENTED(pad == nullptr);
int id = SDL_JoystickInstanceID(SDL_GameControllerGetJoystick(pad));
auto* pad = SDL_OpenGamepad(f.id);
if (pad == nullptr) {
LOGF("Controller: ignoring gamepad %d that could not be opened: %s\n", f.id,
SDL_GetError());
return;
}
int id = SDL_GetJoystickID(SDL_GetGamepadJoystick(pad));
Controller::Connect(id);
}
if (f.removed) {
Controller::Disconnect(f.id);
SDL_GameControllerClose(SDL_GameControllerFromInstanceID(f.id));
if (auto* pad = SDL_GetGamepadFromID(f.id); pad != nullptr) {
Controller::Disconnect(f.id);
SDL_CloseGamepad(pad);
}
}
if (f.down || f.up) {
@@ -401,81 +371,73 @@ void WindowContext::Resize(uint32_t new_width, uint32_t new_height) {
void WindowContext::ProcessWindowEvent(const SDL_WindowEvent& event) {
const auto& window_event = event;
switch (window_event.event) {
case SDL_WINDOWEVENT_SHOWN:
switch (window_event.type) {
case SDL_EVENT_WINDOW_SHOWN:
LOGF("Window %" PRIu32 " shown\n", window_event.windowID);
break;
case SDL_WINDOWEVENT_HIDDEN:
case SDL_EVENT_WINDOW_HIDDEN:
LOGF("Window %" PRIu32 " hidden\n", window_event.windowID);
break;
case SDL_WINDOWEVENT_EXPOSED:
case SDL_EVENT_WINDOW_EXPOSED:
LOGF("Window %" PRIu32 " exposed\n", window_event.windowID);
break;
case SDL_WINDOWEVENT_MOVED:
case SDL_EVENT_WINDOW_MOVED:
LOGF("Window %" PRIu32 " moved to %" PRId32 ",%" PRId32 "\n", window_event.windowID,
window_event.data1, window_event.data2);
break;
case SDL_WINDOWEVENT_RESIZED:
case SDL_EVENT_WINDOW_RESIZED:
LOGF("Window %" PRIu32 " resized to %" PRId32 "x%" PRId32 "\n", window_event.windowID,
window_event.data1, window_event.data2);
LOGF("m: %d\n", static_cast<int>(SDL_ThreadID()));
Resize(window_event.data1, window_event.data2);
break;
case SDL_WINDOWEVENT_SIZE_CHANGED:
case SDL_EVENT_WINDOW_PIXEL_SIZE_CHANGED:
LOGF("Window %" PRIu32 " size changed to %" PRId32 "x%" PRId32 "\n",
window_event.windowID, window_event.data1, window_event.data2);
LOGF("m: %d\n", static_cast<int>(SDL_ThreadID()));
Resize(window_event.data1, window_event.data2);
break;
case SDL_WINDOWEVENT_MINIMIZED:
case SDL_EVENT_WINDOW_MINIMIZED:
LOGF("Window %" PRIu32 " minimized\n", window_event.windowID);
break;
case SDL_WINDOWEVENT_MAXIMIZED:
case SDL_EVENT_WINDOW_MAXIMIZED:
LOGF("Window %" PRIu32 " maximized\n", window_event.windowID);
break;
case SDL_WINDOWEVENT_RESTORED:
case SDL_EVENT_WINDOW_RESTORED:
LOGF("Window %" PRIu32 " restored\n", window_event.windowID);
break;
case SDL_WINDOWEVENT_ENTER:
case SDL_EVENT_WINDOW_MOUSE_ENTER:
LOGF("Mouse entered window %" PRIu32 "\n", window_event.windowID);
break;
case SDL_WINDOWEVENT_LEAVE:
case SDL_EVENT_WINDOW_MOUSE_LEAVE:
LOGF("Mouse left window %" PRIu32 "\n", window_event.windowID);
break;
case SDL_WINDOWEVENT_FOCUS_GAINED:
case SDL_EVENT_WINDOW_FOCUS_GAINED:
LOGF("Window %" PRIu32 " gained keyboard focus\n", window_event.windowID);
break;
case SDL_WINDOWEVENT_FOCUS_LOST:
case SDL_EVENT_WINDOW_FOCUS_LOST:
LOGF("Window %" PRIu32 " lost keyboard focus\n", window_event.windowID);
break;
case SDL_WINDOWEVENT_CLOSE:
case SDL_EVENT_WINDOW_CLOSE_REQUESTED:
LOGF("Window %" PRIu32 " closed\n", window_event.windowID);
break;
default:
LOGF("Window %" PRIu32 " got unknown event %" PRIu8 "\n", window_event.windowID,
window_event.event);
LOGF("Window %" PRIu32 " got unknown event %" PRIu32 "\n", window_event.windowID,
static_cast<uint32_t>(window_event.type));
break;
}
}
void WindowContext::ProcessDisplayEvent(const SDL_DisplayEvent& display) {
bool sdl = false;
switch (display.event) {
case SDL_DISPLAYEVENT_ORIENTATION: sdl = true; [[fallthrough]];
case static_cast<Uint8>(DisplayOrientation::DisplayEventOrientation): {
LOGF("Display %" PRIu32 "[%s] changed orientation to %d - ", display.display,
sdl ? "SDL" : "Kyty", static_cast<int>(display.data1));
switch (display.type) {
case SDL_EVENT_DISPLAY_ORIENTATION: {
LOGF("Display %" PRIu32 " changed orientation to %d - ", display.displayID,
static_cast<int>(display.data1));
switch (display.data1) {
case SDL_ORIENTATION_UNKNOWN: LOGF("UNKNOWN\n"); break;
@@ -489,8 +451,8 @@ void WindowContext::ProcessDisplayEvent(const SDL_DisplayEvent& display) {
break;
}
default:
LOGF("Display %" PRIu32 " got unknown event 0x%" PRIx8 "\n", display.display,
display.event);
LOGF("Display %" PRIu32 " got unknown event 0x%" PRIx32 "\n", display.displayID,
static_cast<uint32_t>(display.type));
break;
}
}
@@ -498,10 +460,9 @@ void WindowContext::ProcessDisplayEvent(const SDL_DisplayEvent& display) {
void WindowContext::ProcessEvent(double time_s) {
auto& game = loop;
auto* event = &game.event;
EXIT_IF(SDL_GetEventState(SDL_DISPLAYEVENT) != SDL_ENABLE);
if ((event->type == SDL_KEYDOWN || event->type == SDL_KEYUP) &&
event->key.keysym.sym == SDLK_F7) {
if (event->type == SDL_KEYDOWN && event->key.repeat == 0) {
if ((event->type == SDL_EVENT_KEY_DOWN || event->type == SDL_EVENT_KEY_UP) &&
event->key.key == SDLK_F7) {
if (event->type == SDL_EVENT_KEY_DOWN && event->key.repeat == 0) {
HostInputToggleMouseToJoystick();
}
return;
@@ -509,59 +470,63 @@ void WindowContext::ProcessEvent(double time_s) {
if (ProcessSystemOverlayInput(*event)) {
return;
}
if (event->type >= SDL_EVENT_WINDOW_FIRST && event->type <= SDL_EVENT_WINDOW_LAST) {
ProcessWindowEvent(event->window);
return;
}
if (event->type >= SDL_EVENT_DISPLAY_FIRST && event->type <= SDL_EVENT_DISPLAY_LAST) {
ProcessDisplayEvent(event->display);
return;
}
switch (event->type) {
case SDL_QUIT: GameEventQuit(game); break;
case SDL_EVENT_QUIT: GameEventQuit(game); break;
case SDL_APP_TERMINATING: GameEventTerminate(game); break;
case SDL_EVENT_TERMINATING: GameEventTerminate(game); break;
case SDL_APP_LOWMEMORY: GameEventLowMemory(); break;
case SDL_EVENT_LOW_MEMORY: GameEventLowMemory(); break;
case SDL_APP_WILLENTERBACKGROUND: GameEventWillEnterBackground(game); break;
case SDL_EVENT_WILL_ENTER_BACKGROUND: GameEventWillEnterBackground(game); break;
case SDL_APP_DIDENTERBACKGROUND: GameEventDidEnterBackground(); break;
case SDL_EVENT_DID_ENTER_BACKGROUND: GameEventDidEnterBackground(); break;
case SDL_APP_WILLENTERFOREGROUND: GameEventWillEnterForeground(); break;
case SDL_EVENT_WILL_ENTER_FOREGROUND: GameEventWillEnterForeground(); break;
case SDL_APP_DIDENTERFOREGROUND: GameEventDidEnterForeground(game); break;
case SDL_EVENT_DID_ENTER_FOREGROUND: GameEventDidEnterForeground(game); break;
case SDL_KEYDOWN:
case SDL_KEYUP: {
case SDL_EVENT_KEY_DOWN:
case SDL_EVENT_KEY_UP: {
EventKeyboard key {};
key.down = (event->type == SDL_KEYDOWN);
key.up = (event->type == SDL_KEYUP);
key.pressed = (event->key.state == SDL_PRESSED);
key.released = (event->key.state == SDL_RELEASED);
key.down = (event->type == SDL_EVENT_KEY_DOWN);
key.up = (event->type == SDL_EVENT_KEY_UP);
key.pressed = event->key.down;
key.released = !event->key.down;
key.repeat = (event->key.repeat != 0u);
key.scan_code = event->key.keysym.scancode;
key.key_code = event->key.keysym.sym;
key.mod = event->key.keysym.mod;
key.scan_code = event->key.scancode;
key.key_code = event->key.key;
key.mod = event->key.mod;
key.timestamp_seconds = time_s;
GameEventKeyboard(game, key);
GameEventKeyboard(key);
break;
}
case SDL_WINDOWEVENT: ProcessWindowEvent(event->window); break;
case SDL_DISPLAYEVENT: ProcessDisplayEvent(event->display); break;
case SDL_MOUSEBUTTONDOWN:
case SDL_MOUSEBUTTONUP: {
case SDL_EVENT_MOUSE_BUTTON_DOWN:
case SDL_EVENT_MOUSE_BUTTON_UP: {
EventMouse mb {};
mb.down = (event->button.type == SDL_MOUSEBUTTONDOWN);
mb.up = (event->button.type == SDL_MOUSEBUTTONUP);
mb.down = (event->button.type == SDL_EVENT_MOUSE_BUTTON_DOWN);
mb.up = (event->button.type == SDL_EVENT_MOUSE_BUTTON_UP);
mb.left = (event->button.button == SDL_BUTTON_LEFT);
mb.middle = (event->button.button == SDL_BUTTON_MIDDLE);
mb.right = (event->button.button == SDL_BUTTON_RIGHT);
mb.x1 = (event->button.button == SDL_BUTTON_X1);
mb.x2 = (event->button.button == SDL_BUTTON_X2);
mb.touch = (event->button.which == SDL_TOUCH_MOUSEID);
mb.pressed = (event->button.state == SDL_PRESSED);
mb.released = (event->button.state == SDL_RELEASED);
mb.pressed = event->button.down;
mb.released = !event->button.down;
mb.num_of_clicks = event->button.clicks;
mb.wheel = false;
mb.x = event->button.x;
@@ -576,7 +541,7 @@ void WindowContext::ProcessEvent(double time_s) {
break;
}
case SDL_MOUSEWHEEL: {
case SDL_EVENT_MOUSE_WHEEL: {
EventMouse mb {};
mb.down = false;
@@ -603,7 +568,7 @@ void WindowContext::ProcessEvent(double time_s) {
break;
}
case SDL_MOUSEMOTION: {
case SDL_EVENT_MOUSE_MOTION: {
EventMouse mb {};
mb.down = false;
@@ -630,16 +595,16 @@ void WindowContext::ProcessEvent(double time_s) {
break;
}
case SDL_FINGERMOTION:
case SDL_FINGERDOWN:
case SDL_FINGERUP: {
case SDL_EVENT_FINGER_MOTION:
case SDL_EVENT_FINGER_DOWN:
case SDL_EVENT_FINGER_UP: {
EventFinger f {};
f.down = (event->tfinger.type == SDL_FINGERDOWN);
f.up = (event->tfinger.type == SDL_FINGERUP);
f.motion = (event->tfinger.type == SDL_FINGERMOTION);
f.finger_id = static_cast<int>(event->tfinger.fingerId);
f.touch_id = static_cast<int>(event->tfinger.touchId);
f.down = (event->tfinger.type == SDL_EVENT_FINGER_DOWN);
f.up = (event->tfinger.type == SDL_EVENT_FINGER_UP);
f.motion = (event->tfinger.type == SDL_EVENT_FINGER_MOTION);
f.finger_id = static_cast<int>(event->tfinger.fingerID);
f.touch_id = static_cast<int>(event->tfinger.touchID);
f.x = event->tfinger.x;
f.y = event->tfinger.y;
f.dx = event->tfinger.dx;
@@ -652,13 +617,13 @@ void WindowContext::ProcessEvent(double time_s) {
break;
}
case SDL_CONTROLLERAXISMOTION: {
case SDL_EVENT_GAMEPAD_AXIS_MOTION: {
EventController c {};
c.id = event->caxis.which;
c.button = SDL_CONTROLLER_BUTTON_INVALID;
c.axis_id = event->caxis.axis;
c.axis_value = event->caxis.value;
c.id = event->gaxis.which;
c.button = SDL_GAMEPAD_BUTTON_INVALID;
c.axis_id = event->gaxis.axis;
c.axis_value = event->gaxis.value;
c.down = false;
c.up = false;
c.added = false;
@@ -674,22 +639,22 @@ void WindowContext::ProcessEvent(double time_s) {
break;
}
case SDL_CONTROLLERBUTTONDOWN:
case SDL_CONTROLLERBUTTONUP: {
case SDL_EVENT_GAMEPAD_BUTTON_DOWN:
case SDL_EVENT_GAMEPAD_BUTTON_UP: {
EventController c {};
c.id = event->cbutton.which;
c.button = event->cbutton.button;
c.axis_id = SDL_CONTROLLER_AXIS_INVALID;
c.id = event->gbutton.which;
c.button = event->gbutton.button;
c.axis_id = SDL_GAMEPAD_AXIS_INVALID;
c.axis_value = 0;
c.down = (event->cbutton.type == SDL_CONTROLLERBUTTONDOWN);
c.up = (event->cbutton.type == SDL_CONTROLLERBUTTONUP);
c.down = event->gbutton.down;
c.up = !event->gbutton.down;
c.added = false;
c.removed = false;
c.remapped = false;
c.axis = false;
c.pressed = (event->cbutton.state == SDL_PRESSED);
c.released = (event->cbutton.state == SDL_RELEASED);
c.pressed = event->gbutton.down;
c.released = !event->gbutton.down;
c.timestamp_seconds = time_s;
GameEventController(c);
@@ -697,44 +662,44 @@ void WindowContext::ProcessEvent(double time_s) {
break;
}
case SDL_CONTROLLERTOUCHPADDOWN:
case SDL_CONTROLLERTOUCHPADMOTION:
case SDL_CONTROLLERTOUCHPADUP:
if (event->ctouchpad.touchpad == 0) {
Controller::SetTouchPad(event->ctouchpad.which, event->ctouchpad.finger,
event->ctouchpad.type != SDL_CONTROLLERTOUCHPADUP,
event->ctouchpad.x, event->ctouchpad.y);
case SDL_EVENT_GAMEPAD_TOUCHPAD_DOWN:
case SDL_EVENT_GAMEPAD_TOUCHPAD_MOTION:
case SDL_EVENT_GAMEPAD_TOUCHPAD_UP:
if (event->gtouchpad.touchpad == 0) {
Controller::SetTouchPad(event->gtouchpad.which, event->gtouchpad.finger,
event->gtouchpad.type != SDL_EVENT_GAMEPAD_TOUCHPAD_UP,
event->gtouchpad.x, event->gtouchpad.y);
}
break;
case SDL_CONTROLLERSENSORUPDATE: {
const auto& sensor = event->csensor;
case SDL_EVENT_GAMEPAD_SENSOR_UPDATE: {
const auto& sensor = event->gsensor;
if (sensor.sensor == SDL_SENSOR_ACCEL || sensor.sensor == SDL_SENSOR_GYRO) {
Controller::SetSensor(sensor.which,
sensor.sensor == SDL_SENSOR_ACCEL ? Controller::Sensor::Accel
: Controller::Sensor::Gyro,
sensor.data,
sensor.timestamp_us != 0
? sensor.timestamp_us
: static_cast<uint64_t>(sensor.timestamp) * 1000);
sensor.sensor_timestamp != 0
? sensor.sensor_timestamp / 1000
: sensor.timestamp / 1000);
}
break;
}
case SDL_CONTROLLERDEVICEADDED:
case SDL_CONTROLLERDEVICEREMOVED:
case SDL_CONTROLLERDEVICEREMAPPED: {
case SDL_EVENT_GAMEPAD_ADDED:
case SDL_EVENT_GAMEPAD_REMOVED:
case SDL_EVENT_GAMEPAD_REMAPPED: {
EventController c {};
c.id = event->cdevice.which;
c.button = SDL_CONTROLLER_BUTTON_INVALID;
c.axis_id = SDL_CONTROLLER_AXIS_INVALID;
c.id = event->gdevice.which;
c.button = SDL_GAMEPAD_BUTTON_INVALID;
c.axis_id = SDL_GAMEPAD_AXIS_INVALID;
c.axis_value = 0;
c.down = false;
c.up = false;
c.added = (event->cdevice.type == SDL_CONTROLLERDEVICEADDED);
c.removed = (event->cdevice.type == SDL_CONTROLLERDEVICEREMOVED);
c.remapped = (event->cdevice.type == SDL_CONTROLLERDEVICEREMAPPED);
c.added = (event->gdevice.type == SDL_EVENT_GAMEPAD_ADDED);
c.removed = (event->gdevice.type == SDL_EVENT_GAMEPAD_REMOVED);
c.remapped = (event->gdevice.type == SDL_EVENT_GAMEPAD_REMAPPED);
c.axis = false;
c.pressed = false;
c.released = false;
@@ -747,47 +712,6 @@ void WindowContext::ProcessEvent(double time_s) {
}
}
void WindowContext::RunOnMainThread(std::function<void()> task) {
if (Common::Thread::IsMainThread()) {
task();
return;
}
uint64_t ticket = 0;
{
Common::LockGuard lock(main_task_mutex);
main_tasks.push_back(std::move(task));
ticket = ++main_tasks_queued;
}
// Wake the main loop in case it is blocked in SDL_WaitEvent.
SDL_Event event {};
event.type = SDL_USEREVENT;
SDL_PushEvent(&event);
Common::LockGuard lock(main_task_mutex);
while (main_tasks_run < ticket) {
main_task_done.Wait(&main_task_mutex);
}
}
void WindowContext::DrainMainThreadTasks() {
std::vector<std::function<void()>> tasks;
{
Common::LockGuard lock(main_task_mutex);
tasks.swap(main_tasks);
}
if (tasks.empty()) {
return;
}
for (auto& task: tasks) {
task();
}
Common::LockGuard lock(main_task_mutex);
main_tasks_run += tasks.size();
main_task_done.SignalAll();
}
void WindowContext::Run() {
Common::Timer timer;
timer.Start();
@@ -797,7 +721,6 @@ void WindowContext::Run() {
loop.paused.store(false, std::memory_order_release);
while (!loop.need_exit) {
DrainMainThreadTasks();
if (loop.paused.load(std::memory_order_acquire)) {
if (!timer.IsPaused()) {
timer.Pause();
@@ -821,17 +744,15 @@ static void WindowCreate(WindowContext& context) {
int width = static_cast<int>(context.graphic_ctx.screen_width);
int height = static_cast<int>(context.graphic_ctx.screen_height);
SDL_SetHint(SDL_HINT_JOYSTICK_HIDAPI_PS4_RUMBLE, "1");
SDL_SetHint(SDL_HINT_JOYSTICK_HIDAPI_PS5_RUMBLE, "1");
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS
SDL_SetHint(SDL_HINT_WINDOWS_DPI_SCALING, "0");
#if KYTY_PLATFORM == KYTY_PLATFORM_LINUX
// RenderDoc can hide Wayland Vulkan surface support; prefer X11 for captures.
if (Config::RenderDocEnabled()) {
SDL_SetHint(SDL_HINT_VIDEO_DRIVER, "x11,wayland");
}
#endif
if (SDL_InitSubSystem(SDL_INIT_VIDEO | SDL_INIT_GAMECONTROLLER) < 0) {
if (!SDL_InitSubSystem(SDL_INIT_VIDEO | SDL_INIT_GAMEPAD)) {
EXIT("%s\n", SDL_GetError());
}
HostInputInit();
InitializeSystemOverlayInput();
LOGF("WindowCreate(): width = %d, height = %d\n", width, height);
@@ -841,15 +762,14 @@ static void WindowCreate(WindowContext& context) {
// MoltenVK loader before calling SDL_CreateWindow. Keep an explicit user
// override, and fall back to SDL's normal loader search when no bundle is present.
if (std::getenv("SDL_VULKAN_LIBRARY") == nullptr) {
if (char* base_path = SDL_GetBasePath(); base_path != nullptr) {
if (const char* base_path = SDL_GetBasePath(); base_path != nullptr) {
const std::string base_path_str = base_path;
SDL_free(base_path);
std::string moltenvk_path = base_path_str + "libMoltenVK.dylib";
if (!Common::File::IsFileExisting(moltenvk_path)) {
moltenvk_path = base_path_str + "../Frameworks/libMoltenVK.dylib";
}
if (Common::File::IsFileExisting(moltenvk_path) &&
SDL_setenv("SDL_VULKAN_LIBRARY", moltenvk_path.c_str(), 0) == 0) {
SDL_setenv_unsafe("SDL_VULKAN_LIBRARY", moltenvk_path.c_str(), 0) == 0) {
LOGF("Vulkan loader: %s\n", moltenvk_path.c_str());
}
}
@@ -862,21 +782,22 @@ static void WindowCreate(WindowContext& context) {
window_flags |= static_cast<uint32_t>(SDL_WINDOW_BORDERLESS);
}
#endif
context.window = SDL_CreateWindow(KYTY_SDL_WINDOW_CAPTION, SDL_WINDOWPOS_CENTERED,
SDL_WINDOWPOS_CENTERED, width, height, window_flags);
context.window = SDL_CreateWindow(KYTY_SDL_WINDOW_CAPTION, width, height, window_flags);
if (context.window == nullptr) {
EXIT("%s\n", SDL_GetError());
}
SDL_SetWindowResizable(context.window, SDL_FALSE);
SDL_SetWindowPosition(context.window, SDL_WINDOWPOS_CENTERED, SDL_WINDOWPOS_CENTERED);
HostInputInit(context.window);
InitializeSystemOverlayInput(context.window);
context.UpdateIcon();
}
uint32_t WindowContext::InitialWindowFlags(bool fullscreen) noexcept {
auto flags = static_cast<uint32_t>(SDL_WINDOW_VULKAN);
auto flags = static_cast<uint32_t>(SDL_WINDOW_VULKAN | SDL_WINDOW_RESIZABLE);
if (fullscreen) {
flags |= static_cast<uint32_t>(SDL_WINDOW_FULLSCREEN_DESKTOP);
flags |= static_cast<uint32_t>(SDL_WINDOW_FULLSCREEN);
}
return flags;
}
@@ -902,11 +823,13 @@ void WindowRun() {
EXIT_IF(g_window == nullptr);
g_window->Run();
Common::LockGuard lock(g_window->render_context->GetMutex());
g_window->render_context->GetPipelineCache().Save();
}
void WindowShutdown() {
if (g_window != nullptr) {
HostInputShutdown();
Controller::EmergencyShutdown();
g_window.reset();
}
@@ -942,7 +865,7 @@ struct WindowIcon {
void* pixels = nullptr;
~WindowIcon() {
SDL_FreeSurface(surface);
SDL_DestroySurface(surface);
stbi_image_free(pixels);
}
};
@@ -966,8 +889,8 @@ static void WindowLoadPngIcon(const std::filesystem::path& path, WindowIcon* ico
EXIT_IF(icon->pixels == nullptr);
icon->surface = SDL_CreateRGBSurfaceWithFormatFrom(icon->pixels, width, height, 32, width * 4,
SDL_PIXELFORMAT_RGBA32);
icon->surface = SDL_CreateSurfaceFrom(width, height, SDL_PIXELFORMAT_RGBA32, icon->pixels,
width * 4);
EXIT_NOT_IMPLEMENTED(icon->surface == nullptr);
}
@@ -1029,7 +952,16 @@ void WindowContext::UpdateTitle() {
(has_title_id ? ", " : ""), (has_app_ver ? app_ver : ""), (has_app_ver ? " " : ""),
device_name, processor_name, frame_num, current_fps);
RunOnMainThread([this, text = std::move(text)] { SDL_SetWindowTitle(window, text.c_str()); });
struct TitleUpdate {
SDL_Window* window;
std::string* text;
} update {window, &text};
EXIT_IF(!SDL_RunOnMainThread(
[](void* data) {
auto& title = *static_cast<TitleUpdate*>(data);
SDL_SetWindowTitle(title.window, title.text->c_str());
},
&update, true));
}
} // namespace Libs::Graphics
@@ -1,15 +1,14 @@
#ifndef EMULATOR_SRC_GRAPHICS_PRESENTATION_WINDOW_WINDOWINTERNAL_H_
#define EMULATOR_SRC_GRAPHICS_PRESENTATION_WINDOW_WINDOWINTERNAL_H_
#include "SDL_events.h"
#include "SDL_video.h"
#include <SDL3/SDL.h>
#include "common/threads.h"
#include "graphics/host_gpu/graphicContext.h"
#include "graphics/host_gpu/vulkanCommon.h"
#include <atomic>
#include <cstdint>
#include <functional>
#include <memory>
#include <vector>
@@ -48,9 +47,6 @@ struct WindowContext {
void ProcessDisplayEvent(const SDL_DisplayEvent& event);
void ProcessEvent(double time_seconds);
void Run();
// SDL window operations must complete on the main thread.
void RunOnMainThread(std::function<void()> task);
void DrainMainThreadTasks();
GraphicContext graphic_ctx;
SDL_Window* window = nullptr;
@@ -61,12 +57,6 @@ struct WindowContext {
WindowLoopState loop;
Common::Mutex mutex;
Common::Mutex main_task_mutex;
Common::CondVar main_task_done;
std::vector<std::function<void()>> main_tasks; // guarded by main_task_mutex
uint64_t main_tasks_queued = 0; // guarded by main_task_mutex
uint64_t main_tasks_run = 0; // guarded by main_task_mutex
};
} // namespace Libs::Graphics
@@ -15,7 +15,6 @@
#include "graphics/shader/recompiler/ir/passes/ResourceMaterialization.h"
#include "graphics/shader/recompiler/ir/passes/ResourceTracking.h"
#include "graphics/shader/recompiler/ir/passes/ShaderInfoCollection.h"
#include "graphics/shader/recompiler/ir/passes/SrtWalker.h"
#include "graphics/shader/recompiler/ir/passes/SsaRewrite.h"
#include <algorithm>
@@ -533,30 +532,40 @@ TranslateResult TranslateProgram(std::span<const uint32_t> code, const CompileOp
LOGF("%s phase begin: stage=%s hash=0x%016" PRIx64 " CFG BuildGraph\n", GetDumpLabel(options),
StageName(options.stage), options.shader_hash);
auto cfg = CFG::BuildGraph(decoded);
auto native_cfg = CFG::BuildGraph(decoded);
CFG::Graph structured_cfg;
auto* selected_cfg = &native_cfg;
LOGF("%s phase end: stage=%s hash=0x%016" PRIx64 " CFG BuildGraph blocks=%" PRIu64
" loops=%" PRIu64 " back_edges=%" PRIu64 " elapsed_ms=%" PRIu64 "\n",
GetDumpLabel(options), StageName(options.stage), options.shader_hash,
static_cast<uint64_t>(cfg.blocks.size()), static_cast<uint64_t>(cfg.natural_loops.size()),
static_cast<uint64_t>(cfg.back_edges.size()), phase_ms());
if (cfg.irreducible) {
LogDispatcherFallback(options, cfg, "build");
static_cast<uint64_t>(native_cfg.blocks.size()),
static_cast<uint64_t>(native_cfg.natural_loops.size()),
static_cast<uint64_t>(native_cfg.back_edges.size()), phase_ms());
if (native_cfg.irreducible) {
LogDispatcherFallback(options, native_cfg, "build");
} else {
LOGF("%s phase begin: stage=%s hash=0x%016" PRIx64 " CFG Structurize\n",
GetDumpLabel(options), StageName(options.stage), options.shader_hash);
if (!CFG::Structurize(cfg)) {
LogDispatcherFallback(options, cfg, "structurize");
structured_cfg = CFG::Structurize(native_cfg);
if (structured_cfg.unsupported) {
native_cfg.unsupported = true;
native_cfg.failure_kind = structured_cfg.failure_kind;
native_cfg.failure_block = structured_cfg.failure_block;
native_cfg.unsupported_reason = structured_cfg.unsupported_reason;
LogDispatcherFallback(options, native_cfg, "structurize");
} else {
selected_cfg = &structured_cfg;
LOGF("%s structured CFG success: blocks=%" PRIu64 "\n", GetDumpLabel(options),
static_cast<uint64_t>(cfg.blocks.size()));
static_cast<uint64_t>(selected_cfg->blocks.size()));
}
LOGF("%s phase end: stage=%s hash=0x%016" PRIx64 " CFG Structurize blocks=%" PRIu64
" loops=%" PRIu64 " elapsed_ms=%" PRIu64 "\n",
GetDumpLabel(options), StageName(options.stage), options.shader_hash,
static_cast<uint64_t>(cfg.blocks.size()),
static_cast<uint64_t>(cfg.natural_loops.size()), phase_ms());
static_cast<uint64_t>(selected_cfg->blocks.size()),
static_cast<uint64_t>(selected_cfg->natural_loops.size()), phase_ms());
}
const auto& cfg = *selected_cfg;
Frontend::EmbeddedFetchPlan embedded_fetch;
if ((options.stage == ShaderType::Vertex || options.stage == ShaderType::Local) &&
options.input_info.vertex != nullptr && options.input_info.vertex->fetch_embedded) {
@@ -585,7 +594,7 @@ TranslateResult TranslateProgram(std::span<const uint32_t> code, const CompileOp
GetDumpLabel(options), StageName(options.stage), options.shader_hash,
static_cast<uint64_t>(ir.blocks.size()), phase_ms());
IR::RewriteToSsa(ir.blocks);
IR::ConstantPropagationPass(ir.blocks);
IR::ConstantPropagationPass(ir.blocks, ir.wave_size);
IR::ResolveControlFlowIdentities(ir);
IR::RemoveIdentities(ir.blocks);
IR::EliminateDeadCode(ir.blocks);
@@ -593,21 +602,26 @@ TranslateResult TranslateProgram(std::span<const uint32_t> code, const CompileOp
if (read_lane_stats.rewritten_reads != 0) {
LOGF("%s read-lane elimination: reads=%" PRIu32 "\n", GetDumpLabel(options),
read_lane_stats.rewritten_reads);
IR::ConstantPropagationPass(ir.blocks);
IR::ConstantPropagationPass(ir.blocks, ir.wave_size);
IR::ResolveControlFlowIdentities(ir);
IR::RemoveIdentities(ir.blocks);
IR::EliminateDeadCode(ir.blocks);
}
LowerTessellationMemory(ir, options);
IR::BuildSrtPlan(ir);
IR::EliminateDeadCode(ir.blocks);
IR::TrackResources(ir);
IR::EliminateDeadCode(ir.blocks);
std::string cfg_dump;
if (options.dump_ir) {
cfg_dump = CFG::GraphToString(cfg);
if (options.early_dump) {
LOGF("%s native IR before resource tracking:\n%s", GetDumpLabel(options),
MakeIrDump(cfg_dump, ir).c_str());
}
}
IR::TrackResources(ir, decoded, native_cfg);
TranslateResult result;
result.program = std::move(ir);
if (options.dump_ir) {
result.decoded_dump = std::move(decoded_dump);
result.cfg_dump = CFG::GraphToString(cfg);
result.cfg_dump = std::move(cfg_dump);
}
return result;
}
@@ -618,6 +632,35 @@ CompileResult CompileProgram(TranslateResult translated, const CompileOptions& o
const auto emit_begin = std::chrono::steady_clock::now();
auto& ir = translated.program;
IR::ApplyResourceSpecialization(ir, specialization);
// The resource plan owns host descriptor evaluation now. Keep only dependencies consumed
// by GPU memory operations; bound descriptor dwords must not retain shader instructions.
for (auto& inst: ir.value_storage) {
inst.Invalidate();
}
for (auto* block: ir.blocks) {
for (auto& inst: *block) {
const auto op = inst.GetOpcode();
uint32_t first = 0;
if (op == IR::ValueOpcode::GetBufferResource) {
if (std::ranges::any_of(inst.Uses(), [&](const IR::Use& use) {
return ir.memory_info[use.user->Flags<IR::MemoryFlags>().index].kind ==
IR::ResourceKind::IndirectBuffer;
})) {
continue;
}
} else if (op == IR::ValueOpcode::GetImageResource) {
const auto resource = inst.Flags<uint32_t>();
first = resource < ir.info.images.size() &&
ir.info.images[resource].indirect_root == resource ? 1u : 0u;
} else if (op != IR::ValueOpcode::GetSamplerResource) {
continue;
}
for (size_t index = first; index < inst.NumArgs(); index++) {
inst.SetArg(index, IR::Value(0u));
}
}
}
ir.value_storage.clear();
IR::RemoveIdentities(ir.blocks);
IR::EliminateDeadCode(ir.blocks);
@@ -217,7 +217,7 @@ void LowerTessellationMemory(IR::Program& program, const CompileOptions& options
2, ActiveAddress(*block, it, it->Arg(2), it->Arg(it->NumArgs() - 1u), resolved));
}
}
ConstantPropagationPass(program.blocks);
ConstantPropagationPass(program.blocks, program.wave_size);
uint32_t reads = 0, writes = 0, factors = 0;
for (auto* block: program.blocks) {
for (auto it = block->begin(); it != block->end(); ++it) {
@@ -317,7 +317,7 @@ void LowerTessellationMemory(IR::Program& program, const CompileOptions& options
inst.Invalidate();
}
}
ConstantPropagationPass(program.blocks);
ConstantPropagationPass(program.blocks, program.wave_size);
RemoveIdentities(program.blocks);
EliminateDeadCode(program.blocks);
LOGF("%s tessellation lowering: reads=%u writes=%u factors=%u\n",
@@ -3,6 +3,7 @@
#include "common/assert.h"
#include "graphics/shader/recompiler/backend/spirv/spirvEmitterInternal.h"
#include "graphics/shader/recompiler/ir/ShaderIR.h"
#include "graphics/shader/recompiler/ir/passes/BindingLayout.h"
#include <algorithm>
#include <array>
@@ -34,16 +35,13 @@ void ValidateNativeProgram(const IR::Program& program) {
present[index] = true;
expected[index] = std::move(resources);
};
if (!program.info.buffers.empty()) {
Expect(Kind::Buffers, Dense(program.info.buffers.size()));
}
for (uint32_t i = 0; i < program.info.images.size(); i++) {
const auto kind = IR::DescriptorBindingForImage(program.info.images[i]);
if (!kind.has_value()) {
Fail(program, "native shader plan has an invalid image class");
}
present[static_cast<size_t>(*kind)] = true;
const auto dynamic = program.info.images[i].mip_mode == IR::ImageMipMode::DynamicStorage;
const auto dynamic = program.info.images[i].mip_mode == IR::ImageMipMode::Dynamic;
const auto count = dynamic ? program.info.images[i].mip_count : 1u;
if (count == 0u || (!dynamic && program.info.images[i].mip_count != 1u)) {
Fail(program, "native shader plan has an invalid image mip descriptor count");
@@ -54,23 +52,9 @@ void ValidateNativeProgram(const IR::Program& program) {
if (!program.info.samplers.empty()) {
Expect(Kind::Samplers, Dense(program.info.samplers.size()));
}
bool uses_gds = false;
for (const auto* block: program.blocks) {
for (const auto& inst: *block) {
if (IR::SharedAccessOf(inst.GetOpcode()) == IR::SharedAccess::None) {
continue;
}
const auto index = inst.Flags<IR::MemoryFlags>().index;
if (index >= program.memory_info.size()) {
Fail(program, "shared operation has invalid memory metadata");
}
const auto kind = program.memory_info[index].kind;
if (kind != IR::ResourceKind::Lds && kind != IR::ResourceKind::Gds) {
Fail(program, "shared operation has invalid resource kind");
}
uses_gds |= kind == IR::ResourceKind::Gds;
}
}
auto& buffers = expected[static_cast<size_t>(Kind::Buffers)];
const bool uses_gds = IR::CollectMemoryResources(program, buffers);
present[static_cast<size_t>(Kind::Buffers)] = !buffers.empty();
if (uses_gds) {
Expect(Kind::Gds);
}
@@ -78,12 +62,7 @@ void ValidateNativeProgram(const IR::Program& program) {
Expect(Kind::BdaPagetable);
Expect(Kind::FaultBuffer);
}
const bool uses_flattened_runtime =
!program.srt_reads.empty() ||
std::ranges::any_of(program.info.images, [](const IR::ImageResource& image) {
return image.indirect_search_iterations != 0u;
});
if (uses_flattened_runtime) {
if (IR::UsesFlattenedSrt(program)) {
Expect(Kind::FlattenedSrt);
}
if (program.bindings.ShaderDataDwords() != 0 && !program.bindings.UsesPushData()) {
@@ -109,7 +88,7 @@ void ValidateNativeProgram(const IR::Program& program) {
if ((program.bindings.UsesPushData() &&
!IR::PushData::CanFit(program.bindings.push_data_start_dword, shader_data_dwords)) ||
program.bindings.memory_offset_dword != program.bindings.user_data_registers.size() ||
program.bindings.memory_offset_count != program.info.buffers.size() ||
program.bindings.memory_offset_count != buffers.size() ||
has_shader_data_storage != (shader_data_dwords != 0 && !program.bindings.UsesPushData()) ||
!std::is_sorted(program.bindings.user_data_registers.begin(),
program.bindings.user_data_registers.end()) ||
@@ -132,12 +111,33 @@ void ValidateNativeProgram(const IR::Program& program) {
program.memory_info[index].planning_only;
});
};
const auto indirect_buffer_handle = [&](const IR::Inst& handle) {
return program.info.uses_dma && handle.NumArgs() == 4u && !handle.Uses().empty() &&
std::ranges::all_of(handle.Uses(), [&](const IR::Use& use) {
if (IR::BufferAccessOf(use.user->GetOpcode()) != IR::BufferAccess::Read) {
return false;
}
const auto index = use.user->Flags<IR::MemoryFlags>().index;
return index < program.memory_info.size() &&
program.memory_info[index].kind == IR::ResourceKind::IndirectBuffer;
});
};
const auto local_flat_handle = [&](const IR::Inst& handle) {
return !handle.Uses().empty() &&
std::ranges::all_of(handle.Uses(), [&](const IR::Use& use) {
if (IR::AddressOpcodeInfoOf(use.user->GetOpcode()).access == IR::AddressAccess::None)
return false;
const auto index = use.user->Flags<IR::MemoryFlags>().index;
return index < program.memory_info.size() &&
program.memory_info[index].kind == IR::ResourceKind::FlatLocal;
});
};
for (const auto* block: program.blocks) {
for (const auto& inst: *block) {
const auto dense = inst.Flags<uint32_t>();
switch (inst.GetOpcode()) {
case IR::ValueOpcode::GetBufferResource:
if (planning_only_handle(inst)) {
if (planning_only_handle(inst) || indirect_buffer_handle(inst)) {
break;
}
if (dense >= program.info.buffers.size()) {
@@ -148,7 +148,7 @@ void ValidateNativeProgram(const IR::Program& program) {
if (planning_only_handle(inst)) {
break;
}
if (inst.NumArgs() != 2 || !program.info.uses_dma) {
if (inst.NumArgs() != 2 || (!program.info.uses_dma && !local_flat_handle(inst))) {
Fail(program, "typed address handle has invalid DMA metadata");
}
break;
@@ -187,6 +187,7 @@ Emitter::SpirvRequirements Emitter::AnalyzeProgramRequirements(const IR::Program
SpirvRequirements requirements {};
for (const auto* block: program.blocks) {
for (const auto& inst: *block) {
requirements.float64 |= inst.GetType() == IR::Type::F64;
if (IR::BufferAccessOf(inst.GetOpcode()) == IR::BufferAccess::Atomic &&
inst.GetType() == IR::Type::U64) {
requirements.buffer_int64_atomics = true;
@@ -197,11 +198,16 @@ Emitter::SpirvRequirements Emitter::AnalyzeProgramRequirements(const IR::Program
if (memory_index >= program.memory_info.size()) {
Fail(program, "address operation has invalid memory metadata");
}
if (program.memory_info[memory_index].kind == IR::ResourceKind::Scratch) {
const auto kind = program.memory_info[memory_index].kind;
if (kind == IR::ResourceKind::Scratch || kind == IR::ResourceKind::FlatLocal) {
if (program.scratch_dwords == 0) {
Fail(program, "scratch operation has no per-thread storage");
}
requirements.function_scratch = true;
if (kind == IR::ResourceKind::FlatLocal && program.stage != ShaderType::Compute &&
program.stage != ShaderType::Mesh) {
requirements.function_lds = true;
}
} else if (address_access == IR::AddressAccess::Write) {
Fail(program, "writable FLAT/GLOBAL addresses require GPU ownership tracking");
}
@@ -212,7 +218,11 @@ Emitter::SpirvRequirements Emitter::AnalyzeProgramRequirements(const IR::Program
Fail(program, "buffer operation has invalid memory metadata");
}
const auto& memory = program.memory_info[memory_index];
if (memory.kind == IR::ResourceKind::IndirectBuffer) {
requirements.subgroup_local_invocation_id = true;
}
if (memory.kind == IR::ResourceKind::Buffer) {
requirements.coherent_buffers |= memory.coherent;
if (memory.resource >= program.info.buffers.size()) {
Fail(program, "buffer operation has invalid resource metadata");
}
@@ -234,6 +244,13 @@ Emitter::SpirvRequirements Emitter::AnalyzeProgramRequirements(const IR::Program
if (kind != IR::ResourceKind::Lds && kind != IR::ResourceKind::Gds) {
Fail(program, "shared operation has invalid resource kind");
}
if (shared_access == IR::SharedAccess::Atomic &&
IR::SharedComponentCount(inst.GetOpcode()) == 2u) {
if (kind != IR::ResourceKind::Lds || program.stage != ShaderType::Compute) {
Fail(program, "64-bit shared atomics require compute LDS");
}
requirements.shared_int64_atomics = true;
}
if (program.stage != ShaderType::Compute && program.stage != ShaderType::Mesh &&
kind == IR::ResourceKind::Lds) {
requirements.function_lds = true;
@@ -246,6 +263,7 @@ Emitter::SpirvRequirements Emitter::AnalyzeProgramRequirements(const IR::Program
}
}
switch (inst.GetOpcode()) {
case IR::ValueOpcode::BvhIntersect: requirements.bvh = true; break;
case IR::ValueOpcode::Ballot: requirements.subgroup_ballot = true; break;
case IR::ValueOpcode::DppMoveU32:
case IR::ValueOpcode::ReadFirstLane:
@@ -291,9 +291,25 @@ uint32_t EmitConvertU32F32(EmitterState& state, uint32_t arg0) {
return EmitF32ToU32(state, arg0, false);
}
uint32_t EmitConvertF32S32(EmitterState& state, uint32_t arg0) {
const auto signed_value = Unary(state, spv::OpBitcast, TypeI32(state), arg0);
return EmitNative<spv::OpConvertSToF, IR::Type::F32>(state, signed_value);
uint32_t EmitConvertF32F64(EmitterState& state, uint32_t arg0) {
const auto converted = Unary(state, spv::OpFConvert, TypeF32(state), arg0);
const auto source = EmitNative<spv::OpCompositeExtract, IR::Type::U32>(
state, Unary(state, spv::OpBitcast, TypeU64(state), arg0), 1u);
const auto exponent = EmitAndConstant(state, source, 0x7ff00000u);
const auto overflow =
Binary(state, spv::OpLogicalAnd, TypeBool(state),
EmitCompareU32Constant(state, spv::OpUGreaterThan, exponent, 0x47e00000u),
EmitCompareU32Constant(state, spv::OpULessThan, exponent, 0x7ff00000u));
const auto clamped = EmitOrU32(state, EmitAndConstant(state, source, 0x80000000u),
ConstantU32(state, 0x7f7fffffu));
return EmitFlushF32DenormToSignedZero(
state, Select(state, TypeF32(state), overflow,
Unary(state, spv::OpBitcast, TypeF32(state), clamped), converted));
}
uint32_t EmitConvertF64F32(EmitterState& state, uint32_t arg0) {
return EmitNative<spv::OpFConvert, IR::Type::F64>(state,
EmitFlushF32DenormToSignedZero(state, arg0));
}
uint32_t EmitCompositeExtractU64(EmitterState& state, uint32_t arg0, IR::Value arg1) {
@@ -428,6 +444,18 @@ uint32_t EmitUGreaterThan64(EmitterState& state, uint32_t arg0, uint32_t arg1) {
return CompareOrdered64(state, arg0, arg1, spv::OpUGreaterThan, spv::OpUGreaterThan);
}
uint32_t EmitSLessThanEqual64(EmitterState& state, uint32_t arg0, uint32_t arg1) {
return CompareOrdered64(state, arg0, arg1, spv::OpSLessThan, spv::OpULessThanEqual);
}
uint32_t EmitULessThanEqual64(EmitterState& state, uint32_t arg0, uint32_t arg1) {
return CompareOrdered64(state, arg0, arg1, spv::OpULessThan, spv::OpULessThanEqual);
}
uint32_t EmitUGreaterThanEqual64(EmitterState& state, uint32_t arg0, uint32_t arg1) {
return CompareOrdered64(state, arg0, arg1, spv::OpUGreaterThan, spv::OpUGreaterThanEqual);
}
uint32_t EmitFPIsNan32(EmitterState& state, uint32_t arg0) {
return EmitNative<spv::OpFUnordNotEqual, IR::Type::U1>(state, arg0, arg0);
}
@@ -453,6 +481,11 @@ uint32_t EmitFPRecip32(EmitterState& state, uint32_t arg0) {
return Binary(state, spv::OpFDiv, TypeF32(state), ConstantF32(state, 0x3f800000u), source);
}
uint32_t EmitFPRecip64(EmitterState& state, uint32_t arg0) {
const auto one = state.builder.Constant(spv::OpConstant, TypeF64(state), 0u, 0x3ff00000u);
return EmitNative<spv::OpFDiv, IR::Type::F64>(state, one, arg0);
}
uint32_t EmitFPRecipIFlag32(EmitterState& state, uint32_t arg0) {
// Integer-to-float inputs used by IFLAG cannot be denormal.
return Binary(state, spv::OpFDiv, TypeF32(state), ConstantF32(state, 0x3f800000u), arg0);
@@ -203,9 +203,7 @@ uint32_t EmitFlushF32DenormToSignedZero(EmitterState& state, uint32_t value) {
const auto bits = state.builder.AllocateId();
const auto abs_bits = state.builder.AllocateId();
const auto sign_bits = state.builder.AllocateId();
const auto non_zero = state.builder.AllocateId();
const auto subnormal = state.builder.AllocateId();
const auto flush = state.builder.AllocateId();
const auto selected = state.builder.AllocateId();
const auto ret = state.builder.AllocateId();
state.builder.AddFunction(spv::OpBitcast, TypeU32(state), bits, value);
@@ -213,12 +211,9 @@ uint32_t EmitFlushF32DenormToSignedZero(EmitterState& state, uint32_t value) {
ConstantU32(state, 0x7fffffffu));
state.builder.AddFunction(spv::OpBitwiseAnd, TypeU32(state), sign_bits, bits,
ConstantU32(state, 0x80000000u));
state.builder.AddFunction(spv::OpINotEqual, TypeBool(state), non_zero, abs_bits,
ConstantU32(state, 0));
state.builder.AddFunction(spv::OpULessThan, TypeBool(state), subnormal, abs_bits,
ConstantU32(state, 0x00800000u));
state.builder.AddFunction(spv::OpLogicalAnd, TypeBool(state), flush, non_zero, subnormal);
state.builder.AddFunction(spv::OpSelect, TypeU32(state), selected, flush, sign_bits, bits);
state.builder.AddFunction(spv::OpSelect, TypeU32(state), selected, subnormal, sign_bits, bits);
state.builder.AddFunction(spv::OpBitcast, TypeF32(state), ret, selected);
return ret;
}
@@ -124,13 +124,21 @@ uint32_t ImageType(EmitterState& state, const IR::ImageResource& image) {
sampled = 2;
if (image.atomic) {
EXIT_IF(image.numeric_class != Prospero::TextureNumericClass::Uint);
format = spv::ImageFormatR32ui;
format = image.atomic64 ? spv::ImageFormatR64ui : spv::ImageFormatR32ui;
if (image.atomic64) {
state.builder.RequireExtension("SPV_EXT_shader_image_int64");
state.builder.RequireCapability(spv::CapabilityInt64);
state.builder.RequireCapability(spv::CapabilityInt64Atomics);
state.builder.RequireCapability(spv::CapabilityInt64ImageEXT);
}
}
} else {
EXIT("invalid image resource class");
}
const auto& info = ImageDimensionInfoFor(image.dimension);
return state.builder.Type(spv::OpTypeImage, ImageScalarType(state, image.numeric_class),
const auto scalar_type = image.atomic64 ? TypeScalarU64(state)
: ImageScalarType(state, image.numeric_class);
return state.builder.Type(spv::OpTypeImage, scalar_type,
info.spirv_dimension, image.depth_compare ? 1u : 0u, info.arrayed,
info.multisampled, sampled, format);
}
@@ -144,20 +152,11 @@ uint32_t ImageViewSizeType(EmitterState& state, ImageDimension dimension) {
}
}
uint32_t LoadSampledImageDescriptor(EmitterState& state, uint32_t resource) {
const auto& image_resource = state.program.info.images.at(resource);
EXIT_IF(image_resource.resource_class != IR::ImageResourceClass::Sampled);
const auto kind = IR::DescriptorBindingForImage(image_resource);
EXIT_IF(!kind.has_value());
const auto array_index = ResourceForDescriptor(state, *kind, resource);
const auto variable = state.image_variables[IR::ImageBindingIndex(*kind)];
const auto pointer_type = state.builder.Type(
spv::OpTypePointer, spv::StorageClassUniformConstant, ImageType(state, image_resource));
const auto pointer =
DescriptorElementPointer(state, pointer_type, variable, array_index, *kind, resource,
"sampled image descriptor array was not emitted");
uint32_t LoadImageDescriptor(EmitterState& state, uint32_t resource, uint32_t mip) {
const auto pointer = ImageDescriptorPointer(state, resource, mip);
const auto image = state.builder.AllocateId();
state.builder.AddFunction(spv::OpLoad, ImageType(state, image_resource), image, pointer);
state.builder.AddFunction(spv::OpLoad, ImageType(state, state.program.info.images.at(resource)),
image, pointer);
return image;
}
@@ -175,10 +174,10 @@ uint32_t LoadSamplerDescriptor(EmitterState& state, uint32_t sampler) {
return sampler_id;
}
uint32_t MakeSampledImage(EmitterState& state, uint32_t resource, uint32_t sampler) {
uint32_t MakeSampledImage(EmitterState& state, uint32_t resource, uint32_t sampler_id, uint32_t mip) {
const auto& image_resource = state.program.info.images.at(resource);
const auto image = LoadSampledImageDescriptor(state, resource);
const auto sampler_id = LoadSamplerDescriptor(state, sampler);
EXIT_IF(image_resource.resource_class != IR::ImageResourceClass::Sampled);
const auto image = LoadImageDescriptor(state, resource, mip);
const auto sampled_image = state.builder.AllocateId();
const auto sampled_type =
state.builder.Type(spv::OpTypeSampledImage, ImageType(state, image_resource));
@@ -186,17 +185,17 @@ uint32_t MakeSampledImage(EmitterState& state, uint32_t resource, uint32_t sampl
return sampled_image;
}
uint32_t StorageImageDescriptorPointer(EmitterState& state, uint32_t resource) {
uint32_t ImageDescriptorPointer(EmitterState& state, uint32_t resource, uint32_t mip) {
const auto& image = state.program.info.images.at(resource);
EXIT_IF(image.resource_class != IR::ImageResourceClass::Storage);
EXIT_IF(mip >= image.mip_count);
const auto kind = IR::DescriptorBindingForImage(image);
EXIT_IF(!kind.has_value());
const auto array_index = ResourceForDescriptor(state, *kind, resource);
const auto array_index = ResourceForDescriptor(state, *kind, resource) + mip;
const auto pointer_type = state.builder.Type(
spv::OpTypePointer, spv::StorageClassUniformConstant, ImageType(state, image));
const auto variable = state.image_variables[IR::ImageBindingIndex(*kind)];
return DescriptorElementPointer(state, pointer_type, variable, array_index, *kind, resource,
"storage image descriptor array was not emitted");
"image descriptor array was not emitted");
}
void EmitStorageImageWrite(EmitterState& state, uint32_t resource, uint32_t mip_lod, uint32_t coord,
@@ -206,46 +205,15 @@ void EmitStorageImageWrite(EmitterState& state, uint32_t resource, uint32_t mip_
if (!image.atomic) {
state.builder.RequireCapability(spv::CapabilityStorageImageWriteWithoutFormat);
}
const auto kind = IR::DescriptorBindingForImage(image);
EXIT_IF(!kind.has_value());
const auto array_index = ResourceForDescriptor(state, *kind, resource);
const auto image_type = ImageType(state, image);
const auto pointer_type =
state.builder.Type(spv::OpTypePointer, spv::StorageClassUniformConstant, image_type);
const auto variable = state.image_variables[IR::ImageBindingIndex(*kind)];
const auto LoadAt = [&](uint32_t array_index) {
const auto pointer =
DescriptorElementPointer(state, pointer_type, variable, array_index, *kind, resource,
"storage image descriptor array was not emitted");
const auto descriptor = state.builder.AllocateId();
state.builder.AddFunction(spv::OpLoad, image_type, descriptor, pointer);
return descriptor;
const auto EmitWrite = [&](uint32_t mip) {
state.builder.AddFunction(spv::OpImageWrite, LoadImageDescriptor(state, resource, mip),
coord, texel);
};
if (image.mip_mode != IR::ImageMipMode::DynamicStorage) {
state.builder.AddFunction(spv::OpImageWrite, LoadAt(array_index), coord, texel);
if (image.mip_mode != IR::ImageMipMode::Dynamic) {
EmitWrite(0);
return;
}
if (image.mip_count == 0u) {
ExitDescriptorBindingFailure(state, *kind, resource,
"dynamic storage image has no mip descriptors");
}
const auto merge_label = state.builder.AllocateId();
std::vector<uint32_t> labels(image.mip_count);
std::vector<uint32_t> words {spv::OpSwitch, mip_lod, merge_label};
for (uint32_t mip = 0; mip < image.mip_count; mip++) {
labels[mip] = state.builder.AllocateId();
words.push_back(mip);
words.push_back(labels[mip]);
}
state.builder.AddFunction(spv::OpSelectionMerge, merge_label, spv::SelectionControlMaskNone);
state.builder.AddFunction(words);
for (uint32_t mip = 0; mip < image.mip_count; mip++) {
EmitLabel(state, labels[mip]);
state.builder.AddFunction(spv::OpImageWrite, LoadAt(array_index + mip), coord, texel);
state.builder.AddFunction(spv::OpBranch, merge_label);
}
EmitLabel(state, merge_label);
EmitImageMipSwitch(state, mip_lod, image.mip_count, 0, EmitWrite);
}
spv::ExecutionModel ExecutionModelForStage(ShaderType stage) {
@@ -0,0 +1,348 @@
#include "graphics/shader/recompiler/backend/spirv/spirvEmitterInstructions.h"
namespace Libs::Graphics::ShaderRecompiler::Spirv::Emitter {
namespace {
using Vec3 = std::array<uint32_t, 3>;
uint32_t Extract(EmitterState& s, uint32_t type, uint32_t vector, uint32_t index) {
const auto result = s.builder.AllocateId();
s.builder.AddFunction(spv::OpCompositeExtract, type, result, vector, index);
return result;
}
uint32_t Vector(EmitterState& s, uint32_t type, std::span<const uint32_t> components) {
const auto result = s.builder.AllocateId();
s.builder.AddFunction(spv::OpCompositeConstruct, type, result, components);
return result;
}
uint32_t NodeWord(EmitterState& s, uint32_t block, uint32_t word) {
const auto offset = Binary(s, spv::OpShiftLeftLogical, TypeU32(s), word, ConstantU32(s, 2));
const auto address = Binary(s, spv::OpIAdd, TypeScalarU64(s), block,
Unary(s, spv::OpUConvert, TypeScalarU64(s), offset));
const auto pointer = Unary(s, spv::OpConvertUToPtr, TypePhysicalU32Pointer(s), address);
const auto result = s.builder.AllocateId();
s.builder.AddFunction(spv::OpLoad, TypeU32(s), result, pointer, spv::MemoryAccessAlignedMask, 4u);
return result;
}
uint32_t Triangle(EmitterState& s, uint32_t block, uint32_t kind, uint32_t bary,
Vec3 origin, Vec3 direction) {
const auto f = TypeF32(s), u = TypeU32(s), b = TypeBool(s);
const auto zero = ConstantF32Value(s, 0.0f);
const auto op = [&](spv::Op code, uint32_t x, uint32_t y) {
const auto value = Binary(s, code, f, x, y);
s.builder.AddAnnotation(spv::OpDecorate, value, spv::DecorationNoContraction);
return value;
};
const auto cmp = [&](spv::Op code, uint32_t x, uint32_t y) { return Binary(s, code, b, x, y); };
const auto either = [&](uint32_t x, uint32_t y) { return Binary(s, spv::OpLogicalOr, b, x, y); };
const auto both = [&](uint32_t x, uint32_t y) { return Binary(s, spv::OpLogicalAnd, b, x, y); };
const auto abs = [&](uint32_t x) { return EmitGlsl<GLSLstd450FAbs, IR::Type::F32>(s, x); };
const auto is_kind = [&](uint32_t n) { return cmp(spv::OpIEqual, kind, ConstantU32(s, n)); };
// The four triangles are (0,1,2), (1,3,2), (2,3,4), (2,4,0).
const std::array vertices {
EmitGlsl<GLSLstd450UMin, IR::Type::U32>(s, kind, ConstantU32(s, 2)),
Select(s, u, is_kind(0), ConstantU32(s, 1),
Select(s, u, is_kind(3), ConstantU32(s, 4), ConstantU32(s, 3))),
Select(s, u, is_kind(2), ConstantU32(s, 4),
Select(s, u, is_kind(3), ConstantU32(s, 0), ConstantU32(s, 2))) };
std::array<Vec3, 3> positions;
for (uint32_t i = 0; i < 3; ++i) {
const auto first = Binary(s, spv::OpIMul, u, vertices[i], ConstantU32(s, 3));
for (uint32_t axis = 0; axis < 3; ++axis)
positions[i][axis] = Unary(s, spv::OpBitcast, f,
NodeWord(s, block, Binary(s, spv::OpIAdd, u, first, ConstantU32(s, axis))));
}
const auto flag = NodeWord(s, block, ConstantU32(s, 15));
const auto y_largest = cmp(spv::OpFOrdLessThan, abs(direction[0]), abs(direction[1]));
const auto z_largest = cmp(spv::OpFOrdLessThan,
Select(s, f, y_largest, abs(direction[1]), abs(direction[0])), abs(direction[2]));
const auto rotate = [&](Vec3 value) {
Vec3 result;
for (uint32_t i = 0; i < 3; ++i)
result[i] = Select(s, f, z_largest, value[i],
Select(s, f, y_largest, value[(i + 2) % 3], value[(i + 1) % 3]));
return result;
};
origin = rotate(origin);
direction = rotate(direction);
std::array<Vec3, 3> projected;
for (uint32_t i = 0; i < 3; ++i) {
const auto position = rotate(positions[i]);
Vec3 relative;
for (uint32_t axis = 0; axis < 3; ++axis)
relative[axis] = op(spv::OpFSub, position[axis], origin[axis]);
projected[i] = {op(spv::OpFSub, op(spv::OpFMul, relative[0], direction[2]),
op(spv::OpFMul, direction[0], relative[2])),
op(spv::OpFSub, op(spv::OpFMul, relative[1], direction[2]),
op(spv::OpFMul, direction[1], relative[2])), relative[2]};
}
Vec3 edge, weighted;
for (uint32_t i = 0; i < 3; ++i) {
const auto& first = projected[(i + 1) % 3];
const auto& second = projected[(i + 2) % 3];
edge[i] = op(spv::OpFSub, op(spv::OpFMul, second[0], first[1]),
op(spv::OpFMul, second[1], first[0]));
weighted[i] = op(spv::OpFMul, edge[i], direction[2]);
}
auto numerator = op(spv::OpFAdd, op(spv::OpFAdd,
op(spv::OpFMul, edge[0], projected[0][2]), op(spv::OpFMul, edge[1], projected[1][2])),
op(spv::OpFMul, edge[2], projected[2][2]));
auto denominator = op(spv::OpFAdd, op(spv::OpFAdd, weighted[0], weighted[1]), weighted[2]);
uint32_t negative = ConstantBool(s, false), positive = negative;
for (auto value: edge) {
negative = either(negative, cmp(spv::OpFOrdLessThan, value, zero));
positive = either(positive, cmp(spv::OpFOrdGreaterThan, value, zero));
}
const auto winding = cmp(spv::OpFOrdGreaterThan, denominator, zero);
auto missed = either(both(negative, positive), cmp(spv::OpFOrdEqual, denominator, zero));
missed = either(missed, Unary(s, spv::OpIsNan, b, numerator));
missed = either(missed, cmp(spv::OpFOrdLessThan,
Select(s, f, winding, numerator, Unary(s, spv::OpFNegate, f, numerator)), zero));
for (uint32_t i = 0; i < 3; ++i) {
const auto first = projected[(i + 1) % 3][1], second = projected[(i + 2) % 3][1];
const auto first_zero = cmp(spv::OpFOrdEqual, first, zero);
const auto horizontal = both(first_zero, cmp(spv::OpFOrdEqual, second, zero));
const auto below = either(cmp(spv::OpFOrdLessThan, first, zero),
both(first_zero, cmp(spv::OpFOrdGreaterThan, second, zero)));
const auto right = Binary(s, spv::OpLogicalNotEqual, b, below, winding);
const auto excluded = Select(s, b, horizontal,
cmp(spv::OpFOrdGreaterThan, projected[i][1], zero), right);
missed = either(missed, both(cmp(spv::OpFOrdEqual, edge[i], zero), excluded));
}
const auto remap = Binary(s, spv::OpShiftRightLogical, u, flag,
Binary(s, spv::OpIMul, u, kind, ConstantU32(s, 8)));
const auto coordinate = [&](uint32_t shift) {
const auto index = Binary(s, spv::OpBitwiseAnd, u,
Binary(s, spv::OpShiftRightLogical, u, remap, ConstantU32(s, shift)), ConstantU32(s, 3));
return Unary(s, spv::OpBitcast, u, Select(s, f,
cmp(spv::OpIEqual, index, ConstantU32(s, 1)), weighted[1],
Select(s, f, cmp(spv::OpIEqual, index, ConstantU32(s, 2)), weighted[2], weighted[0])));
};
const std::array result {
Unary(s, spv::OpBitcast, u, Select(s, f, missed, ConstantF32(s, 0x7f800000), numerator)),
Unary(s, spv::OpBitcast, u, Select(s, f, missed, ConstantF32Value(s, 1.0f), denominator)),
Select(s, u, bary, coordinate(0), Binary(s, spv::OpIAdd, u, flag, kind)),
Select(s, u, bary, coordinate(2), Select(s, u, missed, ConstantU32(s, 0), ConstantU32(s, 1))) };
return Vector(s, TypeU32Vector(s, 4), result);
}
std::array<uint32_t, 6> Box(EmitterState& s, uint32_t first, uint32_t second,
uint32_t half, uint32_t child) {
const auto f = TypeF32(s);
const auto half_label = s.builder.AllocateId(), full_label = s.builder.AllocateId();
const auto merge = s.builder.AllocateId();
s.builder.AddFunction(spv::OpSelectionMerge, merge, spv::SelectionControlMaskNone);
s.builder.AddFunction(spv::OpBranchConditional, half, half_label, full_label);
EmitLabel(s, half_label);
std::array<uint32_t, 6> packed, full, result;
for (uint32_t pair = 0; pair < 3; ++pair) {
const auto word = NodeWord(s, first, ConstantU32(s, 4 + child * 3 + pair));
const auto unpacked = EmitGlsl<GLSLstd450UnpackHalf2x16, IR::Type::F32x2>(s, word);
packed[pair * 2] = Extract(s, f, unpacked, 0);
packed[pair * 2 + 1] = Extract(s, f, unpacked, 1);
}
s.builder.AddFunction(spv::OpBranch, merge);
EmitLabel(s, full_label);
for (uint32_t component = 0; component < 6; ++component) {
const auto word = 4 + child * 6 + component;
full[component] = Unary(s, spv::OpBitcast, f,
NodeWord(s, word < 16 ? first : second, ConstantU32(s, word % 16)));
}
s.builder.AddFunction(spv::OpBranch, merge);
EmitLabel(s, merge);
for (uint32_t component = 0; component < 6; ++component) {
result[component] = s.builder.AllocateId();
s.builder.AddFunction(spv::OpPhi, f, result[component], packed[component], half_label,
full[component], full_label);
}
return result;
}
uint32_t Boxes(EmitterState& s, uint32_t first, uint32_t second, uint32_t half,
uint32_t grow, uint32_t sort, uint32_t extent, const Vec3& origin, const Vec3& inverse) {
const auto f = TypeF32(s), u = TypeU32(s), b = TypeBool(s);
const auto zero = ConstantF32Value(s, 0.0f);
std::array<uint32_t, 4> depths, indices;
Vec3 forward;
for (uint32_t axis = 0; axis < 3; ++axis)
forward[axis] = Binary(s, spv::OpFOrdGreaterThanEqual, b, inverse[axis], zero);
for (uint32_t child = 0; child < 4; ++child) {
const auto box = Box(s, first, second, half, child);
Vec3 near, far;
for (uint32_t axis = 0; axis < 3; ++axis) {
const auto lo = Binary(s, spv::OpFMul, f,
Binary(s, spv::OpFSub, f, box[axis], origin[axis]), inverse[axis]);
const auto hi = Binary(s, spv::OpFMul, f,
Binary(s, spv::OpFSub, f, box[axis + 3], origin[axis]), inverse[axis]);
near[axis] = Select(s, f, forward[axis], lo, hi);
far[axis] = Select(s, f, forward[axis], hi, lo);
}
const auto entry = EmitGlsl<GLSLstd450FMax, IR::Type::F32>(s,
EmitGlsl<GLSLstd450FMax, IR::Type::F32>(s, near[0], near[1]), near[2]);
const auto exit = EmitGlsl<GLSLstd450FMin, IR::Type::F32>(s,
EmitGlsl<GLSLstd450FMin, IR::Type::F32>(s, far[0], far[1]), far[2]);
// Valid hits have nonnegative exit distance. Saturate ULP growth at +infinity.
const auto bits = Unary(s, spv::OpBitcast, u, Select(s, f,
Binary(s, spv::OpFOrdEqual, b, exit, zero), zero, exit));
const auto grown_bits = EmitGlsl<GLSLstd450UMin, IR::Type::U32>(s,
Binary(s, spv::OpIAdd, u, bits, grow), ConstantU32(s, 0x7f800000));
const auto grown = Unary(s, spv::OpBitcast, f, grown_bits);
auto hit = Binary(s, spv::OpLogicalAnd, b,
Binary(s, spv::OpFOrdGreaterThanEqual, b, exit, zero),
Binary(s, spv::OpFOrdLessThan, b, entry, extent));
hit = Binary(s, spv::OpLogicalAnd, b, hit,
Binary(s, spv::OpFOrdLessThanEqual, b, entry, grown));
depths[child] = Unary(s, spv::OpBitcast, u, Select(s, f, hit,
Select(s, f, Binary(s, spv::OpFOrdGreaterThan, b, entry, zero), entry, zero),
ConstantF32(s, 0x7f800000)));
indices[child] = Select(s, u, hit, NodeWord(s, first, ConstantU32(s, child)), ConstantU32(s, ~0u));
}
// A select network keeps the original child order when sorting is disabled.
constexpr std::array<std::pair<uint32_t, uint32_t>, 5> network {{{0,1}, {2,3}, {0,2}, {1,3}, {1,2}}};
for (size_t step = 0; step < network.size(); ++step) {
const auto [a, c] = network[step];
const auto swap = Binary(s, spv::OpLogicalAnd, b, sort,
Binary(s, spv::OpUGreaterThan, b, depths[a], depths[c]));
const auto depth = depths[a], index = indices[a];
const auto later = [&](uint32_t position) {
return std::ranges::any_of(std::span(network).subspan(step + 1), [=](auto pair) {
return pair.first == position || pair.second == position;
});
};
if (later(a)) depths[a] = Select(s, u, swap, depths[c], depth);
if (later(c)) depths[c] = Select(s, u, swap, depth, depths[c]);
indices[a] = Select(s, u, swap, indices[c], index);
indices[c] = Select(s, u, swap, index, indices[c]);
}
return Vector(s, TypeU32Vector(s, 4), indices);
}
} // namespace
void DefineBvhIntersect(EmitterState& s) {
if (!s.requirements.bvh) return;
const auto u = TypeU32(s), f = TypeF32(s), b = TypeBool(s), wide = TypeScalarU64(s);
const auto vec4 = TypeU32Vector(s, 4), vec3 = TypeF32Vector(s, 3);
const auto signature = s.builder.Type(spv::OpTypeFunction, vec4, vec4, u, f, vec3, vec3, vec3);
s.bvh_intersect_function = s.builder.AllocateId();
s.builder.AddName(s.bvh_intersect_function, "bvh_intersect");
s.builder.AddFunction(spv::OpFunction, vec4, s.bvh_intersect_function,
spv::FunctionControlMaskNone, signature);
std::array<uint32_t, 6> args;
const std::array types {vec4, u, f, vec3, vec3, vec3};
for (uint32_t i = 0; i < args.size(); ++i) {
args[i] = s.builder.AllocateId();
s.builder.AddFunction(spv::OpFunctionParameter, types[i], args[i]);
}
EmitLabel(s, s.builder.AllocateId());
const auto descriptor = args[0], node = args[1], extent = args[2];
Vec3 origin, direction, inverse;
for (uint32_t i = 0; i < 3; ++i) {
origin[i] = Extract(s, f, args[3], i);
direction[i] = Extract(s, f, args[4], i);
inverse[i] = Extract(s, f, args[5], i);
}
std::array<uint32_t, 4> words;
for (uint32_t i = 0; i < 4; ++i) words[i] = Extract(s, u, descriptor, i);
const auto and_bits = [&](uint32_t value, uint32_t mask) {
return Binary(s, spv::OpBitwiseAnd, u, value, ConstantU32(s, mask));
};
const auto shr = [&](uint32_t value, uint32_t shift) {
return Binary(s, spv::OpShiftRightLogical, u, value, ConstantU32(s, shift));
};
const auto kind = and_bits(node, 7);
const auto triangle = Binary(s, spv::OpULessThan, b, kind, ConstantU32(s, 4));
const auto full = Binary(s, spv::OpIEqual, b, kind, ConstantU32(s, 5));
const auto half = Binary(s, spv::OpIEqual, b, kind, ConstantU32(s, 4));
const auto bary = Binary(s, spv::OpINotEqual, b, and_bits(words[3], 0x01000000), ConstantU32(s, 0));
const auto sort = Binary(s, spv::OpINotEqual, b, and_bits(words[1], 0x80000000), ConstantU32(s, 0));
const auto grow = and_bits(shr(words[1], 23), 0xff);
const auto base = Binary(s, spv::OpShiftLeftLogical, wide,
DeviceAddressFromWords(s, words[0], and_bits(words[1], 0xff)), ConstantDeviceAddress(s, 8));
const auto index = Unary(s, spv::OpUConvert, wide, shr(node, 3));
const auto last_index = Binary(s, spv::OpIAdd, wide, index,
Select(s, wide, full, ConstantDeviceAddress(s, 1), ConstantDeviceAddress(s, 0)));
const auto address = Binary(s, spv::OpIAdd, wide, base,
Binary(s, spv::OpShiftLeftLogical, wide, index, ConstantDeviceAddress(s, 6)));
const auto last_address = Binary(s, spv::OpIAdd, wide, base,
Binary(s, spv::OpShiftLeftLogical, wide, last_index, ConstantDeviceAddress(s, 6)));
auto valid = Binary(s, spv::OpIEqual, b, shr(words[3], 28), ConstantU32(s, 8));
const auto require = [&](uint32_t condition) {
valid = Binary(s, spv::OpLogicalAnd, b, valid, condition);
};
require(Binary(s, spv::OpULessThanEqual, b, kind, ConstantU32(s, 5)));
require(Binary(s, spv::OpULessThanEqual, b, last_index, DeviceAddressFromWords(s, words[2], and_bits(words[3], 0x3ff))));
require(Binary(s, spv::OpULessThan, b, last_address, ConstantDeviceAddress(s, uint64_t{1} << 48)));
require(Unary(s, spv::OpLogicalNot, b, Unary(s, spv::OpIsNan, b, extent)));
for (uint32_t i = 0; i < 3; ++i) {
for (const auto value: {origin[i], direction[i], inverse[i]})
require(Unary(s, spv::OpLogicalNot, b, Unary(s, spv::OpIsNan, b, value)));
require(Unary(s, spv::OpLogicalNot, b, Unary(s, spv::OpIsInf, b, origin[i])));
}
const std::array triangle_invalid {ConstantU32(s, 0x7f800000), ConstantU32(s, 0x3f800000),
ConstantU32(s, ~0u), ConstantU32(s, 0)};
const std::array box_invalid {ConstantU32(s, ~0u), ConstantU32(s, ~0u),
ConstantU32(s, ~0u), ConstantU32(s, ~0u)};
std::array<uint32_t, 4> invalid_words;
for (uint32_t i = 0; i < 4; ++i)
invalid_words[i] = Select(s, u, triangle, triangle_invalid[i], box_invalid[i]);
const auto invalid = Vector(s, vec4, invalid_words);
const auto result = EmitValueOrDefaultIfCondition(s, valid, vec4, invalid, [&] {
const auto first = GetBdaPointer(s, address);
const auto second = EmitValueOrDefaultIfCondition(s, full, wide, ConstantDeviceAddress(s, 0), [&] {
return GetBdaPointer(s, Binary(s, spv::OpIAdd, wide, address, ConstantDeviceAddress(s, 64)));
});
const auto present = Binary(s, spv::OpLogicalAnd, b,
Binary(s, spv::OpINotEqual, b, first, ConstantDeviceAddress(s, 0)),
Binary(s, spv::OpLogicalOr, b, Unary(s, spv::OpLogicalNot, b, full),
Binary(s, spv::OpINotEqual, b, second, ConstantDeviceAddress(s, 0))));
return EmitValueOrDefaultIfCondition(s, present, vec4, invalid, [&] {
const auto tri_label = s.builder.AllocateId(), box_label = s.builder.AllocateId();
const auto merge = s.builder.AllocateId();
s.builder.AddFunction(spv::OpSelectionMerge, merge, spv::SelectionControlMaskNone);
s.builder.AddFunction(spv::OpBranchConditional, triangle, tri_label, box_label);
EmitLabel(s, tri_label);
const auto tri_value = Triangle(s, first, kind, bary, origin, direction);
const auto tri_exit = s.current_label;
s.builder.AddFunction(spv::OpBranch, merge);
EmitLabel(s, box_label);
const auto box_value = Boxes(s, first, second, half, grow, sort, extent, origin, inverse);
const auto box_exit = s.current_label;
s.builder.AddFunction(spv::OpBranch, merge);
EmitLabel(s, merge);
const auto value = s.builder.AllocateId();
s.builder.AddFunction(spv::OpPhi, vec4, value, tri_value, tri_exit, box_value, box_exit);
return value;
});
});
s.builder.AddFunction(spv::OpReturnValue, result);
s.builder.AddFunction(spv::OpFunctionEnd);
}
uint32_t EmitBvhIntersect(ValueEmitContext& ctx, const IR::Inst& inst) {
auto& s = ctx.state;
const auto* ray = ctx.ImageAddress(inst.Arg(1));
if (ray == nullptr || ray->NumArgs() < 11 || s.bvh_intersect_function == 0)
ctx.Fail(inst, "BVH intersection requires eleven address components and its shared function");
const auto vector = [&](uint32_t first) {
Vec3 values;
for (uint32_t i = 0; i < 3; ++i)
values[i] = Unary(s, spv::OpBitcast, TypeF32(s), ctx.Arg(*ray, first + i));
return Vector(s, TypeF32Vector(s, 3), values);
};
const auto descriptor = ctx.Arg(inst, 0), node = ctx.Arg(*ray, 0);
const auto extent = Unary(s, spv::OpBitcast, TypeF32(s), ctx.Arg(*ray, 1));
const auto origin = vector(2), direction = vector(5), inverse = vector(8);
return EmitValueOrDefaultIfCondition(s, ctx.Arg(inst, 2), TypeU32Vector(s, 4),
ConstantU32CompositeZero(s, 4), [&] {
const auto result = s.builder.AllocateId();
s.builder.AddFunction(spv::OpFunctionCall, TypeU32Vector(s, 4), result,
s.bvh_intersect_function, descriptor, node, extent, origin, direction, inverse);
return result;
});
}
} // namespace Libs::Graphics::ShaderRecompiler::Spirv::Emitter
@@ -452,6 +452,11 @@ void EmitSetAttribute(ValueEmitContext& ctx, const IR::Inst& inst) {
if (exp.kind == IR::ExportTargetKind::Null || exp.en == 0u) {
return;
}
// Skip dormant color exports after their valid mask; MRT1 is reserved for logical alpha.
if (state.program.stage == ShaderType::Pixel && exp.kind == IR::ExportTargetKind::Mrt &&
exp.index != 0 && state.input_info.pixel->alpha_blend_source_remap) {
return;
}
EmitIfCondition(state, exec, [&]() {
const auto data = ctx.Arg(inst, 0);
if (exp.kind == IR::ExportTargetKind::Primitive) {
@@ -492,6 +497,18 @@ void EmitSetAttribute(ValueEmitContext& ctx, const IR::Inst& inst) {
const bool uint_output = MrtOutputMode(state, exp) == 7u;
const auto vector_type = uint_output ? TypeU32Vector(state, 4) : TypeF32Vector(state, 4);
auto value = ExportVector(ctx, data, exp, uint_output);
if (state.program.stage == ShaderType::Pixel && exp.kind == IR::ExportTargetKind::Mrt &&
exp.index == 0 && !uint_output && state.input_info.pixel->alpha_blend_source_remap) {
// Broadcast logical alpha before swizzling the primary output.
const auto blend_output =
OutputVariableForExport(state, {.kind = IR::ExportTargetKind::Mrt, .index = 1});
if (blend_output != 0) {
const auto alpha = state.builder.AllocateId();
state.builder.AddFunction(spv::OpVectorShuffle, vector_type, alpha, value, value,
3u, 3u, 3u, 3u);
state.builder.AddFunction(spv::OpStore, blend_output, alpha);
}
}
if (state.program.stage == ShaderType::Pixel && exp.kind == IR::ExportTargetKind::Mrt &&
exp.index < state.input_info.pixel->target_export_mapping.size()) {
const auto mapping = state.input_info.pixel->target_export_mapping[exp.index];
@@ -559,6 +576,10 @@ void EmitVoid(ValueEmitContext&) {}
void EmitBarrier(EmitterState& state) {
const auto tessellation = state.program.stage == ShaderType::TessellationControl;
if (!tessellation && ShaderWorkgroupInput(state.program.stage, state.input_info) == nullptr) {
// Independent graphics invocations have no native workgroup left to synchronize.
return;
}
const auto memory_scope = tessellation ? spv::ScopeInvocation : spv::ScopeWorkgroup;
const auto semantics = tessellation ? spv::MemorySemanticsMaskNone
: spv::MemorySemanticsAcquireReleaseMask |
@@ -632,6 +653,29 @@ uint32_t EmitDppUpdateU32(ValueEmitContext& ctx, const IR::Inst& inst) {
ctx.Arg(inst, 1));
}
uint32_t EmitConditionRef(ValueEmitContext& ctx, const IR::Inst& inst) {
if (ctx.other_half == nullptr) return ctx.Arg(inst, 0);
// A native scalar branch makes one decision for both emulated wave halves.
if (ctx.half != 0) return ctx.other_half->Def(IR::Value(&inst));
const auto kind = inst.Flags<CFG::BranchCondition>();
if (kind == CFG::BranchCondition::ScalarInstruction) return ctx.Arg(inst, 0);
const auto ballot = ctx.Ballot(inst.Arg(0));
const auto low = ctx.state.builder.AllocateId();
const auto high = ctx.state.builder.AllocateId();
const auto combined = ctx.state.builder.AllocateId();
const auto result = ctx.state.builder.AllocateId();
ctx.state.builder.AddFunction(spv::OpCompositeExtract, TypeU32(ctx.state), low, ballot, 0);
ctx.state.builder.AddFunction(spv::OpCompositeExtract, TypeU32(ctx.state), high, ballot, 1);
const bool zero = kind == CFG::BranchCondition::ExecZero ||
kind == CFG::BranchCondition::VccZero || kind == CFG::BranchCondition::SccZero;
ctx.state.builder.AddFunction(zero ? spv::OpBitwiseAnd : spv::OpBitwiseOr,
TypeU32(ctx.state), combined, low, high);
ctx.state.builder.AddFunction(zero ? spv::OpIEqual : spv::OpINotEqual,
TypeBool(ctx.state), result, combined,
ConstantU32(ctx.state, zero ? ~0u : 0u));
return result;
}
uint32_t EmitBallot(ValueEmitContext& ctx, IR::Value predicate) {
return ctx.Ballot(predicate);
}
@@ -3,9 +3,7 @@
#include "graphics/shader/recompiler/frontend/decode/ImageOps.h"
#include <algorithm>
#include <atomic>
#include <bit>
#include <cstdio>
namespace Libs::Graphics::ShaderRecompiler::Spirv::Emitter {
namespace {
@@ -63,10 +61,10 @@ uint32_t AddressF32(ValueEmitContext& ctx, const IR::MemoryInfo& mem, const IR::
: Unary(ctx.state, spv::OpBitcast, TypeF32(ctx.state), value);
}
ImageSampleLayout Layout(const IR::MemoryInfo& mem, ImageDimension dimension) {
ImageSampleLayout Layout(const IR::MemoryInfo& mem) {
ImageSampleLayout layout;
uint32_t cursor = 0;
const auto& info = ImageDimensionInfoFor(dimension);
const auto& info = ImageDimensionInfoFor(mem.image_dimension);
if (HasFlag(mem, Decoder::ImageSampleFlagOffset)) layout.offset = cursor++;
if (HasFlag(mem, Decoder::ImageSampleFlagBias)) layout.bias = cursor++;
if (HasFlag(mem, Decoder::ImageSampleFlagCompare)) layout.dref = cursor++;
@@ -86,6 +84,46 @@ uint32_t ZeroF32(EmitterState& state) {
return ConstantF32(state, 0);
}
uint32_t GatherMip(ValueEmitContext& ctx, const IR::Inst& inst, const IR::MemoryInfo& mem,
const IR::Inst& address, const ImageSampleLayout& layout, uint32_t mip_count) {
auto& state = ctx.state;
const auto Extract = [&](uint32_t word, uint32_t first, uint32_t count) {
return EmitBitFieldUExtract(state, word, ConstantU32(state, first),
ConstantU32(state, count));
};
const auto LodFixed = [&](uint32_t word, uint32_t first) {
return Binary(state, spv::OpFMul, TypeF32(state),
Unary(state, spv::OpConvertUToF, TypeF32(state), Extract(word, first, 12)),
ConstantF32Value(state, 1.0f / 256.0f));
};
const auto sampler_control = ctx.Arg(inst, 6);
const auto preclamp = Binary(state, spv::OpINotEqual, TypeBool(state),
Extract(sampler_control, 28, 1), ConstantU32(state, 0));
const auto half = ConstantF32Value(state, 0.5f);
const auto zero = ZeroF32(state);
const auto base = Unary(state, spv::OpConvertUToF, TypeF32(state),
Extract(ctx.Arg(inst, 4), 12, 4));
const auto minimum = Binary(state, spv::OpFSub, TypeF32(state),
LodFixed(ctx.Arg(inst, 3), 8), base);
const auto last = ConstantF32Value(state, static_cast<float>(mip_count - 1u));
auto lod = Binary(state, spv::OpFAdd, TypeF32(state),
AddressF32(ctx, mem, address, layout.lod),
Select(state, TypeF32(state), preclamp, half, zero));
lod = EmitGlsl<GLSLstd450FClamp, IR::Type::F32>(
state, lod, LodFixed(ctx.Arg(inst, 5), 0), LodFixed(ctx.Arg(inst, 5), 12));
// T# minimum LOD is in physical mip space and applies after the S# view-relative clamp.
lod = EmitGlsl<GLSLstd450FMax, IR::Type::F32>(state, lod, minimum);
lod = EmitGlsl<GLSLstd450FClamp, IR::Type::F32>(state, lod, zero, last);
lod = Binary(state, spv::OpFAdd, TypeF32(state), lod,
Select(state, TypeF32(state), preclamp, zero, half));
// The clamped value is nonnegative, so conversion performs the integer mip rounding.
const auto mip = EmitUMin32(state, Unary(state, spv::OpConvertFToU, TypeU32(state), lod),
ConstantU32(state, mip_count - 1u));
const auto mip_none = Binary(state, spv::OpIEqual, TypeBool(state),
Extract(sampler_control, 26, 2), ConstantU32(state, 0));
return Select(state, TypeU32(state), mip_none, ConstantU32(state, 0), mip);
}
uint32_t CubeAxis(EmitterState& state, uint32_t value) {
return Binary(state, spv::OpFSub, TypeF32(state), value, ConstantF32(state, 0x3f800000u));
}
@@ -104,8 +142,7 @@ uint32_t CubeLayer(EmitterState& state, uint32_t value) {
}
uint32_t CoordF32(ValueEmitContext& ctx, const IR::MemoryInfo& mem, const IR::Inst& address,
uint32_t first, uint32_t components) {
const bool cube = ctx.state.program.info.images.at(mem.resource).cube;
uint32_t first, uint32_t components, bool cube = false) {
auto x = AddressF32(ctx, mem, address, first);
if (components == 1u) return x;
auto y = mem.image_address_components > first + 1u ? AddressF32(ctx, mem, address, first + 1u)
@@ -253,7 +290,7 @@ uint32_t QueryDimensions(ValueEmitContext& ctx, const IR::MemoryInfo& mem,
const IR::Inst& address) {
const auto dimension = ctx.state.program.info.images.at(mem.resource).dimension;
const auto& info = ImageDimensionInfoFor(dimension);
const auto image = LoadSampledImageDescriptor(ctx.state, mem.resource);
const auto image = LoadImageDescriptor(ctx.state, mem.resource);
const auto size = ctx.state.builder.AllocateId();
if (info.multisampled != 0u) {
ctx.state.builder.AddFunction(spv::OpImageQuerySize,
@@ -434,7 +471,7 @@ uint32_t EmitOneDimensionalGatherLz(ValueEmitContext& ctx, const IR::MemoryInfo&
uint32_t coord, Prospero::TextureNumericClass numeric_class) {
auto& state = ctx.state;
state.builder.RequireCapability(spv::CapabilityImageQuery);
const auto image = LoadSampledImageDescriptor(state, mem.resource);
const auto image = LoadImageDescriptor(state, mem.resource);
const auto width = state.builder.AllocateId();
state.builder.AddFunction(spv::OpImageQuerySizeLod, TypeU32(state), width, image,
ConstantU32(state, 0));
@@ -447,7 +484,8 @@ uint32_t EmitOneDimensionalGatherLz(ValueEmitContext& ctx, const IR::MemoryInfo&
Binary(state, spv::OpFMul, TypeF32(state), coord, width_f32),
ConstantF32(state, 0x3f000000u)));
const auto sampled = MakeSampledImage(state, mem.resource, mem.sampler);
const auto sampled = MakeSampledImage(state, mem.resource,
LoadSamplerDescriptor(state, mem.sampler));
const auto vector_type = ImageVectorType(state, numeric_class, 4);
const auto scalar_type = ImageScalarType(state, numeric_class);
const auto component =
@@ -540,19 +578,6 @@ uint32_t StoreTexel(ValueEmitContext& ctx, const IR::MemoryInfo& mem, uint32_t d
return PackImageTexel(ctx, mem, texel);
}
spv::Op ImageAtomicOpcode(IR::ValueOpcode opcode) {
switch (opcode) {
case IR::ValueOpcode::ImageAtomicSwap32: return spv::OpAtomicExchange;
case IR::ValueOpcode::ImageAtomicIAdd32: return spv::OpAtomicIAdd;
case IR::ValueOpcode::ImageAtomicUMin32: return spv::OpAtomicUMin;
case IR::ValueOpcode::ImageAtomicUMax32: return spv::OpAtomicUMax;
case IR::ValueOpcode::ImageAtomicAnd32: return spv::OpAtomicAnd;
case IR::ValueOpcode::ImageAtomicOr32: return spv::OpAtomicOr;
case IR::ValueOpcode::ImageAtomicXor32: return spv::OpAtomicXor;
default: return spv::OpNop;
}
}
} // namespace
void EmitImage(ValueEmitContext& ctx, const IR::Inst& inst) {
@@ -572,11 +597,13 @@ void EmitImage(ValueEmitContext& ctx, const IR::Inst& inst) {
if (op == IR::ValueOpcode::ImageQueryLod) {
state.builder.RequireCapability(spv::CapabilityImageQuery);
const auto dimension = image.dimension;
const auto sampled = MakeSampledImage(state, mem.resource, mem.sampler);
const auto sampled = MakeSampledImage(state, mem.resource,
LoadSamplerDescriptor(state, mem.sampler));
const auto lod = state.builder.AllocateId();
state.builder.AddFunction(
spv::OpImageQueryLod, TypeF32Vector(state, 2), lod, sampled,
CoordF32(ctx, mem, *address, 0, ImageDimensionInfoFor(dimension).spatial_components));
CoordF32(ctx, mem, *address, 0, ImageDimensionInfoFor(dimension).spatial_components,
image.cube));
uint32_t values[4] = {ConstantU32(state, 0), ConstantU32(state, 0), ConstantU32(state, 0),
ConstantU32(state, 0)};
for (uint32_t index = 0; index < 2u; index++) {
@@ -601,7 +628,7 @@ void EmitImage(ValueEmitContext& ctx, const IR::Inst& inst) {
EmitValueOrDefaultIfCondition(
state, condition, TypeU32Vector(state, 4), ConstantU32CompositeZero(state, 4),
[&]() {
const auto descriptor = LoadSampledImageDescriptor(state, mem.resource);
const auto descriptor = LoadImageDescriptor(state, mem.resource);
const auto color = state.builder.AllocateId();
const auto coord = CoordU32(ctx, mem, *address, dimension);
if (dimension_info.multisampled != 0u) {
@@ -625,7 +652,7 @@ void EmitImage(ValueEmitContext& ctx, const IR::Inst& inst) {
const auto dimension = image.dimension;
EmitIfCondition(state, ctx.Arg(inst, 3), [&]() {
const auto mip_lod =
state.program.info.images[mem.resource].mip_mode == IR::ImageMipMode::DynamicStorage
state.program.info.images[mem.resource].mip_mode == IR::ImageMipMode::Dynamic
? LodU32(ctx, mem, *address, dimension)
: 0u;
const auto coord = CoordU32(ctx, mem, *address, dimension);
@@ -637,7 +664,7 @@ void EmitImage(ValueEmitContext& ctx, const IR::Inst& inst) {
if (op == IR::ValueOpcode::ImageSampleRaw || op == IR::ValueOpcode::ImageGatherRaw) {
const auto dimension = image.dimension;
const auto& dimension_info = ImageDimensionInfoFor(dimension);
const auto layout = Layout(mem, dimension);
const auto layout = Layout(mem);
const auto numeric_class = image.numeric_class;
const bool dref = HasFlag(mem, Decoder::ImageSampleFlagCompare);
if (dref && state.program.info.images[mem.resource].conversion_format !=
@@ -645,16 +672,9 @@ void EmitImage(ValueEmitContext& ctx, const IR::Inst& inst) {
ctx.Fail(inst, "uses depth comparison with a packed integer image");
return;
}
const auto coord =
CoordF32(ctx, mem, *address, layout.coord, dimension_info.coordinate_components);
if (op == IR::ValueOpcode::ImageGatherRaw) {
if (HasFlag(mem, Decoder::ImageSampleFlagLod)) {
static std::atomic_flag warned = ATOMIC_FLAG_INIT;
if (!warned.test_and_set(std::memory_order_relaxed)) {
std::fputs("Warning: approximating IMAGE_GATHER4_L at mip level 0; explicit LOD is ignored.\n",
stderr);
}
}
const auto coord = CoordF32(ctx, mem, *address, layout.coord,
dimension_info.coordinate_components, image.cube);
if (dimension == ImageDimension::Dim1D) {
if (dref || !HasFlag(mem, Decoder::ImageSampleFlagLevelZero) ||
HasFlag(mem, Decoder::ImageSampleFlagOffset) ||
@@ -671,44 +691,48 @@ void EmitImage(ValueEmitContext& ctx, const IR::Inst& inst) {
ctx.Fail(inst, "has an unsupported 1D-array gather");
return;
}
const auto sampled = MakeSampledImage(state, mem.resource, mem.sampler);
const auto sample = state.builder.AllocateId();
std::vector<uint32_t> words;
const auto result_numeric_class = dref ? Prospero::TextureNumericClass::Float
: numeric_class;
const auto result_type = ImageVectorType(state, result_numeric_class, 4);
uint32_t component_or_dref;
if (dref) {
auto dref_value = ZeroF32(state);
if (layout.dref != NoImageComponent) {
dref_value = AddressF32(ctx, mem, *address, layout.dref);
}
words = {spv::OpImageDrefGather,
TypeF32Vector(state, 4),
sample,
sampled,
coord,
dref_value};
component_or_dref = layout.dref != NoImageComponent
? AddressF32(ctx, mem, *address, layout.dref)
: ZeroF32(state);
} else {
uint32_t component = 0;
if (ImageConversionFormat(state, mem).format == Prospero::BufferFormat::kInvalid) {
component = ImageGatherComponent(mem.dmask);
}
words = {spv::OpImageGather,
ImageVectorType(state, numeric_class, 4),
sample,
sampled,
coord,
ConstantU32(state, component)};
component_or_dref = ConstantU32(
state, ImageConversionFormat(state, mem).format == Prospero::BufferFormat::kInvalid
? ImageGatherComponent(mem.dmask) : 0u);
}
uint32_t operand_mask = 0;
uint32_t offset = 0;
if (HasFlag(mem, Decoder::ImageSampleFlagGatherHorizontal)) {
words.push_back(spv::ImageOperandsConstOffsetsMask);
words.push_back(HorizontalOffsets(state, dimension));
operand_mask = spv::ImageOperandsConstOffsetsMask;
offset = HorizontalOffsets(state, dimension);
} else if (layout.offset != NoImageComponent) {
words.push_back(spv::ImageOperandsOffsetMask);
words.push_back(PackedOffset(ctx, mem, *address, layout, dimension));
}
state.builder.AddFunction(words);
auto result_numeric_class = numeric_class;
if (dref) {
result_numeric_class = Prospero::TextureNumericClass::Float;
operand_mask = spv::ImageOperandsOffsetMask;
offset = PackedOffset(ctx, mem, *address, layout, dimension);
}
const auto sampler_id = LoadSamplerDescriptor(state, mem.sampler);
const auto EmitGather = [&](uint32_t mip) {
const auto sampled = MakeSampledImage(state, mem.resource, sampler_id, mip);
const auto sample = state.builder.AllocateId();
std::vector<uint32_t> words {
static_cast<uint32_t>(dref ? spv::OpImageDrefGather : spv::OpImageGather), result_type,
sample, sampled, coord, component_or_dref};
if (operand_mask != 0u) {
words.push_back(operand_mask);
words.push_back(offset);
}
state.builder.AddFunction(words);
return sample;
};
const auto sample = image.mip_mode == IR::ImageMipMode::Dynamic && image.mip_count > 1u
? EmitImageMipSwitch(
state, GatherMip(ctx, inst, mem, *address, layout,
image.mip_count),
image.mip_count, result_type, EmitGather)
: EmitGather(0);
ctx.Define(inst, ResultVector(ctx, UnpackImageGather(ctx, mem, sample),
result_numeric_class, false, mem, true));
return;
@@ -751,8 +775,14 @@ void EmitImage(ValueEmitContext& ctx, const IR::Inst& inst) {
operand_mask |= spv::ImageOperandsBiasMask;
operands.push_back(AddressF32(ctx, mem, *address, layout.bias));
}
const auto sampler_id = LoadSamplerDescriptor(state, mem.sampler);
const auto EmitSample = [&](uint32_t resource) {
const auto sampled = MakeSampledImage(state, resource, mem.sampler);
const auto& candidate = state.program.info.images[resource];
const auto coord =
CoordF32(ctx, mem, *address, layout.coord,
ImageDimensionInfoFor(candidate.dimension).coordinate_components,
candidate.cube);
const auto sampled = MakeSampledImage(state, resource, sampler_id);
const auto sample = state.builder.AllocateId();
std::vector<uint32_t> sample_operands {result_type, sample, sampled, coord};
if (dref) {
@@ -779,11 +809,11 @@ void EmitImage(ValueEmitContext& ctx, const IR::Inst& inst) {
? &state.program.descriptor_sources[image.source]
: nullptr;
if (handle == nullptr || source == nullptr || !source->indirect_image.has_value() ||
source->indirect_image->key_arg >= handle->NumArgs()) {
handle->NumArgs() == 0u) {
ctx.Fail(inst, "has invalid indirect image key provenance");
return;
}
const auto key = ctx.Def(handle->Arg(source->indirect_image->key_arg));
const auto key = ctx.Def(handle->Arg(0));
if (state.flattened_srt_variable == 0 || image.indirect_search_iterations == 0u ||
image.indirect_resources.size() < 2u) {
ctx.Fail(inst, "has no indirect image runtime mapping");
@@ -871,23 +901,31 @@ void EmitImage(ValueEmitContext& ctx, const IR::Inst& inst) {
ctx.Define(inst, ResultVector(ctx, result, numeric_class, dref, mem));
return;
}
const auto atomic_opcode = ImageAtomicOpcode(op);
if (atomic_opcode != spv::OpNop) {
if (image_info.access == IR::ImageAccess::Atomic) {
const auto dimension = image.dimension;
ctx.Define(inst, EmitValueOrZeroIfCondition(state, ctx.Arg(inst, 3), [&]() {
const auto result_type = TypeId(state, inst.GetType());
const auto zero = image.atomic64 ? ConstantU64(state, 0) : ConstantU32(state, 0);
ctx.Define(inst, EmitValueOrDefaultIfCondition(
state, ctx.Arg(inst, inst.NumArgs() - 1), result_type, zero, [&]() {
const auto pointer = state.builder.AllocateId();
const auto pointer_type = state.builder.Type(
spv::OpTypePointer, spv::StorageClassImage, TypeU32(state));
spv::OpTypePointer, spv::StorageClassImage,
image.atomic64 ? TypeScalarU64(state) : TypeU32(state));
state.builder.AddFunction(spv::OpImageTexelPointer, pointer_type, pointer,
StorageImageDescriptorPointer(state, mem.resource),
ImageDescriptorPointer(state, mem.resource),
CoordU32(ctx, mem, *address, dimension),
ConstantU32(state, 0));
const auto old = state.builder.AllocateId();
state.builder.AddFunction(atomic_opcode, TypeU32(state), old, pointer,
ConstantU32(state, spv::ScopeDevice),
ConstantU32(state, spv::MemorySemanticsMaskNone),
ctx.Arg(inst, 2));
EmitDeviceAtomicMemoryBarrier(state);
if (op == IR::ValueOpcode::ImageAtomicFMin32 ||
op == IR::ValueOpcode::ImageAtomicFMax32) {
return AtomicUpdate(state, pointer, IR::ResourceKind::Image,
[&](uint32_t old) {
return EmitFloatAtomicReplacement(
state, old, ctx.Arg(inst, 2),
op == IR::ValueOpcode::ImageAtomicFMax32);
});
}
const auto old = EmitAtomicOperation(ctx, inst, pointer, spv::ScopeDevice);
EmitAtomicMemoryBarrier(state, IR::ResourceKind::Image);
return old;
}));
return;
@@ -2,6 +2,8 @@
#include "graphics/shader/recompiler/backend/spirv/spirvEmitterInternal.h"
#include <bit>
namespace Libs::Graphics::ShaderRecompiler::Spirv::Emitter {
// Operand signatures distinguish SPIR-V IDs (uint32_t), raw literals/values
// (IR::Value), and instructions whose metadata or operand loads must stay lazy.
@@ -13,14 +15,25 @@ inline constexpr auto EmitConvertU32U16 = EmitBitCastU16F16;
inline constexpr auto EmitConvertU32U8 = EmitBitCastU16F16;
inline constexpr auto EmitBitCastU32F32 = EmitBitCastU16F16;
EMIT_NATIVE(BitCastF32U32, OpBitcast, F32, uint32_t)
EMIT_NATIVE(BitCastU64F64, OpBitcast, U64, uint32_t)
EMIT_NATIVE(BitCastF64U64, OpBitcast, F64, uint32_t)
uint32_t EmitConvertU16U32(EmitterState& state, uint32_t arg0);
uint32_t EmitConvertU8U32(EmitterState& state, uint32_t arg0);
uint32_t EmitConvertF16F32(EmitterState& state, uint32_t arg0);
inline constexpr auto EmitConvertF32F16 = EmitF16BitsToF32;
uint32_t EmitConvertS32F32(EmitterState& state, uint32_t arg0);
uint32_t EmitConvertU32F32(EmitterState& state, uint32_t arg0);
uint32_t EmitConvertF32S32(EmitterState& state, uint32_t arg0);
template <IR::Type type>
uint32_t EmitConvertSigned32ToFloat(EmitterState& state, uint32_t arg0) {
const auto signed_value = Unary(state, spv::OpBitcast, TypeI32(state), arg0);
return EmitNative<spv::OpConvertSToF, type>(state, signed_value);
}
inline constexpr auto EmitConvertF32S32 = EmitConvertSigned32ToFloat<IR::Type::F32>;
inline constexpr auto EmitConvertF64S32 = EmitConvertSigned32ToFloat<IR::Type::F64>;
uint32_t EmitConvertF32F64(EmitterState& state, uint32_t arg0);
uint32_t EmitConvertF64F32(EmitterState& state, uint32_t arg0);
EMIT_NATIVE(ConvertF32U32, OpConvertUToF, F32, uint32_t)
EMIT_NATIVE(ConvertF64U32, OpConvertUToF, F64, uint32_t)
EMIT_NATIVE(CompositeConstructU64, OpCompositeConstruct, U64, uint32_t, uint32_t)
EMIT_NATIVE(CompositeConstructU32x2, OpCompositeConstruct, U32x2, uint32_t, uint32_t)
EMIT_NATIVE(CompositeConstructU32x3, OpCompositeConstruct, U32x3, uint32_t, uint32_t, uint32_t)
@@ -98,28 +111,55 @@ uint32_t EmitIEqual64(EmitterState& state, uint32_t arg0, uint32_t arg1);
uint32_t EmitINotEqual64(EmitterState& state, uint32_t arg0, uint32_t arg1);
uint32_t EmitULessThan64(EmitterState& state, uint32_t arg0, uint32_t arg1);
uint32_t EmitSLessThan64(EmitterState& state, uint32_t arg0, uint32_t arg1);
uint32_t EmitSLessThanEqual64(EmitterState& state, uint32_t arg0, uint32_t arg1);
uint32_t EmitULessThanEqual64(EmitterState& state, uint32_t arg0, uint32_t arg1);
uint32_t EmitUGreaterThanEqual64(EmitterState& state, uint32_t arg0, uint32_t arg1);
uint32_t EmitUGreaterThan64(EmitterState& state, uint32_t arg0, uint32_t arg1);
EMIT_NATIVE(LogicalOr, OpLogicalOr, U1, uint32_t, uint32_t)
EMIT_NATIVE(LogicalAnd, OpLogicalAnd, U1, uint32_t, uint32_t)
EMIT_NATIVE(LogicalXor, OpLogicalNotEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(LogicalNot, OpLogicalNot, U1, uint32_t)
EMIT_NATIVE(FPOrdEqual32, OpFOrdEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordEqual32, OpFUnordEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdNotEqual32, OpFOrdNotEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordNotEqual32, OpFUnordNotEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdLessThan32, OpFOrdLessThan, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordLessThan32, OpFUnordLessThan, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdGreaterThan32, OpFOrdGreaterThan, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordGreaterThan32, OpFUnordGreaterThan, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdLessThanEqual32, OpFOrdLessThanEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordLessThanEqual32, OpFUnordLessThanEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdGreaterThanEqual32, OpFOrdGreaterThanEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordGreaterThanEqual32, OpFUnordGreaterThanEqual, U1, uint32_t, uint32_t)
template <spv::Op opcode>
uint32_t EmitFloatCompare32(ValueEmitContext& ctx, const IR::Inst& inst) {
// SPIR-V's DenormFlushToZero mode does not require comparison operands to flush.
const bool flush = inst.Flags<IR::FPCompareFlags>().flush_input_denorms;
const auto operand = [&](size_t index) {
const auto value = inst.Arg(index);
if (flush && value.IsImmediate()) {
const auto bits = std::bit_cast<uint32_t>(value.F32Value());
return ConstantF32(ctx.state, (bits & 0x7fffffffu) < 0x00800000u
? bits & 0x80000000u : bits);
}
const auto id = ctx.Arg(inst, index);
return flush ? EmitFlushF32DenormToSignedZero(ctx.state, id) : id;
};
return EmitNative<opcode, IR::Type::U1>(ctx.state, operand(0), operand(1));
}
inline constexpr auto EmitFPOrdEqual32 = EmitFloatCompare32<spv::OpFOrdEqual>;
EMIT_NATIVE(FPOrdEqual64, OpFOrdEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdLessThanEqual64, OpFOrdLessThanEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdGreaterThanEqual64, OpFOrdGreaterThanEqual, U1, uint32_t, uint32_t)
inline constexpr auto EmitFPUnordEqual32 = EmitFloatCompare32<spv::OpFUnordEqual>;
inline constexpr auto EmitFPOrdNotEqual32 = EmitFloatCompare32<spv::OpFOrdNotEqual>;
inline constexpr auto EmitFPUnordNotEqual32 = EmitFloatCompare32<spv::OpFUnordNotEqual>;
inline constexpr auto EmitFPOrdLessThan32 = EmitFloatCompare32<spv::OpFOrdLessThan>;
inline constexpr auto EmitFPUnordLessThan32 = EmitFloatCompare32<spv::OpFUnordLessThan>;
inline constexpr auto EmitFPOrdGreaterThan32 = EmitFloatCompare32<spv::OpFOrdGreaterThan>;
inline constexpr auto EmitFPUnordGreaterThan32 = EmitFloatCompare32<spv::OpFUnordGreaterThan>;
inline constexpr auto EmitFPOrdLessThanEqual32 = EmitFloatCompare32<spv::OpFOrdLessThanEqual>;
inline constexpr auto EmitFPUnordLessThanEqual32 = EmitFloatCompare32<spv::OpFUnordLessThanEqual>;
inline constexpr auto EmitFPOrdGreaterThanEqual32 = EmitFloatCompare32<spv::OpFOrdGreaterThanEqual>;
inline constexpr auto EmitFPUnordGreaterThanEqual32 = EmitFloatCompare32<spv::OpFUnordGreaterThanEqual>;
uint32_t EmitFPIsNan32(EmitterState& state, uint32_t arg0);
inline constexpr auto EmitFPCmpClass32 = EmitClassMaskF32;
EMIT_NATIVE(FPAdd32, OpFAdd, F32, uint32_t, uint32_t)
EMIT_NATIVE(FPSub32, OpFSub, F32, uint32_t, uint32_t)
EMIT_NATIVE(FPMul32, OpFMul, F32, uint32_t, uint32_t)
EMIT_NATIVE(FPAdd64, OpFAdd, F64, uint32_t, uint32_t)
EMIT_NATIVE(FPMul64, OpFMul, F64, uint32_t, uint32_t)
inline constexpr auto EmitFPFma64 =
EmitGlsl<GLSLstd450Fma, IR::Type::F64, uint32_t, uint32_t, uint32_t>;
uint32_t EmitFPRecip64(EmitterState& state, uint32_t arg0);
inline constexpr auto EmitFPFma32 =
EmitGlsl<GLSLstd450Fma, IR::Type::F32, uint32_t, uint32_t, uint32_t>;
uint32_t EmitFPMin32(EmitterState& state, uint32_t arg0, uint32_t arg1);
@@ -169,6 +209,7 @@ uint32_t EmitDppUpdateU32(ValueEmitContext& ctx, const IR::Inst& in
uint32_t EmitWqmU64(EmitterState& state, uint32_t value);
uint32_t EmitLaneId(EmitterState& state);
uint32_t EmitBallot(ValueEmitContext& ctx, IR::Value predicate);
uint32_t EmitConditionRef(ValueEmitContext& ctx, const IR::Inst& inst);
uint32_t EmitReadFirstLane(ValueEmitContext& ctx, const IR::Inst& inst);
uint32_t EmitReadLane(ValueEmitContext& ctx, const IR::Inst& inst);
uint32_t EmitWriteLane(ValueEmitContext& ctx, const IR::Inst& inst);
@@ -188,6 +229,7 @@ void EmitLoadMemory(ValueEmitContext& ctx, const IR::Inst& inst
void EmitStoreMemory(ValueEmitContext& ctx, const IR::Inst& inst);
uint32_t EmitAtomic32(ValueEmitContext& ctx, const IR::Inst& inst);
uint32_t EmitBufferAtomic64(ValueEmitContext& ctx, const IR::Inst& inst);
void EmitSharedAtomic64(ValueEmitContext& ctx, const IR::Inst& inst);
uint32_t EmitBufferFloatAtomic(ValueEmitContext& ctx, const IR::Inst& inst);
void EmitSharedFloatAtomic(ValueEmitContext& ctx, const IR::Inst& inst);
uint32_t EmitSharedIncDec(ValueEmitContext& ctx, const IR::Inst& inst);
@@ -223,6 +265,7 @@ inline constexpr auto EmitBufferAtomicUMin32 = EmitAtomic32;
inline constexpr auto EmitBufferAtomicSMax32 = EmitAtomic32;
inline constexpr auto EmitBufferAtomicUMax32 = EmitAtomic32;
inline constexpr auto EmitBufferAtomicAnd32 = EmitAtomic32;
inline constexpr auto EmitBufferAtomicAnd64 = EmitBufferAtomic64;
inline constexpr auto EmitBufferAtomicOr32 = EmitAtomic32;
inline constexpr auto EmitBufferAtomicOr64 = EmitBufferAtomic64;
inline constexpr auto EmitBufferAtomicXor32 = EmitAtomic32;
@@ -244,6 +287,7 @@ inline constexpr auto EmitSharedAtomicFMin32 = EmitSharedFloatAtomic;
inline constexpr auto EmitSharedAtomicFMax32 = EmitSharedFloatAtomic;
inline constexpr auto EmitSharedAtomicSwap32 = EmitAtomic32;
inline constexpr auto EmitSharedAtomicIAdd32 = EmitAtomic32;
inline constexpr auto EmitSharedAtomicIAdd64 = EmitSharedAtomic64;
inline constexpr auto EmitSharedAtomicISub32 = EmitAtomic32;
inline constexpr auto EmitSharedAtomicInc32 = EmitSharedIncDec;
inline constexpr auto EmitSharedAtomicDec32 = EmitSharedIncDec;
@@ -253,24 +297,32 @@ inline constexpr auto EmitSharedAtomicSMax32 = EmitAtomic32;
inline constexpr auto EmitSharedAtomicUMax32 = EmitAtomic32;
inline constexpr auto EmitSharedAtomicAnd32 = EmitAtomic32;
inline constexpr auto EmitSharedAtomicOr32 = EmitAtomic32;
inline constexpr auto EmitSharedAtomicOr64 = EmitSharedAtomic64;
inline constexpr auto EmitSharedAtomicXor32 = EmitAtomic32;
inline constexpr auto EmitDataAppend = EmitAppendConsume;
inline constexpr auto EmitDataConsume = EmitAppendConsume;
uint32_t EmitSwizzleU32(ValueEmitContext& ctx, const IR::Inst& inst);
void EmitImage(ValueEmitContext& ctx, const IR::Inst& inst);
uint32_t EmitBvhIntersect(ValueEmitContext& ctx, const IR::Inst& inst);
inline constexpr auto EmitImageQueryDimensions = EmitImage;
inline constexpr auto EmitImageQueryLod = EmitImage;
inline constexpr auto EmitImageRead = EmitImage;
inline constexpr auto EmitImageWrite = EmitImage;
inline constexpr auto EmitImageSampleRaw = EmitImage;
inline constexpr auto EmitImageGatherRaw = EmitImage;
inline constexpr auto EmitImageAtomicCompareSwap32 = EmitImage;
inline constexpr auto EmitImageAtomicSwap32 = EmitImage;
inline constexpr auto EmitImageAtomicIAdd32 = EmitImage;
inline constexpr auto EmitImageAtomicSMin32 = EmitImage;
inline constexpr auto EmitImageAtomicUMin32 = EmitImage;
inline constexpr auto EmitImageAtomicSMax32 = EmitImage;
inline constexpr auto EmitImageAtomicUMax32 = EmitImage;
inline constexpr auto EmitImageAtomicUMax64 = EmitImage;
inline constexpr auto EmitImageAtomicAnd32 = EmitImage;
inline constexpr auto EmitImageAtomicOr32 = EmitImage;
inline constexpr auto EmitImageAtomicXor32 = EmitImage;
inline constexpr auto EmitImageAtomicFMin32 = EmitImage;
inline constexpr auto EmitImageAtomicFMax32 = EmitImage;
void EmitUnreachable(ValueEmitContext& ctx, const IR::Inst& inst);
inline constexpr auto EmitPhi = EmitUnreachable;
inline constexpr auto EmitTessellationBase = EmitUnreachable;
@@ -19,6 +19,7 @@
#include <map>
#include <spirv/unified1/GLSL.std.450.h>
#include <string>
#include <type_traits>
#include <unordered_map>
#include <utility>
#include <vector>
@@ -58,6 +59,7 @@ constexpr std::array<ImageDimensionInfo, 7> ImageDimensions {{
const ImageDimensionInfo& ImageDimensionInfoFor(ImageDimension dimension);
struct SpirvRequirements {
bool bvh = false;
bool subgroup_ballot = false;
bool subgroup_shuffle = false;
bool subgroup_local_invocation_id = false;
@@ -67,6 +69,9 @@ struct SpirvRequirements {
bool function_scratch = false;
bool pixel_valid_mask = false;
bool buffer_int64_atomics = false;
bool shared_int64_atomics = false;
bool coherent_buffers = false;
bool float64 = false;
};
SpirvRequirements AnalyzeProgramRequirements(const IR::Program& program);
@@ -80,6 +85,20 @@ struct EmitterState {
Builder builder;
const IR::Program& program;
ShaderStageInputInfo input_info;
uint32_t void_type = 0;
uint32_t bool_type = 0;
uint32_t u32_type = 0;
uint32_t native_u64_type = 0;
uint32_t i32_type = 0;
uint32_t f32_type = 0;
uint32_t f64_type = 0;
uint32_t u32_pair_type = 0;
uint32_t i32_pair_type = 0;
uint32_t function_type = 0;
std::array<uint32_t, 3> bool_vector_types {};
std::array<uint32_t, 3> u32_vector_types {};
std::array<uint32_t, 3> i32_vector_types {};
std::array<uint32_t, 3> f32_vector_types {};
std::array<uint32_t, 6> tess_variables {};
uint32_t tess_inner_variable = 0;
uint32_t tess_patch_base = 0;
@@ -93,12 +112,14 @@ struct EmitterState {
uint32_t bda_pagetable_variable = 0;
uint32_t fault_buffer_variable = 0;
uint32_t bda_pointer_function = 0;
uint32_t bvh_intersect_function = 0;
uint32_t gds_variable = 0;
uint32_t gds_length = 0;
uint32_t push_constant_variable = 0;
uint32_t shader_data_storage_variable = 0;
uint32_t flattened_srt_variable = 0;
uint32_t lds_variable = 0;
uint32_t lds_u64_variable = 0;
std::array<uint32_t, 2> scratch_variable {};
std::array<uint32_t, IR::ImageBindingCount> image_variables {};
uint32_t sampler_variable = 0;
@@ -138,6 +159,7 @@ uint32_t TypeU32Pair(EmitterState& state);
uint32_t TypeI32(EmitterState& state);
uint32_t TypeI32Pair(EmitterState& state);
uint32_t TypeF32(EmitterState& state);
uint32_t TypeF64(EmitterState& state);
uint32_t TypeU32Vector(EmitterState& state, uint32_t components);
uint32_t TypeU32Composite(EmitterState& state, uint32_t components);
@@ -166,6 +188,10 @@ template <spv::Op opcode, IR::Type type, typename... Args>
uint32_t EmitNative(EmitterState& state, Args... args) {
const auto result = state.builder.AllocateId();
state.builder.AddFunction(opcode, TypeId(state, type), result, args...);
if constexpr (type == IR::Type::F64 &&
(opcode == spv::OpFMul || opcode == spv::OpFDiv || opcode == spv::OpExtInst)) {
state.builder.AddAnnotation(spv::OpDecorate, result, spv::DecorationNoContraction);
}
return result;
}
@@ -289,13 +315,14 @@ uint32_t ImageType(EmitterState& state, const IR::ImageResource& image);
uint32_t ImageViewSizeType(EmitterState& state, ImageDimension dimension);
uint32_t LoadSampledImageDescriptor(EmitterState& state, uint32_t resource);
uint32_t ImageDescriptorPointer(EmitterState& state, uint32_t resource, uint32_t mip = 0);
uint32_t LoadImageDescriptor(EmitterState& state, uint32_t resource, uint32_t mip = 0);
uint32_t LoadSamplerDescriptor(EmitterState& state, uint32_t sampler);
uint32_t MakeSampledImage(EmitterState& state, uint32_t resource, uint32_t sampler);
uint32_t StorageImageDescriptorPointer(EmitterState& state, uint32_t resource);
uint32_t MakeSampledImage(EmitterState& state, uint32_t resource, uint32_t sampler_id,
uint32_t mip = 0);
void EmitStorageImageWrite(EmitterState& state, uint32_t resource, uint32_t mip_lod, uint32_t coord,
uint32_t texel);
@@ -369,14 +396,16 @@ Prospero::BufferFormat StorageBufferFormat(const EmitterState& state, const IR::
void EmitMemoryOffsets(EmitterState& state);
uint32_t LdsDwordCount(const EmitterState& state);
void EnsureLdsStorage(EmitterState& state);
struct MemoryResourceAccess {
IR::ResourceKind kind = IR::ResourceKind::None;
uint32_t object_pointer = 0;
uint32_t length = 0;
uint32_t index_offset = 0;
uint32_t byte_offset = 0;
bool add_index_offset = false;
IR::ResourceKind kind = IR::ResourceKind::None;
uint32_t object_pointer = 0;
uint32_t length = 0;
uint32_t index_offset = 0;
uint32_t byte_offset = 0;
bool add_index_offset = false;
spv::MemoryAccessMask memory_access = spv::MemoryAccessMaskNone;
};
MemoryResourceAccess PrepareMemoryResourceAccess(EmitterState& state, const IR::MemoryInfo& mem);
@@ -409,7 +438,12 @@ uint32_t EmitUFloatToF32Bits(EmitterState& state, uint32_t raw, uint32_t bits);
uint32_t NormalizeFormatComponent(EmitterState& state, const Format::BufferFormatInfo& info,
uint32_t component, uint32_t raw);
void EmitDeviceAtomicMemoryBarrier(EmitterState& state);
spv::Op SpirvAtomicOpcode(IR::ValueOpcode opcode);
uint32_t EmitAtomicOperation(ValueEmitContext& ctx, const IR::Inst& inst, uint32_t pointer,
uint32_t scope);
void EmitAtomicMemoryBarrier(EmitterState& state, IR::ResourceKind kind);
uint32_t EmitFloatAtomicReplacement(EmitterState& state, uint32_t old, uint32_t source,
bool max_value);
@@ -476,8 +510,51 @@ uint32_t EmitF16BitsToF32(EmitterState& state, uint32_t bits);
void EmitProgram(EmitterState& state);
void DefineGetBdaPointer(EmitterState& state);
void DefineBvhIntersect(EmitterState& state);
uint32_t GetBdaPointer(EmitterState& state, uint32_t address);
uint32_t ConstantDeviceAddress(EmitterState& state, uint64_t value);
uint32_t DeviceAddressFromWords(EmitterState& state, uint32_t low, uint32_t high);
// These templates accept local lambdas from several emitter translation units.
template <typename Fn>
auto EmitImageMipSwitch(EmitterState& state, uint32_t mip_lod, uint32_t mip_count,
uint32_t result_type, Fn&& emit) {
constexpr bool has_result = !std::is_void_v<std::invoke_result_t<Fn, uint32_t>>;
EXIT_IF(mip_count == 0u);
const auto merge_label = state.builder.AllocateId();
std::vector<uint32_t> labels(mip_count);
std::vector<uint32_t> words {spv::OpSwitch, mip_lod, merge_label};
for (uint32_t mip = 0; mip < mip_count; mip++) {
labels[mip] = state.builder.AllocateId();
words.push_back(mip);
words.push_back(labels[mip]);
}
if constexpr (has_result) {
words[2] = labels.front();
}
state.builder.AddFunction(spv::OpSelectionMerge, merge_label, spv::SelectionControlMaskNone);
state.builder.AddFunction(words);
std::vector<uint32_t> phi;
if constexpr (has_result) {
phi = {spv::OpPhi, result_type, state.builder.AllocateId()};
}
for (uint32_t mip = 0; mip < mip_count; mip++) {
EmitLabel(state, labels[mip]);
if constexpr (has_result) {
phi.push_back(emit(mip));
phi.push_back(state.current_label);
} else {
emit(mip);
}
state.builder.AddFunction(spv::OpBranch, merge_label);
}
EmitLabel(state, merge_label);
if constexpr (has_result) {
state.builder.AddFunction(phi);
return phi[2];
}
}
template <typename Fn>
void EmitIfCondition(EmitterState& state, uint32_t condition, Fn&& fn) {
const auto then_label = state.builder.AllocateId();
@@ -494,22 +571,18 @@ template <typename Fn>
uint32_t EmitValueOrDefaultIfCondition(EmitterState& state, uint32_t condition, uint32_t type,
uint32_t default_value, Fn&& fn) {
const auto then_label = state.builder.AllocateId();
const auto then_exit = state.builder.AllocateId();
const auto else_label = state.builder.AllocateId();
const auto header = state.current_label;
const auto merge_label = state.builder.AllocateId();
state.builder.AddFunction(spv::OpSelectionMerge, merge_label, spv::SelectionControlMaskNone);
state.builder.AddFunction(spv::OpBranchConditional, condition, then_label, else_label);
state.builder.AddFunction(spv::OpBranchConditional, condition, then_label, merge_label);
EmitLabel(state, then_label);
const auto then_value = fn();
state.builder.AddFunction(spv::OpBranch, then_exit);
EmitLabel(state, then_exit);
state.builder.AddFunction(spv::OpBranch, merge_label);
EmitLabel(state, else_label);
const auto then_exit = state.current_label;
state.builder.AddFunction(spv::OpBranch, merge_label);
EmitLabel(state, merge_label);
const auto value = state.builder.AllocateId();
state.builder.AddFunction(spv::OpPhi, type, value, then_value, then_exit, default_value,
else_label);
header);
return value;
}
@@ -521,14 +594,7 @@ uint32_t EmitValueOrZeroIfCondition(EmitterState& state, uint32_t condition, Fn&
template <typename Fn>
uint32_t AtomicUpdate(EmitterState& state, uint32_t pointer, IR::ResourceKind kind, Fn&& desired) {
const auto scope = kind == IR::ResourceKind::Lds ? spv::ScopeWorkgroup : spv::ScopeDevice;
const auto memory = [&] {
switch (kind) {
case IR::ResourceKind::Lds: return spv::MemorySemanticsWorkgroupMemoryMask;
case IR::ResourceKind::Image: return spv::MemorySemanticsImageMemoryMask;
default: return spv::MemorySemanticsUniformMemoryMask;
}
}();
const auto scope = kind == IR::ResourceKind::Lds ? spv::ScopeWorkgroup : spv::ScopeDevice;
const auto preheader = state.builder.AllocateId();
const auto header = state.builder.AllocateId();
const auto cont = state.builder.AllocateId();
@@ -557,8 +623,7 @@ uint32_t AtomicUpdate(EmitterState& state, uint32_t pointer, IR::ResourceKind ki
EmitLabel(state, cont);
state.builder.AddFunction(spv::OpBranch, header);
EmitLabel(state, merge);
state.builder.AddFunction(spv::OpMemoryBarrier, ConstantU32(state, scope),
ConstantU32(state, spv::MemorySemanticsAcquireReleaseMask | memory));
EmitAtomicMemoryBarrier(state, kind);
return observed;
}
@@ -1,6 +1,7 @@
#include "graphics/shader/recompiler/backend/spirv/spirvEmitterInstructions.h"
#include "graphics/host_gpu/renderer/cache/bufferCache.h"
#include "graphics/shader/recompiler/frontend/decode/ShaderDecoder.h"
#include <algorithm>
@@ -27,61 +28,71 @@ uint32_t EmitDsMaskedLaneRead(EmitterState& state, uint32_t source, uint32_t tar
return Select(state, TypeU32(state), source_active, shuffled, ConstantU32(state, 0));
}
uint32_t BufferByteAddress(ValueEmitContext& ctx, const IR::Inst& inst, const IR::MemoryInfo& mem,
uint32_t index, uint32_t offset, uint32_t soffset) {
auto& state = ctx.state;
const uint32_t packed = StorageBufferPackedStride(state, mem);
const uint32_t stride = packed & 0x3fffu;
const bool swizzle = stride != 0u && ((packed >> 14u) & 1u) != 0u;
if (((packed >> 20u) & 1u) != 0u) {
const auto lane = Binary(state, spv::OpBitwiseAnd, TypeU32(state),
EmitSubgroupLocalInvocationId(state), ConstantU32(state, 63));
index = Binary(state, spv::OpIAdd, TypeU32(state), index, lane);
}
if (mem.offset != 0u) {
offset = Binary(state, spv::OpIAdd, TypeU32(state), offset, ConstantU32(state, mem.offset));
}
struct BufferAddress {
uint32_t offset;
uint32_t byte;
};
uint32_t address = 0;
if (!swizzle) {
if (stride == 0u) {
address = offset;
} else {
const auto indexed = stride == 1u ? index
: Binary(state, spv::OpIMul, TypeU32(state), index,
ConstantU32(state, stride));
address = Binary(state, spv::OpIAdd, TypeU32(state), indexed, offset);
}
} else {
const uint32_t stride_enum = (packed >> 16u) & 3u;
const uint32_t index_stride = 8u << stride_enum;
const auto index_msb = Binary(state, spv::OpShiftRightLogical, TypeU32(state), index,
ConstantU32(state, stride_enum + 3u));
const auto index_lsb = Binary(state, spv::OpBitwiseAnd, TypeU32(state), index,
ConstantU32(state, index_stride - 1u));
const auto offset_msb =
BufferAddress CalculateBufferAddress(EmitterState& state, uint32_t index, uint32_t offset,
uint32_t soffset, uint32_t immediate, uint32_t stride,
uint32_t swizzle, uint32_t index_stride) {
const auto zero = ConstantU32(state, 0);
const auto one = ConstantU32(state, 1);
const auto add = [&](uint32_t lhs, uint32_t rhs) {
return lhs == zero ? rhs : rhs == zero ? lhs
: Binary(state, spv::OpIAdd, TypeU32(state), lhs, rhs);
};
const auto mul = [&](uint32_t lhs, uint32_t rhs) {
return lhs == zero || rhs == zero ? zero
: lhs == one ? rhs
: rhs == one ? lhs
: Binary(state, spv::OpIMul, TypeU32(state), lhs, rhs);
};
if (immediate != 0u) {
offset = add(offset, ConstantU32(state, immediate));
}
auto address = add(mul(index, stride), offset);
if (swizzle != 0u) {
const auto index_shift =
Binary(state, spv::OpIAdd, TypeU32(state), index_stride, ConstantU32(state, 3));
const auto indices = Binary(state, spv::OpShiftLeftLogical, TypeU32(state),
ConstantU32(state, 1), index_shift);
const auto index_msb =
Binary(state, spv::OpShiftRightLogical, TypeU32(state), index, index_shift);
const auto index_lsb =
Binary(state, spv::OpBitwiseAnd, TypeU32(state), index,
Binary(state, spv::OpISub, TypeU32(state), indices, ConstantU32(state, 1)));
const auto offset_msb =
Binary(state, spv::OpBitwiseAnd, TypeU32(state), offset, ConstantU32(state, ~3u));
const auto offset_lsb =
Binary(state, spv::OpBitwiseAnd, TypeU32(state), offset, ConstantU32(state, 3u));
const auto indexed_msb = stride == 1u ? index_msb
: Binary(state, spv::OpIMul, TypeU32(state),
index_msb, ConstantU32(state, stride));
const auto msb = Binary(state, spv::OpIMul, TypeU32(state),
Binary(state, spv::OpIAdd, TypeU32(state), indexed_msb, offset_msb),
ConstantU32(state, index_stride));
const auto lsb = Binary(state, spv::OpIAdd, TypeU32(state),
Binary(state, spv::OpShiftLeftLogical, TypeU32(state), index_lsb,
ConstantU32(state, 2u)),
offset_lsb);
address = Binary(state, spv::OpIAdd, TypeU32(state), msb, lsb);
const auto msb = mul(add(mul(index_msb, stride), offset_msb), indices);
const auto lsb = add(Binary(state, spv::OpShiftLeftLogical, TypeU32(state), index_lsb,
ConstantU32(state, 2u)), offset_lsb);
address = Select(state, TypeU32(state), swizzle, add(msb, lsb), address);
}
return {offset, add(address, soffset)};
}
const auto soffset_value = inst.Arg(3).Resolve();
if (soffset_value.IsImmediate() && soffset_value.GetType() == IR::Type::U32 &&
soffset_value.U32() == 0u) {
return address;
uint32_t BufferLane(EmitterState& state) {
return Binary(state, spv::OpBitwiseAnd, TypeU32(state), EmitSubgroupLocalInvocationId(state),
ConstantU32(state, 63));
}
uint32_t BufferByteAddress(ValueEmitContext& ctx, const IR::Inst& inst, const IR::MemoryInfo& mem) {
auto& state = ctx.state;
const auto packed = StorageBufferPackedStride(state, mem);
const auto stride = packed & 0x3fffu;
auto index = ctx.Arg(inst, 1);
if ((packed & (1u << 20u)) != 0u) {
index = Binary(state, spv::OpIAdd, TypeU32(state), index, BufferLane(state));
}
return Binary(state, spv::OpIAdd, TypeU32(state), address, soffset);
const bool swizzle = stride != 0u && (packed & (1u << 14u)) != 0u;
return CalculateBufferAddress(state, index, ctx.Arg(inst, 2), ctx.Arg(inst, 3), mem.offset,
ConstantU32(state, stride),
swizzle ? ConstantBool(state, true) : 0u,
ConstantU32(state, (packed >> 16u) & 3u))
.byte;
}
uint32_t AddU64Low(EmitterState& state, uint32_t low, uint32_t high, uint32_t add_low,
@@ -95,31 +106,23 @@ uint32_t AddU64Low(EmitterState& state, uint32_t low, uint32_t high, uint32_t ad
return result;
}
uint32_t ScratchByteAddress(ValueEmitContext& ctx, const IR::MemoryInfo& mem, uint32_t low,
uint32_t high) {
auto& state = ctx.state;
auto immediate = static_cast<int32_t>(mem.offset);
uint32_t AddAddressOffset(EmitterState& state, const IR::MemoryInfo& mem, uint32_t low,
uint32_t& high) {
const auto immediate = static_cast<int32_t>(mem.offset);
if (immediate == 0) return low;
const auto immediate_low = ConstantU32(state, static_cast<uint32_t>(immediate));
const auto immediate_high = ConstantU32(state, immediate < 0 ? UINT32_MAX : 0u);
low = AddU64Low(state, low, high, immediate_low, immediate_high, high);
return AddU64Low(state, low, high, immediate_low, immediate_high, high);
}
uint32_t ScratchByteAddress(ValueEmitContext& ctx, const IR::MemoryInfo& mem, uint32_t low,
uint32_t high) {
auto& state = ctx.state;
low = AddAddressOffset(state, mem, low, high);
const auto valid = Binary(state, spv::OpIEqual, TypeBool(state), high, ConstantU32(state, 0));
return Select(state, TypeU32(state), valid, low, ConstantU32(state, UINT32_MAX));
}
uint32_t ConstantDeviceAddress(EmitterState& state, uint64_t value) {
return state.builder.Constant(spv::OpConstant, TypeScalarU64(state),
static_cast<uint32_t>(value),
static_cast<uint32_t>(value >> 32u));
}
uint32_t DeviceAddressFromWords(EmitterState& state, uint32_t low, uint32_t high) {
const auto low64 = Unary(state, spv::OpUConvert, TypeScalarU64(state), low);
const auto high64 = Binary(state, spv::OpShiftLeftLogical, TypeScalarU64(state),
Unary(state, spv::OpUConvert, TypeScalarU64(state), high),
ConstantDeviceAddress(state, 32));
return Binary(state, spv::OpBitwiseOr, TypeScalarU64(state), low64, high64);
}
uint32_t GuestAddress(ValueEmitContext& ctx, const IR::Inst& inst, const IR::MemoryInfo& mem) {
auto& state = ctx.state;
auto low = ctx.Arg(inst, 1);
@@ -136,7 +139,12 @@ uint32_t GuestAddress(ValueEmitContext& ctx, const IR::Inst& inst, const IR::Mem
ctx.Fail(inst, "has no address base pair");
return ConstantDeviceAddress(state, 0);
}
const auto base = DeviceAddressFromWords(state, ctx.Arg(*handle, 0), ctx.Arg(*handle, 1));
auto base_low = ctx.Arg(*handle, 0);
if (mem.kind == IR::ResourceKind::ScalarAddress) {
base_low = Binary(state, spv::OpBitwiseAnd, TypeU32(state), base_low,
ConstantU32(state, ~3u));
}
const auto base = DeviceAddressFromWords(state, base_low, ctx.Arg(*handle, 1));
address = Binary(state, spv::OpIAdd, TypeScalarU64(state), base,
Unary(state, spv::OpUConvert, TypeScalarU64(state), low));
}
@@ -171,17 +179,9 @@ void RecordBdaFault(EmitterState& state, uint32_t page) {
Binary(state, spv::OpBitwiseOr, TypeU32(state), value, bit));
}
uint32_t GetBdaPointer(ValueEmitContext& ctx, uint32_t address) {
auto& state = ctx.state;
const auto result = state.builder.AllocateId();
state.builder.AddFunction(spv::OpFunctionCall, TypeScalarU64(state), result,
state.bda_pointer_function, address);
return result;
}
uint32_t LoadBdaDword(ValueEmitContext& ctx, uint32_t address) {
auto& state = ctx.state;
const auto bda = GetBdaPointer(ctx, address);
const auto bda = GetBdaPointer(state, address);
const auto present =
Binary(state, spv::OpINotEqual, TypeBool(state), bda, ConstantDeviceAddress(state, 0));
return EmitValueOrZeroIfCondition(state, present, [&]() {
@@ -196,11 +196,8 @@ uint32_t LoadBdaDword(ValueEmitContext& ctx, uint32_t address) {
});
}
uint32_t LoadBda(ValueEmitContext& ctx, const IR::Inst& inst, const IR::MemoryInfo& mem,
uint32_t bits) {
auto& state = ctx.state;
const auto address = GuestAddress(ctx, inst, mem);
const auto active = ctx.Arg(inst, inst.NumArgs() - 1);
uint32_t LoadBda(ValueEmitContext& ctx, uint32_t address, uint32_t active, uint32_t bits) {
auto& state = ctx.state;
return EmitValueOrZeroIfCondition(state, active, [&]() {
const auto aligned = Binary(state, spv::OpBitwiseAnd, TypeScalarU64(state), address,
ConstantDeviceAddress(state, ~uint64_t {3}));
@@ -236,8 +233,7 @@ uint32_t LoadBda(ValueEmitContext& ctx, const IR::Inst& inst, const IR::MemoryIn
uint32_t ByteAddress(ValueEmitContext& ctx, const IR::Inst& inst, const IR::MemoryInfo& mem) {
if (mem.kind == IR::ResourceKind::Buffer) {
return BufferByteAddress(ctx, inst, mem, ctx.Arg(inst, 1), ctx.Arg(inst, 2),
ctx.Arg(inst, 3));
return BufferByteAddress(ctx, inst, mem);
}
if (mem.kind == IR::ResourceKind::Lds || mem.kind == IR::ResourceKind::Gds) {
if (mem.offset == 0u) {
@@ -307,7 +303,8 @@ uint32_t LoadWordInBounds(ValueEmitContext& ctx, const MemoryResourceAccess& res
uint32_t index) {
const auto value = ctx.state.builder.AllocateId();
const auto pointer = EmitMemoryElementPointer(ctx.state, resource, index);
ctx.state.builder.AddFunction(spv::OpLoad, TypeU32(ctx.state), value, pointer);
ctx.state.builder.AddFunction(spv::OpLoad, TypeU32(ctx.state), value, pointer,
resource.memory_access);
return value;
}
@@ -492,21 +489,65 @@ void StoreSubword(ValueEmitContext& ctx, const IR::Inst& inst, IR::MemoryInfo me
});
}
void StoreWordInBounds(ValueEmitContext& ctx, const MemoryResourceAccess& resource, uint32_t index,
uint32_t data) {
ctx.state.builder.AddFunction(spv::OpStore,
EmitMemoryElementPointer(ctx.state, resource, index), data,
resource.memory_access);
}
template <typename Fn>
void ForEachLocalFlatAccess(ValueEmitContext& ctx, const IR::Inst& inst, Fn&& emit) {
auto& state = ctx.state;
auto mem = ctx.Memory(inst);
auto high = ctx.Arg(inst, 2);
const auto low = AddAddressOffset(state, mem, ctx.Arg(inst, 1), high);
const auto index = Binary(state, spv::OpShiftRightLogical, TypeU32(state), low, ConstantU32(state, 2));
const auto aligned = Binary(state, spv::OpIEqual, TypeBool(state), ConstantU32(state, 0),
Binary(state, spv::OpBitwiseAnd, TypeU32(state), low, ConstantU32(state, 3)));
for (const auto kind: {IR::ResourceKind::Scratch, IR::ResourceKind::Lds}) {
mem.kind = kind;
const auto resource = PrepareMemoryResourceAccess(state, mem);
const auto aperture = kind == IR::ResourceKind::Scratch ? Decoder::PrivateApertureHigh
: Decoder::SharedApertureHigh;
const auto selected = Binary(state, spv::OpIEqual, TypeBool(state), high, ConstantU32(state, aperture));
const auto valid = AndCondition(state, aligned,
AndCondition(state, selected, EmitMemoryElementInBounds(state, resource, index)));
emit(resource, index, valid);
}
}
uint32_t LoadLocalFlat(ValueEmitContext& ctx, const IR::Inst& inst) {
return EmitValueOrZeroIfCondition(ctx.state, ctx.Arg(inst, inst.NumArgs() - 1), [&]() {
uint32_t result = 0;
ForEachLocalFlatAccess(ctx, inst, [&](const auto& resource, uint32_t index, uint32_t valid) {
const auto value = EmitValueOrZeroIfCondition(ctx.state, valid, [&]() {
return LoadWordInBounds(ctx, resource, index);
});
result = result == 0 ? value : Binary(ctx.state, spv::OpBitwiseOr, TypeU32(ctx.state), result, value);
});
return result;
});
}
void StoreLocalFlat(ValueEmitContext& ctx, const IR::Inst& inst) {
EmitIfCondition(ctx.state, ctx.Arg(inst, inst.NumArgs() - 1), [&]() {
ForEachLocalFlatAccess(ctx, inst, [&](const auto& resource, uint32_t index, uint32_t valid) {
EmitIfCondition(ctx.state, valid, [&]() {
StoreWordInBounds(ctx, resource, index, ctx.Arg(inst, inst.NumArgs() - 2));
});
});
});
}
void StoreWordPrepared(ValueEmitContext& ctx, const IR::Inst& inst, const IR::MemoryInfo& mem,
const MemoryResourceAccess& resource, uint32_t data) {
const auto index = EmitMemoryElementIndex(ctx.state, resource, DwordIndex(ctx, inst, mem));
EmitIfCondition(ctx.state, EmitMemoryElementInBounds(ctx.state, resource, index), [&]() {
ctx.state.builder.AddFunction(spv::OpStore,
EmitMemoryElementPointer(ctx.state, resource, index), data);
StoreWordInBounds(ctx, resource, index, data);
});
}
void StoreWordInBounds(ValueEmitContext& ctx, const MemoryResourceAccess& resource, uint32_t index,
uint32_t data) {
ctx.state.builder.AddFunction(spv::OpStore,
EmitMemoryElementPointer(ctx.state, resource, index), data);
}
void StoreWord(ValueEmitContext& ctx, const IR::Inst& inst, IR::MemoryInfo mem) {
EmitIfCondition(ctx.state, ctx.Arg(inst, inst.NumArgs() - 1), [&]() {
const auto resource = PrepareMemoryResourceAccess(ctx.state, mem);
@@ -514,54 +555,6 @@ void StoreWord(ValueEmitContext& ctx, const IR::Inst& inst, IR::MemoryInfo mem)
});
}
spv::Op SpirvAtomicOpcode(IR::ValueOpcode opcode) {
switch (opcode) {
case IR::ValueOpcode::BufferAtomicCmpSwap32: return spv::OpAtomicCompareExchange;
case IR::ValueOpcode::BufferAtomicSwap32:
case IR::ValueOpcode::BufferAtomicSwap64:
case IR::ValueOpcode::SharedAtomicSwap32: return spv::OpAtomicExchange;
case IR::ValueOpcode::BufferAtomicIAdd32:
case IR::ValueOpcode::SharedAtomicIAdd32: return spv::OpAtomicIAdd;
case IR::ValueOpcode::BufferAtomicISub32:
case IR::ValueOpcode::SharedAtomicISub32: return spv::OpAtomicISub;
case IR::ValueOpcode::BufferAtomicSMin32:
case IR::ValueOpcode::SharedAtomicSMin32: return spv::OpAtomicSMin;
case IR::ValueOpcode::BufferAtomicUMin32:
case IR::ValueOpcode::SharedAtomicUMin32: return spv::OpAtomicUMin;
case IR::ValueOpcode::BufferAtomicSMax32:
case IR::ValueOpcode::SharedAtomicSMax32: return spv::OpAtomicSMax;
case IR::ValueOpcode::BufferAtomicUMax32:
case IR::ValueOpcode::SharedAtomicUMax32: return spv::OpAtomicUMax;
case IR::ValueOpcode::BufferAtomicAnd32:
case IR::ValueOpcode::SharedAtomicAnd32: return spv::OpAtomicAnd;
case IR::ValueOpcode::BufferAtomicOr32:
case IR::ValueOpcode::BufferAtomicOr64:
case IR::ValueOpcode::SharedAtomicOr32: return spv::OpAtomicOr;
case IR::ValueOpcode::BufferAtomicXor32:
case IR::ValueOpcode::SharedAtomicXor32: return spv::OpAtomicXor;
default: return spv::OpNop;
}
}
uint32_t EmitAtomicOperation(ValueEmitContext& ctx, const IR::Inst& inst, uint32_t pointer,
uint32_t scope) {
const auto old = ctx.state.builder.AllocateId();
if (inst.GetOpcode() == IR::ValueOpcode::BufferAtomicCmpSwap32) {
const auto desired = ctx.Arg(inst, inst.NumArgs() - 3);
const auto comparator = ctx.Arg(inst, inst.NumArgs() - 2);
ctx.state.builder.AddFunction(
spv::OpAtomicCompareExchange, TypeU32(ctx.state), old, pointer,
ConstantU32(ctx.state, scope), ConstantU32(ctx.state, spv::MemorySemanticsMaskNone),
ConstantU32(ctx.state, spv::MemorySemanticsMaskNone), desired, comparator);
} else {
const auto value = ctx.Arg(inst, inst.NumArgs() - 2);
ctx.state.builder.AddFunction(SpirvAtomicOpcode(inst.GetOpcode()), TypeU32(ctx.state), old,
pointer, ConstantU32(ctx.state, scope),
ConstantU32(ctx.state, spv::MemorySemanticsMaskNone), value);
}
return old;
}
template <typename Fn>
uint32_t EmitAtomicAccess(ValueEmitContext& ctx, const IR::Inst& inst,
const IR::MemoryInfo& mem, Fn&& operation) {
@@ -699,22 +692,66 @@ uint32_t FormattedOutOfBoundsValue(ValueEmitContext& ctx, const IR::MemoryInfo&
return ConstructU32Composite(ctx.state, components, values);
}
void StoreFormattedInBounds(ValueEmitContext& ctx, const IR::MemoryInfo& mem,
const PreparedFormattedMemory& plan, uint32_t component,
uint32_t data) {
if (component >= plan.info.component_count) return;
const auto bits = plan.info.component_bits[component];
if (plan.info.type == Format::ComponentType::Snorm && bits == 16u) {
uint32_t EncodeFormattedStoreComponent(ValueEmitContext& ctx,
const Format::BufferFormatInfo& info,
uint32_t component, uint32_t data) {
const auto bits = info.component_bits[component];
if (info.type == Format::ComponentType::Unorm) {
const auto value = EmitBitCastF32U32(ctx.state, data);
data = EmitPackSnorm2x16(
ctx.state, EmitCompositeConstructF32x2(ctx.state, value, ConstantF32Value(ctx.state, 0.0f)));
const auto clamped = EmitFPMin32(
ctx.state, EmitFPMax32(ctx.state, value, ConstantF32Value(ctx.state, 0.0f)),
ConstantF32Value(ctx.state, 1.0f));
const auto scaled = EmitFPMul32(
ctx.state, clamped, ConstantF32Value(ctx.state, float((1u << bits) - 1u)));
return Unary(ctx.state, spv::OpConvertFToU, TypeU32(ctx.state),
EmitFPRoundEven32(ctx.state, scaled));
}
if (bits == 8u || bits == 16u) {
StoreSubwordInBounds(ctx, mem, plan.resource, plan.addresses[component],
plan.indices[component], bits, data);
} else {
StoreWordInBounds(ctx, plan.resource, plan.indices[component], data);
if (bits == 16u && (info.type == Format::ComponentType::Snorm ||
info.type == Format::ComponentType::Float)) {
const auto value = EmitBitCastF32U32(ctx.state, data);
const auto pair = EmitCompositeConstructF32x2(ctx.state, value,
ConstantF32Value(ctx.state, 0.0f));
return info.type == Format::ComponentType::Float ? EmitPackHalf2x16(ctx.state, pair)
: EmitPackSnorm2x16(ctx.state, pair);
}
return data;
}
void StoreFormattedPrepared(ValueEmitContext& ctx, const IR::Inst& inst,
const IR::MemoryInfo& mem, const MemoryResourceAccess& resource,
const Format::BufferFormatInfo& info, uint32_t data,
uint32_t components) {
// RDNA2 formatted stores transfer the entire format, zero-filling missing VGPRs.
const auto plan = PrepareFormattedMemory(ctx, inst, mem, resource, info, info.component_count,
FormattedAccess::Store);
EmitIfCondition(ctx.state, plan.in_bounds, [&]() {
auto packed = ConstantU32(ctx.state, 0);
for (uint32_t component = 0; component < info.component_count; component++) {
auto value = ConstantU32(ctx.state, 0);
if (component < components) {
value = data;
if (components != 1u) {
value = ctx.state.builder.AllocateId();
ctx.state.builder.AddFunction(spv::OpCompositeExtract, TypeU32(ctx.state),
value, data, component);
}
}
value = EncodeFormattedStoreComponent(ctx, info, component, value);
const auto bits = info.component_bits[component];
if (info.packed_bitfield) {
packed = EmitBitFieldInsert(
ctx.state, packed, value,
ConstantU32(ctx.state, info.component_bit_offset[component]),
ConstantU32(ctx.state, bits));
} else if (bits == 8u || bits == 16u) {
StoreSubwordInBounds(ctx, mem, resource, plan.addresses[component],
plan.indices[component], bits, value);
} else {
StoreWordInBounds(ctx, resource, plan.indices[component], value);
}
}
if (info.packed_bitfield) StoreWordInBounds(ctx, resource, plan.indices[0], packed);
});
}
void FormattedStore(ValueEmitContext& ctx, const IR::Inst& inst, const IR::MemoryInfo& mem) {
@@ -726,20 +763,90 @@ void FormattedStore(ValueEmitContext& ctx, const IR::Inst& inst, const IR::Memor
StoreWordPrepared(ctx, inst, RebaseRawComponent(mem, 0u), resource, data);
return;
}
const auto plan = PrepareFormattedMemory(ctx, inst, mem, resource, info, 1u,
FormattedAccess::Store);
EmitIfCondition(ctx.state, plan.in_bounds, [&]() {
StoreFormattedInBounds(ctx, mem, plan, 0u, data);
});
StoreFormattedPrepared(ctx, inst, mem, resource, info, data, 1u);
});
}
uint32_t LoadIndirectBuffer(ValueEmitContext& ctx, const IR::Inst& inst, uint32_t components) {
auto& state = ctx.state;
const auto& handle = *inst.Arg(0).ResolveInstruction();
const auto word1 = ctx.Arg(handle, 1);
const auto records = ctx.Arg(handle, 2);
const auto word3 = ctx.Arg(handle, 3);
const auto field = [&](uint32_t word, uint32_t first, uint32_t count) {
return EmitBitFieldUExtract(state, word, ConstantU32(state, first),
ConstantU32(state, count));
};
const auto nonzero = [&](uint32_t value) {
return Binary(state, spv::OpINotEqual, TypeBool(state), value, ConstantU32(state, 0));
};
const auto has_dword = [&](uint32_t offset, uint32_t size) {
return AndCondition(
state,
Binary(state, spv::OpUGreaterThanEqual, TypeBool(state), size, ConstantU32(state, 4)),
Binary(state, spv::OpULessThanEqual, TypeBool(state), offset,
Binary(state, spv::OpISub, TypeU32(state), size, ConstantU32(state, 4))));
};
const auto stride = field(word1, 16, 14);
const auto swizzle = AndCondition(state, nonzero(stride), nonzero(field(word1, 31, 1)));
const auto index_stride = field(word3, 21, 2);
const auto add_tid = nonzero(field(word3, 23, 1));
const auto index =
Binary(state, spv::OpIAdd, TypeU32(state), ctx.Arg(inst, 1),
Select(state, TypeU32(state), add_tid, BufferLane(state), ConstantU32(state, 0)));
const auto soffset = ctx.Arg(inst, 3);
const auto base = DeviceAddressFromWords(state, ctx.Arg(handle, 0), field(word1, 0, 16));
const auto valid_format = nonzero(field(word3, 12, 7));
const auto mode = field(word3, 28, 2);
const auto index_in_bounds = Binary(state, spv::OpULessThan, TypeBool(state), index, records);
const auto scalar_in_bounds =
Binary(state, spv::OpULessThanEqual, TypeBool(state), soffset, records);
const auto raw_records = Binary(state, spv::OpISub, TypeU32(state), records, soffset);
const auto raw_index_in_bounds =
Binary(state, spv::OpULessThan, TypeBool(state), index, raw_records);
std::array<uint32_t, 4> values {};
for (uint32_t component = 0; component < components; component++) {
const auto address = CalculateBufferAddress(state, index, ctx.Arg(inst, 2), soffset,
ctx.Memory(inst).offset + component * 4u,
stride, swizzle, index_stride);
// RDNA2 raw DWORD vectors check each component, unlike formatted accesses.
const auto structured_bounds =
AndCondition(state, index_in_bounds,
Binary(state, spv::OpULessThan, TypeBool(state), address.offset, stride));
// OOB_SELECT=3 reduces NUM_RECORDS by SOFFSET before the offset/index check.
const auto raw_bounds = AndCondition(
state, scalar_in_bounds,
Select(state, TypeBool(state), swizzle,
AndCondition(state, raw_index_in_bounds, has_dword(address.offset, stride)),
has_dword(address.offset, raw_records)));
auto in_bounds =
Select(state, TypeBool(state),
Binary(state, spv::OpIEqual, TypeBool(state), mode, ConstantU32(state, 0)),
structured_bounds, index_in_bounds);
in_bounds = Select(
state, TypeBool(state),
Binary(state, spv::OpULessThan, TypeBool(state), mode, ConstantU32(state, 2)),
in_bounds,
Select(state, TypeBool(state),
Binary(state, spv::OpIEqual, TypeBool(state), mode, ConstantU32(state, 2)),
nonzero(records), raw_bounds));
const auto guest =
Binary(state, spv::OpIAdd, TypeScalarU64(state), base,
Unary(state, spv::OpUConvert, TypeScalarU64(state), address.byte));
values[component] = LoadBda(ctx, guest, AndCondition(state, valid_format, in_bounds), 32u);
}
return ConstructU32Composite(state, components, values);
}
uint32_t LoadWideBuffer(ValueEmitContext& ctx, const IR::Inst& inst, uint32_t components) {
auto& state = ctx.state;
return EmitValueOrDefaultIfCondition(
state, ctx.Arg(inst, inst.NumArgs() - 1), TypeU32Composite(state, components),
ConstantU32CompositeZero(state, components), [&]() {
const auto mem = ctx.Memory(inst);
if (mem.kind == IR::ResourceKind::IndirectBuffer) {
return LoadIndirectBuffer(ctx, inst, components);
}
const auto resource = PrepareMemoryResourceAccess(state, mem);
const auto info = Format::GetFormatInfo(
mem.formatted ? BufferFormat(ctx, mem) : Prospero::BufferFormat::kInvalid);
@@ -774,16 +881,7 @@ void StoreWideBuffer(ValueEmitContext& ctx, const IR::Inst& inst, uint32_t compo
const auto info = Format::GetFormatInfo(
mem.formatted ? BufferFormat(ctx, mem) : Prospero::BufferFormat::kInvalid);
if (info.type != Format::ComponentType::Unknown) {
const auto plan = PrepareFormattedMemory(ctx, inst, mem, resource, info, components,
FormattedAccess::Store);
EmitIfCondition(state, plan.in_bounds, [&]() {
for (uint32_t component = 0; component < components; component++) {
const auto data = state.builder.AllocateId();
state.builder.AddFunction(spv::OpCompositeExtract, TypeU32(state), data,
composite, component);
StoreFormattedInBounds(ctx, mem, plan, component, data);
}
});
StoreFormattedPrepared(ctx, inst, mem, resource, info, composite, components);
return;
}
for (uint32_t component = 0; component < components; component++) {
@@ -843,6 +941,27 @@ void StoreWideShared(ValueEmitContext& ctx, const IR::Inst& inst, uint32_t compo
} // namespace
uint32_t ConstantDeviceAddress(EmitterState& state, uint64_t value) {
return state.builder.Constant(spv::OpConstant, TypeScalarU64(state),
static_cast<uint32_t>(value),
static_cast<uint32_t>(value >> 32u));
}
uint32_t DeviceAddressFromWords(EmitterState& state, uint32_t low, uint32_t high) {
const auto low64 = Unary(state, spv::OpUConvert, TypeScalarU64(state), low);
const auto high64 = Binary(state, spv::OpShiftLeftLogical, TypeScalarU64(state),
Unary(state, spv::OpUConvert, TypeScalarU64(state), high),
ConstantDeviceAddress(state, 32));
return Binary(state, spv::OpBitwiseOr, TypeScalarU64(state), low64, high64);
}
uint32_t GetBdaPointer(EmitterState& state, uint32_t address) {
const auto result = state.builder.AllocateId();
state.builder.AddFunction(spv::OpFunctionCall, TypeScalarU64(state), result,
state.bda_pointer_function, address);
return result;
}
void DefineGetBdaPointer(EmitterState& state) {
if (!state.program.info.uses_dma) {
return;
@@ -858,7 +977,14 @@ void DefineGetBdaPointer(EmitterState& state) {
state.builder.AddFunction(spv::OpFunctionParameter, type, address);
EmitLabel(state, entry_label);
const auto page64 = Binary(state, spv::OpShiftRightLogical, type, address,
const auto extended = Binary(state, spv::OpUGreaterThanEqual, TypeBool(state), address,
ConstantDeviceAddress(state, LibKernel::Memory::kExtendedMemoryBase));
const auto packed = Select(state, type, extended,
Binary(state, spv::OpISub, type, address,
ConstantDeviceAddress(state,
LibKernel::Memory::kExtendedMemoryBase - LOWER_ADDRESS_SIZE)),
address);
const auto page64 = Binary(state, spv::OpShiftRightLogical, type, packed,
ConstantDeviceAddress(state, BufferCache::CACHING_PAGEBITS));
const auto page = Unary(state, spv::OpUConvert, TypeU32(state), page64);
const auto entry_pointer = state.builder.AllocateId();
@@ -899,14 +1025,7 @@ uint32_t EmitAtomic32(ValueEmitContext& ctx, const IR::Inst& inst) {
const auto scope =
mem.kind == IR::ResourceKind::Lds ? spv::ScopeWorkgroup : spv::ScopeDevice;
const auto old = EmitAtomicOperation(ctx, inst, pointer, scope);
if (mem.kind == IR::ResourceKind::Lds) {
const auto semantics =
spv::MemorySemanticsAcquireReleaseMask | spv::MemorySemanticsWorkgroupMemoryMask;
ctx.state.builder.AddFunction(spv::OpMemoryBarrier, ConstantU32(ctx.state, scope),
ConstantU32(ctx.state, semantics));
} else {
EmitDeviceAtomicMemoryBarrier(ctx.state);
}
EmitAtomicMemoryBarrier(ctx.state, mem.kind);
return old;
});
}
@@ -925,21 +1044,41 @@ uint32_t EmitBufferAtomic64(ValueEmitContext& ctx, const IR::Inst& inst) {
return EmitValueOrDefaultIfCondition(
state, EmitMemoryElementInBounds(state, resource, index), TypeU64(state),
ConstantU64(state, 0), [&]() {
const auto value = Unary(state, spv::OpBitcast, TypeScalarU64(state),
ctx.Arg(inst, inst.NumArgs() - 2));
const auto old = state.builder.AllocateId();
state.builder.AddFunction(
SpirvAtomicOpcode(inst.GetOpcode()), TypeScalarU64(state), old,
EmitStorageBufferElementPointer(state, resource, index,
TypeStorageBufferU64ElementPointer(state)),
ConstantU32(state, spv::ScopeDevice),
ConstantU32(state, spv::MemorySemanticsMaskNone), value);
EmitDeviceAtomicMemoryBarrier(state);
return Unary(state, spv::OpBitcast, TypeU64(state), old);
const auto pointer = EmitStorageBufferElementPointer(
state, resource, index, TypeStorageBufferU64ElementPointer(state));
const auto old = EmitAtomicOperation(ctx, inst, pointer, spv::ScopeDevice);
EmitAtomicMemoryBarrier(state, mem.kind);
return old;
});
});
}
void EmitSharedAtomic64(ValueEmitContext& ctx, const IR::Inst& inst) {
auto& state = ctx.state;
const auto& mem = ctx.Memory(inst);
EnsureLdsStorage(state);
EmitIfCondition(state, ctx.Arg(inst, 2), [&]() {
const auto address = Binary(state, spv::OpBitwiseAnd, TypeU32(state),
ByteAddress(ctx, inst, mem), ConstantU32(state, 0xfff8u));
const auto index = Binary(state, spv::OpShiftRightLogical, TypeU32(state), address,
ConstantU32(state, 3u));
const auto in_bounds = Binary(state, spv::OpULessThan, TypeBool(state), index,
ConstantU32(state, LdsDwordCount(state) / 2u));
EmitIfCondition(state, in_bounds, [&]() {
const auto pointer = state.builder.AllocateId();
state.builder.AddFunction(spv::OpAccessChain,
TypePointer(state, spv::StorageClassWorkgroup, TypeScalarU64(state)),
pointer, state.lds_u64_variable, ConstantU32(state, 0), index);
const auto value = Unary(state, spv::OpBitcast, TypeScalarU64(state), ctx.Arg(inst, 1));
state.builder.AddFunction(
SpirvAtomicOpcode(inst.GetOpcode()), TypeScalarU64(state), state.builder.AllocateId(),
pointer, ConstantU32(state, spv::ScopeWorkgroup),
ConstantU32(state, spv::MemorySemanticsAcquireReleaseMask |
spv::MemorySemanticsWorkgroupMemoryMask), value);
});
});
}
uint32_t EmitBufferFloatAtomic(ValueEmitContext& ctx, const IR::Inst& inst) {
const auto& mem = ctx.Memory(inst);
const bool max_value = inst.GetOpcode() == IR::ValueOpcode::BufferAtomicFMax32;
@@ -987,16 +1126,27 @@ uint32_t EmitAppendConsume(ValueEmitContext& ctx, const IR::Inst& inst) {
if (ctx.half == 1) {
return ctx.other_half->Def(IR::Value(const_cast<IR::Inst*>(&inst)));
}
const auto m0 = ctx.Arg(inst, 0);
const auto base =
Binary(state, spv::OpShiftRightLogical, TypeU32(state), m0, ConstantU32(state, 16));
const auto size =
Binary(state, spv::OpBitwiseAnd, TypeU32(state), m0, ConstantU32(state, 0xffffu));
const auto address = Binary(state, spv::OpIAdd, TypeU32(state), base,
ConstantU32(state, ctx.Memory(inst).offset));
const auto mem = ctx.Memory(inst);
const auto offset = mem.offset & 0xfffcu;
auto address = ConstantU32(state, offset);
uint32_t region_bounds = 0;
if (mem.kind == IR::ResourceKind::Gds) {
const auto m0 = ctx.Arg(inst, 0);
const auto base =
Binary(state, spv::OpShiftRightLogical, TypeU32(state), m0, ConstantU32(state, 16));
const auto size =
Binary(state, spv::OpBitwiseAnd, TypeU32(state), m0, ConstantU32(state, 0xffffu));
address = Binary(state, spv::OpIAdd, TypeU32(state), base, address);
// The entire M0 region must fit the PS5's 48 KiB GDS partition.
region_bounds = AndCondition(
state, Binary(state, spv::OpULessThan, TypeBool(state),
ConstantU32(state, offset + 3u), size),
Binary(state, spv::OpULessThanEqual, TypeBool(state),
Binary(state, spv::OpIAdd, TypeU32(state), base, size),
ConstantU32(state, 0xc000u)));
}
const auto raw_index =
Binary(state, spv::OpShiftRightLogical, TypeU32(state), address, ConstantU32(state, 2));
const auto mem = ctx.Memory(inst);
const auto access = PrepareMemoryResourceAccess(state, mem);
const auto index = EmitMemoryElementIndex(state, access, raw_index);
const auto exec = ctx.Arg(inst, 1);
@@ -1015,19 +1165,17 @@ uint32_t EmitAppendConsume(ValueEmitContext& ctx, const IR::Inst& inst) {
: first;
const auto is_first = Binary(state, spv::OpIEqual, TypeBool(state),
EmitSubgroupLocalInvocationId(state), source_lane);
const auto storage_bounds = EmitMemoryElementInBounds(state, access, index);
const auto m0_bounds =
mem.kind == IR::ResourceKind::Gds
? Binary(state, spv::OpINotEqual, TypeBool(state), size, ConstantU32(state, 0))
: Binary(state, spv::OpULessThan, TypeBool(state),
ConstantU32(state, ctx.Memory(inst).offset + 3u), size);
auto bounds = EmitMemoryElementInBounds(state, access, index);
if (mem.kind == IR::ResourceKind::Gds) {
bounds = AndCondition(state, bounds, region_bounds);
}
const auto condition = AndCondition(
state, is_first,
AndCondition(state,
state.lane_count == 2 ? Binary(state, spv::OpINotEqual, TypeBool(state), count,
ConstantU32(state, 0))
: exec,
AndCondition(state, storage_bounds, m0_bounds)));
bounds));
const auto atomic = EmitValueOrZeroIfCondition(state, condition, [&]() {
const auto value = state.builder.AllocateId();
state.builder.AddFunction(append ? spv::OpAtomicIAdd : spv::OpAtomicISub, TypeU32(state),
@@ -1061,10 +1209,11 @@ void EmitReadConstBuffer(ValueEmitContext& ctx, const IR::Inst& inst) {
if (mem.planning_only) return;
auto& state = ctx.state;
mem.kind = IR::ResourceKind::ScalarBuffer;
const auto address = Binary(state, spv::OpIAdd, TypeU32(state), ctx.Arg(inst, 1),
ConstantU32(state, mem.offset));
const auto index =
Binary(state, spv::OpShiftRightLogical, TypeU32(state), address, ConstantU32(state, 2));
auto index = Binary(state, spv::OpShiftRightLogical, TypeU32(state), ctx.Arg(inst, 1),
ConstantU32(state, 2));
if (mem.offset >= 4u) {
index = Binary(state, spv::OpIAdd, TypeU32(state), index, ConstantU32(state, mem.offset >> 2u));
}
const auto access = PrepareMemoryResourceAccess(state, mem);
const auto element = EmitMemoryElementIndex(state, access, index);
const auto condition = EmitMemoryElementInBounds(state, access, element);
@@ -1082,13 +1231,18 @@ void EmitLoadMemory(ValueEmitContext& ctx, const IR::Inst& inst) {
const auto shared_components = IR::SharedComponentCount(op);
const auto address_info = IR::AddressOpcodeInfoOf(op);
uint32_t value;
if (buffer_components > 1u)
if (mem.kind == IR::ResourceKind::FlatLocal)
value = LoadLocalFlat(ctx, inst);
else if (buffer_components > 1u)
value = LoadWideBuffer(ctx, inst, buffer_components);
else if (shared_components > 1u)
value = LoadWideShared(ctx, inst, shared_components);
else if (mem.kind == IR::ResourceKind::ScalarAddress)
value = LoadBdaDword(ctx, GuestAddress(ctx, inst, mem));
else if (address_info.access == IR::AddressAccess::Read &&
mem.kind != IR::ResourceKind::Scratch)
value = LoadBda(ctx, inst, mem, address_info.data_bits);
value = LoadBda(ctx, GuestAddress(ctx, inst, mem), ctx.Arg(inst, inst.NumArgs() - 1),
address_info.data_bits);
else if (op == IR::ValueOpcode::LoadBufferU32 && mem.formatted)
value = FormattedLoad(ctx, inst, mem);
else if (inst.GetType() == IR::Type::U8)
@@ -1106,7 +1260,9 @@ void EmitStoreMemory(ValueEmitContext& ctx, const IR::Inst& inst) {
const auto buffer_components = IR::BufferComponentCount(op);
const auto shared_components = IR::SharedComponentCount(op);
const auto type = inst.Arg(inst.NumArgs() - 2).GetType();
if (buffer_components > 1u)
if (mem.kind == IR::ResourceKind::FlatLocal)
StoreLocalFlat(ctx, inst);
else if (buffer_components > 1u)
StoreWideBuffer(ctx, inst, buffer_components);
else if (shared_components > 1u)
StoreWideShared(ctx, inst, shared_components);
@@ -63,16 +63,34 @@ uint32_t LdsDwordCount(const EmitterState& state) {
return workgroup != nullptr ? workgroup->lds_size_dwords : 8192u;
}
static void EnsureLdsStorage(EmitterState& state) {
void EnsureLdsStorage(EmitterState& state) {
if (state.lds_variable != 0) {
return;
}
if (ShaderWorkgroupInput(state.program.stage, state.input_info) == nullptr) {
EXIT("function LDS was not prepared before SPIR-V function emission\n");
}
state.lds_variable = state.builder.DefineGlobalVariable(
TypeU32ArrayPointer(state, spv::StorageClassWorkgroup, LdsDwordCount(state)),
spv::StorageClassWorkgroup);
const auto define = [&](uint32_t type, uint32_t bytes) {
const auto array = state.builder.DecoratedType(
spv::OpTypeArray, {{spv::OpDecorate, {spv::DecorationArrayStride, bytes}}}, type,
ConstantU32(state, std::max(LdsDwordCount(state) * 4u / bytes, 1u)));
const auto block = state.builder.DecoratedType(
spv::OpTypeStruct, {{spv::OpMemberDecorate, {0, spv::DecorationOffset, 0}},
{spv::OpDecorate, {spv::DecorationBlock}}}, array);
const auto variable = state.builder.DefineGlobalVariable(
TypePointer(state, spv::StorageClassWorkgroup, block), spv::StorageClassWorkgroup);
state.builder.AddAnnotation(spv::OpDecorate, variable, spv::DecorationAliased);
return variable;
};
if (state.requirements.shared_int64_atomics) {
state.lds_variable = define(TypeU32(state), 4u);
state.lds_u64_variable = define(TypeScalarU64(state), 8u);
state.builder.AddName(state.lds_u64_variable, "lds_qwords");
} else {
state.lds_variable = state.builder.DefineGlobalVariable(
TypeU32ArrayPointer(state, spv::StorageClassWorkgroup, LdsDwordCount(state)),
spv::StorageClassWorkgroup);
}
state.builder.AddName(state.lds_variable, "lds_dwords");
}
@@ -86,7 +104,9 @@ MemoryResourceAccess PrepareStorageBufferResourceAccess(EmitterState& state,
}
const auto array_index =
ResourceForDescriptor(state, IR::DescriptorBindingKind::Buffers, mem.resource);
MemoryResourceAccess access {.kind = mem.kind};
MemoryResourceAccess access {
.kind = mem.kind,
.memory_access = mem.coherent ? spv::MemoryAccessVolatileMask : spv::MemoryAccessMaskNone};
access.object_pointer = state.builder.AllocateId();
state.builder.AddFunction(spv::OpAccessChain, pointer_type, access.object_pointer, variable,
ConstantU32(state, array_index));
@@ -165,6 +185,11 @@ uint32_t EmitMemoryElementPointer(EmitterState& state, const MemoryResourceAcces
: ShaderWorkgroupInput(state.program.stage, state.input_info) != nullptr
? spv::StorageClassWorkgroup
: spv::StorageClassFunction;
if (access.kind == IR::ResourceKind::Lds && state.requirements.shared_int64_atomics) {
state.builder.AddFunction(spv::OpAccessChain, TypeU32ElementPointer(state, storage_class),
pointer, access.object_pointer, ConstantU32(state, 0), index);
return pointer;
}
state.builder.AddFunction(spv::OpAccessChain, TypeU32ElementPointer(state, storage_class),
pointer, access.object_pointer, index);
return pointer;
@@ -281,11 +306,83 @@ uint32_t NormalizeFormatComponent(EmitterState& state, const Format::BufferForma
}
}
void EmitDeviceAtomicMemoryBarrier(EmitterState& state) {
const auto semantics =
spv::MemorySemanticsAcquireReleaseMask | spv::MemorySemanticsUniformMemoryMask;
state.builder.AddFunction(spv::OpMemoryBarrier, ConstantU32(state, spv::ScopeDevice),
ConstantU32(state, semantics));
spv::Op SpirvAtomicOpcode(IR::ValueOpcode opcode) {
switch (opcode) {
case IR::ValueOpcode::ImageAtomicCompareSwap32:
case IR::ValueOpcode::BufferAtomicCmpSwap32: return spv::OpAtomicCompareExchange;
case IR::ValueOpcode::ImageAtomicSwap32:
case IR::ValueOpcode::BufferAtomicSwap32:
case IR::ValueOpcode::BufferAtomicSwap64:
case IR::ValueOpcode::SharedAtomicSwap32: return spv::OpAtomicExchange;
case IR::ValueOpcode::ImageAtomicIAdd32:
case IR::ValueOpcode::BufferAtomicIAdd32:
case IR::ValueOpcode::SharedAtomicIAdd64:
case IR::ValueOpcode::SharedAtomicIAdd32: return spv::OpAtomicIAdd;
case IR::ValueOpcode::BufferAtomicISub32:
case IR::ValueOpcode::SharedAtomicISub32: return spv::OpAtomicISub;
case IR::ValueOpcode::ImageAtomicSMin32:
case IR::ValueOpcode::BufferAtomicSMin32:
case IR::ValueOpcode::SharedAtomicSMin32: return spv::OpAtomicSMin;
case IR::ValueOpcode::ImageAtomicUMin32:
case IR::ValueOpcode::BufferAtomicUMin32:
case IR::ValueOpcode::SharedAtomicUMin32: return spv::OpAtomicUMin;
case IR::ValueOpcode::ImageAtomicSMax32:
case IR::ValueOpcode::BufferAtomicSMax32:
case IR::ValueOpcode::SharedAtomicSMax32: return spv::OpAtomicSMax;
case IR::ValueOpcode::ImageAtomicUMax32:
case IR::ValueOpcode::ImageAtomicUMax64:
case IR::ValueOpcode::BufferAtomicUMax32:
case IR::ValueOpcode::SharedAtomicUMax32: return spv::OpAtomicUMax;
case IR::ValueOpcode::ImageAtomicAnd32:
case IR::ValueOpcode::BufferAtomicAnd32:
case IR::ValueOpcode::BufferAtomicAnd64:
case IR::ValueOpcode::SharedAtomicAnd32: return spv::OpAtomicAnd;
case IR::ValueOpcode::ImageAtomicOr32:
case IR::ValueOpcode::BufferAtomicOr32:
case IR::ValueOpcode::BufferAtomicOr64:
case IR::ValueOpcode::SharedAtomicOr64:
case IR::ValueOpcode::SharedAtomicOr32: return spv::OpAtomicOr;
case IR::ValueOpcode::ImageAtomicXor32:
case IR::ValueOpcode::BufferAtomicXor32:
case IR::ValueOpcode::SharedAtomicXor32: return spv::OpAtomicXor;
default: return spv::OpNop;
}
}
uint32_t EmitAtomicOperation(ValueEmitContext& ctx, const IR::Inst& inst, uint32_t pointer,
uint32_t scope) {
const auto opcode = SpirvAtomicOpcode(inst.GetOpcode());
const auto old = ctx.state.builder.AllocateId();
const bool wide = inst.GetType() == IR::Type::U64;
const auto type = wide ? TypeScalarU64(ctx.state) : TypeU32(ctx.state);
if (opcode == spv::OpAtomicCompareExchange) {
const auto desired = ctx.Arg(inst, inst.NumArgs() - 3);
const auto comparator = ctx.Arg(inst, inst.NumArgs() - 2);
ctx.state.builder.AddFunction(
spv::OpAtomicCompareExchange, TypeU32(ctx.state), old, pointer,
ConstantU32(ctx.state, scope), ConstantU32(ctx.state, spv::MemorySemanticsMaskNone),
ConstantU32(ctx.state, spv::MemorySemanticsMaskNone), desired, comparator);
} else {
auto value = ctx.Arg(inst, inst.NumArgs() - 2);
if (wide) value = Unary(ctx.state, spv::OpBitcast, type, value);
ctx.state.builder.AddFunction(opcode, type, old, pointer,
ConstantU32(ctx.state, scope),
ConstantU32(ctx.state, spv::MemorySemanticsMaskNone), value);
}
return wide ? Unary(ctx.state, spv::OpBitcast, TypeU64(ctx.state), old) : old;
}
void EmitAtomicMemoryBarrier(EmitterState& state, IR::ResourceKind kind) {
const auto scope = kind == IR::ResourceKind::Lds ? spv::ScopeWorkgroup : spv::ScopeDevice;
const auto memory = [&] {
switch (kind) {
case IR::ResourceKind::Lds: return spv::MemorySemanticsWorkgroupMemoryMask;
case IR::ResourceKind::Image: return spv::MemorySemanticsImageMemoryMask;
default: return spv::MemorySemanticsUniformMemoryMask;
}
}();
state.builder.AddFunction(spv::OpMemoryBarrier, ConstantU32(state, scope),
ConstantU32(state, spv::MemorySemanticsAcquireReleaseMask | memory));
}
uint32_t EmitFloatAtomicReplacement(EmitterState& state, uint32_t old, uint32_t source,
@@ -1,4 +1,4 @@
#include "graphics/shader/recompiler/backend/spirv/spirvEmitterInternal.h"
#include "graphics/shader/recompiler/backend/spirv/spirvEmitterInstructions.h"
namespace Libs::Graphics::ShaderRecompiler::Spirv::Emitter {
namespace {
@@ -127,10 +127,7 @@ void EmitMeshEntryPoint(EmitterState& state) {
state.builder.AddFunction(spv::OpFunctionCall, TypeVoid(state), state.builder.AllocateId(),
state.mesh_guest_func);
// All guest waves finish before the uniform Vulkan allocation and output stores.
state.builder.AddFunction(spv::OpControlBarrier, ConstantU32(state, spv::ScopeWorkgroup),
ConstantU32(state, spv::ScopeWorkgroup),
ConstantU32(state, spv::MemorySemanticsAcquireReleaseMask |
spv::MemorySemanticsWorkgroupMemoryMask));
EmitBarrier(state);
const auto vertices = MeshLoad(state, state.mesh_allocation, spv::StorageClassWorkgroup,
TypeU32(state), ConstantU32(state, 0));
const auto primitives = MeshLoad(state, state.mesh_allocation, spv::StorageClassWorkgroup,
@@ -6,19 +6,32 @@
namespace Libs::Graphics::ShaderRecompiler::Spirv::Emitter {
uint32_t TypeVoid(EmitterState& state) {
return state.builder.Type(spv::OpTypeVoid);
if (state.void_type == 0) {
state.void_type = state.builder.Type(spv::OpTypeVoid);
}
return state.void_type;
}
uint32_t TypeBool(EmitterState& state) {
return state.builder.Type(spv::OpTypeBool);
if (state.bool_type == 0) {
state.bool_type = state.builder.Type(spv::OpTypeBool);
}
return state.bool_type;
}
uint32_t TypeBoolVector(EmitterState& state, uint32_t components) {
return state.builder.Type(spv::OpTypeVector, TypeBool(state), components);
auto& type = state.bool_vector_types[components - 2u];
if (type == 0) {
type = state.builder.Type(spv::OpTypeVector, TypeBool(state), components);
}
return type;
}
uint32_t TypeU32(EmitterState& state) {
return state.builder.Type(spv::OpTypeInt, 32, 0);
if (state.u32_type == 0) {
state.u32_type = state.builder.Type(spv::OpTypeInt, 32, 0);
}
return state.u32_type;
}
uint32_t TypeU64(EmitterState& state) {
@@ -26,29 +39,55 @@ uint32_t TypeU64(EmitterState& state) {
}
uint32_t TypeScalarU64(EmitterState& state) {
return state.builder.Type(spv::OpTypeInt, 64, 0);
if (state.native_u64_type == 0) {
state.native_u64_type = state.builder.Type(spv::OpTypeInt, 64, 0);
}
return state.native_u64_type;
}
uint32_t TypeU32Pair(EmitterState& state) {
const auto element = TypeU32(state);
return state.builder.Type(spv::OpTypeStruct, element, element);
if (state.u32_pair_type == 0) {
const auto element = TypeU32(state);
state.u32_pair_type = state.builder.Type(spv::OpTypeStruct, element, element);
}
return state.u32_pair_type;
}
uint32_t TypeI32(EmitterState& state) {
return state.builder.Type(spv::OpTypeInt, 32, 1);
if (state.i32_type == 0) {
state.i32_type = state.builder.Type(spv::OpTypeInt, 32, 1);
}
return state.i32_type;
}
uint32_t TypeI32Pair(EmitterState& state) {
const auto element = TypeI32(state);
return state.builder.Type(spv::OpTypeStruct, element, element);
if (state.i32_pair_type == 0) {
const auto element = TypeI32(state);
state.i32_pair_type = state.builder.Type(spv::OpTypeStruct, element, element);
}
return state.i32_pair_type;
}
uint32_t TypeF32(EmitterState& state) {
return state.builder.Type(spv::OpTypeFloat, 32);
if (state.f32_type == 0) {
state.f32_type = state.builder.Type(spv::OpTypeFloat, 32);
}
return state.f32_type;
}
uint32_t TypeF64(EmitterState& state) {
if (state.f64_type == 0) {
state.f64_type = state.builder.Type(spv::OpTypeFloat, 64);
}
return state.f64_type;
}
uint32_t TypeU32Vector(EmitterState& state, uint32_t components) {
return state.builder.Type(spv::OpTypeVector, TypeU32(state), components);
auto& type = state.u32_vector_types[components - 2u];
if (type == 0) {
type = state.builder.Type(spv::OpTypeVector, TypeU32(state), components);
}
return type;
}
uint32_t TypeU32Composite(EmitterState& state, uint32_t components) {
@@ -57,11 +96,19 @@ uint32_t TypeU32Composite(EmitterState& state, uint32_t components) {
}
uint32_t TypeI32Vector(EmitterState& state, uint32_t components) {
return state.builder.Type(spv::OpTypeVector, TypeI32(state), components);
auto& type = state.i32_vector_types[components - 2u];
if (type == 0) {
type = state.builder.Type(spv::OpTypeVector, TypeI32(state), components);
}
return type;
}
uint32_t TypeF32Vector(EmitterState& state, uint32_t components) {
return state.builder.Type(spv::OpTypeVector, TypeF32(state), components);
auto& type = state.f32_vector_types[components - 2u];
if (type == 0) {
type = state.builder.Type(spv::OpTypeVector, TypeF32(state), components);
}
return type;
}
uint32_t TypePointer(EmitterState& state, spv::StorageClass storage_class, uint32_t pointee) {
@@ -69,7 +116,10 @@ uint32_t TypePointer(EmitterState& state, spv::StorageClass storage_class, uint3
}
uint32_t TypeFunction(EmitterState& state) {
return state.builder.Type(spv::OpTypeFunction, TypeVoid(state));
if (state.function_type == 0) {
state.function_type = state.builder.Type(spv::OpTypeFunction, TypeVoid(state));
}
return state.function_type;
}
uint32_t StorageRuntimeArrayType(EmitterState& state) {
@@ -201,6 +251,16 @@ void DefineDescriptors(EmitterState& state) {
state.builder.AddAnnotation(spv::OpDecorate, state.storage_buffer_u64_variable,
spv::DecorationAliased);
}
if (state.requirements.coherent_buffers) {
// RDNA2 stores publish to L2 even without GLC; every alias of the buffer
// must participate in visibility for cache-bypassing polling loads.
state.builder.AddAnnotation(spv::OpDecorate, state.storage_buffer_variable,
spv::DecorationCoherent);
if (state.storage_buffer_u64_variable != 0) {
state.builder.AddAnnotation(spv::OpDecorate, state.storage_buffer_u64_variable,
spv::DecorationCoherent);
}
}
break;
case IR::DescriptorBindingKind::BdaPagetable:
state.bda_pagetable_variable = Define(StorageBufferU64Type(state), "bda_pagetable");
@@ -612,10 +672,15 @@ void DefineModule(EmitterState& state) {
state.builder.RequireCapability(spv::CapabilityPhysicalStorageBufferAddresses);
state.builder.RequireExtension("SPV_KHR_physical_storage_buffer");
}
if (state.requirements.buffer_int64_atomics) {
if (state.requirements.buffer_int64_atomics || state.requirements.shared_int64_atomics) {
state.builder.RequireCapability(spv::CapabilityInt64);
state.builder.RequireCapability(spv::CapabilityInt64Atomics);
}
if (state.requirements.shared_int64_atomics) {
state.builder.RequireVersion(0x00010400u);
state.builder.RequireExtension("SPV_KHR_workgroup_memory_explicit_layout");
state.builder.RequireCapability(spv::CapabilityWorkgroupMemoryExplicitLayoutKHR);
}
if (state.clip_distance_variable != 0) {
state.builder.RequireCapability(spv::CapabilityClipDistance);
}
@@ -669,6 +734,19 @@ void DefineModule(EmitterState& state) {
// contract prevents host compilers from treating synthesized IEEE values as finite.
state.builder.AddExecutionMode(state.main_func, spv::ExecutionModeSignedZeroInfNanPreserve,
32u);
if (state.requirements.float64) {
EXIT_NOT_IMPLEMENTED(state.program.stage == ShaderType::Compute &&
state.input_info.compute->float_mode != 0xc0);
// MODE=0xc0 uses round-to-nearest-even and preserves FP64 input/output denormals.
state.builder.RequireCapability(spv::CapabilityFloat64);
state.builder.RequireCapability(spv::CapabilityRoundingModeRTE);
state.builder.AddExecutionMode(state.main_func, spv::ExecutionModeSignedZeroInfNanPreserve,
64u);
// FP64 denormal preservation is temporarily disabled.
// state.builder.RequireCapability(spv::CapabilityDenormPreserve);
// state.builder.AddExecutionMode(state.main_func, spv::ExecutionModeDenormPreserve, 64u);
state.builder.AddExecutionMode(state.main_func, spv::ExecutionModeRoundingModeRTE, 32u);
}
if (const auto* cs = ShaderWorkgroupInput(state.program.stage, state.input_info)) {
uint32_t local_x = state.requirements.compute_derivatives ? 2u : 1u;
uint32_t local_y = state.requirements.compute_derivatives ? 2u : 1u;
@@ -97,30 +97,6 @@ void EmitReturn(ValueEmitContext& ctx) {
ctx.state.builder.AddFunction(spv::OpReturn);
}
uint32_t BranchCondition(ValueEmitContext& ctx, const IR::BlockInfo& info) {
// Scalar-instruction conditions already test the full wave's raw register values.
if (ctx.other_half == nullptr ||
info.terminator.condition == CFG::BranchCondition::ScalarInstruction ||
info.terminator.condition == CFG::BranchCondition::GotoVariable) {
return ctx.Def(info.condition);
}
const auto ballot = ctx.Ballot(info.condition);
const auto low = ctx.state.builder.AllocateId();
const auto high = ctx.state.builder.AllocateId();
const auto result = ctx.state.builder.AllocateId();
ctx.state.builder.AddFunction(spv::OpCompositeExtract, TypeU32(ctx.state), low, ballot, 0);
ctx.state.builder.AddFunction(spv::OpCompositeExtract, TypeU32(ctx.state), high, ballot, 1);
const auto kind = info.terminator.condition;
const bool zero = kind == CFG::BranchCondition::ExecZero ||
kind == CFG::BranchCondition::VccZero ||
kind == CFG::BranchCondition::SccZero;
const auto combined =
EmitBinaryU32(ctx.state, zero ? spv::OpBitwiseAnd : spv::OpBitwiseOr, low, high);
ctx.state.builder.AddFunction(zero ? spv::OpIEqual : spv::OpINotEqual, TypeBool(ctx.state),
result, combined, ConstantU32(ctx.state, zero ? ~0u : 0u));
return result;
}
void EmitStructuredTerminator(ValueEmitContext& ctx, const IR::Block* block,
const IR::BlockInfo& info) {
const auto& program = ctx.state.program;
@@ -160,7 +136,7 @@ void EmitStructuredTerminator(ValueEmitContext& ctx, const IR::Block* block,
EmitReturn(ctx);
return;
}
const auto condition = BranchCondition(ctx, info);
const auto condition = ctx.Def(info.condition);
emit_merge();
ctx.state.builder.AddFunction(spv::OpBranchConditional, condition,
ctx.Label(true_block), ctx.Label(false_block));
@@ -193,7 +169,7 @@ uint32_t EmitDispatcherNextPc(ValueEmitContext& ctx, const DispatcherFunctionSta
EmitDispatcherTarget(ctx, dispatcher, block, term.false_block);
const auto selected = ctx.state.builder.AllocateId();
ctx.state.builder.AddFunction(
spv::OpSelect, TypeU32(ctx.state), selected, BranchCondition(ctx, info),
spv::OpSelect, TypeU32(ctx.state), selected, ctx.Def(info.condition),
ConstantU32(ctx.state, term.true_block), ConstantU32(ctx.state, term.false_block));
return selected;
}
@@ -267,6 +243,12 @@ void Invoke(Return (*emit)(Context&, Args...), ValueEmitContext& ctx, const IR::
}
void EmitDirectInstruction(ValueEmitContext& ctx, const IR::Inst& inst) {
if (ctx.half != 0 && (inst.GetOpcode() == IR::ValueOpcode::Ballot ||
inst.GetOpcode() == IR::ValueOpcode::ReadFirstLane)) {
// Both operations already combine both emulated halves into one whole-wave result.
ctx.Define(inst, ctx.other_half->Result(inst));
return;
}
switch (inst.GetOpcode()) {
#define VALUE_OPCODE(name, ...) \
case IR::ValueOpcode::name: return Invoke(Emit##name, ctx, inst);
@@ -440,6 +422,7 @@ uint32_t TypeId(EmitterState& state, IR::Type type) {
case IR::Type::U64: return TypeU64(state);
case IR::Type::U32x2: return TypeU32Pair(state);
case IR::Type::F32: return TypeF32(state);
case IR::Type::F64: return TypeF64(state);
case IR::Type::U32x3: return TypeU32Vector(state, 3);
case IR::Type::U32x4: return TypeU32Vector(state, 4);
case IR::Type::F32x2: return TypeF32Vector(state, 2);
@@ -702,6 +685,7 @@ void EmitProgram(EmitterState& state) {
}
}
DefineGetBdaPointer(state);
DefineBvhIntersect(state);
for (const auto* block: program.blocks) {
if (std::ranges::any_of(*block, [](const IR::Inst& inst) {
return inst.GetOpcode() == IR::ValueOpcode::SwizzleU32 ||
Loaded 100 of 252 files, more files were not shown because too many files have changed in this diff. Show more