Compare commits

...
Author SHA1 Message Date
nmzik e930657bf6 tesselation support in GUI 2026-09-20 17:16:19 +04:00
nmzik 0ad9149c90 Fix three-vertex rectangle rendering 2026-09-20 17:16:19 +04:00
nmzik 25f2699523 Fix video aspect ratio 2026-09-20 17:16:19 +04:00
nmzik a849f25376 Make tessellation optional 2026-09-20 17:16:19 +04:00
nmzik fa51da4d76 Fix AvPlayer end-of-file handling 2026-09-20 17:16:19 +04:00
nmzik 6b79b7a003 Fix AvPlayer startup state 2026-09-20 17:16:19 +04:00
nmzik 102a10f0fe Fix signed sampler comparisons 2026-09-20 17:16:19 +04:00
nmzik 8d66c4ef6d user service: return logged-out error for unavailable user names 2026-09-20 14:03:08 +04:00
nmzikandAugusto Domingos 8cecaf4e30 Merge PR #716: Fix Unicode paths on Windows
Preserve native Windows paths for filesystem operations, command-line options,
and game icons. Complete Unicode handling in recursive file listings and file
logging, restore unmounting by host directory, and extend regression coverage.

Original implementation by Augusto Domingos (@AghastyGD); retain all five of
his PR commits and include the reviewed follow-up fixes.

Closes #714

Co-authored-by: Augusto Domingos <augustodomingosvasco@gmail.com>
2026-09-20 02:14:22 +04:00
nmzik 82b2711cdf audio: play ATRAC9 sampler blocks 2026-09-20 01:47:43 +04:00
nmzik 35a275e97d avplayer: signal when playback ends 2026-09-20 01:47:43 +04:00
nmzik 303a0203b6 audio: play NGS2 PCM buffers 2026-09-20 01:47:43 +04:00
cfbaee07f0 Persist SaveDataMemory2 save data to disk (fixes Astro Bot progress not saving) (#659)
* Persist save-data memory across restarts

* Fix SetSaveDataMemory2 data loss, split saves per user, add atomic writes everywhere

- SaveDataSetSaveDataMemory2 treated data_num=0 as "no entries" even when
  a valid data pointer was supplied. Astro Bot calls this with data_num=0
  meaning "one entry, at data[0]" (an implicit-count convention), so every
  real save write from it was silently dropped as a no-op. Default
  data_num to 1 when data is non-null, matching that convention.
- SaveDataSetupSaveDataMemory2/Get/SetSaveDataMemory2 rejected any call
  that included a non-null icon pointer. Icon persistence isn't
  implemented, but titles are allowed to pass one anyway; rejecting the
  whole call over it made every title that does so fail outright. Ignore
  icon instead of rejecting.
- SaveDataMount3/SaveDataTransferringMount/SaveDataDelete built their
  save-data path from title+dir_name only, dropping the user_id PS5
  local users are keyed by (unlike the newer SetSaveDataMemory2 layout,
  which already nests by user_id). Nest these under user_id too so saves
  from different local users can no longer collide. No migration for the
  old flat layout - the project is still pre-release/experimental.
- Classic SaveDataMount3-mounted titles (Blasphemous, Dead Cells, ...)
  write their save files directly through the guest's normal file I/O
  once mounted, bypassing SaveData's own write path entirely, so they
  had none of the crash-safety SetSaveDataMemory2 already had. Added the
  same protection at the file-system layer: a guest file opened under a
  /savedataN mount that recreates the file from scratch (O_CREAT and/or
  O_TRUNC) is now written to a .tmp shadow file and atomically replaces
  the real one on close, instead of being truncated in place.
- Both write paths now back up the file they're about to replace into a
  sibling "backups" folder (same filename, no added suffix) before the
  atomic replace, so a manual recovery copy of the previous save always
  survives a write - even one that "succeeds" but wasn't wanted.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* Fix save-memory compatibility and preserve existing saves

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-20 00:51:22 +04:00
Augusto Domingos fe45265fcd preserve native paths when creating directories 2026-09-19 17:53:15 +02:00
Augusto Domingos 8b294eb5b8 Add unicode path regression coverage 2026-09-19 15:48:30 +02:00
Augusto Domingos 07c9978ac6 keep game icon paths native in windows 2026-09-19 14:45:24 +02:00
Augusto Domingos b61b3f2a40 handle UTF-8 command-line paths 2026-09-19 14:12:35 +02:00
Augusto Domingos 001dfea501 Preserve native windows paths in filesystem operations 2026-09-19 13:11:55 +02:00
nmzik f100f785da libs: fill controller and broadcast status outputs 2026-09-19 05:16:16 +04:00
nmzik ba1854bbdc net: start pool IDs at 1 2026-09-19 04:59:28 +04:00
nmzik ea6aae8577 shader: V_MAD_I16 2026-09-19 04:43:48 +04:00
nmzik 5b7d3343cf shader: V_CMP_NLT_F16 2026-09-18 12:25:13 +04:00
nmzik 394e638905 shader: V_MUL_LO_U16 2026-09-18 11:21:57 +04:00
nmzik c2b7b0b58c Revert "BC7-RGBA32_UINT views (Vulkan validation)"
This reverts commit 165234b159.
2026-09-18 11:12:36 +04:00
nmzik 1d2f59dfa4 ajm: reject invalid batch storage 2026-09-18 09:11:55 +04:00
nmzikandBlackCAT 007d7ea2eb audio: infer stereo RAW AAC layout from channel-pair packets
Co-authored-by: BlackCAT <52720489+psnwd@users.noreply.github.com>
2026-09-18 09:11:55 +04:00
nmzikandshadowbeat070 95d3d9d3ad shader: fix packed integer negation
Co-authored-by: shadowbeat070 <70843316+shadowbeat070@users.noreply.github.com>
2026-09-18 09:11:55 +04:00
bfc8cd0e0b shader: DPP8 moves
Co-authored-by: foufouadi <attar.foued@gmail.com>
Co-authored-by: fxpw <fxpw@outlook.com>
2026-09-18 09:11:55 +04:00
165234b159 BC7-RGBA32_UINT views (Vulkan validation)
Co-authored-by: Techx3 <56268269+Techx3@users.noreply.github.com>
Co-authored-by: shadowbeat070 <70843316+shadowbeat070@users.noreply.github.com>
Co-authored-by: SueryRD <the.ai.tester2@gmail.com>
Co-authored-by: BlackCAT <52720489+psnwd@users.noreply.github.com>
2026-09-18 09:11:55 +04:00
nmzik 6493674726 shader: dual-source blending (Vulkan) 2026-09-18 09:11:55 +04:00
nmzik cbedf5707f UDP P2P sockets 2026-09-18 09:11:55 +04:00
nmzik 57c5b206ca fix network errno 2026-09-18 09:11:55 +04:00
nmzik 87b4417aca fix initial NP sign-in callback 2026-09-18 09:11:55 +04:00
nmzik 104530eafc fix Windows build (#680)
fix Windows Vulkan barrier macro
2026-09-17 21:32:54 +04:00
nmzik 6cfde37e98 avoid extra allocations 2026-09-17 20:35:36 +04:00
nmzik c5cebc1d19 remove unused helpers and the -64 suffix 2026-09-17 20:35:36 +04:00
nmzik 697b18b366 replace ByteBuffer with std::vector 2026-09-17 20:35:36 +04:00
nmzik 5b2935e36c remove the extra virtual memory layer 2026-09-17 20:35:36 +04:00
nmzik 5dc644edb4 clean up graphics helpers 2026-09-17 20:35:36 +04:00
nmzik 1211958b4e clean up runtime helpers 2026-09-17 20:35:36 +04:00
nmzik 218e7200bb clean up launcher helpers 2026-09-17 20:35:36 +04:00
nmzik 2bf78a94e0 remove string wrappers 2026-09-17 20:35:36 +04:00
nmzikandSaadi cc8e02380f remove magic_enum wrappers
Co-authored-by: Saadi <cosxmos@gmail.com>
2026-09-17 20:35:36 +04:00
nmzik c66d977c1d shader: encode SNORM16 formatted stores for skinning buffers 2026-09-17 17:40:05 +04:00
nmzik 4b77686566 shader: V_CMP_NGT_F16 2026-09-17 16:56:32 +04:00
c0sxm0s a6144f794b launcher: report cheat file save failures (#669)
PatchesDialog::Save silently ignored QSaveFile failures, leaving the
status label showing the old "Loaded N cheat(s)" text so the user could
not tell that their selection was not saved. Report the failure in the
status label and the log instead.
2026-09-17 16:13:35 +04:00
Mininaut bab1c3614c libc: treat negative dynamic printf precision as omitted (#625) 2026-09-17 15:18:04 +04:00
c0sxm0s de1545e19a launcher: wire cancel->reject in C++; document glslang for Windows (#663)
uic from Qt 6.12.0 crashes (0xC0000005) on any non-empty <connections>
block in .ui files. Move the cancel_button -> reject connection from
configuration_edit_dialog.ui into configurationEditDialog.cpp, matching
how the other buttons are already connected there.

Also document the glslang (glslangValidator) requirement for Windows
builds; CMake requires it via find_program(... REQUIRED).
2026-09-17 15:13:20 +04:00
Komaryandnmzik ce629e0391 input: add gyroscope and motion sensor support (#634)
input: add controller motion sensor support

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-17 12:28:12 +04:00
BlackCAT 6c03b4aaa5 vscode: point IntelliSense at clang instead of MSVC (#564)
* vscode: point IntelliSense at clang instead of MSVC

KYTY_SYSV_ABI expands to __attribute__((sysv_abi)) with no MSVC fallback,
which is correct for a project that rejects cl.exe at configure time. Nothing
told the C/C++ extension that, so it fell back to windows-msvc-x64 and
flagged every guest-facing function in the tree with "__attribute__ is not a
type name".

Set the IntelliSense mode, compiler path and C++ standard to match the
compiler the build actually uses, and export a compile database for it to
read.

* vscode: resolve the clang toolchain through the VS developer environment

The IntelliSense settings named a specific Visual Studio install, so the
compiler and Ninja only resolved on machines with VS 18 Community under
Program Files. Let CMake Tools source the developer environment instead,
which locates the toolchain through vswhere, and drop the explicit
compilerPath so cpptools takes the compiler from compile_commands.json.

Also point C_Cpp.default.compileCommands at the build directory this
workspace actually configures, and drop CMAKE_EXPORT_COMPILE_COMMANDS
because CMake Tools already exports the database by default.
2026-09-17 11:38:40 +04:00
nmzik 0627e23e92 coderabbit: make it quieter 2026-09-17 11:26:44 +04:00
4e6f4f99d3 loader: emulate immediate SSE4a instructions on macOS (#635)
* loader: emulate immediate SSE4a instructions on macOS

* tests: simplify macOS SSE4a test integration

Keep the Darwin target, CTest registration, and aggregate dependency in
one conditional block without a separate target list. Preserve the
standalone target when BUILD_TESTING is disabled.

Lowercase the test filename and diagnostic prefix, and drop unrelated
formatting changes from the instruction emulator.

Credit Brandon Strong's related SSE4a work in
https://github.com/KytyPS5/KytyPS5/pull/467.

Co-authored-by: Brandon Strong <brandostrong@gmail.com>

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
Co-authored-by: Brandon Strong <brandostrong@gmail.com>
2026-09-17 11:23:50 +04:00
Elijaandnmzik f30afc184e shader: support S_ASHR_I64 (#658)
* shader: implement S_ASHR_I64 with compute coverage

* tests: use existing compute suite for S_ASHR_I64

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-17 10:48:05 +04:00
228cf3fbf4 Build the launcher on Qt below 6.6 and cap the reference-clock log (#653)
* Fix launcher build on Qt versions below 6.6

QVulkanInstance::NoPortabilityDrivers is only available since Qt 6.6, so
guard its use the same way as the Apple-specific case.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Stop logging every reference-clock copy

CopyData of the clock counter runs on every GPU wait, so the handler wrote
one line per wait: a single SpongeBob boot that never reaches a frame logs
1312 of them, and the count grows with run time. Keep the first 64 lines
for diagnostics; the copy itself is unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove reference-clock logging comment

---------

Co-authored-by: YuhiAida <21134775+YuhiAida@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-17 10:30:47 +04:00
nmzik 47957faa16 Match stencil associations by guest range and extent
PPSA30279
2026-09-17 09:01:36 +04:00
nmzik 235e24d44f Recognize null-destination SWAPPC shader handoffs
Credits:
BlackCAT (@psnwd):
https://github.com/KytyPS5/KytyPS5/pull/613
YuhiAida (@YuhiAida):
https://github.com/KytyPS5/KytyPS5/pull/655
2026-09-17 08:45:21 +04:00
nmzik e0ecffed70 Approximate explicit-LOD texture gathers at mip zero
Credits:
YuhiAida (@YuhiAida):
https://github.com/KytyPS5/KytyPS5/pull/588
https://github.com/KytyPS5/KytyPS5/pull/589
https://github.com/KytyPS5/KytyPS5/pull/655
frans (@francoisatt):
https://github.com/KytyPS5/KytyPS5/pull/590
BlackCAT (@psnwd):
https://github.com/KytyPS5/KytyPS5/pull/613
shadowbeat070 (@shadowbeat070):
https://github.com/KytyPS5/KytyPS5/pull/654
Pouare514 (@Pouare514):
https://github.com/KytyPS5/KytyPS5/pull/29
https://github.com/KytyPS5/KytyPS5/pull/32
2026-09-17 08:39:57 +04:00
nmzik fa913ec1a7 Accept EXPCLEAR acceleration on expanded depth attachments 2026-09-17 08:25:55 +04:00
nmzik 49b2eedd1c Fix command buffer capacity calculation
Credits:
YuhiAida (@YuhiAida):
https://github.com/KytyPS5/KytyPS5/pull/588
https://github.com/KytyPS5/KytyPS5/pull/589
https://github.com/KytyPS5/KytyPS5/pull/652
Crimson (@Crimson3076):
https://github.com/KytyPS5/KytyPS5/pull/598
2026-09-17 08:16:59 +04:00
nmzik 63fb822f8a Warn and continue mesh draws with primitive restart enabled 2026-09-16 22:33:19 +04:00
nmzikandÇağatay Güçlü f38d738ee2 Honor PS5 texture minimum LOD for streamed mip chains
Fix broken ground textures in ASTRO BOT.

Related work: https://github.com/KytyPS5/KytyPS5/pull/558

Co-authored-by: Çağatay Güçlü <33038942+defektu@users.noreply.github.com>
2026-09-16 22:33:19 +04:00
nmzikandÇağatay Güçlü eda6ba37ec Json2: new ABIs
Related work: https://github.com/KytyPS5/KytyPS5/pull/558

Co-authored-by: Çağatay Güçlü <33038942+defektu@users.noreply.github.com>
2026-09-16 22:33:19 +04:00
nmzikandÇağatay Güçlü b594f8c834 Bind only exported color targets and preserve MRT slots
Related work: https://github.com/KytyPS5/KytyPS5/pull/558

Co-authored-by: Çağatay Güçlü <33038942+defektu@users.noreply.github.com>
2026-09-16 22:33:19 +04:00
nmzikandÇağatay Güçlü bc45aaf604 Track uniform sampler choices across both conditional arms
Related work: https://github.com/KytyPS5/KytyPS5/pull/558

Co-authored-by: Çağatay Güçlü <33038942+defektu@users.noreply.github.com>
2026-09-16 22:33:19 +04:00
nmzik fb94a819c7 Structure early returns with shared loop continuations 2026-09-16 22:33:19 +04:00
nmzikandÇağatay Güçlü 8589731a27 Preserve PS5 mip-tail views and GPU data during image expansion
Related work: https://github.com/KytyPS5/KytyPS5/pull/558

Co-authored-by: Çağatay Güçlü <33038942+defektu@users.noreply.github.com>
2026-09-16 22:33:19 +04:00
nmzik c1dccd7db1 Skip zero-sized compute dispatches before shader preparation 2026-09-16 22:33:19 +04:00
nmzikandÇağatay Güçlü af3a3f4bf6 Preserve uniform conditional sampler choices in resource plans
Related work: https://github.com/KytyPS5/KytyPS5/pull/558

Co-authored-by: Çağatay Güçlü <33038942+defektu@users.noreply.github.com>
2026-09-16 22:33:19 +04:00
nmzik 349fae00d9 Emulate register-form SSE4a EXTRQ 2026-09-16 22:33:19 +04:00
Fag Fag Fag Sahur 10cae50b34 Add Nix flake and development shell (#645) 2026-09-16 22:19:41 +04:00
nmzik 5b541375a3 shader: implement S_CBRANCH_CDBGSYS opcode 2026-09-16 17:04:48 +04:00
nmzik ded6964953 Use native TLS for fiber context across thread migration 2026-09-16 11:22:05 +02:00
nmzik 9b2111dc01 Bundle FFmpeg 2026-09-16 10:13:03 +02:00
nmzik 71f52f9d90 Simplify fiber ownership and preserve small guest stacks 2026-09-16 08:53:56 +02:00
nmzik 86e5e01403 Protect Windows guest red zones when emulating VRSQRTPS 2026-09-16 06:52:36 +02:00
nmzik 77680ce9d2 Make AMD CPU patch optional 2026-09-16 09:48:50 +04:00
nmzik 32e6a433e8 Fix Json2 integer reads 2026-09-16 09:21:44 +04:00
nmzik 7c9148dd47 Support triangle fans in PS5 geometry shaders
Keep the fan center across subgroups and reuse the existing index fetch.
2026-09-16 08:42:47 +04:00
nmzik 957452d6f9 Emulate VRSQRTPS instruction 2026-09-16 08:28:36 +04:00
nmzikandAdrian Endlicher e6156bef05 Cache CI builds with sccache
Co-authored-by: Adrian Endlicher <godlike93@gmail.com>
2026-09-15 22:16:54 +04:00
nmzik 441367b1a1 Fix pixel shader input aliases 2026-09-15 20:46:51 +04:00
nmzik 629ae1ac05 Handle Z-pass predication and query waits 2026-09-15 20:15:39 +04:00
nmzik 0d9e95d96a Fix centroid interpolation weights
PPSA03792
2026-09-15 19:51:30 +04:00
nmzik 9cf7963f44 Use system FFmpeg for VP9 playback on Linux 2026-09-15 19:30:28 +04:00
nmzik acd7435d31 Support byte destinations in V_MOV_B32 2026-09-15 18:25:20 +04:00
nmzik 7af28c267d Map Z/C to L2/R2 2026-09-15 15:20:53 +04:00
nmzik 79120d53ec Handle chained PM4 buffers without recursion 2026-09-15 14:43:15 +04:00
nmzik 6a2987a7b2 Log the game title id 2026-09-15 14:09:24 +04:00
nmzik d1372abf46 Add launcher microphone selection and SDL AudioIn capture 2026-09-15 11:24:05 +04:00
nmzik 98b5d88d67 Implement AudioIn HQ open and close with paced silent input 2026-09-15 11:24:05 +04:00
nmzik 86414760e6 Cull zero homogeneous vertex positions before rasterization
Fix Playroom top right artifact on Nvidia by clipping zero-position vertices.
2026-09-15 11:24:05 +04:00
nmzik a1fbc41146 Preserve shared scalar loads during embedded vertex fetch replacement 2026-09-15 11:24:05 +04:00
nmzik 5702cfc935 Fix transfer-save mounts to honor the source title and missing data 2026-09-15 11:24:05 +04:00
nmzik 7b5a33f878 Fix PS5 front-face VGPR encoding to restore scene lighting 2026-09-15 07:01:10 +04:00
nmzik ea092a9472 Fix DCC clears through aliased image formats
Clear DCC images using the requested view format.
UNORM alpha 0xffff reads as NaN in a FLOAT view.
2026-09-15 07:01:10 +04:00
nmzikandBipin 18a1e0fadc Remove global condition waiter registry that deadlocks exception delivery
Co-authored-by: Bipin <bipinkrish4@gmail.com>
2026-09-14 21:34:01 +04:00
nmzik 437e69ef1b Remove extra DCC tracking 2026-09-14 20:29:20 +04:00
nmzik dd408ffd00 Resolve image aliases before buffer uploads 2026-09-14 20:29:20 +04:00
nmzik bce8924dd6 Use dynamic stencil state 2026-09-14 20:29:20 +04:00
nmzik fe26c1de17 Share SPIR-V module creation 2026-09-14 20:29:20 +04:00
nmzik 01df42a5f3 Fix DCC clears after metadata writes 2026-09-14 20:29:20 +04:00
nmzik 968cf3c03f Use the correct pixel shader wave size 2026-09-14 20:29:20 +04:00
nmzik 586cbd4a0d Fix separate stencil test and write values 2026-09-14 20:29:20 +04:00
nmzik 940e42f130 Share stencil setup for both faces 2026-09-14 20:29:20 +04:00
nmzik 3f80151a6f Respect vertex buffer sizes 2026-09-14 20:29:20 +04:00
nmzik 7e89b97daf Handle front shader allocation state 2026-09-14 20:29:20 +04:00
nmzik d63fb62367 Fix async compute completion interrupts 2026-09-14 20:29:20 +04:00
nmzik 29e9ea6344 Fix masked stencil replacements 2026-09-14 20:29:20 +04:00
nmzik 9a74ef15c3 Move stencil checks into op conversion 2026-09-14 20:29:20 +04:00
nmzikandBlackCAT a9ae04c152 graphics: decode observed FSR setup registers
Inspired by https://github.com/KytyPS5/KytyPS5/pull/607

Co-authored-by: BlackCAT <52720489+psnwd@users.noreply.github.com>
2026-09-14 10:30:28 +04:00
nmzikandBlackCAT 8e35e1233b hmd2: initialize reprojection and virtual headset field of view
Inspired by https://github.com/KytyPS5/KytyPS5/pull/607

Co-authored-by: BlackCAT <52720489+psnwd@users.noreply.github.com>
2026-09-14 10:30:28 +04:00
nmzikandBlackCAT 5610976720 videoout: map buses to dense port slots before opening
Inspired by https://github.com/KytyPS5/KytyPS5/pull/607

Co-authored-by: BlackCAT <52720489+psnwd@users.noreply.github.com>
2026-09-14 10:30:28 +04:00
nmzikandBlackCAT 1133272668 hmd2: initialize a virtual headset with --vr
Inspired by https://github.com/KytyPS5/KytyPS5/pull/607

Co-authored-by: BlackCAT <52720489+psnwd@users.noreply.github.com>
2026-09-14 10:30:28 +04:00
nmzik bbebb6419b graphics: preserve PS5 array mip contents during cache promotion
PPSA05975
2026-09-14 06:58:32 +04:00
nmzik b67cb81359 avplayer: report loop boundaries when decoded output is delivered 2026-09-14 06:58:32 +04:00
nmzikandBlackCAT f3c82a317a kernel: support timed umtx waits through shared address deadlines
Co-authored-by: BlackCAT <52720489+psnwd@users.noreply.github.com>
2026-09-14 06:58:32 +04:00
nmzik f426d60ff1 net: use the shared host resolver for Linux name lookups 2026-09-14 06:58:32 +04:00
nmzikandAli Almohaya b2fdad9e47 memory: preserve live POSIX mappings during partial unmap
Co-authored-by: Ali Almohaya <AliAlmohaya@gmail.com>
2026-09-14 06:58:32 +04:00
nmzik 87971a28f5 gpu: isolate tessellation analysis and SPIR-V emission 2026-09-14 06:58:32 +04:00
nmzik 96d62b3367 gpu: reflect tessellation strides from memory address dataflow 2026-09-14 06:58:32 +04:00
nmzik 7516068662 gpu: compile RDNA2 tessellation through native LS HS and TES stages 2026-09-14 06:58:32 +04:00
nmzik 6d1ba5870a renderer: share graphics resource preparation across shader stages 2026-09-14 06:58:32 +04:00
nmzik 00a16f3120 gpu: retain fused hull shader user-data addresses 2026-09-14 06:58:32 +04:00
nmzik d3d7bd33f8 launcher: Add game size column 2026-09-12 17:56:44 +04:00
Mehmet Cambaz 069f928edc net: return copied bytes for truncated datagram receives (#510)
Normalize Winsock WSAEMSGSIZE on receive to the copied byte count before converting the peer address. Preserve other receive errors and oversized send failures.
2026-09-12 17:51:43 +04:00
Mininautandnmzik 0f3674f531 kernel: release descriptors when rejecting read-only truncation (#504)
Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-12 17:44:45 +04:00
ADBHXNLOLandnmzik a21ffde34e shader: decode and translate V_FRACT_F16 (VOP1 0x5f) and V_CMPX_LT_U16 (VOPC 0xb9) (#582)
shader: support V_FRACT_F16 and V_CMPX_LT_U16

Decode compact, SDWA, and VOP3 forms using the existing arithmetic and comparison paths. Preserve EXEC masking and VCC for the integer compare, and reject unsupported DPP encoding.

Add captured-instruction, operand-selection, floating-point edge, and execution-mask regression coverage.

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-12 17:21:32 +04:00
nmzik 1208d22547 Fix Windows Clang SPIR-V serialization with typed operands 2026-09-12 13:08:23 +02:00
nmzik 3a27f8a76a build: fix SPIR-V enum word conversions 2026-09-12 15:43:47 +04:00
ADBHXNLOL dff2b19c56 json2: export Value::clear (#584)
Export the existing clear implementation while preserving tree ownership and root handler metadata.
2026-09-12 15:15:26 +04:00
Vandvmexe a58db4f023 docs: fix pre-commit setup instructions (#556)
Co-authored-by: vmexe <vmexe@users.noreply.github.com>
2026-09-12 14:58:34 +04:00
Frans 7bcd43dbee shader: support S_MUL_HI_I32 (SOP2 0x36) (#576) 2026-09-12 14:56:14 +04:00
nmzik 7c5473400e config: simplify logging and profiler options. Change to Mailbox 2026-09-12 14:41:34 +04:00
nmzik 23c8a43e37 Simplify and batch buffer cache downloads 2026-09-12 14:17:00 +04:00
nmzik 2d37634ee9 shader: construct SPIR-V module variables in one place 2026-09-12 13:42:15 +04:00
nmzik e026bfc9a2 shader: let structurization own rollback on failure 2026-09-12 13:42:15 +04:00
nmzik 5a31de7955 shader: keep derived frontend metadata with its owner 2026-09-12 13:42:15 +04:00
nmzik 20e57994cf shader: reuse native emitters and simplify memory dispatch 2026-09-12 13:42:15 +04:00
nmzik 59ef88c959 shader: generate SPIR-V dispatch and marshal handler arguments 2026-09-12 13:42:15 +04:00
nmzik 1d1f00bdef shader: use Khronos definitions and simplify compiler ownership 2026-09-12 13:42:15 +04:00
nmzik 47b26e497f pthread: fix per-thread wakeups 2026-09-12 13:40:10 +04:00
nmzik 733842cad4 kernel: index pthread key values per thread 2026-09-12 13:40:10 +04:00
nmzik 73363f1272 memory: remove per-page host protection bookkeeping 2026-09-12 13:40:10 +04:00
nmzik 0cd7682012 Rename texture transfer types and download APIs 2026-09-12 13:40:10 +04:00
nmzik 25a44371e3 refactor(graphics) 2026-09-12 13:40:10 +04:00
nmzik f2455c875e update tracy to 0.14.1 2026-09-10 19:34:37 +04:00
nmzik 2e315a3c62 ajm: parse gapless control data before codec initialization 2026-09-10 18:38:24 +04:00
nmzik d2fa86579b kernel: share directory streams across seeks and reads 2026-09-10 14:36:15 +04:00
nmzik 23e7df21b1 graphics: simplify texture upload and component mapping paths 2026-09-10 14:36:15 +04:00
nmzik a305a6c1c6 graphics: share packed texture and render-target channel layouts
Resolve PS5 5551 textures with alpha in bit 15 and compose host channel order with guest selectors. Share the existing render-target mapping with sampled textures so packed layouts cannot diverge.

Add native GPU readback coverage for 5551 and 4444, arbitrary selectors, constants, and existing 1555 render-target mapping. Verified PPSA24515 through launch.json with Vulkan validation, advancing scenes and over 12,700 guest flips. Independent correctness and cleanliness audits passed.
2026-09-10 14:36:15 +04:00
nmzik 635fb8882b systemservice: use configured language for PS5 parameter 400 2026-09-10 14:11:46 +04:00
nmzik fe6f49632c font: accept supported OpenType CFF faces 2026-09-10 14:11:46 +04:00
nmzikandBrandon Strong c3546577b6 shader: implement S_WQM_B32
Co-authored-by: Brandon Strong <brandostrong@gmail.com>
2026-09-10 14:11:46 +04:00
nmzik c913951f7b shader: implement RDNA2 subvector loop mask and branch semantics 2026-09-10 14:11:46 +04:00
nmzik 3902b42096 shader: share direct and conditional branch classification 2026-09-10 14:11:46 +04:00
nmzik ae573f941a graphics: share vertex format mapping and support packed float colors 2026-09-10 14:11:46 +04:00
nmzik 20ea54cf0c Json2: new ABI 2026-09-10 14:11:46 +04:00
nmzik 0b4e78c0b2 graphics: execute DB_RENDER_OVERRIDE depth and stencil copies 2026-09-09 15:26:52 +04:00
nmzik 57844a07e2 buffer cache: grow stream reserves toward incoming requests 2026-09-09 15:26:52 +04:00
nmzik ba18551d6e ces: implement CP932 string conversion 2026-09-09 15:26:52 +04:00
nmzikandAli Almohaya c7b0d77970 loader: port guest fault diagnostics and remove host stack traces
Co-authored-by: Ali Almohaya <AliAlmohaya@gmail.com>
2026-09-09 13:55:53 +04:00
nmzik 1af19eabf0 shader: fix incorrect per-lane VCCZ/EXECZ reads 2026-09-09 13:22:09 +04:00
nmzik 32ea0869e5 shader: implement S_ORN2_SAVEEXEC_B32 2026-09-09 13:11:48 +04:00
nmzik e04007a7c6 shader: implement DS_INC_RTN_U32 and DS_DEC_RTN_U32 2026-09-09 12:56:20 +04:00
frangametvandnmzik 1a3a15e4cf Fix mount root opens and empty APR prefixes (#439)
* Fix opening mount root directories

* Allow empty APR path prefixes

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-09-09 12:31:23 +04:00
nmzik 679adbfa5e graphics: fix linear render-target pitch alignment 2026-09-09 02:45:59 +02:00
nmzikandEttore Sartori d285efe3b5 graphics: allow absent programmable restart indices
Co-authored-by: Ettore Sartori <85788613+Ekt0re@users.noreply.github.com>
2026-09-08 21:26:59 +04:00
nmzik a64c8d3205 kernel: implement umtx wait/wake and reject malformed timeouts 2026-09-08 21:26:59 +04:00
nmzik 82c52c3fab window: round FPS in the title 2026-09-08 21:26:59 +04:00
nmzik dd1e90fcd7 video: keep decoding after Flush 2026-09-08 21:26:59 +04:00
nmzik ef0155636a shader: support single and split NGG shaders for PPSA11142 2026-09-08 21:26:59 +04:00
nmzik 9d550fbffa graphics: fix last provoking vertex on PS5 2026-09-08 21:26:59 +04:00
nmzik fcef691e0a shader: drop EmitTrueBool 2026-09-08 21:26:59 +04:00
nmzik 839c172612 shader: reuse operands and remove duplicate helpers 2026-09-08 21:26:59 +04:00
nmzik 9f49ac8249 shader: reuse SPIR-V helpers and memory resources 2026-09-08 21:26:59 +04:00
nmzik fde550ef66 graphics: simplify texture cleanup and swizzle decoding 2026-09-08 21:26:59 +04:00
nmzik 2b3993a595 shader: share emitter helpers 2026-09-08 21:26:59 +04:00
nmzik de3e86791c shader: reuse CFG and validation lookups 2026-09-08 21:26:59 +04:00
nmzik 6f66f77b09 shader: drop unused decoder fields 2026-09-08 21:26:59 +04:00
nmzik d853aba955 graphics: skip temporary cache lookup vectors 2026-09-08 21:26:59 +04:00
nmzik cd62693792 shader: share control flow labels 2026-09-08 21:26:59 +04:00
nmzik cb4c8f3878 shader: decode buffer formats once 2026-09-08 21:26:59 +04:00
nmzik 4d371d5f5f shader: reuse initial SSA types 2026-09-08 21:26:59 +04:00
nmzik 72963b01e1 shader: remove redundant decoder and translator state 2026-09-08 21:26:59 +04:00
nmzik 80a18e82b7 graphics: simplify image allocation 2026-09-08 21:26:58 +04:00
nmzik 574e687734 graphics: drop unused completion overload 2026-09-08 21:26:58 +04:00
nmzik c725ec50ff graphics: clean up draw setup 2026-09-08 21:26:58 +04:00
Brandon Strong c6f6e306e0 shader: raise the dense image resource limit to 64 (#457) 2026-09-08 19:37:41 +04:00
nmzik b847135321 launcher: guard optional Qt Vulkan support on macOS 2026-09-08 04:29:01 +04:00
nmzik ca3dc8792a libs: implement NpCommerce lifecycle 2026-09-08 04:25:35 +04:00
nmzik ac3156df3a graphics: fix Vulkan BC image compatibility checks` 2026-09-07 23:41:19 +02:00
257 changed files with 20305 additions and 13691 deletions

No files matched your search

+37
View File
@@ -0,0 +1,37 @@
# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json
# Keep inherited review profiles, automatic reviews, tools, and pre-merge checks.
inheritance: true
reviews:
# Trim supplemental output and keep the remaining walkthrough collapsed.
high_level_summary: false
review_status: false
review_details: false
collapse_walkthrough: true
changed_files_summary: false
sequence_diagrams: false
estimate_code_review_effort: false
related_issues: false
related_prs: false
in_progress_fortune: false
poem: false
enable_prompt_for_ai_agents: false
# Hide standard finishing-touch controls; this also disables their commands.
finishing_touches:
docstrings:
enabled: false
unit_tests:
enabled: false
simplify:
enabled: false
autofix:
enabled: false
fix_ci:
enabled: false
resolve_merge_conflict:
enabled: false
chat:
art: false
+21
View File
@@ -6,7 +6,12 @@ on:
branches: [ main, master ]
pull_request:
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
env:
SCCACHE_GHA_ENABLED: "true"
KYTY_BUILD_ORIGIN: ${{ github.repository != 'KytyPS5/KytyPS5' && 'Fork' || github.event_name == 'push' && 'Official' || 'Source' }}
KYTY_BUILD_REPOSITORY: ${{ github.repository }}
@@ -78,6 +83,9 @@ jobs:
arch: win64_msvc2022_64
cache: true
- name: Setup sccache
uses: mozilla-actions/sccache-action@v0.0.11
- name: Verify toolchain
shell: cmd
run: |
@@ -100,6 +108,8 @@ jobs:
-DKYTY_RELEASE_TAG=${{ needs.metadata.outputs.release_tag }} ^
-DCMAKE_C_COMPILER=clang-cl ^
-DCMAKE_CXX_COMPILER=clang-cl ^
-DCMAKE_C_COMPILER_LAUNCHER=sccache ^
-DCMAKE_CXX_COMPILER_LAUNCHER=sccache ^
-DCMAKE_PREFIX_PATH="%Qt6_DIR%"
- name: Build
@@ -156,6 +166,9 @@ jobs:
arch: clang_64
cache: true
- name: Setup sccache
uses: mozilla-actions/sccache-action@v0.0.11
- name: Verify toolchain
shell: bash
run: |
@@ -180,6 +193,8 @@ jobs:
-DCMAKE_OSX_ARCHITECTURES=x86_64 \
-DCMAKE_C_COMPILER=clang \
-DCMAKE_CXX_COMPILER=clang++ \
-DCMAKE_C_COMPILER_LAUNCHER=sccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=sccache \
-DCMAKE_PREFIX_PATH="$QT_ROOT_DIR"
- name: Build
@@ -304,6 +319,7 @@ jobs:
libxss-dev \
lld \
ninja-build \
pkg-config \
wayland-protocols
- name: Install Qt 6.10.3
@@ -315,6 +331,9 @@ jobs:
arch: linux_gcc_64
cache: true
- name: Setup sccache
uses: mozilla-actions/sccache-action@v0.0.11
- name: Verify toolchain
shell: bash
run: |
@@ -339,6 +358,8 @@ jobs:
-DKYTY_RELEASE_TAG="${{ needs.metadata.outputs.release_tag }}" \
-DCMAKE_C_COMPILER=clang \
-DCMAKE_CXX_COMPILER=clang++ \
-DCMAKE_C_COMPILER_LAUNCHER=sccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=sccache \
-DCMAKE_PREFIX_PATH="$Qt6_DIR" 2>&1 | tee _Build/configure.log
exit "${PIPESTATUS[0]}"
+1 -1
View File
@@ -22,7 +22,7 @@
url = https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator.git
[submodule "3rdparty/ffmpeg-core"]
path = 3rdparty/ffmpeg-core
url = https://github.com/shadps4-emu/ext-ffmpeg-core.git
url = https://github.com/KytyPS5/ext-ffmpeg-core.git
shallow = true
[submodule "3rdparty/fmt"]
path = 3rdparty/fmt
+6 -4
View File
@@ -2,15 +2,17 @@
"cmake.sourceDirectory": "${workspaceFolder}",
"cmake.buildDirectory": "${workspaceFolder}/_Build/vscode-clang",
"cmake.generator": "Ninja",
"cmake.useVsDeveloperEnvironment": "always",
"cmake.environment": {
"CC": "clang-cl",
"CXX": "clang-cl",
"PATH": "${env:ProgramFiles}/Microsoft Visual Studio/18/Community/VC/Tools/Llvm/x64/bin;${env:ProgramFiles}/Microsoft Visual Studio/18/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/Ninja;${env:PATH}"
"CXX": "clang-cl"
},
"cmake.configureSettings": {
"CMAKE_C_COMPILER": "clang-cl",
"CMAKE_CXX_COMPILER": "clang-cl",
"CMAKE_PREFIX_PATH": "C:/Qt/6.11.0/msvc2022_64"
}
},
"C_Cpp.default.intelliSenseMode": "windows-clang-x64",
"C_Cpp.default.compileCommands": "${workspaceFolder}/_Build/vscode-clang/compile_commands.json",
"C_Cpp.default.cppStandard": "c++20"
}
+3
View File
@@ -86,6 +86,9 @@ set(SPDLOG_BUILD_WARNINGS OFF CACHE BOOL "" FORCE)
set(SPDLOG_INSTALL OFF CACHE BOOL "" FORCE)
set(SPDLOG_FMT_EXTERNAL ON CACHE BOOL "" FORCE)
set(SPDLOG_NO_EXCEPTIONS ON CACHE BOOL "" FORCE)
if(WIN32)
set(SPDLOG_WCHAR_FILENAMES ON CACHE BOOL "" FORCE)
endif()
add_subdirectory(spdlog EXCLUDE_FROM_ALL)
if (NOT TARGET FFmpeg::ffmpeg)
+1 -1
+30 -15
View File
@@ -317,24 +317,10 @@ endif()
set(kyty_emulator_link_libraries common Vulkan::Headers spirv-tools-opt spirv-tools SDL2-static imgui xxhash FFmpeg::ffmpeg fmt::fmt nlohmann_json::nlohmann_json LibAtrac9 kyty_git_version Zydis::Zydis xbyak::xbyak)
# Linux system libraries required by the static FFmpeg archive.
# Linux system libraries.
if(LINUX)
find_package(Threads REQUIRED)
list(APPEND kyty_emulator_link_libraries m ${CMAKE_DL_LIBS} Threads::Threads)
# Optional FFmpeg dependencies.
find_package(ZLIB)
if(ZLIB_FOUND)
list(APPEND kyty_emulator_link_libraries ZLIB::ZLIB)
endif()
find_library(KYTY_BZ2_LIBRARY bz2)
if(KYTY_BZ2_LIBRARY)
list(APPEND kyty_emulator_link_libraries ${KYTY_BZ2_LIBRARY})
endif()
find_library(KYTY_LZMA_LIBRARY lzma)
if(KYTY_LZMA_LIBRARY)
list(APPEND kyty_emulator_link_libraries ${KYTY_LZMA_LIBRARY})
endif()
endif()
set(inc_headers
@@ -342,6 +328,7 @@ set(inc_headers
${KYTY_THIRD_PARTY_DIR}/SDL2/include
${KYTY_THIRD_PARTY_DIR}/VulkanMemoryAllocator/include
${KYTY_THIRD_PARTY_DIR}/SPIRV-Tools/include
${KYTY_THIRD_PARTY_DIR}/SPIRV-Headers/include
${KYTY_THIRD_PARTY_DIR}/fmt/include
${KYTY_THIRD_PARTY_DIR}/xxhash
${KYTY_THIRD_PARTY_DIR}/cpuinfo/include
@@ -463,6 +450,12 @@ add_executable(audio_out2_port_tests EXCLUDE_FROM_ALL
target_link_libraries(audio_out2_port_tests common fmt::fmt)
target_include_directories(audio_out2_port_tests PRIVATE ${inc_headers})
add_executable(save_data_memory_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/SaveDataMemoryTests.cpp"
)
target_link_libraries(save_data_memory_tests common fmt::fmt)
target_include_directories(save_data_memory_tests PRIVATE ${inc_headers})
add_executable(event_queue_lifetime_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/EventQueueLifetimeTests.cpp"
"${KYTY_SOURCE_DIR}/kernel/eventQueue.cpp"
@@ -547,9 +540,14 @@ if(BUILD_TESTING)
add_test(NAME kernel_file_system COMMAND $<TARGET_FILE:kernel_file_system_tests>)
add_test(NAME sync_on_address COMMAND $<TARGET_FILE:sync_on_address_tests>)
add_test(NAME audio_out2_port COMMAND $<TARGET_FILE:audio_out2_port_tests>)
add_test(NAME save_data_memory COMMAND $<TARGET_FILE:save_data_memory_tests>)
add_test(NAME shader_recompiler_compute COMMAND $<TARGET_FILE:shader_recompiler_compute_tests>)
add_test(NAME shader_recompiler_alignbyte
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --alignbyte-only)
add_test(NAME graphics_pipeline_rasterization
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --polygon-mode-only)
add_test(NAME graphics_draw_offsets
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --draw-offset-only)
add_test(NAME virtual_memory_allocation
COMMAND $<TARGET_FILE:virtual_memory_allocation_tests>)
add_test(NAME guest_red_zone_patcher
@@ -567,6 +565,8 @@ if(BUILD_TESTING)
add_test(NAME texture_cache_layered_image
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --layered-image-only)
add_test(NAME host_image_allocation
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --host-image-allocation-only)
add_test(NAME texture_cache_image_views
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --image-view-cache-only)
add_test(NAME texture_cache_storage_sampled
@@ -608,6 +608,21 @@ if(BUILD_TESTING)
)
endif()
# Test Darwin signal-context restoration for SSE4a instructions.
if(APPLE AND (CMAKE_OSX_ARCHITECTURES STREQUAL "x86_64" OR
(NOT CMAKE_OSX_ARCHITECTURES AND CMAKE_SYSTEM_PROCESSOR MATCHES "^(x86_64|AMD64)$")))
add_executable(macos_sse4a_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/macosSse4aTests.cpp"
"${KYTY_SOURCE_DIR}/loader/x64InstructionEmulator.cpp"
)
target_link_libraries(macos_sse4a_tests common fmt::fmt Zydis::Zydis)
target_include_directories(macos_sse4a_tests PRIVATE ${inc_headers})
if(BUILD_TESTING)
add_test(NAME macos_sse4a COMMAND $<TARGET_FILE:macos_sse4a_tests>)
set_tests_properties(macos_sse4a PROPERTIES TIMEOUT 15)
add_dependencies(kyty_tests macos_sse4a_tests)
endif()
endif()
add_executable(kyty_emulator "${KYTY_SOURCE_DIR}/main.cpp" ${kyty_emulator_src})
configure_macos_guest_address_space(kyty_emulator)
+35 -4
View File
@@ -90,8 +90,14 @@ starting a large change.
Set up the clang-format hook after cloning:
```powershell
python -m pip install pre-commit
Install `pre-commit` using the method appropriate for your platform:
- **Arch Linux / CachyOS:** `sudo pacman -S pre-commit`
- **Other Linux / macOS / Windows:** `python -m pip install pre-commit`
Then install the Git hook:
```bash
python -m pre_commit install --install-hooks
```
@@ -131,6 +137,7 @@ the Vulkan/SPIR-V validation rules.
- Visual Studio 2022 or Build Tools 2022 with the **Desktop development with C++** workload and
**C++ Clang tools for Windows** component
- Qt 6 for MSVC 2022 64-bit, including Concurrent, Network, and Widgets
- [glslang](https://github.com/KhronosGroup/glslang/releases) (`glslangValidator`) on `PATH`
The Microsoft C++ compiler (`cl.exe`) is not supported; use `clang-cl`.
@@ -165,7 +172,7 @@ build has no working sound and no gamepad hotplug:
```bash
sudo apt-get install --no-install-recommends \
clang lld ninja-build cmake git glslang-tools \
clang lld ninja-build cmake git glslang-tools pkg-config \
libgl1-mesa-dev libx11-dev libxcursor-dev libxext-dev libxfixes-dev \
libxi-dev libxrandr-dev libxss-dev libxkbcommon-dev \
libasound2-dev libpulse-dev libudev-dev libdbus-1-dev libwayland-dev wayland-protocols
@@ -186,13 +193,37 @@ cmake --install _Build/linux --prefix _Build/linux/install
```
The install step copies the Qt libraries and plugins next to the binaries, so
`_Build/linux/install` runs without a matching system Qt.
`_Build/linux/install` runs without a matching system Qt. FFmpeg is linked statically
from the pinned [KytyPS5 FFmpeg core](https://github.com/KytyPS5/ext-ffmpeg-core)
release, including VP9 and WebM support. System FFmpeg packages are not required.
As on Windows, the MSVC compiler is not used; Clang is required. `cl.exe` is rejected at configure
time.
The CMake source root is the repository root.
### Building on NixOS
A development shell provides Clang, CMake, Ninja, Qt 6, the Vulkan headers, and the SDL2 backend
libraries. Enter it and configure exactly as on other Linux distributions; the shell exports
`CMAKE_PREFIX_PATH` and `QT_PLUGIN_PATH`, so the `-DCMAKE_PREFIX_PATH="$Qt6_DIR"` argument is not
needed:
```bash
nix-shell # or: nix develop
git submodule update --init --recursive
cmake -S . -B _Build/linux -G Ninja -DCMAKE_BUILD_TYPE=Release \
-DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
cmake --build _Build/linux --target launcher --parallel
cmake --install _Build/linux --prefix _Build/linux/install
```
The configure step downloads the FFmpeg prebuilts and the `xbyak`/`zydis` sources, so it needs
network access; a fully sandboxed `nix build` would require vendoring those inputs. A Vulkan 1.3
driver must be available at runtime (on NixOS, `hardware.graphics.enable = true`).
### Building on macOS
macOS builds target x86-64 and run under Rosetta 2 on Apple Silicon, so the PS5's x86-64 game
Generated
+27
View File
@@ -0,0 +1,27 @@
{
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1789426278,
"narHash": "sha256-GnotcvKtbTMnoVx6G4E7ZdUX6487Dky74dqq3pU7iRk=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "efe6f071ede9d21c37462d2d6682d5e670099684",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}
+20
View File
@@ -0,0 +1,20 @@
{
description = "KytyPS5 development shell";
inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
outputs = { self, nixpkgs }:
let
systems = [ "x86_64-linux" ];
forAllSystems = nixpkgs.lib.genAttrs systems;
in
{
devShells = forAllSystems (system:
let
pkgs = import nixpkgs { inherit system; };
in
{
default = import ./nix/devshell.nix { inherit pkgs; };
});
};
}
+84
View File
@@ -0,0 +1,84 @@
{ pkgs }:
let
# Build-time dependencies. Every -dev package the bundled SDL2 probes for must
# be listed here, otherwise SDL2 quietly drops the audio/Wayland/udev backends.
buildDeps = with pkgs; [
qt6.qtbase # Concurrent, Network, Widgets (launcher)
vulkan-headers
vulkan-loader
vulkan-tools
vulkan-validation-layers
libglvnd
# X11 / input
libX11
libXcursor
libXext
libXfixes
libXi
libXrandr
libXScrnSaver
libxkbcommon
# Wayland
wayland
wayland-protocols
libdecor
# audio + device/dbus
alsa-lib
libpulseaudio
systemd # libudev
dbus
];
# Libraries the built binaries need at runtime.
runtimeLibs = with pkgs; [
stdenv.cc.cc.lib # libstdc++ / libgcc_s
libgcc.lib
qt6.qtbase # libQt6Concurrent/Network/Widgets
vulkan-loader
libglvnd
alsa-lib
libpulseaudio
libX11
libXcursor
libXext
libXfixes
libXi
libXrandr
libXScrnSaver
libxkbcommon
wayland
libdecor
systemd
dbus
];
in
pkgs.mkShell {
name = "kytyps5";
nativeBuildInputs = with pkgs; [
clang
lld
cmake
ninja
pkg-config
git
glslang # provides glslangValidator for the bundled shaders
];
buildInputs = buildDeps;
shellHook = ''
export CMAKE_PREFIX_PATH="${pkgs.qt6.qtbase}:''${CMAKE_PREFIX_PATH:-}"
export QT_PLUGIN_PATH="${pkgs.qt6.qtbase}/lib/qt-6/plugins:${pkgs.qt6.qtbase}/lib/qt6/plugins:''${QT_PLUGIN_PATH:-}"
export LD_LIBRARY_PATH="${pkgs.lib.makeLibraryPath runtimeLibs}:''${LD_LIBRARY_PATH:-}"
if [ -z "''${KYTY_SHELL_QUIET:-}" ]; then
echo "KytyPS5 dev shell"
echo " cmake -S . -B _Build/linux -G Ninja -DCMAKE_BUILD_TYPE=Release \\"
echo " -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++"
echo " cmake --build _Build/linux --target launcher --parallel"
echo " cmake --install _Build/linux --prefix _Build/linux/install"
fi
'';
}
+4
View File
@@ -0,0 +1,4 @@
let
pkgs = import <nixpkgs> { };
in
import ./nix/devshell.nix { inherit pkgs; }
+25
View File
@@ -0,0 +1,25 @@
// SPDX-FileCopyrightText: 2014 Jannik Vogel <email@jannikvogel.de>
// SPDX-License-Identifier: CC0-1.0
#pragma once
#include <cstddef>
#include <type_traits>
namespace Common {
template <typename T>
[[nodiscard]] constexpr T AlignUp(T value, std::size_t size) {
static_assert(std::is_unsigned_v<T>, "T must be an unsigned value.");
auto mod{static_cast<T>(value % size)};
value -= mod;
return static_cast<T>(mod == T{0} ? value : value + size);
}
template <typename T>
[[nodiscard]] constexpr T AlignDown(T value, std::size_t size) {
static_assert(std::is_unsigned_v<T>, "T must be an unsigned value.");
return static_cast<T>(value - value % size);
}
} // namespace Common
+2 -18
View File
@@ -1,6 +1,5 @@
#include "common/assert.h"
#include "common/debug.h"
#include "common/logging/log.h"
#include "common/subsystems.h"
#include "kytyGitVersion.h"
@@ -12,25 +11,10 @@
namespace Common {
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS && KYTY_BUILD == KYTY_BUILD_DEBUG && \
KYTY_COMPILER == KYTY_COMPILER_CLANG
constexpr int PRINT_STACK_FROM = 4;
#else
constexpr int PRINT_STACK_FROM = 2;
#endif
static std::string BuildFatalReport(const char* title, std::string_view text, const char* file,
int line) {
DebugStack stack;
DebugStack::Trace(&stack);
std::string report = "--- Build ---\n" KYTY_BUILD_LABEL "\n--- Stack Trace ---\n";
for (int i = PRINT_STACK_FROM; i < stack.depth; i++) {
report += fmt::format("[{}] {:016x}\n", i - PRINT_STACK_FROM,
static_cast<uint64_t>(stack.GetAddr(i)));
}
report += fmt::format("{}\n{} in {}:{}\n", title, text, file, line);
return report;
return fmt::format("--- Build ---\n{}\n{}\n{} in {}:{}\n", KYTY_BUILD_LABEL, title, text, file,
line);
}
static int DbgReport(const char* title, std::string_view text, const char* file, int line) {
-166
View File
@@ -1,166 +0,0 @@
#ifndef KYTY_COMMON_BYTEBUFFER_H_
#define KYTY_COMMON_BYTEBUFFER_H_
#include "common/assert.h"
#include "common/common.h"
#include <algorithm>
#include <cstddef> // IWYU pragma: export
#include <cstring>
#include <utility>
#include <vector>
namespace Common {
using Byte = std::byte;
class ByteBuffer {
public:
using DataType = std::vector<Byte>;
static constexpr uint32_t INVALID_INDEX = static_cast<uint32_t>(-1);
using iterator = DataType::iterator;
using const_iterator = DataType::const_iterator;
ByteBuffer() = default;
ByteBuffer(const ByteBuffer&) = default;
ByteBuffer(ByteBuffer&&) = default;
ByteBuffer& operator=(const ByteBuffer&) = default;
ByteBuffer& operator=(ByteBuffer&&) = default;
explicit ByteBuffer(uint32_t size): m_data(size) {}
ByteBuffer(std::initializer_list<uint8_t> list) {
for (const uint8_t& e: list) {
m_data.emplace_back(static_cast<Byte>(e));
}
}
explicit ByteBuffer(const void* buf, uint32_t size)
: ByteBuffer(size) // @suppress("Ambiguous problem")
{
EXIT_IF(buf == nullptr && size != 0);
if (size != 0) {
std::memcpy(m_data.data(), buf, size);
}
}
[[nodiscard]] uint32_t Size() const { return static_cast<uint32_t>(m_data.size()); }
[[nodiscard]] bool IsEmpty() const { return m_data.empty(); }
void Clear() { m_data.clear(); }
void Add(const Byte& val) { m_data.push_back(val); }
void Add(Byte&& val) { m_data.push_back(std::move(val)); }
void Add(const Byte* val, uint32_t num) {
EXIT_IF(val == nullptr && num != 0);
m_data.insert(m_data.end(), val, val + num);
}
void Add(const ByteBuffer& v) {
if (this == &v) {
const ByteBuffer copy(v);
Add(copy.GetDataConst(), copy.Size());
return;
}
Add(v.GetDataConst(), v.Size());
}
void InsertAt(uint32_t index, const Byte& val) {
EXIT_IF(index > Size());
m_data.insert(m_data.begin() + index, val);
}
void InsertAt(uint32_t index, Byte&& val) {
EXIT_IF(index > Size());
m_data.insert(m_data.begin() + index, std::move(val));
}
[[nodiscard]] uint32_t Find(const Byte& t) const {
const auto it = std::find(m_data.begin(), m_data.end(), t);
return (it == m_data.end() ? INVALID_INDEX
: static_cast<uint32_t>(std::distance(m_data.begin(), it)));
}
[[nodiscard]] bool Contains(const Byte& t) const { return Find(t) != INVALID_INDEX; }
bool Remove(const Byte& t) {
const auto it = std::find(m_data.begin(), m_data.end(), t);
if (it == m_data.end()) {
return false;
}
m_data.erase(it);
return true;
}
bool RemoveAt(uint32_t index, uint32_t count = 1) {
if (index >= Size() || count == 0) {
return false;
}
const uint32_t erase_count = std::min(count, Size() - index);
m_data.erase(m_data.begin() + index, m_data.begin() + index + erase_count);
return true;
}
[[nodiscard]] bool IndexValid(uint32_t index) const { return index < Size(); }
Byte& operator[](uint32_t index) {
EXIT_IF(index >= Size());
return m_data[index];
}
const Byte& operator[](uint32_t index) const {
EXIT_IF(index >= Size());
return m_data[index];
}
[[nodiscard]] const Byte& At(uint32_t index) const {
EXIT_IF(index >= Size());
return m_data[index];
}
Byte* GetData() { return m_data.data(); }
[[nodiscard]] const Byte* GetData() const { return m_data.data(); }
[[nodiscard]] const Byte* GetDataConst() const { return m_data.data(); }
void Sort() {
std::sort(m_data.begin(), m_data.end(), [](Byte a, Byte b) {
return std::to_integer<uint8_t>(a) < std::to_integer<uint8_t>(b);
});
}
template <typename OP>
void Sort(OP&& comp_func) {
std::sort(m_data.begin(), m_data.end(), std::forward<OP>(comp_func));
}
bool operator==(const ByteBuffer& s) const { return m_data == s.m_data; }
bool operator!=(const ByteBuffer& s) const { return !(*this == s); }
iterator begin() { return m_data.begin(); } // NOLINT(readability-identifier-naming)
iterator end() { return m_data.end(); } // NOLINT(readability-identifier-naming)
[[nodiscard]] const_iterator begin() const {
return m_data.begin();
} // NOLINT(readability-identifier-naming)
[[nodiscard]] const_iterator end() const {
return m_data.end();
} // NOLINT(readability-identifier-naming)
[[nodiscard]] const_iterator cbegin() const {
return m_data.cbegin();
} // NOLINT(readability-identifier-naming)
[[nodiscard]] const_iterator cend() const {
return m_data.cend();
} // NOLINT(readability-identifier-naming)
private:
DataType m_data;
};
} // namespace Common
#endif /* KYTY_COMMON_BYTEBUFFER_H_ */
-17
View File
@@ -1,20 +1,7 @@
#include "common/debug.h"
#include "common/platform/sysDbg.h"
namespace Common {
void DebugStack::Print(int from, [[maybe_unused]] bool with_name) const {
for (int i = from; i < depth; i++) {
printf("[%d] %016" PRIx64 "\n", i - from, static_cast<uint64_t>(GetAddr(i)));
}
}
void DebugStack::Trace(DebugStack* stack) {
stack->depth = DEBUG_MAX_STACK_DEPTH;
SysStackWalk(stack->stack.data(), &stack->depth);
}
std::string Debug::GetCompiler() {
#if KYTY_COMPILER == KYTY_COMPILER_CLANG
return "clang";
@@ -37,8 +24,4 @@ std::string Debug::GetLinker() {
#endif
}
std::string Debug::GetBitness() {
return "64";
}
} // namespace Common
+1 -27
View File
@@ -2,42 +2,16 @@
#define KYTY_COMMON_DEBUG_H_
#include "common/common.h"
#include "common/stringUtils.h"
#include <array>
#include <cstring>
#include <string>
namespace Common {
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS && KYTY_BUILD == KYTY_BUILD_DEBUG && \
KYTY_COMPILER == KYTY_COMPILER_CLANG
constexpr int DEBUG_MAX_STACK_DEPTH = 20;
#else
constexpr int DEBUG_MAX_STACK_DEPTH = 15;
#endif
namespace Debug {
std::string GetCompiler();
std::string GetLinker();
std::string GetBitness();
} // namespace Debug
struct DebugStack {
int depth {0};
std::array<void*, DEBUG_MAX_STACK_DEPTH> stack {};
[[nodiscard]] uintptr_t GetAddr(int i) const { return reinterpret_cast<uintptr_t>(stack[i]); }
void Print(int from, bool with_name = true) const;
void CopyTo(DebugStack* s) const {
std::memcpy(s->stack.data(), stack.data(), sizeof(void*) * depth);
s->depth = depth;
}
static void Trace(DebugStack* stack);
};
} // namespace Common
#endif /* KYTY_COMMON_DEBUG_H_ */
+20 -4
View File
@@ -43,6 +43,10 @@ int32_t GetUserId() {
return g_config->user_id;
}
const std::string& GetAudioInputDevice() {
return g_config->audio_input_device;
}
PresentMode GetPresentMode() {
return g_config->present_mode;
}
@@ -55,6 +59,14 @@ bool FullscreenEnabled() {
return g_config->fullscreen_enabled;
}
bool VrEnabled() {
return g_config->vr_enabled;
}
bool AmdCpuEnabled() {
return g_config->amd_cpu_enabled;
}
uint32_t GetVblankFrequency() {
return std::clamp(g_config->vblank_frequency, 30u, 360u);
}
@@ -75,7 +87,7 @@ ShaderOptimizationType GetShaderOptimizationType() {
return g_config->shader_optimization_type;
}
ShaderLogDirection GetShaderLogDirection() {
LogDirection GetShaderLogDirection() {
return g_config->shader_log_direction;
}
@@ -95,7 +107,7 @@ bool GraphicsDebugDumpEnabled() {
return g_config->graphics_debug_dump_enabled;
}
OutputDirection GetPrintfDirection() {
LogDirection GetPrintfDirection() {
return g_config->printf_direction;
}
@@ -103,8 +115,8 @@ std::filesystem::path GetPrintfOutputFile() {
return g_config->printf_output_file;
}
ProfilerDirection GetProfilerDirection() {
return g_config->profiler_direction;
bool ProfilerEnabled() {
return g_config->profiler_enabled;
}
bool SpirvDebugPrintfEnabled() {
@@ -123,6 +135,10 @@ bool ReadbackLinearImagesEnabled() {
return g_config->readback_linear_images;
}
bool TessellationEnabled() {
return g_config->tessellation_enabled;
}
bool PlayGoHackEnabled() {
return g_config->playgo_hack_enabled;
}
+16 -12
View File
@@ -21,11 +21,7 @@ struct Lifecycle {
enum class ShaderOptimizationType { None, Size, Performance };
enum class ShaderLogDirection { Silent, Console, File };
enum class ProfilerDirection { None, Network };
enum class OutputDirection { Silent, Console, File };
enum class LogDirection { Silent, Console, File };
enum class PresentMode { Fifo, Mailbox, Immediate };
@@ -47,26 +43,30 @@ struct ConfigOptions {
uint32_t screen_height = 720;
std::string user_name = "Kyty";
int32_t user_id = DEFAULT_USER_ID;
PresentMode present_mode = PresentMode::Fifo;
std::string audio_input_device;
PresentMode present_mode = PresentMode::Mailbox;
int32_t gpu_index = -1;
bool fullscreen_enabled = false;
bool vr_enabled = false;
bool amd_cpu_enabled = false;
uint32_t vblank_frequency = 60;
uint32_t console_language = DEFAULT_CONSOLE_LANGUAGE;
bool vulkan_validation_enabled = false;
bool shader_validation_enabled = false;
ShaderOptimizationType shader_optimization_type = ShaderOptimizationType::None;
ShaderLogDirection shader_log_direction = ShaderLogDirection::Silent;
LogDirection shader_log_direction = LogDirection::Silent;
std::filesystem::path shader_log_folder = "_Shaders";
bool command_buffer_dump_enabled = false;
std::filesystem::path command_buffer_dump_folder = "_Buffers";
bool graphics_debug_dump_enabled = false;
OutputDirection printf_direction = OutputDirection::Silent;
LogDirection printf_direction = LogDirection::Silent;
std::filesystem::path printf_output_file = "_kyty.txt";
ProfilerDirection profiler_direction = ProfilerDirection::None;
bool profiler_enabled = false;
bool spirv_debug_printf_enabled = false;
bool gpu_assisted_validation_enabled = false;
bool renderdoc_enabled = false;
bool readback_linear_images = false;
bool tessellation_enabled = false;
bool playgo_hack_enabled = false;
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS
bool red_zone_protection_enabled = false;
@@ -80,16 +80,19 @@ uint32_t GetScreenWidth();
uint32_t GetScreenHeight();
const std::string& GetUserName();
int32_t GetUserId();
const std::string& GetAudioInputDevice();
PresentMode GetPresentMode();
int32_t GetGpuIndex();
bool FullscreenEnabled();
bool VrEnabled();
bool AmdCpuEnabled();
uint32_t GetVblankFrequency();
uint32_t GetConsoleLanguage();
bool VulkanValidationEnabled();
bool ShaderValidationEnabled();
ShaderOptimizationType GetShaderOptimizationType();
ShaderLogDirection GetShaderLogDirection();
LogDirection GetShaderLogDirection();
std::filesystem::path GetShaderLogFolder();
bool CommandBufferDumpEnabled();
@@ -97,10 +100,10 @@ std::filesystem::path GetCommandBufferDumpFolder();
bool GraphicsDebugDumpEnabled();
OutputDirection GetPrintfDirection();
LogDirection GetPrintfDirection();
std::filesystem::path GetPrintfOutputFile();
ProfilerDirection GetProfilerDirection();
bool ProfilerEnabled();
bool SpirvDebugPrintfEnabled();
@@ -108,6 +111,7 @@ bool GpuAssistedValidationEnabled();
bool RenderDocEnabled();
bool ReadbackLinearImagesEnabled();
bool TessellationEnabled();
bool PlayGoHackEnabled();
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS
bool RedZoneProtectionEnabled();
+53 -84
View File
@@ -136,10 +136,6 @@ bool File::OpenInMem(void* buf, uint32_t buf_size) {
return true;
}
bool File::OpenInMem(ByteBuffer& buf) {
return OpenInMem(buf.GetData(), buf.Size());
}
bool File::CreateInMem() {
EXIT_IF(m_p->f != nullptr);
@@ -249,88 +245,78 @@ void File::Printf(const char* format, ...) {
Write(s.data(), static_cast<uint32_t>(s.size()));
}
static std::filesystem::path WithoutTrailingSeparator(
const std::filesystem::path& path) {
if (!path.empty() && !path.has_filename() &&
path != path.root_path()) {
return path.parent_path();
}
return path;
}
bool File::IsDirectoryExisting(const std::filesystem::path& path) {
auto path_str = PathToGenericString(path);
return SysFileIsDirectoryExisting(
Common::EndsWith(path_str, "/") ? Common::RemoveLast(path_str, 1) : path_str);
return SysFileIsDirectoryExisting(WithoutTrailingSeparator(path));
}
bool File::IsFileExisting(const std::filesystem::path& name) {
return SysFileIsFileExisting(name);
}
bool File::CreateDirectory(
const std::filesystem::path& path) // @suppress("Member declaration not found")
{
auto path_str = PathToGenericString(path);
return SysFileCreateDirectory(Common::EndsWith(path_str, "/") ? Common::RemoveLast(path_str, 1)
: path_str);
bool File::CreateDirectory(const std::filesystem::path& path) {
return SysFileCreateDirectory(
WithoutTrailingSeparator(path));
}
bool File::DeleteDirectory(const std::filesystem::path& path) {
auto path_str = PathToGenericString(path);
return SysFileDeleteDirectory(Common::EndsWith(path_str, "/") ? Common::RemoveLast(path_str, 1)
: path_str);
return SysFileDeleteDirectory(
WithoutTrailingSeparator(path));
}
bool File::CreateDirectories(const std::filesystem::path& path) {
std::string real_path = Common::ReplaceChar(PathToGenericString(path), '\\', '/');
const auto normalized_path =
WithoutTrailingSeparator(path);
std::vector<std::string> list = Common::Split(real_path, "/");
auto current_path = normalized_path.root_path();
std::string p;
for (const auto& component:
normalized_path.relative_path()) {
current_path /= component;
for (uint32_t si = 0; si < list.size(); si++) {
const std::string& s = list[si];
if (IsDirectoryExisting(current_path)) {
continue;
}
if (si != 0 || Common::StartsWith(real_path, "/")) {
p += "/";
}
if (!CreateDirectory(current_path)) {
return false;
}
}
p += s;
if (IsDirectoryExisting(p)) {
continue;
}
if (!CreateDirectory(p)) // @suppress("Invalid arguments")
{
return false;
}
}
return true;
return true;
}
bool File::DeleteDirectories(const std::filesystem::path& path) {
std::string real_path = Common::ReplaceChar(PathToGenericString(path), '\\', '/');
const auto normalized_path =
WithoutTrailingSeparator(path);
std::vector<std::string> list = Common::Split(real_path, "/");
std::vector<std::filesystem::path> directories;
auto current_path = normalized_path.root_path();
std::string p;
std::vector<std::string> list2;
for (const auto& component:
normalized_path.relative_path()) {
current_path /= component;
directories.push_back(current_path);
}
for (uint32_t si = 0; si < list.size(); si++) {
const std::string& s = list[si];
for (auto it = directories.rbegin();
it != directories.rend(); ++it) {
if (!DeleteDirectory(*it)) {
return false;
}
}
if (si != 0 || Common::StartsWith(real_path, "/")) {
p += "/";
}
p += s;
list2.push_back(p);
}
uint32_t num = list2.size();
for (uint32_t si = num - 1; si < num; si--) {
if (!DeleteDirectory(list2[si])) {
return false;
}
}
return true;
return true;
}
bool File::DeleteFile(
@@ -345,7 +331,7 @@ bool File::Flush() {
return SysFileFlush(*m_p->f);
}
ByteBuffer File::ReadWholeBuffer() {
std::vector<std::byte> File::ReadWholeBuffer() {
EXIT_IF(IsInvalid());
EXIT_IF(Tell() != 0);
@@ -353,25 +339,14 @@ ByteBuffer File::ReadWholeBuffer() {
EXIT_IF((s >> 32u) != 0);
ByteBuffer buf(s);
const auto read_size = static_cast<uint32_t>(s);
std::vector<std::byte> buf(read_size);
Read(buf.GetData(), s);
Read(buf.data(), read_size);
return buf;
}
ByteBuffer File::Read(uint32_t size) {
ByteBuffer buf(size);
uint32_t b = 0;
Read(buf.GetData(), size, &b);
buf.RemoveAt(b, size - b);
return buf;
}
void File::Write(const ByteBuffer& buf, uint32_t* bytes_written) {
Write(buf.GetDataConst(), buf.Size(), bytes_written);
}
DateTime File::GetLastAccessTimeUTC(const std::filesystem::path& name) {
SysTimeStruct t {};
SysFileToSystemTimeUtc(SysFileGetLastAccessTimeUtc(name), t);
@@ -475,13 +450,7 @@ std::vector<File::FindInfo> File::FindFiles(const std::filesystem::path& path) {
SysFileFindFiles(path, files);
auto path_str = PathToGenericString(path);
uint32_t len = static_cast<uint32_t>(path_str.size());
if (!Common::EndsWith(path_str, "/") && !Common::EndsWith(path_str, "\\")) {
len++;
}
const auto normalized_path = path.lexically_normal();
std::vector<File::FindInfo> ret;
ret.reserve(files.size());
@@ -489,7 +458,7 @@ std::vector<File::FindInfo> File::FindFiles(const std::filesystem::path& path) {
File::FindInfo r {};
r.path_with_name = f.path_with_name;
r.rel_path_with_name = Common::Mid(PathToGenericString(f.path_with_name), len);
r.rel_path_with_name = f.path_with_name.lexically_normal().lexically_relative(normalized_path);
r.size = f.size;
SysTimeStruct at {};
+4 -7
View File
@@ -1,10 +1,10 @@
#ifndef KYTY_COMMON_FILE_H_
#define KYTY_COMMON_FILE_H_
#include "common/byteBuffer.h"
#include "common/common.h"
#include "common/dateTime.h"
#include <cstddef>
#include <filesystem>
#include <memory>
#include <string>
@@ -51,7 +51,6 @@ public:
bool Create(const std::filesystem::path& name);
bool Open(const std::filesystem::path& name, Mode mode);
bool OpenInMem(void* buf, uint32_t buf_size);
bool OpenInMem(ByteBuffer& buf); // NOLINT(google-runtime-references)
bool CreateInMem();
void Close();
@@ -73,14 +72,12 @@ public:
void GetLastAccessAndWriteTimeUTC(DateTime* access, DateTime* write);
void Read(void* data, uint32_t size, uint32_t* bytes_read = nullptr);
ByteBuffer Read(uint32_t size);
void Write(const void* data, uint32_t size, uint32_t* bytes_written = nullptr);
void Write(const ByteBuffer& buf, uint32_t* bytes_written = nullptr);
void Read(void* data, uint32_t size, uint32_t* bytes_read = nullptr);
void Write(const void* data, uint32_t size, uint32_t* bytes_written = nullptr);
void Printf(const char* format, ...) KYTY_FORMAT_PRINTF(2, 3);
ByteBuffer ReadWholeBuffer();
std::vector<std::byte> ReadWholeBuffer();
static uint64_t Size(const std::filesystem::path& name);
-97
View File
@@ -54,103 +54,6 @@ inline uint32_t hash(const void* key, uint32_t key_len) {
return hash;
}
inline uint32_t hash8(uint8_t key) {
uint32_t hash = 0;
uint8_t* ptr = &key;
hash += ptr[0];
hash += (hash << 10u);
hash ^= (hash >> 6u);
hash += (hash << 3u);
hash ^= (hash >> 11u);
hash += (hash << 15u);
return hash;
}
inline uint32_t hash16(uint16_t key) {
uint32_t hash = 0;
auto* ptr = reinterpret_cast<uint8_t*>(&key);
hash += ptr[0];
hash += (hash << 10u);
hash ^= (hash >> 6u);
hash += ptr[1];
hash += (hash << 10u);
hash ^= (hash >> 6u);
hash += (hash << 3u);
hash ^= (hash >> 11u);
hash += (hash << 15u);
return hash;
}
inline uint32_t hash32(uint32_t key) {
uint32_t hash = 0;
auto* ptr = reinterpret_cast<uint8_t*>(&key);
hash += ptr[0];
hash += (hash << 10u);
hash ^= (hash >> 6u);
hash += ptr[1];
hash += (hash << 10u);
hash ^= (hash >> 6u);
hash += ptr[2];
hash += (hash << 10u);
hash ^= (hash >> 6u);
hash += ptr[3];
hash += (hash << 10u);
hash ^= (hash >> 6u);
hash += (hash << 3u);
hash ^= (hash >> 11u);
hash += (hash << 15u);
return hash;
}
inline uint32_t hash64(uint64_t key) {
uint32_t hash = 0;
auto* ptr = reinterpret_cast<uint8_t*>(&key);
hash += ptr[0];
hash += (hash << 10u);
hash ^= (hash >> 6u);
hash += ptr[1];
hash += (hash << 10u);
hash ^= (hash >> 6u);
hash += ptr[2];
hash += (hash << 10u);
hash ^= (hash >> 6u);
hash += ptr[3];
hash += (hash << 10u);
hash ^= (hash >> 6u);
hash += ptr[4];
hash += (hash << 10u);
hash ^= (hash >> 6u);
hash += ptr[5];
hash += (hash << 10u);
hash ^= (hash >> 6u);
hash += ptr[6];
hash += (hash << 10u);
hash ^= (hash >> 6u);
hash += ptr[7];
hash += (hash << 10u);
hash ^= (hash >> 6u);
hash += (hash << 3u);
hash ^= (hash >> 11u);
hash += (hash << 15u);
return hash;
}
} // namespace Common
#endif /* KYTY_COMMON_HASH_H_ */
+12 -5
View File
@@ -3,7 +3,6 @@
#include "common/assert.h"
#include "common/emulatorConfig.h"
#include "common/stringUtils.h"
#include <cstdio>
#include <filesystem>
@@ -47,7 +46,7 @@ std::shared_ptr<spdlog::logger> MakeFileLogger(std::string name
}
auto sink =
std::make_shared<spdlog::sinks::basic_file_sink_mt>(Common::PathToString(path), true);
std::make_shared<spdlog::sinks::basic_file_sink_mt>(path.native(), true);
return MakeLogger(std::move(name), std::move(sink));
}
@@ -129,6 +128,14 @@ static void WriteImpl(std::string_view text, fmt::text_style style = {}) {
}
}
void WriteToConsoleAndLog(std::string_view text) {
WriteImpl(text);
if (g_initialized && g_direction != Direction::Console) {
WriteStdout(text);
}
Flush();
}
void WriteFatal(std::string_view text) {
if (g_direction == Direction::Silent || !g_initialized) {
WriteStdout(text);
@@ -152,9 +159,9 @@ void WriteFatal(fmt::text_style style, std::string_view text) {
void Initialize() {
g_initialized = true;
switch (Config::GetPrintfDirection()) {
case Config::OutputDirection::Silent: g_direction = Direction::Silent; break;
case Config::OutputDirection::Console: g_direction = Direction::Console; break;
case Config::OutputDirection::File: g_direction = Direction::File; break;
case Config::LogDirection::Silent: g_direction = Direction::Silent; break;
case Config::LogDirection::Console: g_direction = Direction::Console; break;
case Config::LogDirection::File: g_direction = Direction::File; break;
}
g_output_file =
(g_direction == Direction::File ? Config::GetPrintfOutputFile() : std::filesystem::path {});
+1
View File
@@ -25,6 +25,7 @@ Direction GetDirection();
bool IsSilent();
void Write(std::string_view text);
void Write(fmt::text_style style, std::string_view text);
void WriteToConsoleAndLog(std::string_view text);
void WriteFatal(std::string_view text);
void WriteFatal(fmt::text_style style, std::string_view text);
void Flush();
-42
View File
@@ -1,42 +0,0 @@
#ifndef KYTY_COMMON_MAGICENUM_H_
#define KYTY_COMMON_MAGICENUM_H_
#include "common/common.h"
#include "common/stringUtils.h"
#include "magic_enum.hpp" // IWYU pragma: export
namespace Common {
template <typename E>
inline std::string EnumName(E v) {
auto str = magic_enum::enum_name(v);
return std::string(str.data(), str.length());
}
template <typename E>
inline std::string EnumName8(E v) {
auto str = magic_enum::enum_name(v);
return std::string(str.data(), str.length());
}
template <typename E>
inline E EnumValue(const std::string& str, E default_value) {
auto v = magic_enum::enum_cast<E>(str.c_str());
if (v.has_value()) {
return v.value();
}
return default_value;
}
#define KYTY_ENUM_RANGE(e, mx, mn) \
namespace magic_enum::customize { \
template <> \
struct enum_range<e> { \
static constexpr int min = (mx); \
static constexpr int max = (mn); \
}; \
}
} // namespace Common
#endif /* KYTY_COMMON_MAGICENUM_H_ */
+1 -3
View File
@@ -30,8 +30,6 @@ struct sys_dbg_stack_info_t {
#endif
};
void SysStackWalk(void** stack, int* depth);
void SysStackUsage(sys_dbg_stack_info_t& s); // NOLINT(google-runtime-references)
void SysStackUsagePrint(sys_dbg_stack_info_t& stack); // NOLINT(google-runtime-references)
void SysStackUsage(sys_dbg_stack_info_t& s); // NOLINT(google-runtime-references)
#endif /* KYTY_COMMON_PLATFORM_SYSDBG_H_ */
-1
View File
@@ -52,7 +52,6 @@ uint64_t SysFileTell(sys_file_t& f); // NOLINT(google-runt
bool SysFileTruncate(sys_file_t& f, uint64_t size); // NOLINT(google-runtime-references)
bool SysFileUnlink(sys_file_t& f,
const std::filesystem::path& name); // NOLINT(google-runtime-references)
void SysFileWrite(uint32_t n, sys_file_t& f); // NOLINT(google-runtime-references)
uint64_t SysFileSize(const std::filesystem::path& file_name);
bool SysFileIsError(sys_file_t& f); // NOLINT(google-runtime-references)
bool SysFileIsDirectoryExisting(const std::filesystem::path& path);
-51
View File
@@ -8,7 +8,6 @@
#include <cstdlib>
#include <cstring>
#include <execinfo.h>
#include <pthread.h>
#include <sys/param.h>
#include <sys/types.h>
@@ -17,56 +16,6 @@
#include <libgen.h> // POSIX basename() lives here on macOS, not in <cstring>
#endif
// Avoid unwinding a guest-owned stack.
static bool OnOwnStack() {
const char* probe = reinterpret_cast<const char*>(&probe);
pthread_attr_t attr {};
#if defined(__APPLE__)
const auto* top = static_cast<const char*>(pthread_get_stackaddr_np(pthread_self()));
const auto size = pthread_get_stacksize_np(pthread_self());
(void)attr;
return top != nullptr && size != 0 && probe < top && probe >= top - size;
#else
if (pthread_getattr_np(pthread_self(), &attr) != 0) {
return false;
}
void* base = nullptr;
size_t size = 0;
const bool ok = pthread_attr_getstack(&attr, &base, &size) == 0 && base != nullptr && size != 0;
pthread_attr_destroy(&attr);
if (!ok) {
return false;
}
const auto* low = static_cast<const char*>(base);
return probe >= low && probe < low + size;
#endif
}
void SysStackWalk(void** stack, int* depth) {
if (stack == nullptr || depth == nullptr || *depth <= 0) {
if (depth != nullptr) {
*depth = 0;
}
return;
}
if (!OnOwnStack()) {
*depth = 0;
return;
}
const int n = ::backtrace(stack, *depth);
*depth = (n < 0 ? 0 : n);
}
void SysStackUsagePrint(sys_dbg_stack_info_t& stack) {
printf("stack: (0x%" PRIx64 ", %" PRIu64 ")\n", static_cast<uint64_t>(stack.commited_addr),
static_cast<uint64_t>(stack.commited_size));
printf("code: (0x%" PRIx64 ", %" PRIu64 ")\n", static_cast<uint64_t>(stack.code_addr),
static_cast<uint64_t>(stack.code_size));
}
void SysStackUsage(sys_dbg_stack_info_t& s) {
pid_t pid = getpid();
+1 -4
View File
@@ -11,6 +11,7 @@
#include <cerrno>
#include <cstdlib>
#include <cstring>
#include <dirent.h>
#include <fcntl.h>
#include <filesystem>
@@ -151,10 +152,6 @@ void SysFileWrite(const void* data, uint32_t size, sys_file_t& f, uint32_t* byte
}
}
void SysFileWrite(uint32_t n, sys_file_t& f) {
SysFileWrite(&n, 4, f);
}
sys_file_t* SysFileCreate(const std::filesystem::path& file_name) {
auto* ret = new sys_file_t;
+37 -106
View File
@@ -5,7 +5,6 @@
#else
#include "common/assert.h"
#include "common/platform/sysVirtual.h"
#include "common/virtualMemory.h"
#include <atomic>
@@ -28,13 +27,12 @@
#define KYTY_FIXED_NOREPLACE
#endif
namespace Common {
namespace Common::VirtualMemory {
static pthread_mutex_t g_virtual_mutex {};
static std::map<uintptr_t, size_t>* g_allocs = nullptr;
static std::map<uintptr_t, int>* g_protects = nullptr;
static std::map<uintptr_t, size_t>* g_allocs = nullptr;
void SysVirtualInit() {
void Init() {
pthread_mutexattr_t attr {};
pthread_mutexattr_init(&attr);
@@ -46,43 +44,27 @@ void SysVirtualInit() {
pthread_mutex_init(&g_virtual_mutex, &attr);
pthread_mutexattr_destroy(&attr);
g_allocs = new std::map<uintptr_t, size_t>;
g_protects = new std::map<uintptr_t, int>;
g_allocs = new std::map<uintptr_t, size_t>;
}
static int get_protection_flag(VirtualMemory::Mode mode) {
static int get_protection_flag(Mode mode) {
int protect = PROT_NONE;
switch (mode) {
case VirtualMemory::Mode::Read: protect = PROT_READ; break;
case VirtualMemory::Mode::Write:
case VirtualMemory::Mode::ReadWrite: protect = PROT_READ | PROT_WRITE; break; // NOLINT
case VirtualMemory::Mode::Execute: protect = PROT_EXEC; break;
case VirtualMemory::Mode::ExecuteRead: protect = PROT_EXEC | PROT_READ; break; // NOLINT
case VirtualMemory::Mode::ExecuteWrite:
case VirtualMemory::Mode::ExecuteReadWrite:
case Mode::Read: protect = PROT_READ; break;
case Mode::Write:
case Mode::ReadWrite: protect = PROT_READ | PROT_WRITE; break; // NOLINT
case Mode::Execute: protect = PROT_EXEC; break;
case Mode::ExecuteRead: protect = PROT_EXEC | PROT_READ; break; // NOLINT
case Mode::ExecuteWrite:
case Mode::ExecuteReadWrite:
protect = PROT_EXEC | PROT_WRITE | PROT_READ;
break; // NOLINT
case VirtualMemory::Mode::NoAccess:
case Mode::NoAccess:
default: protect = PROT_NONE; break;
}
return protect;
}
static VirtualMemory::Mode get_protection_flag(int mode) {
switch (mode) {
case PROT_NONE: return VirtualMemory::Mode::NoAccess;
case PROT_READ: return VirtualMemory::Mode::Read;
case PROT_WRITE: return VirtualMemory::Mode::Write;
case PROT_READ | PROT_WRITE: return VirtualMemory::Mode::ReadWrite; // NOLINT
case PROT_EXEC: return VirtualMemory::Mode::Execute;
case PROT_EXEC | PROT_WRITE: return VirtualMemory::Mode::ExecuteWrite; // NOLINT
case PROT_EXEC | PROT_READ: return VirtualMemory::Mode::ExecuteRead; // NOLINT
case PROT_EXEC | PROT_WRITE | PROT_READ:
return VirtualMemory::Mode::ExecuteReadWrite; // NOLINT
default: return VirtualMemory::Mode::NoAccess;
}
}
// Keep automatic mappings inside the guest and GPU-addressable low window.
#ifdef KYTY_FIXED_NOREPLACE
static constexpr uintptr_t LOW_ARENA_LIMIT = 0x000000FC00000000ULL; // libc mspace window ceiling
@@ -147,7 +129,7 @@ static void* map_anonymous(uintptr_t addr, size_t size, int protect, int flags)
return mmap(nullptr, size, protect, flags, -1, 0); // NOLINT
}
uint64_t SysVirtualAlloc(uint64_t address, uint64_t size, VirtualMemory::Mode mode) {
uint64_t Alloc(uint64_t address, uint64_t size, Mode mode) {
EXIT_IF(g_allocs == nullptr);
auto addr = static_cast<uintptr_t>(address);
@@ -161,11 +143,6 @@ uint64_t SysVirtualAlloc(uint64_t address, uint64_t size, VirtualMemory::Mode mo
if (ptr != MAP_FAILED) {
pthread_mutex_lock(&g_virtual_mutex);
record_alloc(ret_addr, size);
uintptr_t page_start = ret_addr >> 12u;
uintptr_t page_end = (ret_addr + size - 1) >> 12u;
for (uintptr_t page = page_start; page <= page_end; page++) {
(*g_protects)[page] = protect;
}
pthread_mutex_unlock(&g_virtual_mutex);
}
@@ -176,8 +153,7 @@ static uintptr_t align_up(uintptr_t addr, uint64_t alignment) {
return (addr + alignment - 1) & ~(alignment - 1);
}
uint64_t SysVirtualAllocAligned(uint64_t address, uint64_t size, VirtualMemory::Mode mode,
uint64_t alignment) {
uint64_t AllocAligned(uint64_t address, uint64_t size, Mode mode, uint64_t alignment) {
if (alignment == 0) {
return 0;
}
@@ -246,16 +222,11 @@ uint64_t SysVirtualAllocAligned(uint64_t address, uint64_t size, VirtualMemory::
}
if (ptr == MAP_FAILED) {
return SysVirtualAllocAligned(address, size, mode, alignment << 1u);
return AllocAligned(address, size, mode, alignment << 1u);
}
pthread_mutex_lock(&g_virtual_mutex);
record_alloc(ret_addr, size);
uintptr_t page_start = ret_addr >> 12u;
uintptr_t page_end = (ret_addr + size - 1) >> 12u;
for (uintptr_t page = page_start; page <= page_end; page++) {
(*g_protects)[page] = protect;
}
pthread_mutex_unlock(&g_virtual_mutex);
return ret_addr;
@@ -308,7 +279,7 @@ static bool is_mapped(void* ptr, size_t length) {
}
#endif
bool SysVirtualAllocFixed(uint64_t address, uint64_t size, VirtualMemory::Mode mode) {
bool AllocFixed(uint64_t address, uint64_t size, Mode mode) {
EXIT_IF(g_allocs == nullptr);
auto addr = static_cast<uintptr_t>(address);
@@ -337,11 +308,6 @@ bool SysVirtualAllocFixed(uint64_t address, uint64_t size, VirtualMemory::Mode m
if (ptr != MAP_FAILED) {
pthread_mutex_lock(&g_virtual_mutex);
record_alloc(ret_addr, size);
uintptr_t page_start = ret_addr >> 12u;
uintptr_t page_end = (ret_addr + size - 1) >> 12u;
for (uintptr_t page = page_start; page <= page_end; page++) {
(*g_protects)[page] = protect;
}
pthread_mutex_unlock(&g_virtual_mutex);
return true;
@@ -350,15 +316,15 @@ bool SysVirtualAllocFixed(uint64_t address, uint64_t size, VirtualMemory::Mode m
return false;
}
bool SysVirtualCommit(uint64_t address, uint64_t size, VirtualMemory::Mode mode) {
return SysVirtualProtect(address, size, mode);
bool Commit(uint64_t address, uint64_t size, Mode mode) {
return Protect(address, size, mode);
}
uint64_t SysVirtualReserve(uint64_t address, uint64_t size) {
return SysVirtualReserveAligned(address, size, 1);
uint64_t Reserve(uint64_t address, uint64_t size) {
return ReserveAligned(address, size, 1);
}
uint64_t SysVirtualReserveAligned(uint64_t address, uint64_t size, uint64_t alignment) {
uint64_t ReserveAligned(uint64_t address, uint64_t size, uint64_t alignment) {
if (alignment == 0) {
return 0;
}
@@ -427,7 +393,7 @@ uint64_t SysVirtualReserveAligned(uint64_t address, uint64_t size, uint64_t alig
}
if (ptr == MAP_FAILED) {
return SysVirtualReserveAligned(address, size, alignment << 1u);
return ReserveAligned(address, size, alignment << 1u);
}
pthread_mutex_lock(&g_virtual_mutex);
@@ -437,7 +403,7 @@ uint64_t SysVirtualReserveAligned(uint64_t address, uint64_t size, uint64_t alig
return ret_addr;
}
bool SysVirtualReserveFixed(uint64_t address, uint64_t size) {
bool ReserveFixed(uint64_t address, uint64_t size) {
EXIT_IF(g_allocs == nullptr);
auto addr = static_cast<uintptr_t>(address);
@@ -473,9 +439,9 @@ bool SysVirtualReserveFixed(uint64_t address, uint64_t size) {
return false;
}
bool SysVirtualDecommit(uint64_t address, uint64_t size) {
bool Decommit(uint64_t address, uint64_t size) {
// Drop physical pages while preserving the reservation.
if (!SysVirtualProtect(address, size, VirtualMemory::Mode::NoAccess)) {
if (!Protect(address, size, Mode::NoAccess)) {
return false;
}
@@ -499,7 +465,7 @@ bool SysVirtualDecommit(uint64_t address, uint64_t size) {
return true;
}
bool SysVirtualFree(uint64_t address) {
bool Free(uint64_t address) {
EXIT_IF(g_allocs == nullptr);
size_t size = 0;
@@ -516,21 +482,10 @@ bool SysVirtualFree(uint64_t address) {
return false;
}
if (munmap(reinterpret_cast<void*>(addr), size) == 0) {
uintptr_t page_start = addr >> 12u;
uintptr_t page_end = (addr + size - 1) >> 12u;
pthread_mutex_lock(&g_virtual_mutex);
for (uintptr_t page = page_start; page <= page_end; page++) {
g_protects->erase(page);
}
pthread_mutex_unlock(&g_virtual_mutex);
return true;
}
return false;
return munmap(reinterpret_cast<void*>(addr), size) == 0;
}
bool SysVirtualFreeRange(uint64_t address, uint64_t size) {
bool FreeRange(uint64_t address, uint64_t size) {
EXIT_IF(g_allocs == nullptr);
if (size == 0 || (address & 0xfffu) != 0 || (size & 0xfffu) != 0) {
return false;
@@ -582,46 +537,22 @@ bool SysVirtualFreeRange(uint64_t address, uint64_t size) {
if (end < alloc_end) {
(*g_allocs)[end] = alloc_end - end;
}
for (uintptr_t page = addr >> 12u; page <= (end - 1u) >> 12u; page++) {
g_protects->erase(page);
}
pthread_mutex_unlock(&g_virtual_mutex);
return true;
}
bool SysVirtualProtect(uint64_t address, uint64_t size, VirtualMemory::Mode mode,
VirtualMemory::Mode* old_mode) {
auto addr = static_cast<uintptr_t>(address);
pthread_mutex_lock(&g_virtual_mutex);
if (old_mode != nullptr) {
if (auto s = g_protects->find(addr >> 12u); s != g_protects->end()) {
*old_mode = get_protection_flag(s->second);
} else {
*old_mode = VirtualMemory::Mode::NoAccess;
}
}
pthread_mutex_unlock(&g_virtual_mutex);
uintptr_t page_start = addr >> 12u;
uintptr_t page_end = (addr + size - 1) >> 12u;
if (mprotect(reinterpret_cast<void*>(page_start << 12u), (page_end - page_start + 1) << 12u,
get_protection_flag(mode)) == 0) {
pthread_mutex_lock(&g_virtual_mutex);
for (uintptr_t page = page_start; page <= page_end; page++) {
(*g_protects)[page] = get_protection_flag(mode);
}
pthread_mutex_unlock(&g_virtual_mutex);
return true;
}
return false;
bool Protect(uint64_t address, uint64_t size, Mode mode) {
const auto addr = static_cast<uintptr_t>(address);
const auto page_start = addr >> 12u;
const auto page_end = (addr + size - 1) >> 12u;
return mprotect(reinterpret_cast<void*>(page_start << 12u), (page_end - page_start + 1) << 12u,
get_protection_flag(mode)) == 0;
}
bool SysVirtualFlushInstructionCache(uint64_t /*address*/, uint64_t /*size*/) {
bool FlushInstructionCache(uint64_t /*address*/, uint64_t /*size*/) {
return true;
}
} // namespace Common
} // namespace Common::VirtualMemory
#endif
-53
View File
@@ -1,53 +0,0 @@
#ifndef KYTY_COMMON_PLATFORM_SYSSWAPBYTEORDER_H_
#define KYTY_COMMON_PLATFORM_SYSSWAPBYTEORDER_H_
#include "common/common.h"
#include <type_traits>
inline uint16_t SwapByteOrder16(uint16_t value) {
uint16_t hi = value << 8u;
uint16_t lo = value >> 8u;
return hi | lo;
}
inline uint32_t SwapByteOrder32(uint32_t value) {
return __builtin_bswap32(value);
}
inline uint64_t SwapByteOrder64(uint64_t value) {
return __builtin_bswap64(value);
}
template <typename T>
inline void SwapByteOrder(T& x) {
if (sizeof(x) == 2) {
if (std::is_signed_v<T>) {
x = static_cast<std::make_signed_t<T>>(
SwapByteOrder16(static_cast<std::make_unsigned_t<uint16_t>>(x)));
} else {
x = SwapByteOrder16(x);
}
}
if (sizeof(x) == 4) {
if (std::is_signed_v<T>) {
x = static_cast<std::make_signed_t<T>>(
SwapByteOrder32(static_cast<std::make_unsigned_t<uint32_t>>(x)));
} else {
x = SwapByteOrder32(x);
}
}
if (sizeof(x) == 8) {
if (std::is_signed_v<T>) {
x = static_cast<std::make_signed_t<T>>(
SwapByteOrder64(static_cast<std::make_unsigned_t<uint64_t>>(x)));
} else {
x = SwapByteOrder64(x);
}
}
}
template <typename T>
inline void NoSwapByteOrder(T& x) {}
#endif /* KYTY_COMMON_PLATFORM_SYSSWAPBYTEORDER_H_ */
+1 -5
View File
@@ -151,10 +151,6 @@ inline void SysTimeTToSystem(time_t t, SysTimeStruct& s) {
SysFileToSystemTimeUtc(ft, s);
}
inline time_t SysTimegm(struct tm* tm) {
return timegm(tm);
}
inline void SysSystemToFileTimeUtc(const SysTimeStruct& f, SysFileTimeStruct& t) {
struct tm i {};
@@ -165,7 +161,7 @@ inline void SysSystemToFileTimeUtc(const SysTimeStruct& f, SysFileTimeStruct& t)
i.tm_min = f.Minute;
i.tm_sec = f.Second;
t.is_invalid = (f.is_invalid || (t.time = SysTimegm(&i)) == static_cast<time_t>(-1));
t.is_invalid = (f.is_invalid || (t.time = ::timegm(&i)) == static_cast<time_t>(-1));
}
// Retrieves the current local date and time.
-28
View File
@@ -1,28 +0,0 @@
#ifndef KYTY_COMMON_PLATFORM_SYSVIRTUAL_H_
#define KYTY_COMMON_PLATFORM_SYSVIRTUAL_H_
#include "common/common.h"
#include "common/virtualMemory.h"
namespace Common {
void SysVirtualInit();
uint64_t SysVirtualAlloc(uint64_t address, uint64_t size, VirtualMemory::Mode mode);
uint64_t SysVirtualAllocAligned(uint64_t address, uint64_t size, VirtualMemory::Mode mode,
uint64_t alignment);
bool SysVirtualAllocFixed(uint64_t address, uint64_t size, VirtualMemory::Mode mode);
bool SysVirtualCommit(uint64_t address, uint64_t size, VirtualMemory::Mode mode);
uint64_t SysVirtualReserve(uint64_t address, uint64_t size);
uint64_t SysVirtualReserveAligned(uint64_t address, uint64_t size, uint64_t alignment);
bool SysVirtualReserveFixed(uint64_t address, uint64_t size);
bool SysVirtualDecommit(uint64_t address, uint64_t size);
bool SysVirtualFree(uint64_t address);
bool SysVirtualFreeRange(uint64_t address, uint64_t size);
bool SysVirtualProtect(uint64_t address, uint64_t size, VirtualMemory::Mode mode,
VirtualMemory::Mode* old_mode = nullptr);
bool SysVirtualFlushInstructionCache(uint64_t address, uint64_t size);
} // namespace Common
#endif /* KYTY_COMMON_PLATFORM_SYSVIRTUAL_H_ */
-12
View File
@@ -15,18 +15,6 @@
#include "common/assert.h"
#include "common/platform/sysDbg.h"
void SysStackWalk(void** stack, int* depth) {
*depth = static_cast<int>(CaptureStackBackTrace(0, static_cast<DWORD>(*depth), stack, nullptr));
}
void SysStackUsagePrint(sys_dbg_stack_info_t& stack) {
printf("stack: (0x%" PRIx64 ", %" PRIu64 ") + (0x%" PRIx64 ", %" PRIu64 ") + (0x%" PRIx64
", %" PRIu64 ")\n",
static_cast<uint64_t>(stack.reserved_addr), static_cast<uint64_t>(stack.reserved_size),
static_cast<uint64_t>(stack.guard_addr), static_cast<uint64_t>(stack.guard_size),
static_cast<uint64_t>(stack.commited_addr), static_cast<uint64_t>(stack.commited_size));
}
void SysStackUsage(sys_dbg_stack_info_t& s) {
MEMORY_BASIC_INFORMATION mbi {};
[[maybe_unused]] size_t ss = VirtualQuery(&mbi, &mbi, sizeof(mbi));
+14 -32
View File
@@ -14,6 +14,7 @@
#include "common/stringUtils.h"
#include <cstdlib>
#include <cstring>
#include <vector>
// NOLINTNEXTLINE(readability-identifier-naming)
@@ -135,10 +136,6 @@ void SysFileWrite(const void* data, uint32_t size, sys_file_t& f, uint32_t* byte
}
}
void SysFileWrite(uint32_t n, sys_file_t& f) {
SysFileWrite(&n, 4, f);
}
sys_file_t* SysFileCreate(const std::filesystem::path& file_name) {
auto* ret = new sys_file_t;
@@ -480,18 +477,10 @@ bool SysFileSetLastAccessAndWriteTimeUtc(const std::filesystem::path& name,
}
void SysFileFindFiles(const std::filesystem::path& path, std::vector<sys_file_find_t>& out) {
std::string real_path = Common::ReplaceChar(Common::PathToGenericString(path), '\\', '/');
if (!Common::EndsWith(real_path, "/")) {
real_path += "/";
}
const auto pattern = path / L"*";
std::string pattern = real_path + "*";
HANDLE h = nullptr;
WIN32_FIND_DATAW data;
auto wide_pattern = std::filesystem::path(pattern).wstring();
h = FindFirstFileW(wide_pattern.c_str(), &data);
WIN32_FIND_DATAW data {};
HANDLE h = FindFirstFileW(pattern.c_str(), &data);
if (h == INVALID_HANDLE_VALUE) {
return;
@@ -499,18 +488,19 @@ void SysFileFindFiles(const std::filesystem::path& path, std::vector<sys_file_fi
do {
std::filesystem::path file_name(data.cFileName);
auto file_name_str = Common::PathToString(file_name);
if (file_name_str == "." || file_name_str == "..") {
if (file_name == L"." || file_name == L"..") {
continue;
}
const auto full_path = path / file_name;
if ((data.dwFileAttributes & static_cast<DWORD>(FILE_ATTRIBUTE_DIRECTORY)) != 0u) {
SysFileFindFiles(std::filesystem::path(real_path) / file_name, out);
SysFileFindFiles(full_path, out);
} else {
sys_file_find_t r {};
r.path_with_name = std::filesystem::path(real_path) / file_name;
r.path_with_name = full_path;
r.size = (static_cast<uint64_t>(data.nFileSizeHigh) << 32u) +
static_cast<uint64_t>(data.nFileSizeLow);
r.last_access_time.is_invalid = false;
@@ -518,7 +508,7 @@ void SysFileFindFiles(const std::filesystem::path& path, std::vector<sys_file_fi
r.last_write_time.is_invalid = false;
r.last_write_time.time = data.ftLastWriteTime;
out.push_back(r);
out.push_back(std::move(r));
}
} while (FindNextFileW(h, &data) != 0);
@@ -527,18 +517,10 @@ void SysFileFindFiles(const std::filesystem::path& path, std::vector<sys_file_fi
}
void SysFileGetDents(const std::filesystem::path& path, std::vector<sys_dir_entry_t>& out) {
std::string real_path = Common::ReplaceChar(Common::PathToGenericString(path), '\\', '/');
if (!Common::EndsWith(real_path, "/")) {
real_path += "/";
}
const auto pattern = path / L"*";
std::string pattern = real_path + "*";
HANDLE h = nullptr;
WIN32_FIND_DATAW data;
auto wide_pattern = std::filesystem::path(pattern).wstring();
h = FindFirstFileW(wide_pattern.c_str(), &data);
WIN32_FIND_DATAW data {};
HANDLE h = FindFirstFileW(pattern.c_str(), &data);
if (h == INVALID_HANDLE_VALUE) {
return;
@@ -552,7 +534,7 @@ void SysFileGetDents(const std::filesystem::path& path, std::vector<sys_dir_entr
r.is_file = ((data.dwFileAttributes & static_cast<DWORD>(FILE_ATTRIBUTE_DIRECTORY)) == 0u);
r.name = Common::PathToString(file_name);
out.push_back(r);
out.push_back(std::move(r));
} while (FindNextFileW(h, &data) != 0);
+31 -49
View File
@@ -10,7 +10,6 @@
#include <windows.h> // IWYU pragma: keep
#include "common/assert.h"
#include "common/platform/sysVirtual.h"
#include "common/virtualMemory.h"
// IWYU pragma: no_include <basetsd.h>
@@ -22,45 +21,33 @@
// IWYU pragma: no_include <winerror.h>
// IWYU pragma: no_include <wtypes.h>
namespace Common {
namespace Common::VirtualMemory {
static DWORD GetProtectionFlag(VirtualMemory::Mode mode) {
static DWORD GetProtectionFlag(Mode mode) {
DWORD protect = PAGE_NOACCESS;
switch (mode) {
case VirtualMemory::Mode::Read: protect = PAGE_READONLY; break;
case Mode::Read: protect = PAGE_READONLY; break;
case VirtualMemory::Mode::Write:
case VirtualMemory::Mode::ReadWrite: protect = PAGE_READWRITE; break;
case Mode::Write:
case Mode::ReadWrite: protect = PAGE_READWRITE; break;
case VirtualMemory::Mode::Execute: protect = PAGE_EXECUTE; break;
case Mode::Execute: protect = PAGE_EXECUTE; break;
case VirtualMemory::Mode::ExecuteRead: protect = PAGE_EXECUTE_READ; break;
case Mode::ExecuteRead: protect = PAGE_EXECUTE_READ; break;
case VirtualMemory::Mode::ExecuteWrite:
case VirtualMemory::Mode::ExecuteReadWrite: protect = PAGE_EXECUTE_READWRITE; break;
case Mode::ExecuteWrite:
case Mode::ExecuteReadWrite: protect = PAGE_EXECUTE_READWRITE; break;
case VirtualMemory::Mode::NoAccess:
case Mode::NoAccess:
default: protect = PAGE_NOACCESS; break;
}
return protect;
}
static VirtualMemory::Mode GetProtectionFlag(DWORD mode) {
switch (mode) {
case PAGE_NOACCESS: return VirtualMemory::Mode::NoAccess;
case PAGE_READONLY: return VirtualMemory::Mode::Read;
case PAGE_READWRITE: return VirtualMemory::Mode::ReadWrite;
case PAGE_EXECUTE: return VirtualMemory::Mode::Execute;
case PAGE_EXECUTE_READ: return VirtualMemory::Mode::ExecuteRead;
case PAGE_EXECUTE_READWRITE: return VirtualMemory::Mode::ExecuteReadWrite;
default: return VirtualMemory::Mode::NoAccess;
}
}
void Init() {}
void SysVirtualInit() {}
uint64_t SysVirtualAlloc(uint64_t address, uint64_t size, VirtualMemory::Mode mode) {
auto ptr = (address == 0 ? SysVirtualAllocAligned(address, size, mode, 1)
uint64_t Alloc(uint64_t address, uint64_t size, Mode mode) {
auto ptr = (address == 0 ? AllocAligned(address, size, mode, 1)
: reinterpret_cast<uintptr_t>(VirtualAlloc(
reinterpret_cast<LPVOID>(static_cast<uintptr_t>(address)), size,
static_cast<DWORD>(MEM_COMMIT) | static_cast<DWORD>(MEM_RESERVE),
@@ -71,7 +58,7 @@ uint64_t SysVirtualAlloc(uint64_t address, uint64_t size, VirtualMemory::Mode mo
if (err != ERROR_INVALID_ADDRESS) {
printf("VirtualAlloc() failed: 0x%08" PRIx32 "\n", err);
} else {
return SysVirtualAllocAligned(address, size, mode, 1);
return AllocAligned(address, size, mode, 1);
}
}
return ptr;
@@ -92,8 +79,7 @@ static uint64_t AlignUp(uint64_t addr, uint64_t alignment) {
return (addr + alignment - 1) & ~(alignment - 1);
}
uint64_t SysVirtualAllocAligned(uint64_t address, uint64_t size, VirtualMemory::Mode mode,
uint64_t alignment) {
uint64_t AllocAligned(uint64_t address, uint64_t size, Mode mode, uint64_t alignment) {
if (alignment == 0) {
printf("VirtualAlloc2 failed: 0x%08" PRIx32 "\n", static_cast<uint32_t>(GetLastError()));
return 0;
@@ -147,13 +133,13 @@ uint64_t SysVirtualAllocAligned(uint64_t address, uint64_t size, VirtualMemory::
printf("VirtualAlloc2(alignment = 0x%016" PRIx64 ") failed: 0x%08" PRIx32 "\n",
alignment, err);
} else {
return SysVirtualAllocAligned(address, size, mode, alignment << 1u);
return AllocAligned(address, size, mode, alignment << 1u);
}
}
return ptr;
}
bool SysVirtualAllocFixed(uint64_t address, uint64_t size, VirtualMemory::Mode mode) {
bool AllocFixed(uint64_t address, uint64_t size, Mode mode) {
auto ptr = reinterpret_cast<uintptr_t>(VirtualAlloc(
reinterpret_cast<LPVOID>(static_cast<uintptr_t>(address)), size,
static_cast<DWORD>(MEM_COMMIT) | static_cast<DWORD>(MEM_RESERVE), GetProtectionFlag(mode)));
@@ -173,7 +159,7 @@ bool SysVirtualAllocFixed(uint64_t address, uint64_t size, VirtualMemory::Mode m
return true;
}
bool SysVirtualCommit(uint64_t address, uint64_t size, VirtualMemory::Mode mode) {
bool Commit(uint64_t address, uint64_t size, Mode mode) {
auto ptr = reinterpret_cast<uintptr_t>(
VirtualAlloc(reinterpret_cast<LPVOID>(static_cast<uintptr_t>(address)), size,
static_cast<DWORD>(MEM_COMMIT), GetProtectionFlag(mode)));
@@ -191,8 +177,8 @@ bool SysVirtualCommit(uint64_t address, uint64_t size, VirtualMemory::Mode mode)
return true;
}
uint64_t SysVirtualReserve(uint64_t address, uint64_t size) {
auto ptr = (address == 0 ? SysVirtualReserveAligned(address, size, 1)
uint64_t Reserve(uint64_t address, uint64_t size) {
auto ptr = (address == 0 ? ReserveAligned(address, size, 1)
: reinterpret_cast<uintptr_t>(VirtualAlloc(
reinterpret_cast<LPVOID>(static_cast<uintptr_t>(address)), size,
static_cast<DWORD>(MEM_RESERVE), PAGE_NOACCESS)));
@@ -202,13 +188,13 @@ uint64_t SysVirtualReserve(uint64_t address, uint64_t size) {
if (err != ERROR_INVALID_ADDRESS) {
printf("VirtualAlloc(MEM_RESERVE) failed: 0x%08" PRIx32 "\n", err);
} else {
return SysVirtualReserveAligned(address, size, 1);
return ReserveAligned(address, size, 1);
}
}
return ptr;
}
uint64_t SysVirtualReserveAligned(uint64_t address, uint64_t size, uint64_t alignment) {
uint64_t ReserveAligned(uint64_t address, uint64_t size, uint64_t alignment) {
if (alignment == 0) {
printf("VirtualAlloc2(MEM_RESERVE) failed: 0x%08" PRIx32 "\n",
static_cast<uint32_t>(GetLastError()));
@@ -262,13 +248,13 @@ uint64_t SysVirtualReserveAligned(uint64_t address, uint64_t size, uint64_t alig
"\n",
alignment, err);
} else {
return SysVirtualReserveAligned(address, size, alignment << 1u);
return ReserveAligned(address, size, alignment << 1u);
}
}
return ptr;
}
bool SysVirtualReserveFixed(uint64_t address, uint64_t size) {
bool ReserveFixed(uint64_t address, uint64_t size) {
auto ptr = reinterpret_cast<uintptr_t>(
VirtualAlloc(reinterpret_cast<LPVOID>(static_cast<uintptr_t>(address)), size,
static_cast<DWORD>(MEM_RESERVE), PAGE_NOACCESS));
@@ -288,7 +274,7 @@ bool SysVirtualReserveFixed(uint64_t address, uint64_t size) {
return true;
}
bool SysVirtualDecommit(uint64_t address, uint64_t size) {
bool Decommit(uint64_t address, uint64_t size) {
if (VirtualFree(reinterpret_cast<LPVOID>(static_cast<uintptr_t>(address)), size,
MEM_DECOMMIT) == 0) {
printf("VirtualFree(MEM_DECOMMIT) failed: 0x%08" PRIx32 "\n",
@@ -298,7 +284,7 @@ bool SysVirtualDecommit(uint64_t address, uint64_t size) {
return true;
}
bool SysVirtualFree(uint64_t address) {
bool Free(uint64_t address) {
if (VirtualFree(reinterpret_cast<LPVOID>(static_cast<uintptr_t>(address)), 0, MEM_RELEASE) ==
0) {
printf("VirtualFree() failed: 0x%08" PRIx32 "\n", static_cast<uint32_t>(GetLastError()));
@@ -307,7 +293,7 @@ bool SysVirtualFree(uint64_t address) {
return true;
}
bool SysVirtualFreeRange(uint64_t address, uint64_t size) {
bool FreeRange(uint64_t address, uint64_t size) {
if (address == 0 || size == 0) {
return false;
}
@@ -325,24 +311,20 @@ bool SysVirtualFreeRange(uint64_t address, uint64_t size) {
}
current = next;
}
return current - address == size && SysVirtualFree(address);
return current - address == size && Free(address);
}
bool SysVirtualProtect(uint64_t address, uint64_t size, VirtualMemory::Mode mode,
VirtualMemory::Mode* old_mode) {
bool Protect(uint64_t address, uint64_t size, Mode mode) {
DWORD old_protect = 0;
if (VirtualProtect(reinterpret_cast<LPVOID>(static_cast<uintptr_t>(address)), size,
GetProtectionFlag(mode), &old_protect) == 0) {
printf("VirtualProtect() failed: 0x%08" PRIx32 "\n", static_cast<uint32_t>(GetLastError()));
return false;
}
if (old_mode != nullptr) {
*old_mode = GetProtectionFlag(old_protect);
}
return true;
}
bool SysVirtualFlushInstructionCache(uint64_t address, uint64_t size) {
bool FlushInstructionCache(uint64_t address, uint64_t size) {
if (::FlushInstructionCache(GetCurrentProcess(),
reinterpret_cast<LPVOID>(static_cast<uintptr_t>(address)),
size) == 0) {
@@ -353,6 +335,6 @@ bool SysVirtualFlushInstructionCache(uint64_t address, uint64_t size) {
return true;
}
} // namespace Common
} // namespace Common::VirtualMemory
#endif
+7 -13
View File
@@ -55,19 +55,13 @@ void SetThreadName(const char* name) {
}
void Initialize() {
switch (Config::GetProfilerDirection()) {
case Config::ProfilerDirection::Network:
if (!tracy::ProfilerAvailable()) {
tracy::StartupProfiler();
TracySetProgramName("KytyPS5");
::printf("Tracy profiler enabled: client %d.%d.%d, protocol %u, "
"broadcast %u, connect to 127.0.0.1:8086\n",
tracy::Version::Major, tracy::Version::Minor, tracy::Version::Patch,
tracy::ProtocolVersion, tracy::BroadcastVersion);
}
break;
case Config::ProfilerDirection::None:
default: break;
if (Config::ProfilerEnabled() && !tracy::ProfilerAvailable()) {
tracy::StartupProfiler();
TracySetProgramName("KytyPS5");
::printf("Tracy profiler enabled: client %d.%d.%d, protocol %u, "
"broadcast %u, connect to 127.0.0.1:8086\n",
tracy::Version::Major, tracy::Version::Minor, tracy::Version::Patch,
tracy::ProtocolVersion, tracy::BroadcastVersion);
}
}
+8 -178
View File
@@ -1,7 +1,6 @@
#ifndef KYTY_COMMON_STRING_UTILS_H_
#define KYTY_COMMON_STRING_UTILS_H_
#include "common/byteBuffer.h"
#include "common/common.h"
#include <algorithm>
@@ -9,7 +8,6 @@
#include <charconv>
#include <codecvt>
#include <filesystem>
#include <fmt/format.h>
#include <locale>
#include <string>
#include <string_view>
@@ -18,8 +16,6 @@
namespace Common {
constexpr uint32_t FIND_INVALID_INDEX = static_cast<uint32_t>(-1);
inline std::string PathToString(const std::filesystem::path& path) {
#if defined(__cpp_char8_t)
auto u8 = path.u8string();
@@ -38,8 +34,12 @@ inline std::string PathToGenericString(const std::filesystem::path& path) {
#endif
}
inline bool IsSpace(char c) {
return std::isspace(static_cast<unsigned char>(c)) != 0;
inline std::filesystem::path PathFromUtf8(std::string_view text) {
#if defined(__cpp_char8_t)
return std::filesystem::path(std::u8string(text.begin(), text.end()));
#else
return std::filesystem::u8path(text.begin(), text.end());
#endif
}
inline bool EqualNoCase(std::string_view lhs, std::string_view rhs) {
@@ -63,65 +63,6 @@ inline std::string ToLower(std::string_view text) {
return ret;
}
inline bool ContainsStr(std::string_view text, std::string_view needle) {
return text.find(needle) != std::string_view::npos;
}
inline bool StartsWith(std::string_view text, std::string_view prefix) {
return text.starts_with(prefix);
}
inline bool StartsWith(std::string_view text, char prefix) {
return !text.empty() && text.front() == prefix;
}
inline bool EndsWith(std::string_view text, std::string_view suffix) {
return text.ends_with(suffix);
}
inline bool EndsWith(std::string_view text, char suffix) {
return !text.empty() && text.back() == suffix;
}
inline bool ContainsChar(std::string_view text, char needle) {
return text.find(needle) != std::string_view::npos;
}
inline bool IndexValid(std::string_view text, uint32_t index) {
return index < text.size();
}
inline uint32_t FindIndex(std::string_view text, std::string_view needle, size_t from = 0) {
const auto pos = text.find(needle, from);
return pos == std::string_view::npos ? FIND_INVALID_INDEX : static_cast<uint32_t>(pos);
}
inline uint32_t FindIndex(std::string_view text, char needle, size_t from = 0) {
const auto pos = text.find(needle, from);
return pos == std::string_view::npos ? FIND_INVALID_INDEX : static_cast<uint32_t>(pos);
}
inline uint32_t FindLastIndex(std::string_view text, std::string_view needle,
size_t from = FIND_INVALID_INDEX) {
if (needle.empty()) {
return 0;
}
if (from == FIND_INVALID_INDEX || from > text.size()) {
from = text.size();
}
const auto pos = text.rfind(needle, from);
return pos == std::string_view::npos ? FIND_INVALID_INDEX : static_cast<uint32_t>(pos);
}
inline uint32_t FindLastIndex(std::string_view text, char needle,
size_t from = FIND_INVALID_INDEX) {
if (from == FIND_INVALID_INDEX || from > text.size()) {
from = text.size();
}
const auto pos = text.rfind(needle, from);
return pos == std::string_view::npos ? FIND_INVALID_INDEX : static_cast<uint32_t>(pos);
}
inline std::string Mid(std::string_view text, size_t first, size_t count = std::string_view::npos) {
if (first >= text.size()) {
return {};
@@ -129,49 +70,18 @@ inline std::string Mid(std::string_view text, size_t first, size_t count = std::
return std::string(text.substr(first, count));
}
inline std::string Left(std::string_view text, size_t count) {
return Mid(text, 0, count);
}
inline std::string Right(std::string_view text, size_t count) {
if (count >= text.size()) {
return std::string(text);
}
return Mid(text, text.size() - count, count);
}
inline std::string RemoveLast(std::string_view text, size_t count) {
if (count >= text.size()) {
return {};
}
return Left(text, text.size() - count);
return std::string(text.substr(0, text.size() - count));
}
inline std::string RemoveFirst(std::string_view text, size_t count) {
if (count >= text.size()) {
return {};
}
return Right(text, text.size() - count);
}
inline std::string TrimRight(std::string_view text) {
size_t end = text.size();
while (end > 0 && IsSpace(text[end - 1])) {
end--;
}
return std::string(text.substr(0, end));
}
inline std::string TrimLeft(std::string_view text) {
size_t first = 0;
while (first < text.size() && IsSpace(text[first])) {
first++;
}
return std::string(text.substr(first));
}
inline std::string Trim(std::string_view text) {
return TrimRight(TrimLeft(text));
return std::string(text.substr(count));
}
inline std::string ReplaceChar(std::string_view text, char old_char, char new_char) {
@@ -204,16 +114,6 @@ inline std::string FilenameWithoutDirectory(std::string_view text) {
return pos == std::string_view::npos ? std::string(text) : std::string(text.substr(pos + 1));
}
inline std::string FilenameWithoutExtension(std::string_view text) {
const auto pos = text.find_last_of('.');
return pos == std::string_view::npos ? std::string(text) : std::string(text.substr(0, pos));
}
inline std::string ExtensionWithoutFilename(std::string_view text) {
const auto pos = text.find_last_of('.');
return pos == std::string_view::npos ? std::string() : std::string(text.substr(pos));
}
inline std::string FixFilenameSlash(std::string_view text) {
return ReplaceChar(text, '\\', '/');
}
@@ -257,65 +157,12 @@ inline std::vector<std::string> Split(std::string_view text, char sep, bool keep
return Split(text, std::string_view(&sep, 1), keep_empty);
}
inline std::string Concat(const std::vector<std::string>& list, std::string_view sep) {
std::string ret;
for (size_t i = 0; i < list.size(); i++) {
if (i != 0) {
ret += sep;
}
ret += list[i];
}
return ret;
}
inline std::string Concat(const std::vector<std::string>& list, char sep) {
return Concat(list, std::string_view(&sep, 1));
}
inline uint32_t ToUint32(std::string_view text, int base = 10) {
uint32_t value = 0;
std::from_chars(text.data(), text.data() + text.size(), value, base);
return value;
}
inline uint64_t ToUint64(std::string_view text, int base = 10) {
uint64_t value = 0;
std::from_chars(text.data(), text.data() + text.size(), value, base);
return value;
}
inline int32_t ToInt32(std::string_view text, int base = 10) {
int32_t value = 0;
std::from_chars(text.data(), text.data() + text.size(), value, base);
return value;
}
inline int64_t ToInt64(std::string_view text, int base = 10) {
int64_t value = 0;
std::from_chars(text.data(), text.data() + text.size(), value, base);
return value;
}
inline double ToDouble(std::string_view text) {
return std::stod(std::string(text));
}
inline float ToFloat(std::string_view text) {
return std::stof(std::string(text));
}
inline std::string SafeCsv(std::string_view text) {
const bool add_space =
!text.empty() && (text.front() == '+' || text.front() == '=' || text.front() == '-');
if (text.find('"') != std::string_view::npos || text.find(';') != std::string_view::npos ||
text.find('+') != std::string_view::npos || text.find('=') != std::string_view::npos ||
text.find('-') != std::string_view::npos) {
return "\"" + std::string(add_space ? " " : "") +
ReplaceStr(std::string(text), "\"", "\"\"") + "\"";
}
return std::string(text);
}
inline std::string Utf16ToUtf8(std::u16string_view utf16) {
if (utf16.empty()) {
return {};
@@ -324,23 +171,6 @@ inline std::string Utf16ToUtf8(std::u16string_view utf16) {
return convert.to_bytes(utf16.data(), utf16.data() + utf16.size());
}
inline ByteBuffer HexToBin(std::string_view text) {
ByteBuffer ret;
for (size_t i = 0; i + 1 < text.size(); i += 2) {
ret.Add(static_cast<Byte>((ToUint32(text.substr(i, 2), 16) & 0xffu)));
}
return ret;
}
inline std::string HexFromBin(const ByteBuffer& bin) {
std::string ret;
ret.reserve(bin.Size() * 2);
for (uint32_t i = 0; i < bin.Size(); i++) {
ret += fmt::format("{:02X}", std::to_integer<uint8_t>(bin.At(i)));
}
return ret;
}
} // namespace Common
#endif /* KYTY_COMMON_STRING_UTILS_H_ */
+8 -51
View File
@@ -1,7 +1,6 @@
#include "common/threads.h"
#include "common/assert.h"
#include "common/debug.h"
#include <algorithm>
#include <atomic>
@@ -9,7 +8,6 @@
#include <chrono> // IWYU pragma: keep
#include <condition_variable> // IWYU pragma: keep
#include <mutex>
#include <vector>
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS && KYTY_COMPILER == KYTY_COMPILER_CLANG
#define KYTY_WIN_CS
@@ -198,37 +196,7 @@ struct CondVarPrivate {
#endif
};
static std::recursive_mutex g_cond_waiters_mutex;
static std::vector<std::pair<int, CondVarPrivate*>> g_cond_waiters;
static wait_poll_func_t g_cond_wait_poll_callback = nullptr;
static void WakeCondVar(CondVarPrivate* cond_var) {
#ifdef KYTY_WIN_CS
static auto func = ResolveWakeAllConditionVariable();
EXIT_NOT_IMPLEMENTED(func == nullptr);
func(&cond_var->m_cv);
#else
cond_var->m_cv.notify_all();
#endif
}
static void RegisterCondWaiter(CondVarPrivate* cond_var) {
std::lock_guard lock(g_cond_waiters_mutex);
g_cond_waiters.emplace_back(Thread::GetThreadIdUnique(), cond_var);
}
static void UnregisterCondWaiter(CondVarPrivate* cond_var) {
const auto thread_id = Thread::GetThreadIdUnique();
std::lock_guard lock(g_cond_waiters_mutex);
const auto it = std::find_if(g_cond_waiters.begin(), g_cond_waiters.end(),
[thread_id, cond_var](const auto& waiter) {
return waiter.first == thread_id && waiter.second == cond_var;
});
if (it != g_cond_waiters.end()) {
g_cond_waiters.erase(it);
}
}
static wait_poll_func_t g_cond_wait_poll_callback = nullptr;
struct ThreadPrivate {
ThreadPrivate(thread_func_t f, void* a): func(f), arg(a), m_thread(&Run, this) {}
@@ -285,10 +253,6 @@ void Thread::Detach() {
m_thread->m_thread.detach();
}
void Thread::Sleep(uint32_t millis) {
std::this_thread::sleep_for(std::chrono::milliseconds(millis));
}
void Thread::SleepMicro(uint32_t micros) {
#ifdef KYTY_WIN_CS
SleepHighResolution100ns(static_cast<uint64_t>(micros) * 10);
@@ -366,7 +330,6 @@ CondVar::~CondVar() {
}
void CondVar::Wait(Mutex* mutex) {
RegisterCondWaiter(m_cond_var.get());
#ifndef KYTY_WIN_CS
std::unique_lock<std::recursive_mutex> cpp_lock(mutex->m_mutex->m_mutex, std::adopt_lock_t());
#endif
@@ -407,7 +370,6 @@ void CondVar::Wait(Mutex* mutex) {
}
cpp_lock.release();
#endif
UnregisterCondWaiter(m_cond_var.get());
}
void CondVar::SetWaitPollCallback(wait_poll_func_t callback) {
@@ -416,7 +378,6 @@ void CondVar::SetWaitPollCallback(wait_poll_func_t callback) {
bool CondVar::WaitFor(Mutex* mutex, uint32_t micros) {
bool ok = false;
RegisterCondWaiter(m_cond_var.get());
#ifndef KYTY_WIN_CS
std::unique_lock<std::recursive_mutex> cpp_lock(mutex->m_mutex->m_mutex, std::adopt_lock_t());
#endif
@@ -431,7 +392,6 @@ bool CondVar::WaitFor(Mutex* mutex, uint32_t micros) {
std::cv_status::no_timeout);
cpp_lock.release();
#endif
UnregisterCondWaiter(m_cond_var.get());
return ok;
}
@@ -446,16 +406,13 @@ void CondVar::Signal() {
}
void CondVar::SignalAll() {
WakeCondVar(m_cond_var.get());
}
void CondVar::SignalThread(int thread_id) {
std::lock_guard lock(g_cond_waiters_mutex);
for (const auto& waiter: g_cond_waiters) {
if (waiter.first == thread_id) {
WakeCondVar(waiter.second);
}
}
#ifdef KYTY_WIN_CS
static auto func = ResolveWakeAllConditionVariable();
EXIT_NOT_IMPLEMENTED(func == nullptr);
func(&m_cond_var->m_cv);
#else
m_cond_var->m_cv.notify_all();
#endif
}
int Thread::GetThreadIdUnique() {
-2
View File
@@ -31,7 +31,6 @@ public:
// The id is unique and can't be reused by another thread.
[[nodiscard]] int GetUniqueId() const;
static void Sleep(uint32_t millis);
static void SleepMicro(uint32_t micros);
static void SleepNano(uint64_t nanos);
static bool IsMainThread();
@@ -77,7 +76,6 @@ public:
void Signal();
void SignalAll();
static void SignalThread(int thread_id);
static void SetWaitPollCallback(wait_poll_func_t callback);
KYTY_CLASS_NO_COPY(CondVar);
-17
View File
@@ -62,23 +62,6 @@ double Timer::GetTimeS() const {
return (static_cast<double>(current_time - m_StartTime)) / static_cast<double>(m_Frequency);
}
// return time in ticks
uint64_t Timer::GetTicks() const {
if (m_is_paused) {
return (m_PauseTime - m_StartTime);
}
uint64_t current_time = 0;
SysQueryPerformanceCounter(&current_time);
return (current_time - m_StartTime);
}
// return ticks frequency
uint64_t Timer::GetFrequency() const {
return m_Frequency;
}
uint64_t Timer::QueryPerformanceFrequency() {
uint64_t ret = 0;
SysQueryPerformanceFrequency(&ret);
-6
View File
@@ -24,12 +24,6 @@ public:
// return time in seconds
[[nodiscard]] double GetTimeS() const;
// return time in ticks
[[nodiscard]] uint64_t GetTicks() const;
// return ticks frequency
[[nodiscard]] uint64_t GetFrequency() const;
KYTY_CLASS_NO_COPY(Timer);
[[nodiscard]] static uint64_t QueryPerformanceFrequency();
-63
View File
@@ -1,63 +0,0 @@
#include "common/virtualMemory.h"
#include "common/platform/sysVirtual.h"
namespace Common {
namespace VirtualMemory {
void Init() {
SysVirtualInit();
}
uint64_t Alloc(uint64_t address, uint64_t size, Mode mode) {
return SysVirtualAlloc(address, size, mode);
}
uint64_t AllocAligned(uint64_t address, uint64_t size, Mode mode, uint64_t alignment) {
return SysVirtualAllocAligned(address, size, mode, alignment);
}
bool AllocFixed(uint64_t address, uint64_t size, Mode mode) {
return SysVirtualAllocFixed(address, size, mode);
}
bool Commit(uint64_t address, uint64_t size, Mode mode) {
return SysVirtualCommit(address, size, mode);
}
uint64_t Reserve(uint64_t address, uint64_t size) {
return SysVirtualReserve(address, size);
}
uint64_t ReserveAligned(uint64_t address, uint64_t size, uint64_t alignment) {
return SysVirtualReserveAligned(address, size, alignment);
}
bool ReserveFixed(uint64_t address, uint64_t size) {
return SysVirtualReserveFixed(address, size);
}
bool Decommit(uint64_t address, uint64_t size) {
return SysVirtualDecommit(address, size);
}
bool Free(uint64_t address) {
return SysVirtualFree(address);
}
bool FreeRange(uint64_t address, uint64_t size) {
return SysVirtualFreeRange(address, size);
}
bool Protect(uint64_t address, uint64_t size, Mode mode, Mode* old_mode) {
return SysVirtualProtect(address, size, mode, old_mode);
}
bool FlushInstructionCache(uint64_t address, uint64_t size) {
return SysVirtualFlushInstructionCache(address, size);
}
} // namespace VirtualMemory
} // namespace Common
+1 -1
View File
@@ -35,7 +35,7 @@ bool ReserveFixed(uint64_t address, uint64_t size);
bool Decommit(uint64_t address, uint64_t size);
bool Free(uint64_t address);
bool FreeRange(uint64_t address, uint64_t size);
bool Protect(uint64_t address, uint64_t size, Mode mode, Mode* old_mode = nullptr);
bool Protect(uint64_t address, uint64_t size, Mode mode);
bool FlushInstructionCache(uint64_t address, uint64_t size);
} // namespace VirtualMemory
+4
View File
@@ -196,6 +196,10 @@ void Run(const RunOptions& options) {
ClearDebugTextureFolder();
PrintSystemInfo();
std::string title_id;
if (Loader::SystemContentParamSfoGetString("TITLE_ID", &title_id) && !title_id.empty()) {
Log::WriteToConsoleAndLog(fmt::format("Title ID: {}\n", title_id));
}
int ok = atexit(KytyClose);
EXIT_NOT_IMPLEMENTED(ok != 0);
@@ -32,11 +32,12 @@ private:
struct BufferCursor {
std::span<const uint32_t> commands;
uint32_t offset_dw = 0;
uint32_t deferred_advance_dw = 0;
uint32_t offset_dw = 0;
};
std::vector<BufferCursor> m_buffer_stack;
std::span<const uint32_t> m_next_buffer;
bool m_chain = false;
bool m_suspended = false;
bool m_made_progress = false;
};
@@ -129,7 +130,7 @@ public:
[[nodiscard]] bool ShouldSkipPredicatedPackets() const { return m_predicate_skip; }
Pm4ProcessResult Process(Pm4Execution& execution, std::span<const uint32_t> commands);
void ProcessIndirectBuffer(std::span<const uint32_t> commands);
void ProcessIndirectBuffer(std::span<const uint32_t> commands, bool chain);
void SetFlip(const FlipInfo& flip) { m_flip = flip; }
@@ -143,12 +144,11 @@ private:
uint32_t cache_action, uint32_t event_index, uint32_t event_write_source,
void* dst_gpu_addr, T value, uint32_t interrupt_selector,
uint32_t interrupt_context_id);
void ProcessPm4(Pm4Execution& execution, size_t stop_depth);
void ProcessPm4(Pm4Execution& execution);
void SuspendPm4();
CommandScheduler& GetScheduler() const { return m_renderer.GetCommandScheduler(); }
CommandBuffer& CurrentBuffer() { return GetScheduler().Current(); }
void CheckBuffer() const { GetScheduler().CheckActive(); }
GpuResourceManager& GetGpuResources() const { return m_renderer.GetGpuResources(); }
RenderContext& m_renderer;
HW::Context m_ctx;
@@ -315,8 +315,17 @@ static void HwCtxSetDepthBoundsRegister(CommandProcessor& cp, uint32_t cmd_offse
}
}
static void HwCtxIgnoreDepthMetadataRegister([[maybe_unused]] uint32_t cmd_offset,
[[maybe_unused]] uint32_t value) {}
static void HwCtxSetDepthMetadataRegister(CommandProcessor& cp, uint32_t cmd_offset,
uint32_t value) {
if (cmd_offset == Pm4::DB_RENDER_OVERRIDE) {
HW::DepthRenderOverride control;
control.force_z_valid = (value & 0x20000000u) != 0;
control.force_z_dirty = (value & 0x08000000u) != 0;
control.force_stencil_valid = (value & 0x40000000u) != 0;
control.force_stencil_dirty = (value & 0x10000000u) != 0;
cp.GetCtx().SetDepthRenderOverride(control);
}
}
static void HwCtxIgnoreFovWindow([[maybe_unused]] uint32_t value) {}
@@ -389,7 +398,7 @@ KYTY_HW_CTX_PARSER(HwCtxSetDepthMetadataRegisters) {
auto num_values = KYTY_PM4_LEN(cmd_id) - 2u;
for (uint32_t i = 0; i < num_values; i++) {
HwCtxIgnoreDepthMetadataRegister(cmd_offset + i, buffer[i]);
HwCtxSetDepthMetadataRegister(cp, cmd_offset + i, buffer[i]);
}
return num_values;
@@ -1486,10 +1495,10 @@ KYTY_CP_OP_PARSER(CpOpBranch) {
reinterpret_cast<uint64_t>(else_buffer), else_num_dw);
if (take_then) {
cp.ProcessIndirectBuffer({then_buffer, then_num_dw});
cp.ProcessIndirectBuffer({then_buffer, then_num_dw}, true);
} else if (mode == 2 && else_num_dw != 0) {
EXIT_NOT_IMPLEMENTED(else_buffer == nullptr);
cp.ProcessIndirectBuffer({else_buffer, else_num_dw});
cp.ProcessIndirectBuffer({else_buffer, else_num_dw}, true);
}
return payload_dw;
@@ -1950,7 +1959,7 @@ KYTY_CP_OP_PARSER(CpOpIndirectBuffer) {
GraphicsDbgDumpDcb("ci", indirect_num_dw, indirect_buffer);
cp.ProcessIndirectBuffer({indirect_buffer, indirect_num_dw});
cp.ProcessIndirectBuffer({indirect_buffer, indirect_num_dw}, (control & (1u << 20u)) != 0);
return 3;
}
@@ -3003,28 +3012,28 @@ void GraphicsInitJmpTablesCxIndirect() {
HwCtxIgnoreCbDccControl(value);
};
g_hw_ctx_indirect_func[Pm4::DB_COUNT_CONTROL] = [](KYTY_HW_CTX_INDIRECT_ARGS) {
HwCtxIgnoreDepthMetadataRegister(cmd_offset, value);
HwCtxSetDepthMetadataRegister(cp, cmd_offset, value);
};
for (auto cmd_offset = Pm4::DB_SRESULTS_COMPARE_STATE0;
cmd_offset <= Pm4::DB_SRESULTS_COMPARE_STATE1; cmd_offset++) {
g_hw_ctx_indirect_func[cmd_offset] = [](KYTY_HW_CTX_INDIRECT_ARGS) {
HwCtxIgnoreDepthMetadataRegister(cmd_offset, value);
HwCtxSetDepthMetadataRegister(cp, cmd_offset, value);
};
}
g_hw_ctx_indirect_func[Pm4::DB_RENDER_OVERRIDE] = [](KYTY_HW_CTX_INDIRECT_ARGS) {
HwCtxIgnoreDepthMetadataRegister(cmd_offset, value);
HwCtxSetDepthMetadataRegister(cp, cmd_offset, value);
};
g_hw_ctx_indirect_func[Pm4::DB_RENDER_OVERRIDE2] = [](KYTY_HW_CTX_INDIRECT_ARGS) {
HwCtxIgnoreDepthMetadataRegister(cmd_offset, value);
HwCtxSetDepthMetadataRegister(cp, cmd_offset, value);
};
g_hw_ctx_indirect_func[Pm4::DB_DFSM_CONTROL] = [](KYTY_HW_CTX_INDIRECT_ARGS) {
HwCtxIgnoreDepthMetadataRegister(cmd_offset, value);
HwCtxSetDepthMetadataRegister(cp, cmd_offset, value);
};
g_hw_ctx_indirect_func[Pm4::DB_RMI_L2_CACHE_CONTROL] = [](KYTY_HW_CTX_INDIRECT_ARGS) {
HwCtxIgnoreDepthMetadataRegister(cmd_offset, value);
HwCtxSetDepthMetadataRegister(cp, cmd_offset, value);
};
g_hw_ctx_indirect_func[Pm4::CB_RMI_GL2_CACHE_CONTROL] = [](KYTY_HW_CTX_INDIRECT_ARGS) {
HwCtxIgnoreDepthMetadataRegister(cmd_offset, value);
HwCtxSetDepthMetadataRegister(cp, cmd_offset, value);
};
g_hw_ctx_indirect_func[Pm4::TA_BC_BASE_ADDR] = [](KYTY_HW_CTX_INDIRECT_ARGS) {
HwCtxIgnoreBorderColorTableAddr(cmd_offset, value);
@@ -3626,6 +3635,11 @@ void GraphicsInitJmpTablesShIndirect() {
g_hw_sh_indirect_func[Pm4::SPI_SHADER_PGM_RSRC4_GS] = [](KYTY_HW_SH_INDIRECT_ARGS) {
HwShIgnoreShaderRegister(cmd_offset, value);
};
// Toolkit state restoration emits the compiler's GS-front allocation
// metadata here. Native GS resource registers already carry that allocation.
g_hw_sh_indirect_func[0x0ca] = [](KYTY_HW_SH_INDIRECT_ARGS) {
HwShIgnoreShaderRegister(cmd_offset, value);
};
g_hw_sh_indirect_func[Pm4::SPI_GRAPHICS_SHADER_CONTROL_GS] = [](KYTY_HW_SH_INDIRECT_ARGS) {
HwShIgnoreShaderRegister(cmd_offset, value);
};
@@ -3645,12 +3659,13 @@ void GraphicsInitJmpTablesShIndirect() {
g_hw_sh_indirect_func[Pm4::SPI_GRAPHICS_SHADER_CONTROL_HS] = [](KYTY_HW_SH_INDIRECT_ARGS) {
HwShIgnoreShaderRegister(cmd_offset, value);
};
g_hw_sh_indirect_func[Pm4::SPI_SHADER_USER_DATA_ADDR_LO_HS] = [](KYTY_HW_SH_INDIRECT_ARGS) {
HwShIgnoreShaderRegister(cmd_offset, value);
};
g_hw_sh_indirect_func[Pm4::SPI_SHADER_USER_DATA_ADDR_HI_HS] = [](KYTY_HW_SH_INDIRECT_ARGS) {
HwShIgnoreShaderRegister(cmd_offset, value);
};
for (uint32_t offset = Pm4::SPI_SHADER_USER_DATA_ADDR_LO_HS;
offset <= Pm4::SPI_SHADER_USER_DATA_ADDR_HI_HS; offset++) {
g_hw_sh_indirect_func[offset] = [](KYTY_HW_SH_INDIRECT_ARGS) {
cp.GetShCtx().SetHsUserDataAddress(cmd_offset - Pm4::SPI_SHADER_USER_DATA_ADDR_LO_HS,
value);
};
}
g_hw_sh_indirect_func[Pm4::SPI_SHADER_PGM_LO_HS] = [](KYTY_HW_SH_INDIRECT_ARGS) {
auto base = cp.GetShCtx().GetVs().hs_regs.data_addr;
base &= 0xFFFFFF00000000FFull;
@@ -3809,6 +3824,25 @@ void GraphicsInitJmpTablesUcIndirect() {
for (auto& func: g_hw_uc_indirect_func) {
func = nullptr;
}
for (uint32_t i = 0; i < 4; ++i) {
g_hw_uc_indirect_func[Pm4::FSR_CONTROL_POINTS_LEFT_X + i] = [](KYTY_HW_UC_INDIRECT_ARGS) {
cp.GetUcfg().SetFsrControlPoint(0, cmd_offset - Pm4::FSR_CONTROL_POINTS_LEFT_X, value);
};
g_hw_uc_indirect_func[Pm4::FSR_CONTROL_POINTS_LEFT_Y + i] = [](KYTY_HW_UC_INDIRECT_ARGS) {
cp.GetUcfg().SetFsrControlPoint(1, cmd_offset - Pm4::FSR_CONTROL_POINTS_LEFT_Y, value);
};
}
for (uint32_t i = 0; i < 2; ++i) {
g_hw_uc_indirect_func[Pm4::FSR_ALPHA_LEFT_X + i] = [](KYTY_HW_UC_INDIRECT_ARGS) {
cp.GetUcfg().SetFsrAlpha(0, cmd_offset - Pm4::FSR_ALPHA_LEFT_X, value);
};
g_hw_uc_indirect_func[Pm4::FSR_ALPHA_LEFT_Y + i] = [](KYTY_HW_UC_INDIRECT_ARGS) {
cp.GetUcfg().SetFsrAlpha(1, cmd_offset - Pm4::FSR_ALPHA_LEFT_Y, value);
};
g_hw_uc_indirect_func[Pm4::FSR_WINDOW_LEFT + i] = [](KYTY_HW_UC_INDIRECT_ARGS) {
cp.GetUcfg().SetFsrWindow(cmd_offset - Pm4::FSR_WINDOW_LEFT, value);
};
}
g_hw_uc_indirect_func[Pm4::GE_CNTL] = [](KYTY_HW_UC_INDIRECT_ARGS) {
HW::GeControl r;
+4
View File
@@ -26,6 +26,10 @@ enum class PrimitiveType : uint32_t {
kPolygon = 21,
};
constexpr bool IsRectList(PrimitiveType type) {
return type == PrimitiveType::kRectList || type == PrimitiveType::kRectListLegacy;
}
enum class IndexType : uint32_t {
kIndex16 = 0,
kIndex32 = 1,
+72 -67
View File
@@ -77,7 +77,7 @@ private:
static bool GraphicsRunDebugDumpEnabled() {
return Config::GraphicsDebugDumpEnabled() &&
Config::GetPrintfDirection() != Config::OutputDirection::Silent;
Config::GetPrintfDirection() != Config::LogDirection::Silent;
}
GuestGpu::GuestGpu(RenderContext& renderer): m_renderer(renderer) {
@@ -386,8 +386,12 @@ void CommandProcessor::WriteReferenceClock(uint64_t dst_address, uint32_t num_by
}
const auto value = Sync::ReadReferenceClock();
std::memcpy(reinterpret_cast<void*>(dst_address), &value, num_bytes);
LOGF("\t copy_data reference clock: dst=0x%016" PRIx64 " value=0x%016" PRIx64 " size=%u\n",
dst_address, value, num_bytes);
static std::atomic<uint32_t> clock_log_count {0};
if (clock_log_count.fetch_add(1) < 64) {
LOGF("\t copy_data reference clock: dst=0x%016" PRIx64 " value=0x%016" PRIx64
" size=%u\n",
dst_address, value, num_bytes);
}
}
void CommandProcessor::DmaData(uint8_t engine, uint8_t dst_sel, uint8_t dst_cache_policy,
@@ -427,7 +431,7 @@ void CommandProcessor::DmaData(uint8_t engine, uint8_t dst_sel, uint8_t dst_cach
if (!decode_gds(dst_sel, dst_gds)) {
EXIT("unsupported dmaData destination selector 0x%02" PRIx8 "\n", dst_sel);
}
auto& buffer_cache = GetGpuResources().GetBufferCache();
auto& buffer_cache = m_renderer.GetBufferCache();
if (src_sel == 2) {
buffer_cache.FillBuffer(
dst_address_or_offset, num_bytes,
@@ -603,11 +607,11 @@ bool GuestGpu::Process(Submission& submission) {
}
if (progressed) {
if (complete) {
m_renderer.GetGpuResources().RunGarbageCollector();
m_renderer.RunGarbageCollector();
}
cp.BufferFlush();
} else if (complete) {
m_renderer.GetGpuResources().RunGarbageCollector();
m_renderer.RunGarbageCollector();
}
break;
}
@@ -629,16 +633,16 @@ bool GuestGpu::Process(Submission& submission) {
Pm4ProcessResult::Complete;
if (submission.command_execution.MadeProgress()) {
if (complete) {
m_renderer.GetGpuResources().RunGarbageCollector();
m_renderer.RunGarbageCollector();
}
cp.BufferFlush();
} else if (complete) {
m_renderer.GetGpuResources().RunGarbageCollector();
m_renderer.RunGarbageCollector();
}
break;
}
case SubmissionType::FlipPreparation:
m_renderer.GetGpuResources().RunGarbageCollector();
m_renderer.RunGarbageCollector();
cp.PrepareCpuFlip(submission.flip_request_id);
break;
}
@@ -672,20 +676,16 @@ Pm4ProcessResult CommandProcessor::Process(Pm4Execution& execution,
Pm4Execution* previous_execution;
} execution_scope(*this, execution);
ProcessPm4(execution, 0);
ProcessPm4(execution);
return execution.m_buffer_stack.empty() ? Pm4ProcessResult::Complete
: Pm4ProcessResult::Blocked;
}
void CommandProcessor::ProcessIndirectBuffer(std::span<const uint32_t> commands) {
void CommandProcessor::ProcessIndirectBuffer(std::span<const uint32_t> commands, bool chain) {
EXIT_IF(g_current_execution == nullptr);
if (commands.empty()) {
return;
}
auto& execution = *g_current_execution;
const auto stop_depth = execution.m_buffer_stack.size();
execution.m_buffer_stack.push_back({commands});
ProcessPm4(execution, stop_depth);
EXIT_IF(!g_current_execution->m_next_buffer.empty());
g_current_execution->m_next_buffer = commands;
g_current_execution->m_chain = chain;
}
void CommandProcessor::SuspendPm4() {
@@ -693,21 +693,13 @@ void CommandProcessor::SuspendPm4() {
g_current_execution->m_suspended = true;
}
void CommandProcessor::ProcessPm4(Pm4Execution& execution, size_t stop_depth) {
while (execution.m_buffer_stack.size() > stop_depth) {
void CommandProcessor::ProcessPm4(Pm4Execution& execution) {
while (!execution.m_buffer_stack.empty()) {
if (g_gpu_state != nullptr) {
g_gpu_state->ProcessCommands();
}
const auto buffer_index = execution.m_buffer_stack.size() - 1;
auto& cursor = execution.m_buffer_stack[buffer_index];
auto& cursor = execution.m_buffer_stack.back();
EXIT_IF(cursor.offset_dw > cursor.commands.size());
if (cursor.deferred_advance_dw != 0) {
EXIT_IF(cursor.deferred_advance_dw > cursor.commands.size() - cursor.offset_dw);
cursor.offset_dw += cursor.deferred_advance_dw;
cursor.deferred_advance_dw = 0;
execution.m_made_progress = true;
continue;
}
if (cursor.offset_dw == cursor.commands.size()) {
execution.m_buffer_stack.pop_back();
continue;
@@ -783,14 +775,19 @@ void CommandProcessor::ProcessPm4(Pm4Execution& execution, size_t stop_depth) {
handler(*this, packet_header & ~1u, packet + 1, remaining_dw, total_dw) + 1;
EXIT_IF(packet_dw > remaining_dw);
if (execution.m_suspended) {
if (execution.m_buffer_stack.size() > buffer_index + 1) {
execution.m_buffer_stack[buffer_index].deferred_advance_dw = packet_dw;
}
return;
}
EXIT_IF(execution.m_buffer_stack.size() != buffer_index + 1);
execution.m_buffer_stack[buffer_index].offset_dw += packet_dw;
cursor.offset_dw += packet_dw;
execution.m_made_progress = true;
if (!execution.m_next_buffer.empty()) {
// Chains and taken branches reuse the fetcher; only calls retain a return cursor.
if (execution.m_chain) {
cursor = {execution.m_next_buffer};
} else {
execution.m_buffer_stack.push_back({execution.m_next_buffer});
}
execution.m_next_buffer = {};
}
}
}
@@ -825,36 +822,55 @@ void CommandProcessor::SetNumInstances(uint32_t num_instances) {
void CommandProcessor::SetPredication(uint32_t condition, uint32_t op, uint32_t wait_op,
const volatile void* address, uint32_t count_in_dwords) {
if (wait_op != 0) {
BufferFlushAndWait();
}
(void)count_in_dwords;
uint64_t value = 0;
switch (op) {
case 0x00: {
case 0x00:
m_predicate_skip = false;
} break;
case 0x03: {
return;
case 0x01: {
EXIT_NOT_IMPLEMENTED(address == nullptr);
auto value = *reinterpret_cast<const volatile uint64_t*>(address);
switch (condition) {
case 0x00: m_predicate_skip = (value != 0); break;
case 0x01: m_predicate_skip = (value == 0); break;
default: EXIT("unknown predication condition: 0x%08" PRIx32 "\n", condition);
}
static std::atomic<uint32_t> log_count {0};
if (log_count.fetch_add(1) < 128) {
LOGF("\t bool predication: addr=0x%016" PRIx64 ", value=0x%016" PRIx64
", condition=%" PRIu32 ", skip=%u, wait_op=%" PRIu32 "\n",
reinterpret_cast<uint64_t>(address), value, condition,
m_predicate_skip ? 1u : 0u, wait_op);
// One begin/end pair per DB; bit 63 marks each counter ready.
constexpr uint64_t ready_bit = 1ull << 63u;
const auto* results = reinterpret_cast<const volatile uint64_t*>(address);
for (uint32_t db = 0; db < 16u; db++) {
const auto begin = results[db * 2u];
const auto end = results[db * 2u + 1u];
if ((begin & end & ready_bit) == 0) {
if (wait_op == 0) {
SuspendPm4();
} else {
m_predicate_skip = false;
}
return;
}
value += end - begin;
}
} break;
case 0x03:
if (wait_op != 0) {
BufferFlushAndWait();
}
EXIT_NOT_IMPLEMENTED(address == nullptr);
value = *reinterpret_cast<const volatile uint64_t*>(address);
break;
default: EXIT("unknown predication op: 0x%08" PRIx32 "\n", op);
}
switch (condition) {
case 0x00: m_predicate_skip = (value != 0); break;
case 0x01: m_predicate_skip = (value == 0); break;
default: EXIT("unknown predication condition: 0x%08" PRIx32 "\n", condition);
}
if (op == 0x03) {
static std::atomic<uint32_t> log_count {0};
if (log_count.fetch_add(1) < 128) {
LOGF("\t bool predication: addr=0x%016" PRIx64 ", value=0x%016" PRIx64
", condition=%" PRIu32 ", skip=%u, wait_op=%" PRIu32 "\n",
reinterpret_cast<uint64_t>(address), value, condition,
m_predicate_skip ? 1u : 0u, wait_op);
}
}
}
void CommandProcessor::DrawIndex(DrawIndexArgs args) {
@@ -1090,17 +1106,6 @@ void CommandProcessor::DispatchDirect(uint32_t thread_group_x, uint32_t thread_g
// local_x = std::max(cs.num_thread_x, 1u);
// local_y = std::max(cs.num_thread_y, 1u);
// local_z = std::max(cs.num_thread_z, 1u);
if (cs.wave_size == 64u) {
static std::atomic_bool logged_wave64_shader {false};
if (!logged_wave64_shader.exchange(true, std::memory_order_relaxed)) {
LOGF("warning: executing wave64 compute shader cs=0x%016" PRIx64 "\n",
cs.data_addr);
std::printf("warning: executing wave64 compute shader cs=0x%016" PRIx64 "\n",
cs.data_addr);
std::fflush(stdout);
}
}
m_renderer.GetRenderExecutor().DispatchDirect(m_submit_id, CurrentBuffer(), thread_group_x,
thread_group_y, thread_group_z, mode);
}
@@ -1292,6 +1297,7 @@ void CommandProcessor::WriteAtEndOfPipe(uint32_t cache_policy, uint32_t event_wr
break;
case 0x14:
case 0x28:
case 0x2f:
if (event_index == 0x00) {
write64(false);
return;
@@ -1299,7 +1305,6 @@ void CommandProcessor::WriteAtEndOfPipe(uint32_t cache_policy, uint32_t event_wr
break;
case 0x2b:
case 0x2d:
case 0x2f:
case 0x30:
if (event_index == 0x00 && !with_interrupt) {
write64(false);
+50 -14
View File
@@ -202,6 +202,13 @@ struct RenderControl {
uint8_t copy_sample = 0;
};
struct DepthRenderOverride {
bool force_z_valid = false;
bool force_z_dirty = false;
bool force_stencil_valid = false;
bool force_stencil_dirty = false;
};
struct GdsOaCounter {
uint32_t counter = 0;
uint32_t address = 0;
@@ -491,7 +498,8 @@ struct LsStageRegisters {
};
struct HsStageRegisters {
uint64_t data_addr = 0;
uint64_t data_addr = 0;
uint64_t user_data_addr = 0;
HsShaderResource1 rsrc1;
HsShaderResource2 rsrc2;
};
@@ -809,6 +817,12 @@ public:
void SetClipControl(const ClipControl& control) { m_clip_control = control; }
[[nodiscard]] const RenderControl& GetRenderControl() const { return m_render_control; }
void SetRenderControl(const RenderControl& control) { m_render_control = control; }
[[nodiscard]] const DepthRenderOverride& GetDepthRenderOverride() const {
return m_depth_render_override;
}
void SetDepthRenderOverride(const DepthRenderOverride& control) {
m_depth_render_override = control;
}
[[nodiscard]] const DepthControl& GetDepthControl() const { return m_depth_control; }
void SetDepthControl(const DepthControl& control) { m_depth_control = control; }
[[nodiscard]] const ModeControl& GetModeControl() const { return m_mode_control; }
@@ -904,15 +918,16 @@ private:
uint32_t m_shader_stages = 0;
DepthRenderTarget m_depth_render_target;
RenderControl m_render_control;
DepthControl m_depth_control;
StencilControl m_stencil_control;
StencilMask m_stencil_mask;
float m_depth_clear_value = 0.0f;
float m_depth_bounds_min = 0.0f;
float m_depth_bounds_max = 1.0f;
uint8_t m_stencil_clear_value = 0;
DepthRenderTarget m_depth_render_target;
RenderControl m_render_control;
DepthRenderOverride m_depth_render_override;
DepthControl m_depth_control;
StencilControl m_stencil_control;
StencilMask m_stencil_mask;
float m_depth_clear_value = 0.0f;
float m_depth_bounds_min = 0.0f;
float m_depth_bounds_max = 1.0f;
uint8_t m_stencil_clear_value = 0;
ModeControl m_mode_control;
PolyOffset m_poly_offset;
@@ -921,6 +936,12 @@ private:
ShaderRegisters m_sh_regs;
};
struct FsrView {
uint32_t control_points[2][4] {};
uint32_t alphas[2][2] {};
uint32_t window[2] {};
};
class UserConfig {
public:
UserConfig() = default;
@@ -950,8 +971,17 @@ public:
void SetGdsOaCntl(uint32_t value) { m_gds_oa.cntl = value; }
void SetGdsOaCounter(uint32_t value) { m_gds_oa.counters[m_gds_oa.GetIndex()].counter = value; }
void SetGdsOaAddress(uint32_t value) { m_gds_oa.counters[m_gds_oa.GetIndex()].address = value; }
void SetFsrControlPoint(uint32_t axis, uint32_t index, uint32_t value) {
m_fsr_view.control_points[axis][index] = value;
}
void SetFsrAlpha(uint32_t axis, uint32_t index, uint32_t value) {
m_fsr_view.alphas[axis][index] = value;
}
void SetFsrWindow(uint32_t index, uint32_t value) { m_fsr_view.window[index] = value; }
[[nodiscard]] const FsrView& GetFsrView() const { return m_fsr_view; }
private:
FsrView m_fsr_view;
Prospero::PrimitiveType m_prim_type = Prospero::PrimitiveType::kNone;
uint32_t m_index_offset = 0;
uint32_t m_object_id = 0;
@@ -974,14 +1004,14 @@ public:
void SetEsShaderBase(uint64_t addr) { m_vs.es_regs.data_addr = addr; }
void SetLsShaderBase(uint64_t addr) { m_vs.ls_regs.data_addr = addr; }
void SetHsShaderBase(uint64_t addr) { m_vs.hs_regs.data_addr = addr; }
void SetHsUserDataAddress(uint32_t word, uint32_t value) {
SetUserDataAddressWord(m_vs.hs_regs.user_data_addr, word, value);
}
void SetHsShaderResource1(const HsShaderResource1& rsrc1) { m_vs.hs_regs.rsrc1 = rsrc1; }
void SetHsShaderResource2(const HsShaderResource2& rsrc2) { m_vs.hs_regs.rsrc2 = rsrc2; }
void SetGsShaderBase(uint64_t addr) { m_vs.gs_regs.data_addr = addr; }
void SetGsUserDataAddress(uint32_t word, uint32_t value) {
const auto shift = word * 32u;
auto& address = m_vs.gs_regs.user_data_addr;
address = (address & ~(uint64_t {0xffffffffu} << shift)) |
(static_cast<uint64_t>(value) << shift);
SetUserDataAddressWord(m_vs.gs_regs.user_data_addr, word, value);
}
void SetGsShaderResource1(const GsShaderResource1& rsrc1) { m_vs.gs_regs.rsrc1 = rsrc1; }
void SetGsShaderResource2(const GsShaderResource2& rsrc2) { m_vs.gs_regs.rsrc2 = rsrc2; }
@@ -1023,6 +1053,12 @@ public:
[[nodiscard]] const ComputeShaderInfo& GetCs() const { return m_cs; }
private:
static void SetUserDataAddressWord(uint64_t& address, uint32_t word, uint32_t value) {
const auto shift = word * 32u;
address = (address & ~(uint64_t {0xffffffffu} << shift)) |
(static_cast<uint64_t>(value) << shift);
}
VertexShaderInfo m_vs;
PixelShaderInfo m_ps;
ComputeShaderInfo m_cs;
+6
View File
@@ -987,6 +987,12 @@ constexpr uint32_t GE_USER_VGPR_EN_EN_USER_VGPR2_MASK = 0x1;
constexpr uint32_t GE_USER_VGPR_EN_EN_USER_VGPR3_SHIFT = 2;
constexpr uint32_t GE_USER_VGPR_EN_EN_USER_VGPR3_MASK = 0x1;
constexpr uint32_t FSR_WINDOW_LEFT = 0x2BC;
constexpr uint32_t FSR_CONTROL_POINTS_LEFT_X = 0x2C4;
constexpr uint32_t FSR_CONTROL_POINTS_LEFT_Y = 0x2CC;
constexpr uint32_t FSR_ALPHA_LEFT_X = 0x2E4;
constexpr uint32_t FSR_ALPHA_LEFT_Y = 0x2E8;
constexpr uint32_t TA_CS_BC_BASE_ADDR = 0x380;
constexpr uint32_t TA_CS_BC_BASE_ADDR_HI = 0x381;
constexpr uint32_t TEXTURE_GRADIENT_FACTORS = 0x383;
+17 -30
View File
@@ -1,24 +1,18 @@
#include "graphics/guest_gpu/tile.h"
#include "common/alignment.h"
#include "common/assert.h"
#include "common/logging/log.h"
#include "common/profiler.h"
#include "common/stringUtils.h"
#include "graphics/guest_gpu/gpu_defs.h"
#include "graphics/guest_gpu/gpu_format.h"
#include <algorithm>
#include <array>
#include <bit>
#include <fmt/format.h>
#include <vector>
namespace Libs::Graphics {
static uint32_t AlignUp(uint32_t value, uint32_t alignment) {
return (value + alignment - 1u) & ~(alignment - 1u);
}
static uint32_t ShiftCeil(uint32_t value, uint32_t shift) {
return static_cast<uint32_t>((static_cast<uint64_t>(value) + (1ull << shift) - 1ull) >> shift);
}
@@ -33,7 +27,7 @@ static uint32_t CalcLinearAlignedLevelPitch(uint32_t base_width, uint32_t base_h
const uint32_t level_width = ShiftCeil(base_width, level);
const uint32_t level_height = ShiftCeil(base_height, level);
const uint32_t padded_width =
AlignUp(std::max(level_width, 1u), CalcLinearBlockWidth(bytes_per_element));
Common::AlignUp(std::max(level_width, 1u), CalcLinearBlockWidth(bytes_per_element));
const uint64_t size =
static_cast<uint64_t>(padded_width) * std::max(level_height, 1u) * bytes_per_element;
EXIT_NOT_IMPLEMENTED(size > 0xffffffffull);
@@ -61,7 +55,7 @@ static uint32_t SetLinearMipChainLayout(uint32_t levels, const uint32_t* mip_pit
offset += mip_size[level];
}
return AlignUp(offset, 256u);
return Common::AlignUp(offset, 256u);
}
struct Log2BlockDimensions {
@@ -366,8 +360,8 @@ bool TileGetTiledTextureLayout(const TileSurfaceDescription& description, TileSu
((description.width >> level) + texture.texel_width - 1u) / texture.texel_width, 1u);
mip.height = std::max(
((description.height >> level) + texture.texel_height - 1u) / texture.texel_height, 1u);
mip.padded_width = AlignUp(std::max(ShiftCeil(width0, level), 1u), block.block_width);
mip.padded_height = AlignUp(std::max(ShiftCeil(height0, level), 1u), block.block_height);
mip.padded_width = Common::AlignUp(std::max(ShiftCeil(width0, level), 1u), block.block_width);
mip.padded_height = Common::AlignUp(std::max(ShiftCeil(height0, level), 1u), block.block_height);
mip.size = static_cast<uint64_t>(block.block_depth) * mip.padded_width * mip.padded_height *
block.bytes_per_element;
result.block_slice_size += mip.size;
@@ -1121,8 +1115,8 @@ static bool TileGetHtileSize(uint32_t width, uint32_t height, TileSizeAlign& hti
}
// Prospero HTile stores one DWORD per depth tile. Its 32 KiB allocation blocks cover
// 1024x512 pixels, independently of the attachment's fragment count.
const uint64_t size = static_cast<uint64_t>(AlignUp(width, 1024u) / 1024u) *
(AlignUp(height, 512u) / 512u) * 32768u;
const uint64_t size = static_cast<uint64_t>(Common::AlignUp(width, 1024u) / 1024u) *
(Common::AlignUp(height, 512u) / 512u) * 32768u;
if (size == 0 || size > UINT32_MAX) {
return false;
}
@@ -1154,13 +1148,13 @@ bool TileGetDepthSize(uint32_t width, uint32_t height, uint32_t pitch,
&stencil_block_height));
const uint32_t fragments = 1u << num_fragments_log2;
const uint64_t depth_bytes_total =
valid_blocks ? static_cast<uint64_t>(AlignUp(width, depth_block_width)) *
AlignUp(height, depth_block_height) * depth_bytes * fragments
valid_blocks ? static_cast<uint64_t>(Common::AlignUp(width, depth_block_width)) *
Common::AlignUp(height, depth_block_height) * depth_bytes * fragments
: 0;
const uint64_t stencil_bytes_total =
stencil_format == Prospero::StencilFormat::k8UInt && valid_blocks
? static_cast<uint64_t>(AlignUp(width, stencil_block_width)) *
AlignUp(height, stencil_block_height) * fragments
? static_cast<uint64_t>(Common::AlignUp(width, stencil_block_width)) *
Common::AlignUp(height, stencil_block_height) * fragments
: 0;
TileSizeAlign calculated_htile {};
const bool htile_valid = !htile || TileGetHtileSize(width, height, calculated_htile);
@@ -1187,8 +1181,7 @@ uint32_t TileGetRenderTargetPitch(uint32_t width, uint32_t bytes_per_element,
&block_width, &block_height)) {
return 0;
}
const uint64_t pitch = (static_cast<uint64_t>(width) + block_width - 1u) &
~static_cast<uint64_t>(block_width - 1u);
const uint64_t pitch = Common::AlignUp(static_cast<uint64_t>(width), block_width);
return pitch <= UINT32_MAX ? static_cast<uint32_t>(pitch) : 0;
}
@@ -1209,8 +1202,7 @@ bool TileGetRenderTargetSize(uint32_t width, uint32_t height, uint32_t pitch,
pitch != TileGetRenderTargetPitch(width, bytes_per_element, num_fragments_log2)) {
return false;
}
const uint64_t padded_height = (static_cast<uint64_t>(height) + block_height - 1u) &
~static_cast<uint64_t>(block_height - 1u);
const uint64_t padded_height = Common::AlignUp(static_cast<uint64_t>(height), block_height);
const uint64_t size = static_cast<uint64_t>(pitch) * padded_height * bytes_per_element *
(1u << num_fragments_log2);
if (size == 0 || size > UINT32_MAX) {
@@ -1327,13 +1319,8 @@ void TileGetTextureSize(Prospero::BufferFormat format, uint32_t width, uint32_t
return;
}
if (total_size != nullptr && total_size->size == 0) {
std::vector<std::string> list;
list.push_back(fmt::format("format = {}", static_cast<uint32_t>(format)));
list.push_back(fmt::format("width = {}", width));
list.push_back(fmt::format("height = {}", height));
list.push_back(fmt::format("levels = {}", levels));
list.push_back(fmt::format("tile = {}", static_cast<uint32_t>(tile)));
EXIT("unknown format:\n%s\n", Common::Concat(list, '\n').c_str());
EXIT("unknown format:\nformat = %u\nwidth = %u\nheight = %u\nlevels = %u\ntile = %u\n",
static_cast<uint32_t>(format), width, height, levels, static_cast<uint32_t>(tile));
}
}
@@ -1372,7 +1359,7 @@ uint32_t TileGetTexturePitch(Prospero::BufferFormat format, uint32_t width,
TileTextureElementLayout element {};
if (TileGetTextureElementLayout(format, element) && element.texel_width == 1 &&
element.texel_height == 1) {
pitch = AlignUp(pitch, CalcLinearBlockWidth(element.bytes));
pitch = Common::AlignUp(pitch, CalcLinearBlockWidth(element.bytes));
}
break;
}
@@ -1383,7 +1370,7 @@ uint32_t TileGetTexturePitch(Prospero::BufferFormat format, uint32_t width,
case Prospero::TileMode::kRenderTarget: {
TileTextureBlockLayout layout {};
if (TileGetTextureBlockLayout(format, tile, false, layout)) {
pitch = AlignUp(pitch, layout.block.block_width * layout.texel_width);
pitch = Common::AlignUp(pitch, layout.block.block_width * layout.texel_width);
}
break;
}
+1
View File
@@ -86,6 +86,7 @@ struct TileMipLayout {
uint32_t padded_height = 0;
uint32_t tail_x = 0;
uint32_t tail_y = 0;
bool operator==(const TileMipLayout&) const = default;
};
struct TileSurfaceLayout {
+3 -13
View File
@@ -15,7 +15,6 @@
namespace Libs::Graphics {
struct VulkanImage;
struct VulkanMemory;
inline constexpr uint32_t VULKAN_TARGET_API_VERSION = VK_API_VERSION_1_3;
@@ -28,10 +27,11 @@ struct GraphicContext {
vk::Device device = nullptr;
VmaAllocator allocator = nullptr;
bool memory_budget_ext_enabled = false;
bool rt_extensions_enabled = false;
bool compute_subgroup_size_control_enabled = false;
bool sample_rate_shading_enabled = false;
bool attachment_feedback_loop_enabled = false;
bool provoking_vertex_last_enabled = false;
bool supports_block_texel_view = false;
bool mesh_shader_enabled = false;
vk::PhysicalDeviceMeshShaderPropertiesEXT mesh_shader_properties = {};
uint32_t subgroup_size = 0;
@@ -98,10 +98,6 @@ struct GraphicContext {
[[nodiscard]] uint64_t GetTotalMemoryBudget() const;
[[nodiscard]] bool CreateImage(const vk::ImageCreateInfo& info, VulkanImage& image);
void DeleteImage(VulkanImage& image);
void AppendHardwareRayTracingDeviceExtensions(
const std::vector<vk::ExtensionProperties>& available_extensions,
std::vector<const char*>& device_extensions);
void LoadHardwareRayTracingFunctions() const;
uint32_t screen_width = 0;
uint32_t screen_height = 0;
@@ -116,12 +112,6 @@ private:
m_image_format_properties;
};
struct VulkanMemory {
vk::MemoryPropertyFlags property = {};
vk::MemoryPropertyFlags preferred_property = {};
VmaAllocation allocation = nullptr;
};
struct VulkanImageState {
vk::PipelineStageFlags2 pl_stage = vk::PipelineStageFlagBits2::eAllCommands;
vk::AccessFlags2 access_mask = vk::AccessFlagBits2::eNone;
@@ -143,7 +133,7 @@ struct VulkanImage {
vk::Image image = nullptr;
VulkanImageState state;
std::vector<VulkanImageState> subresource_states;
Graphics::VulkanMemory memory;
VmaAllocation allocation = nullptr;
};
+3 -5
View File
@@ -1,5 +1,6 @@
#include "graphics/host_gpu/memoryTracker.h"
#include "common/alignment.h"
#include "common/assert.h"
namespace Libs::Graphics {
@@ -8,9 +9,6 @@ static_assert(std::atomic<void*>::is_always_lock_free);
MemoryTracker::MemoryTracker(PageManager& page_manager): m_page_manager(page_manager) {
m_regions = std::make_unique<std::atomic<RegionManager*>[]>(REGION_COUNT);
for (size_t i = 0; i < REGION_COUNT; i++) {
m_regions[i].store(nullptr, std::memory_order_relaxed);
}
}
MemoryTracker::~MemoryTracker() = default;
@@ -34,8 +32,8 @@ void MemoryTracker::ValidateGpuDirtyPages(const RangeSet& dirty, uint64_t vaddr,
void MemoryTracker::ValidateGpuDirtyOwnership(const RangeSet& dirty, uint64_t vaddr, uint64_t size,
const char* operation) {
ValidateRange(vaddr, size);
const auto begin = vaddr & ~(TRACKER_PAGE_SIZE - 1);
const auto end = (vaddr + size + TRACKER_PAGE_SIZE - 1) & ~(TRACKER_PAGE_SIZE - 1);
const auto begin = Common::AlignDown(vaddr, TRACKER_PAGE_SIZE);
const auto end = Common::AlignUp(vaddr + size, TRACKER_PAGE_SIZE);
for (auto page = begin; page < end; page += TRACKER_PAGE_SIZE) {
const bool has_dirty_bytes = dirty.Intersects(page, TRACKER_PAGE_SIZE);
if (IsRegionGpuModified(page, TRACKER_PAGE_SIZE) != has_dirty_bytes) {
+10 -35
View File
@@ -62,43 +62,18 @@ public:
void ValidateGpuDirtyOwnership(const RangeSet&, uint64_t, uint64_t, const char*) {}
#endif
template <bool clear, typename Preflight, typename Func>
void ForEachDownloadRange(uint64_t vaddr, uint64_t size, Preflight&& preflight, Func&& func) {
static_assert(std::is_nothrow_invocable_v<Preflight&, uint64_t, uint64_t>);
static_assert(std::is_nothrow_invocable_v<Func&, uint64_t, uint64_t>);
CheckNotInUploadCallback();
std::vector<RegionManager*> managers;
Iterate<false>(vaddr, size, [&](RegionManager* manager, uint64_t, uint64_t) {
managers.push_back(manager);
});
std::vector<std::unique_lock<TrackingSpinLock>> locks;
locks.reserve(managers.size());
for (auto* manager: managers) {
locks.emplace_back(manager->lock);
}
Iterate<false>(vaddr, size, [&](RegionManager* manager, uint64_t offset, uint64_t bytes) {
const auto address = manager->GetCpuAddr() + offset;
manager->template ForEachModifiedRange<DirtySource::Gpu, false>(address, bytes,
preflight);
});
Iterate<false>(vaddr, size, [&](RegionManager* manager, uint64_t offset, uint64_t bytes) {
manager->template ForEachModifiedRange<DirtySource::Gpu, false>(
manager->GetCpuAddr() + offset, bytes, func);
});
if constexpr (clear) {
Iterate<false>(vaddr, size,
[&](RegionManager* manager, uint64_t offset, uint64_t bytes) {
const auto address = manager->GetCpuAddr() + offset;
manager->template ForEachModifiedRange<DirtySource::Gpu, true>(
address, bytes, [](uint64_t, uint64_t) noexcept {});
});
}
}
template <bool clear, typename Func>
void ForEachDownloadRange(uint64_t vaddr, uint64_t size, Func&& func) {
ForEachDownloadRange<clear>(
vaddr, size, [](uint64_t, uint64_t) noexcept {}, std::forward<Func>(func));
static_assert(std::is_nothrow_invocable_v<Func&, uint64_t, uint64_t>);
CheckNotInUploadCallback();
Iterate<false>(vaddr, size, [&](RegionManager* manager, uint64_t offset, uint64_t bytes) {
std::scoped_lock lock(manager->lock);
const auto address = manager->GetCpuAddr() + offset;
manager->template ForEachModifiedRange<DirtySource::Gpu, false>(address, bytes, func);
if constexpr (clear) {
manager->template ChangeState<DirtySource::Gpu, false>(address, bytes);
}
});
}
template <typename RangeFunc, typename UploadFunc>
+15 -64
View File
@@ -1,6 +1,7 @@
#include "graphics/host_gpu/pageManager.h"
#include "common/platform/sysDbg.h"
#include "common/alignment.h"
#include "common/virtualMemory.h"
#include "graphics/host_gpu/regionDefinitions.h"
#include "kernel/memory.h"
@@ -21,10 +22,7 @@
#include <windows.h>
#undef min
#undef max
#elif defined(__APPLE__)
#include <unistd.h>
#else
#include <execinfo.h>
#include <unistd.h>
#endif
@@ -36,38 +34,12 @@ constexpr uint64_t REGION_SIZE = TRACKER_REGION_SIZE;
constexpr uint64_t ADDRESS_SIZE = TRACKER_ADDRESS_SIZE;
constexpr uint64_t REGION_COUNT = ADDRESS_SIZE / REGION_SIZE;
#if KYTY_PLATFORM != KYTY_PLATFORM_WINDOWS
// The tracker reuses Win32 memory-protection tags as internal page-state values.
// Mirror their canonical numeric values so the shared state-machine logic is identical.
constexpr uint32_t PAGE_NOACCESS = 0x01;
constexpr uint32_t PAGE_READONLY = 0x02;
constexpr uint32_t PAGE_READWRITE = 0x04;
#endif
constexpr uint64_t REGION_PAGES = REGION_SIZE / PAGE_SIZE;
constexpr uint32_t NO_ACCESS_PROTECTION = PAGE_NOACCESS;
constexpr uint32_t READ_ONLY_PROTECTION = PAGE_READONLY;
constexpr uint32_t READ_WRITE_PROTECTION = PAGE_READWRITE;
[[noreturn]] void FailFast(const char* reason = nullptr) noexcept {
std::fputs("PageManager fail-fast: ", stderr);
std::fputs(reason != nullptr ? reason : "invalid page state", stderr);
std::fputc('\n', stderr);
#if !defined(__APPLE__)
void* frames[16] {};
int frame_count = static_cast<int>(std::size(frames));
SysStackWalk(frames, &frame_count);
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS
const auto image_base = reinterpret_cast<uintptr_t>(GetModuleHandleW(nullptr));
for (int i = 0; i < frame_count; i++) {
const auto address = reinterpret_cast<uintptr_t>(frames[i]);
std::fprintf(stderr, " frame[%d]=0x%016" PRIxPTR " image_rva=0x%016" PRIxPTR "\n", i,
address, address >= image_base ? address - image_base : 0);
}
#else
::backtrace_symbols_fd(frames, frame_count, STDERR_FILENO);
#endif
#endif
std::fflush(stderr);
#if KYTY_PLATFORM == KYTY_PLATFORM_WINDOWS
TerminateProcess(GetCurrentProcess(), static_cast<UINT>(EXCEPTION_NONCONTINUABLE_EXCEPTION));
@@ -86,15 +58,6 @@ constexpr uint32_t READ_WRITE_PROTECTION = PAGE_READWRITE;
std::_Exit(322);
}
Common::VirtualMemory::Mode ToMemoryMode(uint32_t protection) {
switch (protection) {
case NO_ACCESS_PROTECTION: return Common::VirtualMemory::Mode::NoAccess;
case READ_ONLY_PROTECTION: return Common::VirtualMemory::Mode::Read;
case READ_WRITE_PROTECTION: return Common::VirtualMemory::Mode::ReadWrite;
default: Fatal("unmappable protection 0x%08" PRIx32, protection);
}
}
class SpinGuard final {
public:
explicit SpinGuard(std::atomic_flag& lock): m_lock(lock) {
@@ -115,15 +78,6 @@ void ValidateRange(uint64_t vaddr, uint64_t size) {
}
}
uint64_t PageStart(uint64_t vaddr) {
return vaddr & ~(PAGE_SIZE - 1);
}
uint64_t PageEnd(uint64_t vaddr, uint64_t size) {
ValidateRange(vaddr, size);
return PageStart(vaddr + size - 1) + PAGE_SIZE;
}
} // namespace
struct PageManager::Impl {
@@ -131,14 +85,14 @@ struct PageManager::Impl {
uint8_t write_watchers : 7 = 0;
uint8_t access_watchers : 1 = 0;
[[nodiscard]] uint32_t Perms() const noexcept {
[[nodiscard]] Common::VirtualMemory::Mode Perms() const noexcept {
if (access_watchers != 0) {
return NO_ACCESS_PROTECTION;
return Common::VirtualMemory::Mode::NoAccess;
}
if (write_watchers != 0) {
return READ_ONLY_PROTECTION;
return Common::VirtualMemory::Mode::Read;
}
return READ_WRITE_PROTECTION;
return Common::VirtualMemory::Mode::ReadWrite;
}
template <int delta, bool is_read>
@@ -202,9 +156,6 @@ struct PageManager::Impl {
}
#endif
regions = std::make_unique<std::atomic<Region*>[]>(REGION_COUNT);
for (uint64_t i = 0; i < REGION_COUNT; i++) {
regions[i].store(nullptr, std::memory_order_relaxed);
}
}
~Impl() {
@@ -239,11 +190,10 @@ struct PageManager::Impl {
return ptr;
}
void Protect(uint64_t vaddr, uint64_t size, uint32_t protection) noexcept {
if (!Libs::LibKernel::Memory::ProtectGuestHostMemory(vaddr, size,
ToMemoryMode(protection))) {
Fatal("address-space protection failed at 0x%016" PRIx64 ", new=0x%08" PRIx32, vaddr,
protection);
void Protect(uint64_t vaddr, uint64_t size, Common::VirtualMemory::Mode mode) noexcept {
if (!Libs::LibKernel::Memory::ProtectGuestHostMemory(vaddr, size, mode)) {
Fatal("address-space protection failed at 0x%016" PRIx64 ", mode=0x%08" PRIx32, vaddr,
static_cast<uint32_t>(mode));
}
}
@@ -300,11 +250,12 @@ struct PageManager::Impl {
template <bool track, bool is_read>
void UpdatePageWatchers(uint64_t vaddr, uint64_t size) {
const auto begin = PageStart(vaddr);
const auto end = PageEnd(vaddr, size);
ValidateRange(vaddr, size);
const auto begin = Common::AlignDown(vaddr, PAGE_SIZE);
const auto end = Common::AlignUp(vaddr + size, PAGE_SIZE);
for (auto chunk_begin = begin; chunk_begin < end;) {
const auto chunk_end = std::min(end, (chunk_begin / REGION_SIZE + 1) * REGION_SIZE);
const auto region_base = chunk_begin / REGION_SIZE * REGION_SIZE;
const auto chunk_end = std::min(end, Common::AlignUp(chunk_begin + 1, REGION_SIZE));
const auto region_base = Common::AlignDown(chunk_begin, REGION_SIZE);
auto* region = track ? GetOrCreateRegion(chunk_begin) : FindRegion(chunk_begin);
if (region == nullptr) {
Fatal("untracking unknown page 0x%016" PRIx64, chunk_begin);
+2 -14
View File
@@ -6,17 +6,11 @@
#include <algorithm>
#include <cstdint>
#include <map>
#include <vector>
namespace Libs::Graphics {
class RangeSet final {
public:
struct Range {
uint64_t address = 0;
uint64_t size = 0;
};
void Add(uint64_t address, uint64_t size) {
const auto end = End(address, size);
auto it = m_ranges.lower_bound(address);
@@ -62,12 +56,6 @@ public:
}
}
[[nodiscard]] std::vector<Range> Intersections(uint64_t address, uint64_t size) const {
std::vector<Range> result;
ForEachIntersection(address, size, [&result](Range range) { result.push_back(range); });
return result;
}
[[nodiscard]] bool Intersects(uint64_t address, uint64_t size) const {
const auto end = End(address, size);
auto it = m_ranges.lower_bound(address);
@@ -88,7 +76,7 @@ public:
}
template <typename Func>
void ForEachIntersection(uint64_t address, uint64_t size, Func&& func) const {
void ForEachInRange(uint64_t address, uint64_t size, Func&& func) const {
const auto end = End(address, size);
auto it = m_ranges.upper_bound(address);
if (it != m_ranges.begin()) {
@@ -98,7 +86,7 @@ public:
const auto begin = std::max(address, it->first);
const auto last = std::min(end, it->second);
if (begin < last) {
func(Range {begin, last - begin});
func(begin, last);
}
}
}
+20 -39
View File
@@ -16,6 +16,7 @@
#include <windows.h>
#undef min
#undef max
#undef MemoryBarrier
#elif defined(__APPLE__)
#include <pthread.h>
#elif defined(__linux__)
@@ -111,56 +112,43 @@ public:
bits.UnsetRange(start, end);
}
if constexpr (source == DirtySource::Cpu) {
UpdateCpuProtection<!enable>();
UpdateProtection<!enable, false>();
} else {
UpdateGpuProtection<enable>();
UpdateProtection<enable, true>();
}
}
template <DirtySource source, bool clear, typename Func>
void ForEachModifiedRange(uint64_t vaddr, uint64_t size, Func&& func) {
const auto [start, end] = GetPageRange(vaddr, size);
RegionBits mask(GetBits<source>(), start, end);
auto& bits = GetBits<source>();
RegionBits mask(bits, start, end);
if constexpr (clear) {
GetBits<source>().UnsetRange(start, end);
bits.UnsetRange(start, end);
if constexpr (source == DirtySource::Cpu) {
UpdateProtection<true, false>();
} else {
UpdateProtection<false, true>();
}
}
if constexpr (source == DirtySource::Cpu && clear) {
UpdateCpuProtection<true>();
ForEachRange(mask, std::forward<Func>(func));
return;
for (const auto [first, last]: mask) {
func(m_cpu_addr + first * TRACKER_PAGE_SIZE, (last - first) * TRACKER_PAGE_SIZE);
}
if constexpr (source == DirtySource::Gpu && clear) {
UpdateGpuProtection<false>();
}
ForEachRange(mask, std::forward<Func>(func));
}
TrackingSpinLock lock;
private:
template <bool track>
void UpdateCpuProtection() {
auto mask = m_cpu_dirty ^ m_writable;
m_writable = m_cpu_dirty;
template <bool track, bool is_read>
void UpdateProtection() {
const auto protection = is_read ? ~m_gpu_dirty : m_cpu_dirty;
auto& previous = is_read ? m_readable : m_writable;
auto mask = protection ^ previous;
if (mask.None()) {
return;
}
m_page_manager.UpdatePageWatchersForRegion<track>(m_cpu_addr, mask);
}
template <bool track>
void UpdateGpuProtection() {
auto readable = ~m_gpu_dirty;
auto mask = readable ^ m_readable;
m_readable = readable;
if (mask.None()) {
return;
}
if constexpr (track) {
m_page_manager.UpdatePageWatchersForRegion<true, true>(m_cpu_addr, mask);
} else {
m_page_manager.UpdatePageWatchersForRegion<false, true>(m_cpu_addr, mask);
}
previous = protection;
m_page_manager.UpdatePageWatchersForRegion<track, is_read>(m_cpu_addr, mask);
}
template <DirtySource source>
@@ -191,13 +179,6 @@ private:
static_cast<size_t>((offset + size + TRACKER_PAGE_SIZE - 1) / TRACKER_PAGE_SIZE)};
}
template <typename Func>
void ForEachRange(const RegionBits& bits, Func&& func) const {
for (const auto [start, end]: bits) {
func(m_cpu_addr + start * TRACKER_PAGE_SIZE, (end - start) * TRACKER_PAGE_SIZE);
}
}
PageManager& m_page_manager;
uint64_t m_cpu_addr = 0;
RegionBits m_cpu_dirty;
+102 -149
View File
@@ -1,5 +1,6 @@
#include "graphics/host_gpu/renderer/cache/bufferCache.h"
#include "common/alignment.h"
#include "common/assert.h"
#include "common/logging/log.h"
#include "common/profiler.h"
@@ -42,13 +43,6 @@ void BufferCache::WriteDataBuffer(Buffer& buffer, uint64_t address, const void*
}
}
struct BufferCache::DownloadCopy {
Buffer* buffer = nullptr;
uint64_t source_offset = 0;
uint64_t address = 0;
uint64_t size = 0;
};
void BufferCache::Register(BufferId id) {
ChangeRegister<true>(id);
}
@@ -103,8 +97,7 @@ void BufferCache::TouchBuffer(const Buffer& buffer) {
}
void BufferCache::DeleteBuffer(BufferId id) {
auto* buffer = m_slot_buffers.try_get(id);
if (buffer == nullptr || buffer->is_deleted) {
if (IsBufferInvalid(id)) {
return;
}
Unregister(id);
@@ -115,79 +108,70 @@ void BufferCache::DeleteBuffer(BufferId id) {
}
}
std::pair<uint64_t, uint64_t> BufferCache::DownloadEnvelope(const DownloadCopy& copy) {
if (copy.buffer == nullptr || copy.size == 0 || copy.source_offset > copy.buffer->Size() ||
copy.size > copy.buffer->Size() - copy.source_offset) {
EXIT("BufferCache: invalid download copy\n");
bool BufferCache::DownloadBufferMemory(Buffer& buffer, uint64_t vaddr, uint64_t size) {
std::vector<vk::BufferCopy> copies;
uint64_t total_size = 0;
const auto buffer_address = buffer.CpuAddress();
m_memory_tracker.ForEachDownloadRange<false>(
vaddr, size, [&](uint64_t address, uint64_t bytes) noexcept {
m_memory_tracker.ValidateGpuDirtyPages(m_gpu_modified_ranges, address, bytes,
"buffer download");
m_gpu_modified_ranges.ForEachInRange(address, bytes, [&](uint64_t start, uint64_t end) {
copies.emplace_back(start - buffer_address, total_size, end - start);
// Keep packed ranges on separate cache lines, as in shadPS4.
total_size += Common::AlignUp(end - start, 64);
});
m_gpu_modified_ranges.Subtract(address, bytes);
});
if (copies.empty()) {
return false;
}
const auto begin = copy.source_offset & ~uint64_t {3};
if (copy.source_offset > UINT64_MAX - copy.size ||
copy.source_offset + copy.size > UINT64_MAX - 3) {
EXIT("BufferCache: download copy alignment overflow\n");
}
const auto end = (copy.source_offset + copy.size + 3) & ~uint64_t {3};
if (end > copy.buffer->Size()) {
EXIT("BufferCache: aligned download copy exceeds its owner\n");
}
return {begin, end - begin};
}
void BufferCache::DownloadBufferMemory(std::span<const DownloadCopy> copies) {
std::vector<DownloadCopy> batch;
batch.reserve(copies.size());
uint64_t packed_size = 0;
auto& download = m_download_buffer;
const auto flush = [&] {
const auto [mapped, base_offset] = download.Map(packed_size, DOWNLOAD_ALIGNMENT);
EXIT_IF(mapped == nullptr);
uint64_t cursor = 0;
for (const auto& copy: batch) {
const auto [source_begin, envelope_size] = DownloadEnvelope(copy);
download.CopyFrom(m_scheduler.Current(), *copy.buffer, source_begin, base_offset + cursor,
envelope_size, vk::AccessFlagBits::eMemoryWrite, vk::AccessFlags {},
vk::AccessFlagBits::eMemoryRead | vk::AccessFlagBits::eMemoryWrite,
vk::AccessFlagBits::eHostRead);
cursor += AlignDownload(envelope_size);
}
download.Commit();
const auto completion_tick = m_scheduler.CurrentTick();
m_scheduler.Finish();
m_scheduler.WaitPriorityOperations(completion_tick);
cursor = 0;
for (const auto& copy: batch) {
const auto [source_begin, envelope_size] = DownloadEnvelope(copy);
const auto offset = cursor + copy.source_offset - source_begin;
download.Invalidate(base_offset + offset, copy.size);
Libs::LibKernel::Memory::WriteBacking(copy.address, mapped + offset, copy.size);
cursor += AlignDownload(envelope_size);
}
batch.clear();
packed_size = 0;
};
for (auto copy: copies) {
while (copy.size != 0) {
const auto available = download.Size() - packed_size;
const auto prefix = copy.source_offset & 3u;
const auto bytes = std::min(copy.size, available - prefix);
DownloadCopy part {copy.buffer, copy.source_offset, copy.address, bytes};
const auto [source_begin, envelope_size] = DownloadEnvelope(part);
(void)source_begin;
packed_size += AlignDownload(envelope_size);
batch.push_back(part);
copy.source_offset += bytes;
copy.address += bytes;
copy.size -= bytes;
if (packed_size == download.Size()) {
flush();
}
}
const auto [mapped, offset] = m_download_buffer.Map(total_size, 64);
if (mapped == nullptr) {
EXIT("BufferCache: download exceeds 32 MiB staging buffer capacity\n");
}
if (!batch.empty()) {
flush();
}
for (const auto& copy: copies) {
m_gpu_modified_ranges.Subtract(copy.address, copy.size);
m_download_buffer.Commit();
for (auto& copy: copies) {
copy.dstOffset += offset;
}
auto& command = m_scheduler.Current();
command.EndRendering();
const auto native = command.Handle();
vk::BufferMemoryBarrier before {};
before.srcAccessMask = vk::AccessFlagBits::eMemoryRead | vk::AccessFlagBits::eMemoryWrite;
before.dstAccessMask = vk::AccessFlagBits::eTransferRead;
before.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
before.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
before.buffer = buffer.Handle();
before.offset = 0;
before.size = buffer.Size();
native.pipelineBarrier(vk::PipelineStageFlagBits::eAllCommands,
vk::PipelineStageFlagBits::eTransfer, {}, 0, nullptr, 1, &before, 0,
nullptr);
native.copyBuffer(buffer.Handle(), m_download_buffer.Handle(),
static_cast<uint32_t>(copies.size()), copies.data());
auto after = before;
after.srcAccessMask = vk::AccessFlagBits::eTransferWrite;
after.dstAccessMask = vk::AccessFlagBits::eHostRead;
after.buffer = m_download_buffer.Handle();
after.offset = offset;
after.size = total_size;
native.pipelineBarrier(vk::PipelineStageFlagBits::eTransfer,
vk::PipelineStageFlagBits::eAllCommands |
vk::PipelineStageFlagBits::eHost,
{}, 0, nullptr, 1, &after, 0, nullptr);
m_scheduler.DeferPriorityOperation([this, mapped, offset, total_size, buffer_address,
copies = std::move(copies)] {
m_download_buffer.Invalidate(offset, total_size);
for (const auto& copy: copies) {
Libs::LibKernel::Memory::WriteBacking(buffer_address + copy.srcOffset,
mapped + (copy.dstOffset - offset), copy.size);
}
});
return true;
}
BufferCache::BufferCache(GraphicContext& graphics, CommandScheduler& scheduler,
@@ -252,44 +236,29 @@ void BufferCache::ReadMemory(uint64_t vaddr, uint64_t size, bool is_write) {
"addr=0x%016" PRIx64 " size=0x%016" PRIx64 "\n",
vaddr, size);
}
m_scheduler.Context().GetGpu().SendCommandSync(
[this, vaddr, size, is_write] { ReadMemoryOnGpu(vaddr, size, is_write); });
}
m_scheduler.Context().GetGpu().SendCommandSync([this, vaddr, size, is_write] {
if (is_write && !IsRegionRegistered(vaddr, size)) {
return;
}
auto& buffer = m_slot_buffers[FindBuffer(vaddr, size)];
void BufferCache::ReadMemoryOnGpu(uint64_t vaddr, uint64_t size, bool is_write) {
if (is_write && !IsRegionRegistered(vaddr, size)) {
return;
}
auto& buffer = m_slot_buffers[FindBuffer(vaddr, size)];
// Widen nearby CPU reads so they share one GPU drain.
constexpr uint64_t WindowSize = 512 * 1024;
const auto buffer_begin = buffer.CpuAddress();
const auto buffer_end = buffer_begin + buffer.Size();
const auto window_begin = std::max(Common::AlignDown(vaddr, WindowSize), buffer_begin);
const auto window_end = std::min(std::max(window_begin + WindowSize, vaddr + size), buffer_end);
// Widen nearby CPU reads so they share one GPU drain.
constexpr uint64_t WindowSize = 512 * 1024;
const auto buffer_begin = buffer.CpuAddress();
const auto buffer_end = buffer_begin + buffer.Size();
const auto window_begin = std::max(vaddr & ~(WindowSize - 1), buffer_begin);
const auto window_end = std::min(std::max(window_begin + WindowSize, vaddr + size), buffer_end);
std::vector<DownloadCopy> copies;
m_memory_tracker.ForEachDownloadRange<false>(
window_begin, window_end - window_begin,
[&](uint64_t address, uint64_t bytes) noexcept {
m_memory_tracker.ValidateGpuDirtyPages(m_gpu_modified_ranges, address, bytes,
"memory invalidation");
},
[&](uint64_t address, uint64_t bytes) noexcept {
for (const auto range: m_gpu_modified_ranges.Intersections(address, bytes)) {
copies.push_back(
{&buffer, buffer.Offset(range.address), range.address, range.size});
}
});
if (!copies.empty()) {
DownloadBufferMemory(copies);
// The enumeration covered whole dirty pages and every exact interval on them.
m_memory_tracker.UnmarkRegionAsGpuModified(window_begin, window_end - window_begin);
}
if (is_write) {
m_memory_tracker.MarkRegionAsCpuModified(vaddr, size);
}
if (DownloadBufferMemory(buffer, window_begin, window_end - window_begin)) {
const auto tick = m_scheduler.CurrentTick();
m_scheduler.Wait(tick);
m_scheduler.WaitPriorityOperations(tick);
m_memory_tracker.UnmarkRegionAsGpuModified(window_begin, window_end - window_begin);
}
if (is_write) {
m_memory_tracker.MarkRegionAsCpuModified(vaddr, size);
}
});
}
BufferId BufferCache::FindBuffer(uint64_t vaddr, uint64_t size) {
@@ -340,10 +309,12 @@ BufferCache::OverlapResult BufferCache::ResolveOverlaps(uint64_t vaddr, uint64_t
end = std::max(end, buffer_end);
if (!has_stream_leap && (stream_score += buffer.StreamScore()) > StreamLeapThreshold) {
has_stream_leap = true;
if (expands_right) {
// Reserve space in the incoming stream's direction of growth.
// The old buffer extending left of the request predicts growth to the right, and vice versa.
if (expands_left) {
end += std::min(StreamLeapSize, PageTable::kAddressSpaceSize - end);
}
if (expands_left) {
if (expands_right) {
const auto minimum = CACHING_PAGESIZE * 2;
if (begin > minimum) {
begin -= std::min(StreamLeapSize, begin - minimum);
@@ -369,8 +340,8 @@ void BufferCache::JoinOverlap(BufferId new_id, BufferId overlap_id, bool accumul
BufferId BufferCache::CreateBuffer(uint64_t vaddr, uint64_t size) {
EXIT_IF(m_scheduler.Current().IsInvalid());
const auto end = (vaddr + size + CACHING_PAGESIZE - 1) & ~(CACHING_PAGESIZE - 1);
vaddr &= ~(CACHING_PAGESIZE - 1);
const auto end = Common::AlignUp(vaddr + size, CACHING_PAGESIZE);
vaddr = Common::AlignDown(vaddr, CACHING_PAGESIZE);
size = end - vaddr;
const auto overlap = ResolveOverlaps(vaddr, size);
@@ -483,17 +454,16 @@ std::pair<Buffer*, uint64_t> BufferCache::ObtainBuffer(uint64_t vaddr, uint64_t
}
}
auto* buffer = m_slot_buffers.try_get(id);
if (buffer == nullptr || buffer->is_deleted || !buffer->IsInBounds(vaddr, size)) {
id = FindBuffer(vaddr, size);
buffer = &m_slot_buffers[id];
if (IsBufferInvalid(id) || !m_slot_buffers[id].IsInBounds(vaddr, size)) {
id = FindBuffer(vaddr, size);
}
TouchBuffer(*buffer);
(void)SynchronizeBuffer(*buffer, vaddr, size, is_written, is_texel_buffer);
auto& buffer = m_slot_buffers[id];
TouchBuffer(buffer);
(void)SynchronizeBuffer(buffer, vaddr, size, is_written, is_texel_buffer);
if (is_written) {
m_gpu_modified_ranges.Add(vaddr, size);
}
return {buffer, buffer->Offset(vaddr)};
return {&buffer, buffer.Offset(vaddr)};
}
std::pair<Buffer*, uint64_t> BufferCache::ObtainBufferForImage(uint64_t vaddr, uint64_t size) {
@@ -545,9 +515,7 @@ void BufferCache::FillBuffer(uint64_t vaddr, uint64_t size, uint32_t value, bool
}
m_texture_cache.InvalidateMemoryFromGPU(vaddr, size);
const auto id = FindBuffer(vaddr, size);
auto [dst, dst_offset] = ObtainBuffer(vaddr, size, true, true, id);
EXIT_IF(dst == nullptr);
auto [dst, dst_offset] = ObtainBuffer(vaddr, size, true, true);
dst->Fill(dst_offset, size, value);
}
@@ -581,10 +549,6 @@ void BufferCache::CopyBuffer(uint64_t dst_vaddr, uint64_t src_vaddr, uint64_t si
: std::pair {&m_gds_buffer, src_vaddr};
auto [dst, dst_offset] = dst_memory ? ObtainBuffer(dst_vaddr, size, true, true, dst_id)
: std::pair {&m_gds_buffer, dst_vaddr};
EXIT_IF(src == nullptr || dst == nullptr);
if (src == dst && src_offset < dst_offset + size && dst_offset < src_offset + size) {
EXIT("BufferCache: resolved Vulkan copy ranges overlap\n");
}
dst->CopyFrom(command, *src, src_offset, dst_offset, size);
}
@@ -628,8 +592,7 @@ void BufferCache::RunGarbageCollector() {
const size_t limit = aggressive ? 64 : 32;
std::vector<BufferId> dirty_buffers;
std::vector<DownloadCopy> copies;
size_t retire_count = 0;
size_t retire_count = 0;
m_lru_cache.ForEachItemBelow(tick - age, [&](BufferId id) {
auto& buffer = m_slot_buffers[id];
EXIT_IF(buffer.is_deleted);
@@ -640,19 +603,7 @@ void BufferCache::RunGarbageCollector() {
return false;
}
if (dirty) {
m_memory_tracker.ForEachDownloadRange<false>(
buffer.CpuAddress(), buffer.Size(),
[&](uint64_t dirty_address, uint64_t dirty_size) noexcept {
m_memory_tracker.ValidateGpuDirtyPages(m_gpu_modified_ranges, dirty_address,
dirty_size, "garbage collection");
},
[&](uint64_t dirty_address, uint64_t dirty_size) noexcept {
m_gpu_modified_ranges.ForEachIntersection(
dirty_address, dirty_size, [&](RangeSet::Range range) {
copies.push_back({&buffer, range.address - buffer.CpuAddress(),
range.address, range.size});
});
});
EXIT_IF(!DownloadBufferMemory(buffer, buffer.CpuAddress(), buffer.Size()));
dirty_buffers.push_back(id);
} else {
m_memory_tracker.UntrackMemory(buffer.CpuAddress(), buffer.Size());
@@ -664,8 +615,10 @@ void BufferCache::RunGarbageCollector() {
return;
}
EXIT_IF(copies.empty());
DownloadBufferMemory(copies);
// Publish all queued downloads before releasing their tracked pages and owners.
const auto completion_tick = m_scheduler.CurrentTick();
m_scheduler.Wait(completion_tick);
m_scheduler.WaitPriorityOperations(completion_tick);
for (const auto id: dirty_buffers) {
auto& buffer = m_slot_buffers[id];
m_memory_tracker.UnmarkRegionAsGpuModified(buffer.CpuAddress(), buffer.Size());
+7 -8
View File
@@ -74,6 +74,11 @@ public:
private:
friend struct BufferCacheTestAccess;
bool IsBufferInvalid(BufferId id) const {
const auto* buffer = m_slot_buffers.try_get(id);
return buffer == nullptr || buffer->is_deleted;
}
using BufferMap = std::map<uint64_t, BufferId>;
struct OverlapResult {
BufferMap::iterator first;
@@ -83,14 +88,8 @@ private:
bool has_stream_leap;
};
struct DownloadCopy;
using PageTable = MultiLevelPageTable<BufferId, CACHING_PAGEBITS, 40, 16>;
static_assert(CACHING_PAGESIZE == (uint64_t {1} << PageTable::kPageBits));
static constexpr uint64_t DOWNLOAD_ALIGNMENT = 64;
[[nodiscard]] static constexpr uint64_t AlignDownload(uint64_t size) noexcept {
return (size + DOWNLOAD_ALIGNMENT - 1) & ~(DOWNLOAD_ALIGNMENT - 1);
}
[[nodiscard]] static std::pair<uint64_t, uint64_t> DownloadEnvelope(const DownloadCopy& copy);
void WriteDataBuffer(Buffer& buffer, uint64_t address, const void* source, uint64_t size);
void TouchBuffer(const Buffer& buffer);
[[nodiscard]] OverlapResult ResolveOverlaps(uint64_t vaddr, uint64_t size);
@@ -106,8 +105,8 @@ private:
[[nodiscard]] vk::Buffer UploadCopies(Buffer& buffer, std::span<vk::BufferCopy> copies,
uint64_t total_size);
[[nodiscard]] bool SynchronizeBufferFromImage(Buffer& buffer, uint64_t vaddr, uint64_t size);
void DownloadBufferMemory(std::span<const DownloadCopy> copies);
void ReadMemoryOnGpu(uint64_t vaddr, uint64_t size, bool is_write);
// Queues backing publication; callers wait before clearing dirty pages or reusing their data.
[[nodiscard]] bool DownloadBufferMemory(Buffer& buffer, uint64_t vaddr, uint64_t size);
GraphicContext& m_graphics;
CommandScheduler& m_scheduler;
+1 -6
View File
@@ -44,12 +44,7 @@ FaultManager::FaultManager(GraphicContext& graphics, CommandScheduler& scheduler
&m_fault_process_desc_layout),
"create fault-buffer descriptor layout");
vk::ShaderModuleCreateInfo module_info {};
module_info.codeSize = std::size(FAULT_BUFFER_PROCESS_SPV) * sizeof(uint32_t);
module_info.pCode = FAULT_BUFFER_PROCESS_SPV;
vk::ShaderModule module = nullptr;
RequireVulkanSuccess(m_graphics.device.createShaderModule(&module_info, nullptr, &module),
"create fault-buffer shader module");
const auto module = CompileSPV(FAULT_BUFFER_PROCESS_SPV, m_graphics.device);
vk::PipelineLayoutCreateInfo pipeline_layout_info {};
pipeline_layout_info.setLayoutCount = 1;
@@ -1,96 +0,0 @@
#include "graphics/host_gpu/renderer/cache/gpuResourceManager.h"
#include "common/assert.h"
#include "graphics/guest_gpu/graphicsRun.h"
#include "graphics/host_gpu/renderer/commandScheduler.h"
namespace Libs::Graphics {
GpuResourceManager::GpuResourceManager(GraphicContext& graphics, CommandScheduler& scheduler)
: m_scheduler(scheduler), m_buffer_cache(graphics, scheduler, m_page_manager, m_texture_cache),
m_texture_cache(graphics, scheduler, m_page_manager, m_buffer_cache) {}
GpuResourceManager::~GpuResourceManager() = default;
bool GpuResourceManager::HandleFault(PageFaultAccess access, uint64_t fault_vaddr) noexcept {
// The host reports the faulting byte, not the instruction's access width. Both caches
// resolve its page; guessing a width can cross the end of a valid guest mapping.
constexpr uint64_t fault_size = 1;
if (!IsMapped(fault_vaddr, fault_size)) {
return false;
}
if (access == PageFaultAccess::Write) {
m_buffer_cache.InvalidateMemory(fault_vaddr, fault_size);
m_texture_cache.InvalidateMemory(fault_vaddr, fault_size);
} else {
m_buffer_cache.ReadMemory(fault_vaddr, fault_size);
}
return true;
}
bool GpuResourceManager::InvalidateMemory(uint64_t vaddr, uint64_t size) {
if (!IsMapped(vaddr, size)) {
return false;
}
m_buffer_cache.InvalidateMemory(vaddr, size);
m_texture_cache.InvalidateMemory(vaddr, size);
return true;
}
bool GpuResourceManager::IsMapped(uint64_t vaddr, uint64_t size) const noexcept {
if (!GuestRange {vaddr, size}.Valid()) {
return false;
}
std::shared_lock lock(m_mapped_ranges_mutex);
return m_mapped_ranges.Contains(vaddr, size);
}
void GpuResourceManager::MapMemory(uint64_t vaddr, uint64_t size) {
{
std::lock_guard lock(m_mapped_ranges_mutex);
m_mapped_ranges.Add(vaddr, size);
}
}
void GpuResourceManager::UnmapMemory(uint64_t vaddr, uint64_t size) {
if (CommandScheduler::InDeferredOperation()) {
EXIT("unsupported memory unmap from an asynchronous GPU completion, "
"addr=0x%016" PRIx64 " size=0x%016" PRIx64 "\n",
vaddr, size);
}
const auto unmap = [this, vaddr, size] {
if (m_scheduler.Active()) {
const auto tick = m_scheduler.CurrentTick();
m_scheduler.Finish();
m_scheduler.WaitPriorityOperations(tick);
}
m_buffer_cache.InvalidateMemory(vaddr, size);
m_texture_cache.UnmapMemory(vaddr, size);
std::lock_guard lock(m_mapped_ranges_mutex);
m_mapped_ranges.Subtract(vaddr, size);
};
if (m_gpu == nullptr) {
unmap();
return;
}
m_gpu->SendCommandSync(unmap);
}
void GpuResourceManager::PrepareBda() {
std::shared_lock lock(m_mapped_ranges_mutex);
m_mapped_ranges.ForEach([this](uint64_t start, uint64_t end) {
m_buffer_cache.SynchronizeBuffersInRange(start, end - start);
});
m_fault_process_pending = true;
}
void GpuResourceManager::RunGarbageCollector() {
if (m_fault_process_pending) {
m_fault_process_pending = false;
m_buffer_cache.ProcessFaultBuffer();
}
m_texture_cache.ProcessDownloadImages();
m_texture_cache.RunGarbageCollector();
m_buffer_cache.RunGarbageCollector();
}
} // namespace Libs::Graphics
@@ -1,49 +0,0 @@
#ifndef EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_GPURESOURCEMANAGER_H_
#define EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_GPURESOURCEMANAGER_H_
#include "common/abi.h"
#include "common/common.h"
#include "graphics/host_gpu/pageManager.h"
#include "graphics/host_gpu/renderer/cache/bufferCache.h"
#include "graphics/host_gpu/renderer/cache/textureCache.h"
#include <cstdint>
#include <shared_mutex>
namespace Libs::Graphics {
class CommandScheduler;
class GuestGpu;
class GpuResourceManager {
public:
GpuResourceManager(GraphicContext& graphics, CommandScheduler& scheduler);
~GpuResourceManager();
KYTY_CLASS_NO_COPY(GpuResourceManager);
[[nodiscard]] BufferCache& GetBufferCache() { return m_buffer_cache; }
[[nodiscard]] TextureCache& GetTextureCache() { return m_texture_cache; }
void SetGpu(GuestGpu* gpu) noexcept { m_gpu = gpu; }
[[nodiscard]] bool HandleFault(PageFaultAccess access, uint64_t fault_vaddr) noexcept;
[[nodiscard]] bool InvalidateMemory(uint64_t vaddr, uint64_t size);
[[nodiscard]] bool IsMapped(uint64_t vaddr, uint64_t size) const noexcept;
void MapMemory(uint64_t vaddr, uint64_t size);
void UnmapMemory(uint64_t vaddr, uint64_t size);
void PrepareBda();
void RunGarbageCollector();
private:
PageManager m_page_manager;
CommandScheduler& m_scheduler;
BufferCache m_buffer_cache;
TextureCache m_texture_cache;
mutable std::shared_mutex m_mapped_ranges_mutex;
RangeSet m_mapped_ranges;
GuestGpu* m_gpu = nullptr;
bool m_fault_process_pending = false;
};
} // namespace Libs::Graphics
#endif // EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_GPURESOURCEMANAGER_H_
+4 -15
View File
@@ -1,12 +1,12 @@
#include "graphics/host_gpu/renderer/cache/streamBuffer.h"
#include "common/alignment.h"
#include "common/assert.h"
#include "common/profiler.h"
#include "graphics/host_gpu/graphicContext.h"
#include "graphics/host_gpu/renderer/commandScheduler.h"
#include <cstring>
#include <limits>
#include <numeric>
#include <vk_mem_alloc.h>
@@ -45,16 +45,11 @@ constexpr size_t WATCHES_RESERVE_CHUNK = 0x1000;
result = value;
return true;
}
const auto remainder = value % alignment;
if (remainder == 0) {
result = value;
return true;
}
const auto increment = alignment - remainder;
if (value > std::numeric_limits<uint64_t>::max() - increment) {
const auto aligned = Common::AlignUp(value, alignment);
if (aligned < value) {
return false;
}
result = value + increment;
result = aligned;
return true;
}
@@ -124,12 +119,6 @@ bool Buffer::IsInBounds(uint64_t address, uint64_t size) const noexcept {
return address >= m_cpu_address && size <= Size() && address - m_cpu_address <= Size() - size;
}
void Buffer::Write(uint64_t offset, const void* source, uint64_t size) {
EXIT_IF(source == nullptr || m_mapped.empty() || offset > Size() || size > Size() - offset);
std::memcpy(m_mapped.data() + offset, source, static_cast<size_t>(size));
Flush(offset, size);
}
void Buffer::Flush(uint64_t offset, uint64_t size) {
EXIT_IF(m_mapped.empty() || offset > Size() || size > Size() - offset);
if (!IsCoherent() && size != 0) {
-1
View File
@@ -55,7 +55,6 @@ public:
[[nodiscard]] bool IsInBounds(uint64_t address, uint64_t size) const noexcept;
void IncreaseStreamScore(int score) noexcept { stream_score += score; }
[[nodiscard]] int StreamScore() const noexcept { return stream_score; }
void Write(uint64_t offset, const void* source, uint64_t size);
void Flush(uint64_t offset, uint64_t size);
void Invalidate(uint64_t offset, uint64_t size);
void CopyFrom(CommandBuffer& command, const Buffer& source, uint64_t source_offset,
+199 -239
View File
@@ -1,5 +1,6 @@
#include "graphics/host_gpu/renderer/cache/textureCache.h"
#include "common/alignment.h"
#include "common/assert.h"
#include "common/emulatorConfig.h"
#include "common/logging/log.h"
@@ -32,22 +33,24 @@ namespace {
constexpr uint64_t NumFramesBeforeRemoval = 32;
[[nodiscard]] bool DecodeDccClear(const TextureCache::ImageDesc& desc, vk::Format format,
uint32_t fill, vk::ClearColorValue& clear) {
const auto code = static_cast<uint8_t>(fill);
if (fill != static_cast<uint32_t>(code) * 0x01010101u) {
return false;
[[nodiscard]] bool DecodeDccClear(const TextureCache::ImageDesc& desc, uint8_t code,
vk::ClearColorValue& clear) {
switch (code) {
case 0x00:
case 0x20:
case 0x40:
case 0x80:
case 0xc0: break;
default: return false;
}
const auto& metadata = desc.info.metadata;
const auto format = desc.view_info.format;
if (code == 0x20) {
// Clear-to-register is a color-buffer operation; the texture pipe cannot decode it.
return desc.type == TextureCache::BindingType::RenderTarget &&
metadata.dcc_clear_register_valid &&
DecodePackedColorClear(format, metadata.dcc_clear_word, clear);
}
if (code != 0x00 && code != 0x40 && code != 0x80 && code != 0xc0) {
return false;
}
clear = {};
if (code == 0x00) {
return true;
@@ -187,15 +190,11 @@ TextureCache::TextureCache(GraphicContext& graphics, CommandScheduler& scheduler
}
TextureCache::~TextureCache() {
std::vector<ImageId> registered;
m_slot_images.ForEach([&](ImageId id, const Image& image) {
if (image.registered) {
registered.push_back(id);
UnregisterImage(id);
}
});
for (const auto id: registered) {
UnregisterImage(id);
}
}
bool TextureCache::SameBacking(const ImageInfo& cached, const ImageInfo& requested,
@@ -209,6 +208,10 @@ bool TextureCache::SameBacking(const ImageInfo& cached, const ImageInfo& request
if (cached.extent != requested.extent) {
return false;
}
if (cached.resources.levels < requested.resources.levels ||
cached.resources.layers < requested.resources.layers) {
return false;
}
if (cached.samples != requested.samples) {
return false;
}
@@ -270,9 +273,9 @@ void TextureCache::RegisterImage(ImageId id) {
if (!ImagePageTable::TryGetPageRange(image.info.data.address, image.info.data.size, pages)) {
EXIT("TextureCache: image registration is outside the guest address space\n");
}
for (size_t page = pages.first; page < pages.last_exclusive; ++page) {
ForEachPage(image.info.data.address, image.info.data.size, [this, id](uint64_t page) {
m_image_page_table[page].push_back(id);
}
});
image.registered = true;
image.lru_id = m_lru_cache.Insert(id, m_gc_tick);
m_total_used_memory += image.AccountedSize();
@@ -288,12 +291,12 @@ void TextureCache::UnregisterImage(ImageId id) {
if (!ImagePageTable::TryGetPageRange(image.info.data.address, image.info.data.size, pages)) {
EXIT("TextureCache: registered image is outside the guest address space\n");
}
for (size_t page = pages.first; page < pages.last_exclusive; ++page) {
ForEachPage(image.info.data.address, image.info.data.size, [this, id](uint64_t page) {
auto* owners = m_image_page_table.Find(page);
if (owners == nullptr || !owners->Erase(id)) {
EXIT("TextureCache: image missing from page owner index\n");
}
}
});
m_lru_cache.Free(image.lru_id);
const auto accounted = image.AccountedSize();
if (accounted > m_total_used_memory) {
@@ -316,11 +319,7 @@ void TextureCache::DeleteImage(ImageId id) {
}
});
for (const auto association: associations) {
auto& associated = m_slot_images[association];
if (associated.IsGpuModified()) {
associated.ClearGpuModified();
}
DeleteImage(association);
FreeImage(association);
}
}
if (image->IsGpuModified()) {
@@ -330,10 +329,8 @@ void TextureCache::DeleteImage(ImageId id) {
if (image->info.HasMetadata()) {
const auto metadata = m_surface_metas.find(image->info.metadata.range.address);
if (metadata != m_surface_metas.end() &&
((image->info.metadata.kind == ImageMetadataKind::Dcc &&
metadata->second.type == MetaDataInfo::Type::Dcc) ||
(image->info.metadata.kind == ImageMetadataKind::Htile &&
metadata->second.type == MetaDataInfo::Type::HTile))) {
image->info.metadata.kind == ImageMetadataKind::Htile &&
metadata->second.type == MetaDataInfo::Type::HTile) {
// A later binding may have reused this address for another metadata type.
m_surface_metas.erase(metadata);
}
@@ -447,7 +444,7 @@ void TextureCache::UntrackImageHead(ImageId id) {
if (!image.IsTracked() || begin < image.track_addr) {
return;
}
const auto address = (begin + TRACKER_PAGE_SIZE) & ~(TRACKER_PAGE_SIZE - 1);
const auto address = Common::AlignDown(begin + TRACKER_PAGE_SIZE, TRACKER_PAGE_SIZE);
const auto size = address - begin;
image.track_addr = address;
if (image.track_addr == image.track_addr_end) {
@@ -464,7 +461,7 @@ void TextureCache::UntrackImageTail(ImageId id) {
if (!image.IsTracked() || image.track_addr_end < end) {
return;
}
const auto address = end & ~(TRACKER_PAGE_SIZE - 1);
const auto address = Common::AlignDown(end, TRACKER_PAGE_SIZE);
const auto size = end - address;
image.track_addr_end = address;
if (image.track_addr == image.track_addr_end) {
@@ -498,10 +495,10 @@ TextureCache::ImageIds TextureCache::FindImagesInRegion(uint64_t address, uint64
}
ImageIds result;
for (size_t page = pages.first; page < pages.last_exclusive; ++page) {
ForEachPage(address, size, [&](uint64_t page) {
const auto* owners = m_image_page_table.Find(page);
if (owners == nullptr) {
continue;
return;
}
owners->ForEach([&](ImageId id) {
auto* image = m_slot_images.try_get(id);
@@ -516,7 +513,7 @@ TextureCache::ImageIds TextureCache::FindImagesInRegion(uint64_t address, uint64
result.push_back(id);
}
});
}
});
return result;
}
@@ -817,17 +814,10 @@ TextureCache::OverlapResult TextureCache::ResolveOverlap(const ImageInfo& reques
const bool safe_to_delete =
current_tick - std::min(current_tick, cached.tick_accessed_last) > NumFramesBeforeRemoval;
if (requested.data.address == cached.info.data.address) {
const uint32_t requested_block = requested.bytes_per_block * requested.samples;
const uint32_t cached_block = cached.info.bytes_per_block * cached.info.samples;
if (requested.BlockExtent() != cached.info.BlockExtent() ||
requested_block != cached_block) {
if (safe_to_delete) {
FreeImage(cached_id);
}
return {merged_id};
}
const uint32_t requested_block = requested.bytes_per_block * requested.samples;
const uint32_t cached_block = cached.info.bytes_per_block * cached.info.samples;
if (requested.data.address == cached.info.data.address &&
requested.BlockExtent() == cached.info.BlockExtent() && requested_block == cached_block) {
if (const auto depth_id = ResolveDepthOverlap(requested, binding, cached_id)) {
return {depth_id};
}
@@ -857,6 +847,16 @@ TextureCache::OverlapResult TextureCache::ResolveOverlap(const ImageInfo& reques
ImageViewOps::FormatsCompatible(cached.info.pixel_format, requested.pixel_format)) {
return {ExpandImage(requested, cached_id)};
}
// PS5 mip tails can expose more levels without increasing the guest allocation.
if (requested.pixel_format == cached.info.pixel_format &&
requested.type == cached.info.type && requested.resources > cached.info.resources &&
(requested.data.size > cached.info.data.size ||
(requested.data.size == cached.info.data.size &&
requested.extent == cached.info.extent &&
cached.info.resources.levels > 1 &&
requested.resources.layers == cached.info.resources.layers))) {
return {ExpandImage(requested, cached_id)};
}
if (requested.pixel_format != cached.info.pixel_format ||
requested.data.size <= cached.info.data.size) {
const auto result_id = merged_id ? merged_id : cached_id;
@@ -866,9 +866,6 @@ TextureCache::OverlapResult TextureCache::ResolveOverlap(const ImageInfo& reques
? result_id
: ImageId {}};
}
if (requested.type == cached.info.type && requested.resources > cached.info.resources) {
return {ExpandImage(requested, cached_id)};
}
EXIT("TextureCache: unresolvable equal-address image overlap, address=0x%016" PRIx64
" requested=%ux%u "
"cached=%ux%u requested_size=0x%016" PRIx64 " cached_size=0x%016" PRIx64
@@ -880,38 +877,27 @@ TextureCache::OverlapResult TextureCache::ResolveOverlap(const ImageInfo& reques
static_cast<uint32_t>(cached.info.tile_mode));
}
if (requested.data.address > cached.info.data.address) {
const int32_t mip = requested.MipOf(cached.info);
if (mip >= 0) {
const int32_t layer = requested.SliceOf(cached.info, mip);
if (layer >= 0) {
return {cached_id, mip, layer};
}
}
if (safe_to_delete) {
FreeImage(cached_id);
}
return {};
const int32_t requested_mip = requested.MipOf(cached.info);
if (requested_mip >= 0) {
const int32_t layer = requested.SliceOf(cached.info, requested_mip);
return {cached_id, requested_mip, layer};
}
const int32_t mip = cached.info.MipOf(requested);
if (mip >= 0) {
const int32_t layer = cached.info.SliceOf(requested, mip);
if (layer >= 0) {
if (cached.binding.is_target) {
cached.binding.needs_rebind = true;
if (merged_id) {
m_slot_images[merged_id].binding.is_target = true;
}
FreeImage(cached_id);
return {merged_id};
}
if (merged_id) {
CopyImageMip(merged_id, cached_id, static_cast<uint32_t>(mip),
static_cast<uint32_t>(layer));
FreeImage(cached_id);
}
if (!merged_id) {
return {ExpandImage(requested, cached_id)};
}
cached.binding.needs_rebind |= cached.binding.is_bound || cached.binding.is_target;
m_slot_images[merged_id].binding.is_target |= cached.binding.is_target;
CopyImageMip(merged_id, cached_id, static_cast<uint32_t>(mip),
static_cast<uint32_t>(layer));
FreeImage(cached_id);
return {merged_id};
}
if (requested.data.address >= cached.info.data.address && safe_to_delete) {
FreeImage(cached_id);
}
return {merged_id};
}
@@ -926,12 +912,19 @@ ImageId TextureCache::ExpandImage(const ImageInfo& info, ImageId source_id) {
source.binding.needs_rebind = true;
}
InitializeImage(expanded_id);
CopyImage(expanded_id, source_id);
const int32_t mip = source.info.MipOf(info);
const int32_t layer = source.info.SliceOf(info, mip);
if (layer >= 0) {
CopyImageMip(expanded_id, source_id, static_cast<uint32_t>(mip),
static_cast<uint32_t>(layer));
} else {
CopyImage(expanded_id, source_id);
}
FreeImage(source_id);
return expanded_id;
}
struct TextureCache::TextureTransferPlan {
struct TextureCache::TextureTransfer {
TextureUploadLayout layout;
std::vector<vk::BufferImageCopy> regions;
std::vector<GpuTileInfo> tiles;
@@ -947,13 +940,13 @@ struct TextureCache::TextureTransferPlan {
}
};
struct TextureCache::DownloadPlan {
TextureTransferPlan texture;
struct TextureCache::ImageDownload {
TextureTransfer texture;
bool depth_target = false;
bool valid = false;
};
TextureCache::TextureTransferPlan
TextureCache::TextureTransfer
TextureCache::BuildTextureTransfer(const Image& image, BindingType binding,
TransferDirection direction) const {
const auto& info = image.info;
@@ -966,8 +959,8 @@ TextureCache::BuildTextureTransfer(const Image& image, BindingType binding,
bool allow_depth_tile = upload;
const char* owner = "TextureCache readback";
TextureTransferPlan plan;
plan.swap_bgra16 = info.bgra16 && (!upload || render_target || video_out);
TextureTransfer transfer;
transfer.swap_bgra16 = info.bgra16 && (!upload || render_target || video_out);
if (render_target) {
format = ImageOps::RenderTargetTransferFormat(info.bytes_per_block);
}
@@ -997,44 +990,44 @@ TextureCache::BuildTextureTransfer(const Image& image, BindingType binding,
}
}
plan.layout = TextureCalcUploadLayout(format, info.extent.width, info.extent.height,
transfer.layout = TextureCalcUploadLayout(format, info.extent.width, info.extent.height,
info.resources.levels, layers, info.tile_mode,
info.data.size, allow_depth_tile, volume, owner);
plan.regions = TextureBuildImageCopies(plan.layout);
transfer.regions = TextureBuildImageCopies(transfer.layout);
if (info.IsDepth()) {
for (auto& region: plan.regions) {
for (auto& region: transfer.regions) {
region.imageSubresource.aspectMask = vk::ImageAspectFlagBits::eDepth;
}
}
if (plan.layout.surface.description.tile_mode != Prospero::TileMode::kLinear) {
if (!TextureBuildGpuTileInfos(info.data.size, plan.regions, plan.layout,
info.resources.levels, plan.tiles)) {
return plan;
if (transfer.layout.surface.description.tile_mode != Prospero::TileMode::kLinear) {
if (!TextureBuildGpuTileInfos(info.data.size, transfer.regions, transfer.layout,
info.resources.levels, transfer.tiles)) {
return transfer;
}
}
plan.valid = true;
return plan;
transfer.valid = true;
return transfer;
}
TextureCache::DownloadPlan TextureCache::BuildDownload(const Image& image) const {
TextureCache::ImageDownload TextureCache::BuildDownload(const Image& image) const {
const auto& info = image.info;
const auto binding = UploadBinding(image);
DownloadPlan plan {.depth_target = binding == BindingType::DepthTarget};
ImageDownload transfer {.depth_target = binding == BindingType::DepthTarget};
if (info.samples != 1 || image.backing.samples != 1) {
return plan;
return transfer;
}
if (plan.depth_target) {
plan.valid = IsSupportedDepthPlaneReadback(info) && info.resources.layers != 0 &&
if (transfer.depth_target) {
transfer.valid = IsSupportedDepthPlaneReadback(info) && info.resources.layers != 0 &&
info.data.size % info.resources.layers == 0 &&
Prospero::NumBytesPerElement(info.guest_format) == info.bytes_per_block;
return plan;
return transfer;
}
if (info.metadata.compression != VideoOutCompression::Uncompressed) {
return plan;
return transfer;
}
plan.texture = BuildTextureTransfer(image, binding, TransferDirection::Download);
plan.valid = plan.texture.valid;
return plan;
transfer.texture = BuildTextureTransfer(image, binding, TransferDirection::Download);
transfer.valid = transfer.texture.valid;
return transfer;
}
void TextureCache::UploadImage(Image& image, Buffer& source, uint64_t source_offset) {
@@ -1048,26 +1041,26 @@ void TextureCache::UploadImage(Image& image, Buffer& source, uint64_t source_off
};
if (binding != BindingType::DepthTarget) {
auto plan = BuildTextureTransfer(image, binding, TransferDirection::Upload);
if (!plan.valid) {
auto transfer = BuildTextureTransfer(image, binding, TransferDirection::Upload);
if (!transfer.valid) {
EXIT("TextureCache: invalid texture upload: binding=%u addr=0x%016" PRIx64
" size=0x%016" PRIx64 " format=%u tile=%u family=%u extent=%ux%ux%u "
"pitch=%u levels=%u layers=%u samples=%u\n",
static_cast<uint32_t>(binding), info.data.address, info.data.size,
static_cast<uint32_t>(info.guest_format), static_cast<uint32_t>(info.tile_mode),
static_cast<uint32_t>(plan.layout.surface.texture.block.family), info.extent.width,
static_cast<uint32_t>(transfer.layout.surface.texture.block.family), info.extent.width,
info.extent.height, info.extent.depth, info.pitch, info.resources.levels,
info.resources.layers, info.samples);
}
TileManager::Result linear {source.Handle(), source_offset, info.data.size};
if (!plan.tiles.empty()) {
if (!transfer.tiles.empty()) {
linear = m_tiler.Detile(source.Handle(), source_offset, info.data.size,
plan.LinearSize(), plan.tiles);
transfer.LinearSize(), transfer.tiles);
}
if (plan.swap_bgra16) {
if (transfer.swap_bgra16) {
linear = m_tiler.SwapBgra16(linear);
}
upload(plan.regions, linear);
upload(transfer.regions, linear);
return;
}
@@ -1142,53 +1135,69 @@ void TextureCache::InitializeImage(ImageId id) {
}
}
void TextureCache::PrepareDccClear(ImageId id, const ImageDesc& desc) {
void TextureCache::MaterializeDccClear(ImageId id, const ImageDesc& desc,
uint32_t metadata_base_layer) {
if (desc.info.metadata.kind != ImageMetadataKind::Dcc) {
return;
}
auto& image = m_slot_images[id];
image.info.metadata = desc.info.metadata;
auto [entry, inserted] = m_surface_metas.try_emplace(
desc.info.metadata.range.address, MetaDataInfo {.type = MetaDataInfo::Type::Dcc});
auto& metadata = entry->second;
if (!inserted && metadata.type == MetaDataInfo::Type::PendingDcc) {
metadata.type = MetaDataInfo::Type::Dcc;
} else if (metadata.type != MetaDataInfo::Type::Dcc) {
EXIT("TextureCache: image reuses non-DCC metadata\n");
const auto range = desc.info.metadata.range;
{
std::scoped_lock lock {m_lock};
auto& image = m_slot_images[id];
image.info.metadata = desc.info.metadata;
// A native DCC allocation must not retain a reused HTile/CMask/FMask interpretation.
m_surface_metas.erase(range.address);
if (range.size == 0 || desc.info.resources.levels != 1 || image.info.resources.levels != 1) {
return;
}
}
if (metadata.clear_mask == 0 || image.info.resources.levels != 1 ||
desc.info.metadata.range.size == 0 || metadata.fill_size < desc.info.metadata.range.size) {
return;
}
vk::ClearValue clear {};
if (!DecodeDccClear(desc, image.backing.format, metadata.fill_value, clear.color)) {
return;
const auto layers = desc.info.TransferLayers();
// These one-mip surfaces use complete 4 KiB DCC metadata blocks.
constexpr uint64_t MetadataBlockSize = 0x1000;
if (!range.Valid() || range.address % MetadataBlockSize != 0 || layers == 0 ||
range.size % layers != 0 || (range.size / layers) % MetadataBlockSize != 0) {
EXIT("TextureCache: DCC slices must contain aligned 4 KiB blocks\n");
}
const auto& view = desc.view_info;
const bool volume_texture = image.info.IsVolume() && view.type == vk::ImageViewType::e3D;
const auto first = volume_texture ? 0u : view.base_layer;
const auto count = volume_texture ? std::max(image.info.extent.depth >> view.base_level, 1u)
: view.layer_count;
if (first >= 32 || count > 32 - first) {
return;
const bool volume_texture = desc.info.IsVolume() && view.type == vk::ImageViewType::e3D;
const auto first = volume_texture ? 0u : metadata_base_layer;
const auto image_first = volume_texture ? 0u : view.base_layer;
const auto count = volume_texture ? desc.info.extent.depth : view.layer_count;
if (first >= layers || count > layers - first) {
EXIT("TextureCache: DCC view exceeds its native metadata slices\n");
}
// The metadata fill covers the complete allocation. Consume each layer only after its
// native image contents exist; already materialized layers may have been rendered since.
for (uint32_t layer = first; layer < first + count;) {
if ((metadata.clear_mask & (1u << layer)) == 0) {
layer++;
// Finish native metadata writes before reading backing bytes. This can submit the scheduler,
// so discovery runs before final draw uploads and never holds the texture lock across it.
if (m_buffer_cache.IsRegionGpuModified(range.address, range.size)) {
m_buffer_cache.ReadMemory(range.address, range.size, false);
}
const auto slice_size = range.size / layers;
for (uint32_t slice = 0; slice < count; slice++) {
const auto address = range.address + slice_size * (first + slice);
uint8_t code = 0;
if (!LibKernel::Memory::TryReadBacking(address, &code, sizeof(code))) {
EXIT("TextureCache: failed to read DCC metadata backing\n");
}
vk::ClearValue clear {};
if (!DecodeDccClear(desc, code, clear.color)) {
continue;
}
const auto start = layer;
uint32_t mask = 0;
do {
mask |= 1u << layer++;
} while (layer < first + count && (metadata.clear_mask & (1u << layer)) != 0);
ClearImage(m_scheduler.Current(), id,
{vk::ImageAspectFlagBits::eColor, view.base_level, view.level_count, start,
layer - start},
clear);
metadata.clear_mask &= ~mask;
std::vector<uint8_t> bytes(slice_size);
if (!LibKernel::Memory::TryReadBacking(address, bytes.data(), bytes.size())) {
EXIT("TextureCache: failed to read DCC metadata slice\n");
}
if (!std::all_of(bytes.begin(), bytes.end(), [code](uint8_t byte) { return byte == code; })) {
continue;
}
{
std::scoped_lock lock {m_lock};
ClearImage(m_scheduler.Current(), id, view.format,
{vk::ImageAspectFlagBits::eColor, view.base_level, view.level_count,
image_first + slice, 1}, clear);
}
// Native expanded keys own consumption. Existing buffer tracking publishes this CPU
// write to future GPU readers; FillBuffer can fault and must run outside the texture lock.
m_buffer_cache.FillBuffer(address, slice_size, UINT32_MAX, false);
}
}
@@ -1228,7 +1237,8 @@ void TextureCache::AssociateStencil(ImageId depth_id, GuestRange stencil) {
ImageId association {};
for (const auto id: FindImagesInRegion(stencil.address, stencil.size, false)) {
const auto owner = m_slot_images.try_get(id);
if (owner != nullptr && owner->info.data.address == stencil.address) {
if (owner != nullptr && owner->info.data == stencil &&
owner->info.extent == depth.info.extent) {
association = id;
}
}
@@ -1253,6 +1263,7 @@ ImageId TextureCache::FindImage(ImageDesc& desc, bool exact_format) {
std::scoped_lock lock {m_lock};
return GetNullImage(desc);
}
const auto metadata_base_layer = desc.view_info.base_layer;
ImageId result {};
{
@@ -1320,6 +1331,7 @@ ImageId TextureCache::FindImage(ImageDesc& desc, bool exact_format) {
image.tick_accessed_last = m_scheduler.CurrentTick();
TouchImage(image);
}
MaterializeDccClear(result, desc, metadata_base_layer);
return result;
}
@@ -1334,8 +1346,8 @@ ImageId TextureCache::FindImageFromRange(uint64_t address, uint64_t size, bool e
if (!GuestRange {address, size}.Valid()) {
return {};
}
std::scoped_lock lock {m_lock};
std::vector<ImageId> matches;
std::scoped_lock lock {m_lock};
ImageIds matches;
for (const auto id: FindImagesInRegion(address, size, false)) {
auto owner = m_slot_images.try_get(id);
if (owner == nullptr || owner->info.data.address != address) {
@@ -1383,16 +1395,12 @@ vk::ImageView TextureCache::FindTexture(ImageId id, const ImageDesc& desc) {
image.MarkGpuModified();
}
if (!image.info.data.Empty()) {
PrepareDccClear(id, desc);
RefreshImage(id);
}
switch (desc.type) {
case BindingType::Texture: break;
case BindingType::Storage:
if (!image.info.data.Empty()) {
if (!image.registered || image.depth_id) {
EXIT("TextureCache: cannot acquire an unavailable storage image\n");
}
CommitGpuWrite(image);
}
TrackImageDownload(id, image);
@@ -1416,7 +1424,6 @@ vk::ImageView TextureCache::FindRenderTarget(ImageId id, const ImageDesc& desc)
TouchImage(image);
image.MarkGpuModified();
image.usage.render_target = true;
PrepareDccClear(id, desc);
RefreshImage(id);
CommitGpuWrite(image);
TrackImageDownload(id, image);
@@ -1440,16 +1447,9 @@ vk::ImageView TextureCache::FindDepthTarget(ImageId id, const ImageDesc& desc) {
RefreshImage(id);
if (desc.info.HasMetadata()) {
image.info.metadata = desc.info.metadata;
auto [metadata, inserted] =
m_surface_metas.try_emplace(desc.info.metadata.range.address,
MetaDataInfo {.type = MetaDataInfo::Type::HTile,
.clear_mask = image.info.htile_clear_mask});
if (!inserted && metadata->second.type != MetaDataInfo::Type::HTile) {
// PS5 allocations can reuse DCC storage as HTile while the old color image is cached.
// The depth binding defines the new type; incompatible fill state cannot carry over.
metadata->second = {.type = MetaDataInfo::Type::HTile,
.clear_mask = image.info.htile_clear_mask};
}
m_surface_metas.emplace(desc.info.metadata.range.address,
MetaDataInfo {.type = MetaDataInfo::Type::HTile,
.clear_mask = image.info.htile_clear_mask});
}
CommitGpuWrite(image);
if (desc.info.HasStencil()) {
@@ -1537,12 +1537,12 @@ bool TextureCache::ClearImageFromBuffer(CommandBuffer& command, uint64_t address
}
clear.depthStencil.stencil = stencil_clear;
}
ClearImage(command, selected,
ClearImage(command, selected, image.backing.format,
{aspect, 0, image.info.resources.levels, 0, image.info.TransferLayers()}, clear);
return true;
}
void TextureCache::ClearImage(CommandBuffer& command, ImageId id,
void TextureCache::ClearImage(CommandBuffer& command, ImageId id, vk::Format format,
const vk::ImageSubresourceRange& range, const vk::ClearValue& clear) {
auto& image = m_slot_images[id];
const auto aspects = image.info.IsDepth() ? ImageViewOps::DepthAspectMask(image.backing.format)
@@ -1567,11 +1567,12 @@ void TextureCache::ClearImage(CommandBuffer& command, ImageId id,
}
}
command.EndRendering();
if (image.info.IsVolume() && !full_image) {
// Transfer clears use the backing format; aliased clears must encode through their view.
if (format != image.backing.format || (image.info.IsVolume() && !full_image)) {
EXIT_NOT_IMPLEMENTED(range.aspectMask != vk::ImageAspectFlagBits::eColor ||
range.levelCount != 1);
ImageViewInfo view {};
view.format = image.backing.format;
view.format = format;
view.type = range.layerCount == 1 ? vk::ImageViewType::e2D : vk::ImageViewType::e2DArray;
view.base_level = range.baseMipLevel;
view.base_layer = range.baseArrayLayer;
@@ -1679,12 +1680,12 @@ void TextureCache::DownloadDepth(Image& image, Buffer& destination, uint64_t des
destination_offset, info.data.size, tiles);
}
void TextureCache::DownloadImageData(Image& image, Buffer& destination, uint64_t destination_offset,
uint64_t destination_size, DownloadPlan plan) {
if (!plan.valid) {
EXIT("TextureCache: invalid image download plan\n");
void TextureCache::DownloadImage(Image& image, Buffer& destination, uint64_t destination_offset,
uint64_t destination_size, ImageDownload transfer) {
if (!transfer.valid) {
EXIT("TextureCache: invalid image download transfer\n");
}
if (plan.depth_target) {
if (transfer.depth_target) {
if (destination_size != image.info.data.size) {
EXIT("TextureCache: partial depth image download is unsupported\n");
}
@@ -1692,7 +1693,7 @@ void TextureCache::DownloadImageData(Image& image, Buffer& destination, uint64_t
return;
}
auto& texture = plan.texture;
auto& texture = transfer.texture;
const auto transform = texture.swap_bgra16 ? TileManager::ColorTransform::SwapBgra16
: TileManager::ColorTransform::None;
if (texture.tiles.empty()) {
@@ -1753,15 +1754,15 @@ bool BufferCache::SynchronizeBufferFromImage(Buffer& buffer, uint64_t vaddr, uin
if (copy_size == 0) {
return false;
}
auto plan = m_texture_cache.BuildDownload(image);
if (!plan.valid) {
auto transfer = m_texture_cache.BuildDownload(image);
if (!transfer.valid) {
return false;
}
if (plan.depth_target && copy_size != image.info.data.size) {
if (transfer.depth_target && copy_size != image.info.data.size) {
return false;
}
if (!plan.depth_target && levels < image.info.resources.levels) {
auto& texture = plan.texture;
if (!transfer.depth_target && levels < image.info.resources.levels) {
auto& texture = transfer.texture;
std::erase_if(texture.regions, [levels](const vk::BufferImageCopy& region) {
return region.imageSubresource.mipLevel >= levels;
});
@@ -1776,17 +1777,17 @@ bool BufferCache::SynchronizeBufferFromImage(Buffer& buffer, uint64_t vaddr, uin
}
}
}
m_texture_cache.DownloadImageData(image, buffer, buf_offset, copy_size, std::move(plan));
m_texture_cache.DownloadImage(image, buffer, buf_offset, copy_size, std::move(transfer));
return true;
}
bool TextureCache::TryDownloadImage(ImageId id) {
bool TextureCache::DownloadImageMemory(ImageId id) {
auto& image = m_slot_images[id];
if (image.depth_id) {
return false;
}
auto plan = BuildDownload(image);
if (!plan.valid || !SafeToDownload(image)) {
auto transfer = BuildDownload(image);
if (!transfer.valid || !SafeToDownload(image)) {
return false;
}
const auto range = image.info.data;
@@ -1802,7 +1803,7 @@ bool TextureCache::TryDownloadImage(ImageId id) {
}
download.Flush(offset, range.size);
DownloadImageData(image, download, offset, range.size, std::move(plan));
DownloadImage(image, download, offset, range.size, std::move(transfer));
vk::BufferMemoryBarrier barrier {};
barrier.srcAccessMask = vk::AccessFlagBits::eMemoryWrite | vk::AccessFlagBits::eTransferWrite |
vk::AccessFlagBits::eShaderWrite;
@@ -1855,8 +1856,8 @@ bool TextureCache::IsRegionGpuModified(uint64_t address, uint64_t size) {
}
void TextureCache::InvalidateCpuAliases(uint64_t address, uint64_t size) {
const auto page_begin = address & ~(TRACKER_PAGE_SIZE - 1);
const auto page_end = (address + size + TRACKER_PAGE_SIZE - 1) & ~(TRACKER_PAGE_SIZE - 1);
const auto page_begin = Common::AlignDown(address, TRACKER_PAGE_SIZE);
const auto page_end = Common::AlignUp(address + size, TRACKER_PAGE_SIZE);
for (const auto id: FindImagesInRegion(address, size, true)) {
auto owner = m_slot_images.try_get(id);
if (owner == nullptr) {
@@ -1882,72 +1883,32 @@ void TextureCache::InvalidateCpuAliases(uint64_t address, uint64_t size) {
bool TextureCache::IsMeta(uint64_t address) {
std::scoped_lock lock {m_lock};
const auto found = m_surface_metas.find(address);
return found != m_surface_metas.end() && found->second.type != MetaDataInfo::Type::PendingDcc;
return found != m_surface_metas.end();
}
bool TextureCache::IsMetaCleared(uint64_t address, uint32_t slice, uint32_t* fill_value) {
bool TextureCache::IsMetaCleared(uint64_t address, uint32_t slice) {
std::scoped_lock lock {m_lock};
const auto found = m_surface_metas.find(address);
if (found == m_surface_metas.end() || found->second.type == MetaDataInfo::Type::PendingDcc ||
slice >= 32) {
if (found == m_surface_metas.end() || slice >= 32) {
return false;
}
if (fill_value != nullptr) {
*fill_value = found->second.fill_value;
}
return (found->second.clear_mask & (1u << slice)) != 0;
}
bool TextureCache::ClearMeta(uint64_t address) {
std::scoped_lock lock {m_lock};
const auto found = m_surface_metas.find(address);
if (found == m_surface_metas.end() || found->second.type == MetaDataInfo::Type::PendingDcc ||
found->second.type == MetaDataInfo::Type::Dcc) {
// Preserve the broad metadata-clear operation for CMask/FMask/HTile. DCC requires a
// validated fill value, so an arbitrary compute write must not clear it.
if (found == m_surface_metas.end()) {
return false;
}
found->second.clear_mask = UINT32_MAX;
return true;
}
void TextureCache::TrackDccFill(uint64_t address, uint64_t size, uint32_t fill_value) {
if (!GuestRange {address, size}.Valid()) {
EXIT("TextureCache: invalid DCC fill range\n");
}
// DCC fills use a repeated byte code. Require all four bytes of the detected dword to agree,
// and mark only recognized deferred-clear encodings as logically clear.
const auto dcc_clear_mask = [fill_value] {
const auto code = static_cast<uint8_t>(fill_value);
if (fill_value != static_cast<uint32_t>(code) * 0x01010101u) {
return 0u;
}
switch (code) {
case 0x00:
case 0x20:
case 0x40:
case 0x80:
case 0xc0: return UINT32_MAX;
default: return 0u;
}
}();
std::scoped_lock lock {m_lock};
// The guest dispatch still writes metadata. An unknown address remains PendingDcc until an
// image descriptor confirms its role; never reinterpret CMask/FMask/HTile as DCC.
const auto found = m_surface_metas.try_emplace(address).first;
if (found->second.type == MetaDataInfo::Type::PendingDcc ||
found->second.type == MetaDataInfo::Type::Dcc) {
found->second.clear_mask = dcc_clear_mask;
found->second.fill_value = fill_value;
found->second.fill_size = size;
}
}
bool TextureCache::TouchMeta(uint64_t address, uint32_t slice, bool is_clear) {
std::scoped_lock lock {m_lock};
const auto found = m_surface_metas.find(address);
if (found == m_surface_metas.end() || found->second.type == MetaDataInfo::Type::PendingDcc ||
slice >= 32) {
if (found == m_surface_metas.end() || slice >= 32) {
return false;
}
if (is_clear) {
@@ -1963,10 +1924,13 @@ void TextureCache::UnmapMemory(uint64_t address, uint64_t size) {
EXIT("TextureCache: invalid unmap range\n");
}
std::scoped_lock lock {m_lock};
const auto end = address + size;
for (auto metadata = m_surface_metas.lower_bound(address);
metadata != m_surface_metas.end() && metadata->first < end;) {
metadata = m_surface_metas.erase(metadata);
for (auto metadata = m_surface_metas.begin(); metadata != m_surface_metas.end();) {
const auto base = metadata->first;
if (base >= address && base < address + size) {
metadata = m_surface_metas.erase(metadata);
} else {
++metadata;
}
}
auto images = FindImagesInRegion(address, size, false);
for (const auto id: images) {
@@ -1974,10 +1938,7 @@ void TextureCache::UnmapMemory(uint64_t address, uint64_t size) {
if (owner == nullptr) {
continue;
}
if (owner->IsGpuModified()) {
owner->ClearGpuModified();
}
DeleteImage(id);
FreeImage(id);
}
}
@@ -2020,12 +1981,11 @@ void TextureCache::RunGarbageCollector() {
if (safe && !pressured) {
continue;
}
if (safe && !TryDownloadImage(id)) {
if (safe && !DownloadImageMemory(id)) {
continue;
}
owner->ClearGpuModified();
}
DeleteImage(id);
FreeImage(id);
if (m_total_used_memory < m_critical_gc_memory && aggressive) {
deletions >>= 2;
aggressive = false;
@@ -2047,7 +2007,7 @@ void TextureCache::ProcessDownloadImages() {
for (const auto id: m_download_images) {
const auto owner = m_slot_images.try_get(id);
if (owner != nullptr && owner->registered && owner->IsGpuModified()) {
(void)TryDownloadImage(id);
(void)DownloadImageMemory(id);
}
}
m_download_images.clear();
+33 -24
View File
@@ -13,6 +13,7 @@
#include "graphics/host_gpu/renderer/image/tiler.h"
#include <map>
#include <type_traits>
#include <unordered_map>
#include <unordered_set>
#include <vector>
@@ -63,11 +64,8 @@ public:
[[nodiscard]] bool IsRegionGpuModified(uint64_t address, uint64_t size);
[[nodiscard]] bool IsMeta(uint64_t address);
[[nodiscard]] bool IsMetaCleared(uint64_t address, uint32_t slice,
uint32_t* fill_value = nullptr);
[[nodiscard]] bool IsMetaCleared(uint64_t address, uint32_t slice);
[[nodiscard]] bool ClearMeta(uint64_t address);
// Record deferred DCC state while the original guest dispatch writes the metadata.
void TrackDccFill(uint64_t address, uint64_t size, uint32_t fill_value);
[[nodiscard]] bool TouchMeta(uint64_t address, uint32_t slice, bool is_clear);
void UnmapMemory(uint64_t address, uint64_t size);
@@ -76,21 +74,14 @@ public:
private:
enum class TransferDirection { Upload, Download };
struct TextureTransferPlan;
struct DownloadPlan;
struct TextureTransfer;
struct ImageDownload;
struct MetaDataInfo {
// A guest metadata-fill dispatch may initialize DCC before its render target is bound.
// PendingDcc retains that exact fill until an image binding classifies the address,
// without exposing an unconfirmed buffer address to the normal metadata heuristics.
// Keep all surface metadata in one entry so CMask/FMask can be
// registered beside HTile and DCC without introducing parallel tracking paths.
enum class Type : uint8_t { PendingDcc, CMask, FMask, HTile, Dcc };
enum class Type : uint8_t { CMask, FMask, HTile };
Type type = Type::PendingDcc;
uint32_t clear_mask = 0;
uint32_t fill_value = 0xffffffffu;
uint64_t fill_size = 0;
Type type;
uint32_t clear_mask = UINT32_MAX;
};
struct OverlapResult {
@@ -102,6 +93,23 @@ private:
using ImageIds = InlinePageOwnerList<ImageId, 16>;
using ImagePageTable = MultiLevelPageTable<ImageIds, 20, 40, 10>;
// Callers have validated the nonempty 40-bit range with TryGetPageRange.
template <typename Func>
static void ForEachPage(uint64_t address, size_t size, Func&& func) {
using FuncReturn = typename std::invoke_result<Func, uint64_t>::type;
static constexpr bool RETURNS_BOOL = std::is_same_v<FuncReturn, bool>;
const uint64_t page_end = (address + size - 1) >> ImagePageTable::kPageBits;
for (uint64_t page = address >> ImagePageTable::kPageBits; page <= page_end; ++page) {
if constexpr (RETURNS_BOOL) {
if (func(page)) {
break;
}
} else {
func(page);
}
}
}
[[nodiscard]] ImageId InsertImage(const ImageInfo& info);
[[nodiscard]] ImageId GetNullImage(const ImageDesc& desc);
void RegisterImage(ImageId id);
@@ -131,19 +139,20 @@ private:
ImageId cached);
[[nodiscard]] ImageId ExpandImage(const ImageInfo& info, ImageId source);
void RefreshImage(ImageId id);
void PrepareDccClear(ImageId id, const ImageDesc& desc);
void MaterializeDccClear(ImageId id, const ImageDesc& desc,
uint32_t metadata_base_layer);
void InitializeImage(ImageId id);
[[nodiscard]] TextureTransferPlan
[[nodiscard]] TextureTransfer
BuildTextureTransfer(const Image& image, BindingType binding, TransferDirection direction) const;
[[nodiscard]] DownloadPlan BuildDownload(const Image& image) const;
[[nodiscard]] ImageDownload BuildDownload(const Image& image) const;
void UploadImage(Image& image, Buffer& source, uint64_t source_offset);
void DownloadImageData(Image& image, Buffer& destination, uint64_t destination_offset,
uint64_t destination_size, DownloadPlan plan);
void DownloadImage(Image& image, Buffer& destination, uint64_t destination_offset,
uint64_t destination_size, ImageDownload transfer);
void DownloadDepth(Image& image, Buffer& destination, uint64_t destination_offset);
void CommitGpuWrite(Image& image);
// Caller holds m_lock. Volume layer ranges select depth slices.
void ClearImage(CommandBuffer& command, ImageId id, const vk::ImageSubresourceRange& range,
const vk::ClearValue& clear);
void ClearImage(CommandBuffer& command, ImageId id, vk::Format format,
const vk::ImageSubresourceRange& range, const vk::ClearValue& clear);
void PrepareImageCopy(Image& image);
void RefreshCopySource(ImageId id);
[[nodiscard]] bool CopyD16(Image& destination, Image& source);
@@ -153,7 +162,7 @@ private:
void ValidateImageDesc(const ImageDesc& desc) const;
void InvalidateCpuAliases(uint64_t address, uint64_t size);
[[nodiscard]] bool TryDownloadImage(ImageId id);
[[nodiscard]] bool DownloadImageMemory(ImageId id);
GraphicContext& m_graphics;
CommandScheduler& m_scheduler;
@@ -41,16 +41,13 @@ static bool DccAlphaOnMsb(const HW::ColorInfo& info) {
}
// NOLINTNEXTLINE(readability-function-cognitive-complexity)
void RenderExecutor::ResolveRenderColorTarget(uint64_t submit_id, CommandBuffer& buffer,
RenderColorInfo& r,
void RenderExecutor::ResolveRenderColorTarget(CommandBuffer& buffer, RenderColorInfo& r,
uint32_t render_target_slice_offset,
uint32_t render_target_slot, bool ignore_target_mask,
uint32_t rt_slot, bool ignore_target_mask,
bool exact_format) {
KYTY_PROFILER_FUNCTION();
const auto& hw = buffer.GetRegisters();
const auto rt_slot = (render_target_slot == UINT32_MAX ? render_target_first_bound_slot(buffer)
: render_target_slot);
const auto& rt = hw.GetRenderTarget(rt_slot);
auto mask = render_target_mask_slot(hw.GetRenderTargetMask(), rt_slot);
if (ignore_target_mask && rt.base.addr != 0 && mask == 0) {
@@ -206,17 +203,14 @@ void RenderExecutor::ResolveRenderColorTarget(uint64_t submit_id, CommandBuffer&
" layer=%u/%u\n",
rt.attrib3.dimension, rt.attrib3.depth, view.base_layer, view.image_layers);
}
if (tile) {
if (volume || texture_tile) {
pitch = TileGetTexturePitch(transfer_format, width, rt.attrib3.tile_mode);
} else {
pitch = TileGetRenderTargetPitch(width, bytes_per_element, rt.attrib.num_fragments);
}
if (pitch == 0) {
EXIT("unsupported render-target pitch: width=%u bytes=%u\n", width, bytes_per_element);
}
// PPSA28068: the linear EULA target and its sampled view must share the padded pitch.
if (!tile || volume || texture_tile) {
pitch = TileGetTexturePitch(transfer_format, width, rt.attrib3.tile_mode);
} else {
pitch = width;
pitch = TileGetRenderTargetPitch(width, bytes_per_element, rt.attrib.num_fragments);
}
if (pitch == 0) {
EXIT("unsupported render-target pitch: width=%u bytes=%u\n", width, bytes_per_element);
}
TileSizeOffset mip_sizes[16] {};
@@ -208,13 +208,7 @@ void CommandScheduler::Wait(uint64_t tick) {
}
void CommandScheduler::PopPendingOperations() {
PopPendingOperations(true);
}
void CommandScheduler::PopPendingOperations(bool refresh_gpu_tick) {
if (refresh_gpu_tick) {
m_master.Refresh();
}
m_master.Refresh();
for (;;) {
PendingOperation operation;
{
@@ -81,7 +81,6 @@ private:
};
void BeginNext();
void PopPendingOperations(bool refresh_gpu_tick);
void PriorityOperationsThread(std::stop_token stop);
void RunOperation(Common::UniqueFunction<void>&& operation);
+2 -2
View File
@@ -61,11 +61,11 @@ void CommandBuffer::SetDebugInfo(uint32_t op, uint64_t submit_id, uint32_t arg0,
}
void CommandBuffer::BeginRendering(const RenderState& state) const {
EXIT_IF(state.width == 0 || state.height == 0 || state.num_layers == 0 ||
state.num_color_attachments > RENDER_COLOR_ATTACHMENTS_MAX);
if (m_rendering && m_render_state == state) {
return;
}
EXIT_IF(state.width == 0 || state.height == 0 || state.num_layers == 0 ||
state.num_color_attachments > RENDER_COLOR_ATTACHMENTS_MAX);
EndRendering();
std::array<vk::RenderingAttachmentInfo, RENDER_COLOR_ATTACHMENTS_MAX> colors {};
+69 -124
View File
@@ -4,7 +4,6 @@
#include "common/common.h"
#include "common/emulatorConfig.h"
#include "common/logging/log.h"
#include "common/stringUtils.h"
#include "graphics/guest_gpu/gpu_defs.h"
#include "graphics/guest_gpu/hardwareContext.h"
#include "graphics/host_gpu/renderer/render.h"
@@ -57,7 +56,7 @@ uint32_t render_target_first_bound_slot(const CommandBuffer& buffer) {
bool graphics_debug_dump_enabled() {
return Config::GraphicsDebugDumpEnabled() &&
Config::GetPrintfDirection() != Config::OutputDirection::Silent;
Config::GetPrintfDirection() != Config::LogDirection::Silent;
}
void uc_print(const char* func, const HW::UserConfig& uc) {
@@ -89,91 +88,75 @@ void uc_check(const HW::UserConfig& uc) {
EXIT_NOT_IMPLEMENTED(user_en.vgpr3 != false);
}
void sh_print(const char* func, const HW::Shader& /*uc*/) {
LOGF("%s\n", func);
}
std::string rt_print(const char* func, const HW::RenderTarget& rt) {
std::string dst;
dst.reserve(4096);
std::vector<std::string> rt_print(const char* func, const HW::RenderTarget& rt) {
std::vector<std::string> dst;
dst.reserve(40);
dst += fmt::format("{}\n", func);
dst.push_back(fmt::format("{}\n", func));
dst += fmt::format("\t base.addr = 0x{:016x}\n", rt.base.addr);
dst += fmt::format("\t view.base_array_slice_index = 0x{:08x}\n",
rt.view.base_array_slice_index);
dst += fmt::format("\t view.last_array_slice_index = 0x{:08x}\n",
rt.view.last_array_slice_index);
dst += fmt::format("\t view.current_mip_level = 0x{:08x}\n", rt.view.current_mip_level);
dst += fmt::format("\t info.fmask_compression_enable = {}\n",
rt.info.fmask_compression_enable ? "true" : "false");
dst.push_back(fmt::format("\t base.addr = 0x{:016x}\n", rt.base.addr));
dst.push_back(fmt::format("\t view.base_array_slice_index = 0x{:08x}\n",
rt.view.base_array_slice_index));
dst.push_back(fmt::format("\t view.last_array_slice_index = 0x{:08x}\n",
rt.view.last_array_slice_index));
dst.push_back(
fmt::format("\t view.current_mip_level = 0x{:08x}\n", rt.view.current_mip_level));
dst.push_back(fmt::format("\t info.fmask_compression_enable = {}\n",
rt.info.fmask_compression_enable ? "true" : "false"));
dst += fmt::format("\t info.fmask_data_compression_disable = {}\n",
rt.info.fmask_data_compression_disable ? "true" : "false");
dst += fmt::format("\t info.fmask_one_frag_mode = {}\n",
rt.info.fmask_one_frag_mode ? "true" : "false");
// dst.push_back(fmt::format("\t info.fmask_compression_mode = 0x{:08x}\n", //
// rt.info.fmask_compression_mode));
dst.push_back(fmt::format("\t info.fmask_data_compression_disable = {}\n",
rt.info.fmask_data_compression_disable ? "true" : "false"));
dst.push_back(fmt::format("\t info.fmask_one_frag_mode = {}\n",
rt.info.fmask_one_frag_mode ? "true" : "false"));
dst.push_back(fmt::format("\t info.cmask_fast_clear_enable = {}\n",
rt.info.cmask_fast_clear_enable ? "true" : "false"));
dst.push_back(fmt::format("\t info.dcc_compression_enable = {}\n",
rt.info.dcc_compression_enable ? "true" : "false"));
dst.push_back(fmt::format("\t info.format = 0x{:08x}\n",
static_cast<uint32_t>(rt.info.format)));
dst.push_back(fmt::format("\t info.channel_type = 0x{:08x}\n",
static_cast<uint32_t>(rt.info.channel_type)));
dst.push_back(fmt::format("\t info.channel_order = 0x{:08x}\n",
static_cast<uint32_t>(rt.info.channel_order)));
dst.push_back(fmt::format("\t info.blend_bypa = {}\n",
rt.info.blend_bypass ? "true" : "false"));
dst.push_back(fmt::format("\t info.blend_clamp = {}\n",
rt.info.blend_clamp ? "true" : "false"));
dst.push_back(fmt::format("\t info.round_mode = {}\n",
rt.info.round_mode ? "true" : "false"));
dst.push_back(fmt::format("\t attrib.force_dest_alpha_to_one = {}\n",
rt.attrib.force_dest_alpha_to_one ? "true" : "false"));
dst.push_back(
fmt::format("\t attrib.num_samples = 0x{:08x}\n", rt.attrib.num_samples));
dst.push_back(
fmt::format("\t attrib.num_fragments = 0x{:08x}\n", rt.attrib.num_fragments));
dst.push_back(fmt::format("\t attrib2.width = 0x{:08x}\n", rt.attrib2.width));
dst.push_back(
fmt::format("\t attrib2.height = 0x{:08x}\n", rt.attrib2.height));
dst.push_back(
fmt::format("\t attrib2.num_mip_levels = 0x{:08x}\n", rt.attrib2.num_mip_levels));
dst.push_back(fmt::format("\t attrib3.depth = 0x{:08x}\n", rt.attrib3.depth));
dst.push_back(fmt::format("\t attrib3.tile_mode = 0x{:08x}\n",
static_cast<uint32_t>(rt.attrib3.tile_mode)));
dst.push_back(
fmt::format("\t attrib3.dimension = 0x{:08x}\n", rt.attrib3.dimension));
dst.push_back(fmt::format("\t attrib3.metadata_pipe_aligned = {}\n",
rt.attrib3.metadata_pipe_aligned ? "true" : "false"));
dst.push_back(fmt::format("\t attrib3.write_vrs_rate_hint_to_cmask = {}\n",
rt.attrib3.write_vrs_rate_hint_to_cmask ? "true" : "false"));
dst.push_back(fmt::format("\t dcc.max_uncompressed_block_size = 0x{:08x}\n",
rt.dcc.max_uncompressed_block_size));
dst.push_back(fmt::format("\t dcc.max_compressed_block_size = 0x{:08x}\n",
rt.dcc.max_compressed_block_size));
dst.push_back(
fmt::format("\t dcc.color_transform = 0x{:08x}\n", rt.dcc.color_transform));
dst.push_back(fmt::format("\t dcc.overwrite_combiner_disable = {}\n",
rt.dcc.overwrite_combiner_disable ? "true" : "false"));
dst.push_back(fmt::format("\t dcc.independent_block_size = 0x{:02x}\n",
static_cast<uint8_t>(rt.dcc.independent_block_size)));
dst.push_back(fmt::format("\t data_write_on_dcc_clear_to_reg = {}\n",
rt.dcc.data_write_on_dcc_clear_to_reg ? "true" : "false"));
dst.push_back(fmt::format("\t dcc.dcc_clear_key_enable = {}\n",
rt.dcc.dcc_clear_key_enable ? "true" : "false"));
dst.push_back(fmt::format("\t cmask.addr = 0x{:016x}\n", rt.cmask.addr));
dst.push_back(fmt::format("\t fmask.addr = 0x{:016x}\n", rt.fmask.addr));
dst.push_back(
fmt::format("\t clear_word0.word0 = 0x{:08x}\n", rt.clear_word0.word0));
dst.push_back(
fmt::format("\t clear_word1.word1 = 0x{:08x}\n", rt.clear_word1.word1));
dst.push_back(
fmt::format("\t dcc_addr.addr = 0x{:016x}\n", rt.dcc_addr.addr));
dst += fmt::format("\t info.cmask_fast_clear_enable = {}\n",
rt.info.cmask_fast_clear_enable ? "true" : "false");
dst += fmt::format("\t info.dcc_compression_enable = {}\n",
rt.info.dcc_compression_enable ? "true" : "false");
dst += fmt::format("\t info.format = 0x{:08x}\n",
static_cast<uint32_t>(rt.info.format));
dst += fmt::format("\t info.channel_type = 0x{:08x}\n",
static_cast<uint32_t>(rt.info.channel_type));
dst += fmt::format("\t info.channel_order = 0x{:08x}\n",
static_cast<uint32_t>(rt.info.channel_order));
dst += fmt::format("\t info.blend_bypa = {}\n",
rt.info.blend_bypass ? "true" : "false");
dst += fmt::format("\t info.blend_clamp = {}\n",
rt.info.blend_clamp ? "true" : "false");
dst += fmt::format("\t info.round_mode = {}\n",
rt.info.round_mode ? "true" : "false");
dst += fmt::format("\t attrib.force_dest_alpha_to_one = {}\n",
rt.attrib.force_dest_alpha_to_one ? "true" : "false");
dst += fmt::format("\t attrib.num_samples = 0x{:08x}\n", rt.attrib.num_samples);
dst += fmt::format("\t attrib.num_fragments = 0x{:08x}\n", rt.attrib.num_fragments);
dst += fmt::format("\t attrib2.width = 0x{:08x}\n", rt.attrib2.width);
dst += fmt::format("\t attrib2.height = 0x{:08x}\n", rt.attrib2.height);
dst += fmt::format("\t attrib2.num_mip_levels = 0x{:08x}\n", rt.attrib2.num_mip_levels);
dst += fmt::format("\t attrib3.depth = 0x{:08x}\n", rt.attrib3.depth);
dst += fmt::format("\t attrib3.tile_mode = 0x{:08x}\n",
static_cast<uint32_t>(rt.attrib3.tile_mode));
dst += fmt::format("\t attrib3.dimension = 0x{:08x}\n", rt.attrib3.dimension);
dst += fmt::format("\t attrib3.metadata_pipe_aligned = {}\n",
rt.attrib3.metadata_pipe_aligned ? "true" : "false");
dst += fmt::format("\t attrib3.write_vrs_rate_hint_to_cmask = {}\n",
rt.attrib3.write_vrs_rate_hint_to_cmask ? "true" : "false");
dst += fmt::format("\t dcc.max_uncompressed_block_size = 0x{:08x}\n",
rt.dcc.max_uncompressed_block_size);
dst += fmt::format("\t dcc.max_compressed_block_size = 0x{:08x}\n",
rt.dcc.max_compressed_block_size);
dst += fmt::format("\t dcc.color_transform = 0x{:08x}\n", rt.dcc.color_transform);
dst += fmt::format("\t dcc.overwrite_combiner_disable = {}\n",
rt.dcc.overwrite_combiner_disable ? "true" : "false");
dst += fmt::format("\t dcc.independent_block_size = 0x{:02x}\n",
static_cast<uint8_t>(rt.dcc.independent_block_size));
dst += fmt::format("\t data_write_on_dcc_clear_to_reg = {}\n",
rt.dcc.data_write_on_dcc_clear_to_reg ? "true" : "false");
dst += fmt::format("\t dcc.dcc_clear_key_enable = {}\n",
rt.dcc.dcc_clear_key_enable ? "true" : "false");
dst += fmt::format("\t cmask.addr = 0x{:016x}\n", rt.cmask.addr);
dst += fmt::format("\t fmask.addr = 0x{:016x}\n", rt.fmask.addr);
dst += fmt::format("\t clear_word0.word0 = 0x{:08x}\n", rt.clear_word0.word0);
dst += fmt::format("\t clear_word1.word1 = 0x{:08x}\n", rt.clear_word1.word1);
dst += fmt::format("\t dcc_addr.addr = 0x{:016x}\n", rt.dcc_addr.addr);
return dst;
}
@@ -478,7 +461,6 @@ static void McCheck(const HW::ModeControl& c) {
logged = true;
}
}
EXIT_NOT_IMPLEMENTED(c.provoking_vtx_last != false);
EXIT_NOT_IMPLEMENTED(c.persp_corr_dis != false);
}
@@ -505,16 +487,7 @@ static void BcPrint(const char* func, const HW::BlendControl& c, const HW::Blend
color.red, color.green, color.blue, color.alpha, cc.mode, cc.op);
}
static void BcCheck(const HW::BlendControl& /*c*/, const HW::BlendColor& color,
const HW::ColorControl& cc) {
// EXIT_NOT_IMPLEMENTED(c.color_srcblend != 0);
// EXIT_NOT_IMPLEMENTED(c.color_comb_fcn != 0);
// EXIT_NOT_IMPLEMENTED(c.color_destblend != 0);
// EXIT_NOT_IMPLEMENTED(c.alpha_srcblend != 0);
// EXIT_NOT_IMPLEMENTED(c.alpha_comb_fcn != 0);
// EXIT_NOT_IMPLEMENTED(c.alpha_destblend != 0);
// EXIT_NOT_IMPLEMENTED(c.separate_alpha_blend != false);
// EXIT_NOT_IMPLEMENTED(c.enable != false);
static void BcCheck(const HW::BlendColor& color, const HW::ColorControl& cc) {
if (color.red != 0.0f || color.green != 0.0f || color.blue != 0.0f || color.alpha != 0.0f) {
static bool logged = false;
if (!logged) {
@@ -577,29 +550,6 @@ static void DPrint(const char* func, const HW::DepthControl& c, const HW::Stenci
sm.stencil_opval_bf);
}
static void DCheck(const HW::DepthControl& c, const HW::StencilControl& s,
const HW::StencilMask& /*sm*/) {
// EXIT_NOT_IMPLEMENTED(c.stencil_enable != false);
// EXIT_NOT_IMPLEMENTED(c.z_enable != false);
// EXIT_NOT_IMPLEMENTED(c.z_write_enable != false);
// EXIT_NOT_IMPLEMENTED(c.zfunc != 0);
// Back-face stencil state is handled separately when enabled.
// EXIT_NOT_IMPLEMENTED(c.stencilfunc != 0);
// EXIT_NOT_IMPLEMENTED(c.stencilfunc_bf != 0);
// EXIT_NOT_IMPLEMENTED(s.stencil_fail != 0);
// EXIT_NOT_IMPLEMENTED(s.stencil_zpass != 0);
// EXIT_NOT_IMPLEMENTED(s.stencil_zfail != 0);
// Back-face stencil ops may legitimately differ from the front-face ops.
// EXIT_NOT_IMPLEMENTED(sm.stencil_testval != 0);
// EXIT_NOT_IMPLEMENTED(sm.stencil_mask != 0);
// EXIT_NOT_IMPLEMENTED(sm.stencil_writemask != 0);
// EXIT_NOT_IMPLEMENTED(sm.stencil_opval != 0);
// EXIT_NOT_IMPLEMENTED(sm.stencil_testval_bf != 0);
// EXIT_NOT_IMPLEMENTED(sm.stencil_mask_bf != 0);
// EXIT_NOT_IMPLEMENTED(sm.stencil_writemask_bf != 0);
// EXIT_NOT_IMPLEMENTED(sm.stencil_opval_bf != 0);
}
static void EqaaPrint(const char* func, const HW::EqaaControl& c) {
LOGF("%s\n", func);
@@ -905,14 +855,10 @@ void hw_check(const CommandBuffer& buffer) {
const auto& hw = buffer.GetRegisters();
const auto rt_slot = render_target_first_bound_slot(buffer);
const auto& rt = hw.GetRenderTarget(rt_slot);
const auto& bc = hw.GetBlendControl(rt_slot);
const auto& bclr = hw.GetBlendColor();
const auto& vp = hw.GetScreenViewport();
const auto& c = hw.GetClipControl();
const auto& rc = hw.GetRenderControl();
const auto& d = hw.GetDepthControl();
const auto& s = hw.GetStencilControl();
const auto& sm = hw.GetStencilMask();
const auto& mc = hw.GetModeControl();
const auto& eqaa = hw.GetEqaaControl();
const auto& cc = hw.GetColorControl();
@@ -938,11 +884,10 @@ void hw_check(const CommandBuffer& buffer) {
log_phase("rc");
RcCheck(rc);
log_phase("depth");
DCheck(d, s, sm);
log_phase("mode");
McCheck(mc);
log_phase("blend");
BcCheck(bc, bclr, cc);
BcCheck(bclr, cc);
log_phase("eqaa");
EqaaCheck(eqaa, ac);
log_phase("aa");
@@ -1001,7 +946,7 @@ void hw_print(const CommandBuffer& buffer) {
hw.GetRenderTargetMask(), hw.GetDepthClearValue(), hw.GetStencilClearValue(),
hw.GetLineWidth(), hw.GetPrimitiveResetIndex());
LOGF("%s", Common::Concat(rt_print("RenderTraget:", rt), "").c_str());
LOGF("%s", rt_print("RenderTraget:", rt).c_str());
ZPrint("DepthRenderTraget:", z);
VpPrint("ScreenViewport:", vp, smc);
+1 -4
View File
@@ -5,7 +5,6 @@
#include <cstdint>
#include <string>
#include <vector>
namespace Libs::Graphics {
@@ -13,7 +12,6 @@ class CommandBuffer;
namespace HW {
class Context;
class Shader;
class UserConfig;
struct RenderTarget;
struct ScanModeControl;
@@ -32,8 +30,7 @@ uint32_t render_target_first_bound_slot(const CommandBuffer& buf
bool graphics_debug_dump_enabled();
void uc_print(const char* func, const HW::UserConfig& uc);
void uc_check(const HW::UserConfig& uc);
void sh_print(const char* func, const HW::Shader& uc);
std::vector<std::string> rt_print(const char* func, const HW::RenderTarget& rt);
std::string rt_print(const char* func, const HW::RenderTarget& rt);
bool RenderIsColorTileModeLinear(Prospero::TileMode tile_mode);
void hw_print(const CommandBuffer& buffer);
void hw_check(const CommandBuffer& buffer);
@@ -10,6 +10,7 @@
#include "graphics/guest_gpu/hardwareContext.h"
#include "graphics/guest_gpu/tile.h"
#include "graphics/host_gpu/graphicContext.h"
#include "graphics/host_gpu/renderer/commandScheduler.h"
#include "graphics/host_gpu/renderer/debug.h"
#include "graphics/host_gpu/renderer/image/imageView.h"
#include "graphics/host_gpu/renderer/image/textureCommon.h"
@@ -18,6 +19,7 @@
#include "graphics/host_gpu/vulkanCommon.h"
#include <algorithm>
#include <array>
#include <atomic>
#include <cmath>
#include <cstdarg>
@@ -44,8 +46,12 @@ static vk::StencilOp ConvertStencilOp(uint8_t value, uint8_t write_mask, uint8_t
switch (static_cast<Prospero::StencilOp>(value)) {
case Prospero::StencilOp::kKeep: return vk::StencilOp::eKeep;
case Prospero::StencilOp::kZero: return vk::StencilOp::eZero;
case Prospero::StencilOp::kReplaceTest:
case Prospero::StencilOp::kReplaceOp: return vk::StencilOp::eReplace;
case Prospero::StencilOp::kReplaceTest: return vk::StencilOp::eReplace;
case Prospero::StencilOp::kReplaceOp:
if ((op_value & write_mask) == 0) {
return vk::StencilOp::eZero;
}
return vk::StencilOp::eReplace;
case Prospero::StencilOp::kAddClamp: return vk::StencilOp::eIncrementAndClamp;
case Prospero::StencilOp::kSubClamp: return vk::StencilOp::eDecrementAndClamp;
case Prospero::StencilOp::kInvert: return vk::StencilOp::eInvert;
@@ -65,9 +71,38 @@ static vk::StencilOp ConvertStencilOp(uint8_t value, uint8_t write_mask, uint8_t
}
}
static bool UsesStencilOpValue(uint8_t fail, uint8_t pass, uint8_t depth_fail) {
constexpr auto replace_op = static_cast<uint8_t>(Prospero::StencilOp::kReplaceOp);
return fail == replace_op || pass == replace_op || depth_fail == replace_op;
static vk::StencilOpState ConvertStencilState(
uint8_t compare, const std::array<uint8_t, 3>& operations, uint8_t op_value,
const vk::StencilOpState& state) {
const auto test_value = state.reference;
auto reference = test_value;
auto required_bits = state.compareMask;
if (compare == static_cast<uint8_t>(vk::CompareOp::eAlways) ||
compare == static_cast<uint8_t>(vk::CompareOp::eNever)) {
required_bits = 0;
}
std::array<vk::StencilOp, 3> converted {};
for (size_t i = 0; i < operations.size(); i++) {
converted[i] = ConvertStencilOp(operations[i], state.writeMask, op_value);
if (converted[i] != vk::StencilOp::eReplace) {
continue;
}
auto replacement = test_value;
if (static_cast<Prospero::StencilOp>(operations[i]) == Prospero::StencilOp::kReplaceOp) {
replacement = op_value;
}
if (((reference ^ replacement) & required_bits & state.writeMask) != 0) {
DepthFatal("unsupported stencil replacement: compare=%u, compare mask=0x%02" PRIx32
", write mask=0x%02" PRIx32 ", operation value=0x%02" PRIx8
", test value=0x%02" PRIx32,
compare, state.compareMask, state.writeMask, op_value, test_value);
}
// Vulkan shares one reference between comparison and every replacement on this face.
reference = (reference & ~state.writeMask) | (replacement & state.writeMask);
required_bits |= state.writeMask;
}
return {converted[0], converted[1], converted[2], static_cast<vk::CompareOp>(compare),
state.compareMask, state.writeMask, reference};
}
[[nodiscard]] static vk::Format ResolveHostDepthAttachmentFormat(const CommandBuffer& buffer,
@@ -96,53 +131,17 @@ static bool UsesStencilOpValue(uint8_t fail, uint8_t pass, uint8_t depth_fail) {
return vk::Format::eUndefined;
}
// NOLINTNEXTLINE(readability-function-cognitive-complexity)
void RenderExecutor::ResolveRenderDepthTarget(uint64_t submit_id, CommandBuffer& buffer,
RenderDepthInfo& r) {
KYTY_PROFILER_FUNCTION();
(void)submit_id;
const auto& hw = buffer.GetRegisters();
const auto& z = hw.GetDepthRenderTarget();
const auto& rc = hw.GetRenderControl();
const auto& dc = hw.GetDepthControl();
const auto& sc = hw.GetStencilControl();
const auto& sm = hw.GetStencilMask();
const bool has_stencil = z.stencil_info.format != Prospero::StencilFormat::kInvalid;
const bool depth_active = dc.z_enable || dc.depth_bounds_enable ||
rc.depth_clear_enable || rc.copy_depth_to_color;
const bool stencil_active =
has_stencil && (dc.stencil_enable || rc.stencil_clear_enable || rc.copy_stencil_to_color);
if (!depth_active && !stencil_active) {
return;
}
static TextureCache::ImageDesc MakeDepthTargetDesc(const CommandBuffer& buffer,
const HW::DepthRenderTarget& z,
bool write_buffer = false) {
const bool has_stencil = z.stencil_info.format != Prospero::StencilFormat::kInvalid;
const auto depth_address = write_buffer ? z.z_write_base_addr : z.z_read_base_addr;
const auto stencil_address =
write_buffer ? z.stencil_write_base_addr : z.stencil_read_base_addr;
if (!z.z_info.HasValidTextureCompatibility() ||
!z.stencil_info.HasValidTextureCompatibility()) {
DepthFatal("invalid PS5 depth texture-compatibility encoding");
}
const bool attachment_unbound =
z.z_info.format == Prospero::DepthFormat::kInvalid &&
z.stencil_info.format == Prospero::StencilFormat::kInvalid && z.z_info.num_samples == 0 &&
z.z_info.texture_compatibility == Prospero::TextureCompatiblePlaneCompression::kDisable &&
!z.z_info.expclear_enabled && !z.z_info.partially_resident && z.z_info.max_mip_level == 0 &&
z.stencil_info.texture_compatibility == Prospero::TextureCompatibleStencil::kDisable &&
!z.stencil_info.expclear_enabled && !z.stencil_info.partially_resident &&
z.depth_view.slice_start == 0 && z.depth_view.slice_max == 0 &&
z.depth_view.current_mip_level == 0 && !z.depth_view.depth_write_disable &&
!z.depth_view.stencil_write_disable && z.z_read_base_addr == 0 && z.z_write_base_addr == 0 &&
z.stencil_read_base_addr == 0 && z.stencil_write_base_addr == 0 &&
z.htile_data_base_addr == 0 &&
// DB_DEPTH_SIZE_XY is independent state and may remain programmed after the attachment
// formats and addresses are unbound. A zero encoding is the valid 1x1 value, so its
// presence alone must not manufacture a depth attachment.
!z.z_info.htile_acceleration && z.shading_rate_encoding == 0 && z.size.x_max == 0 &&
z.size.y_max == 0;
if (attachment_unbound) {
static std::atomic_bool logged = false;
if (!logged.exchange(true, std::memory_order_relaxed)) {
LOGF("DepthTarget: ignoring enabled depth/stencil state without a bound attachment\n");
}
return;
}
const bool has_htile = z.z_info.htile_acceleration;
const bool unsupported_shading_rate_encoding =
z.shading_rate_encoding > 1 || (z.shading_rate_encoding != 0 && !has_htile);
@@ -160,23 +159,16 @@ void RenderExecutor::ResolveRenderDepthTarget(uint64_t submit_id, CommandBuffer&
DepthFatal("invalid depth view: base=%u last=%u", z.depth_view.slice_start,
z.depth_view.slice_max);
}
if (rc.copy_depth_to_color || rc.copy_stencil_to_color || rc.copy_centroid ||
rc.copy_sample != 0 || z.z_info.expclear_enabled ||
z.stencil_info.expclear_enabled || z.z_info.partially_resident ||
// EXPCLEAR permits an HTile acceleration state; the host attachment is already expanded.
if (z.z_info.partially_resident ||
z.stencil_info.partially_resident || z.z_info.max_mip_level != 0 ||
z.depth_view.current_mip_level != 0 || unsupported_shading_rate_encoding ||
z.z_read_base_addr == 0 ||
(!z.depth_view.depth_write_disable && z.z_write_base_addr != z.z_read_base_addr) ||
(z.z_read_base_addr & 0xffffu) != 0 ||
dc.zfunc > static_cast<uint8_t>(vk::CompareOp::eAlways)) {
depth_address == 0 || (depth_address & 0xffffu) != 0) {
DepthFatal("unsupported depth register state");
}
if (has_stencil) {
if (z.stencil_info.format != Prospero::StencilFormat::k8UInt || !htile_stencil_compat ||
z.stencil_read_base_addr == 0 ||
(!z.depth_view.stencil_write_disable &&
z.stencil_write_base_addr != z.stencil_read_base_addr) ||
(z.stencil_read_base_addr & 0xffffu) != 0) {
stencil_address == 0 || (stencil_address & 0xffffu) != 0) {
DepthFatal("unsupported stencil attachment state");
}
} else if (z.stencil_read_base_addr != 0 || z.stencil_write_base_addr != 0) {
@@ -229,67 +221,16 @@ void RenderExecutor::ResolveRenderDepthTarget(uint64_t submit_id, CommandBuffer&
const auto depth_backing_size = depth_size.size * view.image_layers;
const auto stencil_backing_size = stencil_size.size * view.image_layers;
const auto htile_backing_size = htile_size.size * view.image_layers;
if (!GuestRange {z.z_read_base_addr, depth_backing_size}.Valid() ||
(has_stencil && !GuestRange {z.stencil_read_base_addr, stencil_backing_size}.Valid()) ||
if (!GuestRange {depth_address, depth_backing_size}.Valid() ||
(has_stencil && !GuestRange {stencil_address, stencil_backing_size}.Valid()) ||
(has_htile && !GuestRange {z.htile_data_base_addr, htile_backing_size}.Valid())) {
DepthFatal("layered depth backing range is invalid");
}
r.depth_clear_enable = rc.depth_clear_enable;
r.depth_meta_clear_enable = false;
r.depth_load_clear_enable = r.depth_clear_enable;
r.depth_clear_value = hw.GetDepthClearValue();
r.depth_test_enable = dc.z_enable;
r.depth_write_enable = r.depth_test_enable && dc.z_write_enable &&
!z.depth_view.depth_write_disable && !r.depth_clear_enable;
r.depth_compare_op = static_cast<vk::CompareOp>(dc.zfunc);
r.depth_bounds_test_enable = dc.depth_bounds_enable;
r.depth_min_bounds = hw.GetDepthBoundsMin();
r.depth_max_bounds = hw.GetDepthBoundsMax();
r.stencil_clear_enable =
has_stencil && rc.stencil_clear_enable && !z.depth_view.stencil_write_disable;
r.stencil_clear_value = hw.GetStencilClearValue();
r.stencil_test_enable = has_stencil && dc.stencil_enable;
if (r.stencil_test_enable) {
const bool stencil_ops_disabled =
rc.stencil_clear_enable || z.depth_view.stencil_write_disable;
const uint8_t front_write_mask = stencil_ops_disabled ? 0 : sm.stencil_writemask;
const uint8_t back_write_mask = stencil_ops_disabled ? 0 : sm.stencil_writemask_bf;
if (dc.stencilfunc > static_cast<uint8_t>(vk::CompareOp::eAlways) ||
(dc.backface_enable &&
dc.stencilfunc_bf > static_cast<uint8_t>(vk::CompareOp::eAlways)) ||
(front_write_mask != 0 &&
UsesStencilOpValue(sc.stencil_fail, sc.stencil_zpass, sc.stencil_zfail) &&
sm.stencil_opval != sm.stencil_testval) ||
(dc.backface_enable && back_write_mask != 0 &&
UsesStencilOpValue(sc.stencil_fail_bf, sc.stencil_zpass_bf, sc.stencil_zfail_bf) &&
sm.stencil_opval_bf != sm.stencil_testval_bf)) {
DepthFatal("unsupported stencil compare or replacement state");
}
r.stencil_static_front = {
ConvertStencilOp(sc.stencil_fail, front_write_mask, sm.stencil_opval),
ConvertStencilOp(sc.stencil_zpass, front_write_mask, sm.stencil_opval),
ConvertStencilOp(sc.stencil_zfail, front_write_mask, sm.stencil_opval),
static_cast<vk::CompareOp>(dc.stencilfunc)};
r.stencil_dynamic_front = {sm.stencil_mask, front_write_mask, sm.stencil_testval};
if (dc.backface_enable) {
r.stencil_static_back = {
ConvertStencilOp(sc.stencil_fail_bf, back_write_mask, sm.stencil_opval_bf),
ConvertStencilOp(sc.stencil_zpass_bf, back_write_mask, sm.stencil_opval_bf),
ConvertStencilOp(sc.stencil_zfail_bf, back_write_mask, sm.stencil_opval_bf),
static_cast<vk::CompareOp>(dc.stencilfunc_bf)};
r.stencil_dynamic_back = {sm.stencil_mask_bf, back_write_mask, sm.stencil_testval_bf};
} else {
r.stencil_static_back = r.stencil_static_front;
r.stencil_dynamic_back = r.stencil_dynamic_front;
}
}
TextureCache::ImageDesc desc {};
desc.type = TextureCache::BindingType::DepthTarget;
desc.info.data = {z.z_read_base_addr, depth_backing_size};
desc.info.data = {depth_address, depth_backing_size};
desc.info.stencil =
has_stencil ? GuestRange {z.stencil_read_base_addr, stencil_backing_size} : GuestRange {};
has_stencil ? GuestRange {stencil_address, stencil_backing_size} : GuestRange {};
desc.info.pixel_format = format;
desc.info.guest_format = guest_format;
desc.info.type = Prospero::ImageType::kColor2D;
@@ -314,12 +255,164 @@ void RenderExecutor::ResolveRenderDepthTarget(uint64_t submit_id, CommandBuffer&
desc.view_info.base_layer = view.base_layer;
desc.view_info.layer_count = view.layer_count;
desc.view_info.usage = vk::ImageUsageFlagBits::eDepthStencilAttachment;
r.desc = std::move(desc);
auto& cache = m_context.GetTextureCache();
r.image_id = cache.FindImage(r.desc);
return desc;
}
// NOLINTNEXTLINE(readability-function-cognitive-complexity)
void RenderExecutor::ResolveRenderDepthTarget(CommandBuffer& buffer, RenderDepthInfo& r) {
KYTY_PROFILER_FUNCTION();
const auto& hw = buffer.GetRegisters();
const auto& z = hw.GetDepthRenderTarget();
const auto& rc = hw.GetRenderControl();
const auto& dc = hw.GetDepthControl();
const auto& sc = hw.GetStencilControl();
const auto& sm = hw.GetStencilMask();
const bool has_stencil = z.stencil_info.format != Prospero::StencilFormat::kInvalid;
const bool depth_active = dc.z_enable || dc.depth_bounds_enable ||
rc.depth_clear_enable || rc.copy_depth_to_color;
const bool stencil_active =
has_stencil && (dc.stencil_enable || rc.stencil_clear_enable || rc.copy_stencil_to_color);
if (!depth_active && !stencil_active) {
return;
}
const bool attachment_unbound =
z.z_info.format == Prospero::DepthFormat::kInvalid &&
z.stencil_info.format == Prospero::StencilFormat::kInvalid && z.z_info.num_samples == 0 &&
z.z_info.texture_compatibility == Prospero::TextureCompatiblePlaneCompression::kDisable &&
!z.z_info.expclear_enabled && !z.z_info.partially_resident && z.z_info.max_mip_level == 0 &&
z.stencil_info.texture_compatibility == Prospero::TextureCompatibleStencil::kDisable &&
!z.stencil_info.expclear_enabled && !z.stencil_info.partially_resident &&
z.depth_view.slice_start == 0 && z.depth_view.slice_max == 0 &&
z.depth_view.current_mip_level == 0 && !z.depth_view.depth_write_disable &&
!z.depth_view.stencil_write_disable && z.z_read_base_addr == 0 && z.z_write_base_addr == 0 &&
z.stencil_read_base_addr == 0 && z.stencil_write_base_addr == 0 &&
z.htile_data_base_addr == 0 &&
// DB_DEPTH_SIZE_XY is independent state and may remain programmed after the attachment
// formats and addresses are unbound. A zero encoding is the valid 1x1 value, so its
// presence alone must not manufacture a depth attachment.
!z.z_info.htile_acceleration && z.shading_rate_encoding == 0 && z.size.x_max == 0 &&
z.size.y_max == 0;
if (attachment_unbound) {
static std::atomic_bool logged = false;
if (!logged.exchange(true, std::memory_order_relaxed)) {
LOGF("DepthTarget: ignoring enabled depth/stencil state without a bound attachment\n");
}
return;
}
if (rc.copy_depth_to_color || rc.copy_stencil_to_color || rc.copy_centroid ||
rc.copy_sample != 0 || dc.zfunc > static_cast<uint8_t>(vk::CompareOp::eAlways) ||
(!z.depth_view.depth_write_disable && z.z_write_base_addr != z.z_read_base_addr) ||
(has_stencil && !z.depth_view.stencil_write_disable &&
z.stencil_write_base_addr != z.stencil_read_base_addr)) {
DepthFatal("unsupported depth register state");
}
r.desc = MakeDepthTargetDesc(buffer, z);
r.depth_clear_enable = rc.depth_clear_enable;
r.depth_load_clear_enable = r.depth_clear_enable;
r.depth_clear_value = hw.GetDepthClearValue();
r.depth_test_enable = dc.z_enable;
r.depth_write_enable = r.depth_test_enable && dc.z_write_enable &&
!z.depth_view.depth_write_disable && !r.depth_clear_enable;
r.depth_compare_op = static_cast<vk::CompareOp>(dc.zfunc);
r.depth_bounds_test_enable = dc.depth_bounds_enable;
r.depth_min_bounds = hw.GetDepthBoundsMin();
r.depth_max_bounds = hw.GetDepthBoundsMax();
r.stencil_clear_enable =
has_stencil && rc.stencil_clear_enable && !z.depth_view.stencil_write_disable;
r.stencil_clear_value = hw.GetStencilClearValue();
r.stencil_test_enable = has_stencil && dc.stencil_enable;
if (r.stencil_test_enable) {
const bool stencil_ops_disabled =
rc.stencil_clear_enable || z.depth_view.stencil_write_disable;
const uint8_t front_write_mask = stencil_ops_disabled ? 0 : sm.stencil_writemask;
const uint8_t back_write_mask = stencil_ops_disabled ? 0 : sm.stencil_writemask_bf;
if (dc.stencilfunc > static_cast<uint8_t>(vk::CompareOp::eAlways) ||
(dc.backface_enable &&
dc.stencilfunc_bf > static_cast<uint8_t>(vk::CompareOp::eAlways))) {
DepthFatal("unsupported stencil compare state");
}
r.stencil_front = ConvertStencilState(
dc.stencilfunc, {sc.stencil_fail, sc.stencil_zpass, sc.stencil_zfail},
sm.stencil_opval, {.compareMask = sm.stencil_mask, .writeMask = front_write_mask,
.reference = sm.stencil_testval});
if (dc.backface_enable) {
r.stencil_back = ConvertStencilState(
dc.stencilfunc_bf, {sc.stencil_fail_bf, sc.stencil_zpass_bf, sc.stencil_zfail_bf},
sm.stencil_opval_bf, {.compareMask = sm.stencil_mask_bf, .writeMask = back_write_mask,
.reference = sm.stencil_testval_bf});
} else {
r.stencil_back = r.stencil_front;
}
}
auto& cache = m_context.GetTextureCache();
r.image_id = cache.FindImage(r.desc);
BindRenderTarget(r.image_id);
}
bool RenderExecutor::DepthStencilCopy(CommandBuffer& buffer) {
const auto& hw = buffer.GetRegisters();
const auto& z = hw.GetDepthRenderTarget();
const auto& override = hw.GetDepthRenderOverride();
if (hw.GetColorControl().mode != 0) {
return false;
}
const bool depth_copy = override.force_z_dirty && override.force_z_valid &&
z.z_info.format != Prospero::DepthFormat::kInvalid &&
z.z_read_base_addr != 0 && z.z_write_base_addr != 0 &&
z.z_read_base_addr != z.z_write_base_addr;
const bool stencil_copy = override.force_stencil_dirty && override.force_stencil_valid &&
z.stencil_info.format != Prospero::StencilFormat::kInvalid &&
z.stencil_read_base_addr != 0 && z.stencil_write_base_addr != 0 &&
z.stencil_read_base_addr != z.stencil_write_base_addr;
if (!depth_copy && !stencil_copy) {
return false;
}
auto read_desc = MakeDepthTargetDesc(buffer, z);
auto write_desc = MakeDepthTargetDesc(buffer, z, true);
auto& cache = m_context.GetTextureCache();
const auto read_id = cache.FindImage(read_desc);
BindRenderTarget(read_id);
const auto write_id = cache.FindImage(write_desc);
BindRenderTarget(write_id);
cache.UpdateImage(read_id);
cache.UpdateImage(write_id);
cache.MarkGpuWritten(write_id);
auto& source = cache.GetImage(read_id);
auto& destination = cache.GetImage(write_id);
EXIT_IF(read_id == write_id || source.backing.format != destination.backing.format);
auto& scheduler = m_context.GetCommandScheduler();
scheduler.EndRendering();
const auto command = scheduler.Current().Handle();
const ImageSubresourceRange range {read_desc.view_info.base_level, 1,
read_desc.view_info.base_layer,
read_desc.view_info.layer_count};
source.Transit(vk::ImageLayout::eTransferSrcOptimal, vk::AccessFlagBits2::eTransferRead,
range, command);
destination.Transit(vk::ImageLayout::eTransferDstOptimal, vk::AccessFlagBits2::eTransferWrite,
range, command);
std::array<vk::ImageCopy, 2> regions {};
uint32_t count = 0;
for (const auto aspect: {vk::ImageAspectFlagBits::eDepth, vk::ImageAspectFlagBits::eStencil}) {
if ((aspect == vk::ImageAspectFlagBits::eDepth && !depth_copy) ||
(aspect == vk::ImageAspectFlagBits::eStencil && !stencil_copy)) {
continue;
}
auto& region = regions[count++];
region.srcSubresource = {aspect, range.base_level, range.base_layer,
range.layer_count};
region.dstSubresource = region.srcSubresource;
region.extent = write_desc.info.extent;
}
command.copyImage(source.backing.image, vk::ImageLayout::eTransferSrcOptimal,
destination.backing.image, vk::ImageLayout::eTransferDstOptimal,
count, regions.data());
return true;
}
vk::ImageAspectFlags RenderDepthInfo::AttachmentWriteAspects() const {
const auto format = desc.view_info.format;
if (format == vk::Format::eUndefined) {
@@ -336,14 +429,13 @@ vk::ImageAspectFlags RenderDepthInfo::AttachmentWriteAspects() const {
return writes;
}
const auto face_writes = [&](const PipelineStencilStaticState& state,
const PipelineStencilDynamicState& dynamic) {
if (dynamic.writeMask == 0) {
const auto face_writes = [&](const vk::StencilOpState& state) {
if (state.writeMask == 0) {
return false;
}
bool can_pass = state.compareOp != vk::CompareOp::eNever;
bool can_fail = state.compareOp != vk::CompareOp::eAlways;
if (dynamic.compareMask == 0) {
if (state.compareMask == 0) {
switch (state.compareOp) {
case vk::CompareOp::eEqual:
case vk::CompareOp::eLessOrEqual:
@@ -369,8 +461,7 @@ vk::ImageAspectFlags RenderDepthInfo::AttachmentWriteAspects() const {
(can_pass && depth_fail && state.depthFailOp != vk::StencilOp::eKeep);
};
if (stencil_clear_enable ||
(stencil_test_enable && (face_writes(stencil_static_front, stencil_dynamic_front) ||
face_writes(stencil_static_back, stencil_dynamic_back)))) {
(stencil_test_enable && (face_writes(stencil_front) || face_writes(stencil_back)))) {
writes |= vk::ImageAspectFlagBits::eStencil;
}
return writes;
@@ -23,7 +23,6 @@ struct RenderDepthInfo {
TextureCache::ImageDesc desc;
bool depth_clear_enable = false;
bool depth_load_clear_enable = false;
bool depth_meta_clear_enable = false;
float depth_clear_value = 0.0f;
bool depth_test_enable = false;
// Effective draw writes; discovery applies test, target-write and clear controls.
@@ -35,10 +34,8 @@ struct RenderDepthInfo {
bool stencil_clear_enable = false;
uint8_t stencil_clear_value = 0;
bool stencil_test_enable = false;
PipelineStencilStaticState stencil_static_front;
PipelineStencilStaticState stencil_static_back;
PipelineStencilDynamicState stencil_dynamic_front;
PipelineStencilDynamicState stencil_dynamic_back;
vk::StencilOpState stencil_front;
vk::StencilOpState stencil_back;
ImageId image_id;
[[nodiscard]] vk::ImageAspectFlags AttachmentWriteAspects() const;
@@ -10,7 +10,6 @@
#include <algorithm>
#include <array>
#include <iterator>
namespace Libs::Graphics {
@@ -37,9 +36,8 @@ BlitHelper::BlitHelper(GraphicContext& graphics, CommandScheduler& scheduler)
m_graphics.device.createPipelineLayout(&layout_info, nullptr, &m_pipeline_layout),
"create BlitHelper pipeline layout");
m_vertex_shader = CreateShader(GPU_BLIT_FS_TRIANGLE_SPV, std::size(GPU_BLIT_FS_TRIANGLE_SPV));
m_fragment_shader =
CreateShader(GPU_BLIT_COLOR_TO_MS_DEPTH_SPV, std::size(GPU_BLIT_COLOR_TO_MS_DEPTH_SPV));
m_vertex_shader = CompileSPV(GPU_BLIT_FS_TRIANGLE_SPV, m_graphics.device);
m_fragment_shader = CompileSPV(GPU_BLIT_COLOR_TO_MS_DEPTH_SPV, m_graphics.device);
}
BlitHelper::~BlitHelper() {
@@ -60,17 +58,6 @@ BlitHelper::~BlitHelper() {
}
}
vk::ShaderModule BlitHelper::CreateShader(const uint32_t* code, size_t words) const {
EXIT_IF(code == nullptr || words == 0);
vk::ShaderModuleCreateInfo create {};
create.codeSize = words * sizeof(uint32_t);
create.pCode = code;
vk::ShaderModule module = nullptr;
RequireVulkanSuccess(m_graphics.device.createShaderModule(&create, nullptr, &module),
"create BlitHelper shader module");
return module;
}
vk::Pipeline BlitHelper::GetPipeline(PipelineKey key) {
const auto cached = std::ranges::find(m_pipelines, key, &Pipeline::key);
if (cached != m_pipelines.end()) {
@@ -37,8 +37,7 @@ private:
vk::Pipeline handle = nullptr;
};
[[nodiscard]] vk::ShaderModule CreateShader(const uint32_t* code, size_t words) const;
[[nodiscard]] vk::Pipeline GetPipeline(PipelineKey key);
[[nodiscard]] vk::Pipeline GetPipeline(PipelineKey key);
GraphicContext& m_graphics;
CommandScheduler& m_scheduler;
+21 -25
View File
@@ -26,12 +26,13 @@ namespace {
}
}
[[nodiscard]] vk::ImageCreateFlags ImageCreateFlags(const ImageInfo& info) {
[[nodiscard]] vk::ImageCreateFlags ImageCreateFlags(const GraphicContext& graphics,
const ImageInfo& info) {
vk::ImageCreateFlags flags {};
if (DepthAspectTransferFormat(info.pixel_format) == vk::Format::eUndefined) {
flags |= vk::ImageCreateFlagBits::eMutableFormat;
flags |= vk::ImageCreateFlagBits::eExtendedUsage;
if (Prospero::BlockCompressedBytesPerBlock(info.guest_format) != 0) {
if (info.IsBlock() && graphics.supports_block_texel_view) {
flags |= vk::ImageCreateFlagBits::eBlockTexelViewCompatible;
}
}
@@ -47,6 +48,10 @@ namespace {
}
[[nodiscard]] vk::ImageUsageFlags ImageUsageFlags(GraphicContext& graphics, const ImageInfo& info) {
if (info.IsBlock()) {
return vk::ImageUsageFlagBits::eTransferSrc | vk::ImageUsageFlagBits::eTransferDst |
vk::ImageUsageFlagBits::eSampled;
}
const auto properties = graphics.GetFormatProperties(info.pixel_format);
auto usage = vk::ImageUsageFlagBits::eTransferSrc | vk::ImageUsageFlagBits::eTransferDst;
if (HasFormatFeature(properties, vk::FormatFeatureFlagBits::eSampledImage)) {
@@ -656,26 +661,17 @@ Image::Image(GraphicContext& graphics, CommandScheduler& scheduler, const ImageI
return;
}
backing.format = info.pixel_format;
backing.image_type = HostImageType(info.type);
backing.extent = info.extent;
backing.layers = info.IsVolume() ? 1u : info.resources.layers;
backing.mip_levels = info.resources.levels;
backing.samples = info.samples;
backing.flags = ImageCreateFlags(info);
backing.usage = ImageUsageFlags(graphics, info);
vk::ImageCreateInfo create {};
create.flags = backing.flags;
create.imageType = backing.image_type;
create.extent = backing.extent;
create.mipLevels = backing.mip_levels;
create.arrayLayers = backing.layers;
create.format = backing.format;
create.flags = ImageCreateFlags(graphics, info);
create.imageType = HostImageType(info.type);
create.extent = info.extent;
create.mipLevels = info.resources.levels;
create.arrayLayers = info.IsVolume() ? 1u : info.resources.layers;
create.format = info.pixel_format;
create.tiling = vk::ImageTiling::eOptimal;
create.initialLayout = backing.state.layout;
create.usage = backing.usage;
create.samples = vulkan_sample_count(backing.samples);
create.initialLayout = vk::ImageLayout::eUndefined;
create.usage = ImageUsageFlags(graphics, info);
create.samples = vulkan_sample_count(info.samples);
vk::ImageFormatProperties properties {};
if (graphics.GetImageFormatProperties(create.format, create.imageType, create.tiling,
@@ -686,10 +682,9 @@ Image::Image(GraphicContext& graphics, CommandScheduler& scheduler, const ImageI
"flags=0x%x samples=%u\n",
static_cast<int>(create.format), static_cast<int>(create.imageType),
static_cast<vk::ImageUsageFlags::MaskType>(create.usage),
static_cast<vk::ImageCreateFlags::MaskType>(create.flags), backing.samples);
static_cast<vk::ImageCreateFlags::MaskType>(create.flags), info.samples);
}
backing.memory.property = vk::MemoryPropertyFlagBits::eDeviceLocal;
if (!graphics.CreateImage(create, backing)) {
EXIT("failed to create image: extent=%ux%ux%u format=%d layers=%u levels=%u\n",
create.extent.width, create.extent.height, create.extent.depth,
@@ -698,11 +693,12 @@ Image::Image(GraphicContext& graphics, CommandScheduler& scheduler, const ImageI
}
uint64_t Image::HashGuestEdges() const {
constexpr uint64_t page_mask = TRACKER_PAGE_SIZE - 1;
std::array<uint8_t, TRACKER_PAGE_SIZE * 2> bytes {};
const auto range = info.data;
const uint64_t head_end = std::min(range.End(), (range.address + page_mask) & ~page_mask);
const uint64_t tail_begin = std::max(range.address, range.End() & ~page_mask);
const uint64_t head_end =
std::min(range.End(), Common::AlignUp(range.address, TRACKER_PAGE_SIZE));
const uint64_t tail_begin =
std::max(range.address, Common::AlignDown(range.End(), TRACKER_PAGE_SIZE));
const uint64_t head_size = head_end - range.address;
const uint64_t tail_address = tail_begin < head_end ? head_end : tail_begin;
const uint64_t tail_size = range.End() - tail_address;
+2 -1
View File
@@ -1,6 +1,7 @@
#ifndef EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_IMAGE_H_
#define EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_IMAGE_H_
#include "common/alignment.h"
#include "common/assert.h"
#include "common/slotVector.h"
#include "graphics/host_gpu/graphicContext.h"
@@ -133,7 +134,7 @@ public:
}
[[nodiscard]] bool IsTracked() const noexcept { return track_addr != 0 && track_addr_end != 0; }
[[nodiscard]] uint64_t AccountedSize() const noexcept {
return backing.image == nullptr ? 0 : (info.data.size + 1023) & ~uint64_t {1023};
return backing.image == nullptr ? 0 : Common::AlignUp(info.data.size, 1024);
}
[[nodiscard]] uint64_t HashGuestEdges() const;
+44 -103
View File
@@ -82,9 +82,6 @@ struct ImageInfo {
[[nodiscard]] constexpr bool IsVolume() const noexcept {
return type == Prospero::ImageType::kColor3D;
}
[[nodiscard]] constexpr bool IsLayered() const noexcept {
return !IsVolume() && resources.layers > 1;
}
[[nodiscard]] constexpr uint32_t TransferLayers() const noexcept {
return IsVolume() ? extent.depth : resources.layers;
}
@@ -97,9 +94,24 @@ struct ImageInfo {
bytes_per_block == other.bytes_per_block;
}
[[nodiscard]] int32_t MipOf(const ImageInfo& container) const noexcept {
if (!IsCompatible(container) || tile_mode != container.tile_mode || resources.levels != 1 ||
container.resources.layers == 0 ||
container.resources.levels > container.mip_layout.size()) {
if (container.resources.levels > container.mip_layout.size()) {
return -1;
}
for (uint32_t level = 0; level < container.resources.levels; level++) {
if (SliceOf(container, static_cast<int32_t>(level)) >= 0) {
return static_cast<int32_t>(level);
}
}
return -1;
}
[[nodiscard]] int32_t SliceOf(const ImageInfo& container, int32_t mip) const noexcept {
if (!IsCompatible(container) || tile_mode != container.tile_mode || type != container.type ||
IsVolume() || resources.levels != 1 || resources.layers == 0 ||
container.resources.layers == 0 || mip < 0 ||
static_cast<uint32_t>(mip) >= container.resources.levels ||
container.resources.levels > container.mip_layout.size() ||
!data.Valid() || !container.data.Valid() || data.address < container.data.address ||
data.End() > container.data.End()) {
return -1;
}
if (HasStencil() != container.HasStencil() ||
@@ -108,75 +120,36 @@ struct ImageInfo {
return -1;
}
int32_t mip = -1;
for (uint32_t level = 0; level < container.resources.levels; level++) {
const auto& layout = container.mip_layout[level];
if (layout.size == 0 || layout.size % container.resources.layers != 0 ||
container.data.address > UINT64_MAX - layout.offset) {
continue;
}
const auto mip_base = container.data.address + layout.offset;
const auto slice_size = layout.size / container.resources.layers;
if (slice_size == 0 || mip_base > UINT64_MAX - layout.size) {
continue;
}
const auto mip_end = mip_base + layout.size;
if (data.address >= mip_base && data.address < mip_end &&
(data.address - mip_base) % slice_size == 0) {
mip = static_cast<int32_t>(level);
break;
}
}
if (mip < 0) {
const auto level = static_cast<uint32_t>(mip);
const auto& layout = container.mip_layout[level];
if (extent.width != std::max(container.extent.width >> level, 1u) ||
extent.height != std::max(container.extent.height >> level, 1u) ||
extent.depth != container.extent.depth || mip_layout[0].pitch != layout.pitch ||
mip_layout[0].height != layout.height || layout.size == 0 ||
layout.size % container.resources.layers != 0 ||
container.data.size % container.resources.layers != 0 ||
data.size % resources.layers != 0) {
return -1;
}
const auto level = static_cast<uint32_t>(mip);
if (extent.width != std::max(container.extent.width >> level, 1u) ||
extent.height != std::max(container.extent.height >> level, 1u)) {
// PS5 block slices contain the entire reversed mip chain.
const auto slice_stride = container.data.size / container.resources.layers;
const auto child_stride = data.size / resources.layers;
if (child_stride != layout.size / container.resources.layers ||
layout.offset >= slice_stride || child_stride > slice_stride - layout.offset ||
(resources.layers > 1 && child_stride != slice_stride)) {
return -1;
}
const auto mip_depth = std::max(container.extent.depth >> level, 1u);
if (container.type == Prospero::ImageType::kColor3D &&
type == Prospero::ImageType::kColor2D) {
if (resources.layers != mip_depth) {
return -1;
}
} else if (type != container.type) {
const auto address_delta = data.address - container.data.address;
if (address_delta < layout.offset || (address_delta - layout.offset) % slice_stride != 0) {
return -1;
}
return mip;
}
[[nodiscard]] int32_t SliceOf(const ImageInfo& container, int32_t mip) const noexcept {
if (!IsCompatible(container) || type != container.type || mip < 0 ||
static_cast<uint32_t>(mip) >= container.resources.levels ||
container.resources.levels > container.mip_layout.size() ||
container.resources.layers == 0 || data.size == 0) {
const auto layer = (address_delta - layout.offset) / slice_stride;
if (layer > INT32_MAX || layer >= container.resources.layers ||
resources.layers > container.resources.layers - layer) {
return -1;
}
const auto level = static_cast<uint32_t>(mip);
if (extent.width != std::max(container.extent.width >> level, 1u) ||
extent.height != std::max(container.extent.height >> level, 1u)) {
return -1;
}
const auto& layout = container.mip_layout[level];
if (layout.size == 0 || layout.size % container.resources.layers != 0 ||
container.data.address > UINT64_MAX - layout.offset) {
return -1;
}
const auto slice_size = layout.size / container.resources.layers;
if (slice_size == 0 || data.size % slice_size != 0) {
return -1;
}
const auto mip_base = container.data.address + layout.offset;
if (data.address < mip_base) {
return -1;
}
const auto address_delta = data.address - mip_base;
if (address_delta % data.size != 0 || address_delta / data.size > INT32_MAX) {
return -1;
}
return static_cast<int32_t>(address_delta / data.size);
return static_cast<int32_t>(layer);
}
};
@@ -186,6 +159,7 @@ struct ImageViewInfo {
vk::ImageAspectFlags aspect = vk::ImageAspectFlagBits::eColor;
uint32_t base_level = 0;
uint32_t level_count = 1;
uint32_t min_lod = 0; // U4.8 clamp relative to base_level.
uint32_t base_layer = 0;
uint32_t layer_count = 1;
vk::ComponentMapping mapping = {};
@@ -194,9 +168,10 @@ struct ImageViewInfo {
[[nodiscard]] bool operator==(const ImageViewInfo& rhs) const noexcept {
return format == rhs.format && type == rhs.type && aspect == rhs.aspect &&
base_level == rhs.base_level && level_count == rhs.level_count &&
base_layer == rhs.base_layer && layer_count == rhs.layer_count &&
mapping.r == rhs.mapping.r && mapping.g == rhs.mapping.g &&
mapping.b == rhs.mapping.b && mapping.a == rhs.mapping.a && usage == rhs.usage;
min_lod == rhs.min_lod && base_layer == rhs.base_layer &&
layer_count == rhs.layer_count && mapping.r == rhs.mapping.r &&
mapping.g == rhs.mapping.g && mapping.b == rhs.mapping.b &&
mapping.a == rhs.mapping.a && usage == rhs.usage;
}
};
@@ -342,14 +317,6 @@ ClassifyVideoOutCompression(bool compressed, uint64_t metadata_address, uint32_t
}
}
[[nodiscard]] inline constexpr bool
CanUseVideoOutNativeWithoutUpload(VideoOutCompression compression, bool render_target,
bool gpu_modified, bool guest_modified) noexcept {
return compression != VideoOutCompression::Uncompressed &&
compression != VideoOutCompression::Unsupported && !guest_modified &&
(render_target || gpu_modified);
}
struct VideoOutPixelFormatInfo {
vk::Format format = vk::Format::eUndefined;
Prospero::BufferFormat guest_format = Prospero::BufferFormat::kInvalid;
@@ -400,32 +367,6 @@ inline constexpr std::array<VideoOutFormatPolicy, 6> VIDEO_OUT_FORMAT_POLICIES {
return false;
}
[[nodiscard]] inline constexpr bool
IsSupportedDisplayRenderTargetTileMode(Prospero::TileMode tile_mode) noexcept {
return tile_mode == Prospero::TileMode::kRenderTarget;
}
[[nodiscard]] inline constexpr bool IsSupportedStandard64RenderTarget(const ImageInfo& info) {
if (info.tile_mode != Prospero::TileMode::kStandard64KB || info.data.address == 0 ||
(info.data.address & 0xffffu) != 0 || info.extent.width == 0 || info.extent.height == 0 ||
info.bytes_per_block != 4 || info.resources.levels != 1 || info.resources.layers != 1 ||
info.samples != 1) {
return false;
}
const auto expected_pitch =
(static_cast<uint64_t>(info.extent.width) + 127u) & ~uint64_t {127u};
const auto padded_height =
(static_cast<uint64_t>(info.extent.height) + 127u) & ~uint64_t {127u};
return expected_pitch <= UINT32_MAX && info.pitch == expected_pitch &&
expected_pitch <= UINT64_MAX / padded_height / info.bytes_per_block &&
info.data.size == expected_pitch * padded_height * info.bytes_per_block;
}
[[nodiscard]] inline constexpr bool IsTiledRenderTarget(const ImageInfo& info) noexcept {
return info.tile_mode == Prospero::TileMode::kRenderTarget ||
IsSupportedStandard64RenderTarget(info);
}
[[nodiscard]] inline bool DecodePackedColorClear(vk::Format format, uint32_t packed,
vk::ClearColorValue& clear) {
vk::ClearColorValue next {};
@@ -322,6 +322,12 @@ vk::ImageView Image::FindView(const ImageViewInfo& view_info) {
normalized.aspect = vk::ImageAspectFlagBits::eStencil;
}
normalized.usage = is_storage ? vk::ImageUsageFlagBits::eStorage : vk::ImageUsageFlags {};
for (const auto& cached: views) {
if (cached.info == normalized) {
return cached.view;
}
}
const bool format_compatible = normalized.format != vk::Format::eUndefined &&
ImageViewOps::FormatsCompatible(image.format, normalized.format);
const bool slice_view =
@@ -351,17 +357,17 @@ vk::ImageView Image::FindView(const ImageViewInfo& view_info) {
image.layers);
}
for (const auto& cached: views) {
if (cached.info == normalized) {
return cached.view;
}
}
vk::ImageViewUsageCreateInfo usage {};
usage.usage = image.usage;
if (!is_storage) {
usage.usage &= ~vk::ImageUsageFlagBits::eStorage;
}
vk::ImageViewMinLodCreateInfoEXT min_lod {};
if (normalized.min_lod != 0) {
min_lod.minLod = static_cast<float>(normalized.base_level) +
static_cast<float>(normalized.min_lod) / 256.0f;
usage.pNext = &min_lod;
}
vk::ImageViewCreateInfo create {};
create.pNext = &usage;
create.image = image.image;
@@ -1,15 +1,16 @@
#include "graphics/host_gpu/renderer/image/textureCommon.h"
#include "common/alignment.h"
#include "common/assert.h"
#include "graphics/guest_gpu/gpu_defs.h"
#include "graphics/guest_gpu/gpu_format.h"
#include "graphics/host_gpu/renderer/image/tiler.h"
#include "graphics/host_gpu/vulkanCommon.h"
#include "graphics/shader/shader.h"
#include <algorithm>
#include <bit>
#include <array>
#include <cinttypes>
#include <cstring>
namespace Libs::Graphics {
namespace {
@@ -27,13 +28,13 @@ constexpr Prospero::ColorComponentMapping kRenderTargetColorMappings[4][4] = {
Prospero::ColorMappingArgb},
};
struct RenderTargetHostFormatInfo {
struct HostFormatInfo {
vk::Format format = vk::Format::eUndefined;
Prospero::ColorComponentMapping host_to_storage;
};
RenderTargetHostFormatInfo ResolveRenderTargetHostFormat(Prospero::BufferFormat guest_format,
Prospero::ChannelOrder order) {
HostFormatInfo ResolveHostFormat(Prospero::BufferFormat guest_format,
Prospero::ChannelOrder order) {
if (order == Prospero::ChannelOrder::kAlt) {
switch (guest_format) {
case Prospero::BufferFormat::k8_8_8_8UNorm:
@@ -47,17 +48,77 @@ RenderTargetHostFormatInfo ResolveRenderTargetHostFormat(Prospero::BufferFormat
default: break;
}
}
const auto format = VulkanFormat(guest_format);
switch (guest_format) {
case Prospero::BufferFormat::k5_5_5_1UNorm:
return {vk::Format::eA1R5G5B5UnormPack16, Prospero::ColorMappingBgra};
case Prospero::BufferFormat::k5_5_5_1UNorm: return {format, Prospero::ColorMappingBgra};
case Prospero::BufferFormat::k1_5_5_5UNorm:
return {vk::Format::eR5G5B5A1UnormPack16, Prospero::ColorMappingAbgr};
case Prospero::BufferFormat::k4_4_4_4UNorm:
return {vk::Format::eR4G4B4A4UnormPack16, Prospero::ColorMappingAbgr};
default: return {VulkanFormat(guest_format), {}};
case Prospero::BufferFormat::k4_4_4_4UNorm: return {format, Prospero::ColorMappingAbgr};
default: return {format, {}};
}
}
uint32_t GetTextureLevelDepth(uint32_t depth, uint32_t level, bool volume_texture) {
return volume_texture ? std::max(depth >> level, 1u) : depth;
}
size_t GetTextureRegionCount(uint32_t depth, uint32_t levels, bool volume_texture) {
size_t count = 0;
for (uint32_t level = 0; level < levels; level++) {
count += GetTextureLevelDepth(depth, level, volume_texture);
}
return count;
}
uint64_t CalcTextureSliceStride(const TextureUploadMipLayout* mips, uint32_t levels,
uint64_t total_size, uint32_t depth) {
uint64_t stride = 0;
for (uint32_t mip_level = 0; mip_level < levels; ++mip_level) {
stride = std::max(stride, mips[mip_level].offset + mips[mip_level].size);
}
if (depth > 1 && total_size != 0 && total_size % depth == 0) {
stride = std::max(stride, total_size / depth);
}
return stride;
}
uint64_t CalcLinearUploadLevelSize(const TileTextureElementLayout& element, uint32_t pitch,
uint32_t height) {
const uint32_t element_columns =
std::max((pitch + element.texel_width - 1u) / element.texel_width, 1u);
const uint32_t element_rows =
std::max((height + element.texel_height - 1u) / element.texel_height, 1u);
return static_cast<uint64_t>(element_columns) * element_rows * element.bytes;
}
uint64_t SetLinearUploadLevels(TextureUploadMipLayout* mips,
const TileTextureElementLayout& element, uint32_t height,
uint32_t levels, uint32_t base_pitch) {
uint64_t offset = 0;
auto mip_pitch = base_pitch;
auto mip_height = height;
for (uint32_t mip_level = 0; mip_level < levels; ++mip_level) {
const auto size = CalcLinearUploadLevelSize(element, mip_pitch, mip_height);
mips[mip_level] = {offset, size};
offset += size;
if (mip_pitch > 1) {
mip_pitch /= 2;
}
if (mip_height > 1) {
mip_height /= 2;
}
}
return offset;
}
bool FitsBufferRange(uint64_t offset, uint64_t size, uint64_t capacity) {
return offset <= capacity && size <= capacity - offset;
}
} // namespace
RenderTargetFormatInfo TextureGetRenderTargetFormat(Prospero::ChannelLayout layout,
@@ -65,135 +126,61 @@ RenderTargetFormatInfo TextureGetRenderTargetFormat(Prospero::ChannelLayout layo
Prospero::ChannelOrder order) {
const auto encoding = Prospero::ResolveRenderTargetFormat(layout, type);
if (encoding.IsValid() && encoding.SupportsOrder(order)) {
const auto host = ResolveRenderTargetHostFormat(encoding.buffer_format, order);
const auto bytes = Prospero::RenderTargetBytesPerElement(encoding.buffer_format);
if (host.format != vk::Format::eUndefined && bytes != 0) {
const auto host_format = ResolveHostFormat(encoding.buffer_format, order);
const auto bytes = Prospero::RenderTargetBytesPerElement(encoding.buffer_format);
if (host_format.format != vk::Format::eUndefined && bytes != 0) {
const auto order_mapping =
kRenderTargetColorMappings[static_cast<size_t>(order)][encoding.components - 1u];
return {host.format, bytes, host.host_to_storage.Then(order_mapping)};
return {host_format.format, bytes, host_format.host_to_storage.Then(order_mapping)};
}
}
EXIT("unsupported render-target format combination: layout=%u type=%u order=%u\n",
static_cast<uint32_t>(layout), static_cast<uint32_t>(type), static_cast<uint32_t>(order));
}
namespace {
static uint32_t GetTextureLevelDepth(uint32_t depth, uint32_t level, bool volume_texture) {
return volume_texture ? std::max(depth >> level, 1u) : depth;
}
static size_t GetTextureRegionCount(uint32_t depth, uint64_t levels, bool volume_texture) {
size_t count = 0;
for (uint32_t level = 0; level < levels; level++) {
count += GetTextureLevelDepth(depth, level, volume_texture);
vk::ComponentMapping TextureGetComponentMapping(uint32_t swizzle,
Prospero::ColorComponentMapping host_to_storage) {
constexpr std::array host_components {vk::ComponentSwizzle::eR, vk::ComponentSwizzle::eG,
vk::ComponentSwizzle::eB, vk::ComponentSwizzle::eA};
std::array<vk::ComponentSwizzle, 4> storage_components {};
for (uint32_t host_component = 0; host_component < host_components.size(); ++host_component) {
storage_components[host_to_storage.Map(host_component)] = host_components[host_component];
}
return count;
}
vk::ComponentSwizzle TextureGetComponentSwizzle(uint8_t s) {
switch (static_cast<Prospero::CompSwizzle>(s)) {
case Prospero::CompSwizzle::kZero: return vk::ComponentSwizzle::eZero;
case Prospero::CompSwizzle::kOne: return vk::ComponentSwizzle::eOne;
case Prospero::CompSwizzle::kRed: return vk::ComponentSwizzle::eR;
case Prospero::CompSwizzle::kGreen: return vk::ComponentSwizzle::eG;
case Prospero::CompSwizzle::kBlue: return vk::ComponentSwizzle::eB;
case Prospero::CompSwizzle::kAlpha: return vk::ComponentSwizzle::eA;
default: EXIT("unknown swizzle: %d\n", static_cast<int>(s));
}
return vk::ComponentSwizzle::eIdentity;
}
static uint32_t TextureGetDstSel(uint32_t swizzle, uint32_t channel) {
return (swizzle >> (channel * 3u)) & 0x7u;
}
} // namespace
vk::ComponentMapping TextureGetComponentMapping(uint32_t swizzle) {
vk::ComponentMapping components {};
components.r = TextureGetComponentSwizzle(static_cast<uint8_t>(TextureGetDstSel(swizzle, 0)));
components.g = TextureGetComponentSwizzle(static_cast<uint8_t>(TextureGetDstSel(swizzle, 1)));
components.b = TextureGetComponentSwizzle(static_cast<uint8_t>(TextureGetDstSel(swizzle, 2)));
components.a = TextureGetComponentSwizzle(static_cast<uint8_t>(TextureGetDstSel(swizzle, 3)));
return components;
const auto resolve = [&](uint32_t component) {
const auto selector = static_cast<Prospero::CompSwizzle>(GetDstSel(swizzle, component));
switch (selector) {
case Prospero::CompSwizzle::kZero: return vk::ComponentSwizzle::eZero;
case Prospero::CompSwizzle::kOne: return vk::ComponentSwizzle::eOne;
case Prospero::CompSwizzle::kRed: return storage_components[0];
case Prospero::CompSwizzle::kGreen: return storage_components[1];
case Prospero::CompSwizzle::kBlue: return storage_components[2];
case Prospero::CompSwizzle::kAlpha: return storage_components[3];
default: EXIT("unknown swizzle: %u\n", static_cast<uint32_t>(selector));
}
};
return {resolve(0), resolve(1), resolve(2), resolve(3)};
}
SurfaceFormatInfo TextureGetSurfaceFormatInfo(Prospero::BufferFormat format) {
const auto backing_format = Prospero::RemapTextureFormat(format);
const auto vk_format = VulkanFormat(backing_format);
const auto backing_format = Prospero::RemapTextureFormat(format);
const auto host_format = ResolveHostFormat(backing_format, Prospero::ChannelOrder::kStandard);
const auto conversion_format =
backing_format != format ? format : Prospero::BufferFormat::kInvalid;
if (vk_format != vk::Format::eUndefined) {
return SurfaceFormatInfo(vk_format, conversion_format);
if (host_format.format != vk::Format::eUndefined) {
return {host_format.format, conversion_format, host_format.host_to_storage};
}
EXIT("unknown format: fmt = %u\n", static_cast<uint32_t>(format));
return SurfaceFormatInfo(vk::Format::eUndefined, Prospero::BufferFormat::kInvalid);
}
namespace {
uint64_t CalcTextureSliceStride(const TextureUploadMipLayout* mips, uint64_t levels,
uint64_t total_size, uint32_t depth) {
uint64_t stride = 0;
for (uint32_t i = 0; i < levels; i++) {
stride = std::max(stride, mips[i].offset + mips[i].size);
}
if (depth > 1 && total_size != 0 && total_size % depth == 0) {
const auto guest_stride = total_size / depth;
if (guest_stride >= stride) {
stride = guest_stride;
}
}
return stride;
}
uint64_t CalcLinearUploadLevelSize(const TileTextureElementLayout& element, uint32_t pitch,
uint32_t height) {
const uint32_t elements_w =
std::max((pitch + element.texel_width - 1u) / element.texel_width, 1u);
const uint32_t elements_h =
std::max((height + element.texel_height - 1u) / element.texel_height, 1u);
return static_cast<uint64_t>(elements_w) * elements_h * element.bytes;
}
uint64_t SetLinearUploadLevels(TextureUploadMipLayout* mips,
const TileTextureElementLayout& element, uint64_t height,
uint64_t levels, uint32_t base_pitch) {
uint64_t offset = 0;
auto pitch = base_pitch;
auto h = static_cast<uint32_t>(height);
for (uint32_t i = 0; i < levels; i++) {
const auto size = CalcLinearUploadLevelSize(element, pitch, h);
mips[i].size = size;
mips[i].offset = offset;
offset += size;
if (pitch > 1) {
pitch /= 2;
}
if (h > 1) {
h /= 2;
}
}
return offset;
}
} // namespace
TextureUploadLayout TextureCalcUploadLayout(Prospero::BufferFormat format, uint32_t width,
uint32_t height, uint32_t levels, uint32_t depth,
Prospero::TileMode tile, uint64_t upload_size,
Prospero::TileMode tile_mode, uint64_t upload_size,
bool allow_depth_tile, bool volume_texture,
const char* owner) {
TextureUploadLayout layout {};
layout.surface.description = {
format,
tile,
tile_mode,
volume_texture ? TileSurfaceDimension::Dim3D : TileSurfaceDimension::Dim2D,
width,
height,
@@ -202,7 +189,6 @@ TextureUploadLayout TextureCalcUploadLayout(Prospero::BufferFormat format, uint3
volume_texture ? 1u : depth,
};
const auto& description = layout.surface.description;
const auto tile_mode = description.tile_mode;
TileTextureElementLayout element {};
if (format == Prospero::BufferFormat::kInvalid) {
@@ -212,34 +198,19 @@ TextureUploadLayout TextureCalcUploadLayout(Prospero::BufferFormat format, uint3
levels);
}
switch (tile_mode) {
case Prospero::TileMode::kLinear:
if (!TileGetTextureElementLayout(format, element)) {
EXIT("%s: unsupported linear texture format: fmt=%u\n", owner,
static_cast<uint32_t>(format));
}
break;
case Prospero::TileMode::kDepth:
if (!allow_depth_tile || !TileGetTiledTextureLayout(description, layout.surface)) {
EXIT("%s: unsupported typed tiled upload: fmt=%u tile=%u "
"size=%" PRIu64 " extent=%ux%u levels=%u\n",
owner, static_cast<uint32_t>(format),
static_cast<uint32_t>(description.tile_mode), upload_size, width, height,
levels);
}
break;
default:
if (!TileGetTiledTextureLayout(description, layout.surface)) {
EXIT("%s: unsupported typed tiled upload: fmt=%u tile=%u "
"size=%" PRIu64 " extent=%ux%u levels=%u\n",
owner, static_cast<uint32_t>(format),
static_cast<uint32_t>(description.tile_mode), upload_size, width, height,
levels);
}
break;
}
if (tile_mode != Prospero::TileMode::kLinear) {
if (tile_mode == Prospero::TileMode::kLinear) {
if (!TileGetTextureElementLayout(format, element)) {
EXIT("%s: unsupported linear texture format: fmt=%u\n", owner,
static_cast<uint32_t>(format));
}
} else {
if ((tile_mode == Prospero::TileMode::kDepth && !allow_depth_tile) ||
!TileGetTiledTextureLayout(description, layout.surface)) {
EXIT("%s: unsupported typed tiled upload: fmt=%u tile=%u "
"size=%" PRIu64 " extent=%ux%u levels=%u\n",
owner, static_cast<uint32_t>(format), static_cast<uint32_t>(tile_mode),
upload_size, width, height, levels);
}
element = {layout.surface.texture.block.bytes_per_element,
layout.surface.texture.texel_width, layout.surface.texture.texel_height};
}
@@ -272,40 +243,33 @@ TextureUploadLayout TextureCalcUploadLayout(Prospero::BufferFormat format, uint3
std::vector<vk::BufferImageCopy> TextureBuildImageCopies(const TextureUploadLayout& layout) {
const auto& description = layout.surface.description;
const bool volume_texture = description.dimension == TileSurfaceDimension::Dim3D;
const bool linear_texture = description.tile_mode == Prospero::TileMode::kLinear;
const auto depth = volume_texture ? description.depth : description.layers;
uint32_t mip_width = description.width;
uint32_t mip_height = description.height;
uint32_t mip_pitch = volume_texture && description.tile_mode != Prospero::TileMode::kLinear
? description.width
: layout.pitch;
uint32_t mip_pitch = volume_texture && !linear_texture ? description.width : layout.pitch;
std::vector<vk::BufferImageCopy> regions;
regions.reserve(GetTextureRegionCount(depth, description.levels, volume_texture));
for (uint32_t i = 0; i < description.levels; i++) {
EXIT_NOT_IMPLEMENTED(layout.mips[i].size == 0);
const auto mip_depth = GetTextureLevelDepth(depth, i, volume_texture);
for (uint32_t z = 0; z < mip_depth; z++) {
const auto slice_offset = z * layout.slice_stride;
vk::BufferImageCopy region {};
region.bufferOffset = layout.mips[i].offset + slice_offset;
region.imageSubresource = {vk::ImageAspectFlagBits::eColor, i, volume_texture ? 0u : z,
1};
region.imageOffset.z = volume_texture ? static_cast<int>(z) : 0;
region.imageExtent = {mip_width, mip_height, 1};
const bool linear = description.tile_mode == Prospero::TileMode::kLinear;
if (linear) {
region.bufferRowLength = layout.mips[i].row_length;
region.bufferImageHeight = layout.mips[i].image_height;
} else {
const auto texel_width = layout.surface.texture.texel_width;
const auto align = [](uint32_t value, uint32_t block) {
return ((value + block - 1u) / block) * block;
};
const auto pitch = align(mip_pitch, texel_width);
region.bufferRowLength = pitch > align(mip_width, texel_width) ? pitch : 0;
}
for (uint32_t mip_level = 0; mip_level < description.levels; ++mip_level) {
EXIT_NOT_IMPLEMENTED(layout.mips[mip_level].size == 0);
const auto mip_depth = GetTextureLevelDepth(depth, mip_level, volume_texture);
vk::BufferImageCopy region {};
region.imageSubresource = {vk::ImageAspectFlagBits::eColor, mip_level, 0, 1};
region.imageExtent = {mip_width, mip_height, 1};
if (linear_texture) {
region.bufferRowLength = layout.mips[mip_level].row_length;
region.bufferImageHeight = layout.mips[mip_level].image_height;
} else {
const auto texel_width = layout.surface.texture.texel_width;
const auto aligned_pitch = Common::AlignUp(mip_pitch, texel_width);
region.bufferRowLength =
aligned_pitch > Common::AlignUp(mip_width, texel_width) ? aligned_pitch : 0;
}
for (uint32_t slice_index = 0; slice_index < mip_depth; ++slice_index) {
region.bufferOffset = layout.mips[mip_level].offset + slice_index * layout.slice_stride;
region.imageSubresource.baseArrayLayer = volume_texture ? 0u : slice_index;
region.imageOffset.z = volume_texture ? static_cast<int>(slice_index) : 0;
regions.push_back(region);
}
@@ -323,17 +287,9 @@ std::vector<vk::BufferImageCopy> TextureBuildImageCopies(const TextureUploadLayo
return regions;
}
static bool SetGpuTileSize(uint64_t offset, uint64_t length, uint64_t capacity, uint64_t& size) {
if (offset > capacity || length > capacity - offset) {
return false;
}
size = length;
return true;
}
bool TextureBuildGpuTileInfos(uint64_t tiled_size, const std::vector<vk::BufferImageCopy>& regions,
const TextureUploadLayout& layout, uint32_t levels,
std::vector<GpuTileInfo>& out_infos) {
std::vector<GpuTileInfo>& out_tile_infos) {
const auto& description = layout.surface.description;
const bool volume_texture = description.dimension == TileSurfaceDimension::Dim3D;
const auto depth = volume_texture ? description.depth : description.layers;
@@ -351,111 +307,69 @@ bool TextureBuildGpuTileInfos(uint64_t tiled_size, const std::vector<vk::BufferI
return false;
}
std::vector<GpuTileInfo> infos;
infos.reserve(regions.size());
if (volume_texture) {
size_t region_base = 0;
for (uint32_t level = 0; level < levels; ++level) {
const uint32_t mip_depth = GetTextureLevelDepth(depth, level, true);
const bool tail = level >= surface.first_tail_level;
const uint64_t linear_stride = layout.slice_stride;
for (uint32_t z = 0; z < mip_depth; z += block.block_depth) {
const uint32_t copy_depth = std::min(block.block_depth, mip_depth - z);
const auto& region = regions[region_base + z];
const auto pitch =
region.bufferRowLength != 0 ? region.bufferRowLength : region.imageExtent.width;
const auto logical_height = region.bufferImageHeight != 0
? region.bufferImageHeight
: region.imageExtent.height;
GpuTileInfo info {};
info.family = block.family;
info.bytes_per_element = block.bytes_per_element;
info.linear_offset = region.bufferOffset;
info.tiled_offset =
static_cast<uint64_t>(z / block.block_depth) * surface.block_slice_size +
surface.mips[level].offset;
const uint64_t linear_span =
static_cast<uint64_t>(copy_depth - 1u) * linear_stride +
layout.mips[level].size;
if (!SetGpuTileSize(info.linear_offset, linear_span, UINT64_MAX,
info.linear_size) ||
!SetGpuTileSize(info.tiled_offset, surface.mips[level].size, tiled_size,
info.tiled_size)) {
return false;
}
info.linear_slice_stride = linear_stride;
info.width = std::max((region.imageExtent.width + texture.texel_width - 1u) /
texture.texel_width,
1u);
info.height = std::max(
(logical_height + texture.texel_height - 1u) / texture.texel_height, 1u);
info.depth = copy_depth;
info.surface_z =
std::vector<GpuTileInfo> tile_infos;
tile_infos.reserve(regions.size());
const auto slices_per_copy = volume_texture ? block.block_depth : 1u;
size_t region_base = 0;
for (uint32_t mip_level = 0; mip_level < levels; ++mip_level) {
const auto& mip = surface.mips[mip_level];
const auto mip_depth = GetTextureLevelDepth(depth, mip_level, volume_texture);
for (uint32_t slice_index = 0; slice_index < mip_depth; slice_index += slices_per_copy) {
const auto& region = regions[region_base + slice_index];
GpuTileInfo tile_info {};
tile_info.family = block.family;
tile_info.bytes_per_element = block.bytes_per_element;
tile_info.linear_offset = region.bufferOffset;
tile_info.linear_size = layout.mips[mip_level].size;
tile_info.tiled_size = mip.size;
if (volume_texture) {
tile_info.depth = std::min(slices_per_copy, mip_depth - slice_index);
tile_info.linear_slice_stride = layout.slice_stride;
tile_info.linear_size += (tile_info.depth - 1u) * layout.slice_stride;
tile_info.tiled_offset =
mip.offset +
static_cast<uint64_t>(slice_index / slices_per_copy) * surface.block_slice_size;
tile_info.surface_z =
block.block_depth == 1 ? static_cast<uint32_t>(region.imageOffset.z) : 0;
info.pitch = std::max((pitch + texture.texel_width - 1u) / texture.texel_width, 1u);
info.tail_x = tail ? surface.mips[level].tail_x : 0;
info.tail_y = tail ? surface.mips[level].tail_y : 0;
info.tail = tail;
info.tiled_width = surface.mips[level].padded_width;
info.tiled_height = surface.mips[level].padded_height;
infos.push_back(info);
}
region_base += mip_depth;
}
} else {
size_t region_index = 0;
for (uint32_t level = 0; level < levels; level++) {
const auto& level_size = layout.mips[level];
const auto& mip = surface.mips[level];
const bool tail = level >= surface.first_tail_level;
const auto level_depth = GetTextureLevelDepth(depth, level, false);
for (uint32_t z = 0; z < level_depth; z++) {
const auto& region = regions[region_index++];
const auto pitch =
region.bufferRowLength != 0 ? region.bufferRowLength : region.imageExtent.width;
const auto logical_height = region.bufferImageHeight != 0
? region.bufferImageHeight
: region.imageExtent.height;
GpuTileInfo info {};
info.family = block.family;
info.bytes_per_element = block.bytes_per_element;
info.linear_offset = region.bufferOffset;
} else {
const auto source_stride = layout.source_slice_stride != 0
? layout.source_slice_stride
: surface.block_slice_size;
if (z > (UINT64_MAX - mip.offset) / source_stride) {
if (slice_index > (UINT64_MAX - mip.offset) / source_stride) {
return false;
}
info.tiled_offset = mip.offset + static_cast<uint64_t>(z) * source_stride;
if (!SetGpuTileSize(info.linear_offset, level_size.size, UINT64_MAX,
info.linear_size) ||
!SetGpuTileSize(info.tiled_offset, mip.size, tiled_size, info.tiled_size)) {
return false;
}
info.width = std::max((region.imageExtent.width + texture.texel_width - 1u) /
texture.texel_width,
1u);
info.height = std::max(
(logical_height + texture.texel_height - 1u) / texture.texel_height, 1u);
info.surface_z = block.family == TileBlockFamily::RenderTarget64KB ||
block.family == TileBlockFamily::Depth64KB
? region.imageSubresource.baseArrayLayer
: 0;
info.pitch = std::max((pitch + texture.texel_width - 1u) / texture.texel_width, 1u);
info.tail = tail;
info.tail_x = tail ? mip.tail_x : 0;
info.tail_y = tail ? mip.tail_y : 0;
info.tiled_width = mip.padded_width;
info.tiled_height = mip.padded_height;
infos.push_back(info);
tile_info.tiled_offset =
mip.offset + static_cast<uint64_t>(slice_index) * source_stride;
tile_info.surface_z = block.family == TileBlockFamily::RenderTarget64KB ||
block.family == TileBlockFamily::Depth64KB
? region.imageSubresource.baseArrayLayer
: 0;
}
if (!FitsBufferRange(tile_info.linear_offset, tile_info.linear_size, UINT64_MAX) ||
!FitsBufferRange(tile_info.tiled_offset, tile_info.tiled_size, tiled_size)) {
return false;
}
const auto row_length =
region.bufferRowLength != 0 ? region.bufferRowLength : region.imageExtent.width;
const auto image_height = region.bufferImageHeight != 0 ? region.bufferImageHeight
: region.imageExtent.height;
tile_info.width = std::max(
(region.imageExtent.width + texture.texel_width - 1u) / texture.texel_width, 1u);
tile_info.height =
std::max((image_height + texture.texel_height - 1u) / texture.texel_height, 1u);
tile_info.pitch =
std::max((row_length + texture.texel_width - 1u) / texture.texel_width, 1u);
tile_info.tail = mip_level >= surface.first_tail_level;
tile_info.tail_x = tile_info.tail ? mip.tail_x : 0;
tile_info.tail_y = tile_info.tail ? mip.tail_y : 0;
tile_info.tiled_width = mip.padded_width;
tile_info.tiled_height = mip.padded_height;
tile_infos.push_back(tile_info);
}
region_base += mip_depth;
}
if (infos.empty()) {
return false;
}
out_infos = std::move(infos);
out_tile_infos = std::move(tile_infos);
return true;
}
@@ -1,9 +1,7 @@
#ifndef EMULATOR_INCLUDE_EMULATOR_GRAPHICS_HOST_GPU_RENDERER_IMAGE_TEXTURECOMMON_H_
#define EMULATOR_INCLUDE_EMULATOR_GRAPHICS_HOST_GPU_RENDERER_IMAGE_TEXTURECOMMON_H_
#include "common/abi.h"
#include "common/common.h"
#include "graphics/guest_gpu/gpu_format.h"
#include "graphics/guest_gpu/gpu_defs.h"
#include "graphics/guest_gpu/tile.h"
#include "graphics/host_gpu/vulkanCommon.h"
@@ -20,12 +18,9 @@ struct RenderTargetFormatInfo {
};
struct SurfaceFormatInfo {
explicit constexpr SurfaceFormatInfo(vk::Format format,
Prospero::BufferFormat conversion_format)
: vk_format(format), conversion_format(conversion_format) {}
vk::Format vk_format;
Prospero::BufferFormat conversion_format;
vk::Format vk_format;
Prospero::BufferFormat conversion_format;
Prospero::ColorComponentMapping host_to_storage;
};
struct TextureUploadMipLayout {
@@ -43,20 +38,21 @@ struct TextureUploadLayout {
TextureUploadMipLayout mips[16] = {};
};
vk::ComponentMapping TextureGetComponentMapping(uint32_t swizzle);
vk::ComponentMapping TextureGetComponentMapping(uint32_t swizzle,
Prospero::ColorComponentMapping host_to_storage);
SurfaceFormatInfo TextureGetSurfaceFormatInfo(Prospero::BufferFormat format);
RenderTargetFormatInfo TextureGetRenderTargetFormat(Prospero::ChannelLayout layout,
Prospero::ChannelType type,
Prospero::ChannelOrder order);
TextureUploadLayout TextureCalcUploadLayout(Prospero::BufferFormat format, uint32_t width,
uint32_t height, uint32_t levels, uint32_t depth,
Prospero::TileMode tile, uint64_t upload_size,
Prospero::TileMode tile_mode, uint64_t upload_size,
bool allow_depth_tile, bool volume_texture,
const char* owner);
std::vector<vk::BufferImageCopy> TextureBuildImageCopies(const TextureUploadLayout& layout);
bool TextureBuildGpuTileInfos(uint64_t tiled_size, const std::vector<vk::BufferImageCopy>& regions,
const TextureUploadLayout& layout, uint32_t levels,
std::vector<GpuTileInfo>& infos);
std::vector<GpuTileInfo>& out_tile_infos);
} // namespace Libs::Graphics
+17 -30
View File
@@ -1,5 +1,6 @@
#include "graphics/host_gpu/renderer/image/tiler.h"
#include "common/alignment.h"
#include "common/assert.h"
#include "gpu_tiler_shaders/gpu_tiler_demote_d16_spv.h"
#include "gpu_tiler_shaders/gpu_tiler_depth_spv.h"
@@ -221,7 +222,7 @@ void TileManager::Prepare(bool tile, uint64_t tiled_capacity, uint64_t linear_ca
const uint64_t uniform_alignment =
std::max<uint64_t>(limits.minUniformBufferOffsetAlignment, 1);
const uint64_t stride = (sizeof(Push) + uniform_alignment - 1) & ~(uniform_alignment - 1);
const uint64_t stride = Common::AlignUp<uint64_t>(sizeof(Push), uniform_alignment);
EXIT_NOT_IMPLEMENTED(dispatches.size() > UINT64_MAX / stride);
const uint64_t bytes = dispatches.size() * stride;
auto [mapped, offset] = m_stream_buffer.Map(bytes, uniform_alignment);
@@ -259,12 +260,8 @@ vk::Pipeline TileManager::GetPipeline(uint32_t slot) {
const uint32_t values[] {1u << element_index, direction_index};
const vk::SpecializationMapEntry entries[] {{0, 0, 4}, {1, 4, 4}};
const vk::SpecializationInfo specialization {2, entries, sizeof(values), values};
vk::ShaderModuleCreateInfo module_info {};
module_info.codeSize = shaders[family_index].words * sizeof(uint32_t);
module_info.pCode = shaders[family_index].code;
vk::ShaderModule module = nullptr;
RequireVulkanSuccess(m_graphics.device.createShaderModule(&module_info, nullptr, &module),
"create TileManager shader module");
const auto module =
CompileSPV({shaders[family_index].code, shaders[family_index].words}, m_graphics.device);
vk::PipelineShaderStageCreateInfo stage {};
stage.stage = vk::ShaderStageFlagBits::eCompute;
stage.module = module;
@@ -287,12 +284,12 @@ void TileManager::Record(vk::Buffer source, uint64_t source_offset,
const auto& limits = m_graphics.GetPhysicalDeviceProperties().limits;
const uint64_t descriptor_alignment =
std::max<uint64_t>(limits.minStorageBufferOffsetAlignment, 4);
const uint64_t source_descriptor_offset = source_offset & ~(descriptor_alignment - 1);
const uint64_t target_descriptor_offset = target_offset & ~(descriptor_alignment - 1);
const uint64_t source_descriptor_offset = Common::AlignDown(source_offset, descriptor_alignment);
const uint64_t target_descriptor_offset = Common::AlignDown(target_offset, descriptor_alignment);
const uint64_t source_base = source_offset - source_descriptor_offset;
const uint64_t target_base = target_offset - target_descriptor_offset;
const uint64_t source_range = (source_base + source_capacity + 3u) & ~uint64_t {3};
const uint64_t target_range = (target_base + target_capacity + 3u) & ~uint64_t {3};
const uint64_t source_range = Common::AlignUp(source_base + source_capacity, 4);
const uint64_t target_range = Common::AlignUp(target_base + target_capacity, 4);
EXIT_NOT_IMPLEMENTED(source_range > limits.maxStorageBufferRange ||
target_range > limits.maxStorageBufferRange || target_offset % 4 != 0 ||
target_capacity % 4 != 0);
@@ -372,7 +369,7 @@ TileManager::Result TileManager::Detile(vk::Buffer tiled, uint64_t tiled_offset,
const uint64_t source_base = tiled_offset & (descriptor_alignment - 1);
std::vector<Dispatch> dispatches;
Prepare(false, tiled_capacity, linear_capacity, infos, source_base, 0, dispatches);
auto scratch = AllocateScratch((linear_capacity + 3u) & ~uint64_t {3});
auto scratch = AllocateScratch(Common::AlignUp(linear_capacity, 4));
DeferDestroy(scratch);
Record(tiled, tiled_offset, tiled_capacity, scratch.buffer, 0, scratch.size, dispatches,
true);
@@ -406,7 +403,7 @@ void TileManager::TileImage(Image& image, std::span<const vk::BufferImageCopy> r
// Reserve all stream parameters before creating a scheduler-lived scratch dependency:
// StreamBuffer::Map is allowed to submit the current tick when it wraps.
Prepare(true, tiled_capacity, linear_capacity, infos, 0, target_base, dispatches);
auto linear = AllocateScratch((linear_capacity + 3u) & ~uint64_t {3});
auto linear = AllocateScratch(Common::AlignUp(linear_capacity, 4));
DeferDestroy(linear);
image.Download(regions, linear.buffer, 0, linear.size);
Result source {linear.buffer, 0, linear.size};
@@ -418,7 +415,7 @@ void TileManager::TileImage(Image& image, std::span<const vk::BufferImageCopy> r
}
TileManager::Result TileManager::GetScratchBuffer(uint64_t size) {
auto scratch = AllocateScratch((size + 3u) & ~uint64_t {3});
auto scratch = AllocateScratch(Common::AlignUp(size, 4));
DeferDestroy(scratch);
return {scratch.buffer, 0, scratch.size};
}
@@ -426,11 +423,11 @@ TileManager::Result TileManager::GetScratchBuffer(uint64_t size) {
TileManager::StorageBinding TileManager::BindStorage(Result buffer, uint64_t size) const {
const auto& limits = m_graphics.GetPhysicalDeviceProperties().limits;
const auto alignment = std::max<uint64_t>(limits.minStorageBufferOffsetAlignment, 4);
const auto descriptor_offset = buffer.offset - buffer.offset % alignment;
const auto descriptor_offset = Common::AlignDown(buffer.offset, alignment);
const auto base = buffer.offset - descriptor_offset;
EXIT_IF(buffer.buffer == nullptr || size == 0 || buffer.size < size || base > UINT32_MAX ||
size > UINT64_MAX - base || base + size > UINT64_MAX - 3);
const auto range = (base + size + 3) & ~uint64_t {3};
const auto range = Common::AlignUp(base + size, 4);
EXIT_IF(range > limits.maxStorageBufferRange || range > UINT32_MAX);
return {{buffer.buffer, descriptor_offset, range}, static_cast<uint32_t>(base)};
}
@@ -471,12 +468,7 @@ void TileManager::ConvertD16(Result source, Result target, D16Direction directio
code = GPU_TILER_DEMOTE_D16_SPV;
words = std::size(GPU_TILER_DEMOTE_D16_SPV);
}
vk::ShaderModuleCreateInfo module_info {};
module_info.codeSize = words * sizeof(uint32_t);
module_info.pCode = code;
vk::ShaderModule module = nullptr;
RequireVulkanSuccess(m_graphics.device.createShaderModule(&module_info, nullptr, &module),
"create D16 conversion shader module");
const auto module = CompileSPV({code, words}, m_graphics.device);
vk::PipelineShaderStageCreateInfo stage {};
stage.stage = vk::ShaderStageFlagBits::eCompute;
stage.module = module;
@@ -512,8 +504,8 @@ void TileManager::ConvertD16(Result source, Result target, D16Direction directio
const auto target_required = required(layout.height, layout.layers, layout.target_row_stride,
layout.target_slice_stride, target_active);
EXIT_IF(source_required > UINT64_MAX - 3 || target_required > UINT64_MAX - 3);
const auto source_barrier_size = (source_required + 3) & ~uint64_t {3};
const auto target_barrier_size = (target_required + 3) & ~uint64_t {3};
const auto source_barrier_size = Common::AlignUp(source_required, 4);
const auto target_barrier_size = Common::AlignUp(target_required, 4);
EXIT_IF(source.size < source_barrier_size || target.size < target_barrier_size);
m_scheduler.EndRendering();
@@ -610,12 +602,7 @@ void TileManager::ConvertD16(Result source, Result target, D16Direction directio
void TileManager::SwapBgra16(Result input, Result output, uint32_t pixels) {
if (m_swap_bgra16 == nullptr) {
vk::ShaderModuleCreateInfo module_info {};
module_info.codeSize = std::size(GPU_TILER_SWAP_BGRA16_SPV) * sizeof(uint32_t);
module_info.pCode = GPU_TILER_SWAP_BGRA16_SPV;
vk::ShaderModule module = nullptr;
RequireVulkanSuccess(m_graphics.device.createShaderModule(&module_info, nullptr, &module),
"create BGRA16 swap shader module");
const auto module = CompileSPV(GPU_TILER_SWAP_BGRA16_SPV, m_graphics.device);
vk::PipelineShaderStageCreateInfo stage {};
stage.stage = vk::ShaderStageFlagBits::eCompute;
stage.module = module;
@@ -20,7 +20,7 @@ constexpr std::array DescriptorPoolSizes = {
DescriptorHeap::DescriptorHeap(GraphicContext& graphics, MasterSemaphore& master_semaphore)
: m_graphics(graphics), m_master_semaphore(master_semaphore) {
CreatePool();
CreateDescriptorPool();
}
DescriptorHeap::~DescriptorHeap() {
@@ -49,7 +49,7 @@ vk::DescriptorSet DescriptorHeap::Commit(vk::DescriptorSetLayout layout) {
m_pending_pools.pop_front();
EXIT_IF(m_graphics.device.resetDescriptorPool(m_current_pool, {}) != vk::Result::eSuccess);
} else {
CreatePool();
CreateDescriptorPool();
}
m_sets.clear();
@@ -82,7 +82,7 @@ bool DescriptorHeap::Allocate(vk::DescriptorSetLayout layout, Batch& batch) {
}
}
void DescriptorHeap::CreatePool() {
void DescriptorHeap::CreateDescriptorPool() {
vk::DescriptorPoolCreateInfo create {};
create.maxSets = DescriptorHeapCount;
create.poolSizeCount = static_cast<uint32_t>(DescriptorPoolSizes.size());
@@ -31,7 +31,7 @@ private:
};
[[nodiscard]] bool Allocate(vk::DescriptorSetLayout layout, Batch& batch);
void CreatePool();
void CreateDescriptorPool();
GraphicContext& m_graphics;
MasterSemaphore& m_master_semaphore;
@@ -1,5 +1,6 @@
#include "graphics/host_gpu/renderer/pipeline/descriptors.h"
#include "common/alignment.h"
#include "common/assert.h"
#include "common/common.h"
#include "common/file.h"
@@ -14,6 +15,7 @@
#include "graphics/guest_gpu/tile.h"
#include "graphics/host_gpu/graphicContext.h"
#include "graphics/host_gpu/hostMemory.h"
#include "graphics/host_gpu/renderer/colorRenderTarget.h"
#include "graphics/host_gpu/renderer/debug.h"
#include "graphics/host_gpu/renderer/image/imageView.h"
#include "graphics/host_gpu/renderer/image/textureCommon.h"
@@ -92,22 +94,15 @@ static const char* ShaderStageResourceName(ShaderType stage) {
switch (stage) {
case ShaderType::Vertex: return "Vertex";
case ShaderType::Mesh: return "Mesh";
case ShaderType::Local: return "Local";
case ShaderType::TessellationControl: return "Hull";
case ShaderType::TessellationEvaluation: return "Domain";
case ShaderType::Pixel: return "Pixel";
case ShaderType::Compute: return "Compute";
default: return "Unknown";
}
}
static vk::ShaderStageFlags NativeShaderStage(ShaderType stage) {
switch (stage) {
case ShaderType::Vertex: return vk::ShaderStageFlagBits::eVertex;
case ShaderType::Mesh: return vk::ShaderStageFlagBits::eMeshEXT;
case ShaderType::Pixel: return vk::ShaderStageFlagBits::eFragment;
case ShaderType::Compute: return vk::ShaderStageFlagBits::eCompute;
default: EXIT("unknown native shader stage\n");
}
}
static Prospero::ImageType TextureType(const ShaderTextureResource& descriptor) {
const auto type = descriptor.Type();
return type == Prospero::ImageType::kCube ? Prospero::ImageType::kColor2DArray : type;
@@ -145,7 +140,7 @@ NativeStorageBuffer(RenderContext& context, const PreparedBindings::BufferSource
}
auto [buffer, offset] = context.GetBufferCache().ObtainBuffer(address, size, resource.written,
resource.formatted, id);
const auto aligned_offset = offset - offset % alignment;
const auto aligned_offset = Common::AlignDown(offset, alignment);
const auto adjustment = offset - aligned_offset;
const auto max_range = graphics.GetPhysicalDeviceProperties().limits.maxStorageBufferRange;
if (adjustment % sizeof(uint32_t) != 0 || adjustment >= 256 || size > max_range - adjustment) {
@@ -153,7 +148,7 @@ NativeStorageBuffer(RenderContext& context, const PreparedBindings::BufferSource
}
buffer_offset = static_cast<uint32_t>(adjustment);
const vk::DescriptorBufferInfo result {buffer->Handle(), aligned_offset, size + adjustment};
if (resource.formatted && resource.written) {
if (resource.written) {
context.GetTextureCache().InvalidateMemoryFromGPU(address, size);
}
const char* access = "Read";
@@ -169,34 +164,6 @@ NativeStorageBuffer(RenderContext& context, const PreparedBindings::BufferSource
return result;
}
static bool IsSupportedSampledColorResource(const ShaderRecompiler::IR::ImageResource& resource) {
bool supported_dimension = false;
switch (resource.dimension) {
case ShaderRecompiler::Decoder::ImageDimension::Dim1D:
case ShaderRecompiler::Decoder::ImageDimension::Dim1DArray:
case ShaderRecompiler::Decoder::ImageDimension::Dim2D:
case ShaderRecompiler::Decoder::ImageDimension::Dim2DArray:
case ShaderRecompiler::Decoder::ImageDimension::Dim2DMsaa:
case ShaderRecompiler::Decoder::ImageDimension::Dim2DMsaaArray:
supported_dimension = true;
break;
default: break;
}
return resource.resource_class == ShaderRecompiler::IR::ImageResourceClass::Sampled &&
resource.numeric_class != Prospero::TextureNumericClass::Unsupported &&
supported_dimension && resource.mip_mode == ShaderRecompiler::IR::ImageMipMode::None &&
resource.read && !resource.written && !resource.atomic && !resource.depth_compare;
}
bool IsSupportedSampledVideoOutView(const ShaderRecompiler::IR::ImageResource& resource,
const ShaderTextureResource& descriptor, const Image& image) {
return image.usage.video_out && image.info.resources.layers == 1 &&
IsSupportedSampledColorResource(resource) &&
resource.dimension == ShaderRecompiler::Decoder::ImageDimension::Dim2D &&
descriptor.Type() == Prospero::ImageType::kColor2D && descriptor.Depth() == 0 &&
descriptor.BaseArray5() == 0;
}
bool IsSupportedDepthTextureEncoding(const ShaderTextureResource& descriptor, bool r128) {
constexpr uint32_t field1_reserved_mask = 0x200fff00u;
constexpr uint32_t field2_reserved_mask = 0xf0003000u;
@@ -326,13 +293,8 @@ static bool IsSupportedStorageTextureDescriptor(const ShaderRecompiler::IR::Imag
const bool supported_swizzle =
IsValidImageSwizzle(swizzle) &&
(swizzle == DstSel(4, 5, 6, 7) || !resource.read || resource.atomic);
const auto max_mip = resource.r128 ? descriptor.LastLevel() : descriptor.MaxMip();
const auto view_last_level =
resource.mip_mode == ShaderRecompiler::IR::ImageMipMode::DynamicStorage
? descriptor.LastLevel()
: std::min(descriptor.LastLevel(), max_mip);
return (is_1d || is_1d_array || is_2d || is_2d_array || is_3d) && supported_tile &&
descriptor.BaseLevel() <= view_last_level && view_last_level <= max_mip &&
descriptor.BaseLevel() <= descriptor.LastLevel() &&
descriptor.MinLod() == 0 && supported_swizzle && descriptor.BCSwizzle() == 0 &&
!descriptor.MsaaDepth();
}
@@ -404,30 +366,22 @@ void ValidateStorageTexture(const ShaderRecompiler::IR::ImageResource& resource,
descriptor.fields[5], descriptor.fields[6], descriptor.fields[7]);
}
struct NullImageSpec {
vk::Format format;
Prospero::BufferFormat guest_format;
};
static NullImageSpec NullTextureSpec(const ShaderRecompiler::IR::ImageResource& resource) {
switch (resource.numeric_class) {
case Prospero::TextureNumericClass::Float:
return {vk::Format::eR32Sfloat, Prospero::BufferFormat::k32Float};
case Prospero::TextureNumericClass::Uint:
return {vk::Format::eR32Uint, Prospero::BufferFormat::k32UInt};
case Prospero::TextureNumericClass::Sint:
return {vk::Format::eR32Sint, Prospero::BufferFormat::k32SInt};
case Prospero::TextureNumericClass::Unsupported: break;
}
EXIT("null image has unsupported numeric class\n");
}
static TextureCache::ImageDesc NullTextureDesc(const ShaderRecompiler::IR::ImageResource& resource,
TextureCache::BindingType binding) {
const auto spec = NullTextureSpec(resource);
TextureCache::ImageDesc desc {};
desc.info.pixel_format = spec.format;
desc.info.guest_format = spec.guest_format;
switch (resource.numeric_class) {
case Prospero::TextureNumericClass::Float:
desc.info.guest_format = Prospero::BufferFormat::k32Float;
break;
case Prospero::TextureNumericClass::Uint:
desc.info.guest_format = Prospero::BufferFormat::k32UInt;
break;
case Prospero::TextureNumericClass::Sint:
desc.info.guest_format = Prospero::BufferFormat::k32SInt;
break;
default: EXIT("null image has unsupported numeric class\n");
}
desc.info.pixel_format = VulkanFormat(desc.info.guest_format);
desc.info.type = Prospero::ImageType::kColor2D;
desc.info.extent = {1, 1, 1};
desc.info.resources = {1, 1};
@@ -491,17 +445,26 @@ static void PopulateTextureMipLayout(ImageInfo& info) {
static ImageViewInfo TextureViewInfo(const ShaderRecompiler::IR::ImageResource& resource,
const ShaderTextureResource& descriptor, vk::Format format,
bool shader_conversion, bool storage, uint32_t view_levels,
uint32_t image_layers) {
const SurfaceFormatInfo& surface_format, bool storage,
uint32_t view_levels, uint32_t image_layers) {
ImageViewInfo view {};
view.format = format;
view.aspect = vk::ImageAspectFlagBits::eColor;
view.base_level = descriptor.BaseLevel();
view.level_count = view_levels;
view.usage = storage ? vk::ImageUsageFlagBits::eStorage : vk::ImageUsageFlagBits::eSampled;
view.mapping = storage || shader_conversion
? vk::ComponentMapping {}
: TextureGetComponentMapping(descriptor.DstSelXYZW());
if (descriptor.MinLod() > descriptor.LastLevel() * 256u) {
EXIT("texture minimum LOD exceeds last mip level: min_lod=%u last_level=%u\n",
descriptor.MinLod(), descriptor.LastLevel());
}
const auto base_lod = view.base_level * 256u;
if (descriptor.MinLod() > base_lod) {
view.min_lod = descriptor.MinLod() - base_lod;
}
view.usage = storage ? vk::ImageUsageFlagBits::eStorage : vk::ImageUsageFlagBits::eSampled;
view.mapping =
storage || surface_format.conversion_format != Prospero::BufferFormat::kInvalid
? vk::ComponentMapping {}
: TextureGetComponentMapping(descriptor.DstSelXYZW(), surface_format.host_to_storage);
switch (resource.dimension) {
case ShaderRecompiler::Decoder::ImageDimension::Dim1D:
view.type = vk::ImageViewType::e1D;
@@ -547,6 +510,21 @@ static ImageViewInfo TextureViewInfo(const ShaderRecompiler::IR::ImageResource&
return view;
}
static bool TextureViewPreservesMipLayout(const TileSurfaceDescription& description,
uint32_t view_levels) {
TileSurfaceLayout physical {};
TileSurfaceLayout view {};
auto view_description = description;
view_description.levels = view_levels;
return TileGetTiledTextureLayout(description, physical) &&
TileGetTiledTextureLayout(view_description, view) &&
physical.first_tail_level == view.first_tail_level &&
physical.block_slice_size == view.block_slice_size &&
physical.total_size == view.total_size &&
std::equal(std::begin(physical.mips), std::begin(physical.mips) + description.levels,
std::begin(view.mips));
}
TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageResource& resource,
const ShaderRecompiler::IR::DescriptorValue& value) {
auto descriptor = DecodeNativeDescriptor<ShaderTextureResource>(value);
@@ -563,24 +541,22 @@ TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageR
return {id, nullptr, std::move(desc)};
}
const auto address = descriptor.Base40();
const auto width = static_cast<uint32_t>(descriptor.Width5()) + 1u;
const auto height = static_cast<uint32_t>(descriptor.Height5()) + 1u;
const auto base_level = descriptor.BaseLevel();
const auto last_level = descriptor.LastLevel();
const auto type = TextureType(descriptor);
const bool multisampled = IsMultisampledTexture(type);
const auto max_mip = resource.r128 ? last_level : descriptor.MaxMip();
const auto levels = multisampled ? 1u : static_cast<uint32_t>(max_mip) + 1u;
const bool dynamic_storage =
storage && resource.mip_mode == ShaderRecompiler::IR::ImageMipMode::DynamicStorage;
const auto view_last_level =
!multisampled && !dynamic_storage ? std::min(last_level, max_mip) : last_level;
const auto address = descriptor.Base40();
const auto width = static_cast<uint32_t>(descriptor.Width5()) + 1u;
const auto height = static_cast<uint32_t>(descriptor.Height5()) + 1u;
const auto base_level = descriptor.BaseLevel();
const auto last_level = descriptor.LastLevel();
const auto type = TextureType(descriptor);
const bool multisampled = IsMultisampledTexture(type);
const auto max_mip = resource.r128 ? last_level : descriptor.MaxMip();
const auto physical_levels = multisampled ? 1u : static_cast<uint32_t>(max_mip) + 1u;
const auto levels =
multisampled ? 1u : std::max(physical_levels, static_cast<uint32_t>(last_level) + 1u);
const auto tile = descriptor.TileMode();
const bool depth_tile = tile == Prospero::TileMode::kDepth;
const bool msaa_tile = depth_tile || tile == Prospero::TileMode::kRenderTarget;
const bool msaa_array = type == Prospero::ImageType::kColor2DMsaaArray;
if ((!multisampled && (base_level > view_last_level || view_last_level >= levels)) ||
if ((!multisampled && base_level > last_level) ||
(multisampled &&
(base_level != 0 || last_level == 0 || last_level > 3 || max_mip != last_level ||
!msaa_tile || (descriptor.MsaaDepth() && !depth_tile) ||
@@ -599,7 +575,7 @@ TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageR
}
const auto samples = multisampled ? 1u << last_level : 1u;
const auto view_levels =
multisampled ? 1u : static_cast<uint32_t>(view_last_level - base_level) + 1u;
multisampled ? 1u : static_cast<uint32_t>(last_level - base_level) + 1u;
const auto depth = static_cast<uint32_t>(descriptor.Depth()) + 1u;
const auto format = descriptor.Format();
const auto surface_format = TextureGetSurfaceFormatInfo(format);
@@ -618,7 +594,19 @@ TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageR
type == Prospero::ImageType::kColor2DArray ||
type == Prospero::ImageType::kColor2DMsaaArray;
const auto image_layers = layered ? depth : 1u;
uint32_t pitch = 0;
if (levels > physical_levels) {
const TileSurfaceDescription physical {
format, tile, volume ? TileSurfaceDimension::Dim3D : TileSurfaceDimension::Dim2D,
width, height, volume ? depth : 1u, physical_levels, image_layers};
// Texture mip views take precedence over the resource count, but must keep its storage layout.
if (!TextureViewPreservesMipLayout(physical, levels)) {
EXIT("unsupported texture mip view changes physical layout: base=%u last=%u max=%u "
"extent=%ux%ux%u tile=%u\n",
base_level, last_level, max_mip, width, height, depth,
static_cast<uint32_t>(tile));
}
}
uint32_t pitch = 0;
TileSizeAlign size {};
if (multisampled) {
const auto bytes = Prospero::NumBytesPerElement(format);
@@ -631,8 +619,8 @@ TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageR
size.size *= image_layers;
} else {
pitch = TileGetTexturePitch(format, width, tile);
TileGetTextureTotalSize(format, width, height, volume ? depth : image_layers, levels, tile,
volume, size);
TileGetTextureTotalSize(format, width, height, volume ? depth : image_layers,
physical_levels, tile, volume, size);
}
EXIT_NOT_IMPLEMENTED(size.size == 0 || size.align == 0 ||
(address & (static_cast<uint64_t>(size.align) - 1u)) != 0);
@@ -669,7 +657,7 @@ TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageR
!desc.info.IsDepth()) {
TileSizeAlign metadata_size {};
(void)TileGetDccSize(width, height, volume ? depth : image_layers,
desc.info.bytes_per_block, levels, tile, metadata_size,
desc.info.bytes_per_block, physical_levels, tile, metadata_size,
std::countr_zero(samples));
desc.info.metadata.kind = ImageMetadataKind::Dcc;
desc.info.metadata.range = {descriptor.MetaAddr() << 8u, metadata_size.size};
@@ -680,7 +668,7 @@ TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageR
} else {
PopulateTextureMipLayout(desc.info);
}
desc.view_info = TextureViewInfo(resource, descriptor, view_format, shader_conversion, storage,
desc.view_info = TextureViewInfo(resource, descriptor, view_format, surface_format, storage,
view_levels, desc.info.resources.layers);
desc.type = storage ? TextureCache::BindingType::Storage : TextureCache::BindingType::Texture;
@@ -728,13 +716,6 @@ static vk::DescriptorBufferInfo NativeUpload(RenderContext& context,
return {buffer.Handle(), offset, data.size_bytes()};
}
void RenderExecutor::TrackImageBinding(ImageId id) {
EXIT_IF(m_context.GetTextureCache().m_slot_images.try_get(id) == nullptr);
if (std::ranges::find(m_bound_images, id) == m_bound_images.end()) {
m_bound_images.push_back(id);
}
}
void RenderExecutor::BindImage(ImageId id, bool storage) {
auto& image = m_context.GetTextureCache().GetImage(id);
if (image.info.data.Empty()) {
@@ -745,13 +726,13 @@ void RenderExecutor::BindImage(ImageId id, bool storage) {
}
image.binding.is_bound = true;
image.binding.shader_write |= storage;
TrackImageBinding(id);
m_bound_images.push_back(id);
}
void RenderExecutor::BindRenderTarget(ImageId id) {
auto& image = m_context.GetTextureCache().GetImage(id);
image.binding.is_target = true;
TrackImageBinding(id);
m_bound_images.push_back(id);
}
void RenderExecutor::ResetBindings() {
@@ -769,8 +750,7 @@ PreparedBindings RenderExecutor::PrepareBindings(const ShaderStageRuntime& runti
const auto& program = *runtime.program;
const auto& snapshot = runtime.resources;
PreparedBindings prepared;
prepared.program = runtime.program;
prepared.snapshot = &runtime.resources;
prepared.runtime = &runtime;
prepared.images.reserve(program.info.images.size());
for (uint32_t i = 0; i < program.info.images.size(); i++) {
auto binding = ResolveTexture(program.info.images[i], snapshot.images[i]);
@@ -795,9 +775,9 @@ PreparedBindings RenderExecutor::PrepareBindings(const ShaderStageRuntime& runti
void RenderExecutor::FindBuffers(PreparedBindings& prepared) {
KYTY_PROFILER_FUNCTION();
EXIT_IF(prepared.program == nullptr || prepared.snapshot == nullptr);
const auto& program = *prepared.program;
const auto& snapshot = *prepared.snapshot;
EXIT_IF(prepared.runtime == nullptr || !*prepared.runtime);
const auto& program = *prepared.runtime->program;
const auto& snapshot = prepared.runtime->resources;
auto& cache = m_context.GetBufferCache();
prepared.buffer_sources.clear();
@@ -805,10 +785,7 @@ void RenderExecutor::FindBuffers(PreparedBindings& prepared) {
for (uint32_t i = 0; i < program.info.buffers.size(); i++) {
auto descriptor = DecodeNativeDescriptor<ShaderBufferResource>(snapshot.buffers[i]);
const auto address = descriptor.Base48();
const auto stride = descriptor.Stride();
const auto records = descriptor.NumRecords();
// The descriptor has a 14-bit stride and 32-bit record count, so the product fits u64.
const auto requested_size = stride != 0 ? static_cast<uint64_t>(stride) * records : records;
const auto requested_size = descriptor.GetSize();
if (address == 0 || requested_size == 0) {
prepared.buffer_sources.push_back({});
continue;
@@ -820,9 +797,9 @@ void RenderExecutor::FindBuffers(PreparedBindings& prepared) {
void RenderExecutor::RebindBuffers(PreparedBindings& prepared) {
KYTY_PROFILER_FUNCTION();
EXIT_IF(prepared.program == nullptr || prepared.snapshot == nullptr);
const auto& program = *prepared.program;
const auto& snapshot = *prepared.snapshot;
EXIT_IF(prepared.runtime == nullptr || !*prepared.runtime);
const auto& program = *prepared.runtime->program;
const auto& snapshot = prepared.runtime->resources;
const auto& layout = program.bindings;
EXIT_IF(prepared.buffer_sources.size() != program.info.buffers.size());
@@ -855,9 +832,9 @@ void RenderExecutor::RebindBuffers(PreparedBindings& prepared) {
void RenderExecutor::RebindImages(PreparedBindings& prepared) {
KYTY_PROFILER_FUNCTION();
EXIT_IF(prepared.program == nullptr || prepared.snapshot == nullptr);
const auto& program = *prepared.program;
const auto& snapshot = *prepared.snapshot;
EXIT_IF(prepared.runtime == nullptr || !*prepared.runtime);
const auto& program = *prepared.runtime->program;
const auto& snapshot = prepared.runtime->resources;
auto& images = prepared.images;
EXIT_IF(images.size() != program.info.images.size());
auto& texture_cache = m_context.GetTextureCache();
@@ -902,32 +879,39 @@ void RenderExecutor::RebindImages(PreparedBindings& prepared) {
}
}
RenderExecutor::GraphicsBindings
RenderExecutor::PrepareGraphicsBindings(const ShaderStageRuntime& vertex,
const ShaderStageRuntime& pixel, bool pixel_active) {
GraphicsBindings bindings {
.vertex = PrepareBindings(vertex),
};
if (pixel_active) {
bindings.pixel.emplace(PrepareBindings(pixel));
void RenderExecutor::PrepareGraphicsBindings(std::span<PreparedBindings* const> stages,
std::span<RenderColorInfo> colors) {
bool uses_dma = false;
for (auto* stage: stages) {
FindBuffers(*stage);
uses_dma |= stage->runtime->program->info.uses_dma;
}
FindBuffers(bindings.vertex);
if (bindings.pixel) {
FindBuffers(*bindings.pixel);
if (uses_dma) {
m_context.PrepareBda();
}
if (bindings.vertex.program->info.uses_dma ||
(bindings.pixel && bindings.pixel->program->info.uses_dma)) {
m_context.GetGpuResources().PrepareBda();
for (auto* stage: stages) {
RebindImages(*stage);
}
RebindBuffers(bindings.vertex);
if (bindings.pixel) {
RebindBuffers(*bindings.pixel);
auto& cache = m_context.GetTextureCache();
for (auto& target: colors) {
EXIT_IF(!target.image_id);
const auto old_image = cache.m_slot_images.try_get(target.image_id);
if (old_image == nullptr || (!old_image->registered && !old_image->info.data.Empty()) ||
old_image->binding.needs_rebind) {
if (old_image != nullptr) {
old_image->binding = {};
}
target.desc.view_info.base_level = target.guest_mip_level;
target.desc.view_info.base_layer = target.guest_array_layer;
target.image_id = cache.FindImage(target.desc);
BindRenderTarget(target.image_id);
}
}
RebindImages(bindings.vertex);
if (bindings.pixel) {
RebindImages(*bindings.pixel);
// Discovery can read back PS5 metadata and submit the scheduler. Reserve draw buffers only
// after image identities are final; attachment layout transitions follow buffer alias copies.
for (auto* stage: stages) {
RebindBuffers(*stage);
}
return bindings;
}
void RenderExecutor::CommitBindings(CommandBuffer& buffer,
@@ -940,20 +924,21 @@ void RenderExecutor::CommitBindings(CommandBuffer& buffer,
size_t write_count = 0;
ShaderRecompiler::IR::PushData push_data;
bool has_push_data = false;
constexpr auto GraphicsStages = vk::ShaderStageFlagBits::eVertex |
vk::ShaderStageFlagBits::eMeshEXT |
vk::ShaderStageFlagBits::eFragment;
constexpr auto GraphicsStages =
vk::ShaderStageFlagBits::eVertex | vk::ShaderStageFlagBits::eMeshEXT |
vk::ShaderStageFlagBits::eTessellationControl |
vk::ShaderStageFlagBits::eTessellationEvaluation | vk::ShaderStageFlagBits::eFragment;
vk::ShaderStageFlags push_stages = pipeline_bind_point == vk::PipelineBindPoint::eGraphics
? vk::ShaderStageFlagBits::eFragment
: vk::ShaderStageFlags {};
for (const auto* prepared: prepared_bindings) {
EXIT_IF(prepared == nullptr || prepared->program == nullptr ||
prepared->snapshot == nullptr);
write_count += prepared->program->bindings.descriptors.size();
for (const auto& binding: prepared->program->bindings.descriptors) {
EXIT_IF(prepared == nullptr || prepared->runtime == nullptr || !*prepared->runtime);
const auto& program = *prepared->runtime->program;
write_count += program.bindings.descriptors.size();
for (const auto& binding: program.bindings.descriptors) {
descriptor_count += NativeDescriptorCount(binding);
}
const auto shader_stage = NativeShaderStage(prepared->program->stage);
const auto shader_stage = NativeShaderStage(program.stage);
push_stages |= shader_stage;
EXIT_IF((pipeline_bind_point == vk::PipelineBindPoint::eGraphics &&
(shader_stage & GraphicsStages) == vk::ShaderStageFlags {}) ||
@@ -968,7 +953,7 @@ void RenderExecutor::CommitBindings(CommandBuffer& buffer,
m_descriptor_writes.reserve(write_count);
for (auto* prepared: prepared_bindings) {
const auto& program = *prepared->program;
const auto& program = *prepared->runtime->program;
auto& descriptors = *prepared;
const auto shader_stage = NativeShaderStage(program.stage);
const auto shader_stages = ShaderPipelineStages(shader_stage);
@@ -16,9 +16,7 @@
namespace Libs::Graphics {
namespace ShaderRecompiler::IR {
struct ResourceSnapshot;
}
struct ShaderStageRuntime;
struct TextureBinding {
ImageId image_id;
@@ -35,8 +33,8 @@ struct PreparedBindings {
BufferId id;
};
const ShaderRecompiler::IR::CompiledShaderInfo* program = nullptr;
const ShaderRecompiler::IR::ResourceSnapshot* snapshot = nullptr;
// The draw owns the immutable compiled-program/runtime-snapshot association through commit.
const ShaderStageRuntime* runtime = nullptr;
// Keep the resolved guest range through cache preparation; only the host buffer ID may
// become stale and need resolving again when bindings are rebound.
std::vector<BufferSource> buffer_sources;
@@ -68,9 +66,6 @@ template <typename T>
[[nodiscard]] bool IsSupportedDepthTextureEncoding(const ShaderTextureResource& descriptor,
bool r128 = false);
[[nodiscard]] bool
IsSupportedSampledVideoOutView(const ShaderRecompiler::IR::ImageResource& resource,
const ShaderTextureResource& descriptor, const Image& image);
void ValidateStorageTexture(const ShaderRecompiler::IR::ImageResource& resource,
const ShaderTextureResource& descriptor, uint64_t size);
@@ -89,12 +89,7 @@ template <typename... Args>
void PipelineCacheLog(fmt::format_string<Args...> format, Args&&... args) {
auto message = fmt::format(format, std::forward<Args>(args)...);
message += '\n';
if (Log::GetDirection() != Log::Direction::Console) {
std::fwrite(message.data(), 1, message.size(), stdout);
std::fflush(stdout);
}
Log::Write(message);
Log::Flush();
Log::WriteToConsoleAndLog(message);
}
bool ReadShaderGuestMemory(void*, uint64_t address, uint32_t* value) {
@@ -236,6 +231,9 @@ struct PipelineCache::ProgramCache {
switch (options.stage) {
case ShaderType::Vertex: stage_name = "vs"; break;
case ShaderType::Mesh: stage_name = "ms"; break;
case ShaderType::Local: stage_name = "ls"; break;
case ShaderType::TessellationControl: stage_name = "hs"; break;
case ShaderType::TessellationEvaluation: stage_name = "ds"; break;
case ShaderType::Pixel: stage_name = "ps"; break;
case ShaderType::Compute: stage_name = "cs"; break;
default: EXIT("invalid pipeline shader stage\n");
@@ -250,18 +248,9 @@ struct PipelineCache::ProgramCache {
}
DumpShaderSpirv(stage_name, options.shader_hash, result.spirv);
vk::ShaderModuleCreateInfo create_info {};
create_info.codeSize = result.spirv.size() * sizeof(uint32_t);
create_info.pCode = result.spirv.data();
vk::ShaderModule module = nullptr;
RequireVulkanSuccess(device.createShaderModule(&create_info, nullptr, &module),
"create recompiled shader module");
const auto module = CompileSPV(result.spirv, device);
EXIT_IF(module == nullptr);
if (options.dump_ir) {
if (!options.early_dump) {
LOGF("%s decoded RDNA2:\n%s", options.dump_label, result.decoded_dump.c_str());
LOGF("%s IR:\n%s", options.dump_label, result.ir_dump.c_str());
}
LOGF("%s SPIR-V words=%" PRIu64 " wave_size=%u\n", options.dump_label,
static_cast<uint64_t>(result.spirv.size()), options.wave_size);
}
@@ -277,7 +266,7 @@ struct PipelineCache::ProgramCache {
uint32_t& push_data_cursor) {
ShaderType stage;
if constexpr (std::is_same_v<InputInfo, ShaderVertexInputInfo>) {
stage = input_info.mesh.threads_num[0] != 0 ? ShaderType::Mesh : ShaderType::Vertex;
stage = input_info.logical_stage;
} else if constexpr (std::is_same_v<InputInfo, ShaderPixelInputInfo>) {
stage = ShaderType::Pixel;
} else {
@@ -329,6 +318,9 @@ struct PipelineCache::ProgramCache {
switch (stage) {
case ShaderType::Vertex: label = "ShaderRecompiler VS"; break;
case ShaderType::Mesh: label = "ShaderRecompiler MS"; break;
case ShaderType::Local: label = "ShaderRecompiler LS"; break;
case ShaderType::TessellationControl: label = "ShaderRecompiler HS"; break;
case ShaderType::TessellationEvaluation: label = "ShaderRecompiler DS"; break;
case ShaderType::Pixel: label = "ShaderRecompiler PS"; break;
case ShaderType::Compute: label = "ShaderRecompiler CS"; break;
default: EXIT("invalid pipeline shader stage\n");
@@ -338,19 +330,18 @@ struct PipelineCache::ProgramCache {
options.shader_hash = params.hash;
options.user_data = params.user_data;
options.back_code = params.back_code;
options.dump_ir = Config::GetShaderLogDirection() != Config::ShaderLogDirection::Silent;
options.dump_ir = Config::GetShaderLogDirection() != Config::LogDirection::Silent;
options.early_dump = options.dump_ir;
options.dump_label = label;
options.input_info = stage_input;
options.scratch_dwords = input_info.scratch_size_dwords;
if constexpr (std::is_same_v<InputInfo, ShaderVertexInputInfo>) {
options.user_data_base = 8;
if (stage == ShaderType::Mesh) {
if (stage == ShaderType::Mesh || stage == ShaderType::TessellationControl) {
options.user_data_base = 0;
options.wave_size = input_info.mesh.wave_size;
options.scratch_dwords = input_info.mesh.scratch_size_dwords;
options.wave_size = stage == ShaderType::Mesh ? input_info.mesh.wave_size : 64u;
}
} else if constexpr (std::is_same_v<InputInfo, ShaderComputeInputInfo>) {
} else {
options.wave_size = input_info.wave_size;
}
auto translated = ShaderRecompiler::TranslateProgram(params.code, options);
@@ -366,16 +357,19 @@ struct PipelineCache::ProgramCache {
input_info.stage = {.program = &permutation.program, .resources = std::move(resources)};
permutation.program.bindings.AdvancePushData(push_data_cursor);
std::array<size_t, static_cast<size_t>(ShaderType::Mesh) + 1> counts {};
std::array<size_t, static_cast<size_t>(ShaderType::TessellationEvaluation) + 1> counts {};
for (const auto& [key, source]: programs) {
counts[static_cast<size_t>(key.stage)] += source.permutations.size();
}
// Guest geometry shaders are compiled through the host mesh stage.
std::printf("Shaders: VS %zu | PS %zu | CS %zu | GS %zu\n",
std::printf("Shaders: VS %zu | PS %zu | CS %zu | GS %zu | LS %zu | HS %zu | TES %zu\n",
counts[static_cast<size_t>(ShaderType::Vertex)],
counts[static_cast<size_t>(ShaderType::Pixel)],
counts[static_cast<size_t>(ShaderType::Compute)],
counts[static_cast<size_t>(ShaderType::Mesh)]);
counts[static_cast<size_t>(ShaderType::Mesh)],
counts[static_cast<size_t>(ShaderType::Local)],
counts[static_cast<size_t>(ShaderType::TessellationControl)],
counts[static_cast<size_t>(ShaderType::TessellationEvaluation)]);
return permutation.handle;
}
@@ -569,12 +563,18 @@ PipelineCache::GraphicsPrograms PipelineCache::GetGraphicsPrograms(
const HW::VertexShaderInfo& vertex_regs, const HW::PixelShaderInfo& pixel_regs,
const HW::ShaderRegisters& sh, const HW::Context& context, const HW::UserConfig& user_config,
std::span<const Prospero::ColorComponentMapping, 8> target_export_mapping, bool pixel_active,
ShaderVertexInputInfo& vertex_info, ShaderPixelInputInfo& pixel_info) {
const auto vertex_params = PrepareProgram(vertex_regs, context, user_config, vertex_info);
const bool mesh_active = vertex_info.mesh.threads_num[0] != 0;
std::array<ShaderVertexInputInfo, 3>& vertex_info, ShaderPixelInputInfo& pixel_info) {
const bool tess_active = user_config.GetPrimType() == Prospero::PrimitiveType::kPatch;
std::array<ShaderParams, 3> vertex_params;
if (tess_active) {
vertex_params = PrepareTessellationPrograms(vertex_regs, context, vertex_info);
} else {
vertex_params[0] = PrepareProgram(vertex_regs, context, user_config, vertex_info[0]);
}
const bool mesh_active = vertex_info[0].logical_stage == ShaderType::Mesh;
if (mesh_active) {
EXIT_NOT_IMPLEMENTED(!m_graphics.mesh_shader_enabled);
auto& mesh = vertex_info.mesh;
auto& mesh = vertex_info[0].mesh;
mesh.host_subgroup_size = m_graphics.subgroup_size;
const auto& limits = m_graphics.mesh_shader_properties;
const auto logical_threads =
@@ -593,11 +593,26 @@ PipelineCache::GraphicsPrograms PipelineCache::GetGraphicsPrograms(
ShaderParams pixel_params;
if (pixel_active) {
pixel_params = PrepareProgram(pixel_regs, sh, target_export_mapping, pixel_info);
const auto& blend = context.GetBlendControl(0);
const auto is_dual_source = [](uint8_t factor) {
return factor >= static_cast<uint8_t>(Prospero::BlendFactor::kSrc1Color) &&
factor <= static_cast<uint8_t>(Prospero::BlendFactor::kOneMinusSrc1Alpha);
};
pixel_info.dual_source_blending =
blend.enable && !context.GetRenderTarget(0).info.blend_bypass &&
(is_dual_source(blend.color_srcblend) || is_dual_source(blend.color_destblend) ||
(blend.separate_alpha_blend &&
(is_dual_source(blend.alpha_srcblend) || is_dual_source(blend.alpha_destblend))));
if (pixel_info.dual_source_blending) {
// MRT1 supplies a second blend source for the same render target as MRT0.
pixel_info.target_output_mode[1] = pixel_info.target_output_mode[0];
pixel_info.target_export_mapping[1] = pixel_info.target_export_mapping[0];
}
}
if (context.GetClipControl().clip_disable) {
const auto& viewport = context.GetScreenViewport().viewports[0];
const auto& limits = m_graphics.GetPhysicalDeviceProperties().limits;
auto& clip = vertex_info.clip_space;
auto& clip = vertex_info[tess_active ? 2u : 0u].clip_space;
clip.scale[0] = viewport.xscale;
clip.scale[1] = viewport.yscale;
clip.offset[0] = viewport.xoffset;
@@ -615,7 +630,9 @@ PipelineCache::GraphicsPrograms PipelineCache::GetGraphicsPrograms(
if (pixel_active) {
result.pixel = m_program_cache->Get(pixel_params, pixel_info, push_data_cursor);
}
result.vertex = m_program_cache->Get(vertex_params, vertex_info, push_data_cursor);
for (uint32_t i = 0; i < (tess_active ? 3u : 1u); i++) {
result.vertex[i] = m_program_cache->Get(vertex_params[i], vertex_info[i], push_data_cursor);
}
return result;
}
@@ -633,12 +650,14 @@ bool PipelineStaticParameters::operator==(const PipelineStaticParameters& other)
return std::memcmp(this, &other, sizeof(*this)) == 0;
}
PipelineCache::Pipeline& PipelineCache::CreateGraphicsPipeline(
PipelineCache::Pipeline& PipelineCache::GetGraphicsPipeline(
std::span<const RenderColorInfo> colors, const RenderDepthInfo& depth,
const ShaderVertexInputInfo& vs_input_info, CommandBuffer& command,
std::span<const ShaderVertexInputInfo> vertex_info, CommandBuffer& command,
const ShaderPixelInputInfo* ps_input_info, vk::PrimitiveTopology topology,
bool primitive_restart_enable, const ShaderProgram& vertex_program,
const ShaderProgram& pixel_program) {
bool primitive_restart_enable, const GraphicsPrograms& programs) {
const auto& vs_input_info = vertex_info.front();
const auto& vertex_program = programs.vertex[0];
const auto& pixel_program = programs.pixel;
KYTY_PROFILER_BLOCK("PipelineCache::CreatePipeline(Gfx)", profiler::colors::DeepOrangeA200);
EXIT_IF(colors.size() > RENDER_COLOR_ATTACHMENTS_MAX);
@@ -656,23 +675,38 @@ PipelineCache::Pipeline& PipelineCache::CreateGraphicsPipeline(
const auto ps_id = ps_active ? pixel_program.id : 0;
GraphicsPipelineKey key {};
key.vs_shader_id = vs_id;
for (uint32_t i = 0; i < programs.vertex.size(); i++) {
key.vertex_shader_ids[i] = programs.vertex[i].id;
}
key.ps_shader_id = ps_id;
auto& static_params = key.static_params;
auto& rendering = key.rendering;
rendering.color_count = color_count;
rendering.color_count = 0;
uint32_t attachment_samples = 0;
for (uint32_t i = 0; i < color_count; i++) {
const auto slot = colors[i].target_slot;
EXIT_IF(slot >= RENDER_COLOR_ATTACHMENTS_MAX);
rendering.color_count = std::max(rendering.color_count, slot + 1);
EXIT_IF(!colors[i].image_id || colors[i].desc.view_info.format == vk::Format::eUndefined);
static_params.color_mask[i] = colors[i].export_mapping.ApplyMask(
static_params.color_mask[slot] = colors[i].export_mapping.ApplyMask(
render_target_mask_slot(ctx.GetRenderTargetMask(), colors[i].target_slot));
rendering.color_formats[i] = colors[i].desc.view_info.format;
rendering.color_formats[slot] = colors[i].desc.view_info.format;
if (attachment_samples == 0) {
attachment_samples = colors[i].desc.info.samples;
} else if (attachment_samples != colors[i].desc.info.samples) {
EXIT("mixed color attachment sample counts are unsupported: %u and %u\n",
attachment_samples, colors[i].desc.info.samples);
}
const auto& rt = ctx.GetRenderTarget(colors[i].target_slot);
const auto& bc = ctx.GetBlendControl(colors[i].target_slot);
static_params.color_srcblend[slot] = bc.color_srcblend;
static_params.color_comb_fcn[slot] = bc.color_comb_fcn;
static_params.color_destblend[slot] = bc.color_destblend;
static_params.alpha_srcblend[slot] = bc.alpha_srcblend;
static_params.alpha_comb_fcn[slot] = bc.alpha_comb_fcn;
static_params.alpha_destblend[slot] = bc.alpha_destblend;
static_params.separate_alpha_blend[slot] = bc.separate_alpha_blend;
static_params.blend_enable[slot] = bc.enable && !rt.info.blend_bypass;
}
const bool with_depth =
depth.desc.view_info.format != vk::Format::eUndefined && static_cast<bool>(depth.image_id);
@@ -721,29 +755,14 @@ PipelineCache::Pipeline& PipelineCache::CreateGraphicsPipeline(
static_params.depth_bounds_test_enable = depth.depth_bounds_test_enable;
static_params.depth_min_bounds = depth.depth_min_bounds;
static_params.depth_max_bounds = depth.depth_max_bounds;
static_params.stencil_test_enable = depth.stencil_test_enable;
static_params.stencil_front = depth.stencil_static_front;
static_params.stencil_back = depth.stencil_static_back;
const bool rect_list = topology == vk::PrimitiveTopology::ePatchList;
const bool rect_list = Prospero::IsRectList(command.GetUserConfig().GetPrimType());
static_params.cull_back = !rect_list && mc.cull_back;
static_params.cull_front = !rect_list && mc.cull_front;
static_params.face = mc.face;
static_params.provoking_vtx_last = mc.provoking_vtx_last;
static_params.polygon_mode =
ResolvePolygonMode(mc, static_params.cull_front, static_params.cull_back);
for (uint32_t i = 0; i < color_count; i++) {
const auto& rt = ctx.GetRenderTarget(colors[i].target_slot);
const auto& bc = ctx.GetBlendControl(colors[i].target_slot);
static_params.color_srcblend[i] = bc.color_srcblend;
static_params.color_comb_fcn[i] = bc.color_comb_fcn;
static_params.color_destblend[i] = bc.color_destblend;
static_params.alpha_srcblend[i] = bc.alpha_srcblend;
static_params.alpha_comb_fcn[i] = bc.alpha_comb_fcn;
static_params.alpha_destblend[i] = bc.alpha_destblend;
static_params.separate_alpha_blend[i] = bc.separate_alpha_blend;
static_params.blend_enable[i] = bc.enable;
static_params.blend_bypass[i] = rt.info.blend_bypass;
}
if (vs_input_info.stage.program->stage != ShaderType::Mesh) {
EXIT_IF(vs_input_info.buffers_num < 0 ||
vs_input_info.buffers_num > ShaderVertexInputInfo::RES_MAX ||
@@ -787,9 +806,8 @@ PipelineCache::Pipeline& PipelineCache::CreateGraphicsPipeline(
auto cached = std::make_unique<Pipeline>();
LogPipelineTrace("CreatePipelineInternal begin", vs_id, ps_id);
CreatePipelineInternal(m_graphics, *cached, rendering, key.vertex_input, vs_input_info,
vertex_program, ps_input_info, pixel_program, static_params,
m_driver_cache);
CreatePipelineInternal(m_graphics, *cached, rendering, key.vertex_input, vertex_info,
ps_input_info, programs, static_params, m_driver_cache);
LogPipelineTrace("CreatePipelineInternal done", vs_id, ps_id);
EXIT_NOT_IMPLEMENTED(cached->pipeline == nullptr);
@@ -802,8 +820,8 @@ PipelineCache::Pipeline& PipelineCache::CreateGraphicsPipeline(
}
PipelineCache::Pipeline&
PipelineCache::CreateComputePipeline(const ShaderComputeInputInfo& input_info,
const ShaderProgram& compute_program) {
PipelineCache::GetComputePipeline(const ShaderComputeInputInfo& input_info,
const ShaderProgram& compute_program) {
KYTY_PROFILER_BLOCK("PipelineCache::CreatePipeline(Compute)", profiler::colors::RedA100);
EXIT_IF(!compute_program);
@@ -42,13 +42,11 @@ struct PipelineStaticParameters {
bool depth_bounds_test_enable = false;
float depth_min_bounds = 0.0f;
float depth_max_bounds = 0.0f;
bool stencil_test_enable = false;
PipelineStencilStaticState stencil_front;
PipelineStencilStaticState stencil_back;
uint32_t color_mask[RENDER_COLOR_ATTACHMENTS_MAX] = {};
bool cull_front = false;
bool cull_back = false;
bool face = false;
bool provoking_vtx_last = false;
vk::PolygonMode polygon_mode = vk::PolygonMode::eFill;
uint8_t color_srcblend[RENDER_COLOR_ATTACHMENTS_MAX] = {};
uint8_t color_comb_fcn[RENDER_COLOR_ATTACHMENTS_MAX] = {};
@@ -58,7 +56,6 @@ struct PipelineStaticParameters {
uint8_t alpha_destblend[RENDER_COLOR_ATTACHMENTS_MAX] = {};
bool separate_alpha_blend[RENDER_COLOR_ATTACHMENTS_MAX] = {};
bool blend_enable[RENDER_COLOR_ATTACHMENTS_MAX] = {};
bool blend_bypass[RENDER_COLOR_ATTACHMENTS_MAX] = {};
bool operator==(const PipelineStaticParameters& other) const noexcept;
};
@@ -68,7 +65,7 @@ struct PipelineStaticParameters {
static_assert(std::is_trivially_copyable_v<PipelineStaticParameters>);
static_assert(std::is_standard_layout_v<PipelineStaticParameters>);
static_assert(alignof(PipelineStaticParameters) == 1);
static_assert(sizeof(PipelineStaticParameters) == 165);
static_assert(sizeof(PipelineStaticParameters) == 125);
struct PipelineRenderingState {
std::array<vk::Format, RENDER_COLOR_ATTACHMENTS_MAX> color_formats {};
@@ -121,8 +118,10 @@ public:
};
struct GraphicsPrograms {
ShaderProgram vertex;
std::array<ShaderProgram, 3> vertex;
ShaderProgram pixel;
[[nodiscard]] uint32_t VertexStageCount() const { return vertex[1] ? 3u : 1u; }
};
GraphicsPrograms
@@ -130,33 +129,33 @@ public:
const HW::PixelShaderInfo& pixel_regs, const HW::ShaderRegisters& sh,
const HW::Context& context, const HW::UserConfig& user_config,
std::span<const Prospero::ColorComponentMapping, 8> target_export_mapping,
bool pixel_active, ShaderVertexInputInfo& vertex_info,
bool pixel_active, std::array<ShaderVertexInputInfo, 3>& vertex_info,
ShaderPixelInputInfo& pixel_info);
ShaderProgram GetComputeProgram(const HW::ComputeShaderInfo& regs,
const HW::ShaderRegisters& sh,
ShaderComputeInputInfo& input_info);
Pipeline&
CreateGraphicsPipeline(std::span<const RenderColorInfo> colors, const RenderDepthInfo& depth,
const ShaderVertexInputInfo& vs_input_info, CommandBuffer& command,
const ShaderPixelInputInfo* ps_input_info,
vk::PrimitiveTopology topology, bool primitive_restart_enable,
const ShaderProgram& vertex_program, const ShaderProgram& pixel_program);
Pipeline& CreateComputePipeline(const ShaderComputeInputInfo& input_info,
const ShaderProgram& compute_program);
Pipeline& GetGraphicsPipeline(std::span<const RenderColorInfo> colors,
const RenderDepthInfo& depth,
std::span<const ShaderVertexInputInfo> vertex_info,
CommandBuffer& command, const ShaderPixelInputInfo* ps_input_info,
vk::PrimitiveTopology topology, bool primitive_restart_enable,
const GraphicsPrograms& programs);
Pipeline& GetComputePipeline(const ShaderComputeInputInfo& input_info,
const ShaderProgram& compute_program);
private:
struct ProgramCache;
struct GraphicsPipelineKey {
PipelineRenderingState rendering;
uint64_t vs_shader_id = 0;
std::array<uint64_t, 3> vertex_shader_ids {};
uint64_t ps_shader_id = 0;
PipelineVertexInputState vertex_input;
PipelineStaticParameters static_params;
bool operator==(const GraphicsPipelineKey& other) const {
return rendering == other.rendering && vs_shader_id == other.vs_shader_id &&
return rendering == other.rendering && vertex_shader_ids == other.vertex_shader_ids &&
ps_shader_id == other.ps_shader_id && vertex_input == other.vertex_input &&
static_params == other.static_params;
}
@@ -189,7 +188,9 @@ private:
std::size_t operator()(const GraphicsPipelineKey& key) const {
std::size_t hash = 0;
PipelineKeyHash::MixRendering(hash, key.rendering);
PipelineKeyHash::Mix(hash, key.vs_shader_id);
for (const auto id: key.vertex_shader_ids) {
PipelineKeyHash::Mix(hash, id);
}
PipelineKeyHash::Mix(hash, key.ps_shader_id);
PipelineKeyHash::Mix(hash, key.vertex_input.binding_count);
for (uint32_t i = 0; i < key.vertex_input.binding_count; i++) {
@@ -219,12 +220,14 @@ private:
};
void LogPipelineTrace(const char* phase, uint64_t vertex_program_id, uint64_t pixel_program_id);
void CreatePipelineInternal(
GraphicContext& graphics, PipelineCache::Pipeline& pipeline,
const PipelineRenderingState& rendering, const PipelineVertexInputState& vertex_input,
const ShaderVertexInputInfo& vs_input_info, const ShaderProgram& vertex_program,
const ShaderPixelInputInfo* ps_input_info, const ShaderProgram& pixel_program,
const PipelineStaticParameters& static_params, vk::PipelineCache driver_cache);
void CreatePipelineInternal(GraphicContext& graphics, PipelineCache::Pipeline& pipeline,
const PipelineRenderingState& rendering,
const PipelineVertexInputState& vertex_input,
std::span<const ShaderVertexInputInfo> vertex_info,
const ShaderPixelInputInfo* ps_input_info,
const PipelineCache::GraphicsPrograms& programs,
const PipelineStaticParameters& static_params,
vk::PipelineCache driver_cache);
void CreatePipelineInternal(GraphicContext& graphics, PipelineCache::Pipeline& pipeline,
const ShaderComputeInputInfo& input_info,
vk::ShaderModule compute_module, vk::PipelineCache driver_cache);
@@ -8,6 +8,20 @@
namespace Libs::Graphics {
vk::ShaderStageFlagBits NativeShaderStage(ShaderType stage) {
switch (stage) {
case ShaderType::Local:
case ShaderType::Vertex: return vk::ShaderStageFlagBits::eVertex;
case ShaderType::Mesh: return vk::ShaderStageFlagBits::eMeshEXT;
case ShaderType::TessellationControl: return vk::ShaderStageFlagBits::eTessellationControl;
case ShaderType::TessellationEvaluation:
return vk::ShaderStageFlagBits::eTessellationEvaluation;
case ShaderType::Pixel: return vk::ShaderStageFlagBits::eFragment;
case ShaderType::Compute: return vk::ShaderStageFlagBits::eCompute;
default: EXIT("unknown native shader stage\n");
}
}
vk::PipelineStageFlags ShaderPipelineStages(vk::ShaderStageFlags stages) {
vk::PipelineStageFlags result = {};
if (stages & vk::ShaderStageFlagBits::eVertex) {
@@ -16,6 +30,12 @@ vk::PipelineStageFlags ShaderPipelineStages(vk::ShaderStageFlags stages) {
if (stages & vk::ShaderStageFlagBits::eMeshEXT) {
result |= vk::PipelineStageFlagBits::eMeshShaderEXT;
}
if (stages & vk::ShaderStageFlagBits::eTessellationControl) {
result |= vk::PipelineStageFlagBits::eTessellationControlShader;
}
if (stages & vk::ShaderStageFlagBits::eTessellationEvaluation) {
result |= vk::PipelineStageFlagBits::eTessellationEvaluationShader;
}
if (stages & vk::ShaderStageFlagBits::eFragment) {
result |= vk::PipelineStageFlagBits::eFragmentShader;
}
@@ -26,8 +46,8 @@ vk::PipelineStageFlags ShaderPipelineStages(vk::ShaderStageFlags stages) {
return result;
}
VulkanMemoryBarrier MakeShaderWriteDependency() {
VulkanMemoryBarrier barrier {};
vk::MemoryBarrier MakeShaderWriteDependency() {
vk::MemoryBarrier barrier {};
barrier.srcAccessMask = vk::AccessFlagBits::eShaderWrite;
barrier.dstAccessMask = vk::AccessFlagBits::eShaderRead | vk::AccessFlagBits::eShaderWrite |
vk::AccessFlagBits::eVertexAttributeRead |
@@ -38,15 +58,15 @@ VulkanMemoryBarrier MakeShaderWriteDependency() {
return barrier;
}
VulkanMemoryBarrier MakeShaderAccessDependency() {
VulkanMemoryBarrier barrier {};
vk::MemoryBarrier MakeShaderAccessDependency() {
vk::MemoryBarrier barrier {};
barrier.srcAccessMask = vk::AccessFlagBits::eShaderRead | vk::AccessFlagBits::eShaderWrite;
barrier.dstAccessMask = vk::AccessFlagBits::eMemoryRead | vk::AccessFlagBits::eMemoryWrite;
return barrier;
}
VulkanMemoryBarrier MakeShaderWriteHazardDependency() {
VulkanMemoryBarrier barrier {};
vk::MemoryBarrier MakeShaderWriteHazardDependency() {
vk::MemoryBarrier barrier {};
barrier.srcAccessMask = vk::AccessFlagBits::eMemoryRead | vk::AccessFlagBits::eMemoryWrite;
barrier.dstAccessMask = vk::AccessFlagBits::eShaderRead | vk::AccessFlagBits::eShaderWrite;
return barrier;
@@ -8,10 +8,11 @@ namespace Libs::Graphics {
struct ShaderStageRuntime;
vk::ShaderStageFlagBits NativeShaderStage(ShaderType stage);
vk::PipelineStageFlags ShaderPipelineStages(vk::ShaderStageFlags stages);
VulkanMemoryBarrier MakeShaderAccessDependency();
VulkanMemoryBarrier MakeShaderWriteHazardDependency();
VulkanMemoryBarrier MakeShaderWriteDependency();
vk::MemoryBarrier MakeShaderAccessDependency();
vk::MemoryBarrier MakeShaderWriteHazardDependency();
vk::MemoryBarrier MakeShaderWriteDependency();
vk::BufferMemoryBarrier MakeGdsDependency(vk::Buffer buffer);
bool HasShaderBufferWrites(const ShaderStageRuntime& runtime);
void ShaderAccessBarrier(vk::CommandBuffer vk_buffer, vk::PipelineStageFlags source_stages);
@@ -8,10 +8,12 @@
#include "graphics/host_gpu/renderer/debug.h"
#include "graphics/host_gpu/renderer/pipeline/descriptors.h"
#include "graphics/host_gpu/renderer/pipeline/pipelineCache.h"
#include "graphics/host_gpu/renderer/pipeline/shaderResourceBarrier.h"
#include "graphics/host_gpu/renderer/render.h"
#include "graphics/host_gpu/renderer/renderContext.h"
#include "graphics/host_gpu/renderer/renderTarget.h"
#include "graphics/host_gpu/vulkanCommon.h"
#include "graphics/shader/recompiler/BufferFormat.h"
#include "graphics/shader/recompiler/ir/ShaderIR.h"
#include "graphics/shader/rectListShader.h"
#include "graphics/shader/shader.h"
@@ -115,227 +117,13 @@ static void GetInputFormat(const ShaderBufferResource& res, vk::Format& format,
}
format = vk::Format::eR16G16Sfloat;
size = 2;
if (NarrowInputFormat(format, size, used_components)) {
LOGF("InputFormat: narrowing fmt=%u to %s for used_components=%u\n", raw_format,
vk::to_string(format).c_str(), used_components);
}
return;
}
switch (fmt) {
case Prospero::BufferFormat::k32_32_32_32Float:
format = vk::Format::eR32G32B32A32Sfloat;
size = 4;
break;
case Prospero::BufferFormat::k32_32_32_32SInt:
format = vk::Format::eR32G32B32A32Sint;
size = 4;
break;
case Prospero::BufferFormat::k32_32_32_32UInt:
format = vk::Format::eR32G32B32A32Uint;
size = 4;
break;
case Prospero::BufferFormat::k32_32_32Float:
format = vk::Format::eR32G32B32Sfloat;
size = 3;
break;
case Prospero::BufferFormat::k32_32_32SInt:
format = vk::Format::eR32G32B32Sint;
size = 3;
break;
case Prospero::BufferFormat::k32_32_32UInt:
format = vk::Format::eR32G32B32Uint;
size = 3;
break;
case Prospero::BufferFormat::k16_16_16_16Float:
format = vk::Format::eR16G16B16A16Sfloat;
size = 4;
break;
case Prospero::BufferFormat::k16_16_16_16SInt:
format = vk::Format::eR16G16B16A16Sint;
size = 4;
break;
case Prospero::BufferFormat::k16_16_16_16UInt:
format = vk::Format::eR16G16B16A16Uint;
size = 4;
break;
case Prospero::BufferFormat::k16_16_16_16SScaled:
format = vk::Format::eR16G16B16A16Sscaled;
size = 4;
break;
case Prospero::BufferFormat::k16_16_16_16UScaled:
format = vk::Format::eR16G16B16A16Uscaled;
size = 4;
break;
case Prospero::BufferFormat::k16_16_16_16SNorm:
format = vk::Format::eR16G16B16A16Snorm;
size = 4;
break;
case Prospero::BufferFormat::k16_16_16_16UNorm:
format = vk::Format::eR16G16B16A16Unorm;
size = 4;
break;
case Prospero::BufferFormat::k32_32Float:
format = vk::Format::eR32G32Sfloat;
size = 2;
break;
case Prospero::BufferFormat::k32_32SInt:
format = vk::Format::eR32G32Sint;
size = 2;
break;
case Prospero::BufferFormat::k32_32UInt:
format = vk::Format::eR32G32Uint;
size = 2;
break;
case Prospero::BufferFormat::k8_8_8_8UInt:
format = vk::Format::eR8G8B8A8Uint;
size = 4;
break;
case Prospero::BufferFormat::k8_8_8_8SScaled:
format = vk::Format::eR8G8B8A8Sscaled;
size = 4;
break;
case Prospero::BufferFormat::k8_8_8_8UScaled:
format = vk::Format::eR8G8B8A8Uscaled;
size = 4;
break;
case Prospero::BufferFormat::k8_8_8_8SNorm:
format = vk::Format::eR8G8B8A8Snorm;
size = 4;
break;
case Prospero::BufferFormat::k8_8_8_8UNorm:
format = vk::Format::eR8G8B8A8Unorm;
size = 4;
break;
case Prospero::BufferFormat::k10_10_10_2UNorm:
format = vk::Format::eA2B10G10R10UnormPack32;
size = 4;
break;
case Prospero::BufferFormat::k10_10_10_2SNorm:
format = vk::Format::eA2B10G10R10SnormPack32;
size = 4;
break;
case Prospero::BufferFormat::k16_16Float:
format = vk::Format::eR16G16Sfloat;
size = 2;
break;
case Prospero::BufferFormat::k16_16SInt:
format = vk::Format::eR16G16Sint;
size = 2;
break;
case Prospero::BufferFormat::k16_16UInt:
format = vk::Format::eR16G16Uint;
size = 2;
break;
case Prospero::BufferFormat::k16_16SScaled:
format = vk::Format::eR16G16Sscaled;
size = 2;
break;
case Prospero::BufferFormat::k16_16UScaled:
format = vk::Format::eR16G16Uscaled;
size = 2;
break;
case Prospero::BufferFormat::k16_16SNorm:
format = vk::Format::eR16G16Snorm;
size = 2;
break;
case Prospero::BufferFormat::k16_16UNorm:
format = vk::Format::eR16G16Unorm;
size = 2;
break;
case Prospero::BufferFormat::k32Float:
format = vk::Format::eR32Sfloat;
size = 1;
break;
case Prospero::BufferFormat::k32SInt:
format = vk::Format::eR32Sint;
size = 1;
break;
case Prospero::BufferFormat::k32UInt:
format = vk::Format::eR32Uint;
size = 1;
break;
case Prospero::BufferFormat::k8_8SInt:
format = vk::Format::eR8G8Sint;
size = 2;
break;
case Prospero::BufferFormat::k8_8UInt:
format = vk::Format::eR8G8Uint;
size = 2;
break;
case Prospero::BufferFormat::k8_8SScaled:
format = vk::Format::eR8G8Sscaled;
size = 2;
break;
case Prospero::BufferFormat::k8_8UScaled:
format = vk::Format::eR8G8Uscaled;
size = 2;
break;
case Prospero::BufferFormat::k8_8SNorm:
format = vk::Format::eR8G8Snorm;
size = 2;
break;
case Prospero::BufferFormat::k8_8UNorm:
format = vk::Format::eR8G8Unorm;
size = 2;
break;
case Prospero::BufferFormat::k16Float:
format = vk::Format::eR16Sfloat;
size = 1;
break;
case Prospero::BufferFormat::k16SInt:
format = vk::Format::eR16Sint;
size = 1;
break;
case Prospero::BufferFormat::k16UInt:
format = vk::Format::eR16Uint;
size = 1;
break;
case Prospero::BufferFormat::k16SScaled:
format = vk::Format::eR16Sscaled;
size = 1;
break;
case Prospero::BufferFormat::k16UScaled:
format = vk::Format::eR16Uscaled;
size = 1;
break;
case Prospero::BufferFormat::k16SNorm:
format = vk::Format::eR16Snorm;
size = 1;
break;
case Prospero::BufferFormat::k16UNorm:
format = vk::Format::eR16Unorm;
size = 1;
break;
case Prospero::BufferFormat::k8SInt:
format = vk::Format::eR8Sint;
size = 1;
break;
case Prospero::BufferFormat::k8UInt:
format = vk::Format::eR8Uint;
size = 1;
break;
case Prospero::BufferFormat::k8SScaled:
format = vk::Format::eR8Sscaled;
size = 1;
break;
case Prospero::BufferFormat::k8UScaled:
format = vk::Format::eR8Uscaled;
size = 1;
break;
case Prospero::BufferFormat::k8SNorm:
format = vk::Format::eR8Snorm;
size = 1;
break;
case Prospero::BufferFormat::k8UNorm:
format = vk::Format::eR8Unorm;
size = 1;
break;
default:
EXIT("unknown format: fmt = %u\n", raw_format);
format = vk::Format::eUndefined;
size = 4;
break;
format = VulkanFormat(fmt);
size = ShaderRecompiler::Format::GetFormatInfo(fmt).component_count;
if (format == vk::Format::eUndefined || size == 0) {
EXIT("unknown vertex format: fmt = %u\n", raw_format);
}
if (NarrowInputFormat(format, size, used_components)) {
@@ -418,12 +206,18 @@ static void CreateDescriptorLayout(GraphicContext& graphics, PipelineCache::Pipe
}
// NOLINTNEXTLINE(readability-function-cognitive-complexity)
void CreatePipelineInternal(
GraphicContext& graphics, PipelineCache::Pipeline& pipeline,
const PipelineRenderingState& rendering, const PipelineVertexInputState& vertex_input,
const ShaderVertexInputInfo& vs_input_info, const ShaderProgram& vertex_program,
const ShaderPixelInputInfo* ps_input_info, const ShaderProgram& pixel_program,
const PipelineStaticParameters& static_params, vk::PipelineCache driver_cache) {
void CreatePipelineInternal(GraphicContext& graphics, PipelineCache::Pipeline& pipeline,
const PipelineRenderingState& rendering,
const PipelineVertexInputState& vertex_input,
std::span<const ShaderVertexInputInfo> vertex_info,
const ShaderPixelInputInfo* ps_input_info,
const PipelineCache::GraphicsPrograms& programs,
const PipelineStaticParameters& static_params,
vk::PipelineCache driver_cache) {
const auto& vs_input_info = vertex_info.front();
const auto& vertex_program = programs.vertex[0];
const auto& pixel_program = programs.pixel;
const bool tessellation = vertex_info.size() == 3;
const bool ps_active = ps_input_info != nullptr;
EXIT_IF(!vertex_program || (ps_active && !pixel_program));
const bool with_depth = rendering.depth_format != vk::Format::eUndefined ||
@@ -431,72 +225,53 @@ void CreatePipelineInternal(
EXIT_IF(!vs_input_info.stage);
const bool mesh = vs_input_info.stage.program->stage == ShaderType::Mesh;
EXIT_NOT_IMPLEMENTED(mesh && !graphics.mesh_shader_enabled);
const auto vertex_stage =
mesh ? vk::ShaderStageFlagBits::eMeshEXT : vk::ShaderStageFlagBits::eVertex;
const bool rect_list = !mesh && static_params.topology == vk::PrimitiveTopology::ePatchList;
const bool rect_list =
!mesh && !tessellation && static_params.topology == vk::PrimitiveTopology::ePatchList;
vk::ShaderModule tess_control_shader_module = nullptr;
vk::ShaderModule tess_eval_shader_module = nullptr;
vk::ShaderModuleCreateInfo create_info {};
vk::Result result {};
if (rect_list) {
const auto shaders =
BuildRectListShaders(vs_input_info, ps_active ? ps_input_info : nullptr);
create_info.codeSize = shaders.control.size() * 4;
create_info.pCode = shaders.control.data();
result =
graphics.device.createShaderModule(&create_info, nullptr, &tess_control_shader_module);
tess_control_shader_module = CompileSPV(shaders.control, graphics.device);
if (graphics_debug_dump_enabled()) {
LOGF("PipelineTrace: vkCreateShaderModule RectList TCS done result=%s module=%p\n",
vk::to_string(result).c_str(), static_cast<void*>(tess_control_shader_module));
LOGF("PipelineTrace: vkCreateShaderModule RectList TCS done module=%p\n",
static_cast<void*>(tess_control_shader_module));
}
EXIT_NOT_IMPLEMENTED(result != vk::Result::eSuccess);
create_info.codeSize = shaders.evaluation.size() * 4;
create_info.pCode = shaders.evaluation.data();
result =
graphics.device.createShaderModule(&create_info, nullptr, &tess_eval_shader_module);
tess_eval_shader_module = CompileSPV(shaders.evaluation, graphics.device);
if (graphics_debug_dump_enabled()) {
LOGF("PipelineTrace: vkCreateShaderModule RectList TES done result=%s module=%p\n",
vk::to_string(result).c_str(), static_cast<void*>(tess_eval_shader_module));
LOGF("PipelineTrace: vkCreateShaderModule RectList TES done module=%p\n",
static_cast<void*>(tess_eval_shader_module));
}
EXIT_NOT_IMPLEMENTED(result != vk::Result::eSuccess);
}
EXIT_NOT_IMPLEMENTED(
rect_list && (tess_control_shader_module == nullptr || tess_eval_shader_module == nullptr));
vk::PipelineShaderStageCreateInfo vert_shader_stage_info {};
vert_shader_stage_info.stage = vertex_stage;
vert_shader_stage_info.module = vertex_program.module;
vert_shader_stage_info.pName = "main";
vk::PipelineShaderStageCreateInfo frag_shader_stage_info {};
frag_shader_stage_info.stage = vk::ShaderStageFlagBits::eFragment;
frag_shader_stage_info.module = pixel_program.module;
frag_shader_stage_info.pName = "main";
vk::PipelineShaderStageCreateInfo tess_control_shader_stage_info {};
tess_control_shader_stage_info.stage = vk::ShaderStageFlagBits::eTessellationControl;
tess_control_shader_stage_info.module = tess_control_shader_module;
tess_control_shader_stage_info.pName = "main";
vk::PipelineShaderStageCreateInfo tess_eval_shader_stage_info {};
tess_eval_shader_stage_info.stage = vk::ShaderStageFlagBits::eTessellationEvaluation;
tess_eval_shader_stage_info.module = tess_eval_shader_module;
tess_eval_shader_stage_info.pName = "main";
vk::PipelineShaderStageCreateInfo shader_stages[4] = {};
vk::PipelineShaderStageCreateInfo shader_stages[4] {};
uint32_t shader_stage_count = 0;
shader_stages[shader_stage_count++] = vert_shader_stage_info;
for (uint32_t i = 0; i < vertex_info.size(); i++) {
shader_stages[shader_stage_count++] = {.stage =
NativeShaderStage(vertex_info[i].logical_stage),
.module = programs.vertex[i].module,
.pName = "main"};
}
if (rect_list) {
shader_stages[shader_stage_count++] = tess_control_shader_stage_info;
shader_stages[shader_stage_count++] = tess_eval_shader_stage_info;
shader_stages[shader_stage_count++] = {.stage =
vk::ShaderStageFlagBits::eTessellationControl,
.module = tess_control_shader_module,
.pName = "main"};
shader_stages[shader_stage_count++] = {.stage =
vk::ShaderStageFlagBits::eTessellationEvaluation,
.module = tess_eval_shader_module,
.pName = "main"};
}
if (ps_active) {
shader_stages[shader_stage_count++] = frag_shader_stage_info;
shader_stages[shader_stage_count++] = {.stage = vk::ShaderStageFlagBits::eFragment,
.module = pixel_program.module,
.pName = "main"};
}
vk::VertexInputAttributeDescription input_attr[ShaderVertexInputInfo::RES_MAX] {};
@@ -595,6 +370,15 @@ void CreatePipelineInternal(
#if !defined(__APPLE__)
rasterizer.pNext = &clip_ext;
#endif
vk::PipelineRasterizationProvokingVertexStateCreateInfoEXT provoking_vertex {};
EXIT_NOT_IMPLEMENTED(static_params.provoking_vtx_last &&
!graphics.provoking_vertex_last_enabled);
if (graphics.provoking_vertex_last_enabled) {
provoking_vertex.provokingVertexMode = static_params.provoking_vtx_last
? vk::ProvokingVertexModeEXT::eLastVertex : vk::ProvokingVertexModeEXT::eFirstVertex;
provoking_vertex.pNext = rasterizer.pNext;
rasterizer.pNext = &provoking_vertex;
}
rasterizer.cullMode = cull_mode;
rasterizer.frontFace = front_face;
rasterizer.polygonMode = static_params.polygon_mode;
@@ -610,8 +394,7 @@ void CreatePipelineInternal(
EXIT_NOT_IMPLEMENTED((static_params.color_mask[i] & ~0x0fu) != 0);
color_blend_attachment[i].colorWriteMask =
vk::ColorComponentFlags {static_params.color_mask[i]};
color_blend_attachment[i].blendEnable =
(static_params.blend_enable[i] && !static_params.blend_bypass[i]) ? VK_TRUE : VK_FALSE;
color_blend_attachment[i].blendEnable = static_params.blend_enable[i] ? VK_TRUE : VK_FALSE;
color_blend_attachment[i].srcColorBlendFactor =
GetBlendFactor(static_params.color_srcblend[i]);
color_blend_attachment[i].dstColorBlendFactor =
@@ -649,15 +432,19 @@ void CreatePipelineInternal(
color_blending.pAttachments = color_blend_attachment;
std::vector<vk::DescriptorSetLayoutBinding> descriptor_bindings;
AddLayoutBindings(descriptor_bindings, *vs_input_info.stage.program, vertex_stage);
vk::ShaderStageFlags graphics_stages = vk::ShaderStageFlagBits::eFragment;
for (const auto& stage: vertex_info) {
const auto native_stage = NativeShaderStage(stage.logical_stage);
AddLayoutBindings(descriptor_bindings, *stage.stage.program, native_stage);
graphics_stages |= native_stage;
}
if (ps_active) {
EXIT_IF(!ps_input_info->stage);
AddLayoutBindings(descriptor_bindings, *ps_input_info->stage.program,
vk::ShaderStageFlagBits::eFragment);
}
CreateDescriptorLayout(graphics, pipeline, descriptor_bindings);
const auto GraphicsStages = vertex_stage | vk::ShaderStageFlagBits::eFragment;
const vk::PushConstantRange push_constants {GraphicsStages, 0,
const vk::PushConstantRange push_constants {graphics_stages, 0,
ShaderRecompiler::IR::NativePushConstantSize};
vk::PipelineLayoutCreateInfo pipeline_layout_info {};
@@ -673,8 +460,8 @@ void CreatePipelineInternal(
" set_layouts=1 push_constants=%" PRIu32 "\n",
vertex_program.id, ps_active ? pixel_program.id : 0, 1u);
}
result = graphics.device.createPipelineLayout(&pipeline_layout_info, nullptr,
&pipeline.pipeline_layout);
auto result = graphics.device.createPipelineLayout(&pipeline_layout_info, nullptr,
&pipeline.pipeline_layout);
if (graphics_debug_dump_enabled()) {
LOGF("PipelineTrace: vkCreatePipelineLayout done result=%s layout=%p\n",
vk::to_string(result).c_str(), static_cast<void*>(pipeline.pipeline_layout));
@@ -690,15 +477,6 @@ void CreatePipelineInternal(
#else
(static_params.depth_bounds_test_enable ? VK_TRUE : VK_FALSE);
#endif
depth_stencil_info.stencilTestEnable = (static_params.stencil_test_enable ? VK_TRUE : VK_FALSE);
depth_stencil_info.front.failOp = static_params.stencil_front.failOp;
depth_stencil_info.front.passOp = static_params.stencil_front.passOp;
depth_stencil_info.front.depthFailOp = static_params.stencil_front.depthFailOp;
depth_stencil_info.front.compareOp = static_params.stencil_front.compareOp;
depth_stencil_info.back.failOp = static_params.stencil_back.failOp;
depth_stencil_info.back.passOp = static_params.stencil_back.passOp;
depth_stencil_info.back.depthFailOp = static_params.stencil_back.depthFailOp;
depth_stencil_info.back.compareOp = static_params.stencil_back.compareOp;
depth_stencil_info.minDepthBounds = static_params.depth_min_bounds;
depth_stencil_info.maxDepthBounds = static_params.depth_max_bounds;
@@ -711,6 +489,8 @@ void CreatePipelineInternal(
vk::DynamicState::eDepthCompareOp,
vk::DynamicState::eDepthBiasEnable,
vk::DynamicState::eDepthBias,
vk::DynamicState::eStencilTestEnable,
vk::DynamicState::eStencilOp,
vk::DynamicState::eStencilCompareMask,
vk::DynamicState::eStencilReference,
vk::DynamicState::eStencilWriteMask,
@@ -741,8 +521,9 @@ void CreatePipelineInternal(
pipeline_info.pVertexInputState = mesh ? nullptr : &vertex_input_info;
pipeline_info.pInputAssemblyState = mesh ? nullptr : &input_assembly;
vk::PipelineTessellationStateCreateInfo tessellation_state {};
tessellation_state.patchControlPoints = 3;
pipeline_info.pTessellationState = (rect_list ? &tessellation_state : nullptr);
tessellation_state.patchControlPoints =
tessellation ? vs_input_info.tess.input_control_points : 3u;
pipeline_info.pTessellationState = (rect_list || tessellation) ? &tessellation_state : nullptr;
pipeline_info.pViewportState = &viewport_state;
pipeline_info.pRasterizationState = &rasterizer;
pipeline_info.pMultisampleState = &multisampling;
+11 -15
View File
@@ -170,38 +170,34 @@ private:
void DrawAuto(uint64_t submit_id, CommandBuffer& buffer, const DrawAutoArgs& args);
struct GraphicsBindings {
PreparedBindings vertex;
std::array<PreparedBindings, 3> vertex;
std::optional<PreparedBindings> pixel;
};
[[nodiscard]] TextureBinding ResolveTexture(const ShaderRecompiler::IR::ImageResource& resource,
const ShaderRecompiler::IR::DescriptorValue& value);
[[nodiscard]] GraphicsBindings PrepareGraphicsBindings(const ShaderStageRuntime& vertex,
const ShaderStageRuntime& pixel,
bool pixel_active);
void ResolveRenderColorTarget(uint64_t submit_id, CommandBuffer& buffer,
RenderColorInfo& target, uint32_t render_target_slice_offset = 0,
uint32_t render_target_slot = UINT32_MAX,
void PrepareGraphicsBindings(std::span<PreparedBindings* const> stages,
std::span<RenderColorInfo> colors);
void ResolveRenderColorTarget(CommandBuffer& buffer, RenderColorInfo& target,
uint32_t render_target_slice_offset, uint32_t render_target_slot,
bool ignore_target_mask = false, bool exact_format = false);
void ResolveRenderDepthTarget(uint64_t submit_id, CommandBuffer& buffer,
RenderDepthInfo& target);
[[nodiscard]] bool PrepareDrawRenderState(uint64_t submit_id, CommandBuffer& buffer,
void ResolveRenderDepthTarget(CommandBuffer& buffer, RenderDepthInfo& target);
[[nodiscard]] bool DepthStencilCopy(CommandBuffer& buffer);
[[nodiscard]] bool PrepareDrawRenderState(CommandBuffer& buffer,
const DrawCallInfo& draw,
uint32_t render_target_slice_offset,
bool log_setup_phases, DrawRenderState& state);
DrawRenderState& state);
void ExecutePreparedDraw(uint64_t submit_id, CommandBuffer& buffer, const DrawCallInfo& draw,
DrawRenderState& state, vk::PrimitiveTopology topology,
const DrawEmitInfo& emit, const DrawIndexBufferSource& index_source,
bool primitive_restart_enable, bool log_pipeline_phase,
bool set_bind_debug, bool set_auto_debug);
bool primitive_restart_enable);
[[nodiscard]] RenderState AcquireRenderTargets(CommandBuffer& buffer, RenderColorInfo* colors,
uint32_t color_count, RenderDepthInfo& depth,
const std::optional<PreparedBindings>& pixel = std::nullopt);
[[nodiscard]] bool ResolveColorTargets(uint64_t submit_id, CommandBuffer& buffer,
[[nodiscard]] bool ResolveColorTargets(CommandBuffer& buffer,
uint32_t render_target_slice_offset);
void BindImage(ImageId id, bool storage);
void BindRenderTarget(ImageId id);
void TrackImageBinding(ImageId id);
void ResetBindings();
[[nodiscard]] bool TryConsumeComputeMetaClear(const ShaderComputeInputInfo& input,
const CommandBuffer& buffer);
@@ -36,18 +36,12 @@
#include <vector>
namespace Libs::Graphics {
static uint64_t BufferDescriptorSize(const ShaderBufferResource& descriptor) {
const uint64_t records = descriptor.NumRecords();
const uint64_t stride = descriptor.Stride();
return stride == 0 ? records : records * stride;
}
static bool FillSourcesDisjoint(std::span<const ShaderRecompiler::IR::DescriptorValue> sources,
GuestRange destination, uint32_t output_buffer = UINT32_MAX) {
for (uint32_t i = 0; i < sources.size(); ++i) {
if (i == output_buffer) continue;
const auto source = DecodeNativeDescriptor<ShaderBufferResource>(sources[i]);
const auto bytes = BufferDescriptorSize(source);
const auto bytes = source.GetSize();
if (source.Base48() < destination.End() && destination.address < source.Base48() + bytes)
return false;
}
@@ -67,7 +61,7 @@ bool RenderExecutor::TryConsumeComputeMetaClear(const ShaderComputeInputInfo& in
const auto descriptor = DecodeNativeDescriptor<ShaderBufferResource>(resources.buffers[i]);
// A metadata resource that is also read is not proven to be a full overwrite. Execute it
// conservatively instead of replacing the dispatch with a coarse full-surface clear.
if (cache.IsMeta(descriptor.Base48()) && (!resource.written || resource.read)) {
if ((!resource.written || resource.read) && cache.IsMeta(descriptor.Base48())) {
return false;
}
}
@@ -115,7 +109,7 @@ bool ResolveComputeBufferFill(const ShaderComputeInputInfo& input, uint32_t grou
const uint64_t invocations = input.dispatch_thread_dimensions
? group_x
: static_cast<uint64_t>(group_x) * input.threads_num[0];
const auto size = BufferDescriptorSize(descriptor);
const auto size = descriptor.GetSize();
if (invocations != descriptor.NumRecords() || size == 0 || size > UINT32_MAX ||
(input.dispatch_thread_dimensions &&
(group_x % input.threads_num[0] != 0 || input.dispatch_threads_num[0] != group_x ||
@@ -178,7 +172,7 @@ bool RenderExecutor::TryConsumeComputeImageClear(const ShaderComputeInputInfo& i
1, view.base_layer, view.layer_count};
vk::ClearValue clear {};
clear.depthStencil = vk::ClearDepthStencilValue {0.0f, fill.value};
cache.ClearImage(command, binding.image_id, range, clear);
cache.ClearImage(command, binding.image_id, image.backing.format, range, clear);
return true;
}
ShaderBufferResource descriptor;
@@ -189,14 +183,6 @@ bool RenderExecutor::TryConsumeComputeImageClear(const ShaderComputeInputInfo& i
return false;
}
if (!cache.ClearImageFromBuffer(command, descriptor.Base48(), size, packed_clear)) {
// Track deferred DCC state while the original dispatch writes the metadata allocation.
cache.TrackDccFill(descriptor.Base48(), size, packed_clear);
static std::atomic<uint32_t> logged_metadata_clears {0};
if (logged_metadata_clears.fetch_add(1, std::memory_order_relaxed) < 32) {
LOGF("GraphicsRenderDispatchDirect: metadata fill shader=0x%016" PRIx64
" addr=0x%016" PRIx64 " size=0x%016" PRIx64 " value=0x%08" PRIx32 "\n",
input.stage.program->shader_hash, descriptor.Base48(), size, packed_clear);
}
return false;
}
static std::atomic<uint32_t> logged_clears {0};
@@ -220,6 +206,17 @@ void RenderExecutor::DispatchDirect(uint64_t submit_id, CommandBuffer& buffer,
thread_group_x, thread_group_y, thread_group_z, mode,
sh_ctx.GetCs().cs_regs.data_addr);
if (thread_group_x == 0 || thread_group_y == 0 || thread_group_z == 0) {
static std::atomic<uint32_t> log_count {0};
if (log_count.fetch_add(1, std::memory_order_relaxed) < 32) {
LOGF("GraphicsRenderDispatchDirect: skipping zero-sized dispatch groups=%ux%ux%u "
"mode=0x%08" PRIx32 " shader=0x%016" PRIx64 "\n",
thread_group_x, thread_group_y, thread_group_z, mode,
sh_ctx.GetCs().cs_regs.data_addr);
}
return;
}
Common::LockGuard lock(m_context.GetMutex());
if (sh_ctx.GetCs().cs_regs.data_addr == 0) {
LOGF("GraphicsRenderDispatchDirect: temporary: ignoring dispatch with null CS shader, "
@@ -228,7 +225,7 @@ void RenderExecutor::DispatchDirect(uint64_t submit_id, CommandBuffer& buffer,
return;
}
if (!ShaderAddressValid(sh_ctx.GetCs().cs_regs.data_addr)) {
if (sh_ctx.GetCs().cs_regs.data_addr == 0) {
return;
}
@@ -264,9 +261,6 @@ void RenderExecutor::DispatchDirect(uint64_t submit_id, CommandBuffer& buffer,
input_info.dispatch_threads_num[2] = thread_group_z;
}
const uint32_t frame_num = static_cast<uint32_t>(m_context.GetGpu().GetFrameNum());
const bool large_workgroup =
(input_info.threads_num[0] * input_info.threads_num[1] * input_info.threads_num[2] >= 512);
const auto& program = *input_info.stage.program;
const auto& resources = input_info.stage.resources;
if (TryConsumeComputeMetaClear(input_info, buffer)) {
@@ -278,14 +272,17 @@ void RenderExecutor::DispatchDirect(uint64_t submit_id, CommandBuffer& buffer,
ResetBindings();
return;
}
const auto sampled_images = std::count_if(
program.info.images.begin(), program.info.images.end(), [](const auto& image) {
return image.resource_class == ShaderRecompiler::IR::ImageResourceClass::Sampled;
});
const bool large_workgroup =
(input_info.threads_num[0] * input_info.threads_num[1] * input_info.threads_num[2] >= 512);
const bool has_sampler = !program.info.samplers.empty();
static std::atomic<uint32_t> dispatch_log_count {0};
if ((large_workgroup || has_sampler) &&
dispatch_log_count.fetch_add(1, std::memory_order_relaxed) < 512) {
const auto sampled_images = std::count_if(
program.info.images.begin(), program.info.images.end(), [](const auto& image) {
return image.resource_class == ShaderRecompiler::IR::ImageResourceClass::Sampled;
});
const uint32_t frame_num = static_cast<uint32_t>(m_context.GetGpu().GetFrameNum());
LOGF("GraphicsRenderDispatchDirect: frame=%u shader=0x%016" PRIx64
" groups=%ux%ux%u mode=0x%08" PRIx32 " local=%ux%ux%u "
"buffers=%zu textures=%zu sampled=%zu storage=%zu samplers=%zu push=%u\n",
@@ -361,27 +358,16 @@ void RenderExecutor::DispatchDirect(uint64_t submit_id, CommandBuffer& buffer,
}
}
if (thread_group_x == 0 || thread_group_y == 0 || thread_group_z == 0) {
static std::atomic<uint32_t> log_count {0};
if (log_count.fetch_add(1, std::memory_order_relaxed) < 32) {
LOGF("GraphicsRenderDispatchDirect: skipping zero-sized dispatch groups=%ux%ux%u "
"mode=0x%08" PRIx32 " shader=0x%016" PRIx64 "\n",
thread_group_x, thread_group_y, thread_group_z, mode,
sh_ctx.GetCs().cs_regs.data_addr);
}
return;
}
buffer.EndRendering();
auto& pipeline =
m_context.GetPipelineCache().CreateComputePipeline(input_info, compute_program);
m_context.GetPipelineCache().GetComputePipeline(input_info, compute_program);
auto bindings = PrepareBindings(input_info.stage);
FindBuffers(bindings);
if (program.info.uses_dma) {
m_context.GetGpuResources().PrepareBda();
m_context.PrepareBda();
}
RebindBuffers(bindings);
RebindImages(bindings);
RebindBuffers(bindings);
auto vk_buffer = buffer.Handle();
PreparedBindings* descriptor_stage = &bindings;
@@ -14,7 +14,8 @@ RenderContext::RenderContext(GraphicContext& graphics)
: m_graphics(graphics), m_render_executor(*this), m_command_scheduler(*this, graphics),
m_descriptor_heap(graphics, m_command_scheduler.GetMasterSemaphore()),
m_pipeline_cache(graphics), m_sampler_cache(graphics),
m_gpu_resources(graphics, m_command_scheduler) {
m_buffer_cache(graphics, m_command_scheduler, m_page_manager, m_texture_cache),
m_texture_cache(graphics, m_command_scheduler, m_page_manager, m_buffer_cache) {
EXIT_NOT_IMPLEMENTED(!Common::Thread::IsMainThread());
}
@@ -27,12 +28,10 @@ void RenderContext::InitializeGpu(VideoOut::VideoOutDriver* video_out) {
EXIT_IF(m_gpu != nullptr);
m_video_out = video_out;
m_gpu = std::make_unique<GuestGpu>(*this);
m_gpu_resources.SetGpu(m_gpu.get());
}
void RenderContext::ShutdownGpu() {
if (m_gpu != nullptr) {
m_gpu_resources.SetGpu(nullptr);
m_gpu->Shutdown();
m_gpu.reset();
}
@@ -55,6 +54,88 @@ VideoOut::VideoOutDriver& RenderContext::GetVideoOut() const {
return *m_video_out;
}
bool RenderContext::HandleFault(PageFaultAccess access, uint64_t fault_vaddr) noexcept {
// The host reports the faulting byte, not the instruction's access width. Both caches
// resolve its page; guessing a width can cross the end of a valid guest mapping.
constexpr uint64_t fault_size = 1;
if (!IsMapped(fault_vaddr, fault_size)) {
return false;
}
if (access == PageFaultAccess::Write) {
m_buffer_cache.InvalidateMemory(fault_vaddr, fault_size);
m_texture_cache.InvalidateMemory(fault_vaddr, fault_size);
} else {
m_buffer_cache.ReadMemory(fault_vaddr, fault_size);
}
return true;
}
bool RenderContext::InvalidateMemory(uint64_t vaddr, uint64_t size) {
if (!IsMapped(vaddr, size)) {
return false;
}
m_buffer_cache.InvalidateMemory(vaddr, size);
m_texture_cache.InvalidateMemory(vaddr, size);
return true;
}
bool RenderContext::IsMapped(uint64_t vaddr, uint64_t size) const noexcept {
if (!GuestRange {vaddr, size}.Valid()) {
return false;
}
std::shared_lock lock(m_mapped_ranges_mutex);
return m_mapped_ranges.Contains(vaddr, size);
}
void RenderContext::MapMemory(uint64_t vaddr, uint64_t size) {
std::lock_guard lock(m_mapped_ranges_mutex);
m_mapped_ranges.Add(vaddr, size);
}
void RenderContext::UnmapMemory(uint64_t vaddr, uint64_t size) {
if (CommandScheduler::InDeferredOperation()) {
EXIT("unsupported memory unmap from an asynchronous GPU completion, "
"addr=0x%016" PRIx64 " size=0x%016" PRIx64 "\n",
vaddr, size);
}
const auto unmap = [this, vaddr, size] {
if (m_command_scheduler.Active()) {
const auto tick = m_command_scheduler.CurrentTick();
m_command_scheduler.Finish();
m_command_scheduler.WaitPriorityOperations(tick);
}
m_buffer_cache.InvalidateMemory(vaddr, size);
m_texture_cache.UnmapMemory(vaddr, size);
std::lock_guard lock(m_mapped_ranges_mutex);
m_mapped_ranges.Subtract(vaddr, size);
};
// Shutdown still owns the GPU while queued rendering drains, but its command lane no
// longer accepts external work. Use the guest GPU's state for the teardown route.
if (m_gpu == nullptr || m_gpu->IsStopping()) {
unmap();
return;
}
m_gpu->SendCommandSync(unmap);
}
void RenderContext::PrepareBda() {
std::shared_lock lock(m_mapped_ranges_mutex);
m_mapped_ranges.ForEach([this](uint64_t start, uint64_t end) {
m_buffer_cache.SynchronizeBuffersInRange(start, end - start);
});
m_fault_process_pending = true;
}
void RenderContext::RunGarbageCollector() {
if (m_fault_process_pending) {
m_fault_process_pending = false;
m_buffer_cache.ProcessFaultBuffer();
}
m_texture_cache.ProcessDownloadImages();
m_texture_cache.RunGarbageCollector();
m_buffer_cache.RunGarbageCollector();
}
void RenderContext::AddInterruptEq(LibKernel::EventQueue::KernelEqueue eq, int event_id) {
Common::LockGuard lock(m_interrupt_mutex);
+19 -5
View File
@@ -5,8 +5,9 @@
#include "common/assert.h"
#include "common/common.h"
#include "common/threads.h"
#include "graphics/host_gpu/pageManager.h"
#include "graphics/host_gpu/rangeSet.h"
#include "graphics/host_gpu/renderer/cache/bufferCache.h"
#include "graphics/host_gpu/renderer/cache/gpuResourceManager.h"
#include "graphics/host_gpu/renderer/cache/samplerCache.h"
#include "graphics/host_gpu/renderer/cache/textureCache.h"
#include "graphics/host_gpu/renderer/commandScheduler.h"
@@ -15,6 +16,7 @@
#include "kernel/eventQueue.h"
#include <memory>
#include <shared_mutex>
#include <vector>
namespace Libs::VideoOut {
@@ -42,11 +44,18 @@ public:
PipelineCache& GetPipelineCache() { return m_pipeline_cache; }
DescriptorHeap& GetDescriptorHeap() { return m_descriptor_heap; }
SamplerCache& GetSamplerCache() { return m_sampler_cache; }
GpuResourceManager& GetGpuResources() { return m_gpu_resources; }
BufferCache& GetBufferCache() { return m_gpu_resources.GetBufferCache(); }
TextureCache& GetTextureCache() { return m_gpu_resources.GetTextureCache(); }
BufferCache& GetBufferCache() { return m_buffer_cache; }
TextureCache& GetTextureCache() { return m_texture_cache; }
RenderExecutor& GetRenderExecutor() { return m_render_executor; }
[[nodiscard]] bool HandleFault(PageFaultAccess access, uint64_t fault_vaddr) noexcept;
[[nodiscard]] bool InvalidateMemory(uint64_t vaddr, uint64_t size);
[[nodiscard]] bool IsMapped(uint64_t vaddr, uint64_t size) const noexcept;
void MapMemory(uint64_t vaddr, uint64_t size);
void UnmapMemory(uint64_t vaddr, uint64_t size);
void PrepareBda();
void RunGarbageCollector();
void AddInterruptEq(LibKernel::EventQueue::KernelEqueue eq, int event_id);
void DeleteInterruptEq(LibKernel::EventQueue::KernelEqueue eq, int event_id);
void TriggerInterrupt(int event_id, uint32_t context_id);
@@ -64,9 +73,14 @@ private:
DescriptorHeap m_descriptor_heap;
PipelineCache m_pipeline_cache;
SamplerCache m_sampler_cache;
GpuResourceManager m_gpu_resources;
PageManager m_page_manager;
BufferCache m_buffer_cache;
TextureCache m_texture_cache;
mutable std::shared_mutex m_mapped_ranges_mutex;
RangeSet m_mapped_ranges;
std::unique_ptr<GuestGpu> m_gpu;
VideoOut::VideoOutDriver* m_video_out = nullptr;
bool m_fault_process_pending = false;
Common::Mutex m_interrupt_mutex;
std::vector<InterruptEqRegistration> m_interrupt_eqs;
+241 -287
View File
@@ -2,6 +2,7 @@
#include "common/assert.h"
#include "common/common.h"
#include "common/emulatorConfig.h"
#include "common/file.h"
#include "common/logging/log.h"
#include "common/profiler.h"
@@ -15,6 +16,7 @@
#include "graphics/host_gpu/renderer/colorRenderTarget.h"
#include "graphics/host_gpu/renderer/debug.h"
#include "graphics/host_gpu/renderer/depthRenderTarget.h"
#include "graphics/host_gpu/renderer/image/textureCommon.h"
#include "graphics/host_gpu/renderer/pipeline/pipelineCache.h"
#include "graphics/host_gpu/renderer/pipeline/shaderResourceBarrier.h"
#include "graphics/host_gpu/renderer/render.h"
@@ -46,42 +48,34 @@
namespace Libs::Graphics {
int32_t ResolveVertexOffset(uint32_t index_offset, const ShaderVertexInputInfo& vs_input_info) {
if (index_offset != 0 || !vs_input_info.fetch_embedded) {
return static_cast<int32_t>(index_offset);
std::pair<int32_t, uint32_t> ResolveDrawOffsets(uint32_t index_offset,
const ShaderVertexInputInfo& vs_input_info) {
auto vertex_offset = static_cast<int32_t>(index_offset);
uint32_t instance_offset = 0;
if (!vs_input_info.fetch_embedded) {
return {vertex_offset, instance_offset};
}
EXIT_IF(!vs_input_info.stage);
const auto& program = *vs_input_info.stage.program;
const auto& resources = vs_input_info.stage.resources;
if (program.info.vertex_offset_sgpr >= static_cast<int32_t>(program.user_data_base)) {
if (index_offset == 0 &&
program.info.vertex_offset_sgpr >= static_cast<int32_t>(program.user_data_base)) {
const auto index =
static_cast<uint32_t>(program.info.vertex_offset_sgpr) - program.user_data_base;
if (index < resources.user_data.size()) {
return static_cast<int32_t>(resources.user_data[index]);
vertex_offset = static_cast<int32_t>(resources.user_data[index]);
}
}
return 0;
}
uint32_t ResolveInstanceOffset(const ShaderVertexInputInfo& vs_input_info) {
if (!vs_input_info.fetch_embedded) {
return 0;
}
EXIT_IF(!vs_input_info.stage);
const auto& program = *vs_input_info.stage.program;
const auto& resources = vs_input_info.stage.resources;
if (program.info.instance_offset_sgpr >= static_cast<int32_t>(program.user_data_base)) {
const auto index =
static_cast<uint32_t>(program.info.instance_offset_sgpr) - program.user_data_base;
if (index < resources.user_data.size()) {
return resources.user_data[index];
instance_offset = resources.user_data[index];
}
}
return 0;
return {vertex_offset, instance_offset};
}
static std::atomic<uint32_t> g_draw_state_log_count = 0;
@@ -316,24 +310,12 @@ static void LogDrawInputState(const CommandBuffer& buffer, const RenderColorInfo
static void SetGraphicsDynamicParams(const CommandBuffer& buffer, vk::CommandBuffer vk_buffer,
const ShaderVertexInputInfo& vs_input_info,
const RenderColorInfo* colors, uint32_t color_count,
const RenderDepthInfo& depth) {
const RenderDepthInfo& depth, const RenderState& rendering) {
KYTY_PROFILER_FUNCTION();
EXIT_IF(colors == nullptr);
const auto& ctx = buffer.GetRegisters();
const auto& vp = ctx.GetScreenViewport();
vk::Extent2D framebuffer_extent {};
if (color_count > 0 && colors[0].image_id) {
framebuffer_extent = colors[0].Extent();
} else if (depth.image_id) {
framebuffer_extent = {depth.desc.info.extent.width, depth.desc.info.extent.height};
} else {
const auto& limits = buffer.GetGraphics().GetPhysicalDeviceProperties().limits;
framebuffer_extent = {limits.maxFramebufferWidth, limits.maxFramebufferHeight};
}
const auto& vp = ctx.GetScreenViewport();
const vk::Extent2D framebuffer_extent {rendering.width, rendering.height};
const auto& outputs = vs_input_info.stage.program->info.outputs;
const bool indexed_viewports =
std::any_of(outputs.begin(), outputs.end(), [](const auto& output) {
@@ -411,19 +393,16 @@ static void SetGraphicsDynamicParams(const CommandBuffer& buffer, vk::CommandBuf
vk_buffer.setDepthBias(constant_factor, poly_offset.clamp, slope_factor);
}
vk_buffer.setStencilTestEnable(depth.stencil_test_enable ? VK_TRUE : VK_FALSE);
if (depth.stencil_test_enable) {
vk_buffer.setStencilCompareMask(vk::StencilFaceFlagBits::eFront,
depth.stencil_dynamic_front.compareMask);
vk_buffer.setStencilCompareMask(vk::StencilFaceFlagBits::eBack,
depth.stencil_dynamic_back.compareMask);
vk_buffer.setStencilWriteMask(vk::StencilFaceFlagBits::eFront,
depth.stencil_dynamic_front.writeMask);
vk_buffer.setStencilWriteMask(vk::StencilFaceFlagBits::eBack,
depth.stencil_dynamic_back.writeMask);
vk_buffer.setStencilReference(vk::StencilFaceFlagBits::eFront,
depth.stencil_dynamic_front.reference);
vk_buffer.setStencilReference(vk::StencilFaceFlagBits::eBack,
depth.stencil_dynamic_back.reference);
const auto set_stencil = [&](vk::StencilFaceFlagBits face, const vk::StencilOpState& state) {
vk_buffer.setStencilOp(face, state.failOp, state.passOp, state.depthFailOp, state.compareOp);
vk_buffer.setStencilCompareMask(face, state.compareMask);
vk_buffer.setStencilWriteMask(face, state.writeMask);
vk_buffer.setStencilReference(face, state.reference);
};
set_stencil(vk::StencilFaceFlagBits::eFront, depth.stencil_front);
set_stencil(vk::StencilFaceFlagBits::eBack, depth.stencil_back);
}
#if defined(__APPLE__)
@@ -431,15 +410,11 @@ static void SetGraphicsDynamicParams(const CommandBuffer& buffer, vk::CommandBuf
// eColorWriteEnableEXT dynamic state and relies on the static colorWriteMask instead.
#else
vk::Bool32 enable[RENDER_COLOR_ATTACHMENTS_MAX] = {};
// Color-control operation selects special color-buffer paths, not the normal component write
// mask. Attachment availability therefore follows the target write mask.
for (uint32_t i = 0; i < color_count; i++) {
enable[i] = render_target_mask_slot(ctx.GetRenderTargetMask(), colors[i].target_slot) != 0
? VK_TRUE
: VK_FALSE;
for (uint32_t slot = 0; slot < rendering.num_color_attachments; slot++) {
enable[slot] = rendering.color_attachments[slot].image_view != nullptr;
}
if (color_count != 0) {
vk_buffer.setColorWriteEnableEXT(color_count, enable);
if (rendering.num_color_attachments != 0) {
vk_buffer.setColorWriteEnableEXT(rendering.num_color_attachments, enable);
}
#endif
}
@@ -447,7 +422,7 @@ static void SetGraphicsDynamicParams(const CommandBuffer& buffer, vk::CommandBuf
static bool DrawHasValidVertexShader(const HW::Shader& sh_ctx) {
const auto& vs = sh_ctx.GetVs();
return ShaderAddressValid(vs.es_regs.data_addr);
return vs.es_regs.data_addr != 0;
}
static bool PixelShaderHasDepthOrCoverageSideEffects(const HW::ShaderRegisters& sh_regs) {
@@ -461,17 +436,19 @@ struct DrawRenderState {
RenderColorInfo color_info[RENDER_COLOR_ATTACHMENTS_MAX] = {};
uint32_t color_count = 0;
bool ps_active = true;
ShaderVertexInputInfo vs_input_info;
std::array<ShaderVertexInputInfo, 3> vertex_info;
ShaderPixelInputInfo ps_input_info;
PipelineCache::GraphicsPrograms programs;
};
struct DrawCallInfo {
const char* name = nullptr;
CommandBufferDebugOp debug_op = CommandBufferDebugOp::DrawIndex;
uint32_t index_count = 0;
uint32_t instance_count = 0;
uint32_t first_instance = 0;
[[nodiscard]] bool IsIndexed() const { return debug_op == CommandBufferDebugOp::DrawIndex; }
[[nodiscard]] const char* Name() const { return IsIndexed() ? "DrawIndex" : "DrawIndexAuto"; }
};
RenderState RenderExecutor::AcquireRenderTargets(CommandBuffer& buffer, RenderColorInfo* colors,
@@ -483,19 +460,15 @@ RenderState RenderExecutor::AcquireRenderTargets(CommandBuffer& buffer, RenderCo
state.width = std::numeric_limits<uint32_t>::max();
state.height = std::numeric_limits<uint32_t>::max();
state.num_layers = std::numeric_limits<uint32_t>::max();
state.num_color_attachments = color_count;
state.num_color_attachments = 0;
uint32_t attachment_samples = 0;
for (uint32_t i = 0; i < color_count; i++) {
auto& target = colors[i];
EXIT_IF(!target.image_id);
const auto old_image = cache.m_slot_images.try_get(target.image_id);
if (old_image == nullptr || (!old_image->registered && !old_image->info.data.Empty()) ||
old_image->binding.needs_rebind) {
if (old_image != nullptr) {
old_image->binding = {};
}
target.image_id = cache.FindImage(target.desc);
BindRenderTarget(target.image_id);
const auto owner = cache.m_slot_images.try_get(target.image_id);
if (owner == nullptr || (!owner->registered && !owner->info.data.Empty()) ||
owner->binding.needs_rebind) {
EXIT("color target changed after render-state discovery\n");
}
const auto image_view = cache.FindRenderTarget(target.image_id, target.desc);
auto& image = cache.GetImage(target.image_id);
@@ -529,7 +502,8 @@ RenderState RenderExecutor::AcquireRenderTargets(CommandBuffer& buffer, RenderCo
state.width = std::min(state.width, extent.width);
state.height = std::min(state.height, extent.height);
state.num_layers = std::min(state.num_layers, view.layer_count);
auto& attachment = state.color_attachments[i];
state.num_color_attachments = std::max(state.num_color_attachments, target.target_slot + 1);
auto& attachment = state.color_attachments[target.target_slot];
attachment.image_view = image_view;
attachment.image_layout = layout;
}
@@ -544,11 +518,11 @@ RenderState RenderExecutor::AcquireRenderTargets(CommandBuffer& buffer, RenderCo
!cache.ClearMeta(metadata.range.address)) {
EXIT("failed to acquire HTile metadata for a depth clear\n");
}
depth.depth_meta_clear_enable =
const bool meta_clear =
metadata.kind == ImageMetadataKind::Htile &&
cache.IsMetaCleared(metadata.range.address, depth.desc.view_info.base_layer);
depth.depth_load_clear_enable = depth.depth_clear_enable || depth.depth_meta_clear_enable;
if (depth.depth_meta_clear_enable &&
depth.depth_load_clear_enable = depth.depth_clear_enable || meta_clear;
if (meta_clear &&
!cache.TouchMeta(metadata.range.address, depth.desc.view_info.base_layer, false)) {
EXIT("failed to consume HTile clear state\n");
}
@@ -635,7 +609,7 @@ static bool DrawHasActivePixelShader(const CommandBuffer& buffer) {
const auto& ctx = buffer.GetRegisters();
const auto& sh_regs = ctx.GetShaderRegisters();
const bool has_color_output = (ctx.GetRenderTargetMask() & sh_regs.m_cbShaderMask) != 0;
return ShaderAddressValid(buffer.GetShaders().GetPs().ps_regs.data_addr) &&
return buffer.GetShaders().GetPs().ps_regs.data_addr != 0 &&
(has_color_output || PixelShaderHasDepthOrCoverageSideEffects(sh_regs));
}
@@ -662,7 +636,6 @@ static bool ConsumeMetadataColorOperation(const CommandBuffer& buffer) {
}
struct DrawEmitInfo {
bool indexed = false;
int32_t vertex_offset = 0;
uint32_t first_vertex = 0;
uint32_t first_instance = 0;
@@ -716,6 +689,7 @@ struct PreparedVertexBuffers {
std::array<vk::Buffer, MaxBuffers> buffers {};
std::array<vk::DeviceSize, MaxBuffers> offsets {};
std::array<vk::DeviceSize, MaxBuffers> sizes {};
uint32_t count = 0;
};
@@ -800,6 +774,7 @@ static PreparedVertexBuffers AcquireVertexBuffers(CommandBuffer& b
prepared.buffers[i] = range->binding.first->Handle();
prepared.offsets[i] = range->binding.second + vertex.addr - range->base_address;
prepared.sizes[i] = std::min(size, range->acquired_end - vertex.addr);
SetVulkanObjectNameF(
buffer.GetContext().GetGraphics().device, prepared.buffers[i],
"Kyty.VertexBuffer[slot={} guest=0x{:016x} size=0x{:x} stride={} records={}]", i,
@@ -815,8 +790,7 @@ static void SetDrawDebugPhase(CommandBuffer& buffer, uint64_t submit_id, const D
draw.instance_count, draw.first_instance);
}
static bool GetDrawTopology(const HW::UserConfig& ucfg, bool auto_draw,
vk::PrimitiveTopology& topology) {
static bool GetDrawTopology(const HW::UserConfig& ucfg, vk::PrimitiveTopology& topology) {
topology = vk::PrimitiveTopology::ePointList;
@@ -838,14 +812,12 @@ static bool GetDrawTopology(const HW::UserConfig& ucfg, bool auto_draw,
case Prospero::PrimitiveType::kTriStrip:
topology = vk::PrimitiveTopology::eTriangleStrip;
break;
case Prospero::PrimitiveType::kPatch:
if (!Config::TessellationEnabled()) return false;
[[fallthrough]];
case Prospero::PrimitiveType::kRectList:
topology = vk::PrimitiveTopology::ePatchList;
break;
case Prospero::PrimitiveType::kRectListLegacy:
if (!auto_draw) {
EXIT("unknown primitive type: %u\n", static_cast<uint32_t>(ucfg.GetPrimType()));
}
topology = vk::PrimitiveTopology::eTriangleStrip;
topology = vk::PrimitiveTopology::ePatchList;
break;
case Prospero::PrimitiveType::kQuadListLegacy:
topology = vk::PrimitiveTopology::eTriangleFan;
@@ -863,8 +835,8 @@ static bool GetDrawTopology(const HW::UserConfig& ucfg, bool auto_draw,
return true;
}
static bool ResolvePrimitiveRestart(const CommandBuffer& buffer, vk::PrimitiveTopology topology,
uint32_t index_type_and_size) {
static bool ResolvePrimitiveRestart(const CommandBuffer& buffer,
const DrawIndexBufferSource& source) {
const auto control = buffer.GetUserConfig().GetPrimitiveResetControl();
EXIT_NOT_IMPLEMENTED((control & ~0x3u) != 0);
if ((control & 0x1u) == 0) {
@@ -876,66 +848,33 @@ static bool ResolvePrimitiveRestart(const CommandBuffer& buffer, vk::PrimitiveTo
case Prospero::PrimitiveType::kTriStrip: break;
default: return false;
}
if (topology != vk::PrimitiveTopology::eLineStrip &&
topology != vk::PrimitiveTopology::eTriangleStrip &&
topology != vk::PrimitiveTopology::eTriangleFan) {
return false;
}
uint32_t index_mask = 0;
switch (static_cast<Prospero::IndexType>(index_type_and_size)) {
case Prospero::IndexType::kIndex8: index_mask = 0xffu; break;
case Prospero::IndexType::kIndex16: index_mask = 0xffffu; break;
case Prospero::IndexType::kIndex32: index_mask = 0xffffffffu; break;
default: EXIT("unknown index_type_and_size: %u\n", index_type_and_size);
}
const auto reset_index = buffer.GetRegisters().GetPrimitiveResetIndex();
const auto element_size = source.guest_element_size;
const auto index_mask = UINT32_MAX >> ((4 - element_size) * 8);
const auto reset_index = buffer.GetRegisters().GetPrimitiveResetIndex();
if ((control & 0x2u) != 0 && (reset_index & ~index_mask) != 0) {
return false;
}
EXIT_NOT_IMPLEMENTED((reset_index & index_mask) != index_mask);
return true;
const auto restart_index = reset_index & index_mask;
if (restart_index == index_mask) {
// Use native restart; the 8-bit path widens its marker to 0xffff.
return true;
}
// A game can set a custom reset value without using it in the index buffer.
// Keep restart off in that case; fail if we actually find the value.
// Scan before preparing draw resources: readback can restart the command buffer.
EXIT_NOT_IMPLEMENTED(source.address == 0);
const auto* indices = reinterpret_cast<const uint8_t*>(source.address);
for (uint64_t offset = 0; offset < source.size; offset += element_size) {
uint32_t index = 0;
std::memcpy(&index, indices + offset, element_size);
EXIT_NOT_IMPLEMENTED(index == restart_index);
}
return false;
}
bool RenderExecutor::PrepareDrawRenderState(uint64_t submit_id, CommandBuffer& buffer,
const DrawCallInfo& draw,
uint32_t render_target_slice_offset,
bool log_setup_phases, DrawRenderState& state) {
auto& ctx = buffer.GetRegisters();
if (ResolveColorTargets(submit_id, buffer, render_target_slice_offset)) {
return false;
}
if (log_setup_phases) {
LogDrawPhase(draw.name, "ResolveRenderColorTarget");
}
for (uint32_t slot = 0; slot < RENDER_COLOR_ATTACHMENTS_MAX; slot++) {
if (slot == 0 || (render_target_mask_slot(ctx.GetRenderTargetMask(), slot) != 0 &&
ctx.GetRenderTarget(slot).base.addr != 0)) {
ResolveRenderColorTarget(submit_id, buffer, state.color_info[state.color_count],
render_target_slice_offset, slot);
if (state.color_info[state.color_count].image_id) {
state.color_count++;
}
}
}
if (log_setup_phases) {
LogDrawPhase(draw.name, "ResolveRenderDepthTarget");
}
ResolveRenderDepthTarget(submit_id, buffer, state.depth_info);
state.ps_active = DrawHasActivePixelShader(buffer);
if (state.color_count == 0 && !state.depth_info.image_id && !state.ps_active) {
LogFramebufferSkip(draw.name, state.color_info[0], state.depth_info, buffer,
draw.index_count, 0);
return false;
}
return true;
}
static void RefreshShaders(CommandBuffer& buffer, const DrawCallInfo& draw, bool log_phases,
static void RefreshShaders(CommandBuffer& buffer, const DrawCallInfo& draw,
DrawRenderState& state) {
auto& ctx = buffer.GetRegisters();
auto& sh_ctx = buffer.GetShaders();
@@ -948,16 +887,61 @@ static void RefreshShaders(CommandBuffer& buffer, const DrawCallInfo& draw, bool
state.ps_input_info = {};
std::array<Prospero::ColorComponentMapping, RENDER_COLOR_ATTACHMENTS_MAX>
target_export_mapping {};
for (uint32_t i = 0; i < state.color_count; i++) {
target_export_mapping[state.color_info[i].target_slot] = state.color_info[i].export_mapping;
for (uint32_t slot = 0; slot < RENDER_COLOR_ATTACHMENTS_MAX; slot++) {
const auto& rt = ctx.GetRenderTarget(slot);
if (rt.base.addr != 0 && render_target_mask_slot(ctx.GetRenderTargetMask(), slot) != 0) {
target_export_mapping[slot] =
TextureGetRenderTargetFormat(rt.info.format, rt.info.channel_type,
rt.info.channel_order)
.export_mapping;
}
}
auto& pipeline_cache = buffer.GetContext().GetPipelineCache();
if (log_phases) {
LogDrawPhase(draw.name, "GetGraphicsPrograms");
if (draw.IsIndexed()) {
LogDrawPhase(draw.Name(), "GetGraphicsPrograms");
}
state.programs = pipeline_cache.GetGraphicsPrograms(
vertex_shader_info, pixel_shader_info, shader_regs, ctx, buffer.GetUserConfig(),
target_export_mapping, state.ps_active, state.vs_input_info, state.ps_input_info);
target_export_mapping, state.ps_active, state.vertex_info, state.ps_input_info);
}
bool RenderExecutor::PrepareDrawRenderState(CommandBuffer& buffer, const DrawCallInfo& draw,
uint32_t render_target_slice_offset,
DrawRenderState& state) {
state.ps_active = DrawHasActivePixelShader(buffer);
RefreshShaders(buffer, draw, state);
uint32_t mrt_mask = 0;
if (state.ps_active) {
for (const auto& output: state.ps_input_info.stage.program->info.outputs) {
if (output.kind == ShaderRecompiler::IR::StageOutputKind::Mrt) {
mrt_mask |= 1u << output.index;
}
}
}
if (draw.IsIndexed()) {
LogDrawPhase(draw.Name(), "ResolveRenderColorTarget");
}
for (uint32_t slot = 0; slot < RENDER_COLOR_ATTACHMENTS_MAX; slot++) {
if ((mrt_mask & (1u << slot)) != 0) {
ResolveRenderColorTarget(buffer, state.color_info[state.color_count],
render_target_slice_offset, slot);
if (state.color_info[state.color_count].image_id) {
state.color_count++;
}
}
}
if (draw.IsIndexed()) {
LogDrawPhase(draw.Name(), "ResolveRenderDepthTarget");
}
ResolveRenderDepthTarget(buffer, state.depth_info);
if (state.color_count == 0 && !state.depth_info.image_id && !state.ps_active) {
LogFramebufferSkip(draw.Name(), state.color_info[0], state.depth_info, buffer,
draw.index_count, 0);
return false;
}
return true;
}
static PreparedIndexBuffer PrepareIndexBuffer(CommandBuffer& buffer,
@@ -971,17 +955,14 @@ static PreparedIndexBuffer PrepareIndexBuffer(CommandBuffer& buffe
auto& stream = buffer.GetContext().GetBufferCache().GetUtilityBuffer(MemoryUsage::Stream);
prepared.offset = stream.Copy(source.host_data, source.size, 16);
prepared.buffer = stream.Handle();
SetVulkanObjectNameF(buffer.GetContext().GetGraphics().device, prepared.buffer,
"Kyty.IndexBuffer[guest=transient size=0x{:x} type={}]", source.size,
static_cast<uint32_t>(source.type));
} else {
auto [buffer_ptr, offset] =
buffer.GetContext().GetBufferCache().ObtainBuffer(source.address, source.size, false);
prepared.buffer = buffer_ptr->Handle();
prepared.offset = offset;
}
if (source.host_data != nullptr) {
SetVulkanObjectNameF(buffer.GetContext().GetGraphics().device, prepared.buffer,
"Kyty.IndexBuffer[guest=transient size=0x{:x} type={}]", source.size,
static_cast<uint32_t>(source.type));
} else {
SetVulkanObjectNameF(buffer.GetContext().GetGraphics().device, prepared.buffer,
"Kyty.IndexBuffer[guest=0x{:016x} size=0x{:x} type={}]",
source.address, source.size, static_cast<uint32_t>(source.type));
@@ -995,8 +976,9 @@ static void CommitVertexBuffers(vk::CommandBuffer vk_buffer,
EXIT_IF(prepared.buffers[i] == nullptr);
}
if (prepared.count != 0) {
vk_buffer.bindVertexBuffers(0, prepared.count, prepared.buffers.data(),
prepared.offsets.data());
// Guest descriptor bounds must survive allocation merging in the cache.
vk_buffer.bindVertexBuffers2(0, prepared.count, prepared.buffers.data(),
prepared.offsets.data(), prepared.sizes.data(), nullptr);
}
}
@@ -1008,28 +990,26 @@ static void CommitIndexBuffer(vk::CommandBuffer vk_buffer, const PreparedIndexBu
}
static void LogDrawStateIfNeeded(const CommandBuffer& buffer, const DrawCallInfo& draw,
const DrawRenderState& state, bool always_log,
bool force_legacy_rect_log, uint32_t index_type_and_size,
const DrawRenderState& state, uint32_t index_type_and_size,
const void* index_addr) {
if (!graphics_debug_dump_enabled()) {
return;
}
if (!always_log && !force_legacy_rect_log) {
if (!draw.IsIndexed() && !Prospero::IsRectList(buffer.GetUserConfig().GetPrimType())) {
return;
}
if (state.ps_active) {
LogDrawTargetState(draw.name, state.color_info[0], state.depth_info, buffer,
LogDrawTargetState(draw.Name(), state.color_info[0], state.depth_info, buffer,
state.ps_input_info, draw.index_count, 0);
}
LogDrawInputState(buffer, state.color_info[0], state.vs_input_info, index_type_and_size,
LogDrawInputState(buffer, state.color_info[0], state.vertex_info[0], index_type_and_size,
draw.index_count, index_addr);
}
static void EmitDrawPrimitives(const HW::UserConfig& ucfg, vk::CommandBuffer vk_buffer,
const ShaderVertexInputInfo& vs_input_info, const DrawCallInfo& draw,
const DrawEmitInfo& emit) {
const DrawCallInfo& draw, const DrawEmitInfo& emit) {
switch (ucfg.GetPrimType()) {
case Prospero::PrimitiveType::kPointList:
case Prospero::PrimitiveType::kLineList:
@@ -1038,7 +1018,9 @@ static void EmitDrawPrimitives(const HW::UserConfig& ucfg, vk::CommandBuffer vk_
case Prospero::PrimitiveType::kTriFan:
case Prospero::PrimitiveType::kTriStrip:
case Prospero::PrimitiveType::kRectList:
if (emit.indexed) {
case Prospero::PrimitiveType::kRectListLegacy:
case Prospero::PrimitiveType::kPatch:
if (draw.IsIndexed()) {
vk_buffer.drawIndexed(draw.index_count, draw.instance_count, 0, emit.vertex_offset,
emit.first_instance);
} else {
@@ -1046,18 +1028,10 @@ static void EmitDrawPrimitives(const HW::UserConfig& ucfg, vk::CommandBuffer vk_
emit.first_instance);
}
break;
case Prospero::PrimitiveType::kRectListLegacy:
if (emit.indexed) {
EXIT("unknown primitive type: %u\n", static_cast<uint32_t>(ucfg.GetPrimType()));
}
// Sarah
EXIT_NOT_IMPLEMENTED(!(draw.index_count == 3 && vs_input_info.buffers_num == 0));
vk_buffer.draw(4, draw.instance_count, emit.first_vertex, emit.first_instance);
break;
case Prospero::PrimitiveType::kQuadListLegacy:
EXIT_NOT_IMPLEMENTED((draw.index_count & 0x3u) != 0);
for (uint32_t i = 0; i < draw.index_count; i += 4) {
if (emit.indexed) {
if (draw.IsIndexed()) {
vk_buffer.drawIndexed(4, draw.instance_count, i, emit.vertex_offset,
emit.first_instance);
} else {
@@ -1074,16 +1048,23 @@ void RenderExecutor::ExecutePreparedDraw(uint64_t submit_id, CommandBuffer& buff
const DrawCallInfo& draw, DrawRenderState& state,
vk::PrimitiveTopology topology, const DrawEmitInfo& emit,
const DrawIndexBufferSource& index_source,
bool primitive_restart_enable, bool log_pipeline_phase,
bool set_bind_debug, bool set_auto_debug) {
bool primitive_restart_enable) {
auto& ucfg = buffer.GetUserConfig();
const bool mesh_active = state.vs_input_info.stage.program->stage == ShaderType::Mesh;
const auto vertex_stages =
std::span {state.vertex_info.data(), state.programs.VertexStageCount()};
const bool mesh_active = state.vertex_info[0].stage.program->stage == ShaderType::Mesh;
uint32_t mesh_groups = 0;
if (mesh_active) {
const auto& mesh = state.vs_input_info.mesh;
if (primitive_restart_enable || mesh.primitives_per_group == 0) {
const auto& mesh = state.vertex_info[0].mesh;
static std::atomic_bool restart_warned = false;
if (primitive_restart_enable && !restart_warned.exchange(true, std::memory_order_relaxed)) {
std::printf("Warning: primitive restart is not implemented for mesh shaders; "
"continuing draw (primitive=%u indexed=%u)\n",
static_cast<uint32_t>(ucfg.GetPrimType()), draw.IsIndexed());
}
if (mesh.primitives_per_group == 0) {
EXIT("unsupported mesh draw: primitive=%u indexed=%u restart=%u\n",
static_cast<uint32_t>(ucfg.GetPrimType()), emit.indexed, primitive_restart_enable);
static_cast<uint32_t>(ucfg.GetPrimType()), draw.IsIndexed(), primitive_restart_enable);
}
const auto primitives = mesh.InputPrimitiveCount(draw.index_count);
if (primitives == 0 || draw.instance_count == 0) {
@@ -1100,64 +1081,62 @@ void RenderExecutor::ExecutePreparedDraw(uint64_t submit_id, CommandBuffer& buff
}
}
if (mesh_active && emit.indexed) {
if (mesh_active && draw.IsIndexed()) {
// Register the original guest indices for shader reads; PrepareGraphicsBindings
// synchronizes registered BDA ranges before any draw commands are committed.
(void)m_context.GetBufferCache().FindBuffer(
index_source.address, static_cast<uint64_t>(draw.index_count) *
index_source.guest_element_size);
}
LogDrawPhase(draw.name, "PrepareBindings");
auto bindings = PrepareGraphicsBindings(state.vs_input_info.stage, state.ps_input_info.stage,
state.ps_active);
LogDrawPhase(draw.Name(), "PrepareBindings");
GraphicsBindings bindings;
std::array<PreparedBindings*, 4> descriptor_stages {};
uint32_t stage_count = 0;
for (uint32_t i = 0; i < vertex_stages.size(); i++) {
bindings.vertex[i] = PrepareBindings(state.vertex_info[i].stage);
descriptor_stages[stage_count++] = &bindings.vertex[i];
}
if (state.ps_active) {
bindings.pixel.emplace(PrepareBindings(state.ps_input_info.stage));
descriptor_stages[stage_count++] = &*bindings.pixel;
}
const auto stages = std::span {descriptor_stages.data(), stage_count};
PrepareGraphicsBindings(stages, std::span {state.color_info, state.color_count});
PreparedVertexBuffers vertex_bindings;
PreparedIndexBuffer index_binding;
if (!mesh_active) {
LogDrawPhase(draw.name, "PrepareVertexBuffers");
vertex_bindings = AcquireVertexBuffers(buffer, state.vs_input_info);
LogDrawPhase(draw.Name(), "PrepareVertexBuffers");
vertex_bindings = AcquireVertexBuffers(buffer, state.vertex_info[0]);
index_binding = PrepareIndexBuffer(buffer, index_source);
}
const auto rendering =
AcquireRenderTargets(buffer, state.color_info, state.color_count, state.depth_info,
bindings.pixel);
if (log_pipeline_phase) {
LogDrawPhase(draw.name, "CreatePipeline");
if (draw.IsIndexed()) {
LogDrawPhase(draw.Name(), "CreatePipeline");
}
auto& pipeline = m_context.GetPipelineCache().CreateGraphicsPipeline(
std::span {state.color_info, state.color_count}, state.depth_info, state.vs_input_info, buffer,
auto& pipeline = m_context.GetPipelineCache().GetGraphicsPipeline(
std::span {state.color_info, state.color_count}, state.depth_info, vertex_stages, buffer,
state.ps_active ? &state.ps_input_info : nullptr, topology, primitive_restart_enable,
state.programs.vertex, state.programs.pixel);
state.programs);
// Resource preparation above may synchronously finish and restart the scheduler. From this
// point onward, every operation targets the current command buffer and cannot touch guest
// memory.
auto vk_buffer = buffer.Handle();
if (set_bind_debug) {
SetDrawDebugPhase(buffer, submit_id, draw, 0x100u);
}
if (set_auto_debug) {
SetDrawDebugPhase(buffer, submit_id, draw, 0x200u);
}
SetDrawDebugPhase(buffer, submit_id, draw, draw.IsIndexed() ? 0x100u : 0x200u);
if (!mesh_active) {
CommitVertexBuffers(vk_buffer, vertex_bindings);
}
if (bindings.pixel.has_value()) {
if (set_auto_debug) {
SetDrawDebugPhase(buffer, submit_id, draw, 0x300u);
}
if (bindings.pixel && !draw.IsIndexed()) {
SetDrawDebugPhase(buffer, submit_id, draw, 0x300u);
}
std::array<PreparedBindings*, 2> descriptor_stages {&bindings.vertex, nullptr};
const size_t descriptor_stage_count = bindings.pixel.has_value() ? 2u : 1u;
if (bindings.pixel) {
descriptor_stages[1] = &*bindings.pixel;
}
CommitBindings(buffer, vk::PipelineBindPoint::eGraphics, pipeline,
std::span {descriptor_stages.data(), descriptor_stage_count});
CommitBindings(buffer, vk::PipelineBindPoint::eGraphics, pipeline, stages);
if (mesh_active) {
const uint32_t draw_data[] {
draw.index_count,
emit.indexed ? static_cast<uint32_t>(emit.vertex_offset) : emit.first_vertex,
draw.IsIndexed() ? static_cast<uint32_t>(emit.vertex_offset) : emit.first_vertex,
emit.first_instance, index_source.guest_element_size,
static_cast<uint32_t>(index_source.address),
static_cast<uint32_t>(index_source.address >> 32u)};
@@ -1170,8 +1149,7 @@ void RenderExecutor::ExecutePreparedDraw(uint64_t submit_id, CommandBuffer& buff
CommitIndexBuffer(vk_buffer, index_binding);
}
SetGraphicsDynamicParams(buffer, vk_buffer, state.vs_input_info, state.color_info,
state.color_count, state.depth_info);
SetGraphicsDynamicParams(buffer, vk_buffer, vertex_stages.back(), state.depth_info, rendering);
if (m_context.GetGraphics().attachment_feedback_loop_enabled) {
vk_buffer.setAttachmentFeedbackLoopEnableEXT(
rendering.depth_stencil_attachment.image_layout ==
@@ -1180,28 +1158,29 @@ void RenderExecutor::ExecutePreparedDraw(uint64_t submit_id, CommandBuffer& buff
: vk::ImageAspectFlags {});
}
LogDrawPhase(draw.name, "BeginRendering");
if (set_auto_debug) {
LogDrawPhase(draw.Name(), "BeginRendering");
if (!draw.IsIndexed()) {
SetDrawDebugPhase(buffer, submit_id, draw, 0x400u);
}
m_context.GetCommandScheduler().BeginRendering(rendering);
vk_buffer.bindPipeline(vk::PipelineBindPoint::eGraphics, pipeline.pipeline);
if (set_auto_debug) {
if (!draw.IsIndexed()) {
SetDrawDebugPhase(buffer, submit_id, draw, 0x500u);
}
if (mesh_active) {
vk_buffer.drawMeshTasksEXT(mesh_groups, draw.instance_count, 1);
} else {
EmitDrawPrimitives(ucfg, vk_buffer, state.vs_input_info, draw, emit);
EmitDrawPrimitives(ucfg, vk_buffer, draw, emit);
}
if (set_auto_debug) {
if (!draw.IsIndexed()) {
SetDrawDebugPhase(buffer, submit_id, draw, 0x600u);
}
vk::PipelineStageFlags shader_write_stages = {};
if (HasShaderBufferWrites(state.vs_input_info.stage)) {
shader_write_stages |= mesh_active ? vk::PipelineStageFlagBits::eMeshShaderEXT
: vk::PipelineStageFlagBits::eVertexShader;
for (const auto& stage: vertex_stages) {
if (HasShaderBufferWrites(stage.stage)) {
shader_write_stages |= ShaderPipelineStages(NativeShaderStage(stage.logical_stage));
}
}
if (state.ps_active && HasShaderBufferWrites(state.ps_input_info.stage)) {
shader_write_stages |= vk::PipelineStageFlagBits::eFragmentShader;
@@ -1210,8 +1189,8 @@ void RenderExecutor::ExecutePreparedDraw(uint64_t submit_id, CommandBuffer& buff
m_context.GetCommandScheduler().EndRendering();
ShaderWriteBarrier(vk_buffer, shader_write_stages);
}
LogDrawPhase(draw.name, "DrawComplete");
if (set_auto_debug) {
LogDrawPhase(draw.Name(), "DrawComplete");
if (!draw.IsIndexed()) {
SetDrawDebugPhase(buffer, submit_id, draw, 0x700u);
}
}
@@ -1235,7 +1214,8 @@ void RenderExecutor::DrawIndex(uint64_t submit_id, CommandBuffer& buffer,
return;
}
if (ConsumeMetadataColorOperation(buffer)) {
if (ConsumeMetadataColorOperation(buffer) || DepthStencilCopy(buffer) ||
ResolveColorTargets(buffer, args.render_target_slice_offset)) {
ResetBindings();
return;
}
@@ -1245,7 +1225,7 @@ void RenderExecutor::DrawIndex(uint64_t submit_id, CommandBuffer& buffer,
}
if (graphics_debug_dump_enabled()) {
sh_print("GraphicsRenderDrawIndex():Shader:", sh_ctx);
LOGF("GraphicsRenderDrawIndex():Shader:\n");
uc_print("GraphicsRenderDrawIndex():UserConfig:", ucfg);
hw_print(buffer);
@@ -1266,81 +1246,63 @@ void RenderExecutor::DrawIndex(uint64_t submit_id, CommandBuffer& buffer,
hw_check(buffer);
vk::PrimitiveTopology topology = vk::PrimitiveTopology::ePointList;
if (!GetDrawTopology(ucfg, false, topology)) {
if (!GetDrawTopology(ucfg, topology)) {
return;
}
vk::IndexType index_type = vk::IndexType::eUint16;
uint64_t index_size = 0;
bool expand_index8_to_u16 = false;
const bool primitive_restart =
ResolvePrimitiveRestart(buffer, topology, args.index_type_and_size);
DrawIndexBufferSource index_source {};
index_source.address = reinterpret_cast<uint64_t>(args.index_addr);
switch (static_cast<Prospero::IndexType>(args.index_type_and_size)) {
case Prospero::IndexType::kIndex16:
index_type = vk::IndexType::eUint16;
index_size = 2 * static_cast<uint64_t>(args.index_count);
index_source.type = vk::IndexType::eUint16;
index_source.guest_element_size = 2;
break;
case Prospero::IndexType::kIndex32:
index_type = vk::IndexType::eUint32;
index_size = 4 * static_cast<uint64_t>(args.index_count);
index_source.type = vk::IndexType::eUint32;
index_source.guest_element_size = 4;
break;
// Some games use it - need vulkan extension
case Prospero::IndexType::kIndex8:
index_type = vk::IndexType::eUint16;
index_size = static_cast<uint64_t>(args.index_count);
expand_index8_to_u16 = true;
index_source.guest_element_size = 1;
break;
default: EXIT("unknown index_type_and_size: %u\n", args.index_type_and_size);
}
index_source.size = static_cast<uint64_t>(args.index_count) * index_source.guest_element_size;
const bool primitive_restart = ResolvePrimitiveRestart(buffer, index_source);
const DrawCallInfo draw {"DrawIndex", CommandBufferDebugOp::DrawIndex, args.index_count,
args.instance_count, args.first_instance};
std::vector<uint16_t> expanded_indices;
if (expand_index8_to_u16) {
if (index_source.guest_element_size == 1) {
EXIT_NOT_IMPLEMENTED(args.index_addr == nullptr);
const auto* src = static_cast<const uint8_t*>(args.index_addr);
expanded_indices.resize(args.index_count);
for (uint32_t i = 0; i < args.index_count; i++) {
expanded_indices[i] = primitive_restart && src[i] == 0xffu ? 0xffffu : src[i];
}
index_source.host_data = expanded_indices.data();
index_source.size = expanded_indices.size() * sizeof(uint16_t);
}
DrawIndexBufferSource index_source {};
index_source.address = reinterpret_cast<uint64_t>(args.index_addr);
index_source.host_data =
expanded_indices.empty() ? nullptr : static_cast<const void*>(expanded_indices.data());
index_source.size =
expanded_indices.empty() ? index_size : expanded_indices.size() * sizeof(uint16_t);
index_source.type = index_type;
index_source.guest_element_size = static_cast<uint32_t>(index_size / args.index_count);
const DrawCallInfo draw {CommandBufferDebugOp::DrawIndex, args.index_count,
args.instance_count, args.first_instance};
DrawRenderState state {};
if (!PrepareDrawRenderState(submit_id, buffer, draw, args.render_target_slice_offset, true,
state)) {
if (!PrepareDrawRenderState(buffer, draw, args.render_target_slice_offset, state)) {
ResetBindings();
return;
}
RefreshShaders(buffer, draw, true, state);
LogDrawStateIfNeeded(buffer, draw, state, true, false, args.index_type_and_size,
LogDrawStateIfNeeded(buffer, draw, state, args.index_type_and_size,
args.index_addr);
const bool indirect = args.offset_source == DrawOffsetSource::IndirectArgs;
const auto vertex_offset =
indirect
? args.base_vertex
: ResolveVertexOffset(ucfg.GetIndexOffset(), state.vs_input_info) + args.base_vertex;
const auto [vertex_offset, instance_offset] =
indirect ? std::pair<int32_t, uint32_t> {0, args.first_instance}
: ResolveDrawOffsets(ucfg.GetIndexOffset(), state.vertex_info[0]);
DrawEmitInfo emit {};
emit.indexed = true;
emit.vertex_offset = vertex_offset;
emit.first_instance =
indirect ? args.first_instance : ResolveInstanceOffset(state.vs_input_info);
emit.vertex_offset = vertex_offset + args.base_vertex;
emit.first_instance = instance_offset;
ExecutePreparedDraw(submit_id, buffer, draw, state, topology, emit, index_source,
primitive_restart, true, true, false);
primitive_restart);
ResetBindings();
}
@@ -1363,7 +1325,8 @@ void RenderExecutor::DrawAuto(uint64_t submit_id, CommandBuffer& buffer, const D
return;
}
if (ConsumeMetadataColorOperation(buffer)) {
if (ConsumeMetadataColorOperation(buffer) || DepthStencilCopy(buffer) ||
ResolveColorTargets(buffer, args.render_target_slice_offset)) {
ResetBindings();
return;
}
@@ -1373,7 +1336,7 @@ void RenderExecutor::DrawAuto(uint64_t submit_id, CommandBuffer& buffer, const D
}
if (graphics_debug_dump_enabled()) {
sh_print("GraphicsRenderDrawIndexAuto():Shader:", sh_ctx);
LOGF("GraphicsRenderDrawIndexAuto():Shader:\n");
uc_print("GraphicsRenderDrawIndexAuto():UserConfig:", ucfg);
hw_print(buffer);
@@ -1389,26 +1352,23 @@ void RenderExecutor::DrawAuto(uint64_t submit_id, CommandBuffer& buffer, const D
hw_check(buffer);
const DrawCallInfo draw {"DrawIndexAuto", CommandBufferDebugOp::DrawIndexAuto,
const DrawCallInfo draw {CommandBufferDebugOp::DrawIndexAuto,
args.vertex_count, args.instance_count, args.first_instance};
DrawRenderState state {};
if (!PrepareDrawRenderState(submit_id, buffer, draw, args.render_target_slice_offset, false,
state)) {
ResetBindings();
return;
}
vk::PrimitiveTopology topology = vk::PrimitiveTopology::ePointList;
if (!GetDrawTopology(ucfg, true, topology)) {
if (!GetDrawTopology(ucfg, topology)) {
ResetBindings();
return;
}
DrawRenderState state {};
if (!PrepareDrawRenderState(buffer, draw, args.render_target_slice_offset, state)) {
ResetBindings();
return;
}
RefreshShaders(buffer, draw, false, state);
const bool rect_list = topology == vk::PrimitiveTopology::ePatchList;
if (rect_list && state.vs_input_info.buffers_num == 0 &&
state.vs_input_info.stage.program->param_export_mask == 0 &&
const bool rect_list = Prospero::IsRectList(ucfg.GetPrimType());
if (rect_list && state.vertex_info[0].buffers_num == 0 &&
state.vertex_info[0].stage.program->param_export_mask == 0 &&
state.ps_input_info.input_num != 0) {
if (graphics_debug_dump_enabled()) {
LOGF("DrawIndexAuto: skipping rect-list draw with no VS param exports and PS inputs: "
@@ -1420,28 +1380,22 @@ void RenderExecutor::DrawAuto(uint64_t submit_id, CommandBuffer& buffer, const D
return;
}
LogDrawStateIfNeeded(buffer, draw, state, false,
ucfg.GetPrimType() == Prospero::PrimitiveType::kRectListLegacy, 0,
nullptr);
LogDrawStateIfNeeded(buffer, draw, state, 0, nullptr);
const bool indirect = args.offset_source == DrawOffsetSource::IndirectArgs;
const auto vertex_offset =
indirect ? static_cast<int32_t>(args.first_vertex)
: ResolveVertexOffset(ucfg.GetIndexOffset(), state.vs_input_info) +
static_cast<int32_t>(args.first_vertex);
const auto [vertex_offset, instance_offset] =
indirect ? std::pair<int32_t, uint32_t> {0, args.first_instance}
: ResolveDrawOffsets(ucfg.GetIndexOffset(), state.vertex_info[0]);
DrawEmitInfo emit {};
emit.first_vertex = static_cast<uint32_t>(vertex_offset);
emit.first_instance =
indirect ? args.first_instance : ResolveInstanceOffset(state.vs_input_info);
emit.first_vertex = static_cast<uint32_t>(vertex_offset + static_cast<int32_t>(args.first_vertex));
emit.first_instance = instance_offset;
DrawIndexBufferSource index_source {};
ExecutePreparedDraw(submit_id, buffer, draw, state, topology, emit, index_source, false, false,
false, true);
ExecutePreparedDraw(submit_id, buffer, draw, state, topology, emit, index_source, false);
ResetBindings();
}
bool RenderExecutor::ResolveColorTargets(uint64_t submit_id, CommandBuffer& buffer,
uint32_t render_target_slice_offset) {
bool RenderExecutor::ResolveColorTargets(CommandBuffer& buffer, uint32_t render_target_slice_offset) {
const auto& hw = buffer.GetRegisters();
if (hw.GetColorControl().mode != 3) {
return false;
@@ -1455,8 +1409,8 @@ bool RenderExecutor::ResolveColorTargets(uint64_t submit_id, CommandBuffer& buff
RenderColorInfo src {};
RenderColorInfo dst {};
ResolveRenderColorTarget(submit_id, buffer, src, render_target_slice_offset, 0, true, true);
ResolveRenderColorTarget(submit_id, buffer, dst, render_target_slice_offset, 1, true, true);
ResolveRenderColorTarget(buffer, src, render_target_slice_offset, 0, true, true);
ResolveRenderColorTarget(buffer, dst, render_target_slice_offset, 1, true, true);
if (!src.image_id || !dst.image_id) {
return false;
}
Loaded 100 of 257 files, more files were not shown because too many files have changed in this diff. Show more