Compare commits

...
Author SHA1 Message Date
730e1a3cf3 docs: add Fedora and Fedora Atomic (Bazzite) build instructions (#1372)
* docs: add Fedora and Fedora Atomic (Bazzite) build instructions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SscrMzrvwrj17PJsfzqog9

* docs: note --nvidia for Distrobox on NVIDIA hosts

* docs: streamline Fedora build instructions

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
2026-10-11 03:24:15 +02:00
nmzik d4bb7fd786 agc: remove command buffer debug dump 2026-10-11 03:03:26 +02:00
fbea51c684 agc: implement sceAgcSetNop (#1343)
* agc: implement sceAgcSetNop

Star Wars Outlaws calls sceAgcSetNop (K2mciNVxUCE) and crashes right after
the unresolved stub returns, without the packet being disabled.

The call turns a packet into a NOP of the same length. Clear the emulator's
custom NOP code as well, so packets that the HLE encodes as NOPs, such as
end-of-pipe actions, stop executing too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* agc: keep packets disabled by sceAgcSetNop from reading as markers

A disabled packet whose first payload word carries the 0x6875 marker
signature still reached the marker handler, which can flip or stop on an
unknown marker ID. Clear the signature so the packet is skipped like any
other NOP.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* agc: tidy SetNop logging and comments

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
2026-10-11 03:00:09 +02:00
Devran Cosmo Uenal a6f4e8e7f4 AvPlayer: recycle output buffers so the decoder can run ahead 2026-10-11 02:08:27 +02:00
nmzik 448a4c8a6a launcher: reorganize panels 2026-10-11 01:44:04 +02:00
nmzik ece4e28740 gpu: store private format byte sizes in uint8_t 2026-10-11 01:25:47 +02:00
nmzik 58ba1e4cb4 docs: update README wording and grammar 2026-10-11 01:13:14 +02:00
nmzik cf77fb4353 shader: initialize NGG vertex count from active native invocations 2026-10-11 01:05:56 +02:00
nmzik 7e4e142aae gpu: enable native R8 SNORM texture sampling 2026-10-11 01:05:56 +02:00
nmzik b7b335ac63 gpu: admit block-compressed textures in PS5 R64 tiling 2026-10-11 01:05:56 +02:00
nmzik 9a5dd9faff agc: reserve driver backing before guest direct-memory allocation 2026-10-11 01:05:56 +02:00
nmzik 947627d10d docs: clarify pull request scope 2026-10-10 22:47:36 +02:00
Kelvin Oliveiraandnmzik 1cfb9522a3 controller: use Back as the touchpad click on pads without a touchpad (#1355)
* controller: use Back as the touchpad click on pads without a touchpad

Gamepads without a touchpad (Switch, Xbox, 8BitDo, ...) cannot click it, so
games that require the touchpad can't be played with them. The keyboard
already emulates it (Backspace/Tab); gamepads had nothing.

For SDL_GAMEPAD_BUTTON_BACK on a pad that reports no touchpads, report
PAD_BUTTON_TOUCH_PAD plus a finger at the centre of the touchpad. Pads with a
real touchpad are unchanged.

Fixes #1298

* controller: establish touch before fallback click

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-10-10 19:20:34 +02:00
nmzik f63d3befdc Add startup splash screen 2026-10-10 19:07:06 +02:00
nmzik 307fdcf7ad add self updater (#1352)
* add self updater

* remove updater tests after validation
2026-10-10 05:42:32 +02:00
Mac 679be25c37 shader: recognize remaining disabled debug branches 2026-10-10 04:31:55 +02:00
fxpw fa56c5eae9 shader: add scalar zero-bit count and search operations 2026-10-10 04:31:55 +02:00
foufouadi 3b9e176b70 shader: support VOP1/VOP2 DPP8 fetch-inactive 2026-10-10 04:31:55 +02:00
Bartlomiej Wolynczyk 053f5fd3cc shader: emit DPP row_share instead of an identity move 2026-10-10 04:31:55 +02:00
HamsterCL f3836bb831 tests: fix the Linux build of ResourceMaterializationTests (#1333)
tests: avoid empty optional::emplace() on IndirectDescriptor (fixes Linux build)

clang with libstdc++ 14 does not treat std::optional<DescriptorSource::IndirectDescriptor>
as default-constructible (nested struct with default member initializers), so the
argument-less emplace() fails to resolve. Pass an explicit value-initialized
IndirectDescriptor, matching the form already used elsewhere in this file.
2026-10-10 03:47:38 +02:00
popcoder565andClaude Opus 5.5 2f4d0f383e shader: add SDWA support for V_MIN_I32 (#1342)
WWE 2K26 stops building a shader CFG with "unsupported family=VOP2
opcode=0x11 raw=[0x220404f9 0x00060605] reason=VOP2 SDWA modifier is not
supported for opcode".

V_MIN_I32 takes the same integer path as V_MAX_I32, so give it the same
full-destination SDWA profile. The test runs the game's encoding, which
takes the signed minimum with the zero-extended low byte of a register.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-10 03:43:08 +02:00
TheCruZ d247d3d60c shader: complete signed 64-bit integer comparisons 2026-10-10 03:36:25 +02:00
nmzik 93452b24e3 renderer: cache dynamic graphics state 2026-10-10 03:28:02 +02:00
nmzik 5559cfcbb2 renderer: optimize tiling and reuse centralized image layouts 2026-10-10 03:27:13 +02:00
nmzik 1c301c9aab Revert "shader: keep scalar payload and terminal conditions on the GPU" 2026-10-09 21:10:15 +02:00
nmzikandHector Mödinger S. 59183dbfa8 shader: add missing F64 VOPC compares
Co-authored-by: Hector Mödinger S. <hectormodinger@gmail.com>
2026-10-09 21:01:51 +02:00
nmzik c1926f915d update screenshot 2026-10-09 19:07:22 +02:00
nmzik 7da3c4ed21 launcher: add resizable game grid/improve controller setup 2026-10-09 19:03:35 +02:00
nmzik 3e9f88bf79 shader: isolate SWAPPC work from ordinary shader paths 2026-10-09 17:27:07 +02:00
nmzik 8c81386568 shader: share finite descriptor planning with GPU-selected inline samplers 2026-10-09 15:06:39 +00:00
nmzik bb5edbd490 shader: expose graphics compute queue fields to GETREG 2026-10-09 15:06:39 +00:00
nmzikandBehzad 1818aae562 shader: link scalar leaf calls only for SWAPPC programs
Co-authored-by: Behzad <behzadkhan1736@gmail.com>
2026-10-09 15:06:39 +00:00
nmzik 165cd884ca agc: initialize function table in tracked driver memory 2026-10-09 15:06:39 +00:00
nmzik 973809cde0 shader: retain bounded scalar material image selectors 2026-10-09 15:06:39 +00:00
nmzik 71c7261740 shader: share indirect texture dispatch with image loads 2026-10-09 15:06:39 +00:00
nmzik 5b68a26cce shader: keep scalar payload and terminal conditions on the GPU 2026-10-09 15:06:39 +00:00
nmzik 3b1a94707a shader: capture only memory-dependent resource predicates 2026-10-09 15:06:39 +00:00
nmzik 488c0acae9 shader: resolve writable descriptors selected by immutable branches 2026-10-09 15:06:39 +00:00
nmzik 4c8fc53a85 agc: support one-word NOP reservations and share packet sizing 2026-10-09 14:02:06 +02:00
nmzik 3ae5a9b70a agc: implement branch target patching and packet size queries 2026-10-09 13:01:19 +02:00
nmzik a31e2cd361 shader: BUFFER_LOAD_SHORT_D16_HI 2026-10-09 12:57:04 +02:00
nmzik 1bd282ec0a shader: BUFFER_LOAD_SHORT_D16 2026-10-09 12:57:04 +02:00
nmzik 1fc520cd62 shader: FLAT_STORE_DWORD 2026-10-09 12:57:04 +02:00
nmzik 7b9997baea shader: fold exact float class masks into native NaN predicates 2026-10-09 02:54:21 +02:00
nmzik 393257eaae shader: classify float NaNs with one magnitude comparison 2026-10-09 02:54:21 +02:00
nmzik 13e3ca314c shader: unpack buffer offsets from one load per packed word 2026-10-09 02:54:21 +02:00
nmzik b4853a0e08 shader: preserve export vectors instead of rebuilding scalar components 2026-10-09 02:54:21 +02:00
nmzik ee4dd90649 shader: emit native integer operations and remove signedness casts 2026-10-09 02:54:21 +02:00
nmzik 3df456675d shader: own control flow on IR blocks with direct targets 2026-10-09 01:59:41 +02:00
nmzik 8abf99c56e shader: emit bounded SPIR-V operands without temporary vectors 2026-10-09 01:58:39 +02:00
nmzik 7c6121cba5 shader: build CFG labels and targets from ordered storage 2026-10-09 01:55:50 +02:00
nmzik 46997dfc7a shader: retain resource type IDs in SPIR-V emission state 2026-10-09 01:55:50 +02:00
nmzik 97fda7b10f renderer: bind prepared buffer descriptors without copying 2026-10-09 01:49:18 +02:00
nmzik dbe5353f3a shader: eliminate retained SRT lookup aliases 2026-10-09 01:44:34 +02:00
nmzik e1ac893a0c shader: collect usage once and emit descriptor slots directly 2026-10-09 02:17:38 +04:00
nmzik b35cdc39b7 shader: append deferred Phi operands directly to SPIR-V storage 2026-10-09 01:40:52 +04:00
nmzik 52dbc724f8 shader: store SPIR-V definitions directly in IR nodes 2026-10-09 01:40:41 +04:00
Mac 2f4b8a335d shader: allow SDWA byte destinations for bitwise VOP2 ops 2026-10-08 22:12:05 +04:00
nmzik b38b7454a6 Preserve vertex export locations for mixed pixel interpolation 2026-10-08 22:05:53 +04:00
nmzik b65dc12208 Track pad port ownership and reject duplicate opens 2026-10-08 21:58:53 +04:00
Devran Cosmo Uenal 58978d154a font: rasterize glyphs larger than 128 pixels (#1214)
Glyph images were rendered into a fixed 128x128 buffer and their width and
height were clamped to 128. Since glyphs are rasterized at the requested
scale (f96a14a0), large text such as title cards lost every row and column
past 128 pixels, so letters were drawn cut into bands.

The glyph buffers now grow to the glyph's bitmap box and width_byte is the
glyph's width. The VGA fallback font keeps its 128-pixel limit; scalable
glyphs are limited to 4096 pixels only to reject absurd scales.
2026-10-08 20:03:13 +04:00
1f3adf5109 cmake: accept glslang's current tool name and document Python 3 (#1161)
* cmake: accept glslang's current tool name and document Python 3

Official glslang release archives for Windows ship the compiler only as
glslang.exe, so configuring with one of them on PATH could not find
glslangValidator. Fall back to glslang; glslangValidator is still
preferred where both exist.

The bundled SPIRV-Tools needs Python 3 at configure time, which the
Windows build requirements did not mention.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: list Python 3 in the Linux and macOS prerequisites

The bundled SPIRV-Tools calls find_host_package(Python3 REQUIRED) on every platform, so the apt and Homebrew package lists need it too, not only the Windows requirements.

* cmake: remove glslang naming comment

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
2026-10-08 19:46:09 +04:00
NeySlimandnmzik 107d5f750a libNet: finish NpCommerce dialogs at open (#1166)
* libNet: finish NpCommerce dialogs at open

* libNet: complete NpCommerce dialog cancellation

---------

Co-authored-by: nmzik <Nmzik@mail.ru>
2026-10-08 19:31:47 +04:00
f40755173f ampr: keep resolved host paths in native form (#1175)
* ampr: keep resolved host paths in native form

ResolveOnePath stored each resolved file as a UTF-8 std::string
(Common::PathToString) and the read, stat and size paths turned that
string back into a std::filesystem::path. On POSIX this round trip is a
no-op, but on Windows path(std::string) decodes the bytes with the ANSI
code page, not UTF-8 (/utf-8 only sets the source and execution
charsets). Any non-ASCII character in the game folder therefore produced
a different path: the resolve step found the file through the native
path, then every APR read of it failed with ENOENT.

Tokyo Xtreme Racer (#1156) installed under "D:/Инсталляторы/[ps5]/..."
hits this on its first pak read:

  APR read missing host file: .../tokyoxtremeracer/content/paks/global.utoc
  Apr read failure 80020002 at CB offset 0   (LowLevelFatalError, abort)

even though KernelStat and the directory listing see the same file.

Store std::filesystem::path in ResolvedPathInfo and the file-id table
and pass it straight to Common::File. Convert to UTF-8 only for the log
lines.

* tests: cover APR stat and size under a non-ASCII host folder

Resolve a file inside a Cyrillic directory through APR and check that
the stat and size exports find it by ID, matching the Tokyo Xtreme
Racer install layout from #1156. On POSIX the old string round trip
also passed, so this guards the Windows path.

* tests: read the non-ASCII APR fixture through an APR command

Review on #1175 noted that the Unicode fixture only checked the stat
and size exports. A submitted APR read opens the stored host path in
ExecuteCommand/ReadHostFileToGuest, which the fixture did not reach.

Add AprReadMatches, which submits one APR read command for a file ID
and compares the bytes, and use it on the file under the Cyrillic
folder. As with the stat and size checks, POSIX hosts run this path but
only Windows would show an ANSI code page round trip.

* Remove APR path tests from this change

* Remove unrelated formatting changes

* Remove native path explanatory comments

---------

Co-authored-by: overlordxrz-source <overlordxrz-source@users.noreply.github.com>
Co-authored-by: nmzik <Nmzik@mail.ru>
2026-10-08 19:27:10 +04:00
Romain Hedouin 4e14eed770 kernel: start the semaphore wait timer after the early returns (#1173)
Wait() read the clock and looked up the thread id before checking for an available token. Both are only needed once the thread has to block.
2026-10-08 19:21:36 +04:00
139 changed files with 10793 additions and 5607 deletions

No files matched your search

+5 -1
View File
@@ -257,6 +257,7 @@ jobs:
"$install_dir/libMoltenVK.dylib" \
"$app_dir/Contents/MacOS/KytyPS5" \
"$app_dir/Contents/MacOS/kyty_emulator" \
"$app_dir/Contents/MacOS/kyty_updater" \
"$app_dir/Contents/Frameworks/libMoltenVK.dylib"; do
file "$binary"
lipo "$binary" -verify_arch x86_64
@@ -266,6 +267,7 @@ jobs:
"$install_dir/libMoltenVK.dylib" \
"$app_dir/Contents/MacOS/KytyPS5" \
"$app_dir/Contents/MacOS/kyty_emulator" \
"$app_dir/Contents/MacOS/kyty_updater" \
"$app_dir/Contents/Frameworks/libMoltenVK.dylib"; do
codesign --verify --strict "$binary"
done
@@ -407,6 +409,7 @@ jobs:
run: |
file _Build/linux/install/launcher
file _Build/linux/install/kyty_emulator
test -x _Build/linux/install/kyty_updater
file _Build/linux/install/kyty_emulator | grep -q "ELF 64-bit LSB .*x86-64"
ldd _Build/linux/install/kyty_emulator > /dev/null
readelf -d _Build/linux/install/launcher | grep -q 'RPATH.*\$ORIGIN/lib'
@@ -495,7 +498,8 @@ jobs:
test -d "$linux_dir"
chmod a+x \
"$linux_dir/launcher" \
"$linux_dir/kyty_emulator"
"$linux_dir/kyty_emulator" \
"$linux_dir/kyty_updater"
tar -C "$linux_dir" -czf \
"$GITHUB_WORKSPACE/$RELEASE_NAME-Linux-x86_64.tar.gz" .
+5 -2
View File
@@ -244,7 +244,7 @@ file(GLOB kyty_emulator_src CONFIGURE_DEPENDS
"${KYTY_SOURCE_DIR}/loader/*.h"
)
find_program(KYTY_GLSLANG_VALIDATOR glslangValidator REQUIRED)
find_program(KYTY_GLSLANG_VALIDATOR NAMES glslangValidator glslang REQUIRED)
set(gpu_tiler_shader_dir "${KYTY_SOURCE_DIR}/graphics/host_gpu/shaders")
set(gpu_tiler_generated_dir "${PROJECT_BINARY_DIR}/gpu_tiler_shaders")
set(gpu_tiler_shader_names
@@ -393,8 +393,10 @@ add_kyty_full_emulator_test(shader_cfg_tests "${KYTY_TESTS_DIR}/shaderCfgTests.c
add_executable(scalar_provenance_tests EXCLUDE_FROM_ALL
"${KYTY_TESTS_DIR}/ScalarProvenanceTests.cpp"
"${KYTY_SOURCE_DIR}/graphics/guest_gpu/gpu_format.cpp"
"${KYTY_SOURCE_DIR}/graphics/host_gpu/hostMemory.cpp"
"${KYTY_SOURCE_DIR}/graphics/shader/recompiler/ir/passes/ReadLaneElimination.cpp"
"${KYTY_SOURCE_DIR}/graphics/shader/recompiler/ir/passes/ResourceMaterialization.cpp"
"${KYTY_SOURCE_DIR}/graphics/shader/recompiler/ir/passes/SrtWalker.cpp"
)
target_link_libraries(scalar_provenance_tests fmt::fmt)
@@ -477,7 +479,6 @@ add_executable(resource_tracking_tests EXCLUDE_FROM_ALL
"${KYTY_SOURCE_DIR}/graphics/shader/recompiler/ir/passes/ResourceTracking.cpp"
"${KYTY_SOURCE_DIR}/graphics/shader/recompiler/ir/passes/ResourceMaterialization.cpp"
"${KYTY_SOURCE_DIR}/graphics/shader/recompiler/ir/passes/ShaderInfoCollection.cpp"
"${KYTY_SOURCE_DIR}/graphics/shader/recompiler/ir/passes/BindingLayout.cpp"
)
target_link_libraries(resource_tracking_tests fmt::fmt)
target_include_directories(resource_tracking_tests PRIVATE ${inc_headers})
@@ -656,6 +657,8 @@ if(BUILD_TESTING)
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --context-state-only)
add_test(NAME gpu_tiler
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --gpu-tiler-only)
add_test(NAME image_transfer_layout
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --image-layout-only)
add_test(NAME texture_cache_layered_image
COMMAND $<TARGET_FILE:shader_recompiler_compute_tests> --layered-image-only)
+77 -25
View File
@@ -7,12 +7,12 @@
[![Status](https://img.shields.io/badge/status-active%20development-orange.svg)](#current-status)
[![License](https://img.shields.io/badge/license-GPL--2.0-blue.svg)](LICENSE)
**[Weekly updates](https://github.com/KytyPS5/KytyPS5/discussions/862)** — game progress, recent fixes and ongoing development.
**[Weekly updates](https://github.com/KytyPS5/KytyPS5/discussions/862)** — game progress, recent fixes, and ongoing development.
**[Development on Discord](https://discord.gg/UNrkMqGaBg)** — KytyPS5 development.
KytyPS5 is a free and open-source PlayStation 5 emulator written in C++ for Windows and Linux,
with experimental macOS support. It is based on a heavily modified version of
KytyPS5 is a free and open-source PlayStation 5 translation layer written in C++ for Windows
and Linux, with experimental macOS support. It is based on a heavily modified version of
[Kyty](https://github.com/InoriRus/Kyty). The project is in active development, and behavior
can change significantly between builds.
@@ -23,13 +23,13 @@ can change significantly between builds.
## Current Status
KytyPS5 can boot 2D games and a selection of 3D games, including titles built with Unreal Engine
KytyPS5 can boot 2D and 3D games, including titles built with Unreal Engine
4/5, Unity, and custom engines. External low-level emulation modules are neither required nor
planned.
Development is currently focused on expanding game compatibility and improving boot reliability.
Windows and Linux are the primary platforms and receive the most testing.
Windows and Linux are the primary target platforms and receive the most testing.
macOS support is experimental. The emulator is built for x86-64 and runs on Apple Silicon under
Rosetta 2, with Vulkan provided by MoltenVK. A small number of titles have been verified in-game
@@ -52,8 +52,8 @@ or graphical glitches, so please include the version you tested when reporting a
<img src="docs/screenshots/ps5-01.png" width="300" alt="Astro Bot running in KytyPS5">
</td>
<td align="center">
<strong>Dreaming Sarah</strong><br>
<img src="docs/screenshots/ps5-03.png" width="300" alt="Dreaming Sarah running in KytyPS5">
<strong>Saros</strong><br>
<img src="docs/screenshots/ps5-03.png" width="300" alt="Saros running in KytyPS5">
</td>
</tr>
<tr>
@@ -85,14 +85,20 @@ or graphical glitches, so please include the version you tested when reporting a
Testing games and submitting detailed bug reports are useful ways to contribute. Search existing
issues first, then use the **Game Emulation Status Report** template and attach the complete log file.
Code contributions should be focused, build successfully on the platforms they touch, and include
relevant tests where practical. Windows is the primary target, so a change that alters shared code
should not regress it; changes confined to a platform's own code paths only need to build there. Because KytyPS5 is still evolving quickly, consider opening an issue before
starting a large change.
**Pull requests for general fixes, architectural changes, or refactoring that do not address a concrete bug or launch problem in a specific game will be automatically closed.**
**Large, unfocused commits or pull requests that bundle unrelated changes without a clear purpose or game-related benefit will also be automatically closed.**
Keep each PR focused. Name the affected game, describe the exact problem, and provide evidence showing what the change fixes.
Windows and Linux are the primary target platforms, so changes to shared code should not
introduce regressions on either platform. Changes confined to a platform's own code paths
only need to build on that platform. Because KytyPS5 is still evolving quickly, consider
opening an issue before starting a large change.
### Formatting
Set up the clang-format hook after cloning:
Set up the clang-format hook after cloning.
Install `pre-commit` using the method appropriate for your platform:
@@ -141,7 +147,8 @@ the Vulkan/SPIR-V validation rules.
- Visual Studio 2022 or Build Tools 2022 with the **Desktop development with C++** workload and
**C++ Clang tools for Windows** component
- Qt 6 for MSVC 2022 64-bit, including Concurrent, Network, and Widgets
- [glslang](https://github.com/KhronosGroup/glslang/releases) (`glslangValidator`) on `PATH`
- [glslang](https://github.com/KhronosGroup/glslang/releases) (`glslang` or `glslangValidator`) on `PATH`
- Python 3 on `PATH`; the bundled SPIRV-Tools runs it at configure time
The Microsoft C++ compiler (`cl.exe`) is not supported; use `clang-cl`.
@@ -170,20 +177,20 @@ The finished application and its runtime dependencies will be placed in
### Building on Linux
Install the toolchain and the libraries the bundled SDL3 needs. Without the audio, Wayland and
udev development packages SDL3 quietly configures itself without those backends, and the resulting
build has no working sound and no gamepad hotplug:
Install the toolchain and the libraries the bundled SDL3 needs. Without the audio, Wayland, and
udev development packages, SDL3 quietly configures itself without those backends, and the resulting
build has no working audio or gamepad hotplug support:
```bash
sudo apt-get install --no-install-recommends \
clang lld ninja-build cmake git glslang-tools pkg-config \
clang lld ninja-build cmake git glslang-tools python3 pkg-config \
libgl1-mesa-dev libx11-dev libxcursor-dev libxext-dev libxfixes-dev \
libxi-dev libxrandr-dev libxss-dev libxtst-dev libxkbcommon-dev \
libasound2-dev libpulse-dev libudev-dev libdbus-1-dev libwayland-dev wayland-protocols
```
Qt 6 (Concurrent, Network, Widgets) is required for the launcher — either the distribution packages
(`qt6-base-dev`) or an official Qt installation.
Qt 6 (Concurrent, Network, and Widgets) is required for the launcher. Use your distribution's
packages (`qt6-base-dev`) or an official Qt installation.
```bash
git submodule update --init --recursive
@@ -197,7 +204,7 @@ cmake --install _Build/linux --prefix _Build/linux/install
```
The install step copies the Qt libraries and plugins next to the binaries, so
`_Build/linux/install` runs without a matching system Qt. FFmpeg is linked statically
`_Build/linux/install` runs without a matching system Qt installation. FFmpeg is linked statically
from the pinned [KytyPS5 FFmpeg core](https://github.com/KytyPS5/ext-ffmpeg-core)
release, including VP9 and WebM support. System FFmpeg packages are not required.
@@ -241,6 +248,43 @@ sources, so it needs network access; a fully sandboxed `nix build` would require
inputs. A Vulkan 1.3 driver must be available at runtime (on NixOS,
`hardware.graphics.enable = true`).
### Building on Fedora and Fedora Atomic (Bazzite, Bluefin, Silverblue)
On regular Fedora, install dependencies on the host. On Fedora Atomic variants such as Bazzite,
Bluefin and Silverblue, use [Distrobox](https://distrobox.it/) and run the install and build steps
inside the container. Keep the checkout in your home directory so it is shared with the container.
Add `--nvidia` to the create command for hosts using the proprietary NVIDIA driver:
```bash
distrobox create --name kyty-build --image registry.fedoraproject.org/fedora:latest
distrobox enter kyty-build
```
Install the toolchain, Qt 6 and SDL3 dependencies:
```bash
sudo dnf install -y \
clang lld ninja-build cmake git glslang python3 pkgconf-pkg-config \
mesa-libGL-devel libX11-devel libXcursor-devel libXext-devel libXfixes-devel \
libXi-devel libXrandr-devel libXScrnSaver-devel libXtst-devel libxkbcommon-devel \
alsa-lib-devel pulseaudio-libs-devel systemd-devel dbus-devel \
wayland-devel wayland-protocols-devel qt6-qtbase-devel
```
From the repository root, follow the launcher configure, build and install steps in
[Building on Linux](#building-on-linux), omitting `-DCMAKE_PREFIX_PATH="$Qt6_DIR"`; CMake finds
Fedora's packaged Qt 6 automatically.
Run `./_Build/linux/install/launcher` on regular Fedora or inside the container. To launch a
container build from the host, leave the container with `exit`, then run from the repository root:
```bash
distrobox enter kyty-build -- ./_Build/linux/install/launcher
```
Qt libraries and plugins are bundled. You can also launch directly on a Fedora Atomic host
if its other system libraries are compatible with the container build.
### Building on macOS
macOS builds target x86-64 and run under Rosetta 2 on Apple Silicon, so the PS5's x86-64 game
@@ -251,8 +295,8 @@ Requirements:
- An Apple Silicon Mac with Rosetta 2 installed (`softwareupdate --install-rosetta`)
- Xcode (or the Command Line Tools)
- Homebrew packages: `brew install cmake ninja glslang`
- Qt 6 (Concurrent, Network, Widgets) with x86-64 support. The official Qt installation is
- Homebrew packages: `brew install cmake ninja glslang python`
- Qt 6 (Concurrent, Network, and Widgets) with x86-64 support. The official Qt installation is
universal and works; Homebrew's Qt is arm64-only and will not link
```bash
@@ -268,14 +312,14 @@ cmake --install _Build/macos --prefix _Build/macos/install
```
The build re-signs `kyty_emulator` with the JIT entitlements it needs to execute translated
guest code; no manual signing step is required. When the launcher is built, the install
guest code; no manual signing step is required. When the launcher is built, the installation
also produces `_Build/macos/install/KytyPS5.app` — double-click to launch the GUI.
A flat `kyty_emulator` is kept for CLI usage.
Vulkan comes from MoltenVK. Download `MoltenVK-macos.tar` from the
[MoltenVK releases](https://github.com/KhronosGroup/MoltenVK/releases), then copy
`MoltenVK/dynamic/dylib/macOS/libMoltenVK.dylib` next to the flat `kyty_emulator`
(and, for the bundle, into `KytyPS5.app/Contents/Frameworks/`) and ad-hoc sign it:
(and, for the bundle, into `KytyPS5.app/Contents/Frameworks/`) and sign it with an ad hoc signature:
```bash
codesign --force --sign - _Build/macos/install/libMoltenVK.dylib
@@ -337,6 +381,14 @@ folders recursively for game directories containing `eboot.bin` and ZArchive (`.
whose archive root contains `eboot.bin`. Select a detected game and run it from the game list.
ZArchive dumps are mounted read-only and streamed directly; they do not need to be extracted first.
Official release builds include a built-in updater. Leave **Check for updates on startup** enabled,
or click **Check for updates**, then choose **Install Update**. The launcher downloads the package
for your platform from GitHub, checks its SHA-256 checksum, and restarts after installation. Close
any running games first. Settings, saves, patches, and other files outside the release package are
preserved; replacement failures restore the previous files. The installation folder must be writable.
If a package cannot be installed automatically, the updater offers the GitHub release page.
Source builds and builds from forks do not replace themselves with official releases.
The emulator can also be started directly with a legally obtained game directory, ELF file, or
ZArchive dump:
@@ -392,5 +444,5 @@ licenses included with those components.
- [InoriRus/Kyty](https://github.com/InoriRus/Kyty) — KytyPS5 is based on a heavily modified version
of the original Kyty project.
- [shadps4-emu/shadPS4](https://github.com/shadps4-emu/shadPS4) — reference for understanding PS4
memory behavior, GPU resource aliasing and cache coherency,
memory behavior, GPU resource aliasing, cache coherency,
and the AVPlayer implementation.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 326 KiB

After

Width:  |  Height:  |  Size: 1.7 MiB

+1
View File
@@ -54,6 +54,7 @@ struct ConfigOptions {
int32_t gpu_index = -1;
bool fullscreen_enabled = false;
bool hide_cursor_enabled = false;
bool splash_enabled = true;
bool vr_enabled = false;
bool amd_cpu_enabled = false;
uint32_t vblank_frequency = 60;
+4
View File
@@ -11,6 +11,7 @@
#include "common/subsystems.h"
#include "common/systemInfo.h"
#include "common/threads.h"
#include "graphics/presentation/systemOverlay.h"
#include "graphics/presentation/window.h"
#include "kernel/fileSystem.h"
#include "kernel/memory.h"
@@ -138,6 +139,9 @@ static void Init(const Config::ConfigOptions& cfg, const std::filesystem::path&
Libs::LibKernel::Memory::SetFlexibleMemorySize(flexible_memory_size);
}
}
if (cfg.splash_enabled) {
Libs::Graphics::InitializeSplashScreen(param_json.parent_path());
}
// Initialization order is explicit; destruction is automatic and reversed.
subsystems.Initialize<Loader::Timer::Lifecycle>();
+4 -4
View File
@@ -10,9 +10,9 @@ namespace {
struct FormatInfo {
BufferFormat format;
uint32_t bytes_per_element;
uint32_t block_compressed_bytes_per_block;
uint32_t render_target_bytes_per_element;
uint8_t bytes_per_element;
uint8_t block_compressed_bytes_per_block;
uint8_t render_target_bytes_per_element;
bool sampled_texture;
bool uint_texture;
bool sint_texture = false;
@@ -21,7 +21,7 @@ struct FormatInfo {
constexpr FormatInfo kFormatInfo[] = {
{BufferFormat::k8UNorm, 1, 0, 1, true, false},
{BufferFormat::k8UScaled, 1, 0, 0, true, false},
{BufferFormat::k8SNorm, 1, 0, 1, false, false},
{BufferFormat::k8SNorm, 1, 0, 1, true, false},
{BufferFormat::k8UInt, 1, 0, 1, true, true},
{BufferFormat::k16UNorm, 2, 0, 2, true, false},
{BufferFormat::k16SNorm, 2, 0, 2, true, false},
+4 -3
View File
@@ -689,7 +689,7 @@ void CommandProcessor::ProcessPm4(Pm4Execution& execution) {
const auto opcode = (packet_header >> 8u) & 0xffu;
EXIT_NOT_IMPLEMENTED(remaining_dw > total_dw);
if (packet_header == 0x80000000u) {
if (packet_header == 0x80000000u || Pm4::PacketSizeDw(packet_header) == 1u) {
cursor.offset_dw++;
execution.m_made_progress = true;
continue;
@@ -1003,7 +1003,7 @@ void CommandProcessor::DispatchDirect(uint32_t thread_group_x, uint32_t thread_g
}
m_renderer.GetRenderExecutor().DispatchDirect(m_submit_id, CurrentBuffer(), thread_group_x,
thread_group_y, thread_group_z, mode);
thread_group_y, thread_group_z, mode, IsAsyncComputeQueue());
}
}
@@ -1015,7 +1015,8 @@ void CommandProcessor::DispatchIndirect(uint64_t args_addr, uint32_t mode) {
return;
}
m_sh_ctx.SetCsWaveSize(Pm4::ComputeWaveSize(mode));
m_renderer.GetRenderExecutor().DispatchIndirect(m_submit_id, CurrentBuffer(), args_addr, mode);
m_renderer.GetRenderExecutor().DispatchIndirect(m_submit_id, CurrentBuffer(), args_addr, mode,
IsAsyncComputeQueue());
}
void CommandProcessor::DrawIndexAuto(DrawAutoArgs args) {
+3 -3
View File
@@ -120,15 +120,15 @@ void DumpPm4PacketStream(Common::File* file, const uint32_t* cmd_buffer, uint32_
uint32_t len = 0;
const auto packet_type = static_cast<PacketType>(cmd_id >> 30u);
// Type-2 packets are header-only padding; every other packet type requires a body.
EXIT_NOT_IMPLEMENTED(dw < 2 && packet_type != PacketType::Type2);
EXIT_NOT_IMPLEMENTED(dw < 2 && packet_type != PacketType::Type2 &&
PacketSizeDw(cmd_id) != 1u);
switch (packet_type) {
case PacketType::Type3: {
const bool sh_gx = (cmd_id & 0x2u) == 0;
const uint8_t op = ((cmd_id >> 8u) & 0xffu);
const auto r = KYTY_PM4_R(cmd_id);
len = ((cmd_id >> 16u) & 0x3fffu) + 1;
len = PacketSizeDw(cmd_id) - 1u;
EXIT_NOT_IMPLEMENTED(len >= dw);
+4
View File
@@ -19,6 +19,10 @@ class File;
namespace Libs::Graphics::Pm4 {
constexpr uint32_t PacketSizeDw(uint32_t header) {
return (header & 0x3fffff00u) == 0x3fff1000u ? 1u : KYTY_PM4_LEN(header);
}
constexpr uint32_t IT_NOP = 0x10;
constexpr uint32_t IT_SET_BASE = 0x11;
constexpr uint32_t IT_CLEAR_STATE = 0x12;
+2 -2
View File
@@ -299,8 +299,8 @@ bool TileGetTextureBlockLayout(Prospero::BufferFormat format, Prospero::TileMode
if (!TileGetTextureElementLayout(format, element)) {
return false;
}
if ((family == TileBlockFamily::Depth64KB || family == TileBlockFamily::RenderTarget64KB) &&
Prospero::RenderTargetBytesPerElement(format) != element.bytes) {
if (family == TileBlockFamily::Depth64KB &&
(element.texel_width != 1 || element.texel_height != 1)) {
return false;
}
+138 -265
View File
@@ -110,37 +110,6 @@ constexpr uint64_t NumFramesBeforeRemoval = 32;
return true;
}
[[nodiscard]] std::vector<vk::BufferImageCopy> BuildDepthCopies(const ImageInfo& info,
uint64_t slice_stride,
vk::ImageAspectFlags aspect) {
std::vector<vk::BufferImageCopy> copies(info.resources.layers);
for (uint32_t layer = 0; layer < info.resources.layers; ++layer) {
auto& copy = copies[layer];
copy.bufferOffset = slice_stride * layer;
copy.bufferRowLength = info.pitch;
copy.bufferImageHeight = info.extent.height;
copy.imageSubresource = {aspect, 0, layer, 1};
copy.imageExtent = {info.extent.width, info.extent.height, 1};
}
return copies;
}
[[nodiscard]] std::vector<GpuTileInfo> BuildDepthTiles(const ImageInfo& info) {
TileBlockLayout block {};
EXIT_NOT_IMPLEMENTED(
!TileGetBlockLayout(TileBlockFamily::Depth64KB, info.bytes_per_block, block));
const auto full_slice_size = info.data.size / info.resources.layers;
std::vector<GpuTileInfo> tiles;
tiles.reserve(info.resources.layers);
for (uint32_t layer = 0; layer < info.resources.layers; ++layer) {
const auto offset = full_slice_size * layer;
tiles.push_back({block.family, block.bytes_per_element, offset, full_slice_size, offset,
full_slice_size, 0, info.extent.width, info.extent.height, 1, info.pitch});
tiles.back().surface_z = layer;
}
return tiles;
}
} // namespace
TextureCache::TextureCache(GraphicContext& graphics, CommandScheduler& scheduler,
@@ -570,20 +539,20 @@ bool TextureCache::CopyD16(Image& destination, Image& source) {
EXIT_IF(layers == 0 || depth_slice > UINT64_MAX / layers || color_slice > UINT64_MAX / layers);
const auto depth_size = depth_slice * layers;
const auto color_size = color_slice * layers;
std::vector<vk::BufferImageCopy> depth_copies(layers);
std::vector<vk::BufferImageCopy> color_copies(layers);
for (uint32_t layer = 0; layer < layers; layer++) {
depth_copies[layer].bufferOffset = depth_slice * layer;
depth_copies[layer].bufferRowLength = depth.info.pitch;
depth_copies[layer].bufferImageHeight = depth.info.extent.height;
depth_copies[layer].imageSubresource = {vk::ImageAspectFlagBits::eDepth, 0, layer, 1};
depth_copies[layer].imageExtent = depth.info.extent;
color_copies[layer].bufferOffset = color_slice * layer;
color_copies[layer].bufferRowLength = color.info.pitch;
color_copies[layer].bufferImageHeight = color.info.extent.height;
color_copies[layer].imageSubresource = {vk::ImageAspectFlagBits::eColor, 0, layer, 1};
color_copies[layer].imageExtent = color.info.extent;
}
const std::array depth_copies {
vk::BufferImageCopy {0,
depth.info.pitch,
depth.info.extent.height,
{vk::ImageAspectFlagBits::eDepth, 0, 0, layers},
{},
depth.info.extent}};
const std::array color_copies {
vk::BufferImageCopy {0,
color.info.pitch,
color.info.extent.height,
{vk::ImageAspectFlagBits::eColor, 0, 0, layers},
{},
color.info.extent}};
auto depth_buffer = m_tiler.GetScratchBuffer(depth_size);
auto color_buffer = m_tiler.GetScratchBuffer(color_size, depth_buffer.buffer);
@@ -740,7 +709,11 @@ ImageId TextureCache::ResolveDepthOverlap(const ImageInfo& requested, BindingTyp
if (retain_cached_layout) {
info.data = cached.info.data;
info.resources = cached.info.resources;
info.mip_layout = cached.info.mip_layout;
info.mip_layout = cached.info.mip_layout;
info.tiling = cached.info.tiling;
info.tiled_slice_stride = cached.info.tiled_slice_stride;
info.linear_slice_stride = cached.info.linear_slice_stride;
info.first_tail_level = cached.info.first_tail_level;
}
info.htile_clear_mask = 0;
const auto replacement_id = InsertImage(info);
@@ -801,7 +774,14 @@ TextureCache::OverlapResult TextureCache::ResolveOverlap(const ImageInfo& reques
// a level into or out of the mip tail. These are separate guest layouts.
if (requested.tile_mode != cached.info.tile_mode ||
(requested.resources == cached.info.resources &&
requested.mip_layout != cached.info.mip_layout)) {
!std::equal(requested.mip_layout.begin(), requested.mip_layout.end(),
cached.info.mip_layout.begin(),
[](const ImageMipInfo& left, const ImageMipInfo& right) {
return left.offset == right.offset && left.size == right.size &&
left.pitch == right.pitch && left.height == right.height &&
left.tail_x == right.tail_x && left.tail_y == right.tail_y &&
left.surface_z == right.surface_z;
}))) {
if (safe_to_delete) {
FreeImage(cached_id);
}
@@ -913,168 +893,55 @@ ImageId TextureCache::ExpandImage(const ImageInfo& info, ImageId source_id) {
return expanded_id;
}
struct TextureCache::TextureTransfer {
TextureUploadLayout layout;
std::vector<vk::BufferImageCopy> regions;
std::vector<GpuTileInfo> tiles;
ColorTransform color_transform = ColorTransform::None;
bool valid = false;
[[nodiscard]] uint64_t LinearSize() const {
uint64_t size = 0;
for (const auto& tile: tiles) {
size = std::max(size, tile.linear_offset + tile.linear_size);
}
return size;
bool TextureCache::CanDownload(const Image& image) const {
const auto& info = image.info;
if (info.samples != 1 || image.backing.samples != 1 || info.resources.layers == 0 ||
info.data.size % info.resources.layers != 0) {
return false;
}
};
struct TextureCache::ImageDownload {
TextureTransfer texture;
bool depth_target = false;
bool valid = false;
};
TextureCache::TextureTransfer
TextureCache::BuildTextureTransfer(const Image& image, BindingType binding,
TransferDirection direction) const {
const auto& info = image.info;
const bool upload = direction == TransferDirection::Upload;
const bool render_target = binding == BindingType::RenderTarget;
const bool video_out = binding == BindingType::VideoOut;
uint32_t layers = info.TransferLayers();
bool volume = info.IsVolume();
const char* owner = "TextureCache readback";
TextureTransfer transfer;
transfer.color_transform = info.GetColorTransform();
if (upload) {
if ((render_target || video_out) &&
(info.resources.layers == 0 || info.data.size % info.resources.layers != 0 ||
info.samples != 1 || image.backing.samples != 1)) {
EXIT("TextureCache: invalid color-attachment upload\n");
}
owner = "TextureCache";
if (render_target) {
owner = "RenderTarget";
} else if (binding == BindingType::Storage) {
owner = "StorageTextureCache";
} else if (video_out) {
if (info.metadata.compression != VideoOutCompression::Uncompressed) {
EXIT("TextureCache: invalid color-attachment upload\n");
}
layers = info.resources.layers;
volume = false;
owner = "VideoOut";
}
if (UploadBinding(image) == BindingType::DepthTarget) {
return IsSupportedDepthPlaneReadback(info) &&
Prospero::NumBytesPerElement(info.guest_format) == info.bytes_per_block;
}
transfer.layout = TextureCalcUploadLayout(info.guest_format, info.extent.width,
info.extent.height, info.resources.levels, layers,
info.tile_mode, info.data.size, volume, owner);
transfer.regions = TextureBuildImageCopies(transfer.layout);
if (info.IsDepth()) {
for (auto& region: transfer.regions) {
region.imageSubresource.aspectMask = vk::ImageAspectFlagBits::eDepth;
}
}
if (transfer.layout.surface.description.tile_mode != Prospero::TileMode::kLinear) {
if (!TextureBuildGpuTileInfos(info.data.size, transfer.regions, transfer.layout,
info.resources.levels, transfer.tiles)) {
return transfer;
}
}
transfer.valid = true;
return transfer;
}
TextureCache::ImageDownload TextureCache::BuildDownload(const Image& image) const {
const auto& info = image.info;
const auto binding = UploadBinding(image);
ImageDownload transfer {.depth_target = binding == BindingType::DepthTarget};
if (info.samples != 1 || image.backing.samples != 1) {
return transfer;
}
if (transfer.depth_target) {
transfer.valid = IsSupportedDepthPlaneReadback(info) && info.resources.layers != 0 &&
info.data.size % info.resources.layers == 0 &&
Prospero::NumBytesPerElement(info.guest_format) == info.bytes_per_block;
return transfer;
}
if (info.metadata.compression != VideoOutCompression::Uncompressed) {
return transfer;
}
transfer.texture = BuildTextureTransfer(image, binding, TransferDirection::Download);
transfer.valid = transfer.texture.valid;
return transfer;
return info.metadata.compression == VideoOutCompression::Uncompressed &&
(!info.IsTiled() || !Prospero::IsFmaskTextureFormat(info.guest_format)) &&
info.LinearSize() != 0;
}
void TextureCache::UploadImage(Image& image, Buffer& source, uint64_t source_offset) {
auto& destination = image.depth_id ? m_slot_images[image.depth_id] : image;
const auto binding = image.depth_id ? BindingType::DepthTarget : UploadBinding(image);
const auto upload = [&](std::vector<vk::BufferImageCopy>& copies, TileManager::Result linear) {
for (auto& copy: copies) {
copy.bufferOffset += linear.offset;
}
destination.Upload(copies, linear.buffer, linear.offset, linear.size);
};
const auto& info = image.info;
const auto binding = image.depth_id ? BindingType::DepthTarget : UploadBinding(image);
if (binding != BindingType::DepthTarget) {
const auto& info = image.info;
auto transfer = BuildTextureTransfer(image, binding, TransferDirection::Upload);
if (!transfer.valid) {
EXIT("TextureCache: invalid texture upload: binding=%u addr=0x%016" PRIx64
" size=0x%016" PRIx64 " format=%u tile=%u family=%u extent=%ux%ux%u "
"pitch=%u levels=%u layers=%u samples=%u\n",
static_cast<uint32_t>(binding), info.data.address, info.data.size,
static_cast<uint32_t>(info.guest_format), static_cast<uint32_t>(info.tile_mode),
static_cast<uint32_t>(transfer.layout.surface.texture.block.family), info.extent.width,
info.extent.height, info.extent.depth, info.pitch, info.resources.levels,
info.resources.layers, info.samples);
if ((binding == BindingType::RenderTarget || binding == BindingType::VideoOut) &&
(info.resources.layers == 0 || info.data.size % info.resources.layers != 0 ||
info.samples != 1 || image.backing.samples != 1)) {
EXIT("TextureCache: invalid color-attachment upload\n");
}
TileManager::Result linear {source.Handle(), source_offset, info.data.size};
if (!transfer.tiles.empty()) {
linear = m_tiler.Detile(source.Handle(), source_offset, info.data.size,
transfer.LinearSize(), transfer.tiles, transfer.color_transform);
} else if (transfer.color_transform != ColorTransform::None) {
linear = m_tiler.TransformColor(linear, transfer.color_transform, true);
if (binding == BindingType::VideoOut &&
info.metadata.compression != VideoOutCompression::Uncompressed) {
EXIT("TextureCache: invalid video-out upload\n");
}
upload(transfer.regions, linear);
const auto linear = m_tiler.Detile(source.Handle(), source_offset, info);
const auto copies = info.BufferCopies(linear.offset);
destination.Upload(copies, linear.buffer, linear.offset, linear.size);
return;
}
// The stencil plane has its own row pitch and shares the native image
// with the depth plane.
auto info = destination.info;
if (image.depth_id) {
info.data = image.info.data;
info.resources = {image.stencil_subresources.level_count,
image.stencil_subresources.layer_count};
info.guest_format = Prospero::BufferFormat::k8UInt;
info.bytes_per_block = 1;
if (info.IsTiled()) info.pitch = TileGetDepthPitch(info.extent.width, 1, 0);
}
if (info.samples != 1 || destination.backing.samples != 1 ||
info.resources.layers == 0 || info.data.size % info.resources.layers != 0 ||
Prospero::NumBytesPerElement(info.guest_format) != info.bytes_per_block) {
EXIT("TextureCache: invalid depth upload\n");
}
const auto layers = info.resources.layers;
const auto full_slice_size = info.data.size / layers;
auto copies = BuildDepthCopies(info, full_slice_size, image.depth_id
? vk::ImageAspectFlagBits::eStencil
: vk::ImageAspectFlagBits::eDepth);
const auto layers = info.resources.layers;
const auto full_slice_size = info.linear_slice_stride;
auto linear = m_tiler.Detile(source.Handle(), source_offset, info);
auto copies = info.BufferCopies(linear.offset);
if (image.depth_id) {
for (auto& copy: copies) {
copy.imageSubresource.baseArrayLayer += image.stencil_subresources.base_layer;
}
}
TileManager::Result linear {source.Handle(), source_offset, source.Size() - source_offset};
if (info.IsTiled()) {
const auto tiles = BuildDepthTiles(info);
linear =
m_tiler.Detile(source.Handle(), source_offset, info.data.size, info.data.size, tiles);
}
const auto transfer_bytes = image.depth_id ? 1u : DepthAspectTransferBytes(info.pixel_format);
if (transfer_bytes != info.bytes_per_block) {
const uint64_t texels_per_slice = static_cast<uint64_t>(info.pitch) * info.extent.height;
@@ -1095,11 +962,14 @@ void TextureCache::UploadImage(Image& image, Buffer& source, uint64_t source_off
.source_slice_stride = full_slice_size,
.target_slice_stride = transfer_slice});
linear = promoted;
for (uint32_t layer = 0; layer < layers; layer++) {
copies[layer].bufferOffset = transfer_slice * layer;
}
copies.resize(1);
copies[0].bufferOffset = linear.offset;
copies[0].bufferRowLength = info.pitch;
copies[0].bufferImageHeight = info.extent.height;
copies[0].imageSubresource = {vk::ImageAspectFlagBits::eDepth, 0, 0, layers};
copies[0].imageExtent = info.extent;
}
upload(copies, linear);
destination.Upload(copies, linear.buffer, linear.offset, linear.size);
}
void TextureCache::InitializeImage(ImageId id) {
@@ -1257,8 +1127,25 @@ ImageId TextureCache::AssociateStencil(ImageId depth_id, GuestRange stencil) {
}
auto& record = m_slot_images[association];
TouchImage(record);
record.depth_id = depth_id;
record.stencil_subresources = depth.stencil_subresources;
if (record.depth_id != depth_id || record.stencil_subresources != depth.stencil_subresources) {
ImageInfo plane {};
plane.data = stencil;
plane.pixel_format = vk::Format::eS8Uint;
plane.guest_format = Prospero::BufferFormat::k8UInt;
plane.type = depth.info.type;
plane.extent = depth.info.extent;
plane.resources = {depth.stencil_subresources.level_count,
depth.stencil_subresources.layer_count};
plane.tile_mode = depth.info.tile_mode;
plane.pitch =
plane.IsTiled() ? TileGetDepthPitch(plane.extent.width, 1, 0) : depth.info.pitch;
plane.bytes_per_block = 1;
plane.UpdateSize();
record.info = plane;
record.depth_id = depth_id;
record.stencil_subresources = depth.stencil_subresources;
}
return association;
}
@@ -1282,6 +1169,19 @@ ImageId TextureCache::FindImage(ImageDesc& desc, bool exact_format) {
for (const auto id: candidates) {
const auto& image = m_slot_images[id];
// Stencil descriptors name the associated plane; its depth owner supplies
// the native image and the canonical guest tiling.
if ((desc.type == BindingType::Texture || desc.type == BindingType::Storage) &&
image.depth_id && image.info.data == desc.info.data &&
m_slot_images[image.depth_id].info.samples == desc.info.samples &&
image.info.extent == desc.info.extent && image.info.type == desc.info.type &&
image.info.resources.levels >= desc.info.resources.levels &&
image.info.resources.layers >= desc.info.resources.layers &&
desc.info.bytes_per_block == 1 &&
desc.info.guest_format == Prospero::BufferFormat::k8UInt) {
result = id;
break;
}
if (SameBacking(image.info, desc.info, exact_format)) {
result = id;
}
@@ -1303,7 +1203,7 @@ ImageId TextureCache::FindImage(ImageDesc& desc, bool exact_format) {
}
}
if (result) {
if (result && !m_slot_images[result].depth_id) {
auto& resolved = m_slot_images[result];
if (exact_format && resolved.info.pixel_format != desc.info.pixel_format) {
result = {};
@@ -1685,7 +1585,7 @@ void TextureCache::InvalidateMemory(uint64_t address, uint64_t size) {
void TextureCache::DownloadDepth(Image& image, Buffer& destination, uint64_t destination_offset) {
const auto& info = image.info;
const auto layers = info.resources.layers;
const auto full_slice_size = info.data.size / layers;
const auto full_slice_size = info.linear_slice_stride;
const auto transfer_bytes = DepthAspectTransferBytes(info.pixel_format);
const uint64_t texels_per_slice = static_cast<uint64_t>(info.pitch) * info.extent.height;
EXIT_NOT_IMPLEMENTED(transfer_bytes == 0 || texels_per_slice > UINT32_MAX ||
@@ -1696,31 +1596,31 @@ void TextureCache::DownloadDepth(Image& image, Buffer& destination, uint64_t des
EXIT_NOT_IMPLEMENTED(transfer_slice > UINT64_MAX / layers);
const uint64_t transfer_size = transfer_slice * layers;
EXIT_NOT_IMPLEMENTED(guest_slice > full_slice_size);
auto copies = BuildDepthCopies(info, full_slice_size, vk::ImageAspectFlagBits::eDepth);
if (transfer_bytes == info.bytes_per_block) {
if (!info.IsTiled()) {
for (auto& copy: copies) {
copy.bufferOffset += destination_offset;
}
const auto copies = info.BufferCopies(destination_offset);
image.Download(copies, destination.Handle(), destination_offset, info.data.size);
return;
} else {
const auto copies = info.BufferCopies();
m_tiler.TileImage(image, copies, destination.Handle(), destination_offset,
info.data.size);
}
const auto tiles = BuildDepthTiles(info);
m_tiler.TileImage(image, copies, destination.Handle(), destination_offset, info.data.size,
info.data.size, tiles);
return;
}
EXIT_NOT_IMPLEMENTED(info.bytes_per_block != sizeof(uint16_t) ||
transfer_bytes != sizeof(uint32_t));
for (uint32_t layer = 0; layer < layers; layer++) {
copies[layer].bufferOffset = transfer_slice * layer;
}
const std::array copies {vk::BufferImageCopy {0,
info.pitch,
info.extent.height,
{vk::ImageAspectFlagBits::eDepth, 0, 0, layers},
{},
info.extent}};
auto host_linear = m_tiler.GetScratchBuffer(transfer_size);
image.Download(copies, host_linear.buffer, 0, host_linear.size);
const bool tiled = info.IsTiled();
auto guest_linear = tiled ? m_tiler.GetScratchBuffer(info.data.size, host_linear.buffer)
: TileManager::Result {destination.Handle(), destination_offset,
destination.Size() - destination_offset};
auto guest_linear = tiled ? m_tiler.GetScratchBuffer(info.LinearSize(), host_linear.buffer)
: TileManager::Result {destination.Handle(), destination_offset,
destination.Size() - destination_offset};
m_tiler.ConvertD16(host_linear, guest_linear, TileManager::D16Direction::Demote,
DepthAspectTransferFormat(info.pixel_format) == vk::Format::eD32Sfloat,
{.width = info.extent.width,
@@ -1730,47 +1630,39 @@ void TextureCache::DownloadDepth(Image& image, Buffer& destination, uint64_t des
.target_row_stride = static_cast<uint64_t>(info.pitch) * sizeof(uint16_t),
.source_slice_stride = transfer_slice,
.target_slice_stride = full_slice_size});
if (!tiled) {
return;
if (tiled) {
m_tiler.Tile(guest_linear.buffer, guest_linear.offset, guest_linear.size,
destination.Handle(), destination_offset, info.data.size, info);
}
const auto tiles = BuildDepthTiles(info);
m_tiler.Tile(guest_linear.buffer, guest_linear.offset, info.data.size, destination.Handle(),
destination_offset, info.data.size, tiles);
}
void TextureCache::DownloadImage(Image& image, Buffer& destination, uint64_t destination_offset,
uint64_t destination_size, ImageDownload transfer) {
if (!transfer.valid) {
EXIT("TextureCache: invalid image download transfer\n");
uint64_t destination_size, uint32_t levels) {
if (!CanDownload(image)) {
EXIT("TextureCache: invalid image download\n");
}
if (transfer.depth_target) {
if (destination_size != image.info.data.size) {
EXIT("TextureCache: partial depth image download is unsupported\n");
}
if (UploadBinding(image) == BindingType::DepthTarget) {
EXIT_IF(destination_size != image.info.data.size || levels != image.info.resources.levels);
DownloadDepth(image, destination, destination_offset);
return;
}
auto& texture = transfer.texture;
const auto transform = texture.color_transform;
if (texture.tiles.empty()) {
if (transform != ColorTransform::None) {
auto linear = m_tiler.GetScratchBuffer(destination_size);
image.Download(texture.regions, linear.buffer, 0, linear.size);
m_tiler.TransformColor(linear,
{destination.Handle(), destination_offset, destination_size},
transform, false);
return;
}
for (auto& copy: texture.regions) {
copy.bufferOffset += destination_offset;
}
image.Download(texture.regions, destination.Handle(), destination_offset, destination_size);
return;
const auto& info = image.info;
const auto transform = info.GetColorTransform();
if (info.IsTiled()) {
const auto copies = info.BufferCopies(0, levels);
m_tiler.TileImage(image, copies, destination.Handle(), destination_offset, destination_size,
levels);
} else if (transform != ColorTransform::None) {
auto linear = m_tiler.GetScratchBuffer(destination_size);
const auto copies = info.BufferCopies(0, levels);
image.Download(copies, linear.buffer, 0, linear.size);
m_tiler.TransformColor(linear, {destination.Handle(), destination_offset, destination_size},
transform, false);
} else {
const auto copies = info.BufferCopies(destination_offset, levels);
image.Download(copies, destination.Handle(), destination_offset, destination_size);
}
m_tiler.TileImage(image, texture.regions, destination.Handle(), destination_offset,
destination_size, texture.LinearSize(), texture.tiles, transform);
}
bool BufferCache::SynchronizeBufferFromImage(Buffer& buffer, uint64_t vaddr, uint64_t size) {
@@ -1810,30 +1702,12 @@ bool BufferCache::SynchronizeBufferFromImage(Buffer& buffer, uint64_t vaddr, uin
if (copy_size == 0) {
return false;
}
auto transfer = m_texture_cache.BuildDownload(image);
if (!transfer.valid) {
if (!m_texture_cache.CanDownload(image) ||
(TextureCache::UploadBinding(image) == TextureCache::BindingType::DepthTarget &&
copy_size != image.info.data.size)) {
return false;
}
if (transfer.depth_target && copy_size != image.info.data.size) {
return false;
}
if (!transfer.depth_target && levels < image.info.resources.levels) {
auto& texture = transfer.texture;
std::erase_if(texture.regions, [levels](const vk::BufferImageCopy& region) {
return region.imageSubresource.mipLevel >= levels;
});
if (texture.regions.empty()) {
return false;
}
if (!texture.tiles.empty()) {
texture.tiles.clear();
if (!TextureBuildGpuTileInfos(copy_size, texture.regions, texture.layout, levels,
texture.tiles)) {
return false;
}
}
}
m_texture_cache.DownloadImage(image, buffer, buf_offset, copy_size, std::move(transfer));
m_texture_cache.DownloadImage(image, buffer, buf_offset, copy_size, levels);
return true;
}
@@ -1842,8 +1716,7 @@ bool TextureCache::DownloadImageMemory(ImageId id) {
if (image.depth_id) {
return false;
}
auto transfer = BuildDownload(image);
if (!transfer.valid || !image.SafeToDownload()) {
if (!image.SafeToDownload() || !CanDownload(image)) {
return false;
}
const auto range = image.info.data;
@@ -1859,7 +1732,7 @@ bool TextureCache::DownloadImageMemory(ImageId id) {
}
download.Flush(offset, range.size);
DownloadImage(image, download, offset, range.size, std::move(transfer));
DownloadImage(image, download, offset, range.size, image.info.resources.levels);
vk::BufferMemoryBarrier barrier {};
barrier.srcAccessMask = vk::AccessFlagBits::eMemoryWrite | vk::AccessFlagBits::eTransferWrite |
vk::AccessFlagBits::eShaderWrite;
+2 -8
View File
@@ -73,10 +73,6 @@ public:
void RunGarbageCollector();
private:
enum class TransferDirection { Upload, Download };
struct TextureTransfer;
struct ImageDownload;
struct MetaDataInfo {
enum class Type : uint8_t { CMask, FMask, HTile };
@@ -141,12 +137,10 @@ private:
void MaterializeColorClear(ImageId id, const ImageDesc& desc,
uint32_t metadata_base_layer);
void InitializeImage(ImageId id);
[[nodiscard]] TextureTransfer
BuildTextureTransfer(const Image& image, BindingType binding, TransferDirection direction) const;
[[nodiscard]] ImageDownload BuildDownload(const Image& image) const;
[[nodiscard]] bool CanDownload(const Image& image) const;
void UploadImage(Image& image, Buffer& source, uint64_t source_offset);
void DownloadImage(Image& image, Buffer& destination, uint64_t destination_offset,
uint64_t destination_size, ImageDownload transfer);
uint64_t destination_size, uint32_t levels);
void DownloadDepth(Image& image, Buffer& destination, uint64_t destination_offset);
void CommitGpuWrite(Image& image);
// Caller holds m_lock. Volume layer ranges select depth slices.
@@ -139,9 +139,7 @@ void RenderExecutor::ResolveRenderColorTarget(CommandBuffer& buffer, RenderColor
// Nonlinear clear values are still stored as normalized components.
// Metadata clears are materialized during image discovery; render-pass loads preserve contents.
uint32_t width = 0;
uint32_t height = 0;
uint32_t pitch = 0;
uint64_t size = 0;
uint32_t height = 0;
bool tile = false;
const bool standard4 = rt.attrib3.tile_mode == Prospero::TileMode::kStandard4KB;
const bool standard64 = rt.attrib3.tile_mode == Prospero::TileMode::kStandard64KB;
@@ -164,6 +162,9 @@ void RenderExecutor::ResolveRenderColorTarget(CommandBuffer& buffer, RenderColor
if (texture_tile && samples != 1) {
EXIT("texture-tiled color render targets do not support multisampling\n");
}
if (volume && !tile) {
EXIT("linear 3D render targets are unsupported\n");
}
width = rt.attrib2.width + 1;
height = rt.attrib2.height + 1;
@@ -198,80 +199,25 @@ void RenderExecutor::ResolveRenderColorTarget(CommandBuffer& buffer, RenderColor
" layer=%u/%u\n",
rt.attrib3.dimension, rt.attrib3.depth, view.base_layer, view.image_layers);
}
if (samples == 1) {
pitch = TileGetTexturePitch(transfer_format, width, rt.attrib3.tile_mode);
} else {
pitch = TileGetRenderTargetPitch(width, bytes_per_element, rt.attrib.num_fragments);
}
if (pitch == 0) {
EXIT("unsupported render-target pitch: width=%u bytes=%u\n", width, bytes_per_element);
}
TileSizeOffset mip_sizes[16] {};
TilePaddedSize mip_padded[16] {};
TileSurfaceLayout volume_layout {};
uint64_t backing_size = 0;
if (volume) {
const TileSurfaceDescription description {transfer_format,
rt.attrib3.tile_mode,
TileSurfaceDimension::Dim3D,
width,
height,
depth,
levels,
1};
if (!tile || !TileGetTiledTextureLayout(description, volume_layout)) {
EXIT("unsupported 3D render-target layout: %ux%ux%u levels=%u tile=%u\n", width, height,
depth, levels, static_cast<uint32_t>(rt.attrib3.tile_mode));
}
size = volume_layout.block_slice_size;
backing_size = volume_layout.total_size;
} else {
TileSizeAlign layout {};
bool valid_layout = false;
if (samples == 1) {
TileGetTextureSize(transfer_format, width, height, levels, rt.attrib3.tile_mode,
&layout, mip_sizes, mip_padded);
valid_layout = layout.size != 0 && layout.align != 0 &&
(rt.attrib3.tile_mode != Prospero::TileMode::kRenderTarget ||
levels <= std::bit_width(std::max(width, height)));
} else {
valid_layout = TileGetRenderTargetSize(width, height, pitch, bytes_per_element,
layout, rt.attrib.num_fragments);
mip_sizes[0] = {layout.size, 0, 0, 0, 0, 0};
mip_padded[0] = {pitch, height};
}
if (!valid_layout) {
EXIT("unsupported render-target layout: %ux%u pitch=%u bytes=%u levels=%u\n", width,
height, pitch, bytes_per_element, levels);
}
size = layout.size;
}
if (size == 0 || (!volume && size > UINT64_MAX / view.image_layers)) {
EXIT("render-target memory footprint is invalid\n");
}
if (!volume) {
backing_size = size * view.image_layers;
}
if (backing_size == 0) {
EXIT("render-target backing is empty\n");
}
if (!GuestRange {rt.base.addr, backing_size}.Valid()) {
EXIT("render-target backing range is invalid\n");
}
auto& desc = r.desc;
desc.type = TextureCache::BindingType::RenderTarget;
desc.info.data = {rt.base.addr, backing_size};
desc.info.data = {rt.base.addr, 0};
desc.info.pixel_format = target_format.format;
desc.info.guest_format = target_format.guest_format;
desc.info.type = image_type;
desc.info.extent = {width, height, depth};
desc.info.resources = {levels, volume ? 1u : view.image_layers};
desc.info.pitch = pitch;
desc.info.resources = {levels, volume ? 1u : view.image_layers};
desc.info.bytes_per_block = bytes_per_element;
desc.info.samples = samples;
desc.info.tile_mode = rt.attrib3.tile_mode;
if (!volume && rt.attrib3.tile_mode == Prospero::TileMode::kRenderTarget &&
levels > std::bit_width(std::max(width, height))) {
EXIT("unsupported render-target mip count\n");
}
desc.info.UpdateSize();
if (!desc.info.data.Valid()) {
EXIT("render-target backing range is invalid\n");
}
const bool has_dcc = rt.info.dcc_compression_enable && rt.dcc_addr.addr != 0;
const bool has_cmask = !rt.info.dcc_compression_enable && rt.info.cmask_fast_clear_enable &&
rt.cmask.addr != 0 && samples == 1 &&
@@ -293,30 +239,6 @@ void RenderExecutor::ResolveRenderColorTarget(CommandBuffer& buffer, RenderColor
desc.info.metadata.clear_word = rt.clear_word0.word0;
desc.info.metadata.clear_register_valid = true;
}
for (uint32_t level = 0; level < levels; level++) {
if (volume) {
const auto& mip = volume_layout.mips[level];
desc.info.mip_layout[level] = {
mip.offset,
mip.size,
mip.padded_width,
mip.padded_height,
};
continue;
}
const auto level_offset =
mip_sizes[level].src_size != 0 ? mip_sizes[level].src_offset : mip_sizes[level].offset;
const auto level_size =
static_cast<uint64_t>(mip_sizes[level].src_size != 0 ? mip_sizes[level].src_size
: mip_sizes[level].size) *
view.image_layers;
desc.info.mip_layout[level] = {
level_offset,
level_size,
mip_padded[level].width,
mip_padded[level].height,
};
}
desc.view_info.format = target_format.format;
if (is_1d) {
desc.view_info.type =
@@ -334,6 +334,7 @@ CommandBuffer& CommandScheduler::BeginCommand() {
EXIT_IF(!m_command.IsInvalid());
m_command.m_buffer = m_command_pool.Commit();
m_command.Begin();
m_dynamic_state.Invalidate();
return m_command;
}
@@ -3,14 +3,13 @@
#include "common/common.h"
#include "common/uniqueFunction.h"
#include "graphics/host_gpu/renderer/dynamicState.h"
#include "graphics/host_gpu/renderer/masterSemaphore.h"
#include "graphics/host_gpu/renderer/render.h"
#include <condition_variable>
#include <mutex>
#include <queue>
#include <thread>
#include <vector>
@@ -50,6 +49,7 @@ public:
[[nodiscard]] uint64_t CurrentTick() const noexcept { return m_master.CurrentTick(); }
[[nodiscard]] bool IsFree(uint64_t tick);
[[nodiscard]] MasterSemaphore& GetMasterSemaphore() noexcept { return m_master; }
[[nodiscard]] DynamicState& GetDynamicState() noexcept { return m_dynamic_state; }
[[nodiscard]] RenderContext& Context() const noexcept { return m_context; }
[[nodiscard]] GraphicContext& Graphics() const noexcept { return m_graphics; }
@@ -92,6 +92,7 @@ private:
GraphicContext& m_graphics;
CommandPool m_command_pool;
CommandBuffer m_command;
DynamicState m_dynamic_state;
std::queue<PendingOperation> m_pending_operations;
std::queue<PendingOperation> m_priority_operations;
std::mutex m_operation_mutex;
@@ -239,7 +239,7 @@ static TextureCache::ImageDesc MakeDepthTargetDesc(const CommandBuffer& buffer,
desc.info.bytes_per_block = bytes;
desc.info.samples = samples;
desc.info.tile_mode = Prospero::TileMode::kDepth;
desc.info.mip_layout[0] = {0, depth_backing_size, pitch, height};
desc.info.UpdateSize();
desc.info.metadata.range =
has_htile ? GuestRange {z.htile_data_base_addr, htile_backing_size} : GuestRange {};
desc.info.metadata.kind = has_htile ? ImageMetadataKind::Htile : ImageMetadataKind::None;
@@ -0,0 +1,150 @@
#include "graphics/host_gpu/renderer/dynamicState.h"
#include "graphics/host_gpu/graphicContext.h"
namespace Libs::Graphics {
void DynamicState::Commit(const GraphicContext& graphics, vk::CommandBuffer cmdbuf) {
if (dirty_state.viewports) {
dirty_state.viewports = false;
cmdbuf.setViewportWithCount(num_viewports, viewports.data());
}
if (dirty_state.scissors) {
dirty_state.scissors = false;
cmdbuf.setScissorWithCount(num_scissors, scissors.data());
}
if (dirty_state.depth_test_enabled) {
dirty_state.depth_test_enabled = false;
cmdbuf.setDepthTestEnable(depth_test_enabled);
}
if (dirty_state.depth_write_enabled) {
dirty_state.depth_write_enabled = false;
// Note that this must be set in a command buffer even if depth test is disabled.
cmdbuf.setDepthWriteEnable(depth_write_enabled);
}
if (depth_test_enabled && dirty_state.depth_compare_op) {
dirty_state.depth_compare_op = false;
cmdbuf.setDepthCompareOp(depth_compare_op);
}
if (dirty_state.depth_bounds_test_enabled) {
dirty_state.depth_bounds_test_enabled = false;
#if !defined(__APPLE__)
cmdbuf.setDepthBoundsTestEnable(depth_bounds_test_enabled);
#endif
}
if (depth_bounds_test_enabled && dirty_state.depth_bounds) {
dirty_state.depth_bounds = false;
#if !defined(__APPLE__)
cmdbuf.setDepthBounds(depth_bounds_min, depth_bounds_max);
#endif
}
if (dirty_state.depth_bias_enabled) {
dirty_state.depth_bias_enabled = false;
cmdbuf.setDepthBiasEnable(depth_bias_enabled);
}
if (depth_bias_enabled && dirty_state.depth_bias) {
dirty_state.depth_bias = false;
cmdbuf.setDepthBias(depth_bias_constant, depth_bias_clamp, depth_bias_slope);
}
if (dirty_state.stencil_test_enabled) {
dirty_state.stencil_test_enabled = false;
cmdbuf.setStencilTestEnable(stencil_test_enabled);
}
if (stencil_test_enabled) {
if (dirty_state.stencil_front_ops && dirty_state.stencil_back_ops &&
stencil_front_ops == stencil_back_ops) {
dirty_state.stencil_front_ops = false;
dirty_state.stencil_back_ops = false;
cmdbuf.setStencilOp(vk::StencilFaceFlagBits::eFrontAndBack, stencil_front_ops.fail_op,
stencil_front_ops.pass_op, stencil_front_ops.depth_fail_op,
stencil_front_ops.compare_op);
} else {
if (dirty_state.stencil_front_ops) {
dirty_state.stencil_front_ops = false;
cmdbuf.setStencilOp(vk::StencilFaceFlagBits::eFront, stencil_front_ops.fail_op,
stencil_front_ops.pass_op, stencil_front_ops.depth_fail_op,
stencil_front_ops.compare_op);
}
if (dirty_state.stencil_back_ops) {
dirty_state.stencil_back_ops = false;
cmdbuf.setStencilOp(vk::StencilFaceFlagBits::eBack, stencil_back_ops.fail_op,
stencil_back_ops.pass_op, stencil_back_ops.depth_fail_op,
stencil_back_ops.compare_op);
}
}
if (dirty_state.stencil_front_reference && dirty_state.stencil_back_reference &&
stencil_front_reference == stencil_back_reference) {
dirty_state.stencil_front_reference = false;
dirty_state.stencil_back_reference = false;
cmdbuf.setStencilReference(vk::StencilFaceFlagBits::eFrontAndBack,
stencil_front_reference);
} else {
if (dirty_state.stencil_front_reference) {
dirty_state.stencil_front_reference = false;
cmdbuf.setStencilReference(vk::StencilFaceFlagBits::eFront,
stencil_front_reference);
}
if (dirty_state.stencil_back_reference) {
dirty_state.stencil_back_reference = false;
cmdbuf.setStencilReference(vk::StencilFaceFlagBits::eBack, stencil_back_reference);
}
}
if (dirty_state.stencil_front_write_mask && dirty_state.stencil_back_write_mask &&
stencil_front_write_mask == stencil_back_write_mask) {
dirty_state.stencil_front_write_mask = false;
dirty_state.stencil_back_write_mask = false;
cmdbuf.setStencilWriteMask(vk::StencilFaceFlagBits::eFrontAndBack,
stencil_front_write_mask);
} else {
if (dirty_state.stencil_front_write_mask) {
dirty_state.stencil_front_write_mask = false;
cmdbuf.setStencilWriteMask(vk::StencilFaceFlagBits::eFront,
stencil_front_write_mask);
}
if (dirty_state.stencil_back_write_mask) {
dirty_state.stencil_back_write_mask = false;
cmdbuf.setStencilWriteMask(vk::StencilFaceFlagBits::eBack, stencil_back_write_mask);
}
}
if (dirty_state.stencil_front_compare_mask && dirty_state.stencil_back_compare_mask &&
stencil_front_compare_mask == stencil_back_compare_mask) {
dirty_state.stencil_front_compare_mask = false;
dirty_state.stencil_back_compare_mask = false;
cmdbuf.setStencilCompareMask(vk::StencilFaceFlagBits::eFrontAndBack,
stencil_front_compare_mask);
} else {
if (dirty_state.stencil_front_compare_mask) {
dirty_state.stencil_front_compare_mask = false;
cmdbuf.setStencilCompareMask(vk::StencilFaceFlagBits::eFront,
stencil_front_compare_mask);
}
if (dirty_state.stencil_back_compare_mask) {
dirty_state.stencil_back_compare_mask = false;
cmdbuf.setStencilCompareMask(vk::StencilFaceFlagBits::eBack,
stencil_back_compare_mask);
}
}
}
if (dirty_state.blend_constants) {
dirty_state.blend_constants = false;
cmdbuf.setBlendConstants(blend_constants.data());
}
if (dirty_state.color_write_enables) {
dirty_state.color_write_enables = false;
#if !defined(__APPLE__)
if (num_color_write_enables != 0) {
cmdbuf.setColorWriteEnableEXT(num_color_write_enables, color_write_enables.data());
}
#endif
}
if (dirty_state.line_width) {
dirty_state.line_width = false;
cmdbuf.setLineWidth(line_width);
}
if (dirty_state.feedback_loop_aspects && graphics.attachment_feedback_loop_enabled) {
dirty_state.feedback_loop_aspects = false;
cmdbuf.setAttachmentFeedbackLoopEnableEXT(feedback_loop_aspects);
}
}
} // namespace Libs::Graphics
@@ -0,0 +1,258 @@
#ifndef EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_DYNAMICSTATE_H_
#define EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_DYNAMICSTATE_H_
#include "common/assert.h"
#include "graphics/host_gpu/renderer/renderTarget.h"
#include "graphics/host_gpu/vulkanCommon.h"
#include <algorithm>
#include <array>
#include <cstdint>
#include <cstring>
#include <span>
namespace Libs::Graphics {
struct GraphicContext;
struct StencilOps {
vk::StencilOp fail_op {};
vk::StencilOp pass_op {};
vk::StencilOp depth_fail_op {};
vk::CompareOp compare_op {};
bool operator==(const StencilOps& other) const {
return fail_op == other.fail_op && pass_op == other.pass_op &&
depth_fail_op == other.depth_fail_op && compare_op == other.compare_op;
}
};
struct DynamicState {
struct {
bool viewports : 1;
bool scissors : 1;
bool depth_test_enabled : 1;
bool depth_write_enabled : 1;
bool depth_compare_op : 1;
bool depth_bounds_test_enabled : 1;
bool depth_bounds : 1;
bool depth_bias_enabled : 1;
bool depth_bias : 1;
bool stencil_test_enabled : 1;
bool stencil_front_ops : 1;
bool stencil_front_reference : 1;
bool stencil_front_write_mask : 1;
bool stencil_front_compare_mask : 1;
bool stencil_back_ops : 1;
bool stencil_back_reference : 1;
bool stencil_back_write_mask : 1;
bool stencil_back_compare_mask : 1;
bool blend_constants : 1;
bool color_write_enables : 1;
bool line_width : 1;
bool feedback_loop_aspects : 1;
} dirty_state {};
std::array<vk::Viewport, 16> viewports {};
std::array<vk::Rect2D, 16> scissors {};
uint32_t num_viewports {};
uint32_t num_scissors {};
bool depth_test_enabled {};
bool depth_write_enabled {};
vk::CompareOp depth_compare_op {};
bool depth_bounds_test_enabled {};
float depth_bounds_min {};
float depth_bounds_max {};
bool depth_bias_enabled {};
float depth_bias_constant {};
float depth_bias_clamp {};
float depth_bias_slope {};
bool stencil_test_enabled {};
StencilOps stencil_front_ops {};
uint32_t stencil_front_reference {};
uint32_t stencil_front_write_mask {};
uint32_t stencil_front_compare_mask {};
StencilOps stencil_back_ops {};
uint32_t stencil_back_reference {};
uint32_t stencil_back_write_mask {};
uint32_t stencil_back_compare_mask {};
std::array<float, 4> blend_constants {};
std::array<vk::Bool32, RENDER_COLOR_ATTACHMENTS_MAX> color_write_enables {};
uint32_t num_color_write_enables {};
float line_width {};
vk::ImageAspectFlags feedback_loop_aspects {};
/// Commits the dynamic state to the provided command buffer.
void Commit(const GraphicContext& graphics, vk::CommandBuffer cmdbuf);
/// Invalidates all dynamic state to be flushed into the next command buffer.
void Invalidate() { std::memset(&dirty_state, 0xFF, sizeof(dirty_state)); }
void SetViewports(std::span<const vk::Viewport> viewports_) {
EXIT_IF(viewports_.size() > viewports.size());
if (!std::ranges::equal(std::span(viewports).first(num_viewports), viewports_)) {
std::ranges::copy(viewports_, viewports.begin());
num_viewports = static_cast<uint32_t>(viewports_.size());
dirty_state.viewports = true;
}
}
void SetScissors(std::span<const vk::Rect2D> scissors_) {
EXIT_IF(scissors_.size() > scissors.size());
if (!std::ranges::equal(std::span(scissors).first(num_scissors), scissors_)) {
std::ranges::copy(scissors_, scissors.begin());
num_scissors = static_cast<uint32_t>(scissors_.size());
dirty_state.scissors = true;
}
}
void SetDepthTestEnabled(const bool enabled) {
if (depth_test_enabled != enabled) {
depth_test_enabled = enabled;
dirty_state.depth_test_enabled = true;
}
}
void SetDepthWriteEnabled(const bool enabled) {
if (depth_write_enabled != enabled) {
depth_write_enabled = enabled;
dirty_state.depth_write_enabled = true;
}
}
void SetDepthCompareOp(const vk::CompareOp compare_op) {
if (depth_compare_op != compare_op) {
depth_compare_op = compare_op;
dirty_state.depth_compare_op = true;
}
}
void SetDepthBoundsTestEnabled(const bool enabled) {
if (depth_bounds_test_enabled != enabled) {
depth_bounds_test_enabled = enabled;
dirty_state.depth_bounds_test_enabled = true;
}
}
void SetDepthBounds(const float min, const float max) {
if (depth_bounds_min != min || depth_bounds_max != max) {
depth_bounds_min = min;
depth_bounds_max = max;
dirty_state.depth_bounds = true;
}
}
void SetDepthBiasEnabled(const bool enabled) {
if (depth_bias_enabled != enabled) {
depth_bias_enabled = enabled;
dirty_state.depth_bias_enabled = true;
}
}
void SetDepthBias(const float constant, const float clamp, const float slope) {
if (depth_bias_constant != constant || depth_bias_clamp != clamp ||
depth_bias_slope != slope) {
depth_bias_constant = constant;
depth_bias_clamp = clamp;
depth_bias_slope = slope;
dirty_state.depth_bias = true;
}
}
void SetStencilTestEnabled(const bool enabled) {
if (stencil_test_enabled != enabled) {
stencil_test_enabled = enabled;
dirty_state.stencil_test_enabled = true;
}
}
void SetStencilOps(const StencilOps& front_ops, const StencilOps& back_ops) {
if (stencil_front_ops != front_ops) {
stencil_front_ops = front_ops;
dirty_state.stencil_front_ops = true;
}
if (stencil_back_ops != back_ops) {
stencil_back_ops = back_ops;
dirty_state.stencil_back_ops = true;
}
}
void SetStencilReferences(const uint32_t front_reference, const uint32_t back_reference) {
if (stencil_front_reference != front_reference) {
stencil_front_reference = front_reference;
dirty_state.stencil_front_reference = true;
}
if (stencil_back_reference != back_reference) {
stencil_back_reference = back_reference;
dirty_state.stencil_back_reference = true;
}
}
void SetStencilWriteMasks(const uint32_t front_write_mask, const uint32_t back_write_mask) {
if (stencil_front_write_mask != front_write_mask) {
stencil_front_write_mask = front_write_mask;
dirty_state.stencil_front_write_mask = true;
}
if (stencil_back_write_mask != back_write_mask) {
stencil_back_write_mask = back_write_mask;
dirty_state.stencil_back_write_mask = true;
}
}
void SetStencilCompareMasks(const uint32_t front_compare_mask,
const uint32_t back_compare_mask) {
if (stencil_front_compare_mask != front_compare_mask) {
stencil_front_compare_mask = front_compare_mask;
dirty_state.stencil_front_compare_mask = true;
}
if (stencil_back_compare_mask != back_compare_mask) {
stencil_back_compare_mask = back_compare_mask;
dirty_state.stencil_back_compare_mask = true;
}
}
void SetBlendConstants(const std::array<float, 4> blend_constants_) {
if (blend_constants != blend_constants_) {
blend_constants = blend_constants_;
dirty_state.blend_constants = true;
}
}
void SetColorWriteEnables(std::span<const vk::Bool32> enables) {
EXIT_IF(enables.size() > color_write_enables.size());
if (!std::ranges::equal(std::span(color_write_enables).first(num_color_write_enables),
enables)) {
std::ranges::copy(enables, color_write_enables.begin());
num_color_write_enables = static_cast<uint32_t>(enables.size());
dirty_state.color_write_enables = true;
}
}
void SetLineWidth(const float width) {
if (line_width != width) {
line_width = width;
dirty_state.line_width = true;
}
}
void SetAttachmentFeedbackLoopAspects(const vk::ImageAspectFlags aspects) {
if (feedback_loop_aspects != aspects) {
feedback_loop_aspects = aspects;
dirty_state.feedback_loop_aspects = true;
}
}
};
} // namespace Libs::Graphics
#endif // EMULATOR_SRC_GRAPHICS_HOST_GPU_RENDERER_DYNAMICSTATE_H_
@@ -193,6 +193,8 @@ void BlitHelper::ReinterpretColorAsMsDepth(Image& source, Image& destination) {
command.setScissor(0, 1, &scissor);
command.draw(3, 1, 0, 0);
command.endRendering();
// The helper pipeline has static state that invalidates the guest draw's dynamic state.
m_scheduler.GetDynamicState().Invalidate();
}
} // namespace Libs::Graphics
@@ -0,0 +1,246 @@
#include "graphics/host_gpu/renderer/image/imageInfo.h"
#include "common/alignment.h"
namespace Libs::Graphics {
namespace {
uint64_t Multiply(uint64_t left, uint64_t right) {
EXIT_IF(right != 0 && left > UINT64_MAX / right);
return left * right;
}
uint64_t Add(uint64_t left, uint64_t right) {
EXIT_IF(right > UINT64_MAX - left);
return left + right;
}
bool IsDepthPlane(const ImageInfo& info) {
return (info.IsDepth() || info.pixel_format == vk::Format::eS8Uint) &&
(info.tile_mode == Prospero::TileMode::kDepth ||
info.tile_mode == Prospero::TileMode::kLinear);
}
} // namespace
vk::Extent3D ImageInfo::MipExtent(uint32_t level) const {
EXIT_IF(level >= resources.levels || level >= mip_layout.size());
return {std::max(extent.width >> level, 1u), std::max(extent.height >> level, 1u),
IsVolume() ? std::max(extent.depth >> level, 1u) : resources.layers};
}
void ImageInfo::UpdateSize() {
EXIT_IF(guest_format == Prospero::BufferFormat::kInvalid || extent.width == 0 ||
extent.height == 0 || extent.depth == 0 || resources.levels == 0 ||
resources.levels > mip_layout.size() || resources.layers == 0 || samples == 0 ||
!std::has_single_bit(samples));
TileTextureElementLayout element {};
EXIT_NOT_IMPLEMENTED(!TileGetTextureElementLayout(guest_format, element));
if (bytes_per_block == 0) {
bytes_per_block = element.bytes;
}
EXIT_IF(bytes_per_block != element.bytes);
mip_layout = {};
tiling = {};
tiling.texel_width = element.texel_width;
tiling.texel_height = element.texel_height;
tiling.block.bytes_per_element = element.bytes;
tiled_slice_stride = 0;
linear_slice_stride = 0;
first_tail_level = 16;
if (samples > 1) {
EXIT_NOT_IMPLEMENTED(IsVolume() || resources.levels != 1 ||
(tile_mode != Prospero::TileMode::kDepth &&
tile_mode != Prospero::TileMode::kRenderTarget));
const auto fragments = std::countr_zero(samples);
if (pitch == 0) {
pitch = tile_mode == Prospero::TileMode::kDepth
? TileGetDepthPitch(extent.width, bytes_per_block, fragments)
: TileGetRenderTargetPitch(extent.width, bytes_per_block, fragments);
}
EXIT_NOT_IMPLEMENTED(!TileGetTextureBlockLayout(guest_format, tile_mode, false, tiling));
if (data.size == 0) {
TileSizeAlign slice {};
EXIT_NOT_IMPLEMENTED(!TileGetRenderTargetSize(extent.width, extent.height, pitch,
bytes_per_block, slice, fragments));
data.size = Multiply(slice.size, resources.layers);
}
EXIT_IF(data.size % resources.layers != 0);
tiled_slice_stride = data.size / resources.layers;
mip_layout[0] = {0, data.size, pitch, extent.height};
return;
}
if (pitch == 0) {
pitch = TileGetTexturePitch(guest_format, extent.width, tile_mode);
}
EXIT_IF(pitch == 0);
if (IsDepthPlane(*this) && resources.levels == 1) {
EXIT_NOT_IMPLEMENTED(IsVolume());
if (IsTiled()) {
EXIT_NOT_IMPLEMENTED(
!TileGetTextureBlockLayout(guest_format, tile_mode, false, tiling));
}
if (data.size == 0) {
TileSizeAlign slice {};
TileGetTextureSize(guest_format, extent.width, extent.height, 1, tile_mode, &slice,
nullptr, nullptr);
data.size = Multiply(slice.size, resources.layers);
}
EXIT_IF(data.size % resources.layers != 0);
const auto slice = data.size / resources.layers;
EXIT_IF(Multiply(Multiply(pitch, extent.height), bytes_per_block) > slice);
linear_slice_stride = slice;
tiled_slice_stride = IsTiled() ? slice : 0;
mip_layout[0] = {0, data.size, pitch, extent.height, 0, slice, pitch};
return;
}
if (!IsTiled()) {
TileSizeAlign total {};
TileSizeOffset offsets[16] {};
TilePaddedSize padded[16] {};
TileGetTextureSize(guest_format, extent.width, extent.height, resources.levels, tile_mode,
&total, offsets, padded);
if (data.size == 0) {
data.size = Multiply(total.size, TransferLayers());
}
for (uint32_t level = 0; level < resources.levels; ++level) {
auto& mip = mip_layout[level];
mip.offset = offsets[level].offset;
mip.size = Multiply(offsets[level].size, MipExtent(level).depth);
mip.pitch = padded[level].width / element.texel_width;
mip.height = padded[level].height / element.texel_height;
mip.linear_offset = offsets[level].offset;
mip.linear_size = offsets[level].size;
mip.linear_pitch = mip.pitch;
linear_slice_stride =
std::max(linear_slice_stride, Add(mip.linear_offset, mip.linear_size));
}
if (TransferLayers() > 1 && data.size % TransferLayers() == 0) {
linear_slice_stride = std::max(linear_slice_stride, data.size / TransferLayers());
}
} else {
TileSurfaceLayout surface {};
const TileSurfaceDescription description {guest_format,
tile_mode,
IsVolume() ? TileSurfaceDimension::Dim3D
: TileSurfaceDimension::Dim2D,
extent.width,
extent.height,
IsVolume() ? extent.depth : 1u,
resources.levels,
resources.layers};
EXIT_NOT_IMPLEMENTED(!TileGetTiledTextureLayout(description, surface));
tiling = surface.texture;
first_tail_level = surface.first_tail_level;
tiled_slice_stride = surface.block_slice_size;
if (data.size == 0) {
data.size = surface.total_size;
}
if (!IsVolume() && resources.layers > 1 && data.size % resources.layers == 0) {
tiled_slice_stride = std::max(tiled_slice_stride, data.size / resources.layers);
}
uint64_t linear_offset = 0;
uint32_t mip_pitch = IsVolume() ? extent.width : pitch;
for (uint32_t level = 0; level < resources.levels; ++level) {
const auto& source = surface.mips[level];
auto& mip = mip_layout[level];
const auto active = MipExtent(level);
mip.offset = source.offset;
mip.size = Multiply(source.size, IsVolume() ? 1u : resources.layers);
mip.pitch = source.padded_width;
mip.height = source.padded_height;
mip.tail_x = source.tail_x;
mip.tail_y = source.tail_y;
mip.linear_pitch =
Common::AlignUp(mip_pitch, element.texel_width) / element.texel_width;
const auto rows =
Common::AlignUp(active.height, element.texel_height) / element.texel_height;
mip.linear_size = Multiply(Multiply(mip.linear_pitch, rows), element.bytes);
const auto alignment = std::max(element.bytes, 4u);
EXIT_IF(linear_offset > UINT64_MAX - (alignment - 1u));
mip.linear_offset = Common::AlignUp<uint64_t>(linear_offset, alignment);
linear_offset = Add(mip.linear_offset, Multiply(mip.linear_size, active.depth));
mip_pitch = std::max(mip_pitch >> 1u, 1u);
}
}
}
uint64_t ImageInfo::LinearSize(uint32_t levels) const {
levels = levels == 0 ? resources.levels : levels;
EXIT_IF(levels == 0 || levels > resources.levels || levels > mip_layout.size());
if (linear_slice_stride == 0) {
const auto& mip = mip_layout[levels - 1u];
return Add(mip.linear_offset, Multiply(mip.linear_size, MipExtent(levels - 1u).depth));
}
uint64_t size = 0;
for (uint32_t level = 0; level < levels; ++level) {
const auto& mip = mip_layout[level];
const auto slices = MipExtent(level).depth;
const auto stride = linear_slice_stride != 0 ? linear_slice_stride : mip.linear_size;
size = std::max(
size, Add(Add(mip.linear_offset, Multiply(slices - 1u, stride)), mip.linear_size));
}
return size;
}
std::vector<vk::BufferImageCopy> ImageInfo::BufferCopies(uint64_t buffer_offset,
uint32_t levels) const {
levels = levels == 0 ? resources.levels : levels;
EXIT_IF(levels > resources.levels || levels > mip_layout.size() || tiling.texel_width == 0 ||
tiling.texel_height == 0);
const auto aspect = IsDepth() ? vk::ImageAspectFlagBits::eDepth
: pixel_format == vk::Format::eS8Uint ? vk::ImageAspectFlagBits::eStencil
: vk::ImageAspectFlagBits::eColor;
std::vector<vk::BufferImageCopy> copies;
std::array<uint32_t, 16> image_heights {};
uint64_t count = 0;
for (uint32_t level = 0; level < levels; ++level) {
const auto& mip = mip_layout[level];
const auto active = MipExtent(level);
if (linear_slice_stride == 0) {
image_heights[level] = Common::AlignUp(active.height, tiling.texel_height);
} else {
const auto row_bytes = Multiply(mip.linear_pitch, tiling.block.bytes_per_element);
EXIT_IF(row_bytes == 0);
const auto rows = linear_slice_stride / row_bytes;
if (linear_slice_stride % row_bytes == 0 &&
rows >=
(uint64_t {active.height} + tiling.texel_height - 1u) / tiling.texel_height &&
rows <= UINT32_MAX / tiling.texel_height) {
image_heights[level] = static_cast<uint32_t>(rows) * tiling.texel_height;
}
}
count += image_heights[level] != 0 ? 1u : active.depth;
}
copies.reserve(static_cast<size_t>(count));
for (uint32_t level = 0; level < levels; ++level) {
const auto& mip = mip_layout[level];
const auto active = MipExtent(level);
vk::BufferImageCopy copy {};
copy.bufferOffset = Add(buffer_offset, mip.linear_offset);
copy.bufferRowLength = mip.linear_pitch * tiling.texel_width;
copy.bufferImageHeight = image_heights[level] != 0 ? image_heights[level]
: !IsTiled() ? mip.height * tiling.texel_height
: Common::AlignUp(active.height, tiling.texel_height);
copy.imageSubresource = {aspect, level, 0, 1};
copy.imageExtent = {active.width, active.height, 1};
if (image_heights[level] == 0) {
for (uint32_t slice = 0; slice < active.depth; ++slice) {
copy.bufferOffset = Add(Add(buffer_offset, mip.linear_offset),
Multiply(slice, linear_slice_stride));
copy.imageSubresource.baseArrayLayer = IsVolume() ? 0u : slice;
copy.imageOffset.z = IsVolume() ? static_cast<int32_t>(slice) : 0;
copies.push_back(copy);
}
} else {
copy.imageSubresource.layerCount = IsVolume() ? 1u : active.depth;
copy.imageExtent.depth = IsVolume() ? active.depth : 1u;
copies.push_back(copy);
}
}
return copies;
}
} // namespace Libs::Graphics
@@ -4,6 +4,7 @@
#include "common/assert.h"
#include "graphics/guest_gpu/gpu_defs.h"
#include "graphics/guest_gpu/gpu_format.h"
#include "graphics/guest_gpu/tile.h"
#include "graphics/host_gpu/regionDefinitions.h"
#include "graphics/host_gpu/vulkanCommon.h"
@@ -12,6 +13,7 @@
#include <bit>
#include <cmath>
#include <cstdint>
#include <vector>
namespace Libs::Graphics {
@@ -47,11 +49,19 @@ struct ImageSubresourceRange {
};
struct ImageMipInfo {
// Guest mip chains run backwards within each layer or volume block-slice.
// Tail levels share an offset and use tail_x/tail_y within that block.
uint64_t offset = 0;
uint64_t size = 0;
// Padded dimensions in storage elements (compressed blocks for BC formats).
uint32_t pitch = 0;
uint32_t height = 0;
uint64_t linear_offset = 0;
uint64_t linear_size = 0; // Per transfer slice.
uint32_t linear_pitch = 0; // Storage elements.
uint32_t tail_x = 0;
uint32_t tail_y = 0;
uint32_t surface_z = 0;
auto operator<=>(const ImageMipInfo&) const = default;
};
@@ -71,6 +81,17 @@ struct ImageInfo {
Prospero::TileMode tile_mode = Prospero::TileMode::kLinear;
bool bgra16 = false;
std::array<ImageMipInfo, 16> mip_layout {};
TileTextureBlockLayout tiling;
uint64_t tiled_slice_stride = 0;
// Nonzero for guest-linear and depth-plane transfers; tiled colors are mip-major.
uint64_t linear_slice_stride = 0;
uint32_t first_tail_level = 16;
void UpdateSize();
[[nodiscard]] uint64_t LinearSize(uint32_t levels = 0) const;
[[nodiscard]] vk::Extent3D MipExtent(uint32_t level) const;
[[nodiscard]] std::vector<vk::BufferImageCopy> BufferCopies(uint64_t buffer_offset = 0,
uint32_t levels = 0) const;
[[nodiscard]] ColorTransform GetColorTransform() const noexcept {
if (bgra16) return ColorTransform::SwapBgra16;
@@ -1,16 +1,13 @@
#include "graphics/host_gpu/renderer/image/textureCommon.h"
#include "common/alignment.h"
#include "common/assert.h"
#include "graphics/guest_gpu/gpu_defs.h"
#include "graphics/guest_gpu/gpu_format.h"
#include "graphics/host_gpu/renderer/image/tiler.h"
#include "graphics/host_gpu/vulkanCommon.h"
#include "graphics/shader/shader.h"
#include <algorithm>
#include <array>
#include <cinttypes>
namespace Libs::Graphics {
namespace {
@@ -60,68 +57,6 @@ HostFormatInfo ResolveHostFormat(Prospero::BufferFormat guest_format,
}
}
uint32_t GetTextureLevelDepth(uint32_t depth, uint32_t level, bool volume_texture) {
return volume_texture ? std::max(depth >> level, 1u) : depth;
}
size_t GetTextureRegionCount(uint32_t depth, uint32_t levels, bool volume_texture) {
size_t count = 0;
for (uint32_t level = 0; level < levels; level++) {
count += GetTextureLevelDepth(depth, level, volume_texture);
}
return count;
}
uint64_t CalcTextureSliceStride(const TextureUploadMipLayout* mips, uint32_t levels,
uint64_t total_size, uint32_t depth) {
uint64_t stride = 0;
for (uint32_t mip_level = 0; mip_level < levels; ++mip_level) {
stride = std::max(stride, mips[mip_level].offset + mips[mip_level].size);
}
if (depth > 1 && total_size != 0 && total_size % depth == 0) {
stride = std::max(stride, total_size / depth);
}
return stride;
}
uint64_t CalcLinearUploadLevelSize(const TileTextureElementLayout& element, uint32_t pitch,
uint32_t height) {
const uint32_t element_columns =
std::max((pitch + element.texel_width - 1u) / element.texel_width, 1u);
const uint32_t element_rows =
std::max((height + element.texel_height - 1u) / element.texel_height, 1u);
return static_cast<uint64_t>(element_columns) * element_rows * element.bytes;
}
uint64_t SetLinearUploadLevels(TextureUploadMipLayout* mips,
const TileTextureElementLayout& element, uint32_t height,
uint32_t levels, uint32_t base_pitch) {
uint64_t offset = 0;
auto mip_pitch = base_pitch;
auto mip_height = height;
for (uint32_t mip_level = 0; mip_level < levels; ++mip_level) {
const auto size = CalcLinearUploadLevelSize(element, mip_pitch, mip_height);
mips[mip_level] = {offset, size};
offset += size;
if (mip_pitch > 1) {
mip_pitch /= 2;
}
if (mip_height > 1) {
mip_height /= 2;
}
}
return offset;
}
bool FitsBufferRange(uint64_t offset, uint64_t size, uint64_t capacity) {
return offset <= capacity && size <= capacity - offset;
}
} // namespace
RenderTargetFormatInfo TextureGetRenderTargetFormat(Prospero::ChannelLayout layout,
@@ -176,203 +111,4 @@ SurfaceFormatInfo TextureGetSurfaceFormatInfo(Prospero::BufferFormat format) {
EXIT("unknown format: fmt = %u\n", static_cast<uint32_t>(format));
}
TextureUploadLayout TextureCalcUploadLayout(Prospero::BufferFormat format, uint32_t width,
uint32_t height, uint32_t levels, uint32_t depth,
Prospero::TileMode tile_mode, uint64_t upload_size,
bool volume_texture, const char* owner) {
TextureUploadLayout layout {};
layout.surface.description = {
format,
tile_mode,
volume_texture ? TileSurfaceDimension::Dim3D : TileSurfaceDimension::Dim2D,
width,
height,
volume_texture ? depth : 1u,
levels,
volume_texture ? 1u : depth,
};
const auto& description = layout.surface.description;
TileTextureElementLayout element {};
if (format == Prospero::BufferFormat::kInvalid) {
EXIT("%s: legacy texture upload format unsupported: fmt=0 tile=%u size=%" PRIu64
" extent=%ux%u levels=%u\n",
owner, static_cast<uint32_t>(description.tile_mode), upload_size, width, height,
levels);
}
if (tile_mode == Prospero::TileMode::kLinear) {
if (!TileGetTextureElementLayout(format, element)) {
EXIT("%s: unsupported linear texture format: fmt=%u\n", owner,
static_cast<uint32_t>(format));
}
} else {
if (!TileGetTiledTextureLayout(description, layout.surface)) {
EXIT("%s: unsupported typed tiled upload: fmt=%u tile=%u "
"size=%" PRIu64 " extent=%ux%u levels=%u\n",
owner, static_cast<uint32_t>(format), static_cast<uint32_t>(tile_mode),
upload_size, width, height, levels);
}
element = {layout.surface.texture.block.bytes_per_element,
layout.surface.texture.texel_width, layout.surface.texture.texel_height};
}
layout.pitch = TileGetTexturePitch(format, width, description.tile_mode);
if (tile_mode == Prospero::TileMode::kLinear) {
TileSizeOffset level_sizes[16] {};
TilePaddedSize padded_sizes[16] {};
TileGetTextureSize(format, width, height, levels, description.tile_mode, nullptr,
level_sizes, padded_sizes);
for (uint32_t level = 0; level < levels; ++level) {
layout.mips[level] = {level_sizes[level].offset, level_sizes[level].size,
padded_sizes[level].width, padded_sizes[level].height};
}
} else if (volume_texture) {
layout.slice_stride = SetLinearUploadLevels(layout.mips, element, height, levels, width);
} else {
layout.source_slice_stride = layout.surface.block_slice_size;
if (depth > 1 && upload_size != 0 && upload_size % depth == 0) {
layout.source_slice_stride = std::max(layout.source_slice_stride, upload_size / depth);
}
SetLinearUploadLevels(layout.mips, element, height, levels, layout.pitch);
}
if (!volume_texture || tile_mode == Prospero::TileMode::kLinear) {
layout.slice_stride = CalcTextureSliceStride(layout.mips, levels, upload_size, depth);
}
return layout;
}
std::vector<vk::BufferImageCopy> TextureBuildImageCopies(const TextureUploadLayout& layout) {
const auto& description = layout.surface.description;
const bool volume_texture = description.dimension == TileSurfaceDimension::Dim3D;
const bool linear_texture = description.tile_mode == Prospero::TileMode::kLinear;
const auto depth = volume_texture ? description.depth : description.layers;
uint32_t mip_width = description.width;
uint32_t mip_height = description.height;
uint32_t mip_pitch = volume_texture && !linear_texture ? description.width : layout.pitch;
std::vector<vk::BufferImageCopy> regions;
regions.reserve(GetTextureRegionCount(depth, description.levels, volume_texture));
for (uint32_t mip_level = 0; mip_level < description.levels; ++mip_level) {
EXIT_NOT_IMPLEMENTED(layout.mips[mip_level].size == 0);
const auto mip_depth = GetTextureLevelDepth(depth, mip_level, volume_texture);
vk::BufferImageCopy region {};
region.imageSubresource = {vk::ImageAspectFlagBits::eColor, mip_level, 0, 1};
region.imageExtent = {mip_width, mip_height, 1};
if (linear_texture) {
region.bufferRowLength = layout.mips[mip_level].row_length;
region.bufferImageHeight = layout.mips[mip_level].image_height;
} else {
const auto texel_width = layout.surface.texture.texel_width;
const auto aligned_pitch = Common::AlignUp(mip_pitch, texel_width);
region.bufferRowLength =
aligned_pitch > Common::AlignUp(mip_width, texel_width) ? aligned_pitch : 0;
}
for (uint32_t slice_index = 0; slice_index < mip_depth; ++slice_index) {
region.bufferOffset = layout.mips[mip_level].offset + slice_index * layout.slice_stride;
region.imageSubresource.baseArrayLayer = volume_texture ? 0u : slice_index;
region.imageOffset.z = volume_texture ? static_cast<int>(slice_index) : 0;
regions.push_back(region);
}
if (mip_width > 1) {
mip_width /= 2;
}
if (mip_height > 1) {
mip_height /= 2;
}
if (mip_pitch > 1) {
mip_pitch /= 2;
}
}
return regions;
}
bool TextureBuildGpuTileInfos(uint64_t tiled_size, const std::vector<vk::BufferImageCopy>& regions,
const TextureUploadLayout& layout, uint32_t levels,
std::vector<GpuTileInfo>& out_tile_infos) {
const auto& description = layout.surface.description;
const bool volume_texture = description.dimension == TileSurfaceDimension::Dim3D;
const auto depth = volume_texture ? description.depth : description.layers;
if (tiled_size == 0 || levels == 0 || levels > 16 || depth == 0 ||
regions.size() != GetTextureRegionCount(depth, levels, volume_texture) ||
Prospero::IsFmaskTextureFormat(description.format)) {
return false;
}
const auto& surface = layout.surface;
const auto& texture = surface.texture;
const auto& block = texture.block;
if (surface.description.levels < levels || block.bytes_per_element == 0 ||
block.family == TileBlockFamily::Count) {
return false;
}
std::vector<GpuTileInfo> tile_infos;
tile_infos.reserve(regions.size());
const auto slices_per_copy = volume_texture ? block.block_depth : 1u;
size_t region_base = 0;
for (uint32_t mip_level = 0; mip_level < levels; ++mip_level) {
const auto& mip = surface.mips[mip_level];
const auto mip_depth = GetTextureLevelDepth(depth, mip_level, volume_texture);
for (uint32_t slice_index = 0; slice_index < mip_depth; slice_index += slices_per_copy) {
const auto& region = regions[region_base + slice_index];
GpuTileInfo tile_info {};
tile_info.family = block.family;
tile_info.bytes_per_element = block.bytes_per_element;
tile_info.linear_offset = region.bufferOffset;
tile_info.linear_size = layout.mips[mip_level].size;
tile_info.tiled_size = mip.size;
if (volume_texture) {
tile_info.depth = std::min(slices_per_copy, mip_depth - slice_index);
tile_info.linear_slice_stride = layout.slice_stride;
tile_info.linear_size += (tile_info.depth - 1u) * layout.slice_stride;
tile_info.tiled_offset =
mip.offset +
static_cast<uint64_t>(slice_index / slices_per_copy) * surface.block_slice_size;
tile_info.surface_z =
block.block_depth == 1 ? static_cast<uint32_t>(region.imageOffset.z) : 0;
} else {
const auto source_stride = layout.source_slice_stride != 0
? layout.source_slice_stride
: surface.block_slice_size;
if (slice_index > (UINT64_MAX - mip.offset) / source_stride) {
return false;
}
tile_info.tiled_offset =
mip.offset + static_cast<uint64_t>(slice_index) * source_stride;
tile_info.surface_z = block.family == TileBlockFamily::RenderTarget64KB ||
block.family == TileBlockFamily::Depth64KB
? region.imageSubresource.baseArrayLayer
: 0;
}
if (!FitsBufferRange(tile_info.linear_offset, tile_info.linear_size, UINT64_MAX) ||
!FitsBufferRange(tile_info.tiled_offset, tile_info.tiled_size, tiled_size)) {
return false;
}
const auto row_length =
region.bufferRowLength != 0 ? region.bufferRowLength : region.imageExtent.width;
const auto image_height = region.bufferImageHeight != 0 ? region.bufferImageHeight
: region.imageExtent.height;
tile_info.width = std::max(
(region.imageExtent.width + texture.texel_width - 1u) / texture.texel_width, 1u);
tile_info.height =
std::max((image_height + texture.texel_height - 1u) / texture.texel_height, 1u);
tile_info.pitch =
std::max((row_length + texture.texel_width - 1u) / texture.texel_width, 1u);
tile_info.tail = mip_level >= surface.first_tail_level;
tile_info.tail_x = tile_info.tail ? mip.tail_x : 0;
tile_info.tail_y = tile_info.tail ? mip.tail_y : 0;
tile_info.tiled_width = mip.padded_width;
tile_info.tiled_height = mip.padded_height;
tile_infos.push_back(tile_info);
}
region_base += mip_depth;
}
out_tile_infos = std::move(tile_infos);
return true;
}
} // namespace Libs::Graphics
@@ -2,15 +2,10 @@
#define EMULATOR_INCLUDE_EMULATOR_GRAPHICS_HOST_GPU_RENDERER_IMAGE_TEXTURECOMMON_H_
#include "graphics/guest_gpu/gpu_defs.h"
#include "graphics/guest_gpu/tile.h"
#include "graphics/host_gpu/vulkanCommon.h"
#include <vector>
namespace Libs::Graphics {
struct GpuTileInfo;
struct RenderTargetFormatInfo {
vk::Format format = vk::Format::eUndefined;
uint32_t bytes_per_element = 0;
@@ -24,35 +19,12 @@ struct SurfaceFormatInfo {
Prospero::ColorComponentMapping host_to_storage;
};
struct TextureUploadMipLayout {
uint64_t offset = 0;
uint64_t size = 0;
uint32_t row_length = 0;
uint32_t image_height = 0;
};
struct TextureUploadLayout {
uint32_t pitch = 0;
uint64_t slice_stride = 0;
uint64_t source_slice_stride = 0;
TileSurfaceLayout surface;
TextureUploadMipLayout mips[16] = {};
};
vk::ComponentMapping TextureGetComponentMapping(uint32_t swizzle,
Prospero::ColorComponentMapping host_to_storage);
SurfaceFormatInfo TextureGetSurfaceFormatInfo(Prospero::BufferFormat format);
RenderTargetFormatInfo TextureGetRenderTargetFormat(Prospero::ChannelLayout layout,
Prospero::ChannelType type,
Prospero::ChannelOrder order);
TextureUploadLayout TextureCalcUploadLayout(Prospero::BufferFormat format, uint32_t width,
uint32_t height, uint32_t levels, uint32_t depth,
Prospero::TileMode tile_mode, uint64_t upload_size,
bool volume_texture, const char* owner);
std::vector<vk::BufferImageCopy> TextureBuildImageCopies(const TextureUploadLayout& layout);
bool TextureBuildGpuTileInfos(uint64_t tiled_size, const std::vector<vk::BufferImageCopy>& regions,
const TextureUploadLayout& layout, uint32_t levels,
std::vector<GpuTileInfo>& out_tile_infos);
} // namespace Libs::Graphics
+155 -152
View File
@@ -31,7 +31,12 @@ TileManager::TileManager(GraphicContext& graphics, CommandScheduler& scheduler,
StreamBuffer& stream_buffer)
: m_graphics(graphics), m_scheduler(scheduler), m_stream_buffer(stream_buffer) {
static_assert(FamilyCount == 9);
static_assert(sizeof(Push) == 60);
static_assert(sizeof(Mip) == 48);
static_assert(offsetof(TilingParams, mips) == 16);
static_assert(sizeof(TilingParams) == 784);
static_assert(sizeof(TilingParams) <= UINT16_MAX && PipelineCount <= UINT16_MAX);
static_assert(sizeof(Tiling) == 16);
static_assert(sizeof(Conversion) == 32);
std::array<vk::DescriptorSetLayoutBinding, 3> bindings {};
for (uint32_t index = 0; index < 2; index++) {
bindings[index] = {index, vk::DescriptorType::eStorageBuffer, 1,
@@ -48,7 +53,8 @@ TileManager::TileManager(GraphicContext& graphics, CommandScheduler& scheduler,
&m_descriptor_layout),
"create TileManager descriptor layout");
const vk::PushConstantRange push_range {vk::ShaderStageFlagBits::eCompute, 0, sizeof(Push)};
const vk::PushConstantRange push_range {vk::ShaderStageFlagBits::eCompute, 0,
sizeof(Conversion)};
vk::PipelineLayoutCreateInfo layout_info {};
layout_info.setLayoutCount = 1;
layout_info.pSetLayouts = &m_descriptor_layout;
@@ -88,14 +94,25 @@ TileManager::~TileManager() {
}
}
void TileManager::Prepare(bool tile, uint64_t tiled_capacity, uint64_t linear_capacity,
std::span<const GpuTileInfo> infos, uint64_t source_base,
uint64_t target_base, std::vector<Dispatch>& dispatches,
ColorTransform transform) {
EXIT_IF(infos.empty() || tiled_capacity == 0 || linear_capacity == 0);
const auto& limits = m_graphics.GetPhysicalDeviceProperties().limits;
TileManager::Tiling TileManager::Prepare(bool tile, uint64_t tiled_capacity,
uint64_t linear_capacity, const ImageInfo& info,
uint32_t levels, uint64_t source_base,
uint64_t target_base) {
levels = levels != 0 ? levels : info.resources.levels;
EXIT_IF(levels == 0 || levels > MaxMipLevels || levels > info.resources.levels ||
tiled_capacity == 0 || linear_capacity == 0);
EXIT_NOT_IMPLEMENTED(tiled_capacity > UINT32_MAX || linear_capacity > UINT32_MAX);
const auto& block = info.tiling.block;
const auto transform = info.GetColorTransform();
EXIT_NOT_IMPLEMENTED(
!info.IsTiled() || info.samples != 1 || Prospero::IsFmaskTextureFormat(info.guest_format) ||
block.family >= TileBlockFamily::Count || !std::has_single_bit(block.bytes_per_element) ||
block.bytes_per_element > 16 || block.block_width == 0 || block.block_height == 0 ||
block.block_depth == 0 || block.block_size == 0 || info.tiling.texel_width == 0 ||
info.tiling.texel_height == 0 || info.tiled_slice_stride == 0);
EXIT_IF((transform == ColorTransform::SwapBgra16 && block.bytes_per_element != 8u) ||
(transform == ColorTransform::Reverse10_11_11 && block.bytes_per_element != 4u));
const auto checked_multiply = [](uint64_t left, uint64_t right, uint64_t& result) {
return (left == 0 || right <= UINT64_MAX / left) && (result = left * right, true);
};
@@ -106,112 +123,96 @@ void TileManager::Prepare(bool tile, uint64_t tiled_capacity, uint64_t linear_ca
return size != 0 && offset <= capacity && size <= capacity - offset;
};
dispatches.clear();
dispatches.reserve(infos.size());
for (const auto& info: infos) {
EXIT_IF((transform == ColorTransform::SwapBgra16 && info.bytes_per_element != 8u) ||
(transform == ColorTransform::Reverse10_11_11 && info.bytes_per_element != 4u));
TileBlockLayout block {};
const uint32_t tiled_width = info.tiled_width != 0 ? info.tiled_width : info.pitch;
const uint32_t tiled_height = info.tiled_height != 0 ? info.tiled_height : info.height;
const uint64_t groups_x = (static_cast<uint64_t>(info.width) + 7u) / 8u;
const uint64_t groups_y = (static_cast<uint64_t>(info.height) + 7u) / 8u;
EXIT_NOT_IMPLEMENTED(
!TileGetBlockLayout(info.family, info.bytes_per_element, block) || info.width == 0 ||
info.height == 0 || info.depth == 0 || info.pitch < info.width ||
groups_x > limits.maxComputeWorkGroupCount[0] ||
groups_y > limits.maxComputeWorkGroupCount[1] ||
info.depth > limits.maxComputeWorkGroupCount[2] ||
(!info.tail && (tiled_width < info.width || tiled_height < info.height)) ||
!valid_range(info.linear_offset, info.linear_size, linear_capacity) ||
!valid_range(info.tiled_offset, info.tiled_size, tiled_capacity) ||
(block.block_depth == 1 && info.depth != 1));
// ImageInfo owns the layout. Upload only the header and active mip records.
TilingParams params;
params.color_transform = static_cast<uint32_t>(transform);
for (uint32_t level = 0; level < levels; ++level) {
const auto& mip = info.mip_layout[level];
const auto extent = info.MipExtent(level);
const auto width = static_cast<uint32_t>(
(uint64_t {extent.width} + info.tiling.texel_width - 1u) / info.tiling.texel_width);
const auto height = static_cast<uint32_t>(
(uint64_t {extent.height} + info.tiling.texel_height - 1u) / info.tiling.texel_height);
const auto depth = extent.depth;
const bool tail = level >= info.first_tail_level;
EXIT_NOT_IMPLEMENTED(width == 0 || height == 0 || depth == 0 || mip.linear_pitch < width ||
mip.linear_size == 0 || mip.size == 0 ||
(!tail && (mip.pitch < width || mip.height < height)) ||
(!info.IsVolume() && mip.size % depth != 0));
uint64_t pitch_bytes = 0;
EXIT_NOT_IMPLEMENTED(!checked_multiply(info.pitch, info.bytes_per_element, pitch_bytes) ||
pitch_bytes > UINT32_MAX);
uint64_t slice_bytes = info.linear_slice_stride;
EXIT_NOT_IMPLEMENTED(
!checked_multiply(mip.linear_pitch, block.bytes_per_element, pitch_bytes) ||
pitch_bytes > UINT32_MAX);
const uint64_t slice_bytes =
info.linear_slice_stride != 0 ? info.linear_slice_stride : mip.linear_size;
uint64_t minimum_slice = 0;
EXIT_NOT_IMPLEMENTED(!checked_multiply(pitch_bytes, info.height, minimum_slice));
if (slice_bytes == 0) {
slice_bytes = minimum_slice;
}
uint64_t linear_used = 0;
uint64_t bytes = 0;
EXIT_NOT_IMPLEMENTED((info.depth > 1 && slice_bytes < minimum_slice) ||
!checked_multiply(info.depth - 1u, slice_bytes, bytes) ||
uint64_t bytes = 0;
uint64_t linear_span = 0;
EXIT_NOT_IMPLEMENTED(!checked_multiply(pitch_bytes, height, minimum_slice) ||
(depth > 1 && slice_bytes < minimum_slice) ||
!checked_multiply(height - 1u, pitch_bytes, linear_used) ||
!checked_multiply(width, block.bytes_per_element, bytes) ||
!checked_add(linear_used, bytes, linear_used) ||
!checked_multiply(info.height - 1u, pitch_bytes, bytes) ||
!checked_add(linear_used, bytes, linear_used) ||
!checked_multiply(info.width, info.bytes_per_element, bytes) ||
!checked_add(linear_used, bytes, linear_used) ||
linear_used > info.linear_size || slice_bytes > UINT32_MAX);
linear_used > mip.linear_size || slice_bytes > UINT32_MAX ||
!checked_multiply(depth - 1u, slice_bytes, linear_span) ||
!checked_add(linear_span, mip.linear_size, linear_span) ||
!valid_range(mip.linear_offset, linear_span, linear_capacity));
const uint64_t columns =
(static_cast<uint64_t>(tiled_width) + block.block_width - 1u) / block.block_width;
const uint64_t rows =
(static_cast<uint64_t>(tiled_height) + block.block_height - 1u) / block.block_height;
uint64_t blocks_per_slice = 0;
EXIT_NOT_IMPLEMENTED(!checked_multiply(columns, rows, blocks_per_slice) ||
columns > UINT32_MAX || blocks_per_slice > UINT32_MAX);
if (info.tail) {
(uint64_t {mip.pitch} + block.block_width - 1u) / block.block_width;
if (tail) {
EXIT_NOT_IMPLEMENTED(
info.family == TileBlockFamily::Standard256B || info.depth > block.block_depth ||
info.tail_x >= block.block_width || info.width > block.block_width - info.tail_x ||
info.tail_y >= block.block_height ||
info.height > block.block_height - info.tail_y ||
info.tiled_size < block.block_size);
} else {
const uint64_t slices =
(static_cast<uint64_t>(info.depth) + block.block_depth - 1u) / block.block_depth;
uint64_t tiled_used = 0;
EXIT_NOT_IMPLEMENTED(!checked_multiply(blocks_per_slice, slices, tiled_used) ||
!checked_multiply(tiled_used, block.block_size, tiled_used) ||
tiled_used > info.tiled_size);
block.family == TileBlockFamily::Standard256B || mip.tail_x >= block.block_width ||
width > block.block_width - mip.tail_x || mip.tail_y >= block.block_height ||
height > block.block_height - mip.tail_y);
}
const uint64_t tiled_plane = info.IsVolume() ? mip.size : mip.size / depth;
const uint64_t tiled_slice = info.tiled_slice_stride;
const uint64_t slices = (uint64_t {depth} + block.block_depth - 1u) / block.block_depth;
uint64_t tiled_span = 0;
EXIT_NOT_IMPLEMENTED(columns > UINT32_MAX || tiled_slice > UINT32_MAX ||
(slices > 1 && tiled_slice < tiled_plane) ||
!checked_multiply(slices - 1u, tiled_slice, tiled_span) ||
!checked_add(tiled_span, tiled_plane, tiled_span) ||
!valid_range(mip.offset, tiled_span, tiled_capacity));
const uint32_t alignment = std::min(info.bytes_per_element, 4u);
EXIT_NOT_IMPLEMENTED(((info.linear_offset | info.tiled_offset | pitch_bytes | slice_bytes) &
(alignment - 1u)) != 0);
const uint64_t src = source_base + (tile ? info.linear_offset : info.tiled_offset);
const uint64_t dst = target_base + (tile ? info.tiled_offset : info.linear_offset);
EXIT_NOT_IMPLEMENTED(src > UINT32_MAX || dst > UINT32_MAX);
const uint32_t family_index = static_cast<uint32_t>(info.family);
const uint32_t element_index = std::countr_zero(info.bytes_per_element);
EXIT_NOT_IMPLEMENTED(family_index >= FamilyCount || element_index >= BytesPerElementCount);
Dispatch dispatch {};
dispatch.pipeline_slot =
((tile ? FamilyCount : 0u) + family_index) * BytesPerElementCount + element_index;
dispatch.push.src_base = static_cast<uint32_t>(src);
dispatch.push.dst_base = static_cast<uint32_t>(dst);
dispatch.push.width = info.width;
dispatch.push.height = info.height;
dispatch.push.depth = info.depth;
dispatch.push.surface_z = info.surface_z;
dispatch.push.pitch_bytes = static_cast<uint32_t>(pitch_bytes);
dispatch.push.slice_bytes = static_cast<uint32_t>(slice_bytes);
dispatch.push.blocks_per_row = static_cast<uint32_t>(columns);
dispatch.push.blocks_per_slice = static_cast<uint32_t>(blocks_per_slice);
dispatch.push.tail_x = info.tail_x;
dispatch.push.tail_y = info.tail_y;
dispatch.push.tail = info.tail;
dispatch.push.color_transform = static_cast<uint32_t>(transform);
dispatches.push_back(dispatch);
const uint32_t alignment = std::min(block.bytes_per_element, 4u);
EXIT_NOT_IMPLEMENTED(
((mip.linear_offset | mip.offset | pitch_bytes | slice_bytes | tiled_slice) &
(alignment - 1u)) != 0);
uint64_t src = 0, dst = 0, texels = 0;
EXIT_NOT_IMPLEMENTED(
!checked_add(source_base, tile ? mip.linear_offset : mip.offset, src) ||
!checked_add(target_base, tile ? mip.offset : mip.linear_offset, dst) ||
src > UINT32_MAX || dst > UINT32_MAX || !checked_multiply(width, height, texels) ||
!checked_multiply(texels, depth, texels) || texels > UINT32_MAX - params.num_texels);
params.mips[level] = {static_cast<uint32_t>(src),
static_cast<uint32_t>(dst),
width,
height,
static_cast<uint32_t>(pitch_bytes),
static_cast<uint32_t>(slice_bytes),
static_cast<uint32_t>(tiled_slice),
static_cast<uint32_t>(columns),
mip.surface_z,
tail ? mip.tail_x : 0u,
tail ? mip.tail_y : 0u,
static_cast<uint32_t>(texels)};
params.num_texels += static_cast<uint32_t>(texels);
}
const uint64_t uniform_alignment =
std::max<uint64_t>(limits.minUniformBufferOffsetAlignment, 1);
const uint64_t stride = Common::AlignUp<uint64_t>(sizeof(Push), uniform_alignment);
EXIT_NOT_IMPLEMENTED(dispatches.size() > UINT64_MAX / stride);
const uint64_t bytes = dispatches.size() * stride;
auto [mapped, offset] = m_stream_buffer.Map(bytes, uniform_alignment);
EXIT_IF(mapped == nullptr);
for (size_t index = 0; index < dispatches.size(); index++) {
std::memcpy(mapped + index * stride, &dispatches[index].push, sizeof(Push));
dispatches[index].params_offset = offset + index * stride;
}
m_stream_buffer.Commit();
const auto& limits = m_graphics.GetPhysicalDeviceProperties().limits;
const auto params_size =
static_cast<uint16_t>(offsetof(TilingParams, mips) + levels * sizeof(Mip));
const auto offset = m_stream_buffer.Copy(
&params, params_size,
std::max<uint64_t>(limits.minUniformBufferOffsetAlignment, alignof(TilingParams)));
const auto slot =
((tile ? FamilyCount : 0u) + static_cast<uint32_t>(block.family)) * BytesPerElementCount +
std::countr_zero(block.bytes_per_element);
return {offset, params.num_texels, params_size, static_cast<uint16_t>(slot)};
}
vk::Pipeline TileManager::GetPipeline(uint32_t slot) {
@@ -257,10 +258,9 @@ vk::Pipeline TileManager::GetPipeline(uint32_t slot) {
return m_pipelines[slot];
}
void TileManager::Record(vk::Buffer source, uint64_t source_offset,
uint64_t source_capacity, vk::Buffer target, uint64_t target_offset,
uint64_t target_capacity, std::span<Dispatch> dispatches,
bool clear_target) {
void TileManager::Record(vk::Buffer source, uint64_t source_offset, uint64_t source_capacity,
vk::Buffer target, uint64_t target_offset, uint64_t target_capacity,
const Tiling& tiling, bool clear_target) {
const auto& limits = m_graphics.GetPhysicalDeviceProperties().limits;
const uint64_t descriptor_alignment =
std::max<uint64_t>(limits.minStorageBufferOffsetAlignment, 4);
@@ -295,9 +295,8 @@ void TileManager::Record(vk::Buffer source, uint64_t source_offset,
barriers[2].srcAccessMask = vk::AccessFlagBits::eHostWrite;
barriers[2].dstAccessMask = vk::AccessFlagBits::eUniformRead;
barriers[2].buffer = m_stream_buffer.Handle();
barriers[2].offset = dispatches.front().params_offset;
barriers[2].size =
dispatches.back().params_offset - dispatches.front().params_offset + sizeof(Push);
barriers[2].offset = tiling.params_offset;
barriers[2].size = tiling.params_size;
command.pipelineBarrier(
vk::PipelineStageFlagBits::eAllCommands | vk::PipelineStageFlagBits::eHost,
vk::PipelineStageFlagBits::eComputeShader | vk::PipelineStageFlagBits::eTransfer, {}, 0,
@@ -314,24 +313,26 @@ void TileManager::Record(vk::Buffer source, uint64_t source_offset,
const vk::DescriptorBufferInfo source_info {source, source_descriptor_offset, source_range};
const vk::DescriptorBufferInfo target_info {target, target_descriptor_offset, target_range};
for (auto& dispatch: dispatches) {
const vk::DescriptorBufferInfo params_info {m_stream_buffer.Handle(),
dispatch.params_offset, sizeof(Push)};
const vk::DescriptorBufferInfo infos[] {source_info, target_info, params_info};
std::array<vk::WriteDescriptorSet, 3> writes {};
for (uint32_t index = 0; index < writes.size(); index++) {
writes[index].dstBinding = index;
writes[index].descriptorCount = 1;
writes[index].descriptorType = index == 2 ? vk::DescriptorType::eUniformBuffer
: vk::DescriptorType::eStorageBuffer;
writes[index].pBufferInfo = &infos[index];
}
command.pushDescriptorSetKHR(vk::PipelineBindPoint::eCompute, m_pipeline_layout, 0,
static_cast<uint32_t>(writes.size()), writes.data());
command.bindPipeline(vk::PipelineBindPoint::eCompute, GetPipeline(dispatch.pipeline_slot));
command.dispatch((dispatch.push.width + 7u) / 8u, (dispatch.push.height + 7u) / 8u,
dispatch.push.depth);
const vk::DescriptorBufferInfo params_info {m_stream_buffer.Handle(), tiling.params_offset,
tiling.params_size};
const vk::DescriptorBufferInfo infos[] {source_info, target_info, params_info};
std::array<vk::WriteDescriptorSet, 3> writes {};
for (uint32_t index = 0; index < writes.size(); index++) {
writes[index].dstBinding = index;
writes[index].descriptorCount = 1;
writes[index].descriptorType =
index == 2 ? vk::DescriptorType::eUniformBuffer : vk::DescriptorType::eStorageBuffer;
writes[index].pBufferInfo = &infos[index];
}
command.pushDescriptorSetKHR(vk::PipelineBindPoint::eCompute, m_pipeline_layout, 0,
static_cast<uint32_t>(writes.size()), writes.data());
command.bindPipeline(vk::PipelineBindPoint::eCompute, GetPipeline(tiling.pipeline_slot));
const uint64_t groups = (uint64_t {tiling.num_texels} + 63u) / 64u;
const uint32_t groups_y = static_cast<uint32_t>(
(groups + limits.maxComputeWorkGroupCount[0] - 1u) / limits.maxComputeWorkGroupCount[0]);
const uint32_t groups_x = static_cast<uint32_t>((groups + groups_y - 1u) / groups_y);
EXIT_NOT_IMPLEMENTED(groups_y > limits.maxComputeWorkGroupCount[1]);
command.dispatch(groups_x, groups_y, 1);
barriers[1].srcAccessMask = vk::AccessFlagBits::eShaderWrite;
barriers[1].dstAccessMask = vk::AccessFlagBits::eTransferRead | vk::AccessFlagBits::eMemoryRead;
@@ -341,51 +342,53 @@ void TileManager::Record(vk::Buffer source, uint64_t source_offset,
}
TileManager::Result TileManager::Detile(vk::Buffer tiled, uint64_t tiled_offset,
uint64_t tiled_capacity, uint64_t linear_capacity,
std::span<const GpuTileInfo> infos,
ColorTransform transform) {
const ImageInfo& info) {
if (!info.IsTiled()) {
const Result source {tiled, tiled_offset, info.data.size};
return info.GetColorTransform() == ColorTransform::None
? source
: TransformColor(source, info.GetColorTransform(), true);
}
const auto& limits = m_graphics.GetPhysicalDeviceProperties().limits;
const uint64_t descriptor_alignment =
std::max<uint64_t>(limits.minStorageBufferOffsetAlignment, 4);
const uint64_t source_base = tiled_offset & (descriptor_alignment - 1);
std::vector<Dispatch> dispatches;
Prepare(false, tiled_capacity, linear_capacity, infos, source_base, 0, dispatches, transform);
const uint64_t source_base = tiled_offset & (descriptor_alignment - 1);
const auto linear_capacity = info.LinearSize();
const auto tiling = Prepare(false, info.data.size, linear_capacity, info, 0, source_base, 0);
auto scratch = GetScratchBuffer(linear_capacity, tiled);
Record(tiled, tiled_offset, tiled_capacity, scratch.buffer, 0, scratch.size, dispatches,
true);
Record(tiled, tiled_offset, info.data.size, scratch.buffer, 0, scratch.size, tiling, true);
return {scratch.buffer, 0, linear_capacity};
}
void TileManager::Tile(vk::Buffer linear, uint64_t linear_offset, uint64_t linear_capacity,
vk::Buffer tiled, uint64_t tiled_offset, uint64_t tiled_capacity,
std::span<const GpuTileInfo> infos) {
const ImageInfo& info, uint32_t levels) {
const auto& limits = m_graphics.GetPhysicalDeviceProperties().limits;
const uint64_t descriptor_alignment =
std::max<uint64_t>(limits.minStorageBufferOffsetAlignment, 4);
const uint64_t source_base = linear_offset & (descriptor_alignment - 1);
const uint64_t target_base = tiled_offset & (descriptor_alignment - 1);
std::vector<Dispatch> dispatches;
Prepare(true, tiled_capacity, linear_capacity, infos, source_base, target_base, dispatches);
Record(linear, linear_offset, linear_capacity, tiled, tiled_offset, tiled_capacity,
dispatches, false);
const uint64_t source_base = linear_offset & (descriptor_alignment - 1);
const uint64_t target_base = tiled_offset & (descriptor_alignment - 1);
const auto tiling =
Prepare(true, tiled_capacity, linear_capacity, info, levels, source_base, target_base);
Record(linear, linear_offset, linear_capacity, tiled, tiled_offset, tiled_capacity, tiling,
false);
}
void TileManager::TileImage(Image& image, std::span<const vk::BufferImageCopy> regions,
vk::Buffer tiled, uint64_t tiled_offset, uint64_t tiled_capacity,
uint64_t linear_capacity, std::span<const GpuTileInfo> infos,
ColorTransform transform) {
uint32_t levels) {
EXIT_IF(regions.empty());
const auto& limits = m_graphics.GetPhysicalDeviceProperties().limits;
const uint64_t descriptor_alignment =
std::max<uint64_t>(limits.minStorageBufferOffsetAlignment, 4);
const uint64_t target_base = tiled_offset & (descriptor_alignment - 1);
std::vector<Dispatch> dispatches;
const uint64_t target_base = tiled_offset & (descriptor_alignment - 1);
const auto linear_capacity = image.info.LinearSize(levels);
// Reserve stream parameters before recording the image download.
Prepare(true, tiled_capacity, linear_capacity, infos, 0, target_base, dispatches, transform);
const auto tiling =
Prepare(true, tiled_capacity, linear_capacity, image.info, levels, 0, target_base);
auto linear = GetScratchBuffer(linear_capacity, tiled);
image.Download(regions, linear.buffer, 0, linear.size);
Record(linear.buffer, 0, linear_capacity, tiled, tiled_offset, tiled_capacity,
dispatches, false);
Record(linear.buffer, 0, linear_capacity, tiled, tiled_offset, tiled_capacity, tiling, false);
}
TileManager::Result TileManager::GetScratchBuffer(uint64_t size, vk::Buffer input) {
@@ -563,13 +566,13 @@ void TileManager::ConvertD16(Result source, Result target, D16Direction directio
}
command.pushDescriptorSetKHR(vk::PipelineBindPoint::eCompute, m_pipeline_layout, 0,
static_cast<uint32_t>(writes.size()), writes.data());
Push push {};
Conversion push {};
push.src_base = source_binding.base;
push.dst_base = target_binding.base;
push.width = layout.width;
push.height = rows;
push.pitch_bytes = static_cast<uint32_t>(layout.source_row_stride);
push.slice_bytes = static_cast<uint32_t>(layout.target_row_stride);
push.source_pitch = static_cast<uint32_t>(layout.source_row_stride);
push.target_pitch = static_cast<uint32_t>(layout.target_row_stride);
command.pushConstants(m_pipeline_layout, vk::ShaderStageFlagBits::eCompute, 0,
sizeof(push), &push);
command.dispatch(static_cast<uint32_t>(groups_x), rows, 1);
@@ -644,7 +647,7 @@ void TileManager::TransformColor(Result input, Result output, ColorTransform tra
command.bindPipeline(vk::PipelineBindPoint::eCompute, m_color_transform);
command.pushDescriptorSetKHR(vk::PipelineBindPoint::eCompute, m_pipeline_layout, 0,
static_cast<uint32_t>(writes.size()), writes.data());
Push push {};
Conversion push {};
push.src_base = input_binding.base;
push.dst_base = output_binding.base;
push.width = pixels;
+35 -44
View File
@@ -9,7 +9,6 @@
#include <array>
#include <memory>
#include <span>
#include <vector>
namespace Libs::Graphics {
@@ -20,26 +19,6 @@ class StreamBuffer;
struct GraphicContext;
struct TileManagerTestAccess;
struct GpuTileInfo {
TileBlockFamily family = TileBlockFamily::Count;
uint32_t bytes_per_element = 0;
uint64_t linear_offset = 0;
uint64_t linear_size = 0;
uint64_t tiled_offset = 0;
uint64_t tiled_size = 0;
uint64_t linear_slice_stride = 0;
uint32_t width = 0;
uint32_t height = 0;
uint32_t depth = 1;
uint32_t pitch = 0;
uint32_t tail_x = 0;
uint32_t tail_y = 0;
bool tail = false;
uint32_t tiled_width = 0;
uint32_t tiled_height = 0;
uint32_t surface_z = 0;
};
class TileManager final {
public:
enum class D16Direction { Promote, Demote };
@@ -64,15 +43,12 @@ public:
KYTY_CLASS_NO_COPY(TileManager);
// Consume scratch results before the next acquisition, or pass their buffer as input.
[[nodiscard]] Result Detile(vk::Buffer tiled, uint64_t tiled_offset, uint64_t tiled_capacity,
uint64_t linear_capacity, std::span<const GpuTileInfo> infos,
ColorTransform transform = ColorTransform::None);
[[nodiscard]] Result Detile(vk::Buffer tiled, uint64_t tiled_offset, const ImageInfo& info);
void Tile(vk::Buffer linear, uint64_t linear_offset, uint64_t linear_capacity, vk::Buffer tiled,
uint64_t tiled_offset, uint64_t tiled_capacity, std::span<const GpuTileInfo> infos);
uint64_t tiled_offset, uint64_t tiled_capacity, const ImageInfo& info,
uint32_t levels = 0);
void TileImage(Image& image, std::span<const vk::BufferImageCopy> regions, vk::Buffer tiled,
uint64_t tiled_offset, uint64_t tiled_capacity, uint64_t linear_capacity,
std::span<const GpuTileInfo> infos,
ColorTransform transform = ColorTransform::None);
uint64_t tiled_offset, uint64_t tiled_capacity, uint32_t levels = 0);
[[nodiscard]] Result GetScratchBuffer(uint64_t size, vk::Buffer input = nullptr);
void ConvertD16(Result source, Result target, D16Direction direction, bool d32,
const D16Layout& layout);
@@ -86,29 +62,44 @@ private:
static constexpr uint32_t BytesPerElementCount = 5;
static constexpr uint32_t DirectionCount = 2;
static constexpr uint32_t PipelineCount = FamilyCount * BytesPerElementCount * DirectionCount;
static constexpr uint32_t MaxMipLevels = 16;
struct Push {
struct alignas(16) Mip {
uint32_t src_base;
uint32_t dst_base;
uint32_t width;
uint32_t height;
uint32_t depth;
uint32_t surface_z;
uint32_t pitch_bytes;
uint32_t slice_bytes;
uint32_t tiled_slice_bytes;
uint32_t blocks_per_row;
uint32_t blocks_per_slice;
uint32_t surface_z;
uint32_t tail_x;
uint32_t tail_y;
uint32_t tail;
uint32_t num_texels;
};
struct TilingParams {
uint32_t num_texels = 0;
uint32_t color_transform = 0;
std::array<uint32_t, 2> padding {};
std::array<Mip, MaxMipLevels> mips;
};
struct Tiling {
uint64_t params_offset;
uint32_t num_texels;
uint16_t params_size;
uint16_t pipeline_slot;
};
struct Conversion {
uint32_t src_base;
uint32_t dst_base;
uint32_t width;
uint32_t height;
uint32_t source_pitch;
uint32_t target_pitch;
uint32_t color_transform;
uint32_t to_host;
};
struct Dispatch {
Push push {};
uint32_t pipeline_slot = 0;
uint64_t params_offset = 0;
};
struct StorageBinding {
vk::DescriptorBufferInfo info;
uint32_t base = 0;
@@ -119,12 +110,12 @@ private:
uint64_t active, uint32_t remaining,
uint64_t alignment, uint64_t max_range,
uint32_t max_groups) noexcept;
void Prepare(bool tile, uint64_t tiled_capacity, uint64_t linear_capacity,
std::span<const GpuTileInfo> infos, uint64_t source_base, uint64_t target_base,
std::vector<Dispatch>& dispatches, ColorTransform transform = ColorTransform::None);
void Record(vk::Buffer source, uint64_t source_offset, uint64_t source_capacity,
vk::Buffer target, uint64_t target_offset, uint64_t target_capacity,
std::span<Dispatch> dispatches, bool clear_target);
[[nodiscard]] Tiling Prepare(bool tile, uint64_t tiled_capacity, uint64_t linear_capacity,
const ImageInfo& info, uint32_t levels, uint64_t source_base,
uint64_t target_base);
void Record(vk::Buffer source, uint64_t source_offset, uint64_t source_capacity,
vk::Buffer target, uint64_t target_offset, uint64_t target_capacity,
const Tiling& tiling, bool clear_target);
[[nodiscard]] vk::Pipeline GetPipeline(uint32_t slot);
GraphicContext& m_graphics;
@@ -26,7 +26,6 @@
#include "graphics/host_gpu/renderer/renderContext.h"
#include "graphics/host_gpu/vulkanCommon.h"
#include "graphics/shader/recompiler/ir/ShaderIR.h"
#include "graphics/shader/recompiler/ir/passes/BindingLayout.h"
#include "graphics/shader/recompiler/ir/passes/ResourceMaterialization.h"
#include "graphics/shader/shader.h"
#include "kernel/memory.h"
@@ -76,10 +75,6 @@ vk::DescriptorType NativeDescriptorType(BindingKind kind) {
EXIT("invalid native descriptor binding kind");
}
uint32_t NativeDescriptorCount(const ShaderRecompiler::IR::DescriptorBinding& binding) {
return binding.resources.empty() ? 1u : static_cast<uint32_t>(binding.resources.size());
}
vk::DescriptorImageInfo MakeImageInfo(const TextureBinding& texture, uint32_t element) {
vk::ImageView view = nullptr;
if (texture.mip_views.empty()) {
@@ -383,51 +378,6 @@ static TextureCache::ImageDesc NullTextureDesc(const ShaderRecompiler::IR::Image
return desc;
}
static void PopulateTextureMipLayout(ImageInfo& info) {
if (info.IsVolume() && info.tile_mode != Prospero::TileMode::kLinear) {
TileSurfaceLayout surface {};
const TileSurfaceDescription description {
info.guest_format, info.tile_mode, TileSurfaceDimension::Dim3D, info.extent.width,
info.extent.height, info.extent.depth, info.resources.levels, 1};
if (!TileGetTiledTextureLayout(description, surface)) {
EXIT("unsupported normalized volume texture layout\n");
}
for (uint32_t level = 0; level < info.resources.levels; level++) {
const auto& mip = surface.mips[level];
info.mip_layout[level] = {
mip.offset,
mip.size,
mip.padded_width,
mip.padded_height,
};
}
return;
}
TileSizeOffset levels[16] {};
TilePaddedSize padded[16] {};
TileGetTextureSize(info.guest_format, info.extent.width, info.extent.height,
info.resources.levels, info.tile_mode, nullptr, levels, padded);
const auto texel_shift = info.IsBlock() ? 2u : 0u;
for (uint32_t level = 0; level < info.resources.levels; level++) {
const auto offset =
levels[level].src_size != 0 ? levels[level].src_offset : levels[level].offset;
auto size = static_cast<uint64_t>(levels[level].src_size != 0 ? levels[level].src_size
: levels[level].size);
if (info.IsVolume()) {
size *= std::max(info.extent.depth >> level, 1u);
} else {
size *= info.resources.layers;
}
info.mip_layout[level] = {
offset,
size,
padded[level].width >> texel_shift,
padded[level].height >> texel_shift,
};
}
}
static ImageViewInfo TextureViewInfo(const ShaderRecompiler::IR::ImageResource& resource,
const ShaderTextureResource& descriptor, vk::Format format,
const SurfaceFormatInfo& surface_format, bool storage,
@@ -495,45 +445,54 @@ static ImageViewInfo TextureViewInfo(const ShaderRecompiler::IR::ImageResource&
return view;
}
static bool ResolveTextureMipView(const TileSurfaceDescription& description, bool metadata,
uint32_t view_levels, uint32_t& levels, uint32_t& base_level) {
TileSurfaceLayout physical {};
TileSurfaceLayout view {};
auto view_description = description;
view_description.levels = levels;
if (!TileGetTiledTextureLayout(description, physical) ||
!TileGetTiledTextureLayout(view_description, view)) {
static bool ResolveTextureMipView(ImageInfo& info, uint32_t physical_levels, bool metadata,
uint32_t view_levels, uint32_t& base_level) {
if (!info.IsTiled()) {
return false;
}
if (physical.first_tail_level == view.first_tail_level &&
physical.block_slice_size == view.block_slice_size &&
physical.total_size == view.total_size &&
std::equal(std::begin(physical.mips), std::begin(physical.mips) + description.levels,
std::begin(view.mips))) {
return true;
ImageInfo physical = info;
physical.resources.levels = physical_levels;
physical.UpdateSize();
info.UpdateSize();
const auto same_mip = [&](uint32_t target, uint32_t source) {
const auto& left = physical.mip_layout[target];
const auto& right = info.mip_layout[source];
return left.offset == right.offset && left.size == right.size &&
left.pitch == right.pitch && left.height == right.height &&
left.tail_x == right.tail_x && left.tail_y == right.tail_y &&
left.surface_z == right.surface_z &&
physical.MipExtent(target) == info.MipExtent(source);
};
if (physical.first_tail_level == info.first_tail_level &&
physical.tiled_slice_stride == info.tiled_slice_stride &&
physical.data.size == info.data.size) {
bool matches = true;
for (uint32_t level = 0; level < physical_levels; ++level) {
matches &= same_mip(level, level);
}
if (matches) {
return true;
}
}
if (metadata || ((description.layers > 1 || description.depth > 1) &&
physical.block_slice_size != view.block_slice_size)) {
if (metadata ||
(info.TransferLayers() > 1 && physical.tiled_slice_stride != info.tiled_slice_stride)) {
return false;
}
// T# addresses the last mip. A view can select the same stored subresources
// with different mip indices; inaccessible mips need no host representation.
for (uint32_t base = 0; base + view_levels <= description.levels; ++base) {
for (uint32_t base = 0; base + view_levels <= physical_levels; ++base) {
bool matches = true;
for (uint32_t i = 0; i < view_levels; ++i) {
const auto source = base_level + i;
const auto target = base + i;
if (physical.mips[target] != view.mips[source] ||
(target >= physical.first_tail_level) != (source >= view.first_tail_level) ||
std::max(description.width >> target, 1u) != std::max(description.width >> source, 1u) ||
std::max(description.height >> target, 1u) != std::max(description.height >> source, 1u) ||
std::max(description.depth >> target, 1u) != std::max(description.depth >> source, 1u)) {
if (!same_mip(target, source) ||
(target >= physical.first_tail_level) != (source >= info.first_tail_level)) {
matches = false;
break;
}
}
if (matches) {
levels = description.levels;
info = std::move(physical);
base_level = base;
return true;
}
@@ -618,40 +577,6 @@ TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageR
type == Prospero::ImageType::kColor2DMsaaArray;
const auto image_layers = layered ? depth : 1u;
auto view_base = static_cast<uint32_t>(base_level);
if (levels > physical_levels) {
const TileSurfaceDescription physical {
format, tile, volume ? TileSurfaceDimension::Dim3D : TileSurfaceDimension::Dim2D,
width, height, volume ? depth : 1u, physical_levels, image_layers};
if (!ResolveTextureMipView(physical, !resource.r128 && descriptor.MetaCompress(),
view_levels, levels, view_base)) {
EXIT("unsupported texture mip view changes physical layout: base=%u last=%u max=%u "
"extent=%ux%ux%u tile=%u\n",
base_level, last_level, max_mip, width, height, depth,
static_cast<uint32_t>(tile));
}
}
uint32_t pitch = 0;
TileSizeAlign size {};
if (multisampled) {
const auto bytes = Prospero::NumBytesPerElement(format);
pitch = depth_tile ? TileGetDepthPitch(width, bytes, last_level)
: TileGetRenderTargetPitch(width, bytes, last_level);
if (pitch == 0 || !TileGetRenderTargetSize(width, height, pitch, bytes, size, last_level) ||
size.size > UINT32_MAX / image_layers) {
EXIT("unsupported multisample texture layout\n");
}
size.size *= image_layers;
} else {
pitch = TileGetTexturePitch(format, width, tile);
TileGetTextureTotalSize(format, width, height, volume ? depth : image_layers,
physical_levels, tile, volume, size);
}
EXIT_NOT_IMPLEMENTED(size.size == 0 || size.align == 0 ||
(address & (static_cast<uint64_t>(size.align) - 1u)) != 0);
if (storage) {
ValidateStorageTexture(resource, descriptor, size.size);
}
auto pixel_format = surface_format.vk_format;
if (resource.depth_compare) {
if (const auto* depth_format = FindGuestDepthFormatPolicy(format)) {
@@ -668,17 +593,35 @@ TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageR
: pixel_format;
const auto block_bytes = Prospero::BlockCompressedBytesPerBlock(format);
TextureCache::ImageDesc desc {};
desc.info.data = {address, size.size};
desc.info.data = {address, 0};
desc.info.pixel_format = pixel_format;
desc.info.guest_format = format;
desc.info.type = TextureBaseType(type);
desc.info.extent = {width, height, volume ? depth : 1u};
desc.info.resources = {levels, image_layers};
desc.info.pitch = pitch;
desc.info.bytes_per_block =
block_bytes != 0 ? block_bytes : Prospero::NumBytesPerElement(format);
desc.info.samples = samples;
desc.info.tile_mode = tile;
if (levels > physical_levels) {
if (!ResolveTextureMipView(desc.info, physical_levels,
!resource.r128 && descriptor.MetaCompress(), view_levels,
view_base)) {
EXIT("unsupported texture mip view changes physical layout: base=%u last=%u max=%u "
"extent=%ux%ux%u tile=%u\n",
base_level, last_level, max_mip, width, height, depth,
static_cast<uint32_t>(tile));
}
} else {
desc.info.UpdateSize();
}
const auto alignment =
tile == Prospero::TileMode::kLinear ? 256u : desc.info.tiling.block.block_size;
EXIT_NOT_IMPLEMENTED(desc.info.data.size == 0 || alignment == 0 ||
(address & (static_cast<uint64_t>(alignment) - 1u)) != 0);
if (storage) {
ValidateStorageTexture(resource, descriptor, desc.info.data.size);
}
if (!resource.r128 && descriptor.MetaCompress() && tile != Prospero::TileMode::kDepth &&
!desc.info.IsDepth()) {
TileSizeAlign metadata_size {};
@@ -689,11 +632,6 @@ TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageR
desc.info.metadata.range = {descriptor.MetaAddr() << 8u, metadata_size.size};
desc.info.metadata.dcc_alpha_msb = descriptor.DccAlphaPos();
}
if (samples > 1) {
desc.info.mip_layout[0] = {0, size.size, pitch, height};
} else {
PopulateTextureMipLayout(desc.info);
}
desc.view_info = TextureViewInfo(resource, descriptor, view_format, surface_format, storage,
view_levels, desc.info.resources.layers);
desc.view_info.base_level = view_base;
@@ -709,7 +647,8 @@ TextureBinding RenderExecutor::ResolveTexture(const ShaderRecompiler::IR::ImageR
if (storage) {
EXIT("depth target cannot be bound as a storage image\n");
}
ValidateSampledDepthBinding(resource, descriptor, *image, pixel_format, size.size);
ValidateSampledDepthBinding(resource, descriptor, *image, pixel_format,
desc.info.data.size);
} else if (storage) {
ValidateStorageColorView(image->info.pixel_format, view_format, descriptor.DstSelXYZW());
} else {
@@ -799,12 +738,11 @@ void RenderExecutor::PrepareBindings(const ShaderStageRuntime& runtime,
m_context, program, i, snapshot.samplers[program.info.samplers[i].snapshot_index]));
}
prepared.shader_data.reserve(program.bindings.ShaderDataDwords());
for (const auto reg: program.bindings.user_data_registers) {
for (const auto reg: program.info.user_data_registers) {
prepared.shader_data.push_back(snapshot.user_data[reg - program.user_data_base]);
}
prepared.shader_data.resize(program.bindings.ShaderDataDwords());
if (ShaderRecompiler::IR::FindBinding(
program.bindings, ShaderRecompiler::IR::DescriptorBindingKind::Gds) != nullptr) {
if (program.bindings.descriptor_counts[static_cast<size_t>(BindingKind::Gds)] != 0) {
prepared.gds.buffer = m_context.GetBufferCache().GetGdsBuffer()->Handle();
}
}
@@ -817,19 +755,20 @@ void RenderExecutor::FindBuffers(std::span<PreparedBindings* const> stages) {
EXIT_IF(prepared.runtime == nullptr || !*prepared.runtime);
const auto& program = *prepared.runtime->program;
const auto& snapshot = *prepared.runtime->resources;
prepared.buffer_sources.clear();
const auto& layout = program.bindings;
if (layout.memory_offset_count == 0) {
prepared.buffer_sources.resize(layout.descriptor_counts[static_cast<size_t>(BindingKind::Buffers)]);
if (layout.descriptor_counts[static_cast<size_t>(BindingKind::Buffers)] == 0) {
continue;
}
const auto& resources = layout.descriptors.front().resources;
prepared.buffer_sources.reserve(resources.size());
for (const auto resource: resources) {
for (uint32_t resource = 0; resource < program.info.buffers.size(); ++resource) {
const auto& metadata = program.info.buffers[resource];
if (metadata.descriptor_index == UINT32_MAX) continue;
auto& source = prepared.buffer_sources[metadata.descriptor_index];
const auto descriptor = DecodeNativeDescriptor<ShaderBufferResource>(snapshot.buffers[resource]);
const auto address = descriptor.Base48();
auto size = descriptor.GetSize();
if (address == 0 || size == 0) {
prepared.buffer_sources.push_back({});
source = {};
continue;
}
if (descriptor.NumRecords() == UINT32_MAX) {
@@ -853,7 +792,7 @@ void RenderExecutor::FindBuffers(std::span<PreparedBindings* const> stages) {
}
}
size = Libs::LibKernel::Memory::ClampRangeSize(address, size);
prepared.buffer_sources.push_back({address, size, cache.FindBuffer(address, size)});
source = {address, size, cache.FindBuffer(address, size)};
}
}
}
@@ -864,10 +803,9 @@ void RenderExecutor::RebindBuffers(PreparedBindings& prepared) {
const auto& program = *prepared.runtime->program;
const auto& snapshot = *prepared.runtime->resources;
const auto& layout = program.bindings;
EXIT_IF(prepared.buffer_sources.size() != layout.memory_offset_count);
EXIT_IF(prepared.buffer_sources.size() != layout.descriptor_counts[static_cast<size_t>(BindingKind::Buffers)]);
prepared.buffers.clear();
prepared.buffers.reserve(layout.memory_offset_count);
prepared.buffers.resize(layout.descriptor_counts[static_cast<size_t>(BindingKind::Buffers)]);
EXIT_IF(prepared.shader_data.size() != layout.ShaderDataDwords());
std::fill(prepared.shader_data.begin() + layout.memory_offset_dword,
prepared.shader_data.end(), 0);
@@ -876,20 +814,18 @@ void RenderExecutor::RebindBuffers(PreparedBindings& prepared) {
const auto shift = (index % 4u) * 8u;
prepared.shader_data[dword] |= offset << shift;
};
for (uint32_t i = 0; i < layout.memory_offset_count; i++) {
const auto resource = layout.descriptors.front().resources[i];
for (const auto& resource: program.info.buffers) {
const auto index = resource.descriptor_index;
if (index == UINT32_MAX) continue;
uint32_t buffer_offset = 0;
prepared.buffers.push_back(NativeStorageBuffer(m_context, prepared.buffer_sources[i],
program.info.buffers[resource],
buffer_offset));
pack_memory_offset(i, buffer_offset);
prepared.buffers[index] = NativeStorageBuffer(m_context, prepared.buffer_sources[index],
resource, buffer_offset);
pack_memory_offset(index, buffer_offset);
}
if (ShaderRecompiler::IR::FindBinding(
layout, ShaderRecompiler::IR::DescriptorBindingKind::FlattenedSrt) != nullptr) {
if (layout.descriptor_counts[static_cast<size_t>(BindingKind::FlattenedSrt)] != 0) {
prepared.flattened_srt = NativeUpload(m_context, snapshot.flattened_srt);
}
if (ShaderRecompiler::IR::FindBinding(
program.bindings, ShaderRecompiler::IR::DescriptorBindingKind::ShaderData) != nullptr) {
if (layout.descriptor_counts[static_cast<size_t>(BindingKind::ShaderData)] != 0) {
prepared.shader_data_buffer = NativeUpload(m_context, prepared.shader_data);
}
}
@@ -981,7 +917,7 @@ void RenderExecutor::CommitBindings(CommandBuffer& buffer,
std::span<PreparedBindings* const> prepared_bindings) {
KYTY_PROFILER_FUNCTION();
auto vk_buffer = buffer.Handle();
size_t descriptor_count = 0;
size_t image_descriptor_count = 0;
size_t write_count = 0;
ShaderRecompiler::IR::PushData push_data;
bool has_push_data = false;
@@ -995,9 +931,14 @@ void RenderExecutor::CommitBindings(CommandBuffer& buffer,
for (const auto* prepared: prepared_bindings) {
EXIT_IF(prepared == nullptr || prepared->runtime == nullptr || !*prepared->runtime);
const auto& program = *prepared->runtime->program;
write_count += program.bindings.descriptors.size();
for (const auto& binding: program.bindings.descriptors) {
descriptor_count += NativeDescriptorCount(binding);
write_count += std::popcount(program.bindings.descriptor_mask);
for (auto mask = program.bindings.descriptor_mask; mask != 0; mask &= mask - 1) {
const auto index = std::countr_zero(mask);
const auto count = program.bindings.descriptor_counts[index];
if (index >= ShaderRecompiler::IR::FirstImageBinding &&
index <= static_cast<size_t>(BindingKind::Samplers)) {
image_descriptor_count += count;
}
}
const auto shader_stage = NativeShaderStage(program.stage);
push_stages |= shader_stage;
@@ -1029,10 +970,9 @@ void RenderExecutor::CommitBindings(CommandBuffer& buffer,
}
for (const auto* writer: prepared_bindings) {
const auto& program = *writer->runtime->program;
for (uint32_t i = 0; i < writer->buffer_sources.size(); ++i) {
const auto resource = program.bindings.descriptors.front().resources[i];
if (!program.info.buffers[resource].written) continue;
const auto& written = writer->buffer_sources[i];
for (const auto& resource: program.info.buffers) {
if (!resource.written || resource.descriptor_index == UINT32_MAX) continue;
const auto& written = writer->buffer_sources[resource.descriptor_index];
if (written.size != 0 && ImageRangeOverlaps(address, size,
written.address, written.size)) {
EXIT("scalar resource reads overlap a shader buffer write\n");
@@ -1041,12 +981,11 @@ void RenderExecutor::CommitBindings(CommandBuffer& buffer,
}
}
}
m_descriptor_buffers.clear();
m_descriptor_images.clear();
m_descriptor_images.resize(image_descriptor_count);
m_descriptor_writes.clear();
m_descriptor_buffers.reserve(descriptor_count);
m_descriptor_images.reserve(descriptor_count);
m_descriptor_writes.reserve(write_count);
size_t image_cursor = 0;
std::array<vk::DescriptorBufferInfo, 2> address_buffers;
for (auto* prepared: prepared_bindings) {
const auto& program = *prepared->runtime->program;
@@ -1111,37 +1050,37 @@ void RenderExecutor::CommitBindings(CommandBuffer& buffer,
binding.layout = image.backing.state.layout;
}
m_image_occurrences.assign(descriptors.images.size(), 0);
for (const auto& binding: program.bindings.descriptors) {
std::array<size_t, ShaderRecompiler::IR::ImageBindingCount> image_starts {};
for (auto mask = program.bindings.descriptor_mask; mask != 0; mask &= mask - 1) {
const auto index = std::countr_zero(mask);
const auto kind = static_cast<BindingKind>(index);
vk::WriteDescriptorSet write {};
write.dstBinding = ShaderRecompiler::IR::NativeBinding(program.stage, binding.kind);
write.descriptorType = NativeDescriptorType(binding.kind);
write.descriptorCount = NativeDescriptorCount(binding);
const auto buffer_start = m_descriptor_buffers.size();
const auto image_start = m_descriptor_images.size();
if (ShaderRecompiler::IR::ImageBindingResourceClass(binding.kind) !=
write.dstBinding = ShaderRecompiler::IR::NativeBinding(program.stage, kind);
write.descriptorType = NativeDescriptorType(kind);
write.descriptorCount = program.bindings.descriptor_counts[index];
const auto image_start = image_cursor;
if (ShaderRecompiler::IR::ImageBindingResourceClass(kind) !=
ShaderRecompiler::IR::ImageResourceClass::None) {
for (const auto resource: binding.resources) {
m_descriptor_images.push_back(MakeImageInfo(
descriptors.images.at(resource), m_image_occurrences.at(resource)++));
}
image_starts[ShaderRecompiler::IR::ImageBindingIndex(kind)] = image_start;
image_cursor += write.descriptorCount;
} else {
switch (binding.kind) {
switch (kind) {
case BindingKind::Buffers:
EXIT_IF(descriptors.buffers.size() != binding.resources.size());
EXIT_IF(descriptors.buffers.size() != write.descriptorCount);
for (const auto& view: descriptors.buffers) {
EXIT_IF(view.buffer == nullptr);
m_descriptor_buffers.push_back(view);
}
write.pBufferInfo = descriptors.buffers.data();
break;
case BindingKind::BdaPagetable:
case BindingKind::FaultBuffer: {
auto& cache = m_context.GetBufferCache();
const auto* bda_buffer = binding.kind == BindingKind::BdaPagetable
auto& cache = m_context.GetBufferCache();
const auto* bda_buffer = kind == BindingKind::BdaPagetable
? cache.GetBdaPageTableBuffer()
: cache.GetFaultBuffer();
m_descriptor_buffers.emplace_back(bda_buffer->Handle(), 0,
bda_buffer->Size());
auto& view = address_buffers[kind == BindingKind::FaultBuffer];
view = {bda_buffer->Handle(), 0, bda_buffer->Size()};
write.pBufferInfo = &view;
break;
}
case BindingKind::FlattenedSrt:
@@ -1149,42 +1088,41 @@ void RenderExecutor::CommitBindings(CommandBuffer& buffer,
case BindingKind::SharedMemory:
case BindingKind::Gds: {
const vk::DescriptorBufferInfo* view = &descriptors.gds;
if (binding.kind == BindingKind::FlattenedSrt) {
if (kind == BindingKind::FlattenedSrt) {
view = &descriptors.flattened_srt;
} else if (binding.kind == BindingKind::ShaderData) {
} else if (kind == BindingKind::ShaderData) {
view = &descriptors.shader_data_buffer;
} else if (binding.kind == BindingKind::SharedMemory) {
} else if (kind == BindingKind::SharedMemory) {
view = &descriptors.shared_memory;
}
EXIT_IF(view->buffer == nullptr);
m_descriptor_buffers.push_back(*view);
write.pBufferInfo = view;
break;
}
case BindingKind::Samplers:
for (const auto resource: binding.resources) {
const auto sampler = descriptors.samplers.at(resource);
for (const auto sampler: descriptors.samplers) {
EXIT_IF(sampler == nullptr);
m_descriptor_images.emplace_back(sampler, nullptr,
vk::ImageLayout::eUndefined);
m_descriptor_images[image_cursor++] = {sampler, nullptr, vk::ImageLayout::eUndefined};
}
break;
case BindingKind::Count: EXIT("invalid descriptor binding kind");
}
}
if (m_descriptor_buffers.size() != buffer_start) {
write.pBufferInfo = m_descriptor_buffers.data() + buffer_start;
}
if (m_descriptor_images.size() != image_start) {
if (image_cursor != image_start) {
write.pImageInfo = m_descriptor_images.data() + image_start;
}
m_descriptor_writes.push_back(write);
}
for (uint32_t i = 0; i < descriptors.images.size(); i++) {
const auto expected =
descriptors.images[i].mip_views.empty()
? 1u
: static_cast<uint32_t>(descriptors.images[i].mip_views.size());
EXIT_IF(m_image_occurrences[i] != expected);
for (uint32_t i = 0; i < program.info.images.size(); ++i) {
const auto& resource = program.info.images[i];
const auto& texture = descriptors.images[i];
EXIT_IF(resource.mip_count != (texture.mip_views.empty() ? 1u : texture.mip_views.size()));
const auto kind = *ShaderRecompiler::IR::DescriptorBindingForImage(resource);
const auto start = image_starts[ShaderRecompiler::IR::ImageBindingIndex(kind)] +
resource.descriptor_index;
for (uint32_t mip = 0; mip < resource.mip_count; ++mip) {
m_descriptor_images[start + mip] = MakeImageInfo(texture, mip);
}
}
const auto shader_data_dwords = program.bindings.ShaderDataDwords();
@@ -50,8 +50,6 @@ struct PreparedBindings {
[[nodiscard]] vk::DescriptorType
NativeDescriptorType(ShaderRecompiler::IR::DescriptorBindingKind kind);
[[nodiscard]] uint32_t
NativeDescriptorCount(const ShaderRecompiler::IR::DescriptorBinding& binding);
[[nodiscard]] vk::DescriptorImageInfo MakeImageInfo(const TextureBinding& texture,
uint32_t element = 0);
@@ -217,17 +217,53 @@ struct PipelineCache::ProgramCache {
};
struct Permutation {
std::vector<uint32_t> function_code;
ShaderRecompiler::IR::ResourceSpecialization specialization;
ShaderRecompiler::IR::CompiledShaderInfo program;
ShaderProgram handle;
};
struct SourceEntry {
explicit SourceEntry(ShaderRecompiler::IR::ResourcePlan plan)
: resource_plan(std::move(plan)) {
SourceEntry(ShaderRecompiler::IR::ResourcePlan plan,
std::unique_ptr<ShaderRecompiler::ShaderSource> call_source)
: call_source(std::move(call_source)), resource_plan(std::move(plan)) {
permutations.reserve(8);
}
std::span<const uint32_t> FunctionCode() const {
return call_source
? std::span<const uint32_t>(call_source->linked->code).subspan(call_source->code.size())
: std::span<const uint32_t>{};
}
bool RefreshCallSource(const ShaderRecompiler::IR::SrtRuntime& runtime) {
const auto revision = call_source->revision;
ShaderRecompiler::RefreshShaderSource(*call_source, runtime);
resource_plan.source_reads = call_source->reads;
return revision != call_source->revision;
}
template <bool HasCalls>
Permutation* FindPermutation(const ShaderRecompiler::IR::SrtRuntime& runtime,
uint32_t push_data_cursor) {
EXIT_IF(!ShaderRecompiler::IR::MaterializeResources(
resource_plan, runtime, resources, specialization));
for (auto& candidate: permutations) {
const auto& layout = candidate.program.bindings;
if (layout.push_data_start_dword != ShaderRecompiler::IR::PushData::StartFor(
push_data_cursor, layout.ShaderDataDwords()) ||
candidate.specialization != specialization) {
continue;
}
if constexpr (HasCalls) {
if (!std::ranges::equal(candidate.function_code, FunctionCode())) continue;
}
return &candidate;
}
return nullptr;
}
std::unique_ptr<ShaderRecompiler::ShaderSource> call_source;
ShaderRecompiler::IR::ResourcePlan resource_plan;
ShaderRecompiler::IR::ResourceSnapshot resources;
ShaderRecompiler::IR::ResourceSpecialization specialization;
@@ -256,7 +292,8 @@ struct PipelineCache::ProgramCache {
const ShaderRecompiler::CompileOptions& options,
ShaderRecompiler::TranslateResult translated,
ShaderRecompiler::IR::ResourceSpecialization specialization,
uint32_t push_data_start_dword) {
uint32_t push_data_start_dword,
std::span<const uint32_t> function_code) {
auto result = ShaderRecompiler::CompileProgram(std::move(translated), options,
specialization, push_data_start_dword);
if (!ValidateShaderSpirv(options.dump_label, options.shader_hash, result.spirv)) {
@@ -273,6 +310,7 @@ struct PipelineCache::ProgramCache {
static_cast<uint64_t>(result.spirv.size()), options.wave_size);
}
return {
.function_code = {function_code.begin(), function_code.end()},
.specialization = std::move(specialization),
.program = std::move(result.program).TakeCompiledInfo(),
.handle = {.id = ++next_shader_id, .module = module},
@@ -307,21 +345,18 @@ struct PipelineCache::ProgramCache {
if constexpr (std::is_same_v<InputInfo, ShaderComputeInputInfo>) {
runtime.workgroup_counts = input_info.workgroup_counts;
}
bool source_changed = false;
if (entry != programs.end()) {
EXIT_IF(!ShaderRecompiler::IR::MaterializeResources(
entry->second.resource_plan, runtime, entry->second.resources,
entry->second.specialization));
if (const auto permutation = std::ranges::find_if(
entry->second.permutations, [&](const Permutation& candidate) {
const auto& layout = candidate.program.bindings;
return layout.push_data_start_dword ==
ShaderRecompiler::IR::PushData::StartFor(
push_data_cursor, layout.ShaderDataDwords()) &&
candidate.specialization == entry->second.specialization;
});
permutation != entry->second.permutations.end()) {
input_info.stage = {.program = &permutation->program,
.resources = &entry->second.resources};
auto& cached = entry->second;
Permutation* permutation = nullptr;
if (cached.call_source) {
source_changed = cached.RefreshCallSource(runtime);
if (!source_changed) permutation = cached.FindPermutation<true>(runtime, push_data_cursor);
} else {
permutation = cached.FindPermutation<false>(runtime, push_data_cursor);
}
if (permutation != nullptr) {
input_info.stage = {.program = &permutation->program, .resources = &entry->second.resources};
permutation->program.bindings.AdvancePushData(push_data_cursor);
return permutation->handle;
}
@@ -368,16 +403,39 @@ struct PipelineCache::ProgramCache {
options.wave_size = input_info.wave_size;
}
DumpShaderOriginal(stage_name, options.shader_hash, params.code);
auto translated = ShaderRecompiler::TranslateProgram(params.code, options);
ShaderRecompiler::ShaderSource source;
if (entry == programs.end()) source = ShaderRecompiler::PrepareShaderSource(params.code, options);
auto* current = entry == programs.end() ? &source : entry->second.call_source.get();
ShaderRecompiler::TranslateResult translated;
if (current != nullptr) {
if (current->call) {
if (entry == programs.end()) ShaderRecompiler::RefreshShaderSource(*current, runtime);
translated = ShaderRecompiler::TranslateProgram(current->linked->decoded, options);
translated.program.source_reads = current->reads;
} else {
translated = ShaderRecompiler::TranslateProgram(current->decoded, options);
}
} else {
translated = ShaderRecompiler::TranslateProgram(params.code, options);
}
if (entry == programs.end()) {
auto retained = source.call ? std::make_unique<ShaderRecompiler::ShaderSource>(std::move(source)) : nullptr;
entry = programs.try_emplace(lookup_key,
ShaderRecompiler::IR::ExtractResourcePlan(translated.program)).first;
ShaderRecompiler::IR::ExtractResourcePlan(translated.program), std::move(retained)).first;
EXIT_IF(!ShaderRecompiler::IR::MaterializeResources(
entry->second.resource_plan, runtime, entry->second.resources,
entry->second.specialization));
} else if (source_changed) {
entry->second.resource_plan = ShaderRecompiler::IR::ExtractResourcePlan(translated.program);
if (const auto* permutation = entry->second.FindPermutation<true>(runtime, push_data_cursor)) {
input_info.stage = {.program = &permutation->program, .resources = &entry->second.resources};
permutation->program.bindings.AdvancePushData(push_data_cursor);
return permutation->handle;
}
}
entry->second.permutations.push_back(CompilePermutation(
stage_name, options, std::move(translated), entry->second.specialization, push_data_cursor));
stage_name, options, std::move(translated), entry->second.specialization, push_data_cursor,
entry->second.FunctionCode()));
const auto& permutation = entry->second.permutations.back();
input_info.stage = {.program = &permutation.program, .resources = &entry->second.resources};
permutation.program.bindings.AdvancePushData(push_data_cursor);
@@ -617,6 +675,13 @@ PipelineCache::GraphicsPrograms PipelineCache::GetGraphicsPrograms(
ShaderParams pixel_params;
if (pixel_active) {
pixel_params = PrepareProgram(pixel_regs, sh, target_export_mapping, pixel_info);
if (!mesh_active && !tess_active && Prospero::IsRectList(user_config.GetPrimType())) {
pixel_info.parameter_mode = ShaderPixelParameterMode::Rectangle;
} else if (context.GetModeControl().provoking_vtx_last) {
pixel_info.parameter_mode = ShaderPixelParameterMode::LastVertex;
} else {
pixel_info.parameter_mode = ShaderPixelParameterMode::FirstVertex;
}
const auto& blend = context.GetBlendControl(0);
pixel_info.dual_source_blending =
blend.enable && !context.GetRenderTarget(0).info.blend_bypass &&
@@ -19,6 +19,7 @@
#include "graphics/shader/shader.h"
#include <algorithm>
#include <bit>
#include <limits>
#include <span>
#include <vector>
@@ -180,10 +181,12 @@ static vk::BlendOp GetBlendOp(uint32_t op) {
static void AddLayoutBindings(std::vector<vk::DescriptorSetLayoutBinding>& descriptor_bindings,
const ShaderRecompiler::IR::CompiledShaderInfo& program,
vk::ShaderStageFlagBits stage) {
for (const auto& binding: program.bindings.descriptors) {
for (auto mask = program.bindings.descriptor_mask; mask != 0; mask &= mask - 1) {
const auto index = std::countr_zero(mask);
const auto kind = static_cast<ShaderRecompiler::IR::DescriptorBindingKind>(index);
descriptor_bindings.push_back(
{ShaderRecompiler::IR::NativeBinding(program.stage, binding.kind),
NativeDescriptorType(binding.kind), NativeDescriptorCount(binding), stage, nullptr});
{ShaderRecompiler::IR::NativeBinding(program.stage, kind), NativeDescriptorType(kind),
program.bindings.descriptor_counts[index], stage, nullptr});
}
}
+2 -4
View File
@@ -156,9 +156,9 @@ public:
KYTY_CLASS_NO_COPY(RenderExecutor);
void DispatchDirect(uint64_t submit_id, CommandBuffer& buffer, uint32_t thread_group_x,
uint32_t thread_group_y, uint32_t thread_group_z, uint32_t mode);
uint32_t thread_group_y, uint32_t thread_group_z, uint32_t mode, bool async_compute);
void DispatchIndirect(uint64_t submit_id, CommandBuffer& buffer, uint64_t args_addr,
uint32_t mode);
uint32_t mode, bool async_compute);
void PrepareBindings(const ShaderStageRuntime& runtime, PreparedBindings& prepared);
void FindBuffers(std::span<PreparedBindings* const> stages);
@@ -213,10 +213,8 @@ private:
GraphicsBindings m_graphics_bindings;
PreparedBindings m_compute_bindings;
std::vector<ImageId> m_bound_images;
std::vector<vk::DescriptorBufferInfo> m_descriptor_buffers;
std::vector<vk::DescriptorImageInfo> m_descriptor_images;
std::vector<vk::WriteDescriptorSet> m_descriptor_writes;
std::vector<uint32_t> m_image_occurrences;
friend class CommandProcessor;
friend struct RenderExecutorTestAccess;
@@ -20,7 +20,6 @@
#include "graphics/host_gpu/vulkanCommon.h"
#include "graphics/shader/recompiler/ir/ShaderIR.h"
#include "graphics/shader/recompiler/ir/passes/ResourceMaterialization.h"
#include "graphics/shader/recompiler/ir/passes/BindingLayout.h"
#include "graphics/shader/shader.h"
#include "kernel/eventQueue.h"
#include "kernel/pthread.h"
@@ -198,8 +197,8 @@ bool RenderExecutor::TryConsumeComputeImageClear(const ShaderComputeInputInfo& i
static void BindSharedMemory(RenderContext& context, ShaderComputeInputInfo& input,
PreparedBindings& bindings, uint64_t indirect_args = 0) {
if (ShaderRecompiler::IR::FindBinding(input.stage.program->bindings,
ShaderRecompiler::IR::DescriptorBindingKind::SharedMemory) == nullptr) {
if (input.stage.program->bindings.descriptor_counts[
static_cast<size_t>(ShaderRecompiler::IR::DescriptorBindingKind::SharedMemory)] == 0) {
return;
}
auto& cache = context.GetBufferCache();
@@ -227,7 +226,7 @@ static void BindSharedMemory(RenderContext& context, ShaderComputeInputInfo& inp
void RenderExecutor::DispatchDirect(uint64_t submit_id, CommandBuffer& buffer,
uint32_t thread_group_x, uint32_t thread_group_y,
uint32_t thread_group_z, uint32_t mode) {
uint32_t thread_group_z, uint32_t mode, bool async_compute) {
EXIT_IF(buffer.IsInvalid());
m_context.GetCommandScheduler().PopPendingOperations();
auto& ctx = buffer.GetRegisters();
@@ -282,6 +281,7 @@ void RenderExecutor::DispatchDirect(uint64_t submit_id, CommandBuffer& buffer,
const auto& sh_regs = ctx.GetShaderRegisters();
ShaderComputeInputInfo input_info {};
input_info.async_compute = async_compute;
const bool use_thread_dimensions = (mode & DISPATCH_INITIATOR_USE_THREAD_DIMENSIONS) != 0;
input_info.dispatch_thread_dimensions = use_thread_dimensions;
input_info.workgroup_counts[0] = thread_group_x;
@@ -438,7 +438,7 @@ void RenderExecutor::DispatchDirect(uint64_t submit_id, CommandBuffer& buffer,
}
void RenderExecutor::DispatchIndirect(uint64_t submit_id, CommandBuffer& buffer,
uint64_t args_addr, uint32_t mode) {
uint64_t args_addr, uint32_t mode, bool async_compute) {
EXIT_IF(buffer.IsInvalid() || args_addr == 0 || (args_addr & 3u) != 0 ||
(mode & Pm4::COMPUTE_DISPATCH_INITIATOR_USE_THREAD_DIMENSIONS) != 0);
m_context.GetCommandScheduler().PopPendingOperations();
@@ -451,6 +451,7 @@ void RenderExecutor::DispatchIndirect(uint64_t submit_id, CommandBuffer& buffer,
return;
}
ShaderComputeInputInfo input_info {};
input_info.async_compute = async_compute;
const auto compute_program = m_context.GetPipelineCache().GetComputeProgram(
cs_regs, buffer.GetRegisters().GetShaderRegisters(), input_info);
buffer.EndRendering();
+28 -30
View File
@@ -312,9 +312,9 @@ static void LogDrawInputState(const CommandBuffer& buffer, const RenderColorInfo
}
}
static void SetGraphicsDynamicParams(const CommandBuffer& buffer, vk::CommandBuffer vk_buffer,
const ShaderVertexInputInfo& vs_input_info,
const RenderDepthInfo& depth, const RenderState& rendering) {
static void UpdateGraphicsDynamicState(const CommandBuffer& buffer, DynamicState& dynamic_state,
const ShaderVertexInputInfo& vs_input_info,
const RenderDepthInfo& depth, const RenderState& rendering) {
KYTY_PROFILER_FUNCTION();
const auto& ctx = buffer.GetRegisters();
@@ -358,8 +358,8 @@ static void SetGraphicsDynamicParams(const CommandBuffer& buffer, vk::CommandBuf
scissor.extent = {0, 0};
}
}
vk_buffer.setViewportWithCount(viewport_count, viewports.data());
vk_buffer.setScissorWithCount(viewport_count, scissors.data());
dynamic_state.SetViewports(std::span {viewports}.first(viewport_count));
dynamic_state.SetScissors(std::span {scissors}.first(viewport_count));
float line_width = ctx.GetLineWidth();
if (line_width != 1.0f) {
@@ -372,17 +372,17 @@ static void SetGraphicsDynamicParams(const CommandBuffer& buffer, vk::CommandBuf
}
line_width = 1.0f;
}
vk_buffer.setLineWidth(line_width);
dynamic_state.SetLineWidth(line_width);
const auto& blend = ctx.GetBlendColor();
const std::array blend_constants {blend.red, blend.green, blend.blue, blend.alpha};
vk_buffer.setBlendConstants(blend_constants.data());
vk_buffer.setDepthTestEnable(depth.depth_test_enable ? VK_TRUE : VK_FALSE);
vk_buffer.setDepthWriteEnable(depth.depth_write_enable ? VK_TRUE : VK_FALSE);
vk_buffer.setDepthCompareOp(depth.depth_compare_op);
dynamic_state.SetBlendConstants(blend_constants);
dynamic_state.SetDepthTestEnabled(depth.depth_test_enable);
dynamic_state.SetDepthWriteEnabled(depth.depth_write_enable);
dynamic_state.SetDepthCompareOp(depth.depth_compare_op);
#if !defined(__APPLE__)
vk_buffer.setDepthBoundsTestEnable(depth.depth_bounds_test_enable ? VK_TRUE : VK_FALSE);
dynamic_state.SetDepthBoundsTestEnabled(depth.depth_bounds_test_enable);
if (depth.depth_bounds_test_enable) {
vk_buffer.setDepthBounds(ctx.GetDepthBoundsMin(), ctx.GetDepthBoundsMax());
dynamic_state.SetDepthBounds(ctx.GetDepthBoundsMin(), ctx.GetDepthBoundsMax());
}
#endif
@@ -391,7 +391,7 @@ static void SetGraphicsDynamicParams(const CommandBuffer& buffer, vk::CommandBuf
const bool use_front = mode.poly_offset_front_enable && !mode.cull_front;
const bool use_back = mode.poly_offset_back_enable && !mode.cull_back;
const bool depth_bias_enable = use_front || use_back;
vk_buffer.setDepthBiasEnable(depth_bias_enable ? VK_TRUE : VK_FALSE);
dynamic_state.SetDepthBiasEnabled(depth_bias_enable);
if (depth_bias_enable) {
// Vulkan has one bias for both faces. Prefer a visible front face when both are enabled.
const float guest_constant_factor =
@@ -400,19 +400,18 @@ static void SetGraphicsDynamicParams(const CommandBuffer& buffer, vk::CommandBuf
guest_constant_factor, poly_offset, depth.desc.view_info.format);
const float slope_factor =
(use_front ? poly_offset.front_scale : poly_offset.back_scale) / 16.0f;
vk_buffer.setDepthBias(constant_factor, poly_offset.clamp, slope_factor);
dynamic_state.SetDepthBias(constant_factor, poly_offset.clamp, slope_factor);
}
vk_buffer.setStencilTestEnable(depth.stencil_test_enable ? VK_TRUE : VK_FALSE);
dynamic_state.SetStencilTestEnabled(depth.stencil_test_enable);
if (depth.stencil_test_enable) {
const auto set_stencil = [&](vk::StencilFaceFlagBits face, const vk::StencilOpState& state) {
vk_buffer.setStencilOp(face, state.failOp, state.passOp, state.depthFailOp, state.compareOp);
vk_buffer.setStencilCompareMask(face, state.compareMask);
vk_buffer.setStencilWriteMask(face, state.writeMask);
vk_buffer.setStencilReference(face, state.reference);
};
set_stencil(vk::StencilFaceFlagBits::eFront, depth.stencil_front);
set_stencil(vk::StencilFaceFlagBits::eBack, depth.stencil_back);
const auto& front = depth.stencil_front;
const auto& back = depth.stencil_back;
dynamic_state.SetStencilOps({front.failOp, front.passOp, front.depthFailOp, front.compareOp},
{back.failOp, back.passOp, back.depthFailOp, back.compareOp});
dynamic_state.SetStencilCompareMasks(front.compareMask, back.compareMask);
dynamic_state.SetStencilWriteMasks(front.writeMask, back.writeMask);
dynamic_state.SetStencilReferences(front.reference, back.reference);
}
#if defined(__APPLE__)
@@ -423,9 +422,7 @@ static void SetGraphicsDynamicParams(const CommandBuffer& buffer, vk::CommandBuf
for (uint32_t slot = 0; slot < rendering.num_color_attachments; slot++) {
enable[slot] = rendering.color_attachments[slot].image_view != nullptr;
}
if (rendering.num_color_attachments != 0) {
vk_buffer.setColorWriteEnableEXT(rendering.num_color_attachments, enable);
}
dynamic_state.SetColorWriteEnables(std::span {enable}.first(rendering.num_color_attachments));
#endif
}
@@ -1139,10 +1136,11 @@ void RenderExecutor::ExecutePreparedDraw(uint64_t submit_id, CommandBuffer& buff
CommitIndexBuffer(vk_buffer, index_binding);
}
SetGraphicsDynamicParams(buffer, vk_buffer, vertex_stages.back(), state.depth_info, rendering);
if (m_context.GetGraphics().attachment_feedback_loop_enabled) {
vk_buffer.setAttachmentFeedbackLoopEnableEXT(feedback_aspects);
}
auto& dynamic_state = m_context.GetCommandScheduler().GetDynamicState();
UpdateGraphicsDynamicState(buffer, dynamic_state, vertex_stages.back(), state.depth_info,
rendering);
dynamic_state.SetAttachmentFeedbackLoopAspects(feedback_aspects);
dynamic_state.Commit(m_context.GetGraphics(), vk_buffer);
LogDrawPhase(draw.Name(), "BeginRendering");
if (!draw.IsIndexed()) {
@@ -6,20 +6,13 @@ layout(local_size_x = 64) in;
layout(set = 0, binding = 0, std430) readonly buffer Input { uint data[]; } input_buffer;
layout(set = 0, binding = 1, std430) buffer Output { uint data[]; } output_buffer;
layout(push_constant, std430) uniform Push {
layout(push_constant, std430) uniform Conversion {
uint src_base;
uint dst_base;
uint width;
uint height;
uint depth;
uint surface_z;
uint pitch_bytes;
uint slice_bytes;
uint blocks_per_row;
uint blocks_per_slice;
uint tail_x;
uint tail_y;
uint tail;
uint source_pitch;
uint target_pitch;
uint color_transform;
uint to_host;
} params;
@@ -1,66 +1,84 @@
layout(local_size_x = 8, local_size_y = 8) in;
#extension GL_EXT_shader_8bit_storage : require
#extension GL_EXT_shader_16bit_storage : require
layout(local_size_x = 64) in;
layout(set = 0, binding = 0, std430) readonly buffer Input { uint data[]; } input_buffer;
layout(set = 0, binding = 1, std430) buffer Output { uint data[]; } output_buffer;
layout(set = 0, binding = 0, std430) readonly buffer Input8 { uint8_t data[]; } input_buffer8;
layout(set = 0, binding = 1, std430) buffer Output8 { uint8_t data[]; } output_buffer8;
layout(set = 0, binding = 0, std430) readonly buffer Input16 { uint16_t data[]; } input_buffer16;
layout(set = 0, binding = 1, std430) buffer Output16 { uint16_t data[]; } output_buffer16;
layout(set = 0, binding = 2, std140) uniform Push {
struct Mip {
uint src_base;
uint dst_base;
uint width;
uint height;
uint depth;
uint surface_z;
uint pitch_bytes;
uint slice_bytes;
uint tiled_slice_bytes;
uint blocks_per_row;
uint blocks_per_slice;
uint surface_z;
uint tail_x;
uint tail_y;
uint tail;
uint num_texels;
};
layout(set = 0, binding = 2, std140) uniform TilingInfo {
uint num_texels;
uint color_transform;
} params;
Mip mips[16];
} info;
#include "gpu_tiler_color.inc"
void copy_element(uint src, uint dst) {
if ((ELEMENT_BYTES == 4u || ELEMENT_BYTES == 8u) && params.color_transform != 0u) {
if ((ELEMENT_BYTES == 4u || ELEMENT_BYTES == 8u) && info.color_transform != 0u) {
uvec2 value = uvec2(input_buffer.data[src >> 2], 0u);
if (ELEMENT_BYTES == 8u) value.y = input_buffer.data[(src >> 2) + 1u];
value = transform_color(value, params.color_transform, TILE == 0u);
value = transform_color(value, info.color_transform, TILE == 0u);
output_buffer.data[dst >> 2] = value.x;
if (ELEMENT_BYTES == 8u) output_buffer.data[(dst >> 2) + 1u] = value.y;
} else if (ELEMENT_BYTES >= 4) {
for (uint i = 0; i < ELEMENT_BYTES; i += 4) {
output_buffer.data[(dst + i) >> 2] = input_buffer.data[(src + i) >> 2];
}
} else if (ELEMENT_BYTES == 2) {
output_buffer16.data[dst >> 1] = input_buffer16.data[src >> 1];
} else {
uint mask = ELEMENT_BYTES == 1 ? 0xffu : 0xffffu;
uint value = (input_buffer.data[src >> 2] >> ((src & 3u) * 8u)) & mask;
uint shift = (dst & 3u) * 8u;
atomicAnd(output_buffer.data[dst >> 2], ~(mask << shift));
atomicOr(output_buffer.data[dst >> 2], value << shift);
output_buffer8.data[dst] = input_buffer8.data[src];
}
}
void main() {
uvec3 p = gl_GlobalInvocationID;
if (p.x >= params.width || p.y >= params.height || p.z >= params.depth) {
uint group = gl_WorkGroupID.y * gl_NumWorkGroups.x + gl_WorkGroupID.x;
// Reject padding groups before multiplying: the active range may approach 4 GiB.
if (group > (info.num_texels - 1u) / 64u) {
return;
}
uint texel = group * 64u + gl_LocalInvocationID.x;
if (texel >= info.num_texels) {
return;
}
uint mip = 0u;
uint mip_texels = info.mips[mip].num_texels;
while (texel >= mip_texels) {
texel -= mip_texels;
mip_texels = info.mips[++mip].num_texels;
}
Mip params = info.mips[mip];
uint row = texel / params.width;
uint z = row / params.height;
uvec3 p = uvec3(texel - row * params.width, row - z * params.height, z);
uvec3 extent = block_extent();
uvec3 swizzle = p;
uvec3 block = uvec3(0);
if (params.tail != 0) {
swizzle.xy += uvec2(params.tail_x, params.tail_y);
} else {
block = p / extent;
}
uvec3 block = p / extent;
uvec3 swizzle = p + uvec3(params.tail_x, params.tail_y, params.surface_z);
uint block_index = block.z * params.blocks_per_slice +
block.y * params.blocks_per_row + block.x;
uint tiled = block_index * BLOCK_BYTES +
block_offset(uvec3(swizzle.xy, swizzle.z + params.surface_z));
uint block_index = block.y * params.blocks_per_row + block.x;
uint tiled = block.z * params.tiled_slice_bytes + block_index * BLOCK_BYTES +
block_offset(swizzle);
uint linear = p.z * params.slice_bytes + p.y * params.pitch_bytes +
p.x * ELEMENT_BYTES;
if (TILE == 0) {
@@ -6,20 +6,15 @@ layout(local_size_x = 64) in;
layout(set = 0, binding = 0, std430) readonly buffer Input { uint data[]; } input_buffer;
layout(set = 0, binding = 1, std430) buffer Output { uint data[]; } output_buffer;
layout(push_constant, std430) uniform Push {
layout(push_constant, std430) uniform Conversion {
uint src_base;
uint dst_base;
uint width;
uint height;
uint depth;
uint surface_z;
uint pitch_bytes;
uint slice_bytes;
uint blocks_per_row;
uint blocks_per_slice;
uint tail_x;
uint tail_y;
uint tail;
uint source_pitch;
uint target_pitch;
uint color_transform;
uint to_host;
} params;
uint decode(uint value) {
@@ -35,8 +30,8 @@ void main() {
if (x >= params.width || y >= params.height) {
return;
}
uint src = params.src_base + y * params.pitch_bytes + x * 4u;
uint dst = params.dst_base + y * params.slice_bytes + x * 2u;
uint src = params.src_base + y * params.source_pitch + x * 4u;
uint dst = params.dst_base + y * params.target_pitch + x * 2u;
uint shift = (dst & 2u) * 8u;
uint mask = 0xffffu << shift;
uint word = dst >> 2u;
@@ -6,20 +6,15 @@ layout(local_size_x = 64) in;
layout(set = 0, binding = 0, std430) readonly buffer Input { uint data[]; } input_buffer;
layout(set = 0, binding = 1, std430) buffer Output { uint data[]; } output_buffer;
layout(push_constant, std430) uniform Push {
layout(push_constant, std430) uniform Conversion {
uint src_base;
uint dst_base;
uint width;
uint height;
uint depth;
uint surface_z;
uint pitch_bytes;
uint slice_bytes;
uint blocks_per_row;
uint blocks_per_slice;
uint tail_x;
uint tail_y;
uint tail;
uint source_pitch;
uint target_pitch;
uint color_transform;
uint to_host;
} params;
void main() {
@@ -28,12 +23,12 @@ void main() {
if (x >= params.width || y >= params.height) {
return;
}
uint src = params.src_base + y * params.pitch_bytes + x * 2u;
uint src = params.src_base + y * params.source_pitch + x * 2u;
uint packed = input_buffer.data[src >> 2u];
uint value = (packed >> ((src & 2u) * 8u)) & 0xffffu;
uint encoded = D32 != 0u
? floatBitsToUint(float(value) / 65535.0)
: value * 256u + (value + 128u) / 257u;
uint dst = params.dst_base + y * params.slice_bytes + x * 4u;
uint dst = params.dst_base + y * params.target_pitch + x * 4u;
output_buffer.data[dst >> 2u] = encoded;
}
+137
View File
@@ -0,0 +1,137 @@
#include "graphics/presentation/ddsImage.h"
#include "common/assert.h"
#include "common/file.h"
#include "common/logging/log.h"
#include "imgui_impl_vulkan.h"
#include <array>
namespace Libs::Graphics {
DdsImage::~DdsImage() {
if (m_descriptor != VK_NULL_HANDLE) {
ImGui_ImplVulkan_RemoveTexture(m_descriptor);
}
m_graphics.device.destroyImageView(m_view, nullptr);
if (m_image.image != nullptr) {
m_graphics.DeleteImage(m_image);
}
if (m_staging != VK_NULL_HANDLE) {
vmaDestroyBuffer(m_graphics.allocator, m_staging, m_staging_allocation);
}
}
bool DdsImage::Load(const std::filesystem::path& path) {
EXIT_IF(m_image.image != nullptr || m_staging != VK_NULL_HANDLE);
Common::File file(path, Common::File::Mode::Read);
if (file.IsInvalid()) {
return false;
}
std::array<uint32_t, 37> header {};
uint32_t read = 0;
file.Read(header.data(), sizeof(header), &read);
const auto width = header[4];
const auto height = header[3];
if (read != sizeof(header) || header[0] != 0x20534444 || header[1] != 124 || header[19] != 32 ||
header[20] != 4 || header[21] != 0x30315844 || header[32] != 98 || header[33] != 3 ||
header[34] != 0 || header[35] != 1 || header[6] != 0 || header[7] != 1 || header[28] != 0 ||
width == 0 || height == 0 || width > 3840 || height > 2160) {
return false;
}
const uint32_t size = ((width + 3) / 4) * ((height + 3) / 4) * 16;
if (file.Size() != sizeof(header) + size) {
return false;
}
constexpr auto format = vk::Format::eBc7UnormBlock;
constexpr auto features = vk::FormatFeatureFlagBits::eSampledImage |
vk::FormatFeatureFlagBits::eSampledImageFilterLinear |
vk::FormatFeatureFlagBits::eTransferDst;
if ((m_graphics.GetFormatProperties(format).optimalTilingFeatures & features) != features) {
LOGF("[Splash] host does not support BC7 textures\n");
return false;
}
vk::BufferCreateInfo buffer {};
buffer.size = size;
buffer.usage = vk::BufferUsageFlagBits::eTransferSrc;
VmaAllocationCreateInfo allocation {};
allocation.usage = VMA_MEMORY_USAGE_AUTO_PREFER_HOST;
allocation.flags =
VMA_ALLOCATION_CREATE_MAPPED_BIT | VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT;
VmaAllocationInfo mapped {};
if (vmaCreateBuffer(m_graphics.allocator, static_cast<const VkBufferCreateInfo*>(buffer),
&allocation, &m_staging, &m_staging_allocation, &mapped) != VK_SUCCESS) {
LOGF("[Splash] could not allocate DDS upload buffer\n");
return false;
}
file.Read(mapped.pMappedData, size, &read);
if (read != size) {
return false;
}
RequireVulkanSuccess(static_cast<vk::Result>(vmaFlushAllocation(m_graphics.allocator,
m_staging_allocation, 0, size)),
"flush splash image upload");
vk::ImageCreateInfo image {};
image.imageType = vk::ImageType::e2D;
image.extent = {width, height, 1};
image.mipLevels = 1;
image.arrayLayers = 1;
image.format = format;
image.tiling = vk::ImageTiling::eOptimal;
image.usage = vk::ImageUsageFlagBits::eTransferDst | vk::ImageUsageFlagBits::eSampled;
image.samples = vk::SampleCountFlagBits::e1;
if (!m_graphics.CreateImage(image, m_image)) {
LOGF("[Splash] could not allocate DDS image\n");
return false;
}
vk::ImageViewCreateInfo view {};
view.image = m_image.image;
view.viewType = vk::ImageViewType::e2D;
view.format = format;
view.subresourceRange = {vk::ImageAspectFlagBits::eColor, 0, 1, 0, 1};
RequireVulkanSuccess(m_graphics.device.createImageView(&view, nullptr, &m_view),
"create splash image view");
m_descriptor = ImGui_ImplVulkan_AddTexture(static_cast<VkImageView>(m_view),
VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL);
return true;
}
void DdsImage::Upload(vk::CommandBuffer command) {
if (m_image.state.layout == vk::ImageLayout::eShaderReadOnlyOptimal) {
return;
}
vk::ImageMemoryBarrier barrier {};
barrier.dstAccessMask = vk::AccessFlagBits::eTransferWrite;
barrier.oldLayout = vk::ImageLayout::eUndefined;
barrier.newLayout = vk::ImageLayout::eTransferDstOptimal;
barrier.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
barrier.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
barrier.image = m_image.image;
barrier.subresourceRange = {vk::ImageAspectFlagBits::eColor, 0, 1, 0, 1};
command.pipelineBarrier(vk::PipelineStageFlagBits::eTopOfPipe,
vk::PipelineStageFlagBits::eTransfer, {}, 0, nullptr, 0, nullptr, 1,
&barrier);
vk::BufferImageCopy copy {};
copy.imageSubresource = {vk::ImageAspectFlagBits::eColor, 0, 0, 1};
copy.imageExtent = m_image.extent;
command.copyBufferToImage(m_staging, m_image.image, vk::ImageLayout::eTransferDstOptimal, 1,
&copy);
barrier.srcAccessMask = vk::AccessFlagBits::eTransferWrite;
barrier.dstAccessMask = vk::AccessFlagBits::eShaderRead;
barrier.oldLayout = vk::ImageLayout::eTransferDstOptimal;
barrier.newLayout = vk::ImageLayout::eShaderReadOnlyOptimal;
command.pipelineBarrier(vk::PipelineStageFlagBits::eTransfer,
vk::PipelineStageFlagBits::eFragmentShader, {}, 0, nullptr, 0, nullptr,
1, &barrier);
m_image.state.layout = vk::ImageLayout::eShaderReadOnlyOptimal;
}
ImTextureRef DdsImage::Texture() const {
return ImTextureRef {static_cast<ImTextureID>(reinterpret_cast<uintptr_t>(m_descriptor))};
}
} // namespace Libs::Graphics
+32
View File
@@ -0,0 +1,32 @@
#ifndef EMULATOR_SRC_GRAPHICS_PRESENTATION_DDSIMAGE_H_
#define EMULATOR_SRC_GRAPHICS_PRESENTATION_DDSIMAGE_H_
#include "graphics/host_gpu/graphicContext.h"
#include "imgui.h"
#include <filesystem>
namespace Libs::Graphics {
class DdsImage final {
public:
explicit DdsImage(GraphicContext& graphics): m_graphics(graphics) {}
~DdsImage();
KYTY_CLASS_NO_COPY(DdsImage);
bool Load(const std::filesystem::path& path);
void Upload(vk::CommandBuffer command);
[[nodiscard]] ImTextureRef Texture() const;
private:
GraphicContext& m_graphics;
VulkanImage m_image;
vk::ImageView m_view = nullptr;
VkDescriptorSet m_descriptor = VK_NULL_HANDLE;
VkBuffer m_staging = VK_NULL_HANDLE;
VmaAllocation m_staging_allocation = nullptr;
};
} // namespace Libs::Graphics
#endif // EMULATOR_SRC_GRAPHICS_PRESENTATION_DDSIMAGE_H_
+110 -10
View File
@@ -3,9 +3,12 @@
#include <SDL3/SDL.h>
#include "common/assert.h"
#include "common/emulatorConfig.h"
#include "common/file.h"
#include "common/logging/log.h"
#include "common/stringUtils.h"
#include "graphics/host_gpu/graphicContext.h"
#include "graphics/presentation/ddsImage.h"
#include "imgui.h"
#include "imgui_internal.h"
#include "imgui_impl_vulkan.h"
@@ -13,6 +16,7 @@
#include "libs/dialog.h"
#include "libs/ime.h"
#include "libs/imeDialog.h"
#include "loader/systemContent.h"
#include "stb_image.h"
#include <algorithm>
@@ -23,6 +27,7 @@
#include <cstdint>
#include <cstring>
#include <deque>
#include <fmt/format.h>
#include <limits>
#include <memory>
#include <mutex>
@@ -37,6 +42,8 @@ namespace Libs::Graphics {
namespace {
using Clock = std::chrono::steady_clock;
namespace CoreIme = Libs::Ime;
namespace DialogIme = Libs::Dialog::ImeDialog;
namespace SystemDialog = Libs::Dialog::SystemDialog;
@@ -194,6 +201,10 @@ struct TrophyNotification {
std::mutex g_trophy_notification_mutex;
std::deque<TrophyNotification> g_trophy_notifications;
std::atomic<bool> g_trophy_notification_active {false};
std::array<std::filesystem::path, 2> g_splash_paths;
std::atomic<bool> g_splash_active {false};
std::atomic<Clock::duration::rep> g_splash_hidden_at {0};
bool g_splash_fast = false;
std::vector<std::string_view> WrapTrophyTitle(std::string_view text, float font_size, float width) {
std::vector<std::string_view> lines;
@@ -464,6 +475,29 @@ void CheckVulkanResult(VkResult result) {
} // namespace
void InitializeSplashScreen(const std::filesystem::path& sce_sys) {
const auto language = Config::GetConsoleLanguage();
for (size_t i = 0; i < g_splash_paths.size(); ++i) {
auto& path = g_splash_paths[i];
path = sce_sys / fmt::format("pic{}_{:02}.dds", i + 1, language);
if (!Common::File::IsFileExisting(path)) {
path = sce_sys / fmt::format("pic{}.dds", i + 1);
}
}
int32_t attribute = 0;
Loader::SystemContentParamSfoGetInt("ATTRIBUTE3", &attribute);
g_splash_fast = (attribute & 0x1000) != 0;
g_splash_hidden_at.store(0, std::memory_order_release);
g_splash_active.store(Common::File::IsFileExisting(g_splash_paths[0]) &&
Common::File::IsFileExisting(g_splash_paths[1]),
std::memory_order_release);
}
void HideSplashScreen() {
Clock::duration::rep expected = 0;
g_splash_hidden_at.compare_exchange_strong(expected, Clock::now().time_since_epoch().count());
}
void InitializeSystemOverlayInput(SDL_Window* window) {
EXIT_IF(window == nullptr);
const Uint32 type = SDL_RegisterEvents(1);
@@ -481,6 +515,7 @@ void InitializeSystemOverlayInput(SDL_Window* window) {
}
void ShutdownSystemOverlayInput() {
g_splash_active.store(false, std::memory_order_release);
CoreIme::SetVisibilityCallback(nullptr);
DialogIme::SetVisibilityCallback(nullptr);
SystemDialog::SetVisibilityCallback(nullptr);
@@ -515,8 +550,10 @@ SystemOverlayVisualState GetSystemOverlayVisualState() noexcept {
const auto dialog = DialogIme::GetVisualState();
const auto system = SystemDialog::GetVisualState();
const bool trophy_active = g_trophy_notification_active.load(std::memory_order_acquire);
return {core.active || dialog.active || system.active || trophy_active,
core.revision + dialog.revision + system.revision + (trophy_active ? 1u : 0u)};
const bool splash = g_splash_active.load(std::memory_order_acquire);
return {core.active || dialog.active || system.active || trophy_active || splash,
core.revision + dialog.revision + system.revision + (trophy_active ? 1u : 0u) +
(splash ? 2u : 0u)};
}
bool ProcessSystemOverlayInput(const SDL_Event& event) {
@@ -656,8 +693,6 @@ void NotifyTrophyUnlocked(std::string_view name, int32_t grade,
}
struct SystemOverlay::Impl {
using Clock = std::chrono::steady_clock;
explicit Impl(GraphicContext& context): graphics(context) {}
~Impl() {
@@ -770,6 +805,65 @@ struct SystemOverlay::Impl {
return texture;
}
void FinishSplash() {
g_splash_active.store(false, std::memory_order_release);
if (splash_dds[0] || splash_dds[1]) {
Common::LockGuard lock(graphics.queue_mutex);
RequireVulkanSuccess(graphics.queue.waitIdle(), "retire splash images");
splash_dds = {};
}
}
void DrawSplash(vk::Extent2D frame_extent) {
if (!g_splash_active.load(std::memory_order_acquire)) {
return;
}
for (size_t i = 0; i < g_splash_paths.size(); ++i) {
if (splash_dds[i]) {
continue;
}
const auto& path = g_splash_paths[i];
auto image = std::make_unique<DdsImage>(graphics);
if (!image->Load(path)) {
LOGF("[Splash] could not load %s\n", path.string().c_str());
FinishSplash();
return;
}
splash_dds[i] = std::move(image);
}
const auto now = Clock::now();
if (splash_born == Clock::time_point {}) {
splash_born = now;
}
const auto hidden = g_splash_hidden_at.load(std::memory_order_acquire);
const float fade =
hidden == 0
? 0.0f
: std::chrono::duration<float>(now.time_since_epoch() - Clock::duration {hidden})
.count() /
0.3f;
const float alpha = 1.0f - std::clamp(fade, 0.0f, 1.0f);
if (alpha == 0.0f) {
FinishSplash();
return;
}
const float age = std::chrono::duration<float>(now - splash_born).count();
const float logo = g_splash_fast ? 1.0f : std::clamp((age - 1.4f) / 2.2f, 0.0f, 1.0f);
const ImVec2 screen {static_cast<float>(frame_extent.width),
static_cast<float>(frame_extent.height)};
const float fit = std::min(screen.x / 3840.0f, screen.y / 2160.0f);
const ImVec2 size {3840.0f * fit, 2160.0f * fit};
const ImVec2 begin {(screen.x - size.x) * 0.5f, (screen.y - size.y) * 0.5f};
const ImVec2 end {begin.x + size.x, begin.y + size.y};
auto* draw = ImGui::GetBackgroundDrawList();
draw->AddRectFilled({0, 0}, screen, IM_COL32(0, 0, 0, static_cast<int>(255.0f * alpha)));
for (size_t i = 0; i < g_splash_paths.size(); ++i) {
draw->AddImage(
splash_dds[i]->Texture(), begin, end, {0, 0}, {1, 1},
IM_COL32(255, 255, 255, static_cast<int>(255.0f * alpha * (i == 0 ? 1.0f : logo))));
}
}
void EnsureVulkan(vk::Format format, uint32_t image_count) {
EnsureContext();
if (vulkan_initialized) {
@@ -1215,7 +1309,8 @@ struct SystemOverlay::Impl {
OverlaySnapshot snapshot;
const bool has_overlay = GetOverlaySnapshot(&snapshot);
const auto now = Clock::now();
if (!has_overlay && !g_trophy_notification_active.load(std::memory_order_acquire)) {
if (!has_overlay && !g_trophy_notification_active.load(std::memory_order_acquire) &&
!g_splash_active.load(std::memory_order_acquire)) {
return false;
}
const auto prepared_session = snapshot.session;
@@ -1245,12 +1340,9 @@ struct SystemOverlay::Impl {
last_frame = now;
ImGui_ImplVulkan_NewFrame();
ImGui::NewFrame();
DrawSplash(frame_extent);
GetOverlaySnapshot(&snapshot);
if (snapshot.session != prepared_session) {
ImGui::EndFrame();
return false;
}
if (has_overlay) {
if (has_overlay && snapshot.session == prepared_session) {
if (snapshot.session.kind == OverlayKind::Dialog) {
DrawDialog(snapshot.dialog, frame_extent);
} else {
@@ -1264,6 +1356,11 @@ struct SystemOverlay::Impl {
}
void Record(vk::CommandBuffer command, vk::ImageView target) {
for (const auto& image: splash_dds) {
if (image) {
image->Upload(command);
}
}
vk::RenderingAttachmentInfo color {};
color.sType = vk::StructureType::eRenderingAttachmentInfo;
color.imageView = target;
@@ -1290,6 +1387,7 @@ struct SystemOverlay::Impl {
return;
}
ImGui::SetCurrentContext(imgui_context);
splash_dds = {};
ImGui_ImplVulkan_Shutdown();
vulkan_initialized = false;
}
@@ -1309,6 +1407,8 @@ struct SystemOverlay::Impl {
std::optional<TrophyNotification> trophy_notification;
Clock::time_point trophy_notification_born;
std::vector<std::unique_ptr<ImTextureData>> retired_trophy_images;
std::array<std::unique_ptr<DdsImage>, 2> splash_dds;
Clock::time_point splash_born;
SDL_AudioStream* trophy_sound_stream = nullptr;
uint8_t* trophy_sound_data = nullptr;
uint32_t trophy_sound_size = 0;
@@ -5,6 +5,7 @@
#include "graphics/host_gpu/vulkanCommon.h"
#include <cstddef>
#include <filesystem>
#include <memory>
#include <span>
#include <string_view>
@@ -21,6 +22,8 @@ struct SystemOverlayVisualState {
uint64_t revision;
};
void InitializeSplashScreen(const std::filesystem::path& sce_sys);
void HideSplashScreen();
void InitializeSystemOverlayInput(SDL_Window* window);
void ShutdownSystemOverlayInput();
bool ProcessSystemOverlayInput(const SDL_Event& event);
+7 -13
View File
@@ -566,29 +566,23 @@ Graphics::ImageInfo BufferAttributeGroup::ImageInfo(const VideoOutBuffer& buffer
"pixels unchanged (format=0x{:016x}).\n", attribute.pixel_format));
}
}
const auto tile_mode = Graphics::Prospero::TileMode::kRenderTarget;
const auto pitch =
Graphics::TileGetTexturePitch(pixel_format.guest_format, attribute.width, tile_mode);
Graphics::TileSizeAlign total {};
Graphics::TileGetTextureTotalSize(pixel_format.guest_format, attribute.width, attribute.height,
1, 1, tile_mode, false, total);
if (total.size == 0 || total.align != 65536 ||
(buffer.data_address & (total.align - 1u)) != 0) {
EXIT("invalid video-out surface footprint or alignment\n");
}
const auto tile_mode = Graphics::Prospero::TileMode::kRenderTarget;
Graphics::ImageInfo info {};
info.data = {buffer.data_address, total.size};
info.data = {buffer.data_address, 0};
info.pixel_format = pixel_format.format;
info.guest_format = pixel_format.guest_format;
info.type = Graphics::Prospero::ImageType::kColor2D;
info.extent = {attribute.width, attribute.height, 1};
info.resources = {1, 1};
info.pitch = pitch;
info.bytes_per_block = pixel_format.bytes_per_element;
info.samples = 1;
info.tile_mode = tile_mode;
info.bgra16 = pixel_format.bgra16;
info.mip_layout[0] = {0, total.size, pitch, attribute.height};
info.UpdateSize();
if (info.data.size == 0 || info.tiling.block.block_size != 65536 ||
(buffer.data_address & 65535u) != 0) {
EXIT("invalid video-out surface footprint or alignment\n");
}
if (compression != Graphics::VideoOutCompression::Uncompressed) {
Graphics::TileSizeAlign dcc_size {};
if (!Graphics::TileGetDccSize(attribute.width, attribute.height, 1,
@@ -326,6 +326,19 @@ static void GameEventController([[maybe_unused]] const EventController& f) {
const auto button = ControllerButtonToPadButton(f.button);
if (button != 0) {
Controller::SetButton(f.id, button, f.down);
} else if (f.button == SDL_GAMEPAD_BUTTON_BACK) {
// Pads without a touchpad (Switch, Xbox, ...) have no way to click it, which
// blocks games that require it. Use their Back/Select button as a centered click.
auto* pad = SDL_GetGamepadFromID(f.id);
if (pad != nullptr && SDL_GetNumGamepadTouchpads(pad) == 0) {
if (f.down) {
Controller::SetTouchPad(f.id, 0, true, 0.5f, 0.5f);
}
Controller::SetButton(f.id, Controller::PAD_BUTTON_TOUCH_PAD, f.down);
if (f.up) {
Controller::SetTouchPad(f.id, 0, false, 0.5f, 0.5f);
}
}
}
}
@@ -7,8 +7,8 @@
#include "graphics/shader/recompiler/frontend/cfg/ShaderCFG.h"
#include "graphics/shader/recompiler/frontend/decode/ShaderDecoder.h"
#include "graphics/shader/recompiler/frontend/translate/Translate.h"
#include "graphics/shader/recompiler/frontend/translate/Translator.h"
#include "graphics/shader/recompiler/ir/ShaderIR.h"
#include "graphics/shader/recompiler/ir/passes/BindingLayout.h"
#include "graphics/shader/recompiler/ir/passes/ConstantPropagation.h"
#include "graphics/shader/recompiler/ir/passes/DeadCodeElimination.h"
#include "graphics/shader/recompiler/ir/passes/ReadLaneElimination.h"
@@ -21,6 +21,7 @@
#include <array>
#include <chrono>
#include <fmt/format.h>
#include <functional>
#include <map>
#include <span>
#include <utility>
@@ -467,6 +468,7 @@ Decoder::Program DecodeFusedProgram(std::span<const uint32_t> front, std::span<c
Decoder::DecodeInstruction(joined_code, front_words - 1u, result.instructions.back());
Decoder::Program back_program;
Decoder::DecodeProgram(back, back_program);
result.has_swap_pc |= back_program.has_swap_pc;
const auto back_pc = front_words * sizeof(uint32_t);
for (auto& inst: back_program.instructions) {
// A back-stage PC-relative data reference requires its guest code address.
@@ -479,11 +481,227 @@ Decoder::Program DecodeFusedProgram(std::span<const uint32_t> front, std::span<c
return result;
}
void PrepareCallTarget(ShaderSource& source, const CompileOptions& options) {
using namespace IR;
for (uint32_t i = 0; i < source.decoded.instructions.size(); ++i) {
if (source.decoded.instructions[i].opcode != Decoder::Opcode::S_SWAPPC_B64) continue;
EXIT_NOT_IMPLEMENTED(source.call.has_value() || options.stage != ShaderType::Compute);
source.call = ShaderSource::Call {.instruction = i};
}
EXIT_IF(options.input_info.compute == nullptr);
const auto& call = source.decoded.instructions[source.call->instruction];
EXIT_NOT_IMPLEMENTED(call.src0.kind != Decoder::OperandKind::Sgpr ||
call.dst.kind != Decoder::OperandKind::Sgpr ||
call.src0.reg != call.dst.reg || call.dst.reg >= 105u);
const auto graph = CFG::BuildGraph(source.decoded);
Program query;
query.stage = options.stage;
query.user_data_base = options.user_data_base;
query.user_data_count = static_cast<uint32_t>(options.user_data.size());
query.wave_size = options.wave_size;
auto& block = *query.block_storage.emplace_back(std::make_unique<Block>());
query.blocks.push_back(&block);
struct Producer { bool ready = false; std::map<uint32_t, Value> values; };
std::map<uint32_t, Producer> producers;
std::function<Value(uint32_t, uint32_t)> Register;
std::function<Value(uint32_t, Value)> Resolve = [&](uint32_t before, Value value) -> Value {
value = value.Resolve();
auto* inst = value.TryInstruction();
if (inst == nullptr) return value;
uint32_t code = UINT32_MAX;
switch (inst->GetOpcode()) {
case ValueOpcode::GetScalarRegister: code = static_cast<uint32_t>(inst->Arg(0).ScalarRegister()); break;
case ValueOpcode::GetVccLo: code = 106u; break;
case ValueOpcode::GetVccHi: code = 107u; break;
default: break;
}
if (code != UINT32_MAX) {
const auto result = Register(before, code);
inst->ReplaceUsesWith(result);
return result;
}
for (size_t i = 0; i < inst->NumArgs(); ++i) inst->SetArg(i, Resolve(before, inst->Arg(i)));
return value;
};
Register = [&](uint32_t before, uint32_t code) -> Value {
const auto index = CFG::FindScalarDefinition(source.decoded, graph, before, code);
if (index == UINT32_MAX) {
EXIT_IF(code < query.user_data_base || code - query.user_data_base >= query.user_data_count);
return IREmitter(&block).GetUserData(static_cast<ScalarReg>(code));
}
auto [entry, inserted] = producers.try_emplace(index);
auto& producer = entry->second;
if (inserted) {
Block translated;
Frontend::Translator translator(query, &translated, 1u,
(options.input_info.compute->float_mode & 0x10u) == 0u,
!options.input_info.compute->async_compute);
translator.TranslateInstruction(source.decoded.instructions[index]);
for (const auto& inst: translated) {
uint32_t destination = UINT32_MAX;
Value value;
switch (inst.GetOpcode()) {
case ValueOpcode::SetScalarRegister:
destination = static_cast<uint32_t>(inst.Arg(0).ScalarRegister()); value = inst.Arg(1); break;
case ValueOpcode::SetVccLo: destination = 106u; value = inst.Arg(0); break;
case ValueOpcode::SetVccHi: destination = 107u; value = inst.Arg(0); break;
default: break;
}
if (destination != UINT32_MAX) producer.values.insert_or_assign(destination, value);
}
for (auto& [destination, value]: producer.values) value = Resolve(index, value);
for (auto& inst: translated) inst.SetParent(&block);
block.Instructions().splice(block.end(), translated.Instructions());
producer.ready = true;
}
EXIT_IF(!producer.ready || !producer.values.contains(code));
return producer.values.at(code).Resolve();
};
for (uint32_t i = 0; i < 2u; ++i) {
const auto value = Register(source.call->instruction, call.src0.reg + i);
EXIT_IF(!ValidateRuntimeValue(query, value, RuntimeValueType::Integer));
auto& root = query.value_storage.emplace_back(ValueOpcode::ReferenceU32);
root.SetArg(0, value);
source.call->target[i] = Value(&root);
}
RewriteToSsa(query.blocks);
ConstantPropagationPass(query.blocks, query.wave_size);
RemoveIdentities(query.blocks);
EliminateDeadCode(query.blocks);
for (auto& value: source.call->target) value = value.Instruction()->Arg(0).Resolve();
query.value_storage.clear();
for (auto& inst: block) {
inst.SetParent(nullptr);
if (inst.GetOpcode() == ValueOpcode::LoadAddressU32 || inst.GetOpcode() == ValueOpcode::ReadConstBuffer)
inst.SetFlags(SrtReadFlags {.index = inst.Flags<MemoryFlags>().index});
}
query.value_storage.splice(query.value_storage.end(), block.Instructions());
source.call_targets = std::move(static_cast<ResourcePlan&>(query));
}
} // namespace
ShaderSource PrepareShaderSource(std::span<const uint32_t> code, const CompileOptions& options) {
EXIT_IF(code.empty());
ShaderSource source;
if (!options.back_code.empty()) {
source.decoded = DecodeFusedProgram(code, options.back_code, source.code);
} else if (options.stage == ShaderType::Local) {
source.decoded = Decoder::DecodeFrontProgram(code);
auto& handoff = source.decoded.instructions.back();
handoff.opcode = Decoder::Opcode::S_ENDPGM;
handoff.src_count = 0;
} else {
Decoder::DecodeProgram(code, source.decoded);
}
if (source.decoded.has_swap_pc) {
PrepareCallTarget(source, options);
source.code.assign(code.begin(), code.end());
source.decoded.code = source.code;
}
return source;
}
const Decoder::Program& RefreshShaderSource(ShaderSource& source, const IR::SrtRuntime& runtime) {
auto& reads = source.reads;
reads.clear();
IR::SrtReadCapture capture(runtime, reads);
const auto clean = IR::CleanRuntime(capture.ObservedRuntime());
IR::SrtWalker walker(source.call_targets, clean);
uint32_t low = 0, high = 0;
const auto& call = source.decoded.instructions[source.call->instruction];
if (!walker.Evaluate(source.call->target[0], low) || !walker.Evaluate(source.call->target[1], high))
EXIT("shader call at pc 0x%08x has an unavailable scalar target", call.pc);
// SWAPPC reads the old aliased pair and ignores the target's low two bits.
const auto address = ((uint64_t {high} << 32u) | low) & ~uint64_t {3};
EXIT_IF(address == 0u);
const auto query_reads = reads.size();
if (source.linked) {
auto& linked = *source.linked;
const auto function = std::span(linked.code).subspan(source.code.size());
if (clean.read_memory(clean.userdata, address, linked.observed_function) &&
std::ranges::equal(function, linked.observed_function)) {
return linked.decoded;
}
reads.resize(query_reads);
}
ShaderSource::Linked linked {.code = source.code,
.decoded = source.decoded};
const auto& last = source.decoded.instructions.back();
EXIT_NOT_IMPLEMENTED(last.opcode != Decoder::Opcode::S_ENDPGM);
// Keep the caller's data/footer bytes at their native offsets. The leaf's virtual CFG
// PCs follow decoded instructions; its physical code is independently read and owned.
const auto function_pc = last.pc + last.word_count * sizeof(uint32_t);
for (uint32_t index = 0;;) {
// The observed queue helper is a scalar leaf. Scalar encodings need at most one literal.
EXIT_IF(index >= 64u || address > UINT64_MAX - (uint64_t {index} + 2u) * sizeof(uint32_t));
std::array<uint32_t, 2> words {};
if (!clean.read_memory(clean.userdata, address + uint64_t {index} * sizeof(uint32_t),
std::span(words).first(1)))
EXIT("shader call at pc 0x%08x cannot read leaf code", call.pc);
const auto family = Decoder::GetInstructionFamily(words[0]);
EXIT_NOT_IMPLEMENTED(family != Decoder::Family::SOP1 && family != Decoder::Family::SOP2 &&
family != Decoder::Family::SOPC && family != Decoder::Family::SOPK &&
family != Decoder::Family::SOPP);
Decoder::Instruction inst;
Decoder::DecodeInstruction(words, 0u, inst);
if (inst.word_count == 2u) {
if (!clean.read_memory(clean.userdata, address + uint64_t {index + 1u} * sizeof(uint32_t),
std::span(words).last(1)))
EXIT("shader call at pc 0x%08x cannot read leaf literal", call.pc);
inst = {};
Decoder::DecodeInstruction(words, 0u, inst);
}
inst.pc = index * sizeof(uint32_t);
if (Decoder::IsDirectBranch(inst.opcode)) inst.branch_target += inst.pc;
linked.code.insert(linked.code.end(), inst.raw, inst.raw + inst.word_count);
index += inst.word_count;
EXIT_NOT_IMPLEMENTED(inst.opcode == Decoder::Opcode::S_SWAPPC_B64 ||
inst.opcode == Decoder::Opcode::S_GETPC_B64 ||
inst.opcode == Decoder::Opcode::S_ENDPGM);
EXIT_IF(CFG::MayWriteScalarRegister(inst, call.dst.reg) || CFG::MayWriteScalarRegister(inst, call.dst.reg + 1u));
const bool returns = inst.opcode == Decoder::Opcode::S_SETPC_B64;
if (returns) {
EXIT_NOT_IMPLEMENTED(inst.src0.kind != Decoder::OperandKind::Sgpr || inst.src0.reg != call.dst.reg);
inst.opcode = Decoder::Opcode::S_BRANCH;
inst.branch_target = call.pc + call.word_count * sizeof(uint32_t);
} else if (Decoder::IsDirectBranch(inst.opcode)) {
EXIT_IF(inst.branch_target > UINT32_MAX - function_pc);
inst.branch_target += function_pc;
}
inst.pc += function_pc;
linked.decoded.instructions.push_back(std::move(inst));
if (returns) break;
}
const auto function_end = linked.decoded.instructions.back().pc +
linked.decoded.instructions.back().word_count * sizeof(uint32_t);
for (size_t i = source.decoded.instructions.size(); i + 1u < linked.decoded.instructions.size(); ++i) {
const auto& inst = linked.decoded.instructions[i];
if (Decoder::IsDirectBranch(inst.opcode))
EXIT_IF(inst.branch_target < function_pc || inst.branch_target >= function_end);
}
reads.resize(query_reads);
reads.emplace_back(address, (linked.code.size() - source.code.size()) * sizeof(uint32_t));
linked.decoded.instructions[source.call->instruction].branch_target = function_pc;
linked.decoded.code = linked.code;
linked.observed_function.resize(linked.code.size() - source.code.size());
source.linked = std::move(linked);
++source.revision;
return source.linked->decoded;
}
TranslateResult TranslateProgram(std::span<const uint32_t> code, const CompileOptions& options) {
if (code.empty()) {
EXIT("shader recompiler input is empty\n");
auto source = PrepareShaderSource(code, options);
return TranslateProgram(source.decoded, options);
}
TranslateResult TranslateProgram(const Decoder::Program& decoded, const CompileOptions& options) {
EXIT_IF(decoded.instructions.empty());
if (decoded.has_swap_pc) {
for (const auto& inst: decoded.instructions) {
if (inst.opcode == Decoder::Opcode::S_SWAPPC_B64 && inst.branch_target == UINT32_MAX)
EXIT("shader call at pc 0x%08x must be linked before translation", inst.pc);
}
}
if (options.stage != ShaderType::Compute && options.stage != ShaderType::Vertex &&
options.stage != ShaderType::Pixel && options.stage != ShaderType::Mesh &&
@@ -494,34 +712,11 @@ TranslateResult TranslateProgram(std::span<const uint32_t> code, const CompileOp
}
const auto compile_begin = std::chrono::steady_clock::now();
const auto phase_ms = [&compile_begin]() {
const auto phase_ms = [&compile_begin]() {
return static_cast<uint64_t>(std::chrono::duration_cast<std::chrono::milliseconds>(
std::chrono::steady_clock::now() - compile_begin)
.count());
std::chrono::steady_clock::now() - compile_begin).count());
};
LOGF("%s phase begin: stage=%s hash=0x%016" PRIx64 " code_words=%" PRIu64 " decode\n",
GetDumpLabel(options), StageName(options.stage), options.shader_hash,
static_cast<uint64_t>(code.size()));
Decoder::Program decoded;
std::vector<uint32_t> joined_code;
if (!options.back_code.empty()) {
decoded = DecodeFusedProgram(code, options.back_code, joined_code);
} else if (options.stage == ShaderType::Local) {
decoded = Decoder::DecodeFrontProgram(code);
// The separately compiled hull half runs in the next Vulkan stage.
auto& handoff = decoded.instructions.back();
handoff.opcode = Decoder::Opcode::S_ENDPGM;
handoff.src_count = 0;
} else {
Decoder::DecodeProgram(code, decoded);
}
LOGF("%s phase end: stage=%s hash=0x%016" PRIx64 " decode instructions=%" PRIu64
" elapsed_ms=%" PRIu64 "\n",
GetDumpLabel(options), StageName(options.stage), options.shader_hash,
static_cast<uint64_t>(decoded.instructions.size()), phase_ms());
std::string decoded_dump;
if (options.dump_ir) {
decoded_dump = Decoder::ProgramToString(decoded);
@@ -607,6 +802,12 @@ TranslateResult TranslateProgram(std::span<const uint32_t> code, const CompileOp
IR::RemoveIdentities(ir.blocks);
IR::EliminateDeadCode(ir.blocks);
}
if (decoded.has_swap_pc) {
// Leaf returns are resolved CFG edges; native PC values must not escape into GPU data.
for (const auto* block: ir.blocks)
for (const auto& inst: *block)
EXIT_NOT_IMPLEMENTED(inst.GetOpcode() == IR::ValueOpcode::GetShaderBase);
}
LowerTessellationMemory(ir, options);
std::string cfg_dump;
if (options.dump_ir) {
@@ -657,7 +858,11 @@ CompileResult CompileProgram(TranslateResult translated, const CompileOptions& o
const auto resource = inst.Flags<uint32_t>();
first = resource < ir.info.images.size() &&
ir.info.images[resource].indirect_root == resource ? 1u : 0u;
} else if (op != IR::ValueOpcode::GetSamplerResource) {
} else if (op == IR::ValueOpcode::GetSamplerResource) {
const auto resource = inst.Flags<uint32_t>();
first = resource < ir.info.samplers.size() &&
!ir.info.samplers[resource].indirect_resources.empty() ? 1u : 0u;
} else {
continue;
}
for (size_t index = first; index < inst.NumArgs(); index++) {
@@ -670,20 +875,17 @@ CompileResult CompileProgram(TranslateResult translated, const CompileOptions& o
IR::EliminateDeadCode(ir.blocks);
IR::CollectShaderInfo(ir, options.input_info);
IR::AllocateBindings(ir, push_data_start_dword,
ir.stage == ShaderType::Compute && options.input_info.compute != nullptr &&
options.input_info.compute->lds_storage);
std::string ir_dump;
if (options.dump_ir) {
ir_dump = MakeIrDump(translated.cfg_dump, ir);
if (options.early_dump) {
LOGF("%s native IR and bindings (early):\n%s", GetDumpLabel(options), ir_dump.c_str());
LOGF("%s native IR (early):\n%s", GetDumpLabel(options), ir_dump.c_str());
}
}
LOGF("%s phase begin: stage=%s hash=0x%016" PRIx64 " SPIR-V EmitProgram\n",
GetDumpLabel(options), StageName(ir.stage), ir.shader_hash);
auto spirv = Spirv::EmitProgram(ir, options.input_info);
auto spirv = Spirv::EmitProgram(ir, options.input_info, push_data_start_dword);
LOGF("%s phase end: stage=%s hash=0x%016" PRIx64 " SPIR-V EmitProgram words=%" PRIu64
" elapsed_ms=%" PRIu64 "\n",
GetDumpLabel(options), StageName(ir.stage), ir.shader_hash,
@@ -4,8 +4,11 @@
#include "common/common.h"
#include "common/stringUtils.h"
#include "graphics/shader/recompiler/ir/passes/ResourceMaterialization.h"
#include "graphics/shader/recompiler/frontend/decode/ShaderDecoder.h"
#include "graphics/shader/shader.h"
#include <array>
#include <optional>
#include <span>
#include <vector>
@@ -37,6 +40,32 @@ struct CompileResult {
IR::Program program;
};
// Decoded source and typed scalar call queries are retained once per cache source.
struct ShaderSource {
struct Call {
uint32_t instruction;
std::array<IR::Value, 2> target;
};
struct Linked {
std::vector<uint32_t> code;
Decoder::Program decoded;
std::vector<uint32_t> observed_function;
};
std::vector<uint32_t> code;
Decoder::Program decoded;
IR::ResourcePlan call_targets;
std::optional<Call> call;
std::optional<Linked> linked;
uint64_t revision = 0;
std::vector<std::pair<uint64_t, uint64_t>> reads;
};
[[nodiscard]] ShaderSource PrepareShaderSource(std::span<const uint32_t> code,
const CompileOptions& options);
const Decoder::Program& RefreshShaderSource(ShaderSource& source, const IR::SrtRuntime& runtime);
[[nodiscard]] TranslateResult TranslateProgram(const Decoder::Program& decoded,
const CompileOptions& options);
[[nodiscard]] TranslateResult TranslateProgram(std::span<const uint32_t> code,
const CompileOptions& options);
[[nodiscard]] CompileResult CompileProgram(TranslateResult translated,
@@ -171,10 +171,12 @@ void Builder::AddFunction(std::span<const uint32_t> words) {
}
DeferredPhi Builder::AddDeferredPhi(uint32_t type, uint32_t result, size_t incoming_count) {
std::vector<uint32_t> words {spv::OpPhi, type, result};
words.resize(words.size() + incoming_count * 2u);
const DeferredPhi phi {m_functions.size()};
AddFunction(words);
const auto word_count = static_cast<uint32_t>(3u + incoming_count * 2u);
m_functions.push_back((word_count << spv::WordCountShift) | spv::OpPhi);
m_functions.push_back(type);
m_functions.push_back(result);
m_functions.resize(m_functions.size() + incoming_count * 2u);
m_unpatched_phi_incomings += incoming_count;
return phi;
}
@@ -3,7 +3,6 @@
#include "common/assert.h"
#include "graphics/shader/recompiler/backend/spirv/spirvEmitterInternal.h"
#include "graphics/shader/recompiler/ir/ShaderIR.h"
#include "graphics/shader/recompiler/ir/passes/BindingLayout.h"
#include <algorithm>
#include <array>
@@ -19,92 +18,14 @@ namespace {
}
void ValidateNativeProgram(const IR::Program& program, bool lds_storage) {
using Kind = IR::DescriptorBindingKind;
constexpr auto KindCount = static_cast<size_t>(Kind::Count);
std::array<std::vector<uint32_t>, KindCount> expected;
std::array<bool, KindCount> present {};
const auto Dense = [](size_t size) {
std::vector<uint32_t> values(size);
for (uint32_t i = 0; i < values.size(); i++) {
values[i] = i;
}
return values;
uint64_t live_buffers = 0;
bool uses_lds = false;
bool uses_gds = false;
const auto uses_mapping = [](const auto& resource) {
return resource.indirect_root != UINT32_MAX;
};
auto Expect = [&](Kind kind, std::vector<uint32_t> resources = {}) {
const auto index = static_cast<size_t>(kind);
present[index] = true;
expected[index] = std::move(resources);
};
for (uint32_t i = 0; i < program.info.images.size(); i++) {
const auto kind = IR::DescriptorBindingForImage(program.info.images[i]);
if (!kind.has_value()) {
Fail(program, "native shader plan has an invalid image class");
}
present[static_cast<size_t>(*kind)] = true;
const auto dynamic = program.info.images[i].mip_mode == IR::ImageMipMode::Dynamic;
const auto count = dynamic ? program.info.images[i].mip_count : 1u;
if (count == 0u || (!dynamic && program.info.images[i].mip_count != 1u)) {
Fail(program, "native shader plan has an invalid image mip descriptor count");
}
expected[static_cast<size_t>(*kind)].insert(expected[static_cast<size_t>(*kind)].end(),
count, i);
}
if (!program.info.samplers.empty()) {
Expect(Kind::Samplers, Dense(program.info.samplers.size()));
}
auto& buffers = expected[static_cast<size_t>(Kind::Buffers)];
const auto shared = IR::CollectMemoryResources(program, buffers);
present[static_cast<size_t>(Kind::Buffers)] = !buffers.empty();
if (shared.gds) {
Expect(Kind::Gds);
}
if (shared.lds && lds_storage) {
Expect(Kind::SharedMemory);
}
if (program.info.uses_dma) {
Expect(Kind::BdaPagetable);
Expect(Kind::FaultBuffer);
}
if (IR::UsesFlattenedSrt(program)) {
Expect(Kind::FlattenedSrt);
}
if (program.bindings.ShaderDataDwords() != 0 && !program.bindings.UsesPushData()) {
Expect(Kind::ShaderData);
}
std::array<bool, KindCount> seen {};
for (const auto& binding: program.bindings.descriptors) {
const auto kind = static_cast<size_t>(binding.kind);
if (kind >= KindCount || seen[kind] || !present[kind] ||
binding.resources != expected[kind]) {
Fail(program, "native descriptor groups do not match shader topology");
}
seen[kind] = true;
}
for (size_t i = 0; i < KindCount; i++) {
if (present[i] != seen[i]) {
Fail(program, "native shader plan is missing a required descriptor group");
}
}
const auto has_shader_data_storage = present[static_cast<size_t>(Kind::ShaderData)];
const auto shader_data_dwords = program.bindings.ShaderDataDwords();
const auto user_data_dwords = program.bindings.user_data_registers.size();
const bool has_dispatch_threads = program.bindings.dispatch_thread_dword != IR::PushData::NoStart;
if ((program.bindings.UsesPushData() &&
!IR::PushData::CanFit(program.bindings.push_data_start_dword, shader_data_dwords)) ||
(has_dispatch_threads && (program.stage != ShaderType::Compute ||
program.bindings.dispatch_thread_dword != user_data_dwords)) ||
program.bindings.memory_offset_dword != user_data_dwords + (has_dispatch_threads ? 3u : 0u) ||
program.bindings.memory_offset_count != buffers.size() ||
has_shader_data_storage != (shader_data_dwords != 0 && !program.bindings.UsesPushData()) ||
!std::is_sorted(program.bindings.user_data_registers.begin(),
program.bindings.user_data_registers.end()) ||
std::adjacent_find(program.bindings.user_data_registers.begin(),
program.bindings.user_data_registers.end()) !=
program.bindings.user_data_registers.end()) {
Fail(program, "native shader-data layout is inconsistent");
}
bool uses_flattened_srt = std::ranges::any_of(program.info.buffers, uses_mapping) ||
std::ranges::any_of(program.info.images, uses_mapping);
const auto planning_only_handle = [&](const IR::Inst& handle) {
return !handle.Uses().empty() &&
std::ranges::all_of(handle.Uses(), [&](const IR::Use& use) {
@@ -141,6 +62,39 @@ void ValidateNativeProgram(const IR::Program& program, bool lds_storage) {
};
for (const auto* block: program.blocks) {
for (const auto& inst: *block) {
const auto op = inst.GetOpcode();
uses_flattened_srt |= op == IR::ValueOpcode::ReadConst;
if (IR::BufferAccessOf(op) != IR::BufferAccess::None ||
IR::SharedAccessOf(op) != IR::SharedAccess::None ||
IR::AddressOpcodeInfoOf(op).access != IR::AddressAccess::None) {
const auto index = inst.Flags<IR::MemoryFlags>().index;
if (index >= program.memory_info.size()) {
Fail(program, "typed shader contains invalid memory metadata");
}
const auto& memory = program.memory_info[index];
if (!memory.planning_only) {
uses_lds |= memory.kind == IR::ResourceKind::FlatLocal;
if (IR::SharedAccessOf(op) != IR::SharedAccess::None) {
if (memory.kind != IR::ResourceKind::Lds && memory.kind != IR::ResourceKind::Gds) {
Fail(program, "typed shader contains invalid shared-memory metadata");
}
uses_gds |= memory.kind == IR::ResourceKind::Gds;
uses_lds |= memory.kind == IR::ResourceKind::Lds;
} else if (memory.kind == IR::ResourceKind::Buffer ||
memory.kind == IR::ResourceKind::ScalarBuffer) {
if (memory.resource >= program.info.buffers.size()) {
Fail(program, "typed shader contains an invalid buffer resource");
}
live_buffers |= uint64_t{1} << memory.resource;
for (const auto child: program.info.buffers[memory.resource].indirect_resources) {
if (child >= program.info.buffers.size()) {
Fail(program, "typed shader contains an invalid indirect buffer resource");
}
live_buffers |= uint64_t{1} << child;
}
}
}
}
const auto dense = inst.Flags<uint32_t>();
switch (inst.GetOpcode()) {
case IR::ValueOpcode::GetBufferResource:
@@ -186,163 +140,64 @@ void ValidateNativeProgram(const IR::Program& program, bool lds_storage) {
}
}
}
using Kind = IR::DescriptorBindingKind;
std::array<uint32_t, static_cast<size_t>(Kind::Count)> expected {};
auto& buffer_count = expected[static_cast<size_t>(Kind::Buffers)];
for (uint32_t index = 0; index < program.info.buffers.size(); ++index) {
const auto slot = (live_buffers & (uint64_t{1} << index)) != 0 ? buffer_count++ : UINT32_MAX;
if (program.info.buffers[index].descriptor_index != slot) {
Fail(program, "native buffer descriptor index does not match shader topology");
}
}
for (const auto& image: program.info.images) {
const auto kind = IR::DescriptorBindingForImage(image);
if (!kind.has_value()) {
Fail(program, "native shader plan has an invalid image class");
}
if (image.mip_count == 0u ||
(image.mip_mode != IR::ImageMipMode::Dynamic && image.mip_count != 1u)) {
Fail(program, "native shader plan has an invalid image mip descriptor count");
}
auto& count = expected[static_cast<size_t>(*kind)];
if (image.descriptor_index != count) {
Fail(program, "native image descriptor index does not match shader topology");
}
count += image.mip_count;
}
expected[static_cast<size_t>(Kind::Samplers)] = static_cast<uint32_t>(program.info.samplers.size());
expected[static_cast<size_t>(Kind::Gds)] = uses_gds;
expected[static_cast<size_t>(Kind::SharedMemory)] = uses_lds && lds_storage;
expected[static_cast<size_t>(Kind::BdaPagetable)] = program.info.uses_dma;
expected[static_cast<size_t>(Kind::FaultBuffer)] = program.info.uses_dma;
expected[static_cast<size_t>(Kind::FlattenedSrt)] = uses_flattened_srt;
expected[static_cast<size_t>(Kind::ShaderData)] =
program.bindings.ShaderDataDwords() != 0 && !program.bindings.UsesPushData();
uint64_t descriptor_mask = 0;
for (uint32_t index = 0; index < expected.size(); ++index) {
if (expected[index] != 0) descriptor_mask |= uint64_t{1} << index;
}
if (program.bindings.descriptor_counts != expected ||
program.bindings.descriptor_mask != descriptor_mask) {
Fail(program, "native descriptor counts do not match shader topology");
}
const auto shader_data_dwords = program.bindings.ShaderDataDwords();
const auto& registers = program.info.user_data_registers;
const bool has_dispatch_threads = program.bindings.dispatch_thread_dword != IR::PushData::NoStart;
if ((program.bindings.UsesPushData() &&
!IR::PushData::CanFit(program.bindings.push_data_start_dword, shader_data_dwords)) ||
(has_dispatch_threads && (program.stage != ShaderType::Compute ||
program.bindings.dispatch_thread_dword != registers.size())) ||
program.bindings.memory_offset_dword != registers.size() + (has_dispatch_threads ? 3u : 0u) ||
!std::is_sorted(registers.begin(), registers.end()) ||
std::adjacent_find(registers.begin(), registers.end()) != registers.end()) {
Fail(program, "native shader-data layout is inconsistent");
}
}
} // namespace
Emitter::SpirvRequirements Emitter::AnalyzeProgramRequirements(const IR::Program& program) {
SpirvRequirements requirements {};
for (const auto* block: program.blocks) {
for (const auto& inst: *block) {
requirements.float64 |= inst.GetType() == IR::Type::F64;
if (IR::BufferAccessOf(inst.GetOpcode()) == IR::BufferAccess::Atomic &&
inst.GetType() == IR::Type::U64) {
requirements.buffer_int64_atomics = true;
}
const auto address_access = IR::AddressOpcodeInfoOf(inst.GetOpcode()).access;
if (address_access != IR::AddressAccess::None) {
const auto memory_index = inst.Flags<IR::MemoryFlags>().index;
if (memory_index >= program.memory_info.size()) {
Fail(program, "address operation has invalid memory metadata");
}
const auto& memory = program.memory_info[memory_index];
const auto kind = memory.kind;
if (kind == IR::ResourceKind::Scratch || kind == IR::ResourceKind::FlatLocal) {
if (program.scratch_dwords == 0) {
Fail(program, "scratch operation has no per-thread storage");
}
requirements.function_scratch = true;
if (kind == IR::ResourceKind::FlatLocal && program.stage != ShaderType::Compute &&
program.stage != ShaderType::Mesh) {
requirements.function_lds = true;
}
} else if (address_access == IR::AddressAccess::Write) {
Fail(program, "writable FLAT/GLOBAL addresses require GPU ownership tracking");
} else {
// Descriptor stores can alias coherent physical-address loads.
requirements.coherent_buffers |= memory.coherent;
}
}
if (IR::BufferAccessOf(inst.GetOpcode()) != IR::BufferAccess::None) {
const auto memory_index = inst.Flags<IR::MemoryFlags>().index;
if (memory_index >= program.memory_info.size()) {
Fail(program, "buffer operation has invalid memory metadata");
}
const auto& memory = program.memory_info[memory_index];
if (memory.kind == IR::ResourceKind::IndirectBuffer &&
inst.GetOpcode() != IR::ValueOpcode::ReadConstBuffer) {
requirements.subgroup_local_invocation_id = true;
}
if (memory.kind == IR::ResourceKind::Buffer) {
requirements.coherent_buffers |= memory.coherent;
if (memory.resource >= program.info.buffers.size()) {
Fail(program, "buffer operation has invalid resource metadata");
}
const auto bits = StorageBufferElementBits(program, memory);
requirements.buffer_u8 |= bits == 8u;
requirements.buffer_u16 |= bits == 16u;
if ((program.info.buffers[memory.resource].packed_stride & (1u << 20u)) != 0u) {
if (program.stage != ShaderType::Compute) {
Fail(program, "buffer ADD_TID is only valid for compute shaders");
}
requirements.subgroup_local_invocation_id = true;
}
}
}
const auto shared_access = IR::SharedAccessOf(inst.GetOpcode());
if (shared_access != IR::SharedAccess::None) {
const auto index = inst.Flags<IR::MemoryFlags>().index;
if (index >= program.memory_info.size()) {
Fail(program, "shared operation has invalid memory metadata");
}
const auto kind = program.memory_info[index].kind;
if (kind != IR::ResourceKind::Lds && kind != IR::ResourceKind::Gds) {
Fail(program, "shared operation has invalid resource kind");
}
if (shared_access == IR::SharedAccess::Atomic &&
IR::SharedComponentCount(inst.GetOpcode()) == 2u) {
if (kind != IR::ResourceKind::Lds || program.stage != ShaderType::Compute) {
Fail(program, "64-bit shared atomics require compute LDS");
}
requirements.shared_int64_atomics = true;
}
if (program.stage != ShaderType::Compute && program.stage != ShaderType::Mesh &&
kind == IR::ResourceKind::Lds) {
requirements.function_lds = true;
}
if (shared_access == IR::SharedAccess::Append ||
shared_access == IR::SharedAccess::Consume) {
requirements.subgroup_ballot = true;
requirements.subgroup_shuffle = true;
requirements.subgroup_local_invocation_id = true;
}
}
switch (inst.GetOpcode()) {
case IR::ValueOpcode::StoreCompletion: requirements.subgroup_barrier = true; break;
case IR::ValueOpcode::BvhIntersect: requirements.bvh = true; break;
case IR::ValueOpcode::ConditionRef:
requirements.subgroup_ballot |=
inst.Flags<CFG::BranchCondition>() != CFG::BranchCondition::ScalarInstruction;
break;
case IR::ValueOpcode::Ballot: requirements.subgroup_ballot = true; break;
case IR::ValueOpcode::DppMoveU32:
case IR::ValueOpcode::ReadFirstLane:
case IR::ValueOpcode::ReadLane: {
requirements.subgroup_ballot = true;
requirements.subgroup_shuffle = true;
if (inst.GetOpcode() == IR::ValueOpcode::DppMoveU32) {
requirements.subgroup_local_invocation_id = true;
}
break;
}
case IR::ValueOpcode::DppUpdateU32:
case IR::ValueOpcode::WriteLane: {
requirements.subgroup_ballot = true;
requirements.subgroup_local_invocation_id = true;
break;
}
case IR::ValueOpcode::Permlane16U32: {
requirements.subgroup_ballot = true;
requirements.subgroup_shuffle = true;
requirements.subgroup_local_invocation_id = true;
break;
}
case IR::ValueOpcode::SwizzleU32:
case IR::ValueOpcode::PermuteU32:
case IR::ValueOpcode::BpermuteU32: {
requirements.subgroup_ballot = true;
requirements.subgroup_shuffle = true;
requirements.subgroup_local_invocation_id = true;
break;
}
case IR::ValueOpcode::LaneId:
requirements.subgroup_local_invocation_id |=
program.stage != ShaderType::TessellationControl;
break;
case IR::ValueOpcode::ImageQueryLod: requirements.compute_derivatives = true; break;
case IR::ValueOpcode::ImageGatherRaw:
requirements.image_gather_extended = true;
break;
case IR::ValueOpcode::SetAttribute: {
const auto index = inst.Flags<IR::ExportFlags>().index;
if (index >= program.export_info.size()) {
Fail(program, "attribute export has invalid metadata");
}
if (program.stage == ShaderType::Pixel &&
program.export_info[index].vm) {
requirements.pixel_valid_mask = true;
}
break;
}
default: break;
}
}
}
return requirements;
}
std::vector<uint32_t> EmitProgram(const IR::Program& program,
ShaderStageInputInfo input_info) {
std::vector<uint32_t> EmitProgram(IR::Program& program, ShaderStageInputInfo input_info,
uint32_t push_data_start_dword) {
using namespace Emitter;
if (program.stage != ShaderType::Compute && program.stage != ShaderType::Vertex &&
@@ -352,12 +207,14 @@ std::vector<uint32_t> EmitProgram(const IR::Program& program,
Fail(program, "binary SPIR-V emitter received an unsupported shader stage");
}
if (!program.srt_plan_complete || !program.resource_tracking_complete ||
!program.shader_info_complete || !program.binding_layout_complete) {
!program.shader_info_complete) {
Fail(program, "SPIR-V emitter requires a fully planned native shader program");
}
ValidateNativeProgram(program, program.stage == ShaderType::Compute &&
input_info.compute != nullptr && input_info.compute->lds_storage);
if (program.info.buffers.size() > IR::ShaderInfo::MaxBuffers) {
Fail(program, "native shader exceeds the buffer resource limit");
}
IR::ValidateProgram(program, true);
program.bindings = {.push_data_start_dword = push_data_start_dword};
EmitterState state(program, input_info);
const auto* workgroup = ShaderWorkgroupInput(program.stage, input_info);
state.lane_count =
@@ -365,6 +222,8 @@ std::vector<uint32_t> EmitProgram(const IR::Program& program,
? 2u
: 1u;
DefineModule(state);
ValidateNativeProgram(program, program.stage == ShaderType::Compute &&
input_info.compute != nullptr && input_info.compute->lds_storage);
EmitProgram(state);
state.builder.AddEntryPoint(ExecutionModelForStage(state.program.stage), state.main_func,
"main", state.interface_variables);
@@ -9,8 +9,9 @@
namespace Libs::Graphics::ShaderRecompiler::Spirv {
std::vector<uint32_t> EmitProgram(const IR::Program& program,
ShaderStageInputInfo input_info);
// Resets and writes backend IDs and descriptor indices; do not emit the same program concurrently.
std::vector<uint32_t> EmitProgram(IR::Program& program, ShaderStageInputInfo input_info,
uint32_t push_data_start_dword = 0);
} // namespace Libs::Graphics::ShaderRecompiler::Spirv
@@ -56,33 +56,14 @@ uint32_t EmitMulHigh(EmitterState& state, uint32_t lhs, uint32_t rhs, bool signe
return signed_value ? Unary(state, spv::OpBitcast, TypeU32(state), high) : high;
}
uint32_t EmitMinMax3(EmitterState& state, uint32_t a, uint32_t b, uint32_t c, bool signed_value,
bool max_value) {
const auto ab = signed_value ? EmitMinMaxI32Value(state, a, b, max_value)
: EmitMinMaxU32Value(state, a, b, max_value);
return signed_value ? EmitMinMaxI32Value(state, ab, c, max_value)
: EmitMinMaxU32Value(state, ab, c, max_value);
}
uint32_t EmitMed3(EmitterState& state, uint32_t a, uint32_t b, uint32_t c, bool signed_value) {
const auto minimum = EmitMinMax3(state, a, b, c, signed_value, false);
const auto maximum = EmitMinMax3(state, a, b, c, signed_value, true);
const auto ab = Binary(state, spv::OpIAdd, TypeU32(state), a, b);
const auto abc = Binary(state, spv::OpIAdd, TypeU32(state), ab, c);
return Binary(state, spv::OpISub, TypeU32(state),
Binary(state, spv::OpISub, TypeU32(state), abc, minimum), maximum);
}
uint32_t EmitFMinMax3(EmitterState& state, uint32_t a, uint32_t b, uint32_t c, bool max_value) {
return EmitMinMaxF32Value(state, EmitMinMaxF32Value(state, a, b, max_value), c, max_value);
}
uint32_t EmitExt(EmitterState& state, uint32_t type, uint32_t opcode,
std::initializer_list<uint32_t> args) {
const auto result = state.builder.AllocateId();
std::vector<uint32_t> words {spv::OpExtInst, type, result, GlslStd450(state), opcode};
words.insert(words.end(), args.begin(), args.end());
state.builder.AddFunction(words);
const auto result = state.builder.AllocateId();
state.builder.AddFunction(spv::OpExtInst, type, result, GlslStd450(state), opcode, args);
return result;
}
@@ -248,14 +229,6 @@ uint32_t EmitUMulHi(EmitterState& state, uint32_t arg0, uint32_t arg1) {
return EmitMulHigh(state, arg0, arg1, false);
}
uint32_t EmitIAbs32(EmitterState& state, uint32_t arg0) {
const auto value = arg0;
const auto neg = Unary(state, spv::OpSNegate, TypeU32(state), value);
const auto negative =
Binary(state, spv::OpSLessThan, TypeBool(state), value, ConstantU32(state, 0));
return Select(state, TypeU32(state), negative, neg, value);
}
uint32_t EmitBitCount64(EmitterState& state, uint32_t arg0) {
// Vulkan bit-count operands remain 32-bit, including when counting a U64.
const auto unpacked = Unary(state, spv::OpBitcast, TypeU32Vector(state, 2), arg0);
@@ -264,58 +237,30 @@ uint32_t EmitBitCount64(EmitterState& state, uint32_t arg0) {
return Binary(state, spv::OpIAdd, TypeU32(state), pair.low, pair.high);
}
uint32_t EmitFindILsb32(EmitterState& state, uint32_t arg0) {
const auto value = EmitExt(state, TypeI32(state), GLSLstd450FindILsb, {arg0});
return Unary(state, spv::OpBitcast, TypeU32(state), value);
}
uint32_t EmitFindUMsb32(EmitterState& state, uint32_t arg0) {
const auto value = EmitExt(state, TypeI32(state), GLSLstd450FindUMsb, {arg0});
return Unary(state, spv::OpBitcast, TypeU32(state), value);
}
uint32_t EmitSMin32(EmitterState& state, uint32_t arg0, uint32_t arg1) {
return EmitMinMaxI32Value(state, arg0, arg1, false);
}
uint32_t EmitSMax32(EmitterState& state, uint32_t arg0, uint32_t arg1) {
return EmitMinMaxI32Value(state, arg0, arg1, true);
}
uint32_t EmitUMin32(EmitterState& state, uint32_t arg0, uint32_t arg1) {
return EmitMinMaxU32Value(state, arg0, arg1, false);
}
uint32_t EmitUMax32(EmitterState& state, uint32_t arg0, uint32_t arg1) {
return EmitMinMaxU32Value(state, arg0, arg1, true);
}
uint32_t EmitSMinTri32(EmitterState& state, uint32_t arg0, uint32_t arg1, uint32_t arg2) {
return EmitMinMax3(state, arg0, arg1, arg2, true, false);
return EmitSMin32(state, arg0, EmitSMin32(state, arg1, arg2));
}
uint32_t EmitSMaxTri32(EmitterState& state, uint32_t arg0, uint32_t arg1, uint32_t arg2) {
return EmitMinMax3(state, arg0, arg1, arg2, true, true);
return EmitSMax32(state, arg0, EmitSMax32(state, arg1, arg2));
}
uint32_t EmitUMinTri32(EmitterState& state, uint32_t arg0, uint32_t arg1, uint32_t arg2) {
return EmitMinMax3(state, arg0, arg1, arg2, false, false);
return EmitUMin32(state, arg0, EmitUMin32(state, arg1, arg2));
}
uint32_t EmitUMaxTri32(EmitterState& state, uint32_t arg0, uint32_t arg1, uint32_t arg2) {
return EmitMinMax3(state, arg0, arg1, arg2, false, true);
return EmitUMax32(state, arg0, EmitUMax32(state, arg1, arg2));
}
uint32_t EmitSMedTri32(EmitterState& state, uint32_t arg0, uint32_t arg1, uint32_t arg2) {
return EmitMed3(state, arg0, arg1, arg2, true);
const auto high_min = EmitSMin32(state, EmitSMax32(state, arg0, arg1), arg2);
return EmitSMax32(state, EmitSMin32(state, arg0, arg1), high_min);
}
uint32_t EmitUMedTri32(EmitterState& state, uint32_t arg0, uint32_t arg1, uint32_t arg2) {
return EmitMed3(state, arg0, arg1, arg2, false);
}
uint32_t EmitFPIsNan32(EmitterState& state, uint32_t arg0) {
return EmitNative<spv::OpFUnordNotEqual, IR::Type::U1>(state, arg0, arg0);
const auto high_min = EmitUMin32(state, EmitUMax32(state, arg0, arg1), arg2);
return EmitUMax32(state, EmitUMin32(state, arg0, arg1), high_min);
}
uint32_t EmitFPMin32(EmitterState& state, uint32_t arg0, uint32_t arg1) {
@@ -82,35 +82,12 @@ uint32_t EmitF32ToF16RtzBits(EmitterState& state, uint32_t f32) {
return EmitAndConstant(state, EmitSelectValueU32(state, exp_eq_255, special, finite2), 0xffffu);
}
uint32_t EmitMinMaxU32Value(EmitterState& state, uint32_t lhs, uint32_t rhs, bool max_value) {
const auto cond = state.builder.AllocateId();
const auto ret = state.builder.AllocateId();
state.builder.AddFunction(max_value ? spv::OpUGreaterThan : spv::OpULessThan, TypeBool(state),
cond, lhs, rhs);
state.builder.AddFunction(spv::OpSelect, TypeU32(state), ret, cond, lhs, rhs);
return ret;
}
uint32_t EmitMinMaxI32Value(EmitterState& state, uint32_t lhs, uint32_t rhs, bool max_value) {
const auto cond = state.builder.AllocateId();
const auto ret = state.builder.AllocateId();
state.builder.AddFunction(max_value ? spv::OpSGreaterThan : spv::OpSLessThan, TypeBool(state),
cond, lhs, rhs);
state.builder.AddFunction(spv::OpSelect, TypeU32(state), ret, cond, lhs, rhs);
return ret;
}
F32Class EmitClassifyF32Bits(EmitterState& state, uint32_t bits) {
F32Class cls;
cls.bits = bits;
const auto abs_bits = EmitAndConstant(state, cls.bits, 0x7fffffffu);
const auto exponent_bits = EmitAndConstant(state, abs_bits, 0x7f800000u);
const auto mantissa_bits = EmitAndConstant(state, abs_bits, 0x007fffffu);
const auto exponent_max =
EmitCompareU32Constant(state, spv::OpIEqual, exponent_bits, 0x7f800000u);
const auto mantissa_nonzero = EmitCompareU32Constant(state, spv::OpINotEqual, mantissa_bits, 0);
cls.nan = EmitLogicalAndBool(state, exponent_max, mantissa_nonzero);
cls.zero = EmitCompareU32Constant(state, spv::OpIEqual, abs_bits, 0);
cls.bits = bits;
const auto abs_bits = EmitAndConstant(state, cls.bits, 0x7fffffffu);
cls.nan = EmitCompareU32Constant(state, spv::OpUGreaterThan, abs_bits, 0x7f800000u);
cls.zero = EmitCompareU32Constant(state, spv::OpIEqual, abs_bits, 0);
return cls;
}
@@ -59,21 +59,6 @@ uint32_t ConstantU32(EmitterState& state, uint32_t value);
std::abort();
}
uint32_t ResourceForDescriptor(const EmitterState& state, IR::DescriptorBindingKind kind,
uint32_t resource) {
const auto* descriptor = IR::FindBinding(state.program.bindings, kind);
if (descriptor == nullptr) {
ExitDescriptorBindingFailure(state, kind, resource, "descriptor group was not allocated");
}
const auto found =
std::find(descriptor->resources.begin(), descriptor->resources.end(), resource);
if (found == descriptor->resources.end()) {
ExitDescriptorBindingFailure(state, kind, resource,
"resource is absent from descriptor group");
}
return static_cast<uint32_t>(found - descriptor->resources.begin());
}
uint32_t DescriptorElementPointer(EmitterState& state, uint32_t result_ptr_type,
uint32_t variable_id, uint32_t array_index,
IR::DescriptorBindingKind kind, uint32_t resource,
@@ -152,43 +137,58 @@ uint32_t ImageViewSizeType(EmitterState& state, ImageDimension dimension) {
}
}
namespace {
void DecorateNonUniformDescriptor(EmitterState& state, uint32_t id) {
state.builder.RequireExtension("SPV_EXT_descriptor_indexing");
state.builder.RequireCapability(spv::CapabilityShaderNonUniform);
state.builder.RequireCapability(spv::CapabilitySampledImageArrayNonUniformIndexing);
state.builder.AddAnnotation(spv::OpDecorate, id, spv::DecorationNonUniform);
}
} // namespace
uint32_t LoadImageDescriptor(EmitterState& state, uint32_t resource, uint32_t mip,
uint32_t array_index) {
const auto pointer = ImageDescriptorPointer(state, resource, mip, array_index);
const auto image = state.builder.AllocateId();
state.builder.AddFunction(spv::OpLoad, ImageType(state, state.program.info.images.at(resource)),
const auto kind = *IR::DescriptorBindingForImage(state.program.info.images[resource]);
state.builder.AddFunction(spv::OpLoad, state.images[IR::ImageBindingIndex(kind)].type,
image, pointer);
if (array_index != 0u) {
DecorateNonUniformDescriptor(state, image);
}
return image;
}
uint32_t LoadSamplerDescriptor(EmitterState& state, uint32_t sampler) {
const auto array_index =
ResourceForDescriptor(state, IR::DescriptorBindingKind::Samplers, sampler);
const auto sampler_type = state.builder.Type(spv::OpTypeSampler);
const auto pointer_type =
state.builder.Type(spv::OpTypePointer, spv::StorageClassUniformConstant, sampler_type);
uint32_t LoadSamplerDescriptor(EmitterState& state, uint32_t sampler, uint32_t array_index) {
if (sampler >= state.program.info.samplers.size()) {
ExitDescriptorBindingFailure(state, IR::DescriptorBindingKind::Samplers, sampler,
"sampler resource index is out of range");
}
const auto pointer = DescriptorElementPointer(
state, pointer_type, state.sampler_variable, ConstantU32(state, array_index),
state, state.sampler_pointer_type, state.sampler_variable,
array_index == 0u ? ConstantU32(state, sampler) : array_index,
IR::DescriptorBindingKind::Samplers, sampler, "sampler descriptor array was not emitted");
const auto sampler_id = state.builder.AllocateId();
state.builder.AddFunction(spv::OpLoad, sampler_type, sampler_id, pointer);
state.builder.AddFunction(spv::OpLoad, state.sampler_type, sampler_id, pointer);
if (array_index != 0u) DecorateNonUniformDescriptor(state, sampler_id);
return sampler_id;
}
uint32_t MakeSampledImage(EmitterState& state, uint32_t resource, uint32_t sampler_id, uint32_t mip,
uint32_t array_index) {
uint32_t array_index, bool sampler_dynamic) {
const auto& image_resource = state.program.info.images.at(resource);
EXIT_IF(image_resource.resource_class != IR::ImageResourceClass::Sampled);
const auto image = LoadImageDescriptor(state, resource, mip, array_index);
const auto sampled_image = state.builder.AllocateId();
const auto sampled_type =
state.builder.Type(spv::OpTypeSampledImage, ImageType(state, image_resource));
const auto kind = *IR::DescriptorBindingForImage(image_resource);
auto& sampled_type = state.images[IR::ImageBindingIndex(kind)].sampled_type;
if (sampled_type == 0) {
sampled_type = state.builder.Type(
spv::OpTypeSampledImage, state.images[IR::ImageBindingIndex(kind)].type);
}
state.builder.AddFunction(spv::OpSampledImage, sampled_type, sampled_image, image, sampler_id);
if (array_index != 0u) {
state.builder.RequireExtension("SPV_EXT_descriptor_indexing");
state.builder.RequireCapability(spv::CapabilityShaderNonUniform);
state.builder.RequireCapability(spv::CapabilitySampledImageArrayNonUniformIndexing);
state.builder.AddAnnotation(spv::OpDecorate, sampled_image, spv::DecorationNonUniform);
if (array_index != 0u || sampler_dynamic) {
DecorateNonUniformDescriptor(state, sampled_image);
}
return sampled_image;
}
@@ -200,12 +200,11 @@ uint32_t ImageDescriptorPointer(EmitterState& state, uint32_t resource, uint32_t
const auto kind = IR::DescriptorBindingForImage(image);
EXIT_IF(!kind.has_value());
if (array_index == 0u) {
array_index = ConstantU32(state, ResourceForDescriptor(state, *kind, resource) + mip);
array_index = ConstantU32(state, image.descriptor_index + mip);
}
const auto pointer_type = state.builder.Type(
spv::OpTypePointer, spv::StorageClassUniformConstant, ImageType(state, image));
const auto variable = state.image_variables[IR::ImageBindingIndex(*kind)];
return DescriptorElementPointer(state, pointer_type, variable, array_index, *kind, resource,
const auto& definition = state.images[IR::ImageBindingIndex(*kind)];
return DescriptorElementPointer(state, definition.pointer_type, definition.variable,
array_index, *kind, resource,
"image descriptor array was not emitted");
}
@@ -223,7 +223,7 @@ uint32_t Boxes(EmitterState& s, uint32_t first, uint32_t second, uint32_t half,
} // namespace
void DefineBvhIntersect(EmitterState& s) {
if (!s.requirements.bvh) return;
if (!s.program.info.bvh) return;
const auto u = TypeU32(s), f = TypeF32(s), b = TypeBool(s), wide = TypeU64(s);
const auto vec4 = TypeU32Vector(s, 4), vec3 = TypeF32Vector(s, 3);
const auto signature = s.builder.Type(spv::OpTypeFunction, vec4, vec4, wide, f, vec3, vec3, vec3);
@@ -10,7 +10,7 @@ namespace {
bool UserDataDwordIndex(const EmitterState& state, IR::ScalarReg reg, uint32_t& dword_index) {
const auto register_index = IR::RegIndex(reg);
const auto& registers = state.program.bindings.user_data_registers;
const auto& registers = state.program.info.user_data_registers;
const auto found = std::lower_bound(registers.begin(), registers.end(), register_index);
if (found == registers.end() || *found != register_index) {
return false;
@@ -177,6 +177,13 @@ uint32_t EmitAttribute(EmitterState& state, uint32_t attr, uint32_t chan) {
return value;
};
if (input->per_vertex) {
if (PixelParameterIsFlat(state, attr)) {
const auto value = load_per_vertex(
state.input_info.pixel->parameter_mode == ShaderPixelParameterMode::LastVertex ? 2u : 0u);
const auto bits = state.builder.AllocateId();
state.builder.AddFunction(spv::OpBitcast, TypeU32(state), bits, value);
return bits;
}
const auto barycentric_kind = state.input_info.pixel->ps_no_perspective
? IR::StageInputKind::BaryCoordNoPerspective
: IR::StageInputKind::BaryCoordSmooth;
@@ -231,8 +238,12 @@ uint32_t EmitInterpolationParameter(ValueEmitContext& ctx, uint32_t attr, uint32
return value;
};
const auto selected_vertex = (mode + 1u) % 3u;
uint32_t value = load_vertex(selected_vertex);
uint32_t selected_vertex = (mode + 1u) % 3u;
if (mode == 2u && PixelParameterIsFlat(state, attr)) {
selected_vertex =
state.input_info.pixel->parameter_mode == ShaderPixelParameterMode::LastVertex ? 2u : 0u;
}
uint32_t value = load_vertex(selected_vertex);
if (!PixelParameterIsCustom(state, attr) && mode < 2u) {
const auto delta = state.builder.AllocateId();
state.builder.AddFunction(spv::OpFSub, TypeF32(state), delta, value, load_vertex(0));
@@ -293,50 +304,46 @@ uint32_t ExportVector(ValueEmitContext& ctx, uint32_t data, const IR::ExportInfo
f32[0], f32[1], f32[2], f32[3]);
return vector;
}
uint32_t raw[4] = {
ConstantU32(state, 0),
ConstantU32(state, 0),
ConstantU32(state, 0),
ConstantU32(state, uint_output ? 1u : 0x3f800000u),
};
if (exp.compr) {
for (uint32_t pair = 0; pair < 2u; pair++) {
if ((exp.en & (3u << (pair * 2u))) == 0u) {
continue;
}
const auto packed = ExportRawComponent(ctx, data, pair);
for (uint32_t lane = 0; lane < 2u; lane++) {
const auto component = pair * 2u + lane;
if (((exp.en >> component) & 1u) == 0u) {
if (exp.compr || exp.en != 0xfu) {
uint32_t raw[4] = {
ConstantU32(state, 0),
ConstantU32(state, 0),
ConstantU32(state, 0),
ConstantU32(state, uint_output ? 1u : 0x3f800000u),
};
if (exp.compr) {
for (uint32_t pair = 0; pair < 2u; pair++) {
if ((exp.en & (3u << (pair * 2u))) == 0u) {
continue;
}
raw[component] = state.builder.AllocateId();
state.builder.AddFunction(spv::OpBitFieldUExtract, TypeU32(state), raw[component],
packed, ConstantU32(state, lane * 16u),
ConstantU32(state, 16));
const auto packed = ExportRawComponent(ctx, data, pair);
for (uint32_t lane = 0; lane < 2u; lane++) {
const auto component = pair * 2u + lane;
if (((exp.en >> component) & 1u) == 0u) {
continue;
}
raw[component] = state.builder.AllocateId();
state.builder.AddFunction(spv::OpBitFieldUExtract, TypeU32(state), raw[component],
packed, ConstantU32(state, lane * 16u),
ConstantU32(state, 16));
}
}
}
} else {
for (uint32_t component = 0; component < 4u; component++) {
if (((exp.en >> component) & 1u) != 0u) {
raw[component] = ExportRawComponent(ctx, data, component);
} else {
for (uint32_t component = 0; component < 4u; component++) {
if (((exp.en >> component) & 1u) != 0u) {
raw[component] = ExportRawComponent(ctx, data, component);
}
}
}
data = state.builder.AllocateId();
state.builder.AddFunction(spv::OpCompositeConstruct, TypeU32Vector(state, 4), data,
raw[0], raw[1], raw[2], raw[3]);
}
if (uint_output) {
const auto vector = state.builder.AllocateId();
state.builder.AddFunction(spv::OpCompositeConstruct, TypeU32Vector(state, 4), vector,
raw[0], raw[1], raw[2], raw[3]);
return vector;
}
uint32_t f32[4] {};
for (uint32_t component = 0; component < 4u; component++) {
f32[component] = state.builder.AllocateId();
state.builder.AddFunction(spv::OpBitcast, TypeF32(state), f32[component], raw[component]);
return data;
}
const auto vector = state.builder.AllocateId();
state.builder.AddFunction(spv::OpCompositeConstruct, TypeF32Vector(state, 4), vector, f32[0],
f32[1], f32[2], f32[3]);
state.builder.AddFunction(spv::OpBitcast, TypeF32Vector(state, 4), vector, data);
return vector;
}
@@ -461,7 +468,7 @@ void EmitSetAttribute(ValueEmitContext& ctx, const IR::Inst& inst) {
auto& state = ctx.state;
const auto& exp = ctx.Export(inst);
const auto exec = ctx.Arg(inst, 1);
if (state.program.stage == ShaderType::Pixel && exp.vm && state.requirements.pixel_valid_mask &&
if (state.program.stage == ShaderType::Pixel && exp.vm && state.program.info.pixel_valid_mask &&
state.pixel_valid_mask_variable != 0) {
const auto value = state.builder.AllocateId();
state.builder.AddFunction(spv::OpSelect, TypeU32(state), value, exec, ConstantU32(state, 1),
@@ -115,6 +115,16 @@ DppTargetLane EmitDppTargetLane(EmitterState& state, const IR::DppMoveFlags& fla
if (control == 0x141u) {
return EmitDppMirrorTargetLane(state, subid, true);
}
// row_share:N reads lane N of the caller's row of 16.
if (control >= 0x150u && control <= 0x15fu) {
const auto row = state.builder.AllocateId();
const auto target = state.builder.AllocateId();
state.builder.AddFunction(spv::OpBitwiseAnd, TypeU32(state), row, subid,
ConstantU32(state, 0xfffffff0u));
state.builder.AddFunction(spv::OpBitwiseOr, TypeU32(state), target, row,
ConstantU32(state, control & 0xfu));
return {target, ConstantBool(state, true)};
}
if (control >= 0x160u && control <= 0x16fu) {
const auto target = state.builder.AllocateId();
state.builder.AddFunction(spv::OpBitwiseXor, TypeU32(state), target, subid,
@@ -615,6 +615,90 @@ uint32_t StoreTexel(ValueEmitContext& ctx, const IR::MemoryInfo& mem, uint32_t d
return PackImageTexel(ctx, mem, texel);
}
template <typename Emit>
uint32_t EmitImageAccess(ValueEmitContext& ctx, const IR::Inst& inst, uint32_t result_type,
const Emit& emit) {
auto& state = ctx.state;
const auto& mem = ctx.Memory(inst);
const auto& image = state.program.info.images.at(mem.resource);
if (image.indirect_root != mem.resource) return emit(mem.resource, 0u);
const auto* handle = inst.Arg(0).ResolveInstruction();
const auto* source = image.source < state.program.descriptor_sources.size()
? &state.program.descriptor_sources[image.source]
: nullptr;
if (handle == nullptr || source == nullptr || !source->indirect_descriptor.has_value() ||
handle->NumArgs() == 0u) {
ctx.Fail(inst, "has invalid indirect image key provenance");
}
const auto key = ctx.Def(handle->Arg(0));
if (state.flattened_srt_variable == 0 ||
image.indirect_resources.size() < 2u) {
ctx.Fail(inst, "has no indirect image runtime mapping");
}
const auto selected = EmitIndirectResourceIndex(
state, key, image.indirect_mapping_offset, image.indirect_search_iterations, 0u);
struct ImageRun {
uint32_t first;
uint32_t count;
uint32_t resource;
uint32_t slot_bias;
};
std::vector<ImageRun> runs;
for (uint32_t ordinal = 0; ordinal < image.indirect_resources.size(); ++ordinal) {
const auto resource = image.indirect_resources[ordinal];
const auto& candidate = state.program.info.images[resource];
const auto kind = *IR::DescriptorBindingForImage(candidate);
if (!runs.empty()) {
auto& run = runs.back();
const auto& first = state.program.info.images[run.resource];
const auto next_slot = run.slot_bias + ordinal;
if (candidate.dimension == first.dimension && candidate.cube == first.cube &&
IR::DescriptorBindingForImage(first) == kind &&
candidate.mip_count == 1u && first.mip_count == 1u &&
(ordinal == 1u || candidate.descriptor_index == next_slot)) {
// Native roots precede appended children; only the root may have a slot gap.
if (ordinal == 1u)
run.slot_bias = candidate.descriptor_index - ordinal;
++run.count;
continue;
}
}
runs.push_back({ordinal, 1u, resource,
candidate.descriptor_index - ordinal});
}
const auto EmitRun = [&](uint32_t index) {
const auto& run = runs[index];
if (run.count == 1u) return emit(run.resource, 0u);
auto slot = Binary(state, spv::OpIAdd, TypeU32(state), selected,
ConstantU32(state, run.slot_bias));
if (run.first == 0u) {
const auto root_slot = image.descriptor_index;
if (root_slot != run.slot_bias) {
const auto is_root = Binary(state, spv::OpIEqual, TypeBool(state), selected,
ConstantU32(state, 0u));
const auto root_or_child = state.builder.AllocateId();
state.builder.AddFunction(spv::OpSelect, TypeU32(state), root_or_child,
is_root, ConstantU32(state, root_slot), slot);
slot = root_or_child;
}
}
return emit(run.resource, slot);
};
auto run_index = ConstantU32(state, 0u);
for (uint32_t index = 1; index < runs.size(); ++index) {
const auto at_or_after = Binary(state, spv::OpUGreaterThanEqual, TypeBool(state),
selected, ConstantU32(state, runs[index].first));
const auto next = state.builder.AllocateId();
state.builder.AddFunction(spv::OpSelect, TypeU32(state), next, at_or_after,
ConstantU32(state, index), run_index);
run_index = next;
}
return runs.size() == 1u
? EmitRun(0u)
: EmitIndexSwitch(state, run_index, static_cast<uint32_t>(runs.size()),
result_type, EmitRun);
}
} // namespace
void EmitImage(ValueEmitContext& ctx, const IR::Inst& inst) {
@@ -656,29 +740,35 @@ void EmitImage(ValueEmitContext& ctx, const IR::Inst& inst) {
return;
}
if (op == IR::ValueOpcode::ImageRead) {
const auto dimension = image.dimension;
const auto& dimension_info = ImageDimensionInfoFor(dimension);
const auto numeric_class = image.numeric_class;
const auto condition = ctx.Arg(inst, 2);
const auto numeric_class = image.numeric_class;
const auto condition = ctx.Arg(inst, 2);
ctx.Define(
inst,
EmitValueOrDefaultIfCondition(
state, condition, TypeU32Vector(state, 4), ConstantU32CompositeZero(state, 4),
[&]() {
const auto descriptor = LoadImageDescriptor(state, mem.resource);
const auto color = state.builder.AllocateId();
const auto coord = CoordU32(ctx, mem, *address, dimension);
if (dimension_info.multisampled != 0u) {
state.builder.AddFunction(
spv::OpImageFetch, ImageVectorType(state, numeric_class, 4), color,
descriptor, coord, spv::ImageOperandsSampleMask,
AddressU32(ctx, mem, *address, dimension_info.coordinate_components));
} else {
state.builder.AddFunction(spv::OpImageFetch,
ImageVectorType(state, numeric_class, 4), color,
descriptor, coord, spv::ImageOperandsLodMask,
LodU32(ctx, mem, *address, dimension));
}
const auto color = EmitImageAccess(
ctx, inst, ImageVectorType(state, numeric_class, 4),
[&](uint32_t resource, uint32_t array_index) {
const auto& candidate = state.program.info.images[resource];
const auto dimension = candidate.dimension;
const auto& dimension_info = ImageDimensionInfoFor(dimension);
const auto descriptor = LoadImageDescriptor(state, resource, 0u, array_index);
const auto fetched = state.builder.AllocateId();
const auto coord = CoordU32(ctx, mem, *address, dimension);
if (dimension_info.multisampled != 0u) {
state.builder.AddFunction(
spv::OpImageFetch, ImageVectorType(state, candidate.numeric_class, 4), fetched,
descriptor, coord, spv::ImageOperandsSampleMask,
AddressU32(ctx, mem, *address, dimension_info.coordinate_components));
} else {
state.builder.AddFunction(
spv::OpImageFetch, ImageVectorType(state, candidate.numeric_class, 4), fetched,
descriptor, coord, spv::ImageOperandsLodMask,
LodU32(ctx, mem, *address, mem.image_dimension));
}
return fetched;
});
return ResultVector(ctx, UnpackImageTexel(ctx, mem, color), numeric_class,
false, mem);
}));
@@ -752,14 +842,11 @@ void EmitImage(ValueEmitContext& ctx, const IR::Inst& inst) {
const auto EmitGather = [&](uint32_t mip) {
const auto sampled = MakeSampledImage(state, mem.resource, sampler_id, mip);
const auto sample = state.builder.AllocateId();
std::vector<uint32_t> words {
static_cast<uint32_t>(dref ? spv::OpImageDrefGather : spv::OpImageGather), result_type,
sample, sampled, coord, component_or_dref};
if (operand_mask != 0u) {
words.push_back(operand_mask);
words.push_back(offset);
}
state.builder.AddFunction(words);
const std::array operands {operand_mask, offset};
state.builder.AddFunction(
dref ? spv::OpImageDrefGather : spv::OpImageGather, result_type,
sample, sampled, coord, component_or_dref,
std::span(operands).first(operand_mask != 0u ? 2u : 0u));
return sample;
};
const auto sample = image.mip_mode == IR::ImageMipMode::Dynamic && image.mip_count > 1u
@@ -791,134 +878,52 @@ void EmitImage(ValueEmitContext& ctx, const IR::Inst& inst) {
dref_value = AddressF32(ctx, mem, *address, layout.dref);
}
}
uint32_t operand_mask = 0;
std::vector<uint32_t> operands;
std::array<uint32_t, 3> operands {};
size_t operand_count = 0;
if (HasFlag(mem, Decoder::ImageSampleFlagDerivative)) {
operand_mask |= spv::ImageOperandsGradMask;
operands.push_back(
CoordF32(ctx, mem, *address, layout.grad_x, dimension_info.spatial_components));
operands.push_back(
CoordF32(ctx, mem, *address, layout.grad_y, dimension_info.spatial_components));
operands[0] = spv::ImageOperandsGradMask;
operands[1] = CoordF32(ctx, mem, *address, layout.grad_x, dimension_info.spatial_components);
operands[2] = CoordF32(ctx, mem, *address, layout.grad_y, dimension_info.spatial_components);
operand_count = 3;
} else if (explicit_lod) {
operand_mask |= spv::ImageOperandsLodMask;
auto lod = ZeroF32(state);
if (HasFlag(mem, Decoder::ImageSampleFlagLod) && layout.lod != NoImageComponent) {
lod = AddressF32(ctx, mem, *address, layout.lod);
}
operands.push_back(lod);
operands[0] = spv::ImageOperandsLodMask;
operands[1] = lod;
operand_count = 2;
} else if (layout.bias != NoImageComponent) {
operand_mask |= spv::ImageOperandsBiasMask;
operands.push_back(AddressF32(ctx, mem, *address, layout.bias));
operands[0] = spv::ImageOperandsBiasMask;
operands[1] = AddressF32(ctx, mem, *address, layout.bias);
operand_count = 2;
}
const auto sampler_id = LoadSamplerDescriptor(state, mem.sampler);
const auto EmitSample = [&](uint32_t resource, uint32_t array_index = 0u) {
const auto& samplers = state.program.info.samplers.at(mem.sampler).indirect_resources;
uint32_t sampler_index = 0;
if (!samplers.empty()) {
const auto* handle = inst.Arg(1).ResolveInstruction();
if (handle == nullptr || handle->GetOpcode() != IR::ValueOpcode::GetSamplerResource)
ctx.Fail(inst, "has invalid finite sampler key provenance");
sampler_index = EmitIndexSwitch(
state, ctx.Def(handle->Arg(0)), static_cast<uint32_t>(samplers.size()),
TypeU32(state), [&](uint32_t ordinal) { return ConstantU32(state, samplers[ordinal]); });
}
const auto sampler_id = LoadSamplerDescriptor(state, mem.sampler, sampler_index);
const auto EmitSample = [&](uint32_t resource, uint32_t array_index) {
const auto& candidate = state.program.info.images[resource];
const auto coord =
CoordF32(ctx, mem, *address, layout.coord,
ImageDimensionInfoFor(candidate.dimension).coordinate_components,
candidate.cube);
const auto sampled = MakeSampledImage(state, resource, sampler_id, 0u, array_index);
const auto sample = state.builder.AllocateId();
std::vector<uint32_t> sample_operands {result_type, sample, sampled, coord};
if (dref) {
sample_operands.push_back(dref_value);
}
if (operand_mask != 0u) {
sample_operands.push_back(operand_mask);
sample_operands.insert(sample_operands.end(), operands.begin(), operands.end());
}
state.builder.AddFunction(opcode, sample_operands);
const auto sampled = MakeSampledImage(state, resource, sampler_id, 0u, array_index,
sampler_index != 0u);
const auto sample = state.builder.AllocateId();
state.builder.AddFunction(opcode, result_type, sample, sampled, coord,
std::span(&dref_value, dref ? 1u : 0u),
std::span<const uint32_t>(operands).first(operand_count));
return sample;
};
if (image.indirect_root != mem.resource) {
const auto sample = EmitSample(mem.resource);
auto result = sample;
if (!dref) {
result = UnpackImageTexel(ctx, mem, sample);
}
ctx.Define(inst, ResultVector(ctx, result, numeric_class, dref, mem));
return;
}
const auto* handle = image_arg.ResolveInstruction();
const auto* source = image.source < state.program.descriptor_sources.size()
? &state.program.descriptor_sources[image.source]
: nullptr;
if (handle == nullptr || source == nullptr || !source->indirect_descriptor.has_value() ||
handle->NumArgs() == 0u) {
ctx.Fail(inst, "has invalid indirect image key provenance");
return;
}
const auto key = ctx.Def(handle->Arg(0));
if (state.flattened_srt_variable == 0 ||
image.indirect_resources.size() < 2u) {
ctx.Fail(inst, "has no indirect image runtime mapping");
return;
}
const auto selected = EmitIndirectResourceIndex(
state, key, image.indirect_mapping_offset, image.indirect_search_iterations, 0u);
struct SampleRun {
uint32_t first;
uint32_t count;
uint32_t resource;
uint32_t slot_bias;
};
std::vector<SampleRun> runs;
for (uint32_t ordinal = 0; ordinal < image.indirect_resources.size(); ++ordinal) {
const auto resource = image.indirect_resources[ordinal];
const auto& candidate = state.program.info.images[resource];
const auto kind = *IR::DescriptorBindingForImage(candidate);
const auto& binding = *IR::FindBinding(state.program.bindings, kind);
if (!runs.empty()) {
auto& run = runs.back();
const auto& first = state.program.info.images[run.resource];
const auto next_slot = run.slot_bias + ordinal;
if (candidate.dimension == first.dimension && candidate.cube == first.cube &&
IR::DescriptorBindingForImage(first) == kind &&
candidate.mip_count == 1u && first.mip_count == 1u &&
(ordinal == 1u || (next_slot < binding.resources.size() &&
binding.resources[next_slot] == resource))) {
// Native roots precede appended children; only the root may have a slot gap.
if (ordinal == 1u)
run.slot_bias = ResourceForDescriptor(state, kind, resource) - ordinal;
++run.count;
continue;
}
}
runs.push_back({ordinal, 1u, resource,
ResourceForDescriptor(state, kind, resource) - ordinal});
}
const auto EmitRun = [&](uint32_t index) {
const auto& run = runs[index];
if (run.count == 1u) return EmitSample(run.resource);
auto slot = Binary(state, spv::OpIAdd, TypeU32(state), selected,
ConstantU32(state, run.slot_bias));
if (run.first == 0u) {
const auto kind = *IR::DescriptorBindingForImage(image);
const auto root_slot = ResourceForDescriptor(state, kind, mem.resource);
if (root_slot != run.slot_bias) {
const auto is_root = Binary(state, spv::OpIEqual, TypeBool(state), selected,
ConstantU32(state, 0u));
const auto root_or_child = state.builder.AllocateId();
state.builder.AddFunction(spv::OpSelect, TypeU32(state), root_or_child,
is_root, ConstantU32(state, root_slot), slot);
slot = root_or_child;
}
}
return EmitSample(run.resource, slot);
};
auto run_index = ConstantU32(state, 0u);
for (uint32_t index = 1; index < runs.size(); ++index) {
const auto at_or_after = Binary(state, spv::OpUGreaterThanEqual, TypeBool(state),
selected, ConstantU32(state, runs[index].first));
const auto next = state.builder.AllocateId();
state.builder.AddFunction(spv::OpSelect, TypeU32(state), next, at_or_after,
ConstantU32(state, index), run_index);
run_index = next;
}
auto result = runs.size() == 1u
? EmitRun(0u)
: EmitIndexSwitch(state, run_index, static_cast<uint32_t>(runs.size()),
result_type, EmitRun);
auto result = EmitImageAccess(ctx, inst, result_type, EmitSample);
if (!dref) {
result = UnpackImageTexel(ctx, mem, result);
}
@@ -23,13 +23,8 @@ uint32_t EmitConvertF16F32(EmitterState& state, uint32_t arg0);
inline constexpr auto EmitConvertF32F16 = EmitF16BitsToF32;
uint32_t EmitConvertS32F32(EmitterState& state, uint32_t arg0);
uint32_t EmitConvertU32F32(EmitterState& state, uint32_t arg0);
template <IR::Type type>
uint32_t EmitConvertSigned32ToFloat(EmitterState& state, uint32_t arg0) {
const auto signed_value = Unary(state, spv::OpBitcast, TypeI32(state), arg0);
return EmitNative<spv::OpConvertSToF, type>(state, signed_value);
}
inline constexpr auto EmitConvertF32S32 = EmitConvertSigned32ToFloat<IR::Type::F32>;
inline constexpr auto EmitConvertF64S32 = EmitConvertSigned32ToFloat<IR::Type::F64>;
EMIT_NATIVE(ConvertF32S32, OpConvertSToF, F32, uint32_t)
EMIT_NATIVE(ConvertF64S32, OpConvertSToF, F64, uint32_t)
uint32_t EmitConvertF32F64(EmitterState& state, uint32_t arg0);
uint32_t EmitConvertF64F32(EmitterState& state, uint32_t arg0);
EMIT_NATIVE(ConvertF32U32, OpConvertUToF, F32, uint32_t)
@@ -69,7 +64,7 @@ EMIT_NATIVE(IMul64, OpIMul, U64, uint32_t, uint32_t)
EMIT_NATIVE(IAddCarry32, OpIAddCarry, U32x2, uint32_t, uint32_t)
uint32_t EmitSMulHi(EmitterState& state, uint32_t arg0, uint32_t arg1);
uint32_t EmitUMulHi(EmitterState& state, uint32_t arg0, uint32_t arg1);
uint32_t EmitIAbs32(EmitterState& state, uint32_t arg0);
inline constexpr auto EmitIAbs32 = EmitGlsl<GLSLstd450SAbs, IR::Type::U32, uint32_t>;
EMIT_NATIVE(ShiftLeftLogical32, OpShiftLeftLogical, U32, uint32_t, uint32_t)
EMIT_NATIVE(ShiftRightLogical32, OpShiftRightLogical, U32, uint32_t, uint32_t)
EMIT_NATIVE(ShiftRightArithmetic32, OpShiftRightArithmetic, U32, uint32_t, uint32_t)
@@ -96,13 +91,13 @@ EMIT_NATIVE(BitwiseAnd64, OpBitwiseAnd, U64, uint32_t, uint32_t)
EMIT_NATIVE(BitReverse32, OpBitReverse, U32, uint32_t)
EMIT_NATIVE(BitCount32, OpBitCount, U32, uint32_t)
uint32_t EmitBitCount64(EmitterState& state, uint32_t arg0);
uint32_t EmitFindILsb32(EmitterState& state, uint32_t arg0);
uint32_t EmitFindUMsb32(EmitterState& state, uint32_t arg0);
inline constexpr auto EmitFindILsb32 = EmitGlsl<GLSLstd450FindILsb, IR::Type::U32, uint32_t>;
inline constexpr auto EmitFindUMsb32 = EmitGlsl<GLSLstd450FindUMsb, IR::Type::U32, uint32_t>;
uint32_t EmitFindUMsb64(EmitterState& state, uint32_t arg0);
uint32_t EmitSMin32(EmitterState& state, uint32_t arg0, uint32_t arg1);
uint32_t EmitSMax32(EmitterState& state, uint32_t arg0, uint32_t arg1);
uint32_t EmitUMin32(EmitterState& state, uint32_t arg0, uint32_t arg1);
uint32_t EmitUMax32(EmitterState& state, uint32_t arg0, uint32_t arg1);
inline constexpr auto EmitSMin32 = EmitGlsl<GLSLstd450SMin, IR::Type::U32, uint32_t, uint32_t>;
inline constexpr auto EmitSMax32 = EmitGlsl<GLSLstd450SMax, IR::Type::U32, uint32_t, uint32_t>;
inline constexpr auto EmitUMin32 = EmitGlsl<GLSLstd450UMin, IR::Type::U32, uint32_t, uint32_t>;
inline constexpr auto EmitUMax32 = EmitGlsl<GLSLstd450UMax, IR::Type::U32, uint32_t, uint32_t>;
uint32_t EmitSMinTri32(EmitterState& state, uint32_t arg0, uint32_t arg1, uint32_t arg2);
uint32_t EmitSMaxTri32(EmitterState& state, uint32_t arg0, uint32_t arg1, uint32_t arg2);
uint32_t EmitUMinTri32(EmitterState& state, uint32_t arg0, uint32_t arg1, uint32_t arg2);
@@ -151,6 +146,15 @@ inline constexpr auto EmitFPOrdEqual32 = EmitFloatCompare32<spv::OpFOrdEqual>;
EMIT_NATIVE(FPOrdEqual64, OpFOrdEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdLessThanEqual64, OpFOrdLessThanEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdGreaterThanEqual64, OpFOrdGreaterThanEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdNotEqual64, OpFOrdNotEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdLessThan64, OpFOrdLessThan, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPOrdGreaterThan64, OpFOrdGreaterThan, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordEqual64, OpFUnordEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordNotEqual64, OpFUnordNotEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordLessThan64, OpFUnordLessThan, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordGreaterThan64, OpFUnordGreaterThan, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordLessThanEqual64, OpFUnordLessThanEqual, U1, uint32_t, uint32_t)
EMIT_NATIVE(FPUnordGreaterThanEqual64, OpFUnordGreaterThanEqual, U1, uint32_t, uint32_t)
inline constexpr auto EmitFPUnordEqual32 = EmitFloatCompare32<spv::OpFUnordEqual>;
inline constexpr auto EmitFPOrdNotEqual32 = EmitFloatCompare32<spv::OpFOrdNotEqual>;
inline constexpr auto EmitFPUnordNotEqual32 = EmitFloatCompare32<spv::OpFUnordNotEqual>;
@@ -162,7 +166,8 @@ inline constexpr auto EmitFPOrdLessThanEqual32 = EmitFloatCompare32<spv::OpFOrdL
inline constexpr auto EmitFPUnordLessThanEqual32 = EmitFloatCompare32<spv::OpFUnordLessThanEqual>;
inline constexpr auto EmitFPOrdGreaterThanEqual32 = EmitFloatCompare32<spv::OpFOrdGreaterThanEqual>;
inline constexpr auto EmitFPUnordGreaterThanEqual32 = EmitFloatCompare32<spv::OpFUnordGreaterThanEqual>;
uint32_t EmitFPIsNan32(EmitterState& state, uint32_t arg0);
EMIT_NATIVE(FPIsNan32, OpIsNan, U1, uint32_t)
EMIT_NATIVE(FPIsNan64, OpIsNan, U1, uint32_t)
inline constexpr auto EmitFPCmpClass32 = EmitClassMaskF32;
inline constexpr auto EmitFPCmpClass16 = EmitClassMaskF16;
EMIT_NATIVE(FPAdd32, OpFAdd, F32, uint32_t, uint32_t)
@@ -6,7 +6,6 @@
#include "graphics/shader/recompiler/BufferFormat.h"
#include "graphics/shader/recompiler/backend/spirv/SpirvBuilder.h"
#include "graphics/shader/recompiler/ir/ShaderIR.h"
#include "graphics/shader/recompiler/ir/passes/BindingLayout.h"
#include "graphics/shader/recompiler/ir/passes/ResourceMaterialization.h"
#include <algorithm>
@@ -58,42 +57,36 @@ constexpr std::array<ImageDimensionInfo, 7> ImageDimensions {{
const ImageDimensionInfo& ImageDimensionInfoFor(ImageDimension dimension);
struct SpirvRequirements {
bool bvh = false;
bool subgroup_ballot = false;
bool subgroup_barrier = false;
bool subgroup_shuffle = false;
bool subgroup_local_invocation_id = false;
bool compute_derivatives = false;
bool image_gather_extended = false;
bool function_lds = false;
bool function_scratch = false;
bool pixel_valid_mask = false;
bool buffer_int64_atomics = false;
bool buffer_u8 = false;
bool buffer_u16 = false;
bool shared_int64_atomics = false;
bool coherent_buffers = false;
bool float64 = false;
struct BufferDefinition {
uint32_t variable = 0;
uint32_t type = 0;
uint32_t element_type = 0;
uint32_t pointer_type = 0;
uint32_t element_pointer_type = 0;
};
SpirvRequirements AnalyzeProgramRequirements(const IR::Program& program);
struct ImageDefinition {
uint32_t variable = 0;
uint32_t type = 0;
uint32_t pointer_type = 0;
uint32_t sampled_type = 0;
};
struct EmitterState {
EmitterState(const IR::Program& program_, ShaderStageInputInfo input_info_)
EmitterState(IR::Program& program_, ShaderStageInputInfo input_info_)
: builder(program_.stage == ShaderType::Mesh ? 0x00010400u : 0x00010300u),
program(program_), input_info(input_info_),
requirements(AnalyzeProgramRequirements(program_)) {
program(program_), input_info(input_info_) {
if (ShaderWorkgroupInput(program.stage, input_info) != nullptr) {
lds_storage_class = spv::StorageClassWorkgroup;
}
if (IR::FindBinding(program.bindings, IR::DescriptorBindingKind::SharedMemory) != nullptr) {
if (program.info.uses_lds && program.stage == ShaderType::Compute &&
input_info.compute != nullptr && input_info.compute->lds_storage) {
lds_storage_class = spv::StorageClassStorageBuffer;
}
}
Builder builder;
const IR::Program& program;
IR::Program& program;
ShaderStageInputInfo input_info;
uint32_t void_type = 0;
uint32_t bool_type = 0;
@@ -113,13 +106,9 @@ struct EmitterState {
uint32_t tess_inner_variable = 0;
uint32_t tess_patch_base = 0;
const SpirvRequirements requirements;
uint32_t lane_count = 1;
uint32_t lane_half = 0;
uint32_t storage_buffer_variable = 0;
uint32_t storage_buffer_u8_variable = 0;
uint32_t storage_buffer_u16_variable = 0;
uint32_t storage_buffer_u64_variable = 0;
std::array<BufferDefinition, 4> storage_buffers {};
std::array<uint32_t, IR::ShaderInfo::MaxBuffers> memory_byte_offsets {};
uint32_t bda_pagetable_variable = 0;
uint32_t fault_buffer_variable = 0;
@@ -135,8 +124,10 @@ struct EmitterState {
uint32_t lds_variable = 0;
uint32_t lds_u64_variable = 0;
std::array<uint32_t, 2> scratch_variable {};
std::array<uint32_t, IR::ImageBindingCount> image_variables {};
std::array<ImageDefinition, IR::ImageBindingCount> images {};
uint32_t sampler_variable = 0;
uint32_t sampler_type = 0;
uint32_t sampler_pointer_type = 0;
uint32_t main_func = 0;
uint32_t mesh_guest_func = 0;
uint32_t mesh_allocation = 0;
@@ -160,7 +151,6 @@ struct EmitterState {
std::vector<InputBinding> inputs;
std::vector<OutputBinding> outputs;
std::vector<uint32_t> interface_variables;
std::unordered_map<const IR::Block*, uint32_t> labels;
};
uint32_t TypeVoid(EmitterState& state);
@@ -180,8 +170,6 @@ uint32_t TypeI32Vector(EmitterState& state, uint32_t components);
uint32_t TypeF32Vector(EmitterState& state, uint32_t components);
uint32_t TypePointer(EmitterState& state, spv::StorageClass storage_class, uint32_t pointee);
uint32_t TypeFunction(EmitterState& state);
uint32_t TypeStorageBufferElement(EmitterState& state, uint32_t bits);
uint32_t TypeStorageBufferPointer(EmitterState& state, uint32_t bits = 32);
uint32_t TypeStorageBufferElementPointer(EmitterState& state, uint32_t bits = 32);
uint32_t TypePhysicalU32Pointer(EmitterState& state);
uint32_t TypePushConstantElementPointer(EmitterState& state);
@@ -255,12 +243,10 @@ struct ValueEmitContext {
const IR::Inst* ImageAddress(IR::Value value);
const IR::MemoryInfo& Memory(const IR::Inst& inst) const;
const IR::ExportInfo& Export(const IR::Inst& inst) const;
uint32_t Label(const IR::Block* block) const;
[[noreturn]] void Fail(const char* reason) const;
[[noreturn]] void Fail(const IR::Inst& inst, const char* reason) const;
EmitterState& state;
std::unordered_map<const IR::Inst*, uint32_t> definitions;
const std::unordered_map<const IR::Inst*, uint32_t>* dispatcher_spills = nullptr;
std::unordered_map<const IR::Inst*, std::pair<uint32_t, uint32_t>> dispatcher_block_loads;
uint32_t scratch_u32_variable = 0;
@@ -316,9 +302,6 @@ uint32_t EmitSubgroupLocalInvocationId(EmitterState& state);
IR::DescriptorBindingKind kind, uint32_t resource,
const char* reason);
uint32_t ResourceForDescriptor(const EmitterState& state, IR::DescriptorBindingKind kind,
uint32_t resource);
uint32_t DescriptorElementPointer(EmitterState& state, uint32_t result_ptr_type,
uint32_t variable_id, uint32_t array_index,
IR::DescriptorBindingKind kind, uint32_t resource,
@@ -339,10 +322,11 @@ uint32_t ImageDescriptorPointer(EmitterState& state, uint32_t resource, uint32_t
uint32_t LoadImageDescriptor(EmitterState& state, uint32_t resource, uint32_t mip = 0,
uint32_t array_index = 0);
uint32_t LoadSamplerDescriptor(EmitterState& state, uint32_t sampler);
uint32_t LoadSamplerDescriptor(EmitterState& state, uint32_t sampler, uint32_t array_index = 0);
uint32_t MakeSampledImage(EmitterState& state, uint32_t resource, uint32_t sampler_id,
uint32_t mip = 0, uint32_t array_index = 0);
uint32_t mip = 0, uint32_t array_index = 0,
bool sampler_dynamic = false);
void EmitStorageImageWrite(EmitterState& state, uint32_t resource, uint32_t mip_lod, uint32_t coord,
uint32_t texel);
@@ -413,7 +397,6 @@ uint32_t EmitShaderDataDwordLoad(EmitterState& state, uint32_t dword_index);
uint32_t StorageBufferPackedStride(const EmitterState& state, const IR::MemoryInfo& mem);
Prospero::BufferFormat StorageBufferFormat(const EmitterState& state, const IR::MemoryInfo& mem);
uint32_t StorageBufferElementBits(const IR::Program& program, const IR::MemoryInfo& mem);
void EmitMemoryOffsets(EmitterState& state);
@@ -435,7 +418,7 @@ MemoryResourceAccess PrepareMemoryResourceAccess(EmitterState& state, const IR::
MemoryResourceAccess PrepareStorageBufferResourceAccess(EmitterState& state,
const IR::MemoryInfo& mem,
uint32_t variable, uint32_t pointer_type);
BufferDefinition& buffer);
uint32_t EmitMemoryElementInBounds(EmitterState& state, const MemoryResourceAccess& access,
uint32_t index);
@@ -489,10 +472,6 @@ uint32_t EmitSubConstantMinusU32(EmitterState& state, uint32_t constant, uint32_
uint32_t EmitF32ToF16RtzBits(EmitterState& state, uint32_t f32);
uint32_t EmitMinMaxU32Value(EmitterState& state, uint32_t lhs, uint32_t rhs, bool max_value);
uint32_t EmitMinMaxI32Value(EmitterState& state, uint32_t lhs, uint32_t rhs, bool max_value);
inline constexpr auto EmitBitcastF32ToU32 = EmitNative<spv::OpBitcast, IR::Type::U32, uint32_t>;
inline constexpr auto EmitBitcastU32ToF32 = EmitNative<spv::OpBitcast, IR::Type::F32, uint32_t>;
@@ -251,8 +251,7 @@ uint32_t LoadBda(ValueEmitContext& ctx, uint32_t address, uint32_t active, uint3
uint32_t ByteAddress(ValueEmitContext& ctx, const IR::Inst& inst, const IR::MemoryInfo& mem) {
if (mem.kind == IR::ResourceKind::Buffer) {
const auto resource = ResourceForDescriptor(ctx.state, IR::DescriptorBindingKind::Buffers,
mem.resource);
const auto resource = ctx.state.program.info.buffers.at(mem.resource).descriptor_index;
const auto prefix = ctx.state.memory_byte_offsets[resource];
const auto address = Binary(ctx.state, spv::OpIAdd, TypeU32(ctx.state),
BufferByteAddress(ctx, inst, mem), prefix);
@@ -333,7 +332,8 @@ uint32_t LoadSubwordInBounds(ValueEmitContext& ctx, const MemoryResourceAccess&
uint32_t value;
if (resource.element_bits < 32u) {
const auto loaded = ctx.state.builder.AllocateId();
ctx.state.builder.AddFunction(spv::OpLoad, TypeStorageBufferElement(ctx.state, bits), loaded,
ctx.state.builder.AddFunction(spv::OpLoad,
ctx.state.storage_buffers[bits == 8u ? 0 : 1].element_type, loaded,
EmitMemoryElementPointer(ctx.state, resource, index),
resource.memory_access);
value = Unary(ctx.state, spv::OpUConvert, TypeU32(ctx.state), loaded);
@@ -438,7 +438,8 @@ void StoreSubwordInBounds(ValueEmitContext& ctx, const IR::MemoryInfo& mem,
if (resource.element_bits < 32u) {
ctx.state.builder.AddFunction(
spv::OpStore, pointer,
Unary(ctx.state, spv::OpUConvert, TypeStorageBufferElement(ctx.state, bits), data),
Unary(ctx.state, spv::OpUConvert,
ctx.state.storage_buffers[bits == 8u ? 0 : 1].element_type, data),
resource.memory_access);
return;
}
@@ -1200,8 +1201,8 @@ uint32_t EmitBufferAtomic64(ValueEmitContext& ctx, const IR::Inst& inst) {
auto& state = ctx.state;
return EmitValueOrDefaultIfCondition(
state, ctx.Arg(inst, inst.NumArgs() - 1), TypeU64(state), ConstantU64(state, 0), [&]() {
const auto resource = PrepareStorageBufferResourceAccess(
state, mem, state.storage_buffer_u64_variable, TypeStorageBufferPointer(state, 64));
const auto resource =
PrepareStorageBufferResourceAccess(state, mem, state.storage_buffers[3]);
const auto byte_address = ByteAddress(ctx, inst, mem);
const auto index = Binary(state, spv::OpShiftRightLogical, TypeU32(state), byte_address,
ConstantU32(state, 3u));
@@ -102,23 +102,15 @@ Prospero::BufferFormat StorageBufferFormat(const EmitterState& state, const IR::
return state.program.info.buffers[mem.resource].descriptor_format;
}
uint32_t StorageBufferElementBits(const IR::Program& program, const IR::MemoryInfo& mem) {
if (!mem.formatted) return mem.data_bits;
const auto format = mem.typed ? Format::DecodeTBufferFormat(mem.data_format, mem.number_format)
: program.info.buffers[mem.resource].descriptor_format;
const auto info = Format::GetFormatInfo(format);
return info.type == Format::ComponentType::Unknown || info.packed_bitfield
? 32u : info.component_bits[0];
}
void EmitMemoryOffsets(EmitterState& state) {
for (uint32_t i = 0; i < state.program.bindings.memory_offset_count; i++) {
const auto count = state.program.bindings.descriptor_counts[static_cast<size_t>(IR::DescriptorBindingKind::Buffers)];
for (uint32_t i = 0; i < count; i += 4u) {
const auto word =
EmitShaderDataDwordLoad(state, state.program.bindings.memory_offset_dword + i / 4u);
const auto shift = ConstantU32(state, (i % 4u) * 8u);
state.memory_byte_offsets[i] = EmitBinaryU32(
state, spv::OpBitwiseAnd, EmitBinaryU32(state, spv::OpShiftRightLogical, word, shift),
ConstantU32(state, 0xffu));
for (uint32_t lane = 0; lane < std::min(4u, count - i); ++lane) {
state.memory_byte_offsets[i + lane] = EmitNative<spv::OpBitFieldUExtract, IR::Type::U32>(
state, word, ConstantU32(state, lane * 8u), ConstantU32(state, 8u));
}
}
}
@@ -146,7 +138,7 @@ void EnsureLdsStorage(EmitterState& state) {
state.builder.AddAnnotation(spv::OpDecorate, variable, spv::DecorationAliased);
return variable;
};
if (state.requirements.shared_int64_atomics) {
if (state.program.info.shared_int64_atomics) {
state.lds_variable = define(TypeU32(state), 4u);
state.lds_u64_variable = define(TypeU64(state), 8u);
state.builder.AddName(state.lds_u64_variable, "lds_qwords");
@@ -160,20 +152,22 @@ void EnsureLdsStorage(EmitterState& state) {
MemoryResourceAccess PrepareStorageBufferResourceAccess(EmitterState& state,
const IR::MemoryInfo& mem,
uint32_t variable,
uint32_t pointer_type) {
if (variable == 0) {
BufferDefinition& buffer) {
if (buffer.variable == 0) {
ExitDescriptorBindingFailure(state, IR::DescriptorBindingKind::Buffers, mem.resource,
"storage buffer descriptor array was not emitted");
}
if (buffer.pointer_type == 0) {
buffer.pointer_type = TypePointer(state, spv::StorageClassStorageBuffer, buffer.type);
}
const auto array_index =
ResourceForDescriptor(state, IR::DescriptorBindingKind::Buffers, mem.resource);
state.program.info.buffers.at(mem.resource).descriptor_index;
MemoryResourceAccess access {
.kind = mem.kind,
.memory_access = mem.coherent ? spv::MemoryAccessVolatileMask : spv::MemoryAccessMaskNone};
access.object_pointer = state.builder.AllocateId();
state.builder.AddFunction(spv::OpAccessChain, pointer_type, access.object_pointer, variable,
ConstantU32(state, array_index));
state.builder.AddFunction(spv::OpAccessChain, buffer.pointer_type, access.object_pointer,
buffer.variable, ConstantU32(state, array_index));
access.byte_offset = state.memory_byte_offsets[array_index];
access.length = state.builder.AllocateId();
state.builder.AddFunction(spv::OpArrayLength, TypeU32(state), access.length,
@@ -214,12 +208,14 @@ MemoryResourceAccess PrepareMemoryResourceAccess(EmitterState& state, const IR::
case IR::ResourceKind::ScalarBuffer:
case IR::ResourceKind::Buffer: {
const auto bits = mem.kind == IR::ResourceKind::Buffer
? StorageBufferElementBits(state.program, mem) : 32u;
const auto variable = bits == 8u ? state.storage_buffer_u8_variable
: bits == 16u ? state.storage_buffer_u16_variable
: state.storage_buffer_variable;
access = PrepareStorageBufferResourceAccess(
state, mem, variable, TypeStorageBufferPointer(state, bits));
? IR::StorageBufferElementBits(state.program, mem) : 32u;
auto* buffer = &state.storage_buffers[2];
if (bits == 8u) {
buffer = &state.storage_buffers[0];
} else if (bits == 16u) {
buffer = &state.storage_buffers[1];
}
access = PrepareStorageBufferResourceAccess(state, mem, *buffer);
access.element_bits = bits;
return access;
}
@@ -245,7 +241,7 @@ uint32_t EmitMemoryElementPointer(EmitterState& state, const MemoryResourceAcces
TypeStorageBufferElementPointer(state, 32));
}
const auto pointer = state.builder.AllocateId();
if (access.kind == IR::ResourceKind::Lds && state.requirements.shared_int64_atomics) {
if (access.kind == IR::ResourceKind::Lds && state.program.info.shared_int64_atomics) {
state.builder.AddFunction(spv::OpAccessChain, TypeU32ElementPointer(state, storage_class),
pointer, access.object_pointer, ConstantU32(state, 0), index);
return pointer;
@@ -120,32 +120,39 @@ uint32_t TypeFunction(EmitterState& state) {
return state.function_type;
}
uint32_t TypeStorageBufferElement(EmitterState& state, uint32_t bits) {
return bits == 32u ? TypeU32(state)
: bits == 64u ? TypeU64(state)
: state.builder.Type(spv::OpTypeInt, bits, 0);
}
uint32_t StorageRuntimeArrayType(EmitterState& state, uint32_t bits) {
return state.builder.DecoratedType(
spv::OpTypeRuntimeArray,
{{spv::OpDecorate, {spv::DecorationArrayStride, bits / 8u}}},
TypeStorageBufferElement(state, bits));
}
uint32_t StorageBufferType(EmitterState& state, uint32_t bits = 32) {
return state.builder.DecoratedType(spv::OpTypeStruct,
{{spv::OpMemberDecorate, {0, spv::DecorationOffset, 0}},
{spv::OpDecorate, {spv::DecorationBlock}}},
StorageRuntimeArrayType(state, bits));
}
uint32_t TypeStorageBufferPointer(EmitterState& state, uint32_t bits) {
return TypePointer(state, spv::StorageClassStorageBuffer, StorageBufferType(state, bits));
uint32_t StorageBufferType(EmitterState& state, BufferDefinition& buffer, uint32_t bits = 32) {
if (buffer.type == 0) {
if (bits == 32u) {
buffer.element_type = TypeU32(state);
} else if (bits == 64u) {
buffer.element_type = TypeU64(state);
} else {
buffer.element_type = state.builder.Type(spv::OpTypeInt, bits, 0);
}
const auto array = state.builder.DecoratedType(
spv::OpTypeRuntimeArray,
{{spv::OpDecorate, {spv::DecorationArrayStride, bits / 8u}}}, buffer.element_type);
buffer.type = state.builder.DecoratedType(
spv::OpTypeStruct, {{spv::OpMemberDecorate, {0, spv::DecorationOffset, 0}},
{spv::OpDecorate, {spv::DecorationBlock}}}, array);
}
return buffer.type;
}
uint32_t TypeStorageBufferElementPointer(EmitterState& state, uint32_t bits) {
return TypePointer(state, spv::StorageClassStorageBuffer, TypeStorageBufferElement(state, bits));
auto* buffer = &state.storage_buffers[2];
if (bits == 8u) {
buffer = &state.storage_buffers[0];
} else if (bits == 16u) {
buffer = &state.storage_buffers[1];
} else if (bits == 64u) {
buffer = &state.storage_buffers[3];
}
if (buffer->element_pointer_type == 0) {
buffer->element_pointer_type =
TypePointer(state, spv::StorageClassStorageBuffer, buffer->element_type);
}
return buffer->element_pointer_type;
}
uint32_t TypePhysicalU32Pointer(EmitterState& state) {
@@ -200,6 +207,43 @@ uint32_t F32ArrayType(EmitterState& state, uint32_t count) {
}
void DefineDescriptors(EmitterState& state) {
using Kind = IR::DescriptorBindingKind;
auto& info = state.program.info;
auto& bindings = state.program.bindings;
auto& counts = bindings.descriptor_counts;
auto& buffer_count = counts[static_cast<size_t>(Kind::Buffers)];
for (uint32_t index = 0; index < info.buffers.size(); ++index) {
info.buffers[index].descriptor_index = (info.live_buffers & (uint64_t{1} << index)) != 0
? buffer_count++ : UINT32_MAX;
}
std::array<const IR::ImageResource*, IR::ImageBindingCount> image_types {};
for (auto& image: info.images) {
const auto kind = IR::DescriptorBindingForImage(image);
if (!kind.has_value() || image.mip_count == 0u ||
(image.mip_mode != IR::ImageMipMode::Dynamic && image.mip_count != 1u)) {
EXIT("shader image has an invalid descriptor type or mip count");
}
auto& count = counts[static_cast<size_t>(*kind)];
image.descriptor_index = count;
if (count == 0) image_types[IR::ImageBindingIndex(*kind)] = &image;
count += image.mip_count;
}
counts[static_cast<size_t>(Kind::Samplers)] = static_cast<uint32_t>(info.samplers.size());
counts[static_cast<size_t>(Kind::Gds)] = info.uses_gds;
counts[static_cast<size_t>(Kind::SharedMemory)] =
info.uses_lds && state.lds_storage_class == spv::StorageClassStorageBuffer;
counts[static_cast<size_t>(Kind::BdaPagetable)] = info.uses_dma;
counts[static_cast<size_t>(Kind::FaultBuffer)] = info.uses_dma;
counts[static_cast<size_t>(Kind::FlattenedSrt)] = info.uses_flattened_srt;
bindings.memory_offset_dword = static_cast<uint32_t>(info.user_data_registers.size());
if (info.uses_dispatch_threads) {
bindings.dispatch_thread_dword = bindings.memory_offset_dword;
bindings.memory_offset_dword += 3u;
}
bindings.push_data_start_dword =
IR::PushData::StartFor(bindings.push_data_start_dword, bindings.ShaderDataDwords());
counts[static_cast<size_t>(Kind::ShaderData)] =
bindings.ShaderDataDwords() != 0 && !bindings.UsesPushData();
if (state.program.bindings.UsesPushData() || state.program.stage == ShaderType::Mesh) {
const auto type = PushConstantBlockType(state);
state.push_constant_variable = state.builder.DefineGlobalVariable(
@@ -207,7 +251,10 @@ void DefineDescriptors(EmitterState& state) {
state.builder.AddName(type, "BufferResource");
state.builder.AddName(state.push_constant_variable, "vsharp");
}
for (const auto& binding: state.program.bindings.descriptors) {
for (uint32_t index = 0; index < counts.size(); ++index) {
if (counts[index] == 0) continue;
bindings.descriptor_mask |= uint64_t{1} << index;
const auto kind = static_cast<Kind>(index);
const auto Define = [&](uint32_t type, const char* name,
spv::StorageClass storage = spv::StorageClassStorageBuffer) {
const auto variable =
@@ -215,69 +262,66 @@ void DefineDescriptors(EmitterState& state) {
state.builder.AddName(variable, name);
state.builder.AddAnnotation(spv::OpDecorate, variable, spv::DecorationDescriptorSet, 0);
state.builder.AddAnnotation(spv::OpDecorate, variable, spv::DecorationBinding,
IR::NativeBinding(state.program.stage, binding.kind));
IR::NativeBinding(state.program.stage, kind));
return variable;
};
const auto ArrayType = [&](uint32_t type) {
return state.builder.Type(
spv::OpTypeArray, type,
ConstantU32(state, static_cast<uint32_t>(binding.resources.size())));
ConstantU32(state, counts[index]));
};
switch (binding.kind) {
case IR::DescriptorBindingKind::Buffers:
state.storage_buffer_variable =
Define(ArrayType(StorageBufferType(state)), "buffers");
if (state.requirements.buffer_u8) {
state.storage_buffer_u8_variable =
Define(ArrayType(StorageBufferType(state, 8)), "buffers_u8");
}
if (state.requirements.buffer_u16) {
state.storage_buffer_u16_variable =
Define(ArrayType(StorageBufferType(state, 16)), "buffers_u16");
}
if (state.requirements.buffer_int64_atomics) {
state.storage_buffer_u64_variable =
Define(ArrayType(StorageBufferType(state, 64)), "buffers_u64");
}
for (const auto variable: {state.storage_buffer_variable, state.storage_buffer_u8_variable,
state.storage_buffer_u16_variable, state.storage_buffer_u64_variable}) {
if (variable == 0) continue;
if (state.storage_buffer_u8_variable != 0 || state.storage_buffer_u16_variable != 0 ||
state.storage_buffer_u64_variable != 0) {
state.builder.AddAnnotation(spv::OpDecorate, variable, spv::DecorationAliased);
switch (kind) {
case IR::DescriptorBindingKind::Buffers: {
const std::array used {info.buffer_u8, info.buffer_u16, true, info.buffer_int64_atomics};
constexpr std::array names {"buffers_u8", "buffers_u16", "buffers", "buffers_u64"};
for (uint32_t alias = 0; alias < used.size(); ++alias) {
if (!used[alias]) continue;
auto& buffer = state.storage_buffers[alias];
const auto type = StorageBufferType(state, buffer, 8u << alias);
buffer.variable = Define(ArrayType(type), names[alias]);
if (info.buffer_u8 || info.buffer_u16 || info.buffer_int64_atomics) {
state.builder.AddAnnotation(spv::OpDecorate, buffer.variable, spv::DecorationAliased);
}
// RDNA2 stores publish to L2 even without GLC; every alias of the buffer
// must participate in visibility for cache-bypassing polling loads.
if (state.requirements.coherent_buffers) {
state.builder.AddAnnotation(spv::OpDecorate, variable, spv::DecorationCoherent);
if (info.coherent_buffers) {
state.builder.AddAnnotation(spv::OpDecorate, buffer.variable, spv::DecorationCoherent);
}
}
break;
}
case IR::DescriptorBindingKind::BdaPagetable:
state.bda_pagetable_variable = Define(StorageBufferType(state, 64), "bda_pagetable");
state.bda_pagetable_variable =
Define(StorageBufferType(state, state.storage_buffers[3], 64), "bda_pagetable");
break;
case IR::DescriptorBindingKind::FaultBuffer:
state.fault_buffer_variable = Define(StorageBufferType(state), "fault_buffer");
state.fault_buffer_variable =
Define(StorageBufferType(state, state.storage_buffers[2]), "fault_buffer");
break;
case IR::DescriptorBindingKind::ShaderData:
state.shader_data_storage_variable =
Define(StorageBufferType(state), "shader_data");
Define(StorageBufferType(state, state.storage_buffers[2]), "shader_data");
break;
case IR::DescriptorBindingKind::FlattenedSrt:
state.flattened_srt_variable = Define(StorageBufferType(state), "flattened_srt");
state.flattened_srt_variable =
Define(StorageBufferType(state, state.storage_buffers[2]), "flattened_srt");
break;
case IR::DescriptorBindingKind::Samplers:
state.sampler_variable = Define(ArrayType(state.builder.Type(spv::OpTypeSampler)),
"samplers", spv::StorageClassUniformConstant);
state.sampler_type = state.builder.Type(spv::OpTypeSampler);
state.sampler_pointer_type = TypePointer(
state, spv::StorageClassUniformConstant, state.sampler_type);
state.sampler_variable = Define(ArrayType(state.sampler_type), "samplers",
spv::StorageClassUniformConstant);
break;
case IR::DescriptorBindingKind::Gds:
state.gds_variable = Define(StorageBufferType(state), "gds");
state.gds_variable = Define(StorageBufferType(state, state.storage_buffers[2]), "gds");
break;
case IR::DescriptorBindingKind::SharedMemory:
state.lds_variable = Define(StorageBufferType(state), "lds_dwords");
state.lds_variable = Define(StorageBufferType(state, state.storage_buffers[2]), "lds_dwords");
state.builder.AddAnnotation(spv::OpDecorate, state.lds_variable, spv::DecorationCoherent);
if (state.requirements.shared_int64_atomics) {
state.lds_u64_variable = Define(StorageBufferType(state, 64), "lds_qwords");
if (state.program.info.shared_int64_atomics) {
state.lds_u64_variable =
Define(StorageBufferType(state, state.storage_buffers[3], 64), "lds_qwords");
state.builder.AddAnnotation(spv::OpDecorate, state.lds_u64_variable, spv::DecorationCoherent);
for (const auto variable: {state.lds_variable, state.lds_u64_variable}) {
state.builder.AddAnnotation(spv::OpDecorate, variable, spv::DecorationAliased);
@@ -285,13 +329,16 @@ void DefineDescriptors(EmitterState& state) {
}
break;
default: {
EXIT_IF(IR::ImageBindingResourceClass(binding.kind) ==
EXIT_IF(IR::ImageBindingResourceClass(kind) ==
IR::ImageResourceClass::None);
const auto& image = state.program.info.images.at(binding.resources.front());
const auto name = "image_" + std::to_string(static_cast<uint32_t>(binding.kind));
state.image_variables[IR::ImageBindingIndex(binding.kind)] =
Define(ArrayType(ImageType(state, image)), name.c_str(),
spv::StorageClassUniformConstant);
const auto& image = *image_types[IR::ImageBindingIndex(kind)];
const auto name = "image_" + std::to_string(static_cast<uint32_t>(kind));
auto& definition = state.images[IR::ImageBindingIndex(kind)];
definition.type = ImageType(state, image);
definition.variable = Define(ArrayType(definition.type), name.c_str(),
spv::StorageClassUniformConstant);
definition.pointer_type = TypePointer(
state, spv::StorageClassUniformConstant, definition.type);
if (image.dimension == ImageDimension::Dim1D ||
image.dimension == ImageDimension::Dim1DArray) {
state.builder.RequireCapability(image.resource_class ==
@@ -335,11 +382,11 @@ uint32_t ConstantU64(EmitterState& state, uint64_t value) {
uint32_t ConstantU32CompositeZero(EmitterState& state, uint32_t components) {
EXIT_IF(components < 1u || components > 4u);
const auto zero = ConstantU32(state, 0);
const auto zero = ConstantU32(state, 0);
if (components == 1u) return zero;
std::vector<uint32_t> values(components, zero);
const std::array values {zero, zero, zero, zero};
return state.builder.Constant(spv::OpConstantComposite, TypeU32Composite(state, components),
values);
std::span(values).first(components));
}
uint32_t GlslStd450(EmitterState& state) {
@@ -437,7 +484,7 @@ void DefineInputs(EmitterState& state) {
state.inputs.push_back({{kind, 0, components, name}});
}
};
if (IR::FindBinding(state.program.bindings, IR::DescriptorBindingKind::SharedMemory) != nullptr) {
if (state.program.info.uses_lds && state.lds_storage_class == spv::StorageClassStorageBuffer) {
add_builtin(IR::StageInputKind::WorkgroupId, 3, "gl_WorkGroupID");
add_builtin(IR::StageInputKind::NumWorkgroups, 3, "gl_NumWorkGroups");
}
@@ -524,7 +571,7 @@ void DefineInputs(EmitterState& state) {
builtin);
}
}
if (state.requirements.subgroup_local_invocation_id) {
if (state.program.info.subgroup_local_invocation_id) {
const auto variable = DefineInterfaceVariable(state, TypeU32(state), spv::StorageClassInput,
"gl_SubgroupInvocationID");
state.subgroup_local_invocation_id_variable = variable;
@@ -647,10 +694,10 @@ void DefineModule(EmitterState& state) {
DefineOutputs(state);
DefineTessellationInterfaces(state);
DefineDescriptors(state);
if (state.requirements.function_lds) {
if (state.program.info.function_lds) {
state.lds_variable = state.builder.AllocateId();
}
if (state.requirements.function_scratch) {
if (state.program.info.function_scratch) {
for (uint32_t half = 0; half < state.lane_count; half++) {
state.scratch_variable[half] = state.builder.AllocateId();
}
@@ -675,11 +722,11 @@ void DefineModule(EmitterState& state) {
state.builder.RequireCapability(spv::CapabilityShader);
state.builder.RequireCapability(spv::CapabilitySignedZeroInfNanPreserve);
if (state.requirements.buffer_u8) {
if (state.program.info.buffer_u8) {
state.builder.RequireExtension("SPV_KHR_8bit_storage");
state.builder.RequireCapability(spv::CapabilityStorageBuffer8BitAccess);
}
if (state.requirements.buffer_u16) {
if (state.program.info.buffer_u16) {
state.builder.RequireCapability(spv::CapabilityStorageBuffer16BitAccess);
}
if (state.program.info.uses_dma) {
@@ -687,11 +734,11 @@ void DefineModule(EmitterState& state) {
state.builder.RequireCapability(spv::CapabilityPhysicalStorageBufferAddresses);
state.builder.RequireExtension("SPV_KHR_physical_storage_buffer");
}
if (state.requirements.buffer_int64_atomics || state.requirements.shared_int64_atomics) {
if (state.program.info.buffer_int64_atomics || state.program.info.shared_int64_atomics) {
state.builder.RequireCapability(spv::CapabilityInt64);
state.builder.RequireCapability(spv::CapabilityInt64Atomics);
}
if (state.requirements.shared_int64_atomics &&
if (state.program.info.shared_int64_atomics &&
state.lds_storage_class == spv::StorageClassWorkgroup) {
state.builder.RequireVersion(0x00010400u);
state.builder.RequireExtension("SPV_KHR_workgroup_memory_explicit_layout");
@@ -714,21 +761,21 @@ void DefineModule(EmitterState& state) {
if (InputVariableForKind(state, IR::StageInputKind::SampleId) != 0) {
state.builder.RequireCapability(spv::CapabilitySampleRateShading);
}
if (state.requirements.image_gather_extended) {
if (state.program.info.image_gather_extended) {
state.builder.RequireCapability(spv::CapabilityImageGatherExtended);
}
if (state.lane_count == 2 || state.requirements.subgroup_barrier ||
state.requirements.subgroup_ballot || state.requirements.subgroup_shuffle ||
state.requirements.subgroup_local_invocation_id) {
if (state.lane_count == 2 || state.program.info.subgroup_barrier ||
state.program.info.subgroup_ballot || state.program.info.subgroup_shuffle ||
state.program.info.subgroup_local_invocation_id) {
state.builder.RequireCapability(spv::CapabilityGroupNonUniform);
}
if (state.lane_count == 2 || state.requirements.subgroup_ballot) {
if (state.lane_count == 2 || state.program.info.subgroup_ballot) {
state.builder.RequireCapability(spv::CapabilityGroupNonUniformBallot);
}
if (state.requirements.subgroup_shuffle) {
if (state.program.info.subgroup_shuffle) {
state.builder.RequireCapability(spv::CapabilityGroupNonUniformShuffle);
}
if (state.requirements.compute_derivatives && state.program.stage == ShaderType::Compute) {
if (state.program.info.compute_derivatives && state.program.stage == ShaderType::Compute) {
state.builder.RequireCapability(spv::CapabilityComputeDerivativeGroupQuadsKHR);
state.builder.RequireExtension("SPV_KHR_compute_shader_derivatives");
}
@@ -751,7 +798,7 @@ void DefineModule(EmitterState& state) {
// contract prevents host compilers from treating synthesized IEEE values as finite.
state.builder.AddExecutionMode(state.main_func, spv::ExecutionModeSignedZeroInfNanPreserve,
32u);
if (state.requirements.float64) {
if (state.program.info.float64) {
EXIT_NOT_IMPLEMENTED(state.program.stage == ShaderType::Compute &&
state.input_info.compute->float_mode != 0xc0);
// Use native rounding for MODE=0xc0, consistent with ordinary FP32 arithmetic.
@@ -763,8 +810,8 @@ void DefineModule(EmitterState& state) {
// state.builder.AddExecutionMode(state.main_func, spv::ExecutionModeDenormPreserve, 64u);
}
if (const auto* cs = ShaderWorkgroupInput(state.program.stage, state.input_info)) {
uint32_t local_x = state.requirements.compute_derivatives ? 2u : 1u;
uint32_t local_y = state.requirements.compute_derivatives ? 2u : 1u;
uint32_t local_x = state.program.info.compute_derivatives ? 2u : 1u;
uint32_t local_y = state.program.info.compute_derivatives ? 2u : 1u;
uint32_t local_z = 1u;
local_x = cs->threads_num[0] != 0u ? cs->threads_num[0] : local_x;
local_y = cs->threads_num[1] != 0u ? cs->threads_num[1] : local_y;
@@ -772,7 +819,7 @@ void DefineModule(EmitterState& state) {
if (state.lane_count == 2) {
local_x = ((local_x * local_y * local_z + 63u) / 64u) * 32u;
local_y = local_z = 1u;
if (state.requirements.compute_derivatives) {
if (state.program.info.compute_derivatives) {
local_y = local_x / 2u;
local_x = 2u;
}
@@ -791,14 +838,14 @@ void DefineModule(EmitterState& state) {
state.builder.AddExecutionMode(state.main_func, spv::ExecutionModeEarlyFragmentTests);
}
}
if (state.requirements.compute_derivatives && state.program.stage == ShaderType::Compute) {
if (state.program.info.compute_derivatives && state.program.stage == ShaderType::Compute) {
state.builder.AddExecutionMode(state.main_func, spv::ExecutionModeDerivativeGroupQuadsKHR);
}
state.builder.AddName(state.main_func, "main");
if (state.requirements.function_lds) {
if (state.program.info.function_lds) {
state.builder.AddName(state.lds_variable, "lds_dwords");
}
if (state.requirements.function_scratch) {
if (state.program.info.function_scratch) {
for (uint32_t half = 0; half < state.lane_count; half++) {
state.builder.AddName(state.scratch_variable[half], "scratch_dwords");
}
@@ -83,63 +83,50 @@ void StoreDispatcherPhiEdge(ValueEmitContext& ctx, const DispatcherFunctionState
}
}
const IR::Block* TargetBlock(const IR::Program& program, uint32_t id) {
const auto found = std::ranges::find_if(
program.block_info, [&](const IR::BlockInfo& info) { return info.id == id; });
if (found == program.block_info.end()) {
return nullptr;
}
return program.blocks[static_cast<size_t>(found - program.block_info.begin())];
}
void EmitReturn(ValueEmitContext& ctx) {
EmitKillIfPixelValidMaskInactive(ctx.state);
ctx.state.builder.AddFunction(spv::OpReturn);
}
void EmitStructuredTerminator(ValueEmitContext& ctx, const IR::Block* block,
const IR::BlockInfo& info) {
const auto& program = ctx.state.program;
const auto& term = info.terminator;
void EmitStructuredTerminator(ValueEmitContext& ctx, const IR::Block* block) {
const auto& term = block->terminator;
const auto emit_merge = [&]() {
if (term.loop_header) {
const auto* merge = TargetBlock(program, term.merge_block);
const auto* cont = TargetBlock(program, term.continue_block);
const auto* merge = term.merge_block;
const auto* cont = term.continue_block;
if (merge != nullptr && cont != nullptr) {
ctx.state.builder.AddFunction(spv::OpLoopMerge, ctx.Label(merge), ctx.Label(cont),
spv::LoopControlMaskNone);
ctx.state.builder.AddFunction(spv::OpLoopMerge, merge->Definition(),
cont->Definition(), spv::LoopControlMaskNone);
}
} else if (term.kind == CFG::TerminatorKind::ConditionalBranch &&
term.merge_block != UINT32_MAX) {
if (const auto* merge = TargetBlock(program, term.merge_block); merge != nullptr) {
ctx.state.builder.AddFunction(spv::OpSelectionMerge, ctx.Label(merge),
spv::SelectionControlMaskNone);
}
term.merge_block != nullptr) {
ctx.state.builder.AddFunction(spv::OpSelectionMerge, term.merge_block->Definition(),
spv::SelectionControlMaskNone);
}
};
switch (term.kind) {
case CFG::TerminatorKind::Branch: {
const auto* target = TargetBlock(program, term.true_block);
const auto* target = term.true_block;
if (target == nullptr) {
EmitReturn(ctx);
return;
}
emit_merge();
ctx.state.builder.AddFunction(spv::OpBranch, ctx.Label(target));
ctx.state.builder.AddFunction(spv::OpBranch, target->Definition());
return;
}
case CFG::TerminatorKind::ConditionalBranch: {
const auto* true_block = TargetBlock(program, term.true_block);
const auto* false_block = TargetBlock(program, term.false_block);
if (true_block == nullptr || false_block == nullptr || info.condition.IsEmpty()) {
const auto* true_block = term.true_block;
const auto* false_block = term.false_block;
if (true_block == nullptr || false_block == nullptr || block->condition.IsEmpty()) {
EmitReturn(ctx);
return;
}
const auto condition = ctx.Def(info.condition);
const auto condition = ctx.Def(block->condition);
emit_merge();
ctx.state.builder.AddFunction(spv::OpBranchConditional, condition,
ctx.Label(true_block), ctx.Label(false_block));
true_block->Definition(), false_block->Definition());
return;
}
default: EmitReturn(ctx); return;
@@ -147,8 +134,7 @@ void EmitStructuredTerminator(ValueEmitContext& ctx, const IR::Block* block,
}
void EmitDispatcherTarget(ValueEmitContext& ctx, const DispatcherFunctionState& dispatcher,
const IR::Block* from, uint32_t target) {
const auto* block = TargetBlock(ctx.state.program, target);
const IR::Block* from, const IR::Block* block) {
if (block != nullptr) {
StoreDispatcherPhiEdge(ctx, dispatcher, from, block);
if (ctx.other_half != nullptr) {
@@ -158,41 +144,37 @@ void EmitDispatcherTarget(ValueEmitContext& ctx, const DispatcherFunctionState&
}
uint32_t EmitDispatcherNextPc(ValueEmitContext& ctx, const DispatcherFunctionState& dispatcher,
const IR::Block* block, const IR::BlockInfo& info) {
const auto& term = info.terminator;
const IR::Block* block) {
const auto& term = block->terminator;
switch (term.kind) {
case CFG::TerminatorKind::Branch:
EmitDispatcherTarget(ctx, dispatcher, block, term.true_block);
return ConstantU32(ctx.state, term.true_block);
return ConstantU32(ctx.state, term.true_block->id);
case CFG::TerminatorKind::ConditionalBranch: {
EmitDispatcherTarget(ctx, dispatcher, block, term.true_block);
EmitDispatcherTarget(ctx, dispatcher, block, term.false_block);
const auto selected = ctx.state.builder.AllocateId();
ctx.state.builder.AddFunction(
spv::OpSelect, TypeU32(ctx.state), selected, ctx.Def(info.condition),
ConstantU32(ctx.state, term.true_block), ConstantU32(ctx.state, term.false_block));
ctx.state.builder.AddFunction(spv::OpSelect, TypeU32(ctx.state), selected,
ctx.Def(block->condition),
ConstantU32(ctx.state, term.true_block->id),
ConstantU32(ctx.state, term.false_block->id));
return selected;
}
case CFG::TerminatorKind::IndirectBranch: {
for (const auto target: term.indirect_targets) {
for (const auto* target: block->ImmSuccessors()) {
EmitDispatcherTarget(ctx, dispatcher, block, target);
}
uint32_t selected = ConstantU32(ctx.state, UINT32_MAX);
if (!info.indirect_target.IsEmpty()) {
const auto selector = ctx.Def(info.indirect_target);
const auto& values = term.indirect_selector_code != UINT32_MAX
? term.indirect_selector_values
: term.indirect_target_pcs;
const auto& targets = term.indirect_selector_code != UINT32_MAX
? term.indirect_selector_targets
: term.indirect_targets;
for (size_t index = 0; index < std::min(values.size(), targets.size()); index++) {
if (!block->indirect_target.IsEmpty()) {
const auto selector = ctx.Def(block->indirect_target);
for (const auto& branch: term.cases) {
const auto match = ctx.state.builder.AllocateId();
const auto next = ctx.state.builder.AllocateId();
ctx.state.builder.AddFunction(spv::OpIEqual, TypeBool(ctx.state), match,
selector, ConstantU32(ctx.state, values[index]));
selector, ConstantU32(ctx.state, branch.value));
ctx.state.builder.AddFunction(spv::OpSelect, TypeU32(ctx.state), next, match,
ConstantU32(ctx.state, targets[index]), selected);
ConstantU32(ctx.state, branch.target->id),
selected);
selected = next;
}
}
@@ -267,7 +249,7 @@ void EmitStructuredInstruction(ValueEmitContext& ctx, StructuredFunctionState& s
}
for (size_t index = 0; index < inst.NumArgs(); index++) {
const auto* predecessor = inst.PhiBlock(index);
if (predecessor == nullptr || !ctx.state.labels.contains(predecessor)) {
if (predecessor == nullptr || predecessor->Definition() == 0) {
ctx.Fail(inst, "has a predecessor outside the structured function");
}
}
@@ -301,7 +283,7 @@ void EmitDispatcherInstruction(ValueEmitContext& ctx, const DispatcherFunctionSt
template <typename EmitInstruction>
void EmitBlock(ValueEmitContext& ctx, const IR::Block* block, EmitInstruction&& emit_instruction) {
ctx.state.current_block = block;
EmitLabel(ctx.state, ctx.Label(block));
EmitLabel(ctx.state, block->Definition());
bool emitted_non_phi = false;
for (const auto& inst: *block) {
if (inst.GetOpcode() == IR::ValueOpcode::Phi) {
@@ -342,14 +324,14 @@ void PatchStructuredPhis(ValueEmitContext& ctx, StructuredFunctionState& structu
void EmitStructuredFunction(ValueEmitContext& ctx) {
const auto& program = ctx.state.program;
StructuredFunctionState structured;
ctx.state.builder.AddFunction(spv::OpBranch, ctx.Label(program.blocks.front()));
ctx.state.builder.AddFunction(spv::OpBranch, program.blocks.front()->Definition());
for (size_t index = 0; index < program.blocks.size(); index++) {
const auto* block = program.blocks[index];
EmitBlock(ctx, block, [&](ValueEmitContext& lane, const IR::Inst& inst) {
EmitStructuredInstruction(lane, structured, inst);
});
structured.block_exit_labels.emplace(block, ctx.state.current_label);
EmitStructuredTerminator(ctx, block, program.block_info[index]);
EmitStructuredTerminator(ctx, block);
}
PatchStructuredPhis(ctx, structured);
}
@@ -357,12 +339,11 @@ void EmitStructuredFunction(ValueEmitContext& ctx) {
void EmitDispatcherFunction(ValueEmitContext& ctx, const DispatcherFunctionState& dispatcher) {
auto& state = ctx.state;
const auto* entry = state.program.blocks.front();
state.builder.AddFunction(spv::OpBranch, ctx.Label(entry));
state.builder.AddFunction(spv::OpBranch, entry->Definition());
EmitBlock(ctx, entry, [&](ValueEmitContext& lane, const IR::Inst& inst) {
EmitDispatcherInstruction(lane, dispatcher, inst);
});
const auto initial_pc =
EmitDispatcherNextPc(ctx, dispatcher, entry, state.program.block_info.front());
const auto initial_pc = EmitDispatcherNextPc(ctx, dispatcher, entry);
const auto initial_parent = state.current_label;
state.builder.AddFunction(spv::OpBranch, dispatcher.header_label);
@@ -384,8 +365,8 @@ void EmitDispatcherFunction(ValueEmitContext& ctx, const DispatcherFunctionState
spv::SelectionControlMaskNone);
std::vector<uint32_t> words {spv::OpSwitch, pc, dispatcher.after_switch_label};
for (size_t index = 1; index < state.program.blocks.size(); index++) {
words.push_back(state.program.block_info[index].id);
words.push_back(ctx.Label(state.program.blocks[index]));
words.push_back(state.program.blocks[index]->id);
words.push_back(state.program.blocks[index]->Definition());
}
state.builder.AddFunction(words);
std::vector<uint32_t> next_pc_words {spv::OpPhi, TypeU32(state), next_pc,
@@ -396,8 +377,7 @@ void EmitDispatcherFunction(ValueEmitContext& ctx, const DispatcherFunctionState
[&](ValueEmitContext& lane, const IR::Inst& inst) {
EmitDispatcherInstruction(lane, dispatcher, inst);
});
const auto selected = EmitDispatcherNextPc(ctx, dispatcher, state.program.blocks[index],
state.program.block_info[index]);
const auto selected = EmitDispatcherNextPc(ctx, dispatcher, state.program.blocks[index]);
next_pc_words.push_back(selected);
next_pc_words.push_back(state.current_label);
state.builder.AddFunction(spv::OpBranch, dispatcher.after_switch_label);
@@ -551,23 +531,23 @@ uint32_t ValueEmitContext::Shuffle(const IR::Inst& inst, size_t index, uint32_t
}
uint32_t ValueEmitContext::Result(const IR::Inst& inst) {
if (const auto found = definitions.find(&inst); found != definitions.end()) {
return found->second;
if (const auto id = inst.Definition(half); id != 0) {
return id;
}
const auto id = state.builder.AllocateId();
definitions.emplace(&inst, id);
inst.SetDefinition(half, id);
return id;
}
uint32_t ValueEmitContext::Define(const IR::Inst& inst, uint32_t value) {
if (const auto found = definitions.find(&inst); found != definitions.end()) {
if (found->second != value) {
if (const auto id = inst.Definition(half); id != 0) {
if (id != value) {
state.builder.AddFunction(spv::OpCopyObject, TypeId(state, inst.GetType()),
found->second, value);
id, value);
}
return found->second;
return id;
}
definitions.emplace(&inst, value);
inst.SetDefinition(half, value);
return value;
}
@@ -595,10 +575,6 @@ const IR::ExportInfo& ValueEmitContext::Export(const IR::Inst& inst) const {
return state.program.export_info.at(inst.Flags<IR::ExportFlags>().index);
}
uint32_t ValueEmitContext::Label(const IR::Block* block) const {
return state.labels.at(block);
}
[[noreturn]] void ValueEmitContext::Fail(const char* reason) const {
EXIT("SPIR-V emission failed: hash=0x%016" PRIx64 " stage=%u reason=%s\n",
state.program.shader_hash, static_cast<unsigned>(state.program.stage), reason);
@@ -622,13 +598,16 @@ void EmitProgram(EmitterState& state) {
high.half = 1;
}
std::optional<DispatcherFunctionState> dispatcher;
if (state.program.stage == ShaderType::Pixel && state.requirements.pixel_valid_mask) {
if (state.program.stage == ShaderType::Pixel && state.program.info.pixel_valid_mask) {
state.pixel_valid_mask_variable = state.builder.AllocateId();
state.builder.AddName(state.pixel_valid_mask_variable, "pixel_valid_mask_active");
}
for (const auto* block: program.blocks) {
const auto label = state.builder.AllocateId();
state.labels.emplace(block, label);
// Re-emission may use a different host subgroup width and ID allocation order.
for (const auto& inst: *block) {
inst.ResetDefinitions();
}
block->SetDefinition(state.builder.AllocateId());
}
if (state.program.dispatcher_fallback) {
auto& dispatch = dispatcher.emplace();
@@ -669,8 +648,8 @@ void EmitProgram(EmitterState& state) {
}
}
for (size_t index = 0; index < program.blocks.size(); index++) {
mark_cross_block(program.block_info[index].condition, program.blocks[index]);
mark_cross_block(program.block_info[index].indirect_target, program.blocks[index]);
mark_cross_block(program.blocks[index]->condition, program.blocks[index]);
mark_cross_block(program.blocks[index]->indirect_target, program.blocks[index]);
}
dispatch.header_label = state.builder.AllocateId();
dispatch.select_label = state.builder.AllocateId();
@@ -687,28 +666,23 @@ void EmitProgram(EmitterState& state) {
}
DefineGetBdaPointer(state);
DefineBvhIntersect(state);
for (const auto* block: program.blocks) {
if (std::ranges::any_of(*block, [](const IR::Inst& inst) {
return inst.GetOpcode() == IR::ValueOpcode::SwizzleU32;
})) {
ctx.scratch_u32_variable = state.builder.AllocateId();
if (state.lane_count == 2) {
high.scratch_u32_variable = state.builder.AllocateId();
}
break;
if (program.info.uses_swizzle) {
ctx.scratch_u32_variable = state.builder.AllocateId();
if (state.lane_count == 2) {
high.scratch_u32_variable = state.builder.AllocateId();
}
}
state.builder.AddFunction(spv::OpFunction, TypeVoid(state),
state.mesh_guest_func != 0 ? state.mesh_guest_func : state.main_func,
spv::FunctionControlMaskNone, TypeFunction(state));
EmitLabel(state, state.entry_label);
if (state.requirements.function_lds) {
if (state.program.info.function_lds) {
state.builder.AddFunction(
spv::OpVariable,
TypeU32ArrayPointer(state, spv::StorageClassFunction, LdsDwordCount(state)),
state.lds_variable, spv::StorageClassFunction);
}
if (state.requirements.function_scratch) {
if (state.program.info.function_scratch) {
for (uint32_t half = 0; half < state.lane_count; half++) {
state.builder.AddFunction(
spv::OpVariable,
@@ -8,7 +8,6 @@
#include <deque>
#include <list>
#include <map>
#include <set>
#include <span>
namespace Libs::Graphics::ShaderRecompiler::CFG {
@@ -568,9 +567,8 @@ bool ResolveSetpcTargets(const Decoder::Program& program, uint32_t setpc_index,
return false;
}
bool IsValidTarget(uint32_t target, const std::set<uint32_t>& instruction_pcs, uint32_t first_pc,
uint32_t end_pc) {
return target == end_pc || (target >= first_pc && instruction_pcs.contains(target));
bool IsValidTarget(uint32_t target, std::span<const Instruction> instructions, uint32_t end_pc) {
return target == end_pc || std::ranges::binary_search(instructions, target, {}, &Instruction::pc);
}
std::vector<uint32_t> AllBlockIds(uint32_t count) {
@@ -1455,9 +1453,7 @@ Graph BuildGraph(const Decoder::Program& program) {
const auto first_pc = program.instructions.front().pc;
const auto end_pc = ProgramEndPc(program);
std::set<uint32_t> instruction_pcs;
for (const auto& inst: program.instructions) {
instruction_pcs.insert(inst.pc);
if (inst.opcode == Opcode::UNSUPPORTED) {
ExitBuildFailure(
graph, FailureKind::UnsupportedInstruction, UINT32_MAX,
@@ -1466,23 +1462,22 @@ Graph BuildGraph(const Decoder::Program& program) {
}
}
std::set<uint32_t> labels;
labels.insert(first_pc);
labels.insert(end_pc);
std::vector<uint32_t> labels {first_pc, end_pc};
std::map<uint32_t, SetpcTargetInfo> setpc_targets;
for (uint32_t i = 0; i < program.instructions.size(); i++) {
const auto& inst = program.instructions[i];
const auto next_pc = InstructionEndPc(inst);
if (Decoder::IsDirectBranch(inst.opcode)) {
if (!IsValidTarget(inst.branch_target, instruction_pcs, first_pc, end_pc)) {
if (Decoder::IsDirectBranch(inst.opcode) ||
(inst.opcode == Opcode::S_SWAPPC_B64 && inst.branch_target != UINT32_MAX)) {
if (!IsValidTarget(inst.branch_target, program.instructions, end_pc)) {
ExitBuildFailure(graph, FailureKind::InvalidBranchTarget, UINT32_MAX,
fmt::format("branch at pc 0x{:08x} targets invalid pc 0x{:08x}",
inst.pc, inst.branch_target));
}
labels.insert(inst.branch_target);
labels.push_back(inst.branch_target);
if (next_pc <= end_pc) {
labels.insert(next_pc);
labels.push_back(next_pc);
}
} else if (inst.opcode == Opcode::S_SETPC_B64) {
SetpcTargetInfo target_info;
@@ -1495,30 +1490,32 @@ Graph BuildGraph(const Decoder::Program& program) {
? std::span<const uint32_t>(target_info.target_pcs)
: std::span<const uint32_t>(&target_info.target, 1);
for (const auto target: target_pcs) {
if (!IsValidTarget(target, instruction_pcs, first_pc, end_pc)) {
if (!IsValidTarget(target, program.instructions, end_pc)) {
ExitBuildFailure(
graph, FailureKind::InvalidBranchTarget, UINT32_MAX,
fmt::format("S_SETPC_B64 at pc 0x{:08x} targets invalid pc 0x{:08x}",
inst.pc, target));
}
labels.insert(target);
labels.push_back(target);
}
setpc_targets.emplace(inst.pc, std::move(target_info));
if (next_pc <= end_pc) {
labels.insert(next_pc);
labels.push_back(next_pc);
}
} else if (inst.opcode == Opcode::S_ENDPGM) {
labels.insert(next_pc);
} else if (inst.opcode == Opcode::S_ENDPGM || inst.opcode == Opcode::S_SWAPPC_B64) {
labels.push_back(next_pc);
}
}
std::vector<uint32_t> sorted_labels(labels.begin(), labels.end());
for (uint32_t i = 0; i < sorted_labels.size(); i++) {
const auto start = sorted_labels[i];
SortUnique(labels);
graph.blocks.reserve(labels.size());
for (uint32_t i = 0; i < labels.size(); i++) {
const auto start = labels[i];
if (start > end_pc) {
continue;
}
if (start != end_pc && !instruction_pcs.contains(start)) {
const auto begin = std::ranges::lower_bound(program.instructions, start, {}, &Instruction::pc);
if (start != end_pc && (begin == program.instructions.end() || begin->pc != start)) {
ExitBuildFailure(
graph, FailureKind::InvalidLabel, UINT32_MAX,
fmt::format("CFG label does not start on an instruction: 0x{:08x}", start));
@@ -1527,11 +1524,8 @@ Graph BuildGraph(const Decoder::Program& program) {
BasicBlock block;
block.id = static_cast<uint32_t>(graph.blocks.size());
block.start_pc = start;
block.end_pc = i + 1u < sorted_labels.size() ? sorted_labels[i + 1u] : end_pc;
block.inst_begin = static_cast<uint32_t>(
std::lower_bound(program.instructions.begin(), program.instructions.end(), start,
[](const Instruction& inst, uint32_t pc) { return inst.pc < pc; }) -
program.instructions.begin());
block.end_pc = i + 1u < labels.size() ? labels[i + 1u] : end_pc;
block.inst_begin = static_cast<uint32_t>(begin - program.instructions.begin());
block.inst_end = static_cast<uint32_t>(
std::lower_bound(program.instructions.begin(), program.instructions.end(), block.end_pc,
[](const Instruction& inst, uint32_t pc) { return inst.pc < pc; }) -
@@ -1541,10 +1535,10 @@ Graph BuildGraph(const Decoder::Program& program) {
graph.entry_block = 0;
std::map<uint32_t, uint32_t> pc_to_block;
for (const auto& block: graph.blocks) {
pc_to_block.emplace(block.start_pc, block.id);
}
const auto block_at_pc = [&](uint32_t pc) {
const auto found = std::ranges::lower_bound(graph.blocks, pc, {}, &BasicBlock::start_pc);
return found != graph.blocks.end() && found->start_pc == pc ? found->id : UINT32_MAX;
};
for (auto& block: graph.blocks) {
block.terminator = {};
@@ -1557,6 +1551,9 @@ Graph BuildGraph(const Decoder::Program& program) {
const auto next_pc = InstructionEndPc(last);
if (last.opcode == Opcode::S_ENDPGM) {
block.terminator.kind = TerminatorKind::Return;
} else if (last.opcode == Opcode::S_SWAPPC_B64 && last.branch_target != UINT32_MAX) {
block.terminator.kind = TerminatorKind::Branch;
block.terminator.true_block = block_at_pc(last.branch_target);
} else if (last.opcode == Opcode::S_SETPC_B64) {
const auto& target_info = setpc_targets.at(last.pc);
if (target_info.indirect) {
@@ -1566,7 +1563,7 @@ Graph BuildGraph(const Decoder::Program& program) {
block.terminator.indirect_selector_code = target_info.selector_code;
for (const auto target_pc: target_info.target_pcs) {
block.terminator.indirect_target_pcs.push_back(target_pc);
block.terminator.indirect_targets.push_back(pc_to_block.at(target_pc));
block.terminator.indirect_targets.push_back(block_at_pc(target_pc));
}
const auto selector_count = std::min(target_info.selector_values.size(),
target_info.selector_target_pcs.size());
@@ -1574,35 +1571,35 @@ Graph BuildGraph(const Decoder::Program& program) {
block.terminator.indirect_selector_values.push_back(
target_info.selector_values[i]);
block.terminator.indirect_selector_targets.push_back(
pc_to_block.at(target_info.selector_target_pcs[i]));
block_at_pc(target_info.selector_target_pcs[i]));
}
} else {
block.terminator.kind = TerminatorKind::Branch;
block.terminator.condition = BranchCondition::Always;
block.terminator.true_block = pc_to_block.at(target_info.target);
block.terminator.true_block = block_at_pc(target_info.target);
}
} else if (last.opcode == Opcode::S_BRANCH) {
block.terminator.kind = TerminatorKind::Branch;
block.terminator.condition = BranchCondition::Always;
block.terminator.true_block = pc_to_block.at(last.branch_target);
block.terminator.true_block = block_at_pc(last.branch_target);
} else if (Decoder::IsConditionalBranch(last.opcode)) {
block.terminator.kind = TerminatorKind::ConditionalBranch;
block.terminator.condition = ConditionForOpcode(last.opcode);
block.terminator.true_block = pc_to_block.at(last.branch_target);
const auto fallthrough = pc_to_block.find(next_pc);
if (fallthrough == pc_to_block.end()) {
block.terminator.true_block = block_at_pc(last.branch_target);
const auto fallthrough = block_at_pc(next_pc);
if (fallthrough == UINT32_MAX) {
ExitBuildFailure(
graph, FailureKind::MissingFallthrough, block.id,
fmt::format("conditional branch at pc 0x{:08x} has no fallthrough block",
last.pc));
}
block.terminator.false_block = fallthrough->second;
block.terminator.false_block = fallthrough;
} else {
auto next = pc_to_block.find(block.end_pc);
if (next != pc_to_block.end() && block.end_pc != block.start_pc) {
const auto next = block_at_pc(block.end_pc);
if (next != UINT32_MAX && block.end_pc != block.start_pc) {
block.terminator.kind = TerminatorKind::Branch;
block.terminator.condition = BranchCondition::Always;
block.terminator.true_block = next->second;
block.terminator.true_block = next;
} else {
block.terminator.kind = TerminatorKind::Return;
}
@@ -1683,6 +1680,36 @@ Graph Structurize(const Graph& graph) {
return GotoStructurizer(graph).Run();
}
bool MayWriteScalarRegister(const Decoder::Instruction& inst, uint32_t code) {
uint32_t destination = UINT32_MAX;
// Scalar ALU pair widths are not all decoded. The typed producer must confirm this word.
return InstructionWritesScalarCode(inst, code) ||
(ScalarOperandCode(inst.dst, destination) && code == destination + 1u);
}
uint32_t FindScalarDefinition(const Decoder::Program& program, const Graph& graph,
uint32_t before, uint32_t code) {
EXIT_IF(before >= program.instructions.size());
const auto found = std::ranges::find_if(graph.blocks, [before](const auto& block) {
return block.inst_begin <= before && before < block.inst_end;
});
EXIT_IF(found == graph.blocks.end());
const auto* block = &*found;
for (size_t depth = 0; depth < graph.blocks.size(); ++depth) {
for (uint32_t i = before; i > block->inst_begin;) {
const auto& inst = program.instructions[--i];
EXIT_IF(inst.opcode == Opcode::S_SWAPPC_B64);
if (MayWriteScalarRegister(inst, code)) return i;
}
if (block->id == graph.entry_block) return UINT32_MAX;
EXIT_IF(block->predecessors.size() != 1u);
block = graph.FindBlock(block->predecessors[0]);
EXIT_IF(block == nullptr);
before = block->inst_end;
}
EXIT("scalar shader call source has cyclic reaching definitions");
}
std::string BranchConditionToString(BranchCondition condition) {
switch (condition) {
case BranchCondition::Always: return "always";
@@ -113,6 +113,9 @@ struct Graph {
};
Graph BuildGraph(const Decoder::Program& program);
bool MayWriteScalarRegister(const Decoder::Instruction& inst, uint32_t code);
uint32_t FindScalarDefinition(const Decoder::Program& program, const Graph& graph,
uint32_t before, uint32_t code);
// Returns structured control flow or failure diagnostics without changing the native graph.
// On failure, failure_block is an original block ID or UINT32_MAX.
Graph Structurize(const Graph& graph);
@@ -47,6 +47,8 @@ constexpr MemoryOpcodeInfo MUBUF_OPCODE_LIST[] = {
{0x1du, Opcode::BUFFER_STORE_DWORDX2, 2, 32},
{0x1eu, Opcode::BUFFER_STORE_DWORDX4, 4, 32},
{0x1fu, Opcode::BUFFER_STORE_DWORDX3, 3, 32},
{0x24u, Opcode::BUFFER_LOAD_SHORT_D16, 1, 16},
{0x25u, Opcode::BUFFER_LOAD_SHORT_D16_HI, 1, 16},
{0x30u, Opcode::BUFFER_ATOMIC_SWAP, 1, 32},
{0x31u, Opcode::BUFFER_ATOMIC_CMPSWAP, 1, 32},
{0x32u, Opcode::BUFFER_ATOMIC_ADD, 1, 32},
@@ -298,8 +300,10 @@ void DecodeMubuf(uint32_t pc, std::span<const uint32_t> code, uint32_t word_inde
}
DecodeVectorGpr(vdata, inst.dst);
if (inst.opcode == Opcode::BUFFER_LOAD_FORMAT_D16_X) {
inst.dst.sdwa_sel = 4u;
if (inst.opcode == Opcode::BUFFER_LOAD_FORMAT_D16_X ||
inst.opcode == Opcode::BUFFER_LOAD_SHORT_D16 ||
inst.opcode == Opcode::BUFFER_LOAD_SHORT_D16_HI) {
inst.dst.sdwa_sel = inst.opcode == Opcode::BUFFER_LOAD_SHORT_D16_HI ? 5u : 4u;
}
DecodeVectorGpr(vaddr, inst.src0);
DecodeScalarSource(srsrc * 4u, pc, inst.src1);
@@ -371,7 +375,7 @@ void DecodeFlat(uint32_t pc, std::span<const uint32_t> code, uint32_t word_index
ApplyMemoryInfo(inst, info);
SetRawWords(inst, code, word_index, 2);
if (lds != 0 || inst.glc || inst.slc || seg == 3u) {
if (lds != 0 || inst.glc || (inst.slc && !IsFlatStoreOpcode(inst.opcode)) || seg == 3u) {
SetUnsupported(inst, Family::FLAT, opcode, "FLAT modifiers or segment are not implemented");
return;
}
@@ -47,8 +47,12 @@ constexpr OpcodeMap SOP1_OPCODE_LIST[] = {
{0x0au, Opcode::S_WQM_B64},
{0x0bu, Opcode::S_BREV_B32},
{0x0cu, Opcode::S_BREV_B64},
{0x0du, Opcode::S_BCNT0_I32_B32},
{0x0eu, Opcode::S_BCNT0_I32_B64},
{0x0fu, Opcode::S_BCNT1_I32_B32},
{0x10u, Opcode::S_BCNT1_I32_B64},
{0x11u, Opcode::S_FF0_I32_B32},
{0x12u, Opcode::S_FF0_I32_B64},
{0x13u, Opcode::S_FF1_I32_B32},
{0x14u, Opcode::S_FF1_I32_B64},
{0x15u, Opcode::S_FLBIT_I32_B32},
@@ -61,6 +65,7 @@ constexpr OpcodeMap SOP1_OPCODE_LIST[] = {
{0x1eu, Opcode::S_BITSET1_B64},
{0x1fu, Opcode::S_GETPC_B64},
{0x20u, Opcode::S_SETPC_B64},
{0x21u, Opcode::S_SWAPPC_B64},
{0x24u, Opcode::S_AND_SAVEEXEC_B64},
{0x28u, Opcode::S_ORN2_SAVEEXEC_B64},
{0x2du, Opcode::S_QUADMASK_B64},
@@ -88,7 +93,7 @@ constexpr OpcodeMap SOPK_OPCODE_LIST[] = {
{0x08u, Opcode::S_CMP_LE_I32}, {0x09u, Opcode::S_CMP_EQ_U32}, {0x0au, Opcode::S_CMP_LG_U32},
{0x0bu, Opcode::S_CMP_GT_U32}, {0x0cu, Opcode::S_CMP_GE_U32}, {0x0du, Opcode::S_CMP_LT_U32},
{0x0eu, Opcode::S_CMP_LE_U32}, {0x0fu, Opcode::S_ADD_I32}, {0x10u, Opcode::S_MULK_I32},
{0x13u, Opcode::S_SETREG_B32}, {0x17u, Opcode::S_WAITCNT_VSCNT}, {0x18u, Opcode::S_WAITCNT},
{0x12u, Opcode::S_GETREG_B32}, {0x13u, Opcode::S_SETREG_B32}, {0x17u, Opcode::S_WAITCNT_VSCNT}, {0x18u, Opcode::S_WAITCNT},
{0x19u, Opcode::S_WAITCNT}, {0x1au, Opcode::S_WAITCNT},
{0x1bu, Opcode::S_SUBVECTOR_LOOP_BEGIN}, {0x1cu, Opcode::S_SUBVECTOR_LOOP_END},
};
@@ -111,7 +116,9 @@ constexpr OpcodeMap SOPP_OPCODE_LIST[] = {
{0x12u, Opcode::S_TRAP},
{0x16u, Opcode::S_TTRACEDATA},
{0x17u, Opcode::S_CBRANCH_CDBGSYS},
{0x18u, Opcode::S_CBRANCH_CDBGUSER},
{0x19u, Opcode::S_CBRANCH_CDBGSYS_OR_USER},
{0x1au, Opcode::S_CBRANCH_CDBGSYS_AND_USER},
{0x20u, Opcode::S_INST_PREFETCH},
{0x23u, Opcode::S_WAITCNT_DEPCTR},
};
@@ -155,6 +162,10 @@ void DecodeSop1(uint32_t pc, std::span<const uint32_t> code, uint32_t word_index
}
switch (inst.opcode) {
case Opcode::S_SWAPPC_B64:
inst.branch_target = UINT32_MAX;
inst.data_dwords = 2;
break;
case Opcode::S_GETPC_B64:
inst.src_count = 0;
DecodeScalarDestination(sdst, pc, inst.dst);
@@ -202,7 +213,7 @@ void DecodeSopk(uint32_t pc, std::span<const uint32_t> code, uint32_t word_index
const uint32_t word = code[word_index];
const uint32_t opcode = (word >> 23u) & 0x1fu;
const uint32_t sdst = (word >> 16u) & 0x7fu;
const auto imm = opcode >= 0x09u && opcode <= 0x0eu
const auto imm = (opcode >= 0x09u && opcode <= 0x0eu) || opcode == 0x12u
? static_cast<int32_t>(word & 0xffffu)
: static_cast<int32_t>(static_cast<int16_t>(word & 0xffffu));
@@ -222,6 +233,7 @@ void DecodeSopk(uint32_t pc, std::span<const uint32_t> code, uint32_t word_index
}
switch (inst.opcode) {
case Opcode::S_GETREG_B32:
case Opcode::S_MOVK_I32: DecodeScalarDestination(sdst, pc, inst.dst); return;
case Opcode::S_SUBVECTOR_LOOP_BEGIN:
case Opcode::S_SUBVECTOR_LOOP_END:
@@ -186,7 +186,9 @@ bool IsConditionalBranch(Opcode opcode) {
switch (opcode) {
// Conditional shader debugging is disabled.
case Opcode::S_CBRANCH_CDBGSYS:
case Opcode::S_CBRANCH_CDBGSYS_OR_USER: return false;
case Opcode::S_CBRANCH_CDBGUSER:
case Opcode::S_CBRANCH_CDBGSYS_OR_USER:
case Opcode::S_CBRANCH_CDBGSYS_AND_USER: return false;
case Opcode::S_CBRANCH_SCC0:
case Opcode::S_CBRANCH_SCC1:
case Opcode::S_CBRANCH_VCCZ:
@@ -389,6 +391,7 @@ Program DecodeFrontProgram(std::span<const uint32_t> front) {
while (front_words < front.size()) {
auto& inst = result.instructions.emplace_back();
DecodeInstruction(front, front_words, inst);
result.has_swap_pc |= inst.opcode == Opcode::S_SWAPPC_B64;
front_words += inst.word_count;
if (inst.opcode == Opcode::S_SETPC_B64) {
EXIT_NOT_IMPLEMENTED(inst.src0.kind != OperandKind::Sgpr || inst.src0.reg != 6u);
@@ -406,6 +409,7 @@ void DecodeProgram(std::span<const uint32_t> code, Program& program) {
program.instructions.clear();
program.instructions.reserve(code.size());
program.code = code;
program.has_swap_pc = false;
std::vector<bool> branch_targets;
for (uint32_t word_index = 0; word_index < code.size();) {
@@ -413,6 +417,7 @@ void DecodeProgram(std::span<const uint32_t> code, Program& program) {
DecodeInstruction(code, word_index, program.instructions.back());
const auto& inst = program.instructions.back();
program.has_swap_pc |= inst.opcode == Opcode::S_SWAPPC_B64;
word_index += inst.word_count;
if (IsDirectBranch(inst.opcode)) {
@@ -507,6 +512,10 @@ std::string InstructionToString(const Instruction& inst) {
case Opcode::S_SEXT_I32_I16:
case Opcode::S_BREV_B32:
case Opcode::S_BREV_B64:
case Opcode::S_BCNT0_I32_B32:
case Opcode::S_BCNT0_I32_B64:
case Opcode::S_FF0_I32_B32:
case Opcode::S_FF0_I32_B64:
case Opcode::S_BCNT1_I32_B32:
case Opcode::S_FLBIT_I32_B32:
case Opcode::S_FF1_I32_B32:
@@ -568,7 +577,9 @@ std::string InstructionToString(const Instruction& inst) {
case Opcode::S_CBRANCH_EXECZ:
case Opcode::S_CBRANCH_EXECNZ:
case Opcode::S_CBRANCH_CDBGSYS:
case Opcode::S_CBRANCH_CDBGUSER:
case Opcode::S_CBRANCH_CDBGSYS_OR_USER:
case Opcode::S_CBRANCH_CDBGSYS_AND_USER:
return WithUnsupportedReason(inst, fmt::format("0x{:08x}: {} 0x{:08x}", inst.pc,
magic_enum::enum_name(inst.opcode),
inst.branch_target));
@@ -627,6 +638,8 @@ std::string InstructionToString(const Instruction& inst) {
case Opcode::BUFFER_STORE_FORMAT_D16_X:
case Opcode::BUFFER_LOAD_UBYTE:
case Opcode::BUFFER_LOAD_USHORT:
case Opcode::BUFFER_LOAD_SHORT_D16:
case Opcode::BUFFER_LOAD_SHORT_D16_HI:
case Opcode::BUFFER_LOAD_DWORD:
case Opcode::BUFFER_LOAD_DWORDX2:
case Opcode::BUFFER_LOAD_DWORDX3:
@@ -50,8 +50,12 @@ enum class Opcode {
S_ABSDIFF_I32,
S_BREV_B32,
S_BREV_B64,
S_BCNT0_I32_B32,
S_BCNT0_I32_B64,
S_BCNT1_I32_B32,
S_BCNT1_I32_B64,
S_FF0_I32_B32,
S_FF0_I32_B64,
S_FF1_I32_B32,
S_FF1_I32_B64,
S_FLBIT_I32_B32,
@@ -60,6 +64,7 @@ enum class Opcode {
S_QUADMASK_B64,
S_GETPC_B64,
S_SETPC_B64,
S_SWAPPC_B64,
S_SUBVECTOR_LOOP_BEGIN,
S_SUBVECTOR_LOOP_END,
S_AND_SAVEEXEC_B32,
@@ -373,10 +378,38 @@ enum class Opcode {
V_CMP_NEQ_F32,
V_CMP_NLT_F32,
V_CMP_TRU_F32,
V_CMP_F_F64,
V_CMP_LT_F64,
V_CMP_EQ_F64,
V_CMP_LE_F64,
V_CMP_GT_F64,
V_CMP_LG_F64,
V_CMP_GE_F64,
V_CMP_O_F64,
V_CMP_U_F64,
V_CMP_NGE_F64,
V_CMP_NLG_F64,
V_CMP_NGT_F64,
V_CMP_NLE_F64,
V_CMP_NEQ_F64,
V_CMP_NLT_F64,
V_CMP_TRU_F64,
V_CMPX_F_F64,
V_CMPX_LT_F64,
V_CMPX_EQ_F64,
V_CMPX_LE_F64,
V_CMPX_GT_F64,
V_CMPX_LG_F64,
V_CMPX_GE_F64,
V_CMPX_O_F64,
V_CMPX_U_F64,
V_CMPX_NGE_F64,
V_CMPX_NLG_F64,
V_CMPX_NGT_F64,
V_CMPX_NLE_F64,
V_CMPX_NEQ_F64,
V_CMPX_NLT_F64,
V_CMPX_TRU_F64,
V_CMPX_LT_F32,
V_CMPX_EQ_F32,
V_CMPX_LE_F32,
@@ -458,11 +491,14 @@ enum class Opcode {
V_CMP_NE_U32,
V_CMP_GE_U32,
V_CMP_T_U32,
// Keep V_CMP_F_I64 to V_CMPX_T_U64 contiguous: VectorAluOps.cpp tests this range.
V_CMP_F_I64,
V_CMP_EQ_I64,
V_CMP_LT_I64,
V_CMP_EQ_I64,
V_CMP_LE_I64,
V_CMP_GT_I64,
V_CMP_NE_I64,
V_CMP_GE_I64,
V_CMP_T_I64,
V_CMP_F_U64,
V_CMP_LT_U64,
@@ -476,7 +512,9 @@ enum class Opcode {
V_CMPX_LT_I64,
V_CMPX_EQ_I64,
V_CMPX_LE_I64,
V_CMPX_GT_I64,
V_CMPX_NE_I64,
V_CMPX_GE_I64,
V_CMPX_T_I64,
V_CMPX_F_U64,
V_CMPX_LT_U64,
@@ -518,6 +556,8 @@ enum class Opcode {
BUFFER_LOAD_SBYTE,
BUFFER_LOAD_USHORT,
BUFFER_LOAD_SSHORT,
BUFFER_LOAD_SHORT_D16,
BUFFER_LOAD_SHORT_D16_HI,
BUFFER_LOAD_DWORD,
BUFFER_LOAD_DWORDX2,
BUFFER_LOAD_DWORDX3,
@@ -674,8 +714,11 @@ enum class Opcode {
S_CBRANCH_EXECZ,
S_CBRANCH_EXECNZ,
S_CBRANCH_CDBGSYS,
S_CBRANCH_CDBGUSER,
S_CBRANCH_CDBGSYS_OR_USER,
S_CBRANCH_CDBGSYS_AND_USER,
S_SENDMSG,
S_GETREG_B32,
S_SETREG_B32,
S_SLEEP,
S_TRAP,
@@ -814,6 +857,7 @@ struct Instruction {
struct Program {
std::span<const uint32_t> code;
std::vector<Instruction> instructions;
bool has_swap_pc = false;
};
// Code spans are trusted to contain complete instructions, valid branch targets, and 32-bit PCs.
@@ -39,7 +39,7 @@ constexpr Vop2OpcodeInfo VOP2_OPCODE_LIST[] = {
{0x0bu, Opcode::V_MUL_U32_U24, Vop2SdwaProfile::IntegerPartialDestination},
{0x0fu, Opcode::V_MIN_F32},
{0x10u, Opcode::V_MAX_F32},
{0x11u, Opcode::V_MIN_I32},
{0x11u, Opcode::V_MIN_I32, Vop2SdwaProfile::IntegerFullDestination},
{0x12u, Opcode::V_MAX_I32, Vop2SdwaProfile::IntegerFullDestination},
{0x13u, Opcode::V_MIN_U32, Vop2SdwaProfile::IntegerPartialDestination},
{0x14u, Opcode::V_MAX_U32, Vop2SdwaProfile::IntegerFullDestination},
@@ -236,8 +236,23 @@ constexpr VopcOpcodeInfo VOPC_OPCODE_LIST[] = {
{0x19u, Opcode::V_CMPX_NGE_F32}, {0x1au, Opcode::V_CMPX_NLG_F32},
{0x1bu, Opcode::V_CMPX_NGT_F32}, {0x1cu, Opcode::V_CMPX_NLE_F32},
{0x1du, Opcode::V_CMPX_NEQ_F32}, {0x1eu, Opcode::V_CMPX_NLT_F32},
{0x22u, Opcode::V_CMP_EQ_F64, false}, {0x23u, Opcode::V_CMP_LE_F64, false},
{0x33u, Opcode::V_CMPX_LE_F64, false}, {0x36u, Opcode::V_CMPX_GE_F64, false},
{0x20u, Opcode::V_CMP_F_F64, false}, {0x21u, Opcode::V_CMP_LT_F64, false},
{0x22u, Opcode::V_CMP_EQ_F64, false}, {0x23u, Opcode::V_CMP_LE_F64, false},
{0x24u, Opcode::V_CMP_GT_F64, false}, {0x25u, Opcode::V_CMP_LG_F64, false},
{0x26u, Opcode::V_CMP_GE_F64, false}, {0x27u, Opcode::V_CMP_O_F64, false},
{0x28u, Opcode::V_CMP_U_F64, false}, {0x29u, Opcode::V_CMP_NGE_F64, false},
{0x2au, Opcode::V_CMP_NLG_F64, false}, {0x2bu, Opcode::V_CMP_NGT_F64, false},
{0x2cu, Opcode::V_CMP_NLE_F64, false}, {0x2du, Opcode::V_CMP_NEQ_F64, false},
{0x2eu, Opcode::V_CMP_NLT_F64, false}, {0x2fu, Opcode::V_CMP_TRU_F64, false},
{0x30u, Opcode::V_CMPX_F_F64, false},
{0x31u, Opcode::V_CMPX_LT_F64, false}, {0x32u, Opcode::V_CMPX_EQ_F64, false},
{0x33u, Opcode::V_CMPX_LE_F64, false}, {0x34u, Opcode::V_CMPX_GT_F64, false},
{0x35u, Opcode::V_CMPX_LG_F64, false}, {0x36u, Opcode::V_CMPX_GE_F64, false},
{0x37u, Opcode::V_CMPX_O_F64, false}, {0x38u, Opcode::V_CMPX_U_F64, false},
{0x39u, Opcode::V_CMPX_NGE_F64, false}, {0x3au, Opcode::V_CMPX_NLG_F64, false},
{0x3bu, Opcode::V_CMPX_NGT_F64, false}, {0x3cu, Opcode::V_CMPX_NLE_F64, false},
{0x3du, Opcode::V_CMPX_NEQ_F64, false}, {0x3eu, Opcode::V_CMPX_NLT_F64, false},
{0x3fu, Opcode::V_CMPX_TRU_F64, false},
{0x80u, Opcode::V_CMP_F_I32}, {0x81u, Opcode::V_CMP_LT_I32},
{0x82u, Opcode::V_CMP_EQ_I32}, {0x83u, Opcode::V_CMP_LE_I32},
{0x84u, Opcode::V_CMP_GT_I32}, {0x85u, Opcode::V_CMP_NE_I32},
@@ -263,25 +278,26 @@ constexpr VopcOpcodeInfo VOPC_OPCODE_LIST[] = {
{0xc1u, Opcode::V_CMP_LT_U32}, {0xc2u, Opcode::V_CMP_EQ_U32},
{0xc3u, Opcode::V_CMP_LE_U32}, {0xc4u, Opcode::V_CMP_GT_U32},
{0xc5u, Opcode::V_CMP_NE_U32}, {0xc6u, Opcode::V_CMP_GE_U32},
{0xc7u, Opcode::V_CMP_T_U32}, {0xa2u, Opcode::V_CMP_EQ_I64, false},
{0xa1u, Opcode::V_CMP_LT_I64, false}, {0xa3u, Opcode::V_CMP_LE_I64, false},
{0xa5u, Opcode::V_CMP_NE_I64, false},
{0xa0u, Opcode::V_CMP_F_I64, false}, {0xa7u, Opcode::V_CMP_T_I64, false},
{0xb0u, Opcode::V_CMPX_F_I64, false}, {0xb1u, Opcode::V_CMPX_LT_I64, false},
{0xb2u, Opcode::V_CMPX_EQ_I64, false}, {0xb3u, Opcode::V_CMPX_LE_I64, false},
{0xb7u, Opcode::V_CMPX_T_I64, false},
{0xb5u, Opcode::V_CMPX_NE_I64, false}, {0xd1u, Opcode::V_CMPX_LT_U32},
{0xc7u, Opcode::V_CMP_T_U32}, {0xd1u, Opcode::V_CMPX_LT_U32},
{0xd2u, Opcode::V_CMPX_EQ_U32}, {0xd3u, Opcode::V_CMPX_LE_U32},
{0xd4u, Opcode::V_CMPX_GT_U32}, {0xd5u, Opcode::V_CMPX_NE_U32},
{0xd6u, Opcode::V_CMPX_GE_U32}, {0xe1u, Opcode::V_CMP_LT_U64, false},
{0xe0u, Opcode::V_CMP_F_U64, false}, {0xe7u, Opcode::V_CMP_T_U64, false},
{0xf0u, Opcode::V_CMPX_F_U64, false}, {0xf1u, Opcode::V_CMPX_LT_U64, false},
{0xf2u, Opcode::V_CMPX_EQ_U64, false}, {0xf4u, Opcode::V_CMPX_GT_U64, false},
{0xf6u, Opcode::V_CMPX_GE_U64, false}, {0xf7u, Opcode::V_CMPX_T_U64, false},
{0xe2u, Opcode::V_CMP_EQ_U64, false}, {0xe3u, Opcode::V_CMP_LE_U64, false},
{0xe4u, Opcode::V_CMP_GT_U64, false}, {0xe5u, Opcode::V_CMP_NE_U64, false},
{0xe6u, Opcode::V_CMP_GE_U64, false}, {0xf3u, Opcode::V_CMPX_LE_U64, false},
{0xf5u, Opcode::V_CMPX_NE_U64, false}, {0xc9u, Opcode::V_CMP_LT_F16},
{0xd6u, Opcode::V_CMPX_GE_U32}, {0xa0u, Opcode::V_CMP_F_I64, false},
{0xa1u, Opcode::V_CMP_LT_I64, false}, {0xa2u, Opcode::V_CMP_EQ_I64, false},
{0xa3u, Opcode::V_CMP_LE_I64, false}, {0xa4u, Opcode::V_CMP_GT_I64, false},
{0xa5u, Opcode::V_CMP_NE_I64, false}, {0xa6u, Opcode::V_CMP_GE_I64, false},
{0xa7u, Opcode::V_CMP_T_I64, false}, {0xb0u, Opcode::V_CMPX_F_I64, false},
{0xb1u, Opcode::V_CMPX_LT_I64, false}, {0xb2u, Opcode::V_CMPX_EQ_I64, false},
{0xb3u, Opcode::V_CMPX_LE_I64, false}, {0xb4u, Opcode::V_CMPX_GT_I64, false},
{0xb5u, Opcode::V_CMPX_NE_I64, false}, {0xb6u, Opcode::V_CMPX_GE_I64, false},
{0xb7u, Opcode::V_CMPX_T_I64, false}, {0xe0u, Opcode::V_CMP_F_U64, false},
{0xe1u, Opcode::V_CMP_LT_U64, false}, {0xe2u, Opcode::V_CMP_EQ_U64, false},
{0xe3u, Opcode::V_CMP_LE_U64, false}, {0xe4u, Opcode::V_CMP_GT_U64, false},
{0xe5u, Opcode::V_CMP_NE_U64, false}, {0xe6u, Opcode::V_CMP_GE_U64, false},
{0xe7u, Opcode::V_CMP_T_U64, false}, {0xf0u, Opcode::V_CMPX_F_U64, false},
{0xf1u, Opcode::V_CMPX_LT_U64, false}, {0xf2u, Opcode::V_CMPX_EQ_U64, false},
{0xf3u, Opcode::V_CMPX_LE_U64, false}, {0xf4u, Opcode::V_CMPX_GT_U64, false},
{0xf5u, Opcode::V_CMPX_NE_U64, false}, {0xf6u, Opcode::V_CMPX_GE_U64, false},
{0xf7u, Opcode::V_CMPX_T_U64, false}, {0xc9u, Opcode::V_CMP_LT_F16},
{0xcau, Opcode::V_CMP_EQ_F16}, {0xcbu, Opcode::V_CMP_LE_F16},
{0xccu, Opcode::V_CMP_GT_F16}, {0xcdu, Opcode::V_CMP_LG_F16},
{0xceu, Opcode::V_CMP_GE_F16}, {0xe9u, Opcode::V_CMP_NGE_F16},
@@ -760,9 +776,10 @@ void DecodeVop1Sdwa(uint32_t pc, std::span<const uint32_t> code, uint32_t word_i
void ApplyDppModifier(Operand& operand, uint32_t modifier, uint32_t encoding) {
operand.dpp = true;
if (encoding == 233u) {
operand.dpp8 = true;
operand.dpp_ctrl = modifier >> 8u;
if (encoding == 233u || encoding == 234u) {
operand.dpp8 = true;
operand.dpp_ctrl = modifier >> 8u;
operand.dpp_fetch_inactive = encoding == 234u;
return;
}
operand.negate = ((modifier >> 20u) & 0x1u) != 0u;
@@ -879,10 +896,38 @@ bool IsVopcFloatCompareOpcode(Opcode opcode) {
case Opcode::V_CMP_F_F32:
case Opcode::V_CMP_LT_F32:
case Opcode::V_CMP_EQ_F32:
case Opcode::V_CMP_F_F64:
case Opcode::V_CMP_LT_F64:
case Opcode::V_CMP_EQ_F64:
case Opcode::V_CMP_LE_F64:
case Opcode::V_CMP_GT_F64:
case Opcode::V_CMP_LG_F64:
case Opcode::V_CMP_GE_F64:
case Opcode::V_CMP_O_F64:
case Opcode::V_CMP_U_F64:
case Opcode::V_CMP_NGE_F64:
case Opcode::V_CMP_NLG_F64:
case Opcode::V_CMP_NGT_F64:
case Opcode::V_CMP_NLE_F64:
case Opcode::V_CMP_NEQ_F64:
case Opcode::V_CMP_NLT_F64:
case Opcode::V_CMP_TRU_F64:
case Opcode::V_CMPX_F_F64:
case Opcode::V_CMPX_LT_F64:
case Opcode::V_CMPX_EQ_F64:
case Opcode::V_CMPX_LE_F64:
case Opcode::V_CMPX_GT_F64:
case Opcode::V_CMPX_LG_F64:
case Opcode::V_CMPX_GE_F64:
case Opcode::V_CMPX_O_F64:
case Opcode::V_CMPX_U_F64:
case Opcode::V_CMPX_NGE_F64:
case Opcode::V_CMPX_NLG_F64:
case Opcode::V_CMPX_NGT_F64:
case Opcode::V_CMPX_NLE_F64:
case Opcode::V_CMPX_NEQ_F64:
case Opcode::V_CMPX_NLT_F64:
case Opcode::V_CMPX_TRU_F64:
case Opcode::V_CMP_LE_F32:
case Opcode::V_CMP_GT_F32:
case Opcode::V_CMP_LG_F32:
@@ -997,7 +1042,8 @@ constexpr Vop2SdwaRule VOP2_SDWA_RULES[] = {
{SdwaSelAll(), SdwaSelAll(), SdwaSelAll(), true, false},
{SdwaSelFull(), SdwaSelAll(), SdwaSelAll(), false, false},
{SdwaSelFull(), SdwaSelAll(), SdwaSelAll(), false, false},
{SdwaSelWords() | SdwaSelFull(), SdwaSelAll(), SdwaSelAll(), true, false},
// Bitwise: WriteRawU32 inserts a byte or word destination like any other partial write.
{SdwaSelAll(), SdwaSelAll(), SdwaSelAll(), true, false},
};
static_assert(sizeof(VOP2_SDWA_RULES) / sizeof(VOP2_SDWA_RULES[0]) ==
static_cast<size_t>(Vop2SdwaProfile::Count));
@@ -1201,10 +1247,10 @@ VopcSdwaFields DecodeVopcSdwaFields(uint32_t modifier) {
}
bool SupportsVopcSdwa(Opcode opcode) {
return opcode != Opcode::UNSUPPORTED && opcode != Opcode::V_CMP_NE_I64 &&
opcode != Opcode::V_CMP_EQ_F64 &&
opcode != Opcode::V_CMP_LE_F64 && opcode != Opcode::V_CMPX_LE_F64 &&
opcode != Opcode::V_CMPX_GE_F64;
// SDWA operand handling is not implemented for 64-bit compares.
return opcode != Opcode::UNSUPPORTED &&
(opcode < Opcode::V_CMP_F_F64 || opcode > Opcode::V_CMPX_TRU_F64) &&
(opcode < Opcode::V_CMP_F_I64 || opcode > Opcode::V_CMPX_T_U64);
}
void DecodeVopcSdwa(uint32_t pc, std::span<const uint32_t> code, uint32_t word_index,
@@ -1537,8 +1583,22 @@ void ApplyNativeVop3SourceModifiers(Instruction& inst, uint32_t abs, uint32_t ne
bool IsVopcCompareExec(Opcode opcode) {
switch (opcode) {
case Opcode::V_CMPX_F_F64:
case Opcode::V_CMPX_LT_F64:
case Opcode::V_CMPX_EQ_F64:
case Opcode::V_CMPX_LE_F64:
case Opcode::V_CMPX_GT_F64:
case Opcode::V_CMPX_LG_F64:
case Opcode::V_CMPX_GE_F64:
case Opcode::V_CMPX_O_F64:
case Opcode::V_CMPX_U_F64:
case Opcode::V_CMPX_NGE_F64:
case Opcode::V_CMPX_NLG_F64:
case Opcode::V_CMPX_NGT_F64:
case Opcode::V_CMPX_NLE_F64:
case Opcode::V_CMPX_NEQ_F64:
case Opcode::V_CMPX_NLT_F64:
case Opcode::V_CMPX_TRU_F64:
case Opcode::V_CMPX_LT_F32:
case Opcode::V_CMPX_EQ_F32:
case Opcode::V_CMPX_LE_F32:
@@ -1571,7 +1631,9 @@ bool IsVopcCompareExec(Opcode opcode) {
case Opcode::V_CMPX_LT_I64:
case Opcode::V_CMPX_EQ_I64:
case Opcode::V_CMPX_LE_I64:
case Opcode::V_CMPX_GT_I64:
case Opcode::V_CMPX_NE_I64:
case Opcode::V_CMPX_GE_I64:
case Opcode::V_CMPX_T_I64:
case Opcode::V_CMPX_F_U64:
case Opcode::V_CMPX_LT_U64:
@@ -1625,7 +1687,8 @@ void DecodeVop2(uint32_t pc, std::span<const uint32_t> code, uint32_t word_index
return;
}
switch (src0) {
case 233u: DecodeVop2Dpp(pc, code, word_index, opcode, vdst, vsrc1, inst); return;
case 233u:
case 234u: DecodeVop2Dpp(pc, code, word_index, opcode, vdst, vsrc1, inst); return;
case 249u: DecodeVop2Sdwa(pc, code, word_index, opcode, vdst, vsrc1, inst); return;
case 250u: DecodeVop2Dpp(pc, code, word_index, opcode, vdst, vsrc1, inst); return;
default: break;
@@ -1660,7 +1723,8 @@ void DecodeVop1(uint32_t pc, std::span<const uint32_t> code, uint32_t word_index
return;
}
switch (src0) {
case 233u: DecodeVop1Dpp(pc, code, word_index, opcode, vdst, inst); return;
case 233u:
case 234u: DecodeVop1Dpp(pc, code, word_index, opcode, vdst, inst); return;
case 249u: DecodeVop1Sdwa(pc, code, word_index, opcode, vdst, inst); return;
case 250u: DecodeVop1Dpp(pc, code, word_index, opcode, vdst, inst); return;
default: break;
@@ -1,6 +1,7 @@
#include "graphics/shader/recompiler/frontend/translate/Translator.h"
#include <bit>
#include <utility>
namespace Libs::Graphics::ShaderRecompiler::Frontend {
@@ -27,7 +28,8 @@ void Translator::EmitCompareConstant(const Decoder::Instruction& inst, bool valu
}
void Translator::EmitIntegerCompare(const Decoder::Instruction& inst, IR::ValueOpcode opcode,
IR::Type type, bool scalar, bool cmpx) {
IR::Type type, bool scalar, bool cmpx, bool swap_operands) {
// The equality compares share their IR opcode with the unsigned ones.
const bool signed_64 = opcode == IR::ValueOpcode::SLessThan64 ||
opcode == IR::ValueOpcode::SLessThanEqual64 ||
inst.opcode == Decoder::Opcode::V_CMP_EQ_I64 ||
@@ -42,8 +44,12 @@ void Translator::EmitIntegerCompare(const Decoder::Instruction& inst, IR::ValueO
}
return ReadOperand(operand, type);
};
EmitCompareResult(inst, IR::U1(ir.Emit(opcode, {read(inst.src0), read(inst.src1)})), scalar,
cmpx);
auto lhs = read(inst.src0);
auto rhs = read(inst.src1);
if (swap_operands) {
std::swap(lhs, rhs);
}
EmitCompareResult(inst, IR::U1(ir.Emit(opcode, {lhs, rhs})), scalar, cmpx);
}
void Translator::EmitInteger16Compare(const Decoder::Instruction& inst, IR::ValueOpcode opcode,
@@ -65,11 +71,14 @@ void Translator::EmitFloatCompare(const Decoder::Instruction& inst, IR::ValueOpc
EmitCompareResult(inst, IR::U1(ir.Emit(opcode, {lhs, rhs}, flags)), false, cmpx);
}
void Translator::EmitFloatOrderedCompare(const Decoder::Instruction& inst, bool ordered, bool cmpx) {
const auto lhs = IR::F32(ReadOperand(inst.src0, IR::Type::F32));
const auto rhs = IR::F32(ReadOperand(inst.src1, IR::Type::F32));
const auto unordered = ir.LogicalOr(IR::U1(ir.Emit(IR::ValueOpcode::FPIsNan32, {lhs})),
IR::U1(ir.Emit(IR::ValueOpcode::FPIsNan32, {rhs})));
void Translator::EmitFloatOrderedCompare(const Decoder::Instruction& inst, bool ordered, bool cmpx,
bool is64) {
const auto type = is64 ? IR::Type::F64 : IR::Type::F32;
const auto is_nan_op = is64 ? IR::ValueOpcode::FPIsNan64 : IR::ValueOpcode::FPIsNan32;
const auto lhs = ReadOperand(inst.src0, type);
const auto rhs = ReadOperand(inst.src1, type);
const auto unordered = ir.LogicalOr(IR::U1(ir.Emit(is_nan_op, {lhs})),
IR::U1(ir.Emit(is_nan_op, {rhs})));
EmitCompareResult(inst, ordered ? ir.LogicalNot(unordered) : unordered, false, cmpx);
}
@@ -221,8 +221,12 @@ void Translator::V_XOR3_B32(const Decoder::Instruction& inst) {
WriteOperand(DestinationOperand(inst), result);
}
void Translator::S_FF1_I32_B64(const Decoder::Instruction& inst) {
const auto source = ExtractU64(ReadU64(inst.src0));
void Translator::S_FF_I32_B64(const Decoder::Instruction& inst, bool find_zero) {
auto source = ReadU32Pair(inst.src0);
if (find_zero) {
source[0] = ir.BitwiseNot(source[0]);
source[1] = ir.BitwiseNot(source[1]);
}
const auto low_lsb = IR::U32(ir.Emit(IR::ValueOpcode::FindILsb32, {source[0]}));
const auto high_lsb = IR::U32(ir.Emit(IR::ValueOpcode::FindILsb32, {source[1]}));
const auto high_position = ir.IAdd(high_lsb, IR::U32(IR::Value(32u)));
@@ -973,6 +973,8 @@ void Translator::EmitMemory(const Decoder::Instruction& inst) {
case Decoder::Opcode::BUFFER_LOAD_SBYTE:
case Decoder::Opcode::BUFFER_LOAD_USHORT:
case Decoder::Opcode::BUFFER_LOAD_SSHORT:
case Decoder::Opcode::BUFFER_LOAD_SHORT_D16:
case Decoder::Opcode::BUFFER_LOAD_SHORT_D16_HI:
case Decoder::Opcode::BUFFER_LOAD_DWORD:
case Decoder::Opcode::BUFFER_LOAD_DWORDX2:
case Decoder::Opcode::BUFFER_LOAD_DWORDX3:
@@ -10,13 +10,21 @@ void Translator::EmitScalar(const Decoder::Instruction& inst) {
case O::S_MOV_B64: S_MOV_B64(inst); return;
case O::S_WQM_B32: S_WQM(inst, false); return;
case O::S_WQM_B64: S_WQM(inst, true); return;
case O::S_GETPC_B64: S_GETPC_B64(inst); return;
case O::S_GETPC_B64:
case O::S_SWAPPC_B64: S_GETPC_B64(inst); return;
case O::S_SETPC_B64: return;
case O::S_SUBVECTOR_LOOP_BEGIN: S_SUBVECTOR_LOOP(inst, true); return;
case O::S_SUBVECTOR_LOOP_END: S_SUBVECTOR_LOOP(inst, false); return;
case O::S_CSELECT_B32: S_CSELECT_B32(inst); return;
case O::S_CSELECT_B64: ScalarSelect64(inst, inst.src1); return;
case O::S_CMOV_B64: ScalarSelect64(inst, inst.dst); return;
case O::S_GETREG_B32:
if (!graphics_compute || (inst.src0.value != 0x1818u && inst.src0.value != 0x0a18u))
EXIT("unsupported S_GETREG_B32 at pc 0x%08x: selector=0x%04x graphics_compute=%u",
inst.pc, inst.src0.value, static_cast<unsigned>(graphics_compute));
// HW_ID2 identifies graphics compute as QUEUE=6, ME=0.
WriteOperand(inst.dst, IR::Value(inst.src0.value == 0x1818u ? 6u : 0u));
return;
case O::S_SETREG_B32: EmitControlNop(); return;
case O::S_WAITCNT_VSCNT: S_WAITCNT_VSCNT(inst); return;
case O::S_WAITCNT: return;
@@ -170,12 +178,29 @@ void Translator::EmitScalar(const Decoder::Instruction& inst) {
IR::U32(ir.Emit(IR::ValueOpcode::BitReverse32, {value[0]}))});
return;
}
case O::S_BCNT0_I32_B32:
case O::S_FF0_I32_B32: {
const auto value = ir.BitwiseNot(ReadU32(inst.src0));
const bool count = inst.opcode == O::S_BCNT0_I32_B32;
const auto result = IR::U32(ir.Emit(count ? IR::ValueOpcode::BitCount32
: IR::ValueOpcode::FindILsb32, {value}));
WriteOperand(inst.dst, result);
if (count) {
ir.SetScc(ir.INotEqual(result, IR::U32(IR::Value(0u))));
}
return;
}
case O::S_BCNT1_I32_B32:
return SimpleInteger(inst, IR::ValueOpcode::BitCount32, IR::Type::U32, false, false,
true);
case O::S_BCNT0_I32_B64:
case O::S_BCNT1_I32_B64: {
// Vulkan bit counts operate on 32-bit words; avoid packing only to split again.
const auto value = ReadU32Pair(inst.src0);
auto value = ReadU32Pair(inst.src0);
if (inst.opcode == O::S_BCNT0_I32_B64) {
value[0] = ir.BitwiseNot(value[0]);
value[1] = ir.BitwiseNot(value[1]);
}
const auto low = IR::U32(ir.Emit(IR::ValueOpcode::BitCount32, {value[0]}));
const auto high = IR::U32(ir.Emit(IR::ValueOpcode::BitCount32, {value[1]}));
const auto result = ir.IAdd(low, high);
@@ -213,7 +238,8 @@ void Translator::EmitScalar(const Decoder::Instruction& inst) {
return ComposedIntegerBinary(inst, IR::ValueOpcode::BitwiseOr32, false, true, true);
case O::S_XNOR_B32:
return ComposedIntegerBinary(inst, IR::ValueOpcode::BitwiseXor32, false, true, true);
case O::S_FF1_I32_B64: return S_FF1_I32_B64(inst);
case O::S_FF0_I32_B64: return S_FF_I32_B64(inst, true);
case O::S_FF1_I32_B64: return S_FF_I32_B64(inst, false);
case O::S_FLBIT_I32_B32: return V_FFBH_32(inst, false);
case O::S_FLBIT_I32_B64: return S_FLBIT_I32_B64(inst);
@@ -253,7 +279,9 @@ void Translator::EmitScalar(const Decoder::Instruction& inst) {
case O::S_CBRANCH_EXECZ:
case O::S_CBRANCH_EXECNZ:
case O::S_CBRANCH_CDBGSYS:
case O::S_CBRANCH_CDBGUSER:
case O::S_CBRANCH_CDBGSYS_OR_USER:
case O::S_CBRANCH_CDBGSYS_AND_USER:
case O::S_ENDPGM: return;
default: return FailMissingTranslation(inst);
}
@@ -735,7 +735,7 @@ void Translator::WriteCompareResult(const Decoder::Operand& operand, IR::U1 valu
}
void Translator::AddBranchCondition(const CFG::Graph& graph, const CFG::BasicBlock& source,
IR::BlockInfo& info) {
IR::Block& info) {
const auto native_condition = [&](CFG::BranchCondition kind) -> IR::U1 {
IR::U1 condition;
switch (kind) {
@@ -875,30 +875,70 @@ void IncludeInstructionVectorRegisters(const Decoder::Instruction& inst, uint32_
case Decoder::Opcode::V_FLOOR_F64:
case Decoder::Opcode::V_FRACT_F64: include_vector(inst.dst, 2u); [[fallthrough]];
case Decoder::Opcode::V_CVT_F32_F64: include_vector(inst.src0, 2u); break;
case Decoder::Opcode::V_CMP_F_F64:
case Decoder::Opcode::V_CMP_LT_F64:
case Decoder::Opcode::V_CMP_EQ_F64:
case Decoder::Opcode::V_CMP_LE_F64:
case Decoder::Opcode::V_CMP_GT_F64:
case Decoder::Opcode::V_CMP_LG_F64:
case Decoder::Opcode::V_CMP_GE_F64:
case Decoder::Opcode::V_CMP_O_F64:
case Decoder::Opcode::V_CMP_U_F64:
case Decoder::Opcode::V_CMP_NGE_F64:
case Decoder::Opcode::V_CMP_NLG_F64:
case Decoder::Opcode::V_CMP_NGT_F64:
case Decoder::Opcode::V_CMP_NLE_F64:
case Decoder::Opcode::V_CMP_NEQ_F64:
case Decoder::Opcode::V_CMP_NLT_F64:
case Decoder::Opcode::V_CMP_TRU_F64:
case Decoder::Opcode::V_CMPX_F_F64:
case Decoder::Opcode::V_CMPX_LT_F64:
case Decoder::Opcode::V_CMPX_EQ_F64:
case Decoder::Opcode::V_CMPX_LE_F64:
case Decoder::Opcode::V_CMPX_GT_F64:
case Decoder::Opcode::V_CMPX_LG_F64:
case Decoder::Opcode::V_CMPX_GE_F64:
case Decoder::Opcode::V_CMP_EQ_I64:
case Decoder::Opcode::V_CMPX_O_F64:
case Decoder::Opcode::V_CMPX_U_F64:
case Decoder::Opcode::V_CMPX_NGE_F64:
case Decoder::Opcode::V_CMPX_NLG_F64:
case Decoder::Opcode::V_CMPX_NGT_F64:
case Decoder::Opcode::V_CMPX_NLE_F64:
case Decoder::Opcode::V_CMPX_NEQ_F64:
case Decoder::Opcode::V_CMPX_NLT_F64:
case Decoder::Opcode::V_CMPX_TRU_F64:
case Decoder::Opcode::V_CMP_F_I64:
case Decoder::Opcode::V_CMP_LT_I64:
case Decoder::Opcode::V_CMP_EQ_I64:
case Decoder::Opcode::V_CMP_LE_I64:
case Decoder::Opcode::V_CMP_GT_I64:
case Decoder::Opcode::V_CMP_NE_I64:
case Decoder::Opcode::V_CMP_GE_I64:
case Decoder::Opcode::V_CMP_T_I64:
case Decoder::Opcode::V_CMP_F_U64:
case Decoder::Opcode::V_CMP_LT_U64:
case Decoder::Opcode::V_CMP_EQ_U64:
case Decoder::Opcode::V_CMP_LE_U64:
case Decoder::Opcode::V_CMP_GT_U64:
case Decoder::Opcode::V_CMP_NE_U64:
case Decoder::Opcode::V_CMP_GE_U64:
case Decoder::Opcode::V_CMP_T_U64:
case Decoder::Opcode::V_CMPX_F_I64:
case Decoder::Opcode::V_CMPX_LT_I64:
case Decoder::Opcode::V_CMPX_EQ_I64:
case Decoder::Opcode::V_CMPX_LE_I64:
case Decoder::Opcode::V_CMPX_GT_I64:
case Decoder::Opcode::V_CMPX_NE_I64:
case Decoder::Opcode::V_CMPX_GE_I64:
case Decoder::Opcode::V_CMPX_T_I64:
case Decoder::Opcode::V_CMPX_F_U64:
case Decoder::Opcode::V_CMPX_LT_U64:
case Decoder::Opcode::V_CMPX_EQ_U64:
case Decoder::Opcode::V_CMPX_LE_U64:
case Decoder::Opcode::V_CMPX_GT_U64:
case Decoder::Opcode::V_CMPX_NE_U64:
case Decoder::Opcode::V_CMPX_GE_U64:
case Decoder::Opcode::V_CMPX_T_U64:
include_vector(inst.src0, 2u);
include_vector(inst.src1, 2u);
break;
@@ -1023,45 +1063,67 @@ IR::Program TranslateProgram(const Decoder::Program& decoded, const CFG::Graph&
result.block_storage.reserve(cfg.blocks.size() + 1u);
result.blocks.reserve(cfg.blocks.size() + 1u);
result.block_info.reserve(cfg.blocks.size() + 1u);
const auto max_id = std::ranges::max_element(cfg.blocks, {}, [](const CFG::BasicBlock& block) {
return block.id;
})->id;
if (max_id == UINT32_MAX) {
EXIT("cannot allocate a typed entry block id");
}
CFG::Terminator terminator;
terminator.kind = CFG::TerminatorKind::Branch;
terminator.true_block = cfg.blocks.front().id;
result.block_storage.push_back(std::make_unique<IR::Block>());
result.blocks.push_back(result.block_storage.back().get());
result.block_info.push_back({max_id + 1u, cfg.blocks.front().start_pc,
cfg.blocks.front().start_pc, std::move(terminator)});
auto& entry = *result.blocks.front();
entry.id = max_id + 1u;
entry.start_pc = entry.end_pc = cfg.blocks.front().start_pc;
entry.terminator.kind = CFG::TerminatorKind::Branch;
std::unordered_map<uint32_t, size_t> block_indices;
block_indices.reserve(cfg.blocks.size());
for (const auto& source_block: cfg.blocks) {
if (!block_indices.emplace(source_block.id, result.blocks.size()).second) {
EXIT("CFG contains duplicate block id %u", source_block.id);
std::unordered_map<uint32_t, IR::Block*> blocks_by_id;
blocks_by_id.reserve(cfg.blocks.size());
for (const auto& source: cfg.blocks) {
auto& block = *result.block_storage.emplace_back(std::make_unique<IR::Block>());
if (!blocks_by_id.emplace(source.id, &block).second) {
EXIT("CFG contains duplicate block id %u", source.id);
}
result.block_storage.push_back(std::make_unique<IR::Block>());
result.blocks.push_back(result.block_storage.back().get());
result.block_info.push_back(
{source_block.id, source_block.start_pc, source_block.end_pc, source_block.terminator});
result.blocks.push_back(&block);
block.id = source.id;
block.start_pc = source.start_pc;
block.end_pc = source.end_pc;
}
for (const auto& source_block: cfg.blocks) {
const auto source_index = block_indices.at(source_block.id);
for (const auto successor: source_block.successors) {
const auto target = block_indices.find(successor);
if (target == block_indices.end()) {
EXIT("CFG block %u has unknown successor %u", source_block.id, successor);
const auto target_block = [&](uint32_t id) -> IR::Block* {
return id == UINT32_MAX ? nullptr : blocks_by_id.at(id);
};
entry.terminator.true_block = target_block(cfg.blocks.front().id);
for (const auto& source: cfg.blocks) {
auto& block = *blocks_by_id.at(source.id);
const auto& term = source.terminator;
block.terminator.kind = term.kind;
block.terminator.true_block = target_block(term.true_block);
block.terminator.false_block = target_block(term.false_block);
block.terminator.merge_block = target_block(term.merge_block);
block.terminator.continue_block = target_block(term.continue_block);
block.terminator.loop_header = term.loop_header;
block.terminator.indexed = term.indirect_selector_code != UINT32_MAX;
const auto& values =
block.terminator.indexed ? term.indirect_selector_values : term.indirect_target_pcs;
const auto& targets =
block.terminator.indexed ? term.indirect_selector_targets : term.indirect_targets;
if (values.size() != targets.size()) {
EXIT("CFG block %u has inconsistent indirect targets", source.id);
}
block.terminator.cases.reserve(values.size());
for (size_t index = 0; index < values.size(); ++index) {
block.terminator.cases.push_back({values[index], target_block(targets[index])});
}
for (const auto successor: source.successors) {
const auto target = blocks_by_id.find(successor);
if (target == blocks_by_id.end()) {
EXIT("CFG block %u has unknown successor %u", source.id, successor);
}
result.blocks[source_index]->AddBranch(result.blocks[target->second]);
block.AddBranch(target->second);
}
}
{
result.blocks.front()->AddBranch(result.blocks.at(block_indices.at(cfg.blocks.front().id)));
IR::IREmitter entry_ir(result.blocks.front());
entry.AddBranch(entry.terminator.true_block);
IR::IREmitter entry_ir(&entry);
const auto builtin = [&](IR::StageInputKind kind, uint32_t component = 0u) {
return IR::U32(
entry_ir.Emit(IR::ValueOpcode::GetBuiltin,
@@ -1101,9 +1163,11 @@ IR::Program TranslateProgram(const Decoder::Program& decoded, const CFG::Graph&
}
entry_ir.SetExec(initial_exec);
const auto initial_mask = entry_ir.Emit(IR::ValueOpcode::Ballot, {initial_exec});
entry_ir.SetExecLo(entry_ir.CompositeExtract(initial_mask, 0));
entry_ir.SetExecHi(options.wave_size == 64u ? entry_ir.CompositeExtract(initial_mask, 1)
: IR::U32(IR::Value(0u)));
const auto initial_lo = entry_ir.CompositeExtract(initial_mask, 0);
const auto initial_hi = options.wave_size == 64u ? entry_ir.CompositeExtract(initial_mask, 1)
: IR::U32(IR::Value(0u));
entry_ir.SetExecLo(initial_lo);
entry_ir.SetExecHi(initial_hi);
if (options.stage == ShaderType::Compute) {
const auto* cs = options.input_info.compute;
const auto thread_ids =
@@ -1319,11 +1383,16 @@ IR::Program TranslateProgram(const Decoder::Program& decoded, const CFG::Graph&
}
} else if (options.stage == ShaderType::Vertex) {
// Vulkan owns primitive assembly; each vertex subgroup is one NGG wave.
// Keep its full lane extent: mbcnt(-1) uses lane ordinals, not active counts.
// ESVertCount describes input vertices, independently of lane ordinals.
auto vertex_count = IR::U32(entry_ir.Emit(IR::ValueOpcode::BitCount32, {initial_lo}));
if (options.wave_size == 64u) {
vertex_count = entry_ir.IAdd(vertex_count, IR::U32(entry_ir.Emit(
IR::ValueOpcode::BitCount32, {initial_hi})));
}
entry_ir.SetScalarReg(static_cast<IR::ScalarReg>(2),
IR::U32(IR::Value(options.wave_size << 12u)));
entry_ir.SetScalarReg(static_cast<IR::ScalarReg>(3),
IR::U32(IR::Value((1u << 28u) | options.wave_size)));
entry_ir.BitwiseOr(vertex_count, IR::U32(IR::Value(1u << 28u))));
entry_ir.SetVectorReg(static_cast<IR::VectorReg>(5),
builtin(IR::StageInputKind::VertexIndex));
entry_ir.SetVectorReg(static_cast<IR::VectorReg>(8),
@@ -1331,10 +1400,11 @@ IR::Program TranslateProgram(const Decoder::Program& decoded, const CFG::Graph&
}
}
const bool flush_f32_inputs = options.stage == ShaderType::Compute &&
(options.input_info.compute->float_mode & 0x10u) == 0;
(options.input_info.compute->float_mode & 0x10u) == 0;
for (const auto& cfg_block: cfg.blocks) {
const auto typed_index = block_indices.at(cfg_block.id);
Translator translator(result, result.blocks[typed_index], vector_limit, flush_f32_inputs);
auto* block = blocks_by_id.at(cfg_block.id);
Translator translator(result, block, vector_limit, flush_f32_inputs,
options.stage == ShaderType::Compute && !options.input_info.compute->async_compute);
for (uint32_t index = cfg_block.inst_begin; index < cfg_block.inst_end; index++) {
const auto& instruction = decoded.instructions[index];
if (IsCodeTableLoad(cfg, instruction.pc)) {
@@ -1357,7 +1427,7 @@ IR::Program TranslateProgram(const Decoder::Program& decoded, const CFG::Graph&
}
translator.TranslateInstruction(instruction);
}
translator.AddBranchCondition(cfg, cfg_block, result.block_info[typed_index]);
translator.AddBranchCondition(cfg, cfg_block, *block);
}
IR::ValidateProgram(result, false);
return result;
@@ -10,14 +10,15 @@ namespace Libs::Graphics::ShaderRecompiler::Frontend {
class Translator {
public:
Translator(IR::Program& program, IR::Block* block, uint32_t vector_limit,
bool flush_f32_inputs)
bool flush_f32_inputs, bool graphics_compute)
: program(program), ir(block), current_vector_limit(vector_limit),
flush_f32_inputs(flush_f32_inputs) {}
flush_f32_inputs(flush_f32_inputs), graphics_compute(graphics_compute) {}
void TranslateInstruction(const Decoder::Instruction& inst);
void TranslateEmbeddedFetch(const Decoder::Instruction& inst, uint32_t attribute,
uint32_t component_count, const ShaderBufferResource& resource);
void AddBranchCondition(const CFG::Graph& graph, const CFG::BasicBlock& source, IR::BlockInfo& info);
void AddBranchCondition(const CFG::Graph& graph, const CFG::BasicBlock& source,
IR::Block& info);
private:
const Decoder::Operand& SourceAt(const Decoder::Instruction& inst, uint32_t index);
@@ -121,12 +122,13 @@ private:
void EmitCompareResult(const Decoder::Instruction& inst, IR::U1 value, bool scalar, bool cmpx);
void EmitCompareConstant(const Decoder::Instruction& inst, bool value, bool scalar, bool cmpx);
void EmitIntegerCompare(const Decoder::Instruction& inst, IR::ValueOpcode opcode, IR::Type type,
bool scalar, bool cmpx);
bool scalar, bool cmpx, bool swap_operands = false);
void EmitInteger16Compare(const Decoder::Instruction& inst, IR::ValueOpcode opcode,
bool signed_value, bool cmpx);
void EmitFloatCompare(const Decoder::Instruction& inst, IR::ValueOpcode opcode, bool half,
bool cmpx);
void EmitFloatOrderedCompare(const Decoder::Instruction& inst, bool ordered, bool cmpx);
void EmitFloatOrderedCompare(const Decoder::Instruction& inst, bool ordered, bool cmpx,
bool is64 = false);
void EmitFloatClassCompare(const Decoder::Instruction& inst, bool cmpx, bool half = false);
void V_CVT_F32_UBYTE(const Decoder::Instruction& inst, uint32_t byte_index);
void V_CVT_F32_U32(const Decoder::Instruction& inst);
@@ -185,7 +187,7 @@ private:
void V_AND_OR_B32(const Decoder::Instruction& inst);
void V_OR3_B32(const Decoder::Instruction& inst);
void V_XOR3_B32(const Decoder::Instruction& inst);
void S_FF1_I32_B64(const Decoder::Instruction& inst);
void S_FF_I32_B64(const Decoder::Instruction& inst, bool find_zero);
void V_FFBH_32(const Decoder::Instruction& inst, bool sign);
void S_FLBIT_I32_B64(const Decoder::Instruction& inst);
void Integer24(const Decoder::Instruction& inst, bool sign, bool addend);
@@ -263,6 +265,7 @@ private:
uint32_t current_pc = 0;
uint32_t current_vector_limit = 1;
bool flush_f32_inputs;
bool graphics_compute;
};
} // namespace Libs::Graphics::ShaderRecompiler::Frontend
@@ -145,6 +145,23 @@ void Translator::EmitVector(const Decoder::Instruction& inst) {
case O::V_CMPX_LE_I64:
EmitIntegerCompare(inst, IR::ValueOpcode::SLessThanEqual64, IR::Type::U64, false, true);
return;
// The IR has no signed 64-bit greater-than: a > b is b < a, and a >= b is b <= a.
case O::V_CMP_GT_I64:
EmitIntegerCompare(inst, IR::ValueOpcode::SLessThan64, IR::Type::U64, false, false,
true);
return;
case O::V_CMPX_GT_I64:
EmitIntegerCompare(inst, IR::ValueOpcode::SLessThan64, IR::Type::U64, false, true,
true);
return;
case O::V_CMP_GE_I64:
EmitIntegerCompare(inst, IR::ValueOpcode::SLessThanEqual64, IR::Type::U64, false, false,
true);
return;
case O::V_CMPX_GE_I64:
EmitIntegerCompare(inst, IR::ValueOpcode::SLessThanEqual64, IR::Type::U64, false, true,
true);
return;
case O::V_CMP_LE_U64:
EmitIntegerCompare(inst, IR::ValueOpcode::ULessThanEqual64, IR::Type::U64, false, false);
return;
@@ -242,6 +259,74 @@ void Translator::EmitVector(const Decoder::Instruction& inst) {
case O::V_CMPX_GE_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPOrdGreaterThanEqual64, false, true);
return;
case O::V_CMP_F_F64: EmitCompareConstant(inst, false, false, false); return;
case O::V_CMP_TRU_F64: EmitCompareConstant(inst, true, false, false); return;
case O::V_CMPX_F_F64: EmitCompareConstant(inst, false, false, true); return;
case O::V_CMPX_TRU_F64: EmitCompareConstant(inst, true, false, true); return;
case O::V_CMP_LT_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPOrdLessThan64, false, false);
return;
case O::V_CMPX_LT_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPOrdLessThan64, false, true);
return;
case O::V_CMPX_EQ_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPOrdEqual64, false, true);
return;
case O::V_CMP_GT_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPOrdGreaterThan64, false, false);
return;
case O::V_CMPX_GT_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPOrdGreaterThan64, false, true);
return;
case O::V_CMP_LG_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPOrdNotEqual64, false, false);
return;
case O::V_CMPX_LG_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPOrdNotEqual64, false, true);
return;
case O::V_CMP_GE_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPOrdGreaterThanEqual64, false, false);
return;
case O::V_CMP_NLG_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPUnordEqual64, false, false);
return;
case O::V_CMPX_NLG_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPUnordEqual64, false, true);
return;
case O::V_CMP_NEQ_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPUnordNotEqual64, false, false);
return;
case O::V_CMPX_NEQ_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPUnordNotEqual64, false, true);
return;
case O::V_CMP_NLE_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPUnordGreaterThan64, false, false);
return;
case O::V_CMPX_NLE_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPUnordGreaterThan64, false, true);
return;
case O::V_CMP_NLT_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPUnordGreaterThanEqual64, false, false);
return;
case O::V_CMPX_NLT_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPUnordGreaterThanEqual64, false, true);
return;
case O::V_CMP_NGE_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPUnordLessThan64, false, false);
return;
case O::V_CMPX_NGE_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPUnordLessThan64, false, true);
return;
case O::V_CMP_NGT_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPUnordLessThanEqual64, false, false);
return;
case O::V_CMPX_NGT_F64:
EmitFloatCompare(inst, IR::ValueOpcode::FPUnordLessThanEqual64, false, true);
return;
case O::V_CMP_O_F64: EmitFloatOrderedCompare(inst, true, false, true); return;
case O::V_CMP_U_F64: EmitFloatOrderedCompare(inst, false, false, true); return;
case O::V_CMPX_O_F64: EmitFloatOrderedCompare(inst, true, true, true); return;
case O::V_CMPX_U_F64: EmitFloatOrderedCompare(inst, false, true, true); return;
case O::V_CMPX_EQ_F32:
EmitFloatCompare(inst, IR::ValueOpcode::FPOrdEqual32, false, true);
return;
+25
View File
@@ -1,5 +1,6 @@
#pragma once
#include "graphics/shader/recompiler/frontend/cfg/ShaderCFG.h"
#include "graphics/shader/recompiler/ir/Reg.h"
#include "graphics/shader/recompiler/ir/Value.h"
@@ -37,6 +38,8 @@ public:
[[nodiscard]] const InstructionList& Instructions() const;
[[nodiscard]] std::span<Block* const> ImmPredecessors() const;
[[nodiscard]] std::span<Block* const> ImmSuccessors() const;
[[nodiscard]] uint32_t Definition() const { return definition; }
void SetDefinition(uint32_t value) const { definition = value; }
void SsaSeal();
[[nodiscard]] bool IsSsaSealed() const;
@@ -47,6 +50,27 @@ public:
const_iterator end() const;
bool empty() const;
struct SwitchCase {
uint32_t value;
Block* target;
};
struct Terminator {
CFG::TerminatorKind kind = CFG::TerminatorKind::Return;
Block* true_block = nullptr;
Block* false_block = nullptr;
Block* merge_block = nullptr;
Block* continue_block = nullptr;
std::vector<SwitchCase> cases;
bool indexed = false;
bool loop_header = false;
};
uint32_t id = 0;
uint32_t start_pc = 0;
uint32_t end_pc = 0;
Terminator terminator;
Value condition;
Value indirect_target;
std::array<Value, NumScalarRegs> ssa_sreg_values {};
std::array<Value, NumScalarRegs> ssa_thread_bit_sreg_values {};
std::array<Value, NumScalarRegs> ssa_sreg_mask_tags {};
@@ -57,6 +81,7 @@ private:
std::vector<Block*> predecessors;
std::vector<Block*> successors;
bool ssa_sealed = false;
mutable uint32_t definition = 0;
};
using BlockList = std::vector<Block*>;
+37 -40
View File
@@ -1,5 +1,6 @@
#include "common/assert.h"
#include "graphics/shader/recompiler/ir/ShaderIR.h"
#include "graphics/shader/recompiler/BufferFormat.h"
#include <fmt/format.h>
#include <map>
@@ -202,6 +203,16 @@ Value ResolveActiveU32(Value value, Value active) {
return {};
}
uint32_t StorageBufferElementBits(const Program& program, const MemoryInfo& memory) {
if (!memory.formatted) return memory.data_bits;
const auto format = memory.typed
? Format::DecodeTBufferFormat(memory.data_format, memory.number_format)
: program.info.buffers[memory.resource].descriptor_format;
const auto info = Format::GetFormatInfo(format);
return info.type == Format::ComponentType::Unknown || info.packed_bitfield
? 32u : info.component_bits[0];
}
bool HasShaderMemoryWrites(const Program& program) {
for (const auto* block: program.blocks) {
for (const auto& inst: *block) {
@@ -219,12 +230,11 @@ bool HasShaderMemoryWrites(const Program& program) {
}
void ValidateProgram(const Program& program, bool require_ssa) {
if (program.blocks.size() != program.block_info.size() ||
program.blocks.size() != program.block_storage.size()) {
if (program.blocks.size() != program.block_storage.size()) {
return Fail("value IR block storage is inconsistent");
}
std::unordered_map<const Block*, size_t> block_indices;
std::unordered_map<uint32_t, const Block*> blocks_by_id;
std::unordered_set<uint32_t> block_ids;
std::unordered_map<const Inst*, size_t> instruction_positions;
for (size_t block_index = 0; block_index < program.blocks.size(); block_index++) {
const auto* block = program.blocks[block_index];
@@ -235,10 +245,10 @@ void ValidateProgram(const Program& program, bool require_ssa) {
if (!block_indices.emplace(block, block_index).second) {
return Fail("value IR block pointer is duplicated");
}
if (program.block_info[block_index].id == UINT32_MAX) {
if (block->id == UINT32_MAX) {
return Fail("value IR block uses the reserved exit id");
}
if (!blocks_by_id.emplace(program.block_info[block_index].id, block).second) {
if (!block_ids.insert(block->id).second) {
return Fail("value IR block id is duplicated");
}
size_t position = 0;
@@ -283,15 +293,14 @@ void ValidateProgram(const Program& program, bool require_ssa) {
}
std::unordered_set<const Block*> expected_successors;
const auto add_target = [&](uint32_t id) {
const auto found = blocks_by_id.find(id);
if (found == blocks_by_id.end()) {
const auto add_target = [&](const Block* target) {
if (!block_indices.contains(target)) {
return false;
}
expected_successors.insert(found->second);
expected_successors.insert(target);
return true;
};
const auto& terminator = program.block_info[block_index].terminator;
const auto& terminator = block->terminator;
const auto validate_control_value = [&](Value value, Type type) {
if (value.IsEmpty() || value.GetType() != type) {
return false;
@@ -309,44 +318,32 @@ void ValidateProgram(const Program& program, bool require_ssa) {
if (!add_target(terminator.true_block) || !add_target(terminator.false_block)) {
return Fail("value IR conditional branch target is missing");
}
if (!validate_control_value(program.block_info[block_index].condition, Type::U1)) {
if (!validate_control_value(block->condition, Type::U1)) {
return Fail("value IR conditional branch condition is invalid");
}
break;
case CFG::TerminatorKind::IndirectBranch: {
if (!validate_control_value(program.block_info[block_index].indirect_target,
Type::U32)) {
if (!validate_control_value(block->indirect_target, Type::U32)) {
return Fail("value IR indirect branch selector is invalid");
}
std::unordered_set<uint32_t> indirect_targets;
for (const auto target: terminator.indirect_targets) {
if (!indirect_targets.insert(target).second) {
return Fail("value IR indirect branch target is duplicated");
std::unordered_set<uint32_t> values;
for (const auto& branch: terminator.cases) {
if (!values.insert(branch.value).second) {
return Fail("value IR indirect selector value is duplicated");
}
if (!add_target(target)) {
if (!add_target(branch.target)) {
return Fail("value IR indirect branch target is missing");
}
}
if (terminator.indirect_selector_values.size() !=
terminator.indirect_selector_targets.size()) {
return Fail("value IR indirect selector table is inconsistent");
}
for (const auto target: terminator.indirect_selector_targets) {
const auto found = blocks_by_id.find(target);
if (found == blocks_by_id.end() ||
!expected_successors.contains(found->second)) {
return Fail("value IR indirect selector target is not a CFG successor");
}
}
break;
}
case CFG::TerminatorKind::Return:
case CFG::TerminatorKind::Unsupported: break;
}
if ((terminator.merge_block != UINT32_MAX &&
!blocks_by_id.contains(terminator.merge_block)) ||
(terminator.continue_block != UINT32_MAX &&
!blocks_by_id.contains(terminator.continue_block))) {
if ((terminator.merge_block != nullptr &&
!block_indices.contains(terminator.merge_block)) ||
(terminator.continue_block != nullptr &&
!block_indices.contains(terminator.continue_block))) {
return Fail("value IR structured control target is missing");
}
if (successors != expected_successors) {
@@ -653,7 +650,7 @@ void ValidateProgram(const Program& program, bool require_ssa) {
}
}
}
const auto& info = program.block_info[block_index];
const auto& info = *program.blocks[block_index];
if (!info.condition.IsEmpty() && !control_dominates(info.condition, block)) {
return Fail("value IR branch condition definition does not dominate its use");
}
@@ -664,9 +661,9 @@ void ValidateProgram(const Program& program, bool require_ssa) {
}
void ResolveControlFlowIdentities(Program& program) {
for (auto& info: program.block_info) {
info.condition = info.condition.Resolve();
info.indirect_target = info.indirect_target.Resolve();
for (auto* block: program.blocks) {
block->condition = block->condition.Resolve();
block->indirect_target = block->indirect_target.Resolve();
}
}
@@ -701,9 +698,9 @@ std::string ProgramToString(const Program& program) {
std::string text;
for (size_t block_index = 0; block_index < program.blocks.size(); block_index++) {
text += fmt::format("Block ${} pc=0x{:08x}..0x{:08x}\n", block_index,
program.block_info[block_index].start_pc,
program.block_info[block_index].end_pc);
text +=
fmt::format("Block ${} pc=0x{:08x}..0x{:08x}\n", block_index,
program.blocks[block_index]->start_pc, program.blocks[block_index]->end_pc);
for (const auto& inst: *program.blocks[block_index]) {
const auto type = inst.GetType();
if (type != Type::Void) {
+42 -32
View File
@@ -104,6 +104,7 @@ struct BufferResource {
static constexpr uint32_t NoIndirectBuffer = UINT32_MAX;
uint32_t source = 0;
uint32_t descriptor_index = UINT32_MAX;
uint32_t first_use_pc = 0;
uint32_t max_byte_extent = 0;
uint32_t packed_stride = 0;
@@ -131,6 +132,7 @@ struct ImageResource {
static constexpr uint32_t NoIndirectImage = UINT32_MAX;
uint32_t source = 0;
uint32_t descriptor_index = UINT32_MAX;
uint32_t first_use_pc = 0;
ImageResourceClass resource_class = ImageResourceClass::None;
Prospero::TextureNumericClass numeric_class = Prospero::TextureNumericClass::Unsupported;
@@ -159,6 +161,7 @@ struct SamplerResource {
uint32_t first_use_pc = 0;
// Native filtering/border variants share the original sampler's runtime descriptor.
uint32_t snapshot_index = 0;
std::vector<uint32_t> indirect_resources;
bool force_point_filtering = false;
bool depth_compare = false;
bool integer_border = false;
@@ -424,23 +427,16 @@ DescriptorBindingForImage(const ImageResource& image) {
return static_cast<DescriptorBindingKind>(base + dimension);
}
struct DescriptorBinding {
DescriptorBindingKind kind = DescriptorBindingKind::Buffers;
std::vector<uint32_t> resources;
bool operator==(const DescriptorBinding& other) const = default;
};
struct BindingLayout {
uint32_t push_data_start_dword = PushData::NoStart;
uint32_t dispatch_thread_dword = PushData::NoStart;
uint32_t memory_offset_dword = 0;
uint32_t memory_offset_count = 0;
std::vector<uint32_t> user_data_registers;
std::vector<DescriptorBinding> descriptors;
struct Bindings {
std::array<uint32_t, static_cast<size_t>(DescriptorBindingKind::Count)> descriptor_counts {};
uint64_t descriptor_mask = 0;
uint32_t push_data_start_dword = PushData::NoStart;
uint32_t dispatch_thread_dword = PushData::NoStart;
uint32_t memory_offset_dword = 0;
[[nodiscard]] uint32_t ShaderDataDwords() const {
return memory_offset_dword + (memory_offset_count + 3u) / 4u;
return memory_offset_dword +
(descriptor_counts[static_cast<size_t>(DescriptorBindingKind::Buffers)] + 3u) / 4u;
}
[[nodiscard]] bool UsesPushData() const {
return push_data_start_dword != PushData::NoStart;
@@ -451,7 +447,7 @@ struct BindingLayout {
}
}
bool operator==(const BindingLayout& other) const = default;
bool operator==(const Bindings& other) const = default;
};
struct ShaderInfo {
@@ -466,24 +462,38 @@ struct ShaderInfo {
std::vector<SampledResourcePair> sampled_pairs;
std::vector<StageInput> inputs;
std::vector<StageOutput> outputs;
std::vector<uint32_t> user_data_registers;
uint64_t live_buffers = 0;
std::array<uint8_t, 32> vertex_fetch_components {};
int32_t vertex_offset_sgpr = -1;
int32_t instance_offset_sgpr = -1;
bool has_bitwise_xor = false;
bool uses_dma = false;
bool has_bitwise_xor = false;
bool uses_dma = false;
bool uses_swizzle = false;
bool uses_lds = false;
bool uses_gds = false;
bool uses_flattened_srt = false;
bool uses_dispatch_threads = false;
bool bvh = false;
bool subgroup_ballot = false;
bool subgroup_barrier = false;
bool subgroup_shuffle = false;
bool subgroup_local_invocation_id = false;
bool compute_derivatives = false;
bool image_gather_extended = false;
bool function_lds = false;
bool function_scratch = false;
bool pixel_valid_mask = false;
bool buffer_int64_atomics = false;
bool buffer_u8 = false;
bool buffer_u16 = false;
bool shared_int64_atomics = false;
bool coherent_buffers = false;
bool float64 = false;
bool operator==(const ShaderInfo& other) const = default;
};
struct BlockInfo {
uint32_t id = 0;
uint32_t start_pc = 0;
uint32_t end_pc = 0;
CFG::Terminator terminator;
Value condition;
Value indirect_target;
};
struct DescriptorSource {
struct IndirectDescriptor {
struct SelectorRead {
@@ -542,7 +552,7 @@ struct CompiledShaderInfo {
uint32_t param_export_mask = 0;
bool has_address_writes = false;
ShaderInfo info;
BindingLayout bindings;
Bindings bindings;
};
struct UniformFillPlan {
@@ -580,8 +590,8 @@ struct ResourcePlan {
std::vector<DescriptorSource> descriptor_sources;
std::vector<ResourceBlock> control_flow;
std::vector<SrtRead> srt_reads;
std::vector<uint8_t> clean_flat_slots;
bool requires_specialization_memory = false;
std::vector<std::pair<uint64_t, uint64_t>> source_reads;
bool capture_specialization_reads = false;
bool srt_plan_complete = false;
bool resource_tracking_complete = false;
@@ -614,7 +624,6 @@ struct Program: ResourcePlan {
bool dispatcher_fallback = false;
CFG::FailureKind cfg_failure_kind = CFG::FailureKind::None;
std::string fallback_reason;
std::vector<BlockInfo> block_info;
struct ScalarWrite { uint32_t pc; ScalarReg reg; };
std::vector<ScalarWrite> scalar_writes;
// Typed memory and export instructions reference shader-local metadata by dense index.
@@ -622,11 +631,12 @@ struct Program: ResourcePlan {
std::vector<ExportInfo> export_info;
bool has_address_writes = false;
bool shader_info_complete = false;
BindingLayout bindings;
bool binding_layout_complete = false;
Bindings bindings;
};
uint32_t StorageBufferElementBits(const Program& program, const MemoryInfo& memory);
std::string ProgramToString(const Program& program);
bool HasShaderMemoryWrites(const Program& program);
+8 -1
View File
@@ -129,6 +129,12 @@ public:
}
return evaluation_index;
}
// Backend definitions belong to one emission. Wave64 emulation needs one per half.
[[nodiscard]] uint32_t Definition(uint32_t half) const { return definitions[half]; }
void SetDefinition(uint32_t half, uint32_t definition) const {
definitions[half] = definition;
}
void ResetDefinitions() const { definitions = {}; }
void SetParent(Block* block);
void SetArg(size_t index, Value value);
@@ -165,6 +171,7 @@ private:
uint8_t num_args;
bool live = false;
mutable uint32_t evaluation_index = UINT32_MAX;
mutable std::array<uint32_t, 2> definitions {};
uint64_t flags;
Block* parent = nullptr;
union {
@@ -175,6 +182,6 @@ private:
std::vector<Use> uses;
};
static_assert(sizeof(Inst) <= 112, "Inst operand storage unintentionally increased");
static_assert(sizeof(Inst) <= 120, "Inst operand and backend storage unintentionally increased");
} // namespace Libs::Graphics::ShaderRecompiler::IR
@@ -162,6 +162,15 @@ VALUE_OPCODE(FPOrdEqual32, U1, F32, F32)
VALUE_OPCODE(FPOrdEqual64, U1, F64, F64)
VALUE_OPCODE(FPOrdLessThanEqual64, U1, F64, F64)
VALUE_OPCODE(FPOrdGreaterThanEqual64, U1, F64, F64)
VALUE_OPCODE(FPOrdNotEqual64, U1, F64, F64)
VALUE_OPCODE(FPOrdLessThan64, U1, F64, F64)
VALUE_OPCODE(FPOrdGreaterThan64, U1, F64, F64)
VALUE_OPCODE(FPUnordEqual64, U1, F64, F64)
VALUE_OPCODE(FPUnordNotEqual64, U1, F64, F64)
VALUE_OPCODE(FPUnordLessThan64, U1, F64, F64)
VALUE_OPCODE(FPUnordGreaterThan64, U1, F64, F64)
VALUE_OPCODE(FPUnordLessThanEqual64, U1, F64, F64)
VALUE_OPCODE(FPUnordGreaterThanEqual64, U1, F64, F64)
VALUE_OPCODE(FPUnordEqual32, U1, F32, F32)
VALUE_OPCODE(FPOrdNotEqual32, U1, F32, F32)
VALUE_OPCODE(FPUnordNotEqual32, U1, F32, F32)
@@ -174,6 +183,7 @@ VALUE_OPCODE(FPUnordLessThanEqual32, U1, F32, F32)
VALUE_OPCODE(FPOrdGreaterThanEqual32, U1, F32, F32)
VALUE_OPCODE(FPUnordGreaterThanEqual32, U1, F32, F32)
VALUE_OPCODE(FPIsNan32, U1, F32)
VALUE_OPCODE(FPIsNan64, U1, F64)
VALUE_OPCODE(FPCmpClass32, U1, F32, U32)
VALUE_OPCODE(FPCmpClass16, U1, U32, U32)
@@ -1,197 +0,0 @@
#include "graphics/shader/recompiler/ir/passes/BindingLayout.h"
#include "common/assert.h"
#include "graphics/shader/recompiler/ir/ShaderIR.h"
#include <algorithm>
#include <array>
#include <utility>
namespace Libs::Graphics::ShaderRecompiler::IR {
namespace {
[[noreturn]] void BindingFail(const char* message) {
EXIT("shader binding layout failed: %s", message);
std::abort();
}
void CollectShaderData(const Program& program, BindingLayout& layout) {
std::array<bool, NumScalarRegs> registers {};
bool uses_dispatch_threads = false;
for (const auto* block: program.blocks) {
for (const auto& inst: *block) {
if (!inst.HasUses()) {
continue;
}
uses_dispatch_threads |= inst.GetOpcode() == ValueOpcode::GetDispatchThreadExtent;
if (inst.GetOpcode() != ValueOpcode::GetUserData) {
continue;
}
if (inst.Arg(0).GetType() != Type::ScalarReg) {
BindingFail("typed shader contains an invalid user-data register");
}
const auto index = RegIndex(inst.Arg(0).ScalarRegister());
if (index >= NumScalarRegs) {
BindingFail("typed shader contains an invalid user-data register");
}
registers[index] = true;
}
}
for (uint32_t index = 0; index < registers.size(); index++) {
if (registers[index]) {
layout.user_data_registers.push_back(index);
}
}
layout.memory_offset_dword = static_cast<uint32_t>(layout.user_data_registers.size());
if (uses_dispatch_threads) {
layout.dispatch_thread_dword = layout.memory_offset_dword;
layout.memory_offset_dword += 3u;
}
}
void AddBinding(BindingLayout& layout, DescriptorBindingKind kind,
std::vector<uint32_t> resources = {}) {
layout.descriptors.push_back({kind, std::move(resources)});
}
} // namespace
SharedMemoryResources CollectMemoryResources(const Program& program, std::vector<uint32_t>& buffers) {
std::array<bool, ShaderInfo::MaxBuffers> live_buffers {};
SharedMemoryResources shared;
for (const auto* block: program.blocks) {
for (const auto& inst: *block) {
const auto op = inst.GetOpcode();
if (BufferAccessOf(op) == BufferAccess::None &&
SharedAccessOf(op) == SharedAccess::None &&
AddressOpcodeInfoOf(op).access == AddressAccess::None) {
continue;
}
const auto index = inst.Flags<MemoryFlags>().index;
if (index >= program.memory_info.size()) {
BindingFail("typed shader contains invalid memory metadata");
}
const auto& memory = program.memory_info[index];
if (memory.planning_only) {
continue;
}
shared.lds |= memory.kind == ResourceKind::FlatLocal;
if (SharedAccessOf(op) != SharedAccess::None) {
if (memory.kind != ResourceKind::Lds && memory.kind != ResourceKind::Gds) {
BindingFail("typed shader contains invalid shared-memory metadata");
}
shared.gds |= memory.kind == ResourceKind::Gds;
shared.lds |= memory.kind == ResourceKind::Lds;
} else if (memory.kind == ResourceKind::Buffer || memory.kind == ResourceKind::ScalarBuffer) {
EXIT_IF(memory.resource >= program.info.buffers.size());
live_buffers.at(memory.resource) = true;
for (const auto child: program.info.buffers[memory.resource].indirect_resources) {
live_buffers.at(child) = true;
}
}
}
}
for (uint32_t i = 0; i < program.info.buffers.size(); i++) {
if (live_buffers.at(i)) {
buffers.push_back(i);
}
}
return shared;
}
bool UsesFlattenedSrt(const Program& program) {
const auto uses_mapping = [](const auto& resource) {
return resource.indirect_root != UINT32_MAX;
};
return std::ranges::any_of(program.blocks, [](const Block* block) {
return std::ranges::any_of(*block, [](const Inst& inst) {
return inst.GetOpcode() == ValueOpcode::ReadConst;
});
}) || std::ranges::any_of(program.info.buffers, uses_mapping) ||
std::ranges::any_of(program.info.images, uses_mapping);
}
void AllocateBindings(Program& program, uint32_t push_data_start_dword, bool lds_storage) {
if (!program.shader_info_complete || program.binding_layout_complete) {
EXIT("shader binding layout failed: %s", !program.shader_info_complete
? "shader info is not ready"
: "binding layout already allocated");
}
BindingLayout next;
std::vector<uint32_t> buffers;
const auto shared = CollectMemoryResources(program, buffers);
CollectShaderData(program, next);
next.memory_offset_count = static_cast<uint32_t>(buffers.size());
next.push_data_start_dword =
PushData::StartFor(push_data_start_dword, next.ShaderDataDwords());
if (!buffers.empty()) {
// Draw binding accesses this first group directly when memory_offset_count is nonzero.
AddBinding(next, DescriptorBindingKind::Buffers, std::move(buffers));
}
std::array<std::vector<uint32_t>, ImageBindingCount> image_groups;
for (uint32_t i = 0; i < program.info.images.size(); i++) {
const auto kind = DescriptorBindingForImage(program.info.images[i]);
if (!kind.has_value()) {
EXIT("shader binding layout failed: image %u has an invalid binding class", i);
}
const auto group = ImageBindingIndex(*kind);
if (group >= image_groups.size()) {
EXIT("shader binding layout failed: image %u has an unmapped binding class", i);
}
auto& resources = image_groups[group];
const auto dynamic = program.info.images[i].mip_mode == ImageMipMode::Dynamic;
const auto count = dynamic ? program.info.images[i].mip_count : 1u;
if (count == 0u || (!dynamic && program.info.images[i].mip_count != 1u)) {
EXIT("shader binding layout failed: image %u has invalid specialized mip count %u", i,
program.info.images[i].mip_count);
}
resources.insert(resources.end(), count, i);
}
for (uint32_t i = 0; i < image_groups.size(); i++) {
if (!image_groups[i].empty()) {
AddBinding(next, static_cast<DescriptorBindingKind>(FirstImageBinding + i),
std::move(image_groups[i]));
}
}
if (!program.info.samplers.empty()) {
std::vector<uint32_t> resources(program.info.samplers.size());
for (uint32_t i = 0; i < resources.size(); i++) {
resources[i] = i;
}
AddBinding(next, DescriptorBindingKind::Samplers, std::move(resources));
}
if (shared.gds) {
AddBinding(next, DescriptorBindingKind::Gds);
}
if (shared.lds && lds_storage) {
AddBinding(next, DescriptorBindingKind::SharedMemory);
}
if (program.info.uses_dma) {
AddBinding(next, DescriptorBindingKind::BdaPagetable);
AddBinding(next, DescriptorBindingKind::FaultBuffer);
}
if (UsesFlattenedSrt(program)) {
AddBinding(next, DescriptorBindingKind::FlattenedSrt);
}
if (next.ShaderDataDwords() != 0 && !next.UsesPushData()) {
AddBinding(next, DescriptorBindingKind::ShaderData);
}
program.bindings = std::move(next);
program.binding_layout_complete = true;
}
const DescriptorBinding* FindBinding(const BindingLayout& layout, DescriptorBindingKind kind) {
for (const auto& binding: layout.descriptors) {
if (binding.kind == kind) {
return &binding;
}
}
return nullptr;
}
} // namespace Libs::Graphics::ShaderRecompiler::IR
@@ -1,23 +0,0 @@
#ifndef EMULATOR_INCLUDE_EMULATOR_GRAPHICS_SHADER_RECOMPILER_BINDINGLAYOUT_H_
#define EMULATOR_INCLUDE_EMULATOR_GRAPHICS_SHADER_RECOMPILER_BINDINGLAYOUT_H_
#include "graphics/shader/recompiler/ir/ShaderIR.h"
namespace Libs::Graphics::ShaderRecompiler::IR {
void AllocateBindings(Program& program, uint32_t push_data_start_dword = 0,
bool lds_storage = false);
struct SharedMemoryResources {
bool lds = false;
bool gds = false;
};
SharedMemoryResources CollectMemoryResources(const Program& program, std::vector<uint32_t>& buffers);
bool UsesFlattenedSrt(const Program& program);
const DescriptorBinding* FindBinding(const BindingLayout& layout, DescriptorBindingKind kind);
} // namespace Libs::Graphics::ShaderRecompiler::IR
#endif /* EMULATOR_INCLUDE_EMULATOR_GRAPHICS_SHADER_RECOMPILER_BINDINGLAYOUT_H_ */
@@ -215,6 +215,24 @@ void FoldInstruction(Block& block, Block::iterator instruction,
return;
case ValueOpcode::SelectU32:
case ValueOpcode::SelectF32: FoldSelect(inst); return;
case ValueOpcode::FPCmpClass32: {
const auto mask = Arg(inst, 1);
if (!IsImmediate(mask, Type::U32)) {
return;
}
switch (mask.U32() & 0x3ffu) {
case 0u: Replace(inst, Value(false)); break;
case 0x3ffu: Replace(inst, Value(true)); break;
case 3u: {
const auto result = block.PrependNewInst(
instruction, ValueOpcode::FPIsNan32, {Arg(inst, 0)});
Replace(inst, Value(&*result));
break;
}
default: break;
}
return;
}
case ValueOpcode::BitFieldInsert: {
const auto base = Arg(inst, 0);
const auto insert = Arg(inst, 1);
@@ -10,7 +10,6 @@
#include <array>
#include <bit>
#include <cstdio>
#include <cstring>
#include <fmt/format.h>
#include <functional>
#include <numeric>
@@ -141,31 +140,6 @@ bool DecodeBufferDescriptor(const DescriptorValue& descriptor, ShaderBufferResou
return true;
}
struct ReadCapture {
SrtRuntime source;
std::vector<std::pair<uint64_t, uint64_t>>& ranges;
};
bool CaptureStrictRead(void* userdata, uint64_t address, std::span<uint32_t> values) {
auto& capture = *static_cast<ReadCapture*>(userdata);
if (!capture.source.read_specialization_memory(capture.source.userdata, address, values)) {
return false;
}
capture.ranges.emplace_back(address, values.size_bytes());
return true;
}
bool CaptureOrdinaryRead(void* userdata, uint64_t address, std::span<uint32_t> values) {
auto& capture = *static_cast<ReadCapture*>(userdata);
if (capture.source.read_memory != nullptr) {
if (!capture.source.read_memory(capture.source.userdata, address, values)) return false;
} else {
std::memcpy(values.data(), reinterpret_cast<const void*>(address), values.size_bytes());
}
capture.ranges.emplace_back(address, values.size_bytes());
return true;
}
const DescriptorSource* Source(const ResourcePlan& program, uint32_t source) {
if (source >= program.descriptor_sources.size()) {
return nullptr;
@@ -173,32 +147,28 @@ const DescriptorSource* Source(const ResourcePlan& program, uint32_t source) {
return &program.descriptor_sources[source];
}
void MarkCleanFlatSlots(const ResourcePlan& program, const DescriptorSource* source,
std::span<uint8_t> slots, Value extra = {}) {
void MarkCleanReads(const std::unordered_set<const Inst*>& planned_reads,
const DescriptorSource* source, Value extra = {}) {
if (source == nullptr && extra.IsEmpty()) {
return;
}
std::vector<Value> pending;
std::vector<Value> pending;
if (source != nullptr) {
pending.assign(source->dwords.begin(), source->dwords.begin() + source->dword_count);
}
if (!extra.IsEmpty()) pending.push_back(extra);
std::vector<const Inst*> visited;
while (!pending.empty()) {
auto value = pending.back().Resolve();
auto* inst = pending.back().Resolve().TryInstruction();
pending.pop_back();
const auto* inst = value.TryInstruction();
if (inst == nullptr || std::ranges::find(visited, inst) != visited.end()) {
continue;
}
visited.push_back(inst);
if (inst->GetOpcode() == ValueOpcode::ReadConst) {
const auto slot = inst->Arg(1).Resolve();
if (slot.IsImmediate() && slot.GetType() == Type::U32 && slot.U32() < slots.size()) {
slots[slot.U32()] = 1u;
pending.push_back(program.srt_reads[slot.U32()].value);
}
continue;
if (planned_reads.contains(inst)) {
auto flags = inst->Flags<SrtReadFlags>();
flags.clean = 1u;
inst->SetFlags(flags);
}
for (size_t arg = 0; arg < inst->NumArgs(); arg++) {
pending.push_back(inst->Arg(arg));
@@ -370,20 +340,22 @@ bool MaterializeIndirectDescriptor(const ResourcePlan& p
} else if (!indirect.selector_first.IsEmpty()) {
ShaderBufferResource material;
uint32_t first = 0, count = 0;
if (!DecodeBufferDescriptor(material_value, material) || material.Type() != 0u ||
material.SwizzleEnabled() ||
material.AddTid() || material.OutOfBounds() != 0u ||
uint64_t {selector->offset} + 4u > material.Stride() ||
if (!DecodeBufferDescriptor(material_value, material) ||
!clean.Evaluate(indirect.selector_first, first) ||
!clean.Evaluate(indirect.key_count, count) || count > MaxIndirectDescriptorProbes ||
uint64_t {first} + count > material.NumRecords())
!clean.Evaluate(indirect.key_count, count) || count > MaxIndirectDescriptorProbes)
return false;
if (count != 0u && (uint64_t {first} + count - 1u) * material.Stride() +
selector->offset + 4u >
uint64_t {UINT32_MAX} + 1u)
// SMEM uses its instruction's byte stride; V# stride only supplies its bounds.
const auto step = selector->stride != 0u ? selector->stride : material.Stride();
if (selector->stride == 0u &&
(material.Type() != 0u || material.SwizzleEnabled() || material.AddTid() ||
material.OutOfBounds() != 0u || uint64_t {selector->offset} + 4u > step ||
uint64_t {first} + count > material.NumRecords())) return false;
const auto end = count != 0u
? (uint64_t {first} + count - 1u) * step + selector->offset + 4u : 0u;
if (end > uint64_t {UINT32_MAX} + 1u || end > material.GetSize())
return false;
if (!read_keys(material, uint64_t {first} * step + selector->offset, step, count))
return false;
if (!read_keys(material, uint64_t {first} * material.Stride() + selector->offset,
material.Stride(), count)) return false;
} else if (!indirect.selector_mask.IsEmpty()) {
uint32_t mask = 0;
uint32_t count = 0;
@@ -663,6 +635,10 @@ bool BuildSamplerPlan(const ShaderInfo& base, SamplerPlan& plan) {
return false;
}
usage[pair.sampler] |= 1u << static_cast<uint32_t>(ClassifySampler(base.images[pair.image]));
for (const auto candidate: base.samplers[pair.sampler].indirect_resources) {
if (candidate >= base.samplers.size()) return false;
usage[candidate] |= usage[pair.sampler];
}
}
for (uint32_t index = 0; index < base.samplers.size(); index++) {
auto& mapping = plan.mapping[index];
@@ -684,30 +660,29 @@ bool BuildSamplerPlan(const ShaderInfo& base, SamplerPlan& plan) {
template <typename Predicate>
static std::vector<ResourceBlock> ResourceControlFlow(const Program& program, const Predicate& predicate) {
// The renderer checks captured scalar reads against final buffer and image write ranges.
if (program.blocks.size() != program.block_info.size() || program.has_address_writes) {
if (program.has_address_writes) {
return {};
}
std::unordered_map<uint32_t, uint32_t> indices;
for (uint32_t i = 0; i < program.block_info.size(); i++) {
if (!indices.emplace(program.block_info[i].id, i).second) {
std::unordered_map<const Block*, uint32_t> indices;
for (uint32_t i = 0; i < program.blocks.size(); i++) {
if (!indices.emplace(program.blocks[i], i).second) {
return {};
}
}
std::vector<ResourceBlock> blocks(program.blocks.size());
for (uint32_t i = 0; i < blocks.size(); i++) {
auto& block = blocks[i];
const auto& info = program.block_info[i];
const auto& info = *program.blocks[i];
const auto& terminator = info.terminator;
std::vector<uint32_t> successors;
const std::array targets {terminator.true_block, terminator.false_block};
std::span<Block* const> successors;
switch (terminator.kind) {
case CFG::TerminatorKind::Branch: successors.push_back(terminator.true_block); break;
case CFG::TerminatorKind::Branch: successors = std::span(targets).first(1); break;
case CFG::TerminatorKind::ConditionalBranch:
successors = {terminator.true_block, terminator.false_block};
successors = targets;
block.condition = info.condition;
break;
case CFG::TerminatorKind::IndirectBranch:
successors = terminator.indirect_targets;
break;
case CFG::TerminatorKind::IndirectBranch: successors = info.ImmSuccessors(); break;
case CFG::TerminatorKind::Return: break;
default: return {};
}
@@ -828,17 +803,16 @@ static std::optional<std::array<uint64_t, 3>> FillIndex(Value value, uint32_t ax
}
static UniformFillPlan AnalyzeUniformFill(const Program& program) {
if (program.stage != ShaderType::Compute || program.blocks.empty() ||
program.blocks.size() != program.block_info.size() || program.info.uses_dma ||
if (program.stage != ShaderType::Compute || program.blocks.empty() || program.info.uses_dma ||
!program.info.samplers.empty()) {
return {};
}
std::unordered_set<uint32_t> visited;
uint32_t index = 0;
const Inst* store = nullptr;
std::unordered_set<const Block*> visited;
const auto* block = program.blocks.front();
const Inst* store = nullptr;
for (;;) {
if (!visited.insert(index).second) return {};
for (const auto& inst: *program.blocks[index]) {
if (block == nullptr || !visited.insert(block).second) return {};
for (const auto& inst: *block) {
if (AddressOpcodeInfoOf(inst.GetOpcode()).access != AddressAccess::None) return {};
if (!inst.MayHaveSideEffects()) continue;
if (store != nullptr || (BufferAccessOf(inst.GetOpcode()) != BufferAccess::Write &&
@@ -846,12 +820,10 @@ static UniformFillPlan AnalyzeUniformFill(const Program& program) {
return {};
store = &inst;
}
const auto& term = program.block_info[index].terminator;
const auto& term = block->terminator;
if (term.kind == CFG::TerminatorKind::Return) break;
if (term.kind != CFG::TerminatorKind::Branch) return {};
const auto next = std::ranges::find(program.block_info, term.true_block, &BlockInfo::id);
if (next == program.block_info.end()) return {};
index = static_cast<uint32_t>(next - program.block_info.begin());
block = term.true_block;
}
if (store == nullptr || visited.size() != program.blocks.size()) return {};
for (const auto& buffer: program.info.buffers) {
@@ -935,6 +907,7 @@ ResourcePlan ExtractResourcePlan(const Program& program) {
plan.user_data_count = program.user_data_count;
plan.info = program.info;
plan.memory_info = program.memory_info;
plan.source_reads = program.source_reads;
plan.srt_plan_complete = program.srt_plan_complete;
plan.resource_tracking_complete = program.resource_tracking_complete;
@@ -945,6 +918,12 @@ ResourcePlan ExtractResourcePlan(const Program& program) {
if (source == nullptr) {
return value;
}
if (source->GetOpcode() == ValueOpcode::ReadConst) {
const auto slot = source->Arg(1).Resolve();
EXIT_IF(!slot.IsImmediate() || slot.GetType() != Type::U32 ||
slot.U32() >= program.srt_reads.size());
return Clone(program.srt_reads[slot.U32()].value);
}
if (source->GetOpcode() == ValueOpcode::ReadFirstLane &&
ValidateRuntimeValue(program, source->Arg(0))) {
// Uniform values need no new EXEC context; preserve their shared evaluation memo.
@@ -965,6 +944,11 @@ ResourcePlan ExtractResourcePlan(const Program& program) {
}
auto& target =
plan.value_storage.emplace_back(source->GetOpcode(), source->Flags<uint64_t>());
if (source->GetOpcode() == ValueOpcode::LoadAddressU32 ||
source->GetOpcode() == ValueOpcode::ReadConstBuffer ||
source->GetOpcode() == ValueOpcode::LoadBufferU32) {
target.SetFlags(SrtReadFlags {.index = source->Flags<MemoryFlags>().index});
}
cloned.emplace(source, &target);
if (source->GetOpcode() == ValueOpcode::Phi) {
for (size_t index = 0; index < source->NumArgs(); index++) {
@@ -978,12 +962,15 @@ ResourcePlan ExtractResourcePlan(const Program& program) {
return Value(&target);
};
bool capture_indirect_reads = false;
plan.descriptor_sources.reserve(program.descriptor_sources.size());
for (const auto& source: program.descriptor_sources) {
auto& target = plan.descriptor_sources.emplace_back();
target.dword_count = source.dword_count;
target.indirect_descriptor = source.indirect_descriptor;
if (target.indirect_descriptor.has_value()) {
plan.requires_specialization_memory = true;
capture_indirect_reads = true;
target.indirect_descriptor->key_count = Clone(target.indirect_descriptor->key_count);
target.indirect_descriptor->selector_first =
Clone(target.indirect_descriptor->selector_first);
@@ -1001,10 +988,15 @@ ResourcePlan ExtractResourcePlan(const Program& program) {
// A proven uniform factor can decide a branch even when its other lanes are unknown.
// Keep only that Boolean structure, never the varying shader dependency graph.
Value unknown;
bool capture_control_reads = false;
std::function<Value(Value)> ClonePredicate = [&](Value value) -> Value {
value = value.Resolve();
if (value.IsEmpty()) return {};
if (ValidateRuntimeValue(program, value, RuntimeValueType::Integer)) return Clone(value);
if (ValidateRuntimeValue(program, value, RuntimeValueType::Integer)) {
capture_control_reads |=
!ValidateRuntimeValue(program, value, RuntimeValueType::ImmutableInteger);
return Clone(value);
}
const auto* inst = value.TryInstruction();
if (inst == nullptr) return {};
const auto op = inst->GetOpcode();
@@ -1034,59 +1026,61 @@ ResourcePlan ExtractResourcePlan(const Program& program) {
for (uint32_t i = 0; i < plan.uniform_fill.fill.words; ++i) {
plan.uniform_fill.values[i] = Clone(plan.uniform_fill.values[i]);
}
plan.clean_flat_slots.resize(plan.srt_reads.size());
bool capture_indirect_reads = false;
for (const auto& source: plan.descriptor_sources) {
if (!source.indirect_descriptor.has_value()) continue;
const auto& indirect = *source.indirect_descriptor;
plan.requires_specialization_memory = true;
capture_indirect_reads = true;
if (indirect.selector)
MarkCleanFlatSlots(plan, Source(plan, indirect.selector->source), plan.clean_flat_slots);
MarkCleanFlatSlots(plan, nullptr, plan.clean_flat_slots, indirect.selector_mask);
MarkCleanFlatSlots(plan, nullptr, plan.clean_flat_slots, indirect.selector_first);
MarkCleanFlatSlots(plan, nullptr, plan.clean_flat_slots, indirect.key_count);
if (indirect.sources.empty()) {
MarkCleanFlatSlots(plan, Source(plan, indirect.table_source), plan.clean_flat_slots);
} else {
for (const auto candidate: indirect.sources) {
MarkCleanFlatSlots(plan, Source(plan, candidate), plan.clean_flat_slots);
plan.capture_specialization_reads |=
capture_indirect_reads || capture_control_reads || !plan.source_reads.empty();
if (plan.capture_specialization_reads) {
// Every former flat alias now targets its retained raw read. Mark those reads
// on the normalized graph, so discarded Phi/EXEC dependencies stay discarded.
std::unordered_set<const Inst*> planned_reads;
for (const auto& read: plan.srt_reads) {
auto* inst = read.value.ResolveInstruction();
EXIT_IF(inst == nullptr || (inst->GetOpcode() != ValueOpcode::LoadAddressU32 &&
inst->GetOpcode() != ValueOpcode::ReadConstBuffer));
planned_reads.insert(inst);
}
for (const auto& source: plan.descriptor_sources) {
if (!source.indirect_descriptor.has_value()) continue;
const auto& indirect = *source.indirect_descriptor;
if (indirect.selector)
MarkCleanReads(planned_reads, Source(plan, indirect.selector->source));
MarkCleanReads(planned_reads, nullptr, indirect.selector_mask);
MarkCleanReads(planned_reads, nullptr, indirect.selector_first);
MarkCleanReads(planned_reads, nullptr, indirect.key_count);
if (indirect.sources.empty()) {
MarkCleanReads(planned_reads, Source(plan, indirect.table_source));
} else {
for (const auto candidate: indirect.sources) {
MarkCleanReads(planned_reads, Source(plan, candidate));
}
}
}
}
plan.capture_specialization_reads |= capture_indirect_reads || !plan.control_flow.empty();
if (plan.capture_specialization_reads) {
// Clean writable addresses prove that shader writes cannot overlap captured reads.
for (const auto& buffer: plan.info.buffers) {
if (buffer.written) MarkCleanFlatSlots(plan, Source(plan, buffer.source), plan.clean_flat_slots);
if (buffer.written) MarkCleanReads(planned_reads, Source(plan, buffer.source));
}
for (const auto& image: plan.info.images) {
if (image.written) MarkCleanFlatSlots(plan, Source(plan, image.source), plan.clean_flat_slots);
if (image.written) MarkCleanReads(planned_reads, Source(plan, image.source));
}
}
if (capture_indirect_reads) plan.resource_tracking_complete &= !program.has_address_writes;
if (capture_indirect_reads || !plan.source_reads.empty())
plan.resource_tracking_complete &= !program.has_address_writes;
return plan;
}
bool MaterializeResources(const ResourcePlan& program, const SrtRuntime& runtime,
ResourceSnapshot& snapshot, ResourceSpecialization& specialization) {
if (!program.resource_tracking_complete ||
(program.requires_specialization_memory && runtime.read_specialization_memory == nullptr)) {
((program.requires_specialization_memory || !program.source_reads.empty()) &&
runtime.read_specialization_memory == nullptr)) {
return false;
}
const bool capture_reads = program.capture_specialization_reads;
const bool capture_reads = program.capture_specialization_reads || !program.source_reads.empty();
auto& reads = snapshot.specialization_reads;
reads.clear();
ReadCapture capture {runtime, reads};
SrtRuntime observed = runtime;
if (capture_reads) {
observed.userdata = &capture;
observed.read_specialization_memory = runtime.read_specialization_memory != nullptr
? CaptureStrictRead : nullptr;
observed.read_memory = CaptureOrdinaryRead;
}
reads = program.source_reads;
SrtReadCapture capture {runtime, reads};
const auto observed = capture_reads ? capture.ObservedRuntime() : runtime;
SrtWalker clean(program, CleanRuntime(observed));
SrtWalker walker(program, observed, program.clean_flat_slots,
SrtWalker walker(program, observed,
capture_reads || program.requires_specialization_memory ? &clean : nullptr);
if (!walker.RefreshFlatBuffer(snapshot.flattened_srt)) {
return false;
@@ -1210,7 +1204,14 @@ bool MaterializeResources(const ResourcePlan& program, const SrtRuntime& runtime
}
snapshot.samplers.resize(program.info.samplers.size());
for (uint32_t i = 0; i < program.info.samplers.size(); ++i) {
if (!evaluate(program.info.samplers[i].source, snapshot.samplers[i])) {
auto source = program.info.samplers[i].source;
if (source >= program.descriptor_sources.size()) return false;
const auto& descriptor = program.descriptor_sources[source];
if (descriptor.indirect_descriptor) {
if (descriptor.indirect_descriptor->sources.empty()) return false;
source = descriptor.indirect_descriptor->sources[0];
}
if (!evaluate(source, snapshot.samplers[i])) {
return false;
}
if (program.info.samplers[i].gather_lod) {
@@ -1229,8 +1230,7 @@ bool MaterializeResources(const ResourcePlan& program, const SrtRuntime& runtime
}
void ApplyResourceSpecialization(Program& program, const ResourceSpecialization& specialization) {
EXIT_IF(!program.resource_tracking_complete || program.shader_info_complete ||
program.binding_layout_complete);
EXIT_IF(!program.resource_tracking_complete || program.shader_info_complete);
EXIT_IF(program.info.buffers.size() > specialization.buffers.size() ||
program.info.images.size() > specialization.images.size());
@@ -1303,11 +1303,20 @@ void ApplyResourceSpecialization(Program& program, const ResourceSpecialization&
samplers[index].force_point_filtering = binding.type == SamplerClass::PointInteger;
samplers[index].integer_border = binding.type != SamplerClass::Float;
}
for (uint32_t index = 0; index < sampler_plan.sampler_count; index++) {
const auto& binding = sampler_plan.bindings[index];
for (auto& candidate: samplers[index].indirect_resources) {
candidate = sampler_plan.mapping[candidate][static_cast<uint32_t>(binding.type)];
EXIT_IF(candidate == UINT32_MAX);
}
}
for (auto& pair: sampled_pairs) {
const auto type = static_cast<uint32_t>(ClassifySampler(images[pair.image]));
pair.sampler = sampler_plan.mapping[pair.sampler][type];
EXIT_IF(pair.sampler == UINT32_MAX);
samplers[pair.sampler].depth_compare |= images[pair.image].depth_compare;
for (const auto candidate: samplers[pair.sampler].indirect_resources)
samplers[candidate].depth_compare |= images[pair.image].depth_compare;
}
auto& memory_info = program.memory_info;
@@ -1395,7 +1404,8 @@ void ApplyResourceSpecialization(Program& program, const ResourceSpecialization&
EXIT_IF(memory.sampler == UINT32_MAX);
}
EXIT_IF(image.indirect_root == memory.resource &&
inst.GetOpcode() != ValueOpcode::ImageSampleRaw);
inst.GetOpcode() != ValueOpcode::ImageSampleRaw &&
inst.GetOpcode() != ValueOpcode::ImageRead);
}
}
for (auto& memory: memory_info) {
@@ -430,7 +430,7 @@ public:
plan.handle->SetArg(word, Value(0u));
continue;
}
if (plan.handle->GetOpcode() == ValueOpcode::GetBufferResource) {
if (plan.handle->NumArgs() == 4u) {
plan.handle->SetArg(0, plan.key);
for (uint32_t word = 1; word < 4u; ++word) plan.handle->SetArg(word, Value(0u));
} else {
@@ -609,24 +609,37 @@ private:
return selected.IsEmpty() ? value : selected;
}
Value LowerDescriptorPhi(Value value) {
value = value.Resolve();
const auto* phi = value.TryInstruction();
if (m_shader_writes || phi == nullptr || phi->GetOpcode() != ValueOpcode::Phi ||
phi->NumArgs() != 2u || phi->NumPhiBlocks() != 2u || phi->GetType() != Type::U32 ||
m_program.blocks.size() != m_program.block_info.size()) {
return value;
}
const auto* merge = phi->Parent();
const auto* branch = phi->PhiBlock(0);
if (merge == nullptr || branch == nullptr || phi->PhiBlock(1) == nullptr ||
branch == phi->PhiBlock(1)) {
return value;
}
for (const auto& [original, selected]: m_descriptor_selections) {
if (original == phi) {
return selected;
Value LowerDescriptorValue(Value value) {
value = value.Resolve();
const auto* inst = value.TryInstruction();
if (inst == nullptr) return value;
// The scalar planner owns reads and lowers their address handles separately.
uint32_t memory_index = 0;
if (ScalarReadMemory(*inst, memory_index) != nullptr) return value;
const auto [cached, inserted] = m_descriptor_values.emplace(inst, value);
if (!inserted) return cached->second;
if (inst->GetOpcode() != ValueOpcode::Phi) {
Inst* projected = nullptr;
for (size_t arg = 0; arg < inst->NumArgs(); ++arg) {
const auto lowered = LowerDescriptorValue(inst->Arg(arg));
if (lowered == inst->Arg(arg).Resolve()) continue;
if (projected == nullptr) {
projected = &m_program.value_storage.emplace_back(inst->GetOpcode(), inst->Flags<uint64_t>());
for (size_t i = 0; i < inst->NumArgs(); ++i) projected->SetArg(i, inst->Arg(i));
}
projected->SetArg(arg, lowered);
}
if (projected != nullptr) cached->second = Value(projected);
return cached->second;
}
if (inst->NumArgs() != 2u || inst->NumPhiBlocks() != 2u || inst->GetType() != Type::U32) {
return value;
}
const auto* merge = inst->Parent();
const auto* branch = inst->PhiBlock(0);
if (merge == nullptr || branch == nullptr || inst->PhiBlock(1) == nullptr ||
branch == inst->PhiBlock(1)) {
return value;
}
if (branch->ImmSuccessors().size() != 2u) {
if (branch->ImmPredecessors().size() != 1u) {
@@ -637,9 +650,9 @@ private:
if (branch == merge || branch->ImmSuccessors().size() != 2u) {
return value;
}
std::array<uint32_t, 2> target_ids;
std::array<const Block*, 2> targets;
for (uint32_t arm = 0; arm < 2; arm++) {
const auto* incoming = phi->PhiBlock(arm);
const auto* incoming = inst->PhiBlock(arm);
if (incoming == merge ||
(incoming != branch &&
(incoming->ImmPredecessors().size() != 1u ||
@@ -648,35 +661,27 @@ private:
incoming->ImmSuccessors()[0] != merge))) {
return value;
}
const auto* target = incoming == branch ? merge : incoming;
const auto it = std::ranges::find(m_program.blocks, target);
if (it == m_program.blocks.end()) {
return value;
}
target_ids[arm] = m_program.block_info[it - m_program.blocks.begin()].id;
targets[arm] = incoming == branch ? merge : incoming;
}
const auto branch_it = std::ranges::find(m_program.blocks, branch);
if (branch_it == m_program.blocks.end()) {
return value;
}
const auto& info = m_program.block_info[branch_it - m_program.blocks.begin()];
const auto& info = *branch;
const auto& term = info.terminator;
if (term.kind != CFG::TerminatorKind::ConditionalBranch ||
!((term.true_block == target_ids[0] && term.false_block == target_ids[1]) ||
(term.false_block == target_ids[0] && term.true_block == target_ids[1])) ||
!ValidateRuntimeValue(m_program, info.condition, RuntimeValueType::Integer) ||
!ValidateRuntimeValue(m_program, phi->Arg(0)) ||
!ValidateRuntimeValue(m_program, phi->Arg(1))) {
!((term.true_block == targets[0] && term.false_block == targets[1]) ||
(term.false_block == targets[0] && term.true_block == targets[1])) ||
!ValidateRuntimeValue(m_program, info.condition, m_shader_writes
? RuntimeValueType::ImmutableInteger : RuntimeValueType::Integer) ||
!ValidateRuntimeValue(m_program, inst->Arg(0)) ||
!ValidateRuntimeValue(m_program, inst->Arg(1))) {
return value;
}
// Retain a host expression; replacing the GPU Phi would break SSA dominance.
const auto true_arg = term.true_block == target_ids[0] ? 0u : 1u;
const auto true_arg = term.true_block == targets[0] ? 0u : 1u;
auto& selected = m_program.value_storage.emplace_back(ValueOpcode::SelectU32);
selected.SetArg(0, info.condition);
selected.SetArg(1, phi->Arg(true_arg));
selected.SetArg(2, phi->Arg(true_arg ^ 1u));
m_descriptor_selections.emplace_back(phi, Value(&selected));
return Value(&selected);
selected.SetArg(1, inst->Arg(true_arg));
selected.SetArg(2, inst->Arg(true_arg ^ 1u));
cached->second = Value(&selected);
return cached->second;
}
void MakeSource(const Inst& handle, uint32_t width, bool sampler, bool sample_adjust,
@@ -696,7 +701,7 @@ private:
for (uint32_t i = 0; i < width; i++) {
const auto value = base_reg != UINT32_MAX
? NativeDescriptorSource(handle.Arg(i), base_reg + i, pc) : handle.Arg(i);
descriptor.dwords[i] = LowerDescriptorPhi(value);
descriptor.dwords[i] = LowerDescriptorValue(value);
}
if (sample_adjust) {
descriptor.dwords[3] = CanonicalizeSampleAdjustDword3(descriptor.dwords[3]);
@@ -852,9 +857,9 @@ private:
if (entry.source.dwords[word].Resolve() == Value(read))
entry.source.dwords[word] = flat;
}
for (auto& info: m_program.block_info) {
if (info.condition.Resolve() == Value(read)) info.condition = flat;
if (info.indirect_target.Resolve() == Value(read)) info.indirect_target = flat;
for (auto* block: m_program.blocks) {
if (block->condition.Resolve() == Value(read)) block->condition = flat;
if (block->indirect_target.Resolve() == Value(read)) block->indirect_target = flat;
}
if (keep) {
if (resolved->planning_handle.IsEmpty()) {
@@ -1022,9 +1027,6 @@ private:
};
bool NonzeroOnEntry(Value value, const Block* block) const {
if (m_program.blocks.size() != m_program.block_info.size()) {
return false;
}
// Each unique predecessor must execute before this use. Stop at joins: an
// unrelated comparison is not a bound on FindILsb's zero-input sentinel.
for (size_t depth = 0; block != nullptr && depth < m_program.blocks.size(); ++depth) {
@@ -1098,15 +1100,11 @@ private:
}
std::optional<EdgePredicate> ConditionalEdge(const Block* from, const Block* to) const {
const auto position = std::ranges::find(m_program.blocks, from);
const auto target = std::ranges::find(m_program.blocks, to);
if (position == m_program.blocks.end() || target == m_program.blocks.end()) return {};
const auto& info = m_program.block_info[position - m_program.blocks.begin()];
const auto& term = info.terminator;
const auto id = m_program.block_info[target - m_program.blocks.begin()].id;
const auto& term = from->terminator;
if (term.kind != CFG::TerminatorKind::ConditionalBranch ||
(term.true_block == id) == (term.false_block == id)) return {};
EdgePredicate edge {info.condition, term.true_block == id};
(term.true_block == to) == (term.false_block == to))
return {};
EdgePredicate edge {from->condition, term.true_block == to};
while (const auto* inst = edge.condition.Resolve().TryInstruction()) {
if (inst->GetOpcode() == ValueOpcode::LogicalNot) {
edge.positive = !edge.positive;
@@ -1470,9 +1468,9 @@ private:
const Inst* BoundedLoop(Value key, const Block* use, const auto& accepts_bound) const {
const auto* phi = key.Resolve().TryInstruction();
if (phi == nullptr || phi->GetOpcode() != ValueOpcode::Phi ||
phi->GetType() != Type::U32 || phi->NumArgs() != 2u || phi->NumPhiBlocks() != 2u ||
m_program.blocks.size() != m_program.block_info.size()) return {};
if (phi == nullptr || phi->GetOpcode() != ValueOpcode::Phi || phi->GetType() != Type::U32 ||
phi->NumArgs() != 2u || phi->NumPhiBlocks() != 2u)
return {};
const Block* increment_block = nullptr;
uint32_t initial_arm = 0;
for (uint32_t initial = 0; initial < 2u; ++initial) {
@@ -1480,7 +1478,18 @@ private:
const auto* step = phi->Arg(initial ^ 1u).Resolve().TryInstruction();
if (!zero.IsImmediate() || zero.GetType() != Type::U32 || zero.U32() != 0u ||
step == nullptr || step->GetOpcode() != ValueOpcode::IAdd32 ||
step->Parent() != phi->PhiBlock(initial ^ 1u)) continue;
step->Parent() == nullptr) continue;
// Structurization can forward the native increment through a synthetic continue block.
const auto* incoming = phi->PhiBlock(initial ^ 1u);
const auto* next = phi->Parent();
for (size_t edges = 0; incoming != step->Parent() && edges < m_program.blocks.size(); ++edges) {
if (incoming == nullptr || incoming->terminator.kind != CFG::TerminatorKind::Branch ||
incoming->terminator.true_block != next || incoming->ImmPredecessors().size() != 1u)
break;
next = incoming;
incoming = incoming->ImmPredecessors()[0];
}
if (incoming != step->Parent()) continue;
uint32_t increment = 0;
if ((step->Arg(0).Resolve() == key &&
ImmediateU32(step->Arg(1), increment) && increment == 1u) ||
@@ -1518,6 +1527,30 @@ private:
});
}
bool MatchScalarMaterialKey(Value key, DescriptorSource::IndirectDescriptor& indirect,
DescriptorSource& material_source) {
const auto* read = key.Resolve().TryInstruction();
if (read == nullptr || read->GetOpcode() != ValueOpcode::ReadConstBuffer) return false;
uint32_t memory_index = 0;
const auto* memory = ScalarReadMemory(*read, memory_index);
Value index;
uint32_t offset = 0, stride = 0;
if (memory == nullptr || memory->kind != ResourceKind::ScalarBuffer ||
memory->offset > INT32_MAX || !MemoryIndexBelongsTo(memory_index, *read) ||
!MatchTableOffset(read->Arg(1), index, offset, stride) ||
((offset | stride | memory->offset) & 3u) != 0u ||
uint64_t {offset} + memory->offset > UINT32_MAX) return false;
const auto* bound = BoundedLoop(index, read->Parent());
if (bound == nullptr || bound->GetOpcode() != ValueOpcode::ULessThan32 ||
!MakeRuntimeTableSource(*read, material_source)) return false;
indirect.selector.emplace(DescriptorSource::IndirectDescriptor::SelectorRead {
.source = InternSource(material_source), .stride = stride,
.offset = offset + memory->offset});
indirect.selector_first = Value(0u);
indirect.key_count = bound->Arg(1);
return true;
}
void FoldBoundedLoopSelectors() {
for (auto* header: m_program.blocks) {
for (auto& induction: *header) {
@@ -1753,6 +1786,10 @@ private:
}
if ((table_offset & 3u) != 0u ||
(bitscan && table_offset > UINT32_MAX - (32u * 32u - 1u))) return false;
} else if (!plan.table_indexed &&
MatchScalarMaterialKey(key, indirect, material_source)) {
// The host enumerates the guarded scalar loop; its key and T# payload stay on the GPU.
plan.retain_reads = true;
} else {
// A bounded V# supplies the complete image table. Leave every GPU selector
// and descriptor read in the shader; the host only translates table bytes.
@@ -1904,38 +1941,33 @@ private:
}
bool ImpossibleSwitchEdge(const Block* from, const Block* to) const {
const auto from_it = std::ranges::find(m_program.blocks, from);
const auto to_it = std::ranges::find(m_program.blocks, to);
if (from_it == m_program.blocks.end() || to_it == m_program.blocks.end() ||
m_program.block_info.size() != m_program.blocks.size()) return false;
const auto& info = m_program.block_info[from_it - m_program.blocks.begin()];
const auto& term = info.terminator;
if (term.kind != CFG::TerminatorKind::IndirectBranch ||
term.indirect_selector_code == UINT32_MAX ||
term.indirect_selector_values.empty() ||
term.indirect_selector_values.size() != term.indirect_selector_targets.size()) return false;
const auto target = m_program.block_info[to_it - m_program.blocks.begin()].id;
const auto maximum = SelectorMaximum(info.indirect_target);
const auto& term = from->terminator;
if (term.kind != CFG::TerminatorKind::IndirectBranch || !term.indexed) return false;
const auto maximum = SelectorMaximum(from->indirect_target);
bool found = false;
for (size_t i = 0; i < term.indirect_selector_values.size(); ++i) {
if (term.indirect_selector_targets[i] != target) continue;
if (term.indirect_selector_values[i] <= maximum) return false;
for (const auto& branch: term.cases) {
if (branch.target != to) continue;
if (branch.value <= maximum) return false;
found = true;
}
return found;
}
bool TryMakeFiniteImage(Inst& handle, IndirectDescriptorPlan& plan) {
if (handle.GetOpcode() != ValueOpcode::GetImageResource || handle.NumArgs() != 8u)
bool TryMakeFiniteDescriptor(Inst& handle, IndirectDescriptorPlan& plan) {
const bool sampler = handle.GetOpcode() == ValueOpcode::GetSamplerResource;
if ((!sampler && handle.GetOpcode() != ValueOpcode::GetImageResource) ||
handle.NumArgs() != (sampler ? 4u : 8u))
return false;
if (std::ranges::any_of(handle.Uses(), [](const Use& use) {
if (std::ranges::any_of(handle.Uses(), [&](const Use& use) {
const auto op = use.user->GetOpcode();
return op != ValueOpcode::ImageSampleRaw && op != ValueOpcode::ImageGatherRaw;
if (op != ValueOpcode::ImageSampleRaw) return sampler || op != ValueOpcode::ImageGatherRaw;
return sampler && (m_program.memory_info[use.user->Flags<MemoryFlags>().index]
.image_sample_flags & Decoder::ImageSampleFlagAdjust) != 0u;
})) return false;
bool has_phi = false;
for (size_t word = 0; word < handle.NumArgs(); ++word) has_phi |= handle.Arg(word).Resolve().IsPhi();
if (!has_phi) return false;
// Normalize the eight synchronized descriptor words to one GPU ordinal.
// Normalize synchronized descriptor words to one GPU ordinal.
// Build the complete graph before changing IR so an unsupported leaf is transactional.
struct Choice {
DescriptorSource descriptor;
@@ -1951,7 +1983,15 @@ private:
const auto index = static_cast<uint32_t>(choices.size());
choices.push_back({descriptor});
uint32_t bad = 0;
if (ValidateSource(descriptor, bad)) return index;
if (ValidateSource(descriptor, bad)) {
// Finite inline samplers have literal leaves; live descriptor tables use the
// existing direct sampler planning path.
if (sampler && !std::ranges::all_of(
std::span(descriptor.dwords).first(descriptor.dword_count),
[](Value word) { return word.IsImmediate() && word.GetType() == Type::U32; }))
return UINT32_MAX;
return index;
}
const auto* branch = descriptor.dwords[bad].TryInstruction();
if (branch == nullptr || branch->Parent() == nullptr ||
branch->GetOpcode() != ValueOpcode::Phi ||
@@ -1988,14 +2028,14 @@ private:
return index;
};
DescriptorSource root;
root.dword_count = 8u;
root.dword_count = handle.NumArgs();
for (uint32_t word = 0; word < root.dword_count; ++word) root.dwords[word] = handle.Arg(word);
if (visit(visit, root) == UINT32_MAX || choices.front().branch == nullptr) return false;
DescriptorSource image_source;
image_source.dword_count = 8u;
image_source.dwords.fill(Value(0u));
image_source.indirect_descriptor.emplace(DescriptorSource::IndirectDescriptor{});
auto& sources = image_source.indirect_descriptor->sources;
DescriptorSource finite_source;
finite_source.dword_count = root.dword_count;
finite_source.dwords.fill(Value(0u));
finite_source.indirect_descriptor.emplace(DescriptorSource::IndirectDescriptor{});
auto& sources = finite_source.indirect_descriptor->sources;
for (auto& choice: choices) {
if (choice.branch != nullptr) continue;
const auto source = InternSource(choice.descriptor);
@@ -2024,9 +2064,9 @@ private:
}
}
plan.handle = &handle;
plan.source = InternSource(image_source);
plan.source = InternSource(finite_source);
plan.key = choices.front().key;
plan.roots = image_source.dwords;
plan.roots = finite_source.dwords;
plan.reads.fill(nullptr);
return true;
}
@@ -2057,27 +2097,33 @@ private:
inst.NumArgs() == 0u) {
continue;
}
auto* handle = inst.Arg(0).Resolve().TryInstruction();
if (handle == nullptr || FindIndirectDescriptor(*handle) != nullptr) {
continue;
}
const auto flags = inst.Flags<MemoryFlags>();
const auto& memory = m_program.memory_info[flags.index];
DescriptorSource descriptor;
MakeSource(*handle, inst.GetOpcode() == ValueOpcode::StoreBufferU32 ? 4u : 8u,
false, false, memory.resource * 4u, descriptor, flags.pc);
uint32_t bad_dword = 0;
if (ValidateSource(descriptor, bad_dword)) continue;
IndirectDescriptorPlan plan;
if (TryMakeIndirectImage(*handle, descriptor, plan) || TryMakeFiniteImage(*handle, plan) ||
TryMakeIndirectBuffer(*handle, descriptor, plan)) {
for (const auto& previous: m_indirect_descriptors) {
if (plan.reads[0] != nullptr && previous.reads[0] == plan.reads[0] &&
!EquivalentValue(m_program, previous.key, plan.key))
Fail(flags.pc, "shared descriptor read has incompatible keys");
const auto plan_handle = [&](Value value, uint32_t width, uint32_t base, bool sampler) {
auto* handle = value.Resolve().TryInstruction();
if (handle == nullptr || FindIndirectDescriptor(*handle) != nullptr) return;
DescriptorSource descriptor;
MakeSource(*handle, width, sampler, false, base, descriptor, flags.pc);
uint32_t bad_dword = 0;
if (ValidateSource(descriptor, bad_dword)) return;
IndirectDescriptorPlan plan;
if (sampler ? TryMakeFiniteDescriptor(*handle, plan)
: (TryMakeIndirectImage(*handle, descriptor, plan) ||
TryMakeFiniteDescriptor(*handle, plan) ||
TryMakeIndirectBuffer(*handle, descriptor, plan))) {
for (const auto& previous: m_indirect_descriptors) {
if (plan.reads[0] != nullptr && previous.reads[0] == plan.reads[0] &&
!EquivalentValue(m_program, previous.key, plan.key))
Fail(flags.pc, "shared descriptor read has incompatible keys");
}
m_indirect_descriptors.push_back(std::move(plan));
}
m_indirect_descriptors.push_back(std::move(plan));
}
};
plan_handle(inst.Arg(0), inst.GetOpcode() == ValueOpcode::StoreBufferU32 ? 4u : 8u,
memory.resource * 4u, false);
if (inst.GetOpcode() == ValueOpcode::ImageSampleRaw &&
(memory.image_sample_flags & Decoder::ImageSampleFlagAdjust) == 0u)
plan_handle(inst.Arg(1), 4u, memory.sampler * 4u, true);
}
}
}
@@ -2206,7 +2252,20 @@ private:
return UINT32_MAX;
}
m_info.samplers.push_back({source, pc});
return static_cast<uint32_t>(m_info.samplers.size() - 1);
const auto root = static_cast<uint32_t>(m_info.samplers.size() - 1);
const auto* descriptor = Source(source);
if (descriptor != nullptr && descriptor->indirect_descriptor) {
const auto& sources = descriptor->indirect_descriptor->sources;
if (sources.empty()) Fail(pc, "finite sampler has no candidates");
std::vector<uint32_t> resources {root};
for (size_t i = 1; i < sources.size(); ++i) {
const auto candidate = AddSampler(sources[i], pc);
if (candidate == UINT32_MAX) return UINT32_MAX;
resources.push_back(candidate);
}
m_info.samplers[root].indirect_resources = std::move(resources);
}
return root;
}
void AddSampledPair(uint32_t image, uint32_t sampler, uint32_t pc) {
@@ -2366,8 +2425,15 @@ private:
uint32_t sampler_source = 0;
const bool sample_adjust =
(memory.image_sample_flags & Decoder::ImageSampleFlagAdjust) != 0;
GetHandle(inst.Arg(1), ValueOpcode::GetSamplerResource, 4, flags.pc,
memory.sampler * 4u, sampler_handle, sampler_source, true, sample_adjust);
sampler_handle = inst.Arg(1).Resolve().TryInstruction();
const auto* selected_sampler = sampler_handle != nullptr
? FindIndirectDescriptor(*sampler_handle) : nullptr;
if (selected_sampler != nullptr) {
sampler_source = selected_sampler->source;
} else {
GetHandle(inst.Arg(1), ValueOpcode::GetSamplerResource, 4, flags.pc,
memory.sampler * 4u, sampler_handle, sampler_source, true, sample_adjust);
}
sampler = AddSampler(sampler_source, flags.pc);
if (sampler == UINT32_MAX) {
Fail(flags.pc, "sampler resource limit exceeded");
@@ -2423,7 +2489,7 @@ private:
std::vector<HandlePatch> m_handle_patches;
std::vector<MemoryPatch> m_memory_patches;
std::vector<IndirectDescriptorPlan> m_indirect_descriptors;
std::vector<std::pair<const Inst*, Value>> m_descriptor_selections;
std::map<const Inst*, Value> m_descriptor_values;
bool m_shader_writes = false;
};
@@ -75,128 +75,11 @@ void ValidateOptions(const Program& program, ShaderStageInputInfo input_info) {
}
}
void ValidateValueReferences(const Program& program, ShaderStageInputInfo input_info) {
for (const auto* block: program.blocks) {
for (const auto& inst: *block) {
switch (inst.GetOpcode()) {
case ValueOpcode::GetAttribute: {
if (!inst.Arg(0).IsImmediate() || inst.Arg(0).GetType() != Type::U32 ||
!inst.Arg(1).IsImmediate() || inst.Arg(1).GetType() != Type::U32) {
return Fail("typed attribute reference is not constant");
}
if ((program.stage == ShaderType::Vertex ||
program.stage == ShaderType::Local) &&
(inst.Arg(1).U32() >= 4u ||
inst.Arg(0).U32() >=
static_cast<uint32_t>(input_info.vertex->resources_num))) {
return Fail("vertex input reference is out of range");
}
if (program.stage == ShaderType::Pixel &&
(inst.Arg(1).U32() >= 4u ||
inst.Arg(0).U32() >= input_info.pixel->input_num)) {
return Fail("pixel input reference is out of range");
}
break;
}
case ValueOpcode::GetInterpolationParameter: {
if (program.stage != ShaderType::Pixel || !inst.Arg(0).IsImmediate() ||
inst.Arg(0).GetType() != Type::U32 || !inst.Arg(1).IsImmediate() ||
inst.Arg(1).GetType() != Type::U32 || !inst.Arg(2).IsImmediate() ||
inst.Arg(2).GetType() != Type::U32) {
return Fail("interpolation parameter reference is invalid");
}
if (inst.Arg(0).U32() >= input_info.pixel->input_num ||
inst.Arg(1).U32() >= 4u || inst.Arg(2).U32() >= 3u) {
return Fail("interpolation parameter reference is out of range");
}
break;
}
case ValueOpcode::GetBuiltin: {
if (!inst.Arg(0).IsImmediate() || inst.Arg(0).GetType() != Type::U32 ||
!inst.Arg(1).IsImmediate() || inst.Arg(1).GetType() != Type::U32) {
return Fail("typed builtin reference is not constant");
}
const auto kind = static_cast<StageInputKind>(inst.Arg(0).U32());
const auto component = inst.Arg(1).U32();
switch (kind) {
case StageInputKind::PackedAncillary:
return Fail("packed pixel ancillary input has an unsupported live use");
case StageInputKind::Layer:
case StageInputKind::SampleId:
if (program.stage != ShaderType::Pixel || component != 0u) {
return Fail("typed pixel scalar input is invalid");
}
break;
case StageInputKind::VertexIndex:
case StageInputKind::InvocationId:
case StageInputKind::PrimitiveId:
case StageInputKind::InstanceIndex:
case StageInputKind::FrontFacing:
case StageInputKind::LocalInvocationIndex:
if (component != 0u) {
return Fail("typed scalar builtin component is out of range");
}
break;
case StageInputKind::FragCoord:
if (component >= 4u) {
return Fail("typed fragment-coordinate component is out of range");
}
break;
case StageInputKind::BaryCoordSmooth:
case StageInputKind::BaryCoordSmoothSample:
case StageInputKind::BaryCoordSmoothCentroid:
case StageInputKind::BaryCoordNoPerspective:
if (component >= 2u) {
return Fail("typed barycentric component is out of range");
}
break;
case StageInputKind::TessCoord:
case StageInputKind::WorkgroupId:
case StageInputKind::LocalInvocationId:
case StageInputKind::GlobalInvocationId:
if (component >= 3u) {
return Fail("typed invocation builtin component is out of range");
}
break;
case StageInputKind::Parameter:
default: return Fail("typed builtin kind is invalid");
}
break;
}
case ValueOpcode::SetAttribute: {
const auto index = inst.Flags<ExportFlags>().index;
if (index >= program.export_info.size()) {
return Fail("typed export metadata index is out of range");
}
const auto& exp = program.export_info[index];
if (exp.kind == ExportTargetKind::Position && exp.index != 0 && exp.en != 0 &&
program.stage != ShaderType::Vertex && program.stage != ShaderType::Mesh &&
program.stage != ShaderType::TessellationEvaluation) {
return Fail("auxiliary position export requires a vertex, mesh, or "
"tessellation evaluation shader");
}
break;
}
default: break;
}
}
}
}
void CollectVertexInputs(const Program& program, const ShaderVertexInputInfo* vertex,
ShaderInfo& info) {
void CollectVertexInputs(const ShaderVertexInputInfo* vertex, ShaderInfo& info,
const std::array<uint8_t, 32>& used_components) {
AddInput(info, StageInputKind::VertexIndex, 0, 1, "gl_VertexIndex");
AddInput(info, StageInputKind::InstanceIndex, 0, 1, "gl_InstanceIndex");
uint32_t used_components[ShaderVertexInputInfo::RES_MAX] = {};
for (const auto* block: program.blocks) {
for (const auto& inst: *block) {
if (inst.GetOpcode() == ValueOpcode::GetAttribute) {
const auto attr = inst.Arg(0).U32();
const auto chan = inst.Arg(1).U32();
used_components[attr] = std::max(used_components[attr], chan + 1u);
}
}
}
for (uint32_t attr = 0; attr < static_cast<uint32_t>(vertex->resources_num); attr++) {
if (used_components[attr] != 0) {
AddInput(info, StageInputKind::Parameter, attr, used_components[attr],
@@ -205,37 +88,25 @@ void CollectVertexInputs(const Program& program, const ShaderVertexInputInfo* ve
}
}
void CollectPixelInputs(const Program& program, const ShaderPixelInputInfo* pixel,
ShaderInfo& info) {
void CollectPixelInputs(const ShaderPixelInputInfo* pixel, ShaderInfo& info,
std::array<bool, 32>& per_vertex,
const std::array<uint8_t, 32>& used_components) {
if (pixel->HasPositionInput()) {
AddInput(info, StageInputKind::FragCoord, 0, 4, "gl_FragCoord");
}
if (pixel->ps_front_face) {
AddInput(info, StageInputKind::FrontFacing, 0, 1, "gl_FrontFacing");
}
std::array<bool, 32> per_vertex {};
std::array<bool, 32> interpolated {};
for (const auto* block: program.blocks) {
for (const auto& inst: *block) {
if (inst.GetOpcode() == ValueOpcode::GetAttribute) {
interpolated[inst.Arg(0).U32()] = true;
} else if (inst.GetOpcode() == ValueOpcode::GetInterpolationParameter) {
const auto input = inst.Arg(0).U32();
const auto mode = inst.Arg(2).U32();
per_vertex[input] =
per_vertex[input] || mode < 2u || !ShaderPixelParameterIsFlat(*pixel, input);
}
}
}
// Aliases of a vertex output share one SPIR-V interface variable. If any
// alias reads raw vertices, interpolate the other aliases from those too.
// Mixed interpolation modes share raw vertices at their guest export slot.
// Rectangle expansion alone supplies separate flat and smooth outputs.
for (uint32_t input = 0; input < pixel->input_num; input++) {
for (uint32_t alias = 0; alias < pixel->input_num; alias++) {
const bool same_mode = ShaderPixelParameterIsFlat(*pixel, input) ==
ShaderPixelParameterIsFlat(*pixel, alias);
if (ShaderPixelParameterMappedLocation(*pixel, input) ==
ShaderPixelParameterMappedLocation(*pixel, alias) &&
ShaderPixelParameterIsFlat(*pixel, input) ==
ShaderPixelParameterIsFlat(*pixel, alias)) {
per_vertex[input] = per_vertex[input] || per_vertex[alias];
(pixel->parameter_mode != ShaderPixelParameterMode::Rectangle || same_mode)) {
per_vertex[input] = per_vertex[input] || per_vertex[alias] || !same_mode;
}
}
}
@@ -244,7 +115,8 @@ void CollectPixelInputs(const Program& program, const ShaderPixelInputInfo* pixe
per_vertex[input]);
}
for (uint32_t input = 0; input < pixel->input_num; input++) {
if (interpolated[input] && per_vertex[input]) {
if (used_components[input] != 0 && per_vertex[input] &&
!ShaderPixelParameterIsFlat(*pixel, input)) {
const auto kind = pixel->ps_no_perspective ? StageInputKind::BaryCoordNoPerspective
: StageInputKind::BaryCoordSmooth;
AddInput(info, kind, 0, 3,
@@ -269,147 +141,411 @@ void CollectComputeInputs(const ShaderComputeInputInfo* compute, ShaderInfo& inf
}
}
void CollectBuiltinInputs(const Program& program, ShaderInfo& info) {
for (const auto* block: program.blocks) {
for (const auto& inst: *block) {
if (program.stage == ShaderType::TessellationControl &&
inst.GetOpcode() == ValueOpcode::LaneId) {
AddInput(info, StageInputKind::InvocationId, 0, 1, "gl_InvocationID");
}
if (inst.GetOpcode() != ValueOpcode::GetBuiltin) {
continue;
}
const auto kind = static_cast<StageInputKind>(inst.Arg(0).U32());
switch (kind) {
case StageInputKind::VertexIndex:
AddInput(info, kind, 0, 1, "gl_VertexIndex");
break;
case StageInputKind::InstanceIndex:
AddInput(info, kind, 0, 1, "gl_InstanceIndex");
break;
case StageInputKind::InvocationId:
AddInput(info, kind, 0, 1, "gl_InvocationID");
break;
case StageInputKind::PrimitiveId:
AddInput(info, kind, 0, 1, "gl_PrimitiveID");
break;
case StageInputKind::TessCoord: AddInput(info, kind, 0, 3, "gl_TessCoord"); break;
case StageInputKind::FragCoord: AddInput(info, kind, 0, 4, "gl_FragCoord"); break;
case StageInputKind::FrontFacing:
AddInput(info, kind, 0, 1, "gl_FrontFacing");
break;
case StageInputKind::Layer: AddInput(info, kind, 0, 1, "gl_Layer"); break;
case StageInputKind::SampleId: AddInput(info, kind, 0, 1, "gl_SampleID"); break;
case StageInputKind::BaryCoordSmoothSample:
AddInput(info, StageInputKind::SampleId, 0, 1, "gl_SampleID");
[[fallthrough]];
case StageInputKind::BaryCoordSmooth:
case StageInputKind::BaryCoordSmoothCentroid:
AddInput(info, StageInputKind::BaryCoordSmooth, 0, 3, "gl_BaryCoordKHR");
break;
case StageInputKind::BaryCoordNoPerspective:
AddInput(info, kind, 0, 3, "gl_BaryCoordNoPerspKHR");
break;
case StageInputKind::WorkgroupId:
AddInput(info, kind, 0, 3, "gl_WorkGroupID");
break;
case StageInputKind::LocalInvocationId:
AddInput(info, kind, 0, 3, "gl_LocalInvocationID");
break;
case StageInputKind::LocalInvocationIndex:
AddInput(info, kind, 0, 1, "gl_LocalInvocationIndex");
break;
case StageInputKind::GlobalInvocationId:
AddInput(info, kind, 0, 3, "gl_GlobalInvocationID");
break;
case StageInputKind::PackedAncillary:
case StageInputKind::Parameter: break;
}
}
void AddBuiltinInput(ShaderInfo& info, StageInputKind kind) {
switch (kind) {
case StageInputKind::VertexIndex:
AddInput(info, kind, 0, 1, "gl_VertexIndex");
break;
case StageInputKind::InstanceIndex:
AddInput(info, kind, 0, 1, "gl_InstanceIndex");
break;
case StageInputKind::InvocationId:
AddInput(info, kind, 0, 1, "gl_InvocationID");
break;
case StageInputKind::PrimitiveId:
AddInput(info, kind, 0, 1, "gl_PrimitiveID");
break;
case StageInputKind::TessCoord: AddInput(info, kind, 0, 3, "gl_TessCoord"); break;
case StageInputKind::FragCoord: AddInput(info, kind, 0, 4, "gl_FragCoord"); break;
case StageInputKind::FrontFacing:
AddInput(info, kind, 0, 1, "gl_FrontFacing");
break;
case StageInputKind::Layer: AddInput(info, kind, 0, 1, "gl_Layer"); break;
case StageInputKind::SampleId: AddInput(info, kind, 0, 1, "gl_SampleID"); break;
case StageInputKind::BaryCoordSmoothSample:
AddInput(info, StageInputKind::SampleId, 0, 1, "gl_SampleID");
[[fallthrough]];
case StageInputKind::BaryCoordSmooth:
case StageInputKind::BaryCoordSmoothCentroid:
AddInput(info, StageInputKind::BaryCoordSmooth, 0, 3, "gl_BaryCoordKHR");
break;
case StageInputKind::BaryCoordNoPerspective:
AddInput(info, kind, 0, 3, "gl_BaryCoordNoPerspKHR");
break;
case StageInputKind::WorkgroupId:
AddInput(info, kind, 0, 3, "gl_WorkGroupID");
break;
case StageInputKind::LocalInvocationId:
AddInput(info, kind, 0, 3, "gl_LocalInvocationID");
break;
case StageInputKind::LocalInvocationIndex:
AddInput(info, kind, 0, 1, "gl_LocalInvocationIndex");
break;
case StageInputKind::GlobalInvocationId:
AddInput(info, kind, 0, 3, "gl_GlobalInvocationID");
break;
case StageInputKind::PackedAncillary:
case StageInputKind::Parameter: break;
}
}
void CollectOutputs(const Program& program, ShaderStageInputInfo input_info, ShaderInfo& info) {
void CollectOutput(const Program& program, ShaderStageInputInfo input_info,
ShaderInfo& info, const Inst& inst) {
const bool alpha_remap = program.stage == ShaderType::Pixel && input_info.pixel != nullptr &&
input_info.pixel->alpha_blend_source != ShaderAlphaBlendSource::None;
for (const auto* block: program.blocks) {
for (const auto& inst: *block) {
if (inst.GetOpcode() != ValueOpcode::SetAttribute) {
continue;
const auto& export_info = program.export_info[inst.Flags<ExportFlags>().index];
if (export_info.kind == ExportTargetKind::MrtZ) {
if (program.stage == ShaderType::Pixel && (export_info.en & 0x1u) != 0 &&
input_info.pixel->ps_depth_export_enable) {
AddOutput(info, StageOutputKind::Depth, 0, 0, "gl_FragDepth");
}
if (program.stage == ShaderType::Pixel && (export_info.en & 0x4u) != 0 &&
input_info.pixel->ps_sample_mask_export_enable) {
AddOutput(info, StageOutputKind::SampleMask, 0, 0, "gl_SampleMask");
}
return;
}
if (export_info.en == 0) {
return;
}
switch (export_info.kind) {
case ExportTargetKind::Position:
if (export_info.index == 0) {
AddOutput(info, StageOutputKind::Position, 0, 0, "out_position");
break;
}
const auto& export_info = program.export_info[inst.Flags<ExportFlags>().index];
if (export_info.kind == ExportTargetKind::MrtZ) {
if (program.stage == ShaderType::Pixel && (export_info.en & 0x1u) != 0 &&
input_info.pixel->ps_depth_export_enable) {
AddOutput(info, StageOutputKind::Depth, 0, 0, "gl_FragDepth");
}
if (program.stage == ShaderType::Pixel && (export_info.en & 0x4u) != 0 &&
input_info.pixel->ps_sample_mask_export_enable) {
AddOutput(info, StageOutputKind::SampleMask, 0, 0, "gl_SampleMask");
}
continue;
if (export_info.compr) {
return Fail("compressed auxiliary position export is unsupported");
}
if (export_info.en == 0) {
continue;
for (uint32_t component = 0; component < 4; component++) {
if ((export_info.en & (1u << component)) == 0) {
continue;
}
const auto output = DecodePositionExportComponent(
input_info.vertex->pa_cl_vs_out_cntl, export_info.index, component);
if (output.viewport) {
AddOutput(info, StageOutputKind::ViewportIndex, 0, 0, "gl_ViewportIndex");
}
if (output.point_size) {
AddOutput(info, StageOutputKind::PointSize, 0, 0, "gl_PointSize");
}
if (output.layer) {
AddOutput(info, StageOutputKind::Layer, 0, 0, "gl_Layer");
}
if (output.clip_distance != UINT32_MAX) {
AddOutput(info, StageOutputKind::ClipDistance, output.clip_distance, 0,
"gl_ClipDistance");
}
if (output.cull_distance != UINT32_MAX) {
AddOutput(info, StageOutputKind::CullDistance, output.cull_distance, 0,
"gl_CullDistance");
}
}
switch (export_info.kind) {
case ExportTargetKind::Position:
if (export_info.index == 0) {
AddOutput(info, StageOutputKind::Position, 0, 0, "out_position");
break;
}
if (export_info.compr) {
return Fail("compressed auxiliary position export is unsupported");
}
for (uint32_t component = 0; component < 4; component++) {
if ((export_info.en & (1u << component)) == 0) {
continue;
}
const auto output = DecodePositionExportComponent(
input_info.vertex->pa_cl_vs_out_cntl, export_info.index, component);
if (output.viewport) {
AddOutput(info, StageOutputKind::ViewportIndex, 0, 0, "gl_ViewportIndex");
}
if (output.point_size) {
AddOutput(info, StageOutputKind::PointSize, 0, 0, "gl_PointSize");
}
if (output.layer) {
AddOutput(info, StageOutputKind::Layer, 0, 0, "gl_Layer");
}
if (output.clip_distance != UINT32_MAX) {
AddOutput(info, StageOutputKind::ClipDistance, output.clip_distance, 0,
"gl_ClipDistance");
}
if (output.cull_distance != UINT32_MAX) {
AddOutput(info, StageOutputKind::CullDistance, output.cull_distance, 0,
"gl_CullDistance");
}
}
break;
case ExportTargetKind::Parameter:
AddOutput(info, StageOutputKind::Parameter, export_info.index,
export_info.index, fmt::format("out_param_{}", export_info.index));
break;
case ExportTargetKind::Mrt: {
if (alpha_remap && export_info.index != 0) {
break;
}
const auto slot = input_info.pixel->dual_source_blending
? 0u : ShaderPixelExportTarget(input_info.pixel->target_shader_mask,
export_info.index);
if (slot >= 8) {
break;
}
AddOutput(info, StageOutputKind::Mrt, export_info.index, slot,
fmt::format("out_mrt_{}", export_info.index));
if (alpha_remap) {
AddOutput(info, StageOutputKind::Mrt, 1, 0, "out_mrt_1");
break;
case ExportTargetKind::Parameter:
AddOutput(info, StageOutputKind::Parameter, export_info.index,
export_info.index, fmt::format("out_param_{}", export_info.index));
break;
case ExportTargetKind::Mrt: {
if (alpha_remap && export_info.index != 0) {
break;
}
const auto slot = input_info.pixel->dual_source_blending
? 0u : ShaderPixelExportTarget(input_info.pixel->target_shader_mask,
export_info.index);
if (slot >= 8) {
break;
}
AddOutput(info, StageOutputKind::Mrt, export_info.index, slot,
fmt::format("out_mrt_{}", export_info.index));
if (alpha_remap) {
AddOutput(info, StageOutputKind::Mrt, 1, 0, "out_mrt_1");
}
break;
}
default: break;
}
}
struct InputUsage {
std::array<uint8_t, 32> components {};
std::array<bool, 32> per_vertex {};
std::array<bool, NumScalarRegs> user_data {};
std::array<StageInputKind, static_cast<size_t>(StageInputKind::Parameter)> builtins {};
uint32_t builtin_mask = 0;
uint32_t builtin_count = 0;
void AddBuiltin(StageInputKind kind) {
const auto bit = 1u << static_cast<uint32_t>(kind);
if ((builtin_mask & bit) == 0) {
builtin_mask |= bit;
builtins[builtin_count++] = kind;
}
}
};
void Visit(Program& program, ShaderStageInputInfo input_info, InputUsage& inputs,
const Inst& inst) {
auto& info = program.info;
const auto op = inst.GetOpcode();
const auto buffer_access = BufferAccessOf(op);
const auto address_access = AddressOpcodeInfoOf(op).access;
const auto shared_access = SharedAccessOf(op);
info.float64 |= inst.GetType() == Type::F64;
info.buffer_int64_atomics |= buffer_access == BufferAccess::Atomic &&
inst.GetType() == Type::U64;
if (buffer_access != BufferAccess::None || address_access != AddressAccess::None ||
shared_access != SharedAccess::None) {
const auto index = inst.Flags<MemoryFlags>().index;
if (index >= program.memory_info.size()) {
return Fail("typed shader contains invalid memory metadata");
}
const auto& memory = program.memory_info[index];
const auto kind = memory.kind;
if (!memory.planning_only) {
info.uses_lds |= kind == ResourceKind::FlatLocal;
if (shared_access != SharedAccess::None) {
info.uses_lds |= kind == ResourceKind::Lds;
info.uses_gds |= kind == ResourceKind::Gds;
} else if (kind == ResourceKind::Buffer || kind == ResourceKind::ScalarBuffer) {
if (memory.resource >= info.buffers.size() || memory.resource >= ShaderInfo::MaxBuffers) {
return Fail("buffer operation has invalid resource metadata");
}
info.live_buffers |= uint64_t{1} << memory.resource;
for (const auto child: info.buffers[memory.resource].indirect_resources) {
if (child >= info.buffers.size()) {
return Fail("indirect buffer operation has invalid resource metadata");
}
break;
info.live_buffers |= uint64_t{1} << child;
}
default: break;
}
}
if (address_access != AddressAccess::None) {
if (kind == ResourceKind::Scratch || kind == ResourceKind::FlatLocal) {
if (program.scratch_dwords == 0) {
return Fail("scratch operation has no per-thread storage");
}
info.function_scratch = true;
info.function_lds |= kind == ResourceKind::FlatLocal &&
program.stage != ShaderType::Compute && program.stage != ShaderType::Mesh;
} else if (address_access == AddressAccess::Write) {
return Fail("writable FLAT/GLOBAL addresses require GPU ownership tracking");
} else {
info.coherent_buffers |= memory.coherent;
}
}
if (buffer_access != BufferAccess::None) {
if (kind == ResourceKind::IndirectBuffer && op != ValueOpcode::ReadConstBuffer) {
info.subgroup_local_invocation_id = true;
}
if (kind == ResourceKind::Buffer) {
info.coherent_buffers |= memory.coherent;
if (memory.resource >= info.buffers.size()) {
return Fail("buffer operation has invalid resource metadata");
}
const auto bits = StorageBufferElementBits(program, memory);
info.buffer_u8 |= bits == 8u;
info.buffer_u16 |= bits == 16u;
if ((info.buffers[memory.resource].packed_stride & (1u << 20u)) != 0u) {
if (program.stage != ShaderType::Compute) {
return Fail("buffer ADD_TID is only valid for compute shaders");
}
info.subgroup_local_invocation_id = true;
}
}
}
if (shared_access != SharedAccess::None) {
if (kind != ResourceKind::Lds && kind != ResourceKind::Gds) {
return Fail("shared operation has invalid resource kind");
}
if (shared_access == SharedAccess::Atomic && SharedComponentCount(op) == 2u) {
if (kind != ResourceKind::Lds || program.stage != ShaderType::Compute) {
return Fail("64-bit shared atomics require compute LDS");
}
info.shared_int64_atomics = true;
}
info.function_lds |= kind == ResourceKind::Lds &&
program.stage != ShaderType::Compute && program.stage != ShaderType::Mesh;
if (shared_access == SharedAccess::Append || shared_access == SharedAccess::Consume) {
info.subgroup_ballot = true;
info.subgroup_shuffle = true;
info.subgroup_local_invocation_id = true;
}
}
}
switch (inst.GetOpcode()) {
case ValueOpcode::GetAttribute: {
if (!inst.Arg(0).IsImmediate() || inst.Arg(0).GetType() != Type::U32 ||
!inst.Arg(1).IsImmediate() || inst.Arg(1).GetType() != Type::U32) {
return Fail("typed attribute reference is not constant");
}
if ((program.stage == ShaderType::Vertex ||
program.stage == ShaderType::Local) &&
(inst.Arg(1).U32() >= 4u ||
inst.Arg(0).U32() >=
static_cast<uint32_t>(input_info.vertex->resources_num))) {
return Fail("vertex input reference is out of range");
}
if (program.stage == ShaderType::Pixel &&
(inst.Arg(1).U32() >= 4u ||
inst.Arg(0).U32() >= input_info.pixel->input_num)) {
return Fail("pixel input reference is out of range");
}
if (program.stage == ShaderType::Vertex || program.stage == ShaderType::Local ||
program.stage == ShaderType::Pixel) {
auto& components = inputs.components[inst.Arg(0).U32()];
components = std::max(components, static_cast<uint8_t>(inst.Arg(1).U32() + 1u));
}
break;
}
case ValueOpcode::GetInterpolationParameter: {
if (program.stage != ShaderType::Pixel || !inst.Arg(0).IsImmediate() ||
inst.Arg(0).GetType() != Type::U32 || !inst.Arg(1).IsImmediate() ||
inst.Arg(1).GetType() != Type::U32 || !inst.Arg(2).IsImmediate() ||
inst.Arg(2).GetType() != Type::U32) {
return Fail("interpolation parameter reference is invalid");
}
if (inst.Arg(0).U32() >= input_info.pixel->input_num ||
inst.Arg(1).U32() >= 4u || inst.Arg(2).U32() >= 3u) {
return Fail("interpolation parameter reference is out of range");
}
const auto input = inst.Arg(0).U32();
inputs.per_vertex[input] |= inst.Arg(2).U32() < 2u ||
!ShaderPixelParameterIsFlat(*input_info.pixel, input);
break;
}
case ValueOpcode::GetBuiltin: {
if (!inst.Arg(0).IsImmediate() || inst.Arg(0).GetType() != Type::U32 ||
!inst.Arg(1).IsImmediate() || inst.Arg(1).GetType() != Type::U32) {
return Fail("typed builtin reference is not constant");
}
const auto kind = static_cast<StageInputKind>(inst.Arg(0).U32());
const auto component = inst.Arg(1).U32();
switch (kind) {
case StageInputKind::PackedAncillary:
return Fail("packed pixel ancillary input has an unsupported live use");
case StageInputKind::Layer:
case StageInputKind::SampleId:
if (program.stage != ShaderType::Pixel || component != 0u) {
return Fail("typed pixel scalar input is invalid");
}
break;
case StageInputKind::VertexIndex:
case StageInputKind::InvocationId:
case StageInputKind::PrimitiveId:
case StageInputKind::InstanceIndex:
case StageInputKind::FrontFacing:
case StageInputKind::LocalInvocationIndex:
if (component != 0u) {
return Fail("typed scalar builtin component is out of range");
}
break;
case StageInputKind::FragCoord:
if (component >= 4u) {
return Fail("typed fragment-coordinate component is out of range");
}
break;
case StageInputKind::BaryCoordSmooth:
case StageInputKind::BaryCoordSmoothSample:
case StageInputKind::BaryCoordSmoothCentroid:
case StageInputKind::BaryCoordNoPerspective:
if (component >= 2u) {
return Fail("typed barycentric component is out of range");
}
break;
case StageInputKind::TessCoord:
case StageInputKind::WorkgroupId:
case StageInputKind::LocalInvocationId:
case StageInputKind::GlobalInvocationId:
if (component >= 3u) {
return Fail("typed invocation builtin component is out of range");
}
break;
case StageInputKind::Parameter:
default: return Fail("typed builtin kind is invalid");
}
inputs.AddBuiltin(kind);
break;
}
case ValueOpcode::SetAttribute: {
const auto index = inst.Flags<ExportFlags>().index;
if (index >= program.export_info.size()) {
return Fail("typed export metadata index is out of range");
}
const auto& exp = program.export_info[index];
if (exp.kind == ExportTargetKind::Position && exp.index != 0 && exp.en != 0 &&
program.stage != ShaderType::Vertex && program.stage != ShaderType::Mesh &&
program.stage != ShaderType::TessellationEvaluation) {
return Fail("auxiliary position export requires a vertex, mesh, or "
"tessellation evaluation shader");
}
info.pixel_valid_mask |= program.stage == ShaderType::Pixel && exp.vm;
CollectOutput(program, input_info, info, inst);
break;
}
case ValueOpcode::GetUserData: {
if (!inst.HasUses()) break;
if (inst.Arg(0).GetType() != Type::ScalarReg) {
return Fail("typed shader contains an invalid user-data register");
}
const auto index = RegIndex(inst.Arg(0).ScalarRegister());
if (index >= NumScalarRegs) {
return Fail("typed shader contains an invalid user-data register");
}
inputs.user_data[index] = true;
break;
}
case ValueOpcode::GetDispatchThreadExtent:
info.uses_dispatch_threads |= inst.HasUses();
break;
case ValueOpcode::ReadConst: info.uses_flattened_srt = true; break;
case ValueOpcode::BitwiseXor32: info.has_bitwise_xor = true; break;
case ValueOpcode::StoreCompletion: info.subgroup_barrier = true; break;
case ValueOpcode::BvhIntersect: info.bvh = true; break;
case ValueOpcode::ConditionRef:
info.subgroup_ballot |=
inst.Flags<CFG::BranchCondition>() != CFG::BranchCondition::ScalarInstruction;
break;
case ValueOpcode::Ballot: info.subgroup_ballot = true; break;
case ValueOpcode::DppMoveU32:
case ValueOpcode::ReadFirstLane:
case ValueOpcode::ReadLane: {
info.subgroup_ballot = true;
info.subgroup_shuffle = true;
if (inst.GetOpcode() == ValueOpcode::DppMoveU32) {
info.subgroup_local_invocation_id = true;
}
break;
}
case ValueOpcode::DppUpdateU32:
case ValueOpcode::WriteLane: {
info.subgroup_ballot = true;
info.subgroup_local_invocation_id = true;
break;
}
case ValueOpcode::SwizzleU32:
info.uses_swizzle = true;
[[fallthrough]];
case ValueOpcode::Permlane16U32:
case ValueOpcode::PermuteU32:
case ValueOpcode::BpermuteU32: {
info.subgroup_ballot = true;
info.subgroup_shuffle = true;
info.subgroup_local_invocation_id = true;
break;
}
case ValueOpcode::LaneId:
if (program.stage == ShaderType::TessellationControl) {
inputs.AddBuiltin(StageInputKind::InvocationId);
} else {
info.subgroup_local_invocation_id = true;
}
break;
case ValueOpcode::ImageQueryLod: info.compute_derivatives = true; break;
case ValueOpcode::ImageGatherRaw:
info.image_gather_extended = true;
break;
default: break;
}
}
@@ -421,29 +557,49 @@ void CollectShaderInfo(Program& program, ShaderStageInputInfo input_info) {
: "shader info already collected");
}
ValidateOptions(program, input_info);
ValidateValueReferences(program, input_info);
auto& next = program.info;
next.inputs.clear();
next.outputs.clear();
next.has_bitwise_xor =
std::any_of(program.blocks.begin(), program.blocks.end(), [](const auto* block) {
return std::any_of(block->begin(), block->end(), [](const auto& inst) {
return inst.GetOpcode() == ValueOpcode::BitwiseXor32;
});
});
auto& info = program.info;
if (info.buffers.size() > ShaderInfo::MaxBuffers) {
return Fail("buffer resource limit exceeded");
}
info.inputs.clear();
info.outputs.clear();
info.user_data_registers.clear();
info.live_buffers = 0;
info.has_bitwise_xor = info.uses_lds = info.uses_gds = info.uses_flattened_srt =
info.uses_dispatch_threads = info.uses_swizzle = false;
info.bvh = info.subgroup_ballot = info.subgroup_barrier = info.subgroup_shuffle =
info.subgroup_local_invocation_id = info.compute_derivatives = info.image_gather_extended =
info.function_lds = info.function_scratch = info.pixel_valid_mask = info.buffer_int64_atomics =
info.buffer_u8 = info.buffer_u16 = info.shared_int64_atomics = info.coherent_buffers = info.float64 = false;
InputUsage inputs;
for (const auto* block: program.blocks) {
for (const auto& inst: *block) {
Visit(program, input_info, inputs, inst);
}
}
for (uint32_t index = 0; index < inputs.user_data.size(); index++) {
if (inputs.user_data[index]) info.user_data_registers.push_back(index);
}
const auto uses_mapping = [](const auto& resource) {
return resource.indirect_root != UINT32_MAX;
};
info.uses_flattened_srt |= std::ranges::any_of(info.buffers, uses_mapping) ||
std::ranges::any_of(info.images, uses_mapping);
switch (program.stage) {
case ShaderType::Vertex:
case ShaderType::Local: CollectVertexInputs(program, input_info.vertex, next); break;
case ShaderType::Local: CollectVertexInputs(input_info.vertex, info, inputs.components); break;
case ShaderType::TessellationControl:
case ShaderType::TessellationEvaluation:
case ShaderType::Mesh: break;
case ShaderType::Pixel: CollectPixelInputs(program, input_info.pixel, next); break;
case ShaderType::Compute: CollectComputeInputs(input_info.compute, next); break;
case ShaderType::Pixel:
CollectPixelInputs(input_info.pixel, info, inputs.per_vertex, inputs.components);
break;
case ShaderType::Compute: CollectComputeInputs(input_info.compute, info); break;
default: return Fail("unsupported shader stage for info collection");
}
CollectBuiltinInputs(program, next);
CollectOutputs(program, input_info, next);
for (uint32_t i = 0; i < inputs.builtin_count; i++) {
AddBuiltinInput(info, inputs.builtins[i]);
}
program.shader_info_complete = true;
}
@@ -5,8 +5,7 @@
namespace Libs::Graphics::ShaderRecompiler::IR {
// Completes the immutable shader interface after resource tracking. On failure Program::info and
// all completion state remain unchanged.
// Collects the shader interface, resource usage and instruction features after resource tracking.
void CollectShaderInfo(Program& program, ShaderStageInputInfo input_info);
} // namespace Libs::Graphics::ShaderRecompiler::IR
@@ -11,6 +11,35 @@
namespace Libs::Graphics::ShaderRecompiler::IR {
SrtRuntime SrtReadCapture::ObservedRuntime() {
auto runtime = m_source;
runtime.userdata = this;
runtime.read_specialization_memory = m_source.read_specialization_memory != nullptr
? ReadStrict : nullptr;
runtime.read_memory = ReadOrdinary;
return runtime;
}
bool SrtReadCapture::ReadStrict(void* userdata, uint64_t address, std::span<uint32_t> values) {
auto& capture = *static_cast<SrtReadCapture*>(userdata);
if (!capture.m_source.read_specialization_memory(capture.m_source.userdata, address, values)) {
return false;
}
capture.m_ranges.emplace_back(address, values.size_bytes());
return true;
}
bool SrtReadCapture::ReadOrdinary(void* userdata, uint64_t address, std::span<uint32_t> values) {
auto& capture = *static_cast<SrtReadCapture*>(userdata);
if (capture.m_source.read_memory != nullptr) {
if (!capture.m_source.read_memory(capture.m_source.userdata, address, values)) return false;
} else {
std::memcpy(values.data(), reinterpret_cast<const void*>(address), values.size_bytes());
}
capture.m_ranges.emplace_back(address, values.size_bytes());
return true;
}
SrtRuntime CleanRuntime(SrtRuntime runtime) {
runtime.read_memory = runtime.read_specialization_memory != nullptr
? runtime.read_specialization_memory
@@ -159,7 +188,7 @@ private:
bool Validate(Value value, bool require_uniform = true) {
value = value.Resolve();
// Host floating-point evaluation does not model shader rounding/denormal modes.
if (m_type == RuntimeValueType::Integer &&
if (m_type != RuntimeValueType::Any &&
TypesOverlap(value.GetType(), Type::F16 | Type::F32 | Type::F32x2)) {
return false;
}
@@ -188,6 +217,13 @@ private:
return valid;
};
const auto op = inst->GetOpcode();
if (m_type == RuntimeValueType::ImmutableInteger &&
(op == ValueOpcode::ReadConst || BufferAccessOf(op) != BufferAccess::None ||
AddressOpcodeInfoOf(op).access != AddressAccess::None ||
ImageOpcodeInfoOf(op).access != ImageAccess::None ||
SharedAccessOf(op) != SharedAccess::None)) {
return finish(false);
}
if (op == ValueOpcode::ReadConst) {
const auto slot = inst->NumArgs() == 2 ? inst->Arg(1).Resolve() : Value {};
if (inst->NumArgs() != 2 || inst->Arg(0).Resolve().TryInstruction() == nullptr ||
@@ -197,7 +233,7 @@ private:
slot.U32() >= m_program.srt_reads.size()) {
return finish(false);
}
if (m_type == RuntimeValueType::Integer) {
if (m_type != RuntimeValueType::Any) {
const auto active_mask = m_active_mask;
m_active_mask = {};
const bool valid = Validate(m_program.srt_reads[slot.U32()].value);
@@ -209,7 +245,7 @@ private:
if (!m_active_mask.IsEmpty() && IsRuntimeSelect(op) && inst->NumArgs() == 3 &&
inst->Arg(0).Resolve() == m_active_mask) {
// Empty EXEC reads lane zero, so ignored operands still require integer types.
if (m_type == RuntimeValueType::Integer && !Validate(inst->Arg(2), false)) {
if (m_type != RuntimeValueType::Any && !Validate(inst->Arg(2), false)) {
return finish(false);
}
return finish(Validate(inst->Arg(1)));
@@ -237,7 +273,7 @@ private:
return finish(true);
}
if (op == ValueOpcode::Phi) {
if (m_type == RuntimeValueType::Integer && !ValidateArguments(*inst, false)) {
if (m_type != RuntimeValueType::Any && !ValidateArguments(*inst, false)) {
return finish(false);
}
const auto invariant = ResolveInvariantPhi(m_program, value);
@@ -251,7 +287,7 @@ private:
inst->Arg(1).GetType() != Type::U1) {
return finish(false);
}
if (m_type == RuntimeValueType::Integer && !Validate(inst->Arg(1), false)) {
if (m_type != RuntimeValueType::Any && !Validate(inst->Arg(1), false)) {
return finish(false);
}
const auto active_mask = m_active_mask;
@@ -324,10 +360,9 @@ private:
} // namespace
SrtWalker::SrtWalker(const ResourcePlan& program, const SrtRuntime& runtime,
std::span<const uint8_t> clean_flat_slots, SrtWalker* clean_evaluator,
Value active_mask)
: m_program(program), m_runtime(runtime), m_clean_flat_slots(clean_flat_slots),
m_clean_evaluator(clean_evaluator), m_active_mask(active_mask.Resolve()),
SrtWalker* clean_evaluator, Value active_mask)
: m_program(program), m_runtime(runtime), m_clean_evaluator(clean_evaluator),
m_active_mask(active_mask.Resolve()),
m_context(AcquireContext(program)) {}
SrtWalker::~SrtWalker() { --m_program.evaluation_depth; }
@@ -455,7 +490,7 @@ bool SrtWalker::EvaluateExtract(const Inst& inst, uint64_t& result) {
}
bool SrtWalker::EvaluateRawRead(const Inst& inst, uint64_t& result) {
const auto flags = inst.Flags<MemoryFlags>();
const auto flags = inst.Flags<SrtReadFlags>();
if (flags.index >= m_program.memory_info.size()) {
return false;
}
@@ -557,9 +592,8 @@ bool SrtWalker::EvaluateInst(const Inst& inst, uint64_t& result) {
return EvaluateWide(inst.Arg(0), result);
}
const auto clean_runtime = CleanRuntime(m_runtime);
SrtWalker clean_active(m_program, clean_runtime, {}, nullptr, inst.Arg(1));
SrtWalker active(m_program, m_runtime, m_clean_flat_slots, &clean_active,
inst.Arg(1));
SrtWalker clean_active(m_program, clean_runtime, nullptr, inst.Arg(1));
SrtWalker active(m_program, m_runtime, &clean_active, inst.Arg(1));
return active.EvaluateWide(inst.Arg(0), result);
}
case ValueOpcode::BitCastU32F32:
@@ -573,25 +607,14 @@ bool SrtWalker::EvaluateInst(const Inst& inst, uint64_t& result) {
result = static_cast<uint32_t>(a) |
(static_cast<uint64_t>(static_cast<uint32_t>(b)) << 32u);
return true;
case ValueOpcode::ReadConst: {
const auto slot = inst.Arg(1).Resolve();
if (!slot.IsImmediate() || slot.GetType() != Type::U32 ||
slot.U32() >= m_program.srt_reads.size()) {
return false;
}
if (slot.U32() < m_clean_flat_slots.size() &&
m_clean_flat_slots[slot.U32()] != 0u && m_clean_evaluator != nullptr) {
return m_clean_evaluator->EvaluateWide(m_program.srt_reads[slot.U32()].value,
result);
}
return EvaluateWide(m_program.srt_reads[slot.U32()].value, result);
}
case ValueOpcode::LoadAddressU32:
case ValueOpcode::ReadConstBuffer:
case ValueOpcode::LoadBufferU32:
if (IsRawRead(m_program, inst)) {
if (inst.GetOpcode() == ValueOpcode::LoadBufferU32 && m_clean_evaluator != nullptr &&
m_clean_evaluator->m_active_mask == m_active_mask) {
if (m_clean_evaluator != nullptr &&
(inst.Flags<SrtReadFlags>().clean != 0u ||
(inst.GetOpcode() == ValueOpcode::LoadBufferU32 &&
m_clean_evaluator->m_active_mask == m_active_mask))) {
return m_clean_evaluator->EvaluateWide(Value(const_cast<Inst*>(&inst)), result);
}
return EvaluateRawRead(inst, result);
@@ -934,8 +957,8 @@ bool SrtWalker::RefreshFlatBuffer(std::vector<uint32_t>& flat) {
const auto refresh = [&](uint32_t slot) {
if (slot >= m_program.srt_reads.size()) return false;
const auto& read = m_program.srt_reads[slot];
const bool clean = read.flat_offset < m_clean_flat_slots.size() &&
m_clean_flat_slots[read.flat_offset] != 0u;
const auto* inst = read.value.ResolveInstruction();
const bool clean = inst != nullptr && inst->Flags<SrtReadFlags>().clean != 0u;
if (clean && (m_clean_evaluator == nullptr || m_runtime.read_specialization_memory == nullptr))
return false;
auto& evaluator = clean ? *m_clean_evaluator : *this;
@@ -972,7 +995,8 @@ bool SrtWalker::RefreshFlatBuffer(std::vector<uint32_t>& flat) {
}
uint32_t condition = 0;
auto& predicate = m_clean_evaluator != nullptr ? *m_clean_evaluator : *this;
if (!block.condition.IsEmpty() && m_runtime.read_specialization_memory != nullptr &&
if (!block.condition.IsEmpty() &&
(m_runtime.read_specialization_memory != nullptr || !m_program.capture_specialization_reads) &&
predicate.Evaluate(block.condition, condition)) {
pending.push_back(block.successors[condition != 0u ? 0u : 1u]);
} else {
@@ -20,7 +20,28 @@ struct SrtRuntime {
std::span<const uint32_t> workgroup_counts;
};
enum class RuntimeValueType { Any, Integer };
class SrtReadCapture {
public:
SrtReadCapture(SrtRuntime source, std::vector<std::pair<uint64_t, uint64_t>>& ranges):
m_source(source), m_ranges(ranges) {}
SrtRuntime ObservedRuntime();
private:
static bool ReadStrict(void* userdata, uint64_t address, std::span<uint32_t> values);
static bool ReadOrdinary(void* userdata, uint64_t address, std::span<uint32_t> values);
SrtRuntime m_source;
std::vector<std::pair<uint64_t, uint64_t>>& m_ranges;
};
// Retained reads no longer need the guest instruction PC. A clean read evaluates
// its address and bounds through the strict reader as well as the final DWORD.
struct SrtReadFlags {
uint32_t index = 0;
uint32_t clean = 0;
};
enum class RuntimeValueType { Any, Integer, ImmutableInteger };
bool ValidateRuntimeValue(const ResourcePlan& program, Value value,
RuntimeValueType type = RuntimeValueType::Any);
@@ -31,8 +52,7 @@ SrtRuntime CleanRuntime(SrtRuntime runtime);
class SrtWalker {
public:
SrtWalker(const ResourcePlan& program, const SrtRuntime& runtime,
std::span<const uint8_t> clean_flat_slots = {}, SrtWalker* clean_evaluator = nullptr,
Value active_mask = {});
SrtWalker* clean_evaluator = nullptr, Value active_mask = {});
~SrtWalker();
SrtWalker(const SrtWalker&) = delete;
SrtWalker& operator=(const SrtWalker&) = delete;
@@ -54,7 +74,6 @@ private:
const ResourcePlan& m_program;
SrtRuntime m_runtime;
std::span<const uint8_t> m_clean_flat_slots;
SrtWalker* m_clean_evaluator = nullptr;
Value m_active_mask;
ResourcePlan::EvaluationContext& m_context;
+5 -1
View File
@@ -574,10 +574,12 @@ static void ShaderGetStaticInputInfoPS(
static void ShaderGetStaticInputInfoCS(const HW::ComputeShaderInfo& regs,
const HW::ShaderRegisters& /*sh*/,
const ShaderMappedData& data, ShaderComputeInputInfo& info) {
const bool async_compute = info.async_compute;
const bool dispatch_thread_dimensions = info.dispatch_thread_dimensions;
const auto host_subgroup_size = info.host_subgroup_size;
const auto workgroup_counts = std::to_array(info.workgroup_counts);
info = {};
info.async_compute = async_compute;
info.dispatch_thread_dimensions = dispatch_thread_dimensions;
info.host_subgroup_size = host_subgroup_size;
std::ranges::copy(workgroup_counts, info.workgroup_counts);
@@ -663,6 +665,7 @@ void BuildStageStaticKey(const ShaderPixelInputInfo& info, std::vector<uint32_t>
key.push_back(static_cast<uint32_t>(info.ps_front_face));
key.push_back(static_cast<uint32_t>(info.ps_ancillary));
key.push_back(static_cast<uint32_t>(info.ps_no_perspective));
key.push_back(static_cast<uint32_t>(info.parameter_mode));
key.push_back(static_cast<uint32_t>(info.ps_pixel_kill_enable));
key.push_back(static_cast<uint32_t>(info.ps_depth_export_enable));
key.push_back(static_cast<uint32_t>(info.ps_sample_mask_export_enable));
@@ -692,7 +695,8 @@ void BuildStageStaticKey(const ShaderComputeInputInfo& info, std::vector<uint32_
key.push_back(info.thread_ids_num);
key.push_back(info.lds_size_dwords);
key.push_back(info.scratch_size_dwords);
key.push_back(static_cast<uint32_t>(info.dispatch_thread_dimensions));
key.push_back(static_cast<uint32_t>(info.dispatch_thread_dimensions) |
(static_cast<uint32_t>(info.async_compute) << 1u));
for (int i = 0; i < 3; i++) {
key.push_back(info.threads_num[i]);
key.push_back(static_cast<uint32_t>(info.group_id[i]));
+3
View File
@@ -146,6 +146,7 @@ struct ShaderVertexInputInfo {
};
struct ShaderComputeInputInfo: ShaderWorkgroupInputInfo {
bool async_compute = false;
uint8_t float_mode = 0xc0;
uint32_t dispatch_threads_num[3] = {0, 0, 0};
uint32_t workgroup_counts[3] = {0, 0, 0};
@@ -159,6 +160,7 @@ struct ShaderComputeInputInfo: ShaderWorkgroupInputInfo {
};
enum class ShaderAlphaBlendSource : uint8_t { None, SourceAlpha, SourceAlphaOne, SourceAlphaZero };
enum class ShaderPixelParameterMode : uint8_t { FirstVertex, LastVertex, Rectangle };
struct ShaderPixelInputInfo {
uint32_t interpolator_settings[32] = {0};
@@ -180,6 +182,7 @@ struct ShaderPixelInputInfo {
bool ps_front_face = false;
bool ps_ancillary = false;
bool ps_no_perspective = false;
ShaderPixelParameterMode parameter_mode = ShaderPixelParameterMode::FirstVertex;
bool ps_pixel_kill_enable = false;
bool ps_depth_export_enable = false;
bool ps_sample_mask_export_enable = false;
+4 -2
View File
@@ -30,6 +30,9 @@ uint32_t ShaderPixelParameterMappedLocation(const ShaderPixelInputInfo& info, ui
uint32_t ShaderPixelParameterLocation(const ShaderPixelInputInfo& info,
std::span<const uint32_t> active_inputs, uint32_t input) {
if (info.parameter_mode != ShaderPixelParameterMode::Rectangle) {
return ShaderPixelParameterMappedLocation(info, input);
}
std::array<bool, 32> used_locations {};
for (const auto active_input: active_inputs) {
used_locations[ShaderPixelParameterMappedLocation(info, active_input)] = true;
@@ -42,8 +45,7 @@ uint32_t ShaderPixelParameterLocation(const ShaderPixelInputInfo& info,
auto& location = group_locations[group];
if (location == UINT32_MAX) {
location = mapped;
// Smooth and custom interpolation read the same vertex output. Only
// differing flat/smooth rectangle outputs need separate locations.
// Rectangle expansion broadcasts flat aliases through separate outputs.
if (group_locations[group ^ 1u] != UINT32_MAX) {
location = 0;
while (location < used_locations.size() && used_locations[location]) {
+6 -6
View File
@@ -198,12 +198,6 @@ KernelSemaPrivate::Result KernelSemaPrivate::Wait(int need_count, uint32_t* ptr_
infinitely = false;
}
uint32_t elapsed = 0;
Common::Timer t;
t.Start();
int id = Common::Thread::GetThreadIdUnique();
if (m_count >= need_count) {
m_count -= need_count;
if (ptr_micros != nullptr) {
@@ -216,6 +210,12 @@ KernelSemaPrivate::Result KernelSemaPrivate::Wait(int need_count, uint32_t* ptr_
return Result::TimedOut;
}
uint32_t elapsed = 0;
Common::Timer t;
t.Start();
int id = Common::Thread::GetThreadIdUnique();
WaitingThread waiter {};
waiter.id = id;
waiter.priority = (m_fifo_order ? 0 : Libs::LibKernel::PthreadGetCurrentPriorityForKernel());
+13 -1
View File
@@ -1,5 +1,6 @@
find_package(Qt6 COMPONENTS Concurrent Network Widgets REQUIRED)
add_subdirectory("${KYTY_SOURCE_DIR}/updater" "${CMAKE_CURRENT_BINARY_DIR}/updater")
set(CMAKE_AUTOUIC_SEARCH_PATHS forms)
set(CMAKE_INCLUDE_CURRENT_DIR ON)
@@ -42,7 +43,13 @@ elseif(LINUX)
set_target_properties(launcher PROPERTIES LINK_FLAGS "${KYTY_LD_OPTIONS}")
endif()
add_dependencies(launcher kyty_emulator)
add_custom_target(stage_updater
COMMAND ${CMAKE_COMMAND} -E make_directory $<TARGET_FILE_DIR:launcher>
COMMAND ${CMAKE_COMMAND} -E copy_if_different $<TARGET_FILE:kyty_updater>
$<TARGET_FILE_DIR:launcher>/$<TARGET_FILE_NAME:kyty_updater>
DEPENDS kyty_updater
VERBATIM)
add_dependencies(launcher kyty_emulator stage_updater)
if(KYTY_CLANG_CL)
add_custom_command(TARGET launcher POST_BUILD
@@ -96,6 +103,7 @@ install(TARGETS launcher
if(APPLE)
install(TARGETS kyty_emulator DESTINATION "KytyPS5.app/Contents/MacOS")
install(TARGETS kyty_updater DESTINATION "KytyPS5.app/Contents/MacOS")
install(FILES "${KYTY_TROPHY_UNLOCK_SOUND}" DESTINATION "KytyPS5.app/Contents/Resources/assets/sounds")
# Deploy and ad-hoc sign the Qt frameworks and plugins in the bundle.
find_program(KYTY_MACDEPLOYQT NAMES macdeployqt PATHS "${Qt6_DIR}/../../../bin" NO_DEFAULT_PATH)
@@ -112,6 +120,10 @@ if(APPLE)
")
endif()
if(NOT APPLE)
install(TARGETS kyty_updater RUNTIME DESTINATION .)
endif()
if(WIN32)
install(CODE "
execute_process(
+413 -384
View File
@@ -16,394 +16,423 @@
<property name="locale">
<locale language="English" country="UnitedStates"/>
</property>
<layout class="QGridLayout" name="gridLayout">
<property name="horizontalSpacing">
<number>8</number>
</property>
<property name="verticalSpacing">
<number>8</number>
</property>
<item row="0" column="0" colspan="2">
<layout class="QGridLayout" name="sectionsLayout" columnstretch="1,1">
<property name="horizontalSpacing">
<number>8</number>
</property>
<property name="verticalSpacing">
<number>8</number>
</property>
<item row="0" column="0">
<widget class="QGroupBox" name="user_profile_group">
<property name="title">
<string>User profile</string>
<layout class="QVBoxLayout" name="dialogLayout">
<property name="spacing"><number>8</number></property>
<item>
<widget class="QTabWidget" name="settings_tabs">
<property name="currentIndex"><number>0</number></property>
<widget class="QWidget" name="settings_tab">
<attribute name="title"><string>General</string></attribute>
<layout class="QGridLayout" name="gridLayout">
<property name="horizontalSpacing">
<number>8</number>
</property>
<layout class="QFormLayout" name="userProfileLayout">
<property name="fieldGrowthPolicy">
<enum>QFormLayout::AllNonFixedFieldsGrow</enum>
</property>
<item row="0" column="0">
<widget class="QLabel" name="label_user_name">
<property name="text"><string>User name:</string></property>
</widget>
</item>
<item row="0" column="1">
<widget class="MandatoryLineEdit" name="lineEdit_user_name">
<property name="toolTip"><string>Local user name exposed to games</string></property>
<property name="clearButtonEnabled"><bool>true</bool></property>
</widget>
</item>
<item row="1" column="0">
<widget class="QLabel" name="label_user_id">
<property name="text"><string>User ID:</string></property>
</widget>
</item>
<item row="1" column="1">
<widget class="QSpinBox" name="spinBox_user_id">
<property name="toolTip"><string>Local user ID exposed to games</string></property>
<property name="maximum"><number>2147483647</number></property>
</widget>
</item>
<item row="2" column="0">
<widget class="QLabel" name="label_console_language">
<property name="text"><string>Console language:</string></property>
</widget>
</item>
<item row="2" column="1">
<widget class="QComboBox" name="comboBox_console_language">
<property name="toolTip"><string>Language</string></property>
</widget>
</item>
</layout>
</widget>
</item>
<item row="2" column="0">
<widget class="QGroupBox" name="audio_group">
<property name="title"><string>Audio</string></property>
<layout class="QFormLayout" name="audioLayout">
<property name="fieldGrowthPolicy">
<enum>QFormLayout::AllNonFixedFieldsGrow</enum>
</property>
<item row="0" column="0">
<widget class="QLabel" name="label_audio_input_device">
<property name="text"><string>Microphone:</string></property>
</widget>
</item>
<item row="0" column="1">
<widget class="QComboBox" name="comboBox_audio_input_device"/>
</item>
</layout>
</widget>
</item>
<item row="0" column="1" rowspan="3">
<widget class="QGroupBox" name="graphics_group">
<property name="title">
<string>Graphics &amp;&amp; display</string>
<property name="verticalSpacing">
<number>8</number>
</property>
<layout class="QFormLayout" name="graphicsLayout">
<property name="fieldGrowthPolicy">
<enum>QFormLayout::AllNonFixedFieldsGrow</enum>
</property>
<item row="0" column="0">
<widget class="QLabel" name="label_screen_resolution">
<property name="text"><string>Screen resolution:</string></property>
</widget>
</item>
<item row="0" column="1">
<widget class="QComboBox" name="comboBox_screen_resolution">
<property name="toolTip"><string>Window resolution</string></property>
</widget>
</item>
<item row="1" column="0">
<widget class="QLabel" name="label_present_mode">
<property name="text"><string>Present mode:</string></property>
</widget>
</item>
<item row="1" column="1">
<widget class="QComboBox" name="comboBox_present_mode">
<property name="toolTip"><string>Vulkan swapchain presentation mode</string></property>
</widget>
</item>
<item row="2" column="0">
<widget class="QLabel" name="label_vblank_frequency">
<property name="text"><string>Vblank frequency:</string></property>
</widget>
</item>
<item row="2" column="1">
<widget class="QSpinBox" name="spinBox_vblank_frequency">
<property name="toolTip"><string>Virtual display refresh rate used for frame pacing</string></property>
<property name="minimum"><number>30</number></property>
<property name="maximum"><number>360</number></property>
<property name="value"><number>60</number></property>
</widget>
</item>
<item row="3" column="0">
<widget class="QLabel" name="label_shader_optimization">
<property name="text"><string>Shader optimization:</string></property>
</widget>
</item>
<item row="3" column="1">
<widget class="QComboBox" name="comboBox_shader_optimization_type">
<property name="toolTip"><string>Optimize shaders for code size or performance</string></property>
</widget>
</item>
<item row="4" column="0">
<widget class="QLabel" name="label_gpu">
<property name="text"><string>GPU:</string></property>
</widget>
</item>
<item row="4" column="1">
<widget class="QComboBox" name="comboBox_gpu">
<property name="toolTip"><string>Graphics device used by the emulator</string></property>
</widget>
</item>
<item row="5" column="1">
<widget class="QCheckBox" name="checkBox_fullscreen">
<property name="toolTip"><string>Run in borderless desktop fullscreen</string></property>
<property name="text"><string>Fullscreen</string></property>
</widget>
</item>
<item row="6" column="1">
<widget class="QCheckBox" name="checkBox_readback">
<property name="toolTip"><string>Read back writable linear images</string></property>
<property name="text"><string>Enable readback</string></property>
</widget>
</item>
<item row="7" column="1">
<widget class="QCheckBox" name="checkBox_tessellation">
<property name="toolTip"><string>Draw tessellated geometry. Disabled by default; unsupported tessellation may cause crashes.</string></property>
<property name="text"><string>Enable tessellation support (experimental)</string></property>
</widget>
</item>
<item row="8" column="1">
<widget class="QCheckBox" name="checkBox_hide_cursor">
<property name="toolTip"><string>Hide the host cursor after 2 seconds without mouse activity</string></property>
<property name="text"><string>Auto-hide cursor</string></property>
</widget>
</item>
<item row="9" column="1">
<widget class="QCheckBox" name="checkBox_skip_notice_screen">
<property name="toolTip"><string>Skip startup logos and notices in supported games.</string></property>
<property name="text"><string>Skip boot logos</string></property>
</widget>
</item>
<item row="10" column="1">
<widget class="QCheckBox" name="checkBox_sync_raw_image_buffers">
<property name="toolTip"><string>Can fix missing textures and incorrect transparency in some games. May reduce performance.</string></property>
<property name="text"><string>Accurate texture synchronization</string></property>
</widget>
</item>
</layout>
</widget>
</item>
<item row="3" column="0">
<widget class="QGroupBox" name="notifications_group">
<property name="title"><string>Notifications</string></property>
<layout class="QVBoxLayout" name="notificationsLayout">
<item>
<widget class="QCheckBox" name="checkBox_trophy_notifications">
<property name="toolTip"><string>Show trophy unlock notifications and play their sound.
To use your own sound, replace the emulator's assets/sounds/trophy-unlock.wav with a WAV file and restart the game.</string></property>
<property name="text"><string>Enable trophy notifications</string></property>
<property name="checked"><bool>true</bool></property>
</widget>
</item>
</layout>
</widget>
</item>
<item row="4" column="0" colspan="2">
<widget class="QGroupBox" name="debug_group">
<property name="title">
<string>Debugging &amp;&amp; logs</string>
</property>
<layout class="QGridLayout" name="debugLayout" columnstretch="0,1,0,1">
<property name="horizontalSpacing"><number>8</number></property>
<property name="verticalSpacing"><number>6</number></property>
<item row="0" column="0" colspan="2">
<widget class="QCheckBox" name="checkBox_vulkan_validation">
<property name="toolTip"><string>Enable Vulkan validation layers</string></property>
<property name="text"><string>Vulkan validation</string></property>
<property name="checked"><bool>true</bool></property>
</widget>
</item>
<item row="0" column="2" colspan="2">
<widget class="QCheckBox" name="checkBox_shader_validation">
<property name="toolTip"><string>Validate SPIR-V binary</string></property>
<property name="text"><string>Shader validation</string></property>
<property name="checked"><bool>true</bool></property>
</widget>
</item>
<item row="1" column="0" colspan="2">
<widget class="QCheckBox" name="checkBox_renderdoc_capture">
<property name="toolTip"><string>Enable RenderDoc capture</string></property>
<property name="text"><string>RenderDoc capture</string></property>
</widget>
</item>
<item row="1" column="2" colspan="2">
<widget class="QCheckBox" name="checkBox_cmd_dump">
<property name="toolTip"><string>Dump command buffers</string></property>
<property name="text"><string>Command buffer dump</string></property>
</widget>
</item>
<item row="2" column="0">
<widget class="QLabel" name="label_shader_log_direction">
<property name="text"><string>Shader logging:</string></property>
</widget>
</item>
<item row="2" column="1">
<widget class="QComboBox" name="comboBox_shader_log_direction">
<property name="toolTip"><string>Dump shaders to file or console window. If enabled may decrease emulator performance</string></property>
</widget>
</item>
<item row="2" column="2">
<widget class="QLabel" name="label_shader_log_folder">
<property name="text"><string>Shader log folder:</string></property>
</widget>
</item>
<item row="2" column="3">
<widget class="MandatoryLineEdit" name="lineEdit_shader_log_folder">
<property name="toolTip"><string>Specify directory to dump shaders</string></property>
<property name="clearButtonEnabled"><bool>true</bool></property>
</widget>
</item>
<item row="3" column="0">
<widget class="QLabel" name="label_printf_direction">
<property name="text"><string>Printf output:</string></property>
</widget>
</item>
<item row="3" column="1">
<widget class="QComboBox" name="comboBox_printf_direction">
<property name="toolTip"><string>Print logs to file or console window. If enabled may decrease emulator performance</string></property>
</widget>
</item>
<item row="3" column="2">
<widget class="QLabel" name="label_printf_file">
<property name="text"><string>Printf output file:</string></property>
</widget>
</item>
<item row="3" column="3">
<widget class="MandatoryLineEdit" name="lineEdit_printf_file">
<property name="toolTip"><string>Specify file to dump logs</string></property>
<property name="clearButtonEnabled"><bool>true</bool></property>
</widget>
</item>
<item row="4" column="0">
<widget class="QCheckBox" name="checkBox_profiler">
<property name="toolTip"><string>Enable the Tracy profiler. Profiling may decrease emulator performance</string></property>
<property name="text"><string>Tracy profiler</string></property>
</widget>
</item>
<item row="4" column="2">
<widget class="QLabel" name="label_cmd_dump_folder">
<property name="text"><string>Command buffer folder:</string></property>
</widget>
</item>
<item row="4" column="3">
<widget class="MandatoryLineEdit" name="lineEdit_cmd_dump_folder">
<property name="toolTip"><string>Specify directory to dump command buffers</string></property>
<property name="clearButtonEnabled"><bool>true</bool></property>
</widget>
</item>
</layout>
</widget>
</item>
<item row="1" column="0">
<widget class="QGroupBox" name="compatibility_group">
<property name="title">
<string>Compatibility</string>
</property>
<layout class="QVBoxLayout" name="compatibilityLayout">
<item>
<widget class="QCheckBox" name="checkBox_amd_cpu">
<property name="toolTip"><string>Patch CPU instructions that differ between AMD and the host CPU.</string></property>
<property name="text"><string>AMD CPU patch (experimental)</string></property>
</widget>
</item>
<item>
<widget class="QCheckBox" name="checkBox_red_zone_protection">
<property name="toolTip"><string>Helps prevent crashes in some games on Windows</string></property>
<property name="text"><string>Windows SysV red zone crash protection (experimental)</string></property>
</widget>
</item>
</layout>
</widget>
</item>
</layout>
</item>
<item row="1" column="1">
<widget class="QGroupBox" name="controller_group">
<property name="title"><string>Controller</string></property>
<layout class="QFormLayout" name="controllerLayout">
<property name="fieldGrowthPolicy"><enum>QFormLayout::AllNonFixedFieldsGrow</enum></property>
<item row="0" column="0">
<widget class="QLabel" name="label_controller_color">
<property name="text"><string>DualSense lightbar:</string></property>
</widget>
</item>
<item row="0" column="1">
<layout class="QHBoxLayout" name="controllerColorLayout">
<item>
<widget class="QPushButton" name="button_controller_color">
<property name="toolTip"><string>Choose and preview a fixed LED color. Bluetooth DualSense may take several seconds to update.</string></property>
<property name="text"><string>Custom</string></property>
</widget>
</item>
<item>
<widget class="QPushButton" name="button_controller_color_reset">
<property name="toolTip"><string>Let the game control the lightbar</string></property>
<property name="text"><string>Default</string></property>
</widget>
</item>
</layout>
</item>
<item row="1" column="0">
<widget class="QLabel" name="label_controller_vibration">
<property name="text"><string>Vibration intensity:</string></property>
</widget>
</item>
<item row="1" column="1">
<layout class="QHBoxLayout" name="controllerVibrationLayout">
<item>
<widget class="QSlider" name="slider_controller_vibration">
<property name="toolTip"><string>Scale DualSense vibration over USB and Bluetooth</string></property>
<property name="maximum"><number>100</number></property>
<property name="value"><number>100</number></property>
<property name="orientation"><enum>Qt::Horizontal</enum></property>
</widget>
</item>
<item>
<widget class="QLabel" name="label_controller_vibration_value">
<property name="minimumSize"><size><width>40</width><height>0</height></size></property>
<property name="text"><string>100%</string></property>
</widget>
</item>
</layout>
</item>
<item row="2" column="0">
<widget class="QLabel" name="label_controller_volume">
<property name="text"><string>Speaker volume:</string></property>
</widget>
</item>
<item row="2" column="1">
<layout class="QHBoxLayout" name="controllerVolumeLayout">
<item>
<widget class="QSlider" name="slider_controller_volume">
<property name="toolTip"><string>Scale the DualSense speaker over USB and Bluetooth. 50% is the previous default; 100% adds 6 dB of gain.</string></property>
<property name="maximum"><number>100</number></property>
<property name="value"><number>50</number></property>
<property name="orientation"><enum>Qt::Horizontal</enum></property>
</widget>
</item>
<item>
<widget class="QLabel" name="label_controller_volume_value">
<property name="minimumSize"><size><width>40</width><height>0</height></size></property>
<property name="text"><string>50%</string></property>
</widget>
</item>
</layout>
</item>
</layout>
<item row="0" column="0" colspan="2">
<layout class="QGridLayout" name="sectionsLayout" columnstretch="1,1">
<property name="horizontalSpacing">
<number>8</number>
</property>
<property name="verticalSpacing">
<number>8</number>
</property>
<item row="0" column="0">
<widget class="QGroupBox" name="user_profile_group">
<property name="title">
<string>User profile</string>
</property>
<layout class="QFormLayout" name="userProfileLayout">
<property name="fieldGrowthPolicy">
<enum>QFormLayout::AllNonFixedFieldsGrow</enum>
</property>
<item row="0" column="0">
<widget class="QLabel" name="label_user_name">
<property name="text"><string>User name:</string></property>
</widget>
</item>
<item row="0" column="1">
<widget class="MandatoryLineEdit" name="lineEdit_user_name">
<property name="toolTip"><string>Local user name exposed to games</string></property>
<property name="clearButtonEnabled"><bool>true</bool></property>
</widget>
</item>
<item row="1" column="0">
<widget class="QLabel" name="label_user_id">
<property name="text"><string>User ID:</string></property>
</widget>
</item>
<item row="1" column="1">
<widget class="QSpinBox" name="spinBox_user_id">
<property name="toolTip"><string>Local user ID exposed to games</string></property>
<property name="maximum"><number>2147483647</number></property>
</widget>
</item>
<item row="2" column="0">
<widget class="QLabel" name="label_console_language">
<property name="text"><string>Console language:</string></property>
</widget>
</item>
<item row="2" column="1">
<widget class="QComboBox" name="comboBox_console_language">
<property name="toolTip"><string>Language</string></property>
</widget>
</item>
</layout>
</widget>
</item>
<item row="2" column="0">
<widget class="QGroupBox" name="audio_group">
<property name="title"><string>Audio</string></property>
<layout class="QFormLayout" name="audioLayout">
<property name="fieldGrowthPolicy">
<enum>QFormLayout::AllNonFixedFieldsGrow</enum>
</property>
<item row="0" column="0">
<widget class="QLabel" name="label_audio_input_device">
<property name="text"><string>Microphone:</string></property>
</widget>
</item>
<item row="0" column="1">
<widget class="QComboBox" name="comboBox_audio_input_device"/>
</item>
</layout>
</widget>
</item>
<item row="0" column="1" rowspan="3">
<widget class="QGroupBox" name="graphics_group">
<property name="title">
<string>Graphics &amp;&amp; display</string>
</property>
<layout class="QFormLayout" name="graphicsLayout">
<property name="fieldGrowthPolicy">
<enum>QFormLayout::AllNonFixedFieldsGrow</enum>
</property>
<item row="0" column="0">
<widget class="QLabel" name="label_screen_resolution">
<property name="text"><string>Screen resolution:</string></property>
</widget>
</item>
<item row="0" column="1">
<widget class="QComboBox" name="comboBox_screen_resolution">
<property name="toolTip"><string>Window resolution</string></property>
</widget>
</item>
<item row="1" column="0">
<widget class="QLabel" name="label_present_mode">
<property name="text"><string>Present mode:</string></property>
</widget>
</item>
<item row="1" column="1">
<widget class="QComboBox" name="comboBox_present_mode">
<property name="toolTip"><string>Vulkan swapchain presentation mode</string></property>
</widget>
</item>
<item row="2" column="0">
<widget class="QLabel" name="label_vblank_frequency">
<property name="text"><string>Vblank frequency:</string></property>
</widget>
</item>
<item row="2" column="1">
<widget class="QSpinBox" name="spinBox_vblank_frequency">
<property name="toolTip"><string>Virtual display refresh rate used for frame pacing</string></property>
<property name="minimum"><number>30</number></property>
<property name="maximum"><number>360</number></property>
<property name="value"><number>60</number></property>
</widget>
</item>
<item row="3" column="0">
<widget class="QLabel" name="label_shader_optimization">
<property name="text"><string>Shader optimization:</string></property>
</widget>
</item>
<item row="3" column="1">
<widget class="QComboBox" name="comboBox_shader_optimization_type">
<property name="toolTip"><string>Optimize shaders for code size or performance</string></property>
</widget>
</item>
<item row="4" column="0">
<widget class="QLabel" name="label_gpu">
<property name="text"><string>GPU:</string></property>
</widget>
</item>
<item row="4" column="1">
<widget class="QComboBox" name="comboBox_gpu">
<property name="toolTip"><string>Graphics device used by the emulator</string></property>
</widget>
</item>
<item row="5" column="1">
<widget class="QCheckBox" name="checkBox_readback">
<property name="toolTip"><string>Read back writable linear images</string></property>
<property name="text"><string>Enable readback</string></property>
</widget>
</item>
<item row="6" column="1">
<widget class="QCheckBox" name="checkBox_tessellation">
<property name="toolTip"><string>Draw tessellated geometry. Disabled by default; unsupported tessellation may cause crashes.</string></property>
<property name="text"><string>Enable tessellation support (experimental)</string></property>
</widget>
</item>
<item row="7" column="1">
<widget class="QCheckBox" name="checkBox_sync_raw_image_buffers">
<property name="toolTip"><string>Can fix missing textures and incorrect transparency in some games. May reduce performance.</string></property>
<property name="text"><string>Accurate texture synchronization</string></property>
</widget>
</item>
</layout>
</widget>
</item>
<item row="3" column="0">
<widget class="QGroupBox" name="notifications_group">
<property name="title"><string>Notifications</string></property>
<layout class="QVBoxLayout" name="notificationsLayout">
<item>
<widget class="QCheckBox" name="checkBox_trophy_notifications">
<property name="toolTip"><string>Show trophy unlock notifications and play their sound.
To use your own sound, replace the emulator's assets/sounds/trophy-unlock.wav with a WAV file and restart the game.</string></property>
<property name="text"><string>Enable trophy notifications</string></property>
<property name="checked"><bool>true</bool></property>
</widget>
</item>
</layout>
</widget>
</item>
<item row="3" column="1">
<widget class="QGroupBox" name="general_group">
<property name="title"><string>General</string></property>
<layout class="QVBoxLayout" name="generalLayout">
<item>
<widget class="QCheckBox" name="checkBox_fullscreen">
<property name="toolTip"><string>Run in borderless desktop fullscreen</string></property>
<property name="text"><string>Run in fullscreen</string></property>
</widget>
</item>
<item>
<widget class="QCheckBox" name="checkBox_hide_cursor">
<property name="toolTip"><string>Hide the host cursor after 2 seconds without mouse activity</string></property>
<property name="text"><string>Auto-hide cursor</string></property>
</widget>
</item>
<item>
<widget class="QCheckBox" name="checkBox_skip_notice_screen">
<property name="toolTip"><string>Skip startup logos and notices in supported games.</string></property>
<property name="text"><string>Skip boot logos</string></property>
</widget>
</item>
</layout>
</widget>
</item>
<item row="1" column="0">
<widget class="QGroupBox" name="compatibility_group">
<property name="title">
<string>Compatibility</string>
</property>
<layout class="QVBoxLayout" name="compatibilityLayout">
<item>
<widget class="QCheckBox" name="checkBox_amd_cpu">
<property name="toolTip"><string>Patch CPU instructions that differ between AMD and the host CPU.</string></property>
<property name="text"><string>AMD CPU patch (experimental)</string></property>
</widget>
</item>
<item>
<widget class="QCheckBox" name="checkBox_red_zone_protection">
<property name="toolTip"><string>Helps prevent crashes in some games on Windows</string></property>
<property name="text"><string>Windows SysV red zone crash protection (experimental)</string></property>
</widget>
</item>
</layout>
</widget>
</item>
</layout>
</item>
<item row="1" column="1">
<widget class="QGroupBox" name="controller_group">
<property name="title"><string>Controller</string></property>
<layout class="QFormLayout" name="controllerLayout">
<property name="fieldGrowthPolicy"><enum>QFormLayout::AllNonFixedFieldsGrow</enum></property>
<item row="0" column="0">
<widget class="QLabel" name="label_controller_color">
<property name="text"><string>DualSense lightbar:</string></property>
</widget>
</item>
<item row="0" column="1">
<layout class="QHBoxLayout" name="controllerColorLayout">
<item>
<widget class="QPushButton" name="button_controller_color">
<property name="toolTip"><string>Choose and preview a fixed LED color. Bluetooth DualSense may take several seconds to update.</string></property>
<property name="text"><string>Custom</string></property>
</widget>
</item>
<item>
<widget class="QPushButton" name="button_controller_color_reset">
<property name="toolTip"><string>Let the game control the lightbar</string></property>
<property name="text"><string>Default</string></property>
</widget>
</item>
</layout>
</item>
<item row="1" column="0">
<widget class="QLabel" name="label_controller_vibration">
<property name="text"><string>Vibration intensity:</string></property>
</widget>
</item>
<item row="1" column="1">
<layout class="QHBoxLayout" name="controllerVibrationLayout">
<item>
<widget class="QSlider" name="slider_controller_vibration">
<property name="toolTip"><string>Scale DualSense vibration over USB and Bluetooth</string></property>
<property name="maximum"><number>100</number></property>
<property name="value"><number>100</number></property>
<property name="orientation"><enum>Qt::Horizontal</enum></property>
</widget>
</item>
<item>
<widget class="QLabel" name="label_controller_vibration_value">
<property name="minimumSize"><size><width>40</width><height>0</height></size></property>
<property name="text"><string>100%</string></property>
</widget>
</item>
</layout>
</item>
<item row="2" column="0">
<widget class="QLabel" name="label_controller_volume">
<property name="text"><string>Speaker volume:</string></property>
</widget>
</item>
<item row="2" column="1">
<layout class="QHBoxLayout" name="controllerVolumeLayout">
<item>
<widget class="QSlider" name="slider_controller_volume">
<property name="toolTip"><string>Scale the DualSense speaker over USB and Bluetooth. 50% is the previous default; 100% adds 6 dB of gain.</string></property>
<property name="maximum"><number>100</number></property>
<property name="value"><number>50</number></property>
<property name="orientation"><enum>Qt::Horizontal</enum></property>
</widget>
</item>
<item>
<widget class="QLabel" name="label_controller_volume_value">
<property name="minimumSize"><size><width>40</width><height>0</height></size></property>
<property name="text"><string>50%</string></property>
</widget>
</item>
</layout>
</item>
</layout>
</widget>
</item>
</layout>
</widget>
<widget class="QWidget" name="debug_tab">
<attribute name="title"><string>Debug</string></attribute>
<layout class="QVBoxLayout" name="debugTabLayout">
<item>
<widget class="QGroupBox" name="debug_group">
<property name="title">
<string>Debugging &amp;&amp; logs</string>
</property>
<layout class="QGridLayout" name="debugLayout" columnstretch="0,1,0,1">
<property name="horizontalSpacing"><number>8</number></property>
<property name="verticalSpacing"><number>6</number></property>
<item row="0" column="0" colspan="2">
<widget class="QCheckBox" name="checkBox_vulkan_validation">
<property name="toolTip"><string>Enable Vulkan validation layers</string></property>
<property name="text"><string>Vulkan validation</string></property>
<property name="checked"><bool>true</bool></property>
</widget>
</item>
<item row="0" column="2" colspan="2">
<widget class="QCheckBox" name="checkBox_shader_validation">
<property name="toolTip"><string>Validate SPIR-V binary</string></property>
<property name="text"><string>Shader validation</string></property>
<property name="checked"><bool>true</bool></property>
</widget>
</item>
<item row="1" column="0" colspan="2">
<widget class="QCheckBox" name="checkBox_renderdoc_capture">
<property name="toolTip"><string>Enable RenderDoc capture</string></property>
<property name="text"><string>RenderDoc capture</string></property>
</widget>
</item>
<item row="1" column="2" colspan="2">
<widget class="QCheckBox" name="checkBox_cmd_dump">
<property name="toolTip"><string>Dump command buffers</string></property>
<property name="text"><string>Command buffer dump</string></property>
</widget>
</item>
<item row="2" column="0">
<widget class="QLabel" name="label_shader_log_direction">
<property name="text"><string>Shader logging:</string></property>
</widget>
</item>
<item row="2" column="1">
<widget class="QComboBox" name="comboBox_shader_log_direction">
<property name="toolTip"><string>Dump shaders to file or console window. If enabled may decrease emulator performance</string></property>
</widget>
</item>
<item row="2" column="2">
<widget class="QLabel" name="label_shader_log_folder">
<property name="text"><string>Shader log folder:</string></property>
</widget>
</item>
<item row="2" column="3">
<widget class="MandatoryLineEdit" name="lineEdit_shader_log_folder">
<property name="toolTip"><string>Specify directory to dump shaders</string></property>
<property name="clearButtonEnabled"><bool>true</bool></property>
</widget>
</item>
<item row="3" column="0">
<widget class="QLabel" name="label_printf_direction">
<property name="text"><string>Printf output:</string></property>
</widget>
</item>
<item row="3" column="1">
<widget class="QComboBox" name="comboBox_printf_direction">
<property name="toolTip"><string>Print logs to file or console window. If enabled may decrease emulator performance</string></property>
</widget>
</item>
<item row="3" column="2">
<widget class="QLabel" name="label_printf_file">
<property name="text"><string>Printf output file:</string></property>
</widget>
</item>
<item row="3" column="3">
<widget class="MandatoryLineEdit" name="lineEdit_printf_file">
<property name="toolTip"><string>Specify file to dump logs</string></property>
<property name="clearButtonEnabled"><bool>true</bool></property>
</widget>
</item>
<item row="4" column="0">
<widget class="QCheckBox" name="checkBox_profiler">
<property name="toolTip"><string>Enable the Tracy profiler. Profiling may decrease emulator performance</string></property>
<property name="text"><string>Tracy profiler</string></property>
</widget>
</item>
<item row="4" column="2">
<widget class="QLabel" name="label_cmd_dump_folder">
<property name="text"><string>Command buffer folder:</string></property>
</widget>
</item>
<item row="4" column="3">
<widget class="MandatoryLineEdit" name="lineEdit_cmd_dump_folder">
<property name="toolTip"><string>Specify directory to dump command buffers</string></property>
<property name="clearButtonEnabled"><bool>true</bool></property>
</widget>
</item>
</layout>
</widget>
</item>
<item>
<spacer name="debugSpacer">
<property name="orientation"><enum>Qt::Vertical</enum></property>
<property name="sizeHint" stdset="0"><size><width>20</width><height>40</height></size></property>
</spacer>
</item>
</layout>
</widget>
</widget>
</item>
<item row="2" column="0" colspan="2">
<item>
<layout class="QHBoxLayout" name="buttonLayout">
<item>
<widget class="QPushButton" name="clear_button">
+112 -43
View File
@@ -204,6 +204,72 @@
</property>
</spacer>
</item>
<item>
<widget class="QSlider" name="grid_size_slider">
<property name="maximumSize">
<size><width>120</width><height>16777215</height></size>
</property>
<property name="toolTip">
<string>Game image size</string>
</property>
<property name="accessibleName">
<string>Game image size</string>
</property>
<property name="minimum">
<number>128</number>
</property>
<property name="maximum">
<number>320</number>
</property>
<property name="singleStep">
<number>16</number>
</property>
<property name="pageStep">
<number>32</number>
</property>
<property name="value">
<number>224</number>
</property>
<property name="orientation">
<enum>Qt::Horizontal</enum>
</property>
</widget>
</item>
<item>
<spacer name="gridSizeSpacer">
<property name="orientation">
<enum>Qt::Horizontal</enum>
</property>
<property name="sizeType">
<enum>QSizePolicy::Fixed</enum>
</property>
<property name="sizeHint" stdset="0">
<size><width>8</width><height>0</height></size>
</property>
</spacer>
</item>
<item>
<widget class="QToolButton" name="grid_view_button">
<property name="minimumSize">
<size><width>36</width><height>36</height></size>
</property>
<property name="text">
<string>Grid view</string>
</property>
<property name="toolTip">
<string>Toggle grid view</string>
</property>
<property name="iconSize">
<size><width>32</width><height>32</height></size>
</property>
<property name="checkable">
<bool>true</bool>
</property>
<property name="autoRaise">
<bool>true</bool>
</property>
</widget>
</item>
<item>
<widget class="QLineEdit" name="search_line_edit">
<property name="sizePolicy">
@@ -238,50 +304,53 @@
</layout>
</item>
<item>
<widget class="GameListTreeWidget" name="cfgs_list">
<property name="rootIsDecorated">
<bool>false</bool>
</property>
<column>
<property name="text">
<string>Game</string>
<widget class="QStackedWidget" name="game_views">
<widget class="GameListTreeWidget" name="cfgs_list">
<property name="rootIsDecorated">
<bool>false</bool>
</property>
</column>
<column>
<property name="text">
<string>Serial</string>
</property>
</column>
<column>
<property name="text">
<string>Game Version</string>
</property>
</column>
<column>
<property name="text">
<string>Firmware Version</string>
</property>
</column>
<column>
<property name="text">
<string>Size</string>
</property>
</column>
<column>
<property name="text">
<string>Path</string>
</property>
</column>
<column>
<property name="text">
<string>Status</string>
</property>
</column>
<column>
<property name="text">
<string>Comments</string>
</property>
</column>
<column>
<property name="text">
<string>Game</string>
</property>
</column>
<column>
<property name="text">
<string>Serial</string>
</property>
</column>
<column>
<property name="text">
<string>Game Version</string>
</property>
</column>
<column>
<property name="text">
<string>Firmware Version</string>
</property>
</column>
<column>
<property name="text">
<string>Size</string>
</property>
</column>
<column>
<property name="text">
<string>Path</string>
</property>
</column>
<column>
<property name="text">
<string>Status</string>
</property>
</column>
<column>
<property name="text">
<string>Comments</string>
</property>
</column>
</widget>
<widget class="QListView" name="cfgs_grid"/>
</widget>
</item>
</layout>
+8
View File
@@ -0,0 +1,8 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32">
<g fill="none" stroke="#000000" stroke-width="2" stroke-linejoin="round">
<rect x="5" y="5" width="8" height="8" rx="1"/>
<rect x="19" y="5" width="8" height="8" rx="1"/>
<rect x="5" y="19" width="8" height="8" rx="1"/>
<rect x="19" y="19" width="8" height="8" rx="1"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 363 B

@@ -12,9 +12,9 @@
class ConfigurationItem;
class CompatibilityDatabase;
class ControllerPreview;
class QEvent;
class QTreeWidgetItem;
class QPoint;
namespace Ui {
class ConfigurationListWidget;
@@ -29,6 +29,7 @@ public:
void SetRunEnabled(bool flag) { m_run_enabled = flag; }
void SetRuntimeDirectory(const QString& directory) { m_runtime_directory = directory; }
void SetControllerPreview(ControllerPreview* preview) { m_controller_preview = preview; }
[[nodiscard]] const ConfigurationItem* GetSelectedItem() const { return m_selected_item; }
ConfigurationItem* GetSelectedItem() { return m_selected_item; }
@@ -65,13 +66,13 @@ protected slots:
void edit_global_settings();
void edit_input_mapping();
void list_itemDoubleClicked(QTreeWidgetItem* witem, int column);
void show_context_menu(const QPoint& pos);
void open_game_folder();
void remove_save_data();
void filter_configurations(const QString& text);
private:
void SelectItem(QTreeWidgetItem* witem);
void ShowContextMenu(QTreeWidgetItem* witem);
void ApplyCompatibility();
void UpdateToolbarIcons();
void ImportGameSettings(QWidget* parent);
@@ -87,6 +88,7 @@ private:
Configuration m_global_info;
QMap<QString, Configuration*> m_custom_infos;
CompatibilityDatabase* m_compatibility = nullptr;
ControllerPreview* m_controller_preview = nullptr;
};
#endif // CONFIGURATION_LIST_WIDGET_H
-29
View File
@@ -1,29 +0,0 @@
#ifndef CONTROLLER_LIGHTBAR_H
#define CONTROLLER_LIGHTBAR_H
#include <QColor>
#include <QHash>
#include <QObject>
#include <QString>
#include <QTimer>
#include <SDL3/SDL_gamepad.h>
class ControllerLightbar final: public QObject {
public:
explicit ControllerLightbar(QObject* parent = nullptr);
~ControllerLightbar() override;
void SetColor(const QString& color);
void Stop();
private:
void Refresh();
void ApplyColor(SDL_Gamepad* gamepad) const;
QTimer m_timer;
QHash<SDL_JoystickID, SDL_Gamepad*> m_gamepads;
QColor m_color;
};
#endif // CONTROLLER_LIGHTBAR_H
+53
View File
@@ -0,0 +1,53 @@
#ifndef CONTROLLER_PREVIEW_H
#define CONTROLLER_PREVIEW_H
#include <QColor>
#include <QMap>
#include <QObject>
#include <QPointF>
#include <QSet>
#include <QString>
#include <QTimer>
#include <SDL3/SDL_gamepad.h>
class ControllerPreview final: public QObject {
Q_OBJECT
public:
explicit ControllerPreview(QObject* parent = nullptr);
~ControllerPreview() override;
void SetColor(const QString& color);
void SetPreviewEnabled(bool enabled);
void Stop();
void SetSuspended(bool suspended);
[[nodiscard]] const QSet<QString>& Pressed() const { return m_pressed; }
[[nodiscard]] QPointF LeftStick() const { return m_left_stick; }
[[nodiscard]] QPointF RightStick() const { return m_right_stick; }
[[nodiscard]] const QString& DeviceName() const { return m_device_name; }
[[nodiscard]] const QString& Error() const { return m_error; }
signals:
void StateChanged();
private:
void UpdateRunning();
void OpenGamepad(SDL_JoystickID id);
void Refresh();
void ApplyColor(SDL_Gamepad* gamepad) const;
QTimer m_timer;
QMap<SDL_JoystickID, SDL_Gamepad*> m_gamepads;
QColor m_color;
bool m_preview_enabled = false;
bool m_suspended = false;
QSet<QString> m_pressed;
QPointF m_left_stick;
QPointF m_right_stick;
QString m_device_name;
QString m_error;
};
#endif // CONTROLLER_PREVIEW_H
Loaded 100 of 139 files, more files were not shown because too many files have changed in this diff. Show more