Shaders are small programs compiled to run on a graphics processing unit (GPU). Older hardware and consoles use "fixed function" rendering where programmers call a limited number of dedicated functions to make visual effects on screen. Modern GPU hardware like the Tegra X1 are "programmable", meaning that programmers can write their own code to control parts of how models are rendered on screen.
Shaders control distinct stages of the rendering process like how vertices are transformed for creating camera and animations. Shaders also control how pixels or "fragments" are shaded on screen for effects like lighting, texture mapping, and also post processing effects like bloom.
Each draw command or "draw call" requires a certain amount of settings to be configured. This collection of state is often referred to as "pipeline state" since it controls the entire renderig "pipeline" for that draw command. These settings include the compiled shaders used for each of the rendering stages as well as additional state like alpha blending. Not all rendering state is configured via compiled shaders. Some state like depth testing or alpha blending is configured using graphics API calls. Each mesh object in Smash Ultimate uses a single draw command or draw call with the pipeline state split across a number of files like the .numatb materials, .nushdb shader, .numshb parameters, stage lighting, render param, etc. Tools like RenderDoc can show the current pipeline state for each draw call when debugging an emulator or application with a supported graphics API like SSBH Editor or ssbh_wgpu.
The shaders for Smash Ultimate are precompiled and stored in .nushdb files. Each .nushdb file contains the precompiled code for a number of shaders. This code can be decompiled using Ryujinx. The shader files in game do not store any plaintext soure code or symbols, so there are no variable or function names available. The game contains thousands of shaders and possible state configurations, so each .nushdb file contains metadata for each shader to tell the game how inputs should be assigned to the shader. This includes texture names, vertex attribute names, and the location of uniform buffer parameters available in the shader code like material parameters or lighting data. The metadata also describes the attributes passed between shader stages such as the SH ambient lighting passed from the vertex shader to the fragment shader. The .nushdb metadata is available in the shader info dump.
The .nufxlb files contain databases of shader program information. Multiple compiled shaders from the .nushdb files are linked together into a shader program. Each shader program is identified by a shader label name like SFX_PBS_0100000008008269_opaque. The ending tag like "_opaque" or "_sort" helps group draw calls into render passes. The .nufxlb file also contains information on the required material parameters and mesh vertex attributes for the shader program. Note that not all the required inputs listed in the .nuflxb file are actually used by the shader code. Determining which input or parameter values are used requires looking at the .nushdb metadata and analyzing the decompiled source code. Automated analaysis of the decompiled source code is used to generate some of the data for the shader info JSON file. See the smush_materials Rust project for details.
Each material specifies a single shader prorgram entry in one of the the .nufxlb files based on the material's shader label. Materials are assigned to any number of meshes in the model.numdlb file. Some shaders are selected during rendering through other means like post processing or vertex skinning shaders.
The in game shaders are precompiled to a format designed for the Tegra X1. Emulators must first translate this byte code into a format that desktop GPUs can use. This is typically a high level language like GLSL. Ryujinx exposes its shader decompiler as a separate tool that was used for the decompiled shader dump as well as much of the shader source code analysis. These decompiled shaders can also be extracted from debugging the emulator using a tool like RenderDoc.
While the code generated by Ryujinx won't be identical to the code used to author the game, the decompiled code is still highly useful for analysis purposes. Accurate decompiled code should produce the same results as the original even if it performs the operations in slightly different ways. The shader decompilers in Ryujinx and Yuzu are good at generating code that functions equivalently to the precompiled code. This enables analyzing the algorithms and formulas used for the in game code and creating accurate high level recreations where appropriate without needing to know anything about the Nintendo Switch hardware itself.
Note that shader code compilers transform and optimize the human readable source code into assembly instructions, so the decompiled code may have non idiomatic patterns. Operations that can be expressed in a single line of high level code will often take multiple lines in the decompiled output since each line of decompiled code is a single shader instruction. For example, high level functions like pow(x, 5.0) in handwritten GLSL will appear as exp2(log2(abs(x)) * 5.0) split across multiple lines in the decompiled shader code. Converting all the optimized code to readable form requires too much manual effort to be feasible, so only select sections of code are hand translated when developing programs like ssbh_wgpu. The glsl_dependencies subcommand can automate the process of finding the relevant lines to analyze.
The shaders decompiled by Ryujinx and Yuzu use modern GLSL syntax and features like explicit layout locations and grouping uniforms into uniform and storage buffers. The layouts of uniform and storage buffers can be assumed to be the same as the in game locations in the .nushdb files. Emulators differ in how they order and assign certain inputs, so special care should be taken when identifying the shader locations for certain resources like textures. The ssbh_wgpu rendering library already implements some shader information collected by mapping .nushdb metadata to decompiled source code. This data is generated as part of the shader info JSON file.