For the past few days I have been testing the FMLA32 (FP16/BF16/TF32 -> FP32 accumulation) unlock on my Windows 11 machine with a Palit RTX 4070 Ti (10de:2782, 12 GB, 300W power limit). The unlock itself works as advertised, but the setup process was pretty inconvenient if you couldn't find a repository made specifically for your hardware. Linux users had to jump between separate repositories for Turing, Ampere, and Ada. On Windows, existing UEFI builds were mostly tied to the profile of a single specific board. To run them on your own card, you had to pull your PCI Device/Subsystem IDs, VRAM offsets, and the Windows EFI boot partition GUID, manually edit local_profile.h, and compile BOOTX64.EFI yourself. While adapting this for my own GPU, I had the idea to refine the existing work and simplify the process by bringing everything into a single tool. The result is a unified cross-platform package that combines support for RTX 40 (Ada), RTX 30 (Ampere), and RTX 20 (Turing) series cards on Windows 10/11 and Linux: https://github.com/MindAllik/GPU-Tensor-Unlock-Universal Full credit goes to the authors of the foundational research, payloads, and loader code that this project builds upon: TOTORONG/AD103_win_unlocker, Zagitta/gpu-unlock, elias-gaeros/gpu-unlock, JustAResearcher/gpu-unlock-ada, KohakuBlueleaf/KohakUnlock, impacifist/Windows-GPU-UNLOCK, Goldlionren/gpu-unlock-4080s, haohervchb/Turing-unlock, and of course Jon Pry. On Windows (Windows_Launch_UI.bat), the core_detector.py script cross-queries NVML, CUDA Driver API (sm_89 / sm_86 / sm_75), and pci.sys to automatically detect the PCI Device ID, Subsystem ID, bus number, VRAM size, and the EFI partition GUID on the boot drive. It checks the PCI ID -> Silicon Die -> PMC_BOOT_0 mapping to make sure the profile is accurate, generates the headers in a temp folder, and compiles BOOTX64.EFI onto a FAT32 USB drive (with a 30-second confirmation timer on pressing U at boot). To avoid black-screen hangs, a two-phase pre-flight check (PromCheck) was added. Instead of relying on a single hardcoded ROM hash or disconnecting the display blindly, the loader first reads the card's VBIOS (BAR0 + 0x300000), verifies the BIT table, the Falcon descriptor (V3 RSA-3K on Ada/Ampere, V2 AES-128 on Turing), and the DMAP structure, and then runs a dry-run FRTS patch in RAM before calling DisconnectController. If a vendor VBIOS layout is non-standard, it safely aborts the operation and boots into Windows normally. On Linux (Linux_Launch_UI.sh or via CLI make install), the tool installs the gpu_unlock.ko DKMS module and rebuilds initramfs for mkinitcpio, dracut, or initramfs-tools. For experimental Ada and Turing cards, an explicit single-card PCI BDF (0000:xx:00.0) is required instead of targets=all so multi-GPU builds don't touch a neighboring card. It also includes tools/read_sm_rates.py, which queries /dev/nvidiactl via ioctl 0x20801230 to check the fmla32 = 0 (full) status directly from the driver without creating a CUDA context. A 7-kernel PTX JIT benchmark is also built into the console (tensor_benchmark.py: FP16 -> FP16, FP16 -> FP32, BF16 -> FP32, TF32 -> FP32, FP8 E4M3 -> FP32, INT8 -> INT32, and Scalar FP32). As KohakuBlueleaf noted earlier, standard mmapeak filled matrix fragments with compile-time zeros, which allowed ptxas to optimize away calculations inside the loop. This benchmark generates random non-zero data at runtime and verifies both the loop iteration signature (iter ^ 0x67b05a5a) and finite IEEE-754 accumulator values (and a button to raise the Power Limit to the VBIOS maximum via NVML was also added to the interface). On my test bench (Palit RTX 4070 Ti 10de:2782, Windows 11 UEFI, 300W limit), the unthrottled FP16 -> FP16 rate (FMLA16 control kernel) is 172.8 TF/s. At stock, FP16 -> FP32 was around 86.4 TF/s, and after the UEFI unlock both FP16 -> FP32 (172.8 TF/s) and BF16 -> FP32 (172.7 TF/s) reached the full 2.00x rate. For comparison: GPU PCI ID / Die FP16->FP16 (Control) FP16->FP32 Stock FP16->FP32 Unlocked BF16->FP32 Unlocked Source RTX 4070 Ti 10de:2782 (AD104) 172.8 TF/s 86.4 TF/s 172.8 TF/s (2.00x) 172.7 TF/s (2.00x) Tested by me (Win 11 UEFI, Palit 300W) RTX 4090 10de:2684 (AD102) ~331.6 TF/s 165.6 TF/s 331.6 TF/s (2.00x) 330.6 TF/s (2.00x) Upstream report (elias-gaeros, Linux 450W) RTX 4070 Ti SUPER 10de:2705 (AD103) ~190.0 TF/s 95.0 TF/s 189.9 TF/s (2.00x) 189.7 TF/s (2.00x) Upstream report (JustAResearcher, Linux 240W) All operations happen in volatile MMIO registers prior to OS driver load and reset on a cold power-off (VBIOS flash memory is not touched at all, and the script downloads the proprietary NVIDIA firmware binaries directly from the official 610.43.03 package and verifies its SHA-256 locally). Right now, RTX 4070 Ti (10de:2782), RTX 4090 (2684), RTX 4070 Ti SUPER (2705), RTX 3090 (2204), RTX 3080 (2206), and RTX 3060 12GB (2504) are verified on real hardware. The remaining desktop AD10x (40-series), GA10x (30-series), and TU10x (20-series / CMP) profiles in the matrix currently have the experimental_simulated status (verified on a 106-scenario simulator under GCC ASan/UBSan). If you decide to test one of the experimental cards on Windows or Linux, please share your EFI\UNLOCK\REPORT.TXT (or dmesg) and benchmark numbers in GitHub Issues or here in the comments, as the report is very important both when a run succeeds and when it stops at the pre-flight check so each board profile can be confirmed (: Separately, I want to mention that this is unlikely to noticeably speed up single-stream token generation (batch=1 decode is bound by memory bandwidth) and is unlikely to affect pure integer INT8/INT4 matmuls in llama.cpp. Where it actually helps is compute-bound FP16, BF16, and FP8 matrix math with FP32 accumulation, such as reading long prompts (prefill in vLLM, ExLlamaV2, FlashAttention, and Marlin INT4 -> FP16 kernels), LoRA / QLoRA fine-tuning in BF16, and diffusion models. P.S. I used Gemini to translate this text into English and make it prettier and more concise :)   submitted by   /u/MindAllik [link]   [comments]