Kernel Forge: An Agent Harness for LLM-based Generation and Optimization of CUDA Kernels

Kernel Forge: An Agent Harness for LLM-based Generation and Optimization of CUDA Kernels
Joshua Brodsky, Dhravid Kumar, Savini Kashmira, Jayanaka Danatanarayana, Jason Mars, Krisztian Flautner, Lingjia Tang (University of Michigan; the first two authors contributed equally)
arXiv:2607.24762v1 [cs.AI; cs.PF], submitted 2 June 2026 · arXiv:2607.24762 · code repository TheJoshBrod/KernelForge (MIT)
Code status: public. The repository contains model ingestion, CUDA generation and validation, the MCTS controller, .cast packaging, a CLI and a graphical interface; the paper’s benchmark harness lives under paper_benchmarks/.

One-Sentence Summary

Kernel Forge leaves the PyTorch model untouched. It first captures the operators, tensor shapes, dtypes, strides and arguments that occur during a real forward pass, turns them into operator cards, and asks an LLM to write CUDA for those concrete variants. A persistent MCTS controller explores several revision paths; a candidate is exported only if it compiles, matches the captured eager outputs numerically, and beats the measured eager latency. With 50 iterations per kernel on an NVIDIA GB10, 14 kernels outperform eager across four vision, diffusion and language workloads, with the largest local result reaching 2.83x on softmax.

1. The Hard Part Is Not Writing a Fast Kernel; It Is Reattaching One Safely

Figure 1 (paper Figure 1): the Kernel Forge graphical interface. The numbered views cover project launch, project creation, the operator workbench, per-operator results, and a kernel workbench with the generated revisions and MCTS tree.

Model runtime is concentrated in a small set of operators: matrix multiplication, convolution, normalization, softmax and elementwise arithmetic. It is tempting to conclude that replacing those operators with faster CUDA should make the model faster. In production, that inference breaks in at least three places. An isolated kernel is often tuned on random tensors or a fixed shape, while the same operator appears in a model with a particular dtype, stride, layout, broadcast pattern and set of scalar arguments. The generated code is often a standalone source file that still needs bindings, output allocation and a call-site replacement. And a performance comparison against a weak reference implementation may win a benchmark without improving the path the model actually executes.

The paper organizes the gap into four challenges. First, many tools output optimized kernels as standalone code and leave integration to the developer. Second, some systems concentrate on LLM serving or benchmark kernels, while vision, diffusion and audio models invoke the same operator with different shapes and arguments; a kernel tuned for one workload need not transfer. Third, linear refinement makes the outcome depend heavily on early design choices, while beam-style search can discard a candidate that is temporarily slower but becomes useful after later edits. Fourth, many research prototypes are exposed as scripts or benchmark workflows rather than integrated tools for inspecting candidates, tracking progress and debugging failures.

Those challenges imply a stricter evaluation question. A kernel-optimization system should not only report whether generated code is fast on a test. It should establish whether the code is correct for a concrete call observed in a model, whether it is faster than the path that call would otherwise take, and what happens when it is not. Kernel Forge is built around that question; speed is one decision variable in a longer chain.

2. System Overview: Ingestion and Forging Meet at the Operator Card

Figure 2 (paper Figure 2): the two-stage architecture. The ingestion pipeline produces operator cards from real model execution; the Forge generates, validates, benchmarks and optimizes candidates under an MCTS controller.

The system has two sequential stages. Model Ingestion executes the original PyTorch model on user-provided inputs, records the supported operators that run, groups calls into runtime-compatible variants, and emits operator cards. The Forge consumes a card, asks an LLM for CUDA and a launch wrapper, compiles and validates the candidate, benchmarks it against eager, and places it in a revision tree.

The split has an engineering consequence: ingestion does not need a CUDA generator, and generation does not need to rerun the whole model to choose its next action. A card compresses model-side evidence into a reusable optimization target. It records the operator family, call count, eager time, runtime share, variant manifest and validation examples. The generator sees one concrete shape, dtype and argument configuration; the validator replays the same captured inputs.

The unit is a “workload-specific variant,” not an operator name. A linear call in Gemma 4 E2B decoding and a linear call in a diffusion text encoder can differ in input dimension, dtype and memory layout. Treating them as one target would discard exactly the information that makes specialization possible. The definition also constrains every claim later in the paper: an improvement belongs to a captured variant, not to the end-to-end model.

3. Ingestion: Turning a Forward Pass into Replayable Evidence

Kernel Forge hooks PyTorch dispatch while the unmodified model runs. It records supported high-level operators such as conv2d, linear, relu, normalization, pooling and softmax. For each call it stores tensor shapes, dtypes, strides, layouts, scalar and optional arguments, device placement, output metadata, and example inputs and outputs. In the repository, src/optimizer/benchmarking/profile_project.py uses a unique_case path to deduplicate calls while preserving samples that can be replayed and compared.

After capture, the system groups calls by runtime compatibility. Two calls with the same public operator name become different variants if their shape, dtype, layout, arguments, backend behavior or replacement requirements differ. The grouping is induced from an execution trace rather than from static analysis, so it preserves the path PyTorch actually selected on the target device.

Runtime share is estimated from call counts and eager time inside the captured operator region. For variant $v$ with $n_v$ calls, mean eager time $t_v$, and captured variant set $\mathcal{V}$,

$$w_v=\frac{n_v t_v}{\sum_{u\in\mathcal{V}} n_u t_u}. \tag{1}$$

This weight exposes a fact that headline speedups can hide. A 2.8x improvement on an operator responsible for 0.19% of runtime has far less deployment impact than a 1.05x improvement on an operator responsible for 53%. The evaluation repeatedly pairs local speedup with $w_v$ so that a tall bar is not mistaken for a model-level result.

Ingestion also stores eager outputs as the correctness reference. Validation compares against the same model, the same input and the same dtype, not against a CPU implementation or a different library. That makes the comparison deployment-relevant and raises the bar: generated CUDA must compete with the framework’s own path, including vendor libraries, rather than with a textbook kernel.

4. The Generator: Produce a Working Extension, Then Revise a Parent

In the first round, the generator receives tensor metadata, argument values, output metadata and the launch interface from the operator card. It asks the LLM for a single kernel.cu containing device code and a C++ host wrapper named launch. The system prompt forbids delegating the computation to PyTorch, ATen, cuDNN, cuBLAS or another wrapper and requires the operation to be implemented in the kernel. A prompt is an instruction, not a guarantee; later audit labels distinguish generated CUDA from a path that quietly falls back to a backend library.

Later rounds do not restart from scratch. The MCTS controller selects a previously validated parent, and the generator receives the parent source plus its lineage. The revision is asked to reduce measured latency while preserving the behavior described by the card. Lineage records earlier modification directions, so the model continues along a path with history instead of re-guessing the design every iteration.

The prompt also carries hardware and runtime information such as the GPU model, compute capability, compiler and available memory. Those details influence tile sizes, shared-memory use, block configuration and vector width. The paper does not encode a fixed optimization template; it makes the target environment part of the generation context so that the same operator can receive different candidates on different GPUs.

5. Validation: Compile, Run, Compare, and Repair Within a Bound

A candidate is first compiled with nvcc. If compilation fails, the compiler error is returned to the generator through a bounded repair loop and the repaired source is compiled again. If compilation succeeds, the kernel is launched on the inputs stored in the card. Launch failures, crashes and illegal memory accesses enter the same repair path with their corresponding diagnostics.

Compiling and launching are necessary but not sufficient. The output must match the captured eager output. For floating-point tensors the check follows torch.allclose:

$$|x_i-y_i|\leq \mathrm{atol}+\mathrm{rtol}\,|y_i|. \tag{2}$$

The repository defaults to $\mathrm{atol}=10^{-2}$ and $\mathrm{rtol}=10^{-1}$; integer and other non-floating outputs use exact equality. The quantized TinyGEMM path has its own ABI and tolerance settings. The tolerance is not cosmetic. It determines whether fp16 or bf16 accumulation error, softmax normalization and reduction-order differences are accepted. The paper explicitly calls validation a workload-specific filter rather than a proof of correctness for all inputs: passing means the candidate matches the captured examples, not that it is safe outside them.

Only a validated candidate enters the MCTS tree. Failed candidates do not contaminate the search statistics, although their diagnostics can drive repairs. The order matters. If speed were ranked before correctness, the tree would fill with fast but unusable code and later selection would expand around invalid nodes. Validating first makes every node at least an executable, comparable implementation.

6. The MCTS Controller Searches Revision Paths, Not Hyperparameters

Each variant has a persistent revision tree. A node stores source code, parent and child links, visit count, optimization notes, audit status and measured latency. The tree lives in the project’s SQLite database, nodes.db, so a process restart does not erase the search. A node is a concrete CUDA implementation and its history, not a point in a hyperparameter grid; an edge means that one version was derived from another.

Selection has two parts. The first is progressive widening. For a node $x$ with $N_x$ visits and root visit count $R$, the number of children is capped by

$$\left|\mathrm{children}(x)\right|\leq\left\lfloor N_x^{\alpha(R)}\right\rfloor. \tag{3}$$

The exponent anneals linearly from 0.5 to 0.3 over the first 1000 root visits:

$$\alpha(R)=0.5-(0.5-0.3)\min\!\left(1,\frac{R}{1000}\right). \tag{4}$$

Early in search the larger exponent permits more branching and a wider set of ideas. Later it slows the creation of new children and spends more budget deepening existing branches. This is the mechanism that keeps a temporarily slower candidate alive long enough to be revised. A linear refinement chain would commit to the first design; a beam search that ranks only current latency might prune the intermediate version that later becomes the best parent.

When a node is not widened, the controller descends using a minimization form of UCT:

$$\mathrm{score}(c\mid p)=L_c-C\sqrt{\frac{\log N_p}{N_c}},\qquad C=1.0. \tag{5}$$

Here $L_c$ is the best latency observed in the child’s subtree, falling back to the child’s own latency; $N_p$ and $N_c$ are parent and child visit counts. The first term exploits known good results. The second rewards branches with fewer visits. Lower is better because the objective is milliseconds, not reward. A stagnation rule skips a child that has been visited often enough while its subtree has never improved beyond its own value, which prevents the controller from sampling a dead end indefinitely.

After each measurement, the best latency propagates to the ancestors:

$$L_x=\min\!\left(\ell_x,\ \min_{c\in\mathrm{children}(x)}L_c\right), \tag{6}$$

where $\ell_x$ is the node’s own measured latency. Propagation gives a node with a strong descendant continued value even if the node itself is not the fastest. The tree retains valid slower candidates because such a candidate may be the necessary intermediate state for a later structural change.

7. Packaging and Guarded Dispatch: Fallback Is the Default

Selected candidates are exported into .cast packages containing source, launch metadata, validation status, timing, workload-specific operator metadata and audit labels. The repository’s docs/FileFormat.md defines the package as a ZIP archive whose first entry must be HEADER.json, with checksums.sha256 for integrity. paper_benchmarks/paper_bench/cast_export.py uses the selection policy auto_best_fastest_valid, exporting only candidates that are valid and faster.

Runtime replacement is a conjunction. Let $V_k$ be the validation result for candidate $k$, $C_k$ its compatibility with the captured call pattern, $t_k$ its latency and $t_{\mathrm{eager}}$ the eager latency. Generated CUDA is used only when

$$\mathrm{use}(k)=V_k\land C_k\land\left(t_k<t_{\mathrm{eager}}\right). \tag{7}$$

Otherwise the original PyTorch eager path runs. This turns optimization from a global replacement into a per-variant decision. In one model, softmax may use generated CUDA while linear stays on cuBLAS; conv2d may stay on cuDNN while tensor addition uses a generated kernel. The translucent caps in the result figures show the performance recovered by fallback. They are not additional generated-kernel speedup; they visualize the guard.

Audit labels separate three outcomes: generated CUDA, a wrapper or backend path, and fallback. A package can improve the captured operator-region result while still relying on fallback for some operators. It can also contain generated code that internally calls a backend library. The paper excludes wrapper and backend paths from generated-CUDA wins. That distinction looks like reporting hygiene, but it determines whether the conclusion is credible.

8. Experimental Setup: Four Real Workloads on One GB10

All experiments run on an NVIDIA DGX Spark with a GB10 GPU reporting compute capability 12.1 (sm_121). The software stack is PyTorch 2.10.0+cu130, CUDA 13.0 with nvcc 13.0.88, driver 580.126.09, and cuDNN 9.15.1. Generation and optimization prompts use Anthropic Claude Opus 4.7. Cost is computed at 5 dollars per million input tokens and 25 dollars per million output tokens. Validation, timing, guarded fallback and cost accounting are performed by the local harness.

Workload	Model and precision	Inputs and timing policy	Captured calls
Vision	TorchVision ResNet-50 V1.5, IMAGENET1K_V2, fp16	ImageNetV2 matched-frequency, 224×224; 20 warmup batches and 100 timed batches	2,800
Diffusion	Stable Diffusion 3.5 Medium in Diffusers, bf16	T2I-CompBench prompts; batch 1, 1024×1024, 28 steps, guidance 3.5, max sequence 256; 20 warmup and 100 timed generations	75,544
Language	Gemma 4 E2B, bf16, CUDA/offload, eager attention	100 fixed prompts (60 MT-Bench, 20 ShareGPT, 20 LongBench multi_news); batch 1, greedy decoding, KV cache, 128 new tokens	3,840
Language	Qwen 3.5 35B-A3B, bf16, CUDA/offload, eager attention	Same 100 prompts; batch 1, greedy decoding, KV cache, 128 new tokens	912

Table 1: workloads, capture sources and timing policy, based on Table 1 of the paper.

The search budget has five arms: opt0, opt5, opt10, opt20 and opt50. opt0 is the initial generated candidate, and opt$i$ denotes the audit point after $i$ optimization iterations. The authors state that the arms need not improve monotonically. More search can discover a better candidate, but it can also spend budget on variants with little runtime share or on operators whose eager baseline is already strong. Every reported number is an operator-region measurement, not an end-to-end model speedup.

The baseline is the PyTorch eager path observed in the original model, including ATen, cuBLAS, cuDNN and SDPA. That is a harder baseline than a naive handwritten kernel and a more realistic one for deployment. The paper separates 24 open-source or native operators from 9 proprietary or vendor-backed operators, which prevents a large number of small native wins from hiding the vendor-backed losses.

9. Results: High Local Speedups Do Not Line Up with Runtime Share

Figure 3 (paper Figure 3): ResNet-50 operator-level results at opt50. Blue bars are open-source/native paths; yellow bars are proprietary/vendor-backed paths. Translucent caps show fallback to 1.0x, and asterisks mark wrapper/backend operators.

ResNet-50’s wins are concentrated in native operators. Tensor addition reaches 1.257x at 13.71% runtime share; ReLU reaches 1.004x at 11.90%; max pooling reaches 1.212x at 1.63%; adaptive average pooling reaches 1.515x but accounts for only 0.12%. The dominant operator, conv2d at 53.35%, reaches 0.999x. Batch normalization is 0.992x, linear is 0.903x and flatten is 0.890x. The guard keeps eager for those paths.

This is the deployment pattern in miniature. Generated code can win where the framework has not specialized every small operator, while most of the model’s time sits on operators the framework already optimizes heavily. ResNet-50 spends more than half of its captured time in conv2d; Kernel Forge does not force a replacement there. The 1.515x adaptive-pooling result is real, but multiplied by a 0.12% share it barely moves the model clock.

Figure 4 (paper Figure 4): Stable Diffusion 3.5 Medium operator-level results at opt50.

Diffusion shows the same structure more sharply. Group normalization reaches 1.699x, layer normalization 1.049x, SiLU 1.052x, interpolation 1.445x, padding 1.409x and embedding 1.009x. Yet group normalization, layer normalization and SiLU together account for only 10.48% of captured time. Linear accounts for 53.40% and scaled dot-product attention for 27.12%; together they cover 80.52%. Their generated candidates reach only 0.495x and 0.021x. Convolution reaches 0.299x at a 2.55% share.

The attention result is the most informative failure in the paper. Against a highly optimized SDPA path, fifty rounds of LLM-generated attention do not come close. The authors do not hide the number; the guard blocks it. For a reader, that is more useful than another favorable bar: a harness must know when not to use its own output.

Figure 5 (paper Figure 5): Gemma 4 E2B operator-level results at opt50.

Gemma 4 E2B pushes the mismatch to its limit. Softmax reaches 2.827x, the highest local result in the paper, but accounts for 5.93% of runtime. Linear accounts for 90.13% and reaches only 0.246x; gelu reaches 0.766x and embedding 0.905x. The safe package enables softmax while the vast majority of execution stays on eager. The 2.83x number is a real operator-level measurement. It is not a claim that Gemma is 2.83x faster end to end, nor even that the model is 5.93% faster, because softmax is only one part of the captured region.

Figure 6 (paper Figure 6): Qwen 3.5 35B-A3B operator-level results at opt50.

Qwen 3.5 35B-A3B follows the same pattern. Softmax reaches 1.544x at a 0.19% share, conv1d 1.292x at 0.40%, and embedding 1.138x at roughly 0.002%. Grouped matrix multiplication accounts for 93.62% and reaches 0.616x; linear accounts for 4.98% and reaches 0.363x. Softplus, sigmoid, padding and SiLU sit below or near 1.0x. Guarded dispatch admits a few small operators and leaves the dominant time on vendor paths.

Workload	Operator	opt50 speedup	Captured runtime share	Path
ResNet-50	tensor add	1.257x	13.71%	native
ResNet-50	ReLU	1.004x	11.90%	native
ResNet-50	max pooling	1.212x	1.63%	native
ResNet-50	adaptive avgpool	1.515x	0.12%	native
ResNet-50	conv2d	0.999x	53.35%	wrapper/backend
SD3.5 Medium	group norm	1.699x	group norm + layer norm + SiLU = 10.48%	native
SD3.5 Medium	linear	0.495x	53.40%	proprietary
SD3.5 Medium	SDPA	0.021x	27.12%	proprietary
Gemma 4 E2B	softmax	2.827x	5.93%	native
Gemma 4 E2B	linear	0.246x	90.13%	proprietary
Qwen 3.5 35B-A3B	softmax	1.544x	0.19%	native
Qwen 3.5 35B-A3B	conv1d	1.292x	0.40%	native
Qwen 3.5 35B-A3B	grouped matmul	0.616x	93.62%	proprietary

Table 2: key operators from Figures 3–6. Speedup is the generated candidate relative to the eager path for the same variant; the share is the runtime weight inside the captured operator region.

The aggregate count makes the distribution clear. Among the displayed opt50 operators, 13 of 24 open-source/native operators beat eager, compared with 1 of 9 proprietary/vendor-backed operators. This is consistent with mature library paths being harder to beat from scratch, but the paper does not turn it into a proof of impossibility. It is a statement about these models, this GPU, and a 50-iteration budget.

10. Search Cost: opt50 Spends More, Not Necessarily Where It Matters

The paper records Opus 4.7 API spend for each search arm and reports both incremental and cumulative cost. With $T_{\mathrm{in}}$ input tokens and $T_{\mathrm{out}}$ output tokens, the cost of a call under the stated prices is

$$c=5\cdot\frac{T_{\mathrm{in}}}{10^6}+25\cdot\frac{T_{\mathrm{out}}}{10^6}. \tag{8}$$

Cost rises sharply at opt50 because that arm adds many more generation and repair attempts. The increase is not matched by a proportional runtime-weighted benefit. The largest local speedups often land on low-share operators, while the dominant operators remain on strong vendor paths. Restricting the accounting to operators with at least a 1% runtime share preserves the pattern: the later arms account for a large fraction of total spend without necessarily changing the aggregate outcome.

The conclusion is not that search is useless. It is that a uniform per-operator budget is misaligned with deployment impact. A better policy would allocate depth by runtime share and baseline strength: stop early on low-impact variants, spend more on dominant operators that can plausibly be challenged, and stop after repeated failures to approach eager. The paper presents this as future work rather than fitting a more attractive policy after the fact.

11. Code Correspondence: Every Paper Component Has a Repository Location

The MCTS controller is implemented in src/optimizer/core/mcts.py; node structure and the UCT score are in src/optimizer/core/types.py; the widening exponents, stagnation threshold and exploration constant are in src/optimizer/config/settings.py. The tree is persisted in each project’s nodes.db, with columns for source, visits, median latency, best-subtree value and phase.

Capture and replay live in src/optimizer/benchmarking/profile_project.py. CUDA compilation, execution and allclose validation live in src/optimizer/backends/cuda/verifier.py; quantized tolerances and the TinyGEMM ABI live in src/optimizer/quantized.py. Export policy is in paper_benchmarks/paper_bench/cast_export.py, the package format is documented in docs/FileFormat.md, and the generator’s prohibition on delegating to a wrapper is in src/generator/prompts/GeneratorSystemPrompt.md. These files make “guarded,” “audited” and “persistent” concrete properties rather than adjectives.

Paper component	Repository location	Key implementation
Model ingestion and variant capture	src/optimizer/benchmarking/profile_project.py	dispatch hook; unique_case deduplication; captured inputs, outputs and metadata
CUDA generation constraints	src/generator/prompts/GeneratorSystemPrompt.md	single kernel.cu; launch wrapper; no delegation to backend libraries
Compilation and numerical validation	src/optimizer/backends/cuda/verifier.py	nvcc / load_inline; repair loop; torch.allclose
Quantized tolerances	src/optimizer/quantized.py	default $10^{-2}/10^{-1}$; TinyGEMM-specific ABI and tolerance
MCTS control	src/optimizer/core/mcts.py, types.py, config/settings.py	SQLite tree; progressive widening; minimizing UCT; stagnation skip
Packaging and export	paper_benchmarks/paper_bench/cast_export.py, docs/FileFormat.md	auto_best_fastest_valid; .cast ZIP; checksums and audit labels

Table 3: mapping between paper components and the open-source implementation, based on this reading of the repository.

12. The Whole Pipeline in One Diagram

flowchart LR
    A["PyTorch model
weights + sample inputs"] --> B["Forward run
dispatch hook"]
    B --> C["Captured calls
shape / dtype / stride / args / output"]
    C --> D["Runtime-compatible variants"]
    D --> E["Operator cards
calls / eager time / runtime share"]
    E --> F["Generator agent
CUDA source + launch wrapper"]
    F --> G["nvcc compile"]
    G -->|"failure"| H["Bounded repair loop"]
    H --> G
    G -->|"success"| I["Run on captured inputs"]
    I -->|"crash or mismatch"| H
    I -->|"allclose passes"| J["Benchmark against eager"]
    J --> K["Persistent MCTS tree
widening + UCT"]
    K -->|"select parent + lineage"| F
    K --> L["Export .cast
source / ABI / validation / timing / audit"]
    L --> M{"valid and compatible
and faster"}
    M -->|"yes"| N["Dispatch generated CUDA"]
    M -->|"no"| O["Fallback to PyTorch eager"]

There is no shortcut in which one prompt generates every kernel and the model simply runs faster. Each variant goes through compilation, validation, timing and search. Each exported package re-evaluates the replacement condition at runtime. “End to end” here means that every segment from model execution to safe dispatch is connected, and that some segments are allowed to decide not to replace anything.

13. Limitations: What the Authors Admit and What the Evidence Does Not Cover

First, validation is a workload-specific filter, not a proof. A candidate matches eager on the captured input, dtype and arguments; a different batch size, padding mode or sequence length may change its behavior. The paper states this in Section 3.3. Guarded dispatch can only operate inside the captured call pattern, which is why compatibility is part of the replacement condition.

Second, the results are operator-region measurements, not end-to-end speedups. The paper does not report a whole-model latency reduction, and it does not combine per-operator speedups into one aggregate number. Replacement changes memory layout, launch count, synchronization and cache behavior, so operator-level gains need not add linearly. A 2.83x operator result cannot be written as “Gemma is 2.83x faster.”

Third, a fixed per-operator budget is misaligned with deployment impact. opt50 gives every variant the same depth, but dominant operators are often already served by mature libraries and low-share operators cannot move the model clock even with a large local win. The authors propose runtime-share-aware and baseline-aware budgeting. Until that exists, the current results are closer to an upper-bound probe than a cost-optimal production policy.

Fourth, experimental coverage is narrow. All results come from one GB10 (sm_121), four models, inference only, fixed input sets and a single LLM. Training, multi-GPU execution, dynamic shapes, long context and other GPU architectures are not tested. The MCTS settings ($C=1.0$, 0.5 to 0.3 annealing, 1000 root visits, stagnation threshold 10) are not ablated, so the paper cannot claim that this search policy beats linear refinement or beam search. It shows that the combination works in this setting.

Fifth, the audit boundary is limited. Labels distinguish generated CUDA, wrapper/backend paths and fallback, but they are produced by the export pipeline, not by a formal runtime guarantee. Generated code can still read out of bounds, leak memory or return a wrong answer on an uncaptured input. Guarded dispatch reduces risk; it does not turn LLM-generated CUDA into a trusted execution environment.

14. Conclusion: Put “Faster” Back Inside “Usable”

Kernel Forge’s contribution is not merely that an LLM can write CUDA. It connects real model execution, operator variants, numerical validation, a persistent search tree, packaging and fallback into one auditable chain, then exercises that chain on four workloads from different modalities. The most important result is structural rather than the 2.83x headline: generated kernels win more often on native, low-share operators; high-share operators are usually defended by mature libraries; guarded dispatch refuses a regression instead of handing it to the user.

The same pattern appears in robotics and embodied systems. Perception, planning and control pipelines contain many fixed-shape, frequently called operators, while edge GPUs have less library coverage than data-center accelerators. A useful optimizer must know which calls came from real trajectories, which outputs require elementwise agreement, and which candidates are safe only for a particular sensor input. Kernel Forge offers an engineering template for encoding those constraints, not a slogan that the model will write its own kernels.

Quote

“The largest local speedups often occur on operators with small runtime contribution, while dominant operators are frequently backed by mature framework or vendor implementations.”

That sentence is the paper’s real thesis. Kernel Forge turns it into measurable, reversible and auditable system behavior.