#Large Language Models (1)

TokenGate: Learnable Token Pruning for Fast, Memory-Efficient LLM Prefill

TokenGate learns a residual scorer at a middle layer, retains top-K document tokens plus request scaffolding, and uses token-gated attention to train through the discrete selection. It reports 1.4-11.3x faster TTFT and 1.5-3.4x lower peak prefill memory on Qwen3 and Qwen3.5 without accuracy loss; robotics performance remains untested.