Open Source

TensorRT-LLM: NVIDIA's specialised-kernel and rack-scale engine, with agentic serving as a first-class topic

NVIDIA's inference optimisation library for LLMs and visual generation models, fully open source since 2025-03-22. Its 28 tech blogs form an auditable engineering ledger: Skip Softmax and sparse attention for long context, the three-part expert parallelism series with one-sided AlltoAll over NVLink, DWDP for NVL72, guided decoding cooperating with speculative decoding, inference-time compute, and evaluating agentic serving with trace replay and job-level metrics. Claims include Llama 4 Maverick above 1,000 TPS per user and 40,000+ tok/s on B200; Bing and NAVER Place run it in production.

Inference EngineCUDAKernels

Python15k2.8k123

XiaomiMiMo/MiMo-CodeMIT

MiMo Code: Xiaomi terminal coding agent betting on memory and self-evolution

Xiaomi open-sourced terminal-native AI coding assistant, TypeScript built with bun. The source is MIT but usage is additionally bound by USE_RESTRICTIONS.md, the MiMo terms of service and the trademark policy - read those before treating it as plain MIT. The README states it is a fork of OpenCode: it keeps the multi-provider, TUI, LSP, MCP and plugin core and adds persistent memory (SQLite FTS5 full-text search across four kinds - project MEMORY.md, session checkpoints, scratch notes and task progress - injected automatically on session resume), intelligent context management (near the limit it rebuilds from the latest checkpoint plus project memory plus task progress plus retained recent messages, ranked by importance against a token budget), goals and stop conditions (/goal sets the condition, and when the agent wants to stop a separate judge model assesses whether it was truly met, which targets optimistic early quitting), and deterministic JS workflows in a sandbox (compose splits independent tasks into isolated git worktrees with per-task TDD; also deep-research, fact-check with three-reviewer adversarial voting, and research-experiment with an anti-metric-gaming audit). Twenty built-in skills (arxiv, claude-code, codex, docx, pdf, pptx, xlsx, html-to-video, product-design and more), compatible with four skill roots - .agents/skills, .claude/skills, .codex/skills, .opencode/skills - where a user skill of the same name overrides the built-in. /dream and /distill are its signature: the first distils recent session trajectories into project memory and prunes stale entries, the second finds your repeated manual routines and packages high-confidence candidates into reusable skills. Model agnostic - the Xiaomi platform, Codex/ChatGPT OAuth, or any OpenAI-compatible endpoint. 13.4k stars. Not installed or run yet; memory-restore accuracy, judge-model effectiveness and the vendor-stated cache hit rates are unverified, so this is graded as pending reproduction.

Coding AgentAgent HarnessPersistent Memory

TypeScript13k1.4k76

handsomestWei/patent-disclosure-skillMIT

patent-disclosure-skill

patent-disclosure-skill (China Patents skill) is an open-source Agent Skills suite (MIT, Python 3.9+, Playwright-powered) covering the full Chinese patent workflow, built so that hands-on R&D engineers stop getting stuck at "how do I mine patentable points, write the prior-art search, and deliver figures plus an editable Word file". Eight sub-skills: patent-disclosure drafts invention/utility/design disclosures (mines protectable points from design docs and code, runs prior-art checks, sanitizes, writes, and iterates with multi-version archives and change traces, plus protective 1+N portfolio planning); patent-application converts disclosures into claims, spec, abstract and figures; patent-docket runs a disclosure-engineer-vs-patent-attorney role-play workflow that plans autonomously and asks rather than fabricates; patent-reader turns a publication number or PDF into plain-language notes and graphs, accumulating into a private Obsidian patent knowledge base; patent-map renders semantic terrain maps, applicant quadrants, family citation networks, tech-effect matrices and dashboards, all locally; patent-oa assists office-action responses (clause-by-clause Q&A plus RAG over distilled practitioner expertise); patent-search queries by name/company/classification and reverse-engineers search strings from a product photo or a claim; patent-exam-policy tracks CNIPA announcements and flags which drafting techniques may be outdated. It reads and generates figures: design line art from product photos, structure line drawings with part-number callouts, and isometric multi-view projections from CAD models. Disclosures land as timestamped output directories with mermaid diagrams; revisions coexist as new versions.

patentObsidianAgent Skills

Python11k1.1k29

Untrivial-ai/agent-orchestratorApache-2.0

Agent Orchestrator: Plan, Run, and Supervise Coding Agent Fleets from One Place

AO is a local desktop workspace: each coding task gets its own agent, isolated worktree branch and feedback loop; a project-aware orchestrator shapes plans and spawns workers; a live Kanban tracks every PR, CI run and review. Supports 26 coding agents.

Agent HarnessDevelopment Tools

Go11k1.5k36

deepbeepmeep/Wan2GPNOASSERTION

Wan2GP: run frontier generative models on a 6 GB GPU

A one-stop local generation app for video, image, audio and speech models, built on MMGP: block swapping, VRAM preloading and seven memory profiles let models larger than your VRAM run, starting at 6 GB.

VideoGenerationImageGenerationTextToSpeech

Python10k1.6k111

lipku/LiveTalkingApache-2.0

LiveTalking: a real-time streaming digital-human engine

LiveTalking connects LLMs, TTS and three lip-sync backends into a real-time digital-human pipeline with WebRTC, RTMP, virtual-camera output, interruption, recording and multi-session support.

Digital HumanTalking HeadLip Sync

Python9.7k1.5k75

NVlabs/SanaApache-2.0

SANA: 4K Text-to-Image That Fits in Laptop VRAM

SANA is NVIDIA Labs open-source codebase for high-resolution image and video generation (9k+ stars, Apache-2.0, PyTorch). Its efficiency comes from two structural changes: linear attention replaces vanilla attention in the transformer, and a deep compression autoencoder compresses images 32x instead of the usual 8x. Official figures: the 0.6B model renders a 1024 image in 0.9 s, roughly one fortieth of FLUX-dev 23 s, while FID improves from 10.15 to 5.61. The series now spans one-step generation, video, world models and streaming editing, and runs 4K in 8 GB of VRAM under 4-bit quantization. No benchmarks reproduced; recorded as unverified.

Image GenerationText-to-ImageDiffusion Transformer

Python9.2k737102

lexmount/moliApache-2.0

Moli: A Rust Headless Browser Built for AI Agents

Moli is Lexmount open-source headless browser: a standalone Rust kernel rather than a Chromium wrapper, built for AI agents (9k+ stars, dual Apache-2.0 / MIT). Its core design is on-demand layout and paint — by default it reads page structure without triggering layout or paint; –layout enables real geometry, hit-testing and screenshots, with layout frozen into a one-shot FrozenLayoutTree and paint state discarded after use. CDP, WebDriver Classic and WebDriver BiDi share one kernel, so no separate driver or browser install is needed. Official self-tests: 73 MiB median RSS on a 192-site crawl (headless Chrome: 773 MiB) and 81.88% success over 1,308 comparable Lexbench tasks (Chrome 99.85%). No benchmarks reproduced; recorded as unverified.

Agent HarnessHeadless BrowserBrowser Automation

Rust9.1k27718

TencentCloud/OctopMIT

Octop: a self-hosted multi-user, multi-agent assistant in one Python process

Tencent Cloud's open-source, self-hosted AI assistant platform (MIT, 8,777 stars). One process serves the FastAPI backend, React dashboard, CLI, IM channels and cron, with a SQLite WAL control plane and optional PostgreSQL. It composes four runtime libraries (octop-harness, octop-gateway, octop-memory, octop-browser) into a single self-hostable product, adding an expert library with AgentTeams, bidirectional ACP, a cross-instance Bridge, tool guardrails and PII redaction, with ADRs and line-level specs as reviewable evidence.

Agent HarnessMulti-agentSelf-Hosted

Python8.8k1.1k63

mobile-next/mobile-mcpApache-2.0

mobile-mcp: the mobile automation MCP server driven by the native accessibility tree

mobile-next/mobile-mcp (8.2k stars, Apache-2.0) connects iOS simulators and real devices plus Android emulators and real devices to any MCP client, including Claude Code, Codex, Gemini and Copilot. Its first principle is accessibility-first: read the native accessibility tree instead of screenshots, no vision model and no image tokens, falling back to screenshots with coordinates only when necessary. Roughly 30 tools in seven groups cover device management, remote devices through Mobile Next Cloud, app management, screen interaction with recording, input and navigation, logs and crashes, and batch commands that compress N rounds into one. The README examples are all cross-app long-horizon tasks, the hardest class for mobile agents.

MCPMobile AutomationiOS

TypeScript8.2k71347

ai-dynamo/dynamoNOASSERTION

Dynamo: the datacenter-scale orchestration layer above inference engines, with KV-aware routing, tiered KVBM and an SLA-driven planner

NVIDIA's open-source (Apache-2.0) orchestration layer, Rust for the performance path and Python for extensibility. It explicitly does not replace SGLang, TensorRT-LLM or vLLM - it wires them into a coordinated multi-node system: disaggregated prefill/decode, KV-aware routing (2x faster TTFT on Qwen3-Coder 480B), KVBM offloading KV cache to CPU/SSD/remote, ModelExpress streaming weights over NVLink for 7x faster cold starts, an SLA-driven Planner (80% fewer breaches at 5% lower TCO in Alibaba production), and Grove for topology-aware NVL72 scheduling.

Inference OrchestrationKV CacheDisaggregated Serving

Rust8.2k1.6k78

lightningpixel/modlyNOASSERTION

Modly: Local, Open-Source Image-to-3D Mesh Generation Desktop App

Modly is an open-source desktop app from Lightning Pixel (Windows, Linux, Apple Silicon macOS; v0.4.3; MIT plus an attribution clause) that turns a photo into a usable 3D mesh with open models running on your own GPU, so inference never leaves the machine. An Electron shell supervises a Python FastAPI backend bound to 127.0.0.1:8765. The default model is TripoSR (about 2.4 GB); Hunyuan3D 2 Mini, TripoSG and Trellis2 GGUF arrive as extensions, where an extension is a GitHub repo with a manifest.json that can declare multi-repo model_sources, shared weight_groups and separately installable weight_variants carrying size_gb and vram_gb. Generation is expressed as a node-graph workflow (Image to Generate Mesh to Add to Scene), scenes follow the modly.scene-manifest.v1 contract, and export covers glb/stl/obj/ply with a dedicated stl/obj printing path that scales to millimetres for OrcaSlicer. Three agent surfaces ship in the repo: a stdlib-only JSON-first CLI with its own SKILL.md, an MCP server exposing nine modly_* tools, and an in-app tool-calling agent that can run on a local llama.cpp GGUF pool or an external model endpoint. AMD ROCm auto-detection, memory-budgeted Apple Silicon workflows and headless Jetson runs are documented with measured numbers.

Image-to-3D3D GenerationElectron

TypeScript8.1k75755

As an Amazon Associate, we earn from qualifying purchases.