Skip to content
View mbsdeepak's full-sized avatar

Block or report mbsdeepak

Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
mbsdeepak/README.md
Bhargava Sai Deepak Mathukumalli — building LLM-agent infrastructure from first principles

I build cloud cost-intelligence and data platforms professionally, and on my own time I implement the hard parts from the ground up — LLM-agent infrastructure, data and distributed systems, and models — one small, single-purpose library per problem, no framework in the way. The bias throughout: build the thing to understand it — minimal dependencies, code you can read end-to-end, behaviour you can inspect.


⌁   An agent platform, built in layers

Each hard part of running an LLM agent, implemented as an independent library. They stand alone — and compose into a complete platform.

flowchart LR
  T(["request"]) --> BH["🛡️ bulkhead<br/><i>gateway</i>"]
  BH --> CG["⚙️ cogs<br/><i>runtime</i>"]
  LM["🧵 loom<br/><i>context</i>"] --> CG
  CG --> EM["🔥 ember<br/><i>serving</i>"]
  CG --> SN["📡 sonar<br/><i>tracing</i>"]
  CG -. evals .-> GT["🎯 gauntlet"]
  classDef n fill:#161b22,stroke:#30363d,color:#e6edf3,rx:8,ry:8;
  class T,BH,CG,LM,EM,SN,GT n;
Loading
Project What it solves Notable internals
cogs The agent runtime Agent loop, typed tool protocol, provider abstraction, context management, permissions, and deterministic record/replay — in ~1.5k lines.
ember LLM inference serving A mini-vLLM — paged KV cache + continuous batching, GPT-2 from scratch loading real weights, and an OpenAI-compatible streaming server.
loom Context engineering Chunking, embeddings, vector retrieval, history compaction, and token-budgeted context assembly.
bulkhead Provider resilience Retries, circuit breaking, rate limiting, caching, failover, and cost governance in front of any provider.
gauntlet Agentic evaluation Deterministic simulated tool environments, state/trajectory/LLM-judge grading, and pass@k with Wilson confidence intervals.
sonar Run observability OpenTelemetry-style span tracer, cost/latency meters, and text/HTML trace timelines.

⌁   Data & ML, from first principles

The same bias, pointed at data systems and models — each one built to see how the real thing works underneath.

Project What it solves Notable internals
qwery SQL over raw files Hand-written tokenizer + recursive-descent parser feeding a Volcano-model executor over CSV/Parquet — the design behind DuckDB, in pure Python.
athena-cost-guard  ·  PyPI Know a query's cost before you run it Parses SQL with sqlglot, resolves partitions through Glue, and estimates bytes-scanned from S3 — then a @cost_guard decorator blocks queries over budget. pip install athena-cost-guard.
kalman-timeseries Denoising & forecasting noisy signals Self-tuning adaptive/robust Kalman filter with an RTS smoother, NumPy-only — ~51% lower error than the plain baseline, with an accompanying paper.
text-diffusion-fashion-mnist  ·  🤗 Text-to-image from scratch A tiny Stable Diffusion — CLIP text conditioning, a U-Net denoiser, DDIM sampling, and classifier-free guidance, trained on Fashion-MNIST. Weights on Hugging Face.
scribe  ·  🤗 A small language model from scratch A ~30M-param GPT trained from scratch on TinyStories — hand-written causal attention, GELU MLP, tied LM head. Built GPT-2-compatible on purpose, so ember serves the very weights it trains. Weights on Hugging Face.

⌁   Distributed systems, from scratch

Consensus and storage internals, built to see how a scalable stateful system actually holds together.

Project What it solves Notable internals
quorum A distributed key-value database A from-scratch LSM storage engine (WAL, skiplist memtable, SSTables with bloom filters, compaction) replicated by a hand-written Raft — leader election, log replication with fast-backup, crash-safe persistence, and snapshot log compaction. In Go, race-tested end to end.

⌁   Currently

Cloud cost-intelligence — an LLM agent that answers cost questions over large-scale usage data, and the analytics pipelines behind it on Kubernetes.

Reach for:   Python   Go   C++   AWS · Bedrock · Athena   Kubernetes · Argo   SQL


$ contact  ·  LinkedIn  ·  mbsdeepak3@gmail.com

Pinned Loading

  1. cogs cogs Public

    A minimal but real LLM agent runtime — the core of a coding-agent harness (agent loop, typed tool protocol, provider abstraction, context management, permissions, and deterministic record/replay) i…

    Python 2

  2. gauntlet gauntlet Public

    A rigorous agentic tool-use eval set for LLMs: deterministic simulated tool environments, state/trajectory/LLM-judge grading, pass@k with Wilson confidence intervals, and cost/latency tracking.

    Python 2

  3. sonar sonar Public

    Observability for agent runs — an OpenTelemetry-style span tracer, cost/latency meters, trace ingest for cogs & gauntlet, and text/HTML timelines, in ~2k readable lines of Python.

    Python 2

  4. athena-cost-guard athena-cost-guard Public

    Estimate what an AWS Athena query will scan and cost before you run it — and block queries over budget.

    Python 2

  5. kalman-timeseries kalman-timeseries Public

    NumPy-only Kalman filter for denoising and forecasting noisy time series (temperature demo, RTS smoother, tests)

    Python 2

  6. text-diffusion-fashion-mnist text-diffusion-fashion-mnist Public

    Text-conditioned diffusion model (tiny Stable Diffusion) built from scratch in PyTorch, trained on Fashion-MNIST — CLIP text conditioning, DDIM sampling, classifier-free guidance.

    Python 2