Yee Jiun (YJ) Song

The biggest difference between Google and Waymo, almost one year in: Waymo has largely adopted the Amazonian practice of presenting information in documents, not slides. Meetings start with a few minutes of pre-read / frantic commenting on a Google Doc, followed by a live burn-down of the comments that require discussion. It's efficient and information-dense. I love it.

2,183

63 Comments

I was asked recently: what is one thing that has surprised me about leadership? Now, keep in mind I went to Stanford to be a great engineer, and I started at Intel in the engineering department. Engineering was where I lived. As I moved up within the organization, I had to start spending time with other areas of the business. Departments like marketing, business, legal, communications, etc. Guess which one I spent the most time in? Trick question -- it was the psychology department! A huge part of a leader’s job is seeking to understand and influence people. It's one of the most important skills in this role. I would never have believed this as an 18-year-old engineer! Some of the writings and efforts of Pat Lencioni, the The Table Group and the Five Dysfunctions of a team have been really helpful to me over the years to skill up in this area. 

887

44 Comments

Google Research Director Denny Zhou, who founded the LLM Reasoning Team at DeepMind, recently gave a great talk at Stanford’s CS25 class: Large Language Model Reasoning. This was one of the most intuitive talks about reasoning model training and application. He outlined new directions for training models to handle questions where answers aren’t easily verifiable. My team is currently working on an enterprise LLM project focused on using reasoning to extract relevant context and generate responses for unverifiable cases. It’s a tough but promising area, and it’s an interesting time to be building. [Link to Denny Zhou's talk: https://lnkd.in/gdJ6Mzi8]

1,026

8 Comments

Our Huxley-Gödel Machine learns to rewrite its own code, estimating its own long-term self-improvement potential. It generalizes on new tasks (SWE-Bench Lite), matching the best officially checked human-engineered agents. With Wenyi Wang, Piotr Piękos, Li Nanbo, Firas Laakom, Yimeng Chen, Mateusz Ostaszewski, Mingchen Zhuge. ArXiv: https://lnkd.in/e3zbgJQe Github: https://lnkd.in/e5_UW2MK

728

33 Comments

What I'm Reading This Week 📚 ⚡ Diffusion Coming to LLMs https://lnkd.in/gMrhRsvj Lots of announcements at Google IO this week. One that caught my eye is Gemini's new Diffusion model—applying image diffusion algorithms to text generation for faster speeds. This could be a game changer for latency-sensitive applications like IDE auto-completion and real-time code editing. 🔄 Real-Time Collaboration Architecture https://lnkd.in/ghwrBAxT Matthew Weidner delivers a brilliant deep dive into synchronizing state between central servers and multiple clients. If you've ever wondered how Google Docs manages concurrent edits without chaos, this is your answer. His exploration of operational transformation and server reconciliation patterns gives me new appreciation for the complexity behind seamless collaboration. 📈 Economic Reality Check https://lnkd.in/gfTf2TG9 Thomas Klitgaard from the NY Fed provides a down-to-earth explanation of why the US consistently runs trade deficits. While the current administration's trade war rhetoric continues, one thing is clear: rebalancing trade won't happen without painful adjustments for American consumers. The macroeconomic forces at play are more complex than political soundbites suggest. 🎯 Algebraic Effects Explained https://lnkd.in/g8zeFzWX A fascinating introduction to algebraic effects—essentially resumable exceptions that enable elegant control flow. While this cutting-edge programming language feature might be hard to wrap your head around, the Ante team does an excellent job explaining when and why you'd need such abstractions. 🤖 Google's Coding Agent https://jules.google/ Google launches Jules, their take on cloud-hosted agentic coding loops. Between AI-powered IDEs, local agents like Claude Code, and these cloud solutions, we're clearly still figuring out the right form factor for agentic programming. The overloaded "Codex" branding from OpenAI isn't helping clarity either, but the underlying trend toward autonomous code generation is undeniable. What's your take on the current state of AI coding assistants? Are you finding them more helpful or distracting in your development workflow? #TechArchitecture #AICodeGeneration #DistributedSystems #EconomicPolicy #FutureOfProgramming #AgenticDevelopment #DeveloperTools

15

🚀 Join Us for the Roblox Engineering Productivity Brunch on March 14th in San Mateo! At Roblox, we’re always looking for ways to help engineers build faster, better, and with fewer headaches. That’s why we’re hosting an Engineering Productivity Brunch, a chance to connect with industry pros, dive into the tools and strategies driving engineering efficiency at scale, and, of course, enjoy some great food. Here’s what’s in store: ✅ Tech Talks – Learn how we’re making engineering at scale frictionless with innovative tools that let teams focus on building, not busywork. ✅ Panel Discussion – Engineering leaders from Replit, Google, and Slack will share insights on tackling complex, career-defining challenges. ✅ Brunch & Networking – Meet like-minded engineers, swap ideas, and discuss the future of engineering productivity over a solid brunch spread. At Roblox, our engineers rely on powerful tools like our microservice lifecycle platform, code center, and advanced observability platform to streamline workflows, reduce downtime, and drive innovation. Space is limited, so grab your spot now! 👉 Register here https://lnkd.in/d2Uw_-Wi Want a preview of what we’ll be talking about? Check out our latest blog on how our engineering tools are shaping the future of immersive experiences: Read more. https://lnkd.in/gQm6WCMN Hope to see you there! 🚀 Max Ross, Jared Rosoff, Deon Poncini, David Rahn, Ying Dai, Xiaofeng Han, Scott Kennedy, Niranjan Tulpule, Sridhar Ramakrishnan

113

4 Comments

10/25: Tenth in the series on 25 Google papers for 25 years of WSC: Google-Wide Profiling The "Always On" Monitor: Most companies turn profiling off in production because they fear the overhead. At Google, we keep it on. This paper (Google-Wide Profiling) explains how to get continuous, fleet-wide visibility with a tiny tax: just 0.01% overhead. Why bother, you might ask? Because "lab" environments can be misleading. By profiling in prod, you find: Hardware differences you didn't expect; "Cold" code that is actually running hot; Library bottlenecks across every binary in the fleet. It’s the difference between guessing where the performance went and _knowing_. And for today's nano-banana poster for the paper, I tried a different prompt: [Make an infographic academic-style poster for the following paper. Use a visual aesthetic similar to the architectural blueprint roll -- Large blue sheets with white technical lines, measurements, and room labels.]

378

12 Comments

Get faster and smarter MOE inference straight out of the box. 👇Deep dive on scaling expert parallelism with TensorRT-LLM. LLMs with MOE promise higher model capacity without linearly increasing compute costs - but they introduce new challenges -- more conditional computation, dynamic routing, and non-uniform GPU utilization -- solved by TensorRT-LLM. ✨ New in TensorRT-LLM has native support for expert parallelism—designed for fast, efficient inference with MoE models like Mixtral (Mistral AI) and DeepSeek (DeepSeek AI). This gives you: ✅ Dynamic expert routing: Automatically route tokens to the top-k experts with minimal overhead. ✅ Efficient expert scheduling: Balance expert loads across GPUs using smart sharding and token bucketization. ✅ Memory-aware execution: Maximize hardware utilization while respecting memory budgets. ✅ Drop-in support: Use @HuggingFace models with minimal code changes via TensorRT-LLM's #Python API. 🧠 How it works: MoE models activate only a subset of "experts" for each token. This dynamic nature is powerful—but hard to optimize. It’s all done under the hood using custom #CUDA kernels and NCCL-based communication primitives—giving you low latency, high throughput, and better GPU scaling. ✨ TensorRT-LLM handles: ✅Token-expert mapping using the gating network. ✅Token sorting to batch same-expert tokens together. ✅Expert parallel execution across GPUs. ✅Merging outputs for final predictions. 🛠️ Developer workflow - here is the code to get started. # Clone the repo git clone https://lnkd.in/g-GiDX23 # Use included examples to load and run a Mixtral model cd TensorRT-LLM/examples/mixtral From there, the Python API lets you load the model, convert it with TensorRT, and run expert parallel inference—all with a few lines of code. Results? 📈 Performance at scale. Tests show up to 2.3x faster inference throughput compared to standard tensor parallelism when using 8 GPUs and top-2 experts per token. Even better—TensorRT-LLM keeps efficiency high across increasing batch sizes. Want to see it in action or contribute? 👉 Read the full tech blog: https://lnkd.in/g_7YV3vV 👉 Explore the code on GitHub: https://lnkd.in/gNjQ5W2U 👉 Follow updates in the TensorRT-LLM repo: https://lnkd.in/gqSHYQ4u Share your experiences with us.

78

3 Comments

Warehouse Scale Computing (WSC) sounds so easy, but its not. Read the retrospective from my colleagues Partha Ranganathan and Urs Hölzle and the three key lessons and insights they share after Google has been building an WSC infrastructure for 25 years now! Tl;dr: 🛬 Landings over launches 🚀 The power of roofshots 🛡️The primacy of security What an amazing read, thanks Partha and Urs for your amazing leadership over the years! Read more here 👇 https://lnkd.in/d8bR-YN9 And you must love their conclusion: As AI/ML becomes an increasingly important part of life, the scale of needed computing is likely to continue growing. The challenges and opportunities for innovation in WSC are similarly huge. Advancing AI may soon be able to help design WSCs themselves, opening whole new technical frontiers. [...] The best of WSC is yet to come. If the next 25 years are anything like the last, we are in for an exhilarating ride.

28

1 Comment

Early in my career everything was about building large scale homogeneous systems. What’s been more interesting over the last 5y has been heterogeneous multi cloud systems. The tradeoffs are more complex to build simple (irreducible complexity) and elegant systems in that space.

41

2 Comments