Infrastructure Management

Microsoft and Google just rewrote the map for data centre growth. The next chapter starts now. Let’s break it down 👇 → Microsoft: 55 MW, all-renewable, new cloud region in Sweden (that’s five in the Nordics!). Powered by wind and hydro, built for AI, connected with fiber and subsea cables. Fast, green, and built for low-latency workloads. → Google: 150 MW hyperscale campus in Chile, using solar and geothermal. The goal? 60% renewables by 2035. This site will drive local jobs, fuel AI, and unlock a new digital backbone for South America. But here’s the real headline: Hyperscaler CapEx will almost DOUBLE in two years. → $170B (2024) → $320B (2026) Let that sink in! Why the surge? Three forces: 1️⃣ AI is exploding-training and inference need more power, everywhere. 2️⃣ Data sovereignty-regulations demand regional builds. 3️⃣ Sustainability-green certification is now a must-have. What’s changed? The old way: Centralised mega-cities with “cheap land.” The new way: Go where clean power, fast networks, and local laws align. Here’s what I’m seeing on the ground: → Every site selection starts with power-how green? How reliable? → Latency matters-close to users, close to the edge. → Policy is king-compliance first, not an afterthought. For companies looking to scale: • Pick sites with renewable energy as a priority. Power = competitive edge. • Build for low latency and local rules. It pays off later. • Watch where the hyperscalers invest. The ecosystem follows. Microsoft in Sweden. Google in Chile. These are not one-offs. They’re signals for the whole industry. The new geography of hyperscale is about green power, distributed sites, and AI at the core. Where do you see the biggest shift-energy, policy, or connectivity? Share how you’re navigating the new map.

𝐀𝐫𝐞 𝐘𝐨𝐮 𝐏𝐢𝐜𝐤𝐢𝐧𝐠 𝐭𝐡𝐞 𝐑𝐢𝐠𝐡𝐭 𝐀𝐳𝐮𝐫𝐞 𝐒𝐞𝐫𝐯𝐢𝐜𝐞𝐬 𝐀𝐜𝐫𝐨𝐬𝐬 𝐀𝐥𝐥 𝟖 𝐋𝐚𝐲𝐞𝐫𝐬 𝐨𝐟 𝐭𝐡𝐞 𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐀𝐈 𝐒𝐭𝐚𝐜𝐤? "We'll build it on Azure" sounds simple. Then you open the console and realize Azure has eight different layers of agentic AI services and picking wrong on any one gets expensive fast. 𝐖𝐡𝐚𝐭 𝐝𝐨𝐞𝐬 𝐭𝐡𝐞 𝐟𝐮𝐥𝐥 𝐬𝐭𝐚𝐜𝐤 𝐥𝐨𝐨𝐤 𝐥𝐢𝐤𝐞? 1. Deployment and Infrastructure:  Azure ML Managed Endpoints, Container Apps, AKS, Functions, App Service, Container Registry, GPU VMs (NC/ND series). The choice is mostly about control vs abstraction same agent, very different ops cost. 2. Evaluation and Monitoring:  Azure AI Foundry Evaluations (groundedness, relevance, safety), Azure Monitor, Application Insights, Content Safety, Microsoft Purview, Responsible AI dashboard. Evaluation isn't optional in production it's the difference between catching a regression and shipping one. 3. Foundation Models:  Azure OpenAI (GPT-4o, GPT-4.1, o-series), plus Mistral, Meta Llama, Cohere, DeepSeek, Microsoft Phi, Grok. Available as serverless API or self-hosted via the Foundry model catalog. 4. Orchestration Frameworks:  Microsoft Agent Framework 1.0, Azure AI Foundry Agent Service, Semantic Kernel, AutoGen, Prompt Flow, Logic Apps (1,400+ connectors), LangChain/LangGraph. Note: Semantic Kernel and AutoGen are now in maintenance mode. Microsoft Agent Framework is the forward-looking choice. 5. Vector Databases:  Azure AI Search, Cosmos DB (vector search), PostgreSQL with pgvector, Azure Cache for Redis. Third-party: Qdrant, Weaviate, Milvus. 6. Embedding Models:  Azure OpenAI embeddings (text-embedding-3-large, -small, ada-002), Cohere Embed v3/v4, Azure AI Vision for multimodal. Use the same model for indexing and retrieval change one without the other and quality silently collapses. 7. Data Ingestion and Extraction:  Document Intelligence, AI Search indexers, Microsoft Fabric/OneLake, Data Factory, Functions for custom ingestion, Content Understanding for multimodal. Most RAG quality is decided here, before the model is ever called. 8. Memory and Context:  Foundry Agent Service built-in state, Cosmos DB, Redis, AI Search, Microsoft Agent Framework session management and checkpointing. PS: Found this useful? Join 3,000+ AI architects and engineering leaders from Microsoft, Google, IBM, PwC and others reading my weekly newsletter 𝗗𝗶𝗮𝗿𝘆 𝗼𝗳 𝗮𝗻 𝗔𝗜 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁. I break down real enterprise AI systems, agentic patterns, and what actually works in production. ✉️ Free subscription: https://lnkd.in/exc4upeq #AzureAI #AgenticAI #CloudArchitecture

The G7 Toolkit for Artificial Intelligence in the Public Sector, prepared by the OECD.AI and UNESCO, provides a structured framework for guiding governments in the responsible use of AI and aims to balance the opportunities & risks of AI across public services. ✅ a resource for public officials seeking to leverage AI while balancing risks. It emphasizes ethical, human-centric development w/appropriate governance frameworks, transparency,& public trust. ✅ promotes collaborative/flexible strategies to ensure AI's positive societal impact. ✅will influence policy decisions as governments aim to make public sectors more efficient, responsive, & accountable through AI. Key Insights/Recommendations: 𝐆𝐨𝐯𝐞𝐫𝐧𝐚𝐧𝐜𝐞 & 𝐍𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐒𝐭𝐫𝐚𝐭𝐞𝐠𝐢𝐞𝐬: ➡️importance of national AI strategies that integrate infrastructure, data governance, & ethical guidelines. ➡️ different G7 countries adopt diverse governance structures—some opt for decentralized governance; others have a single leading institution coordinating AI efforts. 𝐁𝐞𝐧𝐞𝐟𝐢𝐭𝐬 & 𝐂𝐡𝐚𝐥𝐥𝐞𝐧𝐠𝐞𝐬 ➡️ AI can enhance public services, policymaking efficiency, & transparency, but governments to address concerns around security, privacy, bias, & misuse. ➡️ AI usage in areas like healthcare, welfare, & administrative efficiency demonstrates its potential; ethical risks like discrimination or lack of transparency are a challenge. 𝐄𝐭𝐡𝐢𝐜𝐚𝐥 𝐆𝐮𝐢𝐝𝐞𝐥𝐢𝐧𝐞𝐬 & 𝐅𝐫𝐚𝐦𝐞𝐰𝐨𝐫𝐤𝐬 ➡️ focus on human-centric AI development while ensuring fairness, transparency, & privacy. ➡️Some members have adopted additional frameworks like algorithmic transparency standards & impact assessments to govern AI's role in decision-making. 𝐏𝐮𝐛𝐥𝐢𝐜 𝐒𝐞𝐜𝐭𝐨𝐫 𝐈𝐦𝐩𝐥𝐞𝐦𝐞𝐧𝐭𝐚𝐭𝐢𝐨𝐧 ➡️provides a phased roadmap for developing AI solutions—from framing the problem, prototyping, & piloting solutions to scaling up and monitoring their outcomes. ➡️ engagement + stakeholder input is critical throughout this journey to ensure user needs are met & trust is built. 𝐄𝐱𝐚𝐦𝐩𝐥𝐞𝐬 𝐨𝐟 𝐀𝐈 𝐢𝐧 𝐔𝐬𝐞 ➡️Use cases include AI tools in policy drafting, public service automation, & fraud prevention. The UK’s Algorithmic Transparency Recording Standard (ATRS) and Canada's AI impact assessments serve as examples of operational frameworks. 𝐃𝐚𝐭𝐚 & 𝐈𝐧𝐟𝐫𝐚𝐬𝐭𝐫𝐮𝐜𝐭𝐮𝐫𝐞: ➡️G7 members to open up government datasets & ensure interoperability. ➡️Countries are investing in technical infrastructure to support digital transformation, such as shared data centers and cloud platforms. 𝐅𝐮𝐭𝐮𝐫𝐞 𝐎𝐮𝐭𝐥𝐨𝐨𝐤 & 𝐈𝐧𝐭𝐞𝐫𝐧𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐂𝐨𝐥𝐥𝐚𝐛𝐨𝐫𝐚𝐭𝐢𝐨𝐧: ➡️ importance of collaboration across G7 members & international bodies like the EU and Global Partnership on Artificial Intelligence (GPAI) to advance responsible AI. ➡️Governments are encouraged to adopt incremental approaches, using pilot projects & regulatory sandboxes to mitigate risks & scale successful initiatives gradually.

If you’re building anything with LLMs, your system architecture matters more than your prompts. Most people stop at “call the model, get the output.” But LLM-native systems need workflows, blueprints that define how multiple LLM calls interact, how routing, evaluation, memory, tools, or chaining come into play. Here’s a breakdown of 6 core LLM workflows I see in production: 🧠 LLM Augmentation Classic RAG + tools setup. The model augments its own capabilities using: → Retrieval (e.g., from vector DBs) → Tool use (e.g., calculators, APIs) → Memory (short-term or long-term context) 🔗 Prompt Chaining Workflow Sequential reasoning across steps. Each output is validated (pass/fail) → passed to the next model. Great for multi-stage tasks like reasoning, summarizing, translating, and evaluating. 🛣 LLM Routing Workflow Input routed to different models (or prompts) based on the type of task. Example: classification → Q&A → summarization all handled by different call paths. 📊 LLM Parallelization Workflow (Aggregator) Run multiple models/tasks in parallel → aggregate the outputs. Useful for ensembling or sourcing multiple perspectives. 🎼 LLM Parallelization Workflow (Synthesizer) A more orchestrated version with a control layer. Think: multi-agent systems with a conductor + synthesizer to harmonize responses. 🧪 Evaluator–Optimizer Workflow The most underrated architecture. One LLM generates. Another evaluates (pass/fail + feedback). This loop continues until quality thresholds are met. If you’re an AI engineer, don’t just build for single-shot inference. Design workflows that scale, self-correct, and adapt. 📌 Save this visual for your next project architecture review. 〰️〰️〰️ Follow me (Aishwarya Srinivasan) for more AI insight and subscribe to my Substack to find more in-depth blogs and weekly updates in AI: https://lnkd.in/dpBNr6Jg

🚀 Revolutionizing Infrastructure Application Management: A GitOps Journey with Terraform, ArgoCD, Kargo🚀 In the ever-evolving world of cloud-native development, a game-changing approach to infrastructure and application deployment that's transformed our team's efficiency and reliability. By combining Terraform's powerful Infrastructure as Code (IaC) capabilities with GitOps principles using ArgoCD, we can create a seamless, version-controlled deployment ecosystem that brings unprecedented clarity and control to our infrastructure management. The Power of Terraform IaC : Terraform has been a game-changer in how we define and provision infrastructure. Instead of manual configurations and error-prone click-ops, we now: 1. Describe our entire infrastructure as code 2. Ensure consistent, repeatable deployments 3. Manage complex multi-cloud environments with ease 4. Leverage state management for precise infrastructure tracking GitOps: A Single Source of Truth : Integrating ArgoCD has taken our deployment strategy to the next level. Now, our entire infrastructure and application state is declaratively defined and automatically synchronized from Git repositories. This means: 1. Every infrastructure change is a pull request 2. Complete audit trail of all modifications 3. Self-healing infrastructure that automatically converges to the desired state 4. Simplified rollbacks and version control Real-World Impact What started as an experiment has become the standard approach. We can dramatically reduced deployment errors, increased team collaboration, and gained unprecedented visibility into our infrastructure lifecycle. 💡 Pro Tip: Start small. Begin by converting one service or environment to this approach and watch the benefits compound. 💡💡💡💡💡 #DevOps #CloudNative #Terraform #GitOps #Kubernetes #ArgoCD #CloudEngineering

After my post a couple of days back on Agentic AI architecture, a few folks pinged me asking a very practical question. If you are already on a hyperscaler, should you build these agentic components yourself or simply adopt the in-house AI stack? This is the classic build vs buy dilemma, but with a very specific twist for Generative AI in 2025. My simple take is, adopt the gravity components and build the edge components. Data Registries and RAG infrastructure sit close to your data. Native tools win here because of data gravity. But Agent Registries, MCP Registries, and Observability need more flexibility and custom control. The native versions often feel too rigid for fast moving enterprise AI needs. Let me try to break it down when advising platform and product teams. 💠 Data Registry: Go Native Azure Purview, AWS DataZone, and GCP Dataplex win because they live inside your cloud estate. They give you governance, lineage, and access control across lakes and warehouses from day one. Purview stands out if you are already deep in the Microsoft ecosystem. 💠 RAG Systems: Hybrid Start with native if your use case is simple. Bedrock Knowledge Bases or Azure AI Search get you to value fast. Move to custom only when you need advanced retrieval like graph RAG, reranking, or hierarchical retrieval. Tools like LlamaIndex or LangChain give you that flexibility on top of managed vector stores. 💠 Agent Registry: Go Custom This is the part that surprises many people. Native agent services look convenient, but they limit how your agent reasons, loops, or manages state. If you want to switch from ReAct to Plan-and-Solve, or add human approval inside the chain, the native tools slow you down. A cleaner strategy is to build agents as microservices and register them yourself. Use LangGraph, CrewAI, or Semantic Kernel, and deploy them as containers. Treat the cloud as runtime, not the brain. 💠 MCP Registry: Depends Azure is ahead here. Azure API Center allows you to register MCP servers and maintain a private organizational catalog. If you are on Azure, use it. On AWS or GCP, you will end up building a simple internal directory that maps tool names to endpoints. 💠 Observability: Use Specialized Tools CloudWatch and Azure Monitor are great for servers, but they cannot tell you why an LLM hallucinated or why a retrieval step failed. Tools like Langfuse, LangSmith, or Arize give you trace visibility, prompt history, cost tracking, and failure debugging. You can self-host them if needed. In a nutshell, the strategy that I usually follow is, ➡️ Native for data. ➡️ Custom for orchestration and agents. ➡️ Native for governance. ➡️ Specialized tools for observability. I would love to hear other viewpoints on this topic. I write about #artificialintelligence | #technology | #startups | #mentoring | #leadership | #financialindependence   PS: All views are personal Vignesh Kumar

We are asking the wrong question about AI and jobs. The conversation is still stuck on prompt engineers, coding copilots, and automation of white-collar tasks. But AI is no longer just a productivity tool. It is becoming infrastructure — embedded in energy grids, financial systems, logistics networks, healthcare workflows, robotics fleets, and national policy. Infrastructure does not simply replace tasks. It reorganizes responsibility. The durable careers of the next decade will not belong to those who master the most visible #AI tools. They will belong to those who understand how AI systems behave under constraint — across regulation, physical environments, energy limits, and institutional accountability. In this extended article, I explore: • Why most students are preparing for the wrong battlefield • How AI shifts labor from execution to stewardship • Where orchestration, governance, and integration roles will grow • Why blue-collar + AI may expand faster than white-collar automation • How universities must redesign preparation pathways The future of work in the Accelerated AI era is not about competing with machines. It is about designing, stabilizing, governing, and owning them responsibly. Would value your perspective — especially from educators, policymakers, and students preparing for 2026–2035. DC*

If you're an AI Engineer wanting to move out of the simple LLM API calling paradigm and understand how LLM inference actually works, this is a nice starting point. - Explains what LLM inference is, how it differs from training, and how it works. - Covers deployment options like serverless vs. self-hosted, and OpenAI-compatible APIs. - Guides model selection, GPU memory planning, fine-tuning, quantization, and tool integration. - Details advanced inference techniques like batching, KV caching, speculative decoding, and parallelism. - Discusses infrastructure needs, challenges, and trade-offs in building scalable, efficient LLM inference systems. - Emphasizes the importance of observability, cost management, and operations (InferenceOps) for reliability. Link to guide by BentoML: https://bentoml.com/llm/ #AI #LLMs #GenAI

Imagine you’re running dozens or even hundreds of little apps, each wrapped in its own container, and you don’t want to manually babysit them. Kubernetes acts like an orchestra conductor, making sure every container knows when to start, stop, or move to a different “seat” when the music changes. What I really appreciate about Kubernetes is that it hides a lot of the complexity behind simple concepts like pods, nodes, and clusters. A pod is just a container (or a small group of them) bundled together, a node is the machine running those pods, and a cluster is the big group of nodes working as one. The magic is in how Kubernetes schedules, balances, and heals things automatically. If one pod crashes, Kubernetes just spins up another. If traffic spikes, it can add more pods. That resiliency is where it shines. Of course, Kubernetes can feel like overkill for small projects. I’ve seen teams struggle because they set it up too early without really needing it, and the learning curve can eat you alive. But when you do hit that stage where uptime, scaling, and automation matter, Kubernetes earns its reputation. It’s not so much about being the shiny tool but about bringing order to chaos when your architecture grows faster than your capacity to manage it manually. Kubernetes is less about technology and more about mindset. It forces you to think in terms of declarative infrastructure — you don’t tell it how to do something, you tell it what you want, and it figures out the rest. That shift in thinking has ripple effects across engineering culture. It’s one of those tools that, once you’ve lived with it for a while, changes the way you look at building and running software altogether. #Kubernetes #CloudComputing #DevOps #Containers #Microservices #CloudNative

AI data centres may be the only game in town right now… but what goes inside them besides AI chips? 💡 8 things you need to know When we think "data centre," the conversation usually stops at GPUs, TPUs, HBMs and high-performance processors. But in reality, these facilities are vast ecosystems of specialised technologies, each critical to keeping our digital world running 24/7. Here are the unsung enablers: 1️⃣ Medium voltage (MV) power distribution – Companies like ABB, Siemens, Hitachi & Schneider Electric are at the forefront of MV power distribution in data centres, providing reliable and efficient systems essential for smooth operations. 2️⃣ Backup power – The need for uninterrupted power is critical, and companies like GE, Caterpillar Inc., Generac & Cummins Inc. lead in generators and backup systems to ensure uptime during outages. 3️⃣ Uninterruptible power systems (UPS) – Rolls-Royce, Eaton, Vertiv & EnerSys provide UPS solutions that protect against disruptions and maintain continuous power supply. 4️⃣ Building automation – Johnson Controls, Trane Technologies, Cisco & Honeywell enable efficient management of data centre operations, from power usage to cooling and security. 5️⃣ Security systems – Palo Alto Networks, Bosch & Delta Electronics safeguard data centres against both digital and physical threats through advanced surveillance, access control, and network security. 6️⃣ Heating, ventilation, and air conditioning (HVAC) solutions – Munters, Mitsubishi Electric & Dover Corporation maintain optimal temperature and humidity to ensure equipment longevity and reliability. 7️⃣ Server cabinets –Hewlett Packard Enterprise, Fujitsu & Dell Technologies design cabinets that organise and protect critical hardware, supporting effective cable management, cooling, and security. 8️⃣ Low voltage (LV) power distribution – Companies like Vertiv, nVent & MPS Limited provide systems that ensure the safe and efficient distribution of low voltage power within data centres. 💡 Why it matters: The AI Data-centre revolution rides on more than just semiconductors. It’s the infrastructure stack: power, cooling, security, automation, that transforms silicon into real-world capability. Each layer represents opportunities for innovation, investment, and strategic positioning in the global digital economy. 📍 For Malaysia: If we want to play big in AI, we shouldn’t just think about chips. We should aim to be a hub for the full data centre value chain. 💬 Which of these 8 do you think Malaysia could lead in? As we approach the end of 2025, I’m re-posting one of my most popular posts of the year. I share semiconductor insights everyday. Follow me 👉 Andrew Chan Yik Hong for actionable perspectives on policy, strategy and industry shifts and ring the bell 🔔 to get notified whenever I post. 💬 If this post resonates with you, re post, drop a comment or leave a like. I would love to hear your thoughts.