Lead AI Engineer (Agentic Systems)

About the Role: Grade Level (for internal use): 11 Lead AI Engineer (Agentic Systems)   Role Summary   As the Lead AI Engineer (Agentic Systems), you will   help   architect and build the organization’s next generation of autonomous AI workflows. This is a multidisciplinary technical role   operating   at the intersection of Software Engineering, Data Engineering, and Machine Learning   Engineering . You will move beyond simple "chatbots" to design production-grade Agentic Systems: intelligent applications capable of reasoning, planning, and executing complex tasks autonomously.   Responsibilities   Agentic Systems Architecture & Core Engineering   Architect & Build Multi-Agent Workflows: Lead the hands-on design and coding of stateful, production-grade agentic systems using Python and orchestration frameworks like   LangGraph ,   CrewAI , or   AutoGen .   Agent-to-Agent (A2A) Communication: Design and implement robust A2A protocols enabling autonomous agents to collaborate, hand off sub-tasks, and negotiate execution paths dynamically within multi-agent environments.   State Management & Orchestration: Engineer robust control flows for non-deterministic agents; implement complex message passing, memory persistence, and interruptible state handling to support long-running autonomous tasks.   Tool Interface Design (MCP): Implement and standardize the Model Context Protocol (MCP) to create universal interfaces between agents, data sources, and operational tools, ensuring modularity and scalability.   Model Integration & Optimization:   Utilize   proxy services ( i.e.   LiteLLM )   to manage model routing and fallback strategies;   optimize   context windows and inference costs across proprietary and open-source models.     Production Deployment: Containerize agentic workloads using Docker and orchestrate deployments on Kubernetes; leverage AWS   AgentCore   or similar cloud-native services for scalable infrastructure.   Data Engineering & Operational Real-Time Integration   Build Agent Data Pipelines: Write and   maintain   high-throughput ingestion pipelines (using Databricks or Python-based ETL) that transform raw operational signals into structured context for agents.   Real-Time Context Injection: Ensure agents have access to "operational real-time" data (seconds/minutes latency) by   optimizing   retrieval architectures and vector store performance.     Cross-Functional Engineering: Act as the technical bridge between Data Engineering and AI teams; translate complex agent requirements into concrete data schemas and pipeline specifications, while stepping in to resolve hands-on bottlenecks in data availability.   Observability, Governance & Human-in-the-Loop   LLMOps   & Tracing: Implement comprehensive observability using tools like   Langfuse   to trace agent reasoning steps,   monitor   token usage, and debug latency issues in production.     Safety & Control Frameworks: Design hybrid execution modes ranging from Human-in-the-Loop (HITL) for sensitive operations to fully autonomous execution; build "break-glass" mechanisms and guardrails for automated decision-making.     Evaluation & Reliability:   Establish   technical standards for testing non-deterministic outputs; automate evaluation pipelines to measure agent accuracy, hallucination rates, and drift before deployment.     Technical Leadership & Strategy   Technical Roadmap Definition: Partner with Product and Engineering leadership to scope feasibility for autonomous projects; define the "Agentic Architecture" roadmap.   Mentorship & Standards: Define code quality standards, architectural patterns, and PR review processes for the AI engineering team; upskill team members on the latest agentic frameworks and methodologies.   Innovation: Proactively prototype with emerging tools (e.g., new reasoning models, graph-based RAG) to solve high-value business problems, moving successful experiments into the production roadmap.     Qualifications   Required   Experience: 7+ years of total technical experience in Software Engineering, Data Engineering, or Machine Learning.   GenAI Specialization: 2+ years of specific experience building and deploying LLM-based applications or Agentic Systems in production.     Database & Lakehouse Mastery:   E xperience architecting storage layers for AI, including Vector Databases (e.g., Pinecone,   Weaviate ,   Qdrant ), NoSQL/Relational Databases (PostgreSQL, DynamoDB), and modern Data   Lakehouses   (specifically Databricks or Snowflake).     Cloud & Infrastructure:   E xpertise   in cloud architecture and container orchestration   (AWS, GCP, or Azure)   using Kubernetes and Docker. You must be comfortable deploying and scaling your own applications.   LLM Ecosystem:   F amiliarity with common LLM frameworks and orchestration libraries (e.g.,   LangGraph ,   LangChain ,   CrewAI ,   AutoGen ). You understand the mechanics of RAG, embeddings, and context window management.     Hybrid Engineering Skillset: A unique blend of Data Science (understanding model behavior, probability, and prompting) and Software Engineering (CI/CD, API design, asynchronous programming, and system reliability).     Language Proficiency: Advanced   proficiency   in Python for systems engineering, capable of writing modular, testable, and maintainable production code.     Education:   Bachelor’s degree in Computer Science , Engineering, Mathematics, or   a related   technical field.   Preferred   Advanced Education: Master’s degree or PhD in Computer Science, Artificial Intelligence, or a related quantitative field.   NLP Expertise: 5+ years of hands-on experience in Natural Language Processing (NLP), ranging from foundational techniques (e.g., text processing, embeddings, classification) to modern architectures.   Graph Technologies: Experience with Knowledge Graphs (e.g., Neo4j, AWS Neptune), Graph Databases, and   GraphML   (Graph Machine Learning) to support complex reasoning and relationship modeling.     Agentic Tooling: Specific experience with   LangGraph ,   LiteLLM ,   Langfuse , AWS   AgentCore , or implementing the Model Context Protocol (MCP).     Advanced Architectures: Proven   track record   of implementing Agent-to-Agent (A2A) communication, swarm intelligence, or multi-modal agent workflows.   Real-Time Operations: Experience working in environments requiring operational real-time processing (e.g., FinTech, Energy, Logistics).     Why This Role Matters   You   won't   just be building chatbots here; you will be architecting the organization’s "central nervous system." As the Lead AI Engineer for Agentic Systems, you are bridging the gap between static data models and active decision-making. The autonomous workflows you design—capable of planning, collaborating (A2A), and executing tasks—will fundamentally change how we   operate , moving us from human-dependent processes to self-healing, intelligent systems. This is a rare opportunity to define the standards for Agentic AI in a production environment, working with a stack that   represents   the absolute   cutting edge   of the industry.     About S&P Global Energy At S&P Global Energy, our comprehensive view of global energy and commodities markets enables our customers to make superior decisions and create long-term, sustainable value. Our four core capabilities are: Platts for news and pricing; CERA for research and advisory; Horizons for energy expansion and sustainability solutions; and Events for industry collaboration. S&P Global Energy is a division of S&P Global (NYSE: SPGI). S&P Global enables businesses, governments, and individuals with trusted data, expertise, and technology to make decisions with conviction. We are Advancing Essential Intelligence through world-leading benchmarks, data, and insights that customers need in order to plan confidently, act decisively, and thrive economically in a rapidly changing global landscape. Learn more at   www.spglobal.c