Principal Engineer – Agentic AI Development – Remote – Pennsylvania

Principal Engineer – Agentic AI Development

Location: Remote – Pennsylvania
Employment Type: Contract

 

Job Description:

Core Technical Skills

Must Have:

Azure AI Agent Service | Azure AI Foundry | Azure AI Search | Azure OpenAI Service | Microsoft Copilot Studio | MCP Development | LLM Orchestration | Prompt Flow | Python | TypeScript | JavaScript | RAG | Multi-Agent AI | Semantic Kernel

 

Required Qualifications

• 12+ years and at least 2+ years of experience working with AI/ML, conversational AI, Generative AI, or Agentic AI systems.
• Strong hands-on experience with Microsoft Copilot Studio, including low-code authoring and pro-code extensions.
• Strong production experience with Azure AI Foundry and related Azure AI services.
• Hands-on experience with:
• Azure OpenAI Service
• Azure AI Agent Service
• Azure AI Search
• Prompt Flow
• Semantic Kernel
• Strong understanding of Agentic AI architecture and LLM orchestration.
• Experience with multi-agent systems, tool/function calling, RAG, embeddings, agent memory, evaluation, and guardrails.
• Hands-on experience developing MCP / Model Context Protocol solutions.
• Strong programming skills in Python and TypeScript/JavaScript; C# is a plus.
• Experience with at least two agent/orchestration frameworks outside the Microsoft ecosystem, such as:
• LangChain
• LangGraph
• AutoGen
• CrewAI
• Comparable frameworks
• Strong understanding of REST APIs, GraphQL, event-driven architectures, CI/CD, and infrastructure-as-code.
• Experience with Power Platform administration, governance, ALM, custom connectors, and DLP policies.
• Excellent communication skills with the ability to work with both technical teams and business stakeholders.
• Bachelor's degree in Computer Science, Engineering, AI/ML, or a related field. Master's degree preferred.

 

Key Responsibilities

Agentic AI Development

• Architect and develop production-grade agentic AI applications using Azure AI Foundry, Azure OpenAI Service, Azure AI Agent Service, Azure AI Search, Prompt Flow, and Semantic Kernel.
• Build custom AI agents, multi-agent workflows, tool integrations, and orchestration frameworks.
• Develop enterprise RAG pipelines, vector search solutions, embeddings, indexing, and retrieval strategies.
• Implement agent memory, state management, context handling, and long-running autonomous workflows.
• Develop and maintain Model Context Protocol (MCP) servers and integrations.

Microsoft Copilot Studio

• Develop and extend enterprise solutions using Microsoft Copilot Studio.
• Build custom connectors, actions, Power Platform components, and reusable agent components.
• Implement advanced capabilities including multi-agent orchestration, MCP integration, human-in-the-loop workflows, and Copilot Tuning.
• Establish development patterns and reusable templates for citizen developers.
• Integrate Copilot Studio solutions with Azure AI Foundry when advanced pro-code capabilities are required.

Software Engineering & Integration

• Write production-quality code using Python, TypeScript, JavaScript, and/or C#.
• Design RESTful and GraphQL APIs and enterprise system integrations.
• Build event-driven architectures and scalable agent integration layers.
• Develop CI/CD pipelines and infrastructure automation.
• Build custom connectors and integration services for enterprise applications.
• Utilize Terraform, Bicep, Docker, and cloud-native technologies where appropriate.

AI Evaluation, Quality & Governance

• Develop comprehensive testing and evaluation frameworks for AI agents.
• Implement LLM-as-a-Judge, benchmark testing, regression testing, red-team testing, and human evaluation.
• Establish agent quality standards covering accuracy, hallucination, latency, cost, and reliability.
• Implement AI guardrails, content safety, PII protection, prompt-injection defenses, and responsible AI controls.
• Build agent observability and monitoring using Application Insights, telemetry, tracing, dashboards, and performance metrics.
• Support production incident response, root-cause analysis, and continuous improvement.

Low-Code Enablement

• Develop training and enablement programs for business users and citizen developers using Copilot Studio.
• Create reusable agent templates, connector libraries, development standards, and deployment checklists.
• Establish governance standards for decentralized agent development.
• Monitor adoption, usage, quality, compliance, and business value of enterprise agents.
• Identify when low-code solutions require transition to pro-code implementations.
•