Building Self-Evolving LLM Agents on AWS
This blog series explores the design and implementation of self-evolving LLM agents on AWS. The primary use case is an LLM-powered agent that handles user requests related to AWS services via WhatsApp. The system comprises two key agents:
AWS Operator Agent: Manages user tasks related to AWS services. If it cannot fulfill a request, it automatically creates a user story in Azure DevOps. Developer Agent: Picks up the user story, develops the required changes, and submits a pull request. After manual review and commit, the DevOps pipeline updates the AWS Operator with the newly integrated tool.
Introduction
Seamless Multi-Channel Communication
The system supports multiple communication channels, including WhatsApp and Email, allowing users to interact effortlessly with LLM agents. Messages from these channels are captured via "Receivers," which convert the channel messages into a standardized format for LLM processing.
WhatsApp Integration: Uses a webhook, processed via API Gateway and AWS Lambda. Email Integration: Amazon SES -> SNS -> Lambda. Message Routing: All incoming messages are routed through a single Amazon SQS queue, ensuring a uniform message format.
This design also enables cross-channel communication, allowing users to switch seamlessly between communication platforms while maintaining session continuity.
Agents (Task Handlers)
The AWS Operator Agent and Developer Agent are built using LangGraph, providing a structured approach to managing workflows.
AWS Operator Agent:
Executes AWS operations and manages user tasks. Creates user stories in Azure DevOps for unhandled requests. Stores session and state data in DynamoDB. Communicates back to users via WhatsApp or email.
Developer Agent:
Develops code for user stories created by the AWS Operator. Submits pull requests for review and merging.
LLM Router (Model & Guardrails Layer)
The LLM Router connects to multiple LLM providers (OpenAI, AWS, Azure, Google Cloud, and Anthropic) and applies governance controls using NVIDIA NeMo Guardrails. It enforces safeguards such as:
Content moderation Off-topic detection PII handling Hallucination prevention (RAD enforcement) Jailbreak detection
This layer is powered by AWS Lambda, ensuring secure and policy-compliant model interactions.
State Management
To keep graph execution efficient and avoid token bloat, we use DynamoDB-based checkpointing combined with message pruning via the langgraph-reducer package. This setup allows safe trimming of growing message histories without breaking tool/AI message chains. It supports both inline reducers and dedicated pruning nodes.
Code Repos
WhatsApp Webhook: https://github.com/skamalj/whatsappwebhook.git WhatsApp Processor: https://github.com/skamalj/whatsapp-processor Email Processor: https://github.com/skamalj/sns-to-sqs-lambda.git Operator Agent: https://github.com/skamalj/computeagent.git Developer Agent: https://github.com/skamalj/devagent.git