Our agent-based data assessment: modern AI workflows on AWS hyperscaler infrastructure

AI agents in the corporate context

The Data Assessment System combines modern conversational AI with a sophisticated AWS-based architecture to support companies in the structured collection and analysis of their data requirements. The agent guides potential customers through an intelligent self-service assessment that systematically captures use cases, data sources and KPIs.

Technological basics

At the core of the system is AWS Bedrock Agent with the Claude 3.7 Sonnet Foundation Model from Anthropic. This combination enables the agent to conduct complex data analysis conversations while providing both technical depth and a natural conversational style.

Scalable cloud architecture

The technical architecture follows modern serverless principles and makes optimum use of the advantages of the AWS hyperscaler infrastructure:

Frontend layer

• Responsive web interface: Static HTML/CSS/JavaScript application with modern chat interface
• Markdown rendering: Dynamic display of agent responses for optimal user experience
• Multi-language supportAutomatic speech recognition and localization

API and security layer

• AWS API GatewayHTTP API with CORS support for seamless integration
• Encrypted communicationEnd-to-end HTTPS encryption

AI processing layer

• Bedrock AgentCentral AI engine with custom prompt engineering
• Session Management: Persistent conversations with idle timeout
• Lambda FunctionsEfficient mediation between API Gateway and Bedrock Agent

Infrastructure as Code: Modern DevOps Practices

A particular focus is on the fully automated infrastructure. The use of Terraform as Infrastructure as Code (IaC) offers decisive advantages for data assessment. All infrastructure components are defined as code and can be versioned, enabling complete reproducibility of environments and eliminating "works on my machine" issues. Modular Terraform configurations allow dev, staging and production environments to be deployed consistently and automatically, with changes first tested in the development environment before being pushed to production.

# Beispiel: Bedrock Agent Konfiguration
resource "aws_bedrockagent_agent" "data_assessment_agent" {
  agent_name                  = "${var.project}-${var.stage}-agent"
  agent_resource_role_arn     = aws_iam_role.bedrock_agent_role.arn
  foundation_model            = var.foundation_model
  idle_session_ttl_in_seconds = var.session_ttl
  instruction                 = local_file.agent_prompt.content
}

Monitoring and cost optimization

The system has a Cloudwatch-based monitoring system that covers both technical and economic aspects. It enables continuous monitoring of all system components. Errors are automatically detected and reported. In addition, automatic recognition of completed data assessments is implemented so that the interested party can be contacted directly.

AWS budgets are used for cost control, which send proactive notifications when defined thresholds are exceeded. The serverless architecture helps to minimize operating costs, as resources are only billed when they are actually used.

Scalability and performance

The architecture is based on serverless components that automatically adapt to the current load. The API gateways and Lambda functions are optimized for low latency and ensure fast response times. The multi-AZ deployment ensures high availability through automatic failover in the event of individual availability zone failures.

Outlook: Integrated knowledge base

The current MVP already shows the potential for extended functionalities:

Data source integration

The integration of different data sources enables a more precise assessment of data quality. For example, users of the data assessment could upload anonymized product data or aggregated sales statistics in order to analyse the completeness and consistency of the data records. The direct connection to databases with structured metadata would also make it possible to use live data for real-time assessments.

Advanced AI functions

The integration of advanced AI functions would enable intelligent and context-aware data analysis. Orchestrating multiple specialized agents for different domains such as financial data, customer data or production data would allow complex data landscapes to be analyzed with domain-specific expertise, while predictive analytics based on historical data quality metrics could enable predictions about future data problems and thus support proactive measures to improve data quality.