AI features with Bedrock: a hands-on guide for engineers

AI features with Bedrock: a hands-on guide for engineers. Generative AI inside your AWS account.

Most teams building modern applications face a common dilemma: they want to add AI capabilities without managing ML infrastructure or dealing with external APIs that expose sensitive data. AWS Bedrock solves this by bringing generative AI into your account with a simple API, letting you work with models from multiple providers while keeping everything secure and compliant.

What Bedrock is and why it matters

Bedrock is a fully managed service that gives you access to foundation models—large language models and image generation models—through a single API. Instead of provisioning your own infrastructure or building custom integrations with multiple AI vendors, you invoke models directly from your AWS account. The service handles authentication, scaling, and billing in a way that feels familiar to developers already using other AWS services.

For teams working with regulated data or strict compliance requirements, this matters because your prompts and model responses never leave your AWS account by default. You're not sending customer PII to third-party SaaS platforms or relying on external vendor security practices. That makes a real difference when you're handling healthcare records, financial data, or any information that demands privacy-first architecture.

Model choice — Access models from several providers through one API

One of Bedrock's strongest features is vendor diversity. You get access to models from Anthropic (Claude), Meta (Llama), Cohere, AI21 Labs, and Stability AI, all through the same interface. This matters because different models have different strengths: Claude excels at reasoning and nuanced analysis, Llama offers efficient inference for cost-conscious workloads, and Cohere provides strong performance on specific NLP tasks.

Instead of writing adapter code or maintaining separate SDKs, you pick your model and call it through the Bedrock API. Your code doesn't change structurally when you switch models—you just change a parameter. This flexibility lets you experiment without major refactoring.

Here's how you'd invoke Claude through the Python SDK:

import boto3
import json

client = boto3.client('bedrock-runtime', region_name='us-east-1')

response = client.invoke_model(
    modelId='anthropic.claude-3-sonnet-20240229-v1:0',
    body=json.dumps({
        'anthropic_version': 'bedrock-2023-06-01',
        'max_tokens': 1024,
        'messages': [
            {
                'role': 'user',
                'content': 'Summarize the key changes in our payment processing system.'
            }
        ]
    })
)

result = json.loads(response['body'].read())
print(result['content'][0]['text'])

Notice the response format: you get structured data back with the model's output. Each provider has slightly different response shapes, so you'll want to normalize that in production. The key insight is that once you handle one provider's format, adding another is straightforward—it's just a different model ID and potentially a different response parser.

When choosing a model, start by thinking about latency and cost. Claude 3.5 Sonnet balances intelligence and speed well for most general-purpose tasks. For high-volume workloads where you're processing thousands of documents daily, Llama 2 or 3 often gives better throughput per dollar. Run quick benchmarks with your actual use cases; don't assume the most capable model is the right fit financially.

Data stays in your account — Useful for privacy and compliance requirements

The privacy model is straightforward: when you call Bedrock, AWS doesn't store your prompts or responses for model improvement by default. The data stays in your account and region. AWS won't use it to train future versions of third-party models. This is a hard requirement for regulated industries and something you should verify in writing with your legal team before going live.

This becomes critical in scenarios like these: your healthcare platform processes patient queries, your fintech app analyzes transaction patterns, or your legal tech startup ingests confidential contracts. Sending that data to a general-purpose SaaS AI tool creates audit trail nightmares and compliance violations. Bedrock keeps it local.

You can strengthen this further by using Bedrock within a VPC endpoint, ensuring traffic never crosses the public internet. Set up a VPC endpoint for Bedrock in your region, then route all model invocations through it:

aws ec2 create-vpc-endpoint \
    --vpc-id vpc-12345678 \
    --service-name com.amazonaws.us-east-1.bedrock-runtime \
    --vpc-endpoint-type Interface

Now all communication between your application and Bedrock stays within AWS's network backbone. Add restrictive security group rules to limit which subnets can reach the endpoint, and you've got a tight privacy boundary. Combined with encryption in transit (TLS by default) and at rest (KMS integration), this gives you the data isolation that compliance auditors want to see.

Knowledge bases — Managed retrieval for document question answering

Knowledge bases solve a specific problem: you have a collection of documents—internal wikis, product documentation, customer support articles, or proprietary research—and you want an AI that can answer questions by searching and citing those documents. Without this, you'd need to build retrieval-augmented generation (RAG) yourself: chunk documents, embed them, store embeddings in a vector database, retrieve similar chunks, and pass them to the model. Knowledge bases handle all of that for you.

The workflow is simple. You upload documents to an S3 bucket, create a Bedrock knowledge base pointing to that bucket, and let AWS ingest and embed your content. Then when you ask a question, Bedrock retrieves relevant passages and sends them as context to your chosen model. The model answers based on those retrieved documents, and you get citations so users can verify the source.

Here's a practical example: imagine you're building a customer support chatbot for a SaaS platform. Your knowledge base contains 500 help articles, FAQs, and API documentation. A customer asks "How do I enable single sign-on for my team?" Knowledge bases retrieves the three most relevant articles, passes them to Claude, and Claude synthesizes an answer with proper citations.

Setting up a knowledge base takes a few steps:

1. Create an S3 bucket and upload your documents (PDFs, HTML, markdown, etc.)
2. Create an IAM role that allows Bedrock to read from that bucket
3. Create the knowledge base through the AWS console or CLI, specifying your bucket and chunking preferences
4. Sync the knowledge base to ingest and embed all documents
5. Query it from your application code

Here's the query pattern in Python:

import boto3
import json

client = boto3.client('bedrock-agent-runtime', region_name='us-east-1')

response = client.retrieve_and_generate(
    input={
        'text': 'How do I enable single sign-on for my team?'
    },
    retrieveAndGenerateConfiguration={
        'type': 'KNOWLEDGE_BASE',
        'knowledgeBaseConfiguration': {
            'knowledgeBaseId': 'KB-12345678',
            'modelArn': 'arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-sonnet-20240229-v1:0'
        }
    }
)

answer = response['output']['text']
citations = response.get('citations', [])

The service handles chunking strategy, embedding model selection, and storage. You focus on quality documents and user experience. One caveat: knowledge bases embed documents using AWS's own embedding model. If you need fine-tuned embeddings or control over the chunking logic, you might build RAG manually with OpenSearch or DynamoDB, but for most teams, the managed service saves weeks of engineering work.

Guardrails — Filters for sensitive content and topics

Generative models sometimes produce harmful, biased, or off-topic outputs. Guardrails let you define rules about what content the model can generate. You specify blocked topics, harmful content filters, word blocklists, and custom criteria. When a prompt or response violates a guardrail, Bedrock either refuses to process it or returns a pre-configured message.

This is especially important in customer-facing applications. You don't want your AI chatbot discussing topics you haven't reviewed, or generating content that violates your brand guidelines. Guardrails act as a content policy enforcement layer.

Common use cases for guardrails:

• Customer support bots: Block political discussions, block harmful advice, ensure responses stay relevant to your product
• Internal tools: Prevent the model from discussing competitors, sensitive pricing, or confidential initiatives
• Educational platforms: Block profanity, ensure responses are age-appropriate, prevent cheating by refusing to write essays wholesale
• Financial advisory: Block discussions of specific investment picks, restrict to general education, prevent legal advice

Creating a guardrail is straightforward. Define your rules as structured policies, apply the guardrail to your model invocation, and Bedrock enforces them. The enforcement is deterministic—the same prompt with the same guardrail always produces the same filtering decision, which matters for compliance audits.

When a guardrail blocks content, you get structured feedback in the response indicating which rule was violated. This lets you log incidents, alert ops, or adjust your guardrail policies based on real-world usage patterns.

Common mistakes

Ignoring model-specific response formats: Each provider's API returns slightly different JSON structures. Claude nests content in a messages list, while Cohere returns text directly. Not normalizing this across providers leads to bugs when you switch models. Build an abstraction layer early.

Using knowledge bases for real-time data: Knowledge bases are great for static documentation but not ideal if your documents change every hour. The ingestion and embedding process takes time. For dynamic data, consider calling your database directly and passing results as context to the model instead.

Underestimating guardrail latency: Guardrail enforcement adds milliseconds to each request. In high-throughput scenarios, this compounds. Test your end-to-end latency with guardrails enabled before committing to the architecture.

Forgetting to set up VPC endpoints: If you're handling sensitive data, VPC endpoints aren't optional—they're a control requirement. Plan for them from day one; retrofitting network architecture is painful.

Key takeaways

• Bedrock gives you one API for multiple foundation models, letting you experiment and optimize without code rewrites
• Data privacy is built in: prompts and responses stay in your account and region by default, critical for compliance-heavy workloads
• Knowledge bases turn static documents into searchable, cited answers, handling RAG complexity you'd otherwise need to build yourself
• Guardrails enforce content policies at the model layer, reducing risk in customer-facing and regulated applications
• Start with model choice experiments in development, use VPC endpoints in production for regulated data, and test guardrail performance impact early