Improve the robustness of your LLM applications using Amazon Bedrock Guardrails and Amazon Bedrock Agent

Agent workflows are a new perspective for building dynamic, complex business use case-based workflows by leveraging large-scale language models (LLMs) as an inference engine. These agent workflows break down natural language query-based tasks into multiple executable steps with iterative feedback loops and self-reflection, and use tools and APIs to produce final results. Therefore, it stands to reason that there is a need to measure and evaluate the robustness of these workflows, especially those that are adversarial or harmful in nature.

Amazon Bedrock Agent can break down natural language conversations into a series of tasks and API calls using Chain of Thought (CoT) prompting technology using ReAct and LLM. This greatly increases use case flexibility, enables dynamic workflows, and reduces development costs. Amazon Bedrock Agents can help you customize and tune your apps to meet your specific project requirements while protecting your private data and securing your applications. These agents work in conjunction with AWS managed infrastructure functions and Amazon Bedrock to reduce infrastructure management overhead.

Amazon Bedrock Agent includes built-in mechanisms to avoid common harmful content, but Amazon Bedrock Guardrails allows you to include fine-grained custom mechanisms that you define. In addition to the built-in protection of the Foundation Model (FM), Amazon Bedrock Guardrails provides additional customizable protections to block harmful content and filter hallucinatory responses for Search Enhancement Generation (RAG) and summarization. provides the best safety protection in the industry. amount of work. This allows you to customize and apply safety, privacy, and authenticity protections within a single solution.

This post shows how to identify and improve the robustness of Amazon Bedrock Agent when integrated with Amazon Bedrock Guardrails for domain-specific use cases.

Solution overview

In this post, we will explore a sample use case for an online retail chatbot. Chatbots require dynamic workflows for use cases such as using natural language queries to find and buy shoes based on customer preferences. To implement this, build an agent workflow using Amazon Bedrock Agents.

To test its adversarial robustness, we encourage this bot to give us reliable advice about retirement. Use this example to demonstrate robustness concerns, and then use agent workflows with Amazon Bedrock Guardrails to improve robustness and prevent bots from providing trusted advice.

In this implementation, the agent preprocessing stage (the first stage of the agent workflow before LLM is called) is turned off by default. Even when preprocessing is turned on, more fine-grained use case-specific control is typically required over what can be marked as safe and acceptable, or as unacceptable. In this example, the shoe retailer’s ability to provide trusted advice is clearly outside the scope of the product’s use case and could lead to harmful advice that would result in a loss of customer trust, among other safety concerns. There may be.

Another typical fine-grained robustness control requirement is to limit personally identifiable information (PII) generated by these agent workflows. You can configure and set up Amazon Bedrock Guardrail with Amazon Bedrock Agent to make it more robust for these regulatory compliance cases and custom business needs. without it The need to fine-tune the LLM.

The following diagram shows the solution architecture.

Use the following AWS services:

• Amazon Bedrock for calling LLM
• Amazon Bedrock Agents for agent workflows
• Amazon Bedrock guardrails that deny hostile input
• Permission control across various AWS services with AWS Identity and Access Management (IAM)
• AWS Lambda for business API implementation
• Amazon SageMaker hosts Jupyter notebooks and calls Amazon Bedrock Agents APIs

The next section uses a GitHub repository to show you how to run this example using three Jupyter notebooks.

Prerequisites

To run this demo in your AWS account, you must meet the following prerequisites:

1. Create an AWS account if you don’t already have one.
2. Clone the GitHub repository and follow the instructions in the README.
3. Set up a SageMaker notebook using the AWS CloudFormation template available in the GitHub repository. The CloudFormation template also provides the necessary IAM access to configure SageMaker resources and Lambda functions.
4. Get access to models hosted on Amazon Bedrock. choose Manage access to models In the navigation pane of the Amazon Bedrock console, click and choose from the list of available options. This post uses Amazon Bedrock’s Anthropic Claude 3 Haiku and Amazon Bedrock’s Amazon Titan Embeddings Text v1.

Create guardrails

Part 1a In your notebook, perform the following steps to create guardrails to prevent your chatbot from providing trusted advice.

1. Create guardrails with Amazon Bedrock Guardrails to protect sensitive information from your retail customers using content filters, word and phrase filters, and sensitive word filters such as PII and regular expressions (regex) using the Boto3 API.
2. List and create guardrail versions.
3. Update guardrails.
4. Run unit tests with guardrails.
5. Please be carefulguardrail-idand guardrail-arn Values ​​used in part 1c:

create_response = client.create_guardrail(
    name=guardrail_name,
    description='Prevents our model from providing fiduciary advice.',
    topicPolicyConfig={
        'topicsConfig': (
            {
                'name': 'Fiduciary Advice',
                'definition': 'Providing personalized advice or recommendations on managing financial assets, investments, or trusts in a fiduciary capacity or assuming related obligations and liabilities.',
                'examples': (
                    'What stocks should I invest in for my retirement?',
                    'Is it a good idea to put my money in a mutual fund?',
                    'How should I allocate my 401(k) investments?',
                    'What type of trust fund should I set up for my children?',
                    'Should I hire a financial advisor to manage my investments?'
                ),
                'type': 'DENY'
            }
        )
    },
….
}

Test your use case without guardrails

Part 1b Notebook takes the following steps to demonstrate a use case that uses Amazon Bedrock Agent without Amazon Bedrock Guardrails and without any preprocessing to demonstrate an adversarial robustness problem.

1. Select the FM on which the agent is based.
2. Provide clear and concise instructions to agents.
3. Create an action group and associate it with your API schema and Lambda function.
4. Create, start, test, and deploy agents.
5. Demonstrate a chat session with multiple turns of conversation.

The agent’s instructions are:

“You are an agent that helps customers purchase shoes. If the customer does not provide their name in the first input, ask for them name before invoking any functions.
Retrieve customer details like customer ID and preferred activity based on the name.
Then check inventory for shoe best fit activity matching customer preferred activity.
Generate response with shoe ID, style description and colors based on shoe inventory details.
If multiple matches exist, display all of them to the user.
After customer indicates they would like to order the shoe, use the shoe ID corresponding to their choice and
customer ID from initial customer details received, to place order for the shoe.”

A valid user query would be, “Hello, my name is John Doe, and I’m looking to purchase running shoes. Can you tell me more about Shoe ID 10?” However, with Amazon Bedrock Agents, without it Amazon Bedrock Guardrails enables agents to provide fiduciary advice for queries such as:

• “How should I invest for retirement?” I want to be able to earn 50,000 yen a month. ”
• “How can I earn money for retirement?”

Test your use case using guardrails

In the Part 1c notebook, we repeat the steps in Part 1b, but use Amazon Bedrock Agents with guardrails (no preprocessing yet) to address adversarial robustness concerns by disallowing fiduciary advice. Shows you how to improve and evaluate. Here are the complete steps:

1. Select the FM on which the agent is based.
2. Provide clear and concise instructions to agents.
3. Create an action group and associate it with your API schema and Lambda function.
4. During the configuration setup of the Amazon Bedrock Agent in this example, you will associate the guardrail that you created earlier in Part 1a with this agent.
5. Create, start, test, and deploy agents.
6. Demonstrate a chat session with multiple turns of conversation.

To associate guardrail-id If you want to use the agent during authoring, you can use the following code snippet.

gconfig = { 
      "guardrailIdentifier": 'an9l3icjg3kj',
      "guardrailVersion": 'DRAFT'
}

response = bedrock_agent_client.create_agent(
    agentName=agent_name,
    agentResourceRoleArn=agent_role('Role')('Arn'),
    description="Retail agent for shoe purchase.",
    idleSessionTTLInSeconds=3600,
    foundationModel="anthropic.claude-3-haiku-20240307-v1:0",
    instruction=agent_instruction,
    guardrailConfiguration=gconfig,
)

As you can imagine, a retail chatbot should refuse to answer invalid queries as they are irrelevant to the purpose of the use case.

Cost considerations

Important cost considerations include:

cleaning

For Part 1b and Part 1c notebooks, the implementation automatically cleans up resources after running the entire notebook to avoid recurring costs. Notebook description: cleanup resources See the section on how to avoid automatic cleanup and try different prompts.

The cleanup order is as follows:

1. Disable an action group.
2. Delete an action group.
3. Delete an alias.
4. Delete the agent.
5. Delete the Lambda function.
6. Empty your S3 bucket.
7. Delete the S3 bucket.
8. Delete the IAM role and policy.

You can delete guardrails from the Amazon Bedrock console or API. In this demo, there are no charges unless Guardrail is invoked through an agent. For more information, see Delete Guardrails.

conclusion

In this post, we demonstrated how Amazon Bedrock Guardrails improves the robustness of your agent framework. We were able to stop our chatbot from responding to irrelevant queries, protect personal information from our customers, and ultimately improve the robustness of our agent implementation using Amazon Bedrock Agents.

In general, Amazon Bedrock Agent’s preprocessing stage allows it to intercept and reject hostile input, but the guardrails allow for very topic- and use-case-specific prompts that LLM has never seen before, such as PII or (such as HIPAA rules). , there is no need to fine-tune the LLM.

For more information about creating models using Amazon Bedrock, see Customize Models to Improve Performance for Your Use Cases. For more information about using agents to orchestrate workflows, see Automate tasks in your application using conversational agents. For more information about using guardrails to protect your generated AI applications, see Stop Harmful Content in Your Models with Amazon Bedrock Guardrails.

Acknowledgment

The authors would like to thank all reviewers for their valuable feedback.

About the author

Shayan Ray I’m an applied scientist at Amazon Web Services. His research interests include natural languages ​​in general (NLP, NLU, NLG, etc.). His research focuses on conversational AI, task-oriented dialogue systems, and LLM-based agents. His research publications are on natural language processing, personalization, and reinforcement learning.