Build generative AI applications on Amazon Bedrock using the AWS SDK for Python (Boto3)

Amazon Bedrock is a fully managed service that provides a selection of high-performance foundational models (FM) from leading AI companies, including AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon, through a single API. Broad feature set for building generative AI applications with security, privacy, and responsible AI. With Amazon Bedrock, you can experiment and evaluate top FMs for your use case, privately customize them with your data using techniques such as fine-tuning and acquisition augmentation generation (RAG), and develop enterprise systems and data. You can use it to build agents to perform tasks. sauce. Amazon Bedrock is serverless, so you don’t have to manage any infrastructure and can securely integrate and deploy generated AI capabilities into your applications using the AWS services you know.

This post shows how to incorporate FM programmatically using Amazon Bedrock and the AWS SDK for Python (Boto3).

Solution overview

This solution uses an AWS SDK for Python script with the ability to call Anthropic’s Claude 3 Sonnet on Amazon Bedrock. This FM uses the prompt as input to generate output. The following diagram shows the solution architecture.

Prerequisites

Before calling the Amazon Bedrock API, make sure you have the following:

Deploy the solution

After you complete the prerequisites, you can start using Amazon Bedrock. Start by creating the script using the following steps.

1. Import the required libraries.

2. Configure the Boto3 client to use the Amazon Bedrock runtime and specify the AWS Region.

# Set up the Amazon Bedrock client
bedrock_client = boto3.client(
    	service_name="bedrock-runtime",
    region_name="us-east-1"
)

3. Define the model to call using the model ID. This example uses Anthropic’s Claude 3 Sonnet on Amazon Bedrock.

# Define the model ID
model_id = "anthropic.claude-3-sonnet-20240229-v1:0"

4. Assign prompts. Prompts are messages used to interact with FM during invocation.

# Prepare the input prompt.
prompt = "Hello, how are you?"

Rapid engineering techniques improve FM performance and improve results.

Before invoking an Amazon Bedrock model, you must define a payload that serves as a set of instructions and information to guide the model generation process. This payload structure varies depending on the model you choose. This example uses Anthropic’s Claude 3 Sonnet from Amazon Bedrock. Think of this payload as a blueprint for your model, and provide it with the context and parameters it needs to generate the desired text based on your specific prompts. Let’s analyze the key elements within this payload.

• anthropic_version – This specifies the exact version of Amazon Bedrock that you are using.
• max_tokens – This sets a limit on the total number of tokens that the model can generate in the response. Tokens are the smallest meaningful units of text (words, punctuation marks, subwords) that are processed and produced by large-scale language models (LLMs).
• Temperature – This parameter controls the level of randomness of the generated text. Higher values ​​can give you more creative and unexpected output, while lower values ​​can give you more conservative and consistent results.
• Top_k – This defines the number of most likely candidate words considered at each step during the generation process.
• Top_p – This affects the sampling probability distribution for selecting the next word. Higher values ​​prioritize frequently used words, while lower values ​​provide more variety and potentially surprising choices.
• message – This is an array containing the individual messages that the model processes.
• role – This defines the sender’s role within the message (user of the specified prompt).
• Content – This array holds the actual prompt text itself, represented as an object of type “text”.

5. Define the payload as follows:

payload = {
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 2048,
    "temperature": 0.9,
    "top_k": 250,
    "top_p": 1,
    "messages": (
        {
            "role": "user",
            "content": (
                {
                    "type": "text",
                    "text": prompt
                }
            )
        }
    )
}

6. I have set up the FM with which I want to interact with the parameters. Next, send a request to Amazon Bedrock, specifying the FM to interact with and the payload you defined.

# Invoke the Amazon Bedrock model
response = bedrock_client.invoke_model(
    modelId=model_id,
    body=json.dumps(payload)
)

7. After your request is processed, you can view the text results generated from Amazon Bedrock.

# Process the response
result = json.loads(response("body").read())
generated_text = "".join((output("text") for output in result("content")))
print(f"Response: {generated_text}")

Let’s take a look at the complete script.

import boto3
import json

# Set up the Amazon Bedrock client
bedrock_client = boto3.client(
    service_name="bedrock-runtime",
    region_name="us-east-1"
)

# Define the model ID
model_id = "anthropic.claude-3-sonnet-20240229-v1:0"

# Prepare the input prompt
prompt = "Hello, how are you?"

# Create the request payload
payload = {
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 2048,
    "temperature": 0.9,
    "top_k": 250,
    "top_p": 1,
    "messages": (
        {
            "role": "user",
            "content": (
                {
                    "type": "text",
                    "text": prompt
                }
            )
        }
    )
}

# Invoke the Amazon Bedrock model
response = bedrock_client.invoke_model(
    modelId=model_id,
    body=json.dumps(payload)
)

# Process the response
result = json.loads(response("body").read())
generated_text = "".join((output("text") for output in result("content")))
print(f"Response: {generated_text}")

Call the model with the prompt “Hello, how are you?” You will get the result shown in the following screenshot.

cleaning

When you finish using Amazon Bedrock, clean up temporary resources such as IAM users and Amazon CloudWatch logs to avoid unnecessary charges. Cost considerations depend on frequency of use, pricing of the model you choose, and resource utilization during script execution. For pricing details and cost optimization strategies such as choosing the right model, optimizing prompts, and monitoring usage, see Amazon Bedrock Pricing.

conclusion

This post showed how to use Boto3 to programmatically interact with Amazon Bedrock FM. We considered specific FM invocations and processing of generated text, and demonstrated the potential for developers to use these models in applications for a variety of use cases, including:

• Text generation – Generate creative content such as poems, scripts, songs, and even various programming languages
• code completion – Increase developer productivity by suggesting relevant code snippets based on existing code or prompts
• Data summary – Extract key insights and generate concise summaries from large datasets
• Conversational AI – Develop chatbots and virtual assistants that can converse in natural language

stay curious And explore how generative AI can revolutionize various industries. Explore different models and APIs and compare how each model provides different outputs. Find the model that suits your use case and use this script as a base to create agents and integrations within your solution.

About the author

Merlin Naidoo He is a Senior Technical Account Manager at AWS with over 15 years of experience in digital transformation and innovative technology solutions. His passion is connecting people of all backgrounds and leveraging technology to create meaningful opportunities that empower everyone. When he is not immersed in the world of technology, he can be found participating in active sports.