Integrate the underlying model into your code using Amazon Bedrock

The rise of large-scale language models (LLMs) and fundamental models (FMs) has revolutionized the fields of natural language processing (NLP) and artificial intelligence (AI). Trained on vast amounts of data, these powerful models can generate human-like text, answer questions, and even engage in creative writing tasks. However, training and deploying such models from scratch is a complex and resource-intensive process, often requiring specialized knowledge and significant computational resources.

Amazon Bedrock is a fully managed service that provides developers with seamless access to cutting-edge FM through a simple API. Amazon Bedrock streamlines the integration of cutting-edge generative AI capabilities for developers, providing pre-trained models that can be customized and deployed without the need for extensive model training from scratch. Amazon simplifies the process while maintaining flexibility in model customization, making it easy for developers to use cutting-edge generative AI technology in their applications. Amazon Bedrock allows you to integrate advanced NLP features into your applications, such as language understanding, text generation, and question answering.

In this post, we explore how you can integrate Amazon Bedrock FM into your codebase to easily build powerful AI-driven applications. It guides you through the process of setting up your environment, creating an Amazon Bedrock client, prompting and wrapping your code, invoking your model, and working with different models and streaming calls. By the end of this post, you will have the knowledge and tools to harness the power of Amazon Bedrock FM, accelerate your product development timelines, and provide advanced AI capabilities to your applications.

Solution overview

Amazon Bedrock provides a simple and efficient way to use powerful FM through its API without training custom models. In this post, you’ll run your code in a Jupyter notebook inside VS Code and use Python. The process of integrating Amazon Bedrock into your codebase includes the following steps:

1. Set up your development environment by importing the required dependencies and creating an Amazon Bedrock client. This client serves as the entry point for interacting with Amazon Bedrock FM.
2. After setting up the Amazon Bedrock client, you can define prompts or code snippets to use to interact with FM. These prompts can contain natural language instructions or code snippets based on which the model processes and produces output.
3. After you define a prompt, you can call Amazon Bedrock FM by passing the prompt to the client. Amazon Bedrock supports a variety of models, each with their own strengths and features, so you can choose the model that best fits your use case.
4. Depending on your model and the prompts you provide, Amazon Bedrock produces output that includes natural language text, code snippets, or a combination of both. You can then process and integrate this output into your application as needed.
5. For certain models and use cases, Amazon Bedrock supports streaming calls, which allows you to interact with your models in real time. This is especially useful in conversational AI or interactive applications where multiple prompts and responses need to be exchanged with the model.

This post provides detailed code examples and explanations for each step to help you seamlessly integrate Amazon Bedrock FM into your codebase. These powerful models allow you to enhance your applications with advanced NLP features, accelerate your development process, and deliver innovative solutions to your users.

Prerequisites

Before starting the integration process, ensure that the following prerequisites are met:

• AWS account – To access and use Amazon Bedrock, you need an AWS account. If you don’t have one, you can create a new account.
• Development environment – Set up an integrated development environment (IDE) using your preferred coding language and tools. You can interact with Amazon Bedrock using the AWS SDKs available for Python, Java, Node.js, and more.
• AWS credentials – Configure AWS credentials in your development environment to authenticate with AWS services. The AWS documentation for your chosen SDK explains how to do this. In this post, we’ll look at a Python example.

With these prerequisites in place, you are ready to integrate Amazon Bedrock FM into your code.

Create a new file in your IDE. This example uses a Jupyter notebook (kernel: Python 3.12.0).

The next section shows how to implement the solution in a Jupyter notebook.

Set up the environment

First, import the dependencies required to interact with Amazon Bedrock. Below is an example of how to do this in Python.

The first step is importing boto3 and json:

Next, create an instance of the Amazon Bedrock client. This client serves as the entry point for interacting with FM. Below is a code example of how to create a client.

bedrock_runtime = boto3.client(
    service_name="bedrock-runtime",
    region_name="us-east-1"
)

Define prompts and code snippets

After setting up the Amazon Bedrock client, define the prompts and code snippets used to interact with FM. These prompts can contain natural language instructions or code snippets based on which the model processes and produces output.

In this example, we asked the model: “Hello, who are you?”.

To send a prompt to an API endpoint, you need to pass some keyword arguments. You can get these arguments from the Amazon Bedrock console.

1. In the Amazon Bedrock console, base model in the navigation pane.
   

2. choice Titan Text G1 – Express.
   

3. Please select a model name (Titan Text G1 – Express) to go to the API request.
   

4. Copy the API request.

{
"modelId": "amazon.titan-text-express-v1",
"contentType": "application/json",
"accept": "application/json",
"body": "{\"inputText\":\"this is where you place your input text\",\"textGenerationConfig\":{\"maxTokenCount\":8192,\"stopSequences\":(),\"temperature\":0,\"topP\":1}}"
}

5. Insert this code into a Jupyter notebook with the following slight changes:
   • Posts an API request to keyword arguments (kwargs).
   • The following changes are made on the prompt. Replace “Place your input text here” with “Hello, who are you?”
6. Print keyword arguments.

kwargs = {
 "modelId": "amazon.titan-text-express-v1",
 "contentType": "application/json",
 "accept": "application/json",
 "body": "{\"inputText\":\"Hello, who are you?\",\"textGenerationConfig\":{\"maxTokenCount\":8192,\"stopSequences\":(),\"temperature\":0,\"topP\":1}}"
}
print(kwargs)

This should give you the following output:

{'modelId': 'amazon.titan-text-express-v1', 'contentType': 'application/json', 'accept': 'application/json', 'body': '{"inputText":"Hello, who are you?","textGenerationConfig":{"maxTokenCount":8192,"stopSequences":(),"temperature":0,"topP":1}}'}

call model

Defining a prompt allows you to call Amazon Bedrock FM.

1. Pass the prompt to the client.

response = bedrock_runtime.invoke_model(**kwargs)
response

This launches the Amazon Bedrock model with the provided prompt and outputs the generated streaming body object response.

{'ResponseMetadata': {'RequestId': '3cfe2718-b018-4a50-94e3-59e2080c75a3',
'HTTPStatusCode': 200,
'HTTPHeaders': {'date': 'Fri, 18 Oct 2024 11:30:14 GMT',
'content-type': 'application/json',
'content-length': '255',
'connection': 'keep-alive',
'x-amzn-requestid': '3cfe2718-b018-4a50-94e3-59e2080c75a3',
'x-amzn-bedrock-invocation-latency': '1980',
'x-amzn-bedrock-output-token-count': '37',
'x-amzn-bedrock-input-token-count': '6'},
'RetryAttempts': 0},
'contentType': 'application/json',
'body': <botocore.response.StreamingBody at 0x105e8e7a0>}

The Amazon Bedrock runtime invocation model described above works for whichever FM you choose to invoke.

2. Unzip the JSON string as follows:

response_body = json.loads(response.get('body').read())
response_body

You should get a response similar to the following (this is the response you get from the Titan Text G1 – Express model to the prompt you provided):

{'inputTextTokenCount': 6, 'results': ({'tokenCount': 37, 'outputText': '\nI am Amazon Titan, a large language model built by AWS. It is designed to assist you with tasks and answer any questions you may have. How may I help you?', 'completionReason': 'FINISH'})}

try different models

Amazon Bedrock offers a variety of FMs, each with their own strengths and features. You can specify the model to use by passing . model_name Parameters when creating an Amazon Bedrock client.

1. Similar to the previous Titan Text G1 – Express example, get the API request from the Amazon Bedrock console. This time we will use Anthropic’s Claude on Amazon Bedrock.
   

{
"modelId": "anthropic.claude-v2",
"contentType": "application/json",
"accept": "*/*",
"body": "{\"prompt\":\"\\n\\nHuman: Hello world\\n\\nAssistant:\",\"max_tokens_to_sample\":300,\"temperature\":0.5,\"top_k\":250,\"top_p\":1,\"stop_sequences\":(\"\\n\\nHuman:\"),\"anthropic_version\":\"bedrock-2023-05-31\"}"
}

Anthropic’s Claude accepts prompts in a different way (\\n\\nHuman:), so the Amazon Bedrock console API request provides a prompt in a way that Anthropic’s Claude can accept.

2. Edit the API request and add in the keyword arguments.

   kwargs = {
     "modelId": "anthropic.claude-v2",
     "contentType": "application/json",
     "accept": "*/*",
     "body": "{\"prompt\":\"\\n\\nHuman: we have received some text without any context.\\nWe will need to label the text with a title so that others can quickly see what the text is about \\n\\nHere is the text between these <text></text> XML tags\\n\\n<text>\\nToday I sent to the beach and saw a whale. I ate an ice-cream and swam in the sea\\n</text>\\n\\nProvide title between <title></title> XML tags\\n\\nAssistant:\",\"max_tokens_to_sample\":300,\"temperature\":0.5,\"top_k\":250,\"top_p\":1,\"stop_sequences\":(\"\\n\\nHuman:\"),\"anthropic_version\":\"bedrock-2023-05-31\"}"
   }
   print(kwargs)

You should get the following response:

{'modelId': 'anthropic.claude-v2', 'contentType': 'application/json', 'accept': '*/*', 'body': '{"prompt":"\\n\\nHuman: we have received some text without any context.\\nWe will need to label the text with a title so that others can quickly see what the text is about \\n\\nHere is the text between these <text></text> XML tags\\n\\n<text>\\nToday I sent to the beach and saw a whale. I ate an ice-cream and swam in the sea\\n</text>\\n\\nProvide title between <title></title> XML tags\\n\\nAssistant:","max_tokens_to_sample":300,"temperature":0.5,"top_k":250,"top_p":1,"stop_sequences":("\\n\\nHuman:"),"anthropic_version":"bedrock-2023-05-31"}'}

3. After you define a prompt, you can pass the prompt to your client to call Amazon Bedrock FM.

response = bedrock_runtime.invoke_model(**kwargs)
response

You should get the following output:

{'ResponseMetadata': {'RequestId': '72d2b1c7-cbc8-42ed-9098-2b4eb41cd14e', 'HTTPStatusCode': 200, 'HTTPHeaders': {'date': 'Thu, 17 Oct 2024 15:07:23 GMT', 'content-type': 'application/json', 'content-length': '121', 'connection': 'keep-alive', 'x-amzn-requestid': '72d2b1c7-cbc8-42ed-9098-2b4eb41cd14e', 'x-amzn-bedrock-invocation-latency': '538', 'x-amzn-bedrock-output-token-count': '15', 'x-amzn-bedrock-input-token-count': '100'}, 'RetryAttempts': 0}, 'contentType': 'application/json', 'body': <botocore.response.StreamingBody at 0x1200b5990>}

4. Unzip the JSON string as follows:

response_body = json.loads(response.get('body').read())
response_body

This gives the following output for the given text title:

{'type': 'completion',
'completion': ' <title>A Day at the Beach</title>',
'stop_reason': 'stop_sequence',
'stop': '\n\nHuman:'}

5. Outputs completion.

completion = response_body.get('completion')
completion

The response is returned in the XML tags you defined, so you can consume the response and display it on the client.

' <title>A Day at the Beach</title>'

Call the model with streaming code

For certain models and use cases, Amazon Bedrock supports streaming calls, which allows you to interact with your models in real time. This is especially useful in conversational AI or conversational applications where multiple prompts and responses need to be exchanged with the model. For example, if you commission an article or story from FM, you may want to stream the output of the content generated.

1. Import dependencies and create an Amazon Bedrock client.

import boto3, json
bedrock_runtime = boto3.client(
service_name="bedrock-runtime",
region_name="us-east-1"
)

2. Define the prompt as follows:

prompt = "write an article about fictional planet Foobar"

3. Edit the API request and fill in the keyword arguments as before.
   Use API requests for the claude-v2 model.

kwargs = {
  "modelId": "anthropic.claude-v2",
  "contentType": "application/json",
  "accept": "*/*",
  "body": "{\"prompt\":\"\\n\\nHuman: " + prompt + "\\nAssistant:\",\"max_tokens_to_sample\":300,\"temperature\":0.5,\"top_k\":250,\"top_p\":1,\"stop_sequences\":(\"\\n\\nHuman:\"),\"anthropic_version\":\"bedrock-2023-05-31\"}"
}

4. You can now call Amazon Bedrock FM by passing a prompt to the client.
   what we use is invoke_model_with_response_stream instead of invoke_model.

response = bedrock_runtime.invoke_model_with_response_stream(**kwargs)

stream = response.get('body')
if stream:
    for event in stream:
        chunk = event.get('chunk')
        if chunk:
            print(json.loads(chunk.get('bytes')).get('completion'), end="")

I get a response like this as streaming output:

Here is a draft article about the fictional planet Foobar: Exploring the Mysteries of Planet Foobar Far off in a distant solar system lies the mysterious planet Foobar. This strange world has confounded scientists and explorers for centuries with its bizarre environments and alien lifeforms. Foobar is slightly larger than Earth and orbits a small, dim red star. From space, the planet appears rusty orange due to its sandy deserts and red rock formations. While the planet looks barren and dry at first glance, it actually contains a diverse array of ecosystems. The poles of Foobar are covered in icy tundra, home to resilient lichen-like plants and furry, six-legged mammals. Moving towards the equator, the tundra slowly gives way to rocky badlands dotted with scrubby vegetation. This arid zone contains ancient dried up riverbeds that point to a once lush environment. The heart of Foobar is dominated by expansive deserts of fine, deep red sand. These deserts experience scorching heat during the day but drop to freezing temperatures at night. Hardy cactus-like plants manage to thrive in this harsh landscape alongside tough reptilian creatures. Oases rich with palm-like trees can occasionally be found tucked away in hidden canyons. Scattered throughout Foobar are pockets of tropical jungles thriving along rivers and wetlands.

conclusion

In this post, you learned how to integrate Amazon Bedrock FM into your codebase. Amazon Bedrock enables you to use cutting-edge generative AI capabilities without training custom models, accelerating your development process and enabling you to build powerful applications with advanced NLP capabilities.

Whether you’re building a conversational AI assistant, a code generation tool, or another application that requires NLP capabilities, Amazon Bedrock provides a simple and efficient solution. By leveraging the power of FM through the Amazon Bedrock API, you can focus on building innovative solutions and delivering value to your users without worrying about the underlying complexity of your language model.

As you continue to explore and integrate Amazon Bedrock into your projects, be sure to stay up to date with the latest updates and features the service has to offer. Additionally, consider exploring other AWS services and tools that can complement and power your AI-driven applications, such as Amazon SageMaker for training and deploying machine learning models and Amazon Lex for building conversational interfaces. Please.

To further explore the capabilities of Amazon Bedrock, see the following resources:

Share and learn with the Community.aws Generative AI community.

Have fun coding and building with Amazon Bedrock!

About the author

Rajakumar Sampathkumar is a Principal Technical Account Manager at AWS, providing customers with guidance on business technology alignment and helping them reinvent their cloud operating models and processes. He is passionate about cloud and machine learning. Raj is also a machine learning specialist who works with AWS customers to design, deploy, and manage AWS workloads and architectures.

Yadukishore Tatavarti As a Senior Partner Solutions Architect at Amazon Web Services, I support customers and partners around the world. For the past 20 years, he has helped clients build enterprise data strategies and advised on generative AI, cloud implementation, migration, reference architecture creation, data modeling best practices, and data lake/warehouse architecture.