Achieve cost-effective AI inference using Amazon Bedrock serverless capabilities with models trained on Amazon SageMaker

This post shows you how to use Amazon Bedrock, with its fully managed on-demand API, with trained or fine-tuned models in Amazon SageMaker.

Amazon Bedrock is a fully managed service that provides a selection of high-performance foundational models (FM) from leading AI companies, including AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon, through a single API. Broad feature set for building generative AI applications with security, privacy, and responsible AI.

Previously, if you wanted to use your own custom, fine-tuned model in Amazon Bedrock, you had to either self-manage the inference infrastructure in SageMaker or train the model directly within Amazon Bedrock, which required costly provisioning. throughput was required.

Amazon Bedrock Custom Model Import allows you to use new or existing models that have been trained or fine-tuned within SageMaker using Amazon SageMaker JumpStart. Once you import a supported architecture into Amazon Bedrock, you can access it on demand through Amazon Bedrock’s fully managed invocation model API.

Solution overview

At the time of writing, Amazon Bedrock supports importing custom models from the following architectures:

• Mistral
• Franc
• Meta Llama 2 and Llama 3

This post uses the Hugging Face Flan-T5 Base model.

The following section provides instructions for training a model with SageMaker JumpStart and importing it into Amazon Bedrock. You can then interact with your custom model through Amazon Bedrock Playground.

Prerequisites

Before you begin, make sure you have an AWS account with access to Amazon SageMaker Studio and Amazon Bedrock.

If you don’t already have an instance of SageMaker Studio, see Launching Amazon SageMaker Studio for instructions on creating one.

Train a model with SageMaker JumpStart

To train a Flan model with SageMaker JumpStart, follow these steps:

1. Open the AWS Management Console and navigate to SageMaker Studio.

2. In SageMaker Studio, select jump start in the navigation pane.

SageMaker JumpStart allows machine learning (ML) practitioners to choose from a wide selection of publicly available FMs with pre-built machine learning solutions that can be deployed in a few clicks.

3. Search for and select. Hug Face Fran-T5 Base

On the model details page, you can see a brief description of the model, how to deploy it, how to fine-tune it, and the format of training data required to customize the model.

4. choose train Start fine-tuning your model based on your training data.

Create a training job using default settings. By default, training jobs have recommended settings.

5. The examples in this post use a preconfigured sample dataset. If you use your own data, please specify its location. data section to ensure that it meets the formatting requirements.

6. Configure security settings such as AWS Identity and Access Management (IAM) roles, Virtual Private Cloud (VPC), and encryption.
7. Notice the value of Output artifact location (S3 URI) Use it later.
8. Submit a job to start training.

You can monitor jobs by selecting . training in Recruitment Dropdown menu. If the training job status looks like this: completionthe job is done. With default settings, training takes approximately 10 minutes.

Import the model into Amazon Bedrock

Once your model is trained, you can import it into Amazon Bedrock. Follow these steps:

1. In the Amazon Bedrock console, imported model under basic model in the navigation pane.
2. choose import model.

3. for Model nameenter a recognizable name for your model.
4. under Model import settingsselect Amazon SageMaker Model Select the radio button next to your model.

5. under service accessselect Create and use a new service role Enter a name for the role.
6. choose import model.

7. Importing the model takes approximately 15 minutes.

8. under playground In the navigation pane, select sentence.
9. choose Please select a model.

10. for categorychoose imported model.
11. for modelchoose flan-t5-tweak.
12. for throughputchoose on demand.
13. choose apply.

You can now work with your custom model. The following screenshot uses a custom model example to summarize the discussion about Amazon Bedrock.

cleaning

To clean up your resources, follow these steps:

1. If you do not want to continue using SageMaker, delete your SageMaker domain.
2. If you no longer need to maintain your model artifacts, delete the Amazon Simple Storage Service (Amazon S3) bucket where your model artifacts are stored.
3. To delete the imported model from Amazon Bedrock, imported model Select your model on the Amazon Bedrock console page, select the options menu (three dots), and select erase.

conclusion

In this post, you learned how to use Amazon Bedrock’s custom model import feature to make your own custom trained or fine-tuned models available for cost-effective inference on demand. By integrating SageMaker model training capabilities with Amazon Bedrock’s fully managed and scalable infrastructure, you can seamlessly deploy specialized models and access them through a simple API.

Whether you prefer the user-friendly SageMaker Studio console or the flexibility of SageMaker notebooks, you can train and import your models into Amazon Bedrock. This allows you to focus on developing innovative applications and solutions without the burden of managing complex ML infrastructure.

As the capabilities of large language models continue to evolve, the ability to integrate custom models into applications becomes increasingly valuable. With the Amazon Bedrock Custom Model Import feature, you can unlock the full potential of your specialized models and deliver customized experiences to your customers, while benefiting from the scalability and cost efficiency of a fully managed service. .

For more information about fine-tuning in SageMaker, see Instruction Fine-Tuning for FLAN T5 XL Using Amazon SageMaker Jumpstart. For more hands-on experience with Amazon Bedrock, check out the Building with Amazon Bedrock workshop.

About the author

joseph sadler He is a Senior Solutions Architect on AWS’s Global Public Sector team, specializing in cybersecurity and machine learning. With experience in the public and private sectors, he has expertise in cloud security, artificial intelligence, threat detection, and incident response. His diverse background helps him build robust and secure solutions that use cutting-edge technology to protect mission-critical systems.