Managing generative AI in the enterprise with Amazon SageMaker Canvas

With the rise of powerful foundational models (FMs) powered by services like Amazon Bedrock and Amazon SageMaker JumpStart, enterprises want granular control over which users and groups can access and use these models, which is critical for compliance, security, and governance.

Launched in 2021, Amazon SageMaker Canvas is a visual, point-and-click service that enables business analysts and citizen data scientists to use ready-to-use machine learning (ML) models or build custom ML models to generate accurate predictions without writing any code. SageMaker Canvas provides a no-code interface to use a wide range of FMs from both services out of the box. You can also customize model responses using Retrieval Augmented Generation (RAG) workflows using Amazon Kendra as a knowledge base or fine-tune using labeled datasets. This makes generative artificial intelligence (AI) capabilities more accessible to business analysts and data scientists, without the need for technical knowledge or writing code, making them more productive.

In this post, we analyze strategies for using AWS Identity and Access Management (IAM) policies to manage access to Amazon Bedrock and SageMaker JumpStart models from within SageMaker Canvas. You learn how to create fine-grained permissions to control invocation of out-of-the-box Amazon Bedrock models and prevent provisioning of SageMaker endpoints with specified SageMaker JumpStart models. We provide code examples aligned to common enterprise governance scenarios. At the end, you understand how to lock down access to generative AI capabilities based on your organization’s requirements and continue to use state-of-the-art AI within the no-code SageMaker Canvas environment in a secure and compliant manner.

This post covers a topic that will become increasingly important as more powerful AI models become available, making it a valuable resource for ML operators, security teams, and anyone managing AI within the enterprise.

Solution overview

The following diagram shows the solution architecture:

The architecture of SageMaker Canvas enables business analysts and data scientists to work with ML models without writing code. However, managing access to these models is essential to maintain security and compliance. When a user interacts with SageMaker Canvas, operations such as invoking a model or creating an endpoint are performed by a SageMaker service role. SageMaker user profiles can inherit default roles from the SageMaker domain or have user-specific roles.

By customizing the policies attached to this role, you can control which actions are allowed or denied and manage access to generative AI capabilities. In this post, we discuss the IAM policies you can use for this role to control operations within SageMaker Canvas, such as invoking models or creating endpoints, based on your enterprise organization’s requirements. We analyze two patterns for both Amazon Bedrock and SageMaker JumpStart models: restricting access to all models from the service or restricting access to specific models.

Managing Access to SageMaker Canvas from Amazon Bedrock

To work with Amazon Bedrock models, SageMaker Canvas calls the following Amazon Bedrock APIs:

• Foundation: InvokeModel – Call the model synchronously
• bedrock:InvokeModelWithResponseStream – Invoke the model synchronously and stream the response over a socket, as shown in the following diagram.

Additionally, SageMaker Canvas can call the bedrock:FineTune API to fine-tune large-scale language models (LLMs) on Amazon Bedrock. At the time of writing, SageMaker Canvas only allows fine-tuning of Amazon Titan models.

To use a specific LLM from Amazon Bedrock, SageMaker Canvas uses the model ID of the selected LLM as part of the API call. At the time of writing, SageMaker Canvas supports the following models from Amazon Bedrock, grouped by model provider:

• AI21
  • Jurassic 2 Mid: j2-mid-v1
  • Jurassic 2 Ultra: j2-ultra-v1
• Amazon
  • Titan: titan-text-premier-v1:*
  • Titan Large: titan-text-lite-v1
  • Titan Express: titan-text-express-v1
• Anthropological
  • Claude 2: claude-v2
  • Claude Instant: claude-instant-v1
• Kohia
  • Command Text: command-text-*
  • Command Write: command-light-text-*
• Meta
  • Llama 2 13B: llama2-13b-chat-v1
  • Llama 2 70B: llama2-70b-chat-v1

For a complete list of Amazon Bedrock model IDs, see Amazon Bedrock Model IDs.

Restrict access to all Amazon Bedrock models

To restrict access to all Amazon Bedrock models, you can modify the SageMaker role to explicitly deny these APIs, which will prevent any user from calling Amazon Bedrock models through SageMaker Canvas.

Here is an example IAM policy to achieve this:

{
    "Version": "2012-10-17",
    "Statement": (
        {
            "Effect": "Deny",
            "Action": (
                "bedrock:InvokeModel",
                "bedrock:InvokeModelWithResponseStream"
            ),
            "Resource": "*"
        }
    )
}

The policy uses the following parameters:

• "Effect": "Deny" Specifies that the following actions are denied:
• "Action": ("bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream") Specifies the Amazon Bedrock API that is denied.
• "Resource": "*" Indicates that the denial applies to all Amazon Bedrock models

Restricting access to specific Amazon Bedrock models

You can extend the preceding IAM policy to restrict access to specific Amazon Bedrock models by specifying the model ID in the resource section of the policy, allowing users to call only the allowed models.

The following is an example of an extended IAM policy:

{
    "Version": "2012-10-17",
    "Statement": (
        {
            "Effect": "Deny",
            "Action": (
                "bedrock:InvokeModel",
                "bedrock:InvokeModelWithResponseStream"
            ),
            "Resource": (
                "arn:aws:bedrock:<region-or-*>::foundation-model/<model-id-1>",
                "arn:aws:bedrock:<region-or-*>::foundation-model/<model-id-2>"
            )
        }
    )
}

In this policy, the resources array lists the specific Amazon Bedrock models that are to be denied, specifying the AWS Region, account, and model ID appropriate for your environment.

Manage SageMaker JumpStart access to SageMaker Canvas

To enable SageMaker Canvas to use LLM from SageMaker JumpStart, you must do the following:

1. Select LLM from the list of SageMaker Canvas or JumpStart model IDs (link below).
2. Create an endpoint configuration and deploy LLM to the real-time endpoint.
3. Call the endpoint to generate a prediction.

The following diagram illustrates this workflow:

For a list of available JumpStart model IDs, see JumpStart Available Models Table. At the time of writing, SageMaker Canvas supports the following model IDs:

• huggingface-textgeneration1-mpt-7b-*
• huggingface-llm-mistral-*
• meta-textgeneration-llama-2-*
• huggingface-llm-falcon-*
• huggingface-textgeneration-dolly-v2-*
• huggingface-text2text-flan-t5-*

To identify a suitable model from SageMaker JumpStart, SageMaker Canvas uses aws:RequestTag/sagemaker-sdk:jumpstart-model-id As part of the endpoint configuration. For more information about other techniques for restricting access to your SageMaker JumpStart models using IAM permissions, see Managing Access to Amazon SageMaker JumpStart Foundation Models Using Private Hubs.

Configure permissions for deploying endpoints through the UI

You can configure SageMaker Canvas to deploy SageMaker endpoints on the SageMaker domain settings page of the AWS Management Console SageMaker page. This option also enables deployment of real-time endpoints for traditional ML models such as time series forecasting and classification. To enable model deployment, follow these steps:

1. In the Amazon SageMaker console, navigate to your domain.
2. Above Domain Details On the page, Configure the app

3. So canvas Select by section edit.

4. Turn it on Allows direct rendering of Canvas models of MLOps configuration

Restrict access to all SageMaker JumpStart models

To restrict access to all SageMaker JumpStart models, configure a SageMaker role: CreateEndpointConfig and CreateEndpoint API for any SageMaker JumpStart model IDs. This prevents the creation of endpoints using these models. See the following code:

{
    "Version": "2012-10-17",
    "Statement": (
        {
            "Effect": "Deny",
            "Action": (
                "sagemaker:CreateEndpointConfig",
                "sagemaker:CreateEndpoint"
            ),
            "Resource": "*",
"Condition": {
                "Null": {
                    "aws:RequestTag/sagemaker-sdk:jumpstart-model-id":”*”
		    }
		}
        }
    )
}

This policy uses the following parameters:

• "Effect": "Deny" Specifies that the following actions are denied:
• "Action": ("sagemaker:CreateEndpointConfig", "sagemaker:CreateEndpoint") Specifies the SageMaker API that will be denied.
• of "Null" Condition operators in AWS IAM policies are used to check if a key exists or not. They only check for the presence or absence of a key, not its value.
• "aws:RequestTag/sagemaker-sdk:jumpstart-model-id":”*” Indicates that the rejection applies to all SageMaker JumpStart models.

Restrict access and deployment to specific SageMaker JumpStart models

Similar to Amazon Bedrock models, you can restrict access to specific SageMaker JumpStart models by specifying the model ID in an IAM policy. To achieve this, an administrator must prevent users from creating endpoints with models that they are not authorized to use. For example, to deny access to the Hugging Face FLAN T5 and MPT models, use the following code:

{
    "Version": "2012-10-17",
    "Statement": (
        {
            "Effect": "Deny",
            "Action": (
                "sagemaker:CreateEndpointConfig",
                "sagemaker:CreateEndpoint"
            ),
            "Resource": "*",
            "Condition": {
                "StringLike": {
                    "aws:RequestTag/sagemaker-sdk:jumpstart-model-id": (
                        "huggingface-textgeneration1-mpt-7b-*",
                        "huggingface-text2text-flan-t5-*"
                    )
                }
            }
        }
    )
}

In this policy, "StringLike" The condition allows for pattern matching, so that the policy can apply to multiple model IDs with similar prefixes.

cleaning

To avoid incurring future workspace instance charges, log out of SageMaker Canvas when you are done using your application. Optionally, you can configure SageMaker Canvas to automatically shut down when idle.

Conclusion

This post described how SageMaker Canvas invokes LLM with Amazon Bedrock and SageMaker JumpStart, and how enterprises can manage access to these models (whether to limit access to specific models or to limit access to any models in either service). The IAM policies presented in this post can be combined into the same IAM roles for complete control.

By following these guidelines, companies can ensure that the use of generative AI models is safe and compliant with organizational policies. This approach not only protects sensitive data, but also empowers business analysts and data scientists to harness the full potential of AI within a controlled environment.

Now that your environment is configured according to your enterprise standards, we recommend you read the following posts to discover what you can do with generative AI in SageMaker Canvas:

About the Author

Davide Galittelli Sr. Specialist Solutions Architect GenAI/ML. Italian, based in Brussels, working closely with clients around the world on Generative AI workloads and Low-Code No-Code ML technologies. A developer since childhood, he started coding at the age of 7. After graduating from university he started learning AI/ML and has been obsessed with it ever since.

Lijiang Kniir Lijan is a Senior Technical Account Manager at AWS. He enjoys helping AWS enterprise customers build reliable, cost-effective systems with operational excellence. Lijan has over 25 years of experience developing solutions for finance and consulting firms.

Saptarshi Banerjee As a Senior Partner Solutions Architect at AWS, he works closely with AWS Partners to design and build mission-critical solutions. Specializing in Generative AI, AI/ML, Serverless architectures and cloud-based solutions, Saptarshi is committed to driving performance, innovation, scalability and cost-efficiency for AWS Partners within the cloud ecosystem.