Integrate Amazon Bedrock Knowledge Base with Microsoft SharePoint as a Data Source

Amazon Bedrock Knowledge Base provides contextual information from your company’s private data sources to Amazon Bedrock’s Foundation Models (FMs) and agents for Search Augmentation Generation (RAG), providing more relevant, accurate, and customized responses. Amazon Bedrock Knowledge Base provides a fully managed RAG experience.

The data sources you can connect to your knowledge base are constantly expanding. In this post, we’ll show you how to use one of the data source connectors: Microsoft SharePoint. Microsoft SharePoint is an integrated content management and collaboration tool that many organizations use to store, organize, and share their internal data. For a complete list of supported data source connectors, see Data Source Connectors.

Solution overview

Below are some of the relevant features of SharePoint data sources in the Amazon Bedrock knowledge base:

• It provides access to information stored in SharePoint. The RAG architecture queries and retrieves relevant information from SharePoint sources and provides contextual responses based on user input.
• It provides the ability to extract structured data, metadata, and other information from documents ingested from SharePoint and provide relevant search results based on user queries.
• It provides the ability to continuously synchronize incremental updates to SharePoint content.
• Provides the source attribute of the response generated by the FM.

The following sections provide steps to create a knowledge base, configure the data source, and test the solution.

Prerequisites

The following prerequisites are required to implement an Amazon Bedrock Knowledge Base using SharePoint as a connector:

Creating a Knowledge Base and Connecting to a Data Source

To set up a knowledge base in Amazon Bedrock and connect it to a SharePoint data source, follow these steps:

1. On the Amazon Bedrock console, Knowledge Base In the navigation pane.
2. choose Create a knowledge base.

3. So More on the knowledge base In the section, change the default name if necessary and enter a description for your knowledge base.
4. So IAM permissions section, select the IAM role that grants Amazon Bedrock permissions to access other AWS services. You can have Amazon Bedrock create a service role for you, or you can select a custom role that you’ve created.
5. So Select Data Source Section, Selection Sharepoint.
6. Optionally, add tags to your knowledge base. For more information, see Tags Resource.
7. choose Next.

8. So Name and description In the section, change the default data source name if necessary and enter a description for the data source.
9. So sauce In the section, provide the following information:
   1. for Site URLenter the site URL that you want to use to crawl and index content for the RAG.
   2. for domainenter the domain name associated with your data source. For example, if your site URL is https://deloittedasits.sharepoint.com/xyz.aspxThe domain value is deloittedasits.
   3. under Advanced SettingsLeave the default selection.

When Amazon Bedrock converts your data to an embedding, it encrypts the data by default with AWS owned and managed keys. To use your own AWS Key Management Service (AWS KMS) keys, Customize encryption settings (details) Select a key. For more information, see Encryption of temporary data storage during data ingestion.

You can also choose from the following options for data deletion policies for your data sources:

• erase – When you delete a Knowledge Base or Data Source resource, all the underlying data belonging to the Data Source will be deleted from the Vector Store. Note that the Vector Store itself is not deleted, only the underlying data. If your AWS account is deleted, this flag will be ignored.
• Retention – When you delete a knowledge base or data source resource, all of the underlying data in the vector store is preserved.

For more information about managing your knowledge base, see Managing Data Sources.

10. So certification In the section, the supported authentication methods are set to OAuth 2.0.
    1. for Tenant IDand enter your Tenant ID. To get your Tenant ID, see the Register a new application in the Microsoft Azure Portal section of this post.
    2. for AWS Secrets Manager Enter the AWS Secrets Manager to get the secret. To get the secret, see the Create a Secrets Manager secret for the SharePoint data source section in this article.

The SharePoint data source requires credentials to connect to the SharePoint Online site using the Microsoft Graph API. This is achieved by creating a new Secrets Manager secret. These credentials are not used in the access logs for the SharePoint Online site.

11. So Metadata Settings In the sections, select the content types you want to include or exclude, as appropriate.

12. So Content chunking and parsing Section, Selection Defaults.

13. choose Next.
14. So Embedding Model Section, Selection Titan Embeddings G1 – Text Or another embedding model if desired.
15. So Vector Database Section, Selection Quickly create a new vector store Create a vector store for embedding.
16. choose Next.

17. Above Review and Create Review your selections on the page and click Create.

Your knowledge base should now be created.

You have now created a knowledge base with SharePoint as the data source, but you need to synchronize the data source to crawl the site URLs and index the relevant content.

18. To start this process, select a data source on the knowledge base details page, Synchronization.

Register a new application in the Microsoft Azure portal

In this section, you will register a new application in the Microsoft Azure portal. You will get the Tenant ID during this step and use it when configuring the Amazon Bedrock Knowledge Base data source. Follow these steps:

1. Open the Azure portal and log in with your Microsoft account. If you don’t have an account, create one or contact your organization’s administration team.
2. choose Sign up.
3. Please provide the following information:
   1. for namespecify a name for your application. We’ll call this application TargetApp. Amazon Bedrock Knowledge Bases uses TargetApp to connect to your SharePoint site and crawl and index your data.
   2. for Who can use this application or access this API? choose Only accounts in this organizational directory (<テナント名> only – single tenant).
   3. choose register.
   4. Note down the Application (client) ID and the Directory (tenant) ID. overview You will need it later if you need it TargetApp-ClientId and TenantId.
4. choose API Permissions In the navigation pane.
5. Set the permissions as follows:
   1. choose Adding permissions.
   2. choose Microsoft Graph.
   3. choose Delegated authority.
   4. choose Read the full story of user section.
   5. choose Read the full story of Group Members section.
   6. choose Full Control. All of site section.
   7. choose Adding permissionsThis permission allows the app to read data about the signed-in user in your organization’s directory.
   8. In the options menu (three dots), Remove Permissions.
   9. Delete the original Read – Delegation permission.
   10. choose Grant admin consent Default directory.

6. choose Certificates and Secrets In the navigation pane.
   1. choose New client secret.
   2. for explanationand enter a description such as Describe your client secret.
   3. Please select a value ExpiredIn a production environment, you should manually rotate secrets before they expire.
   4. choose addition.
   5. Make a note of the new secret value, you will need it later when asked for the client secret (TargetApp-ClientSecret).
7. Select options Owner Add additional owners for the application. Owners will be able to manage permissions for the Azure AD app (TargetApp).

Create a Secrets Manager secret for the SharePoint data source

To create a Secrets Manager secret to connect to a SharePoint Online site that is listed as a site URL in your data source, follow these steps:

1. In the Secrets Manager console, Save your new secret.
2. for Secret Type,choice Other Types of Secrets.
3. for Key/Value PairsEnter the following:
   1. Username
   2. password
   3. Client ID
   4. Client Secret
4. for Encryption Keychoose aws/secretsmanager.
5. choose Next.
6. So Secret name and description In the section, enter a name for your secret and an optional description.
7. Add related tags tag
8. Leave Resource Permissions and Replication Secret As default.
9. choose Next.
10. So Set rotation In the sections, leave the defaults or change them according to your organization’s policies.
11. choose Next.
12. Review the options you selected and shop.
13. On the Secret details page, note the Secret ARN value to use as the secret while creating your Amazon Bedrock knowledge base.

Test the solution

To test the knowledge base you created, follow these steps:

1. On the Amazon Bedrock console, Knowledge Base In the navigation pane.
2. Select the knowledge base you created and test.

3. Select the appropriate model for testing, Apply.

4. Enter a question about the content that’s stored on your SharePoint site.

cleaning

If you created a new knowledge base to experiment with this article and you don’t plan to use it in the future, delete the knowledge base to avoid accumulating costs in your AWS account. For instructions, see Managing Knowledge Bases.

Conclusion

In this article, we have shown you how to set up an Amazon Bedrock knowledge base with SharePoint Online as a data source. By connecting SharePoint Online as a data source, your employees can use natural language to interact with your organization’s knowledge and data stored in SharePoint, making it easier for them to find relevant information, extract key takeaways, and derive valuable insights. This can significantly improve productivity, decision-making, and knowledge sharing within your organization.

Try this feature today in the Amazon Bedrock console, and to learn more, see the Amazon Bedrock knowledge base.

About the Author

Surendar Gajaveli He is a Senior Solutions Architect based in Nashville, Tennessee. He is a passionate technology enthusiast who enjoys working with clients to build innovative solutions.

Abi Patrolla is a Senior Solutions Architect based in the New York City area helping clients with their cloud transformation, AI/ML and data initiatives. As a strategic and technical leader, he advises executives and engineers on cloud strategies that drive innovation and positive impact.