Converting credit decisions using AI generated using Rich Data Co and AWS

This post is co-authored with RDC’s Gordon Campbell, Charles Guan, and Hendra Suryanto.

Rich Data Co (RDC)’s mission is to expand access to sustainable credit globally. Its software As-a-Service (SaaS) solution features deep customer insights and AI-driven decision-making capabilities for leading banks and lenders.

Using AI to make credit decisions can be challenging, and data science and portfolio teams need to integrate complex thematic information and work together productively. To solve this challenge, RDC used generator AI to enable teams to use the solution more effectively.

• Data Science Assistant – Designed for data science teams, this agent helps develop, build and deploy AI models within a regulated environment. By answering complex technical queries throughout the lifecycle of Machine Learning Operations (MLOPS) you can extract team efficiency from a comprehensive knowledge base that includes environmental documentation, AI and data science expertise, and Python code generation. It is intended to raise it.
• Portfolio Assistant – Designed for portfolio managers and analysts, this agent promotes natural language enquiries about loan portfolios. It provides important insights into performance, risk exposure, and credit policy adjustments, enabling informed commercial decisions without the need for detailed analytical skills. Assistants are proficient in high-level questions (such as identifying high-risk segments and potential growth opportunities) and one-time questions, allowing them to diversify their portfolios.

In this post, we will explain how RDCs can use generated AI on Amazon bedrock to build these assistants and accelerate their overall mission to democratize access to sustainable credit.

Solution Overview: Building a Multi-agent Generated AI Solution

We anticipated common user questions, starting with a carefully crafted set of over 200 prompts. Our first approach combined rapid engineering with traditional search and augmented generation (RAG). However, I ran into a challenge. In particular, accuracy fell below 90% for more complex questions.

To overcome the challenges, we adopted an agent approach and decomposed the problem into specialized use cases. This strategy allowed us to tailor the most appropriate basic model (FM) and tools for each task. Our multi-agent framework is coordinated using Langgraph.

1. Orchestrator – Orchestrators are responsible for routeing user questions to the appropriate agents. In this example, we start with a Data Science or Portfolio Agent. However, we are assuming more agents in the future. Orchestrators can also use user contexts such as user roles to determine routing to the appropriate agent.
2. agent – Agents are designed for professional tasks. It is equipped with the right FM for your tasks and the tools you need to perform actions and access knowledge. You can also handle multi-turn conversations and coordinate multiple calls to FM to reach the solution.
3. tool – Tools extend agent functionality beyond FM. Provides access to external data and APIs, or enable specific actions and calculations. To use the model’s context window efficiently, build a tool selector that retrieves only relevant tools based on agent state information. This helps to simplify debugging in case of errors, ultimately making the agent more effective and cost-effective.

This approach provides the right tools for the right job. Increases the ability to efficiently and accurately process complex queries while providing flexibility for future improvements and agents.

The following image is a high-level architecture diagram of the solution.

Data Science Agent: RAG and Code Generation

To increase productivity for the data science team, we focused on rapid understanding of advanced knowledge, including industry-specific models from curated knowledge bases. Here, RDC provides an integrated development environment (IDE) for Python coding, covering a variety of team roles. One role is a model validator that closely assesses whether the model matches the bank or lender’s policy. We designed an agent with two tools to support the evaluation process.

1. Content Retriever Tool – Amazon Bedrock Knowledge Bases enhances search for intelligent content through streamlined RAG implementations. The service automatically converts text documents to vector representations using Amazon Titan text embedding and stores them in Amazon OpenSearch ServerLess. Because knowledge is huge, we perform semantic chunking to ensure that knowledge is organized by topic and fits within the context window of FM. When users interact with agents, Amazon Bedrock Knowledge Base uses OpenSearch Serverless to provide fast in-memory semantic searches, allowing agents to obtain the most relevant chunks of knowledge for context responses related to users I’ll make it possible.
2. Code Generator Tool – With Code Generation, I chose Anthropic’s Claude model in Amazon Bedrock due to its inherent ability to understand and generate code. This tool is grounded to answer queries related to data science and can generate Python code for quick implementation. He is also skilled in troubleshooting coding errors.

Portfolio Agent: SQL and Self-correction from Text

We focused on two key areas to increase productivity for our credit portfolio team. High levels of commercial insights were prioritized for portfolio managers. Deep Dive Data Exploration has been enabled for analysts. This approach reinforced both roles with rapid understanding and practical insights that streamline the decision-making process across the team.

Our solution required a natural language understanding of the structured portfolio data stored in Amazon Aurora. This was based on a solution based on intertext models to efficiently bridge the gap between natural language and SQL.

To reduce errors beyond the capabilities of the model and tackle complex queries, I developed three tools using Anthropic’s Claude model on Amazon Bedrock for self-correction.

1. Check out the query tools – Validate and fix SQL queries and address common issues such as data type mismatches and incorrect function usage
2. Check out the Results Tool – Validate the results of the query, provide relevance, and prompt for search or user clarification if necessary
3. Try again from the user tool – Engage users for additional information if the queries that lead interactions based on database information and user input are too broad or have no details

These tools work with agent systems, allowing for accurate database interactions and improve query results through iterative improvements and user engagement.

To improve accuracy, we tested fine-tuning the model and trained the model with general queries and contexts (such as database schemas and their definitions). This approach reduces inference costs and improves response time compared to prompts on each call. I used Amazon Sagemaker Jumpstart to fine-tune Meta’s Llama model by providing expected prompts, intended answers, and related context. Amazon Sagemaker Jumpstart offers a cost-effective alternative to third-party models and provides a viable pathway for future applications. However, we were unable to deploy the fine-tuned model, especially for complex questions, as we experimentally observed that prompts in Anthropic’s Claude model provided better generalization. We also evaluate structured data searches on Amazon bedrock knowledge base to reduce operational overhead.

Conclusion and the next steps for RDC

To facilitate development, RDC collaborated with AWS Startups and AWS Generation AI Innovation Center. Through an iterative approach, RDC rapidly strengthened its generation AI capabilities, deploying the initial version into production in just three months. The solution successfully met the stringent security standards required in a regulated banking environment, providing both innovation and compliance.

“The integration of generation AI into our solutions marks a pivotal moment in our mission to revolutionize faith-ready decision-making. By using AI assistants for both data scientists and portfolio managers , not only improve efficiency, but also change the way financial institutions approach lending.”

– Gordon Campbell, RDC co-founder and chief customer officer

RDC envisions generator AI that plays a key role in increasing productivity in the banking and credit industry. Using this technology, RDCs can provide key insights to their customers, improve solution adoption, accelerate model lifecycle, and reduce customer support burden. Looking ahead, RDC plans to further improve and expand AI capabilities and explore new use cases and integrations as the industry evolves.

For more information about how to work with RDC and AWS, contact your AWS Account Manager or visit Rich Data Co to help bank customers around the world and understand how to use AI in credit decisions.

For more information about AWS Generating AI, see the following resources:

About the author

Daniel Willho He is a solution architect at AWS and focuses on fintech and SaaS startups. As a former startup CTO, he enjoys working with founders and engineering leaders to promote AWS growth and innovation. Outside of work, Daniel enjoys taking a walk with coffee, appreciating nature, and learning new ideas.

Xuefeng liu He leads the science team at the AWS Generated AI Innovation Centre in the Asia-Pacific region. His team is partnering with AWS customers on Generated AI projects with the goal of accelerating the adoption of Generated AI.

Iman Abbasnejad I am a computer scientist at the Generation AI Innovation Center at Amazon Web Services (AWS), working on generator AI and complex multi-agent systems.

Gordon Campbell He is RDC’s Chief Customer Officer and Co-Founder, and has used RDC’s leading AI decision-making platform for business and commercial lenders for over 30 years. With its proven track record in product strategy and development across three global software companies, Gordon commits to advances in customer success, advocacy, and financial inclusion through data and AI.

Charles Guan He is RDC’s Chief Technology Officer and Co-Founder. With over 20 years of experience in data analytics and enterprise applications, he has driven innovation in both the public and private sectors. At RDC, Charles leads research, development and product advancements. It is collaborating with the university to leverage advanced analytics and AI. He is dedicated to promoting financial inclusion and providing a positive impact for communities around the world.

Hendra Suryanto He is a leading data scientist at RDC and has over 20 years of experience in data science, big data and business intelligence. Before joining RDC, he served as lead data scientist at KPMG and advised clients globally. In RDC, Hendra designs end-to-end analytics solutions within an agile DevOps framework. He holds a PhD in Artificial Intelligence and has completed postdoctoral research in machine learning.