In the era of generative AI, agents that simulate human behavior and behavior are emerging as powerful tools for enterprises to create production-ready applications. Agents can interact with users, perform tasks, and mimic human-like intelligence to provide decision-making capabilities. By combining agents with Amazon Titan’s Foundation Model (FM) from the Amazon Bedrock family, customers can develop multimodal, complex applications where agents can understand and generate natural language and images.
For example, in the fashion retail industry, assistants powered by agents and multimodal models can provide customers with personalized and immersive experiences. Assistants can communicate in natural language and understand customer preferences and intentions. Multimodal features can then be used to analyze clothing images and make recommendations based on customer input. Additionally, agents can generate visual aids such as clothing suggestions to improve the overall customer experience.
In this post, you will implement a Fashion Assistant agent using the Amazon Bedrock agent and the Amazon Titan family model. Fashion Assistant provides a personalized, multimodal conversational experience. Among other things, you can use Amazon Titan Image Generator’s ability to inpaint and outpaint images to generate fashion inspiration and edit your photos. The Amazon Titan multimodal embedding model allows you to search for styles on your database using both prompt text or user-provided reference images to find similar styles. Anthropic Claude 3 Sonnet is used by agents to coordinate their actions. For example, search for the current weather and receive recommendations for weather-appropriate clothing. A simple web UI through Streamlit provides users with the best experience for interacting with agents.
Fashion Assistant Agent integrates seamlessly into existing e-commerce platforms and mobile applications, providing a seamless and enjoyable experience for your customers. Customers can also upload their own images, describe their desired style, and provide reference images, and agents will generate personalized recommendations and visual inspiration.
The code used in this solution is available in a GitHub repository.
Solution overview
The Fashion Assistant agent leverages the power of Amazon Titan models and Amazon Bedrock agents to provide users with a comprehensive set of style-related features.
- Image-to-image or text-to-image search – This tool allows customers to find products similar to their preferred style in the catalog, improving the user experience. Embed each product image using the Titan Multimodal Embeddings model and store it in Amazon OpenSearch Serverless for future searches.
- Text to image generation – If the desired style is not in the database, this tool generates a unique customized image based on the user’s query, allowing the creation of personalized styles.
- Weather API connection – By retrieving weather information for a specific location mentioned in a user prompt, agents can suggest appropriate styles for the occasion and ensure customers are dressed for the weather. .
- Top coat – Users can request background changes by uploading images, allowing them to visualize their preferred style with different settings.
- repair – This tool allows you to change specific clothing items within an uploaded image, including changing the design or color, while keeping the background intact.
The following flowchart shows the decision-making process.
The corresponding architecture diagram is:
Prerequisites
To set up a Fashion Assistant agent, make sure you have the following:
- An active AWS account and AWS Identity and Access Management (IAM) role with Amazon Bedrock, AWS Lambda, and Amazon Simple Storage (Amazon S3) access
- Installing required Python libraries such as Streamlit
- Anthropic Claude 3 Sonnet, Amazon Titan Image Generator, and Amazon Titan Multimodal Embeddings models are now enabled in Amazon Bedrock. You can confirm that these are enabled. model access Amazon Bedrock console page. When these models are enabled, the access status is displayed as follows: access grantedas shown in the following screenshot.
Before you start building your infrastructure by running the notebooks provided in the GitHub repository, ensure that your AWS account has the following permissions:
- Create a managed IAM role and policy
- Create and invoke a Lambda function
- Create, read, and write to S3 buckets
- Access and manage Amazon Bedrock agents and models
If you want to enable image-to-image or text-to-image search features, you need additional permissions to your AWS account.
- Create security policies, access policies, collections, indexes, and index mappings with OpenSearch Serverless
- please call
BatchGetCollection
About OpenSearch Serverless
Set up a fashion assistant agent
To set up a Fashion Assistant agent, follow these steps:
- Clone a GitHub repository using the command
- Meet the prerequisites and grant sufficient privileges
- Follow the installation instructions in README.md.
- (Optional) If you use
image_lookup
Run a feature, code snippetopensearch_ingest.ipynb
To embed and save a sample image using Amazon Titan multimodal embedding - Run the Streamlit UI and interact with the agent using the following commands:
By following these steps, you can create a powerful and engaging Fashion Assistant agent that combines the features of the Amazon Titan model with the automation and decision-making capabilities of the Amazon Bedrock agent.
Test the fashion assistant
Once you have set up Fashion Assistant, you will be able to interact with it through the Streamlit UI. Follow these steps:
- Navigate to the Streamlit UI as shown in the following screenshot.
- Follow the desired action, such as image search, image generation, outpainting, or inpainting, by uploading an image or entering a text prompt that describes the desired style. The following screenshot shows an example prompt.
- Press Enter to send the prompt to the agent. You can view the agent’s Chain of Thought (CoT) process in the UI, as shown in the following screenshot.
- Once the response is ready, you can view the agent’s response in the UI, as shown in the following screenshot. Responses may include generated images, recommendations for similar styles, or images modified based on your request. You can download the generated images directly from the UI or view them in your S3 bucket.
cleaning
Be sure to remove the resources used by this solution to avoid unnecessary costs. To do this, run the following command:
conclusion
Fashion Assistant Agents, powered by Amazon Titan models and Amazon Bedrock Agents, is an example of how retailers can create innovative applications that improve the customer experience and drive business growth. With this solution, retailers can gain a competitive edge by providing their customers with personalized style recommendations, visual inspiration, and interactive fashion advice.
We encourage you to check out the samples available in the aws-samples GitHub repository to explore the possibilities of building more agents like this fashion assistant.
About the author
Akarsha Sehwag He is a data scientist and ML engineer at AWS Professional Services with over 5 years of experience building ML-based solutions. We leverage our expertise in computer vision and deep learning to help customers efficiently harness the power of ML in the AWS Cloud. With the emergence of Generative AI, she worked with many customers to identify great use cases and build them into production-ready solutions.
Yangyang Zhang She is a Senior Generative AI Data Scientist at Amazon Web Services and a Generative AI Specialist working on cutting-edge AI/ML technologies to help customers leverage GenAI to achieve their desired outcomes. Yanyang graduated from Texas A&M University and received his Ph.D. Degree in Electrical Engineering. Outside of work, I love traveling, working out, and exploring new things.
Antonia Wiebeler She is a data scientist in the AWS Generative AI Innovation Center and enjoys building proofs of concepts for customers. Her passion is exploring how generative AI can solve real-world problems and create value for customers. When she’s not coding, she enjoys running and competing in triathlons.
alex newton is a data scientist at the AWS Generative AI Innovation Center, helping customers solve complex problems using generative AI and machine learning. He enjoys applying cutting-edge ML solutions to solve real-world challenges. In his free time, Alex can be found playing in a band or watching live music.
Chris Pecora Generative AI Data Scientist at Amazon Web Services. He is passionate about building innovative products and solutions, while also focusing on customer-focused science. When I’m not experimenting or keeping up with the latest developments in generative AI, I love spending time with kids.
Mayra Ladeira Tanke I am a Senior Generative AI Data Scientist at AWS. With a background in machine learning, she has over 10 years of experience designing and building AI applications with customers across a variety of industries. As a technology leader, she helps customers accelerate the achievement of business value through Amazon Bedrock’s generative AI solutions. In her free time, Myra enjoys traveling, playing with her cats, and spending time with her family in warm places.