This post was co-authored with NVIDIA’s Abhishek sawarkar, Eliuth Triana, Jiahong Liu, and Kshitiz Gupta.
At AWS re:Invent 2024, we’re excited to introduce Amazon Bedrock Marketplace. This is an innovative new feature within Amazon Bedrock that serves as a central hub for discovering, testing, and implementing foundational models (FM). This gives developers and organizations access to an extensive catalog of over 100 popular, emerging, and specialized FMs, complementing Amazon Bedrock’s existing selection of industry-leading models. Bedrock Marketplace allows you to subscribe to and deploy models through managed endpoints while maintaining the simplicity of the Amazon Bedrock integration API.
The NVIDIA Nemotron family, available as NVIDIA NIM microservices, offers a state-of-the-art language model suite available through Amazon Bedrock Marketplace and represents a significant milestone in AI model accessibility and deployment.
In this post, we will discuss the benefits and features of the Bedrock Marketplace and Nemotron models, as well as how to get started.
About the Amazon Bedrock Marketplace
Bedrock Marketplace plays a pivotal role in democratizing access to advanced AI capabilities through several key benefits:
- Comprehensive model selection – Bedrock Marketplace offers a wide range of models, from proprietary to commonly available options, allowing organizations to find the best fit for their specific use case.
- A unified, secure experience – By providing a single point of access for all models through the Amazon Bedrock API, Bedrock Marketplace greatly simplifies the integration process. Organizations can safely use these models with Amazon Bedrock’s robust toolkit, including Amazon Bedrock Agents, Amazon Bedrock Knowledge Bases, Amazon Bedrock Guardrails, and Amazon Bedrock Flows, for models that are compatible with the Amazon Bedrock Converse API. can be used.
- Scalable infrastructure – Bedrock Marketplace offers configurable scalability through managed endpoints, allowing organizations to choose the number of instances they need, select the appropriate instance type, and create a custom platform that dynamically adjusts to workload demands. Define autoscaling policies to help you optimize costs while maintaining performance.
About the NVIDIA Nemotron model family
At the forefront of the NVIDIA Nemotron model family is Nemotron-4, which NVIDIA says is a powerful multilingual large-scale language model (LLM) trained on a staggering 8 trillion text tokens. and is specifically optimized for English, multilingual, and coding tasks. The main features are:
- Generating synthetic data – Ability to create high-quality, domain-specific training data at scale
- Multilingual support – Trained on extensive text corpora and supports multiple languages and tasks
- high performance inference – Optimized for efficient deployment on GPU-accelerated infrastructure
- Various model sizes – Including variations like Nemotron-4 15B with 15 billion parameters
- open license – Offers a unique permissive open model license, giving companies a scalable way to generate and own synthetic data that helps build powerful LLMs
The Nemotron model offers transformative potential for AI developers by addressing key challenges in AI development.
- Data augmentation – Solve data scarcity problems by generating synthetic, high-quality training datasets
- cost effectiveness – Reduce the cost of manual data annotation and time-consuming data collection processes.
- Enhanced model training – Improve AI model performance with high-quality synthetic data generation
- Flexible integration – Supports seamless integration with existing AWS services and workflows, enabling developers to build sophisticated AI solutions faster
These features make the Nemotron model particularly suitable for organizations looking to accelerate their AI initiatives while maintaining high standards of performance and security.
Get started with Bedrock Marketplace and Nemotron
To start using Amazon Bedrock Marketplace, open the Amazon Bedrock console. From there, you can explore the Bedrock Marketplace interface, which offers a comprehensive catalog of FMs from various providers. You can browse the available options to find different AI capabilities and specializations. This search will help you find NVIDIA’s model products, including Nemotron-4.
The following sections describe these steps.
Open Amazon Bedrock Marketplace
Navigating to Amazon Bedrock Marketplace is easy.
- In the Amazon Bedrock console, model catalog in the navigation pane.
- under filterselect Bedrock Marketplace.
Once you enter Bedrock Marketplace, you will see a well-organized interface with various categories and filters to help you find the right model for your needs. Browse by provider and modality.
- Use the search feature to quickly find a specific provider and explore models cataloged on the Bedrock Marketplace.
Deploy the NVIDIA Nemotron model
When you find NVIDIA model products on Bedrock Marketplace, you can narrow it down to Nemotron models. To subscribe and deploy Nemotron-4, follow these steps:
- filter condition Nemotron under provider Or search by model name.
- Choose from available models.
Nemotron-4 15B.
The model details page provides detailed specifications, features, and pricing. The Nemotron-4 model offers superior multilingual and coding capabilities.
- choose View subscription options To subscribe to models.
- Review and select available options Subscribe.
- choose expand Follow the prompts to configure deployment options such as instance type and scaling policy.
This process is user-friendly and allows you to quickly integrate these powerful AI capabilities into your projects using the Amazon Bedrock API.
conclusion
The launch of NVIDIA Nemotron models on Amazon Bedrock Marketplace marks a significant milestone in making advanced AI capabilities more accessible to developers and organizations. Featuring a 15 billion parameter architecture trained on 8 trillion text tokens, Nemotron-4 15B brings powerful multilingual and coding capabilities to Amazon Bedrock.
Through Bedrock Marketplace, organizations can use Nemotron’s advanced capabilities while benefiting from AWS’s scalable infrastructure and NVIDIA’s robust technology. We encourage you to start exploring the capabilities of NVIDIA Nemotron models through the Amazon Bedrock Marketplace today and experience first-hand how this powerful language model can transform your AI applications.
About the author
james park I’m a Solutions Architect at Amazon Web Services. He works with Amazon.com to design, build, and deploy technology solutions on AWS, and has a particular interest in AI and machine learning. In my free time, I enjoy exploring new cultures, new experiences, and keeping up with the latest technology trends. You can find him on LinkedIn.
Saurabh Trikhande Senior Product Manager for Amazon Bedrock and SageMaker Inference. He is passionate about working with customers and partners, motivated by the goal of democratizing AI. He focuses on key challenges related to deploying complex AI applications, inference with multi-tenant models, optimizing costs, and making the deployment of generative AI models more accessible. In my free time, I enjoy hiking, learning about innovative technology, following TechCrunch, and spending time with my family.
Melanie LeeWith a Ph.D., she is a Senior Generative AI Specialist Solutions Architect at AWS based in Sydney, Australia, where she focuses on collaborating with customers to build solutions that leverage cutting-edge AI and machine learning tools. I’m leaving it there. She has been actively involved in multiple generative AI initiatives across APJ, leveraging the power of large-scale language models (LLM). Prior to joining AWS, Dr. Lee held data science roles in the financial and retail industries.
mark karp I’m an ML Architect on the Amazon SageMaker Service team. He focuses on helping customers design, deploy, and manage ML workloads at scale. In my free time, I enjoy traveling and exploring new places.
Abhishek Sawarkar He is a product manager on the NVIDIA AI Enterprise team, working on integrating NVIDIA AI software into the cloud MLOps platform. He focuses on integrating the NVIDIA AI end-to-end stack within cloud platforms and improving the user experience with accelerated computing.
Eleus Triana Developer Relations Manager at NVIDIA and Amazon’s AI MLOps, DevOps, Scientist, and AWS Technology Expert masters the NVIDIA compute stack for data curation, GPU training, model inference, and production deployments on AWS GPU instances. Generative AI Foundation helps you accelerate and optimize your models. . Additionally, Eliuth is a passionate mountain biker, skier, tennis and poker player.
Liu Jiahong I’m a Solutions Architect on NVIDIA’s Cloud Service Provider team. He helps clients deploy machine learning and AI solutions that leverage NVIDIA accelerated computing to address training and inference challenges. In my free time, I enjoy origami, DIY projects, and basketball.
kshitis gupta I’m a Solutions Architect at NVIDIA. He enjoys educating cloud customers about GPU AI technology from NVIDIA and helping them accelerate machine learning and deep learning applications. Outside of work, I enjoy running, hiking, and watching wildlife.
