Enhanced performance for Amazon Bedrock Custom Model Import

You can now achieve significant performance improvements when using Amazon Bedrock Custom Model Import, with reduced end-to-end latency, faster time-to-first-token, and improved throughput through advanced PyTorch compilation and CUDA graph optimizations. With Amazon Bedrock Custom Model Import you can to bring your own foundation models to Amazon Bedrock for deployment and inference at scale.

These performance enhancements typically come with model initialization overhead that could impact container cold-start times. Amazon Bedrock addresses this with compilation artifact caching. This innovation delivers performance improvements while maintaining existing cold-start performance metrics that customers expect from CMI.

When deploying models with these optimizations, customers will experience a one-time initialization delay during the first model startup, but each subsequent model instance will spin up without this overhead—balancing performance with fast startup times during scaling.

In this post, we introduce how to use the improvements in Amazon Bedrock Custom Model Import.

How the optimization works ...