Introducing flexible reservations

Reservations ensure access to guaranteed capacity at fixed prices. Their downside is that they require a commitment to consuming (or at least paying for) that capacity for a fixed time, while your actual workload might take a different shape. This leads to difficult decisions, trading off the safety of securing capacity against the cost of potentially wasted resources.

For example, a company launching a new AI product involving customizing a model for each user will need to ensure it has access to GPUs at launch time to deliver the service. But new service usage is often unpredictable. The service could be a sustained hit or fail to get much traction. It could also take the form of high initial usage by tire-kickers, followed by a substantial drop, and then sustained growth as it develops its user base.

To allow companies to optimize their capacity planning in such situations, Mithril is introducing flexible reservations. We’re actually making all reservations flexible.

Here is the form it takes in practice. For a given reservation request (defined by an instance type, a number of instances, start/end dates, and a region), Mithril will generate reservation offers, available in the console or through the API.

A reservation offer is defined by three numbers:

• The Reserved Unit Price: Your maximum hourly rate—the most you’ll ever pay.

• The Pause Credit: The rate you earn credit for pausing instances. Use this to save money during dips while keeping the right to scale back up whenever needed.

• The Return Credit: A higher credit rate for capacity you release permanently. Use this if your work finishes early or you want to pivot to a different GPU type.

Here’s an example of a 17-day reservation for 8 instances which is fully used for 3 days, then 2 instances are paused for 2 days, then those 2 instances are returned, then 2 more instances are paused on day 9, and those are returned 2 days later. The diagram below describes over what time period and for how many instances this reservation would earn pause credit (in blue) and return credit (in green).

In the Mithril console (or via the API), we will generate two such reservation offers for any reservation request you make. One of them will be a “High Flexibility” offer with a higher Unit Price but also higher Pause Credit and higher Return Credit. This offer is most advantageous if you expect significant variability in your workload, and that you’ll take advantage of the flexibility mechanisms.

The other offer will be a “Low Flexibility” offer with the lowest available Unit Price but also with lower Pause and Return Credits. This offer is most advantageous if you expect to make use of most of the reserved capacity and don’t need as much flexibility.

Based on this logic (lower Unit Price coming with lower credits for Pause and Return) you might be looking for a “no flexibility” offer (with no credits) in order to secure the lowest possible Unit Price, for cases where you don’t expect to need any flexibility. That offer exists and it is… the “Low Flexibility” offer. Thanks to the economics of the Mithril spot market, we’re able to offer those “Low Flexibility” credit rates with no added premium to the Unit Price. In other words, those credit opportunities come as a built-in perk for all Mithril reservations. If you don’t need them, just don’t use them. But if you find yourself temporarily not needing some instances, you can just pause them and opportunistically collect some credits.

Here is the reservation screen, where you can decide between the "Low Flexibility" and the "High Flexibility" offer, on the right side. The offers are calculated based on the reservation parameters selected on the left side.

For long-term reservations, not currently available via the console, please contact sales@mithril.ai and we will present you with options that match your needs, with similar flexibility mechanisms. Also note that earned credits apply to future use of Mithril, that those credits expire after six months, and that Pause Credits only apply to instances which have been paused for at least one hour. Please see the documentation for all the details.

We hope you find these flexible reservations a great complement to the Mithril spot market, especially since it is possible to use instances from either order type as part of the same Kubernetes cluster.