Azure Compute Fleet – Simply Explained

Hello everyone and welcome to another episode of Microsoft Knowledge Nuggets here on M365. FM, I'm Mirko Peters. Imagine you need one virtual machine for a small app. You pick a region, choose a VM size, see the price and create it. Simple, right? Now imagine that same app suddenly needs 2,000 VMs overnight. You might be running simulations, processing a huge data set or testing software with thousands of short-lived workers. Suddenly that one easy VM choice turns into a real headache. The size you picked might not have enough capacity in your region. A different size could work, but the price shifts. Spot VMs cost way less, but Azure can take them back at any time. You could build multiple virtual machine scale sets and write custom scripts to hunt for capacity, but that quickly becomes a management nightmare. Here's the question, Azure Compute Fleet is built to answer. Why should one exact VM size keep a large deployment from starting at all? By the end of this episode, you'll know what Compute Fleet actually does, what it doesn't do, and where it fits alongside virtual machine scale sets. Compute Fleet in plain English. Azure Compute Fleet is a managed service designed to get you a large pool of VM compute capacity. Notice I didn't say it creates one specific machine. You tell Azure what kind of capacity you can use, how much you need, where you're willing to run it, and whether you want standard on-demand VMs, lower cost spot VMs, or both. Then Azure scans for capacity across those acceptable options. Think of it like a modern office building. You don't walk in demanding desk 27 on floor 8, in one exact room facing one exact window. Instead, you tell the building manager, "I need 500 desks for my team." These floors work, these room layouts are fine. Here's our budget. The manager looks at what's free and gets everyone seated. Compute Fleet works the same way. Your request might say, "I need a thousand workers, they need this much CPU and memory. These VM sizes are acceptable. These availability zones are acceptable. Use on-demand for part of the workload and spot where it makes sense." That wider net makes it much easier to find available capacity. A VM SKU is just a particular size and type a virtual machine. It describes things like CPU, memory, local storage, and processor family. If you ask for only one SKU, Azure can only search for that one exact shape. If that capacity is busy, your deployment waits or fails. With Compute Fleet, you list multiple acceptable VM sizes. Azure can then place some workers on one size and others on another as long as your workload runs on both. For big workloads, that flexibility is the whole point. Compute Fleet can handle up to 10,000 VMs in a single deployment. You no longer need to create thousands of machines one by one or build separate scripts for every acceptable VM size. Instead, you create one fleet request. Behind the scenes, Azure checks the capacity choices you allowed, finds a workable mix, and creates the VM groups that make up the fleet. You still have to provide the VM image, the network setup, and the software each worker should run. Fleet doesn't understand your business job on its own. It doesn't know whether a worker is calculating financial risk, rendering a video frame, testing software, or processing a genome sample. It also doesn't root web traffic, replace a load balancer, or fix code inside a VM. Fleet's role is narrower. It helps you acquire and manage a large pool of compute capacity when more than one path can get you there. That's the shift. Instead of handing Azure a rigid shopping list that says, "I need 1,000 of this exact machine." You give it a practical request that says, "I need this much compute, and here are the choices that work." The next part is where that request becomes useful, because you need to decide how much freedom you're willing to give Azure. The three choices that give Azure room to work. The first choice is the list of VM sizes you'll accept. Most people start with a name like D8's, F16, or some exact SKU that makes sense when you're running one server and you know exactly what it needs. But at Fleet scale, the better question isn't "which processor name do I want?" It's, "What does my work actually need?" Imagine a risk model that runs thousands of separate calculations. Each worker needs enough, CPU and memory to process one slice of the data. It might run just as well on several compute-focused VM families, even if the underlying processor is different. The job cares about finishing calculations correctly. It doesn't care whether every worker has the same label on the box. So instead of allowing one VM size, you can allow several sizes that meet the same practical need that gives as you have more capacity options to search through, while your workers still get the resources they need. There is a limit to this flexibility though. Your software has to run correctly on every VM type you allow. If one family needs a special driver, a different processor design or a particular local disk setup, don't add it just because it looks cheaper. A wider list only helps when every option you include is a real option. The second choice is what you want Azure to favor when it finds those options. Sometimes finishing the work matters more than finding the lowest possible price. Maybe a team meets to complete an overnight calculation before the market opens. In that case, you'd lean toward a capacity-focused choice. Your telling Azure find available compute that meets my rules. The job needs to start. This doesn't mean Azure creates capacity from nowhere. It means Azure can search through the sizes, zones and purchase choices you allowed, then favor the options with the best chance of getting your workers running. Other jobs can take a different path. Maybe you're creating test environments, running a large simulation that can retry later, or processing a backlog of files with no hard deadline. Cost may matter more than speed in those cases. A price-focused choice tells Azure to favor lower cost eligible capacity. That can reduce your compute bill, but it can also mean waiting longer for enough workers. The least expensive choice isn't always available when you need it, and it may not give you the full amount of capacity immediately. You've probably seen this outside Azure. A cheap flight is perfect if your schedule is flexible. If you need to be somewhere for an important meeting tomorrow morning, the lowest fare probably isn't the decision that helps you. Compute fleet lets you make the same trade-off with compute. Then there's the middle ground. Many teams don't want to pay the highest price for every worker, but they also can't accept a fleet that never reaches a useful size. A balanced approach gives Azure a room to consider both price and available capacity. You set the boundaries and Azure works inside them. That's a much more honest design than pretending every workload has one simple goal. Some jobs need a deadline, some jobs need a budget, most need both. The third choice is how you describe the machines themselves. You can list exact SKUs, which works well when you've already tested a small set of sizes, but Azure also has an attribute-based approach, currently available in Preview, where you describe the shape of compute you need. For example, you might describe a worker that needs a certain amount of CPU, memory, local storage, or an accelerator like a GPU. Azure can then find VM sizes that match those requirements. Think about the difference. An SKU list says, use these named models. An attribute request says, "Find machines with these physical needs." The second approach can reduce the work of updating your fleet definition when newer VM generations become available, but you still need to test your image and software against the kinds of machines as you may choose. Don't confuse flexibility with handing over every decision. You define the safe boundaries and Azure selects within them. So those are the three choices, which VM sizes or shapes will work, whether capacity or price comes first, and how broadly you describe the compute you need. Lower cost capacity can make a fleet far cheaper, but it comes with one rule that every team needs to plan for. Sometimes Azure can take a worker away. Spot and on demand, cheap seats and reserved seats. So let's talk about the two ways you can pay for the VMs in a fleet. On demand VMs are the stable option. You ask Azure for a VM, you pay the normal rate while it runs, and Azure doesn't reclaim it just because another customer needs that capacity. For work that must keep moving, on demand capacity gives you a dependable base. Think of it like booking confirmed seats for the people who must be at the event. Your job controller might run there, your queue service might depend on it, or maybe you need a minimum number of workers running all night because missing the morning deadline isn't an option. Spot VMs work differently. Azure sells unused compute capacity at a lower price, sometimes much lower than the normal on demand rate. That makes Spot useful when you have a lot of work to process and can handle interruptions. But the lower price comes with a condition. Azure can reclaim a Spot VM when it needs that capacity back. The VM can disappear while it's working. That doesn't mean Spot is unreliable in a bad way. It means you need to use it honestly. A Spot VM is not the place for the only copy of an important database, a single user facing server, or a job that loses hours of progress when one worker stops. You plan for the worker to leave. This is why a mixed fleet often makes sense. You keep enough on demand VMs for the work that must continue. Then you add Spot VMs for the extra processing power. When Spot capacity is available, the fleet can process much more work for less money. When some Spot workers disappear, the baseline on demand workers keep going. That setup gives you a practical balance. The stable workers protect your minimum throughput while the Spot workers help you finish sooner or process more work when Azure has spare capacity. Compute Fleet helps manage the capacity side of that mix. It can use the allocation choices you set, look for eligible Spot or on-demand options, and react when Spot VMs are evicted. Still, Fleet cannot protect work that exists only inside of VM. Your workload design has to do that. Imagine a video rendering job. One worker renders frame one through frame one hundred. Another handles the next group. If a Spot worker disappears halfway through a frame, the system should mark that frame as unfinished and send it back to the queue. The next worker starts it again. No drama, no loss project. That same pattern works for simulations, data extraction and transformation jobs, software builds, automated tests, development environments and genomics processing where many samples can run separately. These jobs work well because they can split into smaller pieces. One worker can process one file, one worker can calculate one scenario, one worker can test one build if the worker goes away, another worker can pick up that small piece later. But a single database server is a very different job. If the database stores its only copy of the data on a Spot VM, an eviction can interrupt the service and put recovery at risk. The same applies to stateful software that cannot restart cleanly or a customer facing application that depends on one VM staying online. Spot doesn't turn those designs into safe designs. For a Spot-friendly workload, keep progress somewhere durable outside the worker VM. That might mean a storage account, a database or another service that records what has finished and what still needs work. Use a durable queue so workers pull tasks one at a time. Save checkpoints during long running jobs. Make each task safe to retry, so running it again doesn't create duplicate results or corrupt data. That way, when Azure reclaims a Spot VM, you lose only the small piece it was processing, not the entire job. This can save a lot of money but cost alone doesn't tell you whether compute fleet belongs in your design. The next question is whether you need to fleet at all or whether a virtual machine scale set fits the job better. Compute fleet versus virtual machine scale sets. You might be wondering, doesn't Azure already have a service called virtual machine scale sets? It does and the two are related because they both help you run groups of Azure VMs instead of managing each one by hand. But here's the thing, they solve different problems starting from different questions. A virtual machine scale set or VMSS for short starts with your application. Let's say you have a web app, an API or an internal business service, you create a group of VMs with a similar setup, put them behind a load balancer and let the group grow or shrink as demand changes. Traffic rises, so the scale set adds more application servers, traffic falls and it removes servers, so you aren't paying for machines you no longer need. That's a very normal pattern for a long running application tier. Your VMs usually run the same image, the same application and a closely related VM setup. Azure can use health checks to see whether the app is responding and it can add or remove instances based on CPU use, a schedule or another metric you choose. Think of a VM scale set like a store manager planning a regular team. Every morning the manager needs cashiers, shelf stockers and people at the service desk. They know the jobs, the location and the basic tools each person needs. When the store gets busy, they call in more people from the same team. That's what VM assess does well. It runs a known application pattern and adjusts the number of workers as demand changes. Compute fleets start somewhere else. Instead of asking out how many web servers does my app need right now? It asks how can I get a very large amount of acceptable compute capacity for this job? That distinction changes how you design the request. With a fleet the VM size can vary across the workers as long as each worker can perform the task. You might accept several machine families, use different purchase options and let Azure choose a workable mix based on your rules. Think of compute fleet as a staffing agency trying to fill thousands of shifts before a deadline. The agency doesn't care whether every worker wears the same shoe size, it cares whether each person can do the work, whether enough people arrive on time and what the full shift costs. That's a much better fit for a job that can split across many independent workers, a large data run, a simulation, a rendering project or a big set of automated tests. The difference isn't that one service is better than the other, they solve different problems. So choose virtual machine scale sets when you run a steady application tier that receives traffic. That could be a customer facing website, an API, a line of business app or a group of application servers behind Azure load balancer or application gateway. In those cases you usually care about application health and traffic handling. You need instances with a known setup, you need a way to add servers when users arrive and remove them when demand drops. VMSS fits that model. Choose compute fleet when the main challenge is getting a lot of compute for work that can spread out. The work might last for hours, days or only a short burst but it doesn't depend on one identical application server receiving user requests. Fleet works well when a worker can take a task, complete it, save the result and move on. That's common in high performance computing, batch processing, large simulations, data processing and large temporary worker pools. There's also a practical point that trips people up. Compute fleet does not replace your application design. It doesn't replace the queue that hands out work, the checkpoints that save progress or a load balancer for a web app. And it doesn't turn a single stateful server into a fault tolerant system. As you can help acquire the VMS but your software still needs to know what each VMS should do, where its data lives, and what should happen if a worker stops halfway through a task. That shared responsibility matters. As your manages the physical servers and the fleet capacity rules, you design the workload so it can survive the normal events of cloud computing, including a VM restart, a failed task or an interrupted worker. So a simple decision guide looks like this. If you're scaling an application because user traffic changes, start with virtual machine scale sets. If you're gathering a large amount of flexible compute because a job needs many workers, look at compute fleet. And if your architecture includes both needs, you can use both services. A VM scale set can run the customer facing application tier, while compute fleet supplies temporary workers for the heavy background job. Picture a team that needs thousands of workers overnight to finish a calculation before the next business date. That's where these choices stop being product names and start changing the outcome. A fleet example. Finishing a large job without betting on one VM size. Imagine a finance team that runs risk calculations every night. They need to process a huge amount of market data, test thousands of possible outcomes and produce results before people arrive the next morning. The team has one clear goal. Finish the calculations by morning without paying the normal on-demand rate for every single worker. Their first attempt looks simple. They choose one compute focused VM size, request a large number of workers, and start the job. Then as you cannot find enough of that exact size in the region. Nothing is wrong with the VM size or the code. There just isn't enough capacity for that one exact choice when the team needs it. If their whole design depends on that one SKU, the job starts late. A late start means fewer results by morning or a rushed and expensive move to another setup. So the team changes the request. Instead of demanding one VM size, they identify several CPU focused sizes that their risk software can use. Each option has enough memory for a task, enough CPU for a calculation, and works with the same VM image. They also allow more than one availability zone where their data and workload can run. That gives Azure more places to find workers. Next, the team decides how much capacity must stay available no matter what. They choose a group of on-demand VMs as their reliable base. Those workers give the job a minimum processing rate throughout the night. Even if lower cost capacity becomes scarce, the run continues. Above that base they allow spot VMs. Those spot workers do the extra work. When Azure has spare capacity, the team can process many more calculations without using on-demand VMs for every task. The request now says something like this. Give us enough stable workers to guarantee a minimum result, then add as many lower cost workers as you can from this approved list of VM sizes. That changes the situation. The team isn't betting the whole overnight run on one size, one zone, or one price type. But there's another part of the design that matters just as much as the fleet request. The risk calculations cannot live only inside the worker VMs. Before the fleet starts, a job system breaks the large calculation into small pieces. One task might calculate the outcome for one market scenario. Another might handle a different portfolio or time range. Each task enters a queue. A worker takes one task, processes it, writes the result to durable storage, and then asks for another task. Progress lives outside the worker. That means a worker can disappear without taking the whole job with it. Supposer spot VM receives an eviction notice halfway through a calculation. The VM stops. The unfinished task remains marked as incomplete in the queue. Another worker, perhaps a new spot VM or one of the on-demand workers, can take that task and run it again. The team loses one small piece of work, but they don't lose the whole overnight calculation. And boom, that's the real benefit of a fleet-friendly design. Azure handles finding and managing the compute capacity. The team's job system handles the work, retries, and saved results. Neither side can do the other side's job. Compute fleet cannot decide whether rerunning a calculation creates the wrong financial result. The team has to make tasks safe to retry and prevent duplicate processing. At the same time, the team doesn't need custom scripts that search through every approved VM size and keep trying to create workers one group at a time. They define the acceptable choices and let Azure work within those rules. By morning, the exact number of spot workers may have changed during the night. That's expected. The stable workers kept processing. Extra spot workers increased throughput when capacity existed, interrupted work returned to the queue. The final cost stayed under more control than an all-on-demand run. This isn't a promise that every task will finish at the lowest price. It's a way to avoid tying the whole job to one narrow capacity choice. The same pattern appears in life sciences. A research team may need to process a large set of genome samples. Each sample can move through the same analysis pipeline, but each sample is separate from the next. The team stores input data and results outside the worker VMs. A worker processes one sample or one stage of a sample, saves the output and moves on. If that worker stops, another worker resumes from the last save point. Compute fleet can supply the large, flexible worker pool, but the pipeline still needs to track what finished, what failed, and what must run again. Before you put a fleet into a real design though, there are a few decisions that prevent you from choosing it for the wrong kind of workload. When to use it and what to decide first. What exactly is a compute fleet workload? It's work that can spread across many independent workers. That's the starting point. Here's the simple question you need to answer. If one worker stops can another worker take over without you having to fix the job by hand? If the answer is yes, you probably have a good fleet workload, batch processing, test workers, analysis jobs, and large file processing often fit because the work splits into separate units that run in parallel. Start with the workload behavior, not the VM name. Think of it like a team of freelancers. If one drops out, another picks up the task without losing progress. Can the work run without keeping its state inside one VM? If it needs state, can it save progress somewhere durable and restart from that point? Can the same task run again safely if a worker never reports back? Those answers tell you far more than any pricing page ever will. Now decide the minimum capacity that must stay available. This is the part of the job you cannot afford to lose if lower cost capacity becomes unavailable or gets interrupted. Use on-demand VMs for that baseline then decide how much extra capacity you are willing to add through spot VMs. That number should come from your deadline, your job backlog, and your budget. Not from a guest that spot will always be there. After that define your VM choices carefully. You might approve several related VM families that your software already supports. Or if the preview feature fits your design, you can describe the CPU, memory, storage, or accelerator needs and let azure pick matching machines keep the list practical. A narrow list limits the places as you can search. An overly broad list can put your workload on machines that behave differently from the ones you tested. Every allowed choice needs to work with your image, your network setup, and your software. You also need a clear ceiling. Set the target capacity for the job then set a maximum cost exposure before launching. Large compute jobs can grow fast when a queue contains more work than expected. Or when a deadline pushes more work onto on-demand capacity. A budget is not just a fine and setting, it is a design boundary. Where your workload and region supported spread the fleet across availability zones. This gives azure more placement choices and reduces the chance that one local capacity issue affects every worker. Keep an eye on the job, not just the VM count. Monitor the capacity fleet has actually acquired. Watch the queue backlog, failed tasks, retry count, eviction events, completion rate, and cost. A fleet with many healthy VMs can still be failing the business goal if the queue is growing or results are not being saved. Here's the mistake most people make. They treat compute fleet as automatic application recovery. It does not do that. It manages capacity. Your job system still needs to recover the work. Once those pieces are in place, one flexible request can replace a pile of manual skew checks, separate spot plans and scripts that retry capacity one VM group at a time. So pick one workload that already runs as a batch job. Or one that you can safely retry. Write down five VM choices that can genuinely run it. Not five names you found in a list, but five tested options your software supports. Then next to that list write down four more things. The minimum on demand capacity you need. The extra spot capacity you can use. Where progress is saved. And exactly how an unfinished task returns for retry. Add a cost limit before you launch. That small design exercise turns compute fleet from a product name into a real plan. It changes the request from I need this exact VM to I need this work completed and these compute choices are safe for long running application tiers that serve user traffic continue with virtual machine scale sets next Azure can find the workers but your workload design decides what happens when one disappears.