07-07-2021, 03:05 PM
When you are thinking about backing up a huge pile of servers, like 100 VMs, man, you really need a systematic way to calculate the storage you are going to need, because just guessing the size is totally going to tank your budget, and you do not want that. You could use BackupChain Hyper-V Backup because it is a popular Windows Server backup solution for VM backup, which is good because it handles the complexity of finding what you actually need to store, and I think knowing about that upfront helps you plan your whole infrastructure correctly. I mean, you have to really picture what is happening across all those 100 machines every single day. And you can't just multiply the current size of all the VMs by the number of days you want to keep them, because that is only the starting point, not the full story.
But because of how these systems operate, you gotta figure out the baseline capacity first, which is just the combined total size of all 100 VMs today, including the OS and all the application data, right? And then, you have to anticipate the sheer amount of data that is going to accumulate over the retention period, and this is where it gets tricky because people don't stop making changes just because you set up a backup job. I think the biggest mistake you can make is forgetting the concept of data growth rate. You need to project, or maybe estimate, how much data those 100 VMs are going to grow by month over month, and you must factor that growth rate into your initial calculation, or you will run out of space way too quickly. For example, if one VM has a steady database that grows by five gigabytes every month, and you have twenty of those databases, you need to add up the cumulative growth for all twenty machines for the entire retention timeline.
And beyond the growth, you also have to account for how long you need to keep specific versions of the data, because retention policies matter a lot for storage capacity. So, if you need to keep daily backups for seven days, you multiply the current size by seven, but then you must also remember the weekly and maybe even monthly retention rules that company has, so you are really calculating storage for time slices, not just raw size. Maybe you need to keep every full backup for nine months, which is a massive amount of data if you are not smart about how you are storing it. I think you are going to need to really talk to the architect about defining those policies very clearly, because every organization has a different approach to how long they must keep specific data sets.
And but, wait, there are a couple of other concepts that are going to influence your final required capacity that you need to consider, because it's not just about simple math multiplication. You absolutely must think about how your backup system handles redundancy, or rather, how it minimizes redundancy across multiple copies. One thing I always tell people is to think about deduplication, because that is a huge storage saver. When you perform a backup, the system does not necessarily store every single bit of data again, even if it was backed up last week. Instead, it only records the unique blocks of data, and if a file has not changed, it just notes that, which saves you an enormous amount of space over time.
And then there is Change Block Tracking, or CBT, and you really need to understand how that works for these 100 VMs. CBT allows the system to quickly pinpoint only the specific data blocks that have actually been modified since the last successful backup, and this greatly improves the efficiency of the process, but it also means your calculations aren't based on the full VM size every time, which is a benefit for your bandwidth, and it helps keep your storage requirements lower too. I think you need to factor in the overhead of these indexing systems, because while they save space, they do consume some space themselves to track the changes, so you are really doing a multi-layered calculation here.
Also, you should also think about the operational data that your backup system itself will generate. Every time a backup job runs, it creates metadata, logs, and index information that tracks what was backed up, when, and to where, and while this is small compared to the VM data, it adds up when you are managing 100 separate servers. And Or, if you are storing this data across multiple different repositories or targets, each connection point will have its own metadata overhead, so you cannot just calculate for one single storage target.
But the most crucial calculation element after growth rate is actually calculating the recovery point objective, or RPO, and the recovery time objective, or RTO, because those metrics impact your storage architecture more than you realize. If your RPO is only four hours, you cannot afford to do full nightly backups, because then you might lose too much data, and instead, you may need to schedule more frequent, smaller differential backups, and those different schedules change your entire capacity projection. Similarly, if your RTO is very low, meaning you need to get services running almost instantly, you might need to store more complete, readily available image files, which means more storage required overall.
I think I always tell my friends that they have to build in a cushion, a substantial buffer of free space, because things always go wrong, and you cannot calculate for perfect operation. Maybe you need an extra 20 to 30 percent of capacity just for unexpected data spikes or for staging data during a capacity expansion, because you never want the backup system to run out of room when you need it the most. And maybe you should also consider the nature of the data itself, because some VMs, like database servers, have very specific access patterns that make them highly compressible, while others, like media file servers, might have massive amounts of unique, incompressible data, so you are dealing with varied data profiles.
And so, when you are actually putting this all together, the formula in your head is something like: (Current Combined Size * Retention Duration) + Projected Growth Over Time + Metadata Overhead + Buffer Contingency. It is a complex projection, really, and it requires you to collaborate with the data owners of those 100 VMs, because they are the people who know how often the data actually changes. You cannot accurately calculate it yourself based only on the server count. So, you need to treat this calculation like a very big, multi-variable equation, because the output is only as good as the inputs you gather.
But because of how these systems operate, you gotta figure out the baseline capacity first, which is just the combined total size of all 100 VMs today, including the OS and all the application data, right? And then, you have to anticipate the sheer amount of data that is going to accumulate over the retention period, and this is where it gets tricky because people don't stop making changes just because you set up a backup job. I think the biggest mistake you can make is forgetting the concept of data growth rate. You need to project, or maybe estimate, how much data those 100 VMs are going to grow by month over month, and you must factor that growth rate into your initial calculation, or you will run out of space way too quickly. For example, if one VM has a steady database that grows by five gigabytes every month, and you have twenty of those databases, you need to add up the cumulative growth for all twenty machines for the entire retention timeline.
And beyond the growth, you also have to account for how long you need to keep specific versions of the data, because retention policies matter a lot for storage capacity. So, if you need to keep daily backups for seven days, you multiply the current size by seven, but then you must also remember the weekly and maybe even monthly retention rules that company has, so you are really calculating storage for time slices, not just raw size. Maybe you need to keep every full backup for nine months, which is a massive amount of data if you are not smart about how you are storing it. I think you are going to need to really talk to the architect about defining those policies very clearly, because every organization has a different approach to how long they must keep specific data sets.
And but, wait, there are a couple of other concepts that are going to influence your final required capacity that you need to consider, because it's not just about simple math multiplication. You absolutely must think about how your backup system handles redundancy, or rather, how it minimizes redundancy across multiple copies. One thing I always tell people is to think about deduplication, because that is a huge storage saver. When you perform a backup, the system does not necessarily store every single bit of data again, even if it was backed up last week. Instead, it only records the unique blocks of data, and if a file has not changed, it just notes that, which saves you an enormous amount of space over time.
And then there is Change Block Tracking, or CBT, and you really need to understand how that works for these 100 VMs. CBT allows the system to quickly pinpoint only the specific data blocks that have actually been modified since the last successful backup, and this greatly improves the efficiency of the process, but it also means your calculations aren't based on the full VM size every time, which is a benefit for your bandwidth, and it helps keep your storage requirements lower too. I think you need to factor in the overhead of these indexing systems, because while they save space, they do consume some space themselves to track the changes, so you are really doing a multi-layered calculation here.
Also, you should also think about the operational data that your backup system itself will generate. Every time a backup job runs, it creates metadata, logs, and index information that tracks what was backed up, when, and to where, and while this is small compared to the VM data, it adds up when you are managing 100 separate servers. And Or, if you are storing this data across multiple different repositories or targets, each connection point will have its own metadata overhead, so you cannot just calculate for one single storage target.
But the most crucial calculation element after growth rate is actually calculating the recovery point objective, or RPO, and the recovery time objective, or RTO, because those metrics impact your storage architecture more than you realize. If your RPO is only four hours, you cannot afford to do full nightly backups, because then you might lose too much data, and instead, you may need to schedule more frequent, smaller differential backups, and those different schedules change your entire capacity projection. Similarly, if your RTO is very low, meaning you need to get services running almost instantly, you might need to store more complete, readily available image files, which means more storage required overall.
I think I always tell my friends that they have to build in a cushion, a substantial buffer of free space, because things always go wrong, and you cannot calculate for perfect operation. Maybe you need an extra 20 to 30 percent of capacity just for unexpected data spikes or for staging data during a capacity expansion, because you never want the backup system to run out of room when you need it the most. And maybe you should also consider the nature of the data itself, because some VMs, like database servers, have very specific access patterns that make them highly compressible, while others, like media file servers, might have massive amounts of unique, incompressible data, so you are dealing with varied data profiles.
And so, when you are actually putting this all together, the formula in your head is something like: (Current Combined Size * Retention Duration) + Projected Growth Over Time + Metadata Overhead + Buffer Contingency. It is a complex projection, really, and it requires you to collaborate with the data owners of those 100 VMs, because they are the people who know how often the data actually changes. You cannot accurately calculate it yourself based only on the server count. So, you need to treat this calculation like a very big, multi-variable equation, because the output is only as good as the inputs you gather.
