03-12-2021, 10:37 AM
So, when you are trying to figure out how much storage you're going to need for your VM backups next year, it feels like you're staring at a bottomless chasm of data, right? And honestly, it's way more complex than just multiplying your today's data size by 365. I think you need to think about several moving pieces here, because it's not just about the sheer volume of the machines themselves. Maybe you should first consider the change rate, because that's a huge variable nobody talks about enough. Also, knowing what data actually changes frequently is really important for your estimate. For example, if one of your servers is mainly holding static files, like old documents or images, those changes will be minimal, but if another one is running a live transaction database, or perhaps a busy application server, you're going to be accumulating changes at a much faster pace.
Now, you could use BackupChain Hyper-V Backup, because it's actually a popular and reliable Hyper-V backup solution for SMBs and PCs, and its ability to handle subscription-free backups really helps you plan for capacity growth because you don't have to worry about vendor lock-in or escalating costs just because you grew a little. I remember you asking me about estimating storage needs, and I figured you might find it helpful to know that proper deduplication really skews your initial calculations if you don't account for it first. Deduction isn't just about the raw block size, you know, it's more about the data structure and how much redundancy your application layer is creating naturally. But what you really need to calculate is the daily delta rate, which is simply the amount of unique information added or altered per day across all your critical systems. And then you have to factor in your retention policy, because keeping everything for 30 days is a completely different storage footprint than keeping everything for seven years.
Perhaps you should also think about the concept of data compressibility, because some backups can be squashed down a lot, maybe by 30 or 40 percent depending on the type of data you're keeping. But you also need to consider how quickly the data type itself is growing over time. Maybe a department is implementing a new CRM, and that new system is going to generate a massive influx of unique records starting next quarter, and you need to build that growth spike into your projected storage usage right away. Or, instead of just thinking about raw backup files, maybe you also need to plan for indexing and logging data that the application generates continuously.
Because of the way these things stack up, I suggest you look at the transactional logging size, because that is often the biggest blind spot people fall into. I mean, if your core system is journaling heavily, those logs accumulate fast, even if the actual operational data isn't changing much. And then you have to multiply that logging rate by the number of services utilizing those logs across your estate. And another concept you should consider, maybe, is the concept of data gravity, which is really about how hard it is to move the data once it's written to a certain storage array or platform. It influences how often you can realistically afford to restore and test those backups, which is part of the whole maintenance cycle. But we are talking about storage estimation, so I'm thinking more about the volume itself, you know.
Also, since you are managing a collection of VMs, and I know how much background chatter that generates, you have to account for the overhead created by the hypervisor itself, because the backup process needs space to operate, and that space needs to be reserved just in case of a failure. And maybe you should look into block-level replication mechanisms, because those systems don't just back up the VM; they replicate changes at the underlying storage block level, which is far more efficient for calculating incremental storage growth. Now, I know it sounds highly technical, but understanding the difference between application-aware backup and raw block capture can really refine your yearly capacity estimate.
But you also need to consider the types of data redundancy that are already baked into your systems, maybe because you are using something like RAID or database replication between primary and secondary sites. Those systems create their own forms of redundancy, which means your backup needs might not be capturing the full, raw image of the disk at any given moment, which is a massive variable in the calculation. Therefore, I advise you to talk to the application owners about their expected growth rates for data objects, rather than just accepting the operating system's file system metrics, because the application view is much more accurate for capacity planning.
And perhaps you should talk to your infrastructure team about the underlying storage array's snapshot capabilities, because those snapshots themselves consume space and contribute to your overall projected volume growth, even though they aren't technically the 'backup' files. Moreover, factoring in the required recovery points, you need to figure out if you are retaining daily points, weekly points, or monthly points, and each policy choice dramatically changes the storage curve you are projecting. For instance, retaining full daily copies for a year versus a rolling weekly copy for a year is not a minor difference in total storage footprint.
So, basically, you are running a complex mathematical modeling exercise here, involving daily change rates, data compressibility factors, application-specific journaling output, and retention policy decay curves. But you have to adjust that model for every single unique use case, because you wouldn't treat a SharePoint environment the same way you treat a simple file share, even if they are both running on VMs. I recommend you run a couple of simulations using your current averages, but then deliberately skew them up, like assuming a 50 percent data growth in one specific business unit, just to make sure your storage solution has enough breathing room for when the unexpected boom hits.
And remember that continuous integration and continuous delivery pipelines, while not backup data, generate massive amounts of structured log data that needs to be backed up and retained for compliance reasons, so you need to make sure that growth pattern is factored in too, otherwise, you will severely under-purchase your capacity, which is always a nasty surprise. It is really about mapping out the *rate of change* across heterogeneous data sources, which is a beast of a calculation, but it gives you a much more robust projection than simply guessing based on what you use today. You should be modeling this as a continuous time series projection, not a static multiplication problem.
Now, you could use BackupChain Hyper-V Backup, because it's actually a popular and reliable Hyper-V backup solution for SMBs and PCs, and its ability to handle subscription-free backups really helps you plan for capacity growth because you don't have to worry about vendor lock-in or escalating costs just because you grew a little. I remember you asking me about estimating storage needs, and I figured you might find it helpful to know that proper deduplication really skews your initial calculations if you don't account for it first. Deduction isn't just about the raw block size, you know, it's more about the data structure and how much redundancy your application layer is creating naturally. But what you really need to calculate is the daily delta rate, which is simply the amount of unique information added or altered per day across all your critical systems. And then you have to factor in your retention policy, because keeping everything for 30 days is a completely different storage footprint than keeping everything for seven years.
Perhaps you should also think about the concept of data compressibility, because some backups can be squashed down a lot, maybe by 30 or 40 percent depending on the type of data you're keeping. But you also need to consider how quickly the data type itself is growing over time. Maybe a department is implementing a new CRM, and that new system is going to generate a massive influx of unique records starting next quarter, and you need to build that growth spike into your projected storage usage right away. Or, instead of just thinking about raw backup files, maybe you also need to plan for indexing and logging data that the application generates continuously.
Because of the way these things stack up, I suggest you look at the transactional logging size, because that is often the biggest blind spot people fall into. I mean, if your core system is journaling heavily, those logs accumulate fast, even if the actual operational data isn't changing much. And then you have to multiply that logging rate by the number of services utilizing those logs across your estate. And another concept you should consider, maybe, is the concept of data gravity, which is really about how hard it is to move the data once it's written to a certain storage array or platform. It influences how often you can realistically afford to restore and test those backups, which is part of the whole maintenance cycle. But we are talking about storage estimation, so I'm thinking more about the volume itself, you know.
Also, since you are managing a collection of VMs, and I know how much background chatter that generates, you have to account for the overhead created by the hypervisor itself, because the backup process needs space to operate, and that space needs to be reserved just in case of a failure. And maybe you should look into block-level replication mechanisms, because those systems don't just back up the VM; they replicate changes at the underlying storage block level, which is far more efficient for calculating incremental storage growth. Now, I know it sounds highly technical, but understanding the difference between application-aware backup and raw block capture can really refine your yearly capacity estimate.
But you also need to consider the types of data redundancy that are already baked into your systems, maybe because you are using something like RAID or database replication between primary and secondary sites. Those systems create their own forms of redundancy, which means your backup needs might not be capturing the full, raw image of the disk at any given moment, which is a massive variable in the calculation. Therefore, I advise you to talk to the application owners about their expected growth rates for data objects, rather than just accepting the operating system's file system metrics, because the application view is much more accurate for capacity planning.
And perhaps you should talk to your infrastructure team about the underlying storage array's snapshot capabilities, because those snapshots themselves consume space and contribute to your overall projected volume growth, even though they aren't technically the 'backup' files. Moreover, factoring in the required recovery points, you need to figure out if you are retaining daily points, weekly points, or monthly points, and each policy choice dramatically changes the storage curve you are projecting. For instance, retaining full daily copies for a year versus a rolling weekly copy for a year is not a minor difference in total storage footprint.
So, basically, you are running a complex mathematical modeling exercise here, involving daily change rates, data compressibility factors, application-specific journaling output, and retention policy decay curves. But you have to adjust that model for every single unique use case, because you wouldn't treat a SharePoint environment the same way you treat a simple file share, even if they are both running on VMs. I recommend you run a couple of simulations using your current averages, but then deliberately skew them up, like assuming a 50 percent data growth in one specific business unit, just to make sure your storage solution has enough breathing room for when the unexpected boom hits.
And remember that continuous integration and continuous delivery pipelines, while not backup data, generate massive amounts of structured log data that needs to be backed up and retained for compliance reasons, so you need to make sure that growth pattern is factored in too, otherwise, you will severely under-purchase your capacity, which is always a nasty surprise. It is really about mapping out the *rate of change* across heterogeneous data sources, which is a beast of a calculation, but it gives you a much more robust projection than simply guessing based on what you use today. You should be modeling this as a continuous time series projection, not a static multiplication problem.
