How We Saved 10 Terabytes of Disk Space Using Windows ReFS with Veeam (Including Script)

As we were running low on space on the local repository of our backup server at Paessler, we decided to utilize the benefits of Windows 2019 Resilient File System in combination with Veeam Backup & Replication.

In this blog, I'll show you the basics about Windows ReFS, the benefits it brings in combination with the Veeam backup solution, and how we monitor our entire backup environment with PRTG Network Monitor.

Windows ReFS at a Glance

For technical definitions, a short introduction via Wikipedia is always recommended. There it states:

The Resilent or Reliable File System (ReFS) was designed to overcome problems that had become significant over the years since NTFS was conceived, which are related to how data storage requirements had changed. The key design advantages of ReFS include automatic integrity checking and data scrubbing, removal of the need for running chkdsk, protection against data degradation, built-in handling of hard disk drive failure and redundancy, integration of RAID functionality, a switch to copy/allocate on write for data and metadata updates, handling of very long paths and filenames, and storage virtualization and pooling, including almost arbitrarily sized logical volumes (unrelated to the physical sizes of the used drives).

Source: wikipedia.org/wiki/ReFS

ReFS basically supports the same functions as NTFS. However, there are some differences and it has features that are not possible in ReFS. ReFS volumes cannot be compressed, nor can they be used as an encrypted file system. Also disk quotas are not supported. Also, shadow copies cannot be configured on ReFS drives and ReFS volumes are not bootable. These are the principle reasons why ReFS is mainly used for storing data or archiving. ReFS is the only file system that can handle virtual disks spanning over multiple physical disks of different types.

Resilient File System and Veeam

So what are the benefits of Resilient File System when it comes to data storage, and what happens when Veeam comes into play? Veeam Backup & Replication offers some significant benefits that provide greater performance and new space- and time-saving features when using the ReFS integration. The feature leads to faster disk cloning in shorter backup intervals and reduced backup storage capacity. All you need is a Windows Server 2016 or later, and the Veeam Backup & Replication solution.

How Does a Synthetic Backup Typically Work?

Imagine you store your data with a typical incremental chain. Every block in each incremental backup is new; there is no space saving. A synthetic operation looks different. Suppose we use the following configuration for our day-to-day backup: daily backups, incremental backups from Monday to Saturday, and a synthetic full backup on every Sunday. After one week the synthetic operation starts.

Veeam reads all blocks of all incremental backups and adds them to the previous full backup. The new full backup is created from this. That' s why it is called a synthetic backup: the new full backup is created synthetically from an existing full backup and the daily incremental backups without having to create a new full backup of the production environment. This saves a lot of I/Os including the storage space for an additional full backup.

Now Let’s See What Happens When ReFS Comes into Play!

With ReFS and the BlockClone technology, only the pointers of the modified blocks are updated. The contents of the blocks are not written to disk a second time. The updated Veeam backup file is generated from the new pointers to these blocks.

This means that only one physical copy of a block exists on the file system at a time. A block can therefore contain many pointers. The pointers need a bit of storage space for the metadata of the pointer itself, but the complete space for the contents of the block is not needed. As long as a block has at least one pointer, it is not deleted from the file system. This saves a lot of disk space, depending on configuration and data structure.

ReFS and Veeam Really Saved Us 10 Terabytes of Disk Space

Before we switched our backups, all the data backed up was 19.2 terabytes in total. Since formatting our backup hard disks with ReFS and setting up the new Veeam backup repository, the total backups occupy only 9.2 terabytes of space. This saves us approximately 50% capacity on the hard disks.

Next Step: Monitoring All Relevant Data – including the Space Savings – with PRTG Network Monitor

Of course, we want to monitor everything I have talked about so far. More specifically: whether the necessary hardware is ready for use, that backups have been created successfully, and how disk space savings are changing. PRTG Network Monitor provides us with all this data, reliably and 24/7.

1. Backup Hardware Monitoring

Let's start with hardware monitoring. Our physical backup server is equipped with various PRTG sensors. We use the Ping sensor to check the network accessibility. Sensors for CPU, disk, memory, pagefile usage and uptime provide us with additional vital values of the server in detail. With the Windows Physical Disk I/O Sensor we monitor the I/O values of the physical hard disks. The network cards installed in the server are monitored with bandwidth sensors. In addition, our sysadmin team has placed a Windows Update Status Sensor on the device, which constantly monitors how current the updates are and provides feedback if necessary.

2. Backup Application Monitoring

Next, the Veeam Backup Sensor comes into action. This sensor is based on a PRTG EXE/XML sensor, which is fed with an awesome PowerShell script. The script reads a huge number of values from the Veeam backup environment, including the status of the backup, copies and replications created, the status of the repository and of course the size of the backup. You can download the script from Github. Find all the necessary information in our PRTG Sensor Hub.

3. ReFS Space Savings Monitoring (Including Script)

Finally, we would also like to know how the disk space savings will evolve with the introduction of ReFS in conjunction with Veeam. We use an HTTP Push Data Sensor for this. So let’s see how this works in detail.

On our backup server, there’s a scheduled task running a PowerShell script once a day. The script calculates the total savings and total fragments over all files. The HTTP Push Data Sensor in PRTG grabs the script result and show all the necessary information in PRTG. In the sensor settings you just have to set the request method to POST, insert an identification token (or use the provided code) and choose the desired primary channel for the sensor display. You can also adjust the scanning interval to the running interval of the scheduled task.

So that you don't have to make all the settings by hand, we have prepared a comprehensive script for you. You can download the script and the necessary files in one package here.

The script consists of the elements below:

Get everything running with the following steps:

1. Create an HTTP Push Sensor in PRTG 
2. Download the script and extract the files to disk
3. Run Create-ScheduledTask.ps1 and follow the instructions
4. Be amazed by the amount of information about the disk space savings of your ReFS environment.

That’s pretty much it to have a transparent overview of your whole backup environment. 

Conclusion

Veeam in combination with the Windows Resilent File System saves our company a lot of physical disk storage, network resources and backup time. So check out if this solution is suitable for your environment as well. And if you want to monitor your IT infrastructure - and that's what we definitely recommend you to do 😉 - keep PRTG Network Monitor in mind!