Storage Spaces Direct in Windows Server 2019

Storage Spaces Direct (S2D) is the award-winning software-defined storage (SDS) tecnhology that was first introduced in Windows Server 2016. Since that moment, S2D has become my one of the favourite and frequently used role in Windows Server (on-premises and even in Azure).

Windows Server 2019 that is going to be generally available in the second half of this year (September or October – in accordance with WS 2012-2016 release dates) continues to develop S2D by adding new features. Why don’t we discuss them right now?

TIP: Windows Server 2019 preview build is available here

Scalability

More volumes, bigger capacity per server and cluster

Storage Spaces Direct Scalability in Windows Server 2019

USB Witness

Storage Spaces Direct can be configured with just two nodes. However, we will need to design witness placement to protect our cluster from unexpected failure (in other words, we should achieve quorum). In Windows Server 2016, you can place witness on a file share, cloud (details here) or disk witness (it’d be very strange if it was actually used in S2D clusters). What about customers who don’t have any AD infrastructure or Azure/Internet access? Here is a breakthrough – USB Witness.

usb witness windows server 2019

In short, you will be able to configure a true two-node S2D by using USB thumb drive connected to the router that is already used for VM/management traffic between nodes, for instance. Other two network interfaces (shown on the picture above), could be RDMA-adapters (recommended and supported) or Thunderbolt (POC, Project Kepler-47).

Simply insert the USB drive into the port on the router, set the share name and access information, configure witness in PowerShell: Set-ClusterQuorum -FileShareWitness \path\ -Credential  and you are ready to go.

TIP: the router should support SMB2+ and USB. And, given that a witness.log is a quite small file (just kilobytes), you can use any-sized USB drives. The list of the supported routers will be available later.

Data Deduplication

ReFS  is the recommended file system for S2D, improves VHDX creation/expansion speed (enables Accelerated VHDX operations), provides higher stability by detecting corruptions and allowing you to repair them with no volume downtime. However, some features such as ODX, Data Deduplication are not supported by ReFS in Windows Server 2016.

Starting with the Windows Server 2019 (1709 and later), Data Deduplication has been fully supported for ReFS. It means that you no longer need to choose between NTFS and ReFS file systems while planning S2D volumes. Create ReFS volume, enable Data Deduplication (PowerShell/Windows Admin Center), and then check the savings of storage space (use PowerShell and Get-DedupVolume cmdlet).

data deduplication s2d windows server 2019

Proactive outlier detection

It was quite challenging to investigate  performance issues in S2D in Windows Server 2016. We had to use PowerShell or performance counters (+VMFleet) to get a full picture of our setup’s behavior. Windows Server 2019 significantly simplifies that. S2D now records the outcome (success/failure) and latency (elapsed time) for every read/write IO to every drive without any performance impact. Therefore, drives with latency/outcome issues will be marked in PowerShell and Windows Admin Center as “Abnormal Latency” status. In addition, you can organize pooled drives into peer groups, and then compare latency against peers to quickly find any bottlenecks (new cmdlet: Get-PhysicalDiskIoReport).

This azure-inspired mechanism works on the lower level than performance counters and enabled by default for every SATA,SAS,NVMe drives.

latency outlier detection s2d 2019

Others

-Faster mirror-accelerated parity volumes (~x2)

-PMEM (Persistent Memory) drives support (Intel Optane/NVDIMM-N) for use as cache and capacity

-Deep integration with Windows Admin Center (a free HTML5-based management interface for entire Windows Server infrastructure. We’ll look at this a bit later)

-New networking recommendations for high performance, at scale, or deployments of 4+ nodes:  25 Gbps (or higher) NICs (two or more) that are remote-direct memory access (RDMA) capable, iWARP (recommended) or RoCE

My first book is published! (VMM 2016 Cookbook)

My first book that I have been working day and night on for the last four months is published and globally available! The System Center Virtual Machine Manager 2016 Cookbook (English, 575 pages) includes multiple tips, tricks and techniques to help you make the perfect VMM fabric (see What this book covers  section at the bottom of this post for details)

I have done a complete makeover of the previous edition and,therefore, we have the  chapters written from the scratch, plenty of new recipes and revised old ones to meet changes in VMM 2016.  This book is essentially intended to system engineers, solution architects, administrators and anyone who want to learn and master Virtual Machine Manager 2016 (however, since we have two channels (LTSB/SAC), you will also find references to the latest VMM 1801 release in the semi-annual channel).

The book is available in paperback and ebook formats at Amazon and Packt

Please don’t forget to write your feedback/review on Amazon.

Acknowledgements

I am grateful to all of those with whom I have had the pleasure to work during this project. This book would not be possible without Packt team who found me and offered to take up the writing. I say thank you to Devika Battike, Manish Shanbhag, Heramb Bhavsar and Prateek Bharadwaj for supporting and helping me along the way.

A special thank goes to Edvaldo Alessandro Cardoso, an author of two previous editions, for an active participation and for being a technical reviewer together with Tomica Kaniski. I could not have finished this edition without your help.

In addition, I wish to mention the Microsoft team for helping to make this book as accurate as possible, in particular: Steven Ekren, Elden Christensen Sai Prasanna Vudataneni, Krupesh Dhruva and Sonal Agarwal. I am pretty sure our long discussions and your feedbacks will be appreciated by readers.

I have worked nonstop on the book for the last months and truly couldn’t survive without my family. Nobody has been more important to me than you. Thank you very much for your support, inspiration and love.

What this book covers
  • Chapter 1, VMM 2016 Architecture, provides an understanding of the VMM modular architecture, which is useful when designing VMM and troubleshooting deployment. This chapter also covers all requirements that must be satisfied to make a private cloud.
  • Chapter 2, Upgrading from Previous Versions, walks through all the necessary steps to upgrade the previous version of Virtual Machine Manager to the new VMM 2016, covering its database, highly available configurations and post-upgrade tasks.
  • Chapter 3, Installing VMM 2016, focus on deploying VMM and it’s dependencies. It gives also a plenty of tips and tricks to install and automate VMM and SQL Server deployments in both Windows Server Core and Full environments.
  • Chapter 4, Installing a High Available VMM Server, dives into more advanced VMM configuration, and provides an understanding how VMM has become a critical part of the private cloud infrastructure. You will also learn how to make a highly available library server and VMM configuration database.
  • Chapter 5, Configuring Fabric Resources, discusses building a new fabric in VMM by configuring compute, storage and networking resources. It starts by adding hosts group and ends by creating a hyper-converged cluster with Storage Spaces Direct and Hyper-V. It also covers a deployment of a Network Controller providing a good start point for network virtualization implementation.
  • Chapter 6, Configuring Guarded Fabric, walks you through the recipes to help protect confidential data by deploying new shielded VMs as a part of Guarded Fabric consisting of Guarded Hosts and Host Guardian Service. It also discusses how to convert existing VMs to shielded and manage them through VMM.
  • Chapter 7, Deploying Virtual Machines and Services, provides information to help the administrator to create,deploy and manage private clouds, virtual machines, templates, and services in VMM 2016; it provides recipes to assist you in getting the most our of deployment.
  • Chapter 8, Managing VMware ESXi Hosts, shows you how to manage and make VMware recources available to private cloud deployments. It also covers converting VMware machines to Hyper-V (V2V), deploying virtual machines and templates, all from the VMM console.
  • Chapter 9, Managing Clouds, Fabric Updates, Resources, Clusters and the New Features of 2016, covers other new features of VMM 2016 such as Cluster OS Rolling upgrade and Production Checkpoints. You will also learn how to integrate VMM 2016 with Windows Azure Pack for VM Clouds management.
  • Chapter 10, Integration with System Center Operations Manager 2016, guides you through the steps required to complete integration of SCOM 2016 with VMM in order to enable monitoring of the private cloud infrastructure.

VMM 2016 Cookbook

Code Files