We will discuss about VMFS filesystem and VMFS locking mechanisms in this post along with VMFS 5 and VMFS 6 characteristics in vSphere 7. Also we will discuss VMFS upgrade possibilities from VMFS 5 to VMFS 6.
VMFS 5
- VMFS 5 and 6 do share most of the characteristics except few like sub block sizes,drives. Do check my other post on VMFS 5. I have listed characteristics of VMFS 5 there already.
VMFS 6
- VMware proprietary file system designed to be cluster aware and optimised for virtualized platform and workloads.
- Dynamically expandable
- Maximum datastore size: 64 TB
- Maximum usable size: 62 TB
- Block size: 1 MB
- Sub-Block Size: 64 KB
- Files that are less than 1 KB are stored in metadata space directly.
- Maximum number of datastore per host: 1024
- Maximum number of paths : 4096
- Provides on-disk, block-level locking.
- Supported storage drives: 512n/512e/4kn
- Supported Sparse format: SEsparse
Can we upgrade to VMFS 6 from VMFS 5?
There is structural differences in metadata between VMFS 5 and VMFS 6, so we cannot directly upgrade from VMFS 5 to VMFS 6. However, since we do have co-existence available, we can simply migrate the data using storage vMotion from VMFS 5 datastore to VMFS 6 datastore and later decommission the VMFS 5 datastore if required.
VMFS Locking mechanisms
There are two types of locking mechanisms that are supported in VMFS filesystem:
- SCSI reservations (aka SCSI-2 Reservations or SCSI-Reserve and Release)
- Primitive ATS (aka Hardware assisted locking)
Before we dig into these methods, let’s understand the purpose of using these locking mechanisms. VMFS datastore uses locking mechanisms like SCSI reservation to prevent multiple hosts from concurrently writing to the metadata and ensure the data integrity is maintained while metadata operations for datastore or virtual machines are being performed by a host. Metadata update is required each time when metadata operations are being performed.
The Metadata operations are:
- Creating, growing, or locking a virtual machine file
- Changing attributes of a file
- Powering a virtual machine on or off
- Creating or deleting a VMFS datastore
- Expanding a VMFS datastore
- Creating a template
- Deploying a virtual machine from a template
- Migrating a virtual machine with vMotion
With that being discussed, lets move on to our next block and look into each of the locking mechanisms. i.e. SCSI reservation and ATS
SCSI reservations
- Provides two commands:
- SCSI Reserve – To lock the target
- SCSI Release – To release the target
SCSI reservations allows a host to lock the entire LUN and block access to other hosts while performing an operation that requires metadata protection. We already talked about these metadata operations earlier in this post. Reserving the LUN prevents any possible data corruption that may be caused if other hosts overwrites the metadata.
An initiator on ESXi host sets the reservation on target LUN by issuing SCSI Reserve command. Once the metadata operation is completed, the lock owner host releases the lock by issuing SCSI Release command.
However, there is one more possibility of lock getting released i.e. SCSI bus reset. If a SCSI bus reset is performed due to some error recovery, that would cause the reservation to be released without lock owner releasing the lock.
VAAI primitive ATS
- Atomic test and set (ATS) .
- Used for storage devices that support T10 standard-based VAAI specifications
ATS is anytime preferred over SCSI reservations due to the fact that ATS allows disk locking per sector, unlike SCSI reservation where entire LUN getting locked. Hence the performance degradation does not happen with ATS. ESXi host uses ATS for VMFS datastore’s that are backed by an array with VAAI support.
Usage in VMFS
VMFS datastore can use ATS-only or the combination of ATS and SCSI reservations (i.e. ATS+SCSI). With ATS + SCSI combination, VMFS datastore first attempts to use ATS if possible, or else resorts to SCSI reservations as alternate method.
ATS+SCSI mechanism is used by VMFS5 datastore’s that are upgraded from VMFS 3 and also for new VMFS5 or VMFS6 datastore’s that are backed by storage that does not support VAAI.
To display information related to VMFS locking mechanisms, use esxcli cmdlet.
esxcli storage vmfs lockmode list
Check more info here in VMware Docs for locking modes.
Thats all in this post. Feel free to comment to discuss more. Also check about SCSI-3 Persistent reservation which is added in vSphere 7 at VMDK level.