Site icon Virtual Maestro

Transparent Page Sharing (TPS) in vSphere 6.0 PART3

Abhijeet D
Advertisements

On an ESXi host, Many ESXi workloads present opportunities for sharing memory across virtual machines (as well as within a single virtual machine). So instead of creating multiple duplicate pages in host memory for virtual machines, TPS is used to perform memory page sharing.

In vSphere 6, intra-VM TPS is enabled by default and inter-VM TPS is disabled by default, due to some security concerns as described in VMware KB 2080735.

With page sharing, the hypervisor reclaims the redundant copies and keeps only one copy, which is shared by multiple virtual machines in the host physical memory. As a result, the total virtual machine host memory consumption is reduced and a high memory over commitment is possible.

How TPS works? 

ESXi scans the content of guest physical memory for sharing opportunities. Instead of comparing each byte of a candidate guest physical page to other pages, ESXi uses hashing to identify potentially identical pages.

Image: VMware
  • A hash value is generated based on the virtual machines physical page’s (GA) content and stored in global hash table. Each entry in global hash table includes a hash value and the physical page number of a shared page
  • The hash value is used to look up a global hash table. If the hash value of virtual machines physical page matches an existing entry in hash table, a bit-by-bit comparison of the page contents is performed to exclude any false match.
  • Once the virtual machine physical page’s content matches with the content of an existing shared host physical page, the guest physical (GA) to host physical mapping (HA) of the virtual machine physical page is changed to the shared host physical page, and the redundant host memory copy (the page pointed to by the dashed arrow in above image) is reclaimed.
  • This remapping is invisible to the virtual machine and inaccessible to the guest operating system. Because of this invisibility, sensitive information cannot be leaked from one virtual machine to another.
  • Any attempt to write to the shared pages will generate a minor page fault. In the page fault handler, the hypervisor will transparently create a private copy of the page for the virtual machine and remap the affected guest physical page to this private copy. A standard copy-on-write (CoW) technique is used to handle writes to the shared host physical pages.

In hardware-assisted memory virtualization (Intel EPT and AMD RVI) systems, ESXi will not share large pages because:

  • The probability of finding two large pages having identical contents is low
  • The overhead of doing a bit-by-bit comparison for a 2MB page is much larger than for a 4KB page

Since ESXi will not swap out large pages, the large page (2MB) will be broken into small pages (4KB) during host swapping so that these pre-generated hashes can be used to share the small pages before they are swapped out.

What is Salting in TPS?

Salting is used to allow more granular management of the virtual machines participating in TPS. Salting is enabled after the ESXi update mentioned below are deployed.

• ESXi 5.0 Patch ESXi500-201502001
• ESXi 5.1 Update 3
• ESXi 5.5, Patch ESXi550-201501001
• ESXi 6.0

By default, salting is set Mem.ShareForceSalting=2 and each virtual machine has a different salt. This means page sharing does not occur across the virtual machines (inter-VM TPS) and only happens inside a virtual machine (intra VM).

When salting is enabled (Mem.ShareForceSalting=1 or 2) in order to share a page between two virtual machines both salt and the content of the page must be same. A salt value is a configurable vmx option for each virtual machine. You can manually specify the salt values in the virtual machine’s vmx file with the new vmx option sched.mem.pshare.salt. If this option is not present in the virtual machine’s vmx file, then the value of vc.uuid vmx option is taken as the default value. Since the vc.uuid is unique to each virtual machine, by default TPS happens only among the pages belonging to a particular virtual machine (Intra-VM).

How can I enable or disable salting?

  1. Log in to ESX (i)/vCenter with the VI-Client.
  2. Select ESX (i) relevant host.
  3. In the Configuration tab, click Advanced Settings (link) under the software section.
  4. In the Advanced Settings window, click Mem.
  5. Search for Mem.ShareForceSalting and set the value to 1 or 2 (enable salting), 0(disable salting).
  6. Click OK.
  7. For the changes to take effect do either of the two:
    • Migrate all the virtual machines to another host in cluster and then back to original host. Or
    • Shutdown and power-on the virtual machines.

Steps to specify the salt value for a virtual machine:

  1. Power off the virtual machine on which you want to set salt value.
  2. Right click on virtual machine, click on Edit settings.
  3. Select options menu, click on General under Advanced section.
  4. Click on Configuration Parameters….
  5. Click on Add Row, new row will be added.
  6. On LHS add text sched.mem.pshare.salt and on RHS specify the unique string.
  7. Power on the virtual machine to take effect of salting.
  8. Repeat steps 1 to 7 to set the salt value for individuals virtual machine.

Note: Same salting values can be specified to achieve the page sharing across virtual machines.

You can change the TPS behavior by applying the salting mechanism as described in VMware KB 2097593.

Below is the list of articles in this series for further reading.

Exit mobile version