Multi-Target Virtual-Memory Objects

• Typ der Arbeit: Bachelor-/Masterarbeit
• Status der Arbeit: reserviert
• Projekte: ParPerOS
• Betreuer: Alexander Halbuer, Daniel Lohmann

Multiple synchronized page-table sub trees with one tree beeing the leader. [Generated with AI]

Context

Current computer systems offer main memory capacities that were not even imaginable decades ago. However, memory management and abstractions are still mostly based on tiny 4-KiB fragments, allowing for fine-grained provisioning but at the expense of high management overhead for large objects. With Morsels, we introduced a novel memory-management paradigm that shifts from the management of individual pages to larger virtual-memory objects, technically represented as subtrees of the page-table hierarchy. This reduces management overhead and enables very fast transfer between address spaces.

Problem

Morsels are based on hardware-specific page-table formats, which limit them to a specific architecture, such as the AMD64 MMU format. Although IO-page-table formats differ, we have previously demonstrated that it is possible to share Morsels with the IOMMU, enabling direct access from devices like NVMe SSDs, when both formats are similar (as shown in the referenced master thesis). However, this approach has several drawbacks. It restricts the use of advanced features (e.g., memory protection keys) and reduces the number of bits available for software use within each page table entry. Furthermore, this method is infeasible if the formats are too different and important fields deviate.

To address these limitations, we propose an alternative approach, where multiple representations exist, one per MMU format. One representation, likely the existing AMD64 implementation, will serve as the authoritative truth (primary), while others will be translation views (secondary) of the same memory object. The authoritative truth ensures synchronization between threads and crash consistency using the existing log-free algorithms. In contrast, translation views directly follow the object layout and therefore do not need to be persisted before system reboot. Instead, they can be restored on first use and, for supported targets, populated lazily and partially on first access to a page within the Morsel.

Goal

The objective of this thesis is to design and develop a secondary Morsel view that mirrors the object layout of the primary Morsel. To simplify debugging, you may initially focus on using AMD64 as the common architecture for both views.

Essential features of the implementation will be:

• Creation: Create a secondary Morsel from a given primary Morsel.
• Lazy population: Initially, the secondary Morsel should be empty to reduce creation time.
  • On first access to a page, the secondary Morsel takes over the mapping from the primary Morsel.
  • If the page does not exist on the primary Morsel, it must be populated first.
• Tracking: For each primary Morsel, a list of secondary Morsels should track their existence.
• Eviction: When removing pages from the primary Morsel, the changes must be propagated to all secondary Morsels.

For Master Thesis Only

Up to this point, the implementation provides the fundamental features and allows for quantification of the additional bookkeeping overhead. The next step - the integration into an application scenario - typically is beyond the scope of a bachelor thesis. Therefore, it leaves room for a follow-up thesis or, on the other hand, allows scaling this topic to a master thesis.

The next major step will be the migration of the implementation to a different page table format. Especially well suited is the Extended Page Table (EPT) format, which serves as the foundation for nested paging (see references). This will enable us to map a Morsel into a virtual machine, where it appears as a contiguous memory range in the guest's physical address space. So, the second task will be the integration into QEMU to enable sharing of Morsels between host processes and virtual machine instances.

Schedule

The thesis will follow these key steps (bachelor: 1-3, master: 1-6):

1. Getting started: Familiarize with kernel development, set up a suitable development environment, and establish a functional test setup.
2. Implementation: Develop a secondary view for the same architecture as the primary view, allowing for simplified debugging and testing.
3. Evaluation 1: Analyze the additional bookkeeping overhead in suitable synthetic scenarios.
4. Format Migration: Port the secondary view to an alternative page table format (EPT).
5. Virtual Machine Integration: Integrate an interface into QEMU to allow sharing of Morsels between host processes and virtual machine instances.
6. Evaluation 2: Assess the implementation using a real-world application scenario.

Topics: C, Linux kernel, paging, virtual memory, QEMU, virtualization

References

Web Links

Nested Paging: An additional address translation step in the MMU creates the perception of an exclusive physical address space for each virtual machine running on a host system.

Papers

DIMES Workshop
Morsels: Explicit Virtual Memory Objects

Alexander Halbuer, Christian Dietrich, Florian Rommel, Daniel LohmannProceedings of the 1st Workshop on Disruptive Memory SystemsAssociation for Computing Machinery2023.
PDF Details Slides 10.1145/3609308.3625267 [BibTex]

×

BibTex Entry

@inproceedings{halbuer:23:dimes,
author = {Alexander Halbuer and Christian Dietrich and Florian Rommel and Daniel Lohmann},
booktitle = {Proceedings of the 1st Workshop on Disruptive Memory Systems},
date = {2023-10},
doi = {10.1145/3609308.3625267},
entrysubtype = {Workshop},
location = {Koblenz, Germany},
month = {October},
publisher = {Association for Computing Machinery},
series = {DIMES '23},
title = {Morsels: Explicit Virtual Memory Objects},
year = {2023},
}

Close

Related Theses