Optimizing Memory Metadata: Dynamic Allocation of Struct Pages in the Linux Kernel

• Typ der Arbeit: Masterarbeit
• Status der Arbeit: abgeschlossen
• Projekte: ParPerOS
• Betreuer: Lars Wrenger, Alexander Halbuer, Daniel Lohmann
• Bearbeiter: Paul Aumann
• Ende der Arbeit: 14. Jun 2024

Commonly, OSes need a certain amount of metadata for every physical memory page frame. Linux (among other OSes) uses a large array with a 64B entry for every 4K page frame, summing up to about 1.6 percent of total memory installed. For a server system with 384GiB of memory, this means blocking 6GiB for management purposes. Moreover, this metadata is quite difficult to maintain as it is tightly packed, and most fields are overloaded depending on the state of the corresponding page frame.

We redesigned two major parts of the Linux Kernel as part of the ParPerOS project: The page frame allocator (LLFree) and the new virtual memory objects (Morsels). Both are self-contained, meaning they do not store any metadata in the struct pages. As a result, in some cases, the struct page is not required at all, but currently, the struct page array is mostly statically allocated.

The goal of this thesis is to dynamically allocate a struct page in cases where it is needed and to omit the memory overhead in cases where it is not. We want to examine at which cost these memory savings come and what changes are required to the allocation policy and the memory subsystem to adapt such a deeply entangled paradigm as struct page.

Related Publications

USENIX Conference A Distinguished Artifact Award
LLFree: Scalable and Optionally-Persistent Page-Frame Allocation

Lars Wrenger, Florian Rommel, Alexander Halbuer, Christian Dietrich, Daniel Lohmann2023 USENIX Annual Technical Conference (USENIX '23)USENIX Association2023Distinguished Artifact Award.
PDF Details Slides [BibTex]

×

BibTex Entry

@inproceedings{wrenger:23:atc,
location = {Boston, MA},
author = {Wrenger, Lars and Rommel, Florian and Halbuer, Alexander and Dietrich, Christian and Lohmann, Daniel},
booktitle = {2023 USENIX Annual Technical Conference (USENIX '23)},
entrysubtype = {Conference},
isbn = {978-1-939133-35-9},
month = {jul},
pages = {897–914},
publisher = {USENIX Association},
title = {{LLFree}: Scalable and {Optionally-Persistent} {Page-Frame} Allocation},
url = {https://www.usenix.org/conference/atc23/presentation/wrenger},
year = {2023},
}

Close

DIMES Workshop
Morsels: Explicit Virtual Memory Objects

Alexander Halbuer, Christian Dietrich, Florian Rommel, Daniel LohmannProceedings of the 1st Workshop on Disruptive Memory SystemsAssociation for Computing Machinery2023.
PDF Details Slides 10.1145/3609308.3625267 [BibTex]

×

BibTex Entry

@inproceedings{halbuer:23:dimes,
author = {Alexander Halbuer and Christian Dietrich and Florian Rommel and Daniel Lohmann},
booktitle = {Proceedings of the 1st Workshop on Disruptive Memory Systems},
date = {2023-10},
doi = {10.1145/3609308.3625267},
entrysubtype = {Workshop},
location = {Koblenz, Germany},
month = {October},
publisher = {Association for Computing Machinery},
series = {DIMES '23},
title = {Morsels: Explicit Virtual Memory Objects},
url = {https://doi.org/10.1145/3609308.3625267},
year = {2023},
}

Close

References

• https://www.kernel.org/doc/gorman/html/understand/understand005.html
• https://www.kernel.org/doc/html/v5.6/vm/memory-model.html
• https://lwn.net/Articles/565097/