Storage System is a unique course due to its sole focus on NVM storage and its impact on research and education. We take inspiration from the 2018 Data Storage Research Vision 2025 report which identifies (see section 6.1)
“Many students may only associate storage systems with hard disk drives or a specific file system, which is obviously less attractive compared to, say, self-driving cars. This situation is partly due to the fact that there is no clearly defined course on storage systems in the majority of universities.”
Figure 1: The new storage hierarchy
Storage Systems (VU catalogue) is a MSc-level course that is first established and offered in 2020. The course covers the rise of Non-Volatile Memory (NVM) storage technologies in commodity computing, their impact on system design (architecture, operating system), distributed systems, storage services, and application designs. We cover the following topics (2021):
1. Introduction: History, NAND flash, internal organization, the new triangle of Storage Hierarchy.
2. Host Interfacing and Software Implications: NVMe, storage and block-layer optimizations.
3. Flash FTL and Garbage Collection: FTL and GC designs, concerns, and host-managed FTLs.
4. Flash Filesystems: Log-structured file systems, F2FS, DFS, and Nameless writes.
5. Flash KV Stores: B+ Trees, Hash Tables, and LSM trees on flash (LOCS, WiscKey, uTree, SILK).
6. Byte-addressable Persistent Memories: Optane, NVHeap, and Pmem/PMDK project.
7. Networked Flash: Disaggregated storage, NVMoF, Disaggregated Flash, and FlashNet.
8. Programmable Storage: What is CSD, Willow, Biscuit, INSIDER.
9. Distributed Storage - I: Distributed temporary data storage and formats (Crail and Albis).
10. Distributed Storage - II: Talks about Corfu and Tango distributed transaction systems.
All slides are freely available at (under the CC license): https://drive.google.com/drive/folders/1Ob994kg2UBFdrdgeEAReNILmCx7mSlxT?usp=sharing.
Feel free to modify and use the slides in your course as you see fit.
PS~ We are on a lookout for a cool course mascot…if you have ideas, drop me an email ☺︎
For the practical work, students develop an NVM flash translation layer (the essential part of any modern NVM storage device) and integrate a file system in RocksDB. There are five milestone in the practical work:
The project handbook is publicly available here PDF. Drop me an email if you want access to more project related resources.
This course content are distributed under the Creative Commons Attribution 4.0 International (CC BY 4.0), https://creativecommons.org/licenses/by/4.0/
The project work is generously supported by Western Digital with their donation of ZNS devices and software support.