Item — HN Reader

Interesting!

Incidentally (since this may be somewhat related), I'm wondering, what are your thoughts on the Persistent Memory Manager approach, as in the following:

Justin Meza, Yixin Luo, Samira Khan, Jishen Zhao, Yuan Xie, and Onur Mutlu: "A Case for Efficient Hardware/Software Cooperative Management of Storage and Memory." Workshop on Energy-Efficient Design, 2013.

Context: "emerging high-performance NVM technologies enable a renewed focus on the unification of storage and memory: a hardware-accelerated single-level store, or persistent memory, which exposes a large, persistent virtual address space supported by hardware-accelerated management of heterogeneous storage and memory devices. The implications of such an interface for system efficiency are immense: A persistent memory can provide a unified load/store-like interface to access all data in a system without the overhead of software-managed metadata storage and retrieval and with hardware-assisted data persistence guarantees."

The stated goals/benefits include eliminating operating system calls for file operations, eliminating file system operations, and efficient data mapping.

Paper: http://justinmeza.com/bin/meza_weed13.pdf

Presentation: https://users.ece.cmu.edu/~omutlu/pub/mutlu_weed13_talk.pdf

2 comments

One giant flat address space is not the answer. Hardware people tend to come up with approaches like that because flat address spaces and caching are well understood hardware. It's the same thinking that leads to "storing into device registers" as an approach to I/O control, even when the interface is really packets over a serial cable as in FireWire or USB or PCI Express.

File systems and databases are useful abstractions, from an ease of use, security, and robustness perspective. The challenge is to make them go faster. Pushing the machinery behind them out to special-purpose hardware can do that.

The straightforward thing to do first is to to take some FPGA part and use it to implement a large key/value store using non-volatile solid state memory. That's been done at Stanford[1], Berkeley[2], and MIT[3], and was suggested on YC about six years ago.[4] One could go further, and implement more of an SQL database back end. It's an interesting data structure problem; the optimal data structures are different when you don't have to wait for disk rotation, but do need persistence and reliability.

[1] http://csl.stanford.edu/~christos/publications/2014.hwkvs.nv... [2] https://www.cs.berkeley.edu/~kubitron/courses/cs262a-F14/pro... [3] https://dspace.mit.edu/handle/1721.1/91829 [4] https://news.ycombinator.com/item?id=1628550

OK I find it easier to follow these ideas when thinking about how loads/stores to volatile memory are organized. Memory is not accessed via a syscall. Instead the OS sets up some data structures in the MMU and lets the application run. Some kind of fault happens when control must be transferred back to the OS.

Going back to non-volatile memory the question is what kind of abstraction should be implemented in hardware? Presumably something simple that the OS and applications can then use to implement higher level abstractions like file systems and databases. Pushing parts of a SQL database engine into the hardware does not intuitively seem like a right solution.

Thanks, that is something that I was looking for! Nice to see it really happening!