tomasz kapela, @tomaszkapela linuxcon europe, berlin 2016...the fancy typed pointer...
TRANSCRIPT
Tomasz Kapela, @tomaszkapela
LinuxCon Europe, Berlin 2016
Hype
3
The NVM programming model
The solution
5
NVMLOpen source set of libraries available at www.github.com/pmem/nvml
Library Description
libpmem The basic library, flush to persistence
libvmem Volatile Memory Allocator
libvmmalloc Transparent use of libvmem
libpmemblk Persistent memory carved into blocks
libpmemlog Log file (append-mostly)
libpmemobj Transactional Object Store
6
libpmemobj
Application
libpmem
Load/Store
memory-mapped pmem
libpmemobj
transactionsatomic
operations
allocatorlists
locks
API API API
Pointers by libpmemobj
8
The fancy typed pointer – TOID
Type safety introduced to a fancy based pointer.
9
The fancy typed pointer – TOID
Type safety introduced to a fancy based pointer.
10
The fancy typed pointer – TOID
Type safety introduced to a fancy based pointer.
11
The fancy typed pointer – TOID
Type safety introduced to a fancy based pointer.
12
The fancy typed pointer – TOID
Type safety introduced to a fancy based pointer.
Type declarations necessary
13
The fancy typed pointer – TOID
Type safety introduced to a fancy based pointer.
14
The fancy typed pointer – TOID
Type safety introduced to a fancy based pointer.
Type declarations necessary
A lot of macros to make this feasible
15
The fancy typed pointer – TOID
Type safety introduced to a fancy based pointer.
16
The fancy typed pointer – TOID
Type safety introduced to a fancy based pointer.
Type declarations necessary
A lot of macros to make this feasible
A lot of macros to handle the TOID
17
The fancy typed pointer – TOID
Type safety introduced to a fancy based pointer.
18
The fancy typed pointer – TOID
Type safety introduced to a fancy based pointer.
Type declarations necessary
A lot of macros to make this feasible
A lot of macros to handle the TOID
– Why?
19
The fancy typed pointer – TOID
Type safety introduced to a fancy based pointer.
20
The fancy typed pointer – TOID
Type safety introduced to a fancy based pointer.
Type declarations necessary
A lot of macros to make this feasible
A lot of macros to handle the TOID
– Why?
Is it usable?
21
The fancy typed pointer – TOID
Type safety introduced to a fancy based pointer.
Type declarations necessary
A lot of macros to make this feasible
A lot of macros to handle the TOID
– Why?
Is it usable?
Addressing the pain points
23
The fancy typed pointer – persistent_ptr
Encapsulate type info into a smart pointer.
No explicit type declaration
24
The fancy typed pointer – persistent_ptr
Encapsulate type info into a smart pointer.
No explicit type declaration
– typeid(T).hash_code()
25
The fancy typed pointer – persistent_ptr
Encapsulate type info into a smart pointer.
No explicit type declaration
– typeid(T).hash_code()
No more macros!
26
The fancy typed pointer – persistent_ptr
Encapsulate type info into a smart pointer.
27
The fancy typed pointer – persistent_ptr
Encapsulate type info into a smart pointer.
No explicit type declaration
– typeid(T).hash_code()
No more macros!
Mimics a well known construct
– std::shared_ptr
Now is it usable?
28
The fancy typed pointer – persistent_ptr
Encapsulate type info into a smart pointer.
29
The fancy typed pointer – persistent_ptr
Encapsulate type info into a smart pointer.
30
The fancy typed pointer – persistent_ptr
Known issues
No polymorphic types allowed in persistent memory
– Vtables rebuild issue
typeid(T).hash_code()
– Returns an unspecified value, except that within a single execution of the program, it shall return the same value for any two type_info objects which compare equal.
Transactions by libpmemobj
32
Transactions
Consistency critical roll-back transactions.
ACID like properties
33
Transactions
Consistency critical roll-back transactions.
ACID like properties
„C” exceptions
34
Transactions
Consistency critical roll-back transactions.
ACID like properties
„C” exceptions
35
Transactions
Consistency critical roll-back transactions.
ACID like properties
„C” exceptions
36
Transactions
Consistency critical roll-back transactions.
ACID like properties
„C” exceptions
Macros to make this usable
37
Transactions
Consistency critical roll-back transactions.
ACID like properties
„C” exceptions
Macros to make this usable
38
Transactions
Consistency critical roll-back transactions.
ACID like properties
„C” exceptions
Macros to make this usable
39
Transactions
Consistency critical roll-back transactions.
ACID like properties
„C” exceptions
Macros to make this usable
A bit error prone
– Even with the macros
40
Transactions
Consistency critical roll-back transactions.
41
Transactions
Consistency critical roll-back transactions.
ACID like properties
„C” exceptions
Macros to make this usable
A bit error prone
– Even with the macros
Snapshotting
42
Transactions
Consistency critical roll-back transactions.
Addressing the pain points
44
Transactions
Making life easier with RAII and lambdas.
Manual and automatic transaction
45
Transactions
Making life easier with RAII and lambdas.
Manual and automatic transaction
46
Transactions
Making life easier with RAII and lambdas.
Manual and automatic transaction
– The problem with std::uncaught_exception
47
Transactions
Making life easier with RAII and lambdas.
Manual and automatic transaction
– The problem with std::uncaught_exception
– CppCon2015/Declarative Control Flow - Andrei Alexandrescu
48
Transactions
Making life easier with RAII and lambdas.
Manual and automatic transaction
– The problem with std::uncaught_exception
– Still can’t throw an exception though
49
Transactions
Making life easier with RAII and lambdas.
Manual and automatic transaction
– The problem with std::uncaught_exception
– Still can’t throw an exception though
The lambda version
50
Transactions
Making life easier with RAII and lambdas.
Manual and automatic transaction
– The problem with std::uncaught_exception
– Still can’t throw an exception though
The lambda version
51
Transactions
Making life easier with RAII and lambdas.
Manual and automatic transaction
– The problem with std::uncaught_exception
– Still can’t throw an exception though
The lambda version
52
Transactions
Making life easier with RAII and lambdas.
Manual and automatic transaction
– The problem with std::uncaught_exception
– Still can’t throw an exception though
The lambda version
– Self contained
– Somewhat ugly looking
Using the containers
54
Persistent memory containers
A proof of concept under way.
Targeting libc++ and libstdc++
55
Persistent memory containers
A proof of concept under way.
Targeting libc++ and libstdc++
56
Persistent memory containers
A proof of concept under way.
Targeting libc++ and libstdc++
– Libc++ is mostly working
– Libstdc++ needs a bit more work
57
Persistent memory containers
A proof of concept under way.
Targeting libc++ and libstdc++
– Libc++ is mostly working
– Libstdc++ needs a bit more work
58
Persistent memory containers
A proof of concept under way.
Targeting libc++ and libstdc++
– Libc++ is mostly working
– Libstdc++ needs a bit more work
Main pros
– It’s for free! (well, almost)
– Familiar interface
– Well tested implementation of containers
59
Persistent memory containers
What are the known issues
Internally held data is not „aware” of persistence
T* used instead of allocator_traits::pointer
Layout versioning
Current implementation works only on x86_64
61
Handy links
The allocator branch:
https://github.com/tomaszkapela/nvml/tree/cpp-allocator
Libpmemobj++ documentation:
http://pmem.io/nvml/cpp_obj/
Pmem Google Groups page:
https://groups.google.com/forum/#!forum/pmem