Add some notes on current project state.

NOTES.md

@@ -4,6 +4,58 @@ The new version of pls uses a more complicated/less user friendly
 API in favor of performance and memory guarantees.
 For the old version refer to the second half of this document.
 
+# 13.03.2020 - Coroutines and 'Nice API'
+
+Before Christmas we finished the 'ugly' inline/object based API.
+Because this was very hard to use (even for us), the only options
+where to go to a class based solution like TBB (everything MUST be a
+class, as soon as you call into user code promises are gone) or to
+go with a fully fledged continuation based API.
+
+We decide to go for the continuation API, as one of our project questions
+was if we can incorporate a nice high level API with a library only solution.
+
+Following this we need coroutines and we also need the stealing to work for more
+than two child tasks.
+
+## Coroutines basic implementation
+
+We go for an implementation similar to [boost.context](https://github.com/boostorg/context),
+where the context is stored and changed using custom assembly.
+We do so, as jumpbuf is not portable at all (i.e. behaviour for the jumps
+we intent to do are EXPLICITLY not allowed in the standard, see 
+[docs](https://en.cppreference.com/w/c/program/setjmp) on jumping other
+than what expections are allowed to do).
+
+Inspirations for what we use in our end-product are the [deboost.context](https://github.com/septag/deboost.context),
+ [tinycoroutine](https://github.com/soundsrc/tinycoroutine).
+ 
+Our final implementatino uses custom assembly on supported platforms,
+being faster than boost.context as we choose to ommit unneccesary setup steps,
+as the focus is on quick switching into a new routine. We plan to also use
+this for 'tail calls' in our future code (and think that this is a good fit if
+we plan for composable APIs in future work).
+ 
+## TSan support...
+ 
+We want to integrate tsan into our fiber for sure, as it has helped us in the
+past to catch various bugs. Unfortionetly, we found that our programm crashes 
+when running it with it. The first solution was to add the new (mid 2019) API
+for explicit fiber switching. This seemed to work, however, the programm crased
+after some time. After long research, we found that it was a memory leak in
+thet tsan implementation. [We fixed it](https://reviews.llvm.org/D76073) 
+and integrated support for it into a seperate coroutine library.
+
+## Multiple child steal
+
+We then needed to adopt our stealing procedure/queue to support multiple
+child tasks. To do so in a lock-free manner required some tinkering.
+-> further details are in our paper notebook for now. In summary,
+we use a flag to atomicially trade in resources when stealing a tasks
+and use a lock free stack to store the resources pending on a task.
+
+
+
 # 05.11.2019 - Memory Allocation, 'Fat Objects'
 
 We change our memory allocation for all memory the scheduler requires