las3_pub / predictable_parallel_patterns

We still see very sporadic crashes, however the current version is at least a starting point for refactoring and debugging. Next steps have to be to re-enable tooling support (i.e. add code to let sanitizers do their work).

The current version has race conditions and is hard to debug (especially because of the fibers, if a wrong thread executes on a fiber we get segfalts very fast). To combat this mess we now refactor the code bit by bit while also adding tests where it can be done with reasonably effort).

The project is currently really messy and there are sporadic sigsevs. This indicates that we still have a race in our code. Thread Sanitizer does not work with our current implementation, as it needs annotations for fibers.

The next step is to clean up the project and maybe add thread sanitizer support to our fiber implementation. This should help finding the remaining bugs.

The deque trades tasks when stealing. Right now only the fast local path is tested and implemented. For the next step to work we also need to add the resource stack and resource tarding to the system.

The current state shows the minimum actions taken to execute a parallel call: get the thread local, find the active frame, execute on the next frame and return to the active frame.

The idea is to exclude as many sources as possible that could lead to issues with contention and cache misses. After some experimentation, we think that hyperthreading is simply not working very well with our kind of workload. In the future we might simply test on other hardware.

It is still not working, however we now have no more redundant code, making debugging it simpler.

Everything so far is untested. We only made sure tha fast path still seems to function correctly. Next up is writing tests for both the fast and slow path to then introduce the slow path. After that we can look at performance optimizations.

This showcases the expected performance when a task executes a sub-tree without inference from other threads. We target to stay about 6x slower than a normal function call.

This first sketch of the classes captures what we think is needed in terms of general interface and very mich WIP.

We changed how the memory is allocated from passing char* buffers to then store objects into to creating 'fat objects' for all scheduler state. This eases development for us, as we can make changes to data structures without too much effort (e.g. add a second array to manage tasks if required).

Our stack is not calling deconstructors of its elements. This is problematic for e.g. the graph implementation where reference counted images are hold in tasks. To solve this for now we manually call the deconstructor after each tasks (we do so, because a generic, virtual deconstructor adds runtime costs to primitive tasks, requiring us to re-run all benchmarks; with this change we do not need to do this and as we re-work the scheduler anyways we postpone a clean implementation for then).

The scheduler yields if it failed to steal any work due to the task
lists being empty. This should improve performance on multiprogrammed
systems, as it potentially makes room for other worker threads which
still have work to perform.

This allows the stack and deque class to use the same offset, making it work better with each other.

Threads must be joined on scheduler termination and tasks must be pushed onto the stack to allow better memory management.

This seems to be only possible with two different function names, as variadic arguments and default arguments wont work together.

Please note that this does currently not execute, but only removed the 'old aspecs'. We are still missing launching the root fork_join task.