From 73264cdac568475c04d00eea307947b77f26f40c Mon Sep 17 00:00:00 2001 From: FritzFlorian Date: Mon, 24 Jun 2019 13:42:50 +0200 Subject: [PATCH] Add notes on our current measurement setup. --- NOTES.md | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++ media/2ad04ce5_fft_boxplot_isolated.png | Bin 0 -> 72907 bytes media/2ad04ce5_fft_boxplot_multiprogrammed.png | Bin 0 -> 77520 bytes media/2ad04ce5_matrix_boxplot_isolated.png | Bin 0 -> 77300 bytes media/2ad04ce5_matrix_boxplot_multiprogrammed.png | Bin 0 -> 73050 bytes 5 files changed, 53 insertions(+) create mode 100644 media/2ad04ce5_fft_boxplot_isolated.png create mode 100644 media/2ad04ce5_fft_boxplot_multiprogrammed.png create mode 100644 media/2ad04ce5_matrix_boxplot_isolated.png create mode 100644 media/2ad04ce5_matrix_boxplot_multiprogrammed.png diff --git a/NOTES.md b/NOTES.md index 96e97ce..ef4845c 100644 --- a/NOTES.md +++ b/NOTES.md @@ -4,6 +4,59 @@ A collection of stuff that we noticed during development. Useful later on two write a project report and to go back in time to find out why certain decisions where made. +## 24.06.2019 - Further Benchmark Settings and First Results + +As a further option to reduce potential inference of our benchmark +process we add the `CPUAffinity` option to systemd, setting the +CPU affinity of all processes started by the startup procedure +to core 0, thus isolating cores 1 to 7 for our benchmarks. +For docs on this see [the systemd manual](https://www.freedesktop.org/software/systemd/man/systemd.exec.html). +(Please note that this is not prefect, it seems that a kernel parameter +would perform even better, but our current measurements show this +working well enough to not go that deep a long as we do not see the +need in further testing). + +We could potentially better our results further by using a preemtive +patch on a newer kernel, e.g. Arch linux with preemt patch. + +Our first results look promising, and show that our measurement +methodology can indeed work (measured on the banana PI with the +above settings). Backgroud tasks are started from core 1 to 7 +with 500us work, 5000us pause. Benchmarks are started pinned +to core 1 to 7: + +FFT without background tasks: + + + + +FFT with background tasks: + + + +Matrix without background tasks: + + + +Matrix with background tasks: + + + + +The plot's include all outliers, showing that the workst case times can +be measured and compared under these circumstances. +One option we might change in the future is how/when the background +preempting workers are executed, to better 'produce' bad situations +for the scheduler, e.g. have a higher active time for the workers +(close to maybe 1/4 of the benchmark times), to actually fully +remove one worker from a few tasks (we would like to do that with +our tests on different queue types, to e.g. see if the locknig queue +actualy is worse than lock free implementations by blocking up other +tasks). + + + + ## 14.06.2019 - Benchmark Settings/Real Time Execution on Linux Our set goal of this project is predictable execution times. diff --git a/media/2ad04ce5_fft_boxplot_isolated.png b/media/2ad04ce5_fft_boxplot_isolated.png new file mode 100644 index 0000000..53a10e6 Binary files /dev/null and b/media/2ad04ce5_fft_boxplot_isolated.png differ diff --git a/media/2ad04ce5_fft_boxplot_multiprogrammed.png b/media/2ad04ce5_fft_boxplot_multiprogrammed.png new file mode 100644 index 0000000..3b62236 Binary files /dev/null and b/media/2ad04ce5_fft_boxplot_multiprogrammed.png differ diff --git a/media/2ad04ce5_matrix_boxplot_isolated.png b/media/2ad04ce5_matrix_boxplot_isolated.png new file mode 100644 index 0000000..5ea9653 Binary files /dev/null and b/media/2ad04ce5_matrix_boxplot_isolated.png differ diff --git a/media/2ad04ce5_matrix_boxplot_multiprogrammed.png b/media/2ad04ce5_matrix_boxplot_multiprogrammed.png new file mode 100644 index 0000000..9552893 Binary files /dev/null and b/media/2ad04ce5_matrix_boxplot_multiprogrammed.png differ -- libgit2 0.26.0