Commit 02888721 by FritzFlorian

Finalize banana pi setup notes.

parent 4c6e770f
Pipeline #1436 passed with stages
in 3 minutes 26 seconds
...@@ -90,7 +90,7 @@ Here are some good articles discussing jitter on linux systems: ...@@ -90,7 +90,7 @@ Here are some good articles discussing jitter on linux systems:
We use the following settings: We use the following settings:
```shell script ```shell script
mce=ignore_ce nosoftlockup nmi_watchdog=0 transparent_hugepage=never processor.max_cstate=1 idle=poll nohz=on nohz_full=1-7 isolcpus=1-7 mce=ignore_ce nosoftlockup nmi_watchdog=0 transparent_hugepage=never processor.max_cstate=1 idle=poll nohz=on nohz_full=1-7
``` ```
- ***mce=ignore_ce*** do not scan for hw errors. Reduce the jitter introduced by periodic runs - ***mce=ignore_ce*** do not scan for hw errors. Reduce the jitter introduced by periodic runs
...@@ -99,15 +99,13 @@ mce=ignore_ce nosoftlockup nmi_watchdog=0 transparent_hugepage=never processor.m ...@@ -99,15 +99,13 @@ mce=ignore_ce nosoftlockup nmi_watchdog=0 transparent_hugepage=never processor.m
- ***transparent_hugepage=never*** do not scan for small pages to combine to hugepages. We have no issues with memory usage, spare us of this periodic jitter. - ***transparent_hugepage=never*** do not scan for small pages to combine to hugepages. We have no issues with memory usage, spare us of this periodic jitter.
- ***processor.max_cstate=1 idle=poll*** do not switch to CPU power saving modes (c-states). Just run all cores at full speed all the time (we do not care about energy during our tests). - ***processor.max_cstate=1 idle=poll*** do not switch to CPU power saving modes (c-states). Just run all cores at full speed all the time (we do not care about energy during our tests).
- ***nohz=on nohz_full=1-7*** disable houskeeping os ticks on our isolated benchmark cores. core 0 will handle these when needed. - ***nohz=on nohz_full=1-7*** disable houskeeping os ticks on our isolated benchmark cores. core 0 will handle these when needed.
- ***isolcpus=1-7*** ignores cores 1 to 7 for scheduling processes (expect system processes that are bound to cores).
### Pin all other processes to core 0 (crgoups) ### Pin all other processes to core 0 (crgoups)
We want to isolate our measurements to cores 1 to 7 and use core 0 for all non benchmark related processes. We want to isolate our measurements to cores 1 to 7 and use core 0 for all non benchmark related processes.
isolcpus does some of this during startup (TODO: see if there are issues with task migration if we use isolcpus), isolcpus is often used for this, however, we found that it disables the scheduler from balancing tasks
however, we need to configure the system to schedule our threads specifically onto the isolated cores. between the isolated cores. A better approach is to use cgroups.
See the tutorial for further information: https://github.com/lpechacek/cpuset/blob/master/doc/tutorial.txt
cgroups is a tool well suited for this. See the tutorial for further information: https://github.com/lpechacek/cpuset/blob/master/doc/tutorial.txt
Essentially, we can partition our cores into two isolated groups, then map all tasks that can be moved away from Essentially, we can partition our cores into two isolated groups, then map all tasks that can be moved away from
our benchmark cores, to ensure low influence of background tasks. Cgroups also nicely interact with our benchmark cores, to ensure low influence of background tasks. Cgroups also nicely interact with
the real time scheduler, as described here https://www.linuxjournal.com/article/10165, because the real time scheduler, as described here https://www.linuxjournal.com/article/10165, because
...@@ -196,6 +194,7 @@ echo "Some might fail because they can not be mapped (e.g. core specific timers ...@@ -196,6 +194,7 @@ echo "Some might fail because they can not be mapped (e.g. core specific timers
echo "" echo ""
echo "" echo ""
echo 1 > /proc/irq/default_smp_affinity
for dir in /proc/irq/*/ for dir in /proc/irq/*/
do do
echo "Mapping $dir ..." echo "Mapping $dir ..."
......
...@@ -28,7 +28,7 @@ software components with arch rolling releases very hard. Even with tinkering we ...@@ -28,7 +28,7 @@ software components with arch rolling releases very hard. Even with tinkering we
to update the system without the kernel and without boot issues. to update the system without the kernel and without boot issues.
Following the above discussion we see that armbian is easy to setup, but has no prebuild rt image. Following the above discussion we see that armbian is easy to setup, but has no prebuild rt image.
Because of this we decide to use the armbian build script and add the rt patch to it, giving us the best Because of this, we decide to use the armbian build script and add the rt patch to it, giving us the best
of both worlds. of both worlds.
Full setup documentation for the os and how to further tune isolated performance can be found in a Full setup documentation for the os and how to further tune isolated performance can be found in a
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment