rarias/ovni
rarias/ovni
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Add more documentation

Browse Source
This commit is contained in:
Rodrigo Arias 2022-08-31 11:31:55 +02:00
parent 135f3dc7aa
commit 2679655b2a
30 changed files with 1042 additions and 84 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

31
doc/concepts.md Normal file
View File

@ -0,0 +1,31 @@
# Overview
The objective of the ovni project is to provide insight into what
happened at execution of a program.
![Instrumentation process](fig/instrumentation.svg)
The key pieces of software involved are instrumented so they emit events
during the execution which allow the reconstruction of the execution
later on.
During the execution phase, the information gathered in the events is
kept very short and simple, so the overhead is kept at minimum to avoid
disturbing the execution process. Here is an example of a single event
emitted during the execution phase, informing the current thread to
finish the execution:
00 4f 48 65 52 c0 27 b4 d3 ec 01 00
During the emulation phase, the events are read and processed in the
emulator, reconstructing the execution. State transitions are recorded
in a Paraver trace. Here is an example of the same thread ceasing the
execution:
2:0:1:1:1:50105669:1:0
Finally, loading the trace in the Paraver program, we can generate a
timeline visualization of the state change. Here is the example for the
same state transition of the thread stopping the execution:
![Visualization](fig/visualization.png)

62
doc/developer-guide/index.md Normal file
View File

@ -0,0 +1,62 @@
# Developer guide
To contribute to the ovni project, please read carefully this guide and
follow the guidelines to have your contribution accepted in the project.
## First principle: keep things simple
The most important principle in the emulator is to keep things simple.
It is extremely important to avoid any complexities, including
sacrificing the performance if necessary to keep the design simple.
## Use a formal model
Design a model of the library or runtime that you want to instrument and
all posible state transitions to be checked by the emulator. Detect
as many error conditions as posible. Otherwise, it becomes very hard to
analyze millions of state transitions by the naked eye to determine if
something went wrong.
## Emit redundant information
At runtime, emit redundant information in the events, unless they affect
the performance very badly. This information may include the ID of the
current task or any other identification, which the emulator already
knows. This redundant information is the only bit of data the emulator
can use to perform consistency checks and ensure the emulated state
follows the runtime state.
The extra information also allows a human to read the binary trace and
understand what is happening without the need to keep track of the
emulation state all the time.
## Document your model
Explain your model and the concepts behind it, so a user can understand
how to interpret the information provided by the timeline views.
The emulator only tries to represent the execution, but is never
perfect. Sometimes it is difficult or impossible to instrument what
happened at runtime, so an approximation is the only posible approach.
Explain in the documentation how the model approximates the real
execution so the user can determine what information is only an
approximation.
## No dependencies
Don't pull any extra dependencies. If you need them, you are probably
violating the first principle (keeping things simple). Alternatively,
place a portable implementation in ovni, so we don't need to depend on
any other library when running the emulator.
## Testing
The ovni project has two types of tests: manual tests, which
artificially create the events in the trace, and only test the emulation
part. And runtime tests, which run the complete program with the
instrumentation enabled, generate a trace and then run the emulator.
When adding support for a new library or model, ensure you add enough
tests to cover the most important problems. Specially those hard to
detect without the current runtime state. See the `test/` directory for
examples of other models.

5
doc/emulation/events.md
View File

@ -161,8 +161,9 @@ KCI Is back in the CPU due to a context switch
6Ha Attaches to Nanos6 as external thread 6Ha Attaches to Nanos6 as external thread
6HA Detaches from Nanos6 as external thread 6HA Detaches from Nanos6 as external thread
6Hs Begins the spawn of a function
6HS Ends the spawn of a function 6s[ Begins to spawn a function via spawnFunction()
6s] Ends spawning a function
6Dr Begins the registration of a task's accesses 6Dr Begins the registration of a task's accesses
6DR Ends the registration of a task's accesses 6DR Ends the registration of a task's accesses

66
doc/emulation/fig/model.dot Normal file
View File

@ -0,0 +1,66 @@
digraph {
#rankdir=LR;
graph [ranksep="1"];
newrank=true;
subgraph cluster_resource {
color="#ffeeee";
style=filled;
label="Resouce model";
cluster -> {"node 0", "node 1"}
"node 0" -> {"cpu 0.0", "cpu 0.1"}
"node 1" -> {"cpu 1.0", "cpu 1.1"}
}
subgraph cluster_process {
color="#eeffee";
style=filled;
label="Process model"
genesis -> {"loom 0", "loom 1"}
"loom 0" -> {"pid 0.0", "pid 0.1"}
"loom 1" -> {"pid 1.0", "pid 1.1"}
"pid 0.0" -> {"th 0.0.0", "th 0.0.1"}
"pid 0.1" -> {"th 0.1.0", "th 0.1.1"}
}
subgraph cluster_task {
color="#eeeeff";
style=filled;
label="Task model";
workload -> { "app 0", "app 1"};
"app 0" -> {"rank 0.0", "rank 0.1"};
"app 1" -> {"rank 1.0", "rank 1.1"};
"rank 0.0" -> {"task 0.0.0","task 0.0.1"};
"rank 0.1" -> {"task 0.1.0", "task 0.1.1"};
}
{rank = same; workload; cluster; genesis;}
{
rank = same;
"node 0"; "node 1"; "loom 0"; "loom 1";
}
{ rank=same; "app 0"; "app 1"; }
{
rank = same; "rank 0.0"; "pid 0.0";
}
{
rank = same; "th 0.0.0"; "task 0.0.0";
"cpu 0.0"; "cpu 0.1"; "cpu 1.0"; "cpu 1.1";
}
edge [style=dashed, constraint=false];
"loom 0" -> "node 0" [color=blue, label="mapped to"];
"rank 0.0" -> "pid 0.0" [color=blue, label="mapped to"];
"task 0.0.0" -> "th 0.0.0" [color=red, label="runs at"];
"th 0.0.0" -> "cpu 0.0" [color=red, label="runs at"];
}

0
doc/emulation/model.png → doc/emulation/fig/model.png
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 158 KiB

After

Width:  |  Height:  |  Size: 158 KiB

385
doc/emulation/fig/model.svg Normal file
View File

@ -0,0 +1,385 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Generated by graphviz version 5.0.0 (0)
-->
<!-- Pages: 1 -->
<svg width="1439pt" height="491pt"
viewBox="0.00 0.00 1439.00 491.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 487)">
<polygon fill="white" stroke="transparent" points="-4,4 -4,-487 1435,-487 1435,4 -4,4"/>
<g id="clust1" class="cluster">
<title>cluster_resource</title>
<polygon fill="#ffeeee" stroke="#ffeeee" points="8,-8 8,-475 438,-475 438,-8 8,-8"/>
<text text-anchor="middle" x="223" y="-459.8" font-family="Times,serif" font-size="14.00">Resouce model</text>
</g>
<g id="clust5" class="cluster">
<title>cluster_process</title>
<polygon fill="#eeffee" stroke="#eeffee" points="446,-8 446,-475 892,-475 892,-8 446,-8"/>
<text text-anchor="middle" x="669" y="-459.8" font-family="Times,serif" font-size="14.00">Process model</text>
</g>
<g id="clust11" class="cluster">
<title>cluster_task</title>
<polygon fill="#eeeeff" stroke="#eeeeff" points="900,-8 900,-475 1423,-475 1423,-8 900,-8"/>
<text text-anchor="middle" x="1161.5" y="-459.8" font-family="Times,serif" font-size="14.00">Task model</text>
</g>
<!-- cluster -->
<g id="node1" class="node">
<title>cluster</title>
<ellipse fill="none" stroke="black" cx="223" cy="-403" rx="42.79" ry="18"/>
<text text-anchor="middle" x="223" y="-399.3" font-family="Times,serif" font-size="14.00">cluster</text>
</g>
<!-- node 0 -->
<g id="node2" class="node">
<title>node 0</title>
<ellipse fill="none" stroke="black" cx="169" cy="-280" rx="42.49" ry="18"/>
<text text-anchor="middle" x="169" y="-276.3" font-family="Times,serif" font-size="14.00">node 0</text>
</g>
<!-- cluster&#45;&gt;node 0 -->
<g id="edge1" class="edge">
<title>cluster&#45;&gt;node 0</title>
<path fill="none" stroke="black" d="M215.49,-385.18C206.51,-365.06 191.38,-331.16 180.83,-307.51"/>
<polygon fill="black" stroke="black" points="183.89,-305.78 176.62,-298.07 177.5,-308.63 183.89,-305.78"/>
</g>
<!-- node 1 -->
<g id="node3" class="node">
<title>node 1</title>
<ellipse fill="none" stroke="black" cx="277" cy="-280" rx="42.49" ry="18"/>
<text text-anchor="middle" x="277" y="-276.3" font-family="Times,serif" font-size="14.00">node 1</text>
</g>
<!-- cluster&#45;&gt;node 1 -->
<g id="edge2" class="edge">
<title>cluster&#45;&gt;node 1</title>
<path fill="none" stroke="black" d="M230.51,-385.18C239.49,-365.06 254.62,-331.16 265.17,-307.51"/>
<polygon fill="black" stroke="black" points="268.5,-308.63 269.38,-298.07 262.11,-305.78 268.5,-308.63"/>
</g>
<!-- cpu 0.0 -->
<g id="node4" class="node">
<title>cpu 0.0</title>
<ellipse fill="none" stroke="black" cx="61" cy="-34" rx="44.69" ry="18"/>
<text text-anchor="middle" x="61" y="-30.3" font-family="Times,serif" font-size="14.00">cpu 0.0</text>
</g>
<!-- node 0&#45;&gt;cpu 0.0 -->
<g id="edge3" class="edge">
<title>node 0&#45;&gt;cpu 0.0</title>
<path fill="none" stroke="black" d="M161.58,-262.23C143.08,-220.44 94.83,-111.43 72.57,-61.13"/>
<polygon fill="black" stroke="black" points="75.72,-59.62 68.48,-51.89 69.32,-62.45 75.72,-59.62"/>
</g>
<!-- cpu 0.1 -->
<g id="node5" class="node">
<title>cpu 0.1</title>
<ellipse fill="none" stroke="black" cx="169" cy="-34" rx="44.69" ry="18"/>
<text text-anchor="middle" x="169" y="-30.3" font-family="Times,serif" font-size="14.00">cpu 0.1</text>
</g>
<!-- node 0&#45;&gt;cpu 0.1 -->
<g id="edge4" class="edge">
<title>node 0&#45;&gt;cpu 0.1</title>
<path fill="none" stroke="black" d="M169,-261.84C169,-220.1 169,-112.99 169,-62.33"/>
<polygon fill="black" stroke="black" points="172.5,-62.14 169,-52.14 165.5,-62.14 172.5,-62.14"/>
</g>
<!-- cpu 1.0 -->
<g id="node6" class="node">
<title>cpu 1.0</title>
<ellipse fill="none" stroke="black" cx="277" cy="-34" rx="44.69" ry="18"/>
<text text-anchor="middle" x="277" y="-30.3" font-family="Times,serif" font-size="14.00">cpu 1.0</text>
</g>
<!-- node 1&#45;&gt;cpu 1.0 -->
<g id="edge5" class="edge">
<title>node 1&#45;&gt;cpu 1.0</title>
<path fill="none" stroke="black" d="M277,-261.84C277,-220.1 277,-112.99 277,-62.33"/>
<polygon fill="black" stroke="black" points="280.5,-62.14 277,-52.14 273.5,-62.14 280.5,-62.14"/>
</g>
<!-- cpu 1.1 -->
<g id="node7" class="node">
<title>cpu 1.1</title>
<ellipse fill="none" stroke="black" cx="385" cy="-34" rx="44.69" ry="18"/>
<text text-anchor="middle" x="385" y="-30.3" font-family="Times,serif" font-size="14.00">cpu 1.1</text>
</g>
<!-- node 1&#45;&gt;cpu 1.1 -->
<g id="edge6" class="edge">
<title>node 1&#45;&gt;cpu 1.1</title>
<path fill="none" stroke="black" d="M284.15,-262.06C298.32,-228.63 331.06,-151.89 360,-88 363.97,-79.25 368.41,-69.74 372.44,-61.22"/>
<polygon fill="black" stroke="black" points="375.63,-62.66 376.76,-52.12 369.31,-59.65 375.63,-62.66"/>
</g>
<!-- genesis -->
<g id="node8" class="node">
<title>genesis</title>
<ellipse fill="none" stroke="black" cx="685" cy="-403" rx="46.29" ry="18"/>
<text text-anchor="middle" x="685" y="-399.3" font-family="Times,serif" font-size="14.00">genesis</text>
</g>
<!-- loom 0 -->
<g id="node9" class="node">
<title>loom 0</title>
<ellipse fill="none" stroke="black" cx="633" cy="-280" rx="42.49" ry="18"/>
<text text-anchor="middle" x="633" y="-276.3" font-family="Times,serif" font-size="14.00">loom 0</text>
</g>
<!-- genesis&#45;&gt;loom 0 -->
<g id="edge7" class="edge">
<title>genesis&#45;&gt;loom 0</title>
<path fill="none" stroke="black" d="M677.77,-385.18C669.13,-365.06 654.55,-331.16 644.39,-307.51"/>
<polygon fill="black" stroke="black" points="647.5,-305.88 640.34,-298.07 641.07,-308.64 647.5,-305.88"/>
</g>
<!-- loom 1 -->
<g id="node10" class="node">
<title>loom 1</title>
<ellipse fill="none" stroke="black" cx="737" cy="-280" rx="42.49" ry="18"/>
<text text-anchor="middle" x="737" y="-276.3" font-family="Times,serif" font-size="14.00">loom 1</text>
</g>
<!-- genesis&#45;&gt;loom 1 -->
<g id="edge8" class="edge">
<title>genesis&#45;&gt;loom 1</title>
<path fill="none" stroke="black" d="M692.23,-385.18C700.87,-365.06 715.45,-331.16 725.61,-307.51"/>
<polygon fill="black" stroke="black" points="728.93,-308.64 729.66,-298.07 722.5,-305.88 728.93,-308.64"/>
</g>
<!-- loom 0&#45;&gt;node 0 -->
<g id="edge27" class="edge">
<title>loom 0&#45;&gt;node 0</title>
<path fill="none" stroke="blue" stroke-dasharray="5,2" d="M602.07,-292.52C564.32,-307.07 498.2,-330.2 439,-339 355.62,-351.4 259.53,-319 207.44,-297.44"/>
<polygon fill="blue" stroke="blue" points="208.5,-294.09 197.93,-293.42 205.78,-300.54 208.5,-294.09"/>
<text text-anchor="middle" x="401" y="-337.8" font-family="Times,serif" font-size="14.00">mapped to</text>
</g>
<!-- pid 0.0 -->
<g id="node11" class="node">
<title>pid 0.0</title>
<ellipse fill="none" stroke="black" cx="529" cy="-157" rx="42.79" ry="18"/>
<text text-anchor="middle" x="529" y="-153.3" font-family="Times,serif" font-size="14.00">pid 0.0</text>
</g>
<!-- loom 0&#45;&gt;pid 0.0 -->
<g id="edge9" class="edge">
<title>loom 0&#45;&gt;pid 0.0</title>
<path fill="none" stroke="black" d="M618.97,-262.68C601.04,-241.82 569.86,-205.54 549.33,-181.65"/>
<polygon fill="black" stroke="black" points="551.98,-179.36 542.8,-174.06 546.67,-183.93 551.98,-179.36"/>
</g>
<!-- pid 0.1 -->
<g id="node12" class="node">
<title>pid 0.1</title>
<ellipse fill="none" stroke="black" cx="633" cy="-157" rx="42.79" ry="18"/>
<text text-anchor="middle" x="633" y="-153.3" font-family="Times,serif" font-size="14.00">pid 0.1</text>
</g>
<!-- loom 0&#45;&gt;pid 0.1 -->
<g id="edge10" class="edge">
<title>loom 0&#45;&gt;pid 0.1</title>
<path fill="none" stroke="black" d="M633,-261.92C633,-242.04 633,-208.92 633,-185.42"/>
<polygon fill="black" stroke="black" points="636.5,-185.38 633,-175.38 629.5,-185.38 636.5,-185.38"/>
</g>
<!-- pid 1.0 -->
<g id="node13" class="node">
<title>pid 1.0</title>
<ellipse fill="none" stroke="black" cx="737" cy="-157" rx="42.79" ry="18"/>
<text text-anchor="middle" x="737" y="-153.3" font-family="Times,serif" font-size="14.00">pid 1.0</text>
</g>
<!-- loom 1&#45;&gt;pid 1.0 -->
<g id="edge11" class="edge">
<title>loom 1&#45;&gt;pid 1.0</title>
<path fill="none" stroke="black" d="M737,-261.92C737,-242.04 737,-208.92 737,-185.42"/>
<polygon fill="black" stroke="black" points="740.5,-185.38 737,-175.38 733.5,-185.38 740.5,-185.38"/>
</g>
<!-- pid 1.1 -->
<g id="node14" class="node">
<title>pid 1.1</title>
<ellipse fill="none" stroke="black" cx="841" cy="-157" rx="42.79" ry="18"/>
<text text-anchor="middle" x="841" y="-153.3" font-family="Times,serif" font-size="14.00">pid 1.1</text>
</g>
<!-- loom 1&#45;&gt;pid 1.1 -->
<g id="edge12" class="edge">
<title>loom 1&#45;&gt;pid 1.1</title>
<path fill="none" stroke="black" d="M751.03,-262.68C768.96,-241.82 800.14,-205.54 820.67,-181.65"/>
<polygon fill="black" stroke="black" points="823.33,-183.93 827.2,-174.06 818.02,-179.36 823.33,-183.93"/>
</g>
<!-- th 0.0.0 -->
<g id="node15" class="node">
<title>th 0.0.0</title>
<ellipse fill="none" stroke="black" cx="501" cy="-34" rx="46.59" ry="18"/>
<text text-anchor="middle" x="501" y="-30.3" font-family="Times,serif" font-size="14.00">th 0.0.0</text>
</g>
<!-- pid 0.0&#45;&gt;th 0.0.0 -->
<g id="edge13" class="edge">
<title>pid 0.0&#45;&gt;th 0.0.0</title>
<path fill="none" stroke="black" d="M525.05,-138.92C520.42,-118.92 512.7,-85.55 507.25,-62.02"/>
<polygon fill="black" stroke="black" points="510.6,-60.95 504.93,-51.99 503.78,-62.52 510.6,-60.95"/>
</g>
<!-- th 0.0.1 -->
<g id="node16" class="node">
<title>th 0.0.1</title>
<ellipse fill="none" stroke="black" cx="613" cy="-34" rx="46.59" ry="18"/>
<text text-anchor="middle" x="613" y="-30.3" font-family="Times,serif" font-size="14.00">th 0.0.1</text>
</g>
<!-- pid 0.0&#45;&gt;th 0.0.1 -->
<g id="edge14" class="edge">
<title>pid 0.0&#45;&gt;th 0.0.1</title>
<path fill="none" stroke="black" d="M540.68,-139.18C554.91,-118.68 579.08,-83.85 595.53,-60.16"/>
<polygon fill="black" stroke="black" points="598.59,-61.89 601.42,-51.68 592.84,-57.9 598.59,-61.89"/>
</g>
<!-- th 0.1.0 -->
<g id="node17" class="node">
<title>th 0.1.0</title>
<ellipse fill="none" stroke="black" cx="725" cy="-34" rx="46.59" ry="18"/>
<text text-anchor="middle" x="725" y="-30.3" font-family="Times,serif" font-size="14.00">th 0.1.0</text>
</g>
<!-- pid 0.1&#45;&gt;th 0.1.0 -->
<g id="edge15" class="edge">
<title>pid 0.1&#45;&gt;th 0.1.0</title>
<path fill="none" stroke="black" d="M645.41,-139.68C661.05,-119.11 688.07,-83.57 706.24,-59.67"/>
<polygon fill="black" stroke="black" points="709.23,-61.53 712.5,-51.45 703.66,-57.29 709.23,-61.53"/>
</g>
<!-- th 0.1.1 -->
<g id="node18" class="node">
<title>th 0.1.1</title>
<ellipse fill="none" stroke="black" cx="837" cy="-34" rx="46.59" ry="18"/>
<text text-anchor="middle" x="837" y="-30.3" font-family="Times,serif" font-size="14.00">th 0.1.1</text>
</g>
<!-- pid 0.1&#45;&gt;th 0.1.1 -->
<g id="edge16" class="edge">
<title>pid 0.1&#45;&gt;th 0.1.1</title>
<path fill="none" stroke="black" d="M656.8,-141.88C693.15,-120.32 762.61,-79.12 803.87,-54.65"/>
<polygon fill="black" stroke="black" points="805.9,-57.51 812.72,-49.4 802.33,-51.49 805.9,-57.51"/>
</g>
<!-- th 0.0.0&#45;&gt;cpu 0.0 -->
<g id="edge30" class="edge">
<title>th 0.0.0&#45;&gt;cpu 0.0</title>
<path fill="none" stroke="red" stroke-dasharray="5,2" d="M484.26,-50.97C468.31,-65.83 442.8,-86.38 416,-95 351.68,-115.69 180.53,-68.48 115,-52 111.44,-51.1 107.77,-50.09 104.1,-49.01"/>
<polygon fill="red" stroke="red" points="105.06,-45.64 94.47,-46.03 102.99,-52.33 105.06,-45.64"/>
<text text-anchor="middle" x="390" y="-91.8" font-family="Times,serif" font-size="14.00">runs at</text>
</g>
<!-- workload -->
<g id="node19" class="node">
<title>workload</title>
<ellipse fill="none" stroke="black" cx="1190" cy="-403" rx="53.09" ry="18"/>
<text text-anchor="middle" x="1190" y="-399.3" font-family="Times,serif" font-size="14.00">workload</text>
</g>
<!-- app 0 -->
<g id="node20" class="node">
<title>app 0</title>
<ellipse fill="none" stroke="black" cx="1132" cy="-280" rx="36.29" ry="18"/>
<text text-anchor="middle" x="1132" y="-276.3" font-family="Times,serif" font-size="14.00">app 0</text>
</g>
<!-- workload&#45;&gt;app 0 -->
<g id="edge17" class="edge">
<title>workload&#45;&gt;app 0</title>
<path fill="none" stroke="black" d="M1181.94,-385.18C1172.2,-364.86 1155.71,-330.47 1144.36,-306.79"/>
<polygon fill="black" stroke="black" points="1147.47,-305.19 1140,-297.68 1141.16,-308.21 1147.47,-305.19"/>
</g>
<!-- app 1 -->
<g id="node21" class="node">
<title>app 1</title>
<ellipse fill="none" stroke="black" cx="1249" cy="-280" rx="36.29" ry="18"/>
<text text-anchor="middle" x="1249" y="-276.3" font-family="Times,serif" font-size="14.00">app 1</text>
</g>
<!-- workload&#45;&gt;app 1 -->
<g id="edge18" class="edge">
<title>workload&#45;&gt;app 1</title>
<path fill="none" stroke="black" d="M1198.2,-385.18C1208.11,-364.86 1224.88,-330.47 1236.42,-306.79"/>
<polygon fill="black" stroke="black" points="1239.63,-308.2 1240.87,-297.68 1233.34,-305.14 1239.63,-308.2"/>
</g>
<!-- rank 0.0 -->
<g id="node22" class="node">
<title>rank 0.0</title>
<ellipse fill="none" stroke="black" cx="1015" cy="-157" rx="49.29" ry="18"/>
<text text-anchor="middle" x="1015" y="-153.3" font-family="Times,serif" font-size="14.00">rank 0.0</text>
</g>
<!-- app 0&#45;&gt;rank 0.0 -->
<g id="edge19" class="edge">
<title>app 0&#45;&gt;rank 0.0</title>
<path fill="none" stroke="black" d="M1116.7,-263.18C1096.58,-242.37 1061.05,-205.63 1037.77,-181.55"/>
<polygon fill="black" stroke="black" points="1040.14,-178.96 1030.67,-174.21 1035.11,-183.83 1040.14,-178.96"/>
</g>
<!-- rank 0.1 -->
<g id="node23" class="node">
<title>rank 0.1</title>
<ellipse fill="none" stroke="black" cx="1132" cy="-157" rx="49.29" ry="18"/>
<text text-anchor="middle" x="1132" y="-153.3" font-family="Times,serif" font-size="14.00">rank 0.1</text>
</g>
<!-- app 0&#45;&gt;rank 0.1 -->
<g id="edge20" class="edge">
<title>app 0&#45;&gt;rank 0.1</title>
<path fill="none" stroke="black" d="M1132,-261.92C1132,-242.04 1132,-208.92 1132,-185.42"/>
<polygon fill="black" stroke="black" points="1135.5,-185.38 1132,-175.38 1128.5,-185.38 1135.5,-185.38"/>
</g>
<!-- rank 1.0 -->
<g id="node24" class="node">
<title>rank 1.0</title>
<ellipse fill="none" stroke="black" cx="1249" cy="-157" rx="49.29" ry="18"/>
<text text-anchor="middle" x="1249" y="-153.3" font-family="Times,serif" font-size="14.00">rank 1.0</text>
</g>
<!-- app 1&#45;&gt;rank 1.0 -->
<g id="edge21" class="edge">
<title>app 1&#45;&gt;rank 1.0</title>
<path fill="none" stroke="black" d="M1249,-261.92C1249,-242.04 1249,-208.92 1249,-185.42"/>
<polygon fill="black" stroke="black" points="1252.5,-185.38 1249,-175.38 1245.5,-185.38 1252.5,-185.38"/>
</g>
<!-- rank 1.1 -->
<g id="node25" class="node">
<title>rank 1.1</title>
<ellipse fill="none" stroke="black" cx="1366" cy="-157" rx="49.29" ry="18"/>
<text text-anchor="middle" x="1366" y="-153.3" font-family="Times,serif" font-size="14.00">rank 1.1</text>
</g>
<!-- app 1&#45;&gt;rank 1.1 -->
<g id="edge22" class="edge">
<title>app 1&#45;&gt;rank 1.1</title>
<path fill="none" stroke="black" d="M1264.3,-263.18C1284.42,-242.37 1319.95,-205.63 1343.23,-181.55"/>
<polygon fill="black" stroke="black" points="1345.89,-183.83 1350.33,-174.21 1340.86,-178.96 1345.89,-183.83"/>
</g>
<!-- rank 0.0&#45;&gt;pid 0.0 -->
<g id="edge28" class="edge">
<title>rank 0.0&#45;&gt;pid 0.0</title>
<path fill="none" stroke="blue" stroke-dasharray="5,2" d="M982.2,-170.78C917.51,-196.09 769.17,-244.84 647,-216 615.71,-208.61 583.26,-191.59 560.32,-177.72"/>
<polygon fill="blue" stroke="blue" points="562.06,-174.68 551.72,-172.39 558.38,-180.63 562.06,-174.68"/>
<text text-anchor="middle" x="685" y="-214.8" font-family="Times,serif" font-size="14.00">mapped to</text>
</g>
<!-- task 0.0.0 -->
<g id="node26" class="node">
<title>task 0.0.0</title>
<ellipse fill="none" stroke="black" cx="965" cy="-34" rx="56.59" ry="18"/>
<text text-anchor="middle" x="965" y="-30.3" font-family="Times,serif" font-size="14.00">task 0.0.0</text>
</g>
<!-- rank 0.0&#45;&gt;task 0.0.0 -->
<g id="edge23" class="edge">
<title>rank 0.0&#45;&gt;task 0.0.0</title>
<path fill="none" stroke="black" d="M1008.05,-139.18C999.74,-119.06 985.73,-85.16 975.95,-61.51"/>
<polygon fill="black" stroke="black" points="979.11,-59.98 972.05,-52.07 972.64,-62.65 979.11,-59.98"/>
</g>
<!-- task 0.0.1 -->
<g id="node27" class="node">
<title>task 0.0.1</title>
<ellipse fill="none" stroke="black" cx="1096" cy="-34" rx="56.59" ry="18"/>
<text text-anchor="middle" x="1096" y="-30.3" font-family="Times,serif" font-size="14.00">task 0.0.1</text>
</g>
<!-- rank 0.0&#45;&gt;task 0.0.1 -->
<g id="edge24" class="edge">
<title>rank 0.0&#45;&gt;task 0.0.1</title>
<path fill="none" stroke="black" d="M1026.26,-139.18C1039.98,-118.68 1063.3,-83.85 1079.16,-60.16"/>
<polygon fill="black" stroke="black" points="1082.18,-61.94 1084.83,-51.68 1076.36,-58.04 1082.18,-61.94"/>
</g>
<!-- task 0.1.0 -->
<g id="node28" class="node">
<title>task 0.1.0</title>
<ellipse fill="none" stroke="black" cx="1227" cy="-34" rx="56.59" ry="18"/>
<text text-anchor="middle" x="1227" y="-30.3" font-family="Times,serif" font-size="14.00">task 0.1.0</text>
</g>
<!-- rank 0.1&#45;&gt;task 0.1.0 -->
<g id="edge25" class="edge">
<title>rank 0.1&#45;&gt;task 0.1.0</title>
<path fill="none" stroke="black" d="M1145.21,-139.18C1161.37,-118.59 1188.88,-83.55 1207.49,-59.85"/>
<polygon fill="black" stroke="black" points="1210.48,-61.71 1213.9,-51.68 1204.97,-57.39 1210.48,-61.71"/>
</g>
<!-- task 0.1.1 -->
<g id="node29" class="node">
<title>task 0.1.1</title>
<ellipse fill="none" stroke="black" cx="1358" cy="-34" rx="56.59" ry="18"/>
<text text-anchor="middle" x="1358" y="-30.3" font-family="Times,serif" font-size="14.00">task 0.1.1</text>
</g>
<!-- rank 0.1&#45;&gt;task 0.1.1 -->
<g id="edge26" class="edge">
<title>rank 0.1&#45;&gt;task 0.1.1</title>
<path fill="none" stroke="black" d="M1158.81,-141.64C1199.18,-120.03 1275.66,-79.08 1321.21,-54.7"/>
<polygon fill="black" stroke="black" points="1323,-57.71 1330.16,-49.9 1319.69,-51.54 1323,-57.71"/>
</g>
<!-- task 0.0.0&#45;&gt;th 0.0.0 -->
<g id="edge29" class="edge">
<title>task 0.0.0&#45;&gt;th 0.0.0</title>
<path fill="none" stroke="red" stroke-dasharray="5,2" d="M922.11,-45.88C837.96,-68.38 654.09,-113.41 593,-95 568.4,-87.59 544.34,-71.21 526.99,-57.29"/>
<polygon fill="red" stroke="red" points="528.98,-54.39 519.04,-50.7 524.51,-59.78 528.98,-54.39"/>
<text text-anchor="middle" x="619" y="-91.8" font-family="Times,serif" font-size="14.00">runs at</text>
</g>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 18 KiB

11
doc/emulation/fig/nanos6-task-model.dot Normal file
View File

@ -0,0 +1,11 @@
digraph {
rankdir=LR;
{ rank=same; Running; Paused; }
Unknown -> Created;
Created -> Running;
Running -> Paused;
Paused -> Running;
Running -> Dead;
}

72
doc/emulation/fig/nanos6-task-model.svg Normal file
View File

@ -0,0 +1,72 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Generated by graphviz version 5.0.0 (0)
-->
<!-- Pages: 1 -->
<svg width="490pt" height="98pt"
viewBox="0.00 0.00 490.37 98.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 94)">
<polygon fill="white" stroke="transparent" points="-4,4 -4,-94 486.37,-94 486.37,4 -4,4"/>
<!-- Running -->
<g id="node1" class="node">
<title>Running</title>
<ellipse fill="none" stroke="black" cx="327.43" cy="-18" rx="50.09" ry="18"/>
<text text-anchor="middle" x="327.43" y="-14.3" font-family="Times,serif" font-size="14.00">Running</text>
</g>
<!-- Paused -->
<g id="node2" class="node">
<title>Paused</title>
<ellipse fill="none" stroke="black" cx="327.43" cy="-72" rx="44.39" ry="18"/>
<text text-anchor="middle" x="327.43" y="-68.3" font-family="Times,serif" font-size="14.00">Paused</text>
</g>
<!-- Running&#45;&gt;Paused -->
<g id="edge3" class="edge">
<title>Running&#45;&gt;Paused</title>
<path fill="none" stroke="black" d="M356.99,-32.55C364.84,-38.68 366.89,-44.81 363.15,-50.94"/>
<polygon fill="black" stroke="black" points="360.59,-48.55 356.08,-58.13 365.59,-53.45 360.59,-48.55"/>
</g>
<!-- Dead -->
<g id="node5" class="node">
<title>Dead</title>
<ellipse fill="none" stroke="black" cx="447.92" cy="-18" rx="34.39" ry="18"/>
<text text-anchor="middle" x="447.92" y="-14.3" font-family="Times,serif" font-size="14.00">Dead</text>
</g>
<!-- Running&#45;&gt;Dead -->
<g id="edge5" class="edge">
<title>Running&#45;&gt;Dead</title>
<path fill="none" stroke="black" d="M377.65,-18C386.05,-18 394.74,-18 402.99,-18"/>
<polygon fill="black" stroke="black" points="403.17,-21.5 413.17,-18 403.17,-14.5 403.17,-21.5"/>
</g>
<!-- Paused&#45;&gt;Running -->
<g id="edge4" class="edge">
<title>Paused&#45;&gt;Running</title>
<path fill="none" stroke="black" d="M298.77,-58.13C290.67,-52.14 288.11,-46.14 291.11,-40.15"/>
<polygon fill="black" stroke="black" points="293.83,-42.35 297.87,-32.55 288.6,-37.69 293.83,-42.35"/>
</g>
<!-- Unknown -->
<g id="node3" class="node">
<title>Unknown</title>
<ellipse fill="none" stroke="black" cx="54.6" cy="-18" rx="54.69" ry="18"/>
<text text-anchor="middle" x="54.6" y="-14.3" font-family="Times,serif" font-size="14.00">Unknown</text>
</g>
<!-- Created -->
<g id="node4" class="node">
<title>Created</title>
<ellipse fill="none" stroke="black" cx="193.29" cy="-18" rx="48.19" ry="18"/>
<text text-anchor="middle" x="193.29" y="-14.3" font-family="Times,serif" font-size="14.00">Created</text>
</g>
<!-- Unknown&#45;&gt;Created -->
<g id="edge1" class="edge">
<title>Unknown&#45;&gt;Created</title>
<path fill="none" stroke="black" d="M109.23,-18C117.63,-18 126.34,-18 134.82,-18"/>
<polygon fill="black" stroke="black" points="135.02,-21.5 145.02,-18 135.02,-14.5 135.02,-21.5"/>
</g>
<!-- Created&#45;&gt;Running -->
<g id="edge2" class="edge">
<title>Created&#45;&gt;Running</title>
<path fill="none" stroke="black" d="M241.62,-18C249.83,-18 258.47,-18 266.95,-18"/>
<polygon fill="black" stroke="black" points="267.2,-21.5 277.2,-18 267.2,-14.5 267.2,-21.5"/>
</g>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.2 KiB

15
doc/emulation/fig/thread-model.dot Normal file
View File

@ -0,0 +1,15 @@
digraph {
graph [nodesep=1, ranksep=1];
{rank = same; unknown; "running"; "dead";}
//{rank = same; cooling; paused; warming;}
unknown -> "running" [label="eXecute *"];
"running" -> "cooling" [label="Cool"];
"cooling" -> "paused" [label="* Pause"];
"running" -> "paused" [label="* Pause"];
"paused" -> "running" [label="Resume *"];
"paused" -> "warming" [label="Warm *"];
"warming" -> "running" [label="Resume"];
"running" -> "dead" [label="* End"];
}

0
doc/emulation/ovni/thread-model.png → doc/emulation/fig/thread-model.png
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 58 KiB

104
doc/emulation/fig/thread-model.svg Normal file
View File

@ -0,0 +1,104 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Generated by graphviz version 5.0.0 (0)
-->
<!-- Pages: 1 -->
<svg width="596pt" height="413pt"
viewBox="0.00 0.00 596.34 413.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 409)">
<polygon fill="white" stroke="transparent" points="-4,4 -4,-409 592.34,-409 592.34,4 -4,4"/>
<!-- unknown -->
<g id="node1" class="node">
<title>unknown</title>
<ellipse fill="none" stroke="black" cx="87.2" cy="-387" rx="52.79" ry="18"/>
<text text-anchor="middle" x="87.2" y="-383.3" font-family="Times,serif" font-size="14.00">unknown</text>
</g>
<!-- running -->
<g id="node2" class="node">
<title>running</title>
<ellipse fill="none" stroke="black" cx="331.2" cy="-387" rx="47.39" ry="18"/>
<text text-anchor="middle" x="331.2" y="-383.3" font-family="Times,serif" font-size="14.00">running</text>
</g>
<!-- unknown&#45;&gt;running -->
<g id="edge1" class="edge">
<title>unknown&#45;&gt;running</title>
<path fill="none" stroke="black" d="M140.14,-387C179.47,-387 233.07,-387 273.49,-387"/>
<polygon fill="black" stroke="black" points="273.72,-390.5 283.72,-387 273.72,-383.5 273.72,-390.5"/>
<text text-anchor="middle" x="211.8" y="-393.8" font-family="Times,serif" font-size="14.00">eXecute *</text>
</g>
<!-- dead -->
<g id="node3" class="node">
<title>dead</title>
<ellipse fill="none" stroke="black" cx="555.2" cy="-387" rx="33.29" ry="18"/>
<text text-anchor="middle" x="555.2" y="-383.3" font-family="Times,serif" font-size="14.00">dead</text>
</g>
<!-- running&#45;&gt;dead -->
<g id="edge8" class="edge">
<title>running&#45;&gt;dead</title>
<path fill="none" stroke="black" d="M378.7,-387C418.03,-387 473.44,-387 511.64,-387"/>
<polygon fill="black" stroke="black" points="511.93,-390.5 521.93,-387 511.93,-383.5 511.93,-390.5"/>
<text text-anchor="middle" x="450.35" y="-393.8" font-family="Times,serif" font-size="14.00">* End</text>
</g>
<!-- cooling -->
<g id="node4" class="node">
<title>cooling</title>
<ellipse fill="none" stroke="black" cx="44.2" cy="-264" rx="44.39" ry="18"/>
<text text-anchor="middle" x="44.2" y="-260.3" font-family="Times,serif" font-size="14.00">cooling</text>
</g>
<!-- running&#45;&gt;cooling -->
<g id="edge2" class="edge">
<title>running&#45;&gt;cooling</title>
<path fill="none" stroke="black" d="M289.6,-378.08C251,-369.83 192.54,-355.01 145.2,-333 118.48,-320.58 90.92,-301.53 71.35,-286.75"/>
<polygon fill="black" stroke="black" points="73.28,-283.82 63.21,-280.5 69.01,-289.37 73.28,-283.82"/>
<text text-anchor="middle" x="161.2" y="-321.8" font-family="Times,serif" font-size="14.00">Cool</text>
</g>
<!-- paused -->
<g id="node5" class="node">
<title>paused</title>
<ellipse fill="none" stroke="black" cx="331.2" cy="-141" rx="44.39" ry="18"/>
<text text-anchor="middle" x="331.2" y="-137.3" font-family="Times,serif" font-size="14.00">paused</text>
</g>
<!-- running&#45;&gt;paused -->
<g id="edge4" class="edge">
<title>running&#45;&gt;paused</title>
<path fill="none" stroke="black" d="M321.28,-369.05C310.18,-349.04 292.9,-314.36 286.2,-282 277.68,-240.9 298.67,-194.84 314.77,-167.15"/>
<polygon fill="black" stroke="black" points="317.82,-168.87 319.99,-158.5 311.83,-165.25 317.82,-168.87"/>
<text text-anchor="middle" x="313.2" y="-260.3" font-family="Times,serif" font-size="14.00">* Pause</text>
</g>
<!-- cooling&#45;&gt;paused -->
<g id="edge3" class="edge">
<title>cooling&#45;&gt;paused</title>
<path fill="none" stroke="black" d="M58.64,-246.83C73.38,-231.3 97.68,-208.29 123.2,-195 172.77,-169.19 235.64,-155.43 279.22,-148.46"/>
<polygon fill="black" stroke="black" points="279.96,-151.89 289.32,-146.91 278.9,-144.97 279.96,-151.89"/>
<text text-anchor="middle" x="150.2" y="-198.8" font-family="Times,serif" font-size="14.00">* Pause</text>
</g>
<!-- paused&#45;&gt;running -->
<g id="edge5" class="edge">
<title>paused&#45;&gt;running</title>
<path fill="none" stroke="black" d="M340.01,-158.67C344.91,-168.85 350.54,-182.32 353.2,-195 365.77,-255.04 368.14,-273.04 355.2,-333 353.24,-342.08 349.58,-351.48 345.71,-359.8"/>
<polygon fill="black" stroke="black" points="342.47,-358.47 341.15,-368.98 348.73,-361.58 342.47,-358.47"/>
<text text-anchor="middle" x="398.7" y="-260.3" font-family="Times,serif" font-size="14.00">Resume *</text>
</g>
<!-- warming -->
<g id="node6" class="node">
<title>warming</title>
<ellipse fill="none" stroke="black" cx="470.2" cy="-18" rx="51.99" ry="18"/>
<text text-anchor="middle" x="470.2" y="-14.3" font-family="Times,serif" font-size="14.00">warming</text>
</g>
<!-- paused&#45;&gt;warming -->
<g id="edge6" class="edge">
<title>paused&#45;&gt;warming</title>
<path fill="none" stroke="black" d="M349.08,-124.43C373.24,-103.4 416.46,-65.78 444.21,-41.62"/>
<polygon fill="black" stroke="black" points="446.69,-44.1 451.94,-34.9 442.1,-38.82 446.69,-44.1"/>
<text text-anchor="middle" x="436.2" y="-75.8" font-family="Times,serif" font-size="14.00">Warm *</text>
</g>
<!-- warming&#45;&gt;running -->
<g id="edge7" class="edge">
<title>warming&#45;&gt;running</title>
<path fill="none" stroke="black" d="M475.21,-35.95C478.71,-49.65 482.54,-69.47 481.2,-87 474.51,-174.43 485.56,-201.33 451.2,-282 434.89,-320.27 396.12,-349.56 367.03,-367.15"/>
<polygon fill="black" stroke="black" points="365.21,-364.16 358.35,-372.24 368.74,-370.21 365.21,-364.16"/>
<text text-anchor="middle" x="504.2" y="-198.8" font-family="Times,serif" font-size="14.00">Resume</text>
</g>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 5.5 KiB

20
doc/emulation/index.md
View File

@ -13,10 +13,10 @@ The emulator uses several models to identify how the resources are being
used. The following diagram despicts the resource, process and task used. The following diagram despicts the resource, process and task
model. model.
![Model](model.png) [![Model](fig/model.svg)](fig/model.svg)
The resource model directly maps to the available hardware on the The resource model directly maps to the available hardware on the
machine. It consists of clusters which contains nodes, where each node machine. It consists of a cluster which contains nodes, where each node
contains a set of CPUs that can execute instructions. contains a set of CPUs that can execute instructions.
The process model tracks the state of processes and threads. Processes The process model tracks the state of processes and threads. Processes
@ -39,3 +39,19 @@ processing of events can keep the disk writting as the bottleneck.
The linter mode enables more tests which are disabled from the default The linter mode enables more tests which are disabled from the default
mode to prevent costly operations running in the emulator by default. mode to prevent costly operations running in the emulator by default.
The linter tests are enabled when running the ovni testsuite. The linter tests are enabled when running the ovni testsuite.
## Emulation models
Each component is implemented in an emulation model, which consists of
the following elements:
- A single byte model identification (for example `O`).
- A set of runtime events with that model identification (see the [list
of events](events)).
- Rules that determine which sequences of events are valid.
- The emulation hooks that process each event and modify the state of
the emulator.
- A set of channels that output the states from the emulator.
- A set of Paraver views that present the information in a timeline.
All the models are independent and can be instrumented at the same time.

119
doc/emulation/nanos6.md
View File

@ -1,54 +1,74 @@
# Nanos6 model # Nanos6 model
The Nanos6 emulator generates four different Paraver views, which are The Nanos6 runtime library implements the OmpSs-2 tasking model, which
explained in this document. schedules the execution of tasks with dependencies. For more information
see the [OmpSs-2 website][oss] and the [Nanos6 repository][nanos6].
## Task id [oss]: https://pm.bsc.es/ompss-2
[nanos6]: https://github.com/bsc-pm/nanos6
The task id view represents the id of the Nanos6 task instance that is The library is instrumented to track the execution of tasks and also the
currently executing on each thread/cpu. This id is a monotonically execution path inside the runtime library to identify what is happening.
increasing identifier assigned on task creation. Lower ids correspond to This information is typically used by both the users and the developers
tasks created at an earlier point than higher ids. of the Nanos6 library to analyze problems and unwanted behaviors.
Towards this goal, four different Paraver views are generated, which are
explained below.
## Task type The state of each task is modelled in a simple finite state machine,
which identifies the main state changes of the task. The task is set to
the *Running* state only when is executing the body of the task,
consisting of user defined code. The states can be observed in the
following diagram:
![Nanos6 task states](fig/nanos6-task-model.svg)
## Task ID view
The task ID view represents the ID of the Nanos6 task instance that is
currently executing on each thread. This ID is a monotonically
increasing identifier assigned on task creation. Lower IDs correspond to
tasks created at an earlier point than higher IDs.
## Task type view
Every task in Nanos6 contains a task type, which roughly corresponds to Every task in Nanos6 contains a task type, which roughly corresponds to
the actual location in the code a task was declared. For example if a the actual location in the code a task was declared. For example if a
function fn() is declared as a Nanos6 task, and it is called multiple function is declared as a Nanos6 task, and it is called multiple times
times in a program, every created task will have a different id, but the in a program, every created task will have a different ID, but the same
same type. type.
In the view, each type is shown with a label declared in the source with In the view, each type is shown with a label declared in the source with
the label() attribute of the task. If no label was specified, one is the label attribute of the task. If no label was specified, one is
automatically generated for each type. automatically generated for each type.
Note that in this view, event value is a hash function of the type Note that in this view, the numeric event value is a hash function of
label, so two distinct types (tasks declared in different parts of the the type label, so two distinct types (tasks declared in different parts
code) with the same label will share event value and will hence be of the code) with the same label will share the event value and have the
indistinguishable. same color.
## MPI Rank ## MPI rank view
Represents the current MPI rank for the currently running task in a This view shows the numeric MPI rank of the process running the current
thread or cpu. task. It is only shown when the task is in the running state. This view
is specially useful to identify task in a distributed workload, which
spans several nodes.
## Subsystem view ## Subsystem view
The subsystem view attempts to provide a general overview of what the The subsystem view attempts to provide a general overview of what Nanos6
runtime is doing at any point in time. This view is more complex to is doing at any point in time. This view is more complex to understand
understand than the others but is generally the most useful to than the others but is generally the most useful to understand what is
understand what is happening and debug problems related with the happening and debug problems related with Nanos6 itself.
runtime.
The view shows the state of the runtime for each thread (and for each The view shows the state of the runtime for each thread (and for each
CPU, the state of the running thread in that CPU). CPU, the state of the running thread in that CPU).
The state is computed by the following method: the runtime code is The state is computed by the following method: the runtime code is
completely divided into sections of code (machine instructions) S1, S2, completely divided into sections of code (machine instructions) $`S_1,
..., SN, which are instrumented (an event is emitted when entering and S_2, \ldots, S_N`$, which are instrumented (an event is emitted when entering
exiting each section), and one common section of code which is shared and exiting each section), and one common section of code which is
across the subsystems, U, of no interest. We also assume any other code shared across the subsystems, $`U`$, of no interest. We also assume any
not belonging to the runtime to be in the U section. other code not belonging to the runtime belongs to the $`U`$ section.
!!! remark !!! remark
@ -58,36 +78,35 @@ To determine the state of a thread, we look into the stack to see what
is the top-most instrumented section. is the top-most instrumented section.
At any given point in time, a thread may be executing code with a stack At any given point in time, a thread may be executing code with a stack
that spawns multiple sections, for example *S1, U, S2, S3* and *U* (the that spawns multiple sections, for example $`[ S_1, U, S_2, S_3, U ]`$
last is on top). The subsystem view selects the last subsystem section (the last is current stack frame). The subsystem view selects the last
from the stack ignoring the common section *U*, and presents that section subsystem section from the stack ignoring the common section $`U`$, and
as the current state of the execution, in this case the *S3*. presents that section as the current state of the execution, in this
case the section $`S_3`$.
Additionally, the runtime sections are grouped together in subsystems, Additionally, the runtime sections $`S_i`$ are grouped together in
which form a group of closely related functions. A complete set of subsystems, which form a closely related group of functions. The
states for each subsystem is listed below. complete list of subsystems and sections is shown below.
When there is no instrumented section in the thread stack, the state is When there is no instrumented section in the thread stack, the state is
set to **No subsystem**. set to **No subsystem**.
### Task subsystem Task subsystem
: The **Task** subsystem contains the code that controls the life cycle
The **Task** subsystem contains the code that controls the lifecycle of of tasks. It contains the following sections:
tasks. It contains the following sections:
- **Running**: Executing the body of the task (user defined code). - **Running**: Executing the body of the task (user defined code).
- **Spawning function**: Registering a new spawn function - **Spawning function**: Spawning a function as task (it will be
(programmatically created task) submitted to the scheduler for later execution).
- **Creating**: Creating a new task, through `nanos6_create_task` - **Creating**: Creating a new task, through `nanos6_create_task`
- **Submitting**: Submitting a recently created task, through - **Submitting**: Submitting a recently created task, through
`nanos6_submit_task` `nanos6_submit_task`
### Scheduler subsystem Scheduler subsystem
: The **Scheduler** system groups the actions that relate to the queueing
The **Scheduler** system groups the actions that relate to the queueing
and dequeueing of ready tasks. It contains the following sections: and dequeueing of ready tasks. It contains the following sections:
- **Waiting for tasks**: Actively waiting for tasks inside the - **Waiting for tasks**: Actively waiting for tasks inside the
@ -99,9 +118,8 @@ to other threads
- **Adding ready tasks**: Adding tasks to the scheduler queues, - **Adding ready tasks**: Adding tasks to the scheduler queues,
but outside of the scheduler lock. but outside of the scheduler lock.
### Dependency subsystem Dependency subsystem
: The **Dependency** system only contains the code that manages the
The **Dependency** system only contains the code that manages the
registration of task dependencies. It contains the following sections: registration of task dependencies. It contains the following sections:
- **Registering**: Registering a task's dependencies - **Registering**: Registering a task's dependencies
@ -109,9 +127,8 @@ registration of task dependencies. It contains the following sections:
- **Unregistering**: Releasing a task's dependencies because - **Unregistering**: Releasing a task's dependencies because
it has ended it has ended
### Blocking subsystem Blocking subsystem
: The **Blocking** subsystem deals with the code stops the thread
The **Blocking** subsystem deals with the code stops the thread
execution. It contains the following sections: execution. It contains the following sections:
- **Taskwait**: Task is blocked while inside a taskwait - **Taskwait**: Task is blocked while inside a taskwait

15
doc/emulation/nosv.md
View File

@ -1,11 +1,16 @@
# nOS-V model # nOS-V model
## nOS-V task type colors The nOS-V runtime is composed of tasks that can be scheduled to run in
threads. Tasks can be paused and resumed, leaving the CPUs free to
execute other tasks. The nOS-V model tracks the state of each task as
well as the state of the runtime internal state.
The color assigned to each nOS-V task type is computed from the task ## Task type colors
type label using a hash function; the task type id doesn't affect in any
way how the color gets assigned. This method provides two desirable In the Paraver timeline, the color assigned to each nOS-V task type is
properties: computed from the task type label using a hash function; the task type
id doesn't affect in any way how the color gets assigned. This method
provides two desirable properties:
- Invariant type colors over time: the order in which task types are - Invariant type colors over time: the order in which task types are
created doesn't affect their color. created doesn't affect their color.

62
doc/emulation/ovni.md
View File

@ -1,5 +1,63 @@
# Ovni model # Ovni model
The ovni model tracks the state of threads and cpus. The ovni model tracks the state of threads and CPUs and provides the
basis for other models.
![Thread states](thread-model.png) ## Thread model
The thread state is modelled as a finite state machine, depicted below:
![Thread states](fig/thread-model.svg){: style="width:80%"}
All threads begin in the `unknown` state and perform a transition to
running then the `execute` event (with MCV `OHx`) is received.
Notice that the runtime thread started the execution **before** emitting
the execute event, as it is not posible to emit the event at the exact
moment the thread begins the execution.
The asterisk `*` in the transitions states when the event is emitted. If
appears *before* the transition name, it happens *before* the real
event. If appears *after* then it is emitted *after* the real event.
In the case of the thread beginning the execution, the event is emitted
*after* the thread begins the execution, so the asterisk appears after.
### Cooling and warming states
In the Linux kernel, it is generally not possible to switch from a
thread A to another thread B in a single atomic operation, from
user-space. The thread A must first wake B and then put itself to sleep.
But the kernel is free to continue the execution of B instead of just
executing the last instructions of A so it can stop.
This limitation causes a time region at which more than one thread is
scheduled to the same CPU, which is a property that is generally
considered an error in parallel runtimes. To avoid this situation, the
additional states *cooling* and *warming* are introduced.
The cooling state indicates that the thread will be put to sleep and may
temporarily wake another thread in the same CPU. Similarly, the warming
state indicates that a thread is waking up, but it may be another thread
in its own CPU for a while.
These states allow the emulator to ignore the threads in the cooling and
warming states when only interested in those in the running state, but
at the same time allow the developers to identify the real transition
sequence to explain potential delays.
## CPU model
The CPU can only execute one thread at a time, and it may have multiple
threads assigned. When two or more threads are in the execution state,
the emulator cannot determine which one is really being executed, so it
will set all the channels to an error state.
!!! Caution
Multiple threads running in the same CPU typically indicates an
error of instrumentation or a problem in the runtime.
The emulator automatically switches the channels from one thread to
another when a thread is switched from the CPU. So the different models
don't need to worry about thread transitions. See the
[channels](../channels) section for more information.

3
doc/extra.css
View File

@ -3,13 +3,14 @@
font-size: 85%; font-size: 85%;
border: none; border: none;
padding: 0px; padding: 0px;
background: #f8f8f8; background: #eee;
} }
.rst-content pre code { .rst-content pre code {
color: inherit; color: inherit;
font-size: 80%; font-size: 80%;
line-height: 1.5em; line-height: 1.5em;
background: #eee;
} }
p { p {

8
doc/fig/instrumentation.dot Normal file
View File

@ -0,0 +1,8 @@
digraph {
rankdir = LR;
Execution, Emulation, Visualization [ shape = circle, width=1.5 ];
"Event trace", "State trace", "Timeline" [ shape = box ];
Execution -> "Event trace" -> Emulation -> "State trace" -> Visualization -> "Timeline";
}

78
doc/fig/instrumentation.svg Normal file
View File

@ -0,0 +1,78 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Generated by graphviz version 5.0.0 (0)
-->
<!-- Pages: 1 -->
<svg width="789pt" height="119pt"
viewBox="0.00 0.00 788.54 118.54" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 114.54)">
<polygon fill="white" stroke="transparent" points="-4,4 -4,-114.54 784.54,-114.54 784.54,4 -4,4"/>
<!-- Execution -->
<g id="node1" class="node">
<title>Execution</title>
<ellipse fill="none" stroke="black" cx="54" cy="-55.27" rx="54" ry="54"/>
<text text-anchor="middle" x="54" y="-51.57" font-family="Times,serif" font-size="14.00">Execution</text>
</g>
<!-- Event trace -->
<g id="node4" class="node">
<title>Event trace</title>
<polygon fill="none" stroke="black" points="244,-73.27 144,-73.27 144,-37.27 244,-37.27 244,-73.27"/>
<text text-anchor="middle" x="194" y="-51.57" font-family="Times,serif" font-size="14.00">Event trace</text>
</g>
<!-- Execution&#45;&gt;Event trace -->
<g id="edge1" class="edge">
<title>Execution&#45;&gt;Event trace</title>
<path fill="none" stroke="black" d="M108.35,-55.27C116.66,-55.27 125.28,-55.27 133.71,-55.27"/>
<polygon fill="black" stroke="black" points="133.85,-58.77 143.85,-55.27 133.85,-51.77 133.85,-58.77"/>
</g>
<!-- Emulation -->
<g id="node2" class="node">
<title>Emulation</title>
<ellipse fill="none" stroke="black" cx="334" cy="-55.27" rx="54" ry="54"/>
<text text-anchor="middle" x="334" y="-51.57" font-family="Times,serif" font-size="14.00">Emulation</text>
</g>
<!-- State trace -->
<g id="node5" class="node">
<title>State trace</title>
<polygon fill="none" stroke="black" points="520,-73.27 424,-73.27 424,-37.27 520,-37.27 520,-73.27"/>
<text text-anchor="middle" x="472" y="-51.57" font-family="Times,serif" font-size="14.00">State trace</text>
</g>
<!-- Emulation&#45;&gt;State trace -->
<g id="edge3" class="edge">
<title>Emulation&#45;&gt;State trace</title>
<path fill="none" stroke="black" d="M388.36,-55.27C396.72,-55.27 405.39,-55.27 413.83,-55.27"/>
<polygon fill="black" stroke="black" points="413.97,-58.77 423.97,-55.27 413.97,-51.77 413.97,-58.77"/>
</g>
<!-- Visualization -->
<g id="node3" class="node">
<title>Visualization</title>
<ellipse fill="none" stroke="black" cx="611.27" cy="-55.27" rx="55.04" ry="55.04"/>
<text text-anchor="middle" x="611.27" y="-51.57" font-family="Times,serif" font-size="14.00">Visualization</text>
</g>
<!-- Timeline -->
<g id="node6" class="node">
<title>Timeline</title>
<polygon fill="none" stroke="black" points="780.54,-73.27 702.54,-73.27 702.54,-37.27 780.54,-37.27 780.54,-73.27"/>
<text text-anchor="middle" x="741.54" y="-51.57" font-family="Times,serif" font-size="14.00">Timeline</text>
</g>
<!-- Visualization&#45;&gt;Timeline -->
<g id="edge5" class="edge">
<title>Visualization&#45;&gt;Timeline</title>
<path fill="none" stroke="black" d="M666.66,-55.27C675.11,-55.27 683.8,-55.27 692.1,-55.27"/>
<polygon fill="black" stroke="black" points="692.38,-58.77 702.38,-55.27 692.38,-51.77 692.38,-58.77"/>
</g>
<!-- Event trace&#45;&gt;Emulation -->
<g id="edge2" class="edge">
<title>Event trace&#45;&gt;Emulation</title>
<path fill="none" stroke="black" d="M244.04,-55.27C252.39,-55.27 261.17,-55.27 269.83,-55.27"/>
<polygon fill="black" stroke="black" points="269.9,-58.77 279.9,-55.27 269.9,-51.77 269.9,-58.77"/>
</g>
<!-- State trace&#45;&gt;Visualization -->
<g id="edge4" class="edge">
<title>State trace&#45;&gt;Visualization</title>
<path fill="none" stroke="black" d="M520.24,-55.27C528.49,-55.27 537.2,-55.27 545.82,-55.27"/>
<polygon fill="black" stroke="black" points="545.86,-58.77 555.86,-55.27 545.86,-51.77 545.86,-58.77"/>
</g>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 3.7 KiB

0
doc/logo.png → doc/fig/logo.png
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 142 KiB

After

Width:  |  Height:  |  Size: 142 KiB

BIN
doc/fig/logo2.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 118 KiB

BIN
doc/fig/visualization.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 20 KiB

22
doc/index.md
View File

@ -1,4 +1,4 @@
![Ovni logo](logo2.png) ![Ovni logo](fig/logo2.png)
This is the documentation of ovni, the Obtuse (but Versatile) Nanoscale This is the documentation of ovni, the Obtuse (but Versatile) Nanoscale
Instrumentation project. Instrumentation project.
@ -6,8 +6,8 @@ Instrumentation project.
!!! Note !!! Note
Preferably write the name of the project as lowercase *ovni* Preferably write the name of the project as lowercase *ovni*
unless the grammar rules suggest otherwise, such as starting a unless the grammar rules suggest otherwise, such as when
new sentence. starting a new sentence.
The ovni project implements a fast instrumentation library that records The ovni project implements a fast instrumentation library that records
small events (starting at 12 bytes) during the execution of programs to small events (starting at 12 bytes) during the execution of programs to
@ -16,7 +16,7 @@ later investigate how the execution happened.
The instrumentation process is split in two stages: [runtime](runtime) The instrumentation process is split in two stages: [runtime](runtime)
tracing and [emulation](emulation/). tracing and [emulation](emulation/).
During runtime, very simple and short events are stored on disk which During runtime, very short binary events are stored on disk which
describe what is happening. Once the execution finishes, the events are describe what is happening. Once the execution finishes, the events are
read and processed to reproduce the execution during the emulation read and processed to reproduce the execution during the emulation
process, and the final execution trace is generated. process, and the final execution trace is generated.
@ -29,5 +29,15 @@ Each event belongs to a model, which has a direct mapping to a target
library or program. Each model is independent of other models, and they library or program. Each model is independent of other models, and they
can be instrumented concurrently. can be instrumented concurrently.
The events are classified by using three identifiers known as *model*, ## Quick start
*category* and *value* (or MCV for short).
To start a trace follow these steps:
- First enable the instrumentation of those libraries or programs that
you are interested in (see their documentation).
- Then simply run the program normally. You will see a new `ovni`
directory containing the runtime trace.
- Finally run the `ovniemu ovni` command to generate the Paraver traces.
- Use the command `wxparaver ovni/cpu.prv` to load the CPU trace.
- Load the configurations from the `cfg/` directory that you are
interested in, to open a timeline view.

BIN
doc/logo2.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 113 KiB

0
doc/runtime/trace_spec/event-jumbo.svg → doc/runtime/fig/event-jumbo.svg
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 44 KiB

After

Width:  |  Height:  |  Size: 44 KiB

0
doc/runtime/trace_spec/event-normal-payload.svg → doc/runtime/fig/event-normal-payload.svg
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 40 KiB

After

Width:  |  Height:  |  Size: 40 KiB

0
doc/runtime/trace_spec/event-normal.svg → doc/runtime/fig/event-normal.svg
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 26 KiB

After

Width:  |  Height:  |  Size: 26 KiB

0
doc/runtime/trace_spec/stream.svg → doc/runtime/fig/stream.svg
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 18 KiB

After

Width:  |  Height:  |  Size: 18 KiB

27
doc/runtime/trace_spec.md
View File

@ -1,8 +1,9 @@
# Trace specification # Trace specification
!!! Note !!! Important
Thie is the trace specification for the version 1 This document refers to the trace specification for
the version 1
The ovni instrumentation library stores the information collected in a The ovni instrumentation library stores the information collected in a
trace following the specification of this document. trace following the specification of this document.
@ -83,7 +84,7 @@ the variable size events just after the header.
The header contains the magic 4 bytes of "ovni" and a version number of The header contains the magic 4 bytes of "ovni" and a version number of
4 bytes too. Here is a figure of the data stored in disk: 4 bytes too. Here is a figure of the data stored in disk:
<img src="stream.svg" alt="Stream" width="400px"/> ![Stream](fig/stream.svg)
Similarly, events have a fixed size header followed by an optional Similarly, events have a fixed size header followed by an optional
payload of varying size. The header has the following information: payload of varying size. The header has the following information:
@ -103,7 +104,7 @@ There are two types of events, depending of the size needed for the
payload: payload:
- Normal events: with a payload up to 16 bytes - Normal events: with a payload up to 16 bytes
- Jumbo events: with a payload up to 2^32 bytes - Jumbo events: with a payload up to $`2^{32}`$ bytes
## Normal events ## Normal events
@ -124,7 +125,7 @@ bytes:
And in the following figure you can see every field annotated: And in the following figure you can see every field annotated:
<img src="event-normal.svg" alt="Normal event without payload" width="400px"/> ![Normal event without payload](fig/event-normal.svg)
Another example of a normal event with 16 bytes of payload, a total of Another example of a normal event with 16 bytes of payload, a total of
28 bytes: 28 bytes:
@ -136,7 +137,7 @@ ff ff ff ff 00 00 00 00 00 00 00 00 |............|
In the following figure you can see each field annotated: In the following figure you can see each field annotated:
<img src="event-normal-payload.svg" alt="Normal event with payload content" width="400px"/> ![Normal event with payload content](fig/event-normal-payload.svg)
## Jumbo events ## Jumbo events
@ -161,7 +162,7 @@ data:
In the following figure you can see each field annotated: In the following figure you can see each field annotated:
<img src="event-jumbo.svg" alt="Jumbo event" width="400px"/> ![Jumbo event](fig/event-jumbo.svg)
## Design considerations ## Design considerations
@ -171,11 +172,13 @@ parser library would take no more than 2 days for a single developer.
The size of the events has been designed to be small, with 12 bytes per The size of the events has been designed to be small, with 12 bytes per
event when no payload is used. event when no payload is used.
**Important:** The events are stored in disk following the endianness of !!! Caution
the machine where they are generated. So a stream generated with a little
endian machine would be different than on a big endian machine. We The events are stored in disk following the endianness of the
assume the same endiannes is used to write the trace at runtime and read machine where they are generated. So a stream generated with a
it after, at the emulation process. little endian machine would be different than on a big endian
machine. We assume the same endiannes is used to write the trace
at runtime and read it after, at the emulation process.
The events are designed to be easily identified when looking at the The events are designed to be easily identified when looking at the
raw stream in binary, as the MCV codes can be read as ASCII characters: raw stream in binary, as the MCV codes can be read as ASCII characters:

6
doc/runtime/tracing.md
View File

@ -1,6 +1,8 @@
# Tracing a program # Tracing a new program
Read carefully this document before using libovni to generate a trace. Read carefully this document before using libovni to instrument a new
component. There are a few rules you must follow to ensure the runtime
trace is correct.
## Trace processes and threads ## Trace processes and threads

15
mkdocs.yml
View File

@ -1,14 +1,25 @@
site_name: ovni site_name: ovni
theme: readthedocs
docs_dir: doc docs_dir: doc
extra_css: [extra.css] extra_css: [extra.css]
theme:
name: readthedocs
sticky_navigation: false
extra_javascript:
- https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.0/MathJax.js?config=TeX-AMS-MML_HTMLorMML
markdown_extensions: markdown_extensions:
- admonition - admonition
- attr_list
- def_list
- mdx_math:
use_gitlab_delimiters: true
enable_dollar_delimiter: true
add_preview: true
- toc: - toc:
permalink: "#" permalink: "#"
separator: "_" separator: "_"
nav: nav:
- index.md - index.md
- concepts.md
- 'Runtime': - 'Runtime':
- runtime/tracing.md - runtime/tracing.md
- runtime/kernel.md - runtime/kernel.md
@ -20,3 +31,5 @@ nav:
- emulation/nanos6.md - emulation/nanos6.md
- emulation/events.md - emulation/events.md
- emulation/channels.md - emulation/channels.md
- 'Developer guide':
- developer-guide/index.md