268 lines
9.0 KiB
Markdown
268 lines
9.0 KiB
Markdown
# Trace specification v3
|
|
|
|
!!! Important
|
|
|
|
This document refers to the trace specification for
|
|
the version 3
|
|
|
|
The ovni instrumentation library libovni stores the information
|
|
collected in a runtime trace following the specification of this document.
|
|
|
|
## Structure
|
|
|
|
An ovni runtime trace (or simply, a trace) is composed of one or more
|
|
[streams](../concepts/trace-model.md#stream), which are directories containing
|
|
two mandatory files:
|
|
|
|
- `stream.json` the stream metadata in JSON format.
|
|
- `stream.obs` the binary stream with events.
|
|
|
|
Each stream is assigned to a single *part* in the [part
|
|
model](../concepts/part-model.md), usually assigned to a given thread.
|
|
|
|
There are no imposed rules on how to organize the several streams into
|
|
directories, but libovni uses the following approach for thread streams:
|
|
|
|
The complete trace is stored in a top-level directory named `ovni`.
|
|
Inside this directory you will find the loom directories. The name of
|
|
the loom directory is built from the `loom` parameter of `ovni_proc_init()`,
|
|
prefixing it with `loom.`.
|
|
|
|
Each loom directory contains one directory per process of that loom. The
|
|
name is composed of the `proc.` prefix and the PID of the process
|
|
specified in the `pid` argument to `ovni_proc_init()`.
|
|
|
|
Inside each process there is one directory for each thread, composed by
|
|
the `thread.` prefix and the TID, which are the streams. The files
|
|
`stream.json` and `stream.obs` reside inside. Example:
|
|
|
|
```
|
|
ovni/loom.mio.nosv-u1000/proc.89719/thread.89719/stream.json
|
|
ovni/loom.mio.nosv-u1000/proc.89719/thread.89719/stream.obs
|
|
```
|
|
|
|
This structure prevents collisions among threads with the same TID among nodes,
|
|
while allowing dumping events from a single thread, process or loom with
|
|
ovnidump.
|
|
|
|
## Stream metadata
|
|
|
|
The `stream.json` metadata file contains information about the part that
|
|
the stream is assigned to. This is generally used to determine the
|
|
hierarchy of the part model.
|
|
|
|
The JSON must be an object (dictionary) with the following mandatory
|
|
keys:
|
|
|
|
- `version`: a number specifying the version of the metadata format.
|
|
Must have the value 3 for this version.
|
|
|
|
The rest of information is stored for each model.
|
|
|
|
In particular, the `ovni` model enforces the use of:
|
|
|
|
- `ovni.part`: the type of part this stream is assigned to, usually
|
|
`thread`.
|
|
- `ovni.require`: a dictionary of model name and version which will
|
|
determine which models are enabled at emulation and the required
|
|
version.
|
|
- `ovni.finished`: must be 1 to ensure the stream is complete (mandatory
|
|
in all streams).
|
|
|
|
### Thread stream metadata
|
|
|
|
For `thread` streams, the following attributes are used.
|
|
|
|
- `ovni.tid`: the TID of the thread (mandatory, per-thread).
|
|
- `ovni.pid`: the PID of the process that the thread belongs to (mandatory, per-thread).
|
|
- `ovni.app_id`: the application ID of the process (optional, per-process).
|
|
- `ovni.rank`: the rank of the MPI process (optional, per-process).
|
|
- `ovni.nranks`: number of total MPI processes (optional, per-process).
|
|
- `ovni.loom`: the name of the loom that the process belongs to (mandatory, per-process).
|
|
- `ovni.loom_cpus`: the array of N CPUs available in the loom
|
|
(mandatory, per-loom). Each element is a dictionary with the keys:
|
|
- `index`: containing the logical CPU index from 0 to N - 1.
|
|
- `phyid`: the number of the CPU as given by the operating system
|
|
(which can exceed N).
|
|
|
|
Notice that some attributes don't need to be present in all thread
|
|
streams. For example, per-process requires that at least one thread
|
|
contains the attribute for each process. Similarly, per-loom requires
|
|
that at least one thread of the loom emits the attribute.
|
|
|
|
The final attribute value will be computed by merging all the values from the
|
|
children metadata. Simple values like numbers or strings must match exactly if
|
|
they appear duplicated, arrays are appended.
|
|
|
|
Other attributes can be used for other models.
|
|
|
|
Here is an example of the `stream.json` file for a thread of a nOS-V
|
|
program:
|
|
|
|
```json
|
|
{
|
|
"version": 3,
|
|
"ovni": {
|
|
"lib": {
|
|
"version": "1.10.0",
|
|
"commit": "dirty"
|
|
},
|
|
"part": "thread",
|
|
"tid": 89719,
|
|
"pid": 89719,
|
|
"loom": "mio.nosv-u1000",
|
|
"app_id": 1,
|
|
"require": {
|
|
"ovni": "1.1.0",
|
|
"nosv": "2.3.0"
|
|
},
|
|
"loom_cpus": [
|
|
{ "index": 0, "phyid": 0 },
|
|
{ "index": 1, "phyid": 1 },
|
|
{ "index": 2, "phyid": 2 },
|
|
{ "index": 3, "phyid": 3 }
|
|
],
|
|
"finished": 1
|
|
},
|
|
"nosv": {
|
|
"can_breakdown": false,
|
|
"lib_version": "2.3.1"
|
|
}
|
|
}
|
|
```
|
|
|
|
## Binary stream
|
|
|
|
!!! Important
|
|
|
|
Binary streams have version 1
|
|
|
|
A binary stream is a binary file named `stream.obs` that contains a
|
|
succession of events with monotonically increasing clock values. They
|
|
have a small header and the variable size events just after the header.
|
|
|
|
The header contains the magic 4 bytes of "ovni" and a version number of
|
|
4 bytes too. Here is a figure of the data stored in disk on a little
|
|
endian machine:
|
|
|
|
|
|
|
|
Similarly, events have a fixed size header followed by an optional
|
|
payload of varying size. The header has the following information:
|
|
|
|
- Event flags
|
|
- Payload size in a special format
|
|
- Model, category and value codes
|
|
- Time in nanoseconds
|
|
|
|
The event size can vary depending on the data stored in the payload. The
|
|
payload size is specified using 4 bits, with the value `0x0` for no
|
|
payload, or with value $`v`$ for $`v + 1`$ bytes of payload. This allows
|
|
us to use 16 bytes of payload with value `0xf` at the cost of
|
|
sacrificing payloads of one byte.
|
|
|
|
There are two types of events, depending of the size needed for the
|
|
payload:
|
|
|
|
- Normal events: with a payload up to 16 bytes
|
|
- Jumbo events: with a payload up to $`2^{32}`$ bytes
|
|
|
|
### Normal events
|
|
|
|
The normal events are composed of:
|
|
|
|
- 4 bits of flags
|
|
- 4 bits of payload size
|
|
- 3 bytes for the MCV
|
|
- 8 bytes for the clock
|
|
- 0 to 16 bytes of payload
|
|
|
|
Here is an example of a normal event without payload, a total of 12
|
|
bytes:
|
|
|
|
```
|
|
00 4f 48 65 01 c5 cf 1d 96 d0 12 00 |.OHe........|
|
|
```
|
|
|
|
And in the following figure you can see every field annotated:
|
|
|
|
|
|
|
|
Another example of a normal event with 16 bytes of payload, a total of
|
|
28 bytes:
|
|
|
|
```
|
|
0f 4f 48 78 58 c1 b0 b5 95 43 11 00 00 00 00 00 |.OHxX....C......|
|
|
ff ff ff ff 00 00 00 00 00 00 00 00 |............|
|
|
```
|
|
|
|
In the following figure you can see each field annotated:
|
|
|
|
|
|
|
|
### Jumbo events
|
|
|
|
The jumbo events are just like normal events but they can hold large
|
|
data. The size of the jumbo data is stored as a 32 bits integer as a
|
|
normal payload, and the jumbo data just follows the event.
|
|
|
|
- 4 bits of flags
|
|
- 4 bits of payload size (always 4 with value 0x3)
|
|
- 3 bytes for the MCV
|
|
- 8 bytes for the clock
|
|
- 4 bytes of payload with the size of the jumbo data
|
|
- 0 to 2^32 bytes of jumbo data
|
|
|
|
Example of a jumbo event of 30 bytes in total, with 14 bytes of jumbo
|
|
data:
|
|
|
|
```
|
|
13 56 59 63 eb c1 4b 1a 96 d0 12 00 0e 00 00 00 |.VYc..K.........|
|
|
01 00 00 00 74 65 73 74 74 79 70 65 31 00 |....testtype1.|
|
|
```
|
|
|
|
In the following figure you can see each field annotated:
|
|
|
|
|
|
|
|
### Design considerations
|
|
|
|
The binary stream format has been designed to be very simple, so writing
|
|
a parser library would take no more than 2 days for a single developer.
|
|
|
|
The size of the events has been designed to be small, with 12 bytes per
|
|
event when no payload is used.
|
|
|
|
!!! Caution
|
|
|
|
The events are stored in disk following the endianness of the
|
|
machine where they are generated. So a stream generated with a
|
|
little endian machine would be different than on a big endian
|
|
machine. We assume the same endiannes is used to write the trace
|
|
at runtime and read it after, at the emulation process.
|
|
|
|
The events are designed to be easily identified when looking at the
|
|
raw stream in binary, as the MCV codes can be read as ASCII characters:
|
|
|
|
```
|
|
00000000 6f 76 6e 69 01 00 00 00 0f 4f 48 78 08 ba 2e 5c |ovni.....OHx...\|
|
|
00000010 b5 b0 00 00 00 00 00 00 ff ff ff ff 00 00 00 00 |................|
|
|
00000020 00 00 00 00 13 56 59 63 3c c2 2e 5c b5 b0 00 00 |.....VYc<..\....|
|
|
00000030 0e 00 00 00 01 00 00 00 74 65 73 74 74 79 70 65 |........testtype|
|
|
00000040 31 00 07 56 54 63 43 cc 2e 5c b5 b0 00 00 01 00 |1..VTcC..\......|
|
|
00000050 00 00 01 00 00 00 03 56 54 78 03 cd 2e 5c b5 b0 |.......VTx...\..|
|
|
00000060 00 00 01 00 00 00 03 56 54 70 2b 7d 37 5c b5 b0 |.......VTp+}7\..|
|
|
00000070 00 00 01 00 00 00 03 56 54 72 c3 4d 40 5c b5 b0 |.......VTr.M@\..|
|
|
00000080 00 00 01 00 00 00 03 56 54 65 03 36 49 5c b5 b0 |.......VTe.6I\..|
|
|
00000090 00 00 01 00 00 00 00 4f 48 65 f5 36 49 5c b5 b0 |.......OHe.6I\..|
|
|
000000a0 00 00 |..|
|
|
```
|
|
|
|
This allows a human to detect signs of corruption by visually inspecting
|
|
the streams.
|
|
|
|
### Limitations
|
|
|
|
The streams are designed to be read only forward, as they only contain
|
|
the size of each event in the header.
|