71a1396955
ds: parse time with perf generator
2021-03-09 11:07:19 +01:00
b600f64fcc
heat: add cache miss experiment and figure
2021-03-05 18:31:31 +01:00
14fbb1499b
ds: add perf stat parser
...
We can only read one output file by now, located at:
.garlic/perf.csv
2021-03-05 18:29:43 +01:00
c1efba1e65
heat: rename test -> granul experiment
2021-03-05 18:28:32 +01:00
29d7245135
heat: add figure with heatmap
2021-03-05 16:21:13 +01:00
363700eb9a
heat: update test experiment
2021-03-05 16:18:51 +01:00
7e10a43b40
heat: update new app version
...
The blocksize is now specified at runtime
2021-03-05 16:16:06 +01:00
6b6b54f757
timetable: add total_time column
2021-03-03 19:00:36 +01:00
5afe819724
osu: add impi figure
2021-03-03 12:42:19 +01:00
651d91ef79
fig: improve indentation
2021-03-03 12:42:19 +01:00
14211c9895
osu: use ggsave and reduce verbosity
2021-03-03 12:42:19 +01:00
6973f48638
osu: add an experiment for Intel MPI tunning
2021-03-03 12:42:19 +01:00
4786953eeb
garlic: fix self/super with correct scope
...
The callPackage function was trying to find packages in bsc.self before
the self of the input parameters.
2021-03-03 12:42:19 +01:00
a6815dc7cf
fig: add article figure directory
2021-03-03 12:41:31 +01:00
4ffb609261
osu: add figures using the fast generators
2021-03-01 12:21:10 +01:00
1d015c7e1e
ds: add osu fast generators
2021-03-01 12:00:58 +01:00
ed932c9921
osu: add bw test
2021-03-01 11:58:23 +01:00
a36d912022
osu: add multithread benchmark
2021-03-01 11:55:13 +01:00
8373751f67
rplot: remove suffix from input link
...
We may have compressed input datasets
2021-03-01 11:41:28 +01:00
2f7032aca6
pp: remove unused derivations and helpers
2021-03-01 11:40:56 +01:00
6dd41fd96f
fig: use the fast timetable generator by default
2021-03-01 11:38:28 +01:00
09a0348b0e
ds: add fast timetable generator
2021-03-01 11:16:03 +01:00
051a74b85d
srun: allow commands to run before srun
2021-02-26 17:00:09 +01:00
8a77900201
srun: don't expand variables on install
2021-02-26 16:59:29 +01:00
1291b90b7f
user guide: correct typo
2021-02-26 12:18:50 +01:00
8e130604aa
machines: set the hardware revision for MN4
...
This change will cause a rebuild of all experiments.
2021-02-25 20:45:20 +01:00
0015c7e4cd
pp: remove launcher
...
It has now been integrated with resultTree in pp/store.nix
2021-02-25 12:29:12 +01:00
9612c69aec
doc: add garlic configuration section
...
Update the garlicd usage as well.
2021-02-25 11:38:29 +01:00
6e0e2f0bf6
garlicd: drop bscpkgs argument requirement
...
The bscpkgs/default.nix is not longer read, as the garlic tool only
needs it for the fetching (-F), when it runs nix-build.
2021-02-25 11:38:29 +01:00
48820ee2d3
resultTree: garlic must be used from the nix shell
2021-02-25 11:38:29 +01:00
9277e60079
timetable: enable verbose processing
2021-02-25 11:38:29 +01:00
c869b6e3b4
garlic: enable verbose rsync fetch
2021-02-25 11:38:29 +01:00
0b95ea20b7
garlicd: allow manual experiment executions
2021-02-25 11:37:58 +01:00
ceb25e5d18
osu: add figure for latency tests
2021-02-23 17:52:48 +01:00
0c9e89dcc0
osu: update experiments using stdexp
2021-02-23 15:22:56 +01:00
ebcbf91fbe
exec: allow manual specification of program path
2021-02-23 15:22:18 +01:00
3e2b369e3e
garlicd: allow nix builders write to the pipes
2021-02-17 10:28:34 +01:00
d4947a40b9
Fix ssh missing shell
2021-02-17 10:28:11 +01:00
243d022620
cn6: update name and add to the shell
2021-02-15 17:44:20 +01:00
0ee2747215
garlicd: avoid no match fail
...
We check the result in the next if.
2021-02-15 16:32:06 +01:00
5fd2a62684
doc: update garlicd usage from the nix-shell
2021-02-15 16:22:45 +01:00
0e0bf9e7a7
garlic: add shell with the garlic tools
2021-02-15 16:22:06 +01:00
cb5bcd7097
garlicd: add to index and check for error
...
The garlicd is now available under garlic.garlid and it requires the
extra-sandbox-path option to be properly set.
2021-02-15 16:20:06 +01:00
d51fe5db48
garlic tool: ensure the mountpoint is enabled
2021-02-15 16:18:21 +01:00
c36b724e9a
Add experimental garlicd doc
2021-02-15 13:00:19 +01:00
cdf48181e5
user guide: add time measurement sections
2021-02-08 19:05:46 +01:00
a6b7b14d5e
user guide: add initialization time limit
2021-02-08 19:05:46 +01:00
2ca58c46b4
user guide: Add postprocessing section
2021-02-08 19:05:46 +01:00
25208a8158
user guide: add tar.gz target for the web
2021-02-08 19:05:19 +01:00
c46feb4bf2
user guide: use ms macros
...
Added HTML output
2021-02-08 19:05:19 +01:00
4d626bff97
user guide: test ms macros
2021-02-08 19:05:19 +01:00
042876a287
user guide: generate html with css
2021-02-08 19:05:19 +01:00
edd71815eb
pp: fix code block for html
2021-02-08 19:05:19 +01:00
39c360b413
user guide: add readme and branch name conventions
2021-02-08 19:04:51 +01:00
3ce0d3934b
user guide: reorder development
2021-02-08 19:04:45 +01:00
60cab85fc4
user guide: expand the develop section
2021-02-08 19:04:36 +01:00
95809bd2bf
user guide: add stub with mm macro
2021-02-08 19:04:30 +01:00
e5561b8735
control: save total execution time
2021-02-08 14:14:08 +01:00
d4dfbb7501
Remove garlic.ds attribute
...
Datasets are now handled directly when creating the figures, via the
timetable attribute.
2021-02-03 15:34:25 +01:00
b65a442cb0
fig: use timetable attribute in other plots
2021-02-03 15:34:10 +01:00
9c6b7a9f87
timetable: missing quote
2021-02-03 13:51:24 +01:00
d84ccf566b
launcher: fix typo
2021-02-03 13:51:04 +01:00
0faf22a43f
fig: add nbody example using timetable attribute
2021-02-03 13:07:55 +01:00
e89139284a
stdexp: add result and timetable targets
...
These targets allow one experiment to directly refer to another
experiment results, thus a dependency chain can be formed to ensure
execution order.
It also simplifies the dataset definition, as they can be automatically
fetched from the experiment directly.
2021-02-03 12:37:54 +01:00
b453c12253
pp: Add automatic launcher
2021-02-03 12:36:59 +01:00
32d8636ae1
timetable: prevent empty time lines
2021-02-03 12:27:23 +01:00
e4e427b7f6
garlicd: add daemon to launch experiments
2021-02-03 12:08:25 +01:00
fe760c0023
garlic: sent trebuchet output to stderr
2021-02-03 11:50:31 +01:00
4591eca1fd
garlic: Add blackbox diagram
2021-02-01 11:11:54 +01:00
9beda65778
lulesh: add experiment with all variants
2021-01-28 15:27:21 +01:00
0f62151dcf
lulesh: follow garlic git branches
2021-01-28 15:27:21 +01:00
0bc81c8943
tools: add range2 function
2021-01-28 15:00:43 +01:00
a3804e31f2
develop: Simplify packages
2021-01-26 14:59:57 +01:00
ed4a9e1bc3
Add branch diagram
2021-01-25 17:54:24 +01:00
3d0e93b4d3
Add slides pdf
2021-01-14 13:06:20 +01:00
8262fd3104
Add slides for meeting 3
2021-01-13 20:19:28 +01:00
2b9c3da911
Add script stage
2021-01-12 18:19:49 +01:00
aeac1a6068
exec: Force newlines
...
Allow single line commands like pre="true"
2021-01-11 19:15:37 +01:00
130fe39c8e
exec: Abort on error
...
We need exit on the first error, as otherwise we cannot track a bad
execution when no exec is done (when post is not empty).
2021-01-11 18:29:30 +01:00
892fb35d27
nbody: Fix infinite recursion
...
We want to override the previous layer (super), not the last one (self).
2021-01-11 14:30:12 +01:00
afd333adef
creams: fix indentation
2020-12-18 12:43:06 +01:00
76f2ef4b95
creams: add figures for scalability
2020-12-18 12:26:40 +01:00
3b80c2fcb9
creams: Merge hybrid and pure datasets into ss.all
2020-12-17 15:26:25 +01:00
b5cadefca9
Allow a space before time tag
...
This matches the fortran format for creams
2020-12-17 15:24:23 +01:00
Pedro Martinez
203dc9f295
Configure the nanos6 environment and get the right hardware attributes
2020-12-15 19:34:49 +01:00
Pedro Martinez
2e18761b48
Use the 'hw' attributes
2020-12-15 19:11:29 +01:00
Pedro Martinez
748d335a39
Define variables 'ntasksPerNode' and 'cpusPerTask' for each experiment and other minor changes
2020-12-15 19:11:29 +01:00
5a8cc1e514
stdexp: Run python snippets and import the result
2020-12-10 15:41:49 +01:00
7d4db6b6de
control: Exit on error
...
This prevents srun from silently returning with an error, without
actually queueing the job of a run.
2020-12-07 16:33:40 +01:00
756c5dff92
Update PM git server
2020-12-07 13:47:17 +01:00
9a0ea08d72
Reorganization
...
- All garlic stuff is moved into garlic/
- Group the overlay index by sections
- Add a garlic/default.nix link to the main default.nix, so we can
build derivations at garlic/
2020-12-07 13:33:42 +01:00
d70316a25a
fwi: disable nanos6 in ModelGenerator
2020-12-04 11:17:15 +01:00
eb4adf9520
ifsker: initial version
2020-12-03 18:49:28 +01:00
266fffdb5f
miniamr: initial version for OmpSs-2
2020-12-03 18:09:47 +01:00
f65e4d01c3
Simplify compiler name variables
2020-12-03 18:06:51 +01:00
1bdeca9e7d
unit: Remove dangerous slash from index names
2020-12-03 16:33:48 +01:00
5e9adf3fe6
nbody: Fix x label
2020-12-03 13:22:48 +01:00
c858f521bf
isolate: add $TMPDIR in the namespace
2020-12-03 13:22:10 +01:00
bdaadd4ef7
nbody: add ctf tests
2020-12-03 13:20:40 +01:00
b8a1ea3f72
develop: Fix inputrc missing key codes
2020-12-03 13:09:42 +01:00
eea9539258
develop: Set shell and hisfile
2020-12-03 12:14:04 +01:00
da4bbf8533
isolate: only load some files from /etc
2020-12-03 12:04:51 +01:00
df1f22c122
develop: support for srun
2020-12-02 13:38:43 +01:00
f87d830218
isolate: preserve TERM
2020-12-02 13:06:55 +01:00
3d352fee19
isolate: allow argument passing
2020-12-02 13:06:35 +01:00
284662d6cd
develop: fix bash PS1
2020-12-02 12:22:20 +01:00
1340d1d2e8
develop: Experimental interactive support
2020-12-02 11:58:00 +01:00
1f841649f8
exec: add support for nixPrefix
2020-12-02 11:57:40 +01:00
4000dbd0b8
Rename slides and generalize makefile
2020-11-24 18:05:16 +01:00
6fa3facfb1
Preliminar version for the slides
2020-11-24 17:58:14 +01:00
ed95cb0a04
Add wip presentation
2020-11-23 19:06:15 +01:00
aca7e36fc7
bigsort: add experiment with input generation
2020-11-20 15:41:27 +01:00
2153e58baf
bigsort: add the shuffle program
2020-11-20 15:39:34 +01:00
ceeb0f7f41
bigsort: add genseq program
2020-11-20 15:38:26 +01:00
a147a396d9
trebuchet: add the experiment as attribute
2020-11-20 15:35:36 +01:00
8bc5656461
tools: recursive getExperiment
...
It allows getExperimentStage to be called from any stage above the
experiment.
2020-11-20 15:34:14 +01:00
d192a59fdc
control: Export the run iteration
2020-11-20 15:32:41 +01:00
734d494d96
stdexp: Allow extra mounts
2020-11-20 15:30:47 +01:00
2863ab6ae1
machines: Use fs topology
2020-11-20 15:29:03 +01:00
4f0da10321
bigsort: Use cpusPerTask instead of cpuBind
2020-11-20 13:57:12 +01:00
David Alvarez
0c438d4dac
Setup for test experiment
2020-11-20 13:57:12 +01:00
David Alvarez
a0dac209e3
First test experiment
2020-11-20 13:57:12 +01:00
David Alvarez
37bd4c33f2
Add BigSort MPI+OpenMP
2020-11-20 13:57:12 +01:00
e8f649327a
exec: Avoid variable expansion at build
...
All bash variables passed in env, pre or post are now expanded at
execution time..
2020-11-20 13:54:45 +01:00
d2d3ccf332
Idea for FS naming convention
2020-11-17 18:33:57 +01:00
e1e34ddf75
exec: add pre and post code to allow cleanup tasks
2020-11-17 16:09:38 +01:00
33f6ae7e55
Add bundled report example
2020-11-17 15:51:09 +01:00
fe0bd8b200
creams: fix pure experiment
...
Use machine agnostic specification for resources
2020-11-17 12:31:03 +01:00
016422cede
Update nbody experiment
...
Generate the input based on the target machine description.
2020-11-17 11:26:35 +01:00
5e50ef19fe
Update experiments with cpusPerTask
...
Try to avoid manually setting the hardware specs and rather use
the hw attrset.
2020-11-17 11:17:57 +01:00
641e752bd5
Add a trace message at unit evaluation
2020-11-17 11:12:12 +01:00
74537e682c
Use divisors in the slurm cpu experiment
2020-11-17 11:01:34 +01:00
433c8864ea
Add divisor generator
2020-11-17 11:01:34 +01:00
e0ca33569b
garlic tools: rename divList -> halfList
2020-11-17 11:01:34 +01:00
dea523460a
Add slurm affinity experiment
2020-11-17 11:01:34 +01:00
b4a3bb0ede
New stdexp resource specification
...
Now the options cpusPerTask ntasksPerNode nodes and jobName are required
for the sbatch stage. Also cpuBind has been removed and is always set to
"cores,verbose" in the srun stage.
2020-11-17 11:01:34 +01:00
dabc6be640
Add more helper functions
2020-11-17 11:01:34 +01:00
18afcb1f44
Avoid nixpkgs reevaluation
...
The bsc attrset is now extensible: replacing a few bsc packages is very
fast. Also we allow the complete bscpkgs to be within other custom
overlays (not tested yet).
2020-11-17 10:49:45 +01:00
288318b556
Merge branch 'saiph' into 'master'
...
Saiph
See merge request rarias/bscpkgs!6
2020-11-13 10:25:47 +01:00
42f2227a9f
sbatch: Use experiment reservation if given
2020-11-13 10:17:54 +01:00
Sandra
4ae66adb9a
Saiph: adding granularity experiment and figures
2020-11-13 09:56:40 +01:00
Sandra
86d1d426ec
Saiph: Removing devMode parameter
2020-11-12 19:10:43 +01:00
5333058741
report: build only required figures
...
Introduces a intermediate derivation that can be imported into the
report derivation, which contains a string cmd that expands the fig
variable as needed.
2020-11-11 19:03:02 +01:00
0b0f6ac9f0
rplot: Add a reference to the dataset
2020-11-11 18:59:57 +01:00
966606b62d
hpcg: precompute the input
2020-11-09 17:48:46 +01:00
5763b91d39
Use the trebuchet only to specify an experiment
2020-11-09 17:46:11 +01:00
47f67dcd85
Extend abstract
2020-11-09 12:16:56 +01:00
48869d6e4a
Clarify some sections
2020-11-09 12:09:54 +01:00
92f58651b8
Increase margins and enable utf8 targets
2020-11-09 12:09:22 +01:00
31f7d17a41
Add manual diagram for nroff
2020-11-06 14:26:11 +01:00
dec183b221
Fix execution out path
2020-11-06 12:31:39 +01:00
92eee2ede8
Use target machine notation
2020-11-06 12:31:31 +01:00
a8208480c1
Add a nix shell for playing with plots
2020-11-05 20:01:26 +01:00
dd0823876a
hpcg: add plot for oss experiment
2020-11-05 19:59:47 +01:00
11ac02da08
heat: Add test experiment and plot
2020-11-05 19:56:26 +01:00
074a75facb
saiph: name the experiment and units in numcomm
2020-11-05 19:53:38 +01:00
7a80d1ca98
heat: Use clang by default
2020-11-05 19:52:37 +01:00
9e477a2313
hpcg: smaller input size
2020-11-05 19:46:34 +01:00
5bd042ef67
nbody: mark the points with bad std
2020-11-05 19:43:39 +01:00
d7be13f88d
Update garlic options in store stage
2020-11-05 19:38:45 +01:00
476c2f20f0
Add manual and update the garlic tool
2020-11-05 19:31:21 +01:00
de6b4864ee
Add garlic tool manual
2020-11-05 19:29:40 +01:00
33682ef48d
Document the results and pp stages
2020-11-05 14:52:57 +01:00
634d2040b5
Add reference index
2020-11-04 13:00:42 +01:00
df4d908f1c
Add more rendered files to ignore
2020-11-04 12:57:22 +01:00
f0122d557f
WIP: postprocessing doc
2020-11-04 12:56:35 +01:00
0c58bb63b5
hpcg: add exp and unit name
2020-11-03 19:10:00 +01:00
de46366985
nbody: plot nb/cpu rather than nb
2020-11-03 19:10:00 +01:00
5eea48c5b0
Add exp and unit name to nbody tampi experiment
2020-11-03 19:10:00 +01:00
c3988dacd2
WIP: documentation for the pp pipeline
2020-11-03 19:10:00 +01:00
317409f6ac
Move index and out inside the user directory
2020-11-03 19:10:00 +01:00
3eae92bdc4
Remove old pp stages
2020-11-03 19:10:00 +01:00
6b40e6f9e9
Experimental garlic tool
2020-11-03 19:10:00 +01:00
d5d42b3c09
Add unit and exp name to nbody test
2020-11-03 19:10:00 +01:00
0bcfe5d25b
Add new store pp stage
2020-11-03 19:10:00 +01:00
5e2797bcde
Create index files for the experiments
2020-11-03 19:10:00 +01:00
efd7df068e
Print full experiment path
2020-11-03 19:10:00 +01:00
7c5345f4bc
report: Idea to reduce build time
2020-11-03 19:10:00 +01:00
43991e9173
nbody: plot nblocks in test
2020-11-03 19:10:00 +01:00
7b26b59988
Use rsync to fetch only the logs
2020-11-03 19:10:00 +01:00
a66cdb52fb
nbody: Fix test experiment
2020-11-03 19:10:00 +01:00
3bd4e61f3f
WIP: Testing with automatic fetching
2020-11-03 19:09:59 +01:00
59346fa97e
control: Add status file
2020-11-03 19:09:59 +01:00
fd1229ddc0
nbody: add simple test figure
2020-11-03 19:09:59 +01:00
8ce88ef046
Add dataset attrset in garlic
...
Modify nbody to evenly distribute blocks per cpu
2020-11-03 19:09:59 +01:00
06c29b573f
Add exp.nbody.tampi variants
2020-11-03 19:09:59 +01:00
7852d86a3f
Fix plot details
2020-11-03 19:09:59 +01:00
4beb069627
WIP: postprocessing pipeline
...
Now each run is executed in a independent folder
2020-11-03 19:09:59 +01:00
1321b6a888
Add experiments with jemalloc and CPU affinity
2020-11-03 19:09:59 +01:00
81d144d716
Remove exp attrset from report
...
Fixes #43
2020-11-03 19:09:59 +01:00
30ad4219d9
Add example report
2020-11-03 19:09:59 +01:00
067fb0c0a2
Add R shell for quick plots
2020-11-03 19:09:59 +01:00
308673f7f6
Increase nbody test cases
2020-11-03 19:09:59 +01:00
1bd9cb6c0f
Move the plot script to R
2020-11-03 19:09:59 +01:00
ede25b6736
Use the stage names as inputs
2020-11-03 19:09:59 +01:00
2680dcb66f
Don't nest the unit results
...
The experiment directory now contains symlinks to the units, keeping the
old structure. The unit results are directly placed in the garlic out
directory.
2020-11-03 19:09:58 +01:00
f33137a55e
WIP: Add experimental figure pipeline
2020-11-03 19:09:58 +01:00
65745e0aaf
WIP: Add another nbody experiment
2020-11-03 19:09:58 +01:00
c3659d316d
Add perf stage
2020-11-03 19:09:58 +01:00
4f901c1b9c
WIP: add postprocessing stages
2020-11-03 19:09:58 +01:00
