forked from rarias/jungle
101 lines
3.3 KiB
Markdown
101 lines
3.3 KiB
Markdown
---
|
|
title: "Chapter 1: Packages"
|
|
description: "Here we show where packages come from"
|
|
date: 2023-06-13T19:36:57+02:00
|
|
weight: 1
|
|
---
|
|
|
|
In this chapter we describe where the packages available in the cluster come
|
|
from and how to load them.
|
|
|
|
## Where packages come from
|
|
|
|
The packages in the jungle cluster are constructed by *layers*. Each layer
|
|
applies some changes over the previous one:
|
|
|
|
1. The first layer is [nixpkgs][1], a large repository of packages maintained by
|
|
the NixOS community. It provides packages like gcc, bash, gcc or the linux
|
|
kernel.
|
|
|
|
[1]: https://github.com/NixOS/nixpkgs/
|
|
|
|
2. The second layer is [bscpkgs][2], it takes the nixpkgs set of packages and
|
|
expands it by adding custom packages from the BSC such as Nanos6, nOS-V,
|
|
NODES, ovni or wxparaver.
|
|
|
|
[2]: https://pm.bsc.es/gitlab/rarias/bscpkgs
|
|
|
|
3. The third layer is [jungle][3], it takes the extended packages from bscpkgs
|
|
and configures them for the jungle cluster. For example, we configure MPICH
|
|
to use the OmniPath network and set it as the default implementation.
|
|
|
|
[3]: https://pm.bsc.es/gitlab/rarias/jungle
|
|
|
|
These layers are called *overlays* in Nix and they are the default mechanism
|
|
used to modify the packages. Generally you will use the packages defined in the
|
|
last layer (jungle) but you can define your own additional layer to specify
|
|
custom changes. For example, instead of choosing MPICH, you may want to use
|
|
Intel MPI instead by default.
|
|
|
|
## Loading packages in an ephemeral shell
|
|
|
|
You can manually load packages in a *new* shell with `nix shell jungle#<pkg>`,
|
|
for example:
|
|
|
|
```
|
|
hut% which ovniemu
|
|
ovniemu not found
|
|
hut% nix shell jungle#bsc.ovni
|
|
hut% which ovniemu
|
|
/nix/store/0yzas8007x9djlpbb0pckcr1vhd0mcfy-ovni-1.3.0/bin/ovniemu
|
|
hut% exit
|
|
hut%
|
|
```
|
|
|
|
You can also specify multiple packages by listing them as parameters of `nix
|
|
shell`:
|
|
|
|
```
|
|
hut% nix shell jungle#bsc.ovni jungle#bsc.osumb
|
|
hut% which osu_bw
|
|
/nix/store/lnjirzllhjn2fadlqzrz7a547iawl8jc-osu-micro-benchmarks-7.1-1/bin/osu_bw
|
|
hut% exit
|
|
```
|
|
|
|
Or make the bash (zsh in this case) shell expand them:
|
|
|
|
```
|
|
hut% echo nix shell jungle#bsc.{ovni,osumb}
|
|
nix shell jungle#bsc.ovni jungle#bsc.osumb
|
|
hut% nix shell jungle#bsc.{ovni,osumb}
|
|
hut% which osu_bw
|
|
/nix/store/lnjirzllhjn2fadlqzrz7a547iawl8jc-osu-micro-benchmarks-7.1-1/bin/osu_bw
|
|
hut% exit
|
|
```
|
|
|
|
You can use TAB to see which packages are available:
|
|
|
|
```
|
|
hut% nix shell jungle#bsc.n<TAB>
|
|
jungle\#bsc.nanos6 jungle\#bsc.nixtools
|
|
jungle\#bsc.nanos6Debug jungle\#bsc.nix-wrap
|
|
jungle\#bsc.nanos6Git jungle\#bsc.nodes
|
|
jungle\#bsc.nanos6GlibcxxDebug jungle\#bsc.nodesGit
|
|
jungle\#bsc.nanos6-icc jungle\#bsc.nodesRelease
|
|
jungle\#bsc.nanos6-icx jungle\#bsc.nodesWithOvni
|
|
jungle\#bsc.nanos6Release jungle\#bsc.nosv
|
|
jungle\#bsc.nix-mn4
|
|
```
|
|
|
|
Notice that these packages are evaluated at the moment the command is invoked.
|
|
So if you come back a month later and run the same command, you may find that
|
|
the packages have been updated and that could be problematic.
|
|
|
|
In the next section we will create a new flake that defines the packages of the
|
|
shell and also records the exact version of the packages that we used at the
|
|
evaluation time for future use.
|
|
|
|
In the [next chapter](../ch2) we will see how to create a permanent shell that
|
|
will retain the same packages even if they are upgraded in the cluster, until we
|
|
decide to upgrade them.
|