jungle/web/content/intro-nix/ch3.md

title	date	weight
Chapter 3: Custom packages	2023-09-15	3

Adding more packages

So far we have define all the packages using:

pkgs.mkShell rec {
  pname = "my-shell";
  buildInputs = with pkgs.bsc; [
    ovni osumb # other packages here...
  ];
};

This line specifies that all packages come from the pkgs.bsc set. We can add additional packages adding them to the list:

pkgs.mkShell rec {
  pname = "my-shell";
  buildInputs = with pkgs.bsc; [
    ovni osumb sonar
  ];
};

And running nix develop again:

hut% nix develop
warning: Git tree '/home/Computational/rarias/jungle-examples' is dirty
[nix-develop]$ printf '%s\n' $buildInputs
/nix/store/0yzas8007x9djlpbb0pckcr1vhd0mcfy-ovni-1.3.0
/nix/store/lnjirzllhjn2fadlqzrz7a547iawl8jc-osu-micro-benchmarks-7.1-1
/nix/store/fjxj4xs0wblw3jyhp4vsrsfnlfwawifa-sonar-0.1.0

In the jungle cluster, the default MPI implementation is currently set to MPICH, as it can be shown with ldd:

[nix-develop]$ ldd $(which ovnisync) | grep mpi
        libmpi.so.12 => /nix/store/nnnaly6hgylravdrmqkhpx1ndg5p79nc-mpich-4.1.2/lib/libmpi.so.12 (0x00007ffff5200000)

Now, what if we want to replace the MPI implementation by another one?

Modifying a package

You notice that the packages we are using are coming directly from the ones specified in jungle. However, what if we need to modify some option at build time or change a dependency?

The Nix language is used to describe how to build each package, and can be extended to create derived versions very easily.

Let's focus on the ovni package. First, to load the definition we can use the nix edit command, which opens the definition file using the editor defined in $EDITOR:

hut% nix edit jungle#bsc.ovni
...

This particular package has several inputs that can be modified directly:

{
  stdenv
, lib
, cmake
, mpi
, fetchFromGitHub
, useGit ? false
, gitBranch ? "master"
, gitUrl ? "ssh://git@bscpm03.bsc.es/rarias/ovni.git"
, gitCommit ? "d0a47783f20f8b177a48418966dae45454193a6a"
, enableDebug ? false
}:
...

For example, the enableDebug flag, currently set to false, affects how the build is configured:

cmakeBuildType = if (enableDebug) then "Debug" else "Release";

Now, to change this option we could replace ovni for our version:

{
  inputs.jungle.url = "jungle";
  nixConfig.bash-prompt = "\[nix-develop\]$ ";

  outputs = { self, jungle }:  
  let
    pkgs = jungle.outputs.packages.x86_64-linux;
    ovniDebug = pkgs.bsc.ovni.override { enableDebug = true; };
  in {
    devShells.x86_64-linux.default = pkgs.mkShell rec {
      pname = "my-shell";
      buildInputs = with pkgs.bsc; [
        ovniDebug osumb sonar
      ];
    };
  };
}

And then, when we now enter the develop shell we can see that ovni gets build with the Debug option:

hut% nix develop -L
warning: Git tree '/home/Computational/rarias/jungle-examples' is dirty
ovni> unpacking sources
ovni> unpacking source archive /nix/store/cz4si0vsw85r9s6dyiqr5ybngh9aympi-source
ovni> source root is source
ovni> patching sources
ovni> updateAutotoolsGnuConfigScriptsPhase
ovni> configuring
ovni> fixing cmake files...
ovni> cmake flags: ... -DCMAKE_BUILD_TYPE=Debug ...
...
[nix-develop]$ which ovniver
/nix/store/hg0xs7fpibwjhsp9ajqfcbffsh69mrsm-ovni-1.3.0/bin/ovniver

[nix-develop]$ file $(which ovniver) | fold
/nix/store/hg0xs7fpibwjhsp9ajqfcbffsh69mrsm-ovni-1.3.0/bin/ovniver: ELF 64-bit L
SB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /nix/st
ore/9la894yvmmksqlapd4v16wvxpaw3rg70-glibc-2.37-8/lib/ld-linux-x86-64.so.2, for
GNU/Linux 3.10.0, with debug_info, not stripped

And we see that the ovniver program is now compiled with debug symbols.

However, this only replaces the ovni package that we specify in the shell. The sonar library also depends on ovni, but that package is still using the old one:

[nix-develop]$ find $buildInputs -name 'libovni.so.1'
/nix/store/hg0xs7fpibwjhsp9ajqfcbffsh69mrsm-ovni-1.3.0/lib/libovni.so.1

[nix-develop]$ find $buildInputs -name 'libsonar-mpi.so'
/nix/store/fjxj4xs0wblw3jyhp4vsrsfnlfwawifa-sonar-0.1.0/lib/libsonar-mpi.so

[nix-develop]$ ldd /nix/store/fjxj4xs0wblw3jyhp4vsrsfnlfwawifa-sonar-0.1.0/lib/libsonar-mpi.so | grep ovni
        libovni.so.1 => /nix/store/0yzas8007x9djlpbb0pckcr1vhd0mcfy-ovni-1.3.0/lib/libovni.so.1 (0x00007ffff7f8d000)

In the next chapter we will see how to replace packages in such a way that all the dependences are automatically updated too.