rarias/jungle
rarias/jungle
2
0
Fork 2
Code Issues 54 Pull Requests 9 Actions Packages Projects Releases Wiki Activity

Add details for xeon07

Browse Source
This commit is contained in:
Rodrigo Arias 2020-07-06 11:19:20 +02:00
parent 018bebc264
commit a95f7fa35e
1 changed files with 116 additions and 78 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

192
README
View File

@ -11,34 +11,50 @@
Some preliminary steps must be done manually to be able to build and Some preliminary steps must be done manually to be able to build and
install packages (derivations in nix jargon). install packages (derivations in nix jargon).
This guide is specific for the nix installation in the xeon07 node,
accessed by the ssfhead.bsc.es login node, but it may be used in other
machines as well.
To easily connect to xeon07 in one step, setup the SSH (for version
7.3 and upwards) configuration file in ~/.ssh/config adding these
lines:
Host cobi
HostName ssflogin.bsc.es
User your-username-here
Host xeon07
ProxyJump cobi
HostName xeon07
User your-username-here
You should be able to connect with:
$ ssh xeon07
1.1 Network access 1.1 Network access
In order to use nix you need to be able to download the sources from In order to use nix would you need to be able to download the sources
Internet. Usually the download requires the ports 22, 80 and 443 to be from Internet. Usually the download requires the ports 22, 80 and 443
open for outgoing traffic. to be open for outgoing traffic.
Unfortunately, in some clusters (as is the case in xeon07) access to Unfortunately, in some clusters (as is the case in xeon07) access to
Internet is disabled. However you can tunnel the connection by SSH to Internet is disabled. However you can tunnel the connection by SSH to
your local machine, and then reach the Internet. your local machine, and then reach the Internet.
There are some guides on how to prepare the proxy server and the In order to tell nix to use the proxy connection, you will need to
tunnel in SSH such as: export the "https_proxy" and "http_proxy" variables. A proxy
connection is already configured in xeon07 and you can automatically
set those variables to the correct address by loading:
https://www.seniorlinuxadmin.co.uk/ssh-over-proxy.html xeon07$ . /scratch/nix/internet
In order to instruct nix to use the proxy connection, you will need to
export the https_proxy and http_proxy variables. In the xeon07 node is
already configured and you can automatically set those variables to
the correct address by loading:
$ . /scratch/nix/internet
Consider adding the command to your ~/.bashrc file so you don't need Consider adding the command to your ~/.bashrc file so you don't need
to do it every time you want to use nix. to do it every time you want to use nix.
Now you should be able to reach the outside world by running: Now you should be able to reach the outside world by running:
$ curl google.com xeon07$ curl google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8"> <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY> <TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1> <H1>301 Moved</H1>
@ -53,13 +69,13 @@
prompt. prompt.
Most repositories at https://pm.bsc.es/gitlab are open to read for Most repositories at https://pm.bsc.es/gitlab are open to read for
logged users, but there are some exceptions for example the nanos6 logged in users, but there are some exceptions (for example the nanos6
repository where you must have explicitly granted read access. repository) where you must have explicitly granted read access.
If you don't have a ssh key at ~/.ssh/*.pub create a new one without If you don't have a ssh key at ~/.ssh/*.pub in xeon07 create a new one
password protection by running: without password protection by running:
$ ssh-keygen xeon07$ ssh-keygen
Generating public/private rsa key pair. Generating public/private rsa key pair.
Enter file in which to save the key (~/.ssh/id_rsa): Enter file in which to save the key (~/.ssh/id_rsa):
Enter passphrase (empty for no passphrase): Enter passphrase (empty for no passphrase):
@ -74,17 +90,15 @@
https://pm.bsc.es/gitlab/profile/keys https://pm.bsc.es/gitlab/profile/keys
If you want to select another key rather than the default Then, configure it for use in the ~/.ssh/config file, adding:
~/.ssh/id_rsa then you must configure it for use in the ~/.ssh/config
file, adding:
Host bscpm02.bsc.es Host bscpm02.bsc.es
IdentityFile ~/.ssh/your-private-key IdentityFile ~/.ssh/id_rsa
Finally verify the SSH connection to the server works and you get a Finally verify the SSH connection to the server works and you get a
greeting from the GitLab server with your username: greeting from the GitLab server with your username:
$ ssh git@bscpm02.bsc.es xeon07$ ssh git@bscpm02.bsc.es
PTY allocation request failed on channel 0 PTY allocation request failed on channel 0
Welcome to GitLab, @rarias! Welcome to GitLab, @rarias!
Connection to bscpm02.bsc.es closed. Connection to bscpm02.bsc.es closed.
@ -97,32 +111,32 @@
Finally, you should be able to download the rarias/nanos6 git Finally, you should be able to download the rarias/nanos6 git
repository without any password interaction by running: repository without any password interaction by running:
$ git clone git@bscpm02-bsc-es:rarias/nanos6.git xeon07$ git clone git@bscpm02-bsc-es:rarias/nanos6.git
1.3 Prepare the bsc-nixpkgs repo 1.3 Prepare the bsc-nixpkgs repo
Once you have Internet and you have granted access to the PM GitLab Once you have Internet and you have granted access to the PM GitLab
repositories you can begin down the rabbit hole of nix. First ensure repositories you can begin down the rabbit hole of nix. First ensure
that the nix binaries are available in your machine: that the nix binaries are available from your shell in xeon07:
$ nix --version xeon07$ nix --version
nix (Nix) 2.3.6 nix (Nix) 2.3.6
Now you are set to install packages with nix. Clone the bsc-nixpkgs Now you are set to install packages with nix. Clone the bsc-nixpkgs
repository: repository:
$ git clone git@bscpm02-bsc-es:rarias/bsc-nixpkgs.git xeon07$ git clone git@bscpm02-bsc-es:rarias/bsc-nixpkgs.git
Nix looks in the current folder for a file named "default.nix" for Nix looks in the current folder for a file named "default.nix" for
packages, so go to the repo: packages, so go to the repo:
$ cd bsc-nixpkgs xeon07$ cd bsc-nixpkgs
Now you should be able to build nanos6 from the git repository: Now you should be able to build nanos6 from the git repository:
$ nix-build -A bsc.nanos6-git xeon07$ nix-build -A bsc.nanos6-git
The output is placed in the "result" symlink. The output is placed in the "result" symbolic link.
2. Basic usage of nix 2. Basic usage of nix
@ -138,10 +152,10 @@
All nix packages are stored under the /nix directory. When you need to All nix packages are stored under the /nix directory. When you need to
"install" some binary from nix, a symlink is added to a folder "install" some binary from nix, a symlink is added to a folder
included in the $PATH variable. In particular, you should have the included in the $PATH variable. In particular, you should have
something similar added to your $PATH: something similar added to your $PATH:
$ echo $PATH | sed 's/:/\n/g' | grep nix xeon07$ echo $PATH | sed 's/:/\n/g' | grep nix
/home/Computational/rarias/.nix-profile/bin /home/Computational/rarias/.nix-profile/bin
/nix/var/nix/profiles/default/bin /nix/var/nix/profiles/default/bin
@ -150,17 +164,35 @@
which contains the nix tools (which are installed in the /nix which contains the nix tools (which are installed in the /nix
directory as well). directory as well).
Use `nix search` to look for official packages in the "nixpkgs"
channel (the default repository of packages):
xeon07$ nix search cowsay
warning: using cached results; pass '-u' to update the cache
* cowsay (cowsay)
A program which generates ASCII pictures of a cow with a message
* neo-cowsay (neo-cowsay)
Cowsay reborn, written in Go
* ponysay (ponysay-3.0.3)
Cowsay reimplemention for ponies
* tewisay (tewisay-unstable-2017-04-14)
Cowsay replacement with unicode and partial ansi escape support
When you need a program that is not available in your environment, When you need a program that is not available in your environment,
much like when you use "module load ..." you can use nix-env to modify much like when you use "module load ..." you can use nix-env to modify
what is currently loaded. For example: what is currently loaded. For example:
$ nix-env -iA nixpkgs.cowsay xeon07$ nix-env -iA nixpkgs.cowsay
The command will download (if not found already in the nix store), Notice that you should specify the prefix "nixpkgs." before. The
compile (if necessary) and load the program `cowsay` from the nixpkgs command will download (if not found already in the nix store), compile
(if necessary) and load the program `cowsay` from the nixpkgs
repository in the environment. You should be able to run it as: repository in the environment. You should be able to run it as:
$ cowsay "hello world" xeon07$ cowsay "hello world"
_____________ _____________
< hello world > < hello world >
------------- -------------
@ -170,17 +202,17 @@
||----w | ||----w |
|| || || ||
You can inspect now the ~/.nix-profile/bin folder, and see that a new You can now inspect the ~/.nix-profile/bin folder, and see that a new
symlink was added to the actual installation of the binary: symlink was added to the actual installation of the binary:
$ file ~/.nix-profile/bin/cowsay xeon07$ file ~/.nix-profile/bin/cowsay
/home/Computational/rarias/.nix-profile/bin/cowsay: symbolic link to /home/Computational/rarias/.nix-profile/bin/cowsay: symbolic link to
`/nix/store/673gczmhr5b449521srz2n7g1klykz6n-cowsay-3.03+dfsg2/bin/cowsay' `/nix/store/673gczmhr5b449521srz2n7g1klykz6n-cowsay-3.03+dfsg2/bin/cowsay'
You can list the current packages installed in your environment by You can list the current packages installed in your environment by
running: running:
$ nix-env -q xeon07$ nix-env -q
cowsay-3.03+dfsg2 cowsay-3.03+dfsg2
nix-2.3.6 nix-2.3.6
@ -191,7 +223,7 @@
You can remove any package from the environment using: You can remove any package from the environment using:
$ nix-env -e cowsay xeon07$ nix-env -e cowsay
See the manual with `nix-env --help` if you want to know more details. See the manual with `nix-env --help` if you want to know more details.
@ -203,63 +235,67 @@
any binary cache server yet. any binary cache server yet.
Nix will handle the build process without any user interaction (with a Nix will handle the build process without any user interaction (with a
few exceptions stated later, which you shouldn't have to worry). If few exceptions which you shouldn't have to worry). If any other user
any other user has already built the package then the build process is has already built the package then the build process is not needed,
not needed, and the package is used as is. and the package is used as is.
In order to build a BSC package go to the `bsc-nixpkgs` directory, and In order to build a BSC package go to the `bsc-nixpkgs` directory, and
run: run:
$ nix-build -A bsc.dummy xeon07$ nix-build -A bsc.dummy
The package will be built and installed in the /nix directory, then a Notice the "bsc." prefix for BSC packages. The package will be built
symlink is placed in the result directory: and installed in the /nix directory, then a symlink is placed in the
result directory:
$ find result/ -type f xeon07$ find result/ -type f
result/ result/
result/bin result/bin
result/bin/dummy result/bin/dummy
The way in which nix handles the packages and dependencies ensures The way in which nix handles the packages and dependencies ensures
that the output of a compilation of any package is exactly the same. that the environment of the build process of any package is exactly
the same, so the generated output should be the same if the builds are
deterministic.
You can check the reproducibility of the build by adding the "--check" You can check the reproducibility of the build by adding the "--check"
flag, which will rebuild the package and compare the checksum of every flag, which will rebuild the package and compare the checksum of every
file with the ones installed: file with the ones previously built:
$ nix-build -A bsc.dummy --check xeon07$ nix-build -A bsc.dummy --check
... ...
$ echo $? xeon07$ echo $?
0 0
A return code of zero ensures the output is bit by bit identical to A return code of zero ensures the output is bit by bit identical to
the one installed. There are some packages that try to include the one installed. There are some packages that include
non-reproducible information in the build process as the timestamp indeterministic information in the build process (such as the
which will produce an error. Those packages must be patched to ensure timestamp of the current time) which will produce an error. Those
the output is deterministic. packages must be patched to ensure the output is deterministic.
Notice that if you "cd" into the result/ directory you will be at /nix Notice that if you "cd" into the "result/" directory you will be at
directory (as you follow the symlink) where you don't have write /nix directory (as you have follow the symlink) where you don't have
permission. Therefore if your program attempts to write to the current write permission. Therefore if your program attempts to write to the
directory it will fail. It is recommended to instead run your program current directory it will fail. It is recommended to instead run your
from the top directory: program from the top directory:
$ result/bin/dummy xeon07$ result/bin/dummy
Hello world! Hello world!
Or you can install it in the environment: Or you can install it in the environment:
$ nix-env -i ./result xeon07$ nix-env -i ./result
And "cd" into any directory where you want to output some files and And "cd" into any directory where you want to output some files and
just run it by the name: just run it by the name:
$ cd /tmp xeon07$ cd /tmp
$ dummy xeon07$ dummy
Hello world! Hello world!
Finally, you can remove it from the environment if you don't need it: Finally, you can remove it from the environment if you don't need it:
$ nix-env -e dummy xeon07$ nix-env -e dummy
If you want to know more details use "nix-build --help" to see the If you want to know more details use "nix-build --help" to see the
manual. manual.
@ -267,18 +303,20 @@
2.3 The build process 2.3 The build process
Each package is built following a programmable configuration Each package is built following a programmable configuration
description in the nix language. Build in nix are performed under very description in the nix language. Builds in nix are performed under
strict conditions. No access any file in the file system is allowed, very strict conditions. No access to any file in the file system is
unless stated in the dependencies of the package which are then allowed, unless stated in the dependencies, which are in the /nix
available in the build environment. store only.
There is no network access in the build process and other restrictions There is no network access in the build process and other restrictions
are enforced that the build is deterministic. See more details here: are enforced so that the build environment is reproducible. See more
details here:
https://nixos.wiki/wiki/Nix#Sandboxing https://nixos.wiki/wiki/Nix#Sandboxing
In the "default.nix" file of the bsc-nixpkgs you can see the The top level "default.nix" file of the bsc-nixpkgs serves as a index
definition for each package, for example the nbody app: of all BSC packages. You can see the definition for each package, for
example the nbody app:
nbody = callPackage ./bsc/apps/nbody/default.nix { nbody = callPackage ./bsc/apps/nbody/default.nix {
stdenv = pkgs.gcc9Stdenv; stdenv = pkgs.gcc9Stdenv;
@ -304,12 +342,12 @@
Then you can rebuild the nbody package: Then you can rebuild the nbody package:
$ nix-build -A bsc.nbody xeon07$ nix-build -A bsc.nbody
... ...
And verify that the binary is indeed linked to MPICH now: And verify that the binary is indeed linked to MPICH now:
$ ldd result/bin/nbody_mpi.N2.2048.exe | grep mpi xeon07$ ldd result/bin/nbody_mpi.N2.2048.exe | grep mpi
libmpi.so.12 => /nix/store/dwkkcv78a5bs8smflpx9ppp3klhz3i98-mpich-3.3.2/lib/libmpi.so.12 (0x00007f6be0f07000) libmpi.so.12 => /nix/store/dwkkcv78a5bs8smflpx9ppp3klhz3i98-mpich-3.3.2/lib/libmpi.so.12 (0x00007f6be0f07000)
If you modify a package which another package requires as a If you modify a package which another package requires as a
@ -333,9 +371,9 @@
The build process now is not required: The build process now is not required:
$ nix-build -A bsc.nbody xeon07$ nix-build -A bsc.nbody
/nix/store/rbq7wrjcmg6fzd6yhrlnkfvzcavdbdpc-nbody /nix/store/rbq7wrjcmg6fzd6yhrlnkfvzcavdbdpc-nbody
$ ldd result/bin/nbody_mpi.N2.2048.exe | grep mpi xeon07$ ldd result/bin/nbody_mpi.N2.2048.exe | grep mpi
libmpifort.so.12 => /nix/store/jvsjvxj2a08340fpdrqbqix9z3mpp3bd-intel-mpi-2019.7.217/lib/libmpifort.so.12 (0x00007f3a00402000) libmpifort.so.12 => /nix/store/jvsjvxj2a08340fpdrqbqix9z3mpp3bd-intel-mpi-2019.7.217/lib/libmpifort.so.12 (0x00007f3a00402000)
libmpi.so.12 => /nix/store/jvsjvxj2a08340fpdrqbqix9z3mpp3bd-intel-mpi-2019.7.217/lib/libmpi.so.12 (0x00007f39fed34000) libmpi.so.12 => /nix/store/jvsjvxj2a08340fpdrqbqix9z3mpp3bd-intel-mpi-2019.7.217/lib/libmpi.so.12 (0x00007f39fed34000)