rarias/jungle
2
0
Fork 3
Code Issues 57 Pull Requests 7 Actions Packages Projects Releases Wiki Activity

Remove blobs #185

Closed
rarias wants to merge 459 commits from remove-blobs into old-master
pull from: remove-blobs
merge into: rarias:old-master
Conversation 2 Commits 459 Files Changed 290 +32347 -470
rarias commented 2025-10-01 16:43:02 +02:00
Owner
Copy Link

Fixes #171

Fixes #171
rarias added 459 commits 2025-10-01 16:43:03 +02:00
Add common directory 15906e5818
Move disk selection to configuration.nix fa757aaaab
Move boot.nix to common 13f72753a5
Move fs.nix to common d6a7a87207
Use partition labels for / and swap f5fa915e09
Move filesystems config to common/fs.nix 1432a26fba
Move boot config to common/boot.nix 32d782be96
Move the remaining hw config to common b739c96882
Move common options from configuration.nix 774350f288
Move users.nix to common ab978758ba
Move overlays.nix to common 020ce58efd
Move ssh.nix to common 51a77b5213
Remove host specific network options from net.nix a476b1758b
Move net.nix to common 7829ba7509
Load overlays from /config ce00282704
Add configuration for xeon01 6f80ebb483
Add script to rebuild configuration f0abbac1c7
Remove xeon07 overlay to load upstream slurm 1b030ab76d
Use xeon07 as control machine a2fe4b552a
Enable slurm in xeon01 513c182d24
Test flakes 0c296c1825
Lock flakes and add inputs 8b4dda98af
Refacto slurm configuration into compute/control 99481e6203
Add xeon02 configuration 2d204c5cc0
Add minimal netboot module to build kexec image 5c9f5e845f
Add steps in install documentation 9267ca51fa
Update ib interface name in xeon02 32b84a5add 
It seems to be plugged in another PCI port
Update nixpkgs b7d52423c9
Update nixpkgs to nixos-unstable 8851dbe212
Disable osnoise and hwlat tracer for now 30a12729f1 
Reuse nix cache to avoid rebuilding the kernel.
Use the latest kernel e7bfba54bc
Increase locked memory to 1 GiB 7b176a3780
Use pmix by default in slurm 70d2255634
Roolback to bash as default shell a6c060a25b 
Zsh doesn't behave properly, it needs further configuration.
Simplify bash prompt 847d516d6d
Allow 5 concurrent buils in the gitlab-runner 81243cbc95
Increase the number of CPUs to 56 for nOS-V docker 61ea1af68b
Add hal ssh key 7828d616fc
Allow public dashboards in grafana f2fc2af77a
Automatically resume restarted nodes in SLURM be90239bbc
Add nixos-config.nix to easily enable nix repl 0a8097231d
Add xeon08 basic config a9f9a1e8e5
Set intel_pstate=passive and disable frequency boost 769317c5a7
Add gitignore 35898c68e7
Improve documentation af03554610
Add ix to common packages 471d44f013
Add cmake to system packages 038340c5d2
Add gnumake to system packages f0af9d87a0
Add file to system packages 26caa390cb
Add tree command 68f7c02555
Serve grafana in https://jungle.bsc.es/grafana b959c72979
xeon08: Add lttng module and tools 0d1a7e59ee
xeon08: Enable lttng lockdep tracepoints 459b29924c
xeon08: Add perf 49ccf0a3f3
xeon08: Add config for kernel non-voluntary preemption 0c4b85be3b
Move arocanon user from xeon08 to common eb9b5e570f
Add ncdu to system packages eae7ed4e10
Remove profiles older than 30 days with gc 68805b337d
Rename xeon07 to hut 218acd6848
Simplify flake and expose host pkgs d679fd6314 
The configuration of the machines is now moved to m/
Add owl and all partition 6ec7353a27
Change owl hostnames 3fab341dc8
Set the name of the slurm cluster to jungle 39f15a1b4f
Add ssh host keys 4afe3121e6
Update rebuild script for all nodes 7bbc526671
Rename xeon08 to eudy afb2bea1c9 
From Eudyptula, a little penguin.
Add eudy host key to known hosts 8b7ffc914a
Add coments in slurm config f51e910aff
Add mpich overlay bae3c75222
Replace mpi inside bsc attribute 73b33c4d6c
Use explicit order in overlays 45d7b31c0a
Add missing parameter to extend 3a9615fce4
Set mpi to mpich by default in bscpkgs 9f03799d34
flake.lock: Update 221ccef956 
Flake lock file updates:

• Updated input 'bscpkgs':
    'git+https://pm.bsc.es/gitlab/rarias/bscpkgs.git?ref=refs%2fheads%2fmaster&rev=c775ee4d6f76aded05b08ae13924c302f18f9b2c' (2023-04-26)
  → 'git+https://pm.bsc.es/gitlab/rarias/bscpkgs.git?ref=refs%2fheads%2fmaster&rev=cbe9af5d042e9d5585fe2acef65a1347c68b2fbd' (2023-06-16)
Add osumb to the system packages 3d0f86ac07
Add rpenacob user b6aaeb8158
Move authorized keys to users.nix b610f12133
Allow srun to specify the cpu binding 2ca7e7383e 
The task/affinity plugin needs to be selected.
Add perf to packages 798e01f9e6
Set perf paranoid to 0 by default 937d8a7637
Lower perf_event_paranoid to -1 d5951483ee
Add DNS tools to resolve hosts e9845cc76a
Use our host names first by default ba75bf8249
Add the ssfhead node as gateway 83acd40880
Enable NTP using the BSC time server 9097811cc0
Add jungle.bsc.es hugo website d88caa8610
eudy: disable all cpu mitigations d5857c0f7d
eudy: Enable memory overcommit c0335c1f95
eudy: Add fcsv3 and intermediate versions for testing 1b8c3eb554
Add koro node cbe53a6f0a
koro: Add vlopez user a094093d95
Add details to request access in the web 17f9a957eb
Kill slurmd remaining processes on upgrade 8c1bf6db42
Upgrade flake: nixpkgs, bscpkgs and agenix a18d56b3ae
GRUB version no longer needed d8e366b444
Allow access to devices for node_exporter ae2007e2fe
Update nixpkgs to fix docker problem ca78313752
Don't set all_proxy cd6e6de2ad
Increase prometheus retention time to one year b80656228d
Monitor power from other nodes via LAN 8fcb5a1079
Update flake 7e789cd062
Add bay node 9c152ec9cc
Remove netboot and fixes f707650724
Add the bay host name 9cd013c4ed
Add ceph config in bay 817c98d37b
Mount the ceph filesystem in hut 94ce6fedf9
Add ceph metrics to prometheus 91270b26bb
Add rarias key for secrets 660a8ae163
Store ceph secret key in age a4fc3d131a 
This allows a node to mount the ceph FS without any extra ceph
configuration in /etc/ceph.
Add agenix to PATH in hut 3c83996e26
Add section to enable serial console 25f06db5f1
Add lake2 bootstrap config 7443a192c6
Prepare lake2 config after bootstrap f0f67f374e 
The disk ID is different under NixOS.
Specify the disk by path 6e89b3f936
Enable netboot again for PXE 33155fcb62
Add PXE helper 686f750c06
Disable pixiecore in hut for now 81baeee5b1
Remove netboot module b63b450111
Use the sda for lake2 cced6b0dc0
Add the lake2 hostname to the hosts 80efd57a11
Enable ceph osd daemons in lake2 d9dea762de
Move pkgs overlay to overlay.nix 3b823ee478
Update ceph to 18.2.0 in overlay 4cea250cf4
Switch ceph logs to journal 6a429fda1b
Add ceph tools in hut too 0fe025e8be
Add fio tool af29f639e2
Add monitoring in the bay node f86114f33e
Scrape metrics from bay 005a1be48a
Also enable monitoring in lake2 9241bda0ac
Scrape lake2 too 042ca9e882
Enable all osd on boot in lake2 e1e879178d
Enable watchdog 409efacf5b
Enable binary emulation for other architectures 76e6ae2f00
Store nixos config in /etc/nixos/config.rev a6d3f43b98
Configure bscpkgs.nixpkgs to follow nixpkgs bf53f01f0a
Keep a log over time with the config commits c63fa494d5
Enable zsh and fix key bindings 515fa49ed0
Set zsh shell for rarias 8ba4f910c3
Set zsh inc_append_history option bc51564a88
Add anavarro user 1f32b8409a
Reorganize secrets and ssh keys 8a027d8b09 
The agenix tools needs to read the secrets from a standalone file, but
we also need the same information for the SSH keys.
Move the ceph client config to an external module 8cedffe040
Clean owl2 configuration 6828273c05
Warn about the owl2 omnipath device 8a0034a867
Mount /ceph in owl1 and owl2 adec7f80fd
Remove old secrets da92154d33
Add agenix module to ceph aa92294907
Add agenix to all nodes 6850bf3a71
flake.lock: Update d57849a954 
Flake lock file updates:

• Updated input 'bscpkgs':
    'git+https://pm.bsc.es/gitlab/rarias/bscpkgs.git?ref=refs/heads/master&rev=18d64c352c10f9ce74aabddeba5a5db02b74ec27' (2023-08-31)
  → 'git+https://pm.bsc.es/gitlab/rarias/bscpkgs.git?ref=refs/heads/master&rev=ee24b910a1cb95bd222e253da43238e843816f2f' (2023-09-01)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/d680ded26da5cf104dd2735a51e88d2d8f487b4d' (2023-08-19)
  → 'github:NixOS/nixpkgs/e56990880811a451abd32515698c712788be5720' (2023-09-02)
Update email contact to jungle mail list 3a31dcd58b
Unlock ovni gitlab runners aa52236a80
flake.lock: Update c38a01c8dc 
Flake lock file updates:

• Updated input 'bscpkgs':
    'git+https://pm.bsc.es/gitlab/rarias/bscpkgs.git?ref=refs/heads/master&rev=ee24b910a1cb95bd222e253da43238e843816f2f' (2023-09-01)
  → 'git+https://pm.bsc.es/gitlab/rarias/bscpkgs.git?ref=refs/heads/master&rev=6122fef92701701e1a0622550ac0fc5c2beb5906' (2023-09-07)
Add IB and IPMI node host names 01140353c6
Poweroff idle slurm nodes after 1 hour 4c88f9a783
Block ssfhead from reaching our slurm daemon 2f258e1cdd
Allow only some ports for srun a6324e47e8
Make exporters listen in localhost only 09ac1d6c13
Remove unused large port hole in firewall 061fc60939
Add encrypted munge key with agenix 79446cebcb
Serve the nix store from hut 66b5074ff1
Enable fstrim service 627c912b87
Add update post to website 73f18d5801
Specify the space available in /ceph 668a65b9c6
Monitor storage nodes via IPMI too 44667e8e40
Update slurm to 23.02.05.1 798fa002cc
Open ports in firewall of compute nodes 1b296f2ce7
Revert "Update slurm to 23.02.05.1" 77276fb6c1 
This reverts commit aaefddc44a.
flake.lock: Update 03822c8b26 
Flake lock file updates:

• Updated input 'bscpkgs':
    'git+https://pm.bsc.es/gitlab/rarias/bscpkgs.git?ref=refs/heads/master&rev=6122fef92701701e1a0622550ac0fc5c2beb5906' (2023-09-07)
  → 'git+https://pm.bsc.es/gitlab/rarias/bscpkgs.git?ref=refs/heads/master&rev=3a4062ac04be6263c64a481420d8e768c2521b80' (2023-09-14)
Don't fetch registry flakes from the net 0ca0da9ffe
Use hut packages as the default package set 1242aad9a3 
Allows the user to directly access nixpkgs and bscpkgs from the top
level as `nix build jungle#htop` and `nix build jungle#bsc.ovni`.
Add bscpkgs and nixpkgs top level attributes 128136c137 
Allows the evaluation of packages of the intermediate overlays.
Remove bscpkgs from the registry and nixPath 642507b255 
This is done to prevent accidental evaluations where the nixpkgs input
of bscpkgs is still pointing to a different version that the one
specified in the jungle flake. Instead use jungle#bscpkgs.X to get a
package from bscpkgs.
Enable direnv integration 6d8fd353d0
Mount the hut nix store for SLURM jobs b84d1d5e26
Document the hut shared nix store for SLURM 6b2351bfe7
Add prometheus-slurm-exporter package 38e23068f2
Enable slurm-exporter service b84066fde5
Set the SLURM_CONF variable ce964b9b65
Remove user/group when using DynamicUsers 7b67b2b703
Allow anonymous access to grafana fbabc06641
Add runner for gitlab.bsc.es 32b9cc17a9
Temporarily disable pm runners 457e403258
Monitor PM webpage via blackbox e9b4d87d9f
Monitor gitlab.bsc.es too 4d362351cb
Add docker runner too ca0937859d
Don't log SLURM connection attempts from ssfhead 9957a0269d
Make blackbox exporter use the proxy a8c0ce5d06 
By default it was trying to reach the targets using the default gateway,
but since the electrical cut of 2023-10-20, the login node has not
enabled forwarding again. So better if we don't rely on it.

Reviewed-By: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable proxy for Grafana too fb8a0cb0a3 
The alerts need to contact the slack endpoint, so we add the proxy
environment variables to the grafana systemd service.

Reviewed-By: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add ICMP probes cd6983223e 
These probes check if we can reach several targets via ICMP, which is
not proxied, so they can be used to see if ICMP forwarding is working in
the login node.

In particular, we test if we can reach the Google (8.8.8.8) and
Cloudflare (1.1.1.1) DNS servers, the BSC gateway which responds to ping
only from the intranet and the login node (ssfhead).

Reviewed-By: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Monitor anella instead of gw.bsc.es 6ac5225ddb 
The target gw.bsc.es doesn't reply to our ICMP probes from hut. However,
the anella hop in the tracepath is a good candidate to identify cuts
between the login and the provider and between the provider and external
hosts like Google or Cloudflare DNS.

Reviewed-By: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Switch bscpkgs URL to sourcehut 0882ad7ecc 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
flake.lock: Update e7555bb1cf 
Flake lock file updates:

• Updated input 'bscpkgs':
    'git+https://pm.bsc.es/gitlab/rarias/bscpkgs.git?ref=refs/heads/master&rev=3a4062ac04be6263c64a481420d8e768c2521b80' (2023-09-14)
  → 'git+https://git.sr.ht/~rodarima/bscpkgs?ref=refs/heads/master&rev=f605f8e5e4a1f392589f1ea2b9ffe2074f72a538' (2023-10-31)

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
BSC packages are no longer in bsc attribute 388a10b666 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
flake.lock: Update 50cf081345 
Flake lock file updates:

• Updated input 'agenix':
    'github:ryantm/agenix/d8c973fd228949736dedf61b7f8cc1ece3236792' (2023-07-24)
  → 'github:ryantm/agenix/daf42cb35b2dc614d1551e37f96406e4c4a2d3e4' (2023-10-08)
• Updated input 'bscpkgs':
    'git+https://git.sr.ht/~rodarima/bscpkgs?ref=refs/heads/master&rev=f605f8e5e4a1f392589f1ea2b9ffe2074f72a538' (2023-10-31)
  → 'git+https://git.sr.ht/~rodarima/bscpkgs?ref=refs/heads/master&rev=e148de50d68b3eeafc3389b331cf042075971c4b' (2023-11-22)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/e56990880811a451abd32515698c712788be5720' (2023-09-02)
  → 'github:NixOS/nixpkgs/e4ad989506ec7d71f7302cc3067abd82730a4beb' (2023-11-19)

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Remove old Ceph package overlay 59d9d19891 
The Ceph package is now integrated in upstream nixpkgs.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Fix warning in slurm exporter using vendorHash 29cdfba328 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Remove complete ceph package from hut 4efd74aad6 
Only the ceph-client is needed.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable runners for pm.bsc.es/gitlab too afae708a48 
The old runners for the PM gitlab were disabled in configuration in the
last outage, but they remained working until we reboot the node. With
this change we enable the runners for both PM and gitlab.bsc.es.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Use tmpfs in /tmp b8fbb6380e 
The /tmp directory was using the SSD disk which is not erased across
boots. Nix will use /tmp to perform the builds, so we want it to be as
fast as possible. In general, all the machines have enough space to
handle large builds like LLVM.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable nixseparatedebuginfod module 024a31dd1b 
The module is only enabled on Hut and Eudy because we noticed activity
on the debuginfod service even if no debug session was active.

Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Monitor https://pm.bsc.es/gitlab/ too 777704a9ce 
The GitLab instance is in the /gitlab endpoint and may fail
independently of https://pm.bsc.es/.

Cc: Víctor López <victor.lopez@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable public-inbox at jungle.bsc.es/lists baa8347753 
The public-inbox service fetches emails from the sourcehut mailing lists
and displays them on the web. The idea is to reduce the dependency on
external services and add a secondary storage for the mailing lists in
case sourcehut goes down or changes the current free plans.

The service is available in https://jungle.bsc.es/lists/ and is open to
the public. It currently mirrors the bscpkgs and jungle mailing list.

We also edited the CSS to improve the readability and have larger fonts
by default.

The service for public-inbox produced by NixOS is not well configured to
fetch emails from an IMAP mail server, so we also manually edit the
service file to enable the network.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Move slurm client in a separate module 0899424de9 
Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Add another HTTPS probe for bsc.es 8ae2870fb3 
As all other HTTPS probes pass through the opsproxy01.bsc.es proxy, we
cannot detect a problem in our proxy or in the BSC one. Adding another
target like bsc.es that doesn't use the ops proxy allows us to discern
where the problem lies.

Instead of monitoring https://www.bsc.es/ directly, which will trigger
the whole Drupal server and take a whole second, we just fetch robots.txt
so the overhead on the server is minimal (and returns in less than 10 ms).

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Use google.com probe instead of bsc.es 6be7365916 
The main website of the BSC is failing every day around 3:00 AM for
almost one hour, so it is not a very good target. Instead, google.com is
used which should be more reliable. The same robots.txt path is fetched,
as it is smaller than the main page.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
flake.lock: Update ebfbca4b48 
Flake lock file updates:

• Updated input 'agenix':
    'github:ryantm/agenix/daf42cb35b2dc614d1551e37f96406e4c4a2d3e4' (2023-10-08)
  → 'github:ryantm/agenix/1381a759b205dff7a6818733118d02253340fd5e' (2024-04-02)
• Updated input 'agenix/darwin':
    'github:lnl7/nix-darwin/87b9d090ad39b25b2400029c64825fc2a8868943' (2023-01-09)
  → 'github:lnl7/nix-darwin/4b9b83d5a92e8c1fbfd8eb27eda375908c11ec4d' (2023-11-24)
• Updated input 'agenix/home-manager':
    'github:nix-community/home-manager/32d3e39c491e2f91152c84f8ad8b003420eab0a1' (2023-04-22)
  → 'github:nix-community/home-manager/3bfaacf46133c037bb356193bd2f1765d9dc82c1' (2023-12-20)
• Added input 'agenix/systems':
    'github:nix-systems/default/da67096a3b9bf56a91d16901293e51ba5b49a27e' (2023-04-09)
• Updated input 'bscpkgs':
    'git+https://git.sr.ht/~rodarima/bscpkgs?ref=refs/heads/master&rev=e148de50d68b3eeafc3389b331cf042075971c4b' (2023-11-22)
  → 'git+https://git.sr.ht/~rodarima/bscpkgs?ref=refs/heads/master&rev=de89197a4a7b162db7df9d41c9d07759d87c5709' (2024-04-24)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/e4ad989506ec7d71f7302cc3067abd82730a4beb' (2023-11-19)
  → 'github:NixOS/nixpkgs/6143fc5eeb9c4f00163267708e26191d1e918932' (2024-04-21)
• Updated input 'nixseparatedebuginfod':
    'github:symphorien/nixseparatedebuginfod/232591f5274501b76dbcd83076a57760237fcd64' (2023-11-05)
  → 'github:symphorien/nixseparatedebuginfod/98d79461660f595637fa710d59a654f242b4c3f7' (2024-03-07)
• Removed input 'nixseparatedebuginfod'
• Removed input 'nixseparatedebuginfod/flake-utils'
• Removed input 'nixseparatedebuginfod/flake-utils/systems'
• Removed input 'nixseparatedebuginfod/nixpkgs'

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Remove nixseparatedebuginfod input a4bf90ddfc 
It has been integrated in nixpkgs, so is no longer required.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Merge pmix outputs for MPICH ab4cab97ba 
MPICH expects headers and libraries to be present in the same directory.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Fix SLURM bug in rank integer sign expansion 08e9db0c3e 
See: https://bugs.schedmd.com/show_bug.cgi?id=19324

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add workaround for MPICH 4.2.0 cafd7ea682 
See: https://github.com/pmodels/mpich/issues/6946

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add firewall rules for Ceph and monitoring c4f539caf6 
The firewall was blocking the monitoring traffic from hut and the Ceph
traffic among OSDs. The rules only allow connecting from the specific
host that they are supposed to be coming from.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add Gitea service b15130744a 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Collect Gitea metrics in Prometheus 0dec0ee519 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add link to Gitea in the website 39378d9544 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Manually add links to the menu b43aca80ca 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Fix bogus separator 24056305c7 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Fix meta in posts entries 93ad04299d 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Allow Ceph traffic to lake2 bb86b04fce
Add msmtp to send notifications via email e0e9dc62d5 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable mail notification in Gitea 27bb7cd69e 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable Grafana email alerts b89c656ff7 
Allows sending Grafana alerts via email too, so we have a reduntant
mechanism in case Slack fails to deliver them.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add PostgreSQL DB for performance test results 7312a91271 
The database will hold the performance results of the execution of the
benchmarks. We follow the same setup on knights3 for now.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Control user access to each machine 970cdf8dbd 
The users.jungleUsers configuration option behaves like the users.users
option, but defines the list attribute `hosts` for each user, which
filters users so that only the user can only access those hosts.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Split xeon specific configuration from base c8604bf3e0 
To accomodate the raccoon knights workstation, some of the configuration
pulled by m/common/main.nix has to be removed. To solve it, the xeon
specific parts are placed into m/common/xeon.nix and only the common
configuration is at m/common/base.nix.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add raccoon motd file efea776c91 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Move vlopez user to jungleUsers for koro host 338551ec0a 
Access to other machines can be easily added into the "hosts" attribute
without the need to replicate the configuration.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Monitor raccoon machine via IPMI 80eb17c065 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add support for armv7 emulation in hut 7b4cbc57e4 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
eudy: koro: fcs: Fix fcs unprotected cpuid all 6a3728e6c6 
smp_processor_id() was called in a preepmtible context, which could
invalidate the returned value. However, this was not harmful, because
fcs threads in nosv are pinned.

Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Allow incoming traffic to hut proxy 87598e74ae 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Set the default proxy to point to hut bb10c47c2e 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Access private repositories via hut SSH proxy ae96be6915 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Grant rpenacob access to owl1 and owl2 nodes d6d7516e12 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add abonerib user to hut, raccon, owl1 and owl2 4873c881a9 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Allow ptrace to any process of the same user 90d44a95eb 
Allows users to attach GDB to their own processes, without requiring
running the program with GDB from the start. It is only available in
compute nodes, the storage nodes continue with the restricted settings.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
flake.lock: Update f095067deb 
Flake lock file updates:

• Updated input 'agenix':
    'github:ryantm/agenix/1381a759b205dff7a6818733118d02253340fd5e' (2024-04-02)
  → 'github:ryantm/agenix/de96bd907d5fbc3b14fc33ad37d1b9a3cb15edc6' (2024-07-09)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/6143fc5eeb9c4f00163267708e26191d1e918932' (2024-04-21)
  → 'github:NixOS/nixpkgs/693bc46d169f5af9c992095736e82c3488bf7dbb' (2024-07-14)

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Use authentication tokens for PM GitLab runner 8b8fc73225 
Starting with GitLab 16, there is a new mechanism to authenticate the
runners via authentication tokens, so use it instead.  Older tokens and
runners are also removed, as they are no longer used.

With the new way of managing tokens, both the tags and the locked state
are managed from the GitLab web page.

See: https://docs.gitlab.com/ee/ci/runners/new_creation_workflow.html
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Allow other jobs to run in unused cores 60c094030b 
The current select mechanism was using the memory too as a consumable
resource, which by default only sets 1 MiB per node. As each job already
requests 1 MiB, it prevents other jobs from running.

As we are not really concerned with memory usage, we only use the unused
cores in the select criteria.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Set default SLURM job time limit to one hour c737993c9c 
Prevents enless jobs from being left forever, while allow users to
request a larger time limit.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Set gitea and grafana log level to warn cdabc58c09 
Prevents filling the journal logs with information messages.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Enable debuginfod daemon in owl nodes d8ca283b80 
WARNING: This will introduce noise, as the daemon wakes up from time to
time to check for new packages.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Program shutdown for August 2nd for all machines f1373e5227 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Emulate other architectures in owl nodes too fd76a36c36 
Allows cross-compilation of packages for RISC-V that are known to try to
run RISC-V programs in the host.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Don't mount the nix store in owl nodes 240b26d82e 
Initially we planned to run jobs in those nodes by sharing the same nix
store from hut. However, these nodes are now used to build packages
which are not available in hut. Users also ssh to the nodes, which
doesn't mount the hut store, so it doesn't make much sense to keep
mounting it.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add 10 min shutdown jitter to avoid spikes 6f22f683a9 
The shutdown timer will fire at slightly different times for the
different nodes, so we slowly decrease the power consumption.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add documentation section about GRUB chain loading 291feda7ff 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove setLdLibraryPath and driSupport options fd23d84da8 
They have been removed from NixOS. The "hardware.opengl" group is now
renamed to "hardware.graphics".

See: 98cef4c273
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Set the serial console to ttyS1 in raccoon 92482721b4 
Apparently the ttyS0 console doesn't exist but ttyS1 does:

  raccoon% sudo stty -F /dev/ttyS0
  stty: /dev/ttyS0: Input/output error
  raccoon% sudo stty -F /dev/ttyS1
  speed 9600 baud; line = 0;
  -brkint -imaxbel

The dmesg line agrees:

  00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A

The console configuration is then moved from base to xeon to allow
changing it for the raccoon machine.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add dbautist user with access to hut 7c381b2b65 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Delay nix-gc until /home is mounted c4e12872d9 
Prevents starting the garbage collector before the remote FS are
mounted, in particular /home. Otherwise, all the gcroots which have
symlinks in /home will be considered stale and they will be removed.

See: #79
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Mount the NVME disk in /nvme e76e10ec19 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Use nginx to serve website and other services 8a150555a6 
Instead of using multiple tunels to forward all our services to the VM
that serves jungle.bsc.es, just use nginx to redirect the traffic from
hut. This allows adding custom rules for paths that are not posible
otherwise.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add p command to paste files b8db4ad3cd 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add paste documentation in jungle website a5b512dd67 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Create paste directories in /ceph/p 8789f6d1fe 
Ensure that all hut users have a paste directory in /ceph/p owned by
themselves. We need to wait for the ceph mount point to create them, so
we use a systemd service that waits for the remote-fs.target.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Ignore misc directory 3ebd3852f6 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Log the client IP not the proxy 0a3429ed8f 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Ignore logging requests from the gitea runner 53e7ce6b64 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Keep host header for Grafana requests 2d487e9722 
This was breaking requests due to CSRF check.

See: https://github.com/grafana/grafana/issues/45117#issuecomment-1033842787
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Use NVME as root 4d7c8378bf 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Use SSD for boot, then switch to NVME 74c0ad07ad 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove exception to fetch task endpoint 3ce5dd7c68 
It causes the request to go to the website rather than the Gitea
service.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add custom GPFS exporter for MN5 93546953aa 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Collect statistics from logged users 23b58839de 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add BSC machines to ssh config 37ed60eb09 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add script to monitor GPFS 693a96878a 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Set nixpkgs to track nixos-24.11 ae2194debe 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
flake.lock: Update 32367fbc07 
Flake lock file updates:

• Updated input 'agenix':
    'github:ryantm/agenix/de96bd907d5fbc3b14fc33ad37d1b9a3cb15edc6' (2024-07-09)
  → 'github:ryantm/agenix/f6291c5935fdc4e0bef208cfc0dcab7e3f7a1c41' (2024-08-10)
• Updated input 'bscpkgs':
    'git+https://git.sr.ht/~rodarima/bscpkgs?ref=refs/heads/master&rev=de89197a4a7b162db7df9d41c9d07759d87c5709' (2024-04-24)
  → 'git+https://git.sr.ht/~rodarima/bscpkgs?ref=refs/heads/master&rev=6782fc6c5b5a29e84a7f2c2d1064f4bcb1288c0f' (2024-11-29)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/693bc46d169f5af9c992095736e82c3488bf7dbb' (2024-07-14)
  → 'github:NixOS/nixpkgs/9c6b49aeac36e2ed73a8c472f1546f6d9cf1addc' (2025-01-14)

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Fix papermod theme in website for new hugo c0531bbe8a 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Fix MPICH build by fetching upstream patches too 7e5211d049 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Update PM GitLab tokens to new URL 7d84c9e088 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add new fox machine c0f5db745b 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add fox IPMI monitoring b18a9f99ef 
Use agenix to store the credentials safely.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Use IPMI host names instead of IP addresses 226dba428e 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Exclude fox from being suspended by slurm 6af214dfa3 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Mount NVME disks in /nvme{0,1} 8b64f53dac 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add fox page in jungle website 4adfc0297f 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add dalvare1 user e8d65e70e9 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add users to fox 094801a362 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reject SSH connections without SLURM allocation a7aa3b79a1 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add quickstart guide 9839206f4e 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Don't move doc in web output 5abdc0da89 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add abonerib user to fox a886d6c943 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Adjust fox slurm config after disabling SMT 2938acc3e4 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add varcila user to hut and fox 97c1fb240d 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove SLURM partition all fc75b32c5f 
We no longer have homogeneous nodes so it doesn't make much sense to
allocate a mix of them.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add new GitLab runner for gitlab.bsc.es 61df5d4ddb 
It uses docker based on alpine and the host nix store, so we can perform
builds but isolate them from the system.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Fix nginx /cache regex 0ecf221730 
`nix-serve` does not handle duplicates in the path:
```
hut$ curl http://127.0.0.1:5000/nix-cache-info
StoreDir: /nix/store
WantMassQuery: 1
Priority: 30
hut$ curl http://127.0.0.1:5000//nix-cache-info
File not found.
```

This meant that the cache was not accessible via:
`curl https://jungle.bsc.es/cache/nix-cache-info` but
`curl https://jungle.bsc.es/cachenix-cache-info` worked.

Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Make nginx listen on all interfaces 85d4e8ad5c 
Needed for local hosts to contact the nix cache via HTTP directly.
We also allow the incoming traffic on port 80.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Clean all iptables rules on stop 5b14172646 
Prevents the "iptables: Chain already exists." error by making sure that
we don't leave any chain on start. The ideal solution is to use
iptables-restore instead, which will do the right job. But this needs to
be changed in NixOS entirely.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Use hut nix cache in owl1, owl2 and raccoon 03cdf10cbc 
For owl1 and owl2 directly connect to hut via LAN with HTTP, but for
raccoon pass via the proxy using jungle.bsc.es with HTTPS. There is no
risk of tampering as packages are signed.

Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Add nix cache documentation section dcfd74f387 
Include usage from NixOS and non-NixOS hosts and a test with curl to
ensure it can be reached.

Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Add bscpm04.bsc.es SSH host and public key b1795ba5be 
Allows fetching repositories from hut and other machines in jungle
without the need to do any extra configuration.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Allow traffic from docker to enter port 23080 166508fc4f 
Before:

  hut% sudo docker run -it --rm alpine /bin/ash -xc 'true | nc -w 3 -v 10.0.40.7 23080'
  + true
  + nc -w 3 -v 10.0.40.7 23080
  nc: 10.0.40.7 (10.0.40.7:23080): Operation timed out

After:

  hut% sudo docker run -it --rm alpine /bin/ash -xc 'true | nc -w 3 -v 10.0.40.7 23080'
  + true
  + nc -w 3 -v 10.0.40.7 23080
  10.0.40.7 (10.0.40.7:23080) open

Fixes: #94
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Don't forward any docker traffic 41e83fc5ee 
Access to the 23080 local port will be done by applying the INPUT rules,
which pass through nixos-fw.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Increase data retention to 5 years f2cfc4a415 
Now that we have more space, we can extend the retention time to 5 years
to hold the monitoring metrics. For a year we have:

	# du -sh /var/lib/prometheus2
	13G     /var/lib/prometheus2

So we can expect it to increase to about 65 GiB. In the future we may
want to reduce some adquisition frequency.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add raccoon node exporter monitoring b7e1e4faa8 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Set keep-outputs to true in all machines edca5972ee 
From the documentation of keep-outputs, setting it to true would prevent
the GC from removing build time dependencies:

If true, the garbage collector will keep the outputs of non-garbage
derivations. If false (default), outputs will be deleted unless they are
GC roots themselves (or reachable from other roots).

In general, outputs must be registered as roots separately. However,
even if the output of a derivation is registered as a root, the
collector will still delete store paths that are used only at build time
(e.g., the C compiler, or source tarballs downloaded from the network).
To prevent it from doing so, set this option to true.

See: https://nix.dev/manual/nix/2.24/command-ref/conf-file.html#conf-keep-outputs
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add custom nix-daemon exporter 102d8706dd 
Allows us to see which derivations are being built in realtime. It is a
bit of a hack, but it seems to work. We simply look at the environment
of the child processes of nix-daemon (usually bash) and then look for
the $name variable which should hold the current derivation being
built. Needs root to be able to read the environ file of the different
nix-daemon processes as they are owned by the nixbld* users.

See: https://discourse.nixos.org/t/query-ongoing-builds/23486
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add meteocat exporter 371b8b6a23 
Allows us to track ambient temperature changes and estimate the
temperature delta between the server room and exterior temperature.
We should be able to predict when we would need to stop the machines due
to excesive temperature as summer approaches.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add UPC temperature sensor monitoring 5dbc100738 
These sensors are part of their air quality measurements, which just
happen to be very close to our server room.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove pam_slurm_adopt from fox 3e0674f872 
We no longer will be able to use SLURM from jungle.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove fox from SLURM 40ef1d4886 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove Ceph module from fox f7cad2381c 
It will no longer be accesible from the UPC.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Distrust fox SSH key 7f16280fd5 
We no longer will share secrets with fox until we can regain our trust.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Rotate fox SSH host key 2af538e99b 
Prevent decrypting old secrets by reading the git history.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Rekey all secrets 8887b9e1f8 
Fox is no longer able to use munge or ceph, so we remove the key and
rekey them.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Disable home via NFS in fox dbedeb3613 
It won't be accesible anymore as we won't be in the same LAN.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Update configuration for UPC network 1b090785a0 
The fox machine will be placed in the UPC network, so we update the
configuration with the new IP and gateway. We won't be able to reach hut
directly so we also remove the host entry and proxy.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Monitor fox, gateway and UPC anella via ICMP dad5225486 
Fox should reply once the machine is connected to the UPC network.
Monitoring also the gateway and UPC anella allows us to estimate if the
whole network is down or just fox.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove fox monitoring via IPMI 6f4fe9bb22 
We will need to setup an VPN to be able to access fox in its new
location, so for now we simply remove the IPMI monitoring.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add machine map file c2bfe806fe 
Documents the location, board and serial numbers so we can track the
machines if they move around. Some information is unkown.

Using the Nix language to encode the machines location and properties
allows us to later use that information in the configuration of the
machines themselves.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Only proxy SSH git remotes via hut in xeon a3e48bc83c 
Other machines like raccoon have direct access.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable linger for user rarias d53c7a3acb 
Allows services to run without a login session.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Allow X11 forwarding via SSH 0271ba399f 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add node exporter monitoring in raccoon fe344ea31a 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add dbautist user to raccoon machine 384ef9e9df 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Disable nix garbage collector in raccoon 54a30e063c 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable binfmt emulation in raccoon ac46401243 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Make raccoon use performance governor 3e080465a4 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable nixdebuginfod in raccoon 0b330e5274 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Extend perf support in raccoon a8c68a630f 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Create tracing group and add arocanon in raccoon 2760355358 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Use UPC time servers as others are blocked d6b7421f3f 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Use DHCP for Ethernet in fox d76f38b502 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Use extra- for substituters and trusted-public-keys ce4c653eb2 
From the nix manual:

> A configuration setting usually overrides any previous value. However,
> for settings that take a list of items, you can prefix the name of the
> setting by extra- to append to the previous value.

Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Add hut as nix cache in fox 9c3274d068 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Use performance governor in fox 3797a8ecaf 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Only allow Vincent to access fox for now 7905969874 
Needed to run benchmarks without interference.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Create specific SSF rack configuration 7a312bd01c 
Allow xeon machines to optionally inherit SSF configuration such as the
NFS mount point and the network configuration.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add tent machine from xeon04 144d87008d 
We moved the tent machine to the server room in the BSC building and is
now directly connected to the raccoon via NAT.

Fixes: #106
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Don't use proxy in base preset 5dce13c512 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add hut SSH configuration from outside SSF LAN 0022dfab63 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add access to tent to all hut users too df0ce98526 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add software RAID in tent using 3 disks ddcf158758 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Create directories in /vault/home for tent users 2006a5fb05 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add tent host key and admin keys 3cb9563738 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Rekey secrets with tent keys 59961d1351 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Disable nix garbage collector in tent ef2f31510c 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Add monitoring in tent 5d25805c6a 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add nginx server in tent de99ff3414 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Serve Grafana from subpath 017b57670b 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable nix cache 8b2c9dcacd 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Remove jungle nix cache from tent c51ca035b7 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add tent key for nix-serve 71bbdd5922 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable jungle robot emails for Grafana in tent 01891b9bef 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add GitLab shell runner in tent for PM 96d7f186d2 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add docker GitLab runner for BSC GitLab 085b92ce0f 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add public html files to tent a4328fe380 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Use IPv4 for blackbox exporter 03209b6bfc 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Monitor AXLE machine too 965dca422a 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add bsc.es to resolve domain names 2e94b6795b 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable gitea in tent e1359d134e 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable public-inbox service in tent e94ef5a08d 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add p service for pastes eeb8557b96 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Move nix-daemon exporter to modules d5227db996 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Monitor nix-daemon in tent 1210d96ae9 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add GitLab runner with debian docker for PM 28e094d4c1 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable msmtp configuration in tent 6e316e57be 
Allows gitea to send notifications via email.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Disable registration in Gitea dfee33038b 
Get rid of all the spam accounts they are trying to register.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable SSH X11 forwarding df70515bc8 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Load amd_uncore module in fox e6f526bbc6 
Needed for L3 events in perf.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Disable NUMA balancing in fox 98b882507b 
See: https://www.kernel.org/doc/html/latest/admin-guide/sysctl/kernel.html#numa-balancing

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Disable kptr_restrict in fox d07855e5c5 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Update access instructions bc64d62a7f 
We no longer need to request a petition through BSC, as we will be in
charge of the login. Remove link to the old repository as well and
prefer only email.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add ac.upc.edu as name search server d83ad5decb 
Allows referring to fox.ac.upc.edu directly as fox.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add OpenVPN service to connect to fox BMC 8eb14acdf9 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Monitor fox via VPN 5928c68720 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Restrict DAC VPN to fox-ipmi machine only 43c20c544d 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Monitor Fox BMC with ICMP probes too ad834cebd6 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add all terminfo files in environment 5318fb1a0d 
Fixes problems with the kitty terminal when opening vim or kakoune.

Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Revert "Only allow Vincent to access fox for now" c47c880ff2 
This reverts commit efac36b186.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add access to fox for rpenacob user daba3eca18 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add pmartin1 user with access to fox 57b9450a59 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add new configuration for apex 52bd019cdd 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add storcli utility to apex 3d02a231a5 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove proxy configuration from environment 6826a372d5 
All machines have now direct connection with the outside world.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove proxy from hut HTTP probes fd99204913 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add users to apex machine f70074b216 
They need to be able to login to apex to access any other machine from
the SSF rack.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove SSH proxy to access BSC clusters 327c0d61a2 
We now have direct connection to them.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Disable root_squash from NFS 7be793da3d 
Allows root to read files in the NFS export, so we can directly run
`nixos-rebuild switch` from /home.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Make NFS mount async to improve latency d5a9086afa 
Don't wait to flush writes, as we don't care about consistency on a
crash:

> This option allows the NFS server to violate the NFS protocol and
> reply to requests before any changes made by that request have been
> committed to stable storage (e.g. disc drive).
>
> Using this option usually improves performance, but at the cost that
> an unclean server restart (i.e. a crash) can cause data to be lost or
> corrupted.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Use IPv4 in blackbox probes cd37f86b09 
Otherwise they simply fail as IPv6 doesn't work.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove unused blackbox configuration modules 1899ad89db 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add proxy configuration for internal hosts 88d2de454b 
Access internal hosts via apex proxy. From the compute nodes we first
open an SSH connection to apex, and then tunnel it through the HTTP
proxy with netcat.

This way we allow reaching internal GitLab repositories without
requiring the user to have credentials in the remote host, while we can
use multiple remotes to provide redundancy.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Prevent accidental use of nftables c7a52e2999 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove extra flush commands on firewall stop a36b403bd4 
They are not needed as they are already flushed when the firewall
starts or stops.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add weasel machine configuration 24128e22f4 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Rotate anavarro password and SSH key d487241db2 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Silently ban OpenVAS BSC scanner from apex 99741382ab 
It is spamming our logs with refused connection lines:

apex% sudo journalctl -b0 | grep 'refused connection.*SRC=192.168.8.16' | wc -l
13945

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Upgrade nixpkgs to nixos 25.05 acd8fd6c51 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
flake.lock: Update 957da4b1fd 
Flake lock file updates:

• Updated input 'agenix':
    'github:ryantm/agenix/f6291c5935fdc4e0bef208cfc0dcab7e3f7a1c41?narHash=sha256-b%2Buqzj%2BWa6xgMS9aNbX4I%2BsXeb5biPDi39VgvSFqFvU%3D' (2024-08-10)
  → 'github:ryantm/agenix/531beac616433bac6f9e2a19feb8e99a22a66baf?narHash=sha256-9P1FziAwl5%2B3edkfFcr5HeGtQUtrSdk/MksX39GieoA%3D' (2025-06-17)
• Updated input 'agenix/darwin':
    'github:lnl7/nix-darwin/4b9b83d5a92e8c1fbfd8eb27eda375908c11ec4d?narHash=sha256-gzGLZSiOhf155FW7262kdHo2YDeugp3VuIFb4/GGng0%3D' (2023-11-24)
  → 'github:lnl7/nix-darwin/43975d782b418ebf4969e9ccba82466728c2851b?narHash=sha256-dyN%2BteG9G82G%2Bm%2BPX/aSAagkC%2BvUv0SgUw3XkPhQodQ%3D' (2025-04-12)
• Updated input 'agenix/home-manager':
    'github:nix-community/home-manager/3bfaacf46133c037bb356193bd2f1765d9dc82c1?narHash=sha256-7ulcXOk63TIT2lVDSExj7XzFx09LpdSAPtvgtM7yQPE%3D' (2023-12-20)
  → 'github:nix-community/home-manager/abfad3d2958c9e6300a883bd443512c55dfeb1be?narHash=sha256-YZCh2o9Ua1n9uCvrvi5pRxtuVNml8X2a03qIFfRKpFs%3D' (2025-04-24)
• Updated input 'bscpkgs':
    'git+https://git.sr.ht/~rodarima/bscpkgs?ref=refs/heads/master&rev=6782fc6c5b5a29e84a7f2c2d1064f4bcb1288c0f' (2024-11-29)
  → 'git+https://git.sr.ht/~rodarima/bscpkgs?ref=refs/heads/master&rev=9d1944c658929b6f98b3f3803fead4d1b91c4405' (2025-06-11)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/9c6b49aeac36e2ed73a8c472f1546f6d9cf1addc?narHash=sha256-i/UJ5I7HoqmFMwZEH6vAvBxOrjjOJNU739lnZnhUln8%3D' (2025-01-14)
  → 'github:NixOS/nixpkgs/dfcd5b901dbab46c9c6e80b265648481aafb01f8?narHash=sha256-Kt1UIPi7kZqkSc5HVj6UY5YLHHEzPBkgpNUByuyxtlw%3D' (2025-07-13)

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Remove package ix as it is gone 0a39169d17 
Fails with: "error: ix has been removed from Nixpkgs, as the ix.io
pastebin has been offline since Dec. 2023".

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Remove merged MPICH patch a81aebc788 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Update weasel IPMI hostname for monitoring 9c11076a43 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Ban another scanner trying to connect via SSH 9e5ce89600 
It is constantly spamming out logs:

apex# journalctl | grep 'Connection closed by 84.88.52.176' | wc -l
2255

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove option allowUnfree from fox and raccoon ef166563d8 
It is already set to true for all machines.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Enable open source NVidia driver in fox 111c13d200 
It is recommended for newer versions.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Select proprietary NVIDIA driver in raccoon 1d2dca5869 
The NVIDIA GTX 960 from 2016 has the Maxwell architecture, and NixOS
suggests using the proprietary driver for older than Turing:

> It is suggested to use the open source kernel modules on Turing or
> later GPUs (RTX series, GTX 16xx), and the closed source modules
> otherwise.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Enable automatic Nix GC in raccoon deb3370cdc 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Replace xeon07 by hut in ssh config ef2f2115de 
The xeon07 machine has been renamed to hut.

Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Move shared nvidia settings to a separate module a05ef0c3eb 
Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Enable cuda systemFeature in raccoon and fox e2d8a17fad 
This allows running derivations which depend on cuda runtime without
breaking the sandbox. We only need to add `requiredSystemFeatures = [ "cuda" ];`
to the derivation.

Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Add missing symlink in cuda sandbox 4db07cb9c3 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add cudainfo program to test CUDA eaeff6a564 
The cudainfo program checks that we can initialize the CUDA RT library
and communicate with the driver. It can be used as standalone program or
built with cudainfo.gpuCheck so it is executed inside the build sandbox
to see if it also works fine. It uses the autoAddDriverRunpath hook to
inject in the runpath the location of the library directory for CUDA
libraries.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Disable automatic August shutdown for Fox 49ffca7492 
The UPC has different dates for the yearly power cut, and Fox can
recover properly from a power loss, so we don't need to have it turned
off before the power cut. Simply disabling the timer is enough.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Move August shutdown to 3rd at 22h 9fa66aaa41 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add NixOS module to control power policy a433a1686b 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Set power policy to always turn on 0805684887 
In all machines, as soon as we recover the power, turn the machine back
on. We cannot rely on the previous state as we will shut them down
before the power is cut to prevent damage on the power supply
monitoring circuit.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Move StartLimit* options to unit section 5eb2e2516a 
The StartLimitBurst and StartLimitIntervalSec options belong to the
[Unit] section, otherwise they are ignored in [Service]:

> Unknown key 'StartLimitIntervalSec' in section [Service], ignoring.

When using [Unit], the limits are properly set:

  apex% systemctl show power-policy.service | grep StartLimit
  StartLimitIntervalUSec=10min
  StartLimitBurst=10
  StartLimitAction=none

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Access gitlab via raccoon in fox 3e378ae523 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add csiringo user with access to apex and weasel f19f343330 
Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Enable nix-ld in weasel 96a85c107b 
Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Fix typo in csiringo ssh key b1a2effdd6 
Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Move slurm control server to apex 86ec606121 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Split slurm configuration for client and server 9daa2a2649 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Only configure apex as slurm server 6a99b096e8 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove hut from slurm 8d0b759524 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add firewall rules to slurm server b5d22b2d27 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Use writeShellScript for suspend.sh and resume.sh c8444b696a 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add wireguard server in fox 725a826ea4 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable wireguard in apex 710982378b 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add local host fox in apex 8de6b0c1c3 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Make apex host specific to each machine d41aeba923 
Allows direct contact via the VPN when accessing from fox, but use
Internet when using the rest of the machines.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Trust fox for compute node secrets 7e60c2c0dd 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Rekey secrets with trusted fox key cbfaa4cb6f 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add fox machine to SLURM cf0692795c 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Accept fox connection to slurm controller 6bde1db500 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Accept connections from apex to fox slurmd 2a1782c8ad 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Enable fail2ban in fox 35cac2964c 
Protect fox against ssh bruteforce attacks:

fox% sudo lastb | head
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:24 - 11:24  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:24 - 11:24  (00:00)

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Revert "Remove pam_slurm_adopt from fox" 85996dd334 
This reverts commit 64a52801ed.

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Update fox documentation for SLURM and FS c1dea8fdf0 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Use 10.106.0.0/24 subnet to avoid collisions 0de9a4c9c4 
The 106 byte is the code for 'j' (jungle) in ASCII:

	% printf j | od -t d
	0000000         106
	0000001

Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Allow access to NFS via wireguard subnet 76c36b1c41 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Mount home via NFS from apex in fox f793d3efdc 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add /nfs/home to fox documentation d43d432a98 
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
Add AMD uProf package and driver db57b52db4 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add AMD uProf module and enable it in fox 814e3b4f0b 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Use CFLAGS_MODULE instead of EXTRA_CFLAGS 7f8a8c8ea3 
Fixes the build in Linux 6.15.6, as it was not able to find the include
files.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Fix hrtimer new interface ceb93bdd6b 
The hrtimer_init() is now done via hrtimer_setup() with the callback
function as argument.

See: https://lwn.net/Articles/996598/
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Fix amd-uprof dependencies with patchelf 9ea0d5f7ac 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Disable NMI watchdog in fox 723860aea0 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Fix hidden dependencies for AMDuProfSys 9fcb7c4caa 
It tries to dlopen libcrypt.so.1 and libstdc++.so.6, so we make sure
they are available by adding them to the runpath.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add AMD uProf section to fox documentation 6d4ec8dedc 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add amd_hsmp module in fox for AMD uProf 113097f2fd 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Fix AMDuProfPcm so it finds libnuma.so 59fd50fbee 
We change the search procedure so it detects NixOS from /etc/os-release
and uses "libnuma.so" when calling dlopen, instead of harcoding a full
path to /usr. The full patch of libnuma is stored in the runpath, so
dlopen can find it.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Tested-by: Vincent Arcila <vincent.arcila@bsc.es>
Share a public folder for documents 36c5befc5b 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Use lowercase peer hostnames 407e93eff3 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Restrict fox peer to a single IP 71b24e1b28 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add raccoon host key 5afcdbba5e 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add raccoon peer to wireguard 4021e98def 
It routes traffic from fox, apex and the compute nodes so that we can
reach the git servers and tent.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove extra SSH jump configuration cb8f27071d 
We now have direct visibility among nodes so we don't need any extra
SSH configuration to reach them.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
Mount apex /home via NFS in raccoon dbeb973863 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add web post update for 2025 8cbb9dd58a 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Remove machine access for user csiringo 31a3ac4a4d 
Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Use hut substituter in all nodes 15098cb89f 
Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Lower connect timeout when using hut substituter 7c34907f52 
Reviewed-by: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Restart slurmd on failure 7350983bb3 
A failure to reach the control node can cause slurmd to fail and the
unit remains in the failed state until is manually restarted. Instead,
try to restart the service every 30 seconds, forever:

    owl1% systemctl show slurmd | grep -E 'Restart=|RestartUSec='
    Restart=on-failure
    RestartUSec=30s
    owl1% pgrep slurmd
    5903
    owl1% sudo kill -SEGV 5903
    owl1% pgrep slurmd
    6137

Fixes: #177
Reviewed-by: Aleix Boné <abonerib@bsc.es>
Add acinca user 8e0345b866 
Reviewed-by: Aleix Boné <abonerib@bsc.es>
abonerib approved these changes 2025-10-01 16:51:04 +02:00
abonerib left a comment
Collaborator
Copy Link

LGTM XD

LGTM XD
rarias commented 2025-10-01 17:23:20 +02:00
Author
Owner
Copy Link

I will move the website to another repository as well, and fix the commit dates. Will fix in another PR.

I will move the website to another repository as well, and fix the commit dates. Will fix in another PR.
👍 1
rarias closed this pull request 2025-10-01 17:23:20 +02:00

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No Reviewers
abonerib
Labels
Clear labels
bug
ceph
ci
config
cross
hw
io
kernel
mpi
net
nix
ops
repair
slurm
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
abonerib arocanon varcila rarias
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: rarias/jungle#185
Delete Branch "remove-blobs"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?