40 Commits

Author SHA1 Message Date
8161a21fbc users: Create tracing group and add arocanon 2025-05-15 12:32:54 +02:00
a2b9b155f6 raccoon: Add lttng and extend perf support 2025-05-15 12:21:26 +02:00
00e686a1d8 raccoon: Enable nixdebuginfod 2025-05-06 14:44:12 +02:00
f00575640c Set keep-outputs to true in all machines
From the documentation of keep-outputs, setting it to true would prevent
the GC from removing build time dependencies:

If true, the garbage collector will keep the outputs of non-garbage
derivations. If false (default), outputs will be deleted unless they are
GC roots themselves (or reachable from other roots).

In general, outputs must be registered as roots separately. However,
even if the output of a derivation is registered as a root, the
collector will still delete store paths that are used only at build time
(e.g., the C compiler, or source tarballs downloaded from the network).
To prevent it from doing so, set this option to true.

See: https://nix.dev/manual/nix/2.24/command-ref/conf-file.html#conf-keep-outputs
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
2025-05-05 11:46:46 +02:00
5899d8847c Use nix cache from hut 2025-05-05 11:15:18 +02:00
2be8c0dad3 Make raccoon use performance governor 2025-05-05 11:15:14 +02:00
cf4ce9fa18 Enable binfmt emulation in raccoon 2025-03-21 17:51:41 +01:00
b513cd8982 Disable nix garbage collector 2025-03-18 16:48:47 +01:00
b87d79e1cc Add dbautist user to raccoon machine 2025-03-03 13:55:23 +01:00
4807337098 Add node exporter monitoring in raccoon 2025-02-25 17:11:09 +01:00
82ed40d386 Add FPGA u280 firmware (working ok) 2025-02-25 17:08:13 +01:00
e81b7cc158 Add U280 support back in the xocl driver
They removed U280 from the list of supported devices, but that doesn't
mean it is will stop working.
2025-02-21 10:30:42 +01:00
0533a48a98 Add udev rules 2025-02-20 17:32:31 +01:00
a9c1cb26ea Load Xilinx modules at boot 2025-02-20 16:27:14 +01:00
4ab38212b3 Add Xilinx xocl derivation
Need to move to raccoon to access "config" attribute.
2025-02-20 16:04:52 +01:00
df8b5b2d67 First successful build of Xilinx XRT
What an absolute nightmare, and we are far from the end.
2025-02-20 12:07:21 +01:00
c5400955c7 Allow X11 forwarding via SSH 2025-02-18 16:19:04 +01:00
165a60b0d2 Enable linger for user rarias
Allows services to run without a login session.
2024-10-14 19:12:25 +02:00
bd4dabf03a Only proxy SSH git remotes via hut in xeon
Other machines like raccoon have direct access.
2024-10-14 18:31:04 +02:00
9a9161fc55 Delay nix-gc until /home is mounted
Prevents starting the garbage collector before the remote FS are
mounted, in particular /home. Otherwise, all the gcroots which have
symlinks in /home will be considered stale and they will be removed.

See: #79
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
2024-09-20 09:45:30 +02:00
1a0cf96fc4 Add dbautist user with access to hut
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
2024-09-20 09:42:02 +02:00
4bd1648074 Set the serial console to ttyS1 in raccoon
Apparently the ttyS0 console doesn't exist but ttyS1 does:

  raccoon% sudo stty -F /dev/ttyS0
  stty: /dev/ttyS0: Input/output error
  raccoon% sudo stty -F /dev/ttyS1
  speed 9600 baud; line = 0;
  -brkint -imaxbel

The dmesg line agrees:

  00:03: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A

The console configuration is then moved from base to xeon to allow
changing it for the raccoon machine.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:56 +02:00
15b114ffd6 Remove setLdLibraryPath and driSupport options
They have been removed from NixOS. The "hardware.opengl" group is now
renamed to "hardware.graphics".

See: 98cef4c273
Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:53 +02:00
dd6d8c9735 Add documentation section about GRUB chain loading
Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:47 +02:00
e15a3867d4 Add 10 min shutdown jitter to avoid spikes
The shutdown timer will fire at slightly different times for the
different nodes, so we slowly decrease the power consumption.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:44 +02:00
5cad208de6 Don't mount the nix store in owl nodes
Initially we planned to run jobs in those nodes by sharing the same nix
store from hut. However, these nodes are now used to build packages
which are not available in hut. Users also ssh to the nodes, which
doesn't mount the hut store, so it doesn't make much sense to keep
mounting it.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:42 +02:00
c8687f7e45 Emulate other architectures in owl nodes too
Allows cross-compilation of packages for RISC-V that are known to try to
run RISC-V programs in the host.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:39 +02:00
d988ef2eff Program shutdown for August 2nd for all machines
Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:36 +02:00
b07929eab3 Enable debuginfod daemon in owl nodes
WARNING: This will introduce noise, as the daemon wakes up from time to
time to check for new packages.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:30 +02:00
b3e397eb4c Set gitea and grafana log level to warn
Prevents filling the journal logs with information messages.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:27 +02:00
5ad2c683ed Set default SLURM job time limit to one hour
Prevents enless jobs from being left forever, while allow users to
request a larger time limit.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:24 +02:00
1f06f0fa0c Allow other jobs to run in unused cores
The current select mechanism was using the memory too as a consumable
resource, which by default only sets 1 MiB per node. As each job already
requests 1 MiB, it prevents other jobs from running.

As we are not really concerned with memory usage, we only use the unused
cores in the select criteria.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:22 +02:00
8ca1d84844 Use authentication tokens for PM GitLab runner
Starting with GitLab 16, there is a new mechanism to authenticate the
runners via authentication tokens, so use it instead.  Older tokens and
runners are also removed, as they are no longer used.

With the new way of managing tokens, both the tags and the locked state
are managed from the GitLab web page.

See: https://docs.gitlab.com/ee/ci/runners/new_creation_workflow.html
Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:16 +02:00
998f599be3 flake.lock: Update
Flake lock file updates:

• Updated input 'agenix':
    'github:ryantm/agenix/1381a759b205dff7a6818733118d02253340fd5e' (2024-04-02)
  → 'github:ryantm/agenix/de96bd907d5fbc3b14fc33ad37d1b9a3cb15edc6' (2024-07-09)
• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/6143fc5eeb9c4f00163267708e26191d1e918932' (2024-04-21)
  → 'github:NixOS/nixpkgs/693bc46d169f5af9c992095736e82c3488bf7dbb' (2024-07-14)

Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:13 +02:00
fcfc6ac149 Allow ptrace to any process of the same user
Allows users to attach GDB to their own processes, without requiring
running the program with GDB from the start. It is only available in
compute nodes, the storage nodes continue with the restricted settings.

Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:09 +02:00
6e87130166 Add abonerib user to hut, raccon, owl1 and owl2
Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:07 +02:00
06f9e6ac6b Grant rpenacob access to owl1 and owl2 nodes
Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:05 +02:00
da07aedce2 Access private repositories via hut SSH proxy
Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:36:03 +02:00
61427a8bf9 Set the default proxy to point to hut
Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:35:56 +02:00
958ad1f025 Allow incoming traffic to hut proxy
Reviewed-by: Aleix Boné <abonerib@bsc.es>
2024-09-12 08:35:23 +02:00
37 changed files with 992 additions and 80 deletions

View File

@@ -151,12 +151,26 @@ And update grub.
# nix build .#nixosConfigurations.xeon02.config.system.build.kexecTree -v
```
## Chain NixOS in same disk
## Chain NixOS in same disk with other systems
To install NixOS on a partition along another system which controls the GRUB,
first disable the grub device, so the GRUB is not installed in the disk by
NixOS (only the /boot files will be generated):
```
boot.loader.grub.device = "nodev";
```
Then add the following entry to the old GRUB configuration:
```
menuentry 'NixOS' {
insmod chain
set root=(hd3,1)
search --no-floppy --label nixos --set root
configfile /boot/grub/grub.cfg
}
```
The partition with NixOS must have the label "nixos" for it to be found. New
system configuration entries will be stored in the GRUB configuration managed
by NixOS, so there is no need to change the old GRUB settings.

12
flake.lock generated
View File

@@ -10,11 +10,11 @@
"systems": "systems"
},
"locked": {
"lastModified": 1712079060,
"narHash": "sha256-/JdiT9t+zzjChc5qQiF+jhrVhRt8figYH29rZO7pFe4=",
"lastModified": 1720546205,
"narHash": "sha256-boCXsjYVxDviyzoEyAk624600f3ZBo/DKtUdvMTpbGY=",
"owner": "ryantm",
"repo": "agenix",
"rev": "1381a759b205dff7a6818733118d02253340fd5e",
"rev": "de96bd907d5fbc3b14fc33ad37d1b9a3cb15edc6",
"type": "github"
},
"original": {
@@ -88,11 +88,11 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1713714899,
"narHash": "sha256-+z/XjO3QJs5rLE5UOf015gdVauVRQd2vZtsFkaXBq2Y=",
"lastModified": 1720957393,
"narHash": "sha256-oedh2RwpjEa+TNxhg5Je9Ch6d3W1NKi7DbRO1ziHemA=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "6143fc5eeb9c4f00163267708e26191d1e918932",
"rev": "693bc46d169f5af9c992095736e82c3488bf7dbb",
"type": "github"
},
"original": {

View File

@@ -9,6 +9,10 @@
# Select the this using the ID to avoid mismatches
boot.loader.grub.device = "/dev/disk/by-id/wwn-0x55cd2e414d53562d";
boot.kernel.sysctl = {
"kernel.yama.ptrace_scope" = lib.mkForce "1";
};
environment.systemPackages = with pkgs; [
ceph
];

View File

@@ -3,6 +3,7 @@
# Includes the basic configuration for an Intel server.
imports = [
./base/agenix.nix
./base/august-shutdown.nix
./base/boot.nix
./base/env.nix
./base/fs.nix

View File

@@ -0,0 +1,14 @@
{
# Shutdown all machines on August 2nd at 11:00 AM, so we can protect the
# hardware from spurious electrical peaks on the yearly electrical cut for
# manteinance that starts on August 4th.
systemd.timers.august-shutdown = {
description = "Shutdown on August 2nd for maintenance";
wantedBy = [ "timers.target" ];
timerConfig = {
OnCalendar = "*-08-02 11:00:00";
RandomizedDelaySec = "10min";
Unit = "systemd-poweroff.service";
};
};
}

View File

@@ -11,14 +11,12 @@
terminal_output --append serial
'';
# Enable serial console
boot.kernelParams = [
"console=tty1"
"console=ttyS0,115200"
];
boot.kernel.sysctl = {
"kernel.perf_event_paranoid" = lib.mkDefault "-1";
# Allow ptracing (i.e. attach with GDB) any process of the same user, see:
# https://www.kernel.org/doc/Documentation/security/Yama.txt
"kernel.yama.ptrace_scope" = "0";
};
boot.kernelPackages = pkgs.linuxPackages_latest;

View File

@@ -23,6 +23,7 @@
trusted-users = [ "@wheel" ];
flake-registry = pkgs.writeText "global-registry.json"
''{"flakes":[],"version":2}'';
keep-outputs = true;
};
gc = {
@@ -32,6 +33,21 @@
};
};
# The nix-gc.service can begin its execution *before* /home is mounted,
# causing it to remove all gcroots considering them as stale, as it cannot
# access the symlink. To prevent this problem, we force the service to wait
# until /home is mounted as well as other remote FS like /ceph.
systemd.services.nix-gc = {
# Start remote-fs.target if not already being started and fail if it fails
# to start. It will also be stopped if the remote-fs.target fails after
# starting successfully.
bindsTo = [ "remote-fs.target" ];
# Wait until remote-fs.target fully starts before starting this one.
after = [ "remote-fs.target"];
# Ensure we can access a remote path inside /home
unitConfig.ConditionPathExists = "/home/Computational";
};
# This value determines the NixOS release from which the default
# settings for stateful data, like file locations and database versions
# on your system were taken. Its perfectly fine and recommended to leave

View File

@@ -8,13 +8,6 @@ in
# Enable the OpenSSH daemon.
services.openssh.enable = true;
# Connect to intranet git hosts via proxy
programs.ssh.extraConfig = ''
Host bscpm02.bsc.es bscpm03.bsc.es gitlab-internal.bsc.es alya.gitlab.bsc.es
User git
ProxyCommand nc -X connect -x localhost:23080 %h %p
'';
programs.ssh.knownHosts = hostsKeys // {
"gitlab-internal.bsc.es".publicKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF9arsAOSRB06hdy71oTvJHG2Mg8zfebADxpvc37lZo3";
"bscpm03.bsc.es".publicKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM2NuSUPsEhqz1j5b4Gqd+MWFnRqyqY57+xMvBUqHYUS";

View File

@@ -20,6 +20,7 @@
rarias = {
uid = 1880;
isNormalUser = true;
linger = true;
home = "/home/Computational/rarias";
description = "Rodrigo Arias";
group = "Computational";
@@ -39,7 +40,7 @@
home = "/home/Computational/arocanon";
description = "Aleix Roca";
group = "Computational";
extraGroups = [ "wheel" ];
extraGroups = [ "wheel" "tracing" ];
hashedPassword = "$6$hliZiW4tULC/tH7p$pqZarwJkNZ7vS0G5llWQKx08UFG9DxDYgad7jplMD8WkZh5k58i4dfPoWtnEShfjTO6JHiIin05ny5lmSXzGM/";
openssh.authorizedKeys.keys = [
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF3zeB5KSimMBAjvzsp1GCkepVaquVZGPYwRIzyzaCba aleix@bsc"
@@ -55,7 +56,7 @@
home = "/home/Computational/rpenacob";
description = "Raúl Peñacoba";
group = "Computational";
hosts = [ "hut" ];
hosts = [ "owl1" "owl2" "hut" ];
hashedPassword = "$6$TZm3bDIFyPrMhj1E$uEDXoYYd1z2Wd5mMPfh3DZAjP7ztVjJ4ezIcn82C0ImqafPA.AnTmcVftHEzLB3tbe2O4SxDyPSDEQgJ4GOtj/";
openssh.authorizedKeys.keys = [
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFYfXg37mauGeurqsLpedgA2XQ9d4Nm0ZGo/hI1f7wwH rpenacob@bsc"
@@ -75,6 +76,19 @@
];
};
abonerib = {
uid = 4541;
isNormalUser = true;
home = "/home/Computational/abonerib";
description = "Aleix Boné";
group = "Computational";
hosts = [ "owl1" "owl2" "hut" "raccoon" ];
hashedPassword = "$6$V1EQWJr474whv7XJ$OfJ0wueM2l.dgiJiiah0Tip9ITcJ7S7qDvtSycsiQ43QBFyP4lU0e0HaXWps85nqB4TypttYR4hNLoz3bz662/";
openssh.authorizedKeys.keys = [
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIIFiqXqt88VuUfyANkZyLJNiuroIITaGlOOTMhVDKjf abonerib@bsc"
];
};
vlopez = {
uid = 4334;
isNormalUser = true;
@@ -87,10 +101,24 @@
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGMwlUZRf9jfG666Qa5Sb+KtEhXqkiMlBV2su3x/dXHq victor@arch"
];
};
dbautist = {
uid = 5649;
isNormalUser = true;
home = "/home/Computational/dbautist";
description = "Dylan Bautista Cases";
group = "Computational";
hosts = [ "hut" "raccoon" ];
hashedPassword = "$6$a2lpzMRVkG9nSgIm$12G6.ka0sFX1YimqJkBAjbvhRKZ.Hl090B27pdbnQOW0wzyxVWySWhyDDCILjQELky.HKYl9gqOeVXW49nW7q/";
openssh.authorizedKeys.keys = [
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAb+EQBoS98zrCwnGKkHKwMLdYABMTqv7q9E0+T0QmkS dbautist@bsc-848818791"
];
};
};
groups = {
Computational = { gid = 564; };
tracing = { };
};
};
}

View File

@@ -2,8 +2,9 @@
# Provides the base system for a xeon node.
imports = [
./base.nix
./xeon/console.nix
./xeon/fs.nix
./xeon/getty.nix
./xeon/net.nix
./xeon/ssh.nix
];
}

View File

@@ -5,4 +5,10 @@
wantedBy = [ "getty.target" ];
serviceConfig.Restart = "always";
};
# Enable serial console
boot.kernelParams = [
"console=tty1"
"console=ttyS0,115200"
];
}

View File

@@ -10,7 +10,7 @@
nameservers = ["8.8.8.8"];
proxy = {
default = "http://localhost:23080/";
default = "http://hut:23080/";
noProxy = "127.0.0.1,localhost,internal.domain,10.0.40.40";
# Don't set all_proxy as go complains and breaks the gitlab runner, see:
# https://github.com/golang/go/issues/16715

8
m/common/xeon/ssh.nix Normal file
View File

@@ -0,0 +1,8 @@
{
# Connect to intranet git hosts via proxy
programs.ssh.extraConfig = ''
Host bscpm02.bsc.es bscpm03.bsc.es gitlab-internal.bsc.es alya.gitlab.bsc.es
User git
ProxyCommand nc -X connect -x hut:23080 %h %p
'';
}

View File

@@ -6,6 +6,7 @@
../module/ceph.nix
../module/debuginfod.nix
../module/emulation.nix
../module/slurm-client.nix
./gitlab-runner.nix
./monitoring.nix
@@ -19,8 +20,6 @@
#./pxe.nix
];
boot.binfmt.emulatedSystems = [ "armv7l-linux" "aarch64-linux" "powerpc64le-linux" "riscv64-linux" ];
# Select the this using the ID to avoid mismatches
boot.loader.grub.device = "/dev/disk/by-id/ata-INTEL_SSDSC2BB240G7_PHDV6462004Y240AGN";
@@ -34,5 +33,15 @@
address = "10.0.42.7";
prefixLength = 24;
} ];
firewall = {
extraCommands = ''
# Accept all proxy traffic from compute nodes but not the login
iptables -A nixos-fw -p tcp -s 10.0.40.30 --dport 23080 -j nixos-fw-log-refuse
iptables -A nixos-fw -p tcp -s 10.0.40.0/24 --dport 23080 -j nixos-fw-accept
'';
};
};
# Allow proxy to bind to the ethernet interface
services.openssh.settings.GatewayPorts = "clientspecified";
}

View File

@@ -17,6 +17,7 @@
REGISTER_MANUAL_CONFIRM = true;
ENABLE_NOTIFY_MAIL = true;
};
log.LEVEL = "Warn";
mailer = {
ENABLED = true;

View File

@@ -1,9 +1,8 @@
{ pkgs, lib, config, ... }:
{
age.secrets.ovniToken.file = ../../secrets/ovni-token.age;
age.secrets.gitlabToken.file = ../../secrets/gitlab-bsc-es-token.age;
age.secrets.nosvToken.file = ../../secrets/nosv-token.age;
age.secrets.gitlabRunnerShellToken.file = ../../secrets/gitlab-runner-shell-token.age;
age.secrets.gitlabRunnerDockerToken.file = ../../secrets/gitlab-runner-docker-token.age;
services.gitlab-runner = {
enable = true;
@@ -11,20 +10,14 @@
services = let
common-shell = {
executor = "shell";
tagList = [ "nix" "xeon" ];
registrationFlags = [
# Using space doesn't work, and causes it to misread the next flag
"--locked='false'"
];
environmentVariables = {
SHELL = "${pkgs.bash}/bin/bash";
};
};
common-docker = {
executor = "docker";
dockerImage = "debian:stable";
tagList = [ "docker" "xeon" ];
registrationFlags = [
"--locked='false'"
"--docker-network-mode host"
];
environmentVariables = {
@@ -33,19 +26,12 @@
};
};
in {
# For gitlab.bsc.es
gitlab-bsc-es-shell = common-shell // {
registrationConfigFile = config.age.secrets.gitlabToken.path;
};
gitlab-bsc-es-docker = common-docker // {
registrationConfigFile = config.age.secrets.gitlabToken.path;
};
# For pm.bsc.es/gitlab
gitlab-pm-shell = common-shell // {
registrationConfigFile = config.age.secrets.ovniToken.path;
authenticationTokenConfigFile = config.age.secrets.gitlabRunnerShellToken.path;
};
gitlab-pm-docker = common-docker // {
registrationConfigFile = config.age.secrets.ovniToken.path;
authenticationTokenConfigFile = config.age.secrets.gitlabRunnerDockerToken.path;
};
};
};

View File

@@ -31,6 +31,7 @@
};
feature_toggles.publicDashboards = true;
"auth.anonymous".enabled = true;
log.level = "warn";
};
};

View File

@@ -8,6 +8,10 @@
boot.loader.grub.device = "/dev/disk/by-id/wwn-0x55cd2e414d53563a";
boot.kernel.sysctl = {
"kernel.yama.ptrace_scope" = lib.mkForce "1";
};
environment.systemPackages = with pkgs; [
ceph
];

3
m/module/emulation.nix Normal file
View File

@@ -0,0 +1,3 @@
{
boot.binfmt.emulatedSystems = [ "armv7l-linux" "aarch64-linux" "powerpc64le-linux" "riscv64-linux" ];
}

View File

@@ -47,8 +47,8 @@ in {
];
partitionName = [
"owl Nodes=owl[1-2] Default=YES MaxTime=INFINITE State=UP"
"all Nodes=owl[1-2],hut Default=NO MaxTime=INFINITE State=UP"
"owl Nodes=owl[1-2] Default=YES DefaultTime=01:00:00 MaxTime=INFINITE State=UP"
"all Nodes=owl[1-2],hut Default=NO DefaultTime=01:00:00 MaxTime=INFINITE State=UP"
];
# See slurm.conf(5) for more details about these options.
@@ -83,6 +83,14 @@ in {
# Reduce port range so we can allow only this range in the firewall
SrunPortRange=60000-61000
# Use cores as consumable resources. In SLURM terms, a core may have
# multiple hardware threads (or CPUs).
SelectType=select/cons_tres
# Ignore memory constraints and only use unused cores to share a node with
# other jobs.
SelectTypeParameters=CR_Core
'';
};

View File

@@ -4,9 +4,10 @@
imports = [
../common/xeon.nix
../module/ceph.nix
../module/emulation.nix
../module/slurm-client.nix
../module/slurm-firewall.nix
../module/slurm-hut-nix-store.nix
../module/debuginfod.nix
];
# Select the this using the ID to avoid mismatches

View File

@@ -4,9 +4,10 @@
imports = [
../common/xeon.nix
../module/ceph.nix
../module/emulation.nix
../module/slurm-client.nix
../module/slurm-firewall.nix
../module/slurm-hut-nix-store.nix
../module/debuginfod.nix
];
# Select the this using the ID to avoid mismatches

View File

@@ -3,11 +3,21 @@
{
imports = [
../common/base.nix
../module/emulation.nix
../module/debuginfod.nix
../eudy/kernel/lttng.nix
../eudy/kernel/perf.nix
];
# Don't install Grub on the disk yet
boot.loader.grub.device = "nodev";
# Enable serial console
boot.kernelParams = [
"console=tty1"
"console=ttyS1,115200"
];
networking = {
hostName = "raccoon";
# Only BSC DNSs seem to be reachable from the office VLAN
@@ -19,17 +29,48 @@
} ];
};
# Enable performance governor
powerManagement.cpuFreqGovernor = "performance";
# Configure Nvidia driver to use with CUDA
hardware.nvidia.package = config.boot.kernelPackages.nvidiaPackages.production;
hardware.opengl = {
enable = true;
driSupport = true;
setLdLibraryPath = true;
};
hardware.graphics.enable = true;
nixpkgs.config.allowUnfree = true;
nixpkgs.config.nvidia.acceptLicense = true;
services.xserver.videoDrivers = [ "nvidia" ];
# Disable garbage collection for now
nix.gc.automatic = lib.mkForce false;
# Use nix cache from hut
nix.settings = {
substituters = [ "https://jungle.bsc.es/cache" ];
trusted-public-keys = [ "jungle.bsc.es:pEc7MlAT0HEwLQYPtpkPLwRsGf80ZI26aj29zMw/HH0=" ];
};
services.openssh.settings.X11Forwarding = true;
nixpkgs.overlays = [
(final: prev: {
xilinx-xrt = prev.callPackage ./xilinx-xrt.nix { };
xilinx-fw = prev.callPackage ./xilinx-fw.nix { };
xilinx-xocl = prev.callPackage ./xilinx-xocl.nix {
kernel = config.boot.kernelPackages.kernel;
};
})
];
boot.extraModulePackages = [ pkgs.xilinx-xocl ];
boot.kernelModules = [ "xclmgmt" "xocl" ];
services.udev.packages = [ pkgs.xilinx-xocl ];
services.prometheus.exporters.node = {
enable = true;
enabledCollectors = [ "systemd" ];
port = 9002;
listenAddress = "127.0.0.1";
};
users.motd = ''

335
m/raccoon/xilinx-create-xsabin.sh Executable file
View File

@@ -0,0 +1,335 @@
#!/bin/bash
## (c) Copyright 2020 Xilinx, Inc. All rights reserved.
##
## This file contains confidential and proprietary information
## of Xilinx, Inc. and is protected under U.S. and
## international copyright and other intellectual property
## laws.
##
## DISCLAIMER
## This disclaimer is not a license and does not grant any
## rights to the materials distributed herewith. Except as
## otherwise provided in a valid license issued to you by
## Xilinx, and to the maximum extent permitted by applicable
## law: (1) THESE MATERIALS ARE MADE AVAILABLE "AS IS" AND
## WITH ALL FAULTS, AND XILINX HEREBY DISCLAIMS ALL WARRANTIES
## AND CONDITIONS, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING
## BUT NOT LIMITED TO WARRANTIES OF MERCHANTABILITY, NON-
## INFRINGEMENT, OR FITNESS FOR ANY PARTICULAR PURPOSE; and
## (2) Xilinx shall not be liable (whether in contract or tort,
## including negligence, or under any other theory of
## liability) for any loss or damage of any kind or nature
## related to, arising under or in connection with these
## materials, including for any direct, or any indirect,
## special, incidental, or consequential loss or damage
## (including loss of data, profits, goodwill, or any type of
## loss or damage suffered as a result of any action brought
## by a third party) even if such damage or loss was
## reasonably foreseeable or Xilinx had been advised of the
## possibility of the same.
##
## CRITICAL APPLICATIONS
## Xilinx products are not designed or intended to be fail-
## safe, or for use in any application requiring fail-safe
## performance, such as life-support or safety devices or
## systems, Class III medical devices, nuclear facilities,
## applications related to the deployment of airbags, or any
## other applications that could lead to death, personal
## injury, or severe property or environmental damage
## (individually and collectively, "Critical
## Applications"). Customer assumes the sole risk and
## liability of any use of Xilinx products in Critical
## Applications, subject only to applicable laws and
## regulations governing limitations on product liability.
##
## THIS COPYRIGHT NOTICE AND DISCLAIMER MUST BE RETAINED AS
## PART OF THIS FILE AT ALL TIMES.
# This script must be run with root permissions
# if [[ "$EUID" -ne 0 ]]; then
# echo "This script must be run as root."
# exit
# fi
# Get absolute path to this script with any symlinks resolved
realme=$(realpath $0)
scriptpath="${realme%/*}"
echo "This is create_xsabin.sh running from $scriptpath on $(date)"
# The directory above that is the human-readable installation path - probably /opt/xilinx/firmware/<card>/<family>/<partition>/
humanpath=${scriptpath%/*}
pushd $humanpath > /dev/null
# This script may be called during firmware upgrade, in which case the firmware product, branch, version and release
# are provided as script arguments, to help this script to select the new firmware file
if [[ "$#" -ge 4 ]]; then
firmware_upgrade_product=$1
firmware_upgrade_branch=$2
firmware_upgrade_version=$3
firmware_upgrade_release=$4
echo "Run for install of firmware $firmware_upgrade_product-$firmware_upgrade_branch version $firmware_upgrade_version release $firmware_upgrade_release"
elif [[ "$#" -eq 3 ]]; then
# If 3 script arguments, these are the partition name, version and release, so that this script can report them for debug
echo "Run for install of partition $1 version $2 release $3"
fi
# Find the partition_metadata.json link in the install directory
jsonlink="partition_metadata.json"
if [[ ! -e "$jsonlink" ]]; then
echo "Cannot find $jsonlink file in $humanpath - install failed"
exit 1
fi
# Find the machine-readable directory for this partition:
# this is the target of the partition_metadata.json link
if [[ ! -h "$jsonlink" ]]; then
echo "$jsonlink in $humanpath should be a symlink, but it is not - install failed"
exit 1
fi
jsonpath=$(readlink $jsonlink)
if [[ $? -ne 0 ]]; then
echo "Failed to read target of symlink $jsonlink in $humanpath - install failed"
exit 1
fi
if [[ ! -e "$jsonpath" ]]; then
echo "Target of symlink $jsonlink in $humanpath is $jsonpath, which does not exist - install failed"
exit 1
fi
echo "Metadata file is $jsonpath"
machinepath=${jsonpath%/*}
json=${jsonpath##*/}
echo "User install path is $humanpath"
echo "Machine-readable path is $machinepath"
pushd $machinepath > /dev/null
# Parse the partition_metadata.json file to find the required firmware
declare -A firmware
firmware_products=()
product=""
branch="mainline"
major="*"
minor="*"
revision="*"
while IFS= read -r line; do
key=${line#*\"}
key=${key%%\"*}
value=${line%\"*}
value=${value##*\"}
numvalue=$value
if [[ "$numvalue" =~ ^0x ]]; then
numvalue=$(($numvalue))
fi
case "$key" in
"firmware")
# Starts a new section: record current one (except if before the first section)
if [[ -n "$product" ]]; then
firmware_products+=($product)
firmware["$product.branch"]=$branch
firmware["$product.version"]="$major.$minor.$revision"
product=""
branch="mainline"
major="*"
minor="*"
revision="*"
fi
;;
"firmware_product_name")
product=${value,,}
;;
"firmware_branch_name")
branch=${value,,}
;;
"firmware_version_major")
major=$numvalue
;;
"firmware_version_minor")
minor=$numvalue
;;
"firmware_version_revision")
revision=$numvalue
;;
esac
done <<< "$(grep '\"firmware' $json)"
# Record last section
firmware_products+=($product)
firmware["$product.branch"]=$branch
firmware["$product.version"]="$major.$minor.$revision"
# Locate the required firmware in existing installed directories, and build xclbinutil options to add firmware
# For each firmware, there is already a symlink in the human-readable directory's firmware directory
# which points to the existing firmware install directory
firmware_opts=""
for product in "${firmware_products[@]}"; do
uc_product=${product^^}
branch=${firmware[$product.branch]}
version=${firmware[$product.version]}
link="$humanpath/firmware/$product-$branch"
if [[ ! -L "$link" ]]; then
echo "Expected symlink $link for required $product firmware, but this either does not exist or is not a symlink - install failed"
exit 1
fi
firmware_path=$(readlink -f $humanpath/firmware/$product-$branch)
if [[ ! -e "$firmware_path" ]]; then
echo "Required $product firmware install directory not found at $firmware_path. Unable to build xsabin files"
exit 1
fi
# Locate the required firmware binary file
case "$product" in
"ert")
# ERT firmware is deployed within XRT and has its own file naming rule
if [[ "$branch" == "mainline" ]] || [[ "$branch" == "legacy" ]] || [[ "$branch" == "" ]]; then
ert_name="sched.bin"
else
ert_name="sched_$branch.bin"
fi
firmware_file="$firmware_path/$ert_name"
if [[ ! -e "$firmware_file" ]]; then
echo "Cannot locate required $product firmware: not found at $firmware_file. Unable to build xsabin files"
exit 1
fi
firmware_opts+=" --add-section SCHED_FIRMWARE:RAW:${firmware_file}"
;;
*)
# All other firmware is deployed in its own package
# Accommodate possible variations in firmware file name, as long as the file name contains the product name
# During firmware upgrade, it is possible that both the old and the new firmware files are both present
# (the old one may not be removed until after this script has run).
# In this situation, the new firmware product, branch and version are provided as script arguments:
# select the appropriate file here (if multiple files are found).
firmware_files=()
for globfile in $firmware_path/*; do
if [[ -e "$globfile" ]] && [[ ! -d "$globfile" ]]; then
globfilename=${globfile##*/}
if [[ "$globfilename" == *"$product"* ]] || [[ "$globfilename" == *"$uc_product"* ]]; then
firmware_files+=($globfile)
fi
fi
done
if [[ "${#firmware_files[@]}" -eq 0 ]]; then
echo "Cannot locate required $product firmware: not found at $firmware_path. Unable to build xsabin files"
exit 1
fi
firmware_file=""
if [[ "${#firmware_files[@]}" -gt 1 ]]; then
IFS=$'\n'
firmware_files=( $(sort -V <<<"${firmware_files[*]}") )
unset IFS
if [[ "$firmware_upgrade_product" == "$product" ]]; then
firmware_file=""
for fw_file in "${firmware_files[@]}"; do
fw_filename=${fw_file##*/}
if [[ "$fw_filename" == *"$firmware_upgrade_version"* ]]; then
firmware_file=$fw_file
fi
done
fi
fi
if [[ -z "$firmware_file" ]]; then
firmware_file="${firmware_files[-1]}"
fi
# Select the correct xsabin section name, depending on the firmware product
section=""
case "$product" in
"cmc")
section="FIRMWARE"
;;
"sc-fw" | "sc")
section="BMC-FW"
;;
*)
echo "Unrecognised firmware product name '$product', unable to select the correct xsabin section name"
exit 1
;;
esac
firmware_opts+=" --add-section ${section}:RAW:${firmware_file}"
# The SC firmware (BMC-FW section) may have a metadata JSON file also to be added
if [[ "$section" == "BMC-FW" && -e "$firmware_path/metadata.json" ]]; then
firmware_opts+=" --add-section BMC-METADATA:JSON:${firmware_path}/metadata.json"
fi
;;
esac
done
# Extract vendor, board, name and version from partition_metadata.json, to build PlatformVBNV
declare -A vbnv
for element in {partition_vendor,partition_card,partition_family,partition_name,installed_package_version}; do
line="$(grep $element $json)"
if [[ -n "$line" ]]; then
value=${line%\"*}
value=${value##*\"}
if [[ "$element" != "partition_vendor" ]] && [[ "$element" != "partition_card" ]]; then
value=${value//-/_}
fi
else
value="UNKNOWN"
fi
vbnv[$element]=$value
done
platform_vbnv="${vbnv[partition_vendor]}:${vbnv[partition_card]}:${vbnv[partition_family]}_${vbnv[partition_name]}:${vbnv[installed_package_version]}"
# Check for VBNV override in partition_metadata.json
line="$(grep vbnv_override $json)"
if [[ -n "$line" ]]; then
value=${line%\"*}
value=${value##*\"}
# Check VBNV override value is correctly formatted (4 fields separated by colons)
fields="$(echo "$value" | tr ':' ' ' | wc -w)"
if [[ "$fields" == "4" ]]; then
platform_vbnv=$value
fi
fi
# Use the XRT standard install path to find xclbinutil
xclbinutil="${xclbinutil:-/opt/xilinx/xrt/bin/xclbinutil}"
if [[ ! -e "$xclbinutil" ]]; then
echo "xclbinutil tool not found at $xclbinutil, unable to build xsabin files"
exit 1
fi
## Must source XRT's setup.sh to set up environment correctly
#if [[ ! -e /opt/xilinx/xrt/setup.sh ]]; then
# echo "XRT setup.sh not found at /opt/xilinx/xrt/setup.sh, XRT installation is bad. Cannot build xsabin files"
# exit 1
#fi
#source /opt/xilinx/xrt/setup.sh
# Build xclbinutil options for creating xsabin files
xclbinopts=" --force"
if [[ -e "partition.mcs" ]]; then
xclbinopts+=" --add-section MCS-PRIMARY:RAW:partition.mcs"
fi
if [[ -e "partition_secondary.mcs" ]]; then
xclbinopts+=" --add-section MCS-SECONDARY:RAW:partition_secondary.mcs"
fi
if [[ -e "partition.bin" ]]; then
xclbinopts+=" --add-section FLASH[BIN]-DATA:RAW:partition.bin"
fi
if [[ -e "bin_metadata.json" ]]; then
xclbinopts+=" --add-section FLASH[BIN]-METADATA:JSON:bin_metadata.json"
fi
if [[ -e "partition.bit" ]]; then
xclbinopts+=" --add-section BITSTREAM:RAW:partition.bit"
fi
if [[ -e "partition.pdi" ]]; then
xclbinopts+=" --add-section PDI:RAW:partition.pdi"
fi
xclbinopts+=" --add-section PARTITION_METADATA:JSON:${json}"
xclbinopts+=$firmware_opts
xclbinopts+=" --key-value SYS:PlatformVBNV:${platform_vbnv}"
# Create partition.xsabin
xsabin="partition.xsabin"
xclbincmd="${xclbinutil} ${xclbinopts} --output $xsabin"
echo $xclbincmd
$xclbincmd
if [[ $? -ne 0 ]]; then
echo "An error occurred while running xclbinutil"
exit 1
fi
if [[ ! -e "$xsabin" ]]; then
echo "xclbinutil did not create output file $xsabin"
exit 1
fi
# And we're done
echo "create_xsabin.sh completed successfully"

75
m/raccoon/xilinx-fw.nix Normal file
View File

@@ -0,0 +1,75 @@
{
stdenv
, lib
, dpkg
, fetchurl
, xilinx-xrt
}:
with lib;
# Must read: https://xilinx.github.io/XRT/master/html/platforms_partitions.html#shell
# Taken from:
# - https://aur.archlinux.org/packages/xilinx-sc-fw-u280
# - https://aur.archlinux.org/packages/xilinx-u280-gen3x16-xdma-base
stdenv.mkDerivation rec {
pname = "xilinx-fw";
version = "1.3.5-3592445";
srcs = [
# List packages with: curl https://packages.xilinx.com/artifactory/debian-packages-cache/pool/
(fetchurl {
url = "https://packages.xilinx.com/artifactory/debian-packages-cache/pool/xilinx-cmc-u280_1.3.5-3592445_all.deb";
hash = "sha256-H48bdeuBc9dK6LExMnw1RCfY85PKntBk/X8CMcAI+zI=";
})
(fetchurl {
url = "https://packages.xilinx.com/artifactory/debian-packages-cache/pool/xilinx-sc-fw-u280_4.3.28-1.ea1b92f_all.deb";
hash = "sha256-JxQal2IqYAgebAgfjs2noFG5ghxC9sJQFppJFUCx6jA=";
})
(fetchurl {
url = "https://packages.xilinx.com/artifactory/debian-packages-cache/pool/xilinx-u280-gen3x16-xdma-base_1-3585717_all.deb";
hash = "sha256-oe84YgmmRFZjNa63j0pIneuFUG0Bb4aA7wulyU4bCrY=";
})
(fetchurl {
url = "https://packages.xilinx.com/artifactory/debian-packages-cache/pool/xilinx-u280-gen3x16-xdma-validate_1-3585755_all.deb";
hash = "sha256-F+IAzR8NVc9FDsgQstpBcKeq3ogH1PI8nuq94sEExCg=";
})
# Needed for the ERT firmware
(fetchurl {
url = "https://packages.xilinx.com/artifactory/debian-packages-cache/pool/xrt_202320.2.16.204_22.04-amd64-xrt.deb";
hash = "sha256-FEhzx2KlIYpunXmTSBjtyAtblbuz5tkvnt2qp21gUho=";
})
];
dontStrip = true;
hardeningDisable = [ "all" ];
nativeBuildInputs = [ dpkg ];
unpackPhase = ''
for f in $srcs; do
dpkg-deb -x "$f" deb
done
sourceRoot=deb
'';
# Generate the xsabin firmware file by fixing the original script
buildPhase = ''
set -x
ln -rs lib/firmware/xilinx/283bab8f654d8674968f4da57f7fa5d7 lib/firmware/xilinx/fb2b2c5a19ed63593fea95f51fbc8eb9
ln -rs lib/firmware/xilinx/283bab8f654d8674968f4da57f7fa5d7/partition_metadata.json opt/xilinx/firmware/u280/gen3x16-xdma/base/partition_metadata.json
ln -rs lib/firmware/xilinx/283bab8f654d8674968f4da57f7fa5d7/partition.xsabin opt/xilinx/firmware/u280/gen3x16-xdma/base/partition.xsabin
ln -rs opt/xilinx/xrt/share/fw opt/xilinx/firmware/u280/gen3x16-xdma/base/firmware/ert-v30
ln -rs opt/xilinx/firmware/cmc/u280 opt/xilinx/firmware/u280/gen3x16-xdma/base/firmware/cmc-u280
ln -rs opt/xilinx/firmware/sc-fw/u280 opt/xilinx/firmware/u280/gen3x16-xdma/base/firmware/sc-fw-u280
find
export xclbinutil=${xilinx-xrt}/xrt/bin/xclbinutil
cp -a ${./xilinx-create-xsabin.sh} opt/xilinx/firmware/u280/gen3x16-xdma/base/scripts/create_xsabin.sh
bash -x opt/xilinx/firmware/u280/gen3x16-xdma/base/scripts/create_xsabin.sh xilinx-u280-gen3x16-xdma-base 1 3585717
set +x
'';
installPhase = ''
mkdir -p $out
cp -a * $out
'';
}

View File

@@ -0,0 +1,27 @@
--- a/driver/xocl/mgmtpf/Makefile 2025-02-20 15:59:28.379826176 +0100
+++ b/driver/xocl/mgmtpf/Makefile 2025-02-20 15:59:42.366892140 +0100
@@ -119,10 +119,6 @@ all:
install: all
$(MAKE) -C $(KERNEL_SRC) M=$(PWD) modules_install
- depmod -a
- install -m 644 99-xclmgmt.rules /etc/udev/rules.d
- -rmmod -s xclmgmt || true
- -modprobe xclmgmt
clean:
rm -rf *.o *.o.d *.o.cmd *~ core .depend .*.cmd *.ko *.ko.unsigned \
--- a/driver/xocl/userpf/Makefile 2025-02-20 16:03:20.751922522 +0100
+++ b/driver/xocl/userpf/Makefile 2025-02-20 16:03:35.377991553 +0100
@@ -138,11 +138,6 @@ all:
install: all
$(MAKE) -C $(KERNEL_SRC) M=$(PWD) modules_install
- depmod -a
- install -m 644 99-xocl.rules /etc/udev/rules.d
- -rmmod -s xocl || true
- -rmmod -s xdma || true
- -modprobe xocl
clean:
rm -rf *.o *.o.d *~ core .depend .*.cmd *.ko *.ko.unsigned *.mod.c \

35
m/raccoon/xilinx-xocl.nix Normal file
View File

@@ -0,0 +1,35 @@
{
stdenv
, lib
, kernel
, xilinx-xrt
}:
with lib;
# See: https://iotlab.sdsu.edu/index.php/flash-base-image-on-xilinx-alveo-u280/
stdenv.mkDerivation rec {
pname = "xilinx-xocl";
version = "2.19.0";
src = "${xilinx-xrt}/src/xrt-${version}";
dontStrip = true;
preBuild = ''
cd driver/xocl
'';
patches = [
./xilinx-xocl-depmod.patch
];
buildFlags = [ "KERNEL_SRC=${kernel.dev}/lib/modules/${kernel.modDirVersion}/build" ];
installFlags = [
"KERNEL_SRC=${kernel.dev}/lib/modules/${kernel.modDirVersion}/build"
"INSTALL_MOD_PATH=${placeholder "out"}"
];
postInstall = ''
mkdir -p $out/etc/udev/rules.d
install -m 644 userpf/99-xocl.rules $out/etc/udev/rules.d
install -m 644 mgmtpf/99-xclmgmt.rules $out/etc/udev/rules.d
'';
nativeBuildInputs = kernel.moduleBuildDependencies;
hardeningDisable = [ "all" ];
}

View File

@@ -0,0 +1,25 @@
--- a/src/runtime_src/core/common/aiebu/src/cpp/aiebu/utils/asm/CMakeLists.txt
+++ b/src/runtime_src/core/common/aiebu/src/cpp/aiebu/utils/asm/CMakeLists.txt
@@ -23,8 +23,6 @@ add_executable(aiebu-asm $<TARGET_OBJECTS:aiebu_asm_objects>)
target_link_libraries(aiebu-asm PRIVATE aiebu_static)
if (${CMAKE_SYSTEM_NAME} STREQUAL "Linux")
- target_link_options(aiebu-asm PRIVATE "-static")
- set_target_properties(aiebu-asm PROPERTIES INSTALL_RPATH "" BUILD_RPATH "")
# Create a dynamically linked executable. aiebu-asm-dyn, on Linux for running
# valgrind, etc. This binary is not released for deployment but only used for
@@ -35,13 +33,6 @@ if (${CMAKE_SYSTEM_NAME} STREQUAL "Linux")
target_link_libraries(aiebu-asm-dyn PRIVATE aiebu_static)
endif()
-# This custom target fails if aiebu-asm has any dynamic dependencies
-add_custom_target(check_dynamic_deps ALL
- COMMAND ${CMAKE_COMMAND} -E echo "Checking for dynamic dependencies ..."
- COMMAND ${CMAKE_COMMAND} -P "${AIEBU_SOURCE_DIR}/cmake/depends.cmake" $<TARGET_FILE:aiebu-asm> aiebu-asm_depends.txt
- DEPENDS aiebu-asm
- )
-
install(TARGETS aiebu-asm
RUNTIME DESTINATION ${AIEBU_INSTALL_BIN_DIR}
CONFIGURATIONS Debug Release COMPONENT Runtime

View File

@@ -0,0 +1,13 @@
diff --git a/src/CMake/icd.cmake b/src/CMake/icd.cmake
index 255a2e3d8..460a6d4c7 100644
--- a/src/CMake/icd.cmake
+++ b/src/CMake/icd.cmake
@@ -10,7 +10,7 @@ configure_file (
${ICD_FILE_NAME}
)
-set(OCL_ICD_INSTALL_PREFIX "/etc/OpenCL/vendors")
+set(OCL_ICD_INSTALL_PREFIX "${CMAKE_INSTALL_PREFIX}/etc/OpenCL/vendors")
install (FILES ${CMAKE_CURRENT_BINARY_DIR}/${ICD_FILE_NAME}
DESTINATION ${OCL_ICD_INSTALL_PREFIX}

View File

@@ -0,0 +1,204 @@
From 6f64871f2e679ad5d3b140c8a2732edaae2dcf6a Mon Sep 17 00:00:00 2001
From: Rodrigo Arias Mallo <rodrigo.arias@bsc.es>
Date: Thu, 20 Feb 2025 18:49:54 +0100
Subject: [PATCH] Revert "Removed support for u50lv, u55n and u280 platforms in
XRT (#7901)"
This reverts commit 41f4221433c6b173316b61cb2e7e3ee5152d8075.
---
.../core/pcie/driver/linux/xocl/devices.h | 103 ++++++++++++++++++
1 file changed, 103 insertions(+)
diff --git a/src/runtime_src/core/pcie/driver/linux/xocl/devices.h b/src/runtime_src/core/pcie/driver/linux/xocl/devices.h
index 971ad73d2..5fe329cfa 100644
--- a/src/runtime_src/core/pcie/driver/linux/xocl/devices.h
+++ b/src/runtime_src/core/pcie/driver/linux/xocl/devices.h
@@ -2064,6 +2064,14 @@ struct xocl_subdev_map {
.subdev_num = ARRAY_SIZE(USER_RES_DSA52), \
}
+#define XOCL_BOARD_USER_DSA52_U280 \
+ (struct xocl_board_private){ \
+ .flags = 0, \
+ .subdev_info = USER_RES_DSA52, \
+ .subdev_num = ARRAY_SIZE(USER_RES_DSA52), \
+ .p2p_bar_sz = 64, \
+ }
+
#define XOCL_BOARD_USER_SMARTN \
(struct xocl_board_private){ \
.flags = XOCL_DSAFLAG_SMARTN, \
@@ -2370,6 +2378,30 @@ struct xocl_subdev_map {
.flash_type = FLASH_TYPE_SPI, \
}
+
+#define MGMT_RES_XBB_DSA52_U280 \
+ ((struct xocl_subdev_info []) { \
+ XOCL_DEVINFO_FEATURE_ROM, \
+ XOCL_DEVINFO_PRP_IORES_MGMT, \
+ XOCL_DEVINFO_AXIGATE_ULP, \
+ XOCL_DEVINFO_CLOCK_HBM, \
+ XOCL_DEVINFO_AF_DSA52, \
+ XOCL_DEVINFO_XMC, \
+ XOCL_DEVINFO_XVC_PRI, \
+ XOCL_DEVINFO_MAILBOX_MGMT, \
+ XOCL_DEVINFO_ICAP_MGMT, \
+ XOCL_DEVINFO_FMGR, \
+ XOCL_DEVINFO_FLASH, \
+ })
+
+#define XOCL_BOARD_MGMT_XBB_DSA52_U280 \
+ (struct xocl_board_private){ \
+ .flags = 0, \
+ .subdev_info = MGMT_RES_XBB_DSA52_U280, \
+ .subdev_num = ARRAY_SIZE(MGMT_RES_XBB_DSA52_U280), \
+ .flash_type = FLASH_TYPE_SPI, \
+ }
+
#define MGMT_RES_XBB_SMARTN \
((struct xocl_subdev_info []) { \
XOCL_DEVINFO_FEATURE_ROM_SMARTN, \
@@ -2772,6 +2804,24 @@ struct xocl_subdev_map {
.board_name = "u50" \
}
+#define XOCL_BOARD_U55N_USER_RAPTOR2 \
+ (struct xocl_board_private){ \
+ .flags = XOCL_DSAFLAG_DYNAMIC_IP, \
+ .board_name = "u55n", \
+ .subdev_info = RES_USER_VSEC, \
+ .subdev_num = ARRAY_SIZE(RES_USER_VSEC), \
+ }
+
+#define XOCL_BOARD_U55N_MGMT_RAPTOR2 \
+ (struct xocl_board_private){ \
+ .flags = XOCL_DSAFLAG_DYNAMIC_IP, \
+ .subdev_info = RES_MGMT_VSEC, \
+ .subdev_num = ARRAY_SIZE(RES_MGMT_VSEC), \
+ .flash_type = FLASH_TYPE_SPI, \
+ .board_name = "u55n", \
+ .vbnv = "xilinx_u55n" \
+ }
+
#define XOCL_BOARD_U55C_USER_RAPTOR2 \
(struct xocl_board_private){ \
.flags = XOCL_DSAFLAG_DYNAMIC_IP, \
@@ -2790,6 +2840,24 @@ struct xocl_subdev_map {
.vbnv = "xilinx_u55c" \
}
+#define XOCL_BOARD_U50LV_USER_RAPTOR2 \
+ (struct xocl_board_private){ \
+ .flags = XOCL_DSAFLAG_DYNAMIC_IP, \
+ .board_name = "u50lv", \
+ .subdev_info = RES_USER_VSEC, \
+ .subdev_num = ARRAY_SIZE(RES_USER_VSEC), \
+ }
+
+#define XOCL_BOARD_U50LV_MGMT_RAPTOR2 \
+ (struct xocl_board_private){ \
+ .flags = XOCL_DSAFLAG_DYNAMIC_IP, \
+ .subdev_info = RES_MGMT_VSEC, \
+ .subdev_num = ARRAY_SIZE(RES_MGMT_VSEC), \
+ .flash_type = FLASH_TYPE_SPI, \
+ .board_name = "u50lv", \
+ .vbnv = "xilinx_u50lv" \
+ }
+
#define XOCL_BOARD_U50C_USER_RAPTOR2 \
(struct xocl_board_private){ \
.flags = XOCL_DSAFLAG_DYNAMIC_IP, \
@@ -2834,6 +2902,14 @@ struct xocl_subdev_map {
.p2p_bar_sz = 64, \
}
+#define XOCL_BOARD_U280_USER_RAPTOR2 \
+ (struct xocl_board_private){ \
+ .flags = XOCL_DSAFLAG_DYNAMIC_IP, \
+ .subdev_info = RES_USER_VSEC, \
+ .subdev_num = ARRAY_SIZE(RES_USER_VSEC), \
+ .board_name = "u280", \
+ }
+
#define XOCL_BOARD_U250_MGMT_RAPTOR2 \
(struct xocl_board_private){ \
.flags = XOCL_DSAFLAG_DYNAMIC_IP, \
@@ -2843,6 +2919,15 @@ struct xocl_subdev_map {
.board_name = "u250" \
}
+#define XOCL_BOARD_U280_MGMT_RAPTOR2 \
+ (struct xocl_board_private){ \
+ .flags = XOCL_DSAFLAG_DYNAMIC_IP, \
+ .subdev_info = RES_MGMT_VSEC, \
+ .subdev_num = ARRAY_SIZE(RES_MGMT_VSEC), \
+ .flash_type = FLASH_TYPE_SPI, \
+ .board_name = "u280" \
+ }
+
#define XOCL_BOARD_VERSAL_USER_RAPTOR2 \
(struct xocl_board_private){ \
.flags = XOCL_DSAFLAG_DYNAMIC_IP | \
@@ -3435,6 +3520,8 @@ struct xocl_subdev_map {
{ XOCL_PCI_DEVID(0x10EE, 0x6A8F, 0x4353, MGMT_6A8F_DSA52) }, \
{ XOCL_PCI_DEVID(0x10EE, 0x5000, PCI_ANY_ID, MGMT_XBB_DSA52_U200) }, \
{ XOCL_PCI_DEVID(0x10EE, 0x5004, PCI_ANY_ID, MGMT_XBB_DSA52) }, \
+ { XOCL_PCI_DEVID(0x10EE, 0x5008, PCI_ANY_ID, MGMT_XBB_DSA52_U280) },\
+ { XOCL_PCI_DEVID(0x10EE, 0x500C, PCI_ANY_ID, MGMT_XBB_DSA52_U280) },\
{ XOCL_PCI_DEVID(0x10EE, 0x5020, PCI_ANY_ID, MGMT_U50) }, \
{ XOCL_PCI_DEVID(0x10EE, 0x5028, PCI_ANY_ID, MGMT_VERSAL) }, \
{ XOCL_PCI_DEVID(0x10EE, 0x5044, PCI_ANY_ID, MGMT_VERSAL) }, \
@@ -3448,7 +3535,9 @@ struct xocl_subdev_map {
{ XOCL_PCI_DEVID(0x10EE, 0x5078, PCI_ANY_ID, VERSAL_MGMT_RAPTOR2) }, \
{ XOCL_PCI_DEVID(0x10EE, 0x5050, PCI_ANY_ID, MGMT_U25) }, \
{ XOCL_PCI_DEVID(0x10EE, 0x504E, PCI_ANY_ID, U26Z_MGMT_RAPTOR2) }, \
+ { XOCL_PCI_DEVID(0x10EE, 0x5058, PCI_ANY_ID, U55N_MGMT_RAPTOR2) },\
{ XOCL_PCI_DEVID(0x10EE, 0x505C, PCI_ANY_ID, U55C_MGMT_RAPTOR2) },\
+ { XOCL_PCI_DEVID(0x10EE, 0x5060, PCI_ANY_ID, U50LV_MGMT_RAPTOR2) },\
{ XOCL_PCI_DEVID(0x10EE, 0x506C, PCI_ANY_ID, U50C_MGMT_RAPTOR2) },\
{ XOCL_PCI_DEVID(0x10EE, 0x5074, PCI_ANY_ID, X3522PV_MGMT_RAPTOR2) }, \
{ XOCL_PCI_DEVID(0x13FE, 0x006C, PCI_ANY_ID, MGMT_6A8F) }, \
@@ -3457,6 +3546,8 @@ struct xocl_subdev_map {
{ XOCL_PCI_DEVID(0x10EE, 0xF987, PCI_ANY_ID, XBB_MFG("samsung_efuse")) },\
{ XOCL_PCI_DEVID(0x10EE, 0xD000, PCI_ANY_ID, XBB_MFG("u200")) },\
{ XOCL_PCI_DEVID(0x10EE, 0xD004, PCI_ANY_ID, XBB_MFG("u250")) },\
+ { XOCL_PCI_DEVID(0x10EE, 0xD008, PCI_ANY_ID, XBB_MFG("u280-es1")) }, \
+ { XOCL_PCI_DEVID(0x10EE, 0xD00C, PCI_ANY_ID, XBB_MFG("u280")) },\
{ XOCL_PCI_DEVID(0x10EE, 0xD030, PCI_ANY_ID, XBB_MFG("poc1465")) },\
{ XOCL_PCI_DEVID(0x10EE, 0xD020, PCI_ANY_ID, XBB_MFG_U50) }, \
{ XOCL_PCI_DEVID(0x10EE, 0xD03C, PCI_ANY_ID, XBB_MFG_U30) }, \
@@ -3495,11 +3586,15 @@ struct xocl_subdev_map {
{ XOCL_PCI_DEVID(0x10EE, 0x7990, 0x4352, USER_DSA52) }, \
{ XOCL_PCI_DEVID(0x10EE, 0x5001, PCI_ANY_ID, USER_DSA52) }, \
{ XOCL_PCI_DEVID(0x10EE, 0x5005, PCI_ANY_ID, USER_DSA52) }, \
+ { XOCL_PCI_DEVID(0x10EE, 0x5009, PCI_ANY_ID, USER_DSA52_U280) }, \
+ { XOCL_PCI_DEVID(0x10EE, 0x500D, PCI_ANY_ID, USER_DSA52_U280) }, \
{ XOCL_PCI_DEVID(0x10EE, 0x5021, PCI_ANY_ID, USER_U50) }, \
{ XOCL_PCI_DEVID(0x10EE, 0x5051, PCI_ANY_ID, USER_U25) }, \
{ XOCL_PCI_DEVID(0x10EE, 0x504F, PCI_ANY_ID, U26Z_USER_RAPTOR2) }, \
{ XOCL_PCI_DEVID(0x10EE, 0x513D, PCI_ANY_ID, U30_USER_RAPTOR2) }, \
+ { XOCL_PCI_DEVID(0x10EE, 0x5059, PCI_ANY_ID, U55N_USER_RAPTOR2) },\
{ XOCL_PCI_DEVID(0x10EE, 0x505D, PCI_ANY_ID, U55C_USER_RAPTOR2) },\
+ { XOCL_PCI_DEVID(0x10EE, 0x5061, PCI_ANY_ID, U50LV_USER_RAPTOR2) },\
{ XOCL_PCI_DEVID(0x10EE, 0x506D, PCI_ANY_ID, U50C_USER_RAPTOR2) },\
{ XOCL_PCI_DEVID(0x10EE, 0x5075, PCI_ANY_ID, X3522PV_USER_RAPTOR2) }, \
{ XOCL_PCI_DEVID(0x13FE, 0x0065, PCI_ANY_ID, USER_XDMA) }, \
@@ -3561,6 +3656,14 @@ struct xocl_subdev_map {
.vbnv = "xilinx_u250", \
.priv_data = &XOCL_BOARD_U250_MGMT_RAPTOR2, \
.type = XOCL_DSAMAP_RAPTOR2 }, \
+ { 0x10EE, 0x500D, PCI_ANY_ID, \
+ .vbnv = "xilinx_u280", \
+ .priv_data = &XOCL_BOARD_U280_USER_RAPTOR2, \
+ .type = XOCL_DSAMAP_RAPTOR2 }, \
+ { 0x10EE, 0x500C, PCI_ANY_ID, \
+ .vbnv = "xilinx_u280", \
+ .priv_data = &XOCL_BOARD_U280_MGMT_RAPTOR2, \
+ .type = XOCL_DSAMAP_RAPTOR2 }, \
{ 0x10EE, 0x5020, PCI_ANY_ID, \
.vbnv = "xilinx_u50", \
.priv_data = &XOCL_BOARD_U50_MGMT_RAPTOR2, \
--
2.45.2

74
m/raccoon/xilinx-xrt.nix Normal file
View File

@@ -0,0 +1,74 @@
{
stdenv
, fetchFromGitHub
, enableDebug ? false
, lib
, cmake
, pkg-config
, libdrm
, libelf
, opencl-headers
, ocl-icd
, git
, boost
, ncurses
, openssl
, rapidjson
, protobuf
, python3
, libuuid
, curl
, libsystemtap
, libxcrypt
, udev
}:
with lib;
stdenv.mkDerivation rec {
name = "xilinx-xrt";
version = "dc81a9cc";
src = fetchFromGitHub {
owner = "Xilinx";
repo = "XRT";
rev = "dc81a9cc852bf44e71aa3edde7c8f7d54f355eab";
hash = "sha256-SG1gIO8Bvgs5XQ7HswjWNavPH+m8xHXqauztuJa6aEo=";
fetchSubmodules = true;
};
dontStrip = true;
patches = [
./xilinx-xrt-aiebu.patch
./xilinx-xrt-icd.patch
./xilinx-xrt-u280-support.patch
];
cmakeFlags = [
"-DXRT_INSTALL_PREFIX=${placeholder "out"}"
"-DXRT_INSTALL_DIR=${placeholder "out"}"
"-DXRT_NATIVE_BUILD=yes"
"-DCMAKE_BUILD_WITH_INSTALL_RPATH=ON"
# Enable debug
"-DCMAKE_BUILD_TYPE=RelWithDebInfo"
#"-DCMAKE_BUILD_TYPE=Debug"
#"-DXOCL_VERBOSE=1"
#"-DXRT_VERBOSE=1"
];
# A directory named "build" already exists
cmakeBuildDir = "the-build";
# Replace all occurences of /usr to $out, although some are not correct. By
# default they are replaced by /var/empty
dontFixCmake = true;
preConfigure = ''
find "." -type f \( -name "*.cmake" -o -name "*.cmake.in" -o -name CMakeLists.txt \) -print |
while read fn; do
sed -e 's^/usr\([ /]\|$\)^'$out'\1^g' -e 's^/opt\([ /]\|$\)^'$out'\1^g' < "$fn" > "$fn.tmp"
mv "$fn.tmp" "$fn"
done
'';
nativeBuildInputs = [ cmake pkg-config git ];
buildInputs = [ libdrm.dev opencl-headers ocl-icd boost.dev ncurses
openssl.dev rapidjson protobuf python3 libelf libuuid.dev curl.dev
libsystemtap libxcrypt udev.out udev.dev
];
hardeningDisable = [ "all" ];
}

View File

@@ -1,11 +0,0 @@
age-encryption.org/v1
-> ssh-ed25519 HY2yRg caTbx0NBmsTSmZH4HtBaxhsauWqWUDTesJqT08UsoEQ
8ND31xuco+H8d5SKg8xsCFRPVDhU4d8UKwV1BnmKVjQ
-> ssh-ed25519 CAWG4Q 4ETYuhCwHHECkut4DWDknMMgpAvFqtzLWVC2Wi2L8FM
BGMvRnAfd8qZG5hzLefmk32FkGvwzE9pqBUyx4JY0co
-> ssh-ed25519 MSF3dg hj5QL4ZfylN8/W/MXQHvVqtI7mRvlQOYr8HsaQEmPB0
kvB7sljmmkswSGZDQnrwdTbTsN78EAwH3pz1pPe0Hu0
-> )Q-grease vHF} [8p1> @7z;C"/
tgSUKFyyrf2jLXZp+pakigwB2fRO/WFj2Qnt1aPjtVPEK92JbJ4
--- xzM0AhV4gTQE0Q7inJNo9vFj+crJQxWeI7u9pl7bqAI
<05>6nGJ<47>0B<03><>7F<37><46><EFBFBD>b<EFBFBD>ٽ2<D9BD>L<EFBFBD><4C><EFBFBD>]<18>2zl<7A><6C>&e<>K<EFBFBD>x<EFBFBD><78><EFBFBD>9SWN<19>V"Mf<4D><66><EFBFBD><EFBFBD>KHUC:1b;9St<53><74><EFBFBD><EFBFBD>Duѧ<75><D1A7>Ϣ<EFBFBD>̟<04><02><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>f<EFBFBD>7<1D><>1<>I(<28>d<><64><EFBFBD><EFBFBD><EFBFBD> <0C>

View File

@@ -0,0 +1,9 @@
age-encryption.org/v1
-> ssh-ed25519 HY2yRg WvKK6U1wQtx2pbUDfuaUIXTQiCulDkz7hgUCSwMfMzQ
jLktUMqKuVxukqzz++pHOKvmucUQqeKYy5IwBma7KxY
-> ssh-ed25519 CAWG4Q XKGuNNoYFl9bdZzsqYYTY7GsEt5sypLW4R+1uk78NmU
8dIA2GzRAwTGM5CDHSM2BUBsbXzEAUssWUz2PY2PaTg
-> ssh-ed25519 MSF3dg T630RsKuZIF/bp+KITnIIWWHsg6M/VQGqbWQZxqT+AA
SraZcgZJVtmUzHF/XR9J7aK5t5EDNpkC/av/WJUT/G8
--- /12G8pj9sbs591OM/ryhoLnSWWmzYcoqprk9uN/3g18
<EFBFBD><EFBFBD><EFBFBD><01>‡%<25>]yi"<22><><EFBFBD>L<EFBFBD> <0B><>H`<60>a$<24><>)<29>9ve<76>.0<EFBFBD>m<EFBFBD>K<EFBFBD>v<EFBFBD><EFBFBD> <0B>u"|1c<31>-%<25><>"<22>WF<12><><EFBFBD>A<EFBFBD><41>h<EFBFBD>$<05><>j<e<><65>x<EFBFBD>Lx<4C><78>.?<3F><><EFBFBD>:L<><4C><EFBFBD><EFBFBD>,<2C>u<EFBFBD>|<7C><>F|<7C>i<EFBFBD><69><EFBFBD>

Binary file not shown.

View File

@@ -1,11 +0,0 @@
age-encryption.org/v1
-> ssh-ed25519 HY2yRg hrdS7Dl/j+u3XVfM79ZJpZSlre9TcD7DTQ+EEAT6kEE
avUO96P1h7w2BYWgrQ7GpUgdaCV9AZL7eOTTcF9gfro
-> ssh-ed25519 CAWG4Q A5raRY1CAgFYZgoQ92GMyNejYNdHx/7Y6uTS+EjLPWA
FRFqT2Jz7qRcybaxkQTKHGl797LVXoHpYG4RZSrX/70
-> ssh-ed25519 MSF3dg D+R80Bg7W9AuiOMAqtGFZQl994dRBIegYRLmmTaeZ3o
BHvZsugRiuZ91b4jk91h30o3eF3hadSnVCwxXge95T8
-> BT/El`a-grease W{nq|Vm )bld 2Nl}4 N$#JGB4t
oLG+0S1aGfO/ohCfgGmhDhwwLi4H
--- 2I5C+FvBG/K1ZHh7C5QD39feTSLoFGwcTeZAmeILNsI
<EFBFBD><EFBFBD>W<EFBFBD>o<> <14><>d;<3B><>C<EFBFBD>.<2E><>_(<28>u

Binary file not shown.

View File

@@ -6,10 +6,9 @@ let
safe = keys.hostGroup.safe ++ adminsKeys;
in
{
"gitlab-bsc-es-token.age".publicKeys = hut;
"gitea-runner-token.age".publicKeys = hut;
"ovni-token.age".publicKeys = hut;
"nosv-token.age".publicKeys = hut;
"gitlab-runner-docker-token.age".publicKeys = hut;
"gitlab-runner-shell-token.age".publicKeys = hut;
"nix-serve.age".publicKeys = hut;
"jungle-robot-password.age".publicKeys = hut;