Compare commits
1 Commits
0c6fdc9183
...
maintenanc
| Author | SHA1 | Date | |
|---|---|---|---|
| e8d7ae345d |
156
doc/2025-05-maintenance-purchase.md
Normal file
156
doc/2025-05-maintenance-purchase.md
Normal file
@@ -0,0 +1,156 @@
|
||||
# Maintenance purchase 2025-05
|
||||
|
||||
We need to buy some components to replace broken parts or to have spare ones for
|
||||
when they break. We also need some tools to do basic repairs.
|
||||
|
||||
Here is the list:
|
||||
|
||||
- 11 x Power supply DELTA DPS-750XB A (700 W) (this is critical)
|
||||
- 57.69€/unit, 634.59€ total <https://es.aliexpress.com/item/1005004090017186.html>
|
||||
|
||||
- 8 x RAM DDR4 2400MHz PC4-19200 ECC Registered
|
||||
- 128.85€/pair, 515.40€ total <https://www.amazon.es/PC4-19200-REGISTERED-MEMORY-WORKSTATIONS-MOTHERBOARDS/dp/B06W9P3RKF>
|
||||
|
||||
- 1 x Set of screwdrivers
|
||||
- 23.99€ <https://www.amazon.es/BLOSTM-Juego-Destornilladores-Profesionales-Destornillador/dp/B09W9R8J3S>
|
||||
|
||||
- 1 x UART adaptor
|
||||
- 14.99€ <https://www.amazon.es/DSD-TECH-SH-U09C5-convertidor-Soporte/dp/B07WX2DSVB>
|
||||
|
||||
- 1 x SSD SATA disk of 2 TB
|
||||
- 135.99€ <https://www.amazon.es/Crucial-BX500-pulgadas-interno-CT2000BX500SSD101/dp/B0CCN9QWKT>
|
||||
|
||||
Total: 1324.96 €
|
||||
|
||||
# Rationale
|
||||
|
||||
Below is the search procedure I followed to come up with that list.
|
||||
|
||||
## Power supplies
|
||||
|
||||
They are the first components to fail. We already have some problems with the
|
||||
monitoring of some power supplies. They will soon stop being manufactured, so we
|
||||
should increase out stack.
|
||||
|
||||
Most Xeon nodes use the DELTA DPS-750XB A:
|
||||
|
||||
hut% sudo ipmitool fru
|
||||
...
|
||||
FRU Device Description : Pwr Supply 1 FRU (ID 2)
|
||||
Product Manufacturer : DELTA
|
||||
Product Name : DPS-750XB A
|
||||
Product Part Number : E98791-010
|
||||
Product Version : 05
|
||||
Product Serial : XXXXXXXXXXXXXXXXX
|
||||
|
||||
And we only have one per node. We should make the power supply redundant so we
|
||||
can tolerate it to fail without bringing down the node.
|
||||
|
||||
They are available on Amazon, but they are very expensive (287.54 €):
|
||||
|
||||
<https://www.amazon.es/DPS-750XB-E98791-010-alimentaci%C3%B3n-conmutada-Platinum/dp/B0DB65G4VT>
|
||||
|
||||
On Aliexpress they are much cheaper (57.69 €):
|
||||
|
||||
<https://es.aliexpress.com/item/1005004090017186.html>
|
||||
|
||||
We have 11 nodes plus the login, but I'm not able to figure out which power
|
||||
supply the login is using.
|
||||
|
||||
The login uses another one, AXX1100PCRPS, and only has one slot populated. We
|
||||
may want to also we another one, but I would need to reset the FRU and I don't
|
||||
have access to the login node. So I will leave this for Operations to deal with.
|
||||
We can live without the login if needed.
|
||||
|
||||
## RAM DIMM
|
||||
|
||||
The DIMM modules also experience errors, which are monitored by Linux. In some
|
||||
nodes we see non-recoverable errors that are no longer corrected by the ECC. We
|
||||
need to replace the bad modules.
|
||||
|
||||
Having two spare modules per node would be enough to cover most problems in the
|
||||
future.
|
||||
|
||||
> 16 GB, 2400 MHz RDIMM
|
||||
|
||||
The module from dmidecode:
|
||||
|
||||
Handle 0x0026, DMI type 17, 40 bytes
|
||||
Memory Device
|
||||
Array Handle: 0x0020
|
||||
Error Information Handle: Not Provided
|
||||
Total Width: 72 bits
|
||||
Data Width: 64 bits
|
||||
Size: 16 GB
|
||||
Form Factor: DIMM
|
||||
Set: None
|
||||
Locator: DIMM_B1
|
||||
Bank Locator: NODE 1
|
||||
Type: DDR4
|
||||
Type Detail: Synchronous
|
||||
Speed: 2400 MT/s
|
||||
Manufacturer: Micron
|
||||
Serial Number: XXXXXXXX
|
||||
Asset Tag:
|
||||
Part Number: 36ASF2G72PZ-2G3B1
|
||||
Rank: 2
|
||||
Configured Memory Speed: 2400 MT/s
|
||||
Minimum Voltage: Unknown
|
||||
Maximum Voltage: Unknown
|
||||
Configured Voltage: Unknown
|
||||
|
||||
Which is this module:
|
||||
|
||||
<https://www.amazon.com/Micron-PC4-19200-DDR4-2400MHz-Registered-MTA36ASF2G72PZ-2G3B1/dp/B01KBCNEGI>
|
||||
|
||||
But they have only one in stock. Here is more details:
|
||||
|
||||
> 16GB PC4-19200 DDR4-2400MHz
|
||||
|
||||
The must have the following features:
|
||||
|
||||
- 16 GB
|
||||
- DDR4
|
||||
- Speed at least 2400 MT/s
|
||||
- ECC
|
||||
- Registered
|
||||
- Best if from Micron
|
||||
|
||||
I would say having 8 spare modules would be enough for now, as we only have a
|
||||
few that are currently failing. We could upgrade the modules later, as they
|
||||
don't have much risk of stopping being manufactured like the power supplies.
|
||||
|
||||
These may work:
|
||||
|
||||
- 1 x 16GB, 69,11€ <https://www.amazon.es/PC4-19200-REGISTRADO-SERVIDORES-Estaciones-CHIPKILL/dp/B06X42HC9N>
|
||||
|
||||
- 2 x 16GB, 128,85€ <https://www.amazon.es/PC4-19200-REGISTERED-MEMORY-WORKSTATIONS-MOTHERBOARDS/dp/B06W9P3RKF>
|
||||
|
||||
It is cheaper to buy them by pairs, so let's use the last one.
|
||||
|
||||
## Screwdriver set
|
||||
|
||||
In order to change and replace the machine parts we need a set of screwdrivers.
|
||||
Instead of having to bring my own from home, I want to have one at BSC. These
|
||||
are enough and come in a nice box so I don't lose them:
|
||||
|
||||
<https://www.amazon.es/BLOSTM-Juego-Destornilladores-Profesionales-Destornillador/dp/B09W9R8J3S>
|
||||
|
||||
## Serial port adaptor
|
||||
|
||||
In order to debug problems with several components, we need to be able to plug
|
||||
to the serial port of the CPU. As we may deal with different voltages and
|
||||
pinouts, the most versatile option is to just be able to select the voltage and
|
||||
expose a pin interface.
|
||||
|
||||
This one would do:
|
||||
|
||||
<https://www.amazon.es/DSD-TECH-SH-U09C5-convertidor-Soporte/dp/B07WX2DSVB>
|
||||
|
||||
## Storage for raccoon
|
||||
|
||||
Given that we are currently using raccoon for builds too, we would need to
|
||||
increase its current storage. We only have available 270 GB, so we can benefit
|
||||
from another disk. Using 2 TiB would be plenty. This one seems enough:
|
||||
|
||||
- 135,99€ <https://www.amazon.es/Crucial-BX500-pulgadas-interno-CT2000BX500SSD101/dp/B0CCN9QWKT>
|
||||
@@ -18,7 +18,6 @@ in
|
||||
{
|
||||
nixosConfigurations = {
|
||||
hut = mkConf "hut";
|
||||
tent = mkConf "tent";
|
||||
owl1 = mkConf "owl1";
|
||||
owl2 = mkConf "owl2";
|
||||
eudy = mkConf "eudy";
|
||||
|
||||
5
keys.nix
5
keys.nix
@@ -9,12 +9,11 @@ rec {
|
||||
koro = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIImiTFDbxyUYPumvm8C4mEnHfuvtBY1H8undtd6oDd67 koro";
|
||||
bay = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICvGBzpRQKuQYHdlUQeAk6jmdbkrhmdLwTBqf3el7IgU bay";
|
||||
lake2 = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINo66//S1yatpQHE/BuYD/Gfq64TY7ZN5XOGXmNchiO0 lake2";
|
||||
fox = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDwItIk5uOJcQEVPoy/CVGRzfmE1ojrdDcI06FrU4NFT fox";
|
||||
fox = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDa9lId4rB/EKGkkCCVOy0cuId2SYLs+8W8kx0kmpO1y fox";
|
||||
};
|
||||
|
||||
hostGroup = with hosts; rec {
|
||||
untrusted = [ fox ];
|
||||
compute = [ owl1 owl2 ];
|
||||
compute = [ owl1 owl2 fox ];
|
||||
playground = [ eudy koro ];
|
||||
storage = [ bay lake2 ];
|
||||
monitor = [ hut ];
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
{
|
||||
imports = [
|
||||
../common/ssf.nix
|
||||
../common/xeon.nix
|
||||
../module/monitoring.nix
|
||||
];
|
||||
|
||||
|
||||
@@ -8,6 +8,17 @@ in
|
||||
# Enable the OpenSSH daemon.
|
||||
services.openssh.enable = true;
|
||||
|
||||
# Connect to intranet git hosts via proxy
|
||||
programs.ssh.extraConfig = ''
|
||||
Host bscpm02.bsc.es bscpm03.bsc.es bscpm04.bsc.es gitlab-internal.bsc.es alya.gitlab.bsc.es
|
||||
User git
|
||||
ProxyCommand nc -X connect -x hut:23080 %h %p
|
||||
|
||||
# Connect to BSC machines via hut proxy too
|
||||
Host amdlogin1.bsc.es armlogin1.bsc.es hualogin1.bsc.es glogin1.bsc.es glogin2.bsc.es fpgalogin1.bsc.es
|
||||
ProxyCommand nc -X connect -x hut:23080 %h %p
|
||||
'';
|
||||
|
||||
programs.ssh.knownHosts = hostsKeys // {
|
||||
"gitlab-internal.bsc.es".publicKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF9arsAOSRB06hdy71oTvJHG2Mg8zfebADxpvc37lZo3";
|
||||
"bscpm03.bsc.es".publicKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM2NuSUPsEhqz1j5b4Gqd+MWFnRqyqY57+xMvBUqHYUS";
|
||||
|
||||
@@ -20,7 +20,6 @@
|
||||
rarias = {
|
||||
uid = 1880;
|
||||
isNormalUser = true;
|
||||
linger = true;
|
||||
home = "/home/Computational/rarias";
|
||||
description = "Rodrigo Arias";
|
||||
group = "Computational";
|
||||
@@ -40,7 +39,7 @@
|
||||
home = "/home/Computational/arocanon";
|
||||
description = "Aleix Roca";
|
||||
group = "Computational";
|
||||
extraGroups = [ "wheel" "tracing" ];
|
||||
extraGroups = [ "wheel" ];
|
||||
hashedPassword = "$6$hliZiW4tULC/tH7p$pqZarwJkNZ7vS0G5llWQKx08UFG9DxDYgad7jplMD8WkZh5k58i4dfPoWtnEShfjTO6JHiIin05ny5lmSXzGM/";
|
||||
openssh.authorizedKeys.keys = [
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIF3zeB5KSimMBAjvzsp1GCkepVaquVZGPYwRIzyzaCba aleix@bsc"
|
||||
@@ -108,7 +107,7 @@
|
||||
home = "/home/Computational/dbautist";
|
||||
description = "Dylan Bautista Cases";
|
||||
group = "Computational";
|
||||
hosts = [ "hut" "raccoon" ];
|
||||
hosts = [ "hut" ];
|
||||
hashedPassword = "$6$a2lpzMRVkG9nSgIm$12G6.ka0sFX1YimqJkBAjbvhRKZ.Hl090B27pdbnQOW0wzyxVWySWhyDDCILjQELky.HKYl9gqOeVXW49nW7q/";
|
||||
openssh.authorizedKeys.keys = [
|
||||
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAb+EQBoS98zrCwnGKkHKwMLdYABMTqv7q9E0+T0QmkS dbautist@bsc-848818791"
|
||||
@@ -144,7 +143,6 @@
|
||||
|
||||
groups = {
|
||||
Computational = { gid = 564; };
|
||||
tracing = { };
|
||||
};
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,9 +0,0 @@
|
||||
{
|
||||
# Provides the base system for a xeon node in the SSF rack.
|
||||
imports = [
|
||||
./xeon.nix
|
||||
./ssf/fs.nix
|
||||
./ssf/net.nix
|
||||
./ssf/ssh.nix
|
||||
];
|
||||
}
|
||||
@@ -1,8 +0,0 @@
|
||||
{
|
||||
# Connect to intranet git hosts via proxy
|
||||
programs.ssh.extraConfig = ''
|
||||
# Connect to BSC machines via hut proxy too
|
||||
Host amdlogin1.bsc.es armlogin1.bsc.es hualogin1.bsc.es glogin1.bsc.es glogin2.bsc.es fpgalogin1.bsc.es
|
||||
ProxyCommand nc -X connect -x hut:23080 %h %p
|
||||
'';
|
||||
}
|
||||
@@ -1,7 +1,9 @@
|
||||
{
|
||||
# Provides the base system for a xeon node, not necessarily in the SSF rack.
|
||||
# Provides the base system for a xeon node.
|
||||
imports = [
|
||||
./base.nix
|
||||
./xeon/fs.nix
|
||||
./xeon/console.nix
|
||||
./xeon/net.nix
|
||||
];
|
||||
}
|
||||
|
||||
@@ -85,6 +85,10 @@
|
||||
10.0.40.8 eudy xeon08 xeon08-eth0
|
||||
10.0.42.8 eudy-ib xeon08-ib0
|
||||
10.0.40.108 eudy-ipmi xeon08-ipmi0 xeon08-ipmi
|
||||
|
||||
# fox
|
||||
10.0.40.26 fox
|
||||
10.0.40.126 fox-ipmi
|
||||
'';
|
||||
};
|
||||
}
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
{
|
||||
imports = [
|
||||
../common/ssf.nix
|
||||
../common/xeon.nix
|
||||
#(modulesPath + "/installer/netboot/netboot-minimal.nix")
|
||||
|
||||
./kernel/kernel.nix
|
||||
|
||||
@@ -2,9 +2,11 @@
|
||||
|
||||
{
|
||||
imports = [
|
||||
../common/base.nix
|
||||
../common/xeon/console.nix
|
||||
../common/xeon.nix
|
||||
../module/ceph.nix
|
||||
../module/emulation.nix
|
||||
../module/slurm-client.nix
|
||||
../module/slurm-firewall.nix
|
||||
];
|
||||
|
||||
# Select the this using the ID to avoid mismatches
|
||||
@@ -20,30 +22,11 @@
|
||||
hardware.cpu.intel.updateMicrocode = lib.mkForce false;
|
||||
|
||||
networking = {
|
||||
defaultGateway = "147.83.30.130";
|
||||
nameservers = [ "8.8.8.8" ];
|
||||
hostName = "fox";
|
||||
interfaces.enp1s0f0np0.ipv4.addresses = [
|
||||
{
|
||||
# UPC network
|
||||
# Public IP configuration:
|
||||
# - Hostname: fox.ac.upc.edu
|
||||
# - IP: 147.83.30.141
|
||||
# - Gateway: 147.83.30.130
|
||||
# - NetMask: 255.255.255.192
|
||||
# Private IP configuration for BMC:
|
||||
# - Hostname: fox-ipmi.ac.upc.edu
|
||||
# - IP: 147.83.35.27
|
||||
# - Gateway: 147.83.35.2
|
||||
# - NetMask: 255.255.255.0
|
||||
address = "147.83.30.141";
|
||||
prefixLength = 26; # 255.255.255.192
|
||||
}
|
||||
];
|
||||
extraHosts = ''
|
||||
147.83.30.141 fox.ac.upc.edu
|
||||
147.83.35.27 fox-ipmi.ac.upc.edu
|
||||
'';
|
||||
interfaces.enp1s0f0np0.ipv4.addresses = [ {
|
||||
address = "10.0.40.26";
|
||||
prefixLength = 24;
|
||||
} ];
|
||||
};
|
||||
|
||||
# Configure Nvidia driver to use with CUDA
|
||||
@@ -73,4 +56,20 @@
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
serviceConfig.ExecStart = script;
|
||||
};
|
||||
|
||||
# Only allow SSH connections from users who have a SLURM allocation
|
||||
# See: https://slurm.schedmd.com/pam_slurm_adopt.html
|
||||
security.pam.services.sshd.rules.account.slurm = {
|
||||
control = "required";
|
||||
enable = true;
|
||||
modulePath = "${pkgs.slurm}/lib/security/pam_slurm_adopt.so";
|
||||
args = [ "log_level=debug5" ];
|
||||
order = 999999; # Make it last one
|
||||
};
|
||||
|
||||
# Disable systemd session (pam_systemd.so) as it will conflict with the
|
||||
# pam_slurm_adopt.so module. What happens is that the shell is first adopted
|
||||
# into the slurmstepd task and then into the systemd session, which is not
|
||||
# what we want, otherwise it will linger even if all jobs are gone.
|
||||
security.pam.services.sshd.startSession = lib.mkForce false;
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
{
|
||||
imports = [
|
||||
../common/ssf.nix
|
||||
../common/xeon.nix
|
||||
|
||||
../module/ceph.nix
|
||||
../module/debuginfod.nix
|
||||
|
||||
@@ -3,10 +3,7 @@
|
||||
{
|
||||
imports = [
|
||||
../module/slurm-exporter.nix
|
||||
../module/meteocat-exporter.nix
|
||||
../module/upc-qaire-exporter.nix
|
||||
./gpfs-probe.nix
|
||||
./nix-daemon-exporter.nix
|
||||
];
|
||||
|
||||
age.secrets.grafanaJungleRobotPassword = {
|
||||
@@ -111,9 +108,6 @@
|
||||
"127.0.0.1:${toString config.services.prometheus.exporters.smartctl.port}"
|
||||
"127.0.0.1:9341" # Slurm exporter
|
||||
"127.0.0.1:9966" # GPFS custom exporter
|
||||
"127.0.0.1:9999" # Nix-daemon custom exporter
|
||||
"127.0.0.1:9929" # Meteocat custom exporter
|
||||
"127.0.0.1:9928" # UPC Qaire custom exporter
|
||||
"127.0.0.1:${toString config.services.prometheus.exporters.blackbox.port}"
|
||||
];
|
||||
}];
|
||||
@@ -169,9 +163,6 @@
|
||||
"8.8.8.8"
|
||||
"ssfhead"
|
||||
"anella-bsc.cesca.cat"
|
||||
"upc-anella.cesca.cat"
|
||||
"fox.ac.upc.edu"
|
||||
"arenys5.ac.upc.edu"
|
||||
];
|
||||
}];
|
||||
relabel_configs = [
|
||||
@@ -267,6 +258,17 @@
|
||||
}
|
||||
];
|
||||
}
|
||||
{
|
||||
job_name = "ipmi-fox";
|
||||
metrics_path = "/ipmi";
|
||||
static_configs = [
|
||||
{ targets = [ "127.0.0.1:9290" ]; }
|
||||
];
|
||||
params = {
|
||||
target = [ "fox-ipmi" ];
|
||||
module = [ "fox" ];
|
||||
};
|
||||
}
|
||||
];
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,26 +0,0 @@
|
||||
#!/bin/sh
|
||||
|
||||
# Locate nix daemon pid
|
||||
nd=$(pgrep -o nix-daemon)
|
||||
|
||||
# Locate children of nix-daemon
|
||||
pids1=$(tr ' ' '\n' < "/proc/$nd/task/$nd/children")
|
||||
|
||||
# For each children, locate 2nd level children
|
||||
pids2=$(echo "$pids1" | xargs -I @ /bin/sh -c 'cat /proc/@/task/*/children' | tr ' ' '\n')
|
||||
|
||||
cat <<EOF
|
||||
HTTP/1.1 200 OK
|
||||
Content-Type: text/plain; version=0.0.4; charset=utf-8; escaping=values
|
||||
|
||||
# HELP nix_daemon_build Nix daemon derivation build state.
|
||||
# TYPE nix_daemon_build gauge
|
||||
EOF
|
||||
|
||||
for pid in $pids2; do
|
||||
name=$(cat /proc/$pid/environ 2>/dev/null | tr '\0' '\n' | rg "^name=(.+)" - --replace '$1' | tr -dc ' [:alnum:]_\-\.')
|
||||
user=$(ps -o uname= -p "$pid")
|
||||
if [ -n "$name" -a -n "$user" ]; then
|
||||
printf 'nix_daemon_build{user="%s",name="%s"} 1\n' "$user" "$name"
|
||||
fi
|
||||
done
|
||||
@@ -1,23 +0,0 @@
|
||||
{ pkgs, config, lib, ... }:
|
||||
let
|
||||
script = pkgs.runCommand "nix-daemon-exporter.sh" { }
|
||||
''
|
||||
cp ${./nix-daemon-builds.sh} $out;
|
||||
chmod +x $out
|
||||
''
|
||||
;
|
||||
in
|
||||
{
|
||||
systemd.services.nix-daemon-exporter = {
|
||||
description = "Daemon to export nix-daemon metrics";
|
||||
path = [ pkgs.procps pkgs.ripgrep ];
|
||||
wantedBy = [ "default.target" ];
|
||||
serviceConfig = {
|
||||
Type = "simple";
|
||||
ExecStart = "${pkgs.socat}/bin/socat TCP4-LISTEN:9999,fork EXEC:${script}";
|
||||
# Needed root to read the environment, potentially unsafe
|
||||
User = "root";
|
||||
Group = "root";
|
||||
};
|
||||
};
|
||||
}
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
{
|
||||
imports = [
|
||||
../common/ssf.nix
|
||||
../common/xeon.nix
|
||||
#(modulesPath + "/installer/netboot/netboot-minimal.nix")
|
||||
|
||||
../eudy/cpufreq.nix
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
{
|
||||
imports = [
|
||||
../common/ssf.nix
|
||||
../common/xeon.nix
|
||||
../module/monitoring.nix
|
||||
];
|
||||
|
||||
|
||||
70
m/map.nix
70
m/map.nix
@@ -1,70 +0,0 @@
|
||||
{
|
||||
# In physical order from top to bottom (see note below)
|
||||
ssf = {
|
||||
# Switches for Ethernet and OmniPath
|
||||
switch-C6-S1A-05 = { pos=42; size=1; model="Dell S3048-ON"; };
|
||||
switch-opa = { pos=41; size=1; };
|
||||
|
||||
# SSF login
|
||||
ssfhead = { pos=39; size=2; label="SSFHEAD"; board="R2208WTTYSR"; contact="operations@bsc.es"; };
|
||||
|
||||
# Storage
|
||||
bay = { pos=38; size=1; label="MDS01"; board="S2600WT2R"; sn="BQWL64850303"; contact="rodrigo.arias@bsc.es"; };
|
||||
lake1 = { pos=37; size=1; label="OSS01"; board="S2600WT2R"; sn="BQWL64850234"; contact="rodrigo.arias@bsc.es"; };
|
||||
lake2 = { pos=36; size=1; label="OSS02"; board="S2600WT2R"; sn="BQWL64850266"; contact="rodrigo.arias@bsc.es"; };
|
||||
|
||||
# Compute xeon
|
||||
owl1 = { pos=35; size=1; label="SSF-XEON01"; board="S2600WTTR"; sn="BQWL64954172"; contact="rodrigo.arias@bsc.es"; };
|
||||
owl2 = { pos=34; size=1; label="SSF-XEON02"; board="S2600WTTR"; sn="BQWL64756560"; contact="rodrigo.arias@bsc.es"; };
|
||||
xeon03 = { pos=33; size=1; label="SSF-XEON03"; board="S2600WTTR"; sn="BQWL64750826"; contact="rodrigo.arias@bsc.es"; };
|
||||
# Slot 34 empty
|
||||
koro = { pos=31; size=1; label="SSF-XEON05"; board="S2600WTTR"; sn="BQWL64954293"; contact="rodrigo.arias@bsc.es"; };
|
||||
xeon06 = { pos=30; size=1; label="SSF-XEON06"; board="S2600WTTR"; sn="BQWL64750846"; contact="antoni.navarro@bsc.es"; };
|
||||
hut = { pos=29; size=1; label="SSF-XEON07"; board="S2600WTTR"; sn="BQWL64751184"; contact="rodrigo.arias@bsc.es"; };
|
||||
eudy = { pos=28; size=1; label="SSF-XEON08"; board="S2600WTTR"; sn="BQWL64756586"; contact="aleix.rocanonell@bsc.es"; };
|
||||
|
||||
# 16 KNL nodes, 4 per chassis
|
||||
knl01_04 = { pos=26; size=2; label="KNL01..KNL04"; board="HNS7200APX"; };
|
||||
knl05_08 = { pos=24; size=2; label="KNL05..KNL18"; board="HNS7200APX"; };
|
||||
knl09_12 = { pos=22; size=2; label="KNL09..KNL12"; board="HNS7200APX"; };
|
||||
knl13_16 = { pos=20; size=2; label="KNL13..KNL16"; board="HNS7200APX"; };
|
||||
|
||||
# Slot 19 empty
|
||||
|
||||
# EPI (hw team, guessed order)
|
||||
epi01 = { pos=18; size=1; contact="joan.cabre@bsc.es"; };
|
||||
epi02 = { pos=17; size=1; contact="joan.cabre@bsc.es"; };
|
||||
epi03 = { pos=16; size=1; contact="joan.cabre@bsc.es"; };
|
||||
anon = { pos=14; size=2; }; # Unlabeled machine. Operative
|
||||
|
||||
# These are old and decommissioned (off)
|
||||
power8 = { pos=12; size=2; label="BSCPOWER8N3"; decommissioned=true; };
|
||||
powern1 = { pos=8; size=4; label="BSCPOWERN1"; decommissioned=true; };
|
||||
gustafson = { pos=7; size=1; label="gustafson"; decommissioned=true; };
|
||||
odap01 = { pos=3; size=4; label="ODAP01"; decommissioned=true; };
|
||||
amhdal = { pos=2; size=1; label="AMHDAL"; decommissioned=true; }; # sic
|
||||
moore = { pos=1; size=1; label="moore (earth)"; decommissioned=true; };
|
||||
};
|
||||
|
||||
bsc2218 = {
|
||||
raccoon = { board="W2600CR"; sn="QSIP22500829"; contact="rodrigo.arias@bsc.es"; };
|
||||
tent = { label="SSF-XEON04"; board="S2600WTTR"; sn="BQWL64751229"; contact="rodrigo.arias@bsc.es"; };
|
||||
};
|
||||
|
||||
upc = {
|
||||
fox = { board="H13DSG-O-CPU"; sn="UM24CS600392"; prod="AS-4125GS-TNRT"; prod_sn="E508839X5103339"; contact="rodrigo.arias@bsc.es"; };
|
||||
};
|
||||
|
||||
# NOTE: Position is specified in "U" units (44.45 mm) and starts at 1 from the
|
||||
# bottom. Example:
|
||||
#
|
||||
# | ... | - [pos+size] <--- Label in chassis
|
||||
# +--------+
|
||||
# | node | - [pos+1]
|
||||
# | 2U | - [pos]
|
||||
# +------- +
|
||||
# | ... | - [pos-1]
|
||||
#
|
||||
# NOTE: The board and sn refers to the FRU information (Board Product and
|
||||
# Board Serial) via `ipmitool fru print 0`.
|
||||
}
|
||||
@@ -1,17 +0,0 @@
|
||||
{ config, lib, pkgs, ... }:
|
||||
|
||||
with lib;
|
||||
|
||||
{
|
||||
systemd.services."prometheus-meteocat-exporter" = {
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
after = [ "network.target" ];
|
||||
serviceConfig = {
|
||||
Restart = mkDefault "always";
|
||||
PrivateTmp = mkDefault true;
|
||||
WorkingDirectory = mkDefault "/tmp";
|
||||
DynamicUser = mkDefault true;
|
||||
ExecStart = "${pkgs.meteocat-exporter}/bin/meteocat-exporter";
|
||||
};
|
||||
};
|
||||
}
|
||||
@@ -43,11 +43,13 @@ in {
|
||||
clusterName = "jungle";
|
||||
nodeName = [
|
||||
"owl[1,2] Sockets=2 CoresPerSocket=14 ThreadsPerCore=2 Feature=owl"
|
||||
"fox Sockets=2 CoresPerSocket=96 ThreadsPerCore=1 Feature=fox"
|
||||
"hut Sockets=2 CoresPerSocket=14 ThreadsPerCore=2"
|
||||
];
|
||||
|
||||
partitionName = [
|
||||
"owl Nodes=owl[1-2] Default=YES DefaultTime=01:00:00 MaxTime=INFINITE State=UP"
|
||||
"fox Nodes=fox Default=NO DefaultTime=01:00:00 MaxTime=INFINITE State=UP"
|
||||
];
|
||||
|
||||
# See slurm.conf(5) for more details about these options.
|
||||
@@ -75,7 +77,7 @@ in {
|
||||
SuspendTimeout=60
|
||||
ResumeProgram=${resumeProgram}
|
||||
ResumeTimeout=300
|
||||
SuspendExcNodes=hut
|
||||
SuspendExcNodes=hut,fox
|
||||
|
||||
# Turn the nodes off after 1 hour of inactivity
|
||||
SuspendTime=3600
|
||||
|
||||
@@ -1,9 +0,0 @@
|
||||
{
|
||||
programs.ssh.extraConfig = ''
|
||||
Host ssfhead
|
||||
HostName ssflogin.bsc.es
|
||||
Host hut
|
||||
ProxyJump ssfhead
|
||||
HostName xeon07
|
||||
'';
|
||||
}
|
||||
@@ -1,17 +0,0 @@
|
||||
{ config, lib, pkgs, ... }:
|
||||
|
||||
with lib;
|
||||
|
||||
{
|
||||
systemd.services."prometheus-upc-qaire-exporter" = {
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
after = [ "network.target" ];
|
||||
serviceConfig = {
|
||||
Restart = mkDefault "always";
|
||||
PrivateTmp = mkDefault true;
|
||||
WorkingDirectory = mkDefault "/tmp";
|
||||
DynamicUser = mkDefault true;
|
||||
ExecStart = "${pkgs.upc-qaire-exporter}/bin/upc-qaire-exporter";
|
||||
};
|
||||
};
|
||||
}
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
{
|
||||
imports = [
|
||||
../common/ssf.nix
|
||||
../common/xeon.nix
|
||||
../module/ceph.nix
|
||||
../module/emulation.nix
|
||||
../module/slurm-client.nix
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
{
|
||||
imports = [
|
||||
../common/ssf.nix
|
||||
../common/xeon.nix
|
||||
../module/ceph.nix
|
||||
../module/emulation.nix
|
||||
../module/slurm-client.nix
|
||||
|
||||
@@ -3,10 +3,6 @@
|
||||
{
|
||||
imports = [
|
||||
../common/base.nix
|
||||
../module/emulation.nix
|
||||
../module/debuginfod.nix
|
||||
../module/ssh-hut-extern.nix
|
||||
../eudy/kernel/perf.nix
|
||||
];
|
||||
|
||||
# Don't install Grub on the disk yet
|
||||
@@ -27,18 +23,6 @@
|
||||
address = "84.88.51.152";
|
||||
prefixLength = 25;
|
||||
} ];
|
||||
interfaces.enp5s0f1.ipv4.addresses = [ {
|
||||
address = "10.0.44.1";
|
||||
prefixLength = 24;
|
||||
} ];
|
||||
nat = {
|
||||
enable = true;
|
||||
internalInterfaces = [ "enp5s0f1" ];
|
||||
externalInterface = "eno0";
|
||||
};
|
||||
hosts = {
|
||||
"10.0.44.4" = [ "tent" ];
|
||||
};
|
||||
};
|
||||
|
||||
nix.settings = {
|
||||
@@ -46,9 +30,6 @@
|
||||
trusted-public-keys = [ "jungle.bsc.es:pEc7MlAT0HEwLQYPtpkPLwRsGf80ZI26aj29zMw/HH0=" ];
|
||||
};
|
||||
|
||||
# Enable performance governor
|
||||
powerManagement.cpuFreqGovernor = "performance";
|
||||
|
||||
# Configure Nvidia driver to use with CUDA
|
||||
hardware.nvidia.package = config.boot.kernelPackages.nvidiaPackages.production;
|
||||
hardware.graphics.enable = true;
|
||||
@@ -56,18 +37,6 @@
|
||||
nixpkgs.config.nvidia.acceptLicense = true;
|
||||
services.xserver.videoDrivers = [ "nvidia" ];
|
||||
|
||||
# Disable garbage collection for now
|
||||
nix.gc.automatic = lib.mkForce false;
|
||||
|
||||
services.openssh.settings.X11Forwarding = true;
|
||||
|
||||
services.prometheus.exporters.node = {
|
||||
enable = true;
|
||||
enabledCollectors = [ "systemd" ];
|
||||
port = 9002;
|
||||
listenAddress = "127.0.0.1";
|
||||
};
|
||||
|
||||
users.motd = ''
|
||||
⠀⠀⠀⠀⠀⠀⠀⣀⣀⣄⣠⣀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
|
||||
⠀⠀⠀⠀⠀⠀⢰⠇⡀⠀⠙⠻⡿⣦⣀⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡀⠀⠀⠀⠀
|
||||
|
||||
@@ -1,45 +0,0 @@
|
||||
{ config, pkgs, ... }:
|
||||
|
||||
{
|
||||
imports = [
|
||||
../common/xeon.nix
|
||||
../module/emulation.nix
|
||||
../module/debuginfod.nix
|
||||
../module/ssh-hut-extern.nix
|
||||
];
|
||||
|
||||
# Select the this using the ID to avoid mismatches
|
||||
boot.loader.grub.device = "/dev/disk/by-id/wwn-0x55cd2e414d537675";
|
||||
|
||||
networking = {
|
||||
hostName = "tent";
|
||||
interfaces.eno1.ipv4.addresses = [
|
||||
{
|
||||
address = "10.0.44.4";
|
||||
prefixLength = 24;
|
||||
}
|
||||
];
|
||||
|
||||
# Only BSC DNSs seem to be reachable from the office VLAN
|
||||
nameservers = [ "84.88.52.35" "84.88.52.36" ];
|
||||
defaultGateway = "10.0.44.1";
|
||||
};
|
||||
|
||||
nix.settings = {
|
||||
substituters = [ "https://jungle.bsc.es/cache" ];
|
||||
trusted-public-keys = [ "jungle.bsc.es:pEc7MlAT0HEwLQYPtpkPLwRsGf80ZI26aj29zMw/HH0=" ];
|
||||
};
|
||||
|
||||
services.prometheus.exporters.node = {
|
||||
enable = true;
|
||||
enabledCollectors = [ "systemd" ];
|
||||
port = 9002;
|
||||
listenAddress = "127.0.0.1";
|
||||
};
|
||||
|
||||
programs.ssh.extraConfig = ''
|
||||
Host hut
|
||||
ProxyJump ssfhead
|
||||
HostName xeon07
|
||||
'';
|
||||
}
|
||||
@@ -1,25 +0,0 @@
|
||||
{ python3Packages, lib }:
|
||||
|
||||
python3Packages.buildPythonApplication rec {
|
||||
pname = "meteocat-exporter";
|
||||
version = "1.0";
|
||||
|
||||
src = ./.;
|
||||
|
||||
doCheck = false;
|
||||
|
||||
build-system = with python3Packages; [
|
||||
setuptools
|
||||
];
|
||||
|
||||
dependencies = with python3Packages; [
|
||||
beautifulsoup4
|
||||
lxml
|
||||
prometheus-client
|
||||
];
|
||||
|
||||
meta = with lib; {
|
||||
description = "MeteoCat Prometheus Exporter";
|
||||
platforms = platforms.linux;
|
||||
};
|
||||
}
|
||||
@@ -1,54 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import time
|
||||
from prometheus_client import start_http_server, Gauge
|
||||
from bs4 import BeautifulSoup
|
||||
from urllib import request
|
||||
|
||||
# Configuration -------------------------------------------
|
||||
meteo_station = "X8" # Barcelona - Zona Universitària
|
||||
listening_port = 9929
|
||||
update_period = 60 * 5 # Each 5 min
|
||||
# ---------------------------------------------------------
|
||||
|
||||
metric_tmin = Gauge('meteocat_temp_min', 'Min temperature')
|
||||
metric_tmax = Gauge('meteocat_temp_max', 'Max temperature')
|
||||
metric_tavg = Gauge('meteocat_temp_avg', 'Average temperature')
|
||||
metric_srad = Gauge('meteocat_solar_radiation', 'Solar radiation')
|
||||
|
||||
def update(st):
|
||||
url = 'https://www.meteo.cat/observacions/xema/dades?codi=' + st
|
||||
response = request.urlopen(url)
|
||||
data = response.read()
|
||||
soup = BeautifulSoup(data, 'lxml')
|
||||
table = soup.find("table", {"class" : "tblperiode"})
|
||||
rows = table.find_all('tr')
|
||||
row = rows[-1] # Take the last row
|
||||
row_data = []
|
||||
header = row.find('th')
|
||||
header_text = header.text.strip()
|
||||
row_data.append(header_text)
|
||||
for col in row.find_all('td'):
|
||||
row_data.append(col.text)
|
||||
try:
|
||||
# Sometimes it will return '(s/d)' and fail to parse
|
||||
metric_tavg.set(float(row_data[1]))
|
||||
metric_tmax.set(float(row_data[2]))
|
||||
metric_tmin.set(float(row_data[3]))
|
||||
metric_srad.set(float(row_data[10]))
|
||||
#print("ok: temp_avg={}".format(float(row_data[1])))
|
||||
except:
|
||||
print("cannot parse row: {}".format(row))
|
||||
metric_tavg.set(float("nan"))
|
||||
metric_tmax.set(float("nan"))
|
||||
metric_tmin.set(float("nan"))
|
||||
metric_srad.set(float("nan"))
|
||||
|
||||
if __name__ == '__main__':
|
||||
start_http_server(port=listening_port, addr="localhost")
|
||||
while True:
|
||||
try:
|
||||
update(meteo_station)
|
||||
except:
|
||||
print("update failed")
|
||||
time.sleep(update_period)
|
||||
@@ -1,11 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
|
||||
from setuptools import setup, find_packages
|
||||
|
||||
setup(name='meteocat-exporter',
|
||||
version='1.0',
|
||||
# Modules to import from other scripts:
|
||||
packages=find_packages(),
|
||||
# Executables
|
||||
scripts=["meteocat-exporter"],
|
||||
)
|
||||
@@ -54,6 +54,4 @@ final: prev:
|
||||
});
|
||||
|
||||
prometheus-slurm-exporter = prev.callPackage ./slurm-exporter.nix { };
|
||||
meteocat-exporter = prev.callPackage ./meteocat-exporter/default.nix { };
|
||||
upc-qaire-exporter = prev.callPackage ./upc-qaire-exporter/default.nix { };
|
||||
}
|
||||
|
||||
@@ -1,24 +0,0 @@
|
||||
{ python3Packages, lib }:
|
||||
|
||||
python3Packages.buildPythonApplication rec {
|
||||
pname = "upc-qaire-exporter";
|
||||
version = "1.0";
|
||||
|
||||
src = ./.;
|
||||
|
||||
doCheck = false;
|
||||
|
||||
build-system = with python3Packages; [
|
||||
setuptools
|
||||
];
|
||||
|
||||
dependencies = with python3Packages; [
|
||||
prometheus-client
|
||||
requests
|
||||
];
|
||||
|
||||
meta = with lib; {
|
||||
description = "UPC Qaire Prometheus Exporter";
|
||||
platforms = platforms.linux;
|
||||
};
|
||||
}
|
||||
@@ -1,11 +0,0 @@
|
||||
#!/usr/bin/env python
|
||||
|
||||
from setuptools import setup, find_packages
|
||||
|
||||
setup(name='upc-qaire-exporter',
|
||||
version='1.0',
|
||||
# Modules to import from other scripts:
|
||||
packages=find_packages(),
|
||||
# Executables
|
||||
scripts=["upc-qaire-exporter"],
|
||||
)
|
||||
@@ -1,74 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
import time
|
||||
from prometheus_client import start_http_server, Gauge
|
||||
import requests, json
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
# Configuration -------------------------------------------
|
||||
listening_port = 9928
|
||||
update_period = 60 * 5 # Each 5 min
|
||||
# ---------------------------------------------------------
|
||||
|
||||
metric_temp = Gauge('upc_c6_s302_temp', 'UPC C6 S302 temperature sensor')
|
||||
|
||||
def genparams():
|
||||
d = {}
|
||||
d['topic'] = 'TEMPERATURE'
|
||||
d['shift_dates_to'] = ''
|
||||
d['datapoints'] = 301
|
||||
d['devicesAndColors'] = '1148418@@@#40ACB6'
|
||||
|
||||
now = datetime.now()
|
||||
|
||||
d['fromDate'] = now.strftime('%d/%m/%Y')
|
||||
d['toDate'] = now.strftime('%d/%m/%Y')
|
||||
d['serviceFrequency'] = 'NONE'
|
||||
|
||||
# WTF!
|
||||
for i in range(7):
|
||||
for j in range(48):
|
||||
key = 'week.days[{}].hours[{}].value'.format(i, j)
|
||||
d[key] = 'OPEN'
|
||||
|
||||
return d
|
||||
|
||||
def measure():
|
||||
# First we need to load session
|
||||
s = requests.Session()
|
||||
r = s.get("https://upc.edu/sirena")
|
||||
if r.status_code != 200:
|
||||
print("bad HTTP status code on new session: {}".format(r.status_code))
|
||||
return
|
||||
|
||||
if s.cookies.get("JSESSIONID") is None:
|
||||
print("cannot get JSESSIONID")
|
||||
return
|
||||
|
||||
# Now we can pull the data
|
||||
url = "https://upcsirena.app.dexma.com/l_12535/analysis/by_datapoints/data.json"
|
||||
r = s.post(url, data=genparams())
|
||||
|
||||
if r.status_code != 200:
|
||||
print("bad HTTP status code on data: {}".format(r.status_code))
|
||||
return
|
||||
|
||||
#print(r.text)
|
||||
j = json.loads(r.content)
|
||||
|
||||
# Just take the last one
|
||||
last = j['data']['chartElementList'][-1]
|
||||
temp = last['values']['1148418-Temperatura']
|
||||
|
||||
return temp
|
||||
|
||||
if __name__ == '__main__':
|
||||
start_http_server(port=listening_port, addr="localhost")
|
||||
while True:
|
||||
try:
|
||||
metric_temp.set(measure())
|
||||
except:
|
||||
print("measure failed")
|
||||
metric_temp.set(float("nan"))
|
||||
|
||||
time.sleep(update_period)
|
||||
Binary file not shown.
Binary file not shown.
@@ -1,10 +1,11 @@
|
||||
age-encryption.org/v1
|
||||
-> ssh-ed25519 HY2yRg XPOFoZqY+AnKC77jrgNqAm1ADphurfuhO4NRrfiuUDc
|
||||
iCfMMpGHyaYHGy6ci8sqjUtcPeteLlyvLGEF79VPOEc
|
||||
-> ssh-ed25519 CAWG4Q 6OsGrnM+/c5lTN81Rvp166K+ygmSIFeSYzXxYg25KGE
|
||||
Av1zTw2zK4Gufzti9kQaye7C362GCiDRRHzCqBLR33g
|
||||
-> ssh-ed25519 MSF3dg 8CHqJ7mEDvjvqbmF+eE6Em1Wi6eHAzEUpiExC1gm7S0
|
||||
bdwzYHw3RAbdHq+RsiFUP++sQ586VUlSnAzAOhiQUjI
|
||||
--- gA5XSUfjUBol938sC5DbUf8PvQUIr2pNkS2nL95OF9c
|
||||
<EFBFBD><EFBFBD><EFBFBD>Ea1G7<47><37>ݩ[R<><52>\{~$Go<47>cQ<63>wKP&<26><><EFBFBD>w<EFBFBD><77><EFBFBD>6]
|
||||
<EFBFBD><EFBFBD>ѣ<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>^z<7A>̄ 1k<31><6B><03><><EFBFBD><EFBFBD><EFBFBD>Y<EFBFBD><59>2<15>p<>2<EFBFBD><32>K<EFBFBD><4B><EFBFBD><EFBFBD>nok<>/X<><58>pt''<27><>$0co=<3D><EFBFBD>
|
||||
-> ssh-ed25519 HY2yRg WSdjyQPzBJ4JbzQpGeq1AAYpWKoXmLI1ZtmNmM5QOzs
|
||||
qGDlDT31DQF1DdHen0+5+52DdsQlabJdA2pOB5O1I6g
|
||||
-> ssh-ed25519 CAWG4Q wioWMDxQjN+d4JdIbCwZg0DLQu1OH2mV6gukRprjuAs
|
||||
670fE61hidOEh20hHiQAhP0+CjDF0WMBNzgwkGT8Yqg
|
||||
-> ssh-ed25519 MSF3dg DN19uvAEtqq4708P6HpuX9i/o/qAvHX6dj69dCF2H1o
|
||||
4Lu9GnjiFLMeXJ2C7aVPJsCHCQVlhylNWJi896Av92s
|
||||
--- 7cKBwOYNOUZ2h3/kAY09aSMASZSxX7hZIT4kvlIiT6w
|
||||
<EFBFBD>6<EFBFBD><02><><EFBFBD><06>fQF5=<3D>bX+<2B>v e`<60>7/<2F><05>A~P<><50>Ѧ7<15><>
|
||||
<EFBFBD><EFBFBD>A<EFBFBD>)<29>h<05><>=oZ<6F>$<24> ^<5E>V0<56>/܅<>r
|
||||
k<EFBFBD>u<EFBFBD>bĶ:R<><52>>^g<><67><06>ik_* % <0B>a7<61>KG<4B><47><EFBFBD><EFBFBD><EFBFBD><EFBFBD>&<26>PI<50><49>n
|
||||
@@ -1,9 +1,10 @@
|
||||
age-encryption.org/v1
|
||||
-> ssh-ed25519 HY2yRg pXNTB/ailRwSEJG1pXvrzzpz5HqkDZdWVWnOH7JGeQ4
|
||||
NzA+2fxfkNRy/u+Zq96A02K1Vxy0ETYZjMkDVTKyCY8
|
||||
-> ssh-ed25519 CAWG4Q 7CLJWn+EAxoWDduXaOSrHaBFHQ4GIpYP/62FFTj3ZTI
|
||||
vSYV1pQg2qI2ngCzM0nCZAnqdz1tbT4hM5m+/TyGU2c
|
||||
-> ssh-ed25519 MSF3dg Akmp4NcZcDuaYHta/Vej6zulNSrAOCd5lmSV+OiBGC4
|
||||
qTxqVzTyywur+GjtUQdbaIUdH1fqCqPe6qPf8iHRa4w
|
||||
--- uCKNqD1TmZZThOzlpsecBKx/k+noIWhCVMr/pzNwBr8
|
||||
r'<27>Ƌs4<15>˺<EFBFBD>Aĥ<41>P <05>L<EFBFBD><4C>7` <09><02><>) H<><48>-<2D>0<EFBFBD>AH<41>5<16><> <1F><>L<EFBFBD>Qe<51><65>H2b<32> <1D>B<42>CJG<4A>"-S<> <1C>\<5C><> <0C>H<n<>V
|
||||
-> ssh-ed25519 HY2yRg GdmdkW+BqqwBgu30b846jv3J7jtCM+a3rgOERuA050A
|
||||
FeGqM75jG9egesR+yyVKHm0/M+uBBp5Hclg4+qN0BR8
|
||||
-> ssh-ed25519 CAWG4Q a0wTWHgulQUYDAMZmXf3dOf6PdYgCqNtSylzWVVRNVM
|
||||
Bx+WSYaiY4ZwlSZJo2a1XPMQmbKOU7F0tKAqVRLBOPo
|
||||
-> ssh-ed25519 MSF3dg KccUvZZUbxbCrRWUWrX8KcHF6vQ5FV/BqUqI59G7dj4
|
||||
CFr7GXpZ9rPgy7HBfOyiYF9FnZUw6KcZwq9f7/0KaU8
|
||||
--- E0Rp6RR/8+o0jvB1lRdhnlabxvI6uu/IgL2ZpPXzTc8
|
||||
<EFBFBD><13>#<23><>H<EFBFBD>$<24>F;<3B><EFBFBD><7F> %<25><>6<><02>2<EFBFBD><32>rfX<66>\Dn <20>ш<EFBFBD>ȉ<EFBFBD>x<EFBFBD><78>><3E><>&;<3B>c<EFBFBD>U<EFBFBD>I=<3D><>M<EFBFBD><4D><EFBFBD>?T<><54>Ǹ<EFBFBD><16>"px<70>ӭ\s<><73><EFBFBD>bF<62><46><EFBFBD><EFBFBD> WD<>{<7B>
|
||||
AW>?U<><55><EFBFBD><17><>HԳ
|
||||
@@ -1,9 +1,9 @@
|
||||
age-encryption.org/v1
|
||||
-> ssh-ed25519 HY2yRg s6iI9f25xulF4KXt+XY07kXXPKxXo7f2Ql/OTHN55Hk
|
||||
WO4Fd2H9c+HL3+XhUF3BmEZVILlcchGxSrSmL2OEdGw
|
||||
-> ssh-ed25519 CAWG4Q TBkdpx8k8K1NvW3wcvaF7omKFwEJ2DxWJp3tIOTjwCA
|
||||
LcYgWRix23AQnw0OQ7f8+8S3J84CHUElX1vKZSETiLE
|
||||
-> ssh-ed25519 MSF3dg WzrF8kjTP7BXXDjmUp7kPCKguthAW12RPo6Vy2RMmh4
|
||||
8C3mT9ktudCTANDxhyNszUkbeDG6X4wOJdx825++dYM
|
||||
--- /w3YQ2UeTi67H1JR0GsdPz2KoLN2Y7BIZfFY+//AWjY
|
||||
<EFBFBD>ӣ-`P<>@<40><> ބ<><DE84><EFBFBD>)9<>9 l<EFBFBD><EFBFBD> <EFBFBD><EFBFBD>Zf<EFBFBD><EFBFBD>V?I><>w鉐<77> z40 <20>2{i@<40>Z<EFBFBD><5A>x<EFBFBD><78>AHn<48>%<25><> <0C>ʤ<EFBFBD>/W<EFBFBD><EFBFBD>Ĕ<EFBFBD>l<EFBFBD><EFBFBD><EFBFBD><EFBFBD>}<7D>&<08>얶<EFBFBD>(<EFBFBD><EFBFBD><EFBFBD>K<EFBFBD><EFBFBD><EFBFBD>S<EFBFBD><EFBFBD>o<EFBFBD>z<EFBFBD>= <1F>d
|
||||
-> ssh-ed25519 HY2yRg xWRxJGWSzA5aplRYCYLB6aBwrUrQQJ2MtDYaD75V5nI
|
||||
J07XF3NQiaYKKKNRcNWi9MloJD2wXHd+2K7bo6lF+QU
|
||||
-> ssh-ed25519 CAWG4Q jNWymbyCczcm8RcaIEbFQBlOMALsuxTl4+pLUi0aR20
|
||||
z5NixlrRD+Y7Z/aFPs6hiDW4/lp8CBQCeJYpbuG9yYM
|
||||
-> ssh-ed25519 MSF3dg QsUQloEKN3k1G49FQnNR/Do6ILgGpjFcw3zu5kk1Ako
|
||||
IHwyFWUEWqCStNcFprnpBa8L5J6zKIsn+7HcgGRv3sM
|
||||
--- oUia0fsL6opeYWACyXtHAu/Ld+bUIt/7S1VszYTvwgU
|
||||
<EFBFBD><EFBFBD>V<EFBFBD><16>*<2A>t<1B>2-<2D>7<><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>h<EFBFBD>&<26><>͢_!տ+<2B><><EFBFBD><EFBFBD>(<28><0F><11>n<EFBFBD><6E> <09><>(<28><19><EFBFBD>/}<EFBFBD><EFBFBD><EFBFBD><EFBFBD>C<EFBFBD>Nͷ|<04>N<>u<EFBFBD>5<EFBFBD>ù勚K<E58B9A><EFBFBD>l<EFBFBD>"<EFBFBD><EFBFBD>klOX<EFBFBD>y<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>A<EFBFBD><EFBFBD>e<><65>$
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Reference in New Issue
Block a user