16 Commits

Author SHA1 Message Date
777704a9ce Monitor https://pm.bsc.es/gitlab/ too
The GitLab instance is in the /gitlab endpoint and may fail
independently of https://pm.bsc.es/.

Cc: Víctor López <victor.lopez@bsc.es>
Reviewed-by: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
2025-10-01 16:40:16 +02:00
6ac5225ddb Monitor anella instead of gw.bsc.es
The target gw.bsc.es doesn't reply to our ICMP probes from hut. However,
the anella hop in the tracepath is a good candidate to identify cuts
between the login and the provider and between the provider and external
hosts like Google or Cloudflare DNS.

Reviewed-By: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
2025-10-01 16:40:16 +02:00
cd6983223e Add ICMP probes
These probes check if we can reach several targets via ICMP, which is
not proxied, so they can be used to see if ICMP forwarding is working in
the login node.

In particular, we test if we can reach the Google (8.8.8.8) and
Cloudflare (1.1.1.1) DNS servers, the BSC gateway which responds to ping
only from the intranet and the login node (ssfhead).

Reviewed-By: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
2025-10-01 16:40:16 +02:00
fb8a0cb0a3 Enable proxy for Grafana too
The alerts need to contact the slack endpoint, so we add the proxy
environment variables to the grafana systemd service.

Reviewed-By: Aleix Roca Nonell <aleix.rocanonell@bsc.es>
2025-10-01 16:40:16 +02:00
4d362351cb Monitor gitlab.bsc.es too 2025-10-01 16:40:16 +02:00
e9b4d87d9f Monitor PM webpage via blackbox 2025-10-01 16:40:16 +02:00
fbabc06641 Allow anonymous access to grafana 2025-10-01 16:40:16 +02:00
b84066fde5 Enable slurm-exporter service 2025-10-01 16:40:16 +02:00
09ac1d6c13 Make exporters listen in localhost only 2025-10-01 16:40:16 +02:00
042ca9e882 Scrape lake2 too 2025-10-01 16:40:16 +02:00
005a1be48a Scrape metrics from bay 2025-10-01 16:40:16 +02:00
91270b26bb Add ceph metrics to prometheus 2025-10-01 16:40:16 +02:00
8fcb5a1079 Monitor power from other nodes via LAN 2025-10-01 16:40:15 +02:00
b80656228d Increase prometheus retention time to one year 2025-10-01 16:40:15 +02:00
ae2007e2fe Allow access to devices for node_exporter 2025-10-01 16:40:15 +02:00
d679fd6314 Simplify flake and expose host pkgs
The configuration of the machines is now moved to m/
2025-10-01 16:40:15 +02:00