rarias/jungle
rarias/jungle
2
0
Fork 3
Code Issues 53 Pull Requests 10 Actions Packages Projects Releases Wiki Activity

Add fox to SLURM #167

Manually merged
rarias merged 13 commits from fox-slurm into master 2025-09-03 12:14:38 +02:00
Conversation 3 Commits 13 Files Changed 28 +214 -66
rarias commented 2025-09-02 16:54:47 +02:00
Owner
Copy Link

Configures a wireguard tunnel between fox and apex so we can safely exchange SLURM traffic. Then adds fox to the slurm config and makes the appropriate firewall rules to only accept traffic from fox coming from the tunnel in apex. Seems to be working fine:

apex% sinfo
PARTITION AVAIL  TIMELIMIT  NODES  STATE NODELIST
owl*         up   infinite      2  idle~ owl[1-2]
fox          up   infinite      1   idle fox
apex% srun -p fox grep Cpus_allowed_list /proc/self/status
Cpus_allowed_list:	0
apex% srun -p fox -c 8 grep Cpus_allowed_list /proc/self/status
Cpus_allowed_list:	0-7
apex% srun -p fox --exclusive grep Cpus_allowed_list /proc/self/status
Cpus_allowed_list:	0-191

NFS may be possible in the future.

Fixes: #150

Configures a wireguard tunnel between fox and apex so we can safely exchange SLURM traffic. Then adds fox to the slurm config and makes the appropriate firewall rules to only accept traffic from fox coming from the tunnel in apex. Seems to be working fine: ``` apex% sinfo PARTITION AVAIL TIMELIMIT NODES STATE NODELIST owl* up infinite 2 idle~ owl[1-2] fox up infinite 1 idle fox apex% srun -p fox grep Cpus_allowed_list /proc/self/status Cpus_allowed_list: 0 apex% srun -p fox -c 8 grep Cpus_allowed_list /proc/self/status Cpus_allowed_list: 0-7 apex% srun -p fox --exclusive grep Cpus_allowed_list /proc/self/status Cpus_allowed_list: 0-191 ``` NFS may be possible in the future. Fixes: #150
rarias added 10 commits 2025-09-02 16:54:48 +02:00 
Allows direct contact via the VPN when accessing from fox, but use
Internet when using the rest of the machines.
Protect fox against ssh bruteforce attacks:

fox% sudo lastb | head
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:25 - 11:25  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:24 - 11:24  (00:00)
root     ssh:notty    200.124.28.102   Mon Sep  1 11:24 - 11:24  (00:00)
rarias added 1 commit 2025-09-02 17:20:07 +02:00 
This reverts commit 64a52801ed8d5c4a57650c2c434254a9986c1901.
rarias added 1 commit 2025-09-02 17:22:42 +02:00
rarias requested review from arocanon 2025-09-03 10:04:40 +02:00
rarias commented 2025-09-03 11:05:55 +02:00
Author
Owner
Copy Link

Damm, the 10.100.0.0/16 subnet is colliding with the one used by the BSC VPN. We should change it to avoid collisions.

Damm, the 10.100.0.0/16 subnet is colliding with the one used by the BSC VPN. We should change it to avoid collisions.
rarias added 1 commit 2025-09-03 11:18:05 +02:00 
The 106 byte is the code for 'j' (jungle) in ASCII:

	% printf j | od -t d
	0000000         106
	0000001
arocanon requested changes 2025-09-03 11:42:31 +02:00
Dismissed
arocanon left a comment
Collaborator
Copy Link

Looks good to me! I have just added a minor comment

Looks good to me! I have just added a minor comment
m/apex/wireguard.nix Outdated
@ -0,0 +20,4 @@
{
name = "Fox";
publicKey = "VfMPBQLQTKeyXJSwv8wBhc6OV0j2qAxUpX3kLHunK2Y=";
allowedIPs = [ "10.106.0.0/24" ];
arocanon commented 2025-09-03 11:23:48 +02:00
Collaborator
Copy Link

Would it make sense to narrow it down to "10.100.0.1/32"?

Would it make sense to narrow it down to "10.100.0.1/32"?
rarias commented 2025-09-03 11:47:58 +02:00
Author
Owner
Copy Link

If we were only planning on supporting one node yes, but I may add other machines in the future, so I reserved the 256 IPs.

If we were only planning on supporting one node yes, but I may add other machines in the future, so I reserved the 256 IPs.
arocanon marked this conversation as resolved
arocanon approved these changes 2025-09-03 11:49:01 +02:00
rarias force-pushed fox-slurm from 67a3f57362 to 52380eae59 2025-09-03 12:03:31 +02:00 Compare
rarias manually merged commit 52380eae59 into master 2025-09-03 12:14:38 +02:00
Sign in to join this conversation.
Reviewers
No Reviewers
arocanon
Labels
Clear labels
bug
ceph
ci
config
cross
hw
io
kernel
mpi
net
nix
ops
repair
slurm
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
abonerib arocanon varcila rarias
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: rarias/jungle#167
No description provided.
Delete Branch "fox-slurm"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?