rarias/jungle
2
0
Fork 3
Code Issues 57 Pull Requests 7 Actions Packages Projects Releases Wiki Activity

Enable cuda system feature in raccoon and fox via nix-required-mounts #146

Manually merged
rarias merged 4 commits from abonerib/jungle:cuda-systemFeature into master 2025-07-23 11:57:47 +02:00
Conversation 2 Commits 4 Files Changed 7 +679 -10
abonerib commented 2025-07-18 11:41:16 +02:00
Collaborator
Copy Link

### EDIT
After testing in fox, we found that nix-required-mounts does not work properly. It does not find libcuda (/var/lib/opengl-driver does not seem to be included in the sandbox bypass?). As such, I leave that part of the MR for a later date.

Hopefully it is fixed by #147 (comment)

closes #147

This should allow running derivations that need a CUDA to run in the sandbox.

programs.nix-required-mounts.enable = true;
programs.nix-required-mounts.presets.nvidia-gpu.enable = true;

This should list /dev/dri and /dev/nvidia*

{
  inputs.jungle.url = "jungle";

  outputs = { self, jungle }: {
    packages.x86_64-linux.default =
      let
        pkgs = import jungle.inputs.nixpkgs {
          system = "x86_64-linux";
          allowUnfree = true;
          # cudaSupport = true; # may be needed for triton?
        };
      in
      pkgs.runCommand "list_dev" { requiredSystemFeatures = [ "cuda" ]; } ''
        ls /dev >$out
      '';
  };
}

I have tested nix build jungle#python3Packages.triton.gpuCheck in the current configuration in fox and it fails:

error: a 'x86_64-linux' with features {cuda} is required to build '/nix/store/5g96p5c72j4i8kk5jazs77j9jd8c7j00-triton-pytest-3.1.0.drv', but I am a 'x86_64-linux' with features {benchmark, big-parallel, kvm, nixos-test}
~~### EDIT~~ ~~After testing in fox, we found that `nix-required-mounts` does not work properly. It does not find libcuda (/var/lib/opengl-driver does not seem to be included in the sandbox bypass?). As such, I leave that part of the MR for a later date.~~ Hopefully it is fixed by https://jungle.bsc.es/git/rarias/jungle/issues/147#issuecomment-4079 closes #147 ---- This should allow running derivations that need a CUDA to run in the sandbox. ```nix programs.nix-required-mounts.enable = true; programs.nix-required-mounts.presets.nvidia-gpu.enable = true; ``` This should list `/dev/dri` and `/dev/nvidia*` ```nix { inputs.jungle.url = "jungle"; outputs = { self, jungle }: { packages.x86_64-linux.default = let pkgs = import jungle.inputs.nixpkgs { system = "x86_64-linux"; allowUnfree = true; # cudaSupport = true; # may be needed for triton? }; in pkgs.runCommand "list_dev" { requiredSystemFeatures = [ "cuda" ]; } '' ls /dev >$out ''; }; } ``` I have tested `nix build jungle#python3Packages.triton.gpuCheck` in the current configuration in `fox` and it fails: ``` error: a 'x86_64-linux' with features {cuda} is required to build '/nix/store/5g96p5c72j4i8kk5jazs77j9jd8c7j00-triton-pytest-3.1.0.drv', but I am a 'x86_64-linux' with features {benchmark, big-parallel, kvm, nixos-test} ```
abonerib added 2 commits 2025-07-18 11:41:17 +02:00
Move shared nvidia settings to a separate module cfdada0008
Enable cuda system feature in raccoon and fox via nix-required-mounts 1a72f65584 
This allows running derivations which depend on cuda runtime without
breaking the sandbox. We only need to add `requiredSystemFeatures = [ "cuda" ];`
to the derivation.
abonerib changed title from Enable cuda system feature in raccoon and fox via nix-required-mounts to WIP: Enable cuda system feature in raccoon and fox via nix-required-mounts 2025-07-18 15:32:30 +02:00
abonerib force-pushed cuda-systemFeature from 1a72f65584 to 42d69bf0ec 2025-07-18 15:52:02 +02:00 Compare
abonerib changed title from WIP: Enable cuda system feature in raccoon and fox via nix-required-mounts to Move shared nvidia settings to a separate module 2025-07-18 15:52:29 +02:00
rarias added a new dependency 2025-07-21 17:16:39 +02:00
#148 Fixes for raccoon after the upgrade to NixOS 25.05
abonerib added 2 commits 2025-07-21 17:24:09 +02:00
Enable cuda system feature in raccoon and fox via nix-required-mounts 41adc3b2dc 
This allows running derivations which depend on cuda runtime without
breaking the sandbox. We only need to add `requiredSystemFeatures = [ "cuda" ];`
to the derivation.
Add missing symlink in cuda sandbox 0f70c245c1
abonerib changed title from Move shared nvidia settings to a separate module to Enable cuda system feature in raccoon and fox via nix-required-mounts 2025-07-21 17:26:12 +02:00
abonerib force-pushed cuda-systemFeature from 0f70c245c1 to 378716c0fa 2025-07-21 18:22:58 +02:00 Compare
rarias added 1 commit 2025-07-22 15:29:03 +02:00
Add cudainfo program to test CUDA in sandbox f4563008b1 
The cudainfo program is executed inside the build sandbox so we can see
if it works properly. It uses the autoAddDriverRunpath hook to inject in
the runpath the location of the library directory for CUDA libraries.
rarias force-pushed cuda-systemFeature from f4563008b1 to 8b1df39edf 2025-07-22 16:16:33 +02:00 Compare
rarias approved these changes 2025-07-22 17:05:53 +02:00
abonerib reviewed 2025-07-22 17:16:42 +02:00
pkgs/cudainfo/default.nix Outdated
@@ -0,0 +24,4 @@
requiredSystemFeatures = [ "cuda" ];
dontBuild = true;
nativeCheckInputs = [
cudainfo
abonerib commented 2025-07-22 17:16:42 +02:00
Author
Collaborator
Copy Link

We could try to use the finalAttrs pattern and finalAttrs.finalPackage instead of cudainfo so we can do (cudainfo.overrideAtrrs (prev: {})).gpuCheck without running into weirdness.

We could try to use the `finalAttrs` pattern and `finalAttrs.finalPackage` instead of `cudainfo` so we can do `(cudainfo.overrideAtrrs (prev: {})).gpuCheck` without running into weirdness.
👍 1
rarias marked this conversation as resolved
rarias force-pushed cuda-systemFeature from 8b1df39edf to 8d6735f799 2025-07-22 17:27:34 +02:00 Compare
rarias force-pushed cuda-systemFeature from 8d6735f799 to 9b681ab7ce 2025-07-23 11:56:31 +02:00 Compare
rarias manually merged commit 9b681ab7ce into master 2025-07-23 11:57:47 +02:00
Sign in to join this conversation.
Reviewers
No Reviewers
rarias
abonerib
Labels
Clear labels
bug
ceph
ci
config
cross
hw
io
kernel
mpi
net
nix
ops
repair
slurm
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
abonerib arocanon varcila rarias
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Depends on
#148 Fixes for raccoon after the upgrade to NixOS 25.05
rarias/jungle
Reference: rarias/jungle#146
Delete Branch "abonerib/jungle:cuda-systemFeature"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?