rarias/jungle
rarias/jungle
2
0
Fork 1
Code Issues 34 Pull Requests 4 Packages Projects Releases Wiki Activity

WIP: Add fpga u280 #86

Draft
rarias wants to merge 8 commits from add-fpga-u280 into master
pull from: add-fpga-u280
merge into: rarias:master
Conversation 3 Commits 8 Files Changed 11 +399 -8
rarias commented 2025-02-20 16:38:30 +01:00
Owner
Copy Link

Closes #85

Closes https://jungle.bsc.es/git/rarias/jungle/issues/85
rarias added 6 commits 2025-02-20 16:38:31 +01:00 
Other machines like raccoon have direct access.
Allows services to run without a login session.
What an absolute nightmare, and we are far from the end.
Need to move to raccoon to access "config" attribute.
rarias commented 2025-02-20 16:42:33 +01:00
Author
Owner
Copy Link

The msd daemon fails to run, somehow the daemons still use the harcoded path:

raccoon% /nix/store/b6skv0iyl2djirf77q65sysq01qqncrn-xilinx-xrt/xrt/bin/msd
XRT build version: 2.19.0
Build hash:
Build date: 1980-01-01 00:00:00
Git branch:
PID: 281888
UID: 1880
[Thu Feb 20 15:41:16 2025 GMT]
HOST: raccoon
EXE: /nix/store/b6skv0iyl2djirf77q65sysq01qqncrn-xilinx-xrt/xrt/bin/msd
[WARNING] ERROR: No such library '/opt/xilinx/xrt/lib/libxrt_core.so.2'
The msd daemon fails to run, somehow the daemons still use the harcoded path: ``` raccoon% /nix/store/b6skv0iyl2djirf77q65sysq01qqncrn-xilinx-xrt/xrt/bin/msd XRT build version: 2.19.0 Build hash: Build date: 1980-01-01 00:00:00 Git branch: PID: 281888 UID: 1880 [Thu Feb 20 15:41:16 2025 GMT] HOST: raccoon EXE: /nix/store/b6skv0iyl2djirf77q65sysq01qqncrn-xilinx-xrt/xrt/bin/msd [WARNING] ERROR: No such library '/opt/xilinx/xrt/lib/libxrt_core.so.2' ```
rarias commented 2025-02-20 16:59:24 +01:00
Author
Owner
Copy Link

Probably I'm missing the udev rules to make a device.

raccoon% find . -name '*.rules'
./src/runtime_src/core/pcie/driver/aws/kernel/mgmt/10-awsmgmt.rules
./src/runtime_src/core/pcie/driver/linux/xocl/userpf/99-xocl.rules
./src/runtime_src/core/pcie/driver/linux/xocl/mgmtpf/99-xclmgmt.rules
./src/runtime_src/core/edge/drm/zocl/10-zocl.rules
raccoon% cat ./src/runtime_src/core/pcie/driver/linux/xocl/mgmtpf/99-xclmgmt.rules
KERNEL=="xclmgmt*",MODE="0666",SYMLINK="%k"
KERNEL=="xfpga/xvc_pub.m*",MODE="0666",SYMLINK="xvc_pub.m%n"
KERNEL=="xfpga/xvc_pri.m*",MODE="0600",SYMLINK="xvc_pri.m%n"
KERNEL=="xfpga/nifd_pri.m*",MODE="0600"
KERNEL=="xfpga/icap.m*",MODE="0200"
raccoon% cat ./src/runtime_src/core/pcie/driver/linux/xocl/userpf/99-xocl.rules
KERNEL=="renderD*",MODE="0666"
KERNEL=="xfpga/xvc_pub.u*",MODE="0666",SYMLINK="xvc_pub.u%n"
KERNEL=="xfpga/dma.qdma.u*",MODE="0666"
KERNEL=="xfpga/aximm_mon.u*",MODE="0666"
KERNEL=="xfpga/accel_mon.u*",MODE="0666"
KERNEL=="xfpga/axistream_mon.u*",MODE="0666"
KERNEL=="xfpga/trace_fifo_lite.u*",MODE="0666"
KERNEL=="xfpga/trace_fifo_full.u*",MODE="0666"
KERNEL=="xfpga/trace_funnel.u*",MODE="0666"
KERNEL=="xfpga/trace_s2mm.u*",MODE="0666"
KERNEL=="xfpga/accel_deadlock.u*",MODE="0666"
Probably I'm missing the udev rules to make a device. ``` raccoon% find . -name '*.rules' ./src/runtime_src/core/pcie/driver/aws/kernel/mgmt/10-awsmgmt.rules ./src/runtime_src/core/pcie/driver/linux/xocl/userpf/99-xocl.rules ./src/runtime_src/core/pcie/driver/linux/xocl/mgmtpf/99-xclmgmt.rules ./src/runtime_src/core/edge/drm/zocl/10-zocl.rules raccoon% cat ./src/runtime_src/core/pcie/driver/linux/xocl/mgmtpf/99-xclmgmt.rules KERNEL=="xclmgmt*",MODE="0666",SYMLINK="%k" KERNEL=="xfpga/xvc_pub.m*",MODE="0666",SYMLINK="xvc_pub.m%n" KERNEL=="xfpga/xvc_pri.m*",MODE="0600",SYMLINK="xvc_pri.m%n" KERNEL=="xfpga/nifd_pri.m*",MODE="0600" KERNEL=="xfpga/icap.m*",MODE="0200" raccoon% cat ./src/runtime_src/core/pcie/driver/linux/xocl/userpf/99-xocl.rules KERNEL=="renderD*",MODE="0666" KERNEL=="xfpga/xvc_pub.u*",MODE="0666",SYMLINK="xvc_pub.u%n" KERNEL=="xfpga/dma.qdma.u*",MODE="0666" KERNEL=="xfpga/aximm_mon.u*",MODE="0666" KERNEL=="xfpga/accel_mon.u*",MODE="0666" KERNEL=="xfpga/axistream_mon.u*",MODE="0666" KERNEL=="xfpga/trace_fifo_lite.u*",MODE="0666" KERNEL=="xfpga/trace_fifo_full.u*",MODE="0666" KERNEL=="xfpga/trace_funnel.u*",MODE="0666" KERNEL=="xfpga/trace_s2mm.u*",MODE="0666" KERNEL=="xfpga/accel_deadlock.u*",MODE="0666" ```
rarias added 2 commits 2025-02-21 10:31:39 +01:00 
They removed U280 from the list of supported devices, but that doesn't
mean it is will stop working.
rarias commented 2025-02-21 10:33:56 +01:00
Author
Owner
Copy Link

They had removed support for the U280 device. Reverting the commit makes the driver perform the initialization:

[198196.595845] xclmgmt init()
[198196.597370] xclmgmt 0000:83:00.0:  ffff8bfa058d50c0 xclmgmt_probe: Driver: 2.19.0,
[198196.597385] xclmgmt 0000:83:00.0:  ffff8bfa058d50c0 xclmgmt_probe: probe(pdev = 0x0000000039654fe7, pci_id = 0x00000000d0a106f3)
[198196.598732] xclmgmt 0000:83:00.0:  ffff8bfa058d50c0 xocl_subdev_vsec: No Vendor Specific Capability found.
[198196.599155] xclmgmt 0000:83:00.0:  ffff8bfa058d50c0 xclmgmt_probe: minimum initialization done
[198196.599641] xclmgmt 0000:83:00.0: flash.m.8388608 ffff8c0145c1ac10 qspi_probe: QSPI FIFO depth is: 256
[198196.599674] xclmgmt 0000:83:00.0: flash.m.8388608 ffff8c0145c1ac10 qspi_probe: Number of slave chips is: 1
[198196.599700] xclmgmt 0000:83:00.0: flash.m.8388608 ffff8c0145c1ac10 flash_get_info: Flash vendor: micron
[198196.599707] xclmgmt 0000:83:00.0: flash.m.8388608 ffff8c0145c1ac10 flash_get_info: Flash size: 128MB
[198196.599751] xclmgmt 0000:83:00.0:  ffff8bfa058d50c0 __xocl_subdev_create: Created subdev flash inst 8388608 level 0
[198196.601020] xclmgmt 0000:83:00.0:  ffff8bfa058d50c0 xocl_subdev_vsec: No Vendor Specific Capability found.
[198196.602101] xclmgmt 0000:83:00.0:  ffff8bfa058d50c0 xocl_subdev_vsec: No Vendor Specific Capability found.
[198196.603172] xclmgmt 0000:83:00.0:  ffff8bfa058d50c0 xocl_subdev_vsec: No Vendor Specific Capability found.

And now we see the FPGA:

raccoon% xrt/xrt/bin/xbmgmt examine
System Configuration
  OS Name              : Linux
  Release              : 6.9.9
  Machine              : x86_64
  CPU Cores            : 32
  Memory               : 64356 MB
  Distribution         : NixOS 24.11 (Vicuna)
  GLIBC                : 2.39
  Model                : W2600CR
  BIOS Vendor          : Intel Corp.
  BIOS Version         : SE5C600.86B.99.99.x044.020120121354

XRT
  Version              : 2.19.0
  Branch               :
  Hash                 :
  Hash Date            : 1980-01-01 00:00:00
  xocl                 : 2.19.0,
  xclmgmt              : 2.19.0,

Device(s) Present
|BDF             |Shell                 |Logic UUID  |Device ID  |Device Ready*  |
|----------------|----------------------|------------|-----------|---------------|
|[0000:83:00.0]  |xilinx_u280_GOLDEN_8  |n/a         |n/a        |No             |


* Devices that are not ready will have reduced functionality when using XRT tools
They had removed support for the U280 device. Reverting the commit makes the driver perform the initialization: ``` [198196.595845] xclmgmt init() [198196.597370] xclmgmt 0000:83:00.0: ffff8bfa058d50c0 xclmgmt_probe: Driver: 2.19.0, [198196.597385] xclmgmt 0000:83:00.0: ffff8bfa058d50c0 xclmgmt_probe: probe(pdev = 0x0000000039654fe7, pci_id = 0x00000000d0a106f3) [198196.598732] xclmgmt 0000:83:00.0: ffff8bfa058d50c0 xocl_subdev_vsec: No Vendor Specific Capability found. [198196.599155] xclmgmt 0000:83:00.0: ffff8bfa058d50c0 xclmgmt_probe: minimum initialization done [198196.599641] xclmgmt 0000:83:00.0: flash.m.8388608 ffff8c0145c1ac10 qspi_probe: QSPI FIFO depth is: 256 [198196.599674] xclmgmt 0000:83:00.0: flash.m.8388608 ffff8c0145c1ac10 qspi_probe: Number of slave chips is: 1 [198196.599700] xclmgmt 0000:83:00.0: flash.m.8388608 ffff8c0145c1ac10 flash_get_info: Flash vendor: micron [198196.599707] xclmgmt 0000:83:00.0: flash.m.8388608 ffff8c0145c1ac10 flash_get_info: Flash size: 128MB [198196.599751] xclmgmt 0000:83:00.0: ffff8bfa058d50c0 __xocl_subdev_create: Created subdev flash inst 8388608 level 0 [198196.601020] xclmgmt 0000:83:00.0: ffff8bfa058d50c0 xocl_subdev_vsec: No Vendor Specific Capability found. [198196.602101] xclmgmt 0000:83:00.0: ffff8bfa058d50c0 xocl_subdev_vsec: No Vendor Specific Capability found. [198196.603172] xclmgmt 0000:83:00.0: ffff8bfa058d50c0 xocl_subdev_vsec: No Vendor Specific Capability found. ``` And now we see the FPGA: ``` raccoon% xrt/xrt/bin/xbmgmt examine System Configuration OS Name : Linux Release : 6.9.9 Machine : x86_64 CPU Cores : 32 Memory : 64356 MB Distribution : NixOS 24.11 (Vicuna) GLIBC : 2.39 Model : W2600CR BIOS Vendor : Intel Corp. BIOS Version : SE5C600.86B.99.99.x044.020120121354 XRT Version : 2.19.0 Branch : Hash : Hash Date : 1980-01-01 00:00:00 xocl : 2.19.0, xclmgmt : 2.19.0, Device(s) Present |BDF |Shell |Logic UUID |Device ID |Device Ready* | |----------------|----------------------|------------|-----------|---------------| |[0000:83:00.0] |xilinx_u280_GOLDEN_8 |n/a |n/a |No | * Devices that are not ready will have reduced functionality when using XRT tools ```
This pull request has changes conflicting with the target branch.
  • m/common/base/ssh.nix
View command line instructions.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin add-fpga-u280:add-fpga-u280
git checkout add-fpga-u280
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
ceph

   
ci

   
config

   
hw

   
io

   
kernel

   
mpi

   
net

   
nix

   
ops
No Label
ceph
ci
config
hw
io
kernel
mpi
net
nix
ops
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: rarias/jungle#86
No description provided.
Delete Branch "add-fpga-u280"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?