From 97c0dd685924a09d9a816c2468edc719067d00d8 Mon Sep 17 00:00:00 2001 From: Rodrigo Arias Mallo Date: Fri, 12 Jul 2024 14:50:59 +0200 Subject: [PATCH] Remove reserved region hack --- JOURNAL.md | 200 ++++++++++++++++++++++++++++++++++++++++++++++++++++ ox-plic.dts | 4 -- 2 files changed, 200 insertions(+), 4 deletions(-) diff --git a/JOURNAL.md b/JOURNAL.md index 642d562..ae48e9e 100644 --- a/JOURNAL.md +++ b/JOURNAL.md @@ -1870,3 +1870,203 @@ However, now I cannot boot the kernel, as it is overwritting the FDT: Could not find a valid device tree Let's move the FDT to 0x80100000. + +Now I can load the kernel, but it gets stuck in the middle of the boot: + + GGGGGGGGG + OpenSBI v1.5 + ____ _____ ____ _____ + / __ \ / ____| _ \_ _| + | | | |_ __ ___ _ __ | (___ | |_) || | + | | | | '_ \ / _ \ '_ \ \___ \| _ < | | + | |__| | |_) | __/ | | |____) | |_) || |_ + \____/| .__/ \___|_| |_|_____/|____/_____| + | | + |_| + + sbi_timer_init: begins + sbi_timer_init: got Zicntr extension + fdt_timer_cold_init: pos = 0 + fdt_timer_cold_init: got match, name = riscv,aclint-mtimer + fdt_timer_cold_init: enabled + timer_mtimer_cold_init: begins, is_clint = 0 + timer_mtimer_cold_init: mtime_addr = 0x40002000 + timer_mtimer_cold_init: mtime_size = 0x00000008 + timer_mtimer_cold_init: mtimecmp_addr = 0x40002008 + timer_mtimer_cold_init: mtimecmp_size = 0x00000008 + fdt_timer_cold_init: drc->cold_init = 0 + fdt_timer_cold_init: pos = 1 + fdt_timer_cold_init: returns 0 + Platform Name : Barcelona Supercomputing Center - Lagarto Ox (NixOS) + Platform Features : medeleg + Platform HART Count : 1 + Platform IPI Device : --- + Platform Timer Device : aclint-mtimer @ 50000Hz + Platform Console Device : uart8250 + Platform HSM Device : --- + Platform PMU Device : --- + Platform Reboot Device : --- + Platform Shutdown Device : --- + Platform Suspend Device : --- + Platform CPPC Device : --- + Firmware Base : 0x80000000 + Firmware Size : 327 KB + Firmware RW Offset : 0x40000 + Firmware RW Size : 71 KB + Firmware Heap Offset : 0x49000 + Firmware Heap Size : 35 KB (total), 2 KB (reserved), 11 KB (used), 21 KB (free) + Firmware Scratch Size : 4096 B (total), 408 B (used), 3688 B (free) + Runtime SBI Version : 2.0 + + Domain0 Name : root + Domain0 Boot HART : 0 + Domain0 HARTs : 0* + Domain0 Region00 : 0x0000000040002000-0x000000004000200f M: (I,R,W) S/U: () + Domain0 Region01 : 0x0000000040001000-0x0000000040001fff M: (I,R,W) S/U: (R,W) + Domain0 Region02 : 0x0000000080040000-0x000000008005ffff M: (R,W) S/U: () + Domain0 Region03 : 0x0000000080000000-0x000000008003ffff M: (R,X) S/U: () + Domain0 Region04 : 0x0000000040800000-0x0000000040bfffff M: (I,R,W) S/U: (R,W) + Domain0 Region05 : 0x0000000000000000-0xffffffffffffffff M: () S/U: (R,W,X) + Domain0 Next Address : 0x0000000080200000 + Domain0 Next Arg1 : 0x0000000080100000 + Domain0 Next Mode : S-mode + Domain0 SysReset : yes + Domain0 SysSuspend : yes + + Boot HART ID : 0 + Boot HART Domain : root + Boot HART Priv Version : v1.10 + Boot HART Base ISA : rv64imafdc + Boot HART ISA Extensions : zicntr,zihpm,sdtrig + Boot HART PMP Count : 0 + Boot HART PMP Granularity : 0 bits + Boot HART PMP Address Bits: 0 + Boot HART MHPM Info : 29 (0xfffffff8) + Boot HART Debug Triggers : 0 triggers + Boot HART MIDELEG : 0x0000000000000222 + Boot HART MEDELEG : 0x000000000000b109 + + + Core: 12 devices, 8 uclasses, devicetree: board + Loading Environment from nowhere... OK + In: serial,usbkbd + Out: serial,vidconsole + Err: serial,vidconsole + No working controllers found + Net: No ethernet found. + Working FDT set to 80100000 + Hit any key to stop autoboot: 0 + + Device 0: unknown device + + Device 1: unknown device + scanning bus for devices... + + Device 0: unknown device + starting USB... + No working controllers found + No ethernet found. + No ethernet found. + => setenv bootargs "root=/dev/ram0 loglevel=7 debug rw earlycon=sbi console=hvc0" + => setenv ramdisk_size 12614846 + => booti ${kernel_addr_r} ${ramdisk_addr_r}:${ramdisk_size} ${fdtcontroladdr} + Moving Image from 0x84000000 to 0x80200000, end=83044650 + ## Flattened Device Tree blob at 80100000 + Booting using the fdt blob at 0x80100000 + Working FDT set to 80100000 + ERROR: reserving fdt memory region failed (addr=80000000 size=4000000 flags=4) + Using Device Tree in place at 0000000080100000, end 0000000080103dd0 + Working FDT set to 80100000 + + Starting kernel ... + + [ 0.000000] Linux version 6.9.7 (nixbld@localhost) (riscv64-unknown-linux-gnu-gcc (GCC) 13.3.0, GNU ld (GNU Binutils) 2.41) #1-NixOS Thu Jun 27 11:52:32 UTC 2024 + [ 0.000000] Machine model: Barcelona Supercomputing Center - Lagarto Ox (NixOS) + [ 0.000000] SBI specification v2.0 detected + [ 0.000000] SBI implementation ID=0x1 Version=0x10005 + [ 0.000000] SBI TIME extension detected + [ 0.000000] SBI IPI extension detected + [ 0.000000] SBI RFENCE extension detected + [ 0.000000] SBI DBCN extension detected + [ 0.000000] earlycon: sbi0 at I/O port 0x0 (options '') + [ 0.000000] printk: legacy bootconsole [sbi0] enabled + [ 0.000000] OF: reserved mem: Reserved memory: failed to reserve memory for node 'reserved@80000000': base 0x0000000080000000, size 64 MiB + [ 0.000000] OF: reserved mem: OVERLAP DETECTED! + [ 0.000000] mmode_resv1@80000000 (0x0000000080000000--0x0000000080040000) overlaps with reserved@80000000 (0x0000000080000000--0x0000000084000000) + [ 0.000000] OF: reserved mem: OVERLAP DETECTED! + [ 0.000000] reserved@80000000 (0x0000000080000000--0x0000000084000000) overlaps with mmode_resv0@80040000 (0x0000000080040000--0x0000000080060000) + [ 0.000000] Reserved memory: created DMA memory pool at 0x0000000060000000, size 256 MiB + [ 0.000000] OF: reserved mem: initialized node dma_pool@60000000, compatible id shared-dma-pool + [ 0.000000] OF: reserved mem: 0x0000000060000000..0x000000006fffffff (262144 KiB) map non-reusable dma_pool@60000000 + [ 0.000000] Reserved memory: created DMA memory pool at 0x0000000070000000, size 256 MiB + [ 0.000000] OF: reserved mem: initialized node dma_pool@70000000, compatible id shared-dma-pool + [ 0.000000] OF: reserved mem: 0x0000000070000000..0x000000007fffffff (262144 KiB) map non-reusable dma_pool@70000000 + [ 0.000000] OF: reserved mem: 0x0000000080000000..0x000000008003ffff (256 KiB) nomap non-reusable mmode_resv1@80000000 + [ 0.000000] OF: reserved mem: 0x0000000080000000..0x0000000083ffffff (65536 KiB) nomap non-reusable reserved@80000000 + [ 0.000000] OF: reserved mem: 0x0000000080040000..0x000000008005ffff (128 KiB) nomap non-reusable mmode_resv0@80040000 + [ 0.000000] cma: Reserved 16 MiB at 0x00000000af000000 on node -1 + [ 0.000000] Zone ranges: + [ 0.000000] DMA32 [mem 0x0000000080000000-0x00000000afffffff] + [ 0.000000] Normal empty + [ 0.000000] Movable zone start for each node + [ 0.000000] Early memory node ranges + [ 0.000000] node 0: [mem 0x0000000080000000-0x000000008005ffff] + [ 0.000000] node 0: [mem 0x0000000080060000-0x00000000afffffff] + [ 0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x00000000afffffff] + [ 0.000000] Falling back to deprecated "riscv,isa" + [ 0.000000] riscv: base ISA extensions adfim + [ 0.000000] riscv: ELF capabilities adfim + [ 0.000000] pcpu-alloc: s0 r0 d131072 u131072 alloc=1*131072 + [ 0.000000] pcpu-alloc: [0] 0 + [ 0.000000] Kernel command line: root=/dev/ram0 loglevel=7 debug rw earlycon=sbi console=hvc0 + [ 0.000000] Dentry cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) + [ 0.000000] Inode-cache hash table entries: 65536 (order: 7, 524288 bytes, linear) + [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 193536 + [ 0.000000] mem auto-init: stack:all(zero), heap alloc:off, heap free:off + [ 0.000000] Virtual kernel memory layout: + [ 0.000000] fixmap : 0xffffffc6fea00000 - 0xffffffc6ff000000 (6144 kB) + [ 0.000000] pci io : 0xffffffc6ff000000 - 0xffffffc700000000 ( 16 MB) + [ 0.000000] vmemmap : 0xffffffc700000000 - 0xffffffc800000000 (4096 MB) + [ 0.000000] vmalloc : 0xffffffc800000000 - 0xffffffd800000000 ( 64 GB) + [ 0.000000] modules : 0xffffffff02e45000 - 0xffffffff80000000 (2001 MB) + [ 0.000000] lowmem : 0xffffffd800000000 - 0xffffffd830000000 ( 768 MB) + [ 0.000000] kernel : 0xffffffff80000000 - 0xffffffffffffffff (2047 MB) + [ 0.000000] Memory: 386632K/786432K available (17075K kernel code, 9047K rwdata, 10240K rodata, 8737K init, 917K bss, 383416K reserved, 16384K cma-reserved) + [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 + [ 0.000000] ftrace: allocating 46961 entries in 184 pages + [ 0.000000] ftrace: allocated 184 pages with 4 groups + [ 0.000000] trace event string verifier disabled + [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1. + [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1. + [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 + [ 0.000000] riscv-intc: 64 local interrupts mapped + [ 0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x179dd7f66, max_idle_ns: 56421785867800 ns + [ 0.000020] sched_clock: 64 bits at 50kHz, resolution 20000ns, wraps every 70368744170000ns + [ 0.015020] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) + [ 0.035900] Console: colour dummy device 80x25 + [ 0.041720] Calibrating delay loop (skipped), value calculated using timer frequency.. 0.10 BogoMIPS (lpj=200) + [ 0.052480] pid_max: default: 32768 minimum: 301 + [ 0.187380] LSM: initializing lsm=capability,landlock,yama + [ 0.479640] landlock: Up and running. + [ 0.483700] Yama: becoming mindful. + [ 0.521600] Mount-cache hash table entries: 2048 (order: 2, 16384 bytes, linear) + [ 0.529780] Mountpoint-cache hash table entries: 2048 (order: 2, 16384 bytes, linear) + [ 0.784220] riscv: ELF compat mode unsupported + [ 0.784600] ASID allocator disabled (0 bits) + [ 0.867620] devtmpfs: initialized + [ 0.986340] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns + [ 0.996900] futex hash table entries: 256 (order: 1, 12288 bytes, linear) + [ 1.084560] pinctrl core: initialized pinctrl subsystem + [ 1.195460] NET: Registered PF_NETLINK/PF_ROUTE protocol family + [ 1.244960] DMA: preallocated 128 KiB GFP_KERNEL pool for atomic allocations + [ 1.255960] DMA: preallocated 128 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations + [ 1.266500] audit: initializing netlink subsys (disabled) + [ 1.317920] thermal_sys: Registered thermal governor 'step_wise' + [ 1.319800] cpuidle: using governor ladder + +This is very interesting, because all these lockups show the same symptoms and +they may be related with the kernel trying to access an area of memory that it +shouldn't. + +First, let's remove the reserved region, as now OpenSBI properly forwards the +regions to the kernel. diff --git a/ox-plic.dts b/ox-plic.dts index 1499c37..18ae4a5 100644 --- a/ox-plic.dts +++ b/ox-plic.dts @@ -84,10 +84,6 @@ #address-cells = <2>; /* Starting address and size */ #size-cells = <2>; /* 64 bits memory addresses */ ranges; - reserved@80000000 { - reg = <0x0 0x80000000 0x0 0x04000000>; - no-map; - }; eth_pool: dma_pool@60000000 { reg = <0x0 0x60000000 0x0 0x10000000>; compatible = "shared-dma-pool";