Driver crashes when loading a 7B model with 4-bit quantization. Solution: The driver’s memory scrubber may be too aggressive. Add siudi_npu.memory_scrub=0 to your kernel boot parameters.
sudo apt-get install linux-headers-$(uname -r) sudo dpkg -i siudi-7b-driver_2.1.0_arm64.deb Siudi 7b Driver
wget https://repos.siudi.ai/drivers/siudi-7b-driver_2.1.0_arm64.deb The driver compiles against your current kernel. Driver crashes when loading a 7B model with
+-----------------------------------------------------------------------------+ | Siudi 7b Driver Version: 2.1.0 NPU Clk: 1.2 GHz Temp: 45C | |-----------------------------------------------------------------------------| | GPU Name Bus-Id Memory-Usage Power | | 0 Siudi X7 0000:01:00.0 4580MiB / 8192MiB 15W | +-----------------------------------------------------------------------------+ Installing the driver is only half the battle. To truly run a 7B model smoothly, you need to adjust driver parameters. Setting the Power Governor The default governor prioritizes battery life. For chat applications, switch to performance mode: switch to performance mode: