108

cross-posted from: https://lemmit.online/post/4242386

This is an automated archive made by the Lemmit Bot.

The original was posted on /r/pcmasterrace by /u/trander6face on 2024-10-24 11:11:47+00:00.

you are viewing a single comment's thread
view the rest of the comments
[-] j4k3@piefed.world 10 points 5 days ago

You need a model compiled for the architecture. I saw some for the RK35xx devices when shopping for hardware. I do not think there is software made to split up or run models in general on a NPU. The models must be configured for the physical hardware topology. The stuff that runs on most devices is very small, and these either need a ton of custom fine tuning or they are barely capable of simple tasks.

On the other hand, segmentation models are small, and that makes layers, object identification, and background removal stuff work. Looking at your CPU speed, and available memory, it is unlikely to make much difference. You are also memory constrained for running models, though you could use deepspeed to load from a disk drive too.

this post was submitted on 31 Jan 2026
108 points (95.8% liked)

Opensource

5416 readers
219 users here now

A community for discussion about open source software! Ask questions, share knowledge, share news, or post interesting stuff related to it!

CreditsIcon base by Lorc under CC BY 3.0 with modifications to add a gradient



founded 2 years ago
MODERATORS