GitHub - cactus-compute/cactus: Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots. · GitHub

Cactus is a source-available runtime for running models on the device rather than in a datacentre: quantization, kernels, runtime and inference engine, aimed at phones, wearables, smart home hardware and robots. The engine page carries the pitch -- quantized models with hardware-specific acceleration, tuned for battery rather than throughput, one SDK across iOS, Android, macOS and wearables, and compatibility with whatever runtime a team already has in place. The on-device category is getting crowded. What separates this entry is that the stack is inspectable all the way down to the kernels instead of arriving as a binary blob behind an API.

Visit Original Article →