I think basically only NVIDIA is reliably supported right now, it would be nice ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		hedgehog on Dec 5, 2024 \| parent \| context \| favorite \| on: Dynamic 4bit Quantization I think basically only NVIDIA is reliably supported right now, it would be nice to have more hardware support to allow splitting models (like HF Accelerate or llama.cpp support).

danielhanchen on Dec 5, 2024 [–]

Yep for now NVIDIA - AMD might work but I'll have to edit the dependencies - more hardware support is coming! I'm trying to add Apple and CPU support!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact