The critical factor seems to be the ability to fit the whole model in RAM (--mlo... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		Joeri on May 6, 2023 \| parent \| context \| favorite \| on: Releasing 3B and 7B RedPajama The critical factor seems to be the ability to fit the whole model in RAM (--mlock option in oobabooga). With Apple's RAM prices most M1/M2 owners probably don't have the 32 GB RAM required to fit a 4bit 30B model.

Semaphor on May 6, 2023 [–]

I have 64 GB RAM, but only a Ryzen 5 3600, and the larger models are very slow ;)

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact