Running Gemma on a GPU-less machine

It is slow but workable for short answers.

@Theo_R9 Good checklist. I'd add: check the licence of the model too.

This 👆

Is there a guide for Gemma without a GPU? I couldn't find one.

@Nadia_S67 Please document exactly what is sent when cloud mode is on.

Qwen 14B runs comfortably on my 32 GB machine.

kamau-k, selinY, Louis_Laurent and 7 others like this.

Thanks everyone, this is really helpful!

Gemma is surprisingly good for short tasks. 💡

ayeshaR, chidi-b, ssato and 4 others like this.

Great points. I'll try the suggestions tonight and report back.

Thanks everyone, this is really helpful!

Update: solved it. Thanks all 🙌

Saving this for later 📌

Isabela_O26, kaya_zeynep, julesM and 17 others like this.

201

@Segun_Ogunleye This is the way 🚀

javier-r likes this.

It's about trust, and trust comes from being able to verify.

Talha_C94 likes this.

That's optimistic. Gemma without a GPU will take years, not months.

Kelechi_Nwosu, ppatel, vtorres and 1 other like this.

Yes please!!

172

@Nadia_S67 I disagree. Gemma without a GPU sounds great on paper but breaks down in real homes and offices. 💡