And call of duty takes 100 of those xD

You can offload them into ram. The response time gets way slower once this happens, but you can do it. I’ve run a 70b llama model on my 3060 12gb at 2 bit quantisation (I do have plenty of ram so no offloading from ram to disk at least lmao). It took like 6-7 minutes to generate replies but it did work.

An external cartridge processing and providing battery power seems like a much better idea than the current solution of wearing the computer on your face. A small shoulder strapped device weighing a few hundred grams with a headset with more of a BSB sort of profile would be ideal for me.

I’m glad to see Apple experimenting with some ideas like this, not so much because I want an Apple headset, but because if it turns out to be a popular idea others will jump on board.

deleted by creator