200 tokens per second isn’t achievable with a 1.5B even on low-midrange GPUs. Unless they’re attaching an external GPU it’s not happening on a raspberry pi.

This article is disjointed and smells like AI.

What’s the deal with Alpine not using GNU? Is it a technical or ideological thing? Or is it another “because we can” type distro?

The model does have a lot of advantages over sdxl with the right prompting, but it seems to fall apart in prompts with more complex anatomy. Hopefully the community can fix it up once we have working trainers.

I assumed this was always the case

The issue is that they have no way of verifying that. We’d have to trust 2 other companies in addition to DDG.

All of Firefox’s ai initiatives including translation and chat are completely local. They have no impact on privacy.

The “why would they make this” people don’t understand how important this type of research is. It’s important to show what’s possible so that we can be ready for it. There are many bad actors already pursuing similar tools if they don’t have them already. The worst case is being blindsided by something not seen before.

The 8B is incredible for it’s size and they’ve managed to do sane refusal training this time for the official instruct.

The rest of the budget kind of sucks but this part makes sense. If you’re making significant profits off of users in a country you should have to pay some of that back. All countries should have this.

They’re already lying to get passed the 13 year requirement so I doubt it would make any difference.

I don’t think the term open-source can be applied to model weights. Even if you have the exact data, config, trainer and cluster it’s basically impossible to reproduce an exact model. Calling a model “open” sort of works but then there’s the distinction between open for research and open for commercial use. I think it’s kind of similar to the “free” software distinction. Maybe there’s some Latin word we could use.

It’s an AI thing. Nearly all small models struggle with separating multiple characters.

I’m sure the machine running it was quite warm actually.

Partnered with Adobe research so we’re never going to get the actual model.

This has more to do with how much chess data was fed into the model than any kind of reasoning ability. A 50M model can learn to play at 1500 elo with enough training: https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html

The “AI PC” specification requires a minimum of 40TOPs of AI compute which is over double the 18TOPs in the current M3s. Direct comparison doesn’t really work though.

What really matters is how it’s made available for development. The Neural engine is basically a black box. It can’t be incorporated into any low level projects because it’s only made available through a high-level swift api. Intel by comparison seems to be targeting pytorch acceleration with their libraries.

Do another 2 day blackout. That’ll show 'em.

This article is grossly overstating the findings of the paper. It’s true that bad generated data hurts model performance, but that’s true of bad human data as well. The paper used opt125M as their generator model, a very small research model with fairly low quality and often incoherent outputs. The higher quality generated data which makes up a majority of the generated text online is far less of an issue. The use of generated data to improve output consistency is a common practice for both text and image models.

It’s size makes it basically useless. It underperforms models even in it’s active weight class. It’s nice that it’s available but Grok-0 would have been far more interesting.

I feel like the whole Reddit AI deal is a trap. If any real judgment comes down about data use Reddit is an easy scapegoat. There was basically nothing stopping them from scraping the site for free.