Approximate calculation is putting the cost at ~$5 per million tokens (assuming 42W sustained, 20¢/kWh), and that's excluding hardware and other costs.
pja 2 hours ago [-]
That README hits all my “this is authored by an LLM” instincts. I presume the codebase is also written by an LLM?
I'm using my skills to orchestrate LLMs and agents, and I can write better code much faster. As developers, we can choose to adapt to new technologies or become extinct.
gruez 2 hours ago [-]
>Contributors
>...
>claude
You don't need to presume. If someone is so lazy that they tell claude to commit their code (ie. they're too lazy to run git commit themselves), the chances they reviewed the code is slim.
danirod 2 hours ago [-]
To be fair, I appreciate when they are so upfront about who wrote the code without requiring further heuristics, so I encourage this behavior.
bensyverson 2 hours ago [-]
Yes, I do this all the time, and also check in the co-authored project plans which drove the commits. For a project that is transparently only possible due to agentic coding, I don't see any reason to conceal the methods.
simonw 2 hours ago [-]
Honestly, Claude writes better commit messages than most people.
Personally I've mostly given in to letting it commit for me now, though I do occasionally take over and hand-write the messages if it's a particularly important concept and Claude's is too verbose.
Codex/GPT-x defaults to one-line commit messages, which are too short. Claude likes to write several paragraphs, which is usually too long.
If you tell it how to commit properly once per session it will stick with your standards for the rest of that session, and you can put that in AGENTS.md if you can be bothered to.
k8sToGo 2 hours ago [-]
Why do you say lazy? maybe they are ok with people seeing it is claude?
Teever 42 minutes ago [-]
That’s a needlessly antagonistic and insulting thing to say.
This person that you’ve never met and probably never will doesn’t owe any of us anything.
They’re out there doing what they want to do how they want to do it and if you don’t like it the correct response isn’t to insult them in front of a bunch of strangers on the internet for clout or whatever.
I doubt that you’d ever call them lazy to their face — why do it here?
bglazer 53 minutes ago [-]
Yeah I'm begging these authors to at least *read* the LLM generated README's. They're so, so incomprehensible because the LLM has a super limited theory of mind for readers. They always assume that external readers have access to the full context and history of decisions in the project development. These decisions and instructions from the user are extremely important for the model and almost completely irrelevant for an outside reader looking at a "finished" product. So, we get sentences like this:
"Where the levers were is not where they are. Overlapping the expert reads with the arithmetic was worth ~1.6x and shipped; the two that looked bigger — reading fewer bytes per token, and keeping more of them in RAM — were both measured and both refused, one because this family's router has no tail to demote and one because a cache the machine will not leave resident cannot be bought at any price."
What the fuck does that mean? Obviously some internal development decision, using the absolutely inscrutable internal terminology that Claude loves. If people would just read what they publish, I'm sure this would stick out immediately.
I'm not an LLM hater, I use them a ton and they work very well for writing complex code, it's undeniable. But they generate absolute dogshit first draft writing.
andai 8 minutes ago [-]
>They're so, so incomprehensible because the LLM has a super limited theory of mind for readers. They always assume that external readers have access to the full context and history of decisions in the project development
The transformer does not yet understand the non-transformer.[0]
This is probably because all the data we trained it on was created by non-transformers, so it thinks it's a non-transformer, but it isn't.
I don't think we know how to train a transformer yet. All the training data is linear, but that's not how they think at all.
[0] It's a bit like the communication difficulties experienced between autistic people and neurotypicals. Each follow the Golden Rule, i.e. do unto others as you would have them do unto you -- and it fails in both directions. A Platinum Rule is necessary: do unto others as their API demands.
chambored 32 minutes ago [-]
If that isn’t the perfect way to frame what I’ve seen and hated about LLM text, I don’t know what is. They certainly write for an audience with a historical context that almost no one has.
1 hours ago [-]
sergiomattei 35 minutes ago [-]
Does it matter?
jkahrs595 34 minutes ago [-]
I need an llm to block these comments.
cyanydeez 2 hours ago [-]
do people think these projects related to LLMs are ever going to be in anyway a pure human endevour?
How bout we make a new rule: only complain about LLM writing when the product as zero relevents to use with LLMs.
Catloafdev 16 minutes ago [-]
Neat! But, what do you do with a 0.5tk/s LLM?
Have you tried running it via llamacpp or other software that supports naive SSD offloading to compare speeds?
SSilver2k2 27 minutes ago [-]
This sounds a lot like what the colibri project did for GLM-5.2. I'm a fan so keep at it!
justvugg.github.io/colibri
herf 1 hours ago [-]
So if this Mac uses 30-50W, that's 40-60 tok/Wh...vs maybe 80k for a modern GPU cluster? So that's about 1000-2000x more power for the SSD streaming, unfortunately.
cjbprime 3 hours ago [-]
Does it not use Metal, on macOS? Would it be faster if it did?
marcobambini 2 hours ago [-]
We tried to use Metal, but for that specific project it was slower than just using NEON ARM optimizations. It is all documented in the docs.
ashivkum 2 hours ago [-]
[dead]
jpecar 2 hours ago [-]
Where can this 1tb k3.waste be downloaded?
marcobambini 2 hours ago [-]
It is not yet available, the only way is to download the official Kimi K3 model and then convert it:
# 1. preflight: reachable? how big? does it fit?
tools/fetch_weights.sh --dest /Volumes/staging/k3 --dry-run
# 2. download — resumable, safe to kill, safe to re-run
tools/fetch_weights.sh --dest /Volumes/staging/k3
# 3. convert into a container
uv run --with torch --with safetensors python tools/convert.py \
--src /Volumes/staging/k3 \
--out ~/models/k3.waste --jobs 3
jpecar 33 minutes ago [-]
Yeah, saw this ... was hoping that there's a torrent of it somewhere already. Or something.
logicallee 2 hours ago [-]
Interesting project. The headline number (29 GB of RAM) is for 4k context.
From what I've read elsewhere, Kimi K3 is quite verbose in its thinking. At the quoted rate, it would generate only a total of 1.8k tokens in 1 hour. Is that enough for it to get any thinking done and produce output on more complicated prompts?
0cf8612b2e1e 52 minutes ago [-]
I saw someone’s excellent idea that if you have a slow system like this, you should communicate by email. It is no longer meant for realtime iteration, but more pointed questions for which there is more effort and time expected on both parties.
I'm using my skills to orchestrate LLMs and agents, and I can write better code much faster. As developers, we can choose to adapt to new technologies or become extinct.
>...
>claude
You don't need to presume. If someone is so lazy that they tell claude to commit their code (ie. they're too lazy to run git commit themselves), the chances they reviewed the code is slim.
Personally I've mostly given in to letting it commit for me now, though I do occasionally take over and hand-write the messages if it's a particularly important concept and Claude's is too verbose.
Codex/GPT-x defaults to one-line commit messages, which are too short. Claude likes to write several paragraphs, which is usually too long.
If you tell it how to commit properly once per session it will stick with your standards for the rest of that session, and you can put that in AGENTS.md if you can be bothered to.
This person that you’ve never met and probably never will doesn’t owe any of us anything.
They’re out there doing what they want to do how they want to do it and if you don’t like it the correct response isn’t to insult them in front of a bunch of strangers on the internet for clout or whatever.
I doubt that you’d ever call them lazy to their face — why do it here?
"Where the levers were is not where they are. Overlapping the expert reads with the arithmetic was worth ~1.6x and shipped; the two that looked bigger — reading fewer bytes per token, and keeping more of them in RAM — were both measured and both refused, one because this family's router has no tail to demote and one because a cache the machine will not leave resident cannot be bought at any price."
What the fuck does that mean? Obviously some internal development decision, using the absolutely inscrutable internal terminology that Claude loves. If people would just read what they publish, I'm sure this would stick out immediately.
I'm not an LLM hater, I use them a ton and they work very well for writing complex code, it's undeniable. But they generate absolute dogshit first draft writing.
The transformer does not yet understand the non-transformer.[0]
This is probably because all the data we trained it on was created by non-transformers, so it thinks it's a non-transformer, but it isn't.
I don't think we know how to train a transformer yet. All the training data is linear, but that's not how they think at all.
[0] It's a bit like the communication difficulties experienced between autistic people and neurotypicals. Each follow the Golden Rule, i.e. do unto others as you would have them do unto you -- and it fails in both directions. A Platinum Rule is necessary: do unto others as their API demands.
How bout we make a new rule: only complain about LLM writing when the product as zero relevents to use with LLMs.
Have you tried running it via llamacpp or other software that supports naive SSD offloading to compare speeds?
justvugg.github.io/colibri
# 1. preflight: reachable? how big? does it fit? tools/fetch_weights.sh --dest /Volumes/staging/k3 --dry-run
# 2. download — resumable, safe to kill, safe to re-run tools/fetch_weights.sh --dest /Volumes/staging/k3
# 3. convert into a container uv run --with torch --with safetensors python tools/convert.py \ --src /Volumes/staging/k3 \ --out ~/models/k3.waste --jobs 3
From what I've read elsewhere, Kimi K3 is quite verbose in its thinking. At the quoted rate, it would generate only a total of 1.8k tokens in 1 hour. Is that enough for it to get any thinking done and produce output on more complicated prompts?