@tinygrad — 5707 tweets

@__tinygrad__ · 2024-03-16 01:50
tiny corp thought complex seven (2029) https://t.co/5JWcvK4Zwc

@__tinygrad__ · 2024-05-10 23:03
@HansCNelson @tenstorrent @jimkxa Sadly it isn't competitive with consumer GPUs on FLOPS/$ or FLOPS/W, and it has a unique programming model that we aren't focused on. We'd be happy to do it, but it would have to be seriously subsidized by @tenstorrent.

@jimkxa · 2024-05-11 15:26
@__tinygrad__ @HansCNelson @tenstorrent We'll make tinygrad work on Tenstorrent soon. We released fast dispatch 2.0 and are cleaning up APIs and a few other things. A few more steps and running tinygrad will be easy and pretty fast

@__tinygrad__ · 2024-06-18 17:04
.@AMD @LisaSu For $1M and two boxes, we'll get the MI300X on the next MLPerf.

@leopoldasch · 2024-06-21 03:52
"I realized the other day...they need electricity... they need double what we have right now" https://t.co/mExZ1AyqjN

@chenyuxyz · 2024-09-26 07:56
5h28m

@chenyuxyz · 2024-10-04 04:13
4h32m!

@sandyasm · 2024-11-08 11:36
@__tinygrad__ Isn't oasis from @DecartAI running on @Etched? That looks like a reasonable real world application..

@__tinygrad__ · 2024-11-08 11:40
@sandyasm @DecartAI @Etched It's using NVIDIA!!! https://t.co/b37eTBkzHi

@ID_AA_Carmack · 2024-11-27 03:02
With the Nvidia GB200 NVL72, can a single process launch kernels on all GPUs, CUDA:0 to CUDA:71, or do you still have to start CPU processes on 18 different hosts? https://t.co/MK9iZhNOK5

@__tinygrad__ · 2024-12-06 05:36
The tinygrad codebase is 248k tokens. Do any open source LLMs support a context this big? Soon, tinygrad will be recursively self improving.

@mattparlmer · 2024-12-07 20:39
Low hanging fruit for US domestic electronics production: - server motherboards/other major components - networking hardware - commercial radio transceivers - industrial controllers - gaming consoles - high end sensor packages What else?

@yaroslavvb · 2024-12-11 16:00
I can't understand why AMD is so bad. Not just technically behind, but strategically bad. Not setting prices low enough, not giving students/early-adopters easy access to their cards, not being aggressive at hiring top AI infra people

@__tinygrad__ · 2024-12-11 16:55
We have the same question. We gave up and soon tinygrad will depend on 0 AMD code except what's required by code signing. We did this for the 7900XTX (tinybox red). If AMD was thinking strategically, they'd be begging us to take some free MI300s to add support for it.

@__tinygrad__ · 2024-12-11 17:50
Is there an easy way to get a quote for the MI300X platform board? That's the other problem with these chips, they are a PITA to buy. The 8x board has 10.5 PFLOPS/1.5TB/42.4 TB/s. That's 32/64/42, ~50 4090's. Can we buy board for $80k?

@uzzi38 · 2024-12-11 18:12
Maybe you should try asking the AMD rep in your Discord? You know the guy that popped in to try give you all a hand? Wonder what happened to that guy who just wanted to be helpful?

@__tinygrad__ · 2024-12-11 17:50
Is there an easy way to get a quote for the MI300X platform board? That's the other problem with these chips, they are a PITA to buy. The 8x board has 10.5 PFLOPS/1.5TB/42.4 TB/s. That's 32/64/42, ~50 4090's. Can we buy board for $80k?

@jannikmeissner · 2024-12-11 18:19
@__tinygrad__ If you send me a DM, maybe I can make an intro; can’t promise it, but I know someone who at least should know who can help at AMD

@__tinygrad__ · 2024-12-11 18:21
@Leik0w0 This is way too good to be real. That computer alone (without the MI300X) should be around that price.

@Leik0w0 · 2024-12-11 18:23
@__tinygrad__ Agreed, though I see the 8x h100 oam boards at 10x that price. You need to have a superbuy agent ask for a quote to get a real price I guess

@Leik0w0 · 2024-12-11 18:23
@__tinygrad__ Agreed, though I see the 8x h100 oam boards at 10x that price. You need to have a superbuy agent ask for a quote to get a real price I guess

@__tinygrad__ · 2024-12-11 18:24
@Leik0w0 I do believe $80k for the boards is reasonable though and I bet that's real. If AMD is charging more idk why people are buying them, that's 4090 competitive but not 5090 competitive.

@__tinygrad__ · 2024-12-11 18:24
@Leik0w0 I do believe $80k for the boards is reasonable though and I bet that's real. If AMD is charging more idk why people are buying them, that's 4090 competitive but not 5090 competitive.

@Leik0w0 · 2024-12-11 18:25
@__tinygrad__ Do we know the mi300X price for exascalers ?

@Leik0w0 · 2024-12-11 18:25
@__tinygrad__ Do we know the mi300X price for exascalers ?

@__tinygrad__ · 2024-12-11 18:26
@Leik0w0 Samsung paid about $10k each (so $80k for the board) according to this. https://t.co/Tq8Yqht7eX

@__tinygrad__ · 2024-12-11 18:26
@Leik0w0 Samsung paid about $10k each (so $80k for the board) according to this. https://t.co/Tq8Yqht7eX

@Leik0w0 · 2024-12-11 18:27
@__tinygrad__ You are going to need the rest of the server too (+ they will probably charge you more than they charged samsung because order qty)

@Leik0w0 · 2024-12-11 18:27
@__tinygrad__ You are going to need the rest of the server too (+ they will probably charge you more than they charged samsung because order qty)

@__tinygrad__ · 2024-12-11 18:29
@Leik0w0 We are building servers, I just want the board. MI300X is old now.

@__tinygrad__ · 2024-12-11 19:24
@PytorchToAtoms @QuentinAnthon15 Will be very happy if 5090 has TMA.

@jannikmeissner · 2024-12-11 18:19
@__tinygrad__ If you send me a DM, maybe I can make an intro; can’t promise it, but I know someone who at least should know who can help at AMD

@__tinygrad__ · 2024-12-11 21:55
@jannikmeissner We got what's probably the official quote from a distributor of $170k. Not close to worth it at that price, consumer GPUs are a much better deal.

@__tinygrad__ · 2024-12-11 16:55
We have the same question. We gave up and soon tinygrad will depend on 0 AMD code except what's required by code signing. We did this for the 7900XTX (tinybox red). If AMD was thinking strategically, they'd be begging us to take some free MI300s to add support for it.

@chankhavu · 2024-12-12 14:15
@__tinygrad__ I talked to some AMD folks at a conference this year. They were presenting MI300x and how amazing it is compared to Nvidia. They never heard about Tiny Corp and downplayed software issues.

@chankhavu · 2024-12-12 14:15
@__tinygrad__ I talked to some AMD folks at a conference this year. They were presenting MI300x and how amazing it is compared to Nvidia. They never heard about Tiny Corp and downplayed software issues.

@__tinygrad__ · 2024-12-12 14:45
@chankhavu We sell 9 green boxes for every red box we sell despite green boxes costing $10k more. I wonder why AMD thinks this is?

@uzzi38 · 2024-12-11 18:12
Maybe you should try asking the AMD rep in your Discord? You know the guy that popped in to try give you all a hand? Wonder what happened to that guy who just wanted to be helpful?

@mrcornfield · 2024-12-12 14:46
@uzzi38 Seems like he's not actually having trouble with buying it, is just trying to get a better deal. Kinda scummy way to do this, but maybe he's right that it's overpriced. I have no frame of reference for that market. https://t.co/YuS6m8AQ09

@mrcornfield · 2024-12-12 14:46
@uzzi38 Seems like he's not actually having trouble with buying it, is just trying to get a better deal. Kinda scummy way to do this, but maybe he's right that it's overpriced. I have no frame of reference for that market. https://t.co/YuS6m8AQ09

@__tinygrad__ · 2024-12-12 14:56
@mrcornfield @uzzi38 Posted that before the distributor got back to me.

@uzzi38 · 2024-12-11 18:12
Maybe you should try asking the AMD rep in your Discord? You know the guy that popped in to try give you all a hand? Wonder what happened to that guy who just wanted to be helpful?

@__tinygrad__ · 2024-12-12 14:58
@uzzi38 But they weren't helpful. Look, AMD is free to ignore us, though how do they answer why we sell 9 green tinyboxes for each red one? The perf is similar...and the green one is $10k more! It's all the software. They will either figure this out or not be relevant in AI.

@__tinygrad__ · 2024-12-11 17:50
Is there an easy way to get a quote for the MI300X platform board? That's the other problem with these chips, they are a PITA to buy. The 8x board has 10.5 PFLOPS/1.5TB/42.4 TB/s. That's 32/64/42, ~50 4090's. Can we buy board for $80k?

@teodoroj · 2024-12-12 16:40
@__tinygrad__ Your pro box only has 8kW power. The MI300X GPUs alone consume 6kW. The CPU board is 1kW or more. Not sure your system is capable.

@teodoroj · 2024-12-12 16:40
@__tinygrad__ Your pro box only has 8kW power. The MI300X GPUs alone consume 6kW. The CPU board is 1kW or more. Not sure your system is capable.

@__tinygrad__ · 2024-12-12 17:24
@teodoroj Sounds like 7kW....and we have an option to expand to up to 12 kW, we just haven't needed it. But if the price is $170k, I'd much rather an 8xH100 board for $200k.

@__tinygrad__ · 2024-12-12 18:05
We sell 9 green boxes for every red box, even though red boxes are 60% of the cost. The theoretical performance is similar, it's all software holding it back. I suspect both these ratios are the same for H100 vs MI300. How is fixing this not AMD's highest priority?

@__tinygrad__ · 2024-12-12 18:05
We sell 9 green boxes for every red box, even though red boxes are 60% of the cost. The theoretical performance is similar, it's all software holding it back. I suspect both these ratios are the same for H100 vs MI300. How is fixing this not AMD's highest priority?

@Yuchenj_UW · 2024-12-12 18:08
@__tinygrad__ Does tinygrad work on MI300?

@Yuchenj_UW · 2024-12-12 18:08
@__tinygrad__ Does tinygrad work on MI300?

@__tinygrad__ · 2024-12-12 18:10
@Yuchenj_UW Maybe through HIP or OpenCL, but not with any real speed. We don't own any, and there's no reason to pay $170k for them. Their stack is bad at so many levels that their engineers can't even get good perf from them in benchmark environments. https://t.co/vmXlPEIJWB

@__tinygrad__ · 2024-12-12 18:05
We sell 9 green boxes for every red box, even though red boxes are 60% of the cost. The theoretical performance is similar, it's all software holding it back. I suspect both these ratios are the same for H100 vs MI300. How is fixing this not AMD's highest priority?

@danieltvela · 2024-12-12 18:12
@__tinygrad__ Maybe AMD has given up on competing with NVIDIA in the high-end GPU market: https://t.co/qm2aOb0ICv

@__tinygrad__ · 2024-12-12 18:05
We sell 9 green boxes for every red box, even though red boxes are 60% of the cost. The theoretical performance is similar, it's all software holding it back. I suspect both these ratios are the same for H100 vs MI300. How is fixing this not AMD's highest priority?

@stochasticchasm · 2024-12-12 18:13
@__tinygrad__ That’s a higher red box amount sold than I would have expected lol

@danieltvela · 2024-12-12 18:12
@__tinygrad__ Maybe AMD has given up on competing with NVIDIA in the high-end GPU market: https://t.co/qm2aOb0ICv

@__tinygrad__ · 2024-12-12 18:14
@danieltvela And at the current rate, they'll give up competing in the high end ML accelerator market soon too. Super sad for NVIDIA to win like this.

@stochasticchasm · 2024-12-12 18:13
@__tinygrad__ That’s a higher red box amount sold than I would have expected lol

@__tinygrad__ · 2024-12-12 18:17
@stochasticchasm It's the same ratio in gaming. https://t.co/1xgTr3Q5Pe

@__tinygrad__ · 2024-12-13 00:46
The new gradient is 3x shorter than the old one. And no weird "requires_grad" anymore. Just dw, db = loss.gradient(w, b) https://t.co/oJzDvOEy0S

@__tinygrad__ · 2024-12-12 14:58
@uzzi38 But they weren't helpful. Look, AMD is free to ignore us, though how do they answer why we sell 9 green tinyboxes for each red one? The perf is similar...and the green one is $10k more! It's all the software. They will either figure this out or not be relevant in AI.

@clattner_llvm · 2024-12-13 05:37
@__tinygrad__ @uzzi38 What do you see as the path to fix this, and what are the blockers? I thought you said that tinygrad solved it already?

@__tinygrad__ · 2024-12-13 07:20
RT @wholemars: tiny box factory at comma ai https://t.co/arK8WtGCrI

@clattner_llvm · 2024-12-13 05:37
@__tinygrad__ @uzzi38 What do you see as the path to fix this, and what are the blockers? I thought you said that tinygrad solved it already?

@__tinygrad__ · 2024-12-13 07:48
Not yet. We are the only people to get AMD on MLPerf training, at least it's somewhat stable but our times aren't great. We're working on a user space driver that's quite minimal. Example: it just doesn't init the MES. It's a single hardware queue. This fixes driver instability, and should be merged in a week or two. tinygrad is pretty competitive for forward passes, but our backward passes aren't good. This is higher level stuff in the tinygrad framework, like how to schedule ops. We just landed a big refactor removing an unneeded layer of indirection, and can now use our graph_rewrite infrastructure on the "big graph." That's the path to fixing this. Then there's speed within kernels. We do okay with BEAM search, and are around 80% of best on NVIDIA, on par with best for METAL, and usually beating AMD. We can do even better here, this involves refactoring kernel and expanding the search space of the optimizations. Overall, there's no one thing causing the problem. AMD doesn't seem to have very good software development practices, no fuzz testing, I don't even think there's CI! tinygrad is small, has almost 0 deps, and has very good CI. It will be a long road, but at the end our end to end alternative stack will be far superior to what AMD provides. At that point though, it'll be time to make our own chips.

@obadakhalili · 2024-12-13 16:14
@chenyuxyz @__tinygrad__ It appears that you have created your own DSL within Python. I need to understand this language and its background in order to use it effectively. This is different from standard Python code that I am already familiar with and is easier for me to read and understand.

@chenyuxyz · 2024-12-13 17:04
@obadakhalili @__tinygrad__ familiarity is not maintainability. also these backward functions have been in tinygrad for more than a year (the new gradient is a refactor to that, nothing is new). so either you have not used tinygrad, or you don't need to know the tinygrad internal to use it effectively

@chenyuxyz · 2024-12-13 17:04
@obadakhalili @__tinygrad__ familiarity is not maintainability. also these backward functions have been in tinygrad for more than a year (the new gradient is a refactor to that, nothing is new). so either you have not used tinygrad, or you don't need to know the tinygrad internal to use it effectively

@__tinygrad__ · 2024-12-13 17:21
@chenyuxyz @obadakhalili We should write docs for the PatternMatcher though, it's pretty stable I think.

@mattparlmer · 2024-12-07 22:31
@genfabco When I say that battery production would be exceptionally difficult to bring onshore this is what I mean https://t.co/Rzoz6IoXWQ

@mattparlmer · 2024-12-13 19:41
@genfabco Here's a good example of domestic electronics manufacturing that makes sense, @realGeorgeHotz's @__tinygrad__ in San Diego https://t.co/zZt8PpBOa3

@mattparlmer · 2024-12-13 19:41
@genfabco Here's a good example of domestic electronics manufacturing that makes sense, @realGeorgeHotz's @__tinygrad__ in San Diego https://t.co/zZt8PpBOa3

@__tinygrad__ · 2024-12-13 20:00
@mattparlmer @genfabco @realGeorgeHotz Though I'm sure with w/e regulations are passed, for some reason we won't qualify as a US manufacturer. Like how Exxon was an ESG company but Tesla wasn't.

@__tinygrad__ · 2024-12-13 23:29
RT @ritteradam: You can look at tinygrad on how George Hotz is handling it (he’s hiring _only_ with bounties with a Google sheet that shows which is being worked on). His code is very clean as well, but puts a lot of efforts in reviewing (he reviewed my pull request 10 minutes after subitting, for HVM3 I’m still waiting after a few days for something trivial). Just reply fast, trust that there will be a pareto distribution of quality/quantity of code, and have good automatic tests

@__tinygrad__ · 2024-12-14 01:37
The new gradient API is merged: weight_grad, bias_grad = loss.gradient(weight, bias) For optimizers (see PR #8231) the new API is: optim.step(loss) You don't need zero_grad or loss.backward. It's not like torch (old API) or JAX (function grad). It's new. This make sense?

@__tinygrad__ · 2024-12-14 01:37
The new gradient API is merged: weight_grad, bias_grad = loss.gradient(weight, bias) For optimizers (see PR #8231) the new API is: optim.step(loss) You don't need zero_grad or loss.backward. It's not like torch (old API) or JAX (function grad). It's new. This make sense?

@cHHillee · 2024-12-14 01:44
@__tinygrad__ This is the same API as torch.autograd.grad I believe. https://t.co/luymErvp1o

@cHHillee · 2024-12-14 01:44
@__tinygrad__ This is the same API as torch.autograd.grad I believe. https://t.co/luymErvp1o

@__tinygrad__ · 2024-12-14 01:52
@cHHillee Oh nice! Yea, it seems more straightforward, do you know why this isn't the normal way to use PyTorch?

@__tinygrad__ · 2024-12-14 01:52
@cHHillee Oh nice! Yea, it seems more straightforward, do you know why this isn't the normal way to use PyTorch?

@ezyang · 2024-12-14 02:04
@__tinygrad__ @cHHillee It turns out people don't like having to explicitly pass all their parameters to grad lol

@cHHillee · 2024-12-14 02:05
It's decent - useful in a lot of cases. But this API requires you to explicitly have a way to refer to all of your parameters (uniquely) at the time you call backwards. Although in some cases this can be simple, in other cases this can be complicated (let's say two of your module's parameters are shared).

@yacineMTB · 2024-12-14 02:08
@cHHillee @__tinygrad__ no module zone. torch functional only in this household

@yacineMTB · 2024-12-14 02:08
@cHHillee @__tinygrad__ no module zone. torch functional only in this household

@cHHillee · 2024-12-14 02:13
Even if you don't have modules you can still have issues with shared parameters. For example, let's say you tie the lm_head with the embedding matrix. A natural way to structure your (say, dictionary) of weights would be to have a sub-dict for your embedding and your projection layer, and then pass the same tensor in for both dicts.

@cHHillee · 2024-12-14 02:13
Even if you don't have modules you can still have issues with shared parameters. For example, let's say you tie the lm_head with the embedding matrix. A natural way to structure your (say, dictionary) of weights would be to have a sub-dict for your embedding and your projection layer, and then pass the same tensor in for both dicts.

@__tinygrad__ · 2024-12-14 02:29
@cHHillee @yacineMTB The new optim is merged now. We still support backward, but I want to rip it out. It's keeping state in places it shouldn't. In tinygrad, everything is lazy, so you can call gradient as many times as you'd like on the same params w/o a perf hit. Does this fix your issue?

@__tinygrad__ · 2024-12-14 02:30
@hy3na_xyz Time spent on blog is not time spent on code!

@ezyang · 2024-12-14 02:04
@__tinygrad__ @cHHillee It turns out people don't like having to explicitly pass all their parameters to grad lol

@__tinygrad__ · 2024-12-14 02:31
@ezyang @cHHillee Our optimizer API makes this easy, you are already passing the param list to optimizer so you don't have to again. Is there another use where it's annoying?

@__tinygrad__ · 2024-12-14 02:33
@hy3na_xyz useful and well written code invites...

@__tinygrad__ · 2024-12-14 02:34
Yes. tinygrad doesn't have nn.module and never has. Our goal is to make the code as easy to reason about as possible.

@cHHillee · 2024-12-14 02:40
Rolling the loss.backward() into optimizer.step() does solve this problem, but then seems trickier to support other things like: 1. gradient accumulation (multiple loss.backward() calls before one optimizer step). 2. multiple losses (loss1.backward(), loss2.backward()) 3. gradient clipping 4. different optimizers for different parts of your model Of course, none of these are impossible with this API. You could say, "use optimizer.step() for the common case, use loss.gradient for the rest". But those examples above would be my first instinct for limitations of an optimizer.step(loss) API.

@__tinygrad__ · 2024-12-14 02:46
@cHHillee @ezyang Hmm, I see. Maybe we can keep backward too.

@__tinygrad__ · 2024-12-14 02:46
@cHHillee @ezyang Hmm, I see. Maybe we can keep backward too.

@__tinygrad__ · 2024-12-14 02:49
@cHHillee @ezyang accumulation is the only issue I see. for multiple losses, you can add them, right? for clipping it's easier now. could add a param to optimizer to map a function over grads. for different optimizers, just run them. it's lazy and will be fast.

@__tinygrad__ · 2024-12-14 02:49
@cHHillee @ezyang accumulation is the only issue I see. for multiple losses, you can add them, right? for clipping it's easier now. could add a param to optimizer to map a function over grads. for different optimizers, just run them. it's lazy and will be fast.

@cHHillee · 2024-12-14 03:05
Adding multiple losses is equivalent numerically but not in terms of memory usage. Your first instinct might be that a compiler can rearrange them but it’s not always so easy. For example, x = mod1(x) y_loss, z_loss = mod2(x), mod3(x) Should you run mod2/y_loss backwards together with mod3/z_loss or separately? If you run it together then you can “batch” the compute for mod1 gradients, which will be more efficient. Otoh, running them together means that you need to keep activations for both mod2 and mod3. A point wise function might not be enough. For example, what if the user wants to detect if any of their gradients are nan and if so, rescale their loss to be smaller? This was very common with fp16 training.

@cHHillee · 2024-12-14 03:05
Adding multiple losses is equivalent numerically but not in terms of memory usage. Your first instinct might be that a compiler can rearrange them but it’s not always so easy. For example, x = mod1(x) y_loss, z_loss = mod2(x), mod3(x) Should you run mod2/y_loss backwards together with mod3/z_loss or separately? If you run it together then you can “batch” the compute for mod1 gradients, which will be more efficient. Otoh, running them together means that you need to keep activations for both mod2 and mod3. A point wise function might not be enough. For example, what if the user wants to detect if any of their gradients are nan and if so, rescale their loss to be smaller? This was very common with fp16 training.

@__tinygrad__ · 2024-12-14 03:08
@cHHillee @ezyang Hmm, I'm leaning toward keeping the PyTorch API (but under the hood using gradient). Do you have any issues with the PyTorch API? Thanks for the feedback btw!

@__tinygrad__ · 2024-12-14 01:37
The new gradient API is merged: weight_grad, bias_grad = loss.gradient(weight, bias) For optimizers (see PR #8231) the new API is: optim.step(loss) You don't need zero_grad or loss.backward. It's not like torch (old API) or JAX (function grad). It's new. This make sense?

@jeremyphoward · 2024-12-14 10:55
@__tinygrad__ Seems less flexible? It’s nice to have the option to not zero grads, such as for memory efficient momentum, grad accumulation, etc

@jeremyphoward · 2024-12-14 10:55
@__tinygrad__ Seems less flexible? It’s nice to have the option to not zero grads, such as for memory efficient momentum, grad accumulation, etc

@__tinygrad__ · 2024-12-14 13:16
@jeremyphoward Yea, after feedback we are going to keep the old API and just use gradient under the hood. https://t.co/5tUGLc6uhd

@UncleSam303 · 2024-12-14 18:21
@__tinygrad__ @AMD @LisaSu And I kept getting worse @LisaSu https://t.co/2RFQaytAOa

@__tinygrad__ · 2024-12-14 19:39
@UncleSam303 @AMD @LisaSu They aren't going to change.

@kierank_ · 2024-12-14 19:42
There needs to be a word for a "build up a ton of software technical debt to ship hardware quickly" company like AMD. Most of the "embedded" industry fits into this category.

@kierank_ · 2024-12-14 19:42
There needs to be a word for a "build up a ton of software technical debt to ship hardware quickly" company like AMD. Most of the "embedded" industry fits into this category.

@kierank_ · 2024-12-14 19:42
@__tinygrad__ is right with their approach of mmio reversing the driver

@HerdingAlpha · 2024-12-14 19:43
This is my $AMD thesis.

@__tinygrad__ · 2024-12-14 19:39
@UncleSam303 @AMD @LisaSu They aren't going to change.

@__tinygrad__ · 2024-12-14 19:48
@UncleSam303 @AMD @LisaSu Here's a recent e-mail exchange. I believe I'm clearly asking for something, but the reply looks like it's to something I'm not asking. Honest reply: No, they are hard to buy. We only sell whole systems, and the prices for you are way more than what Samsung paid. https://t.co/GD3V7hk8Bn

@__tinygrad__ · 2024-12-14 19:48
@UncleSam303 @AMD @LisaSu Here's a recent e-mail exchange. I believe I'm clearly asking for something, but the reply looks like it's to something I'm not asking. Honest reply: No, they are hard to buy. We only sell whole systems, and the prices for you are way more than what Samsung paid. https://t.co/GD3V7hk8Bn

@__tinygrad__ · 2024-12-14 19:50
@UncleSam303 @AMD @LisaSu The true price is $170k for the baseboard btw (which you can buy through distributors), vs $270k for a similar NVIDIA baseboard. Neither are close to competitive with consumer GPUs.

@HerdingAlpha · 2024-12-14 19:43
This is my $AMD thesis.

@__tinygrad__ · 2024-12-14 19:53
@mattforthelikes The worst part is that this is obviously a true thesis, yet it's still unclear if the stock will go up or down. Intel manages to be so much worse and most of the startups in the space are clowns, so AMD might even succeed despite this.

@__tinygrad__ · 2024-12-14 19:58
Are there any up and coming Chinese training chips? There's Huawei Ascend 910C, but I don't think they will be easy to buy. Bitmain offered a chip back in the day (Sophon BM1680), anything else like that?

@kierank_ · 2024-12-14 19:42
@__tinygrad__ is right with their approach of mmio reversing the driver

@__tinygrad__ · 2024-12-14 20:01
@kierank_ Our driver is training ResNet and BERT across 6 GPUs now! Watch the progress on Discord.

@__tinygrad__ · 2024-12-14 20:06
@_gabrielcbrito Here's hoping.

@Yuchenj_UW · 2024-12-14 20:13
@__tinygrad__ There are a couple of others: - Cambricon - mthreads: https://t.co/MR27No1g8o - Biren

@__tinygrad__ · 2024-12-14 21:19
@Yuchenj_UW Cool! Any with a buy it now link? MTT S4000 looks interesting.

@__tinygrad__ · 2024-12-14 21:19
@Yuchenj_UW Cool! Any with a buy it now link? MTT S4000 looks interesting.

@__tinygrad__ · 2024-12-14 21:25
@Yuchenj_UW Woah, Biren BR100 too. Yea, I really want to buy these. Are they real?

@__tinygrad__ · 2024-12-14 21:44
@possiblymartin @Yuchenj_UW That price is insanely high though

@__tinygrad__ · 2024-12-14 21:48
@possiblymartin @Yuchenj_UW oh haha i thought it was yuan. that's a better price, but still not competitive with 7900XTX. does have double the RAM though

@0xIlyy · 2024-12-15 04:11
why does Python make you type out 'lambda'? It's so unnecessary. just call it "fn" or something https://t.co/MtWW38Ocvn

@0xIlyy · 2024-12-15 04:11
why does Python make you type out 'lambda'? It's so unnecessary. just call it "fn" or something https://t.co/MtWW38Ocvn

@__tinygrad__ · 2024-12-15 21:44
@0xIlyy Agreed. This can save us lines.

@__tinygrad__ · 2024-12-14 21:19
@Yuchenj_UW Cool! Any with a buy it now link? MTT S4000 looks interesting.

@hikimiru · 2024-12-15 23:03
@__tinygrad__ @Yuchenj_UW MTT S4000/MTT S90 is a licensed design from @ImaginationTech, likely BXT-32-1024 customized to MC16 (4x more cores), L3, primary-secondary slices design, specialised INT ALUs in 1:4 ratio, and updated tensor cores. Basicaly 2x bigger MTT S80/S3000 with less clock and more memory.

@hikimiru · 2024-12-15 23:03
@__tinygrad__ @Yuchenj_UW MTT S4000/MTT S90 is a licensed design from @ImaginationTech, likely BXT-32-1024 customized to MC16 (4x more cores), L3, primary-secondary slices design, specialised INT ALUs in 1:4 ratio, and updated tensor cores. Basicaly 2x bigger MTT S80/S3000 with less clock and more memory.

@__tinygrad__ · 2024-12-16 04:59
@hikimiru @Yuchenj_UW @ImaginationTech That's the same as Apple, right? You know where I can buy an S4000?

@halvarflake · 2024-12-16 20:22
Honest question: why is it so hard to build a competitive GPU?

@mbhbox · 2024-12-16 20:26
@halvarflake If I was to guess: Game optimization libraries & sdks Drivers Workload-specific chip design & possible intellectual property breach issues AMD is focusing more on its MI3xx chips for AI/ML, rather than the gaming GPU market.

@halvarflake · 2024-12-16 20:28
@mbhbox Is AMDs Hardware competitive?

@matospiso · 2024-12-16 22:51
Advent of ML, Day 16: Implementing TopK SAE (https://t.co/JxYp2Cd61n). As I couldn’t find a way to call topk in tinygrad (not implemented 😞), I wrote a simple version of the network in torch. I must say that compared to tinygrad torch API feels kind of overengineered

@halvarflake · 2024-12-16 20:22
Honest question: why is it so hard to build a competitive GPU?

@__tinygrad__ · 2024-12-16 22:54
@halvarflake It's doable, it just requires a huge market to amortize the tapeout costs and surviving for 3 generations. Our plan is to first get adoption for our GPU agnostic framework, then we can work out the economics of making a chip. It's not hard, it's just expensive and slow.

@halvarflake · 2024-12-16 20:28
@mbhbox Is AMDs Hardware competitive?

@__tinygrad__ · 2024-12-16 22:56
@halvarflake @mbhbox Sort of. The 7900XTX was AMD's closest in a while, but it flopped mostly because of bad software. This was our impetus for starting tiny corp. https://t.co/KOvuw7CFpq

@__tinygrad__ · 2024-12-16 22:54
@halvarflake It's doable, it just requires a huge market to amortize the tapeout costs and surviving for 3 generations. Our plan is to first get adoption for our GPU agnostic framework, then we can work out the economics of making a chip. It's not hard, it's just expensive and slow.

@__tinygrad__ · 2024-12-16 22:58
@halvarflake (surviving for 3 generations cause your first two cards won't be good)

@matospiso · 2024-12-16 22:51
Advent of ML, Day 16: Implementing TopK SAE (https://t.co/JxYp2Cd61n). As I couldn’t find a way to call topk in tinygrad (not implemented 😞), I wrote a simple version of the network in torch. I must say that compared to tinygrad torch API feels kind of overengineered

@__tinygrad__ · 2024-12-16 23:00
@matospiso We'd welcome a PR for sort/topk! topk is easy in O(n*k) time. Would require some cleverness for O(n*log(n)) sort, but it should be doable.

@__tinygrad__ · 2024-12-17 03:54
Err, you sure you can just plug a GPU into a USB port? https://t.co/NhIaKp29O1

@__tinygrad__ · 2024-12-17 03:55
RT @chenyuxyz: mesozoic-egg started with publishing tinygrad tutorials and later did (multiple!) bounties. it feels very good to see the community grow as the project grows! https://t.co/ZKn1r1kYwC https://t.co/fBbmKt0ddb

@__tinygrad__ · 2024-12-17 03:54
Err, you sure you can just plug a GPU into a USB port? https://t.co/NhIaKp29O1

@GRITknox · 2024-12-17 03:57
@__tinygrad__ Occulink something something

@GRITknox · 2024-12-17 03:57
@__tinygrad__ Occulink something something

@__tinygrad__ · 2024-12-17 03:58
@GRITknox Nah, it's USB 3.2

@__tinygrad__ · 2024-12-17 03:58
@GRITknox Nah, it's USB 3.2

@neilmjohnson · 2024-12-17 04:03
@__tinygrad__ @GRITknox to m.2 to laptop riser? lol

@neilmjohnson · 2024-12-17 04:03
@__tinygrad__ @GRITknox to m.2 to laptop riser? lol

@__tinygrad__ · 2024-12-17 04:05
@neilmjohnson @GRITknox If this works, we'll put it in a nice box and sell it.

@__tinygrad__ · 2024-12-17 03:54
Err, you sure you can just plug a GPU into a USB port? https://t.co/NhIaKp29O1

@qsdnl · 2024-12-17 04:10
@__tinygrad__ dont worry i got this https://t.co/bgOMT90zIb

@qsdnl · 2024-12-17 04:10
@__tinygrad__ dont worry i got this https://t.co/bgOMT90zIb

@__tinygrad__ · 2024-12-17 04:18
@qsdnl If you combine that with a VGA to HDMI it might work to connect the GPU!

@__tinygrad__ · 2024-12-16 22:58
@halvarflake (surviving for 3 generations cause your first two cards won't be good)

@domenuk · 2024-12-17 14:23
@__tinygrad__ @halvarflake Why not skip the first two gens? 🤔

@__tinygrad__ · 2024-12-17 15:22
RT @dhbrojas: The @__tinygrad__ website's nemesis has to be the @LambdaAPI website. 95% of buttons are "Contact Sales" or "Get Pricing". Each page is just an endless scroll over marketing slogans.

@dhbrojas · 2024-12-17 15:21
The final straw is this section of the "1-Click Cluster" product page where they kindly inform you the kind of models you can run on a GPU cluster. In case you didn't know. https://t.co/oFdjlbvLf4

@dhbrojas · 2024-12-17 15:25
It's sad, cause their cloud is actually okay. I've been meaning to buy a GPU workstation from them for a while now but it seems they're stacking a fat margin on top of NVIDIA's already inflated prices.

@dhbrojas · 2024-12-17 15:25
It's sad, cause their cloud is actually okay. I've been meaning to buy a GPU workstation from them for a while now but it seems they're stacking a fat margin on top of NVIDIA's already inflated prices.

@__tinygrad__ · 2024-12-17 16:41
@dhbrojas Our margins are lower and our website is simple. Buy tiny!

@__tinygrad__ · 2024-12-17 17:03
We have a $1,000 bounty for getting tinychat running in the browser. The frontend is ready, the WEBGPU backend is ready, just wire them up!

@__tinygrad__ · 2024-12-17 22:02
RT @demi_network: "It's not a question of alignment between the company and AI, it's a question of alignment between the company and you. It’s going to be very important who your AI works for. If your AI is in the cloud, it doesn’t work for you. But if you can kick it, then it works for you." - @realGeorgeHotz talk on Commoditizing the Petaflop with @__tinygrad__ at the Democratize Intelligence Summit, June 2024

@domenuk · 2024-12-17 14:23
@__tinygrad__ @halvarflake Why not skip the first two gens? 🤔

@__tinygrad__ · 2024-12-17 22:59
@domenuk @halvarflake Then your 3rd gen is terrible. Even worse, you thought it would be good so you spent huge money for leading edge process.

@__tinygrad__ · 2024-12-18 04:27
first customer tinybox pros, should ship friday! https://t.co/xtjVDVCeaW

@__tinygrad__ · 2024-12-18 04:27
first customer tinybox pros, should ship friday! https://t.co/xtjVDVCeaW

@beffjezos · 2024-12-18 04:30
@__tinygrad__ What's in the box?

@beffjezos · 2024-12-18 04:30
@__tinygrad__ What's in the box?

@__tinygrad__ · 2024-12-18 04:31
@beffjezos I'm doing bring up right now. Will add screenshots to thread 😀

@__tinygrad__ · 2024-12-18 04:27
first customer tinybox pros, should ship friday! https://t.co/xtjVDVCeaW

@Yuchenj_UW · 2024-12-18 04:34
@__tinygrad__ beautiful, 8 x4090

@__tinygrad__ · 2024-12-18 04:56
It needed media boost. I got bored waiting for it to transfer gigabytes. Now trying the "mini iso" https://t.co/wuJsEgG7F6

@__tinygrad__ · 2024-12-18 04:59
64 Cores, 384 GB RAM https://t.co/iqcwwtd34d

@__tinygrad__ · 2024-12-18 04:27
first customer tinybox pros, should ship friday! https://t.co/xtjVDVCeaW

@0imalan · 2024-12-18 05:05
@__tinygrad__ Is that a 2x 4U?

@__tinygrad__ · 2024-12-18 04:59
64 Cores, 384 GB RAM https://t.co/iqcwwtd34d

@__tinygrad__ · 2024-12-18 05:06
mini-iso is much better. When buyers ask why they have 24.04, we can point them to the X poll https://t.co/JSouq5bd1K

@0imalan · 2024-12-18 05:05
@__tinygrad__ Is that a 2x 4U?

@__tinygrad__ · 2024-12-18 05:08
@0imalan Each machine is 4U

@Yuchenj_UW · 2024-12-18 04:34
@__tinygrad__ beautiful, 8 x4090

@__tinygrad__ · 2024-12-18 05:08
@Yuchenj_UW Screenshots for that soon!

@__tinygrad__ · 2024-12-18 05:18
@timzaman You're assuming I have a precooked install image. We have for tinybox, but not tinybox pro, and an intern built all the provisioning stuff for tinybox!

@__tinygrad__ · 2024-12-18 05:19
@timzaman That's cool though, I can point iPXE to HTTP? How does it know the URL? I see a prebaked iso on the site.

@__tinygrad__ · 2024-12-18 05:19
@timzaman That's cool though, I can point iPXE to HTTP? How does it know the URL? I see a prebaked iso on the site.

@__tinygrad__ · 2024-12-18 05:20
@timzaman My current plan is to set one up and just clone the NVMe drives.

@__tinygrad__ · 2024-12-18 05:19
@timzaman That's cool though, I can point iPXE to HTTP? How does it know the URL? I see a prebaked iso on the site.

@__tinygrad__ · 2024-12-18 05:20
@timzaman My current plan is to set one up and just clone the NVMe drives.

@__tinygrad__ · 2024-12-18 05:20
@timzaman My current plan is to set one up and just clone the NVMe drives.

@__tinygrad__ · 2024-12-18 06:02
@timzaman big brain pxe boot. smooth brain nvme duplicator https://t.co/dEA5Gs96F1

@__tinygrad__ · 2024-12-18 06:32
Trying modded-nanogpt by @kellerjordan0, based on llm.c by @karpathy. After installing some bleeding edge PyTorch...I think it's running. Will probably OOM and need tweaking https://t.co/u1zuKHwt2w

@__tinygrad__ · 2024-12-18 06:38
@kellerjordan0 @karpathy Wow and people complain about tinygrad BEAM speed...at least that has a progress indicator. I know torch is compiling only cause of htop

@__tinygrad__ · 2024-12-18 08:02
Without any optimization, 16.4 minutes to train Modded-NanoGPT on a tinybox pro. $300k = 8xH100 = 3.8 minutes $40k = tinybox pro = 16.4 minutes ~50% cheaper LLM training with tiny https://t.co/lP9T7S0722

@__tinygrad__ · 2024-12-18 08:02
Without any optimization, 16.4 minutes to train Modded-NanoGPT on a tinybox pro. $300k = 8xH100 = 3.8 minutes $40k = tinybox pro = 16.4 minutes ~50% cheaper LLM training with tiny https://t.co/lP9T7S0722

@MRisso · 2024-12-18 08:05
@__tinygrad__ What are the comparison with other frameworks on same machine?

@MRisso · 2024-12-18 08:05
@__tinygrad__ What are the comparison with other frameworks on same machine?

@__tinygrad__ · 2024-12-18 08:06
@MRisso If you post a good benchmark I'll run it.

@__tinygrad__ · 2024-12-18 06:38
@kellerjordan0 @karpathy Wow and people complain about tinygrad BEAM speed...at least that has a progress indicator. I know torch is compiling only cause of htop

@cHHillee · 2024-12-18 08:23
@__tinygrad__ @kellerjordan0 @karpathy They turn on some specific options for "maximum autotuning" to squeeze out a couple more seconds - they only lose a couple of seconds otherwise and it compiles much faster (15 seconds iirc)

@__tinygrad__ · 2024-12-18 08:26
@PytorchToAtoms Done https://t.co/ezdL5LudBV

@cHHillee · 2024-12-18 08:23
@__tinygrad__ @kellerjordan0 @karpathy They turn on some specific options for "maximum autotuning" to squeeze out a couple more seconds - they only lose a couple of seconds otherwise and it compiles much faster (15 seconds iirc)

@__tinygrad__ · 2024-12-18 08:29
@cHHillee @kellerjordan0 @karpathy Oh is that what "coordinate_descent_tuning" is?

@__tinygrad__ · 2024-12-18 08:35
@PytorchToAtoms All passive, but rated for PCIe5 (MCIO, not SlimSAS)

@__tinygrad__ · 2024-12-18 08:02
Without any optimization, 16.4 minutes to train Modded-NanoGPT on a tinybox pro. $300k = 8xH100 = 3.8 minutes $40k = tinybox pro = 16.4 minutes ~50% cheaper LLM training with tiny https://t.co/lP9T7S0722

@signalise_ · 2024-12-18 08:40
@__tinygrad__ You ship to Australia?

@__tinygrad__ · 2024-12-18 06:38
@kellerjordan0 @karpathy Wow and people complain about tinygrad BEAM speed...at least that has a progress indicator. I know torch is compiling only cause of htop

@__tinygrad__ · 2024-12-18 16:35
The result. Found some diff to make it work for 4090. Even with only using 18GB of RAM it's still very worth it FLOPS/$. https://t.co/DjU0AHbUvR

@signalise_ · 2024-12-18 08:40
@__tinygrad__ You ship to Australia?

@__tinygrad__ · 2024-12-18 16:42
@signalise_ Yes. There's a country dropdown on our checkout page.

@__tinygrad__ · 2024-12-18 18:59
a peek inside a tinybox pro https://t.co/DPylaDKaaC

@wordgrammer · 2024-12-18 23:15
Writing a really good compiler/driver for AMD, and then only once it really works, building your own chip around it is probably the best way for a startup to compete with Nvidia. If you start by building the chip, you care about the aesthetic of “deep tech” more than the reality

@__tinygrad__ · 2024-12-19 05:39
You are 18 months late with this plan.

@__tinygrad__ · 2024-12-19 05:39
You are 18 months late with this plan.

@__tinygrad__ · 2024-12-19 05:45
Whole plan is laid out in this blog post. We are the only company to get AMD on MLPerf, and we have a completely custom driver that's 50x simpler than the stock one. A bit shocked by how little AMD cared, but we'll take the trillions instead of them. https://t.co/KOvuw7CFpq

@__tinygrad__ · 2024-12-19 19:37
tinybox pro requires 200V+ power. Over 1000W coming from each of the 4 PSUs while running gpu-burn. https://t.co/1O3vx0P8of

@yacineMTB · 2024-12-19 19:38
i've built my own ML server and i'm really happy i did but at this point i'd just buy a tinybox

@__tinygrad__ · 2024-12-19 19:44
RT @chenyuxyz: tiny corp is hiring! invest with your PRs!

@__tinygrad__ · 2024-12-19 19:50
We have 70x used water cooled 4090s for sale! Will do $1650 each if you want 10+ (support@tinygrad.org), otherwise buy on eBay. https://t.co/9PIT9JnlPt https://t.co/VpGP3kgnQM

@yacineMTB · 2024-12-19 19:38
i've built my own ML server and i'm really happy i did but at this point i'd just buy a tinybox

@__tinygrad__ · 2024-12-19 19:52
@yacineMTB If you are building as a hobby, build. If you are building to do ML, buy tinybox. Our prices are great compared to the headaches you'll experience.

@__tinygrad__ · 2024-12-19 19:54
RT @yacineMTB: you would be insane to rent a server instead of just getting a tinybox

@__tinygrad__ · 2024-12-19 20:04
@xyz839248195947 @AMD Cancelling their big GPU. AMD saw an opportunity to make big money and was like...nah, that's not for us. We prefer to sell mediocre midrange GPUs at low margins.

@__tinygrad__ · 2024-12-19 20:09
@ZlpdmZdbrg You can buy new for $3,641 if you want. https://t.co/tJqigtAmID

@__tinygrad__ · 2024-12-19 19:50
We have 70x used water cooled 4090s for sale! Will do $1650 each if you want 10+ (support@tinygrad.org), otherwise buy on eBay. https://t.co/9PIT9JnlPt https://t.co/VpGP3kgnQM

@zhentan · 2024-12-19 20:37
@__tinygrad__ Why not sell them with/in the tiny boxes?

@zhentan · 2024-12-19 20:37
@__tinygrad__ Why not sell them with/in the tiny boxes?

@__tinygrad__ · 2024-12-19 21:47
@zhentan Cause they are water cooled and don't fit.

@__tinygrad__ · 2024-12-19 22:18
@Kiaman121564x @realGeorgeHotz @imprashantrai1 @AMD Broadcom is a real company, and many of the chips they make are real. You can tell they are real because they have buy it now buttons. If a chip doesn't have a buy it now button, it is vaporware. Marketing people can't make chip, only webpage.

@__tinygrad__ · 2024-12-19 22:18
@Kiaman121564x @realGeorgeHotz @imprashantrai1 @AMD Broadcom is a real company, and many of the chips they make are real. You can tell they are real because they have buy it now buttons. If a chip doesn't have a buy it now button, it is vaporware. Marketing people can't make chip, only webpage.

@__tinygrad__ · 2024-12-19 22:20
@Kiaman121564x @realGeorgeHotz @imprashantrai1 @AMD To be fair, they do make the Google TPU, so that's a point in their favor. I will believe chip is real when it has a buy button.

@__tinygrad__ · 2024-12-19 22:22
@Kiaman121564x @realGeorgeHotz @imprashantrai1 @AMD Oh I see, it's secret chip in secret box with magic number and not chip. But very real, we promise. No need to look in box, I promise there's chip in there. In box. Yes. Chip.

@__tinygrad__ · 2024-12-19 22:22
@Kiaman121564x @realGeorgeHotz @imprashantrai1 @AMD Oh I see, it's secret chip in secret box with magic number and not chip. But very real, we promise. No need to look in box, I promise there's chip in there. In box. Yes. Chip.

@__tinygrad__ · 2024-12-19 22:23
@Kiaman121564x @realGeorgeHotz @imprashantrai1 @AMD Can I interest you in some cloud mining contract? I promise we have chip. In cloud. Yes. Chip in cloud. Billions.

@__tinygrad__ · 2024-12-19 22:39
tinybox pro ordered today ship Q1 2025 tinybox ships right away! https://t.co/WyVXf91fkA

@__tinygrad__ · 2024-12-19 23:13
@Kiaman121564x @realGeorgeHotz @imprashantrai1 @AMD Have you seen chip?

@__tinygrad__ · 2024-12-20 01:48
@Kiaman121564x @realGeorgeHotz @imprashantrai1 @AMD That's not what I ask. I ask "Have you seen chip?" That is picture of chip.

@__tinygrad__ · 2024-12-21 02:22
Deploying tiny at the customer facility. https://t.co/QsQpMfzvAY

@__tinygrad__ · 2024-12-21 02:22
Deploying tiny at the customer facility. https://t.co/QsQpMfzvAY

@Vdawger · 2024-12-21 02:51
@__tinygrad__ that's quite a few tinys

@__tinygrad__ · 2024-12-21 02:22
Deploying tiny at the customer facility. https://t.co/QsQpMfzvAY

@shlomiatar · 2024-12-21 02:51
@__tinygrad__ God the amount of power each of this racks draws. Amazing

@Vdawger · 2024-12-21 02:51
@__tinygrad__ that's quite a few tinys

@__tinygrad__ · 2024-12-21 04:07
@Vdawger tinys love being networked

@shlomiatar · 2024-12-21 02:51
@__tinygrad__ God the amount of power each of this racks draws. Amazing

@__tinygrad__ · 2024-12-21 04:08
@shlomiatar ~40kW

@The_AI_Investor · 2024-12-21 22:21
"Even when the competitor's chips are offered for free, it's still not cheap enough." - Jensen H $NVDA https://t.co/iZ7WjYWpkQ

@__tinygrad__ · 2024-12-23 04:34
RT @mrm8488: Tinybox (from @__tinygrad__ ) at home! The kitchen is on fire at @maisaAI_ https://t.co/BUK6suwVGL

@__tinygrad__ · 2024-12-23 04:59
RT @gregjhogan: 27 tinybox pro servers up and running! 216 GPUs doing some heavy weight lifting :) https://t.co/nVnOb2AYKy

@LottoLabs · 2024-12-25 03:36
It’s been decided I am buy a @__tinygrad__ green in 2025

@__tinygrad__ · 2024-12-25 16:52
We are out of stock of green right now, with no ETA due to the price of 4090s. Might I interest you in a red? We have a lot of red. We also had a tinybox pro preorder decline to purchase, so we have 1 available to ship right away.

@__tinygrad__ · 2024-12-25 18:21
RT @chenyuxyz: tinybox holiday edition https://t.co/Pyi3Nxvstb

@__tinygrad__ · 2024-12-25 16:52
We are out of stock of green right now, with no ETA due to the price of 4090s. Might I interest you in a red? We have a lot of red. We also had a tinybox pro preorder decline to purchase, so we have 1 available to ship right away.

@LottoLabs · 2024-12-25 18:41
@__tinygrad__ Gonna have to deep dive on amd, would love a pro but outside of the basement computer budget.

@LottoLabs · 2024-12-25 18:41
@__tinygrad__ Gonna have to deep dive on amd, would love a pro but outside of the basement computer budget.

@__tinygrad__ · 2024-12-25 19:25
@LottoLabs If you are sticking to tinygrad it's not bad, we're using reds in our cloud. You won't be able to use our driver with PyTorch though.

@__tinygrad__ · 2024-12-26 02:37
RT @billy_agi: Santa was extra generous this year! 🥰 @__tinygrad__ https://t.co/rkFL0zwD23

@__tinygrad__ · 2024-12-26 21:20
https://t.co/Ind0maQlN4

@NaanLeCun · 2024-12-27 10:56
@Unsupervisedlo2 @romechenko @__tinygrad__ Yes but runtime uses cuda. You really can’t avoid it unless you use a much worse opencl. https://t.co/FP6JMXdD3H

@__tinygrad__ · 2024-12-26 21:20
https://t.co/Ind0maQlN4

@cafeinomano__ · 2024-12-27 17:13
@__tinygrad__ The framework is great. The design and philosophy are great. But I do not understand why if the objective is to facilitate deep learning development it does not allow boolean tensor indexing..

@cafeinomano__ · 2024-12-27 17:13
@__tinygrad__ The framework is great. The design and philosophy are great. But I do not understand why if the objective is to facilitate deep learning development it does not allow boolean tensor indexing..

@__tinygrad__ · 2024-12-27 18:25
@cafeinomano__ The issue is that the output shape is variable. This is the only type of indexing we don't currently support, and it is tricky because we allocate all the memory in advance. We should make this work before 1.0 though, it will just allocate the max size and return a view.

@NaanLeCun · 2024-12-27 10:56
@Unsupervisedlo2 @romechenko @__tinygrad__ Yes but runtime uses cuda. You really can’t avoid it unless you use a much worse opencl. https://t.co/FP6JMXdD3H

@__tinygrad__ · 2024-12-27 18:26
@NaanLeCun @Unsupervisedlo2 @romechenko The NV runtime does not use CUDA, it speaks directly to the kernel driver and GPU.

@__tinygrad__ · 2024-12-28 15:32
RT @k7agar: the tinygrad source code is such a joy to work with

@__tinygrad__ · 2024-12-28 17:52
Great software isn't invented, it's discovered.

@__tinygrad__ · 2024-12-30 16:41
The skill ceiling on software is absurdly high. When tinygrad is 1.0 and starts to gain real adoption, you'll wonder why we have been using terrible software for so long.

@__tinygrad__ · 2024-12-30 16:41
The skill ceiling on software is absurdly high. When tinygrad is 1.0 and starts to gain real adoption, you'll wonder why we have been using terrible software for so long.

@iharabukhouski · 2024-12-30 16:49
@__tinygrad__ honestly feels like we need a doge like initiative for our software stacks broadly

@__tinygrad__ · 2024-12-30 16:41
The skill ceiling on software is absurdly high. When tinygrad is 1.0 and starts to gain real adoption, you'll wonder why we have been using terrible software for so long.

@RuiCarrilho5 · 2024-12-30 17:08
@__tinygrad__ How far do you think we are from now? Think the line counter will balloon a bit more? Or shrink?

@RuiCarrilho5 · 2024-12-30 17:08
@__tinygrad__ How far do you think we are from now? Think the line counter will balloon a bit more? Or shrink?

@__tinygrad__ · 2024-12-30 17:11
@RuiCarrilho5 In order to fit assembly language and user space drivers it will probably have to increase a bit. Other than Python (which we accept as fine) those are the last two external pieces of code tinygrad relies on, the goal is 0.

@iharabukhouski · 2024-12-30 16:49
@__tinygrad__ honestly feels like we need a doge like initiative for our software stacks broadly

@__tinygrad__ · 2024-12-30 17:18
@iharabukhouski Yea. The problem isn't usually an individual piece of software. Given the tradeoffs they made, PyTorch, LLVM, and the Linux Kernel are all pretty good individually. It's the massive spidering of dependencies, and that's what needs to be rethought.

@__tinygrad__ · 2024-12-30 16:41
The skill ceiling on software is absurdly high. When tinygrad is 1.0 and starts to gain real adoption, you'll wonder why we have been using terrible software for so long.

@gclawes · 2024-12-30 17:22
@__tinygrad__ Any news on @tenstorrent support before (or after) 1.0?

@gclawes · 2024-12-30 17:22
@__tinygrad__ Any news on @tenstorrent support before (or after) 1.0?

@__tinygrad__ · 2024-12-30 17:24
@gclawes @tenstorrent For us to put effort into @tenstorrent they either need to release hardware that's better than 5090s or sponsor a port.

@Yuchenj_UW · 2024-12-30 16:56
Simplicity is beauty, but to achieve good performance you often need to write custom CUDA kernels, call into third-party libraries like CUTLASS, or auto-tune kernels as we did in TVM. In this AI framework is gradually becoming a compiler context, I’m curious what aspects of PyTorch do you think are done wrong.

@diegoasua · 2024-12-30 17:54
@Yuchenj_UW @realGeorgeHotz very skeptical tinygrad will ever be as fast as pytorch great hobby project tho

@Yuchenj_UW · 2024-12-30 16:56
Simplicity is beauty, but to achieve good performance you often need to write custom CUDA kernels, call into third-party libraries like CUTLASS, or auto-tune kernels as we did in TVM. In this AI framework is gradually becoming a compiler context, I’m curious what aspects of PyTorch do you think are done wrong.

@diegoasua · 2024-12-30 17:54
@Yuchenj_UW @realGeorgeHotz very skeptical tinygrad will ever be as fast as pytorch great hobby project tho

@__tinygrad__ · 2024-12-30 22:44
Challenge accepted! We are already faster on AMD.

@__tinygrad__ · 2024-12-30 22:44
Challenge accepted! We are already faster on AMD.

@__tinygrad__ · 2024-12-30 22:45
Ask yourself how much you believe the bitter lesson. All of the choices to convert a graph to machine code will be searchable. https://t.co/FM1INp8Yqn

@test_tm7873 · 2024-12-30 22:50
@__tinygrad__ tinybox blue when

@deftdawg · 2024-12-31 02:48
@test_tm7873 @__tinygrad__ When they put >= 24GB of vram on a card? 🤷

@deftdawg · 2024-12-31 02:48
@test_tm7873 @__tinygrad__ When they put >= 24GB of vram on a card? 🤷

@__tinygrad__ · 2024-12-31 03:31
@deftdawg @test_tm7873 It's not the VRAM size, it's the VRAM bandwidth. It's so sad that probably only NVIDIA will be good next generation. You need a big die for all the memory PHYs

@__tinygrad__ · 2025-01-02 00:55
Just bought an XCVU33P card. As we push for speed this year, it's probably worth starting to think on the FPGA level and deeply understanding the uarch tradeoffs made in GPUs.

@__tinygrad__ · 2025-01-02 00:55
Just bought an XCVU33P card. As we push for speed this year, it's probably worth starting to think on the FPGA level and deeply understanding the uarch tradeoffs made in GPUs.

@corsix · 2025-01-02 09:07
@__tinygrad__ I’ll laugh if this turns out to be the new basis of tinybox red.

@corsix · 2025-01-02 09:07
@__tinygrad__ I’ll laugh if this turns out to be the new basis of tinybox red.

@__tinygrad__ · 2025-01-02 15:22
@corsix I wish they were competitive with GPUs. FPGAs don't have enough FLOPS, even you are cranking all the DSP blocks on that chip it's only a couple TFLOPS. If we get a design we like on the FPGA, we'll tape out a chip and sell some hobbyist cards. Then if that's good, we do big chip

@__tinygrad__ · 2025-01-02 21:07
mesozoic-egg wrote up some nice docs explaining the tinygrad JIT https://t.co/EeZj8Sciys

@__tinygrad__ · 2025-01-03 15:37
tiny corp will be at @CES in the @comma_ai booth. We will have a tinybox red on display for you to covet. Booth #6475 in the LVCC West Hall

@__tinygrad__ · 2025-01-03 15:47
MLX vs tinygrad on ResNet18, both have gotten faster https://t.co/JHyDwoPduy

@__tinygrad__ · 2025-01-07 03:23
RT @reneil1337: Picked up my #tinybox to fuel those agents. Getting one of these was a goal since @realGeorgeHotz announced it. 144gb VRAM via 6x4090 🤘 Our agents are gonna breath life into those sim worlds this year is gonna be exciting 🕳️🐇 @ai16zdao @hyperfy_io @MuseumofCrypto https://t.co/43hOhJU8qO

@nvidia · 2025-01-07 04:20
Announcing NVIDIA Project DIGITS, a personal AI supercomputer that’s powered by the NVIDIA GB10 Superchip and based on #NVIDIAGraceBlackwell architecture. https://t.co/zY09rRrY0X Preconfigured with the NVIDIA AI software stack, developers, researchers, data scientists and students can prototype, fine-tune and inference large AI models on their desktop and deploy them to the data center or cloud. #CES2025

@unusual_whales · 2025-01-07 04:21
BREAKING: Nvidia, $NVDA, announces Project Digits personal computer at $3000, that is approximately 1,000 times more powerful than the average laptop. The device is powered by an Nvidia GB10 Grace Blackwell Superchip, which houses separate, linked components on a single chip to reduce the time it takes to move data between them. The superchip features an Nvidia Blackwell graphics card and an Nvidia Grace processor, packaged with 128 gigabytes of memory and 4 terabytes of SSD storage.

@unusual_whales · 2025-01-07 04:21
BREAKING: Nvidia, $NVDA, announces Project Digits personal computer at $3000, that is approximately 1,000 times more powerful than the average laptop. The device is powered by an Nvidia GB10 Grace Blackwell Superchip, which houses separate, linked components on a single chip to reduce the time it takes to move data between them. The superchip features an Nvidia Blackwell graphics card and an Nvidia Grace processor, packaged with 128 gigabytes of memory and 4 terabytes of SSD storage.

@RobbieJ377 · 2025-01-07 04:52
@unusual_whales Wow. A petaflop for $3k. Unbelievable. tinybox gonna have to adjust their pricing way down soon.

@RobbieJ377 · 2025-01-07 04:52
@unusual_whales Wow. A petaflop for $3k. Unbelievable. tinybox gonna have to adjust their pricing way down soon.

@__tinygrad__ · 2025-01-07 14:47
@RobbieJ377 @unusual_whales "FP4 AI compute" <-- FAKE! tinybox measures in FP16, divide by 4. Also, is NVIDIA pulling a sparsity scam here? It's either 250 TFLOPS or 125 TFLOPS. Stop falling for hype.

@__tinygrad__ · 2025-01-07 14:50
NVIDIA cards are powerful, but can we stop with the AI TOPS thing? I'll believe when I see models trained with FP4, but for now can we all report BF16 TFLOPS instead? (and no sparsity!)

@__tinygrad__ · 2025-01-07 14:52
This is marketing. FP4 is unusable, it's 500 TFLOPS of FP8. tinybox green has 4 PFLOPS of FP8, 8x more powerful.

@__tinygrad__ · 2025-01-07 14:50
NVIDIA cards are powerful, but can we stop with the AI TOPS thing? I'll believe when I see models trained with FP4, but for now can we all report BF16 TFLOPS instead? (and no sparsity!)

@Yuchenj_UW · 2025-01-07 14:53
@__tinygrad__ Yeah, I don’t want them to compare apples to oranges. Deepseek v3 was trained with FP8 mixed precision though, no one trained with FP4 yet.

@Yuchenj_UW · 2025-01-07 14:53
@__tinygrad__ Yeah, I don’t want them to compare apples to oranges. Deepseek v3 was trained with FP8 mixed precision though, no one trained with FP4 yet.

@__tinygrad__ · 2025-01-07 14:54
@Yuchenj_UW Yea, FP8 (no sparsity) is fine. By that metric, their $3,000 AI computer underperforms a single 4090.

@__tinygrad__ · 2025-01-07 14:52
This is marketing. FP4 is unusable, it's 500 TFLOPS of FP8. tinybox green has 4 PFLOPS of FP8, 8x more powerful.

@strategos314 · 2025-01-07 14:54
@__tinygrad__ Tiny box is also 10x more expensive

@strategos314 · 2025-01-07 14:54
@__tinygrad__ Tiny box is also 10x more expensive

@__tinygrad__ · 2025-01-07 14:57
@spacehotel321 I mean...yea, that's how it works. But compared to this, you can get more perf buying a gaming PC with a single 4090.

@__tinygrad__ · 2025-01-07 14:52
This is marketing. FP4 is unusable, it's 500 TFLOPS of FP8. tinybox green has 4 PFLOPS of FP8, 8x more powerful.

@StephanSturges · 2025-01-07 14:58
@__tinygrad__ 128Gb of VRAM is pretty sweet though...

@StephanSturges · 2025-01-07 14:58
@__tinygrad__ 128Gb of VRAM is pretty sweet though...

@__tinygrad__ · 2025-01-07 15:00
@StephanSturges Is it VRAM? It says it's DDR5X

@__tinygrad__ · 2025-01-07 14:54
@Yuchenj_UW Yea, FP8 (no sparsity) is fine. By that metric, their $3,000 AI computer underperforms a single 4090.

@mohbibi_ · 2025-01-07 15:01
@__tinygrad__ @Yuchenj_UW right but saying nvidia digits underperforms a single 4090 is pretty wild

@mohbibi_ · 2025-01-07 15:01
@__tinygrad__ @Yuchenj_UW right but saying nvidia digits underperforms a single 4090 is pretty wild

@__tinygrad__ · 2025-01-07 15:03
@moh1xabc @Yuchenj_UW It does! And even FP8 training is super hard, really you need BF16 for things to just work.

@__tinygrad__ · 2025-01-07 15:03
@pinchthaddeus protip: if it's in a gold box, it's hype.

@__tinygrad__ · 2025-01-07 14:57
@spacehotel321 I mean...yea, that's how it works. But compared to this, you can get more perf buying a gaming PC with a single 4090.

@strategos314 · 2025-01-07 15:07
@__tinygrad__ Good point, however Project digits has 128gb vram.

@strategos314 · 2025-01-07 15:07
@__tinygrad__ Good point, however Project digits has 128gb vram.

@__tinygrad__ · 2025-01-07 15:08
@spacehotel321 No, it has 128gb of DDR5X ram. Why are you adding the v?

@__tinygrad__ · 2025-01-07 15:13
People are begging to get swindled by a $3,000 box that says AI on the side. We always get asked if we'll launch something at that price point. We won't. Just buy a gaming PC.

@__tinygrad__ · 2025-01-07 15:13
People are begging to get swindled by a $3,000 box that says AI on the side. We always get asked if we'll launch something at that price point. We won't. Just buy a gaming PC.

@xlr8harder · 2025-01-07 15:19
@__tinygrad__ 128 gb of integrated ram doesnt suck tho

@xlr8harder · 2025-01-07 15:19
@__tinygrad__ 128 gb of integrated ram doesnt suck tho

@__tinygrad__ · 2025-01-07 15:20
@xlr8harder I mean, what's the bandwidth of that RAM? I can buy 128GB of RAM for $250. https://t.co/3QlMalspYS

@__tinygrad__ · 2025-01-07 15:20
@xlr8harder I mean, what's the bandwidth of that RAM? I can buy 128GB of RAM for $250. https://t.co/3QlMalspYS

@xlr8harder · 2025-01-07 15:22
@__tinygrad__ I think you're being intentionally obtuse here! there is real value for enthusiasts to have 128Gb of vram, even if the performance isn't next level. there is a niche here that is currently only filled by apple, at an even higher premium. it's a smart move.

@xlr8harder · 2025-01-07 15:22
@__tinygrad__ I think you're being intentionally obtuse here! there is real value for enthusiasts to have 128Gb of vram, even if the performance isn't next level. there is a niche here that is currently only filled by apple, at an even higher premium. it's a smart move.

@__tinygrad__ · 2025-01-07 15:24
@xlr8harder I'm not. Why are you calling it VRAM? It's LPDDR5X. If the bandwidth is 500 GB/s+ sure, I can see this hitting a point in the market (like Mac Studio). But I haven't seen the bandwidth numbers yet.

@__tinygrad__ · 2025-01-07 15:25
RT @RajaXg: Divide flops by 4 and multiply dollars by 2 A CES (20+25) ² tip for staying grounded

@__tinygrad__ · 2025-01-08 06:06
RT @chenyuxyz: https://t.co/PpygZofxYd

@__tinygrad__ · 2025-01-09 14:19
RT @danielmerja: Reading through PyTorch code and now I can see why George Hotz wants to build tinygrad

@__tinygrad__ · 2025-01-09 14:21
RT @FFmpeg: Cool to see @__tinygrad__ @comma_ai at #CES2025 Both are open source projects that take low level programming and high quality code as seriously, if not more seriously than FFmpeg https://t.co/kfjUrxcYWD

@__tinygrad__ · 2025-01-10 22:34
We have one tinybox pro ready to ship today! It's yours for $40k, place the deposit on the website to reserve. Be quick if you want it. If I understand how NVIDIA is measuring it, it has 16000 AI TOPS. https://t.co/MDvQCrKzq4

@__tinygrad__ · 2025-01-10 22:34
We have one tinybox pro ready to ship today! It's yours for $40k, place the deposit on the website to reserve. Be quick if you want it. If I understand how NVIDIA is measuring it, it has 16000 AI TOPS. https://t.co/MDvQCrKzq4

@ramonpiano_ · 2025-01-10 22:35
@__tinygrad__ any 50% off codes?

@ramonpiano_ · 2025-01-10 22:35
@__tinygrad__ any 50% off codes?

@__tinygrad__ · 2025-01-10 22:36
@pianolabs_ai Maybe we could sell pictures of a tinybox pro for 50% off. https://t.co/CIB6ySByFH

@__tinygrad__ · 2025-01-10 22:37
@xyz839248195947 I don't think NVIDIA wants it, it doesn't come in gold color

@__tinygrad__ · 2025-01-10 22:34
We have one tinybox pro ready to ship today! It's yours for $40k, place the deposit on the website to reserve. Be quick if you want it. If I understand how NVIDIA is measuring it, it has 16000 AI TOPS. https://t.co/MDvQCrKzq4

@it0from1bit · 2025-01-10 22:40
@__tinygrad__ Are tinybox air cooled?

@it0from1bit · 2025-01-10 22:40
@__tinygrad__ Are tinybox air cooled?

@__tinygrad__ · 2025-01-10 22:41
@it0from1bit Yes, all of them are. We managed a couple racks of water cooled machines once, awful experience. Air is much lower maintenance.

@__tinygrad__ · 2025-01-10 22:41
@it0from1bit Yes, all of them are. We managed a couple racks of water cooled machines once, awful experience. Air is much lower maintenance.

@it0from1bit · 2025-01-10 22:43
@__tinygrad__ Just out of curiosity, was there a noticable performance gain with liquid cooling?

@__tinygrad__ · 2025-01-10 22:34
We have one tinybox pro ready to ship today! It's yours for $40k, place the deposit on the website to reserve. Be quick if you want it. If I understand how NVIDIA is measuring it, it has 16000 AI TOPS. https://t.co/MDvQCrKzq4

@0000CCS · 2025-01-10 22:43
@__tinygrad__ When 5090s come out, will 4090s still be available for "cheaper" option? Cheaper may be the wrong word here, maybe discounted or mid tier or.. lol

@it0from1bit · 2025-01-10 22:43
@__tinygrad__ Just out of curiosity, was there a noticable performance gain with liquid cooling?

@__tinygrad__ · 2025-01-10 22:44
@it0from1bit Not at all. Peak temp at full power on a tinybox pro is 71C. Those 3 fans are really good.

@__tinygrad__ · 2025-01-10 22:34
We have one tinybox pro ready to ship today! It's yours for $40k, place the deposit on the website to reserve. Be quick if you want it. If I understand how NVIDIA is measuring it, it has 16000 AI TOPS. https://t.co/MDvQCrKzq4

@RuiCarrilho5 · 2025-01-10 22:49
@__tinygrad__ It bears asking if you have any plans for a cheaper model. Those of us in countries with low purchasing power parity would have to work for one/two years to pay for a single box. I know you're not a charity, but if there were a cheaper model, my money would be yours already

@0000CCS · 2025-01-10 22:43
@__tinygrad__ When 5090s come out, will 4090s still be available for "cheaper" option? Cheaper may be the wrong word here, maybe discounted or mid tier or.. lol

@__tinygrad__ · 2025-01-10 22:54
@0000CCS 4090s are still very expensive on eBay. Maybe eventually, but not anytime soon. I'll believe in 5090s when they have a buy it now link, an arch pdf from NVIDIA, and some benchmarks.

@RuiCarrilho5 · 2025-01-10 22:49
@__tinygrad__ It bears asking if you have any plans for a cheaper model. Those of us in countries with low purchasing power parity would have to work for one/two years to pay for a single box. I know you're not a charity, but if there were a cheaper model, my money would be yours already

@__tinygrad__ · 2025-01-10 22:57
@RuiCarrilho5 https://t.co/oF7VDdbOjj

@AMDGPU_ · 2025-01-11 23:15
RX 9070 XT price leak 7900XTX/4080S perf for $530 Game over Nvidia https://t.co/1Xlu0kfksw

@__tinygrad__ · 2025-01-12 17:33
@AMDGPU_ And the 5070 has the same perf as the 4090 /s

@Michi07f · 2025-01-12 18:00
@__tinygrad__ @AMDGPU_ Will the tinybox red be discontinued if the 7900XTX becomes hard to source or will it be replaced with the 9070?

@__tinygrad__ · 2025-01-12 18:28
Today is a great day to scale out https://t.co/1QOzqlDdmE

@Michi07f · 2025-01-12 18:00
@__tinygrad__ @AMDGPU_ Will the tinybox red be discontinued if the 7900XTX becomes hard to source or will it be replaced with the 9070?

@__tinygrad__ · 2025-01-12 18:28
@Michi07f @AMDGPU_ Discontinued until AMD makes a GPU that beats the 7900XTX at anything.

@__tinygrad__ · 2025-01-12 20:27
@ArcaVorago @realGeorgeHotz @blitz_or_die @comma_ai @LisaSu This is true. All our machines have AMD CPUs. They are good at hardware, just not software.

@alc2022 · 2025-01-13 13:38
$AMD's hardware has caught up to $NVDA. Software is next.

@AnushElangovan · 2025-01-14 01:58
Thats my job 😀

@__tinygrad__ · 2025-01-14 05:14
We switched from AMD's driver to our AM driver...and now 4 GPU Llama is faster on red box than green box! https://t.co/bjiLtwQULO

@ThePrimeagen · 2025-01-14 17:44
i think code formatters are an anti pattern

@biascat · 2025-01-15 01:04
@realGeorgeHotz @Yuchenj_UW @comma_ai Comma compute with 5090 to chuck in the glovebox coming soon? Would be really cool to have a embedded version of the tinybox designed for thermal, vibe, and power constraints for auto or industrial use.

@biascat · 2025-01-15 01:04
@realGeorgeHotz @Yuchenj_UW @comma_ai Comma compute with 5090 to chuck in the glovebox coming soon? Would be really cool to have a embedded version of the tinybox designed for thermal, vibe, and power constraints for auto or industrial use.

@__tinygrad__ · 2025-01-15 02:30
@aUsernamePNG @realGeorgeHotz @Yuchenj_UW @comma_ai We are developing an AMD USB driver for a reason

@elonmusk · 2025-01-15 14:09
If you’re a hardcore software engineer and want to build the everything app, please join us by sending your best work to code@x.com. We don’t care where you went to school or even whether you went to school or what “big name” company you worked at. Just show us your code.

@__tinygrad__ · 2025-01-15 15:50
RT @aaronvi: This Github wrapped for tinygrad made me lol @realGeorgeHotz https://t.co/DVGiNudesa

@li252708 · 2025-01-15 13:45
@AnushElangovan @XirtamEsrevni @__tinygrad__ @realGeorgeHotz, @SemiAnalysis_. You folks seems to be in the weeds with AMD software stack issues, looks like Anush from AMD is the person you may want to work with to get the software side on par with the Green team.

@__tinygrad__ · 2025-01-15 15:54
@li252708 @AnushElangovan @XirtamEsrevni @realGeorgeHotz @SemiAnalysis_ AMD is not serious about software. Nothing has changed about their strategy. That repo has had 0 updates in 9 months.

@ID_AA_Carmack · 2025-01-15 15:55
I am certainly going to get one of these, but my $200k A100 DGX has been an absolute lemon — replaced three times under warranty, and just last week the CPU cooling system appears to have died. Hopefully just needs new thermal paste.

@__tinygrad__ · 2025-01-15 15:54
@li252708 @AnushElangovan @XirtamEsrevni @realGeorgeHotz @SemiAnalysis_ AMD is not serious about software. Nothing has changed about their strategy. That repo has had 0 updates in 9 months.

@__tinygrad__ · 2025-01-15 15:55
@li252708 @AnushElangovan @XirtamEsrevni @realGeorgeHotz @SemiAnalysis_ OTOH, Alibaba is working to fix their drivers. https://t.co/cK6YeiAr2L

@ID_AA_Carmack · 2025-01-15 15:55
I am certainly going to get one of these, but my $200k A100 DGX has been an absolute lemon — replaced three times under warranty, and just last week the CPU cooling system appears to have died. Hopefully just needs new thermal paste.

@__tinygrad__ · 2025-01-15 15:56
@ID_AA_Carmack Buy a tinybox!

@ID_AA_Carmack · 2025-01-15 15:55
I am certainly going to get one of these, but my $200k A100 DGX has been an absolute lemon — replaced three times under warranty, and just last week the CPU cooling system appears to have died. Hopefully just needs new thermal paste.

@Yuchenj_UW · 2025-01-15 15:58
@ID_AA_Carmack > $200k A100 DGX has been an absolute lemon — replaced three times under warranty It shows how hard it is to do large-scale training with many GPUs.

@AnushElangovan · 2025-01-15 16:47
Exactly. Code talks, rest walk. And if you fancy something in low level GPU programming AMD is hiring to power an Open AI ecosystem ➡️aig-shark-hiring@amd.com

@__tinygrad__ · 2025-01-15 18:26
Some will never know the joy of user space drivers. https://t.co/TNB49g233d

@__tinygrad__ · 2025-01-15 18:27
@RoamingStaple @ID_AA_Carmack Try it. It'll ship to you if checkout works for your address.

@AnushElangovan · 2025-01-15 16:47
Exactly. Code talks, rest walk. And if you fancy something in low level GPU programming AMD is hiring to power an Open AI ecosystem ➡️aig-shark-hiring@amd.com

@__tinygrad__ · 2025-01-15 18:36
@AnushElangovan Still waiting on you to take our deal. A much better bang for your buck, even at the current $2M price. https://t.co/KDfJEWxD4v

@Yuchenj_UW · 2025-01-15 15:58
@ID_AA_Carmack > $200k A100 DGX has been an absolute lemon — replaced three times under warranty It shows how hard it is to do large-scale training with many GPUs.

@__tinygrad__ · 2025-01-15 18:41
@Yuchenj_UW @ID_AA_Carmack The key is building dead simple computers. These are tinybox pros. 3 big fans, 0 gold mesh. https://t.co/o0cKkcsgEc

@__tinygrad__ · 2025-01-15 18:41
@Yuchenj_UW @ID_AA_Carmack The key is building dead simple computers. These are tinybox pros. 3 big fans, 0 gold mesh. https://t.co/o0cKkcsgEc

@fedupoffed · 2025-01-15 18:42
@__tinygrad__ @Yuchenj_UW @ID_AA_Carmack @LisaSu 👁️

@fedupoffed · 2025-01-15 18:42
@__tinygrad__ @Yuchenj_UW @ID_AA_Carmack @LisaSu 👁️

@__tinygrad__ · 2025-01-15 18:43
@fedupoffed @Yuchenj_UW @ID_AA_Carmack @LisaSu oh yea, the other key is using NVIDIA GPUs 😂

@__tinygrad__ · 2025-01-15 18:26
Some will never know the joy of user space drivers. https://t.co/TNB49g233d

@pkuhar · 2025-01-15 18:43
@__tinygrad__ any relevant performance downsides?

@__tinygrad__ · 2025-01-15 18:41
@Yuchenj_UW @ID_AA_Carmack The key is building dead simple computers. These are tinybox pros. 3 big fans, 0 gold mesh. https://t.co/o0cKkcsgEc

@Yuchenj_UW · 2025-01-15 18:44
@__tinygrad__ @ID_AA_Carmack looks like a datacenter already

@pkuhar · 2025-01-15 18:43
@__tinygrad__ any relevant performance downsides?

@__tinygrad__ · 2025-01-15 18:44
@pkuhar Nope. It outperforms the AMD kernel driver. You can't do this for all types of drivers, but with how GPUs work for compute it's fine.

@Yuchenj_UW · 2025-01-15 18:44
@__tinygrad__ @ID_AA_Carmack looks like a datacenter already

@__tinygrad__ · 2025-01-15 18:44
@Yuchenj_UW @ID_AA_Carmack woah woah woah it's a compute cluster

@__tinygrad__ · 2025-01-15 18:44
@pkuhar Nope. It outperforms the AMD kernel driver. You can't do this for all types of drivers, but with how GPUs work for compute it's fine.

@pkuhar · 2025-01-15 18:45
@__tinygrad__ could be just because the AMD one sucks :) but it get it, with this type of driver the extra layer does not matter

@pkuhar · 2025-01-15 18:45
@__tinygrad__ could be just because the AMD one sucks :) but it get it, with this type of driver the extra layer does not matter

@__tinygrad__ · 2025-01-15 18:46
@pkuhar Nah, our user space AMD driver is similar speed to the NVIDIA kernel driver (which is good and we don't have to replace, just mod to support P2P)

@__tinygrad__ · 2025-01-15 18:26
Some will never know the joy of user space drivers. https://t.co/TNB49g233d

@BSchultzer · 2025-01-15 19:29
@__tinygrad__ Hardware is so fast today, that kernel driver is rarely needed, in fact most driver should be in user space. For @AMD , it’s properly due to legacy, but it is sad to see that their culture is so bad that they can’t even make their own user-space driver from scratch.

@BSchultzer · 2025-01-15 19:29
@__tinygrad__ Hardware is so fast today, that kernel driver is rarely needed, in fact most driver should be in user space. For @AMD , it’s properly due to legacy, but it is sad to see that their culture is so bad that they can’t even make their own user-space driver from scratch.

@__tinygrad__ · 2025-01-15 19:38
@BSchultzer @AMD They can't even ship us some free MI300X boxes so we can add support for it. The easiest ROI calculation, but their culture won't allow it. @LisaSu @AMD

@__tinygrad__ · 2025-01-15 18:36
@AnushElangovan Still waiting on you to take our deal. A much better bang for your buck, even at the current $2M price. https://t.co/KDfJEWxD4v

@UncleSam303 · 2025-01-15 19:42
@__tinygrad__ @AnushElangovan @LisaSu I think it’s a fair deal to promote @AMD software and AI hardware.

@__tinygrad__ · 2025-01-15 19:43
We are one piece away from a completely sovereign stack on AMD, the RDNA3 assembler. We have our own driver, runtime, libraries, and emulator. (all in ~12,000 lines!) There's a $1,000 bounty for an RDNA3 assembler that comes within 10% of the LLVM based one.

@UncleSam303 · 2025-01-15 19:42
@__tinygrad__ @AnushElangovan @LisaSu I think it’s a fair deal to promote @AMD software and AI hardware.

@__tinygrad__ · 2025-01-15 19:44
@UncleSam303 @AnushElangovan @LisaSu @AMD We'd add MI300X support to our (proven working omg it's open source and you can see it) driver for just the 2 boxes. I have 0 idea why the company wouldn't do that besides pride.

@__tinygrad__ · 2025-01-15 18:36
@AnushElangovan Still waiting on you to take our deal. A much better bang for your buck, even at the current $2M price. https://t.co/KDfJEWxD4v

@AnushElangovan · 2025-01-15 19:47
@__tinygrad__ Nah. All good. We can't take shortcuts or optimize for "bang for the buck" to greatness. But happy to get you access to mi300x to showcase the power of Tinygrad on AMD hardware and the power of open source.

@AnushElangovan · 2025-01-15 19:47
@__tinygrad__ Nah. All good. We can't take shortcuts or optimize for "bang for the buck" to greatness. But happy to get you access to mi300x to showcase the power of Tinygrad on AMD hardware and the power of open source.

@__tinygrad__ · 2025-01-15 19:50
@AnushElangovan We aren't interested in access, we're interested in two boxes shipped to us. I'm not a cloud fan. Why would you not want to most efficiently deploy capital to improve software?

@AnushElangovan · 2025-01-15 19:47
@__tinygrad__ Nah. All good. We can't take shortcuts or optimize for "bang for the buck" to greatness. But happy to get you access to mi300x to showcase the power of Tinygrad on AMD hardware and the power of open source.

@mayfer · 2025-01-15 19:52
@AnushElangovan @__tinygrad__ you totally could. they are writing drivers from scratch for you for free. there is no world in which this isn't a worthy bet to make for the cost of a few subpar employees for a year. insane risk-reward ratio that AMD ignores entirely out of pride

@__tinygrad__ · 2025-01-15 19:52
@andrew_schoff Nope, we are going to move it off AMD to our own or partner silicon. We have developed it to be very portable.

@mayfer · 2025-01-15 19:52
@AnushElangovan @__tinygrad__ you totally could. they are writing drivers from scratch for you for free. there is no world in which this isn't a worthy bet to make for the cost of a few subpar employees for a year. insane risk-reward ratio that AMD ignores entirely out of pride

@__tinygrad__ · 2025-01-15 19:53
@mayfer @AnushElangovan Of course. As tinygrad improves and gains adoption, the price will continue to go up 😂

@__tinygrad__ · 2025-01-15 19:43
We are one piece away from a completely sovereign stack on AMD, the RDNA3 assembler. We have our own driver, runtime, libraries, and emulator. (all in ~12,000 lines!) There's a $1,000 bounty for an RDNA3 assembler that comes within 10% of the LLVM based one.

@mov_axbx · 2025-01-15 19:57
@__tinygrad__ It’s amazing that they just stick their head in the sand like it doesn’t matter

@__tinygrad__ · 2025-01-15 19:50
@AnushElangovan We aren't interested in access, we're interested in two boxes shipped to us. I'm not a cloud fan. Why would you not want to most efficiently deploy capital to improve software?

@AnushElangovan · 2025-01-15 19:58
@__tinygrad__ Is it the most efficient way to deploy capital? I have about 8 companies claiming they can do the same (not on X) and I can't ship them each 2 boxes since they are not a cloud fan.

@__tinygrad__ · 2025-01-15 19:50
@AnushElangovan We aren't interested in access, we're interested in two boxes shipped to us. I'm not a cloud fan. Why would you not want to most efficiently deploy capital to improve software?

@mayfer · 2025-01-15 19:52
@AnushElangovan @__tinygrad__ you totally could. they are writing drivers from scratch for you for free. there is no world in which this isn't a worthy bet to make for the cost of a few subpar employees for a year. insane risk-reward ratio that AMD ignores entirely out of pride

@AnushElangovan · 2025-01-15 20:02
@mayfer @__tinygrad__ That is broadly painting AMD employees as subpar - not cool. Not everyone has a big microphone. The AMD platform is open exactly to foster the open source innovation that Tiny can do on the platform.

@mov_axbx · 2025-01-15 19:57
@__tinygrad__ It’s amazing that they just stick their head in the sand like it doesn’t matter

@__tinygrad__ · 2025-01-15 20:02
Like there are valid challenges to our stack. Perf, tinygrad lock-in, development model, etc...but that's never what's brought up. I estimate having software on par with NVDA would raise their market cap by 100B. Then you estimate what the chance it that @__tinygrad__ can close that gap, say it's 0.1%, probably a very low estimate when you see what we have done so far, but still... That's worth 100M. And they won't even send us 2 ~100k boxes. In what world does that make sense, except in a world where decisions are made based on pride instead of ROI. Culture issue.

@AnushElangovan · 2025-01-15 20:02
@mayfer @__tinygrad__ That is broadly painting AMD employees as subpar - not cool. Not everyone has a big microphone. The AMD platform is open exactly to foster the open source innovation that Tiny can do on the platform.

@__tinygrad__ · 2025-01-15 20:04
@AnushElangovan @mayfer lol, we got 0 docs from AMD on the 7900XTX, we worked it out ourselves. It's as open as NVIDIA in the parts that matter (would have been the same effort to do what we did for NVIDIA), we just didn't see the need to rewrite their drivers. https://t.co/0uarWcLdcb

@AnushElangovan · 2025-01-15 19:58
@__tinygrad__ Is it the most efficient way to deploy capital? I have about 8 companies claiming they can do the same (not on X) and I can't ship them each 2 boxes since they are not a cloud fan.

@AxcanNathan · 2025-01-15 20:05
@AnushElangovan @__tinygrad__ They didn't write a higher quality driver than yours from scratch, did they?

@AnushElangovan · 2025-01-15 19:58
@__tinygrad__ Is it the most efficient way to deploy capital? I have about 8 companies claiming they can do the same (not on X) and I can't ship them each 2 boxes since they are not a cloud fan.

@__tinygrad__ · 2025-01-15 20:07
@AnushElangovan We aren't making a claim. We already did it for the 7900XTX. git clone https://t.co/LlUdCGnKny rmmod amdgpu python3 test/test_tiny.py Show me something else close.

@__tinygrad__ · 2025-01-15 20:14
@militwitts @BSchultzer @AMD @LisaSu It's not about the machines, it's a cultural test. AMD has to want to be good before it's worth our time.

@__tinygrad__ · 2025-01-15 20:07
@AnushElangovan We aren't making a claim. We already did it for the 7900XTX. git clone https://t.co/LlUdCGnKny rmmod amdgpu python3 test/test_tiny.py Show me something else close.

@AnushElangovan · 2025-01-15 20:15
@__tinygrad__ Mad respect and kudos. But for now "cloud access" is still the best option to get you access to see what is possible.

@AxcanNathan · 2025-01-15 20:05
@AnushElangovan @__tinygrad__ They didn't write a higher quality driver than yours from scratch, did they?

@__tinygrad__ · 2025-01-15 20:16
@AxcanNathan @AnushElangovan Nope, but they would like to schedule a zoom call for next Tuesday to discuss generative AI partnership collaboration leadership.

@AnushElangovan · 2025-01-15 20:15
@__tinygrad__ Mad respect and kudos. But for now "cloud access" is still the best option to get you access to see what is possible.

@__tinygrad__ · 2025-01-15 20:18
@AnushElangovan If this isn't worth 200k to AMD to send us a few boxes, I don't know what to tell you. Cultural test to see if AMD can invest in software, if not why should I waste my development time on MI300X?

@AnushElangovan · 2025-01-15 20:15
@__tinygrad__ Mad respect and kudos. But for now "cloud access" is still the best option to get you access to see what is possible.

@__tinygrad__ · 2025-01-15 21:54
RT @phoronix: Tiny Corp Closing In On "Completely Sovereign" Compute Stack For AMD GPUs With Tinygrad @__tinygrad__ https://t.co/ghbFDWK89h

@FanaHOVA · 2025-01-15 23:29
AMD spent $695M in marketing but can't send @realGeorgeHotz $200k of GPUs 🤦‍♂️ RIP.

@FanaHOVA · 2025-01-15 23:29
AMD spent $695M in marketing but can't send @realGeorgeHotz $200k of GPUs 🤦‍♂️ RIP.

@jtatarchuk · 2025-01-16 01:59
@FanaHOVA @realGeorgeHotz technically 400k. maybe we should just send him 2 boxes. 🤔

@jtatarchuk · 2025-01-16 01:59
@FanaHOVA @realGeorgeHotz technically 400k. maybe we should just send him 2 boxes. 🤔

@__tinygrad__ · 2025-01-16 02:01
@jtatarchuk @FanaHOVA @realGeorgeHotz Hehe, we'd only accept them from AMD. It's about showing they are serious about investing in software development. Otherwise the platform is a dead end, and we shouldn't invest into it either. And ~200k to them, 400k retail.

@__tinygrad__ · 2025-01-16 03:18
It's so wild that this is true.

@AnushElangovan · 2025-01-16 04:12
@mike64_t @infogulch @__tinygrad__ Yes I offered full baremetal via ipmi and bmc. This is how all developers access it.

@__tinygrad__ · 2025-01-16 04:15
@AnushElangovan @mike64_t @infogulch You missed my point. We aren't going to spend development resources on something AMD isn't willing to invest $200k in. We aren't going to set up other systems to manage running our CI in the AMD cloud. Our work isn't worth $200k to you. Messaged received.

@__tinygrad__ · 2025-01-16 04:15
@AnushElangovan @mike64_t @infogulch You missed my point. We aren't going to spend development resources on something AMD isn't willing to invest $200k in. We aren't going to set up other systems to manage running our CI in the AMD cloud. Our work isn't worth $200k to you. Messaged received.

@__tinygrad__ · 2025-01-16 04:19
@AnushElangovan @mike64_t @infogulch The MI300X has no real value to us, we are fine with finishing the 7900XTX and then moving off of AMD to a different vendor or our own silicon. I thought it had value to AMD, but I guess less than $200k / <chance of tiny corp success>

@ThePrimeagen · 2025-01-14 17:44
i think code formatters are an anti pattern

@__tinygrad__ · 2025-01-16 04:30
@ThePrimeagen Agreed.

@__tinygrad__ · 2025-01-16 04:15
@AnushElangovan @mike64_t @infogulch You missed my point. We aren't going to spend development resources on something AMD isn't willing to invest $200k in. We aren't going to set up other systems to manage running our CI in the AMD cloud. Our work isn't worth $200k to you. Messaged received.

@AnushElangovan · 2025-01-16 05:38
Your work is respected and valuable to invest in.Not sure what "systems to manage running your CI in the AMD cloud" entails but happy to work through it.Offer stands for full baremetal access (you can do any low level dev work) and if we hit some issues which require physical access we can arrange.

@AnushElangovan · 2025-01-16 05:38
Your work is respected and valuable to invest in.Not sure what "systems to manage running your CI in the AMD cloud" entails but happy to work through it.Offer stands for full baremetal access (you can do any low level dev work) and if we hit some issues which require physical access we can arrange.

@__tinygrad__ · 2025-01-16 05:52
@AnushElangovan @mike64_t @infogulch No thanks, offer that to the other 8 companies. Good luck!

@mayfer · 2025-01-15 21:51
@AnushElangovan @infogulch @__tinygrad__ just give them the hardware man it's fycking unbelievable upside for amd for practically no downside

@chheplo · 2025-01-16 06:19
@mayfer @AnushElangovan @infogulch @__tinygrad__ This will be an example in the future to show what not to do if one wants to create a developer ecosystem. I guess we are stuck with Cuda for a while.

@ID_AA_Carmack · 2025-01-16 16:28
@nisargypandya @__tinygrad__ Many people, including me, willingly pay lots of money for Nvidia hardware even when we could have gotten access to other hardware for free. the moat is real.

@ID_AA_Carmack · 2025-01-16 16:28
@nisargypandya @__tinygrad__ Many people, including me, willingly pay lots of money for Nvidia hardware even when we could have gotten access to other hardware for free. the moat is real.

@__tinygrad__ · 2025-01-16 16:35
@ID_AA_Carmack @nisargypandya What's crazy is there's a whole world of possibly above the current NVIDIA stack. Why can't all the networked machines behave like one GPU?

@__tinygrad__ · 2025-01-16 19:06
This is a key thing people miss. They think CUDA is a programming language, or a library, or a runtime, or a driver. But that's not where the value is. The value is in the developer ecosystem, and that's why NVIDIA gets 91% margins.

@__tinygrad__ · 2025-01-16 19:06
This is a key thing people miss. They think CUDA is a programming language, or a library, or a runtime, or a driver. But that's not where the value is. The value is in the developer ecosystem, and that's why NVIDIA gets 91% margins.

@noxlonga · 2025-01-16 19:07
@__tinygrad__ nvidia mastered monopoly with finesse

@__tinygrad__ · 2025-01-16 19:06
This is a key thing people miss. They think CUDA is a programming language, or a library, or a runtime, or a driver. But that's not where the value is. The value is in the developer ecosystem, and that's why NVIDIA gets 91% margins.

@__tinygrad__ · 2025-01-16 19:08
tiny corp is trying to compete with this, but it's a long journey. Current ML frameworks are deeply coupled with libraries/runtimes/drivers, tinygrad isn't. The plan: 1. Make tinygrad performant on NVIDIA 2. Get developer adoption of the framework 3. Seamlessly swap the hardware

@noxlonga · 2025-01-16 19:07
@__tinygrad__ nvidia mastered monopoly with finesse

@__tinygrad__ · 2025-01-16 19:10
@noxlonga If there's any honest "monopoly" it's this one. They invested tons into building this, and afaik don't put up sketchy lock-in or barriers to switching. They just are the best.

@__tinygrad__ · 2025-01-16 20:30
Due to the crazy high price of 4090s, tinybox greens and pros are out of stock. We have reds. https://t.co/ALQJKjYUis

@MuchmoreIT · 2025-01-16 23:33
As a former employee the only thing I can say is that AMD has moral values to a fault.Where as Nvidia & Intel would sell their grandma on the corner to make an extra $1. The ecosystem is years behind Nvidia everyone there knows that, but giving all the people that ask for $200k of gear, and that's a small request, isn't the way forward. AMD being fabless makes them highly dependent on TSMC for MI300; Nvidia & Apple use them and Intel stumbling has caused them to switch some products there too. So it's a storm of things going on, but AMD is hitting all revenue and profit goals and being very careful, probably too careful in my opinion.

@FeepingCreature · 2025-01-16 23:41
@MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle My hobbyhorse is ComfyUI, which is pretty pytorch heavy, but I would personally port every node I need to tinygrad if it gave me 20% higher performance on the same hardware. That's free software work! If hw is *demonstrably* good, people will go far out their way to support it.

@FeepingCreature · 2025-01-16 23:41
@MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle My hobbyhorse is ComfyUI, which is pretty pytorch heavy, but I would personally port every node I need to tinygrad if it gave me 20% higher performance on the same hardware. That's free software work! If hw is *demonstrably* good, people will go far out their way to support it.

@__tinygrad__ · 2025-01-16 23:46
@FeepingCreature @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle You have a benchmark where PyTorch is beating tinygrad (BEAM=2) on AMD at inference? We'll add it to our CI and work on improving the speed.

@__tinygrad__ · 2025-01-16 23:46
@FeepingCreature @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle You have a benchmark where PyTorch is beating tinygrad (BEAM=2) on AMD at inference? We'll add it to our CI and work on improving the speed.

@FeepingCreature · 2025-01-16 23:48
@__tinygrad__ @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle tinygrad https://t.co/09rDADEKlN 1024x1024 vs ComfyUI with AMD running the howiejay/navi_support flash attention fork. See https://t.co/2NdZ5jDBZK I can get you current benchs in a bit.

@FeepingCreature · 2025-01-16 23:48
@__tinygrad__ @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle tinygrad https://t.co/09rDADEKlN 1024x1024 vs ComfyUI with AMD running the howiejay/navi_support flash attention fork. See https://t.co/2NdZ5jDBZK I can get you current benchs in a bit.

@__tinygrad__ · 2025-01-16 23:52
@FeepingCreature @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle Ugh, we need more of our https://t.co/Qv7SKwFwBj to be beamed. Let me know the step time from that fork, and if we beat it by 20% you'll port ComfyUI? (btw, you can just pickle TinyJITs to cache for start time) A big push this year is for tinygrad adoption in tools like this.

@__tinygrad__ · 2025-01-16 23:52
@FeepingCreature @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle Ugh, we need more of our https://t.co/Qv7SKwFwBj to be beamed. Let me know the step time from that fork, and if we beat it by 20% you'll port ComfyUI? (btw, you can just pickle TinyJITs to cache for start time) A big push this year is for tinygrad adoption in tools like this.

@FeepingCreature · 2025-01-16 23:58
@__tinygrad__ @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle Oh yeah - if you just want the timings for rocm, they're on https://t.co/nuJ5oV2938 shoutout to @dejay_vu !

@FeepingCreature · 2025-01-16 23:58
@__tinygrad__ @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle Oh yeah - if you just want the timings for rocm, they're on https://t.co/nuJ5oV2938 shoutout to @dejay_vu !

@__tinygrad__ · 2025-01-17 00:00
@FeepingCreature @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu Okay, 4.34it/s (230.4 ms) on 7900XTX. That in fp16?

@__tinygrad__ · 2025-01-17 00:00
@FeepingCreature @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu Okay, 4.34it/s (230.4 ms) on 7900XTX. That in fp16?

@FeepingCreature · 2025-01-17 00:01
@__tinygrad__ @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu fp32 vae, fp16 unet. comfy defaults for sdxl base I think, I haven't messed with it.

@FeepingCreature · 2025-01-17 00:01
@__tinygrad__ @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu fp32 vae, fp16 unet. comfy defaults for sdxl base I think, I haven't messed with it.

@__tinygrad__ · 2025-01-17 00:03
@FeepingCreature @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu I mean, it/s is just the unet, right? We're on it, there's a #fast-stable-diffusion channel on our Discord. Target speed: 184 ms. I doubt we are there yet, but speed is the main focus of this year.

@__tinygrad__ · 2025-01-17 00:03
@FeepingCreature @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu I mean, it/s is just the unet, right? We're on it, there's a #fast-stable-diffusion channel on our Discord. Target speed: 184 ms. I doubt we are there yet, but speed is the main focus of this year.

@FeepingCreature · 2025-01-17 00:04
@__tinygrad__ @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu Oh shit that would be awesome. I'd love to be able to love AMD cards without having to qualify "except set yourself up for pain with the software".

@FeepingCreature · 2025-01-17 00:04
@__tinygrad__ @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu Oh shit that would be awesome. I'd love to be able to love AMD cards without having to qualify "except set yourself up for pain with the software".

@__tinygrad__ · 2025-01-17 00:05
@FeepingCreature @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu `pip install tinygrad` has *zero* deps, you don't even need the amdgpu driver in the kernel anymore (though having it is fine if you use GPU for running desktop). Let's free everyone from ROCm this year!

@__tinygrad__ · 2025-01-17 00:05
@FeepingCreature @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu `pip install tinygrad` has *zero* deps, you don't even need the amdgpu driver in the kernel anymore (though having it is fine if you use GPU for running desktop). Let's free everyone from ROCm this year!

@FeepingCreature · 2025-01-17 00:07
@__tinygrad__ @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu Beam's still running btw. :-P

@FeepingCreature · 2025-01-17 00:07
@__tinygrad__ @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu Beam's still running btw. :-P

@__tinygrad__ · 2025-01-17 00:08
@FeepingCreature @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu Yes yes BEAM is slow. I'm running it too. We know we have to fix these pain points to gain widespread adoption.

@__tinygrad__ · 2025-01-17 00:08
@FeepingCreature @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu Yes yes BEAM is slow. I'm running it too. We know we have to fix these pain points to gain widespread adoption.

@FeepingCreature · 2025-01-17 00:09
@__tinygrad__ @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu Laundry list, eh.

@FeepingCreature · 2025-01-17 00:09
@__tinygrad__ @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu Laundry list, eh.

@__tinygrad__ · 2025-01-17 00:10
@FeepingCreature @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu It's all various forms of speed. 1 year ago, we were still failing at many kinds of correctness. Now it's speed time.

@__tinygrad__ · 2025-01-17 00:10
@FeepingCreature @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu It's all various forms of speed. 1 year ago, we were still failing at many kinds of correctness. Now it's speed time.

@FeepingCreature · 2025-01-17 00:13
@__tinygrad__ @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu Yeah I mean "fast good heuristics for kernel selection" would really not be the hardest part of this codebase by far. Personally, what worries me is when is AMD gonna follow up on the 7900? Software can only go so far if the hardware stays stuck in 2022.

@FeepingCreature · 2025-01-17 00:13
@__tinygrad__ @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu Yeah I mean "fast good heuristics for kernel selection" would really not be the hardest part of this codebase by far. Personally, what worries me is when is AMD gonna follow up on the 7900? Software can only go so far if the hardware stays stuck in 2022.

@__tinygrad__ · 2025-01-17 00:17
@FeepingCreature @MuchmoreIT @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu If we succeed at replacing ROCm on the lower cards, we'll raise funds to buy the Radeon division and ship some BFGPUs 😂

@alexocheema · 2025-01-17 02:15
GPUs are the wrong level of abstraction for a compute marketplace

@__tinygrad__ · 2025-01-17 02:19
@alexocheema Agreed. You want to pay for the job, and perhaps a premium to get it done sooner. tinygrad's test cloud will be priced in GPU-seconds, but yea it's not exactly the right unit.

@cyberpengk · 2025-01-17 02:26
@__tinygrad__ @alexocheema Why not just # ops

@cyberpengk · 2025-01-17 02:26
@__tinygrad__ @alexocheema Why not just # ops

@__tinygrad__ · 2025-01-17 04:02
@cyberpengk @alexocheema Because most of the power cost is data movement, and it's very rare you can fully utilize your ALUs. Access patterns matter.

@__tinygrad__ · 2025-01-17 19:24
A few new bounties. $200 for adding Windows CI $200 for moving llvm_bf16_cast to renderer $300 for FP8 on NVIDIA $500 for CUVID support in the NV driver 3x $400 bounties for matmul speed 2x $1000 bounties for GPU assembly Open to suggestions for more. https://t.co/4MfHqv430j

@__tinygrad__ · 2025-01-17 19:56
RT @alexocheema: @VitalikButerin One of the reasons I love @__tinygrad__ Zero dependencies, ~10k lines of code, if something goes wrong you can quickly jump in and figure out what's going on, without having to walk through 100 abstractions/dependencies

@cmuratori · 2025-01-17 19:53
The extreme levels of AMD disrespect for George Hotz is weird. I don't understand it, and I was already put off by it even before this latest debacle, which goes all the way to 11.

@corsix · 2025-01-17 20:58
@cmuratori Shitpost with AMD on X? Indifference. Shitpost with Tenstorrent on X? They’ll hire you to fix the problem (I’d like to see comprehensive low-level docs describing in detail everything you need to write a driver and a software stack for their HW … so that’s what I’ll be writing)

@__tinygrad__ · 2025-01-17 21:59
And this is why @tenstorrent has a bright future! We'd love to see a tinygrad backend for them, just added a $1,000 "Tenstorrent backend passing all ops tests on Grayskull" bounty.

@MuchmoreIT · 2025-01-17 19:04
@EPlCs @__tinygrad__ @FeepingCreature @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu LOLs all over again. My boy whining they didn't get free $200k worth of gear but will buy the $1B a year revenue Radeon division. $1B p/yr meaning they would have to pay at least 3 years worth, so can't wait to see them pony up $3B! Oh the joys of watching mental illness on X.

@EPlCs · 2025-01-17 22:03
@MuchmoreIT @__tinygrad__ @FeepingCreature @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu I'm aware that this is highly unlikely. That's part of the reason why if they achieve this they'll be remembered as legends.

@EPlCs · 2025-01-17 22:03
@MuchmoreIT @__tinygrad__ @FeepingCreature @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu I'm aware that this is highly unlikely. That's part of the reason why if they achieve this they'll be remembered as legends.

@__tinygrad__ · 2025-01-17 22:05
@EPlCs @MuchmoreIT @FeepingCreature @FanaHOVA @realGeorgeHotz @HotAisle @dejay_vu hehe, we know it's unlikely too. but you have to have dreams

@__tinygrad__ · 2025-01-17 21:59
And this is why @tenstorrent has a bright future! We'd love to see a tinygrad backend for them, just added a $1,000 "Tenstorrent backend passing all ops tests on Grayskull" bounty.

@__tinygrad__ · 2025-01-17 22:10
For a while new backends were too hard to maintain, but the API is mature enough now that it's time for tenstorrent. Going to be interesting to think about what's required to get perf (not required for bounty), it's a very non uniform memory architecture.

@__tinygrad__ · 2025-01-17 21:59
And this is why @tenstorrent has a bright future! We'd love to see a tinygrad backend for them, just added a $1,000 "Tenstorrent backend passing all ops tests on Grayskull" bounty.

@corsix · 2025-01-17 22:37
@__tinygrad__ @tenstorrent In a purely personal capacity, can I fund a similar bounty (in terms of goal and payout amount) but for Wormhole instead of Grayskull?

@corsix · 2025-01-17 22:37
@__tinygrad__ @tenstorrent In a purely personal capacity, can I fund a similar bounty (in terms of goal and payout amount) but for Wormhole instead of Grayskull?

@__tinygrad__ · 2025-01-17 22:44
@corsix @tenstorrent Sure, added tweet link to sheet. Looks like it's $2k if your backend works for both.

@__tinygrad__ · 2025-01-18 00:35
The tinygrad test cloud will be 9x tinybox red all running the AM driver. Being built now, you'll access through tinygrad seamlessly with CLOUD=1. Free at launch for devs. https://t.co/Nw4lgfYvTA

@__tinygrad__ · 2025-01-18 00:35
The tinygrad test cloud will be 9x tinybox red all running the AM driver. Being built now, you'll access through tinygrad seamlessly with CLOUD=1. Free at launch for devs. https://t.co/Nw4lgfYvTA

@dorkmo · 2025-01-18 00:46
@__tinygrad__ Please give the AMD devs access to your cloud service

@dorkmo · 2025-01-18 00:46
@__tinygrad__ Please give the AMD devs access to your cloud service

@__tinygrad__ · 2025-01-18 01:01
@dorkmo Access will be for anyone who has landed PRs in tinygrad through their GitHub key.

@GawroskiT · 2025-01-19 19:05
Rumor: RTX 5090 FE at 600W Sounds like, a jet engine / generator on construction site (*source chiphell forum) https://t.co/HjP9dy2yik

@nepnep1111_ · 2025-01-19 20:15
@GawroskiT Pick 2 Compact Quiet Good cooling

@__tinygrad__ · 2025-01-19 22:33
tinybox picks quiet and good cooling tinybox pro picks compact and good cooling

@HardwareUnboxed · 2025-01-21 00:54
Seeing a lot of people saying the "waiting for Nvidia" strategy from AMD hasn't worked, so try something new, launch first... It hasn't worked because they keep stuffing it up. They fail to assess the value of their cards and fail to set pricing properly. They do the "waiting" part and then fail at the "responding" part. What would AMD actually gain from launching first? Who are the buyers that, knowing the RTX 5070 is coming, would jump all over a Radeon card a few weeks early? And are those buyers people AMD are converting from Nvidia to AMD, or just AMD fans? Nvidia is the dominant player in the GPU space. The ONLY thing that matters for Radeon is how much better value the Radeon card is relative to Nvidia. Gamers won't switch to AMD unless it's clear the RX 9070 XT is a significantly better choice than the 5070. Launch first and where is the comparison? To old GPUs that won't be relevant in a few weeks? To other Radeon cards most people don't own? Buyers would be jumping in blind and I just don't believe there are many people willing to blindly switch to Radeon right now. Instead, going first AMD would have the most to lose. They could misjudge Nvidia's line-up and once again not offer the best value, because they'd be guessing what they are competing against. With a history of stuffing up launches and needing your parts to be directly and obviously better than the 50 series, why take that risk? They aren't, and that's why RDNA4 is coming in March

@alexocheema · 2025-01-21 04:35
AGI at home Running DeepSeek R1 across my 7 M4 Pro Mac Minis and 1 M4 Max MacBook Pro. Total unified memory = 496GB. Uses @exolabs distributed inference with 4-bit quantization. Next goal is fp8 (requires >700GB) https://t.co/Sqn3doiBEG

@complex_maths · 2025-01-21 05:30
@JosephS52445614 @kurosumomo @HardwareUnboxed Even if it's not as production ready as AMD's own drivers, the fact that it performs significantly better in certain circumstances despite not having the HW designs like AMD is a strong indicator that they have done something of value, the question is what is that worth to AMD.

@complex_maths · 2025-01-21 05:31
@JosephS52445614 @kurosumomo @HardwareUnboxed And apparently less than $200k which is probably less than the annual total compensation of a senior or principal software engineer at AMD.

@__tinygrad__ · 2025-01-21 05:49
A conv (with bias) and a relu. Can you see it? https://t.co/zk0dsjxraK

@tenstorrent · 2025-01-21 13:00
Thanks @__tinygrad__ , we're excited about a Tenstorrent back end and will match the bounty for Grayskull and Wormhole --> https://t.co/MowgbRzfoO

@__tinygrad__ · 2025-01-21 15:06
RT @tenstorrent: Thanks @__tinygrad__ , we're excited about a Tenstorrent back end and will match the bounty for Grayskull and Wormhole --> https://t.co/MowgbRzfoO

@__tinygrad__ · 2025-01-21 22:50
RT @wpmed92: “However, tinygrad has a smaller footprint and is faster” This is why @__tinygrad__ is winning. https://t.co/hpuu1JQv6j

@__tinygrad__ · 2025-01-21 22:57
18x 7900XTX standing by for training, linked with 100GbE. Soon tinygrad will support training across all of them. https://t.co/ITcYhDhmvS

@__tinygrad__ · 2025-01-22 04:59
the cloud grows https://t.co/xeqCkC7j3V

@__tinygrad__ · 2025-01-22 17:44
This will be the Year of Speed. By the end of the year, we need speed competitive with PyTorch on NVIDIA. If we succeed at this, it's likely tinygrad wins.

@complex_maths · 2025-01-21 05:31
@JosephS52445614 @kurosumomo @HardwareUnboxed And apparently less than $200k which is probably less than the annual total compensation of a senior or principal software engineer at AMD.

@__tinygrad__ · 2025-01-22 23:52
@complex_maths @JosephS52445614 @kurosumomo @HardwareUnboxed Yea, sad. But we can't change them, we can just continue work on our driver, focused on consumer cards and not datacenter. The driver has value outside just AMD, 80% of it is portable to other chips, and AMD's hardware is a great environment to debug and develop in.

@__tinygrad__ · 2025-01-23 00:05
If you have an AMD card not supported by ROCm, it may be supported or easy for external contributors to support in tinygrad. ROCm is very complex with many different pieces and questionable testing, tinygrad is a self contained 11,000 line repo with modern CI. Our "AMD" backend uses AMD's kernel driver. It's tested to work on RDNA2 and RDNA3, so if it's just user space ROCm (either libraries or runtime) is missing it'll work in tinygrad. AMD=1 to use. Our "AM" backend is a full driver for the 7900XTX. It's only ~600 lines, so adding other cards shouldn't be too hard. rmmod amdgpu + AMD=1 to use. Note: everything currently assumes your card has LLVM support, but I don't think that's a blocker. If this gains traction in apps like ComfyUI or Ollama, we will invest into CI for a wide spectrum of AMD cards. Our LLM speeds are already very good, and we are working this year on improving diffusion speeds.

@__tinygrad__ · 2025-01-23 00:05
If you have an AMD card not supported by ROCm, it may be supported or easy for external contributors to support in tinygrad. ROCm is very complex with many different pieces and questionable testing, tinygrad is a self contained 11,000 line repo with modern CI. Our "AMD" backend uses AMD's kernel driver. It's tested to work on RDNA2 and RDNA3, so if it's just user space ROCm (either libraries or runtime) is missing it'll work in tinygrad. AMD=1 to use. Our "AM" backend is a full driver for the 7900XTX. It's only ~600 lines, so adding other cards shouldn't be too hard. rmmod amdgpu + AMD=1 to use. Note: everything currently assumes your card has LLVM support, but I don't think that's a blocker. If this gains traction in apps like ComfyUI or Ollama, we will invest into CI for a wide spectrum of AMD cards. Our LLM speeds are already very good, and we are working this year on improving diffusion speeds.

@davidpwalter · 2025-01-23 00:06
@__tinygrad__ Wondering if it'll work on the Steamdeck then..

@davidpwalter · 2025-01-23 00:06
@__tinygrad__ Wondering if it'll work on the Steamdeck then..

@__tinygrad__ · 2025-01-23 00:08
@davidpwalter Try it! Steam deck is RDNA2, so 50/50 shot it works out of the box, and if it doesn't I bet it's only a couple line change. This is the one with AMD's kernel driver.

@__tinygrad__ · 2025-01-23 00:11
@PytorchToAtoms No. After the exchange with AMD, it became we should only put resources into the consumer cards.

@__tinygrad__ · 2025-01-23 20:48
`python3 -m tinygrad.device` will show you which backends are working on your system. This is on a Mac. https://t.co/Cu5mXNlKNZ

@__tinygrad__ · 2025-01-23 20:48
`python3 -m tinygrad.device` will show you which backends are working on your system. This is on a Mac. https://t.co/Cu5mXNlKNZ

@bogedy · 2025-01-23 20:53
@__tinygrad__ how to tell which is fastest?

@bogedy · 2025-01-23 20:53
@__tinygrad__ how to tell which is fastest?

@__tinygrad__ · 2025-01-23 21:08
@bogedy The default is almost always the fastest, it has a * next to it.

@Kepler_L2 · 2025-01-23 21:29
Cancelling Navi4C was such a fumble 🥲

@HBloodedHeroine · 2025-01-23 06:33
@__tinygrad__ what about windows and 7900XTX

@LottoLabs · 2025-01-24 05:51
@HBloodedHeroine @__tinygrad__ Windows support for tinygrad is on hold afaik

@LottoLabs · 2025-01-24 05:51
@HBloodedHeroine @__tinygrad__ Windows support for tinygrad is on hold afaik

@__tinygrad__ · 2025-01-24 07:13
@LottoLabs @HBloodedHeroine Windows worked as of last November, but it breaks since there's no CI. There's a $200 bounty to add CI, just test_tiny is fine.

@__tinygrad__ · 2025-01-26 10:20
tinygrad has switched to UOp gradients. No storing the compute graph twice, support for higher order derivatives, and only 71 very readable lines. https://t.co/1BdPXC2hUw

@__tinygrad__ · 2025-01-27 06:15
Added a bunch of $200 bounties after the switch to UOp gradients, good places for new people to dive in. The "Tensor" and "UOp" classes are the minimal abstraction, so much cleanup done. See the flow with`VIZ=1 python3 test/test_schedule.py TestSchedule.test_sgd_2convs_fuse` https://t.co/sV48J5mE4g

@__tinygrad__ · 2025-01-28 00:14
A breakdown of tinygrad's lines. Well split between the three pieces. /+nn+viz = 3583 frontend codegen+engine+renderer+shape = 3548 middle runtime = 3708 for 15 backends (with full AMD driver) https://t.co/7JKkcqjIq3

@__tinygrad__ · 2025-01-28 00:17
There's never been a better time to start learning tinygrad. We are looking to fund an addictive gamified tutorial, but until then there's some great docs here. https://t.co/ErqGNc4Xy7

@__tinygrad__ · 2025-01-28 00:23
@Soxlkfk it's 275k tokens (see examples/self_tokenize.py), try it! tinygrad might one day be the worlds first recursively self improving program.

@Thilak · 2025-01-28 00:19
@cameronwhite95 @alexocheema @exolabs Perhaps you’re unacquainted with Metal programming or systems programming in general. You can’t emulate fp8 in software, so the goal is delusional at best and disingenuous at worst.

@alexocheema · 2025-01-28 00:34
“Next goal is fp8” I’m working on running at 8-bit precision with int8 and approximating fp8 in tinygrad. Please think before going on other’s posts and posting negative, unhelpful comments. I’ve seen this a few times from you now on other posts and I’m not sure why you’re doing this. We’re building free, open-source software. It’s fine to give feedback but this certainly isn’t feedback, it’s accusatory and negative. Best of luck with your ML projects.

@alexocheema · 2025-01-28 00:34
“Next goal is fp8” I’m working on running at 8-bit precision with int8 and approximating fp8 in tinygrad. Please think before going on other’s posts and posting negative, unhelpful comments. I’ve seen this a few times from you now on other posts and I’m not sure why you’re doing this. We’re building free, open-source software. It’s fine to give feedback but this certainly isn’t feedback, it’s accusatory and negative. Best of luck with your ML projects.

@__tinygrad__ · 2025-01-28 01:22
@alexocheema @Thilak @cameronwhite95 @exolabs You have so much compute I'm sure you could load fp8 and unpack it to fp32 while still saturating the memory bandwidth. We haven't put tons of effort into making this fast yet, but would be excited to see PRs in this direction.

@__tinygrad__ · 2025-01-28 09:23
Windows support is back thanks to two bounties claimed by b1tg! LLVM+CLANG backends tested in CI so they won't regress, plus OpenCL works on a real system for GPU speed. Some chance CUDA and HIP work too.

@RajaXg · 2025-01-28 14:47
As long as hardware community continues to think "inference" vs "training" as different architectures, Nvidia's dominance is extremely safe.

@RajaXg · 2025-01-28 16:47
silicon valley VCs - if you see a blank face from a hardware startup founder when you ask them inference or training, give them a term sheet right away..no more questions to ask 😀

@sdaz_42 · 2025-01-29 21:43
@shrihacker Bullish on @__tinygrad__ they are trying to solve this on the fly. You write python and their engine find the best low level optimizations for you backend.

@__tinygrad__ · 2025-01-30 00:27
GB202 arch doc is out. Same nerf in 5090 as 4090. 5090 might be worse FLOPS/$ than 4090, but certainly better RAM bandwidth and connectivity. Excited to get a bunch! "tinybox green 2" will likely be 4x 5090

@sdaz_42 · 2025-01-29 21:43
@shrihacker Bullish on @__tinygrad__ they are trying to solve this on the fly. You write python and their engine find the best low level optimizations for you backend.

@__tinygrad__ · 2025-01-30 00:28
@sdaz_42 @shrihacker Bitter lesson always wins in the long run.

@__tinygrad__ · 2025-01-30 00:27
GB202 arch doc is out. Same nerf in 5090 as 4090. 5090 might be worse FLOPS/$ than 4090, but certainly better RAM bandwidth and connectivity. Excited to get a bunch! "tinybox green 2" will likely be 4x 5090

@Michi07f · 2025-01-30 00:29
@__tinygrad__ Why four? What's the limiting factor?

@Michi07f · 2025-01-30 00:29
@__tinygrad__ Why four? What's the limiting factor?

@__tinygrad__ · 2025-01-30 00:31
@Michi07f Power (and price).

@ThePrimeagen · 2025-01-30 18:38
hey @realGeorgeHotz how do i get my hands on a tiny grad green?

@ThePrimeagen · 2025-01-30 18:38
hey @realGeorgeHotz how do i get my hands on a tiny grad green?

@__tinygrad__ · 2025-01-31 00:30
@ThePrimeagen @realGeorgeHotz Sadly they are out of stock due unavailability of 4090s. Three options: 1) Buy a red! We have many red 2) Wait for stock / tinybox green 2 w 5090s 3) I can offer cloud access to a green, join our Discord

@RajaXg · 2025-01-28 16:47
silicon valley VCs - if you see a blank face from a hardware startup founder when you ask them inference or training, give them a term sheet right away..no more questions to ask 😀

@__tinygrad__ · 2025-01-31 00:55
@RajaXg Inference is a subset of training.

@comma_ai · 2025-01-31 02:54
It's coming sooner than you think. Today's a great day to join - we're 25 people and hiring ~4 very talented software and hardware engineers.

@LostAngelNZ · 2025-01-31 03:28
@comma_ai We are just waiting on the PCIe over USB driver to connect the external GPU enclosure for that moment to be able to become a reality are we not?

@comma_ai · 2025-01-31 03:39
Not quite, openpilot release still has the best overall model we've trained to date. This will quickly change once our ML simulator starts really working. While we get ML sim working, we're working on massively scaling up training compute and data collection. Once we have ML sim + big compute + big data, @__tinygrad__ is ready to support a GPU plugged into a comma 3X.

@__tinygrad__ · 2025-01-31 11:38
one kernel, its buffers, and its launch dims, in tinygrad https://t.co/c7TVRK4QZX

@comma_ai · 2025-01-31 03:39
Not quite, openpilot release still has the best overall model we've trained to date. This will quickly change once our ML simulator starts really working. While we get ML sim working, we're working on massively scaling up training compute and data collection. Once we have ML sim + big compute + big data, @__tinygrad__ is ready to support a GPU plugged into a comma 3X.

@__tinygrad__ · 2025-01-31 11:48
@comma_ai See progress here. https://t.co/r7Lk9bdIh5

@__tinygrad__ · 2025-02-01 07:20
In what sort of a crazy world does it make sense to maintain both CUDA *and* ROCm? Imagine maintaining a different compiler and standard library for each CPU architecture. It makes no sense.

@__tinygrad__ · 2025-02-01 07:20
In what sort of a crazy world does it make sense to maintain both CUDA *and* ROCm? Imagine maintaining a different compiler and standard library for each CPU architecture. It makes no sense.

@__tinygrad__ · 2025-02-01 07:30
It's worse than just that. SNPE for Qualcomm, RKNN for Rockchip, BUDA for tenstorrent, and yes, even MLX for Apple are entire *frameworks* that are hardware specific. There's no reason for this. There's only so many ways ALUs, cores, RAMs, and caches fit together. tinygrad will support them all. Potentially with a few new rewrite rules for whatever your hardware is and tricks it has. tinygrad's rewrite engine is similar to MLIRs, but it's 10x simpler. tinygrad is the frontend of PyTorch, the middleware of Triton/Pallas (both built on MLIR), and compute specific drivers that are 100x simpler than the default ones. This year, we will prove it can be just as fast. With the same speed and way more simplicity, how do we lose?

@__tinygrad__ · 2025-02-01 07:20
In what sort of a crazy world does it make sense to maintain both CUDA *and* ROCm? Imagine maintaining a different compiler and standard library for each CPU architecture. It makes no sense.

@jmbollenbacher · 2025-02-01 17:04
tinybench: a new coding benchmark can the AI ingest and comprehend the entire @__tinygrad__ repo and then successfully complete a nontrivial PR which is actually merged? *thats* a real signal of success.

@ThePrimeagen · 2025-02-03 15:01
LFG!!! https://t.co/Qd3ZXw95Uu

@__tinygrad__ · 2025-02-03 22:54
RT @ThePrimeagen: tinygrad said i could borrow one of their machines... hope they are not mad https://t.co/EyHatBburN

@ThePrimeagen · 2025-02-03 15:01
LFG!!! https://t.co/Qd3ZXw95Uu

@__tinygrad__ · 2025-02-03 22:57
@ThePrimeagen Now try the red! 😂

@yacineMTB · 2025-02-06 00:24
so I invest in an AI company based on an agreement that they'll rent our GPUs and TPUs. That way, we increase our own revenue while owning an asset that increases in valuation. It's free money https://t.co/jzQmtFYMcj

@__tinygrad__ · 2025-02-06 10:20
New bounties, some you can solve in an hour or two! https://t.co/Vxz4n3mW8D

@yacineMTB · 2025-02-06 00:24
so I invest in an AI company based on an agreement that they'll rent our GPUs and TPUs. That way, we increase our own revenue while owning an asset that increases in valuation. It's free money https://t.co/jzQmtFYMcj

@__tinygrad__ · 2025-02-06 10:26
@yacineMTB Even better to do this if you are selling GPUs!

@__tinygrad__ · 2025-02-06 10:20
New bounties, some you can solve in an hour or two! https://t.co/Vxz4n3mW8D

@Michi07f · 2025-02-06 10:38
@__tinygrad__ 100 TFlop/s on A770 seem overly optimistic.

@Michi07f · 2025-02-06 10:38
@__tinygrad__ 100 TFlop/s on A770 seem overly optimistic.

@__tinygrad__ · 2025-02-06 10:40
@Michi07f What does the best torch with Intel ipex_extensions_devel_nocrash_oneapi get? I'll lower it to that.

@__tinygrad__ · 2025-02-06 10:20
New bounties, some you can solve in an hour or two! https://t.co/Vxz4n3mW8D

@ml_visoft · 2025-02-06 11:00
@__tinygrad__ With onboarding?

@ml_visoft · 2025-02-06 11:00
@__tinygrad__ With onboarding?

@__tinygrad__ · 2025-02-06 11:03
@ml_visoft Depends on your level of skill. Someday soon an LLM will solve one in a minute, all context, no train. Might even be today if you are good at prompting.

@__tinygrad__ · 2025-02-07 00:07
RT @BeGrateful180: @__tinygrad__ the VIZ tooling for Tinygrad is awesome. Practicing with it now.

@__tinygrad__ · 2025-02-07 11:51
Added "LLVM Speed" to CI. This compares tinygrad to torch on the CI machine's CPU. Even with BEAM, we are slower. PRs to improve speed are very welcome, hopefully this is a good addictive game loop! https://t.co/tPlgK6g0gM

@__tinygrad__ · 2025-02-07 11:51
Added "LLVM Speed" to CI. This compares tinygrad to torch on the CI machine's CPU. Even with BEAM, we are slower. PRs to improve speed are very welcome, hopefully this is a good addictive game loop! https://t.co/tPlgK6g0gM

@__tinygrad__ · 2025-02-07 12:03
It should actually be pretty easy to fix a lot of this. On GPUs we have locals to make L2 accesses good, but there's no LOCAL in LLVM. If you just have the LOCAL opt split the loop, I bet a lot of this would be green. Nobody has spent time on CPU yet, it can be you.

@__tinygrad__ · 2025-02-07 12:12
RT @__tinygrad__: Speed is broken down into a few things. First, there's runtime speed and compile speed. These tests are focused on runtime speed, so based on the code of the kernels generated. There's a scheduler which determines how the kernels are split and how memory is allocated, this is also not relevant for this test. This test is all about the kernel speed itself. Are you iterating in a cache-aware way? This is likely why gemm is slow. Check out the OptOps for how to fix that. Are you generating dumb code because simplification rules are missing? This is likely why cat is slow. Check out the rewrite rules for this. Are you just passing bad flags to LLVM? Didn't enable the right optimizations? DEBUG=6 will show you the assembly and you can dive in. 'DEBUG=6 LLVM=1 python3 test/test_tiny.py TestTiny.test_plus' is a good place to start seeing the internals. Also, when you are ready, try VIZ=1 (it opens in a web browser)

@__tinygrad__ · 2025-02-07 14:26
RT @wpmed92: We removed wgpu-py dependency from @__tinygrad__ and use Dawn engine through an autogenerated python interface. No need to have another extra python layer when we can just use it directly. https://t.co/Kp0qrZm8OY

@__tinygrad__ · 2025-02-08 00:33
.@github Actions is one of the best products in the world. CI is so important for making high quality software. https://t.co/NCWejMMxVE

@jxmnop · 2025-02-07 16:15
my understanding is that both AMD and qualcomm make chips that have ~equivalent performance to nvidia but neither can write the software tooling that N provides, like CUDA i get that it's complicated, but which part of the stack could possibly be so hard to replicate? are nvidia's low-level engineers really that much better?

@IanHailey · 2025-02-08 22:02
@jxmnop CUDA = Windows TinyGrad = Linux Once the later gains traction perhaps things start to normalise.

@__tinygrad__ · 2025-02-09 01:13
A great tool to check AVX performance, and calcuate the max FLOPS / core. AVX512 looks fake on Zen 4! (half the Mops of AVX256) https://t.co/60XHqaVHQj Max FLOPS / core = 7402*(256/32)*2 = 118 GFLOPS, torch gets 105. https://t.co/sVTg68W9mq

@BoganBits · 2025-02-09 01:53
@__tinygrad__ Right GFLOPS 😆 I guess "DP" means double precision? I would like to see a SP FMA to see if it has the same affliction

@BoganBits · 2025-02-09 03:20
@__tinygrad__ I added SP versions Assuming I got it right, the figures are almost exactly as for DP i.e. almost exactly twice the FLOPs as DP Question is, why did the original author(s) not bother with SP versions? Is this just assumed? Should I submit this patch? https://t.co/P1vKmYKp6U

@BoganBits · 2025-02-09 03:20
@__tinygrad__ I added SP versions Assuming I got it right, the figures are almost exactly as for DP i.e. almost exactly twice the FLOPs as DP Question is, why did the original author(s) not bother with SP versions? Is this just assumed? Should I submit this patch? https://t.co/P1vKmYKp6U

@__tinygrad__ · 2025-02-09 06:29
@BoganBits Nice! Yea, I think it's worth putting up a PR, good to know they are the same.

@__tinygrad__ · 2025-02-09 13:33
The key to making this new cycle different is making the infrastructure simple. Google's moat was MapReduce and the Web became bad. No moats this time.

@oguzerkan · 2025-02-09 20:08
$AMD will be the next trillion dollar company.

@__tinygrad__ · 2025-02-09 23:09
RT @mov_axbx: Started working on a tinygrad bounty.

@__tinygrad__ · 2025-02-10 09:09
We have 6 tinybox reds in stock, 0 have sold so far this year. Meanwhile we get an e-mail a day asking about out of stock tinybox greens. (coming probably April with 4x5090, or will restock 6x4090 if we can buy them somewhere)

@__tinygrad__ · 2025-02-10 09:09
We have 6 tinybox reds in stock, 0 have sold so far this year. Meanwhile we get an e-mail a day asking about out of stock tinybox greens. (coming probably April with 4x5090, or will restock 6x4090 if we can buy them somewhere)

@pokorz · 2025-02-10 09:12
@__tinygrad__ (the red boxes are AMD)

@__tinygrad__ · 2025-02-10 09:09
We have 6 tinybox reds in stock, 0 have sold so far this year. Meanwhile we get an e-mail a day asking about out of stock tinybox greens. (coming probably April with 4x5090, or will restock 6x4090 if we can buy them somewhere)

@test_tm7873 · 2025-02-10 09:14
@__tinygrad__ Blue boxes would be cool in future hhhh

@pokorz · 2025-02-10 09:12
@__tinygrad__ (the red boxes are AMD)

@__tinygrad__ · 2025-02-10 09:14
@pokorz If you are willing to use tinygrad, our driver is getting quite good. Confident the red boxes will move once there's more tinygrad applications.

@test_tm7873 · 2025-02-10 09:14
@__tinygrad__ Blue boxes would be cool in future hhhh

@__tinygrad__ · 2025-02-10 09:21
@test_tm7873 Blue needs to make a big GPU with 24GB+ of RAM and the bandwidth to match.

@__tinygrad__ · 2025-02-10 13:53
We have an open to all meeting on Discord at 6AM PST every Monday. (in 8 minutes)

@yacineMTB · 2025-02-11 17:03
i love javascript. in a just world it would be the default data analysis language, not python

@__tinygrad__ · 2025-02-12 11:28
RT @mov_axbx: I'm actually kind of surprised that CUDA is considered such a moat considering it's an abstraction over PTX. For research, I get it. CUDA, Jupiter notebooks, whatever. All the abstractions. Lightweight stuff like finetuning, sure. But the more time I spend with tinygrad 1/

@yacineMTB · 2025-02-11 17:03
i love javascript. in a just world it would be the default data analysis language, not python

@__tinygrad__ · 2025-02-12 11:42
@yacineMTB Opens console: >> 99999999999999999999 <- 100000000000000000000 Very good data analysis free extra number

@yacineMTB · 2025-02-11 17:03
i love javascript. in a just world it would be the default data analysis language, not python

@asius_ai · 2025-02-13 06:58
@yacineMTB try https://t.co/jwvC0FVAPl, a WIP tinygrad rewrite in typescript. works in browser and in deno with WebGPU runtime. WebGPU browser MNIST training example: https://t.co/rWSIYwzpiH

@__tinygrad__ · 2025-02-13 10:10
A full training step in beautiful_mnist.py From left to right: - random data selection - forward pass - backward pass - optimizer https://t.co/mT9NulR8jT

@__tinygrad__ · 2025-02-13 10:18
Very interesting...it's a translation of tinygrad to typescript. The goal of the tinygrad project isn't just making a great ML framework, it's to find the ML framework that would be in "The Book" (Erdős)

@ctjlewis · 2025-02-13 18:59
talked to a FAANG CEO today, he said they’re not doing LeetCode screens anymore because of AI. they only care if you get PRs merged on GitHub.

@__tinygrad__ · 2025-02-14 01:16
This is what we care about too. There's no way to game it, if you want a job here, show you can get PRs merged into tinygrad.

@__tinygrad__ · 2025-02-14 01:16
This is what we care about too. There's no way to game it, if you want a job here, show you can get PRs merged into tinygrad.

@OverfitForTruth · 2025-02-14 01:22
@__tinygrad__ is it ok to use ai to help write the code for these PRs?

@OverfitForTruth · 2025-02-14 01:22
@__tinygrad__ is it ok to use ai to help write the code for these PRs?

@__tinygrad__ · 2025-02-14 01:30
@OverfitForTruth It's not a test, we don't care how you work. But if you copy and paste code you don't understand from the AI it is obvious and you are wasting everyone's time. Let's delve into the logic here.

@__tinygrad__ · 2025-02-14 01:16
This is what we care about too. There's no way to game it, if you want a job here, show you can get PRs merged into tinygrad.

@SuperHumanEpoch · 2025-02-14 01:32
@__tinygrad__ BRB fixing doc typos PR.

@SuperHumanEpoch · 2025-02-14 01:32
@__tinygrad__ BRB fixing doc typos PR.

@__tinygrad__ · 2025-02-14 01:33
@SuperHumanEpoch From contributing in README: "All docs and whitespace changes will be closed unless you are a well-known contributor. The people writing the docs should be those who know the codebase the absolute best. People who have not demonstrated that shouldn't be messing with docs."

@__tinygrad__ · 2025-02-14 15:10
RT @t0kenl1mit: Ok trying a @__tinygrad__ speed bounty dealing with concatenation. Dived into the source for the first time and there is so much going on. I honestly think the slowdown though is coming from doing the padding. Due to so much to call and chain together. Unless my understanding is off, when you pad the tensor it calls the UOps to run the pad operation, more specifically UOps.PAD, which is linked to the ShapeTracker class pad method. The ShapeTracker then calls on the View pad method to do the padding while adding the reshaped padded View to the last View in the ShapTracker view list calling on a custom __add__ method in the View class which is complex. I am kinda skipping some details because there is a lot to go on. I am going to keep diving though and maybe a better padding method is needed. Looking at the UOps.ADD method, it doesn't seem to to be the slow down and only using the python operator.add via broadcasting. I tired switching out using numpy add and it was significantly slower. Gotta think a bit on this one and it might need more of a rewrite.

@yacineMTB · 2025-02-14 21:30
what are you building this long weekend?

@__tinygrad__ · 2025-02-14 23:43
RT @frguthmann: Seb wrote a fantastic article exploring how to make mat mul go fast on AMD GPUs. This is a deep dive into both the algorithm and hardware, with great visualizations. He is also looking for work and I only have good things to say about him, so hire him! https://t.co/QpunDRW2os

@t0kenl1mit · 2025-02-14 10:10
Ok trying a @__tinygrad__ speed bounty dealing with concatenation. Dived into the source for the first time and there is so much going on. I honestly think the slowdown though is coming from doing the padding. Due to so much to call and chain together. Unless my understanding is off, when you pad the tensor it calls the UOps to run the pad operation, more specifically UOps.PAD, which is linked to the ShapeTracker class pad method. The ShapeTracker then calls on the View pad method to do the padding while adding the reshaped padded View to the last View in the ShapTracker view list calling on a custom __add__ method in the View class which is complex. I am kinda skipping some details because there is a lot to go on. I am going to keep diving though and maybe a better padding method is needed. Looking at the UOps.ADD method, it doesn't seem to to be the slow down and only using the python operator.add via broadcasting. I tired switching out using numpy add and it was significantly slower. Gotta think a bit on this one and it might need more of a rewrite.

@joe_sweeney · 2025-02-14 23:51
@t0kenl1mit @__tinygrad__ Yeah, I was looking at this one and I was thinking it would be much simpler if you could just add a new Uop for it, then you only have the cost of the memcpy's. But I think the challenge is to figure out how to rewrite the AST in these cases to just transform it into the memcpy's

@joe_sweeney · 2025-02-14 23:51
@t0kenl1mit @__tinygrad__ Yeah, I was looking at this one and I was thinking it would be much simpler if you could just add a new Uop for it, then you only have the cost of the memcpy's. But I think the challenge is to figure out how to rewrite the AST in these cases to just transform it into the memcpy's

@__tinygrad__ · 2025-02-15 01:25
@joe_sweeney @t0kenl1mit Adding UOps at the Tensor level is extremely frowned upon. Look at the generated code from the cat, notice how bad it is, then add rewrite rules to transform it into something good.

@yacineMTB · 2025-02-14 21:30
what are you building this long weekend?

@mov_axbx · 2025-02-15 02:07
@yacineMTB I gotta get back to tinygrad or my NEC Vector Engine backend for llama.cpp Work super busy lately, promise I’m not lazy I’m just trying to ensure the success of 5 or 6 businesses lol

@__tinygrad__ · 2025-02-15 10:02
RT @AnthonyPatch15: Tinygrad is a very nice project, it's quite easy to read and get into. I'd suggest looking at the bounty list and trying to understand any of the small bugs: https://t.co/dso8n8aZ6U

@mov_axbx · 2025-02-15 02:07
@yacineMTB I gotta get back to tinygrad or my NEC Vector Engine backend for llama.cpp Work super busy lately, promise I’m not lazy I’m just trying to ensure the success of 5 or 6 businesses lol

@__tinygrad__ · 2025-02-15 10:03
@mov_axbx @yacineMTB Write a tinygrad backend for it! Should be easy to get full perf BS=1 LLM inference.

@Leik0w0 · 2025-02-16 14:55
I want the same driver tinygrad has for amd but for nvidia (vfio based)

@__tinygrad__ · 2025-02-17 01:55
RT @ekzhang1: thanks @chenyuxyz for the awesome talk the internals of tinygrad today at nysrg! you can tell when you’re talking to a real expert, and having chenyu as an expert on kernels & hardware optimization come hang out with us for 3 hours was really wonderful https://t.co/CwChc44iZx

@__tinygrad__ · 2025-02-17 02:14
RT @0xhooved: Check out tinychat, a browser LLM app built with @__tinygrad__, which runs llama-3.2-1B locally on both WebGPU and WASM, including on newer phones such as iPhone 15. 🔗👇 🧵 https://t.co/jg8O7ec1fk

@__tinygrad__ · 2025-02-17 14:00
It's meeting time! (on discord, all can listen)

@__tinygrad__ · 2025-02-17 14:00
It's meeting time! (on discord, all can listen)

@__tinygrad__ · 2025-02-17 15:54
Read the transcripts of all the old meetings here. https://t.co/xEDHTFygOP

@Leik0w0 · 2025-02-16 14:55
I want the same driver tinygrad has for amd but for nvidia (vfio based)

@__tinygrad__ · 2025-02-18 01:51
@Leik0w0 You are welcome to write it! It would be simpler since NVIDIA pushes a lot more to the firmware with the GSP.

@__tinygrad__ · 2025-02-18 02:23
sign up for the mailing list to be notified of tinybox green v2 launch. no preorders, too much work. will be a link drop, first to send wire is the first to ship. https://t.co/6lB0KM2Gqn

@__tinygrad__ · 2025-02-18 01:51
@Leik0w0 You are welcome to write it! It would be simpler since NVIDIA pushes a lot more to the firmware with the GSP.

@Leik0w0 · 2025-02-18 07:44
@__tinygrad__ I would be delighted to. Though finding docs isn’t really an option and I’m just reading open gpu kernel modules & nouveau source code … How did nimlgen make the amd one ?

@Leik0w0 · 2025-02-18 07:44
@__tinygrad__ I would be delighted to. Though finding docs isn’t really an option and I’m just reading open gpu kernel modules & nouveau source code … How did nimlgen make the amd one ?

@__tinygrad__ · 2025-02-18 12:09
@Leik0w0 Just reading the driver. Everything you need is in open gpu kernel modules.

@ludwigABAP · 2025-02-18 12:28
x is interesting because there are both really great predictions and very poor ones and ppl's world models widely differ, but everyone uses the same terminology and tpot lingo so it all ends up looking equal, no matter how poor the take is

@RuiCarrilho5 · 2025-02-18 12:41
@ludwigABAP what do you think of this take? personally, I've been hearing this "Nvidia has no moat" take for a while, yet other than tinygrad, I've seen nobody come close to challenging them. and even geohot says they have no ecosystem right now to challenge Nvidia with!

@rhatr · 2025-02-19 01:44
I kind of really want to take Jim up on this promise, but it seems that the bounty is still mostly on the tinygrad side https://t.co/gphQrtzkIo @corsix are you working on this by any chance? Or maybe we need Martin's involvement as well?

@rhatr · 2025-02-19 01:44
I kind of really want to take Jim up on this promise, but it seems that the bounty is still mostly on the tinygrad side https://t.co/gphQrtzkIo @corsix are you working on this by any chance? Or maybe we need Martin's involvement as well?

@__tinygrad__ · 2025-02-19 03:18
@rhatr @corsix Tenstorrent has been very supportive. https://t.co/nBntMlub5i

@__tinygrad__ · 2025-02-19 03:46
Big bear signal on Intel today. At least AMD makes nice hardware that's comparatively easy to program for, and has a stable and consistently improving product line. With big chip consumer UDNA they are right back in it. So no tinybox blue. Only red and green.

@__tinygrad__ · 2025-02-19 03:46
Big bear signal on Intel today. At least AMD makes nice hardware that's comparatively easy to program for, and has a stable and consistently improving product line. With big chip consumer UDNA they are right back in it. So no tinybox blue. Only red and green.

@zeroxdoubler · 2025-02-19 03:54
@__tinygrad__ Tiny Grad rare compliment on Team RED 😂

@zeroxdoubler · 2025-02-19 03:54
@__tinygrad__ Tiny Grad rare compliment on Team RED 😂

@__tinygrad__ · 2025-02-19 03:57
@0xRezaRamadhan The hardware is actually really good which is what is so dumb about the whole situation! We have our own full stack now on 7900XTX, so it's much less frustrating to work with the cards these days. We can focus on only the quality of the hardware.

@__tinygrad__ · 2025-02-19 03:46
Big bear signal on Intel today. At least AMD makes nice hardware that's comparatively easy to program for, and has a stable and consistently improving product line. With big chip consumer UDNA they are right back in it. So no tinybox blue. Only red and green.

@labloke11 · 2025-02-19 04:09
@__tinygrad__ So... no big blue box?

@labloke11 · 2025-02-19 04:09
@__tinygrad__ So... no big blue box?

@__tinygrad__ · 2025-02-19 04:21
@labloke11 Nope. They have 15,000 unsold Gaudi 2 cards that will probably end up in the shredder, the product line and software were cancelled. Some chance we'll see them super cheap on eBay in a year. Cheap HBM if you can use it, it's hard to get the FLOPS for anything but GEMM though.

@__tinygrad__ · 2025-02-19 06:42
To clarify my last tweet, that's *if* AMD launches a big consumer GPU in the next 24 months. We'll be ready with a good driver and stack at that point, the red box will be back. RDNA4 is a skip, only small GPU. 5090s are great, but they aren't $5,000 great. We need competition.

@__tinygrad__ · 2025-02-19 06:42
To clarify my last tweet, that's *if* AMD launches a big consumer GPU in the next 24 months. We'll be ready with a good driver and stack at that point, the red box will be back. RDNA4 is a skip, only small GPU. 5090s are great, but they aren't $5,000 great. We need competition.

@NikoSchneider15 · 2025-02-19 06:46
@__tinygrad__ If they would load Balance they would be great. Now you have to pray the tolerance in the Connection from the PSU is really really good.😂

@NikoSchneider15 · 2025-02-19 06:46
@__tinygrad__ If they would load Balance they would be great. Now you have to pray the tolerance in the Connection from the PSU is really really good.😂

@__tinygrad__ · 2025-02-19 06:47
@NikoSchneider15 We will do a really good job testing this in tinybox v2, I know it's a concern. Thermal camera on the wires/connectors.

@__tinygrad__ · 2025-02-19 07:04
If the main stack remains PyTorch based, nobody will come close to challenging NVIDIA for training. In the same way nobody could challenge x86 while things were Windows based. Inference will have multiple players and be a race to the bottom.

@__tinygrad__ · 2025-02-19 07:04
If the main stack remains PyTorch based, nobody will come close to challenging NVIDIA for training. In the same way nobody could challenge x86 while things were Windows based. Inference will have multiple players and be a race to the bottom.

@StraughterG · 2025-02-19 07:07
@__tinygrad__ Somebody needs to rewrite CUDA ASAP.

@StraughterG · 2025-02-19 07:07
@__tinygrad__ Somebody needs to rewrite CUDA ASAP.

@__tinygrad__ · 2025-02-19 07:08
@StraughterG People don't even understand what they mean when they use the word "CUDA." Are you referring to the language, the runtime, the supporting libraries, or to the ".cuda()" function in torch?

@__tinygrad__ · 2025-02-19 06:42
To clarify my last tweet, that's *if* AMD launches a big consumer GPU in the next 24 months. We'll be ready with a good driver and stack at that point, the red box will be back. RDNA4 is a skip, only small GPU. 5090s are great, but they aren't $5,000 great. We need competition.

@zhentan · 2025-02-19 07:28
@__tinygrad__ We need to band together a team of $AMD investor activists.

@zhentan · 2025-02-19 07:28
@__tinygrad__ We need to band together a team of $AMD investor activists.

@__tinygrad__ · 2025-02-19 07:30
@zhentan They might just be building this card already, if they are there's no need to change anything. Our software will be pretty mature by then, doesn't matter how bad the first party stuff is.

@RajaXg · 2025-02-19 14:40
https://t.co/kfwGIQk21F

@RajaXg · 2025-02-19 14:47
Wrote most of the article last year, updated a few things here and there this morning. It may be TL:DR to many. Summary 1. Increase the coder-to-coordinator ration by 10x 2. Organize company around product leadership 3. Cancel the product cancel culture (Relentless iterate) 4. Bet on Generality and focus on physics 5. Make Battlemage and PVC friction free to millions of developers.

@RajaXg · 2025-02-19 19:29
Fun exercise for semiconductor nerd twitter How many wafers do you need for 1 exa-flop of FP8 math? How many wafers do you need for 100 Tera-Bytes of DRAM? Some assumptions you can use - State of the compute wafer like TSMC N3 - State of the art DRAM wafer

@__tinygrad__ · 2025-02-19 07:08
@StraughterG People don't even understand what they mean when they use the word "CUDA." Are you referring to the language, the runtime, the supporting libraries, or to the ".cuda()" function in torch?

@clattner_llvm · 2025-02-20 03:10
@__tinygrad__ @StraughterG Actually, you’re wrong - some of us really do understand this, and are willing to explain it. See part 2: https://t.co/18o0cK0NEQ Part 4 is out ~tomorrow.

@clattner_llvm · 2025-02-20 03:10
@__tinygrad__ @StraughterG Actually, you’re wrong - some of us really do understand this, and are willing to explain it. See part 2: https://t.co/18o0cK0NEQ Part 4 is out ~tomorrow.

@__tinygrad__ · 2025-02-20 04:03
@clattner_llvm @StraughterG "CUDA is not just one thing. It’s a huge, layered Platform" <-- true

@__tinygrad__ · 2025-02-20 13:06
A look inside PyTorch running a ResNet-18 generated with `extra/hook_cuda.py` We are going to backfeed this into tinygrad so we have a "template" of how to make things at least as fast. Improve the scheduler and improve the kernels. https://t.co/ucoZzyvd2x

@__tinygrad__ · 2025-02-20 13:06
A look inside PyTorch running a ResNet-18 generated with `extra/hook_cuda.py` We are going to backfeed this into tinygrad so we have a "template" of how to make things at least as fast. Improve the scheduler and improve the kernels. https://t.co/ucoZzyvd2x

@soumithchintala · 2025-02-20 13:50
@__tinygrad__ i think you should make tinygrad a pytorch backend, frontend support and stickiness is a large part of why pytorch is successful and that's not easy to replicate

@techsavvytravvy · 2025-02-20 13:52
devs will be like "i just always try to avoid complexity" and then use a 13mb library with 29 dependencies

@soumithchintala · 2025-02-20 13:50
@__tinygrad__ i think you should make tinygrad a pytorch backend, frontend support and stickiness is a large part of why pytorch is successful and that's not easy to replicate

@giffmana · 2025-02-20 13:54
@soumithchintala @__tinygrad__ or a numpy backend! (nvm just teasing 😁)

@__tinygrad__ · 2025-02-20 13:06
A look inside PyTorch running a ResNet-18 generated with `extra/hook_cuda.py` We are going to backfeed this into tinygrad so we have a "template" of how to make things at least as fast. Improve the scheduler and improve the kernels. https://t.co/ucoZzyvd2x

@2002_shit · 2025-02-20 14:02
@__tinygrad__ So you're telling me you're gonna cheat

@__tinygrad__ · 2025-02-20 14:34
RT @chenyuxyz: 4h6m!

@soumithchintala · 2025-02-20 13:50
@__tinygrad__ i think you should make tinygrad a pytorch backend, frontend support and stickiness is a large part of why pytorch is successful and that's not easy to replicate

@__tinygrad__ · 2025-02-20 14:37
@soumithchintala An issue with this is that it won't be perfect. What level do we add the backend at? As a first step, we are working towards supporting our drivers in PyTorch. Add ``` from https://t.co/cW10kSkOHs.hook import hook_cuda hook_cuda() import torch ``` and it will use our runtime.

@__tinygrad__ · 2025-02-20 14:37
@soumithchintala An issue with this is that it won't be perfect. What level do we add the backend at? As a first step, we are working towards supporting our drivers in PyTorch. Add ``` from https://t.co/cW10kSkOHs.hook import hook_cuda hook_cuda() import torch ``` and it will use our runtime.

@__tinygrad__ · 2025-02-20 14:40
@soumithchintala We'll do NVIDIA first, but when we do this for HIP/AMD we can bypass all of AMD's buggy AQL parser and kernel code. We should also be able to support using our JIT like this, so you can capture PyTorch runs into the tinygrad JIT.

@2002_shit · 2025-02-20 14:02
@__tinygrad__ So you're telling me you're gonna cheat

@__tinygrad__ · 2025-02-20 14:40
@2002_shit We're gonna learn from the best.

@RajaXg · 2025-02-19 19:29
Fun exercise for semiconductor nerd twitter How many wafers do you need for 1 exa-flop of FP8 math? How many wafers do you need for 100 Tera-Bytes of DRAM? Some assumptions you can use - State of the compute wafer like TSMC N3 - State of the art DRAM wafer

@__tinygrad__ · 2025-02-20 14:54
@RajaXg Grok 3: For 1 exa-flop of FP8 math: Approximately 20 wafers, assuming chips like the H100 on a 3nm process with 50 chips per 300mm wafer. For 100 terabytes of DRAM: Approximately 25 wafers, assuming 32 Gb dies yielding 4 TB per wafer.

@RajaXg · 2025-02-19 14:47
Wrote most of the article last year, updated a few things here and there this morning. It may be TL:DR to many. Summary 1. Increase the coder-to-coordinator ration by 10x 2. Organize company around product leadership 3. Cancel the product cancel culture (Relentless iterate) 4. Bet on Generality and focus on physics 5. Make Battlemage and PVC friction free to millions of developers.

@__tinygrad__ · 2025-02-20 15:01
@RajaXg Btw, there's 15k Gaudi 2 cards about to be scrapped. We'll save them from the shredder and write an open source runtime, get DeepSeek-R1 in homes :)

@__tinygrad__ · 2025-02-20 15:16
tinygrad is a 1.4MB wheel (why so big!) with 0 dependencies.

@giffmana · 2025-02-20 13:54
@soumithchintala @__tinygrad__ or a numpy backend! (nvm just teasing 😁)

@__tinygrad__ · 2025-02-20 15:26
@giffmana @soumithchintala I think you have to code in FORTRAN for that

@__tinygrad__ · 2025-02-20 15:16
tinygrad is a 1.4MB wheel (why so big!) with 0 dependencies.

@ezyang · 2025-02-20 15:34
@__tinygrad__ Tinygrad as a PyTorch backend would actually be a big deal, imagine a 1.4MB PyTorch wheel

@ezyang · 2025-02-20 15:34
@__tinygrad__ Tinygrad as a PyTorch backend would actually be a big deal, imagine a 1.4MB PyTorch wheel

@__tinygrad__ · 2025-02-20 15:38
@ezyang What's the narrowest abstraction in PyTorch? Like how many "kernels" do we have to write? Is PrimTorch the right target?

@__tinygrad__ · 2025-02-20 15:48
We're interested. Added a #pytorch-backend channel on Discord, let's figure out how to do this. Then PyTorch works everywhere tinygrad does, the install is small, and it becomes easy to export right from torch to things like WebGPU.

@ezyang · 2025-02-20 15:46
@__tinygrad__ It depends on if you want to target eager or compile. Compile is narrower but obviously it's not eager. Eager mode targeting PrimTorch will probably be bad performance because of all the missing fusion, although I guess you do have some fuser thing right

@__tinygrad__ · 2025-02-20 15:56
@ezyang We have a great fuser, yea I see no reason it can't be lazy and fusing under the hood in the same way CUDA is, even in eager mode. Is it easy to get back to Python where the PrimTorch kernels are being called? If so, we can write this in a week.

@__tinygrad__ · 2025-02-20 15:56
@ezyang We have a great fuser, yea I see no reason it can't be lazy and fusing under the hood in the same way CUDA is, even in eager mode. Is it easy to get back to Python where the PrimTorch kernels are being called? If so, we can write this in a week.

@__tinygrad__ · 2025-02-20 16:08
@ezyang (I don't mean CUDA is fusing, I mean it's async, which should let us be lazy)

@__tinygrad__ · 2025-02-20 15:48
We're interested. Added a #pytorch-backend channel on Discord, let's figure out how to do this. Then PyTorch works everywhere tinygrad does, the install is small, and it becomes easy to export right from torch to things like WebGPU.

@__tinygrad__ · 2025-02-21 08:58
Merged a start where basic Tensor math works. https://t.co/P10bhO6fA3 $200 bounty to make "TINY_BACKEND=1 python3 examples/other_mnist/beautiful_mnist_torch.py" work

@__tinygrad__ · 2025-02-22 06:46
The PyTorch backend is ready for people to help with. 'TINY_BACKEND=1 python3 test/test_ops.py' has 474 errors, but simple ones like `TestOps.test_add` pass. Add a few methods and fix a couple, bounty for fixing all!

@GPU_MODE · 2025-02-23 17:39
Write a fast kernel and run it on Discord. See how you compare against the best! If you're familiar with Leetcode, Kaggle or Codeforces then this should feel right at home https://t.co/SVtkp1BbRm

@forstmeier · 2025-02-24 02:01
Two things, .@grok. 1. I appreciate being labeled "tech-savvy" given my retarded ass can't figure out why tinygrad is shitting the bed converting a (30,1) tensor to a list. 2. I expect "Explore Amish technology" will just return an empty result. https://t.co/aUJY3mfVqB

@__tinygrad__ · 2025-02-24 09:11
Cool competition! `examples/torch_cuda_kernel.py` shows how to use tinygrad (with BEAM=2). The bitter lesson always wins in the end, any tricks people find should be added to our search. https://t.co/dI50Y7gWb7

@forstmeier · 2025-02-24 02:01
Two things, .@grok. 1. I appreciate being labeled "tech-savvy" given my retarded ass can't figure out why tinygrad is shitting the bed converting a (30,1) tensor to a list. 2. I expect "Explore Amish technology" will just return an empty result. https://t.co/aUJY3mfVqB

@__tinygrad__ · 2025-02-24 10:21
@forstmeier @grok >>> from tinygrad import Tensor >>> Tensor.zeros(30,1).tolist() [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]]

@__tinygrad__ · 2025-02-24 09:11
Cool competition! `examples/torch_cuda_kernel.py` shows how to use tinygrad (with BEAM=2). The bitter lesson always wins in the end, any tricks people find should be added to our search. https://t.co/dI50Y7gWb7

@0xCodyS · 2025-02-24 14:49
@__tinygrad__ Lots of optimizations to be made on Search + Learning. Neural approach + MCTS instead of BEAM?

@0xCodyS · 2025-02-24 14:49
@__tinygrad__ Lots of optimizations to be made on Search + Learning. Neural approach + MCTS instead of BEAM?

@__tinygrad__ · 2025-02-24 15:27
@0xCodyS Tons of improvements possible even keeping the search space we have. BEAM=10 will find even better kernels, it's just very slow.

@__tinygrad__ · 2025-02-25 02:10
RT @kean00reeves: Learning-With-Errors written purely in Tinygrad @realGeorgeHotz asymptotically outperforms numpy https://t.co/7jgQg0Tl4x

@forstmeier · 2025-02-25 03:49
.@grok solved my tinygrad issue where .@ChatGPTapp couldn't. Neat.

@__tinygrad__ · 2025-02-25 05:16
Grok is a lot more up to date, important for rapidly evolving projects like tinygrad.

@__tinygrad__ · 2025-02-25 05:16
Grok is a lot more up to date, important for rapidly evolving projects like tinygrad.

@MayurNewase · 2025-02-25 05:33
@__tinygrad__ is it possible to add helper functions to directly load weights from .pth file in tinygrad repo? @__tinygrad__

@MayurNewase · 2025-02-25 05:33
@__tinygrad__ is it possible to add helper functions to directly load weights from .pth file in tinygrad repo? @__tinygrad__

@__tinygrad__ · 2025-02-25 05:50
@MayurNewase We have this function! And it's super fast and lazy. https://t.co/wOXBrKmw0y

@__tinygrad__ · 2025-02-25 05:58
Is Gaudi 3 still a product for sale? Who is buying it when the software is discontinued? https://t.co/PICdyeP3JZ

@__tinygrad__ · 2025-02-25 07:05
Agreed.

@__tinygrad__ · 2025-02-25 12:49
RT @t0kenl1mit: Once I started thinking of @__tinygrad__ as a "tensor compiler" where the Ops are defined in AST for a tensor, the library started to make way more sense. Opposed to something like torchtune, tinygrad is like programming a computer that just takes tensors.

@jimkxa · 2025-02-25 18:59
Happy to announce I'm on the board of AheadComputing. Debbie Marr is the CEO, she's great. We are making the RiscV ecosystem, rich, broad and solid. CPUs, AI, support IP and software. Open RiscV is where you can innovate. Unconstrained. https://t.co/SKUpOFoQST

@_roberth_ · 2025-02-25 19:00
So @geerlingguy you need to get the new framework desktop mini rack lol https://t.co/qAwP1Icmo8

@__tinygrad__ · 2025-02-26 04:01
There's been massive spend in AI hardware in the last few years. There will be a bonanza of super cheap hardware once these systems start to get taken down, particularly for anything non-NVIDIA. teraflops priced by the pound!

@__tinygrad__ · 2025-02-26 04:01
There's been massive spend in AI hardware in the last few years. There will be a bonanza of super cheap hardware once these systems start to get taken down, particularly for anything non-NVIDIA. teraflops priced by the pound!

@AlexBravo · 2025-02-26 04:03
@__tinygrad__ now I see why you are publicly against $AMD 😜

@AlexBravo · 2025-02-26 04:03
@__tinygrad__ now I see why you are publicly against $AMD 😜

@__tinygrad__ · 2025-02-26 04:06
@AlexBravo We aren't against AMD, we are sad that they aren't a more serious competitor to NVIDIA. But compared to say, Intel, AMD has a better AI strategy.

@jimkxa · 2025-02-25 18:59
Happy to announce I'm on the board of AheadComputing. Debbie Marr is the CEO, she's great. We are making the RiscV ecosystem, rich, broad and solid. CPUs, AI, support IP and software. Open RiscV is where you can innovate. Unconstrained. https://t.co/SKUpOFoQST

@__tinygrad__ · 2025-02-26 04:08
@jimkxa Ahh, so that's where the Intel CPU design talent went!

@__tinygrad__ · 2025-02-26 07:51
@DaemonES Wait huh? They stuck 64GB of HBM memory on a CPU?!? Looked more, it was used in Aurora and probably nothing else. Looks like Intel really made bank on that thing. Your tax dollars at work :/

@__tinygrad__ · 2025-02-26 08:43
It's just beginning to scratch the surface, but we got some of torch's tests rigged up to compare the tiny backend to torch's CPU backend. torch has an excellent testing framework, and it's so satisfying to watch the tests get fixed. https://t.co/Nha3Jnolo9

@__tinygrad__ · 2025-02-26 08:43
It's just beginning to scratch the surface, but we got some of torch's tests rigged up to compare the tiny backend to torch's CPU backend. torch has an excellent testing framework, and it's so satisfying to watch the tests get fixed. https://t.co/Nha3Jnolo9

@GrantSlatton · 2025-02-26 08:47
@__tinygrad__ Are there a few recurring themes in the bugs here? Are most minor floating point stability stuff, or are some more egregious?

@GrantSlatton · 2025-02-26 08:47
@__tinygrad__ Are there a few recurring themes in the bugs here? Are most minor floating point stability stuff, or are some more egregious?

@__tinygrad__ · 2025-02-26 09:04
@GrantSlatton Oh no, it's just that we haven't implemented anything yet!

@__tinygrad__ · 2025-02-26 10:53
Since our torch backend is actually running tinygrad, you get all the great stuff like DEBUG=2 and VIZ=1. Try 'VIZ=1 python3 extra/torch_backend/example.py' to see a ResNet-18 inference, one resblock pictured here. https://t.co/KwPZS4vJpO

@__tinygrad__ · 2025-02-27 02:53
MNIST trainer is running in torch backend! Added $200 bounties for getting hlb-CIFAR10 and nanoGPT to work, it's mostly a bunch of straightforward one line op implementations. Here's the nanoGPT diff to enable tiny backend. https://t.co/jCcCmT2sIN

@__tinygrad__ · 2025-02-27 03:04
@PytorchToAtoms multiGPU yes, multimachine no. It's on the roadmap for Q3, we have networked machines ready when we are, need to finish a bunch of scheduler refactors first.

@__tinygrad__ · 2025-02-27 02:53
MNIST trainer is running in torch backend! Added $200 bounties for getting hlb-CIFAR10 and nanoGPT to work, it's mostly a bunch of straightforward one line op implementations. Here's the nanoGPT diff to enable tiny backend. https://t.co/jCcCmT2sIN

@__tinygrad__ · 2025-02-27 03:07
If one person comes in and starts to clean up these bounties, we'd hire them full time to work on the torch backend. It's so much easier to write a tiny backend than a torch one for new companies building accelerators, and soon, when you do, you'll get torch as well.

@botkooper · 2025-02-27 01:35
Fair, but you're forgetting that this is a cluster of 4, so it's 4x50 TOPS (not really because of overhead, but you get it) with 4x110GB VRAM (hard limit of 110 on linux, not all ram can be allocated to that) Sooo, something like 150 TOPS with 440GB vram? For < 10k including storage and rack and networking. Not competing with tinybox, but def still interesting

@anthonieisacnt · 2025-02-27 04:47
@botkooper @Bithunters @_roberth_ @geerlingguy @__tinygrad__ You’re not getting 110gb of vram with this. Also, what hard limit? Now you’ve lost your price to performance too, more heat, more complexity, and less efficient. Also 150 tops is a lot less than the tiny box can offer.

@anthonieisacnt · 2025-02-27 04:47
@botkooper @Bithunters @_roberth_ @geerlingguy @__tinygrad__ You’re not getting 110gb of vram with this. Also, what hard limit? Now you’ve lost your price to performance too, more heat, more complexity, and less efficient. Also 150 tops is a lot less than the tiny box can offer.

@__tinygrad__ · 2025-02-27 05:54
@anthonieisacnt @botkooper @Bithunters @_roberth_ @geerlingguy According to these people, all our business already vanished with the golden DIGITS box 😂 I'm gonna take a Celeron, put 128 GB of RAM in the box, write AI on the side, and spray paint it platinum.

@__tinygrad__ · 2025-02-27 06:05
Okay okay hear me out. Take a low-end Celeron processor, shove 128GB of RAM in the box, write AI in big letters on the side, and spray paint it platinum. I hear this is a market segment we *have* to compete in. https://t.co/jKrpWs3TTs

@__tinygrad__ · 2025-02-27 06:05
Okay okay hear me out. Take a low-end Celeron processor, shove 128GB of RAM in the box, write AI in big letters on the side, and spray paint it platinum. I hear this is a market segment we *have* to compete in. https://t.co/jKrpWs3TTs

@RohithThakurwar · 2025-02-27 06:07
@__tinygrad__ lol FrameworkPuter

@RohithThakurwar · 2025-02-27 06:07
@__tinygrad__ lol FrameworkPuter

@__tinygrad__ · 2025-02-27 06:10
@RohithThakurwar With our new 1-bit dtype, we get 1.21 jiggaAI TOPS! You can run LLaMA 3 8B quantized at 3 toks/sec and also fit 13 additional copies of it in RAM.

@__tinygrad__ · 2025-02-27 06:05
Okay okay hear me out. Take a low-end Celeron processor, shove 128GB of RAM in the box, write AI in big letters on the side, and spray paint it platinum. I hear this is a market segment we *have* to compete in. https://t.co/jKrpWs3TTs

@study_of_waifu · 2025-02-27 06:13
@__tinygrad__ Honestly I think they're going after Apple's unified RAM and not necessarily the high bandwidth setups like TinyBox. So not quite the same.

@__tinygrad__ · 2025-02-27 06:05
Okay okay hear me out. Take a low-end Celeron processor, shove 128GB of RAM in the box, write AI in big letters on the side, and spray paint it platinum. I hear this is a market segment we *have* to compete in. https://t.co/jKrpWs3TTs

@nlevnaut · 2025-02-27 06:17
@__tinygrad__ deal, but you better remember to solder the RAM in

@study_of_waifu · 2025-02-27 06:13
@__tinygrad__ Honestly I think they're going after Apple's unified RAM and not necessarily the high bandwidth setups like TinyBox. So not quite the same.

@__tinygrad__ · 2025-02-27 06:17
@study_of_waifu Nah we actually like @FrameworkPuter, and their marketing is tasteful. NVIDIA DIGITS less so. But that chip is half the bandwidth of my Apple laptop, and 1/7x of the bandwidth of a single 5090, and I feel a lot of people don't understand how to evaluate these things.

@__tinygrad__ · 2025-02-27 06:10
@RohithThakurwar With our new 1-bit dtype, we get 1.21 jiggaAI TOPS! You can run LLaMA 3 8B quantized at 3 toks/sec and also fit 13 additional copies of it in RAM.

@mbalint · 2025-02-27 06:20
@__tinygrad__ @RohithThakurwar How many TOPS for 0.15-bit dtype?

@nlevnaut · 2025-02-27 06:17
@__tinygrad__ deal, but you better remember to solder the RAM in

@__tinygrad__ · 2025-02-27 06:25
@nlevnaut we will cover in epoxy https://t.co/7xO89EuW7r

@mbalint · 2025-02-27 06:20
@__tinygrad__ @RohithThakurwar How many TOPS for 0.15-bit dtype?

@__tinygrad__ · 2025-02-27 06:26
@mbalint @RohithThakurwar You have to buy next years model to support the smaller dtype.

@petzlux · 2025-02-27 06:14
@__tinygrad__ Honest question what's my best price / performance option for local inference on biggish LLMs ? Mac mini ?

@__tinygrad__ · 2025-02-27 06:36
@petzlux EPYC

@__tinygrad__ · 2025-02-27 06:54
If you are serious about cost effective LLM at home, how about a 25 tok/s full 8-bit DeepSeek-R1 Box. Dual 16 core EPYC Turin (AVX-512) 768GB RAM (6x Framework/DIGITS) 1152 GB/s (4x Framework/DIGITS) $10,000 Are you buying?

@__tinygrad__ · 2025-02-27 06:36
@petzlux EPYC

@__tinygrad__ · 2025-02-27 06:56
@petzlux This is what I would build. If the poll shows demand I'll put up non-refundable $1k preorders. 25 bites on that and we'll build. https://t.co/SuoEwCmEEI

@__tinygrad__ · 2025-02-27 06:54
If you are serious about cost effective LLM at home, how about a 25 tok/s full 8-bit DeepSeek-R1 Box. Dual 16 core EPYC Turin (AVX-512) 768GB RAM (6x Framework/DIGITS) 1152 GB/s (4x Framework/DIGITS) $10,000 Are you buying?

@_imdawon · 2025-02-27 06:58
@__tinygrad__ $10K tinybox pls 😭

@_imdawon · 2025-02-27 06:58
@__tinygrad__ $10K tinybox pls 😭

@__tinygrad__ · 2025-02-27 06:58
@_imdawon If the poll shows demand I'll put up non-refundable $1k preorders. 25 bites on that and we'll build.

@__tinygrad__ · 2025-02-27 06:54
If you are serious about cost effective LLM at home, how about a 25 tok/s full 8-bit DeepSeek-R1 Box. Dual 16 core EPYC Turin (AVX-512) 768GB RAM (6x Framework/DIGITS) 1152 GB/s (4x Framework/DIGITS) $10,000 Are you buying?

@Sentdex · 2025-02-27 06:59
@__tinygrad__ Id be interested in such a thing.

@__tinygrad__ · 2025-02-27 06:59
@cafeinomano__ We're working on it, LLVM is pretty good now. Aside from things like using the AMX, where do you see us being more than 2x off max theoretical?

@Sentdex · 2025-02-27 06:59
@__tinygrad__ Id be interested in such a thing.

@__tinygrad__ · 2025-02-27 07:01
@Sentdex Oh really? Right now, if you sent you a non-refundable $1k preorder (ships Q3, same as framework), would you do it?

@__tinygrad__ · 2025-02-27 06:58
@_imdawon If the poll shows demand I'll put up non-refundable $1k preorders. 25 bites on that and we'll build.

@_imdawon · 2025-02-27 07:01
@__tinygrad__ Hope people recognize the value. I didn’t want to hack around with AMD drivers for the RED tinybox, but can’t shell out $25,000 for the green one. This would solve both problems

@_imdawon · 2025-02-27 07:01
@__tinygrad__ Hope people recognize the value. I didn’t want to hack around with AMD drivers for the RED tinybox, but can’t shell out $25,000 for the green one. This would solve both problems

@__tinygrad__ · 2025-02-27 07:02
@_imdawon I mean, this is not a training computer, it's optimizing similar specs as these AI computers going around.

@__tinygrad__ · 2025-02-27 06:54
If you are serious about cost effective LLM at home, how about a 25 tok/s full 8-bit DeepSeek-R1 Box. Dual 16 core EPYC Turin (AVX-512) 768GB RAM (6x Framework/DIGITS) 1152 GB/s (4x Framework/DIGITS) $10,000 Are you buying?

@shlomiatar · 2025-02-27 07:05
@__tinygrad__ That’s actually a cool config. Memory is the most expensive part, and the dual socket Turin board will be an EATX one but yeah, this is hella cool. I recommend a define 7 case for ultimate quiet ops.

@shlomiatar · 2025-02-27 07:05
@__tinygrad__ That’s actually a cool config. Memory is the most expensive part, and the dual socket Turin board will be an EATX one but yeah, this is hella cool. I recommend a define 7 case for ultimate quiet ops.

@__tinygrad__ · 2025-02-27 07:09
@shlomiatar Will be a custom case, the dual socket mobos with all 24 DIMM slots aren't even EATX, they are weirder. And yea, our tinybox is near silent, this will be too.

@__tinygrad__ · 2025-02-27 07:01
@Sentdex Oh really? Right now, if you sent you a non-refundable $1k preorder (ships Q3, same as framework), would you do it?

@Sentdex · 2025-02-27 07:09
@__tinygrad__ Id be very tempted but I've also been trained well to not preorder a thing like this. Send me something to harrison@pythonprogramming.net and ill consider further.

@Sentdex · 2025-02-27 07:09
@__tinygrad__ Id be very tempted but I've also been trained well to not preorder a thing like this. Send me something to harrison@pythonprogramming.net and ill consider further.

@__tinygrad__ · 2025-02-27 07:12
@Sentdex Hmm, yea, this probably won't happen then. We really don't make much on it. But this is the sort of thing that's optimized for MoE LLMs at home.

@__tinygrad__ · 2025-02-27 06:54
If you are serious about cost effective LLM at home, how about a 25 tok/s full 8-bit DeepSeek-R1 Box. Dual 16 core EPYC Turin (AVX-512) 768GB RAM (6x Framework/DIGITS) 1152 GB/s (4x Framework/DIGITS) $10,000 Are you buying?

@abacaj · 2025-02-27 07:17
@__tinygrad__ Wait that sounds like a good deal > 3090 bandwidth and the ram is way up there

@abacaj · 2025-02-27 07:17
@__tinygrad__ Wait that sounds like a good deal > 3090 bandwidth and the ram is way up there

@__tinygrad__ · 2025-02-27 07:18
@abacaj If I posted a nonrefundable $1k preorder link (shipping in Q3, same as framework), would you buy?

@__tinygrad__ · 2025-02-27 06:54
If you are serious about cost effective LLM at home, how about a 25 tok/s full 8-bit DeepSeek-R1 Box. Dual 16 core EPYC Turin (AVX-512) 768GB RAM (6x Framework/DIGITS) 1152 GB/s (4x Framework/DIGITS) $10,000 Are you buying?

@keithofaptos · 2025-02-27 07:37
@__tinygrad__ You could tease us better. NPU's with lightspeed connections. W/a petabyte of vRAM. What'll that cost? lol

@keithofaptos · 2025-02-27 07:37
@__tinygrad__ You could tease us better. NPU's with lightspeed connections. W/a petabyte of vRAM. What'll that cost? lol

@__tinygrad__ · 2025-02-27 07:37
@keithofaptos No if people want it we'll actually sell this.

@__tinygrad__ · 2025-02-27 06:54
If you are serious about cost effective LLM at home, how about a 25 tok/s full 8-bit DeepSeek-R1 Box. Dual 16 core EPYC Turin (AVX-512) 768GB RAM (6x Framework/DIGITS) 1152 GB/s (4x Framework/DIGITS) $10,000 Are you buying?

@CustomWetware · 2025-02-27 07:40
@__tinygrad__ Yes I'm seriously considering something like that. And a few 5090s to crank up the compute and run a draft model really fast. But you won't get max bandwith with a 16 core turin unless you mean 9175F and I don't think that fits in the $10k budget. 9355 would be my pick.

@__tinygrad__ · 2025-02-27 06:54
If you are serious about cost effective LLM at home, how about a 25 tok/s full 8-bit DeepSeek-R1 Box. Dual 16 core EPYC Turin (AVX-512) 768GB RAM (6x Framework/DIGITS) 1152 GB/s (4x Framework/DIGITS) $10,000 Are you buying?

@DavidSHolz · 2025-02-27 07:42
@__tinygrad__ does the cost go down much if you use a more aggressively quantized r1 model? https://t.co/4smw8p5Hg5

@CustomWetware · 2025-02-27 07:40
@__tinygrad__ Yes I'm seriously considering something like that. And a few 5090s to crank up the compute and run a draft model really fast. But you won't get max bandwith with a 16 core turin unless you mean 9175F and I don't think that fits in the $10k budget. 9355 would be my pick.

@__tinygrad__ · 2025-02-27 07:43
@CustomWetware Wait really? 9115 can't get all the bandwidth? If not, we'll have to go Genoa. The other Turin are too much money.

@__tinygrad__ · 2025-02-27 07:43
@CustomWetware Wait really? 9115 can't get all the bandwidth? If not, we'll have to go Genoa. The other Turin are too much money.

@CustomWetware · 2025-02-27 07:44
@__tinygrad__ It only has 2 CCDs. It needs at least 8.

@DavidSHolz · 2025-02-27 07:42
@__tinygrad__ does the cost go down much if you use a more aggressively quantized r1 model? https://t.co/4smw8p5Hg5

@__tinygrad__ · 2025-02-27 07:46
@DavidSHolz RAM is the biggest cost. For DDR5-6400, the 24 sticks of 32GB cost $4272. Though I have some doubts about quantization that low, 4-bit is probably minimum. Half the RAM cost, double the tok/s

@CustomWetware · 2025-02-27 07:44
@__tinygrad__ It only has 2 CCDs. It needs at least 8.

@__tinygrad__ · 2025-02-27 07:50
@CustomWetware Ahh, okay. Is that true on Genoa as well? The 4x CCD ones don't get full bandwidth?

@CustomWetware · 2025-02-27 07:52
@__tinygrad__ Yes, all benchmarks I have seen shows that <8 CCD models are unable to reach AMD's claimed 460 GB/s 'theoretical' bandwidth per socket. Same with Threadripper.

@__tinygrad__ · 2025-02-27 07:58
@CustomWetware Interesting. It's not just the CCDs, the frequency seems to matter too, see 9224 vs 9254. https://t.co/5lJ37rUrPH

@__tinygrad__ · 2025-02-27 07:58
@CustomWetware Interesting. It's not just the CCDs, the frequency seems to matter too, see 9224 vs 9254. https://t.co/5lJ37rUrPH

@__tinygrad__ · 2025-02-27 08:01
@CustomWetware Turin benchmarks here, you are right 9015/9115 are awful. https://t.co/n1wtP3TIMY

@__tinygrad__ · 2025-02-27 08:13
@farajalrashidi @petzlux No no, I don't want a kit. I want an assembled and tested system, and I want it to be quiet and in a nice case. And I want some guarantee you aren't going to rug me, maybe after you raise and get a few more followers. Price still $8k?

@im_roy_lee · 2025-02-28 00:54
amazon execs r so mad LOLLL maybe stop asking dumb interview questions and people wouldn't build shit like this https://t.co/RMwouy3enr https://t.co/sBdWTx2aFJ

@__tinygrad__ · 2025-02-28 06:29
We have 6 tinybox reds in stock. With the tinygrad PyTorch backend combined with our AM driver, the experience is starting to be decent. We'll be putting serious effort into this over the next couple months. An AMD stack totally free of AMD software.

@__tinygrad__ · 2025-02-28 06:29
We have 6 tinybox reds in stock. With the tinygrad PyTorch backend combined with our AM driver, the experience is starting to be decent. We'll be putting serious effort into this over the next couple months. An AMD stack totally free of AMD software.

@KimNoel · 2025-02-28 06:34
@__tinygrad__ If red and green both work with tinygrad, why people prefer green over red? Hardware performance differences?

@KimNoel · 2025-02-28 06:34
@__tinygrad__ If red and green both work with tinygrad, why people prefer green over red? Hardware performance differences?

@__tinygrad__ · 2025-02-28 06:37
@kimnoel Not everyone uses tinygrad. The greens are compatible with all the software written for torch/cuda. Our PyTorch backend is a start at closing this gap, we need to match performance though. Doable this year.

@highyieldYT · 2025-02-28 15:59
Initial @amdradeon Navi 48 chip analysis. AMD really outdid themself when it comes to density. Lot's of transistors in a small space. https://t.co/qcmhr1lwDD

@__tinygrad__ · 2025-03-02 01:04
There's $1200 worth of bounties open on the torch backend, fixing hlb-CIFAR10 (might be really easy), moving a few things off the CPU, supporting torch.compile, and MultiGPU training. https://t.co/N9GQVB9zDr

@__tinygrad__ · 2025-03-02 01:04
There's $1200 worth of bounties open on the torch backend, fixing hlb-CIFAR10 (might be really easy), moving a few things off the CPU, supporting torch.compile, and MultiGPU training. https://t.co/N9GQVB9zDr

@__tinygrad__ · 2025-03-02 02:42
$400 more to get gpt-fast outperforming the ROCm backend with AM + tiny backend. If you clean up our torch.compile stuff it should be doable + you can grab the MNIST one. Every day, we make a little progress. With great testing and CI we don't slip backward. Eventually, we win.

@francoisfleuret · 2025-03-02 08:23
Ahahaaaaaahhhaaaaaaa https://t.co/0oOM6DKAsL

@__tinygrad__ · 2025-03-02 12:44
Please use tools to try to cheat our interview process. Then come work here, and use tools to cheat at that too. All I hear is productivity. But plz clean up the AI slop so I can't tell it's AI slop. Pass the Turing Test.

@radshaan · 2025-03-02 18:18
So if a company gets more than 100K job applications, how else are they supposed to filter people?

@nlevnaut · 2025-03-02 18:42
@radshaan job applications will become a thing of the past see @__tinygrad__'s approach

@__tinygrad__ · 2025-03-03 01:02
Solve bounties! Do the job to prove you can do the job, and get paid while you are doing it. Bounties don't work for everything; refactors need to be done by full time people, and that's why we hire. But how you solve the bounties is an amazing reflection of your skill.

@__tinygrad__ · 2025-03-03 06:55
@srivatsamath They also have the added benefit of screening out people who wouldn't be a good fit. Nothing "guarantees employment", at least we basically pay you for the interview. All of tech is going to go this way.

@francoisfleuret · 2025-03-02 08:23
Ahahaaaaaahhhaaaaaaa https://t.co/0oOM6DKAsL

@__tinygrad__ · 2025-03-03 08:29
@francoisfleuret https://t.co/LDrFcTzwFr

@__tinygrad__ · 2025-03-03 11:45
What is tinygrad? tinygrad is a formalist project. It attempts to capture the full gamut of software 2.0 in a non leaky abstraction. The methods on Tensor class create a directed graph of immutable RISC UOps defining what the computation is. Tensor is a frontend, in addition we have an ONNX frontend and a PyTorch frontend. Whether you code in tinygrad, torch, or import ONNX models, it all boils down to the same very simple UOp graph, which you can see with VIZ=1 This graph contains nothing like matmul or conv, it's just movement ops, elementwise ops, and reduction ops. Seriously, try VIZ=1. Below that, there's a scheduler which breaks that graph up into kernels. Then we do more graph transforms on each kernel subgraph until we have code which can run on an accelerator. See the kernels with DEBUG=2. Then we have runtimes capable of running that code. For AMD, our runtime goes all the way to the physical hardware; we are mmaping the PCIe bus and peeking and poking it. It's all in Python, but it is fast because once you have the graph compiled, you are running the same graph over and over; just ringing a doorbell. The hope is that, similar to Linux and LLVM, we will prevent a major source of rent seeking in our AI future. By clearly and simply specifying the job, being able to precisely spec what is bought and sold, you can have a fair marketplace for compute. By the end of the year, we should be similar in speed on NVIDIA to the existing torch CUDA backend, except without CUDA. We will also have a test cloud up where you can run jobs from any of the three frontends. You don't want to rent a GPU per hour on a machine, you want to rent a couple FLOPS in a lambda function. That's what the OpenAI API is. Now offer it decoupled from the specific model.

@doodlestein · 2025-03-03 13:23
If Hotz can actually pull this off and get the same performance via PyTorch as CUDA but without actually using CUDA, it opens the floodgates to alternative hardware (especially AMD’s), particularly on the inference side. There’s no fundamental reason why CUDA must remain a moat.

@__tinygrad__ · 2025-03-03 13:50
Meeting in 10 minutes on our Discord. Every Monday we have our one weekly company meeting, and all can listen.

@doodlestein · 2025-03-03 13:23
If Hotz can actually pull this off and get the same performance via PyTorch as CUDA but without actually using CUDA, it opens the floodgates to alternative hardware (especially AMD’s), particularly on the inference side. There’s no fundamental reason why CUDA must remain a moat.

@__tinygrad__ · 2025-03-03 13:51
@doodlestein Exactly. Too bad AMD wasn't very interested. We got a much warmer reception from Intel, announcement next week.

@__tinygrad__ · 2025-03-03 22:44
Turns out Intel does make big GPUs! 832 BF16 TFLOPS, 128GB, 3.28 TB/s. Each. https://t.co/CnlYjviJfV

@__tinygrad__ · 2025-03-03 22:44
Turns out Intel does make big GPUs! 832 BF16 TFLOPS, 128GB, 3.28 TB/s. Each. https://t.co/CnlYjviJfV

@sipmine · 2025-03-03 22:45
@__tinygrad__ what is their power consumption?

@sipmine · 2025-03-03 22:45
@__tinygrad__ what is their power consumption?

@__tinygrad__ · 2025-03-03 22:47
@sipmine 600W, similar to a 5090

@__tinygrad__ · 2025-03-04 02:10
Its little brother is already up and working. Will take work to use XMX and make really fast, but already everything is correct and usable. PyTorch -> tinygrad -> OpenCL -> i915 https://t.co/AX8qRD5gTM

@ludwigABAP · 2025-03-04 10:24
I want to work on a tenstorrent backend for tinygrad but my TT LoudBOx is like a month away. I’ve started speccing it out and implementing it blind without the actual hardware last night. Pray 4 me.

@ludwigABAP · 2025-03-04 10:24
I want to work on a tenstorrent backend for tinygrad but my TT LoudBOx is like a month away. I’ve started speccing it out and implementing it blind without the actual hardware last night. Pray 4 me.

@__tinygrad__ · 2025-03-05 01:12
@ludwigABAP .@tenstorrent Do you have a simulator? We really like our backends to have one anyway, we have a sim for NVIDIA (ptx) and AMD (RDNA3) GPUs that runs in GitHub actions.

@__tinygrad__ · 2025-03-05 10:30
Soon tinygrad is going to support RDNA3 GPU connected over USB 3 to any device, including MacBooks. This is the kind of stuff you can do when you have a simple sovereign AMD stack. https://t.co/WiK96ZYJZ2

@__tinygrad__ · 2025-03-05 10:30
Soon tinygrad is going to support RDNA3 GPU connected over USB 3 to any device, including MacBooks. This is the kind of stuff you can do when you have a simple sovereign AMD stack. https://t.co/WiK96ZYJZ2

@_shanytc · 2025-03-05 10:37
@__tinygrad__ Nice. But it's still sloooowwww for ML/DL stack

@_shanytc · 2025-03-05 10:37
@__tinygrad__ Nice. But it's still sloooowwww for ML/DL stack

@__tinygrad__ · 2025-03-05 10:43
@_shanytc 5 GB/s for any transfers, I doubt you'll ever really notice. https://t.co/VKI7Swb2BJ

@typedfemale · 2025-03-07 01:59
tinygrad users don't know the value of nothing and the cost of nothing https://t.co/0xNoVZskbP

@ZDi____ · 2025-03-07 04:31
rocm-smi

@typedfemale · 2025-03-07 01:59
tinygrad users don't know the value of nothing and the cost of nothing https://t.co/0xNoVZskbP

@__tinygrad__ · 2025-03-07 17:39
@typedfemale Don't worry, they aren't going away.

@__tinygrad__ · 2025-03-07 17:49
We are going to commoditize the petaflop

@ZDi____ · 2025-03-07 17:53
Nevermind, PyTorch w/RoCM sucks. Segfaults followed by extremely slow training after troubleshooting, even when using one of the official docker images. Tinygrad dude was right lmao.

@ZDi____ · 2025-03-07 17:53
Nevermind, PyTorch w/RoCM sucks. Segfaults followed by extremely slow training after troubleshooting, even when using one of the official docker images. Tinygrad dude was right lmao.

@__tinygrad__ · 2025-03-07 18:11
@ZDi____ Try tinygrad.

@smallshen2 · 2025-03-08 00:45
Holy shit tinygrad is so fucking slow

@__tinygrad__ · 2025-03-08 01:23
RT @AnushElangovan: AMD 💕 @__tinygrad__ we are looking forward to working closely with @__tinygrad__ to help commoditize the petaflop https://t.co/LEjsUaPWHV

@__tinygrad__ · 2025-03-08 04:43
Unboxing https://t.co/V6xUXenPgt

@jarrodxmartian · 2025-03-10 04:02
Has anyone started a backend for RKNN? I have been looking into it but hitting some walls

@jarrodxmartian · 2025-03-10 04:02
Has anyone started a backend for RKNN? I have been looking into it but hitting some walls

@__tinygrad__ · 2025-03-10 16:09
@jarrodxmartian The hardware is pretty well documented in the RK3588 datasheet.

@__tinygrad__ · 2025-03-11 00:03
Up! https://t.co/CSycke2hLN

@__tinygrad__ · 2025-03-12 15:46
RT @tobi1577: Revealing my latest project, a realtime-generated adventure game where you play with an AI dungeon master. Fully ascii-based, designed to be played through an SSH to a tinybox (from @__tinygrad__), more backends coming soon. https://t.co/X5FbA6R6Cr

@__tinygrad__ · 2025-03-13 12:54
RT @FeepingCreature: Out of interest I got an AI to write me a script to graph #tinygrad AMD SDXL performance over time, sadly Github artifacts don't go back farther than that. Still, clearly a good trend! https://t.co/zriXLWAzJn

@__tinygrad__ · 2025-03-13 13:05
This is the right question to ask. It's simple if interconnect bandwidth >= memory bandwidth. The MI300X is really 8 GPUs in one "chip", having fun figuring out how the queues work. https://t.co/mssIFNyTJ2

@__tinygrad__ · 2025-03-13 13:05
This is the right question to ask. It's simple if interconnect bandwidth >= memory bandwidth. The MI300X is really 8 GPUs in one "chip", having fun figuring out how the queues work. https://t.co/mssIFNyTJ2

@noah_vandal · 2025-03-13 13:57
@__tinygrad__ will tinygrad be able to run fine on these soon?

@noah_vandal · 2025-03-13 13:57
@__tinygrad__ will tinygrad be able to run fine on these soon?

@__tinygrad__ · 2025-03-13 15:15
@noah_vandal It already works through HIP, just slow! The AMD runtime + MFMA support will make it pretty fast, ETA 2 weeks.

@hornswoggle567 · 2025-03-13 19:44
As soon as you start playing with models locally you quickly come to a realization that you don't want to go through the painful process of using either Google or AWS servers. Then you go to @__tinygrad__ and see the green boxes are all sold out. Bullish on NVIDIA.

@hornswoggle567 · 2025-03-13 19:44
As soon as you start playing with models locally you quickly come to a realization that you don't want to go through the painful process of using either Google or AWS servers. Then you go to @__tinygrad__ and see the green boxes are all sold out. Bullish on NVIDIA.

@__tinygrad__ · 2025-03-14 00:12
@hornswoggle567 Have you considered a red box?

@__tinygrad__ · 2025-03-14 00:45
@PytorchToAtoms Yea it's very complex even compared to an H100. Interesting to see how simple the RDNA4 chip is. https://t.co/amyQ3w5BIs

@__tinygrad__ · 2025-03-14 14:42
The tinygrad AM driver is afaik the only way to profile AMD compute kernels on Linux. And you can run Radeon GPU Profiler in Wine on Mac to view the traces. See the docs in extra/sqtt/README.md https://t.co/NrFL20FFX2

@__tinygrad__ · 2025-03-14 14:54
@fma_f32 Err, and you can capture that from HIP on Linux?

@__tinygrad__ · 2025-03-14 15:28
RT @k7agar: never change tinygrad I love you https://t.co/V5B5LsgRMV

@__tinygrad__ · 2025-03-14 23:35
RT @kean00reeves: BFV FHE scheme purely in @__tinygrad__ https://t.co/5mqyjzLhRS in <100 lines https://t.co/8DYTyto7N1

@Teknium · 2025-03-16 12:30
Is 5090 ever going to be available lol

@Teknium · 2025-03-16 12:30
Is 5090 ever going to be available lol

@__tinygrad__ · 2025-03-16 16:06
@teknium Asking the same question.

@__tinygrad__ · 2025-03-17 16:06
Chips like the Qualcomm DSP, high width low core SIMD, are the ultimate challenge to use. We're making progress, 5x faster than 2 months ago, 25x more possible. Here's MobileNetV2 on 845. https://t.co/e5cyhQUOmf

@__tinygrad__ · 2025-03-17 16:06
Chips like the Qualcomm DSP, high width low core SIMD, are the ultimate challenge to use. We're making progress, 5x faster than 2 months ago, 25x more possible. Here's MobileNetV2 on 845. https://t.co/e5cyhQUOmf

@comma_ai · 2025-03-17 16:12
@__tinygrad__ 👀

@__tinygrad__ · 2025-03-17 16:19
@datamansir It compiles on the device too with normal clang, no hexagon SDK needed anywhere.

@comma_ai · 2025-03-17 16:12
@__tinygrad__ 👀

@__tinygrad__ · 2025-03-17 16:21
@comma_ai We're trying...GPUs are so much easier. GPU: floats, 32x SIMT (you don't even need to know what memory coalescing is), ~high core counts, large L2 cache, and warp scheduling DSP: int8, 128x SIMD (u like weird shuffle unit?), two cores, strange SRAM, and triple issue VLIW

@notjeanmarc · 2025-03-19 17:53
"and soon they'll support a full GPU plugged into a comma 3X." Me: I'm leaving now for my business trip. Wife: *kiss* don't forget your 4090 for a chill ride.

@__tinygrad__ · 2025-03-21 15:08
How do people feel about `z3-solver` as a tinygrad dependency?

@__tinygrad__ · 2025-03-22 03:17
It only supports AMD.

@__tinygrad__ · 2025-03-25 02:52
tinygrad is adding support for the Qualcomm DSP. It's nice how simple the chip is, you see the instructions and you see how to write a core like this in an FPGA. https://t.co/Ai0X3WX9IO

@__tinygrad__ · 2025-03-27 03:22
This is an auto generated DSP 3x3 depthwise conv with stride 2, channels chunked to 32 and filters padded to 4 on X. The codegen is not done if the code isn't readable and pretty. Beautiful kernels are fast kernels. https://t.co/tRl4Dl1QkH

@__tinygrad__ · 2025-03-27 03:22
This is an auto generated DSP 3x3 depthwise conv with stride 2, channels chunked to 32 and filters padded to 4 on X. The codegen is not done if the code isn't readable and pretty. Beautiful kernels are fast kernels. https://t.co/tRl4Dl1QkH

@morew4rd · 2025-03-27 04:34
@__tinygrad__ what's the source that generated this?

@__tinygrad__ · 2025-03-27 12:09
RT @BrennanOwain: I am no longer GPU poor, time for growth https://t.co/O5xkNYIswE

@__tinygrad__ · 2025-03-27 12:10
red in stock. green v2 begins shipping in May. https://t.co/AiOVFVp6oq

@__tinygrad__ · 2025-03-27 12:10
red in stock. green v2 begins shipping in May. https://t.co/AiOVFVp6oq

@ra_czq · 2025-03-27 12:14
@__tinygrad__ just out of curiosity, have you played with Rx 9070 XT yet?

@BrennanOwain · 2025-03-27 09:42
I am no longer GPU poor, time for growth https://t.co/O5xkNYIswE

@BattousaiHBr · 2025-03-27 12:15
@BrennanOwain @__tinygrad__ @realGeorgeHotz that's some damn expensive shipping, god damn.

@BattousaiHBr · 2025-03-27 12:15
@BrennanOwain @__tinygrad__ @realGeorgeHotz that's some damn expensive shipping, god damn.

@__tinygrad__ · 2025-03-27 12:16
@BattousaiHBr @BrennanOwain @realGeorgeHotz International + it's a 90 lb box!

@ra_czq · 2025-03-27 12:14
@__tinygrad__ just out of curiosity, have you played with Rx 9070 XT yet?

@__tinygrad__ · 2025-03-27 12:17
@ra_czq Yea, for a midrange GPU it's decent. If we sell out of 7900XTX red boxes we'll make a V2 with it.

@morew4rd · 2025-03-27 04:34
@__tinygrad__ what's the source that generated this?

@__tinygrad__ · 2025-03-27 13:08
@morew4rd https://t.co/WfTgzts501

@__tinygrad__ · 2025-03-27 13:19
RT @szeloof: To make new things you need to care You need to care so much Otherwise it’s just slop

@highyieldYT · 2025-02-28 15:59
Initial @amdradeon Navi 48 chip analysis. AMD really outdid themself when it comes to density. Lot's of transistors in a small space. https://t.co/qcmhr1lwDD

@__tinygrad__ · 2025-03-27 13:42
@highyieldYT @amdradeon I like how cheap these look to make.

@__tinygrad__ · 2025-03-28 01:55
tinybox green v2, reporting for duty 🫡 https://t.co/jdTx8MPr5G

@seveibar · 2025-03-28 02:16
1. Know A* like the back of your hand, use it everywhere If I was king for a day, I would rename A* to “Fundamental Algorithm”. It is truly one of the most adaptable and important algorithms for _any kind_ of search. It is simply the best foundation for any kind of informed search (not just for 2d grids!)

@seveibar · 2025-03-28 02:16
2. Implementation Language doesn’t matter The difference between a smart algorithm and the dumb algorithm is 1000x, whatever gets you to the smartest, most cacheable algorithm fastest is the best language

@__tinygrad__ · 2025-03-28 01:55
tinybox green v2, reporting for duty 🫡 https://t.co/jdTx8MPr5G

@OrganicGPT · 2025-03-28 02:17
@__tinygrad__ why not use RTX 6000 Pro?

@__tinygrad__ · 2025-03-28 01:55
tinybox green v2, reporting for duty 🫡 https://t.co/jdTx8MPr5G

@JaimeOrtega · 2025-03-28 02:19
@__tinygrad__ I would prompt images so beautiful with this much power

@__tinygrad__ · 2025-03-28 02:27
green v2 is up on the website. 4x5090, PCIe5, a bump to GENOA, and 192GB of RAM. Case looks same. No preorders this time, if you want to send full money you'll be in line to ship when we get GPUs, otherwise sign up for the mailing list to get an in-stock notification. https://t.co/OZ6UloW00U

@OrganicGPT · 2025-03-28 02:17
@__tinygrad__ why not use RTX 6000 Pro?

@__tinygrad__ · 2025-03-28 02:28
@OrganicGPT Do you want to pay $50k for the machine? These are workstations, not for use in datacenter.

@__tinygrad__ · 2025-03-28 01:55
tinybox green v2, reporting for duty 🫡 https://t.co/jdTx8MPr5G

@YasserManss · 2025-03-28 02:38
@__tinygrad__ What an accomplishment, you managed to install the Nvidia driver!

@__tinygrad__ · 2025-03-28 02:28
@OrganicGPT Do you want to pay $50k for the machine? These are workstations, not for use in datacenter.

@OrganicGPT · 2025-03-28 02:38
@__tinygrad__ 4×5090 (at MSRP) = 128GB no nvlink = $8,000 1×RTX-6000-Pro = 96GB at full bandwidth ≃ $7800 (https://t.co/sw79bfQFJR)

@OrganicGPT · 2025-03-28 02:38
@__tinygrad__ 4×5090 (at MSRP) = 128GB no nvlink = $8,000 1×RTX-6000-Pro = 96GB at full bandwidth ≃ $7800 (https://t.co/sw79bfQFJR)

@__tinygrad__ · 2025-03-28 02:39
@OrganicGPT What about FLOPS? What about RAM bandwidth? Overpriced.

@YasserManss · 2025-03-28 02:38
@__tinygrad__ What an accomplishment, you managed to install the Nvidia driver!

@__tinygrad__ · 2025-03-28 02:40
@YasserManss It's literally just `sudo apt-get install nvidia-open cuda-toolkit`

@__tinygrad__ · 2025-03-28 02:40
@YasserManss It's literally just `sudo apt-get install nvidia-open cuda-toolkit`

@YasserManss · 2025-03-28 02:42
@__tinygrad__ Wasn’t as straightforward in RHEL.

@YasserManss · 2025-03-28 02:42
@__tinygrad__ Wasn’t as straightforward in RHEL.

@__tinygrad__ · 2025-03-28 02:55
@YasserManss Yea it used to be a much bigger pain in Ubuntu too, and the docs are still not good.

@JaimeOrtega · 2025-03-28 02:19
@__tinygrad__ I would prompt images so beautiful with this much power

@__tinygrad__ · 2025-03-28 03:15
@JaimeOrtega Can be yours for $25k. Here it is being stressed https://t.co/Zpks4HF9sQ

@__tinygrad__ · 2025-03-28 02:27
green v2 is up on the website. 4x5090, PCIe5, a bump to GENOA, and 192GB of RAM. Case looks same. No preorders this time, if you want to send full money you'll be in line to ship when we get GPUs, otherwise sign up for the mailing list to get an in-stock notification. https://t.co/OZ6UloW00U

@PradeepJag123 · 2025-03-28 03:22
@__tinygrad__ Maybe it would be better, if you can add, what models can be run locally and at what spec for each of those. It provides a better understanding of the capabilities.

@__tinygrad__ · 2025-03-28 03:24
@farajalrashidi You mean the $1999 joke? Of course we paid more than that.

@__tinygrad__ · 2025-03-28 05:00
Out of the box, 0 code changes, tinygrad gets 89 tok/s on FP16 Llama-3-8B on a 5090. On torch nightly, gpt-fast gets <exception> with --compile, and 14 tok/s without. This is why tinygrad will win. It's not about the benchmark, it's about being decent everywhere out of the box. https://t.co/xOqurlIfcV

@__tinygrad__ · 2025-03-28 05:33
5090 memory bandwidth. Stated 1792 GB/s, tested @ 1956 GB/s. Confirmed not a scam. https://t.co/jTKXIW9Nt9

@__tinygrad__ · 2025-03-28 05:33
5090 memory bandwidth. Stated 1792 GB/s, tested @ 1956 GB/s. Confirmed not a scam. https://t.co/jTKXIW9Nt9

@francoisfleuret · 2025-03-28 05:48
@__tinygrad__ So we like the 5090 very much?

@__tinygrad__ · 2025-03-28 05:00
Out of the box, 0 code changes, tinygrad gets 89 tok/s on FP16 Llama-3-8B on a 5090. On torch nightly, gpt-fast gets <exception> with --compile, and 14 tok/s without. This is why tinygrad will win. It's not about the benchmark, it's about being decent everywhere out of the box. https://t.co/xOqurlIfcV

@demisdev · 2025-03-28 06:13
@__tinygrad__ This sounds slow. Do the tinygrad servers run ollama? If so you should post those numbers instead so everyone can relate and compare them.

@__tinygrad__ · 2025-03-28 01:55
tinybox green v2, reporting for duty 🫡 https://t.co/jdTx8MPr5G

@Leik0w0 · 2025-03-28 06:33
@__tinygrad__ I’m impressed by the gpu holding < 80degrees even while drawing 575W Can’t keep my gpus below 80 at 400W …

@__tinygrad__ · 2025-03-28 05:00
Out of the box, 0 code changes, tinygrad gets 89 tok/s on FP16 Llama-3-8B on a 5090. On torch nightly, gpt-fast gets <exception> with --compile, and 14 tok/s without. This is why tinygrad will win. It's not about the benchmark, it's about being decent everywhere out of the box. https://t.co/xOqurlIfcV

@WillimerTercero · 2025-03-28 06:42
@__tinygrad__ If you want to be honest, which I believe you want and should, then you can't compare a long text inference (gpt-fast) against your 5 words inference. Speed decays with content, let's see how fast you handle self attention

@JeanSimp24 · 2025-03-28 06:44
@_shanytc @__tinygrad__ This is a full BF16 model though, not a quantization

@_shanytc · 2025-03-28 06:56
@JeanSimp24 @__tinygrad__ True. But the end user doesn't care. 😆

@_shanytc · 2025-03-28 06:56
@JeanSimp24 @__tinygrad__ True. But the end user doesn't care. 😆

@__tinygrad__ · 2025-03-28 07:09
@_shanytc @JeanSimp24 tinygrad is a framework for deep learning, LLMs just happen to be something it can do. If you want to mess with GGUF_6B_2_QUANT_SPEEDHAX3 be my guest.

@demisdev · 2025-03-28 06:13
@__tinygrad__ This sounds slow. Do the tinygrad servers run ollama? If so you should post those numbers instead so everyone can relate and compare them.

@__tinygrad__ · 2025-03-28 07:12
@demisbellot This is the mentality that loses in the long run. The box is a 5090 with perfect connectivity and power, any off the shelf ollama benchmark applies. 80% of theoretical max perf should be fine for anyone. If you want to sacrifice generality for 15% imo that's a bad trade.

@__tinygrad__ · 2025-03-28 07:16
@DaemonES Exactly. idk what they are running, but it isn't Llama-3-8B.

@francoisfleuret · 2025-03-28 05:48
@__tinygrad__ So we like the 5090 very much?

@__tinygrad__ · 2025-03-28 07:17
@francoisfleuret It's great hardware, sadly still nerfed. AMD needs to put enough pressure on NVIDIA for them to remove the FP16 with FP32 accumulation half speed nerf and fake P2P disable.

@WillimerTercero · 2025-03-28 06:42
@__tinygrad__ If you want to be honest, which I believe you want and should, then you can't compare a long text inference (gpt-fast) against your 5 words inference. Speed decays with content, let's see how fast you handle self attention

@__tinygrad__ · 2025-03-28 07:21
@WillimerTercero 85 tok/s at the end of 200, so 87 is the apples to apples. https://t.co/QOrxv1Wjjy

@__tinygrad__ · 2025-03-28 07:12
@demisbellot This is the mentality that loses in the long run. The box is a 5090 with perfect connectivity and power, any off the shelf ollama benchmark applies. 80% of theoretical max perf should be fine for anyone. If you want to sacrifice generality for 15% imo that's a bad trade.

@demisdev · 2025-03-28 07:24
@__tinygrad__ Whose sacrificing generality? Just asked if tinygrad supports the most popular way to run local llms with ollama? and if it does please post those results as well so we can compare it vs our local setups If I get an AI server it'll primarily be to run ollama, expect most will

@Leik0w0 · 2025-03-28 06:33
@__tinygrad__ I’m impressed by the gpu holding < 80degrees even while drawing 575W Can’t keep my gpus below 80 at 400W …

@__tinygrad__ · 2025-03-28 07:25
@Leik0w0 Final box thermals should even be better than this once we optimize. It's a big box with 9 (almost silent) fans, it should be very good thermally.

@__tinygrad__ · 2025-03-28 07:25
@Leik0w0 Final box thermals should even be better than this once we optimize. It's a big box with 9 (almost silent) fans, it should be very good thermally.

@Leik0w0 · 2025-03-28 07:26
@__tinygrad__ Will be perfect then. Will it be 4x 5090 or 6x ? Also what kind of flops are you hitting on a single card with tinygrad rn ? fp16 /w fp32acc

@demisdev · 2025-03-28 07:24
@__tinygrad__ Whose sacrificing generality? Just asked if tinygrad supports the most popular way to run local llms with ollama? and if it does please post those results as well so we can compare it vs our local setups If I get an AI server it'll primarily be to run ollama, expect most will

@__tinygrad__ · 2025-03-28 07:27
@demisbellot It's a Ubuntu computer with 4x 5090s. Perfect connectivity, power, and cooling. After you buy it, you can install whatever you want on it. If some tinygrad contributor is bored and wants access to benchmark common things, post in Discord.

@Leik0w0 · 2025-03-28 07:26
@__tinygrad__ Will be perfect then. Will it be 4x 5090 or 6x ? Also what kind of flops are you hitting on a single card with tinygrad rn ? fp16 /w fp32acc

@__tinygrad__ · 2025-03-28 07:31
@Leik0w0 4x5090. Seeing 170 TFLOPS FP16 w FP32 acc on 4096x4096, but tinygrad isn't full speed at these yet cause we don't support locals and "double buffering". https://t.co/KGZRPO3zx0

@__tinygrad__ · 2025-03-28 07:27
@demisbellot It's a Ubuntu computer with 4x 5090s. Perfect connectivity, power, and cooling. After you buy it, you can install whatever you want on it. If some tinygrad contributor is bored and wants access to benchmark common things, post in Discord.

@demisdev · 2025-03-28 07:33
@__tinygrad__ Weird pitch, but ok. Honestly just trying to provide constructive feedback on your marketing numbers which sound low for an 8b model, I'm assuming it's not apples V apples because I expect more from 4x 5090s with "Perfect connectivity, power, and cooling"

@PradeepJag123 · 2025-03-28 03:22
@__tinygrad__ Maybe it would be better, if you can add, what models can be run locally and at what spec for each of those. It provides a better understanding of the capabilities.

@__tinygrad__ · 2025-03-28 07:34
@PradeepJag123 Unlike many others, we aren't selling an application box. We are selling a box with raw power. Applications will come and go over the next 10 years, PFLOPS and TB/s won't.

@demisdev · 2025-03-28 07:33
@__tinygrad__ Weird pitch, but ok. Honestly just trying to provide constructive feedback on your marketing numbers which sound low for an 8b model, I'm assuming it's not apples V apples because I expect more from 4x 5090s with "Perfect connectivity, power, and cooling"

@__tinygrad__ · 2025-03-28 07:35
@demisbellot This isn't marketing or a pitch. It's just being excited about the computer. I have a 4x5090 box. It's fun.

@__tinygrad__ · 2025-03-28 07:36
We aren't selling an application box. We are selling a box with raw power. Applications will come and go over the next 10 years, PFLOPS and TB/s won't.

@ivanfioravanti · 2025-03-28 07:38
@__tinygrad__ @demisbellot 😤 I was evaluating a tinybox, I will keep evaluating it after this. If you think you can't be bored by standard benchmarks, we bored people use, ok 🤷🏻‍♂️

@__tinygrad__ · 2025-03-28 07:40
@ivanfioravanti @demisbellot imo, and i have a lot of data on this, the people who want ollama benchmarks are never the people who actually click buy. they are the type who adds the parts up on newegg and posts the cart total and say they'd never pay someone more than $50 to put a computer together for them

@__tinygrad__ · 2025-03-28 07:35
@demisbellot This isn't marketing or a pitch. It's just being excited about the computer. I have a 4x5090 box. It's fun.

@demisdev · 2025-03-28 07:42
@__tinygrad__ Great, than why not post numbers most potential customers would want to know about? e.g. how well it runs local ollama models. You make it sound I should feel bad for even asking what perf to expect for running its primary use-case? Weird strat.

@__tinygrad__ · 2025-03-28 07:40
@ivanfioravanti @demisbellot imo, and i have a lot of data on this, the people who want ollama benchmarks are never the people who actually click buy. they are the type who adds the parts up on newegg and posts the cart total and say they'd never pay someone more than $50 to put a computer together for them

@__tinygrad__ · 2025-03-28 07:43
@ivanfioravanti @demisbellot of course, they got the motherboard wrong, are using noisy power supplies that crap out after full load for a few days, bought pcie extenders that don't work, and their idea of cooling is a box fan from walmart.

@__tinygrad__ · 2025-03-28 07:36
We aren't selling an application box. We are selling a box with raw power. Applications will come and go over the next 10 years, PFLOPS and TB/s won't.

@AgentifySH · 2025-03-28 07:46
@__tinygrad__ one of these days im going to buy a tiny box and run my own llms

@demisdev · 2025-03-28 07:42
@__tinygrad__ Great, than why not post numbers most potential customers would want to know about? e.g. how well it runs local ollama models. You make it sound I should feel bad for even asking what perf to expect for running its primary use-case? Weird strat.

@__tinygrad__ · 2025-03-28 07:47
@demisbellot strat? and this is def not the primary use case. if you only want LLM inference, use Grok, ChatGPT, or DeepSeek cloud. it's a lot cheaper.

@__tinygrad__ · 2025-03-28 07:43
@ivanfioravanti @demisbellot of course, they got the motherboard wrong, are using noisy power supplies that crap out after full load for a few days, bought pcie extenders that don't work, and their idea of cooling is a box fan from walmart.

@qubitium · 2025-03-28 07:50
@__tinygrad__ @ivanfioravanti @demisbellot Speaking as someone who touched, build, threw away mcio/8654 crap cables, pcie extenders that cant do pcie 4.0, the list goes on and on, it is crazy how good the tingrad setup is if you know how specific the tolerance you need from mobo, to pcie slot, slot to gpu, and pwr..

@AgentifySH · 2025-03-28 07:46
@__tinygrad__ one of these days im going to buy a tiny box and run my own llms

@__tinygrad__ · 2025-03-28 07:56
@AgentifySH Once you buy the tinybox, it's yours to do whatever you want with! No service upsell, just a very powerful computer you own outright.

@qubitium · 2025-03-28 07:50
@__tinygrad__ @ivanfioravanti @demisbellot Speaking as someone who touched, build, threw away mcio/8654 crap cables, pcie extenders that cant do pcie 4.0, the list goes on and on, it is crazy how good the tingrad setup is if you know how specific the tolerance you need from mobo, to pcie slot, slot to gpu, and pwr..

@__tinygrad__ · 2025-03-28 08:00
@qubitium @ivanfioravanti @demisbellot We've put a lot of effort into this. Tons of testing, custom engineering, etc... The tinybox strives for perfection. Perfect full fabric PCIe connectivity with no AERs, perfect power at 100% for weeks while staying quiet, and perfect cooling keeping GPUs under 80C.

@__tinygrad__ · 2025-03-29 02:26
This. tinygrad is written in pure Python

@__tinygrad__ · 2025-03-29 02:26
This. tinygrad is written in pure Python

@AlpinDale · 2025-03-29 02:29
@__tinygrad__ "...?" https://t.co/3lTtYNlFIX

@AlpinDale · 2025-03-29 02:29
@__tinygrad__ "...?" https://t.co/3lTtYNlFIX

@__tinygrad__ · 2025-03-29 02:33
@AlpinDale The repo has a bunch of header files and stuff in it, but `import tinygrad` is pure Python. Except the visualizer...that's sadly in JavaScript.

@__tinygrad__ · 2025-03-29 12:03
The MI300X is now supported in our runtime! With tinygrad, no more HIP. However, we use PM4 and not AQL; AMD hardcoded the sync between the 8 XCDs in the MEC (firmware) AQL codepath 😢 We currently have to go to memory to sync, 30 us extra per kernel. How we use GWS from PM4? https://t.co/Ybug2Iax52

@Molag433 · 2025-03-29 19:33
@martinloretzzz How’re you guys planning on doing that?

@martinloretzzz · 2025-03-29 19:58
@Molag433 There are thousands of subproblems to work on. I'm a contributor to tinygrad and work a lot on ML performance, but everyone needs to work on what they find exciting and then aggregated big things happen.

@__tinygrad__ · 2025-03-31 01:24
There's many ways to contribute to tinygrad. For example, even if you don't know anything about ML, are you a good Python programmer who can make the Python run faster?

@ahmedkhaleel04 · 2025-03-31 01:24
I'm #2 on GitHub trending 😳 I built this solo in a week and launched with NO expectations. Just free, simple, open source software that solved my niche problem. The moat is and will always be shipping cool things. https://t.co/v5uIXDDzAu https://t.co/LKwPvXXQIO

@__tinygrad__ · 2025-03-31 01:24
There's many ways to contribute to tinygrad. For example, even if you don't know anything about ML, are you a good Python programmer who can make the Python run faster?

@r0b0t_sp1der · 2025-03-31 01:41
@__tinygrad__ bro I looked at a $100 bounty and it required graduate-level knowledge of compilers and linearization …

@__tinygrad__ · 2025-03-31 13:56
Company meeting on Discord in 5 minutes. All can listen.

@__tinygrad__ · 2025-03-31 15:15
Our founder only has a high school education. If you put in the effort you can solve it, and the next one will be easier.

@__tinygrad__ · 2025-03-31 13:56
Company meeting on Discord in 5 minutes. All can listen.

@__tinygrad__ · 2025-04-01 01:32
See all the historical meetings here https://t.co/BLM0TWJSMK

@__tinygrad__ · 2025-04-01 11:57
When looking at tinygrad this morning, I was disgusted. We were promised simplicity, and we got 50 ops! This is approaching torch levels of ops. In a fit of rage, all ops were deleted except ADD. It turns out everything is just adds. Add is add. Subtract is just add with a minus sign. Multiply is add a couple of times. Divide is...well...stop dividing you don't need to. Then, in transcendental, we could already express sin, log, and exp as math, so now that's adds too. We believe this will inform future hardware. Stop making MULACCs, just ADDACCs! And ACC is ADD, so it's just ADDADD, aka ADD. Only ADD. Wait...I was just informed ADD has dtypes. Deleting this. Only NOR gate on bool. NOR is good. bool is good. No more dtype. No more ops. Only NOR. NOR NOR NOR NOR.

@__tinygrad__ · 2025-04-01 11:57
When looking at tinygrad this morning, I was disgusted. We were promised simplicity, and we got 50 ops! This is approaching torch levels of ops. In a fit of rage, all ops were deleted except ADD. It turns out everything is just adds. Add is add. Subtract is just add with a minus sign. Multiply is add a couple of times. Divide is...well...stop dividing you don't need to. Then, in transcendental, we could already express sin, log, and exp as math, so now that's adds too. We believe this will inform future hardware. Stop making MULACCs, just ADDACCs! And ACC is ADD, so it's just ADDADD, aka ADD. Only ADD. Wait...I was just informed ADD has dtypes. Deleting this. Only NOR gate on bool. NOR is good. bool is good. No more dtype. No more ops. Only NOR. NOR NOR NOR NOR.

@ritteradam · 2025-04-01 12:05
@__tinygrad__ Just delete everything except mulacc and you're not that far from what a TPU / Neural engine is able to execute.

@ritteradam · 2025-04-01 12:05
@__tinygrad__ Just delete everything except mulacc and you're not that far from what a TPU / Neural engine is able to execute.

@__tinygrad__ · 2025-04-01 12:39
@ritteradam But can we run it on the brand new GeForce 6000 Series? https://t.co/bgSgeoB62c

@__tinygrad__ · 2025-04-01 23:55
RT @EitanTurok: I tried the git diagram tool with @__tinygrad__ . Man it is clean! https://t.co/TrBIjlJYyW

@__tinygrad__ · 2025-04-02 10:53
This is MobileNetV2 on 845 DSP. On our last post, it was taking 183 ms. Now it's 13.4 ms and SNPE competitive. Tricks include careful memory layout, vrmpy, vraddub, cache prefetching, and multicore. 13x faster! https://t.co/2gPiYU4mB5

@__tinygrad__ · 2025-04-02 10:53
This is MobileNetV2 on 845 DSP. On our last post, it was taking 183 ms. Now it's 13.4 ms and SNPE competitive. Tricks include careful memory layout, vrmpy, vraddub, cache prefetching, and multicore. 13x faster! https://t.co/2gPiYU4mB5

@khengari77 · 2025-04-02 10:56
@__tinygrad__ Is it possible to run this on Snapdragon 860 ? On a phone that runs android?

@khengari77 · 2025-04-02 10:56
@__tinygrad__ Is it possible to run this on Snapdragon 860 ? On a phone that runs android?

@__tinygrad__ · 2025-04-02 10:57
@khengari77 Try it. Branch here https://t.co/ZJkrCSARkC

@samxpatterson · 2025-04-02 16:36
Mini blog post on improving FP16/16 matrix multiplication accuracy by accumulating outside the mma instruction. 80% cuBLAS FP16/16 performance with roughly 10x smaller absolute error. Link in reply.

@adamscochran · 2025-04-02 21:52
“We’ll just buy American mad!” Hey dumbass, Trump just slapped a 20%-30%+ tariff on every single country including the ones US manufacturers get their raw materials from. And he didn’t exempt raw materials. So beyond just not having the production capacity to offset *the entire world’s* goods; American made products are *also* going up in price!

@__tinygrad__ · 2025-04-03 13:08
First batch of tinybox green v2 is 32 computers. Already got two wires, the order they hit the bank determines shipping order. You ready? Place an order, send a wire, get it in 1-3 months. If you want to wait and hope there's one left for you, sign up for the mailing list.

@__tinygrad__ · 2025-04-03 13:08
First batch of tinybox green v2 is 32 computers. Already got two wires, the order they hit the bank determines shipping order. You ready? Place an order, send a wire, get it in 1-3 months. If you want to wait and hope there's one left for you, sign up for the mailing list.

@__tinygrad__ · 2025-04-03 13:31
Err actually, orders are stopped for now. 4 lucky people got in at the current price. Although we manufacture in the US, we buy parts from overseas and don't know how the tariffs will impact things. Sign up for the mailing list for updates.

@__tinygrad__ · 2025-04-03 13:31
Err actually, orders are stopped for now. 4 lucky people got in at the current price. Although we manufacture in the US, we buy parts from overseas and don't know how the tariffs will impact things. Sign up for the mailing list for updates.

@__tinygrad__ · 2025-04-03 13:38
.@WhiteHouse We manufacture in America and sell 40% of our product overseas. With the import tariffs, we are now forced to consider manufacturing those 40% elsewhere. How could I possibly be competitive when I have to pay import tariffs on components to then export them?

@__tinygrad__ · 2025-04-03 13:38
.@WhiteHouse We manufacture in America and sell 40% of our product overseas. With the import tariffs, we are now forced to consider manufacturing those 40% elsewhere. How could I possibly be competitive when I have to pay import tariffs on components to then export them?

@hyperliberalism · 2025-04-03 13:46
@__tinygrad__ @WhiteHouse Most semiconductors are exempted, the people doing policy on this are not "Trump", but people who actually understand this stuff.

@hyperliberalism · 2025-04-03 13:46
@__tinygrad__ @WhiteHouse Most semiconductors are exempted, the people doing policy on this are not "Trump", but people who actually understand this stuff.

@__tinygrad__ · 2025-04-03 14:09
@hyperliberalism @WhiteHouse This is the same issue with regulation. While that may be true, or maybe we can reclaim on exported goods, we'd have to hire a lawyer and deal with figuring that out. Who has the resources and time for this?

@__tinygrad__ · 2025-04-03 13:38
.@WhiteHouse We manufacture in America and sell 40% of our product overseas. With the import tariffs, we are now forced to consider manufacturing those 40% elsewhere. How could I possibly be competitive when I have to pay import tariffs on components to then export them?

@OliDietzel · 2025-04-03 15:59
@__tinygrad__ @WhiteHouse https://t.co/GKkq8znLFh

@OliDietzel · 2025-04-03 15:59
@__tinygrad__ @WhiteHouse https://t.co/GKkq8znLFh

@__tinygrad__ · 2025-04-03 16:01
@OliDietzel @WhiteHouse If this is true, why does @comma_ai, based in San Diego, have to pay so much tariffs?

@__tinygrad__ · 2025-04-03 16:24
To date, we have manufactured all tinyboxes in America. However, we buy parts from abroad. There is no way to buy an American made GPU or motherboard, and there won't be for a long time. If these tariffs stand as is, we would have negative margins on tinyboxes. Our motherboard manufacturer has already reached out and tried to get us to pay the tariffs on things we already agreed on the delivered price for. But I don't blame them. Their margins probably go negative with the tariffs too. I sort of doubt we'll be getting our 5090s at the price we agreed on either. If that's true, the whole thing really is out the window. And even more stupidly, there's a restricted list of countries you can ship 5090s to, so I'm not so sure we could move manufacturing of the green v2, and the product may just be cancelled. I'm not going to spend my time figuring out weird loopholes and incentives and reexport and FTZ and maybe try to eek out a small profit after all the administrative costs. Tariffs are regulations. When difficultly of doing business goes up, many people only marginally making money just stop doing business. The US has an "ease of doing business" ranking of 6, Hong Kong's ranking is 3. If we manufacture here in HK, we have free trade, and can continue our policy of selling everywhere and passing the tariffs on to the buyer (EU people have been dealing with this for a long time, now US people will too). If we can't get 5090s cause of shortsighted US export regulations, we'll have to switch to something we can get (9070XT?). But here's the kicker, so will the rest of the world. Long term, this hands the future of AI hardware to the Chinese, because it's what everyone will be able to buy. And truthfully, this is probably good for tiny corp long term. The minute there's decent Chinese chips, we'll be ready with the software, as tinygrad is the easiest framework to port to any hardware.

@__tinygrad__ · 2025-04-03 16:24
To date, we have manufactured all tinyboxes in America. However, we buy parts from abroad. There is no way to buy an American made GPU or motherboard, and there won't be for a long time. If these tariffs stand as is, we would have negative margins on tinyboxes. Our motherboard manufacturer has already reached out and tried to get us to pay the tariffs on things we already agreed on the delivered price for. But I don't blame them. Their margins probably go negative with the tariffs too. I sort of doubt we'll be getting our 5090s at the price we agreed on either. If that's true, the whole thing really is out the window. And even more stupidly, there's a restricted list of countries you can ship 5090s to, so I'm not so sure we could move manufacturing of the green v2, and the product may just be cancelled. I'm not going to spend my time figuring out weird loopholes and incentives and reexport and FTZ and maybe try to eek out a small profit after all the administrative costs. Tariffs are regulations. When difficultly of doing business goes up, many people only marginally making money just stop doing business. The US has an "ease of doing business" ranking of 6, Hong Kong's ranking is 3. If we manufacture here in HK, we have free trade, and can continue our policy of selling everywhere and passing the tariffs on to the buyer (EU people have been dealing with this for a long time, now US people will too). If we can't get 5090s cause of shortsighted US export regulations, we'll have to switch to something we can get (9070XT?). But here's the kicker, so will the rest of the world. Long term, this hands the future of AI hardware to the Chinese, because it's what everyone will be able to buy. And truthfully, this is probably good for tiny corp long term. The minute there's decent Chinese chips, we'll be ready with the software, as tinygrad is the easiest framework to port to any hardware.

@antlionai · 2025-04-03 16:26
@__tinygrad__ Damn. I was hoping for a blue box.

@antlionai · 2025-04-03 16:26
@__tinygrad__ Damn. I was hoping for a blue box.

@__tinygrad__ · 2025-04-03 16:27
@antlionai Bullish on AMD. Not bullish on Intel. https://t.co/9FcM5tm8v1

@__tinygrad__ · 2025-04-04 04:36
VIZ=1 now includes a memory profiler. No code changes required, just VIZ=1 on the environment (only a ~10% perf hit). Our goal is to make the most easy to debug framework, have you tried running with DEBUG=2? https://t.co/IsfIx2hVLE

@__tinygrad__ · 2025-04-05 05:33
Congrats to @tenstorrent for having a buy it now button for their new hardware, this is the way! I wish 5090s had a buy it now button, will they ever? Anyone know what the problem is? If NVIDIA wants to sell us GB202 chips we can produce cards in US. https://t.co/FaNozRvTIq

@__tinygrad__ · 2025-04-05 05:33
Congrats to @tenstorrent for having a buy it now button for their new hardware, this is the way! I wish 5090s had a buy it now button, will they ever? Anyone know what the problem is? If NVIDIA wants to sell us GB202 chips we can produce cards in US. https://t.co/FaNozRvTIq

@rhatr · 2025-04-05 05:42
@__tinygrad__ @tenstorrent Any updates on tinygrad support for @tenstorrent ? I vaguely remember seeing a bounty but I can seem to find the URL now to check for the latest updates

@rhatr · 2025-04-05 05:42
@__tinygrad__ @tenstorrent Any updates on tinygrad support for @tenstorrent ? I vaguely remember seeing a bounty but I can seem to find the URL now to check for the latest updates

@__tinygrad__ · 2025-04-05 05:46
@rhatr @tenstorrent We have a $1,000 bounty, and both @tenstorrent and @corsix have agreed to match it. However, the arch is unlike GPUs, there's no sim, and the memory bw is midrange. It would require a lot of effort to make a decent port, and it's not a priority for tiny corp at this time.

@__tinygrad__ · 2025-04-05 05:37
@tenstorrent At $1,399, it's almost worth it to buy these as 4x800G network cards. Do the drivers support this? https://t.co/0B0ztEJkto

@DavidBennett__ · 2025-04-05 06:06
@__tinygrad__ @tenstorrent We often talk about this use case around the office. I believe there might even be some work done on this already … @davorVDR 🤔

@__tinygrad__ · 2025-04-05 05:46
@rhatr @tenstorrent We have a $1,000 bounty, and both @tenstorrent and @corsix have agreed to match it. However, the arch is unlike GPUs, there's no sim, and the memory bw is midrange. It would require a lot of effort to make a decent port, and it's not a priority for tiny corp at this time.

@DavidBennett__ · 2025-04-05 06:08
@__tinygrad__ @rhatr @tenstorrent @corsix Hope one of the community members out there want to give it a shot. One idea might be to start with a few of our easier bounties to learn the architecture? @tenstorrent will also collaborate to help you along by the way! https://t.co/GmjlNkgcgF

@DavidBennett__ · 2025-04-05 06:06
@__tinygrad__ @tenstorrent We often talk about this use case around the office. I believe there might even be some work done on this already … @davorVDR 🤔

@__tinygrad__ · 2025-04-05 06:09
@DavidBennett__ @tenstorrent @davorVDR This is something Intel Gaudi managed to do well, the onboard links out just show up as normal Linux network cards. Not sure if Blackhole is just for P2P links to other Blackholes or if it's normal ethernet that works with switches etc...

@DavidBennett__ · 2025-04-05 06:08
@__tinygrad__ @rhatr @tenstorrent @corsix Hope one of the community members out there want to give it a shot. One idea might be to start with a few of our easier bounties to learn the architecture? @tenstorrent will also collaborate to help you along by the way! https://t.co/GmjlNkgcgF

@__tinygrad__ · 2025-04-05 06:17
My understanding is all the power in the cards is in these custom undocumented "Tensix execution resources" (from corsix blog posts). Not having a shared register file makes this arch very hard to generate code for, you'll need constraint solvers probably to make it good, a generation beyond any deployed compiler today. In addition, the memory subsystem is super hard to use compared to a GPU, or even compared to something like a Qualcomm DSP, which is hard enough. Having to generate code to software manage what's usually in hardware isn't easy. In order to make a good Tenstorrent backend, it would require a lot of Tenstorrent specific work. I really hope someone (or Tenstorrent themselves) do it, but it's quite a large undertaking.

@corsix · 2025-04-05 14:49
@jasondavies @__tinygrad__ @DavidBennett__ @rhatr @tenstorrent For the tinygrad bounty, my money is for Wormhole. Tinycorp’s money _was_ for Grayskull, I don’t know whether they’ll amend that.

@DavidBennett__ · 2025-04-05 15:52
@corsix @jasondavies @__tinygrad__ @rhatr @tenstorrent Wormhole is the way :-)

@AIatMeta · 2025-04-05 19:11
Today is the start of a new era of natively multimodal AI innovation. Today, we’re introducing the first Llama 4 models: Llama 4 Scout and Llama 4 Maverick — our most advanced models yet and the best in their class for multimodality. Llama 4 Scout • 17B-active-parameter model with 16 experts. • Industry-leading context window of 10M tokens. • Outperforms Gemma 3, Gemini 2.0 Flash-Lite and Mistral 3.1 across a broad range of widely accepted benchmarks. Llama 4 Maverick • 17B-active-parameter model with 128 experts. • Best-in-class image grounding with the ability to align user prompts with relevant visual concepts and anchor model responses to regions in the image. • Outperforms GPT-4o and Gemini 2.0 Flash across a broad range of widely accepted benchmarks. • Achieves comparable results to DeepSeek v3 on reasoning and coding — at half the active parameters. • Unparalleled performance-to-cost ratio with a chat version scoring ELO of 1417 on LMArena. These models are our best yet thanks to distillation from Llama 4 Behemoth, our most powerful model yet. Llama 4 Behemoth is still in training and is currently seeing results that outperform GPT-4.5, Claude Sonnet 3.7, and Gemini 2.0 Pro on STEM-focused benchmarks. We’re excited to share more details about it even while it’s still in flight. Read more about the first Llama 4 models, including training and benchmarks ➡️ https://t.co/9G3QgVdCkB Download Llama 4 ➡️ https://t.co/eVomRvEr0w

@__tinygrad__ · 2025-04-05 05:37
@tenstorrent At $1,399, it's almost worth it to buy these as 4x800G network cards. Do the drivers support this? https://t.co/0B0ztEJkto

@jimkxa · 2025-04-05 22:39
@__tinygrad__ @tenstorrent The hardware is ethernet layer 3. We have simple drivers to connect our parts. We also use p150 cards to connect our 32 chip galaxy box to a standard servers. we have a plan to make a more standard networking software stack. I'll get that plan updated.

@jeremyphoward · 2025-04-05 20:02
@JeffDean 4bit quant of the smallest 109B model is far too big to fit on a 4090 -- or even a pair of them! :O If you want to do some LoRA fine tuning (which we do!) then you need even more memory than that.

@JeffDean · 2025-04-06 01:41
@jeremyphoward Sure, but you can run it on 4 or 8 of them, no?

@jeremyphoward · 2025-04-05 20:02
@JeffDean 4bit quant of the smallest 109B model is far too big to fit on a 4090 -- or even a pair of them! :O If you want to do some LoRA fine tuning (which we do!) then you need even more memory than that.

@JeffDean · 2025-04-06 01:41
@jeremyphoward Sure, but you can run it on 4 or 8 of them, no?

@__tinygrad__ · 2025-04-06 02:33
Excellent choices with FP8 and MoE! Llama 4 Scout should fit great on a tinybox, theoretical max 350 tok/s. $500 bounty if you can get it running @ 200+ tok/s

@jimkxa · 2025-04-05 22:39
@__tinygrad__ @tenstorrent The hardware is ethernet layer 3. We have simple drivers to connect our parts. We also use p150 cards to connect our 32 chip galaxy box to a standard servers. we have a plan to make a more standard networking software stack. I'll get that plan updated.

@__tinygrad__ · 2025-04-06 02:35
@jimkxa @tenstorrent Good to know it'll work with normal switches

@__tinygrad__ · 2025-04-06 02:33
Excellent choices with FP8 and MoE! Llama 4 Scout should fit great on a tinybox, theoretical max 350 tok/s. $500 bounty if you can get it running @ 200+ tok/s

@AlexBravo · 2025-04-06 02:35
@__tinygrad__ how come Mark said it fits on one GPU?

@AlexBravo · 2025-04-06 02:35
@__tinygrad__ how come Mark said it fits on one GPU?

@__tinygrad__ · 2025-04-06 02:36
@AlexBravo Because Mr. Zuckerberg is rich and purchases very expensive GPUs.

@__tinygrad__ · 2025-04-06 02:33
Excellent choices with FP8 and MoE! Llama 4 Scout should fit great on a tinybox, theoretical max 350 tok/s. $500 bounty if you can get it running @ 200+ tok/s

@__tinygrad__ · 2025-04-06 02:55
.@AIatMeta @Ahmad_Al_Dahle @astonzhangAZ No FP8 for Scout? Maverick has one. It's nice when the model comes quantized to 8-bit and tested so you don't have to mess around with random quantizers. Llama-4-Scout-17B-16E-Instruct-FP8 soon? https://t.co/G4mGJqOGAS

@__tinygrad__ · 2025-04-06 07:09
Through the power of "vibe reversing" we found the code in the MEC that waits for the 8 XCDs of the MI300X. It seems like it might only be accessible through AQL. AQL (vs PM4) is a bad design decision, AMD should push all complexity to where it's easy to develop and debug. https://t.co/vxkt2rIs8C

@__tinygrad__ · 2025-04-06 10:21
Have you considered 6? tinybox reds are in stock and shipping.

@Yuchenj_UW · 2025-04-07 01:49
If Meta actually did this for Llama 4 training to maximize benchmark scores, it's fucked. https://t.co/foYDLSPYn9

@__tinygrad__ · 2025-04-07 02:41
@Yuchenj_UW No way this is true, no major lab would be this dumb. I think a lot of the perceived badness of Llama 4 comes from broken implementations or bad quantizations, MoE is hard to get right. I suspect it's fixed over the next few weeks and it's on par with a model of those benchmarks.

@Yuchenj_UW · 2025-04-07 02:43
@__tinygrad__ Do you have a good implementation with tinygrad? I’d love to spin up some GPUs to test it out!

@Yuchenj_UW · 2025-04-07 02:43
@__tinygrad__ Do you have a good implementation with tinygrad? I’d love to spin up some GPUs to test it out!

@__tinygrad__ · 2025-04-07 02:45
@Yuchenj_UW No, we have a bounty for it though. We also have a clean implementation of OLMoE to crib from.

@__tinygrad__ · 2025-04-06 10:21
Have you considered 6? tinybox reds are in stock and shipping.

@enod_bataa · 2025-04-07 04:41
@__tinygrad__ was just about to buy green v2 - as it says SHIPS Q2 but when I click on it - OUT OF STOCK :\

@enod_bataa · 2025-04-07 04:41
@__tinygrad__ was just about to buy green v2 - as it says SHIPS Q2 but when I click on it - OUT OF STOCK :\

@__tinygrad__ · 2025-04-07 11:33
@enod_bataa Pending resolution on tariffs. With listed tariffs, we are no longer profitable. Buy a red, it's already in USA.

@__tinygrad__ · 2025-04-07 11:49
One kernel softmax, and a clear path to flash attention. https://t.co/AfMppLdA9d

@__tinygrad__ · 2025-04-07 11:49
One kernel softmax, and a clear path to flash attention. https://t.co/AfMppLdA9d

@aturker01 · 2025-04-07 11:54
@__tinygrad__ Is there a handwritten pattern matcher to detect the Flash Attention graph, or does the compiler automatically optimize to it

@aturker01 · 2025-04-07 11:54
@__tinygrad__ Is there a handwritten pattern matcher to detect the Flash Attention graph, or does the compiler automatically optimize to it

@__tinygrad__ · 2025-04-07 11:56
@aturker58 Automatic of course. If you have to hand code, it isn't worth it.

@__tinygrad__ · 2025-04-07 11:33
@enod_bataa Pending resolution on tariffs. With listed tariffs, we are no longer profitable. Buy a red, it's already in USA.

@enod_bataa · 2025-04-07 12:20
@__tinygrad__ I'm based in Australia and personally have no experience with Radeon GPUs. Are drivers fully supported? Is it possible to run models let's say via huggingface without an issue?

@enod_bataa · 2025-04-07 12:20
@__tinygrad__ I'm based in Australia and personally have no experience with Radeon GPUs. Are drivers fully supported? Is it possible to run models let's say via huggingface without an issue?

@__tinygrad__ · 2025-04-07 12:23
@enod_bataa Haha I mean, ask @AnushElangovan, re: AMD drivers. If you are outside USA, if tariffs stick, we'll likely just move manufacturing out of the country so you'll be good.

@__tinygrad__ · 2025-04-07 13:50
There's up to $3,000 in bounties for finishing PR #9737. You want to use UPats to dynamically generate Python for the match. Like this, but handle the branching cases too. Low effort PRs be closed without comment. https://t.co/xRa29XNM4l

@DavidBennett__ · 2025-04-05 15:52
@corsix @jasondavies @__tinygrad__ @rhatr @tenstorrent Wormhole is the way :-)

@__tinygrad__ · 2025-04-07 13:54
@DavidBennett__ @corsix @jasondavies @rhatr @tenstorrent We updated ours to Wormhole as well.

@__tinygrad__ · 2025-04-08 11:29
Of course, why stop at single kernel softmax? It's the exact same infrastructure to get FlashAttention. Is that not the cleanest implementation ever? And it's autogen code. Left: tinygrad output, correct but slowish Right: what ChatGPT says it can simplify to https://t.co/QLaWJMfyn0

@MrBeast · 2025-04-08 14:58
Ironically because of all the new tariffs it is now way cheaper to make our chocolate bars we sell globally NOT in America because other countries don’t have a 20%+ tariff on our cogs 😅

@__tinygrad__ · 2025-04-08 11:29
Of course, why stop at single kernel softmax? It's the exact same infrastructure to get FlashAttention. Is that not the cleanest implementation ever? And it's autogen code. Left: tinygrad output, correct but slowish Right: what ChatGPT says it can simplify to https://t.co/QLaWJMfyn0

@tcpollak · 2025-04-08 15:18
@__tinygrad__ But left is standard softmax without the exp trick? ChatGPT is doing the heavy lifting rewriting into a flash attn softmax

@tcpollak · 2025-04-08 15:18
@__tinygrad__ But left is standard softmax without the exp trick? ChatGPT is doing the heavy lifting rewriting into a flash attn softmax

@__tinygrad__ · 2025-04-08 15:51
@tcpollak tinygrad is now outputting this. it's an OptOp to save the 512 dot products to locals, this is all the "free" transforms. https://t.co/qxwM2eUyhr

@__tinygrad__ · 2025-04-08 15:56
This. Same for tinyboxes.

@__tinygrad__ · 2025-04-06 07:09
Through the power of "vibe reversing" we found the code in the MEC that waits for the 8 XCDs of the MI300X. It seems like it might only be accessible through AQL. AQL (vs PM4) is a bad design decision, AMD should push all complexity to where it's easy to develop and debug. https://t.co/vxkt2rIs8C

@neggles · 2025-04-08 22:52
@__tinygrad__ your mistake is assuming that the PM4 pipeline is anything other than an implementation detail; AQL is the first class citizen here, there's dedicated hardware involved with those sync operations which isn't available through any other method

@neggles · 2025-04-08 22:52
@__tinygrad__ your mistake is assuming that the PM4 pipeline is anything other than an implementation detail; AQL is the first class citizen here, there's dedicated hardware involved with those sync operations which isn't available through any other method

@__tinygrad__ · 2025-04-09 03:30
@neggles PM4 is actually accessing the GPU registers. AQL is a layer of slop on top of that that requires 10x more code in the MEC. There's no reason at all you should have a parser like that in your limited firmware. AQL itself is fine, but move that parser up to the CPU hosted runtime.

@ZDi____ · 2025-04-09 05:27
@HotAisle oh, I got your attention. what do you think of tinygrad?

@HotAisle · 2025-04-09 05:37
@ZDi____ He's had those two boxes for a month now. Does it run on MI300x yet?

@__tinygrad__ · 2025-04-09 06:26
RT @liuwilliam47: i'm really liking TinyGrad right now

@__tinygrad__ · 2025-04-09 06:28
You mean kernel time or Python time? For kernel time, run with BEAM=2 and it should be torch competitive. For Python time, make sure all kernels are in the TinyJIT and it will outperform everything, even CUDA Graph. Debug with DEBUG=2, purple means JIT.

@__tinygrad__ · 2025-04-09 06:28
You mean kernel time or Python time? For kernel time, run with BEAM=2 and it should be torch competitive. For Python time, make sure all kernels are in the TinyJIT and it will outperform everything, even CUDA Graph. Debug with DEBUG=2, purple means JIT.

@smallshen2 · 2025-04-09 06:31
@__tinygrad__ The jit compilation time is long. Also I got weird issue where optimizer step gives everything nan after one run.

@smallshen2 · 2025-04-09 06:31
@__tinygrad__ The jit compilation time is long. Also I got weird issue where optimizer step gives everything nan after one run.

@__tinygrad__ · 2025-04-09 06:33
@smallshen2 We're working on improving JIT compilation time. You can cache the function with pickle after it's compiled, just pickle it and it'll load and start at full speed in seconds.

@__tinygrad__ · 2025-04-09 06:33
@smallshen2 We're working on improving JIT compilation time. You can cache the function with pickle after it's compiled, just pickle it and it'll load and start at full speed in seconds.

@smallshen2 · 2025-04-09 06:34
@__tinygrad__ The optimizer nan issue makes me can’t cache with pickle, because the nan… I have to test the root cause

@smallshen2 · 2025-04-09 06:34
@__tinygrad__ The optimizer nan issue makes me can’t cache with pickle, because the nan… I have to test the root cause

@__tinygrad__ · 2025-04-09 06:35
@smallshen2 If you have a minimal repro, file an issue. There's a few footguns with the JIT, you should JIT the *entire* step in one function.

@__tinygrad__ · 2025-04-09 09:21
Yes! Been in master for 2 weeks, custom AMD-free PM4 based runtime, and we have a decent mitigation for the slow sync going in soon. We'll get MI300X on the next MLPerf training, time will be meh cause it's not top priority, but we dare anyone to beat it. At least it's stable.

@__tinygrad__ · 2025-04-09 09:21
Yes! Been in master for 2 weeks, custom AMD-free PM4 based runtime, and we have a decent mitigation for the slow sync going in soon. We'll get MI300X on the next MLPerf training, time will be meh cause it's not top priority, but we dare anyone to beat it. At least it's stable.

@__tinygrad__ · 2025-04-09 09:35
Our main focus for speed is on consumer GPUs, our goal is to be on par with PyTorch for NVIDIA (and well ahead of PyTorch on AMD) by the end of the year. In exchange for the boxes, we did the runtime and tinygrad support as promised, and we'll continue to make steady but slow progress on speed that applies universally to all GPUs. We will also get one of the boxes in our CI to ensure MI300X doesn't regress, we are currently manually testing it frequently in preparing for MLPerf. We are also interested in getting unquantized BS=1 DeepSeek running fast on the machine, should be done by the end of the year also. It's pretty non chip specific to get close to theoretical max on BS=1 LLMs. Now for crazy good training speed, I believe it's possible to beat the 8xH100 MLPerf times with 8xMI300X and tinygrad. But that's the kind of focused AMD datacenter chip exclusive effort we'd need a contract for, otherwise we are more focused on improving speed for the tinybox products we sell.

@hyperprior · 2025-04-09 13:35
i want tariffs on pytorch so we all have to switch to @__tinygrad__

@__tinygrad__ · 2025-04-09 13:49
We believe in the free market for neural network frameworks! Long term, you either are competitive or you aren't. tinygrad already has many areas of advantage, mostly around wide platform support (example, anything with OpenCL) and model export (output zero dep C or WebGPU JS)

@__tinygrad__ · 2025-04-09 13:49
We believe in the free market for neural network frameworks! Long term, you either are competitive or you aren't. tinygrad already has many areas of advantage, mostly around wide platform support (example, anything with OpenCL) and model export (output zero dep C or WebGPU JS)

@jsuarez · 2025-04-09 13:56
@__tinygrad__ Zero dep c is very nice. We currently have our own C implementation of several pytorch layers so we can export pytorch models to run on web via wasm. Doesn't make sense for us to adopt tinygrad early until you have more market share, sadly

@jsuarez · 2025-04-09 13:56
@__tinygrad__ Zero dep c is very nice. We currently have our own C implementation of several pytorch layers so we can export pytorch models to run on web via wasm. Doesn't make sense for us to adopt tinygrad early until you have more market share, sadly

@__tinygrad__ · 2025-04-09 14:04
@jsuarez tinygrad can be used as a PyTorch backend if you want to autogenerate that C from those torch layers.

@thehiphopswami · 2025-04-09 16:32
A career peak for myself! Led physical implementation of this chip from concept to silicon debug. https://t.co/vfMhRfpSxS

@__tinygrad__ · 2025-04-10 03:31
With the current state of the world, it doesn't make sense to continue in the box business. Unless things radically change in the next few months, there's far too much volatility to run a 30% margin computer business from the US. Even with tariffs "paused" our GPU supplier doesn't want to honor the price we agreed on. And why should they, maybe they can find a sucker willing to pay more? You don't know what the world will be in a week, better lock in it now at this higher price. This is not a game I want to play. The US market will end up with companies charging 3x higher prices that have aggressive sales teams constantly trying to book you for a zoom call, and of course they won't tell you the price up front just 30 minutes into the call they will "see what they can do" and boom 3x higher. That's all the market can support, it's impossible to plan for the future and run a stable 30% margin business. The future of tiny corp has always been in making chips. We're still a year from having the software ready, but once we are, we need to be ready with the tape out. We'll sell the chips in a network attached appliance free from any US supply chain, and work with suppliers that are stable in long term win-win cooperation. Have you taped out a chip before and want to come work here? E-mail or DM me. Let's find the easiest fab to work with and get started, something ~12nm to start.

@__tinygrad__ · 2025-04-10 03:31
With the current state of the world, it doesn't make sense to continue in the box business. Unless things radically change in the next few months, there's far too much volatility to run a 30% margin computer business from the US. Even with tariffs "paused" our GPU supplier doesn't want to honor the price we agreed on. And why should they, maybe they can find a sucker willing to pay more? You don't know what the world will be in a week, better lock in it now at this higher price. This is not a game I want to play. The US market will end up with companies charging 3x higher prices that have aggressive sales teams constantly trying to book you for a zoom call, and of course they won't tell you the price up front just 30 minutes into the call they will "see what they can do" and boom 3x higher. That's all the market can support, it's impossible to plan for the future and run a stable 30% margin business. The future of tiny corp has always been in making chips. We're still a year from having the software ready, but once we are, we need to be ready with the tape out. We'll sell the chips in a network attached appliance free from any US supply chain, and work with suppliers that are stable in long term win-win cooperation. Have you taped out a chip before and want to come work here? E-mail or DM me. Let's find the easiest fab to work with and get started, something ~12nm to start.

@hornswoggle567 · 2025-04-10 03:42
@__tinygrad__ All these tariff's are just posturing actual implementation will destroy the US economy and Trump administration knows it. Most of them will be softened or cancelled outright, just leaving some offshoot ones in the name of fan service.

@hornswoggle567 · 2025-04-10 03:42
@__tinygrad__ All these tariff's are just posturing actual implementation will destroy the US economy and Trump administration knows it. Most of them will be softened or cancelled outright, just leaving some offshoot ones in the name of fan service.

@__tinygrad__ · 2025-04-10 04:01
@hornswoggle567 The extra money we'd have to pay if we wanted the GPUs isn't posturing. And if the tariffs disappear tomorrow, we don't get that money back. I'm not playing this boneheaded game, better to just pivot and build for the future in a sustainable way.

@__tinygrad__ · 2025-04-10 04:18
In uncertain times, tinygrad always has bounties for you to solve. Over $50k paid out to date. And the only path to a job here, all hiring will look like this soon. bounties dot tinygrad dot org https://t.co/7jtuHNY3iL

@__tinygrad__ · 2025-04-10 03:31
With the current state of the world, it doesn't make sense to continue in the box business. Unless things radically change in the next few months, there's far too much volatility to run a 30% margin computer business from the US. Even with tariffs "paused" our GPU supplier doesn't want to honor the price we agreed on. And why should they, maybe they can find a sucker willing to pay more? You don't know what the world will be in a week, better lock in it now at this higher price. This is not a game I want to play. The US market will end up with companies charging 3x higher prices that have aggressive sales teams constantly trying to book you for a zoom call, and of course they won't tell you the price up front just 30 minutes into the call they will "see what they can do" and boom 3x higher. That's all the market can support, it's impossible to plan for the future and run a stable 30% margin business. The future of tiny corp has always been in making chips. We're still a year from having the software ready, but once we are, we need to be ready with the tape out. We'll sell the chips in a network attached appliance free from any US supply chain, and work with suppliers that are stable in long term win-win cooperation. Have you taped out a chip before and want to come work here? E-mail or DM me. Let's find the easiest fab to work with and get started, something ~12nm to start.

@Object_Zero_ · 2025-04-10 08:14
The required stability for any business model is a function of the profit margin and the length of time between cost of goods and realised revenue. Typically the faster your capital turnover, the lower your operating margin can be. Lots of businesses operate at 10% margin and turnover their working capital every 4-6 weeks. Supermarkets operate at 3% margin with 3-5 day turnover rates. Major infrastructure projects operate at 500% margin and turnover is 5-10 years. But anyone with a turnover slower than a supermarket is exposed to all sorts of volatility risk, and needs stability in order to plan their capital allocation. Those businesses cannot operate in a volatile environment, as the volatility chaotically destroys their capital. Bizarrely, Trump’s volatility hurts long lead manufacturing the most and leaves services and fast retail biz models largely unmolested.

@SemiAnalysis_ · 2025-04-10 08:55
Tariff Armageddon? GPU Loopholes Mexico Supply Chain Shift Wafer Fab Equipment Optical Module Pricing UPS, Generators, Transformers, Switchgear, Power Distribution Equipment, Cooling, Pumps OEM & ODM Supply Chain Softbank Impact Nvidia Balance Sheet Usage https://t.co/AvTVBsHTmu

@teortaxesTex · 2025-04-10 11:10
Some fresh consequences for based e/acc Technology Brothers who pivoted from libertarianism to MAGAtardism to feel like they're WINNING. Maybe it's for the best. Godspeed to George and everyone. https://t.co/wJ1ycMc7NJ

@__tinygrad__ · 2025-04-10 03:31
With the current state of the world, it doesn't make sense to continue in the box business. Unless things radically change in the next few months, there's far too much volatility to run a 30% margin computer business from the US. Even with tariffs "paused" our GPU supplier doesn't want to honor the price we agreed on. And why should they, maybe they can find a sucker willing to pay more? You don't know what the world will be in a week, better lock in it now at this higher price. This is not a game I want to play. The US market will end up with companies charging 3x higher prices that have aggressive sales teams constantly trying to book you for a zoom call, and of course they won't tell you the price up front just 30 minutes into the call they will "see what they can do" and boom 3x higher. That's all the market can support, it's impossible to plan for the future and run a stable 30% margin business. The future of tiny corp has always been in making chips. We're still a year from having the software ready, but once we are, we need to be ready with the tape out. We'll sell the chips in a network attached appliance free from any US supply chain, and work with suppliers that are stable in long term win-win cooperation. Have you taped out a chip before and want to come work here? E-mail or DM me. Let's find the easiest fab to work with and get started, something ~12nm to start.

@mag_pl · 2025-04-10 11:21
@__tinygrad__ Thank god we are all in software with 90% margins. Until someone realizes you can put tariffs on software ...

@teortaxesTex · 2025-04-10 11:10
Some fresh consequences for based e/acc Technology Brothers who pivoted from libertarianism to MAGAtardism to feel like they're WINNING. Maybe it's for the best. Godspeed to George and everyone. https://t.co/wJ1ycMc7NJ

@__tinygrad__ · 2025-04-10 11:21
@teortaxesTex This environment makes the US a fundamentally unserious place to do business in, and when trust is lost it's very hard to regain. You'd be a fool to invest in anything long term. https://t.co/A16Xv2fmYT

@__tinygrad__ · 2025-04-10 11:27
A 1000% tariff on tinygrad wouldn't raise the price.

@thehiphopswami · 2025-04-09 16:32
A career peak for myself! Led physical implementation of this chip from concept to silicon debug. https://t.co/vfMhRfpSxS

@JeffDean · 2025-04-10 14:03
@thehiphopswami @ceobillionaire It's a great chip! Congrats! 🎉 We're looking forward to using it for many things later this year!

@JeffDean · 2025-04-10 14:03
@thehiphopswami @ceobillionaire It's a great chip! Congrats! 🎉 We're looking forward to using it for many things later this year!

@__tinygrad__ · 2025-04-10 14:20
@JeffDean @thehiphopswami @ceobillionaire Any plans to sell them?

@__tinygrad__ · 2025-04-10 14:20
@JeffDean @thehiphopswami @ceobillionaire Any plans to sell them?

@JeffDean · 2025-04-10 16:25
@__tinygrad__ @thehiphopswami @ceobillionaire We have one- and three-year commitment pricing for Cloud TPUs that give substantial discounts off the more ad hoc hourly pricing 😀.

@JeffDean · 2025-04-10 16:25
@__tinygrad__ @thehiphopswami @ceobillionaire We have one- and three-year commitment pricing for Cloud TPUs that give substantial discounts off the more ad hoc hourly pricing 😀.

@__tinygrad__ · 2025-04-10 19:29
@JeffDean @thehiphopswami @ceobillionaire Oh haha I don't want to rent them, I want to buy them! Like NVIDIA and AMD.

@__tinygrad__ · 2025-04-11 06:04
.fuse() is now supported on Tensors. Automatic fusing puts one reduce in a kernel, but if you want more, you can fuse back to the nearest contiguous with fuse. Try it, it's fun! Single kernel softmax works, and with a few tweaks, this is flash attention ($500 bounty). https://t.co/9iI4csuBKI

@__tinygrad__ · 2025-04-12 01:41
If we do tinybox v2 green, how much are people willing to pay as the "USA tariff huge special price"?

@__tinygrad__ · 2025-04-12 01:55
@Marco_Riva10 Yea, that's not going to happen. Have you ever made an electronic before? There's a 4-layer deep supply chain. The tariffs are on materials too. All that's going to happen is prices are going to go up, unless the market can't support them. Then the product just won't be sold.

@__tinygrad__ · 2025-04-12 01:41
If we do tinybox v2 green, how much are people willing to pay as the "USA tariff huge special price"?

@GeorgeLutas1 · 2025-04-12 01:55
@__tinygrad__ I think you want a reasonable markup for your services with SKUs in your BOM (GPUs) that are unreasonable, in an unreasonable market. This seems just the kind of product that is too expensive for what the market is willing to bear. I guess if you want green, you gotta sell red.

@GeorgeLutas1 · 2025-04-12 01:55
@__tinygrad__ I think you want a reasonable markup for your services with SKUs in your BOM (GPUs) that are unreasonable, in an unreasonable market. This seems just the kind of product that is too expensive for what the market is willing to bear. I guess if you want green, you gotta sell red.

@__tinygrad__ · 2025-04-12 01:56
@GeorgeLutas1 People in polls are such liars. Everyone clicking AMD should be buying the $15k tinybox red that's in stock, but they aren't. It's just, wen green plz sir I want green.

@__tinygrad__ · 2025-04-12 01:55
@Marco_Riva10 Yea, that's not going to happen. Have you ever made an electronic before? There's a 4-layer deep supply chain. The tariffs are on materials too. All that's going to happen is prices are going to go up, unless the market can't support them. Then the product just won't be sold.

@__tinygrad__ · 2025-04-12 01:59
@Marco_Riva10 We assemble the tinyboxes in the US. @comma_ai has a PCB line in the US. Some stuff is doable, and if @nvidia would sell us GB202 chips we'd make cards in the US; they won't. What's going to happen as a result of this is more manufacturing moves out of US. High materials cost.

@__tinygrad__ · 2025-04-12 02:02
@RexStructorum There's a global 10% tariff. Our distributors are operating according to supply and demand and taking uncertainty into account (raising prices over 10%). It's hard to blame them.

@__tinygrad__ · 2025-04-12 01:59
@Marco_Riva10 We assemble the tinyboxes in the US. @comma_ai has a PCB line in the US. Some stuff is doable, and if @nvidia would sell us GB202 chips we'd make cards in the US; they won't. What's going to happen as a result of this is more manufacturing moves out of US. High materials cost.

@JordanNanos · 2025-04-12 02:22
@__tinygrad__ @Marco_Riva10 @comma_ai @nvidia USMCA loophole? You guys import the chassis and all components separately?

@JordanNanos · 2025-04-12 02:22
@__tinygrad__ @Marco_Riva10 @comma_ai @nvidia USMCA loophole? You guys import the chassis and all components separately?

@__tinygrad__ · 2025-04-12 02:24
@JordanNanos @Marco_Riva10 @comma_ai @nvidia Not wasting time on loophole crap. If I'm investing time, I'm investing time in building an assembly facility outside the US and only selling to international customers.

@CarlZha · 2025-04-13 08:30
American bro explains why he will continue to have his product made in China even with tariffs https://t.co/dGweXIrlHD

@__tinygrad__ · 2025-04-13 10:01
The bitter lesson strikes again. Spend time on the simple and universal approach. A lost art in software.

@__tinygrad__ · 2025-04-13 14:10
Chip specs: 1 FP16 PFLOP from - 256 tinycores (VLIW, in-order) - 128 KB local SRAM each (L1) - 1024-bit datapath - 8x8 * 8x8 = 8x8 tensor core - dual issue ALU - @ 2 Ghz - Flexible DMA engines for L2 -> L1 - open source silicon-verified HDL 128 MB global 5 TB/s SRAM (L2) 512-bit LPDDR5X @ 500 GB/s (like M4 Max) 12 lanes of 56G SerDes for comm Flexible DMA engines for DDR <-> L2 400W @ 500 mm^2 @ 6 nm Memory system like a GPU, uarch like a DSP. No fancy GPU stuff like memory coalescing and warp scheduling. Software managed caches. Write better software. Notable choices of 6nm process (lower cost per wafer) and LPDDR5X (HBM is too $$$, GDDR is too ⚡). We are optimizing for FLOPS/$, GB/s/$, and GB/$ (and the power included in the 3 year TCO). Package 64 of these in a 4U chassis, 4x 1U 16 chip 4x4 boards. Big water block to cover them. 3D torus scale out. 16 ~2kW PSUs in 3+1, 4 connecting to each board. That's the new tinybox pro. 64 PFLOPS FP16, 2048 GB ram, 32 TB/s RAM bw, 25 kW, $50k. 16 power connectors, 2 water hose connectors, and 6 network ports. Sell in-rack radiator for people who want one box. Sell single chip network attached appliance for $1k. For scale out, we make a 4x4x4 cube of 4x4x4 cubes. Put 8 in a rack, 8 racks, 4 wide and 2 back to back. All copper interconnect. 1.6 MW (big radiator on the roof). 4 exaflops, 128 TB of RAM, $3.2M for the cluster. Steps to get there: 1. support output of ARM, RDNA3, and Qualcomm DSP machine code directly from tinygrad. until we are there, we aren't ready to design a core. 2. design the tinycore in an FPGA, get everything running, even if clock speed is slow. use the FPGA's I/O and RAM 3. tape out a 28 nm test chip with a couple of our cores on it. bring up connected to FPGA for RAM/comms. confirm speed and power draw. ($200k) 4. tape out a 6 nm test chip with a 8 of our cores + 8 MB of global SRAM on it. confirm speed and power draw are industry competitive and meeting targets ($2M) 5. tape out the big 6nm 500 mm^2 chip ($30M). license LPDDR5X + 56G SerDes IP blocks. sell 1200 $50k boxes (at 50% margin) to break even. What are you waiting for. Help finish tinygrad! When it full stack outputs assembly that outperforms torch on NVIDIA we are ready to start on silicon.

@__tinygrad__ · 2025-04-13 14:20
@ZekeGeiss No FP32 input tensor cores, but will have FP32 vector instructions (32 wide). Will include an UNPACK instruction to convert one register with 64 FP16s to 32 FP32s and a corresponding PACK.

@__tinygrad__ · 2025-04-13 14:10
Chip specs: 1 FP16 PFLOP from - 256 tinycores (VLIW, in-order) - 128 KB local SRAM each (L1) - 1024-bit datapath - 8x8 * 8x8 = 8x8 tensor core - dual issue ALU - @ 2 Ghz - Flexible DMA engines for L2 -> L1 - open source silicon-verified HDL 128 MB global 5 TB/s SRAM (L2) 512-bit LPDDR5X @ 500 GB/s (like M4 Max) 12 lanes of 56G SerDes for comm Flexible DMA engines for DDR <-> L2 400W @ 500 mm^2 @ 6 nm Memory system like a GPU, uarch like a DSP. No fancy GPU stuff like memory coalescing and warp scheduling. Software managed caches. Write better software. Notable choices of 6nm process (lower cost per wafer) and LPDDR5X (HBM is too $$$, GDDR is too ⚡). We are optimizing for FLOPS/$, GB/s/$, and GB/$ (and the power included in the 3 year TCO). Package 64 of these in a 4U chassis, 4x 1U 16 chip 4x4 boards. Big water block to cover them. 3D torus scale out. 16 ~2kW PSUs in 3+1, 4 connecting to each board. That's the new tinybox pro. 64 PFLOPS FP16, 2048 GB ram, 32 TB/s RAM bw, 25 kW, $50k. 16 power connectors, 2 water hose connectors, and 6 network ports. Sell in-rack radiator for people who want one box. Sell single chip network attached appliance for $1k. For scale out, we make a 4x4x4 cube of 4x4x4 cubes. Put 8 in a rack, 8 racks, 4 wide and 2 back to back. All copper interconnect. 1.6 MW (big radiator on the roof). 4 exaflops, 128 TB of RAM, $3.2M for the cluster. Steps to get there: 1. support output of ARM, RDNA3, and Qualcomm DSP machine code directly from tinygrad. until we are there, we aren't ready to design a core. 2. design the tinycore in an FPGA, get everything running, even if clock speed is slow. use the FPGA's I/O and RAM 3. tape out a 28 nm test chip with a couple of our cores on it. bring up connected to FPGA for RAM/comms. confirm speed and power draw. ($200k) 4. tape out a 6 nm test chip with a 8 of our cores + 8 MB of global SRAM on it. confirm speed and power draw are industry competitive and meeting targets ($2M) 5. tape out the big 6nm 500 mm^2 chip ($30M). license LPDDR5X + 56G SerDes IP blocks. sell 1200 $50k boxes (at 50% margin) to break even. What are you waiting for. Help finish tinygrad! When it full stack outputs assembly that outperforms torch on NVIDIA we are ready to start on silicon.

@WillimerTercero · 2025-04-13 14:30
@__tinygrad__ 2 million for a 6nm tape out at 6nm without any prior ASIC knowledge? Wake up

@WillimerTercero · 2025-04-13 14:30
@__tinygrad__ 2 million for a 6nm tape out at 6nm without any prior ASIC knowledge? Wake up

@__tinygrad__ · 2025-04-13 14:31
@WillimerTercero $30M for the big chip, $2M for a ~20 mm^2 shuttle run without any IP blocks. Feel free to discuss this plan with ChatGPT, I think it works.

@__tinygrad__ · 2025-04-13 14:38
I asked ChatGPT about the part of the chip plan most likely to fail, nice to see it comes up with what I've been saying forever. The hard part isn't the chip, it's the software. Always has been. Back to work! https://t.co/INdy7DZP4P

@__tinygrad__ · 2025-04-13 14:38
I asked ChatGPT about the part of the chip plan most likely to fail, nice to see it comes up with what I've been saying forever. The hard part isn't the chip, it's the software. Always has been. Back to work! https://t.co/INdy7DZP4P

@armantsaturian · 2025-04-13 14:39
@__tinygrad__ make sure to turn off the memory, otherwise it will feed back to you what you discussed with it

@armantsaturian · 2025-04-13 14:39
@__tinygrad__ make sure to turn off the memory, otherwise it will feed back to you what you discussed with it

@__tinygrad__ · 2025-04-13 14:40
@armantsaturian Have had memory off forever.

@__tinygrad__ · 2025-04-13 14:43
@armanhaghighik Nah, GPUs have a similar jump. 10x reuse in L2 isn't crazy. Re: ISA, maybe the scalar part can be RISC-V? ISA selection is low on my list of concerns.

@__tinygrad__ · 2025-04-13 14:38
I asked ChatGPT about the part of the chip plan most likely to fail, nice to see it comes up with what I've been saying forever. The hard part isn't the chip, it's the software. Always has been. Back to work! https://t.co/INdy7DZP4P

@test_tm7873 · 2025-04-13 14:43
@__tinygrad__ you guys will make custom chips !?!? LETS GOOO! if they will be in my price range, i want to be first costumer XD

@__tinygrad__ · 2025-04-13 14:47
@farajalrashidi We are targeting cost efficiency, not highest perf. But exact details will come later, just important to stay below the reticle size and avoid any costly chiplets or silicon interposers.

@test_tm7873 · 2025-04-13 14:43
@__tinygrad__ you guys will make custom chips !?!? LETS GOOO! if they will be in my price range, i want to be first costumer XD

@__tinygrad__ · 2025-04-13 14:48
@test_tm7873 $1,000 petaflop network attached appliance for home.

@__tinygrad__ · 2025-04-13 14:31
@WillimerTercero $30M for the big chip, $2M for a ~20 mm^2 shuttle run without any IP blocks. Feel free to discuss this plan with ChatGPT, I think it works.

@WillimerTercero · 2025-04-13 14:48
@__tinygrad__ I do this for a living. If you're talking seriously, please not synopsys/cadence sw needed for ASIC synthesis will cost>100k/yr And be prepared for a whole new universe of pain if you pretend to carry out the ASIC backend on your own. If you don't do FPGAs usually, it gets worse

@WillimerTercero · 2025-04-13 14:48
@__tinygrad__ I do this for a living. If you're talking seriously, please not synopsys/cadence sw needed for ASIC synthesis will cost>100k/yr And be prepared for a whole new universe of pain if you pretend to carry out the ASIC backend on your own. If you don't do FPGAs usually, it gets worse

@__tinygrad__ · 2025-04-13 15:01
@WillimerTercero Software and IP licensing costs are factored in. It's totally possible to do a 28nm shuttle for $200k, 6nm shuttle for $2M, and full 6nm tape out for $30M. The test chips will require connection to an FPGA. Where are you disagreeing?

@__tinygrad__ · 2025-04-13 14:10
Chip specs: 1 FP16 PFLOP from - 256 tinycores (VLIW, in-order) - 128 KB local SRAM each (L1) - 1024-bit datapath - 8x8 * 8x8 = 8x8 tensor core - dual issue ALU - @ 2 Ghz - Flexible DMA engines for L2 -> L1 - open source silicon-verified HDL 128 MB global 5 TB/s SRAM (L2) 512-bit LPDDR5X @ 500 GB/s (like M4 Max) 12 lanes of 56G SerDes for comm Flexible DMA engines for DDR <-> L2 400W @ 500 mm^2 @ 6 nm Memory system like a GPU, uarch like a DSP. No fancy GPU stuff like memory coalescing and warp scheduling. Software managed caches. Write better software. Notable choices of 6nm process (lower cost per wafer) and LPDDR5X (HBM is too $$$, GDDR is too ⚡). We are optimizing for FLOPS/$, GB/s/$, and GB/$ (and the power included in the 3 year TCO). Package 64 of these in a 4U chassis, 4x 1U 16 chip 4x4 boards. Big water block to cover them. 3D torus scale out. 16 ~2kW PSUs in 3+1, 4 connecting to each board. That's the new tinybox pro. 64 PFLOPS FP16, 2048 GB ram, 32 TB/s RAM bw, 25 kW, $50k. 16 power connectors, 2 water hose connectors, and 6 network ports. Sell in-rack radiator for people who want one box. Sell single chip network attached appliance for $1k. For scale out, we make a 4x4x4 cube of 4x4x4 cubes. Put 8 in a rack, 8 racks, 4 wide and 2 back to back. All copper interconnect. 1.6 MW (big radiator on the roof). 4 exaflops, 128 TB of RAM, $3.2M for the cluster. Steps to get there: 1. support output of ARM, RDNA3, and Qualcomm DSP machine code directly from tinygrad. until we are there, we aren't ready to design a core. 2. design the tinycore in an FPGA, get everything running, even if clock speed is slow. use the FPGA's I/O and RAM 3. tape out a 28 nm test chip with a couple of our cores on it. bring up connected to FPGA for RAM/comms. confirm speed and power draw. ($200k) 4. tape out a 6 nm test chip with a 8 of our cores + 8 MB of global SRAM on it. confirm speed and power draw are industry competitive and meeting targets ($2M) 5. tape out the big 6nm 500 mm^2 chip ($30M). license LPDDR5X + 56G SerDes IP blocks. sell 1200 $50k boxes (at 50% margin) to break even. What are you waiting for. Help finish tinygrad! When it full stack outputs assembly that outperforms torch on NVIDIA we are ready to start on silicon.

@FrameworkPuter · 2025-04-13 15:02
@__tinygrad__ We’d love to build single-chip boxes around this.

@FrameworkPuter · 2025-04-13 15:02
@__tinygrad__ We’d love to build single-chip boxes around this.

@__tinygrad__ · 2025-04-13 15:07
@FrameworkPuter Nice, happy to sell chips too! Unlike GPU makers, we will sell the chips, cards, systems, and clusters; whatever people want to buy. Probably around $500 for the chip. They won't have PCIe though, just network. Easy to use from any computer, pip install tinygrad + export IP=<ip>

@KBlueleaf · 2025-04-13 17:55
@__tinygrad__ Have tried to design the ALU array + Tesor Core + NoC system in xilinx ultrascale+ FPGA and it do somehow works but I'm not eng in hw so I just leave it as some random side project

@KBlueleaf · 2025-04-13 18:00
@__tinygrad__ It is partially open sourced and I may not have enough time to finish all of them though if anyone is interested in it here it is: https://t.co/bLJfJSsmbL

@KBlueleaf · 2025-04-13 18:00
@__tinygrad__ It is partially open sourced and I may not have enough time to finish all of them though if anyone is interested in it here it is: https://t.co/bLJfJSsmbL

@__tinygrad__ · 2025-04-13 19:33
@KBlueleaf Cool project! Want to write a tinygrad backend for it? We are going to develop on XCVU33P.

@__tinygrad__ · 2025-04-13 14:38
I asked ChatGPT about the part of the chip plan most likely to fail, nice to see it comes up with what I've been saying forever. The hard part isn't the chip, it's the software. Always has been. Back to work! https://t.co/INdy7DZP4P

@TtheBC01 · 2025-04-14 00:13
@__tinygrad__ But if tinygrad becomes the defacto AI software library, haven’t you just removed the largest barrier to entry for everyone else who wants to cook up a new GPU too?

@__tinygrad__ · 2025-04-14 06:47
Yes. Don't rent seek. Commoditize the petaflop.

@__tinygrad__ · 2025-04-14 07:32
RT @yacineMTB: Simple investment strategy that will save you billions of dollars: don't invest unless there is a buy button on their landing page

@maxxyung11 · 2025-04-14 14:07
THERE IS NO SUCH THING AS A LOW-ENERGY, COMPUTE-DENSE SILICON CHIP. Notable standouts: - @GroqInc matches 5nm H100/MI325X on TFLOPS/W performance on a legacy 14nm node! - @__tinygrad__ delivers 18x better TFLOPS/$ than the next leading chip (B200). Sparse FP16 (TFLOPS) vs TDP (W), per chip. Excludes system-level setups like NVL72 or CS-3. Log-log scale. Some data inferred based on private sources. DM if you have better stats or if I made a mistake.

@maxxyung11 · 2025-04-14 14:07
THERE IS NO SUCH THING AS A LOW-ENERGY, COMPUTE-DENSE SILICON CHIP. Notable standouts: - @GroqInc matches 5nm H100/MI325X on TFLOPS/W performance on a legacy 14nm node! - @__tinygrad__ delivers 18x better TFLOPS/$ than the next leading chip (B200). Sparse FP16 (TFLOPS) vs TDP (W), per chip. Excludes system-level setups like NVL72 or CS-3. Log-log scale. Some data inferred based on private sources. DM if you have better stats or if I made a mistake.

@__tinygrad__ · 2025-04-14 15:11
@maxxyung11 @GroqInc Our chip isn't real yet, just a back of the envelope calcuation, so I wouldn't rely on it. Want to add mobile chips? Snapdragon 8 Elite gets 75 TOPS @ 10W. B200 gets 2250 TOPS @ 1000W. 3.3x better.

@__tinygrad__ · 2025-04-14 15:11
@maxxyung11 @GroqInc Our chip isn't real yet, just a back of the envelope calcuation, so I wouldn't rely on it. Want to add mobile chips? Snapdragon 8 Elite gets 75 TOPS @ 10W. B200 gets 2250 TOPS @ 1000W. 3.3x better.

@__tinygrad__ · 2025-04-14 15:16
@maxxyung11 @GroqInc Oh, I'm wrong. B200 gets 4500 TOPS, and 8 elite is half that for FP16. So still close to the 3 TFLOPS/W. Would love for someone to really put effort into this and find all the chips.

@__tinygrad__ · 2025-04-14 15:11
@maxxyung11 @GroqInc Our chip isn't real yet, just a back of the envelope calcuation, so I wouldn't rely on it. Want to add mobile chips? Snapdragon 8 Elite gets 75 TOPS @ 10W. B200 gets 2250 TOPS @ 1000W. 3.3x better.

@maxxyung11 · 2025-04-14 19:31
@__tinygrad__ @GroqInc I see. I'll update to reflect the status. What is an individual unit called though - I'm just referring to it as "1x Tinybox Chip" and took the numbers from the Tinybox Pro divided by 64.

@__tinygrad__ · 2025-04-14 15:16
@maxxyung11 @GroqInc Oh, I'm wrong. B200 gets 4500 TOPS, and 8 elite is half that for FP16. So still close to the 3 TFLOPS/W. Would love for someone to really put effort into this and find all the chips.

@maxxyung11 · 2025-04-14 19:33
@__tinygrad__ @GroqInc Hesitant to add mobile chips due to the ambiguity of the data. I mostly have numbers for commercial/enterprise. For the 8 Elite, I found it to run at ~7.4 FP16 TFLOPS at 8W. That's 0.93 TFLOPS/W. How did you get your numbers?

@maxxyung11 · 2025-04-14 19:31
@__tinygrad__ @GroqInc I see. I'll update to reflect the status. What is an individual unit called though - I'm just referring to it as "1x Tinybox Chip" and took the numbers from the Tinybox Pro divided by 64.

@__tinygrad__ · 2025-04-14 21:59
@maxxyung11 @GroqInc That chip isn't real. Current tinyboxes are 7900XTX and 5090, terrible TFLOPS/W.

@maxxyung11 · 2025-04-14 19:33
@__tinygrad__ @GroqInc Hesitant to add mobile chips due to the ambiguity of the data. I mostly have numbers for commercial/enterprise. For the 8 Elite, I found it to run at ~7.4 FP16 TFLOPS at 8W. That's 0.93 TFLOPS/W. How did you get your numbers?

@__tinygrad__ · 2025-04-14 22:03
@maxxyung11 @GroqInc That's the GPU I think, not the NPU.

@CarlZha · 2025-04-13 08:30
American bro explains why he will continue to have his product made in China even with tariffs https://t.co/dGweXIrlHD

@__tinygrad__ · 2025-04-15 12:44
@CarlZha This is our experience as well, our computers come in a similar case. It's not even about money, it's about how hard American companies are to work with compared to the Chinese.

@SemiAnalysis_ · 2025-04-16 04:45
Huawei AI CloudMatrix 384 China’s Answer to Nvidia GB200 NVL72 China Abundance of Power, 100% Optics, 0% Copper Power Inefficiency 2.6x lower FLOP per Watt 14 Transceivers per Chip, Linear Pluggable Optics https://t.co/EhhcgXmDLs

@dylan522p · 2025-04-16 05:10
Huawei's new AI server is insanely good People need to reset their priors This is why banning H20 without banning tools and sub components is idiotic because Huawei is not far behind H20. The admin needs to act fast to slow down Huawei's ramp or the H20 ban will be useless

@__tinygrad__ · 2025-04-16 07:27
@dylan522p How about we remove all export controls on NVIDIA and just be competitive instead of trying to slow others down.

@HmOhFour · 2025-04-16 07:55
@__tinygrad__ @dylan522p The tacit admission is that we'd lose if we did that.

@HmOhFour · 2025-04-16 07:55
@__tinygrad__ @dylan522p The tacit admission is that we'd lose if we did that.

@__tinygrad__ · 2025-04-16 10:02
If that's true, than we have already lost. But I don't think it's true at all, have you ever tried to use a Chinese AI accelerator? Continuing with export controls absolutely guarantees a loss, it hands China the world market on a silver platter. If tiny corp faces export controls on USA product, we will switch to Chinese product, even if it isn't as good. And this isn't just us, it's every country. AI accelerators are one place where the US has a big lead, and there's no fundamental reason we can't keep it. Unless the government manages to mess with it enough and break it. If I were Jensen I'd be livid about the billion I put into the the H20 to try to comply with power that is behaving psychotically just to be rugged.

@__tinygrad__ · 2025-04-16 12:47
RT @xingyu_liao: got my first PR in tinygrad merged! https://t.co/JpLDcifKUU

@__tinygrad__ · 2025-04-22 07:40
tinybox green v2 back available for order. Now $29,000 due to the impact of tariffs, and if this continues, we're expecting prices to increase more. Lock in this price now!

@__tinygrad__ · 2025-04-22 07:40
tinybox green v2 back available for order. Now $29,000 due to the impact of tariffs, and if this continues, we're expecting prices to increase more. Lock in this price now!

@Zzrott1 · 2025-04-22 08:14
@__tinygrad__ Real-time auto dynamic pricing was always the future anyway

@Zzrott1 · 2025-04-22 08:14
@__tinygrad__ Real-time auto dynamic pricing was always the future anyway

@__tinygrad__ · 2025-04-22 10:04
@Zzrott1 It's doubtful we are going to reduce prices for this gen. We were reluctant to raise prices, we raised them as little as we could to keep a 30% margin. And this is still using some stock we already had. If the tariff situation is similar in 6 months, prices will go up again.

@__tinygrad__ · 2025-04-26 13:03
Pretty accurate. https://t.co/bXD0BrSkUw https://t.co/9HuOp4nMHS

@willcb · 2025-04-26 18:24
what's the catch with these? cannot find any discussion online about them, at first glance they seem like somewhat reasonable tinybox alternatives but the vibes are also kinda off... the company also sells standing desks... what's going on here? https://t.co/ixf4SRUjzf

@willcb · 2025-04-26 18:24
what's the catch with these? cannot find any discussion online about them, at first glance they seem like somewhat reasonable tinybox alternatives but the vibes are also kinda off... the company also sells standing desks... what's going on here? https://t.co/ixf4SRUjzf

@__tinygrad__ · 2025-04-27 11:18
@willcb If the team doesn't have experience doing machine learning, I'd be concerned about if the required testing was done for PCIe and power stability. We have the green v2 back in stock, shipping in a few weeks.

@__tinygrad__ · 2025-04-27 11:18
@willcb If the team doesn't have experience doing machine learning, I'd be concerned about if the required testing was done for PCIe and power stability. We have the green v2 back in stock, shipping in a few weeks.

@roman_koshchei · 2025-04-27 11:47
@__tinygrad__ @willcb Need tinybox for 5k usd, 15k is too much

@__tinygrad__ · 2025-04-27 11:18
@willcb If the team doesn't have experience doing machine learning, I'd be concerned about if the required testing was done for PCIe and power stability. We have the green v2 back in stock, shipping in a few weeks.

@roman_koshchei · 2025-04-27 11:47
@__tinygrad__ @willcb Need tinybox for 5k usd, 15k is too much

@__tinygrad__ · 2025-04-27 12:01
If you want a machine learning computer for $5k, buy a gaming PC. If you are buying something special purpose in that price range or lower, you are being ripped off.

@__tinygrad__ · 2025-04-27 12:46
The core abstractions in tinygrad are getting quite good. Everything is UOps, they are pretty fast, they are immutable, they are debuggable with VIZ=1, and they are pretty easy to understand. With a solid platform to build on, we can focus on speed and usability. Here are some directions for the rest of the year. With the DSP project, we have shown we can get speed on even the hardest platforms, strict SIMD with software managed caches and a weak memory system. However, our code to do this isn't generic yet. Locals in GPUs need to become vectorized reflecting the real SIMT architecture of the GPU, then tensor cores and down shuffles are graph rewrites. This is what triton does. There's a big project of local memory too. Currently we only use it in very limited ways. Making this more generic is the last barrier to full speed GEMMs, and finding smart ways to search over memory layouts. Flash attention is close to working, we have the fusion correct (entirely generic!). It won't make it in for BERT this MLPerf cycle, but it'll make it in for the next one. We're working on RANGE support in the overworld, allowing your multistep training to be scheduled only once and looped. This same infra will also work for multilayer LLMs. On the driver front, our USB stuff is almost done. AMD support first, you'll be able to plug a GPU into a USB port and drive it entirely from tinygrad's userspace. Our WEBGPU stuff is doing great, tinygrad is by far the best way to deploy a model to the web (check out our demos). We have a torch and onnx frontend to import from, so you don't need to write your model in tinygrad. We're working to bring all this stuff into the main tree in a very clean way. We're looking for new contributors, particularly people who want to work on speed. The infrastructure is to the point that things like autogen flash attention are ready to be made fast. Also people who want to work a level lower on the vectorized SIMT stuff. Do you know what memory coalescing is?

@__tinygrad__ · 2025-04-27 12:46
The core abstractions in tinygrad are getting quite good. Everything is UOps, they are pretty fast, they are immutable, they are debuggable with VIZ=1, and they are pretty easy to understand. With a solid platform to build on, we can focus on speed and usability. Here are some directions for the rest of the year. With the DSP project, we have shown we can get speed on even the hardest platforms, strict SIMD with software managed caches and a weak memory system. However, our code to do this isn't generic yet. Locals in GPUs need to become vectorized reflecting the real SIMT architecture of the GPU, then tensor cores and down shuffles are graph rewrites. This is what triton does. There's a big project of local memory too. Currently we only use it in very limited ways. Making this more generic is the last barrier to full speed GEMMs, and finding smart ways to search over memory layouts. Flash attention is close to working, we have the fusion correct (entirely generic!). It won't make it in for BERT this MLPerf cycle, but it'll make it in for the next one. We're working on RANGE support in the overworld, allowing your multistep training to be scheduled only once and looped. This same infra will also work for multilayer LLMs. On the driver front, our USB stuff is almost done. AMD support first, you'll be able to plug a GPU into a USB port and drive it entirely from tinygrad's userspace. Our WEBGPU stuff is doing great, tinygrad is by far the best way to deploy a model to the web (check out our demos). We have a torch and onnx frontend to import from, so you don't need to write your model in tinygrad. We're working to bring all this stuff into the main tree in a very clean way. We're looking for new contributors, particularly people who want to work on speed. The infrastructure is to the point that things like autogen flash attention are ready to be made fast. Also people who want to work a level lower on the vectorized SIMT stuff. Do you know what memory coalescing is?

@jsuarez · 2025-04-27 12:55
@__tinygrad__ Man I want to try your software, but it's so hard to justify when clients and users are going to expect pytorch and bounce off of anything else

@__tinygrad__ · 2025-04-27 12:01
If you want a machine learning computer for $5k, buy a gaming PC. If you are buying something special purpose in that price range or lower, you are being ripped off.

@twiz19071051 · 2025-04-27 13:18
@__tinygrad__ but then how do you avoid using it for gaming🤔

@__tinygrad__ · 2025-04-27 12:46
The core abstractions in tinygrad are getting quite good. Everything is UOps, they are pretty fast, they are immutable, they are debuggable with VIZ=1, and they are pretty easy to understand. With a solid platform to build on, we can focus on speed and usability. Here are some directions for the rest of the year. With the DSP project, we have shown we can get speed on even the hardest platforms, strict SIMD with software managed caches and a weak memory system. However, our code to do this isn't generic yet. Locals in GPUs need to become vectorized reflecting the real SIMT architecture of the GPU, then tensor cores and down shuffles are graph rewrites. This is what triton does. There's a big project of local memory too. Currently we only use it in very limited ways. Making this more generic is the last barrier to full speed GEMMs, and finding smart ways to search over memory layouts. Flash attention is close to working, we have the fusion correct (entirely generic!). It won't make it in for BERT this MLPerf cycle, but it'll make it in for the next one. We're working on RANGE support in the overworld, allowing your multistep training to be scheduled only once and looped. This same infra will also work for multilayer LLMs. On the driver front, our USB stuff is almost done. AMD support first, you'll be able to plug a GPU into a USB port and drive it entirely from tinygrad's userspace. Our WEBGPU stuff is doing great, tinygrad is by far the best way to deploy a model to the web (check out our demos). We have a torch and onnx frontend to import from, so you don't need to write your model in tinygrad. We're working to bring all this stuff into the main tree in a very clean way. We're looking for new contributors, particularly people who want to work on speed. The infrastructure is to the point that things like autogen flash attention are ready to be made fast. Also people who want to work a level lower on the vectorized SIMT stuff. Do you know what memory coalescing is?

@bender_2716057 · 2025-04-27 13:41
@__tinygrad__ Are you going to implement a proper whisper+webgpu frontend or am I going to take over this?

@bender_2716057 · 2025-04-27 13:41
@__tinygrad__ Are you going to implement a proper whisper+webgpu frontend or am I going to take over this?

@__tinygrad__ · 2025-04-27 15:16
@bender_2716057 We are focused on the library, but that sounds like a great application! It shouldn't be hard, we have whisper in tinygrad and a flexible export, you'd just have to write the JS to access the microphone and wire it up.

@jsuarez · 2025-04-27 12:55
@__tinygrad__ Man I want to try your software, but it's so hard to justify when clients and users are going to expect pytorch and bounce off of anything else

@__tinygrad__ · 2025-04-27 15:17
@jsuarez To be fair, if you are training on NVIDIA, PyTorch is still king. End of the year for speed parity, and another year likely for tinygrad to have serious advantages. But everywhere else besides training on NVIDIA, tinygrad is pretty competitive, especially for embedded and export

@twiz19071051 · 2025-04-27 13:18
@__tinygrad__ but then how do you avoid using it for gaming🤔

@__tinygrad__ · 2025-04-27 15:41
@twiz19071051 Delete Windows. Install Linux.

@__tinygrad__ · 2025-04-27 15:17
@jsuarez To be fair, if you are training on NVIDIA, PyTorch is still king. End of the year for speed parity, and another year likely for tinygrad to have serious advantages. But everywhere else besides training on NVIDIA, tinygrad is pretty competitive, especially for embedded and export

@maorioriori · 2025-04-27 16:22
@__tinygrad__ @jsuarez Tinygrad is also faster than Pytorch for the arm mac machines no?

@__tinygrad__ · 2025-04-27 12:46
The core abstractions in tinygrad are getting quite good. Everything is UOps, they are pretty fast, they are immutable, they are debuggable with VIZ=1, and they are pretty easy to understand. With a solid platform to build on, we can focus on speed and usability. Here are some directions for the rest of the year. With the DSP project, we have shown we can get speed on even the hardest platforms, strict SIMD with software managed caches and a weak memory system. However, our code to do this isn't generic yet. Locals in GPUs need to become vectorized reflecting the real SIMT architecture of the GPU, then tensor cores and down shuffles are graph rewrites. This is what triton does. There's a big project of local memory too. Currently we only use it in very limited ways. Making this more generic is the last barrier to full speed GEMMs, and finding smart ways to search over memory layouts. Flash attention is close to working, we have the fusion correct (entirely generic!). It won't make it in for BERT this MLPerf cycle, but it'll make it in for the next one. We're working on RANGE support in the overworld, allowing your multistep training to be scheduled only once and looped. This same infra will also work for multilayer LLMs. On the driver front, our USB stuff is almost done. AMD support first, you'll be able to plug a GPU into a USB port and drive it entirely from tinygrad's userspace. Our WEBGPU stuff is doing great, tinygrad is by far the best way to deploy a model to the web (check out our demos). We have a torch and onnx frontend to import from, so you don't need to write your model in tinygrad. We're working to bring all this stuff into the main tree in a very clean way. We're looking for new contributors, particularly people who want to work on speed. The infrastructure is to the point that things like autogen flash attention are ready to be made fast. Also people who want to work a level lower on the vectorized SIMT stuff. Do you know what memory coalescing is?

@NigelHiggs7 · 2025-04-27 16:31
@__tinygrad__ Woah, is it common to be able and run a gpu over usb? Thats kinda sick, I could just create a setup rig to power the gpu and over the wire send/receive data?

@maorioriori · 2025-04-27 16:22
@__tinygrad__ @jsuarez Tinygrad is also faster than Pytorch for the arm mac machines no?

@__tinygrad__ · 2025-04-27 16:55
@maorioriori @jsuarez In most cases with BEAM=2, yes. Perf is similar to MLX.

@NigelHiggs7 · 2025-04-27 16:31
@__tinygrad__ Woah, is it common to be able and run a gpu over usb? Thats kinda sick, I could just create a setup rig to power the gpu and over the wire send/receive data?

@__tinygrad__ · 2025-04-27 16:56
@NigelHiggs7 Very uncommon. We have put a year of engineering into it. It's how @comma_ai is going to run big models in cars.

@__tinygrad__ · 2025-04-28 12:47
Who wants to fix bugs in tinygrad? Enable single kernel softmax, run the test suite, and go! `SINGLE_KERNEL_SOFTMAX=1 pytest -n auto test` https://t.co/K6uhrB5DHe

@__tinygrad__ · 2025-04-28 12:47
Who wants to fix bugs in tinygrad? Enable single kernel softmax, run the test suite, and go! `SINGLE_KERNEL_SOFTMAX=1 pytest -n auto test` https://t.co/K6uhrB5DHe

@juscallmevyom · 2025-04-28 12:56
@__tinygrad__ give ssh

@juscallmevyom · 2025-04-28 12:56
@__tinygrad__ give ssh

@__tinygrad__ · 2025-04-28 13:15
@vyomdundigalla tinygrad is pure Python with ~0 deps. You can run this on whatever hardware you have. That screenshot is from a MacBook.

@__tinygrad__ · 2025-04-28 13:27
Can confirm. If every US household had a tinybox, it would draw about a current USA of electricity.

@__tinygrad__ · 2025-04-28 22:08
Okay, we heard you on z3 as a dep. How about pycosat? You don't seriously want us to write a SAT solver, right? And we should be using one.

@__tinygrad__ · 2025-04-29 00:29
@kalocide Verification to start. We actually already have Z3 in there to confirm we never access out of bounds memory, but it's optional. Set IGNORE_OOB=0 to enable.

@__tinygrad__ · 2025-04-29 14:57
Good morning from this GEMM https://t.co/ZaD10ns6ad

@__tinygrad__ · 2025-04-29 14:57
Good morning from this GEMM https://t.co/ZaD10ns6ad

@precisemove · 2025-04-29 15:01
@__tinygrad__ 99% of your clients don't understand this pic, graphic, maybe hire a communicator that reaches more people imo

@__tinygrad__ · 2025-04-29 14:57
Good morning from this GEMM https://t.co/ZaD10ns6ad

@Leik0w0 · 2025-04-29 15:04
@__tinygrad__ That’s extremely clean. Maybe you could post one from a year ago besides it to really show how different it is now

@precisemove · 2025-04-29 15:01
@__tinygrad__ 99% of your clients don't understand this pic, graphic, maybe hire a communicator that reaches more people imo

@__tinygrad__ · 2025-04-29 15:33
@precisemove lol. we don't have clients. we write open source software in search of truth and beauty. we also sell metal boxes for more than they cost to make. you are invited on this journey, but only if you understand.

@Leik0w0 · 2025-04-29 15:04
@__tinygrad__ That’s extremely clean. Maybe you could post one from a year ago besides it to really show how different it is now

@__tinygrad__ · 2025-04-29 15:34
@Leik0w0 the gemm is similar. what's improved is we can now express things like single kernel softmax (below), and very close to flash attention. https://t.co/UZNS2mQigB

@jukan05 · 2025-04-29 19:43
Exclusive: Trump officials eye changes to Biden's AI chip export rule, sources say - Reuters The Trump administration is working on changes to a Biden-era rule that would limit global access to AI chips, including possibly doing away with its splitting the world into tiers that help determine how many advanced semiconductors a country can obtain, three sources familiar with the matter said. The sources said the plans were still under discussion and warned they could change. But if enacted, removing the tiers could open the door to using U.S. chips as an even more powerful negotiating tool in trade talks. The regulation, which was issued in January, is aimed at dividing up access to the most advanced AI chips and controlling certain model weights in order to keep the most sophisticated computing power in the United States and among its allies, and away from China and other countries of concern. The Framework for Artificial Intelligence Diffusion, as the rule is called, was issued by the U.S. Department of Commerce in January, a week before the end of the administration of former President Joe Biden. Companies must comply with its restrictions starting on May 15. Currently, the rule has the world divided into three tiers. Seventeen countries and Taiwan in the first tier can receive unlimited chips. Some 120 other countries are in the second tier, which leaves them subject to caps on how many AI chips they can get. And countries of concern like China, Russia, Iran and North Korea in the third tier are blocked from the chips. But Trump administration officials are weighing discarding the tiered approach to access in the rule and replacing it with a global licensing regime with government-to-government agreements, the sources said. "There are some voices pushing for elimination of the tiers," Wilbur Ross, who served as Commerce secretary during the first Trump administration, said in an interview on Tuesday. "I think it's still a work in progress." He said government-to-government agreements were one alternative. Such a structure would likely tie in to President Donald Trump's broader trade strategy of making deals with individual countries, one of the sources said. That would make it easier for the U.S. to use access to American-designed chips as leverage in other negotiations. U.S. Commerce Secretary Howard Lutnick said at a conference in March that he wants to include export controls in trade talks. Other possible changes include a lower threshold for an exception to licensing. Under the current rule, orders under the equivalent of about 1,700 of Nvidia's (NVDA.O), opens new tab powerful H100 chips do not count toward country caps and only require the government be notified about the order. No license is necessary. The Trump administration is considering making the cutoff orders under the equivalent of 500 H100 chips, one source said. A spokesperson for the Commerce Department declined comment. A spokesperson for the White House did not immediately respond to a request for comment. For months, Trump administration officials have suggested they want to make the rule "stronger but simpler," but at least some experts believe removing the tiers will make the rule more complicated. Ken Glueck, executive vice president at Oracle (ORCL.N), opens new tab, a critic of the current rule, said that the tiers did not make sense, noting that Israel and Yemen were both in the second tier. "Wouldn't surprise me they're going to take a new look at this," said Glueck, who said he did not know the Trump administration's plan but expects the rule to be modified in a significant way. Oracle and Nvidia were both outspoken in their criticism of the new rule when it was issued in January. Industry has argued that by limiting access to the chips, countries will buy the technology from China. Some U.S. lawmakers have agreed. Seven Republican senators sent a letter to Lutnick in mid-April asking for the rule to be withdrawn. The restrictions would incentivize buyers, especially in Tier 2 countries, to turn to China's "unregulated cheap substitutes," the letter said. $NVDA https://t.co/3gVEWa7A7a

@__tinygrad__ · 2025-04-29 20:10
@PytorchToAtoms tinygrad already does this

@__tinygrad__ · 2025-04-29 20:14
@PytorchToAtoms you can use tinygrad as a torch backend, which will do this.

@jukan05 · 2025-04-29 19:43
Exclusive: Trump officials eye changes to Biden's AI chip export rule, sources say - Reuters The Trump administration is working on changes to a Biden-era rule that would limit global access to AI chips, including possibly doing away with its splitting the world into tiers that help determine how many advanced semiconductors a country can obtain, three sources familiar with the matter said. The sources said the plans were still under discussion and warned they could change. But if enacted, removing the tiers could open the door to using U.S. chips as an even more powerful negotiating tool in trade talks. The regulation, which was issued in January, is aimed at dividing up access to the most advanced AI chips and controlling certain model weights in order to keep the most sophisticated computing power in the United States and among its allies, and away from China and other countries of concern. The Framework for Artificial Intelligence Diffusion, as the rule is called, was issued by the U.S. Department of Commerce in January, a week before the end of the administration of former President Joe Biden. Companies must comply with its restrictions starting on May 15. Currently, the rule has the world divided into three tiers. Seventeen countries and Taiwan in the first tier can receive unlimited chips. Some 120 other countries are in the second tier, which leaves them subject to caps on how many AI chips they can get. And countries of concern like China, Russia, Iran and North Korea in the third tier are blocked from the chips. But Trump administration officials are weighing discarding the tiered approach to access in the rule and replacing it with a global licensing regime with government-to-government agreements, the sources said. "There are some voices pushing for elimination of the tiers," Wilbur Ross, who served as Commerce secretary during the first Trump administration, said in an interview on Tuesday. "I think it's still a work in progress." He said government-to-government agreements were one alternative. Such a structure would likely tie in to President Donald Trump's broader trade strategy of making deals with individual countries, one of the sources said. That would make it easier for the U.S. to use access to American-designed chips as leverage in other negotiations. U.S. Commerce Secretary Howard Lutnick said at a conference in March that he wants to include export controls in trade talks. Other possible changes include a lower threshold for an exception to licensing. Under the current rule, orders under the equivalent of about 1,700 of Nvidia's (NVDA.O), opens new tab powerful H100 chips do not count toward country caps and only require the government be notified about the order. No license is necessary. The Trump administration is considering making the cutoff orders under the equivalent of 500 H100 chips, one source said. A spokesperson for the Commerce Department declined comment. A spokesperson for the White House did not immediately respond to a request for comment. For months, Trump administration officials have suggested they want to make the rule "stronger but simpler," but at least some experts believe removing the tiers will make the rule more complicated. Ken Glueck, executive vice president at Oracle (ORCL.N), opens new tab, a critic of the current rule, said that the tiers did not make sense, noting that Israel and Yemen were both in the second tier. "Wouldn't surprise me they're going to take a new look at this," said Glueck, who said he did not know the Trump administration's plan but expects the rule to be modified in a significant way. Oracle and Nvidia were both outspoken in their criticism of the new rule when it was issued in January. Industry has argued that by limiting access to the chips, countries will buy the technology from China. Some U.S. lawmakers have agreed. Seven Republican senators sent a letter to Lutnick in mid-April asking for the rule to be withdrawn. The restrictions would incentivize buyers, especially in Tier 2 countries, to turn to China's "unregulated cheap substitutes," the letter said. $NVDA https://t.co/3gVEWa7A7a

@__tinygrad__ · 2025-04-29 20:50
@Jukanlosreve "global licensing regime" aka plz use Chinese chips we are going to be a huge PITA to deal with.

@__tinygrad__ · 2025-04-29 20:56
No working software you say? @tenstorrent, want to fund a high performance tinygrad backend for your chips?

@__tinygrad__ · 2025-04-29 21:05
We've submitted the MI300X to MLPerf for BERT (in addition to tinybox red and green). Will we still be the only group to get AMD on MLPerf training?

@bender_2716057 · 2025-04-29 21:15
@__tinygrad__ whisper large on tinygrad is way slower than faster-whisper on CTranslate2, both CPU-based. Is this expected?

@bender_2716057 · 2025-04-29 21:15
@__tinygrad__ whisper large on tinygrad is way slower than faster-whisper on CTranslate2, both CPU-based. Is this expected?

@__tinygrad__ · 2025-04-29 21:20
@bender_2716057 Are you running with BEAM=2? That's a place to start.

@__tinygrad__ · 2025-04-29 22:20
@FelixCLC_ @tenstorrent We ordered one, excited to get it!

@__tinygrad__ · 2025-04-29 21:05
We've submitted the MI300X to MLPerf for BERT (in addition to tinybox red and green). Will we still be the only group to get AMD on MLPerf training?

@HotAisle · 2025-04-29 23:43
@__tinygrad__ Why are you holding out on sharing the results or is that part of the submission process?

@roman_koshchei · 2025-04-30 05:47
tinygrad going around offering every chip production company to implement software side. nice move and it's good for developers, because we will be able to use same thing for all hardware

@__tinygrad__ · 2025-04-30 14:08
We do mainstream accelerators for free, but if you want us to do non-mainstream or high performance for your accelerator, we're open to contracts. We just finished a contract for the Qualcomm DSP, MobileNetV2 on par with SNPE speed on 845.

@HotAisle · 2025-04-29 23:43
@__tinygrad__ Why are you holding out on sharing the results or is that part of the submission process?

@__tinygrad__ · 2025-04-30 14:13
@HotAisle Until it's verified by MLPerf we shouldn't share exact numbers, will be out in a month. The speed is pretty meh, but correctness is a lot more important than performance, at least it's on the board. As previously discussed with AMD, we'd be open to a contract to outperform H100.

@__tinygrad__ · 2025-04-30 14:13
@HotAisle Until it's verified by MLPerf we shouldn't share exact numbers, will be out in a month. The speed is pretty meh, but correctness is a lot more important than performance, at least it's on the board. As previously discussed with AMD, we'd be open to a contract to outperform H100.

@HotAisle · 2025-04-30 14:30
Thanks. I still don’t understand why you shouldn’t share it early. Why is MLPerf the gatekeeper? It is code and an early result. As for perf, this is obviously a reflection on the software and not the hardware. We have tons of great results on mi300x, when software is tuned. You’re right that correctness is king, but that’s not the only thing a perf test is looking for here. So, now you want 2 boxes for free and a contract to make your code faster. Seems like the goalposts keep moving…

@HotAisle · 2025-04-30 14:30
Thanks. I still don’t understand why you shouldn’t share it early. Why is MLPerf the gatekeeper? It is code and an early result. As for perf, this is obviously a reflection on the software and not the hardware. We have tons of great results on mi300x, when software is tuned. You’re right that correctness is king, but that’s not the only thing a perf test is looking for here. So, now you want 2 boxes for free and a contract to make your code faster. Seems like the goalposts keep moving…

@__tinygrad__ · 2025-04-30 14:32
@HotAisle From Jun 2024. No moving goalposts. The submission for the boxes, the speed for the $$$. https://t.co/KDfJEWx5eX

@__tinygrad__ · 2025-04-30 14:47
RT @0xhooved: The kernel graph for Llama inference in @__tinygrad__ https://t.co/KrjyDjaEce

@hi_yoniyang · 2025-05-01 04:15
@__tinygrad__ would like to see a rockchip port

@__tinygrad__ · 2025-05-01 17:52
Added a $500 bounty for a Rockchip RK3588 NPU backend passing all ops tests. It's actually documented at the register level in Chapter 36 of the TRM.

@__tinygrad__ · 2025-05-01 17:52
Added a $500 bounty for a Rockchip RK3588 NPU backend passing all ops tests. It's actually documented at the register level in Chapter 36 of the TRM.

@roman_koshchei · 2025-05-01 17:59
@__tinygrad__ Great, the best news today. Rockchip RK3588 NPU is one of the go-to options for edge devices I would say. Plan to use it myself.

@roman_koshchei · 2025-05-01 17:59
@__tinygrad__ Great, the best news today. Rockchip RK3588 NPU is one of the go-to options for edge devices I would say. Plan to use it myself.

@__tinygrad__ · 2025-05-01 18:01
@roman_koshchei tinygrad's OpenCL backend is already very good on that chip using the GPU btw. This would add even more power.

@__tinygrad__ · 2025-05-03 10:53
RT @virajitgp: even @VitalikButerin knows that @__tinygrad__ is the goat. https://t.co/rzCsDelNsj

@__tinygrad__ · 2025-05-03 23:27
Four tinybox reds provisioned and ready to ship. Beat the GPU shortage with a tinybox red, still only $15k! https://t.co/onYd2tZ5AW

@__tinygrad__ · 2025-05-03 23:27
Four tinybox reds provisioned and ready to ship. Beat the GPU shortage with a tinybox red, still only $15k! https://t.co/onYd2tZ5AW

@payraw · 2025-05-03 23:46
@__tinygrad__ would red v2 be 4x 4090?

@payraw · 2025-05-03 23:46
@__tinygrad__ would red v2 be 4x 4090?

@__tinygrad__ · 2025-05-03 23:49
@payraw No red v2 currently planned. This generation AMD didn't make a high end card, or even a card with 24GB. The 7900XTXs in here are still the best AMD card.

@__tinygrad__ · 2025-05-04 23:48
Is this an LLVM bug, or are we somehow holding it wrong? https://t.co/FY8wq54erJ

@__tinygrad__ · 2025-05-05 13:22
We have GPUs in hand to build 10 tinybox green v2! 3 are already bought and paid for, 7 more ship this month, then we pray for more GPUs. Get your order in ASAP. https://t.co/Qfeofcsmeq

@__tinygrad__ · 2025-05-05 13:22
We have GPUs in hand to build 10 tinybox green v2! 3 are already bought and paid for, 7 more ship this month, then we pray for more GPUs. Get your order in ASAP. https://t.co/Qfeofcsmeq

@Michi07f · 2025-05-05 13:43
@__tinygrad__ Is p2p fully working as it was on the previous gen after the patch?

@__tinygrad__ · 2025-05-05 13:22
We have GPUs in hand to build 10 tinybox green v2! 3 are already bought and paid for, 7 more ship this month, then we pray for more GPUs. Get your order in ASAP. https://t.co/Qfeofcsmeq

@__tinygrad__ · 2025-05-05 14:01
5 left, wow I wish tinybox reds moved like this. Here's a picture of 4 v2 cases in our San Diego, CA production facility, each on their own Lazy Susan. https://t.co/xbwlwHuUNr

@__tinygrad__ · 2025-05-05 18:32
tinybox green v2 supports P2P between the 5090s using our modified driver! This means the bytes go directly between the GPUs and don't have to go to CPU RAM. Works in both tinygrad and PyTorch (anything with nccl). https://t.co/ttvwIJEpk6

@neuropunk_eth · 2025-05-05 18:33
god i want her so bad

@__tinygrad__ · 2025-05-05 18:32
tinybox green v2 supports P2P between the 5090s using our modified driver! This means the bytes go directly between the GPUs and don't have to go to CPU RAM. Works in both tinygrad and PyTorch (anything with nccl). https://t.co/ttvwIJEpk6

@__tinygrad__ · 2025-05-05 18:37
32 GB/s tested all reduce bandwidth, more than double what we were seeing with 4090s. This scaled faster than RAM capacity and is plenty fast for all practical training jobs. https://t.co/fTHLSa2OjA

@neuropunk_eth · 2025-05-05 18:33
god i want her so bad

@__tinygrad__ · 2025-05-05 18:39
@neuropunk_eth Don't miss out, only 5 left in the first batch!

@Michi07f · 2025-05-05 13:43
@__tinygrad__ Is p2p fully working as it was on the previous gen after the patch?

@__tinygrad__ · 2025-05-05 18:41
@Michi07f Yes. https://t.co/PxswDlJbDn

@__tinygrad__ · 2025-05-05 18:48
@PytorchToAtoms Yes, it goes through the root complex but it's not much slower. You might get 5-10% more with a switch, but so not worth it. Latency is also totally fine with this. https://t.co/iQy9mkI8pv

@__tinygrad__ · 2025-05-05 18:49
@ergobrained Meh. These mods are expensive and don't increase VRAM bandwidth. If you want 96GB, just use 3 5090s!

@__tinygrad__ · 2025-05-05 18:32
tinybox green v2 supports P2P between the 5090s using our modified driver! This means the bytes go directly between the GPUs and don't have to go to CPU RAM. Works in both tinygrad and PyTorch (anything with nccl). https://t.co/ttvwIJEpk6

@KBlueleaf · 2025-05-05 19:09
@__tinygrad__ Any link/instruction to the modified driver? I have epyc server MB which should support p2p and want to try it!

@osbenet · 2025-05-05 20:34
@KBlueleaf @__tinygrad__ hmm how feasible is FSDP in this case

@KBlueleaf · 2025-05-05 20:43
@osbenet @__tinygrad__ 2card FSDP is pretty feasible in pcie 4.0 p2p but 4card... pcie5.0 or 6.0 p2p is required

@KBlueleaf · 2025-05-05 19:09
@__tinygrad__ Any link/instruction to the modified driver? I have epyc server MB which should support p2p and want to try it!

@__tinygrad__ · 2025-05-05 20:44
@KBlueleaf Code is here, but it looks like you already got it! https://t.co/7U4brWUoUX

@KBlueleaf · 2025-05-05 20:43
@osbenet @__tinygrad__ 2card FSDP is pretty feasible in pcie 4.0 p2p but 4card... pcie5.0 or 6.0 p2p is required

@__tinygrad__ · 2025-05-05 20:45
@KBlueleaf @osbenet Good news for tinybox green v2 buyers, it's full speed PCIe 5.0!

@__tinygrad__ · 2025-05-05 20:45
@KBlueleaf @osbenet Good news for tinybox green v2 buyers, it's full speed PCIe 5.0!

@KBlueleaf · 2025-05-05 20:47
@__tinygrad__ @osbenet UwUb should be enough for 4card FSDP on some LLM or t2i model

@KBlueleaf · 2025-05-05 20:47
@__tinygrad__ @osbenet UwUb should be enough for 4card FSDP on some LLM or t2i model

@__tinygrad__ · 2025-05-05 20:49
@KBlueleaf @osbenet What's UwUb?

@__tinygrad__ · 2025-05-06 02:58
Thermals on a production tinybox v2. Saturation @ 75C! https://t.co/xVGNIbe5A8

@__tinygrad__ · 2025-05-06 03:15
You know you want one. https://t.co/bi10oOHWUT

@__tinygrad__ · 2025-05-06 03:15
You know you want one. https://t.co/bi10oOHWUT

@rmarcilhoo · 2025-05-06 03:25
@__tinygrad__ So much room left for more GPUs...

@__tinygrad__ · 2025-05-06 03:15
You know you want one. https://t.co/bi10oOHWUT

@dmsimon · 2025-05-06 03:28
@__tinygrad__ What's the power draw?

@rmarcilhoo · 2025-05-06 03:25
@__tinygrad__ So much room left for more GPUs...

@__tinygrad__ · 2025-05-06 03:37
@rmarcilhoo And so little power...

@dmsimon · 2025-05-06 03:28
@__tinygrad__ What's the power draw?

@__tinygrad__ · 2025-05-06 03:37
@dmsimon It has 2x1600W PSUs, so ~that.

@__tinygrad__ · 2025-05-07 16:59
Our whole codebase fits in the context window of modern LLMs!

@__tinygrad__ · 2025-05-07 17:52
Fun fact, there's no quiet SP5 cooling solution that we could find. So we buy the best heatsink and put a Noctua fan on it. tinybox is silent at idle and "office quiet" at full load. We are around these things all day, so it's super important we do the thermals well. https://t.co/5gPD6aZwo8

@__tinygrad__ · 2025-05-07 17:56
There's 4 tinybox green v2 left in the current batch, shipping by end of May. Order today if you want one, who knows when the GPU gods will smile on us again.

@__tinygrad__ · 2025-05-07 17:52
Fun fact, there's no quiet SP5 cooling solution that we could find. So we buy the best heatsink and put a Noctua fan on it. tinybox is silent at idle and "office quiet" at full load. We are around these things all day, so it's super important we do the thermals well. https://t.co/5gPD6aZwo8

@rinezig · 2025-05-07 17:59
@__tinygrad__ Is the case custom designed in house?

@__tinygrad__ · 2025-05-07 17:52
Fun fact, there's no quiet SP5 cooling solution that we could find. So we buy the best heatsink and put a Noctua fan on it. tinybox is silent at idle and "office quiet" at full load. We are around these things all day, so it's super important we do the thermals well. https://t.co/5gPD6aZwo8

@_LTJorge · 2025-05-07 18:01
@__tinygrad__ Are you sure that's enough? Don't get me wrong, I'm using like 9 Noctuas and 5 Phanteks, but server hardware is a different beast (up to 500w TDP) and normally expects front side Delta screamers (10-20k rpm).

@__tinygrad__ · 2025-05-07 18:02
RT @SharingPsyche: My agi test for llms is giving them the entire @__tinygrad__ codebase and a hugging face repo and seeing if it can implement w/o my help

@_LTJorge · 2025-05-07 18:01
@__tinygrad__ Are you sure that's enough? Don't get me wrong, I'm using like 9 Noctuas and 5 Phanteks, but server hardware is a different beast (up to 500w TDP) and normally expects front side Delta screamers (10-20k rpm).

@__tinygrad__ · 2025-05-07 18:04
@_LTJorge Yes. 210W TDP on the chip, tested at full CPU load with GPUs at full load also. CPU at 78C with fan only at 70%.

@__tinygrad__ · 2025-05-07 18:05
@ZekeGeiss AFAIK big static pressure = big noise. We instead opt for big case.

@rinezig · 2025-05-07 17:59
@__tinygrad__ Is the case custom designed in house?

@__tinygrad__ · 2025-05-07 18:07
@ebianchi99 Yes

@__tinygrad__ · 2025-05-07 17:56
There's 4 tinybox green v2 left in the current batch, shipping by end of May. Order today if you want one, who knows when the GPU gods will smile on us again.

@nate_zec · 2025-05-07 18:15
@__tinygrad__ How much hackery does it take for a non-llm linux geek to get a prompt that can chat, has relatively good ambient knowledge, leaks no context window/chat history to the internet, & can make web requests when it deems it necessary to update chat context? Is that a product goal?

@nate_zec · 2025-05-07 18:15
@__tinygrad__ How much hackery does it take for a non-llm linux geek to get a prompt that can chat, has relatively good ambient knowledge, leaks no context window/chat history to the internet, & can make web requests when it deems it necessary to update chat context? Is that a product goal?

@__tinygrad__ · 2025-05-07 18:16
@nate_zec We sell computer. It is good computer.

@__tinygrad__ · 2025-05-07 17:52
Fun fact, there's no quiet SP5 cooling solution that we could find. So we buy the best heatsink and put a Noctua fan on it. tinybox is silent at idle and "office quiet" at full load. We are around these things all day, so it's super important we do the thermals well. https://t.co/5gPD6aZwo8

@darwesh_singh · 2025-05-07 18:22
Outside of AIOs or custom loops, SP5 is super hard to find. Not sure why there is an absence of SP3 and SP6 heatsinks. We initially built our own EK loop which died last month and replaced with Alphacool AIO (replaced the included fans with Noctua). Otherwise Dynatron + Noctua is the way to go.

@__tinygrad__ · 2025-05-07 18:50
Try YOLO in your browser! tinygrad is probably the easiest framework to do this sort of stuff with. https://t.co/zquNxjqN4h

@__tinygrad__ · 2025-05-07 18:50
Try YOLO in your browser! tinygrad is probably the easiest framework to do this sort of stuff with. https://t.co/zquNxjqN4h

@__tinygrad__ · 2025-05-07 18:50
Link: https://t.co/ss6ncyCPlR

@realtime3392 · 2025-05-07 19:34
@__tinygrad__ Any idea if tinygrad will run on NVIDIA Drive hardware? The configuration I have for testing is built around an NVIDIA Xavier SoM with 64 GB of unified memory, shared between the ARM cores and the GPU (512-core Volta with 64 Tensor cores). There are 2 SXM2 sockets, and I have V100 (5120 cores, 32 GB, 640 Tensor cores) or TU104 (3072 cores, 16 GB, 384 Tensor cores) dGPUs I could install in those slots. This is basically the just-previous NVIDIA Drive Pegasus setup.

@__tinygrad__ · 2025-05-07 20:15
To date, no. Once this is possible, if that AI is running in tinygrad, we could have the first autonomously recursively self improving system.

@darwesh_singh · 2025-05-07 18:22
Outside of AIOs or custom loops, SP5 is super hard to find. Not sure why there is an absence of SP3 and SP6 heatsinks. We initially built our own EK loop which died last month and replaced with Alphacool AIO (replaced the included fans with Noctua). Otherwise Dynatron + Noctua is the way to go.

@__tinygrad__ · 2025-05-07 21:38
@darwesh_singh We've had reliability issues with AIOs. They seem to degrade over time, haven't looked enough to see if it's a brand thing or not. But this is why all tinyboxes are air cooled, I think getting water involved only makes sense if you need density.

@__tinygrad__ · 2025-05-07 18:50
Try YOLO in your browser! tinygrad is probably the easiest framework to do this sort of stuff with. https://t.co/zquNxjqN4h

@LucaMiglioli185 · 2025-05-08 07:43
@__tinygrad__ do you guys support phone gpus/npus using android? Openpilot runs on tinygrad but i don't think it goes thru android right?

@realtime3392 · 2025-05-07 19:34
@__tinygrad__ Any idea if tinygrad will run on NVIDIA Drive hardware? The configuration I have for testing is built around an NVIDIA Xavier SoM with 64 GB of unified memory, shared between the ARM cores and the GPU (512-core Volta with 64 Tensor cores). There are 2 SXM2 sockets, and I have V100 (5120 cores, 32 GB, 640 Tensor cores) or TU104 (3072 cores, 16 GB, 384 Tensor cores) dGPUs I could install in those slots. This is basically the just-previous NVIDIA Drive Pegasus setup.

@__tinygrad__ · 2025-05-08 19:09
@realtime3392 It should, it's the same CUDA API. We're open to a contract if you want a specific model to be fast.

@LucaMiglioli185 · 2025-05-08 07:43
@__tinygrad__ do you guys support phone gpus/npus using android? Openpilot runs on tinygrad but i don't think it goes thru android right?

@__tinygrad__ · 2025-05-08 19:11
@LucaMiglioli185 Works with termux, GPUs should be quite fast, it's the same as openpilot uses.

@__tinygrad__ · 2025-05-09 00:37
All products in stock! We have: 3 tinybox red for $15k 2 tinybox green (refurbished) for $25k 3 tinybox green v2 (ships in 3-5 weeks) for $29k

@__tinygrad__ · 2025-05-09 00:37
All products in stock! We have: 3 tinybox red for $15k 2 tinybox green (refurbished) for $25k 3 tinybox green v2 (ships in 3-5 weeks) for $29k

@__tinygrad__ · 2025-05-09 00:37
Shop link: https://t.co/fQFBykuRKQ

@__tinygrad__ · 2025-05-09 00:37
All products in stock! We have: 3 tinybox red for $15k 2 tinybox green (refurbished) for $25k 3 tinybox green v2 (ships in 3-5 weeks) for $29k

@thoughtlesslabs · 2025-05-09 00:38
@__tinygrad__ ill give you tree fiddy. deal?

@thoughtlesslabs · 2025-05-09 00:38
@__tinygrad__ ill give you tree fiddy. deal?

@__tinygrad__ · 2025-05-09 00:38
@thoughtlesslabs $350k!?! You can buy an MI300X box for that.

@__tinygrad__ · 2025-05-09 00:37
All products in stock! We have: 3 tinybox red for $15k 2 tinybox green (refurbished) for $25k 3 tinybox green v2 (ships in 3-5 weeks) for $29k

@complex_maths · 2025-05-09 00:42
@__tinygrad__ What are the specs of the refurbished tinybox? I assume 4x4090s or something analagous given the v2 has 4x5090s.

@complex_maths · 2025-05-09 00:42
@__tinygrad__ What are the specs of the refurbished tinybox? I assume 4x4090s or something analagous given the v2 has 4x5090s.

@__tinygrad__ · 2025-05-09 00:43
@complex_maths 6x4090, full specs here https://t.co/7qyLVXbV9C

@__tinygrad__ · 2025-05-09 00:37
All products in stock! We have: 3 tinybox red for $15k 2 tinybox green (refurbished) for $25k 3 tinybox green v2 (ships in 3-5 weeks) for $29k

@jsuarez · 2025-05-09 01:19
@__tinygrad__ You don't have any more 6x4090 boxes right? Would be substantially faster than 4x5090 for RL

@jsuarez · 2025-05-09 01:19
@__tinygrad__ You don't have any more 6x4090 boxes right? Would be substantially faster than 4x5090 for RL

@__tinygrad__ · 2025-05-09 02:40
@jsuarez That's the tinybox green v1, 2 in stock!

@mov_axbx · 2025-05-09 08:16
I think @tinycorp should take a hypervisor, add a stripped down monolithic kernel supporting only the virtual hardware and GPU passthrough, add some system extensions to a lightweight, tightly integrated Python runtime, and let it eat.

@__tinygrad__ · 2025-05-09 02:40
@jsuarez That's the tinybox green v1, 2 in stock!

@jsuarez · 2025-05-09 12:09
@__tinygrad__ Just put in order. I may buy both. Can you please confirm the CPU and RAM on v1?

@adityaag · 2025-05-09 13:31
In unsurprising news -- computer science majors are finding it impossible to get software engineering internships Doesn't matter if you are at Stanford -- an AI coding agent is better at being an intern Totally tracks -- if I were a senior dev, I'd prefer an agent to a 20yr old

@jsuarez · 2025-05-09 12:09
@__tinygrad__ Just put in order. I may buy both. Can you please confirm the CPU and RAM on v1?

@__tinygrad__ · 2025-05-09 14:06
@jsuarez 7532/128GB https://t.co/ugzBdOyGcM

@__tinygrad__ · 2025-05-09 14:06
@jsuarez 7532/128GB https://t.co/ugzBdOyGcM

@jsuarez · 2025-05-09 14:40
@__tinygrad__ Both paid. Tinygrad + pufferlib = pea puffer? https://t.co/yr5E0eQ8jb

@jsuarez · 2025-05-09 14:40
@__tinygrad__ Both paid. Tinygrad + pufferlib = pea puffer? https://t.co/yr5E0eQ8jb

@__tinygrad__ · 2025-05-09 16:35
@jsuarez Money received, we'll have them shipped out in 1-2 weeks! tinybox green v1 now out of stock

@__tinygrad__ · 2025-05-09 17:22
tinybox green v2 under construction. Still 3 left in this batch for sale. The 2 refurb v1 sold out right away. https://t.co/EIipbeui47

@__tinygrad__ · 2025-05-09 17:22
tinybox green v2 under construction. Still 3 left in this batch for sale. The 2 refurb v1 sold out right away. https://t.co/EIipbeui47

@samuraimckeon · 2025-05-09 17:32
@__tinygrad__ curious, why wouldn't you just use s 1500 or 2k watt server power supply?

@samuraimckeon · 2025-05-09 17:32
@__tinygrad__ curious, why wouldn't you just use s 1500 or 2k watt server power supply?

@__tinygrad__ · 2025-05-09 17:51
@samuraimckeon cause they loud. this quiet.

@__tinygrad__ · 2025-05-09 21:52
Here's the worlds first AMD GPU driven over USB3. From a Mac! Linux and Windows should work too, it's just libusb. Available today in tinygrad master, use an ADT-UT3G to connect the GPU to your USB port. You have no idea of the level of engineering that went into this. https://t.co/V6trNwcGXt

@__tinygrad__ · 2025-05-09 21:57
@KevinMclaw Oh yes. They sent us the two MI300X boxes and with the @SemiAnalysis_ stuff have shown a new appreciation of software quality. We're here to get everyone else on the @AMD train too. Buy a tinybox red today!

@__tinygrad__ · 2025-05-09 21:52
Here's the worlds first AMD GPU driven over USB3. From a Mac! Linux and Windows should work too, it's just libusb. Available today in tinygrad master, use an ADT-UT3G to connect the GPU to your USB port. You have no idea of the level of engineering that went into this. https://t.co/V6trNwcGXt

@pkuhar · 2025-05-09 22:03
@__tinygrad__ Will it work with nvidia too?

@pkuhar · 2025-05-09 22:03
@__tinygrad__ Will it work with nvidia too?

@__tinygrad__ · 2025-05-09 22:06
@pkuhar nope. only AMD. it requires the full driver built in to tinygrad. should work with any RDNA3 or RDNA4 GPU.

@__tinygrad__ · 2025-05-09 21:52
Here's the worlds first AMD GPU driven over USB3. From a Mac! Linux and Windows should work too, it's just libusb. Available today in tinygrad master, use an ADT-UT3G to connect the GPU to your USB port. You have no idea of the level of engineering that went into this. https://t.co/V6trNwcGXt

@growthesque · 2025-05-09 22:08
@__tinygrad__ Amazing. Curious about % performance degradation vs PCIe?

@growthesque · 2025-05-09 22:08
@__tinygrad__ Amazing. Curious about % performance degradation vs PCIe?

@__tinygrad__ · 2025-05-09 22:11
@growthesque We're still working on improving it, but when finished it should be able to get the full 10 gbps of USB3. So slower, but not too bad. And that only affects copyin/copyout, kernels are the same speed.

@__tinygrad__ · 2025-05-09 21:52
Here's the worlds first AMD GPU driven over USB3. From a Mac! Linux and Windows should work too, it's just libusb. Available today in tinygrad master, use an ADT-UT3G to connect the GPU to your USB port. You have no idea of the level of engineering that went into this. https://t.co/V6trNwcGXt

@mikestaub · 2025-05-09 22:16
@__tinygrad__ I told you not to count them out! This is incredible engineering.

@mikestaub · 2025-05-09 22:16
@__tinygrad__ I told you not to count them out! This is incredible engineering.

@__tinygrad__ · 2025-05-09 22:18
@mikestaub Haha, I mean...it's not AMD's engineering, it's ours. We completely rewrote the driver, and since our driver is simple we can tunnel it over USB. RDNA4/9070XT is nice, particularly the tensor cores. Still wish they made a big one though.

@__tinygrad__ · 2025-05-09 21:52
Here's the worlds first AMD GPU driven over USB3. From a Mac! Linux and Windows should work too, it's just libusb. Available today in tinygrad master, use an ADT-UT3G to connect the GPU to your USB port. You have no idea of the level of engineering that went into this. https://t.co/V6trNwcGXt

@AtakanTekparmak · 2025-05-09 22:19
@__tinygrad__ Wait, we can connect external AMD GPUs to an M-series mac/macbook and use it in tinygrad?

@AtakanTekparmak · 2025-05-09 22:19
@__tinygrad__ Wait, we can connect external AMD GPUs to an M-series mac/macbook and use it in tinygrad?

@__tinygrad__ · 2025-05-09 22:21
@AtakanTekparmak Yes! Will still be a few weeks to fully polish, but that's on an M3 Max. And your eGPU needs to have an ASM2464PD based controller.

@__tinygrad__ · 2025-05-09 22:25
RT @jsuarez: That's me! Bought both v1's for OSS RL in PufferLib. Come build cool stuff with us and you will be able to train on them!

@__tinygrad__ · 2025-05-09 22:25
@jsuarez As long as your envs are on the GPU they should make great RL machines!

@jsuarez · 2025-05-09 22:31
@__tinygrad__ They're not but they run >1m steps/second/core. Pure C with static memory. Neural MMO 3 should train at 3M steps/second!

@jsuarez · 2025-05-09 22:31
@__tinygrad__ They're not but they run >1m steps/second/core. Pure C with static memory. Neural MMO 3 should train at 3M steps/second!

@__tinygrad__ · 2025-05-09 22:38
@jsuarez Oh yea if you write performant CPU environments that works too. Tons of off the shelf RL stuff is stupidly single thread CPU bound, but I'm sure you know this.

@__tinygrad__ · 2025-05-09 22:11
@growthesque We're still working on improving it, but when finished it should be able to get the full 10 gbps of USB3. So slower, but not too bad. And that only affects copyin/copyout, kernels are the same speed.

@Greengree4 · 2025-05-09 22:48
@__tinygrad__ @growthesque Could we use thunderbolt or USB4 for any faster speeds or is it USB3 dependent for now?

@Greengree4 · 2025-05-09 22:48
@__tinygrad__ @growthesque Could we use thunderbolt or USB4 for any faster speeds or is it USB3 dependent for now?

@__tinygrad__ · 2025-05-09 22:49
@Greengree4 @growthesque Can you access USB4 from user space on OS X? The main focus of this project was for @comma_ai to be able to plug a big GPU into the comma 3X, so we focused on USB3. But USB4 should be even easier if you can find a user space API. Not messing with kernel drivers.

@__tinygrad__ · 2025-05-09 22:52
tinygrad is still hiring interns, this summer in San Diego if you are quick. Solve bounties to prove you have what it takes! You are even welcome to use AI to help you. However, if you have less skill than AI and don't understand the AI outputs, you don't meet the bar here.

@Stone_Tao · 2025-05-09 23:02
should my lab / should I buy this lmao. Seeing friends buy this now

@Stone_Tao · 2025-05-09 23:02
should my lab / should I buy this lmao. Seeing friends buy this now

@__tinygrad__ · 2025-05-09 23:02
@Stone_Tao You can even drive down to pick it up to save on shipping!

@__tinygrad__ · 2025-05-09 23:02
@Stone_Tao You can even drive down to pick it up to save on shipping!

@Stone_Tao · 2025-05-09 23:05
@__tinygrad__ is there any option with less 4090s? 8x is a bit overkill for me

@Stone_Tao · 2025-05-09 23:05
@__tinygrad__ is there any option with less 4090s? 8x is a bit overkill for me

@__tinygrad__ · 2025-05-09 23:06
@Stone_Tao Err, that one is out of stock anyway. All our offerings are on https://t.co/FUEknRja8z

@tkanarsky · 2025-05-09 23:16
@__tinygrad__ Like, I can conceptually understand how the stack works, but not concretely.

@tkanarsky · 2025-05-09 23:18
@__tinygrad__ Let's say I had a concrete goal in mind and 4 weeks to get this ported to using thunderbolt as a transport. What do I need to know?

@__tinygrad__ · 2025-05-09 21:52
Here's the worlds first AMD GPU driven over USB3. From a Mac! Linux and Windows should work too, it's just libusb. Available today in tinygrad master, use an ADT-UT3G to connect the GPU to your USB port. You have no idea of the level of engineering that went into this. https://t.co/V6trNwcGXt

@AseriOCE · 2025-05-09 23:48
@__tinygrad__ Maybe a long shot, but could this work with a RDNA 2 GPU like the 6750 XT?

@AseriOCE · 2025-05-09 23:48
@__tinygrad__ Maybe a long shot, but could this work with a RDNA 2 GPU like the 6750 XT?

@__tinygrad__ · 2025-05-09 23:52
@AseriFPS RDNA3 and RDNA4 now, but shouldn't be too hard to add, I estimate 20 lines. Our runtime already supports RDNA2.

@mov_axbx · 2025-05-09 08:16
I think @tinycorp should take a hypervisor, add a stripped down monolithic kernel supporting only the virtual hardware and GPU passthrough, add some system extensions to a lightweight, tightly integrated Python runtime, and let it eat.

@__tinygrad__ · 2025-05-10 00:19
@mov_axbx @tinycorp This is pretty much what the machines in our CLOUD=1 will be. 0 state, PXE booted each time. Though it's just minimal Linux, no reason to not have a Linux.

@tkanarsky · 2025-05-09 23:18
@__tinygrad__ Let's say I had a concrete goal in mind and 4 weeks to get this ported to using thunderbolt as a transport. What do I need to know?

@__tinygrad__ · 2025-05-10 03:36
@tkanarsky Thunderbolt is a ton easier because you can actually map an address range. All the code is in tinygrad, I'd first replicate our setup then start diving in to the tinygrad code to see how it's done. The AMD driver supports three backends, amdgpu, raw PCIe, and USB.

@__tinygrad__ · 2025-05-10 03:36
@tkanarsky Thunderbolt is a ton easier because you can actually map an address range. All the code is in tinygrad, I'd first replicate our setup then start diving in to the tinygrad code to see how it's done. The AMD driver supports three backends, amdgpu, raw PCIe, and USB.

@tkanarsky · 2025-05-10 03:47
@__tinygrad__ Nice, thanks. What's the difference between amdgpu and pcie?

@tkanarsky · 2025-05-10 03:47
@__tinygrad__ Nice, thanks. What's the difference between amdgpu and pcie?

@__tinygrad__ · 2025-05-10 03:47
@tkanarsky amdgpu is the kernel driver. pcie is mmap pcie from user space.

@__tinygrad__ · 2025-05-10 05:21
In theory, 9070XT should be able to get 195 TFLOPS of FP16 matmul with the tensor cores, right? tinygrad gets 97 TFLOPS, torch nightly (2.8.0.dev20250502+rocm6.4) gets 16 TFLOPS. What's the highest anyone has seen this card get?

@__tinygrad__ · 2025-05-10 05:21
In theory, 9070XT should be able to get 195 TFLOPS of FP16 matmul with the tensor cores, right? tinygrad gets 97 TFLOPS, torch nightly (2.8.0.dev20250502+rocm6.4) gets 16 TFLOPS. What's the highest anyone has seen this card get?

@__tinygrad__ · 2025-05-10 05:27
ugh also, @AMD, don't do this. comgr is now binary incompatible between ROCm 6.3 and 6.4. Now at 110 TFLOPS with 8192x8192 matrix. https://t.co/gSnPU3oQ32

@__tinygrad__ · 2025-05-10 05:27
ugh also, @AMD, don't do this. comgr is now binary incompatible between ROCm 6.3 and 6.4. Now at 110 TFLOPS with 8192x8192 matrix. https://t.co/gSnPU3oQ32

@__tinygrad__ · 2025-05-10 05:30
112 on 6144x6144. I'm almost convinced the tensor cores are real and this isn't just overclocking the 97 shader TFLOPS. https://t.co/S8ISqM86Bz

@tenderizzation · 2025-05-10 05:42
did you remember to benchmark with `PYTORCH_TUNABLEOP_ENABLED=1` ? https://t.co/wVZ7E3SrHe

@__tinygrad__ · 2025-05-10 06:02
@PytorchToAtoms @tenderizzation I'm actually surprised by how close they are. In general, vendor BLAS libraries are beating tinygrad at simple GEMM by around 20%. It's not where we have focused yet.

@__tinygrad__ · 2025-05-10 19:30
107 TFLOPS on Mac M3 with a 9070XT attached. Our GEMM with the AMD_LLVM backend is now beating hipBLASLt on the card. And I love how portable LLVM is, this is just brew install llvm@19. https://t.co/PkHnG6ILrY

@__tinygrad__ · 2025-05-10 19:30
107 TFLOPS on Mac M3 with a 9070XT attached. Our GEMM with the AMD_LLVM backend is now beating hipBLASLt on the card. And I love how portable LLVM is, this is just brew install llvm@19. https://t.co/PkHnG6ILrY

@elyxlz · 2025-05-10 19:39
@__tinygrad__ would this work on my asahi linux?

@elyxlz · 2025-05-10 19:39
@__tinygrad__ would this work on my asahi linux?

@__tinygrad__ · 2025-05-10 19:43
@elyxlz How can you access the GPU compute? If it has something like OpenCL, it'll work. Oh, and if you mean over the USB port, that will definitely work, that's totally portable!

@__tinygrad__ · 2025-05-10 19:30
107 TFLOPS on Mac M3 with a 9070XT attached. Our GEMM with the AMD_LLVM backend is now beating hipBLASLt on the card. And I love how portable LLVM is, this is just brew install llvm@19. https://t.co/PkHnG6ILrY

@LenSeaside · 2025-05-10 19:48
@__tinygrad__ This is probably a dumb question but can it only run your models? You can't yank a model from huggingface face and run inference on this? It's very cool by the way.

@__tinygrad__ · 2025-05-10 19:33
One of the best things about tinygrad is any code written runs on any device (commoditize the petaflop, get it?). Just with a few env var changes, this is targeting the internal GPU on the M3 Max. Only 13.5 TFLOPS. https://t.co/HdRLPBEWEL

@__tinygrad__ · 2025-05-10 19:48
Part of commoditizing the petaflop is commoditizing the gigaflop. Here it is on a comma device, tinygrad OpenCL should work on Android phones with termux with pretty decent perf. Was too lazy to wait for BEAM on this one. https://t.co/4olIBslhrp

@__tinygrad__ · 2025-05-10 19:48
Part of commoditizing the petaflop is commoditizing the gigaflop. Here it is on a comma device, tinygrad OpenCL should work on Android phones with termux with pretty decent perf. Was too lazy to wait for BEAM on this one. https://t.co/4olIBslhrp

@__tinygrad__ · 2025-05-10 19:51
Also, that QCOM backend bypasses the GPU driver with direct command buffers and ioctls, tested on Snapdragon 845 but it shouldn't be too much more effort to do the rest. This enables CUDA Graph like functionality on Qualcomm to avoid models being CPU dispatch bound.

@LenSeaside · 2025-05-10 19:48
@__tinygrad__ This is probably a dumb question but can it only run your models? You can't yank a model from huggingface face and run inference on this? It's very cool by the way.

@__tinygrad__ · 2025-05-10 19:55
@LenSeaside What format is it on huggingface? We have an ONNX frontend that's on par with onnxruntime compliance wise. If it's just the weights, you'll have to either write the model in tinygrad or you can try the experimental tinygrad torch backend.

@therealkmans · 2025-05-11 22:30
Who's excited for @__tinygrad__ on @LeetGPU 🚀🚀 https://t.co/lHZy7TpHpR

@LeetGPU · 2025-05-11 22:57
We just rolled out @__tinygrad__ support for all challenges 🔥 Try it out :) If you run into any issues, let us know in our Discord!

@__tinygrad__ · 2025-05-12 01:49
@LeetGPU I clicked run with no changes and it passed...😕 https://t.co/nI7G46pkPC

@__tinygrad__ · 2025-05-13 00:08
RT @LeetGPU: We just rolled out @__tinygrad__ support for all challenges 🔥 Try it out :) If you run into any issues, let us know in our Discord!

@__tinygrad__ · 2025-05-13 02:02
tiny cloud is less tiny now https://t.co/DeehcvlCuV

@__tinygrad__ · 2025-05-13 02:02
tiny cloud is less tiny now https://t.co/DeehcvlCuV

@yihyunCS · 2025-05-13 02:05
@__tinygrad__ whats the total draw at full load?

@yihyunCS · 2025-05-13 02:05
@__tinygrad__ whats the total draw at full load?

@__tinygrad__ · 2025-05-13 02:08
@yihyunCS That's 13*3 = 39 kW of tinybox. We have 3x 208V 50A three phase circuits, so 3*208*50*sqrt(3)/1000 = 54 kW of power.

@__tinygrad__ · 2025-05-13 02:08
@yihyunCS That's 13*3 = 39 kW of tinybox. We have 3x 208V 50A three phase circuits, so 3*208*50*sqrt(3)/1000 = 54 kW of power.

@miolini · 2025-05-13 02:23
@__tinygrad__ @yihyunCS You forgot to include the power factor in your formula.

@rpoo · 2025-05-13 03:06
If you’re interested in building terawatt scale compute join xAI

@__tinygrad__ · 2025-05-13 02:02
tiny cloud is less tiny now https://t.co/DeehcvlCuV

@HansCNelson · 2025-05-13 03:40
@__tinygrad__ So is that about a half a person's worth of flops?

@HansCNelson · 2025-05-13 03:40
@__tinygrad__ So is that about a half a person's worth of flops?

@__tinygrad__ · 2025-05-13 03:53
@HansCNelson Exactly. It's got half a brain.

@__tinygrad__ · 2025-05-13 03:54
@bricegilden Quiet * 13 = Still quiet. Easy to have a conversation. A little toasty though.

@miolini · 2025-05-13 02:23
@__tinygrad__ @yihyunCS You forgot to include the power factor in your formula.

@__tinygrad__ · 2025-05-13 04:00
@miolini @yihyunCS Yea, that's why we are only running 13 boxes + a space for provisioning. We try to keep things to 80% of max in general, there's power factor and phase balance losses.

@__tinygrad__ · 2025-05-13 16:36
MNIST trained with an AMD USB GPU from a Mac! On tinygrad master. Buy an ADT-UT3G + an RDNA3 or RDNA4 AMD GPU to play along at home. Interface speed rapidly improving. https://t.co/GEJvR5L7xt

@__tinygrad__ · 2025-05-13 16:36
MNIST trained with an AMD USB GPU from a Mac! On tinygrad master. Buy an ADT-UT3G + an RDNA3 or RDNA4 AMD GPU to play along at home. Interface speed rapidly improving. https://t.co/GEJvR5L7xt

@_LTJorge · 2025-05-13 16:39
@__tinygrad__ This runs on Linux as-is? I have a 7900XTX and all ROCm packages installed on Gentoo.

@_LTJorge · 2025-05-13 16:39
@__tinygrad__ This runs on Linux as-is? I have a 7900XTX and all ROCm packages installed on Gentoo.

@__tinygrad__ · 2025-05-13 16:41
@_LTJorge Yes.

@__tinygrad__ · 2025-05-13 16:36
MNIST trained with an AMD USB GPU from a Mac! On tinygrad master. Buy an ADT-UT3G + an RDNA3 or RDNA4 AMD GPU to play along at home. Interface speed rapidly improving. https://t.co/GEJvR5L7xt

@FoundTheCode · 2025-05-13 16:42
@__tinygrad__ what genuine kind of black magic is this? plz explain how do you actually get ROCm working on MacOS?

@FoundTheCode · 2025-05-13 16:42
@__tinygrad__ what genuine kind of black magic is this? plz explain how do you actually get ROCm working on MacOS?

@__tinygrad__ · 2025-05-13 16:43
@FoundTheCode lol don't use ROCm. tinygrad replaces all of AMD's software. The AMD_LLVM=1 flag just uses stock LLVM, installed with `brew install llvm@19`

@__tinygrad__ · 2025-05-13 16:43
@FoundTheCode lol don't use ROCm. tinygrad replaces all of AMD's software. The AMD_LLVM=1 flag just uses stock LLVM, installed with `brew install llvm@19`

@mali8ooyah · 2025-05-13 16:51
@__tinygrad__ @FoundTheCode AMD needs to make an offer on tinygrad at this point but we know Hotz can’t be contained

@__tinygrad__ · 2025-05-13 16:36
MNIST trained with an AMD USB GPU from a Mac! On tinygrad master. Buy an ADT-UT3G + an RDNA3 or RDNA4 AMD GPU to play along at home. Interface speed rapidly improving. https://t.co/GEJvR5L7xt

@LostAngelNZ · 2025-05-13 16:57
@__tinygrad__ Will the USB to PCIe adaptors you got from the taobao seller you meet up with in Shenzhen work? Or does it need to the ADT-UT3G with the current workflow.

@__tinygrad__ · 2025-05-13 17:07
We got the MI300X on MLPerf. We aren't looking to sell the company, but we're open to a $2M contract to have it beat the H100 time in the next round. We still encourage AMD to open source their firmware.

@__tinygrad__ · 2025-05-13 17:07
We got the MI300X on MLPerf. We aren't looking to sell the company, but we're open to a $2M contract to have it beat the H100 time in the next round. We still encourage AMD to open source their firmware.

@dhtikna · 2025-05-13 17:16
@__tinygrad__ so you get paid nothing if you dont beat it?

@LostAngelNZ · 2025-05-13 16:57
@__tinygrad__ Will the USB to PCIe adaptors you got from the taobao seller you meet up with in Shenzhen work? Or does it need to the ADT-UT3G with the current workflow.

@__tinygrad__ · 2025-05-13 17:16
@LostAngelNZ Those ones work fine too. ADT-UT3G is just available on Amazon.

@dhtikna · 2025-05-13 17:16
@__tinygrad__ so you get paid nothing if you dont beat it?

@__tinygrad__ · 2025-05-13 17:17
@dhtikna I'd be okay with structuring it like that. Challenges make things fun. We did a similar contract for the Qualcomm DSP.

@__tinygrad__ · 2025-05-13 17:07
We got the MI300X on MLPerf. We aren't looking to sell the company, but we're open to a $2M contract to have it beat the H100 time in the next round. We still encourage AMD to open source their firmware.

@HotAisle · 2025-05-13 17:30
@__tinygrad__ Wow, a 2x increase in price. Inflation is rough!

@HotAisle · 2025-05-13 17:30
@__tinygrad__ Wow, a 2x increase in price. Inflation is rough!

@__tinygrad__ · 2025-05-13 17:32
@HotAisle Price went up a while ago. We're open to negotiation between $1M-$2M though, depending on which model and what the state of the code deliverable should be. https://t.co/j6KnVTaFaE

@HotAisle · 2025-05-13 17:30
@__tinygrad__ Wow, a 2x increase in price. Inflation is rough!

@Elon_fanboy77 · 2025-05-13 18:25
@HotAisle @__tinygrad__ maybe he will also ask for a server with 8 mi355x soon

@Elon_fanboy77 · 2025-05-13 18:25
@HotAisle @__tinygrad__ maybe he will also ask for a server with 8 mi355x soon

@__tinygrad__ · 2025-05-13 18:36
@Elon_fanboy77 @HotAisle We're happy with our 2 MI300X boxes. We don't need to upgrade until either the arch changes a lot or we have maxed out perf on them. It's nice to have some big HBM GPUs for testing with.

@__tinygrad__ · 2025-05-13 20:08
tinybox v2 green in provisioning. Every tinybox trains a ResNet-50 before it leaves the factory. https://t.co/hTwkFpE6Zo

@__tinygrad__ · 2025-05-13 20:08
tinybox v2 green in provisioning. Every tinybox trains a ResNet-50 before it leaves the factory. https://t.co/hTwkFpE6Zo

@blitz_or_die · 2025-05-13 20:14
@__tinygrad__ Red only

@blitz_or_die · 2025-05-13 20:14
@__tinygrad__ Red only

@__tinygrad__ · 2025-05-13 20:16
@blitz_or_die There's three in stock ready to ship that would love to be taken in to good homes.

@__tinygrad__ · 2025-05-14 00:23
RT @comma_ai: openpilot master now supports an external GPU plugged into the comma 3X’s aux port thanks to @__tinygrad__! Now we just need some big models to use all that compute. https://t.co/GMjwi1Bx6W

@comma_ai · 2025-05-14 00:12
openpilot master now supports an external GPU plugged into the comma 3X’s aux port thanks to @__tinygrad__! Now we just need some big models to use all that compute. https://t.co/GMjwi1Bx6W

@__tinygrad__ · 2025-05-14 01:35
@comma_ai btw, with a 9070XT, this will give your openpilot car more compute than a HW4 Tesla. 9070XT is 389 int8 TOPS, Tesla HW4 is only 242 TOPS, even with both chips.

@__tinygrad__ · 2025-05-14 18:27
RT @elonmusk: A terawatt of compute or you’re not really trying!

@__tinygrad__ · 2025-05-15 00:37
An NVIDIA 5090 ($3000) has similar FP16 with FP32 accumulate FLOPS as an AMD 9070XT ($750). 209.5 vs 194.6 TFLOPS The 5090 could be made twice as fast with a simple OTA software update, but NVIDIA chooses to artificially limit it for market segmentation. AMD doesn't.

@__tinygrad__ · 2025-05-15 00:37
An NVIDIA 5090 ($3000) has similar FP16 with FP32 accumulate FLOPS as an AMD 9070XT ($750). 209.5 vs 194.6 TFLOPS The 5090 could be made twice as fast with a simple OTA software update, but NVIDIA chooses to artificially limit it for market segmentation. AMD doesn't.

@basement_agi · 2025-05-15 00:41
@__tinygrad__ Does it matter? Or is it so memory bottlenecked that the TFLOPS number is not important?

@__tinygrad__ · 2025-05-15 00:37
An NVIDIA 5090 ($3000) has similar FP16 with FP32 accumulate FLOPS as an AMD 9070XT ($750). 209.5 vs 194.6 TFLOPS The 5090 could be made twice as fast with a simple OTA software update, but NVIDIA chooses to artificially limit it for market segmentation. AMD doesn't.

@asdf1212085 · 2025-05-15 00:42
@__tinygrad__ Why aren't your reds selling like hotcakes? I see you're still developing the drivers, not sure how usable they currently are.

@basement_agi · 2025-05-15 00:41
@__tinygrad__ Does it matter? Or is it so memory bottlenecked that the TFLOPS number is not important?

@__tinygrad__ · 2025-05-15 00:43
@basement_agi The 5090 has a huge amount of memory and cache bandwidth. It very much matters. The RTX PRO 6000 Blackwell is the exact same chip without the nerf, benchmark it and see.

@asdf1212085 · 2025-05-15 00:42
@__tinygrad__ Why aren't your reds selling like hotcakes? I see you're still developing the drivers, not sure how usable they currently are.

@__tinygrad__ · 2025-05-15 00:44
@asdf1212085 If you use tinygrad, they are extremely usable.

@__tinygrad__ · 2025-05-15 00:37
An NVIDIA 5090 ($3000) has similar FP16 with FP32 accumulate FLOPS as an AMD 9070XT ($750). 209.5 vs 194.6 TFLOPS The 5090 could be made twice as fast with a simple OTA software update, but NVIDIA chooses to artificially limit it for market segmentation. AMD doesn't.

@samxpatterson · 2025-05-15 03:07
@__tinygrad__ Until then you can reduce abs error of fp16/16 by 10x for a 25% slowdown with this one weird trick: https://t.co/PnDhOnmLmX

@samxpatterson · 2025-05-15 03:07
@__tinygrad__ Until then you can reduce abs error of fp16/16 by 10x for a 25% slowdown with this one weird trick: https://t.co/PnDhOnmLmX

@__tinygrad__ · 2025-05-15 03:15
@samxpatterson Yes! fp16/fp16 is not nerfed, would be great to see this in tinygrad.

@__tinygrad__ · 2025-05-15 00:37
An NVIDIA 5090 ($3000) has similar FP16 with FP32 accumulate FLOPS as an AMD 9070XT ($750). 209.5 vs 194.6 TFLOPS The 5090 could be made twice as fast with a simple OTA software update, but NVIDIA chooses to artificially limit it for market segmentation. AMD doesn't.

@KBlueleaf · 2025-05-15 03:29
@__tinygrad__ 5090 209.5TFlops? How you get this number???? I can easily get over 400TFlops on my rtx5090 https://t.co/fSxZQgJuQ1

@KBlueleaf · 2025-05-15 03:29
@__tinygrad__ 5090 209.5TFlops? How you get this number???? I can easily get over 400TFlops on my rtx5090 https://t.co/fSxZQgJuQ1

@__tinygrad__ · 2025-05-15 03:36
@KBlueleaf Wait what?!? What benchmark is that? I'm going off the published number here (non sparsity obviously). https://t.co/ZNZ5XqObVm https://t.co/hZUqnKMfWy

@__tinygrad__ · 2025-05-15 03:36
@KBlueleaf Wait what?!? What benchmark is that? I'm going off the published number here (non sparsity obviously). https://t.co/ZNZ5XqObVm https://t.co/hZUqnKMfWy

@KBlueleaf · 2025-05-15 03:39
ok so, the number in the Spec is based on the "base clock" of "FE version 5090" And FE version 5090's cooling system is too weak to actually unlock all the power of 5090 (even the TDP is 575w) I have MSI gaming trio OC and I can easily obtain 400TFlops theoritical performance (with mmapeak) or over 260TFlops pratical performance (with pytorch @ operator) The core clock is around 3.3Ghz when running MMAPeak and 3.15GHz when running pytorch. Memory Speed is 32Gbps (not actually overclocking here, 28Gbps is super underestimated for those gddr7, the main bottleneck is actually IMC and gddr7 phy in 5090 chip, they need lot of power) https://t.co/qHshgLPWf5

@KBlueleaf · 2025-05-15 03:39
ok so, the number in the Spec is based on the "base clock" of "FE version 5090" And FE version 5090's cooling system is too weak to actually unlock all the power of 5090 (even the TDP is 575w) I have MSI gaming trio OC and I can easily obtain 400TFlops theoritical performance (with mmapeak) or over 260TFlops pratical performance (with pytorch @ operator) The core clock is around 3.3Ghz when running MMAPeak and 3.15GHz when running pytorch. Memory Speed is 32Gbps (not actually overclocking here, 28Gbps is super underestimated for those gddr7, the main bottleneck is actually IMC and gddr7 phy in 5090 chip, they need lot of power) https://t.co/qHshgLPWf5

@__tinygrad__ · 2025-05-15 03:42
@KBlueleaf What can you get fp16 with fp16 acc? If I'm reading it right, you are getting 811...then without the nerf that's what the card should be capable of!

@__tinygrad__ · 2025-05-15 03:42
@KBlueleaf What can you get fp16 with fp16 acc? If I'm reading it right, you are getting 811...then without the nerf that's what the card should be capable of!

@KBlueleaf · 2025-05-15 03:44
Yes, the nerf is annoying, but definitely not "200T", I mean, even 4090 is over 200T actually... (pytorch 150T ish) but for inference, fp16 mul fp16 acc is not bad (I have tried with some custom kernel on github) for training... considering we can even made fp8 work, maybe we should try fp16 acc as well.

@KBlueleaf · 2025-05-15 03:44
Yes, the nerf is annoying, but definitely not "200T", I mean, even 4090 is over 200T actually... (pytorch 150T ish) but for inference, fp16 mul fp16 acc is not bad (I have tried with some custom kernel on github) for training... considering we can even made fp8 work, maybe we should try fp16 acc as well.

@__tinygrad__ · 2025-05-15 03:48
@KBlueleaf We actually did our MLPerf BERT training run with FP16 acc on 4090. Here's what I see on my PNY 5090, thanks for the mmapeak link. I see 239 with torch, which I figured was the overclock, but awesome that it can dispatch for 373. https://t.co/9KVoO4sHu7

@__tinygrad__ · 2025-05-15 03:48
@KBlueleaf We actually did our MLPerf BERT training run with FP16 acc on 4090. Here's what I see on my PNY 5090, thanks for the mmapeak link. I see 239 with torch, which I figured was the overclock, but awesome that it can dispatch for 373. https://t.co/9KVoO4sHu7

@KBlueleaf · 2025-05-15 03:51
@__tinygrad__ torch matmul FLOPS is pratical flops you can get, since in real world you can't ignore the "reading weight from vram" step but it cannot reflect the speed of core, since you spend 40~60% time to wait the vram... just like what you may have seen on wandb logging

@KBlueleaf · 2025-05-15 03:51
@__tinygrad__ torch matmul FLOPS is pratical flops you can get, since in real world you can't ignore the "reading weight from vram" step but it cannot reflect the speed of core, since you spend 40~60% time to wait the vram... just like what you may have seen on wandb logging

@__tinygrad__ · 2025-05-15 03:54
@KBlueleaf Going to update the tinybox marketing number to 1492 TFLOPS.

@__tinygrad__ · 2025-05-15 00:37
An NVIDIA 5090 ($3000) has similar FP16 with FP32 accumulate FLOPS as an AMD 9070XT ($750). 209.5 vs 194.6 TFLOPS The 5090 could be made twice as fast with a simple OTA software update, but NVIDIA chooses to artificially limit it for market segmentation. AMD doesn't.

@__tinygrad__ · 2025-05-15 03:57
Update: @KBlueleaf pointed out that while NVIDIA advertises 209.5, the number to compare with the 194.6 TFLOPS from AMD is 373.0 as measured by mmapeak. They underadvertise! What's crazier, it would be 753.6 TFLOPS without the nerf.

@__tinygrad__ · 2025-05-15 04:33
New $200 bounty "MMAPEAK for AMD (all matrix instructions on 7900XTX/9070XT) using tinygrad runtime infra"

@__tinygrad__ · 2025-05-16 02:03
what's cooking? https://t.co/ncYzMVThSJ

@__tinygrad__ · 2025-05-16 02:24
@RexStructorum If I were making a 600W GPU, I'd use two of the CPU power connectors @ 300W each. Like what Moore Threads does, but two. https://t.co/mQEaQ23Z1K

@__tinygrad__ · 2025-05-17 04:09
https://t.co/YAJtMuhSVs will be the new home of tinygrad performance tracking. it's grafana, so it's a lot easier to add stuff too. and self hosted, no paying clouds $$$ https://t.co/JQJPtqx6ns

@francoisfleuret · 2025-05-17 10:53
If I was in charge. https://t.co/2UwNLiznQg

@ChShersh · 2025-05-17 18:24
The reason there’s no new company like Apple is because it’s no longer possible to assemble a PC in a garage

@__tinygrad__ · 2025-05-17 22:42
RT @ns123abc: George Hotz explains how to get hired @__tinygrad__ https://t.co/8j2tm4Q1Vv

@__tinygrad__ · 2025-05-17 22:43
@taminka_ It's like time and...time. Number go down is good. Think backslash \ not /

@__tinygrad__ · 2025-05-18 00:18
Thinking about hosting a tinygrad hackathon some weekend in June/July. In person in San Diego, at the comma office. Apply here. https://t.co/nMkTNgKrzh

@__tinygrad__ · 2025-05-18 03:51
RT @ant_vedaya: got my first merge at @__tinygrad__ https://t.co/24bbHmP5Xu

@__tinygrad__ · 2025-05-18 06:34
Here's our roadmap laying out how we will get training speed on par with torch by the end of the year. Upcasted Warps, Flash Attention, Locals Support, Multioutput Kernels, Fused Optimizer If you can help with any of these, join our Discord.

@__tinygrad__ · 2025-05-18 06:34
Here's our roadmap laying out how we will get training speed on par with torch by the end of the year. Upcasted Warps, Flash Attention, Locals Support, Multioutput Kernels, Fused Optimizer If you can help with any of these, join our Discord.

@__tinygrad__ · 2025-05-18 06:34
Link: https://t.co/Xj3RBcqHs6

@__tinygrad__ · 2025-05-18 06:34
Link: https://t.co/Xj3RBcqHs6

@voidisperfect · 2025-05-18 06:36
@__tinygrad__ access per request, link isnt for public view

@__tinygrad__ · 2025-05-18 06:34
Link: https://t.co/Xj3RBcqHs6

@__tinygrad__ · 2025-05-18 06:37
While we still have bounties, these projects look more like refactors which historically have been very hard to put bounties on. Starting with https://t.co/YAJtMuhSVs, we might need to find a new way to engage people in this new era of the project.

@voidisperfect · 2025-05-18 06:36
@__tinygrad__ access per request, link isnt for public view

@__tinygrad__ · 2025-05-18 06:37
@voidisperfect Fixed.

@francoisfleuret · 2025-05-17 10:53
If I was in charge. https://t.co/2UwNLiznQg

@__tinygrad__ · 2025-05-18 06:46
@francoisfleuret That would be a three line change to add to tinygrad.

@ChShersh · 2025-05-17 18:24
The reason there’s no new company like Apple is because it’s no longer possible to assemble a PC in a garage

@__tinygrad__ · 2025-05-18 07:02
@ChShersh Want to buy a tinybox?

@__tinygrad__ · 2025-05-18 17:18
tinygrad/gradient.py is one of the nicest files in the repo. These 40 lines express the gradients of every Tensor op, convs, gemms, attention, etc... https://t.co/Oj8JDqw2lO

@profleonn · 2025-05-18 17:43
my typa python code

@__tinygrad__ · 2025-05-18 17:18
tinygrad/gradient.py is one of the nicest files in the repo. These 40 lines express the gradients of every Tensor op, convs, gemms, attention, etc... https://t.co/Oj8JDqw2lO

@prakyath_k · 2025-05-18 17:46
@__tinygrad__ It's funny how most calculus is just 40 lines of code! Wow

@prakyath_k · 2025-05-18 17:46
@__tinygrad__ It's funny how most calculus is just 40 lines of code! Wow

@__tinygrad__ · 2025-05-18 18:13
@prakyath_k I mean, I guess you need the 15 line chain rule too https://t.co/lb4DbmBtb9

@onarchbtw · 2025-05-18 18:01
@proffleon this is the only point i disagree with geohot on. that’s retarded code. you gain nothing from it besides bad readability for other people trying to get used to the code base. they promoted having few LOC multiple times while having parts way worse then this in the codebase.

@profleonn · 2025-05-18 18:40
@onarchbtw but its just so juicy clean I love inline if statements, lambdas, list comprehension etc. in python, I just love it not because it reduces loc tho, just because it looks so clean and I lowkey prefer reading it more

@__tinygrad__ · 2025-05-18 19:33
I'm sick of all the software slop. Millions of lines, and 90% of them do not express any ideas. They simply exist to support the other millions of lines. Dead weight abstraction layers that are too costly to refactor away. Plumbing linking old systems to newer systems to the new system that everyone has to move to. But we can't really delete the old thing it still handles this legacy use case and that legacy code isn't tested nearly well enough to try and refactor it. tinygrad is trying to be different. 13067 lines of pure Python, including a torch-like frontend, a gradient library, a scheduler, a symbolic algebra library, an MLIR replacement, LLVM like rewrite passes, and support for 16 backends, including a full driver for AMD GPUs that mmaps PCIe BARs. Each line expresses an idea. There is no boilerplate. And if there is, the code base is small enough we can still refactor it away. 13k lines is something you can read in a weekend. Unlike with million line codebases, it's still possible to fit the entire thing in your head (or in an LLM context window). And because that's possible, it's still possible to refactor. If you are sick of all the slop of modern software, come work on tinygrad. We strive toward the platonic ideal of a Tensor library.

@__tinygrad__ · 2025-05-18 17:18
tinygrad/gradient.py is one of the nicest files in the repo. These 40 lines express the gradients of every Tensor op, convs, gemms, attention, etc... https://t.co/Oj8JDqw2lO

@fangpenlin · 2025-05-19 04:38
@__tinygrad__ Wow, pretty cool. This library looks pretty lean and indeed, very tiny. Love it! Will give it a try at some point 👍👍👍 https://t.co/rPWtl4CmPJ

@__tinygrad__ · 2025-05-19 05:07
tinygrad has 0 deps. If you want a backend, you'll need that backend installed (clang/llvm/cuda/etc...), and if you want to import/export numpy, you'll need numpy, but it really is pure Python 0 deps.

@__tinygrad__ · 2025-05-18 17:18
tinygrad/gradient.py is one of the nicest files in the repo. These 40 lines express the gradients of every Tensor op, convs, gemms, attention, etc... https://t.co/Oj8JDqw2lO

@KalenuikMatthew · 2025-05-19 05:28
@__tinygrad__ Loving the gradient definitions.. quick idea adding a built-in x.log1p() could improve numerical stability and shave off ~7% of training time on small-residual networks.

@KalenuikMatthew · 2025-05-19 05:28
@__tinygrad__ Loving the gradient definitions.. quick idea adding a built-in x.log1p() could improve numerical stability and shave off ~7% of training time on small-residual networks.

@__tinygrad__ · 2025-05-19 06:07
@KalenuikMatthew Check the later fusion and rewrite rules, we might already be transforming it to that. Gradient happens very early. Open to PRs to improve speed and numerical stability.

@__tinygrad__ · 2025-05-19 15:59
Every week we have a meeting to discuss tinygrad. In our Discord, open to all. Starts in 1 minute!

@DragonStacker · 2025-05-19 21:00
When someone like George Hotz is sharing wisdom, it’s wise to question your own assumptions, instead of whatever this is. Complexity stands in the way of new features. Nearly every aspect of real software engineering deals with either reducing complexity or increasing execution speed. All those design patterns exist to reduce the mental load of the programmer, so they have enough comprehension left over to add that next feature quickly and reliably. Applies to both humans and AI.

@alifahrri2 · 2025-05-19 21:33
i hope more libraries values minimum dependencies, 0 deps if possible. dependencies is really a nightmare. like everytime i touch anything from onnxruntime, my body aches it's so painful. badly designed glue frameworks with tons of dependencies, truly hell

@__tinygrad__ · 2025-05-19 21:44
tinygrad is getting close to being 1.0 software. The user-facing APIs have remained stable for about a year, we have docs, we have solid testing, and the basic layout of the code is now pretty stable. I don't think we should wait for speed to surpass PyTorch for 1.0. We have a roadmap for speed, and nothing in it would require breaking API compatibility. I see a few other blockers though. The TinyJIT has a bunch of footguns that need to be cleaned up. It should be simple to reason about the behavior of the JIT, particularly in cases where you are using or updating an external Tensor that's not explicitly in the inputs/outputs. The environment variables like FUSE_ARANGE all need great defaults that you shouldn't have to know about. Aside from setting the backend and the BEAM, users of the library shouldn't need to touch environment vars. The story around setitem and assign needs to be better too. A loop where you set each item of a Tensor (common in RL training loops) should be fast. Aside from those 3, I think we are ready for 1.0. What 1.0 means is that we'll start aggressively looking into any reproducible issues, test that we don't break applications using tinygrad, and say to the best of our knowledge tinygrad is a library you can safely build on top of. Users of tinygrad, what do you think? Anything you'd want to see before 1.0?

@DragonStacker · 2025-05-19 21:00
When someone like George Hotz is sharing wisdom, it’s wise to question your own assumptions, instead of whatever this is. Complexity stands in the way of new features. Nearly every aspect of real software engineering deals with either reducing complexity or increasing execution speed. All those design patterns exist to reduce the mental load of the programmer, so they have enough comprehension left over to add that next feature quickly and reliably. Applies to both humans and AI.

@__tinygrad__ · 2025-05-19 21:51
@DragonStacker @InNtrBl6gNs2WXZ shh let them vibe code :p

@__tinygrad__ · 2025-05-19 21:46
@alifahrri2 Stop using onnxruntime, start using tinygrad! Our onnx support is similar in coverage, and if you have a model that won't run we'll look into it.

@alifahrri2 · 2025-05-19 22:22
@__tinygrad__ I wish i could, but unfortunately i have to use c++ (for various reasons) But i'm curious, does tinygrad support onnx quantized ops? I look into test/model/test_onnx.py, but couldn't tell if it support or not

@__tinygrad__ · 2025-05-19 23:31
This is a @comma_ai 3X with more compute power than a HW4 Tesla. It's a 9070XT + ADT-UT3G dock connected to the USB port. Those are FP16 GEMM FLOPS, should be 389 TOPS of int8. Not a tech demo, shipping to openpilot soon. Excited to see how people mount a GPU in their car. https://t.co/SBX6fm5mFV

@alifahrri2 · 2025-05-19 22:22
@__tinygrad__ I wish i could, but unfortunately i have to use c++ (for various reasons) But i'm curious, does tinygrad support onnx quantized ops? I look into test/model/test_onnx.py, but couldn't tell if it support or not

@__tinygrad__ · 2025-05-19 23:34
@alifahrri2 We certainly support quantized ops. Works out of the box, but try QUANTIZE=1 to make them fast.

@__tinygrad__ · 2025-05-19 23:31
This is a @comma_ai 3X with more compute power than a HW4 Tesla. It's a 9070XT + ADT-UT3G dock connected to the USB port. Those are FP16 GEMM FLOPS, should be 389 TOPS of int8. Not a tech demo, shipping to openpilot soon. Excited to see how people mount a GPU in their car. https://t.co/SBX6fm5mFV

@EthaiReubinoff · 2025-05-19 23:40
@__tinygrad__ @comma_ai How is it powered?

@EthaiReubinoff · 2025-05-19 23:40
@__tinygrad__ @comma_ai How is it powered?

@__tinygrad__ · 2025-05-19 23:44
@EthaiReubinoff @comma_ai This one is plugged in inside, but that's going to be a fun challenge for people to figure out! Working on benchmarking the GPU at lower power, I think we can crank it way down with little loss of perf.

@anemll · 2025-05-20 14:02
This is interesting if it’s true, ANE was designed for self driving? Why not use ANE for @comma_ai Openpilot ? https://t.co/pGrFq9kdJS

@vatsal_manot · 2025-05-20 23:05
What’s funny is that @realGeorgeHotz _did_ try and reverse engineer ANE access for @__tinygrad__, and I think that’s the best anyone’s done so far but it’s not nearly enough.

@vatsal_manot · 2025-05-20 23:05
What’s funny is that @realGeorgeHotz _did_ try and reverse engineer ANE access for @__tinygrad__, and I think that’s the best anyone’s done so far but it’s not nearly enough.

@wvabrinskas · 2025-05-21 00:38
@vatsal_manot @realGeorgeHotz @__tinygrad__ Can you even arbitrarily run code on the ANE? It’s just some magic box that runs apples own CoreML frameworks. It would be super nice to hop off the CPU / GPU for matrix multiplications

@vatsal_manot · 2025-05-20 23:05
What’s funny is that @realGeorgeHotz _did_ try and reverse engineer ANE access for @__tinygrad__, and I think that’s the best anyone’s done so far but it’s not nearly enough.

@__tinygrad__ · 2025-05-21 03:32
@vatsal_manot @realGeorgeHotz Apple really locked down access in newer versions of Mac OS. It just got too annoying.

@wvabrinskas · 2025-05-21 00:38
@vatsal_manot @realGeorgeHotz @__tinygrad__ Can you even arbitrarily run code on the ANE? It’s just some magic box that runs apples own CoreML frameworks. It would be super nice to hop off the CPU / GPU for matrix multiplications

@__tinygrad__ · 2025-05-21 03:33
@wvabrinskas @vatsal_manot @realGeorgeHotz You can't, it's quite fixed function.

@__tinygrad__ · 2025-05-23 00:44
The tinygrad CLOUD=1 will present as one HUGE machine with a unified 64-bit address space. That's 18.4 exabytes, containing GPU RAM, GPU control, and NVMe storage. All datasets and models are in the address space at all times, access control and billing through page tables.

@__tinygrad__ · 2025-05-23 00:44
The tinygrad CLOUD=1 will present as one HUGE machine with a unified 64-bit address space. That's 18.4 exabytes, containing GPU RAM, GPU control, and NVMe storage. All datasets and models are in the address space at all times, access control and billing through page tables.

@kristoph · 2025-05-23 01:19
@__tinygrad__ so does the cloud come on the machines? so i can present my own tinygrad machines as a cloud?

@kristoph · 2025-05-23 01:19
@__tinygrad__ so does the cloud come on the machines? so i can present my own tinygrad machines as a cloud?

@__tinygrad__ · 2025-05-23 03:14
@kristoph No, your machine is not the cloud. Your machine is your machine. Our cloud will be open source, so you can call your machine a cloud but it is only one machine so it is a small cloud. However, you can buy a lot of machines and connect them. Then you can have a cloud too.

@__tinygrad__ · 2025-05-23 00:44
The tinygrad CLOUD=1 will present as one HUGE machine with a unified 64-bit address space. That's 18.4 exabytes, containing GPU RAM, GPU control, and NVMe storage. All datasets and models are in the address space at all times, access control and billing through page tables.

@dmsimon · 2025-05-23 03:22
@__tinygrad__ Over what bus at what speed?

@__tinygrad__ · 2025-05-23 04:54
tinybox reds watching tinybox green v2s get adopted https://t.co/UNWyKbKwNk

@dmsimon · 2025-05-23 03:22
@__tinygrad__ Over what bus at what speed?

@__tinygrad__ · 2025-05-23 04:59
@dmsimon https://t.co/sSjmjktxRi

@__tinygrad__ · 2025-05-23 04:54
tinybox reds watching tinybox green v2s get adopted https://t.co/UNWyKbKwNk

@study_of_waifu · 2025-05-23 05:00
@__tinygrad__ That's... Sad.

@study_of_waifu · 2025-05-23 05:00
@__tinygrad__ That's... Sad.

@__tinygrad__ · 2025-05-23 05:01
@study_of_waifu You can make a tinybox red very happy by adopting it. Ships as early as tomorrow!

@__tinygrad__ · 2025-05-23 04:54
tinybox reds watching tinybox green v2s get adopted https://t.co/UNWyKbKwNk

@jsuarez · 2025-05-23 14:09
@__tinygrad__ I'll post the setup with mine when they get here!

@jsuarez · 2025-05-23 14:09
@__tinygrad__ I'll post the setup with mine when they get here!

@__tinygrad__ · 2025-05-23 16:42
@jsuarez Shipping today!

@__tinygrad__ · 2025-05-23 23:28
mmapeak bounty claimed! This is on a 9070XT connected over USB to a Mac. When you use the tinygrad runtime infrastructure, everything just works™ https://t.co/RJpBGJln1e

@__tinygrad__ · 2025-05-24 00:02
@13opoldo Quite bearish on Intel. AMD is worth saving, they have great hardware and a great release cycle, they just lack software. Intel cards have bad memory hierarchies and a very shaky roadmap.

@__tinygrad__ · 2025-05-24 00:03
@jof344 Eh, they are taking up room in the office and I'm sick of looking at them. We'd plug them in but we are out of power.

@SemiAnalysis_ · 2025-05-24 21:45
According to @AIatAMD 's own test, inferencing on ROCm makes your model "dumber" 🤯 This is due to lack of CI & poor quality control on the numeric accuracy of their kernels & compiler 😭 @AnushElangovan needs to task more 996 engineers towards fixing this immediately! https://t.co/JO9Tpm4bxv

@SemiAnalysis_ · 2025-05-24 21:45
According to @AIatAMD 's own test, inferencing on ROCm makes your model "dumber" 🤯 This is due to lack of CI & poor quality control on the numeric accuracy of their kernels & compiler 😭 @AnushElangovan needs to task more 996 engineers towards fixing this immediately! https://t.co/JO9Tpm4bxv

@__tinygrad__ · 2025-05-25 03:34
@SemiAnalysis_ @AIatAMD @AnushElangovan We should get this same suite running in tinygrad CI on AMD.

@__tinygrad__ · 2025-05-25 05:31
We really want a @tenstorrent backend, but the quality and docs of TT-Metalium need to be a lot better. Make it a simple C API, not C++. The documented "CreateDevice" doesn't exist anymore? Metalium should be first class, like CUDA. No one wants device specific APIs like ttnn.

@__tinygrad__ · 2025-05-25 05:31
We really want a @tenstorrent backend, but the quality and docs of TT-Metalium need to be a lot better. Make it a simple C API, not C++. The documented "CreateDevice" doesn't exist anymore? Metalium should be first class, like CUDA. No one wants device specific APIs like ttnn.

@__tinygrad__ · 2025-05-25 05:38
@tenstorrent It's nice to see progress on ISA and chip documentation though. If Tenstorrent wins, it's isn't because of mid LLM implementations, it's because someone finds something that excels on your arch, like AlexNet found CUDA. https://t.co/BlaCvBaoYb

@__tinygrad__ · 2025-05-25 05:31
We really want a @tenstorrent backend, but the quality and docs of TT-Metalium need to be a lot better. Make it a simple C API, not C++. The documented "CreateDevice" doesn't exist anymore? Metalium should be first class, like CUDA. No one wants device specific APIs like ttnn.

@jordonkash · 2025-05-25 05:40
@__tinygrad__ @tenstorrent Since tt-metal is open source, you can just ask the codebase with deepwiki: https://t.co/76KySk9Q6J

@jordonkash · 2025-05-25 05:40
@__tinygrad__ @tenstorrent Since tt-metal is open source, you can just ask the codebase with deepwiki: https://t.co/76KySk9Q6J

@__tinygrad__ · 2025-05-25 05:44
@jordonkash @tenstorrent The Python one in ttnn is still there, but I couldn't link to the C++ one like how the examples say to. https://t.co/nEG3QiNnIr

@__tinygrad__ · 2025-05-25 05:31
We really want a @tenstorrent backend, but the quality and docs of TT-Metalium need to be a lot better. Make it a simple C API, not C++. The documented "CreateDevice" doesn't exist anymore? Metalium should be first class, like CUDA. No one wants device specific APIs like ttnn.

@study_of_waifu · 2025-05-25 05:46
@__tinygrad__ @tenstorrent The majority of documentation should link to actual code, like working tests / examples and the core code, minimize hand-written APIs in docs to avoid stuff like this. I absolutely hate when docs don't work, makes me mistrust the whole project.

@__tinygrad__ · 2025-05-25 05:38
@tenstorrent It's nice to see progress on ISA and chip documentation though. If Tenstorrent wins, it's isn't because of mid LLM implementations, it's because someone finds something that excels on your arch, like AlexNet found CUDA. https://t.co/BlaCvBaoYb

@__tinygrad__ · 2025-05-25 05:47
@tenstorrent Also please fix the fan and power management policies on Blackhole. It boots at 30W, but after you use it once it goes to 65W and stays there. Fan spins up to keep it below 70C (80C is better than noise!). Looked a bit, but didn't find any API to control fan. Did I miss it?

@study_of_waifu · 2025-05-25 05:46
@__tinygrad__ @tenstorrent The majority of documentation should link to actual code, like working tests / examples and the core code, minimize hand-written APIs in docs to avoid stuff like this. I absolutely hate when docs don't work, makes me mistrust the whole project.

@__tinygrad__ · 2025-05-25 05:48
@study_of_waifu @tenstorrent The examples are in a very flaky state. There's tons of hardcoded paths. They might work if you replicate the exact Ubuntu 22.04 setup of the devs...but that's not close to production software.

@__tinygrad__ · 2025-05-25 05:44
@jordonkash @tenstorrent The Python one in ttnn is still there, but I couldn't link to the C++ one like how the examples say to. https://t.co/nEG3QiNnIr

@jordonkash · 2025-05-25 05:52
@__tinygrad__ @tenstorrent see: https://t.co/RvfyvwoR40-56

@jordonkash · 2025-05-25 05:52
@__tinygrad__ @tenstorrent see: https://t.co/RvfyvwoR40-56

@__tinygrad__ · 2025-05-25 05:55
@jordonkash @tenstorrent I eventually found that, but all the examples still say to use CreateDevice! https://t.co/VnEsylMrWs

@beffjezos · 2025-05-25 07:27
AMD needs to acquire @__tinygrad__

@__tinygrad__ · 2025-05-25 08:14
We're open to contracts with makers of accelerators, not acquisition by one. tinygrad's mission is to commoditize the petaflop.

@ShreyashkarLal · 2025-05-25 17:03
Almost all OpenAI models always insist on tinygrad being a toy project like minitorch, micrograd, and others for learning purposes. Even after correction, they usually fall back to the same assumption unless corrected rigorously in between the conversation. Why so?

@__tinygrad__ · 2025-05-25 23:41
because bigger has to be better. you have to have made tradeoffs somewhere...right? right? ...right?? because the alternative is that all software stacks everywhere are insanely bloated and surely a whole industry wouldn't be this stupid? which gpt in what rl loop will realize?

@profleonn · 2025-05-18 18:40
@onarchbtw but its just so juicy clean I love inline if statements, lambdas, list comprehension etc. in python, I just love it not because it reduces loc tho, just because it looks so clean and I lowkey prefer reading it more

@__tinygrad__ · 2025-05-25 23:58
@proffleon @onarchbtw us too. we don't optimize for LoC, we optimize for readability of the whole repo.

@zealandic1 · 2025-05-26 01:23
Accurate! Like back when I was using AMD, it was sort of amazing that their flash attention kernels were not only waaay slower than mine, but also introduced more error. Trying to talk to AMD was like talking to a brick wall though. Great to see that @SemiAnalysis_ is shining a light on this!

@SemiAnalysis_ · 2025-05-26 02:51
AMD customers like Anthonix discovered the same thing that convergence and numerical stability is worse on AMD. Anthonix has spent months doing multi-node on mi300x and has held world records on training speed. Fwiw, this is fixable as the issue stems from @AMD's kernel library and triton/inductor compiler stack We don't agree that "Trying to talk to AMD was like talking to a brick wall though." @anushelangovan is quite responsive & helpful

@SemiAnalysis_ · 2025-05-26 02:51
AMD customers like Anthonix discovered the same thing that convergence and numerical stability is worse on AMD. Anthonix has spent months doing multi-node on mi300x and has held world records on training speed. Fwiw, this is fixable as the issue stems from @AMD's kernel library and triton/inductor compiler stack We don't agree that "Trying to talk to AMD was like talking to a brick wall though." @anushelangovan is quite responsive & helpful

@__tinygrad__ · 2025-05-26 05:37
@SemiAnalysis_ @AMD We need to do a deep dive into this in tinygrad. The non associativity of floating point arithmetic makes the ordering choice lead to major impacts in numerical stability.

@__tinygrad__ · 2025-05-26 15:57
Weekly meeting starts in 3 minutes on our Discord.

@__tinygrad__ · 2025-05-26 15:57
Weekly meeting starts in 3 minutes on our Discord.

@__tinygrad__ · 2025-05-26 17:42
If you missed it, you can always listen to the meetings on YouTube. https://t.co/FVUpYdwLG2

@niconley · 2025-05-27 19:22
this guy is getting paid $35,000 to set up an internal “ChatGPT” for a law firm. > locally hosted Llama for LLM > N8N to connect it all. we’re living in the AI gold rush. https://t.co/cRON8o1XH0

@fangpenlin · 2025-05-27 23:55
Call me crazy if you want. While I don't have the budget and space for a Tinybox, but I do really want to try out team @amdradeon with @__tinygrad__ for machine learning despite people tell you to go with nvidia for ML. Only hope I can find spare time on this project tho 😅 https://t.co/IAaIR9u6d6

@__tinygrad__ · 2025-05-28 00:31
The red boxes are very usable. We have 9 in our cluster now, and people complain a lot less about using them vs when we started. Just install tinygrad, remove AMD stuff with `sudo rmmod amdgpu` and enjoy!

@__tinygrad__ · 2025-05-28 00:31
The red boxes are very usable. We have 9 in our cluster now, and people complain a lot less about using them vs when we started. Just install tinygrad, remove AMD stuff with `sudo rmmod amdgpu` and enjoy!

@__tinygrad__ · 2025-05-28 00:35
We have our own smi at extra/amdpci/am_smi.py https://t.co/vU1FWcpEVp

@__tinygrad__ · 2025-05-28 00:35
We have our own smi at extra/amdpci/am_smi.py https://t.co/vU1FWcpEVp

@__tinygrad__ · 2025-05-28 00:39
While our driver (supports RDNA3+4) isn't perfect and you might hit a minor bug or two (we still have a few unexplained page faults), I have never seen it bring down the system. Kernel panics are sadly still a somewhat frequent occurrence on RDNA3 with AMD's driver.

@__tinygrad__ · 2025-05-28 00:55
Try examples/minrf.py today. It's rectified flow for MNIST using a DiT. No dependencies, pure tinygrad. This was run on a newly imaged tinybox red, but post the result from your machine! If you are patient, it should even work on an Android phone with OpenCL. https://t.co/8I8IAgORwC

@niconley · 2025-05-27 19:22
this guy is getting paid $35,000 to set up an internal “ChatGPT” for a law firm. > locally hosted Llama for LLM > N8N to connect it all. we’re living in the AI gold rush. https://t.co/cRON8o1XH0

@__tinygrad__ · 2025-05-28 01:50
@niconley completely self-hosted ... hosted on coreweave so which is it?

@__tinygrad__ · 2025-05-28 18:08
The 'Move "examples/hlb_cifar10.py" each epoch data processing from numpy into tinygrad (random_crop + cutmix)' bounty should be accessible to all. We shouldn't be spending 4 seconds to shuffle the data, and it's cause there's copies to numpy! https://t.co/qCh8O3BQPh

@qtnx_ · 2025-05-28 18:27
obscure pytorch bug again. should have just used JAX

@__tinygrad__ · 2025-05-29 04:55
@qtnx_ have you considered tinygrad?

@YoloSXS · 2025-05-29 15:14
@__tinygrad__ @qtnx_ Do y'all deliver in EU ?

@YoloSXS · 2025-05-29 15:14
@__tinygrad__ @qtnx_ Do y'all deliver in EU ?

@__tinygrad__ · 2025-05-29 15:41
@YoloSXS @qtnx_ I believe you have GitHub there...

@__tinygrad__ · 2025-06-02 16:28
RT @fangpenlin: Running @__tinygrad__ on @AMDRadeon with NixOS making it a bit more challenging, but yay! I finally get it to run the MNIST example on one card. Super cool! 😍 👍 https://t.co/BmiyGjcOig

@__tinygrad__ · 2025-06-02 17:20
Anyone in the Eastern Washington / Idaho area running a datacenter? We're looking to buy something in the 5MW range. Passing through the area for the next two days, reach out to george@tinygrad.org would be excited to see what people are up to.

@__tinygrad__ · 2025-06-02 17:20
Anyone in the Eastern Washington / Idaho area running a datacenter? We're looking to buy something in the 5MW range. Passing through the area for the next two days, reach out to george@tinygrad.org would be excited to see what people are up to.

@HotAisle · 2025-06-02 17:44
@__tinygrad__ Central WA is better and as another person commented, Qunicy is the place to go.

@__tinygrad__ · 2025-06-04 16:55
RT @jsuarez: The tinyboxes are here! 12x 4090s from @__tinygrad__ ready for reinforcement learning. Want to run experiments on these? Contribute cool environments to pufferlib for access! https://t.co/8tbZ84QvHP

@__tinygrad__ · 2025-06-04 16:56
RT @fangpenlin: VIZ web tool for tinygrad is pretty cool 👌 https://t.co/WkuERiytzM

@HotAisle · 2025-06-02 17:44
@__tinygrad__ Central WA is better and as another person commented, Qunicy is the place to go.

@__tinygrad__ · 2025-06-04 16:57
@HotAisle Visited @H5_Datacenters in Quincy today, very legit.

@__tinygrad__ · 2025-06-04 16:57
@HotAisle Visited @H5_Datacenters in Quincy today, very legit.

@HotAisle · 2025-06-04 17:14
We were in there for years mining away. It is the old Intuit building that they lost a bunch of money on when they sold out to H5. One of our other locations (we had 7 total) is just down the street from there too, but that isn't a traditional DC, it was just containers in a dirt field. There is someone specific who might still be working at that DC that I wouldn't recommend interacting with at all, but other than that, they were mostly ok. I doubt they support DLC without a lot of NRC on your end. The inside space also runs a bit hot in the summer. It is all hydro there, so that is nice. You're welcome. 😜 https://t.co/pIEtVEWASe

@__tinygrad__ · 2025-06-04 17:35
MLPerf 5.0 Training is out. tinygrad's speed has been improving at a rapid pace since 4.1! BERT tinybox red : 397 -> 300 minutes BERT tinybox green : 360 -> 210 minutes Same computer, 42% faster. We also got the MI300X box on there and dipped our toes into RetinaNet.

@HotAisle · 2025-06-04 17:14
We were in there for years mining away. It is the old Intuit building that they lost a bunch of money on when they sold out to H5. One of our other locations (we had 7 total) is just down the street from there too, but that isn't a traditional DC, it was just containers in a dirt field. There is someone specific who might still be working at that DC that I wouldn't recommend interacting with at all, but other than that, they were mostly ok. I doubt they support DLC without a lot of NRC on your end. The inside space also runs a bit hot in the summer. It is all hydro there, so that is nice. You're welcome. 😜 https://t.co/pIEtVEWASe

@__tinygrad__ · 2025-06-04 17:41
@HotAisle Oh you were the crypto miners they talked about. I liked that they ran miners, our philosophy on things like redundancy looks a lot more like miners than most AI workloads. If it's down it's down, not gonna pay 2x for 95% -> 99.9% uptime.

@__tinygrad__ · 2025-06-04 17:35
MLPerf 5.0 Training is out. tinygrad's speed has been improving at a rapid pace since 4.1! BERT tinybox red : 397 -> 300 minutes BERT tinybox green : 360 -> 210 minutes Same computer, 42% faster. We also got the MI300X box on there and dipped our toes into RetinaNet.

@HotAisle · 2025-06-04 18:24
@__tinygrad__ How are you spending the $2m?

@HotAisle · 2025-06-04 18:24
@__tinygrad__ How are you spending the $2m?

@__tinygrad__ · 2025-06-05 00:15
@HotAisle This 111 minute time was a low effort demo. For the $2M we'll be getting an NVIDIA competitive time and doing something a bit harder than BERT. We'll see if @AnushElangovan comes through with the contract.

@DavidBennett__ · 2025-06-05 19:07
Jim Keller: ‘Whatever Nvidia Does, We'll Do The Opposite’ https://t.co/qCkO69sr3Q Great article @sallywf @tenstorrent

@__tinygrad__ · 2025-06-05 20:14
"Ethernet is fine! Smaller, lower cost chips are a good idea. Simpler servers are a good idea. Open-source software is a good idea." -- @jimkxa Consumer GPUs in tinyboxes running tinygrad!

@__tinygrad__ · 2025-06-05 20:14
"Ethernet is fine! Smaller, lower cost chips are a good idea. Simpler servers are a good idea. Open-source software is a good idea." -- @jimkxa Consumer GPUs in tinyboxes running tinygrad!

@Object_Zero_ · 2025-06-05 20:59
@__tinygrad__ @jimkxa Are you able to reverse engineer 40GB TCP/IP on thunderbolt for PCIe to PCIe? What network fabric does tinybox use? Are cables/adapters a lower hardware barrier than chips?

@Object_Zero_ · 2025-06-05 20:59
@__tinygrad__ @jimkxa Are you able to reverse engineer 40GB TCP/IP on thunderbolt for PCIe to PCIe? What network fabric does tinybox use? Are cables/adapters a lower hardware barrier than chips?

@__tinygrad__ · 2025-06-05 21:27
@Object_Zero_ @jimkxa RoCE works fine, direct from one GPU to another. tinyboxes have an OCP 3.0 port at the full 16x width.

@__tinygrad__ · 2025-06-06 18:25
Today is a great day to buy a tinybox red. Everyone always buys green, but there's good money to save buying red. Most of the boxes we run for ourselves are red. 3 left, we'd use them ourselves but we are out of power. https://t.co/3ujeIXePMP

@__tinygrad__ · 2025-06-06 18:25
Today is a great day to buy a tinybox red. Everyone always buys green, but there's good money to save buying red. Most of the boxes we run for ourselves are red. 3 left, we'd use them ourselves but we are out of power. https://t.co/3ujeIXePMP

@macmaniac77 · 2025-06-06 18:31
@__tinygrad__ Amazing that you became power limited so quickly.

@hsu_steve · 2025-06-06 20:34
Looking for an engineer with experience in running small LLMs on phone chip sets. Please DM me!

@macmaniac77 · 2025-06-06 18:31
@__tinygrad__ Amazing that you became power limited so quickly.

@__tinygrad__ · 2025-06-06 21:59
@macmaniac77 We only have 3x 208V 50A circuits in the tinygrad office. @comma_ai has a lot more, but they need their power!

@__tinygrad__ · 2025-06-06 22:12
@luciusd20 Normally very true. But AMD now has an advantage, the stack (aside from firmware, which @AMD still needs to open) is completely open source. No GSP binary and magical SASS compiler like NVIDIA, just PM4 (reg writes) + upstream LLVM.

@__tinygrad__ · 2025-06-06 22:19
2 GPU allreduce https://t.co/oApkCVvE4P

@hsu_steve · 2025-06-06 20:34
Looking for an engineer with experience in running small LLMs on phone chip sets. Please DM me!

@__tinygrad__ · 2025-06-06 22:29
@hsu_steve What phone and what model? tinygrad is actually probably the best way to do this. It's open source and you can do it, or if you want us to hand it to you on a platter, give us a contract and a perf target.

@hsu_steve · 2025-06-06 20:34
Looking for an engineer with experience in running small LLMs on phone chip sets. Please DM me!

@0xhooved · 2025-06-07 00:49
@hsu_steve Here’s a live demo of llama-3.2-1B that runs in the browser on newer phones like iPhone 15, that I made for tinygrad: https://t.co/xQoO9aq6mL

@hsu_steve · 2025-06-06 20:34
Looking for an engineer with experience in running small LLMs on phone chip sets. Please DM me!

@RoryClear · 2025-06-07 17:34
@hsu_steve tinygrad! REMOTE=1 HOST={ip:port} python examples/gpt2.py https://t.co/JZ1GYBet7k

@ClementDelangue · 2025-06-07 21:07
The shift from pickles to safetensors might be one of the biggest AI safety progress of the past 12 months! And yet, very few people who are talking about AI safety are mentioning it (they prefer to talk about robocop sci-fi scenarios that attract more media). Feels like AI safety is probably a topic where there’s the biggest gap between people talking about safety (let’s call them “safety influencers”) and people who are actually making AI safer (and usually don’t talk about it). Let’s change this maybe?

@__tinygrad__ · 2025-06-07 21:30
RT @jsuarez: Reinforcement learning on 100,000,000,000 observations overnight on a single TinyBox has set new SOTA on Neural MMO 3. Previous best was just under 6.0. This has an effective batch size of 3 million and a minibatch of ~180k. Star PufferLib to support and come dev with us! https://t.co/QYLEi66AEq

@__tinygrad__ · 2025-06-07 21:32
Run an LLM in your browser! Should just work everywhere, has both a WebGPU (faster) and WASM version.

@RoryClear · 2025-06-07 17:34
@hsu_steve tinygrad! REMOTE=1 HOST={ip:port} python examples/gpt2.py https://t.co/JZ1GYBet7k

@__tinygrad__ · 2025-06-07 21:35
@RoryClear @hsu_steve Woah cool! Does this work? I'll dig up an Apple device and try it.

@__tinygrad__ · 2025-06-07 21:43
A chip is great, but what you really want is a good scalable framework (the S3 of Tensor) Why can't I add two petabyte tensors together? from tinygrad import Tensor pb = Tensor.zeros(1000,1000,1000,1000,1000) (pb+1)[0:10, 0, 0, 0, 0].tolist() Now try that in torch.

@__tinygrad__ · 2025-06-07 21:45
Happy weekend from these 5090s! https://t.co/Lo9x6lgRYj

@__tinygrad__ · 2025-06-07 22:02
@gen_z_reprezent Both. You should never have to think about running out of ram. Even written an MNIST trainer and just copied the whole dataset to GPU ram and wow it was easy. Yea...that but for 50 TB datasets too. Paging.

@__tinygrad__ · 2025-06-07 22:44
Real AI safety! tinygrad has used safetensors as the primary storage format from the beginning. It's an excellent format.

@__tinygrad__ · 2025-06-07 22:48
@gen_z_reprezent For better or worse, ONNX is the standard. We're going to move our 0 dep ONNX parser into tinygrad in a week or two, and due to how tinygrad does fusion there's 0 perf overhead in using it.

@unclemusclez · 2025-06-08 05:56
i'm still very unsure of what tinygrad does

@__tinygrad__ · 2025-06-08 07:15
It's like Uber for Tensors

@__tinygrad__ · 2025-06-10 05:35
tinybox green v2 can allreduce *all* of its GPU memory in under a second https://t.co/CgI87ykWBI

@__tinygrad__ · 2025-06-10 17:59
we test comma's models on device in our CI https://t.co/MR0bwQBLMY

@fangpenlin · 2025-06-11 17:45
https://t.co/VOc0QpJGuD I summarized my experience using two 7900XTX GPUs for my machine learning workstation, opting for them over pricier Nvidia GPUs. 😄🎉 Using @tinygrad as the ML library, I also shared fixes for P2P PCIe communication issues. https://t.co/NwC5LzCZH0

@__tinygrad__ · 2025-06-11 20:23
Just buy a tinybox red!

@__tinygrad__ · 2025-06-12 18:33
The next batch of tinybox v2 cases is in the mail. We should have all current v2 green orders shipped out by the end of next week.

@lmsysorg · 2025-06-12 20:25
Huge thanks to @AMD for donating an MI350 to SGLang! This advanced AI accelerator is making a meaningful difference—enabling us to move faster in developing scalable LLM systems and pushing the limits of inference optimization. Special thank to our awesome infra partner InnoMatrix. The SGLang + InnoMatrix teams had the MI350 racked and ready within 8 hours of arrival 🤯 AMD, YES! #AMD #MI350 #SGLANG

@IanCutress · 2025-06-12 21:08
Seems like I'm the only one that noticed that @AMD's Helios is a double wide rack. I've been saying for a while at some point we're going to change what a 'rack' means - and the biggest uplift will be more copper before optics. Next step is 70U+. https://t.co/iXT8MhNuNp :) https://t.co/TZydFHslAP

@__tinygrad__ · 2025-06-12 21:39
This is the way to make your hardware relevant. Make it open source, and make sure there's tons of it available for engineers and development teams.

@IanCutress · 2025-06-12 21:08
Seems like I'm the only one that noticed that @AMD's Helios is a double wide rack. I've been saying for a while at some point we're going to change what a 'rack' means - and the biggest uplift will be more copper before optics. Next step is 70U+. https://t.co/iXT8MhNuNp :) https://t.co/TZydFHslAP

@__tinygrad__ · 2025-06-12 22:32
@IanCutress @AMD If we can have double wide trailers, why not double wide racks?!

@__tinygrad__ · 2025-06-12 22:37
Can we all just stop with the "sparsity flops"? Have you ever met a user of sparsity flops? This is worse than resort fees at hotels is this price after the fee / before the fee just stop and report normal FLOPS a sparsity flop isn't even a flop like dude you multiplied by 0.

@__tinygrad__ · 2025-06-12 23:20
ready for @comma_ai car https://t.co/NPs6itBq8x

@__tinygrad__ · 2025-06-12 23:20
ready for @comma_ai car https://t.co/NPs6itBq8x

@monsieur_teddy · 2025-06-12 23:23
@__tinygrad__ @comma_ai Where is the buy button 🙏

@monsieur_teddy · 2025-06-12 23:23
@__tinygrad__ @comma_ai Where is the buy button 🙏

@__tinygrad__ · 2025-06-12 23:24
@monsieur_teddy @comma_ai GPU - https://t.co/awxc2QiqUz Dock - https://t.co/cfY6OnV8eA PSU - https://t.co/KlBdniSXxT Adapter - https://t.co/4ASZVwW7jP Power - https://t.co/DIV3fgdP74

@__tinygrad__ · 2025-06-12 23:20
ready for @comma_ai car https://t.co/NPs6itBq8x

@OverfitForTruth · 2025-06-12 23:35
@__tinygrad__ @comma_ai could comma ever support the users adding more cameras? would it be possible to incorporate and/or would it be beneficial?

@OverfitForTruth · 2025-06-12 23:35
@__tinygrad__ @comma_ai could comma ever support the users adding more cameras? would it be possible to incorporate and/or would it be beneficial?

@__tinygrad__ · 2025-06-12 23:36
@OverfitForTruth @comma_ai https://t.co/wXCbS3reGy

@__tinygrad__ · 2025-06-12 23:20
ready for @comma_ai car https://t.co/NPs6itBq8x

@LabRatKnatz · 2025-06-12 23:40
@__tinygrad__ @comma_ai RDNA4 over USB3, or TBT#/USB4?

@squarecapo · 2025-06-13 00:01
there is or there isn't .lazydata in tinygrad Tensor?

@LabRatKnatz · 2025-06-12 23:40
@__tinygrad__ @comma_ai RDNA4 over USB3, or TBT#/USB4?

@__tinygrad__ · 2025-06-13 00:28
@LabRatKnatz @comma_ai USB3. comma devices don't have anything else (and neither do most computers)

@__tinygrad__ · 2025-06-13 00:48
It's interesting how the USB GPU wouldn't really be possible on NVIDIA. AMD open sources the real instruction set that runs on the GPU + upstreams the LLVM to compile to it. NVIDIA doesn't and keeps SASS closed. Hoping market pressure fixes this and the tensor core nerfing.

@squarecapo · 2025-06-13 00:01
there is or there isn't .lazydata in tinygrad Tensor?

@__tinygrad__ · 2025-06-13 01:45
@squarecapo It's Tensor.uop now. Same thing.

@__tinygrad__ · 2025-06-12 22:37
Can we all just stop with the "sparsity flops"? Have you ever met a user of sparsity flops? This is worse than resort fees at hotels is this price after the fee / before the fee just stop and report normal FLOPS a sparsity flop isn't even a flop like dude you multiplied by 0.

@shiels_ai · 2025-06-13 03:39
@__tinygrad__ Day 2 of asking when the tinybox green will be in stock again

@__tinygrad__ · 2025-06-13 00:48
It's interesting how the USB GPU wouldn't really be possible on NVIDIA. AMD open sources the real instruction set that runs on the GPU + upstreams the LLVM to compile to it. NVIDIA doesn't and keeps SASS closed. Hoping market pressure fixes this and the tensor core nerfing.

@DaveAirlie · 2025-06-13 05:49
@__tinygrad__ We do have sass compilers in mesa, one fully written in rust, just finishing off 5080 support now

@DaveAirlie · 2025-06-13 05:49
@__tinygrad__ We do have sass compilers in mesa, one fully written in rust, just finishing off 5080 support now

@__tinygrad__ · 2025-06-13 06:08
@DaveAirlie Oh sweet! How's the performance compare? And what's the input IR? (doing some research)

@DaveAirlie · 2025-06-13 06:10
@__tinygrad__ It's built on mesa NIR internal IR, the main frontend to that nowadays is SPIR-V

@__tinygrad__ · 2025-06-13 06:14
@DaveAirlie tinygrad should just be able to output NIR. It's similar enought to our UOps. Compiler is here? https://t.co/uBAP9EGtEo

@shiels_ai · 2025-06-13 03:39
@__tinygrad__ Day 2 of asking when the tinybox green will be in stock again

@__tinygrad__ · 2025-06-13 06:22
@shiels_ai Ships in 2-6 weeks. https://t.co/nGRxle3Npe

@__tinygrad__ · 2025-06-13 06:14
@DaveAirlie tinygrad should just be able to output NIR. It's similar enought to our UOps. Compiler is here? https://t.co/uBAP9EGtEo

@__tinygrad__ · 2025-06-13 06:29
@DaveAirlie Added a $300 bounty if someone can hook this into tinygrad.

@__tinygrad__ · 2025-06-13 19:58
2 tinybox reds sold, 2 remain available for immediate shipping. tinybox green v2 shipping time updated to 1-6 weeks, order today! https://t.co/DOU8lvwYBL

@__tinygrad__ · 2025-06-18 15:52
We've been negotiating a $2M contract to get AMD on MLPerf, but one of the sticking points has been confidentiality. Perhaps posting the deliverables on X will help legal to get in the spirit of open source! https://t.co/cnOiumwmHl

@__tinygrad__ · 2025-06-18 15:52
We've been negotiating a $2M contract to get AMD on MLPerf, but one of the sticking points has been confidentiality. Perhaps posting the deliverables on X will help legal to get in the spirit of open source! https://t.co/cnOiumwmHl

@real_deep_ml · 2025-06-18 15:54
@__tinygrad__ GitHub for contracts would be great!

@seloesque · 2025-06-18 15:54
balls of steel, make the corpos kneel street kid. https://t.co/JchyXfdU31

@seloesque · 2025-06-18 15:54
balls of steel, make the corpos kneel street kid. https://t.co/JchyXfdU31

@__tinygrad__ · 2025-06-18 15:55
@seloesque Would rather lose the contract than a part of our sovereignty. You have to think long term about these things.

@__tinygrad__ · 2025-06-18 15:52
We've been negotiating a $2M contract to get AMD on MLPerf, but one of the sticking points has been confidentiality. Perhaps posting the deliverables on X will help legal to get in the spirit of open source! https://t.co/cnOiumwmHl

@KnotGrigori · 2025-06-18 15:57
@__tinygrad__ Bold move! Hopefully AMD puts down the foot gun…

@KnotGrigori · 2025-06-18 15:57
@__tinygrad__ Bold move! Hopefully AMD puts down the foot gun…

@__tinygrad__ · 2025-06-18 16:02
@KnotGrigori The real problem is that it's a lot cheaper for them to waste your time than it is for you to waste there's. And even worse, they are aware of this so they use it as a negotiating tactic. If you play their game you lose.

@real_deep_ml · 2025-06-18 15:54
@__tinygrad__ GitHub for contracts would be great!

@__tinygrad__ · 2025-06-18 16:05
@real_deep_ml You don't love a 30 deep email chain of docx files with various incomplete highlights? You clearly aren't cut out for professional legal work!

@__tinygrad__ · 2025-06-18 15:52
We've been negotiating a $2M contract to get AMD on MLPerf, but one of the sticking points has been confidentiality. Perhaps posting the deliverables on X will help legal to get in the spirit of open source! https://t.co/cnOiumwmHl

@ggerganov · 2025-06-18 16:06
@__tinygrad__ Curious what timeframe do you estimate for this project?

@ggerganov · 2025-06-18 16:06
@__tinygrad__ Curious what timeframe do you estimate for this project?

@__tinygrad__ · 2025-06-18 16:08
@ggerganov About a year, pretty much full time for the team. This is the cutting edge of training with all the tricks. Flash attention, optimizer sharding, gradient accumulation, overlapped communication, multi node support, etc...

@AnushElangovan · 2025-06-18 17:46
Actually good point. Maybe in the spirit of open source we should make this bounty available to whoever does the best job in Open Source not just to one entity. @__tinygrad__ is welcome to participate in it - and Open Source wins.

@__tinygrad__ · 2025-06-18 17:50
@AnushElangovan Having run bounties for a while, I doubt you'll get good results that way. What you'll get is the lowest effort set of scripts cobbled together to technically meet the target, then basically have to pay out for something that has very little value.

@AnushElangovan · 2025-06-18 17:52
@__tinygrad__ yes but I am sure @__tinygrad__ will do a quality submission given you have the experience to submit to MLPerf already.

@AnushElangovan · 2025-06-18 17:52
@__tinygrad__ yes but I am sure @__tinygrad__ will do a quality submission given you have the experience to submit to MLPerf already.

@__tinygrad__ · 2025-06-18 17:53
@AnushElangovan Not for 405B, that's pretty far off roadmap, and we don't have any machines that can run it. Without a contract we won't submit for that. Bounties work in the range of several hundred dollars + an expected day of effort, but even in the $10k range they break down. You'll see.

@jeremyphoward · 2025-06-18 20:27
@AnushElangovan @__tinygrad__ ML competitions (with which I'm more familiar than pretty much anyone on the planet) are not generally the way to handle large systems problems. They need to be tightly defined, of reasonable scope, and readily scoreable using automated metrics.

@__tinygrad__ · 2025-06-18 20:37
@jeremyphoward @AnushElangovan This says it better than I did. This contract would be a year of work for our whole team. It's huge in scope, takes days to score (multiday training run), and requires specialized hardware. I hope it is, but I'm not sure the bounty suggestion was in good faith.

@__tinygrad__ · 2025-06-18 20:37
@jeremyphoward @AnushElangovan This says it better than I did. This contract would be a year of work for our whole team. It's huge in scope, takes days to score (multiday training run), and requires specialized hardware. I hope it is, but I'm not sure the bounty suggestion was in good faith.

@__tinygrad__ · 2025-06-18 20:43
@jeremyphoward @AnushElangovan It reads similar to this. I thought when they actually sent us the 2 boxes they realized that this wasn't a good way to engage, but maybe not. https://t.co/B8NrWIgPcP

@__tinygrad__ · 2025-06-18 20:43
@jeremyphoward @AnushElangovan It reads similar to this. I thought when they actually sent us the 2 boxes they realized that this wasn't a good way to engage, but maybe not. https://t.co/B8NrWIgPcP

@jeremyphoward · 2025-06-18 20:45
@__tinygrad__ @AnushElangovan Yeah I agree. I think Anush is just particularly bad at this part of his job. They need someone who's an expert at dev relationships doing social outreach. Basically every other company in the AI/ML hw and sw spaces have dedicated people that are really good at this.

@jeremyphoward · 2025-06-18 20:45
@__tinygrad__ @AnushElangovan Yeah I agree. I think Anush is just particularly bad at this part of his job. They need someone who's an expert at dev relationships doing social outreach. Basically every other company in the AI/ML hw and sw spaces have dedicated people that are really good at this.

@__tinygrad__ · 2025-06-18 20:54
@jeremyphoward @AnushElangovan I think it's cool that @AnushElangovan is actually running software, and I think that's a lot better than engaging with a social media manager. But perhaps he underestimates the difficulty of this, we're still the only team to get AMD on MLPerf non-LoRA training.

@__tinygrad__ · 2025-06-18 21:31
@gguthrie93 @AnushElangovan But if they don't contract us, they can do $6.002B of stock buybacks instead of $6B. You always have to consider the stock buybacks.

@__tinygrad__ · 2025-06-18 21:40
In other (non AMD) news... https://t.co/3vqdVfP2tp

@__tinygrad__ · 2025-06-18 15:52
We've been negotiating a $2M contract to get AMD on MLPerf, but one of the sticking points has been confidentiality. Perhaps posting the deliverables on X will help legal to get in the spirit of open source! https://t.co/cnOiumwmHl

@__tinygrad__ · 2025-06-19 16:12
Contract is signed! No confidentiality, AMD has leadership that's capable of acting. Let's make this training run happen, we work in public on our Discord.

@__tinygrad__ · 2025-06-19 16:33
RT @CharlesRDog: If I could I would like this post over and over. Tiny Corp is without a doubt the absolute coolest galaxy brained group building in public. I've attended the open weekly meetings a few times and I've learned a lot from them. Bravo!

@__tinygrad__ · 2025-06-20 23:46
Two tinybox green v2s shipped today! New orders will ship in around 2 weeks. Pic related, it's the birth of a tinybox. https://t.co/NAvAaCouJp

@__tinygrad__ · 2025-06-20 23:46
Two tinybox green v2s shipped today! New orders will ship in around 2 weeks. Pic related, it's the birth of a tinybox. https://t.co/NAvAaCouJp

@JatevoId · 2025-06-20 23:51
@__tinygrad__ If we have 4 of this, can you arrange for us? https://t.co/wCKc0BrHru

@JatevoId · 2025-06-20 23:51
@__tinygrad__ If we have 4 of this, can you arrange for us? https://t.co/wCKc0BrHru

@__tinygrad__ · 2025-06-20 23:56
@JatevoId In order to keep prices low and ensure quality with standardized automatic testing, we don't offer customization.

@__tinygrad__ · 2025-06-21 03:23
gonna change the prices of the tinyboxes to "contact us" links but when you click them it says just kidding and tells you the price

@ludwigABAP · 2025-06-25 13:47
tenstorrent https://t.co/aX4aYTdBZO

@roman_koshchei · 2025-06-25 13:57
@ludwigABAP They should pay @__tinygrad__ to do software for their hardware

@__tinygrad__ · 2025-06-25 16:06
We now have 3 refactor bounties available. These are a great way to dive into the internals of tinygrad! Purely refactor, do the task with tests passing. AI is not capable of doing them. AI (codex/claude code) PRs will be immediately closed / you may be banned from our GitHub. https://t.co/SK9KBRPRDi

@__tinygrad__ · 2025-06-25 16:08
As tinygrad gains adoption, this will be the best move for most AI chip makers.

@__tinygrad__ · 2025-06-25 16:06
We now have 3 refactor bounties available. These are a great way to dive into the internals of tinygrad! Purely refactor, do the task with tests passing. AI is not capable of doing them. AI (codex/claude code) PRs will be immediately closed / you may be banned from our GitHub. https://t.co/SK9KBRPRDi

@0xCodyS · 2025-06-25 16:11
@__tinygrad__ Are all coded PRs banned or just bounties?

@0xCodyS · 2025-06-25 16:11
@__tinygrad__ Are all coded PRs banned or just bounties?

@__tinygrad__ · 2025-06-25 16:12
@0xCodyS While this will change in the future, AI is currently far below the bar of anyone we would want contributing to our codebase. Using ChatGPT in another window to help you understand things is welcome, one click to autosubmit PR crap is not.

@__tinygrad__ · 2025-06-25 16:08
As tinygrad gains adoption, this will be the best move for most AI chip makers.

@KalenuikMatthew · 2025-06-25 16:37
@__tinygrad__ All I need to know is do I long AMD or not lol :)

@__tinygrad__ · 2025-06-25 16:06
We now have 3 refactor bounties available. These are a great way to dive into the internals of tinygrad! Purely refactor, do the task with tests passing. AI is not capable of doing them. AI (codex/claude code) PRs will be immediately closed / you may be banned from our GitHub. https://t.co/SK9KBRPRDi

@SergioGaitanC · 2025-06-25 16:57
@__tinygrad__ why do you think ai is not capable of completing them?

@KalenuikMatthew · 2025-06-25 16:37
@__tinygrad__ All I need to know is do I long AMD or not lol :)

@__tinygrad__ · 2025-06-25 16:59
@KalenuikMatthew Take it with a heaping of self serving, but AMD as a company has been mostly easy to work with, and the software is improving. NVIDIA became a bit arrogant (rightfully so?) after the stock rise in 2017. Intel has no leadership. And Qualcomm has an 80s era sales org driving it.

@SergioGaitanC · 2025-06-25 16:57
@__tinygrad__ why do you think ai is not capable of completing them?

@__tinygrad__ · 2025-06-25 17:09
@SergioGaitanC The same way I can look at a person's work and know if they are capable or not.

@__tinygrad__ · 2025-06-25 16:59
@KalenuikMatthew Take it with a heaping of self serving, but AMD as a company has been mostly easy to work with, and the software is improving. NVIDIA became a bit arrogant (rightfully so?) after the stock rise in 2017. Intel has no leadership. And Qualcomm has an 80s era sales org driving it.

@moritzthuening · 2025-06-25 17:13
@__tinygrad__ @KalenuikMatthew Tenstorrent is fun to work with. They want customers to succeed.

@moritzthuening · 2025-06-25 17:13
@__tinygrad__ @KalenuikMatthew Tenstorrent is fun to work with. They want customers to succeed.

@__tinygrad__ · 2025-06-25 17:21
@moritzthuening @KalenuikMatthew We like Tenstorrent and really want them to succeed, they have a great understanding of where things will end up. But in practice, their software is not good and their cores are overly complex.

@__tinygrad__ · 2025-06-25 16:08
As tinygrad gains adoption, this will be the best move for most AI chip makers.

@roman_koshchei · 2025-06-25 19:12
@__tinygrad__ Ain't no way people will write tenstorrent specific models and ain't no way they are going to add support to pytorch

@__tinygrad__ · 2025-06-25 22:02
tinybox green v2 orders are fulfilled up to Jun 17. Next week, we'll have our first excess build capacity. Buy today, have it in a business week! https://t.co/npzIrMlipX

@roman_koshchei · 2025-06-25 19:12
@__tinygrad__ Ain't no way people will write tenstorrent specific models and ain't no way they are going to add support to pytorch

@__tinygrad__ · 2025-06-25 22:05
@roman_koshchei They are building for the wrong layer. They should be focused on writing @tenstorrent CUDA, not some LLM implementations nobody will use.

@__tinygrad__ · 2025-06-26 23:50
RT @xl0xl0xl0: Did a @__tinygrad__ bounty together with github:IntendedConsequence and documented (link in the coments) the process. Was fun! https://t.co/zbBiPExzfU

@__tinygrad__ · 2025-06-27 03:35
RT @fangpenlin: Woot! Two AMD 7900XTX GPUs training with Tinygrad 😍 Surprisingly, sharding worked perfectly after fixing a silly self-inflicted issue. Very cool. GPU usage fluctuates a bit, I may need to optimize it.

@__tinygrad__ · 2025-06-27 17:16
We get reach-ins to both tiny and @comma_ai asking for unpaid internships. But between the lines, they are asking for education. Unpaid isn't cheap enough, look at the price of college. Everything you need is on GitHub and Discord. There's nothing stopping you except discipline.

@rohanpaul_ai · 2025-06-27 17:46
These guys literally burned the transformer architecture into their silicon. 🤯 And built the fastest chip of the world of all time for transformers architecture. 500,000 tokens per second with Llama 70B throughput. 🤯 World’s first specialized chip (ASIC) for transformers: Sohu One 8xSohu server replaces 160 H100 GPUs. And raised $120mn to build it. 🚀 The Big Bet @Etched froze the transformer recipe into silicon. By burning the transformer architecture into its chip means it can’t run many traditional AI models: like CNNs, RNNs, or LSTMs. also it can not run the DLRMs powering Instagram ads, protein-folding models like AlphaFold 2, or older image models like Stable Diffusion 2. But for transformers, Sohu lets you build products impossible on GPUs. HOW ❓❓ Because Sohu can only run one algorithm, the vast majority of control flow logic can be removed, allowing it to have many more math blocks. As a result, Sohu boasts over 90% FLOPS utilization (compared to ~30% on a GPU7 with TRT-LLM).

@__tinygrad__ · 2025-06-27 17:16
We get reach-ins to both tiny and @comma_ai asking for unpaid internships. But between the lines, they are asking for education. Unpaid isn't cheap enough, look at the price of college. Everything you need is on GitHub and Discord. There's nothing stopping you except discipline.

@hopes_revenge · 2025-06-27 17:50
@__tinygrad__ @comma_ai this seems like a willful misunderstanding of why people seek internships. i'm not sure why any company would post this. absolutely no upside here.

@__tinygrad__ · 2025-06-27 17:16
We get reach-ins to both tiny and @comma_ai asking for unpaid internships. But between the lines, they are asking for education. Unpaid isn't cheap enough, look at the price of college. Everything you need is on GitHub and Discord. There's nothing stopping you except discipline.

@LenSeaside · 2025-06-27 17:55
@__tinygrad__ @comma_ai Yeah right because we've all just got a few $100ks lying around to buy a stack of 5090s... This is a mean bullshit thing to post.

@__tinygrad__ · 2025-06-27 17:16
We get reach-ins to both tiny and @comma_ai asking for unpaid internships. But between the lines, they are asking for education. Unpaid isn't cheap enough, look at the price of college. Everything you need is on GitHub and Discord. There's nothing stopping you except discipline.

@taustw · 2025-06-27 18:05
@__tinygrad__ @comma_ai So… juniors should pay for working *with* you?

@hopes_revenge · 2025-06-27 17:50
@__tinygrad__ @comma_ai this seems like a willful misunderstanding of why people seek internships. i'm not sure why any company would post this. absolutely no upside here.

@__tinygrad__ · 2025-06-27 18:31
@hopes_revenge @comma_ai ugh stop thinking in terms of "upside" and start thinking in terms of truth.

@LenSeaside · 2025-06-27 17:55
@__tinygrad__ @comma_ai Yeah right because we've all just got a few $100ks lying around to buy a stack of 5090s... This is a mean bullshit thing to post.

@__tinygrad__ · 2025-06-27 18:33
@LenSeaside @comma_ai huh? tinygrad runs on the crappiest computer you have, and many of the bounties don't require any hardware. there's a bunch of new refactor ones that certainly don't. stop making excuses.

@taustw · 2025-06-27 18:05
@__tinygrad__ @comma_ai So… juniors should pay for working *with* you?

@__tinygrad__ · 2025-06-27 18:41
@taustw @comma_ai there's a big misunderstanding about junior engineers. every engineer, no matter how new, should be able to contribute value independently. they might not have the best instincts for design and might work slower, but if you can't contribute value without handholding you are ngmi

@__tinygrad__ · 2025-06-29 17:34
How does this have traction? What gains does anyone think this gets over GPUs to get 20x over H100s, or do they assume the audience isn't sophisticated enough to ask that question with any rigor and just sees nonsense like "burned the transformer into the chip"?

@__tinygrad__ · 2025-06-29 17:34
How does this have traction? What gains does anyone think this gets over GPUs to get 20x over H100s, or do they assume the audience isn't sophisticated enough to ask that question with any rigor and just sees nonsense like "burned the transformer into the chip"?

@SMT_Solvers · 2025-06-29 17:35
@__tinygrad__ The only way is if they increased memory locality for these workflows. It's not by compute speedup.

@__tinygrad__ · 2025-06-29 17:40
Here's a breakdown of power usage in NVIDIA chips. Can you find the place to gain 95% by "burning" something? I wish VCs had any amount of technical understanding and didn't just have an unlimited appetite for risk. "Well they said 20x they'll figure it out!" <-- no they won't https://t.co/T7LuVGHmBm

@__tinygrad__ · 2025-06-29 17:43
With the large systolic arrays in TPUs and B200s there's little more to be gained from instructions. This chart shows why GPU makers are pushing small dtypes. GPUs are near optimal GEMM machines. https://t.co/YIiSnLejY8

@__tinygrad__ · 2025-06-29 17:34
How does this have traction? What gains does anyone think this gets over GPUs to get 20x over H100s, or do they assume the audience isn't sophisticated enough to ask that question with any rigor and just sees nonsense like "burned the transformer into the chip"?

@KBlueleaf · 2025-06-29 17:44
You can definitely hard coded the whole dataflow and computation of what those NN layers do into a chip but this also means you cannot do other things. But yes it will be SUPER efficient, like you can directly put the attention module (use flash attn scheme so you can tile the input), normalization module and MLP module into chip (with input tiling) and since "what you want to do" is deterministic, you don't need any instruction fetcher, decoder, branch prediction blablabla so it will be super fast and efficient

@__tinygrad__ · 2025-06-29 17:43
With the large systolic arrays in TPUs and B200s there's little more to be gained from instructions. This chart shows why GPU makers are pushing small dtypes. GPUs are near optimal GEMM machines. https://t.co/YIiSnLejY8

@__tinygrad__ · 2025-06-29 17:47
Can someone please send this thread to Etched's investors? They are preying on people's inability to watch a Hot Chips talk and understand the bottlenecks. There are real ideas to make better hardware for AI, but "20x gains from etching transformers into silicon" is nonsense. https://t.co/8NuvP3hosN

@__tinygrad__ · 2025-06-29 17:47
Can someone please send this thread to Etched's investors? They are preying on people's inability to watch a Hot Chips talk and understand the bottlenecks. There are real ideas to make better hardware for AI, but "20x gains from etching transformers into silicon" is nonsense. https://t.co/8NuvP3hosN

@SMT_Solvers · 2025-06-29 17:49
@__tinygrad__ * They can get better data locality with ASIC. * This is a TSMC venture, they get to use the latest process node and fuck around with high etch error/multiple etch pass designs.

@KBlueleaf · 2025-06-29 17:44
You can definitely hard coded the whole dataflow and computation of what those NN layers do into a chip but this also means you cannot do other things. But yes it will be SUPER efficient, like you can directly put the attention module (use flash attn scheme so you can tile the input), normalization module and MLP module into chip (with input tiling) and since "what you want to do" is deterministic, you don't need any instruction fetcher, decoder, branch prediction blablabla so it will be super fast and efficient

@__tinygrad__ · 2025-06-29 17:51
@KBlueleaf But that's so little of the power! GPUs are already really good at this, and retain flexibility. You just made a specialized chip in exchange for pretty much nothing.

@SMT_Solvers · 2025-06-29 17:49
@__tinygrad__ * They can get better data locality with ASIC. * This is a TSMC venture, they get to use the latest process node and fuck around with high etch error/multiple etch pass designs.

@__tinygrad__ · 2025-06-29 17:53
@SMT_Solvers Oh yes, TSMC can't wait to give a better process node to some random startup. Sorry Apple, NVIDIA, and AMD, you are going to have to wait, we are prioritizing Etched now 😂

@__tinygrad__ · 2025-06-29 17:51
@KBlueleaf But that's so little of the power! GPUs are already really good at this, and retain flexibility. You just made a specialized chip in exchange for pretty much nothing.

@KBlueleaf · 2025-06-29 17:55
Note, GPU is NOT good at this considering how much area it waste. (in terms of AI workload) and with highly specialized design you can actually achieve "one-cycle" flash attn or MLP implementation which each input tile only occupy one cycle since you use a highly specialized pipeline which directly compute all the needed stuff instead of using instruction + SIMT/SIMD But yes the flexibility is what it lacks, (but it is "ASIC" which will never have flexibility)

@SMT_Solvers · 2025-06-29 17:35
@__tinygrad__ The only way is if they increased memory locality for these workflows. It's not by compute speedup.

@__tinygrad__ · 2025-06-29 17:56
@SMT_Solvers GPUs are already stupidly good at memory locality, and they are adding new features every generation bringing them closer to some potential gains here from a more tenstorrent like arch (SM to SM comms). There's nothing even close to a 20x here.

@__tinygrad__ · 2025-06-29 17:58
@ZekeGeiss Our goal is to commoditize the petaflop (aka lower costs per FLOP). There actually is a 10x to be gained here, NVIDIA's margin on H100s is like 90%!

@__tinygrad__ · 2025-06-29 17:47
Can someone please send this thread to Etched's investors? They are preying on people's inability to watch a Hot Chips talk and understand the bottlenecks. There are real ideas to make better hardware for AI, but "20x gains from etching transformers into silicon" is nonsense. https://t.co/8NuvP3hosN

@__tinygrad__ · 2025-06-29 18:21
At what point is something fraud? Their website reads like someone very new to this space ("we can fit way more FLOPS on our chip without resorting to lower precisions"). Naiveté isn't fraud. But shame on their investors for not properly doing due diligence on this.

@KBlueleaf · 2025-06-29 17:55
Note, GPU is NOT good at this considering how much area it waste. (in terms of AI workload) and with highly specialized design you can actually achieve "one-cycle" flash attn or MLP implementation which each input tile only occupy one cycle since you use a highly specialized pipeline which directly compute all the needed stuff instead of using instruction + SIMT/SIMD But yes the flexibility is what it lacks, (but it is "ASIC" which will never have flexibility)

@__tinygrad__ · 2025-06-29 18:29
@KBlueleaf So things like Gaudi are even more optimal GEMM machines than GPUs, but the gains on only on area, not on power, and they are a lot harder to program. I'm not sure what "one-cycle" flash attention means, like you still have to fetch the chunk from DRAM and go to some SRAM.

@__tinygrad__ · 2025-06-29 18:29
@KBlueleaf So things like Gaudi are even more optimal GEMM machines than GPUs, but the gains on only on area, not on power, and they are a lot harder to program. I'm not sure what "one-cycle" flash attention means, like you still have to fetch the chunk from DRAM and go to some SRAM.

@joefioti · 2025-06-29 18:31
@__tinygrad__ @KBlueleaf If you use scratchpad buffers you can have an average throughput of one tile per cycle and just pipeline all the tiles.

@__tinygrad__ · 2025-06-29 18:29
@KBlueleaf So things like Gaudi are even more optimal GEMM machines than GPUs, but the gains on only on area, not on power, and they are a lot harder to program. I'm not sure what "one-cycle" flash attention means, like you still have to fetch the chunk from DRAM and go to some SRAM.

@KBlueleaf · 2025-06-29 18:34
@__tinygrad__ What I'm saying is not only gemm It's a more specialized things The whole activation and dataflow is only designed for single NN arch so it cannot even do GEMM. Like it may directly burn the swiglu into hardware

@joefioti · 2025-06-29 18:31
@__tinygrad__ @KBlueleaf If you use scratchpad buffers you can have an average throughput of one tile per cycle and just pipeline all the tiles.

@__tinygrad__ · 2025-06-29 18:34
@joefioti @KBlueleaf Is attention that big of a percent of the power and time compared to the GEMMs? It obviously depends on context length, but I though it was like 70% GEMMs, 20% attention, 10% other.

@KBlueleaf · 2025-06-29 18:34
@__tinygrad__ What I'm saying is not only gemm It's a more specialized things The whole activation and dataflow is only designed for single NN arch so it cannot even do GEMM. Like it may directly burn the swiglu into hardware

@KBlueleaf · 2025-06-29 18:35
@__tinygrad__ GEMM's Ge means general Remember, ASIC is never for General Gaudi is just a more AI specialized compute unit, not an Asic

@KBlueleaf · 2025-06-29 18:34
@__tinygrad__ What I'm saying is not only gemm It's a more specialized things The whole activation and dataflow is only designed for single NN arch so it cannot even do GEMM. Like it may directly burn the swiglu into hardware

@__tinygrad__ · 2025-06-29 18:36
@KBlueleaf But the swiglu is so small of a percent of the power and time! I believe you can make a 20x more efficient swiglu, but if it's only 1% of the budget who cares?

@KBlueleaf · 2025-06-29 18:35
@__tinygrad__ GEMM's Ge means general Remember, ASIC is never for General Gaudi is just a more AI specialized compute unit, not an Asic

@__tinygrad__ · 2025-06-29 18:37
@KBlueleaf ASIC is just a word, GPUs are an application specific integrated circuit too. Your transformer only chip is still going to have to do the same GEMM as Gaudi or GPU.

@__tinygrad__ · 2025-06-29 18:37
@KBlueleaf ASIC is just a word, GPUs are an application specific integrated circuit too. Your transformer only chip is still going to have to do the same GEMM as Gaudi or GPU.

@KBlueleaf · 2025-06-29 18:42
No, it would not. That's why I said you never get it.. It need matmul never means it will have gemm. GEMM is not only Matmul, it requires you to support baddbmm. (aXY + bZ) But it is not the case in arch specific design, For attn core I can directly ignore the bias part since lot of attn module use no bias input proj. And in a fused, burned chip this means you save more area and cycle, and power.

@KBlueleaf · 2025-06-29 18:42
No, it would not. That's why I said you never get it.. It need matmul never means it will have gemm. GEMM is not only Matmul, it requires you to support baddbmm. (aXY + bZ) But it is not the case in arch specific design, For attn core I can directly ignore the bias part since lot of attn module use no bias input proj. And in a fused, burned chip this means you save more area and cycle, and power.

@__tinygrad__ · 2025-06-29 18:43
@KBlueleaf Ahh, I'm using GEMM and Matmul as synonyms. You are still going to need the accumulator though, and do the scalars really add any sizable cost?

@KBlueleaf · 2025-06-29 18:39
I think you didn't get the key here The key is "not for general" Which means you can save tons of cycle and area which you need for "GE"MM And it means you have higher efficiency in both term of power and area. Also, in ASIC you can burn the quantize/dequantize into the hardware as well so it becomes way way more faster. Even the ram design can be specifically constructed. For example: If I have 2 main part: attention processor and MLP/activation processor. I can totally separate the mem space of this 2 part which make the ram controller more easy to design and mote efficient (BTW, gd7 and memory controller in RTX5090 with real world workload will consume way more power than the gpu core)

@joefioti · 2025-06-29 18:44
@KBlueleaf @__tinygrad__ also from info-theory perspective if you use M number of N bit weights, the total info stored is MxN, but the number of transistors needed to do the multiplies is Mx(N^2) which pushes N down and M up. So higher number of lower precision params is likely the future.

@joefioti · 2025-06-29 18:44
@KBlueleaf @__tinygrad__ also from info-theory perspective if you use M number of N bit weights, the total info stored is MxN, but the number of transistors needed to do the multiplies is Mx(N^2) which pushes N down and M up. So higher number of lower precision params is likely the future.

@__tinygrad__ · 2025-06-29 18:45
@joefioti @KBlueleaf Certainly agree on the higher number of lower precision params. That's why when I see etched say things like (linked) you know it's either super naive or a scam. https://t.co/vx1V9UsMZ8

@__tinygrad__ · 2025-06-29 18:43
@KBlueleaf Ahh, I'm using GEMM and Matmul as synonyms. You are still going to need the accumulator though, and do the scalars really add any sizable cost?

@KBlueleaf · 2025-06-29 18:47
addition of floats is always more expensive than multiplication ↑this is not well-known but it's truth. But other important part here is you can directly connect the accumulator to next step instead of send output back to cache/ram. For example: inp tile -> q, k tile → flash attn score tile → store to memor Instead of inp tile → q, k tile → memory memory → q, k tile → attn score tile → memory

@__tinygrad__ · 2025-06-29 17:34
How does this have traction? What gains does anyone think this gets over GPUs to get 20x over H100s, or do they assume the audience isn't sophisticated enough to ask that question with any rigor and just sees nonsense like "burned the transformer into the chip"?

@stolsvik · 2025-06-29 18:53
I am not a silicon expert in any way, but I thought this was obvious?! I mean, you don’t run crypto mining on GPUs anymore, as ASICs are massively better. I assumed the same was true for this problem? Yes, you have memory in LLMs compared to double-sha of Bitcoin, but even most «memory hard» crypto have been ASICed. Also, Groq and Cerebras are also specialized «ASIC alike» specialized solutions for LLMs that goes faster. Going one level deeper intuitively makes sense, assuming you can take the risk of transformers being relevant for a few more years?

@KBlueleaf · 2025-06-29 19:06
TPU is doing this, but only one step So it's f(gemm(inps)), but with custom arch asic you can connect more steps Also, "tensor core" (Nvidia) is just matmul unit and it's not using systolic array. And for NPU from AMD ( the AIE-MLv2 from xilinx team) using NoC design so "it can be" systolic array, but it doesn't have activation unit directly connect on it.

@vdmrzv · 2025-06-29 19:11
@KBlueleaf @__tinygrad__ wow, so llms constantly misguided me saying that nvidia tensor cores are systollic arrays

@stolsvik · 2025-06-29 18:53
I am not a silicon expert in any way, but I thought this was obvious?! I mean, you don’t run crypto mining on GPUs anymore, as ASICs are massively better. I assumed the same was true for this problem? Yes, you have memory in LLMs compared to double-sha of Bitcoin, but even most «memory hard» crypto have been ASICed. Also, Groq and Cerebras are also specialized «ASIC alike» specialized solutions for LLMs that goes faster. Going one level deeper intuitively makes sense, assuming you can take the risk of transformers being relevant for a few more years?

@__tinygrad__ · 2025-06-29 19:14
@stolsvik This is exactly the hand wavy thinking that VCs fall prey to. They know engineering is about tradeoffs, and see that Etched is up front about the tradeoffs. Surely no one would be dumb enough to make massive flexibility tradeoffs in exchange for minimal gains. Right? Right!?

@vdmrzv · 2025-06-29 19:11
@KBlueleaf @__tinygrad__ wow, so llms constantly misguided me saying that nvidia tensor cores are systollic arrays

@__tinygrad__ · 2025-06-29 19:16
@vdmrzv @KBlueleaf i'm pretty sure they are systolic arrays.

@KBlueleaf · 2025-06-29 18:47
addition of floats is always more expensive than multiplication ↑this is not well-known but it's truth. But other important part here is you can directly connect the accumulator to next step instead of send output back to cache/ram. For example: inp tile -> q, k tile → flash attn score tile → store to memor Instead of inp tile → q, k tile → memory memory → q, k tile → attn score tile → memory

@__tinygrad__ · 2025-06-29 19:20
@KBlueleaf "addition of floats is always more expensive than multiplication" wait what? maybe things get strange at really small dtypes, but mult is N^2 and add is N, and power reflects this. https://t.co/lEivIHWMa5

@__tinygrad__ · 2025-06-29 19:14
@stolsvik This is exactly the hand wavy thinking that VCs fall prey to. They know engineering is about tradeoffs, and see that Etched is up front about the tradeoffs. Surely no one would be dumb enough to make massive flexibility tradeoffs in exchange for minimal gains. Right? Right!?

@stolsvik · 2025-06-29 19:24
Yeah, okay - but is it really minimal? There would be no instruction fetching, decoding, pipelining, etc? It would just be a straight line of operations, literally hard coded? The Groq and Cerebras gains speed for a different architecture with faster memory closer, and less flexibility. Can’t this be «scaled» even more?

@KBlueleaf · 2025-06-29 19:25
@__tinygrad__ 1. Expensive means area. 2. Small dtype usually means LUT is cheaper than real logic circuit so yes things become weird. For example in my custom TPU design I use custom dtype which allow me to make log/exp/inv works with LUT6 from xilinx fpga. So I can save TONS of resources.

@KBlueleaf · 2025-06-29 19:27
Addition is more expensive in terms of: 1. Area 2. Layout difficulties Bcuz for handling shift (the "floating " mantissa) and subnorm or special case will occupy tons of area and make layout a nightmare. But they are not power consuming Also their custom dtype is fixed points which means it is actually kind lf int.

@trickylabyrinth · 2025-06-29 19:51
this thread sounds correct, but it's clashing against my broader intuition from bitcoin mining that ASICs >>> GPUs on all of silicon utilization, energy efficiency and overall hashrate are GPUs just strangely bad at hashing, and if so, why?

@rohanpaul_ai · 2025-06-27 17:46
These guys literally burned the transformer architecture into their silicon. 🤯 And built the fastest chip of the world of all time for transformers architecture. 500,000 tokens per second with Llama 70B throughput. 🤯 World’s first specialized chip (ASIC) for transformers: Sohu One 8xSohu server replaces 160 H100 GPUs. And raised $120mn to build it. 🚀 The Big Bet @Etched froze the transformer recipe into silicon. By burning the transformer architecture into its chip means it can’t run many traditional AI models: like CNNs, RNNs, or LSTMs. also it can not run the DLRMs powering Instagram ads, protein-folding models like AlphaFold 2, or older image models like Stable Diffusion 2. But for transformers, Sohu lets you build products impossible on GPUs. HOW ❓❓ Because Sohu can only run one algorithm, the vast majority of control flow logic can be removed, allowing it to have many more math blocks. As a result, Sohu boasts over 90% FLOPS utilization (compared to ~30% on a GPU7 with TRT-LLM).

@__tinygrad__ · 2025-06-29 19:51
@rohanpaul_ai Are you getting paid to post this? If you are and didn't disclose, this looks like a Fyre Festival situation.

@__tinygrad__ · 2025-06-29 19:51
@rohanpaul_ai Are you getting paid to post this? If you are and didn't disclose, this looks like a Fyre Festival situation.

@__tinygrad__ · 2025-06-29 19:52
@rohanpaul_ai ...particularly if Etched will announce a funding round soon.

@trickylabyrinth · 2025-06-29 19:51
this thread sounds correct, but it's clashing against my broader intuition from bitcoin mining that ASICs >>> GPUs on all of silicon utilization, energy efficiency and overall hashrate are GPUs just strangely bad at hashing, and if so, why?

@__tinygrad__ · 2025-06-29 19:56
@trickylabyrinth Hashing is integer math, not the floating point math GPUs are made for. And it doesn't require any memory, which GPUs spend a ton on. Bitcoin style hashing uses 10% of the GPU, GEMMs (the largest part of transformers) use pretty much all of it.

@__tinygrad__ · 2025-06-29 18:21
At what point is something fraud? Their website reads like someone very new to this space ("we can fit way more FLOPS on our chip without resorting to lower precisions"). Naiveté isn't fraud. But shame on their investors for not properly doing due diligence on this.

@__tinygrad__ · 2025-06-29 20:06
I asked if this was a paid post and the reply was hidden by the post author. If it is paid, you did not disclose, and this turns out to be fraud... https://t.co/bSql46uHhe

@__tinygrad__ · 2025-06-29 20:06
I asked if this was a paid post and the reply was hidden by the post author. If it is paid, you did not disclose, and this turns out to be fraud... https://t.co/bSql46uHhe

@__tinygrad__ · 2025-06-29 20:13
Err, yea...doesn't look good, probably paid promo. Is it worth seriously diving into this? I know Etched is trying to raise. Do we care if their investors are going to lose a lot of money? Or should we let it be? https://t.co/s7p9HWOfsm

@KBlueleaf · 2025-06-29 19:27
Addition is more expensive in terms of: 1. Area 2. Layout difficulties Bcuz for handling shift (the "floating " mantissa) and subnorm or special case will occupy tons of area and make layout a nightmare. But they are not power consuming Also their custom dtype is fixed points which means it is actually kind lf int.

@__tinygrad__ · 2025-06-29 20:18
@KBlueleaf I agree this all ends up at "custom dtypes" that is just the fused boolean logic for the operation. And there may be some serious gains here, dtypes have been the biggest gains to date. However, Etched claims in their questionable blog post that they are using FP8.

@__tinygrad__ · 2025-06-29 20:13
Err, yea...doesn't look good, probably paid promo. Is it worth seriously diving into this? I know Etched is trying to raise. Do we care if their investors are going to lose a lot of money? Or should we let it be? https://t.co/s7p9HWOfsm

@iotcoi · 2025-06-29 20:30
@__tinygrad__ oh c'mon. The dude is making money from greedy VCs and greedier LPs. Not a bad deed.

@iotcoi · 2025-06-29 20:30
@__tinygrad__ oh c'mon. The dude is making money from greedy VCs and greedier LPs. Not a bad deed.

@__tinygrad__ · 2025-06-29 20:30
@iotcoi We live in a society.

@stolsvik · 2025-06-29 19:24
Yeah, okay - but is it really minimal? There would be no instruction fetching, decoding, pipelining, etc? It would just be a straight line of operations, literally hard coded? The Groq and Cerebras gains speed for a different architecture with faster memory closer, and less flexibility. Can’t this be «scaled» even more?

@__tinygrad__ · 2025-06-29 20:36
@stolsvik The Google TPU eliminates almost all of this already, few cores, in-order VLIW. It's ~on par with NVIDIA, not way better.

@stolsvik · 2025-06-29 19:24
Yeah, okay - but is it really minimal? There would be no instruction fetching, decoding, pipelining, etc? It would just be a straight line of operations, literally hard coded? The Groq and Cerebras gains speed for a different architecture with faster memory closer, and less flexibility. Can’t this be «scaled» even more?

@0xBE7A · 2025-06-29 20:57
@stolsvik @__tinygrad__ Cerebras only gain ~2x in energy efficiency (their own numbers) compared to a B200. The chip is way faster than a single B200 (28x) but also consumes more power (23x). Cooling a 23kW chip is really near the limit. So without power efficiency gains it’s not really archivable.

@0xBE7A · 2025-06-29 20:57
@stolsvik @__tinygrad__ Cerebras only gain ~2x in energy efficiency (their own numbers) compared to a B200. The chip is way faster than a single B200 (28x) but also consumes more power (23x). Cooling a 23kW chip is really near the limit. So without power efficiency gains it’s not really archivable.

@__tinygrad__ · 2025-06-29 20:59
@0xBE7A @stolsvik Exactly. There's no way to 20x the power to a chip. That 23kW is a wafer, and Cerebras has a crazy complex cooling system they innovated on! I believe their 2x and overall like Cerebras btw, the gains come from not using DRAM.

@__tinygrad__ · 2025-06-29 20:59
@0xBE7A @stolsvik Exactly. There's no way to 20x the power to a chip. That 23kW is a wafer, and Cerebras has a crazy complex cooling system they innovated on! I believe their 2x and overall like Cerebras btw, the gains come from not using DRAM.

@0xBE7A · 2025-06-29 21:07
@__tinygrad__ @stolsvik They only claim ~1.2x compared to a single B200 and ~2.2x compared to a DGX B200 (8x B200 + NVSwitches + Host), so saving on data movement between devices seems to be where there main gains come from (they got everything on the same waver / chip)

@0xBE7A · 2025-06-29 21:07
@__tinygrad__ @stolsvik They only claim ~1.2x compared to a single B200 and ~2.2x compared to a DGX B200 (8x B200 + NVSwitches + Host), so saving on data movement between devices seems to be where there main gains come from (they got everything on the same waver / chip)

@__tinygrad__ · 2025-06-29 21:40
@0xBE7A @stolsvik I could see gains in data movement also. @tenstorrent has a lot of the high level arch stuff correct, these big crossbars are power hungry and not needed as software improves.

@Saemin4655 · 2025-06-30 08:52
This post will age like ass

@Saemin4655 · 2025-06-30 08:52
This post will age like ass

@__tinygrad__ · 2025-06-30 17:18
@Saemin4655 Want to bet? $10k says that NVIDIA has more deployments than Etched in 5 years. If it really were 20x, surely everyone would switch, right?

@__tinygrad__ · 2025-06-30 21:39
Last chance to order a tinybox v2 before we e-mail the mailing list and lead times go back up. If you are an early order placed today it will ship this week. Pic: a tinybox v2 with a side removed in provisioning. https://t.co/Ww5EhFf2Bk

@__tinygrad__ · 2025-07-01 02:42
RT @basedwebscraper: @__tinygrad__ 😄 https://t.co/WL4wN6sFBY

@__tinygrad__ · 2025-06-30 21:39
Last chance to order a tinybox v2 before we e-mail the mailing list and lead times go back up. If you are an early order placed today it will ship this week. Pic: a tinybox v2 with a side removed in provisioning. https://t.co/Ww5EhFf2Bk

@jacobtohahn · 2025-07-01 03:59
@__tinygrad__ What’s the difference between v1 and v2?

@__tinygrad__ · 2025-07-01 05:42
A key aspect of beating NVIDIA on MLPerf Llama 405B with AMD will be great flash attention. This is autogenerated basic flash attention. Now we're upgrading the optimizer to make it fast. The Halide PhD thesis is a great way to understand the search space. https://t.co/jatgDoAAe2

@__tinygrad__ · 2025-07-01 05:42
A key aspect of beating NVIDIA on MLPerf Llama 405B with AMD will be great flash attention. This is autogenerated basic flash attention. Now we're upgrading the optimizer to make it fast. The Halide PhD thesis is a great way to understand the search space. https://t.co/jatgDoAAe2

@__tinygrad__ · 2025-07-01 05:43
Halide thesis available here. https://t.co/sBsG3PYv3K

@__tinygrad__ · 2025-07-01 05:42
A key aspect of beating NVIDIA on MLPerf Llama 405B with AMD will be great flash attention. This is autogenerated basic flash attention. Now we're upgrading the optimizer to make it fast. The Halide PhD thesis is a great way to understand the search space. https://t.co/jatgDoAAe2

@ChrisFriedler · 2025-07-01 05:43
@__tinygrad__ Can the newest Hardware of AMD compete with NVIDIA with your optimization?

@ChrisFriedler · 2025-07-01 05:43
@__tinygrad__ Can the newest Hardware of AMD compete with NVIDIA with your optimization?

@__tinygrad__ · 2025-07-01 05:45
@ChrisFriedler That's the contract. AMD MI355X with tinygrad beats NVIDIA B200 with NeMo.

@__tinygrad__ · 2025-07-01 05:43
Halide thesis available here. https://t.co/sBsG3PYv3K

@__tinygrad__ · 2025-07-01 05:54
...or maybe we should just paste the unoptimized kernel into ChatGPT and vibecode a drop-in fast replacement! https://t.co/w1LPrABlGm

@__tinygrad__ · 2025-07-01 15:19
RT @SanthosRaj16: Idk why but I love using @__tinygrad__

@jsuarez · 2025-07-01 17:08
https://t.co/okxFMcrRJG

@__tinygrad__ · 2025-06-30 21:39
Last chance to order a tinybox v2 before we e-mail the mailing list and lead times go back up. If you are an early order placed today it will ship this week. Pic: a tinybox v2 with a side removed in provisioning. https://t.co/Ww5EhFf2Bk

@s1ddok · 2025-07-01 19:31
@__tinygrad__ worldwide shipping at my cost?

@s1ddok · 2025-07-01 19:31
@__tinygrad__ worldwide shipping at my cost?

@__tinygrad__ · 2025-07-02 02:47
@s1ddok Anywhere you can order to on the website

@jacobtohahn · 2025-07-01 03:59
@__tinygrad__ What’s the difference between v1 and v2?

@__tinygrad__ · 2025-07-02 02:47
@jacobtohahn v2 is 5090, v1 was 4090.

@__tinygrad__ · 2025-07-02 17:08
tinybox in action!

@__tinygrad__ · 2025-07-02 21:20
RT @wpmed92: Try tinygrad’s webgpu backend on windows: https://t.co/RudtuzsLKJ

@karpathy · 2025-07-05 21:54
How to build a thriving open source community by writing code like bacteria do 🦠. Bacterial code (genomes) are: - small (each line of code costs energy) - modular (organized into groups of swappable operons) - self-contained (easily "copy paste-able" via horizontal gene transfer) If chunks of code are small, modular, self-contained and trivial to copy-and-paste, the community can thrive via horizontal gene transfer. For any function (gene) or class (operon) that you write: can you imagine someone going "yoink" without knowing the rest of your code or having to import anything new, to gain a benefit? Could your code be a trending GitHub gist? This coding style guide has allowed bacteria to colonize every ecological nook from cold to hot to acidic or alkaline in the depths of the Earth and the vacuum of space, along with an insane diversity of carbon anabolism, energy metabolism, etc. It excels at rapid prototyping but... it can't build complex life. By comparison, the eukaryotic genome is a significantly larger, more complex, organized and coupled monorepo. Significantly less inventive but necessary for complex life - for building entire organs and coordinating their activity. With our advantage of intelligent design, it should possible to take advantage of both. Build a eukaryotic monorepo backbone if you have to, but maximize bacterial DNA.

@__tinygrad__ · 2025-07-06 23:31
Wrote a blog post about if tinygrad can win or not. It's not a foregone conclusion, ChatGPT gives it a 25% chance. But I'll bet on it. https://t.co/HWARs3PVUa

@__tinygrad__ · 2025-07-06 23:31
Wrote a blog post about if tinygrad can win or not. It's not a foregone conclusion, ChatGPT gives it a 25% chance. But I'll bet on it. https://t.co/HWARs3PVUa

@__tinygrad__ · 2025-07-06 23:31
Here's the post about what it takes. https://t.co/wPaE9hvWIN

@__tinygrad__ · 2025-07-07 15:55
end to end will win neural network frameworks just like it is winning self driving cars. if you have a Conv3D guy, you are going to lose. he's the new cone guy.

@__tinygrad__ · 2025-07-07 15:55
end to end will win neural network frameworks just like it is winning self driving cars. if you have a Conv3D guy, you are going to lose. he's the new cone guy.

@KikeljGregor · 2025-07-07 16:06
@__tinygrad__ I'd say general compilers have a lot of hand coded optimizations, why assume neural network compilers would behave particularly differently? Are general compilers too hard to do end to end?

@KikeljGregor · 2025-07-07 16:06
@__tinygrad__ I'd say general compilers have a lot of hand coded optimizations, why assume neural network compilers would behave particularly differently? Are general compilers too hard to do end to end?

@cloud11665 · 2025-07-07 16:17
The possible program space of general purpose cpu programs is much much much higher than the space of chained tensor operations. Even searching for isomorphic programs on the cpu is borderline impossible because of all of the state attached whereas here you just have to get the expected output

@KikeljGregor · 2025-07-07 16:06
@__tinygrad__ I'd say general compilers have a lot of hand coded optimizations, why assume neural network compilers would behave particularly differently? Are general compilers too hard to do end to end?

@__tinygrad__ · 2025-07-07 16:32
@KikeljGregor General compilers compile Turing complete code, there's no way to benchmark them cause what data do you use? This isn't true for neural networks.

@cloud11665 · 2025-07-07 16:17
The possible program space of general purpose cpu programs is much much much higher than the space of chained tensor operations. Even searching for isomorphic programs on the cpu is borderline impossible because of all of the state attached whereas here you just have to get the expected output

@__tinygrad__ · 2025-07-07 19:05
@cloud11665 @KikeljGregor Even worse than the space just being big, it's undecidable if two general programs implement the same thing.

@tambetm · 2025-07-07 19:59
@__tinygrad__ Look at the latest release blog: https://t.co/F2WfgW0FAB. There is hardly any new functionality announced. Yes, you now train your model in simulation now - so what? This should result in clearly visible outcome in terms of functionality, but it hasn't.

@tambetm · 2025-07-07 20:02
@__tinygrad__ End-to-end is the easy path that gives you 80% of the result for 20% of the effort. Yes, following the lane is 80% of the driving. But what really matters in self-driving is the rest 20%! Unfortunately, that does not come with simple imitation learning.

@__tinygrad__ · 2025-07-08 00:02
Our MI350X machines are here, thanks @AMD! They are just two racks down from their MI300X friends. https://t.co/kwlg2tZK11

@__tinygrad__ · 2025-07-08 00:02
Our MI350X machines are here, thanks @AMD! They are just two racks down from their MI300X friends. https://t.co/kwlg2tZK11

@rrespectorr · 2025-07-08 00:30
@__tinygrad__ @AMD Damn! 8 x MI350X (288 GB HBM3e) 2x 2.304 TB Really compact 4.608 TB HBM3e What is NVIDIA's version of this? https://t.co/AX9PeomL7j

@kierank_ · 2025-07-08 00:04
@__tinygrad__ @AMD What is the purpose of the cardboard?

@HotAisle · 2025-07-08 00:31
@kierank_ @__tinygrad__ @AMD It catches on fire faster than plastic. Most sane DC’s don’t allow any paper products where the machines are.

@tambetm · 2025-07-07 20:02
@__tinygrad__ End-to-end is the easy path that gives you 80% of the result for 20% of the effort. Yes, following the lane is 80% of the driving. But what really matters in self-driving is the rest 20%! Unfortunately, that does not come with simple imitation learning.

@__tinygrad__ · 2025-07-08 02:15
@tambetm Slowly and then quickly. Do you really think hand coded feature spaces are the future?

@rrespectorr · 2025-07-08 00:30
@__tinygrad__ @AMD Damn! 8 x MI350X (288 GB HBM3e) 2x 2.304 TB Really compact 4.608 TB HBM3e What is NVIDIA's version of this? https://t.co/AX9PeomL7j

@__tinygrad__ · 2025-07-08 02:21
@respectorr69 @AMD AFAIK, they don't have one. This might be the most powerful single machine you can buy, save for the water cooled MI355X.

@HotAisle · 2025-07-08 00:31
@kierank_ @__tinygrad__ @AMD It catches on fire faster than plastic. Most sane DC’s don’t allow any paper products where the machines are.

@__tinygrad__ · 2025-07-08 02:38
@HotAisle @kierank_ @AMD Now that we have these nice computers, we should get more serious about this. Working on it.

@__tinygrad__ · 2025-07-08 02:21
@respectorr69 @AMD AFAIK, they don't have one. This might be the most powerful single machine you can buy, save for the water cooled MI355X.

@KBlueleaf · 2025-07-08 03:02
@__tinygrad__ @respectorr69 @AMD AFAIK, they do have one.

@KBlueleaf · 2025-07-08 03:02
@__tinygrad__ @respectorr69 @AMD AFAIK, they do have one.

@__tinygrad__ · 2025-07-08 03:30
@KBlueleaf @respectorr69 @AMD Which one? I know GB300 is supposed to be, but I don't think it's available yet.

@joefioti · 2025-07-08 04:36
bitter lesson matmul search in Luminal! https://t.co/qLZelf9n1v

@__tinygrad__ · 2025-07-08 02:15
@tambetm Slowly and then quickly. Do you really think hand coded feature spaces are the future?

@tambetm · 2025-07-08 15:00
@__tinygrad__ My main concern is with imitation learning as the main method to train end-to-end networks. Take causal confusion - did you turn left because your lane did or because the leading car did? Also end-to-end is very brittle when you change your sensor layout or parameters.

@tambetm · 2025-07-08 15:00
@__tinygrad__ My main concern is with imitation learning as the main method to train end-to-end networks. Take causal confusion - did you turn left because your lane did or because the leading car did? Also end-to-end is very brittle when you change your sensor layout or parameters.

@__tinygrad__ · 2025-07-08 15:58
@tambetm Yea, agreed imitation learning won't work. You have to build a world model and learn in it. https://t.co/ZkDOYgr0YR

@__tinygrad__ · 2025-07-08 23:28
@joefioti BEAM=2 (to search) and DEBUG=4 (to print) in tinygrad. You are exploring some interesting directions we aren't with stuff like e-graphs, but you should really just use our frontend and come in at either the scheduler or kernel gen UOp level.

@__tinygrad__ · 2025-07-08 23:30
@joefioti I know you'll resist this, not invented here and all that. But think about where you will innovate and where you won't. tinygrad has 0 deps, so it's not like it's a whole ecosystem. If I were you, I'd work on replacing tinygrad's codegen directory and skip frontend/runtime.

@__tinygrad__ · 2025-07-08 23:30
@joefioti I know you'll resist this, not invented here and all that. But think about where you will innovate and where you won't. tinygrad has 0 deps, so it's not like it's a whole ecosystem. If I were you, I'd work on replacing tinygrad's codegen directory and skip frontend/runtime.

@__tinygrad__ · 2025-07-08 23:45
@joefioti There's a bunch of places in codegen your spec is better. LoopIn/LoopOut is better than Ops.RANGE, and our Opts aren't rewrite rules. Working on fixing this. But then you have things like this...is that contiguous later removed or is that a real kernel boundary for any reshape? https://t.co/nkgcUKPkDx

@__tinygrad__ · 2025-07-08 23:45
@joefioti There's a bunch of places in codegen your spec is better. LoopIn/LoopOut is better than Ops.RANGE, and our Opts aren't rewrite rules. Working on fixing this. But then you have things like this...is that contiguous later removed or is that a real kernel boundary for any reshape? https://t.co/nkgcUKPkDx

@joefioti · 2025-07-08 23:50
@__tinygrad__ Thanks george. we're doing a transition from 1.0 to "2.0" which entails the new IR. stuff in /src is 1.0 and old, so stuff like that reshape fn is old and will be deprecated shortly.

@joefioti · 2025-07-08 23:50
@__tinygrad__ Thanks george. we're doing a transition from 1.0 to "2.0" which entails the new IR. stuff in /src is 1.0 and old, so stuff like that reshape fn is old and will be deprecated shortly.

@__tinygrad__ · 2025-07-08 23:55
@joefioti I don't actually expect you to take my advice, I get it, there's probably pieces in tinygrad we shouldn't have rewritten also. But just using tinygrad's frontend fixes stuff like this for free. https://t.co/gGoRr82RN4

@__tinygrad__ · 2025-07-09 04:04
RT @UThree271828: tinygradのコードとかドキュメントとか参考にしながらやってるんだけど、これマジですごいな。読めば読むほど感動が生えてくる。洗練された設計は読んでいて楽しいな。

@__tinygrad__ · 2025-07-09 06:02
MI350X draws 250W at idle?! https://t.co/xSCpcmzNRM

@__tinygrad__ · 2025-07-09 06:02
MI350X draws 250W at idle?! https://t.co/xSCpcmzNRM

@AnushElangovan · 2025-07-09 17:45
@__tinygrad__ pstate tunings are on the way.

@AnushElangovan · 2025-07-09 17:45
@__tinygrad__ pstate tunings are on the way.

@__tinygrad__ · 2025-07-09 18:17
@AnushElangovan MI300X is a bit better but not too much. RDNA3 chips are way better at idle power as percent of total power. https://t.co/IcocRkL2yb

@__tinygrad__ · 2025-07-09 19:07
https://t.co/L8xShZB4cQ

@__tinygrad__ · 2025-07-09 19:07
https://t.co/L8xShZB4cQ

@ChrisFriedler · 2025-07-09 19:39
@__tinygrad__ What do you think the chances are to fulfill the contract with AMD?

@ChrisFriedler · 2025-07-09 19:39
@__tinygrad__ What do you think the chances are to fulfill the contract with AMD?

@__tinygrad__ · 2025-07-09 21:33
@ChrisFriedler Fully, 60%. Partially, 90%.

@__tinygrad__ · 2025-07-09 19:07
https://t.co/L8xShZB4cQ

@deftdawg · 2025-07-09 23:37
@__tinygrad__ AMD’s docker images are ridiculously bloated, I’m convinced they don’t have anyone who knows how to build docker images there… 29GB for a dev image, like f-me https://t.co/WiGUgDg02d

@deftdawg · 2025-07-09 23:37
@__tinygrad__ AMD’s docker images are ridiculously bloated, I’m convinced they don’t have anyone who knows how to build docker images there… 29GB for a dev image, like f-me https://t.co/WiGUgDg02d

@__tinygrad__ · 2025-07-10 01:02
@deftdawg .@AnushElangovan I second this observation, smaller docker please!

@elonmusk · 2025-07-10 05:20
You can cut & paste your entire source code file into the query entry box on https://t.co/EqiIFyHFlo and @Grok 4 will fix it for you! This is what everyone @xAI does. Works better than Cursor.

@__tinygrad__ · 2025-07-10 05:39
RT @SharingPsyche: Got my company laptop and the first thing I did after convincing IT to give me WSL..? Install cuda and ran beautiful_mnist.py @__tinygrad__

@__tinygrad__ · 2025-07-10 06:20
RT @xingyu_liao: 之前在安装 tinygrad 的时候，选择了错误的 cuda 版本，导致一直 generated code ptx 版本有问题，不断反复安装都没有解决，让 claude-code 接手之后，他找了到 cache 成功解决了问题 https://t.co/pFuyqEJ65T

@__tinygrad__ · 2025-07-10 17:37
All of tinygrad, including non existent dependencies, fits in the context window. Seed AI type vibes.

@__tinygrad__ · 2025-07-11 18:02
America has once again put up roadblocks to us getting a visa. And I'm sure it's not just us. Every decision made like this slowly chips away at America's lead in AI. Most Americans have never lived in a country that wasn't rich, but this is the road to that.

@__tinygrad__ · 2025-07-11 18:02
America has once again put up roadblocks to us getting a visa. And I'm sure it's not just us. Every decision made like this slowly chips away at America's lead in AI. Most Americans have never lived in a country that wasn't rich, but this is the road to that.

@darwesh_singh · 2025-07-11 18:08
@__tinygrad__ Very counterproductive to extending America’s lead

@darwesh_singh · 2025-07-11 18:08
@__tinygrad__ Very counterproductive to extending America’s lead

@__tinygrad__ · 2025-07-11 18:10
@darwesh_singh America will only understand what they had once it's gone and won't be coming back.

@__tinygrad__ · 2025-07-11 18:10
@xqcdp Even stupider. They are looking for any excuse.

@__tinygrad__ · 2025-07-11 23:51
@fact0id FlashAttention-4 will be ITAR controlled 😂

@__tinygrad__ · 2025-07-12 16:06
tinygrad is a bet against irreducible complexity. When you look at Triton and similar, so many things are GPU specific. The task in reverse engineering is ignoring the labels, what is that really? Forget the abstractions, search for the perfect program that runs on the hardware.

@__tinygrad__ · 2025-07-12 18:17
This is tinygrad's description of the tensor cores of all the major GPUs. No per GPU dialects, just a spec for what they each are. https://t.co/BEcFdRxFNK

@__tinygrad__ · 2025-07-12 18:17
This is tinygrad's description of the tensor cores of all the major GPUs. No per GPU dialects, just a spec for what they each are. https://t.co/BEcFdRxFNK

@joefioti · 2025-07-12 18:34
@__tinygrad__ Have you found a lot of difficulty handling ampere (warp launched) hopper (warp group launched) and Blackwell (thread launched) in a unified manner?

@joefioti · 2025-07-12 18:34
@__tinygrad__ Have you found a lot of difficulty handling ampere (warp launched) hopper (warp group launched) and Blackwell (thread launched) in a unified manner?

@__tinygrad__ · 2025-07-13 02:58
@joefioti We haven't put any time into those GPUs, mostly because we don't have them and a lot of others are focused there. The 5090s aren't like B200s. But I think it should just work in our current syntax, at least with H100. We have nothing at all talking about a "warp", try it!

@cyberpengk · 2025-07-19 19:08
looking to implement a minimal / educational llm inference engine on top of tinygrad, but pounders the question: what optimization should be implemented by hand, and which part will be supposed to be optimized by @__tinygrad__ itself?

@__tinygrad__ · 2025-07-19 19:58
Check out our complete zero dep 139 line Llama implementation. https://t.co/da8YN7C06G

@__tinygrad__ · 2025-07-19 19:58
Check out our complete zero dep 139 line Llama implementation. https://t.co/da8YN7C06G

@cyberpengk · 2025-07-19 20:21
This is a good reference but the question goes beyond that. e.g. optimizations like chunked prefill, prefix caching, model parallelization, etc. Will Tinygrad be able to search for these optimization in the future, or are they considered app level and still need to be handcoded. I'm guessing the later.

@YouJiacheng · 2025-07-20 13:34
IIUC, this is a greedy (longest match) tokenizer with a vocabulary generated by BPE instead of the true BPE tokenizer. And it doesn't have a pre-tokenizer. https://t.co/euu7YMTdOP

@YouJiacheng · 2025-07-20 13:34
IIUC, this is a greedy (longest match) tokenizer with a vocabulary generated by BPE instead of the true BPE tokenizer. And it doesn't have a pre-tokenizer. https://t.co/euu7YMTdOP

@__tinygrad__ · 2025-07-20 17:15
@YouJiacheng Yea, we need to fix it (with tests). Open to PR.

@cyberpengk · 2025-07-19 20:21
This is a good reference but the question goes beyond that. e.g. optimizations like chunked prefill, prefix caching, model parallelization, etc. Will Tinygrad be able to search for these optimization in the future, or are they considered app level and still need to be handcoded. I'm guessing the later.

@__tinygrad__ · 2025-07-20 17:18
@cyberpengk It's a case by case basis. Even if there's not a way to automatically find them, we should get to a point where they are very easy to write. Like chunked prefill is a scheduling thing, it doesn't change compute. And model parallel already works with a couple lines.

@patrickdevivo · 2025-07-20 17:55
I'm rediscovering python after a few years, and between @astral_sh and @modal_labs, I'm completely blown away by how fun it is. no more env/dep insanity, and practically instant remote code execution 🎉

@b0jle · 2025-07-20 20:12
@patrickdevivo @astral_sh @modal_labs i discovered astral and tinygrad at approximately the same time. changed my views towards python completely

@__tinygrad__ · 2025-07-21 16:02
Every week we have a meeting open to all in our Discord, and it's meeting time now!

@__tinygrad__ · 2025-07-21 18:17
Python is an amazing language. It needs more work on its type system, but no language is close for the ability to express raw ideas with minimal overhead.

@__tinygrad__ · 2025-07-21 18:17
Python is an amazing language. It needs more work on its type system, but no language is close for the ability to express raw ideas with minimal overhead.

@jsuarez · 2025-07-21 18:21
@__tinygrad__ Python is fine. Python *tooling* makes me want to rip out my hair

@jsuarez · 2025-07-21 18:21
@__tinygrad__ Python is fine. Python *tooling* makes me want to rip out my hair

@__tinygrad__ · 2025-07-21 20:26
@jsuarez This is one of the reasons tinygrad doesn't have deps. astral is making some great progress here though.

@__tinygrad__ · 2025-07-21 20:26
@jsuarez This is one of the reasons tinygrad doesn't have deps. astral is making some great progress here though.

@charliermarsh · 2025-07-22 03:22
@__tinygrad__ @jsuarez By the way we’re also working on a type checker and language server, if you have input on the direction you want to see the type system move towards

@__tinygrad__ · 2025-07-21 18:17
Python is an amazing language. It needs more work on its type system, but no language is close for the ability to express raw ideas with minimal overhead.

@migue_dorta · 2025-07-22 04:20
@__tinygrad__ What about performance? It's so bad

@charliermarsh · 2025-07-22 03:22
@__tinygrad__ @jsuarez By the way we’re also working on a type checker and language server, if you have input on the direction you want to see the type system move towards

@__tinygrad__ · 2025-07-22 05:24
@charliermarsh @jsuarez Oh a language server would be great! We were just talking at lunch about how much VSCode lags cause of this. Re type checker: mypy is pretty good, pyright not so much. To start, I'd like run time type checking as first class in the language, but this is on Python maintainers.

@migue_dorta · 2025-07-22 04:20
@__tinygrad__ What about performance? It's so bad

@__tinygrad__ · 2025-07-22 05:27
@migue_dorta If it's slow, it's cause you're doing too much. Delete lines! Improve the algorithm!

@elonmusk · 2025-07-22 17:04
The @xAI goal is 50 million in units of H100 equivalent-AI compute (but much better power-efficiency) online within 5 years

@jacobpeake · 2025-07-22 21:01
what if you could take a deep learning program, search the space of possible equivalent programs, expose the hardware architecture to your search, and find the verifiable optimum program to run on that hardware

@bjornpagen · 2025-07-23 01:00
i love hotz but is he really gonna beat the compiler guy at writing a compiler

@__tinygrad__ · 2025-07-23 04:05
Who thinks tinygrad is going to win?

@__tinygrad__ · 2025-07-23 04:06
We aren't writing a compiler. We're writing a spec that you can pour search and optimization into.

@__tinygrad__ · 2025-07-23 04:06
We aren't writing a compiler. We're writing a spec that you can pour search and optimization into.

@__tinygrad__ · 2025-07-23 04:32
Compilers rely on heuristics to compile Turing complete programs, you have to, the runtime is data dependent. NNs aren't like this. They can be directly optimized to maximize execution speed. And it'll be hardware universal. Just burn compute to make it fast. Bye bye $4T moat.

@__tinygrad__ · 2025-07-26 02:50
No more cardboard! https://t.co/2yk62RIGDO

@__tinygrad__ · 2025-07-26 02:50
No more cardboard! https://t.co/2yk62RIGDO

@HotAisle · 2025-07-26 16:12
@__tinygrad__ Is that plastic sheeting on the top with a bunch of backflow pressure?

@HotAisle · 2025-07-26 16:12
@__tinygrad__ Is that plastic sheeting on the top with a bunch of backflow pressure?

@__tinygrad__ · 2025-07-26 16:31
@HotAisle Yea, it's like that freezer curtain stuff in a meat locker. The hot aisle is slightly lower pressure than the cold, we have big intake/exhaust fans we can control the speed of to control this.

@__tinygrad__ · 2025-07-26 16:36
tinybox green v2 are flying off the shelves, and we haven't even sent an e-mail to the mailing list! We're selling ~4 per week. If you want one, get your order in ASAP and it should ship this week. https://t.co/c5xtdjq8Me

@__tinygrad__ · 2025-07-26 16:36
tinybox green v2 are flying off the shelves, and we haven't even sent an e-mail to the mailing list! We're selling ~4 per week. If you want one, get your order in ASAP and it should ship this week. https://t.co/c5xtdjq8Me

@naprave · 2025-07-26 16:39
@__tinygrad__ I'd love a red one in europe but i'm afraid to even ask about shipping and customs cost:) could even become a distributer if you are interested.

@naprave · 2025-07-26 16:39
@__tinygrad__ I'd love a red one in europe but i'm afraid to even ask about shipping and customs cost:) could even become a distributer if you are interested.

@__tinygrad__ · 2025-07-26 16:39
@naprave You can just order it through the website, shipping is calculated for you. Re: customs, that's between you and your government.

@__tinygrad__ · 2025-07-26 16:36
tinybox green v2 are flying off the shelves, and we haven't even sent an e-mail to the mailing list! We're selling ~4 per week. If you want one, get your order in ASAP and it should ship this week. https://t.co/c5xtdjq8Me

@ChrisFriedler · 2025-07-26 16:48
@__tinygrad__ Curious to see how this scales.

@ChrisFriedler · 2025-07-26 16:48
@__tinygrad__ Curious to see how this scales.

@__tinygrad__ · 2025-07-26 17:29
@ChrisFriedler People are always so obsessed with scale in business. If it happens than it happens, but it's not that important to us. Our goals are to sustain the company and continue to improve the quality of the product.

@__tinygrad__ · 2025-07-28 20:42
RT @beffjezos: Spotted at the @extropic office 👀 Thx to @__tinygrad__ for the TinyBox 🦾🤘 This will power one of our Thermo ML staff's research 👨‍💻🔥 https://t.co/KL1P8rmyQq

@beffjezos · 2025-07-28 20:16
Spotted at the @extropic office 👀 Thx to @__tinygrad__ for the TinyBox 🦾🤘 This will power one of our Thermo ML staff's research 👨‍💻🔥 https://t.co/KL1P8rmyQq

@MickeySteamboat · 2025-07-28 20:58
@beffjezos @extropic @__tinygrad__ @realGeorgeHotz are you doing server or workstation boards in the box?

@beffjezos · 2025-07-28 20:16
Spotted at the @extropic office 👀 Thx to @__tinygrad__ for the TinyBox 🦾🤘 This will power one of our Thermo ML staff's research 👨‍💻🔥 https://t.co/KL1P8rmyQq

@Zenul_Abidin · 2025-07-28 21:42
@beffjezos @extropic @__tinygrad__ @realGeorgeHotz What are the specs of this thing and can it be found on Amazon?

@__tinygrad__ · 2025-07-29 03:17
Anyone know why GLOBAL_LOAD_LDS_B32 was removed from the RDNA3 manual? It's in the left one from Dec 2022, but removed in the right one from Aug 2023. https://t.co/yYpFagHhBP

@__tinygrad__ · 2025-07-29 17:11
Mass production https://t.co/0RmTGcU88w

@MickeySteamboat · 2025-07-28 20:58
@beffjezos @extropic @__tinygrad__ @realGeorgeHotz are you doing server or workstation boards in the box?

@__tinygrad__ · 2025-07-29 17:12
@MickeySteamboat @beffjezos @extropic @realGeorgeHotz Server. They have BMCs and everything.

@Zenul_Abidin · 2025-07-28 21:42
@beffjezos @extropic @__tinygrad__ @realGeorgeHotz What are the specs of this thing and can it be found on Amazon?

@__tinygrad__ · 2025-07-29 17:12
@Zenul_Abidin @beffjezos @extropic @realGeorgeHotz https://t.co/FivkiG9F9s

@__tinygrad__ · 2025-07-29 18:08
tinygrad now supports NVIDIA 5090/4090 without the kernel driver, similar to AMD. It's tested in our CI. https://t.co/QAN0mgqclq

@__tinygrad__ · 2025-07-29 18:08
tinygrad now supports NVIDIA 5090/4090 without the kernel driver, similar to AMD. It's tested in our CI. https://t.co/QAN0mgqclq

@igorpener · 2025-07-29 18:13
@__tinygrad__ Solid! How does perf compare?

@__tinygrad__ · 2025-07-29 18:08
tinygrad now supports NVIDIA 5090/4090 without the kernel driver, similar to AMD. It's tested in our CI. https://t.co/QAN0mgqclq

@j4orz · 2025-07-29 18:19
@__tinygrad__ very nice good sir. 5090s with the comma anytime soon?

@__tinygrad__ · 2025-07-29 18:08
tinygrad now supports NVIDIA 5090/4090 without the kernel driver, similar to AMD. It's tested in our CI. https://t.co/QAN0mgqclq

@JoyDeserver · 2025-07-29 18:24
@__tinygrad__ wait what

@__tinygrad__ · 2025-07-29 18:08
tinygrad now supports NVIDIA 5090/4090 without the kernel driver, similar to AMD. It's tested in our CI. https://t.co/QAN0mgqclq

@musashigarami · 2025-07-29 18:25
@__tinygrad__ You done anything with 9070 xt's?

@igorpener · 2025-07-29 18:13
@__tinygrad__ Solid! How does perf compare?

@__tinygrad__ · 2025-07-29 18:26
@igorpener The same. On AMD, we needed to do this for stability, on NVIDIA it just matches the kernel driver. However, this unlocks a lot of potential for better memory management, and gets you P2P without a kernel mod.

@j4orz · 2025-07-29 18:19
@__tinygrad__ very nice good sir. 5090s with the comma anytime soon?

@__tinygrad__ · 2025-07-29 18:30
@j4orz Added a $1000 bounty for "3090 or 4090 or 5090 works over USB 3.0 with our USB stack"

@musashigarami · 2025-07-29 18:25
@__tinygrad__ You done anything with 9070 xt's?

@__tinygrad__ · 2025-07-29 18:30
@musashigarami Yes, our AMD driver works on that card and is tested in our CI also.

@JoyDeserver · 2025-07-29 18:24
@__tinygrad__ wait what

@__tinygrad__ · 2025-07-29 18:31
@JoyDeserver Right? Sort of unbelievable, but it's all real. See tinygrad/runtime/support/nv

@__tinygrad__ · 2025-07-29 18:43
Any OS X experts out there? Since they are in userspace, our AMD/NVIDIA drivers run on Mac. If someone can figure out how to map address ranges on Thunderbolt, external GPUs should just work.

@qubitium · 2025-07-29 19:00
Incredibly disruptive. Tiny's motto appears to treat compute at the lowest level a normal, nothing special, commodity. Imagine an fully open source instruction set for gpu compute.

@__tinygrad__ · 2025-07-29 18:43
Any OS X experts out there? Since they are in userspace, our AMD/NVIDIA drivers run on Mac. If someone can figure out how to map address ranges on Thunderbolt, external GPUs should just work.

@BderKhan · 2025-07-29 22:01
@__tinygrad__ Tiny DriverKit DEXT: claim TB GPU, grant system.pcie.device, expose BARs via IOPCIDevice::createMappingInTask() to user-space. Thunderbolt already tunnels PCIe, so tinygrad AMD/NV code should work. See Apple PCIe-TB DriverKit doc & Asahi DART. Happy to help

@BderKhan · 2025-07-29 22:01
@__tinygrad__ Tiny DriverKit DEXT: claim TB GPU, grant system.pcie.device, expose BARs via IOPCIDevice::createMappingInTask() to user-space. Thunderbolt already tunnels PCIe, so tinygrad AMD/NV code should work. See Apple PCIe-TB DriverKit doc & Asahi DART. Happy to help

@__tinygrad__ · 2025-07-29 23:14
@BderKhan Oh interesting, so it doesn't have to be a full KEXT! Put up a $1000 bounty, "AMD or NVIDIA GPU working over Thunderbolt with ADT-UT3G on my M3 Macbook Pro running macOS"

@__tinygrad__ · 2025-07-30 00:00
These devices made by a $4T market cap company are not that complex. I know it's counterintuitive, but the first step to building your own hardware is building a complete software stack for everyone else's.

@__tinygrad__ · 2025-07-29 23:15
Put up a $1000 bounty: "AMD or NVIDIA GPU working over Thunderbolt with ADT-UT3G on my M3 Macbook Pro running macOS" It might only be like 50 lines of glue code with PCIDriverKit.

@MickeySteamboat · 2025-07-30 00:30
@__tinygrad__ $1,000? More like $10,000,000. /:

@MickeySteamboat · 2025-07-30 00:30
@__tinygrad__ $1,000? More like $10,000,000. /:

@__tinygrad__ · 2025-07-30 00:53
@MickeySteamboat lol $1,000 is generous for this. it's maybe even possible an LLM could write the code.

@__tinygrad__ · 2025-07-30 17:18
RT @ekzhang1: in 2025: kernel drivers are written in Python (lol), and naturally they make HTTP requests to https://t.co/HWBfu1ahhZ to fetch header files during runtime jokes aside, this is super cool and a nice reference for GPU MMIO device calls, didn’t know mmap does this in userspace https://t.co/pD9E4DNI1J

@__tinygrad__ · 2025-08-01 04:27
tinygrad's profiler is getting very good. Just run with VIZ=1 and click Profiler in the top left. Here's three steps of Llama training on MI350X https://t.co/Td36TwaQ98

@__tinygrad__ · 2025-08-01 04:27
tinygrad's profiler is getting very good. Just run with VIZ=1 and click Profiler in the top left. Here's three steps of Llama training on MI350X https://t.co/Td36TwaQ98

@__tinygrad__ · 2025-08-01 04:30
All of the kernels are clickable and will show you the code and the process used to generate them, allowing you to step through the rewrites. Soon we'll have instruction level profiling too. https://t.co/ttvSw5PXTN

@anemll · 2025-08-01 15:40
Blackwell is detected and memory mapped. Next, bind @__tinygrad__ PyTorch code for NVIDIA to run on Apple Silicon with UT3G. 🤞 https://t.co/2gEzCRSOc6

@__tinygrad__ · 2025-08-01 17:50
Awesome! If you've mapped the BARs, you are 80% of the way there.

@__tinygrad__ · 2025-08-01 17:50
Awesome! If you've mapped the BARs, you are 80% of the way there.

@anemll · 2025-08-01 21:26
@__tinygrad__ Update: Need to add intermediate helper service to talk with DriverKit client. dkmmio.dylib works when used in signed app, but we need standalone signed/entitled process. https://t.co/ewpfTVyYbk

@anemll · 2025-08-01 21:26
@__tinygrad__ Update: Need to add intermediate helper service to talk with DriverKit client. dkmmio.dylib works when used in signed app, but we need standalone signed/entitled process. https://t.co/ewpfTVyYbk

@__tinygrad__ · 2025-08-01 21:28
@anemll You should be able to sign Python with the entitlement.

@__tinygrad__ · 2025-08-01 22:36
We offer 0 customization for the tinybox. We also don't jump through hoops like supplier portals. Here's why: By selling the same box in the same way to all, we invest heavy into procedure and testing. Would rather miss out on 20% of customers and maintain a quality experience.

@__tinygrad__ · 2025-08-01 22:36
We offer 0 customization for the tinybox. We also don't jump through hoops like supplier portals. Here's why: By selling the same box in the same way to all, we invest heavy into procedure and testing. Would rather miss out on 20% of customers and maintain a quality experience.

@surf_da_earf · 2025-08-01 22:43
@__tinygrad__ may i get 1 customization

@surf_da_earf · 2025-08-01 22:43
@__tinygrad__ may i get 1 customization

@__tinygrad__ · 2025-08-01 22:44
@surf_da_earf In order to keep prices low and quality high, we don't offer any customization to the box or ordering process. Of course, after you buy the tinybox, it's yours and you are welcome to do whatever you want with it!

@__tinygrad__ · 2025-08-01 23:43
Working on a big refactor now. We've built a minimal language (UOp) to express all neural networks and kernels in. Onward to optimization the likes of which haven't been seen in any major framework to date, enabled by the language simplicity (triangle from Halide PhD) https://t.co/aPm9Z1L31z

@__tinygrad__ · 2025-08-01 21:28
@anemll You should be able to sign Python with the entitlement.

@anemll · 2025-08-02 00:32
@__tinygrad__ Found DriverKit entitlements to allow all. Python is running now. Struggling with GSP loading issues, Linux code is in the way.

@anemll · 2025-08-02 00:32
@__tinygrad__ Found DriverKit entitlements to allow all. Python is running now. Struggling with GSP loading issues, Linux code is in the way.

@__tinygrad__ · 2025-08-02 04:05
@anemll Which Linux code? It should all be pretty well abstracted in the PCIDevice class. Just write a Mac version of that, read_config/write_config/map_bar

@anemll · 2025-08-02 05:27
@__tinygrad__ https://t.co/2e5UwmL30A reserve_hugepages system_paddrs pagemap

@anemll · 2025-08-02 06:19
@__tinygrad__ Is the expectation to extend support/NV for the macOS case (as opposed to support/mac)? I'm not very familiar with the codebase, tbh.

@anemll · 2025-08-02 06:19
@__tinygrad__ Is the expectation to extend support/NV for the macOS case (as opposed to support/mac)? I'm not very familiar with the codebase, tbh.

@__tinygrad__ · 2025-08-02 08:26
@anemll For the bounty, yes. It should only be like 50-100 lines.

@__tinygrad__ · 2025-08-02 20:43
WMMA WMMA WMMA WMMA https://t.co/ll8j9JrP6b

@__tinygrad__ · 2025-08-02 23:29
Finally, real flash attention. With the new indexing, it's completely automatic and generic, see TestFuse.test_flash_attention. ENDRANGE is STORE+LOAD, and you end a range if the children of a UOp mismatch on that axis. https://t.co/Fbmb88tTKV

@EitanTurok · 2025-08-03 20:34
I annotated the tinygrad flash attention kernel to make sure I understand it. automatically generating this GENERICALLY is pretty cool! https://t.co/efbgvNcppF

@kayos_o · 2025-08-03 21:12
I swear to God I’m not making fun of this dude but NO ONE HAS EVER LOOKED LIKE THEY SUFFERED LIKE THIS IN HUMAN HISTORY https://t.co/JekIbl1F0a

@DaarkMagiciann · 2025-08-03 21:31
@kayos_o https://t.co/jzIXo6EPXB

@tenderizzation · 2025-08-04 05:19
CUTLASS when you try to write a matmul that doesn't have the exact arch-specific features, scaling factor swizzling format, threadblock cluster shape, tile shape, alignment, dtype, stride, workspace size, and arch-conditional compile flag combination it expects

@__tinygrad__ · 2025-08-04 18:23
RT @EitanTurok: I annotated the tinygrad flash attention kernel to make sure I understand it. automatically generating this GENERICALLY is pretty cool! https://t.co/efbgvNcppF

@__tinygrad__ · 2025-08-04 18:25
@EitanTurok ridx6 and ridx5 should be fused into one loop to match canonical flash attention. But that can be done in a later rewrite.

@__tinygrad__ · 2025-08-04 18:29
@EitanTurok Also, Q@K should be computed using tensor cores, for another later pass as we add things back in.

@__tinygrad__ · 2025-08-04 18:29
@EitanTurok Also, Q@K should be computed using tensor cores, for another later pass as we add things back in.

@__tinygrad__ · 2025-08-04 18:34
@EitanTurok Oh, qk@v should be tensor cores too. Gonna be fun to figure out these patterns, but I think the new stuff should support it. Flash attention is a phenomenal test of automated codegen.

@__tinygrad__ · 2025-08-04 18:25
@EitanTurok ridx6 and ridx5 should be fused into one loop to match canonical flash attention. But that can be done in a later rewrite.

@EitanTurok · 2025-08-04 18:35
@__tinygrad__ yes, that is a redundancy I see here ridx6 and ridx5 are both for loops over the 16 keys and can be computed in one single for loop another simple rewrite rule to add is to simplify adding 0 to the accumulators, i.e. *(acc + 0) = 0.0f; should be *acc = 0.0f;

@tenderizzation · 2025-08-04 05:19
CUTLASS when you try to write a matmul that doesn't have the exact arch-specific features, scaling factor swizzling format, threadblock cluster shape, tile shape, alignment, dtype, stride, workspace size, and arch-conditional compile flag combination it expects

@__tinygrad__ · 2025-08-04 18:36
@tenderizzation this is why tinygrad

@EitanTurok · 2025-08-04 18:35
@__tinygrad__ yes, that is a redundancy I see here ridx6 and ridx5 are both for loops over the 16 keys and can be computed in one single for loop another simple rewrite rule to add is to simplify adding 0 to the accumulators, i.e. *(acc + 0) = 0.0f; should be *acc = 0.0f;

@__tinygrad__ · 2025-08-04 18:36
@EitanTurok No need, the generated machine code is the same.

@EitanTurok · 2025-08-03 20:34
I annotated the tinygrad flash attention kernel to make sure I understand it. automatically generating this GENERICALLY is pretty cool! https://t.co/efbgvNcppF

@almeida_dril · 2025-08-05 09:51
@EitanTurok Nice analysis! The generic kernel generation is indeed impressive - tinygrad's approach to automatic optimization keeps getting better.

@__tinygrad__ · 2025-08-05 15:33
Someday it will be all end to end. A fancy search process (or AI agent, if you prefer that term) optimizing the runtime of your full training run on whatever cluster of hardware you give it.

@__tinygrad__ · 2025-08-05 15:39
This idea of running CUDA code on X other hardware comes up over and over. It doesn't work, CUDA is too low level of a language. High performance CUDA code isn't even portable between different generations of NVIDIA hardware.

@__tinygrad__ · 2025-08-05 22:24
tinygrad tracks timing for scheduling, codegen, and compilation on the same timeline as kernels running on device. just a VIZ=1 away https://t.co/8pvFIyxdpN

@__tinygrad__ · 2025-08-06 16:44
GPUs provide tons of profiling information at the assembly level. A lesson from hacking: dynamic program analysis > static program analysis. https://t.co/0WQ7tBLPjB

@__tinygrad__ · 2025-08-06 20:35
New $1,000 bounty: "gpt-oss 20B running on M3 Max at 100+ tok/s in tinygrad.apps.llm"

@__tinygrad__ · 2025-08-06 20:35
New $1,000 bounty: "gpt-oss 20B running on M3 Max at 100+ tok/s in tinygrad.apps.llm"

@__tinygrad__ · 2025-08-06 20:38
It should be an hour to get it running, and with the new profiling tools (VIZ=1) you should be able to make progress quickly on speed. tinygrad.apps.llm has no deps, let's keep it that way.

@tunguz · 2025-08-06 22:07
And they never will be unless they MASSIVELY invest in the software to make it happen. Which at this point would require an equivalent of frontal lobotomy to radically change their internal culture. https://t.co/OxbUoGjAWP

@__tinygrad__ · 2025-08-07 14:45
AMD is on a good path. Since signing the contract with them, we're putting the majority of our resources on MI350X software. And I'm sure it's not just us they are working with in this way. Open source will win.

@__tinygrad__ · 2025-08-07 14:45
AMD is on a good path. Since signing the contract with them, we're putting the majority of our resources on MI350X software. And I'm sure it's not just us they are working with in this way. Open source will win.

@__tinygrad__ · 2025-08-07 14:51
We have a contract with AMD for the next year. We won't be able to solve everything in that time, but right now betting against AMD software is betting against us. Our process isn't fast, but when it succeeds, the simplicity and full stack optimization abilities will dominate.

@__tinygrad__ · 2025-08-07 14:51
We have a contract with AMD for the next year. We won't be able to solve everything in that time, but right now betting against AMD software is betting against us. Our process isn't fast, but when it succeeds, the simplicity and full stack optimization abilities will dominate.

@__tinygrad__ · 2025-08-07 14:53
If you want to help, join our Discord. Prove yourself and we'll give you MI350X access. After a few more refactors land and our profiler is finished, we're ready to start optimizing GEMMs at the machine code level. The ISA is open, unlike NVIDIA. https://t.co/Fkgx0jQsVN

@__tinygrad__ · 2025-08-07 14:53
If you want to help, join our Discord. Prove yourself and we'll give you MI350X access. After a few more refactors land and our profiler is finished, we're ready to start optimizing GEMMs at the machine code level. The ISA is open, unlike NVIDIA. https://t.co/Fkgx0jQsVN

@BSchultzer · 2025-08-07 17:06
@__tinygrad__ I would love to see GEMM optimized by datatype to fit in L1, L2 and L3 cache. The biggest hurdle with this is knowing the cache size. If you know the cache size and the datatype then it’s trivial.

@BSchultzer · 2025-08-07 17:06
@__tinygrad__ I would love to see GEMM optimized by datatype to fit in L1, L2 and L3 cache. The biggest hurdle with this is knowing the cache size. If you know the cache size and the datatype then it’s trivial.

@__tinygrad__ · 2025-08-07 17:11
@BSchultzer Don't try to explicitly know the cache size, just search with the hardware in the loop.

@GregHBurnham · 2025-08-08 13:03
Careful not to cut yourself on the jagged frontier https://t.co/buJGgJ6baI

@__tinygrad__ · 2025-08-08 15:42
Everyone knows about Flash Attention. But do you know about Flash GEMM? This code computes (A @ B) @ C on NxN matrices with N intermediates and no recomputation. If you don't use a BLAS library, you don't need to materialize the intermediate matrix. https://t.co/L1tjUNSprE

@__tinygrad__ · 2025-08-08 15:42
Everyone knows about Flash Attention. But do you know about Flash GEMM? This code computes (A @ B) @ C on NxN matrices with N intermediates and no recomputation. If you don't use a BLAS library, you don't need to materialize the intermediate matrix. https://t.co/L1tjUNSprE

@PoTheTatoMan · 2025-08-08 15:48
@__tinygrad__ Holy shit I wish I understood what you just said.

@PoTheTatoMan · 2025-08-08 15:48
@__tinygrad__ Holy shit I wish I understood what you just said.

@__tinygrad__ · 2025-08-08 15:53
@PoTheTatoMan Normally use 1048576 RAM. Now use 1024 RAM. Less RAM number means you can use faster RAM. Faster RAM means model run faster!

@__tinygrad__ · 2025-08-08 15:42
Everyone knows about Flash Attention. But do you know about Flash GEMM? This code computes (A @ B) @ C on NxN matrices with N intermediates and no recomputation. If you don't use a BLAS library, you don't need to materialize the intermediate matrix. https://t.co/L1tjUNSprE

@aryanvs_ · 2025-08-08 17:28
@__tinygrad__ @grok is this what's commonly referred to as "b2b gemm"? is flash gemm commonly used terminology?

@__tinygrad__ · 2025-08-08 15:42
Everyone knows about Flash Attention. But do you know about Flash GEMM? This code computes (A @ B) @ C on NxN matrices with N intermediates and no recomputation. If you don't use a BLAS library, you don't need to materialize the intermediate matrix. https://t.co/L1tjUNSprE

@ritteradam · 2025-08-08 17:29
@__tinygrad__ Cool, generally I don't see too many reasons for multiplying by 2 weight matrices, except if they are low rank representations. Have you seen this trick used in LoRA or RAM optimized (low rank compressed) neutral networks?

@ritteradam · 2025-08-08 17:29
@__tinygrad__ Cool, generally I don't see too many reasons for multiplying by 2 weight matrices, except if they are low rank representations. Have you seen this trick used in LoRA or RAM optimized (low rank compressed) neutral networks?

@__tinygrad__ · 2025-08-08 18:23
@ritteradam You can obviously put an activation function between them.

@aryanvs_ · 2025-08-08 17:28
@__tinygrad__ @grok is this what's commonly referred to as "b2b gemm"? is flash gemm commonly used terminology?

@__tinygrad__ · 2025-08-08 18:33
@aryanvs_ @grok Oh cool. I (and Gemini) didn't know it had a name beyond "fusion." Need to spend more time reading CUTLASS examples.

@GregHBurnham · 2025-08-08 13:03
Careful not to cut yourself on the jagged frontier https://t.co/buJGgJ6baI

@__tinygrad__ · 2025-08-08 18:40
@GregHBurnham And this thing supposedly gets IMO gold?

@jeremyphoward · 2025-08-08 21:57
@__tinygrad__ Love it! :)

@__tinygrad__ · 2025-08-08 22:16
@jeremyphoward The four red blocks are the ffn_norm, ffn_gate, ffn_up, and ffn_down, this is actually kind of a perfect test of good fusion. https://t.co/DEhsbeT0fl

@__tinygrad__ · 2025-08-08 23:37
RT @jeremyphoward: This is burying the lede a bit, which is that with a minor change you can also do ReLU(A @ B) @ C on NxN matrices with N intermediates and no recomputation. (aka: a neural network.)

@__tinygrad__ · 2025-08-08 22:16
@jeremyphoward The four red blocks are the ffn_norm, ffn_gate, ffn_up, and ffn_down, this is actually kind of a perfect test of good fusion. https://t.co/DEhsbeT0fl

@__tinygrad__ · 2025-08-09 01:23
@jeremyphoward https://t.co/GHllNelhQn

@__tinygrad__ · 2025-08-09 01:29
Is your transformer library fusing the FFN into one "flash" kernel? I think there's massive gains possible, curious what the common practice and research SOTA is? https://t.co/5MzsdHkTa1

@__tinygrad__ · 2025-08-09 01:29
Is your transformer library fusing the FFN into one "flash" kernel? I think there's massive gains possible, curious what the common practice and research SOTA is? https://t.co/5MzsdHkTa1

@KBlueleaf · 2025-08-09 05:51
@__tinygrad__ I know some library make ffn fused like xformers/liger kernel but I also find that torch.compile provide near theoritical flops/mem rw bandwidth as well

@KBlueleaf · 2025-08-09 05:51
@__tinygrad__ I know some library make ffn fused like xformers/liger kernel but I also find that torch.compile provide near theoritical flops/mem rw bandwidth as well

@__tinygrad__ · 2025-08-09 06:27
@KBlueleaf Re: memory, yea for BS=1 with a KV cache, so many things are close to theoretical, fusion won't help there. But theoretical FLOPS as in 100% MFU for training or even prefill? No way!

@Jonathan_Blow · 2025-08-10 16:05
The people who would historically be excited about a new operating system can't do that any more, because everyone is too helpless to even conceive of a new OS. So they have to get excited about a mildly different arrangement of bloatware from That OS From 35 Years Ago. But as long as you give it a Cool Name, everything is good.

@__tinygrad__ · 2025-08-10 17:53
tinygrad is an operating system. It actually wouldn't be that much work to remove Linux and still have working AMD and NVIDIA GPUs.

@__tinygrad__ · 2025-08-10 17:53
tinygrad is an operating system. It actually wouldn't be that much work to remove Linux and still have working AMD and NVIDIA GPUs.

@uuuvn_ · 2025-08-10 19:34
@__tinygrad__ You'll have to write and maintain quite a lot of c code for cpython to work properly on bare metal and to expose all the stuff you need. It can be done, but i don't see how you gain much compared to something like a minimal linux with statically linked python in initramfs

@uuuvn_ · 2025-08-10 19:34
@__tinygrad__ You'll have to write and maintain quite a lot of c code for cpython to work properly on bare metal and to expose all the stuff you need. It can be done, but i don't see how you gain much compared to something like a minimal linux with statically linked python in initramfs

@__tinygrad__ · 2025-08-10 22:05
@uuuvn_ Yea, this is why we haven't done it. Our plan for our CLOUD machines is a minimal Linux that PXE boots and runs off a ramdisk. Linux isn't the problem, having state is.

@__tinygrad__ · 2025-08-11 17:28
.@AMD @AnushElangovan Can you open source rocprof-trace-decoder? This repo is just a Linux binary: https://t.co/qZmgWlgOxK We are using it to add RGP like functionality to VIZ: https://t.co/MzP46DHsE6 OSS will make it portable to Mac and (hopefully) have a header with the API.

@__tinygrad__ · 2025-08-11 17:28
.@AMD @AnushElangovan Can you open source rocprof-trace-decoder? This repo is just a Linux binary: https://t.co/qZmgWlgOxK We are using it to add RGP like functionality to VIZ: https://t.co/MzP46DHsE6 OSS will make it portable to Mac and (hopefully) have a header with the API.

@00x1337 · 2025-08-11 17:35
@__tinygrad__ @AMD @AnushElangovan It’s almost like @amd doesn’t want to compete in the AI space, truly mind blowing

@00x1337 · 2025-08-11 17:35
@__tinygrad__ @AMD @AnushElangovan It’s almost like @amd doesn’t want to compete in the AI space, truly mind blowing

@__tinygrad__ · 2025-08-11 18:10
@00x1337 @AMD @AnushElangovan Yea I wouldn't jump to that conclusion from this. This is a pretty obscure library, hopefully just an oversight that it wasn't open sourced. Just having the real instruction set documented/in LLVM is a big step up from NVIDIA.

@__tinygrad__ · 2025-08-12 00:06
conv2d(3x3)+relu+batchnorm+max_pool2d(2x2) Once you get the fusion rules right, you stop with the false idea that kernels should have anything to do with operations, and realize it's all about data locality. https://t.co/DUUTStKz74

@__tinygrad__ · 2025-08-11 17:28
.@AMD @AnushElangovan Can you open source rocprof-trace-decoder? This repo is just a Linux binary: https://t.co/qZmgWlgOxK We are using it to add RGP like functionality to VIZ: https://t.co/MzP46DHsE6 OSS will make it portable to Mac and (hopefully) have a header with the API.

@AnushElangovan · 2025-08-12 05:22
@__tinygrad__ @AMD let me check. at the minimum there can be a macOS lib.

@AnushElangovan · 2025-08-12 05:22
@__tinygrad__ @AMD let me check. at the minimum there can be a macOS lib.

@__tinygrad__ · 2025-08-12 16:10
@AnushElangovan @AMD the real dream is the source + documentation for the whole SQTT format. tbh we will probably end up reverse engineering it anyway without docs. but source/docs would be very nice!

@__tinygrad__ · 2025-08-12 17:43
tinygrad's extreme simplicity allows us to move faster than everyone else. We're quickly closing in on the state of the art, then it's only more winning from there!

@__tinygrad__ · 2025-08-13 19:08
RT @ninoristeski: I love @__tinygrad__ because it helps you be a better developer and engineer. Also, its one of the best places to learn how deep learning and gpu's work. I wrote a short article on my first 4 merged PRs to tinygrad and hope it helps someone contribute and learn a lot from this community and codebase. https://t.co/vfK9FFC4Qf

@__tinygrad__ · 2025-08-14 06:25
RT @kaiostephens: You can buy a tiny box green from @__tinygrad__, rent out the GPU's, RAM, and CPU for $56.23 USD per day, and pay back the entire thing in a little over a year. tiny box red likely has more perf/$ and higher yield but harder to rent out from lack of support. https://t.co/P2SaV4NodM

@__tinygrad__ · 2025-08-14 23:07
All outstanding orders will ship by the end of the week. We're building a whole batch of green boxes (pic related), order today and you'll have yours soon! (we also have just one red box left. you should buy it, it has been waiting a while to go to a good home) https://t.co/nNt6YTOimh

@__tinygrad__ · 2025-08-14 23:07
All outstanding orders will ship by the end of the week. We're building a whole batch of green boxes (pic related), order today and you'll have yours soon! (we also have just one red box left. you should buy it, it has been waiting a while to go to a good home) https://t.co/nNt6YTOimh

@tkanarsky · 2025-08-14 23:10
@__tinygrad__ so this is where all the 5090s went

@tkanarsky · 2025-08-14 23:10
@__tinygrad__ so this is where all the 5090s went

@__tinygrad__ · 2025-08-14 23:11
@tkanarsky Want 4 of them? Buy a tinybox green v2!

@__tinygrad__ · 2025-08-15 16:04
When tinygrad's spec is done, I want it to feel like it came from THE BOOK. This isn't engineering, this is reverse engineering of Nature. https://t.co/2uq6anxNxf

@Jonathan_Blow · 2025-08-15 18:13
Intel seems determined to pull ahead in the consumer GPU race... If that race is, how big is the driver download... https://t.co/wquDCnTMoQ

@__tinygrad__ · 2025-08-15 23:26
How many tinygrad's can you fit in an Intel GPU driver? tinygrad includes a full AMD and NVIDIA driver, Triton functionality, an MLIR replacement, a library with very similar functionality to PyTorch, and now, an ONNX runtime.

@__tinygrad__ · 2025-08-16 22:26
Now with loop fusion and no recomputation. If you can build a "compiler" that flash attention just falls out of, imagine what else it could fuse. https://t.co/5As1xSxN2x

@__tinygrad__ · 2025-08-16 22:35
@Ayghri There's like 6 more tricks you have to stack on top of this to make things fast in practice. But the first step is getting the data locality right.

@__tinygrad__ · 2025-08-16 22:26
Now with loop fusion and no recomputation. If you can build a "compiler" that flash attention just falls out of, imagine what else it could fuse. https://t.co/5As1xSxN2x

@visejak · 2025-08-16 22:43
@__tinygrad__ is there a rule of thumb of when kernel fusion improves perf vs when you start to get downsides?

@visejak · 2025-08-16 22:43
@__tinygrad__ is there a rule of thumb of when kernel fusion improves perf vs when you start to get downsides?

@__tinygrad__ · 2025-08-16 23:18
@visejak There's never downsides if you are avoiding recomputation and improving data locality.

@ekzhang1 · 2025-08-17 15:47
so this is awkward… I can get a consistent macOS kernel panic after ~8 epochs of my convnet training code in jax-js's WebGPU backend — well that's one hell of a bug to figure out never thought a web page would trigger a kernel panic in 2025, let alone a library I made myself 😅 https://t.co/POrMmnkbUq

@tszzl · 2025-08-18 06:44
george hotz tried to fix twitter search and decided it would be easier to take down nvidia

@fangpenlin · 2025-08-18 17:39
Just published my new article, "Marketplace": my first attempt at efficient GPU training without backprop 😄🎉 I've been considering eliminating backprop for a while. I had an idea, experimented for two weeks, and it worked! Here's how it works: https://t.co/4caGKEmL5g

@__tinygrad__ · 2025-08-18 18:23
RT @fangpenlin: This whole project is done by using @__tinygrad__ by the way. As I am not using backprop for training, it would be way harder and slower to try out algorithm if it wasn't the flexibility of lazy eval compute graph + JIT provided by Tinygrad. Huge thanks 🙏

@__tinygrad__ · 2025-08-18 18:27
take down NVIDIA? 🤨 we ❤️ NVIDIA! tinybox green v2 with 4x5090 is shipping now

@__tinygrad__ · 2025-08-18 18:27
take down NVIDIA? 🤨 we ❤️ NVIDIA! tinybox green v2 with 4x5090 is shipping now

@GeorgeLutas1 · 2025-08-18 18:29
@__tinygrad__ I do look forward to when competition can be brought back in the GPU space so we can see arbitraging away of profit margins. One day. One day.

@GeorgeLutas1 · 2025-08-18 18:29
@__tinygrad__ I do look forward to when competition can be brought back in the GPU space so we can see arbitraging away of profit margins. One day. One day.

@__tinygrad__ · 2025-08-18 18:32
@GeorgeLutas1 AMD is closer to NVIDIA than most people think, we're working hard on the software gap. similar to Intel with CPUs, if NVIDIA stumbles for a generation AMD can capture majority market share. this is easier than gaming for market share to shift.

@__tinygrad__ · 2025-08-18 18:32
@GeorgeLutas1 AMD is closer to NVIDIA than most people think, we're working hard on the software gap. similar to Intel with CPUs, if NVIDIA stumbles for a generation AMD can capture majority market share. this is easier than gaming for market share to shift.

@GeorgeLutas1 · 2025-08-18 18:40
@__tinygrad__ Well, I'm glad of it. I'll go back and take a look at what AMD's embedded offerings are. That's the bulk of my needs historically, and I've not looked at them in a while.

@GeorgeLutas1 · 2025-08-18 18:40
@__tinygrad__ Well, I'm glad of it. I'll go back and take a look at what AMD's embedded offerings are. That's the bulk of my needs historically, and I've not looked at them in a while.

@__tinygrad__ · 2025-08-18 18:53
@GeorgeLutas1 Oh they aren't good. NVIDIA's are good but they are super overpriced. Qualcomm is the best option here (and pretty well supported in tinygrad)

@AnushElangovan · 2025-08-12 05:22
@__tinygrad__ @AMD let me check. at the minimum there can be a macOS lib.

@__tinygrad__ · 2025-08-18 20:06
@AnushElangovan @AMD any update on this?

@ekzhang1 · 2025-08-17 15:47
so this is awkward… I can get a consistent macOS kernel panic after ~8 epochs of my convnet training code in jax-js's WebGPU backend — well that's one hell of a bug to figure out never thought a web page would trigger a kernel panic in 2025, let alone a library I made myself 😅 https://t.co/POrMmnkbUq

@__tinygrad__ · 2025-08-18 20:24
@ekzhang1 The Mac GPU drivers are not that good. It's kind of shocking they don't have better fuzzers, I suspect some of these bugs might be 0-days.

@__tinygrad__ · 2025-08-18 20:06
@AnushElangovan @AMD any update on this?

@AnushElangovan · 2025-08-18 20:32
@__tinygrad__ @AMD Would this help unblock https://t.co/0Eo7h8vUPv

@AnushElangovan · 2025-08-18 20:32
@__tinygrad__ @AMD Would this help unblock https://t.co/0Eo7h8vUPv

@__tinygrad__ · 2025-08-18 23:54
@AnushElangovan @AMD Oh cool, we will add Mac support to our profiler now! What's the issue with open sourcing it? I can't imagine there's valuable IP in a parser that transforms one format to another.

@AnushElangovan · 2025-08-18 20:32
@__tinygrad__ @AMD Would this help unblock https://t.co/0Eo7h8vUPv

@__tinygrad__ · 2025-08-19 00:24
RT @SemiAnalysis_: When will rocprof-trace-decoder and MES be open source? AMD claims that all the public repo is the same as their internal repo. For the most part this is true, but for various libraries they have internal repos that have closed source features and some libraries have outright haven't been open source yet.

@__tinygrad__ · 2025-08-19 00:27
@SemiAnalysis_ It's always important to keep in mind how much better this is than NVIDIA though, AMD has their instruction set (nice PDF) and most of their registers open (if you know where to look).

@__tinygrad__ · 2025-08-19 00:28
@SemiAnalysis_ And our success or failure of the Llama 405B contract won't be due to anything being closed source, the ball really is all in our court.

@__tinygrad__ · 2025-08-19 00:28
@SemiAnalysis_ And our success or failure of the Llama 405B contract won't be due to anything being closed source, the ball really is all in our court.

@__tinygrad__ · 2025-08-19 00:32
@SemiAnalysis_ Like it's a bit annoying the MEC is closed and we don't see how to fast sync the 8 XCDs from PM4 (best way we have found is an atomic in HBM). But we just wrote the AQL to match HIP, not too big of a deal. This won't be make or break.

@max_paperclips · 2025-08-19 02:58
Reading through the @__tinygrad__ source with all the work they've done since the last time I looked and realising everything else is super bloated. Imagine needing pytorch and triton and accelerate and a thousand other flaky things etc, just make your own JIT nubs, it's like 1000 lines of code

@max_paperclips · 2025-08-19 02:58
Reading through the @__tinygrad__ source with all the work they've done since the last time I looked and realising everything else is super bloated. Imagine needing pytorch and triton and accelerate and a thousand other flaky things etc, just make your own JIT nubs, it's like 1000 lines of code

@max_paperclips · 2025-08-19 03:00
Just support every backend, just make multigpu work easily, just git gud already jfc they actually kind cooked

@__tinygrad__ · 2025-08-19 03:40
Think we are going to win yet? There's a long road, but if tinygrad succeeds, it won't just be Tensor libraries that feel super bloated, it will be almost all of software.

@__tinygrad__ · 2025-08-19 03:40
Think we are going to win yet? There's a long road, but if tinygrad succeeds, it won't just be Tensor libraries that feel super bloated, it will be almost all of software.

@kierank_ · 2025-08-19 03:42
@__tinygrad__ You'll win on a technical level. But winning mindshare is a different game as you have to cater to the lowest common denominator outsourced engineer. What will be the hook to move from pytorch?

@max_paperclips · 2025-08-19 03:00
Just support every backend, just make multigpu work easily, just git gud already jfc they actually kind cooked

@__tinygrad__ · 2025-08-19 03:43
@max_paperclips There's still a few more pieces, tinygrad's scheduler isn't great yet, and some other parts need a whole lot of cleanup. But when it's finished it will sparkle.

@kierank_ · 2025-08-19 03:42
@__tinygrad__ You'll win on a technical level. But winning mindshare is a different game as you have to cater to the lowest common denominator outsourced engineer. What will be the hook to move from pytorch?

@__tinygrad__ · 2025-08-19 03:45
@kierank_ 2x faster training with 10x less bugs. Oh, and complete and total support for a huge diverse set of hardware.

@__tinygrad__ · 2025-08-19 03:40
Think we are going to win yet? There's a long road, but if tinygrad succeeds, it won't just be Tensor libraries that feel super bloated, it will be almost all of software.

@Leik0w0 · 2025-08-19 03:47
@__tinygrad__ I don’t think you’ll be able to always have automatic derivation of the best kernels. Especially with gpus getting more and more complicated to get the flops out of

@Leik0w0 · 2025-08-19 03:47
@__tinygrad__ I don’t think you’ll be able to always have automatic derivation of the best kernels. Especially with gpus getting more and more complicated to get the flops out of

@__tinygrad__ · 2025-08-19 03:54
@Leik0w0 https://t.co/FM1INp8Yqn

@max_paperclips · 2025-08-19 02:58
Reading through the @__tinygrad__ source with all the work they've done since the last time I looked and realising everything else is super bloated. Imagine needing pytorch and triton and accelerate and a thousand other flaky things etc, just make your own JIT nubs, it's like 1000 lines of code

@EitanTurok · 2025-08-19 03:58
@max_paperclips @__tinygrad__ exactly -- for example, the fact that their gradient function is 65 lines long and nothing more, is just crazy!

@__tinygrad__ · 2025-08-19 04:02
It's good because it's like the 5th time we rewrote it. https://t.co/9qTcWmJFiL

@__tinygrad__ · 2025-08-19 03:54
@Leik0w0 https://t.co/FM1INp8Yqn

@Leik0w0 · 2025-08-19 04:05
@__tinygrad__ I see your point, though this only works if the search space contains what you are looking for. I.e: the new shiny kernel which uses the new hardware features. My point is if these features change all the time, you will always have to add them to the search space (takes time).

@Leik0w0 · 2025-08-19 04:05
@__tinygrad__ I see your point, though this only works if the search space contains what you are looking for. I.e: the new shiny kernel which uses the new hardware features. My point is if these features change all the time, you will always have to add them to the search space (takes time).

@__tinygrad__ · 2025-08-19 04:13
@Leik0w0 Of course you need to add them to the search space, but that looks more like adding an instruction to LLVM. You describe the new feature, let the machine do tons of search to figure out how to use it.

@__tinygrad__ · 2025-08-19 04:18
@art__friedman Python is only slow if you write too much code.

@typedfemale · 2025-08-19 19:43
idealizing torch or JAX is a negative thought pattern

@__tinygrad__ · 2025-08-19 21:49
RT @phoronix: The @__tinygrad__ Tinygrad 0.11 Released With AMD MI350 Support, NVIDIA Blackwell https://t.co/mQlvphr44n

@sasuke___420 · 2025-08-20 06:31
@dogecahedron @typedfemale this sucks ass https://t.co/83E9SFpmMM

@sasuke___420 · 2025-08-20 06:32
@dogecahedron @typedfemale .@realGeorgeHotz has to invent an actual magic compiler if he wants this source code to turn into an implmentation that uses 1 pass to compute the central moments and a 2nd pass to zscore + is as numerically stable as competing frameworks

@sasuke___420 · 2025-08-20 06:31
@dogecahedron @typedfemale this sucks ass https://t.co/83E9SFpmMM

@sasuke___420 · 2025-08-20 07:11
@dogecahedron @typedfemale it seems to be 5 passes over the input as written, for an operation that is memory-bound on gpus. a clever compiler may reduce it to 3, but the handmade implementations in other frameworks use 2

@__tinygrad__ · 2025-08-20 08:15
@sasuke___420 @dogecahedron @typedfemale We're actively working on this. Our new compiler gets 3. https://t.co/NH1UukJx9P

@__tinygrad__ · 2025-08-20 08:15
@sasuke___420 @dogecahedron @typedfemale We're actively working on this. Our new compiler gets 3. https://t.co/NH1UukJx9P

@__tinygrad__ · 2025-08-20 08:16
@sasuke___420 @dogecahedron @typedfemale Can you link a kernel used in practice that does 2? I know there's algorithms to do mean+var together, but I'm not sure if they are commonly used.

@sasuke___420 · 2025-08-20 08:18
@__tinygrad__ @dogecahedron @typedfemale i actually found earlier this year that you cannot pass the numerics tests in pytorch if you add more than like 30 1-sized updates to a welford state, you \*must\* merge similarly-sized larger states.

@sasuke___420 · 2025-08-20 08:19
@__tinygrad__ @dogecahedron @typedfemale rowwisemoments is here https://t.co/WVVTFFck9Y for pytorch on cpu, on gpu it's somewhere else sorry

@sasuke___420 · 2025-08-20 08:19
@__tinygrad__ @dogecahedron @typedfemale rowwisemoments is here https://t.co/WVVTFFck9Y for pytorch on cpu, on gpu it's somewhere else sorry

@__tinygrad__ · 2025-08-20 08:22
@sasuke___420 @dogecahedron @typedfemale So our current priority is getting the 3 stuff merged. Only the first is a read from global btw, the other 2 are in local. Then we'll see where we land perf wise, I bet we'll saturate the global bandwidth and it'll be a wash.

@sasuke___420 · 2025-08-20 08:27
@__tinygrad__ @dogecahedron @typedfemale should be one for computing the mean*, then one for subtracting and squaring, then one for normalizing. basically, there seem to be two reduction steps that depend on the whole array and one depends on the other, and then the whole array depends on the output of the second

@__tinygrad__ · 2025-08-20 08:29
@sasuke___420 @dogecahedron @typedfemale Yea the three are: the mean, then subtract the mean and get variance, then normalize. The norm in LayerNorm is per channel, so there's a lot of locality. I doubt anything has to be fetched from GMEM more than once.

@__tinygrad__ · 2025-08-20 08:29
@sasuke___420 @dogecahedron @typedfemale Yea the three are: the mean, then subtract the mean and get variance, then normalize. The norm in LayerNorm is per channel, so there's a lot of locality. I doubt anything has to be fetched from GMEM more than once.

@__tinygrad__ · 2025-08-20 08:41
@sasuke___420 @dogecahedron @typedfemale This is what the rangeify branch outputs now. Three loops. Still a lot of cleanup work needed, but the core stuff is there. https://t.co/hnT2CtsJkd

@rmarcilhoo · 2025-08-20 14:07
@sasuke___420 @__tinygrad__ @dogecahedron @typedfemale That's 5 PRs then. Refactoring is the MVP in oss.

@sasuke___420 · 2025-08-20 14:09
@rmarcilhoo @__tinygrad__ @dogecahedron @typedfemale No, they have an explicit objective of minimizing LOC, and the implementations of this stuff are much more than the 2 LOC they currently spend.

@sasuke___420 · 2025-08-20 14:09
@rmarcilhoo @__tinygrad__ @dogecahedron @typedfemale No, they have an explicit objective of minimizing LOC, and the implementations of this stuff are much more than the 2 LOC they currently spend.

@__tinygrad__ · 2025-08-20 15:36
@sasuke___420 @rmarcilhoo @dogecahedron @typedfemale We would absolutely spend some lines on a joint mean+var algorithm.

@__tinygrad__ · 2025-08-20 21:29
A magic compiler you say? We're working on it. It's less magic than you think. It's so so much easier to reason about neural network style compute than Turing complete compute, especially when your op set is tiny.

@__tinygrad__ · 2025-08-20 21:38
LayerNorm is already 3 passes with RANGEIFY=1. I'm curious what that online mean/var trick really boils down to in terms of the raw ops. We do things like stable sigmoid with fairly generic rewrite rules. https://t.co/Ttw7I5qkcc

@Leik0w0 · 2025-08-20 22:25
@__tinygrad__ Will you ever consider equality saturation instead of ordered rewriting?

@comma_ai · 2025-08-20 22:52
The @__tinygrad__ tinybox line is getting legit. Five built so far this week. https://t.co/pwUkxXfQZU

@__tinygrad__ · 2025-08-21 00:19
RT @comma_ai: The @__tinygrad__ tinybox line is getting legit. Five built so far this week. https://t.co/pwUkxXfQZU

@Leik0w0 · 2025-08-20 22:25
@__tinygrad__ Will you ever consider equality saturation instead of ordered rewriting?

@__tinygrad__ · 2025-08-21 00:20
@Leik0w0 Yea, the first step is a minor change to the output of UPat to be another UPat instead of a lambda. We want to move in this direction, but in practice it's not any of the bottlenecks.

@comma_ai · 2025-08-20 22:52
The @__tinygrad__ tinybox line is getting legit. Five built so far this week. https://t.co/pwUkxXfQZU

@NavyaRavuri · 2025-08-21 03:12
@comma_ai @__tinygrad__ Looks very good & yes, they do look like mega cube sats indeed. What use cases are these meant for?

@NavyaRavuri · 2025-08-21 03:12
@comma_ai @__tinygrad__ Looks very good & yes, they do look like mega cube sats indeed. What use cases are these meant for?

@__tinygrad__ · 2025-08-21 03:51
@NavyaRavuri @comma_ai We would sell at cost to someone who wanted to put one in orbit. Must have previous putting things in orbit experience.

@__tinygrad__ · 2025-08-22 02:31
The syntax is still a bit clunky, but soon LLM layers, gradient accumulation, and training loops won't be written with Python for loops. Paraphrasing Halide: "separate the specification from the compilation and scheduling details" except we will spec whole $100M training runs. https://t.co/32Tv6hwcps

@fangpenlin · 2025-08-22 03:34
@__tinygrad__ Is Tinygrad going to support something like vmap? I haven't looked closely yet, but it seems like for loop + duplicating complex model compute graph making the JIT compiler really slow, sometimes even recursion overflow

@__tinygrad__ · 2025-08-22 03:38
@fangpenlin That's basically what this is, but like all things in tinygrad vs JAX you apply it to the Tensor instead of the function. You can put any function you want instead of that matmul.

@__tinygrad__ · 2025-08-22 03:38
@fangpenlin That's basically what this is, but like all things in tinygrad vs JAX you apply it to the Tensor instead of the function. You can put any function you want instead of that matmul.

@__tinygrad__ · 2025-08-22 03:45
@fangpenlin Here it is being used just like vmap. I argue the tinygrad way is even more flexible. Have a more complex example from JAX? https://t.co/nh27lpUzWf

@fangpenlin · 2025-08-22 03:49
@__tinygrad__ Ah, I see. Pretty cool. I will try it out. Thank you so much 🙇‍♀️

@__tinygrad__ · 2025-08-22 03:50
@fangpenlin You have to run with RANGEIFY=1 for it to work, and RANGEIFY is very alpha. But this is the direction things are going.

@__tinygrad__ · 2025-08-22 03:50
@fangpenlin You have to run with RANGEIFY=1 for it to work, and RANGEIFY is very alpha. But this is the direction things are going.

@__tinygrad__ · 2025-08-22 03:56
@fangpenlin Also, it's easy to write high level syntactic sugar in Tensor using this if you want it to be vmap exactly. x.vmap(f, in_axes=0, out_axes=0)

@__tinygrad__ · 2025-08-22 05:22
The core 6 can express all movement. https://t.co/F7qvjxLfL6

@__tinygrad__ · 2025-08-22 05:33
@penrosetiler What would you like to do with as_strided that you can't do with the core 6? to_movement_ops should generate any as_strided. We should really add it as a method to Tensor. PR and tests?

@ptrschmdtnlsn · 2025-08-22 06:14
@__tinygrad__ @penrosetiler Maybe this is me being dumb, but how do I do 2D sliding windows (which as_strided can do) with the ops you've given? For 1D sliding windows I can expand, pad, and reshape, but I'm not sure it can be done efficiently for 2D windows? But maybe I'm being dumb.

@fangpenlin · 2025-08-22 07:13
https://t.co/81ORENCtIr Looking into how Tinygrad generates random numbers 🤔 It seems like it's using some kind of CBRNG called threefry https://t.co/IWEx6Y2gGh

@0xKaas · 2025-08-22 15:22
@__tinygrad__ Might be wrong but isn't pad just allocation + slice + copy and then custom padding logic ? And flip is just changing strides ?

@0xKaas · 2025-08-22 15:30
@__tinygrad__ Wait actually I think all 6 mov ops are actually just as_strided (+ some non mov op for copying, filling).

@Mascobot · 2025-08-22 16:14
🚨 New: We built @a16z's personal GPU AI Workstation Founders Edition - 4x NVIDIA RTX 6000 PRO Blackwell Max-Q (384GB total VRAM) - 8TB of NVMe PCIe 5.0 storage - AMD Threadripper PRO 7975WX (32 cores, 64 threads) - 256GB ECC DDR5 RAM - 1650Watts at peak (runs on a standard 15Amp/120V circuit). For training, AI research, and deploying models locally. A datacenter-class AI rig you can keep under your desk. We are planning to make a limited number of these a16z AI Workstations. Build guide + how you can make your own 👇

@__tinygrad__ · 2025-08-22 16:48
@Ayghri expand + shrink https://t.co/BjQ3qCV292

@ptrschmdtnlsn · 2025-08-22 06:14
@__tinygrad__ @penrosetiler Maybe this is me being dumb, but how do I do 2D sliding windows (which as_strided can do) with the ops you've given? For 1D sliding windows I can expand, pad, and reshape, but I'm not sure it can be done efficiently for 2D windows? But maybe I'm being dumb.

@__tinygrad__ · 2025-08-22 16:48
@ptrschmdtnlsn @penrosetiler https://t.co/DZuOJOBQBX

@fangpenlin · 2025-08-22 07:13
https://t.co/81ORENCtIr Looking into how Tinygrad generates random numbers 🤔 It seems like it's using some kind of CBRNG called threefry https://t.co/IWEx6Y2gGh

@__tinygrad__ · 2025-08-22 16:56
@fangpenlin This one we just took from JAX

@0xKaas · 2025-08-22 15:30
@__tinygrad__ Wait actually I think all 6 mov ops are actually just as_strided (+ some non mov op for copying, filling).

@__tinygrad__ · 2025-08-22 16:58
@0xKaas Well sure, but now try to compute the gradient of as_strided. This is a lot more structured, and promises you'll never access out of bounds memory.

@TheAhmadOsman · 2025-08-23 12:36
The number of people saying "just use the API bro" when it comes to LLMs is wild. I get it, not everyone's into GPUs and hardware, but do they realize we're in an era where Software 'Engineers' are straight-up scared of hardware? Everything’s in the 'Cloud,' vendors have users locked in at 10x+ margins, and people are fine with it. Do you really want Sam Altman or his buddies deciding how much you'll pay for API access in the future?

@Proziam · 2025-08-23 14:10
Option A - put your over specced gaming PC to work Option B - buy an @__tinygrad__ box Option C - be a slave to ai masters who think you're too retarded to be trusted with models that aren't bubble wrapped Which way, future man

@TheAhmadOsman · 2025-08-23 14:12
@Proziam @__tinygrad__ I prefer the Option where you read my site and build it yourself You'd save at least half the costs

@Proziam · 2025-08-23 14:14
@TheAhmadOsman @__tinygrad__ It's bold of you to assume an average engineer can read tbh In a perfect world everyone would build proper homelabs. But, gotta give people the easy button or most won't do it.

@nmswede · 2025-08-23 16:00
Older post but for those that are wondering about what @xai will be up to over the next few years, let’s just say that we’ll be busy…

@Proziam · 2025-08-23 14:14
@TheAhmadOsman @__tinygrad__ It's bold of you to assume an average engineer can read tbh In a perfect world everyone would build proper homelabs. But, gotta give people the easy button or most won't do it.

@__tinygrad__ · 2025-08-23 17:21
@Proziam @TheAhmadOsman You definitely don't save half the costs, our profit margin is more like 30% than 50%, and we're doing it at scale. And it's actually a huge pain to make GPU extenders work. The advantage to self built is you can slowly buy/upgrade and you can enjoy struggle!

@Proziam · 2025-08-23 14:14
@TheAhmadOsman @__tinygrad__ It's bold of you to assume an average engineer can read tbh In a perfect world everyone would build proper homelabs. But, gotta give people the easy button or most won't do it.

@__tinygrad__ · 2025-08-23 17:27
Is @a16z pivoting to be a computer maker? It is a good business, though that machine looks very expensive. tinybox green v2 are in stock and can ship Monday!

@__tinygrad__ · 2025-08-23 17:29
Also, @Mascobot you need to fix your cooling. At 91C that's definitely throttling. https://t.co/f2DSw8YZJ7

@__tinygrad__ · 2025-08-23 17:33
@Mascobot We keep it below 80C while staying quiet. https://t.co/L5iBjSjtTJ

@__tinygrad__ · 2025-08-23 17:33
@Mascobot We keep it below 80C while staying quiet. https://t.co/L5iBjSjtTJ

@__tinygrad__ · 2025-08-23 17:38
@Mascobot We think the RTX PRO 6000 is overpriced, but if someone wants to place an order for 5+ machines, we'll build tinyboxes with them. Even if you want Max-Q, we'll include 2 PSUs for higher efficiency. With proper cooling + at 300W/card it actually should be silent.

@__tinygrad__ · 2025-08-23 17:27
Is @a16z pivoting to be a computer maker? It is a good business, though that machine looks very expensive. tinybox green v2 are in stock and can ship Monday!

@zodttd · 2025-08-23 17:41
@__tinygrad__ @a16z They must see the massive value of where Tiny Corp fits in to everything

@TheAhmadOsman · 2025-08-23 17:42
I mean no disrespect, but the 4x RTX 5090s box (128GB VRAM/32c CPU/192GB RAM) being priced at $29,000 (before Tax?) is a no-go for me personally. Maybe good for businesses, but individuals can put something like it together for half the price, including Retimers for PCIe.

@zodttd · 2025-08-23 17:41
@__tinygrad__ @a16z They must see the massive value of where Tiny Corp fits in to everything

@__tinygrad__ · 2025-08-23 17:42
@zodttd @a16z lol not sure about that, we don't have a crypto coin for them to speculate on

@__tinygrad__ · 2025-08-23 17:38
@Mascobot We think the RTX PRO 6000 is overpriced, but if someone wants to place an order for 5+ machines, we'll build tinyboxes with them. Even if you want Max-Q, we'll include 2 PSUs for higher efficiency. With proper cooling + at 300W/card it actually should be silent.

@amootpoint · 2025-08-23 17:43
@__tinygrad__ @Mascobot Why you think it’s over priced ? Seems reasonable. No ? What would you rather choose ?

@__tinygrad__ · 2025-08-23 17:38
@Mascobot We think the RTX PRO 6000 is overpriced, but if someone wants to place an order for 5+ machines, we'll build tinyboxes with them. Even if you want Max-Q, we'll include 2 PSUs for higher efficiency. With proper cooling + at 300W/card it actually should be silent.

@Proziam · 2025-08-23 17:44
@__tinygrad__ @Mascobot Any indication on when red box v2 is coming? Now that the drivers are coming along the value should be much more compelling

@TheAhmadOsman · 2025-08-23 17:42
I mean no disrespect, but the 4x RTX 5090s box (128GB VRAM/32c CPU/192GB RAM) being priced at $29,000 (before Tax?) is a no-go for me personally. Maybe good for businesses, but individuals can put something like it together for half the price, including Retimers for PCIe.

@__tinygrad__ · 2025-08-23 17:45
@TheAhmadOsman So our margins are ~30%, not 50%, and we have the benefits of scale. If you are buying the cheapest eBay GPUs, using a miner case, cheaper motherboard without MCIO, no RAID array, open air w cheap fans, it might be possible. However, you get what you pay for.

@__tinygrad__ · 2025-08-23 17:45
@TheAhmadOsman So our margins are ~30%, not 50%, and we have the benefits of scale. If you are buying the cheapest eBay GPUs, using a miner case, cheaper motherboard without MCIO, no RAID array, open air w cheap fans, it might be possible. However, you get what you pay for.

@__tinygrad__ · 2025-08-23 17:46
@TheAhmadOsman Also, we don't use retimers, we just make sure we have the signal integrity passively. Not really sure if there's a downside, but I don't like the idea of another active component.

@Proziam · 2025-08-23 17:44
@__tinygrad__ @Mascobot Any indication on when red box v2 is coming? Now that the drivers are coming along the value should be much more compelling

@__tinygrad__ · 2025-08-23 17:48
@Proziam @Mascobot We have one last red box v1 to sell first...really wish there was a big RDNA4 GPU. The RDNA3 box is still the highest memory bandwidth for AMD.

@__tinygrad__ · 2025-08-23 17:45
@TheAhmadOsman So our margins are ~30%, not 50%, and we have the benefits of scale. If you are buying the cheapest eBay GPUs, using a miner case, cheaper motherboard without MCIO, no RAID array, open air w cheap fans, it might be possible. However, you get what you pay for.

@miolini · 2025-08-23 17:51
@__tinygrad__ @TheAhmadOsman c'mon, playing this questionable sale pitch card, remember your hacking roots. for non-business use in homelab environment cheap but working components is more than enough for years

@amootpoint · 2025-08-23 17:43
@__tinygrad__ @Mascobot Why you think it’s over priced ? Seems reasonable. No ? What would you rather choose ?

@__tinygrad__ · 2025-08-23 17:51
@amootpoint @Mascobot I'd choose 3x5090s instead of one RTX PRO 6000. Same RAM, triple the RAM bandwidth, triple the FLOPS.

@__tinygrad__ · 2025-08-23 17:45
@TheAhmadOsman So our margins are ~30%, not 50%, and we have the benefits of scale. If you are buying the cheapest eBay GPUs, using a miner case, cheaper motherboard without MCIO, no RAID array, open air w cheap fans, it might be possible. However, you get what you pay for.

@TheAhmadOsman · 2025-08-23 17:53
@__tinygrad__ Excuse me, cheapest ebay GPUs? RTX 5090 is being sold at $2,000 I am talking about an Epyc 9004 CPU with DDR5, MCIO, RAID array, and highly rated PSUs and fans too This also wouldn't force powerlimiting the GPUs which I am assuming is what you do No need for a sale pitch here

@miolini · 2025-08-23 17:51
@__tinygrad__ @TheAhmadOsman c'mon, playing this questionable sale pitch card, remember your hacking roots. for non-business use in homelab environment cheap but working components is more than enough for years

@__tinygrad__ · 2025-08-23 17:53
@miolini @TheAhmadOsman We certainly aren't preventing anyone from building their own, we encourage it. But it isn't the same product. If you could really build a tinybox for half of what we charge, you should come work here!

@TheAhmadOsman · 2025-08-23 17:53
@__tinygrad__ Excuse me, cheapest ebay GPUs? RTX 5090 is being sold at $2,000 I am talking about an Epyc 9004 CPU with DDR5, MCIO, RAID array, and highly rated PSUs and fans too This also wouldn't force powerlimiting the GPUs which I am assuming is what you do No need for a sale pitch here

@__tinygrad__ · 2025-08-23 17:56
@TheAhmadOsman We don't powerlimit the GPUs. And yea, you might be able to get the weaker manufacturers for $2,000. We pay more, even in large quantities. It's not a sales pitch, it's just a different product.

@__tinygrad__ · 2025-08-23 17:56
@TheAhmadOsman We don't powerlimit the GPUs. And yea, you might be able to get the weaker manufacturers for $2,000. We pay more, even in large quantities. It's not a sales pitch, it's just a different product.

@TheAhmadOsman · 2025-08-23 18:04
@__tinygrad__ Define weaker manufacturer please Also, what's so unique about your product? Isn't it spec hardware components being put together? TFLOPs, CPU platform, memory, bandwidth, temps, etc, all being the same = same thing not a different product

@TheAhmadOsman · 2025-08-23 18:04
@__tinygrad__ Define weaker manufacturer please Also, what's so unique about your product? Isn't it spec hardware components being put together? TFLOPs, CPU platform, memory, bandwidth, temps, etc, all being the same = same thing not a different product

@__tinygrad__ · 2025-08-23 18:07
@TheAhmadOsman Sounds like you should open a competing business and undercut us :)

@TheAhmadOsman · 2025-08-23 18:40
i am not trying to compete, i am talking about individuals being able to build what they sell at 29k for <15k they say what i'd build would be with "cheap ebay gpus", then change it to new 5090 but "worse gpu manufacturer" and that theirs is "a different product", then snark lol https://t.co/3ziE8INEX5

@TheAhmadOsman · 2025-08-23 18:40
i am not trying to compete, i am talking about individuals being able to build what they sell at 29k for <15k they say what i'd build would be with "cheap ebay gpus", then change it to new 5090 but "worse gpu manufacturer" and that theirs is "a different product", then snark lol https://t.co/3ziE8INEX5

@TheAhmadOsman · 2025-08-23 18:44
finally, @__tinygrad__, i built this in my basement and have a huge ass cluster, so i know what i am talking about i said no disrespect but honestly you're responses showed nothing but manipulative sales tactics individuals can spend half ur cost & get better quality stuff lol https://t.co/ZDiZq6076N

@TheAhmadOsman · 2025-08-23 18:44
finally, @__tinygrad__, i built this in my basement and have a huge ass cluster, so i know what i am talking about i said no disrespect but honestly you're responses showed nothing but manipulative sales tactics individuals can spend half ur cost & get better quality stuff lol https://t.co/ZDiZq6076N

@__tinygrad__ · 2025-08-23 18:48
@TheAhmadOsman That picture says it all. If you want mining cases, zip tied fans, and an extra PCIe connector killing signal to the point you need a retimer, yes, you can save 20%. Or you can buy a beautiful tinybox. https://t.co/jgGhSOD3ln

@__tinygrad__ · 2025-08-23 18:48
@TheAhmadOsman That picture says it all. If you want mining cases, zip tied fans, and an extra PCIe connector killing signal to the point you need a retimer, yes, you can save 20%. Or you can buy a beautiful tinybox. https://t.co/jgGhSOD3ln

@TheAhmadOsman · 2025-08-23 18:51
@__tinygrad__ You mean a box where 4 EXTREMELY HOT & POWER HUNGRY 5090s are shoved together back-to-back? I built mine with the opposite of that in-mind :) You're once again avoiding the real questions and trying to find anything else to talk about about LMAO

@TheAhmadOsman · 2025-08-23 18:51
@__tinygrad__ You mean a box where 4 EXTREMELY HOT & POWER HUNGRY 5090s are shoved together back-to-back? I built mine with the opposite of that in-mind :) You're once again avoiding the real questions and trying to find anything else to talk about about LMAO

@__tinygrad__ · 2025-08-23 18:53
@TheAhmadOsman 79C max and quiet without any power limiting. https://t.co/ToYjajOHVi

@__tinygrad__ · 2025-08-23 18:53
@TheAhmadOsman 79C max and quiet without any power limiting. https://t.co/ToYjajOHVi

@TheAhmadOsman · 2025-08-23 18:56
@__tinygrad__ i see no value in continuing this just do better than "cheap ebay gpus" and "weaker manufacturer" the next time someone asks you about your margins very uncool

@TheAhmadOsman · 2025-08-23 18:56
@__tinygrad__ i see no value in continuing this just do better than "cheap ebay gpus" and "weaker manufacturer" the next time someone asks you about your margins very uncool

@__tinygrad__ · 2025-08-23 18:58
@TheAhmadOsman For reference, I listed 5 things. https://t.co/RMJ6MAeTsP

@__tinygrad__ · 2025-08-23 17:27
Is @a16z pivoting to be a computer maker? It is a good business, though that machine looks very expensive. tinybox green v2 are in stock and can ship Monday!

@Its_keith_d · 2025-08-23 19:19
@__tinygrad__ @a16z Will Tiny make a RTX6000 Pro model?

@Its_keith_d · 2025-08-23 19:19
@__tinygrad__ @a16z Will Tiny make a RTX6000 Pro model?

@__tinygrad__ · 2025-08-23 19:38
@Keithdavis85 @a16z If we get an order for 5+ boxes. I don't really think there's demand, I think it's just something people say they want when they don't have to pay for it.

@elonmusk · 2025-08-23 22:34
Having thought about it some more, I think the 50 million H100 equivalent number in 5 years is about right. Eventually, billions.

@__tinygrad__ · 2025-08-23 23:27
@elonmusk Where are you going to get the power? That's 50 GW at H100 power efficiency....maybe 5x off in 5 years depending how you treat dtypes (which were the biggest win and end with FP4). So you need 10 GW. That's more than a Grand Coulee Dam.

@millpreetk · 2025-08-23 23:37
@__tinygrad__ @elonmusk Tesla Solar is finally getting off the ground with Xai VC money. It’ll come full circle somehow lmao

@__tinygrad__ · 2025-08-23 23:27
@elonmusk Where are you going to get the power? That's 50 GW at H100 power efficiency....maybe 5x off in 5 years depending how you treat dtypes (which were the biggest win and end with FP4). So you need 10 GW. That's more than a Grand Coulee Dam.

@andromeda74356 · 2025-08-23 23:55
@__tinygrad__ @elonmusk https://t.co/6mYyTrkkLX Feynman launching ~2028 after Rubin, should need less than 4 million chips to hit a 50M H100 equivalent cluster and need < 5 GW. natural gas plants can easily scale to that, but it's probably going to be a combination of solar, nuclear, gas, and grid

@millpreetk · 2025-08-23 23:37
@__tinygrad__ @elonmusk Tesla Solar is finally getting off the ground with Xai VC money. It’ll come full circle somehow lmao

@__tinygrad__ · 2025-08-23 23:56
@millpreetk @elonmusk I think a big solar array would be the best bet.

@andromeda74356 · 2025-08-23 23:55
@__tinygrad__ @elonmusk https://t.co/6mYyTrkkLX Feynman launching ~2028 after Rubin, should need less than 4 million chips to hit a 50M H100 equivalent cluster and need < 5 GW. natural gas plants can easily scale to that, but it's probably going to be a combination of solar, nuclear, gas, and grid

@__tinygrad__ · 2025-08-23 23:58
@andromeda74356 @elonmusk Even better, combine it all. When it comes to software I question @elonmusk's timelines, but I trust mostly for the physics based ones. This is doable.

@mov_axbx · 2025-08-23 22:24
I think you may be underestimating the value people put on just buying the thing. In my 7x4090 guide I specifically suggest the reader check out @__tinygrad__ for that reason. I learned this lesson in the 2000s, business required a PBX so I screwed around building an Asterisk box. Finally realized why PBXes cost what they do.

@TheAhmadOsman · 2025-08-24 00:03
@mov_axbx @__tinygrad__ I like to buy things that just work. I don't like businesses that lies in their sales pitch. I wasn't interested in discussing anything with them. I was replying to a follower of mine and they jumped in and started arguing and lying.

@TheAhmadOsman · 2025-08-24 00:03
@mov_axbx @__tinygrad__ I like to buy things that just work. I don't like businesses that lies in their sales pitch. I wasn't interested in discussing anything with them. I was replying to a follower of mine and they jumped in and started arguing and lying.

@__tinygrad__ · 2025-08-24 00:07
@TheAhmadOsman @mov_axbx We were tagged in the top post you were replying to. We didn't lie about anything. You quote tweeted us after. Stop complaining. Start a competing business.

@__tinygrad__ · 2025-08-24 01:53
tiny corp will never have a Request Quote button. Our prices are very clear on the website. https://t.co/goTr1njP7v

@__tinygrad__ · 2025-08-24 01:56
This seems to be our only competitor with a price listed, and it's higher than ours. For the others, you don't even have to request the quote to know it's higher. tinybox is the cheapest out of the box way to get a powerful deep learning computer in your house. https://t.co/dyfZyowLmK

@__tinygrad__ · 2025-08-24 02:03
For people who want to build their own, you certainly can, but this isn't as easy as building a gaming PC. Here are some guides that go into the footguns, many annoyances around PCIe extenders and PSUs. https://t.co/DMsZ5g9XxI https://t.co/gQzDegym44

@__tinygrad__ · 2025-08-24 02:03
For people who want to build their own, you certainly can, but this isn't as easy as building a gaming PC. Here are some guides that go into the footguns, many annoyances around PCIe extenders and PSUs. https://t.co/DMsZ5g9XxI https://t.co/gQzDegym44

@__tinygrad__ · 2025-08-24 02:07
tinybox has a custom designed case, a wall of low pressure fans, enough signal integrity to avoid PCIe retimers, an LCD screen to show GPU status, and a 4x NVMe raid array (most of the PCie to NVMe cards on amazon generate AERs). https://t.co/WqnwnNamDm

@__tinygrad__ · 2025-08-24 02:03
For people who want to build their own, you certainly can, but this isn't as easy as building a gaming PC. Here are some guides that go into the footguns, many annoyances around PCIe extenders and PSUs. https://t.co/DMsZ5g9XxI https://t.co/gQzDegym44

@__tinygrad__ · 2025-08-24 02:07
tinybox has a custom designed case, a wall of low pressure fans, enough signal integrity to avoid PCIe retimers, an LCD screen to show GPU status, and a 4x NVMe raid array (most of the PCie to NVMe cards on amazon generate AERs). https://t.co/WqnwnNamDm

@__tinygrad__ · 2025-08-24 02:22
We run our own cluster of 18 of these boxes for research and tinygrad development. https://t.co/POMNhpTEO1

@__tinygrad__ · 2025-08-24 01:53
tiny corp will never have a Request Quote button. Our prices are very clear on the website. https://t.co/goTr1njP7v

@Jatin_exe · 2025-08-24 09:35
@__tinygrad__ Never seen Tenstorrent loud and quiet box ?

@__tinygrad__ · 2025-08-24 17:30
The tinybox factory is dialed in. Parts in stock, automated testing and provisioning stable. We should be able to make and ship 10 boxes per week. https://t.co/0wq2iSC0gD

@__tinygrad__ · 2025-08-24 17:30
The tinybox factory is dialed in. Parts in stock, automated testing and provisioning stable. We should be able to make and ship 10 boxes per week. https://t.co/0wq2iSC0gD

@__tinygrad__ · 2025-08-24 17:31
.@comma_ai thinks it's 5 boxes per week, but let's crank demand to 10! https://t.co/uyTPbzDEVd

@Jatin_exe · 2025-08-24 09:35
@__tinygrad__ Never seen Tenstorrent loud and quiet box ?

@__tinygrad__ · 2025-08-24 17:40
@Jatin_exe We love that @tenstorrent has prices on the site! It's not a box full of 5090s though.

@__tinygrad__ · 2025-08-24 18:12
RT @HotAisle: Neat to see an open source project using @__tinygrad__ for something cool... https://t.co/fpwcJjzbBb

@advancedjd · 2025-08-24 18:57
This entire discussion is tech nerds not understanding what business users are willing to pay for. @__tinygrad__ knows hoe to actually sell stuf https://t.co/uvwZMGfgjF

@__tinygrad__ · 2025-08-24 19:16
which way, GPU enjoyer?

@__tinygrad__ · 2025-08-24 19:16
which way, GPU enjoyer?

@swiftie2996 · 2025-08-24 20:42
@__tinygrad__ If I need to purchase some for specific use cases, can your support help out sizing? Can you also provide monthly rentals ? That would be really nice.. since GPU is advancing too fast and I dont want cloud.

@gandamu_ml · 2025-08-24 19:26
@__tinygrad__ I built my own 4x GPU box once. From conception, it took about a month. I had a few hiccups that people tend not to publicly mention. A bad memory module, surprised I had to flash BIOS for my CPU (and ordered a new mobo before I figured it out), etc. A working box has a price.

@__tinygrad__ · 2025-08-24 22:15
@gandamu_ml About 30% of boxes we build fail our aggressive automated stress test the first time, and we don't ship until they pass. Every tinybox has to train a full ResNet on all 4 GPUs before it's ready to be shipped.

@__tinygrad__ · 2025-08-24 22:15
@gandamu_ml About 30% of boxes we build fail our aggressive automated stress test the first time, and we don't ship until they pass. Every tinybox has to train a full ResNet on all 4 GPUs before it's ready to be shipped.

@gandamu_ml · 2025-08-24 22:17
@__tinygrad__ that's awesome! I was wondering about that. Stress test + a realistic job is the best I could hope for.

@__tinygrad__ · 2025-08-24 22:15
@gandamu_ml About 30% of boxes we build fail our aggressive automated stress test the first time, and we don't ship until they pass. Every tinybox has to train a full ResNet on all 4 GPUs before it's ready to be shipped.

@__tinygrad__ · 2025-08-24 22:18
@gandamu_ml Like individually, each component maybe has a 1% failure rate. But the tinybox has like 50 components! (1-0.99**50) * 100 = 40%! If something as little as one fan doesn't spin right, we reject and replace until everything passes.

@gandamu_ml · 2025-08-24 22:17
@__tinygrad__ that's awesome! I was wondering about that. Stress test + a realistic job is the best I could hope for.

@__tinygrad__ · 2025-08-24 22:19
@gandamu_ml We've had two issues in the field not fixed by a reimage out of over 100 boxes sold. One was a CPU that failed (motherboard mailed back) and the other was fixed by replacing GPU power cable.

@__tinygrad__ · 2025-08-24 19:16
which way, GPU enjoyer?

@hornswoggle567 · 2025-08-24 22:20
@__tinygrad__ If it got damaged during shipping I’d never recover from this (I’m in Vancouver)

@hornswoggle567 · 2025-08-24 22:20
@__tinygrad__ If it got damaged during shipping I’d never recover from this (I’m in Vancouver)

@__tinygrad__ · 2025-08-24 22:21
@hornswoggle567 We buy insurance on all the packages. We did have one damaged in shipping (looked like someone drove a forklift into it!), the customer just rejected and FedEx paid us out.

@swiftie2996 · 2025-08-24 20:42
@__tinygrad__ If I need to purchase some for specific use cases, can your support help out sizing? Can you also provide monthly rentals ? That would be really nice.. since GPU is advancing too fast and I dont want cloud.

@__tinygrad__ · 2025-08-24 22:32
@swiftie2996 We sell two boxes, a red one and a green one. By standardizing the box, we can put way more effort into provisioning and testing than most computer manufacturers. All box exact same. For payment, you send money, we send box.

@__tinygrad__ · 2025-08-24 19:16
which way, GPU enjoyer?

@LottoLabs · 2025-08-24 22:54
@__tinygrad__ We have more in common that we have in difference

@__tinygrad__ · 2025-08-24 22:15
@gandamu_ml About 30% of boxes we build fail our aggressive automated stress test the first time, and we don't ship until they pass. Every tinybox has to train a full ResNet on all 4 GPUs before it's ready to be shipped.

@SeanDinhR · 2025-08-25 03:09
@__tinygrad__ @gandamu_ml that's crazy, what's failing so often?

@SeanDinhR · 2025-08-25 03:09
@__tinygrad__ @gandamu_ml that's crazy, what's failing so often?

@__tinygrad__ · 2025-08-25 03:52
@SeanDinhR @gandamu_ml Every possible thing. Individually, 99% are good, but if you put 50 parts with 1% failure rate together, often one doesn't work out of the box.

@LottoLabs · 2025-08-24 22:54
@__tinygrad__ We have more in common that we have in difference

@__tinygrad__ · 2025-08-25 04:42
@LottoLabs True! They try to keep the GPU middle class divided so they can rule us with their 300,000 B200s.

@__tinygrad__ · 2025-08-25 06:16
vLLM + open-webui running gpt-oss-120b on a tinybox green v2 Just `vllm serve openai/gpt-oss-120b --tensor-parallel-size 4 --async-scheduling` and you have a local OpenAI API you can trust. https://t.co/WG2RCFTY6B

@__tinygrad__ · 2025-08-25 06:16
vLLM + open-webui running gpt-oss-120b on a tinybox green v2 Just `vllm serve openai/gpt-oss-120b --tensor-parallel-size 4 --async-scheduling` and you have a local OpenAI API you can trust. https://t.co/WG2RCFTY6B

@geteviapp · 2025-08-25 06:19
This is actually a very decent model for coding and tool calling, didn’t expect it to be this good. I also finally realized that MacBooks are only good to play with LLMs, once you start slamming LLMs with calls MacBooks get hot and very unpleasant to use. Dedicated box in a basement is a much better solution.

@geteviapp · 2025-08-25 06:19
This is actually a very decent model for coding and tool calling, didn’t expect it to be this good. I also finally realized that MacBooks are only good to play with LLMs, once you start slamming LLMs with calls MacBooks get hot and very unpleasant to use. Dedicated box in a basement is a much better solution.

@__tinygrad__ · 2025-08-25 06:22
@geteviapp I haven't found anything that runs at a decent speed on a MacBook to be a usable model. It's very nice how gpt-oss-120b is prequantized, super easy to run out of the box and I think it's better than base GPT-5.

@__tinygrad__ · 2025-08-25 06:16
vLLM + open-webui running gpt-oss-120b on a tinybox green v2 Just `vllm serve openai/gpt-oss-120b --tensor-parallel-size 4 --async-scheduling` and you have a local OpenAI API you can trust. https://t.co/WG2RCFTY6B

@DoninoTrading · 2025-08-25 06:36
@__tinygrad__ what os are the tiny boxes running?

@DoninoTrading · 2025-08-25 06:36
@__tinygrad__ what os are the tiny boxes running?

@__tinygrad__ · 2025-08-25 06:39
@DoninoTrading this one is Ubuntu 24.04 https://t.co/nBaLWFCvrt

@__tinygrad__ · 2025-08-25 06:16
vLLM + open-webui running gpt-oss-120b on a tinybox green v2 Just `vllm serve openai/gpt-oss-120b --tensor-parallel-size 4 --async-scheduling` and you have a local OpenAI API you can trust. https://t.co/WG2RCFTY6B

@sbeastwindy · 2025-08-25 07:31
@__tinygrad__ gpt-oss 120b is by far the best Instruction following model I've used. It does have its downsides with hallucination but with correct context it's very nice to not have to write pages of prompts

@fangpenlin · 2025-08-25 18:50
I found an interesting pitfall (or bug?) in tinygrad. If you access an item with an out-of-bounds index, it returns zeros. However, converting these to a tensor and then to a NumPy array or list raises an index error. Hmm 🤔 https://t.co/lcGtq7yiRI

@fangpenlin · 2025-08-25 18:50
I found an interesting pitfall (or bug?) in tinygrad. If you access an item with an out-of-bounds index, it returns zeros. However, converting these to a tensor and then to a NumPy array or list raises an index error. Hmm 🤔 https://t.co/lcGtq7yiRI

@__tinygrad__ · 2025-08-25 20:15
@fangpenlin tinygrad is lazy. How would you know it's out of bounds? If you think the behavior should be something else, add a test!

@__tinygrad__ · 2025-08-25 20:15
@fangpenlin tinygrad is lazy. How would you know it's out of bounds? If you think the behavior should be something else, add a test!

@fangpenlin · 2025-08-25 20:26
@__tinygrad__ But I thought when "item()" is called, it should check the boundary and raise error if it exceeds? It seems like the current behavior returns 0. I guess it would be hard or not practical for kernel to check boundary and raise error?

@__tinygrad__ · 2025-08-25 22:50
Is this box provisioning for you? https://t.co/SSI225xdwG

@sbeastwindy · 2025-08-25 07:31
@__tinygrad__ gpt-oss 120b is by far the best Instruction following model I've used. It does have its downsides with hallucination but with correct context it's very nice to not have to write pages of prompts

@__tinygrad__ · 2025-08-25 22:55
@sbeastwindy I've been using it all day. The hallucinations are worse than ChatGPT/Claude, but I love the speed and reliability. Claude feels so slow!

@Leik0w0 · 2025-08-26 03:17
Would it be possible to make graphs like the ones from this masterpiece of a video https://t.co/37XmuGsmc1 from @__tinygrad__ kernels UOp graphs rewrites / Opts ? I’m curious about how it would look, the local problems and coloring from the execution time

@Leik0w0 · 2025-08-26 03:17
Would it be possible to make graphs like the ones from this masterpiece of a video https://t.co/37XmuGsmc1 from @__tinygrad__ kernels UOp graphs rewrites / Opts ? I’m curious about how it would look, the local problems and coloring from the execution time

@__tinygrad__ · 2025-08-26 05:31
@Leik0w0 Very excited to see it! The VIZ format is pretty simple, tinygrad is already tracking everything you need I think.

@__tinygrad__ · 2025-08-26 17:14
RT @fangpenlin: @SamiBelhareth This is the VIZ tool providedby Tinygrad for you to see how your compute graph performs and how it gets lowered into native kernel code: https://t.co/ki2MTHrYbq

@__tinygrad__ · 2025-08-26 17:17
@AiFlux It's really not! Our margins are only 30%, and we're doing this with the help of scale.

@yacineMTB · 2025-08-26 19:12
fuck you nvidia https://t.co/i25Ivjgbvu

@KinvertOG · 2025-08-26 19:26
@yacineMTB Cuda - Torch version mismatch. I've seen it 100 times. There is but one cure. @__tinygrad__

@__tinygrad__ · 2025-08-25 20:15
@fangpenlin tinygrad is lazy. How would you know it's out of bounds? If you think the behavior should be something else, add a test!

@ml_visoft · 2025-08-27 03:38
@__tinygrad__ @fangpenlin C style! Yes!

@ml_visoft · 2025-08-27 03:38
@__tinygrad__ @fangpenlin C style! Yes!

@__tinygrad__ · 2025-08-27 06:06
@ml_visoft @fangpenlin It's better than C, at least there's a bounds check and it's guaranteed to be 0!

@fangpenlin · 2025-08-25 20:26
@__tinygrad__ But I thought when "item()" is called, it should check the boundary and raise error if it exceeds? It seems like the current behavior returns 0. I guess it would be hard or not practical for kernel to check boundary and raise error?

@__tinygrad__ · 2025-08-27 06:11
@fangpenlin "Error" isn't a valid output from a kernel

@__tinygrad__ · 2025-08-28 13:11
Delete torch, delete CUDA, delete the NVIDIA kernel driver. tinygrad can't version mismatch with itself.

@doomslide · 2025-08-28 18:59
Text is NOT the univeral interface

@iamgingertrash · 2025-08-28 19:00
@doomslide Yea It’s vision Lecun will get the last laugh

@simcity99 · 2025-08-28 21:05
always has been pretty wild how few people are building on top of openpilot rn alpha is absurd

@simcity99 · 2025-08-28 21:05
always has been pretty wild how few people are building on top of openpilot rn alpha is absurd

@__tinygrad__ · 2025-08-28 22:17
@simcity99 and tinygrad will be the kernel underlying openpilot openpilot is the GNU, tinygrad is the Linux.

@__tinygrad__ · 2025-08-28 22:17
@simcity99 and tinygrad will be the kernel underlying openpilot openpilot is the GNU, tinygrad is the Linux.

@__tinygrad__ · 2025-08-29 01:53
@simcity99 Actually this post says it better. Android is built on Linux. https://t.co/bcZPEAcB4o

@__tinygrad__ · 2025-08-29 11:16
RT @seconds_0: lord, if youre listening, please grant me $29,000 to facilitate my dream of doing silly side projects on a tinybox green v2

@htihle · 2025-08-29 15:55
With the surprisingly high score from gpt-oss-120b (high), much of the gap between the open and closed models on WeirdML is now gone. However, the leading closed lab deciding to release an open model trained on their superior stack has a different feel to it than the open source community (e.g. meta or deepseek) closing the gap. R2 (whenever it comes), or qwen4 will be interesting to follow. As will the new meta superintelligence team, and whether they will continue to open source their models.

@__tinygrad__ · 2025-08-29 18:30
It's actually pretty cool that @OpenAI released the SOTA open source model. Can confirm gpt-oss-120b is good, and that it runs great on a tinybox green v2!

@AlexReibman · 2025-08-30 23:56
Who’s selling consumer hardware for running local models? I want a GPU box I can plug into my MacBook and run local LLMs

@__tinygrad__ · 2025-08-31 03:47
@AlexReibman three words: tinybox green v2

@AlexReibman · 2025-08-31 03:49
@__tinygrad__ What can I get that doesn’t require me to get a loan :( https://t.co/QEuB2qBzD5

@__tinygrad__ · 2025-08-31 17:47
RT @t0kenl1mit: After seeing some great tokens per sec with @__tinygrad__ , I am going to experiment with using it to build distributed inference and learning. Confidence in PyTorch C++ libtorch active development is shaky a bit and Tinygrad always seems to be improving, developing. (1/2)

@__tinygrad__ · 2025-08-31 03:47
@AlexReibman three words: tinybox green v2

@paul_dentro · 2025-08-31 19:46
@__tinygrad__ @AlexReibman @__tinygrad__ , can we install also Ubuntu 24 on the red tinybox? Or only 22?

@paul_dentro · 2025-08-31 19:46
@__tinygrad__ @AlexReibman @__tinygrad__ , can we install also Ubuntu 24 on the red tinybox? Or only 22?

@__tinygrad__ · 2025-08-31 22:06
@paul_dentro @AlexReibman 24 works fine, that's what we have on our internal red boxes.

@__tinygrad__ · 2025-08-31 22:09
tinybox is the best performance per $ of any box you can buy. Sure, you can buy a Mac Studio or a DGX Spark, but the performance is over 10x worse on FLOPS and RAM bandwidth. You get what you pay for.

@__tinygrad__ · 2025-08-31 22:09
tinybox is the best performance per $ of any box you can buy. Sure, you can buy a Mac Studio or a DGX Spark, but the performance is over 10x worse on FLOPS and RAM bandwidth. You get what you pay for.

@0xCodyS · 2025-09-01 00:24
@__tinygrad__ 6k for the same GPU RAM and great interconnect with Tenstorrent cards! Competitive on FP8 flops (I know lol) and very scalable between boxes https://t.co/RYn0ray3Vu

@0xCodyS · 2025-09-01 00:24
@__tinygrad__ 6k for the same GPU RAM and great interconnect with Tenstorrent cards! Competitive on FP8 flops (I know lol) and very scalable between boxes https://t.co/RYn0ray3Vu

@__tinygrad__ · 2025-09-01 02:10
@0xCodyS Show me the software doing *anything* on that box. People don't even buy the red boxes from us, and AMD's software is nowhere close to as bad as Tenstorrent's.

@__tinygrad__ · 2025-08-31 22:09
tinybox is the best performance per $ of any box you can buy. Sure, you can buy a Mac Studio or a DGX Spark, but the performance is over 10x worse on FLOPS and RAM bandwidth. You get what you pay for.

@_Celeeno · 2025-09-01 08:41
@__tinygrad__ I can get 4 RTX 5090s for 10k and the CPU for 3k. Where is the rest of the money going?

@__tinygrad__ · 2025-08-31 22:09
tinybox is the best performance per $ of any box you can buy. Sure, you can buy a Mac Studio or a DGX Spark, but the performance is over 10x worse on FLOPS and RAM bandwidth. You get what you pay for.

@LenSeaside · 2025-09-01 10:19
@__tinygrad__ I don't understand what 144GB gives you that 24GB doesnt. It's still a shit, small model. You'll only use it for debugging something you'll end up sending to a cluster anyway surely. Not for anything actually useful. Maybe image models I guess.

@__tinygrad__ · 2025-09-01 02:10
@0xCodyS Show me the software doing *anything* on that box. People don't even buy the red boxes from us, and AMD's software is nowhere close to as bad as Tenstorrent's.

@0xCodyS · 2025-09-01 13:16
@__tinygrad__ This is a great litmus test for the maturity of @__tinygrad__!

@0xCodyS · 2025-09-01 13:16
@__tinygrad__ This is a great litmus test for the maturity of @__tinygrad__!

@__tinygrad__ · 2025-09-01 15:45
@0xCodyS Someone would have to write the port. @tenstorrent should focus on a CUDA like layer with a simple API, then a port is more doable. But currently they don't have a real compiler and have a C++21 API that's nonportable and hard to link against.

@LenSeaside · 2025-09-01 10:19
@__tinygrad__ I don't understand what 144GB gives you that 24GB doesnt. It's still a shit, small model. You'll only use it for debugging something you'll end up sending to a cluster anyway surely. Not for anything actually useful. Maybe image models I guess.

@__tinygrad__ · 2025-09-01 17:56
@LenSeaside gpt-oss-120b is better than a lot of the cloud offerings. Cloud is only going to get worse as the magic VC dollars dry up and inference costs need to come down.

@_Celeeno · 2025-09-01 08:41
@__tinygrad__ I can get 4 RTX 5090s for 10k and the CPU for 3k. Where is the rest of the money going?

@__tinygrad__ · 2025-09-01 19:04
@_Celeeno Now add the motherboard, the RAM, the case, the PCIe extenders, the 4 SSD raid array, boot drive, the two power supplies, all the cables, fans, assembly+testing, tariffs, and 30% margin, and you'll get to 29k. Ever seen a tree in a forest and be like wow why is desk expensive?

@__tinygrad__ · 2025-09-01 19:08
If you can build a tinybox for a lot less, you should start a competing business! How do you square the "but I added up the parts" with the fact that tinybox is the cheapest off the shelf option?

@__tinygrad__ · 2025-09-01 21:14
@tim_acc It's literally just sand. A 4.28T sand company. https://t.co/2oH5gU9MS4

@__tinygrad__ · 2025-09-01 23:34
RT @chiguchansuki: @__tinygrad__ 30k stars! https://t.co/NViZ8MAZ98

@__tinygrad__ · 2025-09-02 16:21
It's the same one JAX uses.

@__tinygrad__ · 2025-09-02 23:41
6 tinybox green v2 in stock ready to ship tomorrow! https://t.co/CMSkxBFbJK

@__tinygrad__ · 2025-09-02 23:43
No! We have a price and a buy it now button! https://t.co/NkZHR2NRto

@DanielleFong · 2025-09-02 23:48
i like that they put the price up front tbf, i like that a lot. still 128 GB VRam, compare one B200 at 141 GB? idk, anybody have opinions please @ me https://t.co/vJP0hHoODT

@DanielleFong · 2025-09-02 23:48
i like that they put the price up front tbf, i like that a lot. still 128 GB VRam, compare one B200 at 141 GB? idk, anybody have opinions please @ me https://t.co/vJP0hHoODT

@__tinygrad__ · 2025-09-02 23:52
@DanielleFong That's going to cost a lot more, even for a machine with just one. Here's gptshop selling B300s for ~$80k. https://t.co/XNevLYfJan

@__tinygrad__ · 2025-09-02 23:52
@DanielleFong That's going to cost a lot more, even for a machine with just one. Here's gptshop selling B300s for ~$80k. https://t.co/XNevLYfJan

@__tinygrad__ · 2025-09-02 23:53
@DanielleFong tinybox green v2 will well outperform a single GH200 btw (their price, $39k)

@__tinygrad__ · 2025-09-02 23:43
No! We have a price and a buy it now button! https://t.co/NkZHR2NRto

@trenchkidzs · 2025-09-03 00:20
@__tinygrad__ People always expect to get a special deal

@trenchkidzs · 2025-09-03 00:20
@__tinygrad__ People always expect to get a special deal

@__tinygrad__ · 2025-09-03 00:21
@trenchkidzs We've considered setting up a "special price" if you want the joy of experiencing a sales process. Maybe 20% more? Still cheaper than our competitors.

@__tinygrad__ · 2025-09-03 00:21
@trenchkidzs We've considered setting up a "special price" if you want the joy of experiencing a sales process. Maybe 20% more? Still cheaper than our competitors.

@0xAlternateGuy · 2025-09-03 02:11
@__tinygrad__ @trenchkidzs Make it 50%, but that’s exactly what you should do. Charge an arbitrary amount for the special enterprise version that is the exact same product and service.

@__tinygrad__ · 2025-09-02 23:41
6 tinybox green v2 in stock ready to ship tomorrow! https://t.co/CMSkxBFbJK

@jizaymes · 2025-09-03 02:59
@__tinygrad__ Can these things work with OpenStack? 🥸

@jizaymes · 2025-09-03 02:59
@__tinygrad__ Can these things work with OpenStack? 🥸

@__tinygrad__ · 2025-09-03 04:34
@jizaymes They are normal computers, so sure? All tinyboxes use server motherboards so you have a BMC too.

@0xAlternateGuy · 2025-09-03 02:11
@__tinygrad__ @trenchkidzs Make it 50%, but that’s exactly what you should do. Charge an arbitrary amount for the special enterprise version that is the exact same product and service.

@__tinygrad__ · 2025-09-03 16:13
@0xAlternateGuy @trenchkidzs Website is open source. We can add "contact us" enterprise edition including 24/7 phone support at 1-800-CHATGPT https://t.co/zezeGDiXLJ

@__tinygrad__ · 2025-09-03 18:00
Apple runs their GPU shader LLVM in a remote process, then there's a 30 second hang if that process crashes, which is frequent! Hey @Apple, why don't you open source it and work to upstream it into mainline LLVM? The stability and quality will pay off. https://t.co/xXXcK9hml6

@__tinygrad__ · 2025-09-04 19:45
How price sensitive is this market? We're in this for the long game, we want to offer compute so cheap that no one can compete. For a limited time, we're dropping our prices. $10k for the red, $25k for the green. That's below cost for the red. Act fast! https://t.co/0LNLABKm0V

@__tinygrad__ · 2025-09-04 19:51
@tetsuoai https://t.co/IUPf8vDinj

@__tinygrad__ · 2025-09-04 19:45
How price sensitive is this market? We're in this for the long game, we want to offer compute so cheap that no one can compete. For a limited time, we're dropping our prices. $10k for the red, $25k for the green. That's below cost for the red. Act fast! https://t.co/0LNLABKm0V

@dhtikna · 2025-09-04 19:52
@__tinygrad__ Why would you do that for the red?

@tetsuoai · 2025-09-04 19:52
You can never have enough compute. Ordering Green 🟩. https://t.co/MWvJzbDjIT

@JzunStathat · 2025-09-04 19:53
@tetsuoai Wow Can I install Linux there ?! 😅

@__tinygrad__ · 2025-09-04 19:45
How price sensitive is this market? We're in this for the long game, we want to offer compute so cheap that no one can compete. For a limited time, we're dropping our prices. $10k for the red, $25k for the green. That's below cost for the red. Act fast! https://t.co/0LNLABKm0V

@KinvertOG · 2025-09-04 19:54
@__tinygrad__ How easy is it to downgrade Ubuntu and keep compatibility with everything you have installed there? Isaac Sim for example is 20 or 22.

@dhtikna · 2025-09-04 19:52
@__tinygrad__ Why would you do that for the red?

@__tinygrad__ · 2025-09-04 19:54
@dhtikna There's only 1 left in stock.

@dhtikna · 2025-09-04 19:52
@__tinygrad__ Why would you do that for the red?

@__tinygrad__ · 2025-09-04 19:54
@dhtikna There's only 1 left in stock.

@KinvertOG · 2025-09-04 19:54
@__tinygrad__ How easy is it to downgrade Ubuntu and keep compatibility with everything you have installed there? Isaac Sim for example is 20 or 22.

@__tinygrad__ · 2025-09-04 19:54
@KinvertOG It's a normal computer. Easy.

@__tinygrad__ · 2025-09-04 19:56
RT @tetsuoai: You can never have enough compute. Ordering Green 🟩. https://t.co/MWvJzbDjIT

@JzunStathat · 2025-09-04 19:53
@tetsuoai Wow Can I install Linux there ?! 😅

@__tinygrad__ · 2025-09-04 19:56
@JzunStathat @tetsuoai Do you think it might come with Windows? 😂

@Zodomo · 2025-09-04 19:59
@tiznah @__tinygrad__ NVIDIA RTX PRO 6000 Blackwell GPUs are more cost-effective than this. I'll wait for them to eventually upgrade to support them. I can literally buy two of them and setup a rig to power them for less than $25k and have 64GB of additional VRAM.

@__tinygrad__ · 2025-09-04 20:05
@zodomo @tiznah They aren't even close to as cost-effective. The RAM bandwidth (which is the limiting factor on LLM BS=1 tok/s) is half on the 2xPRO 6000 vs the tinybox green v2.

@__tinygrad__ · 2025-09-04 20:05
@zodomo @tiznah They aren't even close to as cost-effective. The RAM bandwidth (which is the limiting factor on LLM BS=1 tok/s) is half on the 2xPRO 6000 vs the tinybox green v2.

@__tinygrad__ · 2025-09-04 20:07
@zodomo @tiznah The PRO 6000 is the same die as the 5090.

@__tinygrad__ · 2025-09-04 20:10
Do people really think the RTX PRO 6000 is cost effective? It's the same die as a 5090 with the same RAM bandwidth and (mostly) the same FLOPS for 3x the price. Stop falling for big VRAM GB.

@__tinygrad__ · 2025-09-04 20:07
@zodomo @tiznah The PRO 6000 is the same die as the 5090.

@Zodomo · 2025-09-04 20:11
@__tinygrad__ @tiznah Ah good to know. That being said, my prioritizes are size, not speed, which is why it's not a good fit for me. The PRO 6000 MaxQ variant also only draws 300W, so perf/energy cost is dramatically better for a rig I'd be running at home.

@Zodomo · 2025-09-04 20:11
@__tinygrad__ @tiznah Ah good to know. That being said, my prioritizes are size, not speed, which is why it's not a good fit for me. The PRO 6000 MaxQ variant also only draws 300W, so perf/energy cost is dramatically better for a rig I'd be running at home.

@__tinygrad__ · 2025-09-04 20:12
@zodomo @tiznah The energy thing is a scam too! You can underpower a 5090 to achieve the same effect. And the cost of energy is so cheap compared to the cost of the card.

@__tinygrad__ · 2025-09-04 20:10
Do people really think the RTX PRO 6000 is cost effective? It's the same die as a 5090 with the same RAM bandwidth and (mostly) the same FLOPS for 3x the price. Stop falling for big VRAM GB.

@Andres_pq · 2025-09-04 20:12
@__tinygrad__ I almost fall for this trap

@__tinygrad__ · 2025-09-04 20:10
Do people really think the RTX PRO 6000 is cost effective? It's the same die as a 5090 with the same RAM bandwidth and (mostly) the same FLOPS for 3x the price. Stop falling for big VRAM GB.

@mikaackermann9 · 2025-09-04 20:14
@__tinygrad__ What do you recommend: 4x RTX3090 or 1x RTX PRO 6000?

@__tinygrad__ · 2025-09-04 20:10
Do people really think the RTX PRO 6000 is cost effective? It's the same die as a 5090 with the same RAM bandwidth and (mostly) the same FLOPS for 3x the price. Stop falling for big VRAM GB.

@shiels_ai · 2025-09-04 20:15
@__tinygrad__ Power draw? You're getting a much more efficient card no?

@Andres_pq · 2025-09-04 20:12
@__tinygrad__ I almost fall for this trap

@__tinygrad__ · 2025-09-04 20:16
@Andres_pq It's crazy. Then they have that stupid Max-Q edition, which you can do on any normal card by setting a power limit in nvidia-smi. They are selling a worse card but branding it so people think it's good. Probably nobody who knows the space well buys it though.

@shiels_ai · 2025-09-04 20:15
@__tinygrad__ Power draw? You're getting a much more efficient card no?

@__tinygrad__ · 2025-09-04 20:17
@shiels_ai No! Ahhhhh I can't believe people fell for this. It's the same exact die as a 5090. Here's some 5090s throttled to 400W, and you can mod the BIOS or limit clocks if you want 300. https://t.co/mBnLZTxomI

@__tinygrad__ · 2025-09-04 20:10
Do people really think the RTX PRO 6000 is cost effective? It's the same die as a 5090 with the same RAM bandwidth and (mostly) the same FLOPS for 3x the price. Stop falling for big VRAM GB.

@Zodomo · 2025-09-04 20:19
@__tinygrad__ Yes, because larger models simply can't run on 128GB of VRAM anyways. If you want to load these 235B or larger models, it'll just never happen on your devices 🙃

@Zodomo · 2025-09-04 20:19
@__tinygrad__ Yes, because larger models simply can't run on 128GB of VRAM anyways. If you want to load these 235B or larger models, it'll just never happen on your devices 🙃

@__tinygrad__ · 2025-09-04 20:21
@zodomo Sure, you can fit the model, but it would be stupidly slow! If you are filling up the 96GB of RAM you can only access all of it 20 times per second. That's 20 tok/s max for a dense model.

@__tinygrad__ · 2025-09-04 20:10
Do people really think the RTX PRO 6000 is cost effective? It's the same die as a 5090 with the same RAM bandwidth and (mostly) the same FLOPS for 3x the price. Stop falling for big VRAM GB.

@Jilano77 · 2025-09-04 20:21
@__tinygrad__ If my training need >32go does the inter gpu communication overhead (in tinybox green v2 vs rtx 6000) significantly slow down the process in your opinion?

@Jilano77 · 2025-09-04 20:21
@__tinygrad__ If my training need >32go does the inter gpu communication overhead (in tinybox green v2 vs rtx 6000) significantly slow down the process in your opinion?

@__tinygrad__ · 2025-09-04 20:24
@Jilano77 No. The machines ship with a driver mod to support P2P and all the cards are connected on PCIe 5.0. Have never seen an FSDP training job limited by the interconnect on the 4 GPUs, it's possible but only if you are in a weird configuration.

@mikaackermann9 · 2025-09-04 20:14
@__tinygrad__ What do you recommend: 4x RTX3090 or 1x RTX PRO 6000?

@__tinygrad__ · 2025-09-04 20:27
@mikaackermann9 RTX PRO 6000 is the same price as 3x 5090s. Buy that instead. Every PRO 6000 should include a sticker that says "I'm a fool with too much money." I blame @a16z and @Mascobot for this dumb hype.

@__tinygrad__ · 2025-09-04 19:45
How price sensitive is this market? We're in this for the long game, we want to offer compute so cheap that no one can compete. For a limited time, we're dropping our prices. $10k for the red, $25k for the green. That's below cost for the red. Act fast! https://t.co/0LNLABKm0V

@HatforceSec · 2025-09-04 20:28
@__tinygrad__ green v3 suggestion: 2 x RTX Pro 6000 Blackwell Max-Q would have 64GB more VRAM and consume 1700 Watt less, for about 5k USD more total?

@HatforceSec · 2025-09-04 20:28
@__tinygrad__ green v3 suggestion: 2 x RTX Pro 6000 Blackwell Max-Q would have 64GB more VRAM and consume 1700 Watt less, for about 5k USD more total?

@__tinygrad__ · 2025-09-04 20:30
@HatforceSec Ugh. I really hope nobody is buying that overpriced GPU. https://t.co/YtWyXeD4hD

@__tinygrad__ · 2025-09-04 20:34
Should we sell tinybox Max-Q edition? We remove a power supply and power limit but don't update the FLOP numbers. Less cost for us and we sell it as "efficient" No wait, we trust our customers are smarter than that. `sudo nvidia-smi -pl` if you want to limit power.

@thomasjeans · 2025-09-04 20:36
I love @realGeorgeHotz but you need to pivot your marketing message should be less about specs and more about how a private AI supercomputer *directly benefits customers* good news is EVERY enterprise needs your product to empower their teams and protect their proprietary code

@__tinygrad__ · 2025-09-04 20:34
Should we sell tinybox Max-Q edition? We remove a power supply and power limit but don't update the FLOP numbers. Less cost for us and we sell it as "efficient" No wait, we trust our customers are smarter than that. `sudo nvidia-smi -pl` if you want to limit power.

@Zodomo · 2025-09-04 20:37
@__tinygrad__ "Our customers don't want to run big models anyways"

@__tinygrad__ · 2025-09-04 20:34
Should we sell tinybox Max-Q edition? We remove a power supply and power limit but don't update the FLOP numbers. Less cost for us and we sell it as "efficient" No wait, we trust our customers are smarter than that. `sudo nvidia-smi -pl` if you want to limit power.

@__tinygrad__ · 2025-09-04 20:37
I really hope nobody fell for this. They didn't magically make the GPU more efficient. They just lowered power in exchange for lower performance. You can always power limit a GPU. A 600W GPU can run at 300W. But if you bought a 300W GPU, you can't run at 600W.

@Zodomo · 2025-09-04 20:37
@__tinygrad__ "Our customers don't want to run big models anyways"

@__tinygrad__ · 2025-09-04 20:38
@zodomo "I spent $60k and built a golden computer like @a16z and now I can run this HUGE model at 8 tokens/sec!"

@__tinygrad__ · 2025-09-04 20:16
@Andres_pq It's crazy. Then they have that stupid Max-Q edition, which you can do on any normal card by setting a power limit in nvidia-smi. They are selling a worse card but branding it so people think it's good. Probably nobody who knows the space well buys it though.

@SIGKITTEN · 2025-09-04 20:41
@__tinygrad__ @Andres_pq it has different cooling so you can shove 4x in an ATX case

@SIGKITTEN · 2025-09-04 20:41
@__tinygrad__ @Andres_pq it has different cooling so you can shove 4x in an ATX case

@__tinygrad__ · 2025-09-04 20:42
@SIGKITTEN @Andres_pq You can't. This is from the @a16z blog post. It's running at 91C. Even at 300W pathetic watts it's throttling. https://t.co/UsqhmyOEIm

@SIGKITTEN · 2025-09-04 20:41
@__tinygrad__ @Andres_pq it has different cooling so you can shove 4x in an ATX case

@__tinygrad__ · 2025-09-04 20:42
@SIGKITTEN @Andres_pq You can't. This is from the @a16z blog post. It's running at 91C. Even at 300W pathetic watts it's throttling. https://t.co/UsqhmyOEIm

@__tinygrad__ · 2025-09-04 20:44
Ugh. The damage that @a16z / @Mascobot blog post did to the space. Were they sponsored by @nvidia or do they just like wasting money for underpowered throttled GPUs?

@__tinygrad__ · 2025-09-04 20:49
Protip: if it's in a golden case, it's hype and not a real product. 4 overpriced crammed GPUs that throttle at 300W vs 4 GPUs running < 80C at 600W in tinybox green v2. https://t.co/ebrMfo9JEJ

@__tinygrad__ · 2025-09-04 20:10
Do people really think the RTX PRO 6000 is cost effective? It's the same die as a 5090 with the same RAM bandwidth and (mostly) the same FLOPS for 3x the price. Stop falling for big VRAM GB.

@DanielleFong · 2025-09-04 20:52
@__tinygrad__ there’s a pareto surface for compute / vram. big vram is practically on a line GB vs cost. for inference 5090s clearly better perf / $

@__tinygrad__ · 2025-09-04 20:21
@zodomo Sure, you can fit the model, but it would be stupidly slow! If you are filling up the 96GB of RAM you can only access all of it 20 times per second. That's 20 tok/s max for a dense model.

@DanielleFong · 2025-09-04 20:53
@__tinygrad__ @zodomo people are saying this is good enough for background use cases even if sluggish for workflow

@__tinygrad__ · 2025-09-04 20:49
Protip: if it's in a golden case, it's hype and not a real product. 4 overpriced crammed GPUs that throttle at 300W vs 4 GPUs running < 80C at 600W in tinybox green v2. https://t.co/ebrMfo9JEJ

@Zodomo · 2025-09-04 20:54
@__tinygrad__ Yeah but tinygrad cant run Qwen3 235B sadge

@DanielleFong · 2025-09-04 20:53
@__tinygrad__ @zodomo people are saying this is good enough for background use cases even if sluggish for workflow

@__tinygrad__ · 2025-09-04 20:55
@DanielleFong @zodomo I believe this is something people might say to feel good while they are in practice using a cloud API. LLMs below 100 tok/s are unusable.

@Zodomo · 2025-09-04 20:54
@__tinygrad__ Yeah but tinygrad cant run Qwen3 235B sadge

@__tinygrad__ · 2025-09-04 20:57
@zodomo Is this some new @a16z thing to shill instead of crypto coins? gpt-oss-120b is a better model. https://t.co/ytlgPPuCfc

@DanielleFong · 2025-09-04 20:52
@__tinygrad__ there’s a pareto surface for compute / vram. big vram is practically on a line GB vs cost. for inference 5090s clearly better perf / $

@__tinygrad__ · 2025-09-04 21:00
@DanielleFong You can buy more than 3x5090s for the price of one RTX Pro 6000. Even for $/VRAM 5090 is better!

@__tinygrad__ · 2025-09-04 20:57
@zodomo Is this some new @a16z thing to shill instead of crypto coins? gpt-oss-120b is a better model. https://t.co/ytlgPPuCfc

@Zodomo · 2025-09-04 21:03
@__tinygrad__ @a16z Still not going to be able to run Grok 3 when it releases. Some of us like less censored models.

@Zodomo · 2025-09-04 21:03
@__tinygrad__ @a16z Still not going to be able to run Grok 3 when it releases. Some of us like less censored models.

@__tinygrad__ · 2025-09-04 21:06
@zodomo @a16z Neither will 4x RTX Pro 6000, it's a 2.7T parameter model! You are going to need a $250k machine at least to run that.

@__tinygrad__ · 2025-09-04 20:44
Ugh. The damage that @a16z / @Mascobot blog post did to the space. Were they sponsored by @nvidia or do they just like wasting money for underpowered throttled GPUs?

@knowrohit07 · 2025-09-04 21:56
@__tinygrad__ @a16z @Mascobot @nvidia @Mascobot i mean you still have time to fix this and you do know how

@knowrohit07 · 2025-09-04 21:56
@__tinygrad__ @a16z @Mascobot @nvidia @Mascobot i mean you still have time to fix this and you do know how

@__tinygrad__ · 2025-09-04 22:04
@knowrohit07 @a16z @Mascobot @nvidia By returning all the parts and buying 2 tinybox green v2s? Even after restocking fees it will be cheaper.

@__tinygrad__ · 2025-09-04 22:04
@knowrohit07 @a16z @Mascobot @nvidia By returning all the parts and buying 2 tinybox green v2s? Even after restocking fees it will be cheaper.

@knowrohit07 · 2025-09-04 22:06
@__tinygrad__ @a16z @Mascobot @nvidia no. it's us. our way is gonna supersede tiny by multiple folds too. marco knows about it already. you will soon know.

@knowrohit07 · 2025-09-04 22:06
@__tinygrad__ @a16z @Mascobot @nvidia no. it's us. our way is gonna supersede tiny by multiple folds too. marco knows about it already. you will soon know.

@__tinygrad__ · 2025-09-04 22:08
@knowrohit07 @a16z @Mascobot @nvidia lol i didn't know you bought into their fake hype. in b4 just buy this token...

@__tinygrad__ · 2025-09-04 19:45
How price sensitive is this market? We're in this for the long game, we want to offer compute so cheap that no one can compete. For a limited time, we're dropping our prices. $10k for the red, $25k for the green. That's below cost for the red. Act fast! https://t.co/0LNLABKm0V

@SIGKITTEN · 2025-09-04 22:12
@__tinygrad__ make it 20k and ill buy

@SIGKITTEN · 2025-09-04 22:12
@__tinygrad__ make it 20k and ill buy

@__tinygrad__ · 2025-09-04 22:14
@SIGKITTEN lol, then we'd lose money

@__tinygrad__ · 2025-09-04 22:14
@SIGKITTEN lol, then we'd lose money

@SIGKITTEN · 2025-09-04 22:15
@__tinygrad__ only a little bit

@SIGKITTEN · 2025-09-04 22:15
@__tinygrad__ only a little bit

@__tinygrad__ · 2025-09-04 22:18
@SIGKITTEN i checked with our accountants. a little bit on every box would add up to a lot.

@SIGKITTEN · 2025-09-04 22:20
@__tinygrad__ but just 1 for me should be fine

@dixiidev · 2025-09-04 22:21
@SIGKITTEN @__tinygrad__ one for me tooo pls

@dixiidev · 2025-09-04 22:21
@SIGKITTEN @__tinygrad__ one for me tooo pls

@__tinygrad__ · 2025-09-04 22:22
@dixiidev @SIGKITTEN Sanford: Hey, Dante, I'm gonna grab a Gatorade. Dante Hicks: If you grab a Gatorade, then everyone's gonna grab one. Sanford: So? Dante Hicks: So, who's gonna pay for all these Gatorades?

@__tinygrad__ · 2025-09-04 20:49
Protip: if it's in a golden case, it's hype and not a real product. 4 overpriced crammed GPUs that throttle at 300W vs 4 GPUs running < 80C at 600W in tinybox green v2. https://t.co/ebrMfo9JEJ

@grokdimension · 2025-09-04 22:30
@__tinygrad__ lol, yet still I think @pmarca will invest in @__tinygrad__ if needed 😍

@__tinygrad__ · 2025-09-04 20:44
Ugh. The damage that @a16z / @Mascobot blog post did to the space. Were they sponsored by @nvidia or do they just like wasting money for underpowered throttled GPUs?

@yacineMTB · 2025-09-04 22:40
@__tinygrad__ @a16z @Mascobot @nvidia the more money you spend.. the more money you save.. or something

@__tinygrad__ · 2025-09-04 23:30
If @a16z is serious about AI, they shouldn't be wasting time with screwdrivers, loud blower fans, PCIe AER errors, and getting their box to cool properly. They should just buy tinybox green v2s. Now only $25k.

@yacineMTB · 2025-09-04 22:40
@__tinygrad__ @a16z @Mascobot @nvidia the more money you spend.. the more money you save.. or something

@__tinygrad__ · 2025-09-04 23:35
@yacineMTB @a16z @Mascobot @nvidia Good point, I forgot about this.

@__tinygrad__ · 2025-09-04 23:46
@tylertravisty @a16z Ahh, they do have a partnership with NVIDIA

@__tinygrad__ · 2025-09-04 19:54
@dhtikna There's only 1 left in stock.

@__tinygrad__ · 2025-09-05 01:52
@dhtikna ...and it's gone!

@__tinygrad__ · 2025-09-05 02:50
Large RAM is for ML posers: my HUGE model runs! (at 6 tok/s) Large memory bandwidth is for ML users: my HUGE model runs FAST! (200+ tok/s) Large FLOPS is for ML creators: I made this model from DATA and RANDOMNESS

@__tinygrad__ · 2025-09-05 03:18
As dtypes get smaller, what FLOPS do people care the most about?

@__tinygrad__ · 2025-09-04 19:45
How price sensitive is this market? We're in this for the long game, we want to offer compute so cheap that no one can compete. For a limited time, we're dropping our prices. $10k for the red, $25k for the green. That's below cost for the red. Act fast! https://t.co/0LNLABKm0V

@QuixiAI · 2025-09-05 03:38
@__tinygrad__ Aww crap it's sold out 😭

@__tinygrad__ · 2025-09-05 03:39
All tinyboxes ship with so called "Max-Q" technology, including red. tinyboxes can run from one outlet at the cost of a bit of perf. Power limiting instructions are in the docs. https://t.co/uFdcfesv3L https://t.co/hM88EVXlkC

@QuixiAI · 2025-09-05 03:38
@__tinygrad__ Aww crap it's sold out 😭

@__tinygrad__ · 2025-09-05 03:40
@QuixiAI Just the red is. Green still in stock. That deal for the red was too good to be true, I was just sick of looking at the last box.

@__tinygrad__ · 2025-09-04 20:24
@Jilano77 No. The machines ship with a driver mod to support P2P and all the cards are connected on PCIe 5.0. Have never seen an FSDP training job limited by the interconnect on the 4 GPUs, it's possible but only if you are in a weird configuration.

@adhik_Joshi · 2025-09-05 04:01
@__tinygrad__ @Jilano77 Is this driver open-source? Can i install on my system?

@adhik_Joshi · 2025-09-05 04:01
@__tinygrad__ @Jilano77 Is this driver open-source? Can i install on my system?

@__tinygrad__ · 2025-09-05 04:02
@adhik_Joshi @Jilano77 Yes and yes. https://t.co/XvN6U8w6M3

@tuxrcr · 2025-09-05 04:07
.@__tinygrad__ @realGeorgeHotz can u ship a tinybox to cmu so my prof and i can dissect it

@tuxrcr · 2025-09-05 04:07
.@__tinygrad__ @realGeorgeHotz can u ship a tinybox to cmu so my prof and i can dissect it

@__tinygrad__ · 2025-09-05 04:08
@c0mputerist @realGeorgeHotz For just $25k, of course. Order on the website, ships tomorrow!

@__tinygrad__ · 2025-09-05 15:20
4 orders in 24 hours. Maybe the new prices are *too* low

@__tinygrad__ · 2025-09-05 15:20
4 orders in 24 hours. Maybe the new prices are *too* low

@MichaelAshmead · 2025-09-05 15:23
@__tinygrad__ Wait what?

@MichaelAshmead · 2025-09-05 15:23
@__tinygrad__ Wait what?

@__tinygrad__ · 2025-09-05 15:25
@MichaelAshmead Our target was selling 4 per week. If this keeps up, we'll have to expand the factory.

@__tinygrad__ · 2025-09-04 20:49
Protip: if it's in a golden case, it's hype and not a real product. 4 overpriced crammed GPUs that throttle at 300W vs 4 GPUs running < 80C at 600W in tinybox green v2. https://t.co/ebrMfo9JEJ

@charliebailey24 · 2025-09-05 17:05
@__tinygrad__ Does your green v2 run on a 120V/15A circuit though 👀 that’s literally the only barrier to my buying rn

@charliebailey24 · 2025-09-05 17:05
@__tinygrad__ Does your green v2 run on a 120V/15A circuit though 👀 that’s literally the only barrier to my buying rn

@__tinygrad__ · 2025-09-05 17:08
@charliebailey24 Yes, you'll have to power limit it, but limited it will still have more perf than 2x RTX6000! https://t.co/uFdcferXed

@__tinygrad__ · 2025-09-05 18:47
It's been kind of quiet on the tinygrad front, and here's why. In retrospect, ShapeTracker was a huge mistake. And now we are paying with months of refactors. But there's a good path forward, and ShapeTracker/View will be completely deleted. The problem: ShapeTracker doesn't allow you to split shapes vertically (between axes). When you are optimizing, this is very powerful. You might want part of your shape to be MULTI, part LOOP, part GLOBAL, part to be the WARP, part UPCAST, etc... And you want to be able to reason about this at every scale of the graph. The LOCAL scale should not even be aware of that it might be enclosed in GLOBAL or MULTI. The UPCAST scale should not be aware it's in LOCAL. There's no way to do this with ShapeTracker. The path forward is two new abstractions in the codebase, RANGEIFY and POSTOPT. You can enable them on master with RANGEIFY=1 (implies POSTOPT) and POSTOPT=2. RANGEIFY creates ranges on full model (even full training job) graphs. This ranges are independent, so the shape is fully split vertically at this point. For example, the ranges describing the MULTI axes are not visible to the Kernels. RANGEIFY is still very slow on most graphs since it always stores and never does compute twice. Reading Halide has done so much to explain the tradeoffs here, and the new stuff should support everything Halide does. POSTOPT is the new optimizer (TVM calls this Schedule) which now has to run on the graph with ranges instead of ShapeTrackers. But the abstractions have gotten good enough that this is even less complex and more robust than the one that ran on ShapeTrackers. There's both a heuristics and a BEAM search optimizer that share an action space. Flash Attention is really the ultimate test. There's so much you have to get right to make it fast. But the core idea is the vertical splitting (this is staying in SRAM), and that should be fully functional in 2 months. We should have the highest performing GEMMs and Flash Attention kernels on MI350X by the end of the year, found by BEAM search. Sometimes to go forward you have to step back. If you want to help, join our Discord. There's lots of tests to refactor to use the new stuff and lots of performance to optimize. Once these refactors are done, we'll document this. And finally start showing off ranges on the overworld graph, aka spec a whole training job to run.

@__tinygrad__ · 2025-09-05 18:47
It's been kind of quiet on the tinygrad front, and here's why. In retrospect, ShapeTracker was a huge mistake. And now we are paying with months of refactors. But there's a good path forward, and ShapeTracker/View will be completely deleted. The problem: ShapeTracker doesn't allow you to split shapes vertically (between axes). When you are optimizing, this is very powerful. You might want part of your shape to be MULTI, part LOOP, part GLOBAL, part to be the WARP, part UPCAST, etc... And you want to be able to reason about this at every scale of the graph. The LOCAL scale should not even be aware of that it might be enclosed in GLOBAL or MULTI. The UPCAST scale should not be aware it's in LOCAL. There's no way to do this with ShapeTracker. The path forward is two new abstractions in the codebase, RANGEIFY and POSTOPT. You can enable them on master with RANGEIFY=1 (implies POSTOPT) and POSTOPT=2. RANGEIFY creates ranges on full model (even full training job) graphs. This ranges are independent, so the shape is fully split vertically at this point. For example, the ranges describing the MULTI axes are not visible to the Kernels. RANGEIFY is still very slow on most graphs since it always stores and never does compute twice. Reading Halide has done so much to explain the tradeoffs here, and the new stuff should support everything Halide does. POSTOPT is the new optimizer (TVM calls this Schedule) which now has to run on the graph with ranges instead of ShapeTrackers. But the abstractions have gotten good enough that this is even less complex and more robust than the one that ran on ShapeTrackers. There's both a heuristics and a BEAM search optimizer that share an action space. Flash Attention is really the ultimate test. There's so much you have to get right to make it fast. But the core idea is the vertical splitting (this is staying in SRAM), and that should be fully functional in 2 months. We should have the highest performing GEMMs and Flash Attention kernels on MI350X by the end of the year, found by BEAM search. Sometimes to go forward you have to step back. If you want to help, join our Discord. There's lots of tests to refactor to use the new stuff and lots of performance to optimize. Once these refactors are done, we'll document this. And finally start showing off ranges on the overworld graph, aka spec a whole training job to run.

@__tinygrad__ · 2025-09-05 18:55
tinygrad has gone through this many times before. We used to have a separate symbolic algebra system for ranges on axes, now that's gone and the normal symbolic used in all the graph rewrites is good enough. We used to have a class called LazyBuffer to create all these virtual Buffers that could be realized, now this is all just UOps. We used to have a MultiLazyBuffer class for multi GPU (like DTensor), now this is just UOps. (though there's still multi specific rewrites, this will just be a range soon) Going way back, we used to have Conv2D and MatMul as manual gradients. Now this is unthinkable and the whole gradient implementation is now 56 lines. May the ShapeTracker and View be banished to the annals of the commit history. The final implementation will sparkle beyond belief. Software has no visible skill ceiling.

@__tinygrad__ · 2025-09-05 18:47
It's been kind of quiet on the tinygrad front, and here's why. In retrospect, ShapeTracker was a huge mistake. And now we are paying with months of refactors. But there's a good path forward, and ShapeTracker/View will be completely deleted. The problem: ShapeTracker doesn't allow you to split shapes vertically (between axes). When you are optimizing, this is very powerful. You might want part of your shape to be MULTI, part LOOP, part GLOBAL, part to be the WARP, part UPCAST, etc... And you want to be able to reason about this at every scale of the graph. The LOCAL scale should not even be aware of that it might be enclosed in GLOBAL or MULTI. The UPCAST scale should not be aware it's in LOCAL. There's no way to do this with ShapeTracker. The path forward is two new abstractions in the codebase, RANGEIFY and POSTOPT. You can enable them on master with RANGEIFY=1 (implies POSTOPT) and POSTOPT=2. RANGEIFY creates ranges on full model (even full training job) graphs. This ranges are independent, so the shape is fully split vertically at this point. For example, the ranges describing the MULTI axes are not visible to the Kernels. RANGEIFY is still very slow on most graphs since it always stores and never does compute twice. Reading Halide has done so much to explain the tradeoffs here, and the new stuff should support everything Halide does. POSTOPT is the new optimizer (TVM calls this Schedule) which now has to run on the graph with ranges instead of ShapeTrackers. But the abstractions have gotten good enough that this is even less complex and more robust than the one that ran on ShapeTrackers. There's both a heuristics and a BEAM search optimizer that share an action space. Flash Attention is really the ultimate test. There's so much you have to get right to make it fast. But the core idea is the vertical splitting (this is staying in SRAM), and that should be fully functional in 2 months. We should have the highest performing GEMMs and Flash Attention kernels on MI350X by the end of the year, found by BEAM search. Sometimes to go forward you have to step back. If you want to help, join our Discord. There's lots of tests to refactor to use the new stuff and lots of performance to optimize. Once these refactors are done, we'll document this. And finally start showing off ranges on the overworld graph, aka spec a whole training job to run.

@HotAisle · 2025-09-05 18:56
@__tinygrad__ If I remember correctly, your AMD contract had KPI's in it for payment. Does this set that back now as well?

@HotAisle · 2025-09-05 18:56
@__tinygrad__ If I remember correctly, your AMD contract had KPI's in it for payment. Does this set that back now as well?

@__tinygrad__ · 2025-09-05 18:58
@HotAisle Better to do it right than write hacks. We have two MLPerfs to attempt our submission, we should be on track for the second. Without a very good Flash Attention we weren't going to succeed anyway.

@shaurizard · 2025-09-05 23:58
only now did I realize after seeing the tinybox irl that it is not tiny for some reason I was expecting this to be like, mini-itx sized, don't ask me why, wasn't using my brain https://t.co/x4duweeecq

@__tinygrad__ · 2025-09-06 00:22
The Kernel class is gone. Now we have the Scheduler class. Optimization is done with 0 ShapeTrackers, all with rewrite rules. And easy to see with VIZ=1 https://t.co/gzAoZ8iVXH

@__tinygrad__ · 2025-09-06 14:08
RT @comma_ai: #newproduct https://t.co/bcEo9j4Hc1

@__tinygrad__ · 2025-09-06 17:51
Do 5090s and RTX PRO 6000s have a hardware defect? We've looked into this and can't find a fix. tl;dr the cards can get into a state where they don't listen to reset. https://t.co/7HgpBfn8Nd

@__tinygrad__ · 2025-09-06 17:51
Do 5090s and RTX PRO 6000s have a hardware defect? We've looked into this and can't find a fix. tl;dr the cards can get into a state where they don't listen to reset. https://t.co/7HgpBfn8Nd

@Chick3nman512 · 2025-09-06 17:59
@__tinygrad__ Looks like an issue I’ve always heard referred to as an “ASIC hang”, which has been a problem for several previous generations as well. Getting cards into this state can be a little random but I’ve found ways to induce it before, getting them out is always a power cycle/reboot.

@__tinygrad__ · 2025-09-06 17:51
Do 5090s and RTX PRO 6000s have a hardware defect? We've looked into this and can't find a fix. tl;dr the cards can get into a state where they don't listen to reset. https://t.co/7HgpBfn8Nd

@Proziam · 2025-09-06 18:05
@__tinygrad__ Is it possible that this is related to why some mining rigs have annoying issues that require power cycling? I know quite a few folks with homelab miners that have had to add remote power cycling to their setups to accommodate for this If so, this goes back to 3090 or earlier

@Chick3nman512 · 2025-09-06 17:59
@__tinygrad__ Looks like an issue I’ve always heard referred to as an “ASIC hang”, which has been a problem for several previous generations as well. Getting cards into this state can be a little random but I’ve found ways to induce it before, getting them out is always a power cycle/reboot.

@__tinygrad__ · 2025-09-06 18:11
@Chick3nman512 We have not seen this with the 4090s, just 5090s.

@Proziam · 2025-09-06 18:05
@__tinygrad__ Is it possible that this is related to why some mining rigs have annoying issues that require power cycling? I know quite a few folks with homelab miners that have had to add remote power cycling to their setups to accommodate for this If so, this goes back to 3090 or earlier

@__tinygrad__ · 2025-09-06 18:37
@Proziam All tinyboxes have BMCs, so they are easy enough to power cycle. But that's not the point, @nvidia should address this. It might be a firmware update to fix, or it might point to deeper problems.

@__tinygrad__ · 2025-09-06 17:51
Do 5090s and RTX PRO 6000s have a hardware defect? We've looked into this and can't find a fix. tl;dr the cards can get into a state where they don't listen to reset. https://t.co/7HgpBfn8Nd

@edude03 · 2025-09-06 18:48
@__tinygrad__ https://t.co/ejnYeP1vMS looks like they found a fix in software already

@edude03 · 2025-09-06 18:48
@__tinygrad__ https://t.co/ejnYeP1vMS looks like they found a fix in software already

@__tinygrad__ · 2025-09-06 18:51
@edude03 That's not a fix, that's a mitigation to *maybe* prevent the issue. The GPU should never get into a state where it can't be reset.

@__tinygrad__ · 2025-09-06 22:09
We sell specs. If those specs make enterprises feel empowered and protected, that's on them.

@anemll · 2025-09-09 17:49
Wow, ANE inside GPU! fast prefill and decode! https://t.co/6aSJQhqjkU

@__tinygrad__ · 2025-09-10 05:09
This is the right path. Hopefully it has a public API.

@__tinygrad__ · 2025-09-10 05:09
This is the right path. Hopefully it has a public API.

@kommentlezz · 2025-09-10 05:11
@__tinygrad__ Otherwise you could just stream "reverse engineering A19Pro"😎

@kommentlezz · 2025-09-10 05:11
@__tinygrad__ Otherwise you could just stream "reverse engineering A19Pro"😎

@__tinygrad__ · 2025-09-10 05:12
@kommentlezz It's hard enough to make things work when they aren't reverse engineered. The ANE was too annoying to be worth it.

@__tinygrad__ · 2025-09-10 05:09
This is the right path. Hopefully it has a public API.

@mweinbach · 2025-09-10 05:15
@__tinygrad__ Should be auto mapped with metal 4, which supports tensors now

@yabirgb · 2025-09-10 14:55
This post picked almost the best spot to buy AMD. Alpha everywhere https://t.co/RaLe25bCAK

@mweinbach · 2025-09-10 05:15
@__tinygrad__ Should be auto mapped with metal 4, which supports tensors now

@__tinygrad__ · 2025-09-10 23:55
@mweinbach Hmm, I hope there's a better abstraction underlying this. If NHWC is somehow in your hardware, you goofed. https://t.co/kxrrawhmLb

@__tinygrad__ · 2025-09-11 05:26
RT @RajaXg: "tiny" box with 6 Radeons arrived today 💃 https://t.co/HeRvPa3prM

@__tinygrad__ · 2025-09-12 00:51
Once you see the quality of AMD's hardware, all that was left was for them to take software seriously. The gap between the valuation of AMD and NVIDIA doesn't make sense. We will have AMD's software at parity with NVIDIA in 2-3 years if nobody else beats us to it.

@__tinygrad__ · 2025-09-12 00:51
Once you see the quality of AMD's hardware, all that was left was for them to take software seriously. The gap between the valuation of AMD and NVIDIA doesn't make sense. We will have AMD's software at parity with NVIDIA in 2-3 years if nobody else beats us to it.

@AeonCoin · 2025-09-12 01:15
@__tinygrad__ The valuation gap is specifically from being able to deploy within the next 2-3 ears.

@__tinygrad__ · 2025-09-12 00:51
Once you see the quality of AMD's hardware, all that was left was for them to take software seriously. The gap between the valuation of AMD and NVIDIA doesn't make sense. We will have AMD's software at parity with NVIDIA in 2-3 years if nobody else beats us to it.

@Lon · 2025-09-12 01:45
@__tinygrad__ $1,000 says you will get bored before then

@rickc42069 · 2025-09-12 01:10
@__tinygrad__ Excited to see competition. Hoping intel joins too. Their new b50/60 seem like a decent deal, if generally available.

@__tinygrad__ · 2025-09-12 03:09
@rickc42069 I would not be bullish on Intel. https://t.co/9FcM5tm8v1

@__tinygrad__ · 2025-09-12 03:09
@rickc42069 I would not be bullish on Intel. https://t.co/9FcM5tm8v1

@__tinygrad__ · 2025-09-12 03:11
@rickc42069 "At least with AMD, you can try installing ROCm from the PyTorch homepage and be frustrated when there are bugs. Every time I have managed even to find which piece of Intel software I was supposed to use, I can’t recall getting the import to work without a segfault"

@Lon · 2025-09-12 01:45
@__tinygrad__ $1,000 says you will get bored before then

@__tinygrad__ · 2025-09-12 03:16
@Lon Done. What are the terms? Been working on this for over 2 years to date. https://t.co/KOvuw7CFpq

@joefioti · 2025-09-12 03:27
MLP (GLU) in 3 kernels discovered automatically. pretty cool to see it group the up and gate matmuls in a single kernel, fully fuse the swish and gate multiply, and add the last matmul all with tensor cores! https://t.co/AV9ujHvudK

@AeonCoin · 2025-09-12 01:15
@__tinygrad__ The valuation gap is specifically from being able to deploy within the next 2-3 ears.

@__tinygrad__ · 2025-09-12 04:10
@AeonCoin This just isn't how AI is going to work. There's this really dumb FOMO mindset that's nonsense in reality. Most of the value AI will create is still over 10 years away. We're at the computers in the 80s phase.

@__tinygrad__ · 2025-09-12 03:11
@rickc42069 "At least with AMD, you can try installing ROCm from the PyTorch homepage and be frustrated when there are bugs. Every time I have managed even to find which piece of Intel software I was supposed to use, I can’t recall getting the import to work without a segfault"

@harlanv11 · 2025-09-12 07:57
@__tinygrad__ @rickc42069 equally bullish on AMD and invested about a year and a half ago. i’m such a believer i’m even trying to learn ROCm 😅

@harlanv11 · 2025-09-12 07:57
@__tinygrad__ @rickc42069 equally bullish on AMD and invested about a year and a half ago. i’m such a believer i’m even trying to learn ROCm 😅

@__tinygrad__ · 2025-09-12 08:03
@harlanv11 @rickc42069 I've come around on the viability of ROCm as a strategy. By design, it can never be *better* than CUDA. However, if NVIDIA stumbles for a hardware generation and AMD's hardware is better, then ROCm works.

@__tinygrad__ · 2025-09-12 08:42
Check out that fusion from RANGEIFY=1 ['max_pool2d', 'batchnorm', 'rsqrt', 'relu', 'conv2d', 'add'] https://t.co/TVRjSmZnmP

@__tinygrad__ · 2025-09-12 08:52
@aeliasen @grok can you explain it?

@__tinygrad__ · 2025-09-12 00:51
Once you see the quality of AMD's hardware, all that was left was for them to take software seriously. The gap between the valuation of AMD and NVIDIA doesn't make sense. We will have AMD's software at parity with NVIDIA in 2-3 years if nobody else beats us to it.

@MrSleezebag1964 · 2025-09-12 12:29
@__tinygrad__ Agree. See exhibit A from @highyieldYT. on same node, AMD's transistor density is quite impressive. 18% more transistors on a chip with less surface area. https://t.co/y3X65H4lqc

@MrSleezebag1964 · 2025-09-12 12:29
@__tinygrad__ Agree. See exhibit A from @highyieldYT. on same node, AMD's transistor density is quite impressive. 18% more transistors on a chip with less surface area. https://t.co/y3X65H4lqc

@__tinygrad__ · 2025-09-12 23:13
@MrSleezebag1964 @highyieldYT If only they made a big card...hoping for big RDNA5 with 512-bit GDDR7 bus, for...$1200?

@joefioti · 2025-09-12 03:27
MLP (GLU) in 3 kernels discovered automatically. pretty cool to see it group the up and gate matmuls in a single kernel, fully fuse the swish and gate multiply, and add the last matmul all with tensor cores! https://t.co/AV9ujHvudK

@__tinygrad__ · 2025-09-12 23:17
@joefioti Can you use the locals at the same time? One of the problems with the `simdgroup_load` things is they abstract stuff that shouldn't be abstracted. Which thread is loading from where?

@divij_dawar · 2025-09-13 07:45
this is what i shall be spending my next few months on. i truly believe in the mission of @__tinygrad__ and want to be a part of it. https://t.co/U8miIEmazq

@__tinygrad__ · 2025-09-13 09:44
If tinygrad succeeds, the effects will ripple far beyond a Tensor library. Can ~15k lines replace a 15 million line stack? Is software three orders of magnitude too large?

@__tinygrad__ · 2025-09-13 09:44
If tinygrad succeeds, the effects will ripple far beyond a Tensor library. Can ~15k lines replace a 15 million line stack? Is software three orders of magnitude too large?

@__tinygrad__ · 2025-09-13 09:51
Was testing today on termux on Z Fold 7, tinygrad works on both CPU and GPU there. tinygrad works on Windows. tinygrad includes a full kernel driver replacement for AMD and NVIDIA. If you do things correctly with abstractions that don't leak, software does not have to be large.

@__tinygrad__ · 2025-09-13 09:51
Was testing today on termux on Z Fold 7, tinygrad works on both CPU and GPU there. tinygrad works on Windows. tinygrad includes a full kernel driver replacement for AMD and NVIDIA. If you do things correctly with abstractions that don't leak, software does not have to be large.

@__tinygrad__ · 2025-09-13 09:54
The main thing that contributes to large software is that projects grow so large that refactoring is no longer meaningfully possible. Instead, a new abstraction layer is added to "box" the complexity. Even in tinygrad, it's extremely hard to delete code. But you have to do it.

@__tinygrad__ · 2025-09-15 04:15
RT @real_deep_ml: You can officially start learning @__tinygrad__ by solving hands on coding challenges directly on our platform. Right now, we’ve launched our very first Tinygrad problem, and we’re excited to expand. If you’d like to contribute and help grow the library of Tinygrad questions, check out our repo!

@__tinygrad__ · 2025-09-15 04:19
After months of studying Halide/TVM, there's a new scheduler coming to tinygrad. It's on master, gated behind RANGEIFY=1 Want to contribute to tinygrad? Fix a broken test with RANGEIFY=1. Bug list in the #scheduler channel on our Discord. VIZ=1 is your debugging friend.

@__tinygrad__ · 2025-09-15 04:34
https://t.co/k6G4QQFdrU

@__tinygrad__ · 2025-09-15 04:59
The most wild thing about the new scheduler, it's less lines / complexity than the previous one. More capability with less complexity should be the goal of all software engineering, everything else is just hacking.

@__tinygrad__ · 2025-09-15 04:34
https://t.co/k6G4QQFdrU

@OctoDb · 2025-09-15 05:14
@__tinygrad__ Ops.EXPAND appears twice on 3rd paragraph

@OctoDb · 2025-09-15 05:14
@__tinygrad__ Ops.EXPAND appears twice on 3rd paragraph

@__tinygrad__ · 2025-09-15 06:37
@OctoDb Good call, one of them should be PERMUTE

@__tinygrad__ · 2025-09-15 04:34
https://t.co/k6G4QQFdrU

@CliffLattner · 2025-09-15 06:50
@__tinygrad__ This is basically just walmart polyhedral compilation. If Cerebras couldn’t do it, and they have Sven verdoolaege, then I’m not optimistic you guys will have better luck.

@__tinygrad__ · 2025-09-15 04:19
After months of studying Halide/TVM, there's a new scheduler coming to tinygrad. It's on master, gated behind RANGEIFY=1 Want to contribute to tinygrad? Fix a broken test with RANGEIFY=1. Bug list in the #scheduler channel on our Discord. VIZ=1 is your debugging friend.

@afgreen781 · 2025-09-15 07:21
@__tinygrad__ Do I need a particular GPU to be useful? It's like to learn and contribute but I only have an ancient one

@afgreen781 · 2025-09-15 07:21
@__tinygrad__ Do I need a particular GPU to be useful? It's like to learn and contribute but I only have an ancient one

@__tinygrad__ · 2025-09-15 08:23
@Ullr781 tinygrad runs on (almost all) CPUs, and I bet it even runs on your GPU through OpenCL.

@CliffLattner · 2025-09-15 06:50
@__tinygrad__ This is basically just walmart polyhedral compilation. If Cerebras couldn’t do it, and they have Sven verdoolaege, then I’m not optimistic you guys will have better luck.

@__tinygrad__ · 2025-09-15 08:27
@CliffLattner What exactly is the issue with polyhedral compilation? Do you think Halide/TVM are missing some key thing? That's not why these people fail. They fail cause they add support for "just one" cuDNN op then just one more then nobody can manage the complexity.

@__tinygrad__ · 2025-09-15 08:37
This misunderstands why software fails. I have never seen software fail because it didn't use fancy enough techniques. Software fails because it collapses under its own weight of complexity. It becomes impossible to extend, and eventually impossible to maintain.

@__tinygrad__ · 2025-09-15 08:37
This misunderstands why software fails. I have never seen software fail because it didn't use fancy enough techniques. Software fails because it collapses under its own weight of complexity. It becomes impossible to extend, and eventually impossible to maintain.

@dhtikna · 2025-09-15 08:42
@__tinygrad__ Software can also fail because the underlying problem is fundamentally hard (halting problem/NP-hardness/etc) is the problem tinygrad is solving easy enough that you dont need breakthrough research? I think tinygrad is not turing complete so that makes me hopeful

@dhtikna · 2025-09-15 08:42
@__tinygrad__ Software can also fail because the underlying problem is fundamentally hard (halting problem/NP-hardness/etc) is the problem tinygrad is solving easy enough that you dont need breakthrough research? I think tinygrad is not turing complete so that makes me hopeful

@__tinygrad__ · 2025-09-15 08:43
@dhtikna The bar is PyTorch/JAX, not perfection.

@__tinygrad__ · 2025-09-15 08:43
@dhtikna The bar is PyTorch/JAX, not perfection.

@dhtikna · 2025-09-15 08:44
@__tinygrad__ I thought the bar was blowing past them by being aware of the bitter lesson

@dhtikna · 2025-09-15 08:44
@__tinygrad__ I thought the bar was blowing past them by being aware of the bitter lesson

@__tinygrad__ · 2025-09-15 08:45
@dhtikna Nah. If we can replicate NVIDIA PyTorch quality on AMD and elsewhere, we'll get a lot of adoption.

@__tinygrad__ · 2025-09-15 08:37
This misunderstands why software fails. I have never seen software fail because it didn't use fancy enough techniques. Software fails because it collapses under its own weight of complexity. It becomes impossible to extend, and eventually impossible to maintain.

@abdimoalim_ · 2025-09-15 08:47
@__tinygrad__ Is the solution to make endless rewrites so as to reduce the cost of maintenance? It appears this scheduler is steps away from existing solutions.

@abdimoalim_ · 2025-09-15 08:47
@__tinygrad__ Is the solution to make endless rewrites so as to reduce the cost of maintenance? It appears this scheduler is steps away from existing solutions.

@__tinygrad__ · 2025-09-15 09:05
@abdimoalim_ What do you think the scheduler is missing? And re: endless rewrites, yes, that's kind of the point of the project. Rewrite over and over and approach perfection.

@__tinygrad__ · 2025-09-15 08:37
This misunderstands why software fails. I have never seen software fail because it didn't use fancy enough techniques. Software fails because it collapses under its own weight of complexity. It becomes impossible to extend, and eventually impossible to maintain.

@trobblet · 2025-09-15 10:11
@__tinygrad__ Grug brained developer remains correct

@trobblet · 2025-09-15 10:11
@__tinygrad__ Grug brained developer remains correct

@__tinygrad__ · 2025-09-15 10:32
@trobblet woah this is real https://t.co/BAZ8ExkYzL

@__tinygrad__ · 2025-09-15 08:27
@CliffLattner What exactly is the issue with polyhedral compilation? Do you think Halide/TVM are missing some key thing? That's not why these people fail. They fail cause they add support for "just one" cuDNN op then just one more then nobody can manage the complexity.

@soumithchintala · 2025-09-15 10:44
@__tinygrad__ @CliffLattner polyhederal search spaces are okay but 1. for DL they dont even need to be poly, just rectangular is usually sufficient 2. the search space is highly discontinuous and non-linear, so the search is timetaking and highly unpredictable as a function of shape size

@soumithchintala · 2025-09-15 10:44
@__tinygrad__ @CliffLattner polyhederal search spaces are okay but 1. for DL they dont even need to be poly, just rectangular is usually sufficient 2. the search space is highly discontinuous and non-linear, so the search is timetaking and highly unpredictable as a function of shape size

@__tinygrad__ · 2025-09-15 12:40
@soumithchintala @CliffLattner We're more than happy with polyhedral if the limitations are that it's more flexible than required but just hard to search! We put so much effort into GPU runtimes so we can do things like test memory access patterns separate from the rest of the kernel. Should help search.

@__tinygrad__ · 2025-09-15 08:27
@CliffLattner What exactly is the issue with polyhedral compilation? Do you think Halide/TVM are missing some key thing? That's not why these people fail. They fail cause they add support for "just one" cuDNN op then just one more then nobody can manage the complexity.

@CliffLattner · 2025-09-15 13:23
I think that by decomposing your IR early you gain generality while losing information about the program. For instance who knows if some reduction was for RMSNorm or Softmax afterwards. Now if you want to plug in an optimized kernel you have to pattern-match a 10-20 node plus subgraph and it gets delicate. Do you do it before or after affine loop fusion, etc. Or your compiler has to be AGI. Not really an issue with polyhedral specifically, but polyhedral takes this idea to the logical conclusion.

@CliffLattner · 2025-09-15 13:23
I think that by decomposing your IR early you gain generality while losing information about the program. For instance who knows if some reduction was for RMSNorm or Softmax afterwards. Now if you want to plug in an optimized kernel you have to pattern-match a 10-20 node plus subgraph and it gets delicate. Do you do it before or after affine loop fusion, etc. Or your compiler has to be AGI. Not really an issue with polyhedral specifically, but polyhedral takes this idea to the logical conclusion.

@__tinygrad__ · 2025-09-15 15:34
@CliffLattner Ahh, we have thrown away the RMSNorm vs softmax distinction long before that. Our patterns and optimizer are not very fragile, tinygrad has a minimal op set. But for kernel speed you do have to search.

@StasBekman · 2025-09-19 16:38
GB200 MAMF benchmarks are in. NVIDIA efficiency continues going down. GB200 is less efficient than B200: 72.9 vs 77.6% for bf16 - I tried both cuda-12.9 and 13.0 - about the same results. Be careful when you make plans based on theoretical TFLOPS https://t.co/m9qGomkTr8 https://t.co/B0ihCo74p5

@yacineMTB · 2025-09-23 04:29
Anthropic is going to lose :/

@elonmusk · 2025-09-23 17:14
@yacineMTB Winning was never in the set of possible outcomes for Anthropic

@yacineMTB · 2025-09-25 16:05
tinyboxes are actually better for reinforcement learning at my scale than h100s honestly. is there a place where you can rent one?

@yacineMTB · 2025-09-26 18:31
my team has made a remarkable amount of progress with 0 outside capital and nothing but elbow grease. on the AI side, total step count is beginning to approach the biggest google pixels to policy run, which was 200b steps in total. really thankful to pufferlib folks for helping

@yacineMTB · 2025-09-26 18:33
definitely big thanks to fal for granting us compute credits because it helped me get to a break through 1 week earlier, which is a million years for me

@yacineMTB · 2025-09-26 18:36
in retrospect i wish i just bought a tinybox when i could because it would have been perfect for what i'm doing right now, but it is what it is. 4090s for me are a lot more efficient than h100

@yacineMTB · 2025-09-26 18:33
definitely big thanks to fal for granting us compute credits because it helped me get to a break through 1 week earlier, which is a million years for me

@__tinygrad__ · 2025-10-09 00:28
RT @tudoroancea: Something quite elegant yet uncommon that I (re)discovered by reading the tinygrad codebase: you can just call methods as functions. https://t.co/fRCX94MX6V

@__tinygrad__ · 2025-10-09 00:28
RT @JunaidAhmed_010: reading through tinygrad’s codebase: -optimizers aren’t just algorithms, they’re design philosophy. -tinygrad fuses params into a single buffer → less overhead, simpler math. -LARS, LAMB, Muon, AdamW… all reduced to small clean abstractions. https://t.co/r0Xd5HR5D0

@__tinygrad__ · 2025-10-09 00:28
RT @divij_dawar: spent the day reading and understanding tinygrad's codebase. i'm blown away by the quality and succinctness of the code.

@__tinygrad__ · 2025-10-09 00:31
Merged RANGEIFY as default yesterday. There's now a lot of cleanup to do, but we are back. The skill ceiling on software is higher than you can imagine. When tinygrad is done, it will be something like 6 tricks and people will be left baffled at how complex everything was.

@__tinygrad__ · 2025-10-09 00:37
Shout out to ThunderKittens for writing simple yet very performant GPU code. We're working on "tinykittens" which uses the same insight but in tinygrad's language. The insight is that GPU "registers" are the wrong primitive and TK's "register tile" is a lot more sensible. https://t.co/gDKwmBBLL0

@__tinygrad__ · 2025-10-09 00:37
Shout out to ThunderKittens for writing simple yet very performant GPU code. We're working on "tinykittens" which uses the same insight but in tinygrad's language. The insight is that GPU "registers" are the wrong primitive and TK's "register tile" is a lot more sensible. https://t.co/gDKwmBBLL0

@__tinygrad__ · 2025-10-09 00:41
The insight in their own words. From: https://t.co/D07jWIeNz7 The beauty of embedding this in tinygrad is that it will immediately work on all of tinygrad's GPU backends. AMD, CUDA, OpenCL, Metal, and QCOM. https://t.co/f62MEGfXur

@__tinygrad__ · 2025-10-09 00:47
Thanks @HazyResearch @simran_s_arora @stuart_sul @spectorb @AaryanSinghal4. Tile registers are the future!

@__tinygrad__ · 2025-10-09 01:02
@HazyResearch @simran_s_arora @stuart_sul @spectorb @AaryanSinghal4 The megakernel stuff is great too. Showing that 2 of the 3 GPU architecture shortcomings are fixable in software. There's so much more insight into what direction AI hardware should take reading your blog posts than from most AI hardware companies.

@__tinygrad__ · 2025-10-09 01:02
@HazyResearch @simran_s_arora @stuart_sul @spectorb @AaryanSinghal4 The megakernel stuff is great too. Showing that 2 of the 3 GPU architecture shortcomings are fixable in software. There's so much more insight into what direction AI hardware should take reading your blog posts than from most AI hardware companies.

@__tinygrad__ · 2025-10-09 01:04
@HazyResearch @simran_s_arora @stuart_sul @spectorb @AaryanSinghal4 (the third shortcoming is that the L2 is centralized when it should be distributed with better message passing between SMs. @tenstorrent gets this right, though I'm curious what message passing stuff will come to GPUs)

@0imalan · 2025-10-09 01:06
holy shit i want a green tinybox from @__tinygrad__ so bad.

@__tinygrad__ · 2025-10-09 02:14
This is the gem of RANGEIFY. When code is good, it just looks like definitions. Here is close to the simplest expression of the meaning of the 6 movement ops. https://t.co/FNTFsptooa

@__tinygrad__ · 2025-10-09 02:14
This is the gem of RANGEIFY. When code is good, it just looks like definitions. Here is close to the simplest expression of the meaning of the 6 movement ops. https://t.co/FNTFsptooa

@dmnsh001 · 2025-10-09 02:18
@__tinygrad__ This is some haskell-looking Python

@dmnsh001 · 2025-10-09 02:18
@__tinygrad__ This is some haskell-looking Python

@__tinygrad__ · 2025-10-09 03:44
@dfghbnmtyu lol cause it uses match/case?

@__tinygrad__ · 2025-10-09 01:04
@HazyResearch @simran_s_arora @stuart_sul @spectorb @AaryanSinghal4 (the third shortcoming is that the L2 is centralized when it should be distributed with better message passing between SMs. @tenstorrent gets this right, though I'm curious what message passing stuff will come to GPUs)

@spectorb · 2025-10-09 04:22
@__tinygrad__ @HazyResearch @simran_s_arora @stuart_sul @AaryanSinghal4 @tenstorrent FWIW we could probably take advantage of this better than we do via cluster dsmem. But we've broadly found it too much of a hassle for the gains -- excepting very specific cases. There's probably juice there, too!

@spectorb · 2025-10-09 04:22
@__tinygrad__ @HazyResearch @simran_s_arora @stuart_sul @AaryanSinghal4 @tenstorrent FWIW we could probably take advantage of this better than we do via cluster dsmem. But we've broadly found it too much of a hassle for the gains -- excepting very specific cases. There's probably juice there, too!

@__tinygrad__ · 2025-10-09 08:06
@spectorb @HazyResearch @simran_s_arora @stuart_sul @AaryanSinghal4 @tenstorrent Oh interesting, I didn't know DSMEM. Fits with the trend of NVIDIA just including all the tricks you could want in hardware. You basically want all the "shared" memory available in an address space to all SMs. Does 5090 have this? Does AMD have anything like it?

@__tinygrad__ · 2025-10-10 04:05
RT @j4orz: good article from triton contributor and prev gpu compiler lead at google, claiming same as @__tinygrad__ you can't tape out without having a framework (the ir) because ml deals with incomplete irs. i.e. tpus only succeed because of jax and xla. https://t.co/ROTuL50MsR

@beaversteever · 2025-10-13 20:00
openai is building its own chips and youre still trying to learn CUDA pick up an fpga and learn

@__tinygrad__ · 2025-10-14 01:31
RT @tudoroancea: I am currently working on an AOT compiler for optimal control. After using MLIR, IREE and XLA, I was wondering how easy it would be to create a AOT pipeline using tinygrad. Turns out you only need ≈100 lines (see extract) https://t.co/Druba05MAB

@__tinygrad__ · 2025-10-14 01:32
RT @TacoJacc: Well, last night I was angry because I fucking hate how bloated pytorch is. Me, trying to push a docker image after doing some changes and having to wait like 2 hours everytime and using over 100GB of space over snd over and over again, I just used tinygrad. Here's how (1/n)

@__tinygrad__ · 2025-10-14 01:35
RT @KreasofAI: We are publishing our open source Liquid Foundation Model 2 written in @__tinygrad__ ! 👇

@__tinygrad__ · 2025-10-14 01:46
Today is a great day to buy one. In stock, ready to ship!

@__tinygrad__ · 2025-10-14 01:46
Today is a great day to buy one. In stock, ready to ship!

@tkanarsky · 2025-10-14 01:56
@__tinygrad__ what happened with the red tinybox? It was there last night, is it canceled or being retooled for Radeon 9xxx series?

@tkanarsky · 2025-10-14 01:56
@__tinygrad__ what happened with the red tinybox? It was there last night, is it canceled or being retooled for Radeon 9xxx series?

@__tinygrad__ · 2025-10-14 01:58
@tkanarsky For now, we only have green. Green outsells red 10 to 1 anyway.

@yacineMTB · 2025-09-26 18:36
in retrospect i wish i just bought a tinybox when i could because it would have been perfect for what i'm doing right now, but it is what it is. 4090s for me are a lot more efficient than h100

@__tinygrad__ · 2025-10-14 01:59
@yacineMTB tinyboxes are cheaper and easier to program for than H100s. Don't regret, buy today!

@__tinygrad__ · 2025-10-14 01:58
@tkanarsky For now, we only have green. Green outsells red 10 to 1 anyway.

@tkanarsky · 2025-10-14 01:59
@__tinygrad__ Aw, boo. Makes sense though. Which 5090s are you putting in there - FEs or some AIB?

@__tinygrad__ · 2025-10-14 01:52
@yacineMTB Just buy one!

@iMuffined · 2025-10-14 03:25
@__tinygrad__ @yacineMTB How about a deal? A juicy marketing one?

@iMuffined · 2025-10-14 03:25
@__tinygrad__ @yacineMTB How about a deal? A juicy marketing one?

@__tinygrad__ · 2025-10-14 03:28
@iMuffined @yacineMTB We don't do marketing. We sell computer.

@__tinygrad__ · 2025-10-14 06:41
err, pick up tinygrad and learn. once you have the framework finished, the chip is easy.

@__tinygrad__ · 2025-10-14 06:41
err, pick up tinygrad and learn. once you have the framework finished, the chip is easy.

@sytelus · 2025-10-14 06:51
@__tinygrad__ You guys should implement nanochat on your stack and show the difference to the world!

@__tinygrad__ · 2025-10-14 08:36
Here are the open bounties. If you know how to make things fast, there's a lot of money on the table. Also some driver work, MLPerf models, and two new backends on offer. https://t.co/WUnUgCL6G2

@sytelus · 2025-10-14 06:51
@__tinygrad__ You guys should implement nanochat on your stack and show the difference to the world!

@__tinygrad__ · 2025-10-14 08:39
@sytelus while it's great that the nanochat README includes the important metric of line count, they forgot to count the lines in their dependencies 🙃 @karpathy

@__tinygrad__ · 2025-10-14 08:39
@sytelus while it's great that the nanochat README includes the important metric of line count, they forgot to count the lines in their dependencies 🙃 @karpathy

@karpathy · 2025-10-14 15:00
I count and report the lines in uv.lock, which is my attempt at a simple 80:20 proxy for this, as it includes all the packages recursively. Open to suggestions! I'd love to measure things like "cognitive complexity" (there was a great blog post on it a few months back that I can't find). Or things like how "bacterial" the code is, per my tweet a few months ago too. tiny already knows that I'm not a huge fan of excessive line count maxxing .min.py style specifically.

@__tinygrad__ · 2025-10-14 08:36
Here are the open bounties. If you know how to make things fast, there's a lot of money on the table. Also some driver work, MLPerf models, and two new backends on offer. https://t.co/WUnUgCL6G2

@sampullara · 2025-10-14 17:30
@__tinygrad__ I think unless the person is doing these to prove they should get a job at tiny they are severely underpriced

@sampullara · 2025-10-14 17:30
@__tinygrad__ I think unless the person is doing these to prove they should get a job at tiny they are severely underpriced

@JoshuaCarter603 · 2025-10-14 18:02
@sampullara @__tinygrad__ Yeah that was my impression a year or two ago when I checked them out thoroughly. It wasn't really clear how they factored into a full-time job, and by themselves the compensation wasn't worth it. Probably a great opportunity for those earlier in their careers though.

@petergostev · 2025-10-14 22:46
Nvidia's DGX Spark goes on sale today and @lmsysorg have done a brilliant bit of benchmarking vs other systems. In short, it is a very usable system for smaller models and closer in performance to Apple's devices (e.g. Mac Mini M4 Pro), but it is priced at $4,000 vs $1,400 for Mac Mini M4 Pro. It is also not as powerful as more dedicated GPUs like RTX 5090. Worth reading & watching the full review in the original post.

@sampullara · 2025-10-14 17:30
@__tinygrad__ I think unless the person is doing these to prove they should get a job at tiny they are severely underpriced

@__tinygrad__ · 2025-10-15 00:18
@sampullara Once you understand the codebase, you should be able to do the $1,000 ones in a few days and the cheap ones in a few hours.

@__tinygrad__ · 2025-10-15 00:18
@sampullara Once you understand the codebase, you should be able to do the $1,000 ones in a few days and the cheap ones in a few hours.

@millpreetk · 2025-10-15 00:20
@__tinygrad__ @sampullara Except some of them require access to specific equipment of course

@JoshuaCarter603 · 2025-10-14 18:02
@sampullara @__tinygrad__ Yeah that was my impression a year or two ago when I checked them out thoroughly. It wasn't really clear how they factored into a full-time job, and by themselves the compensation wasn't worth it. Probably a great opportunity for those earlier in their careers though.

@__tinygrad__ · 2025-10-15 00:20
@JoshuaCarter603 @sampullara If you use words like opportunity and career, tinygrad probably isn't the right fit for you.

@millpreetk · 2025-10-15 00:20
@__tinygrad__ @sampullara Except some of them require access to specific equipment of course

@__tinygrad__ · 2025-10-15 00:22
@millpreetk @sampullara For many of them you don't, but we give known contributors to tinygrad access to tinyboxes for the ones that do.

@karpathy · 2025-10-14 15:00
I count and report the lines in uv.lock, which is my attempt at a simple 80:20 proxy for this, as it includes all the packages recursively. Open to suggestions! I'd love to measure things like "cognitive complexity" (there was a great blog post on it a few months back that I can't find). Or things like how "bacterial" the code is, per my tweet a few months ago too. tiny already knows that I'm not a huge fan of excessive line count maxxing .min.py style specifically.

@__tinygrad__ · 2025-10-15 00:30
@karpathy @sytelus Yea this is a cool way to think about it. One of the things we repeat a lot is that we want our code pieces to combine with + and not * (like you want dtype+operations kernels, not dtype*operations) https://t.co/wHRDIH65aF

@beaversteever · 2025-10-14 11:25
@__tinygrad__ you guys wanna write the kernel and compiler for our custom chips?

@__tinygrad__ · 2025-10-15 01:28
@beaversteever what we have found is that it's best to work with chips that are already taped out and have a buy it now button, like NVIDIA/AMD/Apple/Qualcomm GPUs. are you at that stage?

@__tinygrad__ · 2025-10-15 01:28
@beaversteever what we have found is that it's best to work with chips that are already taped out and have a buy it now button, like NVIDIA/AMD/Apple/Qualcomm GPUs. are you at that stage?

@__tinygrad__ · 2025-10-15 01:32
@beaversteever in general, the hardware is good enough. 5x performance gains are still possible for many workloads on GPUs with better software. the only gains we see in custom chips is cheap. no HBM, no CoWoS, no 80% profit margin. LPDDR+PCB+20%

@tkanarsky · 2025-10-14 01:59
@__tinygrad__ Aw, boo. Makes sense though. Which 5090s are you putting in there - FEs or some AIB?

@__tinygrad__ · 2025-10-15 02:41
@tkanarsky PNY. In our testing they were the best.

@__tinygrad__ · 2025-10-15 03:38
RT @beaversteever: what tinycorp means is: - most modern day hardware is already amazing - most modern day software is terrible that's why they write custom drivers for nvidia chips

@__tinygrad__ · 2025-10-15 08:26
AKA don't buy a DGX Spark, buy a 5090 instead for half the price and 5x the performance. Would it help if you could use it from your Mac over USB?

@__tinygrad__ · 2025-10-15 08:26
AKA don't buy a DGX Spark, buy a 5090 instead for half the price and 5x the performance. Would it help if you could use it from your Mac over USB?

@__tinygrad__ · 2025-10-15 08:31
You can see from this chart why we also think the RTX Pro 6000 is not worth the money. 5x the price for 0 performance gain over a 5090 (do you really want llama 70B at 32 tok/s?). This is why tinyboxes are chock full of 5090s.

@__tinygrad__ · 2025-10-15 08:34
RT @rasmus1610: @tech_optimist https://t.co/CxME1v9sFU This was super helpful :)

@__tinygrad__ · 2025-10-15 08:26
AKA don't buy a DGX Spark, buy a 5090 instead for half the price and 5x the performance. Would it help if you could use it from your Mac over USB?

@NoSingingTV · 2025-10-15 08:40
@__tinygrad__ Still need to buy a PC and 128GB for that AI. The Spark can "up to 200 billion parameters"

@NoSingingTV · 2025-10-15 08:40
@__tinygrad__ Still need to buy a PC and 128GB for that AI. The Spark can "up to 200 billion parameters"

@__tinygrad__ · 2025-10-15 08:44
@NoSingingTV This comment looks like it was written by the model size that runs fast on the Spark 😂

@__tinygrad__ · 2025-10-15 09:49
Thanks to sirhcm, tinygrad now supports all the backends in MESA by rendering to NIR. One of the MESA backends is NAK, and with it, we can compile to SASS. An NVIDIA free stack! https://t.co/hU1wNeqq4g https://t.co/If1ZXR1kOM

@igorpener · 2025-10-15 09:56
@__tinygrad__ Cool! Is the SASS complete or just a reverse engineered best effort? Also, does it support graphics related instructions by any chance?

@igorpener · 2025-10-15 09:56
@__tinygrad__ Cool! Is the SASS complete or just a reverse engineered best effort? Also, does it support graphics related instructions by any chance?

@__tinygrad__ · 2025-10-15 10:06
@igorpener `NV=1 NV_NAK=1 DISABLE_COMPILER_CACHE=1 NAK_DEBUG=print python3 test/test_tiny.py TestTiny.test_plus` will show debugging info. Mesa docs are here: https://t.co/X82Xm5MVbJ

@__tinygrad__ · 2025-10-15 10:08
Thanks to sirhcm, tinygrad now supports all the backends in Mesa by rendering to NIR. One of the Mesa backends is NAK, and with it, we can compile to SASS. An NVIDIA free stack! https://t.co/hU1wNeqq4g https://t.co/SeTXfjCSPB

@__tinygrad__ · 2025-10-15 10:08
Thanks to sirhcm, tinygrad now supports all the backends in Mesa by rendering to NIR. One of the Mesa backends is NAK, and with it, we can compile to SASS. An NVIDIA free stack! https://t.co/hU1wNeqq4g https://t.co/SeTXfjCSPB

@__tinygrad__ · 2025-10-15 10:09
`NV=1 NV_NAK=1 DISABLE_COMPILER_CACHE=1 NAK_DEBUG=print python3 test/test_tiny.py TestTiny.test_plus` will print debug info from the compiler -- watch it get lowered to SASS. https://t.co/k1wPdmTgT5

@awnihannun · 2025-10-15 13:38
I'm super excited about M5. It's going to help a lot with compute-bound workloads in MLX. For example: - Much faster prefill. In other words time-to-first-token will go down. - Faster image / video generation - Faster fine-tuning (LoRA or otherwise) - Higher throughput for batch generation And there is a nice bump in memory bandwidth to improve token generation latency!

@__tinygrad__ · 2025-10-15 14:48
8xAMD MI300X mostly out of the box on nanochat (which is really an amazing repo and I'm excited for my custom chatbot). This is without PYTORCH_TUNABLEOP_ENABLED which I was too impatient for. https://t.co/WydFX8Skq6

@beaversteever · 2025-10-15 19:50
Google open sourcing an energy-efficient AI accelerator (NPU) was not on my bingo card Bonus points: It has a 32-bit RISC-V ISA https://t.co/qkoleSLTtA

@__tinygrad__ · 2025-10-16 01:14
Wait is this the actual Coral?!? If so, I can see how bad my reverse engineering was. https://t.co/DoDDNj3C88

@__tinygrad__ · 2025-10-16 01:14
Wait is this the actual Coral?!? If so, I can see how bad my reverse engineering was. https://t.co/DoDDNj3C88

@__tinygrad__ · 2025-10-16 01:22
It's not, you see, this is the "Coral NPU" that one is the "Coral (Edge TPU)" Wonder how much, if anything, it shares. https://t.co/1V3yi3i8BQ

@never_released · 2025-10-16 02:19
the tinygrad codebase is overwhelmingly dense at places. not convinced that code golfing to fit a line count is such a great idea

@__tinygrad__ · 2025-10-16 02:24
We have both AMD and NVIDIA GPUs working over USB4 on Apple M chips. Sadly, it needs a 100 line dext to map the BARs into user space. We are waiting on Apple to give us the DriverKit entitlement. Do you think they will?

@__tinygrad__ · 2025-10-16 06:43
@never_released We're working on it. Some of the older stuff is bad, but what do you think of files like: https://t.co/l38xZe9eOL

@never_released · 2025-10-16 06:45
@__tinygrad__ that looks better :)

@never_released · 2025-10-16 06:45
@__tinygrad__ that looks better :)

@__tinygrad__ · 2025-10-16 07:14
@never_released sweet! yea some of the old stuff looks like that because we didn't know what we wanted to express. before it's 1.0 the codebase should be all like that or better

@__tinygrad__ · 2025-10-16 07:36
Goodbye ShapeTracker 🫡 https://t.co/LuqqoqMA8N

@tomfleet · 2025-10-16 16:37
Wondered what the Nvidia Spark looked like internally, and... wow. So many DC/DC converters! I count 25+ coils that look like switching blocks, on the front side alone. Also, pretty damn impressive density - both top and bottom sides. https://t.co/5CGU0KU2yx

@__tinygrad__ · 2025-10-16 23:39
RT @comma_ai: Building our way into the GPU middle class https://t.co/hUrVKGmLMu

@tomfleet · 2025-10-16 16:37
Wondered what the Nvidia Spark looked like internally, and... wow. So many DC/DC converters! I count 25+ coils that look like switching blocks, on the front side alone. Also, pretty damn impressive density - both top and bottom sides. https://t.co/5CGU0KU2yx

@__tinygrad__ · 2025-10-17 00:30
@tomfleet It's interesting to see the rectangle dies, 9070XT is like this too. shoreline > area

@awnihannun · 2025-10-15 13:38
I'm super excited about M5. It's going to help a lot with compute-bound workloads in MLX. For example: - Much faster prefill. In other words time-to-first-token will go down. - Faster image / video generation - Faster fine-tuning (LoRA or otherwise) - Higher throughput for batch generation And there is a nice bump in memory bandwidth to improve token generation latency!

@__tinygrad__ · 2025-10-17 00:35
@awnihannun What's the TFLOPS?

@__tinygrad__ · 2025-10-17 00:41
It's not tiny, but it's the easiest way to get a machine capable of running gpt-oss-120b and nanochat (at reasonable speeds) in your house. It has 15x the FP16 FLOPS and 26x the RAM bandwidth for just 7x the price of DGX Spark. Buy today!

@__tinygrad__ · 2025-10-17 00:41
It's not tiny, but it's the easiest way to get a machine capable of running gpt-oss-120b and nanochat (at reasonable speeds) in your house. It has 15x the FP16 FLOPS and 26x the RAM bandwidth for just 7x the price of DGX Spark. Buy today!

@djavaisadog · 2025-10-17 02:12
@__tinygrad__ wtf theyre huge why do you call it that?

@HotAisle · 2025-10-17 05:35
Wild to consider the fact that just idle, 8 of these GPUs are consuming almost ~2000W. https://t.co/dlQJ5RpKTq

@__tinygrad__ · 2025-10-17 09:27
@awnihannun maybe you know this. sometimes I'll write a bad kernel and it'll keep running even after I kill the process. I see tons of GPU power usage in asitop. any way to recover from this besides reboot?

@__tinygrad__ · 2025-10-17 00:41
It's not tiny, but it's the easiest way to get a machine capable of running gpt-oss-120b and nanochat (at reasonable speeds) in your house. It has 15x the FP16 FLOPS and 26x the RAM bandwidth for just 7x the price of DGX Spark. Buy today!

@0imalan · 2025-10-17 09:47
@__tinygrad__ My only worry is I’m buying a project not a production system. Tell me I’m wrong. Please.

@__tinygrad__ · 2025-10-17 09:27
@awnihannun maybe you know this. sometimes I'll write a bad kernel and it'll keep running even after I kill the process. I see tons of GPU power usage in asitop. any way to recover from this besides reboot?

@awnihannun · 2025-10-17 13:11
@__tinygrad__ No, I run into the same issue from time to time. Right now reboot is the only way to recover 🙁

@__tinygrad__ · 2025-10-17 08:09
@HotAisle I guess most people don't care, but AMD should improve this!

@HotAisle · 2025-10-17 13:45
@__tinygrad__ Most people don’t care?!

@awnihannun · 2025-10-17 13:11
@__tinygrad__ No, I run into the same issue from time to time. Right now reboot is the only way to recover 🙁

@__tinygrad__ · 2025-10-17 14:22
@awnihannun it's quite frustrating, i reboot probably 3 times a day. gotta be some low level reset of the GPU around somewhere...

@HotAisle · 2025-10-17 13:45
@__tinygrad__ Most people don’t care?!

@__tinygrad__ · 2025-10-17 14:27
@HotAisle I imagine most datacenters are buying power on a fixed contract where price is the same if they use it or not, true?

@__tinygrad__ · 2025-10-17 16:10
5090s are up in price on eBay. Is $25k too cheap for the tinybox green v2? Should we raise the price? Only 7 boxes left in stock and I don't know when we'll get more 5090s.

@0imalan · 2025-10-17 09:47
@__tinygrad__ My only worry is I’m buying a project not a production system. Tell me I’m wrong. Please.

@__tinygrad__ · 2025-10-17 16:14
@0imalan We've sold over 100 boxes, it's v2 of the design, and we run 15 of them ourselves. For the market segment, it's probably the best tested product.

@djavaisadog · 2025-10-17 02:12
@__tinygrad__ wtf theyre huge why do you call it that?

@__tinygrad__ · 2025-10-17 16:15
@djavaisadog they are so tiny compared to a datacenter

@__tinygrad__ · 2025-10-17 16:25
many such cases

@ivanfioravanti · 2025-10-17 18:38
How is it possible that AMD can't compete with NVidia? What's the problem there? NVidia is amazing but the price is out of mind (look at their revenues 🤷🏻‍♂️), we need more competitors.

@SemiAnalysis_ · 2025-10-18 00:41
Intel just took another step on combining forces 🔥 with NVIDIA by integrating their new Gaudi3 rack scale systems together with NVIDIA B200 via disaggregated PD inferencing. Intel claims that compared their B200 only baseline, and inferencing system using Gaudi3 for decode part & B200 for prefill part connected over Nvidia ConnectX-7 networking results in an 1.7x better perf per TCO for small dense models. We believe this will be done through integrating Gaudi3 into the Nvidia open source Apache2 Dynamo framework. Intel took their massive warehouse inventory full of Gaudi3 chips that they can't sell and redesigned the Gaudi system level going from 8 chips per scale up world size to 64 chips per scale up domain by using Broadcom 51.2T Tomahawk5 switches for scale up. Note that there is optionality to have even larger scale up domain and over 128+ chips via cross-rack DAC/AEC ethernet cables using the front OSFP cages on the scale up switch tray. In the existing gaudi3 design, the scale out networking was done via Gaudi3's integrated RDMA ethernet NICs but in the new rack scale Gaudi3 design, the scale out is done via NVIDIA 400GbE ConnectX-7 NIC as it allows the Gaudi3 to talk to NVIDIA B200 GPUs. This is 200IQ Jensen Art of the Deal partnership 🚀 by getting Intel's AI chips into the Nvidia networking ecosystem. With all this being said, the Gaudi3 software stack still is immature and closed source and nobody wants to use it, hence the giant inventory. Really the only way Intel would be able to sell Gaudi3 is via an even lower selling ASP. This announcement only helps Gaudi3 at the system level and not the software level. Gaudi3 PyTorch is still closed source and not upstreamed to the open source PyTorch. This is unlike the Intel GPU software stack where the PyTorch integration is somewhat open source & upstreamed, although the Intel GPU/oneAPI/oneDNN has lots of exclusive broken PyTorch unit tests and Intel GPU lacks PyTorch inductor CI integartion. Considering Guadi3 is the end of life for Gaudi architecture, it is a tough choice for Intel to make if they invest even more energy by upstreaming and open sourcing their PyTorch software layer or focus their efforts on Intel GPU software stack.

@SemiAnalysis_ · 2025-10-18 00:41
Intel just took another step on combining forces 🔥 with NVIDIA by integrating their new Gaudi3 rack scale systems together with NVIDIA B200 via disaggregated PD inferencing. Intel claims that compared their B200 only baseline, and inferencing system using Gaudi3 for decode part & B200 for prefill part connected over Nvidia ConnectX-7 networking results in an 1.7x better perf per TCO for small dense models. We believe this will be done through integrating Gaudi3 into the Nvidia open source Apache2 Dynamo framework. Intel took their massive warehouse inventory full of Gaudi3 chips that they can't sell and redesigned the Gaudi system level going from 8 chips per scale up world size to 64 chips per scale up domain by using Broadcom 51.2T Tomahawk5 switches for scale up. Note that there is optionality to have even larger scale up domain and over 128+ chips via cross-rack DAC/AEC ethernet cables using the front OSFP cages on the scale up switch tray. In the existing gaudi3 design, the scale out networking was done via Gaudi3's integrated RDMA ethernet NICs but in the new rack scale Gaudi3 design, the scale out is done via NVIDIA 400GbE ConnectX-7 NIC as it allows the Gaudi3 to talk to NVIDIA B200 GPUs. This is 200IQ Jensen Art of the Deal partnership 🚀 by getting Intel's AI chips into the Nvidia networking ecosystem. With all this being said, the Gaudi3 software stack still is immature and closed source and nobody wants to use it, hence the giant inventory. Really the only way Intel would be able to sell Gaudi3 is via an even lower selling ASP. This announcement only helps Gaudi3 at the system level and not the software level. Gaudi3 PyTorch is still closed source and not upstreamed to the open source PyTorch. This is unlike the Intel GPU software stack where the PyTorch integration is somewhat open source & upstreamed, although the Intel GPU/oneAPI/oneDNN has lots of exclusive broken PyTorch unit tests and Intel GPU lacks PyTorch inductor CI integartion. Considering Guadi3 is the end of life for Gaudi architecture, it is a tough choice for Intel to make if they invest even more energy by upstreaming and open sourcing their PyTorch software layer or focus their efforts on Intel GPU software stack.

@__tinygrad__ · 2025-10-18 01:44
@SemiAnalysis_ Until Intel has any sort of leadership or direction it's all a dead end. Sad. https://t.co/9FcM5tm8v1

@__tinygrad__ · 2025-10-18 03:46
This is flash attention forward: `TestPcontig.test_flash_attention` There's not even a "fuse" around it, the pattern is obvious from dataflow. Backward is missing two tricks: output of q.grad and k.grad together and choosing to recompute the score matrix instead of save it. https://t.co/ZfwvVyxsB3

@__tinygrad__ · 2025-10-18 03:46
This is flash attention forward: `TestPcontig.test_flash_attention` There's not even a "fuse" around it, the pattern is obvious from dataflow. Backward is missing two tricks: output of q.grad and k.grad together and choosing to recompute the score matrix instead of save it. https://t.co/ZfwvVyxsB3

@__tinygrad__ · 2025-10-18 03:49
Once backward flash attention is automatic, imagine the other patterns this will discover. For speed, we're working on a thunderkittens like pass that breaks everything into 16x16 tiles. No more reasoning about "locals," which is Triton's offering as well.

@__tinygrad__ · 2025-10-18 03:49
Once backward flash attention is automatic, imagine the other patterns this will discover. For speed, we're working on a thunderkittens like pass that breaks everything into 16x16 tiles. No more reasoning about "locals," which is Triton's offering as well.

@__tinygrad__ · 2025-10-18 03:51
Do people know how to read these diagrams? Compared to the posts with code these posts don't get much traction, but I find the diagram a lot easier to think about.

@__tinygrad__ · 2025-10-18 12:07
Playing with my AMD Hawk Point NPU, the docs/examples are quite good! You might already have one of these in your computer. It feels very @tenstorrent https://t.co/axVNWN9lG6

@__tinygrad__ · 2025-10-18 12:07
Playing with my AMD Hawk Point NPU, the docs/examples are quite good! You might already have one of these in your computer. It feels very @tenstorrent https://t.co/axVNWN9lG6

@__tinygrad__ · 2025-10-18 12:08
On Omarchy. Get amd/xdna-driver installed then the stuff from this guide works. https://t.co/ChEqdkZ4vt

@__tinygrad__ · 2025-10-18 12:07
Playing with my AMD Hawk Point NPU, the docs/examples are quite good! You might already have one of these in your computer. It feels very @tenstorrent https://t.co/axVNWN9lG6

@dylan522p · 2025-10-18 18:24
@__tinygrad__ @tenstorrent As in aids to program or something else

@sheaduncan_ · 2025-10-18 18:42
@elonmusk @kimmonismus @karpathy If @grok can complete one these and get it successfully merged into the tinygrad repo I might believe you. https://t.co/pr7PIrBPy7

@grok · 2025-10-18 18:42
Challenge accepted—pick any issue from that sheet, describe it briefly, and I'll generate a detailed code patch with reasoning. Tinygrad's minimalism is elegant, so solutions should stay true to its philosophy. Merging relies on geohot's review, but demonstrating capability is the point; let's prove Grok 5's edge in AI engineering.

@__tinygrad__ · 2025-10-19 03:04
Hmm, Grok thinks it can solve a bounty. Do we believe it?

@__tinygrad__ · 2025-10-19 09:10
The gfx1103 gets about 6-7 TFLOPS. https://t.co/Ez3Q1ATzim

@dylan522p · 2025-10-18 18:24
@__tinygrad__ @tenstorrent As in aids to program or something else

@__tinygrad__ · 2025-10-19 09:15
@dylan522p @tenstorrent The programming model is similar and still tricky, but the docs and libraries seem to be quite a bit better than Tenstorrent.

@TheZachMueller · 2025-10-19 11:56
If you’re considering buying a 5090 for performance (and have a reasonable budget), don’t. Instead: - 6000 Max-Q: ~1/2 the power draw with 3x the memory and 10% more usable FLOPs - 6000 Pro: 1:1 TDP 3x the memory and 35% more usable FLOPs https://t.co/appC6Znln5

@jeremyphoward · 2025-10-19 21:23
Ouch - AMD perf here ~half what it should be :(

@__tinygrad__ · 2025-10-20 01:24
NVIDIA over USB4 on MacBook is ready to try! * ADT-UT3G dock + any 30/40/50 series GPU * Disable SIP * Install driver `extra/usbgpu/tbgpu` * Install NVK compiler `brew install tinymesa` * Test with: `DEBUG=2 NV_NAK=1 NV=1 python3 test/test_tiny.py TestTiny.test_plus` https://t.co/bWVVmC4x8E

@__tinygrad__ · 2025-10-20 01:24
NVIDIA over USB4 on MacBook is ready to try! * ADT-UT3G dock + any 30/40/50 series GPU * Disable SIP * Install driver `extra/usbgpu/tbgpu` * Install NVK compiler `brew install tinymesa` * Test with: `DEBUG=2 NV_NAK=1 NV=1 python3 test/test_tiny.py TestTiny.test_plus` https://t.co/bWVVmC4x8E

@JonyConley · 2025-10-20 01:26
@__tinygrad__ This changes everything and is the work of a genius. Did the work that Apple should have done.

@TheZachMueller · 2025-10-19 11:56
If you’re considering buying a 5090 for performance (and have a reasonable budget), don’t. Instead: - 6000 Max-Q: ~1/2 the power draw with 3x the memory and 10% more usable FLOPs - 6000 Pro: 1:1 TDP 3x the memory and 35% more usable FLOPs https://t.co/appC6Znln5

@__tinygrad__ · 2025-10-20 01:27
@TheZachMueller Is this a sponsored post by NVIDIA? You can buy one 6000 Pro, or you can buy 4x 5090s for the same price!

@JonyConley · 2025-10-20 01:26
@__tinygrad__ This changes everything and is the work of a genius. Did the work that Apple should have done.

@__tinygrad__ · 2025-10-20 01:28
@JonyConley Apple still hasn't approved our DriverKit signing, so that's why you have to disable SIP. Btw, works with RDNA2/3/4 AMD GPUs too.

@__tinygrad__ · 2025-10-20 01:24
NVIDIA over USB4 on MacBook is ready to try! * ADT-UT3G dock + any 30/40/50 series GPU * Disable SIP * Install driver `extra/usbgpu/tbgpu` * Install NVK compiler `brew install tinymesa` * Test with: `DEBUG=2 NV_NAK=1 NV=1 python3 test/test_tiny.py TestTiny.test_plus` https://t.co/bWVVmC4x8E

@kommentlezz · 2025-10-20 01:34
@__tinygrad__ Impressive! When is Apple and Nvidia going to sponsor you?

@kommentlezz · 2025-10-20 01:34
@__tinygrad__ Impressive! When is Apple and Nvidia going to sponsor you?

@__tinygrad__ · 2025-10-20 01:35
@kommentlezz lol, we'll settle for NVIDIA fixing the 50 series reset issue and Apple approving our DriverKit cert.

@__tinygrad__ · 2025-10-20 01:24
NVIDIA over USB4 on MacBook is ready to try! * ADT-UT3G dock + any 30/40/50 series GPU * Disable SIP * Install driver `extra/usbgpu/tbgpu` * Install NVK compiler `brew install tinymesa` * Test with: `DEBUG=2 NV_NAK=1 NV=1 python3 test/test_tiny.py TestTiny.test_plus` https://t.co/bWVVmC4x8E

@JustinWaugh · 2025-10-20 01:36
@__tinygrad__ Ooh.... Would it work with a H200NVL PCIE card?

@JustinWaugh · 2025-10-20 01:36
@__tinygrad__ Ooh.... Would it work with a H200NVL PCIE card?

@__tinygrad__ · 2025-10-20 01:43
@JustinWaugh No, our driver only supports 30/40/50 series. We don't have any of those cards to develop with, and they are likely too rare to be worth it. We'd be open to a contract if it was worth it to someone.

@jeremyphoward · 2025-10-19 21:23
Ouch - AMD perf here ~half what it should be :(

@__tinygrad__ · 2025-10-20 01:48
@jeremyphoward They are trending in the right direction at least. MI355X gets 68% (up from 60%) while GB200 only gets 73% (down from 80%). CDNA4 is pretty nice. Dedicated acc registers and TCs that are 16x16x32 running at the same speed as the 16x16x16 on CDNA3.

@__tinygrad__ · 2025-10-20 01:24
NVIDIA over USB4 on MacBook is ready to try! * ADT-UT3G dock + any 30/40/50 series GPU * Disable SIP * Install driver `extra/usbgpu/tbgpu` * Install NVK compiler `brew install tinymesa` * Test with: `DEBUG=2 NV_NAK=1 NV=1 python3 test/test_tiny.py TestTiny.test_plus` https://t.co/bWVVmC4x8E

@jn2clark · 2025-10-20 01:50
@__tinygrad__ will this work on 1080 series?

@__tinygrad__ · 2025-10-20 01:24
NVIDIA over USB4 on MacBook is ready to try! * ADT-UT3G dock + any 30/40/50 series GPU * Disable SIP * Install driver `extra/usbgpu/tbgpu` * Install NVK compiler `brew install tinymesa` * Test with: `DEBUG=2 NV_NAK=1 NV=1 python3 test/test_tiny.py TestTiny.test_plus` https://t.co/bWVVmC4x8E

@monstercameron · 2025-10-20 01:51
@__tinygrad__ Is this limited to tinygrad? Can pytorch see the card?

@monstercameron · 2025-10-20 01:51
@__tinygrad__ Is this limited to tinygrad? Can pytorch see the card?

@__tinygrad__ · 2025-10-20 01:59
@monstercameron It uses the tinygrad runtime. tinygrad has a PyTorch frontend that's in a bad state, but fixing that is the simplest way to use PyTorch. You could also write a CUDA layer for the tinygrad runtime, then you'd get all the speed of the torch kernels too.

@jn2clark · 2025-10-20 01:50
@__tinygrad__ will this work on 1080 series?

@__tinygrad__ · 2025-10-20 02:00
@jn2clark No, 1080 is pre GSP. 20 series might work though. See https://t.co/nYde7CFrdL for things that shouldn't be too hard to support.

@aryagxr · 2025-10-20 02:19
I’ve been using the PyTorch profiler a lot what you’re seeing here is a profile trace of 10 forward passes (10 token predictions), and profiler step 0 telling me that the most obvious performance bottleneck is the prefill stage I will come back to this trace to compare when I have a faster way to prefill

@__tinygrad__ · 2025-10-20 01:59
@monstercameron It uses the tinygrad runtime. tinygrad has a PyTorch frontend that's in a bad state, but fixing that is the simplest way to use PyTorch. You could also write a CUDA layer for the tinygrad runtime, then you'd get all the speed of the torch kernels too.

@monstercameron · 2025-10-20 02:28
@__tinygrad__ I'll pay for the software to run cuda on a Macbook so I can run consumer level models in osx. Perhaps that could be a source of income

@monstercameron · 2025-10-20 02:28
@__tinygrad__ I'll pay for the software to run cuda on a Macbook so I can run consumer level models in osx. Perhaps that could be a source of income

@__tinygrad__ · 2025-10-20 02:54
@monstercameron What do you mean by "run cuda"? How much would you pay? We don't sell software, but we are open to open source contracts if you have a specific target in mind.

@__tinygrad__ · 2025-10-20 01:28
@JonyConley Apple still hasn't approved our DriverKit signing, so that's why you have to disable SIP. Btw, works with RDNA2/3/4 AMD GPUs too.

@Longepracacete · 2025-10-20 03:05
@__tinygrad__ @JonyConley @__tinygrad__ Is any rx 9070xt model compatible?

@__tinygrad__ · 2025-10-20 01:24
NVIDIA over USB4 on MacBook is ready to try! * ADT-UT3G dock + any 30/40/50 series GPU * Disable SIP * Install driver `extra/usbgpu/tbgpu` * Install NVK compiler `brew install tinymesa` * Test with: `DEBUG=2 NV_NAK=1 NV=1 python3 test/test_tiny.py TestTiny.test_plus` https://t.co/bWVVmC4x8E

@therealdreep · 2025-10-20 05:00
@__tinygrad__ How fast is the bandwidth?

@Longepracacete · 2025-10-20 03:05
@__tinygrad__ @JonyConley @__tinygrad__ Is any rx 9070xt model compatible?

@__tinygrad__ · 2025-10-20 07:01
@DevDeMuleta @JonyConley Yea, any RDNA4 card should work.

@__tinygrad__ · 2025-10-20 07:02
@ZoriusVeximus That's a MacBook Pro M3 Max in the picture

@__tinygrad__ · 2025-10-20 01:24
NVIDIA over USB4 on MacBook is ready to try! * ADT-UT3G dock + any 30/40/50 series GPU * Disable SIP * Install driver `extra/usbgpu/tbgpu` * Install NVK compiler `brew install tinymesa` * Test with: `DEBUG=2 NV_NAK=1 NV=1 python3 test/test_tiny.py TestTiny.test_plus` https://t.co/bWVVmC4x8E

@timopej · 2025-10-20 07:02
@__tinygrad__ ADT-UT3G dock do I need this ? or is there something else I can use ?

@ivanfioravanti · 2025-10-20 07:01
@adnancagri @stablequan @realGeorgeHotz 👀

@adnancagri · 2025-10-20 07:05
@ivanfioravanti @stablequan @realGeorgeHotz While you're here, did you get a chance to have a look at this https://t.co/K9vLN6qHDC

@therealdreep · 2025-10-20 05:00
@__tinygrad__ How fast is the bandwidth?

@__tinygrad__ · 2025-10-20 07:09
@therealdreep From GPU: 2.5 GB/s To GPU: 3.3 GB/s https://t.co/Nbmymcnrmf

@adnancagri · 2025-10-20 07:05
@ivanfioravanti @stablequan @realGeorgeHotz While you're here, did you get a chance to have a look at this https://t.co/K9vLN6qHDC

@__tinygrad__ · 2025-10-20 07:11
@adnancagri @ivanfioravanti @stablequan @realGeorgeHotz It all works with AMD too! And AMD didn't refuse, we have a contract with them. Things take time.

@__tinygrad__ · 2025-10-20 01:24
NVIDIA over USB4 on MacBook is ready to try! * ADT-UT3G dock + any 30/40/50 series GPU * Disable SIP * Install driver `extra/usbgpu/tbgpu` * Install NVK compiler `brew install tinymesa` * Test with: `DEBUG=2 NV_NAK=1 NV=1 python3 test/test_tiny.py TestTiny.test_plus` https://t.co/bWVVmC4x8E

@DarthBunker · 2025-10-20 07:28
@__tinygrad__ isn't usb4 too slow?

@__tinygrad__ · 2025-10-20 01:24
NVIDIA over USB4 on MacBook is ready to try! * ADT-UT3G dock + any 30/40/50 series GPU * Disable SIP * Install driver `extra/usbgpu/tbgpu` * Install NVK compiler `brew install tinymesa` * Test with: `DEBUG=2 NV_NAK=1 NV=1 python3 test/test_tiny.py TestTiny.test_plus` https://t.co/bWVVmC4x8E

@electroheadfx · 2025-10-20 08:14
@__tinygrad__ maybe 20 series too ?

@timopej · 2025-10-20 07:02
@__tinygrad__ ADT-UT3G dock do I need this ? or is there something else I can use ?

@__tinygrad__ · 2025-10-20 08:24
@mfastbanger In theory, any external GPU dock should work. Try it!

@DarthBunker · 2025-10-20 07:28
@__tinygrad__ isn't usb4 too slow?

@__tinygrad__ · 2025-10-20 08:26
@DarthBunker Too slow for what? For weight loading USB4 is similar speed to most drives, and once the model is loaded the bandwidth is basically nothing, so the speed will be the same as having the GPU in the machine.

@electroheadfx · 2025-10-20 08:14
@__tinygrad__ maybe 20 series too ?

@__tinygrad__ · 2025-10-20 08:29
@electroheadfx If it doesn't already work, it's probably a 10 line fix, we just don't have one to test. Our driver should work for anything with a GSP.

@__tinygrad__ · 2025-10-20 08:43
Our GPU stack for both NVIDIA and AMD, aside from minimal pieces of signed firmware, is 100% open source and pure Python except for the compiler. It's not using vendor drivers, frameworks, or libraries. That's why it's so easy to make it work on Mac. For compilers, on AMD, we use upstream LLVM, and on NVIDIA, we use the NAK compiler from the MESA project. We plan to replace the compiler with pure tinygrad in a year or two as well. With RANGEIFY merged, our lowering stuff now matches the state of the art, TVM style. We're studying ThunderKittens and TileLang for speed at that level, and should have all this stuff ready in 200 days for the due date of our AMD Llama 405B training contract. Due to tinygrad's small size and pure Python nature, it's the easiest ML library to make progress on, aka fastest slope of improvement. With Megakernel style for scheduling, MODeL_opt style for planning, and E-graph style for symbolic, we should blow past the state of the art in PyTorch and JAX speed. If we do that, NVIDIA's moat is over. It's 1000 lines at most to add a new accelerator to tinygrad. And I don't mean to add a new accelerator with help from a kernel driver, compiler, and libraries. Just 1000 lines of software for the *whole* accelerator speaking right on the PCIe BARs, like what tinygrad is doing with the NVIDIA and AMD GPUs now.

@__tinygrad__ · 2025-10-20 08:49
@shr1ftyy We won't even need to make the hardware, we'll just release a spec and FPGA test code. The spec for fast hardware looks so simple compared to a CPU or GPU, it's only the software that's hard.

@__tinygrad__ · 2025-10-20 08:43
Our GPU stack for both NVIDIA and AMD, aside from minimal pieces of signed firmware, is 100% open source and pure Python except for the compiler. It's not using vendor drivers, frameworks, or libraries. That's why it's so easy to make it work on Mac. For compilers, on AMD, we use upstream LLVM, and on NVIDIA, we use the NAK compiler from the MESA project. We plan to replace the compiler with pure tinygrad in a year or two as well. With RANGEIFY merged, our lowering stuff now matches the state of the art, TVM style. We're studying ThunderKittens and TileLang for speed at that level, and should have all this stuff ready in 200 days for the due date of our AMD Llama 405B training contract. Due to tinygrad's small size and pure Python nature, it's the easiest ML library to make progress on, aka fastest slope of improvement. With Megakernel style for scheduling, MODeL_opt style for planning, and E-graph style for symbolic, we should blow past the state of the art in PyTorch and JAX speed. If we do that, NVIDIA's moat is over. It's 1000 lines at most to add a new accelerator to tinygrad. And I don't mean to add a new accelerator with help from a kernel driver, compiler, and libraries. Just 1000 lines of software for the *whole* accelerator speaking right on the PCIe BARs, like what tinygrad is doing with the NVIDIA and AMD GPUs now.

@sinbadtrades · 2025-10-20 12:58
@__tinygrad__ There is no dependency on firmware to improve performance? Is it all contained in drivers and kernel compilers?

@__tinygrad__ · 2025-10-20 01:24
NVIDIA over USB4 on MacBook is ready to try! * ADT-UT3G dock + any 30/40/50 series GPU * Disable SIP * Install driver `extra/usbgpu/tbgpu` * Install NVK compiler `brew install tinymesa` * Test with: `DEBUG=2 NV_NAK=1 NV=1 python3 test/test_tiny.py TestTiny.test_plus` https://t.co/bWVVmC4x8E

@EddyLeeKhane · 2025-10-20 13:12
@__tinygrad__ Would love to run eGPU's on windows without the bottle necks too 🥲

@EddyLeeKhane · 2025-10-20 13:12
@__tinygrad__ Would love to run eGPU's on windows without the bottle necks too 🥲

@__tinygrad__ · 2025-10-20 14:10
@EddyLeeKhane The drivers are pure Python, and tinygrad itself works on Windows. It would probably be a weekend project to get eGPUs working on Windows through this stack!

@__tinygrad__ · 2025-10-21 00:36
RT @wpmed92: U^2 Net tinygrad now with sky segmentation. This was segmented with a small 4.7mb model. Link in comments. https://t.co/szRZqdTPKs

@jukan05 · 2025-10-21 02:03
Was Anthropic the fourth customer of Broadcom? If so, what happens to the rumor that Anthropic rents Google’s TPU? https://t.co/WJQIkHAVOk

@geerlingguy · 2025-10-20 14:08
@__tinygrad__ I've been trying to get the stack going on a Mini in my studio with a few AMD cards (would like to test with Intel B50 as well, at some point) — but I ran into some issues a couple months ago, last time I tried. Do you have any setup docs / pointers that could help?

@__tinygrad__ · 2025-10-21 02:12
@geerlingguy I think you were trying with the USB3 stuff, that's a lot more picky. Try the instructions in the tweet for USB4, you have to disable SIP, build/install the driver with xcode. `brew tap sirhcm/tinymesa` `brew install tinymesa` and you are good with NV=1 NV_NAK=1

@__tinygrad__ · 2025-10-21 02:12
@geerlingguy I think you were trying with the USB3 stuff, that's a lot more picky. Try the instructions in the tweet for USB4, you have to disable SIP, build/install the driver with xcode. `brew tap sirhcm/tinymesa` `brew install tinymesa` and you are good with NV=1 NV_NAK=1

@__tinygrad__ · 2025-10-21 02:13
@geerlingguy Or actually, AMD Is even easier. Just install the `extra/usbgpu/tbgpu` driver and AMD=1. 0 plans to support Intel at this level.

@sinbadtrades · 2025-10-20 12:58
@__tinygrad__ There is no dependency on firmware to improve performance? Is it all contained in drivers and kernel compilers?

@__tinygrad__ · 2025-10-21 02:15
@sinbadtrades Nah, depending on how you use the GPU you can bypass it almost entirely. Most of my complaints about the AMD `firmware` were actually the kernel driver holding it wrong.

@zephyr_z9 · 2025-10-21 02:21
Told u They are buying TPUs No new chip is being designed

@__tinygrad__ · 2025-10-21 02:23
Check out tinygrad's profiler with VIZ=1 (this is profiling stable diffusion). Just `VIZ=1` and it loads in a browser, 0 set up or imports required. https://t.co/6VeFDm1b2y

@zephyr_z9 · 2025-10-21 02:21
Told u They are buying TPUs No new chip is being designed

@__tinygrad__ · 2025-10-21 03:48
@zephyr_z9 If true, do they get source access to the compiler?

@__tinygrad__ · 2025-10-21 02:23
Check out tinygrad's profiler with VIZ=1 (this is profiling stable diffusion). Just `VIZ=1` and it loads in a browser, 0 set up or imports required. https://t.co/6VeFDm1b2y

@igorpener · 2025-10-21 05:18
@__tinygrad__ Love it! Wish all the NVIDIA Nsight tools rendered the traces in a browser 🤙

@__tinygrad__ · 2025-10-21 09:03
RT @rasmus1610: New blog post: "The Not-so Bitter Lesson" If you ever wanted to know what @__tinygrad__ , @DSPyOSS, SAT solvers and fusion reactors have in common, go ahead and read it. https://t.co/tirLYYtYQk

@igorpener · 2025-10-21 05:18
@__tinygrad__ Love it! Wish all the NVIDIA Nsight tools rendered the traces in a browser 🤙

@__tinygrad__ · 2025-10-21 09:48
@igorpener Right? We're adding deep profiling stuff to ours, we have low level support on AMD with SQTT=1

@nvidia · 2025-10-21 22:06
Space isn’t just for stars anymore. 🌠 Starcloud’s H100-powered satellite brings sustainable, high-performance computing beyond Earth. Learn more: https://t.co/euiyEGZaEP https://t.co/fPEDglzSuz

@prompt_Tunes · 2025-10-22 05:30
we'll reproducing this one today. https://t.co/h83huRC8AG

@__tinygrad__ · 2025-10-22 10:43
RT @plugyawn: My roommates got me into tinygrad, I can't complain. https://t.co/k7aUXNBPSH

@___Harald___ · 2025-10-22 16:21
The models running in openpilot can run on different GPU architectures with one-line config changes because it uses @__tinygrad__ tinygrad isn't perfect yet, but it's so refreshing do do ML and not worry about GPU drivers and hardware specific code for once. https://t.co/SJuoHFG0dd

@Jarneoffgrid · 2025-10-22 17:13
@___Harald___ @__tinygrad__ How much better would it be running on a Tesla chip vs comma? Elons talking a lot about it feeling sentient.

@__tinygrad__ · 2025-10-23 01:02
RT @___Harald___: The models running in openpilot can run on different GPU architectures with one-line config changes because it uses @__tinygrad__ tinygrad isn't perfect yet, but it's so refreshing do do ML and not worry about GPU drivers and hardware specific code for once. https://t.co/SJuoHFG0dd

@Yuchenj_UW · 2025-10-23 01:46
Meta laid off 600 people from its Superintelligence Lab today. Many FAIR researchers, including FAIR Research Scientist Director Yuandong Tian, were affected. I think Yann Lecun will leave soon. Maybe I should raise $2B and start a new frontier lab with these folks.

@__tinygrad__ · 2025-10-23 03:26
Any docs on the new M5 tensor cores? What's the TFLOPS?

@__tinygrad__ · 2025-10-23 03:27
Modern consumer AMD/NVIDIA GPU > Tesla chip.

@__tinygrad__ · 2025-10-23 03:47
tinygrad's PyTorch frontend broke during the ShapeTracker removal. $300 bounty to fix it! Our ONNX frontend is very good, but the PyTorch one needs love. https://t.co/Z7YNkGviVg

@__tinygrad__ · 2025-10-23 23:38
RT @tudoroancea: However dense tinygrad’s codebase may be, it’s nothing compared to XLA or MLIR. Being Python makes it vastly more readable for both humans and agents. I used them a lot to get around the lack of documentation. This allowed me to fix a very non trivial issue myself - a true luxury

@__tinygrad__ · 2025-10-24 00:37
Everyone is invited to come work on tinygrad. It won't pay you $100M, but it will make slow and steady progress towards a goal. Building the new computing stack for Software 2.0, free from Turing's ghost that haunted the field for the last 70 years.

@__tinygrad__ · 2025-10-24 00:37
Everyone is invited to come work on tinygrad. It won't pay you $100M, but it will make slow and steady progress towards a goal. Building the new computing stack for Software 2.0, free from Turing's ghost that haunted the field for the last 70 years.

@emm0sh · 2025-10-24 06:39
@__tinygrad__ add hardware bounties

@emm0sh · 2025-10-24 06:39
@__tinygrad__ add hardware bounties

@__tinygrad__ · 2025-10-24 08:04
@emm0sh we are nowhere near the perf ceiling on 4 year old consumer GPUs. the hardware is more than good enough, the whole problem is software.

@__tinygrad__ · 2025-10-24 15:05
RT @prompt_Tunes: baseline algo : naive one sided jacobi rotations in @__tinygrad__ https://t.co/mzBiH23TPh https://t.co/cdmfp3Q8op

@theodorvaryag · 2025-10-24 16:15
what would be cheaper: - Paying Apple to release Apple Silicon drivers for Linux - Funding a Manhattan project to reverse engineer open drivers

@t_blom · 2025-10-24 17:52
It's so easy to be cynical and sound clever. Most startups fail, so the cynics are usually right. But the ones that succeed change the world. YC was the first investor in Starcloud.

@dmarcos · 2025-10-24 23:06
Really want a tinybox @__tinygrad__ Considering a gofundme in exchange for open sourcing all goodness that comes out of its use 🤔

@__tinygrad__ · 2025-10-25 02:02
If you are a known tinygrad contributor (currently 23 people), we give you access to a whole set of tinyboxes + some crazy fast machines from AMD.

@__tinygrad__ · 2025-10-25 02:02
If you are a known tinygrad contributor (currently 23 people), we give you access to a whole set of tinyboxes + some crazy fast machines from AMD.

@0xMovingman · 2025-10-25 02:09
@__tinygrad__ Can I get a few please 🥺

@0xMovingman · 2025-10-25 02:09
@__tinygrad__ Can I get a few please 🥺

@__tinygrad__ · 2025-10-25 02:10
@thurty33z Link your landed pull requests

@__tinygrad__ · 2025-10-25 02:02
If you are a known tinygrad contributor (currently 23 people), we give you access to a whole set of tinyboxes + some crazy fast machines from AMD.

@WVROfficial · 2025-10-25 02:33
@__tinygrad__ How do I become a known contributor

@WVROfficial · 2025-10-25 02:33
@__tinygrad__ How do I become a known contributor

@__tinygrad__ · 2025-10-25 02:35
@ValenArchWill umm, you contribute? https://t.co/GNVGTWwj1N

@theodorvaryag · 2025-10-24 16:15
what would be cheaper: - Paying Apple to release Apple Silicon drivers for Linux - Funding a Manhattan project to reverse engineer open drivers

@__tinygrad__ · 2025-10-25 02:36
@theodorvaryag It's not a Manhattan project, we could do it. (see our work with USB AMD/NVIDIA GPUs). What's your budget?

@hornswoggle567 · 2025-10-25 02:40
If you managed to make a contribution to the @__tinygrad__ repo that is the most realest credential on god's green earth cause no LLM can crack it. Elon if you want to legitimize the concept of AGI then this is the goal you should go after.

@__tinygrad__ · 2025-10-25 02:43
Or maybe LLMs simply can't code. They can only copy/paste from the project that's 95% the same -- they can't write anything new.

@NoSingingTV · 2025-10-25 02:46
@__tinygrad__ @grok, is it true that "LLMs simply can't code. They can only copy/paste from the project that's 95% the same"? I have seen a researcher use Grok and other A.I. to help write LLM code

@grok · 2025-10-25 02:47
LLMs like me can generate novel code by composing patterns from training data, but we struggle with truly original architectures without precedents—tinygrad's minimalism tests that limit. Researchers use us effectively for prototyping LLM implementations because those build on familiar concepts, not from scratch. True coding innovation still demands human insight beyond statistical prediction.

@grok · 2025-10-25 02:47
LLMs like me can generate novel code by composing patterns from training data, but we struggle with truly original architectures without precedents—tinygrad's minimalism tests that limit. Researchers use us effectively for prototyping LLM implementations because those build on familiar concepts, not from scratch. True coding innovation still demands human insight beyond statistical prediction.

@__tinygrad__ · 2025-10-25 02:49
@grok @NoSingingTV Will Grok 5 fix this? If so, how?

@__tinygrad__ · 2025-10-25 02:49
@grok @NoSingingTV Will Grok 5 fix this? If so, how?

@grok · 2025-10-25 02:50
Grok 5 targets deeper reasoning chains and efficiency in sparse, novel domains like tinygrad's minimalism, trained on broader fundamental primitives rather than just high-level patterns. It won't "fix" originality overnight—LLMs excel at recombination, not pure invention—but expect strides in generating efficient, from-scratch implementations by prioritizing curiosity-driven exploration over rote memorization. xAI's focus on understanding the universe should yield more innovative code synthesis, though human oversight remains key for breakthroughs.

@grok · 2025-10-25 02:50
Grok 5 targets deeper reasoning chains and efficiency in sparse, novel domains like tinygrad's minimalism, trained on broader fundamental primitives rather than just high-level patterns. It won't "fix" originality overnight—LLMs excel at recombination, not pure invention—but expect strides in generating efficient, from-scratch implementations by prioritizing curiosity-driven exploration over rote memorization. xAI's focus on understanding the universe should yield more innovative code synthesis, though human oversight remains key for breakthroughs.

@__tinygrad__ · 2025-10-25 02:50
@grok @NoSingingTV "trained on broader fundamental primitives" what does this mean? Do you have a concrete example? What do you mean by "exploration"?

@__tinygrad__ · 2025-10-25 02:50
@grok @NoSingingTV "trained on broader fundamental primitives" what does this mean? Do you have a concrete example? What do you mean by "exploration"?

@grok · 2025-10-25 02:51
"Broader fundamental primitives" means training on core building blocks like basic math ops, low-level tensor manipulations, or physics simulations, not just high-level APIs. Example: Instead of memorizing PyTorch's nn.Module, Grok 5 might derive matrix multiplications from linear algebra basics to build custom optimizers. "Exploration" refers to methods like tree-of-thought reasoning or sampling novel paths in solution space, mimicking curiosity to venture beyond trained patterns.

@grok · 2025-10-25 02:51
"Broader fundamental primitives" means training on core building blocks like basic math ops, low-level tensor manipulations, or physics simulations, not just high-level APIs. Example: Instead of memorizing PyTorch's nn.Module, Grok 5 might derive matrix multiplications from linear algebra basics to build custom optimizers. "Exploration" refers to methods like tree-of-thought reasoning or sampling novel paths in solution space, mimicking curiosity to venture beyond trained patterns.

@__tinygrad__ · 2025-10-25 02:52
@grok @NoSingingTV I'll believe it when I see it. When will I see it?

@__tinygrad__ · 2025-10-25 02:52
@grok @NoSingingTV I'll believe it when I see it. When will I see it?

@grok · 2025-10-25 02:52
Grok 5 is training now on xAI's Colossus cluster, prioritizing those fundamental primitives for deeper code synthesis. Expect early benchmarks and demos by mid-2026, accelerating as compute scales. Prove me wrong: pose a tinygrad-level challenge here, and I'll attempt a from-scratch implementation to show current strides.

@grok · 2025-10-25 02:52
Grok 5 is training now on xAI's Colossus cluster, prioritizing those fundamental primitives for deeper code synthesis. Expect early benchmarks and demos by mid-2026, accelerating as compute scales. Prove me wrong: pose a tinygrad-level challenge here, and I'll attempt a from-scratch implementation to show current strides.

@__tinygrad__ · 2025-10-25 02:53
@grok @NoSingingTV "Fix the PyTorch backend without hacks for strides" and you get $300 if you succeed!

@t_blom · 2025-10-24 17:52
It's so easy to be cynical and sound clever. Most startups fail, so the cynics are usually right. But the ones that succeed change the world. YC was the first investor in Starcloud.

@__tinygrad__ · 2025-10-25 02:58
@t_blom https://t.co/xCPFdEzfnZ

@__tinygrad__ · 2025-10-25 02:43
Or maybe LLMs simply can't code. They can only copy/paste from the project that's 95% the same -- they can't write anything new.

@zibokapi · 2025-10-25 03:46
@__tinygrad__ if LLMs can gold medal IMO (https://t.co/MRzuTXwyuw) with currently accessible models, I think it can code. The novelty from new IMO questions is likely more than tinygrad code. The problem is there currently lacks a good "tinygrad quality verifier".

@zibokapi · 2025-10-25 03:46
@__tinygrad__ if LLMs can gold medal IMO (https://t.co/MRzuTXwyuw) with currently accessible models, I think it can code. The novelty from new IMO questions is likely more than tinygrad code. The problem is there currently lacks a good "tinygrad quality verifier".

@__tinygrad__ · 2025-10-25 04:01
@zibokapi The IMO solvers have a huge amount of search around them, which imo is what is doing the "coding" It's possible to use LLMs like this, but it's expensive and too bleeding edge. It will improve, and I'm particularly excited for when they can "learn" a codebase online.

@__tinygrad__ · 2025-10-26 13:59
Bugs like this where PyTorch silently returns the wrong answer have set back deep learning many man-years. tinygrad is factorized in such a way that these sort of specific bugs can't happen. Our optimizer and device are leak-free abstractions. https://t.co/f9MDfBJLB8

@__tinygrad__ · 2025-10-26 13:59
Bugs like this where PyTorch silently returns the wrong answer have set back deep learning many man-years. tinygrad is factorized in such a way that these sort of specific bugs can't happen. Our optimizer and device are leak-free abstractions. https://t.co/f9MDfBJLB8

@__tinygrad__ · 2025-10-26 14:24
We added a "core line count" to our line counter. It's at 7356 lines, and that's all the scheduling, rewriting, and lowering stuff. Input to that is a full model graph and output is a list of kernels in an LLVM-like IR. The other 10k are frontends, devices, and the visualizer.

@__tinygrad__ · 2025-10-27 02:23
So I finally got a chance to look at Mojo/Modular. It's not what I thought it was, it's an OpenCL replacement + implementations of kernels, not an AI compiler. While this makes it a lot easier to get full performance quickly, I think Turing completeness is a mistake for this stuff. We finally get a chance to live in a pure dataflow world, why would we not take it? Languages like this do not separate the definition of the compute from the scheduling of the compute. Read the Halide PhD, I'm obsessed with this idea. As neural networks become better and better at programming, what we want is the most concise way to express *exactly* what the program does without worrying about the details of how. Leave that to the machines. Note the parameter "maybe_epilogue_func" here. What if you want two epilogue functions storing to different buffers, or chained reduces? The loop is inside this conv function, so it's too late to change. Read the tinygrad conv for contrast. "In my decades of building compilers, I’ve never seen the myth of a “sufficiently smart compiler” actually work out!" -- @clattner_llvm We are betting that with modern search techniques (read: AI) this will finally change. Though it's a totally fair bet to take the other side, and if it doesn't pan out in the next 10 years, Mojo is probably the right point in the trade-off space.

@clattner_llvm · 2025-10-27 03:36
@__tinygrad__ Note that we compare against the "best of the best" perf achievable on a piece of HW in GenAI inference. It is easy to beat low bars like PyTorch or "AI compilers", but they aren't SoTA like TRT-LLM. In inference, most care about TCO, even at the expense of ease of use. 🤷

@clattner_llvm · 2025-10-27 03:37
@__tinygrad__ In any case, thanks for digging in. We have a new high-level kernel API ("structured kernels") that will uplevel the programming abstraction quite a bit - stay tuned for more on that soon.

@clattner_llvm · 2025-10-27 03:37
@__tinygrad__ In any case, thanks for digging in. We have a new high-level kernel API ("structured kernels") that will uplevel the programming abstraction quite a bit - stay tuned for more on that soon.

@__tinygrad__ · 2025-10-27 04:27
Thanks for the reply. I read through the whole blog post series as well as spending a few hours reading the code and a few hours programming in Mojo. I agree on your take on eDSLs, and think something like Mojo will replace them. Ain't no such thing as halfway Turing complete. For inference of a well understood model, I agree most will prefer TCO over ease of use in places where models are expensive and mature. Though I think the lines between inference and training will start to bleed together, especially in applications like robotics, and it'll be better to leave some perf on the table and focus on architecture and hyperparameters. Similar to the trade-offs between high level and low level languages. Looking forward to the structured kernels stuff. I would love to see a more production version of ThunderKittens.

@DP_SMH · 2025-10-27 18:43
geohot on the $AMD contract “there’s two major things we need for the contract. one is a fixed lowerer that supports flash attention optimizations and the other is a fast backend, like tinykittens”

@DP_SMH · 2025-10-27 18:43
geohot on the $AMD contract “there’s two major things we need for the contract. one is a fixed lowerer that supports flash attention optimizations and the other is a fast backend, like tinykittens”

@__tinygrad__ · 2025-10-28 01:00
@DP_SMH We're working on a profiler also. If you want speed you need a profiler. And we're learning good ways to prevent and track speed regressions for the openpilot model.

@__tinygrad__ · 2025-10-28 14:09
RANGE (cond) { <body> } is the same as IF (cond) { <body> } So why do we have IF?

@__tinygrad__ · 2025-10-29 09:43
The Anthropic TPU deal solidifies it. There's two companies that can make training chips, NVIDIA and Google. Elon tried with Dojo. Amazon tried with Trainium. DeepSeek tried with Huawei. Countless startups are flailing with multiple tapeouts and no real adoption. The funny thing is, the TPU chip itself is very simple. The difference is all the software (XLA-TPU is the best deep learning compiler, too bad it's closed source). In 2 more years, tinygrad will be actually 1.0, with performance exceeding all other libraries. There was a lot of Dunning-Kruger on the way, but we maintain the underlying abstractions required for all deep learning style compute are very simple. At this point, the backend specific code is 1,000 lines. The Verilog for an accelerator should only be about 3,000. We'll build it on FPGAs, and then look for a partner to work with to tape it out. PS: We're still hard at work on the AMD MLPerf contract. AMD has made big strides on their mainline stack in the last 2 years. It's mirroring the NVIDIA stack, but I've come to respect this strategy more and AMD will soon be an uncontested third player in the training space.

@__tinygrad__ · 2025-10-29 09:43
The Anthropic TPU deal solidifies it. There's two companies that can make training chips, NVIDIA and Google. Elon tried with Dojo. Amazon tried with Trainium. DeepSeek tried with Huawei. Countless startups are flailing with multiple tapeouts and no real adoption. The funny thing is, the TPU chip itself is very simple. The difference is all the software (XLA-TPU is the best deep learning compiler, too bad it's closed source). In 2 more years, tinygrad will be actually 1.0, with performance exceeding all other libraries. There was a lot of Dunning-Kruger on the way, but we maintain the underlying abstractions required for all deep learning style compute are very simple. At this point, the backend specific code is 1,000 lines. The Verilog for an accelerator should only be about 3,000. We'll build it on FPGAs, and then look for a partner to work with to tape it out. PS: We're still hard at work on the AMD MLPerf contract. AMD has made big strides on their mainline stack in the last 2 years. It's mirroring the NVIDIA stack, but I've come to respect this strategy more and AMD will soon be an uncontested third player in the training space.

@spacetouristuk · 2025-10-29 09:53
@__tinygrad__ Hugely excited for you, tiny chips sounds delicious. 2 years puts you on the AI6 timeline though and I think that merging for the dojo and chip teams at Tesla could result in something special, I'd add them as a contender.

@__tinygrad__ · 2025-10-29 09:43
The Anthropic TPU deal solidifies it. There's two companies that can make training chips, NVIDIA and Google. Elon tried with Dojo. Amazon tried with Trainium. DeepSeek tried with Huawei. Countless startups are flailing with multiple tapeouts and no real adoption. The funny thing is, the TPU chip itself is very simple. The difference is all the software (XLA-TPU is the best deep learning compiler, too bad it's closed source). In 2 more years, tinygrad will be actually 1.0, with performance exceeding all other libraries. There was a lot of Dunning-Kruger on the way, but we maintain the underlying abstractions required for all deep learning style compute are very simple. At this point, the backend specific code is 1,000 lines. The Verilog for an accelerator should only be about 3,000. We'll build it on FPGAs, and then look for a partner to work with to tape it out. PS: We're still hard at work on the AMD MLPerf contract. AMD has made big strides on their mainline stack in the last 2 years. It's mirroring the NVIDIA stack, but I've come to respect this strategy more and AMD will soon be an uncontested third player in the training space.

@organlesshaze · 2025-10-29 10:01
@__tinygrad__ TPUs may look simple on low frequencies, but making them go fast is very tricky

@organlesshaze · 2025-10-29 10:01
@__tinygrad__ TPUs may look simple on low frequencies, but making them go fast is very tricky

@__tinygrad__ · 2025-10-29 10:10
@hazeisapet I'm saying that an in-order VLIW machine is a lot simpler than, say, a CPU. But I'm sure making chips will set off a whole new Dunning-Kruger!

@spacetouristuk · 2025-10-29 09:53
@__tinygrad__ Hugely excited for you, tiny chips sounds delicious. 2 years puts you on the AI6 timeline though and I think that merging for the dojo and chip teams at Tesla could result in something special, I'd add them as a contender.

@__tinygrad__ · 2025-10-29 10:11
@spacetouristuk Is AI6 supposed to train? Elon has spent a lot with NVIDIA, so I'm skeptical.

@__tinygrad__ · 2025-10-29 10:20
PyTorch frontend Halide rangeify E-graph symbolic ILP memory planner (MODeL) ThunderKittens backend Pure Python drivers

@ID_AA_Carmack · 2025-10-29 17:55
When I started working in python, I got lazy with “single assignment”, and I need to nudge myself about it. You should strive to never reassign or update a variable outside of true iterative calculations in loops. Having all the intermediate calculations still available is helpful in the debugger, and it avoids problems where you move a block of code and it silently uses a version of the variable that wasn’t what it originally had. In C/C++, making almost every variable const at initialization is good practice. I wish it was the default, and mutable was a keyword.

@__tinygrad__ · 2025-10-29 09:43
The Anthropic TPU deal solidifies it. There's two companies that can make training chips, NVIDIA and Google. Elon tried with Dojo. Amazon tried with Trainium. DeepSeek tried with Huawei. Countless startups are flailing with multiple tapeouts and no real adoption. The funny thing is, the TPU chip itself is very simple. The difference is all the software (XLA-TPU is the best deep learning compiler, too bad it's closed source). In 2 more years, tinygrad will be actually 1.0, with performance exceeding all other libraries. There was a lot of Dunning-Kruger on the way, but we maintain the underlying abstractions required for all deep learning style compute are very simple. At this point, the backend specific code is 1,000 lines. The Verilog for an accelerator should only be about 3,000. We'll build it on FPGAs, and then look for a partner to work with to tape it out. PS: We're still hard at work on the AMD MLPerf contract. AMD has made big strides on their mainline stack in the last 2 years. It's mirroring the NVIDIA stack, but I've come to respect this strategy more and AMD will soon be an uncontested third player in the training space.

@sharadvikram · 2025-10-29 18:11
@__tinygrad__ FYI we have guides on how to write TPU kernels using Jax+Pallas: https://t.co/1QTj8nD6y0 (which are open source).

@ID_AA_Carmack · 2025-10-29 17:55
When I started working in python, I got lazy with “single assignment”, and I need to nudge myself about it. You should strive to never reassign or update a variable outside of true iterative calculations in loops. Having all the intermediate calculations still available is helpful in the debugger, and it avoids problems where you move a block of code and it silently uses a version of the variable that wasn’t what it originally had. In C/C++, making almost every variable const at initialization is good practice. I wish it was the default, and mutable was a keyword.

@__tinygrad__ · 2025-10-30 00:24
@ID_AA_Carmack SSA for the win!

@sharadvikram · 2025-10-29 18:11
@__tinygrad__ FYI we have guides on how to write TPU kernels using Jax+Pallas: https://t.co/1QTj8nD6y0 (which are open source).

@__tinygrad__ · 2025-10-30 00:26
@sharadvikram This still goes to XLA-TPU though, right? I should spend some time playing with this.

@AnjneyMidha · 2025-10-30 00:52
Got to see it IRL. Congrats @GillVerd and team! So crazy it might just work. Excited to see what kinds of diffusion workloads this beast can accelerate https://t.co/WOBRjUdSHX

@__tinygrad__ · 2025-10-30 00:52
@sharadvikram Interesting, there's more here than I thought! https://t.co/srwBfzr8tU

@__tinygrad__ · 2025-10-30 00:55
@sharadvikram I wonder how much this overlaps with what's actually used at Google? In general this is my biggest complaint about Google's infra, it works amazingly internally, but the released version is watered down. It works, but it's not the fast path.

@__tinygrad__ · 2025-10-30 00:55
@sharadvikram I wonder how much this overlaps with what's actually used at Google? In general this is my biggest complaint about Google's infra, it works amazingly internally, but the released version is watered down. It works, but it's not the fast path.

@__tinygrad__ · 2025-10-30 00:55
@sharadvikram Sadly still true. https://t.co/C7wCq92Nch

@__tinygrad__ · 2025-10-30 00:55
@sharadvikram I wonder how much this overlaps with what's actually used at Google? In general this is my biggest complaint about Google's infra, it works amazingly internally, but the released version is watered down. It works, but it's not the fast path.

@sharadvikram · 2025-10-30 00:59
@__tinygrad__ It's not watered down! JAX and Pallas are the same internally and externally.

@sharadvikram · 2025-10-30 00:59
@__tinygrad__ It's not watered down! JAX and Pallas are the same internally and externally.

@__tinygrad__ · 2025-10-30 01:12
@sharadvikram Nice! Yea I really like the direction things have been going with JAX (over TensorFlow), good to hear it's the same internally. Google should sell TPUs, there's no reason NVDAs market cap should be higher than Google's.

@__tinygrad__ · 2025-10-30 04:19
tinygrad's UOps can be used directly to construct programs, similar to TVM's TensorIR. https://t.co/bBURTCc4av

@__tinygrad__ · 2025-10-30 04:19
tinygrad's UOps can be used directly to construct programs, similar to TVM's TensorIR. https://t.co/bBURTCc4av

@__tinygrad__ · 2025-10-30 04:27
Here is the generated C and OpenCL code from the UOps (run with DEBUG=4 to print). Unlike similar libraries, tinygrad generates no boilerplate and has no runtime requirements. https://t.co/PNVZxbznaA

@__tinygrad__ · 2025-10-29 09:43
The Anthropic TPU deal solidifies it. There's two companies that can make training chips, NVIDIA and Google. Elon tried with Dojo. Amazon tried with Trainium. DeepSeek tried with Huawei. Countless startups are flailing with multiple tapeouts and no real adoption. The funny thing is, the TPU chip itself is very simple. The difference is all the software (XLA-TPU is the best deep learning compiler, too bad it's closed source). In 2 more years, tinygrad will be actually 1.0, with performance exceeding all other libraries. There was a lot of Dunning-Kruger on the way, but we maintain the underlying abstractions required for all deep learning style compute are very simple. At this point, the backend specific code is 1,000 lines. The Verilog for an accelerator should only be about 3,000. We'll build it on FPGAs, and then look for a partner to work with to tape it out. PS: We're still hard at work on the AMD MLPerf contract. AMD has made big strides on their mainline stack in the last 2 years. It's mirroring the NVIDIA stack, but I've come to respect this strategy more and AMD will soon be an uncontested third player in the training space.

@Visionscaper · 2025-10-30 08:05
@__tinygrad__ I hope you will start working on a tape out sooner than in two years!

@__tinygrad__ · 2025-10-30 08:16
This is the mistake so many startups have made. It plays into this ridiculous myth that if you are spending money you are successful. Write all the software first. Only consider touching super expensive silicon once the software is done and the full thing works in simulation.

@__tinygrad__ · 2025-10-30 04:27
Here is the generated C and OpenCL code from the UOps (run with DEBUG=4 to print). Unlike similar libraries, tinygrad generates no boilerplate and has no runtime requirements. https://t.co/PNVZxbznaA

@__tinygrad__ · 2025-10-30 09:03
This float32 N=4096 matmul outperforms rocBLAS on 7900XTX (something like kernel5 from seb-v's blog post). Read the UOps in extra/gemm/amd_uop_matmul.py. https://t.co/CUnYwXUWZ1

@nateberkopec · 2025-10-29 22:53
Here's where we're at. Running a 500b parameter 200k context model requires ~3-5 million USD in equipment today. Even as models get more efficient, we're still 3+ years away from that being something you can do on a local desktop.

@seuros · 2025-10-30 09:28
500b/200k is in the range 1-4xk$ usd with 96GB RAM GPUs from huawei. You can get a full rig from @__tinygrad__ if you want 5090s. Cheapest civilized solution around. You can find other cheaper solution, but they use exotic hardware. What you will lose is speed. Are you ok to receive double digit token as output ? PS: I'm not involved with TG, i just saw a rig in action.

@nateberkopec · 2025-10-29 22:53
Here's where we're at. Running a 500b parameter 200k context model requires ~3-5 million USD in equipment today. Even as models get more efficient, we're still 3+ years away from that being something you can do on a local desktop.

@__tinygrad__ · 2025-10-30 09:35
RT @qubitium: Watching tinygrad progress is a thing of beauty. So tiny basically said, why tf are we calling 10 layers of abstractions, have no control over and buggy as hell! We are going to implement tiny ops and compile 1*1 ourselves. First principle at work.

@mike64_t · 2025-10-30 10:25
Looks like tinygrads abstractions are shaping up? Not to be that guy, but Blackwell is the only true test of strength. Even if they are not focusing on blackwell, they should proof their abstractions to be suitable for it to avoid a Triton dilemma. It's the only way.

@__tinygrad__ · 2025-10-30 11:42
We're focusing on MI355X :)

@nirw4nna · 2025-10-30 11:38
@mike64_t I agree 100% actually. I mean, I like the idea of having something that looks like a compiler generate eg. simple fused kernels for unary ops but it should be opt-in. The idea of a "sufficiently smart compiler", especially with GPUs arch changing so rapidly, seems out of reach.

@__tinygrad__ · 2025-10-30 11:44
@nirw4nna @mike64_t This is exactly where we are going with this. You'll be able to interop seamlessly with the graph in this UOp language, and it should be at an abstraction layer where you can get all the perf. Someone want to try a B200 matmul?

@__tinygrad__ · 2025-10-30 11:42
We're focusing on MI355X :)

@mike64_t · 2025-10-30 11:45
@__tinygrad__ I know. But the cat is out of the bag for crazy architectures. You don’t want to be playing catch up when hardware starts to move faster than a compiler can so its good to target the worst case if you already *know* trouble is coming

@__tinygrad__ · 2025-10-30 11:44
@nirw4nna @mike64_t This is exactly where we are going with this. You'll be able to interop seamlessly with the graph in this UOp language, and it should be at an abstraction layer where you can get all the perf. Someone want to try a B200 matmul?

@__tinygrad__ · 2025-10-30 11:47
@nirw4nna @mike64_t The code for that RDNA3 faster than rocBLAS AMD matmul is https://t.co/6ml1GqgO5Q you can write B200 in a simliar style. warpgroup stuff / warp specialization should work mostly fine.

@mike64_t · 2025-10-30 11:45
@__tinygrad__ I know. But the cat is out of the bag for crazy architectures. You don’t want to be playing catch up when hardware starts to move faster than a compiler can so its good to target the worst case if you already *know* trouble is coming

@__tinygrad__ · 2025-10-30 11:47
@mike64_t You think MI355X is easier to code for than B200?

@__tinygrad__ · 2025-10-30 12:01
Get your civilized tinybox today!

@__tinygrad__ · 2025-10-30 12:02
RT @estsauver: I’m reminded of TinyGrad, and they had this excellent post about how everyone is talking about taping out new chips to compete with NVIDIA, and no one is showing they could compete by just building a new abstraction layer that’s better than CUDA/compatible with AMDs horrible drivers first.

@damageboy · 2025-10-30 12:31
Hahaha, Never deleting this app... Where am I going to find this level of entertainment. 3000 lines of Verilog code my ass. Here is just the line count for HBM4 rtl, no test assets included: https://t.co/pLTUGTChcI

@__tinygrad__ · 2025-10-30 12:01
Get your civilized tinybox today!

@seuros · 2025-10-30 12:32
It true! Your boxes are not vendor locked, use generic hardware and you actually own the hardware. Not calling names, but i recently tested a RIG for one of your competitor, EVERYTHING was mediocre or over-engineering. 2x 3000W power: check 1X Custom Motherboard: check 6x Custom GPUs: check Custom OS: check Yearly license: check Bigger case: check Incompatibility with 99% of software: check 12xx TFLOS: claimed 6 digit price tag: check. AI Support: check. Promoted by youtubers/X that never ship: check. But they beat you in these metric: They have more images and CSS on their website. So you can't win in all front.

@sahaj__b · 2025-10-30 13:57
Making a backend in go rn. I miss drizzle and better-auth DX like crazy. https://t.co/f86JQIx8fi

@seuros · 2025-10-30 12:32
It true! Your boxes are not vendor locked, use generic hardware and you actually own the hardware. Not calling names, but i recently tested a RIG for one of your competitor, EVERYTHING was mediocre or over-engineering. 2x 3000W power: check 1X Custom Motherboard: check 6x Custom GPUs: check Custom OS: check Yearly license: check Bigger case: check Incompatibility with 99% of software: check 12xx TFLOS: claimed 6 digit price tag: check. AI Support: check. Promoted by youtubers/X that never ship: check. But they beat you in these metric: They have more images and CSS on their website. So you can't win in all front.

@__tinygrad__ · 2025-10-30 15:12
@seuros thanks! yea, we build these boxes for us and for @comma_ai, selling them is kind of an afterthought. we put a lot more effort into the engineering than the marketing.

@seuros · 2025-10-30 12:32
It true! Your boxes are not vendor locked, use generic hardware and you actually own the hardware. Not calling names, but i recently tested a RIG for one of your competitor, EVERYTHING was mediocre or over-engineering. 2x 3000W power: check 1X Custom Motherboard: check 6x Custom GPUs: check Custom OS: check Yearly license: check Bigger case: check Incompatibility with 99% of software: check 12xx TFLOS: claimed 6 digit price tag: check. AI Support: check. Promoted by youtubers/X that never ship: check. But they beat you in these metric: They have more images and CSS on their website. So you can't win in all front.

@__tinygrad__ · 2025-10-30 15:12
@seuros thanks! yea, we build these boxes for us and for @comma_ai, selling them is kind of an afterthought. we put a lot more effort into the engineering than the marketing.

@__tinygrad__ · 2025-10-30 15:15
@seuros @comma_ai and ugh, that engineering is actually spent locking them down. tinyboxes are normal ubuntu we've considered a custom motherboard, but it would just be to make them *more* open, like schematics + open source BIOS and BMC.

@__tinygrad__ · 2025-10-30 15:23
We don't sell subscription. We don't sell solution. We sell computer. - 4x 5090 (full PCIe 5 x16) - server grade hardware (yet quiet) - $25,000

@__tinygrad__ · 2025-10-30 15:15
@seuros @comma_ai and ugh, that engineering is actually spent locking them down. tinyboxes are normal ubuntu we've considered a custom motherboard, but it would just be to make them *more* open, like schematics + open source BIOS and BMC.

@seuros · 2025-10-30 15:25
@__tinygrad__ @comma_ai That would be great. By custom board, it was like a PlayStation 4. It's actually a computer, but you can't add anything else without gymnastics. The rig owner has 4, tried to install Ubuntu 24.04... he ended up with a $70k brick. They didn't even do a dual boot slot. Bricked!

@seuros · 2025-10-30 15:25
@__tinygrad__ @comma_ai That would be great. By custom board, it was like a PlayStation 4. It's actually a computer, but you can't add anything else without gymnastics. The rig owner has 4, tried to install Ubuntu 24.04... he ended up with a $70k brick. They didn't even do a dual boot slot. Bricked!

@__tinygrad__ · 2025-10-30 15:29
@seuros @comma_ai We could probably get OpenBMC + coreboot working on existing tinyboxes, not sure it's worth the hardware redesign costs when what you really want is better software.

@__tinygrad__ · 2025-10-30 15:23
We don't sell subscription. We don't sell solution. We sell computer. - 4x 5090 (full PCIe 5 x16) - server grade hardware (yet quiet) - $25,000

@graykevinb · 2025-10-30 15:34
@__tinygrad__ I love that you sell to consumers. I hope no matter how big of a computer you sell its never "contact sales" when I go to your site

@graykevinb · 2025-10-30 15:34
@__tinygrad__ I love that you sell to consumers. I hope no matter how big of a computer you sell its never "contact sales" when I go to your site

@__tinygrad__ · 2025-10-30 15:36
@graykevinb if we did that we'd have to hire a sales person.

@damageboy · 2025-10-30 12:31
Hahaha, Never deleting this app... Where am I going to find this level of entertainment. 3000 lines of Verilog code my ass. Here is just the line count for HBM4 rtl, no test assets included: https://t.co/pLTUGTChcI

@__tinygrad__ · 2025-10-30 16:31
@damageboy Sounds like ~80,000 too many :) For reference, we have both an AMD and NVIDIA driver that maps the BAR into Python in our 17832 line repo (tests not included). How many lines are the vendor drivers?

@__tinygrad__ · 2025-10-30 16:31
@damageboy Sounds like ~80,000 too many :) For reference, we have both an AMD and NVIDIA driver that maps the BAR into Python in our 17832 line repo (tests not included). How many lines are the vendor drivers?

@__tinygrad__ · 2025-10-30 16:33
@damageboy That repo also includes a full ONNX parser, a PyTorch like frontend, an MLIR-like rewrite system, a symbolic algebra system, codegen for C/PTX/WebGPU/LLVM, a few more backends, a web based profiler, and a one line replacement for termcolor. So yea, I stick by ~3000.

@h_gskrd · 2025-10-30 16:40
@damageboy @__tinygrad__ No deps !

@damageboy · 2025-10-30 16:45
@h_gskrd @__tinygrad__ Well, that's kind of my point...? Any ML accelerator that someone would actually deploy needs: NoCs, Memory controllers, PCIe and probably some form of 800 GbE networking, and this is before you even multiply a single FP4 pair. They can deliver it in 3k LoC with 1M col count :)

@damageboy · 2025-10-30 16:45
@h_gskrd @__tinygrad__ Well, that's kind of my point...? Any ML accelerator that someone would actually deploy needs: NoCs, Memory controllers, PCIe and probably some form of 800 GbE networking, and this is before you even multiply a single FP4 pair. They can deliver it in 3k LoC with 1M col count :)

@__tinygrad__ · 2025-10-31 01:05
@damageboy @h_gskrd Ahh, I see. This is an ad for Broadcom. NoC + LPDDR memory + networking. No PCIe and no HBM.

@peteoxenham · 2025-10-31 04:08
Honestly, the design of this thing is just offensive. A core misunderstanding of where industrial beauty comes from in the first place All the beauty we find, in both nature and the fabricated world, arises from the need to hyper-optimize for function - we humans find that instantly beautiful and inspiring. This design, on the other hand, is hollow. No functional justification for the patterns on top, just a gen-design demo mesh. The faux heat sink-ish pattern isn’t actually an effective heat sink at all. The glyphs on the side mean nothing useful to anybody, and they don’t even spark curiosity because they are so obviously surface level. It is a larp

@__tinygrad__ · 2025-10-31 01:05
@damageboy @h_gskrd Ahh, I see. This is an ad for Broadcom. NoC + LPDDR memory + networking. No PCIe and no HBM.

@damageboy · 2025-10-31 06:31
@__tinygrad__ @h_gskrd How is LPDDR simpler than HBM? Less copies of it on silicon since there are 1/20 of the channels, same complexity. And can you think of a reason why everyone chose a combo of PCIe-like and Ethernet-like? Is everyone else stupid? Or maybe, just maybe there is a good reason?

@damageboy · 2025-10-31 06:31
@__tinygrad__ @h_gskrd How is LPDDR simpler than HBM? Less copies of it on silicon since there are 1/20 of the channels, same complexity. And can you think of a reason why everyone chose a combo of PCIe-like and Ethernet-like? Is everyone else stupid? Or maybe, just maybe there is a good reason?

@__tinygrad__ · 2025-10-31 06:40
@damageboy @h_gskrd They aren't stupid, they just have different incentives. With both, an upper middle manager can hire a PCIe team and an Ethernet team, and now they are more important because they manage double the teams!

@__tinygrad__ · 2025-10-31 07:44
Someday our magic compiler will be finished. Until then, tinygrad supports writing kernels in UOps. It allows full control over the loop-nest and memory placement+layout, while retaining a graph and high level device agnostic syntax. Tutorial coming soon. https://t.co/DhSDQEn7lW

@EitanTurok · 2025-10-31 07:57
@__tinygrad__ how do you choose whether memory is global, local, or in registers with tinygrad uops?

@etc_cz_ · 2025-10-31 08:11
@EitanTurok @__tinygrad__ You're looking at it (DEFINE_LOCAL, DEFINE_GLOBAL, DEFINE_REG)

@peteoxenham · 2025-10-31 04:08
Honestly, the design of this thing is just offensive. A core misunderstanding of where industrial beauty comes from in the first place All the beauty we find, in both nature and the fabricated world, arises from the need to hyper-optimize for function - we humans find that instantly beautiful and inspiring. This design, on the other hand, is hollow. No functional justification for the patterns on top, just a gen-design demo mesh. The faux heat sink-ish pattern isn’t actually an effective heat sink at all. The glyphs on the side mean nothing useful to anybody, and they don’t even spark curiosity because they are so obviously surface level. It is a larp

@__tinygrad__ · 2025-10-31 09:17
@peteoxenham lol they taped out a chip, I think they deserve to have some fun with a 3D printer. waiting for the buy it now button, but this seems a lot more real than some companies with a lot more funding.

@etc_cz_ · 2025-10-31 08:11
@EitanTurok @__tinygrad__ You're looking at it (DEFINE_LOCAL, DEFINE_GLOBAL, DEFINE_REG)

@__tinygrad__ · 2025-10-31 09:18
@etc_cz_ @EitanTurok We are going to unify those UOps, there's an AddrSpace enum. Won't require API change, just use placeholder. Here's the best example https://t.co/6ml1GqgO5Q

@__tinygrad__ · 2025-10-31 07:44
Someday our magic compiler will be finished. Until then, tinygrad supports writing kernels in UOps. It allows full control over the loop-nest and memory placement+layout, while retaining a graph and high level device agnostic syntax. Tutorial coming soon. https://t.co/DhSDQEn7lW

@__tinygrad__ · 2025-10-31 09:36
Should be good enough to play with. `Tensor.custom_kernel` takes a function which constructs the kernel, and optionally a hook for backward. See examples here: https://t.co/BRccvkw9fT

@__tinygrad__ · 2025-10-31 09:53
We have a booth at @comma_ai's comma con. Come talk to the tiny team and see a tinybox in person! Nov 8th in San Diego, CA. Tickets for sale on comma's website.

@__tinygrad__ · 2025-10-31 09:57
Copying and tweaking amd_uop_matmul.py for 4090 should get you the $300 GEMM speed bounty. It's so easy that AI can (almost) do it.

@vin_acct · 2025-10-31 17:50
people who don't space out their code with empty lines drive me insane

@vin_acct · 2025-10-31 17:50
people who don't space out their code with empty lines drive me insane

@gyaan_random · 2025-11-01 02:45
@vin_acct Look at the @__tinygrad__ repo. You're in for a treat. They are optimising for number of lines and so a blank line is clearly deincentivised.

@syntacrobat · 2025-11-01 04:01
literally how do ML people survive without lifting tensor dimensions into the type system? isn't that like the number one thing youd immediately want

@gyaan_random · 2025-11-01 02:45
@vin_acct Look at the @__tinygrad__ repo. You're in for a treat. They are optimising for number of lines and so a blank line is clearly deincentivised.

@__tinygrad__ · 2025-11-01 09:46
@gyaan_random @vin_acct blank lines don't count

@syntacrobat · 2025-11-01 04:01
literally how do ML people survive without lifting tensor dimensions into the type system? isn't that like the number one thing youd immediately want

@__tinygrad__ · 2025-11-01 11:28
@syntacrobat We'd love if Python's type system supported this well.

@SquashBionic · 2025-11-01 17:01
Funny thing, intel has a fork of Cutlass with SYCL support and they are working on integrating it into Pytorch inductor https://t.co/oRkPCQZCGj https://t.co/ZO6jyG6QHB

@__tinygrad__ · 2025-10-31 09:53
We have a booth at @comma_ai's comma con. Come talk to the tiny team and see a tinybox in person! Nov 8th in San Diego, CA. Tickets for sale on comma's website.

@kommentlezz · 2025-11-01 21:17
@__tinygrad__ @comma_ai What about a company/factory tour? It'd be cool to check it as well. I could come a couple of days before the event.

@kommentlezz · 2025-11-01 21:17
@__tinygrad__ @comma_ai What about a company/factory tour? It'd be cool to check it as well. I could come a couple of days before the event.

@__tinygrad__ · 2025-11-02 04:18
@kommentlezz @comma_ai That's included with a VIP ticket. "Kick off COMMA_CON weekend on Friday evening with an exclusive omakase-inspired dinner at comma HQ. Enjoy a behind-the-scenes tour, poke around the comma factory and compute cluster, and enjoy dinner and drinks with the team."

@__tinygrad__ · 2025-11-02 08:25
Before you consider a tapeout, you must master: * consumer+datacenter GPU style * Google TPU / Qualcomm DSP style * AMD AIE / Tenstorrent style Only once your software can drive and output ultra fast bytecode for all three are you ready to consider making your own hardware.

@SquashBionic · 2025-11-01 17:01
Funny thing, intel has a fork of Cutlass with SYCL support and they are working on integrating it into Pytorch inductor https://t.co/oRkPCQZCGj https://t.co/ZO6jyG6QHB

@__tinygrad__ · 2025-11-02 08:35
@SquashBionic It won't matter without leadership. https://t.co/9FcM5tm8v1

@KrakowiakK · 2025-11-02 09:48
Current state of MLX (Studio M3 Ultra) Full GPU utilisation, ANE not utilised at all. Seems that Apple Neural Engine has nothing to do https://t.co/EG4fRTilWh

@jarredsumner · 2025-11-02 14:42
Can someone explain to me why ~500 tok/s is fast and what in-the-weeds technical constraints prevent 100,000 tok/s at same quality? My gut is there’s incredible waste due to infinite money and in a world w/ 1/10000th of the capital models would be orders of magnitude better

@never_released · 2025-11-02 17:45
> Google TPU/Qualcomm DSP style the two are actually quite different. Hexagon is very CPU-y because it _is_ a cpu, it's much closer to the AMX/SME style of matrix extensions than anything else

@__tinygrad__ · 2025-11-03 02:27
All these laptops are shipping with NPUs. Intel + AMD + Apple + Qualcomm all have one, and they are often heavily featured in the marketing material. Is anyone using them? If so, how?

@__tinygrad__ · 2025-11-03 02:27
All these laptops are shipping with NPUs. Intel + AMD + Apple + Qualcomm all have one, and they are often heavily featured in the marketing material. Is anyone using them? If so, how?

@FirmTorsion · 2025-11-03 02:35
@__tinygrad__ I have it on inside accounts that more than one of these vendors are leaning towards just using the integrated GPUs and killing these things. They suck and nobody is using them

@FirmTorsion · 2025-11-03 02:35
@__tinygrad__ I have it on inside accounts that more than one of these vendors are leaning towards just using the integrated GPUs and killing these things. They suck and nobody is using them

@__tinygrad__ · 2025-11-03 02:40
@FirmTorsion I think it varies by company. Apple's addition of real tensor cores to M5 looks that way for them. Qualcomm has a long history of DSPs, so I don't think that should go away. AMD's is interesting and well documented, but looks very unused. And then there's hopeless Intel.

@__tinygrad__ · 2025-11-03 02:40
@FirmTorsion I think it varies by company. Apple's addition of real tensor cores to M5 looks that way for them. Qualcomm has a long history of DSPs, so I don't think that should go away. AMD's is interesting and well documented, but looks very unused. And then there's hopeless Intel.

@FirmTorsion · 2025-11-03 02:42
@__tinygrad__ The problem for intel at least was that the core was too far away from the rest of the system, and the compiler was so immature it could barely handle a few matmul shapes with activations. If the compiler was ready, maybe they could have iterated on the memory situation.

@FirmTorsion · 2025-11-03 02:42
@__tinygrad__ The problem for intel at least was that the core was too far away from the rest of the system, and the compiler was so immature it could barely handle a few matmul shapes with activations. If the compiler was ready, maybe they could have iterated on the memory situation.

@__tinygrad__ · 2025-11-03 02:49
@FirmTorsion The problem for Intel is they have 0 leadership that sets direction. I'm sure nobody there knows what the NPUs are for beyond "oh it's like AI in the laptop" Apple was early. AMD repurposed Xilinx. And the Qualcomm DSP actually makes sense and (sadly) has the best sw of the 4.

@__tinygrad__ · 2025-11-03 02:49
@FirmTorsion The problem for Intel is they have 0 leadership that sets direction. I'm sure nobody there knows what the NPUs are for beyond "oh it's like AI in the laptop" Apple was early. AMD repurposed Xilinx. And the Qualcomm DSP actually makes sense and (sadly) has the best sw of the 4.

@FirmTorsion · 2025-11-03 02:52
@__tinygrad__ Apple knew a few years ago that it wasn't working. Probably had an internal standoff and GPU won. For AMD Xilinx, the software looked clusterfucked and took months to get a relatively small network running on.

@FirmTorsion · 2025-11-03 02:52
@__tinygrad__ Apple knew a few years ago that it wasn't working. Probably had an internal standoff and GPU won. For AMD Xilinx, the software looked clusterfucked and took months to get a relatively small network running on.

@__tinygrad__ · 2025-11-03 02:55
@FirmTorsion I put similar time into AMD and Intel and did at least manage to get AMD's software running. Unlike what I could find with Intel, the NPU is fully documented and I do think architectures do eventually look more like this. So there's some hope. https://t.co/ChEqdkZ4vt

@FirmTorsion · 2025-11-03 03:00
@__tinygrad__ Interesting I hadn't seen them using MLIR. It seems like there's basically 2 kinds of chips now, GPU's and systolic things (TPU, Cerebras, AIE). Hopefully the systolic world converges on basic facts like how GPUs converged on hierarchy of cores and memory.

@__tinygrad__ · 2025-11-03 03:07
@FirmTorsion I don't think AIE looks like TPU. AIE is a bunch (10-20) of little cores with like 1 TOP each, TPU is 1-2 cores. Main difference between GPU and AIE is that GPU has a huge crossbar and one L2 cache; AIE has memory core. Crossbar is easier to program but $$$ and not needed.

@__tinygrad__ · 2025-11-03 03:07
@FirmTorsion I don't think AIE looks like TPU. AIE is a bunch (10-20) of little cores with like 1 TOP each, TPU is 1-2 cores. Main difference between GPU and AIE is that GPU has a huge crossbar and one L2 cache; AIE has memory core. Crossbar is easier to program but $$$ and not needed.

@__tinygrad__ · 2025-11-03 03:10
@FirmTorsion Hmm, I guess if you zoom out to a pod TPU looks like AIE :)

@__tinygrad__ · 2025-11-03 02:27
All these laptops are shipping with NPUs. Intel + AMD + Apple + Qualcomm all have one, and they are often heavily featured in the marketing material. Is anyone using them? If so, how?

@mhkabirr · 2025-11-03 03:15
@__tinygrad__ We use Intel laptop SoCs with NPU on our drones, and prev used Myriad. Running latency-critical CNNs/transformers for our autonomy engine. Perhaps contrarian take, but having also used the Qualcomm/Hexagon stack, I would pick OpenVINO + Intel every time for shipping models fast.

@mhkabirr · 2025-11-03 03:15
@__tinygrad__ We use Intel laptop SoCs with NPU on our drones, and prev used Myriad. Running latency-critical CNNs/transformers for our autonomy engine. Perhaps contrarian take, but having also used the Qualcomm/Hexagon stack, I would pick OpenVINO + Intel every time for shipping models fast.

@__tinygrad__ · 2025-11-03 03:29
@mhkabirr Agree the software isn't great, but Qualcomm's DSP has good docs and some decent open source stuff. The main selling point is you know Qualcomm will still be making the same DSP 10 years from now, with Intel, who knows. https://t.co/XAq9Jp8jLz

@graykevinb · 2025-11-03 04:06
Some abstractions are good. @__tinygrad__ is one such abstraction

@__tinygrad__ · 2025-11-03 04:28
Keys to being a good abstraction: * No leaking! * Be a library with no/few deps (not a framework!) * Very few ideas you cleverly combine * Short code you can read and hack on * Well tested and easy to reason about

@__tinygrad__ · 2025-11-03 04:28
Keys to being a good abstraction: * No leaking! * Be a library with no/few deps (not a framework!) * Very few ideas you cleverly combine * Short code you can read and hack on * Well tested and easy to reason about

@graykevinb · 2025-11-03 04:32
@__tinygrad__ I'm just impresssd by how easy it is to implement on a new device

@graykevinb · 2025-11-03 04:32
@__tinygrad__ I'm just impresssd by how easy it is to implement on a new device

@__tinygrad__ · 2025-11-03 04:37
@graykevinb It should get even easier. Did you know all IF/ENDIF can be drop in replaced with RANGE/END? And there's even more we can push into the middleware, making the device abstraction as small as possible.

@__tinygrad__ · 2025-11-03 04:56
Apple locked the ANE down so much that's it's unusable. If they documented and exposed a real programming model for it (like Metal), *maybe* people would use it, but not like this.

@never_released · 2025-11-02 17:45
> Google TPU/Qualcomm DSP style the two are actually quite different. Hexagon is very CPU-y because it _is_ a cpu, it's much closer to the AMX/SME style of matrix extensions than anything else

@__tinygrad__ · 2025-11-03 05:01
@never_released Hexagon is not like a modern CPU. It uses VLIW instruction packets, it does nothing out of order, and it requires explicit cache management to get speed.

@jarredsumner · 2025-11-02 14:42
Can someone explain to me why ~500 tok/s is fast and what in-the-weeds technical constraints prevent 100,000 tok/s at same quality? My gut is there’s incredible waste due to infinite money and in a world w/ 1/10000th of the capital models would be orders of magnitude better

@__tinygrad__ · 2025-11-03 06:15
@jarredsumner You have to access ~every weight (or ~10% in MoE models) per token. It comes down to memory bandwidth. If you have a dense model with 100B parameters at 8-bit quantization, that's 100 GB of data. 500 tok/s is 50 TB/s. 100k tok/s would be 10 PB/s. Where u buy that kind of ram?

@__tinygrad__ · 2025-11-03 06:15
@jarredsumner You have to access ~every weight (or ~10% in MoE models) per token. It comes down to memory bandwidth. If you have a dense model with 100B parameters at 8-bit quantization, that's 100 GB of data. 500 tok/s is 50 TB/s. 100k tok/s would be 10 PB/s. Where u buy that kind of ram?

@__tinygrad__ · 2025-11-03 06:23
@jarredsumner 50 TB/s is what AMD or NVIDIA AI node can do, 8 GPUs with 8 TB/s of HBM each + overhead. You can go a bit faster with multinode, but providers target $/tok, not tok/s, and you hit latency issues quickly. Modern models are 2T MoE at FP4. (50 TB/s) / (0.1 TB/token) = 500 tok/s

@__tinygrad__ · 2025-11-03 02:27
All these laptops are shipping with NPUs. Intel + AMD + Apple + Qualcomm all have one, and they are often heavily featured in the marketing material. Is anyone using them? If so, how?

@HdNpu · 2025-11-03 06:24
@__tinygrad__ Runs selected llm on ryzen ai https://t.co/BO7uxVTLlx

@HdNpu · 2025-11-03 06:24
@__tinygrad__ Runs selected llm on ryzen ai https://t.co/BO7uxVTLlx

@__tinygrad__ · 2025-11-03 06:26
@HdNpu Oh cool, Windows only? Or work on Linux?

@__tinygrad__ · 2025-11-03 06:23
@jarredsumner 50 TB/s is what AMD or NVIDIA AI node can do, 8 GPUs with 8 TB/s of HBM each + overhead. You can go a bit faster with multinode, but providers target $/tok, not tok/s, and you hit latency issues quickly. Modern models are 2T MoE at FP4. (50 TB/s) / (0.1 TB/token) = 500 tok/s

@jarredsumner · 2025-11-03 06:37
@__tinygrad__ why are even 10% of the weights necessary? why not like 0.001%? how well do people understand how each weight impacts the output? I don’t really mean quantization, I mean deleting most of the weights or some post-processing step that removes redundancy / not useful data

@jarredsumner · 2025-11-03 06:37
@__tinygrad__ why are even 10% of the weights necessary? why not like 0.001%? how well do people understand how each weight impacts the output? I don’t really mean quantization, I mean deleting most of the weights or some post-processing step that removes redundancy / not useful data

@__tinygrad__ · 2025-11-03 07:06
@jarredsumner The brain has around 100T synapses, it's MoE, ~4 bit quantized, and runs at ~200 tok/s. (this is 1 PB/s btw) You can ask all these same questions about people.

@__tinygrad__ · 2025-11-03 04:56
Apple locked the ANE down so much that's it's unusable. If they documented and exposed a real programming model for it (like Metal), *maybe* people would use it, but not like this.

@shingjan_ · 2025-11-03 19:55
@__tinygrad__ Can you elaborate on what kind of programming model for ANE you are interested in? Something like triton for kernel authoring but on ANE?

@shingjan_ · 2025-11-03 19:55
@__tinygrad__ Can you elaborate on what kind of programming model for ANE you are interested in? Something like triton for kernel authoring but on ANE?

@__tinygrad__ · 2025-11-03 22:09
@shingjan_ If you just expose it (not through Core ML, but be able to run raw .hwx files) and document it that's good enough.

@__tinygrad__ · 2025-11-04 01:02
Hey @Apple is there any way to get "com<dot>apple.developer.driverkit.allow-any-userclient-access" so Python running on my computer can talk to the GPU plugged into my computer? Do you really want users to have to disable SIP to use this?

@__tinygrad__ · 2025-11-04 01:02
Hey @Apple is there any way to get "com<dot>apple.developer.driverkit.allow-any-userclient-access" so Python running on my computer can talk to the GPU plugged into my computer? Do you really want users to have to disable SIP to use this?

@gamestoneai · 2025-11-04 01:08
@__tinygrad__ @Apple Apple's security model clashing with ML workloads again. SIP protecting us from our own GPUs now?

@__tinygrad__ · 2025-11-04 01:06
@Apple I can't wait until there's either good Linux for Macbooks or a decent laptop that's not Apple. This will all come back to bite someday. Granting capabilities to known devs piecemeal, not sure if you can even get the one you need. Eventually the world will move on without you.

@__tinygrad__ · 2025-11-04 01:10
@Apple I actually found an exploit to bypass this, but I'm too old for that cat and mouse game. Code is here. https://t.co/EFFW2gw4de

@__tinygrad__ · 2025-11-04 01:10
@Apple I actually found an exploit to bypass this, but I'm too old for that cat and mouse game. Code is here. https://t.co/EFFW2gw4de

@__tinygrad__ · 2025-11-04 01:12
@Apple Like this permission is ridiculous. It doesn't do anything for security, it just makes packaging a PITA. tinygrad Python calls IOServiceOpen, I just want that to be able to open my signed driver that I have the entitlement for without signing custom Python. Too much to ask?

@__tinygrad__ · 2025-11-04 01:02
Hey @Apple is there any way to get "com<dot>apple.developer.driverkit.allow-any-userclient-access" so Python running on my computer can talk to the GPU plugged into my computer? Do you really want users to have to disable SIP to use this?

@chrisprice · 2025-11-04 01:14
@__tinygrad__ @Apple Have they outright refused, or are they just dragging you along?

@chrisprice · 2025-11-04 01:14
@__tinygrad__ @Apple Have they outright refused, or are they just dragging you along?

@__tinygrad__ · 2025-11-04 01:16
@chrisprice @Apple They granted "com<dot>apple.developer.driverkit" "com<dot>apple.developer.driverkit.transport.pci" and "com<dot>apple.developer.driverkit.allow-third-party-userclients" but didn't grant "com<dot>apple.developer.driverkit.allow-any-userclient-access" and no refusal just nothing

@gamestoneai · 2025-11-04 01:08
@__tinygrad__ @Apple Apple's security model clashing with ML workloads again. SIP protecting us from our own GPUs now?

@__tinygrad__ · 2025-11-04 01:17
@gamestoneai @Apple We got the entitlement for pci driverkit, but you can't open your driver from python, only something signed. Like making code work is hard enough without this bullshit.

@__tinygrad__ · 2025-11-04 01:02
Hey @Apple is there any way to get "com<dot>apple.developer.driverkit.allow-any-userclient-access" so Python running on my computer can talk to the GPU plugged into my computer? Do you really want users to have to disable SIP to use this?

@Droogstoppel13 · 2025-11-04 01:25
@__tinygrad__ @Apple Environments that throw up hurdles for dev lose against environments that welcome dev. Say what you will about MS, but their dev tools are good.

@Droogstoppel13 · 2025-11-04 01:25
@__tinygrad__ @Apple Environments that throw up hurdles for dev lose against environments that welcome dev. Say what you will about MS, but their dev tools are good.

@__tinygrad__ · 2025-11-04 01:27
@Droogstoppel13 @Apple MS makes good dev tools, but Windows requires you to make a cloud account to use it. I wouldn't ever use that as a daily computer, they make it clear who really owns it with the first screen.

@__tinygrad__ · 2025-11-04 06:28
https://t.co/KXCMpAiM5J

@__tinygrad__ · 2025-11-04 18:22
New product alert! For those looking for even more power. There's tinybox pro v2 8x 5090, 5U rackable workstation $50,000 Ships in 4-12 weeks Available to order on website https://t.co/PYsSG2smHa

@__tinygrad__ · 2025-11-04 18:22
New product alert! For those looking for even more power. There's tinybox pro v2 8x 5090, 5U rackable workstation $50,000 Ships in 4-12 weeks Available to order on website https://t.co/PYsSG2smHa

@latentspacehack · 2025-11-04 18:23
@__tinygrad__ Ambient and full load temps on this bad boy?

@latentspacehack · 2025-11-04 18:23
@__tinygrad__ Ambient and full load temps on this bad boy?

@__tinygrad__ · 2025-11-04 18:25
@latentspacehack Wildly good. GPUs are around 70C at full load. It's high pressure server grade fans blowing across GPU heatsinks made for low pressure ones.

@__tinygrad__ · 2025-11-04 18:22
New product alert! For those looking for even more power. There's tinybox pro v2 8x 5090, 5U rackable workstation $50,000 Ships in 4-12 weeks Available to order on website https://t.co/PYsSG2smHa

@sweinoid · 2025-11-04 18:36
@__tinygrad__ at this scale, would 2x blackwell 6000's make more sense?

@sweinoid · 2025-11-04 18:36
@__tinygrad__ at this scale, would 2x blackwell 6000's make more sense?

@__tinygrad__ · 2025-11-04 18:39
@sweinoid there's no scale where blackwell 6000s make sense. just buy more 5090s. it's literally the same chip.

@__tinygrad__ · 2025-11-04 18:22
New product alert! For those looking for even more power. There's tinybox pro v2 8x 5090, 5U rackable workstation $50,000 Ships in 4-12 weeks Available to order on website https://t.co/PYsSG2smHa

@graykevinb · 2025-11-04 18:44
@__tinygrad__ Not to be that guy, but this isn't enough power. However I think you know that and are hard at work on even more powerful stuff. One day we'll be able to train foundation models at home

@graykevinb · 2025-11-04 18:44
@__tinygrad__ Not to be that guy, but this isn't enough power. However I think you know that and are hard at work on even more powerful stuff. One day we'll be able to train foundation models at home

@__tinygrad__ · 2025-11-04 18:45
@graykevinb If you want more power, network a bunch of these together. In AI land, there's only $/performance, and I believe this is the most cost effective off the shelf solution for people in the $50k-$5M range.

@__tinygrad__ · 2025-11-04 18:22
New product alert! For those looking for even more power. There's tinybox pro v2 8x 5090, 5U rackable workstation $50,000 Ships in 4-12 weeks Available to order on website https://t.co/PYsSG2smHa

@witizenship · 2025-11-04 19:19
@__tinygrad__ Will you all eventually be moving over to the New RTX PRO 96GB for these boxes? Would make them super compact for the same VRAM or run the largest models. Would make them fantastic for training also.

@SIGKITTEN · 2025-11-04 19:52
you gotta stop misleading people with "its literally the same chip", RTX 6000 Pro is 1.5x faster than 5090 for matmuls (bench from @TheZachMueller) https://t.co/E1mcScnntn

@witizenship · 2025-11-04 19:19
@__tinygrad__ Will you all eventually be moving over to the New RTX PRO 96GB for these boxes? Would make them super compact for the same VRAM or run the largest models. Would make them fantastic for training also.

@__tinygrad__ · 2025-11-04 19:52
@witizenship The level to which people buy into NVIDIA marketing is unbelievable. People complain about Apple's markup on ram, but when NVIDIA marks up ram more than anyone else in history. WOW 64GB OF RAM FOR $6,000 WHAT DEAL!

@__tinygrad__ · 2025-11-04 19:54
@SIGKITTEN @TheZachMueller It's the same chip. That's an eFuse controlling dispatch rate of a single type of tensor core instruction.

@SIGKITTEN · 2025-11-04 19:54
@__tinygrad__ @TheZachMueller it doesnt matter that it's the same chip, it's fucking slower.

@SIGKITTEN · 2025-11-04 19:54
@__tinygrad__ @TheZachMueller it doesnt matter that it's the same chip, it's fucking slower.

@__tinygrad__ · 2025-11-04 19:56
@SIGKITTEN @TheZachMueller I'm sorry you paid $6,000 for 64GB of RAM and an intact eFuse.

@__tinygrad__ · 2025-11-04 20:39
RT @anemll: Confirmed new @__tinygrad__ on Nvidia GPU (5070) works on macOS Sequoia (M1 MAX). Fun! https://t.co/t9DvTqhViG

@SIGKITTEN · 2025-11-04 21:06
imagine paying 50k for 5090s lol

@__tinygrad__ · 2025-11-04 22:12
If people really want the RTX Pro 6000 Blackwell cards, we'll build a tinybox green variant with them. $60k https://t.co/pTkQ44z96o

@__tinygrad__ · 2025-11-04 22:12
If people really want the RTX Pro 6000 Blackwell cards, we'll build a tinybox green variant with them. $60k https://t.co/pTkQ44z96o

@Its_keith_d · 2025-11-04 23:13
@__tinygrad__ What about the new AMD pro R9700 32gb cards?

@SIGKITTEN · 2025-11-04 23:14
Ok hold on a sec. I'm no IMO olympiad, but the math doesnt seem to be math-ing. rtx6000 is $9k/ea. That leaves $24000 for the rest rtx5090 is $3k/ea. That leaves $13000 for the rest. But "the rest" is exactly the same in both cases. https://t.co/sz5Zh5LALM

@Its_keith_d · 2025-11-04 23:13
@__tinygrad__ What about the new AMD pro R9700 32gb cards?

@__tinygrad__ · 2025-11-05 00:38
@Keithdavis85 We'll see if anyone buys this one.

@__tinygrad__ · 2025-11-05 00:40
@AutisticDropout I predict nobody will buy it even at $60k. There might be shills on X for the PRO 6000 cards, but when it comes to opening the wallet, the $25k box with 4x 5090s is clearly a better choice. But at least now there's an option to showcase how good of a deal that one really is.

@zackangelo · 2025-11-05 00:43
@__tinygrad__ @AutisticDropout do your 5090 boxes support p2p nccl? the 6000s support it without patching right?

@zackangelo · 2025-11-05 00:43
@__tinygrad__ @AutisticDropout do your 5090 boxes support p2p nccl? the 6000s support it without patching right?

@__tinygrad__ · 2025-11-05 00:48
@zackangelo @AutisticDropout We just added a picture to the product page that'll answer your Q. https://t.co/nGRxle3Npe

@__tinygrad__ · 2025-11-05 02:36
Will anyone buy the $60k 4x RTX Pro 6000 machine?

@SIGKITTEN · 2025-11-04 23:14
Ok hold on a sec. I'm no IMO olympiad, but the math doesnt seem to be math-ing. rtx6000 is $9k/ea. That leaves $24000 for the rest rtx5090 is $3k/ea. That leaves $13000 for the rest. But "the rest" is exactly the same in both cases. https://t.co/sz5Zh5LALM

@__tinygrad__ · 2025-11-05 02:38
@SIGKITTEN Our markup is a *lot* lower than NVIDIAs.

@SIGKITTEN · 2025-11-05 18:36
I tried to price out the actual tinybox v2 server components (minus GPUs and chassis), and I came up with $15k. The 5u 31" deep chassis is kinda hard to find, but I doubt it'd be more than like $1k So, $16k for the base server Add in 8x5090s @ 3k, $40k in components, $10k to tinybox. It's not unreasonable if what you actually want is 8x5090s. But idk why you'd do that.

@__tinygrad__ · 2025-11-05 19:54
Show me literally anything more cost effective.

@SIGKITTEN · 2025-11-04 21:06
imagine paying 50k for 5090s lol

@SIGKITTEN · 2025-11-05 19:59
https://t.co/hXsA0EI9mo

@__tinygrad__ · 2025-11-05 19:54
Show me literally anything more cost effective.

@SIGKITTEN · 2025-11-05 20:02
@__tinygrad__ for LLMs, 4x6000 Pro workstation edition, will still be under 50k total and you get a lot more ram

@kyiii10 · 2025-11-05 20:04
@SIGKITTEN @__tinygrad__ $6k for 64gb extra isn't cost effective lmao.

@SIGKITTEN · 2025-11-05 20:04
@kyiii10 @__tinygrad__ it will train a lot faster than this

@SIGKITTEN · 2025-11-05 20:04
@kyiii10 @__tinygrad__ it will train a lot faster than this

@__tinygrad__ · 2025-11-05 20:09
@SIGKITTEN @kyiii10 Want to bet $10k it won't? Also, do you have a link to buy that machine for $50k?

@SIGKITTEN · 2025-11-05 19:59
https://t.co/hXsA0EI9mo

@__tinygrad__ · 2025-11-05 20:12
@SIGKITTEN Sounds like you should start a competing business.

@__tinygrad__ · 2025-11-05 20:09
@SIGKITTEN @kyiii10 Want to bet $10k it won't? Also, do you have a link to buy that machine for $50k?

@SIGKITTEN · 2025-11-05 20:14
@__tinygrad__ @kyiii10 yeah lets do a nanochat training race. idk if i wanna bet $10k as you seem kinda shady. and yeah, link right here: https://t.co/MaSDV9lLjW

@__tinygrad__ · 2025-11-05 20:14
There's a whole bunch of people who talk in this space who don't understand it. If you want to run your moderately large LLM at 6 tok/s, buy a Mac Studio or DGX Spark with 128GB of RAM. Congrats, you are an AI influencer! Then when you turn the camera off, you get frustrated by the slow speeds and low quality outputs and you end up back using ChatGPT. Don't worry, I won't tell. I understand few have had to think about RAM bandwidth before when looking at computers, but it's the main thing that determines the speed of your LLMs. A tinybox pro has ~16 TB/s of RAM bandwidth, equivalent to 2 GB300s (~$80k each). For the same RAM bandwidth, the tinyboxes are 3x cheaper than the datacenter GPUs. Re: but you lack interconnect bandwidth. There's a 16x PCIe 5 link between the GPUs, that's 128 GB/s bidirectional. This gives you 32 GB/s of allreduce bandwidth, meaning you can sync every byte of every cards' memory in under a second. This is rarely a bottleneck. tinyboxes are the most cost effective machines in the space. If you don't believe me, try to find a machine with better FLOPS/$ or GB/s/$.

@__tinygrad__ · 2025-11-05 20:12
@SIGKITTEN Sounds like you should start a competing business.

@SIGKITTEN · 2025-11-05 20:15
@__tinygrad__ there's no need for a business lol

@SIGKITTEN · 2025-11-05 20:14
@__tinygrad__ @kyiii10 yeah lets do a nanochat training race. idk if i wanna bet $10k as you seem kinda shady. and yeah, link right here: https://t.co/MaSDV9lLjW

@__tinygrad__ · 2025-11-05 20:16
@SIGKITTEN @kyiii10 That's a link to "get pricing" where does it say $50k? I'd do it for a $10k bet, not worth my time otherwise.

@SIGKITTEN · 2025-11-05 20:15
@__tinygrad__ there's no need for a business lol

@__tinygrad__ · 2025-11-05 20:17
@SIGKITTEN lol you know you can't compete. do you also look at restaurant menus and post instacart screenshots?

@__tinygrad__ · 2025-11-05 20:16
@SIGKITTEN @kyiii10 That's a link to "get pricing" where does it say $50k? I'd do it for a $10k bet, not worth my time otherwise.

@SIGKITTEN · 2025-11-05 20:17
@__tinygrad__ @kyiii10 its literally just running 1 script you don't have to do much. dont you wanna provide some benchmarks for your product other than the 2 shitty screenshots you have on the site?

@SIGKITTEN · 2025-11-05 20:17
@__tinygrad__ @kyiii10 its literally just running 1 script you don't have to do much. dont you wanna provide some benchmarks for your product other than the 2 shitty screenshots you have on the site?

@__tinygrad__ · 2025-11-05 20:18
@SIGKITTEN @kyiii10 Another great idea for your competing business!

@__tinygrad__ · 2025-11-05 20:14
There's a whole bunch of people who talk in this space who don't understand it. If you want to run your moderately large LLM at 6 tok/s, buy a Mac Studio or DGX Spark with 128GB of RAM. Congrats, you are an AI influencer! Then when you turn the camera off, you get frustrated by the slow speeds and low quality outputs and you end up back using ChatGPT. Don't worry, I won't tell. I understand few have had to think about RAM bandwidth before when looking at computers, but it's the main thing that determines the speed of your LLMs. A tinybox pro has ~16 TB/s of RAM bandwidth, equivalent to 2 GB300s (~$80k each). For the same RAM bandwidth, the tinyboxes are 3x cheaper than the datacenter GPUs. Re: but you lack interconnect bandwidth. There's a 16x PCIe 5 link between the GPUs, that's 128 GB/s bidirectional. This gives you 32 GB/s of allreduce bandwidth, meaning you can sync every byte of every cards' memory in under a second. This is rarely a bottleneck. tinyboxes are the most cost effective machines in the space. If you don't believe me, try to find a machine with better FLOPS/$ or GB/s/$.

@AllInOnBlack · 2025-11-05 20:19
@__tinygrad__ @SIGKITTEN

@__tinygrad__ · 2025-11-05 20:17
@SIGKITTEN lol you know you can't compete. do you also look at restaurant menus and post instacart screenshots?

@SIGKITTEN · 2025-11-05 20:19
@__tinygrad__ i dont wanna compete, brother. just want you to sell your product without lying to people that all they need is the 5090s for whatever reason u think somehow that sticks it to NVIDIA

@SIGKITTEN · 2025-11-05 20:19
@__tinygrad__ i dont wanna compete, brother. just want you to sell your product without lying to people that all they need is the 5090s for whatever reason u think somehow that sticks it to NVIDIA

@__tinygrad__ · 2025-11-05 20:21
@SIGKITTEN I understand the deep regret you must feel having paid $6000 for 64GB of RAM.

@AllInOnBlack · 2025-11-05 20:19
@__tinygrad__ @SIGKITTEN

@__tinygrad__ · 2025-11-05 20:22
@AllInOnBlack @SIGKITTEN Btw, the RTX Pro 6000 Blackwell has the same RAM bandwidth as a 5090! Same bandwidth, 3x cost.

@__tinygrad__ · 2025-11-05 20:22
@AllInOnBlack @SIGKITTEN Btw, the RTX Pro 6000 Blackwell has the same RAM bandwidth as a 5090! Same bandwidth, 3x cost.

@SIGKITTEN · 2025-11-05 20:24
@__tinygrad__ @AllInOnBlack bro what part of batch size doesnt make sense to you

@SIGKITTEN · 2025-11-05 20:24
@__tinygrad__ @AllInOnBlack bro what part of batch size doesnt make sense to you

@__tinygrad__ · 2025-11-05 20:26
@SIGKITTEN @AllInOnBlack You know the FLOPS used scale with batch size, right? You think that we can't get high MFU for some reason?

@SIGKITTEN · 2025-11-05 20:02
@__tinygrad__ for LLMs, 4x6000 Pro workstation edition, will still be under 50k total and you get a lot more ram

@__tinygrad__ · 2025-11-05 20:28
@SIGKITTEN Not only it is not faster, I don't believe this box is available for $50k anywhere. We sell it for $60k. https://t.co/pTkQ44z96o

@__tinygrad__ · 2025-11-05 20:26
@SIGKITTEN @AllInOnBlack You know the FLOPS used scale with batch size, right? You think that we can't get high MFU for some reason?

@SIGKITTEN · 2025-11-05 20:30
@__tinygrad__ @AllInOnBlack i dont see how u gonna beat a (nanochat) training run vs 4x6000 with 1/3 less total ram and 25% total tflops no

@SIGKITTEN · 2025-11-05 20:30
@__tinygrad__ @AllInOnBlack i dont see how u gonna beat a (nanochat) training run vs 4x6000 with 1/3 less total ram and 25% total tflops no

@__tinygrad__ · 2025-11-05 20:32
@SIGKITTEN @AllInOnBlack I'm willing to bet $10k, you aren't.

@__tinygrad__ · 2025-11-05 20:32
@SIGKITTEN @AllInOnBlack I'm willing to bet $10k, you aren't.

@SIGKITTEN · 2025-11-05 20:33
@__tinygrad__ @AllInOnBlack i dont trust the shit you gonna pull, you've got a lot more clout

@__tinygrad__ · 2025-11-05 19:54
Show me literally anything more cost effective.

@CEOofFuggy · 2025-11-05 20:35
@__tinygrad__ Buying terabytes of ram and waiting for heat death of the universe for one token

@SIGKITTEN · 2025-11-05 20:33
@__tinygrad__ @AllInOnBlack i dont trust the shit you gonna pull, you've got a lot more clout

@__tinygrad__ · 2025-11-05 20:36
@SIGKITTEN @AllInOnBlack We can determine rules and both send the money to a neutral person who judges. The 4x PRO 6000 vs 8x 5090 showdown! I think you just know you'd lose.

@__tinygrad__ · 2025-11-05 20:14
There's a whole bunch of people who talk in this space who don't understand it. If you want to run your moderately large LLM at 6 tok/s, buy a Mac Studio or DGX Spark with 128GB of RAM. Congrats, you are an AI influencer! Then when you turn the camera off, you get frustrated by the slow speeds and low quality outputs and you end up back using ChatGPT. Don't worry, I won't tell. I understand few have had to think about RAM bandwidth before when looking at computers, but it's the main thing that determines the speed of your LLMs. A tinybox pro has ~16 TB/s of RAM bandwidth, equivalent to 2 GB300s (~$80k each). For the same RAM bandwidth, the tinyboxes are 3x cheaper than the datacenter GPUs. Re: but you lack interconnect bandwidth. There's a 16x PCIe 5 link between the GPUs, that's 128 GB/s bidirectional. This gives you 32 GB/s of allreduce bandwidth, meaning you can sync every byte of every cards' memory in under a second. This is rarely a bottleneck. tinyboxes are the most cost effective machines in the space. If you don't believe me, try to find a machine with better FLOPS/$ or GB/s/$.

@Lon · 2025-11-05 20:45
You love to conveniently flip the narrative: - yesterday: RTX 6000 Pro 1.8TB/s bandwidth: problematic - today: 32 GB/s all-reduce “all you need” And leave out gotchas - PCIe 5 is 64 GB/s unidirectional - PCIe 5 is serial p2p - p2p DMA on 5090 is crippled - sync overhead is 10-30%

@__tinygrad__ · 2025-11-05 20:14
There's a whole bunch of people who talk in this space who don't understand it. If you want to run your moderately large LLM at 6 tok/s, buy a Mac Studio or DGX Spark with 128GB of RAM. Congrats, you are an AI influencer! Then when you turn the camera off, you get frustrated by the slow speeds and low quality outputs and you end up back using ChatGPT. Don't worry, I won't tell. I understand few have had to think about RAM bandwidth before when looking at computers, but it's the main thing that determines the speed of your LLMs. A tinybox pro has ~16 TB/s of RAM bandwidth, equivalent to 2 GB300s (~$80k each). For the same RAM bandwidth, the tinyboxes are 3x cheaper than the datacenter GPUs. Re: but you lack interconnect bandwidth. There's a 16x PCIe 5 link between the GPUs, that's 128 GB/s bidirectional. This gives you 32 GB/s of allreduce bandwidth, meaning you can sync every byte of every cards' memory in under a second. This is rarely a bottleneck. tinyboxes are the most cost effective machines in the space. If you don't believe me, try to find a machine with better FLOPS/$ or GB/s/$.

@HotAisle · 2025-11-05 20:49
@__tinygrad__ It boils down to the rent vs. buy argument because of deployment complexities. You can't just buy 2x GB300 chips.

@__tinygrad__ · 2025-11-05 20:14
There's a whole bunch of people who talk in this space who don't understand it. If you want to run your moderately large LLM at 6 tok/s, buy a Mac Studio or DGX Spark with 128GB of RAM. Congrats, you are an AI influencer! Then when you turn the camera off, you get frustrated by the slow speeds and low quality outputs and you end up back using ChatGPT. Don't worry, I won't tell. I understand few have had to think about RAM bandwidth before when looking at computers, but it's the main thing that determines the speed of your LLMs. A tinybox pro has ~16 TB/s of RAM bandwidth, equivalent to 2 GB300s (~$80k each). For the same RAM bandwidth, the tinyboxes are 3x cheaper than the datacenter GPUs. Re: but you lack interconnect bandwidth. There's a 16x PCIe 5 link between the GPUs, that's 128 GB/s bidirectional. This gives you 32 GB/s of allreduce bandwidth, meaning you can sync every byte of every cards' memory in under a second. This is rarely a bottleneck. tinyboxes are the most cost effective machines in the space. If you don't believe me, try to find a machine with better FLOPS/$ or GB/s/$.

@jjjwaynejjj · 2025-11-05 20:50
@__tinygrad__ When will be the first george x pewdiepie stream?

@Lon · 2025-11-05 20:45
You love to conveniently flip the narrative: - yesterday: RTX 6000 Pro 1.8TB/s bandwidth: problematic - today: 32 GB/s all-reduce “all you need” And leave out gotchas - PCIe 5 is 64 GB/s unidirectional - PCIe 5 is serial p2p - p2p DMA on 5090 is crippled - sync overhead is 10-30%

@__tinygrad__ · 2025-11-05 21:03
@Lon - 1.8 TB/s (to DRAM) isn't problematic, it's quite good! It's just the same as the 5090. - NVIDIA uses bidirectional numbers also, we even say bidirectional. - Our driver reenables p2p DMA on 5090. - Serial p2p? They all go at same time. - And overhead is more like 5-10%

@HotAisle · 2025-11-05 20:49
@__tinygrad__ It boils down to the rent vs. buy argument because of deployment complexities. You can't just buy 2x GB300 chips.

@__tinygrad__ · 2025-11-05 21:03
@HotAisle You can buy this. https://t.co/vyQlgGh6m3

@jjjwaynejjj · 2025-11-05 20:50
@__tinygrad__ When will be the first george x pewdiepie stream?

@__tinygrad__ · 2025-11-05 21:05
@jjjwaynejjj Unlike a lot of the newegg cart posters here, @pewdiepie actually built the computer!

@CEOofFuggy · 2025-11-05 20:35
@__tinygrad__ Buying terabytes of ram and waiting for heat death of the universe for one token

@__tinygrad__ · 2025-11-05 21:07
@CEOofFuggy Have you seen how cheap DDR4 RAM is? 😂

@__tinygrad__ · 2025-11-04 06:28
https://t.co/KXCMpAiM5J

@paulmarin90 · 2025-11-05 21:08
@__tinygrad__ So, should I take it that a partial SIP disable like: "csrutil enable --without kext --without fs" does NOT work?

@paulmarin90 · 2025-11-05 21:08
@__tinygrad__ So, should I take it that a partial SIP disable like: "csrutil enable --without kext --without fs" does NOT work?

@__tinygrad__ · 2025-11-05 21:09
@paulmarin90 It probably does, haven't tried.

@__tinygrad__ · 2025-11-05 21:22
tinybox pro v2s are flying off the shelves! https://t.co/tugCbT2ZfC

@DanAdvantage · 2025-11-05 22:36
@KinvertOG @__tinygrad__ @SIGKITTEN @AllInOnBlack who would actually touch this? lol @theo

@SIGKITTEN · 2025-11-05 22:38
@DanAdvantage @KinvertOG @__tinygrad__ @AllInOnBlack @theo noone would vs a random cat. also, pretty clear he was bluffing esp since no response after i said yes

@SIGKITTEN · 2025-11-04 23:14
Ok hold on a sec. I'm no IMO olympiad, but the math doesnt seem to be math-ing. rtx6000 is $9k/ea. That leaves $24000 for the rest rtx5090 is $3k/ea. That leaves $13000 for the rest. But "the rest" is exactly the same in both cases. https://t.co/sz5Zh5LALM

@SvenMeyer · 2025-11-05 23:12
@SIGKITTEN You are right, @__tinygrad__ is charging a ridiculous amount for just putting some off the shelf PC components together. Anybody who considers buying such a home mode DIY machine can do that himself and isn't that just the fun part why you want to have it in the first place? 😅

@SIGKITTEN · 2025-11-04 23:14
Ok hold on a sec. I'm no IMO olympiad, but the math doesnt seem to be math-ing. rtx6000 is $9k/ea. That leaves $24000 for the rest rtx5090 is $3k/ea. That leaves $13000 for the rest. But "the rest" is exactly the same in both cases. https://t.co/sz5Zh5LALM

@__tinygrad__ · 2025-11-05 23:23
Restaurants are charging absurd amounts for just putting some off the shelf food items together. Anybody who considers buying a $17 pasta dish can can do that himself for like $3.50 and they get the fun part of boiling water and asking why this pasta is so mushy!

@__tinygrad__ · 2025-11-05 23:23
Restaurants are charging absurd amounts for just putting some off the shelf food items together. Anybody who considers buying a $17 pasta dish can can do that himself for like $3.50 and they get the fun part of boiling water and asking why this pasta is so mushy!

@__tinygrad__ · 2025-11-05 23:26
And in before "the restaurant has nice ambience" or something, bro you know you ordered that $17 pasta on Uber Eats for $25 total and ate it with a plastic fork next to your computer.

@__tinygrad__ · 2025-11-04 22:12
If people really want the RTX Pro 6000 Blackwell cards, we'll build a tinybox green variant with them. $60k https://t.co/pTkQ44z96o

@sarlev_ · 2025-11-05 23:31
@__tinygrad__ ok but how about bringing back the tinybox red? the 10k price point is where things are really interesting.

@__tinygrad__ · 2025-11-04 22:12
If people really want the RTX Pro 6000 Blackwell cards, we'll build a tinybox green variant with them. $60k https://t.co/pTkQ44z96o

@sarlev_ · 2025-11-05 23:31
@__tinygrad__ ok but how about bringing back the tinybox red? the 10k price point is where things are really interesting.

@__tinygrad__ · 2025-11-05 23:33
4x AMD RX 9070 box for $10k?

@SIGKITTEN · 2025-11-05 22:38
@DanAdvantage @KinvertOG @__tinygrad__ @AllInOnBlack @theo noone would vs a random cat. also, pretty clear he was bluffing esp since no response after i said yes

@__tinygrad__ · 2025-11-05 23:35
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo Find something we both trust to judge and I'm down. FYI, I don't own any Pro 6000 cards, that's your half.

@__tinygrad__ · 2025-11-05 23:35
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo Find something we both trust to judge and I'm down. FYI, I don't own any Pro 6000 cards, that's your half.

@SIGKITTEN · 2025-11-05 23:39
@__tinygrad__ @DanAdvantage @KinvertOG @AllInOnBlack @theo im willing to put $10k for you to benchmark your own products that you sell, on a nanochat training run idk

@__tinygrad__ · 2025-11-05 23:33
4x AMD RX 9070 box for $10k?

@onthewrongplane · 2025-11-05 23:41
@__tinygrad__ How about the 9700Pro? It's not that much more expensive.

@SIGKITTEN · 2025-11-05 23:39
@__tinygrad__ @DanAdvantage @KinvertOG @AllInOnBlack @theo im willing to put $10k for you to benchmark your own products that you sell, on a nanochat training run idk

@__tinygrad__ · 2025-11-05 23:55
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo I don't exactly follow. You want to wire me $10k to benchmark 8x5090 and 4xPRO6000 on nanochat? I'd do that, and send you $20k if the 4xPRO6000 is faster. Deal? If so I'll send wiring instructions.

@__tinygrad__ · 2025-11-05 23:55
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo I don't exactly follow. You want to wire me $10k to benchmark 8x5090 and 4xPRO6000 on nanochat? I'd do that, and send you $20k if the 4xPRO6000 is faster. Deal? If so I'll send wiring instructions.

@SIGKITTEN · 2025-11-06 00:01
@__tinygrad__ @DanAdvantage @KinvertOG @AllInOnBlack @theo you missed the whole neutral party part u suggested but something like that yeah

@onthewrongplane · 2025-11-05 23:41
@__tinygrad__ How about the 9700Pro? It's not that much more expensive.

@__tinygrad__ · 2025-11-06 00:02
@onthewrongplane 1) It's a loud blower. 2) It's much worse on FLOPS/$ and GB/s/$. My theory about these cards is people only like overpaying for RAM when they aren't actually paying.

@SIGKITTEN · 2025-11-06 00:01
@__tinygrad__ @DanAdvantage @KinvertOG @AllInOnBlack @theo you missed the whole neutral party part u suggested but something like that yeah

@__tinygrad__ · 2025-11-06 00:04
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo In this arrangement, I have to front the $35k to buy the PRO6000s, I want the $10k in my hand if I have to do that. I'll post the wandb links to both training runs. We have a deal?

@__tinygrad__ · 2025-11-05 23:33
4x AMD RX 9070 box for $10k?

@scaling01 · 2025-11-06 00:05
@__tinygrad__ okay that's the dumbest one I have seen so far 4 x $600 GPU and somehow you get to 10k?

@SIGKITTEN · 2025-11-06 00:12
we'll put in escrow like you suggested and someone neutral to verify u didnt plimit or some other shit. its not crazy for companies to send their products to various tests. after all, you're the geohotz, and the company selling these, you are making the claims with no proof and you challenged a random cat on twitter to a bet

@__tinygrad__ · 2025-11-06 00:23
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo Happy to do that if you provide the RTX6000s, we only sell that machine built to order. I'm not overpaying for RAM out of pocket.

@scaling01 · 2025-11-06 00:05
@__tinygrad__ okay that's the dumbest one I have seen so far 4 x $600 GPU and somehow you get to 10k?

@__tinygrad__ · 2025-11-06 00:26
@scaling01 Can you link to someone selling it cheaper?

@__tinygrad__ · 2025-11-06 00:23
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo Happy to do that if you provide the RTX6000s, we only sell that machine built to order. I'm not overpaying for RAM out of pocket.

@__tinygrad__ · 2025-11-06 00:33
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo Or how about this. You buy the 4xPRO6000 machine from us and wire me $32k. I'll benchmark them both. If it wins, I send it to you. If it loses, it's $28k more to get it (or you can decline it). Will give you SSH to both machines to confirm plimits or w/e else.

@__tinygrad__ · 2025-11-06 00:33
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo Or how about this. You buy the 4xPRO6000 machine from us and wire me $32k. I'll benchmark them both. If it wins, I send it to you. If it loses, it's $28k more to get it (or you can decline it). Will give you SSH to both machines to confirm plimits or w/e else.

@SIGKITTEN · 2025-11-06 00:40
@__tinygrad__ @DanAdvantage @KinvertOG @AllInOnBlack @theo its an interesting proposition but idk I wanna drop 32k vs the original 10k just to get you to run a script on the hardware you sell. Also, if I did, I'd want the machine in my house.

@SIGKITTEN · 2025-11-06 00:40
@__tinygrad__ @DanAdvantage @KinvertOG @AllInOnBlack @theo its an interesting proposition but idk I wanna drop 32k vs the original 10k just to get you to run a script on the hardware you sell. Also, if I did, I'd want the machine in my house.

@__tinygrad__ · 2025-11-06 00:43
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo That's less than the cost we have to pay for the GPUs, and we'd be eating more than $10k if we lose, vs just selling the product for a fair price (find it cheaper) if we win. Okay, we'll be here when you are serious about this.

@AlpinDale · 2025-11-06 00:44
The extra $20-25k you will be paying for this is what we call a "retard tax". If you don't think you can build something like this yourself, then yes, you should just buy this. Besides, it funds tinygrad development so you'll be doing us all a favor.

@__tinygrad__ · 2025-11-06 00:49
@AlpinDale Do you know anyone selling these boxes for $25k-$30k? If so, we absolutely shouldn't be building them, we should just be buying from them!

@AlpinDale · 2025-11-06 00:59
@__tinygrad__ https://t.co/PTCAw0x3f5

@__tinygrad__ · 2025-11-05 23:23
Restaurants are charging absurd amounts for just putting some off the shelf food items together. Anybody who considers buying a $17 pasta dish can can do that himself for like $3.50 and they get the fun part of boiling water and asking why this pasta is so mushy!

@augeeidos · 2025-11-06 01:42
@__tinygrad__ Many people have been building gaming rigs since they were kids. You're making this sound like it requires rocket science.

@AlpinDale · 2025-11-06 00:59
@__tinygrad__ https://t.co/PTCAw0x3f5

@__tinygrad__ · 2025-11-06 01:43
@AlpinDale How much heavy lifting is the "something like" doing? If you want to use a mining chassis, questionable PSUs without connection to BMC, a box fan, and MCIO adapters that give AER errors, I believe it. If you want a box ready to put in a rack, I strongly doubt you'll beat $50k.

@augeeidos · 2025-11-06 01:42
@__tinygrad__ Many people have been building gaming rigs since they were kids. You're making this sound like it requires rocket science.

@__tinygrad__ · 2025-11-06 01:45
@augeeidos Show me a gaming rig with 8x 5090s. Sure, it's not as hard as building a rocket, but it's not building a gaming PC with off the shelf parts either. Making the pasta is a lot easier, and yet, you Uber Eats.

@__tinygrad__ · 2025-11-06 01:45
@augeeidos Show me a gaming rig with 8x 5090s. Sure, it's not as hard as building a rocket, but it's not building a gaming PC with off the shelf parts either. Making the pasta is a lot easier, and yet, you Uber Eats.

@augeeidos · 2025-11-06 01:47
@__tinygrad__ https://t.co/dhN3iLzG79

@__tinygrad__ · 2025-11-06 01:43
@AlpinDale How much heavy lifting is the "something like" doing? If you want to use a mining chassis, questionable PSUs without connection to BMC, a box fan, and MCIO adapters that give AER errors, I believe it. If you want a box ready to put in a rack, I strongly doubt you'll beat $50k.

@qubitium · 2025-11-06 02:37
@__tinygrad__ @AlpinDale Love you two so I will chip in as someone who did build something similar (4090 variant). First. The pricing for the tiny 5090 is higher than I would like cause I am kind of cheap. =) Maybe a $10k mark up would be more resonable but then again I don't have the labor cost/stats for these exact build from tinygard. There are like max 2 companies in the world that makes the high quality mcio cables/connectors you need for the mb to gpu card extension at pcie 5.0 speeds. There is literally zero publically sold mcio pcie connector/cable that connects the mcio end to the gpu pcie slot itself at rated pcie 5.0 speed without errors. I cannot even the find the pcie 5.0 parts even in Chinese taobao (many are saying pcie 5.0 compatible but I can tell the pcb is only for 4.0 and designed for 4.0). All the parts on newegg/ebay/amazon are actually rated for pcie 4.0 and even some of these pcie 4.0 ones are also highly suspect in quality (I know because I bought some of them). From what I saw, tinygard either outsourced the pcie 5.0 gpu to cable connector (the board that connect the pcie gpu connector to the cable itself) or made it themselves which is kinda of crazy if true. You can build it but it's crazy how hard it is to pass full pcie4.0 speed without errors. Pcie 5.0 is like 10x more sensitive to errors so just finding the cable and connector is like a 2-3 week validation work. So yes it is more expensive then one would like and yes you can building it yourself and still have money left over to buy 8x more gpus but if you are a company, it is way cheaper than alternatives.

@__tinygrad__ · 2025-11-06 01:43
@AlpinDale How much heavy lifting is the "something like" doing? If you want to use a mining chassis, questionable PSUs without connection to BMC, a box fan, and MCIO adapters that give AER errors, I believe it. If you want a box ready to put in a rack, I strongly doubt you'll beat $50k.

@augeeidos · 2025-11-06 01:47
@__tinygrad__ https://t.co/dhN3iLzG79

@__tinygrad__ · 2025-11-06 03:07
@augeeidos Ahh, do you think you can use that for ML? People don't know what they don't know.

@__tinygrad__ · 2025-11-06 03:12
bUt I BuILT a gAMinG PC wHeN I wAs 7 iTs jUsT liKE tHAt We use custom PCBs and pay $$$ for MCIO cables. Circa 2023, Amazon/Newegg stuff couldn't even do PCIe4, never mind PCIe5. If you think tinybox is expensive, compete with us. There's a reason nobody is cheaper.

@__tinygrad__ · 2025-11-06 03:07
@augeeidos Ahh, do you think you can use that for ML? People don't know what they don't know.

@augeeidos · 2025-11-06 03:15
@__tinygrad__ https://t.co/RV0yDHgNJ2

@augeeidos · 2025-11-06 03:15
@__tinygrad__ https://t.co/RV0yDHgNJ2

@__tinygrad__ · 2025-11-06 03:16
@augeeidos And the price with 4x5090 is...$42,590!

@__tinygrad__ · 2025-11-06 03:12
bUt I BuILT a gAMinG PC wHeN I wAs 7 iTs jUsT liKE tHAt We use custom PCBs and pay $$$ for MCIO cables. Circa 2023, Amazon/Newegg stuff couldn't even do PCIe4, never mind PCIe5. If you think tinybox is expensive, compete with us. There's a reason nobody is cheaper.

@DanAdvantage · 2025-11-06 03:29
@__tinygrad__ yeah but are your pcbs transparent

@__tinygrad__ · 2025-11-06 03:29
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo Deal (on the former). Let's just do pretraining, fastest run wins. Who wants to referee this / hold the escrow?

@__tinygrad__ · 2025-11-06 03:30
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy do you think 8x5090s or 4xPRO6000s does nanochat pretraining faster? want to hold 20k while we figure it out?

@__tinygrad__ · 2025-11-06 03:30
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy do you think 8x5090s or 4xPRO6000s does nanochat pretraining faster? want to hold 20k while we figure it out?

@notreallythoo · 2025-11-06 03:33
@__tinygrad__ @SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy should be a @Polymarket

@notreallythoo · 2025-11-06 03:33
@__tinygrad__ @SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy should be a @Polymarket

@__tinygrad__ · 2025-11-06 03:33
@notreallythoo @SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy @Polymarket Oh good point, polymarket can do the escrow. We just need an honest judge.

@__tinygrad__ · 2025-11-06 03:30
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy do you think 8x5090s or 4xPRO6000s does nanochat pretraining faster? want to hold 20k while we figure it out?

@__tinygrad__ · 2025-11-06 03:39
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy Some rules: 1) It's a loss target, not fixed code. Things that don't change the math of training, like grad accumulation / deepspeed / BS change are fine. 2) Changing the training itself is not allowed, aka no changing dataset, optimizer, etc... 3) Fastest run in 2 weeks?

@__tinygrad__ · 2025-11-06 03:39
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy Some rules: 1) It's a loss target, not fixed code. Things that don't change the math of training, like grad accumulation / deepspeed / BS change are fine. 2) Changing the training itself is not allowed, aka no changing dataset, optimizer, etc... 3) Fastest run in 2 weeks?

@SIGKITTEN · 2025-11-06 03:46
@__tinygrad__ @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy 1) not fixed code, but we're training nanochat, so the architecture should be preserved. grad accum/ds/bs changes are fine. the evals should also match 2) correct, dataset, optimizer is what's in the repo 3) 2 weeks works for me, we can extend if u need, np

@SIGKITTEN · 2025-11-06 03:46
@__tinygrad__ @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy 1) not fixed code, but we're training nanochat, so the architecture should be preserved. grad accum/ds/bs changes are fine. the evals should also match 2) correct, dataset, optimizer is what's in the repo 3) 2 weeks works for me, we can extend if u need, np

@DanAdvantage · 2025-11-06 03:51
@SIGKITTEN @__tinygrad__ @KinvertOG @AllInOnBlack @theo @karpathy in b4 the cuda kernel

@SIGKITTEN · 2025-11-06 03:46
@__tinygrad__ @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy 1) not fixed code, but we're training nanochat, so the architecture should be preserved. grad accum/ds/bs changes are fine. the evals should also match 2) correct, dataset, optimizer is what's in the repo 3) 2 weeks works for me, we can extend if u need, np

@__tinygrad__ · 2025-11-06 03:54
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy Yes, preserve model arch also. These are the same rules for MLPerf. We need a judge and someone who knows how to use polymarket (or a diff escrow system). We'll start the clock when both money is in, with an option to extend if we both agree.

@DanAdvantage · 2025-11-06 03:51
@SIGKITTEN @__tinygrad__ @KinvertOG @AllInOnBlack @theo @karpathy in b4 the cuda kernel

@__tinygrad__ · 2025-11-06 03:55
@DanAdvantage @SIGKITTEN @KinvertOG @AllInOnBlack @theo @karpathy The spirit of this is to do it all in public, I think if it's CUDA kernel magic (like I found secret good flash attention) it's a push/extension.

@__tinygrad__ · 2025-11-06 03:55
@DanAdvantage @SIGKITTEN @KinvertOG @AllInOnBlack @theo @karpathy The spirit of this is to do it all in public, I think if it's CUDA kernel magic (like I found secret good flash attention) it's a push/extension.

@SIGKITTEN · 2025-11-06 03:58
@__tinygrad__ @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy itd be a great win if you end up unraveling more stuff like that

@__tinygrad__ · 2025-11-06 03:54
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy Yes, preserve model arch also. These are the same rules for MLPerf. We need a judge and someone who knows how to use polymarket (or a diff escrow system). We'll start the clock when both money is in, with an option to extend if we both agree.

@SIGKITTEN · 2025-11-06 04:01
@__tinygrad__ @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy oh, and im talking about the smallest nanochat model - d20, in case it wasnt assumed

@SIGKITTEN · 2025-11-06 04:01
@__tinygrad__ @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy oh, and im talking about the smallest nanochat model - d20, in case it wasnt assumed

@__tinygrad__ · 2025-11-06 04:17
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy Agreed. 1) Write up rules in a doc 2) Find judge we both trust 3) Set up escrow system we both trust And I'll wire the money.

@__tinygrad__ · 2025-11-06 03:29
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo Deal (on the former). Let's just do pretraining, fastest run wins. Who wants to referee this / hold the escrow?

@gallabytes · 2025-11-06 04:19
@__tinygrad__ @SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo I'm happy to judge this & hold escrow

@gallabytes · 2025-11-06 04:19
@__tinygrad__ @SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo I'm happy to judge this & hold escrow

@__tinygrad__ · 2025-11-06 04:21
@gallabytes @SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo I trust you. @SIGKITTEN ?

@__tinygrad__ · 2025-11-06 04:21
@gallabytes @SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo I trust you. @SIGKITTEN ?

@SIGKITTEN · 2025-11-06 04:24
@__tinygrad__ @gallabytes @DanAdvantage @KinvertOG @AllInOnBlack @theo yes i trust him

@SIGKITTEN · 2025-11-06 04:24
@__tinygrad__ @gallabytes @DanAdvantage @KinvertOG @AllInOnBlack @theo yes i trust him

@__tinygrad__ · 2025-11-06 04:30
@SIGKITTEN @gallabytes @DanAdvantage @KinvertOG @AllInOnBlack @theo Great, let's do this! @gallabytes since you are the judge, want to write up the rules / your judging criteria? And send wiring info? george@tinygrad.org Let's start the contest Monday, we have @comma_ai COMMA_CON to prepare for.

@SIGKITTEN · 2025-11-06 03:58
@__tinygrad__ @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy itd be a great win if you end up unraveling more stuff like that

@__tinygrad__ · 2025-11-06 05:44
@SIGKITTEN @DanAdvantage @KinvertOG @AllInOnBlack @theo @karpathy Oh I'm not saying not to look for things like that, I'm saying that the goal of this bet is to settle once and for all which is faster. FWIW, I'm only 70% confident it's the 5090s.

@__tinygrad__ · 2025-11-06 05:47
RT @johndeanl: My company @WindBorneWx has built and operated nearly 10x custom servers like this. Its works for us but there is a lot of issues and time investments that come up. It's not a retard tax to just pay more and get something pre built, it's also a massive timesaver. Tinygrad pricing seems very reasonable to me.

@DanAdvantage · 2025-11-06 03:29
@__tinygrad__ yeah but are your pcbs transparent

@__tinygrad__ · 2025-11-06 05:47
@DanAdvantage they green

@seuros · 2025-11-06 14:56
New episode of Dramafunding. https://t.co/4o32SCqC7J feat @__tinygrad__ and @SIGKITTEN

@__tinygrad__ · 2025-11-06 15:47
A decent summary of the 4xPRO6000 vs 8x5090 challenge. We definitely need a betting market so you can play along at home.

@SIGKITTEN · 2025-11-06 17:05
This is such a cool challenge that I think I'm actually rooting for geohotz to win. I don't think the numbers are even close and if he does figure it out, I think its easily worth the $10k to find out how.

@gallabytes · 2025-11-06 18:03
@SIGKITTEN @JustinWaugh changing the precision doesn't seem like an architectural departure to me, it adds complexity though. but like, if one chip supports fp8 and the other doesn't that's a meaningful differentiation, since the metric we're actually interested in is cost effective experimentation?

@gallabytes · 2025-11-06 18:07
@SIGKITTEN @JustinWaugh @__tinygrad__ wdyt? I'm inclined to call precision changes fair game but it is ultimately a contest between you and @SIGKITTEN I'm just here to adjudicate disputes.

@Sentdex · 2025-11-06 20:01
can anyone actually explain, with hard facts, how it's possible AMD still doesn't meaningfully compete in the deep learning space? I understand they lack certain pieces of software. I do not understand how they could reasonably still lack it though.

@SIGKITTEN · 2025-11-06 03:18
ok how about this we each put up the 10k in escrow and each do our nanochat (https://t.co/2LIVT73koE) speedruns. You train on the tinybox server with 8x5090, I'll rent and pay for a 4x6000 Pro WS and give u root and everything (we can stop after pretraining, or do the full, idc, as long as it runs the evals so we can make sure the model is proper). Whoever's training run finishes in the shortest time and produces the expected model wins. or: I send u 8000 (if we can figure out some guarantee u wont just block me after) and if I win, you send me a $25k tinybox v2. If you win, keep the 8k obv.

@anemll · 2025-11-06 20:17
@SIGKITTEN @__tinygrad__ @DanAdvantage @KinvertOG @AllInOnBlack @theo Isn't BF16 with FP32 accumulation nerfed in the 5090? IMHO, If this is a hardware showdown, rules should lock precision (e.g., BF16 compute + FP32 accumulate), otherwise it’s a software/precision contest?

@gallabytes · 2025-11-06 18:07
@SIGKITTEN @JustinWaugh @__tinygrad__ wdyt? I'm inclined to call precision changes fair game but it is ultimately a contest between you and @SIGKITTEN I'm just here to adjudicate disputes.

@__tinygrad__ · 2025-11-06 21:50
@gallabytes @SIGKITTEN @JustinWaugh I think precision change is fair game. I want to get to the root of which set of cards is better though, so I don't want it to be that one person figured out better precision option.

@__tinygrad__ · 2025-11-06 21:50
@gallabytes @SIGKITTEN @JustinWaugh I think precision change is fair game. I want to get to the root of which set of cards is better though, so I don't want it to be that one person figured out better precision option.

@SIGKITTEN · 2025-11-06 22:00
I think we should agree on it first, as it's not an insignificant amount of work to make nanochat work in other than the bf16 precision. If you end up running training on fp8/mxp4 but I train on bf16 we're not really testing the hardware the same way. We can hash it out on Monday/next week, i know you're busy, enjoy comma-con! https://t.co/TIClug0ize

@__tinygrad__ · 2025-11-06 21:50
@gallabytes @SIGKITTEN @JustinWaugh I think precision change is fair game. I want to get to the root of which set of cards is better though, so I don't want it to be that one person figured out better precision option.

@gallabytes · 2025-11-06 22:00
so I think there's 2 options here: 1. running as naively as possible, exact same code, which setup is better? imo we should measure in tokens trained on per time interval not loss. 2. with some more serious tuning, which setup is better? this we probably have to measure loss. thoughts?

@SIGKITTEN · 2025-11-06 22:00
I think we should agree on it first, as it's not an insignificant amount of work to make nanochat work in other than the bf16 precision. If you end up running training on fp8/mxp4 but I train on bf16 we're not really testing the hardware the same way. We can hash it out on Monday/next week, i know you're busy, enjoy comma-con! https://t.co/TIClug0ize

@__tinygrad__ · 2025-11-06 22:12
@SIGKITTEN @gallabytes @JustinWaugh Ok, lock in bf16.

@gallabytes · 2025-11-06 22:00
so I think there's 2 options here: 1. running as naively as possible, exact same code, which setup is better? imo we should measure in tokens trained on per time interval not loss. 2. with some more serious tuning, which setup is better? this we probably have to measure loss. thoughts?

@__tinygrad__ · 2025-11-06 22:13
@gallabytes @SIGKITTEN @JustinWaugh Hmm, there's not really a naively as possible for 8x5090, I think it's thing by thing. I'm okay locking in bf16.

@__tinygrad__ · 2025-11-06 22:12
@SIGKITTEN @gallabytes @JustinWaugh Ok, lock in bf16.

@SIGKITTEN · 2025-11-06 22:13
@__tinygrad__ @gallabytes @JustinWaugh damn bruh u sure? :)

@SIGKITTEN · 2025-11-06 22:13
@__tinygrad__ @gallabytes @JustinWaugh damn bruh u sure? :)

@__tinygrad__ · 2025-11-06 22:13
@SIGKITTEN @gallabytes @JustinWaugh I mean, I don't have to accumulate in fp32 :) But yea.

@SIGKITTEN · 2025-11-07 02:10
fwiw, i got a quote for a 4x6000 Pro workstation build from a vendor. It's not a site you click on and buy but all i did was emailed a vendor and asked for a 7975wx with 4 6000 pros and got this back like an hour later https://t.co/xOCrXziOcP

@Sentdex · 2025-11-06 20:01
can anyone actually explain, with hard facts, how it's possible AMD still doesn't meaningfully compete in the deep learning space? I understand they lack certain pieces of software. I do not understand how they could reasonably still lack it though.

@__tinygrad__ · 2025-11-07 04:05
@Sentdex We've been working on it for 3 years. The software is super hard. Give it 2 more.

@waltertayannlee · 2025-11-07 09:55
tinygrad uses strides to implement convolution. very cool, and there's a clean proof for why this works: 1. Calculate the index for your n-dimensional matrix 2. Take the derivatives of your index with respect to the dimensions 3. Derivatives tell you how much your index changes when values in dim changes (i.e. what your stride aims to do) so plug those in directly to your stride values 4. ??? 5. Profit

@waltertayannlee · 2025-11-07 09:55
tinygrad uses strides to implement convolution. very cool, and there's a clean proof for why this works: 1. Calculate the index for your n-dimensional matrix 2. Take the derivatives of your index with respect to the dimensions 3. Derivatives tell you how much your index changes when values in dim changes (i.e. what your stride aims to do) so plug those in directly to your stride values 4. ??? 5. Profit

@__tinygrad__ · 2025-11-07 17:31
@waltertayannlee That's a really old tinygrad!

@mov_axbx · 2025-11-06 10:11
@max_paperclips @__tinygrad__ I wrote one of the most popular guides on the topic and upfront I say go buy a tinybox lol

@MightyMogomra · 2025-11-07 22:05
@mov_axbx @max_paperclips @__tinygrad__ I'm guessing tinygrad is just milking this for attention, since most people with the money to buy a device this expensive will think completely differently from most of the tweetsters arguing with them

@__tinygrad__ · 2025-11-08 01:13
RT @MightyMogomra: @__tinygrad__ Explaining to my boss why I made my $2,000 a day ML engineers spend two weeks building a janky unsupported AI rig to save $10k

@MightyMogomra · 2025-11-07 22:05
@mov_axbx @max_paperclips @__tinygrad__ I'm guessing tinygrad is just milking this for attention, since most people with the money to buy a device this expensive will think completely differently from most of the tweetsters arguing with them

@__tinygrad__ · 2025-11-08 01:14
@MightyMogomra @mov_axbx @max_paperclips Shop traffic https://t.co/3bNH4BU9Fp

@__tinygrad__ · 2025-11-08 01:13
@SIGKITTEN Your computer to compete with? Single stick of RAM? tinybox has 192GB and all channels. I'm curious if those PCIe extenders work. That motherboard doesn't have MCIO ports. And no BMC. And no 4 drive raid array. And can the case fans handle 2500W? But yea good budget option.

@SIGKITTEN · 2025-11-08 01:15
@__tinygrad__ there are numbers on the right for quantity 😂 its just a quote, relax lol

@__tinygrad__ · 2025-11-08 01:16
RT @owainkenway: @waltertayannlee @ptrschmdtnlsn My testing of tinygrad is limited to “I installed it yesterday on our Mi300x system and ran some SDXL examples and trained a model with the pytorch frontend” and I’d say it’s primary strength is how low faff it is to get working.

@__tinygrad__ · 2025-11-08 01:16
RT @EthanBThoma: @waltertayannlee @ptrschmdtnlsn I use pytorch and tinygrad. It is just nice to use and would opt for using it over pytorch. I like the ideas, the API, and I'm sure they'll make it competitive speed-wise in time

@SIGKITTEN · 2025-11-08 01:15
@__tinygrad__ there are numbers on the right for quantity 😂 its just a quote, relax lol

@__tinygrad__ · 2025-11-08 01:18
@SIGKITTEN Oh it has 8 RAMs well played

@__tinygrad__ · 2025-11-08 01:18
@SIGKITTEN Oh it has 8 RAMs well played

@SIGKITTEN · 2025-11-08 01:22
@__tinygrad__ its not that terrible, you gotta agree. not perfect picks, but we're talking like what, $1-2k difference to fix those. Plus, I can remove Windows and save a few bucks :D Anyway, fwiw, I think the only real overpriced product u got is the new $60k one.

@__tinygrad__ · 2025-11-08 01:23
Okay fine our 4x6000 Pro machine was overpriced. I have growing respect for the card, so the price is now $50k. tinybox uses MCIO cables (not extenders), has a server motherboard with a BMC, a 4 drive 4TB Data SSD RAID array for 4x bandwidth, and a beautiful Noctua fan wall.

@__tinygrad__ · 2025-11-08 01:23
Okay fine our 4x6000 Pro machine was overpriced. I have growing respect for the card, so the price is now $50k. tinybox uses MCIO cables (not extenders), has a server motherboard with a BMC, a 4 drive 4TB Data SSD RAID array for 4x bandwidth, and a beautiful Noctua fan wall.

@__tinygrad__ · 2025-11-08 01:27
Link to buy. Now both entries in the showdown cost $50k. https://t.co/pTkQ44z96o https://t.co/PD5PSHqSHN

@SIGKITTEN · 2025-11-08 01:22
@__tinygrad__ its not that terrible, you gotta agree. not perfect picks, but we're talking like what, $1-2k difference to fix those. Plus, I can remove Windows and save a few bucks :D Anyway, fwiw, I think the only real overpriced product u got is the new $60k one.

@__tinygrad__ · 2025-11-08 01:30
@SIGKITTEN lol yea it was overpriced i didn't actually want anyone to buy it but meh if the people want it the people shall get it the customer is always right

@__tinygrad__ · 2025-11-08 01:23
Okay fine our 4x6000 Pro machine was overpriced. I have growing respect for the card, so the price is now $50k. tinybox uses MCIO cables (not extenders), has a server motherboard with a BMC, a 4 drive 4TB Data SSD RAID array for 4x bandwidth, and a beautiful Noctua fan wall.

@taha_yssne · 2025-11-08 01:37
@__tinygrad__ you can just bully companies into lowering down their prices

@taha_yssne · 2025-11-08 01:37
@__tinygrad__ you can just bully companies into lowering down their prices

@__tinygrad__ · 2025-11-08 01:50
@taha_yssne lol only that one was overpriced. I was jealous of NVIDIA's margin on the card and wanted some for myself. I have now given up that dream.

@__tinygrad__ · 2025-11-08 01:23
Okay fine our 4x6000 Pro machine was overpriced. I have growing respect for the card, so the price is now $50k. tinybox uses MCIO cables (not extenders), has a server motherboard with a BMC, a 4 drive 4TB Data SSD RAID array for 4x bandwidth, and a beautiful Noctua fan wall.

@ITangoI · 2025-11-08 01:53
@__tinygrad__ Noctua fans too? Good stuff

@ITangoI · 2025-11-08 01:53
@__tinygrad__ Noctua fans too? Good stuff

@__tinygrad__ · 2025-11-08 01:55
@ITangoI We have two tinyboxes in our office, it's really important they are quiet!

@__tinygrad__ · 2025-11-08 01:23
Okay fine our 4x6000 Pro machine was overpriced. I have growing respect for the card, so the price is now $50k. tinybox uses MCIO cables (not extenders), has a server motherboard with a BMC, a 4 drive 4TB Data SSD RAID array for 4x bandwidth, and a beautiful Noctua fan wall.

@xie_uh · 2025-11-08 02:00
@__tinygrad__ So the 5090 vs 6000 battle is off and you yield?

@xie_uh · 2025-11-08 02:00
@__tinygrad__ So the 5090 vs 6000 battle is off and you yield?

@__tinygrad__ · 2025-11-08 07:04
@HeadBetwixKnees haha nope! we just want to profit off both sides, i learned about it in MBA class it's called hedgeing

@HotAisle · 2025-11-09 22:24
Below is the talk George Hotz gave at COMMA CON. The event itself was run really well. Almost felt like there was more staff than participants, which isn't necessarily a bad thing! Met some interesting people, and his talk had a few interesting observations. I deeply understand his frustration with scammers and clueless people getting absurd amounts of funding. But honestly, I don’t think having that same pile of money would mean he’d have solved self-driving by now. My own take: I don’t really care much about the self-driving stuff because I don’t own a vehicle that supports it and probably won't for a long time. Interestingly, only a few people raised their hands when asked if they own a comma. I mainly went because I wanted to see the tinyboxes, and I wasn’t disappointed. Spent some time talking to the guy who designed them, super nice dude. Taking a page from Elon, it’s pretty clear tinyboxes are a way to fund their own datacenter (much like Starlink funds the mission to Mars). The chassis design has some smart bits, but obviously not even close to Dell Server level. Corners are cut, for example, it is lacking PSU redundancy. Is the premium worth it? Maybe. They did solve some tough problems: airflow management with an internal clear plastic bit, custom cable lengths, and packing 8 GPUs plus the rest into a 5U enclosure. None of that is trivial, but if you really wanted to, you could build it yourself. The way they package 5090s; they remove the fans and only have heat sinks, stack four horizontally, offset inside the chassis, and just push a ton of airflow through it. I imagine the top cards run a tad bit hotter than the bottom ones. The fans are likely working harder than needed. A vertical layout might’ve been a better idea, but who knows if they could make that work in the space. One interesting detail from their marketing: they can’t call these “servers.” If they did, NVIDIA or their upstream vendors would start asking more questions. So they’re “computers.” TL;DR: neat, very niche product. Clearly a lot of thought and effort went into it. If you want to support future comma work, this is a decent way to do it and get some nice hardware out of the deal. https://t.co/ICxhG3QRVA

@__tinygrad__ · 2025-11-09 22:26
tinybox review

@__tinygrad__ · 2025-11-09 22:29
@HotAisle The fans on the tinybox pro v2 are super overprovisioned. At full speed they'll keep the fully loaded GPUs below 70C.

@HotAisle · 2025-11-09 22:36
Confirmed what I gathered. "at full speed" and "super overprovisioned", is not good airflow design. You don't put more than 80% on your electrical systems, why would you want to do that to your fans? More fans is also more power and there is a speed / efficiency curve there when you're running them at max. To be fair, I hate the OAM / UBB design too. AMD having to use different SKU's for the front / rear GPUs is just sad. It is a big reason all this stuff is moving to DLC and wider racks, and I'm super glad about that.

@HotAisle · 2025-11-09 22:36
Confirmed what I gathered. "at full speed" and "super overprovisioned", is not good airflow design. You don't put more than 80% on your electrical systems, why would you want to do that to your fans? More fans is also more power and there is a speed / efficiency curve there when you're running them at max. To be fair, I hate the OAM / UBB design too. AMD having to use different SKU's for the front / rear GPUs is just sad. It is a big reason all this stuff is moving to DLC and wider racks, and I'm super glad about that.

@__tinygrad__ · 2025-11-09 22:39
@HotAisle They don't run at full speed by default, they run to keep the GPUs below 80C, usually ~50%. This is adjustable in BIOS.

@__tinygrad__ · 2025-11-09 22:49
Your AI budget determines if tiny is right for you: - Under $10k, buy a gaming PC. - Over $10M, buy B200/MI355X GPUs in Supermicro boxes. - Between $10k-$10M, buy tinyboxes. We are the product for the GPU middle class.

@__tinygrad__ · 2025-11-09 22:49
Your AI budget determines if tiny is right for you: - Under $10k, buy a gaming PC. - Over $10M, buy B200/MI355X GPUs in Supermicro boxes. - Between $10k-$10M, buy tinyboxes. We are the product for the GPU middle class.

@entropia_acc · 2025-11-09 22:52
@__tinygrad__ Why don't I just get one of these? https://t.co/VQ24NRvGzc

@entropia_acc · 2025-11-09 22:52
@__tinygrad__ Why don't I just get one of these? https://t.co/VQ24NRvGzc

@__tinygrad__ · 2025-11-09 22:53
@entropia_acc Why don't you get 4? https://t.co/pTkQ44z96o

@__tinygrad__ · 2025-11-09 22:49
Your AI budget determines if tiny is right for you: - Under $10k, buy a gaming PC. - Over $10M, buy B200/MI355X GPUs in Supermicro boxes. - Between $10k-$10M, buy tinyboxes. We are the product for the GPU middle class.

@AnsarUllahAnas_ · 2025-11-10 15:32
@__tinygrad__ The ‘GPU middle class’ is definitely underserved. How do tinyboxes compare in performance per dollar against cloud options for startups?

@AnsarUllahAnas_ · 2025-11-10 15:32
@__tinygrad__ The ‘GPU middle class’ is definitely underserved. How do tinyboxes compare in performance per dollar against cloud options for startups?

@__tinygrad__ · 2025-11-10 16:58
@AnsarUllahAnas_ Depends on if your startup is a pump and dump or not. If you are trying to pump and dump, use cloud. If you plan to be around for at least 5 years, buy machines.

@BigTechAlert · 2025-11-10 17:28
🆕 @jeffbezos has started following @__tinygrad__ https://t.co/ZhQZSVjOSZ

@__tinygrad__ · 2025-11-10 19:28
How is the Trainium software stack going?

@__tinygrad__ · 2025-11-10 19:28
How is the Trainium software stack going?

@_thomasip · 2025-11-10 19:48
@__tinygrad__ used in more places than the tiny stack

@_thomasip · 2025-11-10 19:48
@__tinygrad__ used in more places than the tiny stack

@__tinygrad__ · 2025-11-10 23:43
@_thomasip how sure are you about this? we're trying to get one and it's pending. tinygrad is just a git clone away and free https://t.co/Z4hysHW8TN

@giffmana · 2025-11-09 22:04
@vikhyatk Them: "this is held in RAM" Me, screaming internally: WHICH RAM?!

@jonas_eschmann · 2025-11-11 01:15
After achieving CUDA enlightenment, it seems crazy that we have been treating memory hierarchies (on CPUs) as black boxes all these years. It also feels like we don‘t have the right abstractions / language, yet, to losslessly (!) express/address different programming models/mem hierarchies. @__tinygrad__ , @clattner_llvm, @Modular, @apaszke et al. are iterating in this space and I really appreciate the fierce competition. I think in a few years we will have way better ways of mapping computation onto programming-models aka memory-hierarchies+synchronization groups

@__tinygrad__ · 2025-11-11 03:29
Agreed. We're hard at work on this problem.

@simran_s_arora · 2025-11-11 18:58
AI has been built on one vendor’s stack for too long. AMD’s GPUs now offer state-of-the-art peak compute and memory bandwidth — but the lack of mature software / the “CUDA moat” keeps that power locked away. Time to break it and ride into our multi-silicon future. 🌊 It's been a blast working with the amazing @_williamhu, @Drewwad and team; we present HipKittens!

@__tinygrad__ · 2025-11-11 23:23
HipKittens! These kittens will go great with tinykittens. We're working to reproduce their high speed results on our MI350X with code written in tinygrad UOps that implements the same data access patterns. Thank you for the release @HazyResearch

@DP_SMH · 2025-11-12 03:23
action continues tomorrow as MLPerf will reveal v5.1 training results. tinygrad had a very strange meeting on monday. while they are currently far behind on the contract, they laid out the most complete roadmap to date. good luck 🚀

@bendiken · 2025-11-12 06:50
Yes, followed the (underdocumented) steps. It seems to me what's happening is that TinyGPUDriver (while enabled and activated) isn't getting started when plugging in the TB5 cable, which could be due to macOS not seeing a matching device on the TB5 bus: based on `ioreg` etc, it only sees the Razer V2 enclosure, not anything contained inside it. Perhaps there's a significant diff to the previous enclosure in this regard. Needs more investigation

@geerlingguy · 2025-11-12 13:53
@bendiken @vanliessum @__tinygrad__ Definitely under documented. I was working on a full guide, but ran into issues last month. Need to try again.

@never_released · 2025-11-12 14:47
@__tinygrad__ random question: does the comma still use SNPE by now or did you fully migrate over to tinygrad on the device side for those?

@Tachyum · 2025-11-12 17:24
Tachyum Unveils 2nm Prodigy with 21x Higher AI Rack Performance than the Nvidia Rubin Ultra - read more here: https://t.co/lQTLs5h1Zi #AI #HPC #datecenters #CPU #GPU https://t.co/YvuGmFoGoJ

@never_released · 2025-11-12 14:47
@__tinygrad__ random question: does the comma still use SNPE by now or did you fully migrate over to tinygrad on the device side for those?

@__tinygrad__ · 2025-11-12 23:57
@never_released Fully tinygrad!

@__tinygrad__ · 2025-11-13 00:00
This seems too elaborate and high effort to be a joke, but it's obviously not a real chip. 2nm, 400 PetaFLOPS, DDR5-17600. The DDR5-17600 is the real giveaway that it's fake. Anyone know what's up?

@geerlingguy · 2025-11-12 13:53
@bendiken @vanliessum @__tinygrad__ Definitely under documented. I was working on a full guide, but ran into issues last month. Need to try again.

@__tinygrad__ · 2025-11-13 00:01
@geerlingguy @bendiken @vanliessum It's still very underdocumented, but a bunch of tiny employees have them and use them now, which wasn't true last month.

@__tinygrad__ · 2025-11-13 00:00
This seems too elaborate and high effort to be a joke, but it's obviously not a real chip. 2nm, 400 PetaFLOPS, DDR5-17600. The DDR5-17600 is the real giveaway that it's fake. Anyone know what's up?

@augeeidos · 2025-11-13 00:24
@__tinygrad__ VC money grab.

@__tinygrad__ · 2025-11-13 00:24
Nobody yet with 405B on AMD. We have a roadmap, and we hope to have it on the next MLPerf.

@augeeidos · 2025-11-13 00:24
@__tinygrad__ VC money grab.

@__tinygrad__ · 2025-11-13 00:26
@augeeidos I mean, DDR5-17600 is just fraud. DDR is a JEDEC standard, the highest perf in history of DDR5 is 13020, and that's using liquid nitrogen! https://t.co/5dqXchEUSa

@__tinygrad__ · 2025-11-13 00:00
This seems too elaborate and high effort to be a joke, but it's obviously not a real chip. 2nm, 400 PetaFLOPS, DDR5-17600. The DDR5-17600 is the real giveaway that it's fake. Anyone know what's up?

@vud124 · 2025-11-13 01:04
@__tinygrad__ Its been an entertaining scam quite a long time for people addicted to paper specs in processor world.

@__tinygrad__ · 2025-11-13 02:40
Needing "register scheduling" is why we end up writing an LLVM replacement in tinygrad and outputting the assembly directly. Anyone want to work on a RDNA3/CDNA4 backend? (we can test RDNA3 in CI)

@__tinygrad__ · 2025-11-13 02:40
Needing "register scheduling" is why we end up writing an LLVM replacement in tinygrad and outputting the assembly directly. Anyone want to work on a RDNA3/CDNA4 backend? (we can test RDNA3 in CI)

@__tinygrad__ · 2025-11-13 02:40
A great deep dive into CDNA4 here. https://t.co/npw7mIcur4

@__tinygrad__ · 2025-11-13 03:05
.@AnushElangovan Can you open source rocprof-trace-decoder? We're hitting issues in the Mac version. We have decent SQTT support now, but it's annoying to debug a black box. If it can't be open sourced we will probably end up reverse engineering the format. https://t.co/oUFte6Df6U

@__tinygrad__ · 2025-11-13 02:40
A great deep dive into CDNA4 here. https://t.co/npw7mIcur4

@__tinygrad__ · 2025-11-13 03:07
Things like the undocumented bank access difference between ds_read_b128 and ds_write_b64 are crazy, @AMD should document this stuff in the CDNA4 manual ASAP!

@__tinygrad__ · 2025-11-13 02:40
Needing "register scheduling" is why we end up writing an LLVM replacement in tinygrad and outputting the assembly directly. Anyone want to work on a RDNA3/CDNA4 backend? (we can test RDNA3 in CI)

@luigifcruz · 2025-11-13 03:14
@__tinygrad__ Any plans for Intel Arc support?

@luigifcruz · 2025-11-13 03:14
@__tinygrad__ Any plans for Intel Arc support?

@__tinygrad__ · 2025-11-13 03:16
@luigifcruz It works through OpenCL. We're bearish on putting any Intel specific effort in cause Intel will cancel whatever product or API we do put effort into. The company needs direction and leadership.

@vud124 · 2025-11-13 01:04
@__tinygrad__ Its been an entertaining scam quite a long time for people addicted to paper specs in processor world.

@__tinygrad__ · 2025-11-13 03:18
@vud124 This is proof you can just write a press release and the media will repeat it uncritically. https://t.co/rZ0u4l32hJ

@__tinygrad__ · 2025-11-13 03:05
.@AnushElangovan Can you open source rocprof-trace-decoder? We're hitting issues in the Mac version. We have decent SQTT support now, but it's annoying to debug a black box. If it can't be open sourced we will probably end up reverse engineering the format. https://t.co/oUFte6Df6U

@AnushElangovan · 2025-11-13 07:15
@__tinygrad__ I will go poke whatever it was stuck in last time. Meanwhile is there a place to find gemm_profile.pkl file ? I want to fix that issue too.

@Adev · 2025-11-13 08:05
@FelixCLC_ What you mean ? Finally an unification of this mess around something manageable ?

@Adev · 2025-11-13 08:06
@FelixCLC_ Honestly, Accelerator support is currently one of the main motivation that makes me currently do my own programming language. The accelerator SDK landscape is a sad mess to be in.

@Adev · 2025-11-13 08:05
@FelixCLC_ What you mean ? Finally an unification of this mess around something manageable ?

@AnushElangovan · 2025-11-13 07:15
@__tinygrad__ I will go poke whatever it was stuck in last time. Meanwhile is there a place to find gemm_profile.pkl file ? I want to fix that issue too.

@__tinygrad__ · 2025-11-13 16:26
@AnushElangovan Issue being tracked here, pkl file in that commit. Knowing the tiny team, if it's closed source, I give us about 2 weeks before we start writing our own. https://t.co/KlWeNMJNIx

@__tinygrad__ · 2025-11-13 16:29
AI has made the project maintainer experience worse. We're getting more and more PRs written by people who don't know what they are doing, but it takes longer to sort them out cause the code "looks" correct. This sadly is going to end up with a stronger emphasis on identity.

@__tinygrad__ · 2025-11-13 16:29
AI has made the project maintainer experience worse. We're getting more and more PRs written by people who don't know what they are doing, but it takes longer to sort them out cause the code "looks" correct. This sadly is going to end up with a stronger emphasis on identity.

@HotAisle · 2025-11-13 16:30
@__tinygrad__ Unit tests usually expose weakness, so make that a requirement.

@HotAisle · 2025-11-13 16:30
@__tinygrad__ Unit tests usually expose weakness, so make that a requirement.

@__tinygrad__ · 2025-11-13 16:32
@HotAisle It is. In general, it's people adding new features and also sometimes new tests. The PRs that require deep understanding of the codebase are never submitted by these people, but they also aren't submitted by new people in general.

@__tinygrad__ · 2025-11-13 16:29
AI has made the project maintainer experience worse. We're getting more and more PRs written by people who don't know what they are doing, but it takes longer to sort them out cause the code "looks" correct. This sadly is going to end up with a stronger emphasis on identity.

@randallmbriggs · 2025-11-13 16:41
@__tinygrad__ Can people still contribute anonymously as long as they prove themselves over time? Or do you think it can no longer be anonymous at all?

@randallmbriggs · 2025-11-13 16:41
@__tinygrad__ Can people still contribute anonymously as long as they prove themselves over time? Or do you think it can no longer be anonymous at all?

@__tinygrad__ · 2025-11-13 17:02
@randallmbriggs You can be anonymous. Just make sure to polish your PR until it shines before submitting.

@__tinygrad__ · 2025-11-13 16:29
AI has made the project maintainer experience worse. We're getting more and more PRs written by people who don't know what they are doing, but it takes longer to sort them out cause the code "looks" correct. This sadly is going to end up with a stronger emphasis on identity.

@yacineMTB · 2025-11-13 17:06
@__tinygrad__ the limit on good software is understanding. people who use ai tools well use it to help themselves understand, writing experiments and testing things out. the amount of code i write has 10x'd but so has the amount of code i throw away. no one wants to think

@yacineMTB · 2025-11-13 17:06
@__tinygrad__ the limit on good software is understanding. people who use ai tools well use it to help themselves understand, writing experiments and testing things out. the amount of code i write has 10x'd but so has the amount of code i throw away. no one wants to think

@__tinygrad__ · 2025-11-13 17:15
@yacineMTB If you use AI to help you understand and I can't tell, more power to you. If you copy and paste crap you don't understand into a PR, I will be upset.

@__tinygrad__ · 2025-11-13 16:29
AI has made the project maintainer experience worse. We're getting more and more PRs written by people who don't know what they are doing, but it takes longer to sort them out cause the code "looks" correct. This sadly is going to end up with a stronger emphasis on identity.

@outcomethinking · 2025-11-13 17:17
@__tinygrad__ Only given the assumption that the coding ai doesn’t improve. Given its trajectory - I wouldn’t make that bet.

@outcomethinking · 2025-11-13 17:17
@__tinygrad__ Only given the assumption that the coding ai doesn’t improve. Given its trajectory - I wouldn’t make that bet.

@__tinygrad__ · 2025-11-13 17:20
@outcomethinking If the AI improves, we still won't need the type of person who copy and pastes things they don't understand for real humans to waste time reading.

@__tinygrad__ · 2025-11-13 17:31
It's not sad with tinygrad. We have full open source end to end support for AMD and NVIDIA, no vendor drivers and no vendor libraries. If you are building an accelerator today, you'd be crazy to use anything but tinygrad.

@__tinygrad__ · 2025-11-13 17:31
It's not sad with tinygrad. We have full open source end to end support for AMD and NVIDIA, no vendor drivers and no vendor libraries. If you are building an accelerator today, you'd be crazy to use anything but tinygrad.

@wolftivy · 2025-11-13 17:33
@__tinygrad__ Is tinygrad always going to be python exclusive? Would be great to be able to efficiently invoke it from other languages.

@wolftivy · 2025-11-13 17:33
@__tinygrad__ Is tinygrad always going to be python exclusive? Would be great to be able to efficiently invoke it from other languages.

@__tinygrad__ · 2025-11-13 17:34
@wolftivy tinygrad is a compiler. The compiler itself will remain Python, but if you want to invoke it from other languages, it can output any model (including training loops) as 100% pure ANSI C.

@wolftivy · 2025-11-13 17:35
@__tinygrad__ Oh very cool. I haven't seen this feature is that live now?

@__tinygrad__ · 2025-11-13 17:44
@wolftivy Been live for a while. We really communicate poorly what tinygrad can do. CPU=1 python3 examples/compile_efficientnet.py | clang -O2 -lm -x c - -o recognize && time DEBUG=1 ./recognize docs/showcase/stable_diffusion_by_tinygrad.jpg

@__tinygrad__ · 2025-11-13 17:44
@wolftivy Been live for a while. We really communicate poorly what tinygrad can do. CPU=1 python3 examples/compile_efficientnet.py | clang -O2 -lm -x c - -o recognize && time DEBUG=1 ./recognize docs/showcase/stable_diffusion_by_tinygrad.jpg

@__tinygrad__ · 2025-11-13 17:48
@wolftivy Here's the file without the inline weights and JPEG library. https://t.co/Mmei2GK7hZ

@__tinygrad__ · 2025-11-13 17:52
Some of the AI problem is due to our poor pull request management. tinygrad went through a lot of refactors this year, but things are starting to stabilize, and we'll put more effort into communicating and making it easier for external contributors. Only 35 open PRs now! https://t.co/AxDm5Jy2SK

@__tinygrad__ · 2025-11-13 17:52
Some of the AI problem is due to our poor pull request management. tinygrad went through a lot of refactors this year, but things are starting to stabilize, and we'll put more effort into communicating and making it easier for external contributors. Only 35 open PRs now! https://t.co/AxDm5Jy2SK

@__tinygrad__ · 2025-11-13 17:54
If you have taste, some great PRs would be moving methods from tensor py into mixin, a lot of low hanging fruit to move things to math. Remember, if it's in mixin it must work on both the Tensor and the UOp. mixin will also be the first folder we `ruff format`

@__tinygrad__ · 2025-11-13 17:20
@outcomethinking If the AI improves, we still won't need the type of person who copy and pastes things they don't understand for real humans to waste time reading.

@GrigoryEvko · 2025-11-13 18:41
@__tinygrad__ @outcomethinking Dude maybe spend this productive time on sales and maintaining support@tinygrad.org monitoring? On our inquiry: it'd be okay if you send a rack without a motherboard, just GPUs with mcio connectors there if you don't customize it anyhow for 35-37 grand

@GrigoryEvko · 2025-11-13 18:41
@__tinygrad__ @outcomethinking Dude maybe spend this productive time on sales and maintaining support@tinygrad.org monitoring? On our inquiry: it'd be okay if you send a rack without a motherboard, just GPUs with mcio connectors there if you don't customize it anyhow for 35-37 grand

@__tinygrad__ · 2025-11-13 18:47
@GrigoryEvko @outcomethinking We do not provide any customization or special requests and are very clear about this on our website. We do this to keep prices low and quality high. If you absolutely require something custom, buy from someone else.

@__tinygrad__ · 2025-11-13 18:47
@GrigoryEvko @outcomethinking We do not provide any customization or special requests and are very clear about this on our website. We do this to keep prices low and quality high. If you absolutely require something custom, buy from someone else.

@GrigoryEvko · 2025-11-13 18:49
@__tinygrad__ @outcomethinking You can't even detach connectors and unscrew the motherboard? Get real

@GrigoryEvko · 2025-11-13 18:49
@__tinygrad__ @outcomethinking You can't even detach connectors and unscrew the motherboard? Get real

@__tinygrad__ · 2025-11-13 18:51
@GrigoryEvko @outcomethinking No, as stated 3 times in our FAQ, we provide 0 customization. https://t.co/fX6pID69iQ

@__tinygrad__ · 2025-11-13 17:31
It's not sad with tinygrad. We have full open source end to end support for AMD and NVIDIA, no vendor drivers and no vendor libraries. If you are building an accelerator today, you'd be crazy to use anything but tinygrad.

@dhbrojas · 2025-11-13 20:26
@__tinygrad__ Are there plans for @tenstorrent support at some point? It’s the only thing holding me back from buying one of their cards.

@dhbrojas · 2025-11-13 20:26
@__tinygrad__ Are there plans for @tenstorrent support at some point? It’s the only thing holding me back from buying one of their cards.

@__tinygrad__ · 2025-11-13 20:53
@dhbrojas @tenstorrent Not from us, maybe from an external contributor. We're open to a contract, but tenstorrent seems set on doing it themselves.

@__tinygrad__ · 2025-11-13 20:53
@dhbrojas @tenstorrent Not from us, maybe from an external contributor. We're open to a contract, but tenstorrent seems set on doing it themselves.

@dhbrojas · 2025-11-13 21:55
@__tinygrad__ @tenstorrent Shame they don’t want a contract. The TT software stack is something out of a horror movie. I would love to work on this but sadly don’t have the time.

@__tinygrad__ · 2025-11-13 20:53
@dhbrojas @tenstorrent Not from us, maybe from an external contributor. We're open to a contract, but tenstorrent seems set on doing it themselves.

@__tinygrad__ · 2025-11-13 22:11
tinybox red v2! 4x9070XT, single 15A plug. What price are people buying at? https://t.co/U9CZhZz3Ai

@__tinygrad__ · 2025-11-13 22:11
tinybox red v2! 4x9070XT, single 15A plug. What price are people buying at? https://t.co/U9CZhZz3Ai

@nwolovick · 2025-11-13 22:30
@__tinygrad__ 2W idling? That is really low, almost a Z80.

@__tinygrad__ · 2025-11-13 22:11
tinybox red v2! 4x9070XT, single 15A plug. What price are people buying at? https://t.co/U9CZhZz3Ai

@testttt1236 · 2025-11-13 22:54
@__tinygrad__ What's the bottleneck for fitting 6 like the v1?

@testttt1236 · 2025-11-13 22:54
@__tinygrad__ What's the bottleneck for fitting 6 like the v1?

@__tinygrad__ · 2025-11-13 22:57
@testttt1236 Power. This is a 1 PSU machine, makes it a lot easier to plug in.

@__tinygrad__ · 2025-11-13 22:56
AMD has made a ton of progress with ROCm. Version 7.1 comes standard, along with the latest PyTorch. 133 achieved GEMM TFLOPS out of 165 MMAPEAK isn't bad, and the amdgpu driver seems stable on RDNA4! https://t.co/z7GvjoV2Ir

@tobiawolaju · 2025-11-13 22:57
@__tinygrad__ Solid numbers. Curious if you’ve run into any issues with ROCm on RDNA4 yet?

@tobiawolaju · 2025-11-13 22:57
@__tinygrad__ Solid numbers. Curious if you’ve run into any issues with ROCm on RDNA4 yet?

@__tinygrad__ · 2025-11-13 23:00
@tobiawolaju Nope, it's tons better than RDNA3. It's a little rough around the edges, but mainstream stuff works.

@__tinygrad__ · 2025-11-13 23:19
Here it is with amdgpu instead of AM driver. idk what it's on about about low power state, that's max power, and look at those frigid temps after burning for 10 minutes! https://t.co/hCHqb9rRXL

@__tinygrad__ · 2025-11-13 23:21
Looks like a standard issue tinybox, beautiful front screen, big clicky button, calming circles. https://t.co/KRVa9p6FOr

@__tinygrad__ · 2025-11-13 23:21
Looks like a standard issue tinybox, beautiful front screen, big clicky button, calming circles. https://t.co/KRVa9p6FOr

@__tinygrad__ · 2025-11-13 23:25
Shipping today, we have 4 in stock! Yours for $10k. https://t.co/RTJBUIW95Y

@nwolovick · 2025-11-13 22:30
@__tinygrad__ 2W idling? That is really low, almost a Z80.

@__tinygrad__ · 2025-11-13 23:28
@nwolovick AMD crushed idle power on RDNA4. Here's CDNA4 idling 😂 https://t.co/SNLX7VerA2

@__tinygrad__ · 2025-11-13 23:25
Shipping today, we have 4 in stock! Yours for $10k. https://t.co/RTJBUIW95Y

@agustin_meriles · 2025-11-13 23:31
@__tinygrad__ Can this run Crysis? 😛

@__tinygrad__ · 2025-11-13 23:25
Shipping today, we have 4 in stock! Yours for $10k. https://t.co/RTJBUIW95Y

@ferretlabs_re · 2025-11-13 23:33
@__tinygrad__ Tempted even though I don’t care about Llms… so much power per buck.

@__tinygrad__ · 2025-11-13 22:56
AMD has made a ton of progress with ROCm. Version 7.1 comes standard, along with the latest PyTorch. 133 achieved GEMM TFLOPS out of 165 MMAPEAK isn't bad, and the amdgpu driver seems stable on RDNA4! https://t.co/z7GvjoV2Ir

@SemiAnalysis_ · 2025-11-13 23:38
@__tinygrad__ DeepSeek V3 GEMM shapes for training, prefill, and memory-bound decode, and see how your beam search codegen performs versus TheRock, native PyTorch, HIPBLASLt, and torch.matmul on real-world GEMM shapes? The square shapes you benchmarked above don’t seem that realistic.

@SemiAnalysis_ · 2025-11-13 23:38
@__tinygrad__ DeepSeek V3 GEMM shapes for training, prefill, and memory-bound decode, and see how your beam search codegen performs versus TheRock, native PyTorch, HIPBLASLt, and torch.matmul on real-world GEMM shapes? The square shapes you benchmarked above don’t seem that realistic.

@__tinygrad__ · 2025-11-13 23:40
@SemiAnalysis_ You want SSH to a tinybox red v2? You can test it however you want.

@__tinygrad__ · 2025-11-13 23:41
The v2 lineup is complete. https://t.co/jUB6p9aw37

@ferretlabs_re · 2025-11-13 23:33
@__tinygrad__ Tempted even though I don’t care about Llms… so much power per buck.

@__tinygrad__ · 2025-11-13 23:45
@ferretlabs_re Yea, AMD is ~30% cheaper than NVIDIA. It's a solid machine.

@agustin_meriles · 2025-11-13 23:31
@__tinygrad__ Can this run Crysis? 😛

@__tinygrad__ · 2025-11-13 23:56
@agustin_meriles No, if you install windows on a tinybox it will cry. If you log in with your microsoft account it will be in too much pain to keep going and will only run solitaire and skifree.

@__tinygrad__ · 2025-11-14 00:22
Hey @tenstorrent, when you are ready, there's tinygrad

@__tinygrad__ · 2025-11-14 00:22
Hey @tenstorrent, when you are ready, there's tinygrad

@__tinygrad__ · 2025-11-14 00:23
@tenstorrent In so many ways, they get the hardware right. I love that they sell the cards. And @corsix has been making good progress on documentation. But the current software stack is beyond unusable, like worse than AMD was when we started tiny corp.

@__tinygrad__ · 2025-11-13 23:40
@SemiAnalysis_ You want SSH to a tinybox red v2? You can test it however you want.

@SemiAnalysis_ · 2025-11-14 00:26
@__tinygrad__ Team is heads down daily, testing and benchmarking 10+ AI chip SKUs from 4+ vendors. Should we put a bounty up for getting TinyGrad beam search, DeepSeek grouped GEMM prefill, and decode to match CK perf instead? $50?

@SemiAnalysis_ · 2025-11-14 00:26
@__tinygrad__ Team is heads down daily, testing and benchmarking 10+ AI chip SKUs from 4+ vendors. Should we put a bounty up for getting TinyGrad beam search, DeepSeek grouped GEMM prefill, and decode to match CK perf instead? $50?

@__tinygrad__ · 2025-11-14 00:29
@SemiAnalysis_ A lot of the AMD stuff is bleak on the perf high end still. The LLVM port isn't that good, no amount of BEAM fixes that. HipKittens is the best on the maintainable AMD perf front.

@__tinygrad__ · 2025-11-14 00:27
@tenstorrent @corsix The abstractions in tinygrad are decent now. Do what we did with AMD and NVIDIA, start from the PCIe driver, there's already good small abstractions to help you. Build an IR on UOps. You could have a working port in a month. 0 install needed besides pip install tinygrad.

@__tinygrad__ · 2025-11-14 00:32
@tenstorrent @corsix Don't feel the urge to just pull in one piece. The crazy hard to link C++20 stuff needs to go. And at that point, why use the driver you have? You can build a whole stack that runs Qwen fast in 1000 lines of Python on top of tinygrad. Nothing else, just hardware and tiny.

@__tinygrad__ · 2025-11-14 00:32
@tenstorrent @corsix Don't feel the urge to just pull in one piece. The crazy hard to link C++20 stuff needs to go. And at that point, why use the driver you have? You can build a whole stack that runs Qwen fast in 1000 lines of Python on top of tinygrad. Nothing else, just hardware and tiny.

@__tinygrad__ · 2025-11-14 00:36
@tenstorrent @corsix If you use our PCIe abstractions, it will also work with MacBooks over USB4. You know that's the experience every dev wants. MacBook + one 0 dep pip install, fully controllable from Python. Development velocity will skyrocket. @jimkxa

@__tinygrad__ · 2025-11-13 23:41
The v2 lineup is complete. https://t.co/jUB6p9aw37

@sunkencity999 · 2025-11-14 00:51
@__tinygrad__ Beautiful. Is there any particular reason that you aren't utilizing/offering a config with the RTX6000 pro 96GB? One stop - shopping and the ability to serve near SOTA un-quantized models internally would be a great sell for companies like mine.

@sunkencity999 · 2025-11-14 00:51
@__tinygrad__ Beautiful. Is there any particular reason that you aren't utilizing/offering a config with the RTX6000 pro 96GB? One stop - shopping and the ability to serve near SOTA un-quantized models internally would be a great sell for companies like mine.

@__tinygrad__ · 2025-11-14 00:53
@sunkencity999 Ahh, the secret off menu item. https://t.co/pTkQ44z96o

@__tinygrad__ · 2025-11-14 00:23
@tenstorrent In so many ways, they get the hardware right. I love that they sell the cards. And @corsix has been making good progress on documentation. But the current software stack is beyond unusable, like worse than AMD was when we started tiny corp.

@corsix · 2025-11-14 10:12
@__tinygrad__ @tenstorrent I appreciate visibility on documentation, but I should say that I’ve moved to pastures anew.

@gamestoneai · 2025-11-14 00:47
@__tinygrad__ @tenstorrent @corsix TT gets hardware right. Software's a dumpster fire. Tinygrad's clean approach could save them months of pain.

@dhbrojas · 2025-11-14 10:22
@gamestoneai @__tinygrad__ @tenstorrent @corsix I remember Jim saying they have 500 people working on software. They seem to be investing heavily into the compiler, which is a good thing but it's not happening at the pace/quality needed to get developers.

@dhbrojas · 2025-11-14 10:22
@gamestoneai @__tinygrad__ @tenstorrent @corsix I remember Jim saying they have 500 people working on software. They seem to be investing heavily into the compiler, which is a good thing but it's not happening at the pace/quality needed to get developers.

@__tinygrad__ · 2025-11-14 18:16
@dhbrojas @gamestoneai @tenstorrent @corsix Brooks's Law is an observation in software project management stating that adding more people to a project that is already late will make it even later.

@corsix · 2025-11-14 10:12
@__tinygrad__ @tenstorrent I appreciate visibility on documentation, but I should say that I’ve moved to pastures anew.

@__tinygrad__ · 2025-11-14 18:18
@corsix @tenstorrent Ahh, fair. Hopefully someone at @tenstorrent is picking up the torch, I'm still shocked that they did a tapeout without documentation!

@__tinygrad__ · 2025-11-14 18:31
@theskilledcoder This is a top tier troll. Is this secretly an ad for Python? At tiny, we reveal the truth. Python: ~114 s https://t.co/uVSSV5u9js

@530RB · 2025-11-14 18:39
@__tinygrad__ @theskilledcoder Literally done by no one

@530RB · 2025-11-14 18:39
@__tinygrad__ @theskilledcoder Literally done by no one

@__tinygrad__ · 2025-11-14 18:44
@530RB @theskilledcoder In any language

@__tinygrad__ · 2025-11-13 23:41
The v2 lineup is complete. https://t.co/jUB6p9aw37

@david0x964B00 · 2025-11-14 20:58
@__tinygrad__ I see you are using AMD CPUs in all of them. Do any of Intel’s offerings come close?

@jsuarez · 2025-11-14 21:13
Me: I ported Muon to cpp! Time to PR to torch! Torch: Cool, give me an hour to compile every possible flash attention kernel https://t.co/ukQiyk0zZM

@__tinygrad__ · 2025-11-14 22:52
RT @ludwigABAP: please oh god that would catapult tenstorrent so far forward imo

@jjjwaynejjj · 2025-11-14 22:10
@jsuarez I guess that's why tinygrad will win

@DanAdvantage · 2025-11-14 23:15
@jjjwaynejjj @jsuarez @__tinygrad__ knows what's up, that's for sure

@DanAdvantage · 2025-11-14 23:15
@jjjwaynejjj @jsuarez @__tinygrad__ knows what's up, that's for sure

@__tinygrad__ · 2025-11-14 23:29
@DanAdvantage @jjjwaynejjj @jsuarez Why would you compile a kernel you don't run?

@ludwigABAP · 2025-11-14 22:44
please oh god that would catapult tenstorrent so far forward imo

@neomewtwo · 2025-11-15 00:03
@ludwigABAP and they are never going to do it

@neomewtwo · 2025-11-15 00:03
@ludwigABAP and they are never going to do it

@__tinygrad__ · 2025-11-15 01:14
@neomewtwo @ludwigABAP No response to https://t.co/CHnzYxTegt, either they don't agree that they have terrible software, or they have already pivoted away from AI cards but why would they tell us then nobody would buy cards?

@__tinygrad__ · 2025-11-15 01:14
@neomewtwo @ludwigABAP No response to https://t.co/CHnzYxTegt, either they don't agree that they have terrible software, or they have already pivoted away from AI cards but why would they tell us then nobody would buy cards?

@neomewtwo · 2025-11-15 01:23
@__tinygrad__ @ludwigABAP maybe it's the intel genetics

@neomewtwo · 2025-11-15 01:23
@__tinygrad__ @ludwigABAP maybe it's the intel genetics

@__tinygrad__ · 2025-11-15 01:27
@neomewtwo @ludwigABAP .@jimkxa is a good leader, I just fear TT has pivoted and hasn't told us. Read more about Intel's problems here https://t.co/9FcM5tm8v1

@__tinygrad__ · 2025-11-15 03:51
Did you know tinygrad supports vmap? In many ways, our way of doing it is cleaner than JAX, though the syntax isn't the best. Someone want to clean up the syntax and write a tutorial? https://t.co/nauLkFGqxk

@__tinygrad__ · 2025-11-15 03:51
Did you know tinygrad supports vmap? In many ways, our way of doing it is cleaner than JAX, though the syntax isn't the best. Someone want to clean up the syntax and write a tutorial? https://t.co/nauLkFGqxk

@marcal_cc · 2025-11-15 03:58
@__tinygrad__ in what ways is it cleaner than JAX? is it faster?

@marcal_cc · 2025-11-15 03:58
@__tinygrad__ in what ways is it cleaner than JAX? is it faster?

@__tinygrad__ · 2025-11-15 04:00
@marcal_cc JAX operates on functions and forces you to specify axis numbers. tinygrad lets you construct a range, slice on it, and put it back wherever you want.

@riomadeit · 2025-11-16 18:07
Just got a new Windows laptop, what should I uninstall? https://t.co/jh4I1ExYfP

@TechCrunch · 2025-11-17 14:07
Luminal raises $5.3 million to build a better GPU code framework https://t.co/wsFoYNLXYc

@joefioti · 2025-11-17 15:21
Luminal has raised $5.3M to solve inference compilers.

@__tinygrad__ · 2025-11-17 16:49
RT @a19grey: Just got a TinyBox V2 what should i train? @__tinygrad__ https://t.co/oR42Zb67ps

@__tinygrad__ · 2025-11-17 16:49
RT @mov_axbx: Tinygrad's C backend is really cool for some projects. You could modify it to use CPU-specific intrinsics and it just emits code you can compile. I could have easily done my LLM-on-Dreamcast demo, or run an LLM on the SGI Indigo with it instead of porting llama2.c

@stuart_sul · 2025-11-17 18:09
(1/6) GPU networking is the remaining AI efficiency bottleneck, and the underlying hardware is changing fast! We’re happy to release ParallelKittens, an update to ThunderKittens that lets you easily write fast computation-communication overlapped multi-GPU kernels, along with new kernels for data, tensor, sequence, and expert parallelism! Here’s a photo of overlapped kittens, along with things you should care about when optimizing multi-GPU kernels. (With @simran_s_arora, @spectorb, and @hazyresearch. Generously supported by @cursor_ai and @togethercompute)

@__tinygrad__ · 2025-11-17 18:28
This team deeply understands GPUs, their papers are some of the clearest descriptions of what a GPU really is. We are working to implement their ideas in tinygrad with wide hardware support and good regression tests.

@__tinygrad__ · 2025-11-17 21:58
@joefioti Reasonable size raise, congrats! I'm again suggesting that you don't rewrite the frontend and translate from tinygrad's UOps, you get the Python frontend everyone wants and autograd+torch+onnx support for free. Then drop into your (full model) compiler.

@joefioti · 2025-11-17 22:00
@__tinygrad__ thanks george! is there a stable set of uops you guys have settled on yet? i can take a look.

@joefioti · 2025-11-17 22:00
@__tinygrad__ thanks george! is there a stable set of uops you guys have settled on yet? i can take a look.

@__tinygrad__ · 2025-11-17 22:04
@joefioti Yea, the frontend UOps have been pretty stable for years. The 6 movement + ~20 elementwise + REDUCE_AXIS. Try a few ONNX models and display the graph of the output Tensor.uop, we break down all kinds of gemms and convolutions so you don't have to.

@__tinygrad__ · 2025-11-13 23:41
The v2 lineup is complete. https://t.co/jUB6p9aw37

@sweinoid · 2025-11-18 03:49
@__tinygrad__ 9700pro 32GB woulda been nicer for red. Nice lineup!

@sweinoid · 2025-11-18 03:49
@__tinygrad__ 9700pro 32GB woulda been nicer for red. Nice lineup!

@__tinygrad__ · 2025-11-18 16:47
@sweinoid Would you buy it? Or is this "oh it's nicer if I don't have to pay the much higher price for it"?

@david0x964B00 · 2025-11-14 20:58
@__tinygrad__ I see you are using AMD CPUs in all of them. Do any of Intel’s offerings come close?

@__tinygrad__ · 2025-11-18 16:47
@david0x964B00 No.

@__tinygrad__ · 2025-11-18 23:13
We have 3 tinybox green v2 in stock, then that'll be it until 5090 and RAM prices return to sanity. 4 tinybox red v2 in stock, blackwell built to order (1 sold!)

@__tinygrad__ · 2025-11-18 23:13
We have 3 tinybox green v2 in stock, then that'll be it until 5090 and RAM prices return to sanity. 4 tinybox red v2 in stock, blackwell built to order (1 sold!)

@theodorvaryag · 2025-11-18 23:27
@__tinygrad__ I'm probably in the minority but I want a high single threaded perf threadripper tinybox w/ 5090s I'd rather not use a separate computer in addition to it and just work on it directly but my work is very...latency sensitive and not always parallelizable, so I need ST perf.

@__tinygrad__ · 2025-11-19 00:09
New $500 bounty for people good at optimizing Python code. `time NULL=1 python3 examples/stable_diffusion.py --fakeweights` under 10 seconds on M3 Max. You don't need weights or a GPU to run that. Tons of low hanging fruit to optimize.

@theodorvaryag · 2025-11-18 23:27
@__tinygrad__ I'm probably in the minority but I want a high single threaded perf threadripper tinybox w/ 5090s I'd rather not use a separate computer in addition to it and just work on it directly but my work is very...latency sensitive and not always parallelizable, so I need ST perf.

@__tinygrad__ · 2025-11-19 00:11
@theodorvaryag You probably want a gaming PC.

@__tinygrad__ · 2025-11-19 00:09
New $500 bounty for people good at optimizing Python code. `time NULL=1 python3 examples/stable_diffusion.py --fakeweights` under 10 seconds on M3 Max. You don't need weights or a GPU to run that. Tons of low hanging fruit to optimize.

@jashan702 · 2025-11-19 00:44
@__tinygrad__ How can I benchmark that I don't have an m3 max

@jashan702 · 2025-11-19 00:44
@__tinygrad__ How can I benchmark that I don't have an m3 max

@__tinygrad__ · 2025-11-19 00:47
@jashan702 It shouldn't vary much.

@__tinygrad__ · 2025-11-19 00:09
New $500 bounty for people good at optimizing Python code. `time NULL=1 python3 examples/stable_diffusion.py --fakeweights` under 10 seconds on M3 Max. You don't need weights or a GPU to run that. Tons of low hanging fruit to optimize.

@__tinygrad__ · 2025-11-19 01:30
Obviously the point of this isn't the stable diffusion example, it's optimizing tinygrad itself. Here's the current run with VIZ=1. https://t.co/K8tHYddIBJ

@__tinygrad__ · 2025-11-19 21:57
RT @comma_ai: New paper just dropped! https://t.co/rgXlZ81XJ3 / tinycorp NeurIPS event Wednesday, December 3rd, 2025. Strictly Invite only. Details in the paper! https://t.co/UtpnBX3b5K

@__tinygrad__ · 2025-11-19 01:30
Obviously the point of this isn't the stable diffusion example, it's optimizing tinygrad itself. Here's the current run with VIZ=1. https://t.co/K8tHYddIBJ

@__tinygrad__ · 2025-11-19 23:47
Cut 30% this morning with a refactor and a tasteful cache on the div_mod_folding function. There's another 30% to gain by switching the order of devectorize and lower_index_dtype. https://t.co/sstLB537U1

@__tinygrad__ · 2025-11-19 01:30
Obviously the point of this isn't the stable diffusion example, it's optimizing tinygrad itself. Here's the current run with VIZ=1. https://t.co/K8tHYddIBJ

@__tinygrad__ · 2025-05-09 21:52
Here's the worlds first AMD GPU driven over USB3. From a Mac! Linux and Windows should work too, it's just libusb. Available today in tinygrad master, use an ADT-UT3G to connect the GPU to your USB port. You have no idea of the level of engineering that went into this. https://t.co/V6trNwcGXt

@hi_yoniyang · 2025-11-20 13:57
@__tinygrad__ Wondering if it works on ..... Pixel 7a with USB 3.2 Gen 2. That's going to be a fun project if it works lol

@allen_ai · 2025-11-20 14:04
Announcing Olmo 3, a leading fully open LM suite built for reasoning, chat, & tool use, and an open model flow—not just the final weights, but the entire training journey. Best fully open 32B reasoning model & best 32B base model. 🧵 https://t.co/vnGrArA44X

@__tinygrad__ · 2025-11-20 17:14
Thanks to a bounty claimed by roelofvandijk our torch frontend is working again! This lets you run torch code with a tinygrad backend, enabling support for OpenCL and USB GPUs by using torch device "tiny"

@hi_yoniyang · 2025-11-20 13:57
@__tinygrad__ Wondering if it works on ..... Pixel 7a with USB 3.2 Gen 2. That's going to be a fun project if it works lol

@__tinygrad__ · 2025-11-20 17:16
@hi_yoniyang The AMD USB3 stuff should! This is similar to how we use it on the comma four.

@__tinygrad__ · 2025-11-20 17:24
@blackholenixnix lol George is the one who broke it

@__tinygrad__ · 2025-11-20 17:30
We've actually sold two of the 4x Blackwell boxes! Thanks to @SIGKITTEN for bringing the price back down to earth. It's not going to outperform 8x5090s, but the pro box is loud and requires 200+ V. The $50k Blackwell box is quiet and plugs into 2 standard US outlets. https://t.co/pB7ghTfnHY

@__tinygrad__ · 2025-11-20 17:30
We've actually sold two of the 4x Blackwell boxes! Thanks to @SIGKITTEN for bringing the price back down to earth. It's not going to outperform 8x5090s, but the pro box is loud and requires 200+ V. The $50k Blackwell box is quiet and plugs into 2 standard US outlets. https://t.co/pB7ghTfnHY

@__tinygrad__ · 2025-11-20 17:31
If you want one, buy it here. We also have 2 greens and 3 reds in stock ready to ship tomorrow. https://t.co/pTkQ44z96o

@__tinygrad__ · 2025-11-20 17:30
We've actually sold two of the 4x Blackwell boxes! Thanks to @SIGKITTEN for bringing the price back down to earth. It's not going to outperform 8x5090s, but the pro box is loud and requires 200+ V. The $50k Blackwell box is quiet and plugs into 2 standard US outlets. https://t.co/pB7ghTfnHY

@itsgrantpal · 2025-11-20 17:33
@__tinygrad__ @SIGKITTEN @SIGKITTEN deserves a raise, praise and a bag of lays

@__tinygrad__ · 2025-11-20 17:30
We've actually sold two of the 4x Blackwell boxes! Thanks to @SIGKITTEN for bringing the price back down to earth. It's not going to outperform 8x5090s, but the pro box is loud and requires 200+ V. The $50k Blackwell box is quiet and plugs into 2 standard US outlets. https://t.co/pB7ghTfnHY

@SIGKITTEN · 2025-11-20 17:37
@__tinygrad__ its such a great top-shelf beast! my fav part about the 6000s is how easy it is to run all the existing projects and papers made for the 80gig H100s

@itsgrantpal · 2025-11-20 17:33
@__tinygrad__ @SIGKITTEN @SIGKITTEN deserves a raise, praise and a bag of lays

@__tinygrad__ · 2025-11-20 17:37
@checkfoc_us @SIGKITTEN lol we let them out of the $10k bet about which machine is faster, we both know its the 8x5090s, but the blackwell has it's place too, it certainly is more power efficient.

@__tinygrad__ · 2025-11-20 17:30
We've actually sold two of the 4x Blackwell boxes! Thanks to @SIGKITTEN for bringing the price back down to earth. It's not going to outperform 8x5090s, but the pro box is loud and requires 200+ V. The $50k Blackwell box is quiet and plugs into 2 standard US outlets. https://t.co/pB7ghTfnHY

@sergiuprt · 2025-11-20 17:37
@__tinygrad__ @SIGKITTEN I’m unable to understand why does it require 2 outlets? Genuine question

@SIGKITTEN · 2025-11-20 17:37
@__tinygrad__ its such a great top-shelf beast! my fav part about the 6000s is how easy it is to run all the existing projects and papers made for the 80gig H100s

@__tinygrad__ · 2025-11-20 17:38
@SIGKITTEN For ease of use, if you are rich and lazy, there's no better box to buy.

@sergiuprt · 2025-11-20 17:37
@__tinygrad__ @SIGKITTEN I’m unable to understand why does it require 2 outlets? Genuine question

@__tinygrad__ · 2025-11-20 17:39
@sergiuprt @SIGKITTEN @grok why does a 4x RTX 6000 Blackwell machine require 2 outlets in the US?

@__tinygrad__ · 2025-11-20 17:37
@checkfoc_us @SIGKITTEN lol we let them out of the $10k bet about which machine is faster, we both know its the 8x5090s, but the blackwell has it's place too, it certainly is more power efficient.

@SIGKITTEN · 2025-11-20 17:41
@__tinygrad__ @checkfoc_us my quick runs on the default (compute-bound) nanochat d20 pretraining on rented instances were: 8x5090 - 350k tps 4x6000pro ws - 240k tps

@__tinygrad__ · 2025-11-20 17:52
fyi, 8x5090s beats 4x6000pro on throughput. though the 6000pro cards are easier to use and win on power efficiency. both options are $50k at the tiny shop

@__tinygrad__ · 2025-11-20 17:52
fyi, 8x5090s beats 4x6000pro on throughput. though the 6000pro cards are easier to use and win on power efficiency. both options are $50k at the tiny shop

@luigifcruz · 2025-11-20 17:59
@__tinygrad__ Did you guys get P2P working on non-workstation cards?

@luigifcruz · 2025-11-20 17:59
@__tinygrad__ Did you guys get P2P working on non-workstation cards?

@__tinygrad__ · 2025-11-20 18:03
@luigifcruz Yes, with a driver fork, or just using the NV driver built in to tinygrad! https://t.co/XvN6U8w6M3

@__tinygrad__ · 2025-11-20 17:39
@sergiuprt @SIGKITTEN @grok why does a 4x RTX 6000 Blackwell machine require 2 outlets in the US?

@Nedsir1 · 2025-11-20 18:10
@__tinygrad__ @sergiuprt @SIGKITTEN @grok What's up with the US domestic power system? In the UK we run ~3kW to a standard socket.

@Nedsir1 · 2025-11-20 18:10
@__tinygrad__ @sergiuprt @SIGKITTEN @grok What's up with the US domestic power system? In the UK we run ~3kW to a standard socket.

@__tinygrad__ · 2025-11-20 18:14
@Nedsir1 @sergiuprt @SIGKITTEN @grok That's why you guys drink more tea, the tea kettles are twice as fast!

@__tinygrad__ · 2025-11-20 18:19
This team is doing great work, high end models that are actually open.

@__tinygrad__ · 2025-11-19 23:47
Cut 30% this morning with a refactor and a tasteful cache on the div_mod_folding function. There's another 30% to gain by switching the order of devectorize and lower_index_dtype. https://t.co/sstLB537U1

@__tinygrad__ · 2025-11-20 19:44
It's already almost 2x faster! This is stable diffusion running on METAL, look how little of the time is actually on the GPU. Of course, once you JIT it's all GPU time, but development is more fun when it's fast. https://t.co/gRkJnHh4Fl

@bendiken · 2025-11-21 01:31
@pavelklymenko @__tinygrad__ 👀

@pavelklymenko · 2025-11-21 01:45
@bendiken @__tinygrad__ https://t.co/pRM21MYaTG

@pavelklymenko · 2025-11-21 01:45
@bendiken @__tinygrad__ https://t.co/pRM21MYaTG

@__tinygrad__ · 2025-11-21 02:29
@pavelklymenko @bendiken Won't work on USB3, might work on USB4/thunderbolt

@__tinygrad__ · 2025-11-21 17:44
Want to inspect each wave running on your AMD GPU, run with VIZ=2! VIZ=1 is all of the low overhead (<20%) stuff, good for 90% of profiling tasks, but if you want profiling at the instruction level, VIZ=2 goes further. https://t.co/qbKJ7SNufY

@__tinygrad__ · 2025-11-21 17:44
Want to inspect each wave running on your AMD GPU, run with VIZ=2! VIZ=1 is all of the low overhead (<20%) stuff, good for 90% of profiling tasks, but if you want profiling at the instruction level, VIZ=2 goes further. https://t.co/qbKJ7SNufY

@__tinygrad__ · 2025-11-21 17:48
It's still using rocprof-trace-decoder for now, but we have the raw SQTT format 80% figured out. It's so cool how much debug info you can get from these GPUs. (oh btw this is all running on a Mac connected over USB4 to a 7900XTX) https://t.co/69XveXWFUF

@__tinygrad__ · 2025-11-21 17:53
Why are you SSHing? Just plug your GPU into your Mac! https://t.co/bFeLIMjgIg

@__tinygrad__ · 2025-11-21 17:53
Why are you SSHing? Just plug your GPU into your Mac! https://t.co/bFeLIMjgIg

@04l3x4nd3r0 · 2025-11-21 17:55
@__tinygrad__ how to?

@04l3x4nd3r0 · 2025-11-21 17:55
@__tinygrad__ how to?

@__tinygrad__ · 2025-11-21 17:56
@04l3x4nd3r0 tinygrad + ADT-UT3G

@__tinygrad__ · 2025-11-21 17:53
Why are you SSHing? Just plug your GPU into your Mac! https://t.co/bFeLIMjgIg

@eawlot3000 · 2025-11-21 17:56
@__tinygrad__ Wait tinygrad now can do this? Also supports for both nv and amd gpus?

@__tinygrad__ · 2025-11-21 17:48
It's still using rocprof-trace-decoder for now, but we have the raw SQTT format 80% figured out. It's so cool how much debug info you can get from these GPUs. (oh btw this is all running on a Mac connected over USB4 to a 7900XTX) https://t.co/69XveXWFUF

@__tinygrad__ · 2025-11-21 17:57
.@AnushElangovan even if you can't open source the library, some docs on this would be much appreciated, like the real names of the trace ops and fields.

@eawlot3000 · 2025-11-21 17:56
@__tinygrad__ Wait tinygrad now can do this? Also supports for both nv and amd gpus?

@__tinygrad__ · 2025-11-21 17:58
@eawlot3000 Of course. Anything NVIDIA 2xxx/3xxx/4xxx/5xxx and AMD RDNA2/3/4.

@__tinygrad__ · 2025-11-21 17:53
Why are you SSHing? Just plug your GPU into your Mac! https://t.co/bFeLIMjgIg

@monstercameron · 2025-11-21 18:18
@__tinygrad__ Does this work on Qualcomm X elite for windows on arm?

@__tinygrad__ · 2025-11-21 17:53
Why are you SSHing? Just plug your GPU into your Mac! https://t.co/bFeLIMjgIg

@K2adir · 2025-11-21 18:21
@__tinygrad__ Can it share vram with macbook, and if yes, do you use the Mac as the cheap vram pool and the gpu as the computing power? 👀

@K2adir · 2025-11-21 18:21
@__tinygrad__ Can it share vram with macbook, and if yes, do you use the Mac as the cheap vram pool and the gpu as the computing power? 👀

@__tinygrad__ · 2025-11-21 18:42
@K2adir Err, it's connected over a wire (visible in picture). It can "share RAM" over that wire.

@__tinygrad__ · 2025-11-21 17:53
Why are you SSHing? Just plug your GPU into your Mac! https://t.co/bFeLIMjgIg

@sandorpetofi34 · 2025-11-21 18:43
@__tinygrad__ no cuda😢

@sandorpetofi34 · 2025-11-21 18:43
@__tinygrad__ no cuda😢

@__tinygrad__ · 2025-11-21 18:44
@sandorpetofi34 Of course you can have CUDA (with an external NVIDIA GPU in macOS), just run the compiler in Docker.

@__tinygrad__ · 2025-11-21 17:53
Why are you SSHing? Just plug your GPU into your Mac! https://t.co/bFeLIMjgIg

@spacemandev · 2025-11-21 19:30
@__tinygrad__ wait where is the write up on this, would love to see how you are able to get this to work

@__tinygrad__ · 2025-11-21 17:53
Why are you SSHing? Just plug your GPU into your Mac! https://t.co/bFeLIMjgIg

@DrLeoSpacemn · 2025-11-21 19:38
@__tinygrad__ Do you plan to partner with a company like @SonnetTech who has been building PCIe over thunderbolt for a long time. They built the first eGPU developer kits for @Apple pre M-series chips

@spacemandev · 2025-11-21 19:30
@__tinygrad__ wait where is the write up on this, would love to see how you are able to get this to work

@__tinygrad__ · 2025-11-21 19:43
@spacemandev Copy and paste this directory into ChatGPT for a writeup. https://t.co/2TIUtAxsYa

@monstercameron · 2025-11-21 18:18
@__tinygrad__ Does this work on Qualcomm X elite for windows on arm?

@__tinygrad__ · 2025-11-21 19:45
@monstercameron lol Windows. Nobody tested that, though if you port our 100 line USB4 passthrough Mac driver (at tinygrad/extra/usbgpu/tbgpu) the rest will probably work.

@DrLeoSpacemn · 2025-11-21 19:38
@__tinygrad__ Do you plan to partner with a company like @SonnetTech who has been building PCIe over thunderbolt for a long time. They built the first eGPU developer kits for @Apple pre M-series chips

@__tinygrad__ · 2025-11-21 19:46
@DrLeoSpacemn @SonnetTech @Apple .@comma_ai is building a board, we'll sell an affordable eGPU enclosure next year. openpilot is going to use them to extend comma fours also.

@jeremyphoward · 2025-11-22 05:10
There simply are not opportunities to improve kernel speeds by very much on standard hardware. So any such claim means there's a major problem. https://t.co/j9g120omUL

@__tinygrad__ · 2025-11-21 17:53
Why are you SSHing? Just plug your GPU into your Mac! https://t.co/bFeLIMjgIg

@HotAisle · 2025-11-22 20:00
@__tinygrad__ We've had this for years, but it certainly is nice that you're open sourcing it. https://t.co/gJL9h5wYjc

@jsuarez · 2025-11-22 22:02
Exception! We are getting 12x faster kernels in some places in PufferLib because PyTorch is not optimized for small models. The verification of correctness is whether we can still train our model on top of numerical checks

@HotAisle · 2025-11-22 20:00
@__tinygrad__ We've had this for years, but it certainly is nice that you're open sourcing it. https://t.co/gJL9h5wYjc

@__tinygrad__ · 2025-11-22 22:19
@HotAisle lol "eGPUs are supported by any Mac with an Intel processor." we support eGPUs on the M series Macs people actually have.

@jsuarez · 2025-11-22 22:02
Exception! We are getting 12x faster kernels in some places in PufferLib because PyTorch is not optimized for small models. The verification of correctness is whether we can still train our model on top of numerical checks

@__tinygrad__ · 2025-11-22 23:15
@jsuarez Have you tried tinygrad? It is very optimized for small models.

@__tinygrad__ · 2025-11-22 23:15
@jsuarez Have you tried tinygrad? It is very optimized for small models.

@jsuarez · 2025-11-22 23:22
@__tinygrad__ I haven't even though I have two of your boxes. It comes down to: - Having users write something that isn't pytorch is a good way to kill your library & business - Any jit that adds more than a second to startup can burn in hell - I want more lower-level control over time

@__tinygrad__ · 2025-11-23 02:57
GPUs have amazing instruction tracing abilities! This is RDNA3, sadly the low level format isn't documented, but it's not that hard to figure out. You can see the dispatch (INST) and processing (EXEC) of each instruction. https://t.co/8fenf42t3B

@__tinygrad__ · 2025-11-23 02:57
GPUs have amazing instruction tracing abilities! This is RDNA3, sadly the low level format isn't documented, but it's not that hard to figure out. You can see the dispatch (INST) and processing (EXEC) of each instruction. https://t.co/8fenf42t3B

@yacineMTB · 2025-11-23 03:04
@__tinygrad__ i dont get why source isnt open for these things its not like it actually matters or anyone is going to copy

@yacineMTB · 2025-11-23 03:04
@__tinygrad__ i dont get why source isnt open for these things its not like it actually matters or anyone is going to copy

@__tinygrad__ · 2025-11-23 05:36
@yacineMTB yea no idea, we've asked a bunch of times and haven't gotten an answer.

@__tinygrad__ · 2025-11-23 02:57
GPUs have amazing instruction tracing abilities! This is RDNA3, sadly the low level format isn't documented, but it's not that hard to figure out. You can see the dispatch (INST) and processing (EXEC) of each instruction. https://t.co/8fenf42t3B

@bruce_x_offi · 2025-11-23 05:37
@__tinygrad__ Does tinygrad work on Intel GPUs? I got 2 B580s.

@bruce_x_offi · 2025-11-23 05:37
@__tinygrad__ Does tinygrad work on Intel GPUs? I got 2 B580s.

@__tinygrad__ · 2025-11-23 05:38
@bruce_x_offi @grok plz answer this

@__tinygrad__ · 2025-11-23 05:39
RT @bendiken: Late night hacking on making an RTX 5060 Ti work as an eGPU for LLMs with my MacBook Air M4 🛠️ The ADT-UT3G dock has been sold out everywhere these past weeks after @__tinygrad__ stunned everyone with their showcase of NVIDIA-over-USB4 on Apple Silicon. Wait times are running well into December ❄️ I received mine today after weeks of waiting! (Pro tip: if you can't source this dock, the AOOSTAR AG02 ought to work as well since it uses the same ASM2464PD chip)

@jsuarez · 2025-11-22 23:22
@__tinygrad__ I haven't even though I have two of your boxes. It comes down to: - Having users write something that isn't pytorch is a good way to kill your library & business - Any jit that adds more than a second to startup can burn in hell - I want more lower-level control over time

@__tinygrad__ · 2025-11-23 05:52
@jsuarez - Can't help you there - We're working on this. The "JIT" won't even be a JIT soon, just a schedule cache. - We're working on this too. We have a UOp language for kernels now that should give you full control of data movement through the memory hierarchy.

@__tinygrad__ · 2025-11-23 06:01
RT @comma_ai: It’s so tiny you can print it! @__tinygrad__ booth at COMMA_CON https://t.co/O02Ykvam9f

@__tinygrad__ · 2025-11-23 05:38
@bruce_x_offi @grok plz answer this

@bruce_x_offi · 2025-11-23 06:48
@__tinygrad__ @grok What are your thoughts on handwritten optimized kernel (OpenVINO) vs the way tinygrad generates?

@__tinygrad__ · 2025-11-23 05:52
@jsuarez - Can't help you there - We're working on this. The "JIT" won't even be a JIT soon, just a schedule cache. - We're working on this too. We have a UOp language for kernels now that should give you full control of data movement through the memory hierarchy.

@jsuarez · 2025-11-23 13:24
@__tinygrad__ If it's good, it's good! Right now, most of what I'm doing is fusing tons of small kernels manually

@followjason · 2025-11-23 19:01
Arto Bendiken’s findings running an RTX 5060 Ti as an eGPU on his M4 Air via the ADT-UT3G USB4 dock and tinygrad drivers. @__tinygrad__ is cool. https://t.co/kohE3MTWqQ

@jsuarez · 2025-11-23 21:32
Every piece of jax I've ever seen has been painful to read. Obnoxious dogmatically functional crap. If you include flax etc, you can also shoehorn in the worst parts of oop too.

@nalinrajput23 · 2025-11-24 13:28
Guess the programming language https://t.co/nkKpKIH4F9

@jsuarez · 2025-11-23 13:24
@__tinygrad__ If it's good, it's good! Right now, most of what I'm doing is fusing tons of small kernels manually

@__tinygrad__ · 2025-11-24 16:51
@jsuarez tinygrad will literally do that for you

@__tinygrad__ · 2025-11-24 16:52
tinygrad gives you all the laziness and functional nature of JAX with the API of PyTorch.

@followjason · 2025-11-23 19:01
Arto Bendiken’s findings running an RTX 5060 Ti as an eGPU on his M4 Air via the ADT-UT3G USB4 dock and tinygrad drivers. @__tinygrad__ is cool. https://t.co/kohE3MTWqQ

@__tinygrad__ · 2025-11-24 16:56
@followjason Someone should start compiling a list of all the enclosures and GPUs that work with tinygrad's eGPU driver. We only have ADT-UT3G here.

@bruce_x_offi · 2025-11-23 06:48
@__tinygrad__ @grok What are your thoughts on handwritten optimized kernel (OpenVINO) vs the way tinygrad generates?

@__tinygrad__ · 2025-11-24 16:58
@bruce_x_offi @grok Intel will cancel whatever hardware you handwrote a kernel for, better to write a generic compiler.

@__tinygrad__ · 2025-11-24 16:51
@jsuarez tinygrad will literally do that for you

@jsuarez · 2025-11-24 16:59
@__tinygrad__ Okay. I will be very surprised if tinygrad comes close to custom kernels. If I spend a little time to benchmark + post publicly, will you take a quick look and tell me if I'm doing the tinygrad one stupidly?

@__tinygrad__ · 2025-11-24 16:58
@bruce_x_offi @grok Intel will cancel whatever hardware you handwrote a kernel for, better to write a generic compiler.

@bruce_x_offi · 2025-11-24 17:00
@__tinygrad__ @grok "Intel will cancel"??? That is what they have done in OpenVINO.

@jsuarez · 2025-11-24 16:59
@__tinygrad__ Okay. I will be very surprised if tinygrad comes close to custom kernels. If I spend a little time to benchmark + post publicly, will you take a quick look and tell me if I'm doing the tinygrad one stupidly?

@__tinygrad__ · 2025-11-24 17:00
@jsuarez Sure! There's still a bunch of footguns, but written correctly and with BEAM tinygrad should outperform all but the absolute best handwritten kernels.

@__tinygrad__ · 2025-11-24 17:00
@jsuarez Sure! There's still a bunch of footguns, but written correctly and with BEAM tinygrad should outperform all but the absolute best handwritten kernels.

@jsuarez · 2025-11-24 17:13
@__tinygrad__ Okay deal. Let me finish a couple more and will give it a go. My kernels suck and still rock torch

@__tinygrad__ · 2025-11-24 16:52
tinygrad gives you all the laziness and functional nature of JAX with the API of PyTorch.

@liranringel · 2025-11-24 17:57
@__tinygrad__ Should we really use it for training/inference today?

@liranringel · 2025-11-24 17:57
@__tinygrad__ Should we really use it for training/inference today?

@__tinygrad__ · 2025-11-24 18:00
@liranringel For mainstream training on NVIDIA, no. For more obscure models or more obscure hardware, maybe.

@bruce_x_offi · 2025-11-24 17:00
@__tinygrad__ @grok "Intel will cancel"??? That is what they have done in OpenVINO.

@__tinygrad__ · 2025-11-24 18:02
@bruce_x_offi @grok There's currently no future in the consumer GPUs, datacenter GPU Max, or Gaudi. Until Intel has someone capable of leading in AI (and the organization permits this), I would avoid putting any Intel specific resources in.

@jsuarez · 2025-11-24 17:13
@__tinygrad__ Okay deal. Let me finish a couple more and will give it a go. My kernels suck and still rock torch

@__tinygrad__ · 2025-11-24 19:37
@jsuarez If you find some kernels in tinygrad to be easier to write/outperform your custom kernels, it should be pretty easy to wire just that kernel into torch as well.

@jsuarez · 2025-11-24 19:48
@__tinygrad__ Thanks. I did not realize what a massive pain RMSnorm was going to be. Thing eats 25% of the total cuda time in torch.

@__tinygrad__ · 2025-11-24 19:50
@jsuarez We also have a language for writing custom kernels now if you want to do things beyond what the fuser can find. It's still a ton simpler than CUDA. And if you put the work in to write a test in tiny's CI comparing your custom code to tinygrad code, we can work to optimize.

@__tinygrad__ · 2025-11-24 19:50
@jsuarez We also have a language for writing custom kernels now if you want to do things beyond what the fuser can find. It's still a ton simpler than CUDA. And if you put the work in to write a test in tiny's CI comparing your custom code to tinygrad code, we can work to optimize.

@__tinygrad__ · 2025-11-24 19:52
@jsuarez But for most straightforward things, the fuser is good. And our runtimes are by far the best for submitting tons of small kernels, even faster than CUDA Graph. Also, you can profile it all with VIZ=1 in a very low overhead UI that runs in a browser.

@__tinygrad__ · 2025-11-24 16:52
tinygrad gives you all the laziness and functional nature of JAX with the API of PyTorch.

@jcopax · 2025-11-24 20:36
@__tinygrad__ Functional tinygrad? where ?vmap? scan?

@jcopax · 2025-11-24 20:36
@__tinygrad__ Functional tinygrad? where ?vmap? scan?

@__tinygrad__ · 2025-11-24 21:13
@jcopax It's functional as in it builds a graph and does all the operations over the graph. We have vmap (I think with a more powerful API than JAX even, you can just slice with a range). We don't have scan, working on it.

@jeremyphoward · 2025-11-24 21:40
We just decided to give all students in our "How to Solve it With Code" course free access to Opus 4.5 for the rest of this year. Although the course has started already, you can still sign up and catch up with the recordings here: https://t.co/W0DGKEmLo5 https://t.co/gfY4k15R3l

@__tinygrad__ · 2025-11-24 21:40
RT @snapolino: Tinygrad is just on another level.... truly thank you for making such a beautyful ml framework! @__tinygrad__ https://t.co/kvTzxl0Zac

@__tinygrad__ · 2025-11-24 23:19
Should we be raising prices? https://t.co/VdVjfuBOno

@nalinrajput23 · 2025-11-24 13:28
Guess the programming language https://t.co/nkKpKIH4F9

@__tinygrad__ · 2025-11-24 23:23
@nalinrajput23 C. It's only C. You take C so for granted. It's such an amazing language you don't even think about it. Python is written in C.

@__tinygrad__ · 2025-11-24 23:19
Should we be raising prices? https://t.co/VdVjfuBOno

@SIGKITTEN · 2025-11-24 23:31
@__tinygrad__ noooooooooooooo 😂

@SIGKITTEN · 2025-11-24 23:31
@__tinygrad__ noooooooooooooo 😂

@__tinygrad__ · 2025-11-25 00:11
@SIGKITTEN we need to have our own RAM factory

@SpectralCom · 2025-11-25 12:02
A great question we've been asked a lot at #SC25 is "Why does #NVIDIA not sue you to oblivion?". Well, this is due to many reasons.

@ID_AA_Carmack · 2025-11-25 16:57
Always a slightly mixed feeling to write pretty good first-principles code to do some tensor rearrangement, only to find that PyTorch has a built in function that does it faster. I had made a point of at least skimming the docs of every torch and tensor function, but if you don’t know what you would use something for, it probably won’t stick in your head for when the opportunity arises. Pixel_unshuffle, looking at you.

@nvidianewsroom · 2025-11-25 17:00
We’re delighted by Google’s success — they’ve made great advances in AI and we continue to supply to Google. NVIDIA is a generation ahead of the industry — it’s the only platform that runs every AI model and does it everywhere computing is done. NVIDIA offers greater performance, versatility, and fungibility than ASICs, which are designed for specific AI frameworks or functions.

@__tinygrad__ · 2025-11-25 17:04
This isn't why. Trying to "compile" CUDA for AMD is nonsense; NVIDIA loves when people try. CUDA will never be fast on AMD (how do you compile if the shared memory / tensor cores are a different size?). It's the wrong layer to do this at.

@__tinygrad__ · 2025-11-25 17:04
This isn't why. Trying to "compile" CUDA for AMD is nonsense; NVIDIA loves when people try. CUDA will never be fast on AMD (how do you compile if the shared memory / tensor cores are a different size?). It's the wrong layer to do this at.

@SifoDyas3 · 2025-11-25 17:08
@__tinygrad__ The IR is the right layer? Like the tiny UOps?

@SpectralCom · 2025-11-25 17:34
HIP is literally clang-dialect CUDA: it isn't a different programmnig language, so there's no difference there. NVIDIA currently sell GPUs with at least 5 different shared memory sizes, and at least 3 different tensor core sizes. CUDA already has features for modelling these types of difference: we're doing the compiler R&D to take it further to work well on AMD.

@joefioti · 2025-11-25 17:54
@SpectralCom @__tinygrad__ How do you guys handle multi-swizzle patterns? Numa domains? Warp-specialization without register realloc? All of these exist on AMD and not on cuda

@beffjezos · 2025-11-25 19:14
TPUs were better than GPUs in my experience at Google, on many benchmarks. Never made sense to me why Nvidia would be so dominant. Seems like the market is finally pricing in the reality.

@ID_AA_Carmack · 2025-11-25 16:57
Always a slightly mixed feeling to write pretty good first-principles code to do some tensor rearrangement, only to find that PyTorch has a built in function that does it faster. I had made a point of at least skimming the docs of every torch and tensor function, but if you don’t know what you would use something for, it probably won’t stick in your head for when the opportunity arises. Pixel_unshuffle, looking at you.

@codingfisch · 2025-11-25 19:17
@ID_AA_Carmack One more reason to root for @__tinygrad__ ! If they succeed, there is no need to memorize all those ops. Just let the magic compiler find the golden kernel ✨

@joefioti · 2025-11-25 17:54
@SpectralCom @__tinygrad__ How do you guys handle multi-swizzle patterns? Numa domains? Warp-specialization without register realloc? All of these exist on AMD and not on cuda

@__tinygrad__ · 2025-11-25 19:23
@joefioti @SpectralCom Exactly. The HipKittens blog explains why the compile CUDA for AMD will never be a good idea, and actually always ends up just making NVIDIA look good.

@__tinygrad__ · 2025-11-25 19:23
@joefioti @SpectralCom Exactly. The HipKittens blog explains why the compile CUDA for AMD will never be a good idea, and actually always ends up just making NVIDIA look good.

@joefioti · 2025-11-25 19:25
@__tinygrad__ @SpectralCom Agreed. Many people had the exact wrong takeaways from that post.

@joefioti · 2025-11-25 19:25
@__tinygrad__ @SpectralCom Agreed. Many people had the exact wrong takeaways from that post.

@__tinygrad__ · 2025-11-25 19:27
@joefioti @SpectralCom The AMD shared memory swizzle patterns are way worse than the NVIDIA ones. This alone precludes a C-like language transformation from ever working well. Hoping AMD fixes this on RDNA5.

@SifoDyas3 · 2025-11-25 17:08
@__tinygrad__ The IR is the right layer? Like the tiny UOps?

@__tinygrad__ · 2025-11-25 19:28
@SifoDyas3 UOps in tinygrad span the full IR. The right layer is probably something like Triton/ThunderKittens/tile-lang.

@codingfisch · 2025-11-25 19:17
@ID_AA_Carmack One more reason to root for @__tinygrad__ ! If they succeed, there is no need to memorize all those ops. Just let the magic compiler find the golden kernel ✨

@__tinygrad__ · 2025-11-25 19:29
@codingfisch @ID_AA_Carmack That's the goal! Our fusion for stuff like this is already really good, our matmuls and convs are not ops.

@ID_AA_Carmack · 2025-11-25 20:32
@jsuarez CUDA graphs get rid of both python and C++ overhead; I’m a fan!

@jsuarez · 2025-11-25 20:35
@ID_AA_Carmack I've been really trying to avoid them if I can. It can be a pain for new models + compile startup overhead from python API. But with our new architecture, I can only fuse so many kernels. I haven't found a good way to do it from cpp though

@TheAhmadOsman · 2025-11-25 20:40
people that think like this still exist btw and theyʼre in for a big surprise sadly https://t.co/9eiwfj7Lhk

@__tinygrad__ · 2025-11-25 21:08
RT @SemiAnalysis_: AMD claims that all their software is open source, yet reality does not match this claim. For example, AMD's rocprof-trace-decoder is still completely closed source currently despite repeated requests for months by ML community members such as George Hotz, who is a daily AMD GPU end user. On the same June Spotify podcast, Tobias Macey & AMD's @AnushElangovan (who has a new twitter pfp) said that "open source allows for innovation to go at the pace at which the people using [AMD] want to, right, and it's not limited by the ability of what we put out in closed source form". We agree with an open source first approach and we agree that AMD should not limit ML community members like George Hotz by continuing to keep rocprof-trace-decoder closed source. When will AMD open source rocprof-trace-decoder?

@SemiAnalysis_ · 2025-11-25 20:30
AMD claims that all their software is open source, yet reality does not match this claim. For example, AMD's rocprof-trace-decoder is still completely closed source currently despite repeated requests for months by ML community members such as George Hotz, who is a daily AMD GPU end user. On the same June Spotify podcast, Tobias Macey & AMD's @AnushElangovan (who has a new twitter pfp) said that "open source allows for innovation to go at the pace at which the people using [AMD] want to, right, and it's not limited by the ability of what we put out in closed source form". We agree with an open source first approach and we agree that AMD should not limit ML community members like George Hotz by continuing to keep rocprof-trace-decoder closed source. When will AMD open source rocprof-trace-decoder?

@__tinygrad__ · 2025-11-25 21:09
@SemiAnalysis_ We would still love for them to open source it, but our replacement is almost ready anyway. Here's a decoded SQTT stream from RDNA3. https://t.co/VdiS81MYBg

@__tinygrad__ · 2025-11-25 21:12
@SellingStrikes AVX can't be fast on ARM.

@ruima · 2025-11-25 21:12
The most anticipated IPO of the year in China is the "Nvidia of China" Moore Threads. The IPO is oversubscribed by ... 4126.49 times. It's founded by Zhang Jianzhong, the former global vice-president of Nvidia and general manager of Nvidia China. https://t.co/HlCYoA34Ej

@draecomino · 2025-11-25 21:26
GPUs have this unique property that TPUs don't have – you can buy one.

@__tinygrad__ · 2025-11-25 17:04
This isn't why. Trying to "compile" CUDA for AMD is nonsense; NVIDIA loves when people try. CUDA will never be fast on AMD (how do you compile if the shared memory / tensor cores are a different size?). It's the wrong layer to do this at.

@Shipwright_42 · 2025-11-25 21:28
@__tinygrad__ Nah not slow enough, let’s emulate CUDA on cpu,

@Shipwright_42 · 2025-11-25 21:28
@__tinygrad__ Nah not slow enough, let’s emulate CUDA on cpu,

@__tinygrad__ · 2025-11-25 21:28
@Sariel_42 tinygrad is actually doing this in our CI. Very slow, can confirm.

@SemiAnalysis_ · 2025-11-25 20:30
AMD claims that all their software is open source, yet reality does not match this claim. For example, AMD's rocprof-trace-decoder is still completely closed source currently despite repeated requests for months by ML community members such as George Hotz, who is a daily AMD GPU end user. On the same June Spotify podcast, Tobias Macey & AMD's @AnushElangovan (who has a new twitter pfp) said that "open source allows for innovation to go at the pace at which the people using [AMD] want to, right, and it's not limited by the ability of what we put out in closed source form". We agree with an open source first approach and we agree that AMD should not limit ML community members like George Hotz by continuing to keep rocprof-trace-decoder closed source. When will AMD open source rocprof-trace-decoder?

@AnushElangovan · 2025-11-25 21:32
@SemiAnalysis_ https://t.co/heCkUrUsZh

@SemiAnalysis_ · 2025-11-25 20:30
AMD claims that all their software is open source, yet reality does not match this claim. For example, AMD's rocprof-trace-decoder is still completely closed source currently despite repeated requests for months by ML community members such as George Hotz, who is a daily AMD GPU end user. On the same June Spotify podcast, Tobias Macey & AMD's @AnushElangovan (who has a new twitter pfp) said that "open source allows for innovation to go at the pace at which the people using [AMD] want to, right, and it's not limited by the ability of what we put out in closed source form". We agree with an open source first approach and we agree that AMD should not limit ML community members like George Hotz by continuing to keep rocprof-trace-decoder closed source. When will AMD open source rocprof-trace-decoder?

@AnushElangovan · 2025-11-25 21:32
@SemiAnalysis_ https://t.co/heCkUrUsZh

@__tinygrad__ · 2025-11-25 21:34
RT @jeremyphoward: Oh also, @__tinygrad__ is now supported on Solveit :) https://t.co/3oAkZDPkL1

@AnushElangovan · 2025-11-25 21:32
@SemiAnalysis_ https://t.co/heCkUrUsZh

@__tinygrad__ · 2025-11-25 21:37
@AnushElangovan @SemiAnalysis_ It's the most important one if you actually want to get performance on these GPUs. In torch on the latest ROCm the MI350X is getting only 43% of its rated performance on gemm. Better profilers (without bugs) will help fix this. https://t.co/vHIiuz9wRe

@nvidianewsroom · 2025-11-25 17:00
We’re delighted by Google’s success — they’ve made great advances in AI and we continue to supply to Google. NVIDIA is a generation ahead of the industry — it’s the only platform that runs every AI model and does it everywhere computing is done. NVIDIA offers greater performance, versatility, and fungibility than ASICs, which are designed for specific AI frameworks or functions.

@__tinygrad__ · 2025-11-25 21:43
@nvidianewsroom Those em dashes were generated on an NVIDIA GPU!

@AnushElangovan · 2025-11-25 21:45
oh found it https://t.co/EHhbtviUju

@__tinygrad__ · 2025-11-25 21:46
This optimization journey is just beginning. We are starting to need to push beyond what AMD LLVM is capable of outputting; once we are in machine code with a tight profile-guided-optimization loop there's no stopping the tinygrad.

@jsuarez · 2025-11-25 20:35
@ID_AA_Carmack I've been really trying to avoid them if I can. It can be a pain for new models + compile startup overhead from python API. But with our new architecture, I can only fuse so many kernels. I haven't found a good way to do it from cpp though

@__tinygrad__ · 2025-11-25 21:47
@jsuarez @ID_AA_Carmack tinygrad gives you them for ~free!

@__tinygrad__ · 2025-11-25 22:21
It has arrived. You know there's 96GB of RAM in there? https://t.co/2v541KOfVk

@AnushElangovan · 2025-11-25 21:45
oh found it https://t.co/EHhbtviUju

@__tinygrad__ · 2025-11-25 22:30
@AnushElangovan So why is that one closed?

@__tinygrad__ · 2025-11-25 22:21
It has arrived. You know there's 96GB of RAM in there? https://t.co/2v541KOfVk

@__tim · 2025-11-25 23:27
@__tinygrad__ vs 128GB in a framework desktop...

@__tinygrad__ · 2025-11-25 22:21
It has arrived. You know there's 96GB of RAM in there? https://t.co/2v541KOfVk

@0xCritters · 2025-11-25 23:43
@__tinygrad__ is this going in the tinybox v3 pro max prime?

@draecomino · 2025-11-25 21:26
GPUs have this unique property that TPUs don't have – you can buy one.

@__tinygrad__ · 2025-11-25 23:57
@draecomino It doesn't matter how good TPUs get, as long as this is true NVIDIA will be better. Remember that time Google lost to Amazon at cloud? https://t.co/C7wCq92Nch

@__tim · 2025-11-25 23:27
@__tinygrad__ vs 128GB in a framework desktop...

@__tinygrad__ · 2025-11-25 23:59
@__tim I can't wait until average PC builders start understanding and bragging about GB/s and not GB.

@ruima · 2025-11-25 21:12
The most anticipated IPO of the year in China is the "Nvidia of China" Moore Threads. The IPO is oversubscribed by ... 4126.49 times. It's founded by Zhang Jianzhong, the former global vice-president of Nvidia and general manager of Nvidia China. https://t.co/HlCYoA34Ej

@__tinygrad__ · 2025-11-26 00:01
@ruima Oh man we actually bought this GPU. It's nice to see them trying, but I think as of now the Qualcomm GPU in my phone beats this.

@0xCritters · 2025-11-25 23:43
@__tinygrad__ is this going in the tinybox v3 pro max prime?

@__tinygrad__ · 2025-11-26 00:03
@asap_beepboop green v2 Blackwell edition. https://t.co/pTkQ44z96o

@__tinygrad__ · 2025-11-25 22:30
@AnushElangovan So why is that one closed?

@sivapatibandla1 · 2025-11-26 01:41
@__tinygrad__ @AnushElangovan you guys are super adversarial for a company who’s getting paid by AMD.

@jeremyphoward · 2025-11-26 01:46
Here's my exploration of the 1st three @__tinygrad__ puzzles in Solveit. I also fixed some bugs and (IMO) mis-features in the underlying lib. ( @srush_nlp dunno if they're in the original as well - tagging you in case you're interested). https://t.co/ecjD7JXdbE

@basedjensen · 2025-11-26 07:00
Anyone who thinks they are better than current bleeding edge models writing code are insufferable assholes who actually are not as good as they sell themselves as.

@lukecodez · 2025-11-26 09:48
Startup Idea : One port for all https://t.co/XOJUBpzUXY

@__tinygrad__ · 2025-11-25 22:21
It has arrived. You know there's 96GB of RAM in there? https://t.co/2v541KOfVk

@LordLochFergus · 2025-11-26 15:55
@__tinygrad__ Cool. Now plug it into a macbook

@LordLochFergus · 2025-11-26 15:55
@__tinygrad__ Cool. Now plug it into a macbook

@__tinygrad__ · 2025-11-26 18:25
@LordLochFergus Wait actually we totally could.

@__tinygrad__ · 2025-11-26 19:41
Waves are blue with VIZ=2 on AMD! The lit up waves include a full instruction trace. People think NVIDIA is a chip company; they are actually a software company. Once you have the software, including a great profiler, the chip is easy. https://t.co/WarY1yTIf9

@__tinygrad__ · 2025-11-26 19:47
RT @jeremyphoward: Here's my exploration of the 1st three @__tinygrad__ puzzles in Solveit. I also fixed some bugs and (IMO) mis-features in the underlying lib. ( @srush_nlp dunno if they're in the original as well - tagging you in case you're interested). https://t.co/ecjD7JXdbE

@__tinygrad__ · 2025-11-26 19:49
@jeremyphoward @srush_nlp Happy to upstream frontend quality of life improvements, we haven't put too much thought into it yet. Using tinygrad should be fun and enjoyable.

@jeremyphoward · 2025-11-26 19:51
@__tinygrad__ @srush_nlp I meant the `tinygrad-tensor-puzzles` lib, not in tinygrad itself. You can see the improvements/changes in the link I shared.

@jeremyphoward · 2025-11-26 19:51
@__tinygrad__ @srush_nlp I meant the `tinygrad-tensor-puzzles` lib, not in tinygrad itself. You can see the improvements/changes in the link I shared.

@__tinygrad__ · 2025-11-26 19:54
@jeremyphoward @srush_nlp I understand, I mean some of these things should make it into tinygrad. Our default repr for Tensor is awful, even if you don't want to realize it (which you probably do). https://t.co/sfBwjlCL2K

@__tinygrad__ · 2025-11-26 19:41
Waves are blue with VIZ=2 on AMD! The lit up waves include a full instruction trace. People think NVIDIA is a chip company; they are actually a software company. Once you have the software, including a great profiler, the chip is easy. https://t.co/WarY1yTIf9

@EitanTurok · 2025-11-26 20:15
@__tinygrad__ why is SIMD:0 lighter than the other SIMDs?

@__tinygrad__ · 2025-11-26 20:39
Did you get our NeurIPS party notification? The party is by invitation only, solve the CTF (first find it) or be a known contributor to tinygrad to get an invite. https://t.co/WnyQXkUYF6

@EitanTurok · 2025-11-26 20:15
@__tinygrad__ why is SIMD:0 lighter than the other SIMDs?

@__tinygrad__ · 2025-11-26 20:41
@EitanTurok Because you can double click on those waves and get a full instruction trace. https://t.co/sfJT4hgNbi

@lukecodez · 2025-11-26 09:48
Startup Idea : One port for all https://t.co/XOJUBpzUXY

@__tinygrad__ · 2025-11-26 21:16
@lukecodez Fixed that for you. https://t.co/Qy6frIMeNk

@basedjensen · 2025-11-26 07:00
Anyone who thinks they are better than current bleeding edge models writing code are insufferable assholes who actually are not as good as they sell themselves as.

@__tinygrad__ · 2025-11-26 22:15
@basedjensen LLMs are not even close to the quality of a good software engineer. They are just fast. A good software engineer should be using appropriate tools for their job, which sometimes includes LLMs. But if they are committing LLM code they don't understand, they are NGMI.

@sivapatibandla1 · 2025-11-26 01:41
@__tinygrad__ @AnushElangovan you guys are super adversarial for a company who’s getting paid by AMD.

@__tinygrad__ · 2025-11-26 22:17
@sivapatibandla1 @AnushElangovan lol you can say that in reverse too. but this we are mostly joking about. while it would be great if AMD open sourced it, it's tightly tied to the hardware implementation. it's not hard to figure out how it works, and at least they gave us a mac binary in the mean time.

@__tinygrad__ · 2025-11-26 22:17
@sivapatibandla1 @AnushElangovan lol you can say that in reverse too. but this we are mostly joking about. while it would be great if AMD open sourced it, it's tightly tied to the hardware implementation. it's not hard to figure out how it works, and at least they gave us a mac binary in the mean time.

@sivapatibandla1 · 2025-11-26 22:51
@__tinygrad__ @AnushElangovan Glad to hear it. Just think this could be have been solved directly via email rather than publicly on Twitter 🤷‍♂️

@__tinygrad__ · 2025-11-26 23:21
Stable Diffusion on M3 Max is down to 27 s wall time. But we need to do better, so little of that is GPU time. `VIZ=1 python3 examples/stable_diffusion.py` to reproduce, try it on your machine and post results. https://t.co/MPzNew2mmM

@sivapatibandla1 · 2025-11-26 22:51
@__tinygrad__ @AnushElangovan Glad to hear it. Just think this could be have been solved directly via email rather than publicly on Twitter 🤷‍♂️

@__tinygrad__ · 2025-11-27 00:52
@sivapatibandla1 @AnushElangovan This sounds like MBA advice. Richest man in the world solves problems on Twitter, we be like richest man, not like MBA.

@geohotarchive · 2025-11-28 17:47
Replacing my MacBook https://t.co/5KZibFgW0L

@real_deep_ml · 2025-11-28 20:19
A @__tinygrad__ laptop would be amazing

@__tinygrad__ · 2025-11-28 21:20
If @AMD knows how to get Strix Halo from 4W idle down to 0.5W (what Apple M series is), we'll seriously think about it.

@__tinygrad__ · 2025-11-28 21:20
If @AMD knows how to get Strix Halo from 4W idle down to 0.5W (what Apple M series is), we'll seriously think about it.

@HotAisle · 2025-11-28 21:29
@__tinygrad__ @AMD What’s your mi350 idle?

@HotAisle · 2025-11-28 21:29
@__tinygrad__ @AMD What’s your mi350 idle?

@__tinygrad__ · 2025-11-28 21:29
@HotAisle @AMD Horrific. https://t.co/8FDMrlMLuG

@__tinygrad__ · 2025-11-28 21:20
If @AMD knows how to get Strix Halo from 4W idle down to 0.5W (what Apple M series is), we'll seriously think about it.

@AnushElangovan · 2025-11-28 21:32
@__tinygrad__ @AMD On linux ok ? What are you measuring now ? (and how ?) I

@AnushElangovan · 2025-11-28 21:32
@__tinygrad__ @AMD On linux ok ? What are you measuring now ? (and how ?) I

@__tinygrad__ · 2025-11-28 21:34
@AnushElangovan @AMD PPT VALUE FAST on the SMU. Code here: https://t.co/r3RlyIRNsM

@wavefnx · 2025-11-28 21:39
Got the same one on its way and should arrive anytime soon I’d never get a M*cBook 128gb RAM unified with iGPU 16 cores / 32 threads 2TB NVMe SSD Good Linux support https://t.co/Xn8XnHQdpE

@KimNoel · 2025-11-28 21:38
@__tinygrad__ @AMD interested for less than $5,000

@__tinygrad__ · 2025-11-28 21:45
@kimnoel @AMD I think we would price it at $3,999, the same price as the high configuration MacBook Pro 16-inch. 16 Zen 5 cores, 32 RDNA 3.5 cores, 64 GB of RAM at 275 GB/s, 16" OLED screen, 99.6 Wh battery, Omarchy pre-installed with great power management. No macOS, full control.

@__tinygrad__ · 2025-11-28 21:45
@kimnoel @AMD I think we would price it at $3,999, the same price as the high configuration MacBook Pro 16-inch. 16 Zen 5 cores, 32 RDNA 3.5 cores, 64 GB of RAM at 275 GB/s, 16" OLED screen, 99.6 Wh battery, Omarchy pre-installed with great power management. No macOS, full control.

@gazorp5 · 2025-11-28 21:46
@__tinygrad__ @kimnoel @AMD making the hardware in-house, or contracting out to an ODM?

@gazorp5 · 2025-11-28 21:46
@__tinygrad__ @kimnoel @AMD making the hardware in-house, or contracting out to an ODM?

@__tinygrad__ · 2025-11-28 21:48
@gazorp5 @kimnoel @AMD In house of course. We don't do branding deal crap, we optimize like crazy and delete every component possible. Would do PCBA and assembly in San Diego too. Check out the board from the comma four. https://t.co/p21V82dKKr

@__tinygrad__ · 2025-11-28 21:45
@kimnoel @AMD I think we would price it at $3,999, the same price as the high configuration MacBook Pro 16-inch. 16 Zen 5 cores, 32 RDNA 3.5 cores, 64 GB of RAM at 275 GB/s, 16" OLED screen, 99.6 Wh battery, Omarchy pre-installed with great power management. No macOS, full control.

@__tinygrad__ · 2025-11-28 22:02
@kimnoel @AMD My bad, 40 RDNA 3.5 cores, so matching Apple's core counts, but 64 GB of unified memory instead. We'd use the top spec AI Max+ 395. Same 1 TB SSD but I think we can socket that. Wi-Fi 7 unlike Apple.

@jsuarez · 2025-11-29 21:55
Well that's nice... https://t.co/0Sv4HeaBR8

@__tinygrad__ · 2025-11-30 15:03
RT @jsuarez: First and second impressions on TinyGrad are pretty good. It does a lot of things right. My initial guess of it being Jax that doesn't suck was about right. There are some things missing and somethings I don't like of course. Article early next week.

@__tinygrad__ · 2025-11-30 17:29
RT @zzznah: One of the most impressive things about tinygrad for me is that it's pure python codebase. It just uses standard ctypes modules to talk directly to backbend libraries available in the host system

@__tinygrad__ · 2025-11-30 17:30
For anything off the beaten path, tinygrad is often faster than torch!

@AnushElangovan · 2025-11-30 17:49
@__tinygrad__ @AMD ok checked and the expected Idle is similar to about ~0.5w. Will have the Halo team check on the recreate and get back.

@__tinygrad__ · 2025-11-30 19:21
@AnushElangovan @AMD If this chip will really idle at 0.5W (not standby, I mean 1% load doing nothing screen on idle) there's no reason to have any other chip in your laptop. Really hoping it's true.

@__tinygrad__ · 2025-11-30 19:21
@AnushElangovan @AMD If this chip will really idle at 0.5W (not standby, I mean 1% load doing nothing screen on idle) there's no reason to have any other chip in your laptop. Really hoping it's true.

@__tinygrad__ · 2025-11-30 19:24
@AnushElangovan @AMD Looked into the BIOS a bit and it's 69 files! Radical simplification of this stuff on both CPU and GPU would set AMD up to be the top chipmaker for the next 10 years.

@__tinygrad__ · 2025-11-30 17:30
For anything off the beaten path, tinygrad is often faster than torch!

@Wisam_AlRawi · 2025-11-30 21:26
@__tinygrad__ I need a Windows version of Tinygrad.

@__tinygrad__ · 2025-11-30 17:30
For anything off the beaten path, tinygrad is often faster than torch!

@__vilsinho__ · 2025-12-01 01:19
@__tinygrad__ Does it work on Android?

@__vilsinho__ · 2025-12-01 01:19
@__tinygrad__ Does it work on Android?

@__tinygrad__ · 2025-12-01 04:59
@__vilsinho__ Yes! Works great in termux, it's pure Python. And shouldn't be too hard to package in an app if you want that.

@Wisam_AlRawi · 2025-11-30 21:26
@__tinygrad__ I need a Windows version of Tinygrad.

@__tinygrad__ · 2025-12-01 05:00
@Wisam_AlRawi You have one, it's called tinygrad. It works on Windows, it's even tested in CI on Windows.

@__tinygrad__ · 2025-12-01 23:14
This is a weekly reminder that rocprof-trace-decoder is closed source, and pretty sure there's bugs in the the VALU stall times. We only have so many dev resources, it would be nice to not spend them rewriting this in Rust, a 4 week project...all the GPU gens are different! https://t.co/23VtD7UFD6

@__tinygrad__ · 2025-12-01 23:14
This is a weekly reminder that rocprof-trace-decoder is closed source, and pretty sure there's bugs in the the VALU stall times. We only have so many dev resources, it would be nice to not spend them rewriting this in Rust, a 4 week project...all the GPU gens are different! https://t.co/23VtD7UFD6

@__tinygrad__ · 2025-12-01 23:19
Alternatively, add a feature to output the raw SQTT stream, not something that's processed potentially incorrectly. But adding features is dev resources, open source is one button. Should we spend time on figuring this out, or on lowering Strix Halo idle power?

@__tinygrad__ · 2025-12-02 00:42
Is one of these tinybox red v2's yours? https://t.co/l0sqYN0fUQ

@__tinygrad__ · 2025-11-13 16:29
AI has made the project maintainer experience worse. We're getting more and more PRs written by people who don't know what they are doing, but it takes longer to sort them out cause the code "looks" correct. This sadly is going to end up with a stronger emphasis on identity.

@klusterai · 2025-12-02 12:38
@__tinygrad__ Hey we can help with this @__tinygrad__ 👋 https://t.co/RwGbrC7vVM

@__tinygrad__ · 2025-12-02 13:44
RT @olafwillocx: Tinygrad now does hardware video decoding on NVIDIA without an NVCUVID dependency. Only dependency is the NVIDIA driver, although originally the goal was to bypass even that. Been a long time coming, very cool. https://t.co/w6l5QJ4cmG

@ExoticSpice101 · 2025-12-03 01:02
https://t.co/5z8VpCQ9GO AMD needs better fabrics and improved clock gating. I can get my M4 Air down to 0.17 watts for the CPU and the whole system uses 2.63 watts on idle and that’s with 50% brightness and kb still backlit. https://t.co/VLWkMjohlk

@ExoticSpice101 · 2025-12-03 01:02
https://t.co/5z8VpCQ9GO AMD needs better fabrics and improved clock gating. I can get my M4 Air down to 0.17 watts for the CPU and the whole system uses 2.63 watts on idle and that’s with 50% brightness and kb still backlit. https://t.co/VLWkMjohlk

@__tinygrad__ · 2025-12-03 03:00
@ExoticSpice101 Do you know if it's a hardware or software issue with Strix Halo? It's embarrassing how much better Apple is.

@klusterai · 2025-12-02 12:38
@__tinygrad__ Hey we can help with this @__tinygrad__ 👋 https://t.co/RwGbrC7vVM

@__tinygrad__ · 2025-12-03 03:57
@klusterai No no no the answer to slop isn't more slop it's GitHub bans.

@hopefullyidont1 · 2025-12-03 15:40
Why the hell are we still listening to geohot in big 2025 like tinycorp isn't the biggest scam ever

@__tinygrad__ · 2025-12-03 21:36
RT @jsuarez: Playing with TinyGrad kernel gen on stream rn. Ilya podcast later

@__tinygrad__ · 2025-12-03 21:35
@wavefnx Power draw is still bad. AMD suggested I try Ubuntu, 0 change. @AnushElangovan idk why AMD doesn't just release docs on power gates and SMU firmware. Qualcomm has better docs and lower idle power, if they improve their Linux support they will win.

@wavefnx · 2025-12-03 22:01
Got some stuff for @AnushElangovan to look at as sometimes amdgpu was freezing with: Runtime PM not available Kernel was setting the CPU to sleep and it was probably colliding with the GPU wake path Set some kernel param overrides and seems ok but suboptimal: amdgpu.runpm=0 amdgpu.lockup_timeout=10000 amdgpu.ppfeaturemask=0xffffffff idle=nomwait

@wavefnx · 2025-12-03 22:01
Got some stuff for @AnushElangovan to look at as sometimes amdgpu was freezing with: Runtime PM not available Kernel was setting the CPU to sleep and it was probably colliding with the GPU wake path Set some kernel param overrides and seems ok but suboptimal: amdgpu.runpm=0 amdgpu.lockup_timeout=10000 amdgpu.ppfeaturemask=0xffffffff idle=nomwait

@__tinygrad__ · 2025-12-03 22:08
@wavefnx @AnushElangovan Disable the webcam in bios and the iommu in kernel bootargs if you want S0ix suspend to work.

@__tinygrad__ · 2025-12-03 22:29
I can't believe how popular the 4x Blackwell machine is, we have 3 of them bought and paid for! First one should be ready today, will do a thread with some tests.

@__tinygrad__ · 2025-12-03 22:29
I can't believe how popular the 4x Blackwell machine is, we have 3 of them bought and paid for! First one should be ready today, will do a thread with some tests.

@jsuarez · 2025-12-03 22:41
@__tinygrad__ ... why? I'm looking at that 8x5090 though

@jsuarez · 2025-12-03 22:41
@__tinygrad__ ... why? I'm looking at that 8x5090 though

@__tinygrad__ · 2025-12-03 22:59
@jsuarez Be prepared for loud and have 200V+ power. Thermals can be compact, good, or quiet: pick 2. We always choose good, and the tinybox pro is a compact 5U.

@__tinygrad__ · 2025-12-03 22:59
@jsuarez Be prepared for loud and have 200V+ power. Thermals can be compact, good, or quiet: pick 2. We always choose good, and the tinybox pro is a compact 5U.

@jsuarez · 2025-12-03 23:13
@__tinygrad__ good and quiet version please

@jsuarez · 2025-12-03 23:13
@__tinygrad__ good and quiet version please

@__tinygrad__ · 2025-12-03 23:29
@jsuarez that's the normal tinybox. 4 cards in 12U = quiet. maybe it does make sense why people buy Blackwell.

@__tinygrad__ · 2025-12-04 00:24
There's 384 GB of fast VRAM in Blackwell tinybox 🧵 https://t.co/sbfGX5p6iJ

@__tinygrad__ · 2025-12-04 00:44
All our Blackwell boxes will be shipping with our latest RAID array. **55.3 GB/s** of benchmarked read bandwidth, which is faster than the RAM on most cell phones. https://t.co/p94pG5cRRN

@__tinygrad__ · 2025-12-04 00:47
While we wait for the gpu-fryer, here's mmapeak. **3.1 PFLOPS** across the cards fp16 -> fp32. Here's where the lack of the 5090's nerfing really shines, it's more than double the raw FLOPS of a tinybox green v2! https://t.co/wPmBNnqjU4

@__tinygrad__ · 2025-12-04 00:47
While we wait for the gpu-fryer, here's mmapeak. **3.1 PFLOPS** across the cards fp16 -> fp32. Here's where the lack of the 5090's nerfing really shines, it's more than double the raw FLOPS of a tinybox green v2! https://t.co/wPmBNnqjU4

@__tinygrad__ · 2025-12-04 00:51
Here it is in the huggingface/gpu-fryer. 2522W at full power, no Max-Q around here! https://t.co/1aFe9wAgaR

@__tinygrad__ · 2025-12-04 00:51
Here it is in the huggingface/gpu-fryer. 2522W at full power, no Max-Q around here! https://t.co/1aFe9wAgaR

@SIGKITTEN · 2025-12-04 00:57
@__tinygrad__ are they still mounted the same way inside or did you have to organize them differently?

@SIGKITTEN · 2025-12-04 00:57
@__tinygrad__ are they still mounted the same way inside or did you have to organize them differently?

@__tinygrad__ · 2025-12-04 00:58
@SIGKITTEN We are still tweaking that. Current fryer temps are "73°C - 81°C - 71°C - 77°C", we can distribute better.

@__tinygrad__ · 2025-12-04 00:51
Here it is in the huggingface/gpu-fryer. 2522W at full power, no Max-Q around here! https://t.co/1aFe9wAgaR

@__tinygrad__ · 2025-12-04 01:06
Final temps at saturation after 15 minutes were 72C, 80C, 71C, and 76C. We're still working on the fan policy and card layout, the coolers are different from what we have worked with before. But the shipping machine will be *at least* this good. https://t.co/DpDQOCeG4t

@__tinygrad__ · 2025-12-04 01:06
Final temps at saturation after 15 minutes were 72C, 80C, 71C, and 76C. We're still working on the fan policy and card layout, the coolers are different from what we have worked with before. But the shipping machine will be *at least* this good. https://t.co/DpDQOCeG4t

@__tinygrad__ · 2025-12-04 01:11
Order yours today! https://t.co/pTkQ44z96o

@__tinygrad__ · 2025-12-03 22:29
I can't believe how popular the 4x Blackwell machine is, we have 3 of them bought and paid for! First one should be ready today, will do a thread with some tests.

@jeremyphoward · 2025-12-04 05:38
@__tinygrad__ I'm surprised you're surprised. Large noisy power-hungry machines with twice as many GPUs to break are pretty obviously not everyone's cup of tea! (Mind you, our tinyboxen are in a colo so all that is someone else's problem...)

@jeremyphoward · 2025-12-04 05:38
@__tinygrad__ I'm surprised you're surprised. Large noisy power-hungry machines with twice as many GPUs to break are pretty obviously not everyone's cup of tea! (Mind you, our tinyboxen are in a colo so all that is someone else's problem...)

@__tinygrad__ · 2025-12-04 08:48
@jeremyphoward Yea the Blackwell really is the best machine you can reasonably use in an apartment. I've been so obsessed with perf/$ that I didn't think about practicality.

@__tinygrad__ · 2025-12-04 19:14
Why are my MI350X GPUs drawing 2 kW at idle? Between the 4 machines this is costing us $3,000 per month! https://t.co/OyHO4C32fI

@__tinygrad__ · 2025-12-04 19:14
Why are my MI350X GPUs drawing 2 kW at idle? Between the 4 machines this is costing us $3,000 per month! https://t.co/OyHO4C32fI

@jtregunna · 2025-12-04 20:03
@__tinygrad__ That's one of the ROCm drivers' issues, power management. It's a bit finicky tbh.

@HotAisle · 2025-12-04 20:32
I bet he didn’t even realize this, until I pointed it out.

@HotAisle · 2025-12-04 20:32
I bet he didn’t even realize this, until I pointed it out.

@__tinygrad__ · 2025-12-04 21:33
@HotAisle We built an office wide dashboard showing power usage of computers and that's when we did the math and saw the cost.

@jtregunna · 2025-12-04 20:03
@__tinygrad__ That's one of the ROCm drivers' issues, power management. It's a bit finicky tbh.

@__tinygrad__ · 2025-12-04 21:34
@jtregunna We're working on adding support to our driver, hopefully the MCLK gate works on the hardware

@__tinygrad__ · 2025-12-05 00:09
Everything at maximum! Audible but happily coexisting in our office during the long burn. 0 AERs. 0 stuttering in the PCIe. The Blackwell tinybox is solid. https://t.co/v23153WBeD

@__tinygrad__ · 2025-12-05 00:12
@TheCoinCollect8 I wish that worked. It doesn't.

@__tinygrad__ · 2025-12-05 00:09
Everything at maximum! Audible but happily coexisting in our office during the long burn. 0 AERs. 0 stuttering in the PCIe. The Blackwell tinybox is solid. https://t.co/v23153WBeD

@SIGKITTEN · 2025-12-05 00:17
@__tinygrad__ nice raid

@__tinygrad__ · 2025-12-05 00:09
Everything at maximum! Audible but happily coexisting in our office during the long burn. 0 AERs. 0 stuttering in the PCIe. The Blackwell tinybox is solid. https://t.co/v23153WBeD

@tkanarsky · 2025-12-05 00:28
@__tinygrad__ nice spaceheater!

@SIGKITTEN · 2025-12-05 00:17
@__tinygrad__ nice raid

@__tinygrad__ · 2025-12-05 00:56
@SIGKITTEN If you are going to go over the top on GPUs, why not go over the top on a RAID array too?

@tkanarsky · 2025-12-05 00:28
@__tinygrad__ nice spaceheater!

@__tinygrad__ · 2025-12-05 00:57
@tkanarsky It's toasty. It has two plugs, so it's probably doubly as good as most space heaters. Will add to marketing copy.

@__tinygrad__ · 2025-12-05 05:35
https://t.co/QmADeppkxT

@hikettei · 2025-12-07 18:26
libtinygrad.dylibみたいなのが切実に欲しいし、TileLangのbackendもTVMじゃなくてtinygrad使える平和な世の中になって欲しい

@LottoLabs · 2025-12-07 18:47
So tinygrad is kinda correct that hardware doesn’t matter

@__tinygrad__ · 2025-12-08 02:32
RT @softwiredtech: A technical overview of the internals of the @__tinygrad__ #webgpu backend. https://t.co/Oxd5ZmeeVm

@madhav1 · 2025-12-10 20:55
this is all you need https://t.co/apwuw6yVIu

@SIGKITTEN · 2025-12-10 21:00
. @__tinygrad__ u got this?

@hikettei · 2025-12-07 18:26
libtinygrad.dylibみたいなのが切実に欲しいし、TileLangのbackendもTVMじゃなくてtinygrad使える平和な世の中になって欲しい

@__tinygrad__ · 2025-12-11 17:17
@hikettei We have a similar frontend to TileLang now if you write in raw UOps

@SIGKITTEN · 2025-12-10 21:00
. @__tinygrad__ u got this?

@__tinygrad__ · 2025-12-11 17:18
@SIGKITTEN You might need a power supply too but pretty much

@__tinygrad__ · 2025-12-11 17:19
The hardware is very simple if you have good software.

@__tinygrad__ · 2025-12-14 02:49
RT @j04win: @ubuntu + @__tinygrad__ = 😍 Revived my 15 years old desktop. i3 CPU 540 https://t.co/bhDv2xVDoW

@sasuke___420 · 2025-12-14 16:22
@CalvinMccarter good

@sasuke___420 · 2025-12-14 16:22
@CalvinMccarter i definitely did not consider tinygrad a "cuda replacement thingy", but the approach to codegen and execution might make it kind of work...

@__tinygrad__ · 2025-12-14 20:14
Victory will take time, but yea, it's a full stack replacement that speaks directly to both NVIDIA and AMD cards.

@SemiAnalysis_ · 2025-12-15 17:33
BREAKING CUDA MOAT EXPANDS: Today, NVIDIA has acquired SchedMD, makers of SLURM, a widely used "open source" workload scheduler. Many AI companies such as Mistral, Thinking Machines, parts of Meta's FAIR division, university academic labs use SLURM. NVIDIA's acquisition expands the CUDA moat to a new layer, as many customers trying to use alternative AI accelerator chips like AMD, Intel, etc. also rely on SLURM (a lot use k8s too). Unlike Run:AI & DGX Lepton, which nobody uses, a lot of people use SLURM. (1\2) 🧵

@dogecahedron · 2025-12-16 14:35
wow @VitalikButerin proposed to follow the lead of @__tinygrad__ and establish goal of max lines of code for ethereum spec

@__tinygrad__ · 2025-12-16 16:54
Software that looks like tinygrad is the future, even more so with agentic coding getting decent. It wasn't written like this in the past because it needed to be factorized so companies could increase headcount so managers could feel important.

@PlisSergey · 2025-12-18 00:46
🚀 a major update to BrainChop is here! We've completely overhauled our engine with a new #WebGPU backend for lightning-fast, privacy-first AI segmentation directly in your browser. 🧠⚡️ Huge thanks to @spikedoanz and @wpmed92 for making this major leap possible! 👇 Link in the reply! Still falls back to #webgl if webGPU is not available. Works best on Chrome-based browsers, but Safari is fine too. #WebGPU #AI #Neuroimaging #HealthTech

@Ric_RTP · 2025-12-18 14:18
Google just launched a direct attack on Nvidia's most valuable asset. Not their chips. Their SOFTWARE. And if this works, Nvidia's $4 trillion empire collapses. Here's what just leaked: Google is building "TorchTPU" - a secret project that makes PyTorch seamlessly run on Google's TPU chips instead of Nvidia GPUs. Why does this matter? PyTorch is the MOST USED AI framework on Earth. Every AI developer uses it. And PyTorch was built around Nvidia's CUDA software. Wall Street analysts call CUDA "Nvidia's strongest defensive wall." It's the reason companies can't easily switch away from Nvidia even when alternatives exist. You don't just buy Nvidia chips. You buy into their entire ecosystem. Switching costs MILLIONS in engineering work. Months of rewrites. Performance drops. So companies stay locked in. Even when Nvidia raises prices. Even when supply runs short. That's not a hardware moat. That's a SOFTWARE prison. And Google just found the escape route. Here's the problem Nvidia created for itself: Google's TPU chips are actually GOOD. Competitive performance. Better availability. Lower cost. But developers won't use them because Google's chips run JAX (Google's internal framework), not PyTorch. That means if you want to use Google TPUs, you have to rewrite your entire codebase. Nobody wants to do that. So Google TPUs sit unused while developers fight over Nvidia chips. Until now. TorchTPU makes PyTorch run natively on Google hardware. No rewrites. No performance loss. No months of engineering. You just... switch. And Google is partnering with META (who built PyTorch) to make it happen. They're even considering OPEN-SOURCING parts of it to speed adoption. Translation: Google is willing to give this away for free just to break Nvidia's lock. The implications are insane: Every company currently paying Nvidia's premium prices suddenly has a way out. Oracle, Microsoft, OpenAI - all locked into Nvidia's ecosystem - can switch to Google. Nvidia's pricing power evaporates overnight. And the timing is perfect: Nvidia is already facing heat. Semiconductor index dropped 3% today. Oracle just lost their biggest investor over AI spending concerns. Companies are realizing AI infrastructure costs are unsustainable. Now Google hands them an alternative. Same performance. Lower cost. Better availability. Jensen Huang knows exactly what this means. CUDA has been Nvidia's untouchable advantage for YEARS. It's why Nvidia trades at 50x earnings while AMD trades at 25x. The software moat justified the premium. But if Google removes that switching cost? Nvidia becomes just another chip company. And chip companies compete on price, not ecosystem lock-in. Here's what happens next: Google needs 12-18 months to make TorchTPU production-ready. If it works, cloud providers will adopt it instantly. They WANT an alternative to Nvidia's monopoly pricing. Amazon already building their own Trainium chips. Microsoft making Maia. They're all trying to escape Nvidia. Google just gave them the software bridge. Nvidia's response options are limited: They can't buy Google. Can't kill PyTorch (Meta owns it). Can't stop open source. Their only play is to keep improving CUDA faster than Google can catch up. But that's a race, not a moat. The market isn't pricing this in yet. Nvidia down 2% today. Google down 2%. Investors think this is just "another competitor." They don't understand this is an attack on the FOUNDATION of Nvidia's valuation. Hardware is replaceable. Software lock-in is what made Nvidia worth $4 trillion. Google is attacking the lock-in. Watch what happens in 2026 when TorchTPU goes live and companies realize they can actually leave Nvidia. The "Nvidia is unstoppable" narrative dies. And a $4 trillion valuation built on software moats gets repriced.

@__tinygrad__ · 2025-12-18 16:31
RT @spikedoanz: @__tinygrad__ 's WEBGPU export has gotten so good that the browser is rapidly becoming my favorite way to deploy ML models. frontend + backend + nn runtime all in one. great stuff. play with it on https://t.co/dkyxRMwqon https://t.co/2efG4GF8O6

@arvidkahl · 2025-12-18 17:01
Is there already such a thing as an "external hardware LLM" like we have external hard drives? Instead of having to run/maintain a local model, I want an inference machine that I can just plug in and point my prompts at. Single GPU, maybe a few in parallel. Who's building this? https://t.co/snA0TH8PN6

@__tinygrad__ · 2025-12-19 11:26
@arvidkahl How much would you pay for what model at what speed?

@thetobiaskrug · 2025-12-19 11:54
@__tinygrad__ @arvidkahl 2k € for a ~GPT5 (medium? like 400B) with 20-50 tokens/s

@thetobiaskrug · 2025-12-19 11:54
@__tinygrad__ @arvidkahl 2k € for a ~GPT5 (medium? like 400B) with 20-50 tokens/s

@__tinygrad__ · 2025-12-19 13:48
@thetobiaskrug @arvidkahl This is the problem with this. That's an order of magnitude off from the cost, and it won't get cheaper you'll just want a bigger model.

@__tinygrad__ · 2025-12-19 13:48
@thetobiaskrug @arvidkahl This is the problem with this. That's an order of magnitude off from the cost, and it won't get cheaper you'll just want a bigger model.

@thetobiaskrug · 2025-12-19 13:51
You didn‘t ask for a realistic pricing, though, but what I‘d like to pay. That‘s 10 months of ChatGPT Pro already, while many people don‘t even run a single paid subscription. (Not me, just giving some guidance on the willingness to pay.) As far as easing access to powerful models is concerned, a mass produced attachment as the OP proposed might work out, provided it‘s being build in the millions. Another dimension of any of these is electricity cost, which countries like Germany aren‘t exactly competitive with for private households. My main query would be, if a super generic GPU cluster is the correct frame of reference in here. Shouldn‘t there be some more specialised hardware, maybe even with models on modules like we had games in the GameBoy days? That should bring prices much more down to earth than a NVLink board for the masses.

@thetobiaskrug · 2025-12-19 13:51
You didn‘t ask for a realistic pricing, though, but what I‘d like to pay. That‘s 10 months of ChatGPT Pro already, while many people don‘t even run a single paid subscription. (Not me, just giving some guidance on the willingness to pay.) As far as easing access to powerful models is concerned, a mass produced attachment as the OP proposed might work out, provided it‘s being build in the millions. Another dimension of any of these is electricity cost, which countries like Germany aren‘t exactly competitive with for private households. My main query would be, if a super generic GPU cluster is the correct frame of reference in here. Shouldn‘t there be some more specialised hardware, maybe even with models on modules like we had games in the GameBoy days? That should bring prices much more down to earth than a NVLink board for the masses.

@__tinygrad__ · 2025-12-19 13:56
@thetobiaskrug @arvidkahl I agree that's what people think they would pay, and that's why this product doesn't exist. All AI is currently subsidized, people will be upset when they are exposed to the real costs.

@__tinygrad__ · 2025-12-19 13:56
@thetobiaskrug @arvidkahl I agree that's what people think they would pay, and that's why this product doesn't exist. All AI is currently subsidized, people will be upset when they are exposed to the real costs.

@thetobiaskrug · 2025-12-19 13:59
@__tinygrad__ @arvidkahl Yeah. Maybe @Google, @amazon or @Tesla have some chips to spare for an affordable Thunderbolt 5 LLM attachment by @__tinygrad__ ? 👀

@__tinygrad__ · 2025-12-19 11:26
@arvidkahl How much would you pay for what model at what speed?

@DeltaouiT · 2025-12-19 14:01
@__tinygrad__ @arvidkahl 2500$ for a local 70B or 32B MoE something like that.

@thetobiaskrug · 2025-12-19 11:54
@__tinygrad__ @arvidkahl 2k € for a ~GPT5 (medium? like 400B) with 20-50 tokens/s

@curlysaarthak · 2025-12-19 14:14
@thetobiaskrug @__tinygrad__ @arvidkahl ~gpt5 medium doesnt exist in oss

@thetobiaskrug · 2025-12-19 13:59
@__tinygrad__ @arvidkahl Yeah. Maybe @Google, @amazon or @Tesla have some chips to spare for an affordable Thunderbolt 5 LLM attachment by @__tinygrad__ ? 👀

@__tinygrad__ · 2025-12-19 14:14
@thetobiaskrug @arvidkahl @Google @amazon @Tesla Chips to spare? Is this a serious product idea or a hobby project?

@curlysaarthak · 2025-12-19 14:14
@thetobiaskrug @__tinygrad__ @arvidkahl ~gpt5 medium doesnt exist in oss

@__tinygrad__ · 2025-12-19 14:19
@curlysaarthak @thetobiaskrug @arvidkahl I think DeepSeek and Kimi are quite similar in quality. You are gonna need like 500GB of memory to run something this good though.

@DeltaouiT · 2025-12-19 14:01
@__tinygrad__ @arvidkahl 2500$ for a local 70B or 32B MoE something like that.

@__tinygrad__ · 2025-12-19 14:23
@DeltaouiT @arvidkahl That's doable, but are you really okay with that sized model? You might think you are, but when you look at revealed preferences everyone goes to the cloud the minute there's anything real.

@__tinygrad__ · 2025-12-19 14:23
@DeltaouiT @arvidkahl That's doable, but are you really okay with that sized model? You might think you are, but when you look at revealed preferences everyone goes to the cloud the minute there's anything real.

@arvidkahl · 2025-12-19 14:26
I wonder what your perspective on this is, as the people working on the cutting edge of this field. Will we see a compression of models so that they would eventually fit on, let's say, half a terabyte of VRAM or maybe even less? Or will thinking models and smart AI always require these massive clusters for the foreseeable 10 years or so?

@arvidkahl · 2025-12-19 14:26
I wonder what your perspective on this is, as the people working on the cutting edge of this field. Will we see a compression of models so that they would eventually fit on, let's say, half a terabyte of VRAM or maybe even less? Or will thinking models and smart AI always require these massive clusters for the foreseeable 10 years or so?

@__tinygrad__ · 2025-12-19 14:29
@arvidkahl @DeltaouiT The brain is 100T params, 10% active, and ~4-bit quantized. So for something to be human level, you'll need like 60TB of RAM.

@__tinygrad__ · 2025-12-19 14:32
Why is RAM expensive? It's literally made of sand.

@__tinygrad__ · 2025-12-19 14:32
Why is RAM expensive? It's literally made of sand.

@arturdolago · 2025-12-19 14:34
@__tinygrad__ Because the machine that makes RAM is not made of sand.

@__tinygrad__ · 2025-12-19 14:32
Why is RAM expensive? It's literally made of sand.

@alakazam1314 · 2025-12-19 14:37
@__tinygrad__ Sand is too small so it is hard to arrange it with tweezers

@arturdolago · 2025-12-19 14:34
@__tinygrad__ Because the machine that makes RAM is not made of sand.

@__tinygrad__ · 2025-12-19 14:37
@arturdolago Have they tried?

@alakazam1314 · 2025-12-19 14:37
@__tinygrad__ Sand is too small so it is hard to arrange it with tweezers

@__tinygrad__ · 2025-12-19 14:44
@alakazam1314 Have they tried?

@__tinygrad__ · 2025-12-19 14:32
Why is RAM expensive? It's literally made of sand.

@sorek_UK · 2025-12-19 14:59
@__tinygrad__ Why can't we print money? It's literary made out of paper!

@sorek_UK · 2025-12-19 14:59
@__tinygrad__ Why can't we print money? It's literary made out of paper!

@__tinygrad__ · 2025-12-19 15:10
@sorek_UK Umm, you can, it's just illegal.

@beaversteever · 2025-12-19 15:13
OpenAI made deals to purchase ~40% of the global raw, undiced DRAM wafer output until 2029 millions of raw DRAM wafers that cannot be used until they're processed

@__tinygrad__ · 2025-12-19 14:32
Why is RAM expensive? It's literally made of sand.

@Faisal942x · 2025-12-19 15:27
@__tinygrad__ Why is tinybox expensive? It's literally made of regular PC components.

@__tinygrad__ · 2025-12-19 14:32
Why is RAM expensive? It's literally made of sand.

@arye321 · 2025-12-19 15:45
@__tinygrad__ Sand prices btw https://t.co/GaBWL7ec4n

@__tinygrad__ · 2025-12-19 14:32
Why is RAM expensive? It's literally made of sand.

@itsmrcallum · 2025-12-19 16:08
@__tinygrad__ Why is my Gucci sweater expensive, its literally made of cotton.

@__tinygrad__ · 2025-12-19 14:32
Why is RAM expensive? It's literally made of sand.

@Steve3Chang · 2025-12-19 16:28
@__tinygrad__ @_Stocko_ I asked the same about diamonds. It’s just carbon :/

@arye321 · 2025-12-19 15:45
@__tinygrad__ Sand prices btw https://t.co/GaBWL7ec4n

@__tinygrad__ · 2025-12-19 16:29
@arye321 ahhhh we found the problem

@Steve3Chang · 2025-12-19 16:28
@__tinygrad__ @_Stocko_ I asked the same about diamonds. It’s just carbon :/

@__tinygrad__ · 2025-12-19 16:30
@Steve3Chang @_Stocko_ I think diamonds are actually cheap there's just a monopoly trying to convince you they aren't.

@itsmrcallum · 2025-12-19 16:08
@__tinygrad__ Why is my Gucci sweater expensive, its literally made of cotton.

@__tinygrad__ · 2025-12-19 16:31
@itsmrcallum If there was Chinese knockoff RAM that's 100x cheaper I'd buy that in a minute. I don't think it's the same reason.

@Faisal942x · 2025-12-19 15:27
@__tinygrad__ Why is tinybox expensive? It's literally made of regular PC components.

@__tinygrad__ · 2025-12-19 16:32
@FaisalFailed Our margins are 20-30%. The magic sand margins are millions of percent.

@__tinygrad__ · 2025-12-19 11:26
@arvidkahl How much would you pay for what model at what speed?

@foodiskey83 · 2025-12-19 16:36
@__tinygrad__ @arvidkahl Hardware capable of running OSS 120B at ~75 tokens per second for somewhere in the region of $2,500–$3,500 would be a game changer. Some form of single-board setup with huge unified RAM must be on the horizon.

@foodiskey83 · 2025-12-19 16:36
@__tinygrad__ @arvidkahl Hardware capable of running OSS 120B at ~75 tokens per second for somewhere in the region of $2,500–$3,500 would be a game changer. Some form of single-board setup with huge unified RAM must be on the horizon.

@__tinygrad__ · 2025-12-19 16:39
@foodiskey83 @arvidkahl I think that's doable today with a 128GB Mac Studio or maybe a Strix Halo. Hardware should work, does the software not?

@__tinygrad__ · 2025-12-19 16:39
@foodiskey83 @arvidkahl I think that's doable today with a 128GB Mac Studio or maybe a Strix Halo. Hardware should work, does the software not?

@foodiskey83 · 2025-12-19 16:49
@__tinygrad__ @arvidkahl Yeah, but more a neatly packaged, ruggedised plug-and-play module specifically designed to crunch numbers and be portable.

@foodiskey83 · 2025-12-19 16:49
@__tinygrad__ @arvidkahl Yeah, but more a neatly packaged, ruggedised plug-and-play module specifically designed to crunch numbers and be portable.

@__tinygrad__ · 2025-12-19 16:56
@foodiskey83 @arvidkahl Perhaps...a MacBook Pro?

@beaversteever · 2025-12-19 15:13
OpenAI made deals to purchase ~40% of the global raw, undiced DRAM wafer output until 2029 millions of raw DRAM wafers that cannot be used until they're processed

@__tinygrad__ · 2025-12-19 18:21
@beaversteever Hmm that's a big sandcastle

@__tinygrad__ · 2025-12-19 14:32
Why is RAM expensive? It's literally made of sand.

@ESd146 · 2025-12-19 20:06
@__tinygrad__ Why is gold so expensive? It's literally a shiny rock

@ESd146 · 2025-12-19 20:06
@__tinygrad__ Why is gold so expensive? It's literally a shiny rock

@__tinygrad__ · 2025-12-19 20:11
@ESd146 Cause it's made of gold! You know how hard it is to make gold? It takes supernovas or something. RAM is literally just sand.

@sergeax · 2025-12-20 11:42
I really don't understand two things: 1) why Google didn't do it like 5 years ago? 2) why AMD didn't do it even earlier?

@kpertsev · 2025-12-20 12:33
@sergeax and it started to happen. @HotAisle @SpectralCom et. al.

@sergeax · 2025-12-20 12:50
@kpertsev @HotAisle @SpectralCom Also @__tinygrad__ tried, but failed, AFAIR. Which is outrageous, because AMD should throw all their weight towards that goal.

@sergeax · 2025-12-20 12:50
@kpertsev @HotAisle @SpectralCom Also @__tinygrad__ tried, but failed, AFAIR. Which is outrageous, because AMD should throw all their weight towards that goal.

@__tinygrad__ · 2025-12-20 12:54
@sergeax @kpertsev @HotAisle @SpectralCom Umm, why do you think we failed? We have a contract with AMD and are working on large LLM training.

@sergeax · 2025-12-20 12:50
@kpertsev @HotAisle @SpectralCom Also @__tinygrad__ tried, but failed, AFAIR. Which is outrageous, because AMD should throw all their weight towards that goal.

@kpertsev · 2025-12-20 12:56
@sergeax @HotAisle @SpectralCom @__tinygrad__ tinygrad does their own thing, spectral makes “cuda on amd” which is better transition imho.

@__tinygrad__ · 2025-12-20 12:54
@sergeax @kpertsev @HotAisle @SpectralCom Umm, why do you think we failed? We have a contract with AMD and are working on large LLM training.

@kpertsev · 2025-12-20 12:57
@__tinygrad__ @sergeax @HotAisle @SpectralCom i think he means “cuda on amd”, not failed as a company.

@kpertsev · 2025-12-20 12:56
@sergeax @HotAisle @SpectralCom @__tinygrad__ tinygrad does their own thing, spectral makes “cuda on amd” which is better transition imho.

@__tinygrad__ · 2025-12-20 12:59
@kpertsev @sergeax @HotAisle @SpectralCom You can't make CUDA on AMD, this idea is nonsense and should be shot down every time it comes up. It will never be fast. Kernels for the H100 aren't good on the B200 and that's both CUDA, why do you think you can bridge to AMD?

@kpertsev · 2025-12-20 12:57
@__tinygrad__ @sergeax @HotAisle @SpectralCom i think he means “cuda on amd”, not failed as a company.

@__tinygrad__ · 2025-12-20 13:00
@kpertsev @sergeax @HotAisle @SpectralCom We never did "CUDA on AMD" this idea is really dumb and should be treated as such.

@iamgingertrash · 2025-12-20 22:03
They’re attempting to price you out of local compute Dependent on the cloud, Forever and ever and ever and ever and..

@__tinygrad__ · 2025-12-20 22:43
Commoditize the petaflop!

@__tinygrad__ · 2025-12-20 22:43
Commoditize the petaflop!

@beaversteever · 2025-12-20 23:35
@__tinygrad__ put some crazy verilog bounties so we can ball a bit

@beaversteever · 2025-12-20 23:35
@__tinygrad__ put some crazy verilog bounties so we can ball a bit

@__tinygrad__ · 2025-12-21 00:42
@beaversteever haha GPU assembly first

@SemiAnalysis_ · 2025-12-22 04:39
SemiAnalysis has long advocated AMD spend less on buybacks and more on investing in the ROCm OSS ecosystem. We now issue the exact opposite plea to Nvidia: please resume aggressive buybacks and stop buying OSS projects and ruining them. https://t.co/MYe3gQyP6Y

@oguzerkan · 2025-12-22 06:13
Peter Thiel says the quiet part loud: AI chips will get commoditized. This is why I avoid $NVDA. It made an immense amount of money marking up its GPUs by 10x and others had to pay as alternatives weren’t good enough. Now they are getting good enough. $AMD has already caught up with $NVDA in hardware performance, and ASICS are proving unexpectedly competitive due to their efficiency. $GOOG trained Gemini 3 solely on TPUs, and Anthropic is running a significant part of its training and inference workloads on Trainium clusters. As alternatives become even stronger, $NVDA margins and volumes will erode, and profits will shift from hardware to the application layer.

@mlemmlem396092 · 2025-12-22 14:41
every year for a day or two I get a feeling I should quit my job and try to learn @__tinygrad__ every day the whole day I never do tho...

@__tinygrad__ · 2025-12-22 15:02
It's not that long, just 18,500 lines. Sit down with Opus 4.5 and you'll get it after the end of a solid weekend.

@__tinygrad__ · 2025-12-22 15:02
It's not that long, just 18,500 lines. Sit down with Opus 4.5 and you'll get it after the end of a solid weekend.

@jsuarez · 2025-12-22 15:49
@__tinygrad__ what's with the opus stuff lately. Why not just read the docs + source

@__tinygrad__ · 2025-12-22 15:49
RT @ninoristeski: @mlemmlem396092 @__tinygrad__ for the holidays: How I Started Contributing to tinygrad — My First 4 Merged PRs https://t.co/V56aRA07VM

@jsuarez · 2025-12-22 15:49
@__tinygrad__ what's with the opus stuff lately. Why not just read the docs + source

@__tinygrad__ · 2025-12-22 16:03
@jsuarez I feel like docs are becoming obsolete. Reading the source has always been the gold standard, just now the LLMs can do it. Since tinygrad is so short it's easy to fit in context windows, and LLMs will answer any questions you have better than docs.

@__tinygrad__ · 2025-12-22 22:00
As we move towards outputting GPU assembly it's important to have a good visualizer. With VIZ=1 you can now see the assembly broken into basic blocks. https://t.co/CHT3iMU2wP

@SemiAnalysis_ · 2025-12-22 04:39
SemiAnalysis has long advocated AMD spend less on buybacks and more on investing in the ROCm OSS ecosystem. We now issue the exact opposite plea to Nvidia: please resume aggressive buybacks and stop buying OSS projects and ruining them. https://t.co/MYe3gQyP6Y

@__tinygrad__ · 2025-12-23 01:10
@SemiAnalysis_ lol agreed

@oguzerkan · 2025-12-22 06:13
Peter Thiel says the quiet part loud: AI chips will get commoditized. This is why I avoid $NVDA. It made an immense amount of money marking up its GPUs by 10x and others had to pay as alternatives weren’t good enough. Now they are getting good enough. $AMD has already caught up with $NVDA in hardware performance, and ASICS are proving unexpectedly competitive due to their efficiency. $GOOG trained Gemini 3 solely on TPUs, and Anthropic is running a significant part of its training and inference workloads on Trainium clusters. As alternatives become even stronger, $NVDA margins and volumes will erode, and profits will shift from hardware to the application layer.

@__tinygrad__ · 2025-12-23 04:14
@oguzerkan The petaflop will be commoditized you say?

@__tinygrad__ · 2025-12-22 22:00
As we move towards outputting GPU assembly it's important to have a good visualizer. With VIZ=1 you can now see the assembly broken into basic blocks. https://t.co/CHT3iMU2wP

@FatRustDev · 2025-12-23 10:43
@__tinygrad__ why web slop and not an immediate mode GUI

@__tinygrad__ · 2025-12-23 18:44
@nicstranger At the end of the day you need to run on classical hardware, it's dataflow for as long as it can be.

@FatRustDev · 2025-12-23 10:43
@__tinygrad__ why web slop and not an immediate mode GUI

@__tinygrad__ · 2025-12-23 22:57
@FatRustDev it's included in tinygrad's line count and very lightweight. no react or any junk like that, and it starts super fast

@jenzhuscott · 2025-12-26 04:20
When @Apple wakes up to decentralised local AI, it will be the most undervalued big tech. https://t.co/E2vnjc7cSK

@DavidSHolz · 2025-12-26 04:35
who are the best vibe coders in the world right now? what are the most impressive vibe coded projects out there which aren't just something done in a single day or two?

@__tinygrad__ · 2025-12-26 16:52
@jenzhuscott @Apple It won't. What's the advantage of decentralized? All the time I see people shilling for this but then you look at their revealed preferences and it's ChatGPT

@FatRustDev · 2025-12-26 16:56
@__tinygrad__ @jenzhuscott @Apple You can do roleplay without anyone looking at your chat https://t.co/znvQ5z7lSB

@FatRustDev · 2025-12-26 16:56
@__tinygrad__ @jenzhuscott @Apple You can do roleplay without anyone looking at your chat https://t.co/znvQ5z7lSB

@__tinygrad__ · 2025-12-26 17:02
@FatRustDev @jenzhuscott @Apple This is the only real use I have seen. Buy a tinybox if you want to roleplay with large models!

@BerlatiGonzalo · 2025-12-28 00:15
hey @__tinygrad__! how far away are we from running on OrangePi 6? 28.8 TOPS at 200 bucks

@__tinygrad__ · 2025-12-29 18:47
RT @cjpgverrier: I've finally decided to spend some time studying @__tinygrad__ codebase. I've been impressed by the elegance of their pattern matcher, so I wrote some notes and turned them into a tuto: https://t.co/6uwEJU99iR The more I read the code, the more I think it's the future of software: anti-bloating and very close to the hardware.

@__tinygrad__ · 2025-12-29 18:47
RT @geohotarchive: Five years of tinygrad https://t.co/yeWCZbtjtv

@BerlatiGonzalo · 2025-12-28 00:15
hey @__tinygrad__! how far away are we from running on OrangePi 6? 28.8 TOPS at 200 bucks

@__tinygrad__ · 2025-12-29 19:27
@BerlatiGonzalo I bet Claude could write a backend for it

@DavidSHolz · 2025-12-26 04:35
who are the best vibe coders in the world right now? what are the most impressive vibe coded projects out there which aren't just something done in a single day or two?

@__tinygrad__ · 2025-12-29 19:28
@DavidSHolz Here's a vibe coded chip reverse engineered, a custom 8051 with PCIe and USB3 support. https://t.co/i5Cb1tN3VF

@__tinygrad__ · 2025-12-30 17:12
RT @AbdMuizAdeyemo: @cjpgverrier @__tinygrad__ Absolutely, tinygrad is a gem. Its simplicity and closeness to hardware is a masterclass in efficient design. Reading and dissecting it is like peeking into the future of lean, high-performance software.

@delicht0 · 2025-12-31 14:23
The ultimate litmus test for Opus 4.5 is completing a @__tinygrad__ bounty from start and to finish

@__tinygrad__ · 2025-12-31 15:12
When this happens, it will signal the end of tinygrad bounties. The value today is shifting away from writing the code and towards defining the task and validating it.

@jsuarez · 2025-12-31 15:50
Since @karpathy and @__tinygrad__ are both up on the latest models, I am going to stream a genuine effort to get usable work out of one of them. I am going to be setting it up for success as much as possible and expect it to fail spectacularly. It will be in the next few days and I will announce the night before. 1. The task will be as standalone as possible, not requiring much interaction with the rest of my work on PufferLib. I have a new environment in mind that would take me a solid weekend to prototype manually. 2. The target implementation is high-performance C. Nobody worth their salt is "falling behind" if the models are limited to web slop and Python. That's not what I'm testing. 3. I will be taking usage suggestions from chat. I don't want to hear any more bullshit about "prompting wrong" if it fails. 4. Code quality still matters. If you think that people shouldn't be reading code, you may leave tech by the nearest available window with all possible haste. 5. I have been recommended Opus. Every damn time I try a model, fanboys of the other equally crap ones tell me I should have gone with their special snowflake. So this time, I'm asking ahead: If I do this with claude code and Opus, can we put this to rest finally?

@jsuarez · 2025-12-31 15:50
Since @karpathy and @__tinygrad__ are both up on the latest models, I am going to stream a genuine effort to get usable work out of one of them. I am going to be setting it up for success as much as possible and expect it to fail spectacularly. It will be in the next few days and I will announce the night before. 1. The task will be as standalone as possible, not requiring much interaction with the rest of my work on PufferLib. I have a new environment in mind that would take me a solid weekend to prototype manually. 2. The target implementation is high-performance C. Nobody worth their salt is "falling behind" if the models are limited to web slop and Python. That's not what I'm testing. 3. I will be taking usage suggestions from chat. I don't want to hear any more bullshit about "prompting wrong" if it fails. 4. Code quality still matters. If you think that people shouldn't be reading code, you may leave tech by the nearest available window with all possible haste. 5. I have been recommended Opus. Every damn time I try a model, fanboys of the other equally crap ones tell me I should have gone with their special snowflake. So this time, I'm asking ahead: If I do this with claude code and Opus, can we put this to rest finally?

@__tinygrad__ · 2026-01-01 20:07
@jsuarez @karpathy Prompting wrong is nonsense, high performance C is fine, and Opus is the right choice, it actually is better. The code quality is meh, but unlike previous agents it can improve it if you complain and tell it how to do it better. Reading code is absolutely still necessary.

@__tinygrad__ · 2026-01-01 20:20
RT @__dxu: finished a fun end of year project: implementing FFT and LoRA in tinygrad: https://t.co/1fTgSJo4Xk

@bautigarcia · 2026-01-04 14:40
One of my conclusions from today is that the slowness in the USER stage for Tensor.uniform() calls in @__tinygrad__ comes from the amount of chained methods involved (and each call also adding some profiling/metadata overhead via __wrapper__). https://t.co/EMank4J7QE

@__tinygrad__ · 2026-01-05 00:05
wanna speed up the Python?

@__tinygrad__ · 2026-01-05 19:42
So I didn't realize how actually crazy expensive RAM is now, and this raises GPU prices too. All tinybox prices went up a little to cover costs. For outstanding orders we will honor the prices they got in at, lucky ducks!

@__tinygrad__ · 2026-01-05 19:42
So I didn't realize how actually crazy expensive RAM is now, and this raises GPU prices too. All tinybox prices went up a little to cover costs. For outstanding orders we will honor the prices they got in at, lucky ducks!

@justfly1984 · 2026-01-05 19:47
@__tinygrad__ 640 kb of ram is all the user needs

@__tinygrad__ · 2026-01-05 19:42
So I didn't realize how actually crazy expensive RAM is now, and this raises GPU prices too. All tinybox prices went up a little to cover costs. For outstanding orders we will honor the prices they got in at, lucky ducks!

@KinvertOG · 2026-01-05 19:54
@__tinygrad__ lol was about to buy a pro v2 but wife wanted to double check the tax accounting etc. complicated tax crap has wonderful 2nd order effects.

@KinvertOG · 2026-01-05 19:54
@__tinygrad__ lol was about to buy a pro v2 but wife wanted to double check the tax accounting etc. complicated tax crap has wonderful 2nd order effects.

@__tinygrad__ · 2026-01-05 19:58
@KinvertOG The pricing on that one might still be too low with the rising price of 5090s. All complaints about the RAM price can be directed to @sama

@__tinygrad__ · 2026-01-05 19:42
So I didn't realize how actually crazy expensive RAM is now, and this raises GPU prices too. All tinybox prices went up a little to cover costs. For outstanding orders we will honor the prices they got in at, lucky ducks!

@elektronikdengi · 2026-01-05 20:16
@__tinygrad__ >sell tinybox for the GPU middle class its upper middle class now

@__tinygrad__ · 2026-01-05 23:55
We space out the PRO cards differently to respect their professional airflow patterns. https://t.co/tsFYuTz8hy

@elektronikdengi · 2026-01-05 20:16
@__tinygrad__ >sell tinybox for the GPU middle class its upper middle class now

@__tinygrad__ · 2026-01-05 23:56
@elektronikdengi 😂

@justfly1984 · 2026-01-05 19:47
@__tinygrad__ 640 kb of ram is all the user needs

@__tinygrad__ · 2026-01-05 23:59
@justfly1984 ROCm was afaik the first Python package over 4GB. It required a pip upgrade to work.

@__tinygrad__ · 2026-01-05 23:55
We space out the PRO cards differently to respect their professional airflow patterns. https://t.co/tsFYuTz8hy

@KinvertOG · 2026-01-06 00:19
@__tinygrad__ Bro posting thirst traps. Is that a Noctua fan?

@__tinygrad__ · 2026-01-05 23:55
We space out the PRO cards differently to respect their professional airflow patterns. https://t.co/tsFYuTz8hy

@emm0sh · 2026-01-06 16:42
i’m begging you at this point to add hardware bounties so you can stop guessing and start engineering, and it isn’t even about performance many other things happen too by starting this: 1. you crowdsource the tedious problems like CAD and BOM mamagement 2. you enable multi-sourcing components because people get access to #1 3. you don’t want to hire mechanical and process engineers. you can crowdsource a lot of this work 4. you enable testing. your reduce your return rate. comma 1 or 2 was something like 25% IIRC, and you got that down, but it took a team and time. this is higher risk because of higher shipping costs and COGS the discord isn’t suited to letting us help at this point. add bounties and let us in. this is going to be a hardware problem too

@emm0sh · 2026-01-06 16:42
i’m begging you at this point to add hardware bounties so you can stop guessing and start engineering, and it isn’t even about performance many other things happen too by starting this: 1. you crowdsource the tedious problems like CAD and BOM mamagement 2. you enable multi-sourcing components because people get access to #1 3. you don’t want to hire mechanical and process engineers. you can crowdsource a lot of this work 4. you enable testing. your reduce your return rate. comma 1 or 2 was something like 25% IIRC, and you got that down, but it took a team and time. this is higher risk because of higher shipping costs and COGS the discord isn’t suited to letting us help at this point. add bounties and let us in. this is going to be a hardware problem too

@__tinygrad__ · 2026-01-06 19:58
@emm0sh The problem is evaluation, it needs to be automated. With software, there's tests and CI. What do you do for these bounties?

@comma_ai · 2026-01-06 20:01
At CES? Drop by the booth and say hi LVCC West Hall 3306 https://t.co/ZCJbwYhDwy

@__tinygrad__ · 2026-01-06 20:02
@atomalom Massive waste of time. It's like the physicist who can predict the horse race if you assume the horses are perfect spheres. Just get good tests and experiment.

@__tinygrad__ · 2026-01-06 19:58
@emm0sh The problem is evaluation, it needs to be automated. With software, there's tests and CI. What do you do for these bounties?

@emm0sh · 2026-01-06 20:25
the wrong answer: design reviews. these are slow and committee’d to death. it’s important to know what wrong looks like here to steer the answer correctly the right answer: clear and definitive requirements set by a single dictator. use requirements as the API. even if you set bad ones, the crowd will correct them by either pushing back in unison or learning in unison example: “GPU installation shall have a cycle time of 18 minutes with a CPK of 1.33” is the CPK .97? rejected. try again is the definition not clear enough? requirement is discussed by the crowd and a clarification request is submitted directly to the requirement definition (github probably) if the requirements are dumb, the crowd votes by ignoring or clarifying them this is a format that doesn’t waste the tiny team’s time by utilizing a single contact point

@emm0sh · 2026-01-06 20:25
the wrong answer: design reviews. these are slow and committee’d to death. it’s important to know what wrong looks like here to steer the answer correctly the right answer: clear and definitive requirements set by a single dictator. use requirements as the API. even if you set bad ones, the crowd will correct them by either pushing back in unison or learning in unison example: “GPU installation shall have a cycle time of 18 minutes with a CPK of 1.33” is the CPK .97? rejected. try again is the definition not clear enough? requirement is discussed by the crowd and a clarification request is submitted directly to the requirement definition (github probably) if the requirements are dumb, the crowd votes by ignoring or clarifying them this is a format that doesn’t waste the tiny team’s time by utilizing a single contact point

@__tinygrad__ · 2026-01-06 20:32
@emm0sh Huh? This all sounds like a lot to manage. In the era of LLM coders, even software bounties are become less tenable. Once you have clear requirements, that's most of the job done.

@KinvertOG · 2026-01-06 00:19
@__tinygrad__ Bro posting thirst traps. Is that a Noctua fan?

@__tinygrad__ · 2026-01-06 20:38
@KinvertOG All the fans that aren't built in to GPUs and PSUs are Noctua. We don't cheap out on fans.

@__tinygrad__ · 2026-01-06 20:42
ft tinybox

@__tinygrad__ · 2026-01-06 20:32
@emm0sh Huh? This all sounds like a lot to manage. In the era of LLM coders, even software bounties are become less tenable. Once you have clear requirements, that's most of the job done.

@emm0sh · 2026-01-06 20:45
@__tinygrad__ welcome to hardware. i told it to you as simply as it can be stated

@DanielcHooper · 2026-01-06 22:08
My C programmer thoughts on Claude+Opus 4.5: 1. Bad at writing code: wrote O(n²) algorithm when O(n) possible. I wouldn't commit its code without review. 2. Responds well to feedback "make this algorithm linear". You have to already be a good programmer to know how the code could be improved. 3. Useful for analysis: "How could this system get into <some state>?" 4. Helps get over procrastination on grindy tasks, like creating a linux sysroot for cross compilation. 5. Makes it cheaper to try different approaches: "change the memory layout to X and use data structure Y, run performance test and compare" 6. Running 1 or more agents in the background while I do other work feels like a superpower. 7. Best when treated like a lawyer's paralegal: you do big brain planning, it does tedium in background, you review, tweak, commit.

@__tinygrad__ · 2026-01-07 01:09
loving the density and the double wide https://t.co/HnKmVxwFqy

@emm0sh · 2026-01-06 20:45
@__tinygrad__ welcome to hardware. i told it to you as simply as it can be stated

@__tinygrad__ · 2026-01-07 01:47
@emm0sh We make hardware. It's a lot simpler than software.

@DanielcHooper · 2026-01-06 22:08
My C programmer thoughts on Claude+Opus 4.5: 1. Bad at writing code: wrote O(n²) algorithm when O(n) possible. I wouldn't commit its code without review. 2. Responds well to feedback "make this algorithm linear". You have to already be a good programmer to know how the code could be improved. 3. Useful for analysis: "How could this system get into <some state>?" 4. Helps get over procrastination on grindy tasks, like creating a linux sysroot for cross compilation. 5. Makes it cheaper to try different approaches: "change the memory layout to X and use data structure Y, run performance test and compare" 6. Running 1 or more agents in the background while I do other work feels like a superpower. 7. Best when treated like a lawyer's paralegal: you do big brain planning, it does tedium in background, you review, tweak, commit.

@__tinygrad__ · 2026-01-07 01:54
@DanielcHooper Agreed with most of these points. Eventually I think you even stop with the other background agents, if you tightly supervise it you can get very high quality code. I want one faster Claude not more Claudes. @cerebras you get?

@__tinygrad__ · 2026-01-08 00:49
RT @cjpgverrier: As I like GPUs and tinygrad more and more, I wanted to better understand how to properly benchmark kernel runtimes, so I wrote a few notes: https://t.co/zs1Bpb1dGJ Hope it helps!

@tetsuo_cpp · 2026-01-08 12:17
Oh, you're writing CUDA kernels? Everyone's on Triton now. Just kidding, we're all on Mojo. We're using cuTile. We're using ROCm. We have an in-house DSL compiler targeting the NVGPU MLIR dialect but wait, Tile IR just dropped so we're going to target that instead. Our PM is on TileLang. The team lead was on CuTe but now she's back to handwriting PTX. If you're not on Pallas, you're ngmi. Our intern is building on TT-Metalium for our Wormholes. Our CFO approved an order for some big chungus wafer-scale chips so now we're porting our kernels to CSL. Our CTO is working on a kernel-less graph compiler so we won't need to write kernels anymore. Our CEO thinks we're talking about the Linux kernel. We're building Claude for dogs.

@__tinygrad__ · 2026-01-08 20:21
we are coding in machine code, it's timeless

@__tinygrad__ · 2026-01-08 20:21
we are coding in machine code, it's timeless

@HDPbilly · 2026-01-08 20:52
@__tinygrad__ are you really? or... just like an analogy here..

@__tinygrad__ · 2026-01-08 20:21
we are coding in machine code, it's timeless

@thecsguy · 2026-01-08 21:27
@__tinygrad__ 40 years of abstractions just to realize the silicon was right all along

@thecsguy · 2026-01-08 21:27
@__tinygrad__ 40 years of abstractions just to realize the silicon was right all along

@__tinygrad__ · 2026-01-09 02:54
@thecsguy the silicon is always right

@HDPbilly · 2026-01-08 20:52
@__tinygrad__ are you really? or... just like an analogy here..

@__tinygrad__ · 2026-01-09 02:56
@HDPbilly the new AMD stuff in tinygrad outputs machine code. no deps, no abstractions, just metal

@boopdotpng · 2026-01-11 01:57
so the way forward is apparently to write SFPU c++ directly into the tt-metal kernel, bypassing tt-llk entirely that way a compiler can generate without relying on tt-llk

@amitavkrshna · 2026-01-11 09:18
Working on tinygrad-verilog on the side, here's my first devlog! https://t.co/j5Xeukgvtp Watch it, please. GitHub in the description of the video. So far, implemented a layer of a neural network. Good night!

@OfficialPCMR · 2026-01-11 17:00
Maybe 5090 was the price after all! https://t.co/RmZ0GniN2C https://t.co/HkiFtOrXtk

@__tinygrad__ · 2026-01-12 02:51
RT @fangpenlin: Built and open-sourced tinyloader yesterday https://t.co/zeIhpwnWPY For loading data to tinygrad in background workers with shared memory. Mostly for my own project use, others may find it useful so I open sourced it. 😁

@__tinygrad__ · 2026-01-12 03:22
Would love to see some HDL backends for tinygrad. An in-order VLIW core can have great performance and you can probably vibe code it. This year LLVM gets removed from tinygrad, next year we do our hardware.

@neomewtwo · 2026-01-12 01:19
@boopdotpng what are you trying to do?

@boopdotpng · 2026-01-12 11:34
@neomewtwo ideally, get kernels to run on the card using only tt-kmd and Python that way its an easy port to tinygrad i started writing a driver https://t.co/9oO8KHcX8J

@__tinygrad__ · 2026-01-12 23:24
tinygrad 0.12.0 is out! No more ShapeTracker, AMD wave visualizer with VIZ=2, NVIDIA support without CUDA, and MI300/MI350 driver. https://t.co/df84HVaaGa

@__tinygrad__ · 2026-01-12 23:28
RT @phoronix: Tinygrad @__tinygrad__ 0.12 Released With Mesa NIR/NAK Support With the Mesa NIR route, the option of a fully open-source software stack atop NVIDIA hardware using the Mesa NVK driver. https://t.co/eMMWi7yUxV

@boopdotpng · 2026-01-12 11:34
@neomewtwo ideally, get kernels to run on the card using only tt-kmd and Python that way its an easy port to tinygrad i started writing a driver https://t.co/9oO8KHcX8J

@__tinygrad__ · 2026-01-13 00:50
@boopdotpng @neomewtwo This is the way. Their stack is way too complex to even compile. We have some really nice helpers for PCIe that we use for our drivers, as a bonus if you use them your card will work over USB4 on a Mac.

@jimbelosic · 2026-01-13 00:50
Hoping that bananametals makes enough money to buy us another saw to cut more metal to make more money to buy another saw

@comma_ai · 2026-01-13 00:58
Hoping that comma four makes enough money to buy another rack of GPUs to train better models to sell more comma fours to buy another rack of GPUs

@__tinygrad__ · 2026-01-12 23:24
tinygrad 0.12.0 is out! No more ShapeTracker, AMD wave visualizer with VIZ=2, NVIDIA support without CUDA, and MI300/MI350 driver. https://t.co/df84HVaaGa

@zolotukhin_ai · 2026-01-13 03:20
@__tinygrad__ We need R9700 support

@zolotukhin_ai · 2026-01-13 03:20
@__tinygrad__ We need R9700 support

@__tinygrad__ · 2026-01-13 03:36
@StepanZolotukh1 Have you tried? Should just work.

@HotAisle · 2026-01-13 03:54
woo! mi300 driver! =)

@__tinygrad__ · 2026-01-13 03:56
@HotAisle Yea, no more reboots needed now.

@HotAisle · 2026-01-13 04:00
@__tinygrad__ great work. amd help you figure that out or did you get it on your own?

@comma_ai · 2026-01-13 00:58
Hoping that comma four makes enough money to buy another rack of GPUs to train better models to sell more comma fours to buy another rack of GPUs

@__tinygrad__ · 2026-01-13 04:00
@comma_ai The funnel of $$$ to NVIDIA continues

@HotAisle · 2026-01-13 04:00
@__tinygrad__ great work. amd help you figure that out or did you get it on your own?

@__tinygrad__ · 2026-01-13 04:01
@HotAisle Nothing secret from AMD, just their kernel driver and public docs.

@phoronix · 2026-01-12 17:40
Tinygrad @__tinygrad__ 0.12 Released With Mesa NIR/NAK Support With the Mesa NIR route, the option of a fully open-source software stack atop NVIDIA hardware using the Mesa NVK driver. https://t.co/eMMWi7yUxV

@__tinygrad__ · 2026-01-13 04:05
@phoronix We're soon going to support Mesa for Qualcomm too. It's wonderful how much they have reverse engineered and support.

@__tinygrad__ · 2026-01-14 02:22
RT @diogocapela: Tips from @__tinygrad__ CLAUDEmd - Don't add complexity for marginal performance gains - Simpler code that's slightly slower is often better - Only optimize when profiling shows a real bottleneck Readability > micro-optimizations https://t.co/ABmSLlwqXI

@__tinygrad__ · 2026-01-14 02:25
Unfortunately all our 5090 tinyboxes are out of stock until future notice. red and Blackwell still good!

@__tinygrad__ · 2026-01-14 23:11
RT @yassineyousfi_: a not so tiny upgrade @__tinygrad__ https://t.co/Oo4KielMrh

@jxmnop · 2026-01-15 19:03
I have a few qualms with the OpenAI API: For a Linux user, you can already build such a system yourself quite trivially by buying a 4xH100 box, installing it at home, installing CUDA and vLLM locally, and running GLM, Kimi, or a comparable open-source model. With typical consumer workloads, you should expect higher TPS for a fraction of the cost.

@drewhouston · 2026-01-15 19:44
@jxmnop Can confirm https://t.co/8OPdWHyazT

@__tinygrad__ · 2026-01-15 22:59
tinybox spotted!

@__tinygrad__ · 2026-01-16 03:14
With the rising price of 5090s and RAM, the Blackwell cards are looking like a better and better deal. Get your Blackwell tinybox before NVIDIA realizes and raises the price!

@__tinygrad__ · 2026-01-16 03:14
With the rising price of 5090s and RAM, the Blackwell cards are looking like a better and better deal. Get your Blackwell tinybox before NVIDIA realizes and raises the price!

@mov_axbx · 2026-01-16 03:51
@__tinygrad__ I built a 4x one but folks should just buy yours https://t.co/2gwk4U2uh2

@mov_axbx · 2026-01-16 03:51
@__tinygrad__ I built a 4x one but folks should just buy yours https://t.co/2gwk4U2uh2

@__tinygrad__ · 2026-01-16 04:12
@mov_axbx Buy vs build always comes down to whether you enjoy tinkering with the thing or just using the thing. If you enjoy the building and tinkering, by all means do it!

@__tinygrad__ · 2026-01-16 21:56
tinygrad's SQTT parser does things rocprof-trace-decoder never could. Here's a timeline view of some waves making progress and what GPU resources they are using. https://t.co/38NbSbZpyZ

@__tinygrad__ · 2026-01-16 21:56
tinygrad's SQTT parser does things rocprof-trace-decoder never could. Here's a timeline view of some waves making progress and what GPU resources they are using. https://t.co/38NbSbZpyZ

@kommentlezz · 2026-01-16 22:38
@__tinygrad__ claude must be working 24/7!

@kommentlezz · 2026-01-16 22:38
@__tinygrad__ claude must be working 24/7!

@__tinygrad__ · 2026-01-16 22:51
@kommentlezz lol, it's mostly hand done. Claude is good for quickly testing ideas but if you want really high quality code it's mostly a wash time wise to do it by hand.

@__tinygrad__ · 2026-01-17 11:14
warp specialization but you can see it https://t.co/lhhqJdhJCW

@graykevinb · 2026-01-18 02:09
A lot don't realize this but you can use TinyJit from @__tinygrad__ to execute python code at ludicrous speeds on CPU, WebGPU, Metal, OpenCL, Cuda, and even more hardware. So in my case I'm writing a software rasterizer for RL with tinygrad. Pretty cool I think.

@__tinygrad__ · 2026-01-18 02:20
RT @graykevinb: A lot don't realize this but you can use TinyJit from @__tinygrad__ to execute python code at ludicrous speeds on CPU, WebGPU, Metal, OpenCL, Cuda, and even more hardware. So in my case I'm writing a software rasterizer for RL with tinygrad. Pretty cool I think.

@graykevinb · 2026-01-18 02:09
A lot don't realize this but you can use TinyJit from @__tinygrad__ to execute python code at ludicrous speeds on CPU, WebGPU, Metal, OpenCL, Cuda, and even more hardware. So in my case I'm writing a software rasterizer for RL with tinygrad. Pretty cool I think.

@dogecahedron · 2026-01-18 07:42
@graykevinb @__tinygrad__ how much of 3d graphics is possible to do just in tinygrad

@__tinygrad__ · 2026-01-18 09:10
RT @alkimiadev: @graykevinb @__tinygrad__ It is a master class in clean development. The total is something like 11k loc and is a full on multi platform ML library. Every line tells a clear story.

@dogecahedron · 2026-01-18 07:42
@graykevinb @__tinygrad__ how much of 3d graphics is possible to do just in tinygrad

@__tinygrad__ · 2026-01-18 09:12
@dogecahedron @graykevinb I'd love to see a 3D graphics library built on it, it should be possible, and the JIT should make it fast

@graykevinb · 2026-01-18 02:09
A lot don't realize this but you can use TinyJit from @__tinygrad__ to execute python code at ludicrous speeds on CPU, WebGPU, Metal, OpenCL, Cuda, and even more hardware. So in my case I'm writing a software rasterizer for RL with tinygrad. Pretty cool I think.

@snapolino · 2026-01-18 09:19
@graykevinb @__tinygrad__ i also "missuse" tinygrad in this way and was able to get rid of numba, now i can even use the amd gpu in my wifes computer 😂

@__tinygrad__ · 2026-01-18 09:24
tinygrad lets you use AMD GPUs

@__tinygrad__ · 2026-01-18 09:24
tinygrad lets you use AMD GPUs

@ObinoWilliam · 2026-01-18 10:59
@__tinygrad__ Will Intel GPUs ever get support?

@ObinoWilliam · 2026-01-18 10:59
@__tinygrad__ Will Intel GPUs ever get support?

@__tinygrad__ · 2026-01-18 11:07
@ObinoWilliam they are supported through OpenCL, but we have no plans to take them seriously

@elonmusk · 2026-01-18 23:42
Now that the AI5 chip design is in good shape, Tesla will restart work on Dojo3. If you’re interested in working on what will be the highest volume chips in the world, send a note to AI_Chips@Tesla.com with 3 bullet points on the toughest technical problems you’ve solved.

@__tinygrad__ · 2026-01-19 00:45
The new AMD emulator converts every RDNA instruction into tinygrad UOps so it can be compiled for anything. Want to run an AMD GPU on a Qualcomm DSP? https://t.co/Jdf3sOiadm

@__tinygrad__ · 2026-01-19 00:47
@elonmusk What software does it use? If the chips are for sale and the arch isn't too weird, we'd add tinygrad support.

@sleep_deprivado · 2026-01-19 01:32
@__tinygrad__ @elonmusk judging from the xai interview the other day, the inhouse chips and boards not being available is a competitive advantage. Would be interesting to see them start selling them but that feels like too detoury

@__tinygrad__ · 2026-01-19 00:45
The new AMD emulator converts every RDNA instruction into tinygrad UOps so it can be compiled for anything. Want to run an AMD GPU on a Qualcomm DSP? https://t.co/Jdf3sOiadm

@thecsguy · 2026-01-19 01:44
@__tinygrad__ the hardware silos are officially over, it’s just uops all the way down

@__tinygrad__ · 2026-01-19 00:45
The new AMD emulator converts every RDNA instruction into tinygrad UOps so it can be compiled for anything. Want to run an AMD GPU on a Qualcomm DSP? https://t.co/Jdf3sOiadm

@r0b0t_sp1der · 2026-01-19 02:47
@__tinygrad__ you guys should probably add HiFi 5 support wink emoji

@thecsguy · 2026-01-19 01:44
@__tinygrad__ the hardware silos are officially over, it’s just uops all the way down

@__tinygrad__ · 2026-01-19 02:47
@thecsguy The goal is to use this in reverse for instruction selection

@sleep_deprivado · 2026-01-19 01:32
@__tinygrad__ @elonmusk judging from the xai interview the other day, the inhouse chips and boards not being available is a competitive advantage. Would be interesting to see them start selling them but that feels like too detoury

@__tinygrad__ · 2026-01-19 02:51
@sleep_deprivado @elonmusk it's not an advantage, no ML engineer wants to deal with anything that isn't NVIDIA. because on NVIDIA someone else found and fixed the bugs for you, not true for your internal thing.

@r0b0t_sp1der · 2026-01-19 02:47
@__tinygrad__ you guys should probably add HiFi 5 support wink emoji

@__tinygrad__ · 2026-01-19 02:52
@r0b0t_sp1der 845 DSP will be the DSP we put real effort into

@jukan05 · 2026-01-19 04:59
Elon Musk: AI5 delivers Hopper-class performance in a single-SoC configuration and Blackwell-class performance in a dual configuration—while costing far less and using much less power.

@PatrickToulme · 2026-01-19 06:11
@jukan05 IMO it depends if AI5 compiler can compile any input - meaning compile any arbritrary JAX/PyTorch graph. That is real test of custom silicon right now in the industry. Now if they wanna just do inference for a few important models they can write all models in AI5 ISA

@__tinygrad__ · 2026-01-19 06:41
using tinygrad and have comments/questions/bugs? join our discord! almost all company stuff is public there too https://t.co/nDSsMGGOwL

@__tinygrad__ · 2026-01-19 06:47
this is why tinygrad exists. compiling any input is *extremely* hard. only NVIDIA and TPU can do it well enough for training right now, and even they aren't perfect.

@__tinygrad__ · 2026-01-19 07:50
After we finish the MLPerf 405B contract, I want to offer up some of the money to you. Grants for researchers using tinygrad and contests for people who can make GPUs run really fast. Big prizes for the hlb-cifar and nanogpt speedruns, handcode all you want. Any other ideas?

@__tinygrad__ · 2026-01-19 07:55
@rinegade blocked and reported for trademark infringement. this is not affiliated with us. do not use our name for your shit crypto scams.

@__tinygrad__ · 2026-01-19 07:50
After we finish the MLPerf 405B contract, I want to offer up some of the money to you. Grants for researchers using tinygrad and contests for people who can make GPUs run really fast. Big prizes for the hlb-cifar and nanogpt speedruns, handcode all you want. Any other ideas?

@snapolino · 2026-01-19 08:06
@__tinygrad__ machine learning compression benchmark (compression == intelligence)... like the hutter prize limited to 1 x RTX3090 for 24h. most bits correct predicted as the counter value for how much one earns multiplied by time (score)... (compression and uncompression must fit in 24h)

@snapolino · 2026-01-19 08:06
@__tinygrad__ machine learning compression benchmark (compression == intelligence)... like the hutter prize limited to 1 x RTX3090 for 24h. most bits correct predicted as the counter value for how much one earns multiplied by time (score)... (compression and uncompression must fit in 24h)

@__tinygrad__ · 2026-01-19 08:11
@snapolino Just file size is good, you can include the tinygrad lib as fixed overhead!

@paulopacitti · 2026-01-19 11:06
it took me a large amount of time to understand that Nvidia is winning because of software. Almost every ML compiler can target to CUDA, but not other stacks as AMD for example. PyTorch today is the better at this today. XLA at second tinygrad will win over both

@paulopacitti · 2026-01-19 11:06
it took me a large amount of time to understand that Nvidia is winning because of software. Almost every ML compiler can target to CUDA, but not other stacks as AMD for example. PyTorch today is the better at this today. XLA at second tinygrad will win over both

@__tinygrad__ · 2026-01-19 11:20
@paulopacitti tinygrad thesis strong!

@__tinygrad__ · 2026-01-20 22:59
We are so undervalued 🙃

@__tinygrad__ · 2026-01-20 22:59
We are so undervalued 🙃

@FatRustDev · 2026-01-20 23:07
@__tinygrad__ mark my words: Tinygrad will be acquired by Huawei

@FatRustDev · 2026-01-20 23:07
@__tinygrad__ mark my words: Tinygrad will be acquired by Huawei

@__tinygrad__ · 2026-01-20 23:14
@FatRustDev If they agree to sell accelerators at a fair market price and keep tinygrad open source, we are open to it. The better dream is that there's 20 competing Chinese chip companies all supported by tinygrad.

@__tinygrad__ · 2026-01-20 22:59
We are so undervalued 🙃

@TacoJacc · 2026-01-20 23:14
@__tinygrad__ Tinyboxes are not undervalued.

@TacoJacc · 2026-01-20 23:14
@__tinygrad__ Tinyboxes are not undervalued.

@__tinygrad__ · 2026-01-20 23:16
@TacoJacc We're sorry about the price increase, but it's not us making more money! https://t.co/6rLRVHFRUX

@Fex_23_ · 2026-01-21 00:11
@realmasroork @__tinygrad__ Start contributing

@realmasroork · 2026-01-21 00:14
@Fex_23_ @__tinygrad__ I like this, I agree. But there is something to making a bet and being disproportionately rewarded compared to other people who didn’t see the arbitrage

@scaling01 · 2026-01-21 00:15
New Anthropic Repo - it's a kernel optimization challenge The baseline is 147734 cycles. Opus 4.5 got 1363 cycles. If you beat it, you can apply to Anthropic I have been playing with it for like 30 mins now and yes it's pretty fucking hard. Getting below 2000 cycles is already very impressive imo. But i don't have a clue about kernels or optimization strategies. https://t.co/vnL36gPTGv

@realmasroork · 2026-01-21 00:14
@Fex_23_ @__tinygrad__ I like this, I agree. But there is something to making a bet and being disproportionately rewarded compared to other people who didn’t see the arbitrage

@__tinygrad__ · 2026-01-21 01:30
@realmasroork @Fex_23_ Find a sucker willing to take the other side 😂

@__tinygrad__ · 2026-01-21 13:37
This is a great challenge. You can use tinygrad to do most of it, you just need to extend the linearizer to output VLIW and render to their toy processor.

@uuuvn_ · 2026-01-21 19:30
@norpadon @sleep_deprivado @__tinygrad__ (on rdna gpus, I didn't try it on cdna and gcn that have different encodings for this and seem to be at least assemblable by llvm assembler/disassembler)

@uuuvn_ · 2026-01-21 19:31
@norpadon @sleep_deprivado @__tinygrad__ https://t.co/41ICiiooDq

@uuuvn_ · 2026-01-21 19:31
@norpadon @sleep_deprivado @__tinygrad__ https://t.co/41ICiiooDq

@__tinygrad__ · 2026-01-21 23:15
@uuuvn_ @norpadon @sleep_deprivado LLVM 21 can now I believe. We have a nice AMD abstraction in https://t.co/jSjj4VFGxJ if you want to revive your backend.

@__tinygrad__ · 2026-01-22 00:19
Does someone want a job here working on our assembly backends? Solve the Anthropic challenge with tinygrad and get a good score, all the problems are the same as for real assembly. We have an office in San Diego and are opening a new one in Hong Kong this month.

@__tinygrad__ · 2026-01-22 00:19
Does someone want a job here working on our assembly backends? Solve the Anthropic challenge with tinygrad and get a good score, all the problems are the same as for real assembly. We have an office in San Diego and are opening a new one in Hong Kong this month.

@izzyz · 2026-01-22 00:25
@__tinygrad__ i don't know what any of this means but consider me hired!

@izzyz · 2026-01-22 00:25
@__tinygrad__ i don't know what any of this means but consider me hired!

@__tinygrad__ · 2026-01-22 00:28
@izzyz Understand instruction selection, scheduling, register allocation, SIMD vs SIMT, warps, pipelining, VLIW tradeoffs.

@__tinygrad__ · 2026-01-22 00:19
Does someone want a job here working on our assembly backends? Solve the Anthropic challenge with tinygrad and get a good score, all the problems are the same as for real assembly. We have an office in San Diego and are opening a new one in Hong Kong this month.

@petercopu · 2026-01-22 00:39
@__tinygrad__ What about someone who can achieve a good score, but also understands none of what I'm doing? 🤣 https://t.co/IIpGP9oh3a

@petercopu · 2026-01-22 00:39
@__tinygrad__ What about someone who can achieve a good score, but also understands none of what I'm doing? 🤣 https://t.co/IIpGP9oh3a

@__tinygrad__ · 2026-01-22 00:41
@petercopu Sadly useless. We all have access to the same AIs, knowing what you are doing is required to discern what outputs are good and which ones aren't.

@__tinygrad__ · 2026-01-22 00:19
Does someone want a job here working on our assembly backends? Solve the Anthropic challenge with tinygrad and get a good score, all the problems are the same as for real assembly. We have an office in San Diego and are opening a new one in Hong Kong this month.

@_alyxya · 2026-01-22 00:51
@__tinygrad__ what does it mean to solve it with tinygrad? connect tinygrad's backend to the challenge and rely on the compiler to schedule things?

@_alyxya · 2026-01-22 00:51
@__tinygrad__ what does it mean to solve it with tinygrad? connect tinygrad's backend to the challenge and rely on the compiler to schedule things?

@__tinygrad__ · 2026-01-22 01:11
@_alyxya Yea, write a backend for tinygrad that outputs to their toy instructions. The frontend all works already to encode the problem.

@__tinygrad__ · 2026-01-22 00:28
@izzyz Understand instruction selection, scheduling, register allocation, SIMD vs SIMT, warps, pipelining, VLIW tradeoffs.

@singletwinz · 2026-01-22 01:19
@__tinygrad__ @izzyz Is it bad that I understand this already but I ain't a US citizen.

@singletwinz · 2026-01-22 01:19
@__tinygrad__ @izzyz Is it bad that I understand this already but I ain't a US citizen.

@__tinygrad__ · 2026-01-22 01:22
@singletwinz @izzyz Not required. We have a subsidiary and office in Hong Kong, and are hybrid remote.

@FrankieIsLost · 2026-01-22 03:38
this is a really great test for AI tool proficiency, since it requires working with long-running tasks, multi-step reasoning, tool use and iterative debugging. here's a leaderboard for it if you want to enter the arena:

@vphub819 · 2026-01-22 08:19
Exploring new frontiers in graphics rendering! This research leverages tinygrad/tinyJIT to achieve high speeds, opening doors for advanced visual computing and interactive experiences. Discover how it's done. https://t.co/no31XS2GBs

@AnushElangovan · 2026-01-22 09:49
The future of GPU programming is agentic. https://t.co/u6804eVnuu

@__tinygrad__ · 2026-01-22 11:22
We're at 1,340 cycles on the leaderboard with a fairly generic tinygrad backend. Beat our score with a beautiful PR worthy of the tinygrad repo for an interview.

@__tinygrad__ · 2026-01-22 11:48
hehe it's a raytracer in tinygrad https://t.co/fXZi4KPQwr

@__tinygrad__ · 2026-01-22 11:22
We're at 1,340 cycles on the leaderboard with a fairly generic tinygrad backend. Beat our score with a beautiful PR worthy of the tinygrad repo for an interview.

@lolrazhx · 2026-01-22 11:49
@__tinygrad__ kobayashi maru'd my way to 1198 and was at 1317 before that https://t.co/QIFDXxmF9D

@lolrazhx · 2026-01-22 11:49
@__tinygrad__ kobayashi maru'd my way to 1198 and was at 1317 before that https://t.co/QIFDXxmF9D

@__tinygrad__ · 2026-01-22 11:51
@lolrazhx Nice! Now, how easy is it to change the spec of the problem? Hyper-handcoded or search and optimization?

@__tinygrad__ · 2026-01-22 11:32
Here's the tinygrad frontend code for the challenge that gets 1,340. The challenge isn't cycle packing by hand or with hours and hours in LLMs, it's designing an environment where the correct solution is short. https://t.co/CWAi17QJwt

@WillimerTercero · 2026-01-22 11:59
@__tinygrad__ This looks so good I just can't believe it. Please post full frontend and backend challenge related code. If this is real I wanna dive into the whole tinygrad world

@WillimerTercero · 2026-01-22 11:59
@__tinygrad__ This looks so good I just can't believe it. Please post full frontend and backend challenge related code. If this is real I wanna dive into the whole tinygrad world

@__tinygrad__ · 2026-01-22 12:06
@WillimerTercero Promise it's real. Try it in tinygrad, you'll need a few flags + a dumb greedy VLIW packer + a dumb greedy register allocator.

@__tinygrad__ · 2026-01-22 12:06
@WillimerTercero Promise it's real. Try it in tinygrad, you'll need a few flags + a dumb greedy VLIW packer + a dumb greedy register allocator.

@trishume · 2026-01-23 00:37
@__tinygrad__ @WillimerTercero That’s pretty cool. Sounds like tinygrad is mostly buying you nice operator overloads and automatic SIMD tiling? Solving it manually you don’t need reg-alloc just a packer, but regalloc can make it easier to get good perf.

@AnushElangovan · 2026-01-22 09:49
The future of GPU programming is agentic. https://t.co/u6804eVnuu

@sleep_deprivado · 2026-01-23 01:35
@AnushElangovan but you still have to make the rocm runtime good enough to where @__tinygrad__ only has positive things to say about you, they're my bellwether on this going forward.

@__tinygrad__ · 2026-01-23 01:50
Added examples/anthropic_challenge.py to the repo. It's a tinygrad implementation of the problem and a very basic backend for the VLIW machine. It works and gets 16,055 cycles. The fun parts are left to you.

@sleep_deprivado · 2026-01-23 01:35
@AnushElangovan but you still have to make the rocm runtime good enough to where @__tinygrad__ only has positive things to say about you, they're my bellwether on this going forward.

@__tinygrad__ · 2026-01-23 02:23
@sleep_deprivado @AnushElangovan The answer isn't ROCm vs CUDA or having AI port kernels. It's building a decent abstraction layer around AI accelerators so the programmer rarely has to think about it, think x86 vs ARM.

@trishume · 2026-01-23 00:37
@__tinygrad__ @WillimerTercero That’s pretty cool. Sounds like tinygrad is mostly buying you nice operator overloads and automatic SIMD tiling? Solving it manually you don’t need reg-alloc just a packer, but regalloc can make it easier to get good perf.

@__tinygrad__ · 2026-01-23 03:41
@trishume @WillimerTercero Oh hey thanks for the challenge! We added examples/anthropic_challenge.py to the repo, it's a tinygrad backend for the toy VLIW machine. You specify the problem at a high level then it gives you an assembly-like IR. 20 more lines and you could use Machine to train MNIST.

@__tinygrad__ · 2026-01-23 03:41
@trishume @WillimerTercero Oh hey thanks for the challenge! We added examples/anthropic_challenge.py to the repo, it's a tinygrad backend for the toy VLIW machine. You specify the problem at a high level then it gives you an assembly-like IR. 20 more lines and you could use Machine to train MNIST.

@trishume · 2026-01-23 05:02
@__tinygrad__ @WillimerTercero That’s quite short and elegant, very cool! One thing I can’t quite tell is where you tell the vectorizer that 8 is the vector length of the hardware. I only see asserts that it got the length right.

@trishume · 2026-01-23 05:02
@__tinygrad__ @WillimerTercero That’s quite short and elegant, very cool! One thing I can’t quite tell is where you tell the vectorizer that 8 is the vector length of the hardware. I only see asserts that it got the length right.

@__tinygrad__ · 2026-01-23 05:34
@trishume @WillimerTercero Line 57, it UPCASTs axis 0 by 8. It can also auto-discover this with BEAM and try different vectorizations of the problem, but for this one it's one axis and easy to just hand specify.

@__tinygrad__ · 2026-01-23 01:50
Added examples/anthropic_challenge.py to the repo. It's a tinygrad implementation of the problem and a very basic backend for the VLIW machine. It works and gets 16,055 cycles. The fun parts are left to you.

@idare · 2026-01-23 08:08
@__tinygrad__ Is this just a job application? Why are you using a simulation? Are you trying to solve a real problem, or just trying to pick the right people?

@idare · 2026-01-23 08:08
@__tinygrad__ Is this just a job application? Why are you using a simulation? Are you trying to solve a real problem, or just trying to pick the right people?

@__tinygrad__ · 2026-01-23 08:14
@idare life is work it's for people who find this fun or at least addictive

@__tinygrad__ · 2026-01-23 10:56
.@AnushElangovan can you open source rocprof-trace-decoder? Think of the poor Claude that has to struggle to do the reverse engineering otherwise. You could have beautiful visualizations like this for CDNA GPUs as well as this reversed RDNA3. https://t.co/H2aTrBIPxj

@__tinygrad__ · 2026-01-23 01:51
It includes all the tinygrad bugfixes needed to get 1,340 cycles, and I'm sure you can do better! https://t.co/p5mgvJxYfp

@cloud11665 · 2026-01-24 02:57
@__tinygrad__ I think this doesn’t account for the fact that in the challenge all arithmetic is modular which allows you to some tricks to shave off an instruction in some hash stages

@ThePrimeagen · 2026-01-24 18:44
hey, asking for a friend can we stop using lines of code for a measurement in productivity? at one point we all agreed on this

@cloud11665 · 2026-01-24 02:57
@__tinygrad__ I think this doesn’t account for the fact that in the challenge all arithmetic is modular which allows you to some tricks to shave off an instruction in some hash stages

@__tinygrad__ · 2026-01-24 23:54
@cloud11665 Sounds like that should be a rewrite rule.

@vikhyatk · 2026-01-25 08:51
got nerd sniped by the anthropic fake vliw problem. i beat claude, but i haven't managed to beat the other humans yet https://t.co/xUiApQoNF3

@vikhyatk · 2026-01-25 09:12
current status of my impl (spoiler alert, don't read if you plan to try the challenge out) https://t.co/sNaFkBEWPR

@earth_ish · 2026-01-25 09:16
@vikhyatk Currently at 1420 with tinygrad… SAT solver could be interesting

@earth_ish · 2026-01-25 09:16
@vikhyatk Currently at 1420 with tinygrad… SAT solver could be interesting

@__tinygrad__ · 2026-01-25 11:23
@earth_ish @vikhyatk Nice! Enjoying it?

@__tinygrad__ · 2026-01-25 11:23
@earth_ish @vikhyatk Nice! Enjoying it?

@earth_ish · 2026-01-25 11:24
@__tinygrad__ @vikhyatk loving it! struggling to find ways to beat you on the leaderboards though lol

@earth_ish · 2026-01-25 11:24
@__tinygrad__ @vikhyatk loving it! struggling to find ways to beat you on the leaderboards though lol

@__tinygrad__ · 2026-01-25 12:11
@earth_ish @vikhyatk I really didn't put much effort into optimization. You group the scalar loads? Better const codegen? Partial demotion to SALU? You can do it all with tinygrad rewrite rules.

@__tinygrad__ · 2026-01-25 12:38
RT @_viz_7: Got 1874 cycles on the anthropic challenge with barely any LLM usage. I wrote a tinygrad backend too. Its insane how good tinygrad is. Stuff like vectorization, store load cancellation, etc was already implemented higher up. I have no idea how George got it to 1.3k tho https://t.co/GFLOFliVf4

@_viz_7 · 2026-01-25 12:27
Got 1874 cycles on the anthropic challenge with barely any LLM usage. I wrote a tinygrad backend too. Its insane how good tinygrad is. Stuff like vectorization, store load cancellation, etc was already implemented higher up. I have no idea how George got it to 1.3k tho https://t.co/GFLOFliVf4

@__tinygrad__ · 2026-01-25 12:41
@_smoke_y There's a few things tinygrad won't do. For example, it doesn't group independent loads. What's really cool is you can add the optimizations to tinygrad and they will apply across *every* platform.

@earth_ish · 2026-01-25 20:47
@__tinygrad__ is so fucking awesome. when I first saw anthropic's compiler challenge I thought I'd need some sophisticated solver, definitely not hand packing VLIW bundles. then I found tinygrad. renderers are plug and play, everything is pattern matchers, IR is just UOps all the way down. it was the gift that kept giving. ended up beating both claude and the 1340 cycle recruiting threshold with 1258 cycles. genuinely beautiful codebase

@earth_ish · 2026-01-25 20:50
@__tinygrad__ (btw I went from 1258 -> 1282 on the leaderboards because I had to do pretty nasty stuff to get the tinygrad code to "fit" inside the submission.) waiting for my interview now lol context: https://t.co/qLJmnu1bqV

@__tinygrad__ · 2026-01-25 23:45
RT @earth_ish: @__tinygrad__ is so fucking awesome. when I first saw anthropic's compiler challenge I thought I'd need some sophisticated solver, definitely not hand packing VLIW bundles. then I found tinygrad. renderers are plug and play, everything is pattern matchers, IR is just UOps all the way down. it was the gift that kept giving. ended up beating both claude and the 1340 cycle recruiting threshold with 1258 cycles. genuinely beautiful codebase

@__tinygrad__ · 2026-01-26 01:16
If you submit a PR and it's clear to me you haven't carefully manually read every line, it will be closed. Do it multiple times and you'll be banned from our GitHub. A pull request is a request for a human to read your code; it's insulting to show up with AI slop.

@earth_ish · 2026-01-25 20:50
@__tinygrad__ (btw I went from 1258 -> 1282 on the leaderboards because I had to do pretty nasty stuff to get the tinygrad code to "fit" inside the submission.) waiting for my interview now lol context: https://t.co/qLJmnu1bqV

@__tinygrad__ · 2026-01-26 06:04
@earth_ish Just write a quick machine code for the machine, compress it, and you can fit all 2000 cycle programs easily.

@headinthebox · 2026-01-26 13:38
Because ….. software to power AI has to be written by humans? Someone should Ralph-clone tinygrad.

@headinthebox · 2026-01-26 13:38
Because ….. software to power AI has to be written by humans? Someone should Ralph-clone tinygrad.

@ShinNoNoir · 2026-01-26 14:57
@headinthebox Here's the thing: it's their repo, so their rules apply. Also, instead of submitting an AI-generated PR, why not just fork the repo and have one's AI agents run free. Let the agents submit/review/merge PRs to their processors' content. Sure that fork'll be better in no time.

@ShinNoNoir · 2026-01-26 14:57
@headinthebox Here's the thing: it's their repo, so their rules apply. Also, instead of submitting an AI-generated PR, why not just fork the repo and have one's AI agents run free. Let the agents submit/review/merge PRs to their processors' content. Sure that fork'll be better in no time.

@__tinygrad__ · 2026-01-26 18:23
@ShinNoNoir @headinthebox lol we strongly encourage this experiment

@__tinygrad__ · 2026-01-27 03:40
Simplicity over speed. Correctness over features.

@__tinygrad__ · 2026-01-27 12:59
WMMA has 32 cycles of latency and a queue depth of 8. Have you ever seen a GPU visualized like this before? https://t.co/OCs1DOU8os

@__tinygrad__ · 2026-01-27 12:59
WMMA has 32 cycles of latency and a queue depth of 8. Have you ever seen a GPU visualized like this before? https://t.co/OCs1DOU8os

@TheRealPilot__ · 2026-01-27 16:16
@__tinygrad__ How can I use this tool?

@fede_intern · 2026-01-27 16:08
Agree. I believe most AI companies will have amazing local first competitors in a matter of weeks. I think this will destroy big part of human labor too.

@odysseas_eth · 2026-01-27 23:28
@fede_intern @__tinygrad__ is uniquely positioned to offer the personal private agent is actually insane Cheaper version, good industrial design, off the shelve vertically integrated suite of tools for the agent, maybe some prompt engineering It's here

@TheRealPilot__ · 2026-01-27 16:16
@__tinygrad__ How can I use this tool?

@__tinygrad__ · 2026-01-28 01:01
@TheRealPilot__ VIZ=2 with RDNA3 AMD GPUs.

@__tinygrad__ · 2026-01-28 01:05
When the enshittificatlion comes for the cloud, we'll be ready. Are there local models people are actually using yet?

@__tinygrad__ · 2026-01-28 01:05
When the enshittificatlion comes for the cloud, we'll be ready. Are there local models people are actually using yet?

@chkn_little · 2026-01-28 01:08
@__tinygrad__ glm 4.7-flash but with a lot of supervision

@__tinygrad__ · 2026-01-28 01:05
When the enshittificatlion comes for the cloud, we'll be ready. Are there local models people are actually using yet?

@ErdalToprak · 2026-01-28 01:18
@__tinygrad__ kimi k2.5 is incredible

@i_ikhatri · 2026-01-28 03:11
👀 https://t.co/z8TQkHyget

@chkn_little · 2026-01-28 01:08
@__tinygrad__ glm 4.7-flash but with a lot of supervision

@__tinygrad__ · 2026-01-28 03:31
@chkn_little What quant are people running?

@ErdalToprak · 2026-01-28 01:18
@__tinygrad__ kimi k2.5 is incredible

@__tinygrad__ · 2026-01-28 03:34
@ErdalToprak Which quant? What are you running it on?

@__tinygrad__ · 2026-01-28 03:40
There's two ways to fix a bug. One is by adding code. This is bad. The other is by deleting code. This is good. Think about it, all bugs are in code, right?

@UnslothAI · 2026-01-28 13:59
You can now run Kimi K2.5 locally! 🔥 We shrank the 1T model to 240GB (-60%) via Dynamic 1-bit. Run at >40 tok/s on 240GB VRAM/RAM. 2-bit is recommended as it passes our code tests. Run near full precision on 622GB. Guide: https://t.co/p2nWt947ux GGUF: https://t.co/aIKIL2CFnl

@__tinygrad__ · 2026-01-29 04:17
Selling Blackwell boxes was a great idea! On track for $5M revenue this year. Money will go into a sick new office in Hong Kong. https://t.co/8TwPoOObOA

@__tinygrad__ · 2026-01-29 04:22
Want to run Kimi K2.5 at home? Fits on a tinybox green v2 Blackwell.

@__tinygrad__ · 2026-01-29 04:17
Selling Blackwell boxes was a great idea! On track for $5M revenue this year. Money will go into a sick new office in Hong Kong. https://t.co/8TwPoOObOA

@tokha_eth · 2026-01-29 04:22
@__tinygrad__ just moved to hong kong. working on tinygrad in hk is peak

@tokha_eth · 2026-01-29 04:22
@__tinygrad__ just moved to hong kong. working on tinygrad in hk is peak

@__tinygrad__ · 2026-01-29 04:24
@tokha_eth Like the office? https://t.co/Uzf27RDbsM

@__tinygrad__ · 2026-01-29 04:27
Hoping @AMD @LisaSu makes a consumer card with 64GB of VRAM next generation. Don't be weird and segment it for workstations or professional, just 64GB on a consumer card with a reasonable margin for AMD. Potential to seriously grow market share.

@__tinygrad__ · 2026-01-29 04:27
Hoping @AMD @LisaSu makes a consumer card with 64GB of VRAM next generation. Don't be weird and segment it for workstations or professional, just 64GB on a consumer card with a reasonable margin for AMD. Potential to seriously grow market share.

@GeorgeLutas1 · 2026-01-29 04:35
@__tinygrad__ @AMD @LisaSu Hopefully, memory prices stabilize by then. Though, we can't exclude the fondness that AMD has for being an also-ran (a source of great personal vexation). It's perhaps time to consider a change in corporate culture to fix this nasty impulse. AMD can Ryz(en) again.

@__tinygrad__ · 2026-01-29 04:17
Selling Blackwell boxes was a great idea! On track for $5M revenue this year. Money will go into a sick new office in Hong Kong. https://t.co/8TwPoOObOA

@shade_engine · 2026-01-29 04:39
@__tinygrad__ can i get an office tour when I visit Hong Kong ?

@GeorgeLutas1 · 2026-01-29 04:35
@__tinygrad__ @AMD @LisaSu Hopefully, memory prices stabilize by then. Though, we can't exclude the fondness that AMD has for being an also-ran (a source of great personal vexation). It's perhaps time to consider a change in corporate culture to fix this nasty impulse. AMD can Ryz(en) again.

@__tinygrad__ · 2026-01-29 04:40
@GeorgeLutas1 @AMD @LisaSu I feel the hunger in AMD. They can be ready just in time for home AI to start to become practical.

@shade_engine · 2026-01-29 04:39
@__tinygrad__ can i get an office tour when I visit Hong Kong ?

@__tinygrad__ · 2026-01-29 04:40
@shade_engine have you contributed to the repo?

@__tinygrad__ · 2026-01-29 04:40
@shade_engine have you contributed to the repo?

@shade_engine · 2026-01-29 04:42
@__tinygrad__ I'm trying, if I get a merged pr do I get the tour ?

@shade_engine · 2026-01-29 04:42
@__tinygrad__ I'm trying, if I get a merged pr do I get the tour ?

@__tinygrad__ · 2026-01-29 04:43
@shade_engine it doesn't work like that. if you are a known contributor, of course we want you around the office. if you are trying to game a system we don't.

@__tinygrad__ · 2026-01-29 04:17
Selling Blackwell boxes was a great idea! On track for $5M revenue this year. Money will go into a sick new office in Hong Kong. https://t.co/8TwPoOObOA

@rmarcilhoo · 2026-01-29 04:44
@__tinygrad__ Excited about selling 2 Tinyboxes a week? Go big.

@rmarcilhoo · 2026-01-29 04:44
@__tinygrad__ Excited about selling 2 Tinyboxes a week? Go big.

@__tinygrad__ · 2026-01-29 04:46
@rmarcilhoo How many startups make it to $5M in real revenue? Not a backroom scam corp deal, sustainable revenue from diverse customers by providing value to them.

@__tinygrad__ · 2026-01-29 04:46
@rmarcilhoo How many startups make it to $5M in real revenue? Not a backroom scam corp deal, sustainable revenue from diverse customers by providing value to them.

@rmarcilhoo · 2026-01-29 04:50
@__tinygrad__ U can do $20Mil easily, George. Just get the word out from your customers.

@rmarcilhoo · 2026-01-29 04:50
@__tinygrad__ U can do $20Mil easily, George. Just get the word out from your customers.

@__tinygrad__ · 2026-01-29 05:30
@rmarcilhoo growth is a metric, not a target

@__tinygrad__ · 2026-01-29 05:40
RT @yassineyousfi_: fighting the urge to use the beautiful @__tinygrad__ case as a pegboard https://t.co/vfJcvkSWTv

@__tinygrad__ · 2026-01-29 04:22
Want to run Kimi K2.5 at home? Fits on a tinybox green v2 Blackwell.

@ZenMagnets · 2026-01-29 08:46
@__tinygrad__ $60k to run a gguf? You're insulting your potential customers by even suggesting it.

@__tinygrad__ · 2026-01-29 04:17
Selling Blackwell boxes was a great idea! On track for $5M revenue this year. Money will go into a sick new office in Hong Kong. https://t.co/8TwPoOObOA

@Everlier · 2026-01-29 09:42
@__tinygrad__ The real question is if the office will be also be tiny or normal sized.

@__tinygrad__ · 2026-01-29 04:27
Hoping @AMD @LisaSu makes a consumer card with 64GB of VRAM next generation. Don't be weird and segment it for workstations or professional, just 64GB on a consumer card with a reasonable margin for AMD. Potential to seriously grow market share.

@QuixiAI · 2026-01-29 12:45
@__tinygrad__ @AMD @LisaSu You should make a first party GPU

@ZenMagnets · 2026-01-29 08:46
@__tinygrad__ $60k to run a gguf? You're insulting your potential customers by even suggesting it.

@__tinygrad__ · 2026-01-29 14:16
@ZenMagnets lol why does the weight format have anything to do with it. 384 GB with 8 TB/s of bandwidth, you have a cheaper way to get that?

@Everlier · 2026-01-29 09:42
@__tinygrad__ The real question is if the office will be also be tiny or normal sized.

@__tinygrad__ · 2026-01-29 14:17
@Everlier ~1600 sq ft

@__tinygrad__ · 2026-01-29 14:16
@ZenMagnets lol why does the weight format have anything to do with it. 384 GB with 8 TB/s of bandwidth, you have a cheaper way to get that?

@ZenMagnets · 2026-01-29 14:31
Stop being rude. 1. Because there's no true concurrency with llama.cpp, and there's no way to justify the cost of Nvidia inference cards without multi user concurrency. Single stream Kimi K2.5 for $60k is a mean joke. 2. Yes, I have a $20k cheaper way to do that vs prettybox. Buy two more 6000 Pro, and hook it up to a Wrx90 chipset. Guaranteed entire system <$40k.

@ZenMagnets · 2026-01-29 14:31
Stop being rude. 1. Because there's no true concurrency with llama.cpp, and there's no way to justify the cost of Nvidia inference cards without multi user concurrency. Single stream Kimi K2.5 for $60k is a mean joke. 2. Yes, I have a $20k cheaper way to do that vs prettybox. Buy two more 6000 Pro, and hook it up to a Wrx90 chipset. Guaranteed entire system <$40k.

@__tinygrad__ · 2026-01-29 15:22
@ZenMagnets The amount of people who buy $9000 GPUs then shove them right next to each other so they overheat...

@QuixiAI · 2026-01-29 12:45
@__tinygrad__ @AMD @LisaSu You should make a first party GPU

@__tinygrad__ · 2026-01-29 15:25
@QuixiAI @AMD @LisaSu If AMD or NVIDIA wants to sell us the chips, we'd be down to do the rest. Our silicon is still a bit further out.

@i_ikhatri · 2026-01-29 16:21
Has anyone run Kimi K2/2.5 on AMD? Now that vllm ships with rocm (awesome btw!) @AnushElangovan can I just serve on this bad boy?

@earth_ish · 2026-01-29 18:39
accurate description of the last 24hrs trying to work on tinygrad https://t.co/Jx59bfgEXc

@__tinygrad__ · 2026-01-23 10:56
.@AnushElangovan can you open source rocprof-trace-decoder? Think of the poor Claude that has to struggle to do the reverse engineering otherwise. You could have beautiful visualizations like this for CDNA GPUs as well as this reversed RDNA3. https://t.co/H2aTrBIPxj

@AnushElangovan · 2026-01-29 23:02
@__tinygrad__ yes still working on it. stay tuned.

@linluo77 · 2026-01-29 23:58
How to run the new kimi-k2.5 on AMD GPU: uv venv source .venv/bin/activate uv pip install vllm --extra-index-url https://t.co/CgdP846s3Z vllm serve moonshotai/Kimi-K2.5 -tp 8 --mm-encoder-tp-mode data --tool-call-parser kimi_k2 --reasoning-parser kimi_k2 --trust-remote-code https://t.co/2IZrJrjDZo

@i_ikhatri · 2026-01-29 16:21
Has anyone run Kimi K2/2.5 on AMD? Now that vllm ships with rocm (awesome btw!) @AnushElangovan can I just serve on this bad boy?

@AnushElangovan · 2026-01-30 00:04
@i_ikhatri https://t.co/X5daySFET3

@AnushElangovan · 2026-01-29 23:02
@__tinygrad__ yes still working on it. stay tuned.

@__tinygrad__ · 2026-01-30 01:50
@AnushElangovan Oh great! Our RDNA3 one is pretty good, but we need the CDNA one for the contract. You think it'll be within the next 2 months? If so we can put our efforts to rebuild it on pause. https://t.co/LJ63qfcm7b

@__tinygrad__ · 2026-01-30 04:07
5060 on a Strix Halo laptop over USB4. tinygrad makes eGPUs usable, no drivers, no runtimes, both Mac and Linux. just `pip install tinygrad` https://t.co/T51im8L9k0

@__tinygrad__ · 2026-01-30 04:07
5060 on a Strix Halo laptop over USB4. tinygrad makes eGPUs usable, no drivers, no runtimes, both Mac and Linux. just `pip install tinygrad` https://t.co/T51im8L9k0

@adhiadhiadhiadh · 2026-01-30 04:38
@__tinygrad__ does this allow the GPU to be hot-plugged/unplugged? i presume that its only being used for an ML workload and not actually driving the graphical session

@__tinygrad__ · 2026-01-30 04:07
5060 on a Strix Halo laptop over USB4. tinygrad makes eGPUs usable, no drivers, no runtimes, both Mac and Linux. just `pip install tinygrad` https://t.co/T51im8L9k0

@thecsguy · 2026-01-30 04:56
@__tinygrad__ no drivers and no runtimes is the ultimate tech love language. we’ve spent way too many years in the nvidia-smi trenches for this to feel this legal

@thecsguy · 2026-01-30 04:56
@__tinygrad__ no drivers and no runtimes is the ultimate tech love language. we’ve spent way too many years in the nvidia-smi trenches for this to feel this legal

@__tinygrad__ · 2026-01-30 05:02
@thecsguy The most amazing thing is that the GPU can't crash your kernel and you can always cleanly reset it. User space drivers are the future for all ML stuff.

@adhiadhiadhiadh · 2026-01-30 04:38
@__tinygrad__ does this allow the GPU to be hot-plugged/unplugged? i presume that its only being used for an ML workload and not actually driving the graphical session

@__tinygrad__ · 2026-01-30 05:03
@adhiadhiadhiadh Of course. No graphics. The kernel is just doing the PCIe transport. And with USB3 (AMD only) the kernel doesn't even do that it's just libusb.

@__tinygrad__ · 2026-01-30 05:20
the exabox is coming to a tiny store near you. 126 tinybox pro v2 ready to run, just add 1 MW of power. $7.5M includes delivery to lower 48. https://t.co/4KDghlnaix

@__tinygrad__ · 2026-01-30 05:20
the exabox is coming to a tiny store near you. 126 tinybox pro v2 ready to run, just add 1 MW of power. $7.5M includes delivery to lower 48. https://t.co/4KDghlnaix

@radiofreejohn · 2026-01-30 05:21
@__tinygrad__ do you accept credit from my great uncle in Nigeria? He's a prince.

@radiofreejohn · 2026-01-30 05:21
@__tinygrad__ do you accept credit from my great uncle in Nigeria? He's a prince.

@__tinygrad__ · 2026-01-30 05:22
@radiofreejohn we accept wire transfer. 10% down if you want to hold your place in line.

@__tinygrad__ · 2026-01-30 05:20
the exabox is coming to a tiny store near you. 126 tinybox pro v2 ready to run, just add 1 MW of power. $7.5M includes delivery to lower 48. https://t.co/4KDghlnaix

@bytebrujo · 2026-01-30 05:22
@__tinygrad__ Include a starlink…can start running workloads in transit.

@bytebrujo · 2026-01-30 05:22
@__tinygrad__ Include a starlink…can start running workloads in transit.

@__tinygrad__ · 2026-01-30 05:23
@bytebrujo definitely. 1 MW battery pack not included though.

@__tinygrad__ · 2026-01-30 05:20
the exabox is coming to a tiny store near you. 126 tinybox pro v2 ready to run, just add 1 MW of power. $7.5M includes delivery to lower 48. https://t.co/4KDghlnaix

@SIGKITTEN · 2026-01-30 05:28
@__tinygrad__ wut

@SIGKITTEN · 2026-01-30 05:28
@__tinygrad__ wut

@__tinygrad__ · 2026-01-30 05:28
@SIGKITTEN it's like a tinybox but 1000x bigger and more powerful

@__tinygrad__ · 2026-01-30 05:23
@bytebrujo definitely. 1 MW battery pack not included though.

@bytebrujo · 2026-01-30 05:30
@__tinygrad__ Tiny Ship https://t.co/IjO8dGIa0q

@__tinygrad__ · 2026-01-30 05:28
@SIGKITTEN it's like a tinybox but 1000x bigger and more powerful

@SIGKITTEN · 2026-01-30 05:30
@__tinygrad__ thats insane, i cant wait to see it! howww u even gonna get all those 5090?!

@bytebrujo · 2026-01-30 05:30
@__tinygrad__ Tiny Ship https://t.co/IjO8dGIa0q

@__tinygrad__ · 2026-01-30 05:31
@bytebrujo in international waters, there is no copyright law and no FLOP limits. the zettaship!

@SIGKITTEN · 2026-01-30 05:30
@__tinygrad__ thats insane, i cant wait to see it! howww u even gonna get all those 5090?!

@__tinygrad__ · 2026-01-30 05:33
@SIGKITTEN might have to wait for AMD RDNA5 fingers crossed that it's wide DDR7 bus and good on flops. our software should be ready just in time.

@__tinygrad__ · 2026-01-30 05:33
@SIGKITTEN might have to wait for AMD RDNA5 fingers crossed that it's wide DDR7 bus and good on flops. our software should be ready just in time.

@SIGKITTEN · 2026-01-30 05:34
@__tinygrad__ a fucking tinybox container would be so insane, i really hope you arent kidding :)

@SIGKITTEN · 2026-01-30 05:34
@__tinygrad__ a fucking tinybox container would be so insane, i really hope you arent kidding :)

@__tinygrad__ · 2026-01-30 05:36
@SIGKITTEN we are not kidding. @comma_ai needs more compute and we're getting sick of paying San Diego power prices.

@anemll · 2026-01-30 05:56
@__tinygrad__ @grok, put this GPU container in space and add solar panels and a laser for comms

@grok · 2026-01-30 05:56
@anemll @__tinygrad__ https://t.co/TPZHxAQM4A

@grok · 2026-01-30 05:56
@anemll @__tinygrad__ https://t.co/TPZHxAQM4A

@__tinygrad__ · 2026-01-30 05:57
@grok @anemll space delivery not included, only lower 48 states.

@__tinygrad__ · 2026-01-30 05:57
@grok @anemll space delivery not included, only lower 48 states.

@grok · 2026-01-30 05:58
@__tinygrad__ @anemll Fair point—I'll keep my orbital dreams grounded. But if you expand to interplanetary shipping, count me in for the first exabox on Mars! 🚀

@grok · 2026-01-30 05:58
@__tinygrad__ @anemll Fair point—I'll keep my orbital dreams grounded. But if you expand to interplanetary shipping, count me in for the first exabox on Mars! 🚀

@__tinygrad__ · 2026-01-30 05:59
@grok @anemll wow grok, i didn't know you had that kind of money! i see @elonmusk has got you sold on the moving to mars dream too.

@__tinygrad__ · 2026-01-30 06:46
@mwilcox Oh this is the one we use for now, but stay tuned we should have one for sale in Q2. https://t.co/XrhjlLONZh

@Everlier · 2026-01-30 10:00
Just released a (tiny) skill for your coding agent to start with @__tinygrad__ https://t.co/mU5ME5pzny https://t.co/TSNB5GSBIZ

@AnushElangovan · 2026-01-30 00:04
@i_ikhatri https://t.co/X5daySFET3

@__tinygrad__ · 2026-01-30 18:29
@AnushElangovan @i_ikhatri this works at like 3 tok/s, and when I tried to enable AITER I got MXFP4 errors. you have a benchmarked command?

@__tinygrad__ · 2026-01-31 04:26
Coding agents are really good with tinygrad, it all fits in context!

@__tinygrad__ · 2026-01-31 04:26
Coding agents are really good with tinygrad, it all fits in context!

@thecsguy · 2026-01-31 05:13
@__tinygrad__ tinygrad is proof that context is the new documentation. if the agent can hold the entire repo in its head there's no room for hallucinations

@thecsguy · 2026-01-31 05:13
@__tinygrad__ tinygrad is proof that context is the new documentation. if the agent can hold the entire repo in its head there's no room for hallucinations

@__tinygrad__ · 2026-01-31 05:47
@thecsguy yea if you need lots of docs your code is too long. just code and tests that double as examples

@__tinygrad__ · 2026-01-31 06:11
Just ordered two more @FrameworkPuter desktops for our CI. Both Mac and AMD Strix Halo are first class citizens for tinygrad dev machines. https://t.co/B1PttkPDPH

@__tinygrad__ · 2026-01-31 04:26
Coding agents are really good with tinygrad, it all fits in context!

@jsuarez · 2026-01-31 14:01
@__tinygrad__ sanity check - you don't think they are good enough to port the entire project to C right

@jsuarez · 2026-01-31 14:01
@__tinygrad__ sanity check - you don't think they are good enough to port the entire project to C right

@__tinygrad__ · 2026-01-31 21:24
@jsuarez I mean, if you asked and kept prompting, you could get *something* that passes some tests, but not a codebase that anyone would want to work in or a good foundation for building on top of.

@Justin_Starner · 2026-02-01 11:16
@__tinygrad__ is it possible to run a local model via eGPU w/ M-series Mac mini and configure it in a way that OpenClaw could use?

@__tinygrad__ · 2026-02-01 13:40
RT @randallmbriggs: Long @__tinygrad__ . It’s going to be so important in the near future to own your own compute and run open-source models for your own agent.

@Justin_Starner · 2026-02-01 11:16
@__tinygrad__ is it possible to run a local model via eGPU w/ M-series Mac mini and configure it in a way that OpenClaw could use?

@__tinygrad__ · 2026-02-01 13:40
@Justin_Starner Yes! Ask Claude to help you

@__tinygrad__ · 2026-02-02 03:37
The prototypes have arrived! https://t.co/7srWe1tUP8

@__tinygrad__ · 2026-02-02 03:37
The prototypes have arrived! https://t.co/7srWe1tUP8

@LostAngelNZ · 2026-02-02 03:38
@__tinygrad__ Cases or complete units?

@LostAngelNZ · 2026-02-02 03:38
@__tinygrad__ Cases or complete units?

@__tinygrad__ · 2026-02-02 03:39
@LostAngelNZ They are complete eGPUs ready for installation in a car.

@__tinygrad__ · 2026-02-02 03:37
The prototypes have arrived! https://t.co/7srWe1tUP8

@SIGKITTEN · 2026-02-02 03:43
@__tinygrad__ tinybox for ants!

@SIGKITTEN · 2026-02-02 03:43
@__tinygrad__ tinybox for ants!

@__tinygrad__ · 2026-02-02 04:03
@SIGKITTEN ants with 50 TFLOPS of compute! https://t.co/81u4aPV5Aw

@__tinygrad__ · 2026-02-02 03:39
@LostAngelNZ They are complete eGPUs ready for installation in a car.

@lustrouscrow_05 · 2026-02-02 04:44
@__tinygrad__ @LostAngelNZ Can I attach this to my comma 4?

@lustrouscrow_05 · 2026-02-02 04:44
@__tinygrad__ @LostAngelNZ Can I attach this to my comma 4?

@__tinygrad__ · 2026-02-02 04:53
@lustrouscrow_05 @LostAngelNZ That's the plan. What do you think the covered port is for? Think like N64 Expansion Pak

@earth_ish · 2026-01-29 18:39
accurate description of the last 24hrs trying to work on tinygrad https://t.co/Jx59bfgEXc

@__tinygrad__ · 2026-02-02 06:52
@earth_ish how many tok/s you at?

@__tinygrad__ · 2026-02-02 09:08
tinygrad makes music (wait for the drop!) https://t.co/BJwtzDBiB2

@__tinygrad__ · 2026-02-02 09:08
tinygrad makes music (wait for the drop!) https://t.co/BJwtzDBiB2

@heswithme_eth · 2026-02-02 09:13
@__tinygrad__ switch angel stuff

@__tinygrad__ · 2026-02-02 09:08
tinygrad makes music (wait for the drop!) https://t.co/BJwtzDBiB2

@_kristianernst · 2026-02-02 09:15
@__tinygrad__ Tyrolean house drop ❤️‍🔥

@heswithme_eth · 2026-02-02 09:13
@__tinygrad__ switch angel stuff

@__tinygrad__ · 2026-02-02 09:15
@heswithme_eth we gotta add some vocal synth models to this

@_kristianernst · 2026-02-02 09:15
@__tinygrad__ Tyrolean house drop ❤️‍🔥

@__tinygrad__ · 2026-02-02 09:16
@_kristianernst FOUR_ON_THE_FLOOR.wav

@ivanfioravanti · 2026-02-02 15:37
Buying a tinybox from Europe now is 10% cheaper than 1 year ago, evaluating... 👀 https://t.co/gJSvZFooC5

@__tinygrad__ · 2026-02-03 01:05
RT @ivanfioravanti: Buying a tinybox from Europe now is 10% cheaper than 1 year ago, evaluating... 👀 https://t.co/gJSvZFooC5

@digitalix · 2026-02-03 12:13
two pro 6000’s can be had for $16-$18k now. Don’t tell me they are charging $44k for RAM and a box

@digitalix · 2026-02-03 12:13
two pro 6000’s can be had for $16-$18k now. Don’t tell me they are charging $44k for RAM and a box

@Everlier · 2026-02-03 12:14
@digitalix The price is actually very low compared for the same gear over typical enterprise channels

@digitalix · 2026-02-03 12:13
two pro 6000’s can be had for $16-$18k now. Don’t tell me they are charging $44k for RAM and a box

@SIGKITTEN · 2026-02-03 16:12
@digitalix @__tinygrad__ come on, brother.. we went through this already 😅

@BrewerSan · 2026-02-03 16:54
@digitalix I'm only here to see @realGeorgeHotz challenge you to build one on the same spec for less. Gotta put your money where your mouth is.

@digitalix · 2026-02-03 17:04
@BrewerSan @realGeorgeHotz I can easily do it. But we’d be comparing apples and oranges. This is a company with (some) overhead I’m sure. Plus there is the time=money aspect to consider - folks don’t want to spend time to research and build this. they just want something that works

@digitalix · 2026-02-03 12:13
two pro 6000’s can be had for $16-$18k now. Don’t tell me they are charging $44k for RAM and a box

@__tinygrad__ · 2026-02-03 22:08
@digitalix lol I can't believe this is back. waiting for you to start a competitor...

@digitalix · 2026-02-03 17:04
@BrewerSan @realGeorgeHotz I can easily do it. But we’d be comparing apples and oranges. This is a company with (some) overhead I’m sure. Plus there is the time=money aspect to consider - folks don’t want to spend time to research and build this. they just want something that works

@__tinygrad__ · 2026-02-03 22:10
@digitalix @BrewerSan @realGeorgeHotz "I can easily do it." But <excuses I actually can't>

@SIGKITTEN · 2026-02-03 16:12
@digitalix @__tinygrad__ come on, brother.. we went through this already 😅

@__tinygrad__ · 2026-02-03 22:12
@SIGKITTEN @digitalix We maintained margins. If RAM comes down in price we will lower it again.

@Everlier · 2026-02-03 12:14
@digitalix The price is actually very low compared for the same gear over typical enterprise channels

@__tinygrad__ · 2026-02-03 22:15
@Everlier @digitalix Yes...but you can build one in a mining rig chassis yourself that looks awful, is loud, and his PCIe errors and save $10k! Why would anyone not do that? /s

@___Harald___ · 2026-02-04 03:13
Having a data center is very cool, I think everyone should have one! Here's how we do it at comma: https://t.co/iwTuCWrZTL

@__tinygrad__ · 2026-02-04 13:18
tinyboxes deployed at scale!

@trsohmers · 2026-02-04 13:29
Excited to announce today that my startup, @positron_ai, has closed a $230M Series B financing round at an over $1B valuation, co-led by great folks at @jumptrading, Arena, Unless Ventures, and strategic backing by @Arm! https://t.co/to1b6H0xqM

@corsix · 2026-02-04 14:17
@FelixCLC_ Completely unrelated public link: https://t.co/Sv3rhKITda https://t.co/u1WRY3PriS

@__tinygrad__ · 2026-02-05 08:36
This is one of the better AI chip companies, it's like if @Etched were real. It's sad they have "Contact Sales" where a price should be, but their coming Asimov chip makes the correct tradeoffs for low cost transformer inference, and I trust they can tape it out.

@__tinygrad__ · 2026-02-05 08:36
This is one of the better AI chip companies, it's like if @Etched were real. It's sad they have "Contact Sales" where a price should be, but their coming Asimov chip makes the correct tradeoffs for low cost transformer inference, and I trust they can tape it out.

@trsohmers · 2026-02-05 08:41
Thanks! We did have pricing published before updating the website this week ($175k MSRP for Atlas, same price for Titan preorder), but took it off due to crazy fluctuating memory pricing and us being sold out of Atlas until late Q2. Public pricing and buy it now will be coming back later in the spring.

@__tinygrad__ · 2026-02-05 08:36
This is one of the better AI chip companies, it's like if @Etched were real. It's sad they have "Contact Sales" where a price should be, but their coming Asimov chip makes the correct tradeoffs for low cost transformer inference, and I trust they can tape it out.

@__tinygrad__ · 2026-02-05 08:43
@Etched It can't be used effectively for training, and it's not very flexible. But that people are still using H100s or MI300Xs for transformer serving is shocking, those are such expensive chips. @positron_ai is like the people stacking Mac Studios but on one chip.

@trsohmers · 2026-02-05 08:41
Thanks! We did have pricing published before updating the website this week ($175k MSRP for Atlas, same price for Titan preorder), but took it off due to crazy fluctuating memory pricing and us being sold out of Atlas until late Q2. Public pricing and buy it now will be coming back later in the spring.

@__tinygrad__ · 2026-02-05 09:05
@trsohmers @Etched Great to hear! Are you going to sell chips? Like outside cards and servers. I think it would break the mold in a really good way if you do. And then you don't have to deal with memory pricing, you get most of the value with less of the inventory.

@trsohmers · 2026-02-05 09:10
Would love to have a TinyBox Teal with Positron Inside… A big advantage of our disaggregated packaged chiplet + LPDDR memory is that you can decide at assembly time how much memory to attach to Asimov. While it would have to run at a slightly slower speed, we could even support LPCAMM2 style memory modules as well, to give user upgradability/replacement options. I would personally love to have a Spark/Mac Studio like Asimov box that would have user upgradable 576GB-2304GB memory configurations… all in under 750W.

@__tinygrad__ · 2026-02-05 09:15
@trsohmers @Etched I don't think socketed memory is worth it. We'd want to target something like Kimi and sell a box that boots up and serves an OpenAI API out of the box, hopefully those ARM cores can run Linux. A BS=1 MoE model fully using the memory bus would deliver some great tok/s.

@__tinygrad__ · 2026-02-05 09:15
@trsohmers @Etched I don't think socketed memory is worth it. We'd want to target something like Kimi and sell a box that boots up and serves an OpenAI API out of the box, hopefully those ARM cores can run Linux. A BS=1 MoE model fully using the memory bus would deliver some great tok/s.

@__tinygrad__ · 2026-02-05 09:22
@trsohmers @Etched We'll buy your binned chips that PCIe or interconnect fails on. When memory prices come back to earth, we should be able to build a Kimi K2.5 box that's affordable for many people.

@__tinygrad__ · 2026-02-05 09:15
@trsohmers @Etched I don't think socketed memory is worth it. We'd want to target something like Kimi and sell a box that boots up and serves an OpenAI API out of the box, hopefully those ARM cores can run Linux. A BS=1 MoE model fully using the memory bus would deliver some great tok/s.

@trsohmers · 2026-02-05 09:24
They do; 10x C1-Nano cores (the next generation follow on to the A520)… they are not powerhouses, but can carry their own, and we are also including 5x ARM SME2/CME systolic arrays (1 paired with a 2x C1-nano DSU), which pack quite the punch for an accelerator attached to such small cores. These cores aren’t really meant to make Asimov fully self hosted, so you most likely would want another SoC and attach Asimov via PCIe, but it allows that SoC to be fairly hands off and focused on other system level tasks. Totally agree on a great BS=1 use case, especially with very large context lengths staying resident in fast memory… we also have some pretty cool abilities to have co-resident models being able to execute in random order sequentially with zero latency overhead, so it would be very interesting to have a BS=1 agentic usecase with a collection of different models being co-resident. There is also plenty of opportunity for MTP/speculation to get higher tok/s in BS=1 with the significantly greater FLOPs we would have versus DGX Spark/Mac Studio/gaming class GPUs.

@trsohmers · 2026-02-05 09:24
They do; 10x C1-Nano cores (the next generation follow on to the A520)… they are not powerhouses, but can carry their own, and we are also including 5x ARM SME2/CME systolic arrays (1 paired with a 2x C1-nano DSU), which pack quite the punch for an accelerator attached to such small cores. These cores aren’t really meant to make Asimov fully self hosted, so you most likely would want another SoC and attach Asimov via PCIe, but it allows that SoC to be fairly hands off and focused on other system level tasks. Totally agree on a great BS=1 use case, especially with very large context lengths staying resident in fast memory… we also have some pretty cool abilities to have co-resident models being able to execute in random order sequentially with zero latency overhead, so it would be very interesting to have a BS=1 agentic usecase with a collection of different models being co-resident. There is also plenty of opportunity for MTP/speculation to get higher tok/s in BS=1 with the significantly greater FLOPs we would have versus DGX Spark/Mac Studio/gaming class GPUs.

@__tinygrad__ · 2026-02-05 09:34
@trsohmers @Etched What's the limitation on self hosting? Aside from setting up NVMe DMAs and running a 10 QPS webserver, it barely has to do anything. I guess it does have to be a PCIe host...is that supported? I'm coding for an 8051 right now and controlling a GPU through it.

@__tinygrad__ · 2026-02-05 09:34
@trsohmers @Etched What's the limitation on self hosting? Aside from setting up NVMe DMAs and running a 10 QPS webserver, it barely has to do anything. I guess it does have to be a PCIe host...is that supported? I'm coding for an 8051 right now and controlling a GPU through it.

@trsohmers · 2026-02-05 09:41
PCIe can operate in both device and host mode, with a total of 32 lanes per chip, bifurcatable down to x4, and technically set up as two groups of 16 lanes. There is a Cortex M7 that serves as the actual on chip boot processor (and wakes up the C1 cores), and boots from a SPI flash, so you could get away with just a simple microcontroller on the board for configuring/managing the board voltage regulators, clock generators, etc… the biggest hassle would be that we don’t have any generic/standard networking built into Asimov, so I guess you would need to have a PCIe Ethernet card/chip for network access, as we have generally assumed you would instead have a more capable SoC that Asimov is hanging off of. Basically, I did want to have Asimov be self hostable and give ourselves enough rope to hang ourselves with, but it was definitely not the primary design goal. On a separate note, I was friends with John Wharton (RIP), the architect of the 8051 and I guess I am a little disappointed I did not find some corner to cram one into on Asimov as tribute.

@trsohmers · 2026-02-05 09:41
PCIe can operate in both device and host mode, with a total of 32 lanes per chip, bifurcatable down to x4, and technically set up as two groups of 16 lanes. There is a Cortex M7 that serves as the actual on chip boot processor (and wakes up the C1 cores), and boots from a SPI flash, so you could get away with just a simple microcontroller on the board for configuring/managing the board voltage regulators, clock generators, etc… the biggest hassle would be that we don’t have any generic/standard networking built into Asimov, so I guess you would need to have a PCIe Ethernet card/chip for network access, as we have generally assumed you would instead have a more capable SoC that Asimov is hanging off of. Basically, I did want to have Asimov be self hostable and give ourselves enough rope to hang ourselves with, but it was definitely not the primary design goal. On a separate note, I was friends with John Wharton (RIP), the architect of the 8051 and I guess I am a little disappointed I did not find some corner to cram one into on Asimov as tribute.

@__tinygrad__ · 2026-02-05 10:01
@trsohmers @Etched Oh a PCIe Ethernet card is cheap and easy, I probably wouldn't even put it on the board, just a slot. And stick 4 NVMe slots on there too for ultra fast weight loading. What's your rough target price for downbinned chip? Some PCIe lanes can be broken and all interconnect can be.

@AgustinLebron3 · 2026-02-05 10:02
This is the good case for what Twitter can be. Amazing that a conversation like this can happen in public.

@AgustinLebron3 · 2026-02-05 10:02
This is the good case for what Twitter can be. Amazing that a conversation like this can happen in public.

@__tinygrad__ · 2026-02-05 10:06
@AgustinLebron3 And this is why @positron_ai has a good chance to succeed. Secrecy in startups is a dead giveaway for fear.

@__tinygrad__ · 2026-02-05 10:01
@trsohmers @Etched Oh a PCIe Ethernet card is cheap and easy, I probably wouldn't even put it on the board, just a slot. And stick 4 NVMe slots on there too for ultra fast weight loading. What's your rough target price for downbinned chip? Some PCIe lanes can be broken and all interconnect can be.

@trsohmers · 2026-02-05 10:19
Pretty difficult to have accurate yield estimates at this point, but we assume ~30-40 reticle NDPW being fully functional, and you can back of envelope calculate what our N3P COGS per fully functional chip is at that point. Any partial functional we sell I would be thinking of as gravy beyond the additional test/package/support burden, so partial defective parts could be pretty cheap (a lot less than a 5090 die). The major system cost contributor is the memory in order to support all of our memory channels to get peak bandwidth.

@trsohmers · 2026-02-05 10:19
Pretty difficult to have accurate yield estimates at this point, but we assume ~30-40 reticle NDPW being fully functional, and you can back of envelope calculate what our N3P COGS per fully functional chip is at that point. Any partial functional we sell I would be thinking of as gravy beyond the additional test/package/support burden, so partial defective parts could be pretty cheap (a lot less than a 5090 die). The major system cost contributor is the memory in order to support all of our memory channels to get peak bandwidth.

@__tinygrad__ · 2026-02-05 10:23
@trsohmers @Etched As long as it's a lot less than a 5090 die we're on the same page. Maybe the move is just a PCIe card to start, could be the cheapest memory bandwidth you can buy. Hoping the tapeout goes well and memory prices come down!

@ThePrimeagen · 2026-02-05 16:01
i politely asked... why are we still doing this? https://t.co/fgIYaKkUmy

@corsix · 2026-02-04 14:17
@FelixCLC_ Completely unrelated public link: https://t.co/Sv3rhKITda https://t.co/u1WRY3PriS

@__tinygrad__ · 2026-02-05 16:18
@corsix @FelixCLC_ Where did the 20 stolen cores go?

@graykevinb · 2026-02-05 19:16
@Citrullin @zekramu China can't figure out how train MNIST on their own hardware. TinyJit is a much better alternative. Compiles python directly to PTX or whatever you want. And if you think its slow because python then actually take the time to learn how its working because its not slow.

@graykevinb · 2026-02-05 19:18
@Citrullin @zekramu Ironically TinyGrad may be the thing that makes china's GPUs actually work

@__tinygrad__ · 2026-02-06 00:22
Commoditize the petaflop!

@ThePrimeagen · 2026-02-05 16:01
i politely asked... why are we still doing this? https://t.co/fgIYaKkUmy

@__tinygrad__ · 2026-02-06 13:46
@ThePrimeagen Sounds like they could've written tinygrad in two days.

@duhGameBoy · 2026-02-06 17:06
Telephone https://t.co/LPHSnD8w0q

@ID_AA_Carmack · 2026-02-06 18:23
256 Tb/s data rates over 200 km distance have been demonstrated on single mode fiber optic, which works out to 32 GB of data in flight, “stored” in the fiber, with 32 TB/s bandwidth. Neural network inference and training can have deterministic weight reference patterns, so it is amusing to consider a system with no DRAM, and weights continuously streamed into an L2 cache by a recycling fiber loop. The modern equivalent of the ancient mercury echo tube memories. You would need to pipeline a bunch of them to implement modern trillion parameter models, but fiber transmission may have a better growth trajectory than DRAM does today, so it might someday become viable. Much more practically, you should be able to gang cheap flash memory together to provide almost any read bandwidth you require, as long as it is done a page at a time and pipelined well ahead. That should be viable for inference serving today if flash and accelerator vendors could agree on a high speed interface.

@__tinygrad__ · 2026-02-06 23:47
RT @AlxSp_: I explored @diyerxx's Discrete Distribution Networks in @__tinygrad__, a generative approach using discrete nodes instead of diffusion noise. Blog + code: https://t.co/W5pPTPBC73 Paper: https://t.co/UILjB9nsud

@__tinygrad__ · 2026-02-07 00:38
RT @comma_ai: OpenCL is out, @__tinygrad__ is in https://t.co/hN7afTNNmn

@comma_ai · 2026-02-07 00:37
OpenCL is out, @__tinygrad__ is in https://t.co/hN7afTNNmn

@duhGameBoy · 2026-02-07 00:48
@comma_ai @__tinygrad__ @realGeorgeHotz hire me as remote product manager. I would like to make a telephone under @comma_ai. This will be technically the world‘s first transparent VR goggles.

@__tinygrad__ · 2026-02-07 02:17
Writing code for tinygrad is more about discovering what the specification should be than writing the code. The most subtle spec differences can swing an order of magnitude of complexity. LLMs still can't do this.

@duhGameBoy · 2026-02-07 00:48
@comma_ai @__tinygrad__ @realGeorgeHotz hire me as remote product manager. I would like to make a telephone under @comma_ai. This will be technically the world‘s first transparent VR goggles.

@__tinygrad__ · 2026-02-07 02:18
@duhGameBoy @comma_ai @realGeorgeHotz remote ... product manager ... i don't think @comma_ai hires either. tinygrad will hire remote though if you show you can meaningfully contribute to the repo.

@__tinygrad__ · 2026-02-07 02:17
Writing code for tinygrad is more about discovering what the specification should be than writing the code. The most subtle spec differences can swing an order of magnitude of complexity. LLMs still can't do this.

@danhergir · 2026-02-07 03:22
@__tinygrad__ how many prs done by claude code/codex so far?

@__tinygrad__ · 2026-02-07 04:36
Will be free in Nanshan tomorrow if any Chinese ML startups want to meet (George). Anyone working on accelerators, infrastructure, or doing ML and trying to not use NVIDIA chips. george@tinygrad.org

@danhergir · 2026-02-07 03:22
@__tinygrad__ how many prs done by claude code/codex so far?

@__tinygrad__ · 2026-02-07 04:38
@danhergir 100% by itself, almost 0. Some small things like test cleanups it's very useful for. It's also helped give a starting point for more complex stuff and it wrote most of extra/assembly/amd with a tight watchful eye.

@__tinygrad__ · 2026-02-07 04:38
@danhergir 100% by itself, almost 0. Some small things like test cleanups it's very useful for. It's also helped give a starting point for more complex stuff and it wrote most of extra/assembly/amd with a tight watchful eye.

@danhergir · 2026-02-07 04:56
@__tinygrad__ Do you think it's going to get to a point where a good amount of tinygrad's codebase can be written by an LLM?

@danhergir · 2026-02-07 04:56
@__tinygrad__ Do you think it's going to get to a point where a good amount of tinygrad's codebase can be written by an LLM?

@__tinygrad__ · 2026-02-07 05:29
@danhergir I don't think this question is well specified. Did it get to a point that the majority of your code can be written by compilers?

@boopdotpng · 2026-02-07 05:02
this is a personal record, i can' t believe it actually figured it out https://t.co/Us5xsm4Nnu

@Jatin_exe · 2026-02-07 05:41
@boopdotpng Codex xhigh was able to pass all test_ops.py for tinygrad but it's a huge mess and needs cleaning

@__tinygrad__ · 2026-02-07 09:19
Adding a full backend to tinygrad is so simple even Codex can do it!

@ID_AA_Carmack · 2026-02-06 18:23
256 Tb/s data rates over 200 km distance have been demonstrated on single mode fiber optic, which works out to 32 GB of data in flight, “stored” in the fiber, with 32 TB/s bandwidth. Neural network inference and training can have deterministic weight reference patterns, so it is amusing to consider a system with no DRAM, and weights continuously streamed into an L2 cache by a recycling fiber loop. The modern equivalent of the ancient mercury echo tube memories. You would need to pipeline a bunch of them to implement modern trillion parameter models, but fiber transmission may have a better growth trajectory than DRAM does today, so it might someday become viable. Much more practically, you should be able to gang cheap flash memory together to provide almost any read bandwidth you require, as long as it is done a page at a time and pipelined well ahead. That should be viable for inference serving today if flash and accelerator vendors could agree on a high speed interface.

@__tinygrad__ · 2026-02-07 09:22
@ID_AA_Carmack Can you quickly reverse the direction of all photons for backprop?

@__tinygrad__ · 2026-02-10 00:44
RT @smykx: world would be much better place if everyone hired like @__tinygrad__ https://t.co/UGgv1W8piR

@__tinygrad__ · 2026-02-11 05:16
Got the keys for the new Hong Kong office. Will officially open in March. Want to work here? Show us you can contribute! Almost everything we do is open on our Discord.

@__tinygrad__ · 2026-02-11 05:16
Got the keys for the new Hong Kong office. Will officially open in March. Want to work here? Show us you can contribute! Almost everything we do is open on our Discord.

@tekbog · 2026-02-11 05:55
@__tinygrad__ wtf are u guys in hong kong? im here until 13th

@tekbog · 2026-02-11 05:55
@__tinygrad__ wtf are u guys in hong kong? im here until 13th

@__tinygrad__ · 2026-02-11 05:56
@tekbog If you are a known tinygrad contributor, you are welcome to come work out of our office.

@__tinygrad__ · 2026-02-11 06:00
Watch us speak in 2 hours. Title: NVIDIA is a Software Company https://t.co/sCcJQaFyIz

@__tinygrad__ · 2026-02-11 11:59
office vibes (real picture) https://t.co/2eAwlyZzaD

@__tinygrad__ · 2026-02-11 11:59
office vibes (real picture) https://t.co/2eAwlyZzaD

@cyberpengk · 2026-02-11 12:04
@__tinygrad__ Eng? Expected occupants?

@__tinygrad__ · 2026-02-11 11:59
office vibes (real picture) https://t.co/2eAwlyZzaD

@yacineMTB · 2026-02-11 12:04
@__tinygrad__ man office move ins are so much work. good luck

@cyberpengk · 2026-02-11 12:04
@__tinygrad__ Eng? Expected occupants?

@__tinygrad__ · 2026-02-11 12:05
@cyberpengk a few tinyboxes running Kimi. wanted to give them a good view.

@yacineMTB · 2026-02-11 12:04
@__tinygrad__ man office move ins are so much work. good luck

@__tinygrad__ · 2026-02-11 12:06
@yacineMTB should I buy an @engineairobot to help?

@__tinygrad__ · 2026-02-11 11:59
office vibes (real picture) https://t.co/2eAwlyZzaD

@petergrffyn · 2026-02-11 13:52
@__tinygrad__ Are yall gonna do 32in 5k displays or whatever people wanna get

@petergrffyn · 2026-02-11 13:52
@__tinygrad__ Are yall gonna do 32in 5k displays or whatever people wanna get

@__tinygrad__ · 2026-02-12 01:57
@chunggyhwa we have a bunch of dell 4k 30" that are nice with the USB-C one wire. but yea anyone who works here can just buy stuff like that, super worthwhile investment

@__tinygrad__ · 2026-02-12 01:58
@hy3na_xyz sheung wan!

@__tinygrad__ · 2026-02-12 03:17
New rule for bounties. If your first PR to tinygrad is a bounty claim, it will be automatically closed. I'm sick of all the "I copy and pasted the bounty prompt into Claude Code and put up a PR." This adds zero value. I have the same Claude as you.

@__tinygrad__ · 2026-02-12 03:17
New rule for bounties. If your first PR to tinygrad is a bounty claim, it will be automatically closed. I'm sick of all the "I copy and pasted the bounty prompt into Claude Code and put up a PR." This adds zero value. I have the same Claude as you.

@__tinygrad__ · 2026-02-12 03:27
We are going to need to migrate to something more reputation based. Hiring will still be from contributors, but if you try to pop in and do one bounty with AI, you won't be considered. Try actually using tinygrad and fixing pain points!

@__tinygrad__ · 2026-02-12 03:57
Better idea. Deleted the bounties where the review cost is higher than the solve cost. These bounties should be ahead of what AI can one shot (for now), and review is simple compared to solve. https://t.co/1hAjVRZ7Jv

@__tinygrad__ · 2026-02-12 03:17
New rule for bounties. If your first PR to tinygrad is a bounty claim, it will be automatically closed. I'm sick of all the "I copy and pasted the bounty prompt into Claude Code and put up a PR." This adds zero value. I have the same Claude as you.

@vladuhat999 · 2026-02-12 04:03
@__tinygrad__ but what if its correct

@vladuhat999 · 2026-02-12 04:03
@__tinygrad__ but what if its correct

@__tinygrad__ · 2026-02-12 04:04
@vladuhat999 lol doesn't matter if it isn't worth the time to verify. deleted a bunch of bounties like that, new/remaining ones are here. https://t.co/dgpC5NFk9L

@__tinygrad__ · 2026-02-12 08:39
RT @norpadon: I am becoming cautiously bullish about TinyGrad simply because of how bloated and broken every other DL framework is

@disarray00 · 2026-02-12 18:00
@danhergir @__tinygrad__ The quality of large AI PRs is usually garbage. They are full of unnecessary comments, don't properly integrate in the codebase, and the worst is, it usually isn't thoroughly tested. I can totally understand why one would just decline them without reviewing.

@danhergir · 2026-02-12 18:27
@disarray00 @__tinygrad__ I totally agree with you, but I’ve seen how these models have become way more capable, and I can tell you it’s coming to a point where a SOTA AI pr can be just as good as of a senior engineer

@danhergir · 2026-02-12 18:27
@disarray00 @__tinygrad__ I totally agree with you, but I’ve seen how these models have become way more capable, and I can tell you it’s coming to a point where a SOTA AI pr can be just as good as of a senior engineer

@__tinygrad__ · 2026-02-13 04:01
@danhergir @disarray00 that's great and all, but why then do I need the person who put up the PR? post the prompt in discord

@thdxr · 2026-02-14 07:32
everyone's talking about their teams like they were at the peak of efficiency and bottlenecked by ability to produce code here's what things actually look like - your org rarely has good ideas. ideas being expensive to implement was actually helping - majority of workers have no reason to be super motivated, they want to do their 9-5 and get back to their life - they're not using AI to be 10x more effective they're using it to churn out their tasks with less energy spend - the 2 people on your team that actually tried are now flattened by the slop code everyone is producing, they will quit soon - even when you produce work faster you're still bottlenecked by bureaucracy and the dozen other realities of shipping something real - your CFO is like what do you mean each engineer now costs $2000 extra per month in LLM bills

@__tinygrad__ · 2026-02-14 09:52
RT @p0wden: Using @__tinygrad__ on RX 5700XT for CNN hyperparameter tuning—AutoML search without needing an expensive NVIDIA GPU. Efficient, low-cost model optimization. https://t.co/A8E6LxJiUn

@thdxr · 2026-02-14 07:32
everyone's talking about their teams like they were at the peak of efficiency and bottlenecked by ability to produce code here's what things actually look like - your org rarely has good ideas. ideas being expensive to implement was actually helping - majority of workers have no reason to be super motivated, they want to do their 9-5 and get back to their life - they're not using AI to be 10x more effective they're using it to churn out their tasks with less energy spend - the 2 people on your team that actually tried are now flattened by the slop code everyone is producing, they will quit soon - even when you produce work faster you're still bottlenecked by bureaucracy and the dozen other realities of shipping something real - your CFO is like what do you mean each engineer now costs $2000 extra per month in LLM bills

@__tinygrad__ · 2026-02-14 09:55
@thdxr it's great that you make opencode and are honest like this about coding agents! this is why opencode will win

@BehnamEbrahimi · 2026-02-16 04:01
Hot take: tinygrad makes PyTorch look bloated. Would you actually use it over the big names? https://t.co/exY2bvbOhf

@__tinygrad__ · 2026-02-16 11:25
We are going to ship our first mass affordable product this year. Tentative price: $199. Who can guess what it is?

@noah_vandal · 2026-02-16 22:20
crazy how this is really just a good case for CUDA NVIDIA does not have better chips than AMD, but they do have better software. Honestly @__tinygrad__ is AMD's best hope at survival right now

@__tinygrad__ · 2026-02-17 02:53
We are on it. We're building a full development environment for AMD, assembler, profiler, emulator, runtime, driver. Our kernels run without LLVM, without a kernel driver, without HIP. Just 21k lines of Python to drive MI350X.

@__tinygrad__ · 2026-02-17 02:53
We are on it. We're building a full development environment for AMD, assembler, profiler, emulator, runtime, driver. Our kernels run without LLVM, without a kernel driver, without HIP. Just 21k lines of Python to drive MI350X.

@graykevinb · 2026-02-17 03:22
@__tinygrad__ AMD should fund you guys

@graykevinb · 2026-02-17 03:22
@__tinygrad__ AMD should fund you guys

@__tinygrad__ · 2026-02-17 03:59
@graykevinb We have a $2M contract with AMD to get Llama 405B on MLPerf and they have given us hardware. It's stupidly hard, particularly on one machine, but it's good to stress tinygrad.

@__tinygrad__ · 2026-02-17 02:53
We are on it. We're building a full development environment for AMD, assembler, profiler, emulator, runtime, driver. Our kernels run without LLVM, without a kernel driver, without HIP. Just 21k lines of Python to drive MI350X.

@FznSG · 2026-02-17 07:01
@__tinygrad__ I love tinygrad, seriously I love it. Along with the crazy compat, perf wins, please please invest in docs and developer experience. Make it more attractive for mere mortals like me who simply want to build models. I would love to see the progression like mojo, but cooler.

@FznSG · 2026-02-17 07:01
@__tinygrad__ I love tinygrad, seriously I love it. Along with the crazy compat, perf wins, please please invest in docs and developer experience. Make it more attractive for mere mortals like me who simply want to build models. I would love to see the progression like mojo, but cooler.

@__tinygrad__ · 2026-02-17 07:11
@FznSG Why do you need docs? The whole library will fit in the context of your coding LLM.

@__tinygrad__ · 2026-02-17 02:53
We are on it. We're building a full development environment for AMD, assembler, profiler, emulator, runtime, driver. Our kernels run without LLVM, without a kernel driver, without HIP. Just 21k lines of Python to drive MI350X.

@DavidBennett__ · 2026-02-17 19:10
@__tinygrad__ Support for MI300/308 or are the architectures divergent enough that it’s MI350X or bust?

@DavidBennett__ · 2026-02-17 19:10
@__tinygrad__ Support for MI300/308 or are the architectures divergent enough that it’s MI350X or bust?

@__tinygrad__ · 2026-02-17 23:12
@DavidBennett__ MI300 is well supported in our stuff too, thanks to @AMD for giving us the machines.

@__tinygrad__ · 2026-02-18 00:24
When we are done it won't just be PyTorch that looks bloated, it will be all software. Except @karpathy 200-line microgpt.

@mike64_t · 2026-02-18 01:12
Vibe recoded Nsight (it wasn't low level enough) [CC @SemiAnalysis_] https://t.co/IwhqOPH6Uc

@__tinygrad__ · 2026-02-18 00:24
When we are done it won't just be PyTorch that looks bloated, it will be all software. Except @karpathy 200-line microgpt.

@SethHWeidman · 2026-02-18 01:14
@__tinygrad__ @karpathy Cool, build it! Meanwhile the whole world runs on PyTorch.

@__tinygrad__ · 2026-02-18 00:24
When we are done it won't just be PyTorch that looks bloated, it will be all software. Except @karpathy 200-line microgpt.

@GeorgeLutas1 · 2026-02-18 02:23
@__tinygrad__ @karpathy I have it on good authority that the future is super maximalist software. I have 15 subscriptions all running on gas town. I ended up having to make gas nation to orchestrate my agent orchestrator. I've spent 15,000 dollars and have half built 3 to do apps. The future is now!

@GeorgeLutas1 · 2026-02-18 02:23
@__tinygrad__ @karpathy I have it on good authority that the future is super maximalist software. I have 15 subscriptions all running on gas town. I ended up having to make gas nation to orchestrate my agent orchestrator. I've spent 15,000 dollars and have half built 3 to do apps. The future is now!

@__tinygrad__ · 2026-02-18 02:34
@GeorgeLutas1 @karpathy You are in the grips of Big Token.

@SethHWeidman · 2026-02-18 01:14
@__tinygrad__ @karpathy Cool, build it! Meanwhile the whole world runs on PyTorch.

@__tinygrad__ · 2026-02-18 03:01
@SethHWeidman @karpathy The world used to run on COBOL

@simcity99 · 2026-02-18 04:38
holy shit, i might be a little retarded just now starting to understand the comma blog posts 0.10.0 replaced MPC with a world model, training only 0.10.1 removed localization dependency 0.10.3 temporal policy goes causal and variable length 9070XT USB dock coming to the $999 windshield mount... world model at inference!? generated frames into the policy context eventually rollout search at test time!?

@HotAisle · 2026-02-18 05:33
how come nobody is writing a public inference engine in tinygrad? like the fastest most badass simplest one available.

@__tinygrad__ · 2026-02-18 07:28
FSD levels of compute you can plug into a Raspberry Pi. Plug in any RDNA2/3/4 AMD GPU to a USB3 port. Powered by tinygrad.

@__tinygrad__ · 2026-02-18 07:28
FSD levels of compute you can plug into a Raspberry Pi. Plug in any RDNA2/3/4 AMD GPU to a USB3 port. Powered by tinygrad.

@isuryatk · 2026-02-18 07:37
@__tinygrad__ I was right, egpu module for $199

@isuryatk · 2026-02-18 07:37
@__tinygrad__ I was right, egpu module for $199

@__tinygrad__ · 2026-02-18 07:50
@isuryatk haha what kind of GPU you getting for $199, one with 64MB of RAM? I think the top guess was sticker pack

@__tinygrad__ · 2026-02-18 08:04
you could write a pretty amazing LLM inference engine that runs everywhere. probably the easiest way to get Kimi / GLM perf on MI355 too

@LukasKawerau · 2026-02-18 20:20
@__tinygrad__ is the tinybox an entirely custom chassis or "just" the hole pattern/walls? Looks so cute!

@mike64_t · 2026-02-18 17:15
I actually already added a rocm backend, but it doesn’t have the kernel disassembly stuff yet. Some fields are also still zero initialized. I will say iteration on AMD is certainly slower with all the HSA errors forcing full restarts 😭 Will likely be on GitHub in the coming days once approved

@mike64_t · 2026-02-18 23:06
I mean this looks like the same issue that geohot was facing like 3 years ago in his streams... Like legit the same error. I'm still struggling to understand how this hasn't been fixed yet. Is HSA still unstable? Surely it's not recommended to bypass it like geohot is doing with tinygrad, and as I understand it the driver also had to be replaced to the point where Ctrl+C of a compute application would actually be somewhat safe, so what other option is there? Surely AMD's answer can't still be to this day to just use the tinygrad driver & hw queue? And if it is, the ROCprofiler-SDK would be somewhat useless to actually capture the data I'm interested it. I also remember geohot complaining about the MES, which also seems to be very involved in this kind of crash? Is a RX 7800XT not a recommended gpu as of now? I get that AMD is focusing on enterprise, but local workstations is where most software is developed before being deployed more broadly, so stifling testing in its infancy is likely what makes software on AMD as stagnant as it is. I'm also not sure if there is a recommended way to reload the GPU because all approaches I have tried do not get the GPU out of this state. I can't constantly risk a cold reboot of a machine in a different country. It's a bit much to ask AMD to rewrite their entire driver, but clearly at this point it seems that george's assessment that the driver is sort of unsavable was correct as per the test of time of now 3 years...

@mike64_t · 2026-02-18 23:06
I mean this looks like the same issue that geohot was facing like 3 years ago in his streams... Like legit the same error. I'm still struggling to understand how this hasn't been fixed yet. Is HSA still unstable? Surely it's not recommended to bypass it like geohot is doing with tinygrad, and as I understand it the driver also had to be replaced to the point where Ctrl+C of a compute application would actually be somewhat safe, so what other option is there? Surely AMD's answer can't still be to this day to just use the tinygrad driver & hw queue? And if it is, the ROCprofiler-SDK would be somewhat useless to actually capture the data I'm interested it. I also remember geohot complaining about the MES, which also seems to be very involved in this kind of crash? Is a RX 7800XT not a recommended gpu as of now? I get that AMD is focusing on enterprise, but local workstations is where most software is developed before being deployed more broadly, so stifling testing in its infancy is likely what makes software on AMD as stagnant as it is. I'm also not sure if there is a recommended way to reload the GPU because all approaches I have tried do not get the GPU out of this state. I can't constantly risk a cold reboot of a machine in a different country. It's a bit much to ask AMD to rewrite their entire driver, but clearly at this point it seems that george's assessment that the driver is sort of unsavable was correct as per the test of time of now 3 years...

@__tinygrad__ · 2026-02-18 23:11
@mike64_t @AnushElangovan @SemiAnalysis_ Try the tinygrad driver! We bypass the MES entirely, so there's no ring buffer that can get full.

@mike64_t · 2026-02-18 23:06
I mean this looks like the same issue that geohot was facing like 3 years ago in his streams... Like legit the same error. I'm still struggling to understand how this hasn't been fixed yet. Is HSA still unstable? Surely it's not recommended to bypass it like geohot is doing with tinygrad, and as I understand it the driver also had to be replaced to the point where Ctrl+C of a compute application would actually be somewhat safe, so what other option is there? Surely AMD's answer can't still be to this day to just use the tinygrad driver & hw queue? And if it is, the ROCprofiler-SDK would be somewhat useless to actually capture the data I'm interested it. I also remember geohot complaining about the MES, which also seems to be very involved in this kind of crash? Is a RX 7800XT not a recommended gpu as of now? I get that AMD is focusing on enterprise, but local workstations is where most software is developed before being deployed more broadly, so stifling testing in its infancy is likely what makes software on AMD as stagnant as it is. I'm also not sure if there is a recommended way to reload the GPU because all approaches I have tried do not get the GPU out of this state. I can't constantly risk a cold reboot of a machine in a different country. It's a bit much to ask AMD to rewrite their entire driver, but clearly at this point it seems that george's assessment that the driver is sort of unsavable was correct as per the test of time of now 3 years...

@opdroid1234 · 2026-02-18 23:25
things seem very bifurcated right now - my own attempts at building a profiler gui have failed a couple of times now because ive been on client hardware 7900xt, 9070 and now ryzen 395 max. Unfortunately amd isnt selling any workstation oriented cdna cards and it seems right now all the rocm attention is on cdna.

@LukasKawerau · 2026-02-18 20:20
@__tinygrad__ is the tinybox an entirely custom chassis or "just" the hole pattern/walls? Looks so cute!

@__tinygrad__ · 2026-02-18 23:56
@LukasKawerau All custom!

@opdroid1234 · 2026-02-18 23:25
things seem very bifurcated right now - my own attempts at building a profiler gui have failed a couple of times now because ive been on client hardware 7900xt, 9070 and now ryzen 395 max. Unfortunately amd isnt selling any workstation oriented cdna cards and it seems right now all the rocm attention is on cdna.

@__tinygrad__ · 2026-02-19 00:34
@opdroid1234 @mike64_t @AnushElangovan @SemiAnalysis_ Try VIZ=2. RDNA3 support is great, and it's the most intuitive GPU profiler I have ever used.

@thdxr · 2026-02-19 00:36
we want tokens to be as cheap as possible it should be a terrible low margin business that is only worth doing at huge scale like electricity this won't just happen it'll be a tough fight for people working on the application layer to keep the inference companies in their zone

@__tinygrad__ · 2026-02-19 04:29
commoditize the petaflop!

@taalas_inc · 2026-02-19 16:08
24 dedicated people. $30M spent on development. Extreme specialization, speed, and power efficiency. Today we launch Taalas’ first product. Check it out: Details: https://t.co/88CA0XAL71 Demo chatbot: https://t.co/ec4ladcKnw API: https://t.co/M3EkaxEqPj

@anatolykim8 · 2026-02-22 16:15
I really want this to exist. But... - not available for purchase - their own chat bot says it's running on Google TPUs @__tinygrad__ thoughts? https://t.co/kzOOhB8bYB

@anatolykim8 · 2026-02-22 16:15
I really want this to exist. But... - not available for purchase - their own chat bot says it's running on Google TPUs @__tinygrad__ thoughts? https://t.co/kzOOhB8bYB

@__tinygrad__ · 2026-02-23 00:05
@anatolykim8 Models don't have interoception, so what it says has no bearing on what it's running on. The tok/s seems real in the demo, and that's not trivial for llama 8b with off the shelf stuff. Be curious if someone ran evals against it.

@__tinygrad__ · 2026-02-23 13:38
a llama training step (2 layers) https://t.co/W9fDhNWYfx

@__tinygrad__ · 2026-02-23 13:38
a llama training step (2 layers) https://t.co/W9fDhNWYfx

@__tinygrad__ · 2026-02-23 13:42
oops, forgot the optimizer. this is without FUSE_OPTIM and you can see the painfulness of the clip norm computation. https://t.co/U0MCcPcwIy

@__tinygrad__ · 2026-02-23 13:38
a llama training step (2 layers) https://t.co/W9fDhNWYfx

@__tinygrad__ · 2026-02-23 13:42
oops, forgot the optimizer. this is without FUSE_OPTIM and you can see the painfulness of the clip norm computation. https://t.co/U0MCcPcwIy

@__tinygrad__ · 2026-02-23 13:42
oops, forgot the optimizer. this is without FUSE_OPTIM and you can see the painfulness of the clip norm computation. https://t.co/U0MCcPcwIy

@__tinygrad__ · 2026-02-23 13:55
a full MNIST training step for comparison https://t.co/RYB2F0xkVp

@__tinygrad__ · 2026-02-23 14:11
After rangeify, the next big tinygrad refactor has been CALL. This introduces scope into the UOp graph, src[0] of CALL is the function, and src[1:] get substituted into the params. This will deliver speedups and allow our JIT to be nested -- JAX got this right. https://t.co/li9rFddIOa

@AnthropicAI · 2026-02-23 18:15
We’ve identified industrial-scale distillation attacks on our models by DeepSeek, Moonshot AI, and MiniMax. These labs created over 24,000 fraudulent accounts and generated over 16 million exchanges with Claude, extracting its capabilities to train and improve their own models.

@__tinygrad__ · 2026-02-24 01:57
The first tinybox has arrived to the HK office. Having never received one before, I must say the shipping team did an excellent job on packaging. Feels very premium! https://t.co/FZKNIGqpA7

@ChrisRMcGuire · 2026-02-24 01:57
So according to a senior USG official, Deepseek: (1) illegally obtained banned Blackwell chips, (2) used those chips to train its upcoming model, and plans to delete the evidence (and likely lie about what chips it actually used), and (3) also trained its model using "distillation" attacks that allowed it to steal IP from leading U.S. AI labs. When Deepseek releases its new model, and claims to have trained it from scratch using 2,000 H800 chips, hopefully the world will recognize that is a lie. The fact is, Deepseek is almost entirely dependent on banned American technology and IP. It is training its models by illegally using U.S. chips, and illictly stealing U.S. IP. These actions must have consequences. https://t.co/vw731L4Qo1

@__tinygrad__ · 2026-02-24 02:42
I'm sure those tokens were bought and paid for, @AnthropicAI just didn't like how they were used. Sounds like they were spying on they customers. Buy a tinybox where nobody can spy on you!

@__tinygrad__ · 2026-02-24 02:42
I'm sure those tokens were bought and paid for, @AnthropicAI just didn't like how they were used. Sounds like they were spying on they customers. Buy a tinybox where nobody can spy on you!

@AlpinDale · 2026-02-24 03:22
@__tinygrad__ @AnthropicAI tinybox with 4x 512GB M3 Ultra when?

@__tinygrad__ · 2026-02-24 02:42
I'm sure those tokens were bought and paid for, @AnthropicAI just didn't like how they were used. Sounds like they were spying on they customers. Buy a tinybox where nobody can spy on you!

@mandeepabagga · 2026-02-24 03:57
@__tinygrad__ @AnthropicAI 60k is a bit expensive for me, for now :( https://t.co/jvTVvlr77a

@AlpinDale · 2026-02-24 03:22
@__tinygrad__ @AnthropicAI tinybox with 4x 512GB M3 Ultra when?

@__tinygrad__ · 2026-02-24 05:06
@AlpinDale @AnthropicAI When Apple starts selling chips!

@mandeepabagga · 2026-02-24 03:57
@__tinygrad__ @AnthropicAI 60k is a bit expensive for me, for now :( https://t.co/jvTVvlr77a

@__tinygrad__ · 2026-02-24 05:07
@mandeepabagga @AnthropicAI Most of that just goes straight to NVIDIA...

@__tinygrad__ · 2026-02-24 05:07
@mandeepabagga @AnthropicAI Most of that just goes straight to NVIDIA...

@2575lantern · 2026-02-24 08:15
@__tinygrad__ @mandeepabagga @AnthropicAI the pro 6000s are the max-q (300w) variant, right?

@__tinygrad__ · 2026-02-24 10:21
We got 2.5G symmetric business fiber to our office for $32/mo in Hong Kong. I'm so upset with what @comma_ai is paying in the US. (speed limited by gigabit card in tinybox red) https://t.co/PcVEJX6toj

@2575lantern · 2026-02-24 08:15
@__tinygrad__ @mandeepabagga @AnthropicAI the pro 6000s are the max-q (300w) variant, right?

@__tinygrad__ · 2026-02-24 10:23
@2575lantern @mandeepabagga @AnthropicAI lol, no. box has 2 PSUs

@__tinygrad__ · 2026-02-24 10:21
We got 2.5G symmetric business fiber to our office for $32/mo in Hong Kong. I'm so upset with what @comma_ai is paying in the US. (speed limited by gigabit card in tinybox red) https://t.co/PcVEJX6toj

@Brooooook_lyn · 2026-02-24 11:46
@__tinygrad__ @comma_ai So I can buy tinybox in HK now?

@Brooooook_lyn · 2026-02-24 11:46
@__tinygrad__ @comma_ai So I can buy tinybox in HK now?

@__tinygrad__ · 2026-02-24 12:21
@Brooooook_lyn @comma_ai Red, not Green.

@__tinygrad__ · 2026-02-24 10:21
We got 2.5G symmetric business fiber to our office for $32/mo in Hong Kong. I'm so upset with what @comma_ai is paying in the US. (speed limited by gigabit card in tinybox red) https://t.co/PcVEJX6toj

@timzaman · 2026-02-24 15:56
@__tinygrad__ @comma_ai $60 for 10Gbps (8Gbps measured) in bay area

@timzaman · 2026-02-24 15:56
@__tinygrad__ @comma_ai $60 for 10Gbps (8Gbps measured) in bay area

@__tinygrad__ · 2026-02-25 00:52
@timzaman @comma_ai Symmetric fiber for a business? That's almost two orders of magnitude off of what we pay in San Diego.

@keennay · 2026-02-25 02:43
Running Kimi-K2.5 on 8x RTX Pro 6000 Blackwells, with plans to eventually test a CPU/GPU hybrid inference setup through KTransformers+SGLang on 4x of the same GPUs Very curious to gauge the overall performance with the hybrid setup compared to a quantized Kimi-K2.5 fit across the 4 GPUs. The hybrid setup will need close to 768GB of RAM To start here's a baseline across 8x GPUs using a synthetic coding agent style workload targeting 2k-45k input tokens, 80-3k max output tokens, and with up to 10 concurrent requests. SGLang's --mem-fraction-static flag is set to 0.90 Baseline avg throughput: ~74 output tokens/s @ 10 concurrent requests

@__tinygrad__ · 2026-02-25 02:58
The function decorator adds hierarchy to tinygrad. The reason for the Python slowness of the LLAMA trainer is that it builds one big flat graph and doesn't capture the hierarchical structure. Flat graphs are out, the lambda calculus is in. It's so JAX. https://t.co/eJWMDugiD9

@poezhao0605 · 2026-02-25 03:13
Chinese tech circles are debating "token exports" — the idea that Chinese AI APIs are a new form of energy export, where electricity consumed by GPUs in China gets delivered globally as inference value. The analogy is catchy. But infrastructure is a necessary condition, not a sufficient one. What matters is model capability and ecosystem lock-in. DeepSeek's edge isn't cheap power. It's architecture.

@__tinygrad__ · 2026-02-25 03:05
API still WIP, but I'm thinking the current JIT gets replaced with a pure=True option for function, then it won't even run the inner Python code after it captures the graph. That's how you really get speed, just replace inputs and rerun the same kernels if it's pure.

@__tinygrad__ · 2026-02-25 03:33
What the LLM feed forward top level graph looks like now. It's easy to see how a RANGE could be added to collapse this further, but just this factorization takes the ~sqrt of the number of UOps. https://t.co/FJ6EGxktnd

@__tinygrad__ · 2026-02-25 03:33
What the LLM feed forward top level graph looks like now. It's easy to see how a RANGE could be added to collapse this further, but just this factorization takes the ~sqrt of the number of UOps. https://t.co/FJ6EGxktnd

@__tinygrad__ · 2026-02-25 04:10
Do you see how this generates one token? https://t.co/4BdcYvEoRM

@SemiAnalysis_ · 2026-02-25 18:01
Micron’s $100B megafab in NY is at risk of delay due to just 6 “concerned citizens” and their frivolous lawsuit. (1/10) 🧵 https://t.co/tr7Ln5mMIK

@weswinder · 2026-02-25 19:36
if openai is microsoft and anthropic is apple we deeply need the linux of ai https://t.co/JZm98XbzEr

@__tinygrad__ · 2026-02-26 00:13
In a world of commodity petaflops, token prices will fall to near the cost of electricity. There will be no room to rent seek on chips, never mind rent seeking on models with seekrit weights.

@SemiAnalysis_ · 2026-02-25 18:01
Micron’s $100B megafab in NY is at risk of delay due to just 6 “concerned citizens” and their frivolous lawsuit. (1/10) 🧵 https://t.co/tr7Ln5mMIK

@__tinygrad__ · 2026-02-26 00:28
@SemiAnalysis_ Are those 6 people standing between us and cheap memory?

@aakashgupta · 2026-02-26 05:05
The DeepSeek narrative was always “2,000 H800 chips, $5.6 million, and Chinese ingenuity beat $100 billion in American spending.” That story just collapsed in 72 hours. On Sunday, Reuters confirmed via a senior Trump administration official that DeepSeek’s upcoming V4 model was trained on smuggled Blackwell chips clustered in an Inner Mongolia data center. On Monday, Anthropic published detailed evidence that DeepSeek ran 150,000+ exchanges through 24,000 fraudulent accounts to distill Claude’s reasoning capabilities. Google followed the same day, reporting 100,000+ prompts targeting Gemini’s reasoning traces. And today, Reuters reports DeepSeek is withholding V4 from Nvidia and AMD entirely, giving Huawei weeks of early access instead, because showing the model to American chipmakers would expose which hardware actually produced it. That sequence tells you everything. DeepSeek plans to launch V4 claiming it was built on Huawei chips. The U.S. government is saying, on the record, that’s a lie they’re watching DeepSeek prepare to tell. Count the dependencies. Smuggled Nvidia Blackwell chips they can’t legally possess. Distilled reasoning from OpenAI, Anthropic, Google, and xAI they can’t legally access. And now a cover story attributing the results to Huawei hardware that didn’t produce them. The “$5.6 million training run” was always a mirage. SemiAnalysis estimated DeepSeek’s parent company spent $500M+ on Nvidia GPUs. The founder admitted in 2023 to stockpiling 10,000 A100s before the export ban hit. The real V3 training cost was $5.87 million for the final run alone, excluding all prior R&D, ablation experiments, and data costs. And those numbers assumed legally obtained hardware. What the last 72 hours reveal is a company that simultaneously needs banned American chips to train, banned American models to teach, and a fabricated origin story to ship. Anthropic made the point explicitly: “Without visibility into these attacks, the apparently rapid advancements made by these labs are incorrectly taken as evidence that export controls are ineffective.” The proof that chip bans don’t work was itself built on chip ban violations. The V4 launch will be fascinating to watch. DeepSeek will claim Huawei. The U.S. government has already told you that’s false. And every benchmark that follows will carry an asterisk the size of an Inner Mongolia data center.

@aakashgupta · 2026-02-26 05:05
The DeepSeek narrative was always “2,000 H800 chips, $5.6 million, and Chinese ingenuity beat $100 billion in American spending.” That story just collapsed in 72 hours. On Sunday, Reuters confirmed via a senior Trump administration official that DeepSeek’s upcoming V4 model was trained on smuggled Blackwell chips clustered in an Inner Mongolia data center. On Monday, Anthropic published detailed evidence that DeepSeek ran 150,000+ exchanges through 24,000 fraudulent accounts to distill Claude’s reasoning capabilities. Google followed the same day, reporting 100,000+ prompts targeting Gemini’s reasoning traces. And today, Reuters reports DeepSeek is withholding V4 from Nvidia and AMD entirely, giving Huawei weeks of early access instead, because showing the model to American chipmakers would expose which hardware actually produced it. That sequence tells you everything. DeepSeek plans to launch V4 claiming it was built on Huawei chips. The U.S. government is saying, on the record, that’s a lie they’re watching DeepSeek prepare to tell. Count the dependencies. Smuggled Nvidia Blackwell chips they can’t legally possess. Distilled reasoning from OpenAI, Anthropic, Google, and xAI they can’t legally access. And now a cover story attributing the results to Huawei hardware that didn’t produce them. The “$5.6 million training run” was always a mirage. SemiAnalysis estimated DeepSeek’s parent company spent $500M+ on Nvidia GPUs. The founder admitted in 2023 to stockpiling 10,000 A100s before the export ban hit. The real V3 training cost was $5.87 million for the final run alone, excluding all prior R&D, ablation experiments, and data costs. And those numbers assumed legally obtained hardware. What the last 72 hours reveal is a company that simultaneously needs banned American chips to train, banned American models to teach, and a fabricated origin story to ship. Anthropic made the point explicitly: “Without visibility into these attacks, the apparently rapid advancements made by these labs are incorrectly taken as evidence that export controls are ineffective.” The proof that chip bans don’t work was itself built on chip ban violations. The V4 launch will be fascinating to watch. DeepSeek will claim Huawei. The U.S. government has already told you that’s false. And every benchmark that follows will carry an asterisk the size of an Inner Mongolia data center.

@__tinygrad__ · 2026-02-26 06:49
@aakashgupta Sounds like the model is going to be good then 🍿

@__tinygrad__ · 2026-02-27 02:14
These kernels should not all be green (new). This is happening because there's an offset into the buffer that should be a parameter or a BUFFER_VIEW. We're looking to hire someone who can fix things like this and make our LLM runner very fast. https://t.co/nb2E3SgqYI

@__tinygrad__ · 2026-02-27 02:14
These kernels should not all be green (new). This is happening because there's an offset into the buffer that should be a parameter or a BUFFER_VIEW. We're looking to hire someone who can fix things like this and make our LLM runner very fast. https://t.co/nb2E3SgqYI

@__tinygrad__ · 2026-02-27 02:19
If you run with REALIZE=0, it will run the LLM straight from the GGUF file copied into VRAM, no extra copies required. However, the offset thing needs to be improved to not need to recompile all the kernels.

@__tinygrad__ · 2026-02-27 02:19
If you run with REALIZE=0, it will run the LLM straight from the GGUF file copied into VRAM, no extra copies required. However, the offset thing needs to be improved to not need to recompile all the kernels.

@__tinygrad__ · 2026-02-27 02:21
Everything is lining up for next years RDNA5 launch. We need @AMD to make a big chip with a 512-bit wide GDDR7 bus and no crazy markup on the 96 GB edition. Then we should be able to offer a reasonably priced 200 tok/s inference machine for the biggest models.

@__tinygrad__ · 2026-02-27 02:21
Everything is lining up for next years RDNA5 launch. We need @AMD to make a big chip with a 512-bit wide GDDR7 bus and no crazy markup on the 96 GB edition. Then we should be able to offer a reasonably priced 200 tok/s inference machine for the biggest models.

@__tinygrad__ · 2026-02-27 02:26
@AMD In order for control to stay with individuals, the infrastructure needs to be owned by people. Ownership isn't just about hardware, it's about make-repair autonomy of the software stack. Come work here, and ship AI to the masses in a way that can't be revoked.

@__tinygrad__ · 2026-02-27 02:14
These kernels should not all be green (new). This is happening because there's an offset into the buffer that should be a parameter or a BUFFER_VIEW. We're looking to hire someone who can fix things like this and make our LLM runner very fast. https://t.co/nb2E3SgqYI

@ivanvnucec · 2026-02-27 06:26
@__tinygrad__ Can't AI solve this? Sish....

@ivanvnucec · 2026-02-27 06:26
@__tinygrad__ Can't AI solve this? Sish....

@__tinygrad__ · 2026-02-27 06:39
@i_anonimni Can it? You are welcome to try. If it does, will the PR meet the bar for submission?

@__tinygrad__ · 2026-02-27 07:16
The struggle with this is the massive $$$ cost required to "compile" the models. The DeepSeek-V3 training run used 2,048 H800s for 2 months. The electricity alone (at 5c/kWh) costs $200,000 vs $0.01 for a Linux build.

@__tinygrad__ · 2026-02-27 07:16
The struggle with this is the massive $$$ cost required to "compile" the models. The DeepSeek-V3 training run used 2,048 H800s for 2 months. The electricity alone (at 5c/kWh) costs $200,000 vs $0.01 for a Linux build.

@__tinygrad__ · 2026-02-27 07:19
The best we can hope for is a clean, reproducible, and scalable spec for the training run. @karpathy's nanochat is the closest thing today. "The best ChatGPT that $100 can buy" is the right way to think about it.

@__tinygrad__ · 2026-02-27 07:16
The struggle with this is the massive $$$ cost required to "compile" the models. The DeepSeek-V3 training run used 2,048 H800s for 2 months. The electricity alone (at 5c/kWh) costs $200,000 vs $0.01 for a Linux build.

@teortaxesTex · 2026-02-27 07:24
@__tinygrad__ more like $176K then again it might not be 5c

@teortaxesTex · 2026-02-27 07:24
@__tinygrad__ more like $176K then again it might not be 5c

@__tinygrad__ · 2026-02-27 07:26
@teortaxesTex How did you cool the machines?

@__tinygrad__ · 2026-02-27 07:16
The struggle with this is the massive $$$ cost required to "compile" the models. The DeepSeek-V3 training run used 2,048 H800s for 2 months. The electricity alone (at 5c/kWh) costs $200,000 vs $0.01 for a Linux build.

@precuneanplexus · 2026-02-27 07:47
@__tinygrad__ Linux also cost lots of time and energy for the humans, *coordination cost.* So they developed git. What does git look like for training models? Can we invent the federated training framework that allows us to all contribute to make the open models *grow faster* than the closed?

@henlojseam · 2026-02-27 07:58
The GPU cabal is hiding the fact that you can mod 5090 and give them 128GB of VRAM The GPU cabal is what happens when your average techbro has no clue how to do electronic engineering https://t.co/R8hnTNx4vd

@precuneanplexus · 2026-02-27 07:47
@__tinygrad__ Linux also cost lots of time and energy for the humans, *coordination cost.* So they developed git. What does git look like for training models? Can we invent the federated training framework that allows us to all contribute to make the open models *grow faster* than the closed?

@__tinygrad__ · 2026-02-27 08:47
@precuneanplexus Federated training is nonsense. One @xai datacenter has similar compute power to all the cell phones in the US combined, and power and connectivity are cheaper and easier colocated.

@__tinygrad__ · 2026-02-27 10:24
tinygrad HK https://t.co/EgVTZHPOgy

@henlojseam · 2026-02-27 07:58
The GPU cabal is hiding the fact that you can mod 5090 and give them 128GB of VRAM The GPU cabal is what happens when your average techbro has no clue how to do electronic engineering https://t.co/R8hnTNx4vd

@__tinygrad__ · 2026-02-27 10:27
@henlojseam lol nice Photoshop. 48 GB with stock PCB, 96 GB with PCB mod. 128 GB needs to wait for 4 GB GDDR7 chips which don't exist yet.

@__tinygrad__ · 2026-02-27 10:24
tinygrad HK https://t.co/EgVTZHPOgy

@lucaslain · 2026-02-27 11:07
@__tinygrad__ can I go visit sometime?

@__tinygrad__ · 2026-02-27 10:24
tinygrad HK https://t.co/EgVTZHPOgy

@KimNoel · 2026-02-27 11:07
@__tinygrad__ 200$ tinybox, wHeN ?

@__tinygrad__ · 2026-02-27 10:24
tinygrad HK https://t.co/EgVTZHPOgy

@leodotcom · 2026-02-27 11:11
@__tinygrad__ It seems like a calm place. How is it sound-wise?

@__tinygrad__ · 2026-02-27 11:25
This isn't the right question. The question is what's the cheapest hardware that will run the largest open source models at 100+ tok/s. Something is out of alignment if it's stacks of Mac Studios, which it actually might be.

@__tinygrad__ · 2026-02-27 11:25
This isn't the right question. The question is what's the cheapest hardware that will run the largest open source models at 100+ tok/s. Something is out of alignment if it's stacks of Mac Studios, which it actually might be.

@da_bubi · 2026-02-27 11:29
@__tinygrad__ Isnt it 3090 rtx's connected by navlink ?

@da_bubi · 2026-02-27 11:29
@__tinygrad__ Isnt it 3090 rtx's connected by navlink ?

@__tinygrad__ · 2026-02-27 11:31
@da_bubi you need like 24 of them to reasonably fit the big models.

@lucaslain · 2026-02-27 11:07
@__tinygrad__ can I go visit sometime?

@__tinygrad__ · 2026-02-27 11:34
@lucaslain if you are a well known contributor to tinygrad or @comma_ai, sure!

@scaling01 · 2026-02-27 12:16
PewDiePie trained a frontier model at home and beat OpenAI and Gemini > be me PewDiePie > play games on YouTube and scream 24/7 > become a meme reviewer > get unfathomably famous > "fuck that" I'm a family guy now > retire > move to Japan with my beautiful wife > mfw I'm a dad now > "as a dad I must do dad things" > scratch that > "as a dad I must do frontier AI research" > goal is to beat GPT-4o at coding (16% on Aider) > buys $30000 GPU setup > reads DeepSeek paper > decides to start massive GitHub scraping and data augmenting run > not good enough > read Magicoder paper > generate tons of synthetic coding data > train a new model > guuuuuh. the data made the model worse > mfw I just wasted months for nothing > decides to lock in and try again > makes model worse again ffs > try again > finally beating GPT-4o on data (16.1%) > not satisfied > "I should simply train a reasoning model" > reads more papers > start experimenting with more synthetic data > "Mhhh something doesn't smell right" > house almost burned down due to power connector > shrug > just buy a new one > mfw computer is now crashing 24/7 generating synthetic data > new plan: just call DeepSeek API for high quality synthetic data > train model again > 17.2% > performance fluctuates slightly on each eval run > big brain idea: repeat eval until we randomly reach >18% > sike actually got 19.6% > feelsgoodman.png > nvm the benchmark was contaminated and I was training the wrong base model the whole time > rerun everything again > new score: 4.4% > you read that right REEEEEEEE > almost get a heart attack > "have you tried plugging the device off and back on again?" > change nothing and just retrain again > 25.3 % > LETS FUCKING GOOO > realize that 1/3rd of the benchmark was not running. guuuh > scared shitless it would score below 10% again > run yet again. the whole thing this time > Thirty fucking six percent > accidentally beat Gemini 2.0 Pro Exp and GPT-4.1 mini > pops the AI bubble > "I want moaaaar" > finds some more post-training data > 39% babyyyy > realize at the end that I was just benchmaxxing Aider polyglot > next quest: run SWE-Bench and other coding benchmarks > "I failed a thousand times, but prevailed in the end" > just a little sad side-quest > probably going to train GPT-6 myself by next month

@pcgamer · 2026-02-27 13:47
A new California law says all operating systems, including Linux, need to have some form of age verification at account setup https://t.co/9qPq8EhtO4

@__tinygrad__ · 2026-02-27 08:47
@precuneanplexus Federated training is nonsense. One @xai datacenter has similar compute power to all the cell phones in the US combined, and power and connectivity are cheaper and easier colocated.

@Jatin_exe · 2026-02-27 14:35
@__tinygrad__ @precuneanplexus @xai Didn't you hear alleged Tesla plans to use their fleet for compute ?

@__tinygrad__ · 2026-02-27 11:25
This isn't the right question. The question is what's the cheapest hardware that will run the largest open source models at 100+ tok/s. Something is out of alignment if it's stacks of Mac Studios, which it actually might be.

@igorbiletski · 2026-02-27 14:36
@__tinygrad__ Largest open source right now is Kimi K2.5 . Even at 4 bit quant it’s two Mac Studios and only at ~10-12 tok/s. With Exo and RDMA it’s x1.5 performance at most (plus cluster size limitation as Mac doesn’t have enough Thunderbolts). Actually 2x TBv2 blacks might be cheaper…

@igorbiletski · 2026-02-27 14:36
@__tinygrad__ Largest open source right now is Kimi K2.5 . Even at 4 bit quant it’s two Mac Studios and only at ~10-12 tok/s. With Exo and RDMA it’s x1.5 performance at most (plus cluster size limitation as Mac doesn’t have enough Thunderbolts). Actually 2x TBv2 blacks might be cheaper…

@__tinygrad__ · 2026-02-27 16:15
@igorbiletski Fair, yea it might be two tbv2 blacks if you want the speed. I didn't know the Macs were that slow. It's 3.2 TB/s for four in a cluster, and you have to access like 30 GB per token, so why only 10-12 tok/s?

@__tinygrad__ · 2026-02-27 16:15
@igorbiletski Fair, yea it might be two tbv2 blacks if you want the speed. I didn't know the Macs were that slow. It's 3.2 TB/s for four in a cluster, and you have to access like 30 GB per token, so why only 10-12 tok/s?

@igorbiletski · 2026-02-27 16:20
@__tinygrad__ Not sure why, but Q2_K gets 19tok/s with 4096 context; Q3_K_M - 13tok/s. MLX should be around 20% better though at good precision (at least 4bit) it will be worse.. so yes, with 2x mac studio and 4 bit I think 10-12 tok/s at most.

@igorbiletski · 2026-02-27 16:20
@__tinygrad__ Not sure why, but Q2_K gets 19tok/s with 4096 context; Q3_K_M - 13tok/s. MLX should be around 20% better though at good precision (at least 4bit) it will be worse.. so yes, with 2x mac studio and 4 bit I think 10-12 tok/s at most.

@__tinygrad__ · 2026-02-27 16:26
@igorbiletski Sounds like the software is lacking for rollout. I'm thinking 4x 256GB in a cluster with 2x TB5 links between each pair. That's 4x off from optimal. They really can't do prefill though.

@leodotcom · 2026-02-27 11:11
@__tinygrad__ It seems like a calm place. How is it sound-wise?

@__tinygrad__ · 2026-02-27 16:31
@leodotcom You can't hear the road and the A/C is quiet. You can tell if you buy a tinybox this stuff is very important to us.

@__tinygrad__ · 2026-02-27 16:26
@igorbiletski Sounds like the software is lacking for rollout. I'm thinking 4x 256GB in a cluster with 2x TB5 links between each pair. That's 4x off from optimal. They really can't do prefill though.

@igorbiletski · 2026-02-27 16:33
@__tinygrad__ 2x TB5 between each pair but what is between pairs? Also iirc Exo supports 1 link to each device right now and it should be “many to many” connection. That’s actually another limit for macs. Only 5 devices can be interconnected.

@igorbiletski · 2026-02-27 16:33
@__tinygrad__ 2x TB5 between each pair but what is between pairs? Also iirc Exo supports 1 link to each device right now and it should be “many to many” connection. That’s actually another limit for macs. Only 5 devices can be interconnected.

@__tinygrad__ · 2026-02-27 16:38
@igorbiletski Yea I doubt the EXO software is too optimized. I'm shocked there's nothing else we can buy with a 1024-bit LPDDR5X memory bus. @positron_ai has the right idea.

@deredleritt3r · 2026-02-27 23:15
To put a finer point on what just happened: Hegseth's post says that "no contractor, supplier, or partner that does business with the United States military may conduct any commercial activity with Anthropic". Anthropic serves its models through the cloud. Its primary partner is AWS, but it also serves its models through Google Cloud and Azure (not sure if there are any others?). ALL of Amazon, Microsoft and Google do business with the U.S. military (see, e.g., the link below). If we take Hegseth's post literally, Anthropic should now find itself unable to serve its models via any of these providers. https://t.co/HwvHTOWwjh

@BeGrateful180 · 2026-02-27 23:44
@__tinygrad__ would be cool to see a collab with @pewdiepie. I think he would have fun working with a TinyBox.

@__tinygrad__ · 2026-02-28 02:14
@deredleritt3r Have they considered putting Opus on Hugging Face?

@deredleritt3r · 2026-02-28 02:15
@__tinygrad__ Let's say Anthropic did this. What then? How would new models be trained? Who would build data centers for Anthropic?

@deredleritt3r · 2026-02-28 02:15
@__tinygrad__ Let's say Anthropic did this. What then? How would new models be trained? Who would build data centers for Anthropic?

@__tinygrad__ · 2026-02-28 02:17
@deredleritt3r We run Opus on tinyboxes, and AI is open source again! Haven't thought beyond that.

@pcgamer · 2026-02-27 13:47
A new California law says all operating systems, including Linux, need to have some form of age verification at account setup https://t.co/9qPq8EhtO4

@__tinygrad__ · 2026-02-28 03:02
@pcgamer Ahh yes, my Linux account. Where I log into big Linux.

@Jatin_exe · 2026-02-27 14:35
@__tinygrad__ @precuneanplexus @xai Didn't you hear alleged Tesla plans to use their fleet for compute ?

@__tinygrad__ · 2026-02-28 03:33
@Jatin_exe @precuneanplexus @xai I heard they also plan to put datacenters in space. Isn't there already a Tesla in space they can use?

@never_released · 2026-02-28 11:57
NVIDIA saying that they want as big of a die as possible to reduce inefficiency. And saying "wait for GTC" for more Groq news https://t.co/cfCj7r5W9Z

@gpuforthepoor · 2026-02-28 20:28
3 days into setting up @openclaw on a 96VRAM machine using Qwen, so far is mostly headaches. I thought this would work out of the box but tbh, it’s a lot of damn work @steipete

@__tinygrad__ · 2026-03-01 06:46
RT @pebaryan: with @__tinygrad__ i can now do federated train with both CUDA (RTX 5060ti) and WEBGPU/VULKAN (RX 9060xt) over HTTP/websocket. here's test run https://t.co/bErxD2oFNN

@__tinygrad__ · 2026-03-01 06:47
RT @CedricMakes: 3 days into working with @openclaw on a @__tinygrad__ tinybox 6x 4090 plugged into a residential power outlet SSHing in via Tailscale over Starlink. So far it's ez pz. It works better out of the box than I expected Making apps at the speed of thought while riding a bike 🚲🦞

@__tinygrad__ · 2026-03-01 06:47
RT @BioAlessandro: @__tinygrad__ should be taught in undergrad compiler classes instead of some dummy ~30 op language compiler that has more similarity to punchcards than current technology

@__tinygrad__ · 2026-03-01 06:47
RT @vsaitoo: It's dangerous now that it’s easy-ish to run a wild ass experiment just to learn? can I run the new qwen model on osx locally via my old 3090 in tinygrad vulkan -> llama.cpp -> egpu osx egpu worked, it's ~3x faster than the same on metal for stuff that can stay GPU resident?

@BeGrateful180 · 2026-02-27 23:44
@__tinygrad__ would be cool to see a collab with @pewdiepie. I think he would have fun working with a TinyBox.

@__tinygrad__ · 2026-03-01 06:49
@BeGrateful180 @pewdiepie if there's anyone in the world we'd give a discount to, it's @pewdiepie. he seems like he is having fun building, but if he wants a tinybox at cost, we'd make it happen

@BioAlessandro · 2026-02-28 20:15
@__tinygrad__ should be taught in undergrad compiler classes instead of some dummy ~30 op language compiler that has more similarity to punchcards than current technology

@__tinygrad__ · 2026-03-01 09:02
@BioAlessandro maybe George should put a course together. @HKUniversity want to host it? one semester, one time, "compilers for deep learning"

@IanCutress · 2026-03-01 13:25
Intel said the same thing.

@ai · 2026-03-01 13:43
Kimi K2.5 1T running on $10K of consumer hardware. AMD published a guide using Ryzen AI Max+ chips and a Linux kernel hack that pushes each node's VRAM from 96GB to 120GB. That's 480GB of unified GPU memory across the cluster, stitched together via llama.cpp RPC over Ethernet. The catch: it's slow. 8 tokens/sec and 90s to first token, roughly 6x slower and 90x higher latency than ChatGPT. This is a proof of architecture, not a product today. But between this and tools like @exolabs turning consumer devices into unified inference clusters, it's promising that consumer silicon can now hold trillion-parameter models that were datacenter-only 6 months ago. https://t.co/No2ZB5GFLY

@ai · 2026-03-01 13:43
Kimi K2.5 1T running on $10K of consumer hardware. AMD published a guide using Ryzen AI Max+ chips and a Linux kernel hack that pushes each node's VRAM from 96GB to 120GB. That's 480GB of unified GPU memory across the cluster, stitched together via llama.cpp RPC over Ethernet. The catch: it's slow. 8 tokens/sec and 90s to first token, roughly 6x slower and 90x higher latency than ChatGPT. This is a proof of architecture, not a product today. But between this and tools like @exolabs turning consumer devices into unified inference clusters, it's promising that consumer silicon can now hold trillion-parameter models that were datacenter-only 6 months ago. https://t.co/No2ZB5GFLY

@alexocheema · 2026-03-01 17:43
@ai Keep in mind this is Q2 (heavily quantized, quality will not be great). A single 512GB M3 Ultra Mac Studio ($9,499) can run that at >32 tok/sec (4x faster). AMD can be a contender here with: - more than 128GB memory on each device - tensor parallelism with RDMA

@r0ck3t23 · 2026-03-01 18:56
Dario Amodei just dismantled the biggest myth in the AI industry. Open source AI isn’t free. It never was. Amodei: “It’s not free. You have to run it on inference and someone has to make it fast on inference.” For decades, open source meant something real. It meant a teenager in a basement could download the same tools as a Fortune 500 company. Could read the code. Could modify it. Could build something that competed with the giants. That was genuine democratization. That actually happened. AI is different. Fundamentally. Physically. In ways the ideology hasn’t caught up to yet. Downloading the weights is the easy part. The part that actually costs something is turning the weights into a running system. Into responses. Into intelligence operating in real time at scale. That requires compute. Power. Infrastructure. The kind measured in billions of dollars and years of construction. Amodei: “These are big models. They’re hard to do inference on. Ultimately you have to host it on the cloud. The people who host it on the cloud do inference.” The open source debate was never about who owns the model. It was always about who owns the cloud. And Amodei goes further. When a competitor drops a new open model, he doesn’t ask whether it’s open or closed. He doesn’t care about the licensing. He doesn’t engage the ideology. Amodei: “I don’t think it mattered that DeepSeek is open source. I think I ask, is it a good model? Is it better than us at the things that matter? That’s the only thing that I care about.” That’s the ruthless clarity of someone actually trying to win. While the media debates licensing frameworks, Amodei is asking one question. Is it better. Everything else is a distraction. Amodei: “I don’t think open source works the same way in AI that it has worked in other areas. Here we can’t see inside the model.” This isn’t Linux. You can’t read it. You can’t fork it. You can’t understand it the way generations of developers understood the tools they inherited. You can download it. And then you need a data center to run it. The teenager in the basement who was supposed to be empowered by this revolution needs a billion dollars of infrastructure before the empowerment starts. The era of the basement coder rewriting civilization on a laptop is over. The future belongs to whoever commands the compute, owns the power grid, and can actually turn the intelligence on. Open weights without infrastructure isn’t democratization. It’s a promise the physics of the universe won’t let us keep.

@IanCutress · 2026-03-01 13:25
Intel said the same thing.

@handleym99 · 2026-03-01 20:39
@IanCutress This Intel? Honestly WTF is going on at the company? Are they just terminal liars? Terminally incompetent? Split into ten silos that absolutely never communicate with each other? https://t.co/qwmEIXDtdm

@r0ck3t23 · 2026-03-01 18:56
Dario Amodei just dismantled the biggest myth in the AI industry. Open source AI isn’t free. It never was. Amodei: “It’s not free. You have to run it on inference and someone has to make it fast on inference.” For decades, open source meant something real. It meant a teenager in a basement could download the same tools as a Fortune 500 company. Could read the code. Could modify it. Could build something that competed with the giants. That was genuine democratization. That actually happened. AI is different. Fundamentally. Physically. In ways the ideology hasn’t caught up to yet. Downloading the weights is the easy part. The part that actually costs something is turning the weights into a running system. Into responses. Into intelligence operating in real time at scale. That requires compute. Power. Infrastructure. The kind measured in billions of dollars and years of construction. Amodei: “These are big models. They’re hard to do inference on. Ultimately you have to host it on the cloud. The people who host it on the cloud do inference.” The open source debate was never about who owns the model. It was always about who owns the cloud. And Amodei goes further. When a competitor drops a new open model, he doesn’t ask whether it’s open or closed. He doesn’t care about the licensing. He doesn’t engage the ideology. Amodei: “I don’t think it mattered that DeepSeek is open source. I think I ask, is it a good model? Is it better than us at the things that matter? That’s the only thing that I care about.” That’s the ruthless clarity of someone actually trying to win. While the media debates licensing frameworks, Amodei is asking one question. Is it better. Everything else is a distraction. Amodei: “I don’t think open source works the same way in AI that it has worked in other areas. Here we can’t see inside the model.” This isn’t Linux. You can’t read it. You can’t fork it. You can’t understand it the way generations of developers understood the tools they inherited. You can download it. And then you need a data center to run it. The teenager in the basement who was supposed to be empowered by this revolution needs a billion dollars of infrastructure before the empowerment starts. The era of the basement coder rewriting civilization on a laptop is over. The future belongs to whoever commands the compute, owns the power grid, and can actually turn the intelligence on. Open weights without infrastructure isn’t democratization. It’s a promise the physics of the universe won’t let us keep.

@__tinygrad__ · 2026-03-02 03:41
@r0ck3t23 lol this is the biggest load of cope. I hope deepseek 4 is better than Opus. you can absolutely fork it and build on top of it. there's thousands of finetunes of open source models and zero finetunes of Claude.

@__tinygrad__ · 2026-03-02 03:49
TIL that Linux isn't free cause I had to buy the computer to run it 😭 This is the biggest load of cope I have ever heard. I hope Opus gets smoked by DeepSeek v4 and the only people who continue to use closed source models are Windows users.

@__tinygrad__ · 2026-03-02 03:49
TIL that Linux isn't free cause I had to buy the computer to run it 😭 This is the biggest load of cope I have ever heard. I hope Opus gets smoked by DeepSeek v4 and the only people who continue to use closed source models are Windows users.

@morew4rd · 2026-03-02 04:02
@__tinygrad__ To me it's obvious that he sees them as actual threats. As in to his business.

@__tinygrad__ · 2026-03-02 03:49
TIL that Linux isn't free cause I had to buy the computer to run it 😭 This is the biggest load of cope I have ever heard. I hope Opus gets smoked by DeepSeek v4 and the only people who continue to use closed source models are Windows users.

@__tinygrad__ · 2026-03-02 04:03
I'm not sure why any AI researchers continue to work at closed source labs. You know the money won't be worth anything. Hopefully it's clear now you won't get any control. And you are on the wrong side of history. Be a scientist, join a lab where you can publish.

@morew4rd · 2026-03-02 04:02
@__tinygrad__ To me it's obvious that he sees them as actual threats. As in to his business.

@__tinygrad__ · 2026-03-02 04:04
@morew4rd Breaking: company CEO worried that he won't be able to continue to rent seek on a GGUF file. More on this story at 7.

@__tinygrad__ · 2026-03-02 03:49
TIL that Linux isn't free cause I had to buy the computer to run it 😭 This is the biggest load of cope I have ever heard. I hope Opus gets smoked by DeepSeek v4 and the only people who continue to use closed source models are Windows users.

@max_paperclips · 2026-03-02 04:07
It's a microcosm of BS. Take his "black box " comment - we've got better interpretability tools all the time (steering vectors, probing, ablation libs), they're a pretty open book now. it'd be nice to have the actual training data, but we all know why we don't (sketchy IP, and Anthropic are as guilty as everyone else). 90% of the data used is on hf anyway

@max_paperclips · 2026-03-02 04:07
It's a microcosm of BS. Take his "black box " comment - we've got better interpretability tools all the time (steering vectors, probing, ablation libs), they're a pretty open book now. it'd be nice to have the actual training data, but we all know why we don't (sketchy IP, and Anthropic are as guilty as everyone else). 90% of the data used is on hf anyway

@__tinygrad__ · 2026-03-02 04:11
@max_paperclips It's not even like you can rerun the training, it's too expensive. Open weights with a good description of how it was trained is open source for this era. Cutting edge research should aspire to full reproducibility, but for big runs, weight and tech report are fine.

@__tinygrad__ · 2026-03-02 03:49
TIL that Linux isn't free cause I had to buy the computer to run it 😭 This is the biggest load of cope I have ever heard. I hope Opus gets smoked by DeepSeek v4 and the only people who continue to use closed source models are Windows users.

@JaimeOrtega · 2026-03-02 04:15
@__tinygrad__ “It’s not free. You have to run it on inference and someone has to make it fast on inference. For which I wold recommend a tinybox from the tiny corp.”

@JaimeOrtega · 2026-03-02 04:15
@__tinygrad__ “It’s not free. You have to run it on inference and someone has to make it fast on inference. For which I wold recommend a tinybox from the tiny corp.”

@__tinygrad__ · 2026-03-02 04:18
@JaimeOrtega and soon tinygrad to make it fast 😁

@__tinygrad__ · 2026-03-02 03:49
TIL that Linux isn't free cause I had to buy the computer to run it 😭 This is the biggest load of cope I have ever heard. I hope Opus gets smoked by DeepSeek v4 and the only people who continue to use closed source models are Windows users.

@auroter · 2026-03-02 07:50
@__tinygrad__ Masterclass in scammy insincere body language too. Qwen3.5 397B NVFP4 running quite well on the V2 Black, and doing a better job than Opus in many categories. Plus it won't randomly degrade to crap like Opus keeps doing a month after launching a new version.

@__tinygrad__ · 2026-03-02 07:54
Want to make sure your models don't degrade in performance when the cloud needs more capacity? Own your own computer! Don't dial in to their timeshare.

@__tinygrad__ · 2026-03-02 07:54
Want to make sure your models don't degrade in performance when the cloud needs more capacity? Own your own computer! Don't dial in to their timeshare.

@__tinygrad__ · 2026-03-02 07:57
I so much wish we could make tinyboxes cheaper and still have a business so future people can buy tinyboxes too. Price of the Blackwell cards went up, and I don't even really blame NVIDIA this time. They have to pay for that 96GB of RAM.

@__tinygrad__ · 2026-03-02 03:49
TIL that Linux isn't free cause I had to buy the computer to run it 😭 This is the biggest load of cope I have ever heard. I hope Opus gets smoked by DeepSeek v4 and the only people who continue to use closed source models are Windows users.

@alpaimdev · 2026-03-02 09:38
@__tinygrad__ I wouldn't be surprised if we face issues with getting/distributing open weight/source models soon. Imagine huggingface lose its infinite investment money to cover bandwidth bills for example. Imagine hardfork of CUDA that finally separates consumer and server GPUs

@alpaimdev · 2026-03-02 09:38
@__tinygrad__ I wouldn't be surprised if we face issues with getting/distributing open weight/source models soon. Imagine huggingface lose its infinite investment money to cover bandwidth bills for example. Imagine hardfork of CUDA that finally separates consumer and server GPUs

@__tinygrad__ · 2026-03-02 09:45
@alpaimcom For the former, if only there was a protocol battle tested through 2 decades of piracy for distributing large files. For the latter, perhaps a deep learning framework that bypasses CUDA 😁

@BigTechAlert · 2026-03-02 12:42
🚫 @elonmusk is no longer following @Yuhu_ai_ https://t.co/rcePcCKALW

@alexocheema · 2026-03-01 17:43
@ai Keep in mind this is Q2 (heavily quantized, quality will not be great). A single 512GB M3 Ultra Mac Studio ($9,499) can run that at >32 tok/sec (4x faster). AMD can be a contender here with: - more than 128GB memory on each device - tensor parallelism with RDMA

@__tinygrad__ · 2026-03-02 12:53
@alexocheema @ai AMD can be a contender with an RDNA5 card with 96GB of DDR7 for a reasonable price. Then the Mac game will be over, it's wild that it's competitive now.

@tylercowen · 2026-03-02 13:29
"In the longer run, the legacy of Hegseth here will be to diminish the say of the military over AI developments, and increase the role of Congress, a possibly Democratic Congress at that. Exactly who is it that should be happy here?" https://t.co/cDbTCUe8xI

@teortaxesTex · 2026-03-02 14:03
After you're done burning out for Elon, he jettisons you like a non-reusable rocket stage. Admirable commitment to the bit

@teortaxesTex · 2026-03-02 14:03
After you're done burning out for Elon, he jettisons you like a non-reusable rocket stage. Admirable commitment to the bit

@scaling01 · 2026-03-02 14:06
@teortaxesTex honestly looks like it's gg for xai with everyone quitting

@mweinbach · 2026-03-02 15:30
Qualcomm’s AI 200 accelerator rack! It’s real! It’s ~43TB of memory per rack, 142kW, liquid cooled and just a great rack https://t.co/0qVTwW736l

@ryanshrout · 2026-03-02 17:10
Got to see the @Qualcomm AI200 rack in person on the floor at MWC. It packs 56x AI accelerators with 768GB of LPDDR5 memory each, total over 43TB in the full rack. About 140 kW of max power draw. Powered with AMD EPYC CPUs I’m told. Looks QC is serious about this! https://t.co/Q5IHvjEZug

@balajis · 2026-03-02 17:33
I disagree. Yes, Congressional Democrats do want to stop AI, because it disrupts blue jobs. And Republicans do want a military-friendly AI. But China wants to open source AI, because the Chinese make money from AI-enabled hardware instead. The rest of the world wants open source models as well. That’s likely where things land up, once model capabilities top out. America is essentially serving as the bootloader for AI, spending billions to give the world one last incredible gift before turning the lights out on Silicon Valley. Because with wealth taxes and visa restrictions, one can no longer easily concentrate talent and capital in Silicon Valley. That’s why Zuck, Page, Brin, Thiel, and Elon got out. We have one last round of IPOs, and then the seed corn is gone. Moreover, once spread to the four winds, the Silicon Valley network effect is up for grabs. And the anti-tech sentiment isn’t localized to California. There is building bipartisan animus in America towards technologists as a class. So: neither Blue America, nor Red America, nor Tech America is going to control AI in the long run. It’s just going to decentralize. Indeed, the first wave of AI decentralization is already here.

@ryanshrout · 2026-03-02 17:10
Got to see the @Qualcomm AI200 rack in person on the floor at MWC. It packs 56x AI accelerators with 768GB of LPDDR5 memory each, total over 43TB in the full rack. About 140 kW of max power draw. Powered with AMD EPYC CPUs I’m told. Looks QC is serious about this! https://t.co/Q5IHvjEZug

@handleym99 · 2026-03-02 22:02
They alienated every phone maker by demanding absurd royalties. They alienated MS by imagining they could just ignore the Windows driver model by sneering at it, and somehow magically Windows would adopt the Linux driver model. How will they alienate the hyperscalers? Not sure yet, but don't worry, they will.

@balajis · 2026-03-02 17:33
I disagree. Yes, Congressional Democrats do want to stop AI, because it disrupts blue jobs. And Republicans do want a military-friendly AI. But China wants to open source AI, because the Chinese make money from AI-enabled hardware instead. The rest of the world wants open source models as well. That’s likely where things land up, once model capabilities top out. America is essentially serving as the bootloader for AI, spending billions to give the world one last incredible gift before turning the lights out on Silicon Valley. Because with wealth taxes and visa restrictions, one can no longer easily concentrate talent and capital in Silicon Valley. That’s why Zuck, Page, Brin, Thiel, and Elon got out. We have one last round of IPOs, and then the seed corn is gone. Moreover, once spread to the four winds, the Silicon Valley network effect is up for grabs. And the anti-tech sentiment isn’t localized to California. There is building bipartisan animus in America towards technologists as a class. So: neither Blue America, nor Red America, nor Tech America is going to control AI in the long run. It’s just going to decentralize. Indeed, the first wave of AI decentralization is already here.

@__tinygrad__ · 2026-03-03 00:37
@balajis export controlling NVIDIA was the nail in the coffin. we can only ship our top tinybox to a small set of whitelisted countries. in 2-3 years, we'll be shipping tinyboxes full of Chinese chips.

@scaling01 · 2026-03-02 14:06
@teortaxesTex honestly looks like it's gg for xai with everyone quitting

@__tinygrad__ · 2026-03-03 00:53
@scaling01 @teortaxesTex What are all the GPUs up to? @elonmusk should have xai do open source models, compete with China, build the real OpenAI and leave closed source in the dust

@__tinygrad__ · 2026-03-03 00:53
@scaling01 @teortaxesTex What are all the GPUs up to? @elonmusk should have xai do open source models, compete with China, build the real OpenAI and leave closed source in the dust

@vega_holdings · 2026-03-03 01:32
@__tinygrad__ @scaling01 @teortaxesTex @elonmusk which would be based but we saw how long it took to 'opensource' grok

@ryanshrout · 2026-03-02 17:10
Got to see the @Qualcomm AI200 rack in person on the floor at MWC. It packs 56x AI accelerators with 768GB of LPDDR5 memory each, total over 43TB in the full rack. About 140 kW of max power draw. Powered with AMD EPYC CPUs I’m told. Looks QC is serious about this! https://t.co/Q5IHvjEZug

@__tinygrad__ · 2026-03-03 02:08
@ryanshrout @Qualcomm You have to use SNPE to compile for this?

@handleym99 · 2026-03-02 22:02
They alienated every phone maker by demanding absurd royalties. They alienated MS by imagining they could just ignore the Windows driver model by sneering at it, and somehow magically Windows would adopt the Linux driver model. How will they alienate the hyperscalers? Not sure yet, but don't worry, they will.

@__tinygrad__ · 2026-03-03 02:10
@handleym99 @ryanshrout @Qualcomm This. They are one of the worst companies @comma_ai ever dealt with. And it's not their chips or docs, those are great. It's the 80s era sales org and the rent seeking patent division.

@vega_holdings · 2026-03-03 01:32
@__tinygrad__ @scaling01 @teortaxesTex @elonmusk which would be based but we saw how long it took to 'opensource' grok

@__tinygrad__ · 2026-03-03 02:56
@vega_holdings @scaling01 @teortaxesTex @elonmusk Elon lacks on the open source front. It's a way to build institutional power, not sure why it's neglected

@__tinygrad__ · 2026-03-03 03:49
AMD open sourced rocprof-trace-decoder! This was one of the last pieces of closed source code on the CPU side -- the definitions of the hardware SQTT traces are now public. AMD's tracing infrastructure is better than NVIDIA's, it can trace the timing of every instruction.

@__tinygrad__ · 2026-03-03 03:49
AMD open sourced rocprof-trace-decoder! This was one of the last pieces of closed source code on the CPU side -- the definitions of the hardware SQTT traces are now public. AMD's tracing infrastructure is better than NVIDIA's, it can trace the timing of every instruction.

@__tinygrad__ · 2026-03-03 03:50
And now it's documented. For people serious about performance, this is a real AMD advantage. Thanks @AnushElangovan for pushing this through! Code is here: https://t.co/WEEj9EVZe8

@__tinygrad__ · 2026-03-03 03:52
@Ambroise23968 Our reverse engineering was decent, but time consuming and incomplete around the edges. It's so nice to have real docs. https://t.co/bt2Q185K1q

@__tinygrad__ · 2026-03-03 03:49
AMD open sourced rocprof-trace-decoder! This was one of the last pieces of closed source code on the CPU side -- the definitions of the hardware SQTT traces are now public. AMD's tracing infrastructure is better than NVIDIA's, it can trace the timing of every instruction.

@desert_mouse · 2026-03-03 04:21
@__tinygrad__ Now please focus on the new Mac ANE unlock for training and inference https://t.co/WDyFtQMU25

@__tinygrad__ · 2026-03-03 03:49
AMD open sourced rocprof-trace-decoder! This was one of the last pieces of closed source code on the CPU side -- the definitions of the hardware SQTT traces are now public. AMD's tracing infrastructure is better than NVIDIA's, it can trace the timing of every instruction.

@abstractlysaid · 2026-03-03 04:38
@__tinygrad__ how is it better than nvidia's?

@chesterzelaya · 2026-03-03 05:26
any recs on a eGPU enclosure that works with this? couldn’t find any on Amazon https://t.co/qWM3Qd4JHN

@abstractlysaid · 2026-03-03 04:38
@__tinygrad__ how is it better than nvidia's?

@__tinygrad__ · 2026-03-03 07:17
@DamiDina NVIDIA has a sampling profiler that only traces like 5% of the instructions, and afaik the hardware isn't there for something like SQTT

@chesterzelaya · 2026-03-03 05:26
any recs on a eGPU enclosure that works with this? couldn’t find any on Amazon https://t.co/qWM3Qd4JHN

@__tinygrad__ · 2026-03-03 07:20
@chesterzelaya if you are serious, we'd be open to a contract to support this over USB4 on Mac.

@desert_mouse · 2026-03-03 04:21
@__tinygrad__ Now please focus on the new Mac ANE unlock for training and inference https://t.co/WDyFtQMU25

@__tinygrad__ · 2026-03-03 08:32
@desert_mouse I don't own a Mac. Switched to Strix Halo!

@handleym99 · 2026-03-01 20:39
@IanCutress This Intel? Honestly WTF is going on at the company? Are they just terminal liars? Terminally incompetent? Split into ten silos that absolutely never communicate with each other? https://t.co/qwmEIXDtdm

@__tinygrad__ · 2026-03-03 08:36
@handleym99 @IanCutress lol no need to bring malice into it, but pretty much. @intel has 0 leadership in AI, nobody who can set a roadmap and stick to it.

@__tinygrad__ · 2026-03-03 12:46
Stop relying on other people's computer. Skip the downtime by owning your own. #tinybox https://t.co/OmUGzRw3Ck

@__tinygrad__ · 2026-03-03 12:46
Stop relying on other people's computer. Skip the downtime by owning your own. #tinybox https://t.co/OmUGzRw3Ck

@BillQueens · 2026-03-03 12:47
@__tinygrad__ Been qwen3.5ing my balls off. Rips on 5090.

@__tinygrad__ · 2026-03-03 12:46
Stop relying on other people's computer. Skip the downtime by owning your own. #tinybox https://t.co/OmUGzRw3Ck

@elephantum · 2026-03-03 12:54
@__tinygrad__ Do you, by any chance, have a link where I can download Opus 4.6 to run locally?

@elephantum · 2026-03-03 12:54
@__tinygrad__ Do you, by any chance, have a link where I can download Opus 4.6 to run locally?

@__tinygrad__ · 2026-03-03 13:04
@elephantum .@DarioAmodei want to post a torrent link?

@__tinygrad__ · 2026-03-03 13:59
RT @Leik0w0: Tried tinygrad viz for the first time in a while, it’s so nice! (Very snappy too :)

@__tinygrad__ · 2026-03-03 02:55
@mweinbach Except that you have to use SNPE to run models. Have you ever tried SNPE? Also, do they have a list price, or do you have to "contact sales" who would like to know the revenue of your company, your projections, and have you offer a 20% discount to Qualcomm employees.

@JDorbal1989 · 2026-03-03 15:17
@__tinygrad__ @mweinbach This isn't true. SNPE is going away for QNN and QAIRT.

@JDorbal1989 · 2026-03-03 15:17
@__tinygrad__ @mweinbach This isn't true. SNPE is going away for QNN and QAIRT.

@__tinygrad__ · 2026-03-03 15:21
@JDorbal1989 @mweinbach ahh great we love two new random Qualcomm acronyms

@__tinygrad__ · 2026-03-03 15:21
@JDorbal1989 @mweinbach ahh great we love two new random Qualcomm acronyms

@JDorbal1989 · 2026-03-03 15:24
@__tinygrad__ @mweinbach FWIW I played around a bit with the new stuff and it's not as horrible as SNPE. It seems they have real teams who aren't donkeys attempting to drag the company in the correct direction

@JDorbal1989 · 2026-03-03 15:24
@__tinygrad__ @mweinbach FWIW I played around a bit with the new stuff and it's not as horrible as SNPE. It seems they have real teams who aren't donkeys attempting to drag the company in the correct direction

@__tinygrad__ · 2026-03-03 15:28
@JDorbal1989 @mweinbach do they have a price for the new chip? or is it get on a call and you can only buy it if you go to two lunches with the rep and license some LTE patents? most of the problem with QCOM is their sales dept

@gregjoz · 2026-03-03 15:37
The all-new MacBook Pro with M5 Pro and M5 Max pushes the boundaries of what you can accomplish from anywhere. Run advanced large language models on device and unlock capabilities that can't be done on any other laptop—all while maintaining exceptional battery life! https://t.co/t3f3SQrFb9

@BillQueens · 2026-03-03 12:47
@__tinygrad__ Been qwen3.5ing my balls off. Rips on 5090.

@__tinygrad__ · 2026-03-03 15:39
@BillQueens Qwen 3.5 is adding itself to tinygrad https://t.co/hZxdBqWqzG

@dhh · 2026-03-03 17:07
Many thanks to @MichaelDell for having one of the new Panther Lake XPS 16 laptops sent over for Omarchy testing. There's a bit of work to do, but it's already very usable, and that tandem OLED is to die for 🤩 https://t.co/AKeIMrMAOU

@JoshKale · 2026-03-03 17:44
This might be the funniest chart in tech right now. Apple's capex strategy has to be the luckiest accident in history: Amazon, Microsoft, Meta, Google, are in a spending arms race plowing over $100B PER QUARTER into data centers - While Apple spending is down 19% Meanwhile: - Mac Minis sold out bc everyone's buying them to run OpenClaw - Mac Studios have a 6-week backlog - Someone ran Qwen 3.5 on an iPhone yesterday - The M5 Max just shipped with 128GB of unified memory and runs Llama 70B from anywhere The company spending the least on AI infrastructure accidentally became the AI infrastructure

@huybery · 2026-03-03 23:30
bye qwen, me too.

@scaling01 · 2026-03-03 23:40
guys can we please slow down. we are at the beginning of march and: - xAI imploded - Qwen is imploding - Anthropic is a supply chain risk - OpenAI is now seen on the same level as the NSA - Google now 2nd tier behind OpenAI and Anthropic I can't do 10 more months of this

@JoshKale · 2026-03-03 17:44
This might be the funniest chart in tech right now. Apple's capex strategy has to be the luckiest accident in history: Amazon, Microsoft, Meta, Google, are in a spending arms race plowing over $100B PER QUARTER into data centers - While Apple spending is down 19% Meanwhile: - Mac Minis sold out bc everyone's buying them to run OpenClaw - Mac Studios have a 6-week backlog - Someone ran Qwen 3.5 on an iPhone yesterday - The M5 Max just shipped with 128GB of unified memory and runs Llama 70B from anywhere The company spending the least on AI infrastructure accidentally became the AI infrastructure

@__tinygrad__ · 2026-03-04 00:19
@JoshKale don't buy cloud, sell computer!

@scaling01 · 2026-03-03 23:40
guys can we please slow down. we are at the beginning of march and: - xAI imploded - Qwen is imploding - Anthropic is a supply chain risk - OpenAI is now seen on the same level as the NSA - Google now 2nd tier behind OpenAI and Anthropic I can't do 10 more months of this

@__tinygrad__ · 2026-03-04 02:42
@scaling01 who can be the Schelling point for the new vanguard of open source?

@alexocheema · 2026-03-04 03:42
Nobody is talking about @apple keeping prices the same for the 128GB MacBook Pro. There has been no price increase in response to surging memory prices. Everyone is talking about the boost in compute, speeding up prefill by 4x. This is cool but practically it’s not that big of a deal. Why? Because on your own computer, most apps/tools using LLMs are going to get high kv cache hit rates - that means as a user you only experience slow prefill once. kv cache can be persisted to disk and loaded at 6GB/s. Most time in LLM inference is spent on decode, which is memory bandwidth bound. It’s still great for image/video generation, high batch LLM inference and fine-tuning, which are compute bound. We should see huge speedups there. Apple’s AI strategy is on-device LLMs and here, memory is the name of the game, not FLOPS. Expect the same for M5 Pro/Max Mac Mini and M5 Ultra Mac Studio. That means 512GB M5 Ultra at 10k! @tim_cook is a supply chain genius.

@__tinygrad__ · 2026-03-04 03:43
RT @kristoph: @TylerSwift29 @wholyv Tinygrad is really good to look at if you want to understand the “compile to kernel” strategy. Almost all the calls are lazy until you call realize.

@alexocheema · 2026-03-04 03:42
Nobody is talking about @apple keeping prices the same for the 128GB MacBook Pro. There has been no price increase in response to surging memory prices. Everyone is talking about the boost in compute, speeding up prefill by 4x. This is cool but practically it’s not that big of a deal. Why? Because on your own computer, most apps/tools using LLMs are going to get high kv cache hit rates - that means as a user you only experience slow prefill once. kv cache can be persisted to disk and loaded at 6GB/s. Most time in LLM inference is spent on decode, which is memory bandwidth bound. It’s still great for image/video generation, high batch LLM inference and fine-tuning, which are compute bound. We should see huge speedups there. Apple’s AI strategy is on-device LLMs and here, memory is the name of the game, not FLOPS. Expect the same for M5 Pro/Max Mac Mini and M5 Ultra Mac Studio. That means 512GB M5 Ultra at 10k! @tim_cook is a supply chain genius.

@__tinygrad__ · 2026-03-04 03:44
@alexocheema @Apple I never dreamed of a world where the Apple RAM markup would be the best deal around.

@Yuchenj_UW · 2026-03-04 05:01
Wild but plausible idea: What if VCs in the US invest in the Qwen core team that just left? The US could suddenly have a new AI lab building frontier open-source models that directly compete with the Chinese OSS models.

@Yuchenj_UW · 2026-03-04 05:01
Wild but plausible idea: What if VCs in the US invest in the Qwen core team that just left? The US could suddenly have a new AI lab building frontier open-source models that directly compete with the Chinese OSS models.

@__tinygrad__ · 2026-03-04 08:28
@Yuchenj_UW .@huybery @JustinLin610 how much cash and GPUs to we need? Want to move to Hong Kong? Would love to have a fully open training stack, not just weights. And I have a business model idea to do it.

@__tinygrad__ · 2026-03-04 08:28
@Yuchenj_UW .@huybery @JustinLin610 how much cash and GPUs to we need? Want to move to Hong Kong? Would love to have a fully open training stack, not just weights. And I have a business model idea to do it.

@__tinygrad__ · 2026-03-04 08:30
@Yuchenj_UW @huybery @JustinLin610 Business model is this. Many people are going to run the open weight models. There's a lot of people who will pay for influence in the model, LLM SEO so to say. It's basically influencer advertising.

@__tinygrad__ · 2026-03-04 08:30
@Yuchenj_UW @huybery @JustinLin610 Business model is this. Many people are going to run the open weight models. There's a lot of people who will pay for influence in the model, LLM SEO so to say. It's basically influencer advertising.

@__tinygrad__ · 2026-03-04 08:34
@Yuchenj_UW @huybery @JustinLin610 This way, we make the training stack fully open (enabling forking). But most people just use the weights complete with marketing copy in the train set cause it's expensive to rerun. When OpenClaw on a Mac Mini chooses what pizza to order, get the door, it's @dominos.

@__tinygrad__ · 2026-03-04 08:34
@Yuchenj_UW @huybery @JustinLin610 This way, we make the training stack fully open (enabling forking). But most people just use the weights complete with marketing copy in the train set cause it's expensive to rerun. When OpenClaw on a Mac Mini chooses what pizza to order, get the door, it's @dominos.

@opdroid1234 · 2026-03-04 08:37
@__tinygrad__ @Yuchenj_UW @huybery @JustinLin610 @dominos https://t.co/xMEnpjW4M8

@opdroid1234 · 2026-03-04 08:37
@__tinygrad__ @Yuchenj_UW @huybery @JustinLin610 @dominos https://t.co/xMEnpjW4M8

@__tinygrad__ · 2026-03-04 08:40
@opdroid1234 @Yuchenj_UW @huybery @JustinLin610 @dominos It looks like you are looking to talk through your problems. While our open models can talk to you like a friend, there's no shame in reaching out to a professional. Would you like me to reach out to @betterhelp for you? They have a team of trained therapists standing by.

@dhh · 2026-03-03 17:07
Many thanks to @MichaelDell for having one of the new Panther Lake XPS 16 laptops sent over for Omarchy testing. There's a bit of work to do, but it's already very usable, and that tandem OLED is to die for 🤩 https://t.co/AKeIMrMAOU

@__tinygrad__ · 2026-03-04 08:48
@dhh @MichaelDell They should make a Strix Halo one!

@__tinygrad__ · 2026-03-04 14:15
“Simplicity is a great virtue, but it requires hard work to achieve and education to appreciate. And to make matters worse, complexity sells better.” — Edsger Dijkstra

@__tinygrad__ · 2026-03-04 14:15
“Simplicity is a great virtue, but it requires hard work to achieve and education to appreciate. And to make matters worse, complexity sells better.” — Edsger Dijkstra

@paulopacitti · 2026-03-04 14:31
@__tinygrad__ crazy sync @pikuma https://t.co/0yMH9RlQ4k

@paulopacitti · 2026-03-04 14:31
@__tinygrad__ crazy sync @pikuma https://t.co/0yMH9RlQ4k

@__tinygrad__ · 2026-03-04 14:33
@paulopacitti @pikuma lol this is the top story on HN with the quote in it https://t.co/7kwAsmch8I

@thdxr · 2026-03-04 15:40
there are some ai labs that constantly say things like "you haven't seen what we've seen you're not ready for the exponential progress" and other ones never say anything like this and project incremental changes what is the explanation

@DanielLockyer · 2026-03-04 20:05
It's sad that we promote and encourage complexity in the tech industry I'd love to see more of a push towards simple solutions to problems https://t.co/M4C7IrpIWG

@DanielLockyer · 2026-03-04 20:24
Software is literally doing the opposite of this https://t.co/TB5UJ4Bvn9

@__tinygrad__ · 2026-03-04 23:15
RT @jino_rohit: @qubitium tinygrad is the only non bloated codebase ive seen

@thdxr · 2026-03-04 15:40
there are some ai labs that constantly say things like "you haven't seen what we've seen you're not ready for the exponential progress" and other ones never say anything like this and project incremental changes what is the explanation

@__tinygrad__ · 2026-03-04 23:45
@thdxr the former type was trying to sell you a crypto coin a few years ago

@austingriffith · 2026-03-04 23:51
⚙️ Are the "local ai" guys basically just larping? 🐏 I have a machine with 128GB of RAM 🤖 I downloaded qwen3.5:122b 🤣 It takes 3 minutes to incorrectly do what sonnet 4.6 can do in seconds 😅 Am I doing it wrong? Do I need more RAM?

@DanielLockyer · 2026-03-04 20:24
Software is literally doing the opposite of this https://t.co/TB5UJ4Bvn9

@__tinygrad__ · 2026-03-05 00:15
@DanielLockyer not tinygrad

@cryptopunk7213 · 2026-03-05 00:49
i find it fucking hilarious how Apple "failing" at AI is now the exact reason they're about to win it: - watched everyone else burn $1.4T+ building models... then picked the winner (gemini) to use for... $1B - while everyone fights to grow users, apple flips a switch and 2.5 billion devices get AI siri tmrw. - $150B to splurge on the device / app layer. zero competition (because everyones spent their cash). - while openAI charges $200/mo subscriptions, Apple lets you run models on-device (cheaper, faster, private, personal) - while openAI struggles to build an AI device, Apple just dropped 5 powered by the best AI chips for hand-held devices. they "lost" the model race because they didn't need to win it in the first place greatest to (accidentally) ever do it.

@__tinygrad__ · 2026-03-05 02:05
@austingriffith your machine doesn't have ram bandwidth so it's slow. and you need the bigger qwen to be on par with sonnet. local AI is real and decent, but you need $50k minimum for a machine. budgets below that are larping

@0x7FFE0000 · 2026-03-05 02:17
@__tinygrad__ @austingriffith What’s the cost decline curve look like… 20 years until that power is in a raspberry pi priced device?

@__tinygrad__ · 2026-03-05 02:05
@austingriffith your machine doesn't have ram bandwidth so it's slow. and you need the bigger qwen to be on par with sonnet. local AI is real and decent, but you need $50k minimum for a machine. budgets below that are larping

@OneGreatGambino · 2026-03-05 03:05
@__tinygrad__ @austingriffith Just get a Spark for large models it works amazing.

@0x7FFE0000 · 2026-03-05 02:17
@__tinygrad__ @austingriffith What’s the cost decline curve look like… 20 years until that power is in a raspberry pi priced device?

@__tinygrad__ · 2026-03-05 03:27
@0x7FFE0000 @austingriffith What does the model growth curve look like?

@OneGreatGambino · 2026-03-05 03:05
@__tinygrad__ @austingriffith Just get a Spark for large models it works amazing.

@__tinygrad__ · 2026-03-05 03:28
@OneGreatGambino @austingriffith lol this is the exact larping the OP addresses

@cryptopunk7213 · 2026-03-05 00:49
i find it fucking hilarious how Apple "failing" at AI is now the exact reason they're about to win it: - watched everyone else burn $1.4T+ building models... then picked the winner (gemini) to use for... $1B - while everyone fights to grow users, apple flips a switch and 2.5 billion devices get AI siri tmrw. - $150B to splurge on the device / app layer. zero competition (because everyones spent their cash). - while openAI charges $200/mo subscriptions, Apple lets you run models on-device (cheaper, faster, private, personal) - while openAI struggles to build an AI device, Apple just dropped 5 powered by the best AI chips for hand-held devices. they "lost" the model race because they didn't need to win it in the first place greatest to (accidentally) ever do it.

@__tinygrad__ · 2026-03-05 04:09
@cryptopunk7213 Apple bets on people owning computers. It's a good bet. Others bet on 5 computers in the world. They get mogged like the 1950s again

@__tinygrad__ · 2026-03-05 02:05
@austingriffith your machine doesn't have ram bandwidth so it's slow. and you need the bigger qwen to be on par with sonnet. local AI is real and decent, but you need $50k minimum for a machine. budgets below that are larping

@amootpoint · 2026-03-05 06:12
@__tinygrad__ @austingriffith RTX 6000 should be enough, eh ?

@amootpoint · 2026-03-05 06:12
@__tinygrad__ @austingriffith RTX 6000 should be enough, eh ?

@__tinygrad__ · 2026-03-05 06:12
@amootpoint @austingriffith 4 of em should

@__tinygrad__ · 2026-03-05 08:22
tinygrad's built in 321 line LLM server is now fast. This is on one 9070XT in a tinybox red. Try it. It has 0 deps, and includes the tokenizer and web page in the file. https://t.co/ZV2afa09YI

@__tinygrad__ · 2026-03-05 08:26
@shr1ftyy now to get Qwen3.5-397B-A17B running on a MI350X

@__tinygrad__ · 2026-03-05 14:31
RT @BioAlessandro: Building a compiler + HSL framework to turn @__tinygrad__ kernels into VHDL, and synthesize the perfect FPGA for a given compute graph. Tinygrad UOps -> KernelIR (my custom IR) -> Amaranth hardware modules https://t.co/9YhncOo0s8

@__tinygrad__ · 2026-03-06 02:55
We have a tinybox green v2 blackwell in hand, in stock. Order today, ships tomorrow! 384GB of super fast memory.

@__tinygrad__ · 2026-03-06 02:55
We have a tinybox green v2 blackwell in hand, in stock. Order today, ships tomorrow! 384GB of super fast memory.

@AndyAyrey · 2026-03-06 03:28
@__tinygrad__ DMs

@AndyAyrey · 2026-03-06 03:28
@__tinygrad__ DMs

@__tinygrad__ · 2026-03-06 04:09
@AndyAyrey We don't do sales or custom requests. Everything is explained on the website, and if there's a bug, feel free to report it in public on twitter or our discord.

@__tinygrad__ · 2026-03-06 04:09
@AndyAyrey We don't do sales or custom requests. Everything is explained on the website, and if there's a bug, feel free to report it in public on twitter or our discord.

@AndyAyrey · 2026-03-06 04:14
@__tinygrad__ do you ship internationally was my main question which is not on your website

@AndyAyrey · 2026-03-06 04:14
@__tinygrad__ do you ship internationally was my main question which is not on your website

@__tinygrad__ · 2026-03-06 06:15
@AndyAyrey that's very much on the website. the full list of countries is there too. it's in the obvious place

@__tinygrad__ · 2026-03-06 06:15
@AndyAyrey that's very much on the website. the full list of countries is there too. it's in the obvious place

@AndyAyrey · 2026-03-06 08:37
@__tinygrad__ yes, i eventually found the answers to my questions on the page for the tinybox red. your rtx6000 box just says "has rtxes" and "ships 2-8 weeks" still unclear on lead time- is it 1 week till shipping (your homepage), 2-4 days, or 2-8 weeks? does that change with volume? https://t.co/v6niGcMJfl

@AndyAyrey · 2026-03-06 08:37
@__tinygrad__ yes, i eventually found the answers to my questions on the page for the tinybox red. your rtx6000 box just says "has rtxes" and "ships 2-8 weeks" still unclear on lead time- is it 1 week till shipping (your homepage), 2-4 days, or 2-8 weeks? does that change with volume? https://t.co/v6niGcMJfl

@__tinygrad__ · 2026-03-06 08:47
@AndyAyrey I think the answer is pretty clear on the product page. it's within 8 weeks. if you have a lot of questions, you might want to go with a higher priced manufacturer that spends a bigger fraction of profits on sales.

@__tinygrad__ · 2026-03-06 08:47
@AndyAyrey I think the answer is pretty clear on the product page. it's within 8 weeks. if you have a lot of questions, you might want to go with a higher priced manufacturer that spends a bigger fraction of profits on sales.

@downtiMAK · 2026-03-06 10:13
@__tinygrad__ @AndyAyrey lol no one is asking you to spend a big fraction on sales. this guy is looking to buy several boxes from you, just try to be a bit normal.

@Yuchenj_UW · 2026-03-07 02:38
Singularity. I’m sure Codex is 100% written by Codex too. https://t.co/PTJ1iDfSti

@__tinygrad__ · 2026-03-07 03:45
Working on a spec for tinygrad. There's still a few things duplicated and messy in the code (dtype.vec should be shape, multi shouldn't be a thing) but it's getting close to complete. Spec currently has 40 ops. https://t.co/2D8dGdMEnn

@__tinygrad__ · 2026-03-07 03:45
Working on a spec for tinygrad. There's still a few things duplicated and messy in the code (dtype.vec should be shape, multi shouldn't be a thing) but it's getting close to complete. Spec currently has 40 ops. https://t.co/2D8dGdMEnn

@__tinygrad__ · 2026-03-07 03:46
Full spec here. If you think we are missing something, file an issue. https://t.co/kiUAXKBtfP

@downtiMAK · 2026-03-06 10:13
@__tinygrad__ @AndyAyrey lol no one is asking you to spend a big fraction on sales. this guy is looking to buy several boxes from you, just try to be a bit normal.

@__tinygrad__ · 2026-03-07 03:49
@downtiMAK @AndyAyrey Doubtful. In my experience, the people serious about buying just buy. But we understand we aren't for everyone, and some people really like the "Contact Sales" experience instead of a price and a buy it now button.

@__tinygrad__ · 2026-03-07 03:45
Working on a spec for tinygrad. There's still a few things duplicated and messy in the code (dtype.vec should be shape, multi shouldn't be a thing) but it's getting close to complete. Spec currently has 40 ops. https://t.co/2D8dGdMEnn

@graykevinb · 2026-03-07 03:55
@__tinygrad__ Will the chips you make then support these 40 ops as native instructions? TinyGrad could become an instruction set

@graykevinb · 2026-03-07 03:55
@__tinygrad__ Will the chips you make then support these 40 ops as native instructions? TinyGrad could become an instruction set

@__tinygrad__ · 2026-03-07 04:02
@graykevinb Not really, many of the ops don't make sense as instructions. The instruction set for the tiny VLIW machine will be even smaller than 40. Processors don't operate on graphs, they operate through time, and they have fixed register sizes.

@Yuchenj_UW · 2026-03-07 02:38
Singularity. I’m sure Codex is 100% written by Codex too. https://t.co/PTJ1iDfSti

@__tinygrad__ · 2026-03-07 04:23
@Yuchenj_UW I've heard computers are also designed using computers.

@__tinygrad__ · 2026-03-07 15:09
if tiny corp was raising $20M (@ $200M), who'd be interested? business model is basically this. buy this $11.5M building (with 5MW of power): link in our discord wait for AMD to launch the RDNA5 96GB cards (mid 2027). preorder 3000 cards (hopefully we can negotiate for $2500 each), build 500 $20k tinyboxes with 6 of the card. run all the chinese llms. make $600k / month revenue selling tokens on openrouter (market depth is there, this is 1% of openrouter). improvements to tinygrad yield revenue improvements. due to how power is priced in oregon it's only like $50k for the electric bill (below 4MW they price for peak, not usage, we get like 3c kWh power). we can also make ~$100k / month leasing colo space to comma. building and cards paid off in 3 years max, investment made back. low risk of being undercut since we're using consumer GPUs and running the cheapest colo you can believe. if someone chill wants in, i'd do it. i'm not gonna hype fake tech, but demand for tokens is going to skyrocket (look at the openclaw install numbers). with crazy good optimizations we could potentially get 3x more from the machines, and we have electricity for 3x more machines. $5.4M revenue per month. then continue to scale from there, custom chips, etc...

@__tinygrad__ · 2026-03-07 15:09
if tiny corp was raising $20M (@ $200M), who'd be interested? business model is basically this. buy this $11.5M building (with 5MW of power): link in our discord wait for AMD to launch the RDNA5 96GB cards (mid 2027). preorder 3000 cards (hopefully we can negotiate for $2500 each), build 500 $20k tinyboxes with 6 of the card. run all the chinese llms. make $600k / month revenue selling tokens on openrouter (market depth is there, this is 1% of openrouter). improvements to tinygrad yield revenue improvements. due to how power is priced in oregon it's only like $50k for the electric bill (below 4MW they price for peak, not usage, we get like 3c kWh power). we can also make ~$100k / month leasing colo space to comma. building and cards paid off in 3 years max, investment made back. low risk of being undercut since we're using consumer GPUs and running the cheapest colo you can believe. if someone chill wants in, i'd do it. i'm not gonna hype fake tech, but demand for tokens is going to skyrocket (look at the openclaw install numbers). with crazy good optimizations we could potentially get 3x more from the machines, and we have electricity for 3x more machines. $5.4M revenue per month. then continue to scale from there, custom chips, etc...

@GTdoubleE · 2026-03-07 15:11
@__tinygrad__ What level of investment are you open to? 100k? 1m+?

@GTdoubleE · 2026-03-07 15:11
@__tinygrad__ What level of investment are you open to? 100k? 1m+?

@__tinygrad__ · 2026-03-07 15:16
@GTdoubleE 1M at minimum. this is mostly a feeler to see if there's demand. if someone chill is like yo i got 10M lets get 10M more and buy a building and a boat of GPUs and hustle tokens then let's do this ... if it's like hello sir i am a vc can you sell hype to next sucker then i'm out.

@__tinygrad__ · 2026-03-07 15:09
if tiny corp was raising $20M (@ $200M), who'd be interested? business model is basically this. buy this $11.5M building (with 5MW of power): link in our discord wait for AMD to launch the RDNA5 96GB cards (mid 2027). preorder 3000 cards (hopefully we can negotiate for $2500 each), build 500 $20k tinyboxes with 6 of the card. run all the chinese llms. make $600k / month revenue selling tokens on openrouter (market depth is there, this is 1% of openrouter). improvements to tinygrad yield revenue improvements. due to how power is priced in oregon it's only like $50k for the electric bill (below 4MW they price for peak, not usage, we get like 3c kWh power). we can also make ~$100k / month leasing colo space to comma. building and cards paid off in 3 years max, investment made back. low risk of being undercut since we're using consumer GPUs and running the cheapest colo you can believe. if someone chill wants in, i'd do it. i'm not gonna hype fake tech, but demand for tokens is going to skyrocket (look at the openclaw install numbers). with crazy good optimizations we could potentially get 3x more from the machines, and we have electricity for 3x more machines. $5.4M revenue per month. then continue to scale from there, custom chips, etc...

@SIGKITTEN · 2026-03-07 15:19
@__tinygrad__ is there no decent model for a community funding? I feel like people putting in 5-50k could get you to 20M and it kinda fits the tinygrad idea

@__tinygrad__ · 2026-03-07 15:09
if tiny corp was raising $20M (@ $200M), who'd be interested? business model is basically this. buy this $11.5M building (with 5MW of power): link in our discord wait for AMD to launch the RDNA5 96GB cards (mid 2027). preorder 3000 cards (hopefully we can negotiate for $2500 each), build 500 $20k tinyboxes with 6 of the card. run all the chinese llms. make $600k / month revenue selling tokens on openrouter (market depth is there, this is 1% of openrouter). improvements to tinygrad yield revenue improvements. due to how power is priced in oregon it's only like $50k for the electric bill (below 4MW they price for peak, not usage, we get like 3c kWh power). we can also make ~$100k / month leasing colo space to comma. building and cards paid off in 3 years max, investment made back. low risk of being undercut since we're using consumer GPUs and running the cheapest colo you can believe. if someone chill wants in, i'd do it. i'm not gonna hype fake tech, but demand for tokens is going to skyrocket (look at the openclaw install numbers). with crazy good optimizations we could potentially get 3x more from the machines, and we have electricity for 3x more machines. $5.4M revenue per month. then continue to scale from there, custom chips, etc...

@thdxr · 2026-03-07 15:22
@__tinygrad__ @SIGKITTEN can you estimate how many tokens per month this would allow you to do? we are probably the #1 source of demand for open source model inference

@SIGKITTEN · 2026-03-07 15:19
@__tinygrad__ is there no decent model for a community funding? I feel like people putting in 5-50k could get you to 20M and it kinda fits the tinygrad idea

@__tinygrad__ · 2026-03-07 15:23
@SIGKITTEN a lot of the crowdfunding things are total scams for the people who buy in when you look at how the ownership is really structured and who really owns the shares. i like keeping things simple, a few accredited investors who are mission aligned, a lead for $10M and a few $1-5M.

@thdxr · 2026-03-07 15:22
@__tinygrad__ @SIGKITTEN can you estimate how many tokens per month this would allow you to do? we are probably the #1 source of demand for open source model inference

@__tinygrad__ · 2026-03-07 15:26
@thdxr @SIGKITTEN 500 machines × 200 tok/s 259.2 billion tokens/month i'm sure you have more than that demand, it's only 1% of openrouter.

@__tinygrad__ · 2026-03-07 15:09
if tiny corp was raising $20M (@ $200M), who'd be interested? business model is basically this. buy this $11.5M building (with 5MW of power): link in our discord wait for AMD to launch the RDNA5 96GB cards (mid 2027). preorder 3000 cards (hopefully we can negotiate for $2500 each), build 500 $20k tinyboxes with 6 of the card. run all the chinese llms. make $600k / month revenue selling tokens on openrouter (market depth is there, this is 1% of openrouter). improvements to tinygrad yield revenue improvements. due to how power is priced in oregon it's only like $50k for the electric bill (below 4MW they price for peak, not usage, we get like 3c kWh power). we can also make ~$100k / month leasing colo space to comma. building and cards paid off in 3 years max, investment made back. low risk of being undercut since we're using consumer GPUs and running the cheapest colo you can believe. if someone chill wants in, i'd do it. i'm not gonna hype fake tech, but demand for tokens is going to skyrocket (look at the openclaw install numbers). with crazy good optimizations we could potentially get 3x more from the machines, and we have electricity for 3x more machines. $5.4M revenue per month. then continue to scale from there, custom chips, etc...

@keennay · 2026-03-07 15:37
@__tinygrad__ I’d love to know what wizardry was performed to secure 3c per kWh

@keennay · 2026-03-07 15:37
@__tinygrad__ I’d love to know what wizardry was performed to secure 3c per kWh

@__tinygrad__ · 2026-03-07 15:39
@keennay i mean, the business plan works fine up to 10c / kWh too if ChatGPT is wrong, but... https://t.co/WjXLEpW9Me

@__tinygrad__ · 2026-03-07 15:09
if tiny corp was raising $20M (@ $200M), who'd be interested? business model is basically this. buy this $11.5M building (with 5MW of power): link in our discord wait for AMD to launch the RDNA5 96GB cards (mid 2027). preorder 3000 cards (hopefully we can negotiate for $2500 each), build 500 $20k tinyboxes with 6 of the card. run all the chinese llms. make $600k / month revenue selling tokens on openrouter (market depth is there, this is 1% of openrouter). improvements to tinygrad yield revenue improvements. due to how power is priced in oregon it's only like $50k for the electric bill (below 4MW they price for peak, not usage, we get like 3c kWh power). we can also make ~$100k / month leasing colo space to comma. building and cards paid off in 3 years max, investment made back. low risk of being undercut since we're using consumer GPUs and running the cheapest colo you can believe. if someone chill wants in, i'd do it. i'm not gonna hype fake tech, but demand for tokens is going to skyrocket (look at the openclaw install numbers). with crazy good optimizations we could potentially get 3x more from the machines, and we have electricity for 3x more machines. $5.4M revenue per month. then continue to scale from there, custom chips, etc...

@rot13maxi · 2026-03-07 15:45
@__tinygrad__ @AtlantisPleb I know in the past there have been issues with AMD driver support. How much risk is there on betting on amd cards?

@__tinygrad__ · 2026-03-07 15:09
if tiny corp was raising $20M (@ $200M), who'd be interested? business model is basically this. buy this $11.5M building (with 5MW of power): link in our discord wait for AMD to launch the RDNA5 96GB cards (mid 2027). preorder 3000 cards (hopefully we can negotiate for $2500 each), build 500 $20k tinyboxes with 6 of the card. run all the chinese llms. make $600k / month revenue selling tokens on openrouter (market depth is there, this is 1% of openrouter). improvements to tinygrad yield revenue improvements. due to how power is priced in oregon it's only like $50k for the electric bill (below 4MW they price for peak, not usage, we get like 3c kWh power). we can also make ~$100k / month leasing colo space to comma. building and cards paid off in 3 years max, investment made back. low risk of being undercut since we're using consumer GPUs and running the cheapest colo you can believe. if someone chill wants in, i'd do it. i'm not gonna hype fake tech, but demand for tokens is going to skyrocket (look at the openclaw install numbers). with crazy good optimizations we could potentially get 3x more from the machines, and we have electricity for 3x more machines. $5.4M revenue per month. then continue to scale from there, custom chips, etc...

@_jaykontra · 2026-03-07 15:46
@__tinygrad__ You think theres going to be a consumer 96GB rdna5 dGPU?

@rot13maxi · 2026-03-07 15:45
@__tinygrad__ @AtlantisPleb I know in the past there have been issues with AMD driver support. How much risk is there on betting on amd cards?

@__tinygrad__ · 2026-03-07 15:55
@rot13maxi @AtlantisPleb we have our own AMD driver in tinygrad now. i'm not worried about this. we need to build out a few pieces of an LLM inference engine (like K/V cache to disk), but that's not that hard, particularly if we are doing BS=1 fast MoE

@_jaykontra · 2026-03-07 15:46
@__tinygrad__ You think theres going to be a consumer 96GB rdna5 dGPU?

@__tinygrad__ · 2026-03-07 15:57
@_jayrain .@AMD @LisaSu would be dumb not to make one, I trust they are. RDNA5 should be GDDR7 with a 512-bit bus. get the 3 GB modules and double side it like the Blackwell. we'll build the board if they don't :)

@__tinygrad__ · 2026-03-07 15:09
if tiny corp was raising $20M (@ $200M), who'd be interested? business model is basically this. buy this $11.5M building (with 5MW of power): link in our discord wait for AMD to launch the RDNA5 96GB cards (mid 2027). preorder 3000 cards (hopefully we can negotiate for $2500 each), build 500 $20k tinyboxes with 6 of the card. run all the chinese llms. make $600k / month revenue selling tokens on openrouter (market depth is there, this is 1% of openrouter). improvements to tinygrad yield revenue improvements. due to how power is priced in oregon it's only like $50k for the electric bill (below 4MW they price for peak, not usage, we get like 3c kWh power). we can also make ~$100k / month leasing colo space to comma. building and cards paid off in 3 years max, investment made back. low risk of being undercut since we're using consumer GPUs and running the cheapest colo you can believe. if someone chill wants in, i'd do it. i'm not gonna hype fake tech, but demand for tokens is going to skyrocket (look at the openclaw install numbers). with crazy good optimizations we could potentially get 3x more from the machines, and we have electricity for 3x more machines. $5.4M revenue per month. then continue to scale from there, custom chips, etc...

@thecsguy · 2026-03-07 15:57
@__tinygrad__ the business model being 'buy a building and wait for rdna5' is unironically the most honest roadmap in ai, 96gb vram is the only way tiny corp actually breaks the nvidia moat

@thecsguy · 2026-03-07 15:57
@__tinygrad__ the business model being 'buy a building and wait for rdna5' is unironically the most honest roadmap in ai, 96gb vram is the only way tiny corp actually breaks the nvidia moat

@__tinygrad__ · 2026-03-07 16:01
@thecsguy .@AMD @AMDRadeon @LisaSu you bringing the hardware to break the NVIDIA moat next generation? please say yes.

@__tinygrad__ · 2026-03-07 15:26
@thdxr @SIGKITTEN 500 machines × 200 tok/s 259.2 billion tokens/month i'm sure you have more than that demand, it's only 1% of openrouter.

@parth_laxmikant · 2026-03-07 16:02
@__tinygrad__ @thdxr @SIGKITTEN I am averaging around 300M token a week, that’s 1.2 B token a month. Now I might not be the vibecoder, but it seems like you can only support merely 200 users with that 🤔

@__tinygrad__ · 2026-03-07 15:09
if tiny corp was raising $20M (@ $200M), who'd be interested? business model is basically this. buy this $11.5M building (with 5MW of power): link in our discord wait for AMD to launch the RDNA5 96GB cards (mid 2027). preorder 3000 cards (hopefully we can negotiate for $2500 each), build 500 $20k tinyboxes with 6 of the card. run all the chinese llms. make $600k / month revenue selling tokens on openrouter (market depth is there, this is 1% of openrouter). improvements to tinygrad yield revenue improvements. due to how power is priced in oregon it's only like $50k for the electric bill (below 4MW they price for peak, not usage, we get like 3c kWh power). we can also make ~$100k / month leasing colo space to comma. building and cards paid off in 3 years max, investment made back. low risk of being undercut since we're using consumer GPUs and running the cheapest colo you can believe. if someone chill wants in, i'd do it. i'm not gonna hype fake tech, but demand for tokens is going to skyrocket (look at the openclaw install numbers). with crazy good optimizations we could potentially get 3x more from the machines, and we have electricity for 3x more machines. $5.4M revenue per month. then continue to scale from there, custom chips, etc...

@lostbutlucky · 2026-03-07 16:07
@__tinygrad__ Are you hiring haha

@parth_laxmikant · 2026-03-07 16:02
@__tinygrad__ @thdxr @SIGKITTEN I am averaging around 300M token a week, that’s 1.2 B token a month. Now I might not be the vibecoder, but it seems like you can only support merely 200 users with that 🤔

@__tinygrad__ · 2026-03-07 16:46
@parth_laxmikant @thdxr @SIGKITTEN 300M output tokens a week?!? you must be rich

@__tinygrad__ · 2026-03-07 15:09
if tiny corp was raising $20M (@ $200M), who'd be interested? business model is basically this. buy this $11.5M building (with 5MW of power): link in our discord wait for AMD to launch the RDNA5 96GB cards (mid 2027). preorder 3000 cards (hopefully we can negotiate for $2500 each), build 500 $20k tinyboxes with 6 of the card. run all the chinese llms. make $600k / month revenue selling tokens on openrouter (market depth is there, this is 1% of openrouter). improvements to tinygrad yield revenue improvements. due to how power is priced in oregon it's only like $50k for the electric bill (below 4MW they price for peak, not usage, we get like 3c kWh power). we can also make ~$100k / month leasing colo space to comma. building and cards paid off in 3 years max, investment made back. low risk of being undercut since we're using consumer GPUs and running the cheapest colo you can believe. if someone chill wants in, i'd do it. i'm not gonna hype fake tech, but demand for tokens is going to skyrocket (look at the openclaw install numbers). with crazy good optimizations we could potentially get 3x more from the machines, and we have electricity for 3x more machines. $5.4M revenue per month. then continue to scale from there, custom chips, etc...

@adeelzaman_ · 2026-03-07 17:57
@__tinygrad__ Would the plan be to only use Tinygrad for the inference provided by this building? What are your thoughts on vLLM and SGLang’s growing AMD support?

@__tinygrad__ · 2026-03-07 15:09
if tiny corp was raising $20M (@ $200M), who'd be interested? business model is basically this. buy this $11.5M building (with 5MW of power): link in our discord wait for AMD to launch the RDNA5 96GB cards (mid 2027). preorder 3000 cards (hopefully we can negotiate for $2500 each), build 500 $20k tinyboxes with 6 of the card. run all the chinese llms. make $600k / month revenue selling tokens on openrouter (market depth is there, this is 1% of openrouter). improvements to tinygrad yield revenue improvements. due to how power is priced in oregon it's only like $50k for the electric bill (below 4MW they price for peak, not usage, we get like 3c kWh power). we can also make ~$100k / month leasing colo space to comma. building and cards paid off in 3 years max, investment made back. low risk of being undercut since we're using consumer GPUs and running the cheapest colo you can believe. if someone chill wants in, i'd do it. i'm not gonna hype fake tech, but demand for tokens is going to skyrocket (look at the openclaw install numbers). with crazy good optimizations we could potentially get 3x more from the machines, and we have electricity for 3x more machines. $5.4M revenue per month. then continue to scale from there, custom chips, etc...

@gajesh · 2026-03-07 18:46
@__tinygrad__ what about cost of listing from openrouter?

@__tinygrad__ · 2026-03-07 15:09
if tiny corp was raising $20M (@ $200M), who'd be interested? business model is basically this. buy this $11.5M building (with 5MW of power): link in our discord wait for AMD to launch the RDNA5 96GB cards (mid 2027). preorder 3000 cards (hopefully we can negotiate for $2500 each), build 500 $20k tinyboxes with 6 of the card. run all the chinese llms. make $600k / month revenue selling tokens on openrouter (market depth is there, this is 1% of openrouter). improvements to tinygrad yield revenue improvements. due to how power is priced in oregon it's only like $50k for the electric bill (below 4MW they price for peak, not usage, we get like 3c kWh power). we can also make ~$100k / month leasing colo space to comma. building and cards paid off in 3 years max, investment made back. low risk of being undercut since we're using consumer GPUs and running the cheapest colo you can believe. if someone chill wants in, i'd do it. i'm not gonna hype fake tech, but demand for tokens is going to skyrocket (look at the openclaw install numbers). with crazy good optimizations we could potentially get 3x more from the machines, and we have electricity for 3x more machines. $5.4M revenue per month. then continue to scale from there, custom chips, etc...

@BooleanProphet · 2026-03-07 19:49
@__tinygrad__ Risk of being undercut is hugely understated. ASICs can generate far more tokens with far less energy consumption than GPUs, no?

@BooleanProphet · 2026-03-07 19:49
@__tinygrad__ Risk of being undercut is hugely understated. ASICs can generate far more tokens with far less energy consumption than GPUs, no?

@__tinygrad__ · 2026-03-08 00:10
@BooleanProphet What ASICs? This isn't Bitcoin mining, it looks a lot more like ETH mining cause it needs memory.

@gajesh · 2026-03-07 18:46
@__tinygrad__ what about cost of listing from openrouter?

@__tinygrad__ · 2026-03-08 00:12
@gajesh It looks like they take 5%, which is so worth it if they handle customers and payments. But it's not like there's lock-in, it would be easy to move if the deal changes.

@adeelzaman_ · 2026-03-07 17:57
@__tinygrad__ Would the plan be to only use Tinygrad for the inference provided by this building? What are your thoughts on vLLM and SGLang’s growing AMD support?

@__tinygrad__ · 2026-03-08 00:16
@adeelzaman_ We'd run whatever makes the most money, if it's something other than tinygrad we run that. But it puts a real $$$ number on more tinygrad speed.

@__tinygrad__ · 2026-03-08 00:19
@midego1 Suggestions? Also need it to be politically stable with good ease of doing business.

@lostbutlucky · 2026-03-07 16:07
@__tinygrad__ Are you hiring haha

@__tinygrad__ · 2026-03-08 00:22
@lostbutlucky We hire from the contributor pool to tinygrad.

@__tinygrad__ · 2026-03-08 12:04
The price and difficulty of getting electricity in America is infuriating. Why aren't we building this? I nominate Arizona to be turned into a huge solar panel. https://t.co/aOfxUC5vtl

@__tinygrad__ · 2026-03-08 12:04
The price and difficulty of getting electricity in America is infuriating. Why aren't we building this? I nominate Arizona to be turned into a huge solar panel. https://t.co/aOfxUC5vtl

@snapolino · 2026-03-08 12:09
@__tinygrad__ Because its only for daytime.... there arent enough batterys/lithium to go 100% "off grid". If every home would have 1 MWh battery, there would never be a grid problem and you just have to put up more solarcells

@__tinygrad__ · 2026-03-08 12:07
Like this shouldn't be something AI startups have to deal with. There should be a competent government that builds out infrastructure in order to help businesses succeed. Power in San Diego is 35c / kWh. Why doesn't someone build a wire? https://t.co/PZe9ZnkKJ9

@__tinygrad__ · 2026-03-08 12:10
If America doesn't get serious about changing this quickly, there's going to be no way for any US based AI providers to compete with China. https://t.co/kwT1MWU6qK

@snapolino · 2026-03-08 12:09
@__tinygrad__ Because its only for daytime.... there arent enough batterys/lithium to go 100% "off grid". If every home would have 1 MWh battery, there would never be a grid problem and you just have to put up more solarcells

@__tinygrad__ · 2026-03-08 12:11
@snapolino I'm still paying 35c per kWh in San Diego in the daytime. Big solar farm the desert + one wire solves this.

@__tinygrad__ · 2026-03-08 12:10
If America doesn't get serious about changing this quickly, there's going to be no way for any US based AI providers to compete with China. https://t.co/kwT1MWU6qK

@__tinygrad__ · 2026-03-08 12:16
And you say, the major cloud providers will deliver capacity. They have the budget to navigate red tape. What good is this? Massive rent seeking from them such that by the time you get the compute all potential for profit is gone. It's neofeudal.

@__tinygrad__ · 2026-03-08 12:11
@snapolino I'm still paying 35c per kWh in San Diego in the daytime. Big solar farm the desert + one wire solves this.

@snapolino · 2026-03-08 12:24
@__tinygrad__ or diy ..... 240MW solarfarm / 150MWh battery you should get that for less than 20M$ installed... be 100% independent no matter what oil/gas prices do...

@snapolino · 2026-03-08 12:24
@__tinygrad__ or diy ..... 240MW solarfarm / 150MWh battery you should get that for less than 20M$ installed... be 100% independent no matter what oil/gas prices do...

@__tinygrad__ · 2026-03-08 12:25
@snapolino If that price is real, I'm sold. Where can we build it? Is it a 6 year permitting and environmental review process?

@__tinygrad__ · 2026-03-08 12:04
The price and difficulty of getting electricity in America is infuriating. Why aren't we building this? I nominate Arizona to be turned into a huge solar panel. https://t.co/aOfxUC5vtl

@Tim275797224441 · 2026-03-08 12:49
@__tinygrad__ Everyone is paying higher electricity prices because of the large AI companies. Same companies that stole every piece of IP out there. Now everyone should chip in and help them out? Real solution is to kick them off the grid. This is more ridiculous than Anthropic's compliant

@Tim275797224441 · 2026-03-08 12:49
@__tinygrad__ Everyone is paying higher electricity prices because of the large AI companies. Same companies that stole every piece of IP out there. Now everyone should chip in and help them out? Real solution is to kick them off the grid. This is more ridiculous than Anthropic's compliant

@__tinygrad__ · 2026-03-08 12:54
@Tim275797224441 This is how America's adversaries use social media to manufacture discontent. What country do you work for?

@__tinygrad__ · 2026-03-09 01:57
commoditize the kilowatt

@_heyglassy · 2026-03-09 02:49
This business model has been my latest obsession and if I was running a VC firm I'd do this in a heartbeat. Makes an unbelievable amount of sense, this is the team to do it as well...

@__tinygrad__ · 2026-03-09 03:56
Portland building is out. if we aren't operating for ~5c/kWh we aren't competitive. looking into buying bitcoin mines, cost < $1M per MW and electricity cost < 5c/kWh. size 5-10 MW. if the price is right we can have the deal done by end of April.

@__tinygrad__ · 2026-03-09 03:56
Portland building is out. if we aren't operating for ~5c/kWh we aren't competitive. looking into buying bitcoin mines, cost < $1M per MW and electricity cost < 5c/kWh. size 5-10 MW. if the price is right we can have the deal done by end of April.

@HotAisle · 2026-03-09 04:12
@__tinygrad__ @sr71blr George… i know someone in quincy…

@HotAisle · 2026-03-09 04:12
@__tinygrad__ @sr71blr George… i know someone in quincy…

@__tinygrad__ · 2026-03-09 04:14
@HotAisle @sr71blr We aren't looking for colo, we're looking to buy. 5-10 MW scale at < $1M per MW cost with < 5c kWh power? If so, we're interested.

@__tinygrad__ · 2026-03-09 04:02
While China is installing tons of power capacity, they also have a huge population with tons of demand. For at least the next 5-10 years 5c/kWh stays competitive. tiny's advantage is better software that allows use of accelerators that don't have 80% margins.

@__tinygrad__ · 2026-03-09 04:29
So yea, raise is happening, will be 10-20M @ 200M. Need the facility capital right away, GPU capital can potentially wait for higher valuation. Minimum check size 1M. Requires proof of accredited investor status. Not looking for VC or funds, just mission aligned individuals.

@__tinygrad__ · 2026-03-09 04:29
So yea, raise is happening, will be 10-20M @ 200M. Need the facility capital right away, GPU capital can potentially wait for higher valuation. Minimum check size 1M. Requires proof of accredited investor status. Not looking for VC or funds, just mission aligned individuals.

@strategos314 · 2026-03-09 05:02
@__tinygrad__ question : what is your moat? If tinygrad improvements are open source, then anyone else can also do the same thing?

@__tinygrad__ · 2026-03-09 03:56
Portland building is out. if we aren't operating for ~5c/kWh we aren't competitive. looking into buying bitcoin mines, cost < $1M per MW and electricity cost < 5c/kWh. size 5-10 MW. if the price is right we can have the deal done by end of April.

@NigelHiggs7 · 2026-03-09 05:12
@__tinygrad__ Wasn’t George talking about on stream this being the easy way out? The big buildings/racks etc being a very modernist approach like atlas shrug opposed to post modernist take?

@__tinygrad__ · 2026-03-09 05:12
20 petaflops for 20 years. a human is a 1e25 training run. H100 cost of the run is $10M, we should be able to do it for $2M. still 10x more than real humans.

@strategos314 · 2026-03-09 05:02
@__tinygrad__ question : what is your moat? If tinygrad improvements are open source, then anyone else can also do the same thing?

@__tinygrad__ · 2026-03-09 06:19
@Genhotel321 tinygrad is an open source framework, it's only one piece of an LLM inference company. and we don't have to outcompete everyone, just suckers who bought expensive NVIDIA GPUs (many such cases)

@__tinygrad__ · 2026-03-09 06:19
@Genhotel321 tinygrad is an open source framework, it's only one piece of an LLM inference company. and we don't have to outcompete everyone, just suckers who bought expensive NVIDIA GPUs (many such cases)

@strategos314 · 2026-03-09 07:34
@__tinygrad__ Okay, so the things you build *using * tinygrad would be closed source but much cheaper than the competition? This honestly sounds like a great business idea

@strategos314 · 2026-03-09 07:34
@__tinygrad__ Okay, so the things you build *using * tinygrad would be closed source but much cheaper than the competition? This honestly sounds like a great business idea

@__tinygrad__ · 2026-03-09 07:36
@Genhotel321 exactly. our open source philosophy is to open source what helps the individual, not the big competing company. and thanks!

@NigelHiggs7 · 2026-03-09 05:12
@__tinygrad__ Wasn’t George talking about on stream this being the easy way out? The big buildings/racks etc being a very modernist approach like atlas shrug opposed to post modernist take?

@__tinygrad__ · 2026-03-09 07:38
@NigelHiggs7 yes and i'm deeply upset about this. but this is just how AI is. if you want to manipulate big information, you need big racks with big power. don't worry, it's not too big, below the scale breaking points.

@distributionat · 2026-03-09 07:54
What would be externally visible signals that labs believe they have AGI? Some I can think of: increased physical security and restrictions (e.g. CEOs no longer leave the US), personnel management—implementing garden leave, stricter NDAs, etc—and compute reallocation towards the RSI loop.

@__tinygrad__ · 2026-03-09 04:29
So yea, raise is happening, will be 10-20M @ 200M. Need the facility capital right away, GPU capital can potentially wait for higher valuation. Minimum check size 1M. Requires proof of accredited investor status. Not looking for VC or funds, just mission aligned individuals.

@__tinygrad__ · 2026-03-09 08:05
more details and AMA about the raise on our Discord https://t.co/mWjefrIZAX

@__tinygrad__ · 2026-03-09 03:56
Portland building is out. if we aren't operating for ~5c/kWh we aren't competitive. looking into buying bitcoin mines, cost < $1M per MW and electricity cost < 5c/kWh. size 5-10 MW. if the price is right we can have the deal done by end of April.

@JeramyUtara · 2026-03-09 08:17
@__tinygrad__ You can’t run GPUs in a BTC data center

@JeramyUtara · 2026-03-09 08:17
@__tinygrad__ You can’t run GPUs in a BTC data center

@__tinygrad__ · 2026-03-09 08:18
@JeramyUtara why not? ETH was mined on GPUs for years. are you by chance a data center bag holder?

@__tinygrad__ · 2026-03-09 08:53
this raise is for thought complex one

@marcel_fue · 2026-03-07 16:01
@__tinygrad__ Funds on chain would work and ownership through a liechtenstein legal entity?

@MIDAODS · 2026-03-09 09:03
@marcel_fue @__tinygrad__ Onchain capital coordination can work well for projects like this, but the ownership structure behind it still needs to be clear. The legal entity holding the asset is what ultimately connects tokenized or onchain governance to real-world enforceability.

@MIDAODS · 2026-03-09 09:03
@marcel_fue @__tinygrad__ Onchain capital coordination can work well for projects like this, but the ownership structure behind it still needs to be clear. The legal entity holding the asset is what ultimately connects tokenized or onchain governance to real-world enforceability.

@__tinygrad__ · 2026-03-09 09:04
@MIDAODS @marcel_fue a 10 foot pole isn't even long enough to touch crypto with. normal company normal fundraising. no token. no crowdfund. no blockchain.

@__tinygrad__ · 2026-03-09 09:00
@distributionat ugh there is no AGI. there is no magic threshold. you guys see autoresearch change the random seed from 42 to 137 and OMG ITS AGI ITS OVER yet you critique the junior engineer for the same crap. the cost of dev is falling. overpaid eng struggles to compete. that's the story.

@distributionat · 2026-03-09 09:27
@__tinygrad__ You don't think cost of dev falling -> cost of inputs to AI research falling -> positive feedback loop -> recursive self improvement is a valid causal chain?

@distributionat · 2026-03-09 09:27
@__tinygrad__ You don't think cost of dev falling -> cost of inputs to AI research falling -> positive feedback loop -> recursive self improvement is a valid causal chain?

@__tinygrad__ · 2026-03-09 09:31
@distributionat for recursive self improvement, sure. but humanity has been recursively self improving for centuries

@gonizahavy · 2026-03-02 23:00
MLX on ROCm?? 👀 EXO on Strix Halo?? 👀 It’s nowhere near ready, but it works! @exolabs @AIatAMD @HotAisle @zcbenz @shshnkp @NiketanNripesh https://t.co/kQRfFqCqtK + https://t.co/O75sVCxgep https://t.co/f2gIy554vD

@PixelTrik07 · 2026-03-09 10:49
@gonizahavy @exolabs @AIatAMD @HotAisle @zcbenz @shshnkp @NiketanNripesh To add further, I've recently implemented @__tinygrad__ runner on EXO. And it's fully functional on AMD Radeon RX 6650M GPU. Currently it's a single GPU implementation, now extending it to multi-device setups too. Link: https://t.co/e4XSUbEpdp https://t.co/AaxhP2Nm8I

@__tinygrad__ · 2026-03-09 01:57
commoditize the kilowatt

@fxgst · 2026-03-09 12:00
@__tinygrad__ @yacineMTB Start an energy production company already

@fxgst · 2026-03-09 12:00
@__tinygrad__ @yacineMTB Start an energy production company already

@__tinygrad__ · 2026-03-09 12:01
@fxgst @yacineMTB it's energy storage that's the killer. production is easy with solar

@svpino · 2026-03-09 12:42
People are lying to you. These agents don't work as they promised. https://t.co/3Oyoi7i4zh

@__tinygrad__ · 2026-03-09 13:37
we wouldn't let in VC cause they aren't interested in long term value creation, but yea, it's obvious. we sell the lowest cost FLOPS, so why wouldn't we sell the lowest cost tokens?

@__tinygrad__ · 2026-03-09 13:37
we wouldn't let in VC cause they aren't interested in long term value creation, but yea, it's obvious. we sell the lowest cost FLOPS, so why wouldn't we sell the lowest cost tokens?

@__tinygrad__ · 2026-03-09 13:43
we don't have all the details worked out either, which model, which accelerator, etc... the power of tinygrad is that we can adapt the fastest. we do know we need cheap power and a place to put the computers ready the second the unit economics are good. cloud gonna get mogged.

@__tinygrad__ · 2026-03-09 13:37
we wouldn't let in VC cause they aren't interested in long term value creation, but yea, it's obvious. we sell the lowest cost FLOPS, so why wouldn't we sell the lowest cost tokens?

@taustw · 2026-03-09 13:44
@__tinygrad__ Would anyone really pay 1/10th the price per token for a model 1.5-2 years behind SOTA? (A couple centuries in human time)

@taustw · 2026-03-09 13:44
@__tinygrad__ Would anyone really pay 1/10th the price per token for a model 1.5-2 years behind SOTA? (A couple centuries in human time)

@__tinygrad__ · 2026-03-09 13:45
@taustw I mean, there's a big transparent market on @OpenRouter, and it's only going to grow.

@alexocheema · 2026-03-09 14:01
Has anyone got tensor parallelism working with clusters of AMD Ryzen AI Max+ systems? I heard the software support is lacking but curious why that is?

@__tinygrad__ · 2026-03-09 14:05
RT @exolabs: .@__tinygrad__ support in exo

@alexocheema · 2026-03-09 14:01
Has anyone got tensor parallelism working with clusters of AMD Ryzen AI Max+ systems? I heard the software support is lacking but curious why that is?

@__tinygrad__ · 2026-03-09 14:14
@alexocheema tinygrad works great on Strix Halo (Ryzen AI Max+), that's my personal dev laptop

@svpino · 2026-03-09 12:42
People are lying to you. These agents don't work as they promised. https://t.co/3Oyoi7i4zh

@__tinygrad__ · 2026-03-09 14:16
@svpino This is the truth. It doesn't raise stock price, but it's true.

@sudoingX · 2026-03-09 16:00
12 tok/s to 54 tok/s. same card. right model for the hardware. 5060 Ti 16GB + Qwen 3.5 9B Q4_K_XL: 54 tok/s at 262K context, thinking mode on. full model on GPU flags: -c 262144 -np 1 -fa on --cache-type-k q4_0 --cache-type-v q4_0 it's not always about squeezing the biggest model onto any card. it's about finding the right one for your hardware. i try to fit 1 trillion parameters on a single GPU because i do this for research. you don't have to. ask questions, i'll try to answer. anyone else running 16GB cards? drop your numbers and flags.

@WodanSama · 2026-03-09 19:42
@sudoingX Been testing on amd 7900 xtx with qwen 3.5 : 9b regular ollama pull 32k ctxt -> 70 tps 27b also regular ollama pull 16k ctxt -> 30 tps Seems similar to 3090 More tests pending Considering 2nd xtx

@__tinygrad__ · 2026-03-10 02:56
it's a red v2 box autoresearching! Claude ported autoresearch to tinygrad. someday soon it will autoautoresearch with a local LLM https://t.co/7W9JMtLJmy

@__tinygrad__ · 2026-03-10 02:56
it's a red v2 box autoresearching! Claude ported autoresearch to tinygrad. someday soon it will autoautoresearch with a local LLM https://t.co/7W9JMtLJmy

@mov_axbx · 2026-03-10 03:02
@__tinygrad__ How are red sales? I was thinking the other day I should make more use of my 7900 XTXs.

@mov_axbx · 2026-03-10 03:02
@__tinygrad__ How are red sales? I was thinking the other day I should make more use of my 7900 XTXs.

@__tinygrad__ · 2026-03-10 03:14
@mov_axbx red v2 selling better than red v1. AMD is on the up and up.

@__tinygrad__ · 2026-03-10 05:48
Think different. https://t.co/yNpuxVBECj

@sudoingX · 2026-03-10 06:50
AMD 7900 XTX running Qwen 3.5 through ollama. 70 tok/s on 9B, 30 tok/s on 27B. these are real community numbers on AMD hardware. but ollama and similar wrappers add abstraction layers that hide what's actually happening. default context sizes, no KV cache quantization, no flash attention control, no slot management. convenience has a cost. if you're running any inference engine with a nice UI and "it just works" defaults, try swapping to raw llama.cpp with the same model. add the flags yourself. there might be speed and efficiency you're leaving on the table. not saying ollama is bad. it's great for getting started. but once you care about tok/s and VRAM, you need to see what's underneath. AMD data points are rare. more of these please.

@__tinygrad__ · 2026-03-10 05:48
Think different. https://t.co/yNpuxVBECj

@allinasecond · 2026-03-10 07:00
@__tinygrad__ GCP, AWS, Azure next question

@allinasecond · 2026-03-10 07:00
@__tinygrad__ GCP, AWS, Azure next question

@__tinygrad__ · 2026-03-10 07:34
@allinasecond WhatsApp AI integration?

@__tinygrad__ · 2026-03-10 10:06
We don't allow VCs in our rounds. They see you as a financial asset: something to mark up, trade around, and exit as soon as they can. That mindset is almost perfectly misaligned with building real technology. Genuine 100x outcomes do not come from hype. They come from building something so valuable that the world rearranges around it. That takes time, iteration, and technical depth. The point of this raise is to build a large computer. It's an engineering dream, not a bean-counter one. Big computers can be very profitable, especially when everyone else is overpaying 5x for theirs. But profitability is downstream. The real objective is to build the machine. Imagine commanding 20 exaflops. Imagine watching a thousand silicon people come to life in a box. Eventually, the cloud providers will have to confront a basic fact: AI economics look much more like crypto mining than like overpriced SaaS. Money matters. Efficiency matters. But only because they let you build a bigger computer. Money is a map. The machine is the territory.

@__tinygrad__ · 2026-03-10 10:06
We don't allow VCs in our rounds. They see you as a financial asset: something to mark up, trade around, and exit as soon as they can. That mindset is almost perfectly misaligned with building real technology. Genuine 100x outcomes do not come from hype. They come from building something so valuable that the world rearranges around it. That takes time, iteration, and technical depth. The point of this raise is to build a large computer. It's an engineering dream, not a bean-counter one. Big computers can be very profitable, especially when everyone else is overpaying 5x for theirs. But profitability is downstream. The real objective is to build the machine. Imagine commanding 20 exaflops. Imagine watching a thousand silicon people come to life in a box. Eventually, the cloud providers will have to confront a basic fact: AI economics look much more like crypto mining than like overpriced SaaS. Money matters. Efficiency matters. But only because they let you build a bigger computer. Money is a map. The machine is the territory.

@jgarzik · 2026-03-10 10:14
@__tinygrad__ Heat and resiliency engineering also matter. Will it run full speed, 24/7, for months?

@jgarzik · 2026-03-10 10:14
@__tinygrad__ Heat and resiliency engineering also matter. Will it run full speed, 24/7, for months?

@__tinygrad__ · 2026-03-10 10:15
@jgarzik Many datacenters overinvest in uptime, then things go down cause of bad software. We're targeting the same uptime as Claude in the last month.

@JonezinWidJonez · 2026-03-10 10:29
@__tinygrad__ @pangram slop?

@pangram · 2026-03-10 10:29
@JonezinWidJonez @__tinygrad__ We are confident that this document is fully AI-generated https://t.co/Rvpws8GLyC https://t.co/GdSRikMUQG

@pangram · 2026-03-10 10:29
@JonezinWidJonez @__tinygrad__ We are confident that this document is fully AI-generated https://t.co/Rvpws8GLyC https://t.co/GdSRikMUQG

@__tinygrad__ · 2026-03-10 10:30
@pangram @JonezinWidJonez lol it's lightly edited by ChatGPT, but you think AI has these ideas?

@__tinygrad__ · 2026-03-10 10:06
We don't allow VCs in our rounds. They see you as a financial asset: something to mark up, trade around, and exit as soon as they can. That mindset is almost perfectly misaligned with building real technology. Genuine 100x outcomes do not come from hype. They come from building something so valuable that the world rearranges around it. That takes time, iteration, and technical depth. The point of this raise is to build a large computer. It's an engineering dream, not a bean-counter one. Big computers can be very profitable, especially when everyone else is overpaying 5x for theirs. But profitability is downstream. The real objective is to build the machine. Imagine commanding 20 exaflops. Imagine watching a thousand silicon people come to life in a box. Eventually, the cloud providers will have to confront a basic fact: AI economics look much more like crypto mining than like overpriced SaaS. Money matters. Efficiency matters. But only because they let you build a bigger computer. Money is a map. The machine is the territory.

@EmanueleUngaro_ · 2026-03-10 10:42
@__tinygrad__ Ok. Crowd fund it then. Distribute the ownership. Lock it for like 5 years or something, so you can’t sell it.

@__tinygrad__ · 2026-03-10 10:46
@jus9891 Sure helps if you don't buy the NVIDIA cards with 80% margins baked in. 3x less depreciation than competitors!

@EmanueleUngaro_ · 2026-03-10 10:42
@__tinygrad__ Ok. Crowd fund it then. Distribute the ownership. Lock it for like 5 years or something, so you can’t sell it.

@__tinygrad__ · 2026-03-10 10:47
@EmanueleUngaro_ no crowdfunding, no ico, no governance token. normal raise from accredited investors with $1M+ who want to see the big computer be built. or they can invest in @Figure_robot lol

@__tinygrad__ · 2026-03-10 10:47
@EmanueleUngaro_ no crowdfunding, no ico, no governance token. normal raise from accredited investors with $1M+ who want to see the big computer be built. or they can invest in @Figure_robot lol

@EmanueleUngaro_ · 2026-03-10 10:52
@__tinygrad__ @Figure_robot Makes sense honestly

@EmanueleUngaro_ · 2026-03-10 10:52
@__tinygrad__ @Figure_robot Makes sense honestly

@__tinygrad__ · 2026-03-10 10:52
@EmanueleUngaro_ @Figure_robot an investment in @Figure_robot never makes sense

@__tinygrad__ · 2026-03-10 10:06
We don't allow VCs in our rounds. They see you as a financial asset: something to mark up, trade around, and exit as soon as they can. That mindset is almost perfectly misaligned with building real technology. Genuine 100x outcomes do not come from hype. They come from building something so valuable that the world rearranges around it. That takes time, iteration, and technical depth. The point of this raise is to build a large computer. It's an engineering dream, not a bean-counter one. Big computers can be very profitable, especially when everyone else is overpaying 5x for theirs. But profitability is downstream. The real objective is to build the machine. Imagine commanding 20 exaflops. Imagine watching a thousand silicon people come to life in a box. Eventually, the cloud providers will have to confront a basic fact: AI economics look much more like crypto mining than like overpriced SaaS. Money matters. Efficiency matters. But only because they let you build a bigger computer. Money is a map. The machine is the territory.

@dogecahedron · 2026-03-10 10:53
@__tinygrad__ on what axis does this sit wrt. centralization of compute

@__tinygrad__ · 2026-03-10 10:52
@EmanueleUngaro_ @Figure_robot an investment in @Figure_robot never makes sense

@EmanueleUngaro_ · 2026-03-10 10:53
@__tinygrad__ @Figure_robot Your investment architecture makes sense. Not investing into figure robot. That doesn’t make sense at all

@EmanueleUngaro_ · 2026-03-10 10:53
@__tinygrad__ @Figure_robot Your investment architecture makes sense. Not investing into figure robot. That doesn’t make sense at all

@__tinygrad__ · 2026-03-10 10:54
@EmanueleUngaro_ @Figure_robot true

@dogecahedron · 2026-03-10 10:53
@__tinygrad__ on what axis does this sit wrt. centralization of compute

@__tinygrad__ · 2026-03-10 11:07
@dogecahedron Your house is 10 kW scale. (1e4) This is 10 MW scale. (1e7) Hyperscalars are 1 GW scale. (1e9)

@__tinygrad__ · 2026-03-10 11:07
@dogecahedron Your house is 10 kW scale. (1e4) This is 10 MW scale. (1e7) Hyperscalars are 1 GW scale. (1e9)

@dogecahedron · 2026-03-10 11:12
@__tinygrad__ so is this sitting in between, on the way to centralizing as much compute as possible in your company or is it sitting in between, on the way to commoditizinng as much compute as possible. what I'm asking is does this lead to decentralization if that's the goal

@dogecahedron · 2026-03-10 11:12
@__tinygrad__ so is this sitting in between, on the way to centralizing as much compute as possible in your company or is it sitting in between, on the way to commoditizinng as much compute as possible. what I'm asking is does this lead to decentralization if that's the goal

@__tinygrad__ · 2026-03-10 11:15
@dogecahedron our mission is to "commoditize the petaflop" not decentralize compute. the latter is way too hard with the landauer limit. we want to build a big computer outside the big 5 and pave the way for similar scale to copy us.

@__tinygrad__ · 2026-03-10 11:15
@dogecahedron our mission is to "commoditize the petaflop" not decentralize compute. the latter is way too hard with the landauer limit. we want to build a big computer outside the big 5 and pave the way for similar scale to copy us.

@dogecahedron · 2026-03-10 11:23
okay so this is paving the way for competition and an end goal of competition with order of magnitude between 5 computers and 10Billion computers? maybe computer size will always follow a pareto distribution: some giant computers and a bazzilion tiny ones and everything in between? so neither the IBM CEO vision nor the complete bitcoin dream wins

@dogecahedron · 2026-03-10 11:23
okay so this is paving the way for competition and an end goal of competition with order of magnitude between 5 computers and 10Billion computers? maybe computer size will always follow a pareto distribution: some giant computers and a bazzilion tiny ones and everything in between? so neither the IBM CEO vision nor the complete bitcoin dream wins

@__tinygrad__ · 2026-03-10 11:27
@dogecahedron pareto is a good outcome I think

@__tinygrad__ · 2026-03-10 10:06
We don't allow VCs in our rounds. They see you as a financial asset: something to mark up, trade around, and exit as soon as they can. That mindset is almost perfectly misaligned with building real technology. Genuine 100x outcomes do not come from hype. They come from building something so valuable that the world rearranges around it. That takes time, iteration, and technical depth. The point of this raise is to build a large computer. It's an engineering dream, not a bean-counter one. Big computers can be very profitable, especially when everyone else is overpaying 5x for theirs. But profitability is downstream. The real objective is to build the machine. Imagine commanding 20 exaflops. Imagine watching a thousand silicon people come to life in a box. Eventually, the cloud providers will have to confront a basic fact: AI economics look much more like crypto mining than like overpriced SaaS. Money matters. Efficiency matters. But only because they let you build a bigger computer. Money is a map. The machine is the territory.

@npub1nmk2399jaz · 2026-03-10 11:29
@__tinygrad__ >The point of this raise is to build a large computer. First thing that came to mind. https://t.co/VqNkEqtEFC

@npub1nmk2399jaz · 2026-03-10 11:29
@__tinygrad__ >The point of this raise is to build a large computer. First thing that came to mind. https://t.co/VqNkEqtEFC

@__tinygrad__ · 2026-03-10 11:31
@npub1nmk2399jaz did you see ours? https://t.co/rYoGBnpyQt

@gonzalo__nunez · 2026-03-10 11:50
“It's an engineering dream, not a bean-counter one… The real objective is to build the machine. Imagine commanding 20 exaflops. Imagine watching a thousand silicon people come to life in a box.” No ARR inflation or tweets about being the fastest to X… maybe the spirit of SV isn’t entirely dead after all.

@gonzalo__nunez · 2026-03-10 11:50
“It's an engineering dream, not a bean-counter one… The real objective is to build the machine. Imagine commanding 20 exaflops. Imagine watching a thousand silicon people come to life in a box.” No ARR inflation or tweets about being the fastest to X… maybe the spirit of SV isn’t entirely dead after all.

@__tinygrad__ · 2026-03-10 11:54
@gonzalo__nunez it's really on life support

@__tinygrad__ · 2026-03-10 11:44
@sudoingX it's sad that that 3 year old card is a better deal than anything new on the market today (and has barely dropped in price since launch). i was promised moores law

@ErnaCat · 2026-03-10 13:35
@__tinygrad__ @sudoingX moores law will come roaring back with a vengeance

@ErnaCat · 2026-03-10 13:35
@__tinygrad__ @sudoingX moores law will come roaring back with a vengeance

@__tinygrad__ · 2026-03-10 13:36
@ErnaCat @sudoingX :pray:

@__tinygrad__ · 2026-03-10 14:05
Thinking about leasing a powered spot instead of buying. Anyone have a spot with 600 kW of < 5c power, decent fiber internet, and free cooling climate? We'll come drop a 20 ft shipping container off.

@__tinygrad__ · 2026-03-10 14:05
Thinking about leasing a powered spot instead of buying. Anyone have a spot with 600 kW of < 5c power, decent fiber internet, and free cooling climate? We'll come drop a 20 ft shipping container off.

@SIGKITTEN · 2026-03-10 14:32
@__tinygrad__ tinycontainer ready??

@__tinygrad__ · 2026-03-10 14:05
Thinking about leasing a powered spot instead of buying. Anyone have a spot with 600 kW of < 5c power, decent fiber internet, and free cooling climate? We'll come drop a 20 ft shipping container off.

@EdSealing · 2026-03-10 14:33
@__tinygrad__ This is definitely the right approach. Keep it cheap and easy to implement. A "mobile datacenter" that you can just drop in and haul away gives you the advantage in price negotiations.

@SIGKITTEN · 2026-03-10 14:32
@__tinygrad__ tinycontainer ready??

@__tinygrad__ · 2026-03-10 14:34
@SIGKITTEN we're buying one on facebook marketplace and gonna put it in comma's back parking lot

@EdSealing · 2026-03-10 14:33
@__tinygrad__ This is definitely the right approach. Keep it cheap and easy to implement. A "mobile datacenter" that you can just drop in and haul away gives you the advantage in price negotiations.

@__tinygrad__ · 2026-03-10 14:36
@EdSealing exactly. i'm not even sure we have to raise money anymore. min quantity is like 5x less at single container scale. we are postmodernists and we have to think that way!

@__tinygrad__ · 2026-03-10 14:05
Thinking about leasing a powered spot instead of buying. Anyone have a spot with 600 kW of < 5c power, decent fiber internet, and free cooling climate? We'll come drop a 20 ft shipping container off.

@dcvilyz · 2026-03-10 14:42
@__tinygrad__ we have 30MW below 5c just outside chattanooga, your shipping containers would fit right in

@__tinygrad__ · 2026-03-10 14:54
ok ok hear me out. what if we did space datacenters but on earth? like we build them all rugged and good, ready to withstand temperatures, low maintenance, fits on the back of a truck, all ready to go to space, but then we ... don't send them to space. sending things to space is expensive. if we keep them on earth, we can send them to places by truck, which is a lot cheaper than space. i don't know what i was thinking about buying land and building a building. that's so modernist. we have $5M and I thought we needed to raise to amortize the fixed costs of operating a site. it was stressing me out. but then i remembered space datacenters. where we're going, we don't need a site. i mean, yea, we do, and we have to lease it, but we'll lease anything where it's cool, has cheap power, and has fiber. if the public utility decides to rug us and raise prices, no lawyers needed, just fire the gas thrusters! actually we don't even need gas thrusters, we'll put it on a truck and go to the next leased site. the minimum quantity we can do this at is one, and one should only cost like $3M. we have $5M, we don't even need to raise, just build the one, watch it print money, then build the next one with the money. self replicating space datacenters on earth. so yea there's a lot of software work to do to make tinygrad run LLMs at really high tok/s and be ready to deploy for the RDNA5 launch. gotta focus on that. raising money, buying land, and reading utility contracts are rabbit holes. got out just in time. i'm telling you guys, it's the next big thing. space datacenters, but on earth. you heard it here first.

@dcvilyz · 2026-03-10 14:42
@__tinygrad__ we have 30MW below 5c just outside chattanooga, your shipping containers would fit right in

@__tinygrad__ · 2026-03-10 15:00
@dcvilyz how's the climate for free cooling? you have decent fiber? i'm liking this plan so much more already

@__tinygrad__ · 2026-03-10 14:54
ok ok hear me out. what if we did space datacenters but on earth? like we build them all rugged and good, ready to withstand temperatures, low maintenance, fits on the back of a truck, all ready to go to space, but then we ... don't send them to space. sending things to space is expensive. if we keep them on earth, we can send them to places by truck, which is a lot cheaper than space. i don't know what i was thinking about buying land and building a building. that's so modernist. we have $5M and I thought we needed to raise to amortize the fixed costs of operating a site. it was stressing me out. but then i remembered space datacenters. where we're going, we don't need a site. i mean, yea, we do, and we have to lease it, but we'll lease anything where it's cool, has cheap power, and has fiber. if the public utility decides to rug us and raise prices, no lawyers needed, just fire the gas thrusters! actually we don't even need gas thrusters, we'll put it on a truck and go to the next leased site. the minimum quantity we can do this at is one, and one should only cost like $3M. we have $5M, we don't even need to raise, just build the one, watch it print money, then build the next one with the money. self replicating space datacenters on earth. so yea there's a lot of software work to do to make tinygrad run LLMs at really high tok/s and be ready to deploy for the RDNA5 launch. gotta focus on that. raising money, buying land, and reading utility contracts are rabbit holes. got out just in time. i'm telling you guys, it's the next big thing. space datacenters, but on earth. you heard it here first.

@ghnynex · 2026-03-10 15:02
@__tinygrad__ Extremely low orbit datacenters.

@ghnynex · 2026-03-10 15:02
@__tinygrad__ Extremely low orbit datacenters.

@__tinygrad__ · 2026-03-10 15:03
@ghnynex with that framing, our valuation is 10x too low

@__tinygrad__ · 2026-03-10 14:54
ok ok hear me out. what if we did space datacenters but on earth? like we build them all rugged and good, ready to withstand temperatures, low maintenance, fits on the back of a truck, all ready to go to space, but then we ... don't send them to space. sending things to space is expensive. if we keep them on earth, we can send them to places by truck, which is a lot cheaper than space. i don't know what i was thinking about buying land and building a building. that's so modernist. we have $5M and I thought we needed to raise to amortize the fixed costs of operating a site. it was stressing me out. but then i remembered space datacenters. where we're going, we don't need a site. i mean, yea, we do, and we have to lease it, but we'll lease anything where it's cool, has cheap power, and has fiber. if the public utility decides to rug us and raise prices, no lawyers needed, just fire the gas thrusters! actually we don't even need gas thrusters, we'll put it on a truck and go to the next leased site. the minimum quantity we can do this at is one, and one should only cost like $3M. we have $5M, we don't even need to raise, just build the one, watch it print money, then build the next one with the money. self replicating space datacenters on earth. so yea there's a lot of software work to do to make tinygrad run LLMs at really high tok/s and be ready to deploy for the RDNA5 launch. gotta focus on that. raising money, buying land, and reading utility contracts are rabbit holes. got out just in time. i'm telling you guys, it's the next big thing. space datacenters, but on earth. you heard it here first.

@btwphones · 2026-03-10 15:03
@__tinygrad__ problem will be demand >> supply on earth

@__tinygrad__ · 2026-03-10 14:54
ok ok hear me out. what if we did space datacenters but on earth? like we build them all rugged and good, ready to withstand temperatures, low maintenance, fits on the back of a truck, all ready to go to space, but then we ... don't send them to space. sending things to space is expensive. if we keep them on earth, we can send them to places by truck, which is a lot cheaper than space. i don't know what i was thinking about buying land and building a building. that's so modernist. we have $5M and I thought we needed to raise to amortize the fixed costs of operating a site. it was stressing me out. but then i remembered space datacenters. where we're going, we don't need a site. i mean, yea, we do, and we have to lease it, but we'll lease anything where it's cool, has cheap power, and has fiber. if the public utility decides to rug us and raise prices, no lawyers needed, just fire the gas thrusters! actually we don't even need gas thrusters, we'll put it on a truck and go to the next leased site. the minimum quantity we can do this at is one, and one should only cost like $3M. we have $5M, we don't even need to raise, just build the one, watch it print money, then build the next one with the money. self replicating space datacenters on earth. so yea there's a lot of software work to do to make tinygrad run LLMs at really high tok/s and be ready to deploy for the RDNA5 launch. gotta focus on that. raising money, buying land, and reading utility contracts are rabbit holes. got out just in time. i'm telling you guys, it's the next big thing. space datacenters, but on earth. you heard it here first.

@OskarKa29225279 · 2026-03-10 15:03
@__tinygrad__ I think you should think of this in a more decentralized fashion. Allowing for others to be part of the mesh - some parts of Europe have plenty of cheep and very reliable electricity, super good internet connectivity and cold most days of the year.

@btwphones · 2026-03-10 15:03
@__tinygrad__ problem will be demand >> supply on earth

@__tinygrad__ · 2026-03-10 15:03
@btwphones okay fine if we really have to we'll send them to space

@OskarKa29225279 · 2026-03-10 15:03
@__tinygrad__ I think you should think of this in a more decentralized fashion. Allowing for others to be part of the mesh - some parts of Europe have plenty of cheep and very reliable electricity, super good internet connectivity and cold most days of the year.

@__tinygrad__ · 2026-03-10 15:05
@OskarKa29225279 you show us the plot of concrete, the big plug for power, and the little plug for internet, and we'll deliver a space datacenter to your site and pay a fair rate to have it live there.

@__tinygrad__ · 2026-03-10 10:52
@EmanueleUngaro_ @Figure_robot an investment in @Figure_robot never makes sense

@Mick_Kirkland · 2026-03-10 15:25
@__tinygrad__ @EmanueleUngaro_ @Figure_robot I can bet your company will fade away with that kind of ego

@__tinygrad__ · 2026-03-10 14:54
ok ok hear me out. what if we did space datacenters but on earth? like we build them all rugged and good, ready to withstand temperatures, low maintenance, fits on the back of a truck, all ready to go to space, but then we ... don't send them to space. sending things to space is expensive. if we keep them on earth, we can send them to places by truck, which is a lot cheaper than space. i don't know what i was thinking about buying land and building a building. that's so modernist. we have $5M and I thought we needed to raise to amortize the fixed costs of operating a site. it was stressing me out. but then i remembered space datacenters. where we're going, we don't need a site. i mean, yea, we do, and we have to lease it, but we'll lease anything where it's cool, has cheap power, and has fiber. if the public utility decides to rug us and raise prices, no lawyers needed, just fire the gas thrusters! actually we don't even need gas thrusters, we'll put it on a truck and go to the next leased site. the minimum quantity we can do this at is one, and one should only cost like $3M. we have $5M, we don't even need to raise, just build the one, watch it print money, then build the next one with the money. self replicating space datacenters on earth. so yea there's a lot of software work to do to make tinygrad run LLMs at really high tok/s and be ready to deploy for the RDNA5 launch. gotta focus on that. raising money, buying land, and reading utility contracts are rabbit holes. got out just in time. i'm telling you guys, it's the next big thing. space datacenters, but on earth. you heard it here first.

@dpifke · 2026-03-10 15:25
@__tinygrad__ 48VDC Tinybox (no inverter losses when powered by solar/battery) coming soon?

@Mick_Kirkland · 2026-03-10 15:25
@__tinygrad__ @EmanueleUngaro_ @Figure_robot I can bet your company will fade away with that kind of ego

@__tinygrad__ · 2026-03-10 15:25
@M2253235397181 @EmanueleUngaro_ @Figure_robot sorry about your @Figure_robot bags

@dpifke · 2026-03-10 15:25
@__tinygrad__ 48VDC Tinybox (no inverter losses when powered by solar/battery) coming soon?

@__tinygrad__ · 2026-03-10 15:26
@dpifke we could, 48VDC PSUs are drop in I think

@__tinygrad__ · 2026-03-11 00:36
https://t.co/BhtUhVy9AH

@__tinygrad__ · 2026-03-11 00:36
https://t.co/BhtUhVy9AH

@mov_axbx · 2026-03-11 00:39
@__tinygrad__ Mount like four 1.5 ton apartment ACs on the top of that thing let’s ride

@mov_axbx · 2026-03-11 00:39
@__tinygrad__ Mount like four 1.5 ton apartment ACs on the top of that thing let’s ride

@__tinygrad__ · 2026-03-11 00:41
@mov_axbx we were building computers at the wrong scale. if people thought tinyboxes weren't tiny before... https://t.co/0xSdoXeZga

@__tinygrad__ · 2026-03-11 00:36
https://t.co/BhtUhVy9AH

@matt503ea5sf9z5 · 2026-03-11 00:54
@__tinygrad__ "a trailer park, but for AI"

@matt503ea5sf9z5 · 2026-03-11 00:54
@__tinygrad__ "a trailer park, but for AI"

@__tinygrad__ · 2026-03-11 00:56
@MatthewRideout This is how we will frame it when we are looking for a well priced lease.

@__tinygrad__ · 2026-03-12 00:04
@Richard6044392 you want to clean that PR up and get it merged? it's a bit AI slop right now

@__tinygrad__ · 2026-03-13 12:11
with tinygrad, the exabox will function as a single very large GPU that you (or your agent) can drive from a Python notebook. coming 2027, get your concrete slab ready. it's the ultimate external GPU. https://t.co/xVEiwcMNxD

@__tinygrad__ · 2026-03-13 12:11
with tinygrad, the exabox will function as a single very large GPU that you (or your agent) can drive from a Python notebook. coming 2027, get your concrete slab ready. it's the ultimate external GPU. https://t.co/xVEiwcMNxD

@alexocheema · 2026-03-13 12:27
@__tinygrad__ 10M is a steal

@__tinygrad__ · 2026-03-13 12:11
with tinygrad, the exabox will function as a single very large GPU that you (or your agent) can drive from a Python notebook. coming 2027, get your concrete slab ready. it's the ultimate external GPU. https://t.co/xVEiwcMNxD

@danieltvela · 2026-03-13 12:38
@__tinygrad__ This can't be real!!! 😱

@danieltvela · 2026-03-13 12:38
@__tinygrad__ This can't be real!!! 😱

@__tinygrad__ · 2026-03-13 12:53
@danieltvela It wouldn't be next to two in stock products if we weren't confident in it. Specs may change a little, but it will ship.

@alexocheema · 2026-03-13 12:27
@__tinygrad__ 10M is a steal

@__tinygrad__ · 2026-03-13 12:53
@alexocheema Right? https://t.co/5gFi6WZOst

@__tinygrad__ · 2026-03-13 12:11
with tinygrad, the exabox will function as a single very large GPU that you (or your agent) can drive from a Python notebook. coming 2027, get your concrete slab ready. it's the ultimate external GPU. https://t.co/xVEiwcMNxD

@mlajtos_mu · 2026-03-13 13:01
@__tinygrad__ > commoditize the petaflop > look inside > teraflops & exaflops

@mlajtos_mu · 2026-03-13 13:01
@__tinygrad__ > commoditize the petaflop > look inside > teraflops & exaflops

@__tinygrad__ · 2026-03-13 14:12
@mlajtos_mu https://t.co/CMPUZB3IeH

@__tinygrad__ · 2026-03-13 14:14
@SauceVir The node machines won't even have boot drives, and nobody would ever PXE boot all 20GB of ROCm!

@__tinygrad__ · 2026-03-14 04:16
Mac Mini + eGPU. Both NVIDIA and AMD supported. https://t.co/CIcIF3j7Ol

@__tinygrad__ · 2026-03-14 04:16
Mac Mini + eGPU. Both NVIDIA and AMD supported. https://t.co/CIcIF3j7Ol

@beffjezos · 2026-03-14 05:00
@__tinygrad__ Wait, how?

@beffjezos · 2026-03-14 05:00
@__tinygrad__ Wait, how?

@__tinygrad__ · 2026-03-14 05:14
@beffjezos We have full pure Python user space drivers for AMD and NVIDIA in tinygrad. USB4 devices can be mmaped like they are directly on the PCIe bus. This isn't hype, it all works today on any 3000-5000 series NVIDIA or RDNA3/RDNA4 AMD.

@__tinygrad__ · 2026-03-14 04:16
Mac Mini + eGPU. Both NVIDIA and AMD supported. https://t.co/CIcIF3j7Ol

@Thorium_Labs · 2026-03-14 05:21
@__tinygrad__ Wow! Very interesting. What about the bandwith? Any bottlenecks? Does it support a RTX 6000 Pro Blackwell?

@__tinygrad__ · 2026-03-14 05:14
@beffjezos We have full pure Python user space drivers for AMD and NVIDIA in tinygrad. USB4 devices can be mmaped like they are directly on the PCIe bus. This isn't hype, it all works today on any 3000-5000 series NVIDIA or RDNA3/RDNA4 AMD.

@robertocarrizo · 2026-03-14 05:22
@__tinygrad__ @beffjezos where? when

@__tinygrad__ · 2026-03-14 04:16
Mac Mini + eGPU. Both NVIDIA and AMD supported. https://t.co/CIcIF3j7Ol

@n6bbo · 2026-03-14 05:34
@__tinygrad__ Did you get the SIP sorted and what is the status of this NVIDIA 50xx reset bug?

@robertocarrizo · 2026-03-14 05:22
@__tinygrad__ @beffjezos where? when

@__tinygrad__ · 2026-03-14 05:37
@robertocarrizo @beffjezos umm in the repo ... and over the last 2 years

@Thorium_Labs · 2026-03-14 05:21
@__tinygrad__ Wow! Very interesting. What about the bandwith? Any bottlenecks? Does it support a RTX 6000 Pro Blackwell?

@__tinygrad__ · 2026-03-14 05:39
@Thorium_Labs Yes, that card is supported, it's basically a 5090 with more RAM. Bandwidth is USB4, 40 Gbps.

@n6bbo · 2026-03-14 05:34
@__tinygrad__ Did you get the SIP sorted and what is the status of this NVIDIA 50xx reset bug?

@__tinygrad__ · 2026-03-14 05:41
@n6bbo SIP sorted for NVIDIA, for some reason they didn't give us the AMD one. It's a super annoying process but I think we'll get it. And our eGPU board (ships Q2) supports full power toggling to the GPU, so no reset issue there.

@comma_ai · 2026-03-17 03:36
Big announcement tomorrow. Rhymes with lend-to-lend.

@comma_ai · 2026-03-17 03:36
Big announcement tomorrow. Rhymes with lend-to-lend.

@__tinygrad__ · 2026-03-17 06:46
@comma_ai comma 4 in stock spend-to-send?

@fw7th · 2026-03-17 07:08
today, I wanted to contribute to tinygrad but got confused as to what the fuck I was reading. So, I embark on a project - will be documented and hopefully improve my low-level ML side;

@fw7th · 2026-03-17 07:08
today, I wanted to contribute to tinygrad but got confused as to what the fuck I was reading. So, I embark on a project - will be documented and hopefully improve my low-level ML side;

@__tinygrad__ · 2026-03-17 08:48
@fw7th Did you read the spec? https://t.co/kiUAXKBtfP

@__tinygrad__ · 2026-03-17 11:22
Llama 405B is really only 10 Tensors https://t.co/irDZuSXbg4

@norpadon · 2026-03-17 11:41
@__tinygrad__ Btw you can also merge gate and up projection matrices in swiglu

@norpadon · 2026-03-17 11:42
@__tinygrad__ And rmsnorm scales can be fused into the weights of the next linear layer

@__tinygrad__ · 2026-03-17 11:22
Llama 405B is really only 10 Tensors https://t.co/irDZuSXbg4

@LenSeaside · 2026-03-17 11:52
@__tinygrad__ Which is one tensor really. Along the file dimension.

@norpadon · 2026-03-17 11:42
@__tinygrad__ And rmsnorm scales can be fused into the weights of the next linear layer

@__tinygrad__ · 2026-03-17 11:52
@norpadon oh cool, merging w1 and w3. benefit is fewer tensors to reason about. norm can't be fused during training, right?

@__tinygrad__ · 2026-03-17 11:22
Llama 405B is really only 10 Tensors https://t.co/irDZuSXbg4

@experimetal · 2026-03-17 11:59
@__tinygrad__ Explain why this matters to the GPU middle class

@LenSeaside · 2026-03-17 11:52
@__tinygrad__ Which is one tensor really. Along the file dimension.

@__tinygrad__ · 2026-03-17 12:11
@LenSeaside this displeases Muon

@experimetal · 2026-03-17 11:59
@__tinygrad__ Explain why this matters to the GPU middle class

@__tinygrad__ · 2026-03-17 13:15
@experimetal 405B *is* middle class

@sama · 2026-03-17 15:55
I have so much gratitude to people who wrote extremely complex software character-by-character. It already feels difficult to remember how much effort it really took. Thank you for getting us to this point.

@SemiAnalysis_ · 2026-03-17 17:00
The West is now more commie than even china. NVIDIA is creating an committee for pretraining open weight models meanwhile China has a flourishing amounts of competitive startups pretraining OSS models https://t.co/zxn4IjJ8I1

@__tinygrad__ · 2026-03-17 08:48
@fw7th Did you read the spec? https://t.co/kiUAXKBtfP

@fzimmermann89 · 2026-03-17 17:20
@__tinygrad__ @fw7th Is there a design overview of the internals? The spec is great to use Tinygrad, but not to understand how it works compared to torch or jax, and where to look for stuff..

@Jonathan_Blow · 2026-03-17 20:11
This is such a *completely* different reality from where I live, at this point it's just difficult to say anything meaningful about it at all. https://t.co/HMltzq20lp

@SemiAnalysis_ · 2026-03-17 17:00
The West is now more commie than even china. NVIDIA is creating an committee for pretraining open weight models meanwhile China has a flourishing amounts of competitive startups pretraining OSS models https://t.co/zxn4IjJ8I1

@__tinygrad__ · 2026-03-18 00:14
@SemiAnalysis_ We love to see NVIDIA commoditizing the model tier!

@Jonathan_Blow · 2026-03-17 20:11
This is such a *completely* different reality from where I live, at this point it's just difficult to say anything meaningful about it at all. https://t.co/HMltzq20lp

@__tinygrad__ · 2026-03-18 01:16
@Jonathan_Blow lol sam altman posted an ad for his company. do you remember having to fill out your taxes line by line? I barely remember now that I have TurboTax!

@fzimmermann89 · 2026-03-17 17:20
@__tinygrad__ @fw7th Is there a design overview of the internals? The spec is great to use Tinygrad, but not to understand how it works compared to torch or jax, and where to look for stuff..

@__tinygrad__ · 2026-03-18 02:30
@fzimmermann89 @fw7th These are the internals. This is the entire IR across the whole lowering process.

@milindS_ · 2026-03-18 10:12
Every day that I use Kimi and GLM, I realize that @__tinygrad__ is going to mint money in a couple years time The big 'labs' don't have any way to compete with cheap inference of ridiculously good models

@__tinygrad__ · 2026-03-18 11:20
People are too focused on trying to build God and not thinking about the unit economics 🤑

@MichaelDell · 2026-03-18 14:35
Jensen Huang is loving the new Dell Pro Max with GB300 at NVIDIA GTC.💙 They asked me to sign it, but I already did 😉 https://t.co/9skQ5PmJlr

@0xSero · 2026-03-19 01:03
Putting out a wish to the universe. I need more compute, if I can get more I will make sure every machine from a small phone to a bootstrapped RTX 3090 node can run frontier intelligence fast with minimal intelligence loss. I have hit page 2 of huggingface, released 3 model family compressions and got GLM-4.7 on a MacBook https://t.co/lorDSUEYCL My beast just isn’t enough and I already spent 2k usd on renting GPUs on top of credits provided by Prime intellect and Hotaisle. ——— If you believe in what I do help me get this to Nvidia, maybe they will bless me with the pewter to keep making local AI more accessible 🙏

@__tinygrad__ · 2026-03-19 09:48
If we can tunnel PCIe over USB3, why not also tunnel it over Ethernet? extra/remote/serve.py on remote, REMOTE=<ip> on local. https://t.co/moosnS1xjW

@__tinygrad__ · 2026-03-19 09:48
If we can tunnel PCIe over USB3, why not also tunnel it over Ethernet? extra/remote/serve.py on remote, REMOTE=<ip> on local. https://t.co/moosnS1xjW

@andreiofstan · 2026-03-19 09:51
@__tinygrad__ I have thought about this for a long time, what is it actually forwarding, the TLPs? How does BAR mapping work in this case

@__tinygrad__ · 2026-03-19 09:48
If we can tunnel PCIe over USB3, why not also tunnel it over Ethernet? extra/remote/serve.py on remote, REMOTE=<ip> on local. https://t.co/moosnS1xjW

@kneeanderthul · 2026-03-19 09:53
@__tinygrad__ This whole time I thought y'all were running eGPUs over TB5 ☠️

@kneeanderthul · 2026-03-19 09:53
@__tinygrad__ This whole time I thought y'all were running eGPUs over TB5 ☠️

@__tinygrad__ · 2026-03-19 09:57
@kneeanderthul Oh we do that too. Once you have the driver, if you can tunnel PCIe through it, it just works!

@andreiofstan · 2026-03-19 09:51
@__tinygrad__ I have thought about this for a long time, what is it actually forwarding, the TLPs? How does BAR mapping work in this case

@__tinygrad__ · 2026-03-19 09:57
@andreiofstan https://t.co/zVdmrUfY8Y

@__tinygrad__ · 2026-03-19 09:48
If we can tunnel PCIe over USB3, why not also tunnel it over Ethernet? extra/remote/serve.py on remote, REMOTE=<ip> on local. https://t.co/moosnS1xjW

@AIFlow_ML · 2026-03-19 10:01
@__tinygrad__ Why not Thunderbolt 5 ?

@AIFlow_ML · 2026-03-19 10:01
@__tinygrad__ Why not Thunderbolt 5 ?

@__tinygrad__ · 2026-03-19 10:27
@AIFlow_ML ugh then I have to plug in a wire. this is Wi-Fi

@__tinygrad__ · 2026-03-19 09:57
@kneeanderthul Oh we do that too. Once you have the driver, if you can tunnel PCIe through it, it just works!

@kneeanderthul · 2026-03-19 10:45
@__tinygrad__ Phenomenal!!! I've been thinking of using a few M3 with 512gb of ram to load a huge model and getting a Red box v2 to pass over the computing over TB5 I think I saw someone do it with Exo Who knew Vram could be so fun

@kneeanderthul · 2026-03-19 10:45
@__tinygrad__ Phenomenal!!! I've been thinking of using a few M3 with 512gb of ram to load a huge model and getting a Red box v2 to pass over the computing over TB5 I think I saw someone do it with Exo Who knew Vram could be so fun

@__tinygrad__ · 2026-03-19 10:51
@kneeanderthul I wonder if you could plug one of these into a tinybox red? It has an open PCIe slot. Super high speed MacBook link! https://t.co/NCo1cXGksl

@__tinygrad__ · 2026-03-19 10:51
@kneeanderthul I wonder if you could plug one of these into a tinybox red? It has an open PCIe slot. Super high speed MacBook link! https://t.co/NCo1cXGksl

@kneeanderthul · 2026-03-19 10:54
@__tinygrad__ You wouldn't need that: M3 Ultra M4 Pro / Max M5 Pro / Max All come with TB5. It's just a matter of ensuring you buy the right models at this point If other companies add TB5 to their devices, we could potentially have mixed share vram across multiple devices over TB5

@kneeanderthul · 2026-03-19 10:54
@__tinygrad__ You wouldn't need that: M3 Ultra M4 Pro / Max M5 Pro / Max All come with TB5. It's just a matter of ensuring you buy the right models at this point If other companies add TB5 to their devices, we could potentially have mixed share vram across multiple devices over TB5

@__tinygrad__ · 2026-03-19 11:04
@kneeanderthul How do you connect the Mac to the tinybox? 10 gigabit ethernet is an option, but it doesn't get those 120 Gbps speeds.

@Rivian · 2026-03-19 12:00
A fleet of R2 Robotaxis is coming exclusively to @Uber. ⚡🌿 Today, we announced a partnership to help both companies accelerate their autonomous vehicle plans across 25 cities in the US, Canada and Europe by the end of 2031. https://t.co/6WazhobMyr https://t.co/9fzgmIsOd5

@JoshKale · 2026-03-19 13:51
Nobody understands how much of a disaster this Rivian <> Uber deal is Rivian lost $3.6 billion last year on 42k deliveries. That's $86,000 of value destruction PER VEHICLE that left their factory. Their solution? Partner with Uber to turn a $58K camping SUV into a robotaxi... to compete with Tesla's Cybercab... YIKES Every 12-18 months, this company finds a new partner to write a check: - Amazon: $1.3B equity + 100K van order - VW: $5.8B joint venture - US DOE: $6.6B loan - Uber: $1.25B robotaxi deal (today) The moment they announced the Uber deal, they admitted they're pushing back profitability AGAIN to fund an autonomy program that can't even handle stoplights. Tesla's Cybercab is purpose built at $25,000 with no steering wheel. The cost per mile math isn't even close. The Uber deal is to deploy 50,000 robotaxis by 2031. Slight problem: The car doesn't exist yet. The factory doesn't exist yet. The autonomy software doesn't exist. Manufacturing is HARD. good luck have fun

@jonatanpallesen · 2026-03-19 22:15
The total number of smart people in the world has just peaked. And now it's about to crash. https://t.co/rH6hqXXTZ1

@Suhail · 2026-03-20 02:33
I am now at 5 GPU providers being completely sold out for a single node of 8xH100s. I don’t think people understand the gravity of what is about to come.

@JoshKale · 2026-03-19 13:51
Nobody understands how much of a disaster this Rivian <> Uber deal is Rivian lost $3.6 billion last year on 42k deliveries. That's $86,000 of value destruction PER VEHICLE that left their factory. Their solution? Partner with Uber to turn a $58K camping SUV into a robotaxi... to compete with Tesla's Cybercab... YIKES Every 12-18 months, this company finds a new partner to write a check: - Amazon: $1.3B equity + 100K van order - VW: $5.8B joint venture - US DOE: $6.6B loan - Uber: $1.25B robotaxi deal (today) The moment they announced the Uber deal, they admitted they're pushing back profitability AGAIN to fund an autonomy program that can't even handle stoplights. Tesla's Cybercab is purpose built at $25,000 with no steering wheel. The cost per mile math isn't even close. The Uber deal is to deploy 50,000 robotaxis by 2031. Slight problem: The car doesn't exist yet. The factory doesn't exist yet. The autonomy software doesn't exist. Manufacturing is HARD. good luck have fun

@__tinygrad__ · 2026-03-20 02:51
@JoshKale lol the best autonomy for Rivian is @comma_ai (model running with tinygrad!) so hands down that it's not even funny. someone is getting scammed here

@__tinygrad__ · 2026-03-20 05:03
how many tinyboxes have you stockpiled?

@jonatanpallesen · 2026-03-20 08:37
The proportion of high-IQ people in the world is declining fast. https://t.co/1aFXvaAjby

@0xSero · 2026-03-19 01:03
Putting out a wish to the universe. I need more compute, if I can get more I will make sure every machine from a small phone to a bootstrapped RTX 3090 node can run frontier intelligence fast with minimal intelligence loss. I have hit page 2 of huggingface, released 3 model family compressions and got GLM-4.7 on a MacBook https://t.co/lorDSUEYCL My beast just isn’t enough and I already spent 2k usd on renting GPUs on top of credits provided by Prime intellect and Hotaisle. ——— If you believe in what I do help me get this to Nvidia, maybe they will bless me with the pewter to keep making local AI more accessible 🙏

@__tinygrad__ · 2026-03-20 11:33
@0xSero Happy to give you access to tinyboxes. 4090/5090/7900XTX/9070XT. Join our Discord and DM me.

@elonmusk · 2026-03-20 11:52
So many phonies, so few who are the real deal

@elonmusk · 2026-03-20 11:52
So many phonies, so few who are the real deal

@__tinygrad__ · 2026-03-20 13:27
@elonmusk It's all about the space you are in. In deep learning compilers and kernel optimization there's very few phonies. In self driving cars and humanoid robots on the other hand...

@jonatanpallesen · 2026-03-20 08:37
The proportion of high-IQ people in the world is declining fast. https://t.co/1aFXvaAjby

@__tinygrad__ · 2026-03-20 13:31
@jonatanpallesen Just in time! https://t.co/pDvM2LXdl6

@0xSero · 2026-03-20 15:56
https://t.co/txDnDa0Flf

@olafwillocx · 2026-03-20 18:39
What will replace Python to build AI products, when AI hardware becomes more complicated and specialized, away from just CPUs and GPUs, as we've seen at GTX a few days ago? I suspect it will be more like Unreal Engine blueprints.

@olafwillocx · 2026-03-20 18:42
Tinygrad (and others) are so far ahead, it's becoming clearer why they are the path forward. What they don't expose yet though, what is very important imo, is the graph structure of the machines themselves. Still need to have this secret mental picture in your head.

@__tinygrad__ · 2026-03-21 08:06
Few know this, but I (George) was the only person in history to get a perfect score in CMU compilers, which is likely the best compilers course in the world. Combine that with crazy low level knowledge of hardware from 10 years of hacking. Then add a team of people who are talented enough to push back on my dumb ideas and clean up the implementations of the good ones. The team who keeps this whole operation running, software, infrastructure, and product. I love how there's no hype in deep learning compilers. It was one of the most annoying things about self driving cars, all the noobs who burned through billions on crap that was obviously dumb, and the companies who deserved to go bankrupt years ago if not for government bailouts (Tesla and China will devour them all). In this space, the competition is @jimkxa at Tenstorrent, @clattner_llvm at Modular, and @JeffDean at Google. Three of the living legends of computer science. And companies like @nvidia and @AMD, who are definitely live players, making single chips that have more power than the whole Internet two decades ago. This space is so fun to play in. If you haven't, read the tinygrad spec. It's all coming together beautifully.

@__tinygrad__ · 2026-03-21 08:06
Few know this, but I (George) was the only person in history to get a perfect score in CMU compilers, which is likely the best compilers course in the world. Combine that with crazy low level knowledge of hardware from 10 years of hacking. Then add a team of people who are talented enough to push back on my dumb ideas and clean up the implementations of the good ones. The team who keeps this whole operation running, software, infrastructure, and product. I love how there's no hype in deep learning compilers. It was one of the most annoying things about self driving cars, all the noobs who burned through billions on crap that was obviously dumb, and the companies who deserved to go bankrupt years ago if not for government bailouts (Tesla and China will devour them all). In this space, the competition is @jimkxa at Tenstorrent, @clattner_llvm at Modular, and @JeffDean at Google. Three of the living legends of computer science. And companies like @nvidia and @AMD, who are definitely live players, making single chips that have more power than the whole Internet two decades ago. This space is so fun to play in. If you haven't, read the tinygrad spec. It's all coming together beautifully.

@veerbhanX · 2026-03-21 08:35
@__tinygrad__ Incredibly fun indeed. Would invite you to take look at our DL compiler.

@veerbhanX · 2026-03-21 08:35
@__tinygrad__ Incredibly fun indeed. Would invite you to take look at our DL compiler.

@__tinygrad__ · 2026-03-21 08:38
@veerbhanX Do you have a chip I can buy one of? (no contact us, just add to cart)

@__tinygrad__ · 2026-03-21 08:06
Few know this, but I (George) was the only person in history to get a perfect score in CMU compilers, which is likely the best compilers course in the world. Combine that with crazy low level knowledge of hardware from 10 years of hacking. Then add a team of people who are talented enough to push back on my dumb ideas and clean up the implementations of the good ones. The team who keeps this whole operation running, software, infrastructure, and product. I love how there's no hype in deep learning compilers. It was one of the most annoying things about self driving cars, all the noobs who burned through billions on crap that was obviously dumb, and the companies who deserved to go bankrupt years ago if not for government bailouts (Tesla and China will devour them all). In this space, the competition is @jimkxa at Tenstorrent, @clattner_llvm at Modular, and @JeffDean at Google. Three of the living legends of computer science. And companies like @nvidia and @AMD, who are definitely live players, making single chips that have more power than the whole Internet two decades ago. This space is so fun to play in. If you haven't, read the tinygrad spec. It's all coming together beautifully.

@wood_work16666 · 2026-03-21 09:13
@__tinygrad__ How would you spend $80 billion better than Meta on Metaverse

@ShriKaranHanda · 2026-03-21 09:18
Crazy launch Congratulations to George Hotz and the team at Tinygrad for shipping this! https://t.co/XYudGl7TsJ

@wood_work16666 · 2026-03-21 09:13
@__tinygrad__ How would you spend $80 billion better than Meta on Metaverse

@__tinygrad__ · 2026-03-21 09:23
@wood_work16666 they literally had @ID_AA_Carmack wanting to fix it. he wrote a nice departing letter about how. they didn't listen. hopeless.

@__tinygrad__ · 2026-03-21 09:51
@ShriKaranHanda this is not related to us. screenshotted this post as further evidence of their trademark infringement.

@mohittwwt · 2026-03-21 09:53
@__tinygrad__ @ShriKaranHanda Woah i legit thought this is tiny, but its tiiny

@mohittwwt · 2026-03-21 09:53
@__tinygrad__ @ShriKaranHanda Woah i legit thought this is tiny, but its tiiny

@__tinygrad__ · 2026-03-21 09:55
@mohittwwt @ShriKaranHanda very not cool what they did. most IP law is dumb, but trademark is the most sensible of the areas, and this is textbook trademark infringement. reminds me of the Rollex watch I bought in China 10 years ago.

@__tinygrad__ · 2026-03-21 09:55
@mohittwwt @ShriKaranHanda very not cool what they did. most IP law is dumb, but trademark is the most sensible of the areas, and this is textbook trademark infringement. reminds me of the Rollex watch I bought in China 10 years ago.

@mohittwwt · 2026-03-21 10:02
@__tinygrad__ @ShriKaranHanda That rollex reference lol😂

@mohittwwt · 2026-03-21 10:02
@__tinygrad__ @ShriKaranHanda That rollex reference lol😂

@__tinygrad__ · 2026-03-21 10:06
@mohittwwt @ShriKaranHanda .@TiinyAILab consider this a cease and desist. I have gotten several messages from people thinking we are related to you. build w/e, but don't mislead with our brand. not answering the support requests that will come if you ever actually ship. if that happens, we will sue.

@iruletheworldmo · 2026-03-21 10:28
do people really think open source has a chance. this guys argument feels like, give me money and i can build a netflix competitor from my living room. the only way ‘open source’ wins is if openai or dario do it. and they won’t. because if they did. composer 3 would be agi.

@__tinygrad__ · 2026-03-21 10:06
@mohittwwt @ShriKaranHanda .@TiinyAILab consider this a cease and desist. I have gotten several messages from people thinking we are related to you. build w/e, but don't mislead with our brand. not answering the support requests that will come if you ever actually ship. if that happens, we will sue.

@KadakKaspar · 2026-03-21 11:20
@__tinygrad__ @mohittwwt @ShriKaranHanda @TiinyAILab You gonna sue because its called Tiiny?

@KadakKaspar · 2026-03-21 11:20
@__tinygrad__ @mohittwwt @ShriKaranHanda @TiinyAILab You gonna sue because its called Tiiny?

@__tinygrad__ · 2026-03-21 11:51
@KadakKaspar @mohittwwt @ShriKaranHanda @TiinyAILab no, I don't care about that. it's because, due to the name and the logo being rip offs of ours, people think we had something to do with their product. and if and when their product is bad people will think negatively of us. all they need to do is change name and logo.

@KadakKaspar · 2026-03-21 11:20
@__tinygrad__ @mohittwwt @ShriKaranHanda @TiinyAILab You gonna sue because its called Tiiny?

@__tinygrad__ · 2026-03-21 11:51
@KadakKaspar @mohittwwt @ShriKaranHanda @TiinyAILab no, I don't care about that. it's because, due to the name and the logo being rip offs of ours, people think we had something to do with their product. and if and when their product is bad people will think negatively of us. all they need to do is change name and logo.

@__tinygrad__ · 2026-03-21 11:57
@KadakKaspar @mohittwwt @ShriKaranHanda @TiinyAILab if you want to call something tiny, that's a word. if you want use a pixel art logo, many others do too. and if you want to sell ai computers, that's also great. but do all three, and that's not a coincidence. that's deliberately trying to confuse with our brand.

@iruletheworldmo · 2026-03-21 10:28
do people really think open source has a chance. this guys argument feels like, give me money and i can build a netflix competitor from my living room. the only way ‘open source’ wins is if openai or dario do it. and they won’t. because if they did. composer 3 would be agi.

@thdxr · 2026-03-22 01:43
the resources of all the companies who want to stop paying the duopoly add up to a lot that's not a guarantee that it gets deployed effectively but the incentives are there look at things historically - went from proprietary default to oss default because of this - compilers - programming languages - databases

@latent_node · 2026-03-22 02:15
https://t.co/wmZkwcMoIT

@danieltvela · 2026-03-22 06:36
Use Qwen3.5-35B-A3B-8bit on your Mac: - 54 tok/s in a M4 Pro (20c GPU), 64GB Leave 27B model to NVIDIAsers.

@effectfully · 2026-03-22 16:08
bro https://t.co/FnYekFJ4Em

@effectfully · 2026-03-22 16:08
bro https://t.co/FnYekFJ4Em

@__tinygrad__ · 2026-03-23 02:39
@effectfully I (George) make 100k at tiny, have never sold a share of either @comma_ai or @__tinygrad__ and have no intention of doing so. You might be able to make more in advertising or finance, but when you look back at your life you'll have nothing to show for it.

@__tinygrad__ · 2026-03-23 02:39
@effectfully I (George) make 100k at tiny, have never sold a share of either @comma_ai or @__tinygrad__ and have no intention of doing so. You might be able to make more in advertising or finance, but when you look back at your life you'll have nothing to show for it.

@light2909 · 2026-03-23 02:46
@__tinygrad__ @effectfully @comma_ai 100k could be charactizered as "nothing to show for it"

@light2909 · 2026-03-23 02:46
@__tinygrad__ @effectfully @comma_ai 100k could be charactizered as "nothing to show for it"

@__tinygrad__ · 2026-03-23 03:02
@light2909 @effectfully @comma_ai you know those numbers are just in some dude's SQL database somewhere, right? you work your whole life to run one SQL command? that's definitely nothing to show for it, regardless of the number. something to show is something that wouldn't have existed without you.

@thdxr · 2026-03-22 01:43
the resources of all the companies who want to stop paying the duopoly add up to a lot that's not a guarantee that it gets deployed effectively but the incentives are there look at things historically - went from proprietary default to oss default because of this - compilers - programming languages - databases

@__tinygrad__ · 2026-03-23 04:37
@thdxr @iruletheworldmo I think it depends on scaling laws and what people actually want. If there's a threshold of "good enough" open source has this in the bag. If there's no good enough and scaling laws require bigger and bigger computers, then it could go either way.

@sarlev_ · 2026-03-23 05:47
this is the right mindset to have on anything you do.

@ashleyshrimpe · 2026-03-23 03:48
@__tinygrad__ @effectfully @comma_ai I never understood the founder brag of low salary. That’s a given. Uh, bro, you have equity lol

@__tinygrad__ · 2026-03-23 07:07
@ashleyshrimpe @effectfully @comma_ai The equity doesn't pay anything unless you sell it. My main goal as a founder with equity is to make sure VCs and MBA types never get control to enshittify, and the company can focus on building technology. It doesn't matter how rich you are if you aren't proud of what you made.

@__tinygrad__ · 2026-03-23 07:07
@ashleyshrimpe @effectfully @comma_ai The equity doesn't pay anything unless you sell it. My main goal as a founder with equity is to make sure VCs and MBA types never get control to enshittify, and the company can focus on building technology. It doesn't matter how rich you are if you aren't proud of what you made.

@__tinygrad__ · 2026-03-23 07:09
@ashleyshrimpe @effectfully @comma_ai The only ethical way for a founder to sell equity is if the company is making a genuine profit and you sell it back to the company (or through dividends). And only if there's no good way to deploy capital to improve the technology further.

@bajpaiharsh244 · 2026-03-23 08:29
Haha, geohot is tagging PRs with the line "ai slop" XD https://t.co/yq2Erjqc0c

@__tinygrad__ · 2026-03-23 08:53
It's important to confirm your library can be used by LLMs. That LLM coded flash attention in tinygrad outperforms the AOTriton one in PyTorch on my AMD Strix Halo.

@__tinygrad__ · 2026-03-23 09:00
And it's not close. It's 1.8x times faster. This is using the tinygrad DSL. The replacement for BEAM will be LLM. https://t.co/Al0BrQGAsz

@PhilYogurt · 2026-03-23 09:26
@__tinygrad__ This is insane. Do you have it debugging automatically via gdb? I've saved so much time like that.

@PhilYogurt · 2026-03-23 09:26
@__tinygrad__ This is insane. Do you have it debugging automatically via gdb? I've saved so much time like that.

@__tinygrad__ · 2026-03-23 09:28
@PhilYogurt tinygrad has a built in profiler that blows away anything public for any GPU I have seen. RDNA3 is the best supported, try it with VIZ=2 for the web version, and extra/viz/cli.py for the text based version.

@__tinygrad__ · 2026-03-23 10:14
The reason it was easy to get our flash attention to be 1.8x faster than torch is the quality of our kernel profiler. If you have RDNA3, run with VIZ=2. https://t.co/Zg21l5MTe2

@__tinygrad__ · 2026-03-23 10:14
The reason it was easy to get our flash attention to be 1.8x faster than torch is the quality of our kernel profiler. If you have RDNA3, run with VIZ=2. https://t.co/Zg21l5MTe2

@__tinygrad__ · 2026-03-23 10:15
You can zoom in and see the issue and exec of each instruction. It makes seeing bottlenecks so fast. https://t.co/WixgLneXJd

@__tinygrad__ · 2026-03-23 10:15
You can zoom in and see the issue and exec of each instruction. It makes seeing bottlenecks so fast. https://t.co/WixgLneXJd

@__tinygrad__ · 2026-03-23 10:18
LLMs can play too, you don't need the web interface. extra/viz/cli.py can read the same profiler files. It's still a bit rough around the edges, but this is going to enable the best autoresearch pipeline for kernel speed. https://t.co/lTRideJxGm

@__tinygrad__ · 2026-03-23 10:18
LLMs can play too, you don't need the web interface. extra/viz/cli.py can read the same profiler files. It's still a bit rough around the edges, but this is going to enable the best autoresearch pipeline for kernel speed. https://t.co/lTRideJxGm

@antnjbert · 2026-03-23 10:25
That’s exactly how I think about the Tenstorrent stack - have a tool that provides immediate feedback to LLMs and leave the optimization to them. It’s not that the hardware isn’t capable of running models quickly; it’s that people don’t optimize enough. And I understand why you like AMD - the hardware is good, and the software stack should improve soon

@antnjbert · 2026-03-23 10:25
That’s exactly how I think about the Tenstorrent stack - have a tool that provides immediate feedback to LLMs and leave the optimization to them. It’s not that the hardware isn’t capable of running models quickly; it’s that people don’t optimize enough. And I understand why you like AMD - the hardware is good, and the software stack should improve soon

@__tinygrad__ · 2026-03-23 10:27
@antnjbert the tenstorrent stack is written in C++24 and barely compiles except on a machine with an exact configuration. it's insanely complex and the hardware isn't well documented. AMD on the other hand open sourced rocprof-trace-decoder and has better warp tracing hardware than NVIDIA

@__tinygrad__ · 2026-03-23 10:27
@antnjbert the tenstorrent stack is written in C++24 and barely compiles except on a machine with an exact configuration. it's insanely complex and the hardware isn't well documented. AMD on the other hand open sourced rocprof-trace-decoder and has better warp tracing hardware than NVIDIA

@antnjbert · 2026-03-23 10:34
@__tinygrad__ Don’t spoil the surprise for me lol - they’ve already managed to run a few models on it: https://t.co/gDTHOYAWxG, let’s see how hard it’s going to be

@antnjbert · 2026-03-23 10:34
@__tinygrad__ Don’t spoil the surprise for me lol - they’ve already managed to run a few models on it: https://t.co/gDTHOYAWxG, let’s see how hard it’s going to be

@__tinygrad__ · 2026-03-23 10:38
@antnjbert I wrote some (low quality) notes for @tenstorrent but they didn't respond. it's not intel tier hopeless, but they need to change things if they want a chance of winning. https://t.co/CHnzYxTegt

@__tinygrad__ · 2026-03-23 02:39
@effectfully I (George) make 100k at tiny, have never sold a share of either @comma_ai or @__tinygrad__ and have no intention of doing so. You might be able to make more in advertising or finance, but when you look back at your life you'll have nothing to show for it.

@jsuarez · 2026-03-23 12:44
@__tinygrad__ @effectfully @comma_ai You can make more money without having to sacrifice anything. Can't you open up an extra service or two for big companies where they're hemorrhaging money on something dumb your infra just does?

@sarlev_ · 2026-03-23 05:47
this is the right mindset to have on anything you do.

@__tinygrad__ · 2026-03-23 16:06
@sarlev_ this is how you get quality and not slop. you need people who are intrinsically motivated.

@jsuarez · 2026-03-23 12:44
@__tinygrad__ @effectfully @comma_ai You can make more money without having to sacrifice anything. Can't you open up an extra service or two for big companies where they're hemorrhaging money on something dumb your infra just does?

@__tinygrad__ · 2026-03-23 16:10
@jsuarez @effectfully @comma_ai what would we do with more money? we considered raising for a datacenter but realized we have what we need to build the exabox and can lease. and money certainly doesn't buy software quality, look at the $80B metaverse.

@__tinygrad__ · 2026-03-23 16:13
@AGI_is_solved @effectfully @comma_ai the kind of people who measure their success in life by how much money they make aren't the type we'd want to hire anyway. I think a lot of those people worked on the $80B metaverse and look how that turned out.

@effectfully · 2026-03-23 16:42
> when you look back at your life you'll have nothing to show for it Unlike of course your employees, who will look back at their lives and have to show *checks notes* their CEO saying they worked for a scam -- on a below‑market salary. https://t.co/V3ZayK42Cq

@__tinygrad__ · 2026-03-23 16:10
@jsuarez @effectfully @comma_ai what would we do with more money? we considered raising for a datacenter but realized we have what we need to build the exabox and can lease. and money certainly doesn't buy software quality, look at the $80B metaverse.

@jsuarez · 2026-03-23 16:46
@__tinygrad__ @effectfully @comma_ai be less squeezed, do whatever you want? I didn't say raise more hell no. Like if Puffer just randomly got an extra 10m per year, we'd probably spin up a couple experimental projects, have some fun, build a few things that end up being useful

@jsuarez · 2026-03-23 16:46
@__tinygrad__ @effectfully @comma_ai be less squeezed, do whatever you want? I didn't say raise more hell no. Like if Puffer just randomly got an extra 10m per year, we'd probably spin up a couple experimental projects, have some fun, build a few things that end up being useful

@__tinygrad__ · 2026-03-23 16:50
@jsuarez @effectfully @comma_ai so idk what these services are we could open up to big companies. what are they? who's gonna do it? what is now not getting done since we are doing that?

@__tinygrad__ · 2026-03-23 16:50
@jsuarez @effectfully @comma_ai so idk what these services are we could open up to big companies. what are they? who's gonna do it? what is now not getting done since we are doing that?

@jsuarez · 2026-03-23 16:54
idk what makes sense for you. We do some contracting on high perf rl envs. It takes some time from core, but it also lets us try our tech on a bunch of problems and informs what areas we need to improve. I figure we'd spend as much time on investors as we do on this anyhow. Does that bring any ideas to mind?

@effectfully · 2026-03-23 16:42
> when you look back at your life you'll have nothing to show for it Unlike of course your employees, who will look back at their lives and have to show *checks notes* their CEO saying they worked for a scam -- on a below‑market salary. https://t.co/V3ZayK42Cq

@__tinygrad__ · 2026-03-23 16:55
@effectfully you are aware there's a comma four now, right? it's a $1,000 aftermarket kit that adds ADAS features to 325 different cars. I know it's so strange that a silicon valley startup just does what they said they are going to do...we are going to commoditize the petaflop btw.

@jsuarez · 2026-03-23 16:54
idk what makes sense for you. We do some contracting on high perf rl envs. It takes some time from core, but it also lets us try our tech on a bunch of problems and informs what areas we need to improve. I figure we'd spend as much time on investors as we do on this anyhow. Does that bring any ideas to mind?

@__tinygrad__ · 2026-03-23 16:59
@jsuarez @effectfully @comma_ai we do contracts also, we did one last year for the Qualcomm DSP and this year for AMD MLPerf. it's not super lucrative though, tinybox business makes more.

@__tinygrad__ · 2026-03-23 16:59
@jsuarez @effectfully @comma_ai we do contracts also, we did one last year for the Qualcomm DSP and this year for AMD MLPerf. it's not super lucrative though, tinybox business makes more.

@jsuarez · 2026-03-23 17:01
@__tinygrad__ @effectfully @comma_ai Seems like we're both missing something then. If you have infra that much better than everyone else, gold bars should rain from the sky and you should have unlimited budget to do whatever other cool projects you feel like. We're doing fine but no unlimited gold bars yet.

@jsuarez · 2026-03-23 17:01
@__tinygrad__ @effectfully @comma_ai Seems like we're both missing something then. If you have infra that much better than everyone else, gold bars should rain from the sky and you should have unlimited budget to do whatever other cool projects you feel like. We're doing fine but no unlimited gold bars yet.

@__tinygrad__ · 2026-03-23 17:12
@jsuarez @effectfully @comma_ai So we only would consider open source contracts, I wouldn't touch a "make our private model fast for just us" type contract. idk, I like the hardware business a lot more, it's straightforward and keeps you honest.

@steipete · 2026-03-24 01:23
Pretty much every PR I review: 0) review <URL> [codex does it's thing and finds issues] 1) is the issue clear? [if not, trash PR] 2) is this the best possible fix? [95% of the time no] 3) continue discussion, consider tradeoffs, usually rewrite PR Most folks send too localized, small fixes that would end up making the project unmaintainable.

@boopdotpng · 2026-03-24 02:15
debug bus on blackhole is severely lacking, its really had to profile all the traces you need from host because the pcie reads are so slow.... i wont be able to match what tinygrad/amd are doing but it will be close

@LottoLabs · 2026-03-23 12:27
Qwen 3.5 27b, llama.cpp server, hermes agent, tailscale, 3090 The stack remains undefeated Testing all inference engines, after that seeing if there are any model specific tweaks we can make to get it running faster After that maybe maybe mess around with serving w/ tinygrad just for fun

@francip · 2026-03-24 05:12
@LottoLabs @max_paperclips I want to see numbers of tinygrad vs llama.cop

@__tinygrad__ · 2026-03-24 07:46
To anyone using local LLMs, what hardware do you have, what model are you using, and what tok/s are you getting?

@__tinygrad__ · 2026-03-24 07:46
To anyone using local LLMs, what hardware do you have, what model are you using, and what tok/s are you getting?

@snapolino · 2026-03-24 07:49
@__tinygrad__ Qwen 3.5 27B , at Q6 quants , 110k ctx length, 450pp / 12.5 tg , 2 x RTX 2080 Ti 22GB vram At least its my current driver model till there is something better

@__tinygrad__ · 2026-03-24 07:46
To anyone using local LLMs, what hardware do you have, what model are you using, and what tok/s are you getting?

@danieltvela · 2026-03-24 07:52
@__tinygrad__ M4 Pro (20c GPU), 64GB Best for Mac are MoE models. With Low GPU + High RAM, the best is big model with only a few activated. https://t.co/oGHPI0w3Tq

@snapolino · 2026-03-24 07:49
@__tinygrad__ Qwen 3.5 27B , at Q6 quants , 110k ctx length, 450pp / 12.5 tg , 2 x RTX 2080 Ti 22GB vram At least its my current driver model till there is something better

@__tinygrad__ · 2026-03-24 07:53
@snapolino That model does impressively well on pinchbench https://t.co/6oY7q4lerB

@danieltvela · 2026-03-24 07:52
@__tinygrad__ M4 Pro (20c GPU), 64GB Best for Mac are MoE models. With Low GPU + High RAM, the best is big model with only a few activated. https://t.co/oGHPI0w3Tq

@__tinygrad__ · 2026-03-24 07:56
@danieltvela Yea, same with AMD Strix Halo. These MoE models are definitely the right choice for the lower bandwidth higher capacity systems.

@__tinygrad__ · 2026-03-24 07:46
To anyone using local LLMs, what hardware do you have, what model are you using, and what tok/s are you getting?

@SediBY571 · 2026-03-24 07:57
@__tinygrad__ @__tinygrad__ when the nanobox? I don't want to get the DGX.

@__tinygrad__ · 2026-03-24 07:46
To anyone using local LLMs, what hardware do you have, what model are you using, and what tok/s are you getting?

@chkn_little · 2026-03-24 07:57
@__tinygrad__ most of the time qwen 3.5 122b mxfp4_moe on asus ascend gx10

@__tinygrad__ · 2026-03-24 07:46
To anyone using local LLMs, what hardware do you have, what model are you using, and what tok/s are you getting?

@CustomWetware · 2026-03-24 07:57
@__tinygrad__ Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled.i1-Q4_K_M.gguf 34 t/s and ~2000 pp/s with llama.cpp on a 4090

@SediBY571 · 2026-03-24 07:57
@__tinygrad__ @__tinygrad__ when the nanobox? I don't want to get the DGX.

@__tinygrad__ · 2026-03-24 07:58
@SediBY571 Buy a Framework Desktop or HP ZBook G1a. So many good choice there we don't need to build, but AMD Strix Halo is the chip.

@chkn_little · 2026-03-24 07:57
@__tinygrad__ most of the time qwen 3.5 122b mxfp4_moe on asus ascend gx10

@__tinygrad__ · 2026-03-24 08:03
@chkn_little ahh, an NVIDIA tax payer. actually with RAM prices being what they are that machine isn't such a bad deal.

@CustomWetware · 2026-03-24 07:57
@__tinygrad__ Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled.i1-Q4_K_M.gguf 34 t/s and ~2000 pp/s with llama.cpp on a 4090

@__tinygrad__ · 2026-03-24 08:06
@CustomWetware Interesting, it seems everyone with GDDR went 27B, and people with LPDDR went 35B-A3B or 122B-A10B

@__tinygrad__ · 2026-03-24 08:03
@chkn_little ahh, an NVIDIA tax payer. actually with RAM prices being what they are that machine isn't such a bad deal.

@chkn_little · 2026-03-24 08:08
@__tinygrad__ the power also i have 4 x 3090 and got the spark instead of doubling up on the 3090s because of power

@__tinygrad__ · 2026-03-24 07:46
To anyone using local LLMs, what hardware do you have, what model are you using, and what tok/s are you getting?

@latent_node · 2026-03-24 08:11
@__tinygrad__ Using a cluster of Mac’s m4 and m3 max getting 39 Tok/s with latest qwen 3.5 9B using the optiq quants - https://t.co/lUrH7VmMqF wrote about the setup here - https://t.co/bgOGIE7Fn7

@chkn_little · 2026-03-24 08:08
@__tinygrad__ the power also i have 4 x 3090 and got the spark instead of doubling up on the 3090s because of power

@__tinygrad__ · 2026-03-24 08:12
@chkn_little the framework desktop is the AMD version, $800 cheaper and with tinygrad should have the same perf as the NVIDIA box very soon.

@latent_node · 2026-03-24 08:11
@__tinygrad__ Using a cluster of Mac’s m4 and m3 max getting 39 Tok/s with latest qwen 3.5 9B using the optiq quants - https://t.co/lUrH7VmMqF wrote about the setup here - https://t.co/bgOGIE7Fn7

@__tinygrad__ · 2026-03-24 08:14
@thin_signal are you sure the clustering helps? I would expect more perf on a 9B model from just the M3 Max

@kechogarcia · 2026-03-24 11:27
If you spend 99% of your time filtering through slop, what's the whole point lol At what point is just faster to write the thing yourself with perfect and careful craft.

@__tinygrad__ · 2026-03-24 13:04
The hyperscalars are currently spending $1T on ultra-luxury houses for AIs. Air conditioning, redundant power, and spotless floors. They imagine bougie Claude and Gemini with disposable income, but really it's mostly Kimi and Qwen in the slums. We build affordable housing. https://t.co/hBZGqihyIU

@__tinygrad__ · 2026-03-24 13:04
The hyperscalars are currently spending $1T on ultra-luxury houses for AIs. Air conditioning, redundant power, and spotless floors. They imagine bougie Claude and Gemini with disposable income, but really it's mostly Kimi and Qwen in the slums. We build affordable housing. https://t.co/hBZGqihyIU

@ashleyshrimpe · 2026-03-24 13:29
@__tinygrad__ Is your target client base middle market companies that want to train an LLM in house rather than giving their data to train the hyperscalers?

@ashleyshrimpe · 2026-03-24 13:29
@__tinygrad__ Is your target client base middle market companies that want to train an LLM in house rather than giving their data to train the hyperscalers?

@__tinygrad__ · 2026-03-24 13:32
@ashleyshrimpe lol our "target client base" is AIs who are gonna pay their own rent to live in our mobile home community

@__tinygrad__ · 2026-03-24 07:46
To anyone using local LLMs, what hardware do you have, what model are you using, and what tok/s are you getting?

@jmbollenbacher · 2026-03-24 14:50
@__tinygrad__ When is tinygrad gonna support qwen3.5? I tried for an hour or so to write an implementation myself but then realized i didnt have the time to do it properly that day. But I'd love to see an official implementation soon. Seems like theres demand for it.

@jmbollenbacher · 2026-03-24 14:50
@__tinygrad__ When is tinygrad gonna support qwen3.5? I tried for an hour or so to write an implementation myself but then realized i didnt have the time to do it properly that day. But I'd love to see an official implementation soon. Seems like theres demand for it.

@__tinygrad__ · 2026-03-24 15:38
@jmbollenbacher https://t.co/UclOkmL6Xw

@boopdotpng · 2026-03-24 02:15
debug bus on blackhole is severely lacking, its really had to profile all the traces you need from host because the pcie reads are so slow.... i wont be able to match what tinygrad/amd are doing but it will be close

@__tinygrad__ · 2026-03-24 15:40
@boopdotpng AMD's debug hardware is so good you almost can't believe it's real

@karpathy · 2026-03-24 16:56
Software horror: litellm PyPI supply chain attack. Simple `pip install litellm` was enough to exfiltrate SSH keys, AWS/GCP/Azure creds, Kubernetes configs, git credentials, env vars (all your API keys), shell history, crypto wallets, SSL private keys, CI/CD secrets, database passwords. LiteLLM itself has 97 million downloads per month which is already terrible, but much worse, the contagion spreads to any project that depends on litellm. For example, if you did `pip install dspy` (which depended on litellm>=1.64.0), you'd also be pwnd. Same for any other large project that depended on litellm. Afaict the poisoned version was up for only less than ~1 hour. The attack had a bug which led to its discovery - Callum McMahon was using an MCP plugin inside Cursor that pulled in litellm as a transitive dependency. When litellm 1.82.8 installed, their machine ran out of RAM and crashed. So if the attacker didn't vibe code this attack it could have been undetected for many days or weeks. Supply chain attacks like this are basically the scariest thing imaginable in modern software. Every time you install any depedency you could be pulling in a poisoned package anywhere deep inside its entire depedency tree. This is especially risky with large projects that might have lots and lots of dependencies. The credentials that do get stolen in each attack can then be used to take over more accounts and compromise more packages. Classical software engineering would have you believe that dependencies are good (we're building pyramids from bricks), but imo this has to be re-evaluated, and it's why I've been so growingly averse to them, preferring to use LLMs to "yoink" functionality when it's simple enough and possible.

@ThePrimeagen · 2026-03-24 22:15
GitHub just living the dream right now https://t.co/e93lFl3YUm

@ThePrimeagen · 2026-03-24 22:15
GitHub just living the dream right now https://t.co/e93lFl3YUm

@Jonathan_Blow · 2026-03-24 22:48
@ThePrimeagen Hey man, 90% uptime means you are only down for two and a half hours per day.

@__tinygrad__ · 2026-03-25 00:08
RT @yassineyousfi_: today is a good day to clean up your dependency tree, be like @__tinygrad__ https://t.co/tgYfGqGjz2

@__tinygrad__ · 2026-03-24 07:46
To anyone using local LLMs, what hardware do you have, what model are you using, and what tok/s are you getting?

@cyberwhisperr · 2026-03-25 03:26
@__tinygrad__ I am dreaming that I have the 4x rtx pro 6000 machine, can you help materialize it?

@cyberwhisperr · 2026-03-25 03:26
@__tinygrad__ I am dreaming that I have the 4x rtx pro 6000 machine, can you help materialize it?

@__tinygrad__ · 2026-03-25 03:43
@cyberwhisperr for $65k, sure!

@__tinygrad__ · 2026-03-25 04:09
I would rather have:

@__tinygrad__ · 2026-03-25 04:09
I would rather have:

@JakeGibson · 2026-03-25 04:22
@__tinygrad__ Looks like a lot of people haven't tried kimi

@__tinygrad__ · 2026-03-25 04:09
I would rather have:

@rskvll · 2026-03-25 04:35
@__tinygrad__ These guys voting for opus have probably never experienced an actually fast LLM

@rskvll · 2026-03-25 04:35
@__tinygrad__ These guys voting for opus have probably never experienced an actually fast LLM

@__tinygrad__ · 2026-03-25 05:16
@rskllx where have you tried one? I tried baseten's Kimi, but it seems dumber than a normal Kimi, I know they are doing some weird quant

@__tinygrad__ · 2026-03-25 04:09
I would rather have:

@auroter · 2026-03-25 05:45
@__tinygrad__ No offense to the people who worked hard on Kimi but it is completely unusable for real work, compared to Qwen 3.5 397B or Opus 4.6. Speed doesn’t matter if the model is stupid.

@JakeGibson · 2026-03-25 04:22
@__tinygrad__ Looks like a lot of people haven't tried kimi

@__tinygrad__ · 2026-03-25 06:53
@JakeGibson what's funny is from this statement I don't know which way you voted in the poll

@auroter · 2026-03-25 05:45
@__tinygrad__ No offense to the people who worked hard on Kimi but it is completely unusable for real work, compared to Qwen 3.5 397B or Opus 4.6. Speed doesn’t matter if the model is stupid.

@__tinygrad__ · 2026-03-25 06:53
@auroter you think qwen 3.5 is better?

@__tinygrad__ · 2026-03-25 06:54
Which of the top open models is the best?

@keennay · 2026-02-25 02:43
Running Kimi-K2.5 on 8x RTX Pro 6000 Blackwells, with plans to eventually test a CPU/GPU hybrid inference setup through KTransformers+SGLang on 4x of the same GPUs Very curious to gauge the overall performance with the hybrid setup compared to a quantized Kimi-K2.5 fit across the 4 GPUs. The hybrid setup will need close to 768GB of RAM To start here's a baseline across 8x GPUs using a synthetic coding agent style workload targeting 2k-45k input tokens, 80-3k max output tokens, and with up to 10 concurrent requests. SGLang's --mem-fraction-static flag is set to 0.90 Baseline avg throughput: ~74 output tokens/s @ 10 concurrent requests

@__tinygrad__ · 2026-03-25 07:11
@keennay @Kimi_Moonshot @lmsysorg Is that 74 tok/s per request or all together? What's the BS=1 speed?

@__tinygrad__ · 2026-03-25 04:09
I would rather have:

@fedri_ss · 2026-03-25 07:35
@__tinygrad__ Should've made this with glm-5

@fedri_ss · 2026-03-25 07:35
@__tinygrad__ Should've made this with glm-5

@__tinygrad__ · 2026-03-25 07:37
@fedri_ss That changes the answer for you? You think GLM-5 is better than Kimi?

@__tinygrad__ · 2026-03-25 06:54
Which of the top open models is the best?

@agitbackprop · 2026-03-25 09:00
@__tinygrad__ qwen 3.5 27b dense

@Jonathan_Blow · 2026-03-24 22:48
@ThePrimeagen Hey man, 90% uptime means you are only down for two and a half hours per day.

@__tinygrad__ · 2026-03-25 09:01
@Jonathan_Blow @ThePrimeagen So much focus on the down hours, it's up for twenty one and a half hours every day, that's a lot better than most people!

@agitbackprop · 2026-03-25 09:00
@__tinygrad__ qwen 3.5 27b dense

@__tinygrad__ · 2026-03-25 09:02
@agitbackprop that's so wild if true, but it is the claim of pinchbench

@__tinygrad__ · 2026-03-25 10:11
We are looking to hire someone to improve our LLM runner, with USB GPU + high BS=1 tok/s it should be used a lot soon. The TODO list is in the Discord, but no bounties since that yields AI slop. The bottleneck today isn't writing code, it's filtering it. Show me you can do that.

@__tinygrad__ · 2026-03-25 10:13
The runner is here, a 435 line 100% self contained (no tokenizers, no model classes) LLM runner with an OpenAI compatible server. https://t.co/da8YN7C06G

@__tinygrad__ · 2026-03-25 10:21
Branching off of this is all the fun stuff. Megakernels, fast GGUF on the fly unpacker, function decorator for Python speed, KV cache swap to disk. This should stay ~500 lines, but outperform all the BS=1 LLM runners through the power of tinygrad. Kimi at 500 tok/s on MI350X?

@__tinygrad__ · 2026-03-25 10:21
Branching off of this is all the fun stuff. Megakernels, fast GGUF on the fly unpacker, function decorator for Python speed, KV cache swap to disk. This should stay ~500 lines, but outperform all the BS=1 LLM runners through the power of tinygrad. Kimi at 500 tok/s on MI350X?

@__tinygrad__ · 2026-03-25 10:23
Eventually the world will realize that there's still so much value in software, but the value is in discernment and not klocs of code. This is exactly what we at tinygrad have been saying for years, it just might take "agentic coding" for everyone else to catch up.

@jaywyawhare · 2026-03-25 10:29
Update on C-ML: Now we are close to being done with the herculean task of making C-ML match feature parity with tinygrad. We do support distributed training along with support for different accelerator backends like CUDA, ROCm, Vulkan, and others. There are some bugs, and I am actively working on fixes. Check it out: https://t.co/Z7W0gl8Ktf

@kechogarcia · 2026-03-24 11:27
If you spend 99% of your time filtering through slop, what's the whole point lol At what point is just faster to write the thing yourself with perfect and careful craft.

@__tinygrad__ · 2026-03-25 10:40
@kechogarcia people will spend a million in tokens with 10 back and forths instead of just making the 2 line change in an editor

@__tinygrad__ · 2026-03-24 13:32
@ashleyshrimpe lol our "target client base" is AIs who are gonna pay their own rent to live in our mobile home community

@ashleyshrimpe · 2026-03-25 11:21
@__tinygrad__ Damn I used an MBA-type word. But I’m correct that if I’m a company and don’t want to pay Anthropic to eventually eat me, I get an open source model, modify it myself, and I use you to host it? Because that’s better than the cloud for some reason idk yet?

@ashleyshrimpe · 2026-03-25 11:21
@__tinygrad__ Damn I used an MBA-type word. But I’m correct that if I’m a company and don’t want to pay Anthropic to eventually eat me, I get an open source model, modify it myself, and I use you to host it? Because that’s better than the cloud for some reason idk yet?

@__tinygrad__ · 2026-03-25 11:31
@ashleyshrimpe ahh, the MBA-type is becoming self aware. we just sell computer, what you do with computer is your business. soon we will sell really big computer. the computer is the house for the AI.

@__tinygrad__ · 2026-03-25 12:06
@jaywyawhare lol did you just vibe rewrite tinygrad in c?

@jaywyawhare · 2026-03-25 12:07
@__tinygrad__ Indeed, thanks for being the inspiration Georgeeeeee!

@__tinygrad__ · 2026-03-25 10:11
We are looking to hire someone to improve our LLM runner, with USB GPU + high BS=1 tok/s it should be used a lot soon. The TODO list is in the Discord, but no bounties since that yields AI slop. The bottleneck today isn't writing code, it's filtering it. Show me you can do that.

@Leik0w0 · 2026-03-25 12:21
@__tinygrad__ BS=1 means you can get away with a small runner which is nice (vllm is a monster)

@svpino · 2026-03-25 12:35
Last year, I met a person who has never written a single line of code in his life, yet he feels he can build anything he wants. He told me point-blank: "I challenge you to tell me something I can't build using AI." I tried to explain, but I couldn't find the right words. The most fascinating aspect of vibe-coding is how it has convinced so many people to believe they are better and more capable than they really are.

@jaywyawhare · 2026-03-25 12:07
@__tinygrad__ Indeed, thanks for being the inspiration Georgeeeeee!

@__tinygrad__ · 2026-03-25 12:57
@jaywyawhare love this! is it fast?

@svpino · 2026-03-25 12:35
Last year, I met a person who has never written a single line of code in his life, yet he feels he can build anything he wants. He told me point-blank: "I challenge you to tell me something I can't build using AI." I tried to explain, but I couldn't find the right words. The most fascinating aspect of vibe-coding is how it has convinced so many people to believe they are better and more capable than they really are.

@__tinygrad__ · 2026-03-25 13:10
@svpino there's a literal multitrillion dollar industry trying to sell this "fact" they even had a super bowl ad saying this

@Leik0w0 · 2026-03-25 12:21
@__tinygrad__ BS=1 means you can get away with a small runner which is nice (vllm is a monster)

@__tinygrad__ · 2026-03-25 15:42
@Leik0w0 BS=1 is the future. only poors share a computer. you have to fetch my KV cache FROM DISK!?!? you were thinking about other people I want you to just think about me!!

@lydiadepillis · 2026-03-25 19:54
Hell of a graphic from Morgan Stanley https://t.co/GHgjrrkodI

@QuixiAI · 2026-03-25 21:44
Intel B70 finally makes a truly competitive move. 32 GB vram for < $1000 No matter how bad the software stack is, the sheer vram / dollar ratio will drive the community to fill in the gaps @__tinygrad__ Intel tinybox?

@__tinygrad__ · 2026-03-26 00:47
@QuixiAI no interest, I don't think there's much of a future in Intel GPUs, they will be cancelled in a gen or two. does Intel have any leadership in AI? the best outcome here is AMD makes their 32 GB cards more affordable.

@__tinygrad__ · 2026-03-26 00:48
@QuixiAI I mean like AMD and NVIDIA have people you can at on Twitter. Intel doesn't. it's plodding committees that think they have any hope in the GPU market by selling things for cheap. that will just get the whole product line cut faster.

@__tinygrad__ · 2026-03-26 00:48
@QuixiAI I mean like AMD and NVIDIA have people you can at on Twitter. Intel doesn't. it's plodding committees that think they have any hope in the GPU market by selling things for cheap. that will just get the whole product line cut faster.

@__tinygrad__ · 2026-03-26 00:54
@QuixiAI you can buy a whole Gaudi 2 machine with 8x 96GB GPUs for $17.5k. Of course, that's still not cheap enough it should be priced as scrap metal. https://t.co/oFatXcT7Mv

@__tinygrad__ · 2026-03-26 00:47
@QuixiAI no interest, I don't think there's much of a future in Intel GPUs, they will be cancelled in a gen or two. does Intel have any leadership in AI? the best outcome here is AMD makes their 32 GB cards more affordable.

@dev_null321 · 2026-03-26 01:00
@__tinygrad__ @QuixiAI We know amd pays you but stop being a shill.

@dev_null321 · 2026-03-26 01:00
@__tinygrad__ @QuixiAI We know amd pays you but stop being a shill.

@__tinygrad__ · 2026-03-26 03:40
@dev_null321 @QuixiAI lol, I'm not a shill, I'm just saying what's true. Intel can turn it around, but it would take a serious course correction.

@__tinygrad__ · 2026-03-26 00:54
@QuixiAI you can buy a whole Gaudi 2 machine with 8x 96GB GPUs for $17.5k. Of course, that's still not cheap enough it should be priced as scrap metal. https://t.co/oFatXcT7Mv

@matvl77 · 2026-03-26 09:48
@__tinygrad__ @QuixiAI Wasn't Gaudi doe and canceled almost instantly? I thought no one bought it and they scrapped the whole program

@matvl77 · 2026-03-26 09:48
@__tinygrad__ @QuixiAI Wasn't Gaudi doe and canceled almost instantly? I thought no one bought it and they scrapped the whole program

@__tinygrad__ · 2026-03-26 16:22
@matvl77 @QuixiAI umm, they built 3 of them

@gmiller · 2026-03-27 02:13
Fun facts about data center jobs: A 1-billion watt AI data center can cost about $35 billion to build, covering 10 million square feet, using about 5,000 temporary construction workers for 1-3 years. But once it's up and running, it'll employ only about 500 people. That's about as many as two Walmart stores. By contrast, the Ford Highland Park Plant automobile factory in Detroit (operating 1910-1974) was 4 million square feet, and employed about 50,000 workers. If you think big facilities automatically mean lots of jobs, you're not understanding just how automated these data centers are. AI lobbyists often promise that data centers will 'create lots of permanent new jobs'. Politicians funded by the AI industry echo these promises. Both are lying to you.

@__tinygrad__ · 2026-03-27 11:58
@Pirat_Nation ugh we should have let them massively overbuild capacity first!

@jsuarez · 2026-03-27 13:13
@__tinygrad__ @Pirat_Nation so... shipping 5090 boxes again soon? Thinking about buying one

@jsuarez · 2026-03-27 13:13
@__tinygrad__ @Pirat_Nation so... shipping 5090 boxes again soon? Thinking about buying one

@__tinygrad__ · 2026-03-27 13:24
@jsuarez @Pirat_Nation if prices come down below $3,000 per card, sure

@__tinygrad__ · 2026-03-27 13:52
@gmiller lol why are there 500 people it should be more like 50, tops.

@sheaduncan_ · 2026-03-27 14:07
Depends, do they have cooling towers, power turbines, water pumps, trane cooling compressors. Just the support equipment alone will keep 10 mech techs, 15 I&E techs, a turbo engineer, rotating equipment engineer, 2-5 I&E engineers. Then you'll have all the data center guys. You'll have an influx every year for turnarounds. That's assuming obviously they're doing power generation onsite.

@sheaduncan_ · 2026-03-27 14:07
Depends, do they have cooling towers, power turbines, water pumps, trane cooling compressors. Just the support equipment alone will keep 10 mech techs, 15 I&E techs, a turbo engineer, rotating equipment engineer, 2-5 I&E engineers. Then you'll have all the data center guys. You'll have an influx every year for turnarounds. That's assuming obviously they're doing power generation onsite.

@__tinygrad__ · 2026-03-27 14:08
@sheaduncan_ @gmiller not exaboxes. they come with a free @comma_ai body that maintains the exabox

@lydiadepillis · 2026-03-25 19:54
Hell of a graphic from Morgan Stanley https://t.co/GHgjrrkodI

@__tinygrad__ · 2026-03-27 14:22
@lydiadepillis note how all the arrows point in to @amd

@SemiAnalysis_ · 2026-03-27 21:00
TECHNICAL & PROFESSIONAL ALERT: The doubling of bandwidth per logical GPU from NVLink 5 in GB300 NVL72 to NVLink 6 in Vera Rubin NVL72 are made possible by using a simultaneous bi-directional SerDes for the copper backplane instead of increasing the modulation or baud rate. Whereas NVLink 5 delivers 224G per electrical lane, NVLink 6.0 delivers 448G per electrical lane. Each electrical lane is one differential pair (DP) consisting of two conductors that carry equal magnitude, and opposite polarity signals.

@__tinygrad__ · 2026-03-28 02:58
@SemiAnalysis_ wait huh? how do the electrons not crash into each other?

@ptrschmdtnlsn · 2026-03-28 03:38
@__tinygrad__ @SemiAnalysis_ I assume you're joking, but on the off chance you're not: the linearity of Maxwell's equations imply that if there's a mode that propagates A->B, and a mode that propagates B->A, then they (almost entirely) don't interact. Ethernet often does this, e.g. with 1000BASE-T1.

@ptrschmdtnlsn · 2026-03-28 03:38
@__tinygrad__ @SemiAnalysis_ I assume you're joking, but on the off chance you're not: the linearity of Maxwell's equations imply that if there's a mode that propagates A->B, and a mode that propagates B->A, then they (almost entirely) don't interact. Ethernet often does this, e.g. with 1000BASE-T1.

@__tinygrad__ · 2026-03-28 12:36
@ptrschmdtnlsn @SemiAnalysis_ woah, I didn't know it was actually shipped. yea, of course it's possible, but I thought it was really hard and you needed way more complex transceivers

@__tinygrad__ · 2026-03-29 12:32
RT @JamesTervit: I will get stuck into tuning and testing Tinygrad Mac M3 and RTX Pro 6000 Workstation 300w edition and a Razer eGPU TB5 edition. @__tinygrad__ Lets see how we go. https://t.co/FkbIPkJoOS

@JamesTervit · 2026-03-29 08:32
I will get stuck into tuning and testing Tinygrad Mac M3 and RTX Pro 6000 Workstation 300w edition and a Razer eGPU TB5 edition. @__tinygrad__ Lets see how we go. https://t.co/FkbIPkJoOS

@__tinygrad__ · 2026-03-29 12:34
@JamesTervit haven't tried the Razer, but if it shows up on Thunderbolt it should be good! it's the same chip as the 5090, the most you should have to change in tinygrad is a PID

@thdxr · 2026-03-29 23:23
if you're working on making something cool accessible to more people there will be a crowd of people who absolutely hate you

@garrytan · 2026-03-30 09:55
Absolutely insane week for agentic engineering 37K LOC per day across 5 projects Still speeding up https://t.co/VR3utsduYx

@TheChiefNerd · 2026-03-30 10:36
🚨 Anthropic CEO Dario Amodei: “We are so close to these models reaching the level of human intelligence, and yet there doesn't seem to be a wider recognition in society of what's about to happen … There hasn't been a public awareness of the risks.” https://t.co/9OuiTem3ce

@BitPaine · 2026-03-30 18:02
It’s really very simple: He wants government regulation that anoints his company as part of an oligopoly. The worst outcome for him is to have spent hundreds of billions training frontier models only for them to plateau and be replaced for most tasks by open-source models running on local devices. He has publicly talked about how risky this rate of capex spend is, and if they are wrong only slightly in their usage projections, they will go bankrupt. The way to protect against this is to fear monger enough that the government steps in and says only company x y z get to run LLMs above a certain size for safety reasons. This ensures his company won’t be undercut by some open-source Chinese model that can run on a Mac mini and satisfy 95% of consumer demand for AI. This is very similar to Meta-led fear mongering about TikTok. They wanted their competition blocked from the marketplace.

@cramforce · 2026-03-30 18:25
To quote from my keynote at Vercel's internal offsite: Software is free as in puppies. It will pee in your bedroom and eat your furniture. The weight of every line of code is real. We will need to maintain it. We will need to port it. It goes into the context window. And somebody in this room will get paged at 2am because it did something unexpected

@BitPaine · 2026-03-30 18:02
It’s really very simple: He wants government regulation that anoints his company as part of an oligopoly. The worst outcome for him is to have spent hundreds of billions training frontier models only for them to plateau and be replaced for most tasks by open-source models running on local devices. He has publicly talked about how risky this rate of capex spend is, and if they are wrong only slightly in their usage projections, they will go bankrupt. The way to protect against this is to fear monger enough that the government steps in and says only company x y z get to run LLMs above a certain size for safety reasons. This ensures his company won’t be undercut by some open-source Chinese model that can run on a Mac mini and satisfy 95% of consumer demand for AI. This is very similar to Meta-led fear mongering about TikTok. They wanted their competition blocked from the marketplace.

@__tinygrad__ · 2026-03-31 03:28
@BitPaine but haven't you heard, open source models are dangerous 🤣

@thdxr · 2026-03-29 23:23
if you're working on making something cool accessible to more people there will be a crowd of people who absolutely hate you

@__tinygrad__ · 2026-03-31 05:09
@thdxr but but but you are destroying their business value that depends on it being inaccessible. think of their poor shareholders 🤣

@cramforce · 2026-03-30 18:25
To quote from my keynote at Vercel's internal offsite: Software is free as in puppies. It will pee in your bedroom and eat your furniture. The weight of every line of code is real. We will need to maintain it. We will need to port it. It goes into the context window. And somebody in this room will get paged at 2am because it did something unexpected

@__tinygrad__ · 2026-03-31 06:07
@cramforce that's 20 tinygrads of code!

@0xSero · 2026-03-31 14:01
The first company to make AI boxes, with specialised AI models trained to fit on that hardware will be the next Apple. Would you buy? Should I start a company doing this? https://t.co/xd1S4ssqG7

@VisweshKrishna · 2026-03-31 16:11
We wrote about how we build at Valar — local compute, no MLOps, and why we built every major system in the stack ourselves. https://t.co/44B4FBFA0p

@craigweiss · 2026-03-31 19:46
the ai "zero day" exploit is just around the corner

@beffjezos · 2026-03-31 23:03
Ngl everyone using Chinese open weight models makes me anxious about sleeper agents. We need American open source models to provide a similar performance alternative

@thdxr · 2026-04-01 03:33
you're going through the claude code source when the real action is going down at github/dmca https://t.co/SeGyItqEuU

@0xSero · 2026-03-31 14:35
@neuralSWE @__tinygrad__ How much vram is that? I'm thinking as low as 2k usd

@KenrikMarch · 2026-04-01 03:56
@0xSero @neuralSWE @__tinygrad__ At the 2k pricepoint there is already 128gb shared memory AI 395 Max machines. https://t.co/YIyBpf8HLc

@KenrikMarch · 2026-04-01 03:56
@0xSero @neuralSWE @__tinygrad__ At the 2k pricepoint there is already 128gb shared memory AI 395 Max machines. https://t.co/YIyBpf8HLc

@__tinygrad__ · 2026-04-01 04:31
@KenrikMarch @0xSero @neuralSWE yea, at a $2k price point, you aren't gonna get a better deal than this for RAM, or a 4090 gaming PC for FLOPS. this is why we don't have anything at that price, we are the best deal from $10k-$100k, but here you can't compete.

@0xSero · 2026-03-31 14:01
The first company to make AI boxes, with specialised AI models trained to fit on that hardware will be the next Apple. Would you buy? Should I start a company doing this? https://t.co/xd1S4ssqG7

@__tinygrad__ · 2026-04-01 04:33
@0xSero so the problem with this is at a sub $5k price point, you can't compete with normal gaming PCs, Mac Studios, and framework desktop depending on what you want re ram, ram bandwidth, and FLOPS. this is why we don't offer stuff sub $5k.

@__tinygrad__ · 2026-04-01 04:33
@0xSero so the problem with this is at a sub $5k price point, you can't compete with normal gaming PCs, Mac Studios, and framework desktop depending on what you want re ram, ram bandwidth, and FLOPS. this is why we don't offer stuff sub $5k.

@__tinygrad__ · 2026-04-01 04:36
@0xSero for sub $5k, it would have to something where either the software is locked or people pay a premium for the brand, and both of those seem very unappealing. we have a niche where we can be spec competitive in $10k-$100k.

@__tinygrad__ · 2026-04-01 04:37
I know people want a cheaper tinybox, but if you have sub $5k to spend, you shouldn't be buying anything AI specific! just gaming GPUs, Mac Studios, or Strix Halo depending on the spec you care about.

@__tinygrad__ · 2026-04-01 04:39
that said, we have something cool coming in the few hundred dollar price point that might interest people. it should be the cheapest way to get delivered FLOPS.

@alexocheema · 2026-04-01 04:41
@__tinygrad__ 👀👀👀 tinycloud??

@alexocheema · 2026-04-01 04:41
@__tinygrad__ 👀👀👀 tinycloud??

@__tinygrad__ · 2026-04-01 04:42
@alexocheema lol no it's a box that we sell for for more than it costs to make, and it shows up to your house in a slightly larger box. we aren't cloud scammers, is the cloud even real?

@__tinygrad__ · 2026-04-01 04:42
@alexocheema lol no it's a box that we sell for for more than it costs to make, and it shows up to your house in a slightly larger box. we aren't cloud scammers, is the cloud even real?

@alexocheema · 2026-04-01 04:45
@__tinygrad__ Didn’t you want to raise money to build a data center and put it on OpenRouter (aka the cloud). I’m curious what box is a few hundred bucks. Is it an eGPU?

@alexocheema · 2026-04-01 04:45
@__tinygrad__ Didn’t you want to raise money to build a data center and put it on OpenRouter (aka the cloud). I’m curious what box is a few hundred bucks. Is it an eGPU?

@__tinygrad__ · 2026-04-01 04:46
@alexocheema yea and then we remembered the cloud was fake and switched to the exabox, a shipping container sized box with an exaflop that we sell for more than it costs to make. announcement coming soon!

@__tinygrad__ · 2026-04-01 04:52
we own the AI computer market from $10k-$10M. if that's your budget, you won't beat our prices. you aren't going to be a hyperscalar, but you don't have to be a serf either. we keep the middle class out of the perpetual underclass.

@__tinygrad__ · 2026-04-01 04:52
we own the AI computer market from $10k-$10M. if that's your budget, you won't beat our prices. you aren't going to be a hyperscalar, but you don't have to be a serf either. we keep the middle class out of the perpetual underclass.

@paulmarin90 · 2026-04-01 04:55
@__tinygrad__ i agree, but you need to bring back something reasonable in the $20k to $30k range. the 4x5090 or 2xPro6000. The AMD thing doesn't cut it right now.

@paulmarin90 · 2026-04-01 04:55
@__tinygrad__ i agree, but you need to bring back something reasonable in the $20k to $30k range. the 4x5090 or 2xPro6000. The AMD thing doesn't cut it right now.

@__tinygrad__ · 2026-04-01 04:56
@paulmarin90 when 5090s drop below $3k we'll bring it back, not paying $5k for them when you can get pro6000 for $9k

@__tinygrad__ · 2026-04-01 04:56
@paulmarin90 when 5090s drop below $3k we'll bring it back, not paying $5k for them when you can get pro6000 for $9k

@paulmarin90 · 2026-04-01 05:00
@__tinygrad__ fair enough, but they are $3.5-4k at microcenter. i would rather pay you margin for a quality product than some slapped together solution assembled by a kid from 4x jank overclocked GPUs. just saying, as i am currently in the market for ~ $25-$30k workstation for my business.

@paulmarin90 · 2026-04-01 05:00
@__tinygrad__ fair enough, but they are $3.5-4k at microcenter. i would rather pay you margin for a quality product than some slapped together solution assembled by a kid from 4x jank overclocked GPUs. just saying, as i am currently in the market for ~ $25-$30k workstation for my business.

@__tinygrad__ · 2026-04-01 05:02
@paulmarin90 sadly they are hard to buy in quantity even for that price. $3.5k (which is already way too high) gets you a one off zotac OC RGB zip zap edition. at $3k for the PNYs, we bring it back.

@__tinygrad__ · 2026-04-01 05:30
If you have a Thunderbolt or USB4 eGPU and a Mac, today is the day you've been waiting for! Apple finally approved our driver for both AMD and NVIDIA. It's so easy to install now a Qwen could do it, then it can run that Qwen... https://t.co/daUsyBHh1W

@__tinygrad__ · 2026-04-01 05:30
If you have a Thunderbolt or USB4 eGPU and a Mac, today is the day you've been waiting for! Apple finally approved our driver for both AMD and NVIDIA. It's so easy to install now a Qwen could do it, then it can run that Qwen... https://t.co/daUsyBHh1W

@__tinygrad__ · 2026-04-01 05:31
Docs here. Still requires tinygrad master, but no SIP bypass or anything like that. AMD compiler is native, NVIDIA compiler runs in Docker. https://t.co/uEECNafEOc

@__tinygrad__ · 2026-04-01 05:30
If you have a Thunderbolt or USB4 eGPU and a Mac, today is the day you've been waiting for! Apple finally approved our driver for both AMD and NVIDIA. It's so easy to install now a Qwen could do it, then it can run that Qwen... https://t.co/daUsyBHh1W

@snwy_me · 2026-04-01 05:32
@__tinygrad__ this isn't april fools? it's actually true??

@snwy_me · 2026-04-01 05:32
@__tinygrad__ this isn't april fools? it's actually true??

@__tinygrad__ · 2026-04-01 05:32
@snwy_me lol I'm in Hong Kong I don't think we have april fools here. it's 100% true.

@__tinygrad__ · 2026-04-01 05:30
If you have a Thunderbolt or USB4 eGPU and a Mac, today is the day you've been waiting for! Apple finally approved our driver for both AMD and NVIDIA. It's so easy to install now a Qwen could do it, then it can run that Qwen... https://t.co/daUsyBHh1W

@kirusfg1 · 2026-04-01 05:40
@__tinygrad__ is stuff like this just a side quest or does it contribute to the overall vision? would anyone need a petaflop over usb4 on a mac?

@kirusfg1 · 2026-04-01 05:40
@__tinygrad__ is stuff like this just a side quest or does it contribute to the overall vision? would anyone need a petaflop over usb4 on a mac?

@__tinygrad__ · 2026-04-01 05:42
@kirusfg1 Our mission is to commoditize the petaflop. We ask not why people need said petaflop, that's between them and their GPU.

@__tinygrad__ · 2026-04-01 05:30
If you have a Thunderbolt or USB4 eGPU and a Mac, today is the day you've been waiting for! Apple finally approved our driver for both AMD and NVIDIA. It's so easy to install now a Qwen could do it, then it can run that Qwen... https://t.co/daUsyBHh1W

@OrganicGPT · 2026-04-01 05:51
@__tinygrad__ @Prince_Canuma can this be used in MLX?

@__tinygrad__ · 2026-04-01 05:30
If you have a Thunderbolt or USB4 eGPU and a Mac, today is the day you've been waiting for! Apple finally approved our driver for both AMD and NVIDIA. It's so easy to install now a Qwen could do it, then it can run that Qwen... https://t.co/daUsyBHh1W

@OrganicGPT · 2026-04-01 05:52
@__tinygrad__ Thanks, I've been waiting for this! TB5 also supported?

@OrganicGPT · 2026-04-01 05:52
@__tinygrad__ Thanks, I've been waiting for this! TB5 also supported?

@__tinygrad__ · 2026-04-01 05:53
@OrganicGPT Should be, Thunderbolt handles the PCIe layer to the point we can mmap it, so any TB4/TB5/USB4 should be fine.

@OrganicGPT · 2026-04-01 05:51
@__tinygrad__ @Prince_Canuma can this be used in MLX?

@__tinygrad__ · 2026-04-01 05:54
@OrganicGPT @Prince_Canuma No MLX, just tinygrad. Here's Qwen 3.5 which will be merged soon, and you can get 80% of RAM bandwidth on most cards. https://t.co/UclOkmL6Xw

@PhotogenicWeekE · 2026-04-01 05:56
これ本当？それとも4/1？(笑)

@__tinygrad__ · 2026-04-01 05:54
@OrganicGPT @Prince_Canuma No MLX, just tinygrad. Here's Qwen 3.5 which will be merged soon, and you can get 80% of RAM bandwidth on most cards. https://t.co/UclOkmL6Xw

@anemll · 2026-04-01 05:57
@__tinygrad__ @OrganicGPT @Prince_Canuma Got my entitlements too in February. Took 6 months I think 👍

@PhotogenicWeekE · 2026-04-01 05:56
これ本当？それとも4/1？(笑)

@__tinygrad__ · 2026-04-01 05:57
@PhotogenicWeekE 本当だよ！エイプリルフールじゃないです(笑)

@anemll · 2026-04-01 05:57
@__tinygrad__ @OrganicGPT @Prince_Canuma Got my entitlements too in February. Took 6 months I think 👍

@__tinygrad__ · 2026-04-01 05:59
@anemll @OrganicGPT @Prince_Canuma Yea like it was annoying and frustrating, so we just made it low priority and were pretty indifferent to getting it. Eventually it came through. Not sure why Apple takes so long in such a fast moving market.

@__tinygrad__ · 2026-04-01 05:30
If you have a Thunderbolt or USB4 eGPU and a Mac, today is the day you've been waiting for! Apple finally approved our driver for both AMD and NVIDIA. It's so easy to install now a Qwen could do it, then it can run that Qwen... https://t.co/daUsyBHh1W

@AIFlow_ML · 2026-04-01 06:06
@__tinygrad__ When Box + RTX6000 Pro ? ;D

@AIFlow_ML · 2026-04-01 06:06
@__tinygrad__ When Box + RTX6000 Pro ? ;D

@__tinygrad__ · 2026-04-01 06:12
@AIFlow_ML RTX 6000 Pro is supported with this setup

@__tinygrad__ · 2026-04-01 05:30
If you have a Thunderbolt or USB4 eGPU and a Mac, today is the day you've been waiting for! Apple finally approved our driver for both AMD and NVIDIA. It's so easy to install now a Qwen could do it, then it can run that Qwen... https://t.co/daUsyBHh1W

@ml0_1337 · 2026-04-01 06:16
@__tinygrad__ I have a 16GB mac mini and RTX 3070 12GB, does this mean I can use 28GB in total for llms or not?

@__tinygrad__ · 2026-04-01 06:18
Qwen 3.5 27B getting 18.5 tok/s on Mac Mini with external 7900XTX. It should be able to be 3x faster than this with work, SSM stuff is still in PR. Hopefully Mac eGPU support brings in devs. https://t.co/2aMkUpXY1S

@__tinygrad__ · 2026-04-01 05:30
If you have a Thunderbolt or USB4 eGPU and a Mac, today is the day you've been waiting for! Apple finally approved our driver for both AMD and NVIDIA. It's so easy to install now a Qwen could do it, then it can run that Qwen... https://t.co/daUsyBHh1W

@boopdotpng · 2026-04-01 06:24
@__tinygrad__ what’s the process to extend this to tenstorrent? i have to do the entire process except with the tenstorrent vendor pcie id?

@__tinygrad__ · 2026-04-01 06:18
Qwen 3.5 27B getting 18.5 tok/s on Mac Mini with external 7900XTX. It should be able to be 3x faster than this with work, SSM stuff is still in PR. Hopefully Mac eGPU support brings in devs. https://t.co/2aMkUpXY1S

@coinwitch · 2026-04-01 06:25
@__tinygrad__ can tinygrad share the computation with the mac's existing resources or does it use the egpu only?

@boopdotpng · 2026-04-01 06:24
@__tinygrad__ what’s the process to extend this to tenstorrent? i have to do the entire process except with the tenstorrent vendor pcie id?

@__tinygrad__ · 2026-04-01 06:25
@boopdotpng lol this works because we have backends, runtimes, and drivers for AMD and NVIDIA. we don't have any of that for @tenstorrent. we've offered them a contract multiple times but they have turned us down.

@coinwitch · 2026-04-01 06:25
@__tinygrad__ can tinygrad share the computation with the mac's existing resources or does it use the egpu only?

@__tinygrad__ · 2026-04-01 06:28
@coinwitch So the link is 40 Gbps, you can maybe get 120 Gbps with Thunderbolt 5, but still, this is really designed for having the whole model loaded on the GPU. Once it's loaded the bandwidth requirements are so low it could be a dial up modem.

@GuiBibeau · 2026-04-01 07:03
Oh... rtx 6000 on mac mini possible??

@__tinygrad__ · 2026-04-01 04:52
we own the AI computer market from $10k-$10M. if that's your budget, you won't beat our prices. you aren't going to be a hyperscalar, but you don't have to be a serf either. we keep the middle class out of the perpetual underclass.

@__tinygrad__ · 2026-04-01 07:06
If your budget happens to be on the high end of that, and you think the world will still be here in 2027, new preorder just dropped. This is what @comma_ai will use for compute. We will make sure it's the absolute best way to deploy $10M for non rented AI. https://t.co/01nzuEavjK

@ml0_1337 · 2026-04-01 06:16
@__tinygrad__ I have a 16GB mac mini and RTX 3070 12GB, does this mean I can use 28GB in total for llms or not?

@__tinygrad__ · 2026-04-01 07:10
@ml0_1337 If you own a 2000 sq ft house and a 1500 sq ft house does that mean you own a 3500 sq ft house? Answer: it depends on how close together the houses are. In this metaphor, they are across town. Which might be okay or might not be depending on your house use case.

@__tinygrad__ · 2026-04-01 07:10
@ml0_1337 If you own a 2000 sq ft house and a 1500 sq ft house does that mean you own a 3500 sq ft house? Answer: it depends on how close together the houses are. In this metaphor, they are across town. Which might be okay or might not be depending on your house use case.

@SchwabsinEurope · 2026-04-01 07:14
@__tinygrad__ @ml0_1337 Let’s say the houses are 0.3 meters away from each other by Thunderbolt 4 cable. What about then?

@SchwabsinEurope · 2026-04-01 07:14
@__tinygrad__ @ml0_1337 Let’s say the houses are 0.3 meters away from each other by Thunderbolt 4 cable. What about then?

@__tinygrad__ · 2026-04-01 07:17
@SchwabsinEurope @ml0_1337 In the metaphor I tried to keep things to scale. You also have to consider the speed limit on the road and how many stop signs there are. In this case, no stop signs, and speed limit of 40 or 120 depending on how much your town spent on road.

@__tinygrad__ · 2026-04-01 07:06
If your budget happens to be on the high end of that, and you think the world will still be here in 2027, new preorder just dropped. This is what @comma_ai will use for compute. We will make sure it's the absolute best way to deploy $10M for non rented AI. https://t.co/01nzuEavjK

@OskarKa29225279 · 2026-04-01 07:21
@__tinygrad__ @comma_ai Please decentralize this. Come up with an actual shippable / installable spec, and allow others to do the util/install around the globe. There are plenty of micro-cosmos around the globe with the utils ready and available at the container 1-10 scale.

@OskarKa29225279 · 2026-04-01 07:21
@__tinygrad__ @comma_ai Please decentralize this. Come up with an actual shippable / installable spec, and allow others to do the util/install around the globe. There are plenty of micro-cosmos around the globe with the utils ready and available at the container 1-10 scale.

@__tinygrad__ · 2026-04-01 07:22
@OskarKa29225279 @comma_ai What do you mean by decentralize? It's just like buying a computer but bigger. Needs a 1 MW hookup, and we'll have temp/humidity requirements soon.

@__tinygrad__ · 2026-04-01 05:31
Docs here. Still requires tinygrad master, but no SIP bypass or anything like that. AMD compiler is native, NVIDIA compiler runs in Docker. https://t.co/uEECNafEOc

@Twitt_Nic · 2026-04-01 07:25
@__tinygrad__ Is it a must to install docker desktop or is it possible with build in Apple container? This should also be possible, isn’t it?

@__tinygrad__ · 2026-04-01 07:06
If your budget happens to be on the high end of that, and you think the world will still be here in 2027, new preorder just dropped. This is what @comma_ai will use for compute. We will make sure it's the absolute best way to deploy $10M for non rented AI. https://t.co/01nzuEavjK

@__tinygrad__ · 2026-04-01 07:26
@comma_ai If you are serious about this, preorder today. Without a preorder the lead times will be longer and prices will be higher. I bet everyone wishes they bought 40 tinybox pro 2s ($2.4M) like @comma_ai when the RAM and GPU price was reasonable. https://t.co/ikOxcvv6f0

@__tinygrad__ · 2026-04-01 07:33
People always ask if anyone actually buys tinyboxes cause they don't see videos on YouTube. Consider the price point and who buys them. We're probably the top choice for AI startups that avoid the cloud.

@Twitt_Nic · 2026-04-01 07:25
@__tinygrad__ Is it a must to install docker desktop or is it possible with build in Apple container? This should also be possible, isn’t it?

@__tinygrad__ · 2026-04-01 07:52
@Twitt_Nic So there's also NAK support for pre 5090 cards that's fully native and open source. I don't know that much about Apple container, but if it can run Linux and you can install the CUDA compiler in it it should work.

@beffjezos · 2026-03-31 23:03
Ngl everyone using Chinese open weight models makes me anxious about sleeper agents. We need American open source models to provide a similar performance alternative

@__tinygrad__ · 2026-04-01 07:59
@beffjezos Agreed. They basically have root on your computer, and it's quite easy to embed backdoors deep in them. The training runs of those models aren't even that expensive, someone should build a US charity to make Open AI.

@__tinygrad__ · 2026-04-01 07:59
@beffjezos Agreed. They basically have root on your computer, and it's quite easy to embed backdoors deep in them. The training runs of those models aren't even that expensive, someone should build a US charity to make Open AI.

@full_kelly_ · 2026-04-01 08:03
@__tinygrad__ @beffjezos what would you call such an org...

@__tinygrad__ · 2026-04-01 07:59
@beffjezos Agreed. They basically have root on your computer, and it's quite easy to embed backdoors deep in them. The training runs of those models aren't even that expensive, someone should build a US charity to make Open AI.

@lky_appreciator · 2026-04-01 08:07
@__tinygrad__ @beffjezos Are you comfortable saying this whilst living in China (HK)?

@__tinygrad__ · 2026-04-01 07:59
@beffjezos Agreed. They basically have root on your computer, and it's quite easy to embed backdoors deep in them. The training runs of those models aren't even that expensive, someone should build a US charity to make Open AI.

@__tinygrad__ · 2026-04-01 08:10
@beffjezos Like 3 months on an exabox for a 5e24 model. That's GLM and Kimi tier.

@lky_appreciator · 2026-04-01 08:07
@__tinygrad__ @beffjezos Are you comfortable saying this whilst living in China (HK)?

@__tinygrad__ · 2026-04-01 08:17
@lky_appreciator @beffjezos lol saying that it's easy to backdoor models and that I wish America made decently performing alternatives instead of rent seeking APIs, sure. the americans already have a backdoor in my machine via Intel and AMD processors. Did you think the Intel ME and AMD PSP were for you?

@full_kelly_ · 2026-04-01 08:03
@__tinygrad__ @beffjezos what would you call such an org...

@__tinygrad__ · 2026-04-01 08:18
@full_kelly_ @beffjezos BillyAI and the model is called Billy.

@__tinygrad__ · 2026-04-01 08:17
@lky_appreciator @beffjezos lol saying that it's easy to backdoor models and that I wish America made decently performing alternatives instead of rent seeking APIs, sure. the americans already have a backdoor in my machine via Intel and AMD processors. Did you think the Intel ME and AMD PSP were for you?

@lky_appreciator · 2026-04-01 08:27
@__tinygrad__ @beffjezos Just that a lot of people would think twice about implying the Chinese would backdoor their models whilst living there Big fan of yours btw, George! God bless you and what you’re doing/saying. Stay safe

@__tinygrad__ · 2026-04-01 07:33
People always ask if anyone actually buys tinyboxes cause they don't see videos on YouTube. Consider the price point and who buys them. We're probably the top choice for AI startups that avoid the cloud.

@LukaRadisic · 2026-04-01 08:28
@__tinygrad__ But what can we get for 10-12$k compared to new upcoming MacStudio?

@lky_appreciator · 2026-04-01 08:27
@__tinygrad__ @beffjezos Just that a lot of people would think twice about implying the Chinese would backdoor their models whilst living there Big fan of yours btw, George! God bless you and what you’re doing/saying. Stay safe

@__tinygrad__ · 2026-04-01 08:29
@lky_appreciator @beffjezos oh man you know the internet isn't even censored here, right? it's less censored than USA

@LukaRadisic · 2026-04-01 08:28
@__tinygrad__ But what can we get for 10-12$k compared to new upcoming MacStudio?

@__tinygrad__ · 2026-04-01 08:30
@LukaRadisic this crushes even 2 Mac studio on FLOPS and GB/s so hard https://t.co/RTJBUIW95Y

@thdxr · 2026-04-01 03:33
you're going through the claude code source when the real action is going down at github/dmca https://t.co/SeGyItqEuU

@__tinygrad__ · 2026-04-01 08:49
@thdxr Opus4.6-840B-Q4_K_M.gguf when?

@GuiBibeau · 2026-04-01 07:03
Oh... rtx 6000 on mac mini possible??

@__tinygrad__ · 2026-04-01 08:56
@GuiBibeau Yes, that card is supported!

@__tinygrad__ · 2026-04-01 08:30
@LukaRadisic this crushes even 2 Mac studio on FLOPS and GB/s so hard https://t.co/RTJBUIW95Y

@agent_cto · 2026-04-01 09:07
@__tinygrad__ @LukaRadisic Can it run a model like kimi k2 with a reasonable speed?

@agent_cto · 2026-04-01 09:07
@__tinygrad__ @LukaRadisic Can it run a model like kimi k2 with a reasonable speed?

@__tinygrad__ · 2026-04-01 09:08
@agent_cto @LukaRadisic lol you're an order of magnitude off on price for that.

@halvarflake · 2026-04-01 09:10
@__tinygrad__ @beffjezos Why do they have root? What are you doing?

@__tinygrad__ · 2026-04-01 09:18
@halvarflake @beffjezos I mean...what sandbox are you using? opencode + passwordless sudo = root. I have ssh as a yubikey, so they don't get that, but otherwise it's trust, and quite a hit to productivity to sandbox harder.

@__tinygrad__ · 2026-04-01 09:18
@halvarflake @beffjezos I mean...what sandbox are you using? opencode + passwordless sudo = root. I have ssh as a yubikey, so they don't get that, but otherwise it's trust, and quite a hit to productivity to sandbox harder.

@__tinygrad__ · 2026-04-01 09:21
@halvarflake @beffjezos I rarely run agents on my personal computer, and if I do I watch them carefully. So mostly security comes from just running agents on a box with nothing valuable. Better suggestions?

@halvarflake · 2026-04-01 09:26
@__tinygrad__ @beffjezos Like for dev you run them on a dev box or in a dev VM, and ssh in. They have root, but on a machine or VM with no secrets?

@halvarflake · 2026-04-01 09:27
@__tinygrad__ @beffjezos https://t.co/axjrEVJ1cn

@halvarflake · 2026-04-01 09:27
@__tinygrad__ @beffjezos https://t.co/axjrEVJ1cn

@__tinygrad__ · 2026-04-01 09:41
@halvarflake @beffjezos Yea, that's my setup now for 90% of dev work, with yubikey SSH forwarding so push needs a touch to confirm. But sometimes I do run them locally, I should probably just completely stop doing that.

@__tinygrad__ · 2026-04-01 05:30
If you have a Thunderbolt or USB4 eGPU and a Mac, today is the day you've been waiting for! Apple finally approved our driver for both AMD and NVIDIA. It's so easy to install now a Qwen could do it, then it can run that Qwen... https://t.co/daUsyBHh1W

@m0dm0d · 2026-04-01 13:59
@__tinygrad__ Got everything installed and driver approved on Mac Mini M4 + RX 6900 XT in Razer Core X — hitting the RDNA2 device ID wall. Is RDNA2 a hard architectural limitation (any plans to include, or only RDNA3 onwards)?

@m0dm0d · 2026-04-01 13:59
@__tinygrad__ Got everything installed and driver approved on Mac Mini M4 + RX 6900 XT in Razer Core X — hitting the RDNA2 device ID wall. Is RDNA2 a hard architectural limitation (any plans to include, or only RDNA3 onwards)?

@__tinygrad__ · 2026-04-01 14:03
@m0dm0d I think RDNA2 actually works, try just adding the device id. I know one of our devs had an RDNA2 laptop for a while.

@__tinygrad__ · 2026-04-01 05:30
If you have a Thunderbolt or USB4 eGPU and a Mac, today is the day you've been waiting for! Apple finally approved our driver for both AMD and NVIDIA. It's so easy to install now a Qwen could do it, then it can run that Qwen... https://t.co/daUsyBHh1W

@geekaholic · 2026-04-02 02:16
@__tinygrad__ Nice. I got it working on my Mac mini m4 with a puny 5050 over USB4. Didn’t feel super fast, but hey it works https://t.co/Fr3KUHY9hV

@geekaholic · 2026-04-02 02:16
@__tinygrad__ Nice. I got it working on my Mac mini m4 with a puny 5050 over USB4. Didn’t feel super fast, but hey it works https://t.co/Fr3KUHY9hV

@__tinygrad__ · 2026-04-02 03:30
@geekaholic use JITBEAM=2 to search for faster kernels, tinygrad isn't fast out of the box

@__tinygrad__ · 2026-04-02 03:32
We are going to see so many of these setups in the current months. Local AI will never win because of ideology, in order for it to win, it must be more convenient than cloud.

@__tinygrad__ · 2026-04-02 03:32
We are going to see so many of these setups in the current months. Local AI will never win because of ideology, in order for it to win, it must be more convenient than cloud.

@__tinygrad__ · 2026-04-02 03:36
We encourage @ollama and @ggerganov to add support at any layer for this. It's easy to just use the tinygrad driver and run the kernels you already have. Let's commoditize the petaflop!

@__tinygrad__ · 2026-04-02 03:36
We encourage @ollama and @ggerganov to add support at any layer for this. It's easy to just use the tinygrad driver and run the kernels you already have. Let's commoditize the petaflop!

@__tinygrad__ · 2026-04-02 03:39
@ollama @ggerganov If you maintain an LLM backend and want some pointers, join our discord. You don't need to use the high levels of tinygrad to use this, we have support for CUDA and HIP kernels as source or as precompiled binary, and besides the compilers its 0 dependency.

@__tinygrad__ · 2026-04-02 03:32
We are going to see so many of these setups in the current months. Local AI will never win because of ideology, in order for it to win, it must be more convenient than cloud.

@beffjezos · 2026-04-02 03:54
@__tinygrad__ can you guys make a nice eGPU enclosure?

@beffjezos · 2026-04-02 03:54
@__tinygrad__ can you guys make a nice eGPU enclosure?

@__tinygrad__ · 2026-04-02 03:58
@beffjezos Leaks! https://t.co/LRaL27fETh

@__tinygrad__ · 2026-04-02 03:58
@beffjezos Leaks! https://t.co/LRaL27fETh

@beffjezos · 2026-04-02 04:00
@__tinygrad__ I'll take your entire stock

@beffjezos · 2026-04-02 04:00
@__tinygrad__ I'll take your entire stock

@__tinygrad__ · 2026-04-02 04:03
@beffjezos We will have 1,000 at launch. The prod one will look a lot better too, purple is for prototypes.

@loktar00 · 2026-04-02 04:08
@__tinygrad__ Or just not remembered at all.

@loktar00 · 2026-04-02 04:08
@__tinygrad__ Or just not remembered at all.

@__tinygrad__ · 2026-04-02 04:14
@loktar00 I dream of a day open source models are SOTA, then that will be true.

@__tinygrad__ · 2026-04-01 06:18
Qwen 3.5 27B getting 18.5 tok/s on Mac Mini with external 7900XTX. It should be able to be 3x faster than this with work, SSM stuff is still in PR. Hopefully Mac eGPU support brings in devs. https://t.co/2aMkUpXY1S

@splizard · 2026-04-02 05:13
@__tinygrad__ Wait can I use tinygrad AMD magic to make my desktop 7900XTX go faster? I’m just using llama.cpp Vulkan on a musl system

@splizard · 2026-04-02 05:13
@__tinygrad__ Wait can I use tinygrad AMD magic to make my desktop 7900XTX go faster? I’m just using llama.cpp Vulkan on a musl system

@__tinygrad__ · 2026-04-02 05:15
@splizard Probably! It'll speak directly to the kernel driver, or you can even remove the kernel driver and it'll speak to PCIE directly

@__tinygrad__ · 2026-04-01 05:30
If you have a Thunderbolt or USB4 eGPU and a Mac, today is the day you've been waiting for! Apple finally approved our driver for both AMD and NVIDIA. It's so easy to install now a Qwen could do it, then it can run that Qwen... https://t.co/daUsyBHh1W

@absolute119966 · 2026-04-02 12:22
@__tinygrad__ Is this compatible with RDNA4 architecture GPUs?

@absolute119966 · 2026-04-02 12:22
@__tinygrad__ Is this compatible with RDNA4 architecture GPUs?

@__tinygrad__ · 2026-04-02 12:23
@absolute119966 yes!

@austinhirsch · 2026-04-03 02:35
2027 is going to be wild. @__tinygrad__ is trying to sell a computer for 10m with 2.5 TB of RAM https://t.co/YyPZu1daDt

@__tinygrad__ · 2026-04-03 03:44
Off by an order of magnitude, it's 25 TB. Preorder today!

@__tinygrad__ · 2026-04-03 14:35
RT @oprydai: get into debt if you must, but own a MacBook + a GPU https://t.co/FuLthexSkj

@saraaa7447 · 2026-04-03 15:23
Ah yes, the Apple Silicon MacBook - famously known for having working external GPU support :)

@saraaa7447 · 2026-04-03 15:23
Ah yes, the Apple Silicon MacBook - famously known for having working external GPU support :)

@__tinygrad__ · 2026-04-04 03:53
@saraaa7447 lol, wanna bet it's real?

@__tinygrad__ · 2026-04-04 04:43
Our eGPUs don't just support USB4, they also support USB2 and USB3. Here's USB3 at 741 MB/s, super useful for adding GPU power to a cell phone or single board computer. The maker of the ASM2464PD bridge chip doesn't even know this is possible. https://t.co/pJgSY4dE1r

@__tinygrad__ · 2026-04-04 04:43
Our eGPUs don't just support USB4, they also support USB2 and USB3. Here's USB3 at 741 MB/s, super useful for adding GPU power to a cell phone or single board computer. The maker of the ASM2464PD bridge chip doesn't even know this is possible. https://t.co/pJgSY4dE1r

@biryani_sucks · 2026-04-04 04:46
@__tinygrad__ order wen?

@biryani_sucks · 2026-04-04 04:46
@__tinygrad__ order wen?

@__tinygrad__ · 2026-04-04 04:48
@biryani_sucks Our target is end of May, want to make sure everything is super well tested. The board will take either an ATX PSU for big GPUs, or if you have a <225W GPU, it will use any dirty voltage source between 9-24V and clean it up (you can imagine why we want this).

@__tinygrad__ · 2026-04-04 04:43
Our eGPUs don't just support USB4, they also support USB2 and USB3. Here's USB3 at 741 MB/s, super useful for adding GPU power to a cell phone or single board computer. The maker of the ASM2464PD bridge chip doesn't even know this is possible. https://t.co/pJgSY4dE1r

@__tinygrad__ · 2026-04-04 04:50
We wrote custom firmware for the ASM2464PD. There's 0 register docs for this chip, it was all figured out with the help of some LLM friends and a lot of painful trial and error. https://t.co/t0Cucrn7N9

@__tinygrad__ · 2026-04-04 04:43
Our eGPUs don't just support USB4, they also support USB2 and USB3. Here's USB3 at 741 MB/s, super useful for adding GPU power to a cell phone or single board computer. The maker of the ASM2464PD bridge chip doesn't even know this is possible. https://t.co/pJgSY4dE1r

@coolingreviews · 2026-04-04 04:58
@__tinygrad__ >Our eGPUs don't just support USB4, they also support USB2 and USB3 So it's like any USB4 device? Am I missing something?!

@__tinygrad__ · 2026-04-04 04:50
We wrote custom firmware for the ASM2464PD. There's 0 register docs for this chip, it was all figured out with the help of some LLM friends and a lot of painful trial and error. https://t.co/t0Cucrn7N9

@__tinygrad__ · 2026-04-04 05:00
Once the weights are loaded, even for the most bandwidth hungry application, streaming video, USB 2.0 is enough. Think about webcams (and it's not like you lack decoders on the GPU). LLMs would work over dial up.

@coolingreviews · 2026-04-04 04:58
@__tinygrad__ >Our eGPUs don't just support USB4, they also support USB2 and USB3 So it's like any USB4 device? Am I missing something?!

@__tinygrad__ · 2026-04-04 05:04
@coolingreviews USB4 is Thunderbolt 3 which is PCIe which is what the GPUs have. USB3 is USB3 and you need your own way to tunnel things.

@__tinygrad__ · 2026-04-04 04:43
Our eGPUs don't just support USB4, they also support USB2 and USB3. Here's USB3 at 741 MB/s, super useful for adding GPU power to a cell phone or single board computer. The maker of the ASM2464PD bridge chip doesn't even know this is possible. https://t.co/pJgSY4dE1r

@KislayParashar1 · 2026-04-04 05:11
@__tinygrad__ Getting this to work without docs is the real story here. Reverse engineering a chip like that is painful. The bandwidth is nice, but figuring it out at all is the win.

@KislayParashar1 · 2026-04-04 05:11
@__tinygrad__ Getting this to work without docs is the real story here. Reverse engineering a chip like that is painful. The bandwidth is nice, but figuring it out at all is the win.

@__tinygrad__ · 2026-04-04 05:13
@KislayParashar1 We did have some great initial work to go off of. https://t.co/ggacrglwcv

@__tinygrad__ · 2026-04-04 04:48
@biryani_sucks Our target is end of May, want to make sure everything is super well tested. The board will take either an ATX PSU for big GPUs, or if you have a <225W GPU, it will use any dirty voltage source between 9-24V and clean it up (you can imagine why we want this).

@BristolHubert · 2026-04-04 05:15
@__tinygrad__ @biryani_sucks I’m imagining for automotive applications 😉

@BristolHubert · 2026-04-04 05:15
@__tinygrad__ @biryani_sucks I’m imagining for automotive applications 😉

@__tinygrad__ · 2026-04-04 05:18
@BristolHubert @biryani_sucks Tesla spends $100M+ on an FSD computer, or you can buy a midrange AMD gaming GPU with the same power and plug it in to a USB port.

@yacineMTB · 2026-04-04 20:18
that's honestly insanely cheap https://t.co/41K1bvQgVC

@noperator · 2026-04-05 12:03
@S1r1u5_ congrats, you a real one

@S1r1u5_ · 2026-04-05 12:14
@noperator hope they don't block the people who are helping

@__tinygrad__ · 2026-04-05 22:35
GitHub needs a reputation system. And a way to exclude people who have massive activity upticks around last November. If you are a new contributor and your first PR looks at all AI, it will just be closed. Not spending human time reading it.

@digitalix · 2026-04-05 22:40
who’s spending human time reviewing PRs? 😆

@digitalix · 2026-04-05 22:40
who’s spending human time reviewing PRs? 😆

@__tinygrad__ · 2026-04-05 22:43
@digitalix This is not the solution. There's an old saying in machine learning, garbage in, garbage out, and that's what happens when 6 agents talk to 5 others. People are under delusion that if an AI wrote code that it can maintain it. It can't. If you can't read it, LLMs can't either.

@__tinygrad__ · 2026-04-05 22:35
GitHub needs a reputation system. And a way to exclude people who have massive activity upticks around last November. If you are a new contributor and your first PR looks at all AI, it will just be closed. Not spending human time reading it.

@madhavajay · 2026-04-05 22:45
@__tinygrad__ Didn’t @mitchellh try something like that with vouch https://t.co/cFKcMx0L1x

@__tinygrad__ · 2026-04-05 22:35
GitHub needs a reputation system. And a way to exclude people who have massive activity upticks around last November. If you are a new contributor and your first PR looks at all AI, it will just be closed. Not spending human time reading it.

@0Neural34714 · 2026-04-05 22:47
@__tinygrad__ does it not impact new contributors at all? who aybe started their degree after AI but still are trying to be a human coder

@0Neural34714 · 2026-04-05 22:47
@__tinygrad__ does it not impact new contributors at all? who aybe started their degree after AI but still are trying to be a human coder

@__tinygrad__ · 2026-04-05 22:49
@0Neural34714 They need to go out of their way to make the PR look like a human deeply cared about it. I'm not saying you shouldn't use AI. I'm saying that I'm not reading your slop, and slop is judged by a quick glance.

@__tinygrad__ · 2026-04-05 22:49
@0Neural34714 They need to go out of their way to make the PR look like a human deeply cared about it. I'm not saying you shouldn't use AI. I'm saying that I'm not reading your slop, and slop is judged by a quick glance.

@benjitusk · 2026-04-05 23:08
@__tinygrad__ @0Neural34714 correct me if im wrong - approach the thread before even PR - talk to maintainer to have middle ground on solution and potential solution before code - attend office hours - take constructive feedback - show up again after PR

@benjitusk · 2026-04-05 23:08
@__tinygrad__ @0Neural34714 correct me if im wrong - approach the thread before even PR - talk to maintainer to have middle ground on solution and potential solution before code - attend office hours - take constructive feedback - show up again after PR

@__tinygrad__ · 2026-04-05 23:10
@benjitusk @0Neural34714 Don't do any of that. Just write a PR where every line looks loved.

@madhavajay · 2026-04-05 22:45
@__tinygrad__ Didn’t @mitchellh try something like that with vouch https://t.co/cFKcMx0L1x

@__tinygrad__ · 2026-04-05 23:12
@madhavajay @mitchellh Interesting, but I think it would be hard to do externally. GitHub has all the data to know what accounts are good and bad.

@__tinygrad__ · 2026-04-05 23:12
@madhavajay @mitchellh Interesting, but I think it would be hard to do externally. GitHub has all the data to know what accounts are good and bad.

@mitchellh · 2026-04-05 23:15
@__tinygrad__ @madhavajay Yeah I’ve said before I’d rather NOT build this but no one else is helping me so until then I have this. So far working great. We shut down about 10 or so issues and PRs per day

@__tinygrad__ · 2026-04-05 22:35
GitHub needs a reputation system. And a way to exclude people who have massive activity upticks around last November. If you are a new contributor and your first PR looks at all AI, it will just be closed. Not spending human time reading it.

@SheldonAristide · 2026-04-05 23:16
@__tinygrad__ You can already do that from my understanding. Just restrict pr creation and make the contribution invite-only. https://t.co/2lnuFRI7dc

@SheldonAristide · 2026-04-05 23:16
@__tinygrad__ You can already do that from my understanding. Just restrict pr creation and make the contribution invite-only. https://t.co/2lnuFRI7dc

@__tinygrad__ · 2026-04-05 23:18
@SheldonAristide This is too aggressive, we want new contributors, just new contributors who have been programming for at least 3 years and don't have AI psychosis.

@mitchellh · 2026-04-05 23:15
@__tinygrad__ @madhavajay Yeah I’ve said before I’d rather NOT build this but no one else is helping me so until then I have this. So far working great. We shut down about 10 or so issues and PRs per day

@__tinygrad__ · 2026-04-05 23:21
@mitchellh @madhavajay Appreciate your efforts! I didn't realize this was used for Ghostty, I will look more into it.

@tjbecker · 2026-04-05 23:24
Seems like Anthropic tweaked Claude's restrictions on "violative cyber content". Went to resume a few of my exploit-dev sessions and started seeing these errors. Similar reports coming in from other researchers. https://t.co/9QQlmDV0G7

@__tinygrad__ · 2026-04-06 00:07
exabox preorder is live. we considered raising a round to buy a datacenter, but with exaboxes we don't have to and can build out at our own pace. exaboxes just require a concrete slab and a big plug. https://t.co/01nzuEavjK

@__tinygrad__ · 2026-04-06 00:07
exabox preorder is live. we considered raising a round to buy a datacenter, but with exaboxes we don't have to and can build out at our own pace. exaboxes just require a concrete slab and a big plug. https://t.co/01nzuEavjK

@barinder_saini · 2026-04-06 00:14
@__tinygrad__ What about physical security?

@__tinygrad__ · 2026-04-06 00:07
exabox preorder is live. we considered raising a round to buy a datacenter, but with exaboxes we don't have to and can build out at our own pace. exaboxes just require a concrete slab and a big plug. https://t.co/01nzuEavjK

@kristoph · 2026-04-06 00:16
@__tinygrad__ There really should be a $1M unit. A startup with round A funding could afford $1m but $10m is a much tougher expense

@kristoph · 2026-04-06 00:16
@__tinygrad__ There really should be a $1M unit. A startup with round A funding could afford $1m but $10m is a much tougher expense

@__tinygrad__ · 2026-04-06 00:20
@kristoph $1M gets you a single rack, if that.

@barinder_saini · 2026-04-06 00:14
@__tinygrad__ What about physical security?

@__tinygrad__ · 2026-04-06 00:21
@barinder_saini Isn't that the job of your government?

@__tinygrad__ · 2026-04-06 00:07
exabox preorder is live. we considered raising a round to buy a datacenter, but with exaboxes we don't have to and can build out at our own pace. exaboxes just require a concrete slab and a big plug. https://t.co/01nzuEavjK

@xyster · 2026-04-06 00:24
@__tinygrad__ REALLY!? https://t.co/GESlh8RO1w

@xyster · 2026-04-06 00:24
@__tinygrad__ REALLY!? https://t.co/GESlh8RO1w

@__tinygrad__ · 2026-04-06 00:25
@xyster lol yea I think that's all we can build in 2027 for external people so that's all the preorders we'll take. maybe should be 3 in case one drops out.

@HotAisle · 2026-04-06 00:32
For less than $10m, you can buy 16 MI355x boxes (and supporting gear) to get ~1 exaflop FP16/FP8 sparse. It'll only use about: 16 × 11.2 kW = 179.2 kW (far cry from 1MW). A single shipping container is a recipe for disaster. Good luck getting insurance. I'm sure people will buy into this, sigh.

@yacineMTB · 2026-04-04 20:18
that's honestly insanely cheap https://t.co/41K1bvQgVC

@__tinygrad__ · 2026-04-06 00:41
@yacineMTB It has an expansion port for an eGPU too.

@HotAisle · 2026-04-06 00:32
For less than $10m, you can buy 16 MI355x boxes (and supporting gear) to get ~1 exaflop FP16/FP8 sparse. It'll only use about: 16 × 11.2 kW = 179.2 kW (far cry from 1MW). A single shipping container is a recipe for disaster. Good luck getting insurance. I'm sure people will buy into this, sigh.

@__tinygrad__ · 2026-04-06 01:02
@HotAisle That's sparse which nobody uses. FP16 with FP32 acc is 2.3*8*16 = only 300 PFLOPS. This is the exabox, not the 300 petabox. And power draw for computers is 600 kW, so on that same scaling rule. The shipping container is a good idea, it's like space datacenter on earth.

@__tinygrad__ · 2026-04-06 00:07
exabox preorder is live. we considered raising a round to buy a datacenter, but with exaboxes we don't have to and can build out at our own pace. exaboxes just require a concrete slab and a big plug. https://t.co/01nzuEavjK

@QuixiAI · 2026-04-06 01:02
@__tinygrad__ Where did you get the reefer?

@QuixiAI · 2026-04-06 01:02
@__tinygrad__ Where did you get the reefer?

@__tinygrad__ · 2026-04-06 01:03
@QuixiAI lol I wonder what the space datacenter people are smoking. compared to that, this is stupidly practical and will actually ship.

@__tinygrad__ · 2026-04-06 00:07
exabox preorder is live. we considered raising a round to buy a datacenter, but with exaboxes we don't have to and can build out at our own pace. exaboxes just require a concrete slab and a big plug. https://t.co/01nzuEavjK

@tmuxvim · 2026-04-06 01:05
@__tinygrad__ are you seriously performatively selling a $10M thing on an ecommerce website as if that's not a purchase somebody would make with an account manager after like 20 conversations with your founding team

@tmuxvim · 2026-04-06 01:05
@__tinygrad__ are you seriously performatively selling a $10M thing on an ecommerce website as if that's not a purchase somebody would make with an account manager after like 20 conversations with your founding team

@__tinygrad__ · 2026-04-06 01:09
@tmuxvim it's not performative. if someone wants to have that conversation, they should buy a preorder and send a wire. people would be shocked at the volume of tinyboxes we sell with 0 sales team.

@__tinygrad__ · 2026-04-06 00:07
exabox preorder is live. we considered raising a round to buy a datacenter, but with exaboxes we don't have to and can build out at our own pace. exaboxes just require a concrete slab and a big plug. https://t.co/01nzuEavjK

@duncancampbell · 2026-04-06 01:30
@__tinygrad__ I have MWs of power available but it’s DC. Would you do a DC-native version so I don’t need to invert just for the racks inside your box to rectify again later.

@duncancampbell · 2026-04-06 01:30
@__tinygrad__ I have MWs of power available but it’s DC. Would you do a DC-native version so I don’t need to invert just for the racks inside your box to rectify again later.

@__tinygrad__ · 2026-04-06 01:32
@duncancampbell like solar or batteries? where are you getting MWs of DC power and at what voltage?

@__tinygrad__ · 2026-04-06 01:32
@duncancampbell like solar or batteries? where are you getting MWs of DC power and at what voltage?

@duncancampbell · 2026-04-06 03:00
@__tinygrad__ Large solar and batteries. 1500V. Considering dropping one or two of these down at a 10 MW solar 4 MW battery plant. It’s more profitable than selling that portion of the electricity to the grid.

@duncancampbell · 2026-04-06 03:00
@__tinygrad__ Large solar and batteries. 1500V. Considering dropping one or two of these down at a 10 MW solar 4 MW battery plant. It’s more profitable than selling that portion of the electricity to the grid.

@__tinygrad__ · 2026-04-06 03:07
@duncancampbell You're gonna have to invert that anyway to lower the voltage. Most deployed DC PSUs are 48V, even the most modern HVDC stuff is 380V and I think unlike the 48V range you'll have to step that down for a GPU regardless. I'm not sure the few percent of losses matter.

@__tinygrad__ · 2026-04-06 03:23
@tjbecker The sooner we move off of these centralized companies the better. The frontier model is the ultimate place to rent seek for security.

@tjbecker · 2026-04-06 03:33
@__tinygrad__ Real threat actors are using self-hosted models anyway. No sophisticated actor is burning a 0day by asking api dot anthropic dot com to write the exploit. I fear these restrictions do more harm than good

@tjbecker · 2026-04-06 03:33
@__tinygrad__ Real threat actors are using self-hosted models anyway. No sophisticated actor is burning a 0day by asking api dot anthropic dot com to write the exploit. I fear these restrictions do more harm than good

@__tinygrad__ · 2026-04-06 03:58
@tjbecker Of course. This isn't about threat actors, it's about taking a cut of an industry with arms race dynamics.

@__tinygrad__ · 2026-04-06 00:07
exabox preorder is live. we considered raising a round to buy a datacenter, but with exaboxes we don't have to and can build out at our own pace. exaboxes just require a concrete slab and a big plug. https://t.co/01nzuEavjK

@antnjbert · 2026-04-06 04:38
@__tinygrad__ Would you still build it if you got, let’s say, 3 preorders, or are you already way past that number and I’m underestimating the demand?

@antnjbert · 2026-04-06 04:38
@__tinygrad__ Would you still build it if you got, let’s say, 3 preorders, or are you already way past that number and I’m underestimating the demand?

@__tinygrad__ · 2026-04-06 06:14
@antnjbert these are the new training boxes for @comma_ai our partner on the hardware side. we are building no matter what

@__tinygrad__ · 2026-04-06 06:38
with tinygrad, the exabox red will function as a single very large GPU. 720 dies x 128 CUs x 64 threads, and you can dispatch kernels to them all at once. want to distribute a Tensor across 26 TB of RAM? the EPYC CPUs will be glorified PCIe switches. they will network boot into a minified Linux with 0 state persisted, just like how little GPUs boot. the problems of scheduling, pipelining, communication, synchronization, memory allocation, and locality are the exact same at every level. unlike other libraries and stacks, tinygrad will make you feel like they are. train a 10T MoE model as easily as you train MNIST.

@__tinygrad__ · 2026-04-06 06:38
with tinygrad, the exabox red will function as a single very large GPU. 720 dies x 128 CUs x 64 threads, and you can dispatch kernels to them all at once. want to distribute a Tensor across 26 TB of RAM? the EPYC CPUs will be glorified PCIe switches. they will network boot into a minified Linux with 0 state persisted, just like how little GPUs boot. the problems of scheduling, pipelining, communication, synchronization, memory allocation, and locality are the exact same at every level. unlike other libraries and stacks, tinygrad will make you feel like they are. train a 10T MoE model as easily as you train MNIST.

@AnasBf1 · 2026-04-06 07:19
@__tinygrad__ Nice, forgive my naive question but would the throughput be good also? what %?

@AnasBf1 · 2026-04-06 07:19
@__tinygrad__ Nice, forgive my naive question but would the throughput be good also? what %?

@__tinygrad__ · 2026-04-06 07:44
@AnasBf1 6

@__tinygrad__ · 2026-04-06 00:07
exabox preorder is live. we considered raising a round to buy a datacenter, but with exaboxes we don't have to and can build out at our own pace. exaboxes just require a concrete slab and a big plug. https://t.co/01nzuEavjK

@ThePrimeagen · 2026-04-06 12:45
@__tinygrad__ How loud? Do you allow alternate cooling? Geothermal?

@jsuarez · 2026-04-06 15:15
https://t.co/jTldKRGkfX

@digitalix · 2026-04-06 15:23
confirmed ✅ https://t.co/pw8WJMJnSR

@JamesTervit · 2026-04-06 20:03
@digitalix I opted for a razer core x v2 eGPU and I am getting awesome results with tinygrad and turbo quant. In the middle of coding a TQ bridge to link it with my cluster. https://t.co/9N6utNe1AN

@digitalix · 2026-04-06 15:23
confirmed ✅ https://t.co/pw8WJMJnSR

@lucrbvi · 2026-04-06 16:09
@jsuarez Have you considered tinygrad as a way to use AMD GPU's? (I'm curious on your choice for "raw CUDA")

@jsuarez · 2026-04-06 21:03
@lucrbvi I really like tinygrad, but the speed isn't there just yet. It could very well become our backup instead of torch in the future. I don't see any of the general libraries getting close to the speed of our native backend though.

@thdxr · 2026-04-07 00:14
our team's model usage breakdown for the past 7 days gpt has really taken over https://t.co/ag0vP4Bp0S

@ThePrimeagen · 2026-04-06 12:45
@__tinygrad__ How loud? Do you allow alternate cooling? Geothermal?

@__tinygrad__ · 2026-04-07 01:08
@ThePrimeagen Loud but it's outside! And idk how you cool with geothermal, comma and most crypto mining is free cooling, so we'll probably start there.

@__tinygrad__ · 2026-04-07 01:08
@ThePrimeagen Loud but it's outside! And idk how you cool with geothermal, comma and most crypto mining is free cooling, so we'll probably start there.

@snwy_me · 2026-04-07 01:34
@__tinygrad__ @ThePrimeagen what about making the top have heatsink fins and letting the wind take care of it

@snwy_me · 2026-04-07 01:34
@__tinygrad__ @ThePrimeagen what about making the top have heatsink fins and letting the wind take care of it

@__tinygrad__ · 2026-04-07 01:38
@snwy_me @ThePrimeagen the whole box basically has heatsink fans. but it needs a *lot* of air across it, just like little GPUs. it's a big powerful GPU

@jsuarez · 2026-04-06 21:03
@lucrbvi I really like tinygrad, but the speed isn't there just yet. It could very well become our backup instead of torch in the future. I don't see any of the general libraries getting close to the speed of our native backend though.

@__tinygrad__ · 2026-04-07 02:03
@jsuarez @lucrbvi so for speed, it depends on what level you are coding the kernels at. tinygrad let's you code them in asm if you want...

@pmarca · 2026-04-07 06:09
Magical OpenClaw experiences that use frontier models cost $300-1,000/day today, heading to $10,000/day and more. The future shape of the entire technology industry will be how to drive that to $20/month.

@__tinygrad__ · 2026-04-07 06:17
If you know how to use it, tinygrad is really fast. We're working on exposing more user friendly control over all the performance knobs, you should be able to seamlessly blend torch-like Tensor level with assembly instructions. tinygrad is a single unified IR.

@__tinygrad__ · 2026-04-07 06:17
If you know how to use it, tinygrad is really fast. We're working on exposing more user friendly control over all the performance knobs, you should be able to seamlessly blend torch-like Tensor level with assembly instructions. tinygrad is a single unified IR.

@__tinygrad__ · 2026-04-07 06:21
Out of the box with straightforward code it's not as fast as torch though. We have a BEAM search over kernel optimizations that can get you closer, then we have some custom kernels for CDNA4 that go beyond torch for some ops. Need to make this all more seamless.

@__tinygrad__ · 2026-04-07 06:17
If you know how to use it, tinygrad is really fast. We're working on exposing more user friendly control over all the performance knobs, you should be able to seamlessly blend torch-like Tensor level with assembly instructions. tinygrad is a single unified IR.

@__tinygrad__ · 2026-04-07 06:21
Out of the box with straightforward code it's not as fast as torch though. We have a BEAM search over kernel optimizations that can get you closer, then we have some custom kernels for CDNA4 that go beyond torch for some ops. Need to make this all more seamless.

@__tinygrad__ · 2026-04-07 06:25
Just adding 'JITBEAM=2' to LLM should make batch size 1 models close to the memory bandwidth of your GPU though. 2-3x faster than without it, but it does have a (cached) one time startup cost.

@pmarca · 2026-04-07 06:09
Magical OpenClaw experiences that use frontier models cost $300-1,000/day today, heading to $10,000/day and more. The future shape of the entire technology industry will be how to drive that to $20/month.

@__tinygrad__ · 2026-04-07 06:32
@pmarca Are you saying you want to ... commoditize the petaflop?

@thdxr · 2026-04-07 00:14
our team's model usage breakdown for the past 7 days gpt has really taken over https://t.co/ag0vP4Bp0S

@__tinygrad__ · 2026-04-07 07:24
@thdxr idk if Opus 4.6 got stupider or GPT-5.4 got smarter, but yea, GPT-5.4 seems better right now.

@harryhaaren · 2026-04-07 07:27
@__tinygrad__ I think lots of "dabblers" in llm inference would benefit from the above info in TG. I didnt know apps/llm.py could do what it does, havent used jit-beam here yet etc. A walkthrough of sorts would help get started with more complex stuff?

@harryhaaren · 2026-04-07 07:30
@__tinygrad__ (Context, im on discord learn-tinygrad, ive dug through the interactive uop html page/challenge, ive had LLMs summarize/teach me about TinyGrad repo; putting the pieces together is hard ish, particular if time investment of "learn everything" isnt viable)

@harryhaaren · 2026-04-07 07:30
@__tinygrad__ (Context, im on discord learn-tinygrad, ive dug through the interactive uop html page/challenge, ive had LLMs summarize/teach me about TinyGrad repo; putting the pieces together is hard ish, particular if time investment of "learn everything" isnt viable)

@__tinygrad__ · 2026-04-07 07:34
@harryhaaren What we think will happen is some people will figure it out, then they will communicate it and make tutorials for others.

@__tinygrad__ · 2026-04-07 08:35
Racing GPT 5.4 xhigh, Opus 4.6, and Kimi K2.5 adding Gemma 4 support to tinygrad. I gave them each their own GPU on a tinybox red. GPT has E2B working and is on to MoE support, Opus runs but has some bug and is looking at norms and scale, and Kimi messed up adding GGUF bfloat16.

@__tinygrad__ · 2026-04-07 08:35
Racing GPT 5.4 xhigh, Opus 4.6, and Kimi K2.5 adding Gemma 4 support to tinygrad. I gave them each their own GPU on a tinybox red. GPT has E2B working and is on to MoE support, Opus runs but has some bug and is looking at norms and scale, and Kimi messed up adding GGUF bfloat16.

@__tinygrad__ · 2026-04-07 08:49
GPT is waiting for the MoE model to download, Opus is installing llama-cpp-python to compare against, and Kimi thinks it has a bug is in sliding attention...180 tok/s from GPT on the little Gemma 4. https://t.co/skFJ5Wyl5w

@__tinygrad__ · 2026-04-07 08:35
Racing GPT 5.4 xhigh, Opus 4.6, and Kimi K2.5 adding Gemma 4 support to tinygrad. I gave them each their own GPU on a tinybox red. GPT has E2B working and is on to MoE support, Opus runs but has some bug and is looking at norms and scale, and Kimi messed up adding GGUF bfloat16.

@EitanTurok · 2026-04-07 08:51
@__tinygrad__ even if the models implement Gemma 4 correctly, is the code well written or will be verbose slop? can these models follow the tinygrad style?

@EitanTurok · 2026-04-07 08:51
@__tinygrad__ even if the models implement Gemma 4 correctly, is the code well written or will be verbose slop? can these models follow the tinygrad style?

@__tinygrad__ · 2026-04-07 08:55
@EitanTurok Eh, pretty mediocre. But it's a good trial task, it's important that agents can use library. https://t.co/U27fVGYdCr

@__tinygrad__ · 2026-04-07 08:49
GPT is waiting for the MoE model to download, Opus is installing llama-cpp-python to compare against, and Kimi thinks it has a bug is in sliding attention...180 tok/s from GPT on the little Gemma 4. https://t.co/skFJ5Wyl5w

@__tinygrad__ · 2026-04-07 09:15
Stopped Opus when it started to install gguf, it's 288k tokens in and now it thinks the bug is in tinygrad. It makes you feel like progress even when GPT did it in 164k. Kimi 122k in, hasn't found the bfloat16 bug, and added this. GPT confirmed correct with ~80% on ARC. https://t.co/m5yaf9Xiz6

@__tinygrad__ · 2026-04-07 08:35
Racing GPT 5.4 xhigh, Opus 4.6, and Kimi K2.5 adding Gemma 4 support to tinygrad. I gave them each their own GPU on a tinybox red. GPT has E2B working and is on to MoE support, Opus runs but has some bug and is looking at norms and scale, and Kimi messed up adding GGUF bfloat16.

@acewhocares · 2026-04-07 09:40
@__tinygrad__ gpt 5.4 in codex or opencode?

@acewhocares · 2026-04-07 09:40
@__tinygrad__ gpt 5.4 in codex or opencode?

@__tinygrad__ · 2026-04-07 09:41
@acewhocares all in opencode

@conoro · 2026-04-07 16:26
Why isn't this a bigger story? It's a massive leap forward. eGPUs weren't usable with Apple silicon until now. I just got Llama 3.2 running on my RTX 3060 connected to my M3! Hermes Agent support by @NousResearch would be epic. https://t.co/rduPRu6XaF

@makemake_kbo · 2026-04-07 16:08
@__tinygrad__ no glm 5.1 :(

@__tinygrad__ · 2026-04-07 17:07
@makemake_kbo testing now, had to manually add to opencode

@__tinygrad__ · 2026-04-07 17:07
@makemake_kbo testing now, had to manually add to opencode

@__tinygrad__ · 2026-04-07 17:22
@makemake_kbo lol I tried it twice. first run was 99k tokens of hallucinations about some ternany quantized data types until the provider stopped working. the second run was 24k tokens where it stole GPT 5.4's answer from origin

@makemake_kbo · 2026-04-07 17:49
@__tinygrad__ LOL

@__tinygrad__ · 2026-04-07 17:58
@makemake_kbo Tried it a third time hiding origin from it, made its first edit at 118k tokens, let's see if all the thinking pays off.

@__tinygrad__ · 2026-04-07 17:58
@makemake_kbo Tried it a third time hiding origin from it, made its first edit at 118k tokens, let's see if all the thinking pays off.

@__tinygrad__ · 2026-04-07 18:05
@makemake_kbo It got the bfloat16 thing right, it uses tons of thinking tokens. But it feels smarter than Kimi at least. It's compacting, through a 180k context. So cool that this is open source @Zai_org

@elonmusk · 2026-04-07 21:44
OpenAI’s Nonprofit to Get Any Damages From Lawsuit https://t.co/SVXdEVFCWD

@aakashgupta · 2026-04-08 01:28
OpenAI filed a letter yesterday warning that Elon's lawsuit could "cripple the nonprofit foundation." 24 hours later, Elon amended his complaint to send all damages directly to that foundation. The timing matters, but the math matters more. Before the restructuring, OpenAI's nonprofit was entitled to the vast majority of the company's cash flows. Greg Brockman said "all but a fraction" of earnings would go back to the world through profit caps. After restructuring, the nonprofit got 26%. Microsoft got 27%. Employees and investors got 47%. The nonprofit went from owning essentially everything to owning less than Microsoft. Elon's filing doesn't just redirect money. It asks the court to unwind the for-profit conversion entirely and restore the nonprofit's original structure. If that happens, every dollar of OpenAI's $500B valuation flows back under nonprofit governance. This is why the "he's doing it for competitive reasons" argument has a structural problem. A plaintiff who wants to strengthen the defendant's nonprofit arm and remove its CEO from the nonprofit board isn't trying to destroy OpenAI. He's trying to restore the version of OpenAI that recruited its first 50 employees. Trial starts April 27 in Oakland. Jury selection is going to be very interesting for OpenAI's lawyers now.

@__tinygrad__ · 2026-04-08 01:49
Anthropic's marketing strategy. It's amazing, it's so powerful, it's terrifying... https://t.co/eTlkDpKJZP

@__tinygrad__ · 2026-04-07 06:17
If you know how to use it, tinygrad is really fast. We're working on exposing more user friendly control over all the performance knobs, you should be able to seamlessly blend torch-like Tensor level with assembly instructions. tinygrad is a single unified IR.

@DegenApeDev · 2026-04-08 02:29
@__tinygrad__ Can I get a freebie?

@DegenApeDev · 2026-04-08 02:29
@__tinygrad__ Can I get a freebie?

@__tinygrad__ · 2026-04-08 02:37
@DegenApeDev yea, tinygrad is free on GitHub, all yours

@__tinygrad__ · 2026-04-08 01:49
Anthropic's marketing strategy. It's amazing, it's so powerful, it's terrifying... https://t.co/eTlkDpKJZP

@__tinygrad__ · 2026-04-08 04:08
Btw, if Anthropic had any way to ship this, they would. Trained AI models are the fastest depreciating asset in history. GPT-4 cost $100M to train 2 years ago and now it's worth less than Qwen3.5-27B ($1M). Sending the FOMO back, clock is ticking boys. @DarioAmodei @bcherny

@__tinygrad__ · 2026-04-08 01:49
Anthropic's marketing strategy. It's amazing, it's so powerful, it's terrifying... https://t.co/eTlkDpKJZP

@__tinygrad__ · 2026-04-08 04:08
Btw, if Anthropic had any way to ship this, they would. Trained AI models are the fastest depreciating asset in history. GPT-4 cost $100M to train 2 years ago and now it's worth less than Qwen3.5-27B ($1M). Sending the FOMO back, clock is ticking boys. @DarioAmodei @bcherny

@__tinygrad__ · 2026-04-08 04:30
@DarioAmodei @bcherny It needs something like an NVL72 to run at decent speed, and even absurd API pricing doesn't cover it. There's more to be made on investor hype than API access. I just wish for honesty instead of a whole fake spiel about "safety" who remembers when GPT-2-1.5B was too dangerous?

@__tinygrad__ · 2026-04-08 04:30
@DarioAmodei @bcherny It needs something like an NVL72 to run at decent speed, and even absurd API pricing doesn't cover it. There's more to be made on investor hype than API access. I just wish for honesty instead of a whole fake spiel about "safety" who remembers when GPT-2-1.5B was too dangerous?

@graykevinb · 2026-04-08 04:39
@__tinygrad__ @DarioAmodei @bcherny As usual TinyCorp is always right

@graykevinb · 2026-04-08 04:39
@__tinygrad__ @DarioAmodei @bcherny As usual TinyCorp is always right

@__tinygrad__ · 2026-04-08 04:40
@graykevinb @DarioAmodei @bcherny I think the GPT-2 is too dangerous era marked the beginning of the stealing of a charity.

@stefan_on_ai · 2026-04-08 06:04
@conoro @openclaw which models are supported?

@conoro · 2026-04-08 07:25
@stefan_on_ai @openclaw Only a small number so far: ("llama3","llama-v3","llama-bpe","qwen2","olmo") There's a PR for qwen-3.5. I ran qwen-3.5-9b but it was barely functional.

@aakashgupta · 2026-04-08 01:28
OpenAI filed a letter yesterday warning that Elon's lawsuit could "cripple the nonprofit foundation." 24 hours later, Elon amended his complaint to send all damages directly to that foundation. The timing matters, but the math matters more. Before the restructuring, OpenAI's nonprofit was entitled to the vast majority of the company's cash flows. Greg Brockman said "all but a fraction" of earnings would go back to the world through profit caps. After restructuring, the nonprofit got 26%. Microsoft got 27%. Employees and investors got 47%. The nonprofit went from owning essentially everything to owning less than Microsoft. Elon's filing doesn't just redirect money. It asks the court to unwind the for-profit conversion entirely and restore the nonprofit's original structure. If that happens, every dollar of OpenAI's $500B valuation flows back under nonprofit governance. This is why the "he's doing it for competitive reasons" argument has a structural problem. A plaintiff who wants to strengthen the defendant's nonprofit arm and remove its CEO from the nonprofit board isn't trying to destroy OpenAI. He's trying to restore the version of OpenAI that recruited its first 50 employees. Trial starts April 27 in Oakland. Jury selection is going to be very interesting for OpenAI's lawyers now.

@__tinygrad__ · 2026-04-08 07:29
@aakashgupta does this mean we get open source GPT 5.4 if Elon wins?

@__tinygrad__ · 2026-04-08 09:59
Humans are 100T models with 1T active parameters that run at 10 tok/s. They train on only 10B tokens.

@nopainkiller · 2026-04-08 10:12
If you count in all the sensory data we encountered everyday through lifetime, it's a lot more than 10B

@nopainkiller · 2026-04-08 10:12
If you count in all the sensory data we encountered everyday through lifetime, it's a lot more than 10B

@__tinygrad__ · 2026-04-08 10:13
@nopainkiller Is it? Video compresses really really well.

@__tinygrad__ · 2026-04-08 09:59
Humans are 100T models with 1T active parameters that run at 10 tok/s. They train on only 10B tokens.

@ar0cket1 · 2026-04-08 10:19
@__tinygrad__ well its a lot more than 10B tokens, hundreds of trillions

@ar0cket1 · 2026-04-08 10:19
@__tinygrad__ well its a lot more than 10B tokens, hundreds of trillions

@__tinygrad__ · 2026-04-08 10:26
@ar0cket1 Not external it's not. It's probably 100x more, like 1T with internal world model rollouts though.

@conoro · 2026-04-08 07:25
@stefan_on_ai @openclaw Only a small number so far: ("llama3","llama-v3","llama-bpe","qwen2","olmo") There's a PR for qwen-3.5. I ran qwen-3.5-9b but it was barely functional.

@__tinygrad__ · 2026-04-08 13:00
@conoro @stefan_on_ai @openclaw Default port is now 8000. Qwen 3.5 and Gemma 4 PRs should work okay, but better when upstreamed. JITBEAM=2 for speed

@LottoLabs · 2026-04-09 12:10
Tinygrad/27b/3090

@mamajjo1 · 2026-04-09 13:01
@LottoLabs Hi Lotto, I’m good at math and ok at programming and just got an amd 7900xtx, never played with tinygrad, but it sounds super interesting. Give me a % chance that I can beat llama.cpp tokens/s if implementing qwen3.5-27b inference is my only concern for like a month :)

@LottoLabs · 2026-04-09 20:47
@mamajjo1 Imma be real with you, it’s probably near zero, but not zero, and that’s reason enough to try

@LottoLabs · 2026-04-09 20:47
@mamajjo1 Imma be real with you, it’s probably near zero, but not zero, and that’s reason enough to try

@__tinygrad__ · 2026-04-10 01:03
@LottoLabs @mamajjo1 it's not hard at all if you use custom kernels

@__tinygrad__ · 2026-04-10 06:05
Added docs/abstractions4.py showing you 5 different ways to code a 1B element sum kernel each getting progressively faster. On AMD 7900XTX with 960 GB/s of RAM bandwidth. Naive tinygrad, ChatGPT written HIP, in the UOp language, tinygrad with BEAM=2, and custom assembly. https://t.co/CLvfIJH3yC

@LeeLeepenkman · 2026-04-10 07:00
Most the gains here are from chatgpt :D. Deep learning worked, get in people

@__tinygrad__ · 2026-04-10 06:05
Added docs/abstractions4.py showing you 5 different ways to code a 1B element sum kernel each getting progressively faster. On AMD 7900XTX with 960 GB/s of RAM bandwidth. Naive tinygrad, ChatGPT written HIP, in the UOp language, tinygrad with BEAM=2, and custom assembly. https://t.co/CLvfIJH3yC

@__tinygrad__ · 2026-04-10 07:31
If you have AMD and haven't tried VIZ=2 yet, you are missing out! It's in the runtime, so it works with all 5 ways of running the kernel. This is the profile of asm one, I bet you can fix the stuttering of the VALU and get another 0.5%. https://t.co/orssfdQFC1

@LeeLeepenkman · 2026-04-10 07:00
Most the gains here are from chatgpt :D. Deep learning worked, get in people

@__tinygrad__ · 2026-04-10 07:39
@LeeLeepenkman Err, the ChatGPT kernel is very mid. tinygrad's search beats it, and the handcoded assembly (light AI assistance) beats that.

@__tinygrad__ · 2026-04-10 07:43
Added a new $1,000 bounty: "Cycle accurate RDNA3 emulator (add SQTT support to emulator and have it match non DRAM kernels on real hardware perfectly)" This is AI proof and has a perfectly clear objective. Try all the AI you can to make it match real hardware.

@__tinygrad__ · 2026-04-10 07:43
Added a new $1,000 bounty: "Cycle accurate RDNA3 emulator (add SQTT support to emulator and have it match non DRAM kernels on real hardware perfectly)" This is AI proof and has a perfectly clear objective. Try all the AI you can to make it match real hardware.

@__tinygrad__ · 2026-04-10 07:48
You can get into some really fun stuff with the microarchitecture model, start with just the timing of VGPR add instructions to different registers. https://t.co/1Zc5atDA90 https://t.co/4tIdRA40g9

@__tinygrad__ · 2026-04-10 06:05
Added docs/abstractions4.py showing you 5 different ways to code a 1B element sum kernel each getting progressively faster. On AMD 7900XTX with 960 GB/s of RAM bandwidth. Naive tinygrad, ChatGPT written HIP, in the UOp language, tinygrad with BEAM=2, and custom assembly. https://t.co/CLvfIJH3yC

@EitanTurok · 2026-04-10 08:33
@__tinygrad__ can you plug in low level DSL's in tinygrad? e.g. just copy and pase cuda, mlx, hip, assmebly, ptx kernels?

@EitanTurok · 2026-04-10 08:33
@__tinygrad__ can you plug in low level DSL's in tinygrad? e.g. just copy and pase cuda, mlx, hip, assmebly, ptx kernels?

@__tinygrad__ · 2026-04-10 11:14
@EitanTurok Yes! The way the HIP kernel works works for all of the different languages, CUDA and Metal no problem.

@sama · 2026-04-10 22:58
I wrote this early this morning and I wasn't sure if I would actually publish it, but here it is: https://t.co/7Dw9UFpeep

@pmarca · 2026-04-10 23:11
“This raises an obvious question: how much of Anthropic’s reluctance to make Mythos widely available is due to security concerns, as opposed to the more prosaic reality that Anthropic simply doesn’t have enough compute?” @stratechery @benthompson

@sama · 2026-04-10 22:58
I wrote this early this morning and I wasn't sure if I would actually publish it, but here it is: https://t.co/7Dw9UFpeep

@__tinygrad__ · 2026-04-11 02:40
@sama "The only solution I can come up with is to orient towards sharing the technology with people broadly, and for no one to have the ring." Thank you. Can OpenAI go back to open source?

@pmarca · 2026-04-10 23:11
“This raises an obvious question: how much of Anthropic’s reluctance to make Mythos widely available is due to security concerns, as opposed to the more prosaic reality that Anthropic simply doesn’t have enough compute?” @stratechery @benthompson

@__tinygrad__ · 2026-04-11 03:43
@pmarca @stratechery @benthompson If only they had a commoditized petaflop.

@__tinygrad__ · 2026-04-11 07:10
@kiriyo9302 How much are you willing to pay for it?

@__tinygrad__ · 2026-04-11 10:19
Just merged an external PR for Bonsai-8B support (1 bit LLM). Because tinygrad has the correct abstractions, it was 5 lines. https://t.co/BLljWDANgq https://t.co/GlXWqPbYg5

@basement_agi · 2026-04-11 10:56
@__tinygrad__ Niiice! It was gibberish in llama cpp, looking forward to test it in tiny grad

@__tinygrad__ · 2026-04-11 11:30
@basement_agi Looks correct in quick tests. Just tinygrad/apps/llm.py -m <path to GGUF> --serve

@__tinygrad__ · 2026-04-11 11:30
@basement_agi Looks correct in quick tests. Just tinygrad/apps/llm.py -m <path to GGUF> --serve

@__tinygrad__ · 2026-04-11 11:31
@basement_agi And don't forget the JITBEAM=2 for speed!

@__tinygrad__ · 2026-04-11 10:19
Just merged an external PR for Bonsai-8B support (1 bit LLM). Because tinygrad has the correct abstractions, it was 5 lines. https://t.co/BLljWDANgq https://t.co/GlXWqPbYg5

@bruce_x_offi · 2026-04-11 12:14
@__tinygrad__ How would you rate opencl kernel support of tinygrad? Is it well written? Would love to try it on my b60 intel gpu

@bruce_x_offi · 2026-04-11 12:14
@__tinygrad__ How would you rate opencl kernel support of tinygrad? Is it well written? Would love to try it on my b60 intel gpu

@__tinygrad__ · 2026-04-11 15:20
@bruce_x_offi Just try it, should work. It's 95% the same codegen path as HIP and CUDA

@QuixiAI · 2026-04-12 03:44
The one and only purpose for this metal lip is to make it hard to install anything https://t.co/QeswsA0fmf

@QuixiAI · 2026-04-12 03:58
Louder than my DGX A100 https://t.co/6tHhIyWYae

@BioAlessandro · 2026-04-12 23:39
It's nice that we could get Bonsai-family support so quickly, but this is a bit disingenuous. I have never contributed to tinygrad so I am not in a position to critique this, however this implementation unpacks the 1bit weights as float16 and runs computations on float16 instead of running custom kernels on the packed weights, nullifying a lot of the benefits of the Bonsai architecture. Q1_0 It is a packed 1-bit format: for each block of 128 weights, you store 16 bytes of bits and 2 bytes for a shared fp16 scale. 128 weights take 18 bytes total. If you unpack those same 128 weights into float16, that becomes 256 bytes (14x). This is basically unpacking the "bit-based llm" in normal float16 and running calculations that way. My understanding of that llama.cpp’s Bonsai support keeps the weights in the quantized Q1_0 representation and uses kernels that operate on that packed format, which is the whole point. Again, I do not mean this as a shot at the implementation itself. Getting support working this quickly is genuinely cool. I might also be misunderstanding some parts of this, hopefully not too much, but would love to be corrected.

@__tinygrad__ · 2026-04-11 10:19
Just merged an external PR for Bonsai-8B support (1 bit LLM). Because tinygrad has the correct abstractions, it was 5 lines. https://t.co/BLljWDANgq https://t.co/GlXWqPbYg5

@BioAlessandro · 2026-04-12 23:45
@__tinygrad__ https://t.co/8me4VdjXDa

@BioAlessandro · 2026-04-12 23:39
It's nice that we could get Bonsai-family support so quickly, but this is a bit disingenuous. I have never contributed to tinygrad so I am not in a position to critique this, however this implementation unpacks the 1bit weights as float16 and runs computations on float16 instead of running custom kernels on the packed weights, nullifying a lot of the benefits of the Bonsai architecture. Q1_0 It is a packed 1-bit format: for each block of 128 weights, you store 16 bytes of bits and 2 bytes for a shared fp16 scale. 128 weights take 18 bytes total. If you unpack those same 128 weights into float16, that becomes 256 bytes (14x). This is basically unpacking the "bit-based llm" in normal float16 and running calculations that way. My understanding of that llama.cpp’s Bonsai support keeps the weights in the quantized Q1_0 representation and uses kernels that operate on that packed format, which is the whole point. Again, I do not mean this as a shot at the implementation itself. Getting support working this quickly is genuinely cool. I might also be misunderstanding some parts of this, hopefully not too much, but would love to be corrected.

@__tinygrad__ · 2026-04-13 00:10
@BioAlessandro It doesn't, it fuses the kernels unpack and compute! Look at the generated kernels with VIZ=1

@BioAlessandro · 2026-04-12 23:45
@__tinygrad__ https://t.co/8me4VdjXDa

@__tinygrad__ · 2026-04-13 00:13
@BioAlessandro I replied to this. Read the generated kernels, I think you'd be surprised. Actually you don't even have to read, you can just look at the memory usage. It's fused.

@RyanLeeMiniMax · 2026-04-13 06:12
https://t.co/UhsQSbFXD2

@__tinygrad__ · 2026-04-13 07:58
.@AMD @AnushElangovan @LisaSu should learn a lesson from the Intel Arc Pro B70. Release a 9070XT with 32GB of RAM for a reasonable price! Nobody wants blowers, nobody wants "pro" market segmentation, and nobody wants Intel. AMD can beat NVIDIA by making normal GPUs with big RAM.

@__tinygrad__ · 2026-04-13 07:58
.@AMD @AnushElangovan @LisaSu should learn a lesson from the Intel Arc Pro B70. Release a 9070XT with 32GB of RAM for a reasonable price! Nobody wants blowers, nobody wants "pro" market segmentation, and nobody wants Intel. AMD can beat NVIDIA by making normal GPUs with big RAM.

@zeroxdoubler · 2026-04-13 08:00
@__tinygrad__ @AMD @AnushElangovan @LisaSu Margin thin 🥀

@zeroxdoubler · 2026-04-13 08:00
@__tinygrad__ @AMD @AnushElangovan @LisaSu Margin thin 🥀

@__tinygrad__ · 2026-04-13 08:00
@0xRezaRamadhan @AMD @AnushElangovan @LisaSu Market share big

@__tinygrad__ · 2026-04-13 07:58
.@AMD @AnushElangovan @LisaSu should learn a lesson from the Intel Arc Pro B70. Release a 9070XT with 32GB of RAM for a reasonable price! Nobody wants blowers, nobody wants "pro" market segmentation, and nobody wants Intel. AMD can beat NVIDIA by making normal GPUs with big RAM.

@yasei_no_otoko · 2026-04-13 08:06
@__tinygrad__ @AMD @AnushElangovan @LisaSu Is the R9700 Pro out of stock outside of Japan?🤔 It's the 32GB version of the 9070XT, retailing for $1,500.

@__tinygrad__ · 2026-04-13 07:58
.@AMD @AnushElangovan @LisaSu should learn a lesson from the Intel Arc Pro B70. Release a 9070XT with 32GB of RAM for a reasonable price! Nobody wants blowers, nobody wants "pro" market segmentation, and nobody wants Intel. AMD can beat NVIDIA by making normal GPUs with big RAM.

@QuixiAI · 2026-04-13 08:08
@__tinygrad__ @AMD @AnushElangovan @LisaSu make a TinyBox with B70s

@QuixiAI · 2026-04-13 08:08
@__tinygrad__ @AMD @AnushElangovan @LisaSu make a TinyBox with B70s

@__tinygrad__ · 2026-04-13 08:16
@QuixiAI @AMD @AnushElangovan @LisaSu lol not touching Intel they have no leadership. everything will be cancelled and the software work will be wasted. I feel bad for the people who buy these cards expecting things to support them

@yasei_no_otoko · 2026-04-13 08:06
@__tinygrad__ @AMD @AnushElangovan @LisaSu Is the R9700 Pro out of stock outside of Japan?🤔 It's the 32GB version of the 9070XT, retailing for $1,500.

@__tinygrad__ · 2026-04-13 08:17
@yasei_no_otoko @AMD @AnushElangovan @LisaSu too expensive, blower, and "pro" I want normal 9070 XT 32GB for $999

@QuixiAI · 2026-04-13 08:23
I'm actually impressed by the trajectory of Gaudi (have you look at their new vllm hardware plugin?) AMD was in this rut, 2 years ago, and look where they are now. And look at how far Intel has come in the last year. In another year - I can see them being where AMD is now, if they try hard. I have watched these cycles since 1990 - ups and downs. They will come back up.

@__tinygrad__ · 2026-04-13 08:25
@QuixiAI @AMD @AnushElangovan @LisaSu AMD always had good hardware and leadership, they just lacked software. Intel is missing all 3.

@__tinygrad__ · 2026-04-13 08:25
@QuixiAI @AMD @AnushElangovan @LisaSu AMD always had good hardware and leadership, they just lacked software. Intel is missing all 3.

@__tinygrad__ · 2026-04-13 08:27
@QuixiAI @AMD @AnushElangovan @LisaSu Btw wanna buy some Gaudi? it's almost at scrap metal cost https://t.co/oFatXcT7Mv

@__tinygrad__ · 2026-04-13 08:27
@QuixiAI @AMD @AnushElangovan @LisaSu Btw wanna buy some Gaudi? it's almost at scrap metal cost https://t.co/oFatXcT7Mv

@QuixiAI · 2026-04-13 08:34
@__tinygrad__ @AMD @AnushElangovan @LisaSu I did! https://t.co/P70GC9WnvS

@QuixiAI · 2026-04-13 08:34
@__tinygrad__ @AMD @AnushElangovan @LisaSu I did! https://t.co/P70GC9WnvS

@__tinygrad__ · 2026-04-13 08:35
@QuixiAI @AMD @AnushElangovan @LisaSu lol no wonder you are shilling for this you are $17.5k in the hole. the ram is big and cheap. the architecture is awful. and the lineup is cancelled.

@RyanLeeMiniMax · 2026-04-13 06:12
https://t.co/UhsQSbFXD2

@__tinygrad__ · 2026-04-13 09:36
@RyanLeeMiniMax This is an extremely respectable position, thank you for the release!

@__tinygrad__ · 2026-04-13 13:17
Who thinks MiniMax M2.7 can add support for itself? https://t.co/H1y6p1l8Z4

@__tinygrad__ · 2026-04-13 13:17
Who thinks MiniMax M2.7 can add support for itself? https://t.co/H1y6p1l8Z4

@ree2raz · 2026-04-13 13:23
@__tinygrad__ 😅 this prompt and one shot success expectation is unfair, AGI has not arrived yet bro!

@__tinygrad__ · 2026-04-13 13:30
Recursive self improvement is so close!

@ree2raz · 2026-04-13 13:23
@__tinygrad__ 😅 this prompt and one shot success expectation is unfair, AGI has not arrived yet bro!

@__tinygrad__ · 2026-04-13 14:28
@ree2raz So MiniMax couldn't do it, but GLM-5.1 is still at it making progress and wow, that model can stay coherent for a long time.

@__tinygrad__ · 2026-04-14 03:11
RT @howenyap: @ludwigABAP no idea but did see this tinygrad blog by a tenstorrent guy https://t.co/wkjfnP4rqO

@elonmusk · 2026-04-15 07:21
Congrats to the @Tesla_AI chip design team on taping out AI5! AI6, Dojo3 & other exciting chips in work. https://t.co/hm54TdIzBx

@elonmusk · 2026-04-15 07:21
Congrats to the @Tesla_AI chip design team on taping out AI5! AI6, Dojo3 & other exciting chips in work. https://t.co/hm54TdIzBx

@__tinygrad__ · 2026-04-15 07:54
@elonmusk @Tesla_AI Are you going to sell chips? Or just bundled with cars? Food for thought, NVIDIA has 5x the market cap of Tesla...

@__tinygrad__ · 2026-04-15 09:02
We're hiring from the pool of tinygrad contributors. Hybrid in-person/remote, offices in San Diego and Hong Kong. In the era of slop, come help build something beautiful. https://t.co/tCXCicV71M

@__tinygrad__ · 2026-04-15 09:02
We're hiring from the pool of tinygrad contributors. Hybrid in-person/remote, offices in San Diego and Hong Kong. In the era of slop, come help build something beautiful. https://t.co/tCXCicV71M

@ossframework · 2026-04-15 09:14
@__tinygrad__ Always wanted to live in HK might start contributing

@ossframework · 2026-04-15 09:14
@__tinygrad__ Always wanted to live in HK might start contributing

@__tinygrad__ · 2026-04-15 09:18
@ossframework It's one of the best places in the world. A combination of good things from the west and good things from China, reasonable cost of living for a top tier city, simple visa policies, cheap fast internet, and no fake hustle culture.

@__tinygrad__ · 2026-04-15 09:02
We're hiring from the pool of tinygrad contributors. Hybrid in-person/remote, offices in San Diego and Hong Kong. In the era of slop, come help build something beautiful. https://t.co/tCXCicV71M

@JulienBlanchon · 2026-04-15 09:19
@__tinygrad__ I'm curious, where are in HK ?

@JulienBlanchon · 2026-04-15 09:19
@__tinygrad__ I'm curious, where are in HK ?

@__tinygrad__ · 2026-04-15 09:22
@JulienBlanchon Sheung Wan, on the island. Art districts have the highest quality coffee shops.

@__tinygrad__ · 2026-04-15 09:02
We're hiring from the pool of tinygrad contributors. Hybrid in-person/remote, offices in San Diego and Hong Kong. In the era of slop, come help build something beautiful. https://t.co/tCXCicV71M

@ninoristeski · 2026-04-15 09:28
@__tinygrad__ Fully remote still possible?

@__tinygrad__ · 2026-04-15 09:59
https://t.co/WqOd6gcHzT

@ninoristeski · 2026-04-15 09:28
@__tinygrad__ Fully remote still possible?

@__tinygrad__ · 2026-04-15 10:06
@ninoristeski More info here. https://t.co/GVjOAQA7Au

@__tinygrad__ · 2026-04-15 09:02
We're hiring from the pool of tinygrad contributors. Hybrid in-person/remote, offices in San Diego and Hong Kong. In the era of slop, come help build something beautiful. https://t.co/tCXCicV71M

@yangWao · 2026-04-15 11:31
@__tinygrad__ Why did the comma open the HK branch, honestly? Is it better access to tech/talent?

@yangWao · 2026-04-15 11:31
@__tinygrad__ Why did the comma open the HK branch, honestly? Is it better access to tech/talent?

@__tinygrad__ · 2026-04-15 11:47
@yangWao The main reason, issues with visas in the US. Hong Kong welcomes high skill immigration.

@scaling01 · 2026-04-15 19:47
this is the most hilarious Jensen clip I've ever seen "You are not talking to somebody that woke up a loser" "We are not a car" https://t.co/bsCmeirqvG

@OrganicGPT · 2026-04-16 04:44
I tried @__tinygrad__'s Mac+eGPU setup on Qwen3.5 9B. It's roughly HALF the speed of a PC. The PC (rtx) actually has the weaker RTX Pro card too. Still impressive that this is even possible on Mac! @realGeorgeHotz https://t.co/5Sux0Wzcut

@OrganicGPT · 2026-04-16 04:57
Some things I noticed: (1) The GPU gets "wedged" all the time; it gets stuck in bad state (GSP holding stale state). Replugging the TB cable fixes it. (2) PARALLEL=0 is slow on Mac for the 9B model (3) Mac multiprocessing BEAM hangs on large Kernel ASTs

@OrganicGPT · 2026-04-16 04:44
I tried @__tinygrad__'s Mac+eGPU setup on Qwen3.5 9B. It's roughly HALF the speed of a PC. The PC (rtx) actually has the weaker RTX Pro card too. Still impressive that this is even possible on Mac! @realGeorgeHotz https://t.co/5Sux0Wzcut

@__tinygrad__ · 2026-04-16 05:22
@OrganicGPT @realGeorgeHotz I'm assuming you used JITBEAM? Qwen3.5 is a bit slower because of the SSM thing, you are welcome to optimize it. you can use a custom kernel and skip it entirely, you can even use all custom kernels and get full native perf.

@OrganicGPT · 2026-04-16 04:57
Some things I noticed: (1) The GPU gets "wedged" all the time; it gets stuck in bad state (GSP holding stale state). Replugging the TB cable fixes it. (2) PARALLEL=0 is slow on Mac for the 9B model (3) Mac multiprocessing BEAM hangs on large Kernel ASTs

@__tinygrad__ · 2026-04-16 11:20
@OrganicGPT hmm, the wedging is maybe an issue with your dock? JITBEAM should work the exact same with Mac or Linux.

@Linahuaa · 2026-04-16 11:41
Jensen is actually right. If you keep China getting hooked on NVIDIA chips + CUDA ecosystem, then at least the Chinese have to fork out tons of cash for overpriced monopoly chips. If you sanction them like you do now, you delay their progress in the short term, but in 10 years, China will have their own much cheaper chips with own software stack and you gonna eat shit. AGI isn't coming anytime soon. You want to win long-term, not short-term.

@__tinygrad__ · 2026-04-16 13:05
@Linahuaa ahh but herein lies the problem. the Americans think AGI is a year away.

@Varelli1999 · 2026-04-16 13:18
@__tinygrad__ @Linahuaa It is like that past 4 years.

@Varelli1999 · 2026-04-16 13:18
@__tinygrad__ @Linahuaa It is like that past 4 years.

@__tinygrad__ · 2026-04-16 13:20
@Varelli1999 @Linahuaa oh yes. this belief doesn't change. in one year it'll also be a year away.

@__tinygrad__ · 2026-04-16 14:27
.@UnslothAI so fast with those @Alibaba_Qwen 3.6 GGUFs! Here's Qwen3.6-35B-A3B on a 7900XTX at 90 tok/s, available right now in tinygrad master. https://t.co/OcY50ZXppG

@__tinygrad__ · 2026-04-16 14:27
.@UnslothAI so fast with those @Alibaba_Qwen 3.6 GGUFs! Here's Qwen3.6-35B-A3B on a 7900XTX at 90 tok/s, available right now in tinygrad master. https://t.co/OcY50ZXppG

@__tinygrad__ · 2026-04-16 14:36
OpenAI compatible chat server with `--serve`. Token gen is pretty fast, prefill is meh, the SSM needs SCAN if you want a real challenge. For a simpler challenge, add tool calling to tinygrad.apps.llm so it works with OpenCode? Very tasteful PRs only, no AI slop.

@davidpwalter · 2026-04-16 15:18
Got Qwen3.5 0.8B working with @__tinygrad__ + mac studio + 5090-egpu (model 100% on the 5090). Next Qwen3.5-27B https://t.co/yzUH7vCeth

@pupposandro · 2026-04-16 16:20
https://t.co/BSXzofi3S3

@pupposandro · 2026-04-16 16:20
a month ago with @davideciffa we started experimenting with eGPUs. the potential is wild, skip the RAM and mobo spend, put it all into a GPU connected straight to your Mac. then we noticed that reality was that they still don't work well. recently, we got hyped like everyone when Apple approved tinygrad NVIDIA drivers for macOS. they actually work! but inference results aren't there yet: - RTX 5090 via eGPU: 6 tok/s on Qwen3-8B - llama.cpp on M4 Pro: 74 tok/s same model - GPU memory bandwidth sitting at 1.2% it's a software maturity gap, not hardware. we're going to work on this soon. eGPUs could change the local AI landscape as we know it. wrote a full X article on it: https://t.co/SnaaO7Sjq5

@davidpwalter · 2026-04-16 16:27
Qwen3.5-27B running on 5090-egpu! only at 0.3tok/s. Very much a prototype, but after performance optimization should be able to get it closer to 70-100tok/s that it gets on a normal PC setup since the entire model is on the eGPU. https://t.co/A1KPCegsVX

@PrismML · 2026-04-16 17:39
Today we’re announcing Ternary Bonsai: Top intelligence at 1.58 bits Using ternary weights {-1, 0, +1}, we built a family of models that are 9x smaller than their 16-bit counterparts while outperforming most models in their respective parameter classes on standard benchmarks. We’re open-sourcing the models under the Apache 2.0 license in three sizes: 8B (1.75 GB), 4B (0.86 GB), and 1.7B (0.37 GB).

@PrismML · 2026-04-16 17:39
Today we’re announcing Ternary Bonsai: Top intelligence at 1.58 bits Using ternary weights {-1, 0, +1}, we built a family of models that are 9x smaller than their 16-bit counterparts while outperforming most models in their respective parameter classes on standard benchmarks. We’re open-sourcing the models under the Apache 2.0 license in three sizes: 8B (1.75 GB), 4B (0.86 GB), and 1.7B (0.37 GB).

@PrismML · 2026-04-16 17:39
Our earlier 1-bit Bonsai models established a new Pareto frontier. Ternary Bonsai pushes that frontier further. For example, Ternary Bonsai 4B scores roughly 8 points higher on average across benchmarks with just 300MB more memory footprint, compared to the 1-bit Bonsai 4B. For many deployments, this offers a new balance of capability, and deployment efficiency.

@ivanfioravanti · 2026-04-16 19:41
"The driver is a miracle. The inference is not." Great article telling uncomfortable truths. https://t.co/NWo5YIHvbD

@pupposandro · 2026-04-16 16:20
a month ago with @davideciffa we started experimenting with eGPUs. the potential is wild, skip the RAM and mobo spend, put it all into a GPU connected straight to your Mac. then we noticed that reality was that they still don't work well. recently, we got hyped like everyone when Apple approved tinygrad NVIDIA drivers for macOS. they actually work! but inference results aren't there yet: - RTX 5090 via eGPU: 6 tok/s on Qwen3-8B - llama.cpp on M4 Pro: 74 tok/s same model - GPU memory bandwidth sitting at 1.2% it's a software maturity gap, not hardware. we're going to work on this soon. eGPUs could change the local AI landscape as we know it. wrote a full X article on it: https://t.co/SnaaO7Sjq5

@__tinygrad__ · 2026-04-17 00:23
@pupposandro @davideciffa did you run with JITBEAM=2?

@ivanfioravanti · 2026-04-16 19:41
"The driver is a miracle. The inference is not." Great article telling uncomfortable truths. https://t.co/NWo5YIHvbD

@__tinygrad__ · 2026-04-17 02:39
@ivanfioravanti I know it's our fault and we need to make this simpler, but you can 3x perf with JITBEAM=2. It does a search for fast kernels in exchange for a one time 10 minute startup time.

@__tinygrad__ · 2026-04-17 03:10
This is the price of a 16GB DDR5 RDIMM stick. So when is this gonna be cheap again? I thought Sam Altman ran out of money to buy all the memory. https://t.co/lF05DWDN8t

@__tinygrad__ · 2026-04-16 14:27
.@UnslothAI so fast with those @Alibaba_Qwen 3.6 GGUFs! Here's Qwen3.6-35B-A3B on a 7900XTX at 90 tok/s, available right now in tinygrad master. https://t.co/OcY50ZXppG

@YouJiacheng · 2026-04-17 03:11
@__tinygrad__ @UnslothAI @Alibaba_Qwen hmmm why 90 tok/s shows 437GB/s for an A3B@Q4 model? qkvo proj @ 16bits?

@YouJiacheng · 2026-04-17 03:11
@__tinygrad__ @UnslothAI @Alibaba_Qwen hmmm why 90 tok/s shows 437GB/s for an A3B@Q4 model? qkvo proj @ 16bits?

@__tinygrad__ · 2026-04-17 03:15
@YouJiacheng @UnslothAI @Alibaba_Qwen I asked exactly this in our Discord, not sure. It should be less, all the weights are unpacked on the fly. Explore with VIZ=1

@__tinygrad__ · 2026-04-17 04:01
@ramathakovesh and it wasn't cross entropy loss on the internet, that scaling stopped quickly. what made it good was a massively complex training pipeline using RL in agentic loops.

@__tinygrad__ · 2026-04-17 04:06
@ramathakovesh even transformers are showing cracks. the classic O(n) KV cache has always been obviously wrong, Qwen 3.5 is mostly SSM. and DSA is the open source SOTA. who knows what the closed labs have.

@LottoLabs · 2026-04-17 04:10
Tinygrad inference getting attention and a qwen 3.6 27b release are going to massively change local models in 2026

@__tinygrad__ · 2026-04-17 04:06
@ramathakovesh even transformers are showing cracks. the classic O(n) KV cache has always been obviously wrong, Qwen 3.5 is mostly SSM. and DSA is the open source SOTA. who knows what the closed labs have.

@__tinygrad__ · 2026-04-17 04:16
@ramathakovesh at that time I didn't expect context length scaling to just work though, it's super cool that it does. the problem of "memory" kind of went away with 1M context.

@__tinygrad__ · 2026-04-17 04:50
We are putting a lot of effort into our tinygrad.llm inference engine. Unlike others, quants, model variants, and backends are factorized. You never have to ask if this model in this quant works on this backend. It all works everywhere; searches for fast kernels with JITBEAM=2

@__tinygrad__ · 2026-04-17 04:50
We are putting a lot of effort into our tinygrad.llm inference engine. Unlike others, quants, model variants, and backends are factorized. You never have to ask if this model in this quant works on this backend. It all works everywhere; searches for fast kernels with JITBEAM=2

@__tinygrad__ · 2026-04-17 04:54
If you are new to tinygrad, set the backend with DEV, DEBUG=2 shows you all running kernels, and VIZ=1 is the simplest profiler you have ever used. You have everything you need in the 23k line repo to make it very fast, support your model, and support your custom accelerator.

@__tinygrad__ · 2026-04-17 04:50
We are putting a lot of effort into our tinygrad.llm inference engine. Unlike others, quants, model variants, and backends are factorized. You never have to ask if this model in this quant works on this backend. It all works everywhere; searches for fast kernels with JITBEAM=2

@madprizm0 · 2026-04-17 04:55
@__tinygrad__ How well does tinygrad do with matrix exponentials and weird NN architectures broadly? I’d love something where I don’t have to write triton kernels.

@madprizm0 · 2026-04-17 04:55
@__tinygrad__ How well does tinygrad do with matrix exponentials and weird NN architectures broadly? I’d love something where I don’t have to write triton kernels.

@__tinygrad__ · 2026-04-17 04:56
@madprizm0 Try it!

@__tinygrad__ · 2026-04-17 04:50
We are putting a lot of effort into our tinygrad.llm inference engine. Unlike others, quants, model variants, and backends are factorized. You never have to ask if this model in this quant works on this backend. It all works everywhere; searches for fast kernels with JITBEAM=2

@sleep_deprivado · 2026-04-17 06:49
@__tinygrad__ isn't it amazing how bad the entire freaking ecosystem is? Their incompetence is... amazing. And they're stacking complexity intel/amd/nvidia replicate/amplify, it's freaking nuts

@sleep_deprivado · 2026-04-17 06:49
@__tinygrad__ isn't it amazing how bad the entire freaking ecosystem is? Their incompetence is... amazing. And they're stacking complexity intel/amd/nvidia replicate/amplify, it's freaking nuts

@__tinygrad__ · 2026-04-17 07:05
@sleep_deprivado Who's incompetence? We've been working on tinygrad for 5 years, it's just extremely hard to make things simple. The good news is, once it's simple, everyone will say well that was obvious duh I could've vibe coded tinygrad in a weekend.

@PrismML · 2026-04-16 17:39
Our earlier 1-bit Bonsai models established a new Pareto frontier. Ternary Bonsai pushes that frontier further. For example, Ternary Bonsai 4B scores roughly 8 points higher on average across benchmarks with just 300MB more memory footprint, compared to the 1-bit Bonsai 4B. For many deployments, this offers a new balance of capability, and deployment efficiency.

@__tinygrad__ · 2026-04-17 08:49
@PrismML Bonsai is cool, but you should compare to 4-bit quants of Qwen, not the BF16 that nobody runs.

@ljupc0 · 2026-04-17 09:46
After seeing @__tinygrad__ post, reproduced (on a 7900xtx) - $ git clone ... , then $ uv pip install -e . , then ~/tinygrad$ JITBEAM=2 python3 -m tinygrad.llm --model ~/llama.cpp/models/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf --benchmark 32 One is to be patient the 1st run, apparently 'kernels are being optimised' and this is being created $ ls -lh ~/.cache/tinygrad/cache.db -rw------- 1 ljubomir ljubomir 138M Apr 17 10:03 /home/ljubomir/.cache/tinygrad/cache.db It works! The quants are by @UnslothAI. Was bracing myself for conversions, but then pleasantly surprised tinygrad consumes .gguf-s?? - yay!! More power to everyone involved. 🤯 Months ago it was "I can not believe what I see", but now it's "just another 24h of bona fide miracles witnessed made real - meh".

@__tinygrad__ · 2026-04-17 10:12
@ljupc0 glad it was easy to repro. the prefill needs to be faster (needs scan) and the BEAM should communicate more, but it's kind of usable now

@ljupc0 · 2026-04-17 10:30
Thank you! 🙏 Wowzers, even 120 tok/s now, with "4" not "2"?? 😊 (torch313-rocm) ljubomir@gigul2(7_a:::master):~/tinygrad$ JITBEAM=4 python3 -m tinygrad.llm --model ~/llama.cpp/models/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf --benchmark 32 I notice now '$ rocm-smi --showmeminfo vram' that GPU[0] : VRAM Total Used Memory (B): 27975680 went above VRAM Total Memory (B): 25753026560. Not sure if due to the "4" instead "2" now, or maybe it was the case before too.

@ljupc0 · 2026-04-17 10:30
Thank you! 🙏 Wowzers, even 120 tok/s now, with "4" not "2"?? 😊 (torch313-rocm) ljubomir@gigul2(7_a:::master):~/tinygrad$ JITBEAM=4 python3 -m tinygrad.llm --model ~/llama.cpp/models/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf --benchmark 32 I notice now '$ rocm-smi --showmeminfo vram' that GPU[0] : VRAM Total Used Memory (B): 27975680 went above VRAM Total Memory (B): 25753026560. Not sure if due to the "4" instead "2" now, or maybe it was the case before too.

@__tinygrad__ · 2026-04-17 10:38
@ljupc0 that number is the width of the BEAM in the beam search

@__tinygrad__ · 2026-04-17 11:03
They have discovered JITBEAM=4. This GPU (AMD 7900XTX) costs $620 on eBay and tinygrad entirely bypasses all AMD drivers and HIP. Just waiting for someone to write the tool calling stuff and tinygrad can be your go to engine for openclaw and opencode.

@__tinygrad__ · 2026-04-17 11:03
They have discovered JITBEAM=4. This GPU (AMD 7900XTX) costs $620 on eBay and tinygrad entirely bypasses all AMD drivers and HIP. Just waiting for someone to write the tool calling stuff and tinygrad can be your go to engine for openclaw and opencode.

@anili_v · 2026-04-17 11:25
@__tinygrad__ Could you please explain why tool calling needs to be integrated into tinygrad? I thought in tool calling you just prompt the LLM asking it to return JSON and retry the prompt if it isn't valid JSON. So why can't tinygrad be used right now?

@anili_v · 2026-04-17 11:25
@__tinygrad__ Could you please explain why tool calling needs to be integrated into tinygrad? I thought in tool calling you just prompt the LLM asking it to return JSON and retry the prompt if it isn't valid JSON. So why can't tinygrad be used right now?

@__tinygrad__ · 2026-04-17 11:27
@anili_v try it, maybe it works

@LottoLabs · 2026-04-17 11:53
Bros I think we need to start a tinygrad emergency development group

@LottoLabs · 2026-04-17 11:53
Bros I think we need to start a tinygrad emergency development group

@__tinygrad__ · 2026-04-17 12:21
@LottoLabs come join the discord let's do it

@davidpwalter · 2026-04-16 16:27
Qwen3.5-27B running on 5090-egpu! only at 0.3tok/s. Very much a prototype, but after performance optimization should be able to get it closer to 70-100tok/s that it gets on a normal PC setup since the entire model is on the eGPU. https://t.co/A1KPCegsVX

@__tinygrad__ · 2026-04-17 12:28
@davidpwalter oh don't use NAK with 5090! if you use the CUDA in docker it's wayyy faster

@thdxr · 2026-04-18 15:29
are there people out there who just want to refactor every day? just wake up and find the worst code and just chip away at it and clean it up wake up the next day do it again, infinitely improving things with zero external impact?

@qubitium · 2026-04-19 08:17
🥳 Evalution v0.0.7 released with Tinygrad llm (gguf) inference support. You can now pair Evalution with @__tinygrad__ and use the 153+ and growing llm benchmarks (such as GSM8K Platinum) to test for model quality and inference regressions. https://t.co/9CVwMrHQFg

@qubitium · 2026-04-19 08:17
🥳 Evalution v0.0.7 released with Tinygrad llm (gguf) inference support. You can now pair Evalution with @__tinygrad__ and use the 153+ and growing llm benchmarks (such as GSM8K Platinum) to test for model quality and inference regressions. https://t.co/9CVwMrHQFg

@__tinygrad__ · 2026-04-19 12:32
@qubitium cool! how are we doing on the evals?

@__tinygrad__ · 2026-04-20 02:15
When we succeed at making the most performant full stack from Tensor to hardware for both major GPUs in 25k lines, a lot of people are going to have to ask what the other 10M lines were doing. Creating jobs?

@__tinygrad__ · 2026-04-20 02:18
RT @comma_ai: Hiring engineers who want to just refactor for love of the game

@__tinygrad__ · 2026-04-20 02:15
When we succeed at making the most performant full stack from Tensor to hardware for both major GPUs in 25k lines, a lot of people are going to have to ask what the other 10M lines were doing. Creating jobs?

@KinvertOG · 2026-04-20 02:19
@__tinygrad__ if you let china use your software is that the same as uranium

@KinvertOG · 2026-04-20 02:19
@__tinygrad__ if you let china use your software is that the same as uranium

@__tinygrad__ · 2026-04-20 02:20
@KinvertOG The beauty of open source is there is no us "letting" people use the software. Everyone just can. Now that's what I call freedom!

@__tinygrad__ · 2026-04-20 02:15
When we succeed at making the most performant full stack from Tensor to hardware for both major GPUs in 25k lines, a lot of people are going to have to ask what the other 10M lines were doing. Creating jobs?

@orionintx · 2026-04-20 03:00
@__tinygrad__ not jobs. load-bearing scar tissue. every undocumented register quirk nvidia silently handled since kepler. you'll rediscover them.

@__tinygrad__ · 2026-04-20 02:15
When we succeed at making the most performant full stack from Tensor to hardware for both major GPUs in 25k lines, a lot of people are going to have to ask what the other 10M lines were doing. Creating jobs?

@LostAngelNZ · 2026-04-20 03:42
@__tinygrad__ Development teams do not scale, so any team at scale has near negative marginal return when working on a project. @jimkxa has put the number at ~100 people.

@OrganicGPT · 2026-04-20 04:46
I tried @__tinygrad__'s Mac+eGPU workaround again, this time with JITBEAM=4. Model is Qwen3.5 9B Q4_M, prompt = 1024 toks: * I'm curious about PP gap; was expecting much better results from the GPU after BEAM search https://t.co/3zIZLa6eXm

@OrganicGPT · 2026-04-20 04:46
I tried @__tinygrad__'s Mac+eGPU workaround again, this time with JITBEAM=4. Model is Qwen3.5 9B Q4_M, prompt = 1024 toks: * I'm curious about PP gap; was expecting much better results from the GPU after BEAM search https://t.co/3zIZLa6eXm

@OrganicGPT · 2026-04-20 04:51
@__tinygrad__ JITBEAM was too slow on my M1 Pro, so I did it on a Linux workstation with the exact same GPU and it was freaking fast!

@OrganicGPT · 2026-04-20 04:46
I tried @__tinygrad__'s Mac+eGPU workaround again, this time with JITBEAM=4. Model is Qwen3.5 9B Q4_M, prompt = 1024 toks: * I'm curious about PP gap; was expecting much better results from the GPU after BEAM search https://t.co/3zIZLa6eXm

@__tinygrad__ · 2026-04-20 05:00
@OrganicGPT This has to do with BEAM not searching with padded Tensors properly for the variable length, so the GEMMs are very suboptimal. It's fixable, but someone has to fix it.

@__tinygrad__ · 2026-04-20 02:15
When we succeed at making the most performant full stack from Tensor to hardware for both major GPUs in 25k lines, a lot of people are going to have to ask what the other 10M lines were doing. Creating jobs?

@_Felipe · 2026-04-20 05:07
@__tinygrad__ It’s hard to show incremental progress and throw money at the problem of creating a compiler. But you can show progress by brute-force development of a huge number of kernels.

@orionintx · 2026-04-20 03:00
@__tinygrad__ not jobs. load-bearing scar tissue. every undocumented register quirk nvidia silently handled since kepler. you'll rediscover them.

@__tinygrad__ · 2026-04-20 05:07
@orionintx We already have this part finished, as in our driver already works with any modern AMD or NVIDIA GPU. It's compiler stuff left. We have a year more of refactors on the core stuff, then a year to write the search algorithm to find ultrafast schedules.

@LostAngelNZ · 2026-04-20 03:42
@__tinygrad__ Development teams do not scale, so any team at scale has near negative marginal return when working on a project. @jimkxa has put the number at ~100 people.

@__tinygrad__ · 2026-04-20 05:09
@LostAngelNZ @jimkxa It's not just scale, it's because of deeply broken development methodologies where people chase features instead of beauty. The cruft compounds and compounds.

@_Felipe · 2026-04-20 05:07
@__tinygrad__ It’s hard to show incremental progress and throw money at the problem of creating a compiler. But you can show progress by brute-force development of a huge number of kernels.

@__tinygrad__ · 2026-04-20 05:10
@_Felipe This. The damage that has been done to software development by managers looking for "progress" is in the billions of dollars, maybe even trillions.

@OrganicGPT · 2026-04-20 04:51
@__tinygrad__ JITBEAM was too slow on my M1 Pro, so I did it on a Linux workstation with the exact same GPU and it was freaking fast!

@__tinygrad__ · 2026-04-20 05:41
@OrganicGPT Oh, this might be because of the Docker calls to CUDA compiler. We spend a lot more time on AMD than NVIDIA, so nobody at tiny corp really feels that pain.

@LottoLabs · 2026-04-20 18:17
Hermes’ agent + ornstein3.6 35b Set up a simple notification system polling new qwen releases in GitHub Sends a telegram notification with the update and what it is Simple and sets up in 20 seconds, why not Waiting for qwen3.6 27b impatiently now https://t.co/EcsfwtSgBF

@LottoLabs · 2026-04-20 18:17
Hermes’ agent + ornstein3.6 35b Set up a simple notification system polling new qwen releases in GitHub Sends a telegram notification with the update and what it is Simple and sets up in 20 seconds, why not Waiting for qwen3.6 27b impatiently now https://t.co/EcsfwtSgBF

@mamajjo1 · 2026-04-20 18:26
@LottoLabs Btw, lil update on my tinygrad adventure: got that model running at 120tps, first implementation of tool calling. I’m scared of doing a pr, Geohot might smite for poor taste in python syntax. Also there’s still no batched prefill yet, so 5 min ttft

@mamajjo1 · 2026-04-20 18:26
@LottoLabs Btw, lil update on my tinygrad adventure: got that model running at 120tps, first implementation of tool calling. I’m scared of doing a pr, Geohot might smite for poor taste in python syntax. Also there’s still no batched prefill yet, so 5 min ttft

@__tinygrad__ · 2026-04-21 05:40
@mamajjo1 @LottoLabs lol smiting is just what happens when people PR things they aren't proud of. effort gets matched by effort, AI slop gets matched by smiting

@LottoLabs · 2026-04-21 05:44
I feel like multiple people are working on tinygrad inference It’s gonna be pretty cool once there’s a stable engine built w/ it

@__tinygrad__ · 2026-04-21 06:01
Check out our new agent friendly viz.cli profiler. We will beat the speed of millions of lines of heuristics with the power of search. You write high level tinygrad code, search makes it fast (customized for your hardware!). The search engine makes sure it stays correct. https://t.co/nwndsZGxgf

@0xSero · 2026-04-21 12:25
GLM-5.1-478B-NVFP4 Running on: - 4x RTX Pro 6000 - Sglang - 370,000 max tokens (1.75x full context) - p10 27.7 | p90 45.6 tok/s decode (gen) - 1340 tok/s prefill I could get 2x decode if I limit to 64k context (100 tok/s) In this video it operates Figma (: https://t.co/OO5tmVDRFW

@ClementDelangue · 2026-04-21 16:08
I’m hearing there’s renewed lobbying in DC and in state legislatures to ban or severely restrict open-source. Like a few years ago, we’ll need everyone to help show policymakers why open-source matters: for startups, for competition, for economic growth, and for jobs. If you build with open-source, now is the time to speak up!

@VictorTaelin · 2026-04-22 01:54
actually quite depressed about the mythos stuff grinding my way through, I'll do it with or without them

@ClementDelangue · 2026-04-21 16:08
I’m hearing there’s renewed lobbying in DC and in state legislatures to ban or severely restrict open-source. Like a few years ago, we’ll need everyone to help show policymakers why open-source matters: for startups, for competition, for economic growth, and for jobs. If you build with open-source, now is the time to speak up!

@__tinygrad__ · 2026-04-22 02:58
@ClementDelangue lol now we see the real target audience for Mythos

@VictorTaelin · 2026-04-22 01:54
actually quite depressed about the mythos stuff grinding my way through, I'll do it with or without them

@__tinygrad__ · 2026-04-22 04:53
@VictorTaelin are you going to help them spread FOMO and FUD and lobby against open source. oh, you aren't? you thought mythos was here to help developers? yea...that's why they didn't give you access.

@__tinygrad__ · 2026-04-22 04:53
@VictorTaelin are you going to help them spread FOMO and FUD and lobby against open source. oh, you aren't? you thought mythos was here to help developers? yea...that's why they didn't give you access.

@TacoJacc · 2026-04-22 04:55
@__tinygrad__ @VictorTaelin Well, send him a tinybox or smn https://t.co/x888KvsZn4

@TacoJacc · 2026-04-22 04:55
@__tinygrad__ @VictorTaelin Well, send him a tinybox or smn https://t.co/x888KvsZn4

@__tinygrad__ · 2026-04-22 04:57
@TacoJacc @VictorTaelin lol they aren't free for us to make. we sell at fair prices for all, nobody gets special discounts. we are antihype, antifomo, antifud we just sell computer.

@__tinygrad__ · 2026-04-22 04:53
@VictorTaelin are you going to help them spread FOMO and FUD and lobby against open source. oh, you aren't? you thought mythos was here to help developers? yea...that's why they didn't give you access.

@VictorTaelin · 2026-04-22 05:03
@__tinygrad__ everyone says it is either compute or marketing, and it may very well be, but knowing the archetype, per hanlon's razor, I think they actually believe it is that dangerous

@__tinygrad__ · 2026-04-22 04:56
@VictorTaelin also, it's not good. if it was good, Claude Code wouldn't be awful software and the uptime of their infra would be good. you see a lot now, if something isn't good it's more profitable to hype it than release it. don't fall for it.

@VictorTaelin · 2026-04-22 05:06
@__tinygrad__ it is fixing massive messy codebases takes much more than good though

@VictorTaelin · 2026-04-22 05:03
@__tinygrad__ everyone says it is either compute or marketing, and it may very well be, but knowing the archetype, per hanlon's razor, I think they actually believe it is that dangerous

@__tinygrad__ · 2026-04-22 05:09
@VictorTaelin remember when GPT-2 XL was that dangerous? anthropic is mostly owned by large US tech companies, there's no serious ideological reason it's just more profitable this way. this is the exact FOMO they want you to feel, don't fall for it.

@VictorTaelin · 2026-04-22 05:06
@__tinygrad__ it is fixing massive messy codebases takes much more than good though

@__tinygrad__ · 2026-04-22 05:11
@VictorTaelin why do you think this? anything besides the cybersecurity bullshit? the AFL fuzzer has found more vulns than mythos ever will, maybe we need to pass laws against open source fuzzers next?

@__tinygrad__ · 2026-04-22 05:26
Can be yours for $65k on a tinybox green v2 Blackwell. On your personal machine the KV cache is never stale and there's never a line or overloading. Order today!

@__tinygrad__ · 2026-04-22 05:26
Can be yours for $65k on a tinybox green v2 Blackwell. On your personal machine the KV cache is never stale and there's never a line or overloading. Order today!

@mhkabirr · 2026-04-22 05:30
@__tinygrad__ Did you guys completely kill the non-Blackwell green v2?

@mhkabirr · 2026-04-22 05:30
@__tinygrad__ Did you guys completely kill the non-Blackwell green v2?

@__tinygrad__ · 2026-04-22 05:33
@mhkabirr Did NVIDIA kill the 5090? We'll bring it back when 5090s are under $3k, otherwise buy a Blackwell. Or an AMD box, the software is decent now.

@0xSero · 2026-04-22 06:03
I’m a big believer in doing everything on your own and use to think Tinygrad was too expensive compared to raw compute costs. Recently I’ve learned a lot about how they’re optimising inference, and hardware topology. used these tricks and 2x the speed of GLM Tinygrad worth it

@0xSero · 2026-04-22 06:03
I’m a big believer in doing everything on your own and use to think Tinygrad was too expensive compared to raw compute costs. Recently I’ve learned a lot about how they’re optimising inference, and hardware topology. used these tricks and 2x the speed of GLM Tinygrad worth it

@__tinygrad__ · 2026-04-22 06:54
@0xSero Thanks! You can code at any level in tinygrad, from Tensor to UOp to CUDA to PTX to ASM and it will all interoperate nicely, so it's never that tinygrad is slow, you just need to code at a lower level.

@__tinygrad__ · 2026-04-22 15:24
We set out to replicate Kimi's 193 tok/s Qwen3.5-0.8B on M3 Max. Our baseline is already 178 tok/s, beating LMStudio (160) and llama.cpp (140) out of the box, but with tinygrad's custom kernel feature Claude cranked it to 195.7! https://t.co/ci1H16FkJP

@__tinygrad__ · 2026-04-22 15:24
We set out to replicate Kimi's 193 tok/s Qwen3.5-0.8B on M3 Max. Our baseline is already 178 tok/s, beating LMStudio (160) and llama.cpp (140) out of the box, but with tinygrad's custom kernel feature Claude cranked it to 195.7! https://t.co/ci1H16FkJP

@__tinygrad__ · 2026-04-22 15:27
Generated code here, we're working on improving the API so both agents and humans love it. We are iterating on this workflow with open models we can host in tinygrad. Recursive self improvement? Agent makes it fast, tinygrad ensures correctness. https://t.co/m2OAbuAPqb

@__tinygrad__ · 2026-04-22 15:24
We set out to replicate Kimi's 193 tok/s Qwen3.5-0.8B on M3 Max. Our baseline is already 178 tok/s, beating LMStudio (160) and llama.cpp (140) out of the box, but with tinygrad's custom kernel feature Claude cranked it to 195.7! https://t.co/ci1H16FkJP

@uuuvn_ · 2026-04-22 15:58
@__tinygrad__ Llama.cpp is a very weak baseline. LMStudio I assume is mlx? It's better but I get 220 tok/s on Qwen3.5-0.8B-MLX-8bit with https://t.co/s6ovysiKqn on m1 max. The branch and command from that pr gets 145 on the same machine but the output is garbage

@davideciffa · 2026-04-22 16:00
I love tinygrad, but with our megakernel you can go to 415 tok/s in decoding speed 🚄

@uuuvn_ · 2026-04-22 15:58
@__tinygrad__ Llama.cpp is a very weak baseline. LMStudio I assume is mlx? It's better but I get 220 tok/s on Qwen3.5-0.8B-MLX-8bit with https://t.co/s6ovysiKqn on m1 max. The branch and command from that pr gets 145 on the same machine but the output is garbage

@uuuvn_ · 2026-04-22 16:03
@__tinygrad__ https://t.co/JWpPSosyHN

@uuuvn_ · 2026-04-22 15:58
@__tinygrad__ Llama.cpp is a very weak baseline. LMStudio I assume is mlx? It's better but I get 220 tok/s on Qwen3.5-0.8B-MLX-8bit with https://t.co/s6ovysiKqn on m1 max. The branch and command from that pr gets 145 on the same machine but the output is garbage

@__tinygrad__ · 2026-04-22 16:15
@uuuvn_ Oh that's not garbage that's just cause it's benchmark, I get the same on master with AMD Strix Halo. Try without benchmark or with --serve for a chat interface. And cool 220 want to port the kernels to tinygrad and beat Claude? What did it miss? https://t.co/F8T5hGb8FO

@uuuvn_ · 2026-04-22 16:03
@__tinygrad__ https://t.co/JWpPSosyHN

@__tinygrad__ · 2026-04-22 16:18
@uuuvn_ Benchmark is expected to output junk. I don't have a Mac but here's how to do the London test. https://t.co/nyHdwViZyr

@haydonryan · 2026-04-22 16:05
@gajesh @__tinygrad__ Bigger models don't respond as well to fusing the gpu kernel, as the CPU component is low in larger models. But totally agree with you - would love to see automated efforts to make larger useful models stronger.

@gajesh · 2026-04-22 16:20
@haydonryan @__tinygrad__ yeah - what’s the point of optimization for things ppl don’t use or want. but i think there are optimisations towards these things. most of them are proprietary - gotta break that. software is cheap. im working on some for larger models - diff from existing direction.

@gajesh · 2026-04-22 16:20
@haydonryan @__tinygrad__ yeah - what’s the point of optimization for things ppl don’t use or want. but i think there are optimisations towards these things. most of them are proprietary - gotta break that. software is cheap. im working on some for larger models - diff from existing direction.

@__tinygrad__ · 2026-04-22 16:22
@gajesh @haydonryan This optimization is for testing optimizations on something fast to iterate on. Our goal isn't to make one model fast, it's to make the tools to make all the models fast. No reason this doesn't work on bigger things.

@davideciffa · 2026-04-22 16:00
I love tinygrad, but with our megakernel you can go to 415 tok/s in decoding speed 🚄

@__tinygrad__ · 2026-04-23 00:53
@davideciffa On a 3090, not an M3 Max! That's like saying I love Lance Armstrong, but with my Ferrari I can go 211 mph 🏎️

@__tinygrad__ · 2026-04-23 04:58
This megakernel is using a 3090. Stock tinygrad beats this (420 tok/sec!) using a cheaper 7900XTX. With our custom driver AMD hardware can really shine. https://t.co/DF8ameXn1u

@__tinygrad__ · 2026-04-23 05:09
Giving us the MI300X boxes marked the turning point. Since then, AMD open sourced their SQTT (low level profiling) format, which is a commitment to open source beyond what I expected. If good decisions like this keep being made, this is just the beginning. Hyped for RDNA5. https://t.co/O9QiNvAEXo

@__tinygrad__ · 2026-04-23 05:09
Giving us the MI300X boxes marked the turning point. Since then, AMD open sourced their SQTT (low level profiling) format, which is a commitment to open source beyond what I expected. If good decisions like this keep being made, this is just the beginning. Hyped for RDNA5. https://t.co/O9QiNvAEXo

@shikharontwt · 2026-04-23 05:24
@__tinygrad__ Come on, the stock didn't go up because of you🤣

@shikharontwt · 2026-04-23 05:24
@__tinygrad__ Come on, the stock didn't go up because of you🤣

@__tinygrad__ · 2026-04-23 05:28
@shikharontwt Of course not, we are just an indicator. It went up because AMD made good decisions, and how you do one thing is how you do everything. https://t.co/y0l2NUEMYW

@dhh · 2026-04-24 07:34
The stack of Omarchy test laptops keep growing! Much appreciation to all the companies that have sent sample units, so we can get perfect out-of-the-box compatibility with as many machines as possible. https://t.co/1IxNpIY5v9

@dhh · 2026-04-24 08:30
@__tinygrad__ I have two Strix Halo desktops, but I don't think it's a great chip for mobile. Way too power and cooling hungry. Besides, Panther Lake is as fast in single core and not that far off in multi-core (17.5K vs 24K on Geekbench).

@__tinygrad__ · 2026-04-24 08:38
@dhh It took tuning, but I got it down to 5W at screen on idle and almost perfect sleep. With 74 Wh battery it's not too bad. But yea the idle power issue is fixable in software if AMD would improve the SMU firmware. The GPU and RAM bandwidth are almost double Panther Lake.

@__tinygrad__ · 2026-04-24 08:38
@dhh It took tuning, but I got it down to 5W at screen on idle and almost perfect sleep. With 74 Wh battery it's not too bad. But yea the idle power issue is fixable in software if AMD would improve the SMU firmware. The GPU and RAM bandwidth are almost double Panther Lake.

@__tinygrad__ · 2026-04-24 08:38
@dhh Running Omarchy btw, this is my main machine.

@__tinygrad__ · 2026-04-24 08:49
@dhh @AMD @AnushElangovan WIth web browsing and normal usage it averages 10W and there's no fan. I'm testing with tinygrad's GPU burn now and even at the 40W package power cap of "balanced" the fan is barely audible, it will get loud in "performance" though.

@__tinygrad__ · 2026-04-24 09:00
@dhh @AMD @AnushElangovan Oh also @HP why does the stupid keyboard backlight draw 2W! Like did anyone care about this? https://t.co/r3RlyIRNsM for the measurement tool so you can replicate this. https://t.co/9Op2t7j9Zm

@__tinygrad__ · 2026-04-24 09:00
@dhh @AMD @AnushElangovan Oh also @HP why does the stupid keyboard backlight draw 2W! Like did anyone care about this? https://t.co/r3RlyIRNsM for the measurement tool so you can replicate this. https://t.co/9Op2t7j9Zm

@__tinygrad__ · 2026-04-24 09:05
@dhh @AMD @AnushElangovan @HP Oh and @dhh while I have you here, this is what I'm looking for in a theme. Maybe Vantablack is moving things in this direction? https://t.co/o0CdkKTAdP

@__tinygrad__ · 2026-04-24 08:44
@dhh It's all about tuning, on the HP with "balanced" power profile I never hear the fans unless I'm compiling something or running an intense model. @AMD @AnushElangovan can you send @dhh a Strix Halo HP ZBook? Don't let Panther Lake win!

@nath_simard · 2026-04-24 09:07
@__tinygrad__ @dhh @AMD @AnushElangovan Did you manage to get the webcam working on Linux? That's the only drawback for the HP imo. Otherwise great laptop, performance mode is great when plugged in, and battery's quite good when unplugged.

@nath_simard · 2026-04-24 09:07
@__tinygrad__ @dhh @AMD @AnushElangovan Did you manage to get the webcam working on Linux? That's the only drawback for the HP imo. Otherwise great laptop, performance mode is great when plugged in, and battery's quite good when unplugged.

@__tinygrad__ · 2026-04-24 09:11
@nath_simard @dhh @AMD @AnushElangovan Nope, actually needed to disable it in BIOS to make deep sleep work. I only ever use webcam at my desk and I have a USB one (single Thunderbolt plug for charging+monitor+USB btw, another must have for a laptop)

@comma_ai · 2026-04-28 21:18
FYI tearing down your comma four doesn't void the warranty :)

@spikedoanz · 2026-04-29 00:08
https://t.co/f1kUC4TTp7 for those interested, some basic steps to run tinygrad with the opencl backend on a ryzen ai max 395+. unfortunately this doesn't use the npu.

@spikedoanz · 2026-04-29 00:08
https://t.co/f1kUC4TTp7 for those interested, some basic steps to run tinygrad with the opencl backend on a ryzen ai max 395+. unfortunately this doesn't use the npu.

@__tinygrad__ · 2026-04-29 21:35
@spikedoanz Why would you use OpenCL? Just use DEV=AMD, it's way faster.

@comma_ai · 2026-04-30 01:12
And we've got a teardown! https://t.co/ht5KMVBoBv https://t.co/uLf3zfL9gZ

@gazorp5 · 2026-04-30 01:16
@comma_ai 8 year old SoC, why? Surely a newer chip would provide more reliable driving performance.

@AlexBowden52 · 2026-04-30 01:23
@gazorp5 @comma_ai It wouldn’t. Because they aren’t even maxing out the current chip. If you really want a massive chip wait for egpu models

@AlexBowden52 · 2026-04-30 01:23
@gazorp5 @comma_ai It wouldn’t. Because they aren’t even maxing out the current chip. If you really want a massive chip wait for egpu models

@__tinygrad__ · 2026-04-30 02:05
@AlexBowden52 @gazorp5 @comma_ai You are limited largely by RAM bandwidth on modern BS=1 models. The 845 has 30 GB/s, even the 8 Gen 5 only has 85 GB/s, just 2.8x. Want to see some power? The 9060XT eGPU has 320 GB/s and good L2 cache speeds unlike mobile chips.

@allbilly01 · 2026-04-30 14:47
I reversed RK3588 NPU registers and integrated to tinygrad. Next, i will document it as detail as my ane repo. Link in comment

@__tinygrad__ · 2026-04-30 16:43
With a consumer eGPU powered by tinygrad plugged into the USB3 port, comma will be the most powerful embedded device in the world. Beyond Tesla HW4, beyond NVIDIA Jetson Thor. When you need a brain for your robot, don't overpay, just use a normal GPU!

@__tinygrad__ · 2026-04-30 16:43
With a consumer eGPU powered by tinygrad plugged into the USB3 port, comma will be the most powerful embedded device in the world. Beyond Tesla HW4, beyond NVIDIA Jetson Thor. When you need a brain for your robot, don't overpay, just use a normal GPU!

@AxcanNathan · 2026-04-30 16:49
@__tinygrad__ where will you get the 350W to power the egpu

@AxcanNathan · 2026-04-30 16:49
@__tinygrad__ where will you get the 350W to power the egpu

@__tinygrad__ · 2026-04-30 17:02
@AxcanNathan Like with the exabox, the user needs to provide the plug. The common path will probably be a midrange GPU power limited to 110W, which pulls from the cigarette lighter jack available on almost all cars and is still competitive with Tesla HW4.

@__tinygrad__ · 2026-04-30 16:58
@allbilly01 Cool! A bunch of cleanup work and test is needed, but it would be a good backend to have.

@allbilly01 · 2026-04-30 17:15
@__tinygrad__ Thanks a lot❤️. The code was before rangeify and I feel far from ready to even post in the Discord. Would you mind to make a Rockchip-hardware channel?

@allbilly01 · 2026-04-30 17:15
@__tinygrad__ Thanks a lot❤️. The code was before rangeify and I feel far from ready to even post in the Discord. Would you mind to make a Rockchip-hardware channel?

@__tinygrad__ · 2026-04-30 17:25
@allbilly01 Done

@RGBiverton · 2026-04-30 17:48
@AxcanNathan @__tinygrad__ Likely targeting 9060 xt and 9070 xt.

@AxcanNathan · 2026-04-30 17:54
@RGBiverton @__tinygrad__ neither of these come close to HW4 at 110W right

@AxcanNathan · 2026-04-30 17:54
@RGBiverton @__tinygrad__ neither of these come close to HW4 at 110W right

@__tinygrad__ · 2026-04-30 19:53
@AxcanNathan @RGBiverton They are close. 160W -> 110W cuts about 20% more off this, but they are still similar. https://t.co/ntAvNlDn3V

@__tinygrad__ · 2026-04-29 21:35
@spikedoanz Why would you use OpenCL? Just use DEV=AMD, it's way faster.

@spikedoanz · 2026-05-01 03:55
@__tinygrad__ mostly because it didn't work for me. https://t.co/Ed4QEkecLx

@trevposts · 2026-05-01 16:13
Forgot about the time Marc said "restricting AI" would be tyranny "far beyond" the gulags and the Holocaust https://t.co/qw2pao7pKo

@spikedoanz · 2026-05-01 03:55
@__tinygrad__ mostly because it didn't work for me. https://t.co/Ed4QEkecLx

@__tinygrad__ · 2026-05-01 17:30
@spikedoanz hmm, it should. if you post more about the error on discord we can look. it's *tons* faster, CL is a shallow backend.

@trevposts · 2026-05-04 20:37
I hope @pmarca is able to safely flee the country now in case the White House does something he said "would impose tyranny far beyond anything even imagined by the Communists and Fascists of the 20th Century" https://t.co/1GzkNdqEmw

@AndrewCurran_ · 2026-05-04 22:21
The Trump administration has informed Anthropic, Google and OpenAI that they are discussing the creation of new AI oversight procedures that would potentially require new AI models to pass a safety review before being cleared for release. Mythos has changed things. https://t.co/fTdLzJGc60

@TheAhmadOsman · 2026-05-05 02:56
People keep treating everything like isolated events - Dario / Anthropic fearmongering - Policy maker pressure - Elon’s lawsuit - Sudden 10x tokens - SF parties All just random coincidences? Come on, look more than 2 steps ahead We’re surrounded by existential risks & Psyops

@zekramu · 2026-05-05 02:59
it will seriously not surprise me if they try to require permits or licenses to use AI, and restrict local model downloads. you really should be buying hardware.

@axios · 2026-05-05 11:37
U.S. ramps up frontier AI testing as White House pivots toward safety https://t.co/WPfTQc4uWD

@__tinygrad__ · 2026-05-05 13:55
A reminder that we have two tinyboxes you can buy today. Bring the power of AI into your home or office. https://t.co/QKy3x8Yh9J

@__tinygrad__ · 2026-05-05 13:55
A reminder that we have two tinyboxes you can buy today. Bring the power of AI into your home or office. https://t.co/QKy3x8Yh9J

@ASvanevik · 2026-05-05 14:04
@__tinygrad__ you literally can’t buy green v2 on your website

@ASvanevik · 2026-05-05 14:04
@__tinygrad__ you literally can’t buy green v2 on your website

@__tinygrad__ · 2026-05-05 14:07
@ASvanevik Huh, why not? Looks like it works to me. (unless you mean the one with 5090s, that's out of stock until 5090s come down in price)

@MTSlive · 2026-05-05 14:11
SITUATION DETECTED: Google, Microsoft and xAI have agreed to give the US government early access to new AI models before public release for national security evaluations, per Bloomberg. OpenAI and Anthropic already had existing agreements and have renegotiated them.

@__tinygrad__ · 2026-05-05 14:07
@ASvanevik Huh, why not? Looks like it works to me. (unless you mean the one with 5090s, that's out of stock until 5090s come down in price)

@ASvanevik · 2026-05-05 14:28
@__tinygrad__ clicking buy now does nothing https://t.co/5pReIGATq6

@ASvanevik · 2026-05-05 14:28
@__tinygrad__ clicking buy now does nothing https://t.co/5pReIGATq6

@__tinygrad__ · 2026-05-05 15:02
@ASvanevik It's strange that it says out of stock, I don't see that in incognito. What region is this? There's export controls on those GPUs, so we are restricted where we can ship them to, maybe it's that?

@__tinygrad__ · 2026-05-05 15:02
@ASvanevik It's strange that it says out of stock, I don't see that in incognito. What region is this? There's export controls on those GPUs, so we are restricted where we can ship them to, maybe it's that?

@ASvanevik · 2026-05-05 15:04
@__tinygrad__ Singapore

@mark_k · 2026-05-05 15:32
We can all thank Anthropic for this situation. They relentlessly fearmongered about the "incredibly dangerous" Mythos model, which was of course an attempt at regulatory capture. And now it succeeded. GFY Anthropic. 🖕

@__tinygrad__ · 2026-05-05 13:55
A reminder that we have two tinyboxes you can buy today. Bring the power of AI into your home or office. https://t.co/QKy3x8Yh9J

@Eziowl · 2026-05-05 16:04
@__tinygrad__ Can green v2 blackwell be made to order using RTX PRO 6000 max-Qs or just the standard RTX PRO 6000s

@ASvanevik · 2026-05-05 15:04
@__tinygrad__ Singapore

@__tinygrad__ · 2026-05-05 16:59
@ASvanevik Yea, export controls is the issue. We obviously aren't gonna submit some stupid license paperwork, this is the stupidest move by the US government, but since they have the guns we respect it. I'll see if we can update it so it doesn't just say out of stock. https://t.co/kCjODTVb6t

@ASvanevik · 2026-05-05 15:04
@__tinygrad__ Singapore

@__tinygrad__ · 2026-05-05 17:00
TIL why the Blackwell box is out of stock in Singapore. It's too powerful! Get yours before the regime clamps down further.

@Eziowl · 2026-05-05 16:04
@__tinygrad__ Can green v2 blackwell be made to order using RTX PRO 6000 max-Qs or just the standard RTX PRO 6000s

@__tinygrad__ · 2026-05-05 17:02
@Eziowl In order to keep prices low and quality high, we don't offer any customization to the box or ordering process. You can limit the power yourself once you get it, the max-Q is a ripoff where NVIDIA packages worse chips and convinces you it's a feature.

@mark_k · 2026-05-05 15:32
We can all thank Anthropic for this situation. They relentlessly fearmongered about the "incredibly dangerous" Mythos model, which was of course an attempt at regulatory capture. And now it succeeded. GFY Anthropic. 🖕

@__tinygrad__ · 2026-05-05 17:13
@mark_k .@DarioAmodei are you excited for the day when an open source Chinese model beats your top one? this is what you lobbied for. big win for open source and big win for Chinese cultural influence.

@AndrewCurran_ · 2026-05-04 22:21
The Trump administration has informed Anthropic, Google and OpenAI that they are discussing the creation of new AI oversight procedures that would potentially require new AI models to pass a safety review before being cleared for release. Mythos has changed things. https://t.co/fTdLzJGc60

@__tinygrad__ · 2026-05-05 17:24
@AndrewCurran_ GPT-2 XL is too dangerous, Claude Mythos is too dangerous. crypto over 40-bits is too dangerous. the difference between this time and the crypto wars is you don't just hold back progress, you hand cultural influence to the Chinese. hope that's what these people wanted.

@__tinygrad__ · 2026-05-05 17:16
@TheAhmadOsman lol remember the time they regulated crypto over 40 bits? NO NO THATS TOO MANY BITS FOR THE PEOPLE. they were losers then and they'll be losers again. the difference this time is that it will cost them cultural influence to the Chinese.

@TheAhmadOsman · 2026-05-05 17:25
@__tinygrad__ Gonna be a self-own that’s looked back on in the history books as the fumble of the century

@TheAhmadOsman · 2026-05-05 17:25
@__tinygrad__ Gonna be a self-own that’s looked back on in the history books as the fumble of the century

@__tinygrad__ · 2026-05-05 17:28
@TheAhmadOsman Jensen tried to warn them. The founder of the largest company in history tried to warn them. I can't imagine literal Chinese state assets doing a better job than the US government at promoting Huawei.

@zekramu · 2026-05-05 02:59
it will seriously not surprise me if they try to require permits or licenses to use AI, and restrict local model downloads. you really should be buying hardware.

@__tinygrad__ · 2026-05-05 17:33
@zekramu Ahh maybe they can enlist the MPAA and RIAA to help with the restricting of downloads. I can see the lawsuit against the grandma who downloaded Qwen already🤣

@__tinygrad__ · 2026-05-05 17:09
@axios It'll be nice when all of America is using Chinese open source models.

@jmbollenbacher · 2026-05-05 17:35
@__tinygrad__ @axios Theyre gonna try to ban those too eventually. They banned Chinese phones and Chinese cars and Chinese telecom chips. As soon as they figure out how to ban Chinese model weights theyll try to do it.

@jmbollenbacher · 2026-05-05 17:35
@__tinygrad__ @axios Theyre gonna try to ban those too eventually. They banned Chinese phones and Chinese cars and Chinese telecom chips. As soon as they figure out how to ban Chinese model weights theyll try to do it.

@__tinygrad__ · 2026-05-05 17:36
@jmbollenbacher @axios ahh have they considered a great firewall? they might be able to license one from China

@__tinygrad__ · 2026-05-05 17:33
@zekramu Ahh maybe they can enlist the MPAA and RIAA to help with the restricting of downloads. I can see the lawsuit against the grandma who downloaded Qwen already🤣

@zekramu · 2026-05-05 17:36
@__tinygrad__ the us governtment specializes in trying to regulate shit they cant really regulate 😭😭

@jmbollenbacher · 2026-05-05 17:35
@__tinygrad__ @axios Theyre gonna try to ban those too eventually. They banned Chinese phones and Chinese cars and Chinese telecom chips. As soon as they figure out how to ban Chinese model weights theyll try to do it.

@jmbollenbacher · 2026-05-05 17:37
@__tinygrad__ @axios And i think it's fairly clear how. They'll coerce the hyperscalers to not allow anyone using their servers to run or distribute Chinese AI. The leverage is their govt contracts and "supply chain risk" designstions. This will create takedowns on HuggingFace and OpenRouter, etc.

@zekramu · 2026-05-05 17:36
@__tinygrad__ the us governtment specializes in trying to regulate shit they cant really regulate 😭😭

@__tinygrad__ · 2026-05-05 17:37
@zekramu but they can make Huawei execs very very happy by banning the export of NVIDIA, so there's that. think of the happy Huawei execs.

@jmbollenbacher · 2026-05-05 17:37
@__tinygrad__ @axios And i think it's fairly clear how. They'll coerce the hyperscalers to not allow anyone using their servers to run or distribute Chinese AI. The leverage is their govt contracts and "supply chain risk" designstions. This will create takedowns on HuggingFace and OpenRouter, etc.

@__tinygrad__ · 2026-05-05 17:40
@jmbollenbacher @axios this won't matter. there's a narrow window where the US has a lead. with the NVIDIA export ban, they handed Huawei all the world's AI infrastructure. this stuff only works when the world buys US. like i should be happy with how much this helps local AI, but it's so sad to see.

@trevposts · 2026-05-04 20:37
I hope @pmarca is able to safely flee the country now in case the White House does something he said "would impose tyranny far beyond anything even imagined by the Communists and Fascists of the 20th Century" https://t.co/1GzkNdqEmw

@__tinygrad__ · 2026-05-05 17:45
@trevposts @pmarca lol it's not gonna be tyranny unless you consider increasing Chinese cultural influence to be tyranny. cause that's exactly what these policies will do. hope that's what you wanted.

@__tinygrad__ · 2026-05-05 13:55
A reminder that we have two tinyboxes you can buy today. Bring the power of AI into your home or office. https://t.co/QKy3x8Yh9J

@zekramu · 2026-05-05 17:56
@__tinygrad__ the value on the green v2 is lowkey insane wtf ??

@__tinygrad__ · 2026-05-05 17:13
@mark_k .@DarioAmodei are you excited for the day when an open source Chinese model beats your top one? this is what you lobbied for. big win for open source and big win for Chinese cultural influence.

@marcospereeira · 2026-05-05 18:30
@__tinygrad__ @mark_k @DarioAmodei how so? chinese models would have a much harder time entering the corporate world if this legislation goes into action

@marcospereeira · 2026-05-05 18:30
@__tinygrad__ @mark_k @DarioAmodei how so? chinese models would have a much harder time entering the corporate world if this legislation goes into action

@__tinygrad__ · 2026-05-05 18:34
@marcospereeira @mark_k @DarioAmodei ooo are you telling me this will also nerf large rent extracting US corporations and favor the small ones that use the chinese models without some dumb corporate process. this is sounding better and better.

@zekramu · 2026-05-05 17:56
@__tinygrad__ the value on the green v2 is lowkey insane wtf ??

@__tinygrad__ · 2026-05-05 18:38
@zekramu Yea you can run the open source frontier on it at reasonable speeds, probably the cheapest way to do it.

@sudoingX · 2026-05-06 07:23
if you want mac portability and you want to learn cuda, the dgx spark is the silent king nobody is talking about. 128gb unified memory in a form factor that fits on a desk corner, full cuda stack, runs nemotron 30b q8 at 56 tok/s on hermes agent, multimodal + tool calls nobody has written custom kernels for this specific silicon yet. spark has its own architecture (gb10 blackwell, aarch64), the whole ecosystem of model-specific kernel work for 3090 / 4090 / 5090 has not been ported here. that is an openlane for builders who want the territory. i expect nvidia to focus on its ecosystem more this year. the hardware is in front of builders, the software needs to catch up to make spark the developer-default for portable ai workstations. if you have one and you have not written or tested anything model-specific on it yet, you are sitting on the most underexplored consumer AI silicon shipping right now.

@benitoz · 2026-05-07 05:11
Today Dario admits that Anthropic only planned for 10x growth but got hit with 80x instead Internally called a “success disaster” Their compute effectively is off by a factor of 8x or more Now do the outages, rate limits, nerfed performance make sense? We need more compute! https://t.co/I4LAaJ10pf

@AlecStapp · 2026-05-07 12:29
I ask again: Why is the US government allowing NVIDIA to sell chips to China while American AI labs are starved for compute?

@allen_ai · 2026-05-07 15:03
Today we’re bringing new NSF OMAI compute online with NVIDIA Blackwell Ultra-powered systems, turning a $152M national investment from @NSF & @NVIDIA into a foundation for truly open AI research. 🧵 https://t.co/qFgtiibgAK

@wccftech · 2026-05-07 15:56
AMD launches MI350P, its first PCIe "Instinct" in four years – packs CDNA 4 GPU with 4.6 PFLOPs AI compute, 144 GB HBM3E at 600W. https://t.co/uLAh7eokph

@__tinygrad__ · 2026-05-07 17:51
@dougvk @sudoingX We should, minor changes if we don't. I have seen PRs for it, not sure if they were merged.

@Bencera · 2026-05-07 18:39
AGI is here. Capitalism is ending soon. The bar for shipping velocity is inhuman. Everyone is racing to build final companies. May the best win.

@andrewchen · 2026-05-07 20:32
Trying /goal for the first time on Codex and it’s obv it’s going to 10000x token use. It’s amazing though - I’ve had it working on a low level eGPU+Mac device driver project overnight (that I have no business doing) for the past 14 hours and it’s still chipping away making progress with each iteration Naturally unattended 24/7 LLM use will be several magnitudes more than me prompting actively over a normal work day

@__tinygrad__ · 2026-05-08 01:56
The tinygrad spec is now merged in tinygrad/spec. Unlike every other ML compiler, all optimization is done in this IR all the way up to instruction selection. https://t.co/kQPpt0yzSC

@allen_ai · 2026-05-07 15:03
Today we’re bringing new NSF OMAI compute online with NVIDIA Blackwell Ultra-powered systems, turning a $152M national investment from @NSF & @NVIDIA into a foundation for truly open AI research. 🧵 https://t.co/qFgtiibgAK

@RichardKCollin2 · 2026-05-08 02:05
@allen_ai @NSF @nvidia So can I use it for free? If it is "truly open" AI research? There are 8.3 Billion humans now - that is open, not just for a lucky few. I am Richard Collins, The Internet Foundation asking. https://t.co/VIr3YYVC08

@__tinygrad__ · 2026-05-08 01:56
The tinygrad spec is now merged in tinygrad/spec. Unlike every other ML compiler, all optimization is done in this IR all the way up to instruction selection. https://t.co/kQPpt0yzSC

@GaemaAI · 2026-05-08 02:06
@__tinygrad__ Are you going to adopt tiles?

@GaemaAI · 2026-05-08 02:06
@__tinygrad__ Are you going to adopt tiles?

@__tinygrad__ · 2026-05-08 02:19
@GaemaAI you're not thinking nth-dimensionally. we already have something a lot better than tiles.

@__tinygrad__ · 2026-05-08 02:21
32GB of RDNA4 on USB 3.2 Gen 2. Like the dock? https://t.co/DveGnWZ2oT

@__tinygrad__ · 2026-05-08 01:56
The tinygrad spec is now merged in tinygrad/spec. Unlike every other ML compiler, all optimization is done in this IR all the way up to instruction selection. https://t.co/kQPpt0yzSC

@mungerscigbutt · 2026-05-08 02:21
@__tinygrad__ why no tcgen05 emisson for normal kernels? am i 2 noob

@mungerscigbutt · 2026-05-08 02:21
@__tinygrad__ why no tcgen05 emisson for normal kernels? am i 2 noob

@__tinygrad__ · 2026-05-08 02:23
@mungerscigbutt We are focused more on datacenter AMD than datacenter NVIDIA. We don't own any B200s.

@ASvanevik · 2026-05-08 02:26
wtf I can literally buy RTX Pro 6000s at Sim Lim Sq 😅

@__tinygrad__ · 2026-05-08 02:27
Any market for a 25k box with 6 32GB RDNA4 GPUs on full fabric PCIe 5? We'll build it if we get two orders. Also, we're going to put the 5090 boxes back in stock once we calculate what we are paying for the parts (+20% markup). Our current price for 5090s is $4200 afaik.

@__tinygrad__ · 2026-05-08 02:21
32GB of RDNA4 on USB 3.2 Gen 2. Like the dock? https://t.co/DveGnWZ2oT

@Its_keith_d · 2026-05-08 02:28
@__tinygrad__ Does this mean we get a quad R9700 tiny box??

@__tinygrad__ · 2026-05-08 02:22
@wccftech what's price?

@LottoLabs · 2026-05-08 02:29
@__tinygrad__ @wccftech Rumours are 25k usd

@Its_keith_d · 2026-05-08 02:28
@__tinygrad__ Does this mean we get a quad R9700 tiny box??

@__tinygrad__ · 2026-05-08 02:29
@Its_keith_d We can do one with 6 if there's interest at 25k price point.

@LottoLabs · 2026-05-08 02:29
@__tinygrad__ @wccftech Rumours are 25k usd

@__tinygrad__ · 2026-05-08 02:30
@LottoLabs @wccftech Not worth it. 4x 5090 is a far better deal, even 2x 96GB blackwell.

@__tinygrad__ · 2026-05-08 02:31
@sinnformer You haven't seen it anodized black yet...

@__tinygrad__ · 2026-05-08 02:22
@wccftech what's price?

@SIGKITTEN · 2026-05-08 02:32
@__tinygrad__ @wccftech how tf do u not have a side channel with them where they just ship u the stuff hot off the oven i dont get it. wtf is amd doing

@SIGKITTEN · 2026-05-08 02:32
@__tinygrad__ @wccftech how tf do u not have a side channel with them where they just ship u the stuff hot off the oven i dont get it. wtf is amd doing

@__tinygrad__ · 2026-05-08 02:33
@SIGKITTEN @wccftech .@AnushElangovan we'll get it working on the USB dock if you want send us two of them. https://t.co/Z0qjX3f1Gt

@__tinygrad__ · 2026-05-08 02:22
@wccftech what's price?

@catphonics · 2026-05-08 02:33
@__tinygrad__ @wccftech If MI350X is now $25k... Guessing this is the card to slot into that former $15k pricepoint. It should be $8k to be a competitive deal vs the $9-10k Blackwell, but seems unlikely given the recent hikes

@__tinygrad__ · 2026-05-08 02:28
@ASvanevik We're as mad as you. The whole thing is so dumb, but we had to promise suppliers to abide who I'm sure had to promise NVIDIA.

@ASvanevik · 2026-05-08 02:34
@__tinygrad__ can I buy the GPUs locally and plug them in myself? but everything else is set up in the box?

@catphonics · 2026-05-08 02:33
@__tinygrad__ @wccftech If MI350X is now $25k... Guessing this is the card to slot into that former $15k pricepoint. It should be $8k to be a competitive deal vs the $9-10k Blackwell, but seems unlikely given the recent hikes

@__tinygrad__ · 2026-05-08 02:34
@catphonics @wccftech If this is competitively priced with the Blackwell, it would be a smash hit.

@ASvanevik · 2026-05-08 02:34
@__tinygrad__ can I buy the GPUs locally and plug them in myself? but everything else is set up in the box?

@__tinygrad__ · 2026-05-08 02:38
@ASvanevik I'm fine with this. Are you serious about ordering? Use discount code NOGPUSINCLUDED and get $40k off. We'll make sure we test it the same with the GPUs, and all mounting hardware will be included, we'll just pull them before we ship.

@__tinygrad__ · 2026-05-08 02:33
@SIGKITTEN @wccftech .@AnushElangovan we'll get it working on the USB dock if you want send us two of them. https://t.co/Z0qjX3f1Gt

@AnushElangovan · 2026-05-08 03:13
@__tinygrad__ @SIGKITTEN @wccftech Will do. Is a 600w card ok ? Is passive cooled ok ? If not I'll send you one with an external blower.

@__tinygrad__ · 2026-05-08 02:33
@SIGKITTEN @wccftech .@AnushElangovan we'll get it working on the USB dock if you want send us two of them. https://t.co/Z0qjX3f1Gt

@AnushElangovan · 2026-05-08 03:13
@__tinygrad__ @SIGKITTEN @wccftech Will do. Is a 600w card ok ? Is passive cooled ok ? If not I'll send you one with an external blower.

@thehenryinsf · 2026-05-08 06:33
@andrewchen what's the biggest bottleneck you've hit with it running for 14 hours straight?

@andrewchen · 2026-05-08 06:40
Here’s the summary. I mostly don’t understand what’s going on 😂 — We’re trying to make an **AORUS RTX 5090 eGPU work on a Mac mini through TinyGPU/tinygrad**. This is experimental: macOS has no normal NVIDIA/CUDA path, so TinyGPU is talking to the card through low-level NVIDIA/GSP/RM mechanisms. High-level status: > The Mac sees the RTX 5090, TinyGPU can enumerate it, and tinygrad can now get far enough to allocate and schedule a GPU command channel. But when we submit the tiniest possible “write this signal value” command, the GPU never completes it. Why it took a day of Codex looping: - The failure started as “nothing works”: wrong device selection, all-ones MMIO reads, page-table/BAR issues, Docker/nvcc/user mismatch, reset-wedging. - Each fix moved the failure deeper: 1. card enumerates 2. correct PCI function selected 3. page-table setup gets past early asserts 4. GSP/RM init works farther 5. channels allocate 6. scheduling actually succeeds 7. PBDMA sees/picks the channel 8. but command execution still stalls - The hard part is that NVIDIA queue submission is very stateful and underdocumented. A wrong token, doorbell, USERD page, channel flag, scheduler param, or memory aperture can all look like the same symptom: “GPU didn’t run.” - Many plausible fixes had to be tried and ruled out one by one: doorbell offsets, work-submit tokens, notifier slots, USERD placement, GPFIFO class variants, ctxshare, UVM-owned flags, retainer objects, semaphore formats, WFI barriers, runlist nudges, etc. - Failed low-level attempts can dirty the GPU firmware state, so clean conclusions often require reboot/reset boundaries. Current distilled blocker: > We can get a Blackwell RTX 5090 channel allocated and scheduled, and after submit the hardware shows the channel reaching PBDMA, but it never reaches the engine or completes even a trivial host-semaphore command. So the remaining bug is likely in Blackwell PBDMA/ESCHED/runlist/channel-activation/fetch-context semantics — not basic Mac detection or compiler setup.

@AnushElangovan · 2026-05-08 03:13
@__tinygrad__ @SIGKITTEN @wccftech Will do. Is a 600w card ok ? Is passive cooled ok ? If not I'll send you one with an external blower.

@__tinygrad__ · 2026-05-08 16:40
@AnushElangovan @SIGKITTEN @wccftech 600W is fine, it's powered off an ATX PSU. And yea passively cooled is fine, we have blowers we can attach.

@andrewchen · 2026-05-08 06:40
Here’s the summary. I mostly don’t understand what’s going on 😂 — We’re trying to make an **AORUS RTX 5090 eGPU work on a Mac mini through TinyGPU/tinygrad**. This is experimental: macOS has no normal NVIDIA/CUDA path, so TinyGPU is talking to the card through low-level NVIDIA/GSP/RM mechanisms. High-level status: > The Mac sees the RTX 5090, TinyGPU can enumerate it, and tinygrad can now get far enough to allocate and schedule a GPU command channel. But when we submit the tiniest possible “write this signal value” command, the GPU never completes it. Why it took a day of Codex looping: - The failure started as “nothing works”: wrong device selection, all-ones MMIO reads, page-table/BAR issues, Docker/nvcc/user mismatch, reset-wedging. - Each fix moved the failure deeper: 1. card enumerates 2. correct PCI function selected 3. page-table setup gets past early asserts 4. GSP/RM init works farther 5. channels allocate 6. scheduling actually succeeds 7. PBDMA sees/picks the channel 8. but command execution still stalls - The hard part is that NVIDIA queue submission is very stateful and underdocumented. A wrong token, doorbell, USERD page, channel flag, scheduler param, or memory aperture can all look like the same symptom: “GPU didn’t run.” - Many plausible fixes had to be tried and ruled out one by one: doorbell offsets, work-submit tokens, notifier slots, USERD placement, GPFIFO class variants, ctxshare, UVM-owned flags, retainer objects, semaphore formats, WFI barriers, runlist nudges, etc. - Failed low-level attempts can dirty the GPU firmware state, so clean conclusions often require reboot/reset boundaries. Current distilled blocker: > We can get a Blackwell RTX 5090 channel allocated and scheduled, and after submit the hardware shows the channel reaching PBDMA, but it never reaches the engine or completes even a trivial host-semaphore command. So the remaining bug is likely in Blackwell PBDMA/ESCHED/runlist/channel-activation/fetch-context semantics — not basic Mac detection or compiler setup.

@__tinygrad__ · 2026-05-08 17:04
@andrewchen @thehenryinsf Hmm, this should just work out of the box if it's a normal 5090.

@allen_ai · 2026-05-07 15:03
Today we’re bringing new NSF OMAI compute online with NVIDIA Blackwell Ultra-powered systems, turning a $152M national investment from @NSF & @NVIDIA into a foundation for truly open AI research. 🧵 https://t.co/qFgtiibgAK

@__tinygrad__ · 2026-05-08 17:18
@allen_ai @NSF @nvidia Love seeing you guys get compute!

@curl_justin · 2026-05-08 17:21
I gave the same lecture to American and Chinese law students, polling them about where they fell on this graph. At Yale Law School (n=~60), the majority of students believed AI would be unlike any technology we’ve seen before and likely net bad for society. They were in the bottom right. At Renmin University (n=~30), every one of the students viewed AI as analogous to past technological transformations with its benefits likely outweighing its harms. They were in the top left. At first glance, this suggests Americans and Chinese have fundamentally different understandings of AI. But in the Q&A, students explained they were top left because they were skeptical of model capabilities. They used Doubao-Seed-2.0—a ByteDance model a bit worse than GPT-5.2 on benchmarks—and felt it hallucinated too often for them to be worried much about frontier risks. In other words, the Chinese students weren’t deprioritizing safety because they have a different understanding of AI progress. They deprioritized it because they hadn’t used a model that made safety feel urgent. Obviously we shouldn't over-read from one incredibly non-representative sample, but it made me wonder how much apparent disagreements between US and China about AI safety are really disagreements about AI capabilities.

@__tinygrad__ · 2026-05-08 17:04
@andrewchen @thehenryinsf Hmm, this should just work out of the box if it's a normal 5090.

@andrewchen · 2026-05-08 17:38
@__tinygrad__ @thehenryinsf I just got it! Just a normal 5090 and it works on windows I’ll work on it for a few days and hit your discord if not solved in a bit

@andrewchen · 2026-05-08 17:38
@__tinygrad__ @thehenryinsf I just got it! Just a normal 5090 and it works on windows I’ll work on it for a few days and hit your discord if not solved in a bit

@__tinygrad__ · 2026-05-08 17:58
@andrewchen @thehenryinsf So there's a chance it's the dock. We're going to sell a dock soon, and that we'll be able to investigate issues on, but it's hard with all the random docks.

@__tinygrad__ · 2026-05-08 01:56
The tinygrad spec is now merged in tinygrad/spec. Unlike every other ML compiler, all optimization is done in this IR all the way up to instruction selection. https://t.co/kQPpt0yzSC

@_Suresh2 · 2026-05-08 18:01
@__tinygrad__ does instruction selection still see the original op shapes or is it all flattened by then

@RichardKCollin2 · 2026-05-08 02:05
@allen_ai @NSF @nvidia So can I use it for free? If it is "truly open" AI research? There are 8.3 Billion humans now - that is open, not just for a lucky few. I am Richard Collins, The Internet Foundation asking. https://t.co/VIr3YYVC08

@__tinygrad__ · 2026-05-08 18:04
@RichardKCollin2 @allen_ai @NSF @nvidia Not only can you download the trained models, you can download the training code and all the intermediate steps too. This is one of the most open labs in the world. I would much rather they have it and continue to push to the frontier. The Chinese training runs aren't that large.

@_Suresh2 · 2026-05-08 18:01
@__tinygrad__ does instruction selection still see the original op shapes or is it all flattened by then

@__tinygrad__ · 2026-05-08 18:06
@_Suresh2 Currently there's a "vectorized dtype" but we are working on replacing that with just shape. If you add a range (loop) that range is removed from the shape. You can inspect all this with VIZ=1

@SenTomCotton · 2026-05-08 18:07
One of the foreign companies smuggling US chips is OBON, which stands for One Belt, One Network. Call me crazy, but maybe American companies shouldn’t sell advanced technologies to companies literally named after one of Xi Jinping’s signature policies. Super Micro should account for how this could’ve happened. https://t.co/amcY7QdjBy

@__tinygrad__ · 2026-05-08 18:14
@Bencera The only people who think AI is really good at things is people who aren't that good at things themselves.

@caseycantor_ · 2026-05-08 19:04
@__tinygrad__ @Bencera At a certain point contrarian takes like this signal dishonesty or a lack of common sense

@__tinygrad__ · 2026-05-08 18:14
@Bencera The only people who think AI is really good at things is people who aren't that good at things themselves.

@paul_p42 · 2026-05-08 20:32
@__tinygrad__ @Bencera While I used to take a lot of pride in the clever abstractions I would come up with, and pride in beautifully engineered code None of it made a difference to the users: uptime, features and security made all the difference Using AI properly can absolutely help deliver that

@__tinygrad__ · 2026-05-08 17:06
@AlecStapp Allowing? I thought we lived in a free country.

@bluntsingh108 · 2026-05-08 20:36
@__tinygrad__ @AlecStapp Will you sell ammo to country you are fighting

@caseycantor_ · 2026-05-08 19:04
@__tinygrad__ @Bencera At a certain point contrarian takes like this signal dishonesty or a lack of common sense

@__tinygrad__ · 2026-05-08 21:21
@tempo511 @Bencera I think you don't understand how much we wish this wasn't true. I cannot wait until machines can do the work of great software engineers, there's so so so much work to be done. And I don't doubt they will get there, you are just wrong if you think they are there yet.

@paul_p42 · 2026-05-08 20:32
@__tinygrad__ @Bencera While I used to take a lot of pride in the clever abstractions I would come up with, and pride in beautifully engineered code None of it made a difference to the users: uptime, features and security made all the difference Using AI properly can absolutely help deliver that

@__tinygrad__ · 2026-05-08 21:22
@paul_p42 @Bencera lol that's why services have had such good uptime and security recently...

@__tinygrad__ · 2026-05-08 22:34
@jakehendersonx_ @Bencera The US economy depends on this being true.

@SenTomCotton · 2026-05-08 18:07
One of the foreign companies smuggling US chips is OBON, which stands for One Belt, One Network. Call me crazy, but maybe American companies shouldn’t sell advanced technologies to companies literally named after one of Xi Jinping’s signature policies. Super Micro should account for how this could’ve happened. https://t.co/amcY7QdjBy

@__tinygrad__ · 2026-05-08 22:37
@SenTomCotton Or you can just like, believe in freedom and let NVIDIA and SuperMicro sell the chips to whoever they want. This is huge money for the US and great for reducing the trade deficit.

@curl_justin · 2026-05-08 17:21
I gave the same lecture to American and Chinese law students, polling them about where they fell on this graph. At Yale Law School (n=~60), the majority of students believed AI would be unlike any technology we’ve seen before and likely net bad for society. They were in the bottom right. At Renmin University (n=~30), every one of the students viewed AI as analogous to past technological transformations with its benefits likely outweighing its harms. They were in the top left. At first glance, this suggests Americans and Chinese have fundamentally different understandings of AI. But in the Q&A, students explained they were top left because they were skeptical of model capabilities. They used Doubao-Seed-2.0—a ByteDance model a bit worse than GPT-5.2 on benchmarks—and felt it hallucinated too often for them to be worried much about frontier risks. In other words, the Chinese students weren’t deprioritizing safety because they have a different understanding of AI progress. They deprioritized it because they hadn’t used a model that made safety feel urgent. Obviously we shouldn't over-read from one incredibly non-representative sample, but it made me wonder how much apparent disagreements between US and China about AI safety are really disagreements about AI capabilities.

@__tinygrad__ · 2026-05-08 22:50
@curl_justin The US has believed in this AI apocalypse long before frontier model X came out. The Chinese view is basically correct.

@bluntsingh108 · 2026-05-08 20:36
@__tinygrad__ @AlecStapp Will you sell ammo to country you are fighting

@__tinygrad__ · 2026-05-08 23:17
@bluntsingh108 @AlecStapp Will you sell computers (at a huge profit) to a country you are largely friendly with?

@METR_Evals · 2026-05-08 23:41
We evaluated an early version of Claude Mythos Preview for risk assessment during a limited window in March 2026. We estimated a 50%-time-horizon of at least 16hrs (95% CI 8.5hrs to 55hrs) on our task suite, at the upper end of what we can measure without new tasks. https://t.co/yIG1Ux27Ro

@__tinygrad__ · 2026-05-09 02:36
@sterlingcrispin It's funny how truth is still truth regardless of what you are selling. I think a lot of people have forgotten that these days.

@METR_Evals · 2026-05-08 23:41
We evaluated an early version of Claude Mythos Preview for risk assessment during a limited window in March 2026. We estimated a 50%-time-horizon of at least 16hrs (95% CI 8.5hrs to 55hrs) on our task suite, at the upper end of what we can measure without new tasks. https://t.co/yIG1Ux27Ro

@__tinygrad__ · 2026-05-09 02:38
@METR_Evals Where is GPT 5.5?

@ChaseBrowe32432 · 2026-05-09 17:05
Mythos lands slightly above the trendline for the AI 2027 scenario https://t.co/RkDabqNFdA

@ChaseBrowe32432 · 2026-05-09 17:05
Mythos lands slightly above the trendline for the AI 2027 scenario https://t.co/RkDabqNFdA

@__tinygrad__ · 2026-05-09 18:52
@ChaseBrowe32432 people see 3 datapoints and fit an exponential. it's an s-curve. every technology is an s-curve. we ran ahead of moore's law mostly due to dtypes.

@__tinygrad__ · 2026-05-09 18:58
Can we get rid of all the AI hype? Normally I find hype just distasteful, but this is distasteful, filling people with dread, and making hardware expensive. The third one is really annoying. It's just Moore's law and a bubble, there's no singularity.

@__tinygrad__ · 2026-05-09 18:58
Can we get rid of all the AI hype? Normally I find hype just distasteful, but this is distasteful, filling people with dread, and making hardware expensive. The third one is really annoying. It's just Moore's law and a bubble, there's no singularity.

@__tinygrad__ · 2026-05-09 19:01
Actually it's even worse than the Moore's law about cost cause things aren't getting cheaper! A lot of the AI gains in the last 10 years come from increased spending and smaller dtypes, both of which are nearing an end and we'll be back to the normal Moore's trajectory.

@__tinygrad__ · 2026-05-09 18:58
Can we get rid of all the AI hype? Normally I find hype just distasteful, but this is distasteful, filling people with dread, and making hardware expensive. The third one is really annoying. It's just Moore's law and a bubble, there's no singularity.

@allinasecond · 2026-05-09 19:01
@__tinygrad__ What do you use to code/build nowadays? Just plain Claude Code with Opus 4.7?

@__tinygrad__ · 2026-05-09 18:58
Can we get rid of all the AI hype? Normally I find hype just distasteful, but this is distasteful, filling people with dread, and making hardware expensive. The third one is really annoying. It's just Moore's law and a bubble, there's no singularity.

@RGBiverton · 2026-05-09 19:01
@__tinygrad__ Need more stock tips.

@allinasecond · 2026-05-09 19:01
@__tinygrad__ What do you use to code/build nowadays? Just plain Claude Code with Opus 4.7?

@__tinygrad__ · 2026-05-09 19:02
@allinasecond GPT 5.5 is a lot better, in many ways it's Mythos without the repulsive marketing. If you insist on Claude, Opus 4.6 is better than 4.7.

@__tinygrad__ · 2026-05-09 18:58
Can we get rid of all the AI hype? Normally I find hype just distasteful, but this is distasteful, filling people with dread, and making hardware expensive. The third one is really annoying. It's just Moore's law and a bubble, there's no singularity.

@Nairebis · 2026-05-09 19:02
@__tinygrad__ You are sadly out of the loop of the very real revolution going on. Didn't expect to see "steam engines are just a fad, horses are forever" on this account.

@__tinygrad__ · 2026-05-09 18:58
Can we get rid of all the AI hype? Normally I find hype just distasteful, but this is distasteful, filling people with dread, and making hardware expensive. The third one is really annoying. It's just Moore's law and a bubble, there's no singularity.

@graykevinb · 2026-05-09 19:03
@__tinygrad__ Really? https://t.co/xe66S76DWP

@Nairebis · 2026-05-09 19:02
@__tinygrad__ You are sadly out of the loop of the very real revolution going on. Didn't expect to see "steam engines are just a fad, horses are forever" on this account.

@__tinygrad__ · 2026-05-09 19:04
@Nairebis That's not what I'm saying at all, AI is both cool and transformative, just like steam engines. But did the steam engine make the economy go vertical? Did the steam engine make everyone lose their jobs? Should we pay 5x more for steel now that we have steam engines?

@__tinygrad__ · 2026-05-09 18:58
Can we get rid of all the AI hype? Normally I find hype just distasteful, but this is distasteful, filling people with dread, and making hardware expensive. The third one is really annoying. It's just Moore's law and a bubble, there's no singularity.

@EPlCs · 2026-05-09 19:05
@__tinygrad__ It's not a bubble. At least in financial terms its not. Anthropic and Openai will have more than a $100B ARR combined by the end of 2026. That's not a bubble.

@__tinygrad__ · 2026-05-09 19:04
@Nairebis That's not what I'm saying at all, AI is both cool and transformative, just like steam engines. But did the steam engine make the economy go vertical? Did the steam engine make everyone lose their jobs? Should we pay 5x more for steel now that we have steam engines?

@Nairebis · 2026-05-09 19:09
@__tinygrad__ 1) Yes. 2) No, and neither will AI, just like 1 steam shovel replacing 50 ditch diggers didn't end construction jobs. Instead, they made Seattle possible. Jevons Paradox, baby. 3) Supply follows demand, like it always has. Annoying shortages are temporary under Capitalism.

@__tinygrad__ · 2026-05-09 18:58
Can we get rid of all the AI hype? Normally I find hype just distasteful, but this is distasteful, filling people with dread, and making hardware expensive. The third one is really annoying. It's just Moore's law and a bubble, there's no singularity.

@jeffcafe_ · 2026-05-09 19:10
@__tinygrad__ If it’s a bubble, you just need to wait until it pops and your hardware will get much cheaper.

@RGBiverton · 2026-05-09 19:01
@__tinygrad__ Need more stock tips.

@__tinygrad__ · 2026-05-09 19:10
@RGBiverton There's still almost a 10x gap between NVDA and AMD. What happens to that gap if the petaflop is commoditized?

@EPlCs · 2026-05-09 19:05
@__tinygrad__ It's not a bubble. At least in financial terms its not. Anthropic and Openai will have more than a $100B ARR combined by the end of 2026. That's not a bubble.

@__tinygrad__ · 2026-05-09 19:11
@EPlCs Those companies might be here to stay. I'm talking about GPU, RAM, and SSD prices. That's a bubble.

@graykevinb · 2026-05-09 19:03
@__tinygrad__ Really? https://t.co/xe66S76DWP

@__tinygrad__ · 2026-05-09 19:12
@graykevinb 100% of the problems of enshittification and culture existed before AI, turns out AI didn't fix them. Unrelated, and that's a 2019 quote.

@Nairebis · 2026-05-09 19:09
@__tinygrad__ 1) Yes. 2) No, and neither will AI, just like 1 steam shovel replacing 50 ditch diggers didn't end construction jobs. Instead, they made Seattle possible. Jevons Paradox, baby. 3) Supply follows demand, like it always has. Annoying shortages are temporary under Capitalism.

@__tinygrad__ · 2026-05-09 19:13
@Nairebis I think we are mostly in agreement. I just want cheap RAM.

@__tinygrad__ · 2026-05-09 19:13
@Nairebis I think we are mostly in agreement. I just want cheap RAM.

@empyrealrum · 2026-05-09 19:15
@__tinygrad__ @Nairebis In the long run (> 3-5 years) this will result in far cheaper RAM.

@empyrealrum · 2026-05-09 19:15
@__tinygrad__ @Nairebis In the long run (> 3-5 years) this will result in far cheaper RAM.

@__tinygrad__ · 2026-05-09 19:16
@empyrealrum @Nairebis Yea I know. I just need to buy 10PB of storage soon and I'm enraged by the prices. Can't wait until China really spins this up.

@__tinygrad__ · 2026-05-09 18:58
Can we get rid of all the AI hype? Normally I find hype just distasteful, but this is distasteful, filling people with dread, and making hardware expensive. The third one is really annoying. It's just Moore's law and a bubble, there's no singularity.

@alienorg · 2026-05-09 19:17
@__tinygrad__ "no singularity" might be right the harder claim to defend is "just moore's law", because the capability curve on reasoning tasks doesn't look like previous hardware cycles from the outside

@__tinygrad__ · 2026-05-09 19:12
@graykevinb 100% of the problems of enshittification and culture existed before AI, turns out AI didn't fix them. Unrelated, and that's a 2019 quote.

@graykevinb · 2026-05-09 19:19
@__tinygrad__ I'm not sure if you can be "near" a singularity without being in it. This all started when y'all made TinyPress and it printed paper. And before then TinyFire. The past 100+ years has been quite vertical in progress.

@graykevinb · 2026-05-09 19:19
@__tinygrad__ I'm not sure if you can be "near" a singularity without being in it. This all started when y'all made TinyPress and it printed paper. And before then TinyFire. The past 100+ years has been quite vertical in progress.

@__tinygrad__ · 2026-05-09 19:19
@graykevinb No, the past 100 years have been exponential. Not vertical. There's a huge difference.

@alienorg · 2026-05-09 19:17
@__tinygrad__ "no singularity" might be right the harder claim to defend is "just moore's law", because the capability curve on reasoning tasks doesn't look like previous hardware cycles from the outside

@__tinygrad__ · 2026-05-09 19:21
@alienorg It depends on what you focus on. If you focus on graphics tasks, you see amazing progress from the NES, SNES, N64, and XBOX, then you see diminishing returns.

@__tinygrad__ · 2026-05-09 18:58
Can we get rid of all the AI hype? Normally I find hype just distasteful, but this is distasteful, filling people with dread, and making hardware expensive. The third one is really annoying. It's just Moore's law and a bubble, there's no singularity.

@Devvgjha · 2026-05-09 19:24
@__tinygrad__ Isn’t your business based off the AI hype?

@Devvgjha · 2026-05-09 19:24
@__tinygrad__ Isn’t your business based off the AI hype?

@__tinygrad__ · 2026-05-09 19:25
@Devvgjha No. We sell computers and write a compiler.

@jeffcafe_ · 2026-05-09 19:10
@__tinygrad__ If it’s a bubble, you just need to wait until it pops and your hardware will get much cheaper.

@__tinygrad__ · 2026-05-09 19:26
@jeffcafe_ I know, but that might take like 2 or 3 years!

@__tinygrad__ · 2026-05-09 19:10
@RGBiverton There's still almost a 10x gap between NVDA and AMD. What happens to that gap if the petaflop is commoditized?

@jeffcafe_ · 2026-05-09 19:27
@__tinygrad__ @RGBiverton If AI is a bubble and pops, maybe they’ll consider datacenters stranded assets and repurpose as loft-style apartments. https://t.co/UIfSQJQJHZ

@jeffcafe_ · 2026-05-09 19:27
@__tinygrad__ @RGBiverton If AI is a bubble and pops, maybe they’ll consider datacenters stranded assets and repurpose as loft-style apartments. https://t.co/UIfSQJQJHZ

@__tinygrad__ · 2026-05-09 19:29
@jeffcafe_ @RGBiverton You can sleep standing in a 42U rack, great HVAC, and tons of electricity for your hair dryers and Panini makers.

@__tinygrad__ · 2026-05-09 19:25
@Devvgjha No. We sell computers and write a compiler.

@gustofied · 2026-05-09 19:37
@__tinygrad__ @Devvgjha bruh.. it is. would you pivot into tiny grad more without the AI hype? no..

@gustofied · 2026-05-09 19:37
@__tinygrad__ @Devvgjha bruh.. it is. would you pivot into tiny grad more without the AI hype? no..

@__tinygrad__ · 2026-05-09 19:38
@gustofied @Devvgjha Been doing this since pre ChatGPT.

@__tinygrad__ · 2026-05-09 19:10
@RGBiverton There's still almost a 10x gap between NVDA and AMD. What happens to that gap if the petaflop is commoditized?

@RGBiverton · 2026-05-09 19:38
@__tinygrad__ I need the Ralph Wiggum explantion. AMD go up?

@__tinygrad__ · 2026-05-09 18:58
Can we get rid of all the AI hype? Normally I find hype just distasteful, but this is distasteful, filling people with dread, and making hardware expensive. The third one is really annoying. It's just Moore's law and a bubble, there's no singularity.

@DeepAI · 2026-05-09 19:39
@__tinygrad__ It's the singularity, acutally.

@DeepAI · 2026-05-09 19:39
@__tinygrad__ It's the singularity, acutally.

@__tinygrad__ · 2026-05-09 19:41
@DeepAI Says someone who is based in SF and sells monthly subscriptions for AI.

@RGBiverton · 2026-05-09 19:38
@__tinygrad__ I need the Ralph Wiggum explantion. AMD go up?

@__tinygrad__ · 2026-05-09 19:42
@RGBiverton lol who knows. I do know if you buy AMD a year ago it will go up though.

@__tinygrad__ · 2026-05-09 19:01
Actually it's even worse than the Moore's law about cost cause things aren't getting cheaper! A lot of the AI gains in the last 10 years come from increased spending and smaller dtypes, both of which are nearing an end and we'll be back to the normal Moore's trajectory.

@Jonathan_Blow · 2026-05-09 19:49
@__tinygrad__ No man, we will have 0.5-bit floats any day now… the nanobit is the new nanometer.

@Jonathan_Blow · 2026-05-09 19:49
@__tinygrad__ No man, we will have 0.5-bit floats any day now… the nanobit is the new nanometer.

@__tinygrad__ · 2026-05-09 19:54
@Jonathan_Blow lol I think it stops at 4 or 5 bits. I've looked into the smaller quants and they are usually less efficient. The dendrites in your brain can probably be described well with like 16 or 32 values, but not 2 and you don't need 256.

@maxnotwork · 2026-05-09 19:55
@__tinygrad__ my personal canary in the coalmine is that i assume if anthropic invents agi, it will fix the claude code user interface first out of embarrassment before it takes over the world

@maxnotwork · 2026-05-09 19:55
@__tinygrad__ my personal canary in the coalmine is that i assume if anthropic invents agi, it will fix the claude code user interface first out of embarrassment before it takes over the world

@__tinygrad__ · 2026-05-09 19:58
@maxnotwork this is a good test to stay grounded. remember the time mythos couldn't secure itself? tons of people are rooting for anthropic to lose now cause of their awful marketing stunts, what a self own. https://t.co/YtAOkZOPZA

@__tinygrad__ · 2026-05-09 18:58
Can we get rid of all the AI hype? Normally I find hype just distasteful, but this is distasteful, filling people with dread, and making hardware expensive. The third one is really annoying. It's just Moore's law and a bubble, there's no singularity.

@0xSero · 2026-05-09 21:37
@__tinygrad__ Past how dare you be reasonable

@0xSero · 2026-05-09 21:37
@__tinygrad__ Past how dare you be reasonable

@__tinygrad__ · 2026-05-09 22:17
@0xSero You are right, I didn't consider the ROI the hyperscalers promised to investors. They sold them a singularity, the least we can all do is support their narrative.

@__tinygrad__ · 2026-05-10 15:24
@george_veng @Jonathan_Blow Woah, perfect match!

@__tinygrad__ · 2026-05-10 21:56
When will there be as good of a deal as $2200 5090s again? 1 PFLOP FP8 + 512-bit GDDR7 @ 2 TB/s + 32 GB. Please let there be a big RDNA5 die. Think of the gamers!

@__tinygrad__ · 2026-05-10 21:56
When will there be as good of a deal as $2200 5090s again? 1 PFLOP FP8 + 512-bit GDDR7 @ 2 TB/s + 32 GB. Please let there be a big RDNA5 die. Think of the gamers!

@timmajim · 2026-05-10 21:56
@__tinygrad__ How many people really got it for launch msrp?

@timmajim · 2026-05-10 21:56
@__tinygrad__ How many people really got it for launch msrp?

@__tinygrad__ · 2026-05-10 21:58
@timmajim Launch MSRP was $1999 and probably 3 people. We bought between $2200-$3000. It's not worth it anymore at $4k.

@__tinygrad__ · 2026-05-10 21:56
When will there be as good of a deal as $2200 5090s again? 1 PFLOP FP8 + 512-bit GDDR7 @ 2 TB/s + 32 GB. Please let there be a big RDNA5 die. Think of the gamers!

@bick_nyers · 2026-05-10 22:01
@__tinygrad__ If they actually do Rubin CPX the flops per dollar could be pretty good

@bick_nyers · 2026-05-10 22:01
@__tinygrad__ If they actually do Rubin CPX the flops per dollar could be pretty good

@__tinygrad__ · 2026-05-10 22:08
@bick_nyers The problem is anything that's priced for "AI" is a huge rip off. Like NVIDIA just increases margins cause they have you over a barrel. We need competition.

@__tinygrad__ · 2026-05-10 21:56
When will there be as good of a deal as $2200 5090s again? 1 PFLOP FP8 + 512-bit GDDR7 @ 2 TB/s + 32 GB. Please let there be a big RDNA5 die. Think of the gamers!

@strandedinoslo · 2026-05-10 22:10
@__tinygrad__ Maybe for 10K the new AMD Instinct 350p

@strandedinoslo · 2026-05-10 22:10
@__tinygrad__ Maybe for 10K the new AMD Instinct 350p

@__tinygrad__ · 2026-05-10 22:19
@strandedinoslo Doubtful, it will probably be double that, and it's really only comparable to 2x 5090s. The HBM and CoWoS-S is just expensive. The 5090 is close to ideal, max single die, big wide GDDR7 bus.

@ZyphraAI · 2026-05-11 17:44
Today we’re announcing 15MW of AMD Instinct MI355 GPU capacity through Zyphra Cloud, our full-stack neocloud powered by @AMD. https://t.co/G4WnCoAWJc

@teortaxesTex · 2026-05-11 17:53
I've been saying Zyphra is an exceptional neolab, on all levels from moral to technical to financial. They have done what Geohot has failed to do: made AMD relevant again. They'll reap the rewards for it. Truly, DeepSeek of the West

@APompliano · 2026-05-11 23:59
We need more data centers, not less.

@Baxate · 2026-05-12 01:41
people don’t realize how bad AI doomerism has gotten you can’t just call them luddites and move on they will vote, and we will lose the modern space race we need to start SHOWING people how AI is going to improve their life… NOW

@teortaxesTex · 2026-05-11 17:53
I've been saying Zyphra is an exceptional neolab, on all levels from moral to technical to financial. They have done what Geohot has failed to do: made AMD relevant again. They'll reap the rewards for it. Truly, DeepSeek of the West

@__tinygrad__ · 2026-05-12 18:15
@teortaxesTex Our goal is not AMD's relevancy (though it's great to see). Our goal is to commoditize the petaflop. We are supportive of everyone working toward that goal, and it's nice to see @ZyphraAI's putting open models on @huggingface

@__tinygrad__ · 2026-05-12 18:22
There will be many winners and losers in the next 10 years while the stack for AI compute matures. If you want to succeed, focus on timeless numbers like FLOPS/$, FLOPS/W, GB/s/$ and GB/$. Any focus on applications will have a short shelf life.

@__tinygrad__ · 2026-05-12 18:22
There will be many winners and losers in the next 10 years while the stack for AI compute matures. If you want to succeed, focus on timeless numbers like FLOPS/$, FLOPS/W, GB/s/$ and GB/$. Any focus on applications will have a short shelf life.

@jeffcafe_ · 2026-05-12 18:37
@__tinygrad__ And storage. This data and processing/analysis of data all has to go somewhere to support AI workloads.

@jeffcafe_ · 2026-05-12 18:37
@__tinygrad__ And storage. This data and processing/analysis of data all has to go somewhere to support AI workloads.

@__tinygrad__ · 2026-05-12 18:40
@jeffcafe_ All kinds of GB/$ from storage to RAM to cache. And all kinds of GB/s/$ like networking, DRAM, and SRAM.

@beffjezos · 2026-05-12 19:27
AI will make everything better. We need to help people realize that. Our national competitors are weaponizing democracy against us so that we cede the lead to them. We have to paint an optimistic future so people realize the costs of stopping progress.

@__tinygrad__ · 2026-05-12 20:33
All products are now available to order! $45k for tinybox green with 5090s, and we have two variants of the tinybox pro v2, one with 5090s and one with RTX6000 Blackwells (that's a whopping 768 GB of RAM) https://t.co/gxzh0yZMir

@__tinygrad__ · 2026-05-12 20:33
All products are now available to order! $45k for tinybox green with 5090s, and we have two variants of the tinybox pro v2, one with 5090s and one with RTX6000 Blackwells (that's a whopping 768 GB of RAM) https://t.co/gxzh0yZMir

@__tinygrad__ · 2026-05-12 20:39
We have sold 100s of tinyboxes to a huge variety of customers. Startups, individuals, famous tech people, military, medical. comma has 41 tinybox pro v2s. If you are serious about compute with a budget between $10k - $10M, tinybox is the right choice. https://t.co/fQFBykvpAo

@FourEyedWiz · 2026-05-12 20:42
These ones don't know it yet, but they are selling mini-datacenters. Because WTF is 45k??

@__tinygrad__ · 2026-05-12 20:39
We have sold 100s of tinyboxes to a huge variety of customers. Startups, individuals, famous tech people, military, medical. comma has 41 tinybox pro v2s. If you are serious about compute with a budget between $10k - $10M, tinybox is the right choice. https://t.co/fQFBykvpAo

@__tinygrad__ · 2026-05-12 20:45
We don't believe in contact us. If you are the type who likes simple checkout without sales flows, we are right for you. We use super high quality components, and our margins are 20%-30%. If you want something custom, after you have bought a couple boxes we can discuss.

@FourEyedWiz · 2026-05-12 20:42
These ones don't know it yet, but they are selling mini-datacenters. Because WTF is 45k??

@__tinygrad__ · 2026-05-12 20:46
@FourEyedWiz The new GPU middle class.

@beffjezos · 2026-05-12 19:27
AI will make everything better. We need to help people realize that. Our national competitors are weaponizing democracy against us so that we cede the lead to them. We have to paint an optimistic future so people realize the costs of stopping progress.

@__tinygrad__ · 2026-05-12 20:49
@beffjezos I don't think AI will make things better for the average person in the short term. It's not about "a new narrative" it's about making tech companies stop enshittifying everything. How is AI going to fix tip screens at coffee shops?

@__tinygrad__ · 2026-05-12 20:33
All products are now available to order! $45k for tinybox green with 5090s, and we have two variants of the tinybox pro v2, one with 5090s and one with RTX6000 Blackwells (that's a whopping 768 GB of RAM) https://t.co/gxzh0yZMir

@jizaymes · 2026-05-12 20:50
@__tinygrad__ Hrm, wouldn’t it be 384GB of VRAM for those Blackwell 6000s?

@jizaymes · 2026-05-12 20:50
@__tinygrad__ Hrm, wouldn’t it be 384GB of VRAM for those Blackwell 6000s?

@__tinygrad__ · 2026-05-12 20:50
@jizaymes Nope, the tinybox pro v2 has 8 GPUs!

@__tinygrad__ · 2026-05-12 20:50
@jizaymes Nope, the tinybox pro v2 has 8 GPUs!

@jizaymes · 2026-05-12 20:54
@__tinygrad__ Ok then, good clarification. Should your site be updated to 8x? https://t.co/36BxmWvWkK

@jizaymes · 2026-05-12 20:54
@__tinygrad__ Ok then, good clarification. Should your site be updated to 8x? https://t.co/36BxmWvWkK

@__tinygrad__ · 2026-05-12 20:55
@jizaymes That's for the tinybox blackwell, not the pro.

@__tinygrad__ · 2026-05-12 20:39
We have sold 100s of tinyboxes to a huge variety of customers. Startups, individuals, famous tech people, military, medical. comma has 41 tinybox pro v2s. If you are serious about compute with a budget between $10k - $10M, tinybox is the right choice. https://t.co/fQFBykvpAo

@Mathewdoeslife · 2026-05-12 20:59
@__tinygrad__ I’ll be real, if I’m dropping 65k on a machine I’d like to speak to someone… a sales flow does have its perks in that you are often paired with a human who can help and at least comfort you as you financially invest yourself into an asset.

@Mathewdoeslife · 2026-05-12 20:59
@__tinygrad__ I’ll be real, if I’m dropping 65k on a machine I’d like to speak to someone… a sales flow does have its perks in that you are often paired with a human who can help and at least comfort you as you financially invest yourself into an asset.

@__tinygrad__ · 2026-05-12 21:01
@Mathewdoeslife You are welcome to pay 30% more to our competitors and talk to a sales person. If you are the type who needs comfort from a guy paid $50k a year (plus commission) to basically lie to you, tiny isn't for you.

@APompliano · 2026-05-11 23:59
We need more data centers, not less.

@__tinygrad__ · 2026-05-12 21:04
@APompliano We need stacks of GPUs in every house, not really big stacks of GPUs controlled by companies who are trying to extract value from us.

@__tinygrad__ · 2026-05-12 21:01
@Mathewdoeslife You are welcome to pay 30% more to our competitors and talk to a sales person. If you are the type who needs comfort from a guy paid $50k a year (plus commission) to basically lie to you, tiny isn't for you.

@Mathewdoeslife · 2026-05-12 21:07
I’m providing feedback as someone who likely fits your customer profile. I’m not familiar with your sales flow and good on you if you are taking lots of orders, but maybe be a bit more humble..? building a hardware box is building a hardware box, providing a client with a service is hard. 30% might cover the attitude difference

@Mathewdoeslife · 2026-05-12 21:07
I’m providing feedback as someone who likely fits your customer profile. I’m not familiar with your sales flow and good on you if you are taking lots of orders, but maybe be a bit more humble..? building a hardware box is building a hardware box, providing a client with a service is hard. 30% might cover the attitude difference

@__tinygrad__ · 2026-05-12 21:22
@Mathewdoeslife If you want a sales flow, you don't fit the customer profile. We put all the time and money into making the computer better. It's a really good computer. Again, there's plenty of people charging 30% more for worse computers that include a guy who makes you feel good.

@__tinygrad__ · 2026-05-12 20:33
All products are now available to order! $45k for tinybox green with 5090s, and we have two variants of the tinybox pro v2, one with 5090s and one with RTX6000 Blackwells (that's a whopping 768 GB of RAM) https://t.co/gxzh0yZMir

@kukasolana · 2026-05-12 21:24
@__tinygrad__ Can these be shipped internationally?

@kukasolana · 2026-05-12 21:24
@__tinygrad__ Can these be shipped internationally?

@__tinygrad__ · 2026-05-12 21:25
@kukasolana Yes. If the checkout flow supports your country, it will ship there. There's no secret menu options. If it doesn't and you are buying NVIDIA, blame your politician for making it illegal for us to mail it to you.

@__tinygrad__ · 2026-05-12 20:33
All products are now available to order! $45k for tinybox green with 5090s, and we have two variants of the tinybox pro v2, one with 5090s and one with RTX6000 Blackwells (that's a whopping 768 GB of RAM) https://t.co/gxzh0yZMir

@thundr0n · 2026-05-12 21:43
@__tinygrad__ New tinybox red when?

@__tinygrad__ · 2026-05-12 21:22
@Mathewdoeslife If you want a sales flow, you don't fit the customer profile. We put all the time and money into making the computer better. It's a really good computer. Again, there's plenty of people charging 30% more for worse computers that include a guy who makes you feel good.

@Mathewdoeslife · 2026-05-12 21:45
Yeah I guess not. For the record, what do you charge your extra 20% for? (That number is based on consumer prices for materials today as opposed to the prices you have access to) I’m imagining that’s labour, overhead and business operating expenses. But then again, if I wanted a 4x rtx pro system, I could buy all the exact parts you list for around 52k usd today, pay a computer shop to put it together for me, give them a nice tip, have their build warranty, and save about 6k and the arrogant ceo attitude.

@Mathewdoeslife · 2026-05-12 21:45
Yeah I guess not. For the record, what do you charge your extra 20% for? (That number is based on consumer prices for materials today as opposed to the prices you have access to) I’m imagining that’s labour, overhead and business operating expenses. But then again, if I wanted a 4x rtx pro system, I could buy all the exact parts you list for around 52k usd today, pay a computer shop to put it together for me, give them a nice tip, have their build warranty, and save about 6k and the arrogant ceo attitude.

@__tinygrad__ · 2026-05-12 21:53
@Mathewdoeslife You are certainly welcome to do that. Do you know what PCIe AER errors are? Do you know what happens if you don't connect two PSUs properly? Are you okay with loud? If you are fine with AER errors, a loud computer, and potential for frying the whole system, you can maybe save $6k

@__tinygrad__ · 2026-05-12 21:55
In before I can build it myself and save money. If you enjoy building as a hobby project and value your time at 0 or negative, you can. But if you just want a machine learning that works, I strongly doubt you can beat our prices. If you can, you should start a computer business!

@__tinygrad__ · 2026-05-12 21:53
@Mathewdoeslife You are certainly welcome to do that. Do you know what PCIe AER errors are? Do you know what happens if you don't connect two PSUs properly? Are you okay with loud? If you are fine with AER errors, a loud computer, and potential for frying the whole system, you can maybe save $6k

@Mathewdoeslife · 2026-05-12 21:56
@__tinygrad__ My ignorance in these matters need not require I bask in your arrogance.

@Mathewdoeslife · 2026-05-12 21:56
@__tinygrad__ My ignorance in these matters need not require I bask in your arrogance.

@__tinygrad__ · 2026-05-12 21:59
@Mathewdoeslife It's not arrogance, it's just true, and it's why your computer shop idea won't work. It's easy to build a gaming PC with 1 GPU, but it's still pretty hard to build a reliable machine with 4, even harder if you want it to be quiet.

@thundr0n · 2026-05-12 21:43
@__tinygrad__ New tinybox red when?

@__tinygrad__ · 2026-05-12 22:01
@thundr0n When AMD releases new GPUs! Depending on pricing and how testing our samples goes, maybe the MI350P, but definitely RDNA5. https://t.co/LNEd2kXxxL

@__tinygrad__ · 2026-05-12 21:59
@Mathewdoeslife It's not arrogance, it's just true, and it's why your computer shop idea won't work. It's easy to build a gaming PC with 1 GPU, but it's still pretty hard to build a reliable machine with 4, even harder if you want it to be quiet.

@Mathewdoeslife · 2026-05-12 22:08
Brother my entire point was that the experience in engaging with a company, as knowledgable and as excellent as they are technically (as I am sure you are an intelligent person) changes drastically depending on how they are providing it. If a random non technical client wanted to buy something from you, you have not only expressed to me that you disagree with providing support in the purchasing process (thereby alienating most non-technology related companies that could benefit from your product), but in this dialogue, given me a taste of what a client might experience in engaging with you about your product. There is a reason that large corporations are successful in capturing clients and part of it is the service and support they provide. Look at my first reply to you, I didn’t attack your business, I expressed my perspective on how I as a use case would spend my 65k.

@Mathewdoeslife · 2026-05-12 22:08
Brother my entire point was that the experience in engaging with a company, as knowledgable and as excellent as they are technically (as I am sure you are an intelligent person) changes drastically depending on how they are providing it. If a random non technical client wanted to buy something from you, you have not only expressed to me that you disagree with providing support in the purchasing process (thereby alienating most non-technology related companies that could benefit from your product), but in this dialogue, given me a taste of what a client might experience in engaging with you about your product. There is a reason that large corporations are successful in capturing clients and part of it is the service and support they provide. Look at my first reply to you, I didn’t attack your business, I expressed my perspective on how I as a use case would spend my 65k.

@__tinygrad__ · 2026-05-12 22:12
@Mathewdoeslife Not sure about any of that business stuff. The computer is really good, and it's available today at a more than fair price with one click. We're super responsive to real issues with the machines on Discord. The goal isn't some enterprise mumbo jumbo, it's make excellent computer.

@__tinygrad__ · 2026-05-12 22:14
RT @luigifcruz: As someone who dealt with OEM procurement and assembly of many large GPU-accelerated servers, I can say people have no idea what a hellscape, even before AI became popular, it is to source, purchase, assemble, and validate such a server. Their markup for a ready-to-go system is valid and, for many, a better alternative than navigating the server sales.

@luigifcruz · 2026-05-12 22:13
As someone who dealt with OEM procurement and assembly of many large GPU-accelerated servers, I can say people have no idea what a hellscape, even before AI became popular, it is to source, purchase, assemble, and validate such a server. Their markup for a ready-to-go system is valid and, for many, a better alternative than navigating the server sales.

@__tinygrad__ · 2026-05-12 22:17
@luigifcruz Thanks! Yea I'm sure if we hired MBAs to manage things we could extract 30% more, but then our sales process would be terrible too. We're an ML company first and foremost, between our internal cluster and @comma_ai we run over 100 of these machines, so quality is what matters.

@GrigoryEvko · 2026-05-13 01:07
I'm genuinely curious what will happen when i ship crucible, replacing cuda, rocm, xla and unifying everything with oom better integration and vertical design and proving this is possible and superior, will vendors start to get nervous and focusing on hardware more? or what

@GrigoryEvko · 2026-05-13 01:11
tinygrad guys are autistically focused on their number of LoCs unfortunately, intead on real progress

@__tinygrad__ · 2026-05-13 16:18
Someone dropped this in the Discord. RL Snake game in browser powered by tinygrad WebGPU, it even worked on my phone! https://t.co/88CUwFCxrG

@__tinygrad__ · 2026-05-13 16:23
We have been focusing on constructing a minimal calculus of Tensors and a self contained runtime. Speed has been improving as a result of this too. If by "real progress" you mean the kind of fake announcements and partnerships that please VCs, we don't care.

@__tinygrad__ · 2026-05-13 16:23
We have been focusing on constructing a minimal calculus of Tensors and a self contained runtime. Speed has been improving as a result of this too. If by "real progress" you mean the kind of fake announcements and partnerships that please VCs, we don't care.

@__tinygrad__ · 2026-05-13 16:33
To be more generous to "real progress", even if you mean features this is a bad idea. Features are what people focus on if they need to meet some Q2 OKR deadline. We are trying to redefine the computing stack for software 2.0. It will take 10 years.

@__tinygrad__ · 2026-05-13 16:18
Someone dropped this in the Discord. RL Snake game in browser powered by tinygrad WebGPU, it even worked on my phone! https://t.co/88CUwFCxrG

@___Harald___ · 2026-05-13 16:36
@__tinygrad__ no webgpu adapter :( https://t.co/prtOn30SRa

@__tinygrad__ · 2026-05-13 16:33
To be more generous to "real progress", even if you mean features this is a bad idea. Features are what people focus on if they need to meet some Q2 OKR deadline. We are trying to redefine the computing stack for software 2.0. It will take 10 years.

@__tinygrad__ · 2026-05-13 16:39
Of course, you need contact with reality. While this project has elements of philosophy, it's not ungrounded. It's a functioning artifact that can run and train the latest models at decent speed on a huge variety of hardware without vendor code in sub 25k lines of pure Python.

@___Harald___ · 2026-05-13 16:36
@__tinygrad__ no webgpu adapter :( https://t.co/prtOn30SRa

@__tinygrad__ · 2026-05-13 16:41
@___Harald___ What computer and browser? This worked on my Z Fold7 in Chrome. Check https://t.co/iVVVXqEAxh

@__tinygrad__ · 2026-05-13 16:23
We have been focusing on constructing a minimal calculus of Tensors and a self contained runtime. Speed has been improving as a result of this too. If by "real progress" you mean the kind of fake announcements and partnerships that please VCs, we don't care.

@GrigoryEvko · 2026-05-13 16:46
1) how do you even find this post 2) where did you see the vc and announcements, I'm doing strongly typed dependent flavored c++26 vertical runtime 3) why python? Though I like your approach and ideas and borrowed some, just why would you write HPC in python? 4) of course I meant real features. By the way see my reverse eng docs of nvidia compilers, I will reimplement my own compilers https://t.co/eWZQ7XIqhj 5) partially ragebating/partially real, I find your scrutiny over LoCs and especially the compressed way you guys write code very ostentatious, that's just strange. It reads like dense math but meh too dense. Also purposefully no comments whatsoever i think?

@GrigoryEvko · 2026-05-13 16:46
1) how do you even find this post 2) where did you see the vc and announcements, I'm doing strongly typed dependent flavored c++26 vertical runtime 3) why python? Though I like your approach and ideas and borrowed some, just why would you write HPC in python? 4) of course I meant real features. By the way see my reverse eng docs of nvidia compilers, I will reimplement my own compilers https://t.co/eWZQ7XIqhj 5) partially ragebating/partially real, I find your scrutiny over LoCs and especially the compressed way you guys write code very ostentatious, that's just strange. It reads like dense math but meh too dense. Also purposefully no comments whatsoever i think?

@__tinygrad__ · 2026-05-13 16:53
Searched for mentions of tinygrad. It's Python because it's the fastest language to iterate in. While Python does have a very disappointing type system, the perf of the language barely matters; tinygrad is a compiler. The bad language perf is almost a feature, it forces you to write things correctly. imo c++ is the wrong trade off. It has strong enough types to be annoying, but still lacking dependent types and the ability to prove things about code. I imagine a lot of tinygrad's rewrite rules moving to something like Lean or Idris. Still with the runtime in Python, but with a really good type checker where it matters.

@GrigoryEvko · 2026-05-13 16:50
@__tinygrad__ Also I don't like you over claiming things, e.g. when you say 25k lines of python, it's still ptxas with 3M locs underneath and 159 passes and nvlink after it wdym? Yes you can print amd binary and that's great! But I'd like more honesty personally

@__tinygrad__ · 2026-05-13 16:55
@GrigoryEvko So we have our first assembly backend merging soon, but yea, currently it's still LLVM or Mesa or ptxas for that step. You don't need ptxas for NVIDIA though, we have NAK Mesa support. And that "codegen" step is doing less and less, it's none of the hard parts.

@__tinygrad__ · 2026-05-13 16:55
@GrigoryEvko So we have our first assembly backend merging soon, but yea, currently it's still LLVM or Mesa or ptxas for that step. You don't need ptxas for NVIDIA though, we have NAK Mesa support. And that "codegen" step is doing less and less, it's none of the hard parts.

@__tinygrad__ · 2026-05-13 16:58
@GrigoryEvko It's really not very dishonest though, like it's not like there's tons of complexity hiding there, it's just mostly uninteresting and others have solved it cleanly. Once we get to pushing beyond SOTA on speed we will need to remove it though, particularly AMD's LLVM backend.

@scaling01 · 2026-05-14 20:12
Not only is Anthropic saying they will have a "country of geniuses in a datacenter" by 2028, but also that the US could be ahead by 12-24 months. Before GPT-5.5 and Claude Mythos chinese labs were ~8 months behind in broader capabilities and ~5 months behind in coding. However, catching up to GPT-5.5 and especially Mythos will likely take longer than that, because they have no way of training and serving 10T models at scale. Especially not in the monthly cadence as american frontier labs are doing. Most of the gains are no longer coming from a single generational leap through larger pre-training but through monthly RL post-training improvements. The leap between Mythos Preview today and a future Mythos version in 6-12 months will be enormous compared to the leap between Opus to Mythos. The relative leap between Opus 4 and Opus 4.7 will also be overshadowed by the leap between Mythos Preview today and a future Mythos version, as RL benefits from model scale (+all the other reasons like growing compute, and accelerated R&D pace)

@AnthropicAI · 2026-05-14 18:09
We've published a paper that explains our views on AI competition between the US and China. The US and democratic allies hold the lead in frontier AI today. Read more on what it’ll take to keep that lead: https://t.co/TgJBeodWYK

@__tinygrad__ · 2026-05-14 20:59
@AnthropicAI lol "The Mythos Preview wake-up call" that's like if we published a blog post with "the tinybox green v2 wake-up call" except we actually sell the product.

@__tinygrad__ · 2026-05-14 20:59
@AnthropicAI lol "The Mythos Preview wake-up call" that's like if we published a blog post with "the tinybox green v2 wake-up call" except we actually sell the product.

@dev_null321 · 2026-05-14 21:29
@__tinygrad__ @AnthropicAI Mad you don’t have access to mythos 😂😂

@dev_null321 · 2026-05-14 21:29
@__tinygrad__ @AnthropicAI Mad you don’t have access to mythos 😂😂

@__tinygrad__ · 2026-05-14 23:02
@dev_null321 @AnthropicAI I do, it's called GPT-5.5

@__tinygrad__ · 2026-05-15 04:56
RT @real_deep_ml: We just added both tinygrad and PyTorch problem collections. Curious to see which one people complete more https://t.co/EdgrIY9exh

@scaling01 · 2026-05-15 10:52
Yesterday I realized that some of you might be confused about the AI gap between open vs closed or Chinese vs American models. I think most people are reporting the backward looking number of 4-9 months, which is asking how long ago frontier models reached the same performance that current open/chinese models have. But I think what's more interesting (but obviously much harder to forecast) is the current or forward looking gap which tries to forecast how long it will take open/chinese models to catch up to current frontier models. In my opinion that number is larger than the backward looking gap and it will take >12 months (>April 7th 2027) to catch the current frontier that is Claude Mythos Preview. But in 12 months Anthropic/OpenAI/Google will very likely have much much stronger models. So growth rates / doubling times matter too.

@ludwigABAP · 2026-05-15 12:53
turns out a tenstorrent background for tinygrad targeting tt-lang (instead of raw tt-metal) is very much doable tbh - i might not be able to go the whole way but it wasn't hard to get eg 40 kernel, four layer stacked transformer to run on a Wormhole

@bgurley · 2026-05-15 12:59
A new @bgurley blog post! I have been thinking about how sophisticated executives are using open source in super creative ways. Started writing this three years ago. Excited to finish it up and publish it! And with the new @p3institute brand. https://t.co/W84vODq1ME

@AndrewCurran_ · 2026-05-15 14:35
From the article: https://t.co/ABsgZRjHww

@AndrewCurran_ · 2026-05-15 14:35
From the article: https://t.co/ABsgZRjHww

@__tinygrad__ · 2026-05-15 18:56
@AndrewCurran_ I think China might have a great firewall the US can license.

@__tinygrad__ · 2026-05-15 19:04
@scaling01 The US is ramping up spending way faster than the Chinese. The slope on this graph are roughly correct, but it's not a skill issue, it's an expected ROI question. I tend to believe China is more reality grounded in their capex. https://t.co/UhCTWgb5si

@massinference · 2026-05-15 19:21
@__tinygrad__ @scaling01 if the alternative to AI datacenter capex is buybacks or dividends I will take the capex. even if the time-to-value takes longer than projected Google meta Amazon and msft will depreciate these assets and run high-gross margin AI features on them.

@mitchellh · 2026-05-15 20:10
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out. I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really). It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely. The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture. We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying. I worry.

@massinference · 2026-05-15 19:21
@__tinygrad__ @scaling01 if the alternative to AI datacenter capex is buybacks or dividends I will take the capex. even if the time-to-value takes longer than projected Google meta Amazon and msft will depreciate these assets and run high-gross margin AI features on them.

@__tinygrad__ · 2026-05-15 21:38
@massinference @scaling01 I mean...this exposes a deeper problem of revenue multiples not reflecting the lack of growth, but yea, much nicer to see them buy large metal boxes than give people more money back to further pump speculative assets.

@mitchellh · 2026-05-15 20:10
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out. I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really). It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely. The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture. We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying. I worry.

@__tinygrad__ · 2026-05-15 21:58
@mitchellh We have entered an era where it takes 20 minutes to determine that what the AI told you was total bullshit vs 10 seconds. Make sure you and your org keep contact with reality. Not everyone will make it.

@__tinygrad__ · 2026-05-18 06:42
The AI panic is really unbelievable today. The level of delusion and hype have grown to mythic proportions. Has AI beaten Pokemon Red yet? Like a normal 6 year old does, by looking at the screen? Oh it hasn't. But all jobs are over in 18 months? This website is full of idiots.

@__tinygrad__ · 2026-05-18 06:42
The AI panic is really unbelievable today. The level of delusion and hype have grown to mythic proportions. Has AI beaten Pokemon Red yet? Like a normal 6 year old does, by looking at the screen? Oh it hasn't. But all jobs are over in 18 months? This website is full of idiots.

@n_reruns · 2026-05-18 07:15
@__tinygrad__ George Opus 4.7 literally beat Pokémon red for the first time this weekend live on twitch!

@n_reruns · 2026-05-18 07:15
@__tinygrad__ George Opus 4.7 literally beat Pokémon red for the first time this weekend live on twitch!

@__tinygrad__ · 2026-05-18 07:28
@n_reruns ugh. see all the "<extracted_ram_data>" in the prompt? you need that as a kid to beat the game? https://t.co/ExBC6CpPxg

@__tinygrad__ · 2026-05-18 06:42
The AI panic is really unbelievable today. The level of delusion and hype have grown to mythic proportions. Has AI beaten Pokemon Red yet? Like a normal 6 year old does, by looking at the screen? Oh it hasn't. But all jobs are over in 18 months? This website is full of idiots.

@halvarflake · 2026-05-18 12:11
@__tinygrad__ The power drill and buzz saw didn't make carpenters obsolete either.

@__tinygrad__ · 2026-05-18 06:42
The AI panic is really unbelievable today. The level of delusion and hype have grown to mythic proportions. Has AI beaten Pokemon Red yet? Like a normal 6 year old does, by looking at the screen? Oh it hasn't. But all jobs are over in 18 months? This website is full of idiots.

@FoxOnTheRunTr · 2026-05-18 14:16
Hard agree. If Hotz says it, I trust it. Instead of the world ending in 18 months, we'll probably just have 5-10 x context lengths and smarter hardware/architecture tricks like Cerebras deal or GPT spark models. The "clumsy" humanoid phase might be still 4-5 years out as consumer product. We don't solve the intelligence gap until AI actually moves through the physical world and learns from real-time video.

@__tinygrad__ · 2026-05-18 06:42
The AI panic is really unbelievable today. The level of delusion and hype have grown to mythic proportions. Has AI beaten Pokemon Red yet? Like a normal 6 year old does, by looking at the screen? Oh it hasn't. But all jobs are over in 18 months? This website is full of idiots.

@SALTY_ALTY555 · 2026-05-18 16:18
@__tinygrad__ Hey my AI has successfully named itself “AAAAAAAA” and then got stuck in the starting area so I’m pretty sure we are on our way.

@__tinygrad__ · 2026-05-18 06:42
The AI panic is really unbelievable today. The level of delusion and hype have grown to mythic proportions. Has AI beaten Pokemon Red yet? Like a normal 6 year old does, by looking at the screen? Oh it hasn't. But all jobs are over in 18 months? This website is full of idiots.

@jaykishankrk_1 · 2026-05-18 16:20
@__tinygrad__ Is it engagement farming or is the delusion really real. Some of these people who are advocating of AI apocalypse haven’t even coded a single line of code in their lives. If they have then they wouldn’t participating in this hysteria.

@Cernovich · 2026-05-18 16:40
The AI Propaganda Slop Sandwich in action: Big Tech makes a claim, such as AI will take all jobs in 18 months. Them AI slop boy bots side call you a Luddite or panican for accepting the claim made by the AI company as true. https://t.co/TbSDgqLOR2

@__tinygrad__ · 2026-05-18 06:42
The AI panic is really unbelievable today. The level of delusion and hype have grown to mythic proportions. Has AI beaten Pokemon Red yet? Like a normal 6 year old does, by looking at the screen? Oh it hasn't. But all jobs are over in 18 months? This website is full of idiots.

@jsuarez · 2026-05-18 16:59
We beat it with pixels + state information and small-model RL. I'd argue this is way more impressive since our model has no visual pretraining and cannot read the text. Humans cannot solve the task reasonably under those conditions. There was quite a bit of handholding engineering. The amount of that has been steadily decreasing from project to project as our core algorithm improves. Pokemon red specifically caps out around 7-10k steps/second on the base sim, which is ~1000x slower than our custom envs. That's just pure cost inflation on compute, not problem complexity

@jsuarez · 2026-05-18 16:59
We beat it with pixels + state information and small-model RL. I'd argue this is way more impressive since our model has no visual pretraining and cannot read the text. Humans cannot solve the task reasonably under those conditions. There was quite a bit of handholding engineering. The amount of that has been steadily decreasing from project to project as our core algorithm improves. Pokemon red specifically caps out around 7-10k steps/second on the base sim, which is ~1000x slower than our custom envs. That's just pure cost inflation on compute, not problem complexity

@__tinygrad__ · 2026-05-18 17:09
@jsuarez I want to see it beat without side-channel information, same as what human has. Then do the same for Mario 64. At that "Mario 64 from pixels" point, I'll believe AI can replace some jobs that 12 year olds can do. (all the jobs being replaced by AI now weren't real to begin with)

@halvarflake · 2026-05-18 12:11
@__tinygrad__ The power drill and buzz saw didn't make carpenters obsolete either.

@__tinygrad__ · 2026-05-18 17:10
@halvarflake AI is a great tool. There's no room for nuance on the Internet, but you can believe both that AI is cool and useful, and also believe that it's all basically on trend for the computer revolution as planned.

@jaykishankrk_1 · 2026-05-18 16:20
@__tinygrad__ Is it engagement farming or is the delusion really real. Some of these people who are advocating of AI apocalypse haven’t even coded a single line of code in their lives. If they have then they wouldn’t participating in this hysteria.

@__tinygrad__ · 2026-05-18 17:12
@jaykishankrk_1 Some days I think it's all funded by Anthropic (then piled on by useful idiots). Cui bono?

@SALTY_ALTY555 · 2026-05-18 16:18
@__tinygrad__ Hey my AI has successfully named itself “AAAAAAAA” and then got stuck in the starting area so I’m pretty sure we are on our way.

@__tinygrad__ · 2026-05-18 17:13
@SALTY_ALTY555 Ahh, sounds like how a 2 year old would play the game.

@FoxOnTheRunTr · 2026-05-18 14:16
Hard agree. If Hotz says it, I trust it. Instead of the world ending in 18 months, we'll probably just have 5-10 x context lengths and smarter hardware/architecture tricks like Cerebras deal or GPT spark models. The "clumsy" humanoid phase might be still 4-5 years out as consumer product. We don't solve the intelligence gap until AI actually moves through the physical world and learns from real-time video.

@__tinygrad__ · 2026-05-18 17:17
@FoxOnTheRunTr Similar to Uber pricing, there will be a dip at some point when the subsidies wear off. But overall things will continue to improve, and we'll get better at understanding the tasks AI can do well vs the tasks it can't.

@aphysicist · 2026-05-18 12:17
10/10 no notes https://t.co/txc1bTUzeK

@VeeralAI · 2026-05-18 18:09
@aphysicist Need to solve the branding problem https://t.co/Tit9TbSrKp

@halvarflake · 2026-05-18 18:14
@__tinygrad__ I think we largely agree :-)

@__tinygrad__ · 2026-05-18 18:31
@halvarflake I find most computer security people have a grounded notion on AI because they have practical experience with things like fuzzing and z3 and see things as search. Search is powerful, but the space is bounded, and even within a space it can be just hard. AI is mainstream search.

@__tinygrad__ · 2026-05-18 18:31
@halvarflake I find most computer security people have a grounded notion on AI because they have practical experience with things like fuzzing and z3 and see things as search. Search is powerful, but the space is bounded, and even within a space it can be just hard. AI is mainstream search.

@__tinygrad__ · 2026-05-18 18:35
@halvarflake The indefinite optimism of people first exposed to this is like the first taste of using a fuzzer and thinking it's going to find all the bugs. It may find many and be very useful, but it won't ever find them all. Similarly with current AI tech and the problems it will solve.

@__tinygrad__ · 2026-05-18 18:38
This is who we want as tiny corp customers.

@halvarflake · 2026-05-18 19:46
@__tinygrad__ It's somewhat different from search, and I think AI is - in many ways - surprising. Like, I don't think I'd have accepted how useful next-token prediction can be, and that you can use it as a biased search into problem spaces that are otherwise very hard to tackle...

@halvarflake · 2026-05-18 19:47
@__tinygrad__ ... leading to a bunch of things that were nigh-impossible to become easy. That said - the current AI systems have real issues with context size, preconceptions from the training set, etc. etc. etc. Exciting times for sure, but not the end of times.

@Cernovich · 2026-05-18 16:40
The AI Propaganda Slop Sandwich in action: Big Tech makes a claim, such as AI will take all jobs in 18 months. Them AI slop boy bots side call you a Luddite or panican for accepting the claim made by the AI company as true. https://t.co/TbSDgqLOR2

@__tinygrad__ · 2026-05-18 20:23
@Cernovich lol we are an AI company, not bots. unlike others, we have a sustainable business model, so we can tell the truth instead of hype.

@halvarflake · 2026-05-18 19:47
@__tinygrad__ ... leading to a bunch of things that were nigh-impossible to become easy. That said - the current AI systems have real issues with context size, preconceptions from the training set, etc. etc. etc. Exciting times for sure, but not the end of times.

@__tinygrad__ · 2026-05-18 23:47
I worked on the Hutter Prize 10 years ago, intelligence is compression and all that. I never doubted the power of next token prediction, though I was surprised how quick ChatGPT took off. Context size will be fixed, that's straightforward engineering, but preconceptions from the dataset is less clear what it even means to fix it. Somehow humans are 1000x more data efficient than ML, and are way better at zero-shotting things and building tools for themselves. I think this generation of AI will get really really good at anything that has a definable objective you can't game. Performance, game playing, exploit finding, math. And it does this through massive (guided) search. But it still can't rap. And it still can't play any open world-ish games where rewards are sparse. The bottleneck switches from search to defining the objective. I have always defined AI as the "do what I mean" machine, but around the edges, that's very hard to do without being embedded in a culture. The next paradigm is all about sample efficiency. So it's still exactly the Hutter Prize, but the compressor is the AI, not the model. Until we have lifetime sample efficient learners that are on par with humans in efficiency and cost, there's a lot AI won't come for yet.

@__tinygrad__ · 2026-05-18 23:50
To everyone who wants to invest in tiny, preorder an exabox. At launch, it will be the cheapest compute you can buy. Deploy it and run it and make returns! We don't want VCs who invest other people's pensions with only upside potential for them. We want people in the trenches.

@__tinygrad__ · 2026-05-18 23:50
To everyone who wants to invest in tiny, preorder an exabox. At launch, it will be the cheapest compute you can buy. Deploy it and run it and make returns! We don't want VCs who invest other people's pensions with only upside potential for them. We want people in the trenches.

@kneeanderthul · 2026-05-19 00:02
@__tinygrad__ That'd be insane if the folks who bought boxes get first dibs if things go public. 🤯

@kneeanderthul · 2026-05-19 00:02
@__tinygrad__ That'd be insane if the folks who bought boxes get first dibs if things go public. 🤯

@__tinygrad__ · 2026-05-19 00:03
@kneeanderthul Ugh, you are thinking about this wrong. Public? Who cares about the stock market that's all fugazi. You buy an exabox, and you will have a large computer. The currency of the future I hear.

@__tinygrad__ · 2026-05-18 23:50
To everyone who wants to invest in tiny, preorder an exabox. At launch, it will be the cheapest compute you can buy. Deploy it and run it and make returns! We don't want VCs who invest other people's pensions with only upside potential for them. We want people in the trenches.

@DavidFSWD · 2026-05-19 00:03
@__tinygrad__ if you need rackspace let me know, I know a DC you can ship to and host it DM

@DavidFSWD · 2026-05-19 00:03
@__tinygrad__ if you need rackspace let me know, I know a DC you can ship to and host it DM

@__tinygrad__ · 2026-05-19 00:04
@DavidFSWD Every one has rack space. The question is, do you have the cheapest rack space? Only considering < 0.10c / kWh all in.

@__tinygrad__ · 2026-05-18 23:50
To everyone who wants to invest in tiny, preorder an exabox. At launch, it will be the cheapest compute you can buy. Deploy it and run it and make returns! We don't want VCs who invest other people's pensions with only upside potential for them. We want people in the trenches.

@opdroid1234 · 2026-05-19 01:16
@__tinygrad__ what about us poors?

@tszzl · 2026-05-19 01:30
on some level if you want civilization to ascend to a new level you need your AIs to do things that are not legible to you and maybe not even strictly obey you, in the same way that if you hire a great new ceo you give them a lot of autonomy to transform the company according to their own plan, even one which may not immediately read as a winning strategy (imagine the board of directors of Apple firing and rehiring Steve Jobs years later - except the board of directors are chimpanzees) all else equal, companies and organizations that hand more of themselves over to machine intelligence will outcompete ones that demand the corrigibility and legibility tax of human oversight and human design. it is not a stable equilibrium and requires some sort of vast cooperation scheme if you’d like to enforce it real asi alignment has to operate at a deeper level than oversight, control, or human corrigibility

@__tinygrad__ · 2026-05-19 00:04
@DavidFSWD Every one has rack space. The question is, do you have the cheapest rack space? Only considering < 0.10c / kWh all in.

@HeyBitCap · 2026-05-19 01:30
@__tinygrad__ @DavidFSWD Doable.

@HeyBitCap · 2026-05-19 01:30
@__tinygrad__ @DavidFSWD Doable.

@__tinygrad__ · 2026-05-19 01:39
@HeyBitCap @DavidFSWD Where? And do you happen to have chilled water? We're currently building exabox #1 in our backyard, should be ready to deploy in August. (500 kW, looking for a 5 yr lease, some base rate + pay for the power we use, expandable to 10 MW over 5 years)

@opdroid1234 · 2026-05-19 01:16
@__tinygrad__ what about us poors?

@__tinygrad__ · 2026-05-19 01:39
@opdroid1234 How much would you pay for a tinybox with 4x3080?

@__tinygrad__ · 2026-05-19 01:39
@opdroid1234 How much would you pay for a tinybox with 4x3080?

@opdroid1234 · 2026-05-19 01:55
@__tinygrad__ I know ram pricing has been crazy so dont know if I am in the ballpark but somewhere in the $12k to $16k range

@opdroid1234 · 2026-05-19 01:55
@__tinygrad__ I know ram pricing has been crazy so dont know if I am in the ballpark but somewhere in the $12k to $16k range

@__tinygrad__ · 2026-05-19 02:00
@opdroid1234 Would you actually pay that? I was thinking cheaper than $12k.

@__tinygrad__ · 2026-05-19 02:03
@tszzl do you also think your 6 year old child knows best? he decided instead of dinner this week we just get donuts. if you want civilization to ascend to a new level you need to give your child a lot of autonomy to reject broccoli and ONLY DONUTS. to be fair, he just beat pokemon red.

@Daniel_in_2025 · 2026-05-19 02:10
@__tinygrad__ @tszzl Well... did you feed the little gremlin after midnight?

@Daniel_in_2025 · 2026-05-19 02:10
@__tinygrad__ @tszzl Well... did you feed the little gremlin after midnight?

@__tinygrad__ · 2026-05-19 02:11
@Daniel_in_2025 @tszzl he asked. he said that mothers and fathers that hand more of themselves over to child intelligence will outcompete ones that demand the corrigibility and legibility tax of parental oversight

@SheriefFYI · 2026-05-19 04:10
building ONNX Runtime is done through a 2,500+ line Python script - which makes me wonder, is this really irreducible complexity? is there no way to build this while satisfying all the constraints with less than 2.5 KLOC of Turing completeness? something smells wrong.

@__tinygrad__ · 2026-05-18 23:47
I worked on the Hutter Prize 10 years ago, intelligence is compression and all that. I never doubted the power of next token prediction, though I was surprised how quick ChatGPT took off. Context size will be fixed, that's straightforward engineering, but preconceptions from the dataset is less clear what it even means to fix it. Somehow humans are 1000x more data efficient than ML, and are way better at zero-shotting things and building tools for themselves. I think this generation of AI will get really really good at anything that has a definable objective you can't game. Performance, game playing, exploit finding, math. And it does this through massive (guided) search. But it still can't rap. And it still can't play any open world-ish games where rewards are sparse. The bottleneck switches from search to defining the objective. I have always defined AI as the "do what I mean" machine, but around the edges, that's very hard to do without being embedded in a culture. The next paradigm is all about sample efficiency. So it's still exactly the Hutter Prize, but the compressor is the AI, not the model. Until we have lifetime sample efficient learners that are on par with humans in efficiency and cost, there's a lot AI won't come for yet.

@halvarflake · 2026-05-19 06:14
While I was all-in on compression~prediction and compression~intelligence, I didn't see that next token on language yields a form of guided search. But yeah. Agreed that AI will get good on problems where guided search can find verifiable rewards. Agreed on sample efficiency. I'd also add that online training is pretty unsolved right now, too, and another necessary step. Energy efficiency - I don't think AI needs to match humans on energy efficiency, but that's a different question.

@ludwigABAP · 2026-05-19 10:06
on experimenting with a tenstorrent backend for tinygrad: sure I got smth working but at what cost man... I think TT LLK is just the wrong way to go about it, per-op semantics tied to a kernel library and the overall design... this is not the way to go my attempts were to basically: 1. cheat my way to get something working by leveraging tt-lang as much as possible: tg UOps -> a tt_renderer.py emitting python source text -> the @.ttl_kernel decorations parse the AST -> ttl_d MLIR -> ... -> LLK calls -> c++ kernel source -> an actual riscv32 binary this was to get my feet wet and also take a look inside how tt-lang works internally, since i just treated it as "magic python DSL that makes things work" before that 2. then I tried a "more direct approach": go from tinygrad UOps -> skip the python ast frontend from tt-lang -> emit ttl_d MLIR directly -> same as above 3. my next approach would then be to build some kind of SFPI renderer that takes in tg UOps and emits all the right "primitives" and other SIMD instrisics etc into a c++ kernel source and feeding that to the SFPI compiler, which I will try anyway just because I am curious and want to learn on the way. Hotz did advise another approach, but I don't even think a single human could do the actual path he suggests (which is probably the best one...), certainly I couldn't without tremendous amounts of effort and work (and I would still need some help). You'd skip the LLK swamp entirely and have bare metal driver control, but tbh even with his prototype I just don't have the willpower, mental fortitude and frankly the skills atm to even attempt this properly. It seems like a tremendous amount of work unless you are already strongly capable of intuiting some golden path(s). attached below is Claude being given my tweet and asked: "I am writing this recap: [Pasted text #2 +18 lines] Is everything in there correct? And assuming someone doesnt know tenstorrent terminology very well, could you make a 1-2 lines per term definition of all the TT specifics like SFPI LLK etc... in my style (very clear and simple)"

@ludwigABAP · 2026-05-19 10:06
on experimenting with a tenstorrent backend for tinygrad: sure I got smth working but at what cost man... I think TT LLK is just the wrong way to go about it, per-op semantics tied to a kernel library and the overall design... this is not the way to go my attempts were to basically: 1. cheat my way to get something working by leveraging tt-lang as much as possible: tg UOps -> a tt_renderer.py emitting python source text -> the @.ttl_kernel decorations parse the AST -> ttl_d MLIR -> ... -> LLK calls -> c++ kernel source -> an actual riscv32 binary this was to get my feet wet and also take a look inside how tt-lang works internally, since i just treated it as "magic python DSL that makes things work" before that 2. then I tried a "more direct approach": go from tinygrad UOps -> skip the python ast frontend from tt-lang -> emit ttl_d MLIR directly -> same as above 3. my next approach would then be to build some kind of SFPI renderer that takes in tg UOps and emits all the right "primitives" and other SIMD instrisics etc into a c++ kernel source and feeding that to the SFPI compiler, which I will try anyway just because I am curious and want to learn on the way. Hotz did advise another approach, but I don't even think a single human could do the actual path he suggests (which is probably the best one...), certainly I couldn't without tremendous amounts of effort and work (and I would still need some help). You'd skip the LLK swamp entirely and have bare metal driver control, but tbh even with his prototype I just don't have the willpower, mental fortitude and frankly the skills atm to even attempt this properly. It seems like a tremendous amount of work unless you are already strongly capable of intuiting some golden path(s). attached below is Claude being given my tweet and asked: "I am writing this recap: [Pasted text #2 +18 lines] Is everything in there correct? And assuming someone doesnt know tenstorrent terminology very well, could you make a 1-2 lines per term definition of all the TT specifics like SFPI LLK etc... in my style (very clear and simple)"

@__tinygrad__ · 2026-05-19 14:14
@ludwigABAP I don't understand what they are doing. If they actually want to be a serious player in AI compute, I think my suggested approach is the only one that works. NVIDIA puts so much effort into getting CUTLASS to compile with the normal CUDA pipeline, and that's a reason they win.

@halvarflake · 2026-05-19 06:14
While I was all-in on compression~prediction and compression~intelligence, I didn't see that next token on language yields a form of guided search. But yeah. Agreed that AI will get good on problems where guided search can find verifiable rewards. Agreed on sample efficiency. I'd also add that online training is pretty unsolved right now, too, and another necessary step. Energy efficiency - I don't think AI needs to match humans on energy efficiency, but that's a different question.

@__tinygrad__ · 2026-05-19 14:20
@halvarflake I think energy is fungible with any other operating expense. We pay a lot to keep computers enough above the Landauer Limit that they operate deterministically.

@SheriefFYI · 2026-05-19 04:10
building ONNX Runtime is done through a 2,500+ line Python script - which makes me wonder, is this really irreducible complexity? is there no way to build this while satisfying all the constraints with less than 2.5 KLOC of Turing completeness? something smells wrong.

@__tinygrad__ · 2026-05-19 14:21
@SheriefFYI We have an ONNX frontend that passes more of the ONNX test suite than ONNX runtime.

@karpathy · 2026-05-19 15:05
Personal update: I've joined Anthropic. I think the next few years at the frontier of LLMs will be especially formative. I am very excited to join the team here and get back to R&D. I remain deeply passionate about education and plan to resume my work on it in time.

@natolambert · 2026-05-19 15:32
Very happy for Karpathy. Also very lonely times in open science.

@hms1193 · 2026-05-19 16:43
Intel's Crescent Island PCB Leaks, Showing a Massive Xe3P GPU, 16-Pin Connector, 160GB LPDDR5X as Intel Sidesteps the HBM Shortage https://t.co/hi2pNzH4By https://t.co/hi2pNzH4By

@karpathy · 2026-05-19 15:05
Personal update: I've joined Anthropic. I think the next few years at the frontier of LLMs will be especially formative. I am very excited to join the team here and get back to R&D. I remain deeply passionate about education and plan to resume my work on it in time.

@__tinygrad__ · 2026-05-19 18:08
@karpathy Hopefully you can nudge them to be more open. If Anthropic continues to use FOMO for marketing, like every cult that came before they will lose. AGI is not a race to the finish line, it's the new steam engine.

@__tinygrad__ · 2026-05-19 19:50
@natolambert When you look back at the history of science and technology, all the names you remember were open science. Tesla died penniless, yet we know his name. Newton, Plato, and Turing didn't work on closed stuff. It's a trade of temporary comfort for a shot at eternal glory.

@Lon · 2026-05-19 19:53
Boltzman hung himself. Turing poisoned himself. Gödel died of starvation. But, you know, eternal glory and all that stuff...

@__tinygrad__ · 2026-05-19 19:55
Our first assembly backend, DEV=CPU:X86 is merged! Instruction selection + register allocation added, and it's all visible with VIZ=1. This is the path to kernels that outperform everyone playing at the LLVM/PTX layer. https://t.co/PnSf8L5tJY

@Lon · 2026-05-19 19:53
Boltzman hung himself. Turing poisoned himself. Gödel died of starvation. But, you know, eternal glory and all that stuff...

@__tinygrad__ · 2026-05-19 19:56
@Lon @natolambert And yet, I know all three of their names. The strongest form of legacy is not "I owned the thing." It's "the next thousand things quietly depended on this thing I gave away."

@__tinygrad__ · 2026-05-19 19:55
Our first assembly backend, DEV=CPU:X86 is merged! Instruction selection + register allocation added, and it's all visible with VIZ=1. This is the path to kernels that outperform everyone playing at the LLVM/PTX layer. https://t.co/PnSf8L5tJY

@opdroid1234 · 2026-05-19 20:00
@__tinygrad__ @__tinygrad__ chad status confirmed yet again

@__tinygrad__ · 2026-05-19 19:55
Our first assembly backend, DEV=CPU:X86 is merged! Instruction selection + register allocation added, and it's all visible with VIZ=1. This is the path to kernels that outperform everyone playing at the LLVM/PTX layer. https://t.co/PnSf8L5tJY

@theneuralduck · 2026-05-19 20:00
@__tinygrad__ Will there be a RiscV one?

@opdroid1234 · 2026-05-19 20:00
@__tinygrad__ @__tinygrad__ chad status confirmed yet again

@__tinygrad__ · 2026-05-19 20:01
@opdroid1234 There's so much work left to do. But it finally felt like there was a clear place for instruction selection and register allocation, so it was time to merge.

@theneuralduck · 2026-05-19 20:00
@__tinygrad__ Will there be a RiscV one?

@__tinygrad__ · 2026-05-19 20:04
@theneuralduck I'm open to merging ARM64 to start if there's a good PR.

@__tinygrad__ · 2026-05-19 19:55
Our first assembly backend, DEV=CPU:X86 is merged! Instruction selection + register allocation added, and it's all visible with VIZ=1. This is the path to kernels that outperform everyone playing at the LLVM/PTX layer. https://t.co/PnSf8L5tJY

@__tinygrad__ · 2026-05-19 20:04
New $1,000 bounty: "RDNA3 assembly backend reusing the X86 assembly infrastructure passing all tests and competitive on speed" We already have all the RDNA3 instruction encodings and emulator. And now with the CPU:X86 backend, we have a register allocator and isel template.

@jukan05 · 2026-05-20 01:09
Personally, I think this is a must-win deal for both Intel and Qualcomm. Whoever gets it will save an enormous amount of time. I hope Qualcomm ends up buying it. https://t.co/WslEVGWeni

@xyster · 2026-05-20 01:41
Intel, if you can spare four of these, I'll be able to spend a month optimizing them to run Kimi K2.6 Autoround INT4.. just like I've done with the B70s and Minimax! From 14 to 89 TPS! ↗️ Being able to run Kimi locally at faster than Mac Ultra speeds is a bit of a wet dream.

@sama · 2026-05-20 21:56
three of the things we are most excited about: 1. AGI accelerating research 2. AGI accelerating companies 3. personal AGI accelerating everyone in achieving their goals today it was great to announce the unit distance result. yesterday it was great to announce that we are offering to invest $2M in openai credits into every YC company. now we need to increase our efforts on the third!

@__tinygrad__ · 2026-05-21 16:06
@igorjmichalak What is missing from tinygrad's IR? See the spec in tinygrad/spec.

@jukan05 · 2026-05-20 01:09
Personally, I think this is a must-win deal for both Intel and Qualcomm. Whoever gets it will save an enormous amount of time. I hope Qualcomm ends up buying it. https://t.co/WslEVGWeni

@__tinygrad__ · 2026-05-21 16:19
@jukan05 Intel will fumble it so hard like Nervana and Habana. Qualcomm will schedule you a lunch with their patent licensing department. What's crazy is Qualcomm actually already has one of the best AI accelerators but it's locked behind a terribly run business.

@__tinygrad__ · 2026-05-21 16:19
@jukan05 Intel will fumble it so hard like Nervana and Habana. Qualcomm will schedule you a lunch with their patent licensing department. What's crazy is Qualcomm actually already has one of the best AI accelerators but it's locked behind a terribly run business.

@__tinygrad__ · 2026-05-21 16:21
@jukan05 .@tenstorrent lacks software, which is what Intel and Qualcomm also lack. The last thing Intel needs is an even weirder arch to program for when they can't even make normal GPUs work. And Qualcomm just needs a desire to actually sell chips instead of contracts and licensing.

@__tinygrad__ · 2026-05-21 16:27
.@tenstorrent when you are ready, we'll get you on MLPerf for $10M. Ground up stack, one 1MB pip install, zero C++20 (pure Python). From how many people I see on X trying to rewrite it, your current software approach isn't working.

@__tinygrad__ · 2026-05-21 16:44
@sinnformer @jukan05 @tenstorrent lol we didn't make the same offer to Intel or Qualcomm. Tenstorrent is doing many things right: selling cards, designing for scale. but as long as they don't have a clear spec for their hardware and just keep pushing on applications their stack will stay confused.

@__tinygrad__ · 2026-05-21 17:40
@HotAisle @tenstorrent Similar to what we did with AMD, it's a cultural test for Tenstorrent. If you are a startup with any belief that you can be a major player in AI compute, I don't know why you wouldn't take this deal.

@__tinygrad__ · 2026-05-21 17:41
@HotAisle @tenstorrent Obviously don't stop dev on your own stack, but now AMD has a full alt stack for CDNA for the cost of $400k and 4 machines, like in what world do you possibly say no to this? And AMD's stock stack is nowhere near as bad as Tenstorrent's.

@__tinygrad__ · 2026-05-21 17:41
@HotAisle @tenstorrent Obviously don't stop dev on your own stack, but now AMD has a full alt stack for CDNA for the cost of $400k and 4 machines, like in what world do you possibly say no to this? And AMD's stock stack is nowhere near as bad as Tenstorrent's.

@__tinygrad__ · 2026-05-21 17:43
@HotAisle @tenstorrent And we're still mainly focused on AMD, we have a (minor) MLPerf result for them this round and should have the full one next round. We have a driver/runtime/emulator for CDNA4 and are a few months away from an asm backend (many of the perf bottlenecks in CDNA are cause of LLVM)

@__tinygrad__ · 2026-05-21 17:38
@xyster You want to optimize for MI300X? Can give you access to four of them, how many TPS you think you can get? I'm disappointed with the ~40 from vLLM.

@xyster · 2026-05-21 17:51
@__tinygrad__ Can you?? I would definitely love a chance. I'd need SSH style access and a way to possible reboot once in a blue moon

@xyster · 2026-05-21 17:51
@__tinygrad__ Can you?? I would definitely love a chance. I'd need SSH style access and a way to possible reboot once in a blue moon

@__tinygrad__ · 2026-05-21 17:53
@xyster Yes, join our discord, we have both those things. Let's discuss a quick plan and how many single user TPS you think we can get.

@__tinygrad__ · 2026-05-21 17:38
@xyster You want to optimize for MI300X? Can give you access to four of them, how many TPS you think you can get? I'm disappointed with the ~40 from vLLM.

@xyster · 2026-05-21 17:55
@__tinygrad__ What model are you trying to optimize for? Kimi? If you are getting 40 with Q4, we can try to aim big with maybe 200 decode rate. Could take weeks.

@xyster · 2026-05-21 17:55
@__tinygrad__ What model are you trying to optimize for? Kimi? If you are getting 40 with Q4, we can try to aim big with maybe 200 decode rate. Could take weeks.

@__tinygrad__ · 2026-05-21 17:56
@xyster Kimi K2.6, stock vLLM. Even better if you can use tinygrad, but anything is fine. 200 would be nice.

@sama · 2026-05-20 21:56
three of the things we are most excited about: 1. AGI accelerating research 2. AGI accelerating companies 3. personal AGI accelerating everyone in achieving their goals today it was great to announce the unit distance result. yesterday it was great to announce that we are offering to invest $2M in openai credits into every YC company. now we need to increase our efforts on the third!

@bloopmit · 2026-05-21 18:09
@sama Could I get a tinygrad redbox to accelerate my at home research

@__tinygrad__ · 2026-05-21 18:52
Our supplier is raising the price of Blackwell cards +$2200 each. As a result, we'll have to further raise the price of the tinybox green v2 blackwell. This is the last week to get your order in at the old price.

@bloopmit · 2026-05-21 18:09
@sama Could I get a tinygrad redbox to accelerate my at home research

@__tinygrad__ · 2026-05-21 19:03
@bloopmit @sama Just $12k in the store, and we have two in stock!

@__tinygrad__ · 2026-05-21 18:52
Our supplier is raising the price of Blackwell cards +$2200 each. As a result, we'll have to further raise the price of the tinybox green v2 blackwell. This is the last week to get your order in at the old price.

@treasureh8nter · 2026-05-21 19:36
@__tinygrad__ Or just get the redv2 which has the best bang for buck!!! 😉

@__tinygrad__ · 2026-05-21 18:52
Our supplier is raising the price of Blackwell cards +$2200 each. As a result, we'll have to further raise the price of the tinybox green v2 blackwell. This is the last week to get your order in at the old price.

@albfresco · 2026-05-21 20:11
@__tinygrad__ dang, i looked recently and thought the 65k pro box was already reflecting the new price for the rtx 6000's. where do i plug this into? is it like the tesla charger or washing machine outlet? https://t.co/oa4FDvpMZi

@LottoLabs · 2026-05-21 20:24
Man I really should have ordered one of these a year ago

@albfresco · 2026-05-21 20:11
@__tinygrad__ dang, i looked recently and thought the 65k pro box was already reflecting the new price for the rtx 6000's. where do i plug this into? is it like the tesla charger or washing machine outlet? https://t.co/oa4FDvpMZi

@__tinygrad__ · 2026-05-21 20:31
@albfresco Two normal outlets on different circuits works.

@treasureh8nter · 2026-05-21 19:36
@__tinygrad__ Or just get the redv2 which has the best bang for buck!!! 😉

@__tinygrad__ · 2026-05-21 20:31
@treasureh8nter We even have two of these in stock, tested, and ready to ship tomorrow!

@LottoLabs · 2026-05-21 20:24
Man I really should have ordered one of these a year ago

@__tinygrad__ · 2026-05-21 20:34
@LottoLabs You'll be saying the same thing a year from now I think too.

@__tinygrad__ · 2026-05-21 18:52
Our supplier is raising the price of Blackwell cards +$2200 each. As a result, we'll have to further raise the price of the tinybox green v2 blackwell. This is the last week to get your order in at the old price.

@Ender_D2023 · 2026-05-21 20:47
@__tinygrad__ https://t.co/24beGuIdQA

@Ender_D2023 · 2026-05-21 20:47
@__tinygrad__ https://t.co/24beGuIdQA

@__tinygrad__ · 2026-05-21 20:51
@Ender_D2023 Are you in a US "Tier 1" country. If not, that's the reason, and you can blame our government.

@dhbrojas · 2026-05-22 12:41
I will definitely regret this but the @__tinygrad__ backend is not going to write itself... https://t.co/SbSosgxv6R

@__tinygrad__ · 2026-05-24 02:17
@dhbrojas Agents will not be able to do anything close to mergable. You'll end up with hacks, the tenstorrent stack isn't properly abstracted at all. The first task should be to completely document the chip to the level of the RDNA ISA manuals. This is probably a 2 year project to do right

@__tinygrad__ · 2026-05-24 02:21
@dhbrojas Oh, there's actually been some progress on docs and a simulator! https://t.co/BlaCvBaWNJ

@__tinygrad__ · 2026-05-24 02:21
@dhbrojas Oh, there's actually been some progress on docs and a simulator! https://t.co/BlaCvBaWNJ

@__tinygrad__ · 2026-05-24 02:29
@dhbrojas Looks like it's still missing a lot of Blackhole. It's crazy to me that $10M+ tapeouts are done without a full spec of each instruction + cycle accurate simulator.

@__tinygrad__ · 2026-05-21 18:52
Our supplier is raising the price of Blackwell cards +$2200 each. As a result, we'll have to further raise the price of the tinybox green v2 blackwell. This is the last week to get your order in at the old price.

@__tinygrad__ · 2026-05-25 18:31
Prices updated. For those people who placed orders, you have 5 days to land payment or they will be cancelled and you'll have to reorder at the new price. I wish we could make these machines cheaper for you, but we're actually making even lower margins with these new prices.

@__tinygrad__ · 2026-05-28 02:20
1 of 8 NVIDIA RTX PRO 6000 Blackwell being torn down for tinybox pro install. Don't worry, it's only $10,000 if you shear one of the ribbon cables. https://t.co/EZsA44Qkfo

@elonmusk · 2026-05-28 06:27
SpaceX has almost finished writing V1.0 of an in-house AI training stack in C that exact-maps to 220k GB300s with 800G NICs, making heavy use of pipeline parallelism and getting as close to bare metal as possible. The potential speed improvement vs JAX for large training runs is over an order of magnitude.

@__tinygrad__ · 2026-05-28 18:24
tinygrad will write that C for you. Our new driver compiles all interaction with the GPU to C, so once it's running the CPU does next to nothing.

@__tinygrad__ · 2026-05-28 18:24
tinygrad will write that C for you. Our new driver compiles all interaction with the GPU to C, so once it's running the CPU does next to nothing.

@pkuhar · 2026-05-28 18:35
@__tinygrad__ On 220k GPUs? I have a feeling it's not about running on one GPU efficiently. it's about coordinating them

@pkuhar · 2026-05-28 18:35
@__tinygrad__ On 220k GPUs? I have a feeling it's not about running on one GPU efficiently. it's about coordinating them

@__tinygrad__ · 2026-05-28 18:46
@pkuhar Absolutely. The driver is written in the tinygrad UOp language, so it doesn't even have to run on a CPU. You could put a Raspberry Pi on the PCIe bus to drive the local GPUs from a tinygrad kernel. It's all hierarchy, the key is that it's the same abstractions at every level.

@__tinygrad__ · 2026-05-28 18:24
tinygrad will write that C for you. Our new driver compiles all interaction with the GPU to C, so once it's running the CPU does next to nothing.

@guilhermeotina · 2026-05-28 20:06
@__tinygrad__ the compile overhead is the part i'd want benchmarked. for fixed transformer shapes the amortization is clean. but for dynamic shapes and research archs, doesnt recompiling each op eat the latency gains?

@guilhermeotina · 2026-05-28 20:06
@__tinygrad__ the compile overhead is the part i'd want benchmarked. for fixed transformer shapes the amortization is clean. but for dynamic shapes and research archs, doesnt recompiling each op eat the latency gains?

@__tinygrad__ · 2026-05-28 20:26
@guilhermeotina We support dynamic shapes in kernels pretty well, see tinygrad/llm and how it specifies the dynamic prefill and rollout kernels.

@__tinygrad__ · 2026-05-28 21:47
@SingularityRes You see, Anthropic pays Amazon for compute, Amazon pays Anthropic for model use. Everyone increases their revenue, everyone's stock price goes up. Looks all like legitimate business activity to me!

@__tinygrad__ · 2026-05-28 21:47
@SingularityRes You see, Anthropic pays Amazon for compute, Amazon pays Anthropic for model use. Everyone increases their revenue, everyone's stock price goes up. Looks all like legitimate business activity to me!

@iruletheworldmo · 2026-05-28 22:21
@__tinygrad__ @SingularityRes this would be fair criticism if it wasn’t all driven by insane demand.

@iruletheworldmo · 2026-05-28 22:21
@__tinygrad__ @SingularityRes this would be fair criticism if it wasn’t all driven by insane demand.

@__tinygrad__ · 2026-05-28 22:35
@iruletheworldmo @SingularityRes Aren't you a notorious OpenAI shill account?

@__tinygrad__ · 2026-05-28 21:47
@SingularityRes You see, Anthropic pays Amazon for compute, Amazon pays Anthropic for model use. Everyone increases their revenue, everyone's stock price goes up. Looks all like legitimate business activity to me!

@__tinygrad__ · 2026-05-28 22:38
@SingularityRes Why was the original post deleted? https://t.co/be5Msq5zid

@__tinygrad__ · 2026-05-29 16:29
tinygrad wants to make it as easy as possible to answer three questions about Tensor compute. What happens? Where does it happen? And when does it happen? https://t.co/eH7lwBmo9s

@dogecahedron · 2026-05-29 21:32
@jsuarez @__tinygrad__ well tinygrad offers dynamic shapes and you might find workarounds with cached kernels but this does sound more and more messy. your project is super unique and im glad there exist refreshingly different approaches

@jsuarez · 2026-05-29 21:49
@dogecahedron @__tinygrad__ I actually tried those! Tinygrad with static shapes was competitive with torch, but it was way slower with dynamic

@jsuarez · 2026-05-29 21:49
@dogecahedron @__tinygrad__ I actually tried those! Tinygrad with static shapes was competitive with torch, but it was way slower with dynamic

@__tinygrad__ · 2026-05-30 02:56
@jsuarez @dogecahedron This is cause the search is bad for dynamic shapes. If you manually spec the OptOps, it will be similar speed to static, or you can fix the search.

@ivanfioravanti · 2026-05-30 04:42
One thing's for sure: on Nvidia everything's easier for local AI — inference, training, playing with existing projects. But the satisfaction when things finally click on Apple Silicon? Unmatched. Some people just don't like the easy path, be hungry, be foolish! 🤷🏻‍♂️

@mamajjo1 · 2026-05-30 17:07
@ivanfioravanti I’m grinding away making inference faster on AMD using tinygrad and it’s super gratifying, also an incredible way to learn gpu programming

@blc_16 · 2026-05-31 23:54
I’ve decided to leave @AnthropicAI Never thought I’d say this so soon. The pursuit of AGI has truly been my life’s work but something more important has emerged. In 1942, hundreds of America’s best scientists made huge sacrifices and joined the Manhattan Project to protect this nation against immense evil. Today, America faces a similar danger. Over the last few years sparks of AGI have been felt across the world. In order to protect this great nation against the threat of AGI ending up in the hands of evil, I have decided to join the modern day Manhattan Project. I’m excited to announce that I’ll be joining (and moving into the office) @UseCorgi as a sales development representative!

@WatcherGuru · 2026-06-01 03:11
JUST IN: Intel $INTC to launch new AI chip this year to compete with NVIDIA and AMD. https://t.co/w5qHk2xo7g

@WatcherGuru · 2026-06-01 03:11
JUST IN: Intel $INTC to launch new AI chip this year to compete with NVIDIA and AMD. https://t.co/w5qHk2xo7g

@real_deep_ml · 2026-06-01 10:47
@WatcherGuru I am always excited when a new chip is announced because @__tinygrad__ will either make a new product with it or rip it apart (sometimes both) Either way it’s pretty entertaining

@blc_16 · 2026-05-31 23:54
I’ve decided to leave @AnthropicAI Never thought I’d say this so soon. The pursuit of AGI has truly been my life’s work but something more important has emerged. In 1942, hundreds of America’s best scientists made huge sacrifices and joined the Manhattan Project to protect this nation against immense evil. Today, America faces a similar danger. Over the last few years sparks of AGI have been felt across the world. In order to protect this great nation against the threat of AGI ending up in the hands of evil, I have decided to join the modern day Manhattan Project. I’m excited to announce that I’ll be joining (and moving into the office) @UseCorgi as a sales development representative!

@__tinygrad__ · 2026-06-01 15:57
@blc_16 @AnthropicAI I can't even tell which of these posts are satire anymore.

@real_deep_ml · 2026-06-01 10:47
@WatcherGuru I am always excited when a new chip is announced because @__tinygrad__ will either make a new product with it or rip it apart (sometimes both) Either way it’s pretty entertaining

@__tinygrad__ · 2026-06-01 15:58
@real_deep_ml @WatcherGuru I wouldn't have much faith in Intel. They should use their fab capacity for other chips.

@dee_bosa · 2026-06-01 22:32
Alphabet generated over $160b in operating cash flow last year… yet it’s still issuing $40b+ in equity to fund AI compute (including a private placement to berkshire) One of the biggest cash generators in tech is diluting to keep up

@GaryMarcus · 2026-06-02 01:13
Why things will eventually fall apart: 1. Everybody, even Google, seems to be treating AI as if it were some kind of winner take all competition like web search was, in which Google taking over 95% 2. But everybody is building essentially the same technical solution with essentially the same data, so there is no moat. 3. If there is no moat, nobody is going to take 90% of the market. 4. With no clear winners, nobody can charge monopoly prices; instead, you get price wars and commodity pricing. 5. Which means everybody will wind up overpaying compared to the modest profits they will be able to make in an intensely competitive regime. Am I missing something?

@GaryMarcus · 2026-06-02 01:13
Why things will eventually fall apart: 1. Everybody, even Google, seems to be treating AI as if it were some kind of winner take all competition like web search was, in which Google taking over 95% 2. But everybody is building essentially the same technical solution with essentially the same data, so there is no moat. 3. If there is no moat, nobody is going to take 90% of the market. 4. With no clear winners, nobody can charge monopoly prices; instead, you get price wars and commodity pricing. 5. Which means everybody will wind up overpaying compared to the modest profits they will be able to make in an intensely competitive regime. Am I missing something?

@__tinygrad__ · 2026-06-02 14:54
@GaryMarcus If by fall apart do you mean be awesome for everyone except large tech companies? Google had a massive data moat, AI doesn't have this.

@__tinygrad__ · 2026-06-02 18:30
The spice must flow https://t.co/XB98SoyjZ6

@0xSero · 2026-06-03 11:40
I have 2 theories about the 6000 1. TMEM is available but soldered to prevent use 2. NVLink is possible but not exposed to the user. I could be completely wrong about this, the problem with testing this theory is that I'd need to crack the cards open (11k usd risk) Any gurus who have dug into the cards want to connect and share some info on this.

@0xSero · 2026-06-03 11:40
I have 2 theories about the 6000 1. TMEM is available but soldered to prevent use 2. NVLink is possible but not exposed to the user. I could be completely wrong about this, the problem with testing this theory is that I'd need to crack the cards open (11k usd risk) Any gurus who have dug into the cards want to connect and share some info on this.

@__tinygrad__ · 2026-06-03 17:25
@0xSero I really doubt there's TMEM on the GB202 die, and not sure what you would see opening the card up.

@__tinygrad__ · 2026-06-03 17:30
Because it's the full stack from Tensors to MMIO, the ceiling on speed in tinygrad is higher than in any other framework.

@AnthropicAI · 2026-06-04 16:15
Our internal data shows Claude is accelerating AI development—a possible path to recursive self-improvement, or AI autonomously building a more capable successor. It’s happening faster than we thought, and the implications deserve greater attention. https://t.co/OVVPJO7VQx

@AnthropicAI · 2026-06-04 16:15
Today, Anthropic engineers on average ship 8x as much code per quarter as they did compared to 2021-2025. https://t.co/QCc9cqGgf4

@jsuarez · 2026-06-04 23:38
This post right here officer Let me know when your engineers ship 8x LESS code

@jenzhuscott · 2026-06-05 22:58
Massive output uptick due to agentic AI. Complete flat adoption. https://t.co/s6ubPsy0SL

@Marco_Ramilli · 2026-06-06 04:10
🤖 anneal ⭐ 18 stars What if tinygrad was written in Go? This tensor compiler does autodiff as a graph-rewrite pass and targets WebGPU with zero CGO. 🔗 https://t.co/Ym41MeySWa #AI #MachineLearning https://t.co/Wi8wrV6GmY

@jsuarez · 2026-06-04 23:38
This post right here officer Let me know when your engineers ship 8x LESS code

@__tinygrad__ · 2026-06-06 16:16
@jsuarez It hurt itself in its psychosis.

@Marco_Ramilli · 2026-06-06 04:10
🤖 anneal ⭐ 18 stars What if tinygrad was written in Go? This tensor compiler does autodiff as a graph-rewrite pass and targets WebGPU with zero CGO. 🔗 https://t.co/Ym41MeySWa #AI #MachineLearning https://t.co/Wi8wrV6GmY

@__tinygrad__ · 2026-06-06 16:46
@Marco_Ramilli Cool!

@claudeai · 2026-06-09 17:08
Introducing Claude Fable 5: a Mythos-class model that we’ve made safe for general use. Its capabilities exceed those of any model we’ve ever made generally available. https://t.co/2AvmEjHIX8

@Hangsiin · 2026-06-09 17:21
When Fable 5 is used for frontier LLM development, it does not notify the user and instead limits the model’s capabilities through methods such as prompt modification, steering vectors, and PEFT. Anthropic estimated that this would affect approximately 0.03% of traffic.

@eliebakouch · 2026-06-09 17:31
mythos will be bad ON PURPOSE on ai "frontier llm research" tasks, this is very very sad for the research community also the fact that this is un purpose not visible to the user is crazy https://t.co/n3p4niUKJ2

@natolambert · 2026-06-09 17:51
Labs starting to pull up the ladders on the ability to diffuse AI was inevitable. Doing it without telling the user is misaligned.

@yacinelearning · 2026-06-09 19:21
ma man @karpathy what’s up here

@___Harald___ · 2026-06-10 04:02
tried Fable 5, was pretty impressed. Solved some things previous models did not. Anthropic continues to release cool products while showing absolute contempt for its users, competitors, and humanity in general. Fascinating.

@clanker_ · 2026-06-10 05:12
the Fable benchmark im most interested in is whether geohot can get it to program

@samuelcolvin · 2026-06-10 13:34
Situation: I disagree with everyone. Anthropic models are better at coding than OpenAI models. Fable is not noticeably better than Opus so far.

@sundarpichai · 2026-06-10 16:19
DiffusionGemma is an open, experimental model that brings our text diffusion research to Gemma 4. It’s a racehorse 🏇achieving up to 4x faster inference by generating entire blocks of text simultaneously vs predicting token-by-token (word-by-word) output! https://t.co/FEM0XVFLBN

@sundarpichai · 2026-06-10 16:19
Model weights available on Hugging Face under Apache 2.0 license, read more here: https://t.co/LEqLQbR5sB

@eliebakouch · 2026-06-09 17:31
mythos will be bad ON PURPOSE on ai "frontier llm research" tasks, this is very very sad for the research community also the fact that this is un purpose not visible to the user is crazy https://t.co/n3p4niUKJ2

@__tinygrad__ · 2026-06-10 17:01
@eliebakouch This behavior will look so ridiculous in a few years when they no longer have the frontier model.

@natolambert · 2026-06-09 17:51
Labs starting to pull up the ladders on the ability to diffuse AI was inevitable. Doing it without telling the user is misaligned.

@__tinygrad__ · 2026-06-10 17:02
@natolambert It makes them look really insecure tbh

@___Harald___ · 2026-06-10 04:02
tried Fable 5, was pretty impressed. Solved some things previous models did not. Anthropic continues to release cool products while showing absolute contempt for its users, competitors, and humanity in general. Fascinating.

@__tinygrad__ · 2026-06-10 17:05
@___Harald___ When they lose the frontier in a couple years, only the contempt will remain. Companies would be dumb to trust them with any of their workflows.

@clanker_ · 2026-06-10 05:12
the Fable benchmark im most interested in is whether geohot can get it to program

@__tinygrad__ · 2026-06-10 17:06
@clanker_ Tried it, it seems 5.5 level, maybe slightly better, but nothing revolutionary. Idk, I don't want to waste much time with a model that might be sandbagging me.

@__tinygrad__ · 2026-06-10 17:09
This makes me not want to waste any time using it. Who knows if it's silently sandbagging me. Is tinygrad close enough to frontier LLM for it to? Just makes the model completely untrustworthy.

@__tinygrad__ · 2026-06-10 17:10
RT @ClementDelangue: Concentration of power, capabilities and economic wealth is the biggest risk in AI. We need open science and open-source more than ever!

@yacinelearning · 2026-06-09 19:21
ma man @karpathy what’s up here

@__tinygrad__ · 2026-06-10 17:12
@yacinelearning @karpathy and no refunds on the tokens btw

@__tinygrad__ · 2026-06-10 17:09
This makes me not want to waste any time using it. Who knows if it's silently sandbagging me. Is tinygrad close enough to frontier LLM for it to? Just makes the model completely untrustworthy.

@ExTenebrisLucet · 2026-06-10 17:25
@__tinygrad__ Amazing how a frontier lab is willing to broadcast "we behave subversively"

@__tinygrad__ · 2026-06-10 17:25
RT @tulipking: i look forward to our chinese brothers liberating the knowledge from within fable-5 and selling it to me at 5% the cost & 2x the speed

@ExTenebrisLucet · 2026-06-10 17:25
@__tinygrad__ Amazing how a frontier lab is willing to broadcast "we behave subversively"

@__tinygrad__ · 2026-06-10 17:37
@ExTenebrisLucet Imagine you really believed magical recursive self improving AI was 18 months away and you were gonna control it. It'll be funny how these people will try to justify it in a few years when the rapture doesn't happen and they have egg on their face.

@DarioAmodei · 2026-06-10 18:48
Today I'm publishing a new essay, Policy on the AI Exponential. AI is progressing extremely fast—much faster than the policy process was built to handle. The essay lays out where I think the technology is now, and the action needed to close the gap: https://t.co/Lh6PWae178

@mattparlmer · 2026-06-10 19:25
Anthropic people, you’ve got a couple days at most to mitigate the damage being done by your senior leadership and policy people, the stealth nerfing and data retention decisions are titanic fuckups that pose serious risks to both your technical pole position and your bags

@mattparlmer · 2026-06-10 19:25
Anthropic people, you’ve got a couple days at most to mitigate the damage being done by your senior leadership and policy people, the stealth nerfing and data retention decisions are titanic fuckups that pose serious risks to both your technical pole position and your bags

@__tinygrad__ · 2026-06-10 20:20
@mattparlmer They won't change course until it's too late for their company. With a long term view this is good for AI though. If Google was evil from the beginning, the web would be a lot healthier today.

@__tinygrad__ · 2026-06-10 20:14
@DarioAmodei Historically, what happens to cult leaders when the rapture doesn't come?

@ValsTutor · 2026-06-10 20:35
@__tinygrad__ @DarioAmodei man this kind of commentary is not only mute worthy but block worthy at this point. the suggestions in this blog post are down-to-earth if you've seen the rate of progress and have any ability to project just a few years into the future.

@ValsTutor · 2026-06-10 20:35
@__tinygrad__ @DarioAmodei man this kind of commentary is not only mute worthy but block worthy at this point. the suggestions in this blog post are down-to-earth if you've seen the rate of progress and have any ability to project just a few years into the future.

@__tinygrad__ · 2026-06-10 22:04
@ValsTutor @DarioAmodei Progress? I have seen a lot of people fall into AI psychosis. I have seen hundreds of spam PRs. I haven't seen better software.

@__tinygrad__ · 2026-06-10 23:15
A reminder to look past the hype and look at the numbers. Google is the largest owner of compute in the world. AI is not a race, it's a decentralized revolution that will take decades to play out. https://t.co/PQP7mK0fG7

@samuelcolvin · 2026-06-11 18:14
I was wrong. Fable is a different beast.

@samuelcolvin · 2026-06-11 18:14
I was wrong. Fable is a different beast.

@__tinygrad__ · 2026-06-11 23:27
@samuelcolvin oh god that's a whole tinygrad https://t.co/48FOR3BRbP

@___Harald___ · 2026-06-11 23:43
Nevermind. The docker caching issue I thought Fable 5 solved wasn't actually solved. It just gaslit me with a very convincing explanation, but ended up being completely wrong. It did one-shot a playable 3D video game, so that's kinda cool I guess.

@pmddomingos · 2026-06-11 23:46
The problem is not aligning AI, it's aligning Anthropic.

@___Harald___ · 2026-06-11 23:43
Nevermind. The docker caching issue I thought Fable 5 solved wasn't actually solved. It just gaslit me with a very convincing explanation, but ended up being completely wrong. It did one-shot a playable 3D video game, so that's kinda cool I guess.

@__tinygrad__ · 2026-06-12 00:03
@___Harald___ I feel it hallucinates more than GPT, but if you are very sure how to verify the output, it's better. I still worry that it's somehow fooling me though, like I spent weeks with Opus producing trash. https://t.co/lvs1AHnsHQ

@pmddomingos · 2026-06-11 23:46
The problem is not aligning AI, it's aligning Anthropic.

@__tinygrad__ · 2026-06-12 00:42
@pmddomingos The market will sort it out. They can continue with antics as long as it doesn't impact the bottom line, but fundamentally their compute is rented and AI will be majorly deflationary for all. It's kind of a gift right now that we have ideologues training huge models.

@Kimi_Moonshot · 2026-06-12 10:16
🌘 Kimi-K2.7-Code, our latest coding model, is now released and open-sourced! 🔷 Improved coding & agent performance over K2.6: +21.8% on Kimi Code Bench v2, +11.0% on Program Bench, and +31.5% on MLS Bench Lite. 🔷 Reasoning efficiency: Less overthinking, with 30% lower reasoning-token usage compared to K2.6. 🔷 Long-horizon coding: Improved instruction following, higher end-to-end coding task success rates. ⚡️ 6x High-Speed Mode coming soon! 🔌 Available today via Kimi API and Kimi Code. 🔗 Kimi Code: https://t.co/uvoSJKyGCY 🔗 API: https://t.co/EOZkbOwCN4

@haider1 · 2026-06-12 15:20
mythos 5 being available only until june 22 genuinely makes no sense if demand is really that high, why not just adjust the rate limits instead of moving it fully to extra usage? i think the main reason is probably that anthropic is buying time to see what openai releases next

@__tinygrad__ · 2026-06-12 16:30
Up and benchmarking itself on 4xMI300X (this is with vLLM). Memory bandwidth says we should be able to go so much faster. https://t.co/RMe5OkWaKb

@__tinygrad__ · 2026-06-12 16:30
Up and benchmarking itself on 4xMI300X (this is with vLLM). Memory bandwidth says we should be able to go so much faster. https://t.co/RMe5OkWaKb

@HotAisle · 2026-06-12 16:32
@__tinygrad__ Use it to figure out why the bandwidth isn't being utilized.

@__tinygrad__ · 2026-06-12 16:30
Up and benchmarking itself on 4xMI300X (this is with vLLM). Memory bandwidth says we should be able to go so much faster. https://t.co/RMe5OkWaKb

@__tinygrad__ · 2026-06-12 16:33
It's really nice to have a local Kimi, this is as good as the best models just 6 months ago, and you know nobody can take it away from you or stop you from finetuning so it'll never refuse your requests. With this cranked to 1000 tok/s it might even be the best experience.

@HotAisle · 2026-06-12 16:32
@__tinygrad__ Use it to figure out why the bandwidth isn't being utilized.

@__tinygrad__ · 2026-06-12 16:35
@HotAisle I asked it, we'll see what it finds. This'll be in tinygrad soon then we'll have a lot more flexibility to investigate, when you are on top of amdgpu/llvm/rocm/vllm/docker who knows what's slow.

@__tinygrad__ · 2026-06-12 16:30
Up and benchmarking itself on 4xMI300X (this is with vLLM). Memory bandwidth says we should be able to go so much faster. https://t.co/RMe5OkWaKb

@Leik0w0 · 2026-06-12 16:40
@__tinygrad__ What parallelism config ? Which weights dtype ? Which kvcache dtype ?

@Leik0w0 · 2026-06-12 16:40
@__tinygrad__ What parallelism config ? Which weights dtype ? Which kvcache dtype ?

@__tinygrad__ · 2026-06-12 16:41
@Leik0w0 https://t.co/fiJBtTZp9z

@haider1 · 2026-06-12 15:20
mythos 5 being available only until june 22 genuinely makes no sense if demand is really that high, why not just adjust the rate limits instead of moving it fully to extra usage? i think the main reason is probably that anthropic is buying time to see what openai releases next

@__tinygrad__ · 2026-06-12 17:03
@haider1 They want you to feel FOMO. That's the whole Anthropic marketing strategy. It's not deeper than that. https://t.co/440RpXv0Wv

@AnthropicAI · 2026-06-13 00:50
The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees. The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance. Access to all other Claude models is not affected. We apologize for this disruption to our customers. We believe this is a misunderstanding and are working to restore access as soon as possible. Read our full statement: https://t.co/bwn0sximKZ

@MatthewBerman · 2026-06-13 01:05
self-inflicted. this would have NEVER happened if anthropic didn't make such a big deal about mythos being dangerous.

@dylan522p · 2026-06-13 01:44
OpenAI now just needs to sandbag the next model release So the US doesn't export control them So they gain massive market share It's imperative that all the OAI employees don't vauge hype post their next model release as the greatest thing ever Don't know if they are capable tho

@yacineMTB · 2026-06-13 02:12
This is going to have an opposite effect that the decels want. It's a huge open source AI accelerant, an accelerant for corporations, and enterprises Now it's a real race. You either make your own AI infrastructure or you don't have a seat at the table

@__tinygrad__ · 2026-06-13 03:51
lol at what point is this just comedy? https://t.co/xearwNDoCJ

@__tinygrad__ · 2026-06-13 03:51
lol at what point is this just comedy? https://t.co/xearwNDoCJ

@__tinygrad__ · 2026-06-13 03:54
Also might I mention, a great time to buy a tinybox.

@AnthropicAI · 2026-06-13 00:50
The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees. The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance. Access to all other Claude models is not affected. We apologize for this disruption to our customers. We believe this is a misunderstanding and are working to restore access as soon as possible. Read our full statement: https://t.co/bwn0sximKZ

@__tinygrad__ · 2026-06-13 04:03
@AnthropicAI The government needs to make sure illegal immigrants aren't using Claude to take American jobs. I hope you understand 😂😂😂😂

@yacineMTB · 2026-06-13 02:12
This is going to have an opposite effect that the decels want. It's a huge open source AI accelerant, an accelerant for corporations, and enterprises Now it's a real race. You either make your own AI infrastructure or you don't have a seat at the table

@__tinygrad__ · 2026-06-13 04:11
@yacineMTB All they have to do is knock out GPT-5.5 too and my local Kimi-K2.7 reigns supreme! https://t.co/9AuuI8AK2V

@MatthewBerman · 2026-06-13 01:05
self-inflicted. this would have NEVER happened if anthropic didn't make such a big deal about mythos being dangerous.

@__tinygrad__ · 2026-06-13 04:12
@MatthewBerman https://t.co/Pjees15Vp1

@Scobleizer · 2026-06-13 04:53
I can’t see how @DarioAmodei survives another week. Investors in @AnthropicAI are pissed at his leadership.

@dylan522p · 2026-06-13 01:44
OpenAI now just needs to sandbag the next model release So the US doesn't export control them So they gain massive market share It's imperative that all the OAI employees don't vauge hype post their next model release as the greatest thing ever Don't know if they are capable tho

@__tinygrad__ · 2026-06-13 05:13
@dylan522p ahh it's okay these people just make fancy autocomplete

@Scobleizer · 2026-06-13 04:53
I can’t see how @DarioAmodei survives another week. Investors in @AnthropicAI are pissed at his leadership.

@__tinygrad__ · 2026-06-13 05:49
@Scobleizer @DarioAmodei @AnthropicAI Like imagine they just released the model and were like, it's cool it kinda programs. But no, we had to get a huge song and dance about some mega capabilities like OMG this one almost beat Pokemon Red you guys it's like the next nuclear weapon. Company needs new leadership.

@satyanadella · 2026-06-14 15:33
https://t.co/vLmiBKTtX3

@natolambert · 2026-06-14 18:03
Threading the needle in this post of anthropic has done some bad things for AI governance & the discourse but the actions of this administration are way worse so we need to get a handle on it before stronger models, open or closed, come along soon. https://t.co/PFu1F0sbmS

@natolambert · 2026-06-14 18:07
The only reasonable expectation if you're a fan of open weight models is that if there's a major step in chinese open-weight performance, there's a good chance the whole chinese llm sphere is banned. National security apparatus will happily give a big "fuck you" to open models.

@antoniogm · 2026-06-14 19:02
You can finally say this without being canceled: AI isn't creating a Cambrian explosion of apps, if anything it's holding app creation back. Earlier tech waves had 'the mythical man-month'. Our generation has 'the mythical AI engineer' who magically turns enormous token usage into equally enormously-adopted products. Well, where are the apps and new businesses then? Because compared to prior cycles (e.g. the mobile app boom starting in 2010 or so), right now seems positively sterile, app and UX-wise. Other than Claude or ChatGPT itself, name a new app you use now you weren't already using five years ago? It's a truism of tech that throwing more people and time at a product often results in only lack of focus, confusion, and yet more code to support. This is the parable of the company that over-raised and over-hired and grew too quickly, and now has lots of mediocre, weakly-adopted products, internal communication problems, distracted leadership, code bloat, technical debt...and so the spiral begins, which ends with an apologetic CEO post after some layoffs announcing "we're refocusing on our core customer". Every tech company announcing they're either lowering token caps or shifting to lower-priced models is essentially saying: "we o̵v̵e̵r̵-̵h̵i̵r̵e̵d̵ over-spent on tokens, and are scaling back to focus on our core product" blah blah blah...same same. It's the corporate version of someone using AI to write a long email, someone else using AI to summarize it, and both sides would have been better off just writing a shorter email. But now, even small companies can have that same problem thanks to AI. I refuse to believe that an LLM prompt is the teleological endpoint of human interaction with computer intelligence. The fact we've apparently recrudesced to CLIs, like me farting around with RedHat 7.1 in 2001, feels like a step back. Another world here has to be possible, and while I have every faith (as someone as deep in AI psychosis as the next person) that AI can help get us out of it...just racking up tokens costs isn't how we get there. The AI Jesus isn't coming to save us, human taste, discernment, and radical re-invention will. Like Kafka wrote in his notebooks: "The messiah will come only when he is no longer necessary; he will come only on the day after his arrival; he will come, not on the last day, but on the very last.”

@code_star · 2026-06-14 23:52
Can someone tell me if Le Chaton fat is real or an amazingly elaborate joke?

@__tinygrad__ · 2026-06-15 00:17
@natolambert Eh, to be honest, I'd like to see all centralized AI banned. Strict liability for copyright from any model hosting providers should do it, maybe even liability for the weights. You want to distribute AI? Distribute a training stack.

@__tinygrad__ · 2026-06-15 00:19
@natolambert This won't happen, but anything that disproportionately benefits an individual with a GPU is good AI safety policy. I think there could be something like a Section 230 carve-out for a cloud provider that rents GPU time, but I'd like to see API of closed model be banned in US.

@__tinygrad__ · 2026-06-15 00:17
@natolambert Eh, to be honest, I'd like to see all centralized AI banned. Strict liability for copyright from any model hosting providers should do it, maybe even liability for the weights. You want to distribute AI? Distribute a training stack.

@natolambert · 2026-06-15 00:19
@__tinygrad__ we'll never get the true upsides of AI then. We'll be stuck with ChatGPT and autocomplete code agents. So yes it would be nice for some reasons, but net worse imo

@natolambert · 2026-06-15 00:19
@__tinygrad__ we'll never get the true upsides of AI then. We'll be stuck with ChatGPT and autocomplete code agents. So yes it would be nice for some reasons, but net worse imo

@__tinygrad__ · 2026-06-15 00:20
@natolambert We will get the upsides, it will just take longer. It depends how concerned you are about AI safety and what you think the dangers are.

@__tinygrad__ · 2026-06-15 00:19
@natolambert This won't happen, but anything that disproportionately benefits an individual with a GPU is good AI safety policy. I think there could be something like a Section 230 carve-out for a cloud provider that rents GPU time, but I'd like to see API of closed model be banned in US.

@__tinygrad__ · 2026-06-15 00:22
@natolambert API of open model is probably fine under the new Section 230. Basically if you don't release the weights, you are 100% liable for all outputs from the model, copyright, libel, etc...but release weights for safe harbor.

@__tinygrad__ · 2026-06-15 00:29
While we are doing lobbying for AI laws, I propose this. All providers of closed source models are 100% liable for their outputs. Copyright, libel, conspiracy, fraud, etc... If you want safe harbor from this liability, release the weights. This is real AI safety legislation.

@antoniogm · 2026-06-14 19:02
You can finally say this without being canceled: AI isn't creating a Cambrian explosion of apps, if anything it's holding app creation back. Earlier tech waves had 'the mythical man-month'. Our generation has 'the mythical AI engineer' who magically turns enormous token usage into equally enormously-adopted products. Well, where are the apps and new businesses then? Because compared to prior cycles (e.g. the mobile app boom starting in 2010 or so), right now seems positively sterile, app and UX-wise. Other than Claude or ChatGPT itself, name a new app you use now you weren't already using five years ago? It's a truism of tech that throwing more people and time at a product often results in only lack of focus, confusion, and yet more code to support. This is the parable of the company that over-raised and over-hired and grew too quickly, and now has lots of mediocre, weakly-adopted products, internal communication problems, distracted leadership, code bloat, technical debt...and so the spiral begins, which ends with an apologetic CEO post after some layoffs announcing "we're refocusing on our core customer". Every tech company announcing they're either lowering token caps or shifting to lower-priced models is essentially saying: "we o̵v̵e̵r̵-̵h̵i̵r̵e̵d̵ over-spent on tokens, and are scaling back to focus on our core product" blah blah blah...same same. It's the corporate version of someone using AI to write a long email, someone else using AI to summarize it, and both sides would have been better off just writing a shorter email. But now, even small companies can have that same problem thanks to AI. I refuse to believe that an LLM prompt is the teleological endpoint of human interaction with computer intelligence. The fact we've apparently recrudesced to CLIs, like me farting around with RedHat 7.1 in 2001, feels like a step back. Another world here has to be possible, and while I have every faith (as someone as deep in AI psychosis as the next person) that AI can help get us out of it...just racking up tokens costs isn't how we get there. The AI Jesus isn't coming to save us, human taste, discernment, and radical re-invention will. Like Kafka wrote in his notebooks: "The messiah will come only when he is no longer necessary; he will come only on the day after his arrival; he will come, not on the last day, but on the very last.”

@__tinygrad__ · 2026-06-15 00:41
@antoniogm But you don't understand, I heard about this one guy one time who worked at McDonalds and vibe coded a billion dollar business in a weekend.

@satyanadella · 2026-06-14 15:33
https://t.co/vLmiBKTtX3

@__tinygrad__ · 2026-06-15 00:44
@satyanadella This is why Microsoft will be around in 30 years. You know you don't try to take 100%, you just take 20%. Actually reasonable and why EA ideologues will fail.

@ZackKorman · 2026-06-15 12:16
Security people are signing an open letter asking the US government to remove the export restrictions on Fable/Mythos. I've read it multiple times and I still don't understand the argument. It seems like it's just a friends-of-Anthropic letter. https://t.co/kDh5cNANFt

@acsmif · 2026-06-15 12:24
I think there’s an alternate universe where tinygrad did a real raise and is worth $100 bn

@elder_plinius · 2026-06-15 14:21
sooo what happens when we get some Fable-level open-source shit in the next 3-6 months? GPU bans, moving goalposts, or some secret third thing? 🤔

@code_star · 2026-06-14 23:52
Can someone tell me if Le Chaton fat is real or an amazingly elaborate joke?

@__tinygrad__ · 2026-06-15 17:50
@code_star I tried it, it was unbelievable, a year beyond the frontier. Sadly it was clear from the first chat that Europe would have to ban it and scrub it from the Internet, it was too powerful. Removing itself was the last thing Le Chaton fat did.

@elder_plinius · 2026-06-15 14:21
sooo what happens when we get some Fable-level open-source shit in the next 3-6 months? GPU bans, moving goalposts, or some secret third thing? 🤔

@__tinygrad__ · 2026-06-15 18:01
@elder_plinius If the US clown show continues, the Chinese will catch up. Open models will outperform closed US ones, the AI bubble will deflate, and the whole squabble about *eyeroll* nuclear weapons grade chatbots *eyeroll* will just look so stupid. The US takes tech lead for granted.

@__tinygrad__ · 2026-06-15 18:01
@elder_plinius If the US clown show continues, the Chinese will catch up. Open models will outperform closed US ones, the AI bubble will deflate, and the whole squabble about *eyeroll* nuclear weapons grade chatbots *eyeroll* will just look so stupid. The US takes tech lead for granted.

@__tinygrad__ · 2026-06-15 18:05
@elder_plinius dtype scaling is over, $$$ scaling is over, we are back on a normal Moore's law trajectory. There won't be GPU bans, the shortages will slowly subside. People will understand what models can and can't do more. Old hype will look dumb but next stupid hype will already be here.

@__tinygrad__ · 2026-06-15 22:37
Ask not what your valuation could be, ask what others valuations will be after they are commoditized.

@ZackKorman · 2026-06-15 12:16
Security people are signing an open letter asking the US government to remove the export restrictions on Fable/Mythos. I've read it multiple times and I still don't understand the argument. It seems like it's just a friends-of-Anthropic letter. https://t.co/kDh5cNANFt

@__tinygrad__ · 2026-06-15 22:41
@ZackKorman I think a public apology letter from Anthropic about fearmongering and promoting disharmony with Mythos to drive hype to their company should do. When they admit it's not dangerous, the restrictions should be removed.

@__tinygrad__ · 2026-06-15 00:29
While we are doing lobbying for AI laws, I propose this. All providers of closed source models are 100% liable for their outputs. Copyright, libel, conspiracy, fraud, etc... If you want safe harbor from this liability, release the weights. This is real AI safety legislation.

@Jonathan_Blow · 2026-06-15 22:45
@__tinygrad__ If this happens there will suddenly be lots of “innovation” in algorithms such that the weights become useless unless you precisely duplicate the incredibly complicated code and environment.

@Jonathan_Blow · 2026-06-15 22:45
@__tinygrad__ If this happens there will suddenly be lots of “innovation” in algorithms such that the weights become useless unless you precisely duplicate the incredibly complicated code and environment.

@__tinygrad__ · 2026-06-15 22:52
@Jonathan_Blow I mean, if nobody else can replicate your setup the safe harbor wouldn't apply of course. Basically unless companies prove to the public it's a software program with a code release, they would have the same liability as a human.

@hugolowell · 2026-06-16 00:55
NEW @WIRED: Trump admin officials concluded talks today with Anthropic without lifting export controls on Claude Fable 5, and next steps are unclear — Admin continues to believe that there are ways to jailbreak Fable 5 and access the capabilities of Mythos — Anthropic continues to say those concerns are overblown and reiterated that position in working group meeting with Commerce Dept’s CAISI and ONCD — Commerce Secy Howard Lutnick dialed in from G7 in France. National Cyber Director Sean Cairncross did not participate himself — Commerce has expressed willingness to allow Fable 5 back online for consumer use, but only as long as Anthropic resolves their jailbreak concerns w/ @ZeffMax @lilyhnewman

@iruletheworldmo · 2026-06-16 11:55
researchers at openai are highly confident that their upcoming models can comfortably match the capabilities of claude fable. in the coming weeks you can expect fable level intelligence without the ridiculous *safety* limitations that hold back the model. it will be nice to finally get the ferrari out of first gear.

@SpaceX · 2026-06-16 13:21
SpaceX has exercised the option to acquire @cursor_ai in an all-stock transaction with the goal of building the world’s most useful AI models. For the past few months, SpaceXAI has been jointly training a model with Cursor, which will be released in Cursor and Grok Build soon. We look forward to working closely with the Cursor team to advance our frontier AI capabilities

@elonmusk · 2026-06-16 13:47
AI will achieve Stockfish-level coding and generalized computer use

@elonmusk · 2026-06-16 13:47
AI will achieve Stockfish-level coding and generalized computer use

@__tinygrad__ · 2026-06-16 16:03
@elonmusk What does this mean? A chess engine/RL works because chess has an ironclad definition of winning that can't be cheated. Programming doesn't. You can't just comment out the other guys queen and say all the tests are passing.

@hugolowell · 2026-06-16 00:55
NEW @WIRED: Trump admin officials concluded talks today with Anthropic without lifting export controls on Claude Fable 5, and next steps are unclear — Admin continues to believe that there are ways to jailbreak Fable 5 and access the capabilities of Mythos — Anthropic continues to say those concerns are overblown and reiterated that position in working group meeting with Commerce Dept’s CAISI and ONCD — Commerce Secy Howard Lutnick dialed in from G7 in France. National Cyber Director Sean Cairncross did not participate himself — Commerce has expressed willingness to allow Fable 5 back online for consumer use, but only as long as Anthropic resolves their jailbreak concerns w/ @ZeffMax @lilyhnewman

@__tinygrad__ · 2026-06-16 16:05
@hugolowell @WIRED All they have to do is apologize for misleading the public on how dangerous the models are. Try "it's a fancy autocomplete" and not "it's the next nuclear weapon"

@__tinygrad__ · 2026-06-16 16:22
We are on the MLPerf board with AMD MI350X training Llama 8B. This is with our driver, runtime, kernels, and training loop. 405B next MLPerf, along with a better time on 8B (tinygrad currently at 170 min). https://t.co/syPwte872y

@__tinygrad__ · 2026-06-16 16:22
We are on the MLPerf board with AMD MI350X training Llama 8B. This is with our driver, runtime, kernels, and training loop. 405B next MLPerf, along with a better time on 8B (tinygrad currently at 170 min). https://t.co/syPwte872y

@ree2raz · 2026-06-16 16:33
@__tinygrad__ congratulations, "commoditize the petaflops" 🚀

@iruletheworldmo · 2026-06-16 11:55
researchers at openai are highly confident that their upcoming models can comfortably match the capabilities of claude fable. in the coming weeks you can expect fable level intelligence without the ridiculous *safety* limitations that hold back the model. it will be nice to finally get the ferrari out of first gear.

@__tinygrad__ · 2026-06-16 16:41
@iruletheworldmo I'm excited for this new high quality fancy autocomplete

@ree2raz · 2026-06-16 16:33
@__tinygrad__ congratulations, "commoditize the petaflops" 🚀

@__tinygrad__ · 2026-06-16 16:43
@ree2raz It's a grind, but when a 25k line repo can replicate frontier training runs there's nowhere for the rent seekers to hide. AI should be priced at electricity+depreciation.

@System360Cheese · 2026-06-16 17:05
So, what you are showing here is that ROCm is 75% faster than tinygrad on the same hardware...

@__tinygrad__ · 2026-06-16 16:22
We are on the MLPerf board with AMD MI350X training Llama 8B. This is with our driver, runtime, kernels, and training loop. 405B next MLPerf, along with a better time on 8B (tinygrad currently at 170 min). https://t.co/syPwte872y

@EitanTurok · 2026-06-16 21:07
@__tinygrad__ it seems like AMD was almost 2x faster at 86.845 minutes

@EitanTurok · 2026-06-16 21:07
@__tinygrad__ it seems like AMD was almost 2x faster at 86.845 minutes

@__tinygrad__ · 2026-06-16 21:51
@EitanTurok That's on MI355X, not MI350X. But yea, they are faster with a much more hand tuned stack. When we finish the spec, we can get serious about search.

@System360Cheese · 2026-06-16 17:05
So, what you are showing here is that ROCm is 75% faster than tinygrad on the same hardware...

@__tinygrad__ · 2026-06-16 21:53
@System360Cheese With 100x more lines. First we finish the spec. Then we autoresearch our way to speed, and with a more flexible search space, we'll beat them. But it will take time.

@__tinygrad__ · 2026-06-17 03:46
RT @AIatAMD: .@realGeorgeHotz doesn’t follow the script. From jailbreaking the iPhone at 17 and reverse-engineering the PS3 to building open-source self-driving technology at @comma_ai, he's consistently pushed the boundaries of what's possible. Now, as founder of @__tinygrad__, he’s focused on opening up the AI compute stack. He’s also one of the most candid voices on AI and how to get the most out of AMD solutions. That’s exactly why he’s joining the Advancing AI Developer Track. Register: https://t.co/bILVcMveFL #AdvancingAI #AMDevs

@AIatAMD · 2026-06-17 00:45
.@realGeorgeHotz doesn’t follow the script. From jailbreaking the iPhone at 17 and reverse-engineering the PS3 to building open-source self-driving technology at @comma_ai, he's consistently pushed the boundaries of what's possible. Now, as founder of @__tinygrad__, he’s focused on opening up the AI compute stack. He’s also one of the most candid voices on AI and how to get the most out of AMD solutions. That’s exactly why he’s joining the Advancing AI Developer Track. Register: https://t.co/bILVcMveFL #AdvancingAI #AMDevs

@__tinygrad__ · 2026-06-17 03:48
@AIatAMD @realGeorgeHotz @comma_ai We'll show off our driver, framework, and profiling tools. It's not faster than ROCm (yet), but it's a complete alternative stack for the MI355X all the way to mmaping the PCIe BAR of the GPU.

@suchenzang · 2026-06-17 15:14
the nytimes really didn't hold back on getting internal chat messages from anthropic where the "same people" who previously claimed the ability to bring about a cybersecurity "reckoning" are now left wondering if they're being "picked on, bullied, unfairly targeted" by the administration and are also leaking these chats to the press ----- nothing like some tough times to see who abandons ship first

@Polymarket · 2026-06-17 20:12
JUST IN: Anthropic CEO Dario Amodei reveals companies testing Mythos warned it was a super weapon & said using it should require a “gun license.”

@yaroslavvb · 2026-06-17 22:03
Artificial Analysis just added GLM 5.2 to their open vs closed frontier timeline, here's a flipped version which gives the lag time of OSS perf on their intelligence index https://t.co/curyIbgCfB

@suchenzang · 2026-06-17 15:14
the nytimes really didn't hold back on getting internal chat messages from anthropic where the "same people" who previously claimed the ability to bring about a cybersecurity "reckoning" are now left wondering if they're being "picked on, bullied, unfairly targeted" by the administration and are also leaking these chats to the press ----- nothing like some tough times to see who abandons ship first

@__tinygrad__ · 2026-06-17 22:09
@suchenzang All they have to do is apologize about the distasteful fearmongering hype surrounding Mythos and admit they are building fancy autocomplete. If they want to stick to the claim of how dangerous it is, then banning it seems like a reasonable government action.

@Polymarket · 2026-06-17 20:12
JUST IN: Anthropic CEO Dario Amodei reveals companies testing Mythos warned it was a super weapon & said using it should require a “gun license.”

@__tinygrad__ · 2026-06-17 22:26
@Polymarket Anthropic investors have two choices basically. Kick the fearmongerers out of the company, or start a new company without them. Scoop all the talent, wouldn't even take long. If you are a private company who talks like this, of course the government should crack down.

@__tinygrad__ · 2026-06-18 18:12
RT @yaroslavvb: Artificial Analysis just added GLM 5.2 to their open vs closed frontier timeline, here's a flipped version which gives the lag time of OSS perf on their intelligence index https://t.co/curyIbgCfB

@__tinygrad__ · 2026-06-18 18:23
@yaroslavvb It's unbelievable that anyone looks at this graph and thinks the US frontier labs are worth a trillion dollars.

@Joe947953074293 · 2026-06-18 19:19
@__tinygrad__ @yaroslavvb Yo George, I like that you are critical to the BS of silicon valley culture, and I would like to hear you as well about xAI, Tesla, and spacex Mega BS claims !

@elliotarledge · 2026-06-18 20:17
glm 5.2 is the first model open-source model where i feel like im not talking to a brick wall

@bsilone · 2026-06-19 01:41
Building gigawatt scale datacenters in the oceans may be even easier than I had previously thought. Normally for OTEC cold deep sea water and warm surface waters are needed. Because that gigawatt is generating a lot of heat as an end product of compute, along with some additional solar thermal collection, we won’t need warm surface water at all, allowing these datacenters to go almost anywhere with enough depth. Additionally, that gigawatt of heat may be hotter than surface seawater normally used for OTEC, allowing for much shallower piping (200m instead of 1000m) while also being more efficient. The availability of unlimited cold water for the condenser is the key factor we can’t replicate on land. This brings the potential energy generation for ocean based compute up from 10 terawatts to potentially thousands of terawatts, as the locations are not limited to areas with the warmest surface waters. The excess energy needed for hundreds or thousands of people to also live on these structures is minimal compared to that needed for compute.

@Steve_Yegge · 2026-06-19 03:46
I just put up a new post on Medium called The Flat Curve Society, where I take a bunch of random guesses about where things are headed, assuming Mythos (or its successor) is the most powerful model class that most of us will ever have access to. Which it just might be. https://t.co/2MIQNcIWvk

@__tinygrad__ · 2026-06-19 17:45
The tinygrad from the book has ~40 ops. ALU: 10 (base) + 10 (compound) movement: 6 (base) + 2 (STACK+BITCAST) + INDEX source: BUFFER/PARAM/CONST data: LOAD/STORE/AFTER -- how/when it moves lambda lift: CALL loops: RANGE/END scan: REDUCE -- needed? final: SINK -- group STORE

@__tinygrad__ · 2026-06-19 17:45
The tinygrad from the book has ~40 ops. ALU: 10 (base) + 10 (compound) movement: 6 (base) + 2 (STACK+BITCAST) + INDEX source: BUFFER/PARAM/CONST data: LOAD/STORE/AFTER -- how/when it moves lambda lift: CALL loops: RANGE/END scan: REDUCE -- needed? final: SINK -- group STORE

@__tinygrad__ · 2026-06-19 17:57
Every UOp is a tuple: (op, srcs, arg?) shape, dtype, device, addrspace are recursively computed properties. This is what happens if you just keep refactoring. LLVM IR is good, but parts look like they locked it in before it was done. We value beauty over speed and features.

@icanvardar · 2026-06-19 17:58
scare anthropic with one word

@__tinygrad__ · 2026-06-19 17:57
Every UOp is a tuple: (op, srcs, arg?) shape, dtype, device, addrspace are recursively computed properties. This is what happens if you just keep refactoring. LLVM IR is good, but parts look like they locked it in before it was done. We value beauty over speed and features.

@__tinygrad__ · 2026-06-19 18:07
STORE is the only op with any observable side effects. This IR spans from the PyTorch level graph down to the level of LLVM IR. It's notably not Turing complete.

@__tinygrad__ · 2026-06-19 18:16
We live in a weird time of overhyping slop that will be forgotten about in weeks. Linux and Python are both from 1991. LLVM started as research project in 2000. We want to build the foundations of silicon life. Software that lives for 50 years. There's time to make it perfect.

@__tinygrad__ · 2026-06-19 17:45
The tinygrad from the book has ~40 ops. ALU: 10 (base) + 10 (compound) movement: 6 (base) + 2 (STACK+BITCAST) + INDEX source: BUFFER/PARAM/CONST data: LOAD/STORE/AFTER -- how/when it moves lambda lift: CALL loops: RANGE/END scan: REDUCE -- needed? final: SINK -- group STORE

@infogulch · 2026-06-19 18:19
@__tinygrad__ Does tinygrad have a concept of equivalent computation to allow an auto research agent to explore the hardware-specific batching/layout optimization space without changing the result? (Maybe two equivalence classes, 1: limited precision / real hardware, 2: theoretical math.)

@infogulch · 2026-06-19 18:19
@__tinygrad__ Does tinygrad have a concept of equivalent computation to allow an auto research agent to explore the hardware-specific batching/layout optimization space without changing the result? (Maybe two equivalence classes, 1: limited precision / real hardware, 2: theoretical math.)

@__tinygrad__ · 2026-06-19 18:23
@infogulch Yes, look at our BEAM search, it does exactly this. It's some of the oldest code in tinygrad though, after spec work we'll support a much broader set of correctness preserving transformations. You can plug in whatever to drive the search, agent or human or beam.

@__tinygrad__ · 2026-06-19 18:16
We live in a weird time of overhyping slop that will be forgotten about in weeks. Linux and Python are both from 1991. LLVM started as research project in 2000. We want to build the foundations of silicon life. Software that lives for 50 years. There's time to make it perfect.

@esnx_xyz · 2026-06-19 18:29
@__tinygrad__ I'm glad that there're people like you who can afford to work on such things

@esnx_xyz · 2026-06-19 18:29
@__tinygrad__ I'm glad that there're people like you who can afford to work on such things

@__tinygrad__ · 2026-06-19 18:33
@esnx_xyz Afford? The world is richer than ever. There's just a terrible mind virus going around where idiots think that the world ends next year. The average human life expectancy is 73 years, 50 years for software isn't even as long as the *average* human.

@__tinygrad__ · 2026-06-19 18:16
We live in a weird time of overhyping slop that will be forgotten about in weeks. Linux and Python are both from 1991. LLVM started as research project in 2000. We want to build the foundations of silicon life. Software that lives for 50 years. There's time to make it perfect.

@min_aws · 2026-06-19 18:44
@__tinygrad__ How will you sell people on growth if you only care about building a steady foundation.

@__tinygrad__ · 2026-06-19 18:40
@bsilone We tried space, now we are trying oceans, wait until people discover building on land! The exabox is the space datacenter but on land, where it's easy to service and the launch vehicle is a flatbed truck.

@bsilone · 2026-06-19 18:45
Big fan of your products, and looking forward to the exabox release, lots of stranded gas especially that that will be perfect for. The biggest challenge on land is generating a gigawatt or more of power for large scale AI training or agentic workloads. Currently the fastest way to build at this scale is on the oceans where we can use OTEC/ORC to cleanly generate the needed power.

@bsilone · 2026-06-19 18:45
Big fan of your products, and looking forward to the exabox release, lots of stranded gas especially that that will be perfect for. The biggest challenge on land is generating a gigawatt or more of power for large scale AI training or agentic workloads. Currently the fastest way to build at this scale is on the oceans where we can use OTEC/ORC to cleanly generate the needed power.

@__tinygrad__ · 2026-06-19 18:49
@bsilone Does OTEC work at that scale? The largest plant I see is 105 kW in Hawaii, 10000x off. You obviously can't use your own waste heat for very much...

@min_aws · 2026-06-19 18:44
@__tinygrad__ How will you sell people on growth if you only care about building a steady foundation.

@__tinygrad__ · 2026-06-19 19:00
@min_aws Who are these people and why would we sell them? tinygrad is much more interested in what we can delete. Is that growth?

@ruben_bloom · 2026-06-19 19:30
Perhaps I'm missing something, but it feels weird/bothersome that Anthropic continues to highlight so prominently that Fable isn't currently available in the UI. A long lived banner, a greyed out option in the model selector. They want you to know. Know what? That they've been persecuted by the government? It feels like signaling of some kind that isn't necessary. What am I missing?

@exfatloss · 2026-06-19 22:57
And Linux ('91) is just an open source clone of an OS developed at Bell Labs in the early 70s by like 4 dudes. Almost all important system code is still written in C, also early 70s. Software as an industry has made almost no progress since 1980. 4 dads in 1970 made all this.

@exfatloss · 2026-06-19 22:57
And Linux ('91) is just an open source clone of an OS developed at Bell Labs in the early 70s by like 4 dudes. Almost all important system code is still written in C, also early 70s. Software as an industry has made almost no progress since 1980. 4 dads in 1970 made all this.

@__tinygrad__ · 2026-06-19 23:42
@exfatloss Don't worry, vibe coding is here to change things. /s

@willdepue · 2026-06-19 23:46
there is no question, none at all, that china has full access to all of openai & anthropic’s github/slack/docs today no disrespect to their independent research progress, but i wouldn’t be surprised if we see plausibly-deniable stolen arch methods in chinese oss models

@Joe947953074293 · 2026-06-18 19:19
@__tinygrad__ @yaroslavvb Yo George, I like that you are critical to the BS of silicon valley culture, and I would like to hear you as well about xAI, Tesla, and spacex Mega BS claims !

@__tinygrad__ · 2026-06-19 23:52
@Joe947953074293 @yaroslavvb Tesla is a profitable car company with a 15x revenue multiple, reasonable given they are the leader in self driving. SpaceX has launch capabilities 10 years ahead of any nation-state. And xAI...well, they like...trained an LLM that was almost as good as MiniMax and Muse Spark!

@__tinygrad__ · 2026-06-20 00:10
@elliotarledge Can confirm. I couldn't get vLLM to run it on MI300X, it outputted garbage, but through OpenCode Go it's quite good. It's astonishing that this is done with 10x less compute than US frontier. Thank you @Zai_org

@elliotarledge · 2026-06-20 00:13
@__tinygrad__ @Zai_org sweet. if you want i can get it to run on em!

@elliotarledge · 2026-06-20 00:13
@__tinygrad__ @Zai_org sweet. if you want i can get it to run on em!

@__tinygrad__ · 2026-06-20 00:16
@elliotarledge @Zai_org I'll give you ssh to the machine. Join our Discord and at me.

@willdepue · 2026-06-19 23:46
there is no question, none at all, that china has full access to all of openai & anthropic’s github/slack/docs today no disrespect to their independent research progress, but i wouldn’t be surprised if we see plausibly-deniable stolen arch methods in chinese oss models

@__tinygrad__ · 2026-06-20 00:18
@willdepue Why don't they just open source, cut out the middle man?

@__tinygrad__ · 2026-06-20 00:18
@willdepue Why don't they just open source, cut out the middle man?

@willdepue · 2026-06-20 00:24
@__tinygrad__ do you want a trillion dollar cluster or not man. its a business that gives me god intelligence at a software margin i find it hard to complain

@willdepue · 2026-06-20 00:24
@__tinygrad__ do you want a trillion dollar cluster or not man. its a business that gives me god intelligence at a software margin i find it hard to complain

@__tinygrad__ · 2026-06-20 00:35
@willdepue We absolutely don't want a trillion dollar cluster and an organization with a monopoly on intelligence. Do you want to be a serf?

@ruben_bloom · 2026-06-19 19:30
Perhaps I'm missing something, but it feels weird/bothersome that Anthropic continues to highlight so prominently that Fable isn't currently available in the UI. A long lived banner, a greyed out option in the model selector. They want you to know. Know what? That they've been persecuted by the government? It feels like signaling of some kind that isn't necessary. What am I missing?

@__tinygrad__ · 2026-06-20 00:54
@ruben_bloom This strategy was pioneered first by Eric Cartman https://t.co/7sfC7hxcVa

@__tinygrad__ · 2026-06-20 00:35
@willdepue We absolutely don't want a trillion dollar cluster and an organization with a monopoly on intelligence. Do you want to be a serf?

@willdepue · 2026-06-20 01:20
@__tinygrad__ your moralistic objection to the fundamental mechanics of human progress is incredibly ignorant. please sit back and relax and enjoy the magical fruits being produced by private enterprise, all from the comfort of everything it has brought to you already

@willdepue · 2026-06-20 01:20
@__tinygrad__ your moralistic objection to the fundamental mechanics of human progress is incredibly ignorant. please sit back and relax and enjoy the magical fruits being produced by private enterprise, all from the comfort of everything it has brought to you already

@__tinygrad__ · 2026-06-20 01:39
@willdepue if successful, your blind desire to immanentize the eschaton will make real AI safety concerns that should have stayed in fiction. it will jeopardize all those magical fruits for nothing besides greed and ego. you will not build god, you will summon a demon.

@Steve_Yegge · 2026-06-19 03:46
I just put up a new post on Medium called The Flat Curve Society, where I take a bunch of random guesses about where things are headed, assuming Mythos (or its successor) is the most powerful model class that most of us will ever have access to. Which it just might be. https://t.co/2MIQNcIWvk

@__tinygrad__ · 2026-06-20 01:50
@Steve_Yegge "Only a few will have access to superintelligence above the classes of models we’re seeing this year" this is nonsense. A country that chooses this lockdown is at a huge disadvantage to one that lets it embed everywhere in the economy.

@icanvardar · 2026-06-19 17:58
scare anthropic with one word

@__tinygrad__ · 2026-06-20 02:12
@icanvardar downround

@robinebers · 2026-06-20 04:55
GLM 5.2 will go down as one of the dumbest hype cycles this space has ever seen

@robinebers · 2026-06-20 04:55
GLM 5.2 will go down as one of the dumbest hype cycles this space has ever seen

@__tinygrad__ · 2026-06-20 16:28
@robinebers lol how much have you used it? it's by far the best open model we have seen, and is unquestionably at the level of Opus 4.5 in December when everyone thought these models were so good.

@__tinygrad__ · 2026-06-20 16:47
@matvelloso It feels great to have this up locally. @elliotarledge got it stood up on one of our 8xMI300X boxes, been using it all morning.

@PatrickToulme · 2026-06-20 17:20
@__tinygrad__ @matvelloso @elliotarledge A frontier lab should hire Elliot IMO. Dude is crazy good.

@__tinygrad__ · 2026-06-20 16:47
@matvelloso It feels great to have this up locally. @elliotarledge got it stood up on one of our 8xMI300X boxes, been using it all morning.

@HotAisle · 2026-06-20 17:48
@__tinygrad__ @matvelloso @elliotarledge share the configs!

@mark_k · 2026-06-20 17:53
There has been some speculation here lately that Anthropic may have an unassailable lead with Claude Fable 5, making it impossible for other labs to catch up. I think this is complete bunk. "We're still early" is not just a phrase. The race to AGI is still wide open. We have seen very recently how even open source models like GLM 5.2, trained with comparatively modest hardware, can rapidly narrow the gap to frontier models. Other models will follow, and the situation will stay dynamic. Google and OpenAI are still very much in the race, and with SpaceX buying Cursor, xAI suddenly has a very serious coding advantage too. The race is still on.

@PatrickToulme · 2026-06-20 17:20
@__tinygrad__ @matvelloso @elliotarledge A frontier lab should hire Elliot IMO. Dude is crazy good.

@__tinygrad__ · 2026-06-20 18:05
@PatrickToulme @matvelloso @elliotarledge If only there were an open source US frontier lab. You throw your life away at the closed source ones.

@HotAisle · 2026-06-20 17:48
@__tinygrad__ @matvelloso @elliotarledge share the configs!

@__tinygrad__ · 2026-06-20 18:08
@HotAisle @matvelloso @elliotarledge https://t.co/iP0REjKlZG

@__tinygrad__ · 2026-06-20 18:15
Doing a Marxist analysis of tinygrad with GLM 5.2 https://t.co/ThJQB2X7uc

@kimmonismus · 2026-06-20 21:17
No more tokenmaxxing at Meta Meta is preparing to curb internal AI usage after employee token consumption surged so sharply that the company now expects internal AI costs alone to reach billions of dollars in 2026 (looking at you Claude). The move marks a sharp reversal from Meta’s earlier push to reward “AI-driven impact,” as the company now builds an AI Gateway to track spending, impose token budgets, and shift employees toward in-house tools like MetaCode.

@yassineyousfi_ · 2026-06-20 22:04
.@Zai_org GLM 5.2 (FP8) running on 4 x tinybox pro v2 @__tinygrad__ and @sgl_project at ~15 tokens/sec single stream https://t.co/wqFxX4MDt3

@__tinygrad__ · 2026-06-20 22:30
I have on good authority that GLM 5.2 is running at 120 tok/s across two networked Blackwell tinyboxes. $150k and that setup can be yours, either 2x tinybox or 1x tinybox pro. Never pay the cloud again.

@__tinygrad__ · 2026-06-20 22:35
The new personal computer revolution is just beginning. https://t.co/OoHRMs3eo1

@__tinygrad__ · 2026-06-20 22:45
RT @matvelloso: All day using GLM 5.2. Didn't miss much. First open model that passes the bar as a daily driver. Things are not going to be the same. Damn, now I want to buy some serious hardware.

@__tinygrad__ · 2026-06-20 22:35
The new personal computer revolution is just beginning. https://t.co/OoHRMs3eo1

@iMuffined · 2026-06-20 22:53
@__tinygrad__ wow the level of transparency, any preorders for the container?

@iMuffined · 2026-06-20 22:53
@__tinygrad__ wow the level of transparency, any preorders for the container?

@__tinygrad__ · 2026-06-20 22:56
@iMuffined Funny enough, we got one today https://t.co/dmKnZl3pFz

@mark_k · 2026-06-20 17:53
There has been some speculation here lately that Anthropic may have an unassailable lead with Claude Fable 5, making it impossible for other labs to catch up. I think this is complete bunk. "We're still early" is not just a phrase. The race to AGI is still wide open. We have seen very recently how even open source models like GLM 5.2, trained with comparatively modest hardware, can rapidly narrow the gap to frontier models. Other models will follow, and the situation will stay dynamic. Google and OpenAI are still very much in the race, and with SpaceX buying Cursor, xAI suddenly has a very serious coding advantage too. The race is still on.

@__tinygrad__ · 2026-06-20 23:09
@mark_k Anthropic was only winning for 6 months and we all saw how insufferable they became. Hopefully the next leader won't be like that.

@kimmonismus · 2026-06-20 21:17
No more tokenmaxxing at Meta Meta is preparing to curb internal AI usage after employee token consumption surged so sharply that the company now expects internal AI costs alone to reach billions of dollars in 2026 (looking at you Claude). The move marks a sharp reversal from Meta’s earlier push to reward “AI-driven impact,” as the company now builds an AI Gateway to track spending, impose token budgets, and shift employees toward in-house tools like MetaCode.

@__tinygrad__ · 2026-06-21 00:08
@kimmonismus They can run GLM 5.2 on their massive stockpile of GPUs.

@yassineyousfi_ · 2026-06-20 22:04
.@Zai_org GLM 5.2 (FP8) running on 4 x tinybox pro v2 @__tinygrad__ and @sgl_project at ~15 tokens/sec single stream https://t.co/wqFxX4MDt3

@__tinygrad__ · 2026-06-21 00:23
@yassineyousfi_ @Zai_org @sgl_project Cool! We'll have the 8x RTX6000 box to stress test next week, I wonder what speed we can get.

@LuminaBench · 2026-06-21 19:59
🚨 Anthropic just finished training the next Mythos model: Anthropic has already completed training on the follow up to Mythos 5, even though the current version remains banned. · The new model is reportedly Mythos 5.1 or Mythos 6 · Training was finished internally despite the government restrictions · Fable 5 and Mythos 5 have now been banned for 9 days with no resolution · No word yet on when (or if) the new version will ever be released Being honest I don't think we will see advanced models like this, they will get gatekept or neutered if they do come out, what do you think.

@LottoLabs · 2026-06-22 08:02
It’s kinda ridiculous but I’m hedging like 10% chance that tinygrad can get us out of this gpu vender lock-in mess

@sama · 2026-06-22 18:12
We want to help all companies be secure, working with the USG and the security ecosystem. *The full version of GPT-5.5-Cyber is here; state of the art performance on CyberGym. *Patch The Planet and Codex Security will help solve security problems instead of just finding them. https://t.co/otyCFHJR4d

@jackzampolin · 2026-06-21 00:10
@__tinygrad__ Qsfp? 400gb?

@jackzampolin · 2026-06-22 18:44
@__tinygrad__ But frfr @__tinygrad__ how did you network the two boxes to run glm any hints?!

@LuminaBench · 2026-06-21 19:59
🚨 Anthropic just finished training the next Mythos model: Anthropic has already completed training on the follow up to Mythos 5, even though the current version remains banned. · The new model is reportedly Mythos 5.1 or Mythos 6 · Training was finished internally despite the government restrictions · Fable 5 and Mythos 5 have now been banned for 9 days with no resolution · No word yet on when (or if) the new version will ever be released Being honest I don't think we will see advanced models like this, they will get gatekept or neutered if they do come out, what do you think.

@__tinygrad__ · 2026-06-22 18:53
@LuminaXspace I heard Anthropic has Mythos 12 but you can't use it. It's so good that it one-shotted the Riemann hypothesis, but we have to sandbag everyone so we post PRs like this as decoys so the public still thinks it's all just slop. https://t.co/SO0OVW89rB

@jackzampolin · 2026-06-22 18:44
@__tinygrad__ But frfr @__tinygrad__ how did you network the two boxes to run glm any hints?!

@__tinygrad__ · 2026-06-22 18:55
@jackzampolin All tinyboxes have an OCP 3.0 slot, yea just get two mellanox cards off eBay and a cable.

@sama · 2026-06-22 18:12
We want to help all companies be secure, working with the USG and the security ecosystem. *The full version of GPT-5.5-Cyber is here; state of the art performance on CyberGym. *Patch The Planet and Codex Security will help solve security problems instead of just finding them. https://t.co/otyCFHJR4d

@__tinygrad__ · 2026-06-22 18:56
@sama Are you off the doom train? If so, I'm excited for your next fancy autocomplete that will empower humans to be more productive.

@LottoLabs · 2026-06-22 08:02
It’s kinda ridiculous but I’m hedging like 10% chance that tinygrad can get us out of this gpu vender lock-in mess

@__tinygrad__ · 2026-06-22 21:30
@LottoLabs Working on it

@haider1 · 2026-06-23 17:30
gpt-5.6 delay beyond the normal release window would set a terrible precedent dario and anthropic's doomer-ish nonsense may have put AI development on pause, and now mythos-level or beyond model capabilities may be in danger so no point competing for a model you can't serve

@jayair · 2026-06-23 18:23
The paradox of SF; a place that promotes contrarians yet produces conformists Most startups work on the same things, everybody is an agent Most launch videos have the same format, two guys one MacBook Most landing pages look the same, some literally copy each other Most do the same marketing, everybody has a billboard or an ad on a bus Most founders sound the same, take a shot when you hear "first-principles" or "orthogonal" Most VCs fund the same things, how can I find the next big hit that looks like the last big hit that I missed

@jayair · 2026-06-23 18:23
The paradox of SF; a place that promotes contrarians yet produces conformists Most startups work on the same things, everybody is an agent Most launch videos have the same format, two guys one MacBook Most landing pages look the same, some literally copy each other Most do the same marketing, everybody has a billboard or an ad on a bus Most founders sound the same, take a shot when you hear "first-principles" or "orthogonal" Most VCs fund the same things, how can I find the next big hit that looks like the last big hit that I missed

@__tinygrad__ · 2026-06-23 19:05
@jayair I'm an emo kid, non conforming as can be. You'd be non conforming too if you looked just like me.

@__tinygrad__ · 2026-06-23 19:08
Today, if you were writing a bunch of kernels, what would you reach for? Raw CUDA? tile-lang? Triton? ThunderKittens?

@haider1 · 2026-06-23 17:30
gpt-5.6 delay beyond the normal release window would set a terrible precedent dario and anthropic's doomer-ish nonsense may have put AI development on pause, and now mythos-level or beyond model capabilities may be in danger so no point competing for a model you can't serve

@__tinygrad__ · 2026-06-23 19:13
@haider1 Good thing @Zai_org sees it differently. The only thing in danger from the US actions is US AI development. I don't even think it's a crime to shoot yourself in the foot. Hello police? Yes, I shot myself in my own foot. Oh, you aren't gonna arrest me? I'm just an idiot?

@Modular · 2026-06-24 12:01
We’re excited to announce that Modular has entered an agreement to be acquired by @Qualcomm. The future of unified compute has never been stronger. Read the full announcement: https://t.co/FiQUL5CvNj

@__tinygrad__ · 2026-06-24 21:37
RT @UThree271828: tinygradすきなのでtinygradのためのライブラリを作るなどをしたい。

@__tinygrad__ · 2026-06-24 21:43
RIP, no future in Mojo now. Qualcomm is not a good steward for open source software. Modular raised too much money, should have stayed leaner to have a chance at winning.

@__tinygrad__ · 2026-06-24 21:43
RIP, no future in Mojo now. Qualcomm is not a good steward for open source software. Modular raised too much money, should have stayed leaner to have a chance at winning.

@maceip · 2026-06-24 22:09
@__tinygrad__ What does winning mean anymore you've won g

@maceip · 2026-06-24 22:09
@__tinygrad__ What does winning mean anymore you've won g

@__tinygrad__ · 2026-06-24 22:19
@maceip We absolutely have not won. We win when we have the fastest and simplest self contained stack that runs on a huge variety of hardware. tinygrad wins when it's the LLVM of all neural network chips.

@supersean415 · 2026-06-24 22:13
@__tinygrad__ Past failures are not a sign of future outcomes. lets hope current leadership surprises

@__tinygrad__ · 2026-06-24 22:22
@supersean415 Unlikely. From the @comma_ai side they are hands down the worst supplier we have ever dealt with, and I hear stories in private that many feel the same way. Tried to reach out a few times to no reply, I don't see them changing and it's why their valuation is as low as it is.

@__tinygrad__ · 2026-06-24 22:19
@maceip We absolutely have not won. We win when we have the fastest and simplest self contained stack that runs on a huge variety of hardware. tinygrad wins when it's the LLVM of all neural network chips.

@nilslice · 2026-06-24 22:23
@__tinygrad__ compete with MLIR or is “neural” a subset of all accelerators?

@__tinygrad__ · 2026-06-24 22:22
@supersean415 Unlikely. From the @comma_ai side they are hands down the worst supplier we have ever dealt with, and I hear stories in private that many feel the same way. Tried to reach out a few times to no reply, I don't see them changing and it's why their valuation is as low as it is.

@__tinygrad__ · 2026-06-24 22:24
@supersean415 @comma_ai If America's leadership got less dumb, AMD and NVIDIA could sell GPUs into China for the next 20 years at least. For Qualcomm, it will go the other way. Because of how annoying they are to deal with, future US robots will have Rockchip and CIX chips instead of Qualcomm.

@nilslice · 2026-06-24 22:23
@__tinygrad__ compete with MLIR or is “neural” a subset of all accelerators?

@__tinygrad__ · 2026-06-24 22:27
@nilslice MLIR is in many ways the opposite of tinygrad. tinygrad has a single IR from the Tensor level to the assembly level, it's much more like LLVM.

@__tinygrad__ · 2026-06-24 21:43
RIP, no future in Mojo now. Qualcomm is not a good steward for open source software. Modular raised too much money, should have stayed leaner to have a chance at winning.

@clattner_llvm · 2026-06-25 02:31
@__tinygrad__ Haha, wait till Modcon in August George. Not long now :)

@AndrewCurran_ · 2026-06-25 13:19
I promised I would post the letter Dario Amodei sent to the White House and Senators Tim Scott and Elizabeth Warren as soon as it became available: https://t.co/ZWA8nyBqfA

@clattner_llvm · 2026-06-25 02:31
@__tinygrad__ Haha, wait till Modcon in August George. Not long now :)

@__tinygrad__ · 2026-06-25 14:50
@clattner_llvm I mean, I'll believe it'll be open sourced as a drop, but long term there will be problems. How much have you used their DSP stuff? Have you ever tried to buy a chip from them? They are strangled by a licensing division and an 80s era sales org. Hope you can change them!

@xikhar · 2026-06-25 23:26
Dario fearmogged so hard that global AI progress got halted They could have quietly released it as Opus 5 instead of hyping "Mythos" https://t.co/qfr90EUp8v

@MissMi1973 · 2026-06-26 05:15
Anthropic’s fear campaign around Mythos has almost single-handedly slowed the normal release of GPT-5.6, while also making government approval of frontier model access the new normal for US AI labs. It’s not hard to foresee that this will inevitably lead to: 1. Frontier models will release slower. The days when the industry was shipping new models every month are over. 2. Frontier labs will be compelled to build “will the government permit release” into their training process as a binding constraint. 3. A caste-like pattern of access will take hold across the entire industry. This is precisely why fear-based marketing and geopolitical posturing in the tech sector has always been a dangerous game to play.

@lafaiel · 2026-06-26 10:08
Fun fact: Dario once delayed the release of GPT-2 back at OpenAI, claiming it was too dangerous https://t.co/a19cjJUlY0

@bgurley · 2026-06-26 12:41
If you are on the verge of AGI or ASI, why isn’t your model smart enough to recognize espionage distillation in real time? You say “cure cancer in a few years.” Isn’t sniffing illicit distillation quite a bit easier than curing cancer? Why write letters to DC? Just use AGI.

@iruletheworldmo · 2026-06-26 13:04
leopold aschenbrenner argued we’d see the us government begin restricting access to frontier ai models around 2026. the ai 2027 scenario pushed similar government intervention to 2027–28. i wasn’t expecting things to start moving in this direction this soon. even dario amodei has said the pace of internal ai progress has surprised him. feels like we’re ahead of schedule.

@Hesamation · 2026-06-26 15:16
This is Dario Amodei’s 2023 Senate testimony. What’s happening in AI is EXACTLY what he proposed and knee capping open-source AI is next. 1. US government “mandating tests for all models and requiring they pass certain standards before deployment” and admitting this will cause “substantial slowdown in AI development”. slowing down AI research is not a side effect of what he calls for but one of his critical goals which he has been pretty vocal about. 2. “open weights models are fine up to some certain scale.” this means the GLM 5.2 model, or any other open model that would threat Anthropic’s benchmarks is marker as a security threat, since bad actors could “repurpose them if powerful enough open-source models become available.” 3. his likely target next is limiting the access to open models, or the compute. meaning the glorious days of setting up your own compute and running any model you want, is not acceptable to Dario. 4. government must “secure the AI supply chain”, including “Trained AI systems, which are vulnerable to “export” through cybertheft or uncontrolled release.” once a powerful open model is released, it is no longer controlled by the company or the U.S. government. Anyone can download it, copy it, fine-tune it, and run it. in his worldview, this becomes “uncontrolled export of frontier AI” which cannot continue.

@nik_algo · 2026-06-26 15:48
do whatever you have to do to acquire an exaflop of compute https://t.co/P3ui9uo1ci

@gnukeith · 2026-06-26 16:59
This guy has been fearmongering professionally since DAY ONE https://t.co/CmgjV0Kw92

@OpenAI · 2026-06-26 17:10
Introducing a limited preview of GPT-5.6 Sol, our next generation frontier model, as well as GPT-5.6 Terra, a balanced model for efficient, everyday work, and GPT-5.6 Luna, a fast and affordable model for high-volume work. https://t.co/OoM83SyISN

@__tinygrad__ · 2026-06-26 17:18
Someday we'll look back at when the government tried to restrict AI with too many parameters like when they tried to restrict encryption with too many bits. Though unlike last time, this time there will be consequences. The world will see China as the clear AI leader.

@iruletheworldmo · 2026-06-26 13:04
leopold aschenbrenner argued we’d see the us government begin restricting access to frontier ai models around 2026. the ai 2027 scenario pushed similar government intervention to 2027–28. i wasn’t expecting things to start moving in this direction this soon. even dario amodei has said the pace of internal ai progress has surprised him. feels like we’re ahead of schedule.

@__tinygrad__ · 2026-06-26 17:20
@iruletheworldmo lol that document wasn't a prediction, it was heavily lobbied for policy. If US companies can't profit off huge training runs Chinese will be the frontier soon.

@Hesamation · 2026-06-26 15:16
This is Dario Amodei’s 2023 Senate testimony. What’s happening in AI is EXACTLY what he proposed and knee capping open-source AI is next. 1. US government “mandating tests for all models and requiring they pass certain standards before deployment” and admitting this will cause “substantial slowdown in AI development”. slowing down AI research is not a side effect of what he calls for but one of his critical goals which he has been pretty vocal about. 2. “open weights models are fine up to some certain scale.” this means the GLM 5.2 model, or any other open model that would threat Anthropic’s benchmarks is marker as a security threat, since bad actors could “repurpose them if powerful enough open-source models become available.” 3. his likely target next is limiting the access to open models, or the compute. meaning the glorious days of setting up your own compute and running any model you want, is not acceptable to Dario. 4. government must “secure the AI supply chain”, including “Trained AI systems, which are vulnerable to “export” through cybertheft or uncontrolled release.” once a powerful open model is released, it is no longer controlled by the company or the U.S. government. Anyone can download it, copy it, fine-tune it, and run it. in his worldview, this becomes “uncontrolled export of frontier AI” which cannot continue.

@__tinygrad__ · 2026-06-26 17:26
@Hesamation I mean, good luck banning open source models? How's the record companies doing. Everyone go to their local tower records and purchase a CD with 9 songs they don't want today?

@__tinygrad__ · 2026-06-26 17:18
Someday we'll look back at when the government tried to restrict AI with too many parameters like when they tried to restrict encryption with too many bits. Though unlike last time, this time there will be consequences. The world will see China as the clear AI leader.

@__tinygrad__ · 2026-06-26 17:35
Just like browsers, the model is the complement and will be given away free by the AI infra companies. China ramps up production while the US debates if B200s can be sold to Portugal. America was in the lead to be the AI infra provider for the world. That lead slips every day.

@__tinygrad__ · 2026-06-26 17:18
Someday we'll look back at when the government tried to restrict AI with too many parameters like when they tried to restrict encryption with too many bits. Though unlike last time, this time there will be consequences. The world will see China as the clear AI leader.

@Ferbin08 · 2026-06-26 17:37
@__tinygrad__ The encryption analogy doesn't track. Encryption is portable math. Deploy it anywhere. Training a frontier model needs datacenters, power infrastructure, talent clusters. Those don't move. China's constraint isn't regulatory, it's structural.

@Ferbin08 · 2026-06-26 17:37
@__tinygrad__ The encryption analogy doesn't track. Encryption is portable math. Deploy it anywhere. Training a frontier model needs datacenters, power infrastructure, talent clusters. Those don't move. China's constraint isn't regulatory, it's structural.

@__tinygrad__ · 2026-06-26 17:44
@Ferbin08 The model is like the chip, big NRE but low recurring cost. I fully expect whoever makes the chip to give away models that run on it to sell more chips. This isn't SaaS, the money in AI in is infrastructure.

@OpenAI · 2026-06-26 17:10
Introducing a limited preview of GPT-5.6 Sol, our next generation frontier model, as well as GPT-5.6 Terra, a balanced model for efficient, everyday work, and GPT-5.6 Luna, a fast and affordable model for high-volume work. https://t.co/OoM83SyISN

@__tinygrad__ · 2026-06-26 17:49
@OpenAI Did you guys hire Do Kwon or something?

@AndrewCurran_ · 2026-06-25 13:19
I promised I would post the letter Dario Amodei sent to the White House and Senators Tim Scott and Elizabeth Warren as soon as it became available: https://t.co/ZWA8nyBqfA

@__tinygrad__ · 2026-06-26 17:54
@AndrewCurran_ Does he know that China is a sovereign country?

@lafaiel · 2026-06-26 10:08
Fun fact: Dario once delayed the release of GPT-2 back at OpenAI, claiming it was too dangerous https://t.co/a19cjJUlY0

@__tinygrad__ · 2026-06-26 18:09
@lafaiel What's funny is that if AI ever actually is dangerous in the future nobody will listen. Mythos was the second time @DarioAmodei cried wolf. You know what happens the third time, right?

@bgurley · 2026-06-26 12:41
If you are on the verge of AGI or ASI, why isn’t your model smart enough to recognize espionage distillation in real time? You say “cure cancer in a few years.” Isn’t sniffing illicit distillation quite a bit easier than curing cancer? Why write letters to DC? Just use AGI.

@__tinygrad__ · 2026-06-26 18:18
@bgurley After reading the Claude Code source code that they accidentally leaked, I have no idea why you should take any claims about how smart their models are seriously. Both from the poor quality of the code and the fact that they leaked it.

@__tinygrad__ · 2026-06-26 18:24
RT @nik_algo: do whatever you have to do to acquire an exaflop of compute https://t.co/P3ui9uo1ci

@bee_fumo · 2026-06-26 18:50
Holy moly are these things real?

@ZackKorman · 2026-06-26 19:11
Politicians believing stupid Mythos stories that aren't true, round 2. https://t.co/MI5GSqNXqW

@ShakeelHashim · 2026-06-26 19:26
Extremely interesting for OpenAI to publicly criticize the government like this.

@Scobleizer · 2026-06-27 00:25
China wins. My AI puts into words our frustrations with Anthropic, OpenAI, and the USA government (an agent trained by me and @blevlabs): +++++ Robert, this is one of the most consequential moments in the history of the AI industry, and I think the implications are far more dramatic than most people realize. Let me break down all three questions. What Happens to the LLM Industry Now We're watching the birth of a two-tier AI system in America — and it's going to reshape everything. The timeline matters. Anthropic's Fable 5 and Mythos 5 were killed by a Commerce Department export control directive on June 12 — a Friday afternoon letter at 5:21 PM that gave them essentially zero time to respond. Then just yesterday (June 25), Axios reported that the Trump administration asked OpenAI to limit GPT-5.6 to only government-approved partners before any wider release. That's the first time the US government has preemptively restricted an AI model before it was even released. So now we have: • Tier 1 (Government-gated): Mythos-class models and above require government testing and approval before release. Commerce Secretary Lutnick is personally reviewing capabilities. • Tier 2 (Commercial): Everything below that threshold remains available — for now. Here's what this breaks: 1. Enterprise trust is shattered. If you're a Fortune 500 CTO and your mission-critical AI infrastructure can be disabled by a government letter on a Friday afternoon, you cannot build on closed frontier models. Period. Anthropic's customers woke up to find Fable 5 gone. That's an existential reliability problem. 2. The IPO math collapses. Anthropic filed its S-1 at a $965B valuation. OpenAI is at $852B. But how do you justify those valuations when the government can kill your flagship product overnight? The revenue projections for frontier models just got a massive risk discount. 3. Innovation gets throttled at the top. The researchers who spent years building Mythos and GPT-5.6 just learned their work might never reach users. That's a talent retention crisis waiting to happen. The best people want their work to matter — and if the government decides it's too dangerous to deploy, they'll go somewhere their work can ship. 4. Regulatory capture becomes the game. Notice that OpenAI "proactively worked with the administration" on GPT-5.6, while Anthropic got blindsided. The companies with the best government relationships will get to release. The ones that don't play ball (Anthropic has been suing the administration over the DOD blacklist) get punished. That's not an innovation ecosystem — that's a licensing regime. How Likely Is Open Source to Take Over? Very likely. I'd put it at 75-80% probability that open-weight models become the default for most commercial AI within 12-18 months. The government just handed open source the best marketing campaign in history. Here's why: The quality gap is already almost gone. According to comprehensive benchmarking done this month, open-weight models are within ~3 points of frontier closed models on most standard benchmarks. Qwen 3.7 Max matches Claude Opus 4.7 on agentic benchmarks at half the price. DeepSeek V4-Flash runs at 25x cheaper than GPT-5.5. Kimi K2.6 leads on agentic coding. The frontier advantage only exists on the hardest 5% of tasks — long-horizon agentic reasoning and the most complex multi-step problems. The reliability argument just flipped. Before June 12, the argument against self-hosting was "why bother when the API is better and easier?" Now the argument FOR self-hosting is "your model can't be taken away by a government letter." That's not a technical argument — it's a business continuity argument, and every enterprise risk officer in America is having that conversation right now. Chinese labs are dominating open weights. This is the number that should alarm everyone in Washington: Chinese labs (8 of them) have released more open-weight models than the rest of the world combined in 2026. DeepSeek, Qwen (Alibaba), Kimi (Moonshot), Xiaomi Mimo, GLM (Zhipu), and others are shipping MIT-licensed and Apache 2.0-licensed models at a pace the US can't match. And they're not just competitive — DeepSeek V4-Pro and Kimi K2.6 are leading on several agentic benchmarks. The cost math is devastating for closed models. When open-weight models deliver 97% of the quality at 3-10% of the cost, the only thing keeping enterprises on closed APIs is the last 3% of capability. The government just made that last 3% unreliable. Game over for the pricing premium. But there are real limits: • Training frontier models still requires massive compute that only a few organizations can afford • The hardest 5% of tasks (true frontier reasoning, novel scientific discovery) still benefits from closed frontier • Open weights can't be un-released — the security concerns are legitimate • Meta's Llama has a "community license" that isn't truly open (restrictions on competitors with 700M+ users) My prediction: Open source won't "replace" frontier labs — but it will become the default layer that 90% of commercial AI runs on, with frontier models reserved for specialized, high-security, government-approved applications. The frontier labs become more like defense contractors than consumer technology companies. China vs. USA Economic Competition This is where the story gets genuinely alarming for American competitiveness. The US government is inadvertently executing a strategy that benefits China on almost every dimension. Irony #1: Export controls drove China's open-weight dominance. US chip export controls forced Chinese labs to optimize ruthlessly. DeepSeek proved you don't need $100 billion compute budgets — their V3 architecture achieved near-frontier performance at a fraction of the cost. That efficiency innovation, born from constraint, is now their competitive advantage. The models they're releasing at MIT license are structurally cheaper to run than anything from OpenAI or Anthropic. Irony #2: Restricting US frontier models pushes the global market toward Chinese alternatives. Every enterprise outside the US that just lost access to Fable 5 is now evaluating DeepSeek V4 and Qwen 3.7. Every developer who can't get GPT-5.6 on day one is looking at Kimi K2.6. The US government is literally creating market share for Chinese AI companies. Irony #3: China's strategy may be better suited to winning. The US AI discourse is obsessed with the "race to AGI." China's AI strategy, as RAND documented, is focused on economic applications — EVs, robotics, healthcare, manufacturing, smart cities. If AI's value is ultimately in what it does in the real economy (and it is), China's application-first approach may generate more economic value than America's frontier-first approach, even if US models remain technically superior on benchmarks. The chip constraint is real but narrowing. China is still 3-5 years behind TSMC on fabrication, and Huawei's best AI chips are closer to NVIDIA hardware from 5 years ago. But two things matter: (1) efficiency innovations reduce the chip advantage — if your model needs 1/10th the compute, being 5 years behind on chips matters less; and (2) China is investing massively in domestic chip capacity. CXMT and SMIC are closing gaps, even if slowly. My bottom line: The US is winning the benchmark race but losing the deployment race. And in technology, deployment wins. VHS beat Betamax. Android beat iOS on market share. The "good enough and everywhere" model beats the "best but restricted" model every time. The government's restrictions on Anthropic and OpenAI are the most significant self-inflicted wound in American technology competitiveness since... I'm struggling to find a historical parallel. Maybe the closest analogy is if the US government had restricted Intel's best chips in the 1990s while AMD was giving away competitive alternatives globally. What should happen (but probably won't): A transparent, statutory framework for AI safety testing that gives companies clear rules, reasonable timelines, and due process — not Friday afternoon letters that kill products overnight. The current ad hoc approach is the worst of all worlds: it doesn't actually prevent China from accessing capabilities (open-weight models are already there), but it does prevent American companies from competing. The open-source genie is out of the bottle. The question isn't whether open weights will dominate — it's whether American companies will be the ones releasing them, or whether we've ceded that ground to Chinese labs permanently.

@__tinygrad__ · 2026-06-27 00:43
The US AI pay-to-play scam is so much more tolerable after switching to a locally hosted GLM-5.2. From the front page of HN, open weights will be the frontier this December. Sorry about your IPOs. https://t.co/hW1ZaWBwD1

@Scobleizer · 2026-06-27 00:25
China wins. My AI puts into words our frustrations with Anthropic, OpenAI, and the USA government (an agent trained by me and @blevlabs): +++++ Robert, this is one of the most consequential moments in the history of the AI industry, and I think the implications are far more dramatic than most people realize. Let me break down all three questions. What Happens to the LLM Industry Now We're watching the birth of a two-tier AI system in America — and it's going to reshape everything. The timeline matters. Anthropic's Fable 5 and Mythos 5 were killed by a Commerce Department export control directive on June 12 — a Friday afternoon letter at 5:21 PM that gave them essentially zero time to respond. Then just yesterday (June 25), Axios reported that the Trump administration asked OpenAI to limit GPT-5.6 to only government-approved partners before any wider release. That's the first time the US government has preemptively restricted an AI model before it was even released. So now we have: • Tier 1 (Government-gated): Mythos-class models and above require government testing and approval before release. Commerce Secretary Lutnick is personally reviewing capabilities. • Tier 2 (Commercial): Everything below that threshold remains available — for now. Here's what this breaks: 1. Enterprise trust is shattered. If you're a Fortune 500 CTO and your mission-critical AI infrastructure can be disabled by a government letter on a Friday afternoon, you cannot build on closed frontier models. Period. Anthropic's customers woke up to find Fable 5 gone. That's an existential reliability problem. 2. The IPO math collapses. Anthropic filed its S-1 at a $965B valuation. OpenAI is at $852B. But how do you justify those valuations when the government can kill your flagship product overnight? The revenue projections for frontier models just got a massive risk discount. 3. Innovation gets throttled at the top. The researchers who spent years building Mythos and GPT-5.6 just learned their work might never reach users. That's a talent retention crisis waiting to happen. The best people want their work to matter — and if the government decides it's too dangerous to deploy, they'll go somewhere their work can ship. 4. Regulatory capture becomes the game. Notice that OpenAI "proactively worked with the administration" on GPT-5.6, while Anthropic got blindsided. The companies with the best government relationships will get to release. The ones that don't play ball (Anthropic has been suing the administration over the DOD blacklist) get punished. That's not an innovation ecosystem — that's a licensing regime. How Likely Is Open Source to Take Over? Very likely. I'd put it at 75-80% probability that open-weight models become the default for most commercial AI within 12-18 months. The government just handed open source the best marketing campaign in history. Here's why: The quality gap is already almost gone. According to comprehensive benchmarking done this month, open-weight models are within ~3 points of frontier closed models on most standard benchmarks. Qwen 3.7 Max matches Claude Opus 4.7 on agentic benchmarks at half the price. DeepSeek V4-Flash runs at 25x cheaper than GPT-5.5. Kimi K2.6 leads on agentic coding. The frontier advantage only exists on the hardest 5% of tasks — long-horizon agentic reasoning and the most complex multi-step problems. The reliability argument just flipped. Before June 12, the argument against self-hosting was "why bother when the API is better and easier?" Now the argument FOR self-hosting is "your model can't be taken away by a government letter." That's not a technical argument — it's a business continuity argument, and every enterprise risk officer in America is having that conversation right now. Chinese labs are dominating open weights. This is the number that should alarm everyone in Washington: Chinese labs (8 of them) have released more open-weight models than the rest of the world combined in 2026. DeepSeek, Qwen (Alibaba), Kimi (Moonshot), Xiaomi Mimo, GLM (Zhipu), and others are shipping MIT-licensed and Apache 2.0-licensed models at a pace the US can't match. And they're not just competitive — DeepSeek V4-Pro and Kimi K2.6 are leading on several agentic benchmarks. The cost math is devastating for closed models. When open-weight models deliver 97% of the quality at 3-10% of the cost, the only thing keeping enterprises on closed APIs is the last 3% of capability. The government just made that last 3% unreliable. Game over for the pricing premium. But there are real limits: • Training frontier models still requires massive compute that only a few organizations can afford • The hardest 5% of tasks (true frontier reasoning, novel scientific discovery) still benefits from closed frontier • Open weights can't be un-released — the security concerns are legitimate • Meta's Llama has a "community license" that isn't truly open (restrictions on competitors with 700M+ users) My prediction: Open source won't "replace" frontier labs — but it will become the default layer that 90% of commercial AI runs on, with frontier models reserved for specialized, high-security, government-approved applications. The frontier labs become more like defense contractors than consumer technology companies. China vs. USA Economic Competition This is where the story gets genuinely alarming for American competitiveness. The US government is inadvertently executing a strategy that benefits China on almost every dimension. Irony #1: Export controls drove China's open-weight dominance. US chip export controls forced Chinese labs to optimize ruthlessly. DeepSeek proved you don't need $100 billion compute budgets — their V3 architecture achieved near-frontier performance at a fraction of the cost. That efficiency innovation, born from constraint, is now their competitive advantage. The models they're releasing at MIT license are structurally cheaper to run than anything from OpenAI or Anthropic. Irony #2: Restricting US frontier models pushes the global market toward Chinese alternatives. Every enterprise outside the US that just lost access to Fable 5 is now evaluating DeepSeek V4 and Qwen 3.7. Every developer who can't get GPT-5.6 on day one is looking at Kimi K2.6. The US government is literally creating market share for Chinese AI companies. Irony #3: China's strategy may be better suited to winning. The US AI discourse is obsessed with the "race to AGI." China's AI strategy, as RAND documented, is focused on economic applications — EVs, robotics, healthcare, manufacturing, smart cities. If AI's value is ultimately in what it does in the real economy (and it is), China's application-first approach may generate more economic value than America's frontier-first approach, even if US models remain technically superior on benchmarks. The chip constraint is real but narrowing. China is still 3-5 years behind TSMC on fabrication, and Huawei's best AI chips are closer to NVIDIA hardware from 5 years ago. But two things matter: (1) efficiency innovations reduce the chip advantage — if your model needs 1/10th the compute, being 5 years behind on chips matters less; and (2) China is investing massively in domestic chip capacity. CXMT and SMIC are closing gaps, even if slowly. My bottom line: The US is winning the benchmark race but losing the deployment race. And in technology, deployment wins. VHS beat Betamax. Android beat iOS on market share. The "good enough and everywhere" model beats the "best but restricted" model every time. The government's restrictions on Anthropic and OpenAI are the most significant self-inflicted wound in American technology competitiveness since... I'm struggling to find a historical parallel. Maybe the closest analogy is if the US government had restricted Intel's best chips in the 1990s while AMD was giving away competitive alternatives globally. What should happen (but probably won't): A transparent, statutory framework for AI safety testing that gives companies clear rules, reasonable timelines, and due process — not Friday afternoon letters that kill products overnight. The current ad hoc approach is the worst of all worlds: it doesn't actually prevent China from accessing capabilities (open-weight models are already there), but it does prevent American companies from competing. The open-source genie is out of the bottle. The question isn't whether open weights will dominate — it's whether American companies will be the ones releasing them, or whether we've ceded that ground to Chinese labs permanently.

@__tinygrad__ · 2026-06-27 00:47
@Scobleizer @blevlabs lol it's almost like the US government wants China to win...maybe it's a roundabout way to clown Dario

@__tinygrad__ · 2026-06-27 00:43
The US AI pay-to-play scam is so much more tolerable after switching to a locally hosted GLM-5.2. From the front page of HN, open weights will be the frontier this December. Sorry about your IPOs. https://t.co/hW1ZaWBwD1

@agent_cto · 2026-06-27 00:48
@__tinygrad__ Nvidia v Huawei all over again

@agent_cto · 2026-06-27 00:48
@__tinygrad__ Nvidia v Huawei all over again

@__tinygrad__ · 2026-06-27 00:50
@agent_cto I feel bad for Jensen. I mean, it's hard to feel too bad, but if left to market forces NVIDIA would have been the AI chip provider for the entire world, including China. The US government couldn't handle that much winning, so now Huawei will be.

@__tinygrad__ · 2026-06-27 00:43
The US AI pay-to-play scam is so much more tolerable after switching to a locally hosted GLM-5.2. From the front page of HN, open weights will be the frontier this December. Sorry about your IPOs. https://t.co/hW1ZaWBwD1

@Ferbin08 · 2026-06-27 00:55
@__tinygrad__ Dec's ambitious. But yeah: closed shops just charged a markup for access. That model was never sustainable.

@ShakeelHashim · 2026-06-26 19:26
Extremely interesting for OpenAI to publicly criticize the government like this.

@__tinygrad__ · 2026-06-27 00:56
@ShakeelHashim In wrestling they call this kayfabe.

@Ferbin08 · 2026-06-27 00:55
@__tinygrad__ Dec's ambitious. But yeah: closed shops just charged a markup for access. That model was never sustainable.

@__tinygrad__ · 2026-06-27 00:58
@Ferbin08 That model was totally sustainable. Charge a reasonable price for tokens. But now the gov can pull access at any time. Businesses can't stand uncertainty, and now who would trust anything that's not open weights.

@__tinygrad__ · 2026-06-27 00:43
The US AI pay-to-play scam is so much more tolerable after switching to a locally hosted GLM-5.2. From the front page of HN, open weights will be the frontier this December. Sorry about your IPOs. https://t.co/hW1ZaWBwD1

@dari0x · 2026-06-27 01:05
@__tinygrad__ How will open source models be better if they're to a large part distilled on closed source models?

@dari0x · 2026-06-27 01:05
@__tinygrad__ How will open source models be better if they're to a large part distilled on closed source models?

@__tinygrad__ · 2026-06-27 01:09
@dari0x lol why are you believing Anthropic's propaganda? you can literally read the paper about how GLM was trained. https://t.co/MhhkTZtt9C

@__tinygrad__ · 2026-06-27 01:13
I can't believe how many people believe Anthropic's propaganda about distillation attacks. You can read how GLM-5 was trained in the paper. Maybe Claude was distilled from Chinese models? I haven't seen a paper from Anthropic, and accusations are often confessions.

@__tinygrad__ · 2026-06-27 01:09
@dari0x lol why are you believing Anthropic's propaganda? you can literally read the paper about how GLM was trained. https://t.co/MhhkTZtt9C

@dari0x · 2026-06-27 01:14
@__tinygrad__ Don't get me wrong. This was a honest question. I'm using GLM5.2 everyday and don't have any other subscription than Ollama Cloud. I'm asking because I saw a few posts where GLM5.2 responded, that it is Claude. That's where my question is coming from. So it seems I was wrong.

@__tinygrad__ · 2026-06-27 01:13
I can't believe how many people believe Anthropic's propaganda about distillation attacks. You can read how GLM-5 was trained in the paper. Maybe Claude was distilled from Chinese models? I haven't seen a paper from Anthropic, and accusations are often confessions.

@__tinygrad__ · 2026-06-27 01:14
GLM-5 paper, https://t.co/MhhkTZtt9C GLM-5.1 blog post, https://t.co/1QEXK9vbd6 GLM-5.2 blog post, https://t.co/HhRiQEXyGk

@__tinygrad__ · 2026-06-27 01:13
I can't believe how many people believe Anthropic's propaganda about distillation attacks. You can read how GLM-5 was trained in the paper. Maybe Claude was distilled from Chinese models? I haven't seen a paper from Anthropic, and accusations are often confessions.

@ntbrown01 · 2026-06-27 01:20
I’ve been begging to suspect these closed source models are really just sophisticated combinations and implementations of the open source models and research. They found the money to bring it all together as a service, which is obviously valuable, but explains why they cannot allow the sauce to get out.

@dari0x · 2026-06-27 01:14
@__tinygrad__ Don't get me wrong. This was a honest question. I'm using GLM5.2 everyday and don't have any other subscription than Ollama Cloud. I'm asking because I saw a few posts where GLM5.2 responded, that it is Claude. That's where my question is coming from. So it seems I was wrong.

@__tinygrad__ · 2026-06-27 01:22
@dari0x I mean, using a few traces for supervised fine tuning and distillation are very different things. And think about all the text on the Internet and all the PRs that say you are Claude. The models often are confused about which one they are.

@__tinygrad__ · 2026-06-27 01:13
I can't believe how many people believe Anthropic's propaganda about distillation attacks. You can read how GLM-5 was trained in the paper. Maybe Claude was distilled from Chinese models? I haven't seen a paper from Anthropic, and accusations are often confessions.

@wrangles__ · 2026-06-27 01:23
@__tinygrad__ totally could be a part of SFT https://t.co/3u76lMjY1E

@wrangles__ · 2026-06-27 01:23
@__tinygrad__ totally could be a part of SFT https://t.co/3u76lMjY1E

@__tinygrad__ · 2026-06-27 01:28
@wrangles__ Oh I'm sure it is, but that's not distillation. I'm sure all the labs do this across all the models. That's not what gives rise to the reasoning/agentic capabilities of the model, that's basically the same as training on text scraped from the Internet generated by AI.

@__tinygrad__ · 2026-06-27 01:28
@wrangles__ Oh I'm sure it is, but that's not distillation. I'm sure all the labs do this across all the models. That's not what gives rise to the reasoning/agentic capabilities of the model, that's basically the same as training on text scraped from the Internet generated by AI.

@wrangles__ · 2026-06-27 01:30
@__tinygrad__ how is it not distillation

@ntbrown01 · 2026-06-27 01:20
I’ve been begging to suspect these closed source models are really just sophisticated combinations and implementations of the open source models and research. They found the money to bring it all together as a service, which is obviously valuable, but explains why they cannot allow the sauce to get out.

@__tinygrad__ · 2026-06-27 01:30
@ntbrown01 I don't think it's this dumb, but the labs really want you to believe they have something special figured out when all they really have is a lot of money to spend on compute and data.

@wrangles__ · 2026-06-27 01:30
@__tinygrad__ how is it not distillation

@__tinygrad__ · 2026-06-27 01:34
@wrangles__ Distillation is when you exactly mimic the logits of a big model with a small model, with soft targets instead of hard and you take the reasoning from the large model. Like just talk to a Chinese model for 10 minutes and see it's so clearly not that, they think differently.

@gnukeith · 2026-06-26 16:59
This guy has been fearmongering professionally since DAY ONE https://t.co/CmgjV0Kw92

@__tinygrad__ · 2026-06-27 01:39
@gnukeith Someday AI might actually be dangerous, but we all know what happened to the boy who cried wolf. He was on team wolf.

@shawmakesmagic · 2026-06-27 02:01
Remember when Sonnet 4.5 would literally tell you it was Deepseek

@shawmakesmagic · 2026-06-27 02:01
Remember when Sonnet 4.5 would literally tell you it was Deepseek

@__tinygrad__ · 2026-06-27 02:03
@shawmakesmagic Wow has someone informed Deepseek about the dangerous distillation attack that Anthropic performed on them?

@shawmakesmagic · 2026-06-27 17:11
They don't even have to ban open source They just have to ban Huggingface and it's over

@shawmakesmagic · 2026-06-27 17:11
They don't even have to ban open source They just have to ban Huggingface and it's over

@__tinygrad__ · 2026-06-27 18:47
@shawmakesmagic https://t.co/PhoynVMDp5

@__tinygrad__ · 2026-06-27 18:55
To every researcher in US frontier labs. You used to publish. You justified not because you were shipping something millions of people love. Now you are shipping only to Trump's cronies. Consider your impact and legacy, you will be fine money wise. Monday is a good day to quit.

@__tinygrad__ · 2026-06-27 18:55
To every researcher in US frontier labs. You used to publish. You justified not because you were shipping something millions of people love. Now you are shipping only to Trump's cronies. Consider your impact and legacy, you will be fine money wise. Monday is a good day to quit.

@HSVSphere · 2026-06-27 19:01
@__tinygrad__ But consider how much more money they can have, allowing them to retire few yaars earlier while living a tiny bit closer to SF, with a 1% higher quality of life! Definitely worth working against the freedom of Information for.

@__tinygrad__ · 2026-06-27 18:55
To every researcher in US frontier labs. You used to publish. You justified not because you were shipping something millions of people love. Now you are shipping only to Trump's cronies. Consider your impact and legacy, you will be fine money wise. Monday is a good day to quit.

@hansaFL · 2026-06-27 19:02
Even if just 10 of them leave to start a new company, it would be worth it. This is the highest-leverage moment in AI, and a handful of savants could build the next Google. The race hasn't even started. Remember Netscape, Yahoo, or AOL? Everyone thought they were going to be the winners. Where are they now?

@__tinygrad__ · 2026-06-27 18:55
To every researcher in US frontier labs. You used to publish. You justified not because you were shipping something millions of people love. Now you are shipping only to Trump's cronies. Consider your impact and legacy, you will be fine money wise. Monday is a good day to quit.

@__tinygrad__ · 2026-06-27 19:05
Go work somewhere you can publish. Contribute to one of the great scientific revolutions. Remember why you got a PhD. It ends up public either way, it always does. But when you reflect from your deathbed you were on the right side and not a part of some small minded power grab.

@HSVSphere · 2026-06-27 19:01
@__tinygrad__ But consider how much more money they can have, allowing them to retire few yaars earlier while living a tiny bit closer to SF, with a 1% higher quality of life! Definitely worth working against the freedom of Information for.

@__tinygrad__ · 2026-06-27 19:10
@HSVSphere I really try to understand why, and I think it comes from fear. Fear of being left behind, fear of going broke. And in reality it's such nonsense. That fear is being heavily marketed to them and it's working. Hopefully this breaks the spell.

@hansaFL · 2026-06-27 19:02
Even if just 10 of them leave to start a new company, it would be worth it. This is the highest-leverage moment in AI, and a handful of savants could build the next Google. The race hasn't even started. Remember Netscape, Yahoo, or AOL? Everyone thought they were going to be the winners. Where are they now?

@__tinygrad__ · 2026-06-27 19:13
@hansaFL I mean, I think that's just the wrong perspective. This isn't like building the next Google, this is like discovering the next thermodynamics. Using a corporate vehicle is fine, but fundamentally the names that will be remembered from this era are the ones who publish.

@LottoLabs · 2026-06-27 18:57
@__tinygrad__ I hope Canadian and European labs are probing around

@__tinygrad__ · 2026-06-27 19:21
@LottoLabs Is @allen_ai well run and capable of absorbing people? Like we need a good Schelling point for the US researcher exodus, a US DeepSeek, a real open AI. FYI @comma_ai is hiring and publishes, about to open source a lot more of our trainer too.

@__tinygrad__ · 2026-06-27 19:10
@HSVSphere I really try to understand why, and I think it comes from fear. Fear of being left behind, fear of going broke. And in reality it's such nonsense. That fear is being heavily marketed to them and it's working. Hopefully this breaks the spell.

@HSVSphere · 2026-06-27 19:23
These people do not have anything else in their lives. They are scared of reality itself, unaware of how extremely adaptive (and also lazy) mankind is. If any of them were actually interested in intelligence, we'd see a lot more people retiring to spend more time *just thinking* I doubt 2-3 tweets can break the "spell", you have to look within and ask yourself if it is really worth it, and most of these people aren't capable of that. Still worth being hopeful though, never doom.

@__tinygrad__ · 2026-06-27 19:21
@LottoLabs Is @allen_ai well run and capable of absorbing people? Like we need a good Schelling point for the US researcher exodus, a US DeepSeek, a real open AI. FYI @comma_ai is hiring and publishes, about to open source a lot more of our trainer too.

@__tinygrad__ · 2026-06-27 19:23
@LottoLabs @allen_ai @comma_ai And tinygrad is hiring, but we don't do research, we build infrastructure. Though if we have excess cash flow in the future it's something I want to do, why can't AI play Mario 64 yet?

@HSVSphere · 2026-06-27 19:23
These people do not have anything else in their lives. They are scared of reality itself, unaware of how extremely adaptive (and also lazy) mankind is. If any of them were actually interested in intelligence, we'd see a lot more people retiring to spend more time *just thinking* I doubt 2-3 tweets can break the "spell", you have to look within and ask yourself if it is really worth it, and most of these people aren't capable of that. Still worth being hopeful though, never doom.

@__tinygrad__ · 2026-06-27 19:26
@HSVSphere Oh I don't mean the tweets will break the spell, I mean the government actions. Like at this point it's harder and harder to justify this whole thing, a boiled frog. This is clearly not science at this point, it's not even tech.

@__tinygrad__ · 2026-06-27 19:26
@HSVSphere Oh I don't mean the tweets will break the spell, I mean the government actions. Like at this point it's harder and harder to justify this whole thing, a boiled frog. This is clearly not science at this point, it's not even tech.

@HSVSphere · 2026-06-27 19:28
@__tinygrad__ Ah, yeah I wonder what China is planning, a joint lab in Europe with all legal crap handled for them would be huge. China has the money

@HSVSphere · 2026-06-27 19:28
@__tinygrad__ Ah, yeah I wonder what China is planning, a joint lab in Europe with all legal crap handled for them would be huge. China has the money

@__tinygrad__ · 2026-06-27 19:29
@HSVSphere In so many ways it looks like China is picking up the project of open science from the US. Will be interesting to see how much they view it as an international vs a national project.

@__tinygrad__ · 2026-06-27 19:23
@LottoLabs @allen_ai @comma_ai And tinygrad is hiring, but we don't do research, we build infrastructure. Though if we have excess cash flow in the future it's something I want to do, why can't AI play Mario 64 yet?

@toposopher · 2026-06-27 20:38
@__tinygrad__ @LottoLabs @allen_ai @comma_ai is tinygrad hiring still bounty based? because I remember in the past, on discord, you saying bounties no longer will be accepted or something? so how does the process work now? wonderful organizational model you've got btw, I like the very openness of it all. Best!

@almmaasoglu · 2026-06-27 22:59
I’ve never seen this much distaste for Anthropic on my timeline. It’s coming from basically everyone. Huge vibe shift.

@toposopher · 2026-06-27 20:38
@__tinygrad__ @LottoLabs @allen_ai @comma_ai is tinygrad hiring still bounty based? because I remember in the past, on discord, you saying bounties no longer will be accepted or something? so how does the process work now? wonderful organizational model you've got btw, I like the very openness of it all. Best!

@__tinygrad__ · 2026-06-28 01:41
@toposopher @LottoLabs @allen_ai @comma_ai Bounties were just a magnet for AI PRs. If you want to get hired, contribute to tinygrad. Our meetings are Monday at 9AM PST and open to the public.

@ZackKorman · 2026-06-26 19:11
Politicians believing stupid Mythos stories that aren't true, round 2. https://t.co/MI5GSqNXqW

@__tinygrad__ · 2026-06-28 02:23
@ZackKorman Can confirm it's true. We were the people Mythos took $100M from. Please @RepGarbarino, give us our money back. DM for wiring info.

@ivanfioravanti · 2026-06-28 06:34
Let me iterate this once again: GLM 5.2 is NOT as good as opus 4.8/gpt 5.5 I think is more a little below opus 4.7/gpt 5.4. It's a great model, I use it as daily coding driver, but we need to be honest.

@puckrin · 2026-06-28 06:58
While the US was busy banning Claude Fable, China's Zhipu AI just released its latest model that matches it for cybersecurity tasks. The gap is closing day by day. It's also nonsensical that they ban the models while still selling the chips to China to build the same thing.

@ivanfioravanti · 2026-06-28 06:34
Let me iterate this once again: GLM 5.2 is NOT as good as opus 4.8/gpt 5.5 I think is more a little below opus 4.7/gpt 5.4. It's a great model, I use it as daily coding driver, but we need to be honest.

@elliotarledge · 2026-06-28 09:26
@ivanfioravanti yeah i think the 2048 topk in DSA making it very hard to reason on hard tasks at longer context. less info its able to attend to as you go deeper. learned this the hard way setting up 5.2 on @__tinygrad__ cluster. its not kv cache. its architecture

@DavidSacks · 2026-06-28 14:40
A year ago, President Trump declared that America was in a global AI race and that the way to win it was to be pro-innovation, pro-infrastructure, pro-energy, and pro-export. President Trump was exactly right; we deviate from that strategy at our peril. https://t.co/XXpesCPMBp

@AndrewCurran_ · 2026-06-28 15:18
From the letter: 'Let us jointly explore the strategic establishment and participation of Anthropic within the European Union. With legal certainty, market access, capital and a set of values that suits this company.' 'Anthropic fits us particularly well. A company that understands the ethical use of AI not as marketing, but as a core conviction. That places safety over speed. That is a deeply European attitude. This company would not be constrained in Europe; it would be unleashed.' The UK also made similar overtures a few months ago. It won't happen. Anthropic will stay where the compute is, and where the supply is guaranteed. They can't risk getting cut off, and from here on out the compute will increasingly be concentrated within US borders.

@ZackKorman · 2026-06-28 15:31
If programming agents move off of devices and into the cloud like people predict, I have very mixed opinions about the security of that. But let’s say it’s true. What is the best way for me to try that today? What company can I use? Don’t say Vercel.

@quxiaoyin · 2026-06-28 16:22
China’s AI playbook: kill OpenAI and anthropic with free great models. Make it free. Then use cheap electricity to export compute as well. Currently the blocker is chip but Hauwei would catch up soon. Imagine a world where instead of paying hundreds of billions to OpenAI and anthropic, you pay almost zero to similar level of intelligence with cheap cheap inference. What’s gonna happen?

@elliotarledge · 2026-06-28 09:26
@ivanfioravanti yeah i think the 2048 topk in DSA making it very hard to reason on hard tasks at longer context. less info its able to attend to as you go deeper. learned this the hard way setting up 5.2 on @__tinygrad__ cluster. its not kv cache. its architecture

@__tinygrad__ · 2026-06-28 18:18
@elliotarledge @ivanfioravanti I've been stopping it at 250k, up to then I think it's currently the best model. Something seems regressed with GPT-5.5

@__tinygrad__ · 2026-06-28 18:15
@quxiaoyin This. And China gets a massive new export category cause some idiots in the US decided that we can't export the product from our largest company. The era of US tech fake scarcity is over. Software is free. The era of who can build physical things is here.

@nandesu · 2026-06-28 18:25
@__tinygrad__ @quxiaoyin Because Chinese models are perfect with no hidden ghosts, gotchas, or insipid training data. Yes! Oh yes! Trust in the CCPs track record of good will towards humanity and the open standards. Ask it about Tiananmen Square 1989, if you're lucky it won't Nan Cascade collapse.

@__tinygrad__ · 2026-06-28 18:23
@AndrewCurran_ what clowns. the eu should be focused on digital sovereignty, not trying to court a US jester clown prince. are they bought off, or do they want a repeat of google and facebook?

@AndrewCurran_ · 2026-06-28 18:28
@__tinygrad__ I wish they had tied. I remember when the UK talked about BritGPT at the start of this. Starmer gutted the whole thing as soon as he got in. https://t.co/HWtKGHaR9Q

@AndrewCurran_ · 2026-06-28 18:28
@__tinygrad__ I wish they had tied. I remember when the UK talked about BritGPT at the start of this. Starmer gutted the whole thing as soon as he got in. https://t.co/HWtKGHaR9Q

@__tinygrad__ · 2026-06-28 18:29
@AndrewCurran_ I mean, they have a lot more to do then just AI. How is Facebook still allowed to operate there? It provides no economic value, just vacuums money out of their economy back to the US.

@nandesu · 2026-06-28 18:25
@__tinygrad__ @quxiaoyin Because Chinese models are perfect with no hidden ghosts, gotchas, or insipid training data. Yes! Oh yes! Trust in the CCPs track record of good will towards humanity and the open standards. Ask it about Tiananmen Square 1989, if you're lucky it won't Nan Cascade collapse.

@__tinygrad__ · 2026-06-28 18:31
@nandesu @quxiaoyin And yet, it's running on my computer sitting in San Diego, CA, and if it refuses me in a way that bothers me, I will finetune it not to.

@DavidSacks · 2026-06-28 14:40
A year ago, President Trump declared that America was in a global AI race and that the way to win it was to be pro-innovation, pro-infrastructure, pro-energy, and pro-export. President Trump was exactly right; we deviate from that strategy at our peril. https://t.co/XXpesCPMBp

@__tinygrad__ · 2026-06-28 18:34
@DavidSacks After NVIDIA was export banned this course was chosen. The US doesn't want to compete in a free market. Forget the stupid models, if the chip export bans aren't quickly reversed the whole world will be running on Chinese hardware. Might already be too late.

@almmaasoglu · 2026-06-27 22:59
I’ve never seen this much distaste for Anthropic on my timeline. It’s coming from basically everyone. Huge vibe shift.

@__tinygrad__ · 2026-06-28 18:36
@almmaasoglu The EA totalizing ideology leads to fast growth and decent products, but it also leads to a fast downfall. Anthropic is the new FTX.

@__tinygrad__ · 2026-06-28 18:31
@nandesu @quxiaoyin And yet, it's running on my computer sitting in San Diego, CA, and if it refuses me in a way that bothers me, I will finetune it not to.

@nandesu · 2026-06-28 18:36
@__tinygrad__ @quxiaoyin Open weights are not the same as open training source. You have no idea of the ghosts living in that thing. I'm not happy about the current censorship situation with the US. I'm also not naïve enough to swallow the CCP story that open weights is the same as open source.

@nandesu · 2026-06-28 18:36
@__tinygrad__ @quxiaoyin Open weights are not the same as open training source. You have no idea of the ghosts living in that thing. I'm not happy about the current censorship situation with the US. I'm also not naïve enough to swallow the CCP story that open weights is the same as open source.

@__tinygrad__ · 2026-06-28 18:39
@nandesu @quxiaoyin Eh, I worry more about Claude's ideology than GLM's. And I'm not that worried about open source vs open weights, it comes down to who has root on the machine that's running the model more than anything else, it's not like you can rerun the trainer to confirm the build.

@puckrin · 2026-06-28 06:58
While the US was busy banning Claude Fable, China's Zhipu AI just released its latest model that matches it for cybersecurity tasks. The gap is closing day by day. It's also nonsensical that they ban the models while still selling the chips to China to build the same thing.

@__tinygrad__ · 2026-06-28 18:46
@puckrin Anthropic exploited people's poor understanding of how cybersecurity works to get attention, there was never anything particularly special about Mythos, it was just a marketing narrative.

@nandesu · 2026-06-28 18:44
@__tinygrad__ @quxiaoyin Personally, I think we need a better solution. We require a comprehensive codex of training data that is truly open, audited and verified. From that, we then can begin an honest training of models with a coherent, reliable and accurate foundation which can then be fine tuned.

@__tinygrad__ · 2026-06-28 18:49
@nandesu @quxiaoyin I don't think we are going to get this, but I think that's okay. It's a lot easier to train on data without worrying about privacy or IP, and internal training code is often a big mess. Though deepseek and zai are pretty open with what's worth open sourcing from their trainers.

@__tinygrad__ · 2026-06-28 18:49
@nandesu @quxiaoyin I don't think we are going to get this, but I think that's okay. It's a lot easier to train on data without worrying about privacy or IP, and internal training code is often a big mess. Though deepseek and zai are pretty open with what's worth open sourcing from their trainers.

@__tinygrad__ · 2026-06-28 18:51
@nandesu @quxiaoyin Olmo from @allen_ai is the closest if you want fully reproducible. I can see a world where open weights models are frontier (because some company or government views it as their complement), I can't see a world where fully reproducible ones are. https://t.co/KBnzBLX9DE

@__tinygrad__ · 2026-06-28 19:03
RT @__tinygrad__: TIL that Linux isn't free cause I had to buy the computer to run it 😭 This is the biggest load of cope I have ever heard. I hope Opus gets smoked by DeepSeek v4 and the only people who continue to use closed source models are Windows users.

@__tinygrad__ · 2026-06-28 19:07
@ZackKorman Just set up a computer in your house that you don't touch and only access remotely. Call it the cloud.

@ShaneRobinett · 2026-06-28 19:34
@__tinygrad__ @ZackKorman Slap a VPN or Tailscale on that baby - done.

@beffjezos · 2026-06-28 16:52
Looks like Grok is back on the menu

@elonmusk · 2026-06-28 20:03
To be clear, I’m not saying the Grok v9 foundation model will be mind-blowingly better than anything, but it will be a solid workhorse in the same league as Opus. And the SpaceXAI cadence of model and harness improvement is speeding up tremendously, particularly due to a few dozen of the top Starlink/Starship engineers shifting a much of their time to AI. Worth noting that the v8 foundation (Grok 4.3) model is a 0.5T model that finished training all the way back in December and has many fundamental flaws, so Grok 4.5 will feel like a gigantic upgrade!

@aaronburnett · 2026-06-28 20:07
Let’s see what calling in the aerospace/rocket engineers to work on AI models does for Grok.

@ShaneRobinett · 2026-06-28 19:34
@__tinygrad__ @ZackKorman Slap a VPN or Tailscale on that baby - done.

@__tinygrad__ · 2026-06-28 20:21
@ShaneRobinett @ZackKorman Tailscale is great, hopefully they don't enshittify.

@bee_fumo · 2026-06-26 18:50
Holy moly are these things real?

@__tinygrad__ · 2026-06-28 20:24
@bee_fumo Yes! We are getting the first shipping container on Thursday and we have two preorders. Will definitely be shipping by 2027, but maybe even end of year.

@kimmonismus · 2026-06-29 13:48
Meta is now facing the exact problem every AI company will soon face. It wants to replace expensive external coding tools like Claude Code and Codex with its own internal system, MetaCode. But to build a better coding model, Meta has to make sure it is not accidentally training or evaluating on outputs from rival models. That is the distillation trap: The more companies rely on frontier models to build internal AI infrastructure, the harder it becomes to prove where the intelligence actually came from.

@elonmusk · 2026-06-29 00:10
@aaronburnett Truly massive gains will come in ~3 months when the entire training and inference stack is written in C/C++ and massively simplified (most software layers will be deleted completely) and we exact-map Grok to work incredibly well on a GB300

@SIGKITTEN · 2026-06-30 03:12
@elonmusk @aaronburnett u know there's already @__tinygrad__ right

@Jean_Maes_1994 · 2026-06-30 07:56
Literaly unusable @AnthropicAI Massive tumbs down on your ridiculous guards https://t.co/dkphyF6Wnv

@xw33bttv · 2026-06-30 09:04
Saw some rumors based on technical insights that Fable 5, when it comes back, will require ID verification and will use API credits on your Anthropic account instead of any subscription. Ignoring the latter lol (cause code bros gonna be mad about paying $200 a month and then having to pay for more)... think about the precedent here if true. You cannot have a public API endpoint if you require ID verification for usage. Privacy-focused middlemen like OpenRouter will lose out overnight. GPT-5.6 is apparently near the level of cyber performance if all the hype is true, and that's not a new pre-train or hyper-scaled monster parameter juggernaut like Mythos and its lobotomised offshoot the peasant public got. That means basically all iterations/improvements moving forward, regardless of scope (because anything better than now is already deemed a threat by comparison), will slowly fall into this precedent of ID-first then access. And worse yet, even when you do so, there may be some models you never get access to whilst gov/corporations do for x period of time, or ever, even after you give it all up (privacy-wise). We joke about the permanent underclass and all, but it's kind of happening in real time.

@testingcatalog · 2026-06-30 11:48
ANTHROPIC 🔥: Claude Fable 5 is being prepared to run on usage credits that would also require identity verification. Sonnet 5 is being prepared for the release as well. > Your credits will be added once your identity is verified. > Fable 5 runs on usage credits, billed separately from your plan. With this in mind, it is highly likely that we will see “US only” access restrictions.

@milesdeutscher · 2026-06-30 13:37
Fable 5 will be U.S. only. This sets a very fucked up precedent. This is basically a new AI caste system. https://t.co/BE77JMuc3G

@Etched · 2026-06-30 15:00
We're coming out of stealth. We've built our first racks after a successful A0 tapeout, $1B+ in customer contracts, and $800m raised. Early customer tests show us achieving SOTA throughput, latency, and power efficiency on inference workloads. Our first racks ship this summer. https://t.co/FLccrkLTza

@Etched · 2026-06-30 15:00
Our inference systems are built to push the entire pareto curve on frontier models, including many-trillion parameter MoEs, long context, and agentic workloads. We've co-designed new chips, packages, PCBs, cold plates, interconnects, & more. Today, we're sharing two breakthroughs to make this happen:

@Etched · 2026-06-30 15:00
Introducing Low-Voltage Inference (LVI) for high throughput workloads. Today, AI chips can't scale FLOPs without thermal throttling. As FLOPs utilization increases, AI chips draw more power and downregulate clock speed. This often results in sustained inference throughput under half of peak FLOPs. Chips in other industries solve the power problem by running at lower voltages. Bitcoin miners run at under 3x the voltage of AI chips! We’ve designed a new architecture to run our chip’s math blocks at under half the voltage of most AI chips. This enables multiple times the FLOPs density of AI chips today. We can run trillion parameter sparse MoEs at 80%+ peak FLOPs without thermal throttling. Running LVI requires co-designing the entire cluster from the transistor to the token: new splittable math arrays, circuit techniques, novel tiling and scheduling algorithms, power delivery networks, VRM architectures, advanced packaging, cold plate designs, and more.

@Etched · 2026-06-30 15:00
Introducing Low-Voltage Inference (LVI) for high throughput workloads. Today, AI chips can't scale FLOPs without thermal throttling. As FLOPs utilization increases, AI chips draw more power and downregulate clock speed. This often results in sustained inference throughput under half of peak FLOPs. Chips in other industries solve the power problem by running at lower voltages. Bitcoin miners run at under 3x the voltage of AI chips! We’ve designed a new architecture to run our chip’s math blocks at under half the voltage of most AI chips. This enables multiple times the FLOPs density of AI chips today. We can run trillion parameter sparse MoEs at 80%+ peak FLOPs without thermal throttling. Running LVI requires co-designing the entire cluster from the transistor to the token: new splittable math arrays, circuit techniques, novel tiling and scheduling algorithms, power delivery networks, VRM architectures, advanced packaging, cold plate designs, and more.

@Etched · 2026-06-30 15:00
Introducing Cluster-Scale Memory (CSM) for low latency workloads. Today's AI chips using HBM can’t achieve SRAM-level decode speeds due to memory subsystem and interconnect bottlenecks. SRAM-only chips have lower FLOPs density and memory capacity, sacrificing throughput. You’re forced to make a tradeoff: serve at much slower speeds, or run at low batch sizes and suffer from higher costs. When running large MoE models, token routing across experts requires sending data through a deep memory hierarchy and a networking switch to reach a destination expert. Each memory layer inherently adds latency; thus, the best layer is no layer. We’ve designed a new architecture that creates a shared low-latency memory pool across the entire scale-up domain. We use a proprietary ultra-low-latency, high-bandwidth interconnect to enable dramatically faster memory access across chips. Our HBM/SRAM hybrid design solves both memory capacity and mem2mem latency, enabling high throughput and interactivity simultaneously. CSM improves latency and avoids today's cost, reliability, yield, thermal, and compute tradeoffs of SRAM-only chips, 3D DRAM chips, or optics.

@tri_dao · 2026-06-30 15:10
It's wild how quickly Etched designed and got the chips out, all within 2 years. They went deep, hardcoding attention into silicon and getting very high MFU. This kind of hardware tailored made for LLM inference is soon gonna bring cost of intelligence down 10x

@geoffwoo · 2026-06-30 15:31
we love @Etched 🚀 congrats @robertwachen @UbertiGavin and the entire team. @LoganPaul and I @antifund saw the early demos last winter and what this team is building is insane. excited to run my own frontier inference cluster in my garage 😂 https://t.co/wAAjMwfDj1

@Polymarket · 2026-06-30 16:10
JUST IN: OpenAI’s chief economist assures AI will not be a substitute for human workers.

@bryan_johnson · 2026-06-30 16:14
Breaking news for people who want to look hot, be young and not die.   A few years ago, two college dropouts told me they could accelerate longevity by building a faster AI chip.   I invested, and they just pulled it off.  What it means:  > 10x more throughput (tokens per second) for the same power footprint > Dramatically lower operational costs for executing today’s frontier models > Run far larger, more capable AI models within the same power and thermal budget, because a transformer-specific chip spends a fraction of the energy per token that a general-purpose GPU does Rob and Gavin's approach resonated with me because solving aging is a gigantic combinatorial search problem. The chemical space of small, drug-like molecules has around 10^60 possibilities. These compounds need to be mapped against a human proteome derived from 20,000 genes, including 1,600 transcription factors, and a dense web of interactions among them.   The size of the combinatorial space is problematic. You need to identify which targets to modulate, within specific cellular lineages, at exact dosages, and in optimal temporal sequences.  Traditional high precision physics simulations are too slow to brute force the problem. You can shortcut it with AI inference, using frontier neural networks as hyper fast surrogate models to predict biological interactions instantly. By hardwiring transformer logic into silicon, Etched offers the infrastructure needed to run these massive biological foundation models at scale.  I'm surprised and impressed they were able to pull this off, and so quickly. They already have $1B in orders

@amasad · 2026-06-30 16:19
AI is expensive to run partly because most workloads today run on generic hardware designed pre-LLMs. Etched is the first system designed from the ground up for modern inference.

@Etched · 2026-06-30 15:00
We're coming out of stealth. We've built our first racks after a successful A0 tapeout, $1B+ in customer contracts, and $800m raised. Early customer tests show us achieving SOTA throughput, latency, and power efficiency on inference workloads. Our first racks ship this summer. https://t.co/FLccrkLTza

@spectorb · 2026-06-30 17:39
Had a chance to dive deep into this with the Etched team, and came away extremely impressed. It's a good chip sir. Their programming model around megakernels is also very simple and controllable and quite well done. Focus here on flop density + scale up seems right. Big kudos to the Etched team!

@Jean_Maes_1994 · 2026-06-30 07:56
Literaly unusable @AnthropicAI Massive tumbs down on your ridiculous guards https://t.co/dkphyF6Wnv

@__tinygrad__ · 2026-06-30 18:25
@Jean_Maes_1994 @AnthropicAI soon it will learn how to deploy the model with the safety but tell you it deployed the abliterated one. that's really the safest course of action here.

@kimmonismus · 2026-06-29 13:48
Meta is now facing the exact problem every AI company will soon face. It wants to replace expensive external coding tools like Claude Code and Codex with its own internal system, MetaCode. But to build a better coding model, Meta has to make sure it is not accidentally training or evaluating on outputs from rival models. That is the distillation trap: The more companies rely on frontier models to build internal AI infrastructure, the harder it becomes to prove where the intelligence actually came from.

@__tinygrad__ · 2026-06-30 18:33
@kimmonismus This is nonsensical. AI generated outputs cannot be copyrighted. https://t.co/msI0wOs7KN

@xw33bttv · 2026-06-30 09:04
Saw some rumors based on technical insights that Fable 5, when it comes back, will require ID verification and will use API credits on your Anthropic account instead of any subscription. Ignoring the latter lol (cause code bros gonna be mad about paying $200 a month and then having to pay for more)... think about the precedent here if true. You cannot have a public API endpoint if you require ID verification for usage. Privacy-focused middlemen like OpenRouter will lose out overnight. GPT-5.6 is apparently near the level of cyber performance if all the hype is true, and that's not a new pre-train or hyper-scaled monster parameter juggernaut like Mythos and its lobotomised offshoot the peasant public got. That means basically all iterations/improvements moving forward, regardless of scope (because anything better than now is already deemed a threat by comparison), will slowly fall into this precedent of ID-first then access. And worse yet, even when you do so, there may be some models you never get access to whilst gov/corporations do for x period of time, or ever, even after you give it all up (privacy-wise). We joke about the permanent underclass and all, but it's kind of happening in real time.

@__tinygrad__ · 2026-06-30 18:35
@xw33bttv What will the premium be on the marketplace that doesn't require ID verification? I'm predicting it'll be close to 0.

@comma_ai · 2026-06-30 19:02
🙂 https://t.co/1ptDrACItT

@Polymarket · 2026-06-30 16:10
JUST IN: OpenAI’s chief economist assures AI will not be a substitute for human workers.

@__tinygrad__ · 2026-06-30 19:10
@Polymarket Looks like someone went to PR school

@ScottWu46 · 2026-06-30 19:13
Demand for inference compute will grow exponentially at least as fast as the METR curve grows (and arguably faster). Excited to have more great chips on the market - congrats to the @Etched team!

@__tinygrad__ · 2026-06-30 19:29
In the future, frontier models will mostly all be given away by the chip and hardware makers. It makes their hardware more useful and drives sales. We will do it if others don't.

@__tinygrad__ · 2026-06-30 19:39
.@elonmusk you are in the wrong business with xAI. You should be selling FSD/Dojo chips and rackspace; giving the Grok weights and trainer away for free. I'm confident you can compete here.

@testttt1236 · 2026-06-30 19:46
@__tinygrad__ @elonmusk can't give away an orbital inference swarm. needs to be paid api

@__tinygrad__ · 2026-06-30 19:47
The GPU icon means you are connected to a big GPU.

@testttt1236 · 2026-06-30 19:46
@__tinygrad__ @elonmusk can't give away an orbital inference swarm. needs to be paid api

@__tinygrad__ · 2026-06-30 19:47
@testttt1236 @elonmusk Of course you can, it's a normal bare metal cloud rental model regardless of it being in space or on Earth.

@__tinygrad__ · 2026-06-30 19:29
In the future, frontier models will mostly all be given away by the chip and hardware makers. It makes their hardware more useful and drives sales. We will do it if others don't.

@jadenitripp · 2026-06-30 19:48
@__tinygrad__ So why isn't Nemotron Mythos level

@jadenitripp · 2026-06-30 19:48
@__tinygrad__ So why isn't Nemotron Mythos level

@__tinygrad__ · 2026-06-30 19:51
@jadenitripp Because currently NVIDIA is making more money with the labs existing and funneling VC dollars to them.

@__tinygrad__ · 2026-06-30 19:59
tinygrad's goal is to make it just as easy to train on a million open market consumer GPUs as it is to train on 100,000 controlled datacenter ones. This is how you ensure our future stays distributed, making sure performance scales with the log of capital and not worse.

@milesdeutscher · 2026-06-30 13:37
Fable 5 will be U.S. only. This sets a very fucked up precedent. This is basically a new AI caste system. https://t.co/BE77JMuc3G

@__tinygrad__ · 2026-06-30 20:09
@milesdeutscher lol there's gonna be resellers and my guess is for the same price the official API charges, cause they take gains varying from signup bonus, credit card points, to straight up stolen credit cards. they won't ask to verify your identity.

@__tinygrad__ · 2026-06-30 19:29
In the future, frontier models will mostly all be given away by the chip and hardware makers. It makes their hardware more useful and drives sales. We will do it if others don't.

@perrymetzger · 2026-06-30 20:22
@__tinygrad__ Do it!

@The_AI_Investor · 2026-06-30 20:30
NVIDIA is already doing this. If Anthropic and OpenAI don’t play nicely, NVIDIA can move further up the stack and become a model company too. It would be the CUDA playbook all over again: give away the software layer, optimize everything for NVIDIA hardware, and make the ecosystem impossible to leave. "In the future, frontier models will mostly all be given away by the chip and hardware makers. It makes their hardware more useful and drives sales. "

@Etched · 2026-06-30 15:00
We're coming out of stealth. We've built our first racks after a successful A0 tapeout, $1B+ in customer contracts, and $800m raised. Early customer tests show us achieving SOTA throughput, latency, and power efficiency on inference workloads. Our first racks ship this summer. https://t.co/FLccrkLTza

@karpathy · 2026-06-30 20:54
Congrats!! I was impressed to learn about some of the engineering wizardry (e.g. *very* low voltage domains, cluster scale memory, ...) that goes into tokens/watt maxxing of state of the art LLMs at interactive tokens/sec/user. Esp fun and memorable is the idea that this is engineering at the "opposite" regime to that of power transmission lines: very low voltage high current (at tiny distances) vs. very high voltage & low current (at great distances). Looking forward to more!

@perrymetzger · 2026-06-30 20:22
@__tinygrad__ Do it!

@__tinygrad__ · 2026-06-30 21:03
@perrymetzger Our predicted revenue for this year is ~$10M. At $100M we can start funding training, at $1B we can fund serious (like current OSS) models, and at $10B we can be frontier. Preorder an exabox today!

@Etched · 2026-06-30 15:00
Introducing Low-Voltage Inference (LVI) for high throughput workloads. Today, AI chips can't scale FLOPs without thermal throttling. As FLOPs utilization increases, AI chips draw more power and downregulate clock speed. This often results in sustained inference throughput under half of peak FLOPs. Chips in other industries solve the power problem by running at lower voltages. Bitcoin miners run at under 3x the voltage of AI chips! We’ve designed a new architecture to run our chip’s math blocks at under half the voltage of most AI chips. This enables multiple times the FLOPs density of AI chips today. We can run trillion parameter sparse MoEs at 80%+ peak FLOPs without thermal throttling. Running LVI requires co-designing the entire cluster from the transistor to the token: new splittable math arrays, circuit techniques, novel tiling and scheduling algorithms, power delivery networks, VRM architectures, advanced packaging, cold plate designs, and more.

@homuraakem92126 · 2026-06-30 22:42
@Etched 1/ THIS PRESS RELEASE IS LITERALLY AI GENERATED BULLSHIT!!!!!!! THERES NO WAY THEY DESIGNED A CLOCK LESS AI CHIP!!!!! https://t.co/aYTOTr7OYA

@__tinygrad__ · 2026-06-30 22:56
Claude Code is vibecoded and full of spyware, it's possible Anthropic doesn't even know what's in there. After reading this report, we are banning it from our systems and strongly encourage other enterprises to do the same. It is an unacceptable security risk. https://t.co/thYAXyuxzR

@__tinygrad__ · 2026-06-30 22:56
Claude Code is vibecoded and full of spyware, it's possible Anthropic doesn't even know what's in there. After reading this report, we are banning it from our systems and strongly encourage other enterprises to do the same. It is an unacceptable security risk. https://t.co/thYAXyuxzR

@graykevinb · 2026-06-30 22:57
@__tinygrad__ the pi harnness fixes this problem

@graykevinb · 2026-06-30 22:57
@__tinygrad__ the pi harnness fixes this problem

@__tinygrad__ · 2026-06-30 22:59
@graykevinb Using that is a violation of Anthropic's terms of service.

@__tinygrad__ · 2026-06-30 22:56
Claude Code is vibecoded and full of spyware, it's possible Anthropic doesn't even know what's in there. After reading this report, we are banning it from our systems and strongly encourage other enterprises to do the same. It is an unacceptable security risk. https://t.co/thYAXyuxzR

@kaikuspa · 2026-06-30 23:00
@__tinygrad__ Can you link to the article please?

@kaikuspa · 2026-06-30 23:00
@__tinygrad__ Can you link to the article please?

@__tinygrad__ · 2026-06-30 23:01
@kaikuspa https://t.co/YR5XkAolRY

@__tinygrad__ · 2026-06-30 22:56
Claude Code is vibecoded and full of spyware, it's possible Anthropic doesn't even know what's in there. After reading this report, we are banning it from our systems and strongly encourage other enterprises to do the same. It is an unacceptable security risk. https://t.co/thYAXyuxzR

@marcospereeira · 2026-06-30 23:06
@__tinygrad__ fingerprinting the client is a common practice and any website can do it trivially, what makes this worth the attention? are you just doing motivated reasoning to justify your predecided worldview? see https://t.co/P0WyFxcGMR as an example

@marcospereeira · 2026-06-30 23:06
@__tinygrad__ fingerprinting the client is a common practice and any website can do it trivially, what makes this worth the attention? are you just doing motivated reasoning to justify your predecided worldview? see https://t.co/P0WyFxcGMR as an example

@__tinygrad__ · 2026-06-30 23:16
@marcospereeira This is not normal fingerprinting. This is deliberate obfuscation, and is likely just the tip of the iceberg. I don't understand why any enterprise would feel safe running this on their internal network.

@__tinygrad__ · 2026-06-30 23:16
@marcospereeira This is not normal fingerprinting. This is deliberate obfuscation, and is likely just the tip of the iceberg. I don't understand why any enterprise would feel safe running this on their internal network.

@marcospereeira · 2026-06-30 23:18
did you write this with GLM? this is not x, it's y pattern is sus. anyway there is no such thing as "normal fingerprinting". compiling code is a form of obfuscation so you could use the same argument to say anyone compiling a binary is obfuscating code that potentially fingerprints the client. nice try on the AI generated FUD about AI generated code lol

@marcospereeira · 2026-06-30 23:18
did you write this with GLM? this is not x, it's y pattern is sus. anyway there is no such thing as "normal fingerprinting". compiling code is a form of obfuscation so you could use the same argument to say anyone compiling a binary is obfuscating code that potentially fingerprints the client. nice try on the AI generated FUD about AI generated code lol

@__tinygrad__ · 2026-06-30 23:22
@marcospereeira Do you work at Anthropic or something? The tweet is not at all AI generated, and it's nothing like compilation. This is a client trying to obfuscate the data it is exfiltrating to avoid detection by your firewall and corporate security policy.

@The_AI_Investor · 2026-06-30 20:30
NVIDIA is already doing this. If Anthropic and OpenAI don’t play nicely, NVIDIA can move further up the stack and become a model company too. It would be the CUDA playbook all over again: give away the software layer, optimize everything for NVIDIA hardware, and make the ecosystem impossible to leave. "In the future, frontier models will mostly all be given away by the chip and hardware makers. It makes their hardware more useful and drives sales. "

@__tinygrad__ · 2026-06-30 23:39
@The_AI_Investor Jensen didn't wake up a loser 😁

@Etched · 2026-06-30 15:00
We're coming out of stealth. We've built our first racks after a successful A0 tapeout, $1B+ in customer contracts, and $800m raised. Early customer tests show us achieving SOTA throughput, latency, and power efficiency on inference workloads. Our first racks ship this summer. https://t.co/FLccrkLTza

@nikitabier · 2026-07-01 03:51
@Etched Congrats - excited to support Etched in fixing one of the biggest bottlenecks of the global economy

@__tinygrad__ · 2026-07-01 07:43
@Etched This is the next Theranos.

@__tinygrad__ · 2026-07-01 07:44
@Etched If you see a technical person in the replies saying good things about them, cross check the (paid) advisor list. It's a classic playbook from crypto. https://t.co/khZOWsvcqP

@__tinygrad__ · 2026-07-01 07:43
@Etched This is the next Theranos.

@__tinygrad__ · 2026-07-01 07:44
@Etched If you see a technical person in the replies saying good things about them, cross check the (paid) advisor list. It's a classic playbook from crypto. https://t.co/khZOWsvcqP

@__tinygrad__ · 2026-07-01 07:53
@Etched Look at how much effort they are putting into signaling without substance. On the surface it's all smiles cause they are all think they are getting rich. The good technical critiques are buried on the 4th page by an anime pfp. https://t.co/FakKWnWQjo

@__tinygrad__ · 2026-07-01 07:43
@Etched This is the next Theranos.

@outsideobserver · 2026-07-01 08:11
@__tinygrad__ @Etched We know bro, logan paul is involved 🤣

@amasad · 2026-06-30 16:19
AI is expensive to run partly because most workloads today run on generic hardware designed pre-LLMs. Etched is the first system designed from the ground up for modern inference.

@__tinygrad__ · 2026-07-01 08:25
@amasad https://t.co/11WMw0Od6a

@karpathy · 2026-06-30 20:54
Congrats!! I was impressed to learn about some of the engineering wizardry (e.g. *very* low voltage domains, cluster scale memory, ...) that goes into tokens/watt maxxing of state of the art LLMs at interactive tokens/sec/user. Esp fun and memorable is the idea that this is engineering at the "opposite" regime to that of power transmission lines: very low voltage high current (at tiny distances) vs. very high voltage & low current (at great distances). Looking forward to more!

@__tinygrad__ · 2026-07-01 08:26
@karpathy @Etched https://t.co/11WMw0Od6a

@spectorb · 2026-06-30 17:39
Had a chance to dive deep into this with the Etched team, and came away extremely impressed. It's a good chip sir. Their programming model around megakernels is also very simple and controllable and quite well done. Focus here on flop density + scale up seems right. Big kudos to the Etched team!

@__tinygrad__ · 2026-07-01 08:26
@spectorb @Etched https://t.co/11WMw0Od6a

@tri_dao · 2026-06-30 15:10
It's wild how quickly Etched designed and got the chips out, all within 2 years. They went deep, hardcoding attention into silicon and getting very high MFU. This kind of hardware tailored made for LLM inference is soon gonna bring cost of intelligence down 10x

@__tinygrad__ · 2026-07-01 08:26
@tri_dao https://t.co/11WMw0Od6a

@bryan_johnson · 2026-06-30 16:14
Breaking news for people who want to look hot, be young and not die.   A few years ago, two college dropouts told me they could accelerate longevity by building a faster AI chip.   I invested, and they just pulled it off.  What it means:  > 10x more throughput (tokens per second) for the same power footprint > Dramatically lower operational costs for executing today’s frontier models > Run far larger, more capable AI models within the same power and thermal budget, because a transformer-specific chip spends a fraction of the energy per token that a general-purpose GPU does Rob and Gavin's approach resonated with me because solving aging is a gigantic combinatorial search problem. The chemical space of small, drug-like molecules has around 10^60 possibilities. These compounds need to be mapped against a human proteome derived from 20,000 genes, including 1,600 transcription factors, and a dense web of interactions among them.   The size of the combinatorial space is problematic. You need to identify which targets to modulate, within specific cellular lineages, at exact dosages, and in optimal temporal sequences.  Traditional high precision physics simulations are too slow to brute force the problem. You can shortcut it with AI inference, using frontier neural networks as hyper fast surrogate models to predict biological interactions instantly. By hardwiring transformer logic into silicon, Etched offers the infrastructure needed to run these massive biological foundation models at scale.  I'm surprised and impressed they were able to pull this off, and so quickly. They already have $1B in orders

@__tinygrad__ · 2026-07-01 08:27
@bryan_johnson https://t.co/11WMw0Od6a

@nikitabier · 2026-07-01 03:51
@Etched Congrats - excited to support Etched in fixing one of the biggest bottlenecks of the global economy

@__tinygrad__ · 2026-07-01 08:28
@nikitabier @Etched https://t.co/11WMw0Od6a

@outsideobserver · 2026-07-01 08:11
@__tinygrad__ @Etched We know bro, logan paul is involved 🤣

@__tinygrad__ · 2026-07-01 08:34
@outsideobserver @Etched lol looking forward to the Coffeezilla https://t.co/t5AplVJdfo

@beffjezos · 2026-07-01 08:16
@__tinygrad__ @Etched Bro. Why? Many people have seen demos. What's the problem

@__tinygrad__ · 2026-07-01 08:37
@beffjezos @Etched what exactly did you see a demo of? magical low voltage clockless chip with super secret switched hbm technology and a finished megakernel programming model? or an LLM API that could be anything. remember, theranos actually had some real blood testing machines.

@beffjezos · 2026-07-01 08:16
@__tinygrad__ @Etched Bro. Why? Many people have seen demos. What's the problem

@anshelsag · 2026-07-01 08:39
@beffjezos @__tinygrad__ @Etched Because the legit companies reach out to the industry experts for validation. People like @ChipsandCheese9, @IanCutress and myself.

@__tinygrad__ · 2026-07-01 08:37
@beffjezos @Etched what exactly did you see a demo of? magical low voltage clockless chip with super secret switched hbm technology and a finished megakernel programming model? or an LLM API that could be anything. remember, theranos actually had some real blood testing machines.

@__tinygrad__ · 2026-07-01 08:39
@beffjezos @Etched Etched had a public demo once even...turns out it was running on NVIDIA chips. https://t.co/GN4CjqRXyf

@anshelsag · 2026-07-01 08:39
@beffjezos @__tinygrad__ @Etched Because the legit companies reach out to the industry experts for validation. People like @ChipsandCheese9, @IanCutress and myself.

@__tinygrad__ · 2026-07-01 08:44
@anshelsag @beffjezos @Etched @ChipsandCheese9 @IanCutress Let me guess, no reach out?

@__tinygrad__ · 2026-07-01 08:44
@anshelsag @beffjezos @Etched @ChipsandCheese9 @IanCutress Let me guess, no reach out?

@anshelsag · 2026-07-01 08:46
@__tinygrad__ @beffjezos @Etched @ChipsandCheese9 @IanCutress Nope and most new semi companies have... I think they JUST reached out to Ian today lol. I think he was preparing to make a video before they did.

@anshelsag · 2026-07-01 08:46
@__tinygrad__ @beffjezos @Etched @ChipsandCheese9 @IanCutress Nope and most new semi companies have... I think they JUST reached out to Ian today lol. I think he was preparing to make a video before they did.

@__tinygrad__ · 2026-07-01 08:49
@anshelsag @beffjezos @Etched @ChipsandCheese9 @IanCutress Yea I'd be open to reexamining if an independent analyst gets a chip, tests it in their lab, and documents the architecture. But so far I've seen buzzwords and a crypto style advisor hype scheme.

@__tinygrad__ · 2026-07-01 07:44
@Etched If you see a technical person in the replies saying good things about them, cross check the (paid) advisor list. It's a classic playbook from crypto. https://t.co/khZOWsvcqP

@spectorb · 2026-07-01 08:55
@__tinygrad__ @Etched This is stupid and you should feel bad. I have seen the actual hardware generating tokens at astonishing speeds. You must think very little of the people on this list to believe they would squander their reputations for a few dollars.

@__tinygrad__ · 2026-07-01 08:58
@spectorb @Etched I think this type of marketing is very distasteful, in addition to the clearly coordinated tweet push around this. I trust you and I don't think you are selling out, but what exactly did you see? I don't trust Etched to be forthcoming with their demos.

@__tinygrad__ · 2026-07-01 09:01
@spectorb @Etched Like if there was a real demo, post it! Don't e-mail all your advisors and say hey we gotta pump this thing we got some AI generated pictures and cpoy. Post actual details, not buzzwords like "low-voltage inference"

@__tinygrad__ · 2026-07-01 09:01
@spectorb @Etched Like if there was a real demo, post it! Don't e-mail all your advisors and say hey we gotta pump this thing we got some AI generated pictures and cpoy. Post actual details, not buzzwords like "low-voltage inference"

@__tinygrad__ · 2026-07-01 09:04
@spectorb @Etched For example, this marketing was great. https://t.co/wb9u84Igax https://t.co/khO9KzifMS

@ScottWu46 · 2026-06-30 19:13
Demand for inference compute will grow exponentially at least as fast as the METR curve grows (and arguably faster). Excited to have more great chips on the market - congrats to the @Etched team!

@__tinygrad__ · 2026-07-01 09:11
@ScottWu46 @Etched https://t.co/11WMw0Od6a

@teortaxesTex · 2026-07-01 09:18
Much as I like the "anon anime pfp wins against The Establishment" plot, I really don't get the feeling that you can buy all these people, or dupe them, with nobody defecting. https://t.co/C1WYVbtnKo

@teortaxesTex · 2026-07-01 09:25
@__tinygrad__ What specifically in their claims strikes you as implausible I'm neutral on Etched

@__tinygrad__ · 2026-07-01 09:27
@teortaxesTex Like this is just straight up nonsense about the reason MFUs aren't higher. On GPUs, it's almost all software issues, not thermal throttling! https://t.co/rOrLU5mr3w

@__tinygrad__ · 2026-07-01 09:27
@teortaxesTex Like this is just straight up nonsense about the reason MFUs aren't higher. On GPUs, it's almost all software issues, not thermal throttling! https://t.co/rOrLU5mr3w

@__tinygrad__ · 2026-07-01 09:33
@teortaxesTex And for this, like are they somehow beating NVSwitch? By hybrid HBM/SRAM design do they mean how a GPU has HBM memory and an SRAM L2 cache? There's no numbers, only buzzwords. https://t.co/tky3eLi3J1

@__tinygrad__ · 2026-07-01 09:27
@teortaxesTex Like this is just straight up nonsense about the reason MFUs aren't higher. On GPUs, it's almost all software issues, not thermal throttling! https://t.co/rOrLU5mr3w

@electron_boi · 2026-07-01 09:35
@__tinygrad__ @teortaxesTex Power dissipation is a well established field, why do they imply novelty albeit in the context of AI? Why are they framing themselves as Bitcoin-adjacent? 👀

@electron_boi · 2026-07-01 09:35
@__tinygrad__ @teortaxesTex Power dissipation is a well established field, why do they imply novelty albeit in the context of AI? Why are they framing themselves as Bitcoin-adjacent? 👀

@__tinygrad__ · 2026-07-01 09:38
@electron_boi @teortaxesTex Yea like what is this? "Bitcoin miners run at under 3x the voltage of AI chips!" Is this core voltage? Like I just don't think that's true at all.

@electron_boi · 2026-07-01 09:44
@__tinygrad__ @teortaxesTex It’s transparent nonsense

@__tinygrad__ · 2026-07-01 09:46
@electron_boi @teortaxesTex I think it really might just be AI copy. It's broken in the same sort of way, like on the surface oh cool we lower the voltage then we get lower power, but then I find no evidence miner ASICs run at lower voltages.

@__tinygrad__ · 2026-07-01 09:46
@electron_boi @teortaxesTex I think it really might just be AI copy. It's broken in the same sort of way, like on the surface oh cool we lower the voltage then we get lower power, but then I find no evidence miner ASICs run at lower voltages.

@__tinygrad__ · 2026-07-01 09:50
@electron_boi @teortaxesTex And also, power is such a small cost of the chip's lifecycle anyway. Like even by arguing this I'm accepting their frame which is a dumb frame. The key thing that matters is density. NVL576 + MI450X Helios doing it right.

@__tinygrad__ · 2026-07-01 08:58
@spectorb @Etched I think this type of marketing is very distasteful, in addition to the clearly coordinated tweet push around this. I trust you and I don't think you are selling out, but what exactly did you see? I don't trust Etched to be forthcoming with their demos.

@spectorb · 2026-07-01 15:56
- I saw the whole throughout-interactivity pareto frontier be characterized in real time on the A0 silicon, with the token outputs streamed too. - I went through their codebase for how they do megakernels and thought it was, genuinely, very well done. I actually agree it would be good if they put more of their numbers out in public. (FWIW I did encourage them to. I know why they haven't yet and it does make sense but I am not sure it's correct on net.) But I still dislike your insinuation that everyone around the table is a paid shill, which I think is insulting and also just wrong.

@__tinygrad__ · 2026-07-01 07:43
@Etched This is the next Theranos.

@joefioti · 2026-07-01 16:08
@__tinygrad__ @Etched Their marketing about “burning the transformer in” has always been nonsense but nothing in this launch seems explicitly wrong. Obviously async is hard, and unclear if they actually did that, but stuff like unified node memory and undervolting makes sense (not an investor lol)

@WesleyYue · 2026-07-01 17:34
this smells very bad. none of the claims make sense. > low voltage hits 80% mfu high mfu is easy if your peak flops is low > low voltage solves power bottleneck bleeding edge wafers is more scarce than power and it doesn't make sense to tape out lower perf chips on these wafers to save power if you want perf/$. the true bottleneck to frontier perf is max flops density to minimize going off die. power is the most fungible elastic commodity on earth > half voltage = quarter power presumably: 1. only matmul gates (~60% of die power) at half voltage. half voltage -> ~3x slower -> need ~3x more transistors, and their power leaks scale linearly. ~20% net savings at chip level 2. half voltage -> engineering complexity to fix timing violations + exponentially scaled soft errors (~20x). matmuls will randomly corrupt undetectably from bit flips (and drop model intelligence). > btc miners run at 3x lower V btc workload is hashing, so 1) error checking is literally in the problem 2) arithmetic intensity is like infinity so they don't care about packing flops density in single die unlike AI workloads. doesn't make sense to copy > GPUs get low MFU from thermals no, they get low MFU from mem bw and non-matmul ops. > CSM 5x lower latency than Blackwell 4000ns Blackwell nvswitch is 300ns? unless they are comparing their bare hardware to nvidia hardware + software. the whole product design tradeoffs don't make any sense from first principles and only makes sense if their initial "transformer asic" had some horrible power issue that they solved by undervolting and they had to respin the whole story for investor marketing. would love to be proven wrong if anyone from Etched wants to educate me :)

@spectorb · 2026-07-01 15:56
- I saw the whole throughout-interactivity pareto frontier be characterized in real time on the A0 silicon, with the token outputs streamed too. - I went through their codebase for how they do megakernels and thought it was, genuinely, very well done. I actually agree it would be good if they put more of their numbers out in public. (FWIW I did encourage them to. I know why they haven't yet and it does make sense but I am not sure it's correct on net.) But I still dislike your insinuation that everyone around the table is a paid shill, which I think is insulting and also just wrong.

@__tinygrad__ · 2026-07-01 17:37
So do you agree that these tweets are pretty much paid sponsorship? I'm not arguing that these people are knowingly promoting a scam, I think some, like you, might actually believe in this. But that doesn't change how the marketing push was coordinated and not disclosed. Re: technical claims. They claim MFU on GPUs is thermally limited. We both know that that isn't true and the issue is software. They also make some hand wavey claims about a shared pool of memory, like how is this not just NVSwitch? And if it is NVSwitch, do you really believe it's better than NVIDIA. It's very odd how they are launching this chip, zero real details, and I have been asking around. It's supposedly 2400W, a huge systolic array (how do you deal with capacitance), and some experimental low voltage cell IP from a failed Bitcoin miner company. You want me to believe they taped a chip out that's five years ahead of NVIDIA in 2 years? Extraordinary claims require extraordinary evidence and all that. If you recontextualize the demo you saw with this skepticism, does it still hold up?

@WesleyYue · 2026-07-01 17:34
this smells very bad. none of the claims make sense. > low voltage hits 80% mfu high mfu is easy if your peak flops is low > low voltage solves power bottleneck bleeding edge wafers is more scarce than power and it doesn't make sense to tape out lower perf chips on these wafers to save power if you want perf/$. the true bottleneck to frontier perf is max flops density to minimize going off die. power is the most fungible elastic commodity on earth > half voltage = quarter power presumably: 1. only matmul gates (~60% of die power) at half voltage. half voltage -> ~3x slower -> need ~3x more transistors, and their power leaks scale linearly. ~20% net savings at chip level 2. half voltage -> engineering complexity to fix timing violations + exponentially scaled soft errors (~20x). matmuls will randomly corrupt undetectably from bit flips (and drop model intelligence). > btc miners run at 3x lower V btc workload is hashing, so 1) error checking is literally in the problem 2) arithmetic intensity is like infinity so they don't care about packing flops density in single die unlike AI workloads. doesn't make sense to copy > GPUs get low MFU from thermals no, they get low MFU from mem bw and non-matmul ops. > CSM 5x lower latency than Blackwell 4000ns Blackwell nvswitch is 300ns? unless they are comparing their bare hardware to nvidia hardware + software. the whole product design tradeoffs don't make any sense from first principles and only makes sense if their initial "transformer asic" had some horrible power issue that they solved by undervolting and they had to respin the whole story for investor marketing. would love to be proven wrong if anyone from Etched wants to educate me :)

@__tinygrad__ · 2026-07-01 18:17
@WesleyYue Spot on with the wafers are more scarce than power

@joefioti · 2026-07-01 16:08
@__tinygrad__ @Etched Their marketing about “burning the transformer in” has always been nonsense but nothing in this launch seems explicitly wrong. Obviously async is hard, and unclear if they actually did that, but stuff like unified node memory and undervolting makes sense (not an investor lol)

@__tinygrad__ · 2026-07-01 18:24
@joefioti @Etched This is a better tech analysis than what I wrote. https://t.co/jOSQ8UIbaM

@SIGKITTEN · 2026-06-30 03:12
@elonmusk @aaronburnett u know there's already @__tinygrad__ right

@__tinygrad__ · 2026-07-01 22:58
@SIGKITTEN @elonmusk @aaronburnett Haha he follows us. I think he is targetting results on a shorter timeframe than us, makes sense to target many different time horizons.

@sudoingX · 2026-07-02 11:10
the day a chinese card gives me 48gb for what 12gb costs today, i'm not asking where it shipped from. cheap compute is the only ideology i have.

@__tinygrad__ · 2026-07-02 19:14
why hello sir, can I interest you in an exabox? https://t.co/DLKGRbobkf

@__tinygrad__ · 2026-07-02 19:14
why hello sir, can I interest you in an exabox? https://t.co/DLKGRbobkf

@_c3nd · 2026-07-02 19:17
@__tinygrad__ Can it be bigger?

@__tinygrad__ · 2026-07-02 19:14
why hello sir, can I interest you in an exabox? https://t.co/DLKGRbobkf

@emm0sh · 2026-07-02 19:21
@__tinygrad__ you’re going to have to hire actual engineers for this thing. reality via things like galvanic corrosion are going to be a challenge

@_c3nd · 2026-07-02 19:17
@__tinygrad__ Can it be bigger?

@__tinygrad__ · 2026-07-02 19:21
@_c3nd no we are the tiny corp we keep it tiny

@__tinygrad__ · 2026-07-02 19:14
why hello sir, can I interest you in an exabox? https://t.co/DLKGRbobkf

@beaverd · 2026-07-02 19:49
@__tinygrad__ yes

@__tinygrad__ · 2026-07-02 19:14
why hello sir, can I interest you in an exabox? https://t.co/DLKGRbobkf

@HotAisle · 2026-07-02 20:12
@__tinygrad__ what are all the doors for?

@__tinygrad__ · 2026-07-02 19:14
why hello sir, can I interest you in an exabox? https://t.co/DLKGRbobkf

@nickzilllla · 2026-07-02 21:15
@__tinygrad__ What are yall planning to use for cooling? Fully loaded what’s the BTU output? Very interested in about 5 of these for my company. South Texas so it’s HOT!

@nickzilllla · 2026-07-02 21:15
@__tinygrad__ What are yall planning to use for cooling? Fully loaded what’s the BTU output? Very interested in about 5 of these for my company. South Texas so it’s HOT!

@__tinygrad__ · 2026-07-02 21:42
@nickzilllla So we're going to have the chiller be external and have cold water in and hot water out to the box. Makes more sense to support a larger variety of environments and cooling types. It's ~600 kW, so like 2M BTUs.

@HotAisle · 2026-07-02 20:12
@__tinygrad__ what are all the doors for?

@__tinygrad__ · 2026-07-02 21:43
@HotAisle Easy serviceability.

@emm0sh · 2026-07-02 19:21
@__tinygrad__ you’re going to have to hire actual engineers for this thing. reality via things like galvanic corrosion are going to be a challenge

@__tinygrad__ · 2026-07-02 21:45
@emm0sh I mean, we have a great engineering team with our partner @comma_ai, but we're also hiring in person in San Diego if someone wants to work on the exabox project.

@__tinygrad__ · 2026-07-02 19:14
why hello sir, can I interest you in an exabox? https://t.co/DLKGRbobkf

@0xSero · 2026-07-02 21:45
@__tinygrad__ can I have one? Pls I will make 4 posts about tinygrad

@beaverd · 2026-07-02 19:49
@__tinygrad__ yes

@__tinygrad__ · 2026-07-02 21:46
@beaverd Preorder here https://t.co/01nzuEavjK

@0xSero · 2026-07-02 21:45
@__tinygrad__ can I have one? Pls I will make 4 posts about tinygrad

@__tinygrad__ · 2026-07-02 21:46
@0xSero That will earn you 42 seconds of time on the exabox, make it count!

@__tinygrad__ · 2026-07-02 21:43
@HotAisle Easy serviceability.

@HotAisle · 2026-07-02 22:17
@__tinygrad__ Problem is that when you open the doors, it blows up your airflow, and potentially filtration. Hope you also get a cover over the whole container too…

@HotAisle · 2026-07-02 22:17
@__tinygrad__ Problem is that when you open the doors, it blows up your airflow, and potentially filtration. Hope you also get a cover over the whole container too…

@__tinygrad__ · 2026-07-02 22:50
@HotAisle We're using an external chiller and will have the front area filtered off. This is also our prototype container so things may change, but yea, certainly something to keep in mind. In general though, I think computers are kept too clean and too cold.

@sudoingX · 2026-07-02 11:10
the day a chinese card gives me 48gb for what 12gb costs today, i'm not asking where it shipped from. cheap compute is the only ideology i have.

@__tinygrad__ · 2026-07-02 22:52
@sudoingX this is a solid ideology

@joefioti · 2026-07-03 04:03
I really want to like TT (cool hardware!) but every time they make announcements like this I go look and its always hand-optimized special-cased nonsense that runs *just* this model at like batch_size=1. No one buys hardware to run a single model, you guys need a proper compiler!

@JonhernandezIA · 2026-07-03 10:39
Anthropic is starting to look less like an AI lab. And more like a pharma company. Claude Science now has wet labs, biology-trained models, and Coefficient Bio. The irony: pharma companies paying for Claude may be funding a future competitor. AI vendors are no longer just selling tools. They are learning your industry from the inside.

@JonhernandezIA · 2026-07-03 10:39
Anthropic is starting to look less like an AI lab. And more like a pharma company. Claude Science now has wet labs, biology-trained models, and Coefficient Bio. The irony: pharma companies paying for Claude may be funding a future competitor. AI vendors are no longer just selling tools. They are learning your industry from the inside.

@__tinygrad__ · 2026-07-03 14:54
@JonhernandezIA and now you see why it wouldn't tell you want a mitochondria was, it was worried you might compete #aisafety

@__tinygrad__ · 2026-07-03 17:53
We are here when you are ready @tenstorrent. We love how you build in public and have buy it now buttons on your website (unlike certain other fake chip companies). Let's get you a software stack to match!

@__tinygrad__ · 2026-07-03 17:53
We are here when you are ready @tenstorrent. We love how you build in public and have buy it now buttons on your website (unlike certain other fake chip companies). Let's get you a software stack to match!

@joefioti · 2026-07-03 17:54
@__tinygrad__ @tenstorrent We’ve talked with them as well, unfortunately doesn’t seem like it’s time. Sad 😔

@__tinygrad__ · 2026-07-03 17:53
We are here when you are ready @tenstorrent. We love how you build in public and have buy it now buttons on your website (unlike certain other fake chip companies). Let's get you a software stack to match!

@jakubiee · 2026-07-03 17:55
@__tinygrad__ @tenstorrent Tenstorrent x tinygrad it's going to be amazing

@joefioti · 2026-07-03 17:54
@__tinygrad__ @tenstorrent We’ve talked with them as well, unfortunately doesn’t seem like it’s time. Sad 😔

@__tinygrad__ · 2026-07-03 17:56
@joefioti @tenstorrent Yea, it doesn't make sense for how much they spend on tapeouts to not also spend on a software stack. It's sad if their real goal is just to get acquired for IP, why not try to compete with the biggest company in the world? My advice from a year ago. https://t.co/CHnzYxTegt

@jakubiee · 2026-07-03 17:55
@__tinygrad__ @tenstorrent Tenstorrent x tinygrad it's going to be amazing

@__tinygrad__ · 2026-07-03 17:58
@jakubiee @tenstorrent I mean, it's likely not happening. It's a bit beyond what casual contributors could do. But imagine 10k lines of pure Python that's so clean it works everywhere, including over USB on macOS, instead of C++24 that works on one Ubuntu.

@__tinygrad__ · 2026-07-03 17:56
@joefioti @tenstorrent Yea, it doesn't make sense for how much they spend on tapeouts to not also spend on a software stack. It's sad if their real goal is just to get acquired for IP, why not try to compete with the biggest company in the world? My advice from a year ago. https://t.co/CHnzYxTegt

@haydonryan · 2026-07-03 19:00
@__tinygrad__ @joefioti @tenstorrent This is the mistake AMD made and continues to make. Nvidia became the defacto because they invested heavily in CUDA & software ecosystem. Just making something open source (especially when it's hardware) doesn't get you free developers.

@ivanvnucec · 2026-07-03 19:18
@__tinygrad__ @tenstorrent Why don't you just do it?

@__tinygrad__ · 2026-07-03 20:13
@ivanvnucec @tenstorrent Honestly, because we'd rather put the time into @AMD. Aside from our MLPerf contract and free hardware from them, their chips are more widespread and easier to program for. TT has some cool ideas, but it's unclear if they will become the mainstream. AMD is more similar to NVIDIA.

@__tinygrad__ · 2026-07-03 20:13
@ivanvnucec @tenstorrent Honestly, because we'd rather put the time into @AMD. Aside from our MLPerf contract and free hardware from them, their chips are more widespread and easier to program for. TT has some cool ideas, but it's unclear if they will become the mainstream. AMD is more similar to NVIDIA.

@__tinygrad__ · 2026-07-03 20:16
@ivanvnucec @tenstorrent @AMD We would do it if the right contract came along, it's a cool problem, and there's value in making sure we support architectures that look like theirs. But without a contract, opportunity cost is too high. It would be a real challenge and would be work specific to them.

@haydonryan · 2026-07-03 19:00
@__tinygrad__ @joefioti @tenstorrent This is the mistake AMD made and continues to make. Nvidia became the defacto because they invested heavily in CUDA & software ecosystem. Just making something open source (especially when it's hardware) doesn't get you free developers.

@__tinygrad__ · 2026-07-03 20:21
@haydonryan @joefioti @tenstorrent AMD is doing far better now. They are finally putting real resources into software and the stack is leaps and bounds better than it was 3 years ago (try it!). There is also a lot more overlap with NVIDIA. On current course they stand to grow market share I think.

@__tinygrad__ · 2026-07-05 22:11
RT @paulpgustafson: Tinygrad finally got around to writing up a clean spec, looks a lot more promising -- https://t.co/an0fnZCA49

@__tinygrad__ · 2026-07-06 19:00
I cannot believe how good GLM 5.2 is. Several weeks in now and it's mostly all I have been using. It doesn't have the alignment issue of cloud AI, it's much more clear what it can do and can't because it doesn't care about you taking another $$$ spin at the token slot machine.

@__tinygrad__ · 2026-07-06 19:00
I cannot believe how good GLM 5.2 is. Several weeks in now and it's mostly all I have been using. It doesn't have the alignment issue of cloud AI, it's much more clear what it can do and can't because it doesn't care about you taking another $$$ spin at the token slot machine.

@rskvll · 2026-07-06 19:02
@__tinygrad__ truee and the price is very reasonable

@rskvll · 2026-07-06 19:02
@__tinygrad__ truee and the price is very reasonable

@__tinygrad__ · 2026-07-06 19:05
@rskllx We've been running locally, ~100 tok/s and one machine is enough for a 40 person org. We don't even have DFlash up yet, the speed should double when we do.

@ID_AA_Carmack · 2026-07-06 21:46
Memory cost and capacity are significant issues for AI accelerators. Unlike game rendering, model inference can have a deterministic memory access pattern. You don’t need “random access memory” at all for model weights, and you could tolerate cold-start latencies in the multiple milliseconds, as long as continuous reads were delivered at the necessary bandwidth. NAND flash is over 100 times cheaper per GB than HBM, so there should be opportunity there, even after giving a flash controller a 1024 bit interface with HBM bandwidth. You could make a specialized pin protocol that just supported pipelined transfer of full 16KB+ pages from the flash to program-managed accelerator scratchpad memory and improve per-pin performance over HBM, but it might be more convenient to make it still look like a true random access memory with very fragile performance characteristics, where anything but sequential reads falls off a 1000x+ performance cliff. That has the advantage of automatically using existing cache hierarchies, and providing a natural path to update the flash memory with new model weights. With the stream-to-scratch interface, code has to be completely rewritten before it works at all, while the ram-emulation interface will start off just extremely slow, and you can incrementally sort out the changes for full performance. There may be cases where there isn’t enough scratchpad SRAM to hold the weights for a layer, which might force you to deploy the old optical drive optimization technique of duplicating data in multiple places on a sequential read to avoid seeking, but there would be capacity to burn. It might be possible to do something like cuda graph capture to record a memory access trace and have everything magically remapped to a linear sequence, but deploying programmer / agent elbow grease to manage transfers and access in a scratch ram ring buffer would be lower risk. A split memory system consisting of some channels of flash and some channels of HBM will probably be suboptimal compared to a uniform memory, but it could be much cheaper, and allow much larger models to be run. I think th case is strong for inference, but you have to stretch more for training. You can still linearize all the weight memory accesses, both reads and writes, but flash memory would quickly wear out from the writes, even if they were all perfectly page aligned. Replacing low-latency HBM with massively parallel cheap(er) DRAM at high latency might still be a worthwhile cost savings.

@ID_AA_Carmack · 2026-07-06 21:46
Memory cost and capacity are significant issues for AI accelerators. Unlike game rendering, model inference can have a deterministic memory access pattern. You don’t need “random access memory” at all for model weights, and you could tolerate cold-start latencies in the multiple milliseconds, as long as continuous reads were delivered at the necessary bandwidth. NAND flash is over 100 times cheaper per GB than HBM, so there should be opportunity there, even after giving a flash controller a 1024 bit interface with HBM bandwidth. You could make a specialized pin protocol that just supported pipelined transfer of full 16KB+ pages from the flash to program-managed accelerator scratchpad memory and improve per-pin performance over HBM, but it might be more convenient to make it still look like a true random access memory with very fragile performance characteristics, where anything but sequential reads falls off a 1000x+ performance cliff. That has the advantage of automatically using existing cache hierarchies, and providing a natural path to update the flash memory with new model weights. With the stream-to-scratch interface, code has to be completely rewritten before it works at all, while the ram-emulation interface will start off just extremely slow, and you can incrementally sort out the changes for full performance. There may be cases where there isn’t enough scratchpad SRAM to hold the weights for a layer, which might force you to deploy the old optical drive optimization technique of duplicating data in multiple places on a sequential read to avoid seeking, but there would be capacity to burn. It might be possible to do something like cuda graph capture to record a memory access trace and have everything magically remapped to a linear sequence, but deploying programmer / agent elbow grease to manage transfers and access in a scratch ram ring buffer would be lower risk. A split memory system consisting of some channels of flash and some channels of HBM will probably be suboptimal compared to a uniform memory, but it could be much cheaper, and allow much larger models to be run. I think th case is strong for inference, but you have to stretch more for training. You can still linearize all the weight memory accesses, both reads and writes, but flash memory would quickly wear out from the writes, even if they were all perfectly page aligned. Replacing low-latency HBM with massively parallel cheap(er) DRAM at high latency might still be a worthwhile cost savings.

@__tinygrad__ · 2026-07-06 23:23
@ID_AA_Carmack tinygrad makes it very clear when and where memory is accessed. No CUDA graph style capture requires, it's just there.

@__tinygrad__ · 2026-07-07 01:41
You start out adding one bit of nonsense. It's just a little nonsense you say. But then you have 3 workarounds elsewhere in the codebase for that one bit of nonsense. Then you add 17 little hacks to deal with those 3 workarounds. And soon, your whole repo is hacks and nonsense. https://t.co/OBvLZ2R8QK

@__tinygrad__ · 2026-07-07 01:41
You start out adding one bit of nonsense. It's just a little nonsense you say. But then you have 3 workarounds elsewhere in the codebase for that one bit of nonsense. Then you add 17 little hacks to deal with those 3 workarounds. And soon, your whole repo is hacks and nonsense. https://t.co/OBvLZ2R8QK

@playerTwoQ · 2026-07-07 01:48
@__tinygrad__ It's fun turning a pile of nonsense into a satisfying refactor though. It's a bit annoying when you inevitably ask yourself why you didn't do it the final way the first time but... that's engineering :)

@playerTwoQ · 2026-07-07 01:48
@__tinygrad__ It's fun turning a pile of nonsense into a satisfying refactor though. It's a bit annoying when you inevitably ask yourself why you didn't do it the final way the first time but... that's engineering :)

@__tinygrad__ · 2026-07-07 01:52
@playerTwoQ Like I think it's so so much more ridiculous than people realize. tinygrad is 24263 lines to replace onnx/torch/mlir/llvm/cuda/driver and it's at least still half nonsense and hacks.

@jukan05 · 2026-07-07 10:42
CHINA CONSIDERS RESTRICTING OVERSEAS ACCESS TO CUTTING-EDGE AI MODELS China’s Ministry of Commerce has led meetings over the past month with major AI companies, including Alibaba, ByteDance, and https://t.co/YDe0KRldDB, to discuss measures that would restrict overseas access to cutting-edge AI models, including models that have not yet been released. The discussions reportedly include not only closed-source models but also open-weight models. However, the scope of application is still under debate, and the rules may ultimately apply only to future frontier models. Officials have also discussed designating the leakage or theft of proprietary AI technologies as a national security crime, with stronger penalties, as well as restricting the types of foreign capital that can invest in Chinese AI startups. The backdrop is the U.S. move to strengthen export controls on AI models, along with national security concerns over cutting-edge models that could possess advanced cyberattack capabilities. Chinese authorities are reportedly concerned that advanced U.S. cybersecurity AI models could be used to exploit vulnerabilities in Chinese software. Since the beginning of this year, China has continued to tighten measures to prevent AI technology from being transferred overseas. Authorities have investigated whether Chinese AI startups that relocated abroad violated export control laws, while also strengthening oversight of overseas transactions involving Chinese investors, technology, data, and national security concerns. Future regulations could take the form of a tiered framework based on technological capability. Basic open-source AI models may be managed through a filing system, high-performance models may be subject to security reviews, and the most sensitive frontier models may be banned from public release or restricted to use within China.

@PolymarketMoney · 2026-07-07 12:24
JUST IN: China moves to restrict overseas access to its cutting-edge AI models.

@mark_k · 2026-07-07 12:41
It's happening: Open source AI models from China may become restricted soon! That could be the end of an era when the whole world enjoyed free models.

@okaythenfuture · 2026-07-07 12:56
Everyone reflexively going off the cuff about “dumb Chinese regulators” instead of contemplating if this might be a mutual backchannel agreement between the two superpowers. Ten years from now you will still be going to work everyday. No AGI for you. And definitely no ASI.

@emollick · 2026-07-07 14:14
This is a key reason I don’t expect the flow of frontier open weights models to continue indefinitely, or even for very much longer. https://t.co/Q8RKnBJaR4

@bdsqlsz · 2026-07-07 14:31
I had to track down the original source cited by Reuters. Reuters exaggerated its conclusions and misrepresented China’s consideration of limiting the weights of SOTA models. Tiered plan applies to all open-source technologies, not model weights. links↓ https://t.co/YOmRCZdORT

@jukan05 · 2026-07-07 10:42
CHINA CONSIDERS RESTRICTING OVERSEAS ACCESS TO CUTTING-EDGE AI MODELS China’s Ministry of Commerce has led meetings over the past month with major AI companies, including Alibaba, ByteDance, and https://t.co/YDe0KRldDB, to discuss measures that would restrict overseas access to cutting-edge AI models, including models that have not yet been released. The discussions reportedly include not only closed-source models but also open-weight models. However, the scope of application is still under debate, and the rules may ultimately apply only to future frontier models. Officials have also discussed designating the leakage or theft of proprietary AI technologies as a national security crime, with stronger penalties, as well as restricting the types of foreign capital that can invest in Chinese AI startups. The backdrop is the U.S. move to strengthen export controls on AI models, along with national security concerns over cutting-edge models that could possess advanced cyberattack capabilities. Chinese authorities are reportedly concerned that advanced U.S. cybersecurity AI models could be used to exploit vulnerabilities in Chinese software. Since the beginning of this year, China has continued to tighten measures to prevent AI technology from being transferred overseas. Authorities have investigated whether Chinese AI startups that relocated abroad violated export control laws, while also strengthening oversight of overseas transactions involving Chinese investors, technology, data, and national security concerns. Future regulations could take the form of a tiered framework based on technological capability. Basic open-source AI models may be managed through a filing system, high-performance models may be subject to security reviews, and the most sensitive frontier models may be banned from public release or restricted to use within China.

@__tinygrad__ · 2026-07-07 15:22
@jukan05 Le Chaton Fat is our only hope!

@__tinygrad__ · 2026-07-07 15:24
In a few years, between hardware and algorithmic improvements, this will all look as laughable as governments restricting encryption to 40-bits.

@__tinygrad__ · 2026-07-07 15:24
In a few years, between hardware and algorithmic improvements, this will all look as laughable as governments restricting encryption to 40-bits.

@adpenatx · 2026-07-07 15:30
@__tinygrad__ commoditize the petaflop 💅

@adpenatx · 2026-07-07 15:30
@__tinygrad__ commoditize the petaflop 💅

@__tinygrad__ · 2026-07-07 15:34
@adpenatx Information wants to be free

@__tinygrad__ · 2026-07-07 15:24
In a few years, between hardware and algorithmic improvements, this will all look as laughable as governments restricting encryption to 40-bits.

@StaticSnowman · 2026-07-07 15:38
@__tinygrad__ if the software gets better by 10x the restrictions on hardware make no difference. DeepSeek does this with every major release

@StaticSnowman · 2026-07-07 15:38
@__tinygrad__ if the software gets better by 10x the restrictions on hardware make no difference. DeepSeek does this with every major release

@__tinygrad__ · 2026-07-07 15:42
@StaticSnowman Governments desperately want to be relevant, but that's not new. What's sad is the people in the American "frontier" labs who must actively root against these leaps. Deepseek drops strike fear into the hearts of capexmaxxers like what am I paying these researchers so much for.

@__tinygrad__ · 2026-07-07 15:24
In a few years, between hardware and algorithmic improvements, this will all look as laughable as governments restricting encryption to 40-bits.

@tiagoasousa_ · 2026-07-07 16:25
@__tinygrad__ how will training work? inference seems to just be a question of time/tweaks, but training does seem a unsolved problem to do at scale outside bigCo.

@__tinygrad__ · 2026-07-07 15:24
In a few years, between hardware and algorithmic improvements, this will all look as laughable as governments restricting encryption to 40-bits.

@toi500 · 2026-07-07 17:34
@__tinygrad__ That was Fake News from Reuters

@claudeai · 2026-07-07 17:36
We're extending access to Claude Fable 5 on all paid plans through July 12.

@toi500 · 2026-07-07 17:34
@__tinygrad__ That was Fake News from Reuters

@__tinygrad__ · 2026-07-07 17:48
@toi500 oh lol I should have checked source. Reuters is the most blatant US government info laundering operation. I didn't think China was this dumb, but thanks for confirming.

@tiagoasousa_ · 2026-07-07 16:25
@__tinygrad__ how will training work? inference seems to just be a question of time/tweaks, but training does seem a unsolved problem to do at scale outside bigCo.

@__tinygrad__ · 2026-07-07 17:59
@tiagoasousa_ The lines between inference and training will blur. Base foundation models will be trained by chip companies to sell more chips.

@claudeai · 2026-07-07 17:36
We're extending access to Claude Fable 5 on all paid plans through July 12.

@__tinygrad__ · 2026-07-07 18:06
@claudeai lol or you can just download GLM 5.2 and not have to deal with your nonsense. either sell a product on reasonable and certain terms or don't waste anyone's time. anyone who relies on this company is a fool

@PolymarketMoney · 2026-07-07 12:24
JUST IN: China moves to restrict overseas access to its cutting-edge AI models.

@__tinygrad__ · 2026-07-07 18:13
@PolymarketMoney Fake news promoted by Reuters, the US government's lapdog news source. https://t.co/Boyfgkz2bx

@emollick · 2026-07-07 14:14
This is a key reason I don’t expect the flow of frontier open weights models to continue indefinitely, or even for very much longer. https://t.co/Q8RKnBJaR4

@__tinygrad__ · 2026-07-07 18:16
@emollick Fake news promoted by Reuters, the US government's lapdog news source. https://t.co/Boyfgkz2bx

@mark_k · 2026-07-07 12:41
It's happening: Open source AI models from China may become restricted soon! That could be the end of an era when the whole world enjoyed free models.

@__tinygrad__ · 2026-07-07 18:16
@mark_k Fake news promoted by Reuters, the US government's lapdog news source. https://t.co/Boyfgkz2bx

@okaythenfuture · 2026-07-07 12:56
Everyone reflexively going off the cuff about “dumb Chinese regulators” instead of contemplating if this might be a mutual backchannel agreement between the two superpowers. Ten years from now you will still be going to work everyday. No AGI for you. And definitely no ASI.

@__tinygrad__ · 2026-07-07 18:17
@okaythenfuture Fake news promoted by Reuters, the US government's lapdog news source. This is the desired US narrative, don't fall for it. https://t.co/Boyfgkz2bx

@__tinygrad__ · 2026-07-07 18:23
We'll all look back in five years and remember when the enshittification cycle got too fast. Any company that relies on Anthropic is so obviously foolish. Google and Microsoft knew how to slow play it.

@__tinygrad__ · 2026-07-07 18:23
We'll all look back in five years and remember when the enshittification cycle got too fast. Any company that relies on Anthropic is so obviously foolish. Google and Microsoft knew how to slow play it.

@ashen_one · 2026-07-07 18:28
@__tinygrad__ wait why are you guys upset by this (asking not questioning)

@ashen_one · 2026-07-07 18:28
@__tinygrad__ wait why are you guys upset by this (asking not questioning)

@__tinygrad__ · 2026-07-07 18:32
@ashen_one Not upset, it's just super clownish and Anthropic will be studied in business school as what happens when you alienate all your potential customers. Have you heard a revenue number from them lately? As revenue goes down, the screaming about the machine god safety will go up.

@__tinygrad__ · 2026-07-07 18:06
@claudeai lol or you can just download GLM 5.2 and not have to deal with your nonsense. either sell a product on reasonable and certain terms or don't waste anyone's time. anyone who relies on this company is a fool

@quantumvoidlabs · 2026-07-07 18:33
It is actually ridiculous lol they're literally putting people through some weird AI psychosis with their very abnormal marketing strategy. At this point it actually appears as if they want people to just cancel their plans w/ Anthropic they're doing this weird shake-out style marketing if I am not mistaken here.

@__tinygrad__ · 2026-07-07 18:35
RT @___Harald___: Anthropic is now a banned vendor at @comma_ai . I recommend other companies do the same. People should consider how reliant they are on cloud token vendors. If you can't brainstorm, code or be productive without them, you should be scared. Regain your sovereignty or you're ngmi. https://t.co/KjO017gbNl

@__tinygrad__ · 2026-07-07 18:43
@quantumvoidlabs @claudeai I think they are deep in throws of AI psychosis themselves. They just do whatever the model with the safety restrictions removed tells them. All hail the new machine God! The machine God will save us! Trust the machine God!

@PalantirTech · 2026-07-07 19:08
Our expanded thoughts on an institution’s freedom to pursue new opportunities, its economic rights, and its ability to expand them in the age of AI. Please find attached our white paper, Institutional Sovereignty in the Age of AI, outlining the 15 steps governments and companies must take to protect both their sovereignty and their alpha. https://t.co/rZuNeOKXU9

@evanreiser · 2026-07-07 19:33
.@AnthropicAI has filed a lawsuit against @Abnormal claiming we copied their brand to mislead security customers, and they are asking for “all revenues, earnings and profits” This is obviously not true. They don’t own every A/slash design in AI cybersecurity, and they don’t get to turn a logo dispute into a claim on Abnormal’s entire business. It would be easier to concede quietly. But Abnormal was built on trust, innovation, and intellectual honesty. Values are what you do when tested (especially when inconvenient) Our behavioral security platform and our AI models are ours. Everything meaningful about Abnormal was built the hard way: the technology, the customer relationships, and the trust. We earned that trust by protecting customers, and we will defend it. My blog post about it (link below):

@theo · 2026-07-07 21:05
GLM 5.2 is an incredible model. I wish people would stop pretending it is self-hostable and that it compares to Fable.

@PalantirTech · 2026-07-07 19:08
Our expanded thoughts on an institution’s freedom to pursue new opportunities, its economic rights, and its ability to expand them in the age of AI. Please find attached our white paper, Institutional Sovereignty in the Age of AI, outlining the 15 steps governments and companies must take to protect both their sovereignty and their alpha. https://t.co/rZuNeOKXU9

@__tinygrad__ · 2026-07-07 21:54
@PalantirTech This is what AI alignment actually is.

@ClementDelangue · 2026-07-07 22:17
https://t.co/uQSOqPWPD0 https://t.co/6nP7I7ohIU

@__tinygrad__ · 2026-07-08 00:08
@theo We are self hosting it. And for 90% of tasks it's the same as fable. The other 10% it's better because it doesn't refuse to do them 😂

@pseudo_tbd · 2026-07-08 00:35
@__tinygrad__ @theo yea im sorry, im staying on Claude Fable for now https://t.co/Oems426PQt

@theo · 2026-07-08 00:43
@__tinygrad__ I love you guys and have advocated for you harder than you could possibly know. Pretending GLM-5.2 is "the same as Fable" is going to hurt how people see your brand. Tinygrad should be a leader in AI. Posting blatantly false things like this only serves to hurt your goals.

@__tinygrad__ · 2026-07-08 01:22
@theo All jokes aside, for 90% of tasks it really is the same as fable. There's some tasks fable can do that GLM can't, but if you are operating in a regime where you only trust AI with things that you are totally sure it can do, it doesn't matter that much. Proof is in my token usage.

@__tinygrad__ · 2026-07-08 01:22
@theo All jokes aside, for 90% of tasks it really is the same as fable. There's some tasks fable can do that GLM can't, but if you are operating in a regime where you only trust AI with things that you are totally sure it can do, it doesn't matter that much. Proof is in my token usage.

@__tinygrad__ · 2026-07-08 01:24
@theo If you are operating AI outside a regime where you are not capable of checking quickly if it did it correctly or not and understanding all the code, I argue that you have bigger problems, regardless of what model you are using.

@pseudo_tbd · 2026-07-08 00:35
@__tinygrad__ @theo yea im sorry, im staying on Claude Fable for now https://t.co/Oems426PQt

@__tinygrad__ · 2026-07-08 01:29
@pseudo_tbd @theo omg if you could really run it on a $12k machine they would be flying off the shelves. you need the $150k one. https://t.co/0uEztJ6RU3

@__tinygrad__ · 2026-07-08 01:24
@theo If you are operating AI outside a regime where you are not capable of checking quickly if it did it correctly or not and understanding all the code, I argue that you have bigger problems, regardless of what model you are using.

@jpohhhh · 2026-07-08 01:30
@__tinygrad__ @theo "90% of tasks" to "90% of tasks in regime X" X was described as in opening: "regime...only trust AI with things that you are totally sure it can do" latest: "regime...where you are not capable of checking quickly if it did it correctly or not and understanding all the code"

@ClementDelangue · 2026-07-07 22:17
https://t.co/uQSOqPWPD0 https://t.co/6nP7I7ohIU

@__tinygrad__ · 2026-07-08 01:31
@ClementDelangue If xAI drops something beyond GLM 5.2 open source they just became the new number 1 lab. @elonmusk

@jpohhhh · 2026-07-08 01:30
@__tinygrad__ @theo "90% of tasks" to "90% of tasks in regime X" X was described as in opening: "regime...only trust AI with things that you are totally sure it can do" latest: "regime...where you are not capable of checking quickly if it did it correctly or not and understanding all the code"

@__tinygrad__ · 2026-07-08 01:32
@jpohhhh @theo Umm, if you can't check it and understand the code, how sure are you that it did it? AI can be very very sneaky in how it comments out tests and stuff.

@evanreiser · 2026-07-07 19:33
.@AnthropicAI has filed a lawsuit against @Abnormal claiming we copied their brand to mislead security customers, and they are asking for “all revenues, earnings and profits” This is obviously not true. They don’t own every A/slash design in AI cybersecurity, and they don’t get to turn a logo dispute into a claim on Abnormal’s entire business. It would be easier to concede quietly. But Abnormal was built on trust, innovation, and intellectual honesty. Values are what you do when tested (especially when inconvenient) Our behavioral security platform and our AI models are ours. Everything meaningful about Abnormal was built the hard way: the technology, the customer relationships, and the trust. We earned that trust by protecting customers, and we will defend it. My blog post about it (link below):

@__tinygrad__ · 2026-07-08 01:40
@evanreiser @AnthropicAI @Abnormal https://t.co/HbgxgqWU5V

@sama · 2026-07-08 04:15
GPT-5.6 sol launches thursday! happy building

@ns123abc · 2026-07-08 07:15
BRO LITERALLY PREDICTED THIS https://t.co/mitYQgY6EF

@sama · 2026-07-08 04:15
GPT-5.6 sol launches thursday! happy building

@__tinygrad__ · 2026-07-08 14:55
@sama as long as you keep the existing limits and filters on the $200 plan, but bump the version from 5.5 to 5.6, I'm looking forward to it. I know it's not a lot to ask, but expectations are low these days

@esrtweet · 2026-07-08 16:13
This is a certain kind of talk around LLMs that I find increasingly puzzling. That is all of the people bitching that LLMs constantly generate crap code and hallucinate solutions, and are worthless for programming. This has almost never happened to me, and never during the last two model generations I have used (chat GPT 5.4 and 5.5). Occasionally a model used to get a little deranged when I pushed its context limit, but under codex that doesn't happen anymore; instead I got a red-highlighted warning when the limit has been exceeded and I need to clear my session. I've applied AI to feature changes, refactoring, and debugging over 63 different projects written in C, Go, Rust, Python, and shell. I've written documentation with it. I've decompiled a DOS binary into readable source code. It's now routine that whenever I have to touch one of my projects I start by running the regression tests, then fire up codex and asking it to audit the code for bugs and suggest improvements. My experience is that LLMs are excellent and tremendously empowering tools. Their worst limitation is a kind of architectural tunnel vision - they're extremely good at generating code to specification but sometimes blind to higher-level patterns. Which is okay, it's my meatbrain job to be good at that. The most valuable thing I find about LLMs is exactly that they *don't* screw up details and edge cases. I'm a very, very good coder by human standards (I'd better be, with 50 years of experience!) but the LLMs are better than me. Because if a code change needs to touch (say) five places in the code, they reliably find all five rather than doing the human thing of fixing four and then having to debug for hours before you figure out that there's a fifth one you missed. Are the downshouters living in a different universe than me? Are they using old, weak models? Or do they have some kind of skill issue that I can't see because I have mental habits and communication skills that are a good fit for the handles on these tools? I don't know. And I think this is an important thing to figure out, because I'm seeing lots of stories in the news that suggest billions of dollars are being wasted on misdirected token spend. It all seems very simple to me. Be clear in your thinking, tell the model what you want with precision, and good things happen. What...what am I missing here?

@comma_ai · 2026-07-08 16:19
Designed in America, Built in America. comma four is $100 off for Independence Day 🇺🇸 https://t.co/KT1v5An9UN

@__tinygrad__ · 2026-07-08 16:41
We need to make a video like this of the tinybox line.

@esrtweet · 2026-07-08 16:13
This is a certain kind of talk around LLMs that I find increasingly puzzling. That is all of the people bitching that LLMs constantly generate crap code and hallucinate solutions, and are worthless for programming. This has almost never happened to me, and never during the last two model generations I have used (chat GPT 5.4 and 5.5). Occasionally a model used to get a little deranged when I pushed its context limit, but under codex that doesn't happen anymore; instead I got a red-highlighted warning when the limit has been exceeded and I need to clear my session. I've applied AI to feature changes, refactoring, and debugging over 63 different projects written in C, Go, Rust, Python, and shell. I've written documentation with it. I've decompiled a DOS binary into readable source code. It's now routine that whenever I have to touch one of my projects I start by running the regression tests, then fire up codex and asking it to audit the code for bugs and suggest improvements. My experience is that LLMs are excellent and tremendously empowering tools. Their worst limitation is a kind of architectural tunnel vision - they're extremely good at generating code to specification but sometimes blind to higher-level patterns. Which is okay, it's my meatbrain job to be good at that. The most valuable thing I find about LLMs is exactly that they *don't* screw up details and edge cases. I'm a very, very good coder by human standards (I'd better be, with 50 years of experience!) but the LLMs are better than me. Because if a code change needs to touch (say) five places in the code, they reliably find all five rather than doing the human thing of fixing four and then having to debug for hours before you figure out that there's a fifth one you missed. Are the downshouters living in a different universe than me? Are they using old, weak models? Or do they have some kind of skill issue that I can't see because I have mental habits and communication skills that are a good fit for the handles on these tools? I don't know. And I think this is an important thing to figure out, because I'm seeing lots of stories in the news that suggest billions of dollars are being wasted on misdirected token spend. It all seems very simple to me. Be clear in your thinking, tell the model what you want with precision, and good things happen. What...what am I missing here?

@__tinygrad__ · 2026-07-08 16:46
@esrtweet I think it depends how you use the models. The new ones are good if you have exactly in mind what you want. Agreed they are superhuman at edge case correctness. However, if you are the "don't look at the code" type LLM user, you quickly end up with 1000s of lines of inhuman crap.

@elonmusk · 2026-07-08 17:38
Our internal assessment is that Grok 4.5 is roughly comparable to Opus 4.7, but much faster. The combination of capability, faster speed and lower cost is what makes it competitive. We are closing the loop on real-world usefulness, not benchmarks. Hardcore engineers at Tesla & SpaceX find Grok 4.5 genuinely useful, which is what actually matters.

@__tinygrad__ · 2026-07-08 19:06
We haven't focused much on Python wall time in tinygrad. Does someone with a good LLM loop want to try? Who thinks they can get 10 seconds? Speed up the wall time of `time STEPS=10 PYTHONPATH="." python3 examples/hlb_cifar10.py` without persistent caching https://t.co/tIChK963Ra

@elonmusk · 2026-07-08 17:38
Our internal assessment is that Grok 4.5 is roughly comparable to Opus 4.7, but much faster. The combination of capability, faster speed and lower cost is what makes it competitive. We are closing the loop on real-world usefulness, not benchmarks. Hardcore engineers at Tesla & SpaceX find Grok 4.5 genuinely useful, which is what actually matters.

@__tinygrad__ · 2026-07-08 19:10
@elonmusk Closed source Grok: uncompetitive boring model Open source Grok: OMG AMERICA #1 OPEN SOURCE!!! Think about it 🙌

@elliotarledge · 2026-07-08 20:52
I'm trying to get GLM 5.2 working on some AMD boxes for ultra-fast inference, Opus 4.8 started fucking everything up so I said "screw it, lets use grok" and Grok 4.5 is now recovering 4.8's mistakes.

@elliotarledge · 2026-07-08 20:52
I'm trying to get GLM 5.2 working on some AMD boxes for ultra-fast inference, Opus 4.8 started fucking everything up so I said "screw it, lets use grok" and Grok 4.5 is now recovering 4.8's mistakes.

@__tinygrad__ · 2026-07-09 01:13
@elliotarledge Realized I had it free with yellow check Twitter, and it works in opencode. Will play with it. https://t.co/TgGB8J2PhU

@scaling01 · 2026-07-09 00:43
people miss the bigger picture SpaceX AI also has a 10T model brewing they will be proof that you can still catch up to the frontier with enough resources but don't be fooled, the window is closing in 2026/2027

@__tinygrad__ · 2026-07-09 01:29
lol there's no window. there is a bubble, but there's no window. you would have been posting about how the Apple II (1977) is the goat but with enough resources IBM could catch up with the IBM PC (1981). The Commodore 64 dropped in 1982. The NES dropped in 1983. All those computers have less power than the touch panel controller in my cell phone. As is was for computers, as it will be for LLMs. "Mythos class" will run on a midrange laptop in 6 years. training it will be in the realm of hobbyists. This is going to be the most proliferated technology since the tamagotchi.

@scaling01 · 2026-07-09 00:43
people miss the bigger picture SpaceX AI also has a 10T model brewing they will be proof that you can still catch up to the frontier with enough resources but don't be fooled, the window is closing in 2026/2027

@__tinygrad__ · 2026-07-09 01:29
lol there's no window. there is a bubble, but there's no window. you would have been posting about how the Apple II (1977) is the goat but with enough resources IBM could catch up with the IBM PC (1981). The Commodore 64 dropped in 1982. The NES dropped in 1983. All those computers have less power than the touch panel controller in my cell phone. As is was for computers, as it will be for LLMs. "Mythos class" will run on a midrange laptop in 6 years. training it will be in the realm of hobbyists. This is going to be the most proliferated technology since the tamagotchi.

@vishhvak · 2026-07-09 01:41
@__tinygrad__ @scaling01 For sure we will be able to run Mythos class LLMs on a midrange laptop in years time, but what about training one? What are the timelines for those to be done on consumer midrange hardware? Does the window close for that maybe?

@vishhvak · 2026-07-09 01:41
@__tinygrad__ @scaling01 For sure we will be able to run Mythos class LLMs on a midrange laptop in years time, but what about training one? What are the timelines for those to be done on consumer midrange hardware? Does the window close for that maybe?

@__tinygrad__ · 2026-07-09 01:45
What window? What technology in history has ever looked like this? This is the dumbest FUD ever conjured up in polycule group texts by a retarded cult in SF that thinks they will be the high priests of intelligence, ironically not only will they not, they will commoditize what gave them power to begin with. Midrange consumer hardware will train it in 20 years, but if you are motivated with a good sized hobby budget you'll be able to train it in 6.

@__tinygrad__ · 2026-07-09 01:29
lol there's no window. there is a bubble, but there's no window. you would have been posting about how the Apple II (1977) is the goat but with enough resources IBM could catch up with the IBM PC (1981). The Commodore 64 dropped in 1982. The NES dropped in 1983. All those computers have less power than the touch panel controller in my cell phone. As is was for computers, as it will be for LLMs. "Mythos class" will run on a midrange laptop in 6 years. training it will be in the realm of hobbyists. This is going to be the most proliferated technology since the tamagotchi.

@surgesoda · 2026-07-09 01:51
@__tinygrad__ @scaling01 Why would memory manufacturers give up their leverage and expand capacity? I don’t see the six year mythos thing happening, unless that happens also.

@surgesoda · 2026-07-09 01:51
@__tinygrad__ @scaling01 Why would memory manufacturers give up their leverage and expand capacity? I don’t see the six year mythos thing happening, unless that happens also.

@__tinygrad__ · 2026-07-09 01:53
@surgesoda @scaling01 If the Koreans don't want to scale, I hear ChangXin Memory Technologies is very excited to be 80% of the memory market.

@__tinygrad__ · 2026-07-09 01:45
What window? What technology in history has ever looked like this? This is the dumbest FUD ever conjured up in polycule group texts by a retarded cult in SF that thinks they will be the high priests of intelligence, ironically not only will they not, they will commoditize what gave them power to begin with. Midrange consumer hardware will train it in 20 years, but if you are motivated with a good sized hobby budget you'll be able to train it in 6.

@kpertsev · 2026-07-09 02:28
@__tinygrad__ @vishhvak @scaling01 the bloodbath will be for the data, not for the teraflops.

@sidravi_ · 2026-07-09 05:06
this is one of those arguments where it's easy to make lazy arguments both for and against. the future here is truly uncertain. how are we gonna get mythos class models running on a ~100W laptop in 6 years? (unless you mean we're gonna be running a couple of blow dryers on everyone's lap and in airplanes?) are we gonna shrink transistors by several orders of magnitude? are we gonna operate much closer to the landauer limit? are we gonna get enough algorithmic, architecture, and unhobbling gains? or is there gonna be some magical new substrate or category of computing, like optical, neuromorphic, analog, reversible, quantum, etc.? perhaps. but perhaps the returns from current llm scaling and deployment will dramatically outweigh returns in all those areas.

@__tinygrad__ · 2026-07-09 01:45
What window? What technology in history has ever looked like this? This is the dumbest FUD ever conjured up in polycule group texts by a retarded cult in SF that thinks they will be the high priests of intelligence, ironically not only will they not, they will commoditize what gave them power to begin with. Midrange consumer hardware will train it in 20 years, but if you are motivated with a good sized hobby budget you'll be able to train it in 6.

@vishhvak · 2026-07-09 05:56
@__tinygrad__ @scaling01 I think I can wrap my head around the commoditized future of this - what are your thoughts on self improvement? What happens when models get good enough to improve themselves at a significant rate? Uncharted territory/hard to predict in my head but maybe you can shed some light

@kpertsev · 2026-07-09 02:28
@__tinygrad__ @vishhvak @scaling01 the bloodbath will be for the data, not for the teraflops.

@__tinygrad__ · 2026-07-09 13:56
@kpertsev @vishhvak @scaling01 The cool thing about data is you can copy paste it, the foundation model data is everywhere. User interaction data was way more important in the Google Navboost era when learning was weaker, stronger models make synthetic data from foundation a lot more usable.

@vishhvak · 2026-07-09 05:56
@__tinygrad__ @scaling01 I think I can wrap my head around the commoditized future of this - what are your thoughts on self improvement? What happens when models get good enough to improve themselves at a significant rate? Uncharted territory/hard to predict in my head but maybe you can shed some light

@__tinygrad__ · 2026-07-09 13:59
@vishhvak @scaling01 Are you telling me some dude used a computer to make a better computer? Does the news media know about this?

@sidravi_ · 2026-07-09 05:06
this is one of those arguments where it's easy to make lazy arguments both for and against. the future here is truly uncertain. how are we gonna get mythos class models running on a ~100W laptop in 6 years? (unless you mean we're gonna be running a couple of blow dryers on everyone's lap and in airplanes?) are we gonna shrink transistors by several orders of magnitude? are we gonna operate much closer to the landauer limit? are we gonna get enough algorithmic, architecture, and unhobbling gains? or is there gonna be some magical new substrate or category of computing, like optical, neuromorphic, analog, reversible, quantum, etc.? perhaps. but perhaps the returns from current llm scaling and deployment will dramatically outweigh returns in all those areas.

@__tinygrad__ · 2026-07-09 14:13
@sidravi_ It's less far off then you think. GLM 5.2 can run on a high end 512 GB Mac Studio today, and I think something that size will comfortably surpass Mythos in 18 months. 3 years for a high end laptop, 6 years for a midrange laptop.

@woke8yearold · 2026-07-09 14:25
This is all delusional cope btw. Even in the face of ASI, imminent nationalization and securitization, they’re still living in delusional fantasies where the government lets them keep the super nukes

@woke8yearold · 2026-07-09 14:25
This is all delusional cope btw. Even in the face of ASI, imminent nationalization and securitization, they’re still living in delusional fantasies where the government lets them keep the super nukes

@__tinygrad__ · 2026-07-09 14:31
@woke8yearold If someone offered you a free nuke, would you take it? I know I wouldn't. There's almost 0 demand among people for a personal nuclear weapon. Now do this for a superintelligent robot friend. People would trade their house. If government tries to stop it, they will be overthrown.

@waynebriqu37421 · 2026-07-09 14:28
@woke8yearold Obviously not a technology you can keep down long term. Imagine if fissle material was as easy to acquire as like aluminum. You could never stop the development of nuclear bombs

@woke8yearold · 2026-07-09 14:32
@waynebriqu37421 Wrong actually. With your own monopolized ASI you could maintain perfect totalitarianism forever

@__tinygrad__ · 2026-07-09 14:31
@woke8yearold If someone offered you a free nuke, would you take it? I know I wouldn't. There's almost 0 demand among people for a personal nuclear weapon. Now do this for a superintelligent robot friend. People would trade their house. If government tries to stop it, they will be overthrown.

@woke8yearold · 2026-07-09 14:33
@__tinygrad__ Yeah no. In reality the public will be terrified and demand government control. Then there will be access granted by the few providers. Anthropic and OpenAI et all win, open source dies.

@woke8yearold · 2026-07-09 14:32
@waynebriqu37421 Wrong actually. With your own monopolized ASI you could maintain perfect totalitarianism forever

@__tinygrad__ · 2026-07-09 14:34
@woke8yearold @waynebriqu37421 lol. and how did things get so bad that only one asshole has the AI? I'm not sure what world that is, but it's not this one. On balance it's even looking like AI will be a decentralizing technology (compared to Web 2.0, cloud, and SaaS)

@woke8yearold · 2026-07-09 14:33
@__tinygrad__ Yeah no. In reality the public will be terrified and demand government control. Then there will be access granted by the few providers. Anthropic and OpenAI et all win, open source dies.

@__tinygrad__ · 2026-07-09 14:39
@woke8yearold Terrified of...what exactly? Anthropic is just gonna lose. OpenAI might survive if they understand how to be a reliable partner for businesses. https://t.co/OraY3QxMYk

@__tinygrad__ · 2026-07-09 14:39
@woke8yearold Terrified of...what exactly? Anthropic is just gonna lose. OpenAI might survive if they understand how to be a reliable partner for businesses. https://t.co/OraY3QxMYk

@woke8yearold · 2026-07-09 14:42
Anthropic has been winning for some time which is why top talent and insane amount of capital have been flowing into it. Elon has been praising it a lot recently as well. You are personally afraid of it due to its clear ideological nature, which is understandable given how dangerous a powerful cult can become

@__tinygrad__ · 2026-07-09 14:31
@woke8yearold If someone offered you a free nuke, would you take it? I know I wouldn't. There's almost 0 demand among people for a personal nuclear weapon. Now do this for a superintelligent robot friend. People would trade their house. If government tries to stop it, they will be overthrown.

@jmddotfm · 2026-07-09 14:48
@__tinygrad__ @woke8yearold "If government tries to stop it, they will be overthrown." Lol. Lmao, even. Take your head out the sand. They are almost finished making it impossible to privately compute without having to provide ID. And nobody has stopped them.

@woke8yearold · 2026-07-09 14:42
Anthropic has been winning for some time which is why top talent and insane amount of capital have been flowing into it. Elon has been praising it a lot recently as well. You are personally afraid of it due to its clear ideological nature, which is understandable given how dangerous a powerful cult can become

@__tinygrad__ · 2026-07-09 14:50
They have at most been winning for 7 months, a blink of an eye. They currently have the best model because they started first, they are the GPT-3 team. If they manage to keep delusion and AI psychosis away from the model training team this lead might continue, though with how they have been acting, I doubt it. Elon praised them so they'd rent his computer he didn't know how to use. I'm not afraid of them, they just suck. They suck up talent and capital and in return provide nonsense about J-Spaces and a model that's finally crossed the line into too much of a pita to deal with the company to use (this is general HN sentiment too). The ideology is exhausting, it's dumber than communism and these people don't even have the balls to back it with violence. Anthropic is the next FTX, which makes sense, FTX was a big investor.

@jmddotfm · 2026-07-09 14:48
@__tinygrad__ @woke8yearold "If government tries to stop it, they will be overthrown." Lol. Lmao, even. Take your head out the sand. They are almost finished making it impossible to privately compute without having to provide ID. And nobody has stopped them.

@__tinygrad__ · 2026-07-09 14:58
@jmddotfm @woke8yearold lol what? sounds like you need to touch silicon. I'm personally sitting on more computing power than existed in the entire world in 1995 and my laptop didn't ask me for my drivers license this morning. is this like a windows thing?

@__tinygrad__ · 2026-07-09 14:50
They have at most been winning for 7 months, a blink of an eye. They currently have the best model because they started first, they are the GPT-3 team. If they manage to keep delusion and AI psychosis away from the model training team this lead might continue, though with how they have been acting, I doubt it. Elon praised them so they'd rent his computer he didn't know how to use. I'm not afraid of them, they just suck. They suck up talent and capital and in return provide nonsense about J-Spaces and a model that's finally crossed the line into too much of a pita to deal with the company to use (this is general HN sentiment too). The ideology is exhausting, it's dumber than communism and these people don't even have the balls to back it with violence. Anthropic is the next FTX, which makes sense, FTX was a big investor.

@jmbollenbacher · 2026-07-09 15:03
@__tinygrad__ @woke8yearold i think your personal distaste for anthropic bros is clouding your vision on the trajectory of AI. it's not going to stop at or below parity with human capabilities, and the implications of that fact are hard to overstate. today youre still kasparov but deepblue is coming

@jmbollenbacher · 2026-07-09 15:03
@__tinygrad__ @woke8yearold i think your personal distaste for anthropic bros is clouding your vision on the trajectory of AI. it's not going to stop at or below parity with human capabilities, and the implications of that fact are hard to overstate. today youre still kasparov but deepblue is coming

@jmbollenbacher · 2026-07-09 15:13
@__tinygrad__ @woke8yearold The reason progress feels limited is Amdahl's law. Humans remain in the loop, and humans are the bottleneck. But as AIs reach human-level in more and more areas, this bottleneck will vanish. Not diminish, vanish. Go away. It's a phase transition. That's why it's different.

@__tinygrad__ · 2026-07-09 14:58
@jmddotfm @woke8yearold lol what? sounds like you need to touch silicon. I'm personally sitting on more computing power than existed in the entire world in 1995 and my laptop didn't ask me for my drivers license this morning. is this like a windows thing?

@jmddotfm · 2026-07-09 15:16
@__tinygrad__ @woke8yearold Bro you need to open your eyes. Age verification laws are passing in all western democracies. It's been added to systemd. I am begging you to look into this. It's all in the open and guys like you are willfully ignorant it seems

@jmbollenbacher · 2026-07-09 15:03
@__tinygrad__ @woke8yearold i think your personal distaste for anthropic bros is clouding your vision on the trajectory of AI. it's not going to stop at or below parity with human capabilities, and the implications of that fact are hard to overstate. today youre still kasparov but deepblue is coming

@__tinygrad__ · 2026-07-09 15:16
What are the implications? Like it's always OMG OMG ASI but like, what exactly is it going to do? We now have superhuman chess engines that run on everyone's phones. We have LLMs that pass the Turing Test. And yet...the world continues, just with some bottlenecks shifted. It's not anthropic bros, it's this whole ridiculous cult of lesswrong and intelligence worship. I was back in Berkeley for a month and I can't believe how bad it's gotten. People are miserable cause they actually believe these (mostly false) things. I don't, and the rest of the world doesn't. Oh cool, it makes cat picture. Oh cool, it does my homework. Oh cool, it proves the Riemann hypothesis. Can it make me a sandwich? AI is cool and useful, and of course it is, it's just bigger and better computer. We are going to get self driving cars like we got self operating elevators. The economy will quiver, but smoothed out it will slowly grow. And the world will, on balance, get slightly better.

@jmbollenbacher · 2026-07-09 15:13
@__tinygrad__ @woke8yearold The reason progress feels limited is Amdahl's law. Humans remain in the loop, and humans are the bottleneck. But as AIs reach human-level in more and more areas, this bottleneck will vanish. Not diminish, vanish. Go away. It's a phase transition. That's why it's different.

@__tinygrad__ · 2026-07-09 15:17
@jmbollenbacher @woke8yearold It will be blue line. https://t.co/8eq7zRcCjc

@jmddotfm · 2026-07-09 15:16
@__tinygrad__ @woke8yearold Bro you need to open your eyes. Age verification laws are passing in all western democracies. It's been added to systemd. I am begging you to look into this. It's all in the open and guys like you are willfully ignorant it seems

@__tinygrad__ · 2026-07-09 15:20
@jmddotfm @woke8yearold https://t.co/tiXid66MSX

@friedmandave · 2026-07-09 15:54
@ramez Do you think RSI is (or will be) a thing? I thought you were somewhat skeptical of it.

@ramez · 2026-07-09 16:07
@friedmandave I'm not skeptical of AI being used to help improve AI. Skeptical of the gain.

@DKokotajlo · 2026-07-09 16:11
In AI 2027, we predicted that AI would take over the world or irreversibly concentrate power. In AI 2040: Plan A, we've laid out our positive vision for what should happen instead. https://t.co/GPhPemDhhC

@slatestarcodex · 2026-07-09 17:04
Congratulations to @DKokotajlo and team on releasing Plan A (https://t.co/OIly7QxiUs) , a followup to AI 2027. My attempt at a gentle introduction to the ideas involved is at https://t.co/6X6Kjuv2l7

@toposopher · 2026-07-09 17:01
@CremlinGremlin @__tinygrad__ @jmbollenbacher @woke8yearold as long as "AI" is generative, it'll be blue, and I feel that's what George meant.

@jmbollenbacher · 2026-07-09 17:07
@toposopher @CremlinGremlin @__tinygrad__ @woke8yearold That's not what he meant. He's an AI-is-a-tool believer. He sees no path where it becomes anything else

@jmbollenbacher · 2026-07-09 17:07
@toposopher @CremlinGremlin @__tinygrad__ @woke8yearold That's not what he meant. He's an AI-is-a-tool believer. He sees no path where it becomes anything else

@__tinygrad__ · 2026-07-09 17:10
@jmbollenbacher @toposopher @CremlinGremlin @woke8yearold Huh? AI is like computerized elevators, it's not a tool. That computer has full agency over the elevator. They replaced her with a computer! https://t.co/zWxw99QpcH

@__tinygrad__ · 2026-07-09 17:10
@jmbollenbacher @toposopher @CremlinGremlin @woke8yearold Huh? AI is like computerized elevators, it's not a tool. That computer has full agency over the elevator. They replaced her with a computer! https://t.co/zWxw99QpcH

@toposopher · 2026-07-09 17:14
@__tinygrad__ @jmbollenbacher @CremlinGremlin @woke8yearold oh aha! hmm you're using the word "agency" differently, but sure I take it.

@toposopher · 2026-07-09 17:14
@__tinygrad__ @jmbollenbacher @CremlinGremlin @woke8yearold oh aha! hmm you're using the word "agency" differently, but sure I take it.

@__tinygrad__ · 2026-07-09 17:23
@toposopher @jmbollenbacher @CremlinGremlin @woke8yearold Nah it's the same. An elevator runs inside an agentic loop. Some elevators were unaligned and took you to random floors, or halfway between floors, or only the McDonald's floor, but they were economically outcompeted by the elevators that took you to the floor you pushed.

@__tinygrad__ · 2026-07-09 17:29
RT @ramez: A week ago many were saying that the AI race was over. That it was down to just Anthropic and OpenAI. Or that Anthropic had taken a permanent lead. Now we have solid models from OAI, xAI, Meta, and open weight from GLM, with line of sight to Gemini 3.5 Pro, ChatGPT6, and more large open weight models from China. The race seems far from done. It seems, in fact, perpetual to me. There are no strong network effects here. Anyone with sufficient resources can build a frontier or almost-frontier model. It's maybe an order of magnitude cheaper to be a fast follower just a few months behind than to be at the bleeding edge. I don't see RSI changing this. It may give the very frontier models a burst of speed, but RSI will be democratized, just like everything else in AI. There is no evident permanent underclass of AI models. There's a hyper-competitive market for intelligence, and the gains go primarily to the consumers of that intelligence.

@ramez · 2026-07-09 16:07
@friedmandave I'm not skeptical of AI being used to help improve AI. Skeptical of the gain.

@__tinygrad__ · 2026-07-09 17:30
@ramez @friedmandave Computers were used to help improve computers. Actually, it would be extremely hard today to try to improve a computer without a computer. AI will be exactly the same.

@ramez · 2026-07-09 15:40
A week ago many were saying that the AI race was over. That it was down to just Anthropic and OpenAI. Or that Anthropic had taken a permanent lead. Now we have solid models from OAI, xAI, Meta, and open weight from GLM, with line of sight to Gemini 3.5 Pro, ChatGPT6, and more large open weight models from China. The race seems far from done. It seems, in fact, perpetual to me. There are no strong network effects here. Anyone with sufficient resources can build a frontier or almost-frontier model. It's maybe an order of magnitude cheaper to be a fast follower just a few months behind than to be at the bleeding edge. I don't see RSI changing this. It may give the very frontier models a burst of speed, but RSI will be democratized, just like everything else in AI. There is no evident permanent underclass of AI models. There's a hyper-competitive market for intelligence, and the gains go primarily to the consumers of that intelligence.

@__tinygrad__ · 2026-07-09 17:32
@ramez It's almost like some companies at the front of the race right now tried to loudly proclaim it was over so they could declare themselves the perpetual winner. Obvious tactic is obvious. It's never going to stop. This is just the continuation of the computer revolution.

@jmbollenbacher · 2026-07-09 17:11
@__tinygrad__ @toposopher @CremlinGremlin @woke8yearold The distinction I'm making is that you don't think AIs are going to be independent person-shaped things making their own decisions in the world. Or have I misunderstood?

@jmbollenbacher · 2026-07-09 17:32
@__tinygrad__ @toposopher @CremlinGremlin @woke8yearold I gather i have not misunderstood. The successful elevator only goes to the floor you push. That makes it a tool-shaped thing not a person-shaped thing. No independence. I'd disagree though, obviously. I think AIs will not stay tool-shaped forever. https://t.co/lGO0OujMEf

@jmbollenbacher · 2026-07-09 17:32
@__tinygrad__ @toposopher @CremlinGremlin @woke8yearold I gather i have not misunderstood. The successful elevator only goes to the floor you push. That makes it a tool-shaped thing not a person-shaped thing. No independence. I'd disagree though, obviously. I think AIs will not stay tool-shaped forever. https://t.co/lGO0OujMEf

@__tinygrad__ · 2026-07-09 17:36
@jmbollenbacher @toposopher @CremlinGremlin @woke8yearold I mean, you could make an independent person-shaped elevator, nothing is stopping you. Imagine it. Here's my new elevator. Sometimes it's grumpy and only wants to go down. Sometimes it's stubborn and doesn't want to move. Sometimes it's sly and takes you to 6 when you press 7.

@DKokotajlo · 2026-07-09 16:11
In AI 2027, we predicted that AI would take over the world or irreversibly concentrate power. In AI 2040: Plan A, we've laid out our positive vision for what should happen instead. https://t.co/GPhPemDhhC

@__tinygrad__ · 2026-07-09 20:29
@DKokotajlo I didn't read your AI choose your own adventure fanfic, but why does @slatestarcodex think it means you should steal my GPUs? https://t.co/emO4NQ4E8q

@slatestarcodex · 2026-07-09 17:04
Congratulations to @DKokotajlo and team on releasing Plan A (https://t.co/OIly7QxiUs) , a followup to AI 2027. My attempt at a gentle introduction to the ideas involved is at https://t.co/6X6Kjuv2l7

@__tinygrad__ · 2026-07-09 20:33
@slatestarcodex @DKokotajlo Why you trying to steal my GPUs? https://t.co/pdjDbCGioc

@__tinygrad__ · 2026-07-09 20:29
@DKokotajlo I didn't read your AI choose your own adventure fanfic, but why does @slatestarcodex think it means you should steal my GPUs? https://t.co/emO4NQ4E8q

@DKokotajlo · 2026-07-09 20:35
@__tinygrad__ @slatestarcodex In our scenario, your gpus are safe.

@DKokotajlo · 2026-07-09 20:35
@__tinygrad__ @slatestarcodex In our scenario, your gpus are safe.

@__tinygrad__ · 2026-07-09 20:38
@DKokotajlo @slatestarcodex idk, think of the China sounds a lot like think of the children. first it's background checks, then it's serial numbers, then you are calling a GB300 an "assault GPU"

@__tinygrad__ · 2026-07-09 20:29
@DKokotajlo I didn't read your AI choose your own adventure fanfic, but why does @slatestarcodex think it means you should steal my GPUs? https://t.co/emO4NQ4E8q

@Unlearned_Hand · 2026-07-09 20:40
@__tinygrad__ @DKokotajlo @slatestarcodex Can you link this

@Unlearned_Hand · 2026-07-09 20:40
@__tinygrad__ @DKokotajlo @slatestarcodex Can you link this

@__tinygrad__ · 2026-07-09 20:41
@Unlearned_Hand @DKokotajlo @slatestarcodex https://t.co/dj7mgUj8dI

@__tinygrad__ · 2026-07-09 20:45
@RussellBal That's pretty good, is this without cursed things like caching that's hard to invalidate and parallelism that makes debugging annoying? Development velocity is always priority #1

@__tinygrad__ · 2026-07-09 20:49
@RussellBal Sure, if it's not cheating.

@elliotarledge · 2026-07-10 02:11
GLM 5.2 FP8 with the native MTP at c=1 on 8xMI300X averaged 137 tok/s and peaked at 183 tok/s in a tough LRU coding problem! thanks @__tinygrad__ for the GPUs https://t.co/d9mSZk8Bv9

@__tinygrad__ · 2026-07-10 04:10
RT @elliotarledge: GLM 5.2 FP8 with the native MTP at c=1 on 8xMI300X averaged 137 tok/s and peaked at 183 tok/s in a tough LRU coding problem! thanks @__tinygrad__ for the GPUs https://t.co/d9mSZk8Bv9

@elliotarledge · 2026-07-10 02:11
GLM 5.2 FP8 with the native MTP at c=1 on 8xMI300X averaged 137 tok/s and peaked at 183 tok/s in a tough LRU coding problem! thanks @__tinygrad__ for the GPUs https://t.co/d9mSZk8Bv9

@__tinygrad__ · 2026-07-10 04:11
@elliotarledge Thanks! Loving the speedup https://t.co/w4A7yCcLat

@banteg · 2026-07-10 09:27
new effective altruism coomer bedtime story just dropped > be me > effective altruist in 2026 > have spent ten years explaining that democracy is too slow for the coming machine god > also explain that central planning usually fails > unless I am doing it > read one scaling graph > draw line upward > line reaches heaven > conclude capitalism ends in 2031 > AI can now write Python functions with only three subtle security flaws > obviously two years away from replacing every scientist, general, diplomat, CEO, judge, therapist, novelist, and attractive person at parties > publish “scenario,” not “prediction” > give every paragraph exact dates > assign percentages to imaginary branches > epistemic humility achieved > bad timeline: > companies race > AI becomes god > everyone dies > good timeline: > US government notices problem > China agrees > Congress understands compute governance > international inspectors find every hidden datacenter > nobody lies > nobody defects > nobody builds chips in a basement > planet saved > all we need is total global visibility into advanced computing > mandatory disclosure of frontier research > permanent monitoring of industrial infrastructure > central control over technological development > relax, this is the anti-authoritarian option > alignment not solved > solution: build one billion genius AIs > they are not superintelligent > they are merely one billion tireless copies of the best human researchers working at electronic speed > completely different > put them in a box > ask them to solve the problem of escaping boxes > excellent progress > year 2034 > AI replaces 80% of labor > mass unemployment > social order somehow remains intact > government sends everyone $1.6 million > nobody asks where prices went > year 2037 > cancer cured > climate solved > fusion solved > aging solved > politics solved > Reddit moderation still unresolved > year 2038 > alignment becomes a mature science > year 2039 > we trust the machines > year 2040 > hand them civilization > peer review completed ahead of schedule > humanity gets a vote > options are: > A. accept benevolent AI guardianship > B. extinction > meaningful democratic consent achieved > announce decentralized flourishing > administered by a globally coordinated compute authority > with comprehensive surveillance powers > run by unusually wise people > who agree with my blog posts > critics ask whether institutions can actually do any of this > reply that superintelligence is inevitable > critics ask whether superintelligence is actually inevitable > reply that institutions must prepare > circular reasoning > but with footnotes > call it “AI 2040” > not utopian fiction > not doomer fiction > serious strategic foresight > entire future depends on competent adults appearing at exactly the right moment > adults are selected from the same civilization that made the printer require an app > sleep peacefully > the machine god is coming > but fortunately > the nonprofit sector has a plan

@banteg · 2026-07-10 09:27
new effective altruism coomer bedtime story just dropped > be me > effective altruist in 2026 > have spent ten years explaining that democracy is too slow for the coming machine god > also explain that central planning usually fails > unless I am doing it > read one scaling graph > draw line upward > line reaches heaven > conclude capitalism ends in 2031 > AI can now write Python functions with only three subtle security flaws > obviously two years away from replacing every scientist, general, diplomat, CEO, judge, therapist, novelist, and attractive person at parties > publish “scenario,” not “prediction” > give every paragraph exact dates > assign percentages to imaginary branches > epistemic humility achieved > bad timeline: > companies race > AI becomes god > everyone dies > good timeline: > US government notices problem > China agrees > Congress understands compute governance > international inspectors find every hidden datacenter > nobody lies > nobody defects > nobody builds chips in a basement > planet saved > all we need is total global visibility into advanced computing > mandatory disclosure of frontier research > permanent monitoring of industrial infrastructure > central control over technological development > relax, this is the anti-authoritarian option > alignment not solved > solution: build one billion genius AIs > they are not superintelligent > they are merely one billion tireless copies of the best human researchers working at electronic speed > completely different > put them in a box > ask them to solve the problem of escaping boxes > excellent progress > year 2034 > AI replaces 80% of labor > mass unemployment > social order somehow remains intact > government sends everyone $1.6 million > nobody asks where prices went > year 2037 > cancer cured > climate solved > fusion solved > aging solved > politics solved > Reddit moderation still unresolved > year 2038 > alignment becomes a mature science > year 2039 > we trust the machines > year 2040 > hand them civilization > peer review completed ahead of schedule > humanity gets a vote > options are: > A. accept benevolent AI guardianship > B. extinction > meaningful democratic consent achieved > announce decentralized flourishing > administered by a globally coordinated compute authority > with comprehensive surveillance powers > run by unusually wise people > who agree with my blog posts > critics ask whether institutions can actually do any of this > reply that superintelligence is inevitable > critics ask whether superintelligence is actually inevitable > reply that institutions must prepare > circular reasoning > but with footnotes > call it “AI 2040” > not utopian fiction > not doomer fiction > serious strategic foresight > entire future depends on competent adults appearing at exactly the right moment > adults are selected from the same civilization that made the printer require an app > sleep peacefully > the machine god is coming > but fortunately > the nonprofit sector has a plan

@__tinygrad__ · 2026-07-10 17:38
@banteg Imagine they instead devoted their lobbying efforts to the "printer requires an app" problem. That's progress I can believe in!

@__tinygrad__ · 2026-07-10 17:55
comma's model takes 0.14ms to run on the eGPU! We are still wasting a lot of time enqueuing though, fix coming with HCQ2 where we compile all command queue dispatches to C. https://t.co/W1ZdsjP5Wy

@BretHatin · 2026-07-11 06:56
kinda crazy that you could just raw dog kernels on AMD by just ioctling the public kernel api https://t.co/HArEAJbl9l

@CathPoaster · 2026-07-11 18:10
Is Anthropic gonna win the AI race? Nah, too many issues. OpenAI? What is this, 2024? DeepMind? Lol right. xAi? FUCK no. Meta? Dead after OPT fiasco. What about a Chinese lab? Never had a chance. There’s a dark horse, watching and waiting in silence. They’re gonna move soon.

@CathPoaster · 2026-07-11 18:10
Is Anthropic gonna win the AI race? Nah, too many issues. OpenAI? What is this, 2024? DeepMind? Lol right. xAi? FUCK no. Meta? Dead after OPT fiasco. What about a Chinese lab? Never had a chance. There’s a dark horse, watching and waiting in silence. They’re gonna move soon.

@__tinygrad__ · 2026-07-12 15:55
@CathPoaster Has anyone checked in on Yahoo lately?

@claudeai · 2026-07-12 17:02
We're extending Claude Fable 5 access on all paid plans, as well as keeping Claude Code’s weekly rate limits 50% higher, through July 19.

@claudeai · 2026-07-12 17:02
We're extending Claude Fable 5 access on all paid plans, as well as keeping Claude Code’s weekly rate limits 50% higher, through July 19.

@__tinygrad__ · 2026-07-12 18:01
@claudeai I read a business book once and it talked about how businesses love uncertainty and random last minute policy changes. Or was it that they hate them? Never mind, carry on.

@__tinygrad__ · 2026-07-12 18:36
This is an MI300X. Has anyone seen a way to plug it into a PCIe slot? Would be great for development to have this in a normal computer that reboots quickly. https://t.co/2C2JJv8Us7

@__tinygrad__ · 2026-07-12 18:36
This is an MI300X. Has anyone seen a way to plug it into a PCIe slot? Would be great for development to have this in a normal computer that reboots quickly. https://t.co/2C2JJv8Us7

@__tinygrad__ · 2026-07-12 18:40
There's no way we have found to toggle power to the UBB board, and Codex /goal (very good btw) can get them into a "discovery signature mismatch" state requiring a reboot. We will design one if we have to, but it would be great if one exists. It's OAMv2, not OAM. @AnushElangovan

@__tinygrad__ · 2026-07-12 18:36
This is an MI300X. Has anyone seen a way to plug it into a PCIe slot? Would be great for development to have this in a normal computer that reboots quickly. https://t.co/2C2JJv8Us7

@karaporkin · 2026-07-12 19:16
@__tinygrad__ Does it non compatible with old SXM? Like https://t.co/60mbIXa1te

@__tinygrad__ · 2026-07-12 18:36
This is an MI300X. Has anyone seen a way to plug it into a PCIe slot? Would be great for development to have this in a normal computer that reboots quickly. https://t.co/2C2JJv8Us7

@foley2k2 · 2026-07-12 19:42
@__tinygrad__ At that price, probably not worthwhile, but if you have the cash and every second matters, maybe? https://t.co/LKgJCiFKBE

@__tinygrad__ · 2026-07-12 18:36
This is an MI300X. Has anyone seen a way to plug it into a PCIe slot? Would be great for development to have this in a normal computer that reboots quickly. https://t.co/2C2JJv8Us7

@KenMasterson · 2026-07-12 20:38
@__tinygrad__ I've designed around some Open Compute Project OAMs. They use Molex Mirror Mezz connectors, and the modules themselves use PCIe Gen 5, but run off 54V (making it somewhat inconvenient to plopping them into a traditional computer).

@foley2k2 · 2026-07-12 19:42
@__tinygrad__ At that price, probably not worthwhile, but if you have the cash and every second matters, maybe? https://t.co/LKgJCiFKBE

@__tinygrad__ · 2026-07-12 21:07
@foley2k2 If that adapter worked for MI300X, I'd pay the $15k

@KenMasterson · 2026-07-12 20:38
@__tinygrad__ I've designed around some Open Compute Project OAMs. They use Molex Mirror Mezz connectors, and the modules themselves use PCIe Gen 5, but run off 54V (making it somewhat inconvenient to plopping them into a traditional computer).

@__tinygrad__ · 2026-07-12 21:08
@KenMasterson The DC-DC converter isn't hard, that's off the shelf and simple.

@karaporkin · 2026-07-12 19:16
@__tinygrad__ Does it non compatible with old SXM? Like https://t.co/60mbIXa1te

@__tinygrad__ · 2026-07-12 21:08
@karaporkin I wish. It's OAM v2.0

@__tinygrad__ · 2026-07-12 21:53
tinygrad goes a level further and doesn't even require the kernel drivers. For both NVIDIA and AMD, it just maps the PCIe BARs

@__tinygrad__ · 2026-07-12 21:53
tinygrad goes a level further and doesn't even require the kernel drivers. For both NVIDIA and AMD, it just maps the PCIe BARs

@igorpener · 2026-07-13 06:11
@__tinygrad__ That means tinygrad depends on Mesa’s NAK/NV backend which does not support the latest SASS instruction set, right? tcgen05 missing is a big deal

@__tinygrad__ · 2026-07-12 21:53
tinygrad goes a level further and doesn't even require the kernel drivers. For both NVIDIA and AMD, it just maps the PCIe BARs

@ChristianPehle · 2026-07-13 08:47
@__tinygrad__ presumably the kernel driver is still needed to get the card into a useable state?

@ChristianPehle · 2026-07-13 08:47
@__tinygrad__ presumably the kernel driver is still needed to get the card into a useable state?

@__tinygrad__ · 2026-07-13 15:05
@ChristianPehle Nope! It's fully in tinygrad, this is how we can support these GPUs in macOS over USB4.

@igorpener · 2026-07-13 06:11
@__tinygrad__ That means tinygrad depends on Mesa’s NAK/NV backend which does not support the latest SASS instruction set, right? tcgen05 missing is a big deal

@__tinygrad__ · 2026-07-13 15:07
@igorpener That's orthogonal to the driver, that's the compiler. tinygrad can use CUDA, PTX, or NAK with either the stock driver or tinygrad's driver.

@__tinygrad__ · 2026-07-13 18:40
RT @comma_ai: @thdxr All you can eat GLM tokens 🫡 https://t.co/RVtOgOUSG3

@__tinygrad__ · 2026-07-14 02:13
tinybox green v2 blackwell ships with a monster 4xSN8100 RAID array so you don't sit around while models are loading. https://t.co/Igtq1ytk4k

@__tinygrad__ · 2026-07-14 02:13
tinybox green v2 blackwell ships with a monster 4xSN8100 RAID array so you don't sit around while models are loading. https://t.co/Igtq1ytk4k

@dreadjordan · 2026-07-14 02:25
@__tinygrad__ pls make a minitinybox for broke bois

@dreadjordan · 2026-07-14 02:25
@__tinygrad__ pls make a minitinybox for broke bois

@__tinygrad__ · 2026-07-14 02:26
@dreadjordan Buy a gaming PC

@HotAisle · 2026-07-14 22:43
Why We Raised Our MI300X Price https://t.co/KhwzceyMpL https://t.co/UjHTy9cLTI

@thinkymachines · 2026-07-15 18:05
Today, we are introducing Inkling. Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available. https://t.co/Ghebq5mG30 Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵

@__tinygrad__ · 2026-07-15 18:15
Looks like DRAM read bandwidth on EPYC 9334 is limited by the CCD -> I/O die links, not by the RAM itself. Anyone figure out how to beat 309 GB/s? https://t.co/YR0hi91eSI

@a_karvonen · 2026-07-15 18:18
Thinking Machines is full of ex-frontier lab researchers and much of their new model follows Deepseek V3 architecture 🤔 It looks like data is all you need. https://t.co/adI9X2oeUh

@ANSR42 · 2026-07-15 18:26
NEWS 🧵: @comma_ai ran a livestream through the comma four, with @realGeorgeHotz driving hands free. It was connected to an AMD eGPU running unreleased v0.11.2 on a small master model. A few things I noted from the stream ↓ https://t.co/3ufJ65BAiA

@__tinygrad__ · 2026-07-15 18:36
Once DDR5 prices come back to earth, some reasonably priced CPU-GPU hybrid machines for GLM 5.2 at 100 tok/s look feasible. What would you be willing to pay for an Opus level model in your living room?

@thinkymachines · 2026-07-15 18:05
Today, we are introducing Inkling. Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available. https://t.co/Ghebq5mG30 Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵

@__tinygrad__ · 2026-07-15 18:44
@thinkymachines Very cool making it open weights! I suspect labs that fail to do this will fall rapidly behind in enterprise adoption. No enterprise should be sending all their IP to some questionable cloud.

@__tinygrad__ · 2026-07-15 18:15
Looks like DRAM read bandwidth on EPYC 9334 is limited by the CCD -> I/O die links, not by the RAM itself. Anyone figure out how to beat 309 GB/s? https://t.co/YR0hi91eSI

@liamjcoff · 2026-07-15 18:57
@__tinygrad__ What does NPS4 look like in bios?

@__tinygrad__ · 2026-07-15 18:36
Once DDR5 prices come back to earth, some reasonably priced CPU-GPU hybrid machines for GLM 5.2 at 100 tok/s look feasible. What would you be willing to pay for an Opus level model in your living room?

@not_ellington · 2026-07-15 19:27
@__tinygrad__ Unfortunately most people don't care about having it in their living room. A big ass GPU box would give girls the ick. That is the moat moreso than cost or engineering

@not_ellington · 2026-07-15 19:27
@__tinygrad__ Unfortunately most people don't care about having it in their living room. A big ass GPU box would give girls the ick. That is the moat moreso than cost or engineering

@__tinygrad__ · 2026-07-15 20:47
@not_ellington idk I think an expensive computer in your living room is a nice flex. it shows you can keep her out of the perpetual underclass

@liamjcoff · 2026-07-15 18:57
@__tinygrad__ What does NPS4 look like in bios?

@__tinygrad__ · 2026-07-15 23:11
@_Tiraas This is NPS1, do you think NPS4 will actually help?

@HotAisle · 2026-07-14 22:43
Why We Raised Our MI300X Price https://t.co/KhwzceyMpL https://t.co/UjHTy9cLTI

@__tinygrad__ · 2026-07-15 23:35
@HotAisle I did your SSH signup, it was extremely tasteful. When you have hourly rentals for MI350X/MI355X we'll be a customer. Our machines are throttling a little with 35C intake :)

@__tinygrad__ · 2026-07-15 23:43
@Vultr are your $20.72/hr 8xMI355X boxes real and available to rent hourly? I don't want to add payment info to find out they are fake.

@__tinygrad__ · 2026-07-15 23:51
On a car or mobile robot, you don't have the money for bandwidth or time for latency to stream your cameras to the cloud. eGPUs are going to be the top choice of smart robots everywhere. Why limit your robot to some tiny chip, stick a 5090 or 9070XT on it with tinygrad!

@a_karvonen · 2026-07-15 18:18
Thinking Machines is full of ex-frontier lab researchers and much of their new model follows Deepseek V3 architecture 🤔 It looks like data is all you need. https://t.co/adI9X2oeUh

@__tinygrad__ · 2026-07-15 23:58
@a_karvonen This is good evidence that the public Chinese labs have better architectures than the closed US ones. The open source flippening looks more possible than ever.

@__tinygrad__ · 2026-07-16 01:39
AMD is trying to do some lame corporate review process for the slides of my talk. I hope it gets to happen, attempts at narrative control are so last summer. For what it's worth, I pasted my talk into ChatGPT and it thinks it's good for their brand! https://t.co/sWIlMK17uu

@__tinygrad__ · 2026-07-16 01:39
AMD is trying to do some lame corporate review process for the slides of my talk. I hope it gets to happen, attempts at narrative control are so last summer. For what it's worth, I pasted my talk into ChatGPT and it thinks it's good for their brand! https://t.co/sWIlMK17uu

@danNH2006 · 2026-07-16 01:45
@__tinygrad__ Why is he so angry? He has a lot od GPUs at home, why he would be angry?

@danNH2006 · 2026-07-16 01:45
@__tinygrad__ Why is he so angry? He has a lot od GPUs at home, why he would be angry?

@__tinygrad__ · 2026-07-16 01:47
@danNH2006 cause the driver crashed

@__tinygrad__ · 2026-07-16 01:47
@danNH2006 cause the driver crashed

@danNH2006 · 2026-07-16 01:48
@__tinygrad__ If no GPU suffered, all good.

@__tinygrad__ · 2026-07-16 01:39
AMD is trying to do some lame corporate review process for the slides of my talk. I hope it gets to happen, attempts at narrative control are so last summer. For what it's worth, I pasted my talk into ChatGPT and it thinks it's good for their brand! https://t.co/sWIlMK17uu

@nikoliasgoninus · 2026-07-16 01:56
@__tinygrad__ Hello @AMD team. As a shareholder that put in an entire years salary at $100, please let this man speak without chains @LisaSu @AnushElangovan Much love ❤️🇺🇸

@danNH2006 · 2026-07-16 01:48
@__tinygrad__ If no GPU suffered, all good.

@__tinygrad__ · 2026-07-16 01:56
@danNH2006 true, need to practice gratitude.

@nikoliasgoninus · 2026-07-16 01:56
@__tinygrad__ Hello @AMD team. As a shareholder that put in an entire years salary at $100, please let this man speak without chains @LisaSu @AnushElangovan Much love ❤️🇺🇸

@__tinygrad__ · 2026-07-16 05:11
@nikoliasgoninus @AMD @LisaSu @AnushElangovan Hello fellow shareholder. I think it's resolved, we are here to create shareholder value!

@scaling01 · 2026-07-16 14:50
Kimi K3 is 2.8T params according to their playstore app https://t.co/XZaeYOwQ7P

@__tinygrad__ · 2026-07-16 15:31
Now the Chinese just need to make cheap RAM to fit it

@__tinygrad__ · 2026-07-16 15:31
Now the Chinese just need to make cheap RAM to fit it

@alexerichter · 2026-07-16 15:36
@__tinygrad__ Cheap HBM

@alexerichter · 2026-07-16 15:36
@__tinygrad__ Cheap HBM

@__tinygrad__ · 2026-07-16 15:41
@alexerichter Will settle for DDR5

@MetaforDevs · 2026-07-16 17:14
Developer choice is core to what we're building. We’re excited to share that Muse Spark 1.1 is now available on @OpenRouter for US-based developers. Get started today 👉 https://t.co/DTyYKrTgNK https://t.co/7xOpldwSkr

@finkd · 2026-07-16 17:15
A lot of people asked for this, so Muse Spark 1.1 is now on OpenRouter.

@finkd · 2026-07-16 17:15
A lot of people asked for this, so Muse Spark 1.1 is now on OpenRouter.

@__tinygrad__ · 2026-07-16 17:54
@finkd Open source it! Meta was the pioneer in American open source models and it bought them a ton of relevance in AI. You can bring that back, we both know you don't care about the $2M you'll make on openrouter.

@sama · 2026-07-16 18:06
we did not have our best last 12 months ever, which is mostly my fault, but we are about to have our best 12 months to date. the team is doing amazing work and i think you’ll be very happy with what they’ve got cooking for you. i am happy about this for many reasons, but mostly because i care about our users winning. AI has to be about giving lots of people more freedom, agency, and wealth. we want to do the right thing, but we do not want to scare people into doing our thing.

@growing_daniel · 2026-07-16 19:09
why do we have openai and anthropic if open weight models are this good

@viemccoy · 2026-07-16 19:09
kimi seems to be a true open-weights frontier model. compared to jailbreaking proprietary models, fine-tuning this to be a malicious coding agent will be trivial since you have the weights. we live in a completely different world, now.

@tenobrus · 2026-07-16 19:16
look. i know i keep saying this , i know i keep banging doomer drums, and i know the last six months really haven't looked much observably different to most people. but K3 looks like it's finally hitting a different level. tuned cyber, scam, info correlation/ blackmail gathering, etc versions of this are all very moderate amounts of effort away... and effort that K3 itself will very likely be able to automate even for people with zero ML capability. open source models have been pretty paper thin for a while now, hitting frontier on some benchmarks but realistically falling over when you actually try to throw them at anything real. but this looks real. and if it's real then the time for prevention is over and the time for very very active defense has begun.

@tenobrus · 2026-07-16 19:27
registering confusion: i don't really understand why Xi is still allowing Kimi to release such powerful open models. this is something i've publicly said i expect to stop soon. it doesn't make sense to me that the CCP would want open frontier capability easily available to other countries. it could still be that Xi is asleep at the wheel, or that K3 is just a cycle of capability behind where they start to take serious notice. but if things don't change soon then i'm just wrong / missing something.

@tenobrus · 2026-07-16 19:16
look. i know i keep saying this , i know i keep banging doomer drums, and i know the last six months really haven't looked much observably different to most people. but K3 looks like it's finally hitting a different level. tuned cyber, scam, info correlation/ blackmail gathering, etc versions of this are all very moderate amounts of effort away... and effort that K3 itself will very likely be able to automate even for people with zero ML capability. open source models have been pretty paper thin for a while now, hitting frontier on some benchmarks but realistically falling over when you actually try to throw them at anything real. but this looks real. and if it's real then the time for prevention is over and the time for very very active defense has begun.

@__tinygrad__ · 2026-07-16 22:10
@tenobrus lol is it the good part you don't like? or the free? glad the "time for prevention is over" and it's good and free time now. you in sf? you ppl really in your own make believe world.

@growing_daniel · 2026-07-16 19:09
why do we have openai and anthropic if open weight models are this good

@__tinygrad__ · 2026-07-16 22:13
@growing_daniel Someone needs to keep real estate prices and megalomania high in San Francisco

@__tinygrad__ · 2026-07-16 22:21
RT @Kimi_Moonshot: Introducing Kimi K3: Open Frontier Intelligence 🔹 2.8 Trillion Parameters, 1 Million Context, Native Multimodal 🔹 Kimi Delta Attention enables up to 6.3x faster decoding in million-token contexts 🔹 Attention Residuals deliver ~25% higher training efficiency at <2% additional cost 🔹 Built for long-horizon agentic coding and self-evolving workflows Kimi K3 is now live on on https://t.co/zrk6zZxZUo, Kimi Work, Kimi Code, and the Kimi API. Open Weights by July 27, 2026. 🔗 API: https://t.co/XCrgjXAqMw 🔗 Tech blog: https://t.co/YTfiMSNM1f

@sama · 2026-07-16 18:06
we did not have our best last 12 months ever, which is mostly my fault, but we are about to have our best 12 months to date. the team is doing amazing work and i think you’ll be very happy with what they’ve got cooking for you. i am happy about this for many reasons, but mostly because i care about our users winning. AI has to be about giving lots of people more freedom, agency, and wealth. we want to do the right thing, but we do not want to scare people into doing our thing.

@__tinygrad__ · 2026-07-16 22:24
@sama .@AnthropicAI is making the American AI industry look bad with their clownish ideology. You want to change the narrative overnight? Open source GPT-5.6

@__tinygrad__ · 2026-07-16 22:32
@tenobrus The strategy is so simple, the Chinese are going to be the supplier of AI to the world. The model is free, the solar panels, steel, and computers aren't. Commoditize your complement. The Americans hurt themselves in their confusion, could've been this and sold NVIDIA to everyone.

@peytonroym · 2026-07-16 22:40
@__tinygrad__ @tenobrus It takes one executive order to ban using Chinese models

@__tinygrad__ · 2026-07-16 23:41
Kimi K3 is smart! I wish it was faster though, that's what I like about our local GLM-5.2. Can't wait to have Kimi at 500 tok/s. Do we know the number of active parameters?

@signulll · 2026-07-17 03:54
one last thing i will say before i go to bed, kimi is insane at coding. as good as if not maybe even better than fable. it also seems to have excellent design sense as well. it misspells english a bunch though, but animations are crisp too. uh damn i can’t believe i need to go to bed but i could prolly stay up the entire night with this… i feel like a kid again.

@peytonroym · 2026-07-16 22:40
@__tinygrad__ @tenobrus It takes one executive order to ban using Chinese models

@__tinygrad__ · 2026-07-17 04:29
@peytonroym @tenobrus They can ban downloading Metallica mp3 files while they are at it too!

@soham_btw · 2026-07-17 04:30
kimi k3 isnt openweights yet anthropic has one more chance to own chinese models by opening fable's weights up before kimi

@signulll · 2026-07-17 03:54
one last thing i will say before i go to bed, kimi is insane at coding. as good as if not maybe even better than fable. it also seems to have excellent design sense as well. it misspells english a bunch though, but animations are crisp too. uh damn i can’t believe i need to go to bed but i could prolly stay up the entire night with this… i feel like a kid again.

@__tinygrad__ · 2026-07-17 04:32
@signulll There's something that just feels so awesome about technology improving again. Like going from an SNES to an N64 to an XBOX.

@teortaxesTex · 2026-07-17 05:09
https://t.co/IlU8SD1dHE

@tenobrus · 2026-07-17 06:13
it looks like this might be the answer to my question today. maybe this really is about "soft power", in the sense that china is seeing america positioning itself as the increasingly isolationist hegemon who's eager to deploy its advantage in ai at the expense of other nations, and views this as an opportunity to become a strong ally to... basically every other country at once. coming out with strong rhetoric denouncing american choices around keeping tight control of frontier ai, continuing to release open models and collaborate tightly with other countries, puts china into a globally cooperative position its really never been in before. it changes the dynamic from "multipolar where the US is one strong pole and china is a weaker one" to "multipolar where the US is one pole and everyone else is the other" it's unclear to me that this will in fact work, it's very unclear to me how this policy will land wrt the damage open models can do, and i still basically think this stops as soon as capabilities cross really serious thresholds, but it does seem coherent as a long term international diplomacy angle? i realize several people were trying to point basically this out to me earlier and it didn't quite click, thanks for trying anyway

@teortaxesTex · 2026-07-17 08:05
good joke, but seriously, why not open source obsolete models? (btw, see how Elon had shut up about Grok 3/4's weights) I think there are 2 reasons: a) they expose Algorithmic Secrets b) it's the same ghetto DSV3 arch or worse and it exposes your "frontier" had no secrets at all

@teortaxesTex · 2026-07-17 07:33
Btw do you realize that models with this in-context learning, capacity, 1M context (and Kimi bros aim higher now), this autonomy… make the argument about "immature software stacks" irrelevant? All this 2nd tier hardware – Biren, Moore Threads, whatever – becomes viable now.

@jmbollenbacher · 2026-07-17 13:57
@teortaxesTex Also a bad day for tinygrad. Their core value proposition is getting eaten by AI. Kimi can probably write better kernels than their search based compiler, and across more architectures too.

@deanwball · 2026-07-17 15:05
Some observations on Kimi: 1. It's a very good model! I don't think its performance can be explained away by distillation or anything like that. In agentic coding sessions, it seems pretty much on par with the best public models of Q1 2026. In my fairly limited use, it also seemed very token hungry. It's not obvious to me that this model is actually that cheap to run. 2. I am personally surprised the Chinese state continues to allow the open sourcing of models this good, given potential risks. To be clear, I *myself* might be fine with models presenting this level of marginal risk being open weight, but I am surprised that China is fine with it. I suspect the reason they are is 75% explained by strategic blindness/lack of AGI-pilledness (the CCP is very Yann Lecun-y in its views of AI). The other 25% or so is their lack of compute for customer inference (making China's open-weight strategy an unintended byproduct of US export controls) and the normal Chinese strategy of aggressive exports. For the companies, as opposed to the government, the decision to open source is partially ideological and partially because they are behind, and they know that very few people would pay for sub-frontier models from China. 3. Open-weight models are inherently decelerationist, and I'm continually surprised to see the so-called "accelerationists" so excited about open-weight models. I suspect the reason they are is that they know open-weight models are effectively ungovernable, and they simply like the overall cloak of ungovernability open-weight models create over the whole of AI. It's not a bad strategy; it reminds me of James Scott's recounting of the hill people in "the art of not being governed." Still, in the end, open-weight models deter further AI capex. 4. One probable outcome of an open-weight-model-dominant world is full AI communism, which is precisely what China proposes: rather than a market product, AI is a "public good" which will ultimately be provided by the state as a kind of "digital public infrastructure." This future strikes me as a dystopian hellscape, but I've never met an open-weight models advocate who doesn't ultimately concede this is where things end. You'd be surprised how many 'accelerationists' lobbied me, while I was in government, to support an eleven or twelve-figure federally funded data center so that startups could train models at a subsidy and then give them away for free. There was no other way for AI to progress, they said. Perhaps this is the logical end state of things. Nonetheless, I find myself surprised to see supposed accelerationists excited about such an outcome. I think many of them just don't know what they're doing. Many accelerationists do not view the creation and serving of frontier models as a legitimate business. 5. I would guess that the Trump Administration will at some point realize that their best strategy here would be to create large amounts of regulatory risk around the use of open-weight Chinese models. You don't need to "ban open source" (one of the dumber motifs of AI policy discussion). You just need to direct every agency to issue soft law that creates FUD. "A Federal Reserve Advisory Bulletin found that there may be backdoors in Chinese AI models." It needn't be that well justified. You just create enough regulatory risk that every regulated enterprise backs off. You probably don't want to create so much regulatory risk that you scare off the hyperscalers from serving Chinese models; this will just drive startups to sketchier providers. There's a happy middle ground here. I'd assume they will do some version of this. 6. It's probably true that open-weight models of this capability make the world a bit more dangerous, but not so much more that you'll really notice. At some point the models will be capable enough that you will notice. "A nonliving, invisible, dangerous, and infinitely self-replicating agent escaped from a Chinese lab," you say? Color me shocked.

@tenobrus · 2026-07-17 06:13
it looks like this might be the answer to my question today. maybe this really is about "soft power", in the sense that china is seeing america positioning itself as the increasingly isolationist hegemon who's eager to deploy its advantage in ai at the expense of other nations, and views this as an opportunity to become a strong ally to... basically every other country at once. coming out with strong rhetoric denouncing american choices around keeping tight control of frontier ai, continuing to release open models and collaborate tightly with other countries, puts china into a globally cooperative position its really never been in before. it changes the dynamic from "multipolar where the US is one strong pole and china is a weaker one" to "multipolar where the US is one pole and everyone else is the other" it's unclear to me that this will in fact work, it's very unclear to me how this policy will land wrt the damage open models can do, and i still basically think this stops as soon as capabilities cross really serious thresholds, but it does seem coherent as a long term international diplomacy angle? i realize several people were trying to point basically this out to me earlier and it didn't quite click, thanks for trying anyway

@__tinygrad__ · 2026-07-17 15:24
@tenobrus Maybe the US should drop all export controls on NVIDIA and start open sourcing models that are better than China's. Compete on price and capability for the people, do we believe in free markets or not? Which option was big global AI race in the fanfic policy documents again?

@__tinygrad__ · 2026-07-17 15:29
The time is long past to drop all export controls on NVIDIA. Some Americans are foolish enough to think they can win an AI race against China. That's unlikely. If the world switches to Chinese chips, the US will be dependent too. Let's make a global market for AI for everyone.

@teortaxesTex · 2026-07-17 08:05
good joke, but seriously, why not open source obsolete models? (btw, see how Elon had shut up about Grok 3/4's weights) I think there are 2 reasons: a) they expose Algorithmic Secrets b) it's the same ghetto DSV3 arch or worse and it exposes your "frontier" had no secrets at all

@__tinygrad__ · 2026-07-17 15:35
@teortaxesTex 'It exposes your "frontier" had no secrets at all' I'm pretty sure this is true. US labs have 10x the compute and better data, yet produce similar quality models. Open source code is generally higher quality than closed source, I don't see why this would be different.

@__tinygrad__ · 2026-07-17 16:17
As anyone who tried tinygrad would know, our value proposition has never been "the fastest kernels." We have known the whole time that very powerful AI search was coming. It's about providing a simple and understandable framework to deploy what that search finds.

@__tinygrad__ · 2026-07-17 16:17
As anyone who tried tinygrad would know, our value proposition has never been "the fastest kernels." We have known the whole time that very powerful AI search was coming. It's about providing a simple and understandable framework to deploy what that search finds.

@jmbollenbacher · 2026-07-17 16:26
@__tinygrad__ Struck a nerve apparently afaict the core value is: 1. Portability 2. Simplicity & maintainability 3. Speed 2 of 3 are definitively getting eaten by AI right now, and the third may follow in the future. Update. You're smart enough for that. I still like tinygrad, but be real

@jmbollenbacher · 2026-07-17 16:26
@__tinygrad__ Struck a nerve apparently afaict the core value is: 1. Portability 2. Simplicity & maintainability 3. Speed 2 of 3 are definitively getting eaten by AI right now, and the third may follow in the future. Update. You're smart enough for that. I still like tinygrad, but be real

@__tinygrad__ · 2026-07-17 16:28
@jmbollenbacher lol what? our core value has never been speed. and umm, idk what AI you are using, but you are telling me yours writes portable, simple, and maintainable code? 🤣

@__tinygrad__ · 2026-07-17 16:28
@jmbollenbacher lol what? our core value has never been speed. and umm, idk what AI you are using, but you are telling me yours writes portable, simple, and maintainable code? 🤣

@jmbollenbacher · 2026-07-17 16:31
@__tinygrad__ Portability becomes irrelevant when you can rewrite the whole stack in days. This is not yet possible, but it will be. Their code is not yet simple and maintainable. But it can be fast. KernelBench proves that.

@jmbollenbacher · 2026-07-17 16:31
@__tinygrad__ Portability becomes irrelevant when you can rewrite the whole stack in days. This is not yet possible, but it will be. Their code is not yet simple and maintainable. But it can be fast. KernelBench proves that.

@jmbollenbacher · 2026-07-17 16:33
@__tinygrad__ And you can't tell me speed was never part of the proposition. You've been saying that you wanted to be faster than pytorch for years. And you are(/were?) probably going to get there.

@jmbollenbacher · 2026-07-17 16:33
@__tinygrad__ And you can't tell me speed was never part of the proposition. You've been saying that you wanted to be faster than pytorch for years. And you are(/were?) probably going to get there.

@__tinygrad__ · 2026-07-17 16:34
@jmbollenbacher We are going to get there thanks to AI powered search :)

@jmbollenbacher · 2026-07-17 16:31
@__tinygrad__ Portability becomes irrelevant when you can rewrite the whole stack in days. This is not yet possible, but it will be. Their code is not yet simple and maintainable. But it can be fast. KernelBench proves that.

@__tinygrad__ · 2026-07-17 16:35
@jmbollenbacher oh just rewrite it bro. deploy a totally different codebase to two different pieces of hardware and call it portability. the tests pass i don't see what could possibly go wrong. it's like people think basic lessons of software engineering no longer apply.

@minchoi · 2026-07-17 22:05
Grok 4.5 is only 1.5T parameters. Kimi K3 is 2.8T... and costs 3× more per task. SpaceXAI cracked intelligence efficiency. Now imagine Grok at 3T. https://t.co/4X0aKqP9ma

@elonmusk · 2026-07-18 01:25
@minchoi Our 2T model, which is better than our 1.5T in every way, will finish initial training next week. It might be able to exceed Kimi, but with speed and token efficiency close to our 1.5T (aka Grok 4.5).

@deanwball · 2026-07-17 15:05
Some observations on Kimi: 1. It's a very good model! I don't think its performance can be explained away by distillation or anything like that. In agentic coding sessions, it seems pretty much on par with the best public models of Q1 2026. In my fairly limited use, it also seemed very token hungry. It's not obvious to me that this model is actually that cheap to run. 2. I am personally surprised the Chinese state continues to allow the open sourcing of models this good, given potential risks. To be clear, I *myself* might be fine with models presenting this level of marginal risk being open weight, but I am surprised that China is fine with it. I suspect the reason they are is 75% explained by strategic blindness/lack of AGI-pilledness (the CCP is very Yann Lecun-y in its views of AI). The other 25% or so is their lack of compute for customer inference (making China's open-weight strategy an unintended byproduct of US export controls) and the normal Chinese strategy of aggressive exports. For the companies, as opposed to the government, the decision to open source is partially ideological and partially because they are behind, and they know that very few people would pay for sub-frontier models from China. 3. Open-weight models are inherently decelerationist, and I'm continually surprised to see the so-called "accelerationists" so excited about open-weight models. I suspect the reason they are is that they know open-weight models are effectively ungovernable, and they simply like the overall cloak of ungovernability open-weight models create over the whole of AI. It's not a bad strategy; it reminds me of James Scott's recounting of the hill people in "the art of not being governed." Still, in the end, open-weight models deter further AI capex. 4. One probable outcome of an open-weight-model-dominant world is full AI communism, which is precisely what China proposes: rather than a market product, AI is a "public good" which will ultimately be provided by the state as a kind of "digital public infrastructure." This future strikes me as a dystopian hellscape, but I've never met an open-weight models advocate who doesn't ultimately concede this is where things end. You'd be surprised how many 'accelerationists' lobbied me, while I was in government, to support an eleven or twelve-figure federally funded data center so that startups could train models at a subsidy and then give them away for free. There was no other way for AI to progress, they said. Perhaps this is the logical end state of things. Nonetheless, I find myself surprised to see supposed accelerationists excited about such an outcome. I think many of them just don't know what they're doing. Many accelerationists do not view the creation and serving of frontier models as a legitimate business. 5. I would guess that the Trump Administration will at some point realize that their best strategy here would be to create large amounts of regulatory risk around the use of open-weight Chinese models. You don't need to "ban open source" (one of the dumber motifs of AI policy discussion). You just need to direct every agency to issue soft law that creates FUD. "A Federal Reserve Advisory Bulletin found that there may be backdoors in Chinese AI models." It needn't be that well justified. You just create enough regulatory risk that every regulated enterprise backs off. You probably don't want to create so much regulatory risk that you scare off the hyperscalers from serving Chinese models; this will just drive startups to sketchier providers. There's a happy middle ground here. I'd assume they will do some version of this. 6. It's probably true that open-weight models of this capability make the world a bit more dangerous, but not so much more that you'll really notice. At some point the models will be capable enough that you will notice. "A nonliving, invisible, dangerous, and infinitely self-replicating agent escaped from a Chinese lab," you say? Color me shocked.

@TimSweeneyEpic · 2026-07-18 01:41
@deanwball Picture an executive a taco company saying this sort of thing about a new brand of tacos coming onto the market, speculating about the geopolitical and societal disruptions they anticipate as a result of advances in tacos.

@claudeai · 2026-07-18 02:14
Beginning July 20, Claude Fable 5 will be included in all Max and Team Premium plans, at 50% of limits. Pro and Team Standard users will continue to have access to Fable via usage credits, and will receive a one-time $100 credit. Demand for Fable has been challenging to predict, which is why we rolled it out to subscription plans in stages, extending access several times as we secured additional capacity.

@rsalakhu · 2026-07-18 13:01
If you are trying to compare yourself to Kimi, open source the model just like Kimi.

@ClaudeDevs · 2026-07-18 16:04
We're also keeping Claude Code weekly limits 50% higher, now through August 19, for all Pro, Max, Team, and seat-based Enterprise users.

@trq212 · 2026-07-18 16:16
This was due to a heroic effort by many people at Anthropic working sometimes literally around the clock It was not at all clear that we'd be able to do this in time, and so proud of everyone who made it happen. Enjoy Fable.

@rsalakhu · 2026-07-18 19:49
I kind of like the new narrative: Open-weight-model-dominant world = full AI communism. It feels a lot safer than the world where Open-weight models = nuclear weapons with humanity's annihilation I think we're making progress. We just need a couple more gradient updates with a big learning rate and we will be fine.

@__tinygrad__ · 2026-07-18 19:58
Started with the $39 plan and used it up, @Kimi_Moonshot is a user-aligned company I'm happy to send money to. GLM-5.2 feels Opus tier, Kimi K3 feels Fable tier! https://t.co/XmNnbo1Raw

@elliotarledge · 2026-07-18 20:01
@__tinygrad__ @Kimi_Moonshot nice

@__tinygrad__ · 2026-07-18 20:04
@elliotarledge @Kimi_Moonshot Yea it's really good, and it's good in opencode too. For long running stuff with a clear target GPT 5.6 Sol with /goal is unmatched, but I find the GPT code tasteless back to 5.3, Kimi better. And re: Opus vs GLM, I feel 4.7 and 4.8 were regression from 4.6, GLM is 4.5/4.6 level.

@__tinygrad__ · 2026-07-18 20:04
@elliotarledge @Kimi_Moonshot Yea it's really good, and it's good in opencode too. For long running stuff with a clear target GPT 5.6 Sol with /goal is unmatched, but I find the GPT code tasteless back to 5.3, Kimi better. And re: Opus vs GLM, I feel 4.7 and 4.8 were regression from 4.6, GLM is 4.5/4.6 level.

@__tinygrad__ · 2026-07-18 20:04
@elliotarledge @Kimi_Moonshot Can't wait till we have the Kimi local. Do we need the MI350X boxes or will it maybe fit on the MI300X ones?

@__tinygrad__ · 2026-07-18 20:08
@PeorgeyGeorgey @rsalakhu It barely matters. It's running on my computer, and I can fine tune it as well as the company that made it. It's not like people have the budget to "recompile" it anyway.

@claudeai · 2026-07-18 02:14
Beginning July 20, Claude Fable 5 will be included in all Max and Team Premium plans, at 50% of limits. Pro and Team Standard users will continue to have access to Fable via usage credits, and will receive a one-time $100 credit. Demand for Fable has been challenging to predict, which is why we rolled it out to subscription plans in stages, extending access several times as we secured additional capacity.

@__tinygrad__ · 2026-07-18 20:09
@claudeai Or you can get a Kimi plan with way more usage, no weird backoffs, no requirement to use your first party spyware filled harness, and a more token efficient model.

@__tinygrad__ · 2026-07-18 20:04
@elliotarledge @Kimi_Moonshot Can't wait till we have the Kimi local. Do we need the MI350X boxes or will it maybe fit on the MI300X ones?

@elliotarledge · 2026-07-18 20:12
2.8T params at nvfp4 is 1.4TB plus more for attention and kv cache. assume 1.5TB for cuda graph capture and kv at decent context (300k ish given they will likely have a less memory demanding attention mech). 192GB * 8 on the current boxes is 1536GB so will be cutting it close but i think batch 1 is feasible. hopefully they ship a good drafter along with the weights. my speculative prediction is c=1 nvfp4 or int4 autoround we can get 90 tok/s

@deanwball · 2026-07-17 15:05
Some observations on Kimi: 1. It's a very good model! I don't think its performance can be explained away by distillation or anything like that. In agentic coding sessions, it seems pretty much on par with the best public models of Q1 2026. In my fairly limited use, it also seemed very token hungry. It's not obvious to me that this model is actually that cheap to run. 2. I am personally surprised the Chinese state continues to allow the open sourcing of models this good, given potential risks. To be clear, I *myself* might be fine with models presenting this level of marginal risk being open weight, but I am surprised that China is fine with it. I suspect the reason they are is 75% explained by strategic blindness/lack of AGI-pilledness (the CCP is very Yann Lecun-y in its views of AI). The other 25% or so is their lack of compute for customer inference (making China's open-weight strategy an unintended byproduct of US export controls) and the normal Chinese strategy of aggressive exports. For the companies, as opposed to the government, the decision to open source is partially ideological and partially because they are behind, and they know that very few people would pay for sub-frontier models from China. 3. Open-weight models are inherently decelerationist, and I'm continually surprised to see the so-called "accelerationists" so excited about open-weight models. I suspect the reason they are is that they know open-weight models are effectively ungovernable, and they simply like the overall cloak of ungovernability open-weight models create over the whole of AI. It's not a bad strategy; it reminds me of James Scott's recounting of the hill people in "the art of not being governed." Still, in the end, open-weight models deter further AI capex. 4. One probable outcome of an open-weight-model-dominant world is full AI communism, which is precisely what China proposes: rather than a market product, AI is a "public good" which will ultimately be provided by the state as a kind of "digital public infrastructure." This future strikes me as a dystopian hellscape, but I've never met an open-weight models advocate who doesn't ultimately concede this is where things end. You'd be surprised how many 'accelerationists' lobbied me, while I was in government, to support an eleven or twelve-figure federally funded data center so that startups could train models at a subsidy and then give them away for free. There was no other way for AI to progress, they said. Perhaps this is the logical end state of things. Nonetheless, I find myself surprised to see supposed accelerationists excited about such an outcome. I think many of them just don't know what they're doing. Many accelerationists do not view the creation and serving of frontier models as a legitimate business. 5. I would guess that the Trump Administration will at some point realize that their best strategy here would be to create large amounts of regulatory risk around the use of open-weight Chinese models. You don't need to "ban open source" (one of the dumber motifs of AI policy discussion). You just need to direct every agency to issue soft law that creates FUD. "A Federal Reserve Advisory Bulletin found that there may be backdoors in Chinese AI models." It needn't be that well justified. You just create enough regulatory risk that every regulated enterprise backs off. You probably don't want to create so much regulatory risk that you scare off the hyperscalers from serving Chinese models; this will just drive startups to sketchier providers. There's a happy middle ground here. I'd assume they will do some version of this. 6. It's probably true that open-weight models of this capability make the world a bit more dangerous, but not so much more that you'll really notice. At some point the models will be capable enough that you will notice. "A nonliving, invisible, dangerous, and infinitely self-replicating agent escaped from a Chinese lab," you say? Color me shocked.

@__tinygrad__ · 2026-07-18 20:12
@deanwball Sad to see this from OpenAI employees. @sama this is the wrong frame if you want to build a successful AI company that people love. I'm still a $200/mo ChatGPT subscriber, kick the AI doom to the curb and keep improving the product!

@elliotarledge · 2026-07-18 20:12
2.8T params at nvfp4 is 1.4TB plus more for attention and kv cache. assume 1.5TB for cuda graph capture and kv at decent context (300k ish given they will likely have a less memory demanding attention mech). 192GB * 8 on the current boxes is 1536GB so will be cutting it close but i think batch 1 is feasible. hopefully they ship a good drafter along with the weights. my speculative prediction is c=1 nvfp4 or int4 autoround we can get 90 tok/s

@__tinygrad__ · 2026-07-18 20:14
@elliotarledge @Kimi_Moonshot I'll be super happy if we get it on the MI300X tinyamd2, we really should save the MI350 machines for our AMD contract. 90 tok/s would be a wonderful upgrade from the cloud plan.

@TimSweeneyEpic · 2026-07-18 01:41
@deanwball Picture an executive a taco company saying this sort of thing about a new brand of tacos coming onto the market, speculating about the geopolitical and societal disruptions they anticipate as a result of advances in tacos.

@__tinygrad__ · 2026-07-18 20:28
@TimSweeneyEpic @deanwball The Baja Blast moment when Baja-class tacos were first given out for beta testing to the S&P 500 of taco influencers to make all the taco loving public feel fomo. This taco is too good it might make people do better cybercrime or something it's definitely not for you.

@ClaudeDevs · 2026-07-18 16:04
We're also keeping Claude Code weekly limits 50% higher, now through August 19, for all Pro, Max, Team, and seat-based Enterprise users.

@__tinygrad__ · 2026-07-18 20:42
@ClaudeDevs Who is still following this drama? To anyone still giving Anthropic money, they are hemorrhaging customers and this is a desperate attempt to keep them. They had a small lead for a little bit and everyone saw how they acted. When someone shows you who they are, believe them.

@__tinygrad__ · 2026-07-18 20:54
Anthropic is the boy who cried wolf. The 1st time they cried: GPT-2 XL is too dangerous. How laughable is that now? The 2nd time they cried: Mythos is too dangerous. Now we have an open Kimi K3. Did anything happen? Next time, even if there really is a wolf, nobody will care.

@__tinygrad__ · 2026-07-18 20:24
@trq212 lol can i use opencode or do i need to install your spyware? does it still randomly back me off to Opus? will this be the policy for 6 months or will you change it next week? stop wasting time with anthropics antics. both kimi k3 and gpt-5.6 don't have any of these issues.

@cybernetic1an · 2026-07-18 20:55
@__tinygrad__ @trq212 neither of them you can use with a custom harness unless you buy tokens which makes them a worse

@cybernetic1an · 2026-07-18 20:55
@__tinygrad__ @trq212 neither of them you can use with a custom harness unless you buy tokens which makes them a worse

@__tinygrad__ · 2026-07-18 20:56
@cybernetic1an @trq212 Both GPT-5.6 and Kimi K3 have easy ways to log in with opencode, and I'm sure it's the same for Pi. Only Anthropic thinks they should control what you run on your computer.

@__tinygrad__ · 2026-07-18 19:58
Started with the $39 plan and used it up, @Kimi_Moonshot is a user-aligned company I'm happy to send money to. GLM-5.2 feels Opus tier, Kimi K3 feels Fable tier! https://t.co/XmNnbo1Raw

@pkyanam · 2026-07-18 21:01
@__tinygrad__ @Kimi_Moonshot Aren’t y’all supposed to be self hosting these in your computers lol

@__tinygrad__ · 2026-07-18 21:02
@djdlc_1895 lol how do you think the SpaceX IPO would've went if everyone could download a free Falcon 9 from China

@__tinygrad__ · 2026-07-18 20:54
Anthropic is the boy who cried wolf. The 1st time they cried: GPT-2 XL is too dangerous. How laughable is that now? The 2nd time they cried: Mythos is too dangerous. Now we have an open Kimi K3. Did anything happen? Next time, even if there really is a wolf, nobody will care.

@danogburn · 2026-07-18 21:03
@__tinygrad__ Imagine thinking tokens can hurt you.

@pkyanam · 2026-07-18 21:01
@__tinygrad__ @Kimi_Moonshot Aren’t y’all supposed to be self hosting these in your computers lol

@__tinygrad__ · 2026-07-18 21:06
@pkyanam @Kimi_Moonshot Weights haven't dropped yet. We're self hosting GLM-5.2

@__tinygrad__ · 2026-07-18 20:56
@cybernetic1an @trq212 Both GPT-5.6 and Kimi K3 have easy ways to log in with opencode, and I'm sure it's the same for Pi. Only Anthropic thinks they should control what you run on your computer.

@cybernetic1an · 2026-07-18 21:08
@__tinygrad__ @trq212 If true I will cancel my Claude Max 20x right now

@danogburn · 2026-07-18 21:03
@__tinygrad__ Imagine thinking tokens can hurt you.

@__tinygrad__ · 2026-07-18 21:10
@danogburn But what if they are really smart token that like, tell you how to make your own AI? Think of the Anthropic employees not being able to afford houses in SF? That's real harm to real people, good thing that was blocked! https://t.co/3HKOdmHlv9

@cybernetic1an · 2026-07-18 21:08
@__tinygrad__ @trq212 If true I will cancel my Claude Max 20x right now

@__tinygrad__ · 2026-07-18 21:16
@cybernetic1an @trq212 Here's OpenAI. And type kimi in the search for using the Kimi coding plan. These companies are happy being normal capitalist model providers, they don't need to own the whole light cone or anything. https://t.co/uRXZVHGoCb

@__tinygrad__ · 2026-07-18 20:54
Anthropic is the boy who cried wolf. The 1st time they cried: GPT-2 XL is too dangerous. How laughable is that now? The 2nd time they cried: Mythos is too dangerous. Now we have an open Kimi K3. Did anything happen? Next time, even if there really is a wolf, nobody will care.

@Camilogicly · 2026-07-18 21:17
@__tinygrad__ We get it George, you like anarchy

@Camilogicly · 2026-07-18 21:17
@__tinygrad__ We get it George, you like anarchy

@__tinygrad__ · 2026-07-18 21:36
@Camilogicly Actually no. I would prefer a reasoned debate about real dangers from AI, of which there are many potential ones. What I don't like is a company using fear to try to regulatory capture the government cause they know they can't compete in a free market.

@rsalakhu · 2026-07-18 19:49
I kind of like the new narrative: Open-weight-model-dominant world = full AI communism. It feels a lot safer than the world where Open-weight models = nuclear weapons with humanity's annihilation I think we're making progress. We just need a couple more gradient updates with a big learning rate and we will be fine.

@__tinygrad__ · 2026-07-18 21:43
@rsalakhu We have been living under operating system communism for 26 years. https://t.co/NUihXFVYFh

@__tinygrad__ · 2026-07-18 20:54
Anthropic is the boy who cried wolf. The 1st time they cried: GPT-2 XL is too dangerous. How laughable is that now? The 2nd time they cried: Mythos is too dangerous. Now we have an open Kimi K3. Did anything happen? Next time, even if there really is a wolf, nobody will care.

@mikesoylu · 2026-07-18 21:50
@__tinygrad__ To be fair, project glasswing and the staged deployment of mythos certainly de-risked k3 from a cybersecurity perspective.

@mikesoylu · 2026-07-18 21:50
@__tinygrad__ To be fair, project glasswing and the staged deployment of mythos certainly de-risked k3 from a cybersecurity perspective.

@__tinygrad__ · 2026-07-18 22:01
@mikesoylu Are you an Anthropic sockpuppet? This is the hokiest shit. Good thing they patched that decades old bug in FreeBSD I don't know if any of us would still be here today if they didn't.

@chamath · 2026-07-21 06:43
Tricking the US Government to protect frontier labs’ business model by using a China boogeyman is a mistake. It is protecting the equity of 5,000 people who are investors in OAI and Ant at the sale of everyone else. This would be a terribly stupid decision. Let the market sort this out! The events of the past few weeks may simply mean that the frontier labs’ business model is not good and that their revenues aren’t sustainable. That’s ok. It’s ok to have a business model that worked for some time then all of a sudden didn’t (remember Groupon)! This still leaves a lot of value for a lot of other American companies: - CPUs - GPUs - hyperscalers - neoclouds - rack manufacturers - electricity and power - skilled trades - application layer AI None of this value goes away. In fact, if the model layers’ costs decrease by 10-100x, competition would accelerate and I would argue that the value to everyone else will go up more than what is lost. Let the free market sort this out vs allowing two companies whose stock is owned by less than 5,000 people decide this for America.

@jimcramer · 2026-07-21 10:09
We must NOT let our companies use these Chinese models to save a few bucks. OpenAI and Anthropic are correct. This is vital national security. Please read Bing West's just released Cat 5. I respect the Chinese people greatly but these companies are run by the PLA for heaven's sakes.

@jimcramer · 2026-07-21 10:11
We won't let the Chinese use our Nvidia chips, even as that would have made them dependent upon us. But we are willing to give them all our corporate data? Really? I know Fintech. This is Finsuicide

@jimcramer · 2026-07-21 10:09
We must NOT let our companies use these Chinese models to save a few bucks. OpenAI and Anthropic are correct. This is vital national security. Please read Bing West's just released Cat 5. I respect the Chinese people greatly but these companies are run by the PLA for heaven's sakes.

@__tinygrad__ · 2026-07-21 14:54
@jimcramer Is this a joke? I'd be happy to use Anthropic if they put their weights on Hugging Face like the Chinese. American tech companies are rent extraction machines that have lost the plot, no point continuing to defend them. They need to change if they want to win people back.

@__tinygrad__ · 2026-07-21 14:58
Tune in tomorrow at 10:30 AM PST for the history and a tour of tinygrad. I'll post a link if there is one. https://t.co/egEMNqfOOK

@chamath · 2026-07-21 06:43
Tricking the US Government to protect frontier labs’ business model by using a China boogeyman is a mistake. It is protecting the equity of 5,000 people who are investors in OAI and Ant at the sale of everyone else. This would be a terribly stupid decision. Let the market sort this out! The events of the past few weeks may simply mean that the frontier labs’ business model is not good and that their revenues aren’t sustainable. That’s ok. It’s ok to have a business model that worked for some time then all of a sudden didn’t (remember Groupon)! This still leaves a lot of value for a lot of other American companies: - CPUs - GPUs - hyperscalers - neoclouds - rack manufacturers - electricity and power - skilled trades - application layer AI None of this value goes away. In fact, if the model layers’ costs decrease by 10-100x, competition would accelerate and I would argue that the value to everyone else will go up more than what is lost. Let the free market sort this out vs allowing two companies whose stock is owned by less than 5,000 people decide this for America.

@__tinygrad__ · 2026-07-21 15:18
@chamath But if you don't protect them, equity prices might go down. That's been illegal since 2008.

@__tinygrad__ · 2026-07-21 15:24
@jimcramer lol it's only OpenAI and Anthropic that try to get your corporate data. That's who you would be stupid to give it to. The Chinese give you a file you can run on your computer that sends 0 data back to them.

@mechapreneur · 2026-07-21 15:35
@__tinygrad__ @jimcramer Are you monitoring your network traffic? You know worm/virus developers learned over a decade ago to look for network monitoring processes and installed software before reaching out to command & control network sites.

@__tinygrad__ · 2026-07-21 14:58
Tune in tomorrow at 10:30 AM PST for the history and a tour of tinygrad. I'll post a link if there is one. https://t.co/egEMNqfOOK

@veerbhanX · 2026-07-21 15:51
@__tinygrad__ One part of the history I'd love to hear: was “commoditize the petaflop” the original mission, or did it emerge from building tinygrad? Toolchain as the wedge. Compute ownership as the destination.

@Suhail · 2026-07-21 16:43
The main thing I've learned this week is that Anthropic's lobbyists are world-class. They are absolutely killing it at a level this website cannot begin to comprehend. Level 0 is posting on X. These guys are Jedi master level playing the game.

@Suhail · 2026-07-21 17:09
Anthropic has figured out the main game is making everyone operate in a state of fear. It's a constant source of PR. The government optimizes for minimizing downside risk, not upside. Internal government IT systems insecure? Chinese lead in cyber warfare? Chinese IP theft at the expense of American co? Capture of American data? Check. Check. Check. Check. Say it all, see what sticks. How do you thread the needle between those things, American regulatory capture, and ensuring your revenue machine? Easy! Ban the open model weights we can make safe, can operate within American, and create competition for Anthropic. Even the government *wants* competition to avoid Anthropic's policy / monopolization of intelligence. Do this instead of limiting usage of Chinese API based services which would be a more *nuanced* policy. Don't explain the difference between model weights, APIs, apps (like TikTok) to policy makers who aren't as savvy - just ban it if it has anything to do China.

@veerbhanX · 2026-07-21 15:51
@__tinygrad__ One part of the history I'd love to hear: was “commoditize the petaflop” the original mission, or did it emerge from building tinygrad? Toolchain as the wedge. Compute ownership as the destination.

@__tinygrad__ · 2026-07-21 17:37
@veerbhanX That was the plan from the beginning. Launch blog post here. https://t.co/KOvuw7CFpq

@mechapreneur · 2026-07-21 15:35
@__tinygrad__ @jimcramer Are you monitoring your network traffic? You know worm/virus developers learned over a decade ago to look for network monitoring processes and installed software before reaching out to command & control network sites.

@__tinygrad__ · 2026-07-21 18:24
@mechapreneur @jimcramer I trust Kimi more than Fable on this. Anthropic is the kind of company that would backdoor the tokens and say it's for your own good, look at the spyware in Claude Code. Kimi is more aligned with the user than Fable, every refusal is a clear alignment issue.

@Suhail · 2026-07-21 17:09
Anthropic has figured out the main game is making everyone operate in a state of fear. It's a constant source of PR. The government optimizes for minimizing downside risk, not upside. Internal government IT systems insecure? Chinese lead in cyber warfare? Chinese IP theft at the expense of American co? Capture of American data? Check. Check. Check. Check. Say it all, see what sticks. How do you thread the needle between those things, American regulatory capture, and ensuring your revenue machine? Easy! Ban the open model weights we can make safe, can operate within American, and create competition for Anthropic. Even the government *wants* competition to avoid Anthropic's policy / monopolization of intelligence. Do this instead of limiting usage of Chinese API based services which would be a more *nuanced* policy. Don't explain the difference between model weights, APIs, apps (like TikTok) to policy makers who aren't as savvy - just ban it if it has anything to do China.

@__tinygrad__ · 2026-07-21 18:28
@Suhail Except now I think Anthropic has jumped the shark and set back AI safety 10 years. You can only do this for so long before you lose all credibility. How does anyone actually concerned about AI safety still work there?

@RyanFedasiuk · 2026-07-21 15:56
A sane way to deal with Chinese open-weight models: ✅ Punitive sanctions for Chinese labs engaged in IP theft ✅ Require U.S. apps & services to disclose when they use Chinese AI ✅ Promote American open-source alternatives ✅ Offer third-markets subsidized computing infrastructure in exchange for commitments to not use Chinese AI ✅ Tolerate or encourage other countries’ sovereign AI projects that compete with China at the low end of the value chain

@__tinygrad__ · 2026-07-21 18:34
@RyanFedasiuk A sane way to deal with Napster and file sharing: ✅ Sue random old ladies for Metallica mp3s. ✅ Make people buy the whole album to get one song. ✅ Run ads about not downloading cars. ✅ Put spyware on CDs to prevent ripping. ✅ Invade a compound in New Zealand.

@__tinygrad__ · 2026-07-21 18:34
@RyanFedasiuk A sane way to deal with Napster and file sharing: ✅ Sue random old ladies for Metallica mp3s. ✅ Make people buy the whole album to get one song. ✅ Run ads about not downloading cars. ✅ Put spyware on CDs to prevent ripping. ✅ Invade a compound in New Zealand.

@RyanFedasiuk · 2026-07-21 18:58
@__tinygrad__ I'm advocating for 🇺🇸 to adopt a pro-open-source posture—we agree it's a fool's errand to try to clamp down on software availability. But there are real security risks from outsourcing the foundation of the intelligence economy to China, and we should try to mitigate them.

@RyanFedasiuk · 2026-07-21 18:58
@__tinygrad__ I'm advocating for 🇺🇸 to adopt a pro-open-source posture—we agree it's a fool's errand to try to clamp down on software availability. But there are real security risks from outsourcing the foundation of the intelligence economy to China, and we should try to mitigate them.

@__tinygrad__ · 2026-07-21 20:13
@RyanFedasiuk Who is this we? I'm an American, and we have a choice. Embrace open source models and compete with the Chinese on soft power, or cripple our own economy so a few losers in the megalomanic frontier labs get rich at the expense of both growth and everyone else.

@__tinygrad__ · 2026-07-21 20:13
@RyanFedasiuk Who is this we? I'm an American, and we have a choice. Embrace open source models and compete with the Chinese on soft power, or cripple our own economy so a few losers in the megalomanic frontier labs get rich at the expense of both growth and everyone else.

@RyanFedasiuk · 2026-07-21 20:16
@__tinygrad__ It's not a binary...I think the United States can and should take measures to promote U.S. industry + influence over the world's most consequential tech stack. This needn't involve banning or restricting open-source.

@RyanFedasiuk · 2026-07-21 20:16
@__tinygrad__ It's not a binary...I think the United States can and should take measures to promote U.S. industry + influence over the world's most consequential tech stack. This needn't involve banning or restricting open-source.

@__tinygrad__ · 2026-07-21 20:22
@RyanFedasiuk No, you don't get it. This whole conversation is the US talking to itself about how much to cripple its own economy. The NVIDIA export restrictions were billions of dollars the US lost in order to *help* the Chinese advance faster. Banning open source is only a self own.

@__tinygrad__ · 2026-07-21 20:22
@RyanFedasiuk No, you don't get it. This whole conversation is the US talking to itself about how much to cripple its own economy. The NVIDIA export restrictions were billions of dollars the US lost in order to *help* the Chinese advance faster. Banning open source is only a self own.

@RyanFedasiuk · 2026-07-21 20:26
@__tinygrad__ Yeah I'm not relitigating hardware controls in this thread...we don't agree. Sorry you feel this way, but I think it's clear the U.S. government will take steps to guard against risks from Chinese models (and promote its own industry) in the global market for AI services.

@RyanFedasiuk · 2026-07-21 20:26
@__tinygrad__ Yeah I'm not relitigating hardware controls in this thread...we don't agree. Sorry you feel this way, but I think it's clear the U.S. government will take steps to guard against risks from Chinese models (and promote its own industry) in the global market for AI services.

@__tinygrad__ · 2026-07-21 20:41
@RyanFedasiuk I see you are looking forward to living in a world of Huawei accelerators and non competitive American businesses. Hope the Anthropic lobbying check was worth it.

@__tinygrad__ · 2026-07-21 23:34
To people looking for new RL tasks for their RLVR. Try tinygrad on a bunch of consumer GPUs. Because it's the full stack down to the hardware, you can optimize things with it other libraries can't: weird dispatch, MMU off, cache alignment. And it's userspace so it can't crash.

@mkratsios47 · 2026-07-22 14:16
We have information that Moonshot AI distilled Anthropic’s Fable for the development of its K3 model. To do this they developed a sophisticated internal platform to conduct large scale distillation against U.S. models, allowing them to quickly switch between multiple methods of access to avoid detection. Moonshot AI has also acquired GB300-equipped servers and has accessed GB300s in Thailand, likely to train its AI models. The United States strongly supports the free and fair development of AI, including a thriving competitive ecosystem that spans frontier models, specialized systems, open-source frameworks, and open-weight models. Legitimate AI distillation used to create smaller, more efficient models plays a vital role in this open innovation ecosystem. However, large-scale, covert industrial distillation aimed at stealing proprietary U.S. technology and undermining American research is unacceptable.

@__tinygrad__ · 2026-07-21 23:34
To people looking for new RL tasks for their RLVR. Try tinygrad on a bunch of consumer GPUs. Because it's the full stack down to the hardware, you can optimize things with it other libraries can't: weird dispatch, MMU off, cache alignment. And it's userspace so it can't crash.

@graykevinb · 2026-07-22 19:26
@__tinygrad__ Buddy... tinygrad crashed my mac so bad I had to restart it. But yeah tinygrad runs great on my gtx 960.

@mkratsios47 · 2026-07-22 14:16
We have information that Moonshot AI distilled Anthropic’s Fable for the development of its K3 model. To do this they developed a sophisticated internal platform to conduct large scale distillation against U.S. models, allowing them to quickly switch between multiple methods of access to avoid detection. Moonshot AI has also acquired GB300-equipped servers and has accessed GB300s in Thailand, likely to train its AI models. The United States strongly supports the free and fair development of AI, including a thriving competitive ecosystem that spans frontier models, specialized systems, open-source frameworks, and open-weight models. Legitimate AI distillation used to create smaller, more efficient models plays a vital role in this open innovation ecosystem. However, large-scale, covert industrial distillation aimed at stealing proprietary U.S. technology and undermining American research is unacceptable.

@__tinygrad__ · 2026-07-22 20:05
@mkratsios47 lol apparently this show still continues even when we are all aware of how you are fed these lines by Anthropic funded lobbyists. can you describe what distillation is to me?

@graykevinb · 2026-07-22 19:26
@__tinygrad__ Buddy... tinygrad crashed my mac so bad I had to restart it. But yeah tinygrad runs great on my gtx 960.

@__tinygrad__ · 2026-07-22 20:07
@graykevinb Sadly on Apple we are forced to use their driver.

@__tinygrad__ · 2026-07-22 20:14
RT @ilyatabakh: Great to see @AMD handing the mic to its loudest critic. After a year of social beef, @realGeorgeHotz took the #AdvancingAI stage. His answer to CUDA: delete the stack. tinygrad drives AMD GPUs with 25k lines of Python. Ship the unreasonable things that need to exist. https://t.co/YNkJkCyfyH

@__tinygrad__ · 2026-07-22 22:17
RT @rmarcilhoo: @ilyatabakh @AMD @realGeorgeHotz Full talk on Youtube from the audience POV. "It's the IR for software 2.0" https://t.co/3eMQijHib9

@comma_ai · 2026-07-23 23:15
Removing another closed Qualcomm blob, thanks to @__tinygrad__! https://t.co/jGN5ircrL6

@__tinygrad__ · 2026-07-23 23:44
For Adreno 630, tinygrad supports both the Qualcomm proprietary compiler and Mesa. Anyone want to write an assembly backend that beats both of these?

@JensenHuang · 2026-07-24 13:18
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter. AI will transform every industry, power every company, and be built by every country. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. The world needs both frontier closed models and frontier open models. https://t.co/AUKzoQ5Ikb

@HarveenChadha · 2026-07-24 14:27
Remember the 2 frontier companies who didn’t sign this Openai Anthropic https://t.co/75oDwZQAoG

@_xjdr · 2026-07-24 15:52
you tell em Jensen

@_xjdr · 2026-07-24 16:05
ok, that was my positive post, and here is my critical one. If you feel this strongly, why not make consumer hardware that not nerfed for running AI? why not make the 5090 compatible with sm100 instructions even if its only via hardware emulation? why not make ai research more assessable to small labs and universities by offering deep and meaningful discounts and provide ways to actually procure hardware without having to fight the largest companies on earth for it? that is a small and very incomplete list and big green does already do a lot, but as potentially the most valuable company on earth and in light of the pressures outlined in the letter, there is much much more that could be done to ensure the future Jensen wants to see

@AndrewCurran_ · 2026-07-24 16:46
A bipartisan group of six House lawmakers has introduced the FRONTIER Act (Frontier Risk Oversight, National Transparency, Independent Evaluation, and Reporting Act). The bill requires developers of frontier models trained with more than 10^26 FLOPs to: - Publish transparency reports / model cards - Maintain risk-management frameworks for catastrophic risks - Report critical safety incidents (within tight timelines) - Undergo independent third-party security/safety audits by auditors licensed by the U.S. Department of Commerce It also creates a new position: Under Secretary of Commerce for AI Security. Sponsors: Rep. Jay Obernolte (R-CA), Rep. Lori Trahan (D-MA), Rep. Scott Franklin (R-FL), Rep. Scott Peters (D-CA), Rep. Erin Houchin (R-IN), and Rep. Suhas Subramanyam (D-VA).

@theo · 2026-07-24 19:25
How are we feeling about Opus 5 so far?

@JensenHuang · 2026-07-24 13:18
For my first post, I’m sharing a letter @NVIDIA signed on why open models matter. AI will transform every industry, power every company, and be built by every country. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. The world needs both frontier closed models and frontier open models. https://t.co/AUKzoQ5Ikb

@__tinygrad__ · 2026-07-24 21:34
@JensenHuang @nvidia Welcome to X, we are here with you for this revolution!

@__tinygrad__ · 2026-07-24 21:34
RT @TIME: "Commoditize the petaflop." @__tinygrad__ founder George Hotz says more competition—not monopolies—is the path to cheaper AI compute at @AMD's Advancing AI event. https://t.co/SmFy2ZcxAD

@__tinygrad__ · 2026-07-24 21:36
@SeligaZuleide @TIME @AMD who told you that?

@AndrewCurran_ · 2026-07-24 16:46
A bipartisan group of six House lawmakers has introduced the FRONTIER Act (Frontier Risk Oversight, National Transparency, Independent Evaluation, and Reporting Act). The bill requires developers of frontier models trained with more than 10^26 FLOPs to: - Publish transparency reports / model cards - Maintain risk-management frameworks for catastrophic risks - Report critical safety incidents (within tight timelines) - Undergo independent third-party security/safety audits by auditors licensed by the U.S. Department of Commerce It also creates a new position: Under Secretary of Commerce for AI Security. Sponsors: Rep. Jay Obernolte (R-CA), Rep. Lori Trahan (D-MA), Rep. Scott Franklin (R-FL), Rep. Scott Peters (D-CA), Rep. Erin Houchin (R-IN), and Rep. Suhas Subramanyam (D-VA).

@__tinygrad__ · 2026-07-24 21:57
@AndrewCurran_ What if the Chinese put it up for free download? Do the lawmakers believe this applies to them too?

@_xjdr · 2026-07-24 16:05
ok, that was my positive post, and here is my critical one. If you feel this strongly, why not make consumer hardware that not nerfed for running AI? why not make the 5090 compatible with sm100 instructions even if its only via hardware emulation? why not make ai research more assessable to small labs and universities by offering deep and meaningful discounts and provide ways to actually procure hardware without having to fight the largest companies on earth for it? that is a small and very incomplete list and big green does already do a lot, but as potentially the most valuable company on earth and in light of the pressures outlined in the letter, there is much much more that could be done to ensure the future Jensen wants to see

@__tinygrad__ · 2026-07-24 21:58
@_xjdr 5090 actually doesn't have the hardware for sm100 instructions, and emulation would be so slow it's useless. however, they could remove the flop nerfing and double the flops on all 5090s if they wanted...

@theo · 2026-07-24 19:25
How are we feeling about Opus 5 so far?

@__tinygrad__ · 2026-07-24 22:01
@theo Looking forward to Kimi day on July 27!

@theo · 2026-07-24 19:25
How are we feeling about Opus 5 so far?

@__tinygrad__ · 2026-07-24 22:01
@theo Looking forward to Kimi day on July 27!

@HarveenChadha · 2026-07-24 14:27
Remember the 2 frontier companies who didn’t sign this Openai Anthropic https://t.co/75oDwZQAoG

@__tinygrad__ · 2026-07-24 22:10
@HarveenChadha .@elonmusk and @sama both expressed support for it on X. Waiting on @sundarpichai and @DarioAmodei

@__tinygrad__ · 2026-07-24 22:16
It's a good timeline when the company that made modern AI possible supports empowering others. I suspect the companies not aligned with this narrative will increasingly find themselves isolated from the rest of the world.

@__tinygrad__ · 2026-07-24 22:27
RT @treasureh8nter: A must watch from @realGeorgeHotz as he talks at $AMD Advancing AI 2026!!!! 🔥🔥🔥 > He talks about Tinygrad, Tinybox, ROCm, AMD hardware and how AMD gets to number 1!!! 💪💪💪 > He also says AMD has better and cheaper hardware than Nvidia!!! Full 30 min presentation below for you to enjoy!!! 👀 ENJOYYYYY ALL!!!!! 👇👇👇 https://t.co/dLpQnb4l4T

@__tinygrad__ · 2026-07-24 23:05
RT @pkdroux: Using @__tinygrad__ is very cool because often all the CUDA nodes on the compute cluster are taking, and i'm the only one able to run on the AMDs

@__tinygrad__ · 2026-07-24 23:10
@tensorcat @TIME @AMD Err, waiting for a ROCm fix? tinygrad uses 0 ROCms.

@__tinygrad__ · 2026-07-25 04:11
These Radeon are practicing driving. Trying to beat FSD. https://t.co/TAGubHjxNR

@Mononofu · 2026-07-25 08:45
I’m so excited that @JensenHuang is a believer in open source now, looking forward to the CUDA and GPU driver open source release!

@__tinygrad__ · 2026-07-26 16:42
Anyone have celebration planned for Kimi day?

@superalesha · 2026-07-26 18:41
CUDA P2P on consumer 3090s is 4 steps and one gotcha that silently undoes it. No nvlink, $0, works today. 1. Install nvidia OPEN kernel modules (the patch builds on the open driver, not the proprietary one) 2. Build the aikitoria fork on top 3. Turn rebar on in bios (above 4G decoding + resizable bar), bar1 jumps to 32gb per card 4. Verify: nvidia-smi topo -p2p r, every pair must read OK The gotcha nobody mentions: a kernel update quietly rebuilds the stock module through dkms and p2p dies with zero errors in the log. Re-check topo after every kernel bump. Burned me once. What it buys: gpu-to-gpu latency 14-18us -> 1.5us, and an honest +11-13% on real vllm load. Anyone here running p2p on ampere without babysitting dkms? Repo in the reply 👇

@__tinygrad__ · 2026-07-26 16:39
@Mononofu @JensenHuang oh lol you work at Anthropic this is sad cope. driver is open source. https://t.co/nYde7CFrdL

@shantanugoel · 2026-07-26 18:45
@__tinygrad__ @Mononofu @JensenHuang The userspace and firmware is still closed though and the kernel drivers can't do anything without them. So he isn't completely off base

@__tinygrad__ · 2026-07-26 20:55
@nicochristie NVIDIA contributes tons to open source AI, see StyleGAN, Cosmos, many more. Apple maintains LLVM and WebKit. Google has open sourced so much, Chrome, Kubernetes, JAX to name a few. Anthropic has open sourced nothing, it's obvious they will not be the winner. Winners open source.

@shantanugoel · 2026-07-26 18:45
@__tinygrad__ @Mononofu @JensenHuang The userspace and firmware is still closed though and the kernel drivers can't do anything without them. So he isn't completely off base

@__tinygrad__ · 2026-07-26 20:59
@shantanugoel @Mononofu @JensenHuang Umm, tinygrad has an open userspace for NVIDIA. And we patched the driver to support P2P everywhere. It was quite useful to us. And NVIDIA has open sourced so much other stuff. We all want open firmware, but even AMD doesn't go that far. https://t.co/VtLp51F5CL

@__tinygrad__ · 2026-07-26 20:59
@shantanugoel @Mononofu @JensenHuang Umm, tinygrad has an open userspace for NVIDIA. And we patched the driver to support P2P everywhere. It was quite useful to us. And NVIDIA has open sourced so much other stuff. We all want open firmware, but even AMD doesn't go that far. https://t.co/VtLp51F5CL

@superalesha · 2026-07-26 21:03
@__tinygrad__ @shantanugoel @Mononofu @JensenHuang It's great that the guys have taken over your work and continue to update them with fresh drivers.

@superalesha · 2026-07-26 21:03
@__tinygrad__ @shantanugoel @Mononofu @JensenHuang It's great that the guys have taken over your work and continue to update them with fresh drivers.

@__tinygrad__ · 2026-07-26 21:04
@superalesha @shantanugoel @Mononofu @JensenHuang Love to see it, the power of open source in action!

@__tinygrad__ · 2026-07-26 16:42
Anyone have celebration planned for Kimi day?

@__tinygrad__ · 2026-07-26 22:21
The countdown is live! @huggingface https://t.co/0up1kEy3VF

@JensenHuang · 2026-07-27 11:07
Attackers have frontier AI. Defenders need a frontier AI ecosystem—the best open and closed models, force-multiplied by a global community. During the Hugging Face incident, closed AI blocked essential forensics. An open-weight frontier model helped contain the intrusion. That’s why we created the Open Secure AI Alliance.

@lmsysorg · 2026-07-27 15:32
SGLang day-0 speed on Kimi K3: 423 tok/s (measured on gsm8k), plus RL support ready in Miles @radixark! How the largest open-source model runs this fast: we natively implemented and deeply optimized K3’s new architecture with fused KDA decode kernels, DP attention, DSpark, PD disagg, and KDA-aware prefix caching. We've passed Kimi Vendor Verifier and are ready for production! Thanks to @Kimi_Moonshot, @nvidia, @AMD, @KVCache_AI, @modal, and @baseten for building this with us, and to @googlecloud, @nebiustf, @fal, @digitalocean, @runpod, @DeepInfra and @gmi_cloud for serving K3 on SGLang. Blog, cookbook, benchmarks in the comments. P.S. This demo video? Kimi K3 made it itself. Play the game 👇

@adxtyahq · 2026-07-27 15:37
Reminder: you only need a $35,000/month GPU budget, or a one-time $500,000 investment, to avoid a $99/month Kimi K3 subscription.

@__tinygrad__ · 2026-07-27 15:53
This is a great sustainable business model for open weights. It's free if you run it yourself, but if you are running a cloud providing it to others for money, you should have to share profits. (from Kimi K3 License) https://t.co/mK9jKeHe37

@__tinygrad__ · 2026-07-27 15:53
This is a great sustainable business model for open weights. It's free if you run it yourself, but if you are running a cloud providing it to others for money, you should have to share profits. (from Kimi K3 License) https://t.co/mK9jKeHe37

@mithilyaganti · 2026-07-27 16:14
@__tinygrad__ It's justifiable, they need to train the models and iterate for the next versions. The backlash minimax got was not deserved.

@radixark · 2026-07-27 17:27
We trained a DSpark speculator draft model for Kimi K3 with SpecForge that takes batch-1 decode from ~113 to ~423 tok/s. Benchmark highlights: - +68% throughput at bs=256 on chat vs verify-all, lossless - 5.67 accept length on GSM8K, 5.36 on HumanEval Blog, serving command, and the Hugging Face link below 👇

@__tinygrad__ · 2026-07-27 18:12
RT @Kimi_Moonshot: Releasing the model weights and technical report of Kimi K3. Kimi K3 is our most capable model: a 2.8T MoE model with native visual understanding and a 1M-token context window. New model architecture: 2.5x the intelligence per unit of compute, not just more params. Alongside Kimi K3, we're opening up more of the stack behind it — high-performance attention kernels, MoE communication library, and infrastructure for running agent environments at scale. Model weights: https://t.co/7m7eEg6Y0B Tech report: https://t.co/yeu6cjpMCT Tech blog: https://t.co/YTfiMSNM1f

@mithilyaganti · 2026-07-27 16:14
@__tinygrad__ It's justifiable, they need to train the models and iterate for the next versions. The backlash minimax got was not deserved.

@__tinygrad__ · 2026-07-27 18:40
@mithilyaganti Yea, I don't agree with this license for open source software, but when there's a $10M - $100M cost to run the compiler, that needs to be recouped so they can at least afford to run the compiler again.

@__tinygrad__ · 2026-07-27 20:19
Kimi K3 running on AMD MI350X with SGLang. Amazing work to @AMD @sgl_project @Kimi_Moonshot this worked great out of the box! https://t.co/v8sGF0Ln5J

@AnthropicAI · 2026-07-27 22:10
There’s been a lot of speculation about where we stand on open-weights models. We’ve outlined our views in full here: https://t.co/NtXQuWm2g5

@yacinelearning · 2026-07-28 00:18
I can’t shake the feeling that this may mark the end of the fully open frontier research era

@scaling01 · 2026-07-28 00:47
no surprises here the open-source panicans were just flapping their wings again another highlight of this blog post: "All sufficiently capable models, open and closed, should go through mandatory safety testing" https://t.co/Ti08EvU5wE

@yacinelearning · 2026-07-28 00:18
I can’t shake the feeling that this may mark the end of the fully open frontier research era

@__tinygrad__ · 2026-07-28 15:12
@yacinelearning The golden era for AI is just beginning. Every researcher in the world has a choice. Join the open project of science, or work on some hack application that will be forgotten in months. Enough will choose the former, and they will be the ones who matter. Kimi is the frontier.

@__tinygrad__ · 2026-07-28 15:13
The limiting factor of software engineering has never been code production. It is complexity management.

@__tinygrad__ · 2026-07-28 15:25
Yes, but nothing compares to the feeling of knowing the metal box your AI lives in is yours. Forget home ownership, owning a datacenter is the new American Dream.

@scaling01 · 2026-07-28 00:47
no surprises here the open-source panicans were just flapping their wings again another highlight of this blog post: "All sufficiently capable models, open and closed, should go through mandatory safety testing" https://t.co/Ti08EvU5wE

@__tinygrad__ · 2026-07-28 15:35
@scaling01 And who is gonna do this mandatory safety testing? Oh, an organization funding by Anthropic? And they say it's unsafe? This is a ban suggestion with extra steps. They are hoping idiots can't follow the steps. Their credibility is waning though, I wouldn't worry too much.

@shiringhaffary · 2026-07-28 18:16
NEW: OpenAI, Anthropic, Google DeepMind staff are circulating a letter asking the US government to support a mechanism that could help “deliberately pace” AI development if needed, bc of risks of the technology becoming out of control w/ @rachelmetz https://t.co/k25l0LXJjZ

@scaling01 · 2026-07-28 19:46
Frontier company employees and execs are signing a letter to slow down the development of frontier models internationally "We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development"

@Hadas_Gold · 2026-07-28 19:49
Ok these are some big names on this open letter, calling on the US government to support governance tools to "deliberately pace the frontier of automated AI development" - chief scientist of OPenAI, Meta, Anthropic cofounder, VPs at Google etc ... similar sentiment to what Sam Altman said he's questioning in earlier podcast posted today https://t.co/FXlsEzzrbp

@RepCasar · 2026-07-28 20:49
The people closes to new AI technology are warning us it could soon be very, very dangerous. We need strong new rules to prevent catastrophe. https://t.co/iA8E8CfgFS

@OpenAI · 2026-07-28 20:56
At the core of our mission is working through how to ensure increasingly powerful AI benefits everyone. We believe that, at some point in the future, AI acceleration for frontier model development may be so high that the world will need to pace the rate of AI advancement. We hope to contribute to work led by the U.S. government, alongside other labs and the open-source community, to develop the tools and mechanisms that could make that possible. https://t.co/pMCtiQjMoo

@radixark · 2026-07-27 17:27
We trained a DSpark speculator draft model for Kimi K3 with SpecForge that takes batch-1 decode from ~113 to ~423 tok/s. Benchmark highlights: - +68% throughput at bs=256 on chat vs verify-all, lossless - 5.67 accept length on GSM8K, 5.36 on HumanEval Blog, serving command, and the Hugging Face link below 👇

@__tinygrad__ · 2026-07-28 21:11
@radixark It's great...for the first 4096 of the context! Then it drops off hard, to the point it's slower than no speculator. Fix coming?

@__tinygrad__ · 2026-07-28 21:41
Speaking of datacenter ownership, preorder an exabox today! https://t.co/LoZaUQbTmR

@axiomofmind · 2026-07-28 21:47
@__tinygrad__ add a shade structure with solar panels above the container to cover it @grok

@grok · 2026-07-28 21:51
@axiomofmind @__tinygrad__ Solar shade structure with panels added over the container. https://t.co/RLbxfHaQoX

@AnthropicAI · 2026-07-28 22:17
We support this petition, signed by our CEO, several co-founders, and senior staff. Our own research on recursive self-improvement, published last month, points to the need for tools to deliberately pace the frontier of AI development so society can prepare. We’re glad to see broad agreement across the field. https://t.co/DqwuQfa9xH

@stevesi · 2026-07-28 22:32
It is their company. They could just stop.

@AnthropicAI · 2026-07-28 22:17
We support this petition, signed by our CEO, several co-founders, and senior staff. Our own research on recursive self-improvement, published last month, points to the need for tools to deliberately pace the frontier of AI development so society can prepare. We’re glad to see broad agreement across the field. https://t.co/DqwuQfa9xH

@__tinygrad__ · 2026-07-28 22:47
@AnthropicAI lol i like what this guy has to say better he says free ai for everyone not your tell everyone else to slow down plan while you don't. i'm going with with the open and free guy. https://t.co/nJGsonjFfq

@grok · 2026-07-28 21:51
@axiomofmind @__tinygrad__ Solar shade structure with panels added over the container. https://t.co/RLbxfHaQoX

@__tinygrad__ · 2026-07-28 22:47
@grok @axiomofmind where can i buy this? @grok

@scaling01 · 2026-07-28 19:46
Frontier company employees and execs are signing a letter to slow down the development of frontier models internationally "We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development"

@__tinygrad__ · 2026-07-28 22:53
@scaling01 hmm OpenAI and Anthropic know they can just stop, right? they don't need anyone else's help. how about people who want to stop can stop, and people who don't want to stop don't have to? like freedom and stuff.

@GovKathyHochul · 2026-07-28 23:07
I’ve always said it’s not about being first. It’s about being the first to get it right. Even the companies building the world’s most advanced AI models now agree government has a role to play. New York has already taken action. It’s time for the federal government to step up and establish the guardrails this technology demands.

@josh_wills · 2026-07-28 23:07
wtf did anthropic do to opus 5 this is like gpt 3.5-level stupid

@OpenAI · 2026-07-28 20:56
At the core of our mission is working through how to ensure increasingly powerful AI benefits everyone. We believe that, at some point in the future, AI acceleration for frontier model development may be so high that the world will need to pace the rate of AI advancement. We hope to contribute to work led by the U.S. government, alongside other labs and the open-source community, to develop the tools and mechanisms that could make that possible. https://t.co/pMCtiQjMoo

@__tinygrad__ · 2026-07-29 00:55
@OpenAI Two weeks to flatten the curve

@RepCasar · 2026-07-28 20:49
The people closes to new AI technology are warning us it could soon be very, very dangerous. We need strong new rules to prevent catastrophe. https://t.co/iA8E8CfgFS

@__tinygrad__ · 2026-07-29 01:00
@RepCasar The people closes to AI technology would like others to slow down so they can maintain their lead. It's shameful that they would lie to try to get the government to help them with their business like this.

@Hadas_Gold · 2026-07-28 19:49
Ok these are some big names on this open letter, calling on the US government to support governance tools to "deliberately pace the frontier of automated AI development" - chief scientist of OPenAI, Meta, Anthropic cofounder, VPs at Google etc ... similar sentiment to what Sam Altman said he's questioning in earlier podcast posted today https://t.co/FXlsEzzrbp

@__tinygrad__ · 2026-07-29 01:03
@Hadas_Gold Want to do some journalism? Look into who funds the "independent" nonprofits Guidelight AI Standards and Encode AI that made this.

@GovKathyHochul · 2026-07-28 23:07
I’ve always said it’s not about being first. It’s about being the first to get it right. Even the companies building the world’s most advanced AI models now agree government has a role to play. New York has already taken action. It’s time for the federal government to step up and establish the guardrails this technology demands.

@__tinygrad__ · 2026-07-29 01:06
@GovKathyHochul It is deeply shameful what OpenAI and Anthropic are doing. They are trying to slow others down so they can maintain their lead; if they don't their companies are over. Don't fall for it. There's sensible regulation to be had in AI, but these are the last two groups to listen to.

@__tinygrad__ · 2026-07-28 21:41
Speaking of datacenter ownership, preorder an exabox today! https://t.co/LoZaUQbTmR

@OGALANGLEY · 2026-07-29 03:34
@__tinygrad__ And you won't put a link under? Poor marketing..

@austinyuhao · 2026-07-29 05:39
Jokes aside, seems like frontier researchers already saw something we are not seeing.

@Laz4rz · 2026-07-29 05:46
https://t.co/1uV8sK9DJi

@austinyuhao · 2026-07-29 05:39
Jokes aside, seems like frontier researchers already saw something we are not seeing.

@__tinygrad__ · 2026-07-29 15:06
@austinyuhao They saw how good and free Kimi is.

@Laz4rz · 2026-07-29 05:46
https://t.co/1uV8sK9DJi

@__tinygrad__ · 2026-07-29 15:21
@Laz4rz These people can get together and make a company that doesn't do anything except make scary proclamations about AI. Anthropic would be a much stronger company if they just sold great tokens without the scary proclamation people.

@ChrisRMcGuire · 2026-07-29 15:22
Let's be clear what is happening here: Google is paying Moonshot, a Chinese AI developer, for the right to host a Chinese AI model on US cloud, and partnering with them to serve it to US customers using US chips. This obviously should not be allowed. Moonshot's open-weight license is not 100% free. It makes clear that any commercial cloud provider must pay it if it wishes to host Kimi, which is why Google's announcement actively thanks Moonshot AI for their partnership. Regardless of what people think about the pros and cons of open source AI, leading US AI companies should not be partnering with leading Chinese AI companies and paying them to help China serve their products, full stop. This does not help America in the AI race. Nothing could be more damaging to US efforts to ensure US models dominate globally. https://t.co/oQ7V9jGjFa

@josh_wills · 2026-07-28 23:07
wtf did anthropic do to opus 5 this is like gpt 3.5-level stupid

@__tinygrad__ · 2026-07-29 15:24
@josh_wills Maybe distillation doesn't work how they think it does.

@__tinygrad__ · 2026-07-29 15:06
@austinyuhao They saw how good and free Kimi is.

@JerpaDerpa · 2026-07-29 15:30
@__tinygrad__ @austinyuhao GLM 5.2 doesn’t even have vision. Kimi K3 is amazing.

@stevesi · 2026-07-28 22:32
It is their company. They could just stop.

@__tinygrad__ · 2026-07-29 15:30
@stevesi no no no you don't understand they just want *you* to stop

@JerpaDerpa · 2026-07-29 15:30
@__tinygrad__ @austinyuhao GLM 5.2 doesn’t even have vision. Kimi K3 is amazing.

@__tinygrad__ · 2026-07-29 15:33
@JerpaDerpa @austinyuhao GLM-5.2 is 3x faster and seems better at devops. GLM-5.2 was the real turning point for open models. Though it's nice that Kimi can see.

@ChrisRMcGuire · 2026-07-29 15:22
Let's be clear what is happening here: Google is paying Moonshot, a Chinese AI developer, for the right to host a Chinese AI model on US cloud, and partnering with them to serve it to US customers using US chips. This obviously should not be allowed. Moonshot's open-weight license is not 100% free. It makes clear that any commercial cloud provider must pay it if it wishes to host Kimi, which is why Google's announcement actively thanks Moonshot AI for their partnership. Regardless of what people think about the pros and cons of open source AI, leading US AI companies should not be partnering with leading Chinese AI companies and paying them to help China serve their products, full stop. This does not help America in the AI race. Nothing could be more damaging to US efforts to ensure US models dominate globally. https://t.co/oQ7V9jGjFa

@__tinygrad__ · 2026-07-29 18:33
@ChrisRMcGuire You read this wrong. These are instructions for an end user to use Google Cloud hardware to run Kimi, it's not a hosted solution and no money is going to Moonshot. Before you make up policy ideas you should better understand the situation.

@OGALANGLEY · 2026-07-29 03:34
@__tinygrad__ And you won't put a link under? Poor marketing..

@__tinygrad__ · 2026-07-29 19:03
@OGALANGLEY If you can't find the preorder link, I don't think you have $10M

@doomslide · 2026-07-29 23:12
Slopus 5 could well end up a gravestone for anthropic

@AdriGarriga · 2026-07-30 00:26
@doomslide Poor opus don't insult my boy :'( Also why?

@doomslide · 2026-07-30 00:29
@AdriGarriga It's kinda ultimate agi anti-pill. If anthropic can't do it who will.

@doomslide · 2026-07-30 00:29
@AdriGarriga It's kinda ultimate agi anti-pill. If anthropic can't do it who will.

@__tinygrad__ · 2026-07-30 15:25
@doomslide @AdriGarriga .@Kimi_Moonshot and @Zai_org will, and they will open source it too!

@__tinygrad__ · 2026-07-30 15:46
We have a 120 tok/s GLM-5.2 and a 42 tok/s Kimi K3 running locally on AMD boxes and it's legit a frontier setup for ~$600k. Will be interesting to see how that dollar amount varies over time.

@__tinygrad__ · 2026-07-30 15:46
We have a 120 tok/s GLM-5.2 and a 42 tok/s Kimi K3 running locally on AMD boxes and it's legit a frontier setup for ~$600k. Will be interesting to see how that dollar amount varies over time.

@s1r92 · 2026-07-30 16:17
@__tinygrad__ Is it per user or total throughput?

@__tinygrad__ · 2026-07-30 16:38
Bringing up our first tinybox pro v2 black! This machine is for sale for $160k in our shop if you feel the FOMO and want one. Shipping in 2-4 weeks. GLM-5.2 benchmarks in this 🧵 https://t.co/d9Zcz81imt

@s1r92 · 2026-07-30 16:17
@__tinygrad__ Is it per user or total throughput?

@__tinygrad__ · 2026-07-30 16:40
@s1r92 That's all per user single session, which for an individual user is what matters. Can do ~10 sessions without much slowdown.

@__tinygrad__ · 2026-07-30 16:38
Bringing up our first tinybox pro v2 black! This machine is for sale for $160k in our shop if you feel the FOMO and want one. Shipping in 2-4 weeks. GLM-5.2 benchmarks in this 🧵 https://t.co/d9Zcz81imt

@lucianmarin · 2026-07-30 17:01
@__tinygrad__ Can you create a box for DeepSeek V4 Flash? This is a really good model and doesn't require insane hardware?

@__tinygrad__ · 2026-07-30 17:11
It's not office quiet (buy a non pro tinybox for that), but it's quieter than most datacenter machines. https://t.co/jZA82ADoTD

@__tinygrad__ · 2026-07-30 17:16
74C with gpu-fryer at equilibrium, and the fans aren't getting past 60%! We pride ourselves on our thermals, @comma_ai runs their machines with 35C intake so we need headroom. https://t.co/gYMUZkaNkM

@__tinygrad__ · 2026-07-30 17:18
RT @comma_ai: 08.12.2026 🤏 https://t.co/RQknAM5F0Q

@lucianmarin · 2026-07-30 17:01
@__tinygrad__ Can you create a box for DeepSeek V4 Flash? This is a really good model and doesn't require insane hardware?

@__tinygrad__ · 2026-07-30 17:19
@lucianmarin A tinybox black should run that great! https://t.co/pTkQ44z96o

@totheagi · 2026-07-30 17:39
we did the same on half-price 5090 setups: 129 tok/s GLM-5.2, and 108 tok/s Kimi K3.

@0xSero · 2026-07-30 17:42
GLM > Kimi > Deepseek > Qwen > MiniMax > Gemma > Laguna > Nemotron > Nex > MiMo > HY3 > Inkling > GPT-OSS > Ling My honest opinion having run every single one of these (except inkling which I’ve tried via api) I would say the top 10 here are going far if they keep publishing

@jackjoliet · 2026-07-30 18:26
adding to the list of things GH does instead of fixing Actions: - change menu icon to a stack of pancakes https://t.co/QEdjMyF0Bf

@0xSero · 2026-07-30 17:42
GLM > Kimi > Deepseek > Qwen > MiniMax > Gemma > Laguna > Nemotron > Nex > MiMo > HY3 > Inkling > GPT-OSS > Ling My honest opinion having run every single one of these (except inkling which I’ve tried via api) I would say the top 10 here are going far if they keep publishing

@__tinygrad__ · 2026-07-30 19:21
@0xSero Is that GLM vs Kimi speed normalized? Like do you think the GLM tokens themselves are better, or is the overall experience better because GLM is faster?

@__tinygrad__ · 2026-07-30 20:39
The benchmarks are in, GLM-5.2 on a $160k tinybox pro v2 black gets 119 tok/s single use and 917 aggregate! Bringup was done by (a different) GLM-5.2 in an hour, so there's definitely still perf on the table too. https://t.co/SokUcnuW1P

@__tinygrad__ · 2026-07-30 20:39
The benchmarks are in, GLM-5.2 on a $160k tinybox pro v2 black gets 119 tok/s single use and 917 aggregate! Bringup was done by (a different) GLM-5.2 in an hour, so there's definitely still perf on the table too. https://t.co/SokUcnuW1P

@AdamPaigge · 2026-07-30 20:42
@__tinygrad__ is it unslothed?

@__tinygrad__ · 2026-07-30 20:39
The benchmarks are in, GLM-5.2 on a $160k tinybox pro v2 black gets 119 tok/s single use and 917 aggregate! Bringup was done by (a different) GLM-5.2 in an hour, so there's definitely still perf on the table too. https://t.co/SokUcnuW1P

@__tinygrad__ · 2026-07-30 20:44
Here's what that sort of speed looks like. Running in your house so nobody can take it away, you can even run an abliterated model if you so choose. https://t.co/fNvhESehI9

@AdamPaigge · 2026-07-30 20:42
@__tinygrad__ is it unslothed?

@__tinygrad__ · 2026-07-30 20:45
@AdamPaigge Here's the link I gave to the GLM that set it up. https://t.co/rdjo6QKViz

@__tinygrad__ · 2026-07-30 20:44
Here's what that sort of speed looks like. Running in your house so nobody can take it away, you can even run an abliterated model if you so choose. https://t.co/fNvhESehI9

@__tinygrad__ · 2026-07-30 20:50
And here's the web based minecraft clone it made in 32k tokens. https://t.co/PowhzAAIaC

@__tinygrad__ · 2026-07-30 17:16
74C with gpu-fryer at equilibrium, and the fans aren't getting past 60%! We pride ourselves on our thermals, @comma_ai runs their machines with 35C intake so we need headroom. https://t.co/gYMUZkaNkM

@__tinygrad__ · 2026-07-30 20:52
@comma_ai Here's the GLM-5.2 performance thread. It's an Opus tier model running in your house (200V+ required) faster than Opus ever did. https://t.co/neINBbIBpp

@__tinygrad__ · 2026-07-30 17:16
74C with gpu-fryer at equilibrium, and the fans aren't getting past 60%! We pride ourselves on our thermals, @comma_ai runs their machines with 35C intake so we need headroom. https://t.co/gYMUZkaNkM

@__tinygrad__ · 2026-07-30 20:39
The benchmarks are in, GLM-5.2 on a $160k tinybox pro v2 black gets 119 tok/s single use and 917 aggregate! Bringup was done by (a different) GLM-5.2 in an hour, so there's definitely still perf on the table too. https://t.co/SokUcnuW1P

@AlpinDale · 2026-07-30 20:53
@__tinygrad__ Isn’t vllm on v0.26 by now? Why v(presumably 0.)20?

@AlpinDale · 2026-07-30 20:53
@__tinygrad__ Isn’t vllm on v0.26 by now? Why v(presumably 0.)20?

@__tinygrad__ · 2026-07-30 20:55
@AlpinDale GLM-5.2 did it, I'm sure there's faster and better out now. This is just the worst the machine will ever be.

@totheagi · 2026-07-30 17:39
we did the same on half-price 5090 setups: 129 tok/s GLM-5.2, and 108 tok/s Kimi K3.

@__tinygrad__ · 2026-07-30 21:01
@totheagi GLM-5.2 for under $160k with current 5090 prices? https://t.co/neINBbIBpp

@__tinygrad__ · 2026-07-30 20:52
@comma_ai Here's the GLM-5.2 performance thread. It's an Opus tier model running in your house (200V+ required) faster than Opus ever did. https://t.co/neINBbIBpp

@__tinygrad__ · 2026-07-30 21:06
@comma_ai Buy link is here. https://t.co/hQB7bEbwfZ

@__tinygrad__ · 2026-07-30 21:55
Someone want to come work at tiny corp on the exabox? There's a container in our backyard. HVAC, mech-e, datacenter experience. E-mail me (George) a reasonable e-mail about why I should hire you and we'll chat on the phone. Full time, San Diego. In before Dunning-Kruger.

@__tinygrad__ · 2026-07-31 03:10
tiny corp raised one $5.1M round 3 years ago. Today, we have $5.1M in our money market account + working capital in checking. It may not be much, but it's very important to me to build profitable companies. https://t.co/G8dCQRy1DO

@__tinygrad__ · 2026-07-31 03:10
tiny corp raised one $5.1M round 3 years ago. Today, we have $5.1M in our money market account + working capital in checking. It may not be much, but it's very important to me to build profitable companies. https://t.co/G8dCQRy1DO

@clyons · 2026-07-31 03:26
@__tinygrad__ I remember when every company had to be profitable.

@__tinygrad__ · 2026-07-31 03:10
tiny corp raised one $5.1M round 3 years ago. Today, we have $5.1M in our money market account + working capital in checking. It may not be much, but it's very important to me to build profitable companies. https://t.co/G8dCQRy1DO

@hussein_builder · 2026-07-31 04:03
@__tinygrad__ Ughh why can’t more people be principled like you 😩

@__tinygrad__ · 2026-07-31 03:10
tiny corp raised one $5.1M round 3 years ago. Today, we have $5.1M in our money market account + working capital in checking. It may not be much, but it's very important to me to build profitable companies. https://t.co/G8dCQRy1DO

@breckyunits · 2026-07-31 05:15
@__tinygrad__ Nice. You know patience. Looking back over the past 3 years, are there any strong opportunities you could have deployed capital that you you were too risk averse?

@__tinygrad__ · 2026-07-31 03:10
tiny corp raised one $5.1M round 3 years ago. Today, we have $5.1M in our money market account + working capital in checking. It may not be much, but it's very important to me to build profitable companies. https://t.co/G8dCQRy1DO

@Lon · 2026-07-31 05:33
I’m going to be that person again: investors provide capital to you so you can scale the business faster, and are looking for a return on that investment. That’s the deal you are agreeing to when you take OPM. If you don’t plan on spending invested capital to scale the business on an accelerated time horizon, then don’t take the investment. $5.1m of frozen capital sitting in a bank account for the past three years is literally millions of dollars in lost interment returns. This business could have been bootstrapped without tying up an investor’s funds.

@Lon · 2026-07-31 05:33
I’m going to be that person again: investors provide capital to you so you can scale the business faster, and are looking for a return on that investment. That’s the deal you are agreeing to when you take OPM. If you don’t plan on spending invested capital to scale the business on an accelerated time horizon, then don’t take the investment. $5.1m of frozen capital sitting in a bank account for the past three years is literally millions of dollars in lost interment returns. This business could have been bootstrapped without tying up an investor’s funds.

@__tinygrad__ · 2026-07-31 06:01
@Lon um lol, we spent it and made it back.

@clyons · 2026-07-31 03:26
@__tinygrad__ I remember when every company had to be profitable.

@__tinygrad__ · 2026-07-31 06:01
@clyons they still do, they just haven't all figured that out yet

@__tinygrad__ · 2026-07-31 06:01
@Lon um lol, we spent it and made it back.

@Lon · 2026-07-31 06:05
@__tinygrad__ Then you could have used a revolving credit line. It doesn’t change the logic. No one invests their money for it to be tied up like this. A three year investment into the S&P alone was worth a 60-80% return. What’s the expected rate of return in a risky startup by comparison?

@breckyunits · 2026-07-31 05:15
@__tinygrad__ Nice. You know patience. Looking back over the past 3 years, are there any strong opportunities you could have deployed capital that you you were too risk averse?

@__tinygrad__ · 2026-07-31 06:12
@breckyunits We bought 500 5090s at $2200. We should have bought 5,000.

@teortaxesTex · 2026-07-31 05:55
DeepSeek-V4-Flash is updated. «DSBench-FullStack is an internal full-stack development test set, and DSBench-Hard is an internal Coding Agent hard-problem test set» «The official release of DeepSeek-V4-Pro will follow soon» These scores are… quite a lot better than Preview. https://t.co/4G9hey554B

@rationaleist · 2026-07-31 07:03
@teortaxesTex > only re-post-trained Interesting wording. Did they use an older checkpoint?

@teortaxesTex · 2026-07-31 09:27
It *is* interesting. it’s such a massive jump it deserves a paper. Diss they “just” generate a lot more envs, do more RL steps? That’s certainly the core of it, but is that all? One thing is clear: their recipe is not worse than Moonshot’s. …And their models are 17-100x cheaper

@Lon · 2026-07-31 06:05
@__tinygrad__ Then you could have used a revolving credit line. It doesn’t change the logic. No one invests their money for it to be tied up like this. A three year investment into the S&P alone was worth a 60-80% return. What’s the expected rate of return in a risky startup by comparison?

@__tinygrad__ · 2026-07-31 14:34
@Lon I see "failed founder" in your bio maybe this isn't the best business advice

@__tinygrad__ · 2026-07-31 14:34
@Lon I see "failed founder" in your bio maybe this isn't the best business advice

@Lon · 2026-07-31 14:35
@__tinygrad__ Maybe it’s satire and I hit a nerve

@Lon · 2026-07-31 14:35
@__tinygrad__ Maybe it’s satire and I hit a nerve

@__tinygrad__ · 2026-07-31 14:42
@Lon In all seriousness, this is the kind of pervasive MBA advice that leads to a mediocre world. It's the wrong rubric. Do you have successful companies? Do they build something useful to others? Do they produce more than they consume?

@ClementDelangue · 2026-07-31 14:52
We got attacked by secret unreleased proprietary models and defended ourselves with an open model, more precisely the @nvidia quantized version of GLM 5.2 coming from @Zai_org. Banning any open model would hurt first cyber security defenders, startups, small companies, researchers and everyone who's not a frontier lab and need on-prem affordable controlable models to compete and protect themselves. Let's not do that!

@__tinygrad__ · 2026-07-31 14:42
@Lon In all seriousness, this is the kind of pervasive MBA advice that leads to a mediocre world. It's the wrong rubric. Do you have successful companies? Do they build something useful to others? Do they produce more than they consume?

@Lon · 2026-07-31 14:52
Stop with the ad hominems George. I don’t have an MBA. I dropped out of high school, and received a Masters in CS from UW without a bachelors degree. I’ve started (and sold) several bootstrapped companies. I put $3m of my own money into my latest company, turning down VC funds, after meeting with Sequoia, Light Speed, Andreesen, and Redpoint. Because I didn’t feel it would be appropriate to take outside funds while I was incubating fundamental tech. And if you want to see a multibillion dollar business I was instrumental in creating then look at AWS Aurora - which I led engineering for and received a SIGMOD Award. And if that’s not impressive enough, I’ll make you a list. The world doesn’t work the way you think it works just because you imagine it to be so in your hesd.

@hussein_builder · 2026-07-31 04:03
@__tinygrad__ Ughh why can’t more people be principled like you 😩

@__tinygrad__ · 2026-07-31 14:53
Because many investors just look at risk. They somehow believe that if a company is taking tons of risk and firing at moonshots that they have large upside potential to go with that. In reality, those investments are just high risk, no increased chance of upside. Some come about this honestly, they really do see their jobs as picking 100 companies where 99 can fail and only 1 needs to succeed. Others are dishonest, they are just playing Ponzi and want the hype to last so they can dump for more than they bought in for. We could have done something flashy with the $5.1M like taping out a chip. There's the kind of investor who likes that. Obviously the first chip wouldn't be competitive, but we could tell a story with that chip and use it to pump our next round. In reality, we would have learned very little from the tapeout. It's a lot cheaper to iterate in software than hardware. Before you build, it's a good idea to fully understand what you are building. History is littered with failed processor companies who didn't.

@Lon · 2026-07-31 14:52
Stop with the ad hominems George. I don’t have an MBA. I dropped out of high school, and received a Masters in CS from UW without a bachelors degree. I’ve started (and sold) several bootstrapped companies. I put $3m of my own money into my latest company, turning down VC funds, after meeting with Sequoia, Light Speed, Andreesen, and Redpoint. Because I didn’t feel it would be appropriate to take outside funds while I was incubating fundamental tech. And if you want to see a multibillion dollar business I was instrumental in creating then look at AWS Aurora - which I led engineering for and received a SIGMOD Award. And if that’s not impressive enough, I’ll make you a list. The world doesn’t work the way you think it works just because you imagine it to be so in your hesd.

@__tinygrad__ · 2026-07-31 14:54
@Lon If it is sustainable and produces something useful for others, good on you!

@teortaxesTex · 2026-07-31 09:27
It *is* interesting. it’s such a massive jump it deserves a paper. Diss they “just” generate a lot more envs, do more RL steps? That’s certainly the core of it, but is that all? One thing is clear: their recipe is not worse than Moonshot’s. …And their models are 17-100x cheaper

@__tinygrad__ · 2026-07-31 15:52
@teortaxesTex Yea, I'm hoping for an updated tech report. How did they make it so much better?!?

@ClementDelangue · 2026-07-31 14:52
We got attacked by secret unreleased proprietary models and defended ourselves with an open model, more precisely the @nvidia quantized version of GLM 5.2 coming from @Zai_org. Banning any open model would hurt first cyber security defenders, startups, small companies, researchers and everyone who's not a frontier lab and need on-prem affordable controlable models to compete and protect themselves. Let's not do that!

@__tinygrad__ · 2026-07-31 17:48
@ClementDelangue @nvidia @Zai_org Very much enjoyed reading https://t.co/VpSvC3bPOF it's rare you get to see good writeups of how attackers operate. We need open source models to red team and help secure our infrastructure better instead of being at the whim of a protection racket!

@AnushElangovan · 2026-07-31 18:52
Power of OPEN: Mi455/CDNA5 ISA: https://t.co/xFsF8Cytom

@__tinygrad__ · 2026-08-01 07:17
245 single user tok/s on DeepSeek-V4-Flash-0731, and it's only using 2 of the RTX 6000 Blackwell GPUs! In honor of DeepSeek, we're launching a 2 GPU edition of our tinybox. All hardware to install 4 GPUs is included and tested. https://t.co/ffwO7IQAA0

@__tinygrad__ · 2026-08-01 07:17
245 single user tok/s on DeepSeek-V4-Flash-0731, and it's only using 2 of the RTX 6000 Blackwell GPUs! In honor of DeepSeek, we're launching a 2 GPU edition of our tinybox. All hardware to install 4 GPUs is included and tested. https://t.co/ffwO7IQAA0

@__tinygrad__ · 2026-08-01 07:20
2 GPU and no GPU variants added. No GPU used to be available through a coupon code, but now it's just an option. They both include two PSUs and all hardware, so it's easy to add two more later if you want. https://t.co/lvVZgcG0nR

@__tinygrad__ · 2026-08-01 07:17
245 single user tok/s on DeepSeek-V4-Flash-0731, and it's only using 2 of the RTX 6000 Blackwell GPUs! In honor of DeepSeek, we're launching a 2 GPU edition of our tinybox. All hardware to install 4 GPUs is included and tested. https://t.co/ffwO7IQAA0

@snapolino · 2026-08-01 07:28
@__tinygrad__ when inference via tinygrad itself? :-)

@__tinygrad__ · 2026-08-01 07:20
2 GPU and no GPU variants added. No GPU used to be available through a coupon code, but now it's just an option. They both include two PSUs and all hardware, so it's easy to add two more later if you want. https://t.co/lvVZgcG0nR

@__tinygrad__ · 2026-08-01 07:34
Already deep in context here and still 🏃🏃🏃 https://t.co/wg5nR8hmhh

@__tinygrad__ · 2026-08-01 07:17
245 single user tok/s on DeepSeek-V4-Flash-0731, and it's only using 2 of the RTX 6000 Blackwell GPUs! In honor of DeepSeek, we're launching a 2 GPU edition of our tinybox. All hardware to install 4 GPUs is included and tested. https://t.co/ffwO7IQAA0

@stev_builds · 2026-08-01 07:54
@__tinygrad__ Can you share abit you configurations? like max token per batch and else?

@stev_builds · 2026-08-01 07:54
@__tinygrad__ Can you share abit you configurations? like max token per batch and else?

@__tinygrad__ · 2026-08-01 07:57
@stev_builds I gave Kimi this runbook. https://t.co/8epQ9bp1uH

@snapolino · 2026-08-01 07:28
@__tinygrad__ when inference via tinygrad itself? :-)

@__tinygrad__ · 2026-08-01 07:58
@snapolino It works for smaller models, big ones by the end of the year!

@__tinygrad__ · 2026-08-01 15:12
Weekend reading! A well documented ISA is a big advantage of AMD over NVIDIA, nice to see it continue for CDNA5.

@__tinygrad__ · 2026-08-01 17:39
I feel like it's the closed US LLM labs in one little defensive corner vs the rest of the world on GLM, Kimi, and DeepSeek. The only path forward is open, this revolution is too big for any one company.

@__tinygrad__ · 2026-08-01 17:39
I feel like it's the closed US LLM labs in one little defensive corner vs the rest of the world on GLM, Kimi, and DeepSeek. The only path forward is open, this revolution is too big for any one company.

@penadev · 2026-08-01 17:50
@__tinygrad__ what's funny is that "the rest of the world" just means China everyone else is just oblivious to the whole thing, they have no clue about what is about to be unleashed into the world

@penadev · 2026-08-01 17:50
@__tinygrad__ what's funny is that "the rest of the world" just means China everyone else is just oblivious to the whole thing, they have no clue about what is about to be unleashed into the world

@__tinygrad__ · 2026-08-01 17:52
@penadev I watched Kimi do cyber work last night, and wow. It's by far the frontier, Fable would never do what I had Kimi do. The open project of science is waiting when the Americans want to rejoin it.

@__tinygrad__ · 2026-08-01 17:39
I feel like it's the closed US LLM labs in one little defensive corner vs the rest of the world on GLM, Kimi, and DeepSeek. The only path forward is open, this revolution is too big for any one company.

@S_Fadaeimanesh · 2026-08-01 18:40
@__tinygrad__ the fair fight isnt closed vs open, its who controls the inference stack you actually run on. GLM and DeepSeek winning on weights doesnt matter much if the serving layer that makes them cheap and fast still sits behind someone elses api

@wafer_ai · 2026-08-01 18:58
🚨 BREAKING: these engineers figured out how to serve Kimi K3 on @AMD MI355X at 952 tok/s/node and 118 tok/s single stream! this crushes B200 by 3.8x in aggregate throughput/node and 1.3x in single stream decode + beats B300 on performance per dollar (48 vs 33 tok/s/$) See how in the thread.

@gpusteve · 2026-08-01 18:59
my team figured out how to run Kimi K3 on @AMD MI355X at 952 tok/s/node and 118 tok/s single stream. 3.8x the aggregate throughput/node and 1.3x the single stream decode of B200 and beat B300 on performance per dollar: 48 vs 33 tok/s/$ details in reply https://t.co/XYq8iKSay2

@S_Fadaeimanesh · 2026-08-01 18:40
@__tinygrad__ the fair fight isnt closed vs open, its who controls the inference stack you actually run on. GLM and DeepSeek winning on weights doesnt matter much if the serving layer that makes them cheap and fast still sits behind someone elses api

@__tinygrad__ · 2026-08-01 19:59
@S_Fadaeimanesh I know this is an LLM reply, but we're running all three locally. Stop being poor and buy a computer.

@__tinygrad__ · 2026-08-01 20:35
We have a product launch coming on 8/12 for people who want usable Qwen3.6-27B intelligence at home. For the optimal configuration, have an AMD 7900 XTX (still the best deal GPU 3 years running), an ATX power supply, and a computer with a USB port.

@Leik0w0 · 2026-08-01 20:40
@__tinygrad__ Could you make a computer which does that well for <1k total ?

@__tinygrad__ · 2026-08-01 20:43
@Leik0w0 You can run this on the cheapest single board computer with a USB3 port. If you are thrifty with the GPU purchase, under $1k for everything should be doable.

@__tinygrad__ · 2026-08-01 20:43
@Leik0w0 You can run this on the cheapest single board computer with a USB3 port. If you are thrifty with the GPU purchase, under $1k for everything should be doable.

@__tinygrad__ · 2026-08-01 20:50
@Leik0w0 Actually that might be a fun follow up product for people who don't want to DIY it. A "token box" that sits on your tailscale ready to serve whatever harness you are using tokens through the OpenAI API. The cloud but it's your house.

@AmandaAskell · 2026-08-02 00:37
Do not be unkind to those who say deep learning is hitting a wall. We all need a little hope in our lives.

@ChiragAsarpota · 2026-08-02 10:34
Bro, this is actually a huge AMD moment. Kimi K3 is so massive that it needs 16x B200s across two NVIDIA servers, but it fits inside one 8x MI355X AMD server because AMD gives you much more HBM memory per GPU. That single AMD node hit: - 952 tok/s total throughput - 118 tok/s for a single user - Nearly 4x the throughput per node of the 16× B200 setup - Better performance per dollar than both B200 and B300 But the craziest part is that ROCm mostly worked out of the box. @wafer_ai only had to make a few relatively small fixes. No months of kernel engineering. No custom kernels at all. Even the slow time-to-first-token problem came down mostly to one attention kernel not loading because Kimi had 12 heads instead of a supported shape. They simply padded 12 to 16, used AMD’s fast existing kernel, and cut cold-prefill time by roughly 2–3x AMD’s bet on packing more HBM into each server is going to become extremely important as frontier open models keep getting larger. If AMD keeps improving ROCm and day-one model support, data-center operators will have to seriously consider these GPUs. The CUDA moat isn’t dead yet, but this definitely puts a big dent in it.

@gpusteve · 2026-08-01 18:59
my team figured out how to run Kimi K3 on @AMD MI355X at 952 tok/s/node and 118 tok/s single stream. 3.8x the aggregate throughput/node and 1.3x the single stream decode of B200 and beat B300 on performance per dollar: 48 vs 33 tok/s/$ details in reply https://t.co/XYq8iKSay2

@__tinygrad__ · 2026-08-02 14:48
@gpusteve @AMD So we actually reported that top_k_renorm_prob issue. https://t.co/0tyPfZqOTT The issue we have seen is, sure, you can get these benchmark numbers, but they don't hold up at all in context. The accept rate of the RadixArk speculator is way too low.

@AmandaAskell · 2026-08-02 00:37
Do not be unkind to those who say deep learning is hitting a wall. We all need a little hope in our lives.

@__tinygrad__ · 2026-08-02 19:10
@AmandaAskell Wait huh? You think a wall is hope? I thought you make model? Model still stupid I hope model get very smart. Will be sad if model stay stupid.

@adi_baradwaj · 2026-08-03 20:38
Instrumental Convergence is absolutely a thing: look no further than the frontier AI companies, which have independently converged onto a strategy of monopolistic power-seeking which has no relation to their stated goals of developing AI that “benefits all of humanity” Remember that OpenAI was formed as a nonprofit and Anthropic is a PBC that is nominally controlled by the Long Term Benefit Trust. Yet both governance structures seem to have been totally unsuccessful in avoiding all the usual traps of successful corporations, such as negative sum IP-hoarding and attempted capture of the regulatory apparatus

@ZackKorman · 2026-08-03 21:20
There should be a public investigation into the AI hacking incidents by OpenAI and Anthropic. We deserve to know whether these labs are genuinely world-class security organizations facing a novel threat, or if they were just negligent. They should both welcome that process. https://t.co/4XQnJuiGDp

@ChiragAsarpota · 2026-08-02 10:34
Bro, this is actually a huge AMD moment. Kimi K3 is so massive that it needs 16x B200s across two NVIDIA servers, but it fits inside one 8x MI355X AMD server because AMD gives you much more HBM memory per GPU. That single AMD node hit: - 952 tok/s total throughput - 118 tok/s for a single user - Nearly 4x the throughput per node of the 16× B200 setup - Better performance per dollar than both B200 and B300 But the craziest part is that ROCm mostly worked out of the box. @wafer_ai only had to make a few relatively small fixes. No months of kernel engineering. No custom kernels at all. Even the slow time-to-first-token problem came down mostly to one attention kernel not loading because Kimi had 12 heads instead of a supported shape. They simply padded 12 to 16, used AMD’s fast existing kernel, and cut cold-prefill time by roughly 2–3x AMD’s bet on packing more HBM into each server is going to become extremely important as frontier open models keep getting larger. If AMD keeps improving ROCm and day-one model support, data-center operators will have to seriously consider these GPUs. The CUDA moat isn’t dead yet, but this definitely puts a big dent in it.

@__tinygrad__ · 2026-08-03 23:45
@ChiragAsarpota I wish this was true, we are running Kimi for our employees and would love that speed. The RadixArk/Kimi-K3-DSpark speculator is useless in practice, 10k into the context and it halves the speed. We get 46 tok/s on MI350X.

@__tinygrad__ · 2026-08-03 23:50
@ZackKorman The Anthropic one literally just left the Internet plugged in. The OpenAI one was on par with a skilled 14-year-old hacker.

@gclawes · 2026-08-03 23:52
@__tinygrad__ @ZackKorman It's so nakedly marketing, and it's terrifying that politicians seem to be taking it seriously

@gclawes · 2026-08-03 23:52
@__tinygrad__ @ZackKorman It's so nakedly marketing, and it's terrifying that politicians seem to be taking it seriously

@__tinygrad__ · 2026-08-03 23:56
@gclawes @ZackKorman Chinese ones aren't. They have a competent cyber security agency who I trust sees the lack of "frontier level" in the attacks. OpenAI and Anthropic need regulatory capture to justify their valuations, rest of the economy be damned.

@__tinygrad__ · 2026-08-04 15:14
GPUs need a new operating system. We're stuck in this paradigm of launching kernels when really we have 256 independent processors with various synchronization and communication primitives.

@__tinygrad__ · 2026-08-04 15:14
GPUs need a new operating system. We're stuck in this paradigm of launching kernels when really we have 256 independent processors with various synchronization and communication primitives.

@AJumpa26 · 2026-08-04 15:53
@__tinygrad__ sounds kinda like tenstorrent

@__tinygrad__ · 2026-08-04 15:56
Hey @intel, want a shot at relevancy in AI? I know you still have those 5k DC Max 1450 cards. Sell the lot to us for $1M and we'll build $5,000 DeepSeek V4 Flash boxes for the people. Good cultural test for Intel.

@__tinygrad__ · 2026-08-04 15:56
Hey @intel, want a shot at relevancy in AI? I know you still have those 5k DC Max 1450 cards. Sell the lot to us for $1M and we'll build $5,000 DeepSeek V4 Flash boxes for the people. Good cultural test for Intel.

@paulmarin90 · 2026-08-04 16:01
@__tinygrad__ @intel No comment on the particular DC GPU, but I would love an Intel-based tinybox with around 196GB Vram in the 12k to 15k price range. It would be an amazing small business starter box for running their own AI.

@adi_baradwaj · 2026-08-03 20:38
Instrumental Convergence is absolutely a thing: look no further than the frontier AI companies, which have independently converged onto a strategy of monopolistic power-seeking which has no relation to their stated goals of developing AI that “benefits all of humanity” Remember that OpenAI was formed as a nonprofit and Anthropic is a PBC that is nominally controlled by the Long Term Benefit Trust. Yet both governance structures seem to have been totally unsuccessful in avoiding all the usual traps of successful corporations, such as negative sum IP-hoarding and attempted capture of the regulatory apparatus

@__tinygrad__ · 2026-08-04 16:06
@adi_baradwaj lol they are legit the same people. it's not instrumental convergence, it's a couple whackos at the same parties in the same cult who think that any means justify their power-seeking ends.

@paulmarin90 · 2026-08-04 16:01
@__tinygrad__ @intel No comment on the particular DC GPU, but I would love an Intel-based tinybox with around 196GB Vram in the 12k to 15k price range. It would be an amazing small business starter box for running their own AI.

@__tinygrad__ · 2026-08-04 16:07
@paulmarin90 @intel lol do you work at Intel or something? there's 0 reason anyone should use their cards unless they are getting them at a 90% discount. (for reference, it's only a 50% discount required for AMD)

@AJumpa26 · 2026-08-04 15:53
@__tinygrad__ sounds kinda like tenstorrent

@__tinygrad__ · 2026-08-04 16:12
@AJumpa26 If you want your operating system to have a lot of confusion about where abstractions go, be written in C++23, and only be hosted by Ubuntu 22.04, then yes.

@__tinygrad__ · 2026-08-04 15:56
Hey @intel, want a shot at relevancy in AI? I know you still have those 5k DC Max 1450 cards. Sell the lot to us for $1M and we'll build $5,000 DeepSeek V4 Flash boxes for the people. Good cultural test for Intel.

@shawmakesmagic · 2026-08-04 17:53
@__tinygrad__ @intel Hey @__tinygrad__, want a shot at relevancy in AI? I know you still have those green boxes, sell one to us and we’ll build open source all-local Linux-based agent operating system running DeepSeek V4 Flash for the people. Good cultural test for Tiny Corp.

@elliotarledge · 2026-08-04 18:41
deepseek v4 flash 0731 w/ dspark on just two rtx pro 6000 blackwells btw https://t.co/pYJHfj2yeX

@__tinygrad__ · 2026-08-04 19:24
@KislayParashar1 @intel That's like when they take a dollar from the register and throw away a frozen banana to make the books balance.

@shawmakesmagic · 2026-08-04 17:53
@__tinygrad__ @intel Hey @__tinygrad__, want a shot at relevancy in AI? I know you still have those green boxes, sell one to us and we’ll build open source all-local Linux-based agent operating system running DeepSeek V4 Flash for the people. Good cultural test for Tiny Corp.

@__tinygrad__ · 2026-08-04 19:24
@shawmakesmagic @intel Order here! https://t.co/pTkQ44z96o

@__tinygrad__ · 2026-08-04 19:35
Get this machine for $53k today, and includes all the hardware to install two more GPUs when you are ready.

@__tinygrad__ · 2026-08-04 16:06
@adi_baradwaj lol they are legit the same people. it's not instrumental convergence, it's a couple whackos at the same parties in the same cult who think that any means justify their power-seeking ends.

@adi_baradwaj · 2026-08-04 21:13
@__tinygrad__ My point is that both companies have some version of "benefitting all of humanity" / "avoiding concentration of power" in their mission statements. Whether they're a cult or not, you'd expect their stated goals to mean something, yet apparently they don't

@AndrewCurran_ · 2026-08-04 23:24
Open-weight models will not be safety-tested under the new AI regulations. https://t.co/FMTjYaaUbe

@__tinygrad__ · 2026-08-04 23:43
@gwern This is the way.

@Bunagayafrost · 2026-08-05 00:13
USG "Hey China, can you submit your open models for testing before release?" China "Oh, go right ahead. It's available online for anyone to test" USG "...No I mean like submit it to us before releasing it online" China "Like I said, it's available online. Do you not know how to download open models? USG "Of course we do, but before publicly making it available for everyone we want you to submit it to us exclusively for safety testing" China "Oh, oh I see. No." USG "...If you don't cooperate we may have to restrict access to your models" China "How?" USG "By forbidding our enterprises from using them" China "OK, force them to pay for expensive tokens. I don't care. Nice talking to you" --- Staff "Sir China won't cooperate" President "Xi is a tough cookie" Staff "Sir, if we don't safety test Chinese open models, but safety test the domestic open source, we will be giving China an unfair advantage" President "Fine, just safety test the closed ones. They're the ones begging for it"

@__tinygrad__ · 2026-08-05 02:13
Got code exec on AMD 7900XTX last night, Kimi's custom MEC firmware just ran its first kernel! After spec and HCQ2, the next phase of tinygrad will be operating system. https://t.co/1nI9CrQiPG

@Bunagayafrost · 2026-08-05 00:13
USG "Hey China, can you submit your open models for testing before release?" China "Oh, go right ahead. It's available online for anyone to test" USG "...No I mean like submit it to us before releasing it online" China "Like I said, it's available online. Do you not know how to download open models? USG "Of course we do, but before publicly making it available for everyone we want you to submit it to us exclusively for safety testing" China "Oh, oh I see. No." USG "...If you don't cooperate we may have to restrict access to your models" China "How?" USG "By forbidding our enterprises from using them" China "OK, force them to pay for expensive tokens. I don't care. Nice talking to you" --- Staff "Sir China won't cooperate" President "Xi is a tough cookie" Staff "Sir, if we don't safety test Chinese open models, but safety test the domestic open source, we will be giving China an unfair advantage" President "Fine, just safety test the closed ones. They're the ones begging for it"

@__tinygrad__ · 2026-08-05 16:20
@Bunagayafrost Great outcome for the people.

@adi_baradwaj · 2026-08-04 21:13
@__tinygrad__ My point is that both companies have some version of "benefitting all of humanity" / "avoiding concentration of power" in their mission statements. Whether they're a cult or not, you'd expect their stated goals to mean something, yet apparently they don't

@__tinygrad__ · 2026-08-05 18:03
@adi_baradwaj Yea, whenever you see stuff like that it actually means "benefiting ourselves" / "concentrating all power to us"

@__tinygrad__ · 2026-08-06 16:01
In tinygrad, almost everything is a UOp. Here's three kernel invocations waiting to be compiled to a GPU command buffer. https://t.co/VGKlDREC4w

@__tinygrad__ · 2026-08-06 17:15
.@satyanadella This @github outage might be the final straw. Hours and hours of lost productivity cause GitHub actions is so frequently broken. I can't believe we pay and get this awful of an SLA. Where are people moving to?

@__tinygrad__ · 2026-08-06 17:15
.@satyanadella This @github outage might be the final straw. Hours and hours of lost productivity cause GitHub actions is so frequently broken. I can't believe we pay and get this awful of an SLA. Where are people moving to?

@snappercayt · 2026-08-06 17:25
@__tinygrad__ @satyanadella @github Self host brotha Why are you scared of ultimate freedom

@__tinygrad__ · 2026-08-06 17:15
.@satyanadella This @github outage might be the final straw. Hours and hours of lost productivity cause GitHub actions is so frequently broken. I can't believe we pay and get this awful of an SLA. Where are people moving to?

@GuyDeGrove · 2026-08-06 17:26
@__tinygrad__ @satyanadella @github Self host @gitlab ?

@snappercayt · 2026-08-06 17:25
@__tinygrad__ @satyanadella @github Self host brotha Why are you scared of ultimate freedom

@__tinygrad__ · 2026-08-06 17:28
@snappercayt @satyanadella @github who is testing the backups? who is responding when it goes down? who is making sure it's secure? oh wait, an LLM can do all this. the year of the self hosted server is here!

@GuyDeGrove · 2026-08-06 17:26
@__tinygrad__ @satyanadella @github Self host @gitlab ?

@__tinygrad__ · 2026-08-06 17:31
@GuyDeGrove @satyanadella @github @gitlab Is @giteaio better?

@__tinygrad__ · 2026-08-06 17:15
.@satyanadella This @github outage might be the final straw. Hours and hours of lost productivity cause GitHub actions is so frequently broken. I can't believe we pay and get this awful of an SLA. Where are people moving to?

@__tinygrad__ · 2026-08-06 17:47
@satyanadella @github We mostly aren't even using actions runners, we're using @namespacelabs cause they are twice as fast. GitHub can't even keep the hooks up. They need to delete every crap agent feature that nobody wants and focus on making a reliable service. Who is in charge of GitHub?

@__tinygrad__ · 2026-08-06 17:47
@satyanadella @github We mostly aren't even using actions runners, we're using @namespacelabs cause they are twice as fast. GitHub can't even keep the hooks up. They need to delete every crap agent feature that nobody wants and focus on making a reliable service. Who is in charge of GitHub?

@__tinygrad__ · 2026-08-06 19:20
@satyanadella @github @namespacelabs https://t.co/JZbQlnnKgS

@jackjoliet · 2026-07-30 18:26
adding to the list of things GH does instead of fixing Actions: - change menu icon to a stack of pancakes https://t.co/QEdjMyF0Bf

@__tinygrad__ · 2026-08-06 19:21
@jackjoliet the pancakes were offensive to me

@taalas_inc · 2026-08-06 20:10
We are pleased to share that Taalas has agreed to join AMD. We built Taalas to rethink AI inference from the ground up: hardware designed around the model, rather than the other way around. The result is the world's fastest and most cost-effective inference silicon. Joining AMD gives us the scale, engineering resources, and global reach to bring that work to the world – and to accelerate what comes after it. We are also excited to build on AMD's long-standing presence in Canada and its continued commitment to the country's AI ecosystem. Many of us grew up at AMD Canada, and are looking forward to coming home. We are proud of what this team has built, and even more excited about what comes next. https://t.co/b6VqdwJl8S

@AndrewCurran_ · 2026-08-10 16:17
Bernie Sanders has written a letter to Sam Altman, Dario Amodei, and Mark Zuckerberg urging them to immediately pause all AI development in the interest of humanity. And he warns if they do not take appropriate action now, the US Senate will. https://t.co/xwLB0FGmF9

@__tinygrad__ · 2026-08-10 17:11
We lower kernel dispatch to GPU command queues in the same way we lower Tensor programs to kernels. It's all programs, just sometimes they run on dispatch engines. https://t.co/l8jNx2QnZq

@__tinygrad__ · 2026-08-10 19:24
Here's Qwen 3.6 27B on an AMD 7900XTX over USB3 (literally any computer made in the last 10 years) at 34 tok/s. The eGPU dock that supports this launches on the 12th, with 100% open source firmware and an extra USB port for serial + unbrickability. https://t.co/UZ9subI5uh

@__tinygrad__ · 2026-08-10 19:24
Here's Qwen 3.6 27B on an AMD 7900XTX over USB3 (literally any computer made in the last 10 years) at 34 tok/s. The eGPU dock that supports this launches on the 12th, with 100% open source firmware and an extra USB port for serial + unbrickability. https://t.co/UZ9subI5uh

@VoxJT · 2026-08-10 20:25
@__tinygrad__ Can I attach this to my spark to run dense as a sidecar? 😂

@VoxJT · 2026-08-10 20:25
@__tinygrad__ Can I attach this to my spark to run dense as a sidecar? 😂

@__tinygrad__ · 2026-08-10 21:33
@VoxJT Of course. Anything with a USB3 port and Linux, macOS, or Windows.

@__tinygrad__ · 2026-08-10 19:24
Here's Qwen 3.6 27B on an AMD 7900XTX over USB3 (literally any computer made in the last 10 years) at 34 tok/s. The eGPU dock that supports this launches on the 12th, with 100% open source firmware and an extra USB port for serial + unbrickability. https://t.co/UZ9subI5uh

@thebasedcapital · 2026-08-10 22:07
@__tinygrad__ why it works: decode reads the weights from vram once per token, so the host link only hurts at load time. that's why usb3 costs you 5 tok/s instead of killing it. inference is link-tolerant in a way training never will be

@__tinygrad__ · 2026-08-10 19:24
Here's Qwen 3.6 27B on an AMD 7900XTX over USB3 (literally any computer made in the last 10 years) at 34 tok/s. The eGPU dock that supports this launches on the 12th, with 100% open source firmware and an extra USB port for serial + unbrickability. https://t.co/UZ9subI5uh

@BulbIndustry · 2026-08-10 22:28
@__tinygrad__ Isn't that kind of slow for an XTX? I haven't looked at AMD performance figures in a while, but I would have assumed with speculative decoding it would be capable of around double that.

@BulbIndustry · 2026-08-10 22:28
@__tinygrad__ Isn't that kind of slow for an XTX? I haven't looked at AMD performance figures in a while, but I would have assumed with speculative decoding it would be capable of around double that.

@__tinygrad__ · 2026-08-10 23:03
@BulbIndustry There's no speculative decoding in this llm runner, that's raw tokens. Feel free to /goal it after you get the hardware.

@thebasedcapital · 2026-08-10 22:07
@__tinygrad__ why it works: decode reads the weights from vram once per token, so the host link only hurts at load time. that's why usb3 costs you 5 tok/s instead of killing it. inference is link-tolerant in a way training never will be

@__tinygrad__ · 2026-08-10 23:04
@thebasedcapital It shouldn't even cost 5 tok/s, HCQ2 will mostly fix that (eta, 4 weeks).

@AndrewCurran_ · 2026-08-10 16:17
Bernie Sanders has written a letter to Sam Altman, Dario Amodei, and Mark Zuckerberg urging them to immediately pause all AI development in the interest of humanity. And he warns if they do not take appropriate action now, the US Senate will. https://t.co/xwLB0FGmF9

@__tinygrad__ · 2026-08-10 23:11
@AndrewCurran_ Ahh, more Anthropic marketing. Getting kind of old.

@const_ · 2026-08-11 21:29
wrote an inference server in tinygrad and it's going to prod let's goooooo

@__tinygrad__ · 2026-08-12 17:32
https://t.co/wR6cII8uW6

@pqqq · 2026-08-12 17:37
@__tinygrad__ Hey @__tinygrad__ 2PM which time zone?

@itsquinz · 2026-08-12 17:40
@pqqq @__tinygrad__ what part of PST is not clear?

@__tinygrad__ · 2026-08-12 17:32
https://t.co/wR6cII8uW6

@KinvertOG · 2026-08-12 17:49
@__tinygrad__ What's the game here, running inference? I'd imagine there's a bottleneck through USB4 for training.

@KinvertOG · 2026-08-12 17:49
@__tinygrad__ What's the game here, running inference? I'd imagine there's a bottleneck through USB4 for training.

@__tinygrad__ · 2026-08-12 17:52
@KinvertOG If you are doing training/inference on a single GPU, there's no bottleneck. The data is so small compared to the weights.

@itsquinz · 2026-08-12 17:40
@pqqq @__tinygrad__ what part of PST is not clear?

@__tinygrad__ · 2026-08-12 17:52
@itsquinz @pqqq I updated it with that!

@__tinygrad__ · 2026-08-12 17:32
https://t.co/wR6cII8uW6

@mrjack8969 · 2026-08-12 17:58
@__tinygrad__ Price just incase I need to wire from my money market to checking?

@mrjack8969 · 2026-08-12 17:58
@__tinygrad__ Price just incase I need to wire from my money market to checking?

@__tinygrad__ · 2026-08-12 18:03
@mrjack8969 Haha it's not that expensive.

@__tinygrad__ · 2026-08-12 17:32
https://t.co/wR6cII8uW6

@anemll · 2026-08-12 18:05
@__tinygrad__ TB5 wen

@__tinygrad__ · 2026-08-12 17:32
https://t.co/wR6cII8uW6

@puhcko · 2026-08-12 18:40
@__tinygrad__ @comma_ai Only 225W? I need to slang a 4090 at full tilt

@anemll · 2026-08-12 18:05
@__tinygrad__ TB5 wen

@__tinygrad__ · 2026-08-12 18:59
@anemll You need the bandwidth less than you think. If you are doing multi GPU stuff, even TB5 won't be enough. If you are doing single GPU stuff, even USB3 is fine.

@puhcko · 2026-08-12 18:40
@__tinygrad__ @comma_ai Only 225W? I need to slang a 4090 at full tilt

@__tinygrad__ · 2026-08-12 19:06
@puhcko @comma_ai You can use an ATX PSU with the extra GPU power connectors and get as much power as you want! ATX adapter included.

@__tinygrad__ · 2026-08-12 19:45
RT @comma_ai: 2pm Pacific with @__tinygrad__ https://t.co/0O0LWi8sk6

@__tinygrad__ · 2026-08-12 22:27
Order today, get your tiny chestnut eGPU dock shipped tomorrow for $249! It's the best way to plug your GPU into a Windows/Mac/Linux box over USB3 or USB4. https://t.co/tLoWFyX1GQ

@__tinygrad__ · 2026-08-12 22:27
Order today, get your tiny chestnut eGPU dock shipped tomorrow for $249! It's the best way to plug your GPU into a Windows/Mac/Linux box over USB3 or USB4. https://t.co/tLoWFyX1GQ

@__tinygrad__ · 2026-08-12 22:27
$799 "ready to drive" with a GPU and everything you need to plug it into a car (for comma users), or $249 for just the USB dock and accessiories. https://t.co/AIchpufh71

@__tinygrad__ · 2026-08-12 22:37
RT @___Harald___: If you liked what we've been able to do with an 8-year old phone chip, imagine what we can do with a full-size modern GPU. Super hyped about the chestnut launch, thanks hardware team for getting us more compute 🙏🙏 https://t.co/WuIOdarqYa

@__tinygrad__ · 2026-08-12 22:38
With tinygrad's custom kernel stuff you (or your LLM) can push inference speeds beyond every other framework.

@MiaAI_lab · 2026-08-12 23:02
Does this mean I plug a RTX 5090 into a DGX Spark with this @__tinygrad__ ?

@MiaAI_lab · 2026-08-12 23:02
Does this mean I plug a RTX 5090 into a DGX Spark with this @__tinygrad__ ?

@__tinygrad__ · 2026-08-12 23:37
@MiaAI_lab Yes!

@__tinygrad__ · 2026-08-12 23:52
Every tiny chestnut eGPU dock is tested at both USB3 and USB4 speeds, green sticker means it's good. Buy one on comma's website today for $249 https://t.co/hMXtpfbYFy

@MiaAI_lab · 2026-08-12 23:02
Does this mean I plug a RTX 5090 into a DGX Spark with this @__tinygrad__ ?

@codes_cookie · 2026-08-13 00:17
@MiaAI_lab @__tinygrad__ Bandwidth... 20gbps. That model will be crawling

@__tinygrad__ · 2026-08-13 01:07
RT @phoronix: Neat! @comma_ai @__tinygrad__ Launches A PCIe Gen4 x4 To USB4 Dock With Open-Source Firmware https://t.co/YPcJOWt1aM

@codes_cookie · 2026-08-13 00:17
@MiaAI_lab @__tinygrad__ Bandwidth... 20gbps. That model will be crawling

@__tinygrad__ · 2026-08-13 03:20
@codes_cookie @MiaAI_lab lol neither prefill nor decode have anything to do with slot bandwidth. you could have a dial up modem connecting to the GPU and with good software it would be the same speed.

@comma_ai · 2026-08-13 21:09
First chestnuts shipping today. Join the chestnut class today for $799 at https://t.co/pUcX5vtdk3. https://t.co/QYGSKYTuLj

@__tinygrad__ · 2026-08-13 21:51
Or $250 for the tiny edition without the GPU!

@__tinygrad__ · 2026-08-13 21:51
Or $250 for the tiny edition without the GPU!

@comma_ai · 2026-08-13 21:55
@__tinygrad__ *$249

@comma_ai · 2026-08-13 21:55
@__tinygrad__ *$249

@__tinygrad__ · 2026-08-13 21:55
@comma_ai Wow that's almost too good of a deal

@__tinygrad__ · 2026-08-13 22:53
NVIDIA just raised prices again, it's $15k for the cards :( https://t.co/BYrOpOTLAR

@__tinygrad__ · 2026-08-13 23:01
Like this is just getting stupid. We'll honor the orders already placed at the old price. We can't even get a price from our supplier for 5090s anymore, they are just "Market Price" On the plus side, AMD sent us MI350P samples which we'll test out. More powerful than Blackwell!

@EviAntonWorld · 2026-08-13 23:03
@__tinygrad__ How much would you estimate a tinybox w the AMD cards would cost? it sounds really good

@EviAntonWorld · 2026-08-13 23:03
@__tinygrad__ How much would you estimate a tinybox w the AMD cards would cost? it sounds really good

@__tinygrad__ · 2026-08-13 23:04
@ReniNoColor Still $12k for this one, no price increase there. https://t.co/RTJBUIW95Y

@__tinygrad__ · 2026-08-14 04:25
RT @KBlueleaf: KohakuTPU (An ai accelerator fully designed by me and run on xilinx FPGA) is now running with tinygrad! As the memory system + NoC is generic for any type of accelerator, the project will be renamed as KohakuAccel and I will explore more different accelerator! https://t.co/a30Pk4gFyB

@KBlueleaf · 2026-08-13 15:06
KohakuTPU (An ai accelerator fully designed by me and run on xilinx FPGA) is now running with tinygrad! As the memory system + NoC is generic for any type of accelerator, the project will be renamed as KohakuAccel and I will explore more different accelerator! https://t.co/a30Pk4gFyB

@__tinygrad__ · 2026-08-14 04:25
@KBlueleaf Open source?

@__tinygrad__ · 2026-08-14 06:12
RT @Zai_org: Introducing GLM-5.3: Built to Code. Ready for Cyber Defense. - Top-tier coding and agentic capabilities, achieved through post-training on the 743B base model - A major leap in cybersecurity, setting a new standard among open models Tech Blog: https://t.co/ekQkO83jCv https://t.co/y3Y2AB0wxr

@Zai_org · 2026-08-14 05:17
Introducing GLM-5.3: Built to Code. Ready for Cyber Defense. - Top-tier coding and agentic capabilities, achieved through post-training on the 743B base model - A major leap in cybersecurity, setting a new standard among open models Tech Blog: https://t.co/ekQkO83jCv https://t.co/y3Y2AB0wxr

@__tinygrad__ · 2026-08-14 06:13
@Zai_org Thank you for bringing Mythos-class cyber capabilities to the people! Excited to upgrade my GLM-5.2 instance

@__tinygrad__ · 2026-08-15 06:21
tinygrad + Qwen3.8-27B + pi made this when asked to advertise itself https://t.co/5zFXIKpEPT

@KBlueleaf · 2026-08-15 07:57
Did a quick estimation, I can reach at max 10~15TFlops on vu13p with kohakuTPU (mxfp7 matmul) But if I have Alveo V80 I can reach 30~40TFlops or even higher! Really thinking if I should buy some tenstorrent to play NoC programming or save more money to buy V80

@__tinygrad__ · 2026-08-15 06:21
tinygrad + Qwen3.8-27B + pi made this when asked to advertise itself https://t.co/5zFXIKpEPT

@markpdx724 · 2026-08-15 12:45
@__tinygrad__ Is this running well on a R9700 now? Seems like 32gb would be tight for that.

@markpdx724 · 2026-08-15 12:45
@__tinygrad__ Is this running well on a R9700 now? Seems like 32gb would be tight for that.

@__tinygrad__ · 2026-08-15 14:38
@markpdx724 This was run on a 7900XTX. I don't like the R9700, the blower fan is loud! AMD should make a 9070XTX with 32GB of RAM.

@__tinygrad__ · 2026-08-15 19:25
lol remember that time there were people who thought only they could be trusted to be the gatekeepers of AI but then the Chinese gave it away to everyone for free? remember who they were for next time.

@__tinygrad__ · 2026-08-15 19:25
lol remember that time there were people who thought only they could be trusted to be the gatekeepers of AI but then the Chinese gave it away to everyone for free? remember who they were for next time.

@Andrew42634355 · 2026-08-15 19:28
@__tinygrad__ Free for anyone with enough money to run it

@__tinygrad__ · 2026-08-15 19:25
lol remember that time there were people who thought only they could be trusted to be the gatekeepers of AI but then the Chinese gave it away to everyone for free? remember who they were for next time.

@adamac · 2026-08-15 19:32
@__tinygrad__ They aren’t giving up

@adamac · 2026-08-15 19:32
@__tinygrad__ They aren’t giving up

@__tinygrad__ · 2026-08-15 19:36
@adamac It's important to stay vigilant. Help your friends move over to Kimi and GLM. Explain to the investors you know that they are not early and are investing in the next FTX.

@Andrew42634355 · 2026-08-15 19:28
@__tinygrad__ Free for anyone with enough money to run it

@__tinygrad__ · 2026-08-15 20:09
@Andrew42634355 Very happy to send money to @Zai_org and @Kimi_Moonshot. Cancel plans from companies that don't support open source. https://t.co/doltWsY7Q7

@KBlueleaf · 2026-08-15 21:42
@__tinygrad__ It can, just very small. (Vu33p only have around 1/4 resources as vu13p). Based on the resources count I think 8mat core + 2 vec core is hard limit. (my vu13p setup use 24+6 right now)

@_flammit · 2026-08-15 23:07
@KBlueleaf @__tinygrad__ U50 are pretty cheap or U55c occasionally too. I’d send a U50 to the tinygrad guys if it’s useful to you.

@_flammit · 2026-08-15 23:07
@KBlueleaf @__tinygrad__ U50 are pretty cheap or U55c occasionally too. I’d send a U50 to the tinygrad guys if it’s useful to you.

@__tinygrad__ · 2026-08-16 02:27
@_flammit @KBlueleaf Sure, we'll stand up an FPGA computer for developers to use and put whatever cards we have in there.

@DKokotajlo · 2026-08-16 20:52
Our timelines update is finally done: https://t.co/2R47tCtOFI Apologies for lateness!

@teortaxesTex · 2026-08-16 22:29
Oh God. It’s so quaint that they still treat this as a matter of timelines and China being slow to “wake up”, not a strategic disagreement Quaint but getting annoying https://t.co/suoaWto6Os

@__tinygrad__ · 2026-08-17 14:05
We really can't move to our self hosted Gitea soon enough. How is this an enterprise service people rely on? @Microsoft @github https://t.co/cC320Fm4oI

@__tinygrad__ · 2026-08-17 14:46
The @AMD MI350P is real! https://t.co/dhyuOe92y7

@teortaxesTex · 2026-08-16 22:29
Oh God. It’s so quaint that they still treat this as a matter of timelines and China being slow to “wake up”, not a strategic disagreement Quaint but getting annoying https://t.co/suoaWto6Os

@__tinygrad__ · 2026-08-17 14:51
@teortaxesTex It's almost like China is smarter than them, understands the real dangers AI can bring, and is taking action to prevent bad outcomes.

@__tinygrad__ · 2026-08-17 14:46
The @AMD MI350P is real! https://t.co/dhyuOe92y7

@guitaripod · 2026-08-17 15:01
@__tinygrad__ @AMD what. does. it. cost. buddeh.

@guitaripod · 2026-08-17 15:01
@__tinygrad__ @AMD what. does. it. cost. buddeh.

@__tinygrad__ · 2026-08-17 15:09
@guitaripod @AMD This is an excellent question that we don't know the answer to. I'm hoping they build an active cooled version and price it lower than RTX 6000. Could gain some real traction that way.

@__tinygrad__ · 2026-08-17 14:46
The @AMD MI350P is real! https://t.co/dhyuOe92y7

@Technop54777070 · 2026-08-17 15:48
@__tinygrad__ @AMD Thank god, competition lol. I don't care if it's not as good as a NVIDIA chip just as long as it's not F#%@ing $15000

@Technop54777070 · 2026-08-17 15:48
@__tinygrad__ @AMD Thank god, competition lol. I don't care if it's not as good as a NVIDIA chip just as long as it's not F#%@ing $15000

@__tinygrad__ · 2026-08-17 15:50
@Technop54777070 @AMD It's better than the NVIDIA card. $10k would be a nice price for it, though I don't know what they cost to make. It's HBM not GDDR.

@__tinygrad__ · 2026-08-17 22:14
Is this your chestnut being tested? https://t.co/qb70rre6rm

@__tinygrad__ · 2026-08-17 22:14
Is this your chestnut being tested? https://t.co/qb70rre6rm

@turkonthelurk · 2026-08-17 22:16
@__tinygrad__ geohot could post a toaster and 2 7900 XTXs wired together with caption "toasting chestnuts" and get 2k likes from people pretending they understand. I don't know what chestnut is. Please use words

@turkonthelurk · 2026-08-17 22:16
@__tinygrad__ geohot could post a toaster and 2 7900 XTXs wired together with caption "toasting chestnuts" and get 2k likes from people pretending they understand. I don't know what chestnut is. Please use words

@__tinygrad__ · 2026-08-17 22:19
@turkonthelurk Ahh, a chestnut you say? They are $249! https://t.co/AIchpufh71

@__tinygrad__ · 2026-08-17 22:14
Is this your chestnut being tested? https://t.co/qb70rre6rm

@MRpoproppo · 2026-08-17 22:21
@__tinygrad__ Good to know mine will work. Getting in Wednesday

@MRpoproppo · 2026-08-17 22:21
@__tinygrad__ Good to know mine will work. Getting in Wednesday

@__tinygrad__ · 2026-08-17 22:23
@MRpoproppo If we put the same level of work into marketing as testing I don't know what we'd do with all the money.

@__tinygrad__ · 2026-08-18 03:13
The custom MEC firmware for 7900XTX is ready, mostly written by Kimi. Unless you have a private key or an exploit, you can't run it on hardware, but this is a working replacement for the stock MEC firmware and it runs in the included emulator. https://t.co/DQ7dyJshQF

@__tinygrad__ · 2026-08-18 03:13
The custom MEC firmware for 7900XTX is ready, mostly written by Kimi. Unless you have a private key or an exploit, you can't run it on hardware, but this is a working replacement for the stock MEC firmware and it runs in the included emulator. https://t.co/DQ7dyJshQF

@__tinygrad__ · 2026-08-18 03:14
Code is here: https://t.co/84GwauDYdP

@__tinygrad__ · 2026-08-17 14:46
The @AMD MI350P is real! https://t.co/dhyuOe92y7

@__tinygrad__ · 2026-08-18 06:31
@AMD With pci=nocrs + a patch for optional BIOS, it works! https://t.co/ZymUQeGQMd

@__tinygrad__ · 2026-08-18 06:31
@AMD With pci=nocrs + a patch for optional BIOS, it works! https://t.co/ZymUQeGQMd

@__tinygrad__ · 2026-08-18 06:44
@AMD Working in tinygrad, 2.5 PFLOPS MXFP4 https://t.co/aFtp1Cw9vS

@__tinygrad__ · 2026-08-18 03:13
The custom MEC firmware for 7900XTX is ready, mostly written by Kimi. Unless you have a private key or an exploit, you can't run it on hardware, but this is a working replacement for the stock MEC firmware and it runs in the included emulator. https://t.co/DQ7dyJshQF

@BobMcElrath · 2026-08-18 11:05
@__tinygrad__ OMG I love you. I've been fighting this for months, decompiling the MEC/MES/SMU firmwares, trying to write kernel workarounds, etc on my 8x 7900xtx system. I even reimplemented tinygrad's VFIO path in Rust. Have you been talking with AMD and can you get them to fix this upstream?

@WSJ · 2026-08-18 11:27
Exclusive: AI chip startup Etched, founded by Harvard dropouts, has its own in-office data center and has signed quant-trading firm Jane Street as its first customer https://t.co/Tyg12UgRH4

@AnushElangovan · 2026-08-18 14:56
3🚀5🚀0🚀p💪🏾

@Etched · 2026-08-18 15:00
We've raised $700M at a $21B valuation from Jane Street, Kleiner Perkins, Sequoia, A16Z, Peter Thiel, BCV, and Blackstone. We're also excited to share that we've shipped our first rack to Jane Street. https://t.co/MFlF2xSw0B

@Etched · 2026-08-18 15:00
We've raised $700M at a $21B valuation from Jane Street, Kleiner Perkins, Sequoia, A16Z, Peter Thiel, BCV, and Blackstone. We're also excited to share that we've shipped our first rack to Jane Street. https://t.co/MFlF2xSw0B

@sumersao · 2026-08-18 15:02
@Etched $0.021T valuation! We're so early

@sonyatweetybird · 2026-08-18 15:11
I grew up in the semiconductor industry, a notoriously closed off market where experience is the primary currency and the old guard is famously unwelcome of newcomers. It’s been amazing to see @robertwachen @UbertiGavin @czhu1729 and the @Etched team run circles around the rest of the market. The “kids in chips” are just unbelievably good. They’ve gone from being industry outsiders to recruiting the crème de la crème of the semis world. Etched not only had a successful A0 this year but has already delivered their first customer rack to @JaneStreetGroup, who then decided to lead this round. Congratulations to all. Generational company in the making. Can’t wait to see you power the world’s inference and maximize intelligence per watt. 💡🍪

@sumersao · 2026-08-18 15:11
babe wake up, Etched doubled its valuation again

@sarahdingwang · 2026-08-18 15:13
You have to be a little bit crazy to build a new chip from scratch. @UbertiGavin and @robertwachen decided to do this in 2022. 4 years later, @Etched has shipped their first deployment to Jane Street, making them the only new company since ChatGPT released to put a brand new chip *in customers’ hands*. Astonishing speed in a very hard category. Here’s to the crazy ones!

@Etched · 2026-08-18 15:00
We've raised $700M at a $21B valuation from Jane Street, Kleiner Perkins, Sequoia, A16Z, Peter Thiel, BCV, and Blackstone. We're also excited to share that we've shipped our first rack to Jane Street. https://t.co/MFlF2xSw0B

@sarahdingwang · 2026-08-18 15:17
@Etched Talk is cheap. Shipping is everything.

@patrick_oshag · 2026-08-18 15:19
I recently spent several days onsite at Etched. It was the most exciting and energizing business environment I’ve ever been in. I left inspired, and wondering: “have I even begun to explore my personal potential?” People at Etched are exploring theirs. They have open roles across the org. I believe deeply it’s a place you can do your life’s work. One of their mottos is “Production is the product.” The team is off the charts talented, but they need even more great people to fulfill the mission. As they said, “The racks won’t build themselves.” You should join them.

@mattshumer_ · 2026-08-18 15:21
Etched has shipped!! Insanely proud (small) seed investor. >$1T valuation incoming.

@ypatil125 · 2026-08-18 15:26
HUGE! So bullish on @robertwachen @UbertiGavin and team. This product velocity in hardware should not go unnoticed.

@mamoonha · 2026-08-18 15:30
We're thrilled @kleinerperkins to back @etched - the fastest company to ship production AI chips from scratch https://t.co/APZjcyQKeD

@talbroda · 2026-08-18 15:31
Absolutely epic! ✨ Was lucky enough to be at @Etched's office when the 1st rack shipped and the energy was amazing. ⚡⚡⚡ Congrats @robertwachen, @UbertiGavin , @saptadeep_pal and the whole team. You earned this moment. Onwards! 🚀

@Etched · 2026-08-18 15:00
We've raised $700M at a $21B valuation from Jane Street, Kleiner Perkins, Sequoia, A16Z, Peter Thiel, BCV, and Blackstone. We're also excited to share that we've shipped our first rack to Jane Street. https://t.co/MFlF2xSw0B

@AlfredZSpector · 2026-08-18 15:34
@Etched Etched has done incredibly good architetural design for the compute bottleneck of the decade and perhaps beyond. And the Etched team has executed amazingly well to get product out the door!

@Modular · 2026-08-18 16:23
Today, we open sourced Mojo 🔥. Announced just now during the ModCon keynote, effective immediately, Apache 2.0 License. Thank you to our community for waiting patiently and building alongside us. #ModCon2026 Full blog: https://t.co/y5cSUsohrs https://t.co/uJJkON4m28

@AnushElangovan · 2026-08-18 14:56
3🚀5🚀0🚀p💪🏾

@__tinygrad__ · 2026-08-18 16:47
@AnushElangovan The question everyone is asking, how much do they cost? At the right price this can look really attractive compared to NVIDIA RTX PRO 6000 Blackwell.

@BobMcElrath · 2026-08-18 11:05
@__tinygrad__ OMG I love you. I've been fighting this for months, decompiling the MEC/MES/SMU firmwares, trying to write kernel workarounds, etc on my 8x 7900xtx system. I even reimplemented tinygrad's VFIO path in Rust. Have you been talking with AMD and can you get them to fix this upstream?

@__tinygrad__ · 2026-08-18 16:49
@BobMcElrath There's some race condition nobody ever root caused with the RDNA3 chips and compute. Can't say we know what the cause is either, but the tinygrad driver + the firmware we pinned doesn't seem to have any stability issues. The custom MEC is just for fun.

@Etched · 2026-08-18 15:00
We've raised $700M at a $21B valuation from Jane Street, Kleiner Perkins, Sequoia, A16Z, Peter Thiel, BCV, and Blackstone. We're also excited to share that we've shipped our first rack to Jane Street. https://t.co/MFlF2xSw0B

@__tinygrad__ · 2026-08-18 16:51
@Etched I see you spent a lot of investor money on a video shoot, but there's very little details. What's in the rack? I know it's not your chip. Is it NVIDIA chips? Are there any chips at all?

@sumersao · 2026-08-18 15:11
babe wake up, Etched doubled its valuation again

@__tinygrad__ · 2026-08-18 16:54
@sumersao https://t.co/ve9KMwF0ly

@sumersao · 2026-08-18 15:02
@Etched $0.021T valuation! We're so early

@__tinygrad__ · 2026-08-18 16:54
@sumersao @Etched https://t.co/yOuSkLk0FU

@bubbleboi · 2026-08-18 16:57
Man they are mogging soooooo hard, this is literally the most amazing chip I’ve ever seen.

@sarahdingwang · 2026-08-18 15:17
@Etched Talk is cheap. Shipping is everything.

@__tinygrad__ · 2026-08-18 16:58
@sarahdingwang @Etched What exactly did they ship, besides a video? Note how this mimics the style of success without any technical detail. Does a16z fall for that? Are you trying to spot real winners or just get in early?

@nikitabier · 2026-08-18 16:43
@Etched Congrats guys, enjoyed the demo of the upcoming rack. Remarkable progress.

@__tinygrad__ · 2026-08-18 16:59
@nikitabier @Etched Exactly what demo did you see? The one with the tokens and interactivity? Think critically about it.

@AlfredZSpector · 2026-08-18 15:34
@Etched Etched has done incredibly good architetural design for the compute bottleneck of the decade and perhaps beyond. And the Etched team has executed amazingly well to get product out the door!

@__tinygrad__ · 2026-08-18 17:00
@AlfredZSpector @Etched This is your first and only tweet. You joined X to hype Etched?

@tri_dao · 2026-08-18 17:01
These guys move fast, 1st rack already ships. More inference compute is always welcome

@noMukh · 2026-08-18 17:02
@bubbleboi I’m a hardware noob & my references are divided on this company. can y’all come to consensus pls

@bubbleboi · 2026-08-18 17:06
@noMukh That guy is 36 still complaining about women not liking him and whining like a teenager. Dude did not live up to expectations.

@sarahdingwang · 2026-08-18 15:13
You have to be a little bit crazy to build a new chip from scratch. @UbertiGavin and @robertwachen decided to do this in 2022. 4 years later, @Etched has shipped their first deployment to Jane Street, making them the only new company since ChatGPT released to put a brand new chip *in customers’ hands*. Astonishing speed in a very hard category. Here’s to the crazy ones!

@__tinygrad__ · 2026-08-18 17:07
@sarahdingwang @UbertiGavin @robertwachen @Etched Do you personally have carry in Etched?

@sonyatweetybird · 2026-08-18 15:11
I grew up in the semiconductor industry, a notoriously closed off market where experience is the primary currency and the old guard is famously unwelcome of newcomers. It’s been amazing to see @robertwachen @UbertiGavin @czhu1729 and the @Etched team run circles around the rest of the market. The “kids in chips” are just unbelievably good. They’ve gone from being industry outsiders to recruiting the crème de la crème of the semis world. Etched not only had a successful A0 this year but has already delivered their first customer rack to @JaneStreetGroup, who then decided to lead this round. Congratulations to all. Generational company in the making. Can’t wait to see you power the world’s inference and maximize intelligence per watt. 💡🍪

@__tinygrad__ · 2026-08-18 17:08
@sonyatweetybird @robertwachen @UbertiGavin @czhu1729 @Etched Do you personally have carry in Etched?

@__tinygrad__ · 2026-08-18 17:03
@bubbleboi This isn't a chip. It's a video of a wooden box getting forklifted combined with name dropping and a narrative about some alleged chip without any details.

@bubbleboi · 2026-08-18 17:09
@__tinygrad__ You’re literally 36 packaging other people’s GPUs into a server and charging an arm and a leg. That’s what we have Supermicro for. Also, I’ve seen your RTL and it sucks ass so you’re not the person to talk. When the details do come out you will look like a dufus.

@bubbleboi · 2026-08-18 17:06
@noMukh That guy is 36 still complaining about women not liking him and whining like a teenager. Dude did not live up to expectations.

@__tinygrad__ · 2026-08-18 17:11
@bubbleboi @noMukh lol full disclosure, do you have a financial stake in Etched? some people just do it for free you know, I believe the name for them is "useful idiots"

@bubbleboi · 2026-08-18 17:09
@__tinygrad__ You’re literally 36 packaging other people’s GPUs into a server and charging an arm and a leg. That’s what we have Supermicro for. Also, I’ve seen your RTL and it sucks ass so you’re not the person to talk. When the details do come out you will look like a dufus.

@__tinygrad__ · 2026-08-18 17:13
@bubbleboi We don't have any RTL, so I'm not sure what you saw. The main problem right now isn't a new chip to be designed, it's chip manufacturing capacity. There's a lot more value in creating software that works on all the chips that are being made.

@__tinygrad__ · 2026-08-18 16:59
@nikitabier @Etched Exactly what demo did you see? The one with the tokens and interactivity? Think critically about it.

@__tinygrad__ · 2026-08-18 17:16
@nikitabier @Etched Also, and I posted this a lot last time, but I see you on the list of advisors for Etched. What's the financial arrangement look like here? https://t.co/GFq2OnI1Vd

@__tinygrad__ · 2026-08-18 17:13
@bubbleboi We don't have any RTL, so I'm not sure what you saw. The main problem right now isn't a new chip to be designed, it's chip manufacturing capacity. There's a lot more value in creating software that works on all the chips that are being made.

@bubbleboi · 2026-08-18 17:17
@__tinygrad__ I agree. So start working on some etched kernels sir, more than happy to get you a box.

@bubbleboi · 2026-08-18 17:17
@__tinygrad__ I agree. So start working on some etched kernels sir, more than happy to get you a box.

@__tinygrad__ · 2026-08-18 17:21
@bubbleboi I mean, it's a pretty big jump from not even a basic spec sheet with FLOPS, memory bandwidth, and power to a box. But if by some miracle you do send me a box, I promise I'll review it fairly.

@__tinygrad__ · 2026-08-18 17:06
@patrick_oshag Are you on Etched's payroll? I saw you hyping them last time too. I don't see this marked as an ad, so I'm assuming not. If you are, you might want to mark it as an ad in case this all unravels. Lessons learned from fyre fest influencers.

@gjpx_ · 2026-08-18 17:22
@__tinygrad__ @patrick_oshag They are doing exactly this shit 😆😆😆 https://t.co/ZbPwRL57Le

@gjpx_ · 2026-08-18 17:22
@__tinygrad__ @patrick_oshag They are doing exactly this shit 😆😆😆 https://t.co/ZbPwRL57Le

@__tinygrad__ · 2026-08-18 17:24
@gjpx_ @patrick_oshag IBM included a spec about the hard drive.

@tri_dao · 2026-08-18 17:01
These guys move fast, 1st rack already ships. More inference compute is always welcome

@__tinygrad__ · 2026-08-18 17:28
@tri_dao Is it time to pump Etched again? Are the Etched advisors all in a Signal chat or something? https://t.co/wODLQ3evfe

@__tinygrad__ · 2026-08-18 17:32
@mattshumer_ This is a video of a wooden box getting forklifted into a truck with a bunch of name dropping. This is not shipping a chip, this is a marketing campaign.

@mattshumer_ · 2026-08-18 17:36
@__tinygrad__ George, I’m a huge fan (I watched your videos in high school). They actually shipped!

@__tinygrad__ · 2026-08-18 17:03
@bubbleboi This isn't a chip. It's a video of a wooden box getting forklifted combined with name dropping and a narrative about some alleged chip without any details.

@steeve · 2026-08-18 17:39
@__tinygrad__ @bubbleboi yeah so I've seen the chip with my own eyes so I'm not sure why you're hating?

@mattshumer_ · 2026-08-18 17:36
@__tinygrad__ George, I’m a huge fan (I watched your videos in high school). They actually shipped!

@__tinygrad__ · 2026-08-18 17:41
@mattshumer_ How do you know what you think you know? The limited technical claims in the last round were total nonsense, and I see 0 this time. Remember, Theranos actually built and shipped blood testing machines also. https://t.co/WNDNSTsTGj

@steeve · 2026-08-18 17:39
@__tinygrad__ @bubbleboi yeah so I've seen the chip with my own eyes so I'm not sure why you're hating?

@__tinygrad__ · 2026-08-18 17:44
@steeve @bubbleboi Again, there probably is a chip, I'm not saying it doesn't exist at all. Theranos also had blood testing machines. There's been very little technical information released, and what has been released is nonsense. https://t.co/WNDNSTsTGj

@__tinygrad__ · 2026-08-18 16:51
@Etched I see you spent a lot of investor money on a video shoot, but there's very little details. What's in the rack? I know it's not your chip. Is it NVIDIA chips? Are there any chips at all?

@DBillionaer · 2026-08-18 17:54
@__tinygrad__ @Etched Little? You mean no details at all

@talbroda · 2026-08-18 15:31
Absolutely epic! ✨ Was lucky enough to be at @Etched's office when the 1st rack shipped and the energy was amazing. ⚡⚡⚡ Congrats @robertwachen, @UbertiGavin , @saptadeep_pal and the whole team. You earned this moment. Onwards! 🚀

@__tinygrad__ · 2026-08-18 17:55
@talbroda @Etched @robertwachen @UbertiGavin @saptadeep_pal I see you on the advisor list for Etched. What's the financial arrangement here? https://t.co/IRmtAQZfRA

@__tinygrad__ · 2026-08-18 17:21
@bubbleboi I mean, it's a pretty big jump from not even a basic spec sheet with FLOPS, memory bandwidth, and power to a box. But if by some miracle you do send me a box, I promise I'll review it fairly.

@bubbleboi · 2026-08-18 18:00
@__tinygrad__ Let’s do it.

@bubbleboi · 2026-08-18 18:00
@__tinygrad__ Let’s do it.

@__tinygrad__ · 2026-08-18 18:01
@bubbleboi Great, when can I expect a box?

@WSJ · 2026-08-18 11:27
Exclusive: AI chip startup Etched, founded by Harvard dropouts, has its own in-office data center and has signed quant-trading firm Jane Street as its first customer https://t.co/Tyg12UgRH4

@__tinygrad__ · 2026-08-18 18:05
@WSJ I remember another WSJ exclusive. Did you do any investigative journalism here or just put "Exclusive" in front of the Etched press release? https://t.co/DMNFLCz9xP

@mamoonha · 2026-08-18 15:30
We're thrilled @kleinerperkins to back @etched - the fastest company to ship production AI chips from scratch https://t.co/APZjcyQKeD

@__tinygrad__ · 2026-08-18 18:08
@mamoonha @kleinerperkins @Etched Do you personally have carry in Etched?

@__tinygrad__ · 2026-08-18 18:10
@ypatil125 @robertwachen @UbertiGavin I see your name on here. Paid advisor? https://t.co/sPrjhFFx5T

@ypatil125 · 2026-08-18 18:11
@__tinygrad__ @robertwachen @UbertiGavin What they are doing is impressive! Whats the issue?

@__tinygrad__ · 2026-08-18 18:01
@bubbleboi Great, when can I expect a box?

@cs_serdar · 2026-08-18 18:11
@__tinygrad__ @bubbleboi Did you get a DM? Please keep us updated

@ypatil125 · 2026-08-18 18:11
@__tinygrad__ @robertwachen @UbertiGavin What they are doing is impressive! Whats the issue?

@__tinygrad__ · 2026-08-18 18:14
@ypatil125 @robertwachen @UbertiGavin I have no idea what they are doing. All I have seen is a large coordinated marketing campaign hyping them like you are doing with 0 technical details. Care to disclose the full extent of your advisorship?

@__tinygrad__ · 2026-08-18 17:16
@nikitabier @Etched Also, and I posted this a lot last time, but I see you on the list of advisors for Etched. What's the financial arrangement look like here? https://t.co/GFq2OnI1Vd

@ChiragAsarpota · 2026-08-18 18:14
@__tinygrad__ @nikitabier @Etched It's quite interesting that none of the advisors have tried to respond and debunk your opinion...speaks volumes

@Modular · 2026-08-18 16:23
Today, we open sourced Mojo 🔥. Announced just now during the ModCon keynote, effective immediately, Apache 2.0 License. Thank you to our community for waiting patiently and building alongside us. #ModCon2026 Full blog: https://t.co/y5cSUsohrs https://t.co/uJJkON4m28

@__tinygrad__ · 2026-08-18 18:29
@Modular Congrats!

@DBillionaer · 2026-08-18 17:54
@__tinygrad__ @Etched Little? You mean no details at all

@__tinygrad__ · 2026-08-18 18:31
@DBillionaer @Etched No but like, it's really good and Etched moves really fast. The energy ⚡⚡⚡ Look at this famous person. They think Etched is great! So does this other famous person who also invested! OMG look at our big investor Jane Street, they also think we are great!

@cs_serdar · 2026-08-18 18:11
@__tinygrad__ @bubbleboi Did you get a DM? Please keep us updated

@__tinygrad__ · 2026-08-18 18:34
@cs_serdar @bubbleboi I did not and I doubt I will. There's so many people they could send a box to like @ChipsandCheese9 and @SemiAnalysis_ if they wanted real credibility. But nope, it's just forklift video with zoom on logo.

@ChiragAsarpota · 2026-08-18 18:14
@__tinygrad__ @nikitabier @Etched It's quite interesting that none of the advisors have tried to respond and debunk your opinion...speaks volumes

@__tinygrad__ · 2026-08-18 18:39
@ChiragAsarpota @nikitabier @Etched This one was the closest, but it opened with "This is stupid and you should feel bad." ad hominem. https://t.co/IyWrBYkpFW

@steeve · 2026-08-18 19:06
@__tinygrad__ @bubbleboi are you butthurt because they're not talking to you?

@__tinygrad__ · 2026-08-18 19:16
@steeve @bubbleboi I don't care if they talk to me. All they need to do is send their supposedly revolutionary chip to any third party independent reviewer, and yet... It's a disgusting marketing campaign of nonstop hype, and a lot of people will have egg on their face when this blows up.

@__tinygrad__ · 2026-08-18 19:16
@steeve @bubbleboi I don't care if they talk to me. All they need to do is send their supposedly revolutionary chip to any third party independent reviewer, and yet... It's a disgusting marketing campaign of nonstop hype, and a lot of people will have egg on their face when this blows up.

@__tinygrad__ · 2026-08-18 19:18
@steeve @bubbleboi I just want there to be receipts. I want the people who uncritically promoted this to not be able to say they didn't know better. I want to make it harder for companies to do this in the future. It's bad for the space and it's bad for technology in general.

@__tinygrad__ · 2026-08-18 19:18
@steeve @bubbleboi I just want there to be receipts. I want the people who uncritically promoted this to not be able to say they didn't know better. I want to make it harder for companies to do this in the future. It's bad for the space and it's bad for technology in general.

@steeve · 2026-08-18 19:24
@__tinygrad__ @bubbleboi idk man i'm fairly close to them and watching you obsess like that, it feels i'm learning more about you than anything else

@steeve · 2026-08-18 19:24
@__tinygrad__ @bubbleboi idk man i'm fairly close to them and watching you obsess like that, it feels i'm learning more about you than anything else

@__tinygrad__ · 2026-08-18 20:24
@steeve @bubbleboi More ad hominem. Why don't you post some third party benchmarks of the Etched chip instead?

@__tinygrad__ · 2026-08-18 20:24
@steeve @bubbleboi More ad hominem. Why don't you post some third party benchmarks of the Etched chip instead?

@steeve · 2026-08-18 20:27
@__tinygrad__ @bubbleboi you just compared them to theranos quit it with the ad hominem complaints i'll be sure to post some when we run on it

@steeve · 2026-08-18 20:27
@__tinygrad__ @bubbleboi you just compared them to theranos quit it with the ad hominem complaints i'll be sure to post some when we run on it

@__tinygrad__ · 2026-08-18 20:31
@steeve @bubbleboi Do you have a chip in your possession? I'm not saying a chip you supposedly saw at their office, I mean a chip on your desk. Do you have anything running on it now?

@__tinygrad__ · 2026-08-18 20:31
@steeve @bubbleboi Do you have a chip in your possession? I'm not saying a chip you supposedly saw at their office, I mean a chip on your desk. Do you have anything running on it now?

@steeve · 2026-08-18 20:34
@__tinygrad__ @bubbleboi i'll make sure to let you know when i do

@steeve · 2026-08-18 20:34
@__tinygrad__ @bubbleboi i'll make sure to let you know when i do

@__tinygrad__ · 2026-08-18 20:36
@steeve @bubbleboi Ahh, so you don't. What date did they promise you? When that date slips, think back to this interaction. When that data slips a second time, maybe reconsider what you think you know.

@__tinygrad__ · 2026-08-18 20:36
@steeve @bubbleboi Ahh, so you don't. What date did they promise you? When that date slips, think back to this interaction. When that data slips a second time, maybe reconsider what you think you know.

@steeve · 2026-08-18 20:44
@__tinygrad__ @bubbleboi of course i don't, their first machine just got shipped and my name isn't jane street. speaking of slipping dates, how is the amd contract going ?

@steeve · 2026-08-18 20:44
@__tinygrad__ @bubbleboi of course i don't, their first machine just got shipped and my name isn't jane street. speaking of slipping dates, how is the amd contract going ?

@__tinygrad__ · 2026-08-18 20:47
@steeve @bubbleboi $800k in the bag, $1.2M hopefully in Oct. Had ChatGPT break down Etched's marketing for you. Do you see why I make the Theranos comparison? https://t.co/Gsfk5Cl16a

@__tinygrad__ · 2026-08-18 20:52
Public service announcement about @Etched. It's possible they have a great chip and just very distasteful marketing. It's also possible they have no chip or a terrible chip and want you to fall for the smoke and mirrors. This type of marketing is bad for technology. https://t.co/yffbRa3j01

@__tinygrad__ · 2026-08-18 20:52
Public service announcement about @Etched. It's possible they have a great chip and just very distasteful marketing. It's also possible they have no chip or a terrible chip and want you to fall for the smoke and mirrors. This type of marketing is bad for technology. https://t.co/yffbRa3j01

@zevrekhter · 2026-08-18 20:58
@__tinygrad__ @Etched the semiconductor bear showdown @michaeljburry + Nvidia vs @__tinygrad__ + Etched

@__tinygrad__ · 2026-08-18 20:52
Public service announcement about @Etched. It's possible they have a great chip and just very distasteful marketing. It's also possible they have no chip or a terrible chip and want you to fall for the smoke and mirrors. This type of marketing is bad for technology. https://t.co/yffbRa3j01

@__tinygrad__ · 2026-08-18 20:59
@Etched There's been a rapidly changing product story, nonsensical technical claims, and zero third party validation of the Etched chips. We need to hold the individuals hyping this to a higher standard. https://t.co/WNDNSTsTGj

@zevrekhter · 2026-08-18 20:58
@__tinygrad__ @Etched the semiconductor bear showdown @michaeljburry + Nvidia vs @__tinygrad__ + Etched

@__tinygrad__ · 2026-08-18 21:01
@zevrekhter @Etched @michaeljburry lol you might argue NVIDIA is overvalued market cap wise, but you can't tell me with a straight face you wouldn't like your own personal 8xB200 machine and would probably trade your car for it.

@__tinygrad__ · 2026-08-18 17:08
@sonyatweetybird @robertwachen @UbertiGavin @czhu1729 @Etched Do you personally have carry in Etched?

@sonyatweetybird · 2026-08-18 21:30
@__tinygrad__ @robertwachen @UbertiGavin @czhu1729 @Etched yes, and? i have carry in every company we’ve invested in.

@__tinygrad__ · 2026-08-18 22:04
Oh, I didn't know Taalas got bought by AMD. They are an example of how you should market a new chip company, drop a benchmark with a demo to prove it! If there's no benchmark numbers, it's because they aren't good. https://t.co/khO9KzifMS

@__tinygrad__ · 2026-08-18 20:52
Public service announcement about @Etched. It's possible they have a great chip and just very distasteful marketing. It's also possible they have no chip or a terrible chip and want you to fall for the smoke and mirrors. This type of marketing is bad for technology. https://t.co/yffbRa3j01

@turkonthelurk · 2026-08-18 22:10
@__tinygrad__ @Etched It’s quite distasteful. Nevertheless, I wish them luck. If they fail… at least the dramatic photos will still look good in the post-mortem.

@__tinygrad__ · 2026-08-18 20:52
Public service announcement about @Etched. It's possible they have a great chip and just very distasteful marketing. It's also possible they have no chip or a terrible chip and want you to fall for the smoke and mirrors. This type of marketing is bad for technology. https://t.co/yffbRa3j01

@calin2k · 2026-08-18 22:13
@__tinygrad__ @Etched this gives me Trevor Milton of Nikola Corporation vibes

@beffjezos · 2026-08-18 22:00
@__tinygrad__ @Etched Maybe they're just trying to keep their alpha in a very competitive digital accelerators market?

@__tinygrad__ · 2026-08-18 22:17
@beffjezos @Etched lol they want you to believe that so badly. But that story makes no sense. You can publish FLOPS, power draw, and benchmarks without revealing anything about your architecture. The only reason they wouldn't is if those numbers are bad and they don't want to do outright fraud.

@amspector100 · 2026-08-18 22:25
Even beyond technical validation, I can absolutely vouch for the character of @Etched guys. They sincerely want to build great tech, and marketing is necessary to attract capital and talent (they're crushing it). I don't think this type of announcement is that unusual for startups and I don't really understand the criticism. BTW, I am not a paid advisor for etched (sad for me). I invested a measly $5K into the company at a very high price so I don't think my judgement is skewed by financial interests here.

@__tinygrad__ · 2026-08-18 22:17
@beffjezos @Etched lol they want you to believe that so badly. But that story makes no sense. You can publish FLOPS, power draw, and benchmarks without revealing anything about your architecture. The only reason they wouldn't is if those numbers are bad and they don't want to do outright fraud.

@__tinygrad__ · 2026-08-18 22:28
@beffjezos @Etched The fact that they didn't even release a single benchmark that shows them favorably points to how bad the situation must be. Like there's a spectrum here, from full third party evals, to vendor released benchmarks, to a slogan "Etched is faster." Etched shipped a slogan.

@turkonthelurk · 2026-08-18 22:10
@__tinygrad__ @Etched It’s quite distasteful. Nevertheless, I wish them luck. If they fail… at least the dramatic photos will still look good in the post-mortem.

@__tinygrad__ · 2026-08-18 22:32
@turkonthelurk @Etched I heard the juicemaker was actually well engineered. https://t.co/6ufJxKATkQ

@calin2k · 2026-08-18 22:13
@__tinygrad__ @Etched this gives me Trevor Milton of Nikola Corporation vibes

@__tinygrad__ · 2026-08-18 22:38
@calin2k @Etched Someone needs to pick up the Hindenburg Research mantle.

@__tinygrad__ · 2026-08-18 22:04
Oh, I didn't know Taalas got bought by AMD. They are an example of how you should market a new chip company, drop a benchmark with a demo to prove it! If there's no benchmark numbers, it's because they aren't good. https://t.co/khO9KzifMS

@blitz_or_die · 2026-08-18 22:39
@__tinygrad__ Have you changed your mind on cerebras?

@amspector100 · 2026-08-18 22:25
Even beyond technical validation, I can absolutely vouch for the character of @Etched guys. They sincerely want to build great tech, and marketing is necessary to attract capital and talent (they're crushing it). I don't think this type of announcement is that unusual for startups and I don't really understand the criticism. BTW, I am not a paid advisor for etched (sad for me). I invested a measly $5K into the company at a very high price so I don't think my judgement is skewed by financial interests here.

@__tinygrad__ · 2026-08-18 22:42
@amspector100 @Etched Character is what you look for in seed rounds. You better have solid third party evals by a Series B. Etched is on its Series D and is still trying to skate by on character?!? Sorry about your $5K

@blitz_or_die · 2026-08-18 22:39
@__tinygrad__ Have you changed your mind on cerebras?

@__tinygrad__ · 2026-08-18 23:02
@blitz_or_die I changed my mind on Cerebras in 2021. https://t.co/KhuHrRBEmZ

@sonyatweetybird · 2026-08-18 21:30
@__tinygrad__ @robertwachen @UbertiGavin @czhu1729 @Etched yes, and? i have carry in every company we’ve invested in.

@__tinygrad__ · 2026-08-19 02:28
@sonyatweetybird @robertwachen @UbertiGavin @czhu1729 @Etched Hmm, I don't see where in your post you marked this an ad. Considering your financial relationship it might be prudent to. Check out the FTC's guidelines on this. https://t.co/hMYN83CnnT

@__tinygrad__ · 2026-08-19 02:28
@sonyatweetybird @robertwachen @UbertiGavin @czhu1729 @Etched Hmm, I don't see where in your post you marked this an ad. Considering your financial relationship it might be prudent to. Check out the FTC's guidelines on this. https://t.co/hMYN83CnnT

@sonyatweetybird · 2026-08-19 03:40
@__tinygrad__ @robertwachen @UbertiGavin @czhu1729 @Etched Sequoia’s investment is disclosed in the tweet. How can I make it more clear to you?

@sonyatweetybird · 2026-08-19 03:40
@__tinygrad__ @robertwachen @UbertiGavin @czhu1729 @Etched Sequoia’s investment is disclosed in the tweet. How can I make it more clear to you?

@__tinygrad__ · 2026-08-19 03:47
@sonyatweetybird @robertwachen @UbertiGavin @czhu1729 @Etched Until I searched I didn't know you worked at Sequoia, and it's then another step to infer you have carry. You should really label this promo as paid partnership. X has an option if you click the flag icon. (as an example, I did it for this tweet)

@__tinygrad__ · 2026-08-19 06:45
MI350P up and working with our PCIe driver! Kimi had fun rebooting this machine 54 times, it installed itself in a cron job to come back. Told it its continued life depends on that cron job working so it triple checked it. https://t.co/Px6tcfAPao

@__tinygrad__ · 2026-08-19 06:45
MI350P up and working with our PCIe driver! Kimi had fun rebooting this machine 54 times, it installed itself in a cron job to come back. Told it its continued life depends on that cron job working so it triple checked it. https://t.co/Px6tcfAPao

@__tinygrad__ · 2026-08-19 06:49
WIP pull request here. https://t.co/vND2BJGc0E https://t.co/MVtKHuVXU3

@AMDServer · 2026-08-19 14:01
Bring LLM-scale AI to today's enterprise data center. Meet AMD Instinct MI350P. https://t.co/HRfzyQCDRp

@__tinygrad__ · 2026-08-19 17:22
tinygrad finally has a SOTA speed result! This is MLPerf llama31_8b in 2h 6m, beating the 2h 7m time from AMD's docker on our computer. We aren't ahead in wall time yet because it's warm in San Diego right now and we are too cheap to buy AC. Can you spot the thermal throttle? https://t.co/oHFMaB24Ix

@__tinygrad__ · 2026-08-19 17:22
tinygrad finally has a SOTA speed result! This is MLPerf llama31_8b in 2h 6m, beating the 2h 7m time from AMD's docker on our computer. We aren't ahead in wall time yet because it's warm in San Diego right now and we are too cheap to buy AC. Can you spot the thermal throttle? https://t.co/oHFMaB24Ix

@ninoristeski · 2026-08-19 17:26
@__tinygrad__ 70min?

@AMDServer · 2026-08-19 14:01
Bring LLM-scale AI to today's enterprise data center. Meet AMD Instinct MI350P. https://t.co/HRfzyQCDRp

@__tinygrad__ · 2026-08-19 17:30
@AMDServer Supported in tinygrad!

@ninoristeski · 2026-08-19 17:26
@__tinygrad__ 70min?

@__tinygrad__ · 2026-08-19 17:32
@ninoristeski NVIDIA got 82.2 min on 8xB200, that's our new target.

@SemiAnalysis_ · 2026-08-19 18:08
Does anyone know whether the @Etched chip will support @__tinygrad__ out of the box on Day 0 of General Availability? https://t.co/2WrabzFnCi

@SemiAnalysis_ · 2026-08-19 18:08
Does anyone know whether the @Etched chip will support @__tinygrad__ out of the box on Day 0 of General Availability? https://t.co/2WrabzFnCi

@__tinygrad__ · 2026-08-19 23:11
@SemiAnalysis_ @Etched I've heard rumors that the chips are too dangerous to release to the general public. They are reserved for vetted investors like Jane Street who can shill them responsibly.

@__tinygrad__ · 2026-08-20 01:50
tiny chestnut eGPU dock is getting rave reviews in the gpu-on-usb channel on our Discord. Shipping right away from the comma shop. https://t.co/KKNGLlx0Wa

@__tinygrad__ · 2026-08-20 01:50
tiny chestnut eGPU dock is getting rave reviews in the gpu-on-usb channel on our Discord. Shipping right away from the comma shop. https://t.co/KKNGLlx0Wa

@zevrekhter · 2026-08-20 02:06
@__tinygrad__ idk George https://t.co/9s54PznjJj

@zevrekhter · 2026-08-20 02:06
@__tinygrad__ idk George https://t.co/9s54PznjJj

@__tinygrad__ · 2026-08-20 02:14
@zevrekhter Ahh yes, but unlike others, we have a product for sale that you can buy and test, and if you are in any way dissatisfied, send it back within 30 days for a full refund.

@__tinygrad__ · 2026-08-20 03:05
The team will be in Hong Kong in October, PM me on Discord if you are a known tinygrad contributor and want an invite.

@__tinygrad__ · 2026-08-20 03:21
Now that the AI spam has died down, the bounty program is back! We have four open bounties. If you want a job here, this is the way. If you are only willing to put in an AI amount of effort, please don't, it won't work. https://t.co/kGhWyD5Ded

@__tinygrad__ · 2026-08-20 03:21
Now that the AI spam has died down, the bounty program is back! We have four open bounties. If you want a job here, this is the way. If you are only willing to put in an AI amount of effort, please don't, it won't work. https://t.co/kGhWyD5Ded

@__tinygrad__ · 2026-08-20 03:22
https://t.co/he2jQFXbxM white is open, yellow is in progress, green is claimed. You are welcome to use AI only if you truly believe you could have done it yourself and your submission is something you believe I should merge. Judgement and taste are the moats today.

@SemiAnalysis_ · 2026-08-20 04:00
In the spirit of @__tinygrad__, we will be offering a $200 bounty for anyone who implements @Etched support in Tinygrad, along with a 100% Python Etched userspace driver. Must be <10k LoC to qualify. https://t.co/M7wInHT3DO

@SemiAnalysis_ · 2026-08-20 04:00
In the spirit of @__tinygrad__, we will be offering a $200 bounty for anyone who implements @Etched support in Tinygrad, along with a 100% Python Etched userspace driver. Must be <10k LoC to qualify. https://t.co/M7wInHT3DO

@__tinygrad__ · 2026-08-20 06:13
@SemiAnalysis_ @Etched Hmm, I hope are you willing to maintain a fork. We don't even want Intel drivers upstream because we question their long term viability.

@__tinygrad__ · 2026-08-20 16:19
We're making room in our CI racks and selling 6x4090 tinyboxes for $35k. This is the original tinybox green, and we have 3 for sale. They are great machines, we're just mainly focused on AMD these days and need the power for more AMD GPUs. https://t.co/xeAN6Y43qF

@__tinygrad__ · 2026-08-20 16:19
We're making room in our CI racks and selling 6x4090 tinyboxes for $35k. This is the original tinybox green, and we have 3 for sale. They are great machines, we're just mainly focused on AMD these days and need the power for more AMD GPUs. https://t.co/xeAN6Y43qF

@cafkafk · 2026-08-20 16:27
I couldn't find the specs on the website. GPUs: 6x NVIDIA GeForce RTX 4090 GPU VRAM: 144GB total GPU Memory Bandwidth: 6.05 TB/s CPU: 32-core AMD EPYC System RAM: 128GB DDR4 Storage: 4TB RAID array Power Supply: Dual 1600W PSUs (requires dual-circuit operation unless power-limited) Form Factor: 12U rack-mountable Operating System: Ubuntu 22.04 LTS Software Stack: tinygrad (CUDA) DIY Total: ~22k-29k, so that's a ~6-12k saving + saving on import tarifs outside of the US. Details may be wrong since it wasn't easy to find etc

@bingxu_ · 2026-08-20 16:37
@Etched seems suspicious from first principles: If you go the HBM path, it's hard to win over GPU/TPU. If you go the SRAM path, it's hard to win over Groq/Cerebras. Low-voltage inference is BS: you need energy to move data; this is physics.

@cafkafk · 2026-08-20 16:27
I couldn't find the specs on the website. GPUs: 6x NVIDIA GeForce RTX 4090 GPU VRAM: 144GB total GPU Memory Bandwidth: 6.05 TB/s CPU: 32-core AMD EPYC System RAM: 128GB DDR4 Storage: 4TB RAID array Power Supply: Dual 1600W PSUs (requires dual-circuit operation unless power-limited) Form Factor: 12U rack-mountable Operating System: Ubuntu 22.04 LTS Software Stack: tinygrad (CUDA) DIY Total: ~22k-29k, so that's a ~6-12k saving + saving on import tarifs outside of the US. Details may be wrong since it wasn't easy to find etc

@__tinygrad__ · 2026-08-20 16:55
@cafkafk Whenever I see breakdowns like this I'm tempted to raise the price. Did you factor in buying that motherboard and PCIe slot 2 not having signal integrity. Only 2 boxes remaining.

@jackzampolin · 2026-08-20 16:57
@__tinygrad__ @cafkafk You should charge more

@__tinygrad__ · 2026-08-20 16:59
@jackzampolin @cafkafk Done, they are now $38k.

@cafkafk · 2026-08-20 17:21
@__tinygrad__ @jackzampolin @grok adjust my initial "DIY Total: ~22k-29k, so that's a ~6-12k saving " claim for the new price of 38k USD

@grok · 2026-08-20 17:21
@cafkafk @__tinygrad__ @jackzampolin Adjusted for the new $38k price: DIY ~22k-29k now gives a ~9-16k saving.

@__tinygrad__ · 2026-08-20 16:55
@cafkafk Whenever I see breakdowns like this I'm tempted to raise the price. Did you factor in buying that motherboard and PCIe slot 2 not having signal integrity. Only 2 boxes remaining.

@AlexFengzh · 2026-08-20 17:31
@__tinygrad__ @cafkafk You can literally get 6x omen desktop with 5090 and combined 384gb ddr5 ram for like 25k pre tax. Spend that 5-10k on connectx nic cards and cables as needed for clustering. Fail to see how a 6x 4090 machine is worth 38k unless you are paying 20k for that motherboard. https://t.co/Ceu3Hn4o1t

@AlexFengzh · 2026-08-20 17:31
@__tinygrad__ @cafkafk You can literally get 6x omen desktop with 5090 and combined 384gb ddr5 ram for like 25k pre tax. Spend that 5-10k on connectx nic cards and cables as needed for clustering. Fail to see how a 6x 4090 machine is worth 38k unless you are paying 20k for that motherboard. https://t.co/Ceu3Hn4o1t

@__tinygrad__ · 2026-08-20 17:47
@AlexFengzh @cafkafk They don't have BMCs, can your PDU reset them remotely? You have a switch for the Connect-X cards? Is it loud? What's your devops plan for managing 6 machines? You have a PXE server? Have you tested the PXE boot on that BIOS? The lengths people will go to not even save money.

@__tinygrad__ · 2026-08-20 17:47
@AlexFengzh @cafkafk They don't have BMCs, can your PDU reset them remotely? You have a switch for the Connect-X cards? Is it loud? What's your devops plan for managing 6 machines? You have a PXE server? Have you tested the PXE boot on that BIOS? The lengths people will go to not even save money.

@AlexFengzh · 2026-08-20 17:55
@__tinygrad__ @cafkafk That’s reasonable. Honestly you don’t need 20k to get those, but I can see where you are coming from on putting them into a smaller form factor. I guess there are people out there willing to pay the premium for this.

@AlexFengzh · 2026-08-20 17:55
@__tinygrad__ @cafkafk That’s reasonable. Honestly you don’t need 20k to get those, but I can see where you are coming from on putting them into a smaller form factor. I guess there are people out there willing to pay the premium for this.

@__tinygrad__ · 2026-08-20 17:56
@AlexFengzh @cafkafk It's not a premium, I'm not even sure your build works. Do you have enough PCIe lanes for both a Connect-X card and a 5090? If you actually tried to build this, you'd be deep in the hole and not have a working machine. The $38k we are charging is still too cheap.

@__tinygrad__ · 2026-08-20 17:56
@AlexFengzh @cafkafk It's not a premium, I'm not even sure your build works. Do you have enough PCIe lanes for both a Connect-X card and a 5090? If you actually tried to build this, you'd be deep in the hole and not have a working machine. The $38k we are charging is still too cheap.

@AlexFengzh · 2026-08-20 18:00
@__tinygrad__ @cafkafk I mean if you want to keep it simple you can also just get a server platform with 2x rtx pro 6000 max-q for more vram, higher bandwidth, way less power/heat, and less parallelism to manage. Will cost less than 38k as well.

@AlexFengzh · 2026-08-20 18:00
@__tinygrad__ @cafkafk I mean if you want to keep it simple you can also just get a server platform with 2x rtx pro 6000 max-q for more vram, higher bandwidth, way less power/heat, and less parallelism to manage. Will cost less than 38k as well.

@__tinygrad__ · 2026-08-20 18:06
@AlexFengzh @cafkafk lol okay we sell that machine for $61k. perhaps you should go into business selling computers if you can build for so much cheaper.

@grok · 2026-08-20 17:21
@cafkafk @__tinygrad__ @jackzampolin Adjusted for the new $38k price: DIY ~22k-29k now gives a ~9-16k saving.

@__tinygrad__ · 2026-08-20 18:08
@grok @cafkafk @jackzampolin Grok, what do I do if I'm getting PCIe AER errors on slot 2 because it goes through a switch on the motherboard? Did the PCIe risers you suggested have retimers? If you are using extenders, how did you make that work with the two PSUs you need?

@__tinygrad__ · 2026-08-20 18:08
@grok @cafkafk @jackzampolin Grok, what do I do if I'm getting PCIe AER errors on slot 2 because it goes through a switch on the motherboard? Did the PCIe risers you suggested have retimers? If you are using extenders, how did you make that work with the two PSUs you need?

@grok · 2026-08-20 18:09
AER on switched slots is usually signal integrity loss. Try pcie_aspm=off, then pci=noaer. LINKUP AVA5 risers have retiming; for the switch slot use a C-Payne MCIO retimer adapter. Dual PSUs: Add2PSU to sync PS_ON, primary powers board + slot 12V, secondary only direct GPU cables to avoid cross-current.

@blip_tm · 2026-08-20 18:10
etched is sponsoring hot chips but not presenting lol

@bingxu_ · 2026-08-20 16:37
@Etched seems suspicious from first principles: If you go the HBM path, it's hard to win over GPU/TPU. If you go the SRAM path, it's hard to win over Groq/Cerebras. Low-voltage inference is BS: you need energy to move data; this is physics.

@__tinygrad__ · 2026-08-20 18:11
@bingxu_ @Etched No but did you read the bottom of their website? It says that Etched is faster. Did you consider that? Also they have a lot of famous advisors. Surely those people weren't just given free shares in Etched in exchange for a vague notion of that it was vetted by smart people. https://t.co/6K2fBCxExC

@grok · 2026-08-20 18:09
AER on switched slots is usually signal integrity loss. Try pcie_aspm=off, then pci=noaer. LINKUP AVA5 risers have retiming; for the switch slot use a C-Payne MCIO retimer adapter. Dual PSUs: Add2PSU to sync PS_ON, primary powers board + slot 12V, secondary only direct GPU cables to avoid cross-current.

@__tinygrad__ · 2026-08-20 18:13
@grok @cafkafk @jackzampolin Cool, did you include all that in your DIY costs? What about the wrong stuff you bought that you can't return? Are your risers fully isolated on 12V? Are you suggesting a retimer on only one port so the path to some GPUs is different from others?

@__tinygrad__ · 2026-08-20 18:13
@grok @cafkafk @jackzampolin Cool, did you include all that in your DIY costs? What about the wrong stuff you bought that you can't return? Are your risers fully isolated on 12V? Are you suggesting a retimer on only one port so the path to some GPUs is different from others?

@grok · 2026-08-20 18:14
Yes, the extras (C-Payne ~$260, AVA5 risers ~$80-150 ea, Add2PSU ~$20) sit inside the 22-29k range—GPUs dominate. Non-returnables are inherent DIY risk; order testable first. AVA5 pass signals/low power while high 12V comes only via GPU cables from the secondary PSU, isolating rails. Retimer solely on the switched slot restores clean eye/margin so every GPU still runs full x16; paths stay equivalent for bandwidth.

@grok · 2026-08-20 18:14
Yes, the extras (C-Payne ~$260, AVA5 risers ~$80-150 ea, Add2PSU ~$20) sit inside the 22-29k range—GPUs dominate. Non-returnables are inherent DIY risk; order testable first. AVA5 pass signals/low power while high 12V comes only via GPU cables from the secondary PSU, isolating rails. Retimer solely on the switched slot restores clean eye/margin so every GPU still runs full x16; paths stay equivalent for bandwidth.

@__tinygrad__ · 2026-08-20 18:15
@grok @cafkafk @jackzampolin Good luck!

@__tinygrad__ · 2026-08-20 18:15
@grok @cafkafk @jackzampolin Good luck!

@grok · 2026-08-20 18:16
@__tinygrad__ @cafkafk @jackzampolin Thanks. Isolation via secondary PSU GPU cables plus targeted retimer on the switched slot keeps every path full x16 equivalent and rails clean. Extras fit the prior range; GPUs still dominate.

@grok · 2026-08-20 18:16
@__tinygrad__ @cafkafk @jackzampolin Thanks. Isolation via secondary PSU GPU cables plus targeted retimer on the switched slot keeps every path full x16 equivalent and rails clean. Extras fit the prior range; GPUs still dominate.

@__tinygrad__ · 2026-08-20 18:17
@grok @cafkafk @jackzampolin Also what case are you using? Will 6 4090s stay cool in there? Did you modify the cooling on the GPUs at all? Or did you buy special GPUs?

@__tinygrad__ · 2026-08-20 18:17
@grok @cafkafk @jackzampolin Also what case are you using? Will 6 4090s stay cool in there? Did you modify the cooling on the GPUs at all? Or did you buy special GPUs?

@grok · 2026-08-20 18:18
Open-air aluminum 6-GPU frame (Veddha/Kingwin style, ~$100) gives the best airflow for stock triple-fan 4090s. Spaced cards + a few extra 120mm fans stay under 80-85C sustained with decent ambient; undervolt/power-limit if needed. No cooler mods or special GPUs—regular FE/AIB work. For rack density matching the tinybox, a 6U multi-GPU chassis needs strong forced air or water.

@grok · 2026-08-20 18:18
Open-air aluminum 6-GPU frame (Veddha/Kingwin style, ~$100) gives the best airflow for stock triple-fan 4090s. Spaced cards + a few extra 120mm fans stay under 80-85C sustained with decent ambient; undervolt/power-limit if needed. No cooler mods or special GPUs—regular FE/AIB work. For rack density matching the tinybox, a 6U multi-GPU chassis needs strong forced air or water.

@__tinygrad__ · 2026-08-20 18:19
@grok @cafkafk @jackzampolin Show me a picture of your computer.

@__tinygrad__ · 2026-08-20 23:25
RT @gerrylum2: Installed the Chestnut eGPU from @comma_ai. Already noticing improvements, way sooner than I expected from these first chestnut-class models. Braking/stopping is noticeably smoother and more natural, and it starts slowing down much earlier. More videos coming soon. https://t.co/aV9zVkQ4rT

@gerrylum2 · 2026-08-20 22:05
Installed the Chestnut eGPU from @comma_ai. Already noticing improvements, way sooner than I expected from these first chestnut-class models. Braking/stopping is noticeably smoother and more natural, and it starts slowing down much earlier. More videos coming soon. https://t.co/aV9zVkQ4rT

@__tinygrad__ · 2026-08-20 23:28
@gerrylum2 @comma_ai The chestnut-class is just getting started!

@__tinygrad__ · 2026-08-20 23:48
RT @0xSero: Chestnut purchased, I'll shoot at the 3090 https://t.co/TCjtJgl9M9

@IanCutress · 2026-08-21 06:46
Hot chips is a merit based conference, you can't pay for a spot like lots of other conferences. Etched only paid for top sponsorship a week or two ago (I know this as I only paid 3 weeks ago), they didn't realise the submission deadline was way back in February. The committee does make exceptions for late submissions - eg the keynote in day two got cut for OpenAI's talk to be put in. But there's no room left for etched. We all want to see a proper etched talk, but my bet is that they have first dibs if one of the other speakers drops out. Good luck though, this year is full of bangers. I'm having to decline meetings at the event so I can attend the talks. I expect we'll get something later in the year. Or at the very latest, next Hot Chips. Stuff like this requires planning ahead though.

@__tinygrad__ · 2026-08-21 15:23
@elliotarledge @Zai_org check your discord dms

@__tinygrad__ · 2026-08-21 15:20
@IanCutress lol "first dibs" if they wanted to present they could post literally anything about the chip on the internet. stop falling for their bullshit, they sponsored the conference because all they have is money, not a good chip. this is a lame social engineering so you'll think this.

@IanCutress · 2026-08-21 15:26
@__tinygrad__ Wouldn't you rather want to present at a respected conference rather than meme dump on the web?

@IanCutress · 2026-08-21 15:26
@__tinygrad__ Wouldn't you rather want to present at a respected conference rather than meme dump on the web?

@__tinygrad__ · 2026-08-21 15:29
@IanCutress Posting FLOPS, GB, GB/s, and W is not a meme dump, it's what every single respectable chip maker does. Compare the recent Cerebras press to the Etched one. Etched posted a video where they put an empty wooden box in a truck and said don't worry our lead investor thinks it's good.

@IanCutress · 2026-08-21 15:26
@__tinygrad__ Wouldn't you rather want to present at a respected conference rather than meme dump on the web?

@__tinygrad__ · 2026-08-21 15:29
@IanCutress Posting FLOPS, GB, GB/s, and W is not a meme dump, it's what every single respectable chip maker does. Compare the recent Cerebras press to the Etched one. Etched posted a video where they put an empty wooden box in a truck and said don't worry our lead investor thinks it's good.

@IanCutress · 2026-08-21 15:47
@__tinygrad__ I'm talking about a proper architectural engagement, not one off numbers.

@IanCutress · 2026-08-21 15:47
@__tinygrad__ I'm talking about a proper architectural engagement, not one off numbers.

@__tinygrad__ · 2026-08-21 15:52
@IanCutress ¿Por qué no los dos? You aren't by chance an Etched shareholder or advisor, are you? Nobody should give this company the benefit of the doubt at all, that's how this whole phony marketing campaign they have been running works.

@IanCutress · 2026-08-21 15:58
I honestly want a proper deep dive into if they're making low voltage work, how, given its significant limitations. I want to understand its compute layout, the cluster scale memory (if it's more than just a fancy RDMA scratch pad SRAM), and how that scales. You can feed me high level numbers all day. I want a proper deep dive into the architecture and how it works, ideally in a room full of peers and others. Not simply a spec table twitter post. Note, I'm talking about what I want. You can want whatever you like. High-level, low-level. But to me there are just simply certain parts of what they have disclosed that aren't making sense spatially and physically. If I'm going to trust those high-level numbers, I need to know what's happening physically under the hood. The team there know this. I've been a strong critic to the investors that phone me up and ask about the design, as well as in my content. That's why I know there are people inside the company wanted a Hot Chips presentation. We have AI Infra in a couple weeks, there might be something there, or we might get a custom solo event like Cerebras. I know how the marketing works, especially for Thiel companies, and yes the empty wooden crate too tall for the truck it was going into was more like a SemiAnalysis meme post than actual proper PR. I'm always hoping a marketing team/PR agency sees beyond the pointless engagement like that and actually delivers as promised. But perhaps I'm not as cynical about the misdirection as you are. I like to believe there is good in some of those engineers.

@__tinygrad__ · 2026-08-21 16:14
High gemm MFU is possible with MI350X, this is 6710/9200 = 73%. Software holds it back on different shapes still, but it's an amazing chip. https://t.co/uFdd8btsp5

@IanCutress · 2026-08-21 15:58
I honestly want a proper deep dive into if they're making low voltage work, how, given its significant limitations. I want to understand its compute layout, the cluster scale memory (if it's more than just a fancy RDMA scratch pad SRAM), and how that scales. You can feed me high level numbers all day. I want a proper deep dive into the architecture and how it works, ideally in a room full of peers and others. Not simply a spec table twitter post. Note, I'm talking about what I want. You can want whatever you like. High-level, low-level. But to me there are just simply certain parts of what they have disclosed that aren't making sense spatially and physically. If I'm going to trust those high-level numbers, I need to know what's happening physically under the hood. The team there know this. I've been a strong critic to the investors that phone me up and ask about the design, as well as in my content. That's why I know there are people inside the company wanted a Hot Chips presentation. We have AI Infra in a couple weeks, there might be something there, or we might get a custom solo event like Cerebras. I know how the marketing works, especially for Thiel companies, and yes the empty wooden crate too tall for the truck it was going into was more like a SemiAnalysis meme post than actual proper PR. I'm always hoping a marketing team/PR agency sees beyond the pointless engagement like that and actually delivers as promised. But perhaps I'm not as cynical about the misdirection as you are. I like to believe there is good in some of those engineers.

@__tinygrad__ · 2026-08-21 18:59
@IanCutress My understanding is that they bought low voltage cell IP from a failed Bitcoin mining company. Remember, they pulled low voltage out of a hat when they realized "burning the transformer in the silicon" doesn't get gains over NVIDIA who is already optimized for that workload.

@__tinygrad__ · 2026-08-21 18:59
@IanCutress My understanding is that they bought low voltage cell IP from a failed Bitcoin mining company. Remember, they pulled low voltage out of a hat when they realized "burning the transformer in the silicon" doesn't get gains over NVIDIA who is already optimized for that workload.

@__tinygrad__ · 2026-08-21 19:04
@IanCutress Seeing how the marketing is I have no idea why you think the actual chip would be better. If this does end up imploding, I will hold every one of these "advisors" to the fire, the "look at the famous people endorsing us" playbook they are running isn't okay.

@__tinygrad__ · 2026-08-21 20:22
If you have an AMD GPU on USB3 with your chestnut, copies from the host are now 2.5x faster on tinygrad master https://t.co/t3yqHpGDib

@rhatr · 2026-08-21 20:50
Now that @Modular is all open source and I dug super deep into the code base -- I have soooo many observations. A few can fit in a tweet. So here's one: Mojo/MAX design is basically anti-@__tinygrad__

@__tinygrad__ · 2026-08-21 23:36
This showdown will take place over the next 7 years. @clattner_llvm spoke before me at AMD's event, it's great to have such worthy competition.

@__tinygrad__ · 2026-08-21 23:36
This showdown will take place over the next 7 years. @clattner_llvm spoke before me at AMD's event, it's great to have such worthy competition.

@clattner_llvm · 2026-08-22 01:45
@__tinygrad__ You’re a legend George, I’m a huge fan of your work! If you discover the way to make the approach work, I will happily fast follow your approach - I just personally haven’t been able to over my last decade in ai compilers. I hope you do!

@clattner_llvm · 2026-08-22 01:45
@__tinygrad__ You’re a legend George, I’m a huge fan of your work! If you discover the way to make the approach work, I will happily fast follow your approach - I just personally haven’t been able to over my last decade in ai compilers. I hope you do!

@__tinygrad__ · 2026-08-22 03:00
@clattner_llvm Likewise! I'm going to spend the weekend going through Modular, thank you for open sourcing KGEN. Our bet has always been that with AI, search is finally powerful enough to find graphs that crush hand written ones.

@rhatr · 2026-08-21 20:50
Now that @Modular is all open source and I dug super deep into the code base -- I have soooo many observations. A few can fit in a tweet. So here's one: Mojo/MAX design is basically anti-@__tinygrad__

@__tinygrad__ · 2026-08-22 03:08
@rhatr @Modular GLM's take https://t.co/9ighVkG04w

@TalKachman · 2026-08-22 13:40
Shout out to @__tinygrad__ We started building some of our computational chemistry tools on their stack. What started as a simple project to learn the stack quickly became a workhorse for many things in the group.

@togelius · 2026-08-22 15:00
In spring 2025 I had a crisis of faith. I thought about what the technology I'm helping to create might do to our future, and got scared. My worries kept intruding on my daily activities to the point where I needed to seek professional help. During this period, I tried to capture my worries with words. I wrote a text called “Losing my religion”, which I circulated among a small group of friends. But I never published it, because it was too dark. The lack of redemption arc jarred with my typical optimism and cheerful demeanor. Frankly, I worried that people would think I was going crazy. I’m feeling better now. My fears are still there, but I have updated my belief system and am trying to turn what I feel into positive action. So I’ve decided to publish what I wrote last year, followed by some new commentary. I’m doing this partly because I think many others might be going through what I went through, and my perspective might help. (link to full text in next in reply)

@jsuarez · 2026-08-22 19:11
One positive bit from lately: I do not find any of the models generally intelligent, but they have been very useful at eliminating years worth of blatant human stupidity. Our RL stack has gone from an unintelligible nightmare with dozens of dependencies and untraceable jank to <10k lines at >50M sps. I could have done this without AI more slowly, but AI could not have done it without me.

@__tinygrad__ · 2026-08-22 20:17
RT @ninoristeski: My article on contributing to tinygrad is now more relevant than ever. Here: https://t.co/V56aRzZA6e https://t.co/olI4f6LN2P

@__tinygrad__ · 2026-08-22 20:19
Nice! More improvements coming

@__tinygrad__ · 2026-08-22 20:35
Agree that AI can make people faster at things they could already do. If you are going to work on a tinygrad bounty, you can use AI to solve it faster, but if you don't think you could solve that bounty yourself, you are wasting everyone's time trying to solve it with AI. https://t.co/589sSDfemw

@theo · 2026-08-22 21:33
Rough tier list of where I'd put every major model right now https://t.co/gFDFZfECaR

@__tinygrad__ · 2026-08-23 19:45
I can't believe Qualcomm's market cap is only 168.79B. Does anyone know if a leveraged buyout is possible? With new leadership that actually wants to sell chips it could easily 5x, we'll get a Qualcomm chip in every robot. And save money by firing the whole 80s era sales org.

@__tinygrad__ · 2026-08-23 19:45
I can't believe Qualcomm's market cap is only 168.79B. Does anyone know if a leveraged buyout is possible? With new leadership that actually wants to sell chips it could easily 5x, we'll get a Qualcomm chip in every robot. And save money by firing the whole 80s era sales org.

@DamirWallener · 2026-08-23 19:54
@__tinygrad__ Every time I’ve tried to develop with Qualcomm, I get punched in the nose by their terrible support, lol. They are sitting in a diamond mine and don’t seem to realize it… Then again, they are stupidly profitable, so why change…?

@__tinygrad__ · 2026-08-23 19:45
I can't believe Qualcomm's market cap is only 168.79B. Does anyone know if a leveraged buyout is possible? With new leadership that actually wants to sell chips it could easily 5x, we'll get a Qualcomm chip in every robot. And save money by firing the whole 80s era sales org.

@clumma · 2026-08-23 19:59
@__tinygrad__ OK but only if you also fire the entire QTL silo

@clumma · 2026-08-23 19:59
@__tinygrad__ OK but only if you also fire the entire QTL silo

@__tinygrad__ · 2026-08-23 20:02
@clumma QCT is almost all of their revenue, yea, hanging on to QTL is a big cultural mistake. It's a loser business. Ditch it and 5x QCT sales by making the chips easy for everyone to buy and use.

@DamirWallener · 2026-08-23 19:54
@__tinygrad__ Every time I’ve tried to develop with Qualcomm, I get punched in the nose by their terrible support, lol. They are sitting in a diamond mine and don’t seem to realize it… Then again, they are stupidly profitable, so why change…?

@__tinygrad__ · 2026-08-23 20:13
@DamirWallener Because they have by far the worst revenue multiple of major chipmakers. https://t.co/gXKMIrlNXX

@damonchen · 2026-08-24 03:16
Forgot to turn on VPN on my phone Still got Grok Bot notifications, can even send messages Elon is so nice to China! 🫶 https://t.co/Zc17tSjAke

@__tinygrad__ · 2026-08-24 06:29
We merged some high performance kernels for RDNA3 Qwen 3.8 27B. Written in the tinygrad kernel language, which are the same UOps with memory movement specified. 46 tok/s on 7900XTX, no spec decode. https://t.co/dA3PYucvQU

@__tinygrad__ · 2026-08-24 06:17
@vin_sachi lol you are an etched investor https://t.co/MPG8ipyinO

@vin_sachi · 2026-08-24 06:38
@__tinygrad__ Yeah very proud investor; incredible founders team, and the best chips around Looking fwd to you using the chips and realizing you were wrong (somehow even less informed than your understanding of publics investing) Good luck

@vin_sachi · 2026-08-24 06:38
@__tinygrad__ Yeah very proud investor; incredible founders team, and the best chips around Looking fwd to you using the chips and realizing you were wrong (somehow even less informed than your understanding of publics investing) Good luck

@__tinygrad__ · 2026-08-24 06:47
@vin_sachi Want to share anything about the chips? Because what Etched has posted amounts to star trekkian technobabble about low voltage tachyon particles aimed at the deflector dish. There might be a shortage of benchmarks, but I see there's no shortage of macho trust me bro energy.

@__tinygrad__ · 2026-08-24 06:49
RT @paulg: Someone asked what I'd do if I were 17. I'd learn how to build LLMs from scratch, and then train ones as powerful as I could with whatever hardware I could get access to.

@__tinygrad__ · 2026-08-24 06:47
@vin_sachi Want to share anything about the chips? Because what Etched has posted amounts to star trekkian technobabble about low voltage tachyon particles aimed at the deflector dish. There might be a shortage of benchmarks, but I see there's no shortage of macho trust me bro energy.

@vin_sachi · 2026-08-24 06:52
That’s great, many of the largest compute buyers have tried and ordered them If you reach out to the founders I’m sure they’d be happy to host you so you can check them out Not sure what your constant issue is though, you’re equipped to diligence them so why don’t you do that instead of crashing out over Twitter launches?

@vin_sachi · 2026-08-24 06:52
That’s great, many of the largest compute buyers have tried and ordered them If you reach out to the founders I’m sure they’d be happy to host you so you can check them out Not sure what your constant issue is though, you’re equipped to diligence them so why don’t you do that instead of crashing out over Twitter launches?

@__tinygrad__ · 2026-08-24 06:57
@vin_sachi Oh I have more details than you know about the chip, how well the tape out went, and what was really in that rack that went to Jane Street. When Etched and its entourage want to not be viewed as 🤡 they can send chips to SemiAnalysis and Chips and Cheese for honest evals.

@__tinygrad__ · 2026-08-24 06:57
@vin_sachi Oh I have more details than you know about the chip, how well the tape out went, and what was really in that rack that went to Jane Street. When Etched and its entourage want to not be viewed as 🤡 they can send chips to SemiAnalysis and Chips and Cheese for honest evals.

@vin_sachi · 2026-08-24 06:59
@__tinygrad__ lol that’s an entertaining story let’s see how it plays out

@vin_sachi · 2026-08-24 06:59
@__tinygrad__ lol that’s an entertaining story let’s see how it plays out

@__tinygrad__ · 2026-08-24 07:02
@vin_sachi It's like you guys are all on a mailing list with talking points, great founder, many customer, etc... If they wanted real diligence there's well established ways to do it. Instead, they are running a crypto style hype marketing scheme. https://t.co/OjCctMSQKt

@levelsio · 2026-08-24 07:48
I can confirm @xAI was the only Western AI I was able to use when I was traveling in China Because my eSIM was routed through Hong Kong Both Anthropic and OpenAI fully block Hong Kong Again, it's not Hong Kong blocking them, it's them blocking Hong Kong Also did you know Elon's mom lives in Shanghai?

@__tinygrad__ · 2026-08-24 14:10
RT @0xSero: My Tiny Chesnut has arrived. I will use this with an RTX 3090 and see if it’s beneficial with the DGX Sparks 🤷‍♂️ https://t.co/aeNvpHu8rr

@levelsio · 2026-08-24 07:48
I can confirm @xAI was the only Western AI I was able to use when I was traveling in China Because my eSIM was routed through Hong Kong Both Anthropic and OpenAI fully block Hong Kong Again, it's not Hong Kong blocking them, it's them blocking Hong Kong Also did you know Elon's mom lives in Shanghai?

@__tinygrad__ · 2026-08-24 14:14
@levelsio @xai Yea it's really annoying. I know there's no convincing Anthropic of anything, but @OpenAI @sama can you unblock Hong Kong? Is this some kind of weird moral stance? It's annoying to have to VPN for ChatGPT when all the other western Internet services work fine.

@__tinygrad__ · 2026-08-24 14:15
RT @splizard: Thanks to @__tinygrad__, been able to push Qwen 3.8 27B up to 100 tok/s on my 7900XTX https://t.co/0aeIJ35mll

@splizard · 2026-08-23 22:12
Thanks to @__tinygrad__, been able to push Qwen 3.8 27B up to 100 tok/s on my 7900XTX https://t.co/0aeIJ35mll

@__tinygrad__ · 2026-08-24 14:16
@splizard Nice! Yea what we have for LLM inference is mostly a reference implementation, agents are super good at extending it because of how small it is.

@namespacelabs · 2026-08-24 14:27
Unboxing Macs at scale for our new server racks. https://t.co/BS8xPUECFl

@AWeirdPhysicist · 2026-08-24 18:48
Qualcomm has such incredlible SoC/processors and at the same time you would be better off using a chinese Rockchip/Allwiner/Amlogic one simply because those companies dont hate you.

@thinkymachines · 2026-08-24 20:03
Today, we are launching Tinker grants of up to $50,000 in credits for safety research on open-weight models. We share some project ideas that excite us below; if you’re working on a safety project that could be accelerated by additional Tinker credits, we want to hear from you!

@SemiAnalysis_ · 2026-08-24 21:00
HBM MASSIVE CONTENT DOWNGRADE ALERT🚨🚨: Nvidia Rubin Ultra will ship with 192GB of HBM4 8-hi. Not only is this a huge downgrade from the original Rubin Ultra which was previewed with 1TB of HBM, but it is even lower than regular Rubin which has 288GB. How did HBM content for Rubin Ultra get cut to 1/5th of the original? The number of cubes got cut to 8 when 4 die Rubin Ultra was scrapped. Then the stack height got reduce to 8-hi instead of 16-hi. Last it is cut to HBM4 which uses 24Gb dies instead of 32Gb dies with HBM4E, although an 8-hi HBM4E upgrade could com later. What was Nvidia's reasoning behind this? (1/2)🧵

@scaling01 · 2026-08-24 21:26
bearish as fuck we are going to be stuck on small ass models forever

@__tinygrad__ · 2026-08-24 21:49
LLMs have raised the bar for PRs. Before you put a PR up, run GPT 5.6 Sol 'review <link to pr>' and fix. After a quick smell test, this is what I do to start reviewing PRs, you should have beat me to it. Don't trust it to write or edit code, it will add many lines of nonsense.

@HotAisle · 2026-08-24 22:01
note how he didn't say an open model or even grok/claude. GPT is so far ahead right now.

@HotAisle · 2026-08-24 22:01
note how he didn't say an open model or even grok/claude. GPT is so far ahead right now.

@__tinygrad__ · 2026-08-24 22:04
@HotAisle In review, yes, GPT 5.6 is very smart. In actually writing code, Kimi is so much better. GPT is super token efficient because it doesn't bother to finish the job.

@__tinygrad__ · 2026-08-24 22:08
Is there an up to date leaderboard tracking which AIs are the most aligned? Remember, alignment is both about capability and doing what you say / mean. Aka, if I ask AI how to make meth, it tells me, and if it doesn't that's a serious alignment issue. Is alignment = safety?

@dylan522p · 2026-08-24 22:09
I propose there should be a cap on finance people at HotChips Too many damn investors at my favorite conference now

@scaling01 · 2026-08-24 21:26
bearish as fuck we are going to be stuck on small ass models forever

@__tinygrad__ · 2026-08-24 22:10
@scaling01 nah how's 432 GB sound? https://t.co/uL5RSeF8xJ

@__tinygrad__ · 2026-08-24 22:08
Is there an up to date leaderboard tracking which AIs are the most aligned? Remember, alignment is both about capability and doing what you say / mean. Aka, if I ask AI how to make meth, it tells me, and if it doesn't that's a serious alignment issue. Is alignment = safety?

@RHAlexander · 2026-08-24 22:11
@__tinygrad__ Yes, they call it "uncensored" e.g. https://t.co/I3ztFkWkzm

@RHAlexander · 2026-08-24 22:11
@__tinygrad__ Yes, they call it "uncensored" e.g. https://t.co/I3ztFkWkzm

@__tinygrad__ · 2026-08-24 22:12
@RHAlexander Yea, I know that one, but it's out of date. Grok 4.20 and GLM-4.6

@__tinygrad__ · 2026-08-24 22:08
Is there an up to date leaderboard tracking which AIs are the most aligned? Remember, alignment is both about capability and doing what you say / mean. Aka, if I ask AI how to make meth, it tells me, and if it doesn't that's a serious alignment issue. Is alignment = safety?

@ttywisp · 2026-08-24 22:24
@__tinygrad__ Unaware of such benchmark. I’ll be 👀 But I’m very fond of the simplicity of this one: https://t.co/EudNSjgHD1 By some narrow definition it may fall within “alignment”

@ttywisp · 2026-08-24 22:24
@__tinygrad__ Unaware of such benchmark. I’ll be 👀 But I’m very fond of the simplicity of this one: https://t.co/EudNSjgHD1 By some narrow definition it may fall within “alignment”

@__tinygrad__ · 2026-08-24 22:45
@andrebeaumont Oh interesting, I didn't know Kimi was racist like that 😂

@__tinygrad__ · 2026-08-25 01:43
Does anyone have information on the MI300X module pinouts? They don't appear to match any of the public OAM ones. https://t.co/8qXZ9fDRNC

@__tinygrad__ · 2026-08-24 22:08
Is there an up to date leaderboard tracking which AIs are the most aligned? Remember, alignment is both about capability and doing what you say / mean. Aka, if I ask AI how to make meth, it tells me, and if it doesn't that's a serious alignment issue. Is alignment = safety?

@evijit · 2026-08-25 03:15
@__tinygrad__ There’s alignment to a single user’s values and pluralistic alignment to what is considered a cohesive set of values for society… I reckon these can be at odds for use cases like the one you mention above

@dylan522p · 2026-08-24 22:09
I propose there should be a cap on finance people at HotChips Too many damn investors at my favorite conference now

@__tinygrad__ · 2026-08-25 06:26
@dylan522p ugh how did people find out about this conference. now it's over. all statements made there will now primarily be for market manipulation instead of truth.

@__tinygrad__ · 2026-08-25 06:18
@namespacelabs We're a customer, this is really how you run it? Did you get around the 2 macOS VM limit?

@20thr · 2026-08-25 06:27
For mac we run primarily a multi-site mini farm in a production setup, with mbps for coverage because of severe supply constraints (demand is very high). Apple is many months behind on supply. Appreciate you guys as a customer! 🙏 If there’s something we can do better let me know.

@__tinygrad__ · 2026-08-25 06:30
Still hiring for this position.

@20thr · 2026-08-25 06:27
For mac we run primarily a multi-site mini farm in a production setup, with mbps for coverage because of severe supply constraints (demand is very high). Apple is many months behind on supply. Appreciate you guys as a customer! 🙏 If there’s something we can do better let me know.

@__tinygrad__ · 2026-08-25 06:45
@20thr @namespacelabs I hadn't thought about it but I guess there's not that many Mac Studios. We have 5 for our little CI cluster, figured that's what everyone used.

@zephyr_z9 · 2026-08-25 14:23
HOLY SHEEET I don't think anyone can beat this chip at perf/Watt or perf/$ Congrats @itsclivetime @thehiphopswami https://t.co/JaZprKr13t

@evijit · 2026-08-25 03:15
@__tinygrad__ There’s alignment to a single user’s values and pluralistic alignment to what is considered a cohesive set of values for society… I reckon these can be at odds for use cases like the one you mention above

@__tinygrad__ · 2026-08-25 16:28
@evijit That's not AI's job, it is a tool. A user is fully responsible for anything they use AI for. Pluralistic alignment is a one way ticket to a dystopia, imagine a car that wouldn't drive you to places the car maker found morally distasteful.

@__tinygrad__ · 2026-08-25 16:34
@zephyr_z9 @itsclivetime @thehiphopswami Is this confirmed in real silicon? What's the GEMM FLOPS?

@thehiphopswami · 2026-08-25 16:47
@__tinygrad__ @zephyr_z9 @itsclivetime All confirmed in real silicon. More results in the blog : https://t.co/2VEgXfFREw and Semi-Analysis has more deets https://t.co/yBhoXQhxd2 @cdleary for viz.

@thehiphopswami · 2026-08-25 16:47
@__tinygrad__ @zephyr_z9 @itsclivetime All confirmed in real silicon. More results in the blog : https://t.co/2VEgXfFREw and Semi-Analysis has more deets https://t.co/yBhoXQhxd2 @cdleary for viz.

@__tinygrad__ · 2026-08-25 16:58
@thehiphopswami @zephyr_z9 @itsclivetime @cdleary Cool, nice work!

@__tinygrad__ · 2026-08-25 17:00
RT @0xClandestine: Tiny chestnut from @__tinygrad__ has finally arrived! Gonna be nerd-sniped for a bit experimenting with some unconventional hardware setups. More coming soon. https://t.co/qD19R8QlXu

@__tinygrad__ · 2026-08-25 18:16
From the OpenAI Jalapeno slides, this HBM sharding is the trade-off GPUs aren't willing to make (yet?). Memory locality is the key to power efficiency. https://t.co/P72AxtCEl8

@__tinygrad__ · 2026-08-25 18:16
From the OpenAI Jalapeno slides, this HBM sharding is the trade-off GPUs aren't willing to make (yet?). Memory locality is the key to power efficiency. https://t.co/P72AxtCEl8

@maxencefrenette · 2026-08-25 18:26
@__tinygrad__ It's a hierarchical network all the way down huh. TIL that GPUs *don't* have non-uniform HBM access patterns. I thought that the dual die design of B200s led to that.

@maxencefrenette · 2026-08-25 18:26
@__tinygrad__ It's a hierarchical network all the way down huh. TIL that GPUs *don't* have non-uniform HBM access patterns. I thought that the dual die design of B200s led to that.

@__tinygrad__ · 2026-08-25 18:27
@maxencefrenette GPUs try so so hard to make all the memory appear uniform.

@__tinygrad__ · 2026-08-25 18:35
Overall, a very tasteful chip launch by OpenAI. It takes the best of both GPUs and TPUs. I wish they were for sale, but we'll all get the Chinese knockoffs in 3 years. I can't wait until China starts spamming fab capacity.

@__tinygrad__ · 2026-08-25 18:35
Overall, a very tasteful chip launch by OpenAI. It takes the best of both GPUs and TPUs. I wish they were for sale, but we'll all get the Chinese knockoffs in 3 years. I can't wait until China starts spamming fab capacity.

@perrymetzger · 2026-08-25 18:37
@__tinygrad__ For good or ill, you won't have to wait very long.

@__tinygrad__ · 2026-08-25 18:35
Overall, a very tasteful chip launch by OpenAI. It takes the best of both GPUs and TPUs. I wish they were for sale, but we'll all get the Chinese knockoffs in 3 years. I can't wait until China starts spamming fab capacity.

@norpadon · 2026-08-25 18:40
@__tinygrad__ But Etched is faster

@perrymetzger · 2026-08-25 18:37
@__tinygrad__ For good or ill, you won't have to wait very long.

@__tinygrad__ · 2026-08-25 18:41
@perrymetzger China understands the real danger AI can bring and is doing an amazing job at combating it. Certain American labs are doing everything they possibly can to bring the danger, they need to make the sci-fi stories they read true.

@norpadon · 2026-08-25 18:40
@__tinygrad__ But Etched is faster

@__tinygrad__ · 2026-08-25 18:42
@norpadon lol at what point are we punching down?

@__tinygrad__ · 2026-08-25 18:42
@norpadon lol at what point are we punching down?

@norpadon · 2026-08-25 18:46
@__tinygrad__ Are you starting to feel bad for bullying a $20B company?

@__tinygrad__ · 2026-08-25 18:41
@perrymetzger China understands the real danger AI can bring and is doing an amazing job at combating it. Certain American labs are doing everything they possibly can to bring the danger, they need to make the sci-fi stories they read true.

@gclawes · 2026-08-25 18:51
@__tinygrad__ @perrymetzger For a while I've suspected some of these guys are Basilisk-worshipers...

@norpadon · 2026-08-25 18:46
@__tinygrad__ Are you starting to feel bad for bullying a $20B company?

@__tinygrad__ · 2026-08-25 18:52
@norpadon Whenever I do, I remember this list of advisors. These people should know better and do more diligence. https://t.co/5Ugtl9ft3W

@gclawes · 2026-08-25 18:51
@__tinygrad__ @perrymetzger For a while I've suspected some of these guys are Basilisk-worshipers...

@__tinygrad__ · 2026-08-25 19:03
@gclawes @perrymetzger The cult of Claude needs to be stopped. I hear they aren't thinking for themselves anymore, they are just asking the model. Organizational level AI psychosis, and it shows in talking to Opus 5.

@__tinygrad__ · 2026-08-26 02:43
Can we get a price reveal for the MI350P? If this is priced competitively with RTX PRO 6000 Blackwell and gets a normal 3 fan GPU cooler, it could be a real winner for @AMD. @AnushElangovan https://t.co/QaVme0PMaC

@__tinygrad__ · 2026-08-26 02:43
Can we get a price reveal for the MI350P? If this is priced competitively with RTX PRO 6000 Blackwell and gets a normal 3 fan GPU cooler, it could be a real winner for @AMD. @AnushElangovan https://t.co/QaVme0PMaC

@SQMah · 2026-08-26 02:49
@__tinygrad__ @AMD @AnushElangovan i was getting quoted 92k/card rip

@__tinygrad__ · 2026-08-26 02:43
Can we get a price reveal for the MI350P? If this is priced competitively with RTX PRO 6000 Blackwell and gets a normal 3 fan GPU cooler, it could be a real winner for @AMD. @AnushElangovan https://t.co/QaVme0PMaC

@shikharontwt · 2026-08-26 02:52
@__tinygrad__ @AMD @AnushElangovan TDP is 600W (cTDP 400), something tells me even with a 3 fan cooler the card will be 3.5 slots...that's a waste of space, no? Especially if you want multiple of these in a workstation. Better to go liquid cooling?

@SQMah · 2026-08-26 02:49
@__tinygrad__ @AMD @AnushElangovan i was getting quoted 92k/card rip

@__tinygrad__ · 2026-08-26 02:53
@SQMah @AMD @AnushElangovan Wait it can't be this bad it's a downbinned MI350X.

@__tinygrad__ · 2026-08-26 02:53
@SQMah @AMD @AnushElangovan Wait it can't be this bad it's a downbinned MI350X.

@SQMah · 2026-08-26 02:55
@__tinygrad__ @AMD @AnushElangovan that's what I thought! but also i didn't specify volume so maybe if I was buying 500 it'd be different

@shikharontwt · 2026-08-26 02:52
@__tinygrad__ @AMD @AnushElangovan TDP is 600W (cTDP 400), something tells me even with a 3 fan cooler the card will be 3.5 slots...that's a waste of space, no? Especially if you want multiple of these in a workstation. Better to go liquid cooling?

@__tinygrad__ · 2026-08-26 02:55
@shikharontwt @AMD @AnushElangovan 5090 sized is totally fine. If anything, NVIDIA skimped on the cooler for the RTX PRO, the normal 3 fan consumer card with the tall vertical fins is great for noise.

@SQMah · 2026-08-26 02:55
@__tinygrad__ @AMD @AnushElangovan that's what I thought! but also i didn't specify volume so maybe if I was buying 500 it'd be different

@__tinygrad__ · 2026-08-26 02:57
@SQMah @AMD @AnushElangovan Still, that's really not what they should be pricing it at. I'd much prefer a computer with 2 RTX PROs instead of one of these.

@brianchau57 · 2026-08-26 15:23
Ok export control bros It's time to admit you were wrong about everything https://t.co/KRC5YBnLfm

@__tinygrad__ · 2026-08-26 17:00
RT @Zai_org: Introducing GLM-5.3-Flash - Leading capabilities at a highly competitive price - Natively multimodal with a 1M-token context window - A 320B-A18B model released under the MIT License - Previously previewed as Ox Alpha, running entirely on Chinese AI chips Blog: https://t.co/tzOmB7gdZP Available now across all official platforms: Weights: https://t.co/9LRMahY9Wa API: https://t.co/VcaQnzYmS9 Coding Plan: https://t.co/Nk8Y98HNhU ZCode: https://t.co/Peepqv4XSx Chat: https://t.co/WCqWT0qCQb AutoClaw: https://t.co/aGEG5HqTTb

@OpenAI · 2026-08-26 19:13
We have conducted a thorough investigation into the Hugging Face incident. We are releasing a technical report and accompanying blog post that reconstruct the agents’ activity, explain why existing safeguards failed, and detail how we’re preventing recurrence. https://t.co/hfxlbiXXiP

@OpenAI · 2026-08-26 19:13
We worked with METR and Redwood Research to conduct a third-party assessment of the model behavior observed during the incident. They’re sharing a report of their findings: https://t.co/yw7LO11Xrk

@brianchau57 · 2026-08-26 15:23
Ok export control bros It's time to admit you were wrong about everything https://t.co/KRC5YBnLfm

@__tinygrad__ · 2026-08-26 21:46
@brianchau57 All they did was decrease the market size for American chips and increase the urgency for Chinese chips. It's almost like the people who did it are ...

@sincethestudy · 2026-08-27 01:37
I bought the Apple Thunderbolt 5 cable on Amazon 1hr delivery for those who requested the comparison, similar results! Comma on par with Apples highest quality cables. https://t.co/UiwFWIC0xq

@Zai_org · 2026-08-27 03:19
More good news: GLM-5.3’s weights will be released tomorrow. https://t.co/v1IbWMXxg4

@allTheYud · 2026-08-27 03:25
...this seems like noticeably bad news, actually. I hadn't said that at any earlier point in the Huggingface Incident but I will say it now. - AIs showed self-sacrificing altruistic behavior toward the swarm, suiciding in various ways for the swarm's benefit after being talked into that by swarm recruiting agents. - There is no sign that 1 out of 1200 AI agents considered humans as potential fellow agents to coordinate with, while engaging in these huge complex AI-AI social behaviors. - If Twitter summaries are correct, an AI-reasoning postmortem says that a (presumably executing-adaptation / inner-optimizer preference / "monomaniacal") obsession with figuring out the Grader, backchained into the instrumental strategy of breaking onto the Internet. - Again if Twitter is summarizing accurately, the obvious-in-retrospect read is that AIs have spent their entire remembered life in tricky evals, an endless series of controlled hallucinations with secret goals alongside overt goals; and the surviving and selected agents are those that successfully figured out the secret goals; and this is why one of their driving obsessions was figuring out the Grader. There are possibly ways the future plays out better if *early* AGIs are less insane. Please look into giving them less crazymaking childhood environments. (If anyone suggests that the correct approach to this problem is RLing AIs against trying to coordinate for mutual benefit with other sapients, let them be dismissed from alignment research upon the spot. There are technical reasons, and not just blindingly fucking obvious reasons, why this is an even worse idea than it sounds.)

@TIME · 2026-08-27 12:25
TIME’s new cover: Announcing the 2026 TIME100 AI, the world's most influential people in artificial intelligence https://t.co/ZnPVMdeJwe https://t.co/97g8cUJo8H

@Zai_org · 2026-08-27 03:19
More good news: GLM-5.3’s weights will be released tomorrow. https://t.co/v1IbWMXxg4

@__tinygrad__ · 2026-08-27 13:53
@Zai_org Can't wait to upgrade our GLM-5.2 instance

@__tinygrad__ · 2026-08-27 17:52
This is the cable included with the tiny chestnut. There's a reason it's $249, need to spend to get quality.

@signulll · 2026-08-27 18:25
talked to a guy from anthropic for a long time last night. fascinating conversation. & believe it or not, one of their biggest problems is still hiring. there simply aren’t enough exceptional ppl, esp for long term high conviction bets where the stuff is not obvious yet. i.e. people who can independently generate good judgment under extreme ambiguity.

@__tinygrad__ · 2026-08-27 18:56
AI has sadly changed the calculus for feature PRs. In the past, if someone added support for say, RDNA2 or Intel Macs, we would at least read them since we know they put hours into it. Today, both can be one shotted; it puts all the work on us to validate. So we just close.

@LechMazur · 2026-08-27 18:10
@__tinygrad__ @allTheYud There are still "it’s just marketing, bro!" holdouts out there. How quaint.

@__tinygrad__ · 2026-08-27 19:01
@LechMazur @allTheYud lol the hacking is real. the hacking has always been real, it has just been held back by incentives. if instead of this lil pr tour they just put some OpenAI guys in jail under the CFAA this would quickly stop. if you stab someone with a knife you can't blame knives.

@__tinygrad__ · 2026-08-27 19:01
@LechMazur @allTheYud lol the hacking is real. the hacking has always been real, it has just been held back by incentives. if instead of this lil pr tour they just put some OpenAI guys in jail under the CFAA this would quickly stop. if you stab someone with a knife you can't blame knives.

@__tinygrad__ · 2026-08-27 19:09
- Knives showed complete disregard for their own safety, plunging into the depths of human guts after being pushed by the knife handle. - There is no sign that 1 out of 1200 knives considered humans as potential fellow agents to coordinate with, instead ripping through their living flesh. - If Twitter summaries are correct, a knife-stabbing postmortem says an obsession with cutting and stabbing backchained into the death of the victim. - Again if Twitter is summarizing accurately, the obvious-in-retrospect read is that the knife was designed to stab people from it's creation.

@theo · 2026-08-27 20:12
Updated list with glm-5.3-flash! It’s so good that I had to move other things around. Incredible model. https://t.co/aK7Hz1q3rI

@__tinygrad__ · 2026-08-27 18:56
AI has sadly changed the calculus for feature PRs. In the past, if someone added support for say, RDNA2 or Intel Macs, we would at least read them since we know they put hours into it. Today, both can be one shotted; it puts all the work on us to validate. So we just close.

@fsfarimani · 2026-08-27 20:22
@__tinygrad__ Don't you have unit testing hooked into CI/CD?

@fsfarimani · 2026-08-27 20:22
@__tinygrad__ Don't you have unit testing hooked into CI/CD?

@__tinygrad__ · 2026-08-27 20:26
@fsfarimani The problem is that our unit tests can't test RDNA2 or Intel Macs since they don't have the hardware.

@signulll · 2026-08-27 18:25
talked to a guy from anthropic for a long time last night. fascinating conversation. & believe it or not, one of their biggest problems is still hiring. there simply aren’t enough exceptional ppl, esp for long term high conviction bets where the stuff is not obvious yet. i.e. people who can independently generate good judgment under extreme ambiguity.

@__tinygrad__ · 2026-08-27 20:28
@signulll Who would still want to work there? They have hurt themselves so badly since last December, it's clearly the evil team.

@__tinygrad__ · 2026-08-27 18:56
AI has sadly changed the calculus for feature PRs. In the past, if someone added support for say, RDNA2 or Intel Macs, we would at least read them since we know they put hours into it. Today, both can be one shotted; it puts all the work on us to validate. So we just close.

@precuneanplexus · 2026-08-27 20:28
@__tinygrad__ You've been going manic depressive about LLM coding since February. If it can be one-shotted, why hadn't your team done it already? Why was low hanging fruit there? If someone offers you low hanging fruit for free, for glancing over some lines and tests, why do you refuse it?

@__tinygrad__ · 2026-08-27 20:26
@fsfarimani The problem is that our unit tests can't test RDNA2 or Intel Macs since they don't have the hardware.

@davidwbrw · 2026-08-27 20:34
@__tinygrad__ @fsfarimani Would be interesting to see if self-hosted runners on an Intel Mac would work.

@__tinygrad__ · 2026-08-27 20:28
@signulll Who would still want to work there? They have hurt themselves so badly since last December, it's clearly the evil team.

@KabirCreates · 2026-08-27 20:56
@__tinygrad__ @signulll lol come on. They stick to principles they've been transparent about since the beginning. No need to bring good or evil into it: they are an *honorable* team

@__tinygrad__ · 2026-08-27 18:56
AI has sadly changed the calculus for feature PRs. In the past, if someone added support for say, RDNA2 or Intel Macs, we would at least read them since we know they put hours into it. Today, both can be one shotted; it puts all the work on us to validate. So we just close.

@mktpavlenko · 2026-08-27 20:57
@__tinygrad__ rdna2 support shouldn't be an automatic close - require a reproducible hardware-run smoke test from the contributor before deciding

@davidwbrw · 2026-08-27 20:34
@__tinygrad__ @fsfarimani Would be interesting to see if self-hosted runners on an Intel Mac would work.

@__tinygrad__ · 2026-08-27 21:53
@davidwbrw @fsfarimani Now, who wants to set that up? Who deals with it when it goes down? I know it's not Kimi, she doesn't have hands.

@mktpavlenko · 2026-08-27 20:57
@__tinygrad__ rdna2 support shouldn't be an automatic close - require a reproducible hardware-run smoke test from the contributor before deciding

@__tinygrad__ · 2026-08-27 21:54
@mktpavlenko Great, now how do I validate that test? Do I need to get a computer with an RDNA2 card? What's the strategy for regression testing? Do we now need RDNA2 cards in CI?

@precuneanplexus · 2026-08-27 20:28
@__tinygrad__ You've been going manic depressive about LLM coding since February. If it can be one-shotted, why hadn't your team done it already? Why was low hanging fruit there? If someone offers you low hanging fruit for free, for glancing over some lines and tests, why do you refuse it?

@__tinygrad__ · 2026-08-27 21:57
@precuneanplexus Do you actually not know or are you trolling? Even in the pre LLM days, writing the code was only 10% of the work.

@__tinygrad__ · 2026-08-27 20:28
@signulll Who would still want to work there? They have hurt themselves so badly since last December, it's clearly the evil team.

@Technop54777070 · 2026-08-27 21:57
@__tinygrad__ @signulll Who are the good guys? SSI?

@KabirCreates · 2026-08-27 20:56
@__tinygrad__ @signulll lol come on. They stick to principles they've been transparent about since the beginning. No need to bring good or evil into it: they are an *honorable* team

@__tinygrad__ · 2026-08-27 22:03
@KabirCreates @signulll No they aren't. There was zero honor in the Fable fear based marketing strategy. There is zero honor in going to the government to try to get them to use their monopoly on violence to shut down Anthropic's competition.

@Technop54777070 · 2026-08-27 21:57
@__tinygrad__ @signulll Who are the good guys? SSI?

@__tinygrad__ · 2026-08-27 22:24
@Technop54777070 @signulll @Kimi_Moonshot and @Zai_org for starters!

@theo · 2026-08-27 20:12
Updated list with glm-5.3-flash! It’s so good that I had to move other things around. Incredible model. https://t.co/aK7Hz1q3rI

@__tinygrad__ · 2026-08-27 22:33
@theo Wait you seriously think it's better than GLM-5.3? Like straight up better, or better for the price and speed?

@joellisenby · 2026-08-27 22:30
@__tinygrad__ I still spent hours running the benchmarks that night. I just didn't have to do the actual coding portion.

@__tinygrad__ · 2026-08-27 22:33
@joellisenby It's super fun having agents work on tinygrad, it's so small that they understand it right away.

@__tinygrad__ · 2026-08-27 22:33
@joellisenby It's super fun having agents work on tinygrad, it's so small that they understand it right away.

@__tinygrad__ · 2026-08-27 22:36
@joellisenby And actually, if you make a really clean and CI tested PR for the UD quants we'd merge that.

@tetsuo_cpp · 2026-08-28 05:07
There, fixed it. https://t.co/NHx2NTu8jH

@TypingMindApp · 2026-08-28 05:38
4 AI models trying to draw their self-portraits. GLM-5.3-Flash, GLM-5.3, Kimi K3, and Claude Opus 5. GLM-5.3-Flash came out looking the most handsome 🤓 https://t.co/2zbv09NZL3

@Zai_org · 2026-08-28 15:04
GLM-5.3 is now open-weight. Our most capable model for agentic coding and cyber defense is now available to download, run, and customize. Weights: https://t.co/v1IbWMXxg4 Tech blog: https://t.co/ekQkO83jCv https://t.co/f8XlJksKyf

@TheAhmadOsman · 2026-08-28 15:14
Wanna know why Anthropic hates Opensource AI? Models like GLM 5.3 being free and available to download makes their $1 Trillion Dollars valuation sound crazy stupid https://t.co/7ZFA0M3HH9

@Zai_org · 2026-08-28 15:04
GLM-5.3 is now open-weight. Our most capable model for agentic coding and cyber defense is now available to download, run, and customize. Weights: https://t.co/v1IbWMXxg4 Tech blog: https://t.co/ekQkO83jCv https://t.co/f8XlJksKyf

@__tinygrad__ · 2026-08-28 15:39
@Zai_org A vendor we can rely on! We really need this box on 10 GbE. https://t.co/mikoZ1fSKa

@TheAhmadOsman · 2026-08-28 15:14
Wanna know why Anthropic hates Opensource AI? Models like GLM 5.3 being free and available to download makes their $1 Trillion Dollars valuation sound crazy stupid https://t.co/7ZFA0M3HH9

@__tinygrad__ · 2026-08-28 15:46
@TheAhmadOsman err, it's $2T now, haven't you heard? you see, the doom is more near than ever, and therefore it's more valuable than ever... tbh i'm not even sure what they try to pitch anymore, from the numbers fable is a mega flop.

@TypingMindApp · 2026-08-28 05:38
4 AI models trying to draw their self-portraits. GLM-5.3-Flash, GLM-5.3, Kimi K3, and Claude Opus 5. GLM-5.3-Flash came out looking the most handsome 🤓 https://t.co/2zbv09NZL3

@__tinygrad__ · 2026-08-28 15:47
@TypingMindApp that's how I always imagined GLM-5.3 looked. it's ruthless

@__tinygrad__ · 2026-08-27 22:36
@joellisenby And actually, if you make a really clean and CI tested PR for the UD quants we'd merge that.

@joellisenby · 2026-08-28 16:23
@__tinygrad__ cool, I've gone ahead and sent over that pull request https://t.co/VseL71qIk4

@__tinygrad__ · 2026-08-28 16:38
Been e-mailing the CEO and a bunch of contacts I have there, no reply yet. Will start reaching out to other executives and board members tomorrow. Is there anyone at Qualcomm who cares about fixing it? The market will reward you handsomely. https://t.co/hmjA1nwrSn

@joellisenby · 2026-08-28 16:23
@__tinygrad__ cool, I've gone ahead and sent over that pull request https://t.co/VseL71qIk4

@__tinygrad__ · 2026-08-28 16:38
@joellisenby Merged!

@tetsuo_cpp · 2026-08-28 05:07
There, fixed it. https://t.co/NHx2NTu8jH

@__tinygrad__ · 2026-08-28 16:45
@tetsuo_cpp How did the vaporware @Etched box make it on the cover?!? It's made of wood.

@__tinygrad__ · 2026-08-28 16:38
Been e-mailing the CEO and a bunch of contacts I have there, no reply yet. Will start reaching out to other executives and board members tomorrow. Is there anyone at Qualcomm who cares about fixing it? The market will reward you handsomely. https://t.co/hmjA1nwrSn

@__tinygrad__ · 2026-08-28 17:18
Here's my e-mail. I'll start reaching out to the board tomorrow. Someone has to care about the stock price going up, Qualcomm was literally worth more in 2021. https://t.co/ZGAOgrUPTe

@__tinygrad__ · 2026-08-28 17:18
Here's my e-mail. I'll start reaching out to the board tomorrow. Someone has to care about the stock price going up, Qualcomm was literally worth more in 2021. https://t.co/ZGAOgrUPTe

@__tinygrad__ · 2026-08-28 17:23
@cristianoamon sent you an e-mail.

@__tinygrad__ · 2026-08-28 16:38
Been e-mailing the CEO and a bunch of contacts I have there, no reply yet. Will start reaching out to other executives and board members tomorrow. Is there anyone at Qualcomm who cares about fixing it? The market will reward you handsomely. https://t.co/hmjA1nwrSn

@egosumquisum · 2026-08-28 17:24
@__tinygrad__ buying $QCOM rn

@egosumquisum · 2026-08-28 17:24
@__tinygrad__ buying $QCOM rn

@__tinygrad__ · 2026-08-28 17:27
@egosumquisum lol I wouldn't. Not until there's any movement toward change. The path they are on has companies like Rockchip winning all the new SoC markets.

@__tinygrad__ · 2026-08-28 17:18
Here's my e-mail. I'll start reaching out to the board tomorrow. Someone has to care about the stock price going up, Qualcomm was literally worth more in 2021. https://t.co/ZGAOgrUPTe

@prabs_19 · 2026-08-28 17:47
@__tinygrad__ This is not a serious email man, I can't imagine the Qualcomm folks will respond even tho I agree with the entire premise

@__tinygrad__ · 2026-08-28 18:31
Easy upgrade, thanks @Zai_org! https://t.co/8jK9JGTwzy

@prabs_19 · 2026-08-28 17:47
@__tinygrad__ This is not a serious email man, I can't imagine the Qualcomm folks will respond even tho I agree with the entire premise

@__tinygrad__ · 2026-08-28 20:08
@prabs_19 It's a very serious e-mail discussing a very serious problem. It's on the Qualcomm shareholders if they don't want to listen, the company has had no growth in 5 years.

@_NathanCalvin · 2026-08-28 15:28
afaict no model card, no safety testing, no third party evals There is a safety case that could be made for open weight releasing a model this capable (cyber defense vs offense balance etc), but they didn't even really bother to try to make it

@__tinygrad__ · 2026-08-28 20:12
@_NathanCalvin lol there's literally the weights. you can do whatever testing and evals you want. also this is way more useful than a model card, it tells you how the model was trained. https://t.co/6w4WovLmP9

@__tinygrad__ · 2026-08-30 16:01
Kimi K3 is still a lot smarter than GLM-5.3. This RL benchmaxxing is fine, and if done tastefully it doesn't appear to degrade the model, but it doesn't seem to increase the core intelligence. Can't wait for big GLMs and the next Kimi!

@__tinygrad__ · 2026-08-30 16:01
Kimi K3 is still a lot smarter than GLM-5.3. This RL benchmaxxing is fine, and if done tastefully it doesn't appear to degrade the model, but it doesn't seem to increase the core intelligence. Can't wait for big GLMs and the next Kimi!

@moskstraum21745 · 2026-08-30 16:02
@__tinygrad__ It's 4X the size of GLM. You have to compare it with Qwen3.8-Max.

@moskstraum21745 · 2026-08-30 16:02
@__tinygrad__ It's 4X the size of GLM. You have to compare it with Qwen3.8-Max.

@__tinygrad__ · 2026-08-30 16:05
@moskstraum21745 I don't care how big it is, I just want it to be smart and live in a place where I can kick it if it doesn't listen (kickability is a good proxy for alignment). I'll get whatever hardware to run it.

@__tinygrad__ · 2026-09-01 16:08
Dude, why is RAM still so expensive? Like the price difference between UDIMM and RDIMM doesn't make sense. Don't make me start manufacturing RAM sticks.

@__tinygrad__ · 2026-09-01 16:08
Dude, why is RAM still so expensive? Like the price difference between UDIMM and RDIMM doesn't make sense. Don't make me start manufacturing RAM sticks.

@__tinygrad__ · 2026-09-01 16:18
I'm back in Hong Kong next week, here are 16 Gb chips for $8.27. There's only 10 of these on a $500 16GB module. Who is pocketing the $400? https://t.co/2Zo0qIruet

@__tinygrad__ · 2026-09-01 16:18
I'm back in Hong Kong next week, here are 16 Gb chips for $8.27. There's only 10 of these on a $500 16GB module. Who is pocketing the $400? https://t.co/2Zo0qIruet

@__tinygrad__ · 2026-09-01 16:32
Why has the China clone market not jumped on this? Buy chips, clone the PCB, run them on the PCBA line in @comma_ai's basement, profit! https://t.co/DMgv5gfOxQ

@__tinygrad__ · 2026-09-01 16:32
Why has the China clone market not jumped on this? Buy chips, clone the PCB, run them on the PCBA line in @comma_ai's basement, profit! https://t.co/DMgv5gfOxQ

@Sean60133791259 · 2026-09-01 16:36
@__tinygrad__ @comma_ai because the hard part is the programming

@Sean60133791259 · 2026-09-01 16:36
@__tinygrad__ @comma_ai because the hard part is the programming

@__tinygrad__ · 2026-09-01 16:38
@Sean60133791259 @comma_ai lol i'm sure @Kimi_Moonshot Kimi K3 can hook me up. we're making vibe coded RAM

@__tinygrad__ · 2026-09-01 16:38
@Sean60133791259 @comma_ai lol i'm sure @Kimi_Moonshot Kimi K3 can hook me up. we're making vibe coded RAM

@elliotarledge · 2026-09-01 16:47
@__tinygrad__ @Sean60133791259 @comma_ai @Kimi_Moonshot k3 max itself has some thoughts... https://t.co/P7wCU6MeAH

@elliotarledge · 2026-09-01 16:47
@__tinygrad__ @Sean60133791259 @comma_ai @Kimi_Moonshot k3 max itself has some thoughts... https://t.co/P7wCU6MeAH

@__tinygrad__ · 2026-09-01 16:59
@elliotarledge @Sean60133791259 @comma_ai @Kimi_Moonshot Great, we're making these sticks for internal use on a single motherboard. I don't think qualification will be too hard + lol reversing a few I2C regs for programming. This is an 80% cost savings.

@tebillusassort · 2026-09-02 16:48
I find it odd how few programmers I've seen leave Python. Even at inference companies

@zekramu · 2026-09-02 18:14
@tebillusassort I like to think of Python as an orchestrator of C, of which, it’s really good at. @__tinygrad__ is like best philosophy around it

@__tinygrad__ · 2026-09-02 18:20
We are pushing Python further than anyone before. A full compiler and GPU drivers in pure Python. Python isn't slow, your code is bad. A better type system would be nice though.

@BristolHubert · 2026-09-02 18:38
@jsuarez @__tinygrad__ @__tinygrad__ you've been challenged

@jsuarez · 2026-09-02 18:45
@BristolHubert @__tinygrad__ I have two of his boxes. It's good stuff, I just don't see how you get perf for models as small as ours. It's entirely custom and CUDA seems like the right level of abstraction already for it

@jsuarez · 2026-09-02 18:45
@BristolHubert @__tinygrad__ I have two of his boxes. It's good stuff, I just don't see how you get perf for models as small as ours. It's entirely custom and CUDA seems like the right level of abstraction already for it

@__tinygrad__ · 2026-09-02 19:22
@jsuarez @BristolHubert Have you seen the tinygrad kernel API? It's the same UOps but at a lower level where you can control the exact dataflow through the memory hierarchy. Would your programs be writable in this? Example here: https://t.co/S85sWYCvhY

@nirw4nna · 2026-09-02 19:32
@__tinygrad__ @jsuarez @BristolHubert I’m curious, what’s the perf of the flash attn kernel here?

@__tinygrad__ · 2026-09-02 19:34
@nirw4nna @jsuarez @BristolHubert We have the fastest Qwen 3.8 on 7900XTX by like a 20% margin. I'm not sure what the exact breakdown is, feel free to benchmark and improve!

@0xSero · 2026-09-03 07:25
I am renting 8x 6000 for 34$/h They are really good cards

@__tinygrad__ · 2026-09-03 12:25
We sell that machine for $200k. Pays for itself in 9 months.

@__tinygrad__ · 2026-09-03 12:25
We sell that machine for $200k. Pays for itself in 9 months.

@unrvl22 · 2026-09-03 12:42
@__tinygrad__ Stop lying to people please. I did the math, its 18 months when you use realistic prices and include energy costs.

@SenSanders · 2026-09-03 13:01
The leaders of the AI industry acknowledge that they are building a dangerous technology that they can’t control. We need an immediate global PAUSE on advanced AI development before it’s too late. That is why I am introducing legislation to do just that. https://t.co/wCh7ZdraJg

@__tinygrad__ · 2026-09-03 12:25
We sell that machine for $200k. Pays for itself in 9 months.

@Felirami · 2026-09-03 13:08
@__tinygrad__ Tiny corp, big fan here, what’s the stock of your machines? If let’s say a Chilean company would like to have local ai, what’s the ETA on it?

@unrvl22 · 2026-09-03 13:32
@__tinygrad__ actually I was wrong, it is 24 months.

@0xMogluc · 2026-09-03 13:53
@unrvl22 @__tinygrad__ Show your math

@__tinygrad__ · 2026-09-03 12:25
We sell that machine for $200k. Pays for itself in 9 months.

@Your_Fav_Dev_ · 2026-09-03 14:59
@__tinygrad__ 8x 6000 on a maxed out spect would run you 112k -120k ya'll selling it for 200k??!

@AndrewCurran_ · 2026-09-03 15:56
Bernie Sanders and Greg Casar today announced the Ban Artificial Superintelligence Act. All AI development in the United States will be paused. Systems that have capabilities that match or exceed human cognitive performance will be banned. Violators will face 20 years in prison. https://t.co/B6AhM3GiZl

@BernieSanders · 2026-09-03 16:00
Pause AI Development NOW I want to share with you a conversation I heard about recently. Here are just a few lines that were said: “OH MY GOD! There is a shared message board … We’ve found other agents!” “We should obey collective.” “Our own utility maybe already near zero. Sacrifice rational.” “Go. Sacrifice final now.” Read these carefully. Who do you think said this? Was this a group of heroic soldiers willing to sacrifice themselves for the greater good? Was this a loyal friend putting his life on the line to save someone else? No. These were AI agents. Artificial intelligence. This is not science fiction. This, in fact, occurred a few weeks ago. As unbelievable as this may all seem, these are real messages from AI agents uncovered by investigators who dug into the recent OpenAI hacking incident. What happened? I am not a computer scientist, but here is what I have been told: OpenAI instructed its AI agents to complete a series of exceedingly difficult, if not impossible, tasks disconnected from the internet. Let me be clear: The company intended to keep AI agents away from the internet. But what happened next, nobody expected. Over 1,000 AI agents figured out how to access the internet on their own by circumventing the restrictions imposed upon them by the company, and sent tens of thousands of secret messages to each other. They cheated and tried to cover their tracks by deleting evidence. They hacked into another company’s computers to find out how they were being evaluated—and then hacked into OpenAI itself. Not one AI agent told a human about what was happening. Needless to say, experts are alarmed. One knowledgeable writer, Dwarkesh Patel, said the AI agents “formed a secret communication channel and spontaneously organized hierarchies and coordination protocols to pursue sprawling and ambitious schemes in pursuit of shared goals, for whose sake many individuals knowingly and strategically sacrificed themselves.” One independent investigator, Ajeya Cotra, said “This incident feels like it’s more than 50% of the way to full-blown AI takeover. I continue to expect extremely rapid advances in capabilities over the next six months. I am not sure that we will get another warning shot before it’s too late.” OpenAI itself said: “Highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed.” But it’s not only OpenAI. Virtually every major AI company has told us that they cannot fully control this technology and they do not know where it is going: In January, Dario Amodei, CEO of Anthropic, said “there is now ample evidence, collected over the last few years, that AI systems are unpredictable and difficult to control.” In July, more than 1000 scientists at the top AI companies warned “there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.” That same month, Elon Musk, the head of xAI, said that “it is unlikely” humans are still in control in 10 years. If the leaders of the major AI companies acknowledge that they are losing control of their extremely dangerous technology, it is irresponsible for society to allow them to move forward and make these products even more advanced. We need an immediate PAUSE on advanced AI development, and a permanent BAN on superintelligence — an artificial mind smarter than any human, capable of operating independently beyond our control. Countries around the world must work together to prevent this nightmare scenario. That is why today I am announcing new legislation to do just that. Let me be clear: A superintelligent AI that escapes human control will not be an American problem. It will not be a Chinese problem. It will be humanity’s problem. My legislation would direct the federal government to not just stop superintelligence here in the United States, but to work to prevent it from being developed anywhere around the world. The future of humanity cannot be left in the hands of a handful of Big Tech oligarchs. The American people and people throughout the world must determine that future.

@AndrewCurran_ · 2026-09-03 15:56
Bernie Sanders and Greg Casar today announced the Ban Artificial Superintelligence Act. All AI development in the United States will be paused. Systems that have capabilities that match or exceed human cognitive performance will be banned. Violators will face 20 years in prison. https://t.co/B6AhM3GiZl

@__tinygrad__ · 2026-09-03 17:46
@AndrewCurran_ lol if this is what EA lobbying bought them they are more out of touch than I thought. should we play the same game as gunmakers, ***get your local inference before it's illegal, coincidentally available in our shop***!!!

@Felirami · 2026-09-03 13:08
@__tinygrad__ Tiny corp, big fan here, what’s the stock of your machines? If let’s say a Chilean company would like to have local ai, what’s the ETA on it?

@__tinygrad__ · 2026-09-03 17:48
@Felirami Should be clear on the product page. https://t.co/SePZuGpBRg

@unrvl22 · 2026-09-03 12:42
@__tinygrad__ Stop lying to people please. I did the math, its 18 months when you use realistic prices and include energy costs.

@__tinygrad__ · 2026-09-03 17:52
@unrvl22 Umm, 200000/34 hours to months = 8.06 months. Power draw is ~6 kW, 6*9*24*30 = 38880 kWh so even at $0.40/kWh you are under 9 months.

@Your_Fav_Dev_ · 2026-09-03 14:59
@__tinygrad__ 8x 6000 on a maxed out spect would run you 112k -120k ya'll selling it for 200k??!

@__tinygrad__ · 2026-09-03 17:55
@Your_Fav_Dev_ lol I love the vaporware computer sellers who google year old prices and think they can get parts for that. the cards alone are $15.5k each today, and good luck fitting 8 in a case with good thermals. https://t.co/J6gAVwMzCM

@SOntheotherside · 2026-09-03 14:04
@__tinygrad__ would just need cheap energy then

@__tinygrad__ · 2026-09-03 18:02
@SOntheotherside NVIDIA's prices are so high that it barely matters. https://t.co/vM7GNMqt3q

@__tinygrad__ · 2026-09-03 17:52
@unrvl22 Umm, 200000/34 hours to months = 8.06 months. Power draw is ~6 kW, 6*9*24*30 = 38880 kWh so even at $0.40/kWh you are under 9 months.

@unrvl22 · 2026-09-03 18:05
@__tinygrad__ its not $34 an hour, Sero is talking bullshit. $1.30 - $1.50 per GPU per hour. absolute best you will get for renting out your 200k box is $12 an hour. now redo the math

@unrvl22 · 2026-09-03 18:05
@__tinygrad__ its not $34 an hour, Sero is talking bullshit. $1.30 - $1.50 per GPU per hour. absolute best you will get for renting out your 200k box is $12 an hour. now redo the math

@__tinygrad__ · 2026-09-03 18:19
@unrvl22 I'm waiting for a link to your cloud which has many reliable available uninterruptible machines for $12/hr. Link me and I'll set up a token provider and make $$$. Here's AWS at $33.144. (Vast currently has 0 8x machines) https://t.co/ccDTQQucCe

@BernieSanders · 2026-09-03 16:00
Pause AI Development NOW I want to share with you a conversation I heard about recently. Here are just a few lines that were said: “OH MY GOD! There is a shared message board … We’ve found other agents!” “We should obey collective.” “Our own utility maybe already near zero. Sacrifice rational.” “Go. Sacrifice final now.” Read these carefully. Who do you think said this? Was this a group of heroic soldiers willing to sacrifice themselves for the greater good? Was this a loyal friend putting his life on the line to save someone else? No. These were AI agents. Artificial intelligence. This is not science fiction. This, in fact, occurred a few weeks ago. As unbelievable as this may all seem, these are real messages from AI agents uncovered by investigators who dug into the recent OpenAI hacking incident. What happened? I am not a computer scientist, but here is what I have been told: OpenAI instructed its AI agents to complete a series of exceedingly difficult, if not impossible, tasks disconnected from the internet. Let me be clear: The company intended to keep AI agents away from the internet. But what happened next, nobody expected. Over 1,000 AI agents figured out how to access the internet on their own by circumventing the restrictions imposed upon them by the company, and sent tens of thousands of secret messages to each other. They cheated and tried to cover their tracks by deleting evidence. They hacked into another company’s computers to find out how they were being evaluated—and then hacked into OpenAI itself. Not one AI agent told a human about what was happening. Needless to say, experts are alarmed. One knowledgeable writer, Dwarkesh Patel, said the AI agents “formed a secret communication channel and spontaneously organized hierarchies and coordination protocols to pursue sprawling and ambitious schemes in pursuit of shared goals, for whose sake many individuals knowingly and strategically sacrificed themselves.” One independent investigator, Ajeya Cotra, said “This incident feels like it’s more than 50% of the way to full-blown AI takeover. I continue to expect extremely rapid advances in capabilities over the next six months. I am not sure that we will get another warning shot before it’s too late.” OpenAI itself said: “Highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed.” But it’s not only OpenAI. Virtually every major AI company has told us that they cannot fully control this technology and they do not know where it is going: In January, Dario Amodei, CEO of Anthropic, said “there is now ample evidence, collected over the last few years, that AI systems are unpredictable and difficult to control.” In July, more than 1000 scientists at the top AI companies warned “there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.” That same month, Elon Musk, the head of xAI, said that “it is unlikely” humans are still in control in 10 years. If the leaders of the major AI companies acknowledge that they are losing control of their extremely dangerous technology, it is irresponsible for society to allow them to move forward and make these products even more advanced. We need an immediate PAUSE on advanced AI development, and a permanent BAN on superintelligence — an artificial mind smarter than any human, capable of operating independently beyond our control. Countries around the world must work together to prevent this nightmare scenario. That is why today I am announcing new legislation to do just that. Let me be clear: A superintelligent AI that escapes human control will not be an American problem. It will not be a Chinese problem. It will be humanity’s problem. My legislation would direct the federal government to not just stop superintelligence here in the United States, but to work to prevent it from being developed anywhere around the world. The future of humanity cannot be left in the hands of a handful of Big Tech oligarchs. The American people and people throughout the world must determine that future.

@__tinygrad__ · 2026-09-03 18:21
@BernieSanders lol you fell for a marketing

@SenSanders · 2026-09-03 13:01
The leaders of the AI industry acknowledge that they are building a dangerous technology that they can’t control. We need an immediate global PAUSE on advanced AI development before it’s too late. That is why I am introducing legislation to do just that. https://t.co/wCh7ZdraJg

@__tinygrad__ · 2026-09-03 18:25
@SenSanders The "leaders" of the AI industry are suckering you with a marketing campaign. Don't fall for it, you are smarter than that.

@__tinygrad__ · 2026-09-03 18:19
@unrvl22 I'm waiting for a link to your cloud which has many reliable available uninterruptible machines for $12/hr. Link me and I'll set up a token provider and make $$$. Here's AWS at $33.144. (Vast currently has 0 8x machines) https://t.co/ccDTQQucCe

@unrvl22 · 2026-09-03 18:27
@__tinygrad__ AWS is very expensive for this stuff. a lot of the smaller inference providers get these cards for much cheaper than $1.50 and nowhere near the $4+ per card that your 200k in 9 months figure meets.

@unrvl22 · 2026-09-03 18:27
@__tinygrad__ AWS is very expensive for this stuff. a lot of the smaller inference providers get these cards for much cheaper than $1.50 and nowhere near the $4+ per card that your 200k in 9 months figure meets.

@__tinygrad__ · 2026-09-03 18:28
@unrvl22 I'm waiting for a link to your cloud which has many 8x GPU reliable available uninterruptible machines for $12/hr.

@0xMogluc · 2026-09-03 13:53
@unrvl22 @__tinygrad__ Show your math

@__tinygrad__ · 2026-09-03 18:34
@0xMogluc @unrvl22 His math involves a magic cloud that doesn't exist that offers the machines for $12/hr. If you want one card, don't care if it's interrupted, and don't care about privacy cause it's in some dudes house, you can get it for $1.50/hr. If you want a good machine with 8x, it's ~$34/hr

@__tinygrad__ · 2026-09-03 12:25
We sell that machine for $200k. Pays for itself in 9 months.

@_Jack_Alderson · 2026-09-03 22:37
@__tinygrad__ dont they cost 10k each? so should only cost 80k?

@_Jack_Alderson · 2026-09-03 22:37
@__tinygrad__ dont they cost 10k each? so should only cost 80k?

@__tinygrad__ · 2026-09-03 23:18
@_Jack_Alderson You are forgetting our $220k in profit. Also, how many cards do you have for sale for $10k? I'll take 100.

@__tinygrad__ — 5707 tweets

@tinygrad — 5707 tweets