But I wonder if there could be room for an ARM-like spec that Google could try and own and license but for AI chips. Arm is to risc-cpu as google-thing is to asic-aichip
Prolly a dumb idea, better to sell the chips or access to them?
I'm not sure the chip spec (or instruction set) is the level of abstraction here?
Something like DirectX (or OpenGL) might be the better level to target? In practice, CUDA is that level of abstraction, but it only really works for Nvidia cards.
It's not that it only works on Nvidia cards, it's only allowed to work on Nvidia cards. A non-clean room implementation of CUDA for other hardware has been done but is a violation of EULA (of the thing that was reverse engineered), copyright on the driver binary interface, and often patents. Nvidia aggressively sends cease-and-desist letters and threatens lawsuits (successfully killed ZLUDA, threatened others). It's an artificial (in a technical sense moat).
To circumvent: you have someone (who might be bound by the EULA, and is otherwise not affiliated with you) dump the data on the internet, and someone else (from your company) can find it there, without being bound by the EULA. Nvidia could only sue the first guy for violating the EULA.
However you are right, that copyright and patents still bite.
Not really, because as usual people misunderstand what CUDA is.
CUDA is hardware designed according to the C++ memory model, with first tier support for C, C++, Fortran and Python GPGPU DSLs, with several languages also having a compiler backend for PTX.
Followed by IDE integration, a graphical debugger and profiler for GPU workloads, and an ecosystem of libraries and frameworks.
Saying just use DirectX, Vulkan, OpenGL instead, misses the tree from the forest that is CUDA, and why researchers rather use CUDA, than deal with yet another shading language or C99 dialect, without anything else.
These are only available in Iowa on GCP, which to me raises this question: do they have them all over the world for their own purposes, or does this limited geography also mean that users of Google AI features get varied experiences depending on their location?
I'll never understand this attitude. Recently I set up a full network with 5 computers, opnsense, xcp-ng and a few things like a pi, switch, AP, etc.
I was migrating from pfsense to Opnsense so I wasn't too familiar with some of the nitty gritty. Was migrating to xcp-ng 8.3 from 8.2 which has some major CLI differences. It was a pretty big migration that took me a full weekend.
OpenAI got things wrong (mostly because it was using old documentation - opnsense had just upgraded) maybe 8 times in the whole project and was able to quickly correct itself when I elaborated on the problem.
If I just had google this would've been a 2 week project easily. I'd have to drudge through extremely dry documentation that mostly doesn't apply to anything I'm doing. Would have to read a bunch of toxic threads demeaning users who don't know everything. Instead I had chatgpt 5 do all that for me and got to the exact same result with a tenth of the effort.
The AI is useless crowd truly makes me scratch my head.
> The AI is useless crowd truly makes me scratch my head.
I think it's because, past autocomplete, for AI to be useful professionally you need to already have a lot of background and experience in what you are using it for, in addition to engineering and project management to keep the scope on track. While demos with agents are impressive in practice autonomy is not there they need strong guidance, so it only works as very smart assistant. What you are describing is very representative of this.
If you don't have that level of seniority then you'll struggle to get value from AI because it'll be hard to guide and keep on track, also spotting and navigating errors and wrong thinking paths. You cannot use it as an assistant, only takes what it says at face value, and given it'll randomly be wrong it makes it useless.
> If I just had google this would've been a 2 week project easily.
But you'd know something new by the end of it.
So many are so fast to skip the human experience element of life that they're turning themselves into mere prompt generators, happy to regurgitate others' knowledge without feeling or understanding.
For this, you might not care to gain meaningful experience, and as a conscious choice, that's fine. But there are an increasing number of developer and developer adjacent people who reach for the LLM first. Who don't understand "their" contributions to projects.
The haters are those of us who have to deal with this slop, and the sloppy people submitting it without thought, care or understanding.
I don't know, the kind of developers doing this are the same that would copy paste from stack overflow in the past. Because if you are interested in knowledge and human experience, LLMs or not you are curious about what you read and take ownership of what you produce. In the past these developers would have created the same slop but at a much slower pace, LLMs are just enabling them to do it faster.
It's the speed that stops you learning anything. Piecing together a dozen scripts from a dozen sources and making them work requires some work. You have to debug it. Some of this knowledge sticks.
It's not just a tech thing. Kid's learning suffering at their ability to just crank out essays they've never even read.
LLMs and AI are getting better. We doomers aren't decrying the technical advances they're making, we're appalled at the human cost of giving people a knowledge-free route through life.
Not just knowledge free, but thought free. Instead of thinking deeply about something and coming to a conclusion yourself, just offload it to an AI to do it for you. Something challenges you in life? No worries, AI is here. Not just to answer your questions, but think for you. What kind of world is that? What kind of society will that lead to?
Google having their own hardware for training and inference is newsworthy, but the link is pretty bad. Here is a much better source https://blog.google/products/google-cloud/ironwood-tpu-age-o...
I'm an idiot and I know nothing
But I wonder if there could be room for an ARM-like spec that Google could try and own and license but for AI chips. Arm is to risc-cpu as google-thing is to asic-aichip
Prolly a dumb idea, better to sell the chips or access to them?
I'm not sure the chip spec (or instruction set) is the level of abstraction here?
Something like DirectX (or OpenGL) might be the better level to target? In practice, CUDA is that level of abstraction, but it only really works for Nvidia cards.
It's not that it only works on Nvidia cards, it's only allowed to work on Nvidia cards. A non-clean room implementation of CUDA for other hardware has been done but is a violation of EULA (of the thing that was reverse engineered), copyright on the driver binary interface, and often patents. Nvidia aggressively sends cease-and-desist letters and threatens lawsuits (successfully killed ZLUDA, threatened others). It's an artificial (in a technical sense moat).
I don't think you can make the EULA bite here?
To circumvent: you have someone (who might be bound by the EULA, and is otherwise not affiliated with you) dump the data on the internet, and someone else (from your company) can find it there, without being bound by the EULA. Nvidia could only sue the first guy for violating the EULA.
However you are right, that copyright and patents still bite.
Spectral just did a thread on that.
https://x.com/SpectralCom/status/1993289178130661838
Not really, because as usual people misunderstand what CUDA is.
CUDA is hardware designed according to the C++ memory model, with first tier support for C, C++, Fortran and Python GPGPU DSLs, with several languages also having a compiler backend for PTX.
Followed by IDE integration, a graphical debugger and profiler for GPU workloads, and an ecosystem of libraries and frameworks.
Saying just use DirectX, Vulkan, OpenGL instead, misses the tree from the forest that is CUDA, and why researchers rather use CUDA, than deal with yet another shading language or C99 dialect, without anything else.
> CUDA is that level of abstraction, but it only really works for Nvidia cards.
There are people actively working on that.
https://scale-lang.com/
Not much real data or news there.
I think we need an analysis of tokens/$1 and tokens/second for Nvidia Blackwell vs Ironwood.
It depends on how they’re utilized , especially at these scales, you have to squeeze every bit out.
> It’s designed for AI with AI
CUDA engineers, your job security has never felt more certain.
These are only available in Iowa on GCP, which to me raises this question: do they have them all over the world for their own purposes, or does this limited geography also mean that users of Google AI features get varied experiences depending on their location?
Running on v6 vs v7 should just be different performance.
So we will be getting wrong answers faster now.
I'll never understand this attitude. Recently I set up a full network with 5 computers, opnsense, xcp-ng and a few things like a pi, switch, AP, etc.
I was migrating from pfsense to Opnsense so I wasn't too familiar with some of the nitty gritty. Was migrating to xcp-ng 8.3 from 8.2 which has some major CLI differences. It was a pretty big migration that took me a full weekend.
OpenAI got things wrong (mostly because it was using old documentation - opnsense had just upgraded) maybe 8 times in the whole project and was able to quickly correct itself when I elaborated on the problem.
If I just had google this would've been a 2 week project easily. I'd have to drudge through extremely dry documentation that mostly doesn't apply to anything I'm doing. Would have to read a bunch of toxic threads demeaning users who don't know everything. Instead I had chatgpt 5 do all that for me and got to the exact same result with a tenth of the effort.
The AI is useless crowd truly makes me scratch my head.
> The AI is useless crowd truly makes me scratch my head.
I think it's because, past autocomplete, for AI to be useful professionally you need to already have a lot of background and experience in what you are using it for, in addition to engineering and project management to keep the scope on track. While demos with agents are impressive in practice autonomy is not there they need strong guidance, so it only works as very smart assistant. What you are describing is very representative of this.
If you don't have that level of seniority then you'll struggle to get value from AI because it'll be hard to guide and keep on track, also spotting and navigating errors and wrong thinking paths. You cannot use it as an assistant, only takes what it says at face value, and given it'll randomly be wrong it makes it useless.
I think most people commenting on HN have the expertise, no?
I use it like a book of openings in Chess. Advanced players also learn openings.
> If I just had google this would've been a 2 week project easily.
But you'd know something new by the end of it.
So many are so fast to skip the human experience element of life that they're turning themselves into mere prompt generators, happy to regurgitate others' knowledge without feeling or understanding.
For this, you might not care to gain meaningful experience, and as a conscious choice, that's fine. But there are an increasing number of developer and developer adjacent people who reach for the LLM first. Who don't understand "their" contributions to projects.
The haters are those of us who have to deal with this slop, and the sloppy people submitting it without thought, care or understanding.
I don't know, the kind of developers doing this are the same that would copy paste from stack overflow in the past. Because if you are interested in knowledge and human experience, LLMs or not you are curious about what you read and take ownership of what you produce. In the past these developers would have created the same slop but at a much slower pace, LLMs are just enabling them to do it faster.
It's the speed that stops you learning anything. Piecing together a dozen scripts from a dozen sources and making them work requires some work. You have to debug it. Some of this knowledge sticks.
It's not just a tech thing. Kid's learning suffering at their ability to just crank out essays they've never even read.
LLMs and AI are getting better. We doomers aren't decrying the technical advances they're making, we're appalled at the human cost of giving people a knowledge-free route through life.
Not just knowledge free, but thought free. Instead of thinking deeply about something and coming to a conclusion yourself, just offload it to an AI to do it for you. Something challenges you in life? No worries, AI is here. Not just to answer your questions, but think for you. What kind of world is that? What kind of society will that lead to?
Similar things were said about the calculator.