I keep seeing these "sovereign" LMs time and time again. In Sweden we had GPT-SW3 (https://www.ai.se/en/project/gpt-sw3) and same story there. Instead of burning money on "sovereign" claims, national research labs should instead focus on building on top of solid baselines (like Qwen/Kimi) and finetuning frontier models with real agentic utility that can be applied across actual use cases and can be widely used by its people, basically for free. Nations should mirror what Cursor has done with Composer 2.5 for example.
Disagree, it’s in the country’s best interest to facilitate internal expertise on the full stack and own their “supply chain” so to speak and fight brain drain. The outcome isn’t just the model, it’s the expertise. Otherwise all their smartest folks will depart for countries where LLM development is strongest.
Got it, so every country should focus on having a mediocre-at-best AI strategy by refusing to work together? Surely this will create a better future instead of pooling resources.
This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man.
On the brain drain problem in particular, one way to ensure talent sticks around is to create a good environment for people to do their best work. In much of Europe, getting bureaucracy out of the way and encouraging real investment would go a long way. People leave because they can make more money and they want to be surrounded by the best people. People would trade some of that off to stick around their home countries, however if you go to California and talk to folks from e.g. NL or DE working on this stuff, they have a lot to say about innovation and working culture back home.
This "vaguely nationalistic world view around tech" is a direct consequence of the US government weaponizing its leading tech firms reach into the EU for nationalistic purposes.
The EU was build on the principles of collaboration, to overcome the nationalistic impulses. Free trade and free movement of people, no need for everyone to replicate what everyone else is doing. Preventing individual nation states from favoring and supporting their home grown firms over other EU firms is a central legal principle.
But this only works if it is reciprocal. And ideally if the partners are roughly on the same level. When you force developing nations in trade deals to not protect local firms, you are also preventing them from moving up the value chain and locking them into the position of "raw material providers".
When trading with China we know that China has absolutely no qualms supporting its strategic industries with a truck load of subsidies. And it is preventing foreign firms from investing and selling on the domestic market.
For the longest time the US was considered a safe partner though. Sure, plenty of disagreements in the details, but in principle someone whom you can rely on. That idea has been decisevly dismantled over the last 10ish years. The US unilaterally cut of the EMails of EU courts. The US has unilaterally decided that EU partners cannot use Fable/Mythos.
The only reasonable reaction is to make sure that the EU can maintain and create its own critical infrastructure.
>This "vaguely nationalistic world view around tech" is a direct consequence of the US government weaponizing its leading tech firms reach into the EU for nationalistic purposes.
And in response NL should weaponize ASML for example. Then both sides would naturally back down. Specialization is the most efficient way of developing tech civilization.
Whereis everybody building their own mediocre versions would be repeating Russia in its attempt to make its own national messenger - a lot of government money sunk, yet people are still using Telegram/etc.
Of NL weaponizes ASML they will just take their IP and leave NL and then they have nothing. They have already pushed a lot of things out of NL and should invest in leveraging the position without weaponizing it.
If ASML is just IP then everyone would have already copied it. They have an extremely deep and high tech supply chain that is nearly impossible to copy.
the story behind it: i was looking for a name to write music under. and i was reading academic papers about subliminal messaging in music and stuff. i came across a paper where they seemed to be absolutely ripping into this "dijksterhuis" guy's previous paper. felt bad for him. felt like he was an underdog. so i ran with that name.
Let's not overstate the case. The EU was built to keep France and Germany from getting into another war, and it has been successful at that. All the other "principles" are just window dressing.
I'm not defining anything. I'm just explaining why the key decision makers actually created the EU. The statements made for public consumption shouldn't be taken too seriously.
Simplistic reply without substance. The EU economic growth is influenced by much more than just the integration. It can be stagnant not because but despite the integration. It could also be the case that EU integration was an attempt to improve economy but it didn't work out.
This "vaguely nationalistic world view around tech" is a direct consequence of the US government ... The EU was build on the principles of collaboration ... But this only works if it is reciprocal.
This sounds great but doesn't really make any sense. What skeptics are saying is there should be a pan-EU effort to build frontier models, rather than one-off toy models built by each country as a box-ticking exercise.
"We built the EU as a powerful supranational organization, so logically the EU's response to a great power challenge in the realm that may well define the 21st century is gonna be to shatter its efforts into 27 useless pieces, because Trump bad" is just absolutely ridiculous and will not lead to anything good. Be the change you want to see, etc.
Look I agree with all you said. The EU should do this but the EU is also structurally bad at large concerted efforts exactly because it is a collaboration mechanism, not a super state.
Whatever topic comes up on this board, if the EU is mentioned people go a bit cray-cray.
If an individual country trains a language model, it's not ambitious enough. If we try to do one for the whole EU they will say it takes too long (you need to get all countries on board, you see). If the EU announces it by executive decision it's a dictatorship and government driven economic intervention, if a EU company does it they'll say it's not good enough.
You can't win with these people. In my opinion, you shouldn't even try to convince them.
Producing local researchers isn’t enough. If you want a competitive model you need to bring the best researchers from all over the world. That means lots of investment that hasn’t been common in Europe.
Sure, but there is still a more dense talent pool with SV companies, and training frontier models requires massive amounts of capital. Are Netherlands and Sweden prepared to invest 10s of billions?
My impression is the talent pool for core model development in SV isn't huge and is already grabbed by the usual suspects. Everyone else sloshing in the vague broader AIsphere are application bros whose value in the supply chain is minimal.
You missed the part where they got 13 million EUR , 3 years ago! and dont have a release yet....Definitely doing it at European pace...they will have release two when AGI is around...
Are there people legitimately brain-rotted such that they believe stuff like this?
Outside of the constant flow of "hey guys, look at the latest dumb thing Trump did" coming from entertainment outfits posing as news, what are we even talking about? Life goes on as normal for the Americans. You are way too online.
Yes, the current administration has been in power for less than two years so obviously the “day to day life” hasn’t changed much for the average American (except everything has gotten more expensive).
The US is no longer looked at as a superpower, it just surrendered to a regional power and will pay $300B USD in tribute.
But for Trump and his cronies life certainly has improved.
But the damage is done and we are witnessing the decline and destruction of the worlds greatest Empire of all time.
The us stopped being looked at as a superpower as a brain rotted Alzheimer’s patients president rose to power on the campaign “don’t let Trump win” and then did little productive except stoke wars and pardon his son and his friends from crimes.
I’m assuming you’re talking about Biden, and no, not really. Still considered a superpower.
Up until the second Trump presidency, most of us outside the USA had basically the same opinion as we had before. Pretty messed up politics , but strong economy and strong military.
The first minor dip was with Trumps first presidency. But shit went downhill fast with his second, unfettered reign when MAGA took total control over the Republican Party.
Now it is 110% clear that the American Dream is dead and buried, except for the well connected wealthy, and your military was state of the art, two decades ago.
I don't know, I don't really buy that this all went downhill with Trump. It's easy to say that about someone you don't like and fail to overlook all the signs with someone you like.
> This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man.
It’s a response to the actually-nationalist practice of the United States. I can understand why it might feel different from California, but things are a bit scary over here right now.
Collectively Europe has talent and money to support it. Whole point of sovreignity around it is to have inhouse the talent and capacity to have essential resources even if it can be have for cheap from other nations/alliances.
That is why people lament at idea of being completely dependent on Russian gas, US tech or Chinese manufacturing.
The EU should absolutely be running frontier labs producing frontier models.
The fact they aren’t points to the moribund nature of things in the EU tech space. Both China and America are doing this although with very different approaches.
wdym? it says 13.5 million euros has been allocated to the project - you aint building a frontier LLM with that kind of money, but its enough to give research opportunities and real-world experience to a handful of researchers, and build up local competence.
These people can then fund for-profit companies once they have a promising approach and bring in private investment.
As for nationalism, like it or not, this is a govt sponsored effort, and governments and universities are funded by the public of specific nations
this exactly. its not nationalistic at all to support your local development. this project has different goals than people perceive and their comments are all missing the plank ;)
Hopefully you understand that there's an obvious selection bias you encounter when talking to folks from NL or DE in California. Ofc the people that moved will think there's a good reason for them moving. But people who live here have good reasons for staying here.
Your argument would suggest the EU developing a European model would be a better direction. A heavy-weight competitor would help advance the field after all.
> getting bureaucracy out of the way and encouraging real investment
I don't think this is really about regulation - it's about network effects. The only way to compete with strong network-effects is to create your own.
> This vaguely-nationalist world view around tech
Nationalism breeds nationalism and it is the fundamental reason European states feel the need to build their own expertise. Can you imagine if your country was subject to the whims of an aspiring dictator?
I don't see it as a nationalist world view around tech in Europe. There is certainly a push for "sovereignty", but that mostly means sovereign from the US. But European countries are actively working together on this.
> Got it, so every country should focus on having a mediocre-at-best AI strategy by refusing to work together? Surely this will create a better future instead of pooling resources.
Yeah, why is Anthropic making their own mediocre models, if they could just pool their resources with OpenAI, that makes no sense.
wdym? it says 13.5 million euros has been allocated to the project - you aint building a frontier LLM with that kind of money, but its enough to give research opportunities and real-world experience to a handful of researchers, and build up local competence.
These people can then fund for-profit companies once they have a promising approach and bring in private investment
wdym? it says 13.5 million euros has been allocated to the project - you aint building a frontier LLM with that kind of money, but its enough to give research opportunities and real-world experience to a handful of researchers, and build up local competence.
Kimi and Qwen come out of China, which means that their training material may be biased e.g. relating to Taiwan [1]. In addition, there is no way to determine what input went into the training, if it was properly licensed, if it was legal (e.g. not contaminated by CSAM), or how the human component of RLHF was sourced - in US models, for example, stories about exploitation like [2] have been floating for years.
Assuming us Europeans finally get our act together, I think it is better for our long-term future (and the ethical problems) if we manage to get a baseline of training input and data ourselves, from scratch, with everything being ethically sourced.
Oh and, while we're at it, the EU has 24 official languages plus a host of minority languages. Most LLMs focus on the English, German, French and Chinese languages, but everything else is... left behind at best. An European model with actual funding and proper data sources might be able to significantly reduce that.
It really doesn't matter if the model sucks and doesn't perform well. Given the funding amount and their lofty ambitions, it seems very unlikely they will be able to pull it off properly.
Yeah China and US models have baises but so will any model. The biases do not get in the way of the product though. You don't open those models just to ask for what happened in Taianaman square or if Taiwan is a state. You dont ask ChatGPT to generate CASM. But they are very good at the tasks you actually expect from a LLM. If you fail at that, nobody will use your model no matter how "ethically sourced" a colonizer-based entity like Europe made it.
> no matter how "ethically sourced" a colonizer-based entity like Europe made it
The attempt is laughable, buy every country should at least try to keep up with frontier technology, even if they fail massively or are massively underfunded.
On the other hand, it's arguably wasteful for an incompetent govt to do something like this, since the money will almost certainly not be well spent. It will just go to people good with MS Word. That's the likely failure mode for such NL innovation projects. The actual solution is a culture shift, but that is much harder if not impossible to pull off and requires decades. But we (NL people and govt) should work towards that. Most likely all these govt led innovation attempts are a sad waste of tax money.
The culture shift that has generated this is the same one that causes the other story on HN this morning about xAIs gas generators being a national security issue. Ie one towards corruption graft and the public ill.
I don’t want Europe to model itself on the US, whatever the economic gain. Hopefully we are large enough to find a third way between China and the US.
You're right, it's not good to idolize this success because it does come with strings attached. You can't have everything. It makes sense to define European success in its own frame of reference.
We do still have to be strategic at the world stage though, or we'll be humiliated and pay dearly.
The Chinese models are almost certainly taught to comply with "Chinese values" in the RLHF step, not from filtering the training data. There may be a few things which are too radioactive to be allowed even in the training material - but that's more likely to be things like child abuse images for a visual model, things non-Chinese values also have an issue with.
I'm pretty sure no county taking a stab at making their own model for sovereignty purposes will let "proper licensing" stand in their way.
> Most LLMs focus on the English, German, French and Chinese languages, but everything else is... left behind at best.
that is not true, so please read before make an opinion. The French Mistral project shipped seven+ years ago with 140 languages for example.. language translation was the first LLM task from 2015
One example is not the same as "most LLMs". My experience is the same with most LLMs. Especially the smaller ones are English oriented (probably makes sense given the size constraints).
> Most LLMs focus on the English, German, French and Chinese languages, but everything else is... left behind at best
Current frontier models (closed and open) are already really good at small languages too. I use them in Finnish sometimes, and the language is immaculate. They underestand even somewhat obscure dialects. Multilinguality seems to be a mostly solved problem.
Specifically, we discard documents shorter than 200 characters (Xue
et al., 2021a), and any page containing the phrase “lorem ipsum,” the word “javascript,” or curly brackets (Raffel et al., 2023)....
It is quite surprising/funny to see all documents with javascript removed.
Uh, some would say it's easy to determine what input went into the training for kimi and qwen.. since they were caught stealing it from American labs. Some cultural cliches may never change.
> since they were caught stealing it from American labs. Some cultural cliches may never change.
Has a formal lawsuit been brought to bear? Given, Anthropic & OpenAI are being dragged through courts for copyright violation (or stealing, as you'd call it, if the companies involved were culturally Chinese) by newspapers, publishing houses etc; one'd think they'd pass on some of that medicine to Alibaba, which does have business entities registered in the US.
It's well-known that all commercial models are based on stolen content. That doesn't mean there is no filtering/censoring, just that the censoring likely depends on where it's happening…
Let’s just gloss over the monstrous amount of copyrighted and pirated material the American labs trained on. China bad. American good. Some cultural cliches never change.
.... Anthropic began buying books in bulk, tearing off the bindings and scanning each page before feeding the digitized versions into its AI model, according to court documents.
Wow. This image of Anthropic employees ripping books apart to use them to train models is a powerful one, seems like an inflection point in the history of information.
If open frontier models start closing up and states start more export controls on AI services and hardware, it might be good to ensure the supply chain is there to reproduce the SotA, or even a couple generations behind it.
There is something to be said for this "most cheapest" approach, there is also something to be said for making models that are entirely ethically sourced:
1. Free of controversy like unlicensed training materials
2. Free of exploitative rlfh loops by people in low-wages countries
3. The leasons learned (and published) from going through the entire training process on "European" hardware: "AI factories" (the term for Slurm HPC/HTC systems with lots of heavy GPU nodes, heavily subsidized by our government [0])
1 and 2 are strong counter-LLM arguments at the moment, and hold back some groups of potential users. Another is energy/water use, so going for maximum green energy would be a nice boon as well. 3 is something I consider to be highly useful for our European identity and "way of the ninja" (for you Naruto fans out there).
one would also hope there'll be less pressure to "make line go up", i.e. not having to do attention-engineering via deliberate sycophancy to trap individuals into using it more and more and more and more and more and more.
but in general, yes, as someone who is vehemently anti-ai GPT-NL has piqued my interest specifically because of the ethical protections / measures they're talking about. question is whether they stick to it.
Sounds like calling those model "open source" did its wicked job. You can not take an open weight model and build a next-generation model using that as a foundation. Once those companies decide it's no longer in their interest to release new open weights everything you've created this way becomes a pile of rapidly deprecating legacy.
OP probably means using the existing open weights as a base for further homegrown development and research, not that the homegrown models are always updated based on whatever US or China are doing in the moment
Do we know for sure how much national corpus of knowledge (like dutch) goes into these "global" models and how that affects "localized" model biases? What's wrong with specialized models?
TNO doesn't serve nor represent global interest and hence does not care about global progress. It exists to enhance knowledge on things within Dutch society primarily with some ripple effect outward to EU because they have interests within the EU.
Its purpose is not to become some kind of OpenAI or global foundation offering services/tools on that scale.
There is a lot of critisism on this project, not invalid, but mostly based in lack of understanding of what the goals are of the organization as well as the people building the thing.
The people building it, are well aware of how it will be less capable than other LLMs on a general reasoning aspect, not only due to having actually purchased _all_ licensed data that has been used as inputs. Not being a multi billion dollar corporation, this means having very little data and should be an obvious signal to observers that it has not the goal to outdo other models.
In my opinion (personal) its a project that has a learning and demonstration value that is not 'look how well our model performs against others', but still offers value.
You keep building on the last available version? Fine tuning is a whole lot cheaper, easier and more useful than pretraining a model from scratch. It's a complete no brainer.
> But good to have researchers and professors "in-house".
I'm not in this field, but I think we already have them. Probably the main difference is that we have most or all of them in academia and next to none insode private companies. But we do have them, and they could start working for private companies if the market moves in that direction in the EU as well
Unfortunately in this game first principles requires massive resources, not "some". Building in-house on top of existing open weights is a good way to bootstrap this process, especially since there's nothing inherently magical or particularly expertise-heavy when it comes to weights themselves
You take current version and build on top of it. You have the weights.
You might not get some n+1 version at some point but the n version you will have will be still most likely much better than whatever you come up with burning good will money of people believing in „sovereignty”.
You are not getting ahead in this game by being „true to your local values” capital expenditure is insane in this game.
It seems like you don't understand. For fine tuning, it's cheaper to fine tune an existing model. For massive changes, it's better to retrain from scratch. Otherwise, model will UNLEARN a lot first, and then you will train about twice longer to the same result.
Ok this „you don’t understand” was uncalled for, I am sorry.
I am speaking from very practical point of view. English and whatever frontier models are trained on is lingua Franca of software/tech/science currently - you don’t want to make massive changes because just exactly like you wrote model will unlearn a lot.
Current models translate easily between the languages.
So from my point of view even if we as a smaller country will have N-2 model that we slightly fine tune or just give it a harness with national RAG it will be better than wasting money on training a model from scratch only on „texts in country own language „ because that is loosing proposition. It’s usefulness is going to be really limited compared to model trained on body of knowledge in English.
It is a lot like company CEO thinking they have to „train AI model” for their company on their company materials - well no, you just make RAG and eventually fine tune some models, give access to dat, give access to MCP.
Because even if you are F1000 company you don’t have resources to train your own generic model, as a nation there is no that have resources to train „national model” on par with frontier labs.
Today, you keep seeing "sovereign" LMs that are subject to the sovereignty of some human-led state. Tomorrow, the "sovereign" LMs will be called that for a completely different reason.
States are the things which can ignore that, and I'm pretty sure US and China already do. No state is going to respect copyright if they think its future is at stake, and apparently even Netherlands thinks the future is at stake.
(Of course states can ignore copyright in a legally polite manner, such as asserting that training on all published material in the National Library is fair game)
Same reasons why every country, or close allies, build their own tanks or space program. You want to keep some level of capability within your control. Compared to weapon programs, AI research is very cheap.
It is crazy that anything Europe gets so much hate. IMO it is important to build models within the boundaries of smaller nations, using their own language. Research has to continue even if it is outside of US and China.
I was somewhat excited about these "sovereign" open models in the beginning, but it became soon apparent that they're not gonna be anything but toys compared to SOTA.
The problem is that there are a lot, at least 30, of these small projects scattered around, funded for a few years as some ad-hoc temporary coalition of universities and businesses. Those simply cannot compete with businesses spending tens of billions on developing these. Especially when you have to bring a spoon to a gunfight restricting to "clean" data.
Multilinguality is essentially a solved problem, and restricting too much on one language with more limited resources is gonna make the model worse in that language too.
Anyone can move the needle. Saying that languages are solved is not accurate as well. You could raise different questions like maybe model grounded in a different language will make it more efficient in some tasks, maybe language structure matters for a multidimentional space, maybe that matters for the distillation, etc. It is all about the ideas and their exchange, not about the investment rounds and MAU.
Even if multilinguality isn't solved, building a benchmark and then testing each model on it and posting the result may be a cheaper accelerator of competence in the language.
I'm from Spain and I also hate these projects with passion. Creating models that speak multiple languages is a solved problem. Having each European Nation train its own useless "sovereign model" in its own language is a total waste of time and resources when we could pool resources and give it a try to training SOTA models that speak in all European languages.
I'd rather have smaller european labs try to give it a go at distributed training. If multiple countries got together and said, "look, we tried training a distributed model that speaks in all of our local languages and that is comparable to 1-year-old Chinese open-source models", that, at least, I would find interesting.
Excuse my ignorance if by "distributed training" you mean a specific process, but couldn't this be considered a step toward distributed training? If nations train models independently and then later distill them into a single model, all the work (both the compute and the research processes) are distributed for the initial training phase.
I mean it as in, train a model across different clusters instead of a centralized cluster. It's been shown that it's possible to train 10B models this way. If more research effort was put into this, that would be great
I don't think your approach would work because you can't create a strong model from distilling several weak models.
Maybe I'm more attuned to this type of thing having grown up as a national of a smaller state living in the shadow of a bigger state but you constantly see actors from the bigger state belittling and condescending anything contributed socially or economically from the smaller state.
And I see this sort of dynamic here in this forum where Americans very frequently talk condescendingly like this about Europe generally and European tech especially (they did it to China too but China smartly ignored this self-interested nonsense and carried on anyway which is what Europeans should do).
It really grates on me and presumably many others. But it serves an agenda too of a lot of the founders and financiers that hang out here that have big fat customers in Europe they'd like to keep sweet and competitors they'd like to keep down.
It’s not that it gets hate so much as it’s akin to watching them make announcements that they’re going to make a European google/facebook/tiktok.
Sure… they can, except at the end of the day it’s a bit late, regulatory burden will make it comparatively useless, and because of that nobody will ever use it. It will be spending a bunch of taxpayer dollars for press releases.
The running joke is that when these “sovereign” EU models launch, they’re going to refuse to answer anything that might involve personal information such as Elon Musk’s birthday.
At least with social networks the network effect is a powerful force. Foregrounding regulatory burden in that context is nonsensical. (That does not apply in the same way to models.)
That’s on Wikipedia, it’s not PII, it’s also not going to be relevant to any meaningful IRL work.
I challenge the assumption you can do meaningful work in this field without blatant disregard for intellectual property.
The idea that it’s all down to training size is clearly incorrect, as every expert human learned their craft without nearly the sum total information of the internet. Clearly there are architectural wins to be found.
Besides that, why would everyone just be fine with Opus level AI at best, as that’s all the US is willing to export, and I doubt China will share beyond that.
Sovereign AI is more important than ever after Friday.
I guess if you are strict about it, making derogatory comments like yours is indeed not hate. But I'm sure you are aware that "getting hate" frequently used in a more extensive meaning online, especially in the context of replies to a post and I don't see the much point in insisting on the stricter definition here.
I kinda agree, the best use of taxpayer money should be in reducing taxes to corporation that would like to compete in the market vs US and China, rather than making governments playing the game (since they very obviously can’t).
If a teenager on your street said he was going to spend $1,000 to customize his Honda Civic for his needs, you'd believe him. If he says he's going to build a brand new car, better than a Honda civic, for $10,000, you'd laugh and say good luck.
If Europe is serious about getting home grown AI fast, three simple steps:
1. Huge tax incentives, let the companies get grossly wealthy while paying minimal taxes. Minimum 10 years with clauses protecting "retribution" taxes there after.
2. Tax incentives for the founders/shareholders, just like above.
3. Drop worker protections to a minimum, make it easy to fire people. You only want serious/dedicated employees anyway.
Within 2-3 years there will be at least a trillion dollars looking to get in.
Don't worry though if reading that made you mad. Its absolutely not going to happen. I can think of few things more antithetical to the European ethos than smart skilled people working 80-100hrs weeks with almost no vacation to gas their founders net worth by tens, hundreds, of billions.
No, we don't need any slavery. The employees at these AI companies will almost certainly out earn most if not all other local white collar workers, while also getting top tier benefits. They can also quit at any time if they don't like it or don't think it is fair.
This "greed is good, and should be rewarded" philosophy is one I see all too often on HN, and the entire reason why Americas political, regulatory, and business leadership have been overrun by the countries most criminally corrupt narcissists and psychopaths; why its democracy has collapsed.
When you reward the most selfish, corrupt, and antisocial behaviours with wealth and power, you're guaranteed to create a selfish, corrupt, and antisocial society. IMHO it's indicative of what I have dubbed Americas "mental illness epidemic"; specifically cluster B personality disorders [0] which are characterised by socially-destructive and self-destructive behaviours.
If that's the world you want for you and your loved ones, congratulations. You've earned it!
If tax incentives to attract companies and at-will employment are already viewed as a destructive collapse of democracy, it's no wonder we are not getting anything done here in Europe.
I suggest you take the time to read independent analysis on the benefits of corporate subsidies in America.
I know I've read of many cases where they cost cities and communities far more than the economic activity they promise to generate (eg Amazon warehouses and sportsball stadiums).
Subsidies could make strategic sense in the case of AI, but it's terrible economics to encourage businesses that take more from communities than they give back, especially those that don't employ many people.
Why should taxpayers anywhere pay private enterprises to create infrastructure they own and control for-profit, rather than just cutting out the monopoly middle man and establishing that infrastructure as a public utility who services the domestic economy, equally?
This is the kind of madness I'm talking about. If taxpayers have to subsidise products and services the "free" market refuses to provide, they should have direct ownership in the profits; not just the losses. Why is the default for society to subsidise expenses and privatise profits? That is ideologically-driven dogmatic insanity, which allows select individuals and foreign entities control over the national economy and its security, and is the entire reason madmen in the USA have the EU (and world) by the balls.
No greed will be rewarded. If European consumers don't like the end AI models that are produced, they can forgo buying them and the investors can watch their fortress burn. For the EU taxpayer, nothing was gained and nothing was lost. All the state did was stand out of the way.
If the models are good though, they will have their sovereign AI and should be happy to pay for it instead of American or Chinese models. You may call it greed, but to me it just sounds fair.
> For the EU taxpayer, nothing was gained and nothing was lost.
Except for the "massive" subsidies you advocated for, right? Except for the massive cost to provide utilities and infrastructure to support those businesses? Except for the cost to support the employees they refuse to provide minimum "protections" for? Except for any damage they do to the local environment?
You seem to think that businesses exist in a vaccum where they have no impact on society or its population, except when they succeed and everyone should be eternally grateful. This is the systemic greed and narcissism America exports to the world; not just through its businesses, but its media and population across forums such as this one, and is exactly what I'm talking about.
There would be no subsidies and the companies would be responsible for funding their own infrastructure. Protections for workers would be minimal, because we wouldn't really be interested in hiring people who want strong protections. They can stay at their current job. You balk at this, but all of Europe's capable tech minds left the EU to go work at tech in the US over the last 30 years. They didn't care about the generous protections, they cared about making stuff and making money. Which they did, overwhelmingly.
What I am describing is an existential battle for the relevance of Europe on the world stage in the future. Right now Europe is looking like a vasal state, picking between the US and China for who it's overlord will be. If Europe wants independence, it needs to mobilize immediately to catch up.
Exactly this. You can't have a competitive industry while at the same time heavily redistributing wealth to the point where people don't have any incentives at all.
They can be serious about home-grown AI without needing to become a libertarian capitalist hellscape. I prefer happiness, safety and privacy over competing with the US / China.
> GPT‑NL is developed within the Netherlands and Europe. This gives us full control over the model, the data and the choices we make. We avoid dependency on non‑European providers and invest in a sustainable AI ecosystem aligned with our laws, values and societal goals.
I love it! So this is our answer to America and China denying foreigners access to their frontier models.. a massive 13,5M€ founding to develop souvereign european ai, trained exclusively on legally obtained documents and highest moral standards as defined in EU AI Act.
The US can only block exports. They cannot force exports. The NL-US relationship is quite toxic, for example the "Dutch America Friendship Treaty". Everything is very one-sided.
Any move like this would be simply by murdering the golden goose. ASMLs stuff is good no doubt but you don’t want to give the world the incentive to develop an alternative. If it was done once it can be done again and once it has been done again say good bye to all those returns on monopoly.
If ASML’s output were absolutely necessary to the US or the rest of the world, and if no replacement could be made, the Netherlands would have infinite leverage and be the most powerful country in the world. That they aren’t tells you all you need to know about exactly how far they could take a strategy like what you’re describing. Yes, everyone needs ASML. But ASML also needs everyone else.
An elite toolmaker is nothing without the factories that use the tools.
The odds of the Dutch attempting to encumber the use of ASML machines by western bloc allies is similar to the odds of the sun going out suddenly. It ain’t gonna happen.
So good to see these developments. Every country should do this. I'd even say every person should gave their own personalized AI running on their own computers. If only the costs involved were not so astronomical.
The government would be far better off figuring out how to take commodity models and applying them to government functions where they can, with deterministic scaffolding and guardrails, to make government more efficient, optionally using RL on traces from their use to improve their performance.
Imagine taking models and fine-tuning them / doing RL rollouts to help automate permit application approvals, as applied specifically to Dutch permit processes. That would be a real help to Dutch businesses!
That type of applied AI is more interesting and effective now than just trying to make another foundational model that isn't going to work well or do anything of economic value.
Another thing they could do is try to attack the the suppply chain issues. Try to form an alliance to block RAM deals or something, or to get fabs on EU soil, making HBM for the people. We have some bargaining chips, especially when banding together with a few large EU states. Not as EU, just a few specic countries. No bureaucracy, just elite trade diplimacy. Probably best done in secret so the big labs don't catch wind of this. Any NL/DE/UK/FR/CH/PL/IT govt people reading this?
"Champions of a European AI model should ask themselves if a European effort would be more effective than Meta, which this year will spend more on chips ($125 billion) than Germany spends on defense ($114 billion) and offer salaries of over $100 million to attract the best researchers, and is still failing to catch up. Elon Musk tried and failed to build a good AI model."
I think at this point what the Netherlands, and any other country that wants a good model in their language should do, is gather up every piece of text ever written in that language and license it to the big AI labs/companies for training. I'm sure there are vast libraries of books and other text that haven't been digitized and aren't a priority for the big labs.
Oh I didn’t mean at all charging them. I mean licensing in the sense of granting rights for the purpose of training. Probably most labs would be fine adding the language to the training for free as long as the dataset quality is high and it improves the results. But yes, pay them if that’s what it takes for them to use it.
Interesting that this got posted now: the project is receiving increasingly more skepticism lately in the Dutch tech scene [0], and I think that’s fully justified.
What is the exact skepticism? The only thing I could get from that was from some "tech entrepreneur":
> GPT-NL was never built to compete with Claude or ChatGPT. It was trained exclusively on licensed data, and is intended more for governments and companies where privacy and compliance matter more than raw performance.”
That's it? That it didn't aim to compete with SOTA models? Maybe this is something you have to start with something, then ramp up, rather do what only a select few labs been able to do, start with really big models. Especially if you're resource constrained, which since this is a government project, I really hope for the sake of the tax payers it was.
TNO is something like semi-DARPA. It gets a lot of stuff tax free and a lot of gov funding, but a lot of their budget is from getting businesses to hire their R&D teams.
They do really good R&D on a lot of stuff. This is just their attempt at public credibility/internal skill building to enter the LLM business.
Doubt its going to be successful, but they "waste" a lot more money on other things that you never heard of. Its not fraud, its just R&D dressed up a little too much too early.
Common approach I've seen is having workflows with paid/larger/hosted models for some workflow where you don't quite know exactly how it'll be when you first put it together, then with time you've locked down how things more or less work yet you still need free-form text parsing of some kind, so you end up replacing the bigger models with carefully post-trained small models.
Besides that, there is a ton of use cases for smaller models for a bunch of different things. We'll be unlikely to be able to run LLMs (actually Large) on smartphones for a while, while the smaller LLMs seem to run already on-device in experiments.
They are starting to deploy to customers now, not sure if that counts as "complete" or not. The big innovation here is that they are doing it all legally.
I don't understand countries (especially governments) wanting to have their own models when there are already pretty solid open source (weights) models out there.
Countries should want control over _where_ the compute is happening rather than _what code_ is running.
What's wrong with a country hosting a Kimi, Qwen or GPT-Oss on their hardware for their government work purpose?
Yes and also, US and Chinese models are censored in different ways. US models are way too prudish for personal use in Europe because they're afraid to piss off religious investors. Chinese models are too censored on history and current affairs, eg the tiananmen massacre never happened stuff like that.
Chinese models aren't censored as much as you think, you can download the model and run it somewhere else and they will happily tell you about Tiananmen Square. Or heck, ask DeepSeek via Openrouter, it will do the same.
The censorship works kind of like with Fabel, it kicks in before the model responds.
There's an absolutely massive cultural and behavioural bias in those models. Models will suggest things like "go to the hospital" for things that require GP appointments, "just drive three hours" while it's faster to go places by train, and so on. They will do it in anglicised Dutch (compound words split, English-like grammar structures) that's perfectly understandable, but the cultural bias is there if you know to look for it.
Furthermore, the expertise in designing and training these models is valuable as well. The existing models are good as a starting point in terms of learning from previous mistakes, but we should not just let a handful of American and Chinese people keep the knowledge and expertise.
One problem with this particular project, though, is that copyright has been enforced for Dutch LLM training before, and the AI industry cannot exist without massive scale piracy, the likes of which has never been seen before. A lot of Dutch training material exists in pirated books that AI companies in countries that do not care about copyright have access to, but are exempted from the training set here. The impact of enforcing copyright on an AI model will be quite interesting to see.
Idk which models you refer to, but I tested a bunch recently, and they performed well on Dutch. Only the smallest, such as qwen 3.6 27B, made up words and switched languages.
There's a large gap between making up words and an actually native text distribution. LLMs have a clear pattern, clear tells, a "feel" in English, and it's normally even more pronounced in non-English languages.
Lots of bias towards English sentence structure, idioms, etiquette, etc.
I didn't notice any of that. Such a bias would be strange, because certainly smaller models don't have the luxury of learning grammar independently: it's still word sequences, and languages are quite well separated.
There would be a bunch of value in having, say, a good 30B-class model that used my local language as well as it does English. There's lots of cases, especially in the government sphere, where local processing is a requirement and frontier-level capabilities aren't required. Making those cheap to run seems like a fine goal.
Yes, but what's the point of a support bot that writes good Dutch when it can't follow instructions, doesn't understand the questions or can't solve problems? I might be wrong, but I don't think atm these models have the cognitive ability to perform any task in a satisfactory manner.
As for accessing pii, I imagine the value here is in the fact they're local, which has nothing to do with the "sovereignty" of these models. If anything, a model is more likely to be tricked by a malicious prompt the farther it is from the sota.
A good harness and engineering is important no matter which model you use.
But Sovereignty of hosting is also important because without it all pii is being leaked.
I don't understand this. Even if that were true (and it isn't in my experience), a model that is trained on a Dutch corpus and arguably "knows Dutch well" but has the reasoning and comprehension abilities of a three year old is useless in any case. I'd rather use a model that can only speak English and put an automatic translator around it.
To be fair. There is a security concern angle: even open-source models could be trained as sleeper agents that act adversarially (for example, adding backdoors) when used in specific national companies in specific settings. This is very difficult to detect or void, so if you want to be sure 100% that this isn't the case, you have to train your own model from scratch.
This gets better short-term results for a fraction of the cost, for sure, but what do you when China places an export control banning the release of open weight models? If you don't have your own talent, you're then relegated to using a base model from 2026 or whatever the cutoff date is, forever. That defeats the purpose of a 'sovereign' model made for and by your people.
Funny, that's what I thought when PewDiePie set up his monster AI rig and what he called a 'council'. Quote:
"PewDiePie has built a custom web UI for self-hosting AI models called "ChatOS" that runs on his custom PC with 2x RTX 4000 Ada cards, along with 8x modded RTX 4090s with 48 GB of VRAM. Running open-source models from Baidu and OpenAI, PewDiePie made a "council" of bots that voted on the best responses, and then built "The Swarm" for data collection that will become the foundation of his own model coming next month."
Calling a bunch of LLMs a "council" is just rebranding well known ensemble methods with shrill marketing hype, nothing original or out of the ordinary. Mixture of experts and every other idea in his stack has a literature older than PewDiePie's career.
Yet another attention craving influencer who shilled crypto scams during the crypto bubble and is now marketing "AI councils" during the AI bubble.
He's not a serious or honest person, AI is just what he pivoted to after crypto. That's not innovation; it's attaching trendy branding to ideas that were already old when Marvin Minsky wrote The Society of Mind in 1986, three years before PewDiePie was zero years old in 1989.
The only thing PewDiePie's brought to the table is cleverly optimized YouTube thumbnails designed to attract clicks. The architecture is decades old; only his marketing and shilling is state of the art.
Does what PewDiePie calls a council not resemble a Poldermodel? That’s all I associated with it. Please don’t lecture me about what to watch and drag up negative associations that have nothing to do with the subject (multiple LLMs voting for the best answer) for a simple observation.
Zonnetje in huis.
edit: also, 7 years on and after closure of the platform I cannot find evidence of rug pulls or exit scams associated with Dlive or any of their underlying crypto funding mechanisms, so before judging others on the content they consume maybe don’t take random YouTubers at their word and then don’t look into it 7 years after the fact.
I thought finetuning data can't contradict foundation models, and anything that are inconsistent with the standard LLM American-Chinese split personality would be rejected?
But you have to have more data than used for pretraining for the added knowledge to take precedent over pretraining, no? If that would be the case, you practically contradict the knowledge in the base model.
I mean ... LLMs are sort of an extreme and living proof of linguistic determinism. Their behaviors are dictated almost entirely by disorganized language data, primarily English and Chinese. So you can't just add a language as native primary language in a quick post training, I think. There's no way that it would work.
Really? Because I'm pretty sure that at least every two days there's a active post with a top voted comment along the lines of "The EU isn't doing AI themselves, they are so hosed".
Please don't move the goalposts. What computer parts does ASML or NXP make?
ASML only makes the lithography machines, 85% of which go outside the EU (let that sink in). And then fabs in Taiwan, Korea or the US use those ASML machines to etch US IP for computer chips. EU doesn't make any computer parts domestically.
And NXP mostly makes various microcontrollers and small chips, not high margin IP decenter centric parts like ASICS, FPGAs, CPUs or GPUs.
So not only are you the ignorant one here, but you also have the audacity to insult others with so much confidence.
@dwa3592 below. Firstly, why are you moving the goalposts in bad faith again just to stir an argument? What does that have to do with my original comment?
And secondly, there's other lithography machines out there, not just ASML.
And thirdly, the IP Nvidia, AMD, etc develop to etch on silicone via ASML machines makes them more valuable than ASML.
Fourthly, repeating my "let that sink in" phrase is just childish and low-IQ trolling, unworthy of this platform.
Europe is currently hosed because we made the mistake of trying to develop economies complementary to the US and china.
That was a big strategic mistake. In the US case it was borne of the mistaken belief that we shared values and were partners.
But don’t mistake the situation for lack of innovation of capability. Europe is currently adapting, but I think the success of Ukraine is one reason to be optimistic that current adversity might actually leave us better off in the long run.
Corrupt countries with broken legal systems tend not to fare that well in the longer run.
>Europe is currently hosed because we made the mistake of trying to develop economies complementary to the US and china.
NO, it's hosed because it's not competitive and slept on the wheel at several key digital economical revolutions plus sleeping at the wheel at preventing obvious geopolitical issues (gas dependence to Russia, losing auto industry to China, losing semiconductor industry, losing SW industry, etc).
You can't be an economic leader if you keep losing on all fronts and only be a leader at how much welfare your spending.
> the mistaken belief we shared values and were partners.
We do share. US and a lot of latin america is mostly European immigrants and European culture, making our cultures are much more similar than the african and middle eastern ones the EU has been importing and adopting. Where we differ is that US still has free speech and isn't devolving into a stasi police state that arrests people for Tweets that the political establishment find uncomfortable.
> might actually leave us better off in the long run.
How? EU's economy has been pretty much stagnant since 2019 when you account for inflation loss.
> But don’t mistake the situation for lack of innovation of capability. Europe is currently adapting,
How? Where is Europe's Nvidia and AMD? Where is Europe's TSMC? ASML can't feed an entire continent.
The EU economy has absorbed covid and the Russian war in that time. It’s pretty unsurprising that things are flat.
The main thing Europe lacks is economies of scale in investment in tech, and a common language. Big data tech wasn’t a priority until we realised it was going to be weaponised against us by an ally. I think we’ll do fine. Most of AWS and Azure is expensive superstitious cover your ass bullshit anyway. It could all be regulated away with proper data residency laws, which I hope are coming soon.
Woah! only lithography machines???? it is literally impossible to make any device capable of running anything close to AI without ASML. Let that sink in.
Funnily ASML owes its current success partly to US funded research (straight from Wikipedia):
> Two years later, it joined a consortium, which included Intel and two other U.S. chipmakers, in order to exploit fundamental research conducted by the US Department of Energy. Because the Cooperative Research and Development Agreement (CRADA) it operates under is funded by the US government, licensing must be approved by Congress.[12]
Why are you acting childish and petty? I said EU hasn't got AI compute manufacturing(aka no equivalent IP to Nvidia and AMD and no equivalent to TSMC or Samsung fabs), not that it doesn't have lithography machines manufacturing.
Surely you understand that while you can have the latter, you can also lack the former.
ASM (International) makes machines that add material to a silicon wafer (deposition).
ASML makes machines that remove material from said wafers (lithography, etching)
(I was a bit surprised that's not combined in 1 machine. But let's move on)
Then Besi makes machines to stack / interconnect / package those ICs into a package. I'm assuming pick & place machines are other companies' turf.
The above are all Dutch companies, operating a pretty important section of the tech stack.
Iirc there were (& probably still are) some IC fabs in Europe, but mostly older nodes (like useful for microcontrollers used by car manufacturers. Wikipedia has a list). So for SOTA smartphone SoCs it's off to Taiwan (TSMC), South Korea (Samsung) or China (who makes everything, including smartphones & the chips going in there).
So as far as EU goes, the capabilities are mostly there. Skilled workforce? Check. Money? This is a rich continent.
What's missing is the guts to say "hey, let's dump €100B into this & make ourselves some laptop & server CPUs!".
But now the important thing: several of such initiatives are starting to bear fruit, and b) confidence that EU can do such things, is growing.
As for bureaucracy / red tape... sigh... (won't be fixed any time soon)
Yes, all true, and all things I didn't disagree with because I wasn't talking about that.
The point I was talking about you only addressed to some extent in a line below that:
>What's missing is the guts to say "hey, let's dump €100B into this & make ourselves some laptop & server CPUs!".
Yeah exactly, the EU doesn't have computer manufacturing capabilities (just like I said 5 layers up) and it never will because it doesn't invest and also doesn't attract investors to invest.
>So as far as EU goes, the capabilities are mostly there. Skilled workforce? Check.
No they're not. We don't have the skilled workers for that. Nobody in EU knows how to design Nvidia and AMD level GPUs and Altera and Xilinx levels of FPGAs that power AI datacenters. Nobody in EU knows how to make competitive 2nm fabs, otherwise EU fabs would have already bought ASML EUV machines and updated their ancient processes to the highly more profitable nodes instead of being stuck making cheap legacy nodes for cars and white goods. Those old nodes are still important to have to an extent, but ask yourself, would you rather sell a die for 10k a pop or sell 1000 dies for 10 cents a pop? Would you rather make more money or less money?
> Money? This is a rich continent.
Money is meaningless if you're not using it right. China is pushing to beat the EU and they have less money than the EU.
Most important machine ... built on US IP, subject to US export restrictions, used to manufacture high value US IP, in factories outside the EU, so profits of those chips goes to US. A point I have addressed over two times already.
Also ASML even threatened to leave the NL if the Dutch government doesn't do what they want on taxes and labor policies. So having only a single card to play that EU can loose at any time, it's not putting EU tech sovereignty argument in a good light.
The "wahabout ASML" that keeps being spammed by people here, isn't proof of EU compute and AI sovereignty. It's the exception which is why it's the only thing people can mention on EU tech and they DDoS you with it as if that changes anything.
Are people here that petty that they can't stay on topic and argue in good faith and instead need to hijack your argument to go on offtopic whataboutism for a cheap gotcha spamming "whatabout ASML" on unrelated arguments?
EU bureaucrats are too busy trying to keep the welfare/pension system from collapsing, defeating Russia, supporting Ukraine, managing the fossil fuels energy shortages, figuring out how to nerf Chinese EVs while supporting domestic car companies, and restricting social media free speech to make sure the "far right" don't win elections.
So of course, semiconductor manufacturing sovereignty is very low on their priority list.
Different scale for those problems. Way different scale. US is monetary rich, oil rich, energy rich, manufacturing rich, and doesn't suffer from russian aggression at its borders. US is so bored from how rich and problem free it is compared to Europe, that it can afford to keep starting foreign wars as if nothing ever happens.
Also back on the topic, the US managed to bring TSMC to open a cutting edge fab in the US and has already been operational for a while. Which already puts it way ahead of the EU on this front as well.
The thing is, US is much better on actually making things happen when push comes to shove. It saw it's deficient and vulnerable on domestic semiconductor manufacturing, it then made it happen with TSMC. It's doing the same with domestic ship building with Korean partners.
US might be slow moving, but somehow EU is even way slower at realizing and addressing its vulnerabilities, only waking up when it's far too late, causing it to pay a much more painful price for sleeping at the wheel (Russian invaded Ukraine in 2014 BTW, not in 2022, and they were building another gas pipeline with them), and when this type of own-goaling keeps repeating enough times you see the correlation with EU's decline as their economic rivals keep biting more and more market share from their industries as they sleep on critical changes and developments.
They’re building a competitive-quality model, from scratch, with fair compensation to content owners, for €13.5 million? Something’s wrong with this picture.
I feel that not only is Europe losing its independence to the US and China, but it does not even try to take part in the race.
Unlike the US, Europe has no California-level VCs. I don't expect hundreds of billions of Euros to be poured into long-shot projects.
Unlike China, Europe has neither cohesive public investment at the global level nor the drive to grow. Long-term investments have a lot of words, a lot of regulations, a lot of proxy goals, but there is neither a lot of money nor urgency. It was captured by this post: https://x.com/piotrsankowski/status/2065795919623438546
So yeah, both in economy and warfare, Europe dooms itself to be in the hands of the US, China, or a mix of both.
> Unlike the US, Europe has no California-level VCs.
Some would consider that a good thing. There is a lot to be said for VC in recent years not being beneficial for the economy, certainly on an individual level, other than "number go up".
And if you ask a bit more, you'll find that those same people are very likely to daily-drive AI models, desktop and phone operating systems and various other software critical to their professional and personal lives from US companies. And buy tons of Chinese products over Chinese or US e-commerce platforms.
What people say matters much less than what they do.
Americans are also far less into AI than a startup forum like HN will make you believe.
In real life, there are a few AI maniacs that make their entire identity about how they use AI, but it's hardly the sensation that the internet will make you believe. I don't believe there is any difference between the USA and Europe, although the lack of employee protection does mean that it's a lot easier for Americans to lose their jobs when their managers get lured in by AI companies.
After all, the entire AI bubble is all about VCs and startups hyping each other up until profits magically appear.
> There is a lot to be said for VC in recent years not being beneficial for the economy
What a wild statement, VC's are behind most of the growth in the US economy, and they directly drive up wages in tech. I'd be fascinated to hear a valid complaint of VC's that isn't just money envy
Some people consider it a good thing that communists boiled people's hands as torture. Some people consider it a good think that Iran massacred 10,000s of its own citizens. Some would consider it a good thing that Israel killed all Palestinians in Gaza.
Is that what I said in your mind? I honestly had to chuckle at the extreme hyperboles you decided to use. Surely having a less favorable view of hyper capitalist economics is on the same level as torture and genocide. Yup, that is a totally reasonable and measured response that just exemplifies acting in good faith.
Anyway, thanks for genuine chuckle your response got out of me.
You don't even say it is a good thing. The closest to morality come is alluding to the fact that someone holds the opinion that it's good. Even though you also likely hold this opinion, you don't even want to go as far as saying that you think it's a good thing.
Europe decided to regulate the hell out of foreign AI instead of investing in their own systems. It's sad to see the European continent lost the race to create a decent startup ecosystem (no decent search engines, social networks, cloud, mobile OS) and now it seems to be hellbent in losing this battle.
>It's sad to see the European continent lost the race to create a decent startup ecosystem
What's ironic and sad at the same time is that pre-2022 Russia's Yandex(domestic Russian variant of Google) was lightyears ahead of what EU, a significantly richer and more capable block, had. IIRC, their reverse image search was so good, they had to nerf it because people were using it to find the identity of people from photos.
Same for Israel, their tech sector is probably greater than the EU one combined
Absolutely shameful how the EU kept managing to snatch defeat from the jaws of victory over and over.
Not surprising. All the Yandex people that moved over to here to the Netherlands that I know of are astounded about the insane difference in the tax burden between what they had in Russia and what they have in Western Europe (when their 5 year tax discount ends, that is). If the government takes the bulk of your income after a certain point, there isn't really that big of a push to create ground-breaking technology.
> If the government takes the bulk of your income after a certain point, there isn't really that big of a push to create ground-breaking technology.
I'm skeptical that high taxes is a large reason to lose to California of all places. Maybe in some important sense CA has "earned" that via talent and funding density while NL hasn't (from the perspective of a company, to be clear)
NL is a corporate tax haven in Europe(and also compared to California), that's why almost all US and foreign companies have their corporate HQs there, Dublin or Luxembourg.
Companies go there because taxes are low for them, not necessarily for their employees(ignoring the NL 5 year tax break for foreigners). It's kinda like the US Delaware of Europe.
I think much of that is because European customers (both private and business) tended to prefer American suppliers over suppliers from European countries that were not their own. That may have something to do with most people in IT being quite fluent in English, while European products were all-to-often half-heartedly translated from German/French/Spanish/Polish/Italian/Ukranian.
In many cases, well-established and well-liked European services have been supplanted by American counterparts that came later and were not really better in any way. They did usually have much more money to burn though, undercutting pricing until competition was dead.
I'm speaking in the past tense, because now for the first time in the couple of decades I can remember, there seems to be a somewhat commonly held preference for European suppliers.
Frankly, being Russian myself, I'm also very disappointed by the state of tech in Europe.
But you know what hurts the most? That I know it wasn't always that way.
I'm sitting right now in the same country that invented the Minitel, built out the TGV network and the Grands Projets, and don't even get me started about the weird and wonderful machines they've got in that museum in Mulhouse, hell, you could go back in time to Gustave Eiffel. Industry and ambition used to be here. It was almost physically painful to discover that it seems to be gone now.
The EU chips act was already largely a failure and they threw plenty of money at it. The problem with the Europe is they look to their bureaucrats to stimulate things then expect different results from the last 50 times.
Europe needs its own private industry that attracts talent and capital. It doesn’t need another EU press release talking about hopes and dreams
>The EU chips act was already largely a failure and they threw plenty of money at it.
Yep, the proof is in the pudding. They managed to grab defeat from the jaws of victory again when US got TSMC to open a fab there.
For example, both German and Austrian semiconductor companies are choosing to expand manufacturing in Malaysia and not domestically.
I don't know what more proof people need that the EU is cooked on manufacturing side.
>The problem with the Europe is they look to their bureaucrats to stimulate things then expect different results from the last 50 times.
The definition of insanity. This is a mentality problem and is deeply ingrained into the EU population. European people are always looking at the state to solve any problems including those of industry and free market, without realizing the state mostly isn't good at those things and they'd actually be better off with less government there. Whenever the state intervenes into the economy it just further supports the massive bloated legacy companies who lobby the politicians never the startup ecosystem. And then people mistake again by voting for more government hoping to fix that which just results in more expensive bloat and more red tape nor a competitive freer market with more capital and innovation.
>Europe needs its own private industry that attracts talent and capital. It doesn’t need another EU press release talking about hopes and dreams
You are saying that as if China or the US are completely isolated from the EU.
We live in a globalized world whether you like it or not, and every supply chain spans multiple countries.
Arguably, staying out of the AI "race" is a good thing
> Unlike the US, Europe has no California-level VCs. I don't expect hundreds of billions of Euros to be poured into long-shot projects.
My ex-neighbor (when I was a teenager, living in Belgium) and very good friend really wanted to make it big. He became a chip engineer, moved to California, raised money for a first startup (it tanked) then raised money for a second startup. He made the world a better place (he created some very specific micro-inverters for solar panels) and made a $$$ exit.
The EU saw exactly zero of the wealth he created and he's never ever coming back to what he considers a failure of a continent.
That's the problem: many of the great minds with the mindset required to do great things already left the EU.
> So yeah, both in economy and warfare, Europe dooms itself to be in the hands of the US, China, or a mix of both.
And in energy (economy is energy and energy is economy, and China really understood that) the EU doomed itself to be in the hands of Russia.
I'll play devil's advocate a little bit - I'm not sure it is losing its "independence" by not taking part in the race. It could very well be that it is gaining independence from tech and choosing a "second mover advantage" to decide how it gets deployed after seeing how it impacts everyone else. Let the US and China experiment on the bleeding edge (and their citizens feel the effect, both good and bad), and then be picky about how you use it.
I don't know if it is the right strategy but there's certainly a legitimate strategy in there.
The problem is recursive self improvement creating a very difficult gap and the fact that power, compute has a lag from when you invest and when data centers come up.
You also can't just spin up a research team out of nowhere.
1. The labs in the US and China don't seem to have any problem selling (or even giving) access to these models right now.
2. If some kind of take-off happens which makes that not true, my bet is all bets are off on what that outcome even looks like. What would the economic paradigm even be under a superintelligent AGI? Do you think "it" is going to listen when Trump says "you can't work with Europe"?
There's a whole bunch of grey in between the two, for example only having access to second rate models, but I'm not sure that particularly matters if the strategy is "second mover."
Sure, but my guess is for "true" super intelligence we won't be able to predict whether that is true or not until it happens. I'm not a doomer, but I also don't really think we can "align" people, much less a "super intelligent" AI.
I feel like building datacenters and filling them with chips may be more valuable than creating sovereign models. xAI I think makes more money renting datacenters to Anthropic than with the models they trained, and they could pivot thanks to their datacenters. By making regulations easier than in the US, this could bring some computing power to Europe, which then can be used to train sovereign models, or rented to big AI labs.
Also, when training models, you create talent that then could go to other countries (brain drain). Restricting that brain drain without imposing authoritarian restrictions on the movements of people seems hard, so it seems hard to keep talent as a competitive advantage. If instead the competitive advantage is datacenters with chips, power capacity building, fast path to building datacenters, I think they are easier to retain while preserving the rights of everyone involved.
This is not an open source model. In that sense I think the sovereign claim is a bit strange. It's the data providers that determine access to the model.
I really think countries should build a sovereign _ecosystem_ and sovereign models are an excuse to achieve it.
An ecosystem is the tribal knowledge, revolving door of talent, known processes etc.
If the end goal is to make a half assed Dutch speaking model, I think it won’t cut it. I don’t see anyone using it over Gemma 4b that runs on my laptop.
An ecosystem is more durable and has desirable second order effects.
We already had GEITje but it was banned by the courts. Of course it can still be found because the entire internet is not subject to Dutch law. But it did manage to stop development :'(
I wonder with these stories. Why are there so many individual country efforts? We know the scale needed with scaling laws / capital / energy. Most of these countries alone can barely compete (even large groups of them would struggle.
Why don't they work together on it? Companies like Airbus have already been able to do that with aircraft.
I'm making a Dutch dictionary and would be interested to see how this model would fair in evals vs non specialized ones. I've tested a variety of models for https://hetnederlands.com content and differences can be big
What really matters is the sovereign capability to finetune the LLM models. Any model could be vetted and tested, but you need finetuning/lora training to prevent the model from being outdated.
What are they going to train with 13.5M really? We're a tiny company in Amsterdam in Holland and we've got "only 64x B300 to train on" so we could never make an LLM I thought, since we've got only 4M in compute.
And they're going to train an LLM with all kinds of extra difficulties compared to OpenAI for just 13.5M?
This is too little, too late. Europe really need to start focussing.
All these tiny niche models are perhaps fun as an academic exercise or great for the researchers resume but I highly doubt that they'll add any value or will be used for anything serious.
Even if this becomes a somewhat decent model with a fantastic understanding of "gezellig", "kring verjaardag" or "pannenkoeken", how many people will interact with it before the limits of it will drive them back to a frontier model?
Even if the purpose of this is government & other regulated industries, do we really want our government to use a poor model? Either do it right or don't do it at all.
Prices for training have dropped immensely in terms of research required, code efficiency, algorithmic/sample efficiency, and possibly also hardware (I'm not qualified to say without looking it FLOPS/dollar, or even to be certain that's the right metric here).
overall, the revenue sharing model is (IMO) more interesting than the fact that it is dutch. Usage of data, and sharing it with the providers of this data, is an inherent part of the creation of these models that is not discussed as much as it should be
Honestly, I used to think the 'sovereign model' was a waste of money. But recently, with the US logic of restricting model exports, I've come to think that if things go south, they could even cut off allied nations. So now the sovereign model seems reasonable to me. That, in turn, means US influence is deteriorating. And that probably isn't such great news for American businesses.
This really isn’t what HN is for. I think you should read the guidelines. I’ve looked at your posting history and it is all like this: relentless negativity with low information content.
I didn't down vote you, but you aren't really adding anything to the conversation. This type of pithy comment might be fun on Reddit, but at HN, we try to provide more constructive, and information rich, comments.
I keep seeing these "sovereign" LMs time and time again. In Sweden we had GPT-SW3 (https://www.ai.se/en/project/gpt-sw3) and same story there. Instead of burning money on "sovereign" claims, national research labs should instead focus on building on top of solid baselines (like Qwen/Kimi) and finetuning frontier models with real agentic utility that can be applied across actual use cases and can be widely used by its people, basically for free. Nations should mirror what Cursor has done with Composer 2.5 for example.
Disagree, it’s in the country’s best interest to facilitate internal expertise on the full stack and own their “supply chain” so to speak and fight brain drain. The outcome isn’t just the model, it’s the expertise. Otherwise all their smartest folks will depart for countries where LLM development is strongest.
Got it, so every country should focus on having a mediocre-at-best AI strategy by refusing to work together? Surely this will create a better future instead of pooling resources.
This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man.
On the brain drain problem in particular, one way to ensure talent sticks around is to create a good environment for people to do their best work. In much of Europe, getting bureaucracy out of the way and encouraging real investment would go a long way. People leave because they can make more money and they want to be surrounded by the best people. People would trade some of that off to stick around their home countries, however if you go to California and talk to folks from e.g. NL or DE working on this stuff, they have a lot to say about innovation and working culture back home.
This "vaguely nationalistic world view around tech" is a direct consequence of the US government weaponizing its leading tech firms reach into the EU for nationalistic purposes.
The EU was build on the principles of collaboration, to overcome the nationalistic impulses. Free trade and free movement of people, no need for everyone to replicate what everyone else is doing. Preventing individual nation states from favoring and supporting their home grown firms over other EU firms is a central legal principle.
But this only works if it is reciprocal. And ideally if the partners are roughly on the same level. When you force developing nations in trade deals to not protect local firms, you are also preventing them from moving up the value chain and locking them into the position of "raw material providers".
When trading with China we know that China has absolutely no qualms supporting its strategic industries with a truck load of subsidies. And it is preventing foreign firms from investing and selling on the domestic market.
For the longest time the US was considered a safe partner though. Sure, plenty of disagreements in the details, but in principle someone whom you can rely on. That idea has been decisevly dismantled over the last 10ish years. The US unilaterally cut of the EMails of EU courts. The US has unilaterally decided that EU partners cannot use Fable/Mythos.
The only reasonable reaction is to make sure that the EU can maintain and create its own critical infrastructure.
>This "vaguely nationalistic world view around tech" is a direct consequence of the US government weaponizing its leading tech firms reach into the EU for nationalistic purposes.
And in response NL should weaponize ASML for example. Then both sides would naturally back down. Specialization is the most efficient way of developing tech civilization.
Whereis everybody building their own mediocre versions would be repeating Russia in its attempt to make its own national messenger - a lot of government money sunk, yet people are still using Telegram/etc.
Of NL weaponizes ASML they will just take their IP and leave NL and then they have nothing. They have already pushed a lot of things out of NL and should invest in leveraging the position without weaponizing it.
If ASML is just IP then everyone would have already copied it. They have an extremely deep and high tech supply chain that is nearly impossible to copy.
ASML wiki for those like me who were confused https://en.wikipedia.org/wiki/ASML
(despite the name, i am not dutch)
I am quite interested to hear how you would pronounce your name, dike-ster-house (assuming you live in the US)?
> dike-ster-house
yep, basically. 3rd one down https://www.howtopronounce.com/dijksterhuis
the story behind it: i was looking for a name to write music under. and i was reading academic papers about subliminal messaging in music and stuff. i came across a paper where they seemed to be absolutely ripping into this "dijksterhuis" guy's previous paper. felt bad for him. felt like he was an underdog. so i ran with that name.
> (despite the name, i am not dutch)
Some close or far ancestor was though. :'-)
Let's not overstate the case. The EU was built to keep France and Germany from getting into another war, and it has been successful at that. All the other "principles" are just window dressing.
Isn't it up to Europeans to define the purpose of the EU? What's your claim to having the right to define it?
I'm not defining anything. I'm just explaining why the key decision makers actually created the EU. The statements made for public consumption shouldn't be taken too seriously.
You're vastly underestimating the economic importance of EU integration.
Is that why the EU economic growth rate is so stagnant?
Simplistic reply without substance. The EU economic growth is influenced by much more than just the integration. It can be stagnant not because but despite the integration. It could also be the case that EU integration was an attempt to improve economy but it didn't work out.
So no evidence of economic importance.
This "vaguely nationalistic world view around tech" is a direct consequence of the US government ... The EU was build on the principles of collaboration ... But this only works if it is reciprocal.
This sounds great but doesn't really make any sense. What skeptics are saying is there should be a pan-EU effort to build frontier models, rather than one-off toy models built by each country as a box-ticking exercise.
"We built the EU as a powerful supranational organization, so logically the EU's response to a great power challenge in the realm that may well define the 21st century is gonna be to shatter its efforts into 27 useless pieces, because Trump bad" is just absolutely ridiculous and will not lead to anything good. Be the change you want to see, etc.
Look I agree with all you said. The EU should do this but the EU is also structurally bad at large concerted efforts exactly because it is a collaboration mechanism, not a super state.
Whatever topic comes up on this board, if the EU is mentioned people go a bit cray-cray.
If an individual country trains a language model, it's not ambitious enough. If we try to do one for the whole EU they will say it takes too long (you need to get all countries on board, you see). If the EU announces it by executive decision it's a dictatorship and government driven economic intervention, if a EU company does it they'll say it's not good enough.
You can't win with these people. In my opinion, you shouldn't even try to convince them.
[dead]
Why would it have to be mediocre?
Both Netherlands and Sweden produce highly competent researchers. Per capita on par with any other location on the planet, including California.
This will be good for Europe as a whole and also the planet.
Producing local researchers isn’t enough. If you want a competitive model you need to bring the best researchers from all over the world. That means lots of investment that hasn’t been common in Europe.
But it's a start
Sure, but there is still a more dense talent pool with SV companies, and training frontier models requires massive amounts of capital. Are Netherlands and Sweden prepared to invest 10s of billions?
My impression is the talent pool for core model development in SV isn't huge and is already grabbed by the usual suspects. Everyone else sloshing in the vague broader AIsphere are application bros whose value in the supply chain is minimal.
Honestly I have a strong suspicion that “the talent pool” will shift towards Europe and China and I believe it’s already happening.
And yes, I do believe Europe will invest in this technology.
You missed the part where they got 13 million EUR , 3 years ago! and dont have a release yet....Definitely doing it at European pace...they will have release two when AGI is around...
>and also the planet.
What a glorious day for Canada and, therefore, the world.
I don’t think it’s good that the USA has so much influence due to their dominance in information technology.
It was only nominally democratic, now it’s a total shit show.
>total shit show
Are there people legitimately brain-rotted such that they believe stuff like this?
Outside of the constant flow of "hey guys, look at the latest dumb thing Trump did" coming from entertainment outfits posing as news, what are we even talking about? Life goes on as normal for the Americans. You are way too online.
Yes, the current administration has been in power for less than two years so obviously the “day to day life” hasn’t changed much for the average American (except everything has gotten more expensive).
The US is no longer looked at as a superpower, it just surrendered to a regional power and will pay $300B USD in tribute.
But for Trump and his cronies life certainly has improved.
But the damage is done and we are witnessing the decline and destruction of the worlds greatest Empire of all time.
Nitpick: it’s 6 years.
The Republican Party looked very different during Trumps first term
The us stopped being looked at as a superpower as a brain rotted Alzheimer’s patients president rose to power on the campaign “don’t let Trump win” and then did little productive except stoke wars and pardon his son and his friends from crimes.
I’m assuming you’re talking about Biden, and no, not really. Still considered a superpower.
Up until the second Trump presidency, most of us outside the USA had basically the same opinion as we had before. Pretty messed up politics , but strong economy and strong military.
The first minor dip was with Trumps first presidency. But shit went downhill fast with his second, unfettered reign when MAGA took total control over the Republican Party.
Now it is 110% clear that the American Dream is dead and buried, except for the well connected wealthy, and your military was state of the art, two decades ago.
The 21st century belongs to China and Europe.
I don't know, I don't really buy that this all went downhill with Trump. It's easy to say that about someone you don't like and fail to overlook all the signs with someone you like.
I didn’t have a problem with Trump during his first term.
It was clear he was unfit to be a president, but he was mostly golfing.
[dead]
> This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man.
It’s a response to the actually-nationalist practice of the United States. I can understand why it might feel different from California, but things are a bit scary over here right now.
> This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man.
It's a direct response to the MAGA / America First attitude of the American electorate.
Collectively Europe has talent and money to support it. Whole point of sovreignity around it is to have inhouse the talent and capacity to have essential resources even if it can be have for cheap from other nations/alliances.
That is why people lament at idea of being completely dependent on Russian gas, US tech or Chinese manufacturing.
The EU should absolutely be running frontier labs producing frontier models.
The fact they aren’t points to the moribund nature of things in the EU tech space. Both China and America are doing this although with very different approaches.
wdym? it says 13.5 million euros has been allocated to the project - you aint building a frontier LLM with that kind of money, but its enough to give research opportunities and real-world experience to a handful of researchers, and build up local competence.
These people can then fund for-profit companies once they have a promising approach and bring in private investment.
As for nationalism, like it or not, this is a govt sponsored effort, and governments and universities are funded by the public of specific nations
this exactly. its not nationalistic at all to support your local development. this project has different goals than people perceive and their comments are all missing the plank ;)
[dead]
> This vaguely-nationalist world view around tech that’s emerging in Europe is dangerous, man.
Guess which country blocked access to a SOTA model based on national security bullshit.
Okay fair point
Hopefully you understand that there's an obvious selection bias you encounter when talking to folks from NL or DE in California. Ofc the people that moved will think there's a good reason for them moving. But people who live here have good reasons for staying here.
> pooling resources
Your argument would suggest the EU developing a European model would be a better direction. A heavy-weight competitor would help advance the field after all.
> getting bureaucracy out of the way and encouraging real investment
I don't think this is really about regulation - it's about network effects. The only way to compete with strong network-effects is to create your own.
> This vaguely-nationalist world view around tech
Nationalism breeds nationalism and it is the fundamental reason European states feel the need to build their own expertise. Can you imagine if your country was subject to the whims of an aspiring dictator?
I don't see it as a nationalist world view around tech in Europe. There is certainly a push for "sovereignty", but that mostly means sovereign from the US. But European countries are actively working together on this.
> Got it, so every country should focus on having a mediocre-at-best AI strategy by refusing to work together? Surely this will create a better future instead of pooling resources.
Yeah, why is Anthropic making their own mediocre models, if they could just pool their resources with OpenAI, that makes no sense.
Users don't need to build wheels. Wheel builders need to first learn to build some mediocre wheels, then they can build better ones.
wdym? it says 13.5 million euros has been allocated to the project - you aint building a frontier LLM with that kind of money, but its enough to give research opportunities and real-world experience to a handful of researchers, and build up local competence.
These people can then fund for-profit companies once they have a promising approach and bring in private investment
wdym? it says 13.5 million euros has been allocated to the project - you aint building a frontier LLM with that kind of money, but its enough to give research opportunities and real-world experience to a handful of researchers, and build up local competence.
if something’s strategically important for safety it’s wise to have competency in it.
Tech sovereignty is a response to American imperialism. It has not gone unnoticed that the White House has summoned the tech bros and weaponised tech.
Surely this just uses state funds to train a pipeline of people to be immediately poached by frontier labs?
Great, now they will get expertise and then get hired by openai and move
Kimi and Qwen come out of China, which means that their training material may be biased e.g. relating to Taiwan [1]. In addition, there is no way to determine what input went into the training, if it was properly licensed, if it was legal (e.g. not contaminated by CSAM), or how the human component of RLHF was sourced - in US models, for example, stories about exploitation like [2] have been floating for years.
Assuming us Europeans finally get our act together, I think it is better for our long-term future (and the ethical problems) if we manage to get a baseline of training input and data ourselves, from scratch, with everything being ethically sourced.
Oh and, while we're at it, the EU has 24 official languages plus a host of minority languages. Most LLMs focus on the English, German, French and Chinese languages, but everything else is... left behind at best. An European model with actual funding and proper data sources might be able to significantly reduce that.
[1] https://www.taiwannews.com.tw/news/6245677
[2] https://www.theguardian.com/technology/2024/apr/16/techscape...
It really doesn't matter if the model sucks and doesn't perform well. Given the funding amount and their lofty ambitions, it seems very unlikely they will be able to pull it off properly.
Yeah China and US models have baises but so will any model. The biases do not get in the way of the product though. You don't open those models just to ask for what happened in Taianaman square or if Taiwan is a state. You dont ask ChatGPT to generate CASM. But they are very good at the tasks you actually expect from a LLM. If you fail at that, nobody will use your model no matter how "ethically sourced" a colonizer-based entity like Europe made it.
> no matter how "ethically sourced" a colonizer-based entity like Europe made it
The attempt is laughable, buy every country should at least try to keep up with frontier technology, even if they fail massively or are massively underfunded.
On the other hand, it's arguably wasteful for an incompetent govt to do something like this, since the money will almost certainly not be well spent. It will just go to people good with MS Word. That's the likely failure mode for such NL innovation projects. The actual solution is a culture shift, but that is much harder if not impossible to pull off and requires decades. But we (NL people and govt) should work towards that. Most likely all these govt led innovation attempts are a sad waste of tax money.
The culture shift that has generated this is the same one that causes the other story on HN this morning about xAIs gas generators being a national security issue. Ie one towards corruption graft and the public ill.
I don’t want Europe to model itself on the US, whatever the economic gain. Hopefully we are large enough to find a third way between China and the US.
You're right, it's not good to idolize this success because it does come with strings attached. You can't have everything. It makes sense to define European success in its own frame of reference.
We do still have to be strategic at the world stage though, or we'll be humiliated and pay dearly.
The Chinese models are almost certainly taught to comply with "Chinese values" in the RLHF step, not from filtering the training data. There may be a few things which are too radioactive to be allowed even in the training material - but that's more likely to be things like child abuse images for a visual model, things non-Chinese values also have an issue with.
I'm pretty sure no county taking a stab at making their own model for sovereignty purposes will let "proper licensing" stand in their way.
> Most LLMs focus on the English, German, French and Chinese languages, but everything else is... left behind at best.
that is not true, so please read before make an opinion. The French Mistral project shipped seven+ years ago with 140 languages for example.. language translation was the first LLM task from 2015
One example is not the same as "most LLMs". My experience is the same with most LLMs. Especially the smaller ones are English oriented (probably makes sense given the size constraints).
> Most LLMs focus on the English, German, French and Chinese languages, but everything else is... left behind at best
Current frontier models (closed and open) are already really good at small languages too. I use them in Finnish sometimes, and the language is immaculate. They underestand even somewhat obscure dialects. Multilinguality seems to be a mostly solved problem.
This already exists https://eurollm.io/
How do people not know about it and keep making stuff from scratch?
I did not know about EuroLLM. I had a look to the paper (https://arxiv.org/abs/2602.05879) describing it:
Specifically, we discard documents shorter than 200 characters (Xue et al., 2021a), and any page containing the phrase “lorem ipsum,” the word “javascript,” or curly brackets (Raffel et al., 2023)....
It is quite surprising/funny to see all documents with javascript removed.
There is something north of 8% OCR error rates.. that will hurt model quality!
Uh, some would say it's easy to determine what input went into the training for kimi and qwen.. since they were caught stealing it from American labs. Some cultural cliches may never change.
> since they were caught stealing it from American labs. Some cultural cliches may never change.
Has a formal lawsuit been brought to bear? Given, Anthropic & OpenAI are being dragged through courts for copyright violation (or stealing, as you'd call it, if the companies involved were culturally Chinese) by newspapers, publishing houses etc; one'd think they'd pass on some of that medicine to Alibaba, which does have business entities registered in the US.
It's well-known that all commercial models are based on stolen content. That doesn't mean there is no filtering/censoring, just that the censoring likely depends on where it's happening…
> It's well-known that all commercial models are based on stolen content.
Does that mean that Chinese models are the "Robin Hood"s of the AI era?
>> Some cultural cliches may never change.
Let’s just gloss over the monstrous amount of copyrighted and pirated material the American labs trained on. China bad. American good. Some cultural cliches never change.
How about, both China and the US bad, Europe at least somewhat decent because we lack the financial incentives to behave like utter arseholes?
> since they were caught stealing it from American labs
...and "good guys" the American labs were caught stealing from authors all over the world[1].
[1]: www.npr.org/2025/09/05/g-s1-87367/anthropic-authors-settlement-pirated-chatbot-training-material
.... Anthropic began buying books in bulk, tearing off the bindings and scanning each page before feeding the digitized versions into its AI model, according to court documents.
Wow. This image of Anthropic employees ripping books apart to use them to train models is a powerful one, seems like an inflection point in the history of information.
If open frontier models start closing up and states start more export controls on AI services and hardware, it might be good to ensure the supply chain is there to reproduce the SotA, or even a couple generations behind it.
There is something to be said for this "most cheapest" approach, there is also something to be said for making models that are entirely ethically sourced:
1. Free of controversy like unlicensed training materials
2. Free of exploitative rlfh loops by people in low-wages countries
3. The leasons learned (and published) from going through the entire training process on "European" hardware: "AI factories" (the term for Slurm HPC/HTC systems with lots of heavy GPU nodes, heavily subsidized by our government [0])
1 and 2 are strong counter-LLM arguments at the moment, and hold back some groups of potential users. Another is energy/water use, so going for maximum green energy would be a nice boon as well. 3 is something I consider to be highly useful for our European identity and "way of the ninja" (for you Naruto fans out there).
[0 https://hpc-portal.eu/funding-opportunities]
one would also hope there'll be less pressure to "make line go up", i.e. not having to do attention-engineering via deliberate sycophancy to trap individuals into using it more and more and more and more and more and more.
but in general, yes, as someone who is vehemently anti-ai GPT-NL has piqued my interest specifically because of the ethical protections / measures they're talking about. question is whether they stick to it.
Sounds like calling those model "open source" did its wicked job. You can not take an open weight model and build a next-generation model using that as a foundation. Once those companies decide it's no longer in their interest to release new open weights everything you've created this way becomes a pile of rapidly deprecating legacy.
OP probably means using the existing open weights as a base for further homegrown development and research, not that the homegrown models are always updated based on whatever US or China are doing in the moment
Do we know for sure how much national corpus of knowledge (like dutch) goes into these "global" models and how that affects "localized" model biases? What's wrong with specialized models?
TNO doesn't serve nor represent global interest and hence does not care about global progress. It exists to enhance knowledge on things within Dutch society primarily with some ripple effect outward to EU because they have interests within the EU.
Its purpose is not to become some kind of OpenAI or global foundation offering services/tools on that scale.
There is a lot of critisism on this project, not invalid, but mostly based in lack of understanding of what the goals are of the organization as well as the people building the thing.
The people building it, are well aware of how it will be less capable than other LLMs on a general reasoning aspect, not only due to having actually purchased _all_ licensed data that has been used as inputs. Not being a multi billion dollar corporation, this means having very little data and should be an obvious signal to observers that it has not the goal to outdo other models.
In my opinion (personal) its a project that has a learning and demonstration value that is not 'look how well our model performs against others', but still offers value.
And what happens once the "solid baselines" become unavailable for a reason or the other?
You keep building on the last available version? Fine tuning is a whole lot cheaper, easier and more useful than pretraining a model from scratch. It's a complete no brainer.
> You keep building on the last available version?
yes but a sovereign can allocate some resources and a few people to stay in the loop from a first principles level. No need to wait for a rug pull.
Of course, it can not compete with the frontier labs. But good to have researchers and professors "in-house". LLMs are here for the long-term.
> But good to have researchers and professors "in-house".
I'm not in this field, but I think we already have them. Probably the main difference is that we have most or all of them in academia and next to none insode private companies. But we do have them, and they could start working for private companies if the market moves in that direction in the EU as well
Unfortunately in this game first principles requires massive resources, not "some". Building in-house on top of existing open weights is a good way to bootstrap this process, especially since there's nothing inherently magical or particularly expertise-heavy when it comes to weights themselves
Seems like you don’t understand.
You take current version and build on top of it. You have the weights.
You might not get some n+1 version at some point but the n version you will have will be still most likely much better than whatever you come up with burning good will money of people believing in „sovereignty”.
You are not getting ahead in this game by being „true to your local values” capital expenditure is insane in this game.
It seems like you don't understand. For fine tuning, it's cheaper to fine tune an existing model. For massive changes, it's better to retrain from scratch. Otherwise, model will UNLEARN a lot first, and then you will train about twice longer to the same result.
https://en.wikipedia.org/wiki/Catastrophic_interference
Ok this „you don’t understand” was uncalled for, I am sorry.
I am speaking from very practical point of view. English and whatever frontier models are trained on is lingua Franca of software/tech/science currently - you don’t want to make massive changes because just exactly like you wrote model will unlearn a lot.
Current models translate easily between the languages.
So from my point of view even if we as a smaller country will have N-2 model that we slightly fine tune or just give it a harness with national RAG it will be better than wasting money on training a model from scratch only on „texts in country own language „ because that is loosing proposition. It’s usefulness is going to be really limited compared to model trained on body of knowledge in English.
It is a lot like company CEO thinking they have to „train AI model” for their company on their company materials - well no, you just make RAG and eventually fine tune some models, give access to dat, give access to MCP.
Because even if you are F1000 company you don’t have resources to train your own generic model, as a nation there is no that have resources to train „national model” on par with frontier labs.
Today, you keep seeing "sovereign" LMs that are subject to the sovereignty of some human-led state. Tomorrow, the "sovereign" LMs will be called that for a completely different reason.
Their legality is very questionable given all the likely copyright infringement going on, and a state can't really ignore that.
States are the things which can ignore that, and I'm pretty sure US and China already do. No state is going to respect copyright if they think its future is at stake, and apparently even Netherlands thinks the future is at stake.
(Of course states can ignore copyright in a legally polite manner, such as asserting that training on all published material in the National Library is fair game)
Same reasons why every country, or close allies, build their own tanks or space program. You want to keep some level of capability within your control. Compared to weapon programs, AI research is very cheap.
It is crazy that anything Europe gets so much hate. IMO it is important to build models within the boundaries of smaller nations, using their own language. Research has to continue even if it is outside of US and China.
I was somewhat excited about these "sovereign" open models in the beginning, but it became soon apparent that they're not gonna be anything but toys compared to SOTA.
The problem is that there are a lot, at least 30, of these small projects scattered around, funded for a few years as some ad-hoc temporary coalition of universities and businesses. Those simply cannot compete with businesses spending tens of billions on developing these. Especially when you have to bring a spoon to a gunfight restricting to "clean" data.
Multilinguality is essentially a solved problem, and restricting too much on one language with more limited resources is gonna make the model worse in that language too.
Anyone can move the needle. Saying that languages are solved is not accurate as well. You could raise different questions like maybe model grounded in a different language will make it more efficient in some tasks, maybe language structure matters for a multidimentional space, maybe that matters for the distillation, etc. It is all about the ideas and their exchange, not about the investment rounds and MAU.
Even if multilinguality isn't solved, building a benchmark and then testing each model on it and posting the result may be a cheaper accelerator of competence in the language.
This platform is running from the US and frequently accessed by US people. What else would you expect?
I'm from Spain and I also hate these projects with passion. Creating models that speak multiple languages is a solved problem. Having each European Nation train its own useless "sovereign model" in its own language is a total waste of time and resources when we could pool resources and give it a try to training SOTA models that speak in all European languages.
I'd rather have smaller european labs try to give it a go at distributed training. If multiple countries got together and said, "look, we tried training a distributed model that speaks in all of our local languages and that is comparable to 1-year-old Chinese open-source models", that, at least, I would find interesting.
Excuse my ignorance if by "distributed training" you mean a specific process, but couldn't this be considered a step toward distributed training? If nations train models independently and then later distill them into a single model, all the work (both the compute and the research processes) are distributed for the initial training phase.
I mean it as in, train a model across different clusters instead of a centralized cluster. It's been shown that it's possible to train 10B models this way. If more research effort was put into this, that would be great
I don't think your approach would work because you can't create a strong model from distilling several weak models.
https://www.primeintellect.ai/blog/intellect-1
https://www.primeintellect.ai/blog/intellect-2-release
Humanity
self-interested hate is it not?
Maybe I'm more attuned to this type of thing having grown up as a national of a smaller state living in the shadow of a bigger state but you constantly see actors from the bigger state belittling and condescending anything contributed socially or economically from the smaller state.
And I see this sort of dynamic here in this forum where Americans very frequently talk condescendingly like this about Europe generally and European tech especially (they did it to China too but China smartly ignored this self-interested nonsense and carried on anyway which is what Europeans should do).
It really grates on me and presumably many others. But it serves an agenda too of a lot of the founders and financiers that hang out here that have big fat customers in Europe they'd like to keep sweet and competitors they'd like to keep down.
It’s not that it gets hate so much as it’s akin to watching them make announcements that they’re going to make a European google/facebook/tiktok.
Sure… they can, except at the end of the day it’s a bit late, regulatory burden will make it comparatively useless, and because of that nobody will ever use it. It will be spending a bunch of taxpayer dollars for press releases.
The running joke is that when these “sovereign” EU models launch, they’re going to refuse to answer anything that might involve personal information such as Elon Musk’s birthday.
At least with social networks the network effect is a powerful force. Foregrounding regulatory burden in that context is nonsensical. (That does not apply in the same way to models.)
That’s on Wikipedia, it’s not PII, it’s also not going to be relevant to any meaningful IRL work.
I challenge the assumption you can do meaningful work in this field without blatant disregard for intellectual property.
The idea that it’s all down to training size is clearly incorrect, as every expert human learned their craft without nearly the sum total information of the internet. Clearly there are architectural wins to be found.
Besides that, why would everyone just be fine with Opus level AI at best, as that’s all the US is willing to export, and I doubt China will share beyond that.
Sovereign AI is more important than ever after Friday.
I guess if you are strict about it, making derogatory comments like yours is indeed not hate. But I'm sure you are aware that "getting hate" frequently used in a more extensive meaning online, especially in the context of replies to a post and I don't see the much point in insisting on the stricter definition here.
I kinda agree, the best use of taxpayer money should be in reducing taxes to corporation that would like to compete in the market vs US and China, rather than making governments playing the game (since they very obviously can’t).
If a teenager on your street said he was going to spend $1,000 to customize his Honda Civic for his needs, you'd believe him. If he says he's going to build a brand new car, better than a Honda civic, for $10,000, you'd laugh and say good luck.
But if a teenager says he’s going to spend 50k going to university to study engineering you might support then.
I agree there is likely some hubris in this sort of announcement, but investing in European expertise and industrial base in this area is important.
If Europe is serious about getting home grown AI fast, three simple steps:
1. Huge tax incentives, let the companies get grossly wealthy while paying minimal taxes. Minimum 10 years with clauses protecting "retribution" taxes there after.
2. Tax incentives for the founders/shareholders, just like above.
3. Drop worker protections to a minimum, make it easy to fire people. You only want serious/dedicated employees anyway.
Within 2-3 years there will be at least a trillion dollars looking to get in.
Don't worry though if reading that made you mad. Its absolutely not going to happen. I can think of few things more antithetical to the European ethos than smart skilled people working 80-100hrs weeks with almost no vacation to gas their founders net worth by tens, hundreds, of billions.
Some additional points to consider:
1. Pay the workers in company scrip and relocate the workers to a company town. That way, all workers are fully dedicated to the company.
2. Start importing slaves from Africa again. It worked to build up massive wealth. Should do the trick for AI as well.
3. Abolish the 8 hour work day. No comment needed.
With these 3 simple tricks, you too can get 6-7 bazillion euro AI mammoths.
No, we don't need any slavery. The employees at these AI companies will almost certainly out earn most if not all other local white collar workers, while also getting top tier benefits. They can also quit at any time if they don't like it or don't think it is fair.
We'll, someone has to do RLHF.
This "greed is good, and should be rewarded" philosophy is one I see all too often on HN, and the entire reason why Americas political, regulatory, and business leadership have been overrun by the countries most criminally corrupt narcissists and psychopaths; why its democracy has collapsed.
When you reward the most selfish, corrupt, and antisocial behaviours with wealth and power, you're guaranteed to create a selfish, corrupt, and antisocial society. IMHO it's indicative of what I have dubbed Americas "mental illness epidemic"; specifically cluster B personality disorders [0] which are characterised by socially-destructive and self-destructive behaviours.
If that's the world you want for you and your loved ones, congratulations. You've earned it!
[0] https://wikipedia.org/wiki/Classification_of_personality_dis...
If tax incentives to attract companies and at-will employment are already viewed as a destructive collapse of democracy, it's no wonder we are not getting anything done here in Europe.
I suggest you take the time to read independent analysis on the benefits of corporate subsidies in America.
I know I've read of many cases where they cost cities and communities far more than the economic activity they promise to generate (eg Amazon warehouses and sportsball stadiums).
Subsidies could make strategic sense in the case of AI, but it's terrible economics to encourage businesses that take more from communities than they give back, especially those that don't employ many people.
Data centers are known to create few local jobs while requiring upgrades to the power grid or water infrastructure.
Wouldn't that make it important to subsidize them at the national level, precisely for that reason? Otherwise it seems likely to be a NIMBY problem.
But now I'm talking about data centers, the main thing we lack here is AI research labs, is it not? I think those should create many jobs.
Why should taxpayers anywhere pay private enterprises to create infrastructure they own and control for-profit, rather than just cutting out the monopoly middle man and establishing that infrastructure as a public utility who services the domestic economy, equally?
This is the kind of madness I'm talking about. If taxpayers have to subsidise products and services the "free" market refuses to provide, they should have direct ownership in the profits; not just the losses. Why is the default for society to subsidise expenses and privatise profits? That is ideologically-driven dogmatic insanity, which allows select individuals and foreign entities control over the national economy and its security, and is the entire reason madmen in the USA have the EU (and world) by the balls.
No greed will be rewarded. If European consumers don't like the end AI models that are produced, they can forgo buying them and the investors can watch their fortress burn. For the EU taxpayer, nothing was gained and nothing was lost. All the state did was stand out of the way.
If the models are good though, they will have their sovereign AI and should be happy to pay for it instead of American or Chinese models. You may call it greed, but to me it just sounds fair.
> For the EU taxpayer, nothing was gained and nothing was lost.
Except for the "massive" subsidies you advocated for, right? Except for the massive cost to provide utilities and infrastructure to support those businesses? Except for the cost to support the employees they refuse to provide minimum "protections" for? Except for any damage they do to the local environment?
You seem to think that businesses exist in a vaccum where they have no impact on society or its population, except when they succeed and everyone should be eternally grateful. This is the systemic greed and narcissism America exports to the world; not just through its businesses, but its media and population across forums such as this one, and is exactly what I'm talking about.
There would be no subsidies and the companies would be responsible for funding their own infrastructure. Protections for workers would be minimal, because we wouldn't really be interested in hiring people who want strong protections. They can stay at their current job. You balk at this, but all of Europe's capable tech minds left the EU to go work at tech in the US over the last 30 years. They didn't care about the generous protections, they cared about making stuff and making money. Which they did, overwhelmingly.
What I am describing is an existential battle for the relevance of Europe on the world stage in the future. Right now Europe is looking like a vasal state, picking between the US and China for who it's overlord will be. If Europe wants independence, it needs to mobilize immediately to catch up.
3 Times As Many Europeans Move to the US, than the Other Way Around.
[1]: https://mises.org/mises-wire/3-times-many-europeans-move-us-...
In 2017. Wonder what it's like today.
"US, for 1st time in 50 years, experienced negative net migration in 2025" [0]
Curious how they jumped for nearly decade old stats, huh?
[0] https://abcnews.com/US/us-1st-time-50-years-experienced-nega...
Butcher worker protections and quality of life across all industries to (hypothethically) benefit a single one?
No thanks.
Why do you feel grinding insane hours would be beneficial to AI progress?
Exactly this. You can't have a competitive industry while at the same time heavily redistributing wealth to the point where people don't have any incentives at all.
They can be serious about home-grown AI without needing to become a libertarian capitalist hellscape. I prefer happiness, safety and privacy over competing with the US / China.
Sure, but good luck finding serious top tier AI researchers who are willing to work for $80k/yr when the US is offering them upwards of $1M/yr.
> GPT‑NL is developed within the Netherlands and Europe. This gives us full control over the model, the data and the choices we make. We avoid dependency on non‑European providers and invest in a sustainable AI ecosystem aligned with our laws, values and societal goals.
I love it! So this is our answer to America and China denying foreigners access to their frontier models.. a massive 13,5M€ founding to develop souvereign european ai, trained exclusively on legally obtained documents and highest moral standards as defined in EU AI Act.
NL could simply say: no more ASML machines, and no more ASM wafers.
ASML’s EUV technology is partially based on US research and so Congress has a degree of control over it, so it’s not that simple: https://web.archive.org/web/20230116222847/https://www.nytim...
The US can only block exports. They cannot force exports. The NL-US relationship is quite toxic, for example the "Dutch America Friendship Treaty". Everything is very one-sided.
it is that simple
Any move like this would be simply by murdering the golden goose. ASMLs stuff is good no doubt but you don’t want to give the world the incentive to develop an alternative. If it was done once it can be done again and once it has been done again say good bye to all those returns on monopoly.
What makes you think they're not trying? The incentive has always been there. They simply fail to succeed.
There’s trying and then there’s trying.
If ASML’s output were absolutely necessary to the US or the rest of the world, and if no replacement could be made, the Netherlands would have infinite leverage and be the most powerful country in the world. That they aren’t tells you all you need to know about exactly how far they could take a strategy like what you’re describing. Yes, everyone needs ASML. But ASML also needs everyone else.
An elite toolmaker is nothing without the factories that use the tools.
The odds of the Dutch attempting to encumber the use of ASML machines by western bloc allies is similar to the odds of the sun going out suddenly. It ain’t gonna happen.
You don't wanna find out how fast american troops would land there..
Faster than in Iran?
So good to see these developments. Every country should do this. I'd even say every person should gave their own personalized AI running on their own computers. If only the costs involved were not so astronomical.
Why? That doesn't make any sense.
The government would be far better off figuring out how to take commodity models and applying them to government functions where they can, with deterministic scaffolding and guardrails, to make government more efficient, optionally using RL on traces from their use to improve their performance.
Imagine taking models and fine-tuning them / doing RL rollouts to help automate permit application approvals, as applied specifically to Dutch permit processes. That would be a real help to Dutch businesses!
That type of applied AI is more interesting and effective now than just trying to make another foundational model that isn't going to work well or do anything of economic value.
Another thing they could do is try to attack the the suppply chain issues. Try to form an alliance to block RAM deals or something, or to get fabs on EU soil, making HBM for the people. We have some bargaining chips, especially when banding together with a few large EU states. Not as EU, just a few specic countries. No bureaucracy, just elite trade diplimacy. Probably best done in secret so the big labs don't catch wind of this. Any NL/DE/UK/FR/CH/PL/IT govt people reading this?
> Why?
Because then the USA can't just turn it off.
I think it will be cost effective at some point. Computers were limited to research institutes before the personal computer arrived.
I hope you're right. I really don't want a future where only corporations and governments have computers.
"Champions of a European AI model should ask themselves if a European effort would be more effective than Meta, which this year will spend more on chips ($125 billion) than Germany spends on defense ($114 billion) and offer salaries of over $100 million to attract the best researchers, and is still failing to catch up. Elon Musk tried and failed to build a good AI model."
https://www.siliconcontinent.com/p/nineteen-thoughts-on-ai-a...
Nvidia will certainly be pleased.
I think at this point what the Netherlands, and any other country that wants a good model in their language should do, is gather up every piece of text ever written in that language and license it to the big AI labs/companies for training. I'm sure there are vast libraries of books and other text that haven't been digitized and aren't a priority for the big labs.
I think they should just make a national security thing and gather every piece of text in every language.
Yeah except replace "license it to the companies for training" with "pay the companies to train on it"
Oh I didn’t mean at all charging them. I mean licensing in the sense of granting rights for the purpose of training. Probably most labs would be fine adding the language to the training for free as long as the dataset quality is high and it improves the results. But yes, pay them if that’s what it takes for them to use it.
Interesting that this got posted now: the project is receiving increasingly more skepticism lately in the Dutch tech scene [0], and I think that’s fully justified.
[0]: https://www.quotenet.nl/zakelijk/a71588202/techondernemers-m...
What is the exact skepticism? The only thing I could get from that was from some "tech entrepreneur":
> GPT-NL was never built to compete with Claude or ChatGPT. It was trained exclusively on licensed data, and is intended more for governments and companies where privacy and compliance matter more than raw performance.”
That's it? That it didn't aim to compete with SOTA models? Maybe this is something you have to start with something, then ramp up, rather do what only a select few labs been able to do, start with really big models. Especially if you're resource constrained, which since this is a government project, I really hope for the sake of the tax payers it was.
I mean if you are wasting funds kind of knowing it's nowhere near remote competitive, then it's kind of a fraud.
TNO is something like semi-DARPA. It gets a lot of stuff tax free and a lot of gov funding, but a lot of their budget is from getting businesses to hire their R&D teams.
They do really good R&D on a lot of stuff. This is just their attempt at public credibility/internal skill building to enter the LLM business.
Doubt its going to be successful, but they "waste" a lot more money on other things that you never heard of. Its not fraud, its just R&D dressed up a little too much too early.
Targeting a niche audience with specific requirements is not fraud.
But why is "competing against remote SOTA models on quality" the only thing that matters here?
What the hell else is there? All the other stuff can be done by an intern with an 8 euro HF Pro subscription.
Other than actual research, which is in a different camp.
Common approach I've seen is having workflows with paid/larger/hosted models for some workflow where you don't quite know exactly how it'll be when you first put it together, then with time you've locked down how things more or less work yet you still need free-form text parsing of some kind, so you end up replacing the bigger models with carefully post-trained small models.
Besides that, there is a ton of use cases for smaller models for a bunch of different things. We'll be unlikely to be able to run LLMs (actually Large) on smartphones for a while, while the smaller LLMs seem to run already on-device in experiments.
TNO literally stands for "Dutch Organisation for Applied Scientific Research" (Nederlandse Organisatie voor Toegepast Natuurwetenschappelijk).
Previously posted on 02-dec-2023 https://news.ycombinator.com/item?id=38497495 3 comments
Two and a half years and still not complete? That's ridiculous.
AMALIA, from Portugal, going the same path!
https://en.wikipedia.org/wiki/Am%C3%A1lia_(LLM)
They are starting to deploy to customers now, not sure if that counts as "complete" or not. The big innovation here is that they are doing it all legally.
I don't understand countries (especially governments) wanting to have their own models when there are already pretty solid open source (weights) models out there.
Countries should want control over _where_ the compute is happening rather than _what code_ is running.
What's wrong with a country hosting a Kimi, Qwen or GPT-Oss on their hardware for their government work purpose?
An LLM is an encoding of a culture, a way of viewing the world.
They are not neutral technology, they are a direct representation of the training set that has been chosen and how they are aligned.
In many ways, they are ideology made code.
If we leave building them to the US and China, only their way of seeing things will be digitized.
I don't like the idea of that.
Yes and also, US and Chinese models are censored in different ways. US models are way too prudish for personal use in Europe because they're afraid to piss off religious investors. Chinese models are too censored on history and current affairs, eg the tiananmen massacre never happened stuff like that.
Chinese models aren't censored as much as you think, you can download the model and run it somewhere else and they will happily tell you about Tiananmen Square. Or heck, ask DeepSeek via Openrouter, it will do the same.
The censorship works kind of like with Fabel, it kicks in before the model responds.
There's an absolutely massive cultural and behavioural bias in those models. Models will suggest things like "go to the hospital" for things that require GP appointments, "just drive three hours" while it's faster to go places by train, and so on. They will do it in anglicised Dutch (compound words split, English-like grammar structures) that's perfectly understandable, but the cultural bias is there if you know to look for it.
Furthermore, the expertise in designing and training these models is valuable as well. The existing models are good as a starting point in terms of learning from previous mistakes, but we should not just let a handful of American and Chinese people keep the knowledge and expertise.
One problem with this particular project, though, is that copyright has been enforced for Dutch LLM training before, and the AI industry cannot exist without massive scale piracy, the likes of which has never been seen before. A lot of Dutch training material exists in pirated books that AI companies in countries that do not care about copyright have access to, but are exempted from the training set here. The impact of enforcing copyright on an AI model will be quite interesting to see.
It is not about the country but the language. Most llms have poor or no support for Dutch.
Idk which models you refer to, but I tested a bunch recently, and they performed well on Dutch. Only the smallest, such as qwen 3.6 27B, made up words and switched languages.
There's a large gap between making up words and an actually native text distribution. LLMs have a clear pattern, clear tells, a "feel" in English, and it's normally even more pronounced in non-English languages.
Lots of bias towards English sentence structure, idioms, etiquette, etc.
I didn't notice any of that. Such a bias would be strange, because certainly smaller models don't have the luxury of learning grammar independently: it's still word sequences, and languages are quite well separated.
There would be a bunch of value in having, say, a good 30B-class model that used my local language as well as it does English. There's lots of cases, especially in the government sphere, where local processing is a requirement and frontier-level capabilities aren't required. Making those cheap to run seems like a fine goal.
Can you provide some examples of these use cases?
Support bots and question answering with access to sensitive pii?
Yes, but what's the point of a support bot that writes good Dutch when it can't follow instructions, doesn't understand the questions or can't solve problems? I might be wrong, but I don't think atm these models have the cognitive ability to perform any task in a satisfactory manner.
As for accessing pii, I imagine the value here is in the fact they're local, which has nothing to do with the "sovereignty" of these models. If anything, a model is more likely to be tricked by a malicious prompt the farther it is from the sota.
A good harness and engineering is important no matter which model you use. But Sovereignty of hosting is also important because without it all pii is being leaked.
I don't understand this. Even if that were true (and it isn't in my experience), a model that is trained on a Dutch corpus and arguably "knows Dutch well" but has the reasoning and comprehension abilities of a three year old is useless in any case. I'd rather use a model that can only speak English and put an automatic translator around it.
To be fair. There is a security concern angle: even open-source models could be trained as sleeper agents that act adversarially (for example, adding backdoors) when used in specific national companies in specific settings. This is very difficult to detect or void, so if you want to be sure 100% that this isn't the case, you have to train your own model from scratch.
Why should Dutch people be expected to make do with models 99% trained on American/Chinese cultural context and language?
Maybe the Dutch really really want an LLM that tells them the truth as straight as possible no matter how harsh - that might be tricky
Understood, but they could fine tune base models on their own cultural context and language. Why reinventing the wheel?
This gets better short-term results for a fraction of the cost, for sure, but what do you when China places an export control banning the release of open weight models? If you don't have your own talent, you're then relegated to using a base model from 2026 or whatever the cutoff date is, forever. That defeats the purpose of a 'sovereign' model made for and by your people.
They could apply the Polder Model of consensus decision making with a mixture of experts.
https://en.wikipedia.org/wiki/Polder_model
Funny, that's what I thought when PewDiePie set up his monster AI rig and what he called a 'council'. Quote:
"PewDiePie has built a custom web UI for self-hosting AI models called "ChatOS" that runs on his custom PC with 2x RTX 4000 Ada cards, along with 8x modded RTX 4090s with 48 GB of VRAM. Running open-source models from Baidu and OpenAI, PewDiePie made a "council" of bots that voted on the best responses, and then built "The Swarm" for data collection that will become the foundation of his own model coming next month."
https://www.tomshardware.com/tech-industry/artificial-intell...
Calling a bunch of LLMs a "council" is just rebranding well known ensemble methods with shrill marketing hype, nothing original or out of the ordinary. Mixture of experts and every other idea in his stack has a literature older than PewDiePie's career.
Yet another attention craving influencer who shilled crypto scams during the crypto bubble and is now marketing "AI councils" during the AI bubble.
He's not a serious or honest person, AI is just what he pivoted to after crypto. That's not innovation; it's attaching trendy branding to ideas that were already old when Marvin Minsky wrote The Society of Mind in 1986, three years before PewDiePie was zero years old in 1989.
The only thing PewDiePie's brought to the table is cleverly optimized YouTube thumbnails designed to attract clicks. The architecture is decades old; only his marketing and shilling is state of the art.
Lighten up dude.
Grow up and stop watching children's entertainment and promoting crypto scammers.
PewDiePie's Shady Promotion of Dlive (Cryptocurrency Scam)
https://www.youtube.com/watch?v=GYyNaQzZTo0
>If you want more concrete and in-depth explanations of what was touched upon in this video, here are the sources I used: Dlive ToS: https://community.dlive.tv/about/terms-of-service/ Dlive/Lino's Shady History: https://steempeak.com/@meno/an-objective-look-at-dlive-s-exi... Dlive overview: https://community.dlive.tv/about/welcome-letter/ 20 Million Chinese Investment: https://web.archive.org/web/20190716124653/https://www.coind... PayPal Suspension: • Is PewDiePie Supporting a Scam? Difference between Lino & Twitch Bits: • PewDiePie & Co-Founder Defending DLive? Is... Refusing to explain how they make profit: https://variety.com/2019/digital/news/pewdiepie-dlive-live-s...
Does what PewDiePie calls a council not resemble a Poldermodel? That’s all I associated with it. Please don’t lecture me about what to watch and drag up negative associations that have nothing to do with the subject (multiple LLMs voting for the best answer) for a simple observation.
Zonnetje in huis.
edit: also, 7 years on and after closure of the platform I cannot find evidence of rug pulls or exit scams associated with Dlive or any of their underlying crypto funding mechanisms, so before judging others on the content they consume maybe don’t take random YouTubers at their word and then don’t look into it 7 years after the fact.
I thought finetuning data can't contradict foundation models, and anything that are inconsistent with the standard LLM American-Chinese split personality would be rejected?
Fine tuning happens on top of pretraining, so of course it can "forget" pretrained defaults when warranted by the new data it's being fine tuned on.
But you have to have more data than used for pretraining for the added knowledge to take precedent over pretraining, no? If that would be the case, you practically contradict the knowledge in the base model.
I mean ... LLMs are sort of an extreme and living proof of linguistic determinism. Their behaviors are dictated almost entirely by disorganized language data, primarily English and Chinese. So you can't just add a language as native primary language in a quick post training, I think. There's no way that it would work.
Oh, it's all fine with cultural context here -- we don't even dub English language movies here because we are that cheap
Really? Because I'm pretty sure that at least every two days there's a active post with a top voted comment along the lines of "The EU isn't doing AI themselves, they are so hosed".
[flagged]
[flagged]
Have you heard of ASML? NXP?
Ignorant comment
Please don't move the goalposts. What computer parts does ASML or NXP make?
ASML only makes the lithography machines, 85% of which go outside the EU (let that sink in). And then fabs in Taiwan, Korea or the US use those ASML machines to etch US IP for computer chips. EU doesn't make any computer parts domestically.
And NXP mostly makes various microcontrollers and small chips, not high margin IP decenter centric parts like ASICS, FPGAs, CPUs or GPUs.
So not only are you the ignorant one here, but you also have the audacity to insult others with so much confidence.
@dwa3592 below. Firstly, why are you moving the goalposts in bad faith again just to stir an argument? What does that have to do with my original comment?
And secondly, there's other lithography machines out there, not just ASML.
And thirdly, the IP Nvidia, AMD, etc develop to etch on silicone via ASML machines makes them more valuable than ASML.
Fourthly, repeating my "let that sink in" phrase is just childish and low-IQ trolling, unworthy of this platform.
Europe is currently hosed because we made the mistake of trying to develop economies complementary to the US and china.
That was a big strategic mistake. In the US case it was borne of the mistaken belief that we shared values and were partners.
But don’t mistake the situation for lack of innovation of capability. Europe is currently adapting, but I think the success of Ukraine is one reason to be optimistic that current adversity might actually leave us better off in the long run.
Corrupt countries with broken legal systems tend not to fare that well in the longer run.
>Europe is currently hosed because we made the mistake of trying to develop economies complementary to the US and china.
NO, it's hosed because it's not competitive and slept on the wheel at several key digital economical revolutions plus sleeping at the wheel at preventing obvious geopolitical issues (gas dependence to Russia, losing auto industry to China, losing semiconductor industry, losing SW industry, etc).
You can't be an economic leader if you keep losing on all fronts and only be a leader at how much welfare your spending.
> the mistaken belief we shared values and were partners.
We do share. US and a lot of latin america is mostly European immigrants and European culture, making our cultures are much more similar than the african and middle eastern ones the EU has been importing and adopting. Where we differ is that US still has free speech and isn't devolving into a stasi police state that arrests people for Tweets that the political establishment find uncomfortable.
> might actually leave us better off in the long run.
How? EU's economy has been pretty much stagnant since 2019 when you account for inflation loss.
> But don’t mistake the situation for lack of innovation of capability. Europe is currently adapting,
How? Where is Europe's Nvidia and AMD? Where is Europe's TSMC? ASML can't feed an entire continent.
The EU economy has absorbed covid and the Russian war in that time. It’s pretty unsurprising that things are flat.
The main thing Europe lacks is economies of scale in investment in tech, and a common language. Big data tech wasn’t a priority until we realised it was going to be weaponised against us by an ally. I think we’ll do fine. Most of AWS and Azure is expensive superstitious cover your ass bullshit anyway. It could all be regulated away with proper data residency laws, which I hope are coming soon.
>>ASML only makes the lithography machines
Woah! only lithography machines???? it is literally impossible to make any device capable of running anything close to AI without ASML. Let that sink in.
Funnily ASML owes its current success partly to US funded research (straight from Wikipedia):
> Two years later, it joined a consortium, which included Intel and two other U.S. chipmakers, in order to exploit fundamental research conducted by the US Department of Energy. Because the Cooperative Research and Development Agreement (CRADA) it operates under is funded by the US government, licensing must be approved by Congress.[12]
Why are you acting childish and petty? I said EU hasn't got AI compute manufacturing(aka no equivalent IP to Nvidia and AMD and no equivalent to TSMC or Samsung fabs), not that it doesn't have lithography machines manufacturing.
Surely you understand that while you can have the latter, you can also lack the former.
In a recent podcast, it was summarized as:
ASM (International) makes machines that add material to a silicon wafer (deposition).
ASML makes machines that remove material from said wafers (lithography, etching)
(I was a bit surprised that's not combined in 1 machine. But let's move on)
Then Besi makes machines to stack / interconnect / package those ICs into a package. I'm assuming pick & place machines are other companies' turf.
The above are all Dutch companies, operating a pretty important section of the tech stack.
Iirc there were (& probably still are) some IC fabs in Europe, but mostly older nodes (like useful for microcontrollers used by car manufacturers. Wikipedia has a list). So for SOTA smartphone SoCs it's off to Taiwan (TSMC), South Korea (Samsung) or China (who makes everything, including smartphones & the chips going in there).
So as far as EU goes, the capabilities are mostly there. Skilled workforce? Check. Money? This is a rich continent.
What's missing is the guts to say "hey, let's dump €100B into this & make ourselves some laptop & server CPUs!".
But now the important thing: several of such initiatives are starting to bear fruit, and b) confidence that EU can do such things, is growing.
As for bureaucracy / red tape... sigh... (won't be fixed any time soon)
>In a recent podcast, it was summarized as: ....
Yes, all true, and all things I didn't disagree with because I wasn't talking about that.
The point I was talking about you only addressed to some extent in a line below that:
>What's missing is the guts to say "hey, let's dump €100B into this & make ourselves some laptop & server CPUs!".
Yeah exactly, the EU doesn't have computer manufacturing capabilities (just like I said 5 layers up) and it never will because it doesn't invest and also doesn't attract investors to invest.
>So as far as EU goes, the capabilities are mostly there. Skilled workforce? Check.
No they're not. We don't have the skilled workers for that. Nobody in EU knows how to design Nvidia and AMD level GPUs and Altera and Xilinx levels of FPGAs that power AI datacenters. Nobody in EU knows how to make competitive 2nm fabs, otherwise EU fabs would have already bought ASML EUV machines and updated their ancient processes to the highly more profitable nodes instead of being stuck making cheap legacy nodes for cars and white goods. Those old nodes are still important to have to an extent, but ask yourself, would you rather sell a die for 10k a pop or sell 1000 dies for 10 cents a pop? Would you rather make more money or less money?
> Money? This is a rich continent.
Money is meaningless if you're not using it right. China is pushing to beat the EU and they have less money than the EU.
The World's Most Important Machine ;)
https://www.youtube.com/watch?v=MiUHjLxm3V0
Most important machine ... built on US IP, subject to US export restrictions, used to manufacture high value US IP, in factories outside the EU, so profits of those chips goes to US. A point I have addressed over two times already.
Also ASML even threatened to leave the NL if the Dutch government doesn't do what they want on taxes and labor policies. So having only a single card to play that EU can loose at any time, it's not putting EU tech sovereignty argument in a good light.
The "wahabout ASML" that keeps being spammed by people here, isn't proof of EU compute and AI sovereignty. It's the exception which is why it's the only thing people can mention on EU tech and they DDoS you with it as if that changes anything.
Are people here that petty that they can't stay on topic and argue in good faith and instead need to hijack your argument to go on offtopic whataboutism for a cheap gotcha spamming "whatabout ASML" on unrelated arguments?
>ASML only makes the shovel making machines
>>Yeah but Europe doesn't build any computer hardware,
Well, then this is will be a good start.
EU bureaucrats are too busy trying to keep the welfare/pension system from collapsing, defeating Russia, supporting Ukraine, managing the fossil fuels energy shortages, figuring out how to nerf Chinese EVs while supporting domestic car companies, and restricting social media free speech to make sure the "far right" don't win elections.
So of course, semiconductor manufacturing sovereignty is very low on their priority list.
Anyone who uses quotation marks around far right has clearly stated their allegiance.
It is no wonder then that such a person would do their best to poo-poo the worlds most successful peace project and the bastion of rule of law.
How many in that list of things is the US also doing?
Different scale for those problems. Way different scale. US is monetary rich, oil rich, energy rich, manufacturing rich, and doesn't suffer from russian aggression at its borders. US is so bored from how rich and problem free it is compared to Europe, that it can afford to keep starting foreign wars as if nothing ever happens.
Also back on the topic, the US managed to bring TSMC to open a cutting edge fab in the US and has already been operational for a while. Which already puts it way ahead of the EU on this front as well.
The thing is, US is much better on actually making things happen when push comes to shove. It saw it's deficient and vulnerable on domestic semiconductor manufacturing, it then made it happen with TSMC. It's doing the same with domestic ship building with Korean partners.
US might be slow moving, but somehow EU is even way slower at realizing and addressing its vulnerabilities, only waking up when it's far too late, causing it to pay a much more painful price for sleeping at the wheel (Russian invaded Ukraine in 2014 BTW, not in 2022, and they were building another gas pipeline with them), and when this type of own-goaling keeps repeating enough times you see the correlation with EU's decline as their economic rivals keep biting more and more market share from their industries as they sleep on critical changes and developments.
The US is a single country. Russia is not on the US' doorstep. The US has its own oil. The US prints the world reserve currency.
That's what bureaucrats are supposed to be doing BTW.
And yet, the ESMC Dresden fab is getting built. And now there's a Chips Act 2.0.
They’re building a competitive-quality model, from scratch, with fair compensation to content owners, for €13.5 million? Something’s wrong with this picture.
Being cheap is on brand for inhabitants of the sea floor. Nothing is wrong
I feel that not only is Europe losing its independence to the US and China, but it does not even try to take part in the race.
Unlike the US, Europe has no California-level VCs. I don't expect hundreds of billions of Euros to be poured into long-shot projects.
Unlike China, Europe has neither cohesive public investment at the global level nor the drive to grow. Long-term investments have a lot of words, a lot of regulations, a lot of proxy goals, but there is neither a lot of money nor urgency. It was captured by this post: https://x.com/piotrsankowski/status/2065795919623438546
So yeah, both in economy and warfare, Europe dooms itself to be in the hands of the US, China, or a mix of both.
> Unlike the US, Europe has no California-level VCs.
Some would consider that a good thing. There is a lot to be said for VC in recent years not being beneficial for the economy, certainly on an individual level, other than "number go up".
Sure.
At the same time, it made in many cases EU dependent on the US. A lot of governments are basically dependent on MS Office or Google Cloud.
With AI, it is even more strategic.
My impression is that in Europe much fewer people are convinced that AI-maxxing is necessary or even a net benefit.
And if you ask a bit more, you'll find that those same people are very likely to daily-drive AI models, desktop and phone operating systems and various other software critical to their professional and personal lives from US companies. And buy tons of Chinese products over Chinese or US e-commerce platforms.
What people say matters much less than what they do.
Much fewer people in Europe are convinced maxxing anything besides work/life/life balance and generous social support are a benefit.
All the stuff that doesn't help an economy grow or pay for the future.
I have rather deep concerns about our future with AI.
Yet, I'm still sending hundreds of dollars to US companies providing it. I'd much rather send it to EU companies.
Americans are also far less into AI than a startup forum like HN will make you believe.
In real life, there are a few AI maniacs that make their entire identity about how they use AI, but it's hardly the sensation that the internet will make you believe. I don't believe there is any difference between the USA and Europe, although the lack of employee protection does mean that it's a lot easier for Americans to lose their jobs when their managers get lured in by AI companies.
After all, the entire AI bubble is all about VCs and startups hyping each other up until profits magically appear.
Which has nothing to do with VCs, just with sourcing decisions.
If there were European MicroSoft or Google, there would be a preference.
Well those companies exist because of VC's, and the nation's business freedom compared to Europe
> There is a lot to be said for VC in recent years not being beneficial for the economy
What a wild statement, VC's are behind most of the growth in the US economy, and they directly drive up wages in tech. I'd be fascinated to hear a valid complaint of VC's that isn't just money envy
Economic growth doesn't mean much if the benefits are concentrated in a small percentage of the population.
And while I can't disagree it has benefited tech wages historically that is just one industry. Then there are also the recent mass layoffs.
Some people consider it a good thing that communists boiled people's hands as torture. Some people consider it a good think that Iran massacred 10,000s of its own citizens. Some would consider it a good thing that Israel killed all Palestinians in Gaza.
Lol, okay bud. Lets go all in on the extreme hyperboles.
I mean if morality is just personal preference why do we even allude to it at all?
Is that what I said in your mind? I honestly had to chuckle at the extreme hyperboles you decided to use. Surely having a less favorable view of hyper capitalist economics is on the same level as torture and genocide. Yup, that is a totally reasonable and measured response that just exemplifies acting in good faith.
Anyway, thanks for genuine chuckle your response got out of me.
You don't even say it is a good thing. The closest to morality come is alluding to the fact that someone holds the opinion that it's good. Even though you also likely hold this opinion, you don't even want to go as far as saying that you think it's a good thing.
This is entirely absurd.
You reply is what we in the game call "moving the goal post".
Some people consider it a good thing that goal posts are moved.
If you are not even here in good faith I can imagine some people do consider that a good thing yeah. No argument from me there.
Europe decided to regulate the hell out of foreign AI instead of investing in their own systems. It's sad to see the European continent lost the race to create a decent startup ecosystem (no decent search engines, social networks, cloud, mobile OS) and now it seems to be hellbent in losing this battle.
>It's sad to see the European continent lost the race to create a decent startup ecosystem
What's ironic and sad at the same time is that pre-2022 Russia's Yandex(domestic Russian variant of Google) was lightyears ahead of what EU, a significantly richer and more capable block, had. IIRC, their reverse image search was so good, they had to nerf it because people were using it to find the identity of people from photos.
Same for Israel, their tech sector is probably greater than the EU one combined
Absolutely shameful how the EU kept managing to snatch defeat from the jaws of victory over and over.
Not surprising. All the Yandex people that moved over to here to the Netherlands that I know of are astounded about the insane difference in the tax burden between what they had in Russia and what they have in Western Europe (when their 5 year tax discount ends, that is). If the government takes the bulk of your income after a certain point, there isn't really that big of a push to create ground-breaking technology.
> If the government takes the bulk of your income after a certain point, there isn't really that big of a push to create ground-breaking technology.
I'm skeptical that high taxes is a large reason to lose to California of all places. Maybe in some important sense CA has "earned" that via talent and funding density while NL hasn't (from the perspective of a company, to be clear)
NL is a corporate tax haven in Europe(and also compared to California), that's why almost all US and foreign companies have their corporate HQs there, Dublin or Luxembourg.
Companies go there because taxes are low for them, not necessarily for their employees(ignoring the NL 5 year tax break for foreigners). It's kinda like the US Delaware of Europe.
We also have much nicer societies than Russia. It's a dictatorship ffs.
And yes having nice things cost money. And a safety net is important.
I would never want to live in America even if I got 3x my wage. Nor Russia of course but that's a foregone conclusion.
I think much of that is because European customers (both private and business) tended to prefer American suppliers over suppliers from European countries that were not their own. That may have something to do with most people in IT being quite fluent in English, while European products were all-to-often half-heartedly translated from German/French/Spanish/Polish/Italian/Ukranian.
In many cases, well-established and well-liked European services have been supplanted by American counterparts that came later and were not really better in any way. They did usually have much more money to burn though, undercutting pricing until competition was dead.
I'm speaking in the past tense, because now for the first time in the couple of decades I can remember, there seems to be a somewhat commonly held preference for European suppliers.
Frankly, being Russian myself, I'm also very disappointed by the state of tech in Europe.
But you know what hurts the most? That I know it wasn't always that way.
I'm sitting right now in the same country that invented the Minitel, built out the TGV network and the Grands Projets, and don't even get me started about the weird and wonderful machines they've got in that museum in Mulhouse, hell, you could go back in time to Gustave Eiffel. Industry and ambition used to be here. It was almost physically painful to discover that it seems to be gone now.
> It was almost physically painful to discover that it seems to be gone now.
It's not gone, it just needs to be re-discovered. And the bureaucrats need to flash some € then get out of the way.
The EU chips act was already largely a failure and they threw plenty of money at it. The problem with the Europe is they look to their bureaucrats to stimulate things then expect different results from the last 50 times.
Europe needs its own private industry that attracts talent and capital. It doesn’t need another EU press release talking about hopes and dreams
>The EU chips act was already largely a failure and they threw plenty of money at it.
Yep, the proof is in the pudding. They managed to grab defeat from the jaws of victory again when US got TSMC to open a fab there.
For example, both German and Austrian semiconductor companies are choosing to expand manufacturing in Malaysia and not domestically.
I don't know what more proof people need that the EU is cooked on manufacturing side.
>The problem with the Europe is they look to their bureaucrats to stimulate things then expect different results from the last 50 times.
The definition of insanity. This is a mentality problem and is deeply ingrained into the EU population. European people are always looking at the state to solve any problems including those of industry and free market, without realizing the state mostly isn't good at those things and they'd actually be better off with less government there. Whenever the state intervenes into the economy it just further supports the massive bloated legacy companies who lobby the politicians never the startup ecosystem. And then people mistake again by voting for more government hoping to fix that which just results in more expensive bloat and more red tape nor a competitive freer market with more capital and innovation.
>Europe needs its own private industry that attracts talent and capital. It doesn’t need another EU press release talking about hopes and dreams
Hey, it worked for the USSR, right? ;)
>It's not gone, it just needs to be re-discovered.
After how much time of not being discovered does it count of it being gone?
> And the bureaucrats need to flash some € then get out of the way.
And when is the EU gonna do that? When pigs fly?
You are saying that as if China or the US are completely isolated from the EU. We live in a globalized world whether you like it or not, and every supply chain spans multiple countries.
Arguably, staying out of the AI "race" is a good thing
Military race isn't a good think either, but you don't want to be on the losing side.
> Unlike the US, Europe has no California-level VCs. I don't expect hundreds of billions of Euros to be poured into long-shot projects.
My ex-neighbor (when I was a teenager, living in Belgium) and very good friend really wanted to make it big. He became a chip engineer, moved to California, raised money for a first startup (it tanked) then raised money for a second startup. He made the world a better place (he created some very specific micro-inverters for solar panels) and made a $$$ exit.
The EU saw exactly zero of the wealth he created and he's never ever coming back to what he considers a failure of a continent.
That's the problem: many of the great minds with the mindset required to do great things already left the EU.
> So yeah, both in economy and warfare, Europe dooms itself to be in the hands of the US, China, or a mix of both.
And in energy (economy is energy and energy is economy, and China really understood that) the EU doomed itself to be in the hands of Russia.
We are a failure of sinking continent.
Europe is a great place to live if you just want to float through life.
The US is a great place to live if you have talent, want to work, and want to reap the rewards.
> economy is energy and energy is economy, and China really understood that
In former times the energy monopoly was called "The Power Company"; we intend to give this name an entirely new meaning."
– CEO Nwabudike Morgan, "The Centauri Monopoly"
I'll play devil's advocate a little bit - I'm not sure it is losing its "independence" by not taking part in the race. It could very well be that it is gaining independence from tech and choosing a "second mover advantage" to decide how it gets deployed after seeing how it impacts everyone else. Let the US and China experiment on the bleeding edge (and their citizens feel the effect, both good and bad), and then be picky about how you use it.
I don't know if it is the right strategy but there's certainly a legitimate strategy in there.
The problem is recursive self improvement creating a very difficult gap and the fact that power, compute has a lag from when you invest and when data centers come up.
You also can't just spin up a research team out of nowhere.
These things are true, however:
1. The labs in the US and China don't seem to have any problem selling (or even giving) access to these models right now.
2. If some kind of take-off happens which makes that not true, my bet is all bets are off on what that outcome even looks like. What would the economic paradigm even be under a superintelligent AGI? Do you think "it" is going to listen when Trump says "you can't work with Europe"?
There's a whole bunch of grey in between the two, for example only having access to second rate models, but I'm not sure that particularly matters if the strategy is "second mover."
I mean, it might listen to him. We have no clue, which is the problem.
Sure, but my guess is for "true" super intelligence we won't be able to predict whether that is true or not until it happens. I'm not a doomer, but I also don't really think we can "align" people, much less a "super intelligent" AI.
Let’s autonomous Russian drones, and Europe is at mercy of two other empires, who capitalize on this opportunity.
Europe is not a country.
Regulations are not even throughout each of the 27 member states. Each country is relatively small in the world stage.
Until EU progresses towards federalization, discussing this is a moot point.
Serious question: what does any of that have to do with the submitted article? Where is the relevance to the topic at hand?
I feel like building datacenters and filling them with chips may be more valuable than creating sovereign models. xAI I think makes more money renting datacenters to Anthropic than with the models they trained, and they could pivot thanks to their datacenters. By making regulations easier than in the US, this could bring some computing power to Europe, which then can be used to train sovereign models, or rented to big AI labs.
Also, when training models, you create talent that then could go to other countries (brain drain). Restricting that brain drain without imposing authoritarian restrictions on the movements of people seems hard, so it seems hard to keep talent as a competitive advantage. If instead the competitive advantage is datacenters with chips, power capacity building, fast path to building datacenters, I think they are easier to retain while preserving the rights of everyone involved.
This is not an open source model. In that sense I think the sovereign claim is a bit strange. It's the data providers that determine access to the model.
So it's a model that's sovereign as in sovereign kingdom of the Netherlands vs sovereign for the people's?
"sovereign" the marketing term basically means "in-house" now, where "house" depends on who says it.
A total of €13.5M has been allocated to the project.
I guess we’re going for GPT2 level capability?
apparently their aim is GPT3.5
I really think countries should build a sovereign _ecosystem_ and sovereign models are an excuse to achieve it.
An ecosystem is the tribal knowledge, revolving door of talent, known processes etc.
If the end goal is to make a half assed Dutch speaking model, I think it won’t cut it. I don’t see anyone using it over Gemma 4b that runs on my laptop.
An ecosystem is more durable and has desirable second order effects.
> A total of €13.5 million has been allocated to the project.
> This public investment underlines the importance of an independent, trustworthy and future‑proof Dutch language model.
It does, but not in the way you think it does.
> It does, but not in the way you think it does.
They're training a model, not funding a startup. €13.5 million is plenty to pre- and post-train a decent model.
We already had GEITje but it was banned by the courts. Of course it can still be found because the entire internet is not subject to Dutch law. But it did manage to stop development :'(
Europe should have a sovereign model on its content and languages that is trained with renewable energy and published as open source.
This looks like a good step in that direction.
I wonder with these stories. Why are there so many individual country efforts? We know the scale needed with scaling laws / capital / energy. Most of these countries alone can barely compete (even large groups of them would struggle.
Why don't they work together on it? Companies like Airbus have already been able to do that with aircraft.
I'm making a Dutch dictionary and would be interested to see how this model would fair in evals vs non specialized ones. I've tested a variety of models for https://hetnederlands.com content and differences can be big
Is it a proposal or a model? And if it is a model, how fies it fare on benchmarks?
What really matters is the sovereign capability to finetune the LLM models. Any model could be vetted and tested, but you need finetuning/lora training to prevent the model from being outdated.
What are they going to train with 13.5M really? We're a tiny company in Amsterdam in Holland and we've got "only 64x B300 to train on" so we could never make an LLM I thought, since we've got only 4M in compute.
And they're going to train an LLM with all kinds of extra difficulties compared to OpenAI for just 13.5M?
The very first Llama was 16M for one training.
This is too little, too late. Europe really need to start focussing.
All these tiny niche models are perhaps fun as an academic exercise or great for the researchers resume but I highly doubt that they'll add any value or will be used for anything serious.
Even if this becomes a somewhat decent model with a fantastic understanding of "gezellig", "kring verjaardag" or "pannenkoeken", how many people will interact with it before the limits of it will drive them back to a frontier model?
Even if the purpose of this is government & other regulated industries, do we really want our government to use a poor model? Either do it right or don't do it at all.
Prices for training have dropped immensely in terms of research required, code efficiency, algorithmic/sample efficiency, and possibly also hardware (I'm not qualified to say without looking it FLOPS/dollar, or even to be certain that's the right metric here).
overall, the revenue sharing model is (IMO) more interesting than the fact that it is dutch. Usage of data, and sharing it with the providers of this data, is an inherent part of the creation of these models that is not discussed as much as it should be
Supposedly this model also aims to treat publishers of all sizes well. Looking forward to its launch soon :)
Maybe it's time to acknowledge that current copyright laws do more harm than good and put another framework in place.
I fear sovereignty is not a adoption-driving feature
Honestly, I used to think the 'sovereign model' was a waste of money. But recently, with the US logic of restricting model exports, I've come to think that if things go south, they could even cut off allied nations. So now the sovereign model seems reasonable to me. That, in turn, means US influence is deteriorating. And that probably isn't such great news for American businesses.
Better work on something at Europe level
So cute.
Burning tax money. I dare to bet this will never lead anywhere.
"Burning" €13.5M of public funds. That's 4000 times less than the Cursor deal from a couple days ago.
I was actually surprised by how little it was.
How do you use it?
"sOvErEiGn"
[flagged]
[dead]
[flagged]
This really isn’t what HN is for. I think you should read the guidelines. I’ve looked at your posting history and it is all like this: relentless negativity with low information content.
Why are you being nasty? And in what world is the Netherlands a second-world nation?
> Excluding harmful content
#define(HARMFUL)
[edit] Downvoters please tell me what the problem is with specifying this?
I didn't down vote you, but you aren't really adding anything to the conversation. This type of pithy comment might be fun on Reddit, but at HN, we try to provide more constructive, and information rich, comments.
How about fixing whatever the hell prevents competitive private LLM vendors from appearing in Europe?
you mean getting rid of checks and balances, environment protection laws, anti-corruption laws?
They’re still debating that, they’ll get back to us soon I’m sure.
> A total of €13.5 million has been allocated to the project.
This is not even funny. If you want a competitive AI industry, you need to invest much more heavily in infrastructure first, building models second.