It obv does have an extremely natural humane sound to it, but I feel like if you pay attention to what it actually says there is a noticeable lack of intelligence behind it.
It honestly sounds a bit like a teenager's stream of consciousness to me, that's how they sound to me anyways.
15-20 sec into any of the demo recordings and I get the sense of 'noone is really there' from the AI responses. I guess it's to be expected from a v1 where the scope is prioritising how the demo conversation flows rather than its actual substance.
My second thought was it sounds like the colorless green ideas have slept furiously, so to speak.
My first thought was: omg, they've harvested the souls of a dozen insipid, self-centered genz' and trapped them in a machine, like The Finn living in an Aleph...
The conversation examples are impressive but this made me say holy shit out loud:
"it runs locally on a MacBook Pro M4 Max or better. And we figured out how to train Dessa on only one NVIDIA GB200 NVL72."
If it really can replicate that low latency + that expressiveness on local devices, that's one heck of an accomplishment for "indie" AI.
Speech is in quite a weird spot when it comes to locally running it. Some of the best STT and TTS models run at faster-than-realtime speed on decent consumer hardware with quite small models (so also no huge RAM requirements), but the tooling and library ecosystem is much more fragmented, making it a lot harder to build actual working products around it. There is also a much more noticeable coverage / quality gap when it comes to languages other than English.
As a speech-to-speech model Dessa is of course in a different ballpark, but given the intelligence it exhibits, it seems to be working with not that many parameters. The developers also seem to be admitting that to some degree ("in fairness, LLMs also have to learn C++ and whatever the hell Angular is"). I'm wondering what the utility of this would be. Given that it lacks both broad world knowledge and tool calling integration in any way, that seems kind of like a dead-end.
> > one NVIDIA GB200 NVL72
That’s around ~$4-6 million in hardware cost (one rack, 72 GPUs + accompanying hardware).
that’s a good clarification, skimming some of the comments here I was thinking they trained it on a single GB200 gpu. THAT would be a huge win for indie AI
Edit: Nvidia GB200 does not refer to a single gpu, Nvidia’s naming scheme is very confusing. Considering past gpus were called H100, H200 etc, I thought GB200 would just be a newer GPU, but it does actually refer to their 30+ gpu rack scale systems. The claim of “we trained this on a single rack of gpus” is definitely much less impressive than “we trained this on a single gpu”
> Nvidia GB200 does not refer to a single gpu, Nvidia’s naming scheme is very confusing. Considering past gpus were called H100, H200 etc, I thought GB200 would just be a newer GPU, but it does actually refer to their 30+ gpu rack scale systems.
It's even more confusing than that :)
GB200 = 1 CPU + 2 B200 GPUs (packaged together)
NV72 = A rack of 72 B200 GPUs (or 32 GB200 superchips)
Think of GB200 as something like Apple's 'Ultra' chip. Not exactly, but similar. They fuse the chips together with a very fast interconnect (connection).
Also a speech to speech model it can't do any tool use or you can't guide it much am I mistaken? How will it be used for agentic uses is the part I don't get. Edit: this is response from Argon if anyone was also curious: You are not mistaken about Dessa: because it was trained solely on raw audio with zero text data, it cannot read text system prompts or output code for tool calls on its own. To use it as an agent, developers must either steer it purely through spoken voice instructions or pair it as a conversational "voice front-end" alongside a separate text LLM that listens to the audio and executes the tools in the background.
> they didn’t have to worry about hosting/billing/subscriptions or any of that yet
Or marketing, which they kinda get for free. So much research goes unseen, there’s huge competition for attention, and most research is very limited in that regard
I would say that’s the biggest factor that makes the team “not indie”
At the same time, like you said, still amazing accomplishment, very cool model, and great work
These bidi/duplex demos always come off as overly artificial to me. It's cool that the model can speak at the same time as you instead of taking turns, but training it to interrupt as much as possible isn't the way to do it. In the very first example, did anyone here find the laugh after the "winter" answer to be convincing? What was the need for it? Why did the human speaker not get to finish a single sentence or finish a thought before the model took over with a monologue?
I'm absolutely going to hate talking to these things as customer service agents, with fake vocal fry and odd amounts of intonation and emotion peppered in.
For sure there's gonna be people falling in love with them though! I find all the anthropomorphization creepy though.
I've been hearing people talk negatively about vocal fry for ages now with no idea what it meant. But this comment was finally the one that made me go look it up. And today I learned that I speak almost entirely in vocal fry.
I listened after reading your comment, and in the first half of the example, Dessa sounds drunk to me, or at least tipsy. So weirdly, the laugh made sense to me.
Maybe it's better with lower latency, but I get really annoyed by this with ChatGPT Voice. I almost always start my chat with "I'm gonna ramble, I'll tell you when it's your turn".
OpenAI ruined advanced voice mode for me with that. I’m in the middle of a sentence and it goes “uh huh” and it kills my whole train of thought. It does not make me feel like I’m talking to a real person, it makes me feel like I’m talking to a stupid stupid AI that I hate.
These full-duplex models are getting very "Her"-ish lately. gpt-live-1 is fantastic. Dessa, though, seems to be aiming at full human parity. There's a huge market for AI girlfriends that they should be looking into in earnest...
human advancement is a function of population, instead of focusing on population decline to solve our resource problems we should focus on efficiency and technology. i am not rooting to revert to the dark ages i really struggle with this ideology of reduction that seems extremely popular. we really take for granted the fragility of human existence we exist in a pico second of reality and could and most likely will be snuffed out eventually, having our own species rooting for this outcome seems suicidal.
I'm a professional cynic and I spread the idea of "population decline to solve our resource problems" every time I have a chance; I even write dystopic science fiction with this theme. I do it not because I think it's the solution (yours is), but to jolt people in a reaction like yours. Let's build a bright future; just in our solar system there's easily enough matter and space for a trillion humans, each enjoying many acres of forest[^1].
No most of us are keen on developing human relationships and intimacy, but it sounds like you view women as a warm fleshlight so I hope that you and your genetic line wind up as dead ends with dry humping a robot as your greatest life accomplishment. Good job lil fella. Give that Claude session your 3.5 inches of American steel.
What do you think human relationships and intimacy entail and why can't some sort of high-tech robot legitimately emulate that?
>it sounds like you view women as a warm fleshlight
I think most people accept the fact that women and men are sexual objects. If you view me as a mysoginist for believing that, then I think I understand why you might think our society is still patriarchal or something.
In fact, I can't say that most straight men expect something cerebral or spiritual out of their partners. But even if it was the case, you could expect that a hyper-inteligent machine would exceed human capabilities in this regard.
The only types of people who aren't going to be humping robots are members of isolated societies who have a grudge against technology. Hence the amish.
There are an increasing number of loser men and women who have these AI partners. I think we may see a tipping point where "genetic dead ends" comprise the majority of the population.
With this level of sexual competition, who can blame them? If you're not at the apex you can put in the effort to reproduce but then your offspring are just faced by the same problem, one generation removed.
The majority of men born in this generation won't reproduce. For the average man, there is no sense in doing anything but devoting your life to some high-risk/high-reward strategy. I don't even know what that entails these days, but apparently it involves a lot of sports betting.
I'm normally endowed, as are like 90% of men (so I think these types of baseless insults fall flat). But if I wasn't, would you rather I pass on 3.5 inches of american steel to the next generation? Your insult stands regardless and reveals your underlying preference. If I was endowed well enough, I'm sure you would see my point.
It makes no difference personally, as a gay man who is uninterested in exploring robosexuality. I prefer to work my clankers.
There's no way I would be able to detect if this was AI from the examples alone. If I got a phone call from a malicious agent using this tech, I probably wouldn't realize it. My plan is to ask a challenge question about a shared memory with people I personally know, if there's something suspicious about a call. I don't know how else to prevent it. It sucks to be this suspicious, but I can't help it.
There's so many potential scenarios.
One scenario: An AI voice trainer trains on someone's social media video posts and get a pretty perfect copy. Then SIM jacks the target's phone number and calls pretending to need help. AIs could be used to mock ransom people, request bail money, ask for a change on a production server, etc.
People with high public visibility are likely targets, especially if they have money and or power. Impersonating them to subordinates would be an easy means of a social engineering attack.
"There's no way I would be able to detect if this was AI from the examples alone."
The hallucination in the second sample - that Bryan Singer was involved in Superman IV: The Quest for Peace - would have rung alarm bells. It just seemed to be free associating on words in the conversation.
Most recent voice-related tech is using voice as an interface or interaction mode. It’s a means to an and, and the end is the value provided by the reasoning system on the other side of the voice.
This model is clearly not so smart. So the voice is the whole point, there’s not much value beyond it, at least on its own.
What that adds up to is, impersonation. That’s what this is for, impersonating humans.
It sounds like it is trained on a lot of podcasts, judging by a quick few second skims over their demo tracks.
Also, the trend of posting on HN with much more sensationalist title first, getting to the front page, then getting the title adjusted by a moderator should probably be penalized; how is 'most authentic ever' not a marketing term? (not just this post, but a bunch of posts recently got to frontpage by making unreasonable claims)
It also seems like it trained on dialog in plays and novels. Some of these responses seem way too "perfect", like a response you would hear in a dramatic soap opera. Honestly it might be using that too. Just my take from listening to first minute or two of dialog.
Telephone conversations are quite different from in person speech. The lack of visual cues from the face, or the lag in video chat, the echo cancellation and the quantisation and comprehension, all reduce the quality subtly.
Now the Stasi archives, that would be interesting. Sometimes people suspected they were being surveilled of course.
The voice seems insanely immersive and uncanny. One issue that the full duplex models always have is tool calling. This is where GPT-Live [1] shines in the sense that you can use it as a fully practical agent. Looking forward to seeing how Dessa folks implement this.
Very technically impressive, but the fake conversation with the machine was cringe worthy to me; this pretending to care doesn't make me want to converse at all. The goal for me is to have an intelligent conversation that _adds_ to my experience; whatever I'm working on at the time. The AI girlfriend experience makes me cry inside rather than rejoice.
> LLMs need an insane amount of text. Dessa learned language from hundreds of times fewer words (in fairness, LLMs also have to learn C++ and whatever the hell Angular is).
Now that a larger proportion of text is AI, I'm appreciating human-written text more and more; this got a good chuckle out of me.
I've definitely seen LLMs compose jokes with this structure a lot where "whatever the hell ___ is" is in parenthesis. Not saying this is AI generated tho
I have a “style guide” markdown that is pretty expansive
I run an RLHF style loop with me in the middle for all my writing now where I give a voice rant that is over expressive (aka redundant samples of concepts that I want to express clearly) in order to manage lossy compression steps
Then I will manually clean up that last 5% after a half dozen rounds
That sequence goes into the style guide to be referenced as a reasoning chain of thought to map rant > output
It’s been three years of tuning I’d suspect at this point
I've recently been using gemini argon as a voice judge. so it goes claude ( with style guide) -> voice judge with argon -> claude adjusts. I do that in a few loops before it hard stops if it can't reconcile, and pushes to me for review.
Agreed, it’s incredible and scary how natural and human it sounds. It’s gonna be pretty much impossible for almost anyone to tell if they’re talking to AI
Yeah. If someone talked to me like that, I’d really want to slap them. I hope no one gets the bright idea of building a customer support bot around this tech.
Is there a way for me to test it online? Also does it support other languages beyond English? I'm greek and there aren't many good solutions for us ...
> we figured out how to train Dessa on only one NVIDIA GB200 NVL72.
This is the crazy part. It seems pretty smart, and LLMs require 1000x more compute to train. Can this kind of efficiency somehow be achieved with text too?
Not if you want the same model that debugs your C++ to also be able to sort objects by shape and generate a description of a video and pick out a good birthday gift for your cousin
"Oh, uh, yeah, pointers and stacks and stuff, uh – lemme spin up Minimax. Just a sec. Ohhh – ok, see where you're casting that float to a double there, uh, line 72..."
Passing a Turing test over a 5 minute conversation:
- Text interface: Done
- Voice interface: 5-10 years I’ll predict
The demo is nice, but enough obstacles remain that very few people would mistake it for a human over even a 5 min conversation. Chat systems configured to intentionally impersonate can already get people to mistake them for humans most of the time.
There’s been great progress in this area, direct response modeling, parallel reasoning, but last challenges are huge.
The reality is LLMs still seem dumb much more often without significant reasoning time (test time inference). On that point alone they are still an order of magnitude too slow for convincing interactive latency.
Of course they don’t need to meet this bar to be useful, systems are good enough today for lots of applications. It will just be interesting to see what the final delta ends up being.
So, I tolerate the "ums" and "ahs" and "uhs" in conversation from humans because they're human. If an AI is going to speak to me (gods forbid), it better be concise and clear and focused, or I'm throwing it out the fucking window.
It is interesting that they've done this reasonably successfully, impressive technology. Novel ideas can have inherent value, absent direct uses they may have, as they can be a stepping stone toward other inventions. But, I'm having a hard time coming up with a use case that isn't predatory or intentionally misleading. Company support agents pretending to be human, for instance. Or, taking the next step on AI girlfriend/boyfriend type interactions. Or, outright scams. Models already do an excellent job with voice cloning, at least for intonation and timbre, adding realistic conversational tics makes it even more likely grandma is going to fall for it when her "nephew" calls needing help.
But, that seems to be extremely common with new tech these days. There's always an angle that is unsavory. Marketing, scams, propaganda, surveillance, killing, that's where the money is, so everything bends toward them.
I also don't get the need for all these imperfections. In some examples it sounds like "she" is half asleep or even tipsy. If I want to talk to an all powered machine that knows my codebase inside out, or that can discuss about technical topics like the universe, science, or a specific technical book, etc... I expect it to be an eloquent speaker, capable of a good cadence, and that's easy to understand. I don't need it to mimic all the imperfections, I want it to be like when I'm talking to a really smart person that keeps me engaged and makes me feel smart.
Especially when you keep in mind that not everyone knows english as their first language. I know he's a bit controversial sometimes but Neil deGrasse Tyson is easy to understand and explained interesting topics in his podcast (I could still understand most of it when I was still getting used to listening to english 15+ years ago).
If conversation is the new interface then we will need UX designers to take care of this.
I came here to say the same thing. I don't want a computer constantly inserting stop words, pauses, laughing unnecessarily and interrupting me. This seems technically impressive but is not something I'd want to interact with.
I know someone who recently came across a scammer where a guy impersonated a sheriff's deputy and tried to get money out of her. She told me the guy sounded like a real sheriff and made her believe the guy until he asked her to pay some money in Bitcoin. I can imagine that this type of scam could go viral soon. Even worse, someone could record our voice and fine-tune a model like this so that it can produce our voice and then talk to our family members to deceive them. It'd be a real trouble.
Already had one call from a scammer AI voice that was pretending to be the official billing dept. of my phone company's larger carrier. Was pretty realistic and beleivable as a human voice, but the very moment i hesitated when asked to provide information.. "you should have that information" i started to say but the agent cut the call at the point between my first and second syllable. The caller ID was already suspect, and when I checked that number it is associated to a sex line. But the voice was polite and official sounding...it wasn't the voice that twigged my suspiciousness. Granny is going to have to start requiring everything via registered mail to not get scammed.
Totally agree. Those scam calls calling as IRS agent, or outsourced criminal shops that try to steal identity, now this makes it much easier. No more foreign accent. They can hyper optimize to sound like a person from that specific locality.
This feels like a parlor trick where the AI was trained to mimic informal telephone mannerisms. The actual conversations were pretty bad. Lots of style, not much substance basically.
Really impressive demo. I have to say that, but I do worry with this kind of fidelity to a human interaction that people will get more drawn into AI relationships and replace human contact.
I like voice as a mode of interaction with AI, but it’s always been clear with current tech that I’m talking to AI (and yet folks still got drawn into fantasy relationships thanks to sycophantic models etc) - I imagine with voice models like Dessa that could be worse for humans. This kind of fidelity blurs the lines of who or what the human is talking to and lets humans fall into an isolated world where they believe they are “socializing.”
Now, we could argue this is not a negative, maybe it is actually healthy for those who are looking for social connection or lonely, but my gut feel is that is not the case for most. It worries me that this kind of model will actually cause further deterioration of human socialisation and human-to-human contact.
I actually thought it sounds really good and natural and it freaked me out in a good way.
You can tell that the robots have no idea of what they're talking about, but the voicing, the emotions, etc, really good.. Like talking to a friend on the phone.
Also creepy, because of the potential for abuse of this technology.
But the interesting part to me is that training on voice only is what allows models to learn emotions - voice transmits a lot of emotional information which text doesn't capture. Language has words for a very limited number of emotions - maybe only the peaks - but there's so many more which we can't describe or transmit via text.
You could determine the phonetic tokens which express different kinds of emotions or 'render' text+emotion using a DSL, etc. Exciting stuff.
If you actually want your "AI" to interact with anything digital, are these voice-to-voice models going to...use speech to text? How does this model interact with anything that isn't an audio channel?
Interesting tech, but if these are the hand-picked examples that best represent it, I suppose we're not quite there yet.
The intelligence sounds lacking. At this point, anyway.
But the idea of learning through voice instead of written text might prove fruitful. After all, text isn't strictly native to humans. It's just a tool we've invented.
Not bad, but I guess I lack the imagination to envision the use cases for such casual conversation. Other than... I suppose a certain demographic would be lining up to use this as the voice for their AI girlfriend.
Also I seem to be missing where/how they are selling this.
Can you instruction-train a duplex model like this? How would that look? I also wonder, if a model trained only on speech can develop world knowledge like this, could you train an image model only on images of text to essentially act as an LLM?
Both speakers in the examples sound unnatural. The human doesn't know what to talk about and seems to have little confidence. The robot says things that only mostly make sense and draws on, saying ultimately very little. "How's the winter stuff?" e.g. is not a sentence a person would say. I might say "How are you dealing with the snow?" or "Are you looking forward to winter?," but not "How's the winter stuff?" And there's just so many examples like this. Also, I'm not convinced that this "human-like" conversation with filler words and interruptions and pauses is actually what anyone wants, especially in absence of a face to interpret or a real human being to understand behind it. Even if it worked perfectly, it would feel uncanny and artificial because I know the person isn't real.
They sound just as unnatural als the annoying actors in the 'Gilmore Girls' series my daughter keeps on watching and re-watching, i.e. stupid valley girl or whatever it is called witless drivel. In that sense this model is tailor-made to create direct script-to-screen voice-overs for pabulum like that.
ya if this is real this is magnitudes better than anything else out there, it honestly feels like it can't be real because its too real, i would be 100% fooled and never even question it wasn't a real person, i was more worried because of that, in that no one would be able to tell and you could have very engaging / emotional conversations that chatgpt/grok/claude etc can't even come close to
It sounds good, but...what the hell is it talking about?
The "Huge range in intonation and speed." example. "I had another ollie special". "I ordered a French vanilla latte, special from Maude [or is that Aldi?], springs and waffles, granny smith"...wtf does any of this mean?
She takes a sip of something and her "mouth is on fire...from all the sugar". She repeatedly calls herself "a ding".
reminds me of the last year hype around SesameAILabs/csm, people losing their mind over how realistic it was, only for them to disappear and never ship. sorry can't get hyped over some mp3s.
They're all building the "Her" companion device. It's almost like those transparent touchscreen phones you'd see in SciFi movies that every phone maker would have their research Labs working towards, Nokia morph was one of them.
Those things all look this in concept and in movies, but their practical utility in real life always seems to be lacking. I'd imagine this is the reality what everyone building companion devices are facing.
With the amount of money, RnD and engineers these firms have this seems to be the only reason.
This would be amazing for making audiobooks more expressive and like real storytelling. I kept telling Audible they needed to invest more into premium audio experiences, kind of the way you had the old radio plays that people used to tune into. The War of the Worlds! We are seeing some of that with full casts of actors doing readings more.
At the same time, imagine if every book was like a living thing you could interact with and buying the book partially bought you access to the model that would tell you the story. The only way you would interact with that particular model, is if you bought the book.
It reads the book to you and you could chime in or it would know when to pause if there's an interruption without you needing to manually pause. You say hey wait, why is this character so angry? Then it kind of naturally reacts and continues the story. Maybe you come back to it 3 years later and it remembers you liked a certain part of the book and injects "here comes the good part!" It hears you start snoring and begins lowering its voice and whispering, but remembers you asked it to wake you up loudly if you start snoring, so it bolts out HEYWAKEUP!
For a lot of people books are a very solitary thing, though of course there are social clubs and book readings. Almost makes me think of those magical books you see fantasy wizards having, where the books themselves are sentient. It would be crazy if they released a The Princess Bride book and had the real narrator's voice and you could interrupt him and he could harass you about all your interruptions.
It wasn't that long ago where we were talking about having games and movies blend so the movies become interactive experiences. We're really getting closer to some of these static mediums becoming more like, trained behavioral meshes that have a personality and you can kind of naturally spend time with it. I'm not sure I'd be big on paintings on the wall coming to life and talking to you like in Harry Potter, but there's some promise.
Game worlds are an obvious extension, not just through NPCs, but if anyone remembers that game called Bastion where there was a narrator explaining the events as they went on. You could imagine a model that could watch you play and speak naturally, being funny or narrating. Almost something like Mystery Science Theater 3000 or how Beavis and Butthead would react to music videos.
I don't think those should be pervasive or the new norm, but there probably is an opportunity there and a market for it. Some kind of social-like presence that doesn't get tired, doesn't get bored, always has the same schedule you do, etc.
Hell, that reminds me of Seaman on the Dreamcast, too. SEGA could make a new Seaman and it would actually be awesome. Like a blend between Nintendogs, a fish, and some AI. There could even be charming goofy antics it tries to pull where it knows it's stuck inside a game and tries ridiculous things to manipulate you into letting it out. Some kind of Pinky and the Brain charm.
Call me a skeptic, obviously the voice sounds plausible and very realistic and human-level but where does the intelligence come from if it's only trained on human speech datasets? LLMs are powerful because they can do anything, if you only have a voice corpus I doubt it is as powerful as all written human knowledge. Like when it says "My day was good", what does it actually mean? Or is it just generating plausible slop.
It is cool but since there is no way to confirm ie. run it, or see it working somewhere, it can be also not real.
I get you are excited to share, but taking into account what your product is, it would be more powerful to let people experience it. As a creator of digital products, why would you put so much text on the homepage, anyhow you get the point.
The speech is great, but you can see some cracks even in the first sound bit. near the end he asks the model to speak louder and initially the voice is louder but then goes back to the same level, also losing what she was saying just seconds ago and he has to lead back to the topic.
I'm surprised it took this long! Clearly the next step is joint training on text and audio. The speech realism is very good, but the intelligence is pretty clearly 2024-level.
Sorry I'm lost. The sample on the homepage sounds absolutely awful. Not one bit of the cadence individually and together, the tonal ranges in responses or the overall structure and feel is comfortable to listen to, it's so far off passing as 'real' that I feel like I must be missing something.
>That doesn't sound like the same bar as "most authentic full-duplex speech model ever built"
It's up there. I think most humans would be convinced they're talking to a human. I assumed you've used other speech models. They are okay, but not as natural and imperfect as Dessa is.
It obv does have an extremely natural humane sound to it, but I feel like if you pay attention to what it actually says there is a noticeable lack of intelligence behind it.
It honestly sounds a bit like a teenager's stream of consciousness to me, that's how they sound to me anyways.
Yes, it's mostly speaking/reacting for speaking sake. A cynic might say that it's ready for deployment among the masses.
15-20 sec into any of the demo recordings and I get the sense of 'noone is really there' from the AI responses. I guess it's to be expected from a v1 where the scope is prioritising how the demo conversation flows rather than its actual substance.
My second thought was it sounds like the colorless green ideas have slept furiously, so to speak.
My first thought was: omg, they've harvested the souls of a dozen insipid, self-centered genz' and trapped them in a machine, like The Finn living in an Aleph...
Extremely obvious in the conversation between this model and GPT-Live-1.
The conversation examples are impressive but this made me say holy shit out loud: "it runs locally on a MacBook Pro M4 Max or better. And we figured out how to train Dessa on only one NVIDIA GB200 NVL72."
If it really can replicate that low latency + that expressiveness on local devices, that's one heck of an accomplishment for "indie" AI.
Speech is in quite a weird spot when it comes to locally running it. Some of the best STT and TTS models run at faster-than-realtime speed on decent consumer hardware with quite small models (so also no huge RAM requirements), but the tooling and library ecosystem is much more fragmented, making it a lot harder to build actual working products around it. There is also a much more noticeable coverage / quality gap when it comes to languages other than English.
As a speech-to-speech model Dessa is of course in a different ballpark, but given the intelligence it exhibits, it seems to be working with not that many parameters. The developers also seem to be admitting that to some degree ("in fairness, LLMs also have to learn C++ and whatever the hell Angular is"). I'm wondering what the utility of this would be. Given that it lacks both broad world knowledge and tool calling integration in any way, that seems kind of like a dead-end.
> that's one heck of an accomplishment for "indie" AI.
It's a nice accomplishment, but I think we have very different definitions of "indie".
https://en.wikipedia.org/wiki/Block,_Inc.#
> one NVIDIA GB200 NVL72
That’s around ~$4-6 million in hardware cost (one rack, 72 GPUs + accompanying hardware).
Many AI shops are funded well into tens of millions if not hundreds, for early rounds.
Also the team size is really small (< 10???). So yes, indie might be a stretch but still very impressive, relatively speaking.
Now taking the other side, they didn’t have to worry about hosting/billing/subscriptions or any of that yet. It’s a research demo at this point.
> > one NVIDIA GB200 NVL72 That’s around ~$4-6 million in hardware cost (one rack, 72 GPUs + accompanying hardware).
that’s a good clarification, skimming some of the comments here I was thinking they trained it on a single GB200 gpu. THAT would be a huge win for indie AI
Edit: Nvidia GB200 does not refer to a single gpu, Nvidia’s naming scheme is very confusing. Considering past gpus were called H100, H200 etc, I thought GB200 would just be a newer GPU, but it does actually refer to their 30+ gpu rack scale systems. The claim of “we trained this on a single rack of gpus” is definitely much less impressive than “we trained this on a single gpu”
> Nvidia GB200 does not refer to a single gpu, Nvidia’s naming scheme is very confusing. Considering past gpus were called H100, H200 etc, I thought GB200 would just be a newer GPU, but it does actually refer to their 30+ gpu rack scale systems.
It's even more confusing than that :)
Think of GB200 as something like Apple's 'Ultra' chip. Not exactly, but similar. They fuse the chips together with a very fast interconnect (connection).Also a speech to speech model it can't do any tool use or you can't guide it much am I mistaken? How will it be used for agentic uses is the part I don't get. Edit: this is response from Argon if anyone was also curious: You are not mistaken about Dessa: because it was trained solely on raw audio with zero text data, it cannot read text system prompts or output code for tool calls on its own. To use it as an agent, developers must either steer it purely through spoken voice instructions or pair it as a conversational "voice front-end" alongside a separate text LLM that listens to the audio and executes the tools in the background.
> No, but I speak the language of binary load lifters, which is very similar to your moisture vaporators
> they didn’t have to worry about hosting/billing/subscriptions or any of that yet
Or marketing, which they kinda get for free. So much research goes unseen, there’s huge competition for attention, and most research is very limited in that regard
I would say that’s the biggest factor that makes the team “not indie”
At the same time, like you said, still amazing accomplishment, very cool model, and great work
Wait, seriously, a rack of 72 GPUs costs $4 million?
Hard to find concrete pricing [0], but apparently a pair can cost as much as $8.8M (depending on config)[1]
[0]: https://www.nvidia.com/en-us/data-center/gb200-nvl72/
[1]: https://www.tomshardware.com/tech-industry/artificial-intell...
This rack has 14 TB of ultra-fast VRAM.
oh, hacker news, where someone will act like a ten person shop with a seven million dollar computer might be indie
Perhaps boutique is a better term?
> it runs locally on a MacBook Pro M4 Max
Open weights or GTFO
I mean cartesia did one of their first demos on a Mac book pro, live.
Still extremely impressive but not unheard of.
Wetware can't stop getting mogged.
By 2030 we'll have this running on old pentiums.
These bidi/duplex demos always come off as overly artificial to me. It's cool that the model can speak at the same time as you instead of taking turns, but training it to interrupt as much as possible isn't the way to do it. In the very first example, did anyone here find the laugh after the "winter" answer to be convincing? What was the need for it? Why did the human speaker not get to finish a single sentence or finish a thought before the model took over with a monologue?
I'm absolutely going to hate talking to these things as customer service agents, with fake vocal fry and odd amounts of intonation and emotion peppered in.
For sure there's gonna be people falling in love with them though! I find all the anthropomorphization creepy though.
I've been hearing people talk negatively about vocal fry for ages now with no idea what it meant. But this comment was finally the one that made me go look it up. And today I learned that I speak almost entirely in vocal fry.
I get irrationally angry when actual people interrupt me while I'm talking, I'll start throwing chairs when AI tech support agents start doing it.
Just make sure the agent on the call on your behalf interrupts more.
I listened after reading your comment, and in the first half of the example, Dessa sounds drunk to me, or at least tipsy. So weirdly, the laugh made sense to me.
Maybe it's better with lower latency, but I get really annoyed by this with ChatGPT Voice. I almost always start my chat with "I'm gonna ramble, I'll tell you when it's your turn".
Likewise; I feel exhausted after trying it for a short time, because it doesn't give any space.
OpenAI ruined advanced voice mode for me with that. I’m in the middle of a sentence and it goes “uh huh” and it kills my whole train of thought. It does not make me feel like I’m talking to a real person, it makes me feel like I’m talking to a stupid stupid AI that I hate.
These full-duplex models are getting very "Her"-ish lately. gpt-live-1 is fantastic. Dessa, though, seems to be aiming at full human parity. There's a huge market for AI girlfriends that they should be looking into in earnest...
Global population collapse imminent.
This is why we never heard from the aliens. Invented AI, retired to their Goonspheres.
What else did you think all the Fallen Empires in Stellaris did ? ;-)
Some say they’re still in there to this day.
Agreed:) no one’s gonna mate and continue the our human lineage, haha
Also with people having fewer ties to the rest of humanity, there will probably be some weird effects on morality
Agreed, and probably has already started with the high usage of the internet and YouTube
There already are. Look how money creates the Epstein class.
Some will, and they will outcompete.
Please let this come sooner...
human advancement is a function of population, instead of focusing on population decline to solve our resource problems we should focus on efficiency and technology. i am not rooting to revert to the dark ages i really struggle with this ideology of reduction that seems extremely popular. we really take for granted the fragility of human existence we exist in a pico second of reality and could and most likely will be snuffed out eventually, having our own species rooting for this outcome seems suicidal.
Cheers and well said :-)
I'm a professional cynic and I spread the idea of "population decline to solve our resource problems" every time I have a chance; I even write dystopic science fiction with this theme. I do it not because I think it's the solution (yours is), but to jolt people in a reaction like yours. Let's build a bright future; just in our solar system there's easily enough matter and space for a trillion humans, each enjoying many acres of forest[^1].
[^2]: https://claude.ai/share/bbab0507-109d-4b4b-b5bb-2a44afad901a
> suicidal
Would the logical opposite be eternity for all, and bring back the deceased? LLM plays the Joker, happily imprisoned?
oh no, I am long past any delusions that any problems will ever be solved, I just want it all to end
So homicidal then.
Honestly I think that people who want to fuck a clanker not passing their genes on is probably for the best.
so pretty much anyone who isn't amish?
In the future you won't be able to tell.
No most of us are keen on developing human relationships and intimacy, but it sounds like you view women as a warm fleshlight so I hope that you and your genetic line wind up as dead ends with dry humping a robot as your greatest life accomplishment. Good job lil fella. Give that Claude session your 3.5 inches of American steel.
What do you think human relationships and intimacy entail and why can't some sort of high-tech robot legitimately emulate that?
>it sounds like you view women as a warm fleshlight
I think most people accept the fact that women and men are sexual objects. If you view me as a mysoginist for believing that, then I think I understand why you might think our society is still patriarchal or something.
In fact, I can't say that most straight men expect something cerebral or spiritual out of their partners. But even if it was the case, you could expect that a hyper-inteligent machine would exceed human capabilities in this regard.
The only types of people who aren't going to be humping robots are members of isolated societies who have a grudge against technology. Hence the amish.
There are an increasing number of loser men and women who have these AI partners. I think we may see a tipping point where "genetic dead ends" comprise the majority of the population.
With this level of sexual competition, who can blame them? If you're not at the apex you can put in the effort to reproduce but then your offspring are just faced by the same problem, one generation removed.
The majority of men born in this generation won't reproduce. For the average man, there is no sense in doing anything but devoting your life to some high-risk/high-reward strategy. I don't even know what that entails these days, but apparently it involves a lot of sports betting.
I'm normally endowed, as are like 90% of men (so I think these types of baseless insults fall flat). But if I wasn't, would you rather I pass on 3.5 inches of american steel to the next generation? Your insult stands regardless and reveals your underlying preference. If I was endowed well enough, I'm sure you would see my point.
It makes no difference personally, as a gay man who is uninterested in exploring robosexuality. I prefer to work my clankers.
does it speak Japanese, tho? What exactly is the point of a waifu if you understand it?
they are, I believe the third clip is dystopian AI girlfriend
There's no way I would be able to detect if this was AI from the examples alone. If I got a phone call from a malicious agent using this tech, I probably wouldn't realize it. My plan is to ask a challenge question about a shared memory with people I personally know, if there's something suspicious about a call. I don't know how else to prevent it. It sucks to be this suspicious, but I can't help it.
There's so many potential scenarios. One scenario: An AI voice trainer trains on someone's social media video posts and get a pretty perfect copy. Then SIM jacks the target's phone number and calls pretending to need help. AIs could be used to mock ransom people, request bail money, ask for a change on a production server, etc.
People with high public visibility are likely targets, especially if they have money and or power. Impersonating them to subordinates would be an easy means of a social engineering attack.
"There's no way I would be able to detect if this was AI from the examples alone."
The hallucination in the second sample - that Bryan Singer was involved in Superman IV: The Quest for Peace - would have rung alarm bells. It just seemed to be free associating on words in the conversation.
Humans never get details of movie credits wrong?
Most recent voice-related tech is using voice as an interface or interaction mode. It’s a means to an and, and the end is the value provided by the reasoning system on the other side of the voice.
This model is clearly not so smart. So the voice is the whole point, there’s not much value beyond it, at least on its own.
What that adds up to is, impersonation. That’s what this is for, impersonating humans.
Doesn't have be all the same model doing all of the things at the same speed. It can do system 1 system 2 type of thing.
all “artificial” intelligence is impersonation.
All your comments are reposts. Do better than an Ai maybe?
I thought it sounded a little stilted and weird and then I realized that was the human half of the conversation.
The conversation with the AI fake wife was a little creepy, albeit honest about what this tech is probably going to be used for.
It sounds like it is trained on a lot of podcasts, judging by a quick few second skims over their demo tracks.
Also, the trend of posting on HN with much more sensationalist title first, getting to the front page, then getting the title adjusted by a moderator should probably be penalized; how is 'most authentic ever' not a marketing term? (not just this post, but a bunch of posts recently got to frontpage by making unreasonable claims)
It also seems like it trained on dialog in plays and novels. Some of these responses seem way too "perfect", like a response you would hear in a dramatic soap opera. Honestly it might be using that too. Just my take from listening to first minute or two of dialog.
The NSA has a much better training set.
Telephone conversations are quite different from in person speech. The lack of visual cues from the face, or the lag in video chat, the echo cancellation and the quantisation and comprehension, all reduce the quality subtly.
Now the Stasi archives, that would be interesting. Sometimes people suspected they were being surveilled of course.
Likely audiobooks too, which certainly wouldn’t help the dramatic tone.
This is why we have a flag.
The voice seems insanely immersive and uncanny. One issue that the full duplex models always have is tool calling. This is where GPT-Live [1] shines in the sense that you can use it as a fully practical agent. Looking forward to seeing how Dessa folks implement this.
[1] https://openai.com/index/introducing-gpt-live/
Very technically impressive, but the fake conversation with the machine was cringe worthy to me; this pretending to care doesn't make me want to converse at all. The goal for me is to have an intelligent conversation that _adds_ to my experience; whatever I'm working on at the time. The AI girlfriend experience makes me cry inside rather than rejoice.
> LLMs need an insane amount of text. Dessa learned language from hundreds of times fewer words (in fairness, LLMs also have to learn C++ and whatever the hell Angular is).
Now that a larger proportion of text is AI, I'm appreciating human-written text more and more; this got a good chuckle out of me.
>and whatever the hell Angular is
I've definitely seen LLMs compose jokes with this structure a lot where "whatever the hell ___ is" is in parenthesis. Not saying this is AI generated tho
I wouldn’t be so sure that that’s human text
I can reliably get my custom models to output a 95% perfect replication of my tone and voice
As somebody who attempted that recently, I seriously doubt that percentage :D
Maybe you just have a voice that's, shall we say, in sparse latent space? ;)
New insult dropped!
I have an insane amount of writing output as training data going back a loooong time so it all comes down to how much training data do you have
Many of us can do it on just our hacker news comments (been there, done that).
What do you mean by your custom models? Do you mean custom prompting?
Guessing - finetuned? Or LoRA?
EDIT: Nevermind, see below. I would definitely go with a LoRA though.
That’s if I was going to build my own model on myself, which I’m not planning on doing at this point
Couple things:
I have a “style guide” markdown that is pretty expansive
I run an RLHF style loop with me in the middle for all my writing now where I give a voice rant that is over expressive (aka redundant samples of concepts that I want to express clearly) in order to manage lossy compression steps
Then I will manually clean up that last 5% after a half dozen rounds
That sequence goes into the style guide to be referenced as a reasoning chain of thought to map rant > output
It’s been three years of tuning I’d suspect at this point
How did you train your style guide, though?
I have a somewhat expansive style guide, and it still reeks of Claude.
My cleanup is much closer to 40%, which still makes it worth the effort, but sometimes when speed matters, I'll apologize for the slop.
I've recently been using gemini argon as a voice judge. so it goes claude ( with style guide) -> voice judge with argon -> claude adjusts. I do that in a few loops before it hard stops if it can't reconcile, and pushes to me for review.
I was expecting to run into some uncanny valley-esque. It was surprisingly good.
It is kinda scary that we're here already, I can see this being pretty problematic for seniors/public prone to getting scammed.
Agreed, it’s incredible and scary how natural and human it sounds. It’s gonna be pretty much impossible for almost anyone to tell if they’re talking to AI
Sounds like they trained it on Boxxy or Kat from Sam and Kat. I guess it does sound like a human talking but in an affected unnatural way.
Yeah. If someone talked to me like that, I’d really want to slap them. I hope no one gets the bright idea of building a customer support bot around this tech.
At least a dozen startups are working on it
Is there a way for me to test it online? Also does it support other languages beyond English? I'm greek and there aren't many good solutions for us ...
> we figured out how to train Dessa on only one NVIDIA GB200 NVL72.
This is the crazy part. It seems pretty smart, and LLMs require 1000x more compute to train. Can this kind of efficiency somehow be achieved with text too?
Not if you want the same model that debugs your C++ to also be able to sort objects by shape and generate a description of a video and pick out a good birthday gift for your cousin
"Oh, uh, yeah, pointers and stacks and stuff, uh – lemme spin up Minimax. Just a sec. Ohhh – ok, see where you're casting that float to a double there, uh, line 72..."
Passing a Turing test over a 5 minute conversation:
- Text interface: Done
- Voice interface: 5-10 years I’ll predict
The demo is nice, but enough obstacles remain that very few people would mistake it for a human over even a 5 min conversation. Chat systems configured to intentionally impersonate can already get people to mistake them for humans most of the time.
There’s been great progress in this area, direct response modeling, parallel reasoning, but last challenges are huge.
The reality is LLMs still seem dumb much more often without significant reasoning time (test time inference). On that point alone they are still an order of magnitude too slow for convincing interactive latency.
Of course they don’t need to meet this bar to be useful, systems are good enough today for lots of applications. It will just be interesting to see what the final delta ends up being.
You are severely underestimating how stupid/gullible the average person is.
Do you think that, much like how OpenAI scaled up GPT-2 to GPT-3, someone else can train a larger model like Dessa on, say, multiple GB200s?
So, I tolerate the "ums" and "ahs" and "uhs" in conversation from humans because they're human. If an AI is going to speak to me (gods forbid), it better be concise and clear and focused, or I'm throwing it out the fucking window.
It is interesting that they've done this reasonably successfully, impressive technology. Novel ideas can have inherent value, absent direct uses they may have, as they can be a stepping stone toward other inventions. But, I'm having a hard time coming up with a use case that isn't predatory or intentionally misleading. Company support agents pretending to be human, for instance. Or, taking the next step on AI girlfriend/boyfriend type interactions. Or, outright scams. Models already do an excellent job with voice cloning, at least for intonation and timbre, adding realistic conversational tics makes it even more likely grandma is going to fall for it when her "nephew" calls needing help.
But, that seems to be extremely common with new tech these days. There's always an angle that is unsavory. Marketing, scams, propaganda, surveillance, killing, that's where the money is, so everything bends toward them.
I also don't get the need for all these imperfections. In some examples it sounds like "she" is half asleep or even tipsy. If I want to talk to an all powered machine that knows my codebase inside out, or that can discuss about technical topics like the universe, science, or a specific technical book, etc... I expect it to be an eloquent speaker, capable of a good cadence, and that's easy to understand. I don't need it to mimic all the imperfections, I want it to be like when I'm talking to a really smart person that keeps me engaged and makes me feel smart.
Especially when you keep in mind that not everyone knows english as their first language. I know he's a bit controversial sometimes but Neil deGrasse Tyson is easy to understand and explained interesting topics in his podcast (I could still understand most of it when I was still getting used to listening to english 15+ years ago).
If conversation is the new interface then we will need UX designers to take care of this.
I came here to say the same thing. I don't want a computer constantly inserting stop words, pauses, laughing unnecessarily and interrupting me. This seems technically impressive but is not something I'd want to interact with.
I know someone who recently came across a scammer where a guy impersonated a sheriff's deputy and tried to get money out of her. She told me the guy sounded like a real sheriff and made her believe the guy until he asked her to pay some money in Bitcoin. I can imagine that this type of scam could go viral soon. Even worse, someone could record our voice and fine-tune a model like this so that it can produce our voice and then talk to our family members to deceive them. It'd be a real trouble.
Already had one call from a scammer AI voice that was pretending to be the official billing dept. of my phone company's larger carrier. Was pretty realistic and beleivable as a human voice, but the very moment i hesitated when asked to provide information.. "you should have that information" i started to say but the agent cut the call at the point between my first and second syllable. The caller ID was already suspect, and when I checked that number it is associated to a sex line. But the voice was polite and official sounding...it wasn't the voice that twigged my suspiciousness. Granny is going to have to start requiring everything via registered mail to not get scammed.
Totally agree. Those scam calls calling as IRS agent, or outsourced criminal shops that try to steal identity, now this makes it much easier. No more foreign accent. They can hyper optimize to sound like a person from that specific locality.
Wondering what kind of tokenization they use after the model referred to the movie Memento as The Mento the first time before being corrected.
Open weights or it didn't happen.
This feels like a parlor trick where the AI was trained to mimic informal telephone mannerisms. The actual conversations were pretty bad. Lots of style, not much substance basically.
Really impressive demo. I have to say that, but I do worry with this kind of fidelity to a human interaction that people will get more drawn into AI relationships and replace human contact.
I like voice as a mode of interaction with AI, but it’s always been clear with current tech that I’m talking to AI (and yet folks still got drawn into fantasy relationships thanks to sycophantic models etc) - I imagine with voice models like Dessa that could be worse for humans. This kind of fidelity blurs the lines of who or what the human is talking to and lets humans fall into an isolated world where they believe they are “socializing.”
Now, we could argue this is not a negative, maybe it is actually healthy for those who are looking for social connection or lonely, but my gut feel is that is not the case for most. It worries me that this kind of model will actually cause further deterioration of human socialisation and human-to-human contact.
Close enough, welcome back Google Duplex!
https://www.youtube.com/watch?v=D5VN56jQMWM
For real though, it's been bothering me, how did they do this in 2018? (Some kind of symbolic logic based AI?)
Also props to the people at Block, Dessa is very cool :)
I actually thought it sounds really good and natural and it freaked me out in a good way. You can tell that the robots have no idea of what they're talking about, but the voicing, the emotions, etc, really good.. Like talking to a friend on the phone. Also creepy, because of the potential for abuse of this technology.
But the interesting part to me is that training on voice only is what allows models to learn emotions - voice transmits a lot of emotional information which text doesn't capture. Language has words for a very limited number of emotions - maybe only the peaks - but there's so many more which we can't describe or transmit via text.
You could determine the phonetic tokens which express different kinds of emotions or 'render' text+emotion using a DSL, etc. Exciting stuff.
If you actually want your "AI" to interact with anything digital, are these voice-to-voice models going to...use speech to text? How does this model interact with anything that isn't an audio channel?
They can perform tool calls through text even though they're speech models, OpenAI showed this off with their latest ones I believe.
The real question is, does it actually understand sound?
Conventional LLMs are just running speech to text and then working on the text, so they can’t understand music or humming or any sort of sound.
Wanted to joke "AI girlfriend just dropped" but they explicitly have an AI girlfriend as the 3rd example.
Modern problems...
Interesting tech, but if these are the hand-picked examples that best represent it, I suppose we're not quite there yet.
The intelligence sounds lacking. At this point, anyway.
But the idea of learning through voice instead of written text might prove fruitful. After all, text isn't strictly native to humans. It's just a tool we've invented.
Not bad, but I guess I lack the imagination to envision the use cases for such casual conversation. Other than... I suppose a certain demographic would be lining up to use this as the voice for their AI girlfriend.
Also I seem to be missing where/how they are selling this.
Video game NPCs that can hold a plausible conversation while getting the plot points in could be amazing.
Can you instruction-train a duplex model like this? How would that look? I also wonder, if a model trained only on speech can develop world knowledge like this, could you train an image model only on images of text to essentially act as an LLM?
Both speakers in the examples sound unnatural. The human doesn't know what to talk about and seems to have little confidence. The robot says things that only mostly make sense and draws on, saying ultimately very little. "How's the winter stuff?" e.g. is not a sentence a person would say. I might say "How are you dealing with the snow?" or "Are you looking forward to winter?," but not "How's the winter stuff?" And there's just so many examples like this. Also, I'm not convinced that this "human-like" conversation with filler words and interruptions and pauses is actually what anyone wants, especially in absence of a face to interpret or a real human being to understand behind it. Even if it worked perfectly, it would feel uncanny and artificial because I know the person isn't real.
They sound just as unnatural als the annoying actors in the 'Gilmore Girls' series my daughter keeps on watching and re-watching, i.e. stupid valley girl or whatever it is called witless drivel. In that sense this model is tailor-made to create direct script-to-screen voice-overs for pabulum like that.
Why is every comment a brief "very impressive, nice job" on a high impact technology followed by a tangential semi-personal rant?
Combination of paid to comment and can't help themselves. The algo down-ranks, but it's ugly seeing how sausage is made.
ya if this is real this is magnitudes better than anything else out there, it honestly feels like it can't be real because its too real, i would be 100% fooled and never even question it wasn't a real person, i was more worried because of that, in that no one would be able to tell and you could have very engaging / emotional conversations that chatgpt/grok/claude etc can't even come close to
It sounds good, but...what the hell is it talking about?
The "Huge range in intonation and speed." example. "I had another ollie special". "I ordered a French vanilla latte, special from Maude [or is that Aldi?], springs and waffles, granny smith"...wtf does any of this mean?
She takes a sip of something and her "mouth is on fire...from all the sugar". She repeatedly calls herself "a ding".
Third example at 0:02:
>Dessa: Hey, baby.
Block is in the AI girlfriend market? Ew.
This is so awesome! How does it work with moving away from the mic, is it just random or does the conversational pattern lead it that way?
The volume on this seemed so low and wouldn't get better even on max on my MacBook Air M5. Hoping it isn't my ears lol
Ok. Sounds cool. Now what? Open source weights? An API somewhere? Available on Openrouter? Are they just testing?
Where is this going?
not at all creepy
Great. I wonder.. is this based on Mochi or PersonaPlex? Also, how big is the model, and do we use it?
Will run on "MacBook Pro M4 Max or better" https://x.com/jack/status/2113792333238161547
reminds me of the last year hype around SesameAILabs/csm, people losing their mind over how realistic it was, only for them to disappear and never ship. sorry can't get hyped over some mp3s.
They're all building the "Her" companion device. It's almost like those transparent touchscreen phones you'd see in SciFi movies that every phone maker would have their research Labs working towards, Nokia morph was one of them.
Those things all look this in concept and in movies, but their practical utility in real life always seems to be lacking. I'd imagine this is the reality what everyone building companion devices are facing.
With the amount of money, RnD and engineers these firms have this seems to be the only reason.
it only works for english now, right? right?
This would be amazing for making audiobooks more expressive and like real storytelling. I kept telling Audible they needed to invest more into premium audio experiences, kind of the way you had the old radio plays that people used to tune into. The War of the Worlds! We are seeing some of that with full casts of actors doing readings more.
At the same time, imagine if every book was like a living thing you could interact with and buying the book partially bought you access to the model that would tell you the story. The only way you would interact with that particular model, is if you bought the book.
It reads the book to you and you could chime in or it would know when to pause if there's an interruption without you needing to manually pause. You say hey wait, why is this character so angry? Then it kind of naturally reacts and continues the story. Maybe you come back to it 3 years later and it remembers you liked a certain part of the book and injects "here comes the good part!" It hears you start snoring and begins lowering its voice and whispering, but remembers you asked it to wake you up loudly if you start snoring, so it bolts out HEYWAKEUP!
For a lot of people books are a very solitary thing, though of course there are social clubs and book readings. Almost makes me think of those magical books you see fantasy wizards having, where the books themselves are sentient. It would be crazy if they released a The Princess Bride book and had the real narrator's voice and you could interrupt him and he could harass you about all your interruptions.
It wasn't that long ago where we were talking about having games and movies blend so the movies become interactive experiences. We're really getting closer to some of these static mediums becoming more like, trained behavioral meshes that have a personality and you can kind of naturally spend time with it. I'm not sure I'd be big on paintings on the wall coming to life and talking to you like in Harry Potter, but there's some promise.
Game worlds are an obvious extension, not just through NPCs, but if anyone remembers that game called Bastion where there was a narrator explaining the events as they went on. You could imagine a model that could watch you play and speak naturally, being funny or narrating. Almost something like Mystery Science Theater 3000 or how Beavis and Butthead would react to music videos.
I don't think those should be pervasive or the new norm, but there probably is an opportunity there and a market for it. Some kind of social-like presence that doesn't get tired, doesn't get bored, always has the same schedule you do, etc.
Hell, that reminds me of Seaman on the Dreamcast, too. SEGA could make a new Seaman and it would actually be awesome. Like a blend between Nintendogs, a fish, and some AI. There could even be charming goofy antics it tries to pull where it knows it's stuck inside a game and tries ridiculous things to manipulate you into letting it out. Some kind of Pinky and the Brain charm.
Anybody on board with this future?
But can it follow a simple customer support script?
Now that a robot can speak over people on Zoom calls for me, I’m finally free to write poems all day!
Call me a skeptic, obviously the voice sounds plausible and very realistic and human-level but where does the intelligence come from if it's only trained on human speech datasets? LLMs are powerful because they can do anything, if you only have a voice corpus I doubt it is as powerful as all written human knowledge. Like when it says "My day was good", what does it actually mean? Or is it just generating plausible slop.
Very impressive demo!
It is cool but since there is no way to confirm ie. run it, or see it working somewhere, it can be also not real.
I get you are excited to share, but taking into account what your product is, it would be more powerful to let people experience it. As a creator of digital products, why would you put so much text on the homepage, anyhow you get the point.
The speech is great, but you can see some cracks even in the first sound bit. near the end he asks the model to speak louder and initially the voice is louder but then goes back to the same level, also losing what she was saying just seconds ago and he has to lead back to the topic.
I'm surprised it took this long! Clearly the next step is joint training on text and audio. The speech realism is very good, but the intelligence is pretty clearly 2024-level.
Haven’t OpenAI models been trained on text + audio for a year or two now?
Those demos are, omg, like, the worst, donchya think?
crazy, if true!
Dessa is written in a swirly feminine font.
There's no male or neutral voice.
All examples are of white male software engineers talking to what I assume will be ultimately marketed as a 'girlfriend' bot (or used for love scams).
The single paragraph hand waving at the importance of 'safe AI empathy' doesn't dissuade me at all.
Sell your idea as what it is, yet another 'perfect' woman shaped object a hypothetical man can own and use how he likes.
The second example is a male.
Sorry I'm lost. The sample on the homepage sounds absolutely awful. Not one bit of the cadence individually and together, the tonal ranges in responses or the overall structure and feel is comfortable to listen to, it's so far off passing as 'real' that I feel like I must be missing something.
Sure it definitely had some strange moment but it’s impressive for a small training data set and low compute
That doesn't sound like the same bar as "most authentic full-duplex speech model ever built"
>That doesn't sound like the same bar as "most authentic full-duplex speech model ever built"
It's up there. I think most humans would be convinced they're talking to a human. I assumed you've used other speech models. They are okay, but not as natural and imperfect as Dessa is.
Not really, there are very obvious unnatural pauses that make me want to slap them instead of continuing to talk to them.
Makes me realize how bad being silent really is.
I think it sounds slightly "off" but its cadence certainly sounds way more like a human than other AI voices I've heard.
Have you tried the alternatives? This is clearly far far superior in terms of fluidity of conversation.
Authentic to what? That annoying clinger who loves interrupting you when you're trying to speak in turns?
Is having a duplex model constantly cutting in with uninteresting quips something people are looking for?