The other day someone sent me a video where the candidate was lip syncing (very badly) the entire interview while someone spoke the answers outside the camera.
I think this will be a security mess, get misused for bullying with even worse outcomes, and hopefully there will be legislation that will tame these kind of applications, regardless of how technology cool it might be.
Completely agree and the recent news about financial fraud perpetrated via video conferencing (as a component of the attack) validates your point.
On the other hand, if tools like these don’t get out into the wild and show the dark arts potential, then it remains a straw man argument against unchecked distribution and use cases.
Things are going to get ugly and it remains to be seen how much weight a check-box legal liability waiver will hold up.
It reminds me of that scene in Johnny Mnemonic where he's on the video call and the bad guy is using internet puppet fingers to impersonate someone. I think about this scene often and I feel like we are getting there. excited to try it when i have a gpu ;)
hmm not sure yet on the open source thing, but how do people normally verify downloadable software has no malware? I guess we could try to distribute it on like reputable distribution channels like the app store
Personally I do take at least a quick look over sources before deciding to trust any new app. I simply won't install apps that don't provide the option.
This used to be an ideological stance but increasingly recently it's the only pragmatic thing to do from a stance of security and safety. The playing field is increasingly hostile and if someone asks you to install their software on you machine and let it record your face and voice but refuse to show what it actually does, that is a red flag. Reasonable exceptions could include video games (which run on dedicated untrusted devices and IMO the IP aspect makes the closed-source stance more understandable there). On the other hand, this app is inherently sensitive and trusted because of its function. I don't see the reason why it needs to be closed-source.
Malware is commonly distrubuted in all app stores. I reported some obviously pretty bad stuff that is still up a year later on Play Store, for example. Google simply doesnt bother if the case is too messy.
> hmm not sure yet on the open source thing
You could start with just go source-available by sharing the source with your users without going full Open Source, if you want to take the time and think about what license to use.
Another thing to add: this is Linux-only and a large amount of Linux users will care about your product being free software or under an open source license for ethical reasons. Source available doesn't mean open source, and open source means your product's license protects distribution and modifications of the code to some extent. This extent is quite debated, but you should certainly read up on this and have a strong defence for why your product isn't open source either way. I can certainly see why you wouldn't want to, but make sure to think about it especially for Linux-only. Windows and Mac users are probably more amenable to proprietary software.
Meticulously with Ghidra I suppose. This trend of saying it’s to protect us is getting old. The only way to prevent malware is by making the source available for us to see for ourselves.
If this isn't monetized, is there any reason against opening its source? I personally would like to be able to disable the usage analytics and crash reports.
> If this isn't monetized, is there any reason against opening its source? I personally would like to be able to disable the usage analytics and crash reports.
Although I personally deplore it and try to stay away from software that requires it, or even opts me in, I nonetheless think that it's reasonable for a developer to impose telemetry as a requirement for people to make use of their freely available software.
The first thing I'd do if it were open source is spin up a PR to allow the telemetry to be opt-in. After two decades of being the product, it's hard to trust any data collection done by companies outside the EU.
GDPR is very clear about how you are not allowed to make data collection a requirement for use.
It is a straightforward set of rules written in simple language and it’s not very long either. It’s not necessary to rely on third party readings or interpretations of it. Just go ahead and read it yourself and you will be well equipped to apply and argue about it.
Wouldn't using this software constitute a crime if using it to "appear as literally anyone"?
IIRC, the have been news stories in the EU about people receiving prison sentences for creating deepfakes, although maybe it was related to adult material. But impersonation and defamation is likely covered similarly. I'd assume that all it takes is for single viewer to believe it, to legally qualify as an act of impersonation.
FATAL:setuid_sandbox_host.cc(158)] The SUID sandbox helper binary was found, but is not configured correctly. Rather than run without sandboxing I'm aborting now. You need to make sure that /tmp/.mount_phazr-wg9XvA/chrome-sandbox is owned by root and has mode 4755.
Trace/breakpoint trap (core dumped)
I haven't installed this yet, but does it require camera access? i.e. does it "transform" your own image to the target image while maintaining facial expression, pose, etc.? Based on the animations, I'd assume it doesn't use the camera since there are techniques that can lipsync from audio.
if its fast enough, I’m very curious if this will get me broader acceptance as a different race in interviews with my same skillset and will report back
If it does work better, I’m sure people will just say the market picked up as opposed to validating my life experience, but as long as I’m collecting bigger paychecks believe whatever you want
I know the culture has changed dramatically in the last ten years, so I'll take this as a sincere question coming from someone who missed out:
Back in the old days, before 2014, people used to make computers do things just for fun. We'd write code because writing code is enjoyable, and hack on projects just to see if we could. If it made something that other people wanted to use that was a bonus, but hacking and experimenting was an end in itself. (The same, incidentally, went for writing blogs and for making YouTube videos).
In the last ten years most of us have lost sight of that in favor of everything needing to have an "audience" or a "use case"—if there's no path to monetization then we struggle to see the point. But some of us still build things just because we can, so from time to time you'll see a project like this that hearkens back to the old school hacker spirit and has no point besides to see if we could.
Thanks for the sincere reply. However, I feel that you avoided the question. There are plenty of projects, actually an infinite variety of things that are fun to build. I make generative line art with a pen plotter. It’s great.
But why this? Why make a tool for anyone to impersonate another person on video calls? I apologize for the cynicism, I actually really hate that my first thought was that this is a tool for scams. But, it’s hard not to see how useful this would be for deceiving people.
The tools for scams are already built and in use, and they're way better than what I'm seeing here. The only example I see with a real photo is a static photo with a mouth wiggling and no other movement. The rest are cartoons, with a similar effect.
I get the concern, but squelching fun use cases isn't going to prevent people from building the bad ones or even slow them down.
I have avoided posting a how-to on YouTube, in part because tech like this could be used to impersonate me. By creating a fake persona with fake voice (this one doesn't do that) we can make those videos.
So the way to not be impersonated is to use the same tools to be a fake person!
People don't have to like what you like and they don't need to give you a justification either. Your tone is needlessly antagonistic and dismissive. Not to mention patronising.
The other day someone sent me a video where the candidate was lip syncing (very badly) the entire interview while someone spoke the answers outside the camera.
A tool like this would be very handy for him.
Could you post the link?
I think this will be a security mess, get misused for bullying with even worse outcomes, and hopefully there will be legislation that will tame these kind of applications, regardless of how technology cool it might be.
Or people will get used to it and over time stop trusting video that much; just like you wouldn’t trust this comment if I wrote that I am Tim Cook[1].
And most likely a bit of both, i.e. some extra laws and some extra awareness.
[1]: https://www.threads.net/@jazzdispensary/post/DFdXpC2SLqU
Completely agree and the recent news about financial fraud perpetrated via video conferencing (as a component of the attack) validates your point.
On the other hand, if tools like these don’t get out into the wild and show the dark arts potential, then it remains a straw man argument against unchecked distribution and use cases.
Things are going to get ugly and it remains to be seen how much weight a check-box legal liability waiver will hold up.
It reminds me of that scene in Johnny Mnemonic where he's on the video call and the bad guy is using internet puppet fingers to impersonate someone. I think about this scene often and I feel like we are getting there. excited to try it when i have a gpu ;)
Do you plan on making source code available? (Or: how can users verify that this is not malware?)
hmm not sure yet on the open source thing, but how do people normally verify downloadable software has no malware? I guess we could try to distribute it on like reputable distribution channels like the app store
Personally I do take at least a quick look over sources before deciding to trust any new app. I simply won't install apps that don't provide the option.
This used to be an ideological stance but increasingly recently it's the only pragmatic thing to do from a stance of security and safety. The playing field is increasingly hostile and if someone asks you to install their software on you machine and let it record your face and voice but refuse to show what it actually does, that is a red flag. Reasonable exceptions could include video games (which run on dedicated untrusted devices and IMO the IP aspect makes the closed-source stance more understandable there). On the other hand, this app is inherently sensitive and trusted because of its function. I don't see the reason why it needs to be closed-source.
Malware is commonly distrubuted in all app stores. I reported some obviously pretty bad stuff that is still up a year later on Play Store, for example. Google simply doesnt bother if the case is too messy.
> hmm not sure yet on the open source thing
You could start with just go source-available by sharing the source with your users without going full Open Source, if you want to take the time and think about what license to use.
thanks for the write up! it's an electron app so i think you can also view the source code easily that way tbh
Another thing to add: this is Linux-only and a large amount of Linux users will care about your product being free software or under an open source license for ethical reasons. Source available doesn't mean open source, and open source means your product's license protects distribution and modifications of the code to some extent. This extent is quite debated, but you should certainly read up on this and have a strong defence for why your product isn't open source either way. I can certainly see why you wouldn't want to, but make sure to think about it especially for Linux-only. Windows and Mac users are probably more amenable to proprietary software.
> you can also view the source code easily that way tbh
sure thing, just gotta download and execute it.... WAIT A MINUTE. YOU ALMOST GOT ME! YOU SOB
> you can also view the source code easily
can just as easily throw it on github, if your intentions were legit
Some day we should have everything sandboxed on the hw level. Like computers in 1980
Reproducible builds are the way. But source available is better for people who are worried about malware
How do you reproduce a build without source?
Meticulously with Ghidra I suppose. This trend of saying it’s to protect us is getting old. The only way to prevent malware is by making the source available for us to see for ourselves.
Yeah, reproducible builds are meaningless for users without a reference to reproduce.
If this isn't monetized, is there any reason against opening its source? I personally would like to be able to disable the usage analytics and crash reports.
> If this isn't monetized, is there any reason against opening its source? I personally would like to be able to disable the usage analytics and crash reports.
Although I personally deplore it and try to stay away from software that requires it, or even opts me in, I nonetheless think that it's reasonable for a developer to impose telemetry as a requirement for people to make use of their freely available software.
The first thing I'd do if it were open source is spin up a PR to allow the telemetry to be opt-in. After two decades of being the product, it's hard to trust any data collection done by companies outside the EU.
As long as it's opt-in, sure. Otherwise it's be an illegal GDPR breach.
Pretty sure if they just don't you use the software without opting in then it isn't a GDPR breach... You aren't entitled to use the software.
GDPR is very clear about how you are not allowed to make data collection a requirement for use.
It is a straightforward set of rules written in simple language and it’s not very long either. It’s not necessary to rely on third party readings or interpretations of it. Just go ahead and read it yourself and you will be well equipped to apply and argue about it.
But then you could not allow access from the EU.
Except I can just deny all of Europe access? Then anyone who wants to use it has to pretend to not be European through a VPN or otherwise.
You can argue specifics and nitpick, but the outcome is effectively the same.
> GDPR is very clear about how you are not allowed to make data collection a requirement for use.
Everyone --- even German newspapers --- flouts this rule. "Consent or pay" is a common strategy.
And they keep losing in courts about it.
eh, don't worry about it. there are some weirdos with trust issues (I'm one) that are vocal about that, but regular people don't have that problem.
Not a critique, but:
Wouldn't using this software constitute a crime if using it to "appear as literally anyone"?
IIRC, the have been news stories in the EU about people receiving prison sentences for creating deepfakes, although maybe it was related to adult material. But impersonation and defamation is likely covered similarly. I'd assume that all it takes is for single viewer to believe it, to legally qualify as an act of impersonation.
FATAL:setuid_sandbox_host.cc(158)] The SUID sandbox helper binary was found, but is not configured correctly. Rather than run without sandboxing I'm aborting now. You need to make sure that /tmp/.mount_phazr-wg9XvA/chrome-sandbox is owned by root and has mode 4755. Trace/breakpoint trap (core dumped)
I haven't installed this yet, but does it require camera access? i.e. does it "transform" your own image to the target image while maintaining facial expression, pose, etc.? Based on the animations, I'd assume it doesn't use the camera since there are techniques that can lipsync from audio.
no camera access needed! it directly generates the image via audio. this is more then just lip sync btw, it's animating the head of the image.
liveportrait or faster-liveportrait are the libraries probably.
similar but we directly audio to the image!
if its fast enough, I’m very curious if this will get me broader acceptance as a different race in interviews with my same skillset and will report back
If it does work better, I’m sure people will just say the market picked up as opposed to validating my life experience, but as long as I’m collecting bigger paychecks believe whatever you want
Hey, is the source open for this?
the best use case for AI seems to be snapchat filters. great stuff
Since no one has asked, why build this? What’s the use case?
I know the culture has changed dramatically in the last ten years, so I'll take this as a sincere question coming from someone who missed out:
Back in the old days, before 2014, people used to make computers do things just for fun. We'd write code because writing code is enjoyable, and hack on projects just to see if we could. If it made something that other people wanted to use that was a bonus, but hacking and experimenting was an end in itself. (The same, incidentally, went for writing blogs and for making YouTube videos).
In the last ten years most of us have lost sight of that in favor of everything needing to have an "audience" or a "use case"—if there's no path to monetization then we struggle to see the point. But some of us still build things just because we can, so from time to time you'll see a project like this that hearkens back to the old school hacker spirit and has no point besides to see if we could.
Thanks for the sincere reply. However, I feel that you avoided the question. There are plenty of projects, actually an infinite variety of things that are fun to build. I make generative line art with a pen plotter. It’s great.
But why this? Why make a tool for anyone to impersonate another person on video calls? I apologize for the cynicism, I actually really hate that my first thought was that this is a tool for scams. But, it’s hard not to see how useful this would be for deceiving people.
The tools for scams are already built and in use, and they're way better than what I'm seeing here. The only example I see with a real photo is a static photo with a mouth wiggling and no other movement. The rest are cartoons, with a similar effect.
I get the concern, but squelching fun use cases isn't going to prevent people from building the bad ones or even slow them down.
I have avoided posting a how-to on YouTube, in part because tech like this could be used to impersonate me. By creating a fake persona with fake voice (this one doesn't do that) we can make those videos.
So the way to not be impersonated is to use the same tools to be a fake person!
People don't have to like what you like and they don't need to give you a justification either. Your tone is needlessly antagonistic and dismissive. Not to mention patronising.
An other possible explanation is that the author's intention is to game interviews.
Why?
This seems great, but why Cuda only?
cuda is just standard for ml. amd port soon maybe
what about Ms Teams?
it works in any video app!
I see only good things happening with this closed-source tech. /s