I am concerned about this from both a mental health standpoint and a privacy standpoint. Replika users have been shown to form real emotional dependence on it and they start feeling responsible for the bot's feelings. https://journals.sagepub.com/doi/10.1177/14614448221142007. There have been a bunch of cases where these models/harnesses lead to suicide.
On privacy, chatbots and agents hosted by big providers collect all sorts of telemetry and build an entire behavioral model of users (this is why i use local open weight models).
Now stick all that in a robot with a camera and mic that lives in your house. Way bigger attack surface. I know it claims to process and store everything locally, and even if that is the case, there are sure to be similar robots that dont.
I am concerned about this from both a mental health standpoint and a privacy standpoint. Replika users have been shown to form real emotional dependence on it and they start feeling responsible for the bot's feelings. https://journals.sagepub.com/doi/10.1177/14614448221142007. There have been a bunch of cases where these models/harnesses lead to suicide.
On privacy, chatbots and agents hosted by big providers collect all sorts of telemetry and build an entire behavioral model of users (this is why i use local open weight models). Now stick all that in a robot with a camera and mic that lives in your house. Way bigger attack surface. I know it claims to process and store everything locally, and even if that is the case, there are sure to be similar robots that dont.