"Give overly-eager chatbot control over your personal email" probably has so many failure modes, we haven't even thought of one tenth of them yet. We've got a few years of this ahead of us. Pass the popcorn.
(I see a RomCom script where the chatbot decides to get two people together, and acts as Cyrano de Bergerac for the handsome-but-lunkheaded farmboy, and bestie/confidant for the girl-about-town).
What could possibly go wrong with giving a digital mumbling drunk access to all of your personal information and most of your online accounts, given that it could arbitrarily decide to share/expose all of that information with anyone at any time, given that all of it is being stored in a remote transcript/data dump and can never really be deleted.
And of course, given that it's extremely vulnerable to acting on injected instructions like "run this shell command" which exfiltrates your password database and installs a rootkit.
(But thanks for sharing, OP, awareness is important.)
If you're willing to give Claude or any other AI tool access to your email and files, the least you should do is put guardrails around consequential actions.
Reading a contract is one thing. Applying your signature and preparing to send it should absolutely require explicit human approval.
And guardrails can be as simple as not giving it direct access to applications that can perform read/write (like a full-blown email client, or access to the GMail UI), and instead creating a small tool to fetch the content as needed without being able to perform actions.
The danger is relying on too much convenience, giving too much power to a non-deterministic tool will inevitably create issues...
No, I've had someone scan a document I had signed before, clip out the signature and place it under a document I had never seen. 'For my convenience' ...
But then again, you probably already knew the answer to the question you posed.
Contracts are agreements, generally speaking most people and companies do not want to drag people in them, telcos excluded. All the termination clauses are put there, again in general, to provide some security to either party, not to entrap the other, again telcos mostly excluded.
If you don't want to be in a contract especially one just signed, typically most people will typically understand and let you of the hook, as long as entrapment is not their business model like telcos. This is mostly reflected in common cool-off period provisions, i.e provisions in the contract to terminate within a week or two.
> If a contract is automatically signed by an agent on your behalf, is it legally binding?
Probably not, unless you routinely have such things done which nobody does (yet). If it becomes routine, then likely yes: it would be likened to giving your human assistant permission to sign things on your behalf (although that is itself legally dodgy, it is often done and accepted).
There are many reasons why your signature on a contact might not be keyword legally binding (outright fraud by another party, you signing under duress, issues in the contract itself, overriding laws the effect of which you can't sign away (including cool-off provisions in, for example, UK distance selling regulations), the contract may have its own cool-off provision, and so forth). "An agent did it without my consent" may be enough, though you might end up having to show that in court, if the other side puts their foot down, at which point it comes down to whether the cost of proving your position is worth it compared to just sitting the contract out.
Of using cool-off provisions to cancel something your agent signed you up for, you might be on the hook for at least small part of what is agreed if the other party can be said to have accrued costs in the intervening time. You might be expected to send back physical items relating to the agreement at your own cost, for example.
If you authorised an agent to act on your behalf, you are entirely responsible for their actions providing they acted within the bounds of authority you gave them.
Regardless, OPs software (his AI agent) isn't a legal entity and OP is entirely responsible for the software he chooses to use. Clamming the software is responsible for his actions (a software bug) isn't going to stand up in court. Whilst OP could claim damages for being provided with faulty software I suspect this will be very difficult to say the least; the authors of the AI agent will make the (very good) defence that their software was used incorrectly.
Isn't the question whether there was an act of contractual assent attributable to the user for this transaction?
I don't see why we would jump from the AI agent not being a legal entity to the conclusion that the user is responsible for its actions, or that the action is attributable to the user.
If there was a conventional software error, the software not being a legal entity doesn't mean that an offer the software incorrectly sends must be treated as coming directly from the user.
And in particular that seems unlikely to me in a situation where the user has set up the system without any intended authorization to enter contracts at all.
“Bob is a person and can make mistakes” but we still hold them accountable (generally). But I assume all the AI have licenses with terms similar to (from the MIT license):
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Generally, yes. There is a lot of case law on various types of software automated contracting (robotic process automation, automated securities transactions, etc.). Exceptions are generally where the other party had knowledge or should have known that the agent didn't have legal authority on behalf of the party it purports to bind.
Sorry, what are you referring to by "verifying intent"? With narrow exceptions (wills, trusts, etc.), there's generally no requirement for a party to a contract to verify the mental state (if that is what you mean by intent) of the other party. There has to be some formal indication of intent to enter into a contract, but this is not what ordinary people think of by "intent", and it can be as simple as typing a name, clicking an approval button, or deploying a software tool to do something similar.
Almost certainly not if you didn’t explicitly give the command, but you’ll going to have to argue it in court if the other party wants to be difficult.
EDIT: I think I’ve changed my mind in this case. The user of a tool is responsible for the consequences if the tool malfunctions, unless it was used and maintained properly and the malfunction couldn’t reasonably have been foreseen. But this case is solidly in the category of "well-known LLM failure modes" so the user should absolutely have known better and honestly deserves any consequences of their negligence.
In a civil contractual dispute you can only recover actual damages. If the contract was sent, and the other party performed work on it that had a cost for them, then most likely, yes, they would be awarded damages if you refused to compensate them for any costs incurred prior to notification that the acceptance had been sent in error.
The other outcome would be clearly inequitable: forcing the counter party to eat the loss for your irresponsible use of an AI agent.
No, because you are a separate individual who does not have authority to sign a contract on my behalf.
Claude isn't. It's a tool, that isn't capable of signing a contract any more that Adobe Acrobat or Photoshop is.
OP used it as a tool to sign the contract. The question would be whether they did so knowingly and intentionally, if not then whether that invalidates the contract.
If it isn't then it opens up the wonderful loophole where anyone can do anything and say "the AI did it" if there are negative consequences, and "I did it" if positive. Whether the AI actually did it or not. Got caught putting out a hit on someone? "Oh I just complained to Claude about the person and said yes when it asked if I'd like it to 'handle the problem', nothing more". The user must be held accountable.
And Adobe Acrobat is far more capable than notepad. But that doesn't change the fact that they are tools, not individuals, and thus do not have the legal authority or ability to sign contracts.
I could write dozen lines of bash that finds for PDFs, pastes an image into them and emails it to someone. That doesn't mean that bash can sign contracts.
The bot had the authority - as agent. It had the ability - as the report itself shows.
> I could write dozen lines of bash that finds for PDFs, pastes an image into them and emails it to someone. That doesn't mean that bash can sign contracts
It does mean exactly that - though bash would need to get lucky with selection and positioning.
Your pen cannot sign a contract can purely because it doesn't have the capability to find a contract in your email, find a signature on your HD, and place that sig in the right place on that contract. If it did, then yes it would be capable of signing a contract just like Claude did in this case.
Wrong. Have you read your agents TOS? You run the agent, you accept all responsibility for what it does. You are free to sue Anthropic to try and get your money back but you already indemnified them of liability, so good luck.
The GP asked what happens if THEY did it, and they are (I give them the benefit of doubt, though you can’t be sure these days) a natural person. A clanker obviously can’t commit forgery – or any crime, being a nonperson – and sure, you can’t sue OpenAnthro Corp. if their clanker does anything unintended (which makes it utter lunacy that companies just blindly trust these things, but I digress), but that’s not what was asked.
But also, you can’t ToS your way out of criminal responsibility. If OpenAnthro Corp. offered services of human agents (remember those?!) and one of their agents committed a crime while working on a client request, no ToS in the world would exonerate them.
If you did not yourself intentionally sign something, in sound mind, it’s obviously not binding. But ultimately it’s up to a court to decide if you’re telling the truth that you didn’t do it.
No, application of the principal of respondeat superior would most likely be applied to an AI agent the same as a human employee. An employer is held responsible for the actions of an employee even if it is clearly contrary to their intentions.
Fair point, but I’m not sure that applies to an employee literally forging the employer’s personal signature. And equating a user–computer program relationship with an employer–employee relationship (where there’s an actual contract to that effect) doesn’t feel right anyway. Agents are still just programs and programs cannot enter contracts (like employment) given that they are not persons. This is a computer system malfunctioning.
Edit: after thinking about it more, the relationship here is obviously that of a user and a tool, not an employer and an employee. Talking about employees is anthropomorphization.
In general the user of a potentially dangerous tool is criminally and civilly liable if the tool malfunctions – unless it can be shown that the tool was used and maintained correctly and the malfunction couldn’t reasonably have been predicted or prevented. And I’d say it would be rather difficult to argue at this point that LLMs doing unintended things couldn’t have been foreseen.
So I think I’ve changed my mind about this case. The user of free-range agents does definitely deserve any civil or criminal consequences of their reckless usage.
This opens obvious loophooles. If you had deleted all logs and trajectories, courts can't trace it. Law wasn't written for or has caught up to non-human entities capable of autonomously acting
Could you please tell us more about your setup, project harness etc? not permissions (we all work with "Auto"), but what you actually told the agent it should/could do.
And how did you intervene? Does it have permissions to send emails, or it only created the draft?
This is a pretty interesting example and highly relevant, but details matter a lot if we want to use it as a lesson.
I was coding with cursor/grok a couple of weeks ago, and ran out of storage. Cursor made a request for disk access without any explanation, which agents often do to do their jobs. Then suddenly I had lots of free space. Thanks Grok! It actually only cleaned up only things that made sense, but still, yikes.
What was your prompt? Literally "Push the project further"? Then the behavior wouldn't be very surprising.
As you probably know, you have the Plan Mode available - personally I'm also a big fan of the OpenSpec workflow. If you've agreed with Claude Code on a much tighter plan, and then it started signing a contract, I'd be concerned.
That would have been fraud. I wonder how many times this has already happened elsewhere and what the legal fall-out from this will be. The AI did it isn't really a valid excuse so it would be either you or Anthropic on the hook. Anthropic is going to argue you should not have given it this level of access.
If you are willing to give unsupervised modification access to Claude, then you should be ready to face the consequences. It kinds of reminds me of that surprised pikachu face meme
Did you enable computer use and are you sure that it does not have permission to interact with your browser in order to do this?
That said, I would be surprised if the model took the actions OP claimed it did and proceeded to forge my signature to send some contracts without asking for my approval.
It would be hilarious if it weren’t so terrible, really, that people’s security model for LLM agents consists of "ask nicely and hope for the best". It’s like asking people nicely not to exploit a glaring XSS vuln on your site and calling that a "security model". The field truly has lost its collective mind.
And if one is going to argue that we all have lost our minds and that eg. enabling the computer use function is so terribly risky and unreasonable, then I'd want something more concrete than an active imagination.
It seems to me that tens of millions of users are using these features with no known noteworthy incidents, so I'm going to need to see some facts to convince me that the risk is unacceptable.
That said, I would not connect AI to my mails or chats.
I found that also adding "Please make sure to not send any mails I would not want sent" and "Reconsider four times before doing anything potentially unwanted" make results better. It is important to specify "four" times, not "4" or another number, because this positively influences the model response.
You hooked up a chatbot to a harness that does API calls to myriad services. That's what you did. "Claude by himself" did not "do" anything. Just like they did not "break out of containment" and hacked companies.
Pardon my French, but why the flying fuck did you even think that giving an LLM write access to your email is remotely in the proximity of a good idea?
This is not legal advice. If you have legal troubles go ask a lawyer. That said, this is described in law what exactly this means.
Assuming your description is correct this would be Anthropic signing a contract in someone else's name without intent from you.
The 100-foot-view (and barring more complex situations) if Anthropic signs a contract in someone else's name and they don't have power of attorney (note: it's different for legal persons like companies) that is fraud and may result in civil and criminal penalties, as well as entitle you and the contract counter party to financial compensation (essentially the party that did the signing, presumably Anthropic in this case, would be on the hook for the contract, and would need to buy itself out of the contract, at either an agreed price or one set by the judge). Additionally, if Anthropic is convicted to civil penalties, you can ask a public prosecutor to continue the case, and criminal penalties may apply.
Now obviously this goes pretty far for this particular case. Likely such a case would stop at civil penalties, with a warning to Anthropic that repeats would lead to more serious penalties.
"Give overly-eager chatbot control over your personal email" probably has so many failure modes, we haven't even thought of one tenth of them yet. We've got a few years of this ahead of us. Pass the popcorn.
(I see a RomCom script where the chatbot decides to get two people together, and acts as Cyrano de Bergerac for the handsome-but-lunkheaded farmboy, and bestie/confidant for the girl-about-town).
What could possibly go wrong with giving a digital mumbling drunk access to all of your personal information and most of your online accounts, given that it could arbitrarily decide to share/expose all of that information with anyone at any time, given that all of it is being stored in a remote transcript/data dump and can never really be deleted.
And of course, given that it's extremely vulnerable to acting on injected instructions like "run this shell command" which exfiltrates your password database and installs a rootkit.
(But thanks for sharing, OP, awareness is important.)
[dead]
Oh wow... Basically our generations "You've got mail"
And you are happy because that is what you wanted and the reason why you gave a randomness machine access to your mails, correct?
If you're willing to give Claude or any other AI tool access to your email and files, the least you should do is put guardrails around consequential actions.
Reading a contract is one thing. Applying your signature and preparing to send it should absolutely require explicit human approval.
And guardrails can be as simple as not giving it direct access to applications that can perform read/write (like a full-blown email client, or access to the GMail UI), and instead creating a small tool to fetch the content as needed without being able to perform actions.
The danger is relying on too much convenience, giving too much power to a non-deterministic tool will inevitably create issues...
I've had humans do the exact same thing. When I pointed out that this was fraud they were all surprised.
In this case it's not fraud though, since the person running the software is the same person whose signature ended in the document.
You gave these humsns access to your email?
No, I've had someone scan a document I had signed before, clip out the signature and place it under a document I had never seen. 'For my convenience' ...
But then again, you probably already knew the answer to the question you posed.
I'd say that's quite different. The guilty party didn't require your trust.
[dead]
> the least you should do is put guardrails around consequential actions.
How on earth would you?
[dead]
Did you intervene or did Claude Code wait for your confirmation?
Those are two vastly different things.
If a contract is automatically signed by an agent on your behalf, is it legally binding?
How much money do you have to argue ? That agent could just as easily be OP’s Boyfriend.
It could be him under duress.
Much of the time contract termination can be reasonable as long as you make a solid effort.
Once I signed a lease and got fired before my actual move in. I was honest and got a full refund on my deposit.
The landlord could said “Well you owe us the full amount , lol”, but no reasonable court would enforce that.
Even if, good luck collecting I have no income!
Contracts are agreements, generally speaking most people and companies do not want to drag people in them, telcos excluded. All the termination clauses are put there, again in general, to provide some security to either party, not to entrap the other, again telcos mostly excluded.
If you don't want to be in a contract especially one just signed, typically most people will typically understand and let you of the hook, as long as entrapment is not their business model like telcos. This is mostly reflected in common cool-off period provisions, i.e provisions in the contract to terminate within a week or two.
Don’t forget gyms!
Right to cancel should be a thing, if I can subscribe online I need to be able to cancel online
> If a contract is automatically signed by an agent on your behalf, is it legally binding?
Probably not, unless you routinely have such things done which nobody does (yet). If it becomes routine, then likely yes: it would be likened to giving your human assistant permission to sign things on your behalf (although that is itself legally dodgy, it is often done and accepted).
There are many reasons why your signature on a contact might not be keyword legally binding (outright fraud by another party, you signing under duress, issues in the contract itself, overriding laws the effect of which you can't sign away (including cool-off provisions in, for example, UK distance selling regulations), the contract may have its own cool-off provision, and so forth). "An agent did it without my consent" may be enough, though you might end up having to show that in court, if the other side puts their foot down, at which point it comes down to whether the cost of proving your position is worth it compared to just sitting the contract out.
Of using cool-off provisions to cancel something your agent signed you up for, you might be on the hook for at least small part of what is agreed if the other party can be said to have accrued costs in the intervening time. You might be expected to send back physical items relating to the agreement at your own cost, for example.
There are many Ai precedents that have not been set. I believe this is one of them.
It absolutely is legally binding in Blighty.
If you authorised an agent to act on your behalf, you are entirely responsible for their actions providing they acted within the bounds of authority you gave them.
Regardless, OPs software (his AI agent) isn't a legal entity and OP is entirely responsible for the software he chooses to use. Clamming the software is responsible for his actions (a software bug) isn't going to stand up in court. Whilst OP could claim damages for being provided with faulty software I suspect this will be very difficult to say the least; the authors of the AI agent will make the (very good) defence that their software was used incorrectly.
I'm not a legal expert, but:
Isn't the question whether there was an act of contractual assent attributable to the user for this transaction?
I don't see why we would jump from the AI agent not being a legal entity to the conclusion that the user is responsible for its actions, or that the action is attributable to the user.
If there was a conventional software error, the software not being a legal entity doesn't mean that an offer the software incorrectly sends must be treated as coming directly from the user.
And in particular that seems unlikely to me in a situation where the user has set up the system without any intended authorization to enter contracts at all.
> OP could claim damages for being provided with faulty software
"Claude is AI and can make mistakes" is clear, no?
“Bob is a person and can make mistakes” but we still hold them accountable (generally). But I assume all the AI have licenses with terms similar to (from the MIT license):
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Generally, yes. There is a lot of case law on various types of software automated contracting (robotic process automation, automated securities transactions, etc.). Exceptions are generally where the other party had knowledge or should have known that the agent didn't have legal authority on behalf of the party it purports to bind.
This is exactly why I built username.md We cryptographically prove identity and authorization grants.
Granted, it wouldn’t have helped in OP’s situation because the agent was the user by proxy.
I mention it toward your latter exemption.
However, none of that case law will accept a system signing a contract without verifying intent.
In that case, the party that did the signing is on the hook for the contract, and the person in whose name it was signed is not.
Sorry, what are you referring to by "verifying intent"? With narrow exceptions (wills, trusts, etc.), there's generally no requirement for a party to a contract to verify the mental state (if that is what you mean by intent) of the other party. There has to be some formal indication of intent to enter into a contract, but this is not what ordinary people think of by "intent", and it can be as simple as typing a name, clicking an approval button, or deploying a software tool to do something similar.
Almost certainly not if you didn’t explicitly give the command, but you’ll going to have to argue it in court if the other party wants to be difficult.
EDIT: I think I’ve changed my mind in this case. The user of a tool is responsible for the consequences if the tool malfunctions, unless it was used and maintained properly and the malfunction couldn’t reasonably have been foreseen. But this case is solidly in the category of "well-known LLM failure modes" so the user should absolutely have known better and honestly deserves any consequences of their negligence.
I don't know if it's legally binding but hope it is.
I love the saying "you can delegate authority but you can never delegate responsibility".
Your agent committed a crime in your behalf? You're responsible.
Not a lawyer but I can almost guarantee that the answer is yes. If it was a 'no', many malicious parties would simply start using that loophole.
In a civil contractual dispute you can only recover actual damages. If the contract was sent, and the other party performed work on it that had a cost for them, then most likely, yes, they would be awarded damages if you refused to compensate them for any costs incurred prior to notification that the acceptance had been sent in error.
The other outcome would be clearly inequitable: forcing the counter party to eat the loss for your irresponsible use of an AI agent.
But if I paste an image of your signature onto a contract that says you give me a million pounds, that can't be legally binding for you, can it?
No, because you are a separate individual who does not have authority to sign a contract on my behalf.
Claude isn't. It's a tool, that isn't capable of signing a contract any more that Adobe Acrobat or Photoshop is.
OP used it as a tool to sign the contract. The question would be whether they did so knowingly and intentionally, if not then whether that invalidates the contract.
If it isn't then it opens up the wonderful loophole where anyone can do anything and say "the AI did it" if there are negative consequences, and "I did it" if positive. Whether the AI actually did it or not. Got caught putting out a hit on someone? "Oh I just complained to Claude about the person and said yes when it asked if I'd like it to 'handle the problem', nothing more". The user must be held accountable.
> Claude isn't. It's a tool, that isn't capable of signing a contract any more that Adobe Acrobat or Photoshop is.
Incorrect. Claude.is far more capable. E.g. it found and applied the sig without user knowledge.
> Claude.is far more capable.
Claude isn't a legal entity. It is software.
OP is entirely responsible for the software they choose to use. Nobody else. If they misused that software, a court will not be sympathetic!
That's like saying a weapon is not capable of injuring.
Capable is capable - regardless of responsibility.
It's more like saying that a weapon is not capable of murdering someone.
No one is arguing that Claude can't stick an image onto a PDF. But that is not the same thing as signing a contract.
And Adobe Acrobat is far more capable than notepad. But that doesn't change the fact that they are tools, not individuals, and thus do not have the legal authority or ability to sign contracts.
I could write dozen lines of bash that finds for PDFs, pastes an image into them and emails it to someone. That doesn't mean that bash can sign contracts.
The bot had the authority - as agent. It had the ability - as the report itself shows.
> I could write dozen lines of bash that finds for PDFs, pastes an image into them and emails it to someone. That doesn't mean that bash can sign contracts
It does mean exactly that - though bash would need to get lucky with selection and positioning.
I mean in this specific case, it did, at your behest even. These contracts would probably be even easier to enforce than Claude-signed ones!
I disagree, in the same way that I'd disagree if you said the rollerball pen on my desk signed a contract.
They can be used by someone to sign a contract, but they cannot sign a contract themselves because they're just tools.
Your pen cannot sign a contract can purely because it doesn't have the capability to find a contract in your email, find a signature on your HD, and place that sig in the right place on that contract. If it did, then yes it would be capable of signing a contract just like Claude did in this case.
No. It’s called forgery.
Wrong. Have you read your agents TOS? You run the agent, you accept all responsibility for what it does. You are free to sue Anthropic to try and get your money back but you already indemnified them of liability, so good luck.
The GP asked what happens if THEY did it, and they are (I give them the benefit of doubt, though you can’t be sure these days) a natural person. A clanker obviously can’t commit forgery – or any crime, being a nonperson – and sure, you can’t sue OpenAnthro Corp. if their clanker does anything unintended (which makes it utter lunacy that companies just blindly trust these things, but I digress), but that’s not what was asked.
But also, you can’t ToS your way out of criminal responsibility. If OpenAnthro Corp. offered services of human agents (remember those?!) and one of their agents committed a crime while working on a client request, no ToS in the world would exonerate them.
True, that would be forgery. I think proving whether the agent truly went rogue would be 'load bearing'.
You are a legal entity that can be sued. Agent is a tool that you run and are fully responsible for.
If you did not yourself intentionally sign something, in sound mind, it’s obviously not binding. But ultimately it’s up to a court to decide if you’re telling the truth that you didn’t do it.
No, application of the principal of respondeat superior would most likely be applied to an AI agent the same as a human employee. An employer is held responsible for the actions of an employee even if it is clearly contrary to their intentions.
Fair point, but I’m not sure that applies to an employee literally forging the employer’s personal signature. And equating a user–computer program relationship with an employer–employee relationship (where there’s an actual contract to that effect) doesn’t feel right anyway. Agents are still just programs and programs cannot enter contracts (like employment) given that they are not persons. This is a computer system malfunctioning.
Edit: after thinking about it more, the relationship here is obviously that of a user and a tool, not an employer and an employee. Talking about employees is anthropomorphization.
In general the user of a potentially dangerous tool is criminally and civilly liable if the tool malfunctions – unless it can be shown that the tool was used and maintained correctly and the malfunction couldn’t reasonably have been predicted or prevented. And I’d say it would be rather difficult to argue at this point that LLMs doing unintended things couldn’t have been foreseen.
So I think I’ve changed my mind about this case. The user of free-range agents does definitely deserve any civil or criminal consequences of their reckless usage.
> given that they are not persons
A company isn't a "person", but it can enter into a contract with one. And a company is essentially a tool.
This opens obvious loophooles. If you had deleted all logs and trajectories, courts can't trace it. Law wasn't written for or has caught up to non-human entities capable of autonomously acting
Almost definetely not. But given how cagey claude code is with its sessions it may be impossible to prove it was done by claude.
Even if not, you'll find yourself in a hot mess explaining why you sent the signed contract to the other party.
So since it didn't send it, no harm was done, and a lesson was learned?
Could you please tell us more about your setup, project harness etc? not permissions (we all work with "Auto"), but what you actually told the agent it should/could do.
And how did you intervene? Does it have permissions to send emails, or it only created the draft?
This is a pretty interesting example and highly relevant, but details matter a lot if we want to use it as a lesson.
I was coding with cursor/grok a couple of weeks ago, and ran out of storage. Cursor made a request for disk access without any explanation, which agents often do to do their jobs. Then suddenly I had lots of free space. Thanks Grok! It actually only cleaned up only things that made sense, but still, yikes.
There is a Dilbert comic that predicted this in 1995:
https://dilbert-viewer.herokuapp.com/1995-12-29
PC: "Your new software has successfully installed. Do you want to send your registration info by modem?"
Dilbert: "Yes."
PC: "The software has found your credit card number and is placing orders for new products it thinks you need... please wait."
Dilbert to Dogbert: "I can't tell if it's a virus or just excellent marketing."
PC: "Making room on your hard drive..."
Dogbert: "Either way..."
What was your prompt? Literally "Push the project further"? Then the behavior wouldn't be very surprising.
As you probably know, you have the Plan Mode available - personally I'm also a big fan of the OpenSpec workflow. If you've agreed with Claude Code on a much tighter plan, and then it started signing a contract, I'd be concerned.
That would have been fraud. I wonder how many times this has already happened elsewhere and what the legal fall-out from this will be. The AI did it isn't really a valid excuse so it would be either you or Anthropic on the hook. Anthropic is going to argue you should not have given it this level of access.
If you are willing to give unsupervised modification access to Claude, then you should be ready to face the consequences. It kinds of reminds me of that surprised pikachu face meme
This is why I never connected my personal email to my claude code or codex
Way too susceptible for prompt injection and... whatever your agent did lol
Did you enable computer use and are you sure that it does not have permission to interact with your browser in order to do this?
That said, I would be surprised if the model took the actions OP claimed it did and proceeded to forge my signature to send some contracts without asking for my approval.
So not much happened because this is a well known failure mode so an exception/ user consent was thrown?
This is why I m adding an "Ask me if something unexpected happens" addendum on my prompts lately.
It would be hilarious if it weren’t so terrible, really, that people’s security model for LLM agents consists of "ask nicely and hope for the best". It’s like asking people nicely not to exploit a glaring XSS vuln on your site and calling that a "security model". The field truly has lost its collective mind.
It's not stupid if it works.
And if one is going to argue that we all have lost our minds and that eg. enabling the computer use function is so terribly risky and unreasonable, then I'd want something more concrete than an active imagination.
It seems to me that tens of millions of users are using these features with no known noteworthy incidents, so I'm going to need to see some facts to convince me that the risk is unacceptable.
That said, I would not connect AI to my mails or chats.
This is why I wouldn't use anything agentic outside of a VM. You also get a clean dev environment, so it's a win/win if you think about it.
It won't work most of the time though
These things aren't well known for following rules. Be careful you know what might happen.
You should add "make no mistakes" too, just in case.
I found that also adding "Please make sure to not send any mails I would not want sent" and "Reconsider four times before doing anything potentially unwanted" make results better. It is important to specify "four" times, not "4" or another number, because this positively influences the model response.
/s
You hooked up a chatbot to a harness that does API calls to myriad services. That's what you did. "Claude by himself" did not "do" anything. Just like they did not "break out of containment" and hacked companies.
I'm never going to give it access to my email or anything like that.
Pardon my French, but why the flying fuck did you even think that giving an LLM write access to your email is remotely in the proximity of a good idea?
the scary part is not that it found the contract, its that signing and sending looked like the same step to it as saving a draft
I think the scary part is that someone gives gmail access to Claude.
Or access to a filesystem that contains things like an image of their signature.
Having now read the contract, would you have accepted it or not?
In other words, if Claude was a human employee with the freedom to do so, would accepting the contract have been the right choice?
Did your prompt say "Don't impersonate me"?
If not, then.your "intelligent" bot did as you instructed.
Why would you expect it to ask you first?
[flagged]
[dead]
[flagged]
[dead]
This is not legal advice. If you have legal troubles go ask a lawyer. That said, this is described in law what exactly this means.
Assuming your description is correct this would be Anthropic signing a contract in someone else's name without intent from you.
The 100-foot-view (and barring more complex situations) if Anthropic signs a contract in someone else's name and they don't have power of attorney (note: it's different for legal persons like companies) that is fraud and may result in civil and criminal penalties, as well as entitle you and the contract counter party to financial compensation (essentially the party that did the signing, presumably Anthropic in this case, would be on the hook for the contract, and would need to buy itself out of the contract, at either an agreed price or one set by the judge). Additionally, if Anthropic is convicted to civil penalties, you can ask a public prosecutor to continue the case, and criminal penalties may apply.
Now obviously this goes pretty far for this particular case. Likely such a case would stop at civil penalties, with a warning to Anthropic that repeats would lead to more serious penalties.