Good intuition, but the enforcement mechanism is a substring blocklist (both "rm -fr /" and "base64 -d | sh" work), and tainting will not work since the model outputs an entirely new string to use as an argument for the tool. It is worth reading the CaMeL paper from DeepMind.
Good intuition, but the enforcement mechanism is a substring blocklist (both "rm -fr /" and "base64 -d | sh" work), and tainting will not work since the model outputs an entirely new string to use as an argument for the tool. It is worth reading the CaMeL paper from DeepMind.
[flagged]
[flagged]