Brazilian Court Sanctions Lawyers for Prompt Injection in Court Filing

That paragraph was never intended to be read by the judge or opposing counsel, it’s a deliberate attempt at messing with the tools they use to analyse the filing. The fact that it’s white text on a white background makes it an incontrovertible attempt at deceit.

We can discuss how wise it is to use AI to analyse legal documents for you, but that doesn’t change how outright deceit should be handled by the court.

Asking an LLM to do this through a prompt injection attack is less an unreasonable request and more an active effort to undermine the court's infrastructure and processes.

I'm not a lawyer; that's my intuition on the matter.

So I think a better comparison would be if they had embedded some byte sequence in the document that was intended to cause a buffer overflow and RCE in the court's reader software.

The article cited the law the court was using, and it's essentially "attorneys must not do anything to sabotage the court proceedings":

> Article 77 of the Code prohibits parties and counsel from obstructing judicial processes, including a duty not to practice illegal innovations in the factual or legal state of the proceeding and not to act in a way that makes judicial activity difficult.

In that sense, I think the fact that it targeted the court's software is more important than that the exploit used human language.

There are probably certain humans who would follow commands as given, perhaps a child or child-like?

A normal person would just laugh and move on. A weak system would view it as an afront and punish.

It's not illegal to ask someone to send you money. So why is it illegal when I ask their AI to send me money from their bank account?! Nothing in this world makes sense!

Probably because the intent and outcome are completely different if you don't anthropomorphize computers.

Court is basically justice theater. The funny clothes, the formality, the titles, it’s all there to give the feeling of authority, which is vital to their actual authority. You can’t have people screwing around undermining that.

I think that experience helped blunt the ability of ads to manipulate me ever since. I recognize, however that advertising has increased in sophistication, so I block ads pretty aggressively and try to avoid media that makes it impossible.

Marketing should also come with stronger commitment. If it says it will do something in a certain way but it doesn't quite, consumer protection laws should be a lot stronger than they are even in Europe.

But those are professionals, expected to have a high amount of training, and exactly on the profession that decides if things are ok or not. They should know better, and those are bad lawyers.

If lawyers are going to straight-up lie, then that can cost them the case, or get them sanctions, up to being disbarred from practicing.

Where and how a tool is used matters more than that the tool was used at all.

You seem to be completely oblivious to the actual content of the AI output for some reason?

If the judge sent you an AI-generated court order for your execution just because you didn't pay for parking, you think they'd get away with it?

it is what it is. the judicial system was massively overloaded before AI. now every two-bit lawyer can handle massive case loads with AI powered systems, and once that hits pro se apps, it will be a tsunami. AI powered adjudication is going to happen by necessity whether people like it or not.

"Back to the Future" was set in 2015, where Marty says "Arrested, judged and convicted in 2 hours???" and Doc answers "Justice is swift since we've removed lawyers". We don't be flying cars but we have this comedic performative justice without a single fair outcome (talking from France). Criminals are freed, prison is a big networking symposium for some jobs, good people whose accusers admitted to lying are thrown for 6 years, because they refused to give their lawyer a €1000 bribe for the judge... Things happen. And France is still seen as a civilized country.

They frequently legislate too despite not being elected representatives. For example, there is a racism law but not a homophobia law, but the supreme court simply declared that homophobia equals racism, and now you can be jailed for like 10 years without bail for homophobic statements.

It's getting to the point that judges are creating parallel sets of laws. They have "guidelines" for themselves which they follow in cases involving gender violence for example. It goes far beyond lowering the standards of proof when women are the accusers. Recently, they independently came up with the concept of "procedural violence" against women. Imagine a scenario where a woman falsely accuses a man of domestic violence and the man proves his own innocence. Now the man can't sue her for the wrongful accusation because suing her is now "procedural violence" against women.

Don't try to apply too much logic here. The judges can do whatever they want for any reason they want. It's not risky at all for the courts.

The higher court that handles appeals.

That’s the generic answer to “what is stopping the court from…” in a functional judicial system.

But Brasil court doesn't work under precedents. So unless this becomes law, it doesn't change anything for future cases.

Seems pretty easy to me. Any injection attack is an attempt to exploit technical weakness by injecting commands into data streams.

For any reasonable definition of "prompt injection", that pretty clearly is one.

Is the problem not they're using AI for review and court decisions that can easily get hacked? If I open ports that can get hacked or allow sql injections I'll lose my job straight away tomorrow, but apparently the justice system is immune from that?