The Dark Side of AI-Generated Contracts: What You Don't Know Can Hurt You

Can AI really step up to the plate and generate a simple legal document? More specifically, will the text generated by AI be considered legally valid?

To get to the bottom of this, lawyers from Lawrina dove into several rounds of research. First, they put AI’s skills to the test by asking it to generate an eviction notice based on a detailed request. The results were nothing short of surprising — you can check them out for yourself.

Next, we tasked GPT-4 with drafting a legal contract for an independent contractor. While the AI did manage to create a template, it fell short of protecting the contractor in case of any issues that might crop up. Understanding contractual obligations is crucial in AI-generated contracts to ensure clarity and legal compliance.

Now, it’s time for round three. We asked the AI to create a simple cleaning service contract and compare it with the ones available on Lawrina Templates. This would give us a better idea of whether AI can deliver.

So, could the AI create a cleaning contract that could protect the parties in case of a dispute? Let’s take a closer look.

With AI, the key is to be as specific as possible. You can also add the sentence, “You can ask me any questions you need to create this document.” — this is a cheat code for many requests. The more information you provide, the better equipped the AI will be to deliver what you need.

Here’s another insider tip: once the AI has generated the contract, ask it to review the text with a lawyer’s eye. Make it clear that you want the AI to scrutinize the document to ensure it’s legally valid and protects your interests in case things go south.

One important caveat to keep in mind:

Finally, our team has found that it’s a good idea to generate multiple document versions with a single request and compare them.

Take a close look at the different versions — is one more detailed than the others? Do they share the same overall structure and formatting, or are there notable differences? This can help you better understand what you’re working with.

Since the cleaning contract is a fairly simple agreement, the request for AI will also be quite simple:

So, what did the AI come up with? We got a contract — or rather, several versions of one - but none of them quite hit the mark. They were more like templates, lacking the crucial details needed to make them legally binding and protect your interests in court.

Let’s take a closer look at what Chat GPT generated. Starting with the first point, Services Provided, the AI produced the following text:

So, what did the lawyers from Lawrina flag as a problem with the text? The issue was that the most critical information — the specifics of the services to be provided — was relegated to a separate schedule, which didn’t give any real insight into the heart of the agreement.

But how should it be done? In this section, the scope of services should be clearly spelled out so the essence of the agreement is immediately clear. That way, all parties involved can get a good grasp of what’s on the table.

Cleaning agreement template by Lawrina:

Next, AI reminds us bout the Terms of Payment, namely:

Let’s examine the major issues with this block, and there are quite a few red flags.

First, the chat only mentions one type of payment, but what about a one-time fixed payment for the cleaning service? It’s a glaring omission.

Moving on to point 2.2, the payment by invoices is described in very general terms, which raises a whole host of questions. For instance:

In fact, the contract doesn’t define “late payment,” leaving the client vulnerable to interpretation.

And then there’s point 2.3, where the interest is tacked on to the payment terms without any clear guidance. We need to add a separate section on payment terms and make sure it doesn’t conflict with this point.

The AI text doesn’t provide this clarity, so it’s possible to manipulate the late payment percentage at will. In contrast, Lawrina’s lawyers have crafted a cleaning contract that clearly outlines when and how much interest will be charged for late payment.

Right from the get-go, the AI has glossed over critical sections, which can ultimately lead to unresolved issues down the line. It’s a stark reminder that AI-generated contracts require close scrutiny to avoid potential pitfalls.

We asked AI to redo this block and got the following:

The document hasn’t improved much. Here’s why:

A well-crafted contract should clearly spell out what each party agrees to do right from the start so the subject of the contract and the parties’ obligations are plain as day.

But in this case, the information is scattered across different points, and it’s unclear what’s expected of each party. It seems like the service provider is shouldering more responsibilities than the client, which could make them hesitant to sign on the dotted line.

Moreover, our legal team didn’t spot any clauses addressing crucial issues like ensuring a safe working environment for the service provider, transporting cleaners to the restaurant, reporting on completed work, or overtime arrangements.

To make matters worse, the contract lacks specificity on key details like the number of cleaners involved, their responsibilities, and who’s accountable for certain areas. The terms “deep cleaning” and “daily cleaning” are vague and need to be defined so that the services can be deemed “duly completed” later on.

It’s also unclear how the services will be accepted, who’s responsible for subpar services, and whether the service provider will get a chance to redo or complete the services. What about payment reductions for incomplete services? These are all critical questions that need to be addressed.

Moving on, the AI adds a block about the Duration and Termination of this cleaning contract. But will this section bring some much-needed clarity, or will it introduce more confusion?

In point 3.1, the chat’s suggested duration is unclear — is this a one-off contract or an ongoing arrangement? The phrase “will continue until terminated” implies it’s meant for long-term cleaning services, but that wasn’t specified in the original task. This ambiguity needs to be ironed out to avoid confusion.

Moving on to point 3.2, it’s surprising that the contract doesn’t include a separate section on how to send termination notices.

Typically, contracts spell out the details, such as whether notices should be sent by email, hand-delivered, or faxed, and provide specific contact information, such as addresses and email addresses.

Since the chat introduced a termination clause, it should have also included a section on notice delivery. Without it, the notice might not reach the party, leading to a breach of contract terms, delayed payments, penalties, and other headaches.

To illustrate what’s missing, let’s look at an example of a Notice section that should cover all the necessary details. This would provide clarity and ensure that all parties are on the same page.

If you ask AI to rephrase the block about Termination, it will produce the following:

The issue with the absence of a section on contract termination remains, just as in the first version of the text.

Next, AI adds a point about Liability:

This is a real sticking point, and it’s unlikely to fly in practice. Imagine a restaurant patron gets hurt due to subpar cleaning. They’ll sue the restaurant, which will then point to this clause and try to shift the blame to the cleaning service provider. But that’s just not how it works.

Restaurants have management teams that are responsible for ensuring the quality of the cleaning services they hire. If the cleaning doesn’t meet standards, the restaurant’s management needs to take ownership of the issue and be accountable to their customers. It’s unrealistic to expect a cleaning service to take on that level of liability for every restaurant or hotel they work with.

Instead of this clause, what’s needed is a provision that outlines the process for the restaurant management to accept the completed services, request corrections or rework, and so on. By detailing this process, both parties can stay on track with the contract terms and avoid potential legal battles with customers down the line. It’s all about creating a clear framework for accountability and communication.

The next item is Confidentiality:

A confidentiality clause usually has some specific terms, like keeping confidential info under wraps for a set period, say three years. This is typically enough to prevent sensitive info from getting out.

But in this case, the chat’s clause is too broad, covering “any disclosed information” without qualification. This doesn’t align with standard commercial practices, as not all shared info is confidential. By making everything confidential, this clause creates an unrealistic burden on the parties.

To fix this, it’s better to clearly spell out what info is considered confidential under this agreement. This would give the parties a clear understanding of what they need to keep quiet about, making it easier for them to adhere to the confidentiality conditions.

For example, the clause could specify what types of info are confidential, such as business strategies, customer data, or financial information. By being more specific, the parties can ensure that they’re protecting sensitive info without creating unnecessary obstacles.

At the end of the document, there are two more important points:

We mentioned in the initial prompt that the restaurant is located in New York, and the AI did manage to correctly identify the Governing Law. However, it’s worth noting that AI can struggle with references — we’ve seen this before in an AI-generated contract for eviction, where the AI mixed up all sorts of unrelated laws and references.

In this case, the AI got it right, but it’s likely because a) the reference is super simple and doesn’t require any specific law citations, and b) it’s a general phrase that doesn’t give the AI much room to get creative and invent its own laws.

Intellectual property is another critical aspect that needs careful consideration in legal agreements. Ensuring that companies retain control over their intangible assets, such as inventions and trademarks, is essential. Legal safeguards for founders and consultants involved in these processes are also crucial.

And that’s a wrap — the last point goes to…

Why is there a problem here?

The schedule of services and payment terms are crucial parts of the contract — without them, it’s impossible to determine whether the contract is correct. However the chat has separated these into separate appendices and didn’t provide any examples. This makes it difficult to form a complete and accurate contract.

In fact, if you try to search for these appendices as separate points, you’ll likely uncover even more errors. It’s much better when these essential details are included directly in the contract text, where they can be clearly linked to the relevant concepts and conditions. This way, everything is cohesive and easy to understand.

After AI struggled to produce a decent eviction notice (a complex document, to be fair), lawyers thought a simple service agreement would be a breeze. But did AI deliver? No.

In reality, almost every clause of the agreement is missing crucial details that make the document valid. Without these specifics about services, termination, and with a liability clause that’s way off — adding all these issues up, you get a document that’s not even a simple legal template. It’s just a hastily written text by someone who glanced at a legal document once.

As of mid-2024, AI still can’t generate a document that meets basic requirements. And although AI is advancing rapidly, with new versions of Chat GPT and all, it’s not quite there yet. However, AI technologies do enhance the efficiency and effectiveness of legal services, particularly in tasks like e-Discovery and legal research.

So, what’s the solution?