AI Draft Modification Rate: How to Measure How Often Your Team Edits AI Replies

BOTrento Team11 November 20267 min read

What the metric is, and what it is not

Define it in one line so everyone on the team counts the same thing:

Modification rate = drafts changed before sending ÷ drafts reviewed, over the same period.

It is a quality signal for the draft, not a score for your people. If someone edits a draft because the customer's situation was unusual, that is the system working as intended: a human is approving what goes out.

It is also not a measure of sales results. A draft sent unchanged can still be a poor reply, and a heavily edited one can win the order. Treat it as a measure of effort and fit, and read it next to metrics such as quote conversion rate.

Count edits in four buckets, not two

"Edited or not" hides too much. A comma fixed and a wrong price corrected are not the same event. Use four buckets:

1. Sent as written. No changes at all.
2. Light edit. Wording, tone, greeting, a shorter sentence. The facts were right.
3. Substantive edit. A fact, price, date, scope or commitment was wrong or missing.
4. Rejected or rewritten. You started again, or the draft was not needed.

Your two headline numbers then become the untouched rate (bucket 1) and the substantive edit rate (buckets 3 and 4). The second one is the number to worry about. Light edits are often personal taste and will never reach zero.

A one-week tracking sheet, step by step

You do not need a dashboard. A shared spreadsheet is enough for the first month.

1. Create five columns: date, enquiry type, bucket (1–4), what changed, who edited.
2. Fill it in at the moment of approval. Doing it at the end of the week means you will forget the reasons.
3. Use a short list for "what changed": tone, missing question, wrong price or scope, wrong date, missing history, too long, wrong language, other.
4. Tag the enquiry type (for example price request, availability check, vague brief, repeat customer). Averages across everything hide where the problem sits.
5. Review once a week for ten minutes. Total each bucket, then sort "what changed" by how often it appears.

Keep the same definitions for at least four weeks before you compare numbers. Changing the rules halfway makes the trend meaningless.

How to read the numbers

There is no universal good rate, and you should be suspicious of anyone who quotes one. Your baseline is your own first two weeks. What matters is the direction and the reasons.

High substantive edits

The AI is missing information. Typical causes: your price logic lives in someone's head, standard questions are not written down, or the enquiry arrives too vague to quote. Fix the inputs, not the person. Writing down a standard question set also cuts back-and-forth, which is covered in the cost of incomplete enquiries.

High light edits, low substantive edits

The facts are right but the voice is not. Collect three or four real replies you are proud of and use them as tone examples. Agree house rules: greeting, sign-off, length, formality.

Very low edits across the board

This can be good, but check two things. First, are people really reading the drafts or just clicking approve? Spot-check a few each week. Second, is the sample too easy, for example mostly repeat customers? A low rate on simple enquiries tells you little about complex ones.

Edits concentrated in one enquiry type

That is the best case for action. If event enquiries with changing scope are edited far more than simple price requests, you know where to add rules or templates. Event and production teams will recognise this pattern; see how event and production companies handle scope-heavy enquiries.

Turn edits into improvements

Every substantive edit is a free lesson. Once a week, take the top two reasons from your sheet and change one thing for each:

• Missing question: add it to your standard question list.
• Wrong price or scope: write the rule down in plain words ("delivery within the city is charged separately").
• Tone: replace the weakest example reply with a better one.
• Missing history: make sure past correspondence is included in the data the draft is built from.
• Wrong language: state the customer's language preference explicitly.

Then check the same reason the following week. If it does not fall, the fix was wrong or the cause is elsewhere. This weekly loop fits neatly into your weekly pipeline review: five of the thirty minutes can go to the edit log.

Remember the other side of the trade-off. If your goal is to reply to enquiries faster, a draft that needs a full rewrite saves you nothing. Track time-to-send alongside the edit rate and you will see both speed and quality at once.

How BOTrento fits this approach

BOTrento Chief of Staff is a managed process for enquiries, quotes and follow-ups. It watches enquiries in your mailbox and uploaded data, shows a morning "Today 1+3" overview on working days, and prepares draft replies, quotes and follow-ups in four languages (Czech, English, German and French). Nothing is sent without human approval, and BOTrento does not send anything by itself. That approval step is exactly where your edit log is filled in.

Be clear about what to expect: BOTrento does not read attachments, only the text of the email, so an enquiry whose details sit in a PDF is a likely source of substantive edits. Log those and ask the customer to put key details in the message body.

AI use is included in the plan, with a monthly usage limit and no automatic extra charges. You can see the flow in the process demo, and pricing lists the plans. If you have questions about data handling, start with the GDPR checklist for AI sales tools; for specific legal cases, speak to a lawyer.

The trial is 14 days, free, with no card and no API key of your own. It starts at your first successful data upload (a mailbox connection or an import), and you get 7 days to prepare. That is long enough to run the tracking sheet above on your own enquiries and see your real baseline.

A simple weekly review note

Paste this into your team chat or notes after each review:

Week of [date]. Drafts reviewed: [n]. Sent as written: [n]. Light edits: [n]. Substantive edits or rewrites: [n].
Top reason for substantive edits: [reason], mostly in [enquiry type].
Change we are making this week: [one specific change].
Check next week: did [reason] appear less often?

One change per reason, one reason at a time. It is slow on purpose, because you can then tell what worked.