AI content quality checks: what our gate tests before a draft merges

An AI content quality check is a rule a script can test without understanding the text: required fields present, banned punctuation absent, link counts right, internal names kept out. Ours are thirteen rules, seven that block a draft and six that only warn, and they run before anything is merged. None of them checks whether a sentence is true.

That last point is the reason for this post. Below is each check, its exact threshold, why it exists, and the gap the gate leaves open. The details were verified against our repository on 3 October 2026. The drafting is done by a tomte, our word for one production agent with one defined job, and this is the gate its drafts must clear.

What the gate is and where it sits

The gate is a short list of mechanical rules run in order on every draft: locally first, then on the pull request, then in CI. It exists so that nothing merges unless it passes every rule a machine can enforce. People are left with judgment, which is the one thing a script cannot do: the topic queue, the rulebook and any new number.

The order of events is fixed: draft, local gates, pull request, CI green, automated merge, deploy, live check. No one approves individual posts, on our blog or on a client install; the gates decide. The table lists every rule and what happens when it fails.

Check	What it tests	Threshold	Result
Frontmatter	Six required fields	title, description, pubDate, author, serviceSlug, proofSlug	Blocks
Build and types	Site compiles, schema valid, types pass	Zero errors	Blocks
Em dash	The character in the body	Zero	Blocks
Banned words	Ten fixed words, plus variants	Zero	Blocks
Link count	Distinct service and proof links	Exactly one each	Blocks
Tomte definition	”our word for” near first use	Within 200 characters	Blocks
Leak scan	Internal names in any text file	Zero hits	Blocks
Title length	Characters	Over 75	Warns
Description	Length, and a digit	100 to 160, one digit	Warns
Service slug	Link matches frontmatter	Must match	Warns
Start links	Links to /start	More than one	Warns
Paragraph length	Words per prose paragraph	Over 95	Warns
Body length	Words, table rows excluded	Under 600 or over 1,700	Warns

Checks that block the draft

Seven rules stop a draft outright: complete frontmatter, a passing build and type check, zero em dashes, no banned words, exactly one service link and one proof link, a definition beside the first mention of a tomte, and a clean leak scan. A failure exits non-zero, so the draft cannot move to a pull request.

Frontmatter. All six fields must be present. The build also validates the schema and coerces the publish date, so a malformed date fails the build instead of shipping a post that sorts wrong. Every downstream surface reads these fields: the sitemap, the feed, the related links.

Em dash. One in the body fails the draft. It is the most reliable visible tell of machine drafting, and a character test costs nothing to run.

Banned words. A fixed list of ten words that every reader of AI output recognises. The match is case-insensitive, catches plurals, -ed and -ing forms and hyphen variants, and strips link URLs first. Our house style lists twelve. The script enforces ten, and CI would not stop either of the other two hype verbs. That is a real gap, and the fix is one line in the list.

Links. Exactly one distinct service link and one distinct proof link. Zero fails, and two different ones fail. Related blog posts are unlimited. The rule keeps each post to one commercial path and one piece of evidence.

Tomte definition. The first use needs “our word for” or “our name for” within 200 characters. A coined term with no definition costs the reader, and it costs any AI engine that quotes the page.

Leak scan. Every line of every text file in the source, public assets, build output and hosting config is checked against a list of internal names. The list is stored encoded in the script, matched as whole words, and the failure message never prints the name it found.

Checks that only warn

Six rules warn without blocking: a title over 75 characters, a description outside 100 to 160 characters or missing a digit, a service link that disagrees with frontmatter, more than one /start link, a prose paragraph over 95 words, and a body under 600 or over 1,700 words. They flag drift. A person decides whether the drift was intended.

These warn because they are judgment calls. The style guide asks for 800 to 1,400 words, so the lint thresholds are deliberately wider and only fire on outliers.

One asymmetry is worth stating. A service link that differs from the frontmatter slug raises a warning. There is no equivalent check for the proof link, so a wrong proof page is caught only by someone reading the page.

What the gate does not catch

The gate does not catch false facts, wrong dates, misquoted sources or a statistic used without its date. A script can count em dashes. It cannot know a number is real. Fact-checking sits outside the gate: it means opening each cited source and comparing the sentence to the page.

The rule that every claim carries a number and a date is a voice rule, not a script. Nothing in the lint tests for it. It lives in the brand document the voice rules come from.

The same is true of the last pass. The anti-tell check covers paragraphs of identical length, a closing heading that restates the post, padded lists and hedged openers, and it is a written checklist, not a script. It follows the idea behind the Wikipedia “Signs of AI writing” page, a living checklist of AI tells grouped by category and maintained by WikiProject AI Cleanup editors. Ours is the same checklist, written into the voice rules the drafting agent follows.

Why the factual hole is the one that matters

Models invent facts at measurable rates even on easy tasks, and search quality guidelines punish unhelpful scale, not AI use itself. So the risk in AI content is wrong or empty text, not the tool that wrote it. That risk is exactly what a mechanical gate cannot see.

The Vectara Hallucination Leaderboard, updated 22 September 2026, tests models on summarising a document they were handed. Rates ran from 1.8% for the best model to 7.0% for Gemini 2.5 Pro, 9.6% for GPT-4o and 10.3% for Claude Sonnet 4. Even with the source text in front of them, models invent roughly 2% to 10% of the time.

Kalai, Nachum, Vempala and Zhang, in “Why Language Models Hallucinate” (4 September 2025), argue that training and evaluation reward guessing over admitting uncertainty. In their test, three models each produced a title and year for a real dissertation, and none was correct.

Google’s Search Quality Rater Guidelines, 11 September 2025 edition, say two things that matter here. Content made at scale with no original value is rated lowest however it was created (section 4.6.5), and using generative AI alone does not set the quality rating (4.6.6). Multiple or significant factual inaccuracies on an informational page make it untrustworthy (4.5).

The Content Marketing Institute’s B2B trends research for 2026, published 8 October 2025 from a survey of 1,015 marketers, found that among organisations using AI content tools, 58% said quality improved and 12% said it decreased.

After merge, CI is advisory

Before merge, the content gates are the real blocker: a red check on the pull request keeps it from merging. After merge they become advisory, because the deploy has already happened. CI on the main branch still runs build, type check, leak scan and lint, and the search ping fires whenever the build succeeded.

A lint failure after merge never skips the IndexNow ping, which tells participating search engines a URL changed. A person fixes the failure in a new pull request, and the post stays live until then.

For the stages around the gate, from queue to live URL check, see our AI content pipeline example. For how gated output performs in search, see AI content automation for SEO.

What the gate leaves for people

The gates decide the merge, on our blog and on client installs. Because they handle fields, links, punctuation and leaked names, people spend their time upstream and at exceptions: choosing topics, writing the rules, signing off new numbers, and fixing what a failed gate or a fix pull request surfaces. Facts and claims stay the hard part, because lint cannot tell a true sentence from a false one.

This is the gate inside our Content Engine: $4,900 setup, 2 to 3 weeks, with an optional $990/mo run fee as of July 2026. Your voice rules become checks the pipeline enforces, and you own the delivered code, data and documentation.

We run the same gate on our own publishing. The inbound machine proof page covers a 363-product B2B catalogue with daily automated publishing, which produced quality inbound within its first month, as of July 2026.

Get a gate like this for your blog

Write down the rules your copy must never break and the claims it may not make, then Start async. You get a written reply within one business day, and there is no call to book.