Reviewing AI-written copy before it goes public

A language model writes sentences that read as checked whether or not anyone checked them. Google's guidance on generative AI gives the reason: generative models do not retrieve facts, they predict a likely sequence of words from their training data, so outputs can contain inaccuracies. Review is therefore a production step with its own method, and this page lays out seven steps for AI-drafted articles and help pages, including a claim ledger and a script that tests quotes against live sources.

What the review has to find

Errors in a model draft rarely look like errors. They look like ordinary sentences. Six kinds account for most of what a reviewer finds.

Google's page names the metadata point explicitly: the manual review covers title elements, meta descriptions, structured data and image alt text, since those can appear in Search results.

Step 1: Freeze the text you review

Review one fixed version. Commit the draft to version control, or save it as a dated file, and write down the commit ID or file name on the ledger you are about to create. Any edit made after review reopens the edited lines. Pasting a paragraph back into a chat tool for polishing produces new text that has not been reviewed, however small the change appears.

Step 2: Turn the draft into a claim ledger

A claim is any sentence a reader could act on, repeat or quote, and which could therefore be wrong. Opinions stay with their author. Everything else gets a row: an ID, the sentence, a type (fact, figure, quote, procedure, date, link), the source address, the quoted words from the source, the date you read it, and a status of keep, rewrite or delete.

Here is a fictional five-sentence excerpt from a draft:

Generative models predict a likely sequence of words rather than retrieving facts. Google requires an AI disclosure on every page. Lighthouse flags "click here" as non-descriptive link text. The helpful content system was introduced last year. Pages with a disclosure rank 40 percent higher.

Each sentence becomes a row, and for each one the reviewer finds the source that should contain the supporting words. The script below fetches each source and reports whether the quoted words appear on the page. It uses Node 18 or later, which includes fetch.

// check-quotes.mjs   usage: node check-quotes.mjs ledger.json
// Row: { "id": "c1", "url": "https://...", "quote": "words from the source" }
import { readFileSync } from 'node:fs';

const norm = (s) => s
  .replace(/<script[\s\S]*?<\/script>|<style[\s\S]*?<\/style>/gi, ' ')
  .replace(/<[^>]+>/g, ' ')
  .replace(/&nbsp;/g, ' ').replace(/&amp;/g, '&')
  .replace(/&#39;|&rsquo;|[’‘]/g, "'").replace(/&quot;|[“”]/g, '"')
  .replace(/\s+/g, ' ').toLowerCase().trim();

const report = (...cols) => console.log(cols.join('\t'));
const rows = JSON.parse(readFileSync(process.argv[2], 'utf8'));
for (const row of rows) {
  try {
    const res = await fetch(row.url, {
      redirect: 'follow',
      headers: { 'user-agent': 'Mozilla/5.0 (claim-check)', 'accept-language': 'en' },
    });
    if (!res.ok) { report(row.id, 'FAIL', 'HTTP ' + res.status, row.url); continue; }
    const found = norm(await res.text()).includes(norm(row.quote));
    report(row.id, found ? 'OK  ' : 'FAIL', found ? 'quote found' : 'quote not on page', res.url);
  } catch (e) {
    report(row.id, 'FAIL', e.cause?.code || e.message, row.url);
  }
}

The accept-language header is there because of something observed while testing this page. A first run of the same ledger without it followed a redirect to a localized copy of a Google documentation page (?hl=...), and an English quotation that exists on the English page was reported missing. The header fixed it, and the script prints the final address after redirects so you can see which copy was read.

Run against the five rows of the excerpt, with the quotes the reviewer chose, the output was:

c1	OK  	quote found	https://developers.google.com/search/docs/fundamentals/using-gen-ai-content
c2	FAIL	quote not on page	https://developers.google.com/search/docs/fundamentals/using-gen-ai-content
c3	OK  	quote found	https://developer.chrome.com/docs/lighthouse/seo/link-text
c4	OK  	quote found	https://developers.google.com/search/blog/2023/02/google-search-and-ai-content
c5	FAIL	quote not on page	https://developers.google.com/search/docs/fundamentals/creating-helpful-content

A match is not a verdict, so the human reads each line against the source:

The script has clear limits. It proves that a page contains some words. It does not prove the words are in context, still current, or what the draft meant, and it cannot read pages that build their text with JavaScript. Row c4 is the reminder: a verbatim match and a wrong sentence can coexist.

Step 3: Rank the sources and read the primary one

A source supports a claim only if it is the origin of the information or a faithful report of it. Sort what you find:

• Primary: the standard, the vendor's own documentation, the statute or regulation text, the product itself behaving in front of you.
• Secondary: a tutorial or news item that summarizes a primary source. Use it to locate the primary source, then read that.
• Not a source: another AI-written page, the model's own reassurance when asked whether it is sure, and any page that repeats your claim in the same words, which suggests a common origin.

A reference list that arrives with the draft is a set of claims too. Open every address and confirm that the page exists and says what the sentence says. An address that returns 200 proves the page exists, and nothing more.

Step 4: Check what cannot be looked up

Some content has no page to quote.

• Procedures and code. Run each command, click each menu path and paste each snippet into a clean project. Record the version of the software you used.
• Numbers. Recompute totals, percentages and conversions yourself, whatever working the draft shows.
• Time words. Replace "recently", "now", "last year" and "the latest version" with a date, or delete them. Note the date shown on each source, and treat any page you read as true only for that date. For topics where facts change often, set a date to re-review.
• Names and quotations. Check spelling against the source. A quotation must match the source word for word, or it should be a paraphrase without quotation marks.

Step 5: Review what readers see outside the body

Search results can show text you never reread: the title, the meta description, image alt text and structured data. Check that the title and description describe what the page delivers (see titles, descriptions and snippets). Check that each alt text states the image's purpose rather than a literal description. Google's structured data guidelines add three rules that suit this step: provide up-to-date information, do not mark up content that is not visible to readers, and do not mark up irrelevant or misleading content. If the JSON-LD was generated along with the copy, compare every field with the visible page, as covered in structured data for a person and a small company.

Step 6: Edit for use

Accurate text can still be useless. Google's helpful-content questions double as an edit test. Does the page go beyond the obvious? If it draws on other sources, does it avoid simply rewriting them and add something of its own? Will the reader need to search again for better information? For each paragraph, delete it mentally and ask what the reader loses. If the answer is nothing, delete it for real. The prompt side of this problem is covered in writing a prompt that produces a better first draft; fixing the brief is cheaper than rewriting the output.

Step 7: Sign off, byline and disclose

Assign one person who answers for the page. Their name goes on the ledger with the date and the commit ID from Step 1. Three points from Google's documentation shape the public side:

1. A byline helps readers understand who created the content, and it should lead to real information about the author. Fabricated creator profiles, including made-up names, AI-generated headshots or false credentials, are described as a form of deception.
2. Giving an AI tool the byline is probably not the best way to show that AI was involved, according to Google's 2023 FAQ. Describe the tool's role in a note instead.
3. A disclosure is useful where a reader might wonder how the content was created. Say what the tool did and what a person verified. A fictional example: "Drafted with an AI tool. Every factual claim was checked against the sources listed below by Maya Okonkwo, reviewed 14 March."

When an edit arrives after sign-off

Sign-off covers the frozen version and nothing later. When someone changes the page, compare it with the reviewed version, for example with git diff --word-diff <reviewed-commit> HEAD -- article.md. Every changed sentence goes back through steps 2 to 5, and the ledger gets new rows with new dates. Unchanged sentences keep their rows. A change that looks cosmetic, such as swapping "can" for "will" or rounding a figure, alters what the sentence claims, so it counts as a change. Finally, give time-sensitive pages a re-review date in the ledger, so that a claim read once in October is read again before it quietly goes out of date.

How much review a page needs

Review effort should follow the cost of being wrong. Google says its systems give even more weight to strong E-E-A-T for topics that could significantly affect health, financial stability or safety, and that content on such topics must be highly accurate and consistent with established expert consensus.

Review time also limits volume. Suppose a careful review of one page takes an hour (a fictional figure): forty pages a week then need forty hours of review. Google's generative AI page warns that generating many pages without adding value for users may violate its spam policy on scaled content abuse. AI-generated content and search: what Google says reads that policy text and its neighbours side by side.

Checklist

• The reviewed text is frozen and identified by a commit ID or dated file.
• Every checkable sentence has a ledger row with a source, a quotation and the date you read it.
• Quotes were tested against the source, and a human compared the meaning, not just the words.
• Procedures were run, numbers recomputed, relative dates replaced and links opened.
• Title, description, alt text and structured data were read against the final body.
• One named person signed off, the byline is a real person, and the disclosure says what the tool did.