Building a Claude Code Skill to Fact-Check the News

The problem

A LinkedIn post: "The world's largest natural hydrogen reserve discovered in Moselle, France! Moselle, the new center of the world!", with a Sciences et Avenir visual and rocket emojis. True, overstated, made up?

To find out: track down the original article, identify who made the discovery and in which lab, verify the numbers, check whether fact-checkers already covered it, compare with other deposits worldwide, and detect presentation biases — typically the conditional that turns into a flat assertion from one repost to the next. One to two hours per news item.

A skill already existed for scientific claims (/science-check), but news fact-checking is a different job: you're not verifying a PubMed study, you're verifying a headline. Hence /fact-check.

What it produces

/fact-check "the world's largest natural hydrogen reserve" in Moselle

Or directly with a screenshot: /fact-check /path/to/screenshot.png. Two minutes later:

# Fact Check

**Claim:** "The world's largest natural hydrogen reserve" in Moselle,
   discovery "unique in the world"
**Source:** Sciences et Avenir — Reliability: Tier 2
**Overall verdict:** OVERSTATED
**Confidence:** HIGH

## Summary

The natural hydrogen discovery in Moselle is real and scientifically
documented by CNRS. The measured concentrations (15% at 1,100m) are
genuinely unprecedented. However, "largest reserve in the world" rests
on theoretical estimates (34-46 Mt) that the researchers themselves
present in the conditional. "Unique in the world" is misleading: other
deposits exist (Mali, Australia, Oman).

## Claim breakdown

Sub-claim                    │ Verdict      │ Confidence
─────────────────────────────┼──────────────┼────────────────────────
H₂ deposit in Moselle        │ TRUE         │ HIGH (CNRS, FDE, AFP)
"Largest reserve"            │ MOSTLY TRUE  │ MEDIUM (conditional)
"Unique in the world"        │ MISLEADING   │ HIGH (Mali, Australia)
"New center of the world"    │ OVERSTATED   │ HIGH (LinkedIn addition)

## Detected biases

- Conditional removed: CNRS headline reads "A giant deposit?"
  with a question mark, reposts drop the "?"
- Resource/reserve confusion: 34-46 Mt = theoretical resource,
  not proven exploitable reserve
- "Unique in the world": unique by concentration, not by the
  existence of the phenomenon (Mali since 1987)

## Sources consulted (12)
 [1] CNRS Le journal — "A giant hydrogen deposit?"
 [2] Ministry of the Economy — PER Trois-Évêchés permit
 [3] The Conversation — Pironon & de Donato (2023)
 ...

The verdict isn't "FALSE" but "OVERSTATED". The discovery is real, the numbers plausible, but the media presentation erases the uncertainty: CNRS writes "could be" with a question mark, and three reposts later it's a bold assertion with rocket emojis.

Why not just extend /science-check?

Two jobs, two skills.

The Google Fact Check API

This is the big addition over /science-check. Before launching web searches, query the global database of professional fact-checks (AFP Factuel, Snopes, PolitiFact, Les Décodeurs). If someone already verified the claim, better to know before spinning up 4 agents.

The API key is free: Google Cloud Console → create a project → enable "Fact Check Tools API" → create a key.

# Fish
set -gx GOOGLE_FACTCHECK_API_KEY "your_key_here"

# Bash/Zsh
export GOOGLE_FACTCHECK_API_KEY="your_key_here"

curl -s "https://factchecktools.googleapis.com/v1alpha1/claims:search\
?query=natural+hydrogen+Moselle\
&languageCode=fr\
&key=${GOOGLE_FACTCHECK_API_KEY}" | python3 -m json.tool

If a professional fact-check exists, you get the verdict, the fact-checker's name and a link to the full article. Otherwise — as with the Moselle hydrogen — the response is {}. Both are useful: either a shortcut, or confirmation you're in uncharted territory. If the key isn't configured, the skill goes straight to web search, no error.

The MCP servers

The skill reuses the same MCPs as /science-check:

• PubMed MCP (mcp-simple-pubmed) — direct access to PubMed's Entrez API
• Paper Search MCP (paper-search-mcp) — multi-source search: PubMed, arXiv, bioRxiv, Google Scholar

Agent D (context and nuance) uses them when the topic warrants it. For a political or economic fact-check, it falls back to WebSearch. If you already have the ~/.claude/mcp.json from /science-check, nothing to add.

The 8-phase workflow

Phases 1-2, extraction and classification. Claude identifies the input type (URL, text, screenshot), extracts the key claims, and classifies the source on a 5-level grid: a CNRS article is Tier 1, a LinkedIn post Tier 4.

Phase 3, PRE-CHECK. Query the Google Fact Check API before mobilising agents. If a professional verdict exists, it's folded in as a Tier 2 source.

Phase 4, four agents in parallel. Agent A hunts raw facts in French and English, Agent B official sources (CNRS, BRGM, ministries), Agent C counter-arguments and criticism, Agent D scientific context via the MCPs.

Why 4 and not 3 like science-check? In science, PubMed is THE source. In news, the primary source can be anywhere: a CNRS press release, a Ministry of the Economy permit, a Council of State ruling. Without agent B dedicated to institutions, early fact-checks systematically missed those.

Phases 5-6, deep dive and cross-validation. Claude always traces back to the primary source — the CNRS article itself, not the local-radio repost. That's where you find the question mark in the headline ("A giant hydrogen deposit in Lorraine**?**") and the systematic conditional, both gone from every repost.

Phase 7, bias detection. The original article goes through a pattern grid:

• Presentation bias: clickbait, unjustified superlatives, numbers without context
• Reasoning bias: correlation/causation, cherry-picking, overgeneralisation
• Omission bias: missing context, erased uncertainty, unmentioned risks
• Source bias: conflict of interest, echo chamber, press release in disguise

Phase 8, synthesis with ultrathink. The ultrathink keyword in SKILL.md activates extended thinking. That's what makes "TRUE in substance but OVERSTATED in presentation" possible instead of a binary call.

The architecture: 6 files, not one big one

The skill lives in ~/.claude/skills/fact-check/ (see on GitHub):

fact-check/
├── SKILL.md              # Main instructions (148 lines)
├── VERDICT_SCALE.md      # 8-level verdict scale
├── SOURCE_RELIABILITY.md # Source reliability grid (5 tiers)
├── BIAS_PATTERNS.md      # Bias and manipulation patterns
├── REPORT_TEMPLATE.md    # Final report template
└── API_INTEGRATION.md    # Google Fact Check API integration

Progressive disclosure: SKILL.md stays short, Claude loads references on demand. Five reference files instead of three for /science-check, because news fact-checking needs more reading grids — cramming it all into three files produced an unreadable mess.

The YAML frontmatter tells Claude when to trigger the skill, the markdown body how to work:

name: fact-check
description: 'Verifies the accuracy of a piece of information, article or claim
  by cross-referencing web sources, scientific papers, fact-checking APIs and
  official positions...'
user-invocable: true
argument-hint: '[URL, text or description of the claim to verify]'
allowed-tools:
  - Agent
  - Bash
  - Read
  - WebSearch
  - WebFetch
  - mcp__pubmed__search_pubmed
  - ...

Three points on the frontmatter:

The description is long, deliberately. Claude tends to under-trigger skills. Listing the domains (science, technology, energy, economics, geopolitics) and the possible phrasings ("whether some info is true, reliable, exaggerated or fake") pushes it to trigger at the right moment.

$ARGUMENTS rather than AskUserQuestion. The first version always asked the user to confirm the claim: redundant. When you type /fact-check "Moselle hydrogen", Claude already has the argument. The AskUserQuestion fallback only applies to a bare /fact-check.

Read in allowed-tools. That's what makes screenshots work: Claude Code is multimodal, and a Read on a .png returns the visual content. The skill can therefore fact-check a screenshot of a tweet or LinkedIn post directly.

The reference files: VERDICT_SCALE.md (8-level scale, far finer than binary true/false — an article can be TRUE in substance and OVERSTATED in presentation, and both must be flagged), SOURCE_RELIABILITY.md (5-tier grid), BIAS_PATTERNS.md, REPORT_TEMPLATE.md (enforced report structure — without it Claude skipped whole sections, notably the biases one time in three) and API_INTEGRATION.md.

Testing

Restart Claude Code (skills load at startup), then:

/fact-check coffee increases cancer risk
/fact-check https://www.example.com/dubious-article
/fact-check /path/to/screenshot.png

Expect 1 to 3 minutes. Well-covered topics run faster; niche topics require the agents to dig deeper.

What this teaches

The conditional makes all the difference. CNRS says "could be the largest reserve". Sciences et Avenir puts quotes around "unique in the world". The LinkedIn post turns it into "Moselle, the new center of the world!". Every repost drops one conditional — insidious, and exactly what the skill detects.

Four agents instead of three for news. The agent dedicated to institutional sources is what separates news fact-checking from scientific fact-checking: the primary source can hide in a PDF on a government site.

The Google Fact Check API is underused. Free, simple, and sometimes it saves 90% of the work. On technical topics it returns {} — but knowing you're the first to verify is information in itself.

Omission biases are the real danger. An article doesn't need to lie to mislead. Failing to say that the 46 million tonnes are a theoretical estimate, that "reserve" and "resource" aren't the same thing in geology, that the Council of State annulled the same operator's methane project weeks earlier — those omissions change perception entirely, and they're the hardest to detect without reading the primary source.

Limitations

It's a tool, not an oracle, and not a journalist. The skill is only as good as the sources available online and Claude's ability to interpret them. On very recent or very local topics, sources are sometimes missing. On technical topics — geology, nuclear physics — Claude can misread a study (a unit-confusion error has already occurred).

But for the everyday case — a LinkedIn post, a headline, a tweet that sounds too good — two minutes are enough to know whether it's TRUE, OVERSTATED or FALSE. And more importantly, to understand why.