JOIN THE COMMUNITY
Crash Test Report CT002
March 20264 / 4 phases complete
Productivity / Workspace AI 3-Week Run

Notion AI

The integration is real, the capability ceiling is low, and the tool will not tell you when you've hit it.

Notion AI scored 59/100 (Mixed) in CTAI’s crash test. It was convenient for drafts and summaries inside a workspace already in use, but weak on deeper tasks. In one documented test, it summarised only the opening of a 6,000-word strategy document and gave no truncation warning.

: I published 58. The calculated score is 59. The verdict band is unchanged.

Primary Failure Point

One long-document test failed quietly.

In one long-document test, it summarised only the opening of a 6,000-word document and gave no truncation warning. A separate meeting-agenda draft invented attendee names, and writing edits flattened the author's voice.

Key Signal One-document coverage
1,400

In one Day 7 test, a 6,000-word strategy document produced a roughly 1,400-word summary covering only the opening, with no truncation warning. This is a single observed test, not a general coverage rate.

Decision Snapshot Best Use

Formatting, boilerplate, low-stakes drafts.

Use for formatting, boilerplate, and low-stakes drafts. Switch tools the moment the task needs depth.

01 Weighted Breakdown

How the 59 was built.

Each metric is scored 1–10 and multiplied by its weight. Reliability and core use-case fit carry more weight than onboarding convenience or visual polish — which is exactly where Notion AI's surface strength stops paying off.

Onboarding×10%
9/10
UX Clarity×10%
8/10
Core Use Case×15%
5/10
Integration×15%
8/10
Output Quality×15%
4/10
Reliability×15%
4/10
Cost / Value×10%
5/10
Operator Ceiling×10%
5/10
02 Phase Findings

Four phases, same method.

Every tool follows the same four-phase method. No phase is skipped, and the score is only assigned once all four are complete.

Initial Impact

First contact

Days 1–3 · onboarding

Zero-friction entry, and it's genuinely true — one click from an existing workspace, no account, no new interface, spacebar surfaces the panel. Fastest activation of any tool tested. First outputs looked polished. But "Improve writing" on my own copy already hinted at the pattern: it looked better and read worse — the voice was gone.

Stress Testing

Under pressure

Days 7–10 · adversarial

In one test, a 6,000-word document was summarised to only its opening ~1,400 words with no truncation warning. A separate meeting-agenda draft invented two attendee names not in the workspace. "Improve writing" flattened voice 0/5 across five samples. The score of 59 reflects the full test run.

Observe & Document

Real workflow

Weeks 2–3 · daily use

Deployed across three client workspaces; it stayed in rotation for one (low-stakes internal ops) and was quietly dropped from the other two. Reliable for reformatting notes and boilerplate first drafts, nothing beyond. The insight: the convenience premium is for proximity, not capability — you pay £10/member/month to not switch tabs.

Verdict & Report · 59

The call

All phases logged

The integration is real, the capability ceiling is low, and the tool will not tell you when you've hit it. Fine for low-stakes drafting; misleading on anything that needs depth, because it sounds complete when it isn't.

03 Instrument Readout

What the data showed.

Headline measurements from the run. Every number traces to a logged observation in the evidence log below.

0/5
Voice preservation

Across five "Improve writing" samples, every output lost its voice — contractions removed, rhythm flattened (Day 10).

1,400 words
One-document summary

In one Day 7 test, a 6,000-word strategy document produced a ~1,400-word summary of the opening sections, without a truncation warning. This is a single test result, not a general coverage rate.

1/10
Setup friction

One-click activation inside the workspace — no account, no config. Fastest of any tool tested (Day 1).

Evidence by phase9 events · from the log below
Initial Impact 3
Stress Testing 3
Observe & Document 3
04 Evidence Log

Logged, as it happened.

Every score traces to an observation logged during the run.

9 events · 3 fail / 3 flag / 3 pass · nothing reconstructed
WhenPhaseObservationResult
Day 1Initial ImpactZero setup friction. One-click activation from an existing workspace — no account, API key, or config. Spacebar surfaces the AI panel. Fastest activation of any tool tested.Pass
Day 2Initial Impact"Improve writing" on my own paragraph — output looked polished at a glance, but the voice was gone, flattened to generic business prose. Looked better, read worse.Flag
Day 3Initial ImpactSummarise on a short meeting note (under 400 words) — fast and accurate, extracted the three key decisions correctly. Strong on short, structured input.Pass
Day 7Stress TestingSummarised a 6,000-word strategy document. Output reflected only ~1,400 words — the opening sections. Final recommendations absent. No truncation warning; the summary looked complete.Fail
Day 9Stress Testing"Draft meeting agenda" on a project page listing team members — the draft included two attendee names not present anywhere in the workspace.Fail
Day 10Stress Testing"Improve writing" across five samples — every output over-formalised, contractions removed, rhythm flattened. Voice preservation 0/5. Works as a voice eraser, not an editor.Fail
Week 2Observe & DocumentDeployed across three client workspaces (agency ops, client strategy, product roadmap). Stayed in rotation for one — low-stakes internal — and was quietly dropped from the other two.Flag
Week 2Observe & DocumentBest consistent use: reformatting unstructured notes into structured prose, and boilerplate first drafts. Meeting recap bulletsprose reliable; process-checklist draft reliable.Pass
Week 3Observe & DocumentThe convenience premium is for proximity, not capability — £10/member/month to not switch tabs. Legitimate for a specific task type; the mistake is believing proximity equals capability.Flag
05 Operator Friction

Where adoption slows down.

The real-world resistance points that determine whether a tool survives daily use or gets abandoned within a week.

AreaFrictionSeverity
OnboardingOne-click activation, no account or config — fastest entry of any tool tested. The separate long-document test exposed a coverage problem.Low
Output qualityPolished on the surface, thin underneath — the test run found problems on deeper tasks and edge cases.High
Context retentionIn one 6,000-word document test, the summary covered only the opening ~1,400 words, with no warning.High
Voice preservation"Improve writing" flattens voice on every sample — 0/5 preserved.High
06 The Balance

What held. What failed.

What held
  • Zero friction — already inside your workflow.
  • Short-document summarisation (observed to work on a meeting note under 400 words).
  • Boilerplate and template generation.
  • Reformatting unstructured notes into structured prose.
  • First drafts of low-stakes internal documents.
What failed
  • In one 6,000-word document test, later sections were missing from the summary without warning.
  • Hallucinated content in meeting and agenda drafts.
  • Destroys voice on every "Improve writing" task.
  • No instruction-conflict detection.
  • Cannot reason across linked databases.
  • Won't flag when it's out of its depth.
Use it if

Your team already lives in Notion and needs formatting, boilerplate, and summaries of short, structured notes.

Skip it if

You need complete summaries of long documents: in CTAI's test, a 6,000-word document was summarised to its opening ~1,400 words without warning. Also skip it if preserving your writing voice is essential (0/5 across five samples).

Would I pay?

Only if your team lives in Notion all day. You're paying for proximity, not capability — worth it for workflow convenience, not for reasoning-heavy work.

07 Operator Verdict
▲ Mixed
59/100

Notion AI's zero-friction integration is valuable for formatting, boilerplate, and short summaries. In CTAI's run, one long-document summary omitted later sections without warning; a separate meeting draft invented names, and five writing edits flattened the author's voice. Convenient, but shallow on tasks needing depth.

Woody
WoodyFounder · Sole tester · CTAI
Did you find this useful? Share
Frequently asked questions Notion AI

Questions about this crash test.

Is Notion AI worth it?

CTAI scored Notion AI 59/100 (Mixed). It was convenient for formatting, boilerplate and low-stakes drafts inside an existing Notion workspace, but the test found a low capability ceiling on tasks that needed depth.

Does Notion AI handle long documents?

In CTAI’s documented test, a 6,000-word strategy document produced a summary of about 1,400 words covering only the opening sections, with no truncation warning. That is the observed test result, not a universal limit for every document.

Tested by
Woody
Woody
Founder · Sole tester · CTAI

Solo operator and the Crash Test Dummy behind CTAI. I put AI to work. No sponsors, no hype.

Let's connect