Notion AI
The integration is real, the capability ceiling is low, and the tool will not tell you when you've hit it.
It fails quietly on anything deeper.
Truncating long documents without warning, inventing names in meeting drafts, and flattening every voice it edits — and it sounds complete when it isn't.
Formatting, boilerplate, low-stakes drafts.
Use for formatting, boilerplate, and low-stakes drafts. Switch tools the moment the task needs depth.
How the 58 was built.
Each metric is scored 1–10 and multiplied by its weight. Reliability and core use-case fit carry more weight than onboarding convenience or visual polish — which is exactly where Notion AI's surface strength stops paying off.
Four phases, same rig.
Every tool runs the same four-phase gauntlet. No phase is skipped, and the score is only assigned once all four are complete.
Zero-friction entry, and it's genuinely true — one click from an existing workspace, no account, no new interface, spacebar surfaces the panel. Fastest activation of any tool tested. First outputs looked polished. But "Improve writing" on my own copy already hinted at the pattern: it looked better and read worse — the voice was gone.
The gap between surface polish and real capability. A 6,000-word document summarised to only its opening ~1,400 words with no truncation warning. A meeting-agenda draft invented two attendee names not in the workspace. "Improve writing" flattened voice 0/5. The score of 58 reflects a consistent pattern, not isolated failures.
Deployed across three client workspaces; it stayed in rotation for one (low-stakes internal ops) and was quietly dropped from the other two. Reliable for reformatting notes and boilerplate first drafts, nothing beyond. The insight: the convenience premium is for proximity, not capability — you pay £10/member/month to not switch tabs.
The integration is real, the capability ceiling is low, and the tool will not tell you when you've hit it. Fine for low-stakes drafting; misleading on anything that needs depth, because it sounds complete when it isn't.
What the data showed.
Headline measurements from the run. Every number traces to a logged observation in the evidence log below.
Across five "Improve writing" samples, every output lost its voice — contractions removed, rhythm flattened (Day 10).
A 6,000-word strategy doc summarised to ~1,400 words — the opening only, with no truncation warning (Day 7).
One-click activation inside the workspace — no account, no config. Fastest of any tool tested (Day 1).
Where adoption slows down.
The real-world resistance points that determine whether a tool survives daily use or gets abandoned within a week.
| Onboarding — One-click activation, no account or config — fastest entry of any tool tested. Only rises slightly on long-document handling. | Low |
| Output quality — Polished on the surface, thin underneath — degrades sharply on edge cases and long documents. | High |
| Context retention — Loses the thread on documents over ~3,000 words, silently and with no warning. | High |
| Voice preservation — "Improve writing" flattens voice on every sample — 0/5 preserved. | High |
What held. What failed.
- Zero friction — already inside your workflow.
- Short-document summarisation (under ~3,000 words).
- Boilerplate and template generation.
- Reformatting unstructured notes into structured prose.
- First drafts of low-stakes internal documents.
- Document truncation on content over ~3,000 words.
- Hallucinated content in meeting and agenda drafts.
- Destroys voice on every "Improve writing" task.
- No instruction-conflict detection.
- Cannot reason across linked databases.
- Won't flag when it's out of its depth.
Your team already lives in Notion and needs formatting, boilerplate, and short-document summaries under 3,000 words.
You work with documents over 3,000 words — summaries silently truncate — or you need your writing voice preserved (0/5).
Only if your team lives in Notion all day. You're paying for proximity, not capability — worth it for workflow convenience, not for reasoning-heavy work.
Sole tester · CTAI
Notion AI's zero-friction integration is genuinely valuable for formatting, boilerplate, and short summaries. But it fails quietly on anything deeper — truncating long documents without warning, inventing names in meeting drafts, and flattening every voice it edits. You're paying for proximity, not capability. Convenient, but shallow.