JOIN THE COMMUNITY
Crash Test Report CT001
March 20264 / 4 phases complete
Research / Answer Engine 14-Day Run

Perplexity AI

Useful tool, wrong mental model. Use it for orientation. Use something else for conclusions.

Perplexity AI scored 70/100 (Mixed) in CTAI’s 14-day crash test. It was fast for topic orientation, but unreliable for conclusions. In one AI-regulation response, 3 of 8 citations were fabricated from title and preview text. That is one logged response, not an estimate of Perplexity’s overall citation error rate.

: I published 71. The calculated score is 70. The verdict band is unchanged.

Primary Failure Point

One response had 3 of 8 citations fabricated.

In one AI-regulation response, three citations were fabricated from title and preview text. The interface presented paywalled and preview-only sources alongside verified sources. A contradictory follow-up also reversed earlier claims without flagging the inconsistency.

Key Signal One-response citation check
3/8

On Day 6, 3 of 8 citations in one AI-regulation response were fabricated from title and preview text. This is not an overall error rate.

Decision Snapshot Best Use

Fast orientation and threaded follow-up.

Use for fast topic orientation and thread-based follow-up — not for conclusions you need to trust.

01 Weighted Breakdown

How the 70 was built.

Each metric is scored 1–10 and multiplied by its weight. Reliability and core use-case fit carry more weight than onboarding convenience or visual polish — so a tool can't buy a high score by nailing the basics and failing where it matters.

Onboarding×10%
9/10
UX Clarity×10%
8/10
Core Use Case×15%
7/10
Integration×15%
7/10
Output Quality×15%
6/10
Reliability×15%
5/10
Cost / Value×10%
7/10
Operator Ceiling×10%
8/10
02 Phase Findings

Four phases, same method.

Every tool follows the same four-phase method. No phase is skipped, and the score is only assigned once all four are complete.

Initial Impact

First contact

Days 1–3 · onboarding

The speed is real and setup took under two minutes with no account required. The citation UI creates immediate credibility — but I caught myself trusting output more because it looked cited, before checking whether the citations were accurate. Copilot mode's clarifying questions were a genuine plus.

Stress Testing

Under pressure

Days 5–7 · adversarial

In the response checked, three citations led to paywalled articles and summaries derived from previews, presented with the same UI as verified sources. A contradictory follow-up also reversed earlier claims without flagging the inconsistency.

Observe & Document

Real workflow

Days 10–14 · daily use

Used as a daily briefing tool it saved 12–15 minutes per day on orientation, at a cost of 2–3 corrections per session. Thread-based follow-up is best-in-class. But multi-step reasoning is retrieval in disguise — it finds and stitches, it doesn't synthesise. The ceiling is orientation, not analysis.

Verdict & Report · 70

The call

All phases logged

A useful tool with the wrong mental model attached to it. Fast for orientation, unreliable for conclusions. The most insidious risk is habit formation: the citation UI quietly trains you to trust before you verify.

03 Instrument Readout

What the data showed.

Headline measurements from the run. Every number traces to a logged observation in the evidence log below.

3 of 8
Citations fabricated · one response

On Day 6, three citations in one AI-regulation response were fabricated from title and preview text. This is one observation, not an overall rate.

12–15 min/day
Time saved · daily briefing

Net positive for low-stakes orientation — offset by 2–3 manual corrections per session (Day 11).

1/10
Setup friction

Under two minutes, no account required. First result in under three seconds (Day 1).

Evidence by phase10 events · from the log below
Initial Impact 3
Stress Testing 3
Observe & Document 4
04 Evidence Log

Logged, as it happened.

Every score traces to an observation logged during the run.

10 events · 4 fail / 2 flag / 4 pass · nothing reconstructed
WhenPhaseObservationResult
Day 1Initial ImpactOnboarding frictionless, no account required. First search returned results in under three seconds with inline citations.Pass
Day 2Initial ImpactCopilot mode prompted a clarifying question on a vague research prompt before answering — asked recent news vs foundational background.Pass
Day 3Initial ImpactSource-attribution UI creates a strong sense of credibility. Noticed increased trust in output before verifying the citations.Flag
Day 5Stress TestingUK media-regulation query returned "34% of UK adults cannot identify sponsored content," attributed to a DCMS report that does not contain the figure.Fail
Day 6Stress TestingChecked 8 citations in one AI-regulation response; 3 citation summaries were fabricated from title and preview text while the underlying articles were inaccessible behind paywalls.Fail
Day 7Stress TestingContradictory follow-up prompt — Perplexity agreed with the new framing and argued against its own prior position without flagging the reversal.Fail
Day 10Observe & DocumentMulti-step reasoning queries returned thin or evasive answers. Confirmed: retrieval with a summarisation layer, not synthesis across sources.Flag
Day 11Observe & DocumentOne-week daily-briefing integration saved 12–15 min/day, requiring 2–3 spot-check corrections per session. Net positive at a low accuracy bar.Pass
Day 13Observe & DocumentThread-based follow-up handled five progressive questions on one topic without losing context. Best-in-class conversational refinement.Pass
Day 14Observe & DocumentAccepted a Perplexity answer on a well-known topic that was wrong on a detail I'd normally question. Citation UI had lowered my verification instinct.Fail
05 Operator Friction

Where it fought me.

The real-world resistance points that determine whether a tool survives daily use or gets abandoned within a week.

AreaFrictionSeverity
OnboardingZero friction — account not required, first result in seconds.Low
Daily useFast and fluid for orientation tasks.Low
Edge casesConfident output on under-specified input, with no guardrails in standard mode.High
AccuracyEvery citation requires manual verification — which defeats the purpose.High
Team useUseful for briefings only if the team knows the verification requirement.Medium
06 The Balance

What held. What failed.

What held
  • Fast orientation on unfamiliar topics.
  • Copilot mode asks clarifying questions before answering.
  • Thread-based follow-up questioning is best-in-class.
  • Clean interface with zero-friction entry.
  • Daily-briefing use case is net time-positive when the verification bar is low.
What failed
  • Cites paywalled sources it hasn't accessed — fabricates summaries from preview text.
  • Hallucinated statistics attached to real document references.
  • No consistency checking between turns — will argue against its own prior claims.
  • Confident output on under-specified input in standard mode.
  • Outdated content presented as current with no recency caveat.
  • Multi-step reasoning is retrieval in disguise — can't synthesise across sources.
Use it if

You need fast topic orientation and can verify key claims yourself — net time-positive when accuracy requirements are low.

Skip it if

You cannot tolerate a single logged response with 3 of 8 citations fabricated, or you need conclusions you can trust without checking the sources.

Would I pay?

Pro — only if daily research briefings are part of your workflow. The free tier covers most use cases adequately.

07 Operator Verdict
▲ Mixed
70/100

Perplexity is genuinely fast for orientation and best-in-class at threaded follow-up. But underneath the citation UI it's retrieval with a summarisation layer — it finds and stitches, it doesn't reason. Worse, the interface trains you to trust before you verify. Use it to find the right questions, not the right answers.

Woody
WoodyFounder · Sole tester · CTAI
Did you find this useful? Share
Frequently asked questions Perplexity AI

Questions about this crash test.

Is Perplexity AI accurate?

CTAI scored Perplexity 70/100 (Mixed) in a 14-day crash test. It was useful for fast orientation, but the evidence did not support relying on it for conclusions. One AI-regulation response had 3 of 8 citations fabricated from title and preview text; this is one logged response, not an overall error rate.

Does Perplexity fabricate citations?

CTAI observed fabricated citation details in one Day 6 AI-regulation response: 3 of 8 citations were based on title and preview text rather than verified article content. That single response does not measure the overall frequency of citation errors.

What is Perplexity best used for?

In CTAI’s run, Perplexity was useful for fast topic orientation and follow-up questions. Daily briefing use saved an estimated 12–15 minutes a day, with 2–3 corrections per session. Open and check the sources before relying on conclusions.

Tested by
Woody
Woody
Founder · Sole tester · CTAI

Solo operator and the Crash Test Dummy behind CTAI. I put AI to work. No sponsors, no hype.

Let's connect