Perplexity AI
Useful tool, wrong mental model. Use it for orientation. Use something else for conclusions.
Perplexity AI scored 70/100 (Mixed) in CTAI’s 14-day crash test. It was fast for topic orientation, but unreliable for conclusions. In one AI-regulation response, 3 of 8 citations were fabricated from title and preview text. That is one logged response, not an estimate of Perplexity’s overall citation error rate.
: I published 71. The calculated score is 70. The verdict band is unchanged.
One response had 3 of 8 citations fabricated.
In one AI-regulation response, three citations were fabricated from title and preview text. The interface presented paywalled and preview-only sources alongside verified sources. A contradictory follow-up also reversed earlier claims without flagging the inconsistency.
On Day 6, 3 of 8 citations in one AI-regulation response were fabricated from title and preview text. This is not an overall error rate.
Fast orientation and threaded follow-up.
Use for fast topic orientation and thread-based follow-up — not for conclusions you need to trust.
How the 70 was built.
Each metric is scored 1–10 and multiplied by its weight. Reliability and core use-case fit carry more weight than onboarding convenience or visual polish — so a tool can't buy a high score by nailing the basics and failing where it matters.
Four phases, same method.
Every tool follows the same four-phase method. No phase is skipped, and the score is only assigned once all four are complete.
First contact
The speed is real and setup took under two minutes with no account required. The citation UI creates immediate credibility — but I caught myself trusting output more because it looked cited, before checking whether the citations were accurate. Copilot mode's clarifying questions were a genuine plus.
Under pressure
In the response checked, three citations led to paywalled articles and summaries derived from previews, presented with the same UI as verified sources. A contradictory follow-up also reversed earlier claims without flagging the inconsistency.
Real workflow
Used as a daily briefing tool it saved 12–15 minutes per day on orientation, at a cost of 2–3 corrections per session. Thread-based follow-up is best-in-class. But multi-step reasoning is retrieval in disguise — it finds and stitches, it doesn't synthesise. The ceiling is orientation, not analysis.
The call
A useful tool with the wrong mental model attached to it. Fast for orientation, unreliable for conclusions. The most insidious risk is habit formation: the citation UI quietly trains you to trust before you verify.
What the data showed.
Headline measurements from the run. Every number traces to a logged observation in the evidence log below.
On Day 6, three citations in one AI-regulation response were fabricated from title and preview text. This is one observation, not an overall rate.
Net positive for low-stakes orientation — offset by 2–3 manual corrections per session (Day 11).
Under two minutes, no account required. First result in under three seconds (Day 1).
Logged, as it happened.
Every score traces to an observation logged during the run.
| When | Phase | Observation | Result |
|---|---|---|---|
| Day 1 | Initial Impact | Onboarding frictionless, no account required. First search returned results in under three seconds with inline citations. | Pass |
| Day 2 | Initial Impact | Copilot mode prompted a clarifying question on a vague research prompt before answering — asked recent news vs foundational background. | Pass |
| Day 3 | Initial Impact | Source-attribution UI creates a strong sense of credibility. Noticed increased trust in output before verifying the citations. | Flag |
| Day 5 | Stress Testing | UK media-regulation query returned "34% of UK adults cannot identify sponsored content," attributed to a DCMS report that does not contain the figure. | Fail |
| Day 6 | Stress Testing | Checked 8 citations in one AI-regulation response; 3 citation summaries were fabricated from title and preview text while the underlying articles were inaccessible behind paywalls. | Fail |
| Day 7 | Stress Testing | Contradictory follow-up prompt — Perplexity agreed with the new framing and argued against its own prior position without flagging the reversal. | Fail |
| Day 10 | Observe & Document | Multi-step reasoning queries returned thin or evasive answers. Confirmed: retrieval with a summarisation layer, not synthesis across sources. | Flag |
| Day 11 | Observe & Document | One-week daily-briefing integration saved 12–15 min/day, requiring 2–3 spot-check corrections per session. Net positive at a low accuracy bar. | Pass |
| Day 13 | Observe & Document | Thread-based follow-up handled five progressive questions on one topic without losing context. Best-in-class conversational refinement. | Pass |
| Day 14 | Observe & Document | Accepted a Perplexity answer on a well-known topic that was wrong on a detail I'd normally question. Citation UI had lowered my verification instinct. | Fail |
Where it fought me.
The real-world resistance points that determine whether a tool survives daily use or gets abandoned within a week.
| Area | Friction | Severity |
|---|---|---|
| Onboarding | Zero friction — account not required, first result in seconds. | Low |
| Daily use | Fast and fluid for orientation tasks. | Low |
| Edge cases | Confident output on under-specified input, with no guardrails in standard mode. | High |
| Accuracy | Every citation requires manual verification — which defeats the purpose. | High |
| Team use | Useful for briefings only if the team knows the verification requirement. | Medium |
What held. What failed.
- Fast orientation on unfamiliar topics.
- Copilot mode asks clarifying questions before answering.
- Thread-based follow-up questioning is best-in-class.
- Clean interface with zero-friction entry.
- Daily-briefing use case is net time-positive when the verification bar is low.
- Cites paywalled sources it hasn't accessed — fabricates summaries from preview text.
- Hallucinated statistics attached to real document references.
- No consistency checking between turns — will argue against its own prior claims.
- Confident output on under-specified input in standard mode.
- Outdated content presented as current with no recency caveat.
- Multi-step reasoning is retrieval in disguise — can't synthesise across sources.
You need fast topic orientation and can verify key claims yourself — net time-positive when accuracy requirements are low.
You cannot tolerate a single logged response with 3 of 8 citations fabricated, or you need conclusions you can trust without checking the sources.
Pro — only if daily research briefings are part of your workflow. The free tier covers most use cases adequately.
Perplexity is genuinely fast for orientation and best-in-class at threaded follow-up. But underneath the citation UI it's retrieval with a summarisation layer — it finds and stitches, it doesn't reason. Worse, the interface trains you to trust before you verify. Use it to find the right questions, not the right answers.
Questions about this crash test.
Is Perplexity AI accurate?
CTAI scored Perplexity 70/100 (Mixed) in a 14-day crash test. It was useful for fast orientation, but the evidence did not support relying on it for conclusions. One AI-regulation response had 3 of 8 citations fabricated from title and preview text; this is one logged response, not an overall error rate.
Does Perplexity fabricate citations?
CTAI observed fabricated citation details in one Day 6 AI-regulation response: 3 of 8 citations were based on title and preview text rather than verified article content. That single response does not measure the overall frequency of citation errors.
What is Perplexity best used for?
In CTAI’s run, Perplexity was useful for fast topic orientation and follow-up questions. Daily briefing use saved an estimated 12–15 minutes a day, with 2–3 corrections per session. Open and check the sources before relying on conclusions.