Quality Indicators
How Threader measures output confidence.
Every AI-generated output in Comms Threader includes confidence scores. These tell you how grounded the output is in your inputs and research. Understanding these scores helps you decide when to refine, when to override, and when outputs are pitch-ready.
Strategic Confidence
The Strategic Confidence score appears on individual tool outputs (like The Story's problem reframe or The Audience's audience analysis). It measures how well the AI output is supported by three factors:
| Factor | What It Measures | How to Improve It |
|---|---|---|
| AI Knowledge | How well the output aligns with established strategic frameworks, category knowledge, and best practices | This is automatic, based on the AI's training data. High scores (90%+) mean the output follows sound strategic principles |
| Client Input | How much specific, actionable information you've provided about the client, brand, market, and brief | Upload documents, add detailed constraints, include research findings, specify objectives clearly |
| Completeness | Whether all required fields are filled and the tool has enough context to generate quality outputs | Fill optional fields when relevant, don't skip uploads, add success metrics if available |
How Scores Are Calculated
Strategic Confidence is a weighted average of multiple factors. The exact weights vary by tool based on what matters most for that type of analysis:
- The Story: AI Knowledge 50%, Client Input 30%, Completeness 20% (problem framing relies heavily on strategic frameworks)
- The Audience: AI Knowledge 40%, Client Input 30%, Segment Quality 20%, Completeness 10% (includes segment-specific validation)
- The Message: AI Knowledge 30%, Client Input 30%, Completeness 28%, Competitive 12% (balanced across multiple positioning factors)
- The Landscape: AI Knowledge 40%, Client Input 30%, Completeness 30% (requires strong category knowledge)
- The Plan & Frame: Cascade-based scoring that inherits from upstream tools (confidence builds on completed pitch flow)
Example: In The Story, if AI Knowledge is 95%, Client Input is 50%, and Completeness is 100%, the overall score is: (0.50 × 95) + (0.30 × 50) + (0.20 × 100) = 82.5%
Why do weights vary by tool? Different strategic tasks require different types of input. The Story needs strong conceptual frameworks (high AI weight), while The Message needs competitive context (balanced across factors). Client input consistently matters across all tools. The more specific information you provide, the higher your confidence scores.
What Scores Mean
| Score Range | Label | Meaning | Action |
|---|---|---|---|
| 85-100% | HIGH | Output is strongly grounded in research and strategic best practices. Ready to use. | Review for accuracy, export if satisfied |
| 70-84% | MEDIUM | Output is sound but could benefit from more client-specific context or research. | Add more detail, upload documents, or use Override to refine |
| Below 70% | LOW | Output may be too generic or speculative. Lacks sufficient grounding. | Upload briefs/research, fill optional fields, or start over with better inputs |
Pitch Confidence
Pitch Confidence appears at the top of your screen in Cascade Mode once you've completed The Story. It tracks the overall quality of your pitch cascade across all five tools.
What it measures: The aggregate confidence of all completed tools. If The Story has 83% confidence, The Audience 88%, and The Message 75%, your Pitch Confidence reflects the average quality across your entire strategic flow.
Pitch Confidence only appears in Cascade Mode. Free Roam mode doesn't cascade data, so there's no cross-tool confidence to track.
Why It Matters
Pitch Confidence helps you spot weak links in your strategic thinking. If your overall pitch confidence is 85% but one tool scores 65%, that's your vulnerability. Strengthen that section before presenting to clients.
Example: Your Story and Audience both score HIGH (85%+), but your Message only scores 68% MEDIUM. This suggests your positioning isn't as well-supported as your problem definition and audience analysis. You might need more competitive research or clearer differentiation.
When to Trust AI Outputs
High Confidence Outputs (85%+)
These are typically safe to use as-is, but always review for:
- Accuracy of extracted facts
- Tone and language fit for your client
- Strategic nuance only you would know
Medium Confidence Outputs (70-84%)
Use these as strong starting points, but plan to:
- Add client-specific examples
- Override generic sections with specific details
- Use Refine to get alternative angles
Low Confidence Outputs (Below 70%)
These need significant work. Either:
- Add substantially more context and regenerate
- Use Override extensively to rewrite sections
- Consider whether you have enough information to brief this tool properly
Common Confidence Issues
Low Client Input Score
Symptom: AI Knowledge 90%, Client Input 40%, Completeness 80%
Fix: Upload client briefs, add budget/timeline constraints, include research docs, specify objectives
Low Completeness Score
Symptom: AI Knowledge 95%, Client Input 75%, Completeness 50%
Fix: Fill optional fields that are relevant, add success metrics, upload supporting documents
Consistently Medium Scores Across Tools
Symptom: All tools score 70-75%
Fix: You're providing adequate input but not excellent input. Add more detail at each step, especially in The Story-better problem definition lifts all downstream tools
Confidence vs Quality
High confidence doesn't mean the AI is right-it means the output is well-supported by your inputs. If your inputs contain wrong information, you'll get confident but inaccurate outputs.
Always validate:
- Factual claims against source documents
- Strategic recommendations against your knowledge of the client
- Competitive insights against reality
Use confidence scores as quality indicators, not replacements for judgment. A 95% confidence score on a badly framed problem is still bad strategy.
Improving Scores Over Time
As you use Threader more, you'll develop a sense for what inputs produce high-confidence outputs:
| If you want higher... | Do this... |
|---|---|
| AI Knowledge | This is automatic and usually high. If low, your request may be unclear or outside strategic norms |
| Client Input | Upload documents, add constraints, specify objectives, include research, fill optional fields with real data |
| Completeness | Fill all required fields, add optional context when relevant, upload supporting documents even when optional |
Technical Notes
Score Persistence
Confidence scores are calculated when outputs are generated and saved with your project. If you edit an output manually (using Override), the score doesn't change-it reflects the AI's original confidence, not your edits.
Cross-Tool Impact
In Cascade Mode, low confidence in upstream tools (like The Story) can lower confidence in downstream tools (like The Audience). If The Story has weak client input, The Audience inherits that weakness because it's building on incomplete context.
Regeneration
Using the Refine button regenerates outputs and recalculates confidence scores. If you added more context since the first generation, the refined output may score higher.