AI Troubleshooting Guide

This guide helps support teams diagnose and resolve AI-related issues reported by users.

Looking Up Request Traces

Every AI request in olllo has a unique correlation ID for tracing. When a user reports an issue, ask for:

1. Approximate time of the issue
2. Feature they were using (reflection, accomplishments, goals, etc.)
3. Error message if any was displayed

Finding the Correlation ID

If the user has the correlation ID from an error message:

# Get full request trace
curl -H "Authorization: Bearer $ADMIN_API_KEY" \
  "https://app.olllo.app/api/admin/ai-dashboard/trace/{correlationId}"

If searching by time and user:

# Query dashboard for recent requests by feature
curl -H "Authorization: Bearer $ADMIN_API_KEY" \
  "https://app.olllo.app/api/admin/ai-dashboard?period=1h&feature=reflection"

Understanding Trace Output

{
  "trace": {
    "telemetry": {
      "correlationId": "550e8400-e29b-41d4-a716-446655440000",
      "userId": "user_xxx",
      "feature": "reflection",
      "status": "SUCCESS",
      "durationMs": 1500,
      "modelName": "claude-sonnet-4-5",
      "usedFallback": false,
      "totalTokens": 800
    },
    "safetyIncident": null,
    "retryAttempts": []
  }
}

Common Error Codes

Rate Limiting Errors

Support Response: “You’ve reached your usage limit for now. This resets automatically. Please try again in a few minutes.”

Provider Errors

Support Response: “We’re experiencing temporary issues with our AI service. Your data has been saved and we’ll process it shortly.”

Safety Errors

Support Response: “Your content was flagged by our safety system. Please review and try rephrasing. Use @mentions for names.”

System Errors

Troubleshooting by Symptom

”AI response was slow”

1. Check durationMs in the trace
2. Normal: < 5 seconds, Slow: 5-15 seconds, Very slow: > 15 seconds
3. Check if usedFallback: true (fallback can be slower)
4. Check retryAttempts - multiple retries add latency

If consistently slow: Escalate to engineering with sample correlation IDs.

”AI response was cut off”

1. Check finishReason in trace
2. "length" = Response hit token limit (normal for long content)
3. "interrupted" = Stream was interrupted (check interruptReason)
4. "safety_block" = Content was blocked mid-stream

Resolution: For length limits, advise user to break into smaller requests.

”AI gave an unhelpful response”

1. This is subjective but may indicate:
   • Prompt issue (engineering may need to tune)
   • Model degradation (compare recent quality)
   • User expectation mismatch
2. Collect specific examples for product team

”Feature says AI is unavailable”

1. Check if system is in degraded mode
2. Verify the specific feature’s status
3. Check for ongoing incidents
4. If isolated to one user, check their rate limit status

Safety Incident Investigation

When a user’s content is blocked:

Viewing the Safety Incident

curl -H "Authorization: Bearer $ADMIN_API_KEY" \
  "https://app.olllo.app/api/admin/ai-dashboard/trace/{correlationId}"

The safetyIncident field shows:

• incidentType: What triggered the block (PII_DETECTED, HARMFUL_CONTENT, etc.)
• severity: low, medium, high
• safetyReason: Human-readable explanation
• actionTaken: What the system did (blocked, warned, etc.)
• userNotified: Whether user saw an error message

Common Safety Scenarios

PII Detection

• User included real names, emails, or phone numbers
• Advise using @mentions instead of real names
• Example: “@manager” instead of “John Smith”

Content Flagged

• AI detected potentially problematic content
• Review the context - may be false positive
• If legitimate, explain our content guidelines

Escalation Paths

Level 1: Support Team

• Rate limit questions
• General “how to use” questions
• Explaining error messages

Level 2: Engineering Support

• Repeated errors for same user
• Errors with no clear cause
• Performance issues affecting multiple users

Level 3: Engineering On-Call

• Complete AI outage
• Security concerns (data exposure, etc.)
• Safety system failures

Information to Include in Escalations

1. Correlation ID(s) - Most important!
2. User ID (anonymized if needed)
3. Timestamp of issue
4. Feature affected
5. Error code if any
6. User’s description of the problem
7. Steps already taken

Quick Reference

API Endpoints

Status Codes

Feature Names

Contact

• Slack: #support-escalations
• Email: support-engineering@olllo.app
• PagerDuty: For Sev 1 issues only