HH/docs/SLA_SYSTEM_SETUP_AND_TESTING_GUIDE.md

SLA System Setup and Testing Guide

Overview

This guide covers the complete SLA (Service Level Agreement) system for complaints, including setup, configuration, testing, and monitoring.

Table of Contents

1. System Architecture
2. Model Structure
3. SLA Configuration
4. Escalation Rules
5. Testing the System
6. Monitoring and Troubleshooting
7. Production Setup

---

System Architecture

Components

The SLA system consists of the following key components:

1. Complaint Model - Stores complaint details and SLA tracking
2. SLA Configuration - Defines time limits for response/resolution
3. Escalation Rules - Defines when and how to escalate complaints
4. Complaint Updates - Timeline of all complaint activities
5. Celery Tasks - Background jobs for SLA reminders and checks
6. Signals - Automatic triggers on complaint creation and updates

Workflow

Complaint Created
    ↓
Calculate Due Date (based on SLA config)
    ↓
Set First/Second Reminder Times
    ↓
Celery Beat Checks Every 15 Minutes
    ↓
Send First Reminder (if enabled and time reached)
    ↓
Send Second Reminder (if enabled and time reached)
    ↓
Escalate (if overdue and escalation rules match)
    ↓
Update Complaint Timeline

---

Model Structure

Complaint Model

Located in: apps/complaints/models.py

Key SLA-related fields:

class Complaint(models.Model):
    # Basic fields
    reference_number = models.CharField(max_length=50, unique=True)
    hospital = models.ForeignKey(Hospital, on_delete=models.PROTECT)
    department = models.ForeignKey(Department, null=True, on_delete=models.PROTECT)
    category = models.ForeignKey(ComplaintCategory, on_delete=models.PROTECT)
    staff = models.ForeignKey(Staff, null=True, on_delete=models.PROTECT)
    
    # Priority and severity
    priority = models.CharField(choices=PRIORITY_CHOICES, default='medium')
    severity = models.CharField(choices=SEVERITY_CHOICES, default='medium')
    
    # Status and SLA tracking
    status = models.CharField(choices=STATUS_CHOICES, default='open')
    
    # SLA deadlines
    due_at = models.DateTimeField(null=True, blank=True)
    
    # SLA reminder tracking
    first_reminder_sent_at = models.DateTimeField(null=True, blank=True)
    second_reminder_sent_at = models.DateTimeField(null=True, blank=True)
    
    # Escalation tracking
    escalation_level = models.IntegerField(default=0)
    escalated_at = models.DateTimeField(null=True, blank=True)
    escalated_to = models.ForeignKey(
        User,
        null=True,
        blank=True,
        on_delete=models.SET_NULL,
        related_name='escalated_complaints'
    )

SLA Configuration Model

class SLAConfig(models.Model):
    hospital = models.ForeignKey(Hospital, on_delete=models.CASCADE)
    department = models.ForeignKey(Department, null=True, on_delete=models.CASCADE)
    category = models.ForeignKey(ComplaintCategory, on_delete=models.CASCADE)
    
    # Time limits (in hours)
    response_time_hours = models.IntegerField()
    resolution_time_hours = models.IntegerField()
    
    # First reminder
    enable_first_reminder = models.BooleanField(default=True)
    first_reminder_hours_before = models.IntegerField(default=24)
    
    # Second reminder
    enable_second_reminder = models.BooleanField(default=True)
    second_reminder_hours_before = models.IntegerField(default=12)
    
    is_active = models.BooleanField(default=True)

Escalation Rule Model

class EscalationRule(models.Model):
    hospital = models.ForeignKey(Hospital, on_delete=models.CASCADE)
    level = models.IntegerField()
    name = models.CharField(max_length=200)
    
    # Trigger conditions
    trigger_type = models.CharField(
        choices=[
            ('overdue', 'Overdue'),
            ('after_reminder', 'After Reminder Sent'),
        ],
        default='overdue'
    )
    hours_overdue = models.IntegerField(null=True, blank=True)
    hours_after_reminder = models.IntegerField(null=True, blank=True)
    
    # Escalation target
    escalate_to = models.CharField(
        choices=[
            ('department_manager', 'Department Manager'),
            ('hospital_admin', 'Hospital Administrator'),
            ('regional_admin', 'Regional Administrator'),
        ],
        default='department_manager'
    )
    
    # Priority/severity filters
    applicable_priorities = models.JSONField(default=list)
    applicable_severities = models.JSONField(default=list)
    
    is_active = models.BooleanField(default=True)

Complaint Update Model

class ComplaintUpdate(models.Model):
    complaint = models.ForeignKey(Complaint, on_delete=models.CASCADE)
    
    update_type = models.CharField(
        choices=[
            ('status_change', 'Status Change'),
            ('comment', 'Comment'),
            ('escalation', 'Escalation'),
            ('reminder_sent', 'Reminder Sent'),
            ('sla_breach', 'SLA Breach'),
        ]
    )
    
    old_status = models.CharField(max_length=50, blank=True)
    new_status = models.CharField(max_length=50, blank=True)
    message = models.TextField()
    created_by = models.ForeignKey(User, null=True, on_delete=models.SET_NULL)

---

SLA Configuration

Setting Up SLA Configurations

SLA configurations define time limits for responding to and resolving complaints based on:

• Hospital
• Department (optional - applies to all departments if null)
• Complaint Category

Configuration Steps

1. Access Admin Panel

   http://localhost:8000/admin/complaints/slaconfig/
   

2. Create SLA Configuration

   • Select Hospital
   • Select Category (required)
   • Select Department (optional)
   • Set Response Time (hours)
   • Set Resolution Time (hours)
   • Configure First Reminder
   • Configure Second Reminder
   • Mark as Active

3. Example Configuration

   For critical complaints:

   • Response Time: 4 hours
   • Resolution Time: 24 hours
   • First Reminder: 2 hours before due
   • Second Reminder: 1 hour before due

   For medium complaints:

   • Response Time: 24 hours
   • Resolution Time: 72 hours
   • First Reminder: 24 hours before due
   • Second Reminder: 12 hours before due

Default Configurations

The system includes default SLA configurations for all complaint categories:

Category	Response Time	Resolution Time	First Reminder	Second Reminder
Clinical Care	24 hours	72 hours	24 hours	12 hours
Staff Behavior	12 hours	48 hours	12 hours	6 hours
Facility	24 hours	72 hours	24 hours	12 hours
Wait Time	4 hours	24 hours	4 hours	2 hours
Billing	24 hours	72 hours	24 hours	12 hours
Communication	12 hours	48 hours	12 hours	6 hours
Other	24 hours	72 hours	24 hours	12 hours

---

Escalation Rules

Understanding Escalation

Escalation automatically assigns complaints to higher-level staff when:

1. Complaint becomes overdue
2. Specified time after reminder has passed

Setting Up Escalation Rules

1. Access Admin Panel

   http://localhost:8000/admin/complaints/escalationrule/
   

2. Create Escalation Rule

   • Select Hospital
   • Set Escalation Level (1, 2, 3, etc.)
   • Name the rule
   • Choose Trigger Type (Overdue or After Reminder)
   • Set Hours parameter
   • Select Escalation Target
   • Filter by Priority/Severity (optional)
   • Mark as Active

Example Escalation Rules

Level 1: Department Manager

• Trigger: Overdue
• Hours overdue: 0 (immediately when due time passes)
• Escalate to: Department Manager

Level 2: Hospital Admin

• Trigger: After Reminder
• Hours after reminder: 12
• Escalate to: Hospital Administrator
• Only for: Critical and High priority

Level 3: Regional Admin

• Trigger: Overdue
• Hours overdue: 48
• Escalate to: Regional Administrator
• Only for: Critical severity

---

Testing the System

Automated Testing

Run the automated SLA test suite:

python test_sla_functionality.py

This test verifies:

• ✓ SLA configuration retrieval
• ✓ Due date calculation
• ✓ First reminder logic
• ✓ Second reminder logic
• ✓ Escalation eligibility
• ✓ Timeline tracking

Manual Testing

Test 1: Complaint Creation and SLA Calculation

1. Create a complaint via admin panel or API
2. Verify:
   • due_at field is populated
   • first_reminder_sent_at and second_reminder_sent_at are null
   • escalation_level is 0
   • Timeline entry shows "Complaint created and registered"

# View complaint details
python manage.py shell -c "
from apps.complaints.models import Complaint
c = Complaint.objects.first()
print(f'ID: {c.id}')
print(f'Title: {c.title}')
print(f'Status: {c.status}')
print(f'Due At: {c.due_at}')
print(f'Escalation Level: {c.escalation_level}')
print(f'First Reminder Sent: {c.first_reminder_sent_at}')
print(f'Second Reminder Sent: {c.second_reminder_sent_at}')
"

Test 2: First Reminder

1. Create a complaint with SLA config
2. Wait for first reminder time or modify due_at to trigger
3. Check complaint for:
   • first_reminder_sent_at is populated
   • Timeline entry shows "First SLA reminder sent"

# Check reminder status
python manage.py shell -c "
from apps.complaints.models import Complaint
from django.utils import timezone
import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

complaint = Complaint.objects.first()
if complaint:
    from apps.complaints.tasks import check_and_send_sla_reminders
    result = check_and_send_sla_reminders()
    print(f'Result: {result}')
    
    # Refresh complaint
    complaint.refresh_from_db()
    print(f'First Reminder Sent At: {complaint.first_reminder_sent_at}')
    print(f'Second Reminder Sent At: {complaint.second_reminder_sent_at}')
"

Test 3: Second Reminder

1. Ensure complaint has first reminder sent
2. Wait for second reminder time or modify timestamps
3. Verify:
   • second_reminder_sent_at is populated
   • Timeline entry shows "Second SLA reminder sent"

Test 4: Escalation

1. Create overdue complaint (set due_at in the past)
2. Ensure escalation rules exist
3. Run escalation check:

python manage.py shell -c "
from apps.complaints.models import Complaint
from django.utils import timezone
import logging

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

complaint = Complaint.objects.first()
if complaint:
    # Make it overdue
    complaint.due_at = timezone.now() - timezone.timedelta(hours=2)
    complaint.save()
    
    from apps.complaints.tasks import check_and_escalate_overdue_complaints
    result = check_and_escalate_overdue_complaints()
    print(f'Result: {result}')
    
    # Refresh complaint
    complaint.refresh_from_db()
    print(f'Escalation Level: {complaint.escalation_level}')
    print(f'Escalated To: {complaint.escalated_to}')
    print(f'Escalated At: {complaint.escalated_at}')
"

Test 5: Seeding Test Data

Generate test complaints for SLA testing:

# Seed 10 complaints (7 Arabic, 3 English)
python manage.py seed_complaints --count 10

# Seed 50 complaints with different distribution
python manage.py seed_complaints --count 50 --arabic-percent 50 --staff-mention-percent 70

# Dry run to preview
python manage.py seed_complaints --count 20 --dry-run

# Clear existing and seed fresh
python manage.py seed_complaints --count 10 --clear

---

Monitoring and Troubleshooting

Checking SLA Status

View all complaints with SLA information:

python manage.py shell -c "
from apps.complaints.models import Complaint
from django.utils import timezone
from django.db.models import Q

now = timezone.now()

# All open complaints
open_complaints = Complaint.objects.filter(status='open')
print(f'Total Open Complaints: {open_complaints.count()}')

# Overdue complaints
overdue = open_complaints.filter(due_at__lt=now)
print(f'Overdue: {overdue.count()}')

# Due within 24 hours
soon = open_complaints.filter(due_at__gte=now, due_at__lte=now + timezone.timedelta(hours=24))
print(f'Due within 24h: {soon.count()}')

# First reminder sent but not second
first_only = open_complaints.filter(first_reminder_sent_at__isnull=False, second_reminder_sent_at__isnull=True)
print(f'First reminder only: {first_only.count()}')

# Escalated complaints
escalated = open_complaints.filter(escalation_level__gt=0)
print(f'Escalated: {escalated.count()}')
"

Celery Task Monitoring

Check if Celery Beat is running:

# Check Celery status
celery -A config inspect active

# View scheduled tasks
celery -A config inspect scheduled

# View registered tasks
celery -A config inspect registered

Email Configuration

Test email sending:

python test_email_sending.py

Ensure email settings in .env:

EMAIL_BACKEND=django.core.mail.backends.smtp.EmailBackend
EMAIL_HOST=smtp.gmail.com
EMAIL_PORT=587
EMAIL_USE_TLS=True
EMAIL_HOST_USER=your-email@gmail.com
EMAIL_HOST_PASSWORD=your-app-password
DEFAULT_FROM_EMAIL=your-email@gmail.com

Common Issues

Issue 1: Reminders not sending

• Check Celery Beat is running
• Verify Celery logs: tail -f logs/celery.log
• Check email configuration
• Verify SLA config has reminders enabled

Issue 2: Escalation not working

• Verify escalation rules exist and are active
• Check complaint matches rule criteria (priority/severity)
• Ensure complaint is actually overdue
• Check Celery logs for errors

Issue 3: Wrong due dates

• Verify SLA configuration exists for complaint's category/hospital
• Check timezone settings in Django
• Review signal handlers in apps/complaints/signals.py

---

Production Setup

1. Configure Celery Beat

Ensure Celery Beat is configured in config/celery.py:

from celery.schedules import crontab

app.conf.beat_schedule = {
    'check-sla-reminders-every-15-minutes': {
        'task': 'apps.complaints.tasks.check_and_send_sla_reminders',
        'schedule': crontab(minute='*/15'),
    },
    'check-overdue-complaints-every-30-minutes': {
        'task': 'apps.complaints.tasks.check_and_escalate_overdue_complaints',
        'schedule': crontab(minute='*/30'),
    },
}

2. Run Celery Workers

# Start Celery worker
celery -A config worker -l INFO

# Start Celery Beat scheduler
celery -A config beat -l INFO --scheduler django_celery_beat.schedulers:DatabaseScheduler

3. Configure Production Email Settings

Use production email service (e.g., SendGrid, AWS SES):

EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'smtp.sendgrid.net'
EMAIL_PORT = 587
EMAIL_USE_TLS = True
EMAIL_HOST_USER = 'apikey'
EMAIL_HOST_PASSWORD = os.getenv('SENDGRID_API_KEY')
DEFAULT_FROM_EMAIL = 'noreply@alhammadi.com.sa'

4. Set Up Monitoring

• Monitor Celery task queue size
• Track SLA breach rate
• Alert on high escalation rates
• Monitor email delivery success rate

5. Database Indexes

Ensure database has proper indexes for SLA queries:

class Complaint(models.Model):
    class Meta:
        indexes = [
            models.Index(fields=['status', 'due_at']),
            models.Index(fields=['status', 'created_at']),
            models.Index(fields=['hospital', 'status']),
            models.Index(fields=['escalation_level']),
        ]

6. Backup Strategy

• Regular database backups
• Backup SLA configurations
• Archive old complaints
• Keep escalation rule history

---

Summary

The SLA system provides:

✓ Automated Time Tracking - Automatically calculates and tracks due dates ✓ Double Reminder System - First and second reminders configurable per SLA ✓ Smart Escalation - Automatic escalation based on rules ✓ Complete Audit Trail - Full timeline of all activities ✓ Multi-level Configuration - Different SLAs by hospital, department, and category ✓ Bilingual Support - Full Arabic/English support for emails and notifications ✓ Flexible Rules - Configure different priorities and severities ✓ Background Processing - Non-blocking Celery tasks ✓ Production Ready - Tested and ready for deployment

---

Next Steps

1. Configure SLA Settings - Set appropriate time limits for your organization
2. Test with Real Data - Use production-like data for testing
3. Monitor Initial Period - Closely monitor SLA performance for first week
4. Adjust Rules - Fine-tune escalation rules based on observed patterns
5. Train Staff - Educate staff on SLA system and their responsibilities
6. Set Up Dashboards - Create monitoring dashboards for SLA metrics

---

Additional Resources

• SLA Testing Guide: docs/SLA_TESTING_GUIDE.md
• SLA System Overview: docs/SLA_SYSTEM_OVERVIEW.md
• SLA Testing Plan: docs/SLA_TESTING_PLAN.md
• Complaint Seeding Guide: docs/COMPLAINT_SEEDING_GUIDE.md
• Email Templates: templates/complaints/emails/

---

Last Updated: January 14, 2026