# PX360 Journey & Survey Engine

## Overview

The Journey & Survey Engine is the core of PX360's patient experience tracking system. It implements an event-driven architecture where external systems (HIS, Lab, Radiology, Pharmacy) send integration events that trigger journey stage completions, which in turn automatically send stage-specific surveys to patients.

## Architecture

### Components

1. **Journey Templates** - Define the stages for each journey type (EMS/Inpatient/OPD)
2. **Journey Instances** - Track actual patient encounters
3. **Stage Templates** - Define individual stages with trigger events
4. **Stage Instances** - Track completion of each stage
5. **Inbound Events** - Integration events from external systems
6. **Survey Templates** - Define surveys for each stage
7. **Survey Instances** - Actual surveys sent to patients

### Flow Diagram

```
External System (HIS/Lab/etc.)
    ↓
POST /api/integrations/events/
    ↓
InboundEvent (status: PENDING)
    ↓
Celery Task: process_inbound_event
    ↓
Find Journey Instance (by encounter_id)
    ↓
Find Matching Stage (by trigger_event_code)
    ↓
Complete Stage → Update timestamps
    ↓
If auto_send_survey=True
    ↓
Create SurveyInstance
    ↓
Send Survey (SMS/WhatsApp/Email)
    ↓
Patient Completes Survey
    ↓
If score < threshold → Create PXAction
```

## Journey Types

### 1. EMS (Emergency Medical Services)
Tracks emergency patient pathway from arrival to discharge.

**Example Stages:**
- Triage
- Emergency Physician Consultation
- Diagnostic Tests
- Treatment
- Discharge

### 2. Inpatient
Tracks admitted patient journey from admission to discharge.

**Example Stages:**
- Admission
- Ward Assignment
- Physician Rounds
- Nursing Care
- Diagnostic Services
- Treatment
- Discharge

### 3. OPD (Outpatient Department)
Tracks outpatient visit from registration to checkout.

**Example Stages:**
- Registration
- MD Consultation
- Lab Tests
- Radiology
- Pharmacy
- Checkout

## Integration Event Contracts

### Event Structure

All integration events must follow this structure:

```json
{
  "source_system": "his|lab|radiology|pharmacy|moh|chi|other",
  "event_code": "EVENT_CODE",
  "encounter_id": "UNIQUE_ENCOUNTER_ID",
  "patient_identifier": "MRN_OR_PATIENT_ID",
  "payload_json": {
    "physician_license": "PHY001",
    "department_code": "DEPT_OPD",
    "timestamp": "2025-12-14T10:30:00Z",
    "additional_data": {}
  }
}
```

### Standard Event Codes

#### OPD Events
- `OPD_REGISTRATION_COMPLETED` - Patient registered
- `OPD_VISIT_COMPLETED` - MD consultation completed
- `OPD_CHECKOUT_COMPLETED` - Patient checked out

#### Lab Events
- `LAB_ORDER_CREATED` - Lab order placed
- `LAB_SAMPLE_COLLECTED` - Sample collected
- `LAB_ORDER_COMPLETED` - Results ready

#### Radiology Events
- `RADIOLOGY_ORDER_CREATED` - Radiology order placed
- `RADIOLOGY_EXAM_COMPLETED` - Exam performed
- `RADIOLOGY_REPORT_FINALIZED` - Report signed off

#### Pharmacy Events
- `PHARMACY_ORDER_RECEIVED` - Prescription received
- `PHARMACY_DISPENSED` - Medication dispensed

#### Inpatient Events
- `ADMISSION_COMPLETED` - Patient admitted
- `WARD_TRANSFER` - Patient transferred to ward
- `DISCHARGE_COMPLETED` - Patient discharged

#### EMS Events
- `EMS_ARRIVAL` - Patient arrived via EMS
- `TRIAGE_COMPLETED` - Triage assessment done
- `ER_PHYSICIAN_CONSULT` - ER physician saw patient

### Example Event Payloads

#### OPD Visit Completed
```json
{
  "source_system": "his",
  "event_code": "OPD_VISIT_COMPLETED",
  "encounter_id": "ENC20251214001",
  "patient_identifier": "MRN123456",
  "payload_json": {
    "physician_license": "PHY001",
    "department_code": "DEPT_OPD",
    "visit_date": "2025-12-14",
    "visit_time": "10:30:00",
    "chief_complaint": "Follow-up",
    "diagnosis_codes": ["Z00.00"],
    "timestamp": "2025-12-14T10:45:00Z"
  }
}
```

#### Lab Order Completed
```json
{
  "source_system": "lab",
  "event_code": "LAB_ORDER_COMPLETED",
  "encounter_id": "ENC20251214001",
  "patient_identifier": "MRN123456",
  "payload_json": {
    "department_code": "DEPT_LAB",
    "order_id": "LAB789",
    "test_codes": ["CBC", "BMP"],
    "collection_time": "2025-12-14T11:00:00Z",
    "result_time": "2025-12-14T14:30:00Z",
    "timestamp": "2025-12-14T14:30:00Z"
  }
}
```

#### Pharmacy Dispensed
```json
{
  "source_system": "pharmacy",
  "event_code": "PHARMACY_DISPENSED",
  "encounter_id": "ENC20251214001",
  "patient_identifier": "MRN123456",
  "payload_json": {
    "department_code": "DEPT_PHARMACY",
    "prescription_id": "RX456",
    "medications": [
      {"name": "Medication A", "quantity": 30},
      {"name": "Medication B", "quantity": 60}
    ],
    "dispensed_time": "2025-12-14T15:00:00Z",
    "timestamp": "2025-12-14T15:00:00Z"
  }
}
```

## Journey Configuration

### Creating a Journey Template

**Step 1: Create Journey Template**
```python
from apps.journeys.models import PatientJourneyTemplate, JourneyType
from apps.organizations.models import Hospital

hospital = Hospital.objects.get(code='HOSP001')

journey_template = PatientJourneyTemplate.objects.create(
    name='Standard OPD Journey',
    name_ar='رحلة العيادات الخارجية القياسية',
    journey_type=JourneyType.OPD,
    hospital=hospital,
    is_active=True,
    is_default=True,
    description='Standard outpatient journey with MD, Lab, Radiology, and Pharmacy stages'
)
```

**Step 2: Create Stage Templates**
```python
from apps.journeys.models import PatientJourneyStageTemplate
from apps.surveys.models import SurveyTemplate

# Get survey templates
md_survey = SurveyTemplate.objects.get(name='MD Consultation Survey')
lab_survey = SurveyTemplate.objects.get(name='Lab Experience Survey')

# Stage 1: MD Consultation
PatientJourneyStageTemplate.objects.create(
    journey_template=journey_template,
    name='MD Consultation',
    name_ar='استشارة الطبيب',
    code='OPD_MD_CONSULT',
    order=1,
    trigger_event_code='OPD_VISIT_COMPLETED',
    survey_template=md_survey,
    auto_send_survey=True,
    survey_delay_hours=2,  # Send 2 hours after visit
    requires_physician=True,
    requires_department=True,
    is_optional=False,
    is_active=True
)

# Stage 2: Lab
PatientJourneyStageTemplate.objects.create(
    journey_template=journey_template,
    name='Laboratory',
    name_ar='المختبر',
    code='LAB',
    order=2,
    trigger_event_code='LAB_ORDER_COMPLETED',
    survey_template=lab_survey,
    auto_send_survey=True,
    survey_delay_hours=1,
    requires_physician=False,
    requires_department=True,
    is_optional=True,  # Not all patients need lab
    is_active=True
)

# Add more stages as needed...
```

### Creating a Journey Instance

**When a patient encounter starts:**
```python
from apps.journeys.models import PatientJourneyInstance
from apps.organizations.models import Patient

patient = Patient.objects.get(mrn='MRN123456')

journey_instance = PatientJourneyInstance.objects.create(
    journey_template=journey_template,
    patient=patient,
    encounter_id='ENC20251214001',
    hospital=hospital,
    department=opd_department
)

# Stage instances are automatically created for all active stages
```

## Event Processing

### Processing Logic

When an event is received:

1. **Event Receipt**
   - POST to `/api/integrations/events/`
   - Event stored with `status=PENDING`
   - Audit log created
   - Celery task queued

2. **Event Processing** (Celery Task)
   - Find journey instance by `encounter_id`
   - Find matching stage where:
     - `stage_template.trigger_event_code == event.event_code`
     - `stage_instance.status IN [PENDING, IN_PROGRESS]`
   - Extract physician and department from payload
   - Complete the stage:
     - Set `status=COMPLETED`
     - Set `completed_at=now()`
     - Attach `physician`, `department`, `event`
     - Store event payload in metadata

3. **Survey Triggering**
   - If `stage_template.auto_send_survey=True`
   - If `stage_template.survey_template` exists
   - Create `SurveyInstance`
   - Queue survey send task with delay

4. **Journey Completion**
   - Check if all required stages completed
   - If yes, mark journey as `COMPLETED`

### Error Handling

**Event Not Found:**
- Status: `IGNORED`
- Reason: "No journey instance found for encounter {encounter_id}"

**Stage Not Found:**
- Status: `IGNORED`
- Reason: "No pending stage found with trigger {event_code}"

**Processing Error:**
- Status: `FAILED`
- Error message stored
- Retry 3 times with exponential backoff
- Can be manually reprocessed by PX Admin

## API Endpoints

### Journey Templates
```
GET    /api/journeys/templates/              # List templates
POST   /api/journeys/templates/              # Create template
GET    /api/journeys/templates/{id}/         # Get template
PUT    /api/journeys/templates/{id}/         # Update template
DELETE /api/journeys/templates/{id}/         # Delete template
```

### Journey Instances
```
GET    /api/journeys/instances/              # List instances
POST   /api/journeys/instances/              # Create instance
GET    /api/journeys/instances/{id}/         # Get instance
GET    /api/journeys/instances/{id}/progress/ # Get progress summary
```

### Integration Events
```
POST   /api/integrations/events/             # Create event (external systems)
POST   /api/integrations/events/bulk_create/ # Bulk create events
GET    /api/integrations/events/             # List events (PX Admin)
GET    /api/integrations/events/{id}/        # Get event details
POST   /api/integrations/events/{id}/reprocess/ # Reprocess failed event
```

## Testing

### Test Scenario: OPD Journey

**1. Create Journey Instance**
```bash
curl -X POST http://localhost:8000/api/journeys/instances/ \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "journey_template": "TEMPLATE_UUID",
    "patient": "PATIENT_UUID",
    "encounter_id": "ENC20251214001",
    "hospital": "HOSPITAL_UUID"
  }'
```

**2. Send MD Visit Completed Event**
```bash
curl -X POST http://localhost:8000/api/integrations/events/ \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "source_system": "his",
    "event_code": "OPD_VISIT_COMPLETED",
    "encounter_id": "ENC20251214001",
    "patient_identifier": "MRN123456",
    "payload_json": {
      "physician_license": "PHY001",
      "department_code": "DEPT_OPD",
      "timestamp": "2025-12-14T10:45:00Z"
    }
  }'
```

**3. Verify Stage Completion**
```bash
curl http://localhost:8000/api/journeys/instances/JOURNEY_UUID/progress/ \
  -H "Authorization: Bearer YOUR_TOKEN"
```

**Expected Response:**
```json
{
  "journey_id": "...",
  "encounter_id": "ENC20251214001",
  "patient": "John Doe",
  "journey_type": "opd",
  "status": "active",
  "completion_percentage": 25,
  "is_complete": false,
  "stages": [
    {
      "name": "MD Consultation",
      "order": 1,
      "status": "completed",
      "completed_at": "2025-12-14T10:45:00Z",
      "survey_sent": true
    },
    {
      "name": "Laboratory",
      "order": 2,
      "status": "pending",
      "completed_at": null,
      "survey_sent": false
    }
  ]
}
```

## Configuration Best Practices

### 1. Survey Delay
- **Immediate (0 hours)**: For time-sensitive feedback (ER, urgent care)
- **2-4 hours**: For outpatient visits (allows patient to leave facility)
- **24 hours**: For inpatient daily surveys
- **48-72 hours**: For discharge surveys

### 2. Optional vs Required Stages
- **Required**: Core pathway stages (MD consultation, admission, discharge)
- **Optional**: Ancillary services (lab, radiology, pharmacy) - not all patients need these

### 3. Trigger Event Codes
- Use consistent naming: `{SERVICE}_{ACTION}_COMPLETED`
- Examples: `OPD_VISIT_COMPLETED`, `LAB_ORDER_COMPLETED`
- Document all codes in integration contracts

### 4. Survey Templates
- Create specific surveys for each stage
- Keep surveys short (5-7 questions max)
- Use NPS or Likert scales
- Include open-ended feedback field

## Monitoring & Troubleshooting

### Check Event Processing Status
```bash
# List pending events
curl http://localhost:8000/api/integrations/events/?status=pending \
  -H "Authorization: Bearer YOUR_TOKEN"

# List failed events
curl http://localhost:8000/api/integrations/events/?status=failed \
  -H "Authorization: Bearer YOUR_TOKEN"
```

### Reprocess Failed Event
```bash
curl -X POST http://localhost:8000/api/integrations/events/EVENT_UUID/reprocess/ \
  -H "Authorization: Bearer YOUR_TOKEN"
```

### View Journey Progress
```bash
curl http://localhost:8000/api/journeys/instances/?encounter_id=ENC123 \
  -H "Authorization: Bearer YOUR_TOKEN"
```

## Audit Trail

All events are logged:
- Event receipt → `integration_event`
- Stage completion → `stage_completed`
- Journey completion → `journey_completed`
- Survey sent → `survey_sent`

View audit logs in Django admin or via API.

## Performance Considerations

### Database Indexes
- `encounter_id` - Fast journey lookup
- `trigger_event_code` - Fast stage matching
- `status` + `received_at` - Efficient pending event queries

### Celery Configuration
- Events processed asynchronously
- Retry logic with exponential backoff
- Max 100 events per batch in periodic task
- Separate queue for event processing (optional)

### Optimization Tips
1. Use `select_related` for foreign keys
2. Use `prefetch_related` for reverse relations
3. Batch process events in periodic task
4. Monitor Celery queue length
5. Scale Celery workers as needed

## Security

### API Authentication
- JWT tokens for internal systems
- API keys for external systems (TODO: implement)
- Rate limiting on event endpoint (TODO: implement)

### Data Validation
- All events validated before storage
- Payload JSON validated
- Encounter ID format validation
- Source system whitelist

## Troubleshooting

### Event Not Processing
1. Check event status in admin
2. Check Celery worker logs
3. Verify journey instance exists
4. Verify stage template has matching trigger code
5. Check stage instance status (must be PENDING or IN_PROGRESS)

### Survey Not Sent
1. Verify `auto_send_survey=True` on stage template
2. Verify `survey_template` is set
3. Check survey instance created
4. Check notification logs (Phase 4)

### Stage Not Completing
1. Verify event code matches `trigger_event_code`
2. Check stage instance status
3. Verify physician/department codes if required
4. Check event processing logs

---

**For API documentation, see:** `/api/docs/`  
**For implementation status, see:** `IMPLEMENTATION_STATUS.md`