HH/docs/OBSERVATION_MODEL_FIXES.md

# Observation Model Fixes

## Overview
Fixed critical issues in the Observation model to align with the system's tenant isolation architecture and improve parity with the Complaint model.

## Issues Fixed

### 1. Missing Hospital ForeignKey (Critical)
**Problem:** The Observation model lacked a `hospital` field, breaking tenant isolation.

**Impact:**
- Cannot filter observations by hospital
- Cannot track which hospital owns the observation
- Prevents proper routing in multi-hospital environment
- Breaks the tenant isolation pattern used throughout the system

**Solution:**
- Added `hospital` ForeignKey to `organizations.Hospital`
- Field is required for new observations
- Added database index for efficient filtering by hospital and status

### 2. Missing Staff ForeignKey
**Problem:** Observations couldn't track which staff member was mentioned in the report.

**Solution:**
- Added optional `staff` ForeignKey to `organizations.Staff`
- Enables AI-matching similar to Complaint model
- Useful for tracking observations about specific staff members

### 3. Missing Source Tracking
**Problem:** No way to track how observations were submitted.

**Solution:**
- Added `source` field with choices:
  - `staff_portal` - Staff Portal (default)
  - `web_form` - Web Form
  - `mobile_app` - Mobile App
  - `email` - Email
  - `call_center` - Call Center
  - `other` - Other

## Model Changes

### Added Fields to Observation Model
```python
hospital = models.ForeignKey(
    'organizations.Hospital',
    on_delete=models.CASCADE,
    related_name='observations',
    help_text="Hospital where observation was made"
)

staff = models.ForeignKey(
    'organizations.Staff',
    on_delete=models.SET_NULL,
    null=True,
    blank=True,
    related_name='observations',
    help_text="Staff member mentioned in observation"
)

source = models.CharField(
    max_length=50,
    choices=[
        ('staff_portal', 'Staff Portal'),
        ('web_form', 'Web Form'),
        ('mobile_app', 'Mobile App'),
        ('email', 'Email'),
        ('call_center', 'Call Center'),
        ('other', 'Other'),
    ],
    default='staff_portal',
    help_text="How the observation was submitted"
)
```

### Added Database Indexes
```python
models.Index(fields=['hospital', 'status', '-created_at'])
```

## Migration Strategy

### Migration 0002_add_missing_fields.py
- Adds `hospital`, `staff`, and `source` fields
- Initially makes `hospital` nullable to support existing data
- Since there are 0 existing observations, no data migration needed

### Future Migration
After deployment, a follow-up migration can make `hospital` required if needed:
```python
migrations.AlterField(
    model_name='observation',
    name='hospital',
    field=models.ForeignKey(
        on_delete=django.db.models.deletion.CASCADE,
        related_name='observations',
        to='organizations.hospital'
    ),
)
```

## Why Not Unify with Complaints?

Despite similarities, the models should remain separate:

### Key Differences
| Aspect | Complaint | Observation |
|--------|-----------|-------------|
| **Purpose** | Patient-reported issues | Staff-reported issues |
| **Reporter** | Patient FK (required) | Anonymous or staff ID |
| **Workflow** | Open → In Progress → Resolved → Closed | New → Triaged → Assigned → In Progress → Resolved → Closed (plus REJECTED, DUPLICATE) |
| **SLA** | Response time SLAs with tracking | Triage timeline, not response time |
| **Tracking** | Reference number (optional) | Tracking code (required, OBS-XXXXXX) |
| **Location** | No location field | Location text field |
| **Incident Time** | No separate field | Incident datetime field |
| **Triage** | No triage fields | Triage tracking (triaged_by, triaged_at) |
| **Resolution Survey** | Links to SurveyInstance | No resolution survey |
| **Encounter ID** | Links to patient encounters | No encounter tracking |

### Different Business Logic
- **Complaints**: Customer service workflow, satisfaction surveys, SLA escalation
- **Observations**: Safety/quality workflow, triage process, anonymous reporting

### Different Stakeholders
- **Complaints**: Patient experience teams, call center
- **Observations**: Quality teams, safety officers, clinical governance

## Next Steps

1. **Run Migration:**
   ```bash
   python manage.py migrate observations
   ```

2. **Update Forms:** Ensure observation forms include hospital selection
   - Public observation form
   - Staff portal observation form
   - Admin observation form

3. **Update Views:** Add hospital filtering to observation list views

4. **Update APIs:** Include hospital field in observation APIs

5. **Update Templates:** Display hospital information in observation detail views

6. **Consider AI Integration:** Add staff name extraction similar to complaints
   - Use `apps.core.ai_service.extract_person_name()`
   - Match extracted names to Staff records
   - Populate `staff` field automatically

## Files Modified

1. **apps/observations/models.py**
   - Added `hospital` ForeignKey
   - Added `staff` ForeignKey
   - Added `source` field
   - Added hospital status index

2. **apps/observations/migrations/0002_add_missing_fields.py**
   - Migration to add new fields

## Backwards Compatibility

- Migration makes `hospital` initially nullable
- Model code enforces `hospital` as required for new records
- Existing code without hospital selection will work but with validation errors for new observations
- Consider updating form validation to ensure hospital is always provided