HH/docs/SIMPLIFIED_SURVEY_INTEGRATION.md

# Simplified Survey Integration Documentation

## Overview

The PX360 survey system has been simplified to directly deliver surveys based on PatientType without the complexity of journey tracking. This document describes the simplified architecture, data flow, and usage.

## Architecture

### Before (Complex)
```
HIS Data → Patient Journey → Journey Stages → Visit Processing → Stage Completion → Survey Creation → SMS Delivery
```

### After (Simplified)
```
HIS Data → PatientType Detection → Survey Template Selection → Survey Creation → SMS Delivery
```

## Key Components

### 1. HIS Adapter (`apps/integrations/services/his_adapter.py`)

The HIS Adapter processes real HIS patient data and creates surveys directly.

**Main Functions:**

- `map_patient_type_to_survey_type()`: Maps HIS PatientType codes to survey types
- `get_survey_template()`: Selects appropriate survey template based on PatientType
- `create_and_send_survey()`: Creates survey instance and triggers SMS delivery
- `process_his_data()`: Main entry point for processing HIS data

**PatientType Mapping:**

| HIS Code | Survey Type | Description |
|----------|-------------|-------------|
| "1" | INPATIENT | Inpatient visits |
| "2" or "O" | OPD | Outpatient visits |
| "3" or "E" | EMS | Emergency visits |
| unknown | OPD (default) | Falls back to OPD |

### 2. HIS Simulator (`apps/simulator/his_simulator.py`)

The simulator generates realistic HIS patient data with proper PatientType distribution.

**PatientType Distribution:**

- OPD: 60% (most common)
- Inpatient: 30%
- EMS: 10%

**Usage:**

```bash
# Run simulator with default settings (5 patients per minute)
python apps/simulator/his_simulator.py

# Run simulator with custom settings
python apps/simulator/his_simulator.py --url http://localhost:8000/api/simulator/his-patient-data/ --delay 10 --max-patients 50
```

**Parameters:**

- `--url`: API endpoint URL
- `--delay`: Delay between patients in seconds
- `--max-patients`: Maximum number of patients (0 = infinite)

### 3. Survey Templates

Survey templates must be named to match PatientType:

- `OPD Survey` or template name containing "OPD"
- `INPATIENT Survey` or template name containing "INPATIENT"
- `EMS Survey` or template name containing "EMS"

**Example Template Creation:**

```python
from apps.surveys.models import SurveyTemplate
from apps.organizations.models import Hospital

hospital = Hospital.objects.get(code="ALH-main")

# OPD Survey
opd_survey = SurveyTemplate.objects.create(
    name="OPD Survey",
    hospital=hospital,
    description="Outpatient Department Survey",
    is_active=True,
    survey_type="patient_satisfaction"
)

# Inpatient Survey
inpatient_survey = SurveyTemplate.objects.create(
    name="INPATIENT Survey",
    hospital=hospital,
    description="Inpatient Care Survey",
    is_active=True,
    survey_type="patient_satisfaction"
)

# EMS Survey
ems_survey = SurveyTemplate.objects.create(
    name="EMS Survey",
    hospital=hospital,
    description="Emergency Medical Services Survey",
    is_active=True,
    survey_type="patient_satisfaction"
)
```

## Data Flow

### 1. HIS Patient Data Reception

The system receives HIS data in this format:

```json
{
  "FetchPatientDataTimeStampList": [{
    "PatientID": "123456",
    "AdmissionID": "ADM-001",
    "HospitalID": "1",
    "HospitalName": "Al Hammadi Hospital - Main",
    "PatientType": "2",
    "AdmitDate": "05-Jun-2025 11:06",
    "DischargeDate": "05-Jun-2025 16:30",
    "PatientName": "Ahmed Al-Saud",
    "MobileNo": "0512345678",
    "Email": "ahmed@example.com",
    "InsuranceCompanyName": "Arabian Shield Cooperative Insurance Company",
    ...
  }],
  "FetchPatientDataTimeStampVisitDataList": [...],
  "Code": 200,
  "Status": "Success"
}
```

### 2. Processing Flow

```
1. Receive HIS patient data
   ↓
2. Extract PatientType from data
   ↓
3. Check if patient is discharged (DischargeDate present)
   ↓
4. Get or create patient record
   ↓
5. Select appropriate survey template based on PatientType
   ↓
6. Create survey instance with metadata
   ↓
7. Send survey via SMS using patient's phone number
   ↓
8. Return success/failure status
```

### 3. Survey Creation

When a discharged patient is received:

```python
# Survey instance created with:
{
  "survey_template": <OPD Survey>,
  "patient": <Patient record>,
  "hospital": <Hospital record>,
  "status": "PENDING",
  "delivery_channel": "SMS",
  "recipient_phone": "0512345678",
  "recipient_email": "ahmed@example.com",
  "metadata": {
    "admission_id": "ADM-001",
    "patient_type": "2",
    "hospital_id": "1",
    "insurance_company": "Arabian Shield Cooperative Insurance Company",
    "is_vip": false
  }
}
```

### 4. SMS Delivery

The survey is sent via SMS to the patient's phone number:

```python
from apps.surveys.services import SurveyDeliveryService

delivery_success = SurveyDeliveryService.deliver_survey(survey)
```

**SMS Message Format:**

```
Thank you for visiting Al Hammadi Hospital! Please complete our survey at:
[Survey URL]

Your feedback helps us improve our services.
```

## Testing

### Automated Tests

Run the test script to verify integration:

```bash
python test_simplified_survey_integration.py
```

**Test Coverage:**

1. ✅ OPD patients receive OPD surveys
2. ✅ Inpatient patients receive Inpatient surveys
3. ✅ EMS patients receive EMS surveys
4. ✅ Non-discharged patients don't receive surveys
5. ✅ Alternative PatientType codes (O, E) work correctly
6. ✅ Survey metadata is stored correctly

### Manual Testing

**Test 1: OPD Patient Survey**

```python
from apps.integrations.services.his_adapter import HISAdapter

his_data = {
    "FetchPatientDataTimeStampList": [{
        "PatientID": "TEST-001",
        "AdmissionID": "ADM-TEST-001",
        "HospitalName": "Al Hammadi Hospital - Main",
        "PatientType": "2",  # OPD
        "AdmitDate": "05-Jun-2025 11:06",
        "DischargeDate": "05-Jun-2025 16:30",
        "PatientName": "Test Patient",
        "MobileNo": "0512345678",
        ...
    }],
    "FetchPatientDataTimeStampVisitDataList": [],
    "Code": 200,
    "Status": "Success"
}

result = HISAdapter.process_his_data(his_data)
print(f"Success: {result['success']}")
print(f"Survey: {result['survey']}")
print(f"Survey URL: {result['survey_url']}")
```

**Test 2: Non-Discharged Patient**

```python
his_data_no_discharge = {
    "FetchPatientDataTimeStampList": [{
        "PatientID": "TEST-002",
        "AdmissionID": "ADM-TEST-002",
        "HospitalName": "Al Hammadi Hospital - Main",
        "PatientType": "2",
        "AdmitDate": "05-Jun-2025 11:06",
        "DischargeDate": None,  # Not discharged
        "PatientName": "Test Patient 2",
        "MobileNo": "0512345679",
        ...
    }],
    "FetchPatientDataTimeStampVisitDataList": [],
    "Code": 200,
    "Status": "Success"
}

result = HISAdapter.process_his_data(his_data_no_discharge)
print(f"Result: {result['message']}")
# Expected: "Patient not discharged - no survey sent"
```

## Configuration

### Required Survey Templates

Each hospital must have at least one survey template for each patient type:

```bash
# Check existing survey templates
python manage.py shell
>>> from apps.surveys.models import SurveyTemplate
>>> SurveyTemplate.objects.filter(hospital__code="ALH-main").values_list('name', 'is_active')
```

### SMS Configuration

Ensure SMS delivery is configured in `apps/surveys/services.py`:

```python
class SurveyDeliveryService:
    @staticmethod
    def deliver_survey(survey: SurveyInstance) -> bool:
        # SMS delivery implementation
        # Returns True if successful, False otherwise
        pass
```

### Logging

Enable logging to track survey delivery:

```python
import logging

logger = logging.getLogger(__name__)
logger.info(f"Survey {survey.id} sent to {survey.patient.get_full_name()}")
```

## Troubleshooting

### Issue: Survey Not Created

**Check:**
1. Patient has discharge date
2. Survey template exists for PatientType
3. Hospital is active

**Debug:**

```python
result = HISAdapter.process_his_data(his_data)
print(f"Message: {result['message']}")
print(f"Success: {result['success']}")
```

### Issue: Wrong Survey Type

**Check:**
1. Survey template name contains PatientType (OPD, INPATIENT, EMS)
2. Template is active (`is_active=True`)
3. Template belongs to correct hospital

### Issue: SMS Not Delivered

**Check:**
1. Patient phone number is valid
2. SMS service is configured
3. Check logs for delivery errors

```python
# Check delivery status
survey = SurveyInstance.objects.get(id=survey_id)
print(f"Status: {survey.status}")
print(f"Delivery Channel: {survey.delivery_channel}")
```

### Issue: Duplicate Surveys

The system prevents duplicate surveys for the same admission:

```python
# Check if survey already exists
existing_survey = SurveyInstance.objects.filter(
    patient=patient,
    hospital=hospital,
    metadata__admission_id="ADM-001"
).first()
```

## Monitoring

### Key Metrics to Track

1. **Survey Creation Rate**: Surveys created per day
2. **Survey Delivery Rate**: SMS delivery success rate
3. **Survey Response Rate**: Patient response rate
4. **Patient Type Distribution**: OPD/Inpatient/EMS breakdown

### Database Queries

```sql
-- Surveys created today
SELECT 
    COUNT(*) as total_surveys,
    COUNT(DISTINCT patient_id) as unique_patients
FROM surveys_surveyinstance
WHERE DATE(created_at) = CURRENT_DATE;

-- Surveys by patient type
SELECT 
    JSON_VALUE(metadata, '$.patient_type') as patient_type,
    COUNT(*) as count
FROM surveys_surveyinstance
WHERE DATE(created_at) = CURRENT_DATE
GROUP BY patient_type;

-- Survey delivery status
SELECT 
    status,
    COUNT(*) as count
FROM surveys_surveyinstance
WHERE DATE(created_at) = CURRENT_DATE
GROUP BY status;
```

## Best Practices

1. **Survey Template Names**: Use clear names containing patient type
   - ✅ "OPD Survey", "INPATIENT Survey", "EMS Survey"
   - ❌ "Patient Survey", "Feedback Survey"

2. **Discharge Detection**: Only send surveys after discharge
   - Always check `DischargeDate` field
   - Don't send surveys for active admissions

3. **Duplicate Prevention**: Check existing surveys before creating new ones
   - Use `admission_id` in metadata to prevent duplicates

4. **Metadata Storage**: Store relevant patient information in metadata
   - PatientType, AdmissionID, InsuranceCompany, VIP status

5. **SMS Delivery**: Use reliable SMS service with retry logic
   - Handle delivery failures gracefully
   - Log all delivery attempts

## Migration from Journey-Based System

If migrating from the complex journey-based system:

1. **Keep Journey Models**: Don't delete journey models - they may be useful for analytics
2. **Update HIS Adapter**: Use simplified `process_his_data()` method
3. **Remove Journey Creation**: No need to create `PatientJourneyInstance`
4. **Direct Survey Creation**: Create surveys directly based on PatientType
5. **Testing**: Run comprehensive tests to ensure surveys are created correctly

## API Endpoints

### Receive HIS Patient Data

**Endpoint:** `POST /api/simulator/his-patient-data/`

**Request Body:**

```json
{
  "FetchPatientDataTimeStampList": [{
    "PatientID": "123456",
    "PatientType": "2",
    "DischargeDate": "05-Jun-2025 16:30",
    ...
  }],
  "FetchPatientDataTimeStampVisitDataList": [],
  "Code": 200,
  "Status": "Success"
}
```

**Response:**

```json
{
  "success": true,
  "message": "Patient data processed successfully",
  "patient": {
    "id": 1,
    "name": "Ahmed Al-Saud"
  },
  "survey": {
    "id": 1,
    "status": "SENT",
    "survey_url": "https://example.com/survey/abc123"
  },
  "survey_sent": true
}
```

## Summary

The simplified survey integration:

✅ **Direct Survey Delivery**: Surveys created immediately upon discharge
✅ **PatientType-Based**: Correct survey template selected automatically
✅ **SMS Delivery**: Surveys sent via SMS to patient's phone
✅ **Duplicate Prevention**: No duplicate surveys for same admission
✅ **Metadata Tracking**: Patient information stored for analytics
✅ **Easy Testing**: Comprehensive test suite available
✅ **Realistic Simulation**: HIS simulator generates realistic patient data

This simplified approach reduces complexity while maintaining all essential functionality for survey delivery based on patient type.