maternal-app/scripts/README-SYNC-SCHEMA.md

# Database Schema Synchronization Script

## Overview

The `sync-database-schema.sh` script automatically synchronizes the production database schema with the development database. It compares both databases and applies necessary changes to keep production up-to-date.

## Features

- ✅ **Creates missing tables** - Automatically creates tables that exist in dev but not in prod
- ✅ **Adds missing columns** - Adds new columns to existing tables
- ✅ **Creates missing indexes** - Synchronizes indexes for better performance
- ✅ **Detects type mismatches** - Warns about data type differences (requires manual intervention)
- ✅ **Safe dry-run mode** - Preview changes before applying them
- ✅ **Detailed logging** - Color-coded output with verbose mode
- ✅ **Automatic cleanup** - Removes temporary files after execution

## Usage

### Basic Synchronization

```bash
# Synchronize production database with development
cd /root/maternal-app/scripts
./sync-database-schema.sh
```

### Dry Run Mode (Preview Changes)

```bash
# See what would be changed without making changes
./sync-database-schema.sh --dry-run
```

### Verbose Mode (Detailed Output)

```bash
# Show detailed SQL statements and operations
./sync-database-schema.sh --verbose
```

### Combined Options

```bash
# Dry run with verbose output
./sync-database-schema.sh --dry-run --verbose
```

## What It Does

### Step 1: Missing Tables
- Compares tables between dev and prod
- Creates any tables that exist in dev but not in prod
- Includes all constraints, indexes, and triggers

### Step 2: Missing Columns
- For each common table, compares columns
- Adds missing columns with correct data types
- Preserves NULL/NOT NULL constraints
- Applies default values

### Step 3: Missing Indexes
- Compares indexes between databases
- Creates missing indexes in production
- Excludes primary keys and unique constraints (handled separately)

### Step 4: Data Type Mismatches
- Detects columns with different data types
- Warns about mismatches requiring manual intervention
- Does NOT automatically change data types (requires manual review)

## Safety Features

1. **Dry Run Mode**: Test changes before applying
2. **Automatic Backups**: Uses temporary files for rollback if needed
3. **Error Handling**: Stops on critical errors
4. **Manual Review Required**: Data type changes need manual approval

## Exit Codes

- `0` - Success (changes made or databases already in sync)
- `1` - Error occurred
- `2` - Dry run mode: changes would be made

## Database Configuration

The script uses these default settings:

```bash
DB_HOST="10.0.0.207"
DB_USER="postgres"
DB_PASSWORD="a3ppq"
DEV_DB="parentflowdev"
PROD_DB="parentflow"
```

## Example Output

```
================================================================================
                    DATABASE SCHEMA SYNCHRONIZATION
================================================================================

Development Database: parentflowdev
Production Database:  parentflow
Dry Run Mode:         false
Verbose Mode:         false

================================================================================

[INFO] Testing database connectivity...
[SUCCESS] Database connectivity verified

[INFO] STEP 1: Checking for missing tables...
[SUCCESS] No missing tables found

[INFO] STEP 2: Checking for missing columns in existing tables...
[WARNING] Table 'users' has missing columns in production
[INFO]   Missing column: new_field (text)
[SUCCESS]   Added column: new_field to users

[INFO] STEP 3: Checking for missing indexes...
[SUCCESS] No missing indexes found

[INFO] STEP 4: Checking for data type mismatches...
[SUCCESS] No data type mismatches found

================================================================================
                              SYNCHRONIZATION SUMMARY
================================================================================

[SUCCESS] Schema synchronization completed successfully!
[INFO] Production database has been updated to match development schema

[INFO] Synchronization completed at 2025-10-08 15:45:23
================================================================================
```

## When to Use

### Recommended Use Cases

1. **After Schema Changes in Dev**: Run after adding tables/columns in development
2. **Before Deployment**: Ensure prod schema is up-to-date before deploying new code
3. **Regular Maintenance**: Weekly/monthly schema sync checks
4. **Post-Migration**: After running migrations in dev

### Not Recommended For

1. **Data Migration**: This script only handles schema, not data
2. **Destructive Changes**: Doesn't drop tables or columns (by design)
3. **Complex Type Changes**: Manual intervention required for data type changes

## Best Practices

1. **Always run with --dry-run first**
   ```bash
   ./sync-database-schema.sh --dry-run
   ```

2. **Review output carefully**
   - Check what tables/columns will be created
   - Verify no unexpected changes

3. **Backup production database before sync**
   ```bash
   pg_dump -h 10.0.0.207 -U postgres parentflow > backup.sql
   ```

4. **Run during maintenance windows**
   - Minimize impact on live traffic
   - Allows time for verification

5. **Test in staging first** (if available)
   - Validate script behavior
   - Identify potential issues

## Troubleshooting

### Connection Errors

```
[ERROR] Cannot connect to production database
```

**Solution**: Check database credentials and network connectivity

### Permission Errors

```
ERROR: permission denied for table users
```

**Solution**: Ensure postgres user has required permissions

### Index Creation Failures

```
[WARNING] Could not create index: idx_name
```

**Solution**: Usually indicates index already exists or has dependencies. Check manually:
```sql
\d table_name
```

### Data Type Mismatches

```
[WARNING] Data type mismatches found in table: users
```

**Solution**: Review differences and manually ALTER TABLE with appropriate type conversion:
```sql
ALTER TABLE users ALTER COLUMN field_name TYPE new_type USING field_name::new_type;
```

## Manual Verification

After running the script, verify changes:

```bash
# Compare table counts
PGPASSWORD=a3ppq psql -h 10.0.0.207 -U postgres -d parentflowdev -c \
  "SELECT COUNT(*) FROM pg_tables WHERE schemaname = 'public';"

PGPASSWORD=a3ppq psql -h 10.0.0.207 -U postgres -d parentflow -c \
  "SELECT COUNT(*) FROM pg_tables WHERE schemaname = 'public';"

# Compare specific table structure
PGPASSWORD=a3ppq psql -h 10.0.0.207 -U postgres -d parentflowdev -c "\d users"
PGPASSWORD=a3ppq psql -h 10.0.0.207 -U postgres -d parentflow -c "\d users"
```

## Automation

### Cron Job (Weekly Sync)

```bash
# Add to crontab
0 2 * * 0 /root/maternal-app/scripts/sync-database-schema.sh >> /var/log/db-sync.log 2>&1
```

### CI/CD Integration

```yaml
# .github/workflows/db-sync.yml
- name: Sync Database Schema
  run: |
    cd scripts
    ./sync-database-schema.sh --dry-run
```

## Support

For issues or questions:
1. Check the troubleshooting section above
2. Review script logs with `--verbose` flag
3. Verify database connectivity and permissions
4. Check PostgreSQL logs for detailed errors

## Version History

- **v1.0** (2025-10-08): Initial release
  - Table creation
  - Column addition
  - Index synchronization
  - Type mismatch detection