feat: Add automatic database schema synchronization script
Some checks failed
ParentFlow CI/CD Pipeline / Backend Tests (push) Has been cancelled
ParentFlow CI/CD Pipeline / Frontend Tests (push) Has been cancelled
ParentFlow CI/CD Pipeline / Security Scanning (push) Has been cancelled
ParentFlow CI/CD Pipeline / Build Docker Images (map[context:maternal-app/maternal-app-backend dockerfile:Dockerfile.production name:backend]) (push) Has been cancelled
ParentFlow CI/CD Pipeline / Build Docker Images (map[context:maternal-web dockerfile:Dockerfile.production name:frontend]) (push) Has been cancelled
ParentFlow CI/CD Pipeline / Deploy to Development (push) Has been cancelled
ParentFlow CI/CD Pipeline / Deploy to Production (push) Has been cancelled
CI/CD Pipeline / Lint and Test (push) Has been cancelled
CI/CD Pipeline / E2E Tests (push) Has been cancelled
CI/CD Pipeline / Build Application (push) Has been cancelled
Some checks failed
ParentFlow CI/CD Pipeline / Backend Tests (push) Has been cancelled
ParentFlow CI/CD Pipeline / Frontend Tests (push) Has been cancelled
ParentFlow CI/CD Pipeline / Security Scanning (push) Has been cancelled
ParentFlow CI/CD Pipeline / Build Docker Images (map[context:maternal-app/maternal-app-backend dockerfile:Dockerfile.production name:backend]) (push) Has been cancelled
ParentFlow CI/CD Pipeline / Build Docker Images (map[context:maternal-web dockerfile:Dockerfile.production name:frontend]) (push) Has been cancelled
ParentFlow CI/CD Pipeline / Deploy to Development (push) Has been cancelled
ParentFlow CI/CD Pipeline / Deploy to Production (push) Has been cancelled
CI/CD Pipeline / Lint and Test (push) Has been cancelled
CI/CD Pipeline / E2E Tests (push) Has been cancelled
CI/CD Pipeline / Build Application (push) Has been cancelled
- Created sync-database-schema.sh for automated schema sync - Compares dev and prod databases automatically - Creates missing tables with full schema - Adds missing columns to existing tables - Creates missing indexes - Detects data type mismatches - Includes dry-run mode for safety - Added comprehensive documentation Features: - Safe dry-run mode to preview changes - Verbose mode for detailed output - Color-coded logging for clarity - Automatic cleanup of temporary files - Error handling and validation 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Andrei
267
scripts/README-SYNC-SCHEMA.md
Normal file
267
scripts/README-SYNC-SCHEMA.md
Normal file
@@ -0,0 +1,267 @@
|
||||
# Database Schema Synchronization Script
|
||||
|
||||
## Overview
|
||||
|
||||
The `sync-database-schema.sh` script automatically synchronizes the production database schema with the development database. It compares both databases and applies necessary changes to keep production up-to-date.
|
||||
|
||||
## Features
|
||||
|
||||
- ✅ **Creates missing tables** - Automatically creates tables that exist in dev but not in prod
|
||||
- ✅ **Adds missing columns** - Adds new columns to existing tables
|
||||
- ✅ **Creates missing indexes** - Synchronizes indexes for better performance
|
||||
- ✅ **Detects type mismatches** - Warns about data type differences (requires manual intervention)
|
||||
- ✅ **Safe dry-run mode** - Preview changes before applying them
|
||||
- ✅ **Detailed logging** - Color-coded output with verbose mode
|
||||
- ✅ **Automatic cleanup** - Removes temporary files after execution
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic Synchronization
|
||||
|
||||
```bash
|
||||
# Synchronize production database with development
|
||||
cd /root/maternal-app/scripts
|
||||
./sync-database-schema.sh
|
||||
```
|
||||
|
||||
### Dry Run Mode (Preview Changes)
|
||||
|
||||
```bash
|
||||
# See what would be changed without making changes
|
||||
./sync-database-schema.sh --dry-run
|
||||
```
|
||||
|
||||
### Verbose Mode (Detailed Output)
|
||||
|
||||
```bash
|
||||
# Show detailed SQL statements and operations
|
||||
./sync-database-schema.sh --verbose
|
||||
```
|
||||
|
||||
### Combined Options
|
||||
|
||||
```bash
|
||||
# Dry run with verbose output
|
||||
./sync-database-schema.sh --dry-run --verbose
|
||||
```
|
||||
|
||||
## What It Does
|
||||
|
||||
### Step 1: Missing Tables
|
||||
- Compares tables between dev and prod
|
||||
- Creates any tables that exist in dev but not in prod
|
||||
- Includes all constraints, indexes, and triggers
|
||||
|
||||
### Step 2: Missing Columns
|
||||
- For each common table, compares columns
|
||||
- Adds missing columns with correct data types
|
||||
- Preserves NULL/NOT NULL constraints
|
||||
- Applies default values
|
||||
|
||||
### Step 3: Missing Indexes
|
||||
- Compares indexes between databases
|
||||
- Creates missing indexes in production
|
||||
- Excludes primary keys and unique constraints (handled separately)
|
||||
|
||||
### Step 4: Data Type Mismatches
|
||||
- Detects columns with different data types
|
||||
- Warns about mismatches requiring manual intervention
|
||||
- Does NOT automatically change data types (requires manual review)
|
||||
|
||||
## Safety Features
|
||||
|
||||
1. **Dry Run Mode**: Test changes before applying
|
||||
2. **Automatic Backups**: Uses temporary files for rollback if needed
|
||||
3. **Error Handling**: Stops on critical errors
|
||||
4. **Manual Review Required**: Data type changes need manual approval
|
||||
|
||||
## Exit Codes
|
||||
|
||||
- `0` - Success (changes made or databases already in sync)
|
||||
- `1` - Error occurred
|
||||
- `2` - Dry run mode: changes would be made
|
||||
|
||||
## Database Configuration
|
||||
|
||||
The script uses these default settings:
|
||||
|
||||
```bash
|
||||
DB_HOST="10.0.0.207"
|
||||
DB_USER="postgres"
|
||||
DB_PASSWORD="a3ppq"
|
||||
DEV_DB="parentflowdev"
|
||||
PROD_DB="parentflow"
|
||||
```
|
||||
|
||||
## Example Output
|
||||
|
||||
```
|
||||
================================================================================
|
||||
DATABASE SCHEMA SYNCHRONIZATION
|
||||
================================================================================
|
||||
|
||||
Development Database: parentflowdev
|
||||
Production Database: parentflow
|
||||
Dry Run Mode: false
|
||||
Verbose Mode: false
|
||||
|
||||
================================================================================
|
||||
|
||||
[INFO] Testing database connectivity...
|
||||
[SUCCESS] Database connectivity verified
|
||||
|
||||
[INFO] STEP 1: Checking for missing tables...
|
||||
[SUCCESS] No missing tables found
|
||||
|
||||
[INFO] STEP 2: Checking for missing columns in existing tables...
|
||||
[WARNING] Table 'users' has missing columns in production
|
||||
[INFO] Missing column: new_field (text)
|
||||
[SUCCESS] Added column: new_field to users
|
||||
|
||||
[INFO] STEP 3: Checking for missing indexes...
|
||||
[SUCCESS] No missing indexes found
|
||||
|
||||
[INFO] STEP 4: Checking for data type mismatches...
|
||||
[SUCCESS] No data type mismatches found
|
||||
|
||||
================================================================================
|
||||
SYNCHRONIZATION SUMMARY
|
||||
================================================================================
|
||||
|
||||
[SUCCESS] Schema synchronization completed successfully!
|
||||
[INFO] Production database has been updated to match development schema
|
||||
|
||||
[INFO] Synchronization completed at 2025-10-08 15:45:23
|
||||
================================================================================
|
||||
```
|
||||
|
||||
## When to Use
|
||||
|
||||
### Recommended Use Cases
|
||||
|
||||
1. **After Schema Changes in Dev**: Run after adding tables/columns in development
|
||||
2. **Before Deployment**: Ensure prod schema is up-to-date before deploying new code
|
||||
3. **Regular Maintenance**: Weekly/monthly schema sync checks
|
||||
4. **Post-Migration**: After running migrations in dev
|
||||
|
||||
### Not Recommended For
|
||||
|
||||
1. **Data Migration**: This script only handles schema, not data
|
||||
2. **Destructive Changes**: Doesn't drop tables or columns (by design)
|
||||
3. **Complex Type Changes**: Manual intervention required for data type changes
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Always run with --dry-run first**
|
||||
```bash
|
||||
./sync-database-schema.sh --dry-run
|
||||
```
|
||||
|
||||
2. **Review output carefully**
|
||||
- Check what tables/columns will be created
|
||||
- Verify no unexpected changes
|
||||
|
||||
3. **Backup production database before sync**
|
||||
```bash
|
||||
pg_dump -h 10.0.0.207 -U postgres parentflow > backup.sql
|
||||
```
|
||||
|
||||
4. **Run during maintenance windows**
|
||||
- Minimize impact on live traffic
|
||||
- Allows time for verification
|
||||
|
||||
5. **Test in staging first** (if available)
|
||||
- Validate script behavior
|
||||
- Identify potential issues
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Connection Errors
|
||||
|
||||
```
|
||||
[ERROR] Cannot connect to production database
|
||||
```
|
||||
|
||||
**Solution**: Check database credentials and network connectivity
|
||||
|
||||
### Permission Errors
|
||||
|
||||
```
|
||||
ERROR: permission denied for table users
|
||||
```
|
||||
|
||||
**Solution**: Ensure postgres user has required permissions
|
||||
|
||||
### Index Creation Failures
|
||||
|
||||
```
|
||||
[WARNING] Could not create index: idx_name
|
||||
```
|
||||
|
||||
**Solution**: Usually indicates index already exists or has dependencies. Check manually:
|
||||
```sql
|
||||
\d table_name
|
||||
```
|
||||
|
||||
### Data Type Mismatches
|
||||
|
||||
```
|
||||
[WARNING] Data type mismatches found in table: users
|
||||
```
|
||||
|
||||
**Solution**: Review differences and manually ALTER TABLE with appropriate type conversion:
|
||||
```sql
|
||||
ALTER TABLE users ALTER COLUMN field_name TYPE new_type USING field_name::new_type;
|
||||
```
|
||||
|
||||
## Manual Verification
|
||||
|
||||
After running the script, verify changes:
|
||||
|
||||
```bash
|
||||
# Compare table counts
|
||||
PGPASSWORD=a3ppq psql -h 10.0.0.207 -U postgres -d parentflowdev -c \
|
||||
"SELECT COUNT(*) FROM pg_tables WHERE schemaname = 'public';"
|
||||
|
||||
PGPASSWORD=a3ppq psql -h 10.0.0.207 -U postgres -d parentflow -c \
|
||||
"SELECT COUNT(*) FROM pg_tables WHERE schemaname = 'public';"
|
||||
|
||||
# Compare specific table structure
|
||||
PGPASSWORD=a3ppq psql -h 10.0.0.207 -U postgres -d parentflowdev -c "\d users"
|
||||
PGPASSWORD=a3ppq psql -h 10.0.0.207 -U postgres -d parentflow -c "\d users"
|
||||
```
|
||||
|
||||
## Automation
|
||||
|
||||
### Cron Job (Weekly Sync)
|
||||
|
||||
```bash
|
||||
# Add to crontab
|
||||
0 2 * * 0 /root/maternal-app/scripts/sync-database-schema.sh >> /var/log/db-sync.log 2>&1
|
||||
```
|
||||
|
||||
### CI/CD Integration
|
||||
|
||||
```yaml
|
||||
# .github/workflows/db-sync.yml
|
||||
- name: Sync Database Schema
|
||||
run: |
|
||||
cd scripts
|
||||
./sync-database-schema.sh --dry-run
|
||||
```
|
||||
|
||||
## Support
|
||||
|
||||
For issues or questions:
|
||||
1. Check the troubleshooting section above
|
||||
2. Review script logs with `--verbose` flag
|
||||
3. Verify database connectivity and permissions
|
||||
4. Check PostgreSQL logs for detailed errors
|
||||
|
||||
## Version History
|
||||
|
||||
- **v1.0** (2025-10-08): Initial release
|
||||
- Table creation
|
||||
- Column addition
|
||||
- Index synchronization
|
||||
- Type mismatch detection
|
||||
Reference in New Issue
Block a user