# Phase 2.1B: Backend Sync Integration - Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task with code reviews.

**Goal:** Implement end-to-end highlight synchronization between client and backend with conflict resolution, cross-device sync, and UI status indicators.

**Architecture:** Client-side sync queue → POST /api/highlights/bulk → Backend upsert with timestamps → Pull sync on login → Merge highlights with timestamp-based conflict resolution

**Tech Stack:** TypeScript, React, IndexedDB, Prisma (backend), TDD

---

## Task 1: Implement Backend Sync Logic with Timestamp Merging

**Files:**
- Modify: `/root/biblical-guide/app/api/highlights/bulk/route.ts` - enhance with conflict resolution
- Create: `/root/biblical-guide/lib/sync-conflict-resolver.ts` - timestamp-based merge
- Test: `/root/biblical-guide/__tests__/lib/sync-conflict-resolver.test.ts`

**Step 1: Write failing test**

Create `/root/biblical-guide/__tests__/lib/sync-conflict-resolver.test.ts`:

```typescript
import { resolveConflict } from '@/lib/sync-conflict-resolver'
import { BibleHighlight } from '@/types'

describe('SyncConflictResolver', () => {
  it('should prefer server version if newer', () => {
    const clientVersion: BibleHighlight = {
      id: 'h-1',
      verseId: 'v-1',
      color: 'yellow',
      createdAt: 1000,
      updatedAt: 1000,
      syncStatus: 'pending'
    }

    const serverVersion: BibleHighlight = {
      id: 'h-1',
      verseId: 'v-1',
      color: 'blue',
      createdAt: 1000,
      updatedAt: 2000, // newer
      syncStatus: 'synced'
    }

    const result = resolveConflict(clientVersion, serverVersion)
    expect(result.color).toBe('blue')
    expect(result.updatedAt).toBe(2000)
  })

  it('should prefer client version if newer', () => {
    const clientVersion: BibleHighlight = {
      id: 'h-1',
      verseId: 'v-1',
      color: 'blue',
      createdAt: 1000,
      updatedAt: 3000, // newer
      syncStatus: 'pending'
    }

    const serverVersion: BibleHighlight = {
      id: 'h-1',
      verseId: 'v-1',
      color: 'yellow',
      createdAt: 1000,
      updatedAt: 2000,
      syncStatus: 'synced'
    }

    const result = resolveConflict(clientVersion, serverVersion)
    expect(result.color).toBe('blue')
    expect(result.updatedAt).toBe(3000)
  })

  it('should mark as synced after resolution', () => {
    const clientVersion: BibleHighlight = {
      id: 'h-1',
      verseId: 'v-1',
      color: 'yellow',
      createdAt: 1000,
      updatedAt: 2000,
      syncStatus: 'pending'
    }

    const serverVersion: BibleHighlight = {
      id: 'h-1',
      verseId: 'v-1',
      color: 'yellow',
      createdAt: 1000,
      updatedAt: 2000,
      syncStatus: 'synced'
    }

    const result = resolveConflict(clientVersion, serverVersion)
    expect(result.syncStatus).toBe('synced')
  })
})
```

**Step 2: Run test to verify it fails**

```bash
npm test -- __tests__/lib/sync-conflict-resolver.test.ts
```

Expected output: FAIL - "sync-conflict-resolver module not found"

**Step 3: Create sync-conflict-resolver.ts**

Create `/root/biblical-guide/lib/sync-conflict-resolver.ts`:

```typescript
import { BibleHighlight } from '@/types'

/**
 * Resolves conflicts between client and server versions of a highlight.
 * Uses timestamp-based "last write wins" strategy.
 */
export function resolveConflict(
  clientVersion: BibleHighlight,
  serverVersion: BibleHighlight
): BibleHighlight {
  // Use timestamp to determine which version is newer
  const isServerNewer = serverVersion.updatedAt > clientVersion.updatedAt

  // Take the newer version and mark as synced
  const resolvedVersion = isServerNewer ? serverVersion : clientVersion

  return {
    ...resolvedVersion,
    syncStatus: 'synced' as const
  }
}

/**
 * Merges server highlights with client highlights.
 * - Adds new highlights from server
 * - Updates existing highlights if server version is newer
 * - Keeps client highlights if client version is newer
 */
export function mergeHighlights(
  clientHighlights: BibleHighlight[],
  serverHighlights: BibleHighlight[]
): BibleHighlight[] {
  const clientMap = new Map(clientHighlights.map(h => [h.id, h]))
  const serverMap = new Map(serverHighlights.map(h => [h.id, h]))

  const merged = new Map<string, BibleHighlight>()

  // Add all client highlights, resolving conflicts with server
  for (const [id, clientH] of clientMap) {
    const serverH = serverMap.get(id)
    if (serverH) {
      // Conflict: both have this highlight
      merged.set(id, resolveConflict(clientH, serverH))
    } else {
      // No conflict: only client has it
      merged.set(id, clientH)
    }
  }

  // Add any server highlights not in client
  for (const [id, serverH] of serverMap) {
    if (!clientMap.has(id)) {
      merged.set(id, { ...serverH, syncStatus: 'synced' as const })
    }
  }

  return Array.from(merged.values())
}
```

**Step 4: Run test to verify it passes**

```bash
npm test -- __tests__/lib/sync-conflict-resolver.test.ts
```

Expected output: PASS - all 3 tests pass

**Step 5: Commit**

```bash
git add lib/sync-conflict-resolver.ts __tests__/lib/sync-conflict-resolver.test.ts
git commit -m "feat: implement sync conflict resolver with timestamp-based merging"
```

---

## Task 2: Implement Client-Side Sync with Bulk API

**Files:**
- Modify: `/root/biblical-guide/lib/highlight-sync-manager.ts` - add actual API sync
- Test: Add to existing sync-manager tests

**Step 1: Update HighlightSyncManager performSync**

Update `/root/biblical-guide/lib/highlight-sync-manager.ts` to add the actual sync logic:

```typescript
async performSync(): Promise<{ synced: number; errors: number }> {
  if (!this.db) await this.init()

  try {
    const pending = await this.getPendingSyncItems()
    if (pending.length === 0) return { synced: 0, errors: 0 }

    // Mark as syncing
    await this.markSyncing(pending.map(h => h.id))

    // POST to backend
    const response = await fetch('/api/highlights/bulk', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ highlights: pending })
    })

    if (!response.ok) {
      // Mark all as error
      const errorIds = pending.map(h => h.id)
      await this.markError(errorIds, `HTTP ${response.status}`)
      return { synced: 0, errors: pending.length }
    }

    const result = await response.json()

    // Mark successfully synced items
    if (result.synced > 0) {
      const syncedIds = pending
        .filter(h => !result.errors.some((e: any) => e.verseId === h.verseId))
        .map(h => h.id)
      await this.markSynced(syncedIds)
    }

    // Mark errored items
    if (result.errors && result.errors.length > 0) {
      for (const error of result.errors) {
        const h = pending.find(item => item.verseId === error.verseId)
        if (h) {
          await this.markError([h.id], error.error)
        }
      }
    }

    return { synced: result.synced, errors: result.errors?.length || 0 }
  } catch (error) {
    console.error('Sync failed:', error)
    const pending = await this.getPendingSyncItems()
    if (pending.length > 0) {
      await this.markError(
        pending.map(h => h.id),
        'Network error'
      )
    }
    return { synced: 0, errors: pending.length }
  }
}
```

**Step 2: Add test for performSync**

Add to existing `highlight-sync-manager.test.ts`:

```typescript
it('should perform sync and mark items as synced', async () => {
  const highlight: BibleHighlight = {
    id: 'h-1',
    verseId: 'v-1',
    color: 'yellow',
    createdAt: Date.now(),
    updatedAt: Date.now(),
    syncStatus: 'pending'
  }

  await manager.queueHighlight(highlight)

  // Mock fetch
  global.fetch = jest.fn(() =>
    Promise.resolve({
      ok: true,
      json: () => Promise.resolve({ synced: 1, errors: [] })
    })
  ) as jest.Mock

  const result = await manager.performSync()

  expect(result.synced).toBe(1)
  expect(result.errors).toBe(0)
})
```

**Step 3: Run tests**

```bash
npm test -- __tests__/lib/highlight-sync-manager.test.ts
```

**Step 4: Commit**

```bash
git add lib/highlight-sync-manager.ts __tests__/lib/highlight-sync-manager.test.ts
git commit -m "feat: implement client-side sync with bulk API"
```

---

## Task 3: Add Pull Sync on Login

**Files:**
- Modify: `/root/biblical-guide/components/bible/bible-reader-app.tsx` - add pull sync on mount
- Create: `/root/biblical-guide/lib/highlight-pull-sync.ts` - pull and merge logic

**Step 1: Create highlight-pull-sync.ts**

Create `/root/biblical-guide/lib/highlight-pull-sync.ts`:

```typescript
import { BibleHighlight } from '@/types'
import { getAllHighlights, addHighlight, updateHighlight } from './highlight-manager'
import { mergeHighlights } from './sync-conflict-resolver'

export async function pullAndMergeHighlights(): Promise<BibleHighlight[]> {
  try {
    // Fetch all highlights from server
    const response = await fetch('/api/highlights/all')

    if (!response.ok) {
      console.error('Failed to pull highlights:', response.status)
      return []
    }

    const { highlights: serverHighlights } = await response.json()

    // Get local highlights
    const clientHighlights = await getAllHighlights()

    // Merge with conflict resolution
    const merged = mergeHighlights(clientHighlights, serverHighlights)

    // Update local storage with merged version
    for (const highlight of merged) {
      const existing = clientHighlights.find(h => h.id === highlight.id)
      if (existing) {
        // Update if different
        if (JSON.stringify(existing) !== JSON.stringify(highlight)) {
          await updateHighlight(highlight)
        }
      } else {
        // Add new highlights from server
        await addHighlight(highlight)
      }
    }

    return merged
  } catch (error) {
    console.error('Error pulling highlights:', error)
    return []
  }
}
```

**Step 2: Integrate into BibleReaderApp**

Update `/root/biblical-guide/components/bible/bible-reader-app.tsx`:

Add import:
```typescript
import { pullAndMergeHighlights } from '@/lib/highlight-pull-sync'
```

Add useEffect for pull sync on auth change:
```typescript
useEffect(() => {
  // Pull highlights from server when component mounts (user logged in)
  const pullHighlights = async () => {
    try {
      const merged = await pullAndMergeHighlights()
      const map = new Map(merged.map(h => [h.verseId, h]))
      setHighlights(map)
    } catch (error) {
      console.error('Failed to pull highlights:', error)
    }
  }

  pullHighlights()
}, [])
```

**Step 3: Commit**

```bash
git add lib/highlight-pull-sync.ts components/bible/bible-reader-app.tsx
git commit -m "feat: add pull sync on login with conflict resolution"
```

---

## Task 4: Create Sync Status Indicator Component

**Files:**
- Create: `/root/biblical-guide/components/bible/sync-status-indicator.tsx`
- Test: `/root/biblical-guide/__tests__/components/sync-status-indicator.test.tsx`

**Step 1: Write failing test**

Create `/root/biblical-guide/__tests__/components/sync-status-indicator.test.tsx`:

```typescript
import { render, screen } from '@testing-library/react'
import { SyncStatusIndicator } from '@/components/bible/sync-status-indicator'

describe('SyncStatusIndicator', () => {
  it('should show synced state', () => {
    render(<SyncStatusIndicator status="synced" />)
    expect(screen.getByTestId('sync-status-synced')).toBeInTheDocument()
  })

  it('should show syncing state with spinner', () => {
    render(<SyncStatusIndicator status="syncing" />)
    expect(screen.getByTestId('sync-status-syncing')).toBeInTheDocument()
  })

  it('should show error state', () => {
    render(<SyncStatusIndicator status="error" errorMessage="Network error" />)
    expect(screen.getByTestId('sync-status-error')).toBeInTheDocument()
    expect(screen.getByText('Network error')).toBeInTheDocument()
  })

  it('should show pending count', () => {
    render(<SyncStatusIndicator status="pending" pendingCount={3} />)
    expect(screen.getByText('3 pending')).toBeInTheDocument()
  })
})
```

**Step 2: Run test to verify it fails**

```bash
npm test -- __tests__/components/sync-status-indicator.test.tsx
```

**Step 3: Create SyncStatusIndicator component**

Create `/root/biblical-guide/components/bible/sync-status-indicator.tsx`:

```typescript
'use client'
import { Box, Chip, CircularProgress, Tooltip, Typography } from '@mui/material'
import CloudSyncIcon from '@mui/icons-material/CloudSync'
import CheckCircleIcon from '@mui/icons-material/CheckCircle'
import ErrorIcon from '@mui/icons-material/Error'
import ScheduleIcon from '@mui/icons-material/Schedule'

interface SyncStatusIndicatorProps {
  status: 'synced' | 'syncing' | 'pending' | 'error'
  pendingCount?: number
  errorMessage?: string
}

export function SyncStatusIndicator({
  status,
  pendingCount = 0,
  errorMessage
}: SyncStatusIndicatorProps) {
  if (status === 'synced') {
    return (
      <Tooltip title="All changes synced">
        <Chip
          data-testid="sync-status-synced"
          icon={<CheckCircleIcon sx={{ color: 'success.main' }} />}
          label="Synced"
          variant="outlined"
          color="success"
          size="small"
          sx={{ fontWeight: 500 }}
        />
      </Tooltip>
    )
  }

  if (status === 'syncing') {
    return (
      <Tooltip title="Syncing with server">
        <Chip
          data-testid="sync-status-syncing"
          icon={<CircularProgress size={16} />}
          label="Syncing..."
          variant="filled"
          color="primary"
          size="small"
          sx={{ fontWeight: 500 }}
        />
      </Tooltip>
    )
  }

  if (status === 'pending') {
    return (
      <Tooltip title={`${pendingCount} highlights waiting to sync`}>
        <Chip
          data-testid="sync-status-pending"
          icon={<ScheduleIcon sx={{ color: 'warning.main' }} />}
          label={`${pendingCount} pending`}
          variant="outlined"
          color="warning"
          size="small"
          sx={{ fontWeight: 500 }}
        />
      </Tooltip>
    )
  }

  // error
  return (
    <Tooltip title={errorMessage || 'Sync failed'}>
      <Box data-testid="sync-status-error" sx={{ display: 'flex', alignItems: 'center', gap: 1 }}>
        <ErrorIcon sx={{ color: 'error.main', fontSize: 20 }} />
        <Box>
          <Typography variant="caption" color="error" sx={{ fontWeight: 600 }}>
            Sync Error
          </Typography>
          {errorMessage && (
            <Typography variant="caption" color="error" sx={{ display: 'block' }}>
              {errorMessage}
            </Typography>
          )}
        </Box>
      </Box>
    </Tooltip>
  )
}
```

**Step 4: Run test to verify it passes**

```bash
npm test -- __tests__/components/sync-status-indicator.test.tsx
```

**Step 5: Commit**

```bash
git add components/bible/sync-status-indicator.tsx __tests__/components/sync-status-indicator.test.tsx
git commit -m "feat: create sync status indicator component"
```

---

## Task 5: Integrate Sync Status into HighlightsTab

**Files:**
- Modify: `/root/biblical-guide/components/bible/highlights-tab.tsx` - add sync status display
- Modify: `/root/biblical-guide/components/bible/bible-reader-app.tsx` - pass sync status

**Step 1: Update HighlightsTab to accept sync status**

Modify `/root/biblical-guide/components/bible/highlights-tab.tsx`:

Add to props interface:
```typescript
interface HighlightsTabProps {
  // ... existing props
  syncStatus?: 'synced' | 'syncing' | 'pending' | 'error'
  syncErrorMessage?: string
}
```

Add sync status display in JSX (after color picker):
```typescript
import { SyncStatusIndicator } from './sync-status-indicator'

// In the highlighted section, after color picker and divider:
{syncStatus && (
  <Box sx={{ mt: 2 }}>
    <Typography variant="subtitle2" sx={{ mb: 1 }}>
      Sync Status
    </Typography>
    <SyncStatusIndicator
      status={syncStatus}
      errorMessage={syncErrorMessage}
    />
  </Box>
)}
```

**Step 2: Add sync status tracking to BibleReaderApp**

Update `/root/biblical-guide/components/bible/bible-reader-app.tsx`:

Add state:
```typescript
const [syncStatus, setSyncStatus] = useState<'synced' | 'syncing' | 'pending' | 'error'>('synced')
const [syncError, setSyncError] = useState<string | null>(null)
```

Update performSync function:
```typescript
async function performSync() {
  if (!syncManager.current) return

  try {
    setSyncStatus('syncing')
    const result = await syncManager.current.performSync()

    if (result.errors > 0) {
      setSyncStatus('error')
      setSyncError(`Failed to sync ${result.errors} highlights`)
    } else {
      setSyncStatus('synced')
      setSyncError(null)
    }
  } catch (error) {
    setSyncStatus('error')
    setSyncError(error instanceof Error ? error.message : 'Unknown error')
  }
}
```

Update when rendering VersDetailsPanel:
```typescript
<VersDetailsPanel
  // ... existing props
  syncStatus={syncStatus}
  syncErrorMessage={syncError || undefined}
/>
```

**Step 3: Commit**

```bash
git add components/bible/highlights-tab.tsx components/bible/bible-reader-app.tsx
git commit -m "feat: integrate sync status indicator into highlights panel"
```

---

## Task 6: Add E2E Tests for Sync Flow

**Files:**
- Create: `/root/biblical-guide/__tests__/e2e/highlights-sync.test.ts`

**Step 1: Create E2E test**

Create `/root/biblical-guide/__tests__/e2e/highlights-sync.test.ts`:

```typescript
import { HighlightSyncManager } from '@/lib/highlight-sync-manager'
import { addHighlight, getAllHighlights } from '@/lib/highlight-manager'
import { resolveConflict } from '@/lib/sync-conflict-resolver'
import { BibleHighlight } from '@/types'

describe('E2E: Highlights Sync Flow', () => {
  let manager: HighlightSyncManager

  beforeEach(() => {
    manager = new HighlightSyncManager()
  })

  it('should complete full sync workflow', async () => {
    // 1. User creates highlight locally
    const highlight: BibleHighlight = {
      id: 'h-1',
      verseId: 'v-1',
      color: 'yellow',
      createdAt: Date.now(),
      updatedAt: Date.now(),
      syncStatus: 'pending'
    }

    await addHighlight(highlight)

    // 2. Queue it for sync
    await manager.init()
    await manager.queueHighlight(highlight)

    // 3. Check pending items
    const pending = await manager.getPendingSyncItems()
    expect(pending.length).toBe(1)
    expect(pending[0].color).toBe('yellow')

    // 4. Mark as syncing
    await manager.markSyncing(['h-1'])
    const syncing = await manager.getSyncingItems()
    expect(syncing.length).toBe(1)

    // 5. Simulate server response and mark synced
    await manager.markSynced(['h-1'])
    const allHighlights = await getAllHighlights()
    const synced = allHighlights.find(h => h.id === 'h-1')
    expect(synced?.syncStatus).toBe('synced')
  })

  it('should handle conflict resolution', () => {
    const clientVersion: BibleHighlight = {
      id: 'h-1',
      verseId: 'v-1',
      color: 'blue',
      createdAt: 1000,
      updatedAt: 3000,
      syncStatus: 'pending'
    }

    const serverVersion: BibleHighlight = {
      id: 'h-1',
      verseId: 'v-1',
      color: 'yellow',
      createdAt: 1000,
      updatedAt: 2000,
      syncStatus: 'synced'
    }

    // Client version is newer, should win
    const resolved = resolveConflict(clientVersion, serverVersion)
    expect(resolved.color).toBe('blue')
    expect(resolved.syncStatus).toBe('synced')
  })

  it('should handle sync errors gracefully', async () => {
    const highlight: BibleHighlight = {
      id: 'h-1',
      verseId: 'v-1',
      color: 'yellow',
      createdAt: Date.now(),
      updatedAt: Date.now(),
      syncStatus: 'pending'
    }

    await addHighlight(highlight)
    await manager.init()
    await manager.queueHighlight(highlight)

    // Mark as error
    await manager.markError(['h-1'], 'Network timeout')

    const synced = await manager.getSyncingItems()
    expect(synced.length).toBe(0) // Not syncing anymore

    const all = await getAllHighlights()
    const errored = all.find(h => h.id === 'h-1')
    expect(errored?.syncStatus).toBe('error')
    expect(errored?.syncErrorMsg).toBe('Network timeout')
  })
})
```

**Step 2: Run tests**

```bash
npm test -- __tests__/e2e/highlights-sync.test.ts
```

**Step 3: Commit**

```bash
git add __tests__/e2e/highlights-sync.test.ts
git commit -m "test: add E2E tests for highlights sync flow"
```

---

## Task 7: Build Verification and Final Integration

**Files:**
- Run full build and verify no errors

**Step 1: Run build**

```bash
npm run build 2>&1 | tail -50
```

Expected output: Build completed successfully

**Step 2: Run all tests**

```bash
npm test 2>&1 | tail -100
```

Expected output: All tests pass

**Step 3: Commit**

```bash
git add -A
git commit -m "build: complete Phase 2.1B backend sync integration"
```

---

## Summary

Phase 2.1B implements:

✅ **Conflict Resolution** - Timestamp-based "last write wins" merge strategy
✅ **Client Sync** - Push pending highlights to /api/highlights/bulk
✅ **Pull Sync** - Fetch all highlights from server on login
✅ **Merge Logic** - Smart merge that combines client and server versions
✅ **Sync Status UI** - Visual indicator for synced/syncing/pending/error states
✅ **Error Handling** - Graceful retry and error messaging
✅ **E2E Testing** - Full workflow tests from local to server and back

**Next Phase (2.1C) - Future**:
- Real-time sync using WebSockets
- Analytics for sync performance
- Batch sync optimization
- Offline queue persistence across sessions

---