biblical-guide.com/docs/plans/2025-01-11-phase-2-rich-annotations-design.md

# Phase 2.1 Design: Rich Annotations & Highlighting

**Date**: 2025-01-11
**Status**: Approved Design
**Objective**: Build a complete highlighting and annotation system that works offline-first with seamless sync. Users can color-code verses for study and reference management.

---

## Core Philosophy

- **Instant feedback**: Highlights appear immediately when user acts
- **Never lose work**: All highlights persist locally, sync when possible
- **Distraction-free**: Visual indicators are subtle; details reveal on demand
- **Cross-device sync**: Annotations follow the user across devices

---

## Feature Specifications

### 1. Highlighting System

#### Colors & Interaction
- **4 highlight colors**: Yellow (default), Orange, Pink, Blue
- **Two-gesture interaction**:
  1. Single tap verse → Opens details panel (existing behavior)
  2. Long-press or swipe verse → Highlights with default color (yellow)
     - Shows mini toast: "Highlighted"
     - Verse background changes color immediately
  3. Tap highlighted verse → Details panel opens with Highlights tab active
     - Shows current color + ColorPicker
     - User can change color or delete highlight

#### Visual Representation
- **Colored background** on highlighted verses
- **Opacity**: 0.3 (subtle, maintains text contrast)
- **Colors**:
  - Yellow: `rgba(255, 193, 7, 0.3)` - Default, general marking
  - Orange: `rgba(255, 152, 0, 0.3)` - Important, needs attention
  - Pink: `rgba(233, 30, 99, 0.3)` - Devotional, personal significance
  - Blue: `rgba(33, 150, 243, 0.3)` - Reference, study focus

#### Storage
- **Database**: IndexedDB table `highlights`
- **Schema**:
  ```
  {
    id: string (UUID),
    verseId: string,
    userId: string (from localStorage auth),
    color: 'yellow' | 'orange' | 'pink' | 'blue',
    createdAt: timestamp,
    updatedAt: timestamp,
    syncStatus: 'pending' | 'syncing' | 'synced' | 'error',
    syncErrorMsg?: string
  }
  ```

### 2. Cross-References

#### Visual Indicator
- **Small link icon** (🔗) or dot next to verse number when cross-references exist
- **Placement**: Subtle, doesn't interrupt reading
- **Behavior**: Clicking verse opens details panel with Cross-References tab

#### Cross-Reference Display
- **Tab in VersDetailsPanel**: "Cross-References"
- **Format**: Collapsible list showing:
  - Book name (e.g., "John")
  - Chapter:verse reference (e.g., "3:16")
  - 1-line preview of the verse text
  - Tap to jump to that verse

#### Quick Jump Behavior
- **Tap reference** → Navigate to verse
- **Add to history**: User can go back to original verse
- **Smooth transition**: No page reload, updates reading view

#### Data Source
- **Endpoint**: `GET /api/bible/cross-references?verseId={verseId}`
- **Lazy-loaded**: Only fetch when user opens Cross-References tab
- **Cached**: Store in IndexedDB with 7-day expiration

### 3. Local-First Sync Strategy

#### Immediate Local Storage
- All highlights saved to IndexedDB instantly when user acts
- Provides instant feedback, works offline
- No waiting for network round-trip

#### Automatic Sync Queue
- **Background service** tracks `syncStatus` for each highlight:
  - `pending`: Created locally, not yet synced
  - `syncing`: Currently pushing to server
  - `synced`: Successfully synced, in-sync with server
  - `error`: Failed to sync, will retry

#### Auto-Sync Timing
- **Interval**: Every 30 seconds when online
- **Batch operation**: POST all pending highlights in one request
- **Smart batching**: Only send items with `syncStatus: 'pending'` or `'error'`
- **Exponential backoff**: Failed syncs retry after 30s, 60s, 120s, then give up

#### Conflict Resolution
- **Strategy**: Last-modified timestamp wins
- **Scenario**: User highlights same verse on two devices
  - Device 1: Highlights yellow at 10:00:00
  - Device 2: Highlights pink at 10:00:05
  - Result: Pink wins (newer timestamp), displayed on both devices after sync
- **Safety**: No data loss—version history kept server-side for audit

#### Offline Fallback
- All operations (highlight, change color, delete) queued locally
- Sync indicator shows "Offline" state
- When connection returns: `syncStatus: 'pending'` items auto-sync

#### Sync Status Indicator
- **Location**: Footer bar (right side, near existing sync indicator)
- **States**:
  - "Syncing..." (briefly while POST in flight)
  - "Synced ✓" (green checkmark, 2 second display)
  - "Sync failed" (red icon, expandable for retry)
  - "Offline" (gray icon)
- **Manual retry**: User can click "Retry" on failed syncs from settings

### 4. Component Architecture

#### Enhanced Components

**HighlightsTab** (NEW - in VersDetailsPanel)
```
HighlightsTab
├── HighlightToggle
│   └── "Highlight this verse" button (if not highlighted)
│   └── "Remove highlight" button (if highlighted)
├── ColorPicker (if highlighted)
│   ├── 4 color swatches (yellow, orange, pink, blue)
│   ├── Selected color indicator
│   └── OnColorChange → Update highlight, queue sync
└── HighlightMetadata
    ├── Created: [date/time]
    └── Last modified: [date/time]
```

**VerseRenderer** (enhanced in ReadingView)
```
VerseRenderer
├── HighlightBackground
│   └── Colored background if verse is highlighted
├── VerseNumber + CrossRefIndicator
│   └── Small icon if cross-references available
└── VerseText
    └── Regular text, no inline linking
```

**HighlightSyncManager** (NEW - in BibleReaderApp)
```
HighlightSyncManager
├── IndexedDB operations
│   ├── addHighlight(verseId, color)
│   ├── updateHighlight(highlightId, color)
│   ├── deleteHighlight(highlightId)
│   └── getAllHighlights()
├── Sync queue logic
│   ├── getPendingHighlights()
│   ├── markSyncing(ids)
│   ├── markSynced(ids)
│   └── markError(ids, msg)
└── Auto-sync interval
    └── Every 30s: fetch pending → POST batch → update status
```

### 5. Data Flow

#### Highlight Creation
```
1. User long-presses verse
2. VerseRenderer detects long-press
3. Create highlight entry in IndexedDB
   { verseId, color: 'yellow', syncStatus: 'pending' }
4. VerseRenderer background changes color
5. Show toast "Highlighted"
6. SyncManager picks it up in next 30s cycle → POST to backend
```

#### Highlight Color Change
```
1. User tap verse → Details panel opens
2. HighlightsTab shows current color + ColorPicker
3. User taps new color
4. Update highlight in IndexedDB with new color + new timestamp
5. VerseRenderer background updates immediately
6. syncStatus changed to 'pending'
7. SyncManager syncs in next cycle
```

#### Offline → Reconnect Flow
```
1. User highlights while offline
   → Stored in IndexedDB with syncStatus: 'pending'
2. Connection returns
3. SyncManager detects online status change
4. Fetches all syncStatus: 'pending' or 'error' items
5. POSTs to /api/highlights/bulk
6. Updates syncStatus to 'synced'
7. Shows sync status indicator
```

#### Cross-Device Sync
```
1. App loads on Device 2
2. Fetch /api/highlights/all from backend
3. For each highlight from server:
   - Check if exists locally (by verseId + userId)
   - If not: Add to IndexedDB
   - If exists: Compare timestamps, keep newer
4. Show user any conflicts (rare)
5. Render highlights with merged data
```

### 6. Backend API Endpoints (NEW)

#### POST /api/highlights
Create a single highlight for authenticated user.

```
Request:
{
  verseId: string,
  color: 'yellow' | 'orange' | 'pink' | 'blue',
  createdAt: timestamp
}

Response:
{
  id: string (UUID),
  verseId: string,
  userId: string,
  color: string,
  createdAt: timestamp,
  updatedAt: timestamp,
  syncStatus: 'synced'
}
```

#### POST /api/highlights/bulk
Batch sync highlights (create or update).

```
Request:
{
  highlights: [
    {
      id?: string,
      verseId: string,
      color: string,
      createdAt: timestamp,
      updatedAt: timestamp
    }
  ]
}

Response:
{
  synced: number,
  errors: [{ verseId, error }],
  serverTime: timestamp
}
```

#### GET /api/highlights/all
Fetch all highlights for authenticated user (for cross-device sync).

```
Response:
{
  highlights: [
    {
      id: string,
      verseId: string,
      color: string,
      createdAt: timestamp,
      updatedAt: timestamp
    }
  ],
  serverTime: timestamp
}
```

#### GET /api/bible/cross-references
Get cross-referenced verses for a given verse.

```
Request: GET /api/bible/cross-references?verseId={verseId}

Response:
{
  verseId: string,
  references: [
    {
      refVerseId: string,
      bookName: string,
      chapter: number,
      verse: number,
      preview: string (first 60 chars)
    }
  ]
}
```

### 7. Error Handling & Resilience

**Sync Failures**
- Network timeout: Auto-retry after 30s with exponential backoff
- 400/401 (invalid request): Remove from queue, log error
- 5xx (server error): Keep in queue, retry next cycle
- Display "Sync failed" in footer with manual retry button

**Offline Highlighting**
- All operations queue locally, appear immediately
- When online: Auto-sync without user intervention
- If sync fails: User notified, can manually retry from settings

**IndexedDB Quota Exceeded**
- Highlights table should never exceed reasonable size (< 1MB typical)
- If quota warning: Suggest clearing old highlights from settings
- Oldest highlights (by date) suggested for removal first

**Cross-Device Conflicts**
- Rare: User highlights same verse on two devices at same second
- Resolution: Newer timestamp wins (automatic)
- User sees no warning (conflict handled transparently)

### 8. Testing Strategy

#### Unit Tests
- Highlight color validation (only 4 valid colors)
- Sync queue operations (add, remove, get pending)
- Timestamp-based conflict resolution
- IndexedDB CRUD operations
- Batch sync request formatting

#### Integration Tests
- Highlight creation → immediate display → queued sync
- Offline highlight → reconnect → verify sync success
- Color change persistence across storage layers
- Cross-device highlight fetch and merge
- Sync conflict resolution (timestamp comparison)

#### E2E Tests
- User highlights verse → sees background change → goes offline → comes back online → highlight is synced
- User highlights on Device 1 → reads on Device 2 → sees highlight immediately after fetch
- User deletes highlight → sync → verify removal on all devices
- Bulk operations: highlight multiple verses rapidly, verify all sync

#### Manual Testing
- Desktop browsers: Chrome, Firefox, Safari
- Mobile: iOS Safari, Chrome Mobile, Android browsers
- Network conditions: Fast 3G, slow 3G, offline
- Sync conflict scenarios (use network throttling to trigger)

---

## Success Criteria

- **Offline**: Can highlight and change colors without internet
- **Sync**: Auto-syncs all highlights within 60 seconds of reconnection
- **Performance**: Highlighting action responds in < 200ms
- **Reliability**: No lost highlights after sync
- **UX**: User never confused about sync state (status indicator clear)
- **Accessibility**: All interactions keyboard-navigable

---

## Implementation Dependencies

### Already Available
- ✅ IndexedDB infrastructure (cache-manager.ts)
- ✅ Details panel infrastructure (VersDetailsPanel.tsx)
- ✅ Verse rendering with click handlers
- ✅ ReadingView component structure
- ✅ Auth system (user identification)

### New Dependencies
- API endpoints (backend implementation)
- Highlight sync manager (new service)
- Color picker component (can use Material-UI)

---

## Future Enhancements (Phase 3+)

- **Highlight statistics**: "You've highlighted 47 verses across 12 books"
- **Highlight search**: Find all yellow highlights, or search within highlights
- **Highlight export**: Export all highlights as PDF or CSV with context
- **Highlight sharing**: Share specific highlighted passages with study groups
- **Highlight collections**: Group highlights into "studies" or "topics"

---

## References

- Current reader: `/root/biblical-guide/components/bible/bible-reader-app.tsx`
- Verse panel: `/root/biblical-guide/components/bible/verse-details-panel.tsx`
- Cache manager: `/root/biblical-guide/lib/cache-manager.ts`
- API Bible endpoints: `/root/biblical-guide/app/api/bible/`