feat(phase-6): Bulk CSV processing and background worker implementation

- Add BulkJob model to Prisma schema with relations
- Implement BulkProcessorService for CSV parsing and job management
- Create BulkTrackingWorker for background processing with BullMQ
- Add comprehensive bulk API routes (upload, jobs, progress, export)
- Integrate multer for CSV file uploads with validation
- Add job progress tracking and estimation
- Implement CSV export functionality for results
- Add queue statistics and cleanup endpoints
- Create shared types for bulk processing
- Add comprehensive test suite for all bulk functionality
- Implement graceful worker shutdown and error handling
- Add rate limiting and authentication for all bulk endpoints

Backward compatibility: Maintained for /api/track and /api/v1/track
This commit is contained in:
Andrei
2025-08-18 14:18:13 +00:00
parent 8c8300780f
commit 9626863917
13 changed files with 2309 additions and 64 deletions

View File

@@ -36,7 +36,10 @@
"mermaid": "^10.6.1",
"markdown-it": "^14.0.0",
"file-type": "^19.0.0",
"mime-types": "^2.1.35"
"mime-types": "^2.1.35",
"csv-parser": "^3.0.0",
"csv-writer": "^1.6.0",
"multer": "^1.4.5-lts.1"
},
"devDependencies": {
"@types/express": "^4.17.21",
@@ -55,6 +58,8 @@
"@types/supertest": "^2.0.16",
"@types/puppeteer": "^7.0.4",
"@types/markdown-it": "^13.0.7",
"@types/mime-types": "^2.1.4"
"@types/mime-types": "^2.1.4",
"@types/multer": "^1.4.11",
"@types/csv-parser": "^3.0.0"
}
}