feat(phase-6): Bulk CSV processing and background worker implementation
- Add BulkJob model to Prisma schema with relations - Implement BulkProcessorService for CSV parsing and job management - Create BulkTrackingWorker for background processing with BullMQ - Add comprehensive bulk API routes (upload, jobs, progress, export) - Integrate multer for CSV file uploads with validation - Add job progress tracking and estimation - Implement CSV export functionality for results - Add queue statistics and cleanup endpoints - Create shared types for bulk processing - Add comprehensive test suite for all bulk functionality - Implement graceful worker shutdown and error handling - Add rate limiting and authentication for all bulk endpoints Backward compatibility: Maintained for /api/track and /api/v1/track
This commit is contained in:
@@ -36,7 +36,10 @@
|
||||
"mermaid": "^10.6.1",
|
||||
"markdown-it": "^14.0.0",
|
||||
"file-type": "^19.0.0",
|
||||
"mime-types": "^2.1.35"
|
||||
"mime-types": "^2.1.35",
|
||||
"csv-parser": "^3.0.0",
|
||||
"csv-writer": "^1.6.0",
|
||||
"multer": "^1.4.5-lts.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/express": "^4.17.21",
|
||||
@@ -55,6 +58,8 @@
|
||||
"@types/supertest": "^2.0.16",
|
||||
"@types/puppeteer": "^7.0.4",
|
||||
"@types/markdown-it": "^13.0.7",
|
||||
"@types/mime-types": "^2.1.4"
|
||||
"@types/mime-types": "^2.1.4",
|
||||
"@types/multer": "^1.4.11",
|
||||
"@types/csv-parser": "^3.0.0"
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user