feat(phase-6): Bulk CSV processing and background worker implementation

- Add BulkJob model to Prisma schema with relations
- Implement BulkProcessorService for CSV parsing and job management
- Create BulkTrackingWorker for background processing with BullMQ
- Add comprehensive bulk API routes (upload, jobs, progress, export)
- Integrate multer for CSV file uploads with validation
- Add job progress tracking and estimation
- Implement CSV export functionality for results
- Add queue statistics and cleanup endpoints
- Create shared types for bulk processing
- Add comprehensive test suite for all bulk functionality
- Implement graceful worker shutdown and error handling
- Add rate limiting and authentication for all bulk endpoints

Backward compatibility: Maintained for /api/track and /api/v1/track
This commit is contained in:
Andrei
2025-08-18 14:18:13 +00:00
parent 8c8300780f
commit 9626863917
13 changed files with 2309 additions and 64 deletions

View File

@@ -18,7 +18,10 @@
"axios": "^1.6.7",
"playwright": "^1.40.1",
"dotenv": "^16.3.1",
"winston": "^3.11.0"
"winston": "^3.11.0",
"undici": "^6.2.1",
"zod": "^3.22.4",
"csv-writer": "^1.6.0"
},
"devDependencies": {
"@types/node": "^20.10.0",