Add prompt injection protection for AI endpoints
Implemented comprehensive security against prompt injection attacks: **Detection Patterns:** - System prompt manipulation (ignore/disregard/forget instructions) - Role manipulation (pretend to be, act as) - Data exfiltration (show system prompt, list users) - Command injection (execute code, run command) - Jailbreak attempts (DAN mode, developer mode, admin mode) **Input Validation:** - Maximum length: 2,000 characters - Maximum line length: 500 characters - Maximum repeated characters: 20 consecutive - Special character ratio limit: 30% - HTML/JavaScript injection blocking **Sanitization:** - HTML tag removal - Zero-width character stripping - Control character removal - Whitespace normalization **Rate Limiting:** - 5 suspicious attempts per minute per user - Automatic clearing on successful validation - Per-user tracking with session storage **Context Awareness:** - Parenting keyword validation - Domain-appropriate scope checking - Lenient validation for short prompts **Implementation:** - lib/security/promptSecurity.ts - Core validation logic - app/api/ai/chat/route.ts - Integrated validation - scripts/test-prompt-injection.mjs - 19 test cases (all passing) - lib/security/README.md - Documentation **Test Coverage:** ✅ Valid parenting questions (2 tests) ✅ System manipulation attempts (4 tests) ✅ Role manipulation (1 test) ✅ Data exfiltration (3 tests) ✅ Command injection (2 tests) ✅ Jailbreak techniques (2 tests) ✅ Length attacks (2 tests) ✅ Character encoding attacks (2 tests) ✅ Edge cases (1 test) All suspicious attempts are logged with user ID, reason, risk level, and timestamp for security monitoring. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -1,8 +1,9 @@
|
||||
import { NextRequest, NextResponse } from 'next/server';
|
||||
import { aiLimiter } from '@/lib/middleware/rateLimiter';
|
||||
import { validateAIPrompt, logSuspiciousPrompt } from '@/lib/security/promptSecurity';
|
||||
|
||||
/**
|
||||
* AI chat endpoint with rate limiting
|
||||
* AI chat endpoint with rate limiting and prompt injection protection
|
||||
* Limited to 10 queries per hour for free tier users
|
||||
*/
|
||||
export async function POST(request: NextRequest) {
|
||||
@@ -14,10 +15,46 @@ export async function POST(request: NextRequest) {
|
||||
const body = await request.json();
|
||||
const { message, childId, conversationId } = body;
|
||||
|
||||
// Validate message input
|
||||
if (!message || typeof message !== 'string') {
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: 'AI_INVALID_INPUT',
|
||||
message: 'Message must be a non-empty string',
|
||||
},
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
|
||||
// Validate and sanitize prompt for injection attempts
|
||||
const validationResult = validateAIPrompt(message);
|
||||
|
||||
if (!validationResult.isValid) {
|
||||
// Log security event
|
||||
logSuspiciousPrompt(
|
||||
message,
|
||||
request.headers.get('x-user-id') || undefined,
|
||||
validationResult.reason || 'Unknown',
|
||||
validationResult.riskLevel
|
||||
);
|
||||
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: 'AI_PROMPT_REJECTED',
|
||||
message: validationResult.reason || 'Your message could not be processed',
|
||||
riskLevel: validationResult.riskLevel,
|
||||
},
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
|
||||
// Use sanitized prompt for AI request
|
||||
const sanitizedMessage = validationResult.sanitizedPrompt || message;
|
||||
|
||||
// TODO: Implement actual AI chat logic
|
||||
// This is a placeholder - actual AI integration will be handled by backend
|
||||
|
||||
// For now, forward to backend API
|
||||
// For now, forward to backend API with sanitized message
|
||||
const backendUrl = process.env.NEXT_PUBLIC_API_URL || 'http://localhost:3020';
|
||||
const response = await fetch(`${backendUrl}/api/v1/ai/chat`, {
|
||||
method: 'POST',
|
||||
@@ -26,7 +63,11 @@ export async function POST(request: NextRequest) {
|
||||
// Forward auth token from client
|
||||
Authorization: request.headers.get('Authorization') || '',
|
||||
},
|
||||
body: JSON.stringify({ message, childId, conversationId }),
|
||||
body: JSON.stringify({
|
||||
message: sanitizedMessage, // Use sanitized message
|
||||
childId,
|
||||
conversationId,
|
||||
}),
|
||||
});
|
||||
|
||||
const data = await response.json();
|
||||
|
||||
Reference in New Issue
Block a user