Initial commit: URL Redirect Tracker application with comprehensive documentation
This commit is contained in:
Andrei
603
comprehensive_app_documentation.md
Normal file
603
comprehensive_app_documentation.md
Normal file
@@ -0,0 +1,603 @@
|
||||
# 🚀 URL Redirect Tracker - Comprehensive Documentation
|
||||
|
||||
**Author:** Based on code review
|
||||
**Date:** Generated from current codebase analysis
|
||||
**Purpose:** Complete overview of current features, data flows, and user workflows
|
||||
|
||||
---
|
||||
|
||||
## 📋 Table of Contents
|
||||
|
||||
1. [Application Overview](#application-overview)
|
||||
2. [Current Features](#current-features)
|
||||
3. [Architecture](#architecture)
|
||||
4. [Data Structures](#data-structures)
|
||||
5. [User Workflows](#user-workflows)
|
||||
6. [API Endpoints](#api-endpoints)
|
||||
7. [Frontend Components](#frontend-components)
|
||||
8. [Security Features](#security-features)
|
||||
9. [Technical Implementation](#technical-implementation)
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Application Overview
|
||||
|
||||
The URL Redirect Tracker is a full-stack web application that analyzes and visualizes HTTP redirect chains. It helps users understand where their URLs ultimately lead, detect security issues, and analyze redirect performance.
|
||||
|
||||
### Core Purpose
|
||||
- Track complete redirect chains from initial URL to final destination
|
||||
- Analyze HTTP status codes, headers, and response metadata
|
||||
- Detect security issues (SSL downgrades, redirect loops)
|
||||
- Provide detailed performance metrics and timing information
|
||||
|
||||
---
|
||||
|
||||
## ✅ Current Features
|
||||
|
||||
### 🔍 Core Tracking Features
|
||||
- **Multi-method Support**: GET, HEAD, POST requests
|
||||
- **Custom User-Agent**: Predefined options (Googlebot, Chrome, iPhone Safari, etc.)
|
||||
- **SSL Certificate Analysis**: Detailed SSL/TLS information extraction
|
||||
- **Response Body Capture**: Truncated response content for analysis
|
||||
- **Timing Metrics**: Individual request duration tracking
|
||||
- **Error Handling**: Graceful handling of failed requests
|
||||
|
||||
### 🛡️ Security & Analysis Features
|
||||
- **Redirect Loop Detection**: Identifies circular redirects
|
||||
- **SSL Downgrade Warning**: Detects HTTPS → HTTP transitions
|
||||
- **Mixed Content Detection**: Security warnings for protocol downgrades
|
||||
- **Certificate Validation**: SSL certificate details and validity
|
||||
- **Tracking Parameter Analysis**: UTM, Facebook, Google click IDs extraction
|
||||
|
||||
### 🎨 User Interface Features
|
||||
- **Dark/Light Mode Toggle**: System preference detection with manual override
|
||||
- **Responsive Design**: Mobile-optimized interface
|
||||
- **Print-Friendly**: Optimized print layouts
|
||||
- **Collapsible Details**: Expandable sections for detailed information
|
||||
- **Copy to Clipboard**: Easy URL copying functionality
|
||||
- **Tabbed Content**: Organized information display (Headers, Body, Metadata, etc.)
|
||||
|
||||
### 📊 Data Visualization
|
||||
- **Summary Statistics**: Redirect count, status code breakdown
|
||||
- **Step-by-Step Breakdown**: Detailed chain visualization
|
||||
- **Performance Metrics**: Response time analysis
|
||||
- **SSL/Non-SSL Indicators**: Visual security status
|
||||
|
||||
### 🔧 API Features
|
||||
- **Versioned API**: `/api/v1/track` with standardized responses
|
||||
- **Rate Limiting**: 100 requests per hour per IP
|
||||
- **Multiple Input Methods**: POST with JSON body, GET with query parameters
|
||||
- **Backward Compatibility**: Legacy `/api/track` endpoint maintained
|
||||
- **API Documentation**: Built-in documentation at `/api/docs`
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Architecture
|
||||
|
||||
### System Architecture (ASCII Diagram)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ CLIENT BROWSER │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
|
||||
│ │ index.html │ │ script.js │ │ styles.css │ │
|
||||
│ │ (UI Layout) │ │ (Logic/API) │ │ (Styling) │ │
|
||||
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ HTTP Requests
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ EXPRESS.JS SERVER │
|
||||
│ (index.js) │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
|
||||
│ │ Rate Limiting │ │ API Routes │ │ Static Files │ │
|
||||
│ │ Middleware │ │ Handler │ │ Serving │ │
|
||||
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ trackRedirects()
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ REDIRECT TRACKING ENGINE │
|
||||
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
|
||||
│ │ HTTP Client │ │ SSL Analysis │ │ Response │ │
|
||||
│ │ (Axios) │ │ (HTTPS Agent) │ │ Processing │ │
|
||||
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
│
|
||||
│ HTTP Requests
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ TARGET WEBSITES │
|
||||
│ (External URLs being tracked) │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Application Flow (ASCII Diagram)
|
||||
|
||||
```
|
||||
User Input → Form Validation → API Request → Redirect Tracking → Response Processing → UI Update
|
||||
|
||||
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
|
||||
│ User enters│ │ Frontend │ │ Backend │ │ External │
|
||||
│ URL │────▶│ validates & │────▶│ tracks │────▶│ websites │
|
||||
│ │ │ sends API │ │ redirects │ │ │
|
||||
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
|
||||
│ │ │
|
||||
▼ ▼ │
|
||||
┌─────────────┐ ┌─────────────┐ │
|
||||
│ Loading │ │ Recursive │ │
|
||||
│ Indicator │ │ Function │◀─────────────┘
|
||||
└─────────────┘ └─────────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌─────────────┐ ┌─────────────┐
|
||||
│ Results │◀───│ Response │
|
||||
│ Display │ │ Data │
|
||||
└─────────────┘ └─────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Data Structures
|
||||
|
||||
### Redirect Object Structure
|
||||
|
||||
```javascript
|
||||
{
|
||||
url: "https://example.com", // Current URL
|
||||
timestamp: 1234567890123, // Request timestamp
|
||||
isSSL: true, // SSL status
|
||||
duration: 245, // Request duration (ms)
|
||||
statusCode: 301, // HTTP status code
|
||||
statusText: "Moved Permanently", // Status text
|
||||
final: false, // Is final destination
|
||||
error: null, // Error message if any
|
||||
metadata: { // Request/response metadata
|
||||
status: 301,
|
||||
statusText: "Moved Permanently",
|
||||
headers: {
|
||||
"location": "https://www.example.com",
|
||||
"content-type": "text/html",
|
||||
"server": "nginx/1.18.0",
|
||||
// ... other headers
|
||||
},
|
||||
contentType: "text/html",
|
||||
contentLength: "178",
|
||||
server: "nginx/1.18.0",
|
||||
date: "Thu, 01 Jan 2024 12:00:00 GMT",
|
||||
protocol: "https:",
|
||||
method: "GET"
|
||||
},
|
||||
responseBody: "<html>...</html>", // Truncated response body
|
||||
sslInfo: { // SSL certificate info
|
||||
valid: true,
|
||||
issuer: { /* certificate issuer */ },
|
||||
subject: { /* certificate subject */ },
|
||||
validFrom: "Jan 1 00:00:00 2024 GMT",
|
||||
validTo: "Jan 1 00:00:00 2025 GMT",
|
||||
fingerprint: "AA:BB:CC:DD:..."
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### API Response Structure (v1)
|
||||
|
||||
```javascript
|
||||
{
|
||||
success: true,
|
||||
status: 200,
|
||||
data: {
|
||||
url: "http://example.com",
|
||||
method: "GET",
|
||||
redirectCount: 2,
|
||||
finalUrl: "https://www.example.com/",
|
||||
finalStatusCode: 200,
|
||||
redirects: [
|
||||
// Array of redirect objects (see above)
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Tracking Parameters Structure
|
||||
|
||||
```javascript
|
||||
{
|
||||
"Source": "google", // utm_source
|
||||
"Medium": "cpc", // utm_medium
|
||||
"Campaign": "summer_sale", // utm_campaign
|
||||
"Google Click ID": "abc123def456", // gclid
|
||||
"Facebook Click ID": "xyz789uvw012" // fbclid
|
||||
// ... other detected parameters
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 👤 User Workflows
|
||||
|
||||
### Primary User Journey (ASCII Diagram)
|
||||
|
||||
```
|
||||
START
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ User visits │
|
||||
│ application │
|
||||
└─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ Enter URL │────▶│ Select │
|
||||
│ in input │ │ HTTP method │
|
||||
│ field │ │ (GET/HEAD/ │
|
||||
└─────────────────┘ │ POST) │
|
||||
│ └─────────────────┘
|
||||
▼ │
|
||||
┌─────────────────┐ │
|
||||
│ Choose User │◀────────────┘
|
||||
│ Agent (opt.) │
|
||||
└─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Click Track │
|
||||
│ Redirects │
|
||||
└─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ Loading │ │ Backend │
|
||||
│ indicator │────▶│ processes │
|
||||
│ shown │ │ request │
|
||||
└─────────────────┘ └─────────────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ Results │◀───│ Redirect │
|
||||
│ displayed │ │ chain │
|
||||
│ │ │ tracked │
|
||||
└─────────────────┘ └─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ User can: │
|
||||
│ • View details│
|
||||
│ • Copy URLs │
|
||||
│ • Print │
|
||||
│ • Toggle mode │
|
||||
└─────────────────┘
|
||||
│
|
||||
▼
|
||||
END
|
||||
```
|
||||
|
||||
### Error Handling Flow
|
||||
|
||||
```
|
||||
Error Occurs
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ Network │ │ Invalid │
|
||||
│ Error │ │ URL │
|
||||
└─────────────────┘ └─────────────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ Display │ │ Form │
|
||||
│ error │ │ validation │
|
||||
│ message │ │ message │
|
||||
└─────────────────┘ └─────────────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ Allow retry │ │ Highlight │
|
||||
│ with same │ │ input field │
|
||||
│ parameters │ │ │
|
||||
└─────────────────┘ └─────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔌 API Endpoints
|
||||
|
||||
### Endpoint Overview
|
||||
|
||||
| Method | Endpoint | Purpose | Rate Limited |
|
||||
|--------|----------|---------|--------------|
|
||||
| POST | `/api/track` | Legacy redirect tracking | No |
|
||||
| POST | `/api/v1/track` | Modern redirect tracking | Yes (100/hr) |
|
||||
| GET | `/api/v1/track` | URL tracking via query params | Yes (100/hr) |
|
||||
| GET | `/api/docs` | API documentation | No |
|
||||
| GET | `/` | Serve main application | No |
|
||||
|
||||
### Request/Response Flow (ASCII Diagram)
|
||||
|
||||
```
|
||||
Client Request
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Rate Limit │
|
||||
│ Check │
|
||||
└─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Input │
|
||||
│ Validation │
|
||||
└─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ URL │
|
||||
│ Normalization │
|
||||
└─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Redirect │
|
||||
│ Tracking │
|
||||
│ (Recursive) │
|
||||
└─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Response │
|
||||
│ Formatting │
|
||||
└─────────────────┘
|
||||
│
|
||||
▼
|
||||
Client Response
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎨 Frontend Components
|
||||
|
||||
### Component Structure
|
||||
|
||||
```
|
||||
Frontend Application
|
||||
├── HTML Structure (index.html)
|
||||
│ ├── Header (Title + Dark mode toggle)
|
||||
│ ├── Form (URL input + options)
|
||||
│ ├── Loading indicator
|
||||
│ ├── Error display
|
||||
│ ├── Warnings display
|
||||
│ └── Results section
|
||||
│ ├── Summary
|
||||
│ ├── Tab navigation (List/Graph)
|
||||
│ └── Content areas
|
||||
│
|
||||
├── JavaScript Logic (script.js)
|
||||
│ ├── Form handling
|
||||
│ ├── API communication
|
||||
│ ├── Results processing
|
||||
│ ├── UI state management
|
||||
│ ├── Dark mode logic
|
||||
│ ├── Copy to clipboard
|
||||
│ ├── Tab switching
|
||||
│ └── Error handling
|
||||
│
|
||||
└── CSS Styling (styles.css)
|
||||
├── Layout & Grid
|
||||
├── Dark/Light themes
|
||||
├── Responsive design
|
||||
├── Component styles
|
||||
└── Print styles
|
||||
```
|
||||
|
||||
### State Management Flow
|
||||
|
||||
```
|
||||
Initial State
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Form Ready │
|
||||
│ • Empty input │
|
||||
│ • Default │
|
||||
│ options │
|
||||
└─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ User Input │────▶│ Validation │
|
||||
│ State │ │ State │
|
||||
└─────────────────┘ └─────────────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ Loading │ │ Error │
|
||||
│ State │ │ State │
|
||||
└─────────────────┘ └─────────────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ Results │ │ Retry │
|
||||
│ State │ │ State │
|
||||
└─────────────────┘ └─────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🛡️ Security Features
|
||||
|
||||
### Security Measures Implemented
|
||||
|
||||
1. **Rate Limiting**
|
||||
- 100 requests per hour per IP address
|
||||
- Prevents abuse and DoS attacks
|
||||
|
||||
2. **Input Validation**
|
||||
- URL format validation
|
||||
- Protocol enforcement (HTTP/HTTPS)
|
||||
- Parameter sanitization
|
||||
|
||||
3. **SSL Certificate Analysis**
|
||||
- Certificate validity checking
|
||||
- Issuer verification
|
||||
- Expiration date monitoring
|
||||
|
||||
4. **Security Warnings**
|
||||
- SSL downgrade detection
|
||||
- Mixed content warnings
|
||||
- Redirect loop identification
|
||||
|
||||
5. **Response Sanitization**
|
||||
- Response body truncation (5000 chars max)
|
||||
- Header filtering
|
||||
- Error message sanitization
|
||||
|
||||
### Security Analysis Flow
|
||||
|
||||
```
|
||||
URL Input
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Protocol │
|
||||
│ Analysis │
|
||||
│ (HTTP/HTTPS) │
|
||||
└─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ SSL Cert │
|
||||
│ Validation │
|
||||
│ (if HTTPS) │
|
||||
└─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Redirect │
|
||||
│ Chain │
|
||||
│ Analysis │
|
||||
└─────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Security │
|
||||
│ Warning │
|
||||
│ Generation │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ Technical Implementation
|
||||
|
||||
### Core Dependencies
|
||||
|
||||
```json
|
||||
{
|
||||
"axios": "^1.6.7", // HTTP client
|
||||
"express": "^4.18.2", // Web framework
|
||||
"express-rate-limit": "^5.5.1" // Rate limiting
|
||||
}
|
||||
```
|
||||
|
||||
### Key Technical Features
|
||||
|
||||
1. **Recursive Redirect Tracking**
|
||||
- Follows redirects programmatically
|
||||
- Captures metadata at each step
|
||||
- Handles relative/absolute URLs
|
||||
|
||||
2. **Custom HTTPS Agent**
|
||||
- SSL certificate extraction
|
||||
- Self-signed certificate handling
|
||||
- Security information gathering
|
||||
|
||||
3. **Request Configuration**
|
||||
- Custom headers support
|
||||
- User-agent spoofing
|
||||
- Timeout handling (15 seconds)
|
||||
|
||||
4. **Response Processing**
|
||||
- Content-type detection
|
||||
- Body size limitation
|
||||
- Header normalization
|
||||
|
||||
### Performance Considerations
|
||||
|
||||
```
|
||||
Request Optimization
|
||||
├── Connection reuse
|
||||
├── Timeout management
|
||||
├── Memory management
|
||||
│ ├── Response truncation
|
||||
│ └── Object cleanup
|
||||
└── Error boundary handling
|
||||
```
|
||||
|
||||
### Error Handling Strategy
|
||||
|
||||
```
|
||||
Error Types
|
||||
├── Network Errors
|
||||
│ ├── Connection timeout
|
||||
│ ├── DNS resolution
|
||||
│ └── Connection refused
|
||||
├── HTTP Errors
|
||||
│ ├── 4xx Client errors
|
||||
│ └── 5xx Server errors
|
||||
├── Application Errors
|
||||
│ ├── Invalid input
|
||||
│ ├── Rate limit exceeded
|
||||
│ └── Processing errors
|
||||
└── Security Errors
|
||||
├── SSL certificate issues
|
||||
└── Protocol violations
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📈 Future Enhancement Opportunities
|
||||
|
||||
Based on the features document analysis, potential improvements include:
|
||||
|
||||
1. **Visual Enhancements**
|
||||
- Mermaid.js graph implementation
|
||||
- Interactive redirect visualization
|
||||
- Performance timeline charts
|
||||
|
||||
2. **Security Scanning**
|
||||
- Google Safe Browsing API integration
|
||||
- robots.txt analysis
|
||||
- Meta redirect detection
|
||||
|
||||
3. **Advanced Analysis**
|
||||
- Response time comparison
|
||||
- Geographic routing analysis
|
||||
- Cache-control header analysis
|
||||
|
||||
4. **Export Features**
|
||||
- JSON/CSV export
|
||||
- Report generation
|
||||
- Historical tracking
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Conclusion
|
||||
|
||||
The URL Redirect Tracker is a comprehensive tool that provides detailed analysis of HTTP redirect chains with strong security features and user-friendly interface. The current implementation covers core functionality with room for enhancement in visualization and advanced security scanning features.
|
||||
|
||||
**Key Strengths:**
|
||||
- Robust redirect tracking with detailed metadata
|
||||
- Security-focused analysis (SSL, loops, downgrades)
|
||||
- Professional UI with dark mode and responsive design
|
||||
- Well-structured API with rate limiting
|
||||
- Comprehensive error handling
|
||||
|
||||
**Architecture Benefits:**
|
||||
- Clean separation of concerns
|
||||
- RESTful API design
|
||||
- Scalable backend structure
|
||||
- Maintainable frontend code
|
||||
- Security-first approach
|
||||
Reference in New Issue
Block a user