andrei/url_tracker_tool
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
url_tracker_tool/comprehensive_app_documentation.md

604 lines
22 KiB
Markdown
Raw Permalink Blame History

# 🚀 URL Redirect Tracker - Comprehensive Documentation
**Author:** Based on code review
**Date:** Generated from current codebase analysis
**Purpose:** Complete overview of current features, data flows, and user workflows
---
## 📋 Table of Contents
1. [Application Overview](#application-overview)
2. [Current Features](#current-features)
3. [Architecture](#architecture)
4. [Data Structures](#data-structures)
5. [User Workflows](#user-workflows)
6. [API Endpoints](#api-endpoints)
7. [Frontend Components](#frontend-components)
8. [Security Features](#security-features)
9. [Technical Implementation](#technical-implementation)
---
## 🎯 Application Overview
The URL Redirect Tracker is a full-stack web application that analyzes and visualizes HTTP redirect chains. It helps users understand where their URLs ultimately lead, detect security issues, and analyze redirect performance.
### Core Purpose
- Track complete redirect chains from initial URL to final destination
- Analyze HTTP status codes, headers, and response metadata
- Detect security issues (SSL downgrades, redirect loops)
- Provide detailed performance metrics and timing information
---
## ✅ Current Features
### 🔍 Core Tracking Features
- **Multi-method Support**: GET, HEAD, POST requests
- **Custom User-Agent**: Predefined options (Googlebot, Chrome, iPhone Safari, etc.)
- **SSL Certificate Analysis**: Detailed SSL/TLS information extraction
- **Response Body Capture**: Truncated response content for analysis
- **Timing Metrics**: Individual request duration tracking
- **Error Handling**: Graceful handling of failed requests
### 🛡️ Security & Analysis Features
- **Redirect Loop Detection**: Identifies circular redirects
- **SSL Downgrade Warning**: Detects HTTPS → HTTP transitions
- **Mixed Content Detection**: Security warnings for protocol downgrades
- **Certificate Validation**: SSL certificate details and validity
- **Tracking Parameter Analysis**: UTM, Facebook, Google click IDs extraction
### 🎨 User Interface Features
- **Dark/Light Mode Toggle**: System preference detection with manual override
- **Responsive Design**: Mobile-optimized interface
- **Print-Friendly**: Optimized print layouts
- **Collapsible Details**: Expandable sections for detailed information
- **Copy to Clipboard**: Easy URL copying functionality
- **Tabbed Content**: Organized information display (Headers, Body, Metadata, etc.)
### 📊 Data Visualization
- **Summary Statistics**: Redirect count, status code breakdown
- **Step-by-Step Breakdown**: Detailed chain visualization
- **Performance Metrics**: Response time analysis
- **SSL/Non-SSL Indicators**: Visual security status
### 🔧 API Features
- **Versioned API**: `/api/v1/track` with standardized responses
- **Rate Limiting**: 100 requests per hour per IP
- **Multiple Input Methods**: POST with JSON body, GET with query parameters
- **Backward Compatibility**: Legacy `/api/track` endpoint maintained
- **API Documentation**: Built-in documentation at `/api/docs`
---
## 🏗️ Architecture
### System Architecture (ASCII Diagram)
```
┌─────────────────────────────────────────────────────────────────┐
│ CLIENT BROWSER │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ index.html │ │ script.js │ │ styles.css │ │
│ │ (UI Layout) │ │ (Logic/API) │ │ (Styling) │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
│
│ HTTP Requests
▼
┌─────────────────────────────────────────────────────────────────┐
│ EXPRESS.JS SERVER │
│ (index.js) │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Rate Limiting │ │ API Routes │ │ Static Files │ │
│ │ Middleware │ │ Handler │ │ Serving │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
│
│ trackRedirects()
▼
┌─────────────────────────────────────────────────────────────────┐
│ REDIRECT TRACKING ENGINE │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ HTTP Client │ │ SSL Analysis │ │ Response │ │
│ │ (Axios) │ │ (HTTPS Agent) │ │ Processing │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
│
│ HTTP Requests
▼
┌─────────────────────────────────────────────────────────────────┐
│ TARGET WEBSITES │
│ (External URLs being tracked) │
└─────────────────────────────────────────────────────────────────┘
```
### Application Flow (ASCII Diagram)
```
User Input → Form Validation → API Request → Redirect Tracking → Response Processing → UI Update
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ User enters│ │ Frontend │ │ Backend │ │ External │
│ URL │────▶│ validates & │────▶│ tracks │────▶│ websites │
│ │ │ sends API │ │ redirects │ │ │
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
│ │ │
▼ ▼ │
┌─────────────┐ ┌─────────────┐ │
│ Loading │ │ Recursive │ │
│ Indicator │ │ Function │◀─────────────┘
└─────────────┘ └─────────────┘
│ │
▼ ▼
┌─────────────┐ ┌─────────────┐
│ Results │◀───│ Response │
│ Display │ │ Data │
└─────────────┘ └─────────────┘
```
---
## 📊 Data Structures
### Redirect Object Structure
```javascript
{
url: "https://example.com", // Current URL
timestamp: 1234567890123, // Request timestamp
isSSL: true, // SSL status
duration: 245, // Request duration (ms)
statusCode: 301, // HTTP status code
statusText: "Moved Permanently", // Status text
final: false, // Is final destination
error: null, // Error message if any
metadata: { // Request/response metadata
status: 301,
statusText: "Moved Permanently",
headers: {
"location": "https://www.example.com",
"content-type": "text/html",
"server": "nginx/1.18.0",
// ... other headers
},
contentType: "text/html",
contentLength: "178",
server: "nginx/1.18.0",
date: "Thu, 01 Jan 2024 12:00:00 GMT",
protocol: "https:",
method: "GET"
},
responseBody: "<html>...</html>", // Truncated response body
sslInfo: { // SSL certificate info
valid: true,
issuer: { /* certificate issuer */ },
subject: { /* certificate subject */ },
validFrom: "Jan 1 00:00:00 2024 GMT",
validTo: "Jan 1 00:00:00 2025 GMT",
fingerprint: "AA:BB:CC:DD:..."
}
}
```
### API Response Structure (v1)
```javascript
{
success: true,
status: 200,
data: {
url: "http://example.com",
method: "GET",
redirectCount: 2,
finalUrl: "https://www.example.com/",
finalStatusCode: 200,
redirects: [
// Array of redirect objects (see above)
]
}
}
```
### Tracking Parameters Structure
```javascript
{
"Source": "google", // utm_source
"Medium": "cpc", // utm_medium
"Campaign": "summer_sale", // utm_campaign
"Google Click ID": "abc123def456", // gclid
"Facebook Click ID": "xyz789uvw012" // fbclid
// ... other detected parameters
}
```
---
## 👤 User Workflows
### Primary User Journey (ASCII Diagram)
```
START
│
▼
┌─────────────────┐
│ User visits │
│ application │
└─────────────────┘
│
▼
┌─────────────────┐ ┌─────────────────┐
│ Enter URL │────▶│ Select │
│ in input │ │ HTTP method │
│ field │ │ (GET/HEAD/ │
└─────────────────┘ │ POST) │
│ └─────────────────┘
▼ │
┌─────────────────┐ │
│ Choose User │◀────────────┘
│ Agent (opt.) │
└─────────────────┘
│
▼
┌─────────────────┐
│ Click Track │
│ Redirects │
└─────────────────┘
│
▼
┌─────────────────┐ ┌─────────────────┐
│ Loading │ │ Backend │
│ indicator │────▶│ processes │
│ shown │ │ request │
└─────────────────┘ └─────────────────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Results │◀───│ Redirect │
│ displayed │ │ chain │
│ │ │ tracked │
└─────────────────┘ └─────────────────┘
│
▼
┌─────────────────┐
│ User can: │
│ • View details│
│ • Copy URLs │
│ • Print │
│ • Toggle mode │
└─────────────────┘
│
▼
END
```
### Error Handling Flow
```
Error Occurs
│
▼
┌─────────────────┐ ┌─────────────────┐
│ Network │ │ Invalid │
│ Error │ │ URL │
└─────────────────┘ └─────────────────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Display │ │ Form │
│ error │ │ validation │
│ message │ │ message │
└─────────────────┘ └─────────────────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Allow retry │ │ Highlight │
│ with same │ │ input field │
│ parameters │ │ │
└─────────────────┘ └─────────────────┘
```
---
## 🔌 API Endpoints
### Endpoint Overview
| Method | Endpoint | Purpose | Rate Limited |
|--------|----------|---------|--------------|
| POST | `/api/track` | Legacy redirect tracking | No |
| POST | `/api/v1/track` | Modern redirect tracking | Yes (100/hr) |
| GET | `/api/v1/track` | URL tracking via query params | Yes (100/hr) |
| GET | `/api/docs` | API documentation | No |
| GET | `/` | Serve main application | No |
### Request/Response Flow (ASCII Diagram)
```
Client Request
│
▼
┌─────────────────┐
│ Rate Limit │
│ Check │
└─────────────────┘
│
▼
┌─────────────────┐
│ Input │
│ Validation │
└─────────────────┘
│
▼
┌─────────────────┐
│ URL │
│ Normalization │
└─────────────────┘
│
▼
┌─────────────────┐
│ Redirect │
│ Tracking │
│ (Recursive) │
└─────────────────┘
│
▼
┌─────────────────┐
│ Response │
│ Formatting │
└─────────────────┘
│
▼
Client Response
```
---
## 🎨 Frontend Components
### Component Structure
```
Frontend Application
├── HTML Structure (index.html)
│ ├── Header (Title + Dark mode toggle)
│ ├── Form (URL input + options)
│ ├── Loading indicator
│ ├── Error display
│ ├── Warnings display
│ └── Results section
│ ├── Summary
│ ├── Tab navigation (List/Graph)
│ └── Content areas
│
├── JavaScript Logic (script.js)
│ ├── Form handling
│ ├── API communication
│ ├── Results processing
│ ├── UI state management
│ ├── Dark mode logic
│ ├── Copy to clipboard
│ ├── Tab switching
│ └── Error handling
│
└── CSS Styling (styles.css)
├── Layout & Grid
├── Dark/Light themes
├── Responsive design
├── Component styles
└── Print styles
```
### State Management Flow
```
Initial State
│
▼
┌─────────────────┐
│ Form Ready │
│ • Empty input │
│ • Default │
│ options │
└─────────────────┘
│
▼
┌─────────────────┐ ┌─────────────────┐
│ User Input │────▶│ Validation │
│ State │ │ State │
└─────────────────┘ └─────────────────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Loading │ │ Error │
│ State │ │ State │
└─────────────────┘ └─────────────────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Results │ │ Retry │
│ State │ │ State │
└─────────────────┘ └─────────────────┘
```
---
## 🛡️ Security Features
### Security Measures Implemented
1. **Rate Limiting**
- 100 requests per hour per IP address
- Prevents abuse and DoS attacks
2. **Input Validation**
- URL format validation
- Protocol enforcement (HTTP/HTTPS)
- Parameter sanitization
3. **SSL Certificate Analysis**
- Certificate validity checking
- Issuer verification
- Expiration date monitoring
4. **Security Warnings**
- SSL downgrade detection
- Mixed content warnings
- Redirect loop identification
5. **Response Sanitization**
- Response body truncation (5000 chars max)
- Header filtering
- Error message sanitization
### Security Analysis Flow
```
URL Input
│
▼
┌─────────────────┐
│ Protocol │
│ Analysis │
│ (HTTP/HTTPS) │
└─────────────────┘
│
▼
┌─────────────────┐
│ SSL Cert │
│ Validation │
│ (if HTTPS) │
└─────────────────┘
│
▼
┌─────────────────┐
│ Redirect │
│ Chain │
│ Analysis │
└─────────────────┘
│
▼
┌─────────────────┐
│ Security │
│ Warning │
│ Generation │
└─────────────────┘
```
---
## ⚙️ Technical Implementation
### Core Dependencies
```json
{
"axios": "^1.6.7", // HTTP client
"express": "^4.18.2", // Web framework
"express-rate-limit": "^5.5.1" // Rate limiting
}
```
### Key Technical Features
1. **Recursive Redirect Tracking**
- Follows redirects programmatically
- Captures metadata at each step
- Handles relative/absolute URLs
2. **Custom HTTPS Agent**
- SSL certificate extraction
- Self-signed certificate handling
- Security information gathering
3. **Request Configuration**
- Custom headers support
- User-agent spoofing
- Timeout handling (15 seconds)
4. **Response Processing**
- Content-type detection
- Body size limitation
- Header normalization
### Performance Considerations
```
Request Optimization
├── Connection reuse
├── Timeout management
├── Memory management
│ ├── Response truncation
│ └── Object cleanup
└── Error boundary handling
```
### Error Handling Strategy
```
Error Types
├── Network Errors
│ ├── Connection timeout
│ ├── DNS resolution
│ └── Connection refused
├── HTTP Errors
│ ├── 4xx Client errors
│ └── 5xx Server errors
├── Application Errors
│ ├── Invalid input
│ ├── Rate limit exceeded
│ └── Processing errors
└── Security Errors
├── SSL certificate issues
└── Protocol violations
```
---
## 📈 Future Enhancement Opportunities
Based on the features document analysis, potential improvements include:
1. **Visual Enhancements**
- Mermaid.js graph implementation
- Interactive redirect visualization
- Performance timeline charts
2. **Security Scanning**
- Google Safe Browsing API integration
- robots.txt analysis
- Meta redirect detection
3. **Advanced Analysis**
- Response time comparison
- Geographic routing analysis
- Cache-control header analysis
4. **Export Features**
- JSON/CSV export
- Report generation
- Historical tracking
---
## 🎯 Conclusion
The URL Redirect Tracker is a comprehensive tool that provides detailed analysis of HTTP redirect chains with strong security features and user-friendly interface. The current implementation covers core functionality with room for enhancement in visualization and advanced security scanning features.
**Key Strengths:**
- Robust redirect tracking with detailed metadata
- Security-focused analysis (SSL, loops, downgrades)
- Professional UI with dark mode and responsive design
- Well-structured API with rate limiting
- Comprehensive error handling
**Architecture Benefits:**
- Clean separation of concerns
- RESTful API design
- Scalable backend structure
- Maintainable frontend code
- Security-first approach
Reference in New Issue View Git Blame Copy Permalink