URL Shortener Microservice - System Design Document
1. Project Overview
1.1 Purpose
Design and implement a robust HTTP URL Shortener Microservice that provides core URL shortening
functionality with comprehensive analytics capabilities.
1.2 Business Requirements
Primary Function: Convert long URLs into short, manageable links
Analytics: Track click statistics and user interactions
Reliability: Handle high availability with proper error handling
Performance: Fast redirection with minimal latency
Compliance: Mandatory logging for audit and monitoring
1.3 Scope
Single microservice architecture
RESTful API endpoints
In-memory data storage (MVP)
Real-time analytics tracking
Automated cleanup of expired URLs
2. System Architecture
2.1 High-Level Architecture
┌─────────────────────────────────────────────────────────┐
│ Client Layer │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐│
│ │ Browser │ │ Mobile │ │ API Clients ││
│ │ │ │ App │ │ (Postman/Curl) ││
│ └─────────────┘ └─────────────┘ └─────────────────────┘│
└─────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────┐
│ Load Balancer (Future) │
└─────────────────────────────────────────────────────────┘
│
┌─────────────────────────────────────────────────────────┐
│ URL Shortener Microservice │
│ ┌─────────────────────────────────────────────────────┐│
│ │ Express.js Application Layer ││
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐││
│ │ │ Logging │ │ Validation │ │ Error Handling │││
│ │ │ Middleware │ │ Middleware │ │ Middleware │││
│ │ └─────────────┘ └─────────────┘ └─────────────────┘││
│ └─────────────────────────────────────────────────────┘│
│ ┌─────────────────────────────────────────────────────┐│
│ │ Business Logic Layer ││
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐││
│ │ │URL Creation │ │ Analytics │ │ Cleanup │││
│ │ │ Service │ │ Service │ │ Service │││
│ │ └─────────────┘ └─────────────┘ └─────────────────┘││
│ └─────────────────────────────────────────────────────┘│
│ ┌─────────────────────────────────────────────────────┐│
│ │ Data Storage Layer ││
│ │ ┌─────────────┐ ┌─────────────────────────────────┐││
│ │ │ URL ││ Analytics │││
│ │ │ Database │ │ Database │││
│ │ │(Map Object) │ │ (Map Object) │││
│ │ └─────────────┘ └─────────────────────────────────┘││
│ └─────────────────────────────────────────────────────┘│
└─────────────────────────────────────────────────────────┘
2.2 Component Architecture
┌──────────────────────────────────────────┐
│ HTTP Server (Express.js) │
└──────────────────────────────────────────┘
│
┌───────────────┼───────────────┐
│ │ │
▼ ▼ ▼
┌─────────┐ ┌─────────────┐ ┌─────────────┐
│ POST │ │ GET │ │ GET │
│/shorturls│ │/:shortcode │ │/shorturls/ │
│ │ │ (Redirect) │ │:shortcode │
│ Create │ │ │ │ (Analytics) │
└─────────┘ └─────────────┘ └─────────────┘
│ │ │
└───────────────┼───────────────┘
┌──────────────────────────────────────────┐
│ Business Logic Layer │
│ ┌─────────────────────────────────────┐ │
│ │ URL Generation & Validation ││
│ │ • Shortcode generation ││
│ │ • Uniqueness checking ││
│ │ • URL format validation ││
│ │ • Expiry calculation ││
│ └─────────────────────────────────────┘ │
│ ┌─────────────────────────────────────┐ │
│ │ Analytics & Click Tracking ││
│ │ • Click event recording ││
│ │ • Geographic data (future) ││
│ │ • User agent parsing ││
│ │ • Referrer tracking ││
│ └─────────────────────────────────────┘ │
└──────────────────────────────────────────┘
│
┌──────────────────────────────────────────┐
│ Data Layer │
│ ┌─────────────────────────────────────┐ │
│ │ URL Storage ││
│ │ Map<shortcode, URLData> ││
│ │ { ││
│ │ originalUrl, ││
│ │ shortcode, ││
│ │ createdAt, ││
│ │ expiresAt, ││
│ │ clickCount ││
│ │ } ││
│ └─────────────────────────────────────┘ │
│ ┌─────────────────────────────────────┐ │
│ │ Analytics Storage ││
│ │ Map<shortcode, ClickData[]> ││
│ │ [{ ││
│ │ timestamp, ││
│ │ ip, ││
│ │ userAgent, ││
│ │ referrer ││
│ │ }] ││
│ └─────────────────────────────────────┘ │
└──────────────────────────────────────────┘
3. API Design
3.1 API Endpoints Specification
3.1.1 Create Short URL
Endpoint: POST /shorturls
Request Body:
"url": "string (required)",
"validity": "integer (optional, default: 30)",
"shortcode": "string (optional, 4-12 alphanumeric)"
Response (201 Created):
"shortLink": "http://hostname:port/abc123",
"expiry": "2025-01-07T14:30:00.000Z"
Error Responses:
400 Bad Request: Invalid URL format, invalid shortcode
409 Conflict: Shortcode already exists
500 Internal Server Error: Unable to generate unique shortcode
3.1.2 Redirect to Original URL
Endpoint: GET /:shortcode
Response:
302 Found: Redirects to original URL
404 Not Found: Shortcode doesn't exist
410 Gone: Link has expired
3.1.3 Get URL Statistics
Endpoint: GET /shorturls/:shortcode
Response (200 OK):
"shortcode": "abc123",
"originalUrl": "https://example.com",
"createdAt": "2025-01-07T13:30:00.000Z",
"expiry": "2025-01-07T14:30:00.000Z",
"totalClicks": 42,
"isExpired": false,
"clickDetails": [
"timestamp": "2025-01-07T13:45:00.000Z",
"ip": "192.168.1.1",
"userAgent": "Mozilla/5.0...",
"referrer": "https://google.com",
"location": "Unknown"
3.2 HTTP Status Codes
Code Usage
200 Successful statistics retrieval
201 Short URL created successfully
302 Redirect to original URL
400 Bad request (validation errors)
404 Short URL not found
409 Shortcode already exists
410 Short URL has expired
500 Internal server error
4. Data Models
4.1 URL Data Model
URLData {
id: string, // Unique identifier
originalUrl: string, // The long URL to redirect to
shortcode: string, // The short identifier (4-12 chars)
createdAt: string, // ISO 8601 timestamp
expiresAt: string, // ISO 8601 timestamp
clickCount: number // Total number of clicks
4.2 Click Analytics Model
ClickData {
timestamp: string, // ISO 8601 timestamp
ip: string, // Client IP address
userAgent: string, // Browser/client information
referrer: string, // Source of the click
location: string // Geographic location (future)
4.3 Storage Structure
// Primary storage
urlDatabase = Map<shortcode: string, URLData>
// Analytics storage
analyticsDatabase = Map<shortcode: string, ClickData[]>
5. System Components
5.1 Core Services
5.1.1 URL Generation Service
5. System Components
5.1 Core Services
5.1.1 URL Generation Service
5.1.2 Analytics Service
class AnalyticsService {
recordClick(shortcode, metadata) // Stores click event
getStatistics(shortcode) // Retrieves analytics data
aggregateMetrics(shortcode) // Computes summary stats
5.1.3 Cleanup Service
class CleanupService {
scheduleCleanup() // Runs every 5 minutes
removeExpiredUrls() // Deletes expired entries
logCleanupActivity() // Records cleanup events
5.2 Middleware Components
5.2.1 Logging Middleware
Logs all incoming requests
Captures request/response metadata
Provides audit trail
Mandatory requirement compliance
5.2.2 Validation Middleware
URL format validation
Shortcode format validation
Request body validation
Input sanitization
5.2.3 Error Handling Middleware
Centralized error processing
Proper HTTP status codes
Structured error responses
Error logging
6. Non-Functional Requirements
6.1 Performance Requirements
Metric Requirement Current Implementation
Response Time < 200ms for redirects In-memory lookup
Throughput 1000+ requests/second Single threaded
Availability 99.9% uptime No redundancy yet
6.2 Scalability Considerations
Current Limitations:
Single Instance: No horizontal scaling
Memory Storage: Limited by available RAM
No Persistence: Data lost on restart
Future Improvements:
┌─────────────────────────────────────────────────┐
│ Load Balancer │
└─────────────────────────────────────────────────┘
│ │ │
▼ ▼ ▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Instance 1 │ │ Instance 2 │ │ Instance N │
└─────────────┘ └─────────────┘ └─────────────┘
│ │ │
└────────────────┼────────────────┘
┌─────────────────────────────────┐
│ Redis/Database │
│ ┌─────────┐ ┌─────────────┐ │
│ │ URLs │ │ Analytics │ │
│ └─────────┘ └─────────────┘ │
└─────────────────────────────────┘
6.3 Data Storage Evolution Path
Phase 1 (Current): In-Memory
// Advantages: Fast, simple
// Disadvantages: Not persistent, limited scale
Map<string, URLData>
Map<string, ClickData[]>
Phase 2: Redis Cache + Database
// Cache Layer (Redis)
SET url:abc123 '{"originalUrl":"...","expiry":"..."}'
LPUSH clicks:abc123 '{"timestamp":"...","ip":"..."}'
// Persistent Layer (PostgreSQL/MongoDB)
CREATE TABLE urls (
shortcode VARCHAR(12) PRIMARY KEY,
original_url TEXT NOT NULL,
created_at TIMESTAMP,
expires_at TIMESTAMP,
click_count INTEGER DEFAULT 0
);
CREATE TABLE clicks (
id SERIAL PRIMARY KEY,
shortcode VARCHAR(12) REFERENCES urls(shortcode),
timestamp TIMESTAMP,
ip_address INET,
user_agent TEXT,
referrer TEXT
);
7. Security Considerations
7.1 Input Validation
URL Validation: Prevent malicious URLs
Shortcode Sanitization: Alphanumeric only, length limits
Rate Limiting: Prevent abuse (future implementation)
7.2 Security Headers
7. Security Considerations
7.1 Input Validation
URL Validation: Prevent malicious URLs
Shortcode Sanitization: Alphanumeric only, length limits
Rate Limiting: Prevent abuse (future implementation)
7.2 Security Headers
// Implemented security middleware
helmet() // Adds security headers:
// - X-Content-Type-Options: nosniff
// - X-Frame-Options: DENY
// - X-XSS-Protection: 1; mode=block
7.3 Potential Security Enhancements
// Rate limiting (future)
app.use('/shorturls', rateLimiter({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100 // limit each IP to 100 requests per windowMs
}));
// URL blacklist checking
const isBlacklisted = (url) => {
const blacklistedDomains = ['malicious.com', 'spam.org'];
return blacklistedDomains.some(domain => url.includes(domain));
};
8. Monitoring and Logging
8.1 Logging Strategy
// Current implementation
const loggingMiddleware = (req, res, next) => {
console.log(`[${timestamp}] ${method} ${url} - IP: ${ip}`);
// Logs: timestamp, method, URL, IP, user-agent, response status
};
8.2 Metrics to Track
Performance Metrics:
Request latency
Response times
