🔍 Code Extractor

A logging utility class that captures and formats HTTP requests and responses in a human-readable format similar to reMarkable app logs, writing them to a file.

Purpose

This class provides structured logging for HTTP request/response cycles, particularly useful for debugging API interactions. It formats requests with method, URL, headers, and body content, handling both text and binary data intelligently. Large bodies are truncated with SHA256 hashes for verification. The logger creates a timestamped log file and appends each request with clear separators for easy reading.

Source Code

class RawRequestLogger:
    """Logs HTTP requests in the same format as reMarkable app logs"""
    
    def __init__(self, log_file: str = "raw_requests.log"):
        self.log_file = Path(log_file)
        self.log_file.parent.mkdir(exist_ok=True)
        
        # Clear log file
        with open(self.log_file, 'w') as f:
            f.write(f"=== RAW REQUEST LOG - {datetime.now().isoformat()} ===\n\n")
    
    def log_request(self, method: str, url: str, headers: dict, body: bytes = None, response_status: int = None, response_text: str = None):
        """Log a request in raw format"""
        with open(self.log_file, 'a') as f:
            # Log request line
            f.write(f"{method} {url}\n")
            
            # Log headers
            for key, value in headers.items():
                f.write(f"{key}: {value}\n")
            
            f.write("\n")  # Empty line before body
            
            # Log body if present
            if body:
                if len(body) > 1000:
                    # For large bodies, show first 500 and last 500 bytes
                    f.write(f"[BODY: {len(body)} bytes]\n")
                    f.write(f"First 500 bytes: {body[:500]}\n")
                    f.write(f"Last 500 bytes: {body[-500:]}\n")
                    f.write(f"SHA256: {hashlib.sha256(body).hexdigest()}\n")
                else:
                    # For small bodies, show full content
                    try:
                        decoded = body.decode('utf-8')
                        f.write(f"[BODY: {len(body)} bytes - UTF-8]\n")
                        f.write(decoded)
                        f.write("\n")
                    except:
                        f.write(f"[BODY: {len(body)} bytes - BINARY]\n")
                        f.write(f"Hex: {body.hex()}\n")
            else:
                f.write("[NO BODY]\n")
            
            # Log response if available
            if response_status is not None:
                f.write(f"\nRESPONSE: {response_status}\n")
                if response_text:
                    f.write(f"Response body: {response_text}\n")
            
            f.write("\n" + "="*80 + "\n\n")

Parameters

Parameter Details

log_file: Path to the log file where HTTP requests will be written. Defaults to 'raw_requests.log' in the current directory. The parent directory will be created if it doesn't exist. The file is cleared on instantiation with a timestamp header.

Return Value

Instantiation returns a RawRequestLogger object that maintains a file handle reference and provides the log_request method. The log_request method returns None as it performs side effects (file writing) only.

Class Interface

Methods

Attributes

Dependencies

• pathlib
• datetime
• hashlib

Required Imports

from pathlib import Path
from datetime import datetime
import hashlib

Usage Example

# Basic usage
logger = RawRequestLogger('api_requests.log')

# Log a simple GET request
logger.log_request(
    method='GET',
    url='https://api.example.com/users',
    headers={'Authorization': 'Bearer token123', 'Content-Type': 'application/json'}
)

# Log a POST request with body and response
request_body = json.dumps({'name': 'John', 'email': 'john@example.com'}).encode('utf-8')
logger.log_request(
    method='POST',
    url='https://api.example.com/users',
    headers={'Content-Type': 'application/json'},
    body=request_body,
    response_status=201,
    response_text='{"id": 123, "name": "John"}'
)

# Log a request with binary data
binary_data = b'\x89PNG\r\n\x1a\n' + b'\x00' * 2000
logger.log_request(
    method='PUT',
    url='https://api.example.com/upload',
    headers={'Content-Type': 'image/png'},
    body=binary_data,
    response_status=200
)

Best Practices

• Instantiate once per logging session to avoid repeatedly clearing the log file
• Be cautious when logging sensitive data (tokens, passwords) in headers or body - consider sanitizing before logging
• Monitor log file size when logging large request bodies, as files can grow quickly
• The log file is cleared on instantiation, so create a new instance with a different filename if you need to preserve previous logs
• For production use, consider implementing log rotation or size limits
• The class is not thread-safe; use separate instances or add locking if logging from multiple threads
• Binary data larger than 1000 bytes is automatically truncated with SHA256 hash for verification

Tags

Similar Components

AI-powered semantic similarity - components with related functionality:

• function logged_request 79.9% similar

  A method that intercepts and logs HTTP requests made to the reMarkable API, capturing request/response details including headers, body content, and response data for debugging purposes.

  From: /tf/active/vicechatdev/e-ink-llm/cloudtest/test_uploads.py

• function logged_request_v2 72.6% similar

  A wrapper function that logs HTTP requests and responses for debugging and analysis purposes, capturing request details, response status, and body content.

  From: /tf/active/vicechatdev/e-ink-llm/cloudtest/test_real_app_upload.py

• function logged_request_v1 67.7% similar

  A wrapper function that intercepts HTTP requests, logs their details (method, URL, headers, body) along with response information, and returns the response object.

  From: /tf/active/vicechatdev/e-ink-llm/cloudtest/raw_request_logger.py

• class InvoiceExtractionLogger 51.1% similar

  A comprehensive logging configuration class for invoice extraction systems that provides console and file logging with optional JSON formatting, request tracking via correlation IDs, and configurable log levels.

  From: /tf/active/vicechatdev/invoice_extraction/utils/logging_utils.py

• class RemarkableAuth 50.5% similar

  Handles the complete authentication flow for reMarkable cloud services, managing device tokens, user tokens, and authenticated HTTP sessions.

  From: /tf/active/vicechatdev/e-ink-llm/cloudtest/auth.py