logo
Sign In
alexgrama7
@alexgrama7
·
a year ago
·
a year ago
function
Enhanced Context Tracker v4
Last Updated
a year ago
Created
a year ago
Function
filter
v4.0
Name
Enhanced Context Tracker v4
Downloads
2.6K+
Saves
19+
Description
A comprehensive context window tracker and metrics dashboard for OpenWebUI that provides real-time feedback on token usage, cost tracking, and performance metrics for all major LLM models.

The Enhanced Context Counter is a sophisticated Function Filter for OpenWebUI that provides real-time monitoring and analytics for LLM interactions. It tracks token usage, estimates costs, monitors performance metrics, and provides actionable insights through a configurable status display. The system supports a wide range of LLMs through multi-source model detection and offers extensive customization options via Valves and UserValves.

Key Features

  • Comprehensive Model Support: Multi-source model detection using OpenRouter API, exports, hardcoded defaults, and user-defined custom models in Valves
  • Advanced Token Counting: Primary tiktoken-based counting with intelligent fallbacks, content-specific adjustments, and calibration factors.
  • Cost Estimation & Budgeting: Precise cost calculation with input/output breakdown and multi-level budget tracking (daily, monthly, session).
  • Performance Analytics: Real-time token rate calculation, adaptive window sizing, and comprehensive session statistics.
  • Intelligent Context Management: Context window monitoring with progress visualization, warnings, and smart trimming suggestions.
  • Persistent Cost Tracking: File-based tracking (cross-chat) with thread-safe operations for user, daily, and monthly costs.
  • Highly Configurable UI: Customizable status line with modular components and visual indicators.

Other Features

  • Image Token Estimation: Heuristic-based calculation using defaults, resolution analysis, and model-specific overrides.
  • Calibration Integration: Status display based on external calibration results for accuracy verification.
  • Error Resilience: Graceful fallbacks for missing dependencies, API failures, and unrecognized models.
  • Content-Type Detection: Specialized handling for different content types (code, JSON, tables, etc.).
  • Cache Optimization: Token counting cache with adaptive pruning for performance enhancement.
  • Cost Optimization Hints: Actionable suggestions for reducing costs based on usage patterns.
  • Extensive Logging: Configurable logging with rotation for diagnostics and troubleshooting.

Valve Configuration Guide

The function offers extensive customization through Valves (global settings) and UserValves (per-user overrides):

Core Valves

  • [Model Detection]: Configure model recognition with fuzzy_match_threshold, vendor_family_map, and heuristic_rules.
  • [Token Counting]: Adjust accuracy with model_correction_factors and content_correction_factors.
  • [Cost/Budget]: Set budget_amount, monthly_budget_amount, and budget_tracking_mode for financial controls.
  • [UI/UX]: Customize display with toggles like show_progress_bar, show_cost, and progress_bar_style.
  • [Performance]: Fine-tune with adaptive_rate_averaging and related window settings.
  • [Cache]: Optimize with enable_token_cache and token_cache_size.
  • [Warnings]: Configure alerts with percentage thresholds for context and budget usage.

UserValves

Users can override global settings with personal preferences:

  • Custom budget amounts and warning thresholds
  • Model aliases for simplified model references
  • Personal correction factors for token counting accuracy
  • Visual style preferences for the status display

UI Status Line Breakdown

The status line provides a comprehensive overview of the current session's metrics in a compact format:

🪙 48/1.0M tokens (0.00%) [▱▱▱▱▱] | 🔽5/🔼43 | 💰 $0.000000 | 🏦 Daily: $0.009221/$100.00 (0.0%) | ⏱️ 5.1s (8.4 t/s) | 🗓️ $99.99 left (0.01%) this month | Text: 48 | 🔧 Not Calibrated

Status Components

  • 🪙 48/1.0M tokens (0.00%): Total tokens used / context window size with percentage
  • [▱▱▱▱▱]: Visual progress bar showing context window usage
  • 🔽5/🔼43: Input/Output token breakdown (5 input, 43 output)
  • 💰 $0.000000: Total estimated cost for the current session
  • 🏦 Daily: $0.009221/$100.00 (0.0%): Daily budget usage (spent/total and percentage)
  • ⏱️ 5.1s (8.4 t/s): Elapsed time and tokens per second rate
  • 🗓️ $99.99 left (0.01%) this month: Monthly budget status (remaining amount and percentage used)
  • Text: 48: Text token count (excludes image tokens if present)
  • 🔧 Not Calibrated: Calibration status of token counting accuracy

Display Modes

The status line adapts to different levels of detail based on configuration:

  1. Minimal: Shows only essential information (tokens, context percentage)

    🪙 48/1.0M tokens (0.00%)

  2. Standard: Includes core metrics (default mode)

    🪙 48/1.0M tokens (0.00%) [▱▱▱▱▱] | 🔽5/🔼43 | 💰 $0.000000 | ⏱️ 5.1s (8.4 t/s)

  3. Detailed: Displays all available metrics including budgets, token breakdowns, and calibration status

    🪙 48/1.0M tokens (0.00%) [▱▱▱▱▱] | 🔽5/🔼43 | 💰 $0.000000 | 🏦 Daily: $0.009221/$100.00 (0.0%) | ⏱️ 5.1s (8.4 t/s) | 🗓️ $99.99 left (0.01%) this month | Text: 48 | 🔧 Not Calibrated

The display automatically adjusts based on available space and configured preferences in the Valves settings.

Roadmap

  1. Enhanced model family detection with ML-based classification
  2. Advanced content-specific token counting with specialized encoders
  3. Interactive UI components for real-time adjustments and analytics
  4. Predictive budget forecasting based on usage patterns
  5. Cross-session analytics with visualization and reporting
  6. API for external integration with monitoring and alerting systems
-1