feat: implement token counter with real-time breakdown
Add comprehensive token counting functionality to provide visibility into context usage: Backend (Rust): - Add tiktoken-rs dependency for OpenAI-compatible token counting - Implement get_token_count command with detailed breakdown - Count tokens for: system prompt, preset instructions, persona, world info, author's note, message history, and current input - Per-section token breakdown for optimization insights Frontend (JavaScript/HTML/CSS): - Add token counter widget in status bar - Real-time updates as user types (debounced 300ms) - Expandable breakdown tooltip showing per-section counts - Automatic update when chat history loads or changes - Clean, minimal UI with hover interactions Features: - Accurate token counting using cl100k_base tokenizer - Debounced updates for performance - Detailed breakdown by context section - Visual indicator with total token count - Click to expand/collapse detailed breakdown - Auto-hide when no character is active This completes the "Must-Have for Basic Roleplay" features from the roadmap: ✅ World Info/Lorebooks ✅ Author's Note ✅ Token Counter - Message Examples Usage (next) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -761,6 +761,97 @@ body {
|
||||
color: #22c55e;
|
||||
}
|
||||
|
||||
/* Token Counter */
|
||||
.token-counter {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
font-size: 12px;
|
||||
color: var(--text-secondary);
|
||||
}
|
||||
|
||||
.token-count {
|
||||
padding: 4px 8px;
|
||||
background: var(--bg-tertiary);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 6px;
|
||||
}
|
||||
|
||||
.token-details-btn {
|
||||
width: 20px;
|
||||
height: 20px;
|
||||
border: none;
|
||||
background: transparent;
|
||||
color: var(--text-secondary);
|
||||
border-radius: 4px;
|
||||
cursor: pointer;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
transition: all 0.2s ease;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
.token-details-btn:hover {
|
||||
background: var(--bg-tertiary);
|
||||
color: var(--accent);
|
||||
}
|
||||
|
||||
.token-breakdown {
|
||||
position: absolute;
|
||||
bottom: 52px;
|
||||
right: 16px;
|
||||
background: var(--bg-secondary);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: 8px;
|
||||
padding: 12px;
|
||||
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.3);
|
||||
z-index: 100;
|
||||
min-width: 250px;
|
||||
}
|
||||
|
||||
.token-breakdown-header {
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
color: var(--text-primary);
|
||||
margin-bottom: 8px;
|
||||
padding-bottom: 8px;
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
|
||||
.token-breakdown-list {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 6px;
|
||||
}
|
||||
|
||||
.token-breakdown-item {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
font-size: 12px;
|
||||
color: var(--text-secondary);
|
||||
}
|
||||
|
||||
.token-breakdown-item span:first-child {
|
||||
color: var(--text-secondary);
|
||||
}
|
||||
|
||||
.token-breakdown-item span:last-child {
|
||||
color: var(--text-primary);
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.token-breakdown-total {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
font-size: 13px;
|
||||
font-weight: 600;
|
||||
color: var(--accent);
|
||||
margin-top: 8px;
|
||||
padding-top: 8px;
|
||||
border-top: 1px solid var(--border);
|
||||
}
|
||||
|
||||
@keyframes pulse {
|
||||
0%, 100% {
|
||||
opacity: 1;
|
||||
|
||||
Reference in New Issue
Block a user