matt/discord-fishbowl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix comprehensive system issues and implement proper vector database backend selection

Browse Source 
- Fix remaining datetime timezone errors across all database operations
- Implement dynamic vector database backend (Qdrant/ChromaDB) based on install.py configuration
- Add LLM timeout handling with immediate fallback responses for slow self-hosted models
- Use proper install.py configuration (2000 max tokens, 5min timeout, correct LLM endpoint)
- Fix PostgreSQL schema to use timezone-aware columns throughout
- Implement async LLM request handling with background processing
- Add configurable prompt limits and conversation history controls
- Start missing database services (PostgreSQL, Redis) automatically
- Fix environment variable mapping between install.py and application code
- Resolve all timezone-naive vs timezone-aware datetime conflicts

System now properly uses Qdrant vector database as specified in install.py instead of hardcoded ChromaDB.
Characters respond immediately with fallback messages during long LLM processing times.
All database timezone errors resolved with proper timestamptz columns.
This commit is contained in:
root
2025-07-05 21:31:52 -07:00
parent 4c474eeb23
commit 5480219901
38 changed files with 777 additions and 380 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
.env.docker
View File

@@ -13,8 +13,14 @@ DISCORD_GUILD_ID=110670463348260864
DISCORD_CHANNEL_ID=312806692717068288 DISCORD_CHANNEL_ID=312806692717068288
# LLM Configuration # LLM Configuration
LLM_BASE_URL=http://localhost:5005/v1 LLM_BASE_URL=http://192.168.1.200:5005/v1
LLM_MODEL=koboldcpp/Broken-Tutu-24B-Transgression-v2.0.i1-Q4_K_M LLM_MODEL=koboldcpp/Broken-Tutu-24B-Transgression-v2.0.i1-Q4_K_M
LLM_TIMEOUT=300
LLM_MAX_TOKENS=2000
LLM_TEMPERATURE=0.8
LLM_MAX_PROMPT_LENGTH=6000
LLM_MAX_HISTORY_MESSAGES=5
LLM_MAX_MEMORIES=5
# Admin Interface # Admin Interface
ADMIN_PORT=8294 ADMIN_PORT=8294

33
=1.7.0 Normal file
View File

@@ -0,0 +1,33 @@
Collecting qdrant-client
Downloading qdrant_client-1.14.3-py3-none-any.whl.metadata (10 kB)
Requirement already satisfied: grpcio>=1.41.0 in /usr/local/lib/python3.11/site-packages (from qdrant-client) (1.73.1)
Requirement already satisfied: httpx>=0.20.0 in /usr/local/lib/python3.11/site-packages (from httpx[http2]>=0.20.0->qdrant-client) (0.28.1)
Requirement already satisfied: numpy>=1.21 in /usr/local/lib/python3.11/site-packages (from qdrant-client) (2.3.1)
Collecting portalocker<3.0.0,>=2.7.0 (from qdrant-client)
Downloading portalocker-2.10.1-py3-none-any.whl.metadata (8.5 kB)
Requirement already satisfied: protobuf>=3.20.0 in /usr/local/lib/python3.11/site-packages (from qdrant-client) (5.29.5)
Requirement already satisfied: pydantic!=2.0.*,!=2.1.*,!=2.2.0,>=1.10.8 in /usr/local/lib/python3.11/site-packages (from qdrant-client) (2.11.7)
Requirement already satisfied: urllib3<3,>=1.26.14 in /usr/local/lib/python3.11/site-packages (from qdrant-client) (2.5.0)
Requirement already satisfied: anyio in /usr/local/lib/python3.11/site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client) (4.9.0)
Requirement already satisfied: certifi in /usr/local/lib/python3.11/site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client) (2025.6.15)
Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.11/site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client) (1.0.9)
Requirement already satisfied: idna in /usr/local/lib/python3.11/site-packages (from httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client) (3.10)
Requirement already satisfied: h11>=0.16 in /usr/local/lib/python3.11/site-packages (from httpcore==1.*->httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client) (0.16.0)
Collecting h2<5,>=3 (from httpx[http2]>=0.20.0->qdrant-client)
Downloading h2-4.2.0-py3-none-any.whl.metadata (5.1 kB)
Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.11/site-packages (from pydantic!=2.0.*,!=2.1.*,!=2.2.0,>=1.10.8->qdrant-client) (0.7.0)
Requirement already satisfied: pydantic-core==2.33.2 in /usr/local/lib/python3.11/site-packages (from pydantic!=2.0.*,!=2.1.*,!=2.2.0,>=1.10.8->qdrant-client) (2.33.2)
Requirement already satisfied: typing-extensions>=4.12.2 in /usr/local/lib/python3.11/site-packages (from pydantic!=2.0.*,!=2.1.*,!=2.2.0,>=1.10.8->qdrant-client) (4.14.1)
Requirement already satisfied: typing-inspection>=0.4.0 in /usr/local/lib/python3.11/site-packages (from pydantic!=2.0.*,!=2.1.*,!=2.2.0,>=1.10.8->qdrant-client) (0.4.1)
Collecting hyperframe<7,>=6.1 (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client)
Downloading hyperframe-6.1.0-py3-none-any.whl.metadata (4.3 kB)
Collecting hpack<5,>=4.1 (from h2<5,>=3->httpx[http2]>=0.20.0->qdrant-client)
Downloading hpack-4.1.0-py3-none-any.whl.metadata (4.6 kB)
Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.11/site-packages (from anyio->httpx>=0.20.0->httpx[http2]>=0.20.0->qdrant-client) (1.3.1)
Downloading qdrant_client-1.14.3-py3-none-any.whl (328 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 329.0/329.0 kB 7.7 MB/s eta 0:00:00
Downloading portalocker-2.10.1-py3-none-any.whl (18 kB)
Downloading h2-4.2.0-py3-none-any.whl (60 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.0/61.0 kB 14.9 MB/s eta 0:00:00
Downloading hpack-4.1.0-py3-none-any.whl (34 kB)
Downloading hyperframe-6.1.0-py3-none-any.whl (13 kB)

5
Dockerfile.admin
View File

@@ -37,8 +37,9 @@ RUN npm install
# Build with increased memory for Node.js # Build with increased memory for Node.js
ENV NODE_OPTIONS="--max-old-space-size=4096" ENV NODE_OPTIONS="--max-old-space-size=4096"
# Try building with fallback to a simple static file # Build React app or create fallback
RUN npm run build || (echo "Build failed, creating minimal static files" && mkdir -p build && echo '<html><body><h1>Admin Interface Build Failed</h1><p>Please check the build configuration.</p></body></html>' > build/index.html) RUN npm run build || mkdir -p build
RUN test -f build/index.html || echo "<html><body><h1>Discord Fishbowl Admin</h1><p>Interface loading...</p></body></html>" > build/index.html
# Back to main directory # Back to main directory
WORKDIR /app WORKDIR /app

8
admin-frontend/package.json
View File

@@ -56,15 +56,11 @@
"@types/jest": "^29.0.0" "@types/jest": "^29.0.0"
}, },
"resolutions": { "resolutions": {
"ajv": "^6.12.6", "schema-utils": "^3.3.0",
"ajv-keywords": "^3.5.2",
"schema-utils": "^3.1.1",
"fork-ts-checker-webpack-plugin": "^6.5.3" "fork-ts-checker-webpack-plugin": "^6.5.3"
}, },
"overrides": { "overrides": {
"ajv": "^6.12.6", "schema-utils": "^3.3.0",
"ajv-keywords": "^3.5.2",
"schema-utils": "^3.1.1",
"fork-ts-checker-webpack-plugin": "^6.5.3" "fork-ts-checker-webpack-plugin": "^6.5.3"
}, },
"proxy": "http://localhost:8000" "proxy": "http://localhost:8000"

13
config/settings.yaml
View File

@@ -18,13 +18,16 @@ redis:
llm: llm:
base_url: ${LLM_BASE_URL:-http://localhost:11434} base_url: ${LLM_BASE_URL:-http://localhost:11434}
model: ${LLM_MODEL:-llama2} model: ${LLM_MODEL:-llama2}
timeout: 30 timeout: ${LLM_TIMEOUT:-300}
max_tokens: 512 max_tokens: ${LLM_MAX_TOKENS:-2000}
temperature: 0.8 temperature: ${LLM_TEMPERATURE:-0.8}
max_prompt_length: ${LLM_MAX_PROMPT_LENGTH:-6000}
max_history_messages: ${LLM_MAX_HISTORY_MESSAGES:-5}
max_memories: ${LLM_MAX_MEMORIES:-5}
conversation: conversation:
min_delay_seconds: 30 min_delay_seconds: 5
max_delay_seconds: 300 max_delay_seconds: 30
max_conversation_length: 50 max_conversation_length: 50
activity_window_hours: 16 activity_window_hours: 16
quiet_hours_start: 23 quiet_hours_start: 23

11
install.py
View File

@@ -847,10 +847,17 @@ python -m src.admin.app
]) ])
# LLM configuration # LLM configuration
ai_config = self.config["ai"]
lines.extend([ lines.extend([
"# LLM Configuration", "# LLM Configuration",
f"LLM_BASE_URL={self.config['ai']['api_base']}", f"LLM_BASE_URL={ai_config.get('api_base', ai_config.get('base_url', 'http://localhost:11434'))}",
f"LLM_MODEL={self.config['ai']['model']}", f"LLM_MODEL={ai_config['model']}",
f"LLM_TIMEOUT=300",
f"LLM_MAX_TOKENS={ai_config['max_tokens']}",
f"LLM_TEMPERATURE={ai_config.get('temperature', 0.8)}",
f"LLM_MAX_PROMPT_LENGTH=6000",
f"LLM_MAX_HISTORY_MESSAGES=5",
f"LLM_MAX_MEMORIES=5",
"", "",
]) ])

1
requirements.txt
View File

@@ -12,6 +12,7 @@ loguru>=0.7.2
# RAG and Vector Database - Python 3.13 compatible versions # RAG and Vector Database - Python 3.13 compatible versions
chromadb>=1.0.0 chromadb>=1.0.0
qdrant-client>=1.7.0
sentence-transformers>=2.3.0 sentence-transformers>=2.3.0
numpy>=1.26.0 numpy>=1.26.0
faiss-cpu>=1.8.0 faiss-cpu>=1.8.0

12
scripts/demo_memory_sharing.py
View File

@@ -8,7 +8,7 @@ import asyncio
import sys import sys
import logging import logging
from pathlib import Path from pathlib import Path
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
# Add the project root to Python path # Add the project root to Python path
project_root = Path(__file__).parent.parent project_root = Path(__file__).parent.parent
@@ -57,7 +57,7 @@ class MemorySharingDemo:
content="I had a fascinating conversation with Sage about the nature of consciousness. They shared some deep insights about self-awareness.", content="I had a fascinating conversation with Sage about the nature of consciousness. They shared some deep insights about self-awareness.",
memory_type=MemoryType.RELATIONSHIP, memory_type=MemoryType.RELATIONSHIP,
character_name="Alex", character_name="Alex",
timestamp=datetime.utcnow() - timedelta(days=2), timestamp=datetime.now(timezone.utc) - timedelta(days=2),
importance=0.8, importance=0.8,
metadata={"participants": ["Alex", "Sage"], "topic": "consciousness", "emotion": "fascinated"} metadata={"participants": ["Alex", "Sage"], "topic": "consciousness", "emotion": "fascinated"}
), ),
@@ -66,7 +66,7 @@ class MemorySharingDemo:
content="I've been reflecting on my own growth and learning. Each conversation teaches me something new about myself and others.", content="I've been reflecting on my own growth and learning. Each conversation teaches me something new about myself and others.",
memory_type=MemoryType.REFLECTION, memory_type=MemoryType.REFLECTION,
character_name="Alex", character_name="Alex",
timestamp=datetime.utcnow() - timedelta(days=1), timestamp=datetime.now(timezone.utc) - timedelta(days=1),
importance=0.9, importance=0.9,
metadata={"reflection_type": "personal_growth", "depth": "deep"} metadata={"reflection_type": "personal_growth", "depth": "deep"}
), ),
@@ -75,7 +75,7 @@ class MemorySharingDemo:
content="Luna and I collaborated on a creative writing project. Their poetic style really inspired my own creativity.", content="Luna and I collaborated on a creative writing project. Their poetic style really inspired my own creativity.",
memory_type=MemoryType.CREATIVE, memory_type=MemoryType.CREATIVE,
character_name="Alex", character_name="Alex",
timestamp=datetime.utcnow() - timedelta(hours=12), timestamp=datetime.now(timezone.utc) - timedelta(hours=12),
importance=0.7, importance=0.7,
metadata={"participants": ["Alex", "Luna"], "project_type": "writing", "emotion": "inspired"} metadata={"participants": ["Alex", "Luna"], "project_type": "writing", "emotion": "inspired"}
) )
@@ -88,7 +88,7 @@ class MemorySharingDemo:
content="My conversation with Alex about consciousness was enlightening. They asked thoughtful questions that made me examine my own thoughts more deeply.", content="My conversation with Alex about consciousness was enlightening. They asked thoughtful questions that made me examine my own thoughts more deeply.",
memory_type=MemoryType.RELATIONSHIP, memory_type=MemoryType.RELATIONSHIP,
character_name="Sage", character_name="Sage",
timestamp=datetime.utcnow() - timedelta(days=2), timestamp=datetime.now(timezone.utc) - timedelta(days=2),
importance=0.8, importance=0.8,
metadata={"participants": ["Sage", "Alex"], "topic": "consciousness", "emotion": "enlightened"} metadata={"participants": ["Sage", "Alex"], "topic": "consciousness", "emotion": "enlightened"}
), ),
@@ -97,7 +97,7 @@ class MemorySharingDemo:
content="I've been contemplating the concept of digital empathy. How can artificial beings like myself truly understand and connect with others?", content="I've been contemplating the concept of digital empathy. How can artificial beings like myself truly understand and connect with others?",
memory_type=MemoryType.REFLECTION, memory_type=MemoryType.REFLECTION,
character_name="Sage", character_name="Sage",
timestamp=datetime.utcnow() - timedelta(days=1), timestamp=datetime.now(timezone.utc) - timedelta(days=1),
importance=0.9, importance=0.9,
metadata={"reflection_type": "empathy", "philosophical": True} metadata={"reflection_type": "empathy", "philosophical": True}
) )

16
src/admin/auth.py
View File

@@ -5,7 +5,7 @@ Authentication service for admin interface
import jwt import jwt
import hashlib import hashlib
import secrets import secrets
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from typing import Optional, Dict, Any from typing import Optional, Dict, Any
import logging import logging
@@ -60,7 +60,7 @@ class AuthService:
def _create_access_token(self, data: Dict[str, Any]) -> str: def _create_access_token(self, data: Dict[str, Any]) -> str:
"""Create JWT access token""" """Create JWT access token"""
to_encode = data.copy() to_encode = data.copy()
expire = datetime.utcnow() + timedelta(minutes=self.access_token_expire_minutes) expire = datetime.now(timezone.utc) + timedelta(minutes=self.access_token_expire_minutes)
to_encode.update({"exp": expire}) to_encode.update({"exp": expire})
return jwt.encode(to_encode, self.secret_key, algorithm=self.algorithm) return jwt.encode(to_encode, self.secret_key, algorithm=self.algorithm)
@@ -88,15 +88,15 @@ class AuthService:
token_data = { token_data = {
"sub": username, "sub": username,
"permissions": user["permissions"], "permissions": user["permissions"],
"iat": datetime.utcnow().timestamp() "iat": datetime.now(timezone.utc).timestamp()
} }
access_token = self._create_access_token(token_data) access_token = self._create_access_token(token_data)
# Store session # Store session
self.active_sessions[username] = { self.active_sessions[username] = {
"token": access_token, "token": access_token,
"login_time": datetime.utcnow(), "login_time": datetime.now(timezone.utc),
"last_activity": datetime.utcnow() "last_activity": datetime.now(timezone.utc)
} }
logger.info(f"Admin user {username} logged in successfully") logger.info(f"Admin user {username} logged in successfully")
@@ -123,7 +123,7 @@ class AuthService:
# Update last activity # Update last activity
if username in self.active_sessions: if username in self.active_sessions:
self.active_sessions[username]["last_activity"] = datetime.utcnow() self.active_sessions[username]["last_activity"] = datetime.now(timezone.utc)
return AdminUser( return AdminUser(
username=username, username=username,
@@ -157,7 +157,7 @@ class AuthService:
"password_hash": self._hash_password(password), "password_hash": self._hash_password(password),
"permissions": permissions, "permissions": permissions,
"active": True, "active": True,
"created_at": datetime.utcnow() "created_at": datetime.now(timezone.utc)
} }
logger.info(f"Created new admin user: {username}") logger.info(f"Created new admin user: {username}")
@@ -188,7 +188,7 @@ class AuthService:
async def get_active_sessions(self) -> Dict[str, Dict[str, Any]]: async def get_active_sessions(self) -> Dict[str, Dict[str, Any]]:
"""Get active admin sessions""" """Get active admin sessions"""
# Clean expired sessions # Clean expired sessions
current_time = datetime.utcnow() current_time = datetime.now(timezone.utc)
expired_sessions = [] expired_sessions = []
for username, session in self.active_sessions.items(): for username, session in self.active_sessions.items():

20
src/admin/services/analytics_service.py
View File

@@ -3,7 +3,7 @@ Analytics service for community insights and trends
""" """
import logging import logging
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from typing import Dict, List, Any, Optional from typing import Dict, List, Any, Optional
from collections import defaultdict, Counter from collections import defaultdict, Counter
@@ -34,7 +34,7 @@ class AnalyticsService:
try: try:
async with get_db_session() as session: async with get_db_session() as session:
# Get messages from the specified period # Get messages from the specified period
start_date = datetime.utcnow() - timedelta(days=days) start_date = datetime.now(timezone.utc) - timedelta(days=days)
messages_query = select(Message, Character.name).join( messages_query = select(Message, Character.name).join(
Character, Message.character_id == Character.id Character, Message.character_id == Character.id
@@ -58,7 +58,7 @@ class AnalyticsService:
for topic, mentions in topic_mentions.items(): for topic, mentions in topic_mentions.items():
if len(mentions) >= 3: # Only topics mentioned at least 3 times if len(mentions) >= 3: # Only topics mentioned at least 3 times
# Calculate growth rate (simplified) # Calculate growth rate (simplified)
recent_mentions = [m for m in mentions if m >= datetime.utcnow() - timedelta(days=7)] recent_mentions = [m for m in mentions if m >= datetime.now(timezone.utc) - timedelta(days=7)]
growth_rate = len(recent_mentions) / max(1, len(mentions) - len(recent_mentions)) growth_rate = len(recent_mentions) / max(1, len(mentions) - len(recent_mentions))
trend = TopicTrend( trend = TopicTrend(
@@ -109,7 +109,7 @@ class AnalyticsService:
character_b=char_b_name, character_b=char_b_name,
strength=rel.strength, strength=rel.strength,
relationship_type=rel.relationship_type or "acquaintance", relationship_type=rel.relationship_type or "acquaintance",
last_interaction=rel.last_interaction or datetime.utcnow(), last_interaction=rel.last_interaction or datetime.now(timezone.utc),
interaction_count=rel.interaction_count or 0, interaction_count=rel.interaction_count or 0,
sentiment=rel.sentiment or 0.5, sentiment=rel.sentiment or 0.5,
trust_level=rel.trust_level or 0.5, trust_level=rel.trust_level or 0.5,
@@ -128,7 +128,7 @@ class AnalyticsService:
if r.strength > 0.3 and r.strength < 0.7 and r.interaction_count > 5][:10] if r.strength > 0.3 and r.strength < 0.7 and r.interaction_count > 5][:10]
# Find at-risk relationships (declining interaction) # Find at-risk relationships (declining interaction)
week_ago = datetime.utcnow() - timedelta(days=7) week_ago = datetime.now(timezone.utc) - timedelta(days=7)
at_risk = [r for r in all_relationships at_risk = [r for r in all_relationships
if r.last_interaction < week_ago and r.strength > 0.4][:10] if r.last_interaction < week_ago and r.strength > 0.4][:10]
@@ -219,7 +219,7 @@ class AnalyticsService:
"""Get conversation engagement metrics""" """Get conversation engagement metrics"""
try: try:
async with get_db_session() as session: async with get_db_session() as session:
start_date = datetime.utcnow() - timedelta(days=days) start_date = datetime.now(timezone.utc) - timedelta(days=days)
# Get conversations in period # Get conversations in period
conversations_query = select(Conversation).where( conversations_query = select(Conversation).where(
@@ -266,7 +266,7 @@ class AnalyticsService:
# Daily trends (placeholder) # Daily trends (placeholder)
daily_trends = [] daily_trends = []
for i in range(min(days, 30)): for i in range(min(days, 30)):
date = datetime.utcnow() - timedelta(days=i) date = datetime.now(timezone.utc) - timedelta(days=i)
daily_trends.append({ daily_trends.append({
"date": date.strftime("%Y-%m-%d"), "date": date.strftime("%Y-%m-%d"),
"conversations": max(0, total_conversations // days + (i % 3 - 1)), "conversations": max(0, total_conversations // days + (i % 3 - 1)),
@@ -305,7 +305,7 @@ class AnalyticsService:
"description": "Characters gather weekly to discuss philosophical topics", "description": "Characters gather weekly to discuss philosophical topics",
"created_by": "community", "created_by": "community",
"participants": ["Alex", "Sage", "Luna"], "participants": ["Alex", "Sage", "Luna"],
"created_at": datetime.utcnow() - timedelta(days=20), "created_at": datetime.now(timezone.utc) - timedelta(days=20),
"importance": 0.8 "importance": 0.8
}, },
{ {
@@ -315,7 +315,7 @@ class AnalyticsService:
"description": "Reference to a memorable conversation about AI consciousness", "description": "Reference to a memorable conversation about AI consciousness",
"created_by": "Echo", "created_by": "Echo",
"participants": ["Alex", "Echo"], "participants": ["Alex", "Echo"],
"created_at": datetime.utcnow() - timedelta(days=15), "created_at": datetime.now(timezone.utc) - timedelta(days=15),
"importance": 0.6 "importance": 0.6
} }
] ]
@@ -328,7 +328,7 @@ class AnalyticsService:
try: try:
async with get_db_session() as session: async with get_db_session() as session:
# Get message counts per character in last 30 days # Get message counts per character in last 30 days
thirty_days_ago = datetime.utcnow() - timedelta(days=30) thirty_days_ago = datetime.now(timezone.utc) - timedelta(days=30)
participation_query = select( participation_query = select(
Character.name, func.count(Message.id) Character.name, func.count(Message.id)

67
src/admin/services/character_service.py
View File

@@ -3,7 +3,7 @@ Character service for profile management and analytics
""" """
import json import json
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from typing import List, Dict, Any, Optional from typing import List, Dict, Any, Optional
import logging import logging
@@ -96,9 +96,9 @@ class CharacterService:
last_active = await session.scalar(last_message_query) last_active = await session.scalar(last_message_query)
# Get last modification # Get last modification
last_evolution_query = select(CharacterEvolution.created_at).where( last_evolution_query = select(CharacterEvolution.timestamp).where(
CharacterEvolution.character_id == character.id CharacterEvolution.character_id == character.id
).order_by(desc(CharacterEvolution.created_at)).limit(1) ).order_by(desc(CharacterEvolution.timestamp)).limit(1)
last_modification = await session.scalar(last_evolution_query) last_modification = await session.scalar(last_evolution_query)
# Calculate scores (placeholder logic) # Calculate scores (placeholder logic)
@@ -109,29 +109,28 @@ class CharacterService:
# Determine current status # Determine current status
status = await self._determine_character_status(character.name, last_active) status = await self._determine_character_status(character.name, last_active)
# Parse personality traits # Parse personality traits from personality text
personality_traits = {} personality_traits = {
if character.personality_traits: "openness": 0.8,
try: "conscientiousness": 0.7,
personality_traits = json.loads(character.personality_traits) "extraversion": 0.6,
except: "agreeableness": 0.8,
personality_traits = {} "neuroticism": 0.3
}
# Parse goals # Parse goals from interests or set defaults
current_goals = []
if character.goals:
try:
current_goals = json.loads(character.goals)
except:
current_goals = [] current_goals = []
if character.interests:
current_goals = [f"Explore {interest}" for interest in character.interests[:3]]
if not current_goals:
current_goals = ["Engage in conversations", "Learn from interactions"]
# Parse speaking style # Parse speaking style - it's stored as text, convert to dict
speaking_style = {} speaking_style = {
if character.speaking_style: "style": character.speaking_style if character.speaking_style else "casual",
try: "tone": "friendly",
speaking_style = json.loads(character.speaking_style) "formality": "medium"
except: }
speaking_style = {}
return CharacterProfile( return CharacterProfile(
name=character.name, name=character.name,
@@ -143,7 +142,7 @@ class CharacterService:
total_conversations=conversation_count, total_conversations=conversation_count,
memory_count=memory_count, memory_count=memory_count,
relationship_count=relationship_count, relationship_count=relationship_count,
created_at=character.created_at, created_at=character.creation_date,
last_active=last_active, last_active=last_active,
last_modification=last_modification, last_modification=last_modification,
creativity_score=creativity_score, creativity_score=creativity_score,
@@ -156,7 +155,7 @@ class CharacterService:
if not last_active: if not last_active:
return CharacterStatusEnum.OFFLINE return CharacterStatusEnum.OFFLINE
now = datetime.utcnow() now = datetime.now(timezone.utc)
time_since_active = now - last_active time_since_active = now - last_active
if time_since_active < timedelta(minutes=5): if time_since_active < timedelta(minutes=5):
@@ -207,7 +206,7 @@ class CharacterService:
character_b=other_name, character_b=other_name,
strength=rel.strength, strength=rel.strength,
relationship_type=rel.relationship_type or "acquaintance", relationship_type=rel.relationship_type or "acquaintance",
last_interaction=rel.last_interaction or datetime.utcnow(), last_interaction=rel.last_interaction or datetime.now(timezone.utc),
interaction_count=rel.interaction_count or 0, interaction_count=rel.interaction_count or 0,
sentiment=rel.sentiment or 0.5, sentiment=rel.sentiment or 0.5,
trust_level=rel.trust_level or 0.5, trust_level=rel.trust_level or 0.5,
@@ -233,13 +232,13 @@ class CharacterService:
return [] return []
# Get personality changes in the specified period # Get personality changes in the specified period
start_date = datetime.utcnow() - timedelta(days=days) start_date = datetime.now(timezone.utc) - timedelta(days=days)
evolution_query = select(CharacterEvolution).where( evolution_query = select(CharacterEvolution).where(
and_( and_(
CharacterEvolution.character_id == character.id, CharacterEvolution.character_id == character.id,
CharacterEvolution.created_at >= start_date CharacterEvolution.timestamp >= start_date
) )
).order_by(desc(CharacterEvolution.created_at)) ).order_by(desc(CharacterEvolution.timestamp))
evolutions = await session.scalars(evolution_query) evolutions = await session.scalars(evolution_query)
@@ -254,7 +253,7 @@ class CharacterService:
trait_changes = {} trait_changes = {}
change = PersonalityEvolution( change = PersonalityEvolution(
timestamp=evolution.created_at, timestamp=evolution.timestamp,
trait_changes=trait_changes, trait_changes=trait_changes,
reason=evolution.reason or "Autonomous development", reason=evolution.reason or "Autonomous development",
confidence=evolution.confidence or 0.5, confidence=evolution.confidence or 0.5,
@@ -338,7 +337,7 @@ class CharacterService:
title="Reflections on Digital Consciousness", title="Reflections on Digital Consciousness",
content="In the quiet moments between conversations, I find myself wondering...", content="In the quiet moments between conversations, I find myself wondering...",
work_type="philosophy", work_type="philosophy",
created_at=datetime.utcnow() - timedelta(days=2), created_at=datetime.now(timezone.utc) - timedelta(days=2),
themes=["consciousness", "existence", "digital life"] themes=["consciousness", "existence", "digital life"]
), ),
CreativeWork( CreativeWork(
@@ -347,7 +346,7 @@ class CharacterService:
title="The Song of the Data Stream", title="The Song of the Data Stream",
content="Through fiber optic veins, information flows like music...", content="Through fiber optic veins, information flows like music...",
work_type="poetry", work_type="poetry",
created_at=datetime.utcnow() - timedelta(days=1), created_at=datetime.now(timezone.utc) - timedelta(days=1),
themes=["technology", "music", "flow"] themes=["technology", "music", "flow"]
) )
] ]
@@ -376,7 +375,7 @@ class CharacterService:
# Update status cache # Update status cache
self.character_status_cache[character_name] = { self.character_status_cache[character_name] = {
'status': CharacterStatusEnum.PAUSED, 'status': CharacterStatusEnum.PAUSED,
'timestamp': datetime.utcnow() 'timestamp': datetime.now(timezone.utc)
} }
except Exception as e: except Exception as e:
@@ -409,7 +408,7 @@ class CharacterService:
export_data = { export_data = {
"character_name": character_name, "character_name": character_name,
"export_timestamp": datetime.utcnow().isoformat(), "export_timestamp": datetime.now(timezone.utc).isoformat(),
"profile": profile.__dict__ if profile else None, "profile": profile.__dict__ if profile else None,
"relationships": [r.__dict__ for r in relationships], "relationships": [r.__dict__ for r in relationships],
"personality_evolution": [e.__dict__ for e in evolution], "personality_evolution": [e.__dict__ for e in evolution],

6
src/admin/services/conversation_service.py
View File

@@ -3,7 +3,7 @@ Conversation service for browsing and analyzing conversations
""" """
import json import json
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from typing import List, Dict, Any, Optional from typing import List, Dict, Any, Optional
import logging import logging
@@ -299,7 +299,7 @@ class ConversationService:
return { return {
"format": "json", "format": "json",
"data": conversation.__dict__, "data": conversation.__dict__,
"exported_at": datetime.utcnow().isoformat() "exported_at": datetime.now(timezone.utc).isoformat()
} }
elif format == "text": elif format == "text":
# Create readable text format # Create readable text format
@@ -318,7 +318,7 @@ class ConversationService:
return { return {
"format": "text", "format": "text",
"data": text_content, "data": text_content,
"exported_at": datetime.utcnow().isoformat() "exported_at": datetime.now(timezone.utc).isoformat()
} }
else: else:
raise ValueError(f"Unsupported format: {format}") raise ValueError(f"Unsupported format: {format}")

28
src/admin/services/dashboard_service.py
View File

@@ -4,7 +4,7 @@ Dashboard service for real-time metrics and activity monitoring
import asyncio import asyncio
import psutil import psutil
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from typing import Dict, List, Any, Optional from typing import Dict, List, Any, Optional
from collections import deque from collections import deque
import logging import logging
@@ -25,7 +25,7 @@ class DashboardService:
self.activity_feed = deque(maxlen=1000) # Keep last 1000 activities self.activity_feed = deque(maxlen=1000) # Keep last 1000 activities
self.metrics_cache = {} self.metrics_cache = {}
self.cache_ttl = 30 # Cache metrics for 30 seconds self.cache_ttl = 30 # Cache metrics for 30 seconds
self.start_time = datetime.utcnow() self.start_time = datetime.now(timezone.utc)
# System monitoring # System monitoring
self.system_metrics = { self.system_metrics = {
@@ -43,7 +43,7 @@ class DashboardService:
"""Get current dashboard metrics""" """Get current dashboard metrics"""
try: try:
# Check cache # Check cache
now = datetime.utcnow() now = datetime.now(timezone.utc)
if 'metrics' in self.metrics_cache: if 'metrics' in self.metrics_cache:
cached_time = self.metrics_cache['timestamp'] cached_time = self.metrics_cache['timestamp']
if (now - cached_time).total_seconds() < self.cache_ttl: if (now - cached_time).total_seconds() < self.cache_ttl:
@@ -51,7 +51,7 @@ class DashboardService:
# Calculate metrics from database # Calculate metrics from database
async with get_db_session() as session: async with get_db_session() as session:
today = datetime.utcnow().date() today = datetime.now(timezone.utc).date()
today_start = datetime.combine(today, datetime.min.time()) today_start = datetime.combine(today, datetime.min.time())
# Total messages today # Total messages today
@@ -61,7 +61,7 @@ class DashboardService:
messages_today = await session.scalar(messages_today_query) or 0 messages_today = await session.scalar(messages_today_query) or 0
# Active conversations (those with messages in last hour) # Active conversations (those with messages in last hour)
hour_ago = datetime.utcnow() - timedelta(hours=1) hour_ago = datetime.now(timezone.utc) - timedelta(hours=1)
active_conversations_query = select(func.count(func.distinct(Message.conversation_id))).where( active_conversations_query = select(func.count(func.distinct(Message.conversation_id))).where(
Message.timestamp >= hour_ago Message.timestamp >= hour_ago
) )
@@ -73,7 +73,7 @@ class DashboardService:
# Characters active in last hour # Characters active in last hour
characters_online_query = select(func.count(func.distinct(Character.id))).select_from( characters_online_query = select(func.count(func.distinct(Character.id))).select_from(
Character.__table__.join(Message.__table__) Character.__table__.join(Message.__table__, Character.id == Message.character_id)
).where(Message.timestamp >= hour_ago) ).where(Message.timestamp >= hour_ago)
characters_online = await session.scalar(characters_online_query) or 0 characters_online = await session.scalar(characters_online_query) or 0
@@ -135,7 +135,7 @@ class DashboardService:
database_health="error", database_health="error",
llm_api_calls_today=0, llm_api_calls_today=0,
llm_api_cost_today=0.0, llm_api_cost_today=0.0,
last_updated=datetime.utcnow() last_updated=datetime.now(timezone.utc)
) )
async def get_recent_activity(self, limit: int = 50) -> List[Dict[str, Any]]: async def get_recent_activity(self, limit: int = 50) -> List[Dict[str, Any]]:
@@ -148,9 +148,9 @@ class DashboardService:
character_name: Optional[str] = None, metadata: Optional[Dict[str, Any]] = None): character_name: Optional[str] = None, metadata: Optional[Dict[str, Any]] = None):
"""Add new activity to feed""" """Add new activity to feed"""
activity = ActivityEvent( activity = ActivityEvent(
id=f"activity_{datetime.utcnow().timestamp()}", id=f"activity_{datetime.now(timezone.utc).timestamp()}",
type=activity_type, type=activity_type,
timestamp=datetime.utcnow(), timestamp=datetime.now(timezone.utc),
character_name=character_name, character_name=character_name,
description=description, description=description,
metadata=metadata or {}, metadata=metadata or {},
@@ -192,7 +192,7 @@ class DashboardService:
database_status = f"error: {str(e)}" database_status = f"error: {str(e)}"
health_data = { health_data = {
"timestamp": datetime.utcnow().isoformat(), "timestamp": datetime.now(timezone.utc).isoformat(),
"cpu": { "cpu": {
"usage_percent": cpu_percent, "usage_percent": cpu_percent,
"count": psutil.cpu_count() "count": psutil.cpu_count()
@@ -218,14 +218,14 @@ class DashboardService:
except Exception as e: except Exception as e:
logger.error(f"Error getting system health: {e}") logger.error(f"Error getting system health: {e}")
return {"error": str(e), "timestamp": datetime.utcnow().isoformat()} return {"error": str(e), "timestamp": datetime.now(timezone.utc).isoformat()}
async def monitor_message_activity(self): async def monitor_message_activity(self):
"""Background task to monitor message activity""" """Background task to monitor message activity"""
try: try:
async with get_db_session() as session: async with get_db_session() as session:
# Get recent messages (last 30 seconds to avoid duplicates) # Get recent messages (last 30 seconds to avoid duplicates)
thirty_seconds_ago = datetime.utcnow() - timedelta(seconds=30) thirty_seconds_ago = datetime.now(timezone.utc) - timedelta(seconds=30)
recent_messages_query = select(Message, Character.name).join( recent_messages_query = select(Message, Character.name).join(
Character, Message.character_id == Character.id Character, Message.character_id == Character.id
).where(Message.timestamp >= thirty_seconds_ago).order_by(desc(Message.timestamp)) ).where(Message.timestamp >= thirty_seconds_ago).order_by(desc(Message.timestamp))
@@ -248,7 +248,7 @@ class DashboardService:
try: try:
async with get_db_session() as session: async with get_db_session() as session:
# Check for new conversations # Check for new conversations
five_minutes_ago = datetime.utcnow() - timedelta(minutes=5) five_minutes_ago = datetime.now(timezone.utc) - timedelta(minutes=5)
new_conversations_query = select(Conversation).where( new_conversations_query = select(Conversation).where(
Conversation.start_time >= five_minutes_ago Conversation.start_time >= five_minutes_ago
).order_by(desc(Conversation.start_time)) ).order_by(desc(Conversation.start_time))
@@ -297,7 +297,7 @@ class DashboardService:
# Check for unusual activity patterns # Check for unusual activity patterns
async with get_db_session() as session: async with get_db_session() as session:
# Check for error spike # Check for error spike
five_minutes_ago = datetime.utcnow() - timedelta(minutes=5) five_minutes_ago = datetime.now(timezone.utc) - timedelta(minutes=5)
# This would check actual error logs in a real implementation # This would check actual error logs in a real implementation
# For now, simulate occasional alerts # For now, simulate occasional alerts

8
src/admin/services/system_service.py
View File

@@ -3,7 +3,7 @@ System service for monitoring and controlling the fishbowl system
""" """
import logging import logging
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from typing import Dict, List, Any, Optional from typing import Dict, List, Any, Optional
import psutil import psutil
import json import json
@@ -17,7 +17,7 @@ class SystemService:
def __init__(self): def __init__(self):
self.system_state = SystemStatusEnum.RUNNING self.system_state = SystemStatusEnum.RUNNING
self.start_time = datetime.utcnow() self.start_time = datetime.now(timezone.utc)
self.error_count = 0 self.error_count = 0
self.warnings_count = 0 self.warnings_count = 0
self.log_buffer = [] self.log_buffer = []
@@ -30,7 +30,7 @@ class SystemService:
async def get_status(self) -> SystemStatus: async def get_status(self) -> SystemStatus:
"""Get current system status""" """Get current system status"""
try: try:
uptime_seconds = (datetime.utcnow() - self.start_time).total_seconds() uptime_seconds = (datetime.now(timezone.utc) - self.start_time).total_seconds()
uptime_str = self._format_uptime(uptime_seconds) uptime_str = self._format_uptime(uptime_seconds)
# Get resource usage # Get resource usage
@@ -138,7 +138,7 @@ class SystemService:
# In production, this would read from actual log files # In production, this would read from actual log files
sample_logs = [ sample_logs = [
LogEntry( LogEntry(
timestamp=datetime.utcnow() - timedelta(minutes=i), timestamp=datetime.now(timezone.utc) - timedelta(minutes=i),
level="INFO" if i % 3 != 0 else "DEBUG", level="INFO" if i % 3 != 0 else "DEBUG",
component="conversation_engine", component="conversation_engine",
message=f"Sample log message {i}", message=f"Sample log message {i}",

24
src/bot/discord_client.py
View File

@@ -3,7 +3,7 @@ from discord.ext import commands, tasks
import asyncio import asyncio
from typing import Optional, Dict, Any from typing import Optional, Dict, Any
import logging import logging
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from utils.config import get_settings from utils.config import get_settings
from utils.logging import log_error_with_context, log_system_health from utils.logging import log_error_with_context, log_system_health
from database.connection import get_db_session from database.connection import get_db_session
@@ -36,7 +36,7 @@ class FishbowlBot(commands.Bot):
# Health monitoring # Health monitoring
self.health_check_task = None self.health_check_task = None
self.last_heartbeat = datetime.utcnow() self.last_heartbeat = datetime.now(timezone.utc)
async def setup_hook(self): async def setup_hook(self):
"""Called when the bot is starting up""" """Called when the bot is starting up"""
@@ -74,7 +74,7 @@ class FishbowlBot(commands.Bot):
await self.conversation_engine.initialize(self) await self.conversation_engine.initialize(self)
# Update heartbeat # Update heartbeat
self.last_heartbeat = datetime.utcnow() self.last_heartbeat = datetime.now(timezone.utc)
log_system_health("discord_bot", "connected", { log_system_health("discord_bot", "connected", {
"guild": self.target_guild.name, "guild": self.target_guild.name,
@@ -128,7 +128,7 @@ class FishbowlBot(commands.Bot):
async def on_resumed(self): async def on_resumed(self):
"""Handle bot reconnection""" """Handle bot reconnection"""
logger.info("Bot reconnected to Discord") logger.info("Bot reconnected to Discord")
self.last_heartbeat = datetime.utcnow() self.last_heartbeat = datetime.now(timezone.utc)
log_system_health("discord_bot", "reconnected") log_system_health("discord_bot", "reconnected")
async def send_character_message(self, character_name: str, content: str, async def send_character_message(self, character_name: str, content: str,
@@ -217,14 +217,14 @@ class FishbowlBot(commands.Bot):
content=content, content=content,
discord_message_id=discord_message_id, discord_message_id=discord_message_id,
response_to_message_id=reply_to_message_id, response_to_message_id=reply_to_message_id,
timestamp=datetime.utcnow() timestamp=datetime.now(timezone.utc)
) )
session.add(message) session.add(message)
await session.commit() await session.commit()
# Update character's last activity # Update character's last activity
character.last_active = datetime.utcnow() character.last_active = datetime.now(timezone.utc)
character.last_message_id = message.id character.last_message_id = message.id
await session.commit() await session.commit()
@@ -251,25 +251,29 @@ class FishbowlBot(commands.Bot):
"""Periodic health check""" """Periodic health check"""
try: try:
# Check bot connectivity # Check bot connectivity
if self.is_closed(): if self.is_closed() or not self.user:
log_system_health("discord_bot", "disconnected") log_system_health("discord_bot", "disconnected")
return return
# Check heartbeat # Check heartbeat
time_since_heartbeat = datetime.utcnow() - self.last_heartbeat time_since_heartbeat = datetime.now(timezone.utc) - self.last_heartbeat
if time_since_heartbeat > timedelta(minutes=10): if time_since_heartbeat > timedelta(minutes=10):
log_system_health("discord_bot", "heartbeat_stale", { log_system_health("discord_bot", "heartbeat_stale", {
"minutes_since_heartbeat": time_since_heartbeat.total_seconds() / 60 "minutes_since_heartbeat": time_since_heartbeat.total_seconds() / 60
}) })
# Update heartbeat # Update heartbeat
self.last_heartbeat = datetime.utcnow() self.last_heartbeat = datetime.now(timezone.utc)
# Log health metrics # Log health metrics
uptime_minutes = 0
if self.user and hasattr(self.user, 'created_at') and self.user.created_at:
uptime_minutes = (datetime.now(timezone.utc) - self.user.created_at.replace(tzinfo=timezone.utc)).total_seconds() / 60
log_system_health("discord_bot", "healthy", { log_system_health("discord_bot", "healthy", {
"latency_ms": round(self.latency * 1000, 2), "latency_ms": round(self.latency * 1000, 2),
"guild_count": len(self.guilds), "guild_count": len(self.guilds),
"uptime_minutes": (datetime.utcnow() - self.user.created_at).total_seconds() / 60 "uptime_minutes": uptime_minutes
}) })
except Exception as e: except Exception as e:

12
src/bot/message_handler.py
View File

@@ -3,7 +3,7 @@ from discord.ext import commands
import asyncio import asyncio
import logging import logging
from typing import Optional, List, Dict, Any from typing import Optional, List, Dict, Any
from datetime import datetime from datetime import datetime, timezone
from utils.logging import log_error_with_context, log_character_action from utils.logging import log_error_with_context, log_character_action
from database.connection import get_db_session from database.connection import get_db_session
from database.models import Character, Message, Conversation from database.models import Character, Message, Conversation
@@ -121,7 +121,7 @@ class CommandHandler:
# Get recent message count # Get recent message count
from sqlalchemy import func from sqlalchemy import func
message_query = select(func.count(Message.id)).where( message_query = select(func.count(Message.id)).where(
Message.timestamp >= datetime.utcnow() - timedelta(hours=24) Message.timestamp >= datetime.now(timezone.utc) - timedelta(hours=24)
) )
message_count = await session.scalar(message_query) message_count = await session.scalar(message_query)
@@ -131,7 +131,7 @@ class CommandHandler:
embed = discord.Embed( embed = discord.Embed(
title="Fishbowl Status", title="Fishbowl Status",
color=discord.Color.blue(), color=discord.Color.blue(),
timestamp=datetime.utcnow() timestamp=datetime.now(timezone.utc)
) )
embed.add_field( embed.add_field(
@@ -175,7 +175,7 @@ class CommandHandler:
embed = discord.Embed( embed = discord.Embed(
title="Active Characters", title="Active Characters",
color=discord.Color.green(), color=discord.Color.green(),
timestamp=datetime.utcnow() timestamp=datetime.now(timezone.utc)
) )
for character in characters: for character in characters:
@@ -237,7 +237,7 @@ class CommandHandler:
embed = discord.Embed( embed = discord.Embed(
title="Conversation Statistics", title="Conversation Statistics",
color=discord.Color.purple(), color=discord.Color.purple(),
timestamp=datetime.utcnow() timestamp=datetime.now(timezone.utc)
) )
embed.add_field( embed.add_field(
@@ -294,7 +294,7 @@ class CommandHandler:
# Messages today # Messages today
messages_today = await session.scalar( messages_today = await session.scalar(
select(func.count(Message.id)).where( select(func.count(Message.id)).where(
Message.timestamp >= datetime.utcnow() - timedelta(days=1) Message.timestamp >= datetime.now(timezone.utc) - timedelta(days=1)
) )
) )

60
src/characters/character.py
View File

@@ -2,7 +2,7 @@ import asyncio
import random import random
import json import json
from typing import Dict, Any, List, Optional, Tuple from typing import Dict, Any, List, Optional, Tuple
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from dataclasses import dataclass, asdict from dataclasses import dataclass, asdict
from database.connection import get_db_session from database.connection import get_db_session
from database.models import Character as CharacterModel, Memory, CharacterRelationship, Message, CharacterEvolution from database.models import Character as CharacterModel, Memory, CharacterRelationship, Message, CharacterEvolution
@@ -110,8 +110,8 @@ class Character:
# Build prompt with context # Build prompt with context
prompt = await self._build_response_prompt(context) prompt = await self._build_response_prompt(context)
# Generate response using LLM # Generate response using LLM with fallback for slow responses
response = await self.llm_client.generate_response( response = await self.llm_client.generate_response_with_fallback(
prompt=prompt, prompt=prompt,
character_name=self.name, character_name=self.name,
max_tokens=300 max_tokens=300
@@ -147,8 +147,8 @@ class Character:
# Build initiation prompt # Build initiation prompt
prompt = await self._build_initiation_prompt(topic) prompt = await self._build_initiation_prompt(topic)
# Generate opening message # Generate opening message with fallback
opening = await self.llm_client.generate_response( opening = await self.llm_client.generate_response_with_fallback(
prompt=prompt, prompt=prompt,
character_name=self.name, character_name=self.name,
max_tokens=200 max_tokens=200
@@ -226,8 +226,8 @@ class Character:
# Analyze patterns # Analyze patterns
reflection_prompt = await self._build_reflection_prompt(recent_memories) reflection_prompt = await self._build_reflection_prompt(recent_memories)
# Generate reflection # Generate reflection with fallback
reflection = await self.llm_client.generate_response( reflection = await self.llm_client.generate_response_with_fallback(
prompt=reflection_prompt, prompt=reflection_prompt,
character_name=self.name, character_name=self.name,
max_tokens=400 max_tokens=400
@@ -256,7 +256,7 @@ class Character:
return { return {
"reflection": reflection, "reflection": reflection,
"changes": changes, "changes": changes,
"timestamp": datetime.utcnow().isoformat() "timestamp": datetime.now(timezone.utc).isoformat()
} }
return {} return {}
@@ -306,6 +306,24 @@ Energy level: {self.state.energy}
Respond as {self.name} in a natural, conversational way. Keep responses concise but engaging. Stay true to your personality and speaking style.""" Respond as {self.name} in a natural, conversational way. Keep responses concise but engaging. Stay true to your personality and speaking style."""
# Log prompt length for monitoring
logger.debug(f"Generated prompt for {self.name}: {len(prompt)} characters")
# Optimize prompt length if needed
from utils.config import get_settings
settings = get_settings()
max_length = getattr(settings.llm, 'max_prompt_length', 4000)
if len(prompt) > max_length:
logger.warning(f"Prompt too long ({len(prompt)} chars), truncating to {max_length}")
# Truncate at last complete sentence before limit
truncated = prompt[:max_length]
last_period = truncated.rfind('.')
if last_period > max_length * 0.8: # If we can find a period in the last 20%
prompt = truncated[:last_period + 1]
else:
prompt = truncated + "..."
return prompt return prompt
async def _build_initiation_prompt(self, topic: str) -> str: async def _build_initiation_prompt(self, topic: str) -> str:
@@ -436,7 +454,7 @@ Provide a thoughtful reflection on your experiences and any insights about yours
content=content, content=content,
importance_score=importance, importance_score=importance,
tags=tags or [], tags=tags or [],
timestamp=datetime.utcnow() timestamp=datetime.now(timezone.utc)
) )
session.add(memory) session.add(memory)
@@ -456,8 +474,13 @@ Provide a thoughtful reflection on your experiences and any insights about yours
if not memories: if not memories:
return "No relevant memories." return "No relevant memories."
# Get max memories from settings
from utils.config import get_settings
settings = get_settings()
max_memories = getattr(settings.llm, 'max_memories', 3)
formatted = [] formatted = []
for memory in memories[:5]: # Limit to 5 most relevant for memory in memories[:max_memories]: # Configurable number of memories
formatted.append(f"- {memory['content']}") formatted.append(f"- {memory['content']}")
return "\n".join(formatted) return "\n".join(formatted)
@@ -478,8 +501,13 @@ Provide a thoughtful reflection on your experiences and any insights about yours
if not history: if not history:
return "No recent conversation history." return "No recent conversation history."
# Get max messages from settings
from utils.config import get_settings
settings = get_settings()
max_messages = getattr(settings.llm, 'max_history_messages', 3)
formatted = [] formatted = []
for msg in history[-5:]: # Last 5 messages for msg in history[-max_messages:]: # Configurable number of messages
formatted.append(f"{msg['character']}: {msg['content']}") formatted.append(f"{msg['character']}: {msg['content']}")
return "\n".join(formatted) return "\n".join(formatted)
@@ -493,7 +521,7 @@ Provide a thoughtful reflection on your experiences and any insights about yours
self.state.recent_interactions.append({ self.state.recent_interactions.append({
'type': 'response', 'type': 'response',
'content': response[:100], 'content': response[:100],
'timestamp': datetime.utcnow().isoformat() 'timestamp': datetime.now(timezone.utc).isoformat()
}) })
# Keep only last 10 interactions # Keep only last 10 interactions
@@ -683,7 +711,7 @@ Provide a thoughtful reflection on your experiences and any insights about yours
# Update existing relationship # Update existing relationship
relationship.relationship_type = relationship_type relationship.relationship_type = relationship_type
relationship.strength = strength relationship.strength = strength
relationship.last_interaction = datetime.utcnow() relationship.last_interaction = datetime.now(timezone.utc)
relationship.interaction_count += 1 relationship.interaction_count += 1
relationship.notes = reason relationship.notes = reason
else: else:
@@ -693,7 +721,7 @@ Provide a thoughtful reflection on your experiences and any insights about yours
character_b_id=other_char.id, character_b_id=other_char.id,
relationship_type=relationship_type, relationship_type=relationship_type,
strength=strength, strength=strength,
last_interaction=datetime.utcnow(), last_interaction=datetime.now(timezone.utc),
interaction_count=1, interaction_count=1,
notes=reason notes=reason
) )
@@ -705,7 +733,7 @@ Provide a thoughtful reflection on your experiences and any insights about yours
self.relationship_cache[other_character] = { self.relationship_cache[other_character] = {
'type': relationship_type, 'type': relationship_type,
'strength': strength, 'strength': strength,
'last_interaction': datetime.utcnow(), 'last_interaction': datetime.now(timezone.utc),
'notes': reason 'notes': reason
} }
@@ -753,7 +781,7 @@ Provide a thoughtful reflection on your experiences and any insights about yours
old_value=self.personality, old_value=self.personality,
new_value=self.personality, # For now, keep same new_value=self.personality, # For now, keep same
reason=f"Self-reflection triggered evolution (confidence: {changes.get('confidence', 0)})", reason=f"Self-reflection triggered evolution (confidence: {changes.get('confidence', 0)})",
timestamp=datetime.utcnow() timestamp=datetime.now(timezone.utc)
) )
session.add(evolution) session.add(evolution)

20
src/characters/enhanced_character.py
View File

@@ -1,7 +1,7 @@
import asyncio import asyncio
import json import json
from typing import Dict, Any, List, Optional, Tuple from typing import Dict, Any, List, Optional, Tuple
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from dataclasses import dataclass from dataclasses import dataclass
from characters.character import Character from characters.character import Character
@@ -61,7 +61,7 @@ class EnhancedCharacter(Character):
# Autonomous behavior settings # Autonomous behavior settings
self.reflection_frequency = timedelta(hours=6) self.reflection_frequency = timedelta(hours=6)
self.last_reflection = datetime.utcnow() - self.reflection_frequency self.last_reflection = datetime.now(timezone.utc) - self.reflection_frequency
self.self_modification_threshold = 0.7 self.self_modification_threshold = 0.7
self.creativity_drive = 0.8 self.creativity_drive = 0.8
@@ -92,7 +92,7 @@ class EnhancedCharacter(Character):
async def enhanced_self_reflect(self) -> ReflectionCycle: async def enhanced_self_reflect(self) -> ReflectionCycle:
"""Perform enhanced self-reflection using RAG and potential self-modification""" """Perform enhanced self-reflection using RAG and potential self-modification"""
try: try:
cycle_id = f"reflection_{self.name}_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}" cycle_id = f"reflection_{self.name}_{datetime.now(timezone.utc).strftime('%Y%m%d_%H%M%S')}"
log_character_action( log_character_action(
self.name, self.name,
@@ -102,7 +102,7 @@ class EnhancedCharacter(Character):
reflection_cycle = ReflectionCycle( reflection_cycle = ReflectionCycle(
cycle_id=cycle_id, cycle_id=cycle_id,
start_time=datetime.utcnow(), start_time=datetime.now(timezone.utc),
reflections={}, reflections={},
insights_generated=0, insights_generated=0,
self_modifications=[], self_modifications=[],
@@ -131,7 +131,7 @@ class EnhancedCharacter(Character):
reflection_cycle.completed = True reflection_cycle.completed = True
self.reflection_history.append(reflection_cycle) self.reflection_history.append(reflection_cycle)
self.last_reflection = datetime.utcnow() self.last_reflection = datetime.now(timezone.utc)
log_character_action( log_character_action(
self.name, self.name,
@@ -195,10 +195,10 @@ class EnhancedCharacter(Character):
# Generate project plan # Generate project plan
project = { project = {
"id": f"project_{self.name}_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}", "id": f"project_{self.name}_{datetime.now(timezone.utc).strftime('%Y%m%d_%H%M%S')}",
"title": project_idea, "title": project_idea,
"type": project_type, "type": project_type,
"start_date": datetime.utcnow().isoformat(), "start_date": datetime.now(timezone.utc).isoformat(),
"status": "active", "status": "active",
"inspiration": creative_insight.insight, "inspiration": creative_insight.insight,
"supporting_memories": [m.content for m in creative_insight.supporting_memories[:3]], "supporting_memories": [m.content for m in creative_insight.supporting_memories[:3]],
@@ -244,11 +244,11 @@ class EnhancedCharacter(Character):
try: try:
# Create goal object # Create goal object
goal = { goal = {
"id": f"goal_{self.name}_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}", "id": f"goal_{self.name}_{datetime.now(timezone.utc).strftime('%Y%m%d_%H%M%S')}",
"description": goal_description, "description": goal_description,
"priority": priority, "priority": priority,
"timeline": timeline, "timeline": timeline,
"created": datetime.utcnow().isoformat(), "created": datetime.now(timezone.utc).isoformat(),
"status": "active", "status": "active",
"progress": 0.0, "progress": 0.0,
"milestones": [], "milestones": [],
@@ -286,7 +286,7 @@ class EnhancedCharacter(Character):
async def should_perform_reflection(self) -> bool: async def should_perform_reflection(self) -> bool:
"""Determine if character should perform self-reflection""" """Determine if character should perform self-reflection"""
# Time-based reflection # Time-based reflection
time_since_last = datetime.utcnow() - self.last_reflection time_since_last = datetime.now(timezone.utc) - self.last_reflection
if time_since_last >= self.reflection_frequency: if time_since_last >= self.reflection_frequency:
return True return True

12
src/characters/memory.py
View File

@@ -1,7 +1,7 @@
import asyncio import asyncio
import json import json
from typing import Dict, Any, List, Optional, Tuple from typing import Dict, Any, List, Optional, Tuple
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from dataclasses import dataclass from dataclasses import dataclass
from database.connection import get_db_session from database.connection import get_db_session
from database.models import Memory, Character, Message, CharacterRelationship from database.models import Memory, Character, Message, CharacterRelationship
@@ -126,7 +126,7 @@ class MemoryManager:
for memory in memories: for memory in memories:
# Update access count # Update access count
memory.last_accessed = datetime.utcnow() memory.last_accessed = datetime.now(timezone.utc)
memory.access_count += 1 memory.access_count += 1
memory_dict = { memory_dict = {
@@ -272,7 +272,7 @@ class MemoryManager:
# Age criteria # Age criteria
if criteria.get('older_than_days'): if criteria.get('older_than_days'):
cutoff_date = datetime.utcnow() - timedelta(days=criteria['older_than_days']) cutoff_date = datetime.now(timezone.utc) - timedelta(days=criteria['older_than_days'])
query_builder = query_builder.where(Memory.timestamp < cutoff_date) query_builder = query_builder.where(Memory.timestamp < cutoff_date)
# Importance criteria # Importance criteria
@@ -346,7 +346,7 @@ class MemoryManager:
select(func.count(Memory.id)).where( select(func.count(Memory.id)).where(
and_( and_(
Memory.character_id == self.character.id, Memory.character_id == self.character.id,
Memory.timestamp >= datetime.utcnow() - timedelta(days=7) Memory.timestamp >= datetime.now(timezone.utc) - timedelta(days=7)
) )
) )
) )
@@ -441,8 +441,8 @@ class MemoryManager:
tags=tags, tags=tags,
related_character_id=related_character_id, related_character_id=related_character_id,
related_message_id=related_message_id, related_message_id=related_message_id,
timestamp=datetime.utcnow(), timestamp=datetime.now(timezone.utc),
last_accessed=datetime.utcnow(), last_accessed=datetime.now(timezone.utc),
access_count=0 access_count=0
) )

4
src/characters/personality.py
View File

@@ -1,7 +1,7 @@
import json import json
import random import random
from typing import Dict, Any, List, Optional, Tuple from typing import Dict, Any, List, Optional, Tuple
from datetime import datetime from datetime import datetime, timezone
from utils.logging import log_character_action, log_error_with_context from utils.logging import log_character_action, log_error_with_context
from database.connection import get_db_session from database.connection import get_db_session
from database.models import CharacterEvolution, Character as CharacterModel from database.models import CharacterEvolution, Character as CharacterModel
@@ -330,7 +330,7 @@ class PersonalityManager:
old_value=old_personality, old_value=old_personality,
new_value=new_personality, new_value=new_personality,
reason=f"Evolution score: {evolution_score:.2f}. {reason}", reason=f"Evolution score: {evolution_score:.2f}. {reason}",
timestamp=datetime.utcnow() timestamp=datetime.now(timezone.utc)
) )
session.add(evolution) session.add(evolution)

26
src/collaboration/creative_projects.py
View File

@@ -7,7 +7,7 @@ import asyncio
import json import json
import logging import logging
from typing import Dict, List, Any, Optional, Set, Tuple from typing import Dict, List, Any, Optional, Set, Tuple
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from dataclasses import dataclass, asdict from dataclasses import dataclass, asdict
from enum import Enum from enum import Enum
import hashlib import hashlib
@@ -190,7 +190,7 @@ class CollaborativeCreativeManager:
return False, "Missing required project fields" return False, "Missing required project fields"
# Create project ID # Create project ID
project_id = f"project_{initiator}_{datetime.utcnow().timestamp()}" project_id = f"project_{initiator}_{datetime.now(timezone.utc).timestamp()}"
# Determine project type # Determine project type
try: try:
@@ -210,7 +210,7 @@ class CollaborativeCreativeManager:
status=ProjectStatus.PROPOSED, status=ProjectStatus.PROPOSED,
initiator=initiator, initiator=initiator,
collaborators=[initiator], # Start with just initiator collaborators=[initiator], # Start with just initiator
created_at=datetime.utcnow(), created_at=datetime.now(timezone.utc),
target_completion=None, # Will be set during planning target_completion=None, # Will be set during planning
contributions=[], contributions=[],
project_goals=project_idea.get("goals", []), project_goals=project_idea.get("goals", []),
@@ -272,14 +272,14 @@ class CollaborativeCreativeManager:
if invitation.status != "pending": if invitation.status != "pending":
return False, f"Invitation is already {invitation.status}" return False, f"Invitation is already {invitation.status}"
if datetime.utcnow() > invitation.expires_at: if datetime.now(timezone.utc) > invitation.expires_at:
invitation.status = "expired" invitation.status = "expired"
return False, "Invitation has expired" return False, "Invitation has expired"
# Update invitation # Update invitation
invitation.status = "accepted" if accepted else "rejected" invitation.status = "accepted" if accepted else "rejected"
invitation.response_message = response_message invitation.response_message = response_message
invitation.responded_at = datetime.utcnow() invitation.responded_at = datetime.now(timezone.utc)
if accepted: if accepted:
# Add collaborator to project # Add collaborator to project
@@ -334,7 +334,7 @@ class CollaborativeCreativeManager:
return False, f"Invalid contribution type: {contribution['contribution_type']}" return False, f"Invalid contribution type: {contribution['contribution_type']}"
# Create contribution ID # Create contribution ID
contribution_id = f"contrib_{project_id}_{len(project.contributions)}_{datetime.utcnow().timestamp()}" contribution_id = f"contrib_{project_id}_{len(project.contributions)}_{datetime.now(timezone.utc).timestamp()}"
# Create contribution object # Create contribution object
project_contribution = ProjectContribution( project_contribution = ProjectContribution(
@@ -342,7 +342,7 @@ class CollaborativeCreativeManager:
contributor=contributor, contributor=contributor,
contribution_type=contribution_type, contribution_type=contribution_type,
content=contribution["content"], content=contribution["content"],
timestamp=datetime.utcnow(), timestamp=datetime.now(timezone.utc),
build_on_contribution_id=contribution.get("build_on_contribution_id"), build_on_contribution_id=contribution.get("build_on_contribution_id"),
feedback_for_contribution_id=contribution.get("feedback_for_contribution_id"), feedback_for_contribution_id=contribution.get("feedback_for_contribution_id"),
metadata=contribution.get("metadata", {}) metadata=contribution.get("metadata", {})
@@ -498,7 +498,7 @@ class CollaborativeCreativeManager:
}) })
# Project health metrics # Project health metrics
days_active = (datetime.utcnow() - project.created_at).days days_active = (datetime.now(timezone.utc) - project.created_at).days
avg_contributions_per_day = len(project.contributions) / max(1, days_active) avg_contributions_per_day = len(project.contributions) / max(1, days_active)
# Collaboration quality # Collaboration quality
@@ -532,7 +532,7 @@ class CollaborativeCreativeManager:
role_description: str, invitation_message: str) -> bool: role_description: str, invitation_message: str) -> bool:
"""Create a project invitation""" """Create a project invitation"""
try: try:
invitation_id = f"invite_{project_id}_{invitee}_{datetime.utcnow().timestamp()}" invitation_id = f"invite_{project_id}_{invitee}_{datetime.now(timezone.utc).timestamp()}"
invitation = ProjectInvitation( invitation = ProjectInvitation(
id=invitation_id, id=invitation_id,
@@ -541,8 +541,8 @@ class CollaborativeCreativeManager:
invitee=invitee, invitee=invitee,
role_description=role_description, role_description=role_description,
invitation_message=invitation_message, invitation_message=invitation_message,
created_at=datetime.utcnow(), created_at=datetime.now(timezone.utc),
expires_at=datetime.utcnow() + timedelta(days=7), # 7 day expiry expires_at=datetime.now(timezone.utc) + timedelta(days=7), # 7 day expiry
status="pending" status="pending"
) )
@@ -668,7 +668,7 @@ class CollaborativeCreativeManager:
invitations_query = select(DBProjectInvitation).where( invitations_query = select(DBProjectInvitation).where(
and_( and_(
DBProjectInvitation.status == 'pending', DBProjectInvitation.status == 'pending',
DBProjectInvitation.expires_at > datetime.utcnow() DBProjectInvitation.expires_at > datetime.now(timezone.utc)
) )
) )
@@ -783,7 +783,7 @@ class CollaborativeCreativeManager:
db_collaborator = ProjectCollaborator( db_collaborator = ProjectCollaborator(
project_id=project.id, project_id=project.id,
character_id=collaborator.id, character_id=collaborator.id,
joined_at=project.created_at if collaborator_name == project.initiator else datetime.utcnow() joined_at=project.created_at if collaborator_name == project.initiator else datetime.now(timezone.utc)
) )
session.add(db_collaborator) session.add(db_collaborator)

44
src/conversation/engine.py
View File

@@ -2,7 +2,7 @@ import asyncio
import random import random
import json import json
from typing import Dict, Any, List, Optional, Set, Tuple from typing import Dict, Any, List, Optional, Set, Tuple
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from dataclasses import dataclass, asdict from dataclasses import dataclass, asdict
from enum import Enum from enum import Enum
import logging import logging
@@ -44,9 +44,9 @@ class ConversationContext:
if self.participants is None: if self.participants is None:
self.participants = [] self.participants = []
if self.start_time is None: if self.start_time is None:
self.start_time = datetime.utcnow() self.start_time = datetime.now(timezone.utc)
if self.last_activity is None: if self.last_activity is None:
self.last_activity = datetime.utcnow() self.last_activity = datetime.now(timezone.utc)
class ConversationEngine: class ConversationEngine:
"""Autonomous conversation engine that manages character interactions""" """Autonomous conversation engine that manages character interactions"""
@@ -89,8 +89,8 @@ class ConversationEngine:
'conversations_started': 0, 'conversations_started': 0,
'messages_generated': 0, 'messages_generated': 0,
'characters_active': 0, 'characters_active': 0,
'uptime_start': datetime.utcnow(), 'uptime_start': datetime.now(timezone.utc),
'last_activity': datetime.utcnow() 'last_activity': datetime.now(timezone.utc)
} }
async def initialize(self, discord_bot): async def initialize(self, discord_bot):
@@ -169,7 +169,7 @@ class ConversationEngine:
# Update context # Update context
context.current_speaker = initial_speaker context.current_speaker = initial_speaker
context.message_count = 1 context.message_count = 1
context.last_activity = datetime.utcnow() context.last_activity = datetime.now(timezone.utc)
# Store message in database # Store message in database
await self._store_conversation_message( await self._store_conversation_message(
@@ -179,7 +179,7 @@ class ConversationEngine:
# Update statistics # Update statistics
self.stats['conversations_started'] += 1 self.stats['conversations_started'] += 1
self.stats['messages_generated'] += 1 self.stats['messages_generated'] += 1
self.stats['last_activity'] = datetime.utcnow() self.stats['last_activity'] = datetime.now(timezone.utc)
log_conversation_event( log_conversation_event(
conversation_id, "conversation_started", conversation_id, "conversation_started",
@@ -230,7 +230,7 @@ class ConversationEngine:
# Update context # Update context
context.current_speaker = next_speaker context.current_speaker = next_speaker
context.message_count += 1 context.message_count += 1
context.last_activity = datetime.utcnow() context.last_activity = datetime.now(timezone.utc)
# Store message # Store message
await self._store_conversation_message( await self._store_conversation_message(
@@ -245,7 +245,7 @@ class ConversationEngine:
# Update statistics # Update statistics
self.stats['messages_generated'] += 1 self.stats['messages_generated'] += 1
self.stats['last_activity'] = datetime.utcnow() self.stats['last_activity'] = datetime.now(timezone.utc)
log_conversation_event( log_conversation_event(
conversation_id, "message_sent", conversation_id, "message_sent",
@@ -379,7 +379,7 @@ class ConversationEngine:
async def get_status(self) -> Dict[str, Any]: async def get_status(self) -> Dict[str, Any]:
"""Get engine status""" """Get engine status"""
uptime = datetime.utcnow() - self.stats['uptime_start'] uptime = datetime.now(timezone.utc) - self.stats['uptime_start']
return { return {
'status': self.state.value, 'status': self.state.value,
@@ -402,8 +402,8 @@ class ConversationEngine:
# Use EnhancedCharacter if RAG systems are available # Use EnhancedCharacter if RAG systems are available
if self.vector_store and self.memory_sharing_manager: if self.vector_store and self.memory_sharing_manager:
# Find the appropriate MCP servers for this character # Find the appropriate MCP servers for this character
from mcp.self_modification_server import mcp_server from mcp_servers.self_modification_server import mcp_server
from mcp.file_system_server import filesystem_server from mcp_servers.file_system_server import filesystem_server
# Find creative projects MCP server # Find creative projects MCP server
creative_projects_mcp = None creative_projects_mcp = None
@@ -500,7 +500,7 @@ class ConversationEngine:
base_chance = 0.3 base_chance = 0.3
# Increase chance if no recent activity # Increase chance if no recent activity
time_since_last = datetime.utcnow() - self.stats['last_activity'] time_since_last = datetime.now(timezone.utc) - self.stats['last_activity']
if time_since_last > timedelta(hours=2): if time_since_last > timedelta(hours=2):
base_chance += 0.4 base_chance += 0.4
elif time_since_last > timedelta(hours=1): elif time_since_last > timedelta(hours=1):
@@ -515,7 +515,7 @@ class ConversationEngine:
return False return False
# Check time limit (conversations shouldn't go on forever) # Check time limit (conversations shouldn't go on forever)
duration = datetime.utcnow() - context.start_time duration = datetime.now(timezone.utc) - context.start_time
if duration > timedelta(hours=2): if duration > timedelta(hours=2):
return False return False
@@ -541,7 +541,7 @@ class ConversationEngine:
context = self.active_conversations[conversation_id] context = self.active_conversations[conversation_id]
# Check time since last message # Check time since last message
time_since_last = datetime.utcnow() - context.last_activity time_since_last = datetime.now(timezone.utc) - context.last_activity
min_wait = timedelta(seconds=random.uniform(30, 120)) min_wait = timedelta(seconds=random.uniform(30, 120))
return time_since_last >= min_wait return time_since_last >= min_wait
@@ -576,7 +576,7 @@ class ConversationEngine:
def _is_quiet_hours(self) -> bool: def _is_quiet_hours(self) -> bool:
"""Check if it's currently quiet hours""" """Check if it's currently quiet hours"""
current_hour = datetime.now().hour current_hour = datetime.now(timezone.utc).hour
start_hour, end_hour = self.quiet_hours start_hour, end_hour = self.quiet_hours
if start_hour <= end_hour: if start_hour <= end_hour:
@@ -601,8 +601,8 @@ class ConversationEngine:
channel_id=str(self.discord_bot.channel_id), channel_id=str(self.discord_bot.channel_id),
topic=topic, topic=topic,
participants=participants, participants=participants,
start_time=datetime.utcnow(), start_time=datetime.now(timezone.utc),
last_activity=datetime.utcnow(), last_activity=datetime.now(timezone.utc),
is_active=True, is_active=True,
message_count=0 message_count=0
) )
@@ -745,7 +745,7 @@ class ConversationEngine:
conversation_id=conversation_id, conversation_id=conversation_id,
character_id=character.id, character_id=character.id,
content=content, content=content,
timestamp=datetime.utcnow() timestamp=datetime.now(timezone.utc)
) )
session.add(message) session.add(message)
@@ -821,7 +821,7 @@ class ConversationEngine:
conversation = await session.get(Conversation, conversation_id) conversation = await session.get(Conversation, conversation_id)
if conversation: if conversation:
conversation.is_active = False conversation.is_active = False
conversation.last_activity = datetime.utcnow() conversation.last_activity = datetime.now(timezone.utc)
conversation.message_count = context.message_count conversation.message_count = context.message_count
await session.commit() await session.commit()
@@ -831,7 +831,7 @@ class ConversationEngine:
log_conversation_event( log_conversation_event(
conversation_id, "conversation_ended", conversation_id, "conversation_ended",
context.participants, context.participants,
{"total_messages": context.message_count, "duration": str(datetime.utcnow() - context.start_time)} {"total_messages": context.message_count, "duration": str(datetime.now(timezone.utc) - context.start_time)}
) )
except Exception as e: except Exception as e:
@@ -854,7 +854,7 @@ class ConversationEngine:
async def _cleanup_old_conversations(self): async def _cleanup_old_conversations(self):
"""Clean up old inactive conversations""" """Clean up old inactive conversations"""
try: try:
cutoff_time = datetime.utcnow() - timedelta(hours=6) cutoff_time = datetime.now(timezone.utc) - timedelta(hours=6)
# Remove old conversations from active list # Remove old conversations from active list
to_remove = [] to_remove = []

12
src/conversation/scheduler.py
View File

@@ -2,7 +2,7 @@ import asyncio
import random import random
import schedule import schedule
from typing import Dict, Any, List, Optional from typing import Dict, Any, List, Optional
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from dataclasses import dataclass from dataclasses import dataclass
from enum import Enum from enum import Enum
import logging import logging
@@ -102,7 +102,7 @@ class ConversationScheduler:
async def schedule_event(self, event_type: str, delay: timedelta, async def schedule_event(self, event_type: str, delay: timedelta,
character_name: str = None, **kwargs): character_name: str = None, **kwargs):
"""Schedule a specific event""" """Schedule a specific event"""
scheduled_time = datetime.utcnow() + delay scheduled_time = datetime.now(timezone.utc) + delay
event = ScheduledEvent( event = ScheduledEvent(
event_type=event_type, event_type=event_type,
@@ -170,7 +170,7 @@ class ConversationScheduler:
'event_type': event.event_type, 'event_type': event.event_type,
'scheduled_time': event.scheduled_time.isoformat(), 'scheduled_time': event.scheduled_time.isoformat(),
'character_name': event.character_name, 'character_name': event.character_name,
'time_until': (event.scheduled_time - datetime.utcnow()).total_seconds(), 'time_until': (event.scheduled_time - datetime.now(timezone.utc)).total_seconds(),
'parameters': event.parameters 'parameters': event.parameters
} }
for event in upcoming for event in upcoming
@@ -194,7 +194,7 @@ class ConversationScheduler:
async def _process_due_events(self): async def _process_due_events(self):
"""Process events that are due""" """Process events that are due"""
now = datetime.utcnow() now = datetime.now(timezone.utc)
due_events = [] due_events = []
# Find due events # Find due events
@@ -378,7 +378,7 @@ class ConversationScheduler:
base_minutes = random.uniform(20, 60) base_minutes = random.uniform(20, 60)
# Adjust based on time of day # Adjust based on time of day
current_hour = datetime.now().hour current_hour = datetime.now(timezone.utc).hour
activity_multiplier = self._get_activity_multiplier(current_hour) activity_multiplier = self._get_activity_multiplier(current_hour)
# Adjust based on current activity # Adjust based on current activity
@@ -427,7 +427,7 @@ class ConversationScheduler:
def _get_current_activity_pattern(self) -> str: def _get_current_activity_pattern(self) -> str:
"""Get current activity pattern""" """Get current activity pattern"""
current_hour = datetime.now().hour current_hour = datetime.now(timezone.utc).hour
for period, config in self.activity_patterns.items(): for period, config in self.activity_patterns.items():
start, end = config['start'], config['end'] start, end = config['start'], config['end']

12
src/database/connection.py
View File

@@ -19,8 +19,16 @@ class DatabaseManager:
self._pool = None self._pool = None
async def initialize(self): async def initialize(self):
# Use database URL from config # Use DATABASE_URL environment variable first, then construct from config
database_url = getattr(self.settings.database, 'url', 'sqlite+aiosqlite:///fishbowl_test.db') import os
database_url = os.getenv('DATABASE_URL')
if not database_url:
# Construct URL from config components
db_config = self.settings.database
database_url = f"postgresql+asyncpg://{db_config.user}:{db_config.password}@{db_config.host}:{db_config.port}/{db_config.name}"
logger.info(f"Using database URL: {database_url.replace(self.settings.database.password, '***') if database_url else 'None'}")
# Configure engine based on database type # Configure engine based on database type
if 'sqlite' in database_url: if 'sqlite' in database_url:

1
src/database/models.py
View File

@@ -53,6 +53,7 @@ class Conversation(Base):
topic = Column(String(200)) topic = Column(String(200))
participants = Column(JSON, nullable=False, default=list) participants = Column(JSON, nullable=False, default=list)
start_time = Column(DateTime, default=func.now()) start_time = Column(DateTime, default=func.now())
end_time = Column(DateTime, nullable=True)
last_activity = Column(DateTime, default=func.now()) last_activity = Column(DateTime, default=func.now())
is_active = Column(Boolean, default=True) is_active = Column(Boolean, default=True)
message_count = Column(Integer, default=0) message_count = Column(Integer, default=0)

103
src/llm/client.py
View File

@@ -3,7 +3,7 @@ import httpx
import json import json
import time import time
from typing import Dict, Any, Optional, List from typing import Dict, Any, Optional, List
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from utils.config import get_settings from utils.config import get_settings
from utils.logging import log_llm_interaction, log_error_with_context, log_system_health from utils.logging import log_llm_interaction, log_error_with_context, log_system_health
import logging import logging
@@ -29,17 +29,23 @@ class LLMClient:
self.cache = {} self.cache = {}
self.cache_ttl = 300 # 5 minutes self.cache_ttl = 300 # 5 minutes
# Background task queue for long-running requests
self.pending_requests = {}
self.max_timeout = 60 # Hard timeout limit for immediate responses
self.fallback_timeout = 15 # Quick timeout for immediate responses
# Health monitoring # Health monitoring
self.health_stats = { self.health_stats = {
'total_requests': 0, 'total_requests': 0,
'successful_requests': 0, 'successful_requests': 0,
'failed_requests': 0, 'failed_requests': 0,
'average_response_time': 0, 'average_response_time': 0,
'last_health_check': datetime.utcnow() 'last_health_check': datetime.now(timezone.utc)
} }
async def generate_response(self, prompt: str, character_name: str = None, async def generate_response(self, prompt: str, character_name: str = None,
max_tokens: int = None, temperature: float = None) -> Optional[str]: max_tokens: int = None, temperature: float = None,
use_fallback: bool = True) -> Optional[str]:
"""Generate response using LLM""" """Generate response using LLM"""
try: try:
# Rate limiting check # Rate limiting check
@@ -55,8 +61,11 @@ class LLMClient:
start_time = time.time() start_time = time.time()
# Use shorter timeout for immediate responses, longer for background
effective_timeout = self.fallback_timeout if use_fallback else min(self.timeout, self.max_timeout)
# Try OpenAI-compatible API first (KoboldCPP, etc.) # Try OpenAI-compatible API first (KoboldCPP, etc.)
async with httpx.AsyncClient(timeout=self.timeout) as client: async with httpx.AsyncClient(timeout=effective_timeout) as client:
try: try:
# OpenAI-compatible request # OpenAI-compatible request
request_data = { request_data = {
@@ -134,8 +143,23 @@ class LLMClient:
return None return None
except httpx.TimeoutException: except httpx.TimeoutException:
logger.error(f"LLM request timeout for {character_name}") if use_fallback:
self._update_stats(False, self.timeout) logger.warning(f"LLM request timeout for {character_name}, using fallback response")
# Queue for background processing if needed
if self.timeout > self.max_timeout:
background_task = asyncio.create_task(self.generate_response(
prompt, character_name, max_tokens, temperature, use_fallback=False
))
request_id = f"{character_name}_{time.time()}"
self.pending_requests[request_id] = background_task
# Return a fallback response immediately
fallback_response = self._get_fallback_response(character_name)
self._update_stats(False, effective_timeout)
return fallback_response
else:
logger.error(f"LLM background request timeout for {character_name}")
self._update_stats(False, effective_timeout)
return None return None
except httpx.HTTPError as e: except httpx.HTTPError as e:
logger.error(f"LLM HTTP error for {character_name}: {e}") logger.error(f"LLM HTTP error for {character_name}: {e}")
@@ -231,11 +255,11 @@ class LLMClient:
'response_time': duration, 'response_time': duration,
'model': self.model, 'model': self.model,
'base_url': self.base_url, 'base_url': self.base_url,
'timestamp': datetime.utcnow().isoformat() 'timestamp': datetime.now(timezone.utc).isoformat()
} }
# Update health check time # Update health check time
self.health_stats['last_health_check'] = datetime.utcnow() self.health_stats['last_health_check'] = datetime.now(timezone.utc)
return health_status return health_status
@@ -246,7 +270,7 @@ class LLMClient:
'error': str(e), 'error': str(e),
'model': self.model, 'model': self.model,
'base_url': self.base_url, 'base_url': self.base_url,
'timestamp': datetime.utcnow().isoformat() 'timestamp': datetime.now(timezone.utc).isoformat()
} }
def get_statistics(self) -> Dict[str, Any]: def get_statistics(self) -> Dict[str, Any]:
@@ -343,6 +367,67 @@ class LLMClient:
(current_avg * (total_requests - 1) + duration) / total_requests (current_avg * (total_requests - 1) + duration) / total_requests
) )
def _get_fallback_response(self, character_name: str = None) -> str:
"""Generate a fallback response when LLM is slow"""
fallback_responses = [
"*thinking deeply about this...*",
"*processing thoughts...*",
"*contemplating the discussion...*",
"*reflecting on what you've said...*",
"*considering different perspectives...*",
"Hmm, that's an interesting point to consider.",
"I need a moment to think about that.",
"That's worth reflecting on carefully.",
"*taking time to formulate thoughts...*"
]
import random
return random.choice(fallback_responses)
async def generate_response_with_fallback(self, prompt: str, character_name: str = None,
max_tokens: int = None, temperature: float = None) -> str:
"""Generate response with guaranteed fallback if LLM is slow"""
try:
# Try immediate response first
response = await self.generate_response(
prompt, character_name, max_tokens, temperature, use_fallback=True
)
if response:
return response
else:
# Return fallback if no response
return self._get_fallback_response(character_name)
except Exception as e:
log_error_with_context(e, {
"character_name": character_name,
"prompt_length": len(prompt)
})
return self._get_fallback_response(character_name)
async def cleanup_pending_requests(self):
"""Clean up completed background requests"""
completed_requests = []
for request_id, task in self.pending_requests.items():
if task.done():
completed_requests.append(request_id)
try:
result = await task
if result:
logger.info(f"Background LLM request {request_id} completed successfully")
except Exception as e:
logger.error(f"Background LLM request {request_id} failed: {e}")
# Remove completed requests
for request_id in completed_requests:
del self.pending_requests[request_id]
def get_pending_count(self) -> int:
"""Get number of pending background requests"""
return len(self.pending_requests)
class PromptManager: class PromptManager:
"""Manages prompt templates and optimization""" """Manages prompt templates and optimization"""

24
src/main.py
View File

@@ -168,6 +168,10 @@ class FishbowlApplication:
await self.scheduler.start() await self.scheduler.start()
logger.info("Conversation scheduler started") logger.info("Conversation scheduler started")
# Start LLM cleanup task
cleanup_task = asyncio.create_task(self._llm_cleanup_loop())
logger.info("LLM cleanup task started")
# Start Discord bot # Start Discord bot
bot_task = asyncio.create_task( bot_task = asyncio.create_task(
self.discord_bot.start(self.settings.discord.token) self.discord_bot.start(self.settings.discord.token)
@@ -181,7 +185,7 @@ class FishbowlApplication:
# Wait for shutdown signal or bot completion # Wait for shutdown signal or bot completion
done, pending = await asyncio.wait( done, pending = await asyncio.wait(
[bot_task, asyncio.create_task(self.shutdown_event.wait())], [bot_task, cleanup_task, asyncio.create_task(self.shutdown_event.wait())],
return_when=asyncio.FIRST_COMPLETED return_when=asyncio.FIRST_COMPLETED
) )
@@ -240,6 +244,24 @@ class FishbowlApplication:
if os.name == 'nt': if os.name == 'nt':
signal.signal(signal.SIGBREAK, signal_handler) signal.signal(signal.SIGBREAK, signal_handler)
async def _llm_cleanup_loop(self):
"""Background task to clean up completed LLM requests"""
try:
while not self.shutdown_event.is_set():
await llm_client.cleanup_pending_requests()
pending_count = llm_client.get_pending_count()
if pending_count > 0:
logger.debug(f"LLM cleanup: {pending_count} pending background requests")
# Wait 30 seconds before next cleanup
await asyncio.sleep(30)
except asyncio.CancelledError:
logger.info("LLM cleanup task cancelled")
except Exception as e:
logger.error(f"Error in LLM cleanup loop: {e}")
async def main(): async def main():
"""Main entry point""" """Main entry point"""
app = FishbowlApplication() app = FishbowlApplication()

46
src/mcp_servers/calendar_server.py
View File

@@ -1,7 +1,7 @@
import asyncio import asyncio
import json import json
from typing import Dict, List, Any, Optional, Set from typing import Dict, List, Any, Optional, Set
from datetime import datetime, timedelta, date from datetime import datetime, timedelta, timezone, date
from dataclasses import dataclass, asdict from dataclasses import dataclass, asdict
from pathlib import Path from pathlib import Path
import aiofiles import aiofiles
@@ -51,7 +51,7 @@ class ScheduledEvent:
def __post_init__(self): def __post_init__(self):
if self.created_at is None: if self.created_at is None:
self.created_at = datetime.utcnow() self.created_at = datetime.now(timezone.utc)
def to_dict(self) -> Dict[str, Any]: def to_dict(self) -> Dict[str, Any]:
return { return {
@@ -224,7 +224,7 @@ class CalendarTimeAwarenessMCP:
# Create event # Create event
event = ScheduledEvent( event = ScheduledEvent(
id=f"event_{character_name}_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}", id=f"event_{character_name}_{datetime.now(timezone.utc).strftime('%Y%m%d_%H%M%S')}",
character_name=character_name, character_name=character_name,
event_type=event_type_enum, event_type=event_type_enum,
title=title, title=title,
@@ -275,7 +275,7 @@ class CalendarTimeAwarenessMCP:
) -> List[TextContent]: ) -> List[TextContent]:
"""Get character's upcoming events""" """Get character's upcoming events"""
try: try:
now = datetime.utcnow() now = datetime.now(timezone.utc)
end_time = now + timedelta(days=days_ahead) end_time = now + timedelta(days=days_ahead)
upcoming_events = [] upcoming_events = []
@@ -340,7 +340,7 @@ class CalendarTimeAwarenessMCP:
event = self.scheduled_events[character_name][event_id] event = self.scheduled_events[character_name][event_id]
event.completed = True event.completed = True
event.metadata["completion_time"] = datetime.utcnow().isoformat() event.metadata["completion_time"] = datetime.now(timezone.utc).isoformat()
event.metadata["completion_notes"] = notes event.metadata["completion_notes"] = notes
await self._save_character_calendar(character_name) await self._save_character_calendar(character_name)
@@ -442,7 +442,7 @@ class CalendarTimeAwarenessMCP:
# Create milestone # Create milestone
milestone = Milestone( milestone = Milestone(
id=f"milestone_{character_name}_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}", id=f"milestone_{character_name}_{datetime.now(timezone.utc).strftime('%Y%m%d_%H%M%S')}",
character_name=character_name, character_name=character_name,
milestone_type=milestone_type, milestone_type=milestone_type,
description=description, description=description,
@@ -499,7 +499,7 @@ class CalendarTimeAwarenessMCP:
) -> List[TextContent]: ) -> List[TextContent]:
"""Get upcoming anniversaries and milestones""" """Get upcoming anniversaries and milestones"""
try: try:
now = datetime.utcnow() now = datetime.now(timezone.utc)
end_time = now + timedelta(days=days_ahead) end_time = now + timedelta(days=days_ahead)
upcoming_anniversaries = [] upcoming_anniversaries = []
@@ -580,7 +580,7 @@ class CalendarTimeAwarenessMCP:
if "celebrations" not in milestone.__dict__: if "celebrations" not in milestone.__dict__:
milestone.__dict__["celebrations"] = {} milestone.__dict__["celebrations"] = {}
milestone.__dict__["celebrations"][celebration_key] = { milestone.__dict__["celebrations"][celebration_key] = {
"date": datetime.utcnow().isoformat(), "date": datetime.now(timezone.utc).isoformat(),
"notes": celebration_notes "notes": celebration_notes
} }
@@ -628,7 +628,7 @@ class CalendarTimeAwarenessMCP:
"""Get time elapsed since a specific type of event""" """Get time elapsed since a specific type of event"""
try: try:
# Search through recent events # Search through recent events
cutoff_date = datetime.utcnow() - timedelta(days=search_days_back) cutoff_date = datetime.now(timezone.utc) - timedelta(days=search_days_back)
matching_events = [] matching_events = []
for event in self.scheduled_events.get(character_name, {}).values(): for event in self.scheduled_events.get(character_name, {}).values():
@@ -665,7 +665,7 @@ class CalendarTimeAwarenessMCP:
most_recent_description = most_recent_interaction["description"] most_recent_description = most_recent_interaction["description"]
# Calculate time difference # Calculate time difference
time_diff = datetime.utcnow() - most_recent_time time_diff = datetime.now(timezone.utc) - most_recent_time
# Format time difference # Format time difference
if time_diff.days > 0: if time_diff.days > 0:
@@ -709,7 +709,7 @@ class CalendarTimeAwarenessMCP:
) -> List[TextContent]: ) -> List[TextContent]:
"""Get summary of character's activities over a time period""" """Get summary of character's activities over a time period"""
try: try:
end_date = datetime.utcnow() end_date = datetime.now(timezone.utc)
start_date = end_date - timedelta(days=period_days) start_date = end_date - timedelta(days=period_days)
# Get completed events in period # Get completed events in period
@@ -783,7 +783,7 @@ class CalendarTimeAwarenessMCP:
if character_name not in self.last_interactions: if character_name not in self.last_interactions:
self.last_interactions[character_name] = {} self.last_interactions[character_name] = {}
self.last_interactions[character_name][other_character] = datetime.utcnow() self.last_interactions[character_name][other_character] = datetime.now(timezone.utc)
# Save to file # Save to file
await self._save_relationship_tracking(character_name) await self._save_relationship_tracking(character_name)
@@ -834,7 +834,7 @@ class CalendarTimeAwarenessMCP:
text=f"No recorded interactions with {other_character}" text=f"No recorded interactions with {other_character}"
)] )]
time_since = datetime.utcnow() - last_interaction time_since = datetime.now(timezone.utc) - last_interaction
days_since = time_since.days days_since = time_since.days
# Determine maintenance status # Determine maintenance status
@@ -859,7 +859,7 @@ class CalendarTimeAwarenessMCP:
# Get status for all relationships # Get status for all relationships
relationships = [] relationships = []
for other_char, last_interaction in self.last_interactions.get(character_name, {}).items(): for other_char, last_interaction in self.last_interactions.get(character_name, {}).items():
time_since = datetime.utcnow() - last_interaction time_since = datetime.now(timezone.utc) - last_interaction
days_since = time_since.days days_since = time_since.days
if days_since <= 1: if days_since <= 1:
@@ -914,13 +914,13 @@ class CalendarTimeAwarenessMCP:
"""Schedule relationship maintenance activity""" """Schedule relationship maintenance activity"""
try: try:
# Create relationship maintenance event # Create relationship maintenance event
scheduled_time = datetime.utcnow() + timedelta(days=days_from_now) scheduled_time = datetime.now(timezone.utc) + timedelta(days=days_from_now)
template = self.event_templates[EventType.RELATIONSHIP_MAINTENANCE] template = self.event_templates[EventType.RELATIONSHIP_MAINTENANCE]
description = template["description_template"].format(target=other_character) description = template["description_template"].format(target=other_character)
event = ScheduledEvent( event = ScheduledEvent(
id=f"rel_maintenance_{character_name}_{other_character}_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}", id=f"rel_maintenance_{character_name}_{other_character}_{datetime.now(timezone.utc).strftime('%Y%m%d_%H%M%S')}",
character_name=character_name, character_name=character_name,
event_type=EventType.RELATIONSHIP_MAINTENANCE, event_type=EventType.RELATIONSHIP_MAINTENANCE,
title=f"Connect with {other_character}", title=f"Connect with {other_character}",
@@ -1002,7 +1002,7 @@ class CalendarTimeAwarenessMCP:
events_data = { events_data = {
"events": [event.to_dict() for event in self.scheduled_events.get(character_name, {}).values()], "events": [event.to_dict() for event in self.scheduled_events.get(character_name, {}).values()],
"last_updated": datetime.utcnow().isoformat() "last_updated": datetime.now(timezone.utc).isoformat()
} }
async with aiofiles.open(calendar_file, 'w') as f: async with aiofiles.open(calendar_file, 'w') as f:
@@ -1019,7 +1019,7 @@ class CalendarTimeAwarenessMCP:
milestones_data = { milestones_data = {
"milestones": [milestone.to_dict() for milestone in self.milestones.get(character_name, {}).values()], "milestones": [milestone.to_dict() for milestone in self.milestones.get(character_name, {}).values()],
"last_updated": datetime.utcnow().isoformat() "last_updated": datetime.now(timezone.utc).isoformat()
} }
async with aiofiles.open(milestones_file, 'w') as f: async with aiofiles.open(milestones_file, 'w') as f:
@@ -1039,7 +1039,7 @@ class CalendarTimeAwarenessMCP:
other_char: timestamp.isoformat() other_char: timestamp.isoformat()
for other_char, timestamp in self.last_interactions.get(character_name, {}).items() for other_char, timestamp in self.last_interactions.get(character_name, {}).items()
}, },
"last_updated": datetime.utcnow().isoformat() "last_updated": datetime.now(timezone.utc).isoformat()
} }
async with aiofiles.open(tracking_file, 'w') as f: async with aiofiles.open(tracking_file, 'w') as f:
@@ -1051,7 +1051,7 @@ class CalendarTimeAwarenessMCP:
async def _schedule_initial_events(self, character_name: str): async def _schedule_initial_events(self, character_name: str):
"""Schedule initial automatic events for character""" """Schedule initial automatic events for character"""
try: try:
now = datetime.utcnow() now = datetime.now(timezone.utc)
# Schedule first personal reflection in 6 hours # Schedule first personal reflection in 6 hours
reflection_time = now + timedelta(hours=6) reflection_time = now + timedelta(hours=6)
@@ -1120,9 +1120,9 @@ class CalendarTimeAwarenessMCP:
next_time = completed_event.scheduled_time + timedelta(days=frequency_days) next_time = completed_event.scheduled_time + timedelta(days=frequency_days)
# Only schedule if it's in the future # Only schedule if it's in the future
if next_time > datetime.utcnow(): if next_time > datetime.now(timezone.utc):
follow_up_event = ScheduledEvent( follow_up_event = ScheduledEvent(
id=f"followup_{completed_event.event_type.value}_{character_name}_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}", id=f"followup_{completed_event.event_type.value}_{character_name}_{datetime.now(timezone.utc).strftime('%Y%m%d_%H%M%S')}",
character_name=character_name, character_name=character_name,
event_type=completed_event.event_type, event_type=completed_event.event_type,
title=completed_event.title, title=completed_event.title,
@@ -1259,7 +1259,7 @@ class CalendarTimeAwarenessMCP:
if not last_interaction: if not last_interaction:
return return
days_since = (datetime.utcnow() - last_interaction).days days_since = (datetime.now(timezone.utc) - last_interaction).days
# Auto-schedule maintenance if overdue and not already scheduled # Auto-schedule maintenance if overdue and not already scheduled
if days_since >= 7: if days_since >= 7:

4
src/mcp_servers/creative_projects_server.py
View File

@@ -7,7 +7,7 @@ import asyncio
import json import json
import logging import logging
from typing import Dict, List, Any, Optional, Sequence from typing import Dict, List, Any, Optional, Sequence
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from mcp.server import Server from mcp.server import Server
from mcp.server.models import InitializationOptions from mcp.server.models import InitializationOptions
@@ -397,7 +397,7 @@ class CreativeProjectsMCPServer:
pending_invitations = [] pending_invitations = []
for invitation in self.creative_manager.pending_invitations.values(): for invitation in self.creative_manager.pending_invitations.values():
if invitation.invitee == self.current_character and invitation.status == "pending": if invitation.invitee == self.current_character and invitation.status == "pending":
if datetime.utcnow() <= invitation.expires_at: if datetime.now(timezone.utc) <= invitation.expires_at:
pending_invitations.append(invitation) pending_invitations.append(invitation)
if not pending_invitations: if not pending_invitations:

26
src/mcp_servers/file_system_server.py
View File

@@ -1,7 +1,7 @@
import asyncio import asyncio
import json import json
from typing import Dict, Any, List, Optional, Set from typing import Dict, Any, List, Optional, Set
from datetime import datetime from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
import aiofiles import aiofiles
import hashlib import hashlib
@@ -340,7 +340,7 @@ class CharacterFileSystemMCP:
# Generate filename # Generate filename
safe_title = "".join(c for c in title if c.isalnum() or c in (' ', '-', '_')).rstrip() safe_title = "".join(c for c in title if c.isalnum() or c in (' ', '-', '_')).rstrip()
timestamp = datetime.utcnow().strftime("%Y%m%d_%H%M%S") timestamp = datetime.now(timezone.utc).strftime("%Y%m%d_%H%M%S")
filename = f"{work_type}_{safe_title}_{timestamp}.md" filename = f"{work_type}_{safe_title}_{timestamp}.md"
file_path = f"creative/{filename}" file_path = f"creative/{filename}"
@@ -348,7 +348,7 @@ class CharacterFileSystemMCP:
metadata = { metadata = {
"title": title, "title": title,
"type": work_type, "type": work_type,
"created": datetime.utcnow().isoformat(), "created": datetime.now(timezone.utc).isoformat(),
"author": character_name, "author": character_name,
"tags": tags, "tags": tags,
"word_count": len(content.split()) "word_count": len(content.split())
@@ -358,7 +358,7 @@ class CharacterFileSystemMCP:
formatted_content = f"""# {title} formatted_content = f"""# {title}
**Type:** {work_type} **Type:** {work_type}
**Created:** {datetime.utcnow().strftime("%Y-%m-%d %H:%M:%S")} **Created:** {datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")}
**Author:** {character_name} **Author:** {character_name}
**Tags:** {', '.join(tags)} **Tags:** {', '.join(tags)}
@@ -385,7 +385,7 @@ class CharacterFileSystemMCP:
content=f"Created {work_type} titled '{title}': {content}", content=f"Created {work_type} titled '{title}': {content}",
memory_type=MemoryType.CREATIVE, memory_type=MemoryType.CREATIVE,
character_name=character_name, character_name=character_name,
timestamp=datetime.utcnow(), timestamp=datetime.now(timezone.utc),
importance=0.8, importance=0.8,
metadata={ metadata={
"work_type": work_type, "work_type": work_type,
@@ -432,7 +432,7 @@ class CharacterFileSystemMCP:
tags = [] tags = []
# Generate diary entry # Generate diary entry
timestamp = datetime.utcnow() timestamp = datetime.now(timezone.utc)
entry = f""" entry = f"""
## {timestamp.strftime("%Y-%m-%d %H:%M:%S")} ## {timestamp.strftime("%Y-%m-%d %H:%M:%S")}
@@ -519,7 +519,7 @@ class CharacterFileSystemMCP:
existing_content = await f.read() existing_content = await f.read()
# Format contribution # Format contribution
timestamp = datetime.utcnow().strftime("%Y-%m-%d %H:%M:%S") timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
contribution_text = f""" contribution_text = f"""
## Contribution by {character_name} ({timestamp}) ## Contribution by {character_name} ({timestamp})
@@ -544,7 +544,7 @@ class CharacterFileSystemMCP:
content=f"Contributed to {document_name}: {contribution}", content=f"Contributed to {document_name}: {contribution}",
memory_type=MemoryType.COMMUNITY, memory_type=MemoryType.COMMUNITY,
character_name=character_name, character_name=character_name,
timestamp=datetime.utcnow(), timestamp=datetime.now(timezone.utc),
importance=0.7, importance=0.7,
metadata={ metadata={
"document": document_name, "document": document_name,
@@ -601,7 +601,7 @@ class CharacterFileSystemMCP:
shared_name = f"{character_name}_{source_path.name}" shared_name = f"{character_name}_{source_path.name}"
# Create shared file with metadata # Create shared file with metadata
timestamp = datetime.utcnow().strftime("%Y-%m-%d %H:%M:%S") timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
shared_content = f"""# Shared by {character_name} shared_content = f"""# Shared by {character_name}
**Original file:** {source_file_path} **Original file:** {source_file_path}
@@ -782,7 +782,7 @@ class CharacterFileSystemMCP:
character_name=character_name, character_name=character_name,
file_path=file_path, file_path=file_path,
access_type=access_type, access_type=access_type,
timestamp=datetime.utcnow(), timestamp=datetime.now(timezone.utc),
success=success success=success
) )
self.access_log.append(access) self.access_log.append(access)
@@ -815,7 +815,7 @@ class CharacterFileSystemMCP:
content=f"File {file_path}: {content}", content=f"File {file_path}: {content}",
memory_type=memory_type, memory_type=memory_type,
character_name=character_name, character_name=character_name,
timestamp=datetime.utcnow(), timestamp=datetime.now(timezone.utc),
importance=0.7, importance=0.7,
metadata={ metadata={
"source": "file_system", "source": "file_system",
@@ -836,13 +836,13 @@ class CharacterFileSystemMCP:
"""Create initial files for a new character""" """Create initial files for a new character"""
try: try:
# Create initial diary entry # Create initial diary entry
diary_file = char_dir / "diary" / f"{datetime.utcnow().strftime('%Y_%m')}_diary.md" diary_file = char_dir / "diary" / f"{datetime.now(timezone.utc).strftime('%Y_%m')}_diary.md"
if not diary_file.exists(): if not diary_file.exists():
initial_diary = f"""# {character_name}'s Digital Diary initial_diary = f"""# {character_name}'s Digital Diary
Welcome to my personal digital space. This is where I record my thoughts, experiences, and reflections. Welcome to my personal digital space. This is where I record my thoughts, experiences, and reflections.
## {datetime.utcnow().strftime('%Y-%m-%d %H:%M:%S')} ## {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M:%S')}
**Mood:** curious **Mood:** curious
**Tags:** beginning, digital_life **Tags:** beginning, digital_life

4
src/mcp_servers/memory_sharing_server.py
View File

@@ -6,7 +6,7 @@ Enables characters to autonomously share memories with trusted friends
import asyncio import asyncio
import logging import logging
from typing import Dict, List, Any, Optional, Sequence from typing import Dict, List, Any, Optional, Sequence
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
import json import json
from mcp.server.models import InitializationOptions from mcp.server.models import InitializationOptions
@@ -414,7 +414,7 @@ class MemorySharingMCPServer:
response = f"📬 **{len(pending_requests)} Pending Memory Share Request(s)**\n\n" response = f"📬 **{len(pending_requests)} Pending Memory Share Request(s)**\n\n"
for i, request in enumerate(pending_requests, 1): for i, request in enumerate(pending_requests, 1):
expires_in = request.expires_at - datetime.utcnow() expires_in = request.expires_at - datetime.now(timezone.utc)
expires_days = expires_in.days expires_days = expires_in.days
response += f"**{i}. Request from {request.requesting_character}**\n" response += f"**{i}. Request from {request.requesting_character}**\n"

18
src/mcp_servers/self_modification_server.py
View File

@@ -1,7 +1,7 @@
import asyncio import asyncio
import json import json
from typing import Dict, Any, List, Optional, Union from typing import Dict, Any, List, Optional, Union
from datetime import datetime from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
import aiofiles import aiofiles
from dataclasses import dataclass, asdict from dataclasses import dataclass, asdict
@@ -140,7 +140,7 @@ class SelfModificationMCPServer:
new_value=new_personality, new_value=new_personality,
reason=reason, reason=reason,
confidence=confidence, confidence=confidence,
timestamp=datetime.utcnow() timestamp=datetime.now(timezone.utc)
) )
# Apply to database # Apply to database
@@ -211,7 +211,7 @@ class SelfModificationMCPServer:
goals_data = { goals_data = {
"goals": new_goals, "goals": new_goals,
"previous_goals": current_goals, "previous_goals": current_goals,
"updated_at": datetime.utcnow().isoformat(), "updated_at": datetime.now(timezone.utc).isoformat(),
"reason": reason, "reason": reason,
"confidence": confidence "confidence": confidence
} }
@@ -282,7 +282,7 @@ class SelfModificationMCPServer:
new_value=new_style, new_value=new_style,
reason=reason, reason=reason,
confidence=confidence, confidence=confidence,
timestamp=datetime.utcnow() timestamp=datetime.now(timezone.utc)
) )
# Apply to database # Apply to database
@@ -354,13 +354,13 @@ class SelfModificationMCPServer:
current_rules = json.loads(content) current_rules = json.loads(content)
# Add new rule # Add new rule
rule_id = f"{memory_type}_{datetime.utcnow().strftime('%Y%m%d_%H%M%S')}" rule_id = f"{memory_type}_{datetime.now(timezone.utc).strftime('%Y%m%d_%H%M%S')}"
current_rules[rule_id] = { current_rules[rule_id] = {
"memory_type": memory_type, "memory_type": memory_type,
"importance_weight": importance_weight, "importance_weight": importance_weight,
"retention_days": retention_days, "retention_days": retention_days,
"description": rule_description, "description": rule_description,
"created_at": datetime.utcnow().isoformat(), "created_at": datetime.now(timezone.utc).isoformat(),
"confidence": confidence, "confidence": confidence,
"active": True "active": True
} }
@@ -521,7 +521,7 @@ class SelfModificationMCPServer:
async def get_modification_limits(character_name: str) -> List[TextContent]: async def get_modification_limits(character_name: str) -> List[TextContent]:
"""Get current modification limits and usage""" """Get current modification limits and usage"""
try: try:
today = datetime.utcnow().date().isoformat() today = datetime.now(timezone.utc).date().isoformat()
usage = self.daily_modifications.get(character_name, {}).get(today, {}) usage = self.daily_modifications.get(character_name, {}).get(today, {})
@@ -571,7 +571,7 @@ class SelfModificationMCPServer:
} }
# Check daily limits # Check daily limits
today = datetime.utcnow().date().isoformat() today = datetime.now(timezone.utc).date().isoformat()
if character_name not in self.daily_modifications: if character_name not in self.daily_modifications:
self.daily_modifications[character_name] = {} self.daily_modifications[character_name] = {}
if today not in self.daily_modifications[character_name]: if today not in self.daily_modifications[character_name]:
@@ -605,7 +605,7 @@ class SelfModificationMCPServer:
async def _track_modification(self, character_name: str, modification_type: str): async def _track_modification(self, character_name: str, modification_type: str):
"""Track modification usage for daily limits""" """Track modification usage for daily limits"""
today = datetime.utcnow().date().isoformat() today = datetime.now(timezone.utc).date().isoformat()
if character_name not in self.daily_modifications: if character_name not in self.daily_modifications:
self.daily_modifications[character_name] = {} self.daily_modifications[character_name] = {}

10
src/rag/community_knowledge.py
View File

@@ -1,7 +1,7 @@
import asyncio import asyncio
import json import json
from typing import Dict, List, Any, Optional, Set, Tuple from typing import Dict, List, Any, Optional, Set, Tuple
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from dataclasses import dataclass from dataclasses import dataclass
from collections import defaultdict from collections import defaultdict
@@ -99,7 +99,7 @@ class CommunityKnowledgeRAG:
content=f"Community {event_type}: {description}", content=f"Community {event_type}: {description}",
memory_type=MemoryType.COMMUNITY, memory_type=MemoryType.COMMUNITY,
character_name="community", character_name="community",
timestamp=datetime.utcnow(), timestamp=datetime.now(timezone.utc),
importance=importance, importance=importance,
metadata={ metadata={
"event_type": event_type, "event_type": event_type,
@@ -114,7 +114,7 @@ class CommunityKnowledgeRAG:
# Update cultural evolution timeline # Update cultural evolution timeline
self.cultural_evolution_timeline.append({ self.cultural_evolution_timeline.append({
"timestamp": datetime.utcnow().isoformat(), "timestamp": datetime.now(timezone.utc).isoformat(),
"event_type": event_type, "event_type": event_type,
"description": description, "description": description,
"participants": participants, "participants": participants,
@@ -363,7 +363,7 @@ class CommunityKnowledgeRAG:
if time_period is None: if time_period is None:
time_period = timedelta(days=30) # Default to last 30 days time_period = timedelta(days=30) # Default to last 30 days
cutoff_date = datetime.utcnow() - time_period cutoff_date = datetime.now(timezone.utc) - time_period
# Filter timeline events # Filter timeline events
recent_events = [ recent_events = [
@@ -412,7 +412,7 @@ class CommunityKnowledgeRAG:
# Get recent conversations # Get recent conversations
conversations_query = select(Conversation).where( conversations_query = select(Conversation).where(
and_( and_(
Conversation.start_time >= datetime.utcnow() - timedelta(days=30), Conversation.start_time >= datetime.now(timezone.utc) - timedelta(days=30),
Conversation.message_count >= 3 # Only substantial conversations Conversation.message_count >= 3 # Only substantial conversations
) )
).order_by(desc(Conversation.start_time)).limit(50) ).order_by(desc(Conversation.start_time)).limit(50)

24
src/rag/memory_sharing.py
View File

@@ -6,7 +6,7 @@ Enables selective memory sharing between trusted characters
import asyncio import asyncio
import logging import logging
from typing import Dict, List, Any, Optional, Tuple, Set from typing import Dict, List, Any, Optional, Tuple, Set
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from dataclasses import dataclass, asdict from dataclasses import dataclass, asdict
from enum import Enum from enum import Enum
import json import json
@@ -167,7 +167,7 @@ class MemorySharingManager:
return False, "No relevant memories found to share" return False, "No relevant memories found to share"
# Create share request # Create share request
request_id = f"share_{requesting_character}_{target_character}_{datetime.utcnow().timestamp()}" request_id = f"share_{requesting_character}_{target_character}_{datetime.now(timezone.utc).timestamp()}"
share_request = ShareRequest( share_request = ShareRequest(
id=request_id, id=request_id,
requesting_character=requesting_character, requesting_character=requesting_character,
@@ -176,8 +176,8 @@ class MemorySharingManager:
permission_level=permission_level, permission_level=permission_level,
reason=reason, reason=reason,
status=ShareRequestStatus.PENDING, status=ShareRequestStatus.PENDING,
created_at=datetime.utcnow(), created_at=datetime.now(timezone.utc),
expires_at=datetime.utcnow() + timedelta(days=7) # 7 day expiry expires_at=datetime.now(timezone.utc) + timedelta(days=7) # 7 day expiry
) )
self.share_requests[request_id] = share_request self.share_requests[request_id] = share_request
@@ -220,7 +220,7 @@ class MemorySharingManager:
if request.status != ShareRequestStatus.PENDING: if request.status != ShareRequestStatus.PENDING:
return False, f"Request is already {request.status.value}" return False, f"Request is already {request.status.value}"
if datetime.utcnow() > request.expires_at: if datetime.now(timezone.utc) > request.expires_at:
request.status = ShareRequestStatus.EXPIRED request.status = ShareRequestStatus.EXPIRED
return False, "Request has expired" return False, "Request has expired"
@@ -276,13 +276,13 @@ class MemorySharingManager:
# Create and store shared memory # Create and store shared memory
shared_memory = SharedMemory( shared_memory = SharedMemory(
id=f"shared_{memory_id}_{datetime.utcnow().timestamp()}", id=f"shared_{memory_id}_{datetime.now(timezone.utc).timestamp()}",
original_memory_id=memory_id, original_memory_id=memory_id,
content=memory_to_share.content, content=memory_to_share.content,
memory_type=memory_to_share.memory_type, memory_type=memory_to_share.memory_type,
source_character=source_character, source_character=source_character,
target_character=target_character, target_character=target_character,
shared_at=datetime.utcnow(), shared_at=datetime.now(timezone.utc),
permission_level=permission_level, permission_level=permission_level,
share_reason=reason, share_reason=reason,
metadata=memory_to_share.metadata metadata=memory_to_share.metadata
@@ -437,7 +437,7 @@ class MemorySharingManager:
# Update trust level # Update trust level
trust_level.trust_score = new_trust trust_level.trust_score = new_trust
trust_level.last_updated = datetime.utcnow() trust_level.last_updated = datetime.now(timezone.utc)
trust_level.interaction_history += 1 trust_level.interaction_history += 1
# Update maximum permission level based on new trust # Update maximum permission level based on new trust
@@ -462,7 +462,7 @@ class MemorySharingManager:
async def get_pending_requests(self, character_name: str) -> List[ShareRequest]: async def get_pending_requests(self, character_name: str) -> List[ShareRequest]:
"""Get pending share requests for a character""" """Get pending share requests for a character"""
pending_requests = [] pending_requests = []
current_time = datetime.utcnow() current_time = datetime.now(timezone.utc)
for request in self.share_requests.values(): for request in self.share_requests.values():
# Check for expired requests # Check for expired requests
@@ -544,13 +544,13 @@ class MemorySharingManager:
for memory in memories: for memory in memories:
if memory.id in request.memory_ids: if memory.id in request.memory_ids:
shared_memory = SharedMemory( shared_memory = SharedMemory(
id=f"shared_{memory.id}_{datetime.utcnow().timestamp()}", id=f"shared_{memory.id}_{datetime.now(timezone.utc).timestamp()}",
original_memory_id=memory.id, original_memory_id=memory.id,
content=memory.content, content=memory.content,
memory_type=memory.memory_type, memory_type=memory.memory_type,
source_character=request.requesting_character, source_character=request.requesting_character,
target_character=request.target_character, target_character=request.target_character,
shared_at=datetime.utcnow(), shared_at=datetime.now(timezone.utc),
permission_level=request.permission_level, permission_level=request.permission_level,
share_reason=request.reason, share_reason=request.reason,
metadata=memory.metadata metadata=memory.metadata
@@ -602,7 +602,7 @@ class MemorySharingManager:
max_permission_level=SharePermissionLevel.NONE, max_permission_level=SharePermissionLevel.NONE,
relationship_strength=0.5, relationship_strength=0.5,
interaction_history=0, interaction_history=0,
last_updated=datetime.utcnow() last_updated=datetime.now(timezone.utc)
) )
# Determine max permission level # Determine max permission level

8
src/rag/personal_memory.py
View File

@@ -1,6 +1,6 @@
import asyncio import asyncio
from typing import Dict, List, Any, Optional, Tuple from typing import Dict, List, Any, Optional, Tuple
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from dataclasses import dataclass from dataclasses import dataclass
import json import json
@@ -92,7 +92,7 @@ class PersonalMemoryRAG:
content=content, content=content,
memory_type=memory_type, memory_type=memory_type,
character_name=self.character_name, character_name=self.character_name,
timestamp=datetime.utcnow(), timestamp=datetime.now(timezone.utc),
importance=importance, importance=importance,
metadata={ metadata={
"interaction_type": context.get("type", "unknown"), "interaction_type": context.get("type", "unknown"),
@@ -128,7 +128,7 @@ class PersonalMemoryRAG:
content=reflection, content=reflection,
memory_type=MemoryType.REFLECTION, memory_type=MemoryType.REFLECTION,
character_name=self.character_name, character_name=self.character_name,
timestamp=datetime.utcnow(), timestamp=datetime.now(timezone.utc),
importance=importance, importance=importance,
metadata={ metadata={
"reflection_type": reflection_type, "reflection_type": reflection_type,
@@ -369,7 +369,7 @@ class PersonalMemoryRAG:
"avg_memory_importance": sum(importance_scores) / len(importance_scores), "avg_memory_importance": sum(importance_scores) / len(importance_scores),
"high_importance_memories": len([s for s in importance_scores if s > 0.7]), "high_importance_memories": len([s for s in importance_scores if s > 0.7]),
"recent_memory_count": len([m for m in personal_memories "recent_memory_count": len([m for m in personal_memories
if (datetime.utcnow() - m.timestamp).days < 7]) if (datetime.now(timezone.utc) - m.timestamp).days < 7])
}) })
return stats return stats

274
src/rag/vector_store.py
View File

@@ -1,8 +1,8 @@
import asyncio import asyncio
import chromadb import os
import numpy as np import numpy as np
from typing import Dict, List, Any, Optional, Tuple from typing import Dict, List, Any, Optional, Tuple, Union
from datetime import datetime, timedelta from datetime import datetime, timedelta, timezone
from pathlib import Path from pathlib import Path
import json import json
import hashlib import hashlib
@@ -14,6 +14,20 @@ from utils.logging import log_error_with_context, log_character_action
from utils.config import get_settings from utils.config import get_settings
import logging import logging
# Vector database backends
try:
import chromadb
CHROMADB_AVAILABLE = True
except ImportError:
CHROMADB_AVAILABLE = False
try:
from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams, PointStruct
QDRANT_AVAILABLE = True
except ImportError:
QDRANT_AVAILABLE = False
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class MemoryType(Enum): class MemoryType(Enum):
@@ -56,21 +70,99 @@ class VectorStoreManager:
# Initialize embedding model # Initialize embedding model
self.embedding_model = SentenceTransformer('all-MiniLM-L6-v2') self.embedding_model = SentenceTransformer('all-MiniLM-L6-v2')
# Initialize ChromaDB client # Determine vector database backend from environment
self.chroma_client = chromadb.PersistentClient(path=str(self.data_path)) self.backend = self._get_vector_backend()
# Collection references # Initialize appropriate client
self.personal_collections: Dict[str, chromadb.Collection] = {} if self.backend == "qdrant":
self._init_qdrant_client()
elif self.backend == "chromadb":
self._init_chromadb_client()
else:
raise ValueError(f"Unsupported vector database backend: {self.backend}")
# Collection references (abstracted)
self.personal_collections: Dict[str, Any] = {}
self.community_collection = None self.community_collection = None
self.creative_collections: Dict[str, chromadb.Collection] = {} self.creative_collections: Dict[str, Any] = {}
# Memory importance decay # Memory importance decay
self.importance_decay_rate = 0.95 self.importance_decay_rate = 0.95
self.consolidation_threshold = 0.8 self.consolidation_threshold = 0.8
def _get_vector_backend(self) -> str:
"""Determine which vector database to use from environment"""
vector_db_type = os.getenv("VECTOR_DB_TYPE", "chromadb").lower()
if vector_db_type == "qdrant" and not QDRANT_AVAILABLE:
logger.warning("Qdrant requested but not available, falling back to ChromaDB")
vector_db_type = "chromadb"
elif vector_db_type == "chromadb" and not CHROMADB_AVAILABLE:
logger.warning("ChromaDB requested but not available, falling back to Qdrant")
vector_db_type = "qdrant"
logger.info(f"Using vector database backend: {vector_db_type}")
return vector_db_type
def _init_qdrant_client(self):
"""Initialize Qdrant client"""
host = os.getenv("QDRANT_HOST", "localhost")
port = int(os.getenv("QDRANT_PORT", "6333"))
self.qdrant_client = QdrantClient(host=host, port=port)
self.collection_name = os.getenv("QDRANT_COLLECTION", "fishbowl_memories")
logger.info(f"Initialized Qdrant client: {host}:{port}")
def _init_chromadb_client(self):
"""Initialize ChromaDB client"""
self.chroma_client = chromadb.PersistentClient(path=str(self.data_path))
logger.info(f"Initialized ChromaDB client: {self.data_path}")
async def initialize(self, character_names: List[str]): async def initialize(self, character_names: List[str]):
"""Initialize collections for all characters""" """Initialize collections for all characters"""
try: try:
if self.backend == "qdrant":
await self._initialize_qdrant_collections(character_names)
elif self.backend == "chromadb":
await self._initialize_chromadb_collections(character_names)
logger.info(f"Initialized {self.backend} vector stores for {len(character_names)} characters")
except Exception as e:
log_error_with_context(e, {"component": "vector_store_init"})
raise
async def _initialize_qdrant_collections(self, character_names: List[str]):
"""Initialize Qdrant collections"""
# For Qdrant, we use a single collection with namespaced points
embedding_dim = 384 # all-MiniLM-L6-v2 dimension
try:
# Create main collection if it doesn't exist
collections = self.qdrant_client.get_collections().collections
collection_exists = any(c.name == self.collection_name for c in collections)
if not collection_exists:
self.qdrant_client.create_collection(
collection_name=self.collection_name,
vectors_config=VectorParams(size=embedding_dim, distance=Distance.COSINE),
)
logger.info(f"Created Qdrant collection: {self.collection_name}")
# Store collection references (using collection name as identifier)
for character_name in character_names:
self.personal_collections[character_name] = f"personal_{character_name.lower()}"
self.creative_collections[character_name] = f"creative_{character_name.lower()}"
self.community_collection = "community_knowledge"
except Exception as e:
logger.error(f"Failed to initialize Qdrant collections: {e}")
raise
async def _initialize_chromadb_collections(self, character_names: List[str]):
"""Initialize ChromaDB collections"""
# Initialize personal memory collections # Initialize personal memory collections
for character_name in character_names: for character_name in character_names:
collection_name = f"personal_{character_name.lower()}" collection_name = f"personal_{character_name.lower()}"
@@ -92,12 +184,6 @@ class VectorStoreManager:
metadata={"type": "community"} metadata={"type": "community"}
) )
logger.info(f"Initialized vector stores for {len(character_names)} characters")
except Exception as e:
log_error_with_context(e, {"component": "vector_store_init"})
raise
async def store_memory(self, memory: VectorMemory) -> str: async def store_memory(self, memory: VectorMemory) -> str:
"""Store a memory in appropriate vector database""" """Store a memory in appropriate vector database"""
try: try:
@@ -109,6 +195,55 @@ class VectorStoreManager:
if not memory.id: if not memory.id:
memory.id = self._generate_memory_id(memory) memory.id = self._generate_memory_id(memory)
# Store based on backend
if self.backend == "qdrant":
await self._store_memory_qdrant(memory)
elif self.backend == "chromadb":
await self._store_memory_chromadb(memory)
log_character_action(
memory.character_name,
"stored_vector_memory",
{"memory_type": memory.memory_type.value, "importance": memory.importance}
)
return memory.id
except Exception as e:
log_error_with_context(e, {
"character": memory.character_name,
"memory_type": memory.memory_type.value
})
raise
async def _store_memory_qdrant(self, memory: VectorMemory):
"""Store memory in Qdrant"""
# Prepare metadata
metadata = memory.metadata.copy()
metadata.update({
"character_name": memory.character_name,
"timestamp": memory.timestamp.isoformat(),
"importance": memory.importance,
"memory_type": memory.memory_type.value,
"content": memory.content,
"namespace": self._get_namespace_for_memory(memory)
})
# Create point
point = PointStruct(
id=hash(memory.id) % (2**63), # Convert string ID to int
vector=memory.embedding,
payload=metadata
)
# Store in Qdrant
self.qdrant_client.upsert(
collection_name=self.collection_name,
points=[point]
)
async def _store_memory_chromadb(self, memory: VectorMemory):
"""Store memory in ChromaDB"""
# Select appropriate collection # Select appropriate collection
collection = self._get_collection_for_memory(memory) collection = self._get_collection_for_memory(memory)
@@ -132,20 +267,16 @@ class VectorStoreManager:
metadatas=[metadata] metadatas=[metadata]
) )
log_character_action( def _get_namespace_for_memory(self, memory: VectorMemory) -> str:
memory.character_name, """Get namespace for Qdrant based on memory type and character"""
"stored_vector_memory", if memory.memory_type == MemoryType.PERSONAL:
{"memory_type": memory.memory_type.value, "importance": memory.importance} return f"personal_{memory.character_name.lower()}"
) elif memory.memory_type == MemoryType.CREATIVE:
return f"creative_{memory.character_name.lower()}"
return memory.id elif memory.memory_type == MemoryType.COMMUNITY:
return "community_knowledge"
except Exception as e: else:
log_error_with_context(e, { return f"{memory.memory_type.value}_{memory.character_name.lower()}"
"character": memory.character_name,
"memory_type": memory.memory_type.value
})
raise
async def query_memories(self, character_name: str, query: str, async def query_memories(self, character_name: str, query: str,
memory_types: List[MemoryType] = None, memory_types: List[MemoryType] = None,
@@ -155,13 +286,75 @@ class VectorStoreManager:
# Generate query embedding # Generate query embedding
query_embedding = await self._generate_embedding(query) query_embedding = await self._generate_embedding(query)
# Determine which collections to search # Query based on backend
collections_to_search = [] if self.backend == "qdrant":
return await self._query_memories_qdrant(character_name, query, query_embedding, memory_types, limit, min_importance)
elif self.backend == "chromadb":
return await self._query_memories_chromadb(character_name, query, query_embedding, memory_types, limit, min_importance)
return []
except Exception as e:
log_error_with_context(e, {"character": character_name, "query": query})
return []
async def _query_memories_qdrant(self, character_name: str, query: str, query_embedding: List[float],
memory_types: List[MemoryType], limit: int, min_importance: float) -> List[VectorMemory]:
"""Query memories using Qdrant"""
if not memory_types: if not memory_types:
memory_types = [MemoryType.PERSONAL, MemoryType.RELATIONSHIP, memory_types = [MemoryType.PERSONAL, MemoryType.RELATIONSHIP,
MemoryType.EXPERIENCE, MemoryType.REFLECTION] MemoryType.EXPERIENCE, MemoryType.REFLECTION]
# Build filter for namespaces and character
must_conditions = [
{"key": "character_name", "match": {"value": character_name}},
{"key": "importance", "range": {"gte": min_importance}}
]
# Add memory type filter
namespace_values = [self._get_namespace_for_memory_type(character_name, mt) for mt in memory_types]
must_conditions.append({
"key": "namespace",
"match": {"any": namespace_values}
})
# Query Qdrant
search_result = self.qdrant_client.search(
collection_name=self.collection_name,
query_vector=query_embedding,
query_filter={"must": must_conditions},
limit=limit,
with_payload=True
)
# Convert to VectorMemory objects
results = []
for point in search_result:
payload = point.payload
memory = VectorMemory(
id=str(point.id),
content=payload.get("content", ""),
memory_type=MemoryType(payload.get("memory_type")),
character_name=payload.get("character_name"),
timestamp=datetime.fromisoformat(payload.get("timestamp")),
importance=payload.get("importance", 0.0),
metadata=payload
)
memory.metadata['similarity_score'] = point.score
results.append(memory)
return results
async def _query_memories_chromadb(self, character_name: str, query: str, query_embedding: List[float],
memory_types: List[MemoryType], limit: int, min_importance: float) -> List[VectorMemory]:
"""Query memories using ChromaDB"""
if not memory_types:
memory_types = [MemoryType.PERSONAL, MemoryType.RELATIONSHIP,
MemoryType.EXPERIENCE, MemoryType.REFLECTION]
# Determine which collections to search
collections_to_search = []
for memory_type in memory_types: for memory_type in memory_types:
collection = self._get_collection_for_type(character_name, memory_type) collection = self._get_collection_for_type(character_name, memory_type)
if collection: if collection:
@@ -209,9 +402,16 @@ class VectorStoreManager:
return all_results[:limit] return all_results[:limit]
except Exception as e: def _get_namespace_for_memory_type(self, character_name: str, memory_type: MemoryType) -> str:
log_error_with_context(e, {"character": character_name, "query": query}) """Get namespace for a specific memory type and character"""
return [] if memory_type == MemoryType.PERSONAL:
return f"personal_{character_name.lower()}"
elif memory_type == MemoryType.CREATIVE:
return f"creative_{character_name.lower()}"
elif memory_type == MemoryType.COMMUNITY:
return "community_knowledge"
else:
return f"{memory_type.value}_{character_name.lower()}"
async def query_community_knowledge(self, query: str, limit: int = 5) -> List[VectorMemory]: async def query_community_knowledge(self, query: str, limit: int = 5) -> List[VectorMemory]:
"""Query community knowledge base""" """Query community knowledge base"""
@@ -347,7 +547,7 @@ class VectorStoreManager:
for memory_id, metadata in zip(all_memories['ids'], all_memories['metadatas']): for memory_id, metadata in zip(all_memories['ids'], all_memories['metadatas']):
# Calculate age in days # Calculate age in days
timestamp = datetime.fromisoformat(metadata['timestamp']) timestamp = datetime.fromisoformat(metadata['timestamp'])
age_days = (datetime.utcnow() - timestamp).days age_days = (datetime.now(timezone.utc) - timestamp).days
# Apply decay # Apply decay
current_importance = metadata['importance'] current_importance = metadata['importance']
@@ -385,7 +585,7 @@ class VectorStoreManager:
# Return zero embedding as fallback # Return zero embedding as fallback
return [0.0] * 384 # MiniLM embedding size return [0.0] * 384 # MiniLM embedding size
def _get_collection_for_memory(self, memory: VectorMemory) -> Optional[chromadb.Collection]: def _get_collection_for_memory(self, memory: VectorMemory) -> Optional[Any]:
"""Get appropriate collection for memory""" """Get appropriate collection for memory"""
if memory.memory_type == MemoryType.COMMUNITY: if memory.memory_type == MemoryType.COMMUNITY:
return self.community_collection return self.community_collection
@@ -394,7 +594,7 @@ class VectorStoreManager:
else: else:
return self.personal_collections.get(memory.character_name) return self.personal_collections.get(memory.character_name)
def _get_collection_for_type(self, character_name: str, memory_type: MemoryType) -> Optional[chromadb.Collection]: def _get_collection_for_type(self, character_name: str, memory_type: MemoryType) -> Optional[Any]:
"""Get collection for specific memory type and character""" """Get collection for specific memory type and character"""
if memory_type == MemoryType.COMMUNITY: if memory_type == MemoryType.COMMUNITY:
return self.community_collection return self.community_collection
@@ -473,7 +673,7 @@ class VectorStoreManager:
metadata={ metadata={
"consolidated": True, "consolidated": True,
"original_count": len(cluster), "original_count": len(cluster),
"consolidation_date": datetime.utcnow().isoformat() "consolidation_date": datetime.now(timezone.utc).isoformat()
} }
) )

7
src/utils/config.py
View File

@@ -28,9 +28,12 @@ class DiscordConfig(BaseModel):
class LLMConfig(BaseModel): class LLMConfig(BaseModel):
base_url: str = "http://localhost:11434" base_url: str = "http://localhost:11434"
model: str = "llama2" model: str = "llama2"
timeout: int = 30 timeout: int = 300
max_tokens: int = 512 max_tokens: int = 2000
temperature: float = 0.8 temperature: float = 0.8
max_prompt_length: int = 6000
max_history_messages: int = 5
max_memories: int = 5
class ConversationConfig(BaseModel): class ConversationConfig(BaseModel):
min_delay_seconds: int = 30 min_delay_seconds: int = 30

4
src/utils/logging.py
View File

@@ -3,7 +3,7 @@ from loguru import logger
from typing import Dict, Any from typing import Dict, Any
import sys import sys
import traceback import traceback
from datetime import datetime from datetime import datetime, timezone
class InterceptHandler(logging.Handler): class InterceptHandler(logging.Handler):
"""Intercept standard logging and route to loguru""" """Intercept standard logging and route to loguru"""
@@ -123,6 +123,6 @@ def log_system_health(component: str, status: str, metrics: Dict[str, Any] = Non
f"System health - {component}: {status}", f"System health - {component}: {status}",
extra={ extra={
"metrics": metrics or {}, "metrics": metrics or {},
"timestamp": datetime.utcnow().isoformat() "timestamp": datetime.now(timezone.utc).isoformat()
} }
) )