`ac30b95`

Raise num_ctx to 16K and lower compaction threshold to 12K to prevent context overflow in multi-turn sessions

Authored by

espadonne 1 month ago

Status	File	+	-
M	`src/loader/agent/loop.py`	1	1
M	`src/loader/llm/ollama.py`	1	1

src/loader/agent/loop.pymodified

      workflow_mode_override: str | None = None
      stream: bool = True  # Stream LLM responses for real-time output
      session_rotate_after_bytes: int = 256 * 1024
 -    session_auto_compaction_input_tokens_threshold: int = 100_000
 +    session_auto_compaction_input_tokens_threshold: int = 12_000  # ~75% of default 16K context
      session_compaction_keep_last_messages: int = 4
      # Reasoning stages configuration

src/loader/llm/ollama.pymodified

          base_url: str = "http://localhost:11434",
          timeout: float | None = None,
          force_react: bool = False,
 -        num_ctx: int = 8192,  # Reasonable context, not too slow
 +        num_ctx: int = 16384,  # 16K context; most models support 32K+
          num_gpu: int = -1,  # Use all GPU layers by default (fast)
      ):
          self.model = model