AI Agent App

An integrated framework for building AI chat agents with support for multiple LLM providers, conversation memory, knowledge bases (RAG), and multi-agent handoffs.

Key Features

• Dual-Provider Support — Build agents using OpenAI (GPT-5, o3) or Claude (Sonnet, Opus, Haiku)
• Unified Interface — Both providers share identical APIs (process_chat, build_messages, create_new_thread)
• Conversation Memory — Automatic persistence across sessions with thread-based isolation
• Knowledge Base (RAG) — Agentic filesystem search over your documents (no embeddings)
• Agent Handoffs — Delegate tasks to specialized sub-agents
• MCP Database Access — Query models through Model Context Protocol
• Streaming Responses — Real-time token streaming via async generators
• Multi-modal Support — Process PDFs and images alongside text

---

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        Your Application                          │
└─────────────────────────────────────────────────────────────────┘
                                │
                                ▼
┌─────────────────────────────────────────────────────────────────┐
│                         Agent Model                              │
│  (Database configuration: system prompt, model, features)        │
└─────────────────────────────────────────────────────────────────┘
                                │
                    ┌───────────┴───────────┐
                    ▼                       ▼
        ┌───────────────────┐   ┌───────────────────┐
        │  BaseOpenAIAgent  │   │  BaseClaudeAgent  │
        │  (OpenAI SDK)     │   │  (Claude SDK)     │
        └───────────────────┘   └───────────────────┘
                    │                       │
                    └───────────┬───────────┘
                                ▼
        ┌───────────────────────────────────────────┐
        │              Core Services                 │
        │  • MemoryStoreService (conversation)       │
        │  • Knowledge base files (filesystem search)│
        └───────────────────────────────────────────┘

---

Quick Start

import uuid
import asyncio
from grit.agent.dataclasses import AgentConfig
from grit.agent.claude_agent import BaseClaudeAgent

config = AgentConfig(
    id="customer-support-agent",
    label="Customer Support Agent",
    description="A minimal customer support agent",
    model_name="claude-haiku-4-5",
    agent_class="grit.agent.claude_agent.BaseClaudeAgent",
    record_usage_for_payment=False,
)

class CustomerSupportAgent(BaseClaudeAgent):
    """A simple agent that provides instructions without a database lookup."""

    def get_agent_instructions(self) -> str:
        return "You are a helpful customer support assistant."

async def main():
    agent = await CustomerSupportAgent.create(config=config)
    chunks = []
    async for chunk in agent.process_chat(
        user=user_example,
        thread_id=str(uuid.uuid4()),
        new_message="Hello, I need help with my order",
        data_type="text",
    ):
        chunks.append(chunk)
    return "".join(chunks)

response = asyncio.run(main())

Note:

• user_example is a User instance representing the authenticated user whose conversation is being processed. In your application, pass the currently logged-in user (e.g. request.user).
• You can also create agents via the Admin panel — the Agent model stores config in the database and provides .get_config() to build an AgentConfig.

---

Agent Providers

The SDK supports two LLM providers with identical interfaces. The provider is auto-detected from the model_name in your configuration.

Claude Agents

from grit.agent.claude_agent import BaseClaudeAgent

agent = await BaseClaudeAgent.create(config=config)

Supported Models:

• claude-opus-4-6 — Highest capability
• claude-sonnet-4-6 — Balanced performance (default)
• claude-haiku-4-6 — Fast responses

Configuration:

# In Agent metadata
{
    "model_name": "claude-sonnet-4-5"
}

OpenAI Agents

Supported Models:

• gpt-5.4 — Latest model
• gpt-5 — Fast, cost-effective (default)
• o3 — Reasoning models

from grit.agent.openai_agent import BaseOpenAIAgent

agent = await BaseOpenAIAgent.create(config=config)

Configuration:

# In Agent metadata
{
    "model_name": "gpt-5",
    "enable_web_search": True,
    "reasoning_effort": "medium"
}

---

Core Features

Conversation Memory

Conversations are automatically persisted per user and thread. The MemoryStoreService handles storage using the ORM.

# Memory is managed automatically by process_chat()
# Each message is persisted immediately to prevent race conditions

# To access memory directly:
from grit.agent.store import MemoryStoreService

memory_service = MemoryStoreService()
namespace = ("memories", str(user.id))

# Get conversation history
memory = memory_service.get_memory(namespace, thread_id)
if memory:
    history = memory.value['conversation_history']

# List recent conversations (up to 20)
recent = memory_service.list_memories(namespace)

Thread Isolation: Each thread_id maintains its own conversation history. Create new threads with agent.create_new_thread(session_key).

Knowledge Base (RAG)

Give an agent reference documents it can consult while answering. Documents are stored as plain text in the KnowledgeBaseFile table (the durable source of truth) and, at chat time, materialized to the local filesystem so the agent can search them with ordinary file tools — the same "agentic search over a filesystem" approach Claude Code uses. There are no embeddings or vector similarity search.

1. Enable Knowledge Base

   # In Agent metadata
   {
       "enable_knowledge_base": True
   }
   

2. Link Knowledge Bases Associate KnowledgeBase records with your Agent via the knowledge_bases many-to-many relationship (e.g. in the Admin).

3. Add Documents

   Sync a GitHub repository folder into a knowledge base. Configure a DataSource on the KnowledgeBase (type: "GITHUB" with a source_config of owner/repo/branch and crawler_config.inclusion_prefixes), then run a full sync:

   from grit.agent.knowledge_base import KnowledgeBaseClient
   
   # Pulls every knowledge base's GitHub data sources into KnowledgeBaseFile
   # rows as plain text. Full sync: files removed upstream are deleted here too.
   KnowledgeBaseClient().sync_data_sources_to_knowledge_files()
   

   Or add a single document directly:

   from grit.agent.knowledge_fs import upsert_knowledge_file
   
   upsert_knowledge_file(
       knowledge_base_id=str(knowledge_base.id),
       path="docs/faq.md",
       content="Your document content here...",
   )
   

   1. Automatic Retrieval When enable_knowledge_base is True, the agent automatically:
   • Materializes each linked knowledge base to <KB_ROOT>/<kb_id>/ on disk (cached per knowledge base, rebuilt when its files change)
   • Searches those files itself — Claude agents use the native Read, Grep, and Glob tools; OpenAI agents get equivalent list_knowledge_files, read_knowledge_file, and search_knowledge function tools
   • Is instructed to cite the file paths it relied on

   The on-disk copy is disposable scratch — durability lives in the database — so an ephemeral container simply rematerializes on its next cold start.

Web Search

The enable_web_search flag gives an agent a web search tool for retrieving live, external information. Availability depends on the provider:

Provider	Web search
OpenAI	Supported
Claude — direct Anthropic API	Supported
Claude — via AWS Bedrock (model_provider='bedrock')	Not supported

Bedrock limitation: any Bedrock-provider agent with enable_web_search=True fails on its first streamed turn. Web search depends on Anthropic's server-side web_search tool, which is not natively supported when Claude is served through AWS Bedrock — Bedrock rejects the tool as soon as the stream opens, so the turn errors before producing any output.

To give a Bedrock agent external context, use one of these options instead:

• Add a knowledge base (recommended). Populate a Knowledge Base — for example, synced from a GitHub repository — so the agent retrieves what it needs from your own documents. The agent stays entirely on Bedrock.
• Route the agent through the direct Claude API. Remove model_provider='bedrock' so the agent runs on the Anthropic API, where web_search is supported. This takes the agent off Bedrock, so it is not a 100% Bedrock-compliant solution — and Bedrock compliance is usually the reason for choosing it in the first place.

Agent Handoffs

Delegate conversations to specialized sub-agents when specific expertise is needed.

1. Configure Sub-Agents In the Admin, add agents to the sub_agents relationship on your main agent.

2. Handoff Behavior The SDK automatically:

   • Detects when a handoff is appropriate
   • Transfers conversation history to the sub-agent
   • Updates the current_agent_id in memory

   # Handoff instructions are auto-generated in the system prompt:
   # "You can transfer conversations to specialized agents..."
   # Available agents:
   # - Billing Support: Handles payment and invoice questions
   # - Technical Support: Handles product technical issues
   

   1. Sub-Agent Configuration Each sub-agent can have its own: - Model (different providers allowed) - System prompt - Knowledge bases - Tools

MCP Database Access

User-mode agents can query models through Model Context Protocol (MCP).

1. Use User-Mode Agent

   from grit.agent.openai_agent import BaseOpenAIUserModeAgent
   from grit.agent.claude_agent import BaseClaudeUserModeAgent
   
   # Pass request context for authentication
   agent = BaseOpenAIUserModeAgent(config=config, request=request)
   await agent.initialize()
   

2. Register Models for MCP. Models must have a scoped manager to be queryable:

   from grit.agent.mcp_server import mcp_registry
   
   # Register in your app's ready() method
   mcp_registry.register(YourModel)
   

3. Available Operations The agent gains an mcp_query tool with these operations:

   • list — Paginated listing with optional filters
   • retrieve — Get single record by primary key
   • search — Full-text search across specified fields

---

Configuration Reference

AgentConfig Fields

Field	Type	Description
id	str	Unique identifier (UUID)
label	str	Display name
description	str	Agent description
model_name	str	LLM model identifier
enable_web_search	bool	Enable web search tool; supported on OpenAI and the direct Claude API, not on AWS Bedrock (see Web Search)
enable_knowledge_base	bool	Enable RAG from linked knowledge bases
knowledge_bases	list	Linked KnowledgeBase records
reasoning_effort	str	For o1/o3 models: "low", "medium", "high"

Agent Metadata Fields

Store these in the metadata JSONField on the Agent model:

{
    "model_name": "gpt-5",
    "description": "Customer support assistant",
    "enable_web_search": True,
    "enable_knowledge_base": False,
    "agent_class": "grit.agent.openai_agent.BaseOpenAIAgent",
    "reasoning_effort": "medium",
    "suggested_messages": [
        "How can I track my order?",
        "I need to return a product"
    ],
    "overview_html": "<p>I help with customer inquiries.</p>",
    "tags": {"Type": ["support", "public"]}
}

Extending Agents

Create custom agent classes by inheriting from the base classes:

from grit.agent.openai_agent import BaseOpenAIAgent

class CustomAgent(BaseOpenAIAgent):
    def get_agent_instructions_context(self) -> dict:
        """Add custom context variables for prompt templates."""
        return {
            "custom_data": self.fetch_custom_data()
        }

    def _build_tools(self):
        """Add custom tools to the agent."""
        tools = super()._build_tools()
        tools.append(self.my_custom_tool)
        return tools

    def on_agent_end(self, user_id, thread_id, new_message, final_output):
        """Hook for post-processing after response completes."""
        super().on_agent_end(user_id, thread_id, new_message, final_output)
        self.log_interaction(user_id, final_output)