Architecture

Memory store (Chroma) with reasoning brain (Claude) and self-audit layer with reflexion.

Case study

HyperCoder - AI Coding Agent

Advanced AI coding agent with 1-shot learning, retrieval-locked generation, and self-audit mechanisms targeting ≤1% hallucination rate.

Overview

HyperCoder is an experimental AI coding assistant that combines memory-augmented generation, multi-model reasoning, and reflexion-based self-correction to minimize hallucinations. Unlike traditional code generation models that often "hallucinate" non-existent APIs or incorrect syntax, HyperCoder verifies all code against a vector-indexed codebase and uses self-audit layers to catch errors before execution.

The system achieves high accuracy through a three-layer architecture: Memory (Chroma vector store), Reasoning (Claude Opus 4), and Audit (Llama-3-70B), with continuous learning from execution feedback.

Architecture Overview

graph TB
    subgraph "Input Layer"
        USER[User Request
Natural Language]
        CONTEXT[Codebase Context
Files + Docs]
    end

    subgraph "Memory Layer"
        CHROMA[(Chroma DB
Vector Store)]
        EMBED[Embeddings
Code + Docs]
        RETRIEVE[Retrieval
Top-K Similar]
    end

    subgraph "Reasoning Layer"
        CLAUDE[Claude Opus 4
Primary Reasoning]
        PLAN[Planning
Task Decomposition]
        CODE[Code Generation
Locked to Retrieved]
    end

    subgraph "Audit Layer"
        LLAMA[Llama-3-70B
Self-Audit]
        VERIFY[Verification
Syntax + Logic]
        REFLEX[Reflexion
Error Correction]
    end

    subgraph "Execution Layer"
        EXEC[Code Execution
Sandboxed]
        FEEDBACK[Feedback Loop
Update Memory]
    end

    USER --> EMBED
    CONTEXT --> EMBED
    EMBED --> CHROMA
    CHROMA --> RETRIEVE
    RETRIEVE --> CLAUDE
    CLAUDE --> PLAN
    PLAN --> CODE
    CODE --> LLAMA
    LLAMA --> VERIFY
    VERIFY --> REFLEX
    REFLEX --> EXEC
    EXEC --> FEEDBACK
    FEEDBACK --> CHROMA

    style CHROMA fill:#4f46e5
    style CLAUDE fill:#dc2626
    style LLAMA fill:#059669

Core Concepts

Retrieval-Locked Generation

Problem: Hallucinations

# Traditional LLM might generate:
import non_existent_library  # ❌ Hallucinated API
result = magic_function()     # ❌ Doesn't exist

Solution: Lock to Retrieved Context

# HyperCoder process:
1. User: "Add user authentication"
2. Retrieve: Search vector DB for auth examples
3. Generate: Use ONLY retrieved code patterns
4. Verify: Check all imports exist in codebase

# Result:
from existing_auth import authenticate  # ✅ Real function
user = authenticate(request)            # ✅ Verified API

1-Shot Learning

Concept: Learn from a single example in the codebase and generalize to new contexts.

# User shows one example:
@app.route("/users/<id>")
def get_user(id):
    user = db.query(User).get(id)
    return jsonify(user.to_dict())

# HyperCoder learns pattern and applies to new request:
# "Create endpoint for products"

@app.route("/products/<id>")
def get_product(id):
    product = db.query(Product).get(id)  # Same pattern!
    return jsonify(product.to_dict())

Reflexion (Self-Correction)

Multi-Pass Generation:

Pass 1: Generate code
    ↓
Pass 2: Self-audit for errors
    ↓
Pass 3: Correct identified issues
    ↓
Pass 4: Verify corrections
    ↓
Output: High-confidence code

Core Components

1. Memory Store (Chroma)

Vector-Indexed Codebase:

from chromadb import Client
from sentence_transformers import SentenceTransformer

class CodebaseMemory:
    """Vector store for codebase knowledge"""

    def __init__(self, codebase_path: str):
        self.client = Client()
        self.collection = self.client.create_collection("codebase")
        self.encoder = SentenceTransformer("code-search-net")

        # Index entire codebase
        self.index_codebase(codebase_path)

    def index_codebase(self, path: str):
        """Chunk and embed all code files"""
        for file in walk_codebase(path):
            # Parse into functions/classes
            chunks = parse_code(file)

            for chunk in chunks:
                # Generate embedding
                embedding = self.encoder.encode(chunk.text)

                # Store with metadata
                self.collection.add(
                    embeddings=[embedding.tolist()],
                    documents=[chunk.text],
                    metadatas=[{
                        "file": file.path,
                        "type": chunk.type,  # function, class, etc.
                        "name": chunk.name
                    }],
                    ids=[f"{file.path}:{chunk.name}"]
                )

    def retrieve(self, query: str, top_k: int = 5) -> List[CodeChunk]:
        """Find most relevant code examples"""
        query_embedding = self.encoder.encode(query)

        results = self.collection.query(
            query_embeddings=[query_embedding.tolist()],
            n_results=top_k
        )

        return [
            CodeChunk(
                text=doc,
                metadata=meta
            )
            for doc, meta in zip(results["documents"][0], results["metadatas"][0])
        ]

2. Primary Reasoning Brain (Claude Opus 4)

LangGraph State Machine:

from langgraph.graph import StateGraph
from anthropic import Anthropic

class HyperCoderAgent:
    """Main coding agent with reasoning"""

    def __init__(self):
        self.claude = Anthropic()
        self.memory = CodebaseMemory("./codebase")
        self.graph = self.build_graph()

    def build_graph(self) -> StateGraph:
        """Define agent workflow"""
        graph = StateGraph()

        # Nodes
        graph.add_node("understand", self.understand_request)
        graph.add_node("retrieve", self.retrieve_context)
        graph.add_node("plan", self.plan_solution)
        graph.add_node("generate", self.generate_code)
        graph.add_node("audit", self.audit_code)

        # Edges
        graph.add_edge("understand", "retrieve")
        graph.add_edge("retrieve", "plan")
        graph.add_edge("plan", "generate")
        graph.add_edge("generate", "audit")

        # Conditional: If audit fails, regenerate
        graph.add_conditional_edges(
            "audit",
            self.should_regenerate,
            {
                "regenerate": "generate",
                "done": "END"
            }
        )

        return graph.compile()

    async def understand_request(self, state: dict) -> dict:
        """Parse user intent"""
        prompt = f"""
        Analyze this coding request:
        {state["user_request"]}

        Extract:
        1. Primary task
        2. Constraints
        3. Required knowledge
        """

        response = await self.claude.messages.create(
            model="claude-opus-4-20250514",
            max_tokens=1000,
            messages=[{"role": "user", "content": prompt}]
        )

        return {
            **state,
            "intent": response.content[0].text
        }

    async def retrieve_context(self, state: dict) -> dict:
        """Find relevant code examples"""
        context = self.memory.retrieve(state["intent"], top_k=5)

        return {
            **state,
            "retrieved_context": context
        }

    async def generate_code(self, state: dict) -> dict:
        """Generate code locked to retrieved context"""
        prompt = f"""
        Task: {state["intent"]}

        Relevant code from codebase:
        {format_context(state["retrieved_context"])}

        Generate code that:
        1. ONLY uses functions/classes from provided context
        2. Follows same patterns as examples
        3. Includes error handling
        4. Has clear comments

        CRITICAL: Do not invent any APIs not shown in context.
        """

        response = await self.claude.messages.create(
            model="claude-opus-4-20250514",
            max_tokens=2000,
            messages=[{"role": "user", "content": prompt}]
        )

        return {
            **state,
            "generated_code": response.content[0].text,
            "generation_count": state.get("generation_count", 0) + 1
        }

3. Self-Audit Layer (Llama-3-70B)

Independent Verification:

from transformers import AutoTokenizer, AutoModelForCausalLM

class CodeAuditor:
    """Self-audit with Llama-3-70B"""

    def __init__(self):
        self.tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3-70b-chat")
        self.model = AutoModelForCausalLM.from_pretrained(
            "meta-llama/Llama-3-70b-chat",
            device_map="auto",
            torch_dtype=torch.float16
        )

    async def audit(
        self,
        generated_code: str,
        context: List[CodeChunk]
    ) -> AuditResult:
        """Check for hallucinations and errors"""

        prompt = f"""
        Review this generated code for errors:

        {generated_code}

        Available APIs (from codebase):
        {format_context(context)}

        Check for:
        1. Hallucinated imports (not in available APIs)
        2. Syntax errors
        3. Logic errors
        4. Undefined variables
        5. Missing error handling

        Return JSON:
        {{
            "valid": bool,
            "errors": [list of issues],
            "confidence": float (0-1)
        }}
        """

        response = await self.generate(prompt)
        result = json.loads(response)

        return AuditResult(
            valid=result["valid"],
            errors=result["errors"],
            confidence=result["confidence"]
        )

    async def suggest_fix(self, error: str, code: str) -> str:
        """Generate correction for identified error"""

        prompt = f"""
        Fix this error in the code:

        Error: {error}

        Code:
        {code}

        Provide corrected version.
        """

        return await self.generate(prompt)

4. Reflexion Loop

Iterative Improvement:

class ReflexionLoop:
    """Self-correction through multiple passes"""

    async def refine(
        self,
        initial_code: str,
        context: List[CodeChunk],
        max_iterations: int = 3
    ) -> str:
        """Iteratively improve code until valid"""

        code = initial_code
        auditor = CodeAuditor()

        for i in range(max_iterations):
            # Audit current code
            audit = await auditor.audit(code, context)

            if audit.valid and audit.confidence > 0.95:
                # High confidence, accept
                return code

            if not audit.errors:
                # No specific errors but low confidence
                # Do one more pass with stronger prompt
                continue

            # Fix identified errors
            for error in audit.errors:
                fix = await auditor.suggest_fix(error, code)
                code = apply_fix(code, fix)

        return code  # Return best effort after max iterations

5. Execution & Feedback

Sandboxed Execution:

import docker

class CodeExecutor:
    """Execute generated code safely"""

    def __init__(self):
        self.client = docker.from_env()

    async def execute(self, code: str, test_cases: List[dict]) -> ExecutionResult:
        """Run code in isolated container"""

        # Create container with resource limits
        container = self.client.containers.run(
            image="python:3.11-slim",
            command=f"python -c '{code}'",
            detach=True,
            mem_limit="512m",
            cpu_quota=50000,  # 50% of 1 CPU
            network_disabled=True  # No network access
        )

        # Wait for completion (with timeout)
        try:
            result = container.wait(timeout=10)
            logs = container.logs()

            # Run test cases
            test_results = [
                self.run_test(code, test) for test in test_cases
            ]

            return ExecutionResult(
                success=result["StatusCode"] == 0,
                output=logs,
                test_results=test_results
            )

        finally:
            container.remove()

    async def provide_feedback(self, result: ExecutionResult):
        """Update memory with execution outcome"""

        if result.success:
            # Store successful pattern
            self.memory.add_positive_example(result.code)
        else:
            # Store failure to avoid repeating
            self.memory.add_negative_example(result.code, result.error)

Key Features

Hallucination Prevention

• Retrieval-Locked: Only uses verified APIs from codebase
• Multi-Model Audit: Independent verification with different model
• Confidence Scoring: Flags uncertain generations for human review
• Execution Feedback: Learns from runtime errors

1-Shot Learning

• Single example in codebase sufficient
• Generalizes patterns to new contexts
• Adapts to project-specific conventions
• No fine-tuning required

Self-Correction

• Reflexion loop with 3+ passes
• Identifies own errors before execution
• Suggests and applies fixes
• Improves with each iteration

Memory-Augmented

• Vector-indexed entire codebase
• Semantic code search
• Learns from execution outcomes
• Growing knowledge base over time

Performance Metrics

• Hallucination Target: ≤1% (aspirational)
• Current Hallucination Rate: ~5-8% (prototype)
• Retrieval Accuracy: 85% (top-5 relevant)
• Self-Audit Catch Rate: 70% of errors detected
• Reflexion Improvement: 40% reduction in errors after 3 passes
• Execution Success: 60% on first try, 85% after reflexion

Technical Stack

Orchestration

{
  "framework": "LangGraph (agentic workflows)",
  "multi-agent": "CrewAI (role-based agents)",
  "state_management": "LangGraph StateGraph"
}

Models

{
  "primary": "Claude Opus 4 (reasoning)",
  "audit": "Llama-3-70B (verification)",
  "embeddings": "CodeSearchNet (code embeddings)"
}

Memory

{
  "vector_db": "Chroma (in-memory or persistent)",
  "embeddings": "SentenceTransformers",
  "indexing": "Codebase + docs + execution history"
}

Use Cases

1. Code Generation

Generate new functions/classes following project patterns with minimal hallucination.

2. Refactoring

Update code to follow new patterns with automatic verification.

3. Bug Fixing

Identify and correct errors using retrieved similar fixes.

4. Documentation

Generate code from natural language specs with verified APIs.

Technical Highlights

• Retrieval-Locked Generation - Constrains LLM output to verified knowledge
• Multi-Model Verification - Independent audit layer catches primary model errors
• Reflexion - Self-correction through iterative refinement
• 1-Shot Learning - Generalizes from single examples without fine-tuning
• Execution Feedback Loop - Continuous learning from runtime outcomes

Limitations & Considerations

Hallucination Rate:

• Current 5-8% vs ≤1% target
• Difficult edge cases remain (novel API combinations)
• Requires more sophisticated verification

Latency:

• Multi-pass generation: 10-30 seconds
• Retrieval + primary + audit + reflexion pipeline
• Not suitable for real-time coding

Context Limitations:

• Chroma retrieval limited to top-K (typically 5-10)
• May miss relevant but syntactically different examples
• Large codebases require hierarchical indexing

Model Costs:

• Claude Opus 4: $15/$75 per 1M tokens (input/output)
• Llama-3-70B: Local inference requires H100 GPU (~$2/hour)
• Reflexion multiplies cost by iteration count

Future Enhancements

• Fine-Tuned Verifier: Train specialized model for code verification
• Hierarchical Retrieval: Multi-level indexing for large codebases
• Interactive Reflexion: Ask user for guidance when uncertain
• Test Generation: Automatically create test cases for generated code
• Streaming Output: Show intermediate reasoning steps

Status

Prototype demonstrating feasibility of retrieval-locked generation with self-audit. Core architecture proven but hallucination rate needs improvement to reach <1% target.

Part of MacLeod Labs AI Development Portfolio

Tech stack