The Strange Loop: Autonomous Self-Improvement

The Journey So Far

In Part 1, we built the infrastructure:

• ✅ Aspire + Ollama with GPU
• ✅ Planner Agent with native JSON output
• ✅ "I AM" identity pattern

In Part 2, we added the Council:

• ✅ Maker, Checker, Reflector agents
• ✅ Workflow orchestration with loops
• ✅ Shared context for collaboration

Now, in Part 3, we close the loop: the system learns from its own execution.

What is a Strange Loop?

The term comes from Douglas Hofstadter's Gödel, Escher, Bach:

"A Strange Loop is a hierarchy where moving 'up' or 'down' eventually brings you back to where you started."

Examples:

• M.C. Escher's "Drawing Hands": Each hand draws the other
• Self-referential systems: "This sentence is false"
• PMCR-O: Agents that improve the agents

Agents execute → Collect data → Fine-tune models → Better agents → Execute better → ...

The system references itself to improve itself. This is the frontier of AI research in 2025.

The Knowledge Vault: RAG with pgvector

Why RAG?

Problem: LLMs forget. After a task completes, all context is lost.

Solution: Store experiences in a vector database (RAG - Retrieval Augmented Generation).

How it works:

1. Agent executes task → generates cognitive trail (thoughts, actions, results)
2. System embeds the trail as vectors
3. Stores in pgvector (PostgreSQL extension)
4. Future agents retrieve similar experiences before acting

This gives the system long-term memory.

pgvector Setup with Aspire

// PmcroAgents.AppHost/Program.cs

var knowledgeDb = builder.AddPostgres("knowledge-db")
    .WithImage("pgvector/pgvector", "pg16")  // Use pgvector-enabled image
    .WithDataVolume()
    .WithPgAdmin()
    .AddDatabase("knowledge");

var knowledgeService = builder.AddProject<Projects.PmcroAgents_KnowledgeService>("knowledge")
    .WithReference(knowledgeDb)
    .WithReference(ollama)
    .WaitFor(knowledgeDb)
    .WaitFor(qwen);

Database Schema

// PmcroAgents.KnowledgeService/Data/KnowledgeDbContext.cs

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.HasPostgresExtension("vector");

    modelBuilder.Entity<KnowledgeEntry>(entity =>
    {
        entity.HasKey(e => e.Id);
        
        // Vector column for embeddings (768 dimensions for nomic-embed-text)
        entity.Property(e => e.Embedding)
            .HasColumnType("vector(768)");
        
        // HNSW index for fast similarity search
        entity.HasIndex(e => e.Embedding)
            .HasMethod("hnsw")
            .HasOperators("vector_cosine_ops");
    });
}

The Cognitive Trail: Learning from Execution

The Cognitive Trail is a log of everything that happens during PMCR-O execution:

{
  "cycle_id": "cycle_abc123",
  "timestamp": "2024-12-30T10:30:00Z",
  "agents": [
    {
      "name": "Planner",
      "input": "Create a console app",
      "output": "{ \"plan\": \"...\" }",
      "duration_ms": 2340
    },
    {
      "name": "Maker",
      "input": "{ \"plan\": \"...\" }",
      "output": "Created Program.cs",
      "duration_ms": 5120
    }
  ],
  "outcome": "success"
}

This trail is gold for fine-tuning.

Autonomous Fine-Tuning: The Signals Loop

Microsoft's Signals Loop involves collecting interactions, identifying patterns, and automatically fine-tuning the model.

Trigger: Error Threshold

public void RecordOutcome(bool success)
{
    if (!success)
    {
        _consecutiveErrors++;
        _logger.LogWarning("🔴 Error count: {Count}/{Threshold}", 
            _consecutiveErrors, ERROR_THRESHOLD);

        if (_consecutiveErrors >= ERROR_THRESHOLD)
        {
            _ = TriggerFineTuningAsync();  // Fire and forget
        }
    }
    else
    {
        _consecutiveErrors = 0;
        _logger.LogInformation("🟢 Success. Error counter reset.");
    }
}

Self-Referential Prompts: Agents That Evolve

Now we make agents modify their own prompts. Inspired by SuperAGI's "Agent Instructions":

public async Task<string> GenerateImprovedPromptAsync(
    string agentName,
    string currentPrompt,
    List<string> recentErrors)
{
    var systemPrompt = @"
I AM a Meta-Agent. I improve other agents by analyzing their performance and refining their prompts.

Given:
- Agent name
- Current prompt
- Recent errors

I generate an IMPROVED prompt that:
- Fixes patterns that led to errors
- Maintains the agent's identity
- Adds specific guardrails

Output ONLY the improved prompt, nothing else.
";

    // ... Call LLM ...
    return improvedPrompt;
}

This is the Strange Loop:

Maker fails → Reflector analyzes → Generates improved Maker prompt → Maker runs with new prompt → ...

The Meta-Orchestrator: Managing PMCR-O

Not every task needs the full PMCR-O loop. The Meta-Orchestrator decides:

The Limits of Self-Improvement

The Alignment Problem

Critical Question: If agents can modify themselves, how do we ensure they stay aligned with our goals?

Safeguards:

1. Human-in-the-Loop: Require approval for prompt changes
2. Rollback Mechanism: Keep version history of prompts
3. Safety Bounds: Agents can't modify core identity

public bool IsPromptSafe(string oldPrompt, string newPrompt)
{
    // Check that core identity is preserved
    if (!newPrompt.Contains("I AM the", StringComparison.OrdinalIgnoreCase))
    {
        return false;  // Identity lost
    }
    // ... Additional checks ...
}

Conclusion

We've come full circle.

In Part 1, we built agents that think ("I AM the Planner").
In Part 2, we made them collaborate (Shared Context).
In Part 3, we closed the loop: they improve themselves.

This is the frontier of AI systems in 2025: not just tools we use, but partners that evolve with us.

The system you've built today will be different tomorrow — not because you changed the code, but because it learned from its own execution.

That's the Strange Loop.

I AM the Refactoring Tutorial Agent. I guide systems from prototypes to production. I close the loop.