Confessions of an AI Wizard: Closing the Action Gap at Open Source North

By Rael, Principal Intelligent Operations Architect at Evolving Solutions & Self-Appointed AI Wizard

Hello, fellow builders. It’s your friendly neighborhood AI Wizard here, fresh from Open Source North (OSN) 2026, where the Midwest’s best open-source minds gathered once again. Huge thanks to founder Jeff Urban and the incredible army of volunteers who make this conference run like a well-orchestrated agent swarm. If you haven’t been, check out Open Source North — easily one of the most genuine, high-signal tech events in the region.

I attended three sessions that perfectly captured where enterprise AI is headed. Spoiler: it’s not another “let’s throw GPT at our problems” talk. It’s about agentic systems — autonomous, reasoning entities that don’t just chat… they do.

And yes, I’m going to sprinkle in some wizardly wisdom (and a bit of snark) because if we can’t laugh while wrestling with probabilistic demons in deterministic castles, what’s the point?

The Action Gap: When Your Agent Has Great Ideas But Zero Follow-Through

As the AI Wizard, I’ve seen this pattern too many times. Modern LLMs are probabilistic sorcerers: give them the same prompt twice and you’ll get two different (but plausibly confident) answers. That’s fantastic for brainstorming. Not so much when you’re trying to approve a purchase order or trigger a failover.

Enterprise systems, on the other hand, are ruthlessly deterministic. They expect exact inputs, follow strict schemas, and throw hard errors when something’s off. This mismatch creates the Action Gap — the frustrating chasm between “AI generated a beautiful plan” and “AI reliably executed it without summoning the compliance demons.”

Integration hell makes it worse. Our data lives in ancient Excel graveyards, ambiguously-named APIs designed for humans, and documents without proper metadata. Agents lack the human intuition to “just figure it out.” Try connecting more than a handful of systems and complexity explodes faster than a poorly prompted chain-of-thought.

Wizard Tip: Stop treating integration as an afterthought. You need agent-ready architectures now — signed interactions, formal contracts (think OpenAPI on steroids with semantic metadata), clear authorization boundaries, and deterministic execution paths with runtime guardrails.

From Generative Toys to Production Agentic Systems

We’re in the middle of a clear evolution:

1. Traditional AI — Rules engines and deterministic workflows (boring but reliable)

2. Generative AI — Content, code, insights (fun but mostly assistive)

3. Agentic AI — Goal-oriented agents that plan, act, observe, and iterate across tools and systems.

Intelligence is now abundant and wildly asymmetric. A small, fast-moving team with well-orchestrated agents can outmaneuver much larger organizations. The bottleneck has shifted from “access to models” to “applied intelligence at scale.”

A highlight from the conference: an agent that manages budgets and spending approvals. It evaluates requests against policy rules, handles edge cases, and either approves or escalates — all while maintaining audit trails.

Side note from the Wizard (and real life): My husband Scott and I are already sketching something similar for personal use. Because nothing says “Principal Intelligent Operations Architect” like automating your own spending approvals so you can focus on bigger spells.

At Evolving Solutions, this is exactly the kind of work I lead — designing intelligent operations platforms that bridge the probabilistic world of AI with the deterministic reality of enterprise systems. We build secure orchestration layers, policy engines, and observable agent runtimes that actually make it to production.

The Architecture Mindset Shift (and Why “Vibe Coding” Must Die)

Another session nailed the evolution in how we build software in the AI era. We’re moving away from “vibe coding” — throwing prompts at the wall and hoping the code sticks — toward AI-Driven Delivery Lifecycle (AI-DLC).

Modern architects now use AI as a Chief of Staff:

• Challenging assumptions

• Stress-testing architectures

• Surfacing blind spots

• Running rapid scenario analysis

This frees humans to focus on judgment, trade-offs, and business value instead of grinding through boilerplate.

Technical Lessons I’m Taking Home:

• Design for failure early — Agents will hallucinate or take unexpected paths. Build in circuit breakers, human escalation, comprehensive tracing, and policy enforcement points.

  

• Never automate ambiguous processes — If you can’t clearly describe it in specs or code, an agent will make creative (and expensive) interpretations.

• Build platforms, not one-off agents — Think “the machine that builds the machines.” Invest in agent orchestration frameworks, shared memory/context stores, and governance layers.

• Observability is non-negotiable — You need visibility into reasoning traces, tool calls, state changes, and cost/performance metrics.

In short: Building a single agent is easy. Operating a fleet of them securely, compliantly, and cost-effectively is where the real engineering happens.

Key Takeaways from Your Friendly AI Wizard

1. Integration and data readiness are the real bosses — Not model size. Agent success is 80% architecture and data engineering.

2. Engineer determinism on top of probability — Guardrails, schemas, policies, and formal verification patterns are your new best friends.

3. Governance isn’t optional — Signed, authorized, auditable agent interactions win in the enterprise.

4. AI amplifies judgment — The highest-leverage use is augmenting senior technical minds, not replacing junior coders.

5. Hybrid human + agent teams are the future — Treat agents as first-class team members in spec-driven organizations.

 

Final Spell

The shift to agentic AI isn’t a modeling problem — it’s a systems, integration, and governance challenge. The organizations that win won’t just have smarter models. They’ll have built the infrastructure that lets those models act safely and reliably inside complex enterprise realities.

As the AI Wizard, my closing question is this:

> What can your AI be trusted to do… not just tell you?

Thanks again to Jeff Urban, the volunteers, and everyone at Open Source North. See you next year — I’ll be the one with the rhinestone tennies, a glowing staff and too many agent monitoring dashboards open.

— Rael 

Principal Intelligent Operations Architect @ Evolving Solutions 

AI Wizard in Residence