Jason Dicken

Prompt engineering · AI evaluation · Applied AI

I build practical AI systems—and test where they fail.

I turn 25 years of restaurant operations into AI workflows for customer support, sales and daily operations. I define the requirements, design the prompts, evaluate the behavior, and own what ships.

Evansville, Indiana · open to remote roles

Selected work

Two projects worth your time

One is measured and public. One runs a business every day and has no evaluation yet. Both say which is which.

Evaluation project · public repository

A support assistant that knows when to stop talking

Problem
A shop's chat assistant answers the easy questions well. The ones that matter are the ones it should not answer: allergy questions, refunds, and angry customers.
My role
I set the behavior rules, wrote every prompt revision, wrote the held-out questions from real customer messages, and read every failure. Coding agents wrote the harness to my spec and drafted the first case set; I reviewed and ran it.
Evidence
Four prompt versions, 40 development cases, 9 held-out questions, five recorded runs — all public, with the judge and the checks beside them.
Status
Complete as an evaluation. Nothing here says the assistant is ready for customers; the held-out run is the reason why.
Development set · 40 cases 27/40 → 40/40

Passing cases, v1 to v4. The set I read the failures from and tuned each prompt against, so this measures fit, not generalization.

Fresh questions · 9 cases 4/9

The same v4 prompt on nine questions it had never seen, run once. Fresh questions exposed five failures the development set missed.

Read the case study The four prompts Test cases The judge

Cobblestone CRM · production agent system

An agent fleet that cannot send anything without me

Problem
The work that closes deals — research, qualification, proposals, follow-up timing — was all manual, and an agent allowed to email a customer unsupervised is a liability, not a feature.
My role
I own the requirements, the architecture, the prompt for Oreo — the orchestrator — and for all 18 specialists, plus the review, the deployment and the daily operation. Coding agents wrote most of the implementation. When it is wrong in production, it is mine to fix.
Evidence
The approval path is diagrammed in the case study from the running system. The repository is private, so there is no code link — a walkthrough on request.
Status
Daily production use, running my own business. Formal evaluation pending: no test set exists for this fleet, so no accuracy claim is made for it anywhere on this site.
  1. A request comes in

    From the dashboard chat, from Telegram, or from one of 20+ scheduled jobs.

  2. Oreo routes it

    The orchestrator holds every tool: pipeline, contacts, activity logging, Gmail read and draft, calendar, and agent dispatch.

  3. A specialist does the work

    Oreo hands off to one of 18, from lead qualification and proposals to compliance review. None has database access of its own.

  4. I approve the send

    Customer-facing drafts queue as a PendingAction row. Email, text and proposals do not leave until a human approves them.

Read the case study

Also shipped

Software real businesses use

Cobblestone POS, and Daisy

Node.js · Express · Firebase · Firestore · Next.js
Production · no evals yet

Off-the-shelf systems are expensive and do not fit how small shops actually run, so I built my own and ran my ice cream shop on it: online ordering, a merchant site, digital menu boards, a DoorDash integration and a bar-inventory module.

Daisy is the assistant inside it. The answers an owner needs are in the POS database, but getting them means knowing which report to run — so the owner asks Daisy in plain English and she routes to agents for sales, food cost, inventory and mid-shift troubleshooting. She has no test set yet, so I cannot tell you how often she is right.

Cobblestone Perks

Loyalty and marketing · part of Cobblestone Restaurant Solutions
Production · no evals yet

Marketing is the first thing an owner drops on a busy week, and loyalty programs die the same way. Perks holds the rewards side — offers and punch cards tied to the POS — and its marketing agent plans and drafts a month of promotions, posts and send dates in under an hour, so the month exists before the week gets away from you.

Cobblestone Galley

React · TypeScript · Supabase · Capacitor
In development · not shipped

Barge kitchens order food by phone and paper, and no one can see what is on the shelf. Galley is an offline-first inventory and ordering app the chef, cook and captain share on one tablet: the chef requests, the captain approves, the purchase order goes out by email. Built by two AI coding agents in one codebase, coordinated with a written baton protocol and a log of locked decisions.

The Owner's Notebook

Python · static site pipeline
Published

A library of food-cost, waste and par-level guides produced by a content pipeline. When the first version read as generic AI voice, I rebuilt the pipeline rather than editing the output.

Approach

How I work

I specify, review and own; agents write most of the code

Claude Code and Codex write the bulk of the implementation. I own requirements, architecture, the agent and system prompts, review, testing, deployment and operation. On the evaluation project that also means the behavior rules in every prompt version, the held-out questions, and reading every failure myself.

Decisions get written down

Every project keeps a handoff document and a dated decision log, so an agent picking up the work reads the same constraints I would otherwise repeat. From Galley's log: vendors are shoppers, not warehouses, so no line prices anywhere and spend is the typed receipt total. That is the part a model cannot infer from the codebase.

One writer at a time

Two agents and a human in one repository will collide. The evals repo carries a baton: a tracked file naming the current holder, with a pre-commit hook that rejects commits from anyone else. Handoff notes are mandatory and written for the next agent.

Prompts drift, so they get audited

A prompt is code, but nothing tests it unless you write the test on purpose: versioned files, a recorded run per version, and an audit against what the system actually does. My CRM orchestrator still described "a team of sixteen specialist agents" months after the fleet had grown to eighteen — true when written, wrong for months, invisible until someone read it against the code.

About

An operator first

I started a restaurant with $10,000 and grew it into a 13-location chain across three states. Later I built an ice cream shop, wrote the software that ran it, and sold it. I spent three years as a SCORE mentor helping small business owners, and before any of that I was a Marine.

That is why I build AI the way I do. In a restaurant, a wrong answer about an allergy is not a bug report. So I test for it first.

  • 2025–Cobblestone Restaurant Solutions: CRM, POS, Perks, Galley, the Owner's Notebook
  • 2026Cobblestone Creamery: built it, wrote its software, sold it
  • 2022–2025SCORE small-business mentor and VP
  • 2006–2021CEO & President, Spudz N Stuff (13 locations)
  • 1998–2001U.S. Marine Corps, Corporal
Evaluation
Test-set design, rubric writing, LLM-as-judge and its failure modes, code-based checks, regression tracking, held-out splits
Applied AI
Prompt design and versioning, tool-using agents, orchestrator and specialist patterns, human-in-the-loop approval, model tiering and cost control
Building
TypeScript, Python/FastAPI, Node/Express, React, SQLite and Postgres, Anthropic API, Twilio, Google Workspace APIs
Operations
Food and labor cost, inventory and par levels, P&L, POS and payments, the domain these systems are judged in

Contact

Hiring for prompt engineering or AI evaluation? Write me.

jason@cobblestonepos.com

Evansville, Indiana. Open to remote work. U.S. Marine Corps veteran.