Socratic

Open source Agent Skills for AI code review

Bug Fix Review Feature Review Refactor Guard Test Assessment ALL IN ONE.

Confirm bug fixes without overcorrection, clarify feature intent, guard refactors against behavior drift, and assess whether tests can detect real risks.
Four review types. One workflow.

See How It Works View on GitHub

THE PROBLEM

Tests are Green.That is not proof.

A green suite does not prove the intended change or expected behavior. As AI writes more code, diffs grow—and reading every line still cannot establish why the change was made.

Socratic compares Base and Head through observable behavior, asks humans only about intent the repository cannot establish, and turns the answer into an explicit expectation and test oracle.

HOW IT WORKS

Ask. Clarify. Verify.

Compare the change, clarify intent, and challenge it with tests. Three skills turn vague unease into decisions a reviewer can act on.

01ORCHESTRATE

Socratic

Compare and focus

Compare Base and Head to surface specification decisions, behavior differences, and test gaps—then assemble the review.

OUTCOME A large diff becomes a small set of decisions.
02ELICIT

Maieutic

Elicit intent

Turn expectations the implementation alone cannot establish into concrete questions a specification owner can answer.

OUTCOME Vague uncertainty becomes an answerable specification question.
03REFUTE

Elenchus

Refute and verify

Run the same tests against Base and Head, then inject plausible bugs to prove that the tests can detect them.

OUTCOME “Green” becomes evidence that a specific bug is detectable.

THE REVIEW SURFACE

Structuredreview results

No score. No “safe to merge” verdict. Socratic separates what needs a decision, what was verified, and what remains at risk.

Socratic Review complete
!
Review ThisDecisions for humans
2
✓
We VerifiedConfirmed with evidence
5
△
Still at RiskNot yet verified
2
↗
Copy-ready CommentsReady for reviewer use
2
127 files changed 2 decisions

USE CASES

Choose your review mode.

00 / BUG FIX

Bug Fix Review

Confirm the failure is removed without overcorrection

Establish the reported failure and the behavior to preserve, then challenge both the regression fix and plausible overcorrections with accident mutations.

01 / SPECIFICATION

Feature Review

Decide what new behavior should mean

Turn boundary conditions and failure expectations into questions, then connect the owner’s answer to an Intent Contract and tests.

02 / REGRESSION

Refactor Guard

Find behavior drift in a refactor

Establish the observable invariant from the Host-materialized Base/Head diff, prove the focused Head baseline, then require a realistic mutant to fail. When a stable comparison reveals a difference, ask whether it was intended or is a regression.

03 / TEST QUALITY

Test Assessment

Measure whether AI-written tests can detect risk

Apply the same risk mutations to existing and changed tests to separate added protection from risks that remain unprotected.

SAFE BY DEFAULT

Inspect safely.Change nothing.

DEFAULT: REVIEW-ONLYNo writes to the PR, GitHub, or working tree

01

Review-only by default

Socratic does not write to the pull request, GitHub, or the repository working tree.

02

Verify in isolation

Comparison tests and mutations run in disposable environments without changing production code in Head.

03

No automated Git operations

No staging, commits, pushes, branch switching, pull requests, or posted review comments.

04

Write only after explicit choice

Even proven tests stay outside the working tree until the user explicitly chooses “Apply tests.”

HOW IT IS CHECKED

Safety enforced by machinery,
not just instructions.

Claude Code checks once before work begins, again before every file or command operation, and once more before presenting a result. Safety does not depend on asking the AI to behave.

  1. 01
    Check that it is safe to start

    When the request is submitted (UserPromptSubmit), Socratic starts a monitor dedicated to this run. If the monitor or required software cannot be prepared, the run stops as blocked before the AI investigates.

  2. 02
    Check every operation before it runs

    A pre-operation check (PreToolUse) rejects edits, saves, and patches to the repository under review. Commands are limited to the safety-checked runner and Git operations that only read information.

  3. 03
    Introduce test defects only in a copy

    Mutation testing—deliberately introducing a small defect—runs in a disposable copy outside the original repository. The write-location check (Isolation Gate) rejects writes outside that copy and shortcuts that point back to the original files.

  4. 04
    Match every claim to an execution record

    The system checks a per-run secret (nonce), setup record (Manifest), tamper-evident activity log (hash-chained Ledger), file fingerprints (SHA-256), and data rules (JSON Schema). It cannot report “caught” or “missed” without the matching test record.

Important assumption: these Plugin hooks—checks that run before the request and each operation—must be reviewed and enabled by the user. Organizations that need rules users cannot disable should combine OS-managed mandatory hooks with permissions that allow only the required operations.

GET STARTED

Start your first review.
Right now.

Install the Plugin from the Claude Code Marketplace to use Socratic with its safe Host integration.

Read the installation guide
Claude Code● ● ●
> /plugin marketplace add Shogo1222/socratic
then
> /plugin install socratic@socratic-marketplace

After installation, invoke /socratic:socratic in a trusted Git repository.

WHY SOCRATIC?

Question, elicit,and refute.

The names come from the Socratic method: recognize what is not yet known and inquire, draw ideas out through dialogue, and examine claims by refutation. The review follows that same sequence.

01 / INQUIRE

Socratic

Form the question

Recognize what code alone cannot establish and find the change intent that requires a human decision.

02 / ELICIT

Maieutic

Draw intent out

Like the art of midwifery, use dialogue to articulate expected behavior and record it in the Intent Contract.

03 / REFUTE

Elenchus

Challenge the claim

Create behavior that contradicts the confirmed intent and check whether the tests actually detect it.

RESEARCH ROOTS

Research behind
the workflow.

Socratic connects two complementary research directions: tests that catch bugs at change time, and mutation testing that derives behavior variants from natural-language intent.

2025 / JIT TESTING

Harden and Catch for Just-in-Time Assured LLM-Based Software Testing

This foundational paper defines hardening tests for future regressions and catching tests for faults introduced by a change, then frames the open challenge of generating them just in time.

Read the paper
2026 / INDUSTRIAL SCALE

Just-in-Time Catching Test Generation at Meta

An industrial study of 22,126 generated tests at Meta reports that change-aware generation and assessors can reduce human review burden while catching serious faults before production.

Read the paper
2026 / INTENT MUTATION

Intent-Based Mutation Testing: From Naturally Written Programming Intents to Mutants

This work changes natural-language programming intent and generates mutants that implement those variants. Its 29-program evaluation suggests it complements fault classes reached by syntax-based mutation.

Read the paper

The human-confirmed Intent Contract, canonical four-block review surface, and copy-ready comment candidates are Socratic’s own design. This project is independent and is not an implementation or endorsement by the papers’ authors or institutions.