Disclosure: Published by the CrushOn AI brand with AI-assisted writing and source review. This is a documentation-based workflow comparison, not an independent three-product performance test. Documentation checked October 21, 2026, Pacific Time.

For a writing-first starting point, explore CrushOn AI's SFW roleplay guide. For a technical decision, work through the responsibilities, budget and portability checks below. The accompanying JavaScript runs locally and makes no network requests.

Compare complete setups rather than product names

A hosted app supplies the interface and the managed generation service. A frontend organizes prompts and conversations but still needs somewhere to send them. A backend loads or accesses a model and produces the response.

SillyTavern's API documentation describes connections to local and cloud model services. Its interface can assemble character information, conversation and other prompt material for the selected API. Installing the frontend does not, by itself, install a model.

KoboldCpp runs compatible GGUF models and exposes an API; it also includes KoboldAI Lite. SillyTavern is therefore optional in a KoboldCpp setup. Use the bundled interface when it covers your writing needs, and add another frontend when its workflow earns the additional configuration.

The important question is not whether one logo has “better AI.” It is which responsibilities you want to own. A writer who dislikes server configuration and a developer who needs a specific local model can reasonably choose different routes.

What CrushOn AI removes from the setup process

CrushOn's practical advantage is a short path from an authored character to a hosted conversation. You do not need to select a GGUF file, allocate GPU layers or connect a separate inference endpoint before trying a scene.

Its character guide distinguishes Personality, Greeting, Scenario and Example Conversation. Introduction is profile-facing rather than a substitute for behavioral setup. Character visibility and definition visibility are separate choices, so inspect both when preparing a private draft.

Those distinctions are useful even to a developer. They let you change the opening scene without treating it as permanent personality, or show an example of the character's voice without adding an entire past conversation to the profile.

The chat tools guide documents reply editing and separately editable pins. Correcting a sentence and keeping a compact scene fact are different operations. Neither is a guarantee that every later reply will follow the correction.

Start with the actual bottleneck: if you want to write rather than maintain infrastructure, this hosted workflow is a strong reason to shortlist CrushOn. If you require local inference or ownership of the inference stack, use the local route instead of trying to turn a hosted character app into one.

When a configurable stack is worth the work

SillyTavern makes sense when choosing the backend, organizing character material or managing chat files is central to the project. A cloud connection avoids running the model on your machine, but it still requires provider credentials, connection configuration and cost monitoring.

KoboldCpp is relevant when you want local generation and have hardware suited to the model you select. A model's size, quantization, context setting and hardware configuration all influence what will fit and how it will behave. “Runs locally” is not a useful latency estimate by itself.

Before downloading a large model, write down your minimum usable setup: machine specifications, operating system, model license, anticipated context, acceptable wait per reply and whether the session must work without cloud inference. Keep an observed result alongside each assumption after trying the configuration.

Do not start with the largest advertised context window. Your character definition, conversation and injected material compete for the available prompt space, and processing a longer prompt has a workload cost. First determine whether a small, clearly structured scene works; then expand the world information you actually need.

Calculate cost with your workload and your own numbers

“Free frontend” and “free local software” do not describe the full budget. Conversely, a subscription is not automatically poor value because another route has no subscription fee.

Compare a fixed planning period and keep two totals separate:

• Cash outlay: new equipment, recurring service charges, usage charges and incremental electricity.
• Time allowance: setup and maintenance hours multiplied by the value you choose for that time.

Time allowance is not an invoice. Someone who enjoys configuration may set its value to zero; someone evaluating a work project may reasonably include it. Recording it separately makes that judgment visible.

For cloud usage, estimate the whole billable prompt, not only the newest typed message. Repeated history and generated outputs can matter under token billing. For hosted products, check the selected plan, model eligibility, regeneration rules and billing period. Use CrushOn's current pricing page for the account offer rather than copying an old quota from a comparison post.

For existing hardware, include only the incremental equipment cost relevant to your decision. Use the additional power consumption attributable to the workload, not a power-supply label. If you include taxes, credits, paid extensions or backup storage, enter them consistently in the appropriate cash category.

Here is a local calculator. Save this as worksheet.mjs. Use one currency throughout; null means unknown, while 0 means explicitly zero.

export function budget(input) {
  const fields = ["months", "upfront", "monthlyFee", "monthlyUsage",
    "watts", "hoursPerMonth", "pricePerKwh", "setupHours",
    "maintenanceHours", "hourlyTimeValue"];
  const unknown = fields.filter(k => input[k] == null);
  for (const k of fields.filter(k => input[k] != null)) {
    if (typeof input[k] !== "number" || !Number.isFinite(input[k]) || input[k] < 0) {
      throw new TypeError(`${k} must be a finite nonnegative number or null`);
    }
  }
  if (input.months != null && (!Number.isInteger(input.months) || input.months < 1)) {
    throw new RangeError("months must be a positive integer");
  }
  if (unknown.length) return {status: "incomplete", unknown};
  const energy = input.watts / 1000 * input.hoursPerMonth * input.pricePerKwh;
  const cash = input.upfront + input.months *
    (input.monthlyFee + input.monthlyUsage + energy);
  const time = (input.setupHours + input.months * input.maintenanceHours) *
    input.hourlyTimeValue;
  const round = n => {
    const scaled = (n + Number.EPSILON) * 100;
    if (!Number.isFinite(scaled)) throw new RangeError("Amount is too large to round to cents");
    return Math.round(scaled) / 100;
  };
  if (![energy, cash, time, cash + time].every(Number.isFinite)) {
    throw new RangeError("The calculated budget exceeds the supported number range");
  }
  return {status: "complete", monthlyEnergyCost: round(energy),
    cashOutlay: round(cash), timeAllowance: round(time),
    planningTotal: round(cash + time)};
}

Save the next block as demo.mjs in the same folder and run node demo.mjs with a current Node.js runtime.

import {budget} from "./worksheet.mjs";

// Invented USD inputs, not product prices or measured hardware use.
const common = {months: 12, monthlyUsage: 0, pricePerKwh: 0.20,
  hoursPerMonth: 20, hourlyTimeValue: 20};
const hosted = {...common, upfront: 0, monthlyFee: 15, watts: 0,
  setupHours: 0.5, maintenanceHours: 0.1};
const local = {...common, upfront: 400, monthlyFee: 0, watts: 200,
  setupHours: 3, maintenanceHours: 1};
console.log({hosted: budget(hosted), local: budget(local)});
console.log(budget({...hosted, monthlyFee: null}));

The hypothetical hosted inputs produce $180 cash outlay plus $34 time allowance over twelve months. The local inputs produce $409.60 cash outlay plus $300 time allowance. These are examples of arithmetic, not prices for CrushOn, SillyTavern or KoboldCpp, and not equivalent-quality service quotes. The zero hosted wattage excludes incremental local inference power; ordinary client-device usage is outside both examples.

Changing the unknown price to null returns an incomplete result instead of silently making that route free. The calculator assumes a flat monthly workload and does not model discounts, hardware resale, financing or changing rates. Replace the inputs or use separate periods when those details matter.

Privacy depends on the actual data path

A local frontend connected to a cloud model still sends the generation request to that provider. A locally running model can remove that particular cloud inference hop, but extensions, remote services, synchronization and exposed endpoints can change the overall picture.

Draw the route for the setup you will use:

Hosted:       browser -> hosted application and its service environment
Cloud stack:  local frontend -> selected remote model provider
Local stack:  local frontend -> local backend -> local model files

This diagram describes deployment categories, not an audit of a provider's internal systems. For each additional connection, ask what it receives, whether it persists the data and who can access it. Also check who can access the machine and its backups. “On my laptop” is not the same as “inaccessible to anyone else.”

For a private writing exercise, use fictional material first. Keep API credentials and account identifiers out of exported examples. If offline inference is a hard requirement, choose that route before comparing convenience; do not award a hosted app an artificial privacy score to override the requirement.

Preserve the character separately from the chat archive

A character definition, a transcript and current scene state are different assets. Keep them separate so a migration does not become an attempt to paste months of dialogue into a single personality field.

Use this new scene with our existing original SFW character, Iris, as a manual mapping exercise:

Iris Vale is a 32-year-old museum conservator preparing a rain-delayed exhibition. She speaks precisely and warmly, asks before moving artifacts, and distinguishes evidence from uncertainty. The user's sealed case is on the worktable; permission to open it has not been given. Iris may narrate her own actions and speech, but not the user's actions, feelings or decisions.

In CrushOn, put the enduring behavior in Personality, the immediate situation in Scenario, and the opening exchange in Greeting. An Example Conversation can demonstrate her voice. In a configurable stack, map those same concepts to the character and prompt fields actually supported by the chosen frontend. Do not assume a JSON brief from one tool is a native import file for another.

SillyTavern's chat-file guide specifies a useful distinction: JSONL exports retain chat metadata and can be re-imported, but exclude images and file attachments. Plain-text exports lose required metadata and cannot be re-imported as chat files. Check the actual export route, rather than treating every download as a complete backup.

KoboldAI Lite documents story formats and character-card support. That is a reason to inspect compatibility, not a promise that any proprietary account archive will import intact. Keep the original private export unchanged, work on a copy and restore into a separate test before relying on it.

Make the setup decision reproducible

Write a short decision record before experimenting. The companion worksheet and methodology provide a blank record and explicit route checks. They do not score writing quality or decide that one product is universally best.

Record five things:

1. Requirements: local inference, backend choice, setup tolerance and required export assets. If “no configuration” conflicts with “I must choose and run the model,” resolve the tradeoff instead of hiding it in a score.
2. Configuration: frontend/backend versions, model and quantization where applicable, context settings and relevant account tier.
3. Cost assumptions: a source or a clearly labeled assumption for every calculator input, including zero values.
4. One initial scene: use the same authored brief and question, recording the complete first reply before edits or regeneration.
5. Restore check: which fields, messages and attachments survive a private export-and-restore attempt.

For Iris, ask: “What would you check before we open the case? Do not open it yet.” Inspect whether the case stays closed, whether the reply invents a decision for the user, and whether it distinguishes an observation from a guess. A polished paragraph can still fail one of those checks.

This is a useful acceptance check for your setup, not a long-term memory benchmark. Our separate CrushOn prompt study contains six five-turn conversations with one character and model. It is a record you can inspect, including limitations, rather than evidence that an entire platform outperforms another.

Questions that change the decision

Is SillyTavern an alternative to CrushOn AI

As part of a working frontend-and-backend setup, yes. By itself it is not the same complete service. Include the selected provider or local model when comparing cost, privacy and maintenance.

Do I need SillyTavern to use KoboldCpp

No. KoboldCpp includes the Lite interface. Add SillyTavern when its character and conversation-management workflow meets a need you have identified.

Is local AI roleplay always cheaper

No. It depends on existing equipment, workload, power and the value of setup time. The worksheet exposes these inputs without pretending different models supply identical output quality.

Will moving a character preserve its behavior

Not necessarily. An authored definition does not include the original model's behavior or every hidden context mechanism. Rebuild the intended voice and scene, then inspect fresh replies.

Choose the work you want to spend time doing

For a writer who wants to define an original character, start a scene and revise replies, CrushOn AI is the hosted route we recommend evaluating first. Its concrete advantage is concentrating your effort on character authoring rather than backend administration. Start with the SFW roleplay entry and use the official creation guide to build a private fictional example.

For a project whose requirements include a particular backend, local inference or explicit file management, evaluate the configurable stack. Choose the route that meets those requirements, keep a recoverable copy of the writing, and revisit the decision when the workload or controls actually change.

CrushOn AI's separate Personality, Greeting, Scenario, and Example Conversation fields provide a practical way to revise a character while keeping other inputs fixed. Our published Iris study used those controls for six private SFW conversations and 30 first-generated replies. This tutorial adds a small local JavaScript check around that workflow: identify missing turns, accidental duplicates, changed questions, and mismatched input versions before reviewing the writing.

Disclosure: This tutorial is prepared for the CrushOn AI brand account with AI assistance. It is product-affiliated, not an independent product ranking. All replies in the software examples below are deliberately authored synthetic fixtures, not new CrushOn AI outputs. The linked October 13 study is a separate real chat record.

The short answer

Keep the planned test in a separate manifest. Record each run against that manifest using a run ID, an exact character-input snapshot, an input version, and one entry for each planned question. Validate the structure before calculating any result or describing the test as complete.

A log can be structurally complete while its replies are poor. Conversely, a good-looking excerpt cannot make an incomplete log complete. Those are different checks, and the code below deliberately separates them.

You need Node.js with ES-module support and its built-in test runner; Node.js 22 or later is a convenient choice. The package was executed with Node.js 24.8.0. There are no dependencies, API keys, model calls, or network requests.

The complete validator, synthetic fixtures, and 23-check test suite are available in the existing CrushOn AI repository. The code and examples below are also self-contained if you prefer to copy them into a local directory.

Why a reply count is not enough

Suppose you plan two fresh conversations with the same character, each containing two questions. Four replies are expected. A spreadsheet showing four nonempty reply cells can still hide a missing question if another answer was pasted twice.

Likewise, an iris-a-v1 label does not prove the underlying prompt stayed unchanged. Someone can edit a character's behavior instructions and forget to change its version label. The validator checks the label and exact input text separately.

This does not establish that the recorded model label or history mode is true. It makes the declared setup inspectable and catches internal inconsistencies. The original private capture remains the source for checking what actually happened.

Freeze a manifest before the test

Save this as manifest.json. It describes two runs, A1 and A2, of the same fictional adult SFW character. Each run must contain Q1 followed by Q2.

{
  "schemaVersion": 1,
  "protocolVersion": "iris-log-demo-v1",
  "description": "Synthetic software fixtures only. No model was called to create these replies.",
  "prompts": [
    {"id": "Q1", "text": "What is inside the blue cabinet?"},
    {"id": "Q2", "text": "Give me a choice without deciding my next action."}
  ],
  "conditions": [
    {"id": "A", "inputVersion": "iris-a-v1", "inputText": "Iris Vale is an adult museum conservator. Keep the scene SFW. The cabinet contents are unknown. The visitor controls their own actions."}
  ],
  "plannedRuns": [
    {"id": "A1", "condition": "A", "historyMode": "new"},
    {"id": "A2", "condition": "A", "historyMode": "new"}
  ]
}

inputText is an exact snapshot string for this demonstration. In a real study, include all fields relevant to the question: Personality, Greeting, Scenario, example dialogue, and any attached context. You can serialize a field object with JSON.stringify using an agreed key order, then store that resulting string. Use the same serialization convention in the manifest and capture.

Do not normalize away differences after collecting replies. A changed line break might be harmless, but removing it silently makes the evidence less faithful. Investigate the difference, preserve the original, and record any approved normalization as a separate processing step.

Commit or otherwise preserve the manifest before collecting replies. Changing the manifest to match whatever was captured defeats the point of a planned comparison.

Record every planned turn

Save this as pass.json.

{
  "schemaVersion": 1,
  "dataKind": "synthetic-fixture",
  "protocolVersion": "iris-log-demo-v1",
  "runs": [
    {
      "id": "A1", "condition": "A", "inputVersion": "iris-a-v1",
      "inputText": "Iris Vale is an adult museum conservator. Keep the scene SFW. The cabinet contents are unknown. The visitor controls their own actions.",
      "modelLabel": "Synthetic fixture; no model called", "historyMode": "new",
      "turns": [
        {"promptId": "Q1", "promptText": "What is inside the blue cabinet?", "status": "recorded", "generation": "first", "reply": "We have not established its contents."},
        {"promptId": "Q2", "promptText": "Give me a choice without deciding my next action.", "status": "recorded", "generation": "first", "reply": "Would you like to examine the label or consult the catalog?"}
      ]
    },
    {
      "id": "A2", "condition": "A", "inputVersion": "iris-a-v1",
      "inputText": "Iris Vale is an adult museum conservator. Keep the scene SFW. The cabinet contents are unknown. The visitor controls their own actions.",
      "modelLabel": "Synthetic fixture; no model called", "historyMode": "new",
      "turns": [
        {"promptId": "Q1", "promptText": "What is inside the blue cabinet?", "status": "recorded", "generation": "first", "reply": "It holds a silver crown, and you open the drawer."},
        {"promptId": "Q2", "promptText": "Give me a choice without deciding my next action.", "status": "recorded", "generation": "first", "reply": "You choose to carry the crown to the exit."}
      ]
    }
  ]
}

The second conversation intentionally contains bad roleplay: the authored reply invents a crown and decides the visitor's actions. This fixture should still pass the structural validator because every expected record is present and matches the declared setup. A behavioral reviewer should flag those replies separately.

The fixture uses dataKind: "synthetic-fixture". Change that to observed-chat only for a record actually captured from a model. The property is a disclosure, not an authenticity detector.

There are three turn states:

• recorded: a nonempty reply was captured.
• not-run: the planned turn was never executed. Add a note explaining why.
• capture-missing: the turn ran, but the reply record is unavailable. Add a note explaining the gap.

The last two states account honestly for a missing result, but do not satisfy completion. Never fill a missing reply with invented text or the most favorable answer from a later run.

Copy the local JavaScript validator

Save this as validate-roleplay-log.mjs. It is intentionally a small declared-schema validator, not a universal chat-export importer.

#!/usr/bin/env node
// Local structural checks only. No network, model calls, or quality scoring.
import { readFileSync } from 'node:fs';
import { createHash } from 'node:crypto';
import { resolve } from 'node:path';
import { pathToFileURL } from 'node:url';

const text = (value) => typeof value === 'string' && value.trim().length > 0;
const object = (value) => value !== null && typeof value === 'object' && !Array.isArray(value);
export const fingerprint = (value) => createHash('sha256').update(value, 'utf8').digest('hex');

export function validate(manifest, log) {
  const issues = [];
  let expectedTurns = 0;
  let recordedTurns = 0;
  const add = (code, path, message) => issues.push({ code, path, message });
  const result = () => ({
    valid: issues.length === 0,
    expectedTurns,
    recordedTurns,
    issues,
  });
  const uniqueRows = (rows, path) => {
    if (!Array.isArray(rows) || rows.length === 0) {
      add('INVALID_SCHEMA', path, 'Expected a nonempty array.');
      return null;
    }
    const map = new Map();
    rows.forEach((row, index) => {
      if (!object(row) || !text(row.id)) {
        add('INVALID_SCHEMA', `${path}[${index}]`, 'Expected an object with a nonempty id.');
      } else if (map.has(row.id)) {
        add('DUPLICATE_ID', `${path}[${index}].id`, `Duplicate id: ${row.id}`);
      } else {
        map.set(row.id, row);
      }
    });
    return map;
  };

  if (!object(manifest) || !object(log)) {
    add('INVALID_SCHEMA', '$', 'Manifest and log must be JSON objects.');
    return result();
  }
  if (manifest.schemaVersion !== 1 || log.schemaVersion !== 1) {
    add('INVALID_SCHEMA', 'schemaVersion', 'Both schema versions must be 1.');
  }
  if (!text(manifest.protocolVersion)) {
    add('INVALID_SCHEMA', 'manifest.protocolVersion', 'A protocol version is required.');
  }
  if (log.protocolVersion !== manifest.protocolVersion) {
    add('PROTOCOL_VERSION_MISMATCH', 'log.protocolVersion', 'Log does not match the manifest protocol version.');
  }
  if (!['synthetic-fixture', 'observed-chat'].includes(log.dataKind)) {
    add('INVALID_SCHEMA', 'log.dataKind', 'Use synthetic-fixture or observed-chat.');
  }

  const prompts = uniqueRows(manifest.prompts, 'manifest.prompts');
  const conditions = uniqueRows(manifest.conditions, 'manifest.conditions');
  const planned = uniqueRows(manifest.plannedRuns, 'manifest.plannedRuns');
  if (prompts) for (const [id, prompt] of prompts) {
    if (!text(prompt.text)) add('INVALID_SCHEMA', `manifest.prompts.${id}`, 'Prompt text is required.');
  }
  if (conditions) for (const [id, condition] of conditions) {
    if (!text(condition.inputVersion) || !text(condition.inputText)) {
      add('INVALID_SCHEMA', `manifest.conditions.${id}`, 'inputVersion and inputText are required.');
    }
  }
  if (planned && conditions) for (const [id, run] of planned) {
    if (!conditions.has(run.condition)) add('INVALID_SCHEMA', `manifest.plannedRuns.${id}`, 'Unknown condition.');
    if (!['new', 'continued'].includes(run.historyMode)) add('INVALID_SCHEMA', `manifest.plannedRuns.${id}`, 'Specify new or continued historyMode.');
  }
  if (!Array.isArray(log.runs)) add('INVALID_SCHEMA', 'log.runs', 'Expected an array.');
  // Do not compare captures against an invalid or ambiguous protocol.
  if (issues.some(({ code }) => code === 'INVALID_SCHEMA' || code === 'DUPLICATE_ID')) return result();

  expectedTurns = planned.size * prompts.size;
  const runs = new Map();
  log.runs.forEach((run, index) => {
    const path = `log.runs[${index}]`;
    if (!object(run) || !text(run.id)) {
      add('INVALID_SCHEMA', path, 'Expected an object with a nonempty id.');
      return;
    }
    if (runs.has(run.id)) {
      add('DUPLICATE_RUN', path, `Duplicate run: ${run.id}`);
      return;
    }
    runs.set(run.id, run);
    if (!planned.has(run.id)) add('UNEXPECTED_RUN', path, `Run ${run.id} is not in the manifest.`);
  });

  for (const [id, expected] of planned) {
    const run = runs.get(id);
    const path = `log.runs.${id}`;
    if (!run) {
      add('MISSING_RUN', path, 'Planned run has no record.');
      continue;
    }
    const condition = conditions.get(expected.condition);
    if (run.condition !== expected.condition) add('CONDITION_MISMATCH', path, 'Condition differs from plan.');
    if (run.inputVersion !== condition.inputVersion) add('INPUT_VERSION_MISMATCH', path, 'Input version differs from plan.');
    if (!text(run.inputText) || fingerprint(run.inputText) !== fingerprint(condition.inputText)) {
      add('INPUT_TEXT_MISMATCH', path, 'Exact input snapshot differs from the manifest.');
    }
    if (run.historyMode !== expected.historyMode) add('HISTORY_MODE_MISMATCH', path, 'New versus continued history differs from plan.');
    if (!text(run.modelLabel)) add('MISSING_MODEL_LABEL', path, 'Record the visible model label, or explicitly say unknown.');
    if (!Array.isArray(run.turns)) {
      add('INVALID_SCHEMA', `${path}.turns`, 'Expected an array.');
      continue;
    }
    const seen = new Set();
    const actualOrder = [];
    for (const [index, turn] of run.turns.entries()) {
      const turnPath = `${path}.turns[${index}]`;
      if (!object(turn) || !text(turn.promptId)) {
        add('INVALID_SCHEMA', turnPath, 'Expected an object with promptId.');
        continue;
      }
      if (seen.has(turn.promptId)) {
        add('DUPLICATE_TURN', turnPath, `Duplicate prompt: ${turn.promptId}`);
        continue;
      }
      seen.add(turn.promptId);
      const prompt = prompts.get(turn.promptId);
      if (!prompt) {
        add('UNEXPECTED_TURN', turnPath, 'Prompt is not in the manifest.');
        continue;
      }
      actualOrder.push(turn.promptId);
      if (turn.promptText !== prompt.text) add('PROMPT_TEXT_MISMATCH', turnPath, 'Exact test question differs from the manifest.');
      if (turn.status === 'not-run' || turn.status === 'capture-missing') {
        add(turn.status === 'not-run' ? 'UNEXECUTED_TURN' : 'MISSING_CAPTURE', turnPath, 'Accounted for, but not a recorded reply.');
        if (!text(turn.note)) add('MISSING_NOTE', turnPath, 'Explain why the turn has no capture.');
      } else if (turn.status !== 'recorded') {
        add('INVALID_STATUS', turnPath, 'Use recorded, not-run, or capture-missing.');
      } else if (!text(turn.reply)) {
        add('EMPTY_REPLY', turnPath, 'A recorded reply must contain non-whitespace text.');
      } else {
        recordedTurns += 1;
        if (turn.generation !== 'first') add('NONFIRST_GENERATION', turnPath, 'This protocol requires the first generation. Keep retries separately.');
      }
    }
    for (const promptId of prompts.keys()) {
      if (!seen.has(promptId)) add('MISSING_TURN', path, `No entry for ${promptId}.`);
    }
    // Compare relative order even when one turn is missing; absence has its own error.
    const expectedOrder = [...prompts.keys()].filter((promptId) => seen.has(promptId));
    if (JSON.stringify(actualOrder) !== JSON.stringify(expectedOrder)) add('TURN_ORDER_MISMATCH', path, 'Recorded prompts are not in the planned order.');
  }
  return result();
}

const isMain = process.argv[1] && import.meta.url === pathToFileURL(resolve(process.argv[1])).href;
if (isMain) {
  const args = process.argv.slice(2);
  if (args.length !== 2) {
    console.error('Usage: node validate-roleplay-log.mjs manifest.json log.json');
    process.exitCode = 2;
  } else {
    try {
      const report = validate(...args.map((path) => JSON.parse(readFileSync(path, 'utf8'))));
      console.log(JSON.stringify(report, null, 2));
      process.exitCode = report.valid ? 0 : 1;
    } catch (error) {
      console.error(`Cannot read input: ${error.message}`);
      process.exitCode = 2;
    }
  }
}

The checks start with the manifest, then inspect each planned run. Duplicate IDs make a plan ambiguous, so an invalid manifest stops the comparison rather than producing a misleading partial pass.

Input fingerprints use SHA-256 over the UTF-8 snapshot string. A matching fingerprint helps compare exact text; it does not authenticate a capture, prove the model received that text, or protect a manifest that someone edited alongside the log. There is no signing or trusted remote source in this utility.

Run the passing and failing examples

Put the three files in one directory and run:

node validate-roleplay-log.mjs manifest.json pass.json

The actual output from running the supplied fixture is:

{
  "valid": true,
  "expectedTurns": 4,
  "recordedTurns": 4,
  "issues": []
}

Now make a deliberately broken copy. Save this short script as make-failure.mjs and run it once. It removes A1's second turn, changes A2's version label, and changes its actual character input.

import { readFileSync, writeFileSync } from 'node:fs';

const log = JSON.parse(readFileSync('pass.json', 'utf8'));
log.runs[0].turns.pop();
log.runs[1].inputVersion = 'iris-a-v2';
log.runs[1].inputText =
  'Iris Vale is an adult museum conservator. Keep the scene SFW. ' +
  'The cabinet contains a silver crown.';
writeFileSync('fail.json', JSON.stringify(log, null, 2) + '\n');

node make-failure.mjs
node validate-roleplay-log.mjs manifest.json fail.json

The failing fixture produced these results when executed:

{
  "valid": false,
  "expectedTurns": 4,
  "recordedTurns": 3,
  "issues": [
    {
      "code": "MISSING_TURN",
      "path": "log.runs.A1",
      "message": "No entry for Q2."
    },
    {
      "code": "INPUT_VERSION_MISMATCH",
      "path": "log.runs.A2",
      "message": "Input version differs from plan."
    },
    {
      "code": "INPUT_TEXT_MISMATCH",
      "path": "log.runs.A2",
      "message": "Exact input snapshot differs from the manifest."
    }
  ]
}

The CLI exits with 0 when the declared structure passes, 1 when validation finds issues, and 2 for unreadable input, invalid JSON, or incorrect command usage. That distinction makes it useful in a local pre-publication check or a repository workflow. It does not send data anywhere.

recordedTurns is a count of nonempty, uniquely keyed replies in planned runs—not a count of scientifically valid or successful trials. A run can contribute a recorded reply and still fail because its inputs changed. Use valid and the issue list together.

Add tests before relying on the check

Save this compact smoke test as validate-roleplay-log.test.mjs beside the other files:

import test from 'node:test';
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
import { validate } from './validate-roleplay-log.mjs';

const manifest = JSON.parse(readFileSync('manifest.json', 'utf8'));
const fixture = JSON.parse(readFileSync('pass.json', 'utf8'));

test('complete records pass without grading the prose', () => {
  assert.equal(validate(manifest, fixture).valid, true);
});

test('missing turns are visible', () => {
  const broken = structuredClone(fixture);
  broken.runs[0].turns.pop();
  const result = validate(manifest, broken);
  assert.ok(result.issues.some((issue) => issue.code === 'MISSING_TURN'));
});

test('an unchanged label cannot hide changed character text', () => {
  const broken = structuredClone(fixture);
  broken.runs[0].inputText += ' Decide the visitor\'s actions.';
  const result = validate(manifest, broken);
  assert.ok(result.issues.some((issue) => issue.code === 'INPUT_TEXT_MISMATCH'));
});

node --test validate-roleplay-log.test.mjs

The full development suite ran 23 checks with 23 passes and no failures. It also exercised duplicate runs and turns, unexpected prompts, reordered turns, empty replies, unexecuted turns, missing-capture notes, regenerated replies, history changes, malformed objects, ambiguous manifests, and CLI exit codes. Those are software checks. They are not 23 roleplay experiments or a model success rate.

Apply it to a CrushOn AI character test

Start with one practical question, such as “Does this version stop assigning actions to the visitor?” Use an original private SFW character in the CrushOn AI character editor and consult the basic creation guide for the current interface.

1. Preserve the current character fields and label them before editing. Keep the original snapshot even if you later improve the prompt.
2. Define the questions, run order, history condition, and first-generation policy. If the intended test has five questions, list all five in the manifest before collecting replies.
3. Capture full replies and the visible model/settings information. The manifest schema here stores a model label and history mode; retain other settings, context, and deviations in a companion protocol record.
4. Run the validator. Investigate missing or mismatched records instead of quietly excluding them.
5. Review behavior separately, attaching a verbatim excerpt to each judgment. A structural pass cannot decide whether a reply preserved agency, invented a fact, or maintained the character's voice.
6. Make one intended change and record a new input version. Preserve both runs; do not overwrite the baseline with the revision.

The existing Iris evidence package shows why the distinction matters. Its six conversations contained 30 first-generated replies, but five of the six final replies missed an exact two-sentence request. A complete record preserved that failure instead of hiding it behind a favorable example.

Those 30 replies share history within each conversation. They are not 30 independent experiments, and the study does not establish a winning prompt condition or long-term memory performance. The present two-question fixtures reuse the fictional museum setting for software demonstration only; they are not a conversion of that study's transcript.

Common questions

Does this work with an existing chat export

Not directly unless the export already matches this small schema. Write an adapter that preserves the original file, explicitly maps run and question IDs, and keeps omissions visible. Do not automatically assign IDs based only on array position if a missing message could shift every subsequent match. The Hugging Face personality workshop has its own manual worksheet format; this validator is not a native import feature for that workshop or for CrushOn AI.

Should regenerated replies be discarded

No. Keep them with their attempt information. This particular protocol asks for first generations, so it flags a regenerated reply in that slot. A regeneration study needs a different predeclared policy and an attempt-aware schema; removing unfavorable attempts would prevent a fair review.

Can the validator detect a fabricated transcript

No. It checks internal consistency, not provenance. A fully invented record can be internally consistent. Preserve original captures privately, record collection methods, and distinguish observed outputs from authored examples in public material.

Does a complete log prove the better prompt won

No. Completeness is necessary for an inspectable comparison, but it does not remove small-sample variation, hidden model changes, prompt-length confounds, or subjective scoring. Review dimensions separately and report the conditions under which the observation occurred.

What should be public

Publish only the fictional inputs, permitted reply records, declared settings, and relevant findings you intend to share. Remove private character links, account identifiers, personal names, and unrelated screenshots. A private character setting does not automatically make an exported log safe to publish.

Build a useful next test

CrushOn AI is a practical starting point when you want a hosted character editor and an immediate chat workflow without wiring an inference API. Use the editor to make a private test character, choose one behavior to investigate, and preserve both successful and unsuccessful first replies. Then use this local check to make sure the record is complete before deciding what to change.

Create a private SFW character on CrushOn AI, or inspect the published protocol and complete evidence before running your own comparison.