Teaching a Tiny Model When to Shut-Up
Helping open-weights AI models learn about double texts, keeping quiet, and when “yeah” means keep going.
This is an active WIP as part of a tooling suite I’m building at curvelabs.org to shape agent personality and behavior: https://github.com/Jiply/tiny-interjection-model-alpha
Agents tend to experience conversations with humans as a sequence of completed messages. Whether it’s a coding agent or friendly companion, the flow is for a message to arrive, for inference to occur, and for a response to appear.
Human conversation has softer edges.
We pause halfway through thoughts, send a sentence across several messages, and correct others while they are speaking. We say “yeah” to invite our conversation partners to continue. Meaning then emerges through timing, sequence, and the changing shape of a given exchange. These temporal nuances seem to matter.
I wanted to explore whether a small model could notice some of this.
Introducing: Tiny Interjection Model (Alpha): a small floor-control model for typed chat. It reads a timestamped event stream and estimates what an agent should do next.
Its vocabulary is intentionally small. It can wait when another message seems likely, respond when the user appears ready, interject when the user redirects an answer in progress, or continue when the assistant is partway through an existing response. When useful, it can produce a short message alongside the decision.
The broader idea is to separate conversational timing from conversational intelligence.
A large model might, and often should remain responsible for reasoning, knowledge, and expression, while a smaller model pays attention to local rhythm surrounding each turn. This creates a lightweight layer between the interface and the main assistant, where timing decisions can happen quickly and cheaply.
Zz, it’s data all the way down.
Typed conversational floor control does not yet have an obvious standard dataset. I created a synthetic one using Qwen3.5 397B A17B using DigitalOcean’s Inference (thank you for supporting our work at Curve Labs through Hatch).
The teacher model generated timestamped conversations consisting of interruptions, incomplete thoughts, corrections, acknowledgements, delayed replies, and multi-part responses. Generation gradually became a questions of timing, semantic continuity, and whether a short message changes the expected direction of an exchange.
Each candidate passed through validation covering schema, action semantics, class balance, duplicates, prompt diversity, timing distribution, and common synthetic placeholders.
The resulting dataset contains 702 approved examples, divided almost evenly across the four actions. Of these, 598 became training examples, 104 became validation examples.
A separate 40-case benchmark stayed outside the training process. It became a small, stable window into how the model’s behavior changed as the dataset and training setup evolved.
Shoutout Nebius for Credits :D
The base model is Qwen3 4B Instruct, fine-tuned with completion-only QLoRA. The configuration uses 4-bit NF4 quantization, LoRA rank 16, LoRA alpha 16, and a maximum sequence length of 4,096 tokens.
The training workflow is packaged as a containerized Nebius Serverless AI Job. It requests an NVIDIA L40S GPU (gpu-l40s-a, 1gpu-8vcpu-32gb).
Inside the job, the container downloads an immutable dataset revision, evaluates the base model, trains the adapter, evaluates the resulting behavior, merges the model, converts it into GGUF, and preserves experiment artifacts.
It’s early. Pls, I did this in <2 days.
The first focused run used 500 examples. It reached full schema validity, 92.5% action accuracy, and 46.875% expected-content accuracy. Expanding the dataset to 702 examples moved expected-content accuracy to 68.75%.
These numbers describe a model learning the broad rhythm of the task more quickly than its finer semantic details. Choosing an action is one problem. Preserving the exact content needed for the accompanying message appears to be another.
The current release threshold is 95% expected-content accuracy. That threshold serves as a direction for future experiments, which will be increasingly focused on subtle continuations, content-preserving corrections, and cases where a tiny change in wording alters the flow of the conversation.
The model, dataset, and benchmark are co-evolving together. Generally, it seems to be an evolving observation that the small model understands the conversational move while losing a particular name, date, or instruction in its response.
TLDR:
The largest model in a given system may not need to make every decision.
Conversation contains many small judgments that depend on recent context rather than broad knowledge. Is the user still typing their thought? Did “okay” close the exchange or invite a follow-up? Did the latest message add detail, replace an instruction, or ask the assistant to stop?
The Tiny Interjection Model is an early exploration these questions. The code, dataset pipeline, evaluation framework, Nebius job configuration, and current experiment results are available on GitHub.
Drop me a DM on X or LinkedIn to discuss more!
Posted from @fdotinc SF for #NebiusServerlessChallenge