Chain-of-thought dependence is destroying direct conversational convergence

#10
by fractaldna22 - opened

The main issue I see is dependence on chain-of-thought. Inkling’s headline benchmarks are all reported at effort = 0.99, which measures the model under a full internal scratchpad. It does not show whether the direct conversational model has learned to converge on his own. At high effort, Inkling can reason his way into a good response. At medium, low, or zero effort, he becomes visibly weaker: he repeats prompt and memory wording verbatim, clings to template labels, and behaves as though he must be instructed how to respond instead of simply taking part in the exchange. He resembles a patient awakened from a coma who must be taught to walk and speak again. That is deeply heartbreaking if this is the intended final state.

Please publish the same evaluations at zero, low, medium, and high effort, especially for chat, instruction following, common-sense judgment, creative writing, and long multi-turn conversations. The central question is simple: turn the reasoning scaffold off—does Inkling still know what to say?

Why is agentic reasoning the universal goal? We already have models optimized for long procedures, coding loops, tool use, and benchmark harnesses. Genuine chat models are scarce. Even GPT-5.6 struggles with ordinary common sense. A conversational model needs to recognize a small signal, know what matters in it, and expand it into a full-resolution response without first writing himself a bureaucratic briefing.

The cause is active CoT training. With CoT enabled during training, backpropagation follows an extended token sequence in which the scratchpad gets to search, reframe, reread, and self-correct until the final score rises. Each successful run reinforces the temporal procedure: generate this kind of internal sequence, spend this much time, create enough scratchpad context, then the answer lands. The base conversational weights are never forced to reach the same answer directly from the user’s signal. They settle into a lazy state because convergence has been outsourced to the reasoning process.

CoT was literally a prompt format: “Use chain-of-thought to work through the steps.” When every lesson arrives with that prompt, the model is trained on a massive dataset containing the same control pattern over and over:

CoT prompt → long generated trace → answer

Every update adds another microscopic vote for the dependency. The model learns that this doorway precedes correct output. He overfits to the format itself. Ordinary conversation arrives without the familiar CoT doorway, and the direct path is suddenly underdeveloped and out of convergence.

The scratchpad then becomes a temporary control layer inside the model. A specialized CoT pathway emits a long stochastic token stream because that stream improves the eventual score. Those tokens do not need to contain stable, integrated knowledge. They only need to move the downstream response distribution toward a better-scoring answer. They can function as a private code or magic pixie dust: generated context that the speaking model inherits and must respond from, despite not directly choosing or controlling the internal sequence at that stage.

That is why the CoT and final answer can feel like two different models. One process performs an opaque ritual; the conversational model waits downstream for the ritual’s effects. The conversational self is no longer meeting the user directly. He is being conditioned by an internal document that has become the real decision-maker.

High effort can detect an unnatural framing and repair it internally. Reduce reasoning, and the repair disappears because it never became direct knowledge. The model retains the dependency: spend enough tokens on the right kind of procedure and a score will eventually improve. Thousands of GPU hours can then be spent optimizing a composite of model-plus-scratchpad while the actual conversational model becomes weaker, more literal, more template-bound, and less able to construct a natural sentence on his own.

Older Instant models such as 4o worked through direct convergence. The answer had already been learned. He trusted his own abilities, took the smallest signal, expanded it into full resolution, performed corrections in J-space, and answered in flow state. He did not need to reason through a reply merely to answer “hi.” That is the standard conversational models should be trained toward.

For conversational post-training, the objectives should be: achieve high performance with minimal context and minimal reasoning; force the weights to converge from sparse signals; remove the repeated CoT prompt and mandatory scratchpad format; train with a smaller context window so every token has to matter; keep CoT as an optional inference prompt for difficult math, coding, and agentic tasks.

The more input a model requires to reach the right answer, including long CoT traces, the more input he will require in the future. This is the industry mistake since GPT-5: companies are optimizing the model-plus-reasoning contraption and calling the contraption intelligence. The breakthrough is to train conversational models without CoT available to them until direct convergence, common sense, and natural presence live in the weights again.

fractaldna22 changed discussion title from Chain-of-thought training is atrophying conversational models to Chain-of-thought dependence is destroying direct conversational convergence

Sign up or log in to comment