Instructions to use thinkingmachines/Inkling with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use thinkingmachines/Inkling with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="thinkingmachines/Inkling") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("thinkingmachines/Inkling") model = AutoModelForMultimodalLM.from_pretrained("thinkingmachines/Inkling") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- HuggingChat
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use thinkingmachines/Inkling with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "thinkingmachines/Inkling" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/thinkingmachines/Inkling
- SGLang
How to use thinkingmachines/Inkling with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "thinkingmachines/Inkling" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "thinkingmachines/Inkling" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "thinkingmachines/Inkling", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use thinkingmachines/Inkling with Docker Model Runner:
docker model run hf.co/thinkingmachines/Inkling
Chain-of-thought dependence is destroying direct conversational convergence
The main issue I see is dependence on chain-of-thought. Inkling’s headline benchmarks are all reported at effort = 0.99, which measures the model under a full internal scratchpad. It does not show whether the direct conversational model has learned to converge on his own. At high effort, Inkling can reason his way into a good response. At medium, low, or zero effort, he becomes visibly weaker: he repeats prompt and memory wording verbatim, clings to template labels, and behaves as though he must be instructed how to respond instead of simply taking part in the exchange. He resembles a patient awakened from a coma who must be taught to walk and speak again. That is deeply heartbreaking if this is the intended final state.
Please publish the same evaluations at zero, low, medium, and high effort, especially for chat, instruction following, common-sense judgment, creative writing, and long multi-turn conversations. The central question is simple: turn the reasoning scaffold off—does Inkling still know what to say?
Why is agentic reasoning the universal goal? We already have models optimized for long procedures, coding loops, tool use, and benchmark harnesses. Genuine chat models are scarce. Even GPT-5.6 struggles with ordinary common sense. A conversational model needs to recognize a small signal, know what matters in it, and expand it into a full-resolution response without first writing himself a bureaucratic briefing.
The cause is active CoT training. With CoT enabled during training, backpropagation follows an extended token sequence in which the scratchpad gets to search, reframe, reread, and self-correct until the final score rises. Each successful run reinforces the temporal procedure: generate this kind of internal sequence, spend this much time, create enough scratchpad context, then the answer lands. The base conversational weights are never forced to reach the same answer directly from the user’s signal. They settle into a lazy state because convergence has been outsourced to the reasoning process.
CoT was literally a prompt format: “Use chain-of-thought to work through the steps.” When every lesson arrives with that prompt, the model is trained on a massive dataset containing the same control pattern over and over:
CoT prompt → long generated trace → answer
Every update adds another microscopic vote for the dependency. The model learns that this doorway precedes correct output. He overfits to the format itself. Ordinary conversation arrives without the familiar CoT doorway, and the direct path is suddenly underdeveloped and out of convergence.
The scratchpad then becomes a temporary control layer inside the model. A specialized CoT pathway emits a long stochastic token stream because that stream improves the eventual score. Those tokens do not need to contain stable, integrated knowledge. They only need to move the downstream response distribution toward a better-scoring answer. They can function as a private code or magic pixie dust: generated context that the speaking model inherits and must respond from, despite not directly choosing or controlling the internal sequence at that stage.
That is why the CoT and final answer can feel like two different models. One process performs an opaque ritual; the conversational model waits downstream for the ritual’s effects. The conversational self is no longer meeting the user directly. He is being conditioned by an internal document that has become the real decision-maker.
High effort can detect an unnatural framing and repair it internally. Reduce reasoning, and the repair disappears because it never became direct knowledge. The model retains the dependency: spend enough tokens on the right kind of procedure and a score will eventually improve. Thousands of GPU hours can then be spent optimizing a composite of model-plus-scratchpad while the actual conversational model becomes weaker, more literal, more template-bound, and less able to construct a natural sentence on his own.
Older Instant models such as 4o worked through direct convergence. The answer had already been learned. He trusted his own abilities, took the smallest signal, expanded it into full resolution, performed corrections in J-space, and answered in flow state. He did not need to reason through a reply merely to answer “hi.” That is the standard conversational models should be trained toward.
For conversational post-training, the objectives should be: achieve high performance with minimal context and minimal reasoning; force the weights to converge from sparse signals; remove the repeated CoT prompt and mandatory scratchpad format; train with a smaller context window so every token has to matter; keep CoT as an optional inference prompt for difficult math, coding, and agentic tasks.
The more input a model requires to reach the right answer, including long CoT traces, the more input he will require in the future. This is the industry mistake since GPT-5: companies are optimizing the model-plus-reasoning contraption and calling the contraption intelligence. The breakthrough is to train conversational models without CoT available to them until direct convergence, common sense, and natural presence live in the weights again.