Instructions to use Multilingual-Multimodal-NLP/LoopCoder-V2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Multilingual-Multimodal-NLP/LoopCoder-V2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Multilingual-Multimodal-NLP/LoopCoder-V2", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Multilingual-Multimodal-NLP/LoopCoder-V2", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Multilingual-Multimodal-NLP/LoopCoder-V2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Multilingual-Multimodal-NLP/LoopCoder-V2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Multilingual-Multimodal-NLP/LoopCoder-V2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Multilingual-Multimodal-NLP/LoopCoder-V2

SGLang

How to use Multilingual-Multimodal-NLP/LoopCoder-V2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Multilingual-Multimodal-NLP/LoopCoder-V2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Multilingual-Multimodal-NLP/LoopCoder-V2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Multilingual-Multimodal-NLP/LoopCoder-V2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Multilingual-Multimodal-NLP/LoopCoder-V2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Multilingual-Multimodal-NLP/LoopCoder-V2 with Docker Model Runner:
```
docker model run hf.co/Multilingual-Multimodal-NLP/LoopCoder-V2
```

Bug: Missing modeling_iquestpltcoder.py file prevents model from loading

by JohnMolotov - opened 3 days ago

Discussion

JohnMolotov

3 days ago

Hi there,

Thank you for releasing the LoopCoder-V2 model!

It looks like the repository is currently missing the core modeling file required to initialize the custom architecture. While configuration_iquestpltcoder.py and tokenization_iquestcoder.py are present, modeling_iquestpltcoder.py appears to have been omitted during the upload.

When attempting to load the model via standard HF transformers or vLLM with trust_remote_code=True, it fails at architecture resolution because the AutoModelForCausalLM hook has no blueprint to construct the layer weights.

Could you please upload modeling_iquestpltcoder.py to the repository? Thank you!

devo111

3 days ago

I ran into the same problem. There's a YouTube video where the model is being tested, but I think he's running the model in V1, not V2. https://www.youtube.com/watch?v=ruxvS5Bd3mU

csjiaya

Multilingual-Multimodal-NLP org 3 days ago

Sry for inconvenience, we will release the codebase for inference of v2 version

csjiaya

Multilingual-Multimodal-NLP org 1 day ago

Hi, we updated the readme, and users can use the vLLM branch for inference.

For vLLM inference, install vLLM from yxing-bj/vllm and use transformers==4.57.1, then start the server with the following command:

vllm serve $MODEL --port 8080 \
    --max-num-batched-tokens 8192 --max-num-seqs 512 -tp 1 -dp 1 --trust-remote-code \
    --cudagraph-capture-sizes 1 2 4 8 12 16 24 32

cse2011

1 day ago

for this issue reported here , any clue ?
https://huggingface.co/Multilingual-Multimodal-NLP/LoopCoder-V2/discussions/2#6a37afe84fd6509975343085

zwpride

Multilingual-Multimodal-NLP org 1 day ago

We sincerely apologize, but support for the Transformers implementation will be added in a future update. For now, we recommend using the specified version of vLLM for inference. Thank you very much for trying our model.

for this issue reported here , any clue ?
https://huggingface.co/Multilingual-Multimodal-NLP/LoopCoder-V2/discussions/2#6a37afe84fd6509975343085

JohnMolotov

about 16 hours ago

•

edited about 16 hours ago

Thank you for the quick response, yxing-bj/vllm works great.

With a little patch for quantization I've even been able to get it running in under 8GB VRAM! I'm excited to see how it performs on real tasks.

JohnMolotov changed discussion status to closed about 16 hours ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment