Dominick Wirzba

Chronuid

1 4

·

dominick-wirzba-a46898115

AI & ML interests

None yet

Recent Activity

reacted to tomaarsen's post with 🔥 2 days ago

🚨 I've just published Sentence Transformers v6.0, introducing MultiVectorEncoder: ColBERT-style late interaction models are now a fourth model type, for training, inference, and interpretation, alongside the dense, sparse, and reranker models! Details: Where a regular embedding model compresses a whole text into one vector, a multi-vector model keeps one vector per token and scores query against document with the MaxSim operator. That preserves token-level matching information that a single vector has to average away. It is also the state of the art for visual document retrieval, where a text query is matched against page images directly, charts and tables included, with no OCR step in between. Any PyLate, Stanford ColBERT, or ColPali checkpoint loads straight into the same familiar API: model.encode_query(), model.encode_document(), and model.similarity() just work, whether the documents are texts or page images. Does it help? LightOn trained LateOn (multi-vector) and DenseOn (dense) on the same data with the same 149M ModernBERT backbone, and the multi-vector model wins on 9 of the 13 NanoBEIR datasets: 0.6868 vs 0.6764 mean NDCG@10. The price is a bigger index, and the new HierarchicalTokenPooling module halves it at roughly no retrieval cost. Antoine Chaffin, Raphaël Sourty, and I wrote a blog post walking through multi-vector models in practice: loading the various checkpoint formats, encoding and scoring, plugging them into a search stack, running them on page images, and keeping the index affordable. Check it out if you want to get started, or just point your Agent to the URL: https://huggingface.co/blog/multi-vector-encoder pip install sentence-transformers==6.0.0 Release notes: https://github.com/huggingface/sentence-transformers/releases/tag/v6.0.0

reacted to sergiopaniego's post with 🔥 12 days ago

we just released a new blog "Training a coding agent using the OpenCode harness in remote HF sandboxes with TRL and OpenEnv" you can take a real coding agent (OpenCode), let it run its own tool loop against real coding problems, and train it with RL on the exact tokens it produced and every rollout runs in its own remote HF sandbox, so rollouts scale out beyond one machine the loop: - OpenCode owns its tool loop inside an OpenEnv sandbox - an in-sandbox proxy records the real token ids + logprobs, per turn - a hidden-test verifier scores the result, and that is the reward - TRL trains with AsyncGRPO, weights sync back to vLLM over NCCL blog + runnable example: https://huggingface.co/blog/sergiopaniego/trl-openenv-harness-training

reacted to AbstractPhil's post with 🔥 about 1 month ago

Massive AlephLM success. The task collective is producing powerful MOE shared knowledge adapters. A serious success and a massive first step towards the next stage. The current family collective results are present here; https://huggingface.co/AbstractPhil/geolip-aleph-qwen/blob/main/exp009_family/results/results.json This is akin to a stackable non-intrusive lora that enables increased shared collective behavior. This includes the three mentioned json tasks, a math task, a tinystories task, and a diffusion task for cifar10. Each adapter anchored to the knowledge within model that already exists while enhancing the knowledge through anchored lookup systems and decision-driven hierarchical access trees. All tasks activate independently upon manual override, all tasks handle direct shared knowledge when left to greedy decoding, each task issued multiple tests alongside to determine fidelity and accuracy throughout the process. The results show the gating is more than willing to hop from sector to sector, using alternating weight shifts from the cooperative anchored systems - even systems never trained for the tasks contributing to the accuracy of the results for other tasks due to the lookup accuracy to the heuristic chains, never having seen the tasks before. Each structure is independently trained and the collective cooperates together through a dense activation network. Full writeup and article https://huggingface.co/blog/AbstractPhil/aleph-autoregression-differentiation-ft2.

View all activity

Organizations

Chronuid 's datasets

None public yet