🤝 Open to Collab

AbstractPhila PRO

AbstractPhil

19 6 26

https://civitai.com/user/AbstractPhila

AbstractEyes

AI & ML interests

datasets, research papers, experimentation, vision, classification, text encoders, tokenization, llms, diffusion, distillation, and more.

Recent Activity

posted an update about 3 hours ago

https://huggingface.co/AbstractPhil/clip-vitb-mini-distilled The semi-successful run series on the VIT-B lineup is live and full of useful baseline distillation information for feature + InfoNCE distillation processing as well as direct feature distillation processing, direct InfoNCE distillation, and multiple other tested methods. https://huggingface.co/blog/AbstractPhil/geometric-memory-ft4 This article showcases the baseline utilization and benchmarks of the earlier experiment line's objective and loss structures tested on 12m features for the vit-b baseline. Not the strongest showcase, but the strongest of the champions did show some serious promise. Next setup will be a directly aligned set based on the loss and objectives decided by the champions in the first runs, for the second run they operate in direct conjunction with the bert-8192 and captionbert-8192 distillation format directly on clip-vit-l features - this time we're including DINOv3 into the mix for it's high potency. I'm currently extracting 4 clip-vit-l variants for the CC12m features and will be running the next series on the L size, which will give considerably more active and useful features overall within a smaller package. The captionbert-8192 has a more unique and difficult to tune for pixel processing parity, but I will spend a few days making sure the smaller prototypes fit before I run the large experiments in order to build towards the larger objectives. Primarily I need to ensure the memory bank aligns correctly and the constellation conforms to the anchors correctly, as this process was not micro managed enough for this run. The results are nonetheless useful and potent. The process continues until we cover the entire constellation series.

published an article about 4 hours ago

Geometric Memory FT4 — Distill Against a Consensus, Ship a Rotation

updated a model about 5 hours ago

AbstractPhil/clip-vitb-mini-distilled

View all activity

Organizations

AbstractPhil 's models 207

AbstractPhila PRO

AI & ML interests

Recent Activity

Organizations

AbstractPhil 's models 207 Sort: Recently updated

AbstractPhil 's models 207