AffectFlow-DINO: Uncertainty-Aware Multi-Task Affect Estimation via Conditional Rectified Flow
Abstract
We present AffectFlow-DINO, a multi-task learning system for the 11th ABAW challenge that extends a standard deterministic architecture with a conditional rectified-flow head to model the inherent ambiguity of in-the-wild facial behavior. Instead of predicting a single affect estimate, the model learns a conditional generative distribution, enabling uncertainty-aware one-to-many predictions through Monte Carlo sampling. The system jointly estimates continuous valence-arousal, classifies eight facial expressions, and detects twelve Action Units from static face images. Built on a frozen DINOv3 ViT-S/16 backbone, extensive ablation studies show that rectified-flow decoding consistently improves deterministic prediction, particularly for valence-arousal estimation (CCC-V +0.058). We further show that post-hoc threshold calibration effectively recovers performance on severely imbalanced rare classes (e.g., Fear: 3.8% rightarrow 33.1%) without retraining. Combined with backbone fine-tuning and flow retuning, the final model achieves P_{MTL=1.177}, substantially outperforming the official challenge baseline of P_{MTL}=0.45.
Community
AffectFlow-DINO models in-the-wild facial affect as a conditional distribution p(y|x), not a single point estimate.
Built on DINOv3, it jointly predicts valence–arousal, 8 expressions, and 12 Action Units, and adds a conditional rectified-flow head for uncertainty-aware, one-to-many predictions on ambiguous faces.
On the 11th ABAW MTL validation set we reach P_MTL = 1.177 (vs. official baseline 0.450), with ablations showing when flow decoding helps and how simple post-hoc calibration recovers rare classes (e.g. Fear F1 3.8% → 33.1%).
Code, pretrained weights, and inference scripts are open:
• Paper: https://arxiv.org/abs/2607.13250
• Code: https://github.com/Bekhouche/AffectFlow-DINO
• Models: https://huggingface.co/Bekhouche/AffectFlow-DINO
Try it in one line:
python inference.py --model finetune-flow-retune-b10 --image face.jpg
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- Task-Specific Feature Fusion Method for Multi-Task Affective Behavior Analysis (2026)
- Causal Supervision of Attention for Affective Behaviour Analysis (2026)
- The Circumplex Degeneracy Behind the Rare-Class Limit in Affect Recognition (2026)
- Learning from Annotation Uncertainty: Entropy-Aware Curriculum for Speech Emotion Recognition (2026)
- SURGELLM: Rethinking Multi-Task Evaluation through Task-Aware Feature Gating with Class-Balanced Normalization (2026)
- TabPFN beyond Tabular Data: Calibration and Accuracy on Multimodal Embeddings (2026)
- What Makes Video World Model Latents Action-Relevant: Prediction over Reconstruction (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Get this paper in your agent:
hf papers read 2607.13250 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 1
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper