ACT — Abstract Compositional Transformer (RAVEN)

ACT is a transformer-based model for Raven's Progressive Matrices (RPM), described in Staged Self-Supervised Learning for Raven Progressive Matrices (IEEE TNNLS, 2025) and Learning Abstract Visual Reasoning via Task Decomposition: A Case Study in Raven Progressive Matrices (AMCS, 2024). For a RAVEN task it predicts a property vector for each panel (arrangement, object presence, type, size, colour, position) — the first stage of the framework described in those publications. The choice makers built on top of ACT (DCM, LCM, CCM) are part of the codebase.

This repository contains the ACT model with the Task tokenizer and Combined masking (Keras/TensorFlow SavedModel), used in the main experiments reported in those publications.

Usage

import os
os.environ["KERAS_BACKEND"] = "tensorflow"
import keras

model = keras.saving.load_model("hf://jkwiatkowski/raven")

The model follows the ACT Task-tokenizer pipeline: 9 panels (8 context + query), 160×160 px, are resized to 84×84 and arranged into a 252×252 task image, encoded by EfficientNetV2B0, processed by the transformer and decoded into property predictions. Input preparation and evaluation utilities are in raven_tools and the main repository.

Training and evaluation

Trained on the RAVEN training split (42,000 tasks) and validated on 14,000 tasks; evaluated on RAVEN, I-RAVEN and RAVEN-FAIR (see the publications). The RAVEN data in Hugging Face format: jkwiatkowski/raven.

Intended uses and limitations

• Research model for RAVEN-style RPM tasks; not a general-purpose vision model.
• Predicts RAVEN-specific property vectors; solving choice tasks requires the choice-maker code from the main repository.
• Trained on RAVEN only — performance may differ on other RPM distributions.

Links

• Code: https://github.com/jakubkwiatkowski/abstract_compositional_transformer
• Utilities: https://github.com/jakubkwiatkowski/raven_tools
• Dataset: https://huggingface.co/datasets/jkwiatkowski/raven
• Demo: https://huggingface.co/spaces/jkwiatkowski/raven

Citation

If you use this model, please cite:

• J. Kwiatkowski, K. Krawiec, "Staged Self-Supervised Learning for Raven Progressive Matrices," IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 9, pp. 16564–16576, 2025. https://ieeexplore.ieee.org/abstract/document/11014503
• J. Kwiatkowski, K. Krawiec, "Learning Abstract Visual Reasoning via Task Decomposition: A Case Study in Raven Progressive Matrices," International Journal of Applied Mathematics and Computer Science, vol. 34, no. 2, pp. 309–321, 2024. https://doi.org/10.61822/amcs-2024-0022