Post
3,037
Maverick 1.1 is out.
When we released the first version, the feedback was clear: real people do not give one clean command at a time. They change their minds, they point and say "put that on the shelf", they ask for a lamp that is in another room, and sometimes the one speaking is a remote expert helping someone on site. 1.1 is built around that.
- Say "no, the other one" and it undoes the last action and repeats it on the other object. If the effect can't be undone, like a button that was already pressed, it tells you.
- It sees what people are looking at or pointing at, and who is speaking.
- It finds objects in other rooms, goes there and acts, or asks which one if there are two.
- It follows a short plan in one sentence: "open the cabinet, put the book and the alarm clock in it, then close it".
In the clip, a remote expert corrects the model, then asks it to highlight what the technician is looking at.
On test sets we sealed before training and ran once, the new abilities score 93 to 100%, where 1.0 scores 33 to 87%. On sentences written by people (the ALFRED set) the two versions are level, at 83.2% and 83.8%.
Where it still falls short: in unfamiliar situations it tends to act when it should ask, and a remote speaker can act in the scene with no permission check, so your app has to enforce that.
Same 4B model and licence (Apache-2.0), the same 11 tools plus
Model: ErenAta00/Maverick-4B-Unity-XR-Agent-1.1
Article and full demo: https://huggingface.co/blog/ErenAta00/maverick-1-1-undo-pointing-other-rooms
Built at the Extended Reality Laboratory (XRLab), Manisa Celal Bayar University:
ExtendedRealityLabMCBU
When we released the first version, the feedback was clear: real people do not give one clean command at a time. They change their minds, they point and say "put that on the shelf", they ask for a lamp that is in another room, and sometimes the one speaking is a remote expert helping someone on site. 1.1 is built around that.
- Say "no, the other one" and it undoes the last action and repeats it on the other object. If the effect can't be undone, like a button that was already pressed, it tells you.
- It sees what people are looking at or pointing at, and who is speaking.
- It finds objects in other rooms, goes there and acts, or asks which one if there are two.
- It follows a short plan in one sentence: "open the cabinet, put the book and the alarm clock in it, then close it".
In the clip, a remote expert corrects the model, then asks it to highlight what the technician is looking at.
On test sets we sealed before training and ran once, the new abilities score 93 to 100%, where 1.0 scores 33 to 87%. On sentences written by people (the ALFRED set) the two versions are level, at 83.2% and 83.8%.
Where it still falls short: in unfamiliar situations it tends to act when it should ask, and a remote speaker can act in the scene with no permission check, so your app has to enforce that.
Same 4B model and licence (Apache-2.0), the same 11 tools plus
undo, still offline on a 4 GB laptop GPU. The Unity package for the new scene format (1.1.0) is on GitHub: https://github.com/mcbu-xrlab/Maverick-UnityModel: ErenAta00/Maverick-4B-Unity-XR-Agent-1.1
Article and full demo: https://huggingface.co/blog/ErenAta00/maverick-1-1-undo-pointing-other-rooms
Built at the Extended Reality Laboratory (XRLab), Manisa Celal Bayar University:
- 5 replies