Feature
Multimodal AI in ClawAI: voice, video and vision
How ClawAI handles voice notes, video notes and images: transcription, sampled video frames, a vision helper for text-only models, and routing by attachment type.
All features · Last reviewed:
Voice and video notes from the composer
The composer has a record button for voice and video notes. It asks for permission first, shows a live waveform while you record, and caps one recording at five minutes. The recording is transcribed — Gemini is tried first and OpenAI Whisper is the fallback — and the answering model is told the message arrived as a voice note, so it replies to what you said rather than to a file.
Audio you already have works the same way: WebM, OGG, MP3, MP4 and M4A, WAV, FLAC and AAC uploads are transcribed before they reach the model.
Video the model can actually follow
An uploaded video is transcribed with timestamps, and up to six frames per video are sampled and shown to the model alongside the transcript, so it can answer questions about what happens on screen as well as what is said. A silent video is reported as having no speech instead of producing an empty transcript, and you can stop processing a long video at any point.
Every plan has a video-length allowance set by the operator; independent of plan, one video can never exceed thirty minutes or 4K resolution. Supported containers are MP4, MOV, WebM, AVI and MPEG.
A vision helper, and routing by attachment type
Not every model can see. When the model answering a message is text-only, a second model can describe up to four images for it, including any text in them, and the answering model is told it is working from a description. In Local-Only and privacy-first conversations, only local helpers served through Ollama or llama.cpp are used.
In Auto mode, the router also ranks candidate models by how well they handle the kind of attachment in the message, so an image, a PDF or a video tends to land on a model that accepts it natively rather than relying on the helper.
Questions people ask
- How long can a voice or video note be?
- One recording made in the composer can run for up to five minutes. Uploaded videos are limited by your plan’s video-length allowance, and never beyond thirty minutes or 4K resolution on any plan.
- What happens if I send an image to a model that cannot see images?
- If the vision helper is enabled on your plan, a vision-capable model describes the image and transcribes its text, and the answering model works from that description, knowing it is a description rather than the image itself.
- Does ClawAI pick a different model because of my attachment?
- In Auto mode, yes: the router ranks candidates by how well they handle the attachment type. If you pin a model yourself, your choice is kept and the vision helper fills the gap where it is enabled.
Try it rather than take our word for it
Voice and video notes, transcription and modality-aware routing are shipped; the vision helper is a plan feature that an operator switches on by assigning a helper model.