---
title: "How Axra's AI Agent Understands Photos, Voice, and Text"
canonical: "https://www.useaxra.com/blog/how-axra-ai-agent-understands-photos-voice-text"
updated: "2026-04-06T10:19:29.976Z"
type: "blog_post"
---

# How Axra's AI Agent Understands Photos, Voice, and Text

> Axra's AI agent processes text, images, and voice through a single unified pipeline across WhatsApp, Telegram, and web. This deep-dive explains the multimodal architecture.

## Key facts

- **Topic:** Multimodal AI banking
- **Published:** 2026-04-06
- **Reading time:** 3 min
- **Article sections:** 7
- **Covers:** multimodal AI banking, AI voice banking, image-based transfers, Claude Vision fintech and WhatsApp banking AI

## The Rise of Multimodal AI Banking

Banking has always been a text-heavy experience. Forms, account numbers, routing codes. But the real world does not operate in neat text fields. A shop owner in Lagos has bank details on a sticker. A driver in Nairobi confirms payment over a voice note. A freelancer in Accra sends an invoice screenshot.

Axra's AI agent processes all three input types -- text, images, and voice -- through a single unified pipeline.

## Architecture: One Pipeline, Every Channel

Every inbound message, regardless of channel, is normalized into a standard format by the Channel Gateway Service. Text, image buffers, and audio buffers all flow through the same processing pipeline.

```
WhatsApp Adapter ---+
Telegram Adapter ---+---> Channel Gateway ---> Media Processing ---> AI Agent
Web Chat Gateway ---+ |-- Image -> Axra's AI vision system
|-- Audio -> Axra's speech recognition engine -> Text
```

A feature built for one channel is instantly available on every channel.

## Image Processing: Axra's AI with Vision

When a user sends an image, the Media Processing Service downloads the buffer, validates its size (max 10MB), and stores an ephemeral copy in S3. The raw image buffer is then passed to Axra's AI as a multimodal content block.

The model extracts:

- **Account numbers**: Every digit of Nigerian NUBAN numbers (exactly 10 digits)
- **Bank names**: GTBank, Access, Zenith, First Bank, UBA, Kuda, OPay, and dozens more
- **Account holder names**: When visible on receipts or confirmations

### Intelligent Model Routing

Axra uses a three-tier routing system:

Images always route to the complex tier. Vision capabilities require a frontier model.

### Tool Chaining from a Single Image

When a user sends a photo with "send $10":

1. **Extract details**: Account number, bank name from image
2. **get_supported_banks("NG")**: Look up bank code
3. **create_beneficiary()**: Save recipient (account name resolved automatically)
4. **initiate_transfer()**: Start the transfer

Four tool calls from one photo and three words.

## Voice Processing: Axra's speech recognition

Voice notes are sent directly to the speech recognition API. No format conversion needed -- The engine accepts OGG, WebM, MP3, M4A, and WAV natively.

```typescript
const formData = new FormData();
formData.append('file', blob, 'audio.ogg');
formData.append('model', 'speech-v1');
formData.append('response_format', 'text');
```

The speech engine handles African English accents, code-switching between English and local languages, and noisy environments.

Once transcribed, the text goes through the same AI pipeline as typed messages.

## Privacy and Security

### Ephemeral Image Storage

Images are processed in-memory. An ephemeral copy is stored in S3 with AES-256 encryption for 24 hours, then automatically deleted.

### Account Number Masking

The AI extracts complete account numbers for tool calls but masks them in responses: `****6789`.

### Rate Limiting

- Text messages: 10 per minute per user
- Media messages: 3 per minute per user

## Cost Engineering

| Input Type | Model | Approximate Cost |
|---|---|---|
| Text (greeting) | a lightweight model | ~$0.001 |

The three-tier routing reduces AI spend by approximately 40% compared to routing everything through a single model.

## What This Means for Users

The technical complexity is invisible. Users see a WhatsApp conversation where they can:

- **Type** "send $20 to mom" and get a confirmation
- **Photograph** a bank sticker and have the transfer set up automatically
- **Record a voice note** saying "what's my balance" and get an instant answer

No app to download. No forms to fill. The AI meets users where they already are.

## Sources

- [How Axra's AI Agent Understands Photos, Voice, and Text](https://www.useaxra.com/blog/how-axra-ai-agent-understands-photos-voice-text)

---

Axra is a product of GoFree and is provided by GoFree Global Inc and its affiliated entities. Please check our FAQ page for information on which GoFree entity provides services in your region, or reach out via in-app chat or support@joingofree.com.

GoFree Global Inc is registered in Delaware, United States, and is registered as a Money Services Business (MSB) with the Financial Crimes Enforcement Network (FinCEN). Registration Number: 20222296774. License Number: 31000281485025.

GoFree Global Technology Limited is registered in Canada and is registered as an MSB and payment service provider with the Financial Transactions and Reports Analysis Centre of Canada (FINTRAC), with RPAA registration in progress with the Bank of Canada. Registration Number: 1001010436. License Number: C100000512.

The registered address for GoFree Global Inc is 1111B S Governors Ave STE 48051, Dover, DE 19904, United States. The registered address for GoFree Global Technology Limited is 2967 Dundas St. W. #1037, Toronto, ON M6P 1Z2, Canada.

Other operating entities include GoFree Global Technology Limited in Nigeria and GoFree Global Technology Limited in Rwanda. We are not a bank; banking services are provided by duly licensed partner banks, and deposits are FDIC insured where applicable.