Accessibility
EchoSense
See the World, Hear the Details
EchoSense is an intelligent visual companion for people with visual or hearing impairments. Powered by AI that runs on your phone, it describes your surroundings, reads documents aloud, identifies currency, translates text, and helps you navigate — without sending your camera view to anyone.
Five tools, one tap apart
EchoSense puts five modes in the bottom toolbar so you can switch between them instantly.
Navigate — understand your surroundings
Point your camera at any scene and EchoSense describes what it sees: objects, layout, people, and how they sit relative to each other. Ask a targeted question by voice, such as where your keys are, and let it search the scene for you.
- Real-time scene description using on-device multimodal AI
- Voice commands for targeted object finding
- Object detection with spatial awareness
- Proximity alerts using depth sensing — LiDAR on iPhone, ARCore on supported Android devices
- Visual overlay showing detected object boundaries
Currency — identify bank notes instantly
Point the camera at a banknote and EchoSense announces the denomination aloud and on screen. Processing happens entirely on your device, so it works with no internet connection at all.
Read — access every word around you
A complete toolkit for getting text out of the physical world, whether that is a menu, a letter, a product label, or a full page of a document.
- Instant Text — continuous camera OCR that reads stable text aloud as you pan
- Guided Document Scan — spoken instructions help you align the page, then auto-capture with perspective correction and image enhancement
- Upload & Read — import images, PDFs, and text documents for extraction and reading
- Colour identification and light-level measurement
- Barcode and QR scanning, with links reported aloud rather than opened automatically
- Accessible Magnifier — full-screen zoom with freeze frame, high contrast, inverted colours, greyscale, and flashlight
- Text-recognition language selector, with auto-detect or a specific language for better accuracy
Assistant — a visual chat companion
Hold a natural, multi-turn conversation by text or voice. Share your camera view, attach an image or document, and ask follow-up questions — the assistant keeps the context of the conversation.
- Multi-turn text and voice chat with full conversation history
- Attach camera images, stored photos, PDFs, or text documents
- Switch between on-device AI and optional online AI using your own API key
- Streamed responses with Markdown formatting
- Automatic fallback to on-device processing when the network is unavailable
Translate — break down language barriers
Pull text out of your surroundings — a sign, a menu, an imported document — and translate it into English, on-device by default.
Your data stays yours
EchoSense is built for on-device processing from the ground up. No account is required to use the core features.
- The on-device model runs entirely on your phone — no images or prompts leave the device
- Optional online AI never connects without your explicit consent, and uses your own API key
- Before any online request, EchoSense shows you exactly what will be sent and asks permission
- Camera frames and analysis results are not intentionally retained after processing
- Conversation history stays in memory only while a chat is active
- Settings stay local, and API keys are held in the platform keychain or credential store
Powerful AI, completely offline
EchoSense's intelligence comes from a downloadable on-device model, managed from Settings. There is no account, no subscription, and no proprietary backend — model metadata is pinned to public download URLs.
- Gemma 3n E2B — the smaller, faster multimodal model, good for quick everyday responses
- Gemma 3n E4B — the larger multimodal model, for deeper analysis when you need it
- Additional text models are available, including Gemma 3 1B, Qwen2.5 1.5B, Phi-4-mini, and DeepSeek-R1-Distill
When you need more power, you choose
For tasks that benefit from a larger cloud model, you can optionally connect your own Gemini or OpenAI API key. Online analysis is always opt-in, and always per request.
- Connect Gemini or OpenAI with your own API key
- Per-request consent: EchoSense tells you what data will be sent before each online analysis
- Choose the default behaviour — ask first, prefer on-device, or prefer online when available
- Automatic fallback to on-device AI when the network is unavailable
Stay aware of your immediate surroundings
In Navigation mode, the Safety feature pairs object detection with depth sensing to warn you about nearby obstacles, announcing the object, its estimated distance, and its bearing.
- iPhone: uses the LiDAR depth sensor for metric distance estimation
- Android: uses the ARCore Depth API on supported devices
- Devices without depth hardware fall back to camera-relative proximity estimates
- A visual overlay marks detected obstacle boundaries
Safety and proximity features are experimental estimates. EchoSense is not a certified mobility aid and should not replace a cane, guide dog, or other established safety practices.
Built for accessibility by design
Accessibility is the primary goal of every feature, not a layer added afterwards.
- Full VoiceOver and TalkBack compatibility across all modes
- Spoken feedback for every analysis result
- Voice commands for hands-free operation
- Accessible Magnifier with zoom, freeze, high contrast, colour inversion, greyscale, and flashlight
- Large, high-contrast buttons with clear labels
- Customisable text overlay preferences
Compatibility & requirements
Android
- Android 12 (API 31) or newer
- ARM64 device (arm64-v8a)
- 6 GB RAM recommended
- Several GB of free storage for the model download
- Camera required for visual features
- An ARCore-supported device is recommended for depth-based Safety features
iOS
- iPhone with a camera
- LiDAR-equipped models give the most accurate distance estimates in Safety
- Several GB of free storage for the model download