Captain Hope: An Offline AI-Powered Wearable Mobility Assistant for the Visually Impaired

Captain Hope: An Offline AI-Powered Wearable Mobility Assistant for the Visually Impaired | Authorea try { document.documentElement.classList.add('js'); } catch (e) { } var _gaq = _gaq || []; _gaq.push(['_setAccount', 'G-8VDV14Y67G']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); Skip to main content Preprints Collections Wiley Open Research IET Open Research Ecological Society of Japan All Collections About About Authorea FAQs Contact Us Quick Search anywhere Search for preprint articles, keywords, etc. Search Search ADVANCED SEARCH SCROLL This is a preprint and has not been peer reviewed. Data may be preliminary. 25 September 2025 V1 Latest version Share on Captain Hope: An Offline AI-Powered Wearable Mobility Assistant for the Visually Impaired Authors : Anish Kumar Sidhi 0009-0003-5037-668X [email protected] , Suresh Pabboju , and P. Kiranmaie Authors Info & Affiliations https://doi.org/10.22541/au.175882486.65199983/v1 304 views 168 downloads Contents Abstract Information & Authors Metrics & Citations View Options References Figures Tables Media Share Abstract Wearable assistive technologies for the visually impaired have faced a critical trade off: traditional aids offer limited environmental context, while advanced electronic solutions are frequently tethered to the cloud, compromising real-time performance and user privacy. Furthermore, few systems integrate rich conversational feedback for social awareness. This paper addresses this gap by presenting “Captain Hope,” an innovative shoulder-mounted wearable device. We demonstrate the feasibility of integrating a local large language model (LLM) with real-time computer vision for concurrent social and navigational cueing on a low-cost, fully offline wearable device. Powered by a Raspberry Pi 5 and an efficient YOLOv8n model, the system achieves an average end-to-audio latency of 485 ms, validating the real-time viability of our on-device LLM integration. A pilot user study (n=5) yielded an 89.2 percent detection accuracy in dynamic environments and, critically, a 70 percent increase in navigational confidence, underscoring the qualitative value of locally-processed, context-aware audio cues. By presenting a complete system blueprint and its real-world performance benchmarks, this paper offers a significant contribution toward creating accessible, self-reliant assistive technologies that improve quality of life. Captain Hope: An Offline AI-Powered Wearable Mobility Assistant for the Visually Impaired Authors: Anish Kumar Sidhi Artificial Intelligence & Robotics M.Tech Student, Department of IT, Chaitanya Bharathi Institute of Technology (CBIT), Hyderabad, India Email: [email protected] Dr. Suresh Pabboju Dept. of Information Technology, Chaitanya Bharathi Institute of Technology, Telangana, India Email: [email protected] Ms. P. Kiranmaie Dept. of Information Technology,Chaitanya Bharathi Institute of Technology, Telangana, India Email: [email protected] Data Availability Statement: The data supporting the findings of this study are available from the corresponding author upon reasonable request. Funding Statement: This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. Conflict of Interest Disclosure: The authors declare no conflict of interest related to this work. Ethics Approval Statement: This study was conducted following institutional ethical guidelines. Patient/Participant Consent Statement: All participants provided informed consent prior to their inclusion in the pilot study. Permission to Reproduce Material: No copyrighted third-party material was reproduced in this manuscript. Clinical Trial Registration: Not applicable Captain Hope- An Offline AI-Powered Wearable Mobility Assistant for the Visually Impaired ABSTRACT Wearable assistive technologies for the visually impaired have faced a critical trade off: traditional aids offer limited environmental context, while advanced electronic solutions are frequently tethered to the cloud, compromising real-time performance and user privacy. Furthermore, few systems integrate rich conversational feedback for social awareness. This paper addresses this gap by presenting “Captain Hope,” an innovative shoulder-mounted wearable device. We demonstrate the feasibility of integrating a local large language model (LLM) with real-time computer vision for concurrent social and navigational cueing on a low-cost, fully offline wearable device. Powered by a Raspberry Pi 5 and an efficient YOLOv8n model, the system achieves an average end-to-audio latency of 485 ms, validating the real-time viability of our on-device LLM integration. A pilot user study (n=5) yielded an 89.2 percent detection accuracy in dynamic environments and, critically, a 70 percent increase in navigational confidence, underscoring the qualitative value of locally-processed, context-aware audio cues. By presenting a complete system blueprint and its real-world performance benchmarks, this paper offers a significant contribution toward creating accessible, self-reliant assistive technologies that improve quality of life. Keywords: Assistive Technology, Computer Vision, Edge AI, Human-Computer Interaction, Local LLM, Object Detection, Visual Impairment, Wearable Computing Introduction The landscape of human experience is profoundly shaped by our ability to perceive and navigate the world around us. For the estimated 2.2 billion people globally living with a near or distance vision impairment, this fundamental interaction is often fraught with challenges that extend beyond simple mobility. The daily act of navigating a bustling street, locating a product in a store, or participating in a social gathering becomes a complex task requiring immense concentration and courage. This constant cognitive load can diminish personal autonomy and foster a sense of social isolation, creating barriers to education, employment, and community engagement. The economic toll is equally staggering, with an estimated annual global productivity loss of over US 411 billion dollars attributed to vision impairment. Traditional mobility aids, most notably the white cane, have been indispensable tools for centuries, providing essential tactile feedback about the immediate ground-level environment. However, their reach is inherently limited; they cannot detect overhanging obstacles, distant hazards, or the dynamic presence of other people. This information gap leaves users vulnerable and often forces a reactive, cautious posture toward the world. In response, the last two decades have seen the rise of Electronic Travel Aids (ETAs) [1, 2, 3], which promise to augment human senses with technological perception. Yet, many of these solutions introduce a new set of compromises. High-end systems often rely on cloud computing for their advanced processing [2, 6], creating a dependency on persistent internet connectivity that renders them unreliable in network dead zones or indoor environments. This reliance also raises significant concerns regarding data privacy and introduces perceptible latency that can be detrimental in time-critical navigation scenarios. Furthermore, their high cost and, in some cases, cumbersome form factors present significant barriers to widespread adoption, particularly in low- and middle-income regions where 90 percent of visually impaired individuals reside. This research is motivated by the conviction that technology should adapt to human needs, not the other way around. It is driven by the need for a new paradigm in assistive technology: one that is affordable, fully autonomous, and designed with a deep understanding of the user’s holistic experience. To this end, we developed ”Captain Hope,” a shoulder-mounted wearable assistant engineered to be a trusted companion for the visually impaired. Its design philosophy is rooted in three core principles: complete offline functionality to ensure reliability and privacy; real-time, multi-modal perception of both physical obstacles and human presence; and intelligent, context-aware audio feedback that enhances social confidence, not just physical safety. Captain Hope addresses systemic deficiencies in existing assistive technologies [1, 3, 8] by providing an economically accessible platform that operates independently of external infrastructure. The project report highlights its use of a Raspberry Pi 5, equipped with a wide-angle camera and a custom 3D-printed shoulder harness, ensuring ergonomic comfort and a lightweight design (280g). The system’s objectives include real-time obstacle detection within an 8-meter radius and context-aware audio responses, validated through bench testing and field evaluations in controlled environments like hallways and pedestrian areas. This paper makes the following scientific contributions: 1. An architectural blueprint for a multi-modal, offline first assistive device that successfully co-locates computer vision, speech recognition, and LLM-based conversational AI on a single-board computer. This provides a validated template for developing complex, on device AI systems for assistive applications. 2. A quantitative evaluation of this architecture, establishing performance benchmarks for end-to-end latency and resource consumption that demonstrate the viability of on-device LLMs for real-time assistive tasks. These benchmarks serve as a crucial reference for future development in the field of edge AI for accessibility. 3. Qualitative and quantitative findings from a pilot user study suggesting that locally-processed, context-aware audio cues (including social presence detection) significantly enhance navigational confidence over traditional obstacle-only alerts [8]. This underscores the importance of designing for social awareness in assistive technologies. Literature Survey ANISH The development of smart wearable robotic systems for visually impaired individuals has emerged as a critical research domain within assistive technology, driven by the fundamental need to enhance spatial awareness and independent mobility [3, 8]. Traditional approaches to navigation assistance have predominantly focused on basic obstacle detection mechanisms, leaving significant gaps in comprehensive environmental understanding and contextual awareness that are essential for confident navigation in complex real-world scenarios. Contemporary research has demonstrated substantial progress in smartphone-based navigation solutions, with systems like DeepNAVI pioneering the integration of on-device deep learning capabilities for enhanced portability and reliability [2]. Kuriakose et al. developed a comprehensive framework that eliminates network dependency while providing detailed environmental analysis, including obstacle classification across twenty distinct categories, precise distance measurements, and motion status evaluation [2]. Their comparative analysis revealed remarkable performance improvements, reducing navigation time from 5.5 minutes with conventional smart canes to 3.5 minutes, while simultaneously enhancing user comfort through local processing that mitigates latency and connectivity-related reliability concerns [2]. However, these smartphone-centric approaches face inherent limitations in sensor positioning flexibility and sustained battery performance during extended navigation sessions [2, 11]. The evolution toward specialized wearable robotic platforms has addressed many limitations of smartphone-based systems while introducing new technical challenges. The eLabrador system, developed by Kan et al., exemplifies advanced wearable architecture through head-mounted RGB-D camera configurations that enable sophisticated 3D semantic environmental mapping using VINS-MONO odometry estimation techniques [3]. This approach facilitates complex global-local path planning while incorporating audio-haptic dual-channel interaction mechanisms for comprehensive multimodal feedback delivery [3]. Despite these technical achievements, the system’s significant weight burden exceeding 3.34 kg fundamentally compromises extended wear comfort and practical deployment feasibility for daily use scenarios [3]. Multi-modal sensor integration has emerged as a promising approach for addressing environmental perception challenges in varying conditions. The AI-Sense Vision system, investigated by Joshi et al., demonstrates innovative fusion of ultrasonic sensors with camera systems to maintain robust performance across diverse lighting conditions through complementary acoustic and visual sensing modalities [1]. Their implementation incorporates YOLOv3 architecture for real-time multi-class object detection while extending functionality through multi-instance object counting and optical character recognition capabilities [1]. Validation studies involving 47 participants, including seven visually impaired individuals, confirmed successful real-world object detection performance with rapid audio response generation, providing valuable empirical evidence for multimodal sensor fusion effectiveness [1]. Advanced cognitive assistance capabilities have been explored through comprehensive multi-target recognition systems that extend beyond basic obstacle detection. Li et al. developed wearable assistance cognitive systems emphasizing simultaneous object detection, facial recognition, and text reading functionality within indoor environments [8]. Their technical architecture incorporates lightweight deep neural networks, including MobileNetV3-YOLOv4-Lite and ArcFace algorithms, strategically balancing computational efficiency with recognition accuracy [8]. Pilot testing across five simulated daily tasks demonstrated impressive performance metrics, with navigation accuracy reaching 91.67 percent and facial recognition achieving 99 percent accuracy, validating multimodal feedback implementation through combined verbal communication and vibration-based haptic feedback via leg-mounted haptic modules [8]. Several critical gaps persist in current wearable robotic solutions for visually impaired users [1, 3, 8]. Weight distribution and extended wear comfort remain fundamental challenges that compromise practical deployment feasibility, with many sophisticated systems exceeding acceptable weight thresholds for daily use [3]. The integration of com prehensive environmental understanding with intuitive user interfaces requires further investigation, particularly regarding the balance between information richness and cognitive load management [8, 9]. This research landscape establishes the foundation for next-generation assistive technologies that must bridge the gap between laboratory demonstrations and real-world deployment effectiveness. Methodology The development of ”Captain Hope” follows a comprehensive methodology that integrates advanced edge computing, computer vision [4, 7], and natural language processing into a cohesive wearable system. This section details the system architecture, hardware selection, and the multi-modal AI pipeline that enables real-time environmental awareness and intelligent interaction. 3.1 System Overview The ”Captain Hope” system is built on a modular, multi-process architecture designed for real-time, concurrent operation on a single-board computer [8]. This architecture is structured in three primary layers: the Hardware Abstraction Layer (HAL), the Middleware Processing Layer, and the Application Interface Layer. This modular design ensures system scalability and maintainability while enabling high-performance AI inference on resource-constrained hardware. The system operates on an event-driven model VOLUME, where multiple concurrent processes handle distinct functions—vision processing, speech recognition, and audio output—while maintaining synchronized operation through efficient inter-process communication. The HAL manages interactions with hardware components, including the Pi Camera Module 3 for visual input and a USB condenser microphone for audio capture. The Core Processing Layer comprises a Computer Vision Engine (YOLOv8 Nano) [4], a Speech Recognition Engine (Vosk), a Natural Language Processing system (Llama 3.2 1B), and a Text-to-Speech Engine (pyttsx3). The Application Layer includes services like the Obstacle Detection Service, Voice Assistant Service, and Audio Feedback Manager, employing a publish-subscribe pattern for seamless integration. Data flow diagrams from the project report illustrate the system’s operation, with a context diagram showing interactions with the user, environment, and file system. The data pipeline begins with two primary inputs: a high frame-rate video stream from the Pi Camera Module and a continuous audio stream from a USB microphone. These streams are handled by two independent, parallel processing pipelines: The Vision Pipeline: Frames from the camera are fed into a dedicated process running the YOLOv8n object detection model [4]. This pipeline is optimized for low-latency inference, identifying and classifying objects such as people, furniture, and other potential hazards. The output is a stream of detection events, including object class, confidence score, and bounding box coordinates. Figure 1: The Vision Processing Pipeline Frames are captured, pre-processed, and fed into the YOLOv8 model. Detections are filtered and enqueued to trigger a corresponding audio alert. The Audio Pipeline: Audio input is processed by the Vosk ASR engine for offline speech-to-text conversion, followed by intent classification and LLM-based response generation, ensuring responsive voice interaction. The figure 2 specifies a Level 1 Data Flow Diagram (DFD) decomposing the system into subsystems like Audio Interface, Object Detection, Command Processor, and Alert and Control, with data stores for configuration and logs. Figure 2: System Architecture Overview (Level 1 DFD) This figure illustrates the high-level data flow between the primary processes, including the Audio Interface, Object Detection, Command Processor, and data stores for logs and configuration. 3.2 Hardware Figure 3: Core Hardware of Captain Hope The central processing unit, a Raspberry Pi 5, is connected to the Pi Camera Module 3 and headphones, forming the heart of the wearable device. The Raspberry Pi 5, with its 2.4GHz CPU and 4GB RAM, supports the system’s computational demands, while the VideoCore VII GPU accelerates inference tasks. The project report notes a 64GB microSD card (Class 10, U3 rated) for storage and a 5V 3A USB-C adapter with a backup battery pack, achieving a continuous-use battery life of 7.5–8 hours—significantly improving upon the original 3–5-hour estimate. Raspberry Pi 5 (8GB LPDDR4X-4267) ARM Cortex-A76 quad-core, VideoCore VII GPU For hardware acceleration Pi Camera Module 3 Wide-Angle 120° FOV for comprehensive environmental scanning Hardware ISP for preprocessing USB Condenser Microphone Omnidirectional capture Reliable voice commands in varied environments Bone Conduction Headphones Preserves ambient sound awareness Critical for user safety Tactile Push Button (GPIO) Hardware-debounced input Reliable dual-gesture system control 20,000mAh USB-C Power Bank Extended operational endurance For daily use 3D-Printed PLA+ Shoulder Mount Lightweight (280g), ergonomic design Extended wear comfort Table 1: Specifications and Technical Justifications for Wearable System Components 3.3 Multi-Modal AI Processing Pipeline The AI pipeline is the software soul of Captain Hope, designed for efficiency and robustness. Computer Vision Processing : The vision pipeline uses a YOLOv8n model [4] optimized for the Raspberry Pi 5. Frames are scaled to 640 × 480 pixels, an empirically determined balance between accuracy and speed. The model is quantized to reduce its memory footprint without significant performance degradation. A confidence threshold of 0.6 and Non-Maximum Suppression (NMS) are applied to filter out weak or redundant detections, ensuring that the alerts are both accurate and relevant. The system focuses on navigation-relevant classes from the COCO dataset, including humans, vehicles, and common furniture items. Advanced Speech and Language Processing : The speech subsystem uses the Vosk ASR engine for offline speech-to-text conversion. It employs a dual-stage wake word detection algorithm to minimize false activations. For conversational intelligence, we integrated a 4-bit quantized version of the Llama 3.2 1B model, reducing its memory requirement to just 800 MB. This allows for sophisticated, context-aware dialogue, such as answering questions about the environment based on real-time visual input [7, 9], a capability rarely seen in fully offline assistive devices. Inference optimization strategies include dynamic context truncation to maintain conversational coherence within a 2048-token limit, adaptive temperature scaling to balance creativity with consistency, and pre-allocated memory buffers to prevent garbage collection pauses during inference. 3.4 Inference & Optimization To achieve real-time performance, the system leverages a multi-process architecture using Python’s multiprocessing library [8]. Each core function—vision, speech, and language—runs in a separate process, allowing them to execute in parallel on the Raspberry Pi’s multi-core CPU. Inter-process communication is handled efficiently: high bandwidth data like video frames are shared via memory segments, while low-latency control signals and text data use message queues. This concurrent design, coupled with a priority-based alert manager, is the key to the system’s ability to deliver immediate safety alerts while simultaneously managing a responsive conversational interface. Watchdog mechanisms are implemented to monitor process health and trigger automatic restarts, ensuring high system availability during extended operation. Experimental Setup and Implementation To rigorously evaluate the real-world performance of Captain Hope, we designed a comprehensive experimental protocol [1, 3]. Recognizing the preliminary nature of this work, we structured the evaluation as a pilot study intended to assess feasibility and gather foundational data on system performance and user experience. 4.1 Multi-Environment Testing Protocol The evaluation was conducted across three distinct environments to test the system’s adaptability [1, 8]: • Controlled Indoor Lab : A structured 15m x 12m space with static furniture and consistent LED lighting (300 400 lux). This environment served as our baseline benchmark, allowing for precise measurement of detection accuracy and latency under optimal conditions. • Dynamic Office Hallway : A busy 30m institutional corridor with unpredictable pedestrian traffic, reflective surfaces, and significant acoustic variability (40-65dB). This scenario was designed to test performance in common indoor social navigation challenges [5]. • Outdoor Urban Park : A complex 2-hectare environment with varied terrain, natural obstacles, and highly variable lighting conditions, ranging from full sunlight (greater than 1000 lux) to deep shade (less than 50 lux). This environment tested the system’s robustness under the most demanding real-world conditions [1]. 4.2 Participant Selection and Study Design Five visually impaired participants were recruited through local community organizations, following approved ethical protocols [1, 8]. The study employed a within-subjects design, where each participant served as their own control. Performance and confidence with Captain Hope were compared against each participant’s established baseline using their traditional mobility aid (white cane or guide dog). Participant demographics are summarized in Table II. P1 35 Acquired White Cane High P2 24 Congenital White Cane Moderate P3 58 Acquired Guide Dog Moderate P4 47 Acquired White Cane High P5 29 Congenital White Cane High Table 2: Participant Demographics for the Pilot User Study (n=5) Each participant completed a 45-minute training session before undertaking a series of navigational tasks in each of the three environments [3, 8]. 4.3 Performance Evaluation Metrics Here evaluation used a mixed-methods approach to capture both objective performance and subjective user experience [1,8]: Quantitative Metrics: Detection Accuracy: Percentage of correctly identified humans and obstacles, validated against manually annotated video recordings [4]. Response Latency: End-to-end time in milliseconds from object appearance to the start of the audio alert, measured using high-precision timestamp logging [1]. System Stability: CPU usage, memory consumption, and system uptime measured during extended operational tests [8]. Qualitative Metrics: Navigational Confidence: Pre- and post-test ratings on a 1-10 Likert scale [3, 8]. User Feedback: Semi-structured interviews and real time feedback collected during trials to assess usability, alert quality, and overall satisfaction, supplemented by the System Usability Scale (SUS) [1, 8]. The project specifies testing on a Raspberry Pi 5 with a 10,000mAh power bank, achieving 4.5 hours of autonomy, though optimized configurations reached 7.5–8 hours. The setup included GPIO pin 17 for button input, with a 0.7 second threshold distinguishing single and double presses, launching vision or voice modules respectively. Testing environments included indoor hallways and outdoor pedestrian areas [5], with calibration for YOLO confidence thresholds and Vosk audio sampling rates. Figure 4: Demonstration of a Live Test Cycle: The terminal confirms initiation of the video stream after a button press Figure 5: Demonstration: The live feed shows a successful human detection with a real-time alert overlay Results The experimental trials yielded strong quantitative results and highly encouraging qualitative feedback, validating the core design principles of Captain Hope. 5.1 Quantitative Performance The system demonstrated robust and reliable performance across all tested environments [1, 8]. Key metrics are summarized in Table III. The average end-to-audio response latency was 485 ms, well within the sub-500 ms target required for safe real-time navigation. Detection accuracy was high, achieving 94.7 percent in the controlled indoor lab and maintaining a respectable 89.2 percent in the challenging outdoor park environment. This 5.5 percent reduction in outdoor performance highlights the system’s real-world robustness while also indicating areas for future refinement, particularly in handling variable lighting and complex backgrounds. The figure 7 shows robust detection performance for key navigational elements like ’Person,’ ’Chair,’ and ’Car,’ validating its effectiveness for real-world environmental understanding. To validate the system’s stability for daily use, we con ducted a 3-hour continuous operation test. As shown by our resource monitoring, CPU usage remained stable, averaging around 50-60 percent, and memory consumption was consistent with no evidence of memory leaks. Figure 6: Demonstration: The on-device LLM generates a contextual response, demonstrating its embedded reasoning capability Figure 7: YOLOv8n Detection Accuracy Across Object Categories Detection Precision (Indoor Lab) 94.7% Controlled environment with static obstacles Detection Precision (Outdoor Park) 89.2% Dynamic environment with moving pedestrians and varied lighting Human Detection Accuracy 92.1% Specific performance for human recognition tasks Obstacle Classification Accuracy 87.3% Recognition of furniture, vehicles, and environmental hazards End-to-Audio Response Latency 485 ms (avg) From object detection to audio alert generation Frame Processing Speed 30 FPS Consistent real-time video processing capability Battery Endurance (continuous) 3.5 hours Full operation with 20,000mAh power bank System Uptime Reliability 99.2% During 48-hour continuous operation testing Table 3: System Performance Evaluation This demonstrates the efficiency of the multi-process architecture and its suitability for long-duration, real-world deployment. Furthermore, analysis of the vision pipeline’s performance showed a consistent frame rate, with inference times remaining low and stable, confirming the system’s real-time processing capabilities even under sustained load. Figure 8: System Resource Usage Over 3-Hour Continuous Test This graph shows stable CPU and memory consumption over an extended period, with no evidence of memory leaks, confirming the efficiency and reliability of the multi-process architecture. 5.2 Qualitative User Experience The qualitative feedback gathered from the five visually impaired participants was overwhelmingly positive and provided deep insights into the system’s real-world value. The most significant finding was a 70 percent average increase in self-reported navigational confidence when using Captain Hope compared to their traditional aids. User feedback, summarized in Table IV, consistently highlighted themes of enhanced awareness, safety, and social integration. Enhanced Navigational Confidence “It’s like having a guide who’s always one step ahead, whispering exactly what I need to know.” The system successfully reduces navigational uncertainty and builds user trust in unfamiliar environments. Social Awareness “The ability to be told ’person approaching from your left’ was transformative for social interaction.” Providing human detection creates opportunities for social engagement rather than just obstacle avoidance. Safety and Independence “I felt like it was my lifeline in a space I didn’t know; it gave me the courage to explore.” The device effectively functions as a trusted safety companion, promoting independent exploration. Alert Quality and Timing “The warnings came at just the right moment—not too late, not overwhelming.” The 485ms response time strikes an optimal balance between timeliness and user comfort. Table 4: Key Qualitative Feedback from User Trials Limitations As a pilot study, this work has several limitations that offer clear directions for future research. The most significant is the small participant sample size (n=5). While the feedback was rich and consistent, a larger, more diverse study is needed to generalize these findings. On a technical level, the system’s performance, while robust, degraded in certain conditions [1, 3]. Detection accuracy dropped by approximately 5 percent in low-light environments, indicating a need for sensor fusion (e.g., adding thermal or ToF sensors) or more advanced low-light image processing algorithms [4]. While the alert manager was designed to prevent information overload [8], some users still found the frequency of alerts in very crowded areas to be overwhelming. This suggests that more intelligent, adaptive filtering algorithms, perhaps ones that learn from user behaviour over time, are required. Finally, the 3–5-hour battery life, while sufficient for many excursions, could be improved with more aggressive power management strategies or higher-capacity power sources to support all-day use. Occasional LLM latency for responses exceeding 150 tokens (up to 3 seconds), reduced accuracy in cluttered scenes [7], and thermal throttling without active cooling. It suggests future enhancements like Tiny LLAMA optimization and directional microphones for noisy environments. Conclusion Captain Hope marks a significant and practical step forward in the landscape of wearable assistive technology [3, 8, 10]. By successfully embedding a suite of advanced AI models directly onto a lightweight, shoulder-mounted device, it delivers rapid, reliable, and context-aware guidance without dependence on external networks [1, 2]. The seamless integration of real-time object detection [4], an offline conversational LLM, and clear audio cues provides users with an intuitive and empowering sense of their surroundings. The modular, offline-first architecture is not merely a proof-of-concept but a robust foundation for the next generation of assistive devices [8, 11]. Future work will focus on enhancing this platform with personalized alert patterns, optional haptic feedback for critical obstacles, and more energy-efficient power modules. Beyond simply avoiding hazards, Captain Hope fosters autonomy, encourages social engagement, and builds confidence—the cornerstones of a truly inclusive future. As on-device intelligence continues to evolve, this platform stands ready to empower visually impaired individuals with unprecedented freedom, safety, and connection to the world around them. Acknowledgements I express my heartfelt gratitude to the visually impaired volunteers who generously dedicated their time and provided invaluable feedback during the testing phases of Captain Hope. Their insights were instrumental in shaping a system that truly serves its users. I thank the reviewers whose constructive feedback refined and strengthened this research. Special thanks to our IT Head of Department, for providing the necessary resources and a conducive environment for conducting this research. His leadership and encouragement have been necessary in the triumphant completion of this study. References [1] R. C. Joshi et al., “AI-SenseVision: A low-cost, artificial intelligence-based, robust, and real-time assistance for visually impaired individuals,” IEEE Access, vol. 10, pp. 10952–10963, 2022. [2] B. Kuriakose et al., “DeepNAVI: A deep learning based smart phone navigation assistant for people with visual impairments,” in Proc. Int. Conf. Computers Helping People with Special Needs (ICCHP), pp. 1–8, 2020. [3] M. Kan et al., “eLabrador: A wearable navigation system for visually impaired individuals,” in Proc. ACM Int. Symp. Wearable Computers (ISWC), pp. 38–45, 2021. [4] Z. Meng et al., “HYDRO-3D: Hybrid object detection and tracking for cooperative perception using 3D LiDAR,” IEEE Trans. Intell. Vehicles, vol. 7, no. 1, pp. 78–89, 2022. [5] Q. Na et al., “Improving walking path generation through biped constraint in indoor navigation system for visually impaired individuals,” in Proc. IEEE Int. Conf. Robotics and Biomimetics (ROBIO), pp. 253–259, 2021. [6] H. Son et al., “Infrastructure enabled guided navigation for visually impaired,” IEEE Internet Things J., vol. 8, no. 15, pp. 12347 12358, 2021. [7] R. Ding et al., “Lowis3D: Language-driven open-world instance level 3D scene understanding,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 3497–3507, 2023. [8] G. Li et al., “Sensing and navigation of wearable assistance cognitive systems for the visually impaired,” IEEE Sensors J., vol. 21, no. 20, pp. 22567–22575, 2021. [9] T. Wang et al., “Visual perception generalization for vision-and language navigation via meta-learning,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 2, pp. 312–328, 2023. [10] H. Chang et al., “MedGlasses: A wearable smart-glasses-based drug pill recognition system using deep learning for medication adherence,” IEEE Access, vol. 8, pp. 17013–17024, 2020. [11] S. Kumar et al., “Mobile based navigation system for visually impaired person,” in Proc. 2023 3rd Int. Conf. Information Communication and Software Engineering (ICICSE), pp. 1–6, 2023. Information & Authors Information Version history V1 Version 1 25 September 2025 Copyright This work is licensed under a Non Exclusive No Reuse License. Keywords computer vision field robotics robotics Authors Affiliations Anish Kumar Sidhi 0009-0003-5037-668X [email protected] Chaitanya Bharathi Institute of Technology View all articles by this author Suresh Pabboju Chaitanya Bharathi Institute of Technology View all articles by this author P. Kiranmaie Chaitanya Bharathi Institute of Technology View all articles by this author Metrics & Citations Metrics Article Usage 304 views 168 downloads .FvxKWukQNSOunydq8rnd { width: 100px; } Citations Download citation Anish Kumar Sidhi, Suresh Pabboju, P. Kiranmaie. Captain Hope: An Offline AI-Powered Wearable Mobility Assistant for the Visually Impaired. Authorea . 25 September 2025. DOI: https://doi.org/10.22541/au.175882486.65199983/v1 If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu . Format Please select one from the list RIS (ProCite, Reference Manager) EndNote BibTex Medlars RefWorks Direct import Tips for downloading citations document.getElementById('citMgrHelpLink').addEventListener('click', function() { popupHelp(this.href); return false; }); $(".js__slcInclude").on("change", function(e){ if ($(this).val() == 'refworks') $('#direct').prop("checked", false); $('#direct').prop("disabled", ($(this).val() == 'refworks')); }); Cited by Loading... View Options View options PDF View PDF Figures Tables Media Share Share Share article link Copy Link Copied! Copying failed. Share Facebook X (formerly Twitter) Bluesky LinkedIn email View full text | Download PDF {"doi":"10.22541/au.175882486.65199983/v1","type":"Article"} Now Reading: Share Figures Tables Close figure viewer Back to article Figure title goes here Change zoom level Go to figure location within the article Download figure Toggle share panel Toggle share panel Share Toggle information panel Toggle information panel Go to previous graphic Go to next graphic Go to previous table Go to next table All figures All tables View all material View all material xrefBack.goTo xrefBack.goTo Request permissions Expand All Collapse Expand Table Show all references SHOW ALL BOOKS Authors Info & Affiliations About FAQs Contact Us Directory RSS Back to top Powered by Research Exchange Preprints Help Terms Privacy Policy Cookie Preferences $(document).ready(() => setTimeout(() => { let _bnw=window,_bna=atob("bG9jYXRpb24="),_bnb=atob("b3JpZ2lu"),_hn=_bnw[_bna][_bnb],_bnt=btoa(_hn+new Array(5 - _hn.length % 4).join(" ")); $.get("/resource/lodash?t="+_bnt); },4000)); (function(){function c(){var b=a.contentDocument||a.contentWindow.document;if(b){var d=b.createElement('script');d.innerHTML="window.__CF$cv$params={r:'9ff084b76efa09d6',t:'MTc3OTMzNDM5Mw=='};var a=document.createElement('script');a.src='/cdn-cgi/challenge-platform/scripts/jsd/main.js';document.getElementsByTagName('head')[0].appendChild(a);";b.getElementsByTagName('head')[0].appendChild(d)}}if(document.body){var a=document.createElement('iframe');a.height=1;a.width=1;a.style.position='absolute';a.style.top=0;a.style.left=0;a.style.border='none';a.style.visibility='hidden';document.body.appendChild(a);if('loading'!==document.readyState)c();else if(window.addEventListener)document.addEventListener('DOMContentLoaded',c);else{var e=document.onreadystatechange||function(){};document.onreadystatechange=function(b){e(b);'loading'!==document.readyState&&(document.onreadystatechange=e,c())}}}})();