GoingEasy® with Crowdsourcing - Building Cyber-Physical Systems for People with Visual Impairment

What is this project about?

      Many practical barriers continue to exist for a blind individual who strives to lead an independent and active life, despite decades of development of assistive technologies. This project addresses the following two most prominent challenges: (1) disparity in information-sharing among people with visual impairment and its limited understanding by the research community; and (2) lack of methods and tools for effectively addressing the disparity. The central idea is to engage visually-impaired people and their families and friends to directly contribute to a joint endeavor of enhancing information flow, increasing awareness, and improving efficiency of assistive practices, through employing social media and participatory Web.

      The research is focused on designing computational methodologies and developing tools that are necessary for building cyber-physical systems for a domain where the tight intertwining of physical and cyber systems plus active participation of the human users are the key to attaining the otherwise unlikely capabilities for improving the quality of living for people with special needs. The key approach is to develop a blind-specific cyber-physical system that supports social-media-based crowdsourcing. This enables visually-impaired people to form loosely-connected groups, actively contribute their information and knowledge, and ask/answer unique questions of special needs.

      Aiming at bridging a significant knowledge gap in addressing the challenge of disparity in information-sharing for people with special needs in the age of social media, the project contributes to the development of a deeper understanding of the principles and methodologies in building new cyber-physical systems that promote and support active participation of users of the system, which is especially important for special-need groups such as the visually impaired, the elderly, etc.



This project is supported by National Science Foundation (Grant #1135616).

Project team









Illustrating the project ideas


The objective of supporting social-media-based crowdsourcing for solving some of the long-lasting challenges faced by the visually-impaired cannot be achieved with exiting social networking sites or on-line services alone. The key idea of the project is to design and develop a novel cyber-physical system with supporting technologies that provides much-needed new CPS capabilities. The figure below illustrates a practical application scenario of such a CPS, which was inspired by demands raised by real users in field studies conducted by the team.


Assume that a Safeway grocery store is located on 7th Street. A user (who might be a blind user or a sighted volunteer of the system) posted a message from her home PC regarding sidewalk constructions on Palm Avenue (which connects 6th Street and 7th Street). Another lady (who is blind) walking on 6th Street with her guide dog calls the CPS from her cell phone, asking about directions to the nearest grocery store. The system will then compute the route (either using the geo-tagged call or her self-report to determine her current location) and provide her with an easy-to-follow verbal instruction to Safeway on 7th Street, without using (unusable) Palm Avenue.


In achieving this, the CPS utilizes multiple deeply-intertwined cyber and physical components: a physical device (the cell phone) for interfacing between the user and the system, a software agent for understanding the user query and extracting information, and software agents for mining and searching for relevant information provided by other users, networking software and hardware supporting making queries to external map services, and a physical server supporting all these processing steps.


While this is only a simple illustration, it reveals some critical features that such a CPS should possess in order to serve the needs of the target population, such as:

·         Being blind-friendly (both the cyber components and the physical components);

·         Being able to provide constantly-updated information (as opposed to being just some static websites);

·         Supporting real-time query for information even when the users are on the go;

·         Providing information that is important to the users’ daily living;

·         Supporting expandability and scalability.



Major research tasks



Designing and Building a Blind-specific Social-networking Site


While making websites more accessible has been extensively studied, the social networking component of the proposed CPS is designed to support much more active involvement of the blind users in contributing and sharing information. Thus a wide spectrum of new issues needs to be thoroughly studied based on the end users, which will in turn result in new design principles/techniques to be used in the development of the CPS. The focus of the project on this regard includes

·         Person-centric, Blind-friendly Design Based on User Study

·         Developing Novel Interfacing Schemes for the CPS

·         Supporting Active Social Interaction on the CPS

·         Delivering Customized Information




Novel, Enabling Cyber-Physical Technologies


To fully support the desired functionalities, a suite of enabling cyber-physical technologies need to be developed. For example, many of the novel features the CPS intends to provide rely on intelligent software agents for community-specific data collection, information mining, and behavior modeling. Also, repurposing visual information is central to many tasks that a user who is blind may encounter in using the CPS. Accordingly, the following R&D efforts are being pursued in the project:

·         Supporting Repurposing Visual Information via Novel CPS Technologies (with a focus on supporting navigation)

·         Data Collection and Fusion through Active Learning

·         Inferencing with Incomplete Data

·         Community-specific Data Mining and Behavior Modeling



Integrating, Deploying, and Evaluating a Fully-Engaged Cyber-Physical System


The cyber and physical components of the proposed CPS need to be deeply intertwined to accomplish the intended goals. Further, users of the system need to have real, active participation in order for the CPS to maintain its relevance to the life of the users. Therefore, ultimately, the research will need to converge to a real working system with active users. That is, integrating the technological components to build and deploy a working system that is to be tested by real users is one of the key tasks of this project. 


~~ A blind-specific social-networking site

An Alpha version of the CPS server, GoingEasy.org, has been running with 20+ local users. Some of the capabilities reflecting the project goals have been deployed and tested on the server. The site was redesigned and revamped multiple iterations based on feedback from the real users, and it is under constant revision.







Relevant Publications, Manuscripts and Presentations:

(Check back often for update)


      Peng Zhang, Xufei Wang, Baoxin Li, On Predicting Twitter Hashtag Trend: Important Factors and Models, IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 2013.  [Poster Paper PDF] [Full manuscript PDF]


      Devi Paladugu, Parag Chandakkar, Peng Zhang, Baoxin Li, Supporting Navigation of Outdoor Shopping Complexs for Visually-Impaired Users Through Multi-model Data Fusion, IEEE International Conference on Multimedia and Expo, 2013. [PDF]


      Devi Paladugu, Baoxin Li,Towards Designing and Sustaining an Online Community for Visually Impaired Users, Poster on IEEE Workshop on Multimodal and Alternative Perception for Visually Impaired People (MAP4VIP) [PDF] [Full description of the work available from this manuscript.]


      Qiongjie Tian, Peng Zhang, Baoxin Li, Predicting the Best Answers in Community-based Question-Answering Services, International Conference on Weblogs and Social Media, 2013. [PDF]


      D. Paladugu, H. Manguluri, Q. Tian, Baoxin Li, “Automated Description Generation for Indoor Floor Maps”, ACM SIG ASSESTS 2012, Live Demo. [PDF]


      X. Wang, H. Liu, P. Zhang, B. Li, “Identifying Information Spreaders in Twitter Follower Networks”, ASU Computer Science & Engineering Technical Report. [PDF]






For more information, contact: Baoxin Li.