Nowadays smartphones are very commonly used. They let users access to rich information; this is particularly useful while the users are on the road. However, many smartphones have adopted a touchscreen input interface. While this is an intuitive interface for general users, it is inconvenient, if not impossible, for the visually impaired persons to use.
CECID has obtained HK$5.49 million funding for an Innovation Technology Fund (ITF) project, "A Smartphone Control Device and Signpost System for Visually Impaired Users", under the Innovation and Technology Support Programme. On top of the ITF funding, the project received industry sponsorships and tremendous support from five industry partners. They include Netgenii Solutions Limited, ICO Limited, MPRHK Limited, The Link Management Limited and Shenzhen CanYou Software Company Limited. Moreover, this project is supported by the following Government departments and NGOs: Commissioner for Rehabilitation of Labour and Welfare Bureau, Architectural Services Department, Transport Department, Housing Authority, Office of the Government Chief Information Officer, The Hong Kong Society for the Blind and Hong Kong Federation of the Blind.
Led by Prof David Cheung, the project commenced on 1 Dec 2011 and will last for 18 months.
The project aims at (1) developing a smartphone assistant for visually impaired users to control and operate mobile applications on smartphones, which helps the users to overcome the barrier of inability to "see" the soft buttons on the touch screens of smartphones; and (2) designing and implementing a portable signpost-for-the-blind system, which disseminates signpost information to the impaired users' smartphones, and thus compensates their inabilities to read signposts visually in their surroundings.
The deliverables of this project include:
- A smartphone assistant device, namely "E-Guide (去街易)", which extends the input function of smartphones, letting users control the phones using physical buttons.
- A portable signpost for the blind system that disseminates direction, location and related information to users' smartphones. This involves hardware and software design of two major subsystems:
- A signal transmitter, which transmit information from the spots (e.g. bus stops, MTR exits, shop entrances, etc.).
- A signal receiver, which includes a Bluetooth Low Energy chip co-located in the smartphone assistant.
- An open application development interface for developing barrier-free mobile applications on the smartphone assistant and signpost infrastructure, and
- Delivery of prototypes and pilot projects.
At present, there are Braille layout plans or Braille plates providing location information to visually impaired persons in malls or train stations, etc. However, since the Braille plates are installed at specific points, it could be a hassle for the visually impaired persons to discover the plates if they are unfamiliar with the area. Our project provides a breakthrough by wirelessly and verbally disseminating indoor and outdoor location information to the users. They can receive the information by our specially designed smartphone control device, namely E-Guide. This handy device will make their lives easier when getting around on their own.
Also, with E-Guide, users will be able to connect to their Android smartphones, and operate the apps running on the smartphones by the keypads on E-Guide. This way, they can enjoy the infinite number of mobile applications (e.g. emails, SMS, e-books, etc.) and Internet information (e.g. weather, finance, etc.) easily, especially while they are traveling around.
In addition, our project will develop an open application development interface for third party developers to develop barrier-free mobile applications to utilize our E-Guide and signpost infrastructure. Below are examples of mobile applications to be developed. They demonstrate the power of our platform on serving the visually impaired user community:
- Audible GPS
- Smart bus stop
- Smart road signs
- Smart shopping mall
- Smart MTR station
- Audible caller display
The following diagram depicts the conceptual framework of the project.
The major feature of this project is to let the visually impaired users know about their current locations. At outdoors locations, global positioning system (GPS) is commonly used. Many smartphones already have built-in GPS chips. To ease the access, E-Guide has designed a hotkey for the users to query their locations quickly and easily. However, at indoor locations such as shopping malls, GPS might no longer provide accurate location information to the users. Therefore, we need to seek for alternate positioning data sources.
In our project, we make use of the Bluetooth v4.0 standard for determining the proximity between the visually impaired users and the positioning data sources. The advantages of Bluetooth v4.0 are manifold. First, Bluetooth is a widely supported technology by the consumer electronics industry. Therefore, the supporting electronic components are abundantly available in the market; this will lower the production cost and stabilize the supply of components for our E-Guide.
Also, Bluetooth v4.0 has a special low energy profile. In our usage scenario, the power consumption of such profile is the lowest among similar wireless technologies (e.g. Active RFID, ZigBee, etc.). Using Bluetooth v4.0 low energy profile, we have designed our signpost tags to operate on ordinary AA batteries. Thanks to the ultra low power consumption, the tags can operate for one year without the need of battery replacement. This way, the maintenance of the signpost tag can be as easy as an annual operational exercise, and no extra electricity wiring is needed during installation. This is an important feature to ease the deployment.
Now with the underlying wireless technology fixed, the technical challenge becomes controlling the proximity sensitivity of our signpost tags and the E-Guide held by the users. Imagine in the shopping scenario, there will be quite many tags installed in the mall. In this case, our requirement is that the E-Guide should report the proximity of signpost tags when the users are fairly close (e.g. within 5 meters). However, in the bus stop scenario, the requirement will be quite different. It is preferable if the users can be notified a bit earlier, e.g. when the buses are 20-30 meters away from the bus stop. Our system design should take care of both situations.
To combat this challenge, we have designed a special computation algorithm to estimate the distance between the tags and the E-Guide effectively. In our experiment in laboratory, we can control our E-Guide to successfully distinguish tags from different ranges: 1m, 5m and over 10m. This helps us to deploy suitable tags in different usage scenarios.
In current phase, we are verifying the usefulness and effectiveness of the features of the proposed technologies through pilot projects. We have obtained full support from the communities of visually impaired users. A focus group has been formed, and users with different levels of visual impairment and different levels of technical awareness will play the tester role.
In addition to end-users testing on operating the mobile device, we have set up pilot projects with shopping mall management companies and the public transport companies. In these pilot projects, the signpost system is deployed and tested in real world environment. We welcome more pilot testing in the coming months. Our objective is to verify the applicability of the portable signpost for the blind system in as many environments and scenarios as possible.
We foresee that, with these technologies developed, tested and installed, visually impaired users could operate the rich applications on smartphones while our novel portable signpost system will let them receive location, direction and related information from the surrounding environment. And in turn these technologies improve their independence of living.