Teaching & Learning

Social Service-related Projects

The College is keen on fostering strategic partnerships with the social services sector to create a more cohesive and caring society. Over the years, our faculty members and students have developed research projects with NGOs in Hong Kong by applying available technology to enhance the delivery of social services. Below are some of the highlighted projects:
 

Project Title:
An App to help the Visually Impaired People to Read Music Sheets

An App to help the Visually...........

Supervisor: Dr Kelvin Shiu Yin Yuen
Student: Ping Keung Shan
Department of Electrical Engineering

Partnered NGO:
Hong Kong Blind Union
Technologies for the Elderly and Disabled people by Youths

Project Description:
This project aims to design an iOS App to help the Visually Impaired People (VIP) to read music sheets. At present, the VIPs need to convert music sheets to braille before they could read them which is very inconvenience and expensive. Through this application, the VIPs can read music sheets via VoiceOver, an inherent accessibility on iOS, when they touch the screen, as if they are reading paper music sheets in braille format. 

Software / Hardware Available:
Prototype of an App

 

Project Title:
Dou Shou Qi for the Visually Impaired

Dou Shou Qi for the Visually Impaired

Supervisor: Dr Kelvin Shiu Yin Yuen
Student: Hei Ching Poon
Department of Electrical Engineering

Partnered NGO:
Hong Kong Federation of the Blind
Jockey Club

Project Description:
This project aims to develop a new version of an iOS App which enables the Visually Impaired People (VIP) to play Dou Shou Qi (Jungle). When the VIPs touch a piece on the screen, the VoiceOver function on an iOS device will tell players the name and position of the piece. This enables the VIPs to play with others via the App.

Software / Hardware Available:
iOS application

Technology available:
VoiceOver on iOS

More about the project: https://drive.google.com/open?id=1TdCJYyz-CxX51cgcoimGLzpSaxFNjpDP

 

Project Title:
Chinese Chess for the Visually Impaired

Chinese Chess for the Visually Impaired

Supervisor: Dr Kelvin Shiu Yin Yuen
Student: Lai Yin Lai
Department of Electrical Engineering

Partnered NGO:
Hong Kong Blind Union
Hong Kong Federation of the Blind

Project Description:
This project aims to develop an iOS App for the visually impaired to play the Chinese chess game, anytime and anywhere. The App is developed with a simple layout of the Chinese chessboard. The real-time status of the game and the pieces movement can be announced by VoiceOver on the iOS device. People with different levels of visual impairment would be able to “see” all the chess pieces on the screen. Other than the visualization of the game situation, the App will be able to connect with the Game Center service. The visually impaired and normal users can battle with each other on the Internet. 

Software / Hardware Available:
App on iOS

More about the project: https://apps.apple.com/hk/app/%E4%B8%AD%E5%9C%8B%E8%B1%A1%E6%A3%8B-vip/id1453873358?l=en

 

Project Title:
An Electronic Guide Dog for the Visually Impaired

An Electronic Guide Dog..........

Supervisor: Dr Kelvin Shiu Yin Yuen
Student: Man Sum Chiu
Department of Electrical Engineering

Partnered NGO:
Hong Kong Federation of the Blind

Project Description:
This project aims to develop an iOS App for the Visually Impaired People (VIP) which can perform indoor navigation. The App is able to access hard wares, including pedometer and digital compass inherent on iOS, so as to record the walked path. It does not rely on GPS technology which is not accurate in indoor environment. By using this App, the VIPs can take the quickest path and get the navigation instructions afterwards. 

Software / Hardware Available:
App on iOS 

Technology Available:
VoiceOver on iOS

 

Project Title:
Real-time Outdoor Objects Recognition and Distance Detection for Visually Impaired People

Real-time Outdoor Objects.........

Supervisor: Dr Leanne Lai Hang Chan
Student: Shing Fung Wong
Department of Electrical Engineering

Partnered NGO:
Hong Kong Blind Union

Project Description:
According to the World Health Organization, there are 257 millions of people with visual disabilities. Among them, 217 million have moderate to severe vision impairment and 36 million are totally blind. According to another study, low mobility is one of the major daily life problem encountered by the visually impaired. Walking on unfamiliar roads can be challenging and possibly dangerous for them. Currently, there are existing applications designed for helping the visually impaired. For example, Microsoft has employed image recognition technology in their Seeing AI application to identify different scenes, colors and emotions. Another application, TapTapSee, describe objects in a photo or short video from user’s smartphone camera. The application uses “CloudSight Image Recognition API” in the pre-processing stage hence the images are able to return correct description even if the picture was taken under narrowed angles or poor lighting conditions. However, majority of the existing application on smartphone are not designed for identifying outdoor objects, and their processing speed are quite slow due to the high latency of cloud computing, combined with issues such as lacking distance detection. The existing applications fail to provide timely notifications regarding the objects surrounding the individual. The objective of this project is to develop an offline smartphone application that performs real-time object recognition and distance detection on common outdoor objects. The application aims to create a low cost and real time application to minimize stress and the risk for visually impaired people when walking around unfamiliar locations.

Technology Available:
IOS Application "SeePath"

Other Information:
Awarded InfoTech Job Market Driven Scholarship 2019

More about the project: https://youtube/DbxMU4qoBZE

 

Project Title:
Digital Empowerment for the Visually Impaired

Digital Empowerment for the Visually Impaired

Supervisor: Dr Ray Chak Chung Cheung
Student: Wing Yin Yu
Department of Electrical Engineering

Partnered NGO:
Hong Kong Federation of the Blind

Project Description:
The main objective of this project is to develop a portable device that help the visually impaired people recognize bus number in real time. This device can assist the VIPs to perform the following 3 functions:

  • To recognize route label of in-coming double-decked buses in Hong Kong
  • To notify the VIPs with a Bluetooth headset once buses are detected
  • To operate in a hands-free manner (i.e.no holding is required)

Prize Obtained:

  • ASM Technology Award 2019 - Outstanding Award
  • CityU EE Project Competition 2019 - Bronze Prize

Software / Hardware Available:
Hardware: a Raspberry Pi 3+, a Bluetooth headset, a power bank, a web-camera

Technology Available:
Computer Vision, Object Detection, Segmentation

 

Project Title:
Game for VIP - Mini Scrabble on iOS

Game for VIP

Dr Kelvin Shiu Yin Yuen
Department of Electrical Engineering

Partnered NGO:
Hong Kong Federation of the Blind

Project Description:
This project aims to present a board game, Scrabble, through the iOS with accessibility for the Visual Impaired People (VIP). The VIP can read the contents on the screen via VoiceOver, an inherent accessibility on iOS, when they touch the screen. They can also tap and flick the screen to select or move elements via VoiceOver Gestures. To a greater extend, the project aims to raise the awareness of developers about the needs of VIP on entertainment and add accessibility when developing iOS Apps.

Software / Hardware Available:
App on iOS

Technology Available:
VoiceOver on iOS

 

Project Title:
Design a Sport Game for the Visually Impaired

Supervisor: Dr Kelvin Shiu Yin Yuen
Student: Wing Sum Tang 

Department of Electrical Engineering

Partnered NGO:
Hong Kong Blind Union
Hong Kong Federation of the Blind

Project Description:
This project introduces a two-player computer tennis game for Visually Impaired People (VIP) to enjoy sports and physical activities. Several speaker-arrays will be set to indicate a tennis ball sound. Players have to reach the corresponding zone by identifying the sounded speaker. A camera will detect the player’s motion and determine whether the player is correct or not. Finally, the scores of each player and the winner will be announced.

GitHub link: https://github.com/charlietang075/Tennis-ball-game-for-VIP

Software / Hardware Available:
1 and 2-player tennis game for VIP

 

Project Title:
Wheelchair Battery Health Condition Monitoring Modules

Professor Henry Shu Hung Chung
Department of Electrical Engineering

Partnered NGO:
Direction Association for the Handicapped

Project Description:
This project aims to develop a diagnostic module for wheelchairs. The module can inform users the real-time state of charge and state of health of the batteries. Moreover, the module can tell users the distance that the batteries can support the wheelchair to travel.

 

Project Title:
Somewhere in Time Museum

Dr Charlie Qiuli Xue
Department of Architecture and Civil Engineering

Partnered NGO:
The Rotary Club of Hong Kong (Kowloon East)

Project Description:
The Rotary Club of Hong Kong (Kowloon East) is planning to build a small museum featuring collections of antiquity in Sai Kung. Led by Dr Charlie Xue, a team of architectural students from the City University of Hong Kong involved in the design of the museum. The team carried out preliminary site analysis and survey, and submitted a report to the Rotary Club in July 2018. All works were conducted on a voluntary basis in support of the community services by the Rotary Club. The project is on-going and it involves government departments such as Lands Department and Buildings Department. In this project, our students are able to apply their skills and knowledge outside of the classroom.

 

Project Title:
Developing an Aid to Help the Visually Impaired to Navigate at MTR Platform

Supervisor: Dr Kelvin Shiu Yin Yuen
Student: Ho Ming Ng 

Department of Electrical Engineering

Partnered NGO:
Hong Kong Blind Union

Project Description:
MTR is one of the major means of transport for the visually impaired people (VIP) in Hong Kong. However, many VIP find that navigating in MTR stations is challenging due to the irregular and complicated structure of the stations, especially for those stations which they seldom visit or have never visited before. Such difficulties could lengthen the travelling time of the VIP and cause inconvenience to their journey. In hopes of helping the VIP to navigate on MTR platforms, an aid utilizing the iBeacon and iPhone is therefore developed to provide them with useful information such as the user’s current location, the location of escalators, the distance to escalators, the exits which the escalator leads to, etc. With the VoiceOver function of iOS switched on and by tapping different parts of the screen, the user will be able to hear the corresponding auditory response reading out the information. This proof-of-concept aid has been field-tested on an MTR platform and is found to work well.

 

Project Title:
Two Chess Games Developed for the Visually Impaired

Supervisor: Dr Kelvin Shiu Yin Yuen
Student: Ming Leong Lam

Department of Electrical Engineering

Partnered NGOs:
Hong Kong Blind Union
The Hong Kong Federation of the Blind

Project Description:
At present, the visually impaired people (VIP) play chess games using a special board and chess pieces. This project aimed to explore innovative ways to present the chess board positions so as to make the chess game experience even more enjoyable for the VIP. Two iOS chess games, “Tic-Tac-Toc” and “Dou Shou Qi”, are developed with accessibility. With the help of VoiceOver, VIP can easily know what is happening on the chess board and play with the user-friendly control.

 

Project Title:
Virtual Sports System for the Visually Impaired
"'

Dr Leanne Lai Hang Chan 
Department of Electrical Engineering

Partnered NGO:
Hong Kong Blind Union

Project Description:
This project aimed to design a virtual sports system, providing a safe and effective exercise platform to the visually impaired, to increase their exercise compliance. The simulated tennis game was written in MATLAB, comprising of the Kinect system and four loudspeakers. Five game zones were designed for users to earn points. Users were asked to reach the corresponding zone after hearing a tennis serving sound generated by MATLAB via the loudspeakers. The Kinect will detect the hand movements of the users while users swing using a racket. Users could earn points by stepping into the correct zones generated by the speakers. Colour and depth sensors of Kinect were used to detect the shirt colour of the user. User’s location was analyzed by using the MATLAB Image Acquisition and Processing toolboxes. Sounds were generated in a randomized order to the loudspeakers and soundcards with their own speaker ID, and the Data Acquisition toolbox. Hand movement detection was achieved by the depth sensor to set the Body Posture Joint Indices of the right hand and made accessible as metadata on the data stream.

 

Project Title:
Scene Recognition Based Navigation System for the Blind

Dr Leanne Lai Hang Chan 
Department of Electrical Engineering

Partnered NGO:
Hong Kong Blind Union

Project Description:
About 17,000 people in Hong Kong (2.4% of the total population) are visually impaired. Besides guide dogs and sticks, the indoor navigation relays on high-cost appendants, such as sensors, RFID and Wi-Fi. Convolutional Neural Network (CNN) has achieved impressive performance in object detection and scenes recognition. However, it is rarely applied in the field of indoor navigation for the visually impaired. This project intends to propose a CNN-based indoor scenes recognition system to improve mobility for the visually impaired.

6,100 images from Hong Kong Blind Union and online image dataset were collected to form the dataset. A CNN model based on AlexNet was optimized on this dataset and deployed in an android app. The resulted model achieves 85% accuracy in test dataset. Furthermore, the model’s behavior was studied through Class Activating Mapping technique, indicating how the model classifies the scenes based on discriminative image regions and spatial features. This study has proved that using CNN is a potential solution to recognize the indoor scenes, which could be applied to giving visually impaired people real-time navigation. Expansion of training dataset and combination with wearable devices could be done to further improve the performance.

 

Project Title:
CityU Apps Lab – Posture Check App

Supervisor: Dr Ray Chak Chung Cheung
Students: Tsz Wing Kwok & Debarun-Dhar

Department of Electrical Engineering

Partnered NGO:
The Chiropractic Doctors' Association of Hong Kong

Project Description:
This project serves to enhance the CityU Apps Lab's Posture Check mobile App by proposing a pipeline for the automatic detection of posture keypoints in the full-body front and side view images of human users. In the proposed method, the keypoint estimation is formulated as a regression problem which is solved using a deep learning approach.

The first half of this project is concerned with the development of a lightweight Convolutional Neural Network (CNN) architecture for human pose estimation on the FashionPose dataset. The model is shown to have comparable results with the current state-of-the-art. In the latter half, a new dataset of 900 images annotated with posture keypoints is used to adapt the pre-trained CNN to the new domain of posture keypoint estimation using transfer learning techniques. Two approaches to transfer learning are explored and evaluated. At the end the quantitative and qualitative results from the completed pipeline are presented. The final results of the pipeline demonstrate its high detection rate while maintaining a fast prediction speed and comparatively small memory footprint.

More about the project: Posture Check App

手機App助查脊骨