Tracking and Classification of In-Air Hand Gesture Based on ThermalGuided Joint Filter

Collection with item attached
2017
Item details URL
http://open-repository.kisti.re.kr/cube/handle/open_repository/486382.do
DOI
10.3390/s17010166
Title
Tracking and Classification of In-Air Hand Gesture Based on ThermalGuided Joint Filter
Description
This work was partly supported by Institute for Information &communications Technology Promotion (IITP) grant funded by the Koreagovernment (MSIP) (R0117-16-0009, Development of the high-precision AR &VR contents based on smart-car sensors) and Institute for Information &Communications Technology Promotion (IITP) grant funded by the Koreagovernment (MSIP) (R0124-16-0002, Emotional Intelligence Technology toInfer Human Emotion and Carry on Dialogue Accordingly).
abstract
The research on hand gestures has attracted many image processing-related studies, as it intuitively conveys the intention of a human as it pertains to motional meaning. Various sensors have been used to exploit the advantages of different modalities for the extraction of important information conveyed by the hand gesture of a user. Although many works have focused on learning the benefits of thermal information from thermal cameras, most have focused on face recognition or human body detection, rather than hand gesture recognition. Additionally, the majority of the works that take advantage of multiple modalities (e.g., the combination of a thermal sensor and a visual sensor), usually adopting simple fusion approaches between the two modalities. As both thermal sensors and visual sensors have their own shortcomings and strengths, we propose a novel joint filter-based hand gesture recognition method to simultaneously exploit the strengths and compensate the shortcomings of each. Our study is motivated by the investigation of the mutual supplementation between thermal and visual information in low feature level for the consistent representation of a hand in the presence of varying lighting conditions. Accordingly, our proposed method leverages the thermal sensor's stability against luminance and the visual sensors textural detail, while complementing the low resolution and halo effect of thermal sensors and the weakness against illumination of visual sensors. A conventional region tracking method and a deep convolutional neural network have been leveraged to track the trajectory of a hand gesture and to recognize the hand gesture, respectively. Our experimental results show stability in recognizing a hand gesture against varying lighting conditions based on the contribution of the joint kernels of spatial adjacency and thermal range similarity.
provenance
Made available in Cube on 2018-09-28T16:06:07Z (GMT). No. of bitstreams: 0
language
English
author
Kim, Seongwan
Ban, Yuseok
Lee, Sangyoun
accessioned
2018-09-28T16:06:07Z
available
2018-09-28T16:06:07Z
issued
2017
citation
SENSORS(17): 1
issn
1424-8220
uri
http://open-repository.kisti.re.kr/cube/handle/open_repository/486382.do
Funder
교육부
Funding Program
BK21플러스사업(0.5)
Project ID
1345274009
Jurisdiction
Rep.of Korea
Project Name
Institute of BioMed-IT, Energy-IT and Smart-IT Technology (BEST)
rights
openAccess
subject
joint filter
thermal sensor
visual sensor
hand gesture tracking
handgesture recognition
varying lighting conditions
type
article


Files in This Item

There are no attached files.