Free-hand interaction for handheld augmented reality using an RGB-depth camera

Huidong Bai; Lei Gao; Jihad El-Sana; Mark Billinghurst

doi:10.1145/2543651.2543667

Free-hand interaction for handheld augmented reality using an RGB-depth camera

Huidong Bai, Lei Gao, Jihad El-Sana, Mark Billinghurst

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

13 Scopus citations

Abstract

In this paper, we present a novel gesture-based interaction method for handheld Augmented Reality (AR) implemented on a tablet with an RGB-Depth camera attached. Compared with conventional device-centric interaction methods like keypad, stylus, or touchscreen input, natural gesture-based interfaces offer a more intuitive experience for AR applications. Combining with depth information, gesture interfaces can extend handheld AR interaction into full 3D space. In our system we retrieve the 3D hand skeleton from color and depth frames, mapping the results to corresponding manipulations of virtual objects in the AR scene. Our method allows users to control virtual objects in 3D space using their bare hands and perform operations such as translation, rotation, and zooming.

Original language	English
Title of host publication	SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013
Publisher	Association for Computing Machinery
ISBN (Print)	9781450326339
DOIs	https://doi.org/10.1145/2543651.2543667
State	Published - 1 Jan 2013
Event	SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013 - Hong Kong, Hong Kong Duration: 19 Nov 2013 → 22 Nov 2013

Publication series

Name	SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013

Conference

Conference	SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013
Country/Territory	Hong Kong
City	Hong Kong
Period	19/11/13 → 22/11/13

Keywords

3D user interfaces
gesture interaction
hand skeletonization
handheld augmented reality

ASJC Scopus subject areas

Computer Graphics and Computer-Aided Design
Human-Computer Interaction
Software

Access to Document

10.1145/2543651.2543667

Cite this

Bai, H., Gao, L., El-Sana, J., & Billinghurst, M. (2013). Free-hand interaction for handheld augmented reality using an RGB-depth camera. In SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013 (SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013). Association for Computing Machinery. https://doi.org/10.1145/2543651.2543667

@inproceedings{96988a9f67044e1d86f5c3b7a8ffc1c5,

title = "Free-hand interaction for handheld augmented reality using an RGB-depth camera",

abstract = "In this paper, we present a novel gesture-based interaction method for handheld Augmented Reality (AR) implemented on a tablet with an RGB-Depth camera attached. Compared with conventional device-centric interaction methods like keypad, stylus, or touchscreen input, natural gesture-based interfaces offer a more intuitive experience for AR applications. Combining with depth information, gesture interfaces can extend handheld AR interaction into full 3D space. In our system we retrieve the 3D hand skeleton from color and depth frames, mapping the results to corresponding manipulations of virtual objects in the AR scene. Our method allows users to control virtual objects in 3D space using their bare hands and perform operations such as translation, rotation, and zooming.",

keywords = "3D user interfaces, gesture interaction, hand skeletonization, handheld augmented reality",

author = "Huidong Bai and Lei Gao and Jihad El-Sana and Mark Billinghurst",

year = "2013",

month = jan,

day = "1",

doi = "10.1145/2543651.2543667",

language = "English",

isbn = "9781450326339",

series = "SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013",

publisher = "Association for Computing Machinery",

booktitle = "SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013",

note = "SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013 ; Conference date: 19-11-2013 Through 22-11-2013",

}

Bai, H, Gao, L, El-Sana, J & Billinghurst, M 2013, Free-hand interaction for handheld augmented reality using an RGB-depth camera. in SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013. SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013, Association for Computing Machinery, SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013, Hong Kong, Hong Kong, 19/11/13. https://doi.org/10.1145/2543651.2543667

Free-hand interaction for handheld augmented reality using an RGB-depth camera. / Bai, Huidong; Gao, Lei; El-Sana, Jihad et al.
SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013. Association for Computing Machinery, 2013. (SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Free-hand interaction for handheld augmented reality using an RGB-depth camera

AU - Bai, Huidong

AU - Gao, Lei

AU - El-Sana, Jihad

AU - Billinghurst, Mark

PY - 2013/1/1

Y1 - 2013/1/1

N2 - In this paper, we present a novel gesture-based interaction method for handheld Augmented Reality (AR) implemented on a tablet with an RGB-Depth camera attached. Compared with conventional device-centric interaction methods like keypad, stylus, or touchscreen input, natural gesture-based interfaces offer a more intuitive experience for AR applications. Combining with depth information, gesture interfaces can extend handheld AR interaction into full 3D space. In our system we retrieve the 3D hand skeleton from color and depth frames, mapping the results to corresponding manipulations of virtual objects in the AR scene. Our method allows users to control virtual objects in 3D space using their bare hands and perform operations such as translation, rotation, and zooming.

AB - In this paper, we present a novel gesture-based interaction method for handheld Augmented Reality (AR) implemented on a tablet with an RGB-Depth camera attached. Compared with conventional device-centric interaction methods like keypad, stylus, or touchscreen input, natural gesture-based interfaces offer a more intuitive experience for AR applications. Combining with depth information, gesture interfaces can extend handheld AR interaction into full 3D space. In our system we retrieve the 3D hand skeleton from color and depth frames, mapping the results to corresponding manipulations of virtual objects in the AR scene. Our method allows users to control virtual objects in 3D space using their bare hands and perform operations such as translation, rotation, and zooming.

KW - 3D user interfaces

KW - gesture interaction

KW - hand skeletonization

KW - handheld augmented reality

UR - http://www.scopus.com/inward/record.url?scp=84897369060&partnerID=8YFLogxK

U2 - 10.1145/2543651.2543667

DO - 10.1145/2543651.2543667

M3 - Conference contribution

AN - SCOPUS:84897369060

SN - 9781450326339

T3 - SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013

BT - SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013

PB - Association for Computing Machinery

T2 - SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013

Y2 - 19 November 2013 through 22 November 2013

ER -

Bai H, Gao L, El-Sana J, Billinghurst M. Free-hand interaction for handheld augmented reality using an RGB-depth camera. In SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013. Association for Computing Machinery. 2013. (SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, SA 2013). doi: 10.1145/2543651.2543667

Free-hand interaction for handheld augmented reality using an RGB-depth camera

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this