| Author: | Sin, Ping Tat |
| Title: | Digitalizing a stuffed toy for interactive play : appeal, tracking and rendering |
| Advisors: | Leong, Hong Va (COMP) |
| Degree: | Ph.D. |
| Year: | 2023 |
| Subject: | Mixed reality Human-computer interaction Hong Kong Polytechnic University -- Dissertations |
| Department: | Department of Computing |
| Pages: | xxx, 217 pages : color illustrations |
| Language: | English |
| Abstract: | In childhood, have you picked up a stuffed toy and pretended to play with it? What if, at that moment, the stuffed toy waved back at you and acted as your ever-loving companion? This thesis proposes the provision of a novel interactive experience via digitalizing a physical stuffed toy. Specifically, a stuffed toy is tracked such that its pose is recognized. When the user interacts with the toy, the user action or the state of the toy can be inferred via the tracked pose of the toy. Via a photo-realistic rendering, the user will be able to see the toy moves "itself" via a display. Succinctly, the works of this thesis can be partitioned into three parts, appeal, tracking and rendering of stuffed toys. This thesis starts with investigating user appeal toward a digitalized stuffed toy for feasibility. First, this is done so via empirical means. A bandage-based prototype is developed to wrap around the joints of the stuffed toy which enables tracking of the toy's articulation. User studies have been conducted to observe how users will respond when interacting via a stuffed toy. It is shown generally that users are interested in and perhaps even prefer playing via the stuffed toy, and that it is a more intuitive way to play extended reality (XR) games in some situations. Another important finding is that children are also quite keen on with playing the toy. The second part of the investigation is done so via a theoretical meta-analysis. Via an inquisitive visit to the literature of XR and play. It is found that a new class of controllers, free-form naturally mapped control interface, can benefit presence, a core focus for many appealing XR experiences. In short, the appeal of playing with a digitalized stuffed toy is empirically and theoretically validated. Then, the research focus moves to investigate how to track the pose and deformation of a stuffed toy. This is necessary as many games will require action recognition and collision detection. In addition, a virtual replica of the stuffed toy's shape will enable a more visually appealing augmentation for the toy. The proposed soft-pose estimator seems to be the first in enabling an interactive session for XR via a tracked stuffed toy. The core idea on how to track the soft-pose is to combine cage-based deformation, a classical computer graphics technique, with a modern pose estimation network. The quantitative evaluation and the user studies show that the soft-pose estimator is effective. More importantly, user feedback shows that the tracked stuffed toy is a competitive modal for play. The investigation then turns toward the issue of rendering. It starts with an exploration of image-based approaches that enable conditional input via latent pathing. To ride on the fidelity that neural rendering can provide, however, further investigation has adopted this increasingly popular set of rendering approaches as a new paradigm. An issue is that the majority of previous neural rendering works seem to focus on static scenes. To enable animation, neural proxy, which uses shell maps, another computer graphics technique, to localize particles sampled from a neural radiance field, is proposed. The quantitative result shows that neural proxy's rendering has a higher fidelity compared to other neural rendering methods. A qualitative result rendering an animated photo-realistic stuffed toy is also presented, further showcasing neural proxy's ability in rendering animated content for 3D interactive environments. With all three areas addressed, an integration of research work is presented at the end to demonstrate how to realize a digitalized stuffed toy. It is hoped that this work not only demonstrates a new way to play, but can also be useful for various childcare applications in the future. |
| Rights: | All rights reserved |
| Access: | open access |
Copyright Undertaking
As a bona fide Library user, I declare that:
- I will abide by the rules and legal ordinances governing copyright regarding the use of the Database.
- I will use the Database for the purpose of my research or private study only and not for circulation or further reproduction or any other purpose.
- I agree to indemnify and hold the University harmless from and against any loss, damage, cost, liability or expenses arising from copyright infringement or unauthorized usage.
By downloading any item(s) listed above, you acknowledge that you have read and understood the copyright undertaking as stated above, and agree to be bound by all of its terms.
Please use this identifier to cite or link to this item:
https://theses.lib.polyu.edu.hk/handle/200/12379