Applications of Projector Phones for Social Computing
Lisa Cowan, William G. Griswold, James D. Hollan
University of California, San Diego 9500 Gilman Dr. La Jolla, CA 92093, USA +1 (858) 534-9669, +1 (858) 534-6898, +1 (858) 534-8156
Although mobile phones can help us stay connected with friends, family, and colleagues, these personal communication devices can also isolate us from the world around us by drawing our attention into a private digital realm. In contrast, public displays can catalyze conversation with people nearby. Mobile phones with integrated pico projectors have the potential to combine the phones computation and communication capabilities with the projected displays ease of sharing, creating a personal, mobile, public display. Enabling the phones content to become visible to co-present people could help bridge the gap between the physical and digital realms, and even enhance face-to-face interaction. In this paper, we explore applications of projector phones for social computing. We characterize technical properties and social implications of projector phones, explore the design space through scenarios, illustrated by low-fidelity prototypes, and discuss design opportunities and challenges.
Figure 1. LG Expo projector phone
2. RELATED WORK
Numerous sociological and ethnographic studies have focused on mobile phone use and related social practices [7, 14]. Other research concerns social interaction around public and situated displays [10, 12], including mobile peripheral displays [4]. Another literature investigates applications of projection, especially camera projector systems, to augmented reality, tangible computing, and ubiquitous computing [13, 16]. Recent work has explored applications and interaction techniques for small, portable projectors. The Hotaru system presented interaction techniques for supporting collaboration with projected displays [15]. Cao, et al., have demonstrated techniques for interacting with virtual information spaces embedded in the physical environment using handheld projectors and passively tracked pens. They present a variety of application scenarios, for single user [1] and multi-user interaction [2]. Mistry, et al., studied tangible and gestural interaction with WUW, a.k.a. 6th sense, a wearable camera projector system [11]. Recent advances in hardware have enabled researchers to study mobile phones with attached pico projectors. The Maurauders Light system projects buddies locations onto paper maps [9]. Greaves and Rukzio have comparatively evaluated mobile phone displays and projectors for photo browsing tasks [5], and have developed a framework for collaborative media viewing and sharing with projector phones [6]. This paper focuses on applications of projector phones for social computing. We characterize the technical properties and social considerations of an early commodity projector phone, and contribute a set of scenarios intended to catalyze future thinking, discussion, and research.
1. INTRODUCTION
Mobile phones can help us stay connected with friends, family, and colleagues. Yet they can also sometimes isolate us from the physical world around us, drawing our attention into a relatively private, personal, digital world of calls, text messages, and email. Further, mobile phones small screen size and limited viewing angle make it difficult to share displayed information with copresent people. One can hold a phone up for others to see or pass it around yet neither strategy adequately facilitates simultaneous group-based communication or lightweight, opportunistic interaction. Public displays, in contrast, present information to many people simultaneously, and have been shown to catalyze conversation and interaction [10, 12]. Mobile phones with integrated pico projectors can potentially combine a phones computation and communication capabilities with a large projected displays ease of sharing, creating a personal, mobile, public display. By enabling users to make their phones content visible to others and to superimpose digital information onto the physical world, projector phones have the potential to bridge the gap between the physical and digital realms, enhance face-to-face interaction, ease social barriers, and engender rich forms of visual communication. In this paper, we contribute scenarios that start to map out the design space of potential applications of projector phones for social computing [3]. We illustrate the scenarios with low-fidelity prototypes implemented on the LG Expo (Figure 1), an early commodity projector phone. First, we discuss related work, and then we discuss some technical characteristics of projector phones and the design considerations that result. Next we present a set of scenarios and low-fidelity prototypes and discuss specific opportunities and challenges in the design of these systems.
3. CHARACTERIZATION
We describe technical characteristics of projector phones, drawing on the specifications of the LG Expo projector phone as an example, and discuss some design considerations.
3.1 Technical specifications
The LG Expo is a Windows Mobile smart phone with a touch screen, stylus, and QWERTY keyboard. It has an integrated, removable projector, which is activated and focused by physical sliders. The HVGA resolution (480320) DLP pico projectors brightness is unspecified, but similar models output in the range of 6-12 lumens. The Expos projection is not visible in bright light. Figure 2 illustrates the Expos brightness in indoor daytime lighting with window shades open or closed, and an incandescent overhead bulb on or off. In moderate lighting conditions, such as these, an approximately 6 ft projected image is easily visible from a 10ft throw, and in darker conditions the size and throw distance can be more than doubled. Our tests indicate that the phones battery power (1500mAh) lasts around 2 hours when the projector is on. The Expos camera and projector cannot be operated simultaneously and in fact are oriented in different planes.
conditions. A projector also places high demands on power; currently, battery life precludes sustained projection unless the phone is plugged in. We expect these limitations to lessen with hardware advances. On the LG Expo, the projected displays content is identical to that of the LCD display. Providing distinct views on each display could enable additional applications and interaction techniques. As sold, the LG Expo does not directly support the class of computer vision-based interaction techniques typically supported by camera/projector systems, because the camera and projector cannot be operated at the same time. Given the Expos hardware configuration, one could reduce the need to look away from the projected display to interact with the phone (via the keyboard and touch screen) by providing some visible indication of a users interactions on the touch screen in the projected display, or by enabling voice activation. A projected display adds a layer of indirection between ones phone and viewers of its content; one can display information without demanding that a particular person attend to it. As discussed later, this indirection could reduce social barriers to interaction, enabling people to share information they might not otherwise share and interact with people with whom they might not otherwise interact. Projecting from a phone risks the privacy of displayed content; information may leak to unintended viewers. Projector phone users can choose from multiple display modes (e.g., projector, LCD) based on the privacy of shared content, security of the physical location, number of viewers, etc. Access controls, such as those demonstrated by Cao, et al. [2], could help. Projection also risks invasiveness; people nearby may see ones display without wanting to, analogous to forced eavesdropping on mobile phone conversations, and some types of projection may be unsuitable or unwelcome. New social norms will likely develop as people adjust to this new technology.
Figure 2. Indoor lighting, clockwise from top left: window and incandescent, window only, neither, and incandescent only
4. SCENARIOS
We envision social applications of commodity projector phones, not directly implied by the business applications to which they have been marketed. We explore several potential social applications through a set of scenarios, illustrated by low-fidelity prototypes, to highlight opportunities and challenges. We ground our discussion of potential applications by considering what is possible with the LG Expo.
3.2 Design considerations
Mobile phones have become ubiquitous in many parts of the world, and integrated pico projectors extend them for ubiquitous, personal, public display. Projector phones are relatively small, light, portable, handheld devices, and their applications can exploit the hardware and software todays mobile phones typically include for positioning and sensing (e.g., accelerometer, GPS, WiFi), multimedia capture and display (e.g., photo, video, audio), and networked communication (e.g., voice, SMS, MMS, Internet). Projector phones provide some of the affordances of large displays, avoiding the size constraints of todays comparatively small mobile phone screens, enabling vision-assistive technology, and making it easier to share information with many people simultaneously. Projector phones also enable people to use a variety of surfaces for ad hoc display and to combine the properties of the projection surface, the projected content, and the motion of the projector to create meaning. Currently, a projector phones limited brightness precludes its use in bright lighting, such as a well-lit area of a typical office or direct sunlight, although an application could dynamically adjust colors and fonts to optimize visibility based on lighting
4.1 Opportunistic sharing
Scenario. Bob, a graduate student attending an academic conference, walks down a hallway. He begins chatting with Sally, a respected researcher who is also walking toward the elevator, and gives her a brief summary of the work hell be presenting at the conference. Sally asks for more details, so Bob takes out his phone and projects his research poster onto the wall near where they are standing. Sally points at a figure and asks about some of his results, and he zooms in on that area of the poster (Figure 3). They step into the elevator and briefly continue to discuss the figure. Sally uses her phone to take a snapshot of the 2-D code in the corner of the projected poster to save a link to Bobs website for future reference. As the elevator arrives at Sallys floor, Bob thanks Sally for the chat and she steps off the elevator.
Figure 3. Opportunistic sharing, catalyzing conversation, humor and play, and collaborative coordination Discussion. In this scenario, a portable projected display provides a shared visual artifact that augments an ad hoc discussion on the go. People near the discussions active participants also share this artifact, so of course public projection is not appropriate when privacy is critical. Currently, such an opportunistic conversation would likely not benefit from supplementary visual information. Also in this example, a displayed 2-D visual code, as discussed by Kray and Rohs [8], becomes a virtual business card. In this sort of mobile scenario, the projector phone is likely to be held in the hand, constraining that hand from gesturing or performing other tasks. Also, for ideal viewing the projector should be held steady, more important with increased throw distance or movement, perhaps held with both hands or rested against the body. eating it), using the content, position, motion, and size of the projected image. We also see how projecting onto a surface is a form of virtual, visual touch at a distance; this type of interaction could escalate to mischief (e.g., projecting a kick me sign onto a persons back) or cross social boundaries (e.g., projecting onto socially restricted parts of the body or onto an unfamiliar person).
4.4 Collaborative coordination
Scenario. Lynne and Frank are with a group of friends, when someone suggests ordering a pizza. Lynne gets out her phone and searches for her favorite pizza restaurants menu. She projects it onto a wall and asks the group, What kind of pizza should we get? As people read and discuss the menu, Frank says, I like [another pizza place] better. What do you guys think? and he projects another restaurants menu onto the wall next to Lynnes menu. A friend of theirs looks at Franks menu and points out a pizza that looks good, and Frank projects his menu a bit higher than Lynnes (Figure 3). Another friend speaks up for a pizza on Lynnes menu, and Lynne pulls her phone back from the wall, to enlarge her projection a bit, and begins to cover Franks menu with hers. Lynne and Frank continue to compete for projection primacy as they attempt to convince the group to choose their preferred pizza places, until a decision is made. Discussion. Here, multiple projected displays support a collaborative decision making process. Currently, a few people can share a physical or digital menu (e.g., on a phone or laptop), yet a large group could not share such a display simultaneously, as they do in this scenario. Also, multiple displays can expand the display area and enable each person to provide their own perspective. We also see how people might use the spatial placement (e.g., size, location, organization) of projections to indicate or compete for priority.
4.2 Catalyzing conversation and interaction
Scenario. Tim is at a party, chatting with his friend John. Tim notices a girl (Jane) standing nearby, and hed like to meet her but hes a bit shy. John asks about Tims recent snowboarding trip, so Tim projects some pictures from the trip onto the wall. Some people standing near Tim and John become interested in the pictures and form a small group around them. Jane is curious and comes over to see what the group is looking at, and she starts to chat with Tim about his trip (Figure 3). Tim is happy and relieved that he didnt have to work up the nerve to talk to her. Discussion. Here we see how a projected display can lower barriers to conversation and social interaction. Currently, you can meet people you dont know at party by directly approaching them and initiating a conversation; this direct approach demands confidence and risks rejection. In the scenario the projected photos ease the social pressure of personal introduction and breaking the ice by providing a topical resource and an indirect approach. One could also imagine a scenario in which the public nature of a projected display invites unwanted attention.
4.3 Humor and play
Scenario. Sue and Rick are friendly colleagues who like to share a laugh. Sues sitting at her desk mid-afternoon and starting to get hungry. She pulls out her phone and searches the Internet for a picture of her favorite donut from a nearby shop that she and Rick frequent. She projects the donut onto the wall and begins moving it closer to Ricks desk. As the projected donut hovers on the wall near him, he glances up from his work, notices the donut, and starts laughing. Are you trying to tell me something? he says, and she replies, Yes, and then projects the donut onto her stomach. As he smiles and considers whether to head out for a quick donut break, she projects the donut onto his stomach (Figure 3), which makes him laugh and convinces him to go. Discussion. In this case, a projector phone facilitates playful interaction and enriches visual communication. It enables people to construct meaning from the combination of virtual and physical elements (e.g., a donut projected onto ones stomach implies
4.5 Situated display
Scenario. Kyle wakes up and glances at his surfboard, beside his bed. He's configured his phone to project a display onto the surfboard when his alarm goes off (Figure 4). He notices that the wave forecast is good and his friend Anne wants to go surfing. He calls her and they meet for a quick surf session before work. Later, in the office, Kyle sets his phone on his desk, where it projects an ambient display onto the wall next to his monitor. It shows his colleagues current tasks and availability, project status updates, and some favorite personal photos. He can see it out of the corner of his eye and his office-mates can see it with a glance in his direction. He selects a task from his to-do list and indicates that he doesnt want to be disturbed. Occasionally he looks up and notices a personal photo, and it cheers him up. He starts working on another task and indicates that hes available. One of his collaborators walks over to discuss a project, and afterwards they chat about some photos of mutual interest that are projected on the wall, and it helps them get to know each other better.
Figure 4. Situated display, personal Expression, Discussion. This scenario demonstrates how a projector phone could be used as a personal, public, situated display. Such a display could provide ambient information and facilitate social awareness without causing undue distraction. The projector phones sensing capabilities enable the displays contents to be tailored to suit the situation (e.g., projection surface, location, activity, people nearby), and its mobility precludes the need for multiple dedicated peripheral display devices (e.g., Chumby). This usage scenario, however, requires the display to be preconfigured and the phone to be plugged in (e.g., to an electrical outlet or a USB port) and set up for viewing.
Observational study of peoples practices and experiences with projector phones is, of course, an important area for future research. The design of projector phone applications that explicitly support face-to-face social interaction presents especially interesting opportunities, and designers should continue exploring solutions to the social and technical issues raise here. Interaction techniques, such as computer-vision based gestural interaction, will also likely evolve as cameras in phones and computation power of phone hardware advance.
6. REFERENCES
[1] Cao, X. and Balakrishnan, R. Interacting with dynamically defined information spaces using a handheld projector and a pen. Proc. UIST '06. [2] Cao, X., Forlines, C., and Balakrishnan, R. Multi-user interaction using handheld projectors. Proc. UIST '07. [3] Carroll, J. M. 2000. Making use: scenarios and scenariobased design. Proc. DIS 2000, ACM Press. [4] Cowan, L., Griswold, W.G., Barkhuus, L., Hollan, J.D., "Engaging the Periphery for Visual Communication on Mobile Phones." Proc. HICSS 2010. [5] Greaves, A. and Rukzio, E. Evaluation of picture browsing using a projector phone. Proc. MobileHCI 2008. [6] Greaves, A. and Rukzio, E. View & share: supporting copresent viewing and sharing of media using personal projection. Proc. MobileHCI 2009. [7] Ito, M., Okabe, D., and Matsuda, M. 2006 Personal, Portable, Pedestrian: Mobile Phones in Japanese Life. The MIT Press. [8] Kray, C., Rohs, M., "Swiss Army Knife meets Camera Phone: Tool Selection and Interaction using Visual Markers", Proc. MIRW 2007. [9] Lchtefeld, M., Schning, J., Rohs, M., and Krger, A. Marauders light: replacing the wand with a mobile camera projector unit. Proc. MUM 2009. [10] McCarthy, J. F., Costa, T. J., and Liongosari, E. S. UniCast, OutCast & GroupCast: Three Steps Toward Ubiquitous, Peripheral Displays. Proc. Ubicomp 2001. [11] Mistry, P., Maes, P., and Chang, L. WUW - wear Ur world: a wearable gestural interface. Ext. Abstr. CHI 2009. [12] O'Hara, K., Perry, M., Churchill, E. and Russell, D. Public and situated displays: Social and interactional aspects of shared display technologies". Kluwer Academic Publishers, 2003. [13] Pinhanez, C. S. The Everywhere Displays Projector: A Device to Create Ubiquitous Graphical Interfaces. Proc. Ubicomp 2001. [14] Plant, S. On the Mobile: the Effects of Mobile Telephones on social and individual life. 2003. http://www.vodafone.com/ flash/receiver/06/articles/pdf/01.pdf. [15] Sugimoto, M., Miyahara, K., Inoue, H., and Tsunesada, Y. Hotaru: Intuitive Manipulation Techniques for Projected Displays of Mobile Devices. Proc. INTERACT 2005. [16] Underkoffler, J., Ullmer, B., and Ishii, H. Emancipated pixels: real-world graphics in the luminous room. Proc. CGIT 1999.
4.6 Personal expression and advertisement
Scenario. As Jerry walks to the subway, he listens to upbeat music and projects bright colors onto the ground around him, to brighten the overcast day and match his mood and outfit. His virtual business card is visible along the edge of the projected display. On the subway he sits next to a girl, dressed in black and surrounded by a projection of dark red flames swirling around her favorite musicians, who scowls at him (Figure 4). He moves over when her projected flames hit his pants leg. When he gets to work, his colleague Lauren notices his cheery projection and observes that he must be in a good mood. Discussion. In this case, a projector phone creates a dynamic personal backdrop that expresses a persons mood and style. It can advertise personal interests, like bumper stickers and t-shirts, or commercial interests, like business cards or flyers. Ones personal projection, however, might be invasive to others for a variety of reasons. The projection surface might be restricted (e.g. an unfamiliar person, a public building), the projected information might be offensive, or a person might simply not wish to see it. Sustained projection, as described here, would require longer battery life than current systems provide.
5. Conclusion
We have begun to map out the design space of potential projector phone applications for social computing through a set of scenarios, illustrated by low-fidelity prototypes. This design space can be classified along dimensions including the intended purpose of projection, the roles and relationships of participants, the spontaneity, duration, and mobility of interaction, and the privacy requirements. We have considered scenarios that highlight how projector phones can facilitate opportunistic sharing, conversation, playful interaction, collaborative coordination, situated display, and personal expression. We have discussed how personal, mobile, projection could enrich communication, lower social barriers, and enhance awareness. It could also risk the privacy of projected information and be invasive to uninvolved people, potentially causing visual clutter, distraction, or social offense. Our scenario-based design exploration is intended to seed discussion and expose opportunities for future research.
