Expressive Machine
An exploration of proposing new expressive nonverbal channels for CA to adjust user expectations

What will be a new form of Human-machine Interaction in creative realm when tool has its own thought? What if we give the right to children to choose the training images depending on their views of the world? How might we design a tablet app to extend creativity and maintain independent thinking for children(6~10) by using machine learning technology?

Project Info
Mar. 2019 - Jun. 2019
Unit 2 A Tool for Whom
Royal College of Art, London
Tools
Python - Machine Learning
Unity - Working Drawing Interface
Sketch - Final APP Interface

What will be a new form of Human-machine Interaction in creative realm when tool has its own thought? What if we give the right to children to choose the training images depending on their views of the world? How might we design a tablet app to extend creativity and maintain independent thinking for children(6~10) by using machine learning technology?

Project Info
Nov. 2018 - Jan. 2019
Unit 1 - Smart Habitat
Royal College of Art, London
Tools
Wekinator - Machine Learning
Leap Motion, Kinect - Gesture Tracking
After Effects - Concept Video
Project Info
Nov. 2019 - Jan. 2020
Unit 4 Design For The Unsettling
Royal College of Art, London
Tools
Google Assistant API - Voice Interaction
Arduino - Functional Prototypes
IFTTT - Online IoT Platform

What will be a new form of Human-machine Interaction in creative realm when tool has its own thought? What if we give the right to children to choose the training images depending on their views of the world?

Project Info
Oct. 2019 - Nov. 2019
Unit 3 - The Exhibitionist
Royal College of Art, London
Tools
Unity, C4D, Premiere - VR Prototype
Raspberry Pi - Tangible Heartbeat Prototype
AR.js, A-Frame - AR Poster
Role & Teammates
Research, Concept Generation, VR + AR + Tangible Interaction Design & Prototyping
Anne-Marie Heck, Rashmi Bidasaria

What will be a new form of Human-machine Interaction in creative realm when tool has its own thought? What if we give the right to children to choose the training images depending on their views of the world?

Project Info
Nov. 2018 - 5 days workshop
AcrossRCA - SenseAbility
Royal College of Art, London
Tools
Arduino - Sensory Interaction Prototype
Scent Making
A/B Testing
Role & Teammates
Research, Concept Generation, Sensory Interaction Design & Prototyping
Moritz Dittrich, Janina Frye, Sushila Pun, Yiling Zhang
Overview

Framed as “dialogue systems often endowed with ‘humanlike’ behaviour”, conversational agents (CA) are becoming ever more common human-computer interfaces. But in the majority of instances, the operation of the CA systems failed to bridge the gap between user expectation and system operation. Playful/humorous interactions such as telling a joke have the effect of reinforcing anthropomorphic qualities, setting unrealistic users’ expectations of CA capability. On the other hand, user interaction with CA remains passive as user always need to say “Hey Siri” or “Ok Google” to start the conversation and voice works as the only channel for expression. 

In order to make the CA more engaging and help user create more accurate expectations, Expressive machine is an exploration of new expressive nonverbal channels for CA in the form of embedding the agent’s personality by different modalities(Skin texture and colour, Smell and Gesture) to express emotional and internal states for interaction.

Machine 1 adapts to the surrounding environment by changing colours and only stands out to attract attention when necessary. Machine 2 diffuses different smells designed from machine perspective to provide information. Machine 3 expresses its internal state using skin texture changes. Machine 4 raises its hands to actively inform but not overburden user. Different combinations between those modalities and sentences even generate richer and funnier meaning of communications. As a result, Expressive machine is being used as an example of significantly enhancing the expressive spectrum of machines for social interaction to support user assessment of system intelligence.

The Process Behind Doodle Pet
view more projects