Work | Tao Design

Independent Study | Summer 2020

VIRTA

Interactive Programming

Virta is a interactive program that I programmed to symbolize people’s interaction in online conferences. It uses facial recognition, gesture reading and eye tracking technologies to capture the movement and attention of users.

Background

The covid-19 virus has made it extremely hard for schools and firms to operate in-person. Therefore, many different kinds of online conference platforms have emerged and become a very important part of people’s lives.

Zoom

Among many of the online conference softwares, zoom is the one that has the most market share.

Zoom Market Share

Zoom’s share takes up to 36.80% of the web conference market.

Under the impact of COVID-19, zoom’s user group grew at an unprecedented speed. It has become one of the most popular online conference platform in the world.

Other Platforms’ Market Share

Existing Problems

Survey

After addressing the problems, I want to find out which of them is the most annoying one to people. I designed a survey asking the following question:

What aspect of having online-classes annoys you the most?

Interview

After the survey, I interviewed a number of people for their views on the problems shown in the survey, and here are some interesting answers.

Brainstorming

After brainstorming and sketching, I’ve decided that my water bottle will have multiple compartments, each compartments will contain a certain amount of water that’s the right amount for runners to get hydrated. Therefore runners can finish one compartment at a time and don’t have to worry about the water shaking in the bottle.

Conclusion

Design Concept

Technology

Eye-tracking

Eye tracking is a sensor technology that makes it possible for a computer or other device to know where a person is looking. An eye tracker can detect the presence, attention and focus of the user. It allows for unique insights into human behavior and facilitates natural user interfaces in a broad range of devices.

Gesture Recognition

Gesture recognition here includes hand gesture recognition and face recognition. These two kinds of sensor technology together can recognize simple gestures of users.

Face recognition is a method of identifying or verifying the identity of an individual using their face. But here only the data like eye-positions, face outline and mouth outline are needed in order to track basic head gestures such as nodding, blinking and smiling.

Gesture recognition is a type of perceptual computing user interface that allows computers to capture and interpret human gestures as commands. The general definition of gesture recognition is the ability of a computer to understand gestures and execute commands based on those gestures

Cluster of Triangles

I started the concept sketching with the idea of visualizing the attention of the users into something that’s simple and easy to understand.

I begun with a triangle, for that a direction can be easily assigned to it. Then I proceed to a cluster of triangles, where a number or an mount can be assigned as well.

Concept Sketches

the Direction

Difference in
number/amount

Basic Functions

The most basic function, also the core function of this design would be to show different user’s focus. The cluster of triangles which represent the user will be following along the focus of the user and also be pointing towards their center of attention.

The blue circle represents the focus of the user, which can be tracked by eye-tracking camera.

Focus

Like the webcam, users will have the right to turn the attention tracking function off. When they do so, their cluster of triangles will remain static and change to color red.