INTERACTIVE INSTALLATION - POLICE
Project description
August 2015
1 month
Policing
2 members
Coordination
Product Strategy
UX/UI Prototype
Graphic Design
Programming
Tester
Blender 3D
Adobe Photoshop
Adobe Illustrator
Unity 3D / C#
Microsoft Kinect SDK
Brazil’s police force is split into four categories/divisions. One of these is the military police, which is responsible for guaranteeing security at large and dealing directly with citizens. Although it shares some of the military’s organizational’s structure, it is its own entity.
The Military Police of the State of Minas Gerais carries out educational campaigns in schools, malls, and public events. To converse with children, the institution has a character called “PM Amigo Legal”, which translates into Cool Military Policeman Friend. This character originally existed as a costume, which is worn by a police officer. There are reports that the costume is heavy and exhausting to worn be on hot or sunny days. Another big problem is that although there are few costumes, the state of Minas Gerais is large, with over 500 municipalities, which creates a big logistical problem regarding the costume’s usage.
The proposal of this project is to model this character in 3D and build an interactive installation that uses motion capture to bring the character to life. The recognition of movements must be performed in real-time so that the installation can be used in real-time activities, within, for example, schools.
Using Microsoft’s Kinect, I created an application that can be used remotely to bring the character to life. The project hides a police officer in a room, with a Ms. Kinect reading their movements, speeches, and gestures, and transports them to the character who is with the children in another environment (a classroom for example).
At this time the children ask questions, clear up doubts and play games with the character. The police officer in the other room sees and hears the children through a simple videoconferencing system.
The solution facilitated educational campaigns because, in addition to solving logistical and operating problems of the costume, it improved the number of events that could be offered simultaneously.
The application also makes it possible to change the background in which the policeman appears, for art, photos or videos. This feature greatly increased the versatility of the tool, which can be customized for each educational context.
The installation had a very positive impact and was presented at the 1st National Congress of Organizational Communication of Military Police.
Problem:
The interactive installation project was very challenging. The entire system should use inexpensive, market-scale technologies. At the time, we had few gesture computing sensors available on the market. Kinect had a public SDK and an affordable price.
In summary, the project’s core challenge was:
Primary Problem: How to create a playful interactive installation where a virtual character can talk and teach concepts of citizenship, traffic education as well as combating domestic violence and drugs?
Secondary Problem: Design playful question/answer games to ensure children’s motivation to use them.
Goals:
- Respond to children’s questions and actions in real-time;
- Being able to customize scenarios to work with different themes;
- Capture the sound of the character’s voice to control the lip sync.
- Convert a person’s movements to a short stature cartoon character;
- Have simple technological infrastructure for transportation;
- Perform remote data transmission to enable the character’s operator to be somewhere else.
Interactive Installation Operation
[in portuguese]
The project is born out of the operational and logistical problems of the character’s costume. The number of events was limited. Some educational materials printed by the Military Police already used the character in 2D cartoon format.
The first phase of development was the 3D modeling of the character and creation of backgrounds themed around citizenship, traffic education, combating domestic violence, and fighting drugs. In this phase, the construction of the Rig (internal skeleton) of the character was carried out in a Kinect-compatible manner.
The prototype is born with the following format:
Students are placed in a room (eg classroom) that has a camera, microphone, and projector. The projector shows the virtual character that converses and interacts with children through audio/video. In another room, the Policeman who controls the character is placed. There, Microsoft Kinect captures movements, a headset microphone captures audio, and the software uses motion tracking and audio waveform to bring the character's articulations and lip-sync to life.
Everything travels over a local TCP/IP network and, as it is only a stream of audio and video, it has very low latency, with effectively no delay.
The installation had a very positive impact and was presented at the 1st National Congress of Organizational Communication of Military Police. Children were also very surprised by the character. Older children initially think that it is a pre-recorded animated video and are surprised when they realize the character can answer questions like: "If you can see me... What color is my shirt?".
The feedback from this project was very gratifying, being able to deliver dreams and learning to children through a technology project.
Presentation at the National Congress of Military Police
[in portuguese]