This website documents the design and research process for the project "THE BORDER".

 

Concept

​

“The Border” is an interactive experience that leverages the medium and technology of videgame and interactive virtual environment to depict the psychological journey of immigrants who escaped from the mainland China to Hong Kong across the Shenzhen border between the 1950s to 1970s. Instead of rendering the environment of the Shenzhen border realistically, the immigrants’ fear, anxiety, and hope for a better future are turned into a metaphorical virtual landscape.

 

Landscape of emotions around the border

​

In order to build the virtual emotional landscape for the project, interviews were being done with people who personally experienced such journeys. The focus during the interviews was their spatial and emotional experiences while navigating the landscape around the border during the escape. Multiple newspaper coverage and the book “Da Tao Gang” written by journalist “陳秉安” were also being taken as references.

 

According to the research, there were 4 routes which people who want to escape to Hong Kong at that time would normally take. The first being the one through Dapeng Bay, the second one through Wutong Mountain, the third one through Shenzhen Bay, and the forth one through Macau. The route through the Wutong Mountain was chosen as the focus of the project.

 

Below summarizes a few key points from the interviews and research.

 

- Many people escaped because of famine and also many could not see a future in their home town.   

 

- The journey from their home village in mainland China to Hong Kong could take over 7 days. It was not only a journey of physical exhaustion but also emotional desperation as many were leaving their families and facing an extremely uncertain future. During the journeys they had to hide from the border guards in the woods of the Wutong Mountain. The forest was filled with fear and anxiety at night.

 

- During their escape, they would hide and take rest during daytime as they would easily be seen by border guards. They would only progress at night.

 

- The landscape of Wutong Mountain was full of high cliffs and strange large rocks, and therefore was very dangerous to navigate at night. Many of the people who escaped through the mountain died from mistepping and falling off high cliffs.

 

- Many people lost directions in the woods of the Wutong Mountain. The mountain was extremely dark at night. Sometimes they could see very little amount of light coming from the very far. They followed the direction of light, because they knew that this would lead them to Hong Kong. In their mind, Hong Kong was the city of light.

 

- Most of the people avoided using flashlight, because that would expose them easily to the border guards. They navigated the forest with the little amount of light coming from the blinking stars in the sky.

 

- There were fences and border guards everywhere, making this a more difficult route compared to other ones. Many people made multiple escape attempts before they could successfully arrive Hong Kong. When they got caught they would probably have to be sent to labor camps.

 

 

How to visualize the emotions

​

After the research, the efforts were put into exploring ways of visualizing the emotions of these people, and also how these emotions could be translated into the virtual landscape.

 

One of the key features of this interactive experience is to have the main character, which the player controls to navigate the virtual environment, to be a dog instead of a human. The reason is to use dog’s outstanding sense of smell as a key mechanic for the player to see the map of human emotions in the virtual landscape.  

 

Dogs read about the world through their noses as their sense of smell is approximately one million times more sensitive than humans. They pick up many things invisible to us with their noses. For example, human shed 50 million skin cells each minute which are visible to the dogs as microscopic snowflakes in which they can read emotions such as fear, anxiety, and sadness. These emotions are particularly very detectable by dogs because these emotions are accompanied by increased blood flow and heart rate, which sends body chemicals more quickly to the skin surface.

​

​

​

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Therefore, when players navigate the virtual landscape with the perceptual experiences of a dog, they see residue of human emotions in the air, left behind by people who were there. For example, in the above screenshot, players can see the person stealthing and hiding from the guards in the forest.

 

​

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

​

​

​

​

In another example, players can see the traces of anxiety from other dogs suffering from starvation and looking for food in the trash.

 

Such mechanic helps to visualize the emotional landscape around the border, including panic from starvation, the sadness of losing close ones , the anxiety when navigating the dangerous woods of Wutung Mountain during escape, etc.

 

 

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

 

 

Apart from visualizing the emotional landscape, the interactive experience also makes use of the fact that dogs has relatively poor vision but outstanding sense of smell to develop an unique interactive mechanic. Players use the arrow keys to control the dog to walk. As the above screenshot indicates, the visuals are black and white by default because dogs have pretty poor vision.

 

​

​

​

​

​

​

​

​

​

​

​

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

​

 

 

 

 

But when the users clicks and holds the right mouse button, the dog would start to smell the ground and that particular part of the ground would turn colored so the users can see the environment with more details. Such interactive mechanic encourages the user to actively explore the environment. 

 

​

Visual design

​

A large amount of research and development work went into developing the visual aesthetics of the interactive experience.

 

​

​

​

​

​

​

​

​

 

 3D Mesh                                                                                                              Point Cloud                                                                                                       Geometry Shader

 

The virtual environment is rendered with a dark and expressive aesthetics. To achieve this visual style, I worked together with 3D artist and developer Kachi Chan to develop custom procedural systems to convert 3D meshes of the virtual environment into billions of vertices. The vertices are then transformed into equal number of tiny triangles via geometry shader, contributing to a noisy and expressive effect similar to a pencil drawing. The down side of this point cloud technique is that from a computational point of view, it consumes a huge amount CPU power and memory, causing a lot of serious performance issues when running the interactive experience. Therefore we also tried using the approach of developing custom shaders for the 3D models of the environment, but we were unable to achieve the same artistic style and therefore went back to the point cloud technique.

​

​

 

​

​

​

​

​

 

 

3D Mesh                                                                                                               Point Cloud                                                                                                       Geometry Shader

 

​

For the traces of emotions from the people, we converted a variety of motion-capture animations into a trail of 3D meshes that look similar to a slow shutter effect in photography, and turned them into millions of vertices and tiny triangles via geometry shader.

​

​

Video

A screencapture video of the interactive experience can be viewed and downloaded here. 

​

​

​

​

​

​

​