Interactive Journey

Interactive Plan & Execution

After finishing off with the mixing and sound design, now I had to create a pipeline of how will the whole changing of clips and interactivity work in the film.
On paper, the basic thing was to make a set of array, put the clips in these array and call them on events to play.

IN LAY MAN’S TERM

Think array as a set of empty boxes, now I would put clips (each clips, with respect to the journey and story) into this boxes. Now think, a ‘call’ as an order, that will be given out to see whats inside of these boxes.
Now imagine the word, ‘Event’ like a well, event and when that event is triggered a call will be coming out of it. That event can be anything, a time, situation, weather etc.

LAYERS:

So the basic structure was this but there are layers to it. One of the layer which I didn't think about was, UI.
User Interface was a very small part of the film, I just needed a UI to help the viewers interact with the Film. Many films have a very unique way to give this UI specially the choice based games and interactive films

There are many types of UI, these can be discribed and differed with their presence and importance in the story and space of a video game. The most known of them are.

1. Non-Diegetic: Doesn't exists in the story and neither in the 3D space or game space

In Super Mario Bros. 3, the stat meter is non-diegetic because it exists outside of the game world and story (characters within the game don’t know it’s there).

1. Diegetic: Exists in Space and story

In the demolition racing game Wreckfest, cars are diegetic UI components. Over the course of a race, they take on visible damage that indicates how near a player is to being knocked out of competition.

1. Spatial: Exists in Space but not in story

The American football franchise Madden has spatial UI components that help players select avatars and understand game scenarios.

1. Meta: Exists in Story but not in space.

The Legend of Zelda utilizes scrolling text (a meta component) to advance the narrative and provide players with helpful tips

GAME UI CHART

Detailed explanation

Now I love the UI that exists in Space and most of the choice based game have UI in the space at it makes it feel more real than just a UI based VIDEO GAME.

Erica and Detroit: Become human

For me as bad as i wanted it to be, it made no sense because It would have increased a lot of work on my side with editing. The process would be me trying to roto the UI in the editing and then use that part in the engine to make that section interactive.
This sounds easy but there was a whole lot of space for error and I would rather choose the another version of UI which are done in many choice based video game and Bandersnatch.

Wolf Among Us and Bandersnatch

This became my reference, planning for a basic animation was important as it would be one of the reason for making my film seamless.

FLOWCHART

Next thing was to create a flowchart for me to understand the basic requirements of the interaction, In a large scale video games development process these flowchart help a lot to jot down things that a production requires, we can call it like a very formal mind map where every little thing is important.

This would help me to know what kind of assets, effects and other things are required other than the things which are already there in the videos.

Basic flowchart stating the bare minimum

This was the first flowchart that I created for the interaction part of it.

This made me realize that there are small things missing in this action. The biggest and most problematic thing was to detail the animation of the UI and the flowchart helped me to think about it, with this I created another more detailed flowchart for the complete interaction action.

Detailed interaction flowchart

This flowchart helped me to be on the correct track and from here I had to put this down into the engine and code it.

CODING

Well I am not good at this thing but I am not bad at it too. I have created lot of games individually and was confident to do thing like this. The major thing here was that, A game contained everything in a 3D space and here I was creating everything in a 2D space, to be more precise I had to create a UI based game.

Get Vinay Kumar Sharma’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

After some YouTube tutorials and talking to my friend I decided to take advantage of the UI aspect of Unity and create the game.

In lay man’s term, lets just think the UI to be like a projector screen, I would be projecting my videos on this screen inside the engine and put a camera in front of it and the camera will then render the projection of this Projector screen to the viewer. Think it like recording a movie in a theater with your camera but in 4K.

UNEXPECTED PROBLEM

This can be like a rant but my I have been using my laptop since a long time and it has always failed to service me at the right time. Not I would expect something wrong to happen but something weird to happen was something which I didn't expect.

The most weirdest thing that happened was that Unity has a codding platform where we use C# to program, and that software aids you in the syntax and other stuff. Not for some reason I was not able to get this autofill syntax and last time I remember was that I am not a good coder, without syntax my life would be a hell.

After all the unnecessary try and not going anywhere I did some research and was finally going to hard code everything but then I found a visual scripting plugin for unity called Fungus.

Fungus lets you do visual coding using nodes just like Da Vinci resolve, this made my life easier and I was able to control even the animation using this tool.

CREATING SCENES

Now according to the flowchart I planned to rather change the scenes than destroying a video and then making other appear, it just seemed very messy to me and creating scenes was little cleaner.

Now let me explain again in a layman term, Think every scene to be like rooms in a house, and each room is there for a reason according to the your requirement.

For example if you need to go to the pee, you choose bathroom. You want to eat you choose kitchen.

Similarly I created rooms in this application where each scenes contained a shot of the film, and with respect to the decision(needs) the viewer chooses they enter that scene.

ANIMATION OF UI

When I created the flowchart it helped me recognize the animation required for the UI appearance and disappearances and also the reaction of UI buttons when mouse would hover on top of it.

The mouse hover had to be done with states, for simple explaination it was about what would happen when mouse will be.

1. Over the button boundary (Highlighted) — The underline would appear
2. Out of the button boundary (Disabled) — The underline would disappear
3. When the button is clicked (Pressed)— UI Disappear animation would play

these three things were the states of the button. When the individual action will occur the buttons will go into the respective states and that led me create the state machine for the buttons and UI Panel.

UI Button state machine

UI Panel State Machine

UI appear and disappear animation

INTERACTION WITH UI

The flow of the interaction was set according to the flowchart where the viewer would click on the choice and the scene would change, this included the wait time for the animation to play.

This let to create the final fungus flow of the game interaction that playing the animation, Wait time for the animation to go back and then loading the scene.

Flow of the game

FINALISATION

After all this the final thing was for me to create all this for each scenes according to the choices that was required for the experience and for each scenes these steps had to be repeated with the perfect timing of animation according to the length of the video. And finally with the polishing the experience was ready.