And so I finally reach the end of a long ol’ road. The ‘My Neighbour Totoro VR – Bus Stop Scene’ is finally done! Just because I know you’re asking, “So where is it??” – it will be out later this week. I need to just have a couple more QA passes to nail any final glitches.
Taming the Totoro
The demo has taken longer than I expected, and this comes down to a couple of issues: Mission Creep – I felt I needed to add a couple of features along the way – and Bugs. The bugs on this are to do primarily with the animation, specifically getting the animation of Totoro from Maya into Unity in the first place and then how the player interacts with that animation and controls it’s progression.
I worked with my animator on the project, Keith Sizemore, to ensure everything matched the scene from the film as closely as possible. Keith has done all the animation in the demo, (I wanted to ensure the animation had an experienced hand guiding it), while I made the demo and did all the rest of the stuff. We had a nightmare with the FBX file format not playing nicely with blend shapes, bind poses and all those other rigging/paint weight aspects Maya does so well and in getting said animation over to Unity. We got there in the end, but two weeks was spent trouble shooting this particular issue. More on that later.
The other issue was with how I wanted to ensure the player has a degree of control over the progression of the demo. When the demo starts the user gets a simple on screen message to ‘find the umbrella to begin’. A simple mouse click/button press on the umbrella kicks off Totoro’s animation and we go from there until Catbus turns up. The difficulties start to creep in when you consider the umbrella has to go from closed on a bus stand to in the players hand, and then be given to Totoro, open, for him then to interact with. It requires scripts to target the umbrella, highlight it, change parenting, all within the Mecanim system of Unity. However across the demo problems crept in. Odd scripting complications I had not considered, and conflicts that are not obvious to fix. Sometime code would work in some situations, yet not in situations superficially identical. Infuriating.
My ambition had been to have the demo loop completely. The umbrella would arrive back at the bus stop (via means I will not mention here), and then you would be able to start all over, without actually reloading the level. Yet it would not play ball and I spent two full days fighting with it. On the second loop it would introduce errors to animation I could not trace, parenting errors, or even crashes (something to do with dis-activating objects I think). After screaming at Monodevelop for a couple of days I gave up. The demo reaches the umbrella as it’s placed back on the stand, but will reload if you interact with it – basically the same as I had planned, but with a fade between the two and puts you back where you started – no big problem, but perhaps I need to simplify my processes.
Totoro and Catbus obviously came first and if nothing else worked on the demo they had to! I needed to ensure they worked just like the scene itself and looked as close as possible. However as I progressed and also rewatched the scene it became clear there was space to add some more in. There’s a little toad in the scene as well, just a couple of shots really, yet it would have felt wrong not to have him, if only to croak when you interacted with it.
The girls however were a different thing altogether and it became clear that adding the girls would not work with this demo.
Scenes from films like this do not translate directly to an interactive experience where the player controls it. Adding the entire scene was out of the question – who would the player play (the older girl), and then what would Mai be doing during the game? In the film she runs about doing stuff and then gets a piggyback from her sister, so do I just have her on your shoulder the whole time? Is there any point? Would that get annoying to have a strange face right in your own vision, possibly clipping into the camera? Do I want to obscure the vision of the trees above or Totoro higher than you by adding an umbrella over you? Too many problems with too few benefits. In the end I decided the person in the scene is YOU. You get to experience the scene as the girls do. I’d love to figure out Avatars properly and perhaps in the next demo it will be a focus. But for this one? Too much trouble when there are more important things to cover!
There are a couple of other characters in there for people to find, so keep an eye out :D
I didn’t just want the scene to just run and you watch it. If it was just going to be a passive experience you might as well watch the film and it would miss the opportunity for the player to affect the scene. In that spirit I chose to give two moments where the player gains control over the progression of the scene – which I think works well and gives players the chance to walk around and take everything in at their leisure.
Going forward into the next demo I want to expand on interaction – to actually look at it changing the progression of the game, rather than a cursory element, while also expanding what I do with characters. Location remains massively important to the experience of VR, so I want to ensure in the next one that the location is just as much the star as the characters.
The location, as with other Ghibli films, is not a clear layout. We don’t see the scene from a couple of angles, so it’s hard to figure out precisely where it all goes, however what was clear was the long road stretching off from the bus stop and the corner with the Shrine.
The shrine is a wonderfully emotive place and I really love the detail here. The bus stop is very simple in theory, but this merely made it all the more important to get right!
The modelling of the scene was done in 3DS, and the texturing (as with Spirited Away) done by hard painting in Photoshop, over pre-baked Ambient Occlusion maps from 3DS (UV unwrapping is also a time consuming yet important part of the process). The entire style takes time, but the results look awesome and natural. Much back and forth is required to ensure it’s looking right, but it’s a satisfying part of the process when it’s done.
The characters have similar challenges. From the front Totoro looks ‘flat’. But from the side, as seen in very few shots, he’s got quite a profile, with a short snout and a tummy that’s not quite completely round – he wants to look correct from each angle, yet turning 2D to 3D will always mean compromise.
Geometry complexity is important to get right – you want to keep it ‘light’ in poly count, but also ensure there is enough to animate with. In retrospect Totoro could have been higher poly – he’s only 6,594, which of course is great for performance, but it could have been higher. Catbus meanwhile is 10,995 polys, which is a healthier amount and the result is perhaps smoother. Rigging however, due to the nature of the animated characters was far more challenging and Keith did an amazing job with his technical Maya magic to bring them to life.
The Animation of the characters was going to be immensely important for this demo. Totoro and Catbus had to look exactly as they do, or as close as possible. Indeed there are technical limitations, not just poly count, that limit what can be animated and how.
I’ll let Keith Sizemore, animator on the demo explain in his own works:
For animation, translating from a traditional animated film to 3D was a task in itself. There are many moments in the film, especially the umbrella section, where Totoro’s arms become much longer. It was especially apparent when Totoro wasn’t able to hold the umbrella in front of his body. I had originally addressed this with stretchy IK arms in the rig, but I learned quickly that Maya’s scale compensate for joints is not recognized by Unity and that forced me to throw that out leaving me with no other choice than to perform multiple animation tricks to try and make it work as best as possible. The facial animation on Totoro was achieved with blend shapes along with joints driving the eyelids and lips. This allowed for us to achieve the extreme poses on Totoro’s mouth. Once the rig was complete we ran into an issue where the rig was suddenly too complicated to send to Unity and required an alternative method through the form of a “puppet rig”. This is where Brave Rabbit’s actor tool came in handy (http://www.braverabbit.de/v2/). This allowed me to have a complicated animation rig in Maya and a more simple actor rig made up of joints, Totoro’s mesh, and his blend shapes. The actor joints were parented to their respective anim rig joints making it simple as baking the animation to the actor joints and the blend shapes then exporting the actor rig and it’s animation to Unity. We then ran into an issue with FBX where our blend shape animation wasn’t coming through to Unity. After scouring the internet we found that blend shapes only seemed to work using the 2010 Maya Binary file format. Once we got all of these problems solved, the rest of the project ran smoothly.
Keith has done an awesome job and I can’t imagine I would have been able to achieve the level I did without him.
But it is now done and I am so happy with result. It was important to feel I’ve done ‘My Neighbour Totoro’, justice – and maybe I have and maybe I haven’t, but I’ve made it to the best of my abilities (and learned/developed some new ones along the way), and I can at least feel good about that.
So… what’s next?
‘Decay Theory’ – a Sci-fi adventure ‘epic’ – is the first full VR game I will be moving on to making, but that’s still being planned/written/designed/worked out, so nothing to show there yet. Scope of the location itself is being expanded, with greater variety of encounters and NPCs allowing for a wider range of experience. I see it as an awesome setting to explore Sci-Fi ideas, so want to ensure people can get the most from it.
I also want to feel I’m coming at it with the core skills/resources in place and I have yet to prove to myself I am capable of everything I want to do. So with that in mind I am going to create a final demo based on an existing film/anime.
To that end I want to ensure what I do next pushes on the areas I have yet to explore fully. Dialogue would be ideal – more exploration, and prototyping certain gameplay styles I want to use in Decay Theory. Location is also important and needs to work with VR well. Finally, the art style wants to fit – Ghibli films locations tend towards the style I want to look to, although I will probably move away from cell shaded for the characters in Decay Theory itself. With that in mind, my final demo before Decay Theory will be:
Man I love this show! It’s such an awesome world and art style and the characters are excellent – and as a source it is ideal as my next demo.
I think people would love exploring the ship, seeing all the locations, interacting with stuff (don’t open the fridge!) – and then having the opportunity to chat with the crew, with a dialogue system implemented to facilitate it. I have a few ideas for scope, however it will all take place on the ship, which I hope to bring in it’s entirety of on-screen location space.
But more than that it also works as a stepping stone to Decay Theory – which will will be Sci-fi set, so a speceship setting is a natural bridge. I also the character interaction feels right. I have some plans I hope to test out, but for now that’s where I am with it :)
Here’s amore images, just for fun!
However I did have a couple of other options:
‘Princess Mononoke’ – Forest of the spirit
This would be a wonderful location (although perhaps similar to Totoro). You would have the opportunity to interact with Mononoke and perhaps the Wolves themselves, and would also get to see the Forest Spirit. Yet I wonder, as lovely as it would be, is it enough? Is it different from Tototo enough? I wonder, yet it would still be awesome.
‘Spirited Away’ – Train scene
A ride and a spectacle, with some great details. It’s popular and indeed would be great to experience, yet I question what opportunities there are to do more than sit and look. I’m not sure that’s enough for me to get my teeth into.
Howls Moving Castle was actually going to be my choice at first. I had planned to create the Castle with it’s entrance in the mountains/hills. You enter to the main ‘kitchen’/Calcifer’s room, where the Witch of the Waste is sitting smoking and can speak to her. Markl might then enter and you could then interact with him. From there you could go upstairs to the bathroom and Howls bedroom (which would have been quite the challenge) and onto the balcony. If that were not enough you would also be able to change the door ‘exit’ wheel and step out into those other locations. This would have covered all the ideas I wanted to cover, but sadly someone else is making up their own Howls VR demo and asked that I not make it as well – which is absolutely fine, and it’s great he got in contact to ask, something I totally respect. Indeed it will be awesome to enjoy an experience I had looked forward to making without having to actually make it!
So there we have it. The ‘My Neighbour Totoro VR – Bus Stop Scene’ will be with you later this week, so please be sure to check in here later, on my Twitter, or keep an eye on /r/oculus. Donations are also welcome and will help to fund the next demo :) Go here for that:
Let me know if there’s anything you’d like to see in the Cowboy Bebop demo!