‘My Neighbour Totoro VR – Bus Stop Scene’ – Complete!

 

 

And so I finally reach the end of a long ol’ road. The ‘My Neighbour Totoro VR – Bus Stop Scene’ is finally done! Just because I know you’re asking, “So where is it??” – it will be out later this week. I need to just have a couple more QA passes to nail any final glitches.

My Neighbour Totoro, VR, Oculus Rift, Bus Stop

Taming the Totoro

The demo has taken longer than I expected, and this comes down to a couple of issues: Mission Creep – I felt I needed to add a couple of features along the way – and Bugs. The bugs on this are to do primarily with the animation, specifically getting the animation of Totoro from Maya into Unity in the first place and then how the player interacts with that animation and controls it’s progression.

I worked with my animator on the project, Keith Sizemore, to ensure everything matched the scene from the film as closely as possible. Keith has done all the animation in the demo, (I wanted to ensure the animation had an experienced hand guiding it), while I made the demo and did all the rest of the stuff. We had a nightmare with the FBX file format not playing nicely with blend shapes, bind poses and all those other rigging/paint weight aspects Maya does so well and in getting said animation over to Unity. We got there in the end, but two weeks was spent trouble shooting this particular issue. More on that later.

The other issue was with how I wanted to ensure the player has a degree of control over the progression of the demo. When the demo starts the user gets a simple on screen message to ‘find the umbrella to begin’. A simple mouse click/button press on the umbrella kicks off Totoro’s animation and we go from there until Catbus turns up. The difficulties start to creep in when you consider the umbrella has to go from closed on a bus stand to in the players hand, and then be given to Totoro, open, for him then to interact with. It requires scripts to target the umbrella, highlight it, change parenting, all within the Mecanim system of Unity. However across the demo problems crept in. Odd scripting complications I had not considered, and conflicts that are not obvious to fix. Sometime code would work in some situations, yet not in situations superficially identical. Infuriating.

My ambition had been to have the demo loop completely. The umbrella would arrive back at the bus stop (via means I will not mention here), and then you would be able to start all over, without actually reloading the level. Yet it would not play ball and I spent two full days fighting with it. On the second loop it would introduce errors to animation I could not trace, parenting errors, or even crashes (something to do with dis-activating objects I think). After screaming at Monodevelop for a couple of days I gave up. The demo reaches the umbrella as it’s placed back on the stand, but will reload if you interact with it – basically the same as I had planned, but with a fade between the two and puts you back where you started – no big problem, but perhaps I need to simplify my processes.

The Characters

Totoro and Catbus obviously came first and if nothing else worked on the demo they had to! I needed to ensure they worked just like the scene itself and looked as close as possible. However as I progressed and also rewatched the scene it became clear there was space to add some more in. There’s a little toad in the scene as well, just a couple of shots really, yet it would have felt wrong not to have him, if only to croak when you interacted with it.

My Neighbour Totoro, VR, Oculus Rift, Catbus

Catbus

The girls however were a different thing altogether and it became clear that adding the girls would not work with this demo.

Scenes from films like this do not translate directly to an interactive experience where the player controls it. Adding the entire scene was out of the question – who would the player play (the older girl), and then what would Mai be doing during the game? In the film she runs about doing stuff and then gets a piggyback from her sister, so do I just have her on your shoulder the whole time? Is there any point? Would that get annoying to have a strange face right in your own vision, possibly clipping into the camera? Do I want to obscure the vision of the trees above or Totoro higher than you by adding an umbrella over you? Too many problems with too few benefits. In the end I decided the person in the scene is YOU. You get to experience the scene as the girls do. I’d love to figure out Avatars properly and perhaps in the next demo it will be a focus. But for this one? Too much trouble when there are more important things to cover!

There are a couple of other characters in there for people to find, so keep an eye out :D

Interaction.

I didn’t just want the scene to just run and you watch it. If it was just going to be a passive experience you might as well watch the film and it would miss the opportunity for the player to affect the scene. In that spirit I chose to give two moments where the player gains control over the progression of the scene – which I think works well and gives players the chance to walk around and take everything in at their leisure.

My Neighbour Totoro, VR, Oculus Rift

Totoro in Rift Vision

Going forward into the next demo I want to expand on interaction – to actually look at it changing the progression of the game, rather than a cursory element, while also expanding what I do with characters. Location remains massively important to the experience of VR, so I want to ensure in the next one that the location is just as much the star as the characters.

Modelling

The location, as with other Ghibli films, is not a clear layout. We don’t see the scene from a couple of angles, so it’s hard to figure out precisely where it all goes, however what was clear was the long road stretching off from the bus stop and the corner with the Shrine.

My Neighbour Totoro, VR, Oculus Rift, Shinto Shrine

The Shrine

The shrine is a wonderfully emotive place and I really love the detail here. The bus stop is very simple in theory, but this merely made it all the more important to get right!

The modelling of the scene was done in 3DS, and the texturing (as with Spirited Away) done by hard painting in Photoshop, over pre-baked Ambient Occlusion maps from 3DS (UV unwrapping is also a time consuming yet important part of the process). The entire style takes time, but the results look awesome and natural. Much back and forth is required to ensure it’s looking right, but it’s a satisfying part of the process when it’s done.

The characters have similar challenges. From the front Totoro looks ‘flat’. But from the side, as seen in very few shots, he’s got quite a profile, with a short snout and a tummy that’s not quite completely round – he wants to look correct from each angle, yet turning 2D to 3D will always mean compromise.

Geometry complexity is important to get right – you want to keep it ‘light’ in poly count, but also ensure there is enough to animate with. In retrospect Totoro could have been higher poly – he’s only 6,594, which of course is great for performance, but it could have been higher. Catbus meanwhile is 10,995 polys, which is a healthier amount and the result is perhaps smoother. Rigging however, due to the nature of the animated characters was far more challenging and Keith did an amazing job with his technical Maya magic to bring them to life.

Animation

The Animation of the characters was going to be immensely important for this demo. Totoro and Catbus had to look exactly as they do, or as close as possible. Indeed there are technical limitations, not just poly count, that limit what can be animated and how.

My Neighbour Totoro, VR, Oculus Rift

I’ll let Keith Sizemore, animator on the demo explain in his own works:

For animation, translating from a traditional animated film to 3D was a task in itself.  There are many moments in the film, especially the umbrella section, where Totoro’s arms become much longer.  It was especially apparent when Totoro wasn’t able to hold the umbrella in front of his body.  I had originally addressed this with stretchy IK arms in the rig, but I learned quickly that Maya’s scale compensate for joints is not recognized by Unity and that forced me to throw that out leaving me with no other choice than to perform multiple animation tricks to try and make it work as best as possible.  The facial animation on Totoro was achieved with blend shapes along with joints driving the eyelids and lips.  This allowed for us to achieve the extreme poses on Totoro’s mouth.  Once the rig was complete we ran into an issue where the rig was suddenly too complicated to send to Unity and required an alternative method through the form of a “puppet rig”.  This is where Brave Rabbit’s actor tool came in handy (http://www.braverabbit.de/v2/).  This allowed me to have a complicated animation rig in Maya and a more simple actor rig made up of joints, Totoro’s mesh, and his blend shapes.  The actor joints were parented to their respective anim rig joints making it simple as baking the animation to the actor joints and the blend shapes then exporting the actor rig and it’s animation to Unity.  We then ran into an issue with FBX where our blend shape animation wasn’t coming through to Unity.  After scouring the internet we found that blend shapes only seemed to work using the 2010 Maya Binary file format.  Once we got all of these problems solved, the rest of the project ran smoothly.

Keith has done an awesome job and I can’t imagine I would have been able to achieve the level I did without him.

But it is now done and I am so happy with result. It was important to feel I’ve done ‘My Neighbour Totoro’, justice – and maybe I have and maybe I haven’t, but I’ve made it to the best of my abilities (and learned/developed some new ones along the way), and I can at least feel good about that.

So… what’s next?

‘Decay Theory’ – a Sci-fi adventure ‘epic’ – is the first full VR game I will be moving on to making, but that’s still being planned/written/designed/worked out, so nothing to show there yet. Scope of the location itself is being expanded, with greater variety of encounters and NPCs allowing for a wider range of experience. I see it as an awesome setting to explore Sci-Fi ideas, so want to ensure people can get the most from it.

I also want to feel I’m coming at it with the core skills/resources in place and I have yet to prove to myself I am capable of everything I want to do. So with that in mind I am going to create a final demo based on an existing film/anime.

To that end I want to ensure what I do next pushes on the areas I have yet to explore fully. Dialogue would be ideal – more exploration, and prototyping certain gameplay styles I want to use in Decay Theory. Location is also important and needs to work with VR well. Finally, the art style wants to fit – Ghibli films locations tend towards the style I want to look to, although I will probably move away from cell shaded for the characters in Decay Theory itself. With that in mind, my final demo before Decay Theory will be:

Cowboy Bebop!

Bebop Interior

The ship and it’s crew!

Man I love this show! It’s such an awesome world and art style and the characters are excellent – and as a source it is ideal as my next demo.

I think people would love exploring the ship, seeing all the locations, interacting with stuff (don’t open the fridge!) – and then having the opportunity to chat with the crew, with a dialogue system implemented to facilitate it. I have a few ideas for scope, however it will all take place on the ship, which I hope to bring in it’s entirety of on-screen location space.

But more than that it also works as a stepping stone to Decay Theory – which will will be Sci-fi set, so a speceship setting is a natural bridge. I also the character interaction feels right. I have some plans I hope to test out, but for now that’s where I am with it :)

Here’s amore images, just for fun!

Bebop InteriorBebop Interior

However I did have a couple of other options:

‘Princess Mononoke’ – Forest of the spirit

Princess Mononoke

Princess Mononoke

This would be a wonderful location (although perhaps similar to Totoro). You would have the opportunity to interact with Mononoke and perhaps the Wolves themselves, and would also get to see the Forest Spirit. Yet I wonder, as lovely as it would be, is it enough? Is it different from Tototo enough? I wonder, yet it would still be awesome.

‘Spirited Away’ – Train scene

Spirited Away Train Scene

Spirited Away Train Scene

A ride and a spectacle, with some great details. It’s popular and indeed would be great to experience, yet I question what opportunities there are to do more than sit and look. I’m not sure that’s enough for me to get my teeth into.

Howls Moving Castle was actually going to be my choice at first. I had planned to create the Castle with it’s entrance in the mountains/hills. You enter to the main ‘kitchen’/Calcifer’s room, where the Witch of the Waste is sitting smoking and can speak to her. Markl might then enter and you could then interact with him. From there you could go upstairs to the bathroom and Howls bedroom (which would have been quite the challenge) and onto the balcony. If that were not enough you would also be able to change the door ‘exit’ wheel and step out into those other locations. This would have covered all the ideas I wanted to cover, but sadly someone else is making up their own Howls VR demo and asked that I not make it as well – which is absolutely fine, and it’s great he got in contact to ask, something I totally respect. Indeed it will be awesome to enjoy an experience I had looked forward to making without having to actually make it!

So there we have it. The ‘My Neighbour Totoro VR – Bus Stop Scene’ will be with you later this week, so please be sure to check in here later, on my Twitter, or keep an eye on /r/oculus. Donations are also welcome and will help to fund the next demo :) Go here for that:

DonatePaypal

Donate Bitcoin

Let me know if there’s anything you’d like to see in the Cowboy Bebop demo!

120135

About these ads

37 thoughts on “‘My Neighbour Totoro VR – Bus Stop Scene’ – Complete!

  1. WOW that’s a lot of unexpected roadblocks along the way. Holy crap. I can imagine all the frustration. Congrats on finishing it, the fact that you pushed through to the end is an accomplishment in itself.
    I would have loved to see the Mononoke scene but I could see it being difficult to recreate the profound scale of the space while ensuring a smooth frame rate. Good luck with the next project!

  2. woooooow man..u did alot of great job..0.0
    The Characters how they look like..the Modelling are fantastic..and the Animation seems perfect..
    i never expected something like that..u realy did very well..i hardly waiting to the demo..and also the other projects u would do..u on the right road buddy ..good luck for u :)

  3. Pingback: Nick Pittom's Final VR Adaption Based on Cowboy Bebop | VR Focus

  4. I absolutely love Cowboy Bebop. Such a big Universe and so many awesome Characters.
    If I were to make a Cowboy Bebop Demo I would probably do something like this:

    Start as Jet in the Bebop on the Bridge drifting in Space in front of a Planet. Maybe Mars.
    Then have the player walk over to the iconic round windows to see Spike fly in and dock the Swordfish II. You could use Audio from the show as radio overlay.
    Once Spike is in the Hanger then you can get the player to walk over to the console and engage the Auto Pilot and fly through a Star Gate. When the Bebop is zooming through the Gate you can either leave the Demo in a continuous flight through or end up at Earth…. The possibilities are endless ;-)

    Pros:
    – You don’t need to program any character animations
    – You can program as much or a little of the Bebop itself (keep the doors shut for areas that aren’t there.
    – You can use lots of audio queues from the Show. maybe even some music

    Extras
    – Have Faye lounge on the couche with Ein.
    – Have Edd sit on the floor and hack away at the yellow Computer in VR.
    – Allow Jet to walk back to the Kitchen and cook some classic Bell Pepper and Beef ;-)
    – Allow the player to turn of the gravity generator
    – Allow the player to walk into the Hanger and see the ships
    – And much more!
    – Can you tell that I love Cowboy Bebop yet?

    Wow, getting carried away there. and that is all only on the Bebop itself….
    Let me know what you think ;-)

    • Hey, thanks for putting those thoughts down :) It’s certainly in line with the sorts of things I’d like to do and I like the idea of the Star Gate. Not sure about the gravity thing – might be a bit vomity in VR lol.

  5. Pingback: My Neighbour Totoro VR coming soon, here’s how it was made… | The Rift Arcade

  6. Hi Nick
    It would be awesome to experience the full movie in one of Miyazaki’s worlds or Cowboy Bebop! Re-playing the movie/series as the main character.
    There would obviously be some licensing issues to make that happen .. but I would spend my life-savings on that. :-P
    Maybe you can negotiate something with Studio Ghibli or even get their assistance. ^_^

    Keep up the good work!

  7. Pingback: Oculus Rift - My Neighbor Totoro: Bus Stop (1440p) - unofficial Oculus Rift Video Blog | daily updates

  8. Pingback: My Neighbor Totoro Hands-On & Video | The Rift Arcade

  9. Wow great work, it really feels like you’ve managed to capture the spirit of this scene.
    I’d like to vote in favor of the train scene from Spirited Away! The train trip to Zeniba’s house is one of my favorite parts in all Miyasaki’s films. No need for complicated interactions, just wait for the train, hop in and enjoy the view.

      • I’d put another vote on Train scene. :) As for more interactions, you could start the scene from when Chihiro climbs out from the building/castle and leave for the train. That would add some ‘actions’ from the player’s point of view and also to start the scene off in a frantic note, but gradually ease into the stillness of the famed train scene.

        The scene was said to be the ‘pinnacle of Studio Ghibli’s animation’ ;)

  10. I’m thoroughly impressed by your work on both this and the Spirited Away demo. One question though: Tou mention 3DSMax and Maya as your 3D tools of choice – have you looked into Blender? After a minor learning curve I think you’ll find it amazingly powerful. I switched from 3DS and Maya to it and have not looked back.

    • I’ve not had the need to be honest, although I understand it’s a great program. I guess if I discovered something I needed to do, that 3DS couldn’t and Blender could I’d look to switch.

  11. I’m introduced to your work via Martyn’s Youtube Channel. I personally would love to see more from the Studio Ghibli scenes.

    In regards to a scene from Princess Mononoke, if the forest scene might look too similar to your work from Totoro, how about the furnace scene in Iron Town? With the songs of the worker singing in the background, the bellow rising up and down, and the sparks and smoke flying out from the furnace flame with every stroke of the bellow.

    • Perhaps – it was an idea to have the city, but I just couldn’t feel excited about it. The amount of work required would outweigh what I could gain from it I think, and perhaps not enough people would recognise it. Something with Mononoke and the wolves felt more in the right direction perhaps, but still not right. But it still remains a favourite film of mine :)

  12. I just saw this on the Martyn’s youtube and I was just geeking out the whole time.
    I grew up with this film in my home and I’ve always love fantasy stories and mythology.
    I would looooove if you could do one for Naruto where you explore the inside of Naruto and Kurama’s cage or where you explore one piece’s thousand sunny!
    Perhaps I’m asking too much, heh heh~
    But otherwise, keep up with the amazing work and good luck to you on your own personal future projects! :3

  13. Pingback: Download 'My Neighbour Totoro VR – Bus Stop Scene' Oculus Rift (video)

  14. Pingback: If Anything Will Sell Me On The Oculus Rift, It’s Virtual Totoro | The Tech Showdown!

  15. Pingback: If Anything Will Sell Me On The Oculus Rift, It's Virtual Totoro - Dustyhawk :: Broken Mirror

  16. Pingback: If Anything Will Sell Me On The Oculus Rift, It's Virtual Totoro | Kotaku Australia

  17. Pingback: Встреться с Тоторо и Автобусом котом лицом к лицу | VR Box

  18. Pingback: “My Neighbor Totoro” Gets A Wondrous Oculus Rift Treatment [Video]

  19. I’m an avid gamer, so naturally I’m excited for games of all sizes and genres to be blasted into my eyes via the Oculus. However, I must say that these kinds of exploratory experiences are probably the ones I’m most excited for – the environments of Miyazaki’s films in particular all feel like old friends and memories, and to be able to step inside them for a moment evokes the most soul-warming sensation; it feels like coming home. You and your team absolutely have done them justice.

    (Oh, and I absolutely can not wait to board the Bebop, by the way! See you, space cowboys.)

  20. Pingback: Propeller Industries » My Neighbor Totoro's Iconic Catbus Scene Gets The Oculus Rift Treatement

  21. Looks amazing, can’t wait to see it in the e-flesh when I get my DK2 next month! (Assuming it arrives on time…).

    Princess Mononoke would be awesome, but as you say would be difficult to keep the scene different from Totoro.

    Will be too complex, but an interesting scene could the chase scene at the beginng where Ashitaka fights the demon while riding Yakul. Could be done from Ashitaka’s perspective. Alternatively the final scenes where Ashitaka and San try to return the head to the Nightwalker. But again the scale and complexity of these scenes would be too much.

    Only seen the Cowboy Bebop movie so don;t know the series that well but looking forward to it! :)

  22. Just want to say its really amazing how you captured the atmosphere not only of Totoro but of Spirited Away before this- it is simply stunning. As a long-time Studio Ghibli fan it is like I’m experiencing a new element to the story- it does not feel out of place/forced/anything. I wonder what Miyazaki would say on these! (Probably that technology has gone too far or something like that.)
    Thank you for making these, I look forward to seeing both original and fanworks from you in the future!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s