There aren’t many media and entertainment companies like Sony. The company’s PlayStation 5 console is still a market leader in consoles. It’s branching into PC games, and it just launched the PlayStation VR 2. On top of that, Sony’s The Last of Us has become a massive hit as television series on HBO.

Heck, in a couple of years, maybe we’ll be playing games in our Sony electric cars.

To Jake Zim, senior vice president of virtual reality at Sony Pictures, this is a unique point in time in the history of gaming. He started diving into VR eight years ago, and it’s nice to see some pay off with new generations of headsets and increasing sales of VR titles. Now the walls are coming down between VR, games, movies and other media. It’s like a single ecosystem.

And one of these days, all of these businesses are likely going to make up a pretty good metaverse experience. I caught up with Zim for a recent interview to talk about this at the Dice Summit in Las Vegas. This kind of theme is what we’ll be talking about at our GamesBeat Summit 2023 event on May 22-23 in Los Angeles.

Here’s an edited transcript of our interview.

GamesBeat: Can you tell me about your background? How long have you been doing this?

Jake Zim: I’m the senior vice president of virtual reality at Sony Pictures. I’ve been at Sony for 12 years. Before that I was at Fox and Lionsgate movie studios. I started in digital marketing, that intersection of new technologies and ways to market movies and get people excited about film products.

About eight years ago somebody put a VR headset on me and it was a horrible experience. It was a mobile phone. But I saw something interesting there. As someone who, at the studio, had been trying to push new technologies to do exciting things, I figured this would be an opportunity to not just start a new way of marketing movies, but also build a new business. Of course as it turns out the sister division at PlayStation was working at PSVR at the time. They shipped that the following year.

I pitched the idea of building a little content business inside the movie studio that would focus on–at that point it was really a new frontier of storytelling. How could we expand the worlds of movies, TV, and even games onto this new platform, following the technology breakthroughs that were expected to come with headsets? Leaning into immersion, presence, agency, all the unique features VR promised, although the tech at the time was still–to this day it’s still nascent in terms of achieving the ultimate goals.

In 2016 I got the job, basically. I started Sony Pictures VR, which is a label. It’s a group. It’s a business inside of Sony Pictures. The mission was to take our franchise IPs, expand those worlds into this new platform, give audiences a chance to engage with their favorite IPs like Ghostbusters, do new things, and monetize them. We’d work with PlayStation, work with Meta, work with any partner out there to distribute content and help build this business. We’ve been doing it ever since. We have a bunch of stuff coming.

GamesBeat: What were some of the titles along the way that built up to where you are now?

Zim: Early on there were two real categories. There was location-based VR. We did a couple of projects with The Void, which you probably remember. Location-based walk-around VR that you did with a backpack on. We did Ghostbusters. We did Jumanji. We did a project with Dreamscape, another location-based VR company, with a Men in Black game.

On the in-home side we did a few games early that were marketing initiatives. We did a promotional Spider-Man game that actually did really well. It’s a free Spider-Man game to promote the live-action movies. We did a couple of those. We did a project early on with D-BOX, the motion chair company. We did a Goosebumps project. These were around 2016, 2017, the tip of the iceberg. Then we moved more into better development. We did a Groundhog Day game, a sequel to the movie, with TequilaWorks. More recently we did Zombieland: Headshot Fever that we released on Quest, developed with a company called XR Games. Just yesterday that launched on PSVR2. Early on it was a mix of location-based, a little promotion, and then building a business with real transactional studio-based games.

GamesBeat: Has Sony ever described how that’s worked out financially over time?

Zim: In terms of the business? The good news is, the focus has always been on how to turn this into a business. We have a variety of partnerships with some of the key platforms out there. As the business is growing, we’re starting to see real return on it. It’s still relatively small in the grand scheme of things, but we see this as an area with tremendous upside as the market grows.

GamesBeat: It seems like there are some folks, some hitmakers, that have thrived in what’s a smaller space than everybody expected, but still a good niche in games. Schell Games has said that all of their games have crossed a million sold. They’ve persevered in the space long enough to see the quality of their games being rewarded. There aren’t as many fish in the ocean feeding with them anymore. Tommy Palm’s company, Resolution Games, has about 100 people working on these games now. It’s become a real business. It seems like a lot of people may not realize that this has turned into a business. It’s not as big as everyone originally predicted, but people are making money in the space.

Zim: Having come from the movie world, there’s an analogy within the movie business. You have big studio labels like Columbia Pictures, which does your Spider-Man movies, Ghostbusters movies. Then there are the smaller labels like Screen Gems and Tri-Star, which have a model where they’re still making the same kind of content, but with a different focus and a different budget.

In some ways, the analogy is you have the big triple-A video games – console games, PC games – as those big labels. High risk, high reward, relatively mature ecosystem. They know who their consumer is. Big studios focus on that. But then this smaller niche of VR games is like those smaller horror movie-type models. You need real expertise. You have to know your audiences. You need developers who, to your point, have persevered over time to understand how to unlock the technology. You have Resolution. You have Vertigo. You have nDreams. You have XR. You have Skydance. Some of these studios that have done very well from a business standpoint because they made it through those very challenging years and released real hit products along the way. They’ve benefited from the curation and the overall pool being a little smaller, and they’ve risen to the top.

GamesBeat: Who do you rely upon for your game developers? Are they in-house at Sony Pictures?

Zim: No, this is what DICE is all about for me. We bring financing and distribution and marketing, but we outsource all of the development. It’s best-in-class VR developers. We have a big Ghostbusters game being developed by nDreams right now. We’ve done projects with XR Games. We have three or four big titles we’ll announce very soon, all with best-in-class VR developers. But it’s the kinds of teams that have proven what they can do in VR with their focus. They’ve shipped games on multiple platforms over time. We work closely with our partners at PlayStation and Meta to vet and ID which studios would make the best fit with our projects. We try to build a partnership, so it’s not just a work-for-hire or one-off deal. The developers feel invested in the project and they’re passionate about the IP.

GamesBeat: How much interaction do you have with some of the wider trends in the industry? The marriage of games and Hollywood seems to be working great with things like The Last of Us. We have the new PlayStation VR. That’s in the game realm, but you can bring movie-related IP into that space. Some of this is starting to feel like a lot of walls are coming down.

Zim: I think we’re in a unique moment in time, when you think about the ability of games to be told as stories on linear platforms successfully, with real art and commerce tied to. Uncharted is a great example. The Last of Us is a great example. Those are Sony IPs, and there are others as well out there. Sony is a unique company in this moment, too, because in theory–the thesis was always that we have all these assets at our fingertips. We have great gaming IP. We have great creative. We have the ability to make movies and TV and market them. We also have electronics. The ecosystem is all set up. But to your point, it took something for the walls to all come down. Some level of mutual respect between the creators on one side, on one platform, the interactive platform, to work closely and collaborate with the creators and storytellers on a linear platform, the writers and showrunners.

What I think is so exciting about the moment that we’re living in right now is that maybe for the first time – someone might debate me on this at some point – there’s a real path to see how these worlds–The Last of Us is a world. Uncharted is a world. Ghostbusters is a world. They can be expanded on different platforms while still staying true to the innate core DNA of what made the project exciting at the beginning. Neil Druckmann hasn’t lost his story by translating The Last of Us to HBO. He gets to sit in that room and make decisions. Even when the TV show diverges from the game story, it’s still part of the canon and the ecosystem. For me it’s the opposite path. How do I take a linear product, a Ghostbusters or a TV show, or even an original story, and expand it into this very immersive, deeply interactive world of VR?

GamesBeat: Looking at where the state of progress in this ecosystem is at right now, where do you expect it to be in a couple of years?

Zim: I always try to take the long view on technology and be surprised when things move faster than expected. The way I look at it, you have a variety of different cultures that are all playing in this space. Meta, even Apple, anybody who has an engineering culture is going to work differently and iterate differently on their hardware and software than a company like us, like Sony, which has a real design focus and a user focus.

The generational leaps are going to be longer between a PSVR and a PSVR2, but they’re going to be more impactful. What you see with the PSVR2 coming out, with the eye tracking and the head rumble and the foveated rendering and all the different technologies, they’re unlocking game design opportunities for smart, creative developers. To me that’s exciting. When you look at what’s happening right now with PSVR2 versus Quest, Quest 2, and going forward, you have this leapfrogging advancement in the technology. That’s really impressive and good for the ecosystem. We need more players in there.

But I think the long game here that everyone needs to recognize is that it takes a long time to get this kind of hardware, which is really the biggest barrier. Getting people to put something on their head, it takes a long time to get that to be comfortable, low heat, aesthetically pleasing, long battery life, at a price point people can afford, and have the capabilities to play really fun games that are social and networked and have a level of mobility. Where are things today versus where I expect them to be? I’m probably an optimist in that I think that things are just where they need to be. I like to be surprised when things move faster than that.

GamesBeat: I’ve been trying out PSVR2. I went into Call of the Mountain. That was a very interesting experience. The visual experience is where it needs to be now, I think. The headset experience is still tough. I wear glasses, and when I turn my head like this, if something bumps my glasses all of a sudden it’s out of focus again. There’s a clunkiness to it that you put up with in order to get the experience that you want from a game.

Zim: That’s a challenge, and I think it’s going to be a challenge for as long as we’re putting something on our face. Not everyone has the same face and head shape to begin with. IPD, all the challenges of people not wanting to mess up their hair, whatever it is at any level, let alone the actual experience in the headset itself. But again, these are things that, over time, are going to get better and better. At some point there is a barrier where people say they’re just not going to do this, in the same way that some people aren’t going to get on a roller coaster or bungee-jump off a bridge. Some things are just a line in the sand for some people. That’s okay.

GamesBeat: With the hardware platforms, I think some of the things that were lacking seven years ago, they seem to have gotten there. Maybe the overall platform is still going to change to a point where the comfort comes in and everything else is in place, whether it’s haptics or the sense of touch. All the things you expect from that holodeck experience. How much do you think about that, whether that’s going to be deliverable at some point through something like the metaverse?

Zim: Eventually that’s the aspiration. Despite the metaverse as a term being a little difficult to define as a business, as a concept I think we can all envision what that feels like and looks like. All of the requirements to get there – hardware, software, compute, thermal, cost – there are a lot of challenges along the way. But I think there’s a shared vision for what everyone wants to have with visual computing. That’s ultimately what we’re talking about here.

To me what’s exciting is the idea that there’s an expertise being developed right now by a lot of these game studios with engine-based interactivity – game engines, real time interactivity – that’s living inside of visual computing hardware. The hardware will get better, faster, cheaper. The form factor will get more comfortable. With more expertise and more compute power, we’re moving toward something that’s going to be the kind of thing that we’ll wear all day, or for a significant amount of time. That’s exciting.

The use cases aren’t that bizarre. The idea that I’m wearing these glasses right now and I can actually just zoom and read the name tag of that gentleman over there, or the street sign over there, that’s a compelling argument for me to wear these things all the time. Right now these are essentially superpowers. I can actually see because they’re prescription glasses. But the idea of enhancing my vision even further seems like a pretty clear use case. Or heat vision so I can see where the cat is hiding in the couch, or whatever it is. All of that is the utility that one would get from being able to wear something on their face.

Of course you add gaming. Of course you add social interaction. Of course you add collaboration tools, the enterprise tools. We have a vision of where this can all go. Moving along, devoting the capital, dealing with the consumer pushback challenges and marketing, that seems like it’s par for the course. It’s nothing we should be afraid of. We just have to keep chipping away at it.

GamesBeat: Is there a way to describe the experience people get in the PSVR2 generation, the things that are possible now?

Zim: The most important way of describing a great VR experience is the idea of feeling. You have to feel something, whether it’s fear, whether it’s vertigo, whether it’s adrenaline, whether it’s an intense shooter experience. From the PSVR2 games that I’ve played, including our own in early builds–I haven’t played Gran Turismo, which I’m really excited to do tomorrow. But it’s the feeling of high visual fidelity, which always adds to immersion and is impressive, and the idea that I’m there. I have presence. Whatever my emotional reaction is to whatever I’m doing, whether it’s a stealth game like Horizon where I’m sneaking up on something, or climbing, or having vertigo because I’m looking down the side of a mountain, I’m feeling something. That’s a constant across all good VR experiences.

When you mix really good one-to-one physical tracking, motion tracking, head and hands especially, eye tracking–to me that’s one of the most exciting unlocks of what PSVR2 is doing. Then you have really cool game design options. This horror game, Switchback VR, there was some great press about this the other day. The idea is that you walk into a room and it says “Don’t blink.” If you blink, the eye tracking is going to activate the zombie on the screen and move them closer. Kind of like Superhot, but with a horror angle. That’s genius. I know other developers are working on really cool eye tracking-based game design with foveated rendering. Knowing what button on the wall you’re looking at, so when you reach up and your hand is tracked, it’s going to push the button that your eyes were looking at. That’s unique to VR, that kind of technology, and therefore it allows games to be unique to VR, which is what the platform needs.

GamesBeat: I don’t know what you call the design challenge of trying to design something that’s only possible in VR, or that uniquely takes advantage of VR. One experience I had was with Tetris. I played Tetris with PSVR, and I realized very quickly that I could play it faster with a normal controller. It doesn’t give you a better experience than you would on a 2D screen. How do we take advantage of the 3D experience? Similarly, people have talked about putting a mall in the metaverse that mimics a physical mall. It could be a million square feet of mall, but even so, why?

Zim: The thing we don’t want to do as an industry is try to replicate and add more barriers to things that can be done efficiently, effectively, without putting something on your head. Tetris is a great game. It’s a mobile phone game. It’s been a console game. But it’s nothing where I need to be immersed in the world of Tetris for it to be a better gaming experience, at least as far as I understand it. There’s a reason why Amazon has a rule about the fewer clicks to the purchase, the better. They want to make it quick and easy for people to buy products. Putting a mall in a virtual metaverse to try to encourage e-commerce may seem like a fanciful idea, but in reality I think people will ultimately reject it. It just adds layers of friction.

What we always try to do is, how do we find a viable experience in VR? We start with our IP. We have a 20-question viability analysis. Ultimately it gets to, is this worth asking someone to put something on their head to do it? How do we provide enough that yes, it’s worth it? Is there a fantasy that we can fulfill by being in this world? Let’s take Ghostbusters. Being a Ghostbuster–we know as movie people that people dress up every Halloween as a Ghostbuster. They drive around in their Ecto-1 mockups. They sing the song. They want to bust ghosts. There’s a fantasy there you can fulfill. There’s a cosplay element to it. That’s one way to look at it. Same thing with Spider-Man. People would love to be Spider-Man. He’s the most beloved superhero in the world. People would love to swing from the skyscrapers in Manhattan.

Number one, then, can we fulfill a fantasy? Then we go down the list. Is that fantasy something we can activate with hands and head? The physicality of VR requires that. Tetris is not a hands and head game. It’s a rotational, two-dimensional, drop an item in–maybe you can make 3D Tetris and view it that way, but it’s not about what you’re doing with your hands and your head and your physicality. The more you can match physicality in gameplay with the technology that tracks it in VR, the more you can fulfill that fantasy and not break immersion. There are a lot of good examples of hit games in VR that have taken that up, from Beat Saber to Walkabout to Superhot. Some of the bestselling VR games really nail that concept.

GamesBeat: I’m curious about how fast the platform can still change. We’re entering this new generation now. It’s brought in things like eye tracking. Can we still do better with haptic feedback, what you feel in your hands, those kinds of things? It feels like there are still many steps to take with the technology.

Zim: To me that’s the most exciting aspect of being in the VR space right now. There’s so much more that we know can already be unlocked, even with today’s technology. There are cost challenges. There are production challenges. There are design challenges. But when I think about the cycle of consoles I grew up with – 2600, 5200, Nintendo, Genesis – it was a linear path to go from generation to generation. PlayStation, PS2, PS3, those were huge leaps put out there for developers to play with. Major generational leaps. In VR I feel like we’re in the PlayStation, PS2 era, waiting to break through to the next moment. As we start to do that it unlocks all these different opportunities for game design.

What’s cool, although always challenging, is the physicality of VR. That’s always going to be inherent to what the game design needs to be. Whether it’s haptics, full body tracking, proximity, voice chat, multiplayer and social on a larger scale, massive instances of people playing together in big battle royales, all of that is unlockable as we start to figure out how to make the tech cheaper and faster. That’s just the hardware generational challenge that’s coming.