Welcome to Meta Minutes, your bite-sized pieces of Metaverse! Rene Schulte kickstarts our brand new series by giving an overview of Metaverse, its types, trends, and future predictions.

Transcript

Hi, Welcome to Meta Minutes, Your bite-sized pieces of Metaverse. My name is Rene from Valorem Reply and I'm happy to kick off our brand-new video show today called Meta Minutes and similar to our sister show QuBites, bite-sized pieces of quantum computing, Meta Minutes is all about bite-sized pieces of Metaverse and our goal is to make it approachable for everyone.

Typically, I would invite expert guests here to talk about a certain specialty of the metaverse where there are experts in and typically, I would ask free questions to them, but since this is the first episode for the whole show, I figured we should rather do an introduction to the Metaverse. And today I'm actually going to do this. I'm going to provide an overview of the metaverse, an introduction to the metaverse topic in general, and I will provide the definition of metaverse, which is driven by market data and you know things you see from market analysts, but also of course it's very much a subjective thing today, right? Because they metaverse is such a dynamic market at the moment and there's so much change and, you know, things are ongoing and constantly changing. You might hear throughout the show various different definitions of the metaverse, right, from our guests. So, don't be surprised if you hear it differently a few times here, but that's the state what we have currently with the metaverse topic and trend, right? We will cover different metaverse flavors. We will talk about virtual metaverse solutions. We will talk about real-world metaverse solutions and of course also enterprise solutions. I will also give you a little bit of an overview of the history of the whole metaverse stuff. And of course, also some motivation when it comes to business numbers and predictions, what's going to happen in the market in the next couple of years, right? Of course, we're also going to talk about the economy like NFTs and the building blocks, like decentralization, virtual presence, decentralized identities, digital humans. The whole stuff will be covered here, right?

Well, if you don't know me, maybe it also makes sense to say a few words about myself. So, my name is Rene Shulte. I'm Innovation director at Valorem Reply, where I'm leading the research and innovation around emerging technologies, and I'm also additionally, the lead for the spatial computing community of practice for whole Reply group. And what we do there is, we centralized R&D for spatial computing research, like things like augmented reality, virtual reality, mixed reality and general extended reality stuff of course, but also spatial computing, depth sensing, 3D scanning, all things metaverse, haptic feedback and all what is related, and we centralized based on the research there, and, you know, provide these as outcomes to all Reply companies, but also a couple of these things will be actually published, and so you can also see some of those in the future. So, take a look, we will publish a bunch of reports as well. But anyhow, this is all about metaverse show today, but maybe it's also worth to say I'm Microsoft MVP for mixed reality as well as a Microsoft regional director. And these are community awards that Microsoft provides to independent experts that share knowledge. So I do a lot of conference talks, video shows but also quite active in social media, so feel free to connect there, on Twitter, LinkedIn and so on, always happy to connect and you will see me constantly sharing stuff there. Additionally, I'm also an advisor for the VR and AR Association and the XR bootcamp. OK, so enough for the intro in the intro. Let us talk about the metaverse and what metaverse is, right?

What is metaverse?

Some might say, “Are we in the metaverse yet”? If you look at the New York Times article, that is cool, how they describe it, right?

Crypto people say they are building it. Gamers might already be living in it. The art world is cashing in on it, and web veterans are trying to save it. But what is the metaverse, right? Good question, because that also aligns quite a bit with the current situation, right, where we already have certain solutions. But more about this later.

So, what is the metaverse?

Well, I like the definition from market research company, Gartner, where they define the metaverse as, “this collective virtual space that is created by the convergence of virtually enhanced physical and digital reality. And the Metaverse is device-independent, persistent, and providing an immersive experience. And the last sentence is especially important, like it needs to be immersive, but you also see a highlighted device, independence, and persistence, because these are super important. You need to be able to access these different solutions with various devices, right? So, it could be a VR headset, but it could also be a web or a desktop application or mobile. Of course, right. And persistence is very important, right? If you change something in one of these virtual worlds, you want to have it persistent. So, if you come back a week later and for example, you took a mic and dropped it on the table. You come back a week later, you want to see it still on the same table, right? So just like the real world behaves, we need to see that also. And of course, this requires a lot of computational power, and a lot of cloud computing is relevant here as well, right? And so, like I said, you cannot just look at the device independence, you will see that it is more about not just this kind of narrow view, which you could look at, like this year the narrow understanding, where you are just looking at the VR market and what you see often today, the Metaverse term is used in general for everything that was called XR or VR, AR, NFT and Blockchain all those that is lumped into one definition. But of course, that is just a very narrow understanding. The broader understanding is really the new Internet. It is the new immersive Internet. And I like this quote from Alan Smithson here- simply put, the metaverse is the Internet, but you can go into it or it can come out to you. So, one is virtual reality, the other one is augmented reality. And you can use it with different devices, doesn’t have to be VR or AR, but I think. this captures very well. It's basically the new immersive Internet, but you can go into it with video or it can come to you with AR. And we also have the seven layers of the metaverse right? And so, this is a model from Jon Radoff here, which you might have seen on his blog or somewhere else before, it is pretty famous. And in fact, I'm not going to talk too much about it today, but you see, all these different building blocks or pieces of it, or layers if you will, because we will have Jon actually as a guest here in the show in the next couple of episodes and so we will talk about this in much more detail with the true expert, with the author of this one.

What are the origins of the metaverse? Well, you can have a look at different books, for example, Snow Crash by Neal Stephenson introduced the term metaverse around 40 years ago, and then we have also Ready Player one, which made it much more famous because of the movie. But when you look at, for example, Ready Player one and the vision, they have there, It's quite dystopian, if you ask me, right? And that's definitely not a vision of the metaverse I can relate to, what you see in the movie, where the world is ruined, all the crises like climate crisis and so on ruined the whole plan. It's pretty dystopian, right? And so, what people do is they put on these headsets to escape this dystopian reality, to go into those beautiful metaverse world called the Oasis, right? And of course, that's not something I can totally relate to. Because I mean, I can totally understand that people want to escape reality at some points, but it should not be the motivation that we can ruin all planet and then just flee into Oasis or something. But anyhow, this is where you might have heard the metaverse terminology before. And it's actually already here, the whole metaverse tech is already here when you look at large online gaming platforms. Of course, you might remember second life back in the days. It was an early form but not really, because it still was lacking a couple of features, but if you look at Roblox Fortnite, Minecraft, and a few other of these large online gaming platforms and they have certain characteristics of the metaverse, but they're not. I wouldn't call them 100% metaverse solutions because they're lacking a bunch of things. But you have virtual characters, you have large open worlds and so on. And it's definitely some first iterations of the metaverse. And well, you can actually, also build custom spaces there. So, this is a custom Reply space inside of Roblox, you can see our Reply man logo there and a couple of other things. So, you can even create in these gaming rules like engaging branded experiences for your clients, right? The metaverse wasn't even invented in the book called Snow Crash. Of course, the term comes from the book snow crash, right? But the whole concept about it, and you know, large online gaming and so on was actually already done much earlier, like for example in 1985 with a game from Lucas called Habitat. And I'm going to play the official trailer here for the habitat game, which is pretty interesting. So, if you take a look, so this is 1985, it was done for the Commodore C 64 and used for the Internet connection. It used a service called Quantum Link, which became AOL later on right? So, this was one of the first internet enabled games for the Commodore C 64 and it's pretty fantastic if you look at the quick trailer here. Take a look.

So yeah, that is quite interesting. Think about it like you have these kind of metaverse solutions in 1985 and then you had, like you know lawyer and couple of other people were imagined that they would join these metaverse spaces and these online games too to have some fun with it. I was actually fortunate enough to meet one of the co-designers of Habitat. I met him in 2017 at a conference where we were both speakers, and so we had a drink and I was talking about it with him about all the amazing Lucas film stuff back in the days, which if you're old enough, like me, we're all big fans of. But anyhow, let's come back to the current state of affairs and talk about metaverse flavor, which is called virtual metaverse, which are basically these large online worlds, where you represent it as a character, as an avatar or different way, and you have to use large online worlds, right? And you're fully immersed into this. So basically you just see the virtual content. So, this is what makes up what I call the virtual metaverse. And you have this nice graphic here, which was done by Vicos dot it, it's an Italian tech site and they categorize different current virtual Network Solutions in this kind of a map of the metaverse 2022. Of course, this is going to highly change, right? This is done in H1 2022. For Q2 quarter 2 of 2022 and this is the current state what we have and if you look at the graphic here, you can see a bunch of different pieces here all over the place. For example, you can categorize them if they're using blockchain technology or not a blockchain technology and then you can also say if they're using a VR device or no VR device, or actually also using VR device and you can see a bunch of these players. These are all the decentralized ones that have some relationship to blockchain, so basically you can do decentralized asset trading and so on and then on the top you have those that don't use a blockchain and some you can use with VR as some you can also additionally use with 2D. So, you might see some of those on both. It's in fact Alt space, should not just be on the VR side, would also be on the no-VR side, because you can also use it without VR headset with a desktop app. But anyhow, so this is the current state.

What we have with different metaverse solutions. Of course this is not complete, there's much more which you can find, but these are the biggest players I would say, which are really interesting to look at, at the moment. One big player you might have heard about it is Meta, which was called Facebook before 2022 as they renamed at the end of 2021 and Facebook is working not just on the hardware like the Oculus Quest which is now called the Meta Quest VR device. A great VR device I have to say, but Facebook is also working on a bunch of virtual worlds and virtual solutions, right? So, they have these Horizon worlds, for example, where you have these large online worlds and where you can meet your friends and so on. And this works of course with VR devices, but also a PC and mobile is upcoming. And then also we have Microsoft working in various different metaverse topics and you know building blocks but also on an end user experience. You might have heard about Mesh before and this is of course also a relevant topic where Microsoft is working on. But Microsoft also has Alt Space VR, which is a large virtual metaverse solution where you can meet and have business meetings as well, but also you can run large events and in fact their Ignite event was done in this metaverse solution last year in Altspace and here you can see the Ignite keynote. A quick example, they had around 10,000 people joined as live, and they can also use a thing called holoportation, where they capture a person in real time and stream that as 3D volumetric video inside of Altspace, so you can see the other person as real time hologram, which is fantastic. So, Alt space another solution that is coming up quite nicely there.

What is the market going to look like? Well, Gartner is saying that in 2026 a quarter of the global population will spend at least one hour per day in the metaverse. So, that might be a steep prediction, but if you look at not just the narrow definition but also a broader definition of the metaverse and also include these online games and so on, that might actually be true, right? So wouldn't be surprised, really. Also, Bloomberg is saying the metaverse opportunity is going to grow to 800 billion in two years, only, right? So this is this is really interesting when you look at this. This is a huge market of course, right? Well, and then we also have this other flavor which I called the real world metaverse and not just me. but also Niantic, for example, which i'm going to show you a little bit.

But what is the real-world metaverse? Well, we have our physical world, the. world we live in, we walk on every day, hopefully. And then we also have these spatial computing devices, right? We have, for example, modern smartphones and we have devices like a quest. Or I could also bring a HoloLens here, right? We also have a HoloLens. And all of these devices have a bunch of interesting sensors built in, for example, of the HoloLens, you see them up front here and with all of these they can sense your environment you're currently in, so your room basically. And if I use my mobile here to scan my room, I can, you know, get that information, the so called spatial map and now think about if I combine the spatial map. If I take it from all of these different devices, but also include photogrammetry data for example, captured with people walking on the street and what not, right? So get all these computer vision data. You get all these frames and videos. If you combine all of this, run it through your computer, what you get is basically a very precise digital twin of the world, a 3D reconstruction as spatial map. And now if we overlay the digital twin of the world, we get the real world metaverse right? Because we get this kind of digital content layer on top of the real world with reference, positions and orientations and so where we can put virtual content. And I love this quote from Satya Nadella here, where he basically is saying that the real world metaverse, in general, the metaverse for him is fundamentally this: increasingly as we embed computing in the real world, you can even embed the real world in computing. And I think this is really the important part about the metaverse. Because, if we don't do this, we just end up with another online world. So, this is super relevant. Highly relevant quote here. Also about the metaverse is that it is not just purely virtual, but we can also bring in some real-world elements into this, right? And here is a quick example. For example, I built these kind of time machine portals, what I called our historic portals. And so what you see here is a demo I developed together with Azure Spatial anchors and what you see here is a video from 1946 which is here from my town Dresden. And if you look at the church towers, for example, there they are not there anymore because you know there was the World War II and bombing and all of this and so everything was rebuilt. So, I can provide this, that's why I call it a time machine or a historic portal. I can look into the past with this here. But that's not why I'm showing it to you. I'm actually showing it to you, not so much for the content you see here, but more about the technology. Because this is an implementation of the real-world metaverse. I can use Azure spatial anchors to put in a spatial anchor point there in the location and attach some virtual item here. In this case, it's a video, but it could be a photo, it could be a 3D model and it could be not just to look in the past but also look into the future. And with this Azure spatial anchors, I can drop this spatial anchor there and I don't need GPS, so it also works indoors and I don't need a QR code. I don't need any marker code or things like this. The real world becomes my marker and this is driven by all the data that comes from these devices that is being captured right and so I can put these virtual anchors there, I can touch virtual content and since it's all stored in this real-world metaverse in cloud, I can then come back or someone else with that device can come, could be iOS or Android or in HoloLens and they could scan it there and then we would re-localize and load the content I left there. And so this is really amazing stuff I think. And this is where we can also connect the real world with the virtual space. Also, Niantic is of course working on this. You might know Niantic from things like Pokémon Go and Harry Potter and a few other Geo location based mobile games, and here you can see one of their ARDK, it's the augmented reality development kit in action from Dynamic Lighthouse platform. Of course, as you can imagine, they have gambled a lot of user data already, and they can provide this digital twin of the real world and we can build some pretty cool, exciting experiences with it. Other tech players are also entering this world with implementations that can be leveraged, and you might also hear about it, called the term augmented reality cloud or AR cloud. It's basically the real world metaverse. Like we have reference positions and we can show them and we can attach virtual content to this. Another interesting point here is what you can see here in this video. Well, we have a spot robot here. Above you can see it there. This is the Boston Dynamics Spot robot, and these robots also constantly scan the world around them. So they basically do a 3D reconstruction about the environment and they do this because they want to navigate, they need to move in this world, so they do the constant slam, the constant scanning of the environment. But if I use my mobile phone or HoloLens, they do a different spatial mapping. And now I can use Azure spatial anchors with Ras SDK. I can basically combine these maps to spatial maps so I can share it, which means I cannot just use mobile phone or HoloLens and so on for these kind of real-world networks experiences, but I can also bring in robots, introduce so-called Robo Wars. And this is pretty amazing because I can run command and control or certain other interesting stuff with mobile robots where I can use my mobile or my HoloLens to control it. Just tap somewhere and the robot would walk there, because both all these devices share the same spatial map of the environment, the same real-world metaverse understanding, if you will.

OK, so let's also talk about the enterprise metaverse because you know, what is relevant for enterprise companies and there is across a bunch of different use cases. One of those could also be activating engaging virtual spaces, and some of these existing major wars platforms. And you know what could happen is also that you end up with more immersive business meetings. And if you take a look here, I'm going to show you a video in the next slide where we tested out one of these platforms together with our community where we tried out various immersive business meetings. So, take a look at this and listen to the audio there and you might hear someone talking. And then I'm going to interrupt the person, but then I'm actually using my hands to indicate that the other person should continue talking. I just won't watch this closely.

So, what I just talked about here in the video you saw what I just did right. I wanted to interrupt my colleague there and then I pointed with my hand. Hey, just continue talking right, just like you would do in a real meeting room and already this is so amazing, right? It's much more immersive. But also, although you see these people not as real people but as avatars, you feel much more immersed and also much more focused in these immersive business meetings. And so if you put on a VR device, of course, you're very much focused, right? And, hand tracking is so amazing you can do handshakes, high fives and whatnot. So this is super-cool and you feel much more immersed. And I can tell you, I memorized much more from these experiences. And so, I can remember many more things from these meetings because I'm also fully focused, you're fully immersed. You put on this headset, and you can see it much more with, you know, full vision and also spatial audio plays a big role. So spatial sound is of course also very much relevant when we talk about those. Because you can hear if someone is talking over there, you can actually hear it here in this ear. So spatial sound is very important, and as you can see this is another platform here, where we're testing and we're basically with our community of practice. We're running through all of these different platforms and experiencing those or testing those out in depth. And of course, give feedback and rate those, and as you can see in all of these, you can have business meetings. You can share your screen, you can see an Excel spreadsheet over there. And we can do some pretty cool stuff I think. And this is Altspace, by the way, and I'm going to switch real quick to a real time Altspace. So here now I'm going to give you a real-time live demo here of another Altspace world. And I'm using here, the Altspace client on my PC, but ofcourse typically I would use it with a VR headset, but look at this, this is a custom world we created at Reply, which is our so-called solo space. So, at reply we have these lounges which are called solo space and now we recreated it here in virtual reality. Or, well, not just virtual reality, but in the metaverse. And there's some gamified things here. But in general, as you can see, it looks pretty fancy. I like it a lot, and I'm showing you this because this will probably also be the stage for a bunch of conversations and expert guest talks I will have here. We will have some coffee talk at this table or some water cooler chat. And so I'm really excited about it and to welcome a bunch of my guests here in this kind of metaverse world for some upcoming episodes. So stay tuned for that, I'm super excited about it.


Coming back to the deck here and also what Microsoft is doing. I mentioned Mesh already and what Microsoft is doing with Mesh for teams is super interesting. As you might know, Microsoft Teams is one of the most used collaboration tools in the world. In fact, they have around 270 million users that are using our teams and basically Mesh for Teams will provide metaverse entry point, a gateway into the metaverse for all these users, and this is exciting because lots of enterprise companies already have this and they have already Microsoft as part of that infrastructure for identity, device management and now we get this kind of immersive mode and where we can have Mesh for Teams. So you can have these corporate spaces like you can see the one I've just shown you which we already created as part of Reply and you have kind of custom ones and then you can meet with your team. Have team meetings but also the cool part is you can also use your avatar that you create and use it instead of your webcam in teams just as a normal teams call. I'm excited to try this out, of course, and this also means your avatar, your digital representation, your virtual presence. You, as a digital human, basically it's a super interesting and important part of any metaverse experience. The challenge we currently have, for example, is with these various different metaverse platforms that each of those has a different kind of avatar system and avatar platform or character creation, whatever you want to call it. But if you take a look here, for example at this one here, this is my Animator avatar that I would use in Horizon from Meta. This is the other one here over there. This is my Altspace avatar and on the other side you see my bitmoji from Snapchat. It's all me basically. And I tried to recreate it as close as I look like, but as you can see, they all provide different kind of look and feel for these characters. And that is a big challenge. Because if you look at social media today, it's super important that you have a similar profile picture on all of these platforms because this is what people will use to recognize you right now. Try to do this with these avatars on different metaverse platforms. There's no real interop at the moment, which is also a very important part when we talk about the metaverse that we have open interoperability between all of these platforms. To make the true vision of the metaverse happening at some point. But anyhow, what we see currently, we have these big players and also smaller players building their own kind of walled gardens platforms. And so, they all have their own avatar system, and I think there's a big opportunity here for decentralization like blockchain and although decentralized protocols but think about you have a decentralized identity that you store on a public. Sure, right so not a platform owns your identity, not meta or not Amazon, not Google, not Microsoft, owns your identity and use that to log into various apps or platforms. But you use your own private public key mechanism basically, and this could be a blockchain or something else, but basically, it's really decentralized identity, right? And that provides also the opportunity that maybe we can have an avatar that is stored on the chain, not the Avatar itself, but an NFT and non fungible token which is connected to your wallet, basically, to your digital identity to your decentralized digital identity, you have your avatar connected as an NFT and then you can hopefully share it with all these other platforms and so this will be super important. I think because we want to be represented in all these different platforms and the end we need real true interop, but in the meantime, we also need a way to use similar overtones on all of these platforms, right? And of course, currently, they all look very comic like, but there's of course amazing progress happening when you look at things like metahuman from Epic here which is providing a really fancy tool to activate super realistic looking avatars and character 3D models or human 3D characters. And it's really amazing because it's a web app powered by remote rendering in the end, so it's using Unreal pixel streaming under the hood, but that's just a technical detail, but if you look at this video, it's pretty fancy what you get there. You have one of the best forms of rendering here with a lot of interesting shading algorithms being used, and you can customize it and create super photorealistic avatars and this might be the one which we want to use for in realistic avatars in metaverse platforms. I think this is needed at some point. But it's still a long way to go in terms of computing power, resources and so on. Of course, this requires a ton of resources. Just imagine 10s or hundreds of these kinds of realistic avatars walking in the virtual world, so great stuff. It's already here, but of course, it's just the beginning. And another thing where we can increase the realism with these virtual platforms are things like haptic feedback. And here, in this demo video, you can see a colleague of mine who is wearing a full body haptic suit and you can see when he's shooting the gun there, he is really feeling the false feedback and so this bodysuit uses Electro stimulation to provide haptic feedback. And here's another case we developed for a VR training application, and you see this is for a welding station, and the gentleman, look at when he reaches out for the red handle, look at his arm, and so when he reaches out for the red handle, he's really making sure that he's reaching out like this above it and not like straight through it. Because if we would reach straight through, it he would hit a collider on the exhaust, would feel the impact on his arm, right? And that's why he's reaching out like this, just like he would have to do in the real world, right? And so, this is a very important part about this kind of solution, I think, because we can simulate this training much more realistically. Although we are much closer to the real world and so we're excited about increasing the immersion in medical solutions with full body haptic feedback. But of course, also these kind of gloves, or like these kind of cubes you can attach to your fingers to provide haptic feedback for your hands and whatnot. And here in the scene you can see the gentleman using a virtual reality. It's kind of a lab training or I don't know, you see that's a chemical lab or something and what's cool about this device. It doesn't just provide the haptic feedback, you can also sample various textures to feel it, but what's really cool about it also provides thermal feedback, which means if you, for example, have a solution where you have a virtual coffee cup, I'll have a real coffee cup. Just imagine I have a virtual coffee cup and when I touch it, this virtual coffee cup, with my hand, I would feel it gets warm at my hand. If I were to release it, it doesn't feel warm anymore. They can even simulate a little bit of cold. And so, this is exciting. The more kind of sense a human has, we can support in these virtual metaverse solutions, the more believable, the more immersive it is. So, what's upcoming is also still of course smell and also taste, right? But there's also solutions up coming on the market that can simulate virtual scent. It's basically attach a little device here to your headset so it's sitting on the nodes and can emit certain perfume and so on. So, you can smell virtual stuff as well. But anyhow in general now there's much more to come. I think there's some amazing stuff we're about to see and this kind of metaverse topics and there's so much more to talk about, and I couldn't cover it. This episode is actually much longer than anticipated for these kind of metaverse episodes while we're trying to keep them a little bit shorter, but I thought it's actually interesting here to talk a bit about it. And basically you know, make sure we're all on the same page what we're going to talk about, but again, in this show, we're going to hear from all of our experts, various diverse views, of course, and some definitions, might be a whole different to what I just said right, but again, this is not a surprise because we're dealing with this highly dynamic metaverse market that is constantly changing. But I think it's really an amazing time to be alive where we see this kind of sci-fi technology becoming a reality. And I think we can all be part of it. And you know, work on it for the greater good and not make it this dystopian world we have seen in sci-fi movies. But make it actually something which we really enjoy and we can use with good, responsible usage patterns. Which is also very much important when we talk about digital humans and whatnot, that we have responsible and ethical usage of this. And so, I'm excited to talk about these different pieces and building blocks and flavors of the metaverse with my guests.

Thank you so much for joining our first episodes here at Meta minutes, your bite sized pieces of the metaverse. Watch our blog and follow our social media channels and subscribe to our YouTube channel to hear all about the upcoming episode and you can always visit our website valoremreply.com to watch all previous and upcoming episodes and also QuBites our sister show. And please don't hesitate to reach out to me whenever you want like social media, email whatever, to suggest new topics, to provide feedback. Well, also if you want to be a guest here for the show reach out, we can chat about it and maybe we see you in the next episode here. Well until then, take care and see you soon in the metaverse.