A new holographic reality for business

Imagine a future where, instead of attending a Zoom call to collaborate with your colleagues or clients, you could each generate accurate, full-size holograms of yourselves, which could be transmitted to a virtual replica of a site or office in real time.

By using mixed-reality solutions such as Microsoft HoloLens, you could then see, hear and interact with these holograms, as if you are all present in the same physical space. You could shake hands, pass around documents and even tweak 3D models or prototypes.

It may sound like science fiction, but Maximilan Doelle, CEO of UK business HoloMeeting, is in the process of developing this technology – and believes it will be commonplace within as little as three years.

“We are planning to invest around £1.5m into our HoloPortation product offering within the next 12 months,” he says. “I believe it will completely revolutionise business interactions and personalise previously remote experiences. Interacting with remote users will become as natural as face-to-face communication.”

Unlike other hologram-generating software, which requires users to have access to volumetric 3D scanners and incomprehensible amounts of processing power, Doelle’s solution requires little more than a laptop camera. “We use a deep neural network that is processed in the Microsoft Azure cloud to generate a 3D avatar from video input from a laptop camera,” Doelle explains. “Clients can self-host the whole solution, which is key for large enterprises that want to maintain complete control of their data. It’s these factors that make what we are offering completely unique.”

The time is ripe for this type of technology to advance. As a direct result of the Covid-19 pandemic, businesses the world over have had to facilitate an entirely remote workforce – and fast. “This expedited move towards home working has given way to a number of challenges that only innovative technologies can meet,” says Annette Jump, senior director analyst for emerging technologies and trends research at Gartner.

According to Mary Hamilton, managing director of Accenture Labs and Technology Innovation for the Americas, the most fundamental challenge has been recreating the presence that business leaders and colleagues once had. “Creating a human connection in a virtual mode of working is incredibly difficult,” she says. “You can see your co-workers in video conferences, but there is still a communication gap. Video conferencing limits our ability to demonstrate or share things. For example, if I wanted to pick up a product model and show you something about it, I couldn’t hand it to you – virtually or otherwise, in a traditional call or video conference.”

Doelle agrees, arguing that the traditional ‘watercooler chat’ is incredibly important, but very difficult to replicate with video conferencing. “The lack of chance encounters is a bigger issue than many realise,” he says. “For example, hearing a sigh of frustration by a colleague might prompt a spontaneous discussion that ends with a quick solution. This spontaneous collaboration is difficult to achieve remotely since it requires a conscious effort from team members to ask for help.”

The move to remote working also presents challenges in terms of training. “Because of social distancing, people cannot be trained en masse at the moment,” says James Watson, chief marketing officer at UK-based virtual-reality (VR) firm Immerse. “This is a big problem as traditional training often involves lots of people in a small space.”

‘As businesses and sectors adapt to entirely new ways of working, the potential of mixed reality is more likely to be realised and this will only accelerate as time goes by.’

Against this backdrop, it’s easy to see the potential that hologram-based solutions like HoloPortation might offer. While this technology is far from being ready, its underlying augmented-reality (AR) platform HoloMeeting – which relies on HoloLens 2 and uses a virtual representation of a person’s head, shoulders and hands, rather than lifelike holograms – is already used by some global corporations.

Doelle believes HoloMeeting provides an immersive office environment that addresses many of the challenges businesses are facing right now. “We ourselves use it for daily stand-ups as well as for holographic colocation,” he says. “In the latter, we simply sit next to each other holographically for one or two hours a day to regain the spontaneous collaboration and personal relationship-building that was previously lost.”

Several other innovative technologies are also gravitating in this direction. US-based Spatial’s technology, for example, is device agnostic and has recently improved access for a much greater number of workers, regardless of whether they have access to a headset or not. People can join co-workers who are collaborating on a HoloLens 1 or 2, Magic Leap or Oculus Quest device, directly from a web browser.

“Connecting via video chat is great, but when it comes to collaborating in real time with colleagues, staring at a face on a screen for hours on end just isn’t the same as being in the same room,” says Anand Agarawala, Spatial’s CEO and co-founder. “In Spatial you get back that real-world experience – real-​time feedback and collaboration, and the ability to draw, build and tweak documents, images and models in 3D space. It hugely reduces the lag time between ideation and go-to-market strategies, connecting the very best people to do the job, regardless of where they are located.”

Agarawala argues that as soon as you take collaboration out of the confines of the 2D screen and into the 3D space around you, the dynamics of that meeting change. “People are instantly more engaged and motivated,” he says. “They become an active part of the meeting, rather than a passive observer. It allows for increased engagement, closer connections and trust, and the more efficient resolution of problems.”

It’s perhaps no surprise that Agarawala has seen a 1,000 per cent uptick in interest in Spatial since the start of the year, expanding beyond its traditional Fortune 1000 customers, which include Mattel, BNP Paribas, Nestle and Pfizer, to small and medium-sized businesses and prosumers.

“The most popular use-cases we see include brainstorming, idea generation and creative moodboarding,” says Agarawala. “You can fill the whole project room with shared ideas – via sticky notes, images generated from the web through HoloLens voice commands, or notes, photos, videos or PowerPoints uploaded from your headset, desktop or phone. Collaboration and iteration is the other key use-case – manufacturers, oil and gas firms or retailers, for example, can quickly discover potential issues, label up changes by annotating directly on the model, see it from every angle and share live feedback.”

Accenture is also working on solutions that leverage virtual, mixed and augmented reality for remote collaboration. Its Synapse Immersive Collaboration Platform is a high-fidelity virtual solution in which users can visit digital twins of Accenture’s Labs and Studios and collaborate side-by-side in a rich environment.

“We’ve also created the ‘Nth Floor’, in partnership with Microsoft AltSpace,” says Hamilton. “This is a virtual office space with a San Francisco backdrop, in which users can move beyond flat-screen video conferencing and into a 3D, interactive environment.”

Meanwhile, Microsoft is spearheading the use of holographic solutions to solve issues around training. For example, the technology has been used to capture highly specialised ventilator production processes, so they could be employed by major UK industrial and engineering firms as part of the VentilatorChallengeUK – an initiative aimed at gearing up to manufacture 10 years’ worth of ventilator production in almost as many weeks.

To enable this, significant digital transformation and leadership has been required, and a key enabler has been using HoloLens with Dynamics 365 Guides for step-by-step instructions and Dynamics 365 Remote Assist for access to remote experts. These out-of-the-box solutions have been rapidly deployed to support operator training across multiple companies and sites to ensure quality standards required for medical-grade ventilators.

Training footage captured on HoloLens has been edited with additional information using the Vuforia Expert Capture app from US-based software firm PTC. “It was then deployed back onto the device for workers to use as a step-by-step guide across the country as they build replica assembly lines from scratch and begin production,” explains Professor Rab Scott, head of digital at the University of Sheffield Advanced Manufacturing Research Centre. “Remote Assist has also allowed workforce training and support to go one step further. Individual operators on the assembly line can quickly problem-solve and manage critical procedures with support from remote specialist teams who can share knowledge and expertise via HoloLens in real time, helping operators learn quicker and retrain faster, in turn reducing waste, saving time and improving productivity.”

At the same time, Immerse is helping firms including DHL, Shell and SAP to address their training challenges through its cloud-based VR platform.

“VR-based training is a fantastic alternative to traditional in-person training. It’s also a far more effective method than other alternatives like video training,” says Immerse’s Watson.

Indeed, recent research from the University of Cambridge has found that VR training can lead to dramatic efficiency improvements in workers’ performance. Training via VR resulted in 28 per cent more productivity, 55 per cent faster learning and 200 per cent fewer mistakes than non-VR methods.

“The fact that people retain information better using VR means that you don’t have to train them as often – which translates to further cost savings,” says Watson. “Because you don’t have to bring everyone together in the same space you can train a large number of people at the same time.”

As well as helping customers resume business-critical training, Watson believes Immerse will help get employees back to work safely. “It can be used to train people in social distancing,” he explains. “For example, we can create a virtual office, which workers can navigate using VR. They can get to grips with a new one-way system, for example, or be alerted when they get too close to virtual colleagues.”

While all these solutions are seeing varying levels of adoption, experts agree that there are still several issues preventing mainstream use.

“One of the biggest hurdles we see is how revolutionary mixed-reality (MR) is from an interface point of view. It will take up to 10 hours to really get used to interacting with holograms and to understand how they interact with physical objects,” says HoloMeeting’s Doelle. “Cost is another problem. A company needs to be willing not only to invest into the hardware, but also into the training of its future users.”

Gartner’s Jump agrees: “The biggest issue is that the technology is immature. Many companies do not understand VR, AR or MR, or where they can benefit. Simply buying the devices isn’t enough – businesses need good software that can integrate with their current systems, as well as a thorough understanding of the benefits. That’s not to mention ongoing training.”

Despite these roadblocks, there is a lot of positivity about the future. “As businesses and sectors adapt to entirely new ways of working, the potential of MR is more likely to be realised and this will only accelerate as time goes by,” says Leila Martine, director of product marketing for MR at Microsoft UK. “Over the coming years, we will continue to see an increase in MR adoption as the technology becomes more deeply embedded into business process. We anticipate an increase in sector-specific MR applications, using out-of-the-box applications to solve business challenges including retail, education, manufacturing and healthcare.”

These technologies are likely to be instrumental in creating a new normal for business in the months and years to come.

“Those executives who were used to hopping on a plane to New York for a three-hour meeting to ‘press the flesh’ and look someone in the eye are now realising that, with the right technology, virtual meetings can be just as effective,” says Watson. “Instead of taking 36 hours out of a week, it would only take two. They cost less and there’s minimal impact on the environment. They also open up new possibilities – new products or buildings that haven’t even been built yet can be visualised and explored in virtual reality. Meanwhile, the studies in virtual training effectiveness show that there’s no need to revert solely to traditional methods.”

He concludes: “We’re already seeing MR hardware becoming smaller, lighter and cheaper, and it’s becoming far easier to use. If there’s one good thing about the current pandemic, it’s that is has resulted in a greater openness to new technology. There’s no question that MR solutions – which will increasingly incorporate more realistic holographic-style images – will be a huge part of business in the future.”