I'm an enthusiast of technology and when I see a mix of innovation with a strong use-case for humanity, I like to dive right in and stay on top of trends. I've always done that since the 1980s when I was involved with cutting-edge 3D modeling, rendering, and animation technologies — then running on mind-boggling (and extremely expensive) Unix workstations from Silicon Graphics. Thanks to that experience and early experimental software, I still remember the excitement of playing a two-user game in a 3D labyrinth where the ultimate goal was to see our opponent's character — basically a very primitive pile of polygons that could move.

Years later, when there was already a company selling Windows NT graphics workstations dreaming of rendering "Toy Story in One Day," I again remember being excited when a flock of birds animated with a complex algorithm to generate random paths and group behaviors surprised us all when one bird passed very close and looked at the (virtual) camera. A unique, magic moment for computer graphics that made us all dream about the possibilities of "AI" because a poorly 3D-modeled bird looked straight at us...

Anyone involved in those early days of 3D computer graphics, when "First Person Shooter" nightmares didn't yet exist, will also remember vividly how we all started to envisage the possibilities to create immersive audio soundscapes, recreating the physical world by manipulating objects and particles, leading to the rediscovery (for me at least) of ambisonics, virtual microphones, soundfield synthesis, etc. In a way, a first foray into what we would call today (real) Spatial Audio. And that was before HD video, before gaming was an industry, before the Internet was in every home, and long before Second Life was imaginable.

Fast forward to the pandemic days, and Facebook discussing metaverse feels like a joke. I've been there, done that. All these years that I have followed the evolution of computer graphics with hyper-realistic ray-tracing images, I have failed to see the excitement of Phong shading basic 3D online gaming environments, much less when seen through Virtual Reality goggles. I never even understood the appeal of "virtual" and I was always excited with "reality" — first by recreating it, then by augmenting it.

Non-fungible experiences stored in our brains, not the blockchain, are like musical emotion. Non-fungible. That is where I believe technology is going in the next few years. And those who have their VR sleep masks will never see it coming...

I truly believe that Augmented Reality (AR) is a key concept for a lot of things and totally key for the audio industry. AR is not a single technology, and will become an enabler principle now that we can tie it with edge computing and edge-AI. That is already visible in a number of implementations. Just like real voice assistants, it will take time to get there, but the benefits are clear. Augmented audio is the ultimate-hyper-high-end audio level, because it's focused on us, on making us hear better. Also, because augmented audio, like AR applications in general, results from a combination of augmenting stimuli to different senses — sound amplified by audio systems plus haptics and — it is my strong conviction — vision. Augmenting natural vision. Because without vision, AR becomes VR, limited to virtual.

This is not about social interaction with "virtual" fake worlds. We understand that Mark Zuckerberg needs that to escape from the mess he created with social media. This is about interaction with the real world, augmenting our senses and experiences.

In fact, it already started with all the many things that are enabled by smartphones (screens), smart watches (wearables), and true wireless earbuds (hearables), which are the first platforms for AR. Augmenting our senses and experiences will never happen with the VR goggles on.

An example applicable to audio AR is foreseeable in the real time collaboration network platforms that studios and musicians are using now and have been trying for a while. Those remote collaboration platforms, particularly now with the pandemic, are evolving to new levels, beyond the basic levels of simple interaction. Elk Live, Sessionwire, Steinberg VST Connect, Spire, Sphere, Soundtrap, and many others, are building dedicated tools to augment the user experience. Some more “video-centric,” others totally DAW-centric, some hardware-based, and others completely software/web-based. But all suffering from the same problem of network latency, which (like the speed of sound) will always be there. Augmented reality can help in such applications by creating an immersive interface for each musician that — like the maestro in an orchestra — helps compensate for the distance, and indirectly the effects of latency. Dear Reality is an example of a company with interesting technologies for that purpose, as long as they keep out of the VR goggles.

The remote collaboration platforms will evolve with dedicated and unique features, much in the same way as voice chat apps for gaming evolved to go beyond what audio-conferencing platforms originally enabled. Also, as we are seeing AI-based audio features being introduced in music production and audio editing tools, with edge-AI platforms we will see audio being processed in real time at the source. Directly from the microphone, and recorded or fed directly to our ears. Augmented Audio means the possibility to have the whole iZotope RX suite available for real-time processing by a team of genius audio engineers, cleaning up the sounds of the world in our earbuds. No longer a question of "how much DSP" horsepower we can have, but having AI making intelligent use of that DSP depending on the context. Augmenting OUR reality.

And what do I see as the ultimate Audio Augmented Reality application? Two weeks ago I discussed the potential for moving away from hearing assistance to augmented hearing and I truly believe that the field of "super-hearing" is a horizon to be explored. (Likewise, imagine having fully adjustable prescription glasses which are also able to zoom...). The other big application is obviously language translation and having virtual voice assistants that are able to deal with multiple language environments.

Of course, there will be deviations in the use of such technologies, the most dangerous of all - particularly for now - being the creation of deep-fakes. Anyone focusing on AI and augmented reality knows that the ultimate goal is to improve and build upon our own abilities and senses. A few twisted minds will of course use it for evil so we will require new levels of artificial intelligence to help spot the deep-fake differences. Remember, we can augment our intelligence, but we can also augment our stupidity.

audioXpress | Voice Coil | LIS 

audioXpress features great articles, projects, tips, and techniques for the best in quality audio. It connects manufacturers and distributors with audio engineers and enthusiasts eager for innovative solutions in sound, acoustic, and electronics.

Voice Coil, the periodical for the loudspeaker industry, delivers product reviews, company profiles, industry news, and design tips straight to professional audio engineers and manufacturers who have the authority to make powerful purchasing decisions.

The Loudspeaker Industry Sourcebook is the most comprehensive collection of listings on loudspeaker material in the industry. Purchasers and decision makers refer to the guide for an entire year when making selections on drivers, finished systems, adhesives, domes, crossovers, voice coils, and everything in between.