Nintendo Wii did it best, the Wii controller was an innovative piece of motion tracking tech. A device that utilised various motion and force sensors in collaboration with software to create realism oriented virtual experiences through Wii games.
World-wide across industries, companies and people are increasingly building interest in popular concepts such as the metaverse and AR/VR/MR experiences. Notably, a barrier to advancement in VR worlds is the ease of use, access, implementation and vitality of VR equipment. VR headsets are a separate tool devised to create a sense of VR immersion by isolating the users’ FOV and introducing virtual elements on a ‘near to face’ display. Often VR devices require a direct connection (wired or OTA) to a computer powerful enough to process the experience then stream it to the device. The immersion is created through motion sensors much like the ones found in the Wii controller, they respond in a similar fashion, allowing the user to visually explore or interact with the VR experience through head movements. These headsets are often combined with a separate controller that operates in the same way but instead provides further interaction through the introduction of hands.
There are a lot of elements required to build the future of entertainment and digital interaction, quite a few systems need optimising and the process to getting inducted into a virtual stream needs refining. The primary barriers to entry remain access to the necessary equipment at affordable costs, and the range/variation of experiences readily available.
Utilising existing household devices to create ‘open view’ VR experiences, as a prelude to ‘isolated’ fully immersive VR experiences.
Disney recently, through Disney+, have taken steps to explore the combination of devices in order to extend & enhance the traditional & expected state of television/passive visual entertainment. Disney created a passively interactive AR experience, engaged with through a companion mobile app exclusive to iOS. The experience works by loading a selected film (‘Remembering’) from the Disney+ app on a separate device (TV, Laptop, Desktop). While watching the RR (Regular Reality) experience on the larger device, the mobile app, activated and influenced by the imagery on screen, reacts to display a constantly changing AR experience on the mobile’s screen through the device’s camera. The AR experience focuses on advancing the RR experience, albeit through a distracting introduction of AR. This means there is innate confusion on where to focus, and which part of the experience to engage with. However, the combination of devices here, in an attempt to extend the viewing experience and increase the entertainment value, is interesting. Interactivity in this style currently comes across as novelty, but has incredible potential to act as a bridge between low-interactivity media and totally engaging fully immersive VR systems.
A cross-device multi-platform controller, an interactive immersive media standard and API.
Imagine, you are watching content on Netflix, you noticed once the show began you received a silent ping on your phone alerting you that the stream you are experiencing is ENGAGE controller compatible. So, you tap the notification, launching the ENGAGE ENABLED mobile app and initiate a wifi connection to your device streaming the Netflix content. Presented with a selection of controllers determined by the kind of content you are streaming, you are given the choice to interact with the streamed content via your mobile. Your mobile device is now a controller, with the ability to influence content on screen in various ways depending on the content.
Some examples: Controlling 360 degree film experiences (like the ones found on YouTube mobile), using your mobile device as a controller, the experience is operated in the same way, but instead of awkwardly moving your head, around making for an uncomfortable streaming experience, you can simply rotate the phone in your hand while looking forwards towards the larger display streaming the content that is reacting to the controller’s movements.
Controlling a virtual element in a film on Netflix, you control the main characters sword, the wind, a mysterious force that reactively spooks the character in odd ways, the movements of their pet, the direction of the car they are driving.
This is all done using the phone’s built in sensors, this ensures a natural learning curve and removes the complexity of having to observe multiple screens at once.
Launching the app on mobile from a notification directly leads the user to a blank screen like such. This allows immediate engagement with the content streamed on the screen.
