Apple Receives VR Headset Patent Which Illustrates The Use Of Powerful Cameras & Base Station Connectivity

arlier this February, the U.S. Patent and Trademark Office officially granted Apple a patent which related to their mixed reality (MR) system, including a head mounted display (HMD) and a base station which could be a Mac, iPhone/iPad or game console to play VR content and more. 

The granted patent, which bypassed the application phase under "Apple" as the assignee, covers a range of equipment and methods for creating mixed reality content for the wearer of the MR headset or smartglasses in the future. 

The mixed reality system includes the MR headset which connects to a separate device through either a wired or wireless connection. The separate device or the base station, according to Apple, could be an iMac, MacBook, iPhone, iPad, game controller and/or game system and will feature software and hardware configured to generate and render frames that include MR content. Whether a gaming system will include VR content from Xbox, PlayStation or a future VR-Apple TV box isn't yet specified.

The headset that Apple has developed features a vast range of cameras both on the interior and exterior which capture the real world as well as the users expressions, hand gestures and eye movements. It also features world-facing sensors which collect data about the wearer’s environment and user-facing sensors which collect data about the wearer including depth and lighting information. The data collected by the sensors can then be transmitted to the base station through the connection.

The world facing sensors are likely to include one or more world mapping sensors, for example either infrared (IR) cameras with an IR illumination source, or Light Detection and Ranging (LIDAR) emitters and receivers/detectors. Incorporating these sensors into the mixed reality headset allows for the capture of depth or range data for objects and surfaces in the wearer’s environment. The data gathered could potentially be used in positioning virtual reality content to be positioned in the real environment at correct depths.

Other world sensors may feature one or more head pose sensors, for example, IR or RGB cameras. These cameras can capture data about the position, orientation, and/or motion of the wearer and/or the wearer’s head in the environment.The data gathered could be used to augment information collected by an inertial-measurement unit of the HMD, essentially determining how to render and display virtual content in the wearer's environment and virtual content within the views. For example, different views of the environment may be rendered based at least in part on the position or orientation of the wearer's head, whether the wearer is currently walking through the environment.

The extensive number of cameras featured in the XR headset supports a recent report which described the HMD having 12 cameras, more specifically, the report from The Information stated that: "A mixed-reality headset Apple is developing will be equipped with more than a dozen cameras for tracking hand movements and showing video of the real world to people wearing it, along with ultra-high-resolution 8K displays and advanced technology for tracking eye-tracking technology." 

It has been reported that Apple have been secretly developing two kinds of wearable devices, a VR Headset and Glasses for some time now. The VR Headset was to be a crazy powerful headset that connected to a Mac or high-end station which was originally rejected by Jony Ive. However, with Ive now gone, it looks like the higher end VR headset could be back on the table as the patent supports a headset taking advantage of higher-end processors in a "base station."