Summary
Sonyis possibly developing a solution for equipping physical controllers with virtual buttons using a variety of technologies and combinations thereof. The hardware-agnostic concept offers some new insights intoSony’s ongoing interest in finger tracking solutions.
The PlayStation maker has been experimenting with finger movement detection techniques for years. A recent uptick in Sony’s patent activity evensuggested that the PS VR2 might support finger tracking, but that didn’t come to pass. As of late 2023, the company has yet to commercialize any such solution.
And yet its research and development efforts on this front continue to this date, as suggested by a newly secured patent describing a way to extend physical controller capabilities using hand and finger tracking in order to achieve virtual button functionality. Specifically,Sonypatented a multi-modal approach to tracking virtual button presses that can leverage anything from wireless receivers and microphones to smartphone cameras and inertial measurement units, devices that track body properties such as force and angular rate of rotation.
Sony’s Patented Virtual Button Tracking Methods
Several implementations described in the newly emerged patent incorporate an analysis of channel state information (CSI) in order to detect both large and subtle gestures. By tracking amplitude variations of Wi-Fi or Bluetooth data packets exchanged between a transmitter and receiver, which can be anything from a controller to a smartphone, Sony believes it can accurately detect actions like waving, typing on a keyboard, and pressing a specific spot on a gamepad, all of which have distinct CSI footprints.
True toSony’s long-standing interest in machine learning, the freshly surfaced patent describes a number of implementations that would leverage an AI model to intelligently interpret sensor data. However, the company believes that even a rudimentary analysis of things like Wi-Fi signals and camera feeds could be used for achieving virtual button functionality. One such solution describes a controller-mounted smartphone that simply records the user’s finger movements, then translates them to button presses.
The overall solution is extremely broad, even by patent standards. The generic nature of Sony’s latest patent suggests that its underlying technology is unlikely to ever be commercialized in its entirety, as some of its described implementations are bound to be more practically viable than others. Nevertheless, the sole existence of this IP filing indicates that Sony is still very much exploring the idea of adding finger tracking to its electronics. Time will tell whether this ongoing R&D effort yields anew version of the DualSenseor a completely novel category of gaming accessories.
