Sony's New AI Patent Predicts Button Presses via Finger Camera

Sony has taken a significant step towards enhancing gaming experiences with a new patent that aims to reduce latency in future hardware. The patent, titled "TIMED INPUT/ACTION RELEASE," was first spotted by Tech4gamers and filed under the number WO2025010132. It details a novel approach to streamline the "timed release of user commands" by predicting a player's next move using an AI model supported by additional sensors.

Sony's initiative comes in the wake of introducing PlayStation Spectral Super Resolution (PSSR) with the PlayStation 5 Pro, which, while capable of upscaling resolutions to 4K, can introduce latency issues when coupled with frame generation technologies. This latency can make games feel less responsive, a critical concern for gamers, especially in genres like twitch shooters where every millisecond counts.

The patent outlines a system where a machine-learning AI model works in tandem with an external sensor, such as a camera focused on the controller, to anticipate the user's next input. Sony explains, "there can be latency between the user's input action and the system's subsequent processing and execution of the command. This in turn results in delayed execution of the command and unintended consequences in the game itself." By predicting the next command, the system aims to execute actions more seamlessly, reducing the delay between input and on-screen action.

An intriguing aspect of the patent is the potential use of the controller's buttons as sensors. Given Sony's history with analog buttons, this could be integrated into a next-generation controller, further refining the input prediction technology.

While it's uncertain whether this exact technology will be implemented in the PlayStation 6, the patent clearly indicates Sony's commitment to tackling latency issues. This is particularly relevant as rendering technologies like FSR 3 and DLSS 3, which can add frame latency, gain popularity. Sony's efforts to minimize latency without sacrificing responsiveness could significantly enhance gameplay, especially in scenarios requiring high framerates and low latency.