Virtual reality (VR) is very sensitive to latency. For an immersive experience, new images must be rendered quickly to reflect the slightest head movements – in a matter of milliseconds. Here, late-warp is a necessary trick, which “re-projects” a rendered image for a more recent view just before displaying it. Can this be useful for desktop gaming too? NVIDIA research presents results from a study that seeks to better understand this application of late-warp.
The HPG2020 paper, Post-Render Warp with Late Input Sampling Improves Aiming Under High Latency Conditions, by NVIDIA research shows how player performance changes when rendered frames are updated based on more recent input.
For example, when a player aims to the right with the mouse it would normally take some time for that action to be received, rendered and displayed. Instead, an existing frame can be moved to the left to show the result much sooner (see above). This is an oversimplification but a real “late-warp” transformation is similar. The paper shows even basic implementations can effectively mitigate latency on the order of 80ms (see below), which might be seen in cloud gaming systems.
Late-warp is a known technique in VR for hiding latency, but its effectiveness in competitive esports is under-explored. Competitive gamers playing esports want to maximize their performance. Computer system latency can significantly affect their ability to play at their best. Cloud gaming adds streaming latency which reduces player performance. Consequently, serious esports players still choose to play on a local setup. Using late-warp to mitigate latency could increase accessibility of many games and gaming platforms for less serious players and when applied on local systems. The same technology could be applicable to other interactive video applications such as remotely-rendered content creation and robotic teleoperation.
The paper presents results from a user study. Participants completed aiming tasks in a purpose-built FPS game, running on PCs with NVIDIA RTX 2080 Ti GPUs. Some tasks required the players to move in order to hit targets. They were subjected to a number of simulated latency conditions, some with late-warp enabled. By simulating latency and late-warp in a single application, all variables in the experiment could be tightly controlled.
Different conditions were chosen to bound the effectiveness of possible late-warp implementations, both giving more broadly applicable results and avoiding confounding effects from specific implementation details. Unsurprisingly, players performed significantly worse in the 105ms latency condition than in the 25ms latency condition. However, all late-warp conditions, including even the simplest condition, let players perform like they did in the 25ms latency condition. More specifically, all late-warp conditions eliminated 80-90% of the player performance penalty caused by high latency.
