Parsing Input in Software and the CPU Limit

Before we get into software, for the sake of sanity, we are going to ignore context switching and we'll pretend that only the operating system kernel and the game are running and always get processor time exactly when they need it for as long as its needed (and never need it at the same time). In real life desktop operating systems, especially on single core processors, there will be added delay due to process scheduling between our game and other tasks (which is handled by the operating system) and OS background tasks. These delays (in extreme cases called starvation) can be somewhere between a handful of nanoseconds or on the microsecond level on modern systems depending on process prioritization, what else is happening, and how the scheduler is implemented.

Once the mouse has sent its report over USB to the PC and the USB root hub receives the data, it is up to the OS (for our purposes, MS Windows) to handle the data next. Our report travels from the USB root hub over the system bus (southbridge through the north bridge to the CPU takes +/- some nanoseconds depending on load), is put on an input stack (in this case the HID (Human Interface Device) stack), and a Windows OS message (WM_INPUT) is generated to let any user space software monitoring raw mouse input know that new data has arrived. Software written to take full advantage of hardware will handle the WM_INPUT message by reading the appropriate data directly from the HID stack after it gets the message that data is waiting.

This particular part of the process (checking windows messages and handling the WM_INPUT message) happens pretty fast and should be on the order of microseconds at worst. This is a hard delay to track down, as the real time this takes is dependent on what the programmer actually does. Latencies here are not guaranteed by either the motherboard chipset or Windows.

Once the software has the data (after at least 1ms and some microseconds in change), it needs to do something with it. This is hugely variable, as developers can choose to implement doing something with input at any of a number of points in the process of updating the game state for the next frame. The thing that makes the most sense to me would be to run your AI based on the previous input data, step through any scripted actions, update physics per object based on last state and AI decisions, then get user data and update player state/physics based on previous state and current input.

There are cases or design decisions that may require getting user input before doing some of these other tasks, so the way I would want to do it might not be practical. This whole part of the pipeline can be quite long as highly intelligent AI and immersive physics (along with other game scripting and state updates) can require massive amounts of work. At the least we have lots of sorting, branching, and necessarily serial computations to worry with.

Depending on when input is collected and the depth and breadth of the simulation, we could see input lag increase up to several milliseconds. This is highly game dependent, but it isn't something the end user has any control over outside of getting the fastest possible CPU (and this still won't likely change things in a perceivable way as there are memory and system latencies to consider and the GPU is largely the bottleneck in modern games). Some games are designed to be highly responsive and some games are designed to be highly accurate. While always having both cranked up to 11 would be great, there are trade offs to be made.

Unfortunately, that leaves us with a highly variable situation. The only way to really determine the input lag caused by game code itself is profile the code (which requires access to the source to be done right) or ask a developer. But knowing the specifics aren't as necessary as knowing that there's not much that can be done by the gamer to mitigate this issue. For the purposes of this article, we will consider game logic to typically add somewhere between 1ms and 10ms of input lag in modern games. This considers things like decoupling simulation and AI threads from rendering and having work done in parallel among other things. If everything were done linearly things would very likely take longer.

When we've got our game state updated, we then setup graphics for rendering. This will involve using our game state to update geometry and display lists on the CPU side before the GPU can start work on the next frame. The speed of this step is again dependent on the implementation and can take up a good bit of time. This will be dependent on the complexity of the scene and the number of triangles required. Again, while this is highly dependent on the game and what's going on, we can typically expect something between 1ms and 10ms for this part of the process as well if we include the time it takes to upload geometry and other data to the GPU.

Now, all the issues we've covered on this page go into making up a key element of game performance: CPU time. The total latency from front to back in this stage of a game engine creates a CPU limit on performance. When what comes after this (rendering on the GPU) takes less time than everything up to this point, we have hit the CPU limit. We can typically see the CPU limit when we drop resolution down to something ridiculously low on a high end card without seeing any real performance gain between that and the next highest resolution.

From the examples I've given here, if both the game logic and the graphics/geometry setup come in at the minimum latencies I've suggested should be typical, we could be CPU limited at as much as 500 frames per second. On the flip side, if both portions of this process push up to the 10ms level, we would never see a frame rate over 50 FPS no matter how fast the GPU rendered anything.

Obviously there is variability in games, and sometimes we see a CPU limit at less than 60 FPS even at the lowest resolution on the highest end hardware. Likewise, we can see framerates hit over 2000 FPS when drawing a static image (where game logic and display lists don't need to be updated) with a menu in front of it (like when a user hits escape in Oblivion with vsync off). And, again, multi-threaded software design on multi-core CPUs really middies up the situation. But this is near enough to illustrate the point.

And now it's on to the portion of realtime 3D graphics that typically incurs the most input lag before we leave the computer: the graphics hardware.

Reflexes and Input Generation Of the GPU and Shading
POST A COMMENT

85 Comments

View All Comments

  • mathew7 - Thursday, September 08, 2011 - link

    What I realized (i)racing in the past year is that a consistent lag can still offer playable experience when everything is under control (like no tire grip losing in racing). I think this is why consoles seem to get away with laggy TVs. What I mean is that the brain will train itself to anticipate any movement. For instance, there was a comment about turning in earlier. But if the lag is inconsistent (let's say +/- 1 frame, that would mean 32ms between lowest and highest lag), the brain cannot adapt to it, as this is unnatural. FPS stutter is such an inconstistency, as is loading something during play (and game controls freezing for much less than 1s).

    But even consistent lag will not offer the best performance (I'm talking about lap times, not FPS), because anticipation is just 1 part, reflex is another, and your reflexes are what input lag affects most. During my racing, I found that I can actually recover from most slides with my LCD TV as opposed to my (much newer) LCD monitor. Some tests with a low-res high-speed camcorder (192x108x240fps), cloned displays and fast steering wheel movement showed 2-4 frames lag between TV and monitor. That is 32-64ms. The TV itself had around 1-2 frame lag compared to the wheel, but this included the complete chain of wheel sampling->USB->game CPU->GPU->TV.
    Reply
  • junior262133 - Monday, March 04, 2013 - link

    Oh sweet baby Jesus thank you for this article. It saddens me knowing that Counter strike Global offensive has horrendous input lag in comparison to counter strike source. This issue has been plaguing me since it was in beta. I would mostly rank first in css when playing, but going over to global offensive was a living nightmare. What baffles me is almost no one seems to notice, considering most of us come from a background of CS:s or 1.6 on a Crt.

    On top of that nvidia has completely removed the option to set pre-rendered frames to zero.

    i have tried submitting tickets in to patch cs:go but they seem to not be getting any attention. Maybe i'm talking to the wrong people. If you could please do so, visit these 2 pages in which i try to address the issues at hand. It wouldn't hurt if you could explain in one comment that the reason there is input lag in Cs:go is because of the game code and not some random pc glitch of mine. Most people seem to be oblivious to the problem.

    http://64bitvps.com/csgo/ticket/mouse-laginput-mis...

    http://64bitvps.com/csgo/ticket/let-us-have-the-op...

    THANK YOU SO F*****G MUCH!!!!!
    Reply
  • jigglywiggly - Saturday, October 05, 2013 - link

    does afr in sli add one frame of input lag? Reply
  • ibaldo - Sunday, October 06, 2013 - link

    I am looking for information about possible difference in lag between monitor connections.
    I mean, comparing VGA vs DVI.
    Thanks for this excellent article!
    Reply
  • willCho - Tuesday, January 15, 2019 - link

    I would like to watch the video, but I can't find it anywhere and can't find it on youtube. Can someone post a link? Reply

Log in

Don't have an account? Sign up now