Parsing Input in Software and the CPU Limit

Before we get into software, for the sake of sanity, we are going to ignore context switching and we'll pretend that only the operating system kernel and the game are running and always get processor time exactly when they need it for as long as its needed (and never need it at the same time). In real life desktop operating systems, especially on single core processors, there will be added delay due to process scheduling between our game and other tasks (which is handled by the operating system) and OS background tasks. These delays (in extreme cases called starvation) can be somewhere between a handful of nanoseconds or on the microsecond level on modern systems depending on process prioritization, what else is happening, and how the scheduler is implemented.

Once the mouse has sent its report over USB to the PC and the USB root hub receives the data, it is up to the OS (for our purposes, MS Windows) to handle the data next. Our report travels from the USB root hub over the system bus (southbridge through the north bridge to the CPU takes +/- some nanoseconds depending on load), is put on an input stack (in this case the HID (Human Interface Device) stack), and a Windows OS message (WM_INPUT) is generated to let any user space software monitoring raw mouse input know that new data has arrived. Software written to take full advantage of hardware will handle the WM_INPUT message by reading the appropriate data directly from the HID stack after it gets the message that data is waiting.

This particular part of the process (checking windows messages and handling the WM_INPUT message) happens pretty fast and should be on the order of microseconds at worst. This is a hard delay to track down, as the real time this takes is dependent on what the programmer actually does. Latencies here are not guaranteed by either the motherboard chipset or Windows.

Once the software has the data (after at least 1ms and some microseconds in change), it needs to do something with it. This is hugely variable, as developers can choose to implement doing something with input at any of a number of points in the process of updating the game state for the next frame. The thing that makes the most sense to me would be to run your AI based on the previous input data, step through any scripted actions, update physics per object based on last state and AI decisions, then get user data and update player state/physics based on previous state and current input.

There are cases or design decisions that may require getting user input before doing some of these other tasks, so the way I would want to do it might not be practical. This whole part of the pipeline can be quite long as highly intelligent AI and immersive physics (along with other game scripting and state updates) can require massive amounts of work. At the least we have lots of sorting, branching, and necessarily serial computations to worry with.

Depending on when input is collected and the depth and breadth of the simulation, we could see input lag increase up to several milliseconds. This is highly game dependent, but it isn't something the end user has any control over outside of getting the fastest possible CPU (and this still won't likely change things in a perceivable way as there are memory and system latencies to consider and the GPU is largely the bottleneck in modern games). Some games are designed to be highly responsive and some games are designed to be highly accurate. While always having both cranked up to 11 would be great, there are trade offs to be made.

Unfortunately, that leaves us with a highly variable situation. The only way to really determine the input lag caused by game code itself is profile the code (which requires access to the source to be done right) or ask a developer. But knowing the specifics aren't as necessary as knowing that there's not much that can be done by the gamer to mitigate this issue. For the purposes of this article, we will consider game logic to typically add somewhere between 1ms and 10ms of input lag in modern games. This considers things like decoupling simulation and AI threads from rendering and having work done in parallel among other things. If everything were done linearly things would very likely take longer.

When we've got our game state updated, we then setup graphics for rendering. This will involve using our game state to update geometry and display lists on the CPU side before the GPU can start work on the next frame. The speed of this step is again dependent on the implementation and can take up a good bit of time. This will be dependent on the complexity of the scene and the number of triangles required. Again, while this is highly dependent on the game and what's going on, we can typically expect something between 1ms and 10ms for this part of the process as well if we include the time it takes to upload geometry and other data to the GPU.

Now, all the issues we've covered on this page go into making up a key element of game performance: CPU time. The total latency from front to back in this stage of a game engine creates a CPU limit on performance. When what comes after this (rendering on the GPU) takes less time than everything up to this point, we have hit the CPU limit. We can typically see the CPU limit when we drop resolution down to something ridiculously low on a high end card without seeing any real performance gain between that and the next highest resolution.

From the examples I've given here, if both the game logic and the graphics/geometry setup come in at the minimum latencies I've suggested should be typical, we could be CPU limited at as much as 500 frames per second. On the flip side, if both portions of this process push up to the 10ms level, we would never see a frame rate over 50 FPS no matter how fast the GPU rendered anything.

Obviously there is variability in games, and sometimes we see a CPU limit at less than 60 FPS even at the lowest resolution on the highest end hardware. Likewise, we can see framerates hit over 2000 FPS when drawing a static image (where game logic and display lists don't need to be updated) with a menu in front of it (like when a user hits escape in Oblivion with vsync off). And, again, multi-threaded software design on multi-core CPUs really middies up the situation. But this is near enough to illustrate the point.

And now it's on to the portion of realtime 3D graphics that typically incurs the most input lag before we leave the computer: the graphics hardware.

Reflexes and Input Generation Of the GPU and Shading
POST A COMMENT

84 Comments

View All Comments

  • RubberJohnny - Friday, July 17, 2009 - link

    OT - I used to be a diehard CRT ONLY user then i realised there is NO ghosting on modern LCD monitors...you may have seen smearing on LCD tvs but thats caused but the scaler resizing SD material to fit the panels native res.
    On monitors there is no scaling = no ghosting.
    Got a 24inch samsung 6 months ago and wish i'd done so earlier, crisper, larger image and widescreen being the main reasons i'll NEVER use a crt again.
    Reply
  • DerekWilson - Friday, July 17, 2009 - link

    I agree that with modern LCD panels ghosting is not as large an issue and color (depending on backlight) and contrast (depending on panel) are much better these days as well.

    Refresh rate is the only real outstanding issue these days (imo). And the only 120Hz display I saw was a bit over saturated / over bright and not high enough contrast.
    Reply
  • jkostans - Saturday, July 18, 2009 - link

    I have yet to see an LCD without ghosting, it may be minor but it's still annoying. And even the 120Hz LCDs supposedly still have measurable input lag regardless of the non-existent ghosting. LCDs are still a downgrade as far as I'm concerned. Reply
  • DerekWilson - Sunday, July 19, 2009 - link

    Well you can actually see how much (or little) ghosting there would be in our high speed footage... even the advertised latencies on the 3007WFP are pretty bad compared to most panels these days (especially TN panels). Despite that there were only a few cases where we could see ghosting take a whole frame (and it never seemed to take more than a whole frame).

    We should test more panels with high speed cameras and see what happens...
    Reply
  • Freeseus - Thursday, July 16, 2009 - link

    I feel like there is a HUGE section of delay missing from this article. Perhaps it was chosen specifically not to be included because rigorous testing/comparison would have to be performed in order to show any sort of suggested "average" numbers. Either way, I feel it should have been addressed... even if it were just at the end of the article.

    I'm referring to the added delay of wireless mice.

    Aside from the added delay of transmitting all the mouse action to the receiver, the biggest issue is the inconsistencies of mouse performance and precision. I'm sure there's a direct correlation between this and the battery. I'm referring specifically to the amount of battery used up in order for the mouse to broadcast continuously at extremely high intervals to insure precise movement. But obviously this includes any issue where the battery needs to be recharged as well. And on top of that, the mouse seems to be non-responsive occasionally during use. Completely unacceptable in a work or 'twitch gaming' environment.

    Anyhow, it would have been nice to see this addressed because many people make the argument that wireless mice are better. And when it comes to FPS gaming or even work, I can't think of a reason not to have a wired mouse. Do I really need to have that accuracy all the time. Yes.
    Reply
  • DerekWilson - Thursday, July 16, 2009 - link

    I agree that the current state of wireless mice is generally pretty poor ... though I've never used a wireless mouse targeted at gaming (gaming mice were the first optical mice I was able to use without mousing too quickly for the sensor even during desktop use).

    Testing wireless mice is definitely something worth looking into.
    Reply
  • Vidmar - Friday, July 17, 2009 - link

    What about PS/2 mice? Are they better or worse than USB? Reply
  • DerekWilson - Friday, July 17, 2009 - link

    PS/2 mice are slower ... iirc they come in at about 100Hz (10ms). Reply
  • Vidmar - Monday, July 20, 2009 - link

    Really?? My old PS/2 MS Wheel Optical Mouse v1.1 is currently running at 200Mhz. IE: 200 reports/second. I've never felt like it doesn't keep up in any game. Reply
  • lopri - Thursday, July 16, 2009 - link

    I'm loving this article and chewing threw every page right now. Just wanted to say thank you for such in-depth analysis as is rarely found elsewhere. Reply

Log in

Don't have an account? Sign up now