Exploiting temporal and spatial coherence

Exploitation of temporal and spatial coherence is among the most powerful tools available to a graphics programmer. Several recent papers explore this area. Accelerating Real-Time Shading with Reverse Reprojection Caching (GH 2007, available here) uses reverse reprojection to reuse values cached from previous frames. An Improved Shading Cache for Modern GPUs (GH 2008, available here) analyzes the performance characteristics of this technique and proposes some efficiency improvements.

Such caching schemes involve analyzing each pixel shader to find appropriate values to cache. Care must be taken to use values which are expensive to compute but have low directional dependence. Automated Reprojection-Based Pixel Shader Optimization (to be published at SIGGRAPH Asia 2008, available here) proposes a method to automate this process. Another option is to apply reprojection caching to a specific, well-defined case like shadow mapping. This is discussed in Pixel-Correct Shadow Maps with Temporal Reprojection and Shadow-Test Confidence (EGSR 2007, paper web page). This paper was also mentioned in our book.

Personally I’m a bit skeptical of reprojection caching techniques, since whenever the view changes abruptly the cache will be completely invalidated resulting in performance dips. Many applications can’t use acceleration techniques which don’t help worst-case performance. Applications with restricted camera motion may certainly benefit. Enhancing these techniques with fallbacks which degrade quality (instead of performance) in cases of abrupt camera motion may make them more generally applicable.

A different approach is discussed in Geometry-Aware Framebuffer Level of Detail (EGSR 2008, available here). Here the idea is to render certain quantities into a lower resolution frame buffer, using a joint bilateral filter to resample them during final rendering. As with the previous technique, care must be taken in selecting intermediate values; they should be both expensive to compute and vary slowly over the screen. This powerful acceleration technique was also used in the paper Image-Based Proxy Accumulation for Real-Time Soft Global Illumination (PG 2007, available here). Variations of this technique have been used in games, with perhaps the most common case being particle rendering, as the Level of Detail blog points out in this interesting post on the subject. The same blog also has insightful posts on many of the papers mentioned here, as well as another related paper.

This and That

I’ll someday run out of titles for these occasional summaries of new(ish) resources, but in the meantime, this one’s “This and That”.

Christer Ericson’s article on dealing with grouping and sorting objects for rendering is excellent. It mostly depends on input latency, but has concepts that can be applied in immediate mode.

An element that continues to renew the field of computer graphics is that the rules change. This article is about taking Quake 2 (from 1997) and moving it to a modern GPU.

If you haven’t seen it yet, Farbrausch’s demo “debris” is truly impressive. It’s only 183,462 bytes, and is absolutely packed with procedural content. Download here (last link works). Or be lazy and watch on YouTube.

NVIDIA’s pulled together its resources for shadow generation and ambient occlusion all onto one handy page (plus ray tracing – just one entry so far, but it’s a good one).

How to deal with various rendering paradigms on multiple platforms? GRAMPS looks intriguing.

Gamasutra put a useful Game Developer article online, all about commercial middleware game engines currently available.

OpenGL will always exist, since Macs and Linux need it. It’s easier to use in college courses because of its clarity and readability. But otherwise the pendulum’s swung far towards DirectX. Phil Taylor comments on and gives some historical context to the controversy around the latest release, OpenGL 3.0.

A nice trend for OpenGL is that people continue to write useful bits, such as GLee, which manages extensions.

New info on older effects: blur and glow, volumetric clouds, and particle systems.

The glorious teapot. I like “a wireframe view”. Yes, the real thing is taller than the synthetic model, as the model makers were compensating for non-square pixels.

“What’s the future hold?” is always a fun topic, one we’ve used each edition to end our book. I liked this presentation on SlideShare for its sheer “here are a hundred things that hurtle us towards the Singularity” feel, though I don’t buy it for a minute. SlideShare, where it is hosted, is a pleasant medium-attention-span kind of place, with all sorts of random and fun slidesets.

Finally, I am pleased to find that LittleBIGPlanet is just as gorgeous as it looked like it would be. I’ve played myself for only a bit, but walking by when my kids are playing I find I have to stop and stare.

Ray Tracing News v. 21 n. 1 is out

I’ve put out the Ray Tracing News for more than 20 years now. New issues come maybe once a year, but there you have it. There’s a little overlap with this blog, but not that much. Find the latest issue here. Now that I’m finally done with this issue I can imagine blogging again (it wasn’t just I3D that was holding me back).

Best Conference Ever

I’ve been busy with papers cochairing for I3D 2009 (he said casually, knowing he’ll probably never get an opportunity to do so again, being a working stiff and not an academic), but hope to get back to blogging soon. In the meantime, here’s the best conference ever: “Foundations of Digital Games“. I3D’s the end of February in Boston, vs. April on a cruise ship between Florida and the Bahamas. Why don’t I get invited to help out at conferences like this?

Corrigenda

“Corrigenda” is a classy publisher’s word for “bugs,” but it also means listing fixes for these errors. Morgan McGuire and his students have been reading our book closely, and have found the first two significant errors in the 3rd edition. These errors and their corrections can be found on our corrigenda page.

Donald Knuth sends checks for $2.56 for each error found in his classic (but still being written) series “The Art of Computer Programming”; Sir James Murray, the editor of the first Oxford English Dictionary, was perhaps the first to reward readers in this way. Knuth has an even more lucrative/costly reward doubling scheme for errors found in his software, with the prize now locked at $327.68.

Tomas offered his students a piece of candy for each error they found in our second edition. I like the idea of rewarding readers in some way, beyond naming them on the page. We’ll think of something; suggestions? More important, have you found any bugs, large or small? Please do pass them on, as it helps everyone.

Interesting bits

I’ve been collecting links via del.icio.us of things for the blog. Let’s go:

Antialiasing thick lines by using textures is an old technique. Areakkusu’s site is nice in that it has good examples and code.

The Level of Detail blog has a great pointer to Slisesix’s amazing demo. “Demo” as in “Demoscene,” where his program is a mere 4k bytes in size. It’s not animated, not real-time, but shows how distance fields could be used for ambient occlusion approximation. Definitely check out all the links: Alex Evan (of LittleBIGPlanet) has a worthwhile talk, and Iñigo’s presentation is even better: good technical content and real-time programs running inside the slides.

I’d rather avoid logrolling in this blog, but did want to mention enjoying Christer Ericson’s post on graphical shader systems. I have to agree that such systems are bad for creating efficient shaders, but these tools do at least allow a wider range of people to experiment and explore. There are a lot of worthwhile followup comments on this thread.

Oogst has a clever trick he calls interior mapping, for rendering walls, floors, and ceilings for buildings seen from the outside. Define a texture to be used for each interior element, and have the pixel shader compute from the eye direction what would be seen inside. There’s no actual geometry, it’s all just computing the ray intersection using (wait for it) a floor function. Humus has demo code available for this technique, using DirectX 10. Admittedly, the various tiles repeat and there are other limits, but actual interiors are vastly superior to the usual dirty or reflective windows currently used in games, with no extra geometry added.

Bavoil and Sainz have a new approach for Screen-Space Ambient Occlusion, using a more elaborate form of horizon mapping: http://developer.nvidia.com/object/siggraph-2008-HBAO.html. Code’s available in NVIDIA’s DX 10 SDK.

If you missed Jon Olick’s talk at SIGGRAPH about voxel octree representation, Timothy Farrar has a summary. Personally, I think Jon’s research is very much that-research, not something that is immediately practical-but I love seeing how changing capabilities and increased flexibility can lead to different approaches.

On Amazon: 4 graphics books for the price of 2, minus the papery bits. Pharr and Humphrey’s “Physically Based Rendering” (PBR) and Luebke’s “Level of Detail for 3D Graphics” are certainly worthwhile, the other two I don’t know about (though look worthwhile and are well-rated). I don’t know a thing about the electronic media used; I’m guessing the books are DRM’ed, not naked PDFs. Searchable is certainly nice. While it’s too bad you can’t just buy the ones you want (I smell a marketing department having some “what can we get them to pay for what bundle?” meetings, given the negligible physical cost), I did notice an interesting thing on Amazon I hadn’t seen before for each book except PBR: “Upgrade this book for $18.39 more, and you can read, search, and annotate every page online.” You can also upgrade books you’ve previously purchased on Amazon.

On Gamasutra, an article summarizing DirectX 11. I liked it: to the point, and with some useful figures.

Every once in awhile someone will say he has a new graphics rendering method that’s awesome, but won’t explain it because of some reason (usually involving money or fame). Here’s one, from Sunfish Studio: no micropolygons, no point sampling. OK, so that leaves-what?-voxels? If anyone knows what this is about, please comment; I’m curious.

GameDeveloperTools.com is a new site that tracks news and has users rate books. To be honest, a lot more voting needs to happen to make the ratings useful-I’d stick with Amazon for now. The main use is that you can look at specific categories, which are a bit better than Amazon’s somewhat random sorting of graphics books (e.g., our book is in three categories on Amazon, competing against artists’ books on using mental ray and RenderMan).

Finally, this, well, this is not interactive graphics, but is just so cool: parking signs understandable from only certain locations.

Latency

Herb Sutter’s site has some interesting material about CPU architectures. His article “The Free Lunch is Over” is a bit dated-everyone should know by now that multicore is upon us-but does a good job pounding home that concurrency is the way of the future (i.e., like, now). It also has some memorable lines, like, “Cache is King,” and “Andy Giveth, and Bill Taketh Away.” I contemplate the latter every time I open up a Word document and it takes 25 seconds to appear.

What I noticed today, due to Eric Preisz’s new indexbuffer site, was that Herb has a newer presentation available, “Machine Architecture: Things Your Programming Language Never Told You.” This covers in-depth the topic of latency and how the CPU attempts to hide it. It’s worth a look if you’re at all interested in the topic; there’s material here that I hadn’t seen presented before. I skimmed over the odd things that compilers might do to code, I must admit, but overall I found it worthwhile.

Face and Skin Papers at SIGGRAPH Asia 2008

Ke-Sen Huang has recently added three papers relating to human face and skin rendering to his excellent list of SIGGRAPH Asia 2008 papers. Human faces are among the hardest objects to render realistically, since people are used to examining faces very closely.

The first two papers focus on modeling the effect of human skin layers on reflectance. The authors of the first paper, “Practical Modeling and Acquisition of Layered Facial Reflectance” work in Paul Debevec’s group at the USC Institue for Creative Technologies, which has done a lot of influential work on acquisition of reflectance from human faces (the results of which are now being offered as a commercial product). Previous work focused on polarization to separate reflectance into specular and diffuse. Here diffuse is further separated into single scattering, shallow multiple scattering, and deep multiple scattering (using structured light). Specular and diffuse albedo are captured per-pixel. Unfortunately specular roughness (lobe width) is only captured for each of several regions and not per-pixel, but since normals are captured at very high resolution they could presumably be used to generate per-pixel roughness values which could be useful in rendering at lower resolutions, as we discuss in Section 7.8.1 of Real-Time Rendering. The scattering model is based on the dipole approximation of subsurface scattering introduced by Henrik Wann Jensen and others. NVIDIA have shown real-time rendering of such models using multiple texture-space diffusion passes.

The authors of the second paper, “A Layered, Heterogeneous Reflectance Model for Acquiring and Rendering Human Skin” have also written several important papers on skin reflectance, focused more on simulating physical processes from first principles. They model human skin as a collection of heterogeneous scattering layers separated by infinitesimally thin heterogeneous absorbing layers. They design their model for efficient GPU evaluation, similar to NVIDIA’s approach mentioned above (one of this paper’s authors also worked on the NVIDIA skin demo). “Efficient” here is a relative term, since their model is too complex to be real-time on current hardware, and as presented is probably too complicated for game use. However, ideas gleaned from this paper are likely to be useful for skin rendering in games. The authors also present a protocol for measuring the parameters of their model

The third paper, “Facial Performance Synthesis Using Deformation-Driven Polynomial Displacement Maps” is also from Debevec’s USC group and focuses on animation rather than reflectance. They use the same facial capturing setup, but with different software to capture animated facial deformations instead of reflectance (this too has been turned into a commercial product). This paper is interesting because it extends previous coarse / fine deformation approaches to multiple scales, and uses a novel method to relate the different scales to each other. They use a polynomial displacement map, which uses the same form as Polynomial Texture Mapping (an interesting technique in its own right) but for deformation rather than shading. This method also bears some resemblance to the wrinkle map approach used by AMD for their Ruby Whiteout demo, which they presented at SIGGRAPH 2007.