Can be found here. Although it has only been up for a few days, Ke-Sen Huang already has 72 out of 97 (74%) papers linked from his SIGGRAPH 2009 papers page. The total of 97 papers presented at SIGGRAPH this year includes 78 actual SIGGRAPH papers, and 19 Transactions on Graphics papers which were selected to be presented at the conference.
SIGGRAPH 2009 courses
No sooner had I written about the full SIGGRAPH 2009 course list not being up yet, and bam! there it is. As I hinted at, there is a lot of stuff there for real-time rendering folks. Some highlights:
Advances in Real-Time Rendering in 3D Graphics and Games (two-parter; second part here): This (somewhat awkwardly-named course) has been my favorite thing at SIGGRAPH for the past three years. Each year it presents all-new material. Previous courses have seen the debut of important rendering techniques algorithms like SSAO, signed distance-field vector texturing, and wrinkle mapping, as well as details on the rendering secrets behind games like Halo 3, Crysis, Starcraft 2, Half-Life 2, Team Fortress 2, and LittleBigPlanet. Not much is known about this year’s content, except that it will include details on Crytek’s new global illumination algorithm; but this is one course I know I’m going to!
Beyond Programmable Shading (another two-parter): GPGPU was promoted by GPU manufacturers in an attempt to find non-graphics uses for their products, and then turned full circle as people realized that drawing pretty pictures is really the best way to use massive general-purpose computing power. Between CUDA, OpenCL, and Larrabee, this has been a pretty hot topic. This is the second year that this course has been presented; last year had information on all the major APIs, and some case studies including a presentation on id’s Voxel Octree research. A subsequent SIGGRAPH Asia presentation added some new material, such as a presentation on real-time implementations of Renderman‘s REYES algorithm. This year, presenters include people from NVIDIA, AMD, Intel and Stanford; I expect this course to add significant new material, given the rapid development of the field.
Efficient Substitutes for Subdivision Surfaces: Tessellation is another hot topic; Direct3D 11 is mostly designed around this and Compute Shaders (the topic of the previous course). There has been a lot of work on mapping subdivision surfaces and other types of high-order surface representations to D3D11 hardware. Including presenters from ILM, Valve, and NVIDIA, this course promises to be a great overview of the state of the art.
Color Imaging: Color is one of the most fundamental topics in rendering. This course is presented by some of the leading researchers on color and HDR imaging, and should be well worth attending.
Advanced Material Appearance Modeling: Previous versions of this course were the basis for an excellent book on material appearance modeling. This is a great overview of an important rendering topic, and well worth attending if you haven’t seen it in previous years.
Visual Algorithms in Post-Production: It is well-known that you can find the latest academic rendering research at SIGGRAPH, but there is always a lot of material from the trenches of film effects and animation production as well. A surprisingly large percentage of this is relevant for real-time and game graphics. This course has presenters from film production as well as graphics academia describing ways in which academic research is used for film post-production. I think our community can learn a lot from the film production folks; this course is high on my list.
The Digital Emily Project: Photoreal Facial Modeling and Animation: Last year, Digital Emily was one of the most impressive technology demonstrations; it was the result of a collaboration between Paul Debevec‘s group at USC and Image Metrics, a leading facial capture company. In this course, presenters from both ICT and Image Metrics describe how this was done, as well as more recent research.
Real-Time Global Illumination for Dynamic Scenes: Real-time global illumination is another active topic of research. This course is presented by the researchers who have done some of the best (and most practical-looking) work in this area. It will be interesting to compare the techniques from this course with Crytek’s technique (presented in the “Advances in Real-Time Rendering” course).
SIGGRAPH 2009 papers
Ke-Sen Huang has had a SIGGRAPH 2009 papers page up for a while, and this weekend he’s added a bunch of new papers.
I found two of these to be of direct relevance to real-time rendering:
Gaussian KD-Trees for High-Dimensional Filtering: This paper generalizes the approach used in the SIGGRAPH 2007 bilateral grid paper. Large-scale image filters are typically performed on downscaled buffers for better performance, but this cannot normally be done for bilateral filters (which are used in real-time rendering for things like filtering SSAO). The bilateral grid is a three-dimensional low-resolution grid, where the third dimension is the image intensity (it could also be depth or some other scalar quantity). However, bilateral grids cannot be used to accelerate bilateral filters based on higher-dimensional quantities like RGB color or depth + surface normal; this paper addresses that limitation.
Modeling Human Color Perception under Extended Luminance Levels: An understanding of human color perception is fundamental to computer graphics; many rendering processes are perceptual rather than physical (such as tone mapping and color correction), but even physical computations are affected by the properties of human vision (such as the range of visible wavelengths and the fact that human color perception is trichromatic, or three-dimensional). Most computer graphics people are familiar with color spaces such as CIE XYZ, but color appearance models such as CIECAM02 are less familiar. These are used to model the effects of adaptation, background, etc. on color perception. Current color appearance models are based on perceptual experiments performed under relatively low luminance values; this paper extends the experiments to high values, up to about 17,000 candelas per square meter (white paper in noon sunlight), and propose a new color appearance model based on their findings. I also found the background and related work sections illuminating for their succinct overview of the current state of color science.
Two more papers, although not directly relevant to real-time rendering, are interesting and thought-provoking:
Single Scattering in Refractive Media with Triangle Mesh Boundaries: This paper finds a rapid (although not quite real-time) solution to refraction in objects composed of faceted or smooth triangle meshes. The methods described here are interesting and look like they could inspire some real-time techniques, perhaps on the next generation of graphics hardware.
Fabricating Microgeometry for Custom Surface Reflectance: This one is not useful for rendering, but is just plain cool. Instead of using the microfacet model to predict the appearance of surfaces based on their structure, they turn the idea around and construct surfaces so that they have a desired appearance. One of the examples they show (inspired by Figure 1 in this paper), is a material with a teapot-shaped highlight! Well, with their current fabrication methods it is really a teapot-shaped reflection cast on a wall, but once manufacturers get their hands on this, all kinds of weird and wonderful materials will start showing up.
So far the yield of papers relevant to real-time rendering practitioners is disappointingly low; perhaps more relevant papers will show up when the official list is published. In any case, the early list of courses has a lot of relevant material, and I have reason to believe the final list will have even more good stuff on it. In addition, the Talks (formerly Sketches) program always has useful stuff, Will Wright is giving a keynote speech, and the Electronic Theater (which is back, renamed as the Evening Theater) now has real-time content, so there are more than enough reasons to attend SIGGRAPH this year (and it’s in New Orleans!). Registration has already started!
CryEngine3 presentation
This detailed presentation on Crytek’s latest engine at the regional Triangle Game Conference slipped completely under my radar, but Wolfgang Engel just pointed it out to me. It’s on Crytek’s presentations page, which has a bunch of other good stuff on it as well.
The presentation includes lots of great information on their new deferred lighting system, which is timely since I am just working on a lengthy blog post on this very subject (hopefully to be finished soon). They also tease about their new dynamic global illumination system, to be presented at SIGGRAPH 2009.
Odds and Ends
It’s 5/7/09, a nice odd sequence, so time for a few odds and ends I’ve collected.
OK, this is worth a few minutes of your life: the elevated demo is awe-inspiring. Terrain generation (be patient when you start it), fly-by’s, and music, all in less than 4096 bytes. By way of comparison, an empty MS Word document is 9834 bytes. (thanks to Steve Worley)
Google has put out a browser-based low-level 3D graphics API called O3D. API here. Demos here. Some initial impressions here. It will be interesting to see if they succeed where so many others have failed.
There is a call for participation out for a new book series called “Game Engine Gems“, edited by Eric Lengyel. (thanks to Marwan Ansari)
The main thing I look at on the SIGGRAPH exhibition floor are the book booths. Good books are such a ridiculous bargain: if a book like Geometric Tools saves a programmer 2 hours of time, it’s paid for itself. One new book that I want to see is Real-Time Cameras, by Mark Haigh-Hutchinson, which came out this April. Looking around for more info, I noticed this sad note. I never met Mark, but we corresponded a few times. He came up with a clever idea to avoid performing division when doing a point in polygon test; I folded this into the CrossingsMultiplyTest Graphics Gems code here, crediting him.
I’ve been looking at GPU capabilities and benchmarking information lately. Some nice resources:
- You probably know about the benchmarking group Futuremark. Me, I hadn’t realized they had useful stats at their site: see the Futuremark ORB links at the bottom of the page and start clicking.
- Two applications that tell you a ton about your card’s capabilities: GPU-Z, with a ton of information and a statistics page & cute map of downloads at their site, and GPU Caps, which also includes CUDA-related information and some nice little OpenGL benchmarks.
- Chris Dragan has a web database that provides a fair amount of data on card support for DirectX capabilities and OpenGL extensions.
- The Notebook Check site had way too much information about many laptop graphics accelerators.
- nHancer is a utility for NVIDIA cards. It lets you get at all sorts of different capabilities on your GPU, on a per-game basis. There are also interesting antialiasing and anisotropic filtering comparison pages (click on the radio buttons). (thanks to Mauricio Vives)
Coincidental world: it turns out there’s a different “Eric Haines” out there that made a well-received 3D graphics game for the iPhone, Realmaze 3D. I’m not sure how it compares to his The Magical Flying Pink Pony Game, which looks awesome. (thanks to Nikolai Sander)
I’ve seen similar real-world illusions, but still thought this one was pretty great. (Addendum: Morgan McGuire found this even-better video of the effect.)
Computer-modeled stop motion in “Coraline”
This is a bit old, but still cool: the film Coraline (based on a great book by one of my favorite authors) was stop-motion, not CG. However, the facial animation was extremely smooth, due to a unique process. The expressions were modeled by computer, exactly as they would have been for a CG animated feature. Then for each frame, the faces were “printed” out on a “3D printer” (rapid prototyping machine). These tweened “frame faces” were swapped in for each stop-motion frame. CGSociety has an article describing the production pipeline for this film, an intriguing combination of CG and stop-motion animation.
Left-Handed vs. Right-Handed Viewing
In my previous post I talked about how I think about left-handed vs. right-handed world coordinate systems. The basic idea is simply that there is an underlying reality, and the coordinate system choice is yours.
I compared notes with Jeff Weeks, a topologist friend (who wrote this cool book The Shape of Space that is not full of math symbols, but is just the opposite – approachable and fun, and what you should read after Flatland and Sphereland). Happily, he agrees. He also introduced me to another word for handedness: chirality, coined by Lord Kelvin. Jeff notes an interesting distinction:
You can ask whether the object’s own intrinsic coordinate system agrees with the ambient world space coordinate system (same chirality) or disagrees with the ambient world space coordinate system (different chirality), but you can’t meaningfully call either one of them “left-handed” or “right-handed”.
In other words, the only time you can meaningfully introduce the words “left-handed” and “right-handed” into the discussion is when a human being enters the picture. Once an object (or a space) gets drawn into the human’s physical space, then “left-handed” means “same chirality as the human’s left thumb-index finger-middle finger” and “right-handed” means “same chirality as the human’s right thumb-index finger-middle finger”.
So in particular, data sitting in a file on disk has no intrinsic chirality. What it has is the author’s intention that it be drawn into the human’s physical space with one chirality or the other (so that, for example, the steering wheel in a car model appears on the left-hand side of the vehicle, as perceived by the human viewing it).
OK, so only for viewing you must also know the handedness of the data. We also know it’s fine to have local coordinates that are RH or LH inside a world that is LH or RH, e.g., you create a frog model by mirroring the right half of your frog model with a mirror matrix, but the world itself is one or the other. So far so good.
Where things get tricky in computer graphics is when we talk about something like a right-handed coordinate world and a left-handed viewing system. Right-handed coordinates are pretty common: all Autodesk applications I know use them. It’s a natural extension of a 2D Cartesian plane to make the Z axis point upwards – a 2D floor-plan that is extruded would be thought to extend upwards from the ground, for example. Also, the determinant of a right-handed viewing matrix is positive; left, negative.
However, a left-handed viewing system is used by default by DirectX: the viewer’s X axis goes to the right, the Y axis is up, and Z goes into the screen. This also feels natural, as you label the screen’s X and Y axes from the lower left corner of the display and Z increases with depth into the screen. OpenGL’s viewing system is right-handed by default, the difference being that +Z goes towards the viewer. This negation leads to a fair bit of confusion in documentation with what near and far mean, but it’s consistent with right-handed world coordinates.
So what if you want to use a left-handed viewing system with right-handed data, or vice versa? All it means: there must be a conversion from one to the other, or expect mirrored images. Like my first post notes, the world coordinate system chosen is arbitrary: your RH system or the mole men’s LH system are both fine, you just have to decide which one to use. However, once you choose, that’s it – the view itself ultimately has to be working in the same system, one way or another. It is honestly meaningless to say “I want to use LH viewing with RH world coordinates”, if you want to do so “without conversion”.
Some transform has to be done to go from RH to LH, but which one? Any mirroring transform can mirror through any arbitrary plane will convert from one chirality to the other. Mirroring along the world’s Z axis is one common solution, i.e., set Z’ = -Z for vertices, normals, light positions, etc. If the scene’s data is converted once to LH space, case closed, your LH camera will work fine.
However, say you don’t want to touch the data. Microsoft gives a way to convert from RH to LH, which boils down to again mirroring along the world’s Z axis as the first transform on the view matrix, i.e., done every frame. No time is lost, since the mirroring is simply part of a different view matrix. The funny bit is that you have to deal with the world in RH, the view in LH, as far as defining where the camera goes and how it is oriented. A common way to define a view is by giving a camera position, a target it is looking at, and an up direction. From this you can form a view basis, a matrix to transform from world to view. By Microsoft’s method, you have to make sure to use LH positions and vectors for forming your camera (with Z negated), vs. the RH coordinate you use for your data. Confusing.
There’s another somewhat sensible place to perform mirroring: go from RH to LH at the end of the transforms, after projection to clip space. You’re given a right-handed camera position, target, and up vector and you use these to make a left-handed view matrix – there’s nothing stopping you. None of these coordinates need to be transformed, they can be defined to be the same in RH and LH spaces. However, what will happen is that the image produced will be mirrored along the vertical axis. That is, left and right sides will be switched, since you didn’t compensate for the difference in chirality. Lots of models are symmetric, in fact, so this sort of mistake is not often immediately noticeable. By adding a simple mirror matrix with X’ = -X you can swap the left and right of the screen. This comes down to negating the upper-left value in the projection matrix.
By using this mirror matrix at the end, you’ve made your left-handed coordinate system into a right-handed one. Each time you move the camera, you’re essentially defining a new and (usually) different mirroring plane in world space, one that passes through your eye, target, and up vector. This mirror matrix will then not affect the eye, target, and up directions, since they lie in this plane. Maybe that’s fine for your system. However, this solution can also mess up basic operations and queries. Say you want to pan the camera to the left. So you query your camera for which direction it thinks is left. It hands you one, you move the camera that direction, but the view shifts the other way. This is because you have a mismatch in chirality: your camera’s basis is in LH, but you correct it at the back-end to be RH. The view matrix returned a left-vector which was also in LH, and needs to be converted to RH. Also confusing.
The whole point of the camera is to transform some chunk of visible space into clip coordinates, which then convert (by dividing by W) to NDC coordinates (visible space being -1 to +1 in X and Y, -1 to +1 in Z for OpenGL, 0 to +1 in Z for DirectX). Neither OpenGL nor DirectX in any way requires LH or RH coordinates, you can feed in whatever view transforms you like. Just make world and view chirality match and be done with it.
That’s about it – just make them match, and do so by setting the view (lookat) matrix to match the world’s chirality. At least, that’s my take, this week (which sounds flip, but I’ve been trying to get a good handle on this area for a long time; I like to think I now have one, as of today). If there are other ways to think about this area, and especially if I’ve made any mental errors, please do comment.
By the way, if you want an enjoyable book about handedness and symmetry, get Martin Gardner’s The New Ambidextrous Universe. It talks about some surprising stuff, such as experiments where parity in the universe is not conserved (so if you’re ever mirrored a few times through the fourth dimension on your way to a distant planet, you’ll know how to test which way you came out).
Oh, and also check out Jeff Weeks’ site – some fun and beautiful OpenGL math applications there for free, Mac and Windows, with source. I should note that he uses LH throughout for his OpenGL applications, the rebel. There’s no “correct” coordinate system, so choose your own.
Eurographics Workshop on Natural Phenomena 2009
EWNP has had interesting papers in recent years, but when it skipped 2008 I thought it was gone. However it came back in 2009 with five papers, all of which are online except for one:
Procedural Modeling of Leather Texture with Structural Elements: Not currently available online, but judging from a previous paper by these authors this appears to be about procedural modeling of the cracks and bumps in leather surfaces. Most real-time applications will use photographed or manually created textures for this, so it is probably not of wide interest to real-time developers.
Interactive Modeling of Virtual Ecosystems: Automatic modeling of plants taking lighting, obstacles, etc. into account. Might be useful as an automatic modeling tool.
A Geometric Algorithm for Snow Distribution in Virtual Scenes: What the title says; might be useful for automated scene modeling, but probably not for runtime use.
Corotated SPH for Deformable Solids: Smoothed Particle Hydrodynamics (SPH) is commonly used in film production for liquids, smoke, etc. This paper discusses how to extend the technique to model deformable solids. Probably not real-time anytime soon.
Real-Time Open Water Environments with Interacting Objects: This combines the Tessendorf FFT-based method for ambient waves with a different method for interactive waves (waves interacting with dynamic objects). This is the most relevant paper for real-time rendering; worth a read.
Tessendorf’s FFT method is the current gold standard for non-interactive ocean waves, and is widely used in game and film production. A description of it can be found on his publication page, under Simulating Ocean Surface. Tessendorf’s publication page has many more papers of interest, including an algorithm (called iWave) for interactive waves and reports on particle and volume rendering for film production.
Insomniac have a particularly efficient and flexible implementation of a variant of Tessendorf’s method, which they extended to support interactive waves as well. This method was used in the game Resistance 2, and Insomniac Games have kindly published not just a white paper on the technique, but actual working code! This is part of their admirable Noctural Initiative for technology sharing. The Noctural Initiative website is highly recommended, as it includes code which has been used in successful game projects by one of the most highly-regarded studios in the industry.
Another interesting approach to interactive waves is Wave Particles, which is described here.
AMD/ATI GDC 2009 slides available
At their Developer Conference Presentations website. The talk titled Your Game Needs Direct3D 11, So Get Started Now! is particularly interesting as an overview of how to port code to D3D11.
Graphics Interface 2009 papers
The list of papers accepted to Graphics Interface 2009 (with abstracts) is now online. Graphics Interface has had some pretty good real-time rendering papers: here is a handful of examples from the last few years. Judging from this year’s abstracts, the following look particularly interesting:
Fast Visualization of Complex 3D Models Using Displacement Mapping: This looks like a combination of the “sparse voxel ray casting” approach popularized by id software with “relief mapping” approaches.
Depth of Field Postprocessing for Layered Scenes Using Constant-Time Rectangle Spreading: This is closely related to one of my favorite I3D 2009 posters, “Faster Filter Spreading and Its Applications”. The basic idea (which has also been discussed in this paper by Dan Piponi) is to “splat” rectangles in constant time (independent of the rectangle size!) by “splatting” just the corners into a buffer, from which a summed-area table is constructed (using existing fast methods), yielding the desired image. This can be extended to more general splats. Although there is no preprint yet, the tech report is available.
An Analytical Approach to Single Scattering for Anisotropic Media and Light Distributions: A follow-on paper to one published in Eurographics 2009, it adds anisotropic phase functions and more general lighting. The basic solution is similar to an earlier paper by Bo Sun and others, but using a slightly different approach that enables increased precision.
Rendering the Effect of Labradorescence: This is of interest to me as an optical reflectance geek, but I doubt anyone will be using it in a game anytime soon. This paper presents a physically-based method of rendering a complex optical phenomena that exists in gems such as Labradorite and Spectrolite.
Ke-Sen Huang’s Graphics Interface 2009 page should be a good place to hunt for preprints of these papers as they appear.