Articles by Naty

You are currently browsing Naty’s articles.

Since it’s quite a long time after GDC 2011 and I never found the time to do a proper conference report, I thought I’d at least do a link roundup.

First, the GDC Vault has a Free Section with many presentation slides. Video for most talks is behind a paywall, but several notable talks have freely available video:

There are several other talks with free video, mostly sponsored by companies such as Intel and NVIDIA.

The rest of this post will cover talks where the authors or their companies have made materials available outside the Vault – I haven’t checked, but I suspect there is a fair bit of overlap with the free section in the Vault.

Presentations from the “Advanced Visual Effects with DirectX 11” tutorial day:

These are hosted on the AMD conference presentations page, which also has a few other AMD presentations:

Presentations from the “Physics for Game Programmers” tutorial day (these are only some of the presentations, it looks like the rest are up on the Vault’s free section):

Presentations from the Technical Artist Bootcamp can be found here – links to individual presentations follow:

DICE had quite a few presentations at GDC, many related to real-time rendering. All DICE presentations can be found on their publications page.

One DICE presentation of particular interest, Approximating Translucency for a Fast, Cheap and Convincing Subsurface Scattering Look (Colin Barre-Brisebois) can also be found on the author’s blog, along with an addendum.

NVIDIA also had a good number of presentations, to be found on their GDC 2011 page. An especially notable one (jointly presented with covered the Epic “Samaritan” demo, intended both to show off the Unreal Engine’s DX11 feature set, and to set a quality bar for the makers of next-generation consoles (the demo was shown on a machine with three GeForce GTX 580 cards connected via SLI, so definitely a “futuristic” system). Online material for the Samaritan demo includes the presentation slides, video of the demo, and some additional technical details on the underlying technology.

Intel also has a dedicated web page for their GDC 2011 presentations. Two especially interesting ones covered Order-Independent Transparency and Dynamic Resolution Rendering. Additional organizations with multiple talks at GDC included AutoDesk and Khronos.

The talk Mega Meshes: Modeling, Rendering and Lighting a World Made of 100 Billion Polygons (Ben Sugden & Michal Iwanicki, Lionhead) presented some unique rendering technology they developed for the (cancelled) game, Milo & Kate. Additional online materials include a video and a second video.

Other rendering talks with online materials include Anti-Aliasing From a Different Perspective (Dmitry Andreev, LucasArts), Practical Occlusion Culling on PS3 (Will Vale, Second Intention), Normal Offset Shadows (poster by Daniel Holbert, High Moon), HTML5 and Other Modern Browser Game Tech (Vincent Scheib, Google), and several presentations that Wolfgang Engel (Confetti) gave at the Intel booth.

Two animation talks also have online materials: The Animation of Halo: Reach: Raising the Bar (Joe Spataro & Tam Armstrong, Bungie), and An Automated Pipeline for Generating Run-Time Rigs (Adam Mechtley, Candlelight Interactive), as well as three non-graphics engineering talks: Forensic Debugging: How To Autopsy, Repair, and Reanimate a Release-built Game (Elan Ruskin, Valve), I Shot You First! Gameplay Networking in Halo: Reach (David Aldridge, Bungie) (there is also a much smaller file without video), and Message Queuing on a Large Scale (Jon Watte, IMVU).

For the sake of completeness, I’ll also list the design, production, and business presentations and videos I found online:

If you find any other presentations online, please put a link in the comments to this post.

Tags: ,

There  have been a lot of awards recently of interest to readers of this blog; I thought it would be useful to provide an overview, as well as covering some of the more obscure awards.

The Oscars are by far the most well-known awards, bestowed annually since 1929 by the Academy of Motion Pictures Arts and Sciences. Oscars are voted on by Academy voting members (who total about 6,000) in the relevant disciplines (e.g. directors vote for Best Director, actors for Best Actor, etc.). The Scientific and Technical Awards are especially notable; they are given in a separate ceremony in early February, two weeks before the main awards ceremony celeb-fest.

Although some of the Sci-Tech awards are still for “analog” stuff like camera mounts, lenses, and film emulsions, in recent times most of the honored developments have been digital. All except three of this year’s winners were for digital advances (the other three were for computer-controlled camera and prop cable suspension systems, so partially digital as well).

The most directly relevant award was for a development described in a SIGGRAPH paper (the 2004 paper, “An Approximate Global Illumination System for Computer Generated Films”, was even mentioned in the award text). The award was given to Eric Tabellion and Arnauld Lamorlette, “for the creation of a computer graphics bounce lighting methodology that is practical at feature film scale”. This technique (as described in the 2004 paper) is a fast one-bounce GI method that uses interesting approximations for both geometry and surface material. The paper is well worth reading; the technique was highly influential for film rendering and some of the ideas are relevant for real-time rendering as well.

Another computer graphics-related award was given to Dr. Mark Sagar “for his early and continuing development of influential facial motion retargeting solutions”. Dr. Sagar pioneered the use of the Facial Animation Coding System (FACS) for film production, starting with Monster House and supervising its use on King Kong and Avatar; this system is now widely used, with growing adoption by the game industry as well. Dr. Sagar also won an Academy Sci-Tech award last year, for his work on Light Stage.

As seen in the cable suspension case, Academy Sci-Tech awards tend to come in “clumps”. As a particular technology area is recognized as important by the Academy, several different groups who did important work in that area receive awards in one year. For example, a bunch of last year’s Sci-Tech awards were related to the digital intermediate (DI) process. The biggest “clump” this year was for another graphics-related topic: render queues (software used to manage render farms, the earliest  – and still most widespread in film production – form of parallel graphics processing):

The final computer graphics-related SciTech award was given to Tony Clark, Alan Rogers, Neil Wilson and Rory McGregor “for the software design and continued development of cineSync, a tool for remote collaboration and review of visual effects” (cineSync is developed and sold by Rising Sun Research).

Some of the main Academy Awards (announced in late February) are also of interest to readers of this blog; there’s a lot of information about these awards out there so I’ll just mention the winners for Visual Effects (Inception), Animated Feature Film (Toy Story 3), and Animated Short Film (The Lost Thing).

The closest video game equivalent to the Oscars are the Interactive Achievement Awards, bestowed annually by the Academy of Interactive Arts and Sciences at the annual D.I.C.E. Summit in early February. Similarly to the Oscars, they are voted for by registered AIAS members, who must be working in the appropriate game development discipline to vote on a given award. This year’s awards of interest to readers of this blog include: Outstanding Achievement in Animation (God of War III), Outstanding Achievement in Art Direction (Red Dead Redemption), and Outstanding Achievement in Visual Engineering (Heavy Rain).

The Game Developers Choice Awards are also prestigious, and are bestowed at the annual Game Developers Conference (which takes place in late February or early March). One must be a registered member of the Gamasutra website (owned by United Business Media, which also owns the Game Developers Conference) to nominate or vote, and the advisory committee which oversees the process is chosen by the editors of Game Developer Magazine (also owned by United Business Media) and Gamasutra. The Game Developers Choice Awards are unusual in thus being managed by a for-profit corporation rather than a nonprofit professional organization. This year’s awards of interest: Best Technology (Red Dead Redemption), and Best Visual Arts (Limbo).

Regarding video game awards, one notable event that happened this year (on February 12th) was the first Grammy award for music composed for a video game. The Grammys don’t have a dedicated award for video game music – this award was for a song, Baba Yetu, originally composed by Christopher Tin for Civilization IV and released on the 2009 album Calling All Dawns (which itself won a Grammy in addition to the award for Baba Yetu).

The awards given by the British Academy of Film and Television Arts (BAFTA) are almost as well-known in the UK as the Oscars are in the US. BAFTA gives awards for TV shows and video games as well as movies.

The British Academy Film Awards were held in mid-January. Awards of interest: Animated Film (Toy Story 3), Short Animation (The Eagleman Stag – which was stop-motion, not CG), and Special Visual Effects (Inception).

One of the British Academy Television Craft Awards was of interest: Visual Effects (The Day of the Triffids).

The British Academy Video Game Awards were held in mid-March. Awards of interest: Artistic Achievement (God of War III), and Technical Innovation (Heavy Rain). A minor controversy erupted after Red Dead Redemption did not win any awards – it turns out that it was not entered by the developers (Rockstar Games), most likely for reasons related to a perceived snub that Grand Theft Auto IV (also developed by Rockstar) received in the 2009 awards.

The last set of awards I will discuss are perhaps the most directly relevant for this blog, though not as well-known as the ones previously mentioned. The Visual Effects Society (VES) is a professional organization representing practitioners in visual effects and computer-generated animation for TV, film and video games. Among their other activities, they host the VES Awards every year in early February. Due to these awards’ focus, most of them are of interest – the full list can be found here. I’ll highlight some of the most interesting awards categories, but first I wanted to mention this year’s VES Lifetime Achievement Award recipient, Ray Harryhausen. Harryhausen is a giant in the field; his pioneering stop-motion effects work on many films, from Mighty Joe Young (1949) to Clash of the Titans (1981) inspired many of today’s most prominent filmmakers. I’ve been going through his films on recent weekends with the wife and kids; most of them are great fun and well worth seeing. I’m not sure why it took the VES nine years to recognize Harryhausen (even the Academy of Motion Pictures Arts and Sciences, which snubbed him for the special effects Oscars throughout his career, finally awarded him the Gordon E. Sawyer Award in 1992).

This year’s VES video game award winners: Outstanding Real-Time Visual Effects in a Video Game (Halo: Reach; presented to Marcus Lehto, Joseph Tung, Stephen Scott, and CJ Cowan from Bungie – two clips related to the submission are available on YouTube: “work to be considered” clip, “before and after” clip), Outstanding Animated Character in a Video Game (StarCraft II – Sarah Kerrigan; presented to Fausto De Martini, Xin Wang, Glenn Ramos, and Scott Lange from Blizzard), Outstanding Visual Effects in a Video Game Trailer (World of Warcraft – for the Cataclysm cinematic; presented to Marc Messenger and Phillip Hillenbrand, Jr. from Blizzard).

Notable VES feature film-related awards: VES Visionary Award (Christopher Nolan), Outstanding Visual Effects in a Visual-Effects Driven Feature Motion Picture (Inception), Outstanding Supporting Visual Effects in a Feature Motion Picture (Hereafter), Outstanding Animation in an Animated Feature Motion Picture (How to Train Your Dragon), Outstanding Achievement in an Animated Short (Day & Night), Outstanding Animated Character in a Live Action Feature Motion Picture (Harry Potter and the Deathly Hallows: Part 1 – Dobby), Outstanding Animated Character in an Animated Feature Motion Picture (How to Train Your Dragon – Toothless), Outstanding Effects Animation in an Animated Feature Motion Picture (How to Train Your Dragon).

Recognition of exceptional work is an important part of the advancement of any professional field; it’s good to see that the field of computer graphics is so well-covered in this respect.

Tags: , , , , , ,

A partial, early list of SIGGRAPH 2011 courses has recently been published. SIGGRAPH has published such preliminary lists in previous years, typically representing around half to a third of the final course list.

The list includes six very promising courses:

  1. Advances in Real-Time Rendering in Games: Part I – this is the next iteration in a course series, organized by Natalya Tatarchuk, that has been presented at SIGGRAPH every year (with new content) since 2006. This course has been a highlight of every SIGGRAPH it has appeared in and I’m pleased to see it coming back. The instructors are not yet listed, but Natasha has always been able to round up a top-notch speaker roster, and I am confident she will do so again this year. “Advances…” has always been a full-day course, though since 2008 (when SIGGRAPH canceled the full-day course format) it’s been divided into two half-day courses. Only one of the two halves appears on this list; hopefully this is a simple oversight and SIGGRAPH didn’t reject the other half of the course!
  2. Character Rigging, Deformations, and Simulations in Film and Game Production – I’m always happy to see “X in film and games”-types courses. If well-organized and presented, such courses detail the current cutting-edge of actual production practice in both industries, emphasizing interesting differences and commonalities between the two. Such crossover content is an important feature of SIGGRAPH not found in industry-specific conferences like GDC. The topic is important; many games don’t put enough of an emphasis on animation quality. The speaker list is strong, including Tim McLaughlin (a graphics researcher at Texas A&M University who also has a nice body of film VFX work he did at ILM), Larry Cutler (a character technical director at Dreamworks Animation, formerly at Pixar), and David Coleman (a Senior CG Supervisor at Electronic Arts Canada, where he leads the EA Sports rigging team).
  3. Cinematography: The Visuals & the Story – I’m very happy to see this course on the list. I have become  increasingly fascinated with cinematography over the last few years; there is a lot that video games can learn from cinematography, from creative topics like lighting and composition to technical ones such as depth of field and tone mapping. This course is taught by Bruce Block, a film producer and visual consultant who wrote a very well-regarded and influential book called The Visual Story, about how visual structure is used to present story in film. I’m trying to get a course put together for next year which would cover the topic from a different angle, as presented by working film cinematographers; the two courses should make a nicely complementary pair.
  4. Destruction and Dynamics for Film and Game Production – Another “X in film and games” course on a key topic, organized by Erwin Coumans (AMD; formerly at SCEA R&D, Havok and Guerrilla Games). Erwin is the creator of the open-source Bullet Physics engine, which has been used in many films and games. Other speakers include Takahiro Harada (a GPU physics researcher at AMD, formerly Havok and the University of Tokyo), Nafees Bin Zafar (a senior production engineer at DreamWorks Animation who won an Academy Scientific & Engineering Award for his fluid simulation work at Digital Domain), Mark Carlson (an FX R&D programmer at DreamWorks Animation, formerly at Disney Animation), Brice Criswell (a senior software engineer at ILM), Michael Baker (no affiliation listed – I’m guessing it’s the Michael Baker who teaches at the Art Institute of Las Vegas and develops tools for the Dynamica Bullet Maya plugin), and Erin Catto (a principal software engineer at Blizzard who also developed the very widely used Box2D open source 2D physics engine).
  5. PhysBAM: Physically Based Simulation – Another physics course, but with a different emphasis. It focuses on the the PhysBAM simulation library developed at Stanford University and used by ILM, Disney Animation, and Pixar. Parts of PhysBAM are already open source – since the course webpage refers to “the soon-to-be-released simulation library PhysBAM”, presumably the rest will be available soon. The course is presented by Craig Schroeder (a PhD student at Stanford).
  6. Storytelling With Color – Anyone who saw my color course last year knows that I believe that getting the technical side of color right is important, for both film and games. But the reason it is important comes from the creative side – the way that a selection of colors can drive story or establish a mood. This course covers that topic, and should be of great interest to many game developers. It will be presented by Kathy Altieri (a production designer at DreamWorks Animation who worked on filmsm including The Prince of Egypt, Over the Hedge, and How to Train Your Dragon, and previously at Disney Animation on The Little Mermaid, Aladdin, and The Lion King).

If the rest of the content will be nearly as good as this preliminary set of courses appears to be, SIGGRAPH 2011 will be a conference to remember!

Tags: ,

The call for submissions for SIGGRAPH Asia 2011 has recently gone live. This fourth iteration of the SIGGRAPH Asia conference will take place in Hong Kong between December 12th and 15th. In previous years, the sketches and course programs have been of similar quality (if reduced quantity) compared to their North American counterparts. The SIGGRAPH Asia Technical Papers have been really good, better in my opinion than the relatively abstruse SIGGRAPH Technical Papers. If you want to see for yourself, the incomparable Ke-Sen Huang has your back, with paper link pages for SIGGRAPH Asia 2008, 2009 and 2010. Ke-Sen deserves an outstanding service award from ACM, instead of the more negative attentions he has received from them.

Here is the 2011 CFS text (a slightly more detailed version can be found here):

SIGGRAPH Asia 2011 sees the return of the Art Gallery and Emerging Technologies programs. Also calling for submissions are: Computer Animation Festival, Courses, Technical Papers, Technical Sketches & Posters.

Submit your research, theories, and innovations and you might be the next to have the valuable opportunity to present your work to audience-packed halls at SIGGRAPH Asia 2011 conference in Hong Kong this December.

For more information on SIGGRAPH Asia 2011, please visit

Tags: ,

GDC has put a bit of a hiatus in my I3D posts; I better get them done soon so I can move onto the GDC posts.

This post describes a talk that David Luebke (Director of Research at NVIDIA) gave during the I3D banquet titled GPU Computing: Past, Present, and Future. Although the slides for the I3D talk are not available, parts of this talk appear to be similar to one David gave a few months ago, which does have video and slides available.

I’ll summarize the talk here; anyone interested in more detail can view the materials linked above.

The first part of the talk (which isn’t in the earlier version) covered the “New Moore’s Law”: computers no longer get faster, just wider; must re-think algorithms to be parallel. David showed examples of several scientists which got profound speedups – from days to minutes. He covered several different techniques, I’ll summarize the most notable four:

  1. A “photonic fence” that zaps mosquitoes with lasers, to reduce the incidence of malaria in third world countries. This application needs fast computer vision combined with low power consumption, which was achieved by using GPUs.
  2. A military vehicle which detects Improvised Explosive Devices (IEDs) using computer vision techniques. The speedup afforded by using GPUs enables the vehicle to drive much faster (an obvious advantage when surrounded by hostile insurgents) while still reliably detecting IEDs.
  3. A method for processing CT scans that enables much reduced radiation exposure for the patient. When running on CPUs, the algorithm was impractically slow; GPUs enabled it to run fast enough to be used in practice.
  4. A motion compensation technique that enables surgery on a beating heart. The video of the heart is motion-compensated to appear static to the surgeon, who operates through a surgical robot that translates the surgeon’s motions into the moving frame of the heart.

David started the next part of the talk (which is very similar to the earlier version linked above)  by going over the heritage of GPU computing. He did so by going over three separate historical threads: graphics hardware, supercomputing, and finally GPU Computing.

The “history of graphics hardware” section started with a brief mention of a different kind of hardware: Dürer‘s perspective machine. The history of electronic graphics hardware started with Ivan Sutherland’s SketchPad and continues through the development of the graphics pipeline by SGI: Geometry Engine (1982), RealityEngine (1993), and InfiniteReality (1997). In the early days, the graphics pipeline was an actual description of the physical hardware structure: each stage was a separate chip or board, with the data flow fixed by the routing of wires between them. Currently, the graphics pipeline is an abstraction; the stages are different threads running on a shared pool of cores, as seen in modern GPU designs such as the GeForce 8, GT200, and Fermi.

The second historical thread was the development of supercomputers. David covered the early development of three ways to build a parallel machine: SIMD (Goddard MPP, Maspar MP-1, Thinking Machines CM-1 and CM-2), hardware multithreading (Tera MTA) and symmetric multiprocessing (SGI Challenge, Sun Enterprise) before returning to Fermi as an example of a design that combines all three.

“GPU computing 1.0” was the use (or abuse) of graphics pipelines and APIs to do general-purpose computing, culminating with BrookGPU. CUDA ushered in “GPU computing 2.0” with an API designed for that purpose. The hardware supported branching and looping, and hid thread divergence from the programmer. David claimed that now GPU computing is in a “3.0” stage, supported by a full ecosystem (multiple APIs, languages, algorithms, tools, IDEs, production lines, etc.). David estimated that there are about 100,000 active GPU compute developers in the world. Currently CUDA includes features such as “GPU Direct” (direct GPU-to-GPU transfer via a unified address space), full C++ support, and a template library.

The “future” part of the talk discussed the workloads that will drive future GPUs. Besides current graphics and high performance computing workloads, David believes a new type of workload, which he calls computational graphics, will be important. In some cases this will be the use of GPU compute to improve (via better performance or flexibility) algorithms typically performed using the graphics pipeline (image histogram analysis for HDR tone mapping, depth of field, bloom, texture-space diffusion for subsurface scattering, tessellation), and in others it will be to perform algorithms for which the graphics pipeline is not well-suited: ray tracing, stochastic rasterization, or dynamic object-space ambient occlusion.

David believes that the next stage of GPU computing (“4.0”) poses challenges to APIs (such as CUDA), to researchers, and to the education community. CUDA needs to be able to elegantly express programming models beyond simple parallelism, it needs to better express locality, and the development environment needs to improve and mature. Researchers need to foster new high-level libraries, languages, and platforms, as well as rethinking their algorithms. Finally, computer science curricula need to start teaching parallel computing in the first year.

Tags: , ,

An industry session kicked off the second day of I3D (there were papers on the first day that I’m skipping over since I’ll do a paper roundup post later).

This session was comprised of two talks by game industry speakers – Dan Baker (Firaxis), and Chris Hecker (currently working on the indie game SpyParty). I’ll summarize each talk as well as the discussion that followed.

Dan Baker’s talk: “From Papers to Pixels: how Research Finds (or Often Doesn’t) its Way into Games”

Dan started by stressing that although most interactive graphics research has games as the primary target application, researcher’s priorities and the needs of the game industry are often misaligned. Game developers prize papers that enable increasing visual quality and/or productivity, describe techniques that are art-directable, and that inspire further work.

Dan complained that researchers appear to prize novelty over practicality. Often papers intended to inspire future research instead create things nobody needs, like a cheese grater-mousetrap combination (an actual patent registration). Papers are judged by other researchers who often lack the background needed to determine their utility.

As Dan pointed out, the vast majority of graphics papers over the years have not been used. He illustrated this by contrasting two papers which had citation rates wildly out of step with their actual real-world impact.

The first example was the progressive meshes paper – one of the most widely cited papers in the field, but it had very little practical impact. Why wasn’t it used? Dan identified three primary reasons:

  1. It solved an already solved problem – artists were already trained to create low-polygon models and weren’t looking for automatic tools which they couldn’t control and which would require changes to the tool chain. Building the mesh was a relatively small part of the art asset pipeline in any case.
  2. The process was fragile and the quality of the final result varied.
  3. Hardware advances rapidly reduced the importance of vertex and triangle throughput as bottlenecks.

In contrast, a paper on vertex cache optimization (written by the same author – Hughes Hoppe) had very low citation rates. However, it had a profound impact in practice. Almost every game pipeline implements similar techniques – it is considered professional negligence at this point not to. Why is this paper so heavily used by game developers?

  1. It was easy to implement and integrate into the asset pipeline.
  2. It did not impact visual quality.
  3. It increased performance across the board.
  4. It remained valuable in the face of ongoing hardware changes.

The reason papers languish is not developers apathy; if a paper offers promise of solving an important problem, game developers will try to implement it. Dan mentioned talking to graphics programmers at a game development conference shortly after the variance shadow maps paper was published – all of them had tried to implement it (although they eventually abandoned it due to artifacts). Dan gave three rules to help researchers seeking to do relevant research for game development:

  1. Hardware advances can rapidly render techniques irrelevant, but this can typically be predicted based on current trends.
  2. Give developers something they need – an incremental improvement to something useful is better than a profound advance in an esoteric area.
  3. Radical changes in the way things are done are difficult to sell.

Dan gave several examples of graphics research used by Firaxis in Civilization V (participating media, subsurface scattering, Banks and Ashikhmin-Shirley anisotropic BRDFs, wavelet splines, and smoothed particle hydrodynamics), and suggested that researchers interact more with game developers (via sabbaticals and internships). He ended by listing some areas he wants to see more research in:

  • MIP generation for linear reconstruction
  • Temporally stable bloom
  • Texture compression
  • Shader anti-aliasing
  • Better normal generation from height data
  • Better cloth shading

Chris Hecker’s talk: “A Game Developer’s Wish List for Researchers”

Chris recorded the audio of his talk, and has a flash animation of the slides synchronized to the audio (as well as separate downloads) on his website.

One comment Chris made resonated with me, although it was unrelated to the main topic of his talk. He said he thought games are around where films were in 1905 in terms of development. I’ve been looking a lot into the history of films lately, and that sounds about right to me – games still have a long way to go. Anyway, back to the theme of stuff game developers want from researchers. Chris stated that although it is commonly thought that the top (indeed only) priority for game developers is performance, the top three game technology priorities in his opinion are robustness, simplicity, and performance, in that order. He went into some more detail on each of these.

Robustness: this is important because of interactivity. Players can get arbitrarily close to things, look at them from different angles, etc. and everything needs to hold up. Chris describes several dimensions of robustness: what are the edge cases (when does it break), what are the failure modes (how does it break), what are the downsides to using the technique, is the parameter space simply connected (i.e. can you tweak and interpolate the parameters and get reasonable results), and are the negative results described (things the researchers tried that didn’t work). Published papers in particular have a robustness problem – when game developers try to implement them they typically don’t work. Page limits mitigate against a proper analysis of drawbacks, implementation tips, etc. Now that most journals and conference proceedings are going paperless, Chris claimed that there is no reason to have these restrictive page limits.

Simplicity: Chris stated that games are always at the edge of systemic complexity failure. If the toolchain is not on the edge of collapse, that means the game developers missed out on an opportunity to add more stuff. It’s the classic “straw that breaks the camel’s back” problem – the added complexity needed to implement a given technique might not seem large by itself, but the existing complexity of the system needs to be taken into account. If the technique does not provide a 10X improvement in some important metric, adding any complexity is not worth it. Simplicity, even crudeness, is a virtue in game development. Like robustness, simplicity also has many dimensions: are the implications of the technique explainable to artists, does it have few parameters, is the output intuitive, are the results art-directable, is it easy to integrate into the tools pipeline, what dependencies (code, preprocessing, markup, order, compatibility) does the technique have, etc.

Performance: As Chris described it, this is different than classic computer science performance. The constant matters, typically more than the O() notation. Researchers need to specify the time it takes for their technique to operate in milliseconds, instead of giving a frames-per-second count that is useless since it includes overheads for rendering some arbitrary scene. It’s important to discuss worst case performance and optimize for that, rather than for the average case. Researchers should not just focus on embarrassingly parallel algorithms, and should do real comparisons of their technique’s performance. A “real comparison” means comparing against real techniques used in practice, against real (typically highly optimized) implementations used in the field, using real inputs, real scenes and real working set sizes. The issue of “real scenes” in particular is more nuanced than commonly thought – it’s not enough to have the same triangle count as a game scene. Any given game scene will have a particular distribution of triangle sizes, and particular “flavors” of overdraw, materials, shadows, lighting, etc. that all have significant performance implications.

Chris talked about the importance of providing source code. Researchers typically think about their paper as being rigorous, and their source code as being “dirty” or “hacky”. However, the source code is the most rigorous description of the algorithm. You can’t handwave details or gloss over edge cases in source code. Availability of source code greatly increases the chance of games developers using the technique. Given that only a small fraction of papers are relevant to game developers (and a small fraction of those work as advertised), the cost of implementing each paper just to figure out if it works is prohibitively high.

As for what to research, Chris stated that researchers should avoid “solutions looking for problems” – they should talk to game developers to find out what the actual urgent problems are. If the research is in the area of graphics, it needs to integrate well with everything else. AI research is especially tricky since AI is game design; the algorithm’s behavior will directly affect the gameplay experience. In the case of animation research, it is important to remember that interactivity is king; the nicest looking animation is a failure if it doesn’t respond fluidly to player commands. Perceptual models and metrics are an important area of research – what is important when it comes to visuals?

Chris ended his talk with a few miscellaneous recommendations to researchers:

  • Don’t patent! If you do, warn us in the abstract so we can skip reading the paper.
  • Put the paper online, not behind a paywall.
  • Publish negative results – knowing what didn’t work is as important as knowing what did.
  • Answer emails – often developers have questions about the technique (not covered in the paper due to page limits).
  • Play games! Without seeing examples of what games are doing now, it’s hard to know what they will need in the future.

Following discussions:

These two talks sparked lots of discussion in the Q&A sections and during subsequent breaks. The primary feeling among the researchers was that game developers have a very one-sided view of the relationship. While researchers do want their research to have a practical impact, they also have more direct needs, such as funding. Computer graphics research used to be largely funded by the military; this source dried up a while ago and many researchers are struggling for funding. If it’s true that games are the primary benefactor of research into computer graphics research, shouldn’t the game industry be the primary funding source as well?

Regarding the use of “real” data, most of the researchers are anxious to do so, but very few game companies will provide it! Valve is a notable exception, and indeed several of the papers at I3D used level data from Left 4 Dead 2. More game developers need to provide their data, if they hope for research which works well on their games.

Companies in other industries do a much better job of working with academic researchers, establishing mutually beneficial relationships. Industry R&D groups (Disney Research, HP Labs, Adobe’s Advanced Technology Labs, Microsoft Research, Intel Research, NVIDIA Research, etc.) are a key interface between industry and academia; if more game companies established such groups, that could help.

Tags: ,

Today was the first day of I3D 2011. I3D (full name: ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games) is a great little conference – it’s small enough so you can talk to most of the people attending, and the percentage of useful papers is relatively high.

I’ll split my I3D report into several smaller blog posts to ensure timely updates; this one will focus on the opening keynote.

The keynote was titled “Image-Based Rendering: A 15-Year Retrospective”, presented  by Richard Szeliski (Microsoft Research), who’s been involved with a lot of the important research in this area. He started with a retrospective of the research in this area, and then followed with a discussion of a specific application: panoramic image stitching. Early research in this area lead to QuickTime VR, and panoramic stitching is now supported in many cameras as a basic feature. Gigapixel panoramas are common – see a great example of one from Obama’s inaugural address by Gigapan (360 cities is another good panorama site). Street-level views such as Google Street View and Bing’s upcoming Street Slide feature use sequences of panoramas.

Richard then discussed the mathematical basis of image-based rendering: a 4D space of light rays (reliant on the fact that radiance is constant over a ray – in participating media this no longer holds and you have to go to a full 5D plenoptic field). Having some geometric scene information is important, it is hard to get good results with just a 4D collection of rays  (this was the main difference between the first two implementations – lumigraphs and light fields). Several variants of these were developed over the years (e.g. unstructured lumigraphs and surface light fields).

Several successful developments used lower-dimensional “2.5D” representations such as layered depth images (a “depth image” is an image with a depth value as well as a color associated with each pixel). Richard remarked that Microsoft’s Kinect has revolutionized this area by making depth cameras (which used to cost many thousands of dollars) into $150 commodities; a lot of academic research is now being done using Kinect cameras.

Image-based modeling started primarily with Paul Debevec’s “Facade” work in 1996. The idea was to augment a very simple geometric model (which back then was created manually) with “view dependent textures” that combine images from several directions to remove occluded pixels and provide additional parallax. Now this can be done automatically at a large scale – Richard showed an aerial model of the city of Graz which was automatically generated in 2009 from several airplane flyovers – it allowed for photorealistic renders with free camera motion in any direction.

Richard also discussed environment matting, as well as video-based techniques such as video rewrite, video matting, and video textures. This last paper in particular seems to me like it should be revisited for possible game applications – it’s a powerful extension of the common practice of looping a short sequence of images on a sprite or particle.

Richard next talked about cases where showing the results of image-based rendering in a less realistic or lower-fidelity way can actually be better – similar to the “uncanny valley” in human animation.

The keynote ended by summarizing the areas where image-based rendering is currently working well, and areas where more work is needed. Automatic 3D pose estimation of the camera from images is pretty robust, as well as automatic aerial and manually augmented ground-level modeling. However, some challenges remain: accurate boundaries and matting, reflections & transparency, integration with recognition algorithms, and user-generated content.

For anyone interested in more in-depth research into these topics, Richard has written a computer vision book, which is also available freely online.

Tags: ,

SIGGRAPH 2011 will be in Vancouver, on August 7-11, 2011. I’ve given presentations at SIGGRAPH several times; each time was a great experience where I learned a lot and met some pretty awesome graphics people from the world’s top research institutions, film production companies, and game development studios.

SIGGRAPH has several programs at which game developers can show their work; I wanted to point out that two of the most important (Talks and Dailies) have deadlines on February 18th, less than two weeks away! Fortunately submitting a proposal to one of these programs doesn’t take much time. However, getting approval from your boss may take a while, so you don’t want to wait.

SIGGRAPH Talks are 20-minute long presentations which typically contain “nuggets” of novel film or game production tech. These can be rendering or shading techniques, tools for artists, enhancements done to support a tricky character design, etc. If it’s something a programmer or technical artist is proud of having done and it’s at least tangentially graphics-related, chances are it would make a good Talk submission. Submitting a talk only requires creating a one-page abstract; if the talk is accepted, you have until August to make 20 minutes worth of slides – not too bad. To get an idea of the level of detail expected in the abstract, and of the variety of possible talks, here are some film and game Talk abstracts from 2009 and 2010: Houdini in a Games Pipeline, Spore API: Accessing a Unique Database of Player Creativity, Radially-Symmetric Reflection Maps, Underground Cave Sequence for Land of the Lost, Hatching an Imaginary Bird, Fast Furry Ray Gathering, and NPR Gabor Noise for Coherent Stylization. If you are reading this, please consider submitted the coolest thing you’ve done last year as a Talk; the small time investment will repay itself many times over.

SIGGRAPH Dailies are relatively new (first introduced at SIGGRAPH 2010). These are very short (under two minutes!) presentations of individual art assets, such as models, animations, particle effects, shaders, etc. Unlike the rest of SIGGRAPH which emphasizes novel techniques, Dailies emphasize excellence in the result. Every good game or movie has many individual bits of excellence, each the result of an artist’s talent, imagination and sweat. These are often overlooked, or unknown outside the studio; Dailies aim to correct that. Dailies submissions are even easier than Talk submissions. All that is required is a short (60-90 second) video of the art asset, no audio, just something simple like an animation loop or model turntable. You will also need a short backstory; something that gives a feeling for the effort that went into the work, including any notable production frustrations, unlikely inspirations, sudden strokes of genius, etc. Don’t write too much – it should take about as long to say as the video length (60-90 seconds). To get a better idea of what a Dailies presentation looks like, here are two examples. The list of Dailies presented at SIGGRAPH 2010 can be found here: it runs the gamut from Pixar and Disney movies to student projects. I suspect not many artists read this blog, so any game programmers reading this, please forward it to the artists at your studio.

Good luck with your submissions!

Tags: ,

Ke-Sen’s I3D 2011 papers page now has the full list. At the moment there are only 8 author preprints available out of 24 papers, but I’m sure more will appear soon. Some of the paper titles look very intriguing – I’ll write a followup blog post about them soon.

In addition, the I3D conference registration page is now up. Early registration prices range from $200-550, depending on whether you are a student, ACM member, etc. Judging from previous years, the quality of the conference is likely to be well worth the cost of attending, especially if you live in the San Francisco Bay Area and don’t have to worry about airfare and hotels.

The conference registration page also has details on hotel registration – the conference is at the Marriott Fisherman’s Wharf in San Francisco and there is a discount rate ($129 per night, only guaranteed until Jan 21, 2011). There are two ways to book a room with the discount rate:

  1. By phone: call 1-800-525-0956 and ask for the ACM Group rate.
  2. Online: go to the hotel registration webpage, enter your arrival/departure dates, Marriott reward number (if applicable), and then one of the following three codes in the ‘Group code’ option: ASSASSA (for single/double occupancy), ASSASSB (for triple occupancy), or ASSASSC (for quadruple occupancy).

Tags: ,

Halo: Reach was one of the big game releases of 2010, so I was pleased to see a detailed technical interview with some of the developers on Eurogamer‘s Digital Foundry website. I recommend you read the whole thing, but I’ll summarize some of the notable rendering tidbits (the interview also covered multiplayer, AI, and animation):

  • The previous two games (Halo 3 and Halo 3: ODST) used a “semi-deferred” approach, not for deferred lighting or shading but for decals. It sounds like they rendered a cut-down g-buffer (probably just diffuse color) in the first geometry pass (skipping small decoration objects like grass and pebbles), then blended decals into this buffer, and finally rendered the geometry a second time to do the lighting. Halo: Reach changed to a deferred lighting approach. Some lights were deferred and some weren’t, objects without decals or deferred lighting were only rendered once (this “hybrid deferred lighting” sounds similar to what Naughty Dog used for the Uncharted series).
  • Halo 3 used spherical harmonics in lightmaps to store directional lighting information (detailed in a GDC 2008 talk, as well as a SIGGRAPH 2008 course – see slides and course notes). For Halo: Reach, Bungie developed an improved light map representation that gave them “the same support for area light sources, improved contrast, fewer artifacts, a smaller memory footprint and much better performance”. This sounds really interesting; I hope they will describe this further in a conference presentation or article.
  • They developed a particle system which performs scene collisions on the GPU, using the depth and normal buffers as an approximate scene description. it can do tens of thousands of collisions / bounces per frame in 0.3 milliseconds (their previous CPU-based colliding particle system had a budget of 7 collisions per frame!). This system will be presented at GDC 2011 (the presentation will also discuss optimizations to their atmospheric effects system).  This is a great idea – techniques like SSAO use depth/normal buffers as approximate scene descriptions for rendering, but this is the first time I have heard of this being done for simulation.
  • Halo 3 used two 8-bit-per-channel frame buffers with different exposure values for HDR effects (primarily bloom). Bungie described this scheme in a presentation called “HDR the Bungie Way” at two Gamefest conferences: USA in 2006 and Europe in 2007 – the 2006 (giant) zip file also contains an audio recording, but the 2007 one has more updated slides (including screenshots). The GDC 2008 talk mentioned above also discusses this scheme briefly towards the end. In contrast, Halo: Reach uses a single 7e3 buffer; this yields higher performance and frees up more EDRAM for shadow buffers but has less dynamic range (the primary result of this is loss of color in some bloom regions).
  • Instead of MSAA, Halo: Reach uses a simple temporal anti-aliasing method. The camera is offset by a half-pixel in alternate frames, and the last two frames are selectively blended (the blending is turned off on pixels that have moved significantly since the last frame).
  • They developed a new LOD system (to be presented at GDC 2011) which automatically generates low-cost models to be used far from the camera. Combined with improved occlusion culling and GPU occlusion queries, this enabled a significant increase in draw distance.

Tags: ,

« Older entries § Newer entries »