Reports

You are currently browsing the archive for the Reports category.

I3D 2011 Report – Part III: Banquet Talk

March 11, 2011 in Reports by Naty | No comments

GDC has put a bit of a hiatus in my I3D posts; I better get them done soon so I can move onto the GDC posts.

This post describes a talk that David Luebke (Director of Research at NVIDIA) gave during the I3D banquet titled GPU Computing: Past, Present, and Future. Although the slides for the I3D talk are not available, parts of this talk appear to be similar to one David gave a few months ago, which does have video and slides available.

I’ll summarize the talk here; anyone interested in more detail can view the materials linked above.

The first part of the talk (which isn’t in the earlier version) covered the “New Moore’s Law”: computers no longer get faster, just wider; must re-think algorithms to be parallel. David showed examples of several scientists which got profound speedups – from days to minutes. He covered several different techniques, I’ll summarize the most notable four:

  1. A “photonic fence” that zaps mosquitoes with lasers, to reduce the incidence of malaria in third world countries. This application needs fast computer vision combined with low power consumption, which was achieved by using GPUs.
  2. A military vehicle which detects Improvised Explosive Devices (IEDs) using computer vision techniques. The speedup afforded by using GPUs enables the vehicle to drive much faster (an obvious advantage when surrounded by hostile insurgents) while still reliably detecting IEDs.
  3. A method for processing CT scans that enables much reduced radiation exposure for the patient. When running on CPUs, the algorithm was impractically slow; GPUs enabled it to run fast enough to be used in practice.
  4. A motion compensation technique that enables surgery on a beating heart. The video of the heart is motion-compensated to appear static to the surgeon, who operates through a surgical robot that translates the surgeon’s motions into the moving frame of the heart.

David started the next part of the talk (which is very similar to the earlier version linked above)  by going over the heritage of GPU computing. He did so by going over three separate historical threads: graphics hardware, supercomputing, and finally GPU Computing.

The “history of graphics hardware” section started with a brief mention of a different kind of hardware: Dürer‘s perspective machine. The history of electronic graphics hardware started with Ivan Sutherland’s SketchPad and continues through the development of the graphics pipeline by SGI: Geometry Engine (1982), RealityEngine (1993), and InfiniteReality (1997). In the early days, the graphics pipeline was an actual description of the physical hardware structure: each stage was a separate chip or board, with the data flow fixed by the routing of wires between them. Currently, the graphics pipeline is an abstraction; the stages are different threads running on a shared pool of cores, as seen in modern GPU designs such as the GeForce 8, GT200, and Fermi.

The second historical thread was the development of supercomputers. David covered the early development of three ways to build a parallel machine: SIMD (Goddard MPP, Maspar MP-1, Thinking Machines CM-1 and CM-2), hardware multithreading (Tera MTA) and symmetric multiprocessing (SGI Challenge, Sun Enterprise) before returning to Fermi as an example of a design that combines all three.

“GPU computing 1.0″ was the use (or abuse) of graphics pipelines and APIs to do general-purpose computing, culminating with BrookGPU. CUDA ushered in “GPU computing 2.0″ with an API designed for that purpose. The hardware supported branching and looping, and hid thread divergence from the programmer. David claimed that now GPU computing is in a “3.0″ stage, supported by a full ecosystem (multiple APIs, languages, algorithms, tools, IDEs, production lines, etc.). David estimated that there are about 100,000 active GPU compute developers in the world. Currently CUDA includes features such as “GPU Direct” (direct GPU-to-GPU transfer via a unified address space), full C++ support, and a template library.

The “future” part of the talk discussed the workloads that will drive future GPUs. Besides current graphics and high performance computing workloads, David believes a new type of workload, which he calls computational graphics, will be important. In some cases this will be the use of GPU compute to improve (via better performance or flexibility) algorithms typically performed using the graphics pipeline (image histogram analysis for HDR tone mapping, depth of field, bloom, texture-space diffusion for subsurface scattering, tessellation), and in others it will be to perform algorithms for which the graphics pipeline is not well-suited: ray tracing, stochastic rasterization, or dynamic object-space ambient occlusion.

David believes that the next stage of GPU computing (“4.0″) poses challenges to APIs (such as CUDA), to researchers, and to the education community. CUDA needs to be able to elegantly express programming models beyond simple parallelism, it needs to better express locality, and the development environment needs to improve and mature. Researchers need to foster new high-level libraries, languages, and platforms, as well as rethinking their algorithms. Finally, computer science curricula need to start teaching parallel computing in the first year.

Tags: , ,

I3D 2011 Report – Part II: Industry Session

February 27, 2011 in Reports by Naty | 4 comments

An industry session kicked off the second day of I3D (there were papers on the first day that I’m skipping over since I’ll do a paper roundup post later).

This session was comprised of two talks by game industry speakers – Dan Baker (Firaxis), and Chris Hecker (currently working on the indie game SpyParty). I’ll summarize each talk as well as the discussion that followed.

Dan Baker’s talk: “From Papers to Pixels: how Research Finds (or Often Doesn’t) its Way into Games”

Dan started by stressing that although most interactive graphics research has games as the primary target application, researcher’s priorities and the needs of the game industry are often misaligned. Game developers prize papers that enable increasing visual quality and/or productivity, describe techniques that are art-directable, and that inspire further work.

Dan complained that researchers appear to prize novelty over practicality. Often papers intended to inspire future research instead create things nobody needs, like a cheese grater-mousetrap combination (an actual patent registration). Papers are judged by other researchers who often lack the background needed to determine their utility.

As Dan pointed out, the vast majority of graphics papers over the years have not been used. He illustrated this by contrasting two papers which had citation rates wildly out of step with their actual real-world impact.

The first example was the progressive meshes paper – one of the most widely cited papers in the field, but it had very little practical impact. Why wasn’t it used? Dan identified three primary reasons:

  1. It solved an already solved problem – artists were already trained to create low-polygon models and weren’t looking for automatic tools which they couldn’t control and which would require changes to the tool chain. Building the mesh was a relatively small part of the art asset pipeline in any case.
  2. The process was fragile and the quality of the final result varied.
  3. Hardware advances rapidly reduced the importance of vertex and triangle throughput as bottlenecks.

In contrast, a paper on vertex cache optimization (written by the same author – Hughes Hoppe) had very low citation rates. However, it had a profound impact in practice. Almost every game pipeline implements similar techniques – it is considered professional negligence at this point not to. Why is this paper so heavily used by game developers?

  1. It was easy to implement and integrate into the asset pipeline.
  2. It did not impact visual quality.
  3. It increased performance across the board.
  4. It remained valuable in the face of ongoing hardware changes.

The reason papers languish is not developers apathy; if a paper offers promise of solving an important problem, game developers will try to implement it. Dan mentioned talking to graphics programmers at a game development conference shortly after the variance shadow maps paper was published – all of them had tried to implement it (although they eventually abandoned it due to artifacts). Dan gave three rules to help researchers seeking to do relevant research for game development:

  1. Hardware advances can rapidly render techniques irrelevant, but this can typically be predicted based on current trends.
  2. Give developers something they need – an incremental improvement to something useful is better than a profound advance in an esoteric area.
  3. Radical changes in the way things are done are difficult to sell.

Dan gave several examples of graphics research used by Firaxis in Civilization V (participating media, subsurface scattering, Banks and Ashikhmin-Shirley anisotropic BRDFs, wavelet splines, and smoothed particle hydrodynamics), and suggested that researchers interact more with game developers (via sabbaticals and internships). He ended by listing some areas he wants to see more research in:

  • MIP generation for linear reconstruction
  • Temporally stable bloom
  • Texture compression
  • Shader anti-aliasing
  • Better normal generation from height data
  • Better cloth shading

Chris Hecker’s talk: “A Game Developer’s Wish List for Researchers”

Chris recorded the audio of his talk, and has a flash animation of the slides synchronized to the audio (as well as separate downloads) on his website.

One comment Chris made resonated with me, although it was unrelated to the main topic of his talk. He said he thought games are around where films were in 1905 in terms of development. I’ve been looking a lot into the history of films lately, and that sounds about right to me – games still have a long way to go. Anyway, back to the theme of stuff game developers want from researchers. Chris stated that although it is commonly thought that the top (indeed only) priority for game developers is performance, the top three game technology priorities in his opinion are robustness, simplicity, and performance, in that order. He went into some more detail on each of these.

Robustness: this is important because of interactivity. Players can get arbitrarily close to things, look at them from different angles, etc. and everything needs to hold up. Chris describes several dimensions of robustness: what are the edge cases (when does it break), what are the failure modes (how does it break), what are the downsides to using the technique, is the parameter space simply connected (i.e. can you tweak and interpolate the parameters and get reasonable results), and are the negative results described (things the researchers tried that didn’t work). Published papers in particular have a robustness problem – when game developers try to implement them they typically don’t work. Page limits mitigate against a proper analysis of drawbacks, implementation tips, etc. Now that most journals and conference proceedings are going paperless, Chris claimed that there is no reason to have these restrictive page limits.

Simplicity: Chris stated that games are always at the edge of systemic complexity failure. If the toolchain is not on the edge of collapse, that means the game developers missed out on an opportunity to add more stuff. It’s the classic “straw that breaks the camel’s back” problem – the added complexity needed to implement a given technique might not seem large by itself, but the existing complexity of the system needs to be taken into account. If the technique does not provide a 10X improvement in some important metric, adding any complexity is not worth it. Simplicity, even crudeness, is a virtue in game development. Like robustness, simplicity also has many dimensions: are the implications of the technique explainable to artists, does it have few parameters, is the output intuitive, are the results art-directable, is it easy to integrate into the tools pipeline, what dependencies (code, preprocessing, markup, order, compatibility) does the technique have, etc.

Performance: As Chris described it, this is different than classic computer science performance. The constant matters, typically more than the O() notation. Researchers need to specify the time it takes for their technique to operate in milliseconds, instead of giving a frames-per-second count that is useless since it includes overheads for rendering some arbitrary scene. It’s important to discuss worst case performance and optimize for that, rather than for the average case. Researchers should not just focus on embarrassingly parallel algorithms, and should do real comparisons of their technique’s performance. A “real comparison” means comparing against real techniques used in practice, against real (typically highly optimized) implementations used in the field, using real inputs, real scenes and real working set sizes. The issue of “real scenes” in particular is more nuanced than commonly thought – it’s not enough to have the same triangle count as a game scene. Any given game scene will have a particular distribution of triangle sizes, and particular “flavors” of overdraw, materials, shadows, lighting, etc. that all have significant performance implications.

Chris talked about the importance of providing source code. Researchers typically think about their paper as being rigorous, and their source code as being “dirty” or “hacky”. However, the source code is the most rigorous description of the algorithm. You can’t handwave details or gloss over edge cases in source code. Availability of source code greatly increases the chance of games developers using the technique. Given that only a small fraction of papers are relevant to game developers (and a small fraction of those work as advertised), the cost of implementing each paper just to figure out if it works is prohibitively high.

As for what to research, Chris stated that researchers should avoid “solutions looking for problems” – they should talk to game developers to find out what the actual urgent problems are. If the research is in the area of graphics, it needs to integrate well with everything else. AI research is especially tricky since AI is game design; the algorithm’s behavior will directly affect the gameplay experience. In the case of animation research, it is important to remember that interactivity is king; the nicest looking animation is a failure if it doesn’t respond fluidly to player commands. Perceptual models and metrics are an important area of research – what is important when it comes to visuals?

Chris ended his talk with a few miscellaneous recommendations to researchers:

  • Don’t patent! If you do, warn us in the abstract so we can skip reading the paper.
  • Put the paper online, not behind a paywall.
  • Publish negative results – knowing what didn’t work is as important as knowing what did.
  • Answer emails – often developers have questions about the technique (not covered in the paper due to page limits).
  • Play games! Without seeing examples of what games are doing now, it’s hard to know what they will need in the future.

Following discussions:

These two talks sparked lots of discussion in the Q&A sections and during subsequent breaks. The primary feeling among the researchers was that game developers have a very one-sided view of the relationship. While researchers do want their research to have a practical impact, they also have more direct needs, such as funding. Computer graphics research used to be largely funded by the military; this source dried up a while ago and many researchers are struggling for funding. If it’s true that games are the primary benefactor of research into computer graphics research, shouldn’t the game industry be the primary funding source as well?

Regarding the use of “real” data, most of the researchers are anxious to do so, but very few game companies will provide it! Valve is a notable exception, and indeed several of the papers at I3D used level data from Left 4 Dead 2. More game developers need to provide their data, if they hope for research which works well on their games.

Companies in other industries do a much better job of working with academic researchers, establishing mutually beneficial relationships. Industry R&D groups (Disney Research, HP Labs, Adobe’s Advanced Technology Labs, Microsoft Research, Intel Research, NVIDIA Research, etc.) are a key interface between industry and academia; if more game companies established such groups, that could help.

Tags: ,

Photos from I3D 2011

February 26, 2011 in Reports by Eric | No comments

Provided by Mauricio Vives: feast your squinties. Lots of headshots, which I’m sure the speakers (and their moms) will appreciate. I certainly do; it’s great putting a face to a name.

Also, save the date: as the last slide shows, March 9-11 2012 is the next I3D, in Costa Mesa, California. I’m suitably impressed that next year’s co-chairs (Sung-Eui Yoon and Gopi Meenakshisundaram) already have a place and date. This was possible since they’re colocating I3D to directly follow IEEE VR, which is March 4-8.

One photo, “Carlo’s Models“:

Tags: ,

I3D 2011 Report – Part I: Keynote

February 19, 2011 in Reports by Naty | 2 comments

Today was the first day of I3D 2011. I3D (full name: ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games) is a great little conference – it’s small enough so you can talk to most of the people attending, and the percentage of useful papers is relatively high.

I’ll split my I3D report into several smaller blog posts to ensure timely updates; this one will focus on the opening keynote.

The keynote was titled “Image-Based Rendering: A 15-Year Retrospective”, presented  by Richard Szeliski (Microsoft Research), who’s been involved with a lot of the important research in this area. He started with a retrospective of the research in this area, and then followed with a discussion of a specific application: panoramic image stitching. Early research in this area lead to QuickTime VR, and panoramic stitching is now supported in many cameras as a basic feature. Gigapixel panoramas are common – see a great example of one from Obama’s inaugural address by Gigapan (360 cities is another good panorama site). Street-level views such as Google Street View and Bing’s upcoming Street Slide feature use sequences of panoramas.

Richard then discussed the mathematical basis of image-based rendering: a 4D space of light rays (reliant on the fact that radiance is constant over a ray – in participating media this no longer holds and you have to go to a full 5D plenoptic field). Having some geometric scene information is important, it is hard to get good results with just a 4D collection of rays  (this was the main difference between the first two implementations – lumigraphs and light fields). Several variants of these were developed over the years (e.g. unstructured lumigraphs and surface light fields).

Several successful developments used lower-dimensional “2.5D” representations such as layered depth images (a “depth image” is an image with a depth value as well as a color associated with each pixel). Richard remarked that Microsoft’s Kinect has revolutionized this area by making depth cameras (which used to cost many thousands of dollars) into $150 commodities; a lot of academic research is now being done using Kinect cameras.

Image-based modeling started primarily with Paul Debevec’s “Facade” work in 1996. The idea was to augment a very simple geometric model (which back then was created manually) with “view dependent textures” that combine images from several directions to remove occluded pixels and provide additional parallax. Now this can be done automatically at a large scale – Richard showed an aerial model of the city of Graz which was automatically generated in 2009 from several airplane flyovers – it allowed for photorealistic renders with free camera motion in any direction.

Richard also discussed environment matting, as well as video-based techniques such as video rewrite, video matting, and video textures. This last paper in particular seems to me like it should be revisited for possible game applications – it’s a powerful extension of the common practice of looping a short sequence of images on a sprite or particle.

Richard next talked about cases where showing the results of image-based rendering in a less realistic or lower-fidelity way can actually be better – similar to the “uncanny valley” in human animation.

The keynote ended by summarizing the areas where image-based rendering is currently working well, and areas where more work is needed. Automatic 3D pose estimation of the camera from images is pretty robust, as well as automatic aerial and manually augmented ground-level modeling. However, some challenges remain: accurate boundaries and matting, reflections & transparency, integration with recognition algorithms, and user-generated content.

For anyone interested in more in-depth research into these topics, Richard has written a computer vision book, which is also available freely online.

Tags: ,

How to Make an Ebook

January 16, 2011 in Reports, Resources by Eric | No comments

Here’s a short guide on creating decent ebooks from scans using Adobe Acrobat. This will not be of interest to 98% of you, but I want to record it somewhere for those of you who may do this in the future. It is written by Iliyan Georgiev, who made the recent PoDIS ebook. Comments are welcome, as usual.

The one piece of software you’ll need that can’t be downloaded for free is Adobe Acrobat, though even this application has a 30-day free trial.

1. Scan the pages of the book using a scanner (a digital camera is a good alternative).

2. Crop the scanned images (and split the pages, if you scanned two pages at once). It’s better for an ebook to have smaller page margins. Also, cropping removes black areas and other artifacts resulting from scanning. An excellent (JPEG-only) batch cropping tool for Windows is JPEGCrops. It has some disadvantages, however, so in practice it’s best to use JPEGCrops to estimate approximate cropping parameters (width, height, x-offset, y-offset) and XnView‘s batch processing mode for the actual cropping. Both applications are free and have portable versions.

3. Assemble all images into a PDF file. Adobe Acrobat has an option to combine multiple files into a single PDF. Use the highest quality settings for the creation.

4. (OPTIONAL) Rearrange/merge/delete pages. Acrobat has excellent tools to achieve these. This can be useful for books that are published in two volumes or for extending the book with additional information, such as errata listings, images, high quality cover pages, etc.

5. Manage blank pages. It might be tempting to delete blank pages inside the book. Such pages are always intentionally left blank by the publishers, as they are important for the printing order. This is particularly important for the first few pages, as well as for the chapters. Many books are created in such a way that all chapters start on an even/odd page, and the large majority have the inner pages typeset for being printed on a specific side (left/right). If you want to optimize the page count anyway, keep in mind how the book would appear when printed out (also using “2 pages per sheet” printing).

6. Number the pages. This is an often-overlooked, but very useful, option. Apart from the default page numbering, the PDF format supports logical page numbering. This can be used to synchronize the PDF page numbers with the actual book page numbers. This is very easy to do in Acrobat and should always be done. To do this, select the necessary pages, right click on them and choose “Number Pages…”.

7. Run OCR (optical character recognition) on the PDF. This is an extremely easy way to make your scanned pages searchable and the text copy/paste-able. Acrobat has a good and easy to use built-in OCR tool. You will find it in the Document menu (Tools pane in Acrobat X). Be sure to disable image resampling, as by default OCR will resample the images, which can easily increase the file size by a huge amount! Keep in mind that OCR is a compute-intensive process and can easily take a couple of hours for a larger book.

8. Optimize document. Acrobat has an option to optimize scanned documents. This runs some image-processing algorithms on the scanned images and compresses them aggressively when it detects text. This is a vital step to keep the size of the document low. It can reduce the file size by a factor of 20! It will also make the antialiasing to look better when pages are minified, if the resolution of the original scans is high enough. This process is also compute-intensive and can easily take an hour for a larger book.

9. (OPTIONAL) Reduce the file size further by using Acrobat’s other optimization options, from which the image downsampling is the most important.

At this point the most important steps are done and you can end here and go to sleep if you see the sunrise through the window. Go on if it’s only 4 AM.

10. (OPTIONAL) Setting the initial view. Open the document properties on the Initial View tab. Here, you can set the initial page, zoom level and which panes (e.g. the bookmarks pane, see below) should be active when the document is opened.

11. (OPTIONAL) Create a PDF table of contents (TOC). The PDF format has a useful (hierarchical) bookmarking feature with a dedicated Bookmarks pane which exists also in Adobe Reader. This feature can be used to reconstruct the book’s TOC for easy document navigation. One simple way to achieve this is the following:
11.a Go to the book’s Contents page, select the chapter title’s text and hit CTRL+B (or right click and choose to add a bookmark from the context menu). Repeat this for each chapter.
11.b Structure the created bookmarks. Rearrange the bookmarks to follow the order and structure of the book’s TOC.
11.c Link the bookmarks to pages. To do this, go over all pages of the book sequentially and every time a new chapter starts, right click on the corresponding bookmark and set the destination to the current page.

12. (OPTIONAL) Create hyperlinks inside the document. The PDF format also supports hyperlinks which can perform actions (e.g. jump to a page or a web site) when clicked. Links can be either rectangles (drawn with a corresponding tool) or text. To create text links, select the text, right click on it and choose to crate a link. There are options to set the link’s appearance and behavior.

You’re done! You have the perfect ebook and you’re late for work!

Tags: , , ,

Gran Turismo on Playstation, PSP, PS2, and PS3

December 13, 2010 in Reports by Naty | 2 comments

This video was published by Eurogamer‘s Digital Foundry department about two weeks ago; it shows footage captured from various games in the Gran Turismo series. What is remarkable about this video is that the same cars and tracks are shown on the original Playstation, the PSP, the Playstation 2 and Playstation 3. Since the developer (Polyphony Digital) has a reputation for squeezing the best visuals out of Sony’s platforms, this promises a rare “apples-to-apples” comparison across multiple hardware generations.

To my eyes, the display resolution changes drown out the more subtle differences in modeling, shading and lighting; it is also apparent to me that Polyphony no longer sits on the graphics throne in this generation. Other first-party PS3 titles such as Uncharted 2 and God of War III look better, in my opinion. The shadows are a particular weak spot: in places their resolution seems no higher than on the original Playstation!

More information on how the video was captured (as well as high-quality download links) can be found in Digital Foundry’s blog post.

Tags:

I3D 2011

July 15, 2010 in Reports by Naty | No comments

The website for I3D 2011 is now up, including the time/place and CFP. I3D will be in San Francisco next year, from February 18-20th. I3D probably has a higher percentage of graphics papers relevant to games than any other conference; this year five of the papers described techniques already in use in games (including high-profile titles like Batman: Arkham Asylum and Civilization 5), and many of the other papers were also highly relevant. Unfortunately, very few game developers attend; I hope next year’s location (San Francisco is home to a large number of developers) will help.

I3D is a great small conference to publish real-time rendering papers. One advantage it has for authors over Eurographics conferences like EGSR, and co-sponsored conferences like HPG and SCA (in “Europe” years) is that it is not subject to Eurographics’ monumentally stupid “authors can’t post copies of their papers for a year after the conference” policy. This policy, of course, hurts the chance of your paper being cited by making it harder for people to read it – brilliant! Hopefully EG will see the error of its ways soon – until then, you are better off sending your papers to non-EG conferences like I3D.

Tags: ,

SIGGRAPH 2010 Game Content Roundup

July 13, 2010 in Reports by Naty | No comments

With less than two weeks until the conference, here’s my final pre-SIGGRAPH roundup of all the game development and real-time rendering content. This is to either to help convince people who are still on the fence about attending (unlikely at this late date) or to help people who are trying to decide which sessions to go to (more likely). If you won’t be able to attend SIGGRAPH this year, this might at least help you figure out which slides, videos, and papers to hunt for after the conference.

First of all, the SIGGRAPH online scheduler is invaluable for helping to sort out all the overlapping sessions (even if you just “download” the results into Eric’s lower-tech version). The iPhone app may show up before the conference, but given the vagaries of iTunes app store approval, I wouldn’t hold my breath.

The second resource is the Games Focus page, which summarizes the relevant content for game developers in one handy place. It makes a good starting point for building your schedule; the rest of this post goes into additional detail.

My previous posts about the panels and the talks, and several posts about the courses go into more detail on the content available in these programs.

Exhibitor Tech Talks are sponsored talks by various vendors, and are often quite good. Although the Games Focus page links to the Exhibitor Tech Talk page, for some reason that page has is no information about the AMD and NVIDIA tech talks (the Intel talk on Inspecting Complex Graphics Scenes in a Direct X Pipeline, about their Graphics Performance Analyzer tool, could be interesting). NVIDIA does have all the details on their tech talks at their SIGGRAPH 2010 page; the ones on OpenGL 4.0 for 2010, Parallel Nsight: GPU Computing and Graphics Development in Visual Studio, and Rapid GPU Ray Tracing Development with NVIDIA OptiX look particularly relevant. AMD has no such information available anywhere: FAIL.

One program not mentioned in the Games Focus page is a new one for this year: SIGGRAPH Dailies! where artists show a specific piece of artwork (animation, cutscene sequence, model, lighting setup, etc.) and discuss it for two minutes. This is a great program, giving artists a unique place to showcase the many bits of excellence that go into any good film or game. Although no game pieces got in this year, the show order includes great work from films such as Toy Story 3, Tangled, Percy Jackson, A Christmas Carol, The Princess and The Frog, Ratatouille, and Up. The show is repeated on Tuesday and Wednesday overlapping the Electronic Theater (which also should not be missed; note that it is shown on Monday evening as well).

One of my favorite things about SIGGRAPH is the opportunity for film and game people to talk to each other. As the Game-Film Synergy Chair, my primary responsibility was to promote content of interest to both. This year there are four such courses (two of which I am organizing and speaking in myself): Global Illumination Across Industries, Color Enhancement and Rendering in Film and Game Production, Physically Based Shading Models in Film and Game Production, and Beyond Programmable Shading I & II.

Besides the content specifically designed to appeal to both industries, a lot of the “pure film” content is also interesting to game developers. The Games Focus page describes one example (the precomputed SH occlusion used in Avatar), and hints at a lot more. But which?

My picks for “film production content most likely to be relevant to game developers”: the course Importance Sampling for Production Rendering, the talk sessions Avatar in Depth, Rendering Intangibles, All About Avatar, and Pipelines and Asset Management, the CAF production sessions Alice in Wonderland: Down the Rabbit Hole, Animation Blockbuster Breakdown, Iron Man 2: Bringing in the “Big Gun”, Making “Avatar”, The Making of TRON: LEGACY, and The Visual Style of How To Train Your Dragon, and the technical papers PantaRay: Fast Ray-Traced Occlusion Caching, An Artist-Friendly Hair Shading System, and Smoothed Local Histogram Filters. (unlike much of the other film production content, paper presentation videos are always recorded, so if a paper presentation conflicts with something else you can safely skip it).

Interesting, but more forward-looking film production stuff (volumetric effects and simulations that aren’t feasible for games now but might be in future): the course Volumetric Methods in Visual Effects, the talk sessions Elemental Training 101, Volumes and Precipitation, Simulation in Production, and Blowing $h!t Up, and the CAF production session The Last Airbender: Harnessing the Elements: Earth, Air, Water, and Fire.

Speaking of forward-looking content, SIGGRAPH papers written by academics (as opposed to film professionals) tend to fall in this category (in the best case; many of them are dead ends). I haven’t had time to look at the huge list of research papers in detail; I highly recommend attending the Technical Papers Fast-Forward to see which papers are worth paying closer attention to (it’s also pretty entertaining).

Some other random SIGGRAPH bits:

  • Posters are of very mixed quality (they have the lowest acceptance bar of any SIGGRAPH content) but quickly skimming them doesn’t take much time, and there is sometimes good stuff there. During lunchtime on Tuesday and Wednesday, the poster authors are available to discuss their work, so if you see anything interesting you might want to come back then and ask some questions.
  • The Studio includes several workshops and presentations of interest, particularly for artists.
  • The Research Challenge has an interesting interactive haunted house concept (Virtual Flashlight for Real-Time Scene Illumination and Discovery) presented by the Square Enix Research and Development Division.
  • The Geek Bar is a good place to relax and watch streaming video of the various SIGGRAPH programs.
  • The SIGGRAPH Reception, the Chapters Party, and various other social events throughout the week are great opportunities to meet, network, and talk graphics with lots of interesting and talented people from outside your regular circle of colleagues.

I will conclude with the list of game studios presenting at SIGGRAPH this year: Activision Studio Central, Avalanche Software, Bizarre Creations, Black Rock Studio, Bungie, Crytek, DICE, Disney Interactive Research, EDEN GAMES, Fantasy Lab, Gearbox, LucasArts, Naughty Dog, Quel Solaar, tri-Ace, SCE Santa Monica Studio, Square Enix R&D, Uber Entertainment, Ubisoft Montreal, United Front Games, Valve, and Volition. I hope for an even longer list in 2011!

Tags: ,

More SIGGRAPH Course Updates

July 12, 2010 in Reports by Naty | 1 comment

After my last SIGGRAPH post, I spent a little more time digging around in the SIGGRAPH online scheduler, and found some more interesting details:

Global Illumination Across Industries

This is another film-game crossover course. It starts with a 15-minute introduction to global illumination by Jaroslav Křivánek, a leading researcher in efficient GI algorithms. It continues with six 25-30 minutes talks:

  • Ray Tracing Solution for Film Production Rendering, by Marcos Fajardo, Solid Angle. Marcos created the Arnold raytracer which was adopted by Sony Pictures Imageworks for all of their production rendering (including CG animation features like Cloudy with a Chance of Meatballs and VFX for films like 2012 and Alice in Wonderland). This is unusual in film production; most VFX and animation houses  use rasterization renderers like Renderman.
  • Point-Based Global Illumination for Film Production, by Per Christensen, Pixar. Per won a Sci-Tech Oscar for this technique, which is widely used in film production.
  • Ray Tracing vs. Point-Based GI for Animated Films, by Eric Tabellion, PDI/Dreamworks. Eric worked on the global illumination (GI) solution which Dreamworks used in Shrek 2; it will be interesting to hear what he has to say on the differences between the two leading film production GI techniques.
  • Adding Real-Time Point-based GI to a Video Game, Michael Bunnell, Fantasy Lab. Mike was also awarded the Oscar for the point-based technique (Christophe Hery was the third winner). He actually originated it as a real-time technique while working at NVIDIA; while Per and Christophe developed it for film rendering, Mike founded Fantasy Lab to further develop the technique for use in games.
  • Pre-computing Lighting in Games, David Larsson, Illuminate Labs. Illuminate Labs make very good prelighting tools for games; I used their Turtle plugin for Maya when working on God of War III and was impressed with its speed, quality and robustness.
  • Dynamic Global Illumination for Games: From Idea to Production, Anton Kaplanyan, Crytek. Anton developed the cascaded light propagation volume technique used in CryEngine 3 for dynamic GI; the I3D 2010 paper describing the technique can be found on Crytek’s publication page.

The course concludes with a 5 minute Q&A session with all speakers.

An Introduction to 3D Spatial Interaction With Videogame Motion Controllers

This course is presented by Joseph LaViola (director of the University of Central Florida Interactive Systems and User Experience Lab) and Richard Marks from Sony Computer Entertainment (principal inventor of the Eyetoy, Playstation Eye, and Playstation Move). Richard Marks gives two 45-minute talks, one on 3D Interfaces With 2D and 3D Cameras and one on 3D Spatial Interaction with the PlayStation Move. Prof. LaViola discusses Common Tasks in 3D User Interfaces, Working With the Nintendo Wiimote, and 3D Gesture Recognition Techniques.

Recent Advances in Real-Time Collision and Proximity Computations for Games and Simulations

After an introduction to the topic of collision detection and proximity queries, this course goes over recent research in collision detection for games including articulated, deformable and fracturing models. It concludes with optimization-oriented talks such as GPU-Based Proximity Computations (presented by Dinesh Manocha, University of North Carolina at Chapel Hill, one of the most prominent researchers in the area of collision detection), Optimizing Proximity Queries for CPU, SPU and GPU (presented by Erwin Coumans, Sony Computer Entertainment US R&D, primary author of the Bullet physics library, which is widely used for both games and feature films), and PhysX and Proximity Queries (presented by Richard Tonge, NVIDIA, one of the architects of the AGEIA  physics processing unit – the company was bought by NVIDIA and their software library formed the basis of the GPU-accelerated PhysX library).

Advanced Techniques in Real-Time Hair Rendering and Simulation

This course is presented by Cem Yuksel (Texas A&M University) and Sarah Tariq (NVIDIA). Between them, they have done a lot of the recent research on efficient rendering and simulation of hair. The course covers all aspects of real-time hair rendering: data management, the rendering pipeline, transparency, antialiasing, shading, shadows, and multiple scattering. It concludes with a discussion of real-time dynamic simulation of hair.

Ray Tracing Solution for Film Production Rendering
Fajardo

2:40 pm
Point-Based Global Illumination for Film Production
Christensen

3:05 pm
Ray Tracing vs. Point-Based GI for Animated Films
Tabellion

3:30 pm
Break 

3:45 pm
Adding Real-Time Point-based GI to a Video Game
Bunnell

4:15 pm
Pre-computing Lighting in Games
Larsson

4:45 pm
Dynamic Global Illumination for Games: From Idea to Production Kaplanyan

5:10 pm
Conclusions, Q & A
Ray Tracing Solution for Film Production Rendering

Fajardo

2:40 pm

Point-Based Global Illumination for Film Production

Christensen

3:05 pm

Ray Tracing vs. Point-Based GI for Animated Films

Tabellion

3:30 pm

Break

3:45 pm

Adding Real-Time Point-based GI to a Video Game

Bunnell

4:15 pm

Pre-computing Lighting in Games

Larsson

4:45 pm

Dynamic Global Illumination for Games: From Idea to Production Kaplanyan

5:10 pm

Conclusions, Q & A

All

All

Tags: ,

Update on Splinter Cell: Conviction Rendering

July 11, 2010 in Reports by Naty | No comments

In my recent post about Gamefest 2010, I discussed Stephen Hill’s great presentation on the rendering techniques used in Splinter Cell: Conviction.

Since then, Stephen contacted me – it turns out I got some details wrong, and he also provided me with some additional details about the techniques in his talk. I will give the corrections and additional details here.

  1. What I described in the post as a “software hierarchical Z-Buffer occlusion system” actually runs completely on the GPU. It was directly inspired by the GPU occlusion system used in ATI’s “March of the Froblins” demo (described here), and indirectly by the original (1993) hierarchical z-buffer paper. Stephen describes his original contribution as “mostly scaling it up to lots of objects on DX9 hardware, piggy-backing other work and the 2-pass shadow culling”. Stephen promises more details on this “in a book chapter and possibly… a blog post or two” – I look forward to it.
  2. The rigid body AO volumes were initially inspired by the Ambient Occlusion Fields paper, but the closest research is an INRIA tech report that was developed in parallel with Stephen’s work (though he did borrow some ideas from it afterwards).
  3. The character occlusion was not performed using capsules, but via nonuniformly-scaled spheres. I’ll let Stephen speak to the details: “we transform the receiver point into ‘ellipsoid’-local space, scale the axes and lookup into a 1D texture (using distance to centre) to get the zonal harmonics for a unit sphere, which are then used to scale the direction vector. This works very well in practice due to the softness of the occlusion. It’s also pretty similar to Hardware Accelerated Ambient Occlusion Techniques on GPUs although they work purely with spheres, which may simplify some things. I checked the P4 history, and our implementation was before their publication, so I’m not sure if there was any direct inspiration. I’m pretty sure our initial version also predated Real-time Soft Shadows in Dynamic Scenes using Spherical Harmonic Exponentiation since I remember attending SIGGRAPH that year and teasing a friend about the fact that we had something really simple.”
  4. My statement that the downsampled AO buffer is applied to the frame using cross-bilateral upsampling was incorrect. Stephen just takes the most representative sample by comparing the full-resolution depth and object IDs against the surrounding down-sampled values. This is a kind of “bilateral point-sampling” which apparently works surprisingly well in practice, and is significantly cheaper than a full bilateral upsample. Interestingly, Stephen did try a more complex filter at one point: “Near the end I did try performing a bilinearly-interpolated lookup for pixels with a matching ID and nearby depth but there were failure cases, so I dropped it due to lack of time. I will certainly be looking at performing more sophisticated upsampling or simply increasing the resolution (as some optimisations near the end paid off) next time around.”

A recent blog post on Jeremy Shopf’s excellent Level of Detail blog mentions similarities between the sphere technique and one used for AMD’s ping-pong demo (the technique is described in the article Deferred Occlusion from Analytic Surfaces in ShaderX7). To me, the basic technique is reminiscent of Inigo Quilez‘ article on analytical sphere ambient occlusion; an HPG 2010 paper by Morgan McGuire does something similar with triangles instead of spheres.

Although the technique builds upon previous ones, it does add several new elements, and works well in the game. The technique does suffer from multiple-occlusion; I wonder if a technique similar to the 1D “compensation map’ used by Morgan McGuire might help.

Tags: ,

« Older entries § Newer entries »