mobile – Real-Time Rendering Tracking the latest developments in interactive rendering techniques Wed, 16 Aug 2017 13:31:51 +0000 en-US hourly 1 Book “WebGL Insights” now free Thu, 27 Jul 2017 18:04:51 +0000 The book WebGL Insights is now free to download as a PDF. Go get it.

Many of the articles are, of course, WebGL-centric, but some articles in the Rendering Section have general interest, especially for mobile developers. WebGL is “trailing edge,” in that it’s tied to OpenGL ES 2.0, which is what most mobile devices run. So techniques in that section will run in mobile apps in general. WebGL 2 (not covered in this book) is ES 3.0, basically, and current has 22% phone support and 8% tablet support – tablets don’t get refreshed as rapidly as phones.


Wow, that worked? Fri, 25 Jan 2013 02:46:51 +0000 The demo I made for the Interactive Rendering course actually runs on my phone (Samsung Galaxy S3). My course assistant found it runs on her HTC phone, too. Good luck hitting the up-arrow on my phone, like the documentation on the screen wants me to do… (I should fix that.) It runs only on Firefox, from what I can tell.

Update: works in Chrome Beta for Android.

Anyway, I’m impressed that my phone can do this at all. It does take a good long while to download and run, but still. 44 frames per second – not bad! Go three.js and WebGL. Oh, and here’s a little video.

click the picture below to go directly to the demo.

Slides & Notes for “Advanced Lighting for Mobile” Talk Wed, 15 Aug 2012 10:39:38 +0000 The slideset and notes for the SIGGRAPH 2012 talk “Advanced Lighting for Mobile”, given by Sam Martin of Geomerics, are up for download. Lots of interesting tidbits, useful even if you’re not programming for mobile devices.

SIGGRAPH 2012 Course Notes, so far Tue, 14 Aug 2012 10:06:23 +0000 Edit: wow, this page has almost everything from SIGGRAPH 2012 (thanks, Sebastian, and especially Stephen Hill!).

Just two course notes links I happen to have, though I’m sure Naty has considerably more:

Beyond Programmable Shading – an interesting course as usual. The introductory talk by Aaron Lefohn points to the rise of the ComputeShader, and Johan Andersson’s “5 Major Challenges” talk gives a thorough overview of the graphics problems currently facing cutting-edge game developers such as DICE.

Unity: iOS and Android – Cross-Platform Challenges and Solutions – from the Mobile session, these are the slides from Renaldas Zioma’s presentation about what they found while moving Unity 3D to mobile devices. I liked this talk so much I asked to host it at our site. Here’s an introduction by Renaldas to the talk itself, with some surprising audience stats.

Eventually the course notes for Advances in Real-Time Rendering in 3D Graphics and Games should also be available.

]]> 3
OpenGL Insights and more Tue, 31 Jul 2012 13:39:42 +0000 In my previous blog post I mentioned the newly-released book OpenGL Insights. It’s worth a second mention, for a few reasons:

  • The editors and some contributors will be signing their book at SIGGRAPH at the CRC booth (#929) at Tuesday, 2 pm. This is a great chance to meet a bunch of OpenGL experts and chat.
  • Five free chapters, which can be found here. In particular, “Performance Tuning for Tile-Based Architectures” is of use to anyone doing 3D on mobile devices. Most of these devices are tile-based, so have a number of significant differences from normal PC GPUs. Reading this chapter and the (previously-mentioned) slideset Bringing AAA Graphics to Mobile Platforms (PDF version) should give you a good sense of the pitfalls and opportunities of mobile tile-based architectures.
  • The book’s website has an OpenGL pipeline map page (direct link to PDF here). Knowing what happens when can clarify some mysteries and solve some bugs.
  • The website also has a tips page, pointing out some of the subtleties of the API and the shading language.

While I’m at it, here are some other worthwhile OpenGL resource links I’ve been collecting:

  • ApiTrace: a simple set of wrapper DLLs that capture graphics API calls (also works for DirectX). You can replay and examine just about everything – think “PIX for OpenGL”, only better. For example, you can edit a shader in a captured run and immediately see the effect. Also, it’s open-source and as of this writing is actively being developed.
  • ANGLE: software to translate OpenGL ES 2.0 calls to DirectX 9 calls. This package is what both Chrome and Firefox use to run WebGL programs on Windows. Open source, of course. Actually, just assume everything here is open-source unless I say different (which I won’t).
    • Edit: Patrick Cozzi (one of the editors of OpenGL Insights) notes that there are several options for WebGL on IE. “Currently, I think the best option is to use Chrome Frame. It painlessly installs without admin rights, and also brings Chrome’s fast JavaScript engine to IE. We use it on and I actually demo it on IE (including installing Chrome Frame) by request quite frequently.”
  • Microsoft Internet Explorer won’t support WebGL, so someone else did as a plug-in.
  • Equalizer: a framework for coarse-parallel OpenGL rendering (think multi-display and multi-machine).
  • Oolong and dEngine: OpenGL ES rendering engines for the iPhone and iPad. Good for learning how things work. Oolong is 90% C++, dEngine is pure C. Each has its own features, e.g. dEngine supports bump mapping and shadow mapping.
  • A bit dated (OpenGL ES 1.1), but might be of interest: a readable rundown of the ancient Wolf3D engine and how it was ported to the iPhone.
  • gl2mark: benchmarking software for OpenGL ES 2.0
  • Matrix libraries: GLM is a full-blown matrix library based on OpenGL naming conventions, libmatrix is a template library for vector and matrix transformations for OpenGL, VMSL is a tiny library for providing modern OpenGL with the modelview/projection matrix functionality in OpenGL 1.0.
  • G3D: well, it’s more a user of OpenGL, but worth a mention. It’s a pretty nice C++ rendering engine that includes deferred shading, as well as ray tracing. I use it a lot for OBJ file display.
  • OpenGL works with cairo, a 2D vector-based drawing engine. Funky.
]]> 1
Seven things for July 25 Thu, 26 Jul 2012 03:16:09 +0000 Here we go:

  • Beautiful demo of various effects, the realtime hybrid raytracing demo RIGID GEMS. Do note there are controls. The foreground blurs for the depth-of-field are a little unconvincing, but the rest is lovely! (thanks to Steve Worley for the tip)
  • Books to check out at SIGGRAPH, or now (I’m sure there are more – let me know):  OpenGL Insights and Shadow Algorithms Data Miner. Five chapters of OpenGL Insights are free to read here. There are quite a few graphics books published since last SIGGRAPH, we have them listed here.
  • Scalable Ambient Obscurance looks worthwhile, and there’s even a demo and source.
  • I can’t say I grok it all yet, but Bringing AAA Graphics to Mobile Platforms (from GDC) has a lot of chewy information on what’s fast and slow on typical mobile hardware, as well as how it works. PDF version on the Unreal Engine site.
  • A somewhat older (a whole year or so old) article on changing resolution on the fly to maintain frame rate. (Thanks, Mauricio)
  • 3D printing opens up a wide range of legal issues, It Will Be Awesome if They Don’t Screw it Up gives an overview of some of these. There are a number of areas where the law hasn’t had to concern itself yet.
  • Echo chamber: stuff you should probably know about already, but just in case. 1) Ouya, a monster money-raiser Kickstarter project for an open console. Tim Lottes comments; my take is “Android games on a console? Weak.” but I’d love to see them succeed. 2) Source Filmmaker, a free film making system from Valve. People are getting busy with it.

To conclude, a photo that looks like a rendering bug; read about it here. If you like these sorts of things, see more at the “2 True 2B Good” collection.

]]> 1
SIGGRAPH 2012 early registration deadline is today Mon, 18 Jun 2012 18:57:13 +0000 If you’re reading this after June 18th, oh well…

Registration page is here.

Me, I wouldn’t rate SIGGRAPH the premier interactive rendering research conference any more: I3D or HPG publish far more relevant results overall. SIGGRAPH still has a lot of other great stuff going on, and there are enough things of interest to me this year that I’m happy to be attending:

  • I guess the courses are the main draw for me right now, and some of these have become informal venues for interactive rendering R&D presentations (e.g. the Advances course).
  • SIGGRAPH Mobile could be interesting. Given the huge profit margins of GPUs for mobile vs. PCs, it’s where the market has moved. It feels a little “back to the future”, with GPU speeds getting reset about a decade vs. PC performance, but there’s some interesting research being done, e.g. this paper (not at SIGGRAPH but at HPG, I noticed it today on Morgan McGuire’s Twitter feed and thought it was fascinating).
  • I was thinking of arriving Sunday afternoon, but then noticed some interesting talks in the Game Worlds talks on Sunday, 2-3:30 pm.
  • Other talks will be of interest, I’ll need to wade through the list.
  • Emerging Technologies and the Exhibition Floor usually have something that grabs my attention (if nothing else, I can browse through new books), and I maybe should give Real Time Live a visit.
  • And, meeting people, of course – it’s inspiring and fun to hear what others are up to. Sometimes a little chance conversation will later have great value.
SIGGRAPH Mobile CFP Thu, 12 Apr 2012 19:44:28 +0000 So, please submit something to SIGGRAPH Mobile so that I can further justify attending this year. Deadline is May 1st.

One key research direction: making the phone no longer fit in your pocket. These are some designs to beat (click on the image for more info).

Seven things for 10/10/11 Tue, 11 Oct 2011 01:38:43 +0000
  • If you can get WebGL running properly on your browser, check out Shader Toy. Coolest thing is that you can edit any shader and immediately try it out.
  • Another odd little WebGL application is a random spaceship maker, with a direct tie-in to Shapeways to buy a 3D version of any model you make.
  • Speaking of Shapeways, I liked their “one coffee cup a day project“. The low-resolution cup is particularly good for computer graphics people, though I’m told that in real life it’s a fair bit more rounded off, due to the way the ceramic sets. Ironic. Also, note that these cups are actually quite small in real life (smaller than even espresso cups), which is too bad. Still, clever.
  • Source code for iOS versions of Castle Wolfenstein and the original DOOM is now available.
  • Patrick Cozzi has a nice rundown of his days at SIGGRAPH this August, with a particular emphasis on OpenGL and mobile. The links for each day are at the bottom of the entry.
  • Nice fractal video generated in near-real time (300 ms/frame) running a GLSL shader using this code. Reddit thread here, about an earlier video now pulled back online.
  • This site gives a darn long list of educational institutions offering videogame design degrees. It’s at least a place to start, if you’re looking for such things. That said, I’ve heard counterarguments from game company professionals to such specialized degrees, “just learn to program well and we’ll teach you the videogames business”.
  • Bonus thing: Draw a curve of your data for a number of years and see what it most closely correlates. Peculiar.

    Predicting the Past Thu, 29 Sep 2011 14:54:12 +0000 Inspired by Bing (a person, not a search engine) and by the acrobatics I saw tonight in Shanghai, time for a blog post.

    So what’s up with graphics APIs? I’ve been working on a project for a fast 3D graphics system for Autodesk for about 4 years now; the base level (which hides the various flavors of DirectX and OpenGL) is used by Maya, Max, AutoCAD, Inventor, and other products. There are various higher-level optimizations we’ve added (and why Microsoft’s fxc effect compiler suddenly got a lot slower is a mystery), with some particularly nice work by one person here in the area of multithreading. Beyond these techniques, minimizing the raw number of calls to the API is the primary way to increase performance. Our rule of thumb is that you get about 1000-1500 calls a frame (CAD isn’t held to a 60 FPS rule, but we still need to be interactive). The usual tricks are to sort by state, and to shove as much geometry and processing as possible into a single draw call and so avoid the small batch problem. So, how silly is that? The best way to make your GPU run fast is to call it as little as possible? That’s an API with a problem.

    This is old news, Tim Sweeney railed against API limitations 3 years ago (sadly, the article’s gone poof). I wrote about his ideas here and added my own two cents. So where are we since then? DirectX 11 has been out awhile, adding three more stages to the pipeline for efficient tessellation of higher-order surfaces. The pipeline’s feeling a bit unwieldy at this point, with a lot of (admittedly optional) stages. There are still some serious headaches for developers, like having to somehow manage to put lighting and material shading in the same pixel shader (one good argument for deferred lighting and similar techniques). Forget about optimization; the arcane API knowledge needed to get even a simple rendering on the screen is considerable.

    I haven’t heard anything of a DirectX 12 in the works (except maybe this breathless posting, which I feel obligated to link to since I’m in China this month), nor can I imagine what they’d add of any significance. I expect there will be some minor XBox 72o (or whatever it will be called) -related tweaks specific to that architecture, if and when it exists. With the various CPU+GPU-on-a-chip products coming out – AMD’s Fusion family, NVIDIA’s Tegra 2, and similar from other companies (I think I counted 5, all totaled) – some access costs between the two processors become much cheaper and so change the rules. However, the API still looks to be the bottleneck.

    Marketwise, and this is based entirely upon my work in scapulimancy, I see things shifting to mobile. If that isn’t at least the 247th time you’ve heard that, you haven’t been wasting enough time on the internet. But, it has some implications: first, DirectX 12 becomes mostly irrelevant. The GPU pipeline is creaky and overburdened enough right now, PC games are an important niche but not the focus, and mobile (specifically, iPad and other tablets) is fine with the functionality defined thus far by existing APIs. OpenGL ES will continue to evolve, but I doubt we’ll see for a good long while any algorithmically (vs. data-slinging) new elements added to the API that the current OpenGL 4.x and DX11 APIs don’t offer.

    Basically, API development feels stalled to me, and that’s how it should be: mobile’s more important, PCs are a (large but slowly evolving) niche, and the current API system feels warped from a programming standpoint, with peculiar constructs like feeding text strings to the API to specify GPU shader effects, and strange contortions performed to avoid calling the API in order to coax the GPU to run fast.

    Is there a way out? I felt a glimmer while attending HPG 2011 this year. The paper “High-Performance Software Rasterization on GPUs” by Samuli Laine and Tero Karras was one of my (and many attendees’) favorites, talking about how to efficiently implement a basic rasterizer using CUDA (code’s open sourced). It’s not as fast as dedicated hardware (no surprise there), but it’s at least in the same ball-park, with hardware being anywhere from 1.5x to 8.1x faster for their test cases, median being 3.6x. What I find exciting is the idea that you could actually program the pipeline, vs. it being locked away. They discuss ideas for optimization such as loosening the “first in, first out” rule for triangles currently enforced by all APIs. With its “yet another language” dependency, I can’t say I hope GPGPU is the future (and certainly CUDA isn’t, since it cuts out non-NVIDIA hardware vendors, but from all reports it’s currently the best way to experiment with GPGPU). Still, it’s nice to see that the fixed-function bits of the GPU, while important, are not an insurmountable limit in considering more flexible and general interactive rasterization programming models. Or, ray tracing – always have to stick that in there.

    So it’s “forward to the past”, looking at traditional algorithms like rasterization and ray tracing and how to gain efficiency (both in raw speed and in development time) on various modern architectures. That’s ultimately what it’s about for me, at least: spending lots of time fighting the API, gluing together strings to make shaders, and all the other craziness is a distraction and a time-waster. That said, there’s a cost/benefit calculation implicit in all of this. For example, using C# or Java is way more productive than C++, I’d say about 2x, mostly because you’re not tracking down memory problems like leaks and access uninitialized or non-existent values. But, there’s so much legacy C++ code around that it’s still the language of graphics, as previously discussed here. Which means I expect none of the API weirdness to change for a solid decade, at the minimum. Please do go ahead and prove me wrong – I’d be thrilled!

    Oh, and acrobatics? Hover your cursor over the image. BTW, the ERA show in Shanghai is wonderful, unlike current APIs.

    ]]> 2