Tag Archives: debugging

Debugging WebGL with SpectorJS

by Sebastien Vandenberghe

With the emerging number of experiences built using WebGL, and all the improvements made in the WebVR/AR space, it is critical to have efficient debugging tools. Whether you are just starting out or are already an experienced developer of 3D applications with WebGL, you likely know how tools can be important for productivity. Looking for such tools, you probably came across Patrick Cozzi’s blog post highlighting the most common ones. Unfortunately, many of these tools are no longer compatible with your project, due to missing WebGL2 features or extensions, such as draw buffers, 3D textures, and so on.

As a core contributor to BabylonJS, working at the engine level, on a daily basis I need to see the entire creation of frames, including all the available information from the WebGL state (Depth, Stencil, Blend, etc.) as well as the list of commands along with their arguments. In order to optimize the engine, I also need information and statistics about memory, draw calls, and primitives. These desires were a big motivation for me to develop SpectorJS. And as we love the WebGL community we decided to make it an Open Source Project, compatible with all existing WebGL 3D engines.

At the end of this walkthrough, you will be able to easily capture and inspect any WebGL frames rendered in your favorites applications. If you have any issues, do not hesitate to report them on Github. To stay informed of all the new features, follow us at @SpectorJS.

 

Table of Contents

Installation

Always looking to save time, the tool is directly available as a browser extension: ChromeFirefox – (more browsers are coming soon)

Embedding the library in your application or side-loading the extension are also possible. More information can be found on Github.

Basic Usage

Once installed, you can now on navigate to any website using WebGL, such as the Babylon JS playground, and you will notice the extension Icon turning red in the toolbar.

This highlights the presence of a canvas with a 3D context in the page or its embedded IFrames. Pressing the toolbar button reloads the page and the icon turns green, as Spector is now ready to capture. During the refresh Spector injects additional debug code that collects state and command information, along with other statistics.

Note: We do not enable it by default, so as to not interfere with any WebGL program unless explicitly requested.

Clicking this green button will display a popup helping you to capture frames.

Following the on-screen instructions and clicking the red circle will trigger a capture. If a canvas is selected, you can also, in this menu, pause or play  the rendered canvas frame by frame. Once the capture has been completed, a result panel will be displayed containing all the information you may need.

The bottom of the menu helps capturing what is happening during the page load on the first canvases present in the document. You can easily choose the number of commands to capture, as well as specify whether or not you would like to capture transient context (context created in the first canvas, even if not part of the DOM).

 

Note: A few reasons might prevent you capturing the context, the main one being that nothing is rendered if the scene is fully static. If this happens, moving the camera after pressing the capture button should be enough to start the capture.

Note: As collecting the information is pretty expensive, the capture may take a long time and you might have to press wait... a few times when the browser notifies you that the page is unresponsive. Unfortunately, we cannot work around this, as the capture needs to happen synchronously during the execution of your code. Without a synchronous capture, the rest of your code continues to react to external events with potential side effects on the capture.

Capture View

On the left side of the screen are displayed all the different visual state changes happening during the creation of the frame. They are displayed alongside their target frame buffer information. This helps to quickly understand how the frame has been built during troubleshooting sessions. Selecting one of the pictures automatically selects the command associated with it. The visual capture handles all the possible renderable outputs such as cube textures, 3D textures, draw buffers, render target texture, render buffers and so on.

The central panel is the commands panel. It displays the list of commands that were executed on the captured context during the frame. These are displayed chronologically. A color code is used to highlight issues and identify draw calls:

  • Orange Background: The selected command.
  • Blue Background: Draw Calls or Clear commands.
  • Green Command Name: Valid Commands (changing state to a new value).
  • Orange Command Name: Redundant Commands (meaning the value applied is the same as the current one which is useful to optimize a WebGL application)
  • Red Command Name: Deprecated WebGL Commands.

Selecting a command leads to display on the right side all of its detailed information including the command name, arguments, and JavaScript call stack. If a draw call has been selected, the various states involved in this call are all available. This is usually a pretty long list of information, as the captures contains the exhaustive list of states, attachments, programs, shaders, attributes, VAOs, uniforms, UBOs, transform feedbacks, and their attached properties. From this panel, the shader source code is also available from the program information, by following the Click to open link:

This opens a beautified view of the shader code, helping to ensure the defines and the code itself are as expected:

Note: Some information might be empty if there is an issue in the engine. For instance, unbound textures might lead to empty uniform information for the sampler. This is usually an interesting warning and more analytics are in progress to help highlight such use cases better.

A few other views are available for each capture.

Init and End State

Once a capture is open, the top command bar includes links to the initial and final state of the capture. This is useful to see how is the context was before the capture and at the end, to help deal with issues happening between frames, for instance.

Context and Frame Information

Commonly there are issues in WebGL applications related to either the canvas or the context setup. To be sure the current setup is correct, the information panel displays all the queryable information. This also contains statistics about the captured frame such as memory information, number of calls of each command, and drawn primitive information.

Sharing Captures

Since we often collaborate with others on projects or use multiple platforms, it is critical to be able to save and share captures. To do this, you can simply navigate to the Captures link in the menu, where all the captures of the session have been stored. Clicking on the floppy icon (nostalgia FTW) downloads the captured JSON file.

To open and view this file, Drag and Drop it on the Extension popup or the Capture list dedicated area. This feature can save a lot of time troubleshooting customer or cross-platform issues.

How to Compare Captures

As it is needed more often than anybody would like, comparing captures after an engine change is a must-have. A full capture comparison is currently under development, but in the meantime, captures can at least be put in different tabs of the browser, making it easier to check differences.

Checking the box in the popup menu forces the next capture to open in a new tab:

Custom Data

Displaying custom information is a nice trick to quickly identify the relationship between a material and its shader or between a mesh and its buffers. Adding custom data to the capture is achievable by adding a special field named __SPECTOR_Metadata to any WebGLObject. Once the field has been set, any command relying on this object displays the related metadata in the property panel.

javascript var cubeVerticesColorBuffer = gl.createBuffer();
cubeVerticesColorBuffer.__SPECTOR_Metadata = { name: "cubeVerticesColorBuffer" };

This enables the visibility of the custom name “cubeVerticesColorBuffer” in the capture Metadata wherever the buffer is in use.

Extension Control

Another interesting feature is the ability to drive the extension by code. Once the extension is enabled, from your browser’s dev tools, or even your code, you can call the following APIs on “spector.”:

  • captureNextFrame(obj: HTMLCanvasElement | RenderingContext) : Call to begin a capture of the next frame of a specific canvas or context.
  • startCapture(obj: HTMLCanvasElement | RenderingContext, commandCount: number) : Start a capture on a specific canvas or context. The capture will stop once it reaches the number of commands specified as a parameter, or after 10 seconds.
  • stopCapture(): ICapture : Stop the current capture and returns the result in JSON. It displays the result if the UI has been displayed. This returns undefined if the capture has not been completed or did not find any commands.
  • setMarker(marker: string) : Adds a marker that is displayed in the capture, helping you analyze the results.
  • clearMarker() : Clears the current marker from the capture for any subsequent calls.

The “spector” object is available on the window for this purpose.

This can be a tremendous help to capture the creation of your shadow maps, for instance. This can also be used to trigger a capture based on a user interaction or to set markers in your code to better analyse the capture.

The following example could be introduced safely in your code:

if (spector) {
    spector.setMarker("Shadow map creation");
}
[your shadow creation code]
if (spector) {
    spector.clearMarker();
}

Using the Standalone Version

If you prefer to use the library in your own application you can find it available on npm: spectorjs

Going Further

This extension being pretty new and under active development, a few features have been discussed for the next releases:

  • Capture Comparison
  • Image Comparison
  • Remote Debugging
  • Shader Editor
  1. Website
  2. Github
  3. Roadmap
  4. Report Issues
  5. Twitter @SpectorJS

I would like to particularly thank Eric Haines for the time spent to review the article, knowing the challenge it represents considering my English 🙂

OpenGL Insights and more

In my previous blog post I mentioned the newly-released book OpenGL Insights. It’s worth a second mention, for a few reasons:

  • The editors and some contributors will be signing their book at SIGGRAPH at the CRC booth (#929) at Tuesday, 2 pm. This is a great chance to meet a bunch of OpenGL experts and chat.
  • Five free chapters, which can be found here. In particular, “Performance Tuning for Tile-Based Architectures” is of use to anyone doing 3D on mobile devices. Most of these devices are tile-based, so have a number of significant differences from normal PC GPUs. Reading this chapter and the (previously-mentioned) slideset Bringing AAA Graphics to Mobile Platforms (PDF version) should give you a good sense of the pitfalls and opportunities of mobile tile-based architectures.
  • The book’s website has an OpenGL pipeline map page (direct link to PDF here). Knowing what happens when can clarify some mysteries and solve some bugs.
  • The website also has a tips page, pointing out some of the subtleties of the API and the shading language.

While I’m at it, here are some other worthwhile OpenGL resource links I’ve been collecting:

  • ApiTrace: a simple set of wrapper DLLs that capture graphics API calls (also works for DirectX). You can replay and examine just about everything – think “PIX for OpenGL”, only better. For example, you can edit a shader in a captured run and immediately see the effect. Also, it’s open-source and as of this writing is actively being developed.
  • ANGLE: software to translate OpenGL ES 2.0 calls to DirectX 9 calls. This package is what both Chrome and Firefox use to run WebGL programs on Windows. Open source, of course. Actually, just assume everything here is open-source unless I say different (which I won’t).
    • Edit: Patrick Cozzi (one of the editors of OpenGL Insights) notes that there are several options for WebGL on IE. “Currently, I think the best option is to use Chrome Frame. It painlessly installs without admin rights, and also brings Chrome’s fast JavaScript engine to IE. We use it on http://cesium.agi.com and I actually demo it on IE (including installing Chrome Frame) by request quite frequently.”
  • Microsoft Internet Explorer won’t support WebGL, so someone else did as a plug-in.
  • Equalizer: a framework for coarse-parallel OpenGL rendering (think multi-display and multi-machine).
  • Oolong and dEngine: OpenGL ES rendering engines for the iPhone and iPad. Good for learning how things work. Oolong is 90% C++, dEngine is pure C. Each has its own features, e.g. dEngine supports bump mapping and shadow mapping.
  • A bit dated (OpenGL ES 1.1), but might be of interest: a readable rundown of the ancient Wolf3D engine and how it was ported to the iPhone.
  • gl2mark: benchmarking software for OpenGL ES 2.0
  • Matrix libraries: GLM is a full-blown matrix library based on OpenGL naming conventions, libmatrix is a template library for vector and matrix transformations for OpenGL, VMSL is a tiny library for providing modern OpenGL with the modelview/projection matrix functionality in OpenGL 1.0.
  • G3D: well, it’s more a user of OpenGL, but worth a mention. It’s a pretty nice C++ rendering engine that includes deferred shading, as well as ray tracing. I use it a lot for OBJ file display.
  • OpenGL works with cairo, a 2D vector-based drawing engine. Funky.

gDEBugger is now free!

Just noticed this on Morgan McGuire’s twitter feed. I don’t know why, but gDEBugger, sort of the PIX equivalent for OpenGL, is now free, go here for a license. They’ll be putting out a newer free version (5.8) by the end of the year, so it’s not like they’re discontinuing the product. Maybe it’s the “get them hooked” business model. Also, there’s talk that the current version doesn’t work that well with OpenGL 3.2 and above. Nonetheless, it’s an excellent product overall. Anyway, screen shots here.

To quote their literature: gDEBugger is an OpenGL, OpenGL ES, and OpenCL Debugger, Profiler and memory analyzer. It traces application activity on top of the OpenGL API to provide the application behavior information you need to find bugs and to optimize application performance. gDEBugger transforms the debugging task of graphic application from a “Black box” into a White box model; using gDEBugger you can peer inside the OpenGL usage to see how individual commands affect the graphic pipeline implementation. gDEBugger has a lot of “standard debugger” abilities, but also contains many special features for graphics software developers: view render context state variables, view allocated textures, textures properties and image data, Shaders programs and source code, break on OpenGL errors. In addition, using its profiling abilities, gDEBugger enables you to pinpoint easily the exact location of the application’s graphic pipeline performance bottleneck to let you optimize the application performance.

Update: Jari Komppa wrote, “This may shed some light on things: http://www.export.gov.il/Eng/_Articles/Article.asp?CategoryID=461&ArticleID=12274

Full text:

AMD to buy Israel’s Graphic Remedy company

The American chip manufacturer AMD is buying Israel’s Graphic Remedy company, the Calcalist financial website reports.

It appears that AMD – Intel’s competitor in manufacturing PC and server chips – will pay a relatively low amount for Graphic Remedy, some $4-5 million.

Graphic Remedy, founded six years ago, is a small company with seven employees. It gained renown for its series of simulation and debugging applications for graphic programs and computer games and became dominant among Cronus’ [sic – they mean Khronos Group’s] Open GL platform developers.

According to Calcalist, AMD seems to be buying Graphic Remedy in an attempt to expand its presence in the home and business graphic
processors market.

Clearing the Queue (a little)

Well, let’s see how far I get tonight in clearing the backlog of 219 potential resources I’ve stored up. Here goes:

  • NShader – If you use MSVC and you write shaders, this one’s for you. It highlights shader text as you’d expect, highlighting function names correctly and generally making code more readable. Worthwhile, I’ve installed it, it’s fine. That said, you can get 90% of the way there (and for sure 100% virus free) by simply using “Options | Text Editor | File Extension” and setting extension .fx (.fxh, etc.), choosing Microsoft Visual C++, then clicking Add. Do it now.
  • Speaking of shaders, I lost much of a day tracking down a bug in Cg: writing code like this, “max(0,someVar);” gave different results than HLSL when someVar was a float. My advice: always use the floating point version of numbers in shaders. So “max(0.0f,someVar);” fixed the problem. Reported to NVIDIA.
  • Morgan McGuire pointed out that John Carmack now has a Twitter account. It’s pretty interesting, in that he’s writing a ray tracer in OpenCL and relearning or rederiving various bits of knowledge that are not really written down anywhere. The guy’s unnervingly productive: “Goal for today: implement photon maps and contrast with my current megatexture radiosity gathering.” But what will he do after lunch?
  • Speaking of Carmack, you must see the Epic Citadel demo for the iPad. Demo video here. Stunning.
  • Speaking of Morgan, his Twitter feed mentioned a number of new resources: a new demo (with complete source) of ambient occlusion volumes at NVIDIA, a demo of sample distribution shadow maps (optimized z partitions for cascading maps) at Intel, and an introduction to DX 11 at Gamasutra. He also points out some entertaining visual bits, like the fascinating style of Devil’s Tuning Fork and a game map scale comparison chart (wow, WoW is small!). Morgan and others’ free multi-platform G3D Innovation Engine is now in release 8.0, and is supposed to be good for students, researchers, and indie game developers.
  • Speaking of Intel, they actually have three new DirectX 11 demos with source, as noted in this Geeks3D article.
  • Implementing some form of an A-buffer (multiple fragments stored in a pixel) is becoming more common. It’s an algorithm that can perform antialiasing and, more importantly, order-independent transparency. We already mentioned AMD’s efforts using DirectX 11; Cyril Crassin has taken their idea and improved cache coherency a bit, creating an OpenGL 4.0+ version of the linked-list approach, with source.
  • I love the idea of driving rendering computations by drawing some simple quad or whatever on the screen and having the pixel shader figure out where, at what depth, and what normal the surface is actually located. The quad is like a bounding volume, the pixel shader essentially a ray caster. Naty sent on this link: the game Hustle Kings uses this technique to great effect for rendering its billiard balls – gorgeous. Video here. Perfect spheres here look lovely. I have to wonder how many triangles in a mesh would be visually identical at closest approach; it seems like the eye is quite good at picking up slight irregularities in tessellated spheres when they are rotating.
  • There’s also a pretty displacement mapping demo in OpenGL 4.0 showing GPU tessellation at work.
  • I like seeing that 3D printers are becoming still cheaper yet, $1500, and this one looks like a fairly clean system (vs. cornstarch dust everywhere, etc.).
  • There’s a basic object/object intersection library now available for XNA, GeometricIntersection.
  • Blending Terrain Textures is a nice little article on just that. Lerping bad, thresholding good.
  • Valve discusses how it worked with vendors to optimize graphics performance on the Mac.
  • NASA provides a fair number of 3D models free to download – 87 models and counting.
  • HWiNFO32 is a free utility which provides tons of information about your system. The thing that appeals to me is that it appears to show GPU memory load over time (and I verified the load indeed increased when I ran a game). I hadn’t seen this before, and thought it was essentially impossible for Vista, so this makes me pretty happy. I did just notice that my desktop icons all turned to generic white document icons and are unusable, and this happened some time around when I tried this utility. Hmmm.
  • Colors in Cultures infographic – I like the concept, it could have been interesting, but it’s mostly just hard to read in this form. Lots of world maps with the countries colored would have perhaps revealed more. Anyway, there’s their data, for remixing.

Now I’m down to 196 potential resources – good! More later.