Resources

You are currently browsing the archive for the Resources category.

by Sebastien Vandenberghe

With the emerging number of experiences built using WebGL, and all the improvements made in the WebVR/AR space, it is critical to have efficient debugging tools. Whether you are just starting out or are already an experienced developer of 3D applications with WebGL, you likely know how tools can be important for productivity. Looking for such tools, you probably came across Patrick Cozzi’s blog post highlighting the most common ones. Unfortunately, many of these tools are no longer compatible with your project, due to missing WebGL2 features or extensions, such as draw buffers, 3D textures, and so on.

As a core contributor to BabylonJS, working at the engine level, on a daily basis I need to see the entire creation of frames, including all the available information from the WebGL state (Depth, Stencil, Blend, etc.) as well as the list of commands along with their arguments. In order to optimize the engine, I also need information and statistics about memory, draw calls, and primitives. These desires were a big motivation for me to develop SpectorJS. And as we love the WebGL community we decided to make it an Open Source Project, compatible with all existing WebGL 3D engines.

At the end of this walkthrough, you will be able to easily capture and inspect any WebGL frames rendered in your favorites applications. If you have any issues, do not hesitate to report them on Github. To stay informed of all the new features, follow us at @SpectorJS.

 

Table of Contents

Installation

Always looking to save time, the tool is directly available as a browser extension: ChromeFirefox – (more browsers are coming soon)

Embedding the library in your application or side-loading the extension are also possible. More information can be found on Github.

Basic Usage

Once the extension has been installed in the browser, navigating to any website leveraging WebGL or WebGL2 3D context such as the Babylon JS playground enables the Spector icon, which turns red in the toolbar.

This highlights the presence of a canvas with a 3D context in the page or its embedded IFrames. Pressing the toolbar button reloads the page and the icon turns green, as Spector is now ready to capture. During the refresh Spector injects additional debug code that collects state and command information, along with other statistics.

Note: We do not enable it by default, so as to not interfere with any WebGL program unless explicitly requested.

Once ready, a click on the button displays the popup containing the capture toolbar.

Following the on-screen instructions and clicking the red button starts a capture. This menu also controls pausing or playing the program frame by frame for the selected canvas. Once the capture has been completed, the result panel is displayed. It contains all the information you may be looking for; if not, do not hesitate to create a feature request ūüôā on Github, we’ll be happy to address it, especially if it will help others.

 

Note: A few reasons might prevent you capturing the context, the main one being that nothing is rendered if the scene is fully static. If this happens, moving the camera after pressing the capture button should be enough to start the capture.

Note: As collecting the information is pretty expensive, the capture may take a long time and you might have to press wait... a few times when the browser notifies you that the page is unresponsive. Unfortunately, we cannot work around this, as the capture needs to happen synchronously during the execution of your code. Without a synchronous capture, the rest of your code continues to react to external events with potential side effects on the capture.

Capture View

On the left side of the screen are displayed all the different visual state changes happening during the creation of the frame. They are displayed alongside their target frame buffer information. This helps to quickly understand how the frame has been built during troubleshooting sessions. Selecting one of the pictures automatically selects the command associated with it. The visual capture handles all the possible renderable outputs such as cube textures, 3D textures, draw buffers, render target texture, render buffers and so on.

The central panel is the commands panel. It displays the list of commands that were executed on the captured context during the frame. These are displayed chronologically. A color code is used to highlight issues and identify draw calls:

  • Orange Background: The selected command.
  • Blue Background: Draw Calls or Clear commands.
  • Green Command Name: Valid Commands (changing state to a new value).
  • Orange Command Name: Redundant Commands (meaning the value applied is the same as the current one which is useful to optimize a WebGL application)
  • Red Command Name: Deprecated WebGL Commands.

Selecting a command leads to display on the right side all of its detailed information including the command name, arguments, and JavaScript call stack. If a draw call has been selected, the various states involved in this call are all available. This is usually a pretty long list of information, as the captures contains the exhaustive list of states, attachments, programs, shaders, attributes, VAOs, uniforms, UBOs, transform feedbacks, and their attached properties. From this panel, the shader source code is also available from the program information, by following the Click to open link:

This opens a beautified view of the shader code, helping to ensure the defines and the code itself are as expected:

Note: Some information might be empty if there is an issue in the engine. For instance, unbound textures might lead to empty uniform information for the sampler. This is usually an interesting warning and more analytics are in progress to help highlight such use cases better.

A few other views are available for each capture.

Init and End State

Once a capture is open, the top command bar includes links to the initial and final state of the capture. This is useful to see how is the context was before the capture and at the end, to help deal with issues happening between frames, for instance.

Context and Frame Information

Common there are issues in WebGL applications related to either the canvas or the context setup. To be sure the current setup is correct, the information panel displays all the queryable information. This also contains statistics about the captured frame such as memory information, number of calls of each command, and drawn primitive information.

Sharing Captures

Since we often collaborate with others on projects or use multiple platforms, it is critical to be able to save and share captures. To do this, you can simply navigate to the Captures link in the menu, where all the captures of the session have been stored. Clicking on the floppy icon (nostalgia FTW) downloads the captured JSON file.

To open and view this file, Drag and Drop it on the Extension popup or the Capture list dedicated area. This feature can save a lot of time troubleshooting customer or cross-platform issues.

How to Compare Captures

As it is needed more often than anybody would like, comparing captures after an engine change is a must-have. A full capture comparison is currently under development, but in the meantime, captures can at least be put in different tabs of the browser, making it easier to check differences.

Checking the box in the popup menu forces the next capture to open in a new tab:

Custom Data

Displaying custom information is a nice trick to quickly identify the relationship between a material and its shader or between a mesh and its buffers. Adding custom data to the capture is achievable by adding a special field named __SPECTOR_Metadata to any WebGLObject. Once the field has been set, any command relying on this object displays the related metadata in the property panel.

javascript var cubeVerticesColorBuffer = gl.createBuffer();
cubeVerticesColorBuffer.__SPECTOR_Metadata = { name: "cubeVerticesColorBuffer" };

This enables the visibility of the custom name “cubeVerticesColorBuffer” in the capture Metadata wherever the buffer is in use.

Going Further

This extension being pretty new and under active development, a few features have been discussed for the next releases:

  • Capture Comparison
  • Image Comparison
  • Remote Debugging
  • Shader Editor
  1. Website
  2. Github
  3. Roadmap
  4. Report Issues
  5. Twitter @SpectorJS

I would like to particularly thank Eric Haines for the time spent to review the article, knowing the challenge it represents considering my English ūüôā

Tags: , , ,

I made a page of thumbnail images of the 297 three.js examples. Here it is:

http://www.realtimerendering.com/threejs/

The three.js site used to have a page like this. I’m not sure why it disappeared, but now I don’t care, as I can more easily find demos I’ve looked at before but then forgot the names.

Bonus links: Stemkoski and Yomotsu also have useful demo pages, which used to be prominently linked from the three.js site but now are not.

Tags:

WebGL Links Page

I got tired of re-finding various useful WebGL and three.js links, so I made a page:

http://www.realtimerendering.com/webgl.html

What cool things am I missing?

I’ve made it a page of links I am likely to want to check out in the future. It’s a bit hard to draw the line. For example, I didn’t bother adding fun demos such as¬†this and this, but I did add the page where I browse new demos. I don’t list development systems such as Goo Create¬†for non-programmers, which is built on this open-source WebGL engine and has some interesting features. Nice things all, but I personally am unlikely to come back to them (or if I do, they’re now in this blog post).

Tags: ,

Some staff at Taylor & Francis kindly dug up some of the¬†supplemental materials (mostly code)¬†for the¬†journal of graphics tools, namely, volumes 10-13. I’ve waded through it all and added these resources to the code repository:

Github JGT repository

If you have code from a JGT article that’s not listed here, please do send it on to me and I’ll add it.

Tags: ,

And, it’s over – Springer appears to have shut the gates a day later. Mistake? Buzz-generating marketing ploy? Who knows? I’ll leave the rest of the post intact, but books¬†are no longer free. Some articles are, such as Knuth’s.

All books from Springer that are ten years old or older are free, go look.

Quoting¬†V√≠t Tuńćek¬†here, from Facebook (reposted by Pete Shirley):

Springer has made a lot of math & physics books available online, for free! Everything that is more than 10 years old.

If you don’t know which book you may want you can start here¬†http://mathoverflow.net/questions/tagged/books

This links to the Graduate Texts in Mathematics series: https://t.co/R1EYrTrz5w

This is for all materials (books, journals, chapters, articles) from all fields: http://goo.gl/cB5rRc

This excellent computational geometry book is available (2nd Edition; the latest, 3rd edition costs money), as is this older-but-worthwhile one. For de Berg’s work,¬†the free version is the second edition; other than these errata fixes, the 3rd edition’s major changes are that Chapter 7 includes information on Voronoi diagrams of line-segments and for farthest point, and Chapter 12 includes BSP trees for low-density scenes.

There are also older computer graphics related books, e.g. this one¬†and this. Ancient, but the price is right, and some of this stuff doesn’t change.

Handy list of direct links for the math & physics PDFs here.

Me, I’m digging around for various recreational math books. One of my favorite books, period, is here: One Jump Ahead. There’s a recreational math book, Tracking the Automatic Ant, a collection from the Mathematical Intelligencer.¬†Some bits of newer¬†stuff from the Mathematical Intelligencer is also available, e.g., an article on¬†mathematical vanity plates¬†by Knuth, of all people. Some books I can’t find, as Springer’s searcher is pretty wonky, e.g. The Science of Cooking appears to somehow not exist, though there’s a short article¬†available by the author.

Happy hunting, and email me or let us know in the comments if you find any other gems related to computer graphics.

Once upon a time the¬†Journal of Graphics Tools¬†had code associated with many of its articles. This was in fact one of the selling points for the journal, which grew out of the Graphics Gems series of books. When Taylor & Francis acquired A.K. Peters back in 2010, they moved the journal to their vast collection of other journals¬†(2671 and counting). The code repository didn’t fit their web template, so they no longer hosted the code. At the time I wasn’t so concerned, as the Wayback Machine¬†mirrored the abstracts and code collection of the old A.K. Peters site. Something happened this year¬†and those backup pages are now gone, e.g., this link used to work.

So, time to rebuild. I’ve maintained the Graphics Gems repository for a few decades; how hard could it be to rebuild a recently-lost code archive? Well, we’ll see. I’ve written Taylor & Francis in hopes that someone knows where they put that DVD or whatever with the code archive. Fingers crossed. In the meantime I’ve been looking around and asking around. Here’s what I’ve collected so far:

“Journal of Graphics Tools” Code Repository

Enjoy what bits are there, and please send me any code you’ve saved that is¬†related to a JGT article. You don’t have to be an author, just a pack-rat. I know that at least one author I asked could not find a backup of his article’s code. I personally can relate: back in 1985 I finished my master’s thesis. A few months later I realized I should get copies of images from my thesis work (in the Utah RLE format – ahhh, memories; astoundingly, that site is still around, things put on cs.utah.edu appear to stay there forever). One backup tape was glitched, so I lost about half my images.

Time for an¬†analogy with the Library of Alexandria¬†(pointed out to me by Jason Mitchell). Go read that article, it’s short, and makes an excellent point. Shorter still is a tweet by the same person¬†which shows the practical effect of our general lack of redundancy. Gamasutra/Game Developer code repository? Gone (AFAIK). The lovely ompf.org forum about real-time ray tracing? Gone. Various game and film company article collections, various useful blogs, various cool resources? Gone baby¬†gone. I encounter this loss every time I update our portal page¬†or the ACM TOG resources page. Of course the “portal” term itself is at least a decade out of style (remember¬†Yahoo?), but knowing where to find the good stuff is still valuable. When some bit of the good stuff goes away, how sad. Sure, there’s Sturgeon’s Law, but the 10% also sometimes disappears.

I’ve had this vague feeling for decades, ever since I started to collect bits and pieces for the Ray Tracing News in 1987, that I’m playing the role of¬†a medieval monk attempting to keep some small bits and pieces of knowledge from disappearing. In fact, the Ray Tracing News archive briefly¬†disappeared when ACM TOG reorganized their site; I moved it to realtimerendering.com. My takeaway for internet resources is “trust no one”, not even myself, since I probably don’t have an infinite life span. The Bret Victor article¬†I mentioned last paragraph (is the article still available? I guess it depends when you read this posting…) points out the problem, but I know of no good solution right now. The Internet Archive’s Wayback Machine is related to Bibliotheca Alexandrina, which is something like naming your new ship the Titanica. The irony, she drips, that this archive somehow lost the Journal of Graphic Tools’ code (update: my guess is someone recently popped in a robots.txt on the dead site). I’m of course now kicking myself for not making a copy of JGT’s code base myself back when I had the chance.

Maybe a better takeaway is “trust everyone.” My one positive suggestion is “make it a zip.” If you have a resource, make the whole site downloadable as one big zip so that it’s easy for you and others to back up. Github offers this automatically. If zipping¬†had been done with JGT’s site, I’d probably have a copy. I have a sort of future sadness for the day Paul Bourke’s¬†or Steve Hollasch’s sites¬†disappear. There are utilities such as HTTrack¬†where one could grab a whole web site, but who wants to mess with that? Well, I now do, but for Hollasch it won’t matter all that much, since he links to lots of external pages (by the way, Xenu is a wonderful free dead link checker). Steve Hollasch’s¬†site, 76 files, sum up to 798 kb – so much more valuable than just about any iPhone photo, which is more than twice as large. Bourke’s site, which includes many PDFs and zips and whatnot, is quite large – it’s still downloading – it might be¬†the size of a season of Rick and Morty¬†(which¬†BTW¬†is a great show). But, once done downloading,¬†I’ll have it, so “don’t you¬†die on me, man” is then a non-issue, at least for me. At a price¬†of less than a nickel per gigabyte, who cares about the cost of storage for code and text?

Sure, a lot of information dies off that absolutely no one will ever care about. But some of the seemingly-useless bits kept around are wonderful to have. Me, I just found the shower that came with our condo-ized piece of the old Victorian we live in – very helpful!¬†It’s said that 90% of the files in any file cabinet are never accessed again. But which 10% are the useful ones? So we naturally keep all the files around. Digital storage is much more compact and painless, yet still easily gets wiped out due to a disk head failure, power surge, or the owner passes away. By the way, if you use Google Docs or Gmail or etc., go set up your Inactive Account Manager right now. On that morbid but practical note, please do remember to send me any code bits you might have from the Journal of Graphics Tools. And do us all a¬†favor and download a zip of the JGT code repository every now and then, just in case Github goes away (I’m thinking of¬†Google Code¬†here, but¬†even BitBucket gets DDOS’ed).

Not so fun fact:¬† 27.5% of the external links on our references page¬†are dead, after 7 years.¬†They’re mostly papers, so perhaps the links have just changed or, at worst, the article can be dug up somewhere (though that’s only true half the time – a lot of game-related articles are not in journals).

Tags: ,

Here it is¬†– you should recognize the third author’s name for sure. I’ve only skimmed a bit, but wow, very interesting. Many of the figures are indeed interactive, which is magical. I also like the idea that you can move the mouse over a term and a pop-up shows more information about it. Seriously, go check out a section right now.

Tags:

More stuff:

  • New interactive 3D graphics books at SIGGRAPH 2015: WebGL Insights, GPU Pro 6¬†(Kindle right now, hardcover in September). Let me know if I missed anything (see full list here, which also includes links to Google Books previews for these new books).
  • Updated book: 7th edition of the OpenGL SuperBible. I would guess that, with Vulkan coming down the pike, and Apple going with Metal and no longer developing OpenGL (it’s back¬†in 2010 at 4.1 in Mavericks), this will be the final edition. Future¬†students having to learn Vulkan or DirectX 12, well, that won’t be much fun at all…
  • I mentioned yesterday how you can download the SIGGRAPH 2015 Proceedings for free this week. There’s more, in theory. Some¬†of the links there have nothing as of right now. The Posters are worth a skim, especially since I didn’t see them at SIGGRAPH.¬†I also liked the Studio PDF. It starts with¬†a bunch of single-page talks that are fun to snack on, followed by a few random slidesets. Emerging Tech also has longer descriptions than on the ETech page¬†(which has more pics and videos, however). If you gotta catch ’em all, there’s also a PDF for¬†Panels.
  • There have been many news articles recently about not viewing screens at bedtime. Right, sure. Michael Herf (former CTO at¬†Picasa) is the president at¬†f.lux, one company that makes¬†screens vary in overall spectra during the day to ameliorate the problem. He pointed me at a¬†useful-to-researchers bit: their¬†fluxometer site, with spectra for many different displays, all downloadable.
  • Oh, and related, a tip from Michael: Pantone stickers with differing colors (using metameric failure)¬†under different temperature lights, so you can ensure you’re showing work under consistent lighting conditions.
  • I was impressed by HALIDE, an MIT licensed open source project for writing high performance image processing code (including GPU versions) from scratch.¬†Most impressive is their case study for local Laplacian filters (p. 28), showing great performance with considerably less code and time coding vs. Adobe Photoshop’s efforts. Google and others use¬†it extensively (p. 32).
  • Path tracing is¬†all the rage for the film industry; the Arnold renderer started it (AFAIK) and others have followed suit. Here’s an entertaining¬†path trace of interior lighting for a Minecraft scene using the free¬†Chunky path tracer.¬†SPP is samples per pixel:

Chunk progressive render

Tags: , , , , ,

Seven things:

  • Stephen Hill’s great collection of SIGGRAPH 2015 links.
  • As he and others have noted, the entire SIGGRAPH 2015 proceedings is available to all for free download until the end of this week. Grab it now if you’re not a SIGGRAPH member. SIGGRAPH members always have Digital Library access to SIGGRAPH-sponsored conferences, even if not Digital Library subscribers, e.g. here’s the SIGGRAPH 2014 proceedings.
  • New term: froxel¬†– frustum voxel.¬†Alex Evans mentioned¬†it in his fascinating talk in the Advances in RTR course; on page 83 he notes, “The term originated at the Sony WWS¬†ATG group, I believe.”¬†Diagram. He’s semi-right that Shadertoy programs do¬†ray marching through froxels at their simplest; a speedup for Shadertoy is using the minimum distance-field distance found to any object as a minimum step size (e.g., line 126 of this demo, most of which they live-coded during the wonderful Shadertoy studio workshop).
  • Evidence that patents appear to not spur research and innovation, even for big pharma. I like The Economist, as it tries to weigh the evidence for & against some idea, vs. knee-jerking it one way or the other.
  • Folklore 1: Jim Blinn confirmed that the teapot model was scaled¬†down vertically because it looked nicer that way, not that the pixels were non-square (incorrectly propagated here and here). Jim & 3D printed teapot.
  • Which reminds me: here’s my random set of pics from SIGGRAPH 2015, with captions. I like, “Hundreds of beautiful designs, and only one or two that suck.” Update:¬†more photos from Mauricio Vives, along with WebGL specific shots. Need more? Everyone’s.
  • Folklore 2: (Updated and corrected)¬†Rendering equation:¬†Kajiya’s used S¬†as a subscript, in PoDIS Glassner used an¬†omega symbol because it looks like a hemisphere, since¬†that’s what was being integrated over. Wikipedia¬†uses it.

unnamed1

Seven more tomorrow.

Tags: , , ,

Andrew Glassner and I are running a fun little workshop called “Freezing Time” this Sunday, as part of Making @ SIGGRAPH. Details: 12:15-1:45 PM,¬†South Hall G – Studio Workstation Area

We’ll be teaching how to use T2Z, “Time To Z”, a program that lets you generate a 2D animation and then turn it into a 3D printable sculpture. Participants will be provided workstations, and there will be high-speed 3D printers available after the workshop. Can’t make it? Read on… Can make it? Get the code now and have fun on the plane ride to Los Angeles.

T2Z takes the frames of your animation and stacks them to form a 3D sculpture.¬†This three.js program shows the transition for a number of animations¬†– use the mouse to¬†change the camera’s view. It’ll also get your cooling fan cranking, if your GPU is like mine. Turning “cycles” down to 1 keeps it sane.

Here’s a simple example. This animation:

squib animation

gives this sculpture when you stack the frames:

squib sculpture

The (self-imposed) challenge is to create an interesting, looping animation that also creates a visually-pleasing and printable sculpture. This example is pretty good, though the animation doesn’t quite perfectly loop. It would be easy enough to make it loop, but then we lose the base it can sit on. Tricky! If you want to hack on this code, it’s the Wobbly animation in T2Z.

There are many more examples in our gallery. I’ve been playing with the idea of data translation in general; you’ll see some experiments there. It’s been a great excuse for me to learn to use various tools at the local makerspace, Artisan’s Asylum, though I’ve not worked up the courage to actually use the plasma cutter yet. There are also plenty of fun & free tools for data manipulation, such as 123D Make, third-generation Photosynth, Sketchfab, 123D Catch, and on and on.

Even if you can’t attend the workshop, you can easily do this sort of experimentation at home. The T2Z code is free and open source, and well-documented. Companies such as Shapeways give you the ability to print high-quality models. We have lots of little animations in Animations.pde – go mess with them! There are also super-hacky “animations” at the end of this file:¬†AnimatedGifReader turns a GIF into an STL 3D print file,¬†FolderOfFramesReader does the same for a set of PNGs, and¬†HeightField takes a grayscale image and uses the gray as a measure of height, e.g.

Skull3dRelief

skull_relief

Processing is entirely fun to hack on (Debugger? We don’t need no stinkin’ debugger, println() is our only friend). It’s Java plus stuff to make graphics easy. I like the fact that to run the program you are faced with the code – the system invites you to start poking at the program from the outset. Andrew wrote most of the code, being a Processing pro (he wrote a book and¬†teaches a course in it; the first half of his¬†course is free). Me, I translated the Marching Cubes code to Processing: each pixel of each image is treated as a voxel, the 3D model is from the isosurface formed between the objects and the background.

We hope to see you on Sunday! Or better yet, online, where we hope to see you sending us animations for the gallery and¬†pull requests for code you’ve added.

 

Where did all this come from? Last year around May Andrew started making a series of looping GIFs using Processing, taking after the Bees & Bombs Tumblr feed. His goal was to make animations worth posting. These can now be found on Andrew’s¬†Tumblr feed.¬†Steve Drucker and I were the critics, over more than half a year.

During this time I was attending and organizing 3D printer meetups in the Boston area. Mark Stock¬†pointed out a fascinating way of modeling: instead of explicitly using union operations on¬†3D models, the traditional CAD approach, he instead deposited objects into a large voxel grid. It’s much simpler and faster to figure out if a voxel is inside some given primitive vs. performing a union or other constructive solid geometry operation on a set of models. For example, computing the union of thousands upon thousands of spheres will bring most CAD modelers to their knees. Voxel in/out functions are trivial to compute for spheres, and Marching Cubes then guarantees a watertight, well-formed model with no geometric singularities, precision problems, etc. 3D printers themselves have limits to precision, so using voxels is a good match. Here’s an example of Mark’s work:

Dendrite by Mark Stock

So, for me, these two things combined: animations could be used to define voxels, and Marching Cubes used to generate 3D representations. I made an exceedingly slow GIF to STL converter in Perl and ran a bunch of Andrew’s GIFs through it. A few interesting forms turned up and that got me started on playing with what I call “323,” converting from some three-dimensional form of data (an animation being 2D plus time) to another (a sculpture).

Seeing the call for Making @ SIGGRAPH, we decided to go further and give a workshop on the process. The T2Z program that resulted is massively faster than my original Perl program, generating sculptures in a few seconds. It’s also much more usable, allowing you to make your own animations, hook up sliders to variables, and easily export them as GIFs, a set of PNGs, or a 3D STL model. Programming all this¬†sucked up way more time than expected, and of course was highly addictive.¬†Andrew made this Processing program do things that Nature did not intend (e.g., binary STL output and multi-window UI).

Personally, I find this whole design¬†process¬†entertaining. In idle moments (or at the dentist) I imagine what might make both an interesting animation and a worthwhile sculpture. It’s a fun way to think about modeling and animation, and one where my intuition doesn’t always pay off. The more I play, the more I learn.

Here’s a screenshot, to whet your appetite – click it for the full-size readable version:

screenshot

So download the thing, install Processing and three little libraries (easy!), and start sliding sliders, pushing buttons, and hacking code! And let us know what you find.

BTW, if you want just one link to bookmark, it’s this: http://bit.ly/t2zspot

Tags: , , , ,

« Older entries