I completely agree with this blog post by Humus on the uselessness of the performance numbers in most rendering papers. This is something that often comes up when reviewing papers. Frames-per-second (fps) numbers are less than useless, since they include extraneous information (the time taken to render parts of the scene not using the technique in question) and make it very difficult to do meaningful comparisons. The performance measurement game developers care about is the time to execute the technique in milliseconds.
Some papers do get it right, for example this one. The authors use milliseconds for detailed performance comparisons, only using fps to show how overall performance varies with camera and light position (which is a rare legitimate use of fps).