I took a day off from development today. But that doesn’t mean that I was away from the game; just that I wasn’t working on the things which were actually on the task list for the milestone 10 build (I’m almost, almost through it, now; only three major tasks left to complete).
Instead, I spent a lot of time looking at the old VectorStorm “bloom” effect. I thought that I’d finally put it to bed and fixed all its problems, but I’ve recently been noticing that the engine’s standard bloom filter was having lots of problems when making narrow lines glow, particularly at low resolutions (say, about 800 pixels tall or less).
Short version: I figured out a solution. The screen here is what it looked like the first time it ran. For the very first time on VectorStorm, I actually had to tone down the bloom effect (it’s now down to 3 passes, from 5). Which means that MT2 (and any future VectorStorm games) will now be slightly less demanding of your GPU than in the previous milestone builds.
Below the fold is the gory technical details of how it works.
The basics of bloom
Fundamentally, a “bloom” filter works by taking a copy of the screen image, blurring it, and then adding it back on top of the screen image. The tricky part is the “blurring it” step.
(In the case of VectorStorm, we don’t actually blur the screen image; instead, while rendering each frame we draw a separate “glow” image. We blur the “glow” image and add that over the screen image the same way that standard games do bloom. But it’s the same general idea).
A naive “blurring it” implementation would draw a blurred version of the screen by, for each pixel, checking the screen color of all the nearby pixels, and then averaging those color values together. If we want to blur by 4 pixels (for example), then each output pixel needs to sample every pixel inside a 9×9 box (that’s the pixel itself, and every pixel within 4 pixels in each direction). That’s 81 texture samples in total, per pixel. This works. But it will also be extremely slow, since sampling all those pixels for every screen pixel is slow. As a general rule of thumb, you can generally get away with doing up to three or four texture samplings in a shader — more than that is starting to push your luck. 81 is completely beyond the pale, and that would only get you a four pixel bloom; which is pretty tiny by anybody’s standards!
So instead of doing that, people will generally do a “separable blur”. In a separable blur, you do your blur in two passes, instead of one. So instead of sampling that 9×9 box of pixels for each pixel, you only sample horizontally; you sampling a 9×1 box instead of a 9×9 one — just 9 pixels from the screen image, instead of 81. And then you take the texture produced by that horizontal blur, and do the same thing again, only blurring vertically. The result of taking an image and blurring it horizontally, and then blurring vertically is actually surprisingly close to doing a full Gaussian blur. This means you’ve done the whole blur process with only 18 texture lookups per pixel, instead of 81. That’s a huge improvement! (though still probably much too expensive for a real game)
And you can go further than that; you can take advantage of a cunning maths trick and do a “linear separable blur”, where you don’t even sample all nine of those pixels. Instead, you sample in between the pixels, and let the GPU implicitly do some of the texture samples for you, in a way which is effectively free. This can bring a 9-wide blur filter down to needing only 5 texture samples per pixel. Which is starting to sound downright reasonable!
But again, we still only have a blur that spans four-pixels, which is pretty small, and we’re doing about as many texture lookups as we can afford to do. What if we want our blur to be bigger? (the absurdly huge blur in the screenshot above is about 40 pixels). Well, what people normally do is they take the screen image, and make a copy of it that’s half the size. They then blur both the large and small versions of the screen image, and blend them together. When we’re operating on a half-size image, the 4-pixel blur corresponds to 8 pixels of blur on the full-size image. So we get blur that extends a further distance, without needing to do an absurd number of samples. And if we want our blur even blurrier, we can halve the size of the screen image again, and have another, even blurrier image to include in the composited blur image.
This last bit is how virtually every modern game does its bloom effect. And it’s how VectorStorm has done its version of the effect as well. (We use just three texture samples each, at five levels of “smaller screen image”) But there’s a problem.
The problem with bloom
I casually mentioned “take a half-size screen image” before, as though that was a trivial thing to do. Actually, it isn’t always trivial. When you have fine image detail, that fine detail tends to get lost when copying the image into a smaller picture. For most games this isn’t a big problem. But in VectorStorm games, I use a lot of very thin lines, which I want to glow brightly and evenly.
And this is a problem. Let’s say that I have a pure white two-pixel-wide line in my screen image, over a flat black background. And let’s say that I want to make an identical version of the screen image with half the image size (for use in the blurring process, mentioned above). If both pixels of the two-pixel-wide line happen to line up precisely with one of the pixels on the smaller image, the line will show up as a single-pixel-wide pure white line. But if it doesn’t line up precisely, then it will appear as a two-pixel-wide grey line on the smaller image (it’s technically “in between” the pixels of the smaller version of the screen image). And there’s no really good solution to this; there are some very clever techniques which try to be smarter about downsampling textures into smaller images without losing image detail, but nothing fast enough that you really want to be running it in real-time.
So if this line is moving around on the screen (for example, because you’re dragging its window around), its glow will be flickering bright and dark, as it alternately lines up and moves out of sync with the pixels of the various smaller versions of the screen image. It’s a very unpleasant, strobe-like effect. And there’s no obvious way to fix it. And it seems like nobody else on the Internet has posted a solution for the problem.
Until I had this thought (and this is where the exciting new idea comes in):
“Self,” I thought, “I’m making all these smaller copies of the screen image, then blurring each of them, and then combining them back together. And the problem is coming from aliasing in the smaller copies of the screen image. And the aliasing is caused by detail levels which are too high. But I’m just going to blur all these images anyway. So what if I blurred each screen image before copying it to the next smaller version, instead of after? That way, the images have already been blurred when the copy happens, which means that there’s no “high detail” to be lost during the copy.
The answer, of course, to why I don’t do that is that doing that would mean that I’m no longer technically doing a correct Gaussian blur; the screen image ends up multiply blurred, and smeared much further out than it really ought to. Which is what you can see in the screenshot above; that’s blur-before-copy instead of blur-after-copy, using exactly the same settings.
But I don’t actually care that what I’m doing now isn’t mathematically Gaussian any more — if I just tone it down a little, it looks great. And since things blur further out in fewer passes, I can get away with doing fewer blur passes, and still have an equivalent glow effect. Plus, doing the blur first means the old “flicker” effect on moving lines is now completely gone. Almost makes me want to go back and update all the old games to use the new engine, just so their bloom would be fixed.
Almost. But there are more important things for me to be working on! I’m talking with artists, I’m almost ready for milestone 10, and Greenlight will come shortly after that. It’s all finally coming together!
