July 8th, 2019
How to enlarge a penguin?
Whilst the clocks ticks down to TAGAP Day of July 31st and the release of Classic TAGAP, time for a little different kind of blog update. Some of you might even find this useful!
So, here we go.
What and why?
Both the Classic TAGAP and the upcoming TAGAP.net overhaul are about one thing; bringing old TAGAP content to the new day. Whilst TAGAP 3 and our upcoming game are built for full HD penguin mayhem, all the older content wasn't. Most of it was done in sub-4k resolutions using sub-4k assets. Whilst with project Classic TAGAP we can programmatically waltz around most of it via filtered upscaling, with website and promotional content we just can't.
So, I needed to create a good workflow for upscaling majority of our content to at least Full HD standard – and in case of official TAGAP promo art, at least 4k. The best solution would've been to re-create everything from scratch, but there aren't enough hours in a day to accomplish this while working on three games!
Thus, researching on heavy-duty upscaling began.
Upscaling basically means enlarging an image and filtering the pixels to avoid, well, obvious pixelation. When it comes to video games, two most typical uses are in emulation – with filters like hq4x and 2x SAI – and with PS4 Pro upscaling sub-4k graphics into crisp 4k with hardware-based checkerboard rendering.
With images, the most typical options is the Photoshop standard bicubic filtering. It works great size reduction and for smaller scale enlargement – say, enlarging an image by 120%. However, it simply doesn't cut it for 200% enlargement, as the results way too blurry.
The latest Photoshop versions – which I don't have access to, because I use Creative Suite, not Creative Cloud – have something called 'preserve details' method of enlarging. This is better for, well, preserving details, but still clearly shows that the image is enlarged from an inferior, tinier source. To my eyes it almost looks as if an image enlarged with bicubic filtering was re-filtered with a dose of 'sharpen'.
Time to bring in Waifu!
So, what is the best way then? The current Big Thing in image generation and computations is, of course, the utilisation of cloud and more importantly, neural networks. You've likely already heard of deep fakes? Those are generated using neural network computing. And if you can achieve stuff like smooth face swaps, surely enlarging an image would be a cakewalk?
Bring in nagadomi's Waifu2x, a browser based, free, neural network driven image enlarger. As the name applies, the algorithms of Waifu2x are designed for rescaling anime style art – with options to choose from drawn and a photo – and so far it's worked great with pretty much anything I've needed to throw at it.
Use of Waifu2x couldn't be simpler. First upload an image and choose between 'artwork' and 'photo'. Then you select the noise reduction scale (from 'none' to 'highest') and the enlargement scale (from none to 2x).
Nagadomi runs Waifu2x on a shoestring budget and it is down at times. If that's the case, the best way is to try again another day. Whilst there are other sites hosting the algorithm – it is freely available on GitHub after all – they lack the options mentioned above. And as you'll see further down this tutorial-of-sorts, having those options is kind of important for getting the best possible results.
Meanwhile, if you like this time-saver of a web app, you can support nagadomi and Waifu2x on Pateron.
Let's get down to it!
An example: Restoring a penguin
Most of my promo artwork is a combination of vectors and raster art. In this example, all the logos and headers are vectors, while Pablo and the background are rasterised. In the beginning, they are in their original layer sets.
For the best results, I will keep all the vector elements separate – vectors natively scale perfectly, retaining the sharpness and smooth gradients. I can leave them as is. However, the rasterised elements, the background and the character, I merge together. I save a copy of the mergedd layers, in 1920×1080 dimensions, into a PNG file (without vector elements!).
Then I enlarge the image to 4k – 3840×2160. That is exactly 2× the size original size. What scaling filtering you use here is irrelevant at this point. Obviously I Save As into a new PSD file.
Go to Waifu2x and use the image upload function to load in the PNG file we made in the previous step.
First, set the image as artwork, set the upscaling to 2× and set the de-noise algorithm to Medium.
Hit Convert. Once the image has loaded into a new tab, copy the image and Paste it to the Photoshop project as a new layer.
Go back to web browser and Waifu2x. Using the same image (no need to re-upload), set the de-noise algorithm to Highest.
Hit Convert. Again, copy the generated image and paste it into the Photoshop project as a new layer.
Now we have three rasterised layers. One is the version enlarged by Photoshop, one is from Waifu2x with details preserved and one is Waifu2x version with the best and sharpest outlines.
For start, you can delete the Photoshop version layer – in above image the Background – unless you want to keep it around for comparison. I give these a boot, it saves on hard disc space.
Next, give the Highest de-noise layer a transparency mask.
Then choose the mask (click the mask in the layer list so that it's highlighted, see above) and hit Ctrl+I. This inverts the mask from default white to black, meaning the 'Highest de-noise' layer just became invisible.
Now, with the mask of the Highest de-noise layer still selected, pick up the air brush tool, set the colour to white – and start 'tracing' the outlines you want to become sharp and surace you want smoothened out.
In the end, in our example, the mask looks like this:
And the final image looks like this:
Here's a zoomed-in comparison the image non-filtered and as how Photoshop resizes it – versus the Waifu2x double-whammy restoration:
Quite a difference, huh?
So yes, I've been doing this to all the sub-4k promo art, converting EVERYTHING into glorious 4k. In the process, I decided to ditch all the SD resolution versions, meaning the new site will carry these in 1920×1080, 1920×1200 and 3840×2160. I haven't seen an SD monitor outside retro YouTube videos and old movies in ages, so there's really no reason to dedicate server space to a load of 4:3 imagery no-one uses, right?
Until next time,
Jouni Lahtinen, the head penguin