Making my computer worse at displaying images, on purpose, for fun

took like two days actually

So this is, to the best of my understanding, the list of things that all interact to make photos of screens look Like That

Most of those can be simulated via postprocesses, some of which are already available in unity

The LCD moire pattern is tricky though. Need accurate coverage %s for every rendered pixel, where each one can overlap a totally arbitrary number of perspective-transformed subpixels

There are very clever ways you could solve this. Good candidate for compute-based rasterization, or since it's a big grid of axis-aligned quads you may even be able to do it analytically

that's a lot of work though so I just made 12,441,600 triangles and let MSAA take the wheel

fun fact I had to use unity's new(ish) Advanced Mesh API that lets you explicitly specify your own vertex and index buffers, because calling SetVertices/SetTriangles/etc on 24,883,200 verts crashed the editor lmao

Without MSAA it's obviously just a mess

8x MSAA cleans it up pretty dramatically, although at this resolution 8 samples is probably still insufficient for accurate coverage values

This is where resolution starts to matter. And like, SPECIFIC resolutions. Moire patterns come from overlapping grids, so the grid parameters make a huge difference

So, both the LCD grid geo, and the CCD sensor grid (i.e. the camera resolution) need to be physically plausible

The LCD grid is 1920x1080, with realistic subpixel dimensions. We need a realistic resolution for the camera

iphone 6 shot at 3264x2448 ("8MP"), so that seems reasonable for a low-end phone

MSAA benefits from the higher res too, so the result actually looks pretty plausible:

So the Moire effect is handled (or at least, close enough without going WAY into some optical weeds)

The next big piece is the Bayer filter. If you're not familiar with Bayer filters I would say just go read the wikipedia article lol; it's too long to explain here

the tl;dr though is: Camera sensors (typically) only sense one primary color at each pixel. The other two colors are reconstructed from the values in their neighbors

so it's like... not exactly Half resolution but three weird overlapping subsampled channels combined

so this is easy to simulate basically perfectly. Just multiply rendered image by the filter colors

the result: looks like ass

(image credit: screenshot from an AUNZ RAILFAN video on youtube. I don't know which one. I have watched Hundreds of these)

Since every pixel is missing two channels, it must be "debayered" to reconstruct the lost values

There are lots of ways to do this. Phones and software that deals with RAW files use good techniques. I'm going to use the Worst technique, because it looks kickass

Nearest-neighbor debayering just grabs the missing channels from the adjacent pixels

it produces these really characteristic zigzag artifacts on sharp edges. Almost nothing uses this anymore, but really shitty phones and old digicams did it and I always though it looked neat

So combining the 8MP Moire sim with the bayer/debayer filters (not really a "sim" since it's the same algorithms an actual camera could use) we get... an even bigger mess

lots of really cool texture though

the moire and bayer effects are the most technically complex bits. With those out of the way, the rest of the effects are fairly straightforward

(also I just noticed I still had the sensor-noise pass enabled in those last screenshots, so uh, spoilers I guess)

Adding a tiny blur to simulate poor focus MASSIVELY cleans up the moire pattern, getting rid of the chunky pixels and making it look much more natural

The blur runs after the LCD sim but BEFORE the bayer, since it's caused by the lens, which is physically between the LCD and CCD

Sensor noise is next. Since it's coming from the electronics and not anything optical, it runs after blur but before bayer filtering

I just made something that looked plausible. It's blue noise (as in, blue spectral distribution, not color (but also the color is blue)), stronger and lower-frequency in the blue channel. Could have used a higher-res texture but ehh it's fine; debayering breaks it up nicely

I'm not actually sure why the noise is both stronger and more coherent in the blue channel, but it seems to be true in real images

if I had to guess it's probably that the CCD is less sensitive to colors in that frequency range, and thus requires higher gain with worse SNR

Also worth noting the noise is straight ADDED to the CCD signal here. It's not screen- or overlay-blended in LDR like you'd do in photoshop; the phenomenon where noise disappears in bright areas happens organically during tonemapping, as it would in a real camera

(come to think of it white noise might actually be more physically correct for the sensor noise, but blue noise is The Best Noise so obviously it's the first thing I reached for)

Next up: Too Much Sharpening. This runs after debayering, as it's a software postprocess

even new iphones do this and it kinda cracks me up. "Our newest phone has SO MANY MEGAPIXELS" and then the optics can't resolve anything finer than a 9px blob so we gotta unsharp mask it lol

Sharpening gets really excited by the zigzag patterns from the debayering, resulting in a weird sort of plaid/checkerboard artifact

real phones don't exactly do this, because they use much better debayering, but like I said I already committed to Bad Debayering for the aesthetic

There's still a bunch more effects that need to be added, but thankfully it's all stuff you can do with unity's own rendering and postprocessing. I don't need to build more custom stuff

Anti-glare coating. Just a big shiny rough plane reflecting an HDRI of an indoor scene

It's very subtle but it makes a pretty big difference in terms of realism, particularly later once all the tonemapping/color stuff is in place

Final step!

Just a big postprocess volume that handles all the remaining effects

Slightly too aggressive exposure curve, slightly wrong white balance, subtle but very broad bloom, vignetting, vignette tinted slightly purplish to fake the LCD color shift toward the edges

Took a bunch of reference photos with different phones and made a couple different postprocess presets for different looks

none of it looks *exactly* like any specific phone, but I got close enough to the general Aesthetic that I'm really happy with the end result

Starting with generally dark images and then overexposing can look really good

tends to bring out the reflected glare layer in a way that looks remarkably realistic

(first pic is one my shader things, second is another AUNZ RAILFAN screenshot)

for anyone wondering about perf: it's what you'd call uhhhh Interactive Framerates

doing blurs at 3264x2448 is pretty heavy to begin with, and it's gotta draw 12 million triangles with 8x MSAA first lol

if you can find Compute Tricks for the moire stuff it'd prob be fast though

So yeah! For a weekend project I'm super happy with it

I think if I saw these by themselves I'd def just assume they were photos of screens