Gonna describe a possible rendering technique because I need to get it out of my head but it'd be way too much work to actually implement lmao

So I'm trying to get environment lighting to affect a volume

What I'm doing here is super naive: Just march through the cloud, summing up opacity, and every step looks at the local gradient and uses that to sample the spherical harmonics that describe the environment lighting

This looks... cool, but weird

The problem is straightforward: Just using the gradient to look up an SH value has nothing to do with scattering. This is essentially lighting the cloud like a weird low-opacity reflector, which is nonsensical

IRL clouds don't really REFLECT much light at all, they mainly transmit/scatter

(distinction between reflection and backscatter is like A Whole Deal)

The SH lookup is telling us how much light is coming from the environment, but what we need to know is how much of that light REACHES the sample

we need some kind of directional attenuation term to modulate the incoming environment light

(to be clear: I'm only concerned with attenuation here, not multiple scattering. This is Wrong ofc but if I try to handle inscatter/backscatter/etc this'll get way out of hand immediately. Need a decent single-scattering approximation first before tackling the rest)

"directional attenuation term" sure sounds like... another SH

So for every voxel (or sample, or however you're doing your volumes) you could sample all around its neighborhood and build an SH that describes how much light can reach it from every direction

So then you'd use the attenuation SH to modulate your environment-lighting SH and the end result SHOULD look fantastic

...This is where it gets into the "too much of a pain to actually write" bc my knowledge of SH math is really vague lol

plus having to store all the SH coefficients means storing way more data per-voxel, and I'm not sure I want that requirement on my volume system at this point

also I'm just using unity's lighting functions to sample the lightprobe SHs that unity generates; I've never actually gone through the math of encoding arbitrary functions into harmonics and then sampling them

not super hard to learn but like... not what I want to do tonight lol

One thing I'm curious about though:
I'd be trying to use one SH to modulate another SH

How do you... do that? Like you're not sampling along a direction anymore, what you'd want is the sum of the whole lightprobe SH weighted by the directional attenuation SH

Can you just like... multiply SH coefficients? Does that modulate the result, or is it like Fourier space where multiplication becomes convolution and you need new rules for new space? I know stuff like rotating them is nontrivial...

I did find this page, which at least proves I'm def not the first person to think of this:

They're only describing building the "visibility transfer function" though, not going into detail on how you'd actually use that to modulate incoming light

Notably the "march opacity but use the local gradient for shading" technique is useless for clouds but actually great for visualizing CT scans. In that case you actually WANT to see opaque reflective surfaces that are still, paradoxically, semitransparent

If you'd like to think about computer graphics for a bit, I came up with a much clearer explanation of the idea

Note that the last two steps (multiply SHs and sum the result) are the parts I'm unsure about, due to unfamiliarity with SH maths. Can those be done easily/fast?