adobe photoshop - Сlip layer to "outer glow"

I have a folder that contains many strings. This folder has layer effects applied:

Color overlay (bright yellow) Stroke (black) outer glow (can be any color, currently white.)

I want to clip a layer with a pattern to the outer glow of the layer effect applied to the folder. basically I want the Outer glow not to be a solid color, but to have a image/pattern overlayed.

How can this be done?

I found one way that works but is a bit of a hassle and the result is not perfect:

I convert the whole folder to a smart object (layer effect is applied INSIDE the smart object) then I clip the layer with the pattern to the smart object and use blend if to apply it only to the bright white outer glow.

Its not perfect because the font itself is quite bright and it has some fringing around the outer stroke. Also, if possible, I would like to avoid converting to smart object..