I'd for sure be removing that spigot after every brew and paying it some extra attention. Mine stays out of my big mouth bubbler till I use it next and even then, it gets another sanitation round.
Personally, I'm not a big fan of s-04. I get fruity/estery flavors that I just don't care for and no matter what temps I have tried fermenting at, is just off putting to me. Might not be the case with you, but some flavors are characteristics of the yeast, I and know s-04 doesn't do well with temp fluctuations during fermentation. Especially if things warm up at all. If I even get it around 65f and a bit above, it's no bueno for me. I get really fruity/apple cidery type flavors.