Can't tell where the issue lies, but as a stormzon right now, I began the game with about 109k defense, had a skill shrine going through the destruction den of evil, boosted to 116k defense. Skill shrine has been gone a few minutes now, and my defense is still at 116k defense, so I'm not sure if somehow the TCD boost on Asterism remained or the flat defense from Balance. One of the two seems to be remaining as a prolonged boost past the effects of the shrine. I can't test if it resets when I die because playing on Hardcore, but will edit this when I make a new game to check.
Edit 1: Got hit with Spellbind from the lizards in Dark Wood and after that curse, I got reverted back to 109k again.
Edit 2: Seems any time anything affecting your character status will then revert your defense to normal (level-ups, curse effects ending, etc). Things like Paragon returned to normal after the shrine bonus, but defense remained.
Edit3: It must be Asterism because the energy/dexterity bonus also remained after the shrine effect.
Edit4: Seems to only be caused by skill shrine. Elixir of Power works fine and reverts to normal defense after the duration.
Here are screenshots of a test for before/during/after: