Monsters around him have charge, you are probably desyncing a lot, and it looks like they are one-shotting you from distance. You can handle them with thunderslam.
I don't remember, maybe you need to reroll area few times and he gets different immunities.
You have cold damage too, and it is possible to melee him, just takes some luck and practice. I've killed him many times with build that has less damage and walks much slower, even without using fortress, so really it is doable. Few times it was just bursting him with melee attack.