you're still put at a significant disadvantage if you don't get your elemental prism, for instance -- or miss the level 80 class charm boost
i suppose the argument shouldn't lie in making ubers doable without charms but instead getting rid of level requirements for certain boosts seeing as it's less of a "bonus challenge" (as i believe they were initially intended to be) and more of a amazing-race-esque roadblock where you HAVE to stop what you're doing at level 45 and clear TA and you HAVE to stop what youre doing the moment you can handle hatred K3KBA and get your class charm or else you might as well not aspire to clear the end game content -- losing +4 to all skills, 20% increased speed, -5% fire/light/cold pierce, 20% all resists, bonuses like +1% defense per socketed rune, might as well be a death knell for a run
of course the implementation of items going "HEY GO HERE" helps prevent newbie druids from not getting goodberries for instance but why even have that restriction at all?