Are The-One-Above-All and The Presence really the most powerful?

Yes, they are the supreme being of their own universe.
Nothing more above than creator, nothing more greater than creating omni reality.

But only in their own universe

Some character can be more powerful than them but it's not confirmed. Sometimes they try to mention more powerful being than them just to confuse readers.

More powerful dudes exists,like Leviathan,enlil,utu,gilgamesh (unwritten gilgamesh not fate one )and overvoid,are superior to presence,toaa is one of the weakest 1A out there due to crippling cosmology and retcon.