Wait, so... it makes no difference, but you have to be stronger. I think your sense of logic needs to be stronger.
And obviously, being taller makes inverts probably harder, since even when you tuck yourself in the smallest possible ball, you're still going to be bigger than the small guys. That being said, if everything wasproportionnal (jump size, rail size, etc.), it should not make a difference.