I think it depends on who you pass. If you pass a gaper in any near proximity going faster than 15mph they'll claim you were out of control. Somebody could pass me doing 50, as long as they don't hit me I don't care. Gapers also view any tricks/newschool stuff as reckless half the time. Even if you can ski backwards better than they can ski forwards they see you as reckless.
People are too concerned with what could have happened and don't realize what really happened. Of course an out of control skier creates more of risk but you're skiing, it's not exactly the safest sport. Just like driving, I'm just as concerned with my driving as all the other idiots on the road.