if I remember correctly from getting it drilled into my head in hs football, it goes something like this
you shouldnt carry more then 20 lbs extra BULK (muscle) then your frame should allow. you can google what your ideal weight is (a combo of height , body build, and age)
So say your 5'6 with broad shoulders (basicaly not a SKINNY build). Your ideal weight (healthy) is between 160-170. You can substain 20 lbs more muscle mass.
Once you get into heavy body building, and put more weight then you should have on, you become less agile, because you actually need more muscle to move your extra weight around.
Look at the NFL. your linebackers/running backs are some of the most agile athletes you can find, and in most cases, they arent giganically huge. Your lineman are massive, and they are slow
I may be incorrect a bit up there, cause im doing it from memory, but dont go crazy on body building for skiing.