Moving people left and right has absolutely zero effect on the controlability of the plane, because the ailerons can compensate for much larger movements than that. What it might have an effect on, though, is the overall efficiency with which the plane flies. It is more efficient to have the center of lift and the center of gravity matched, than not.