When you sit more frequently than you exercise, your muscles weaken – and quickly. A sedentary lifestyle causes what's called disuse atrophy (that's opposed to muscle atrophy due to neurogenic causes, such as muscular dystrophy). Basically, that means when you don't use your muscles you lose your muscles. Muscle fibers lose their flexibility and become stiff, they lose bulk and mass, and they gain fat storage instead of turning those fats into muscle energy.
Additionally, when you're in a sitting position you're basically training your muscles to be better-suited for sitting. Your glutes and your leg muscles, for example, shorten compared to if you spent similar time standing -- and that means aches, pains and balance issues. Sitting weakens the muscles you use for posture and the abdominal, pelvic and spinal muscle groups, as well.