Let's start with normal stairs and understand why they burn calories, and then look at stair machines as a special case.
The HowStuffWorks article How Horsepower Works explains the unit of power known as horsepower. One horsepower is 550 foot-pounds or work per second, or 33,000 foot-pounds of work per minute, or 1,980,000 foot-pounds of work per hour. If a horse (or an engine) were to raise a 550-pound load at a rate of one foot per second, it would be exerting 1 horsepower of effort.
So let's imagine that you are standing at the bottom of the Empire State Building in New York and you start climbing the stairs. Let's also assume that you weigh 150 pounds. If each step raises you 1 foot, and if you walk up the stairs at a rate of one step every second, then you are doing 150 foot-pounds of work every second, or a little bit more than a quarter of a horsepower. If you were to run up four steps every second, you would be doing 600 foot-pounds of work every second -- 1.09 horsepower.
One horsepower exerted over an hour is equal to 641 Calories. So if you were to run up stairs at a rate of 550 foot-pounds of work every second for an hour, you would burn off 641 Calories. It is hard for human beings to maintain that level of effort for an entire hour. A normal human might be able to maintain a rate that burns 200 or 300 Calories per hour. Which is unfair when you consider that you can gain it all back by eating a candy bar!
On a stair machine, it does not appear that you are going anywhere, but really you are. Imagine that you are climbing a long ladder, but the ladder is sinking in the mud as fast as you are climbing. If you were to stop climbing, you would sink into the mud too, so you have to keep climbing. It would appear that you were going nowhere, but in fact you would be climbing at the same rate that the ladder is sinking, so you are doing work. The lack of movement is an illusion. A stair machine creates the same illusion -- as anyone who has used one can tell you, you are doing work!