Three factors affect the size of pores: oily skin, age and sex [source: British Journal of Dermatology].
We all have oil coating our skin to keep it hydrated and safe from bacteria, but some of us have it in abundance. Oily skin and large pores tend to go together. When your pores are full with oil, for example, they expand.
Age contributes in a different way. Your skin loses elasticity as it ages, and this slackening of the skin can make pores look larger. Men, in general, seem to have bigger pores than women do, although hormones make a difference for women. During, before or after menstruation, hormonal shifts can cause skin changes, as well.
You can't change your skin type, your age or your biological sex. In fact, you can't "shrink" your pores at all. They don't open or close, because they don't have muscles, so you can't exercise them into submission. Shrinking pores is really more about tricking the eye than changing your skin structure.