Doctors disagree about whether or not you should take a day or two off from work when you come down with a cold. However, they do agree that extra rest helps. Staying away from work may be a good idea from a prevention standpoint, too; your coworkers will probably appreciate your not spreading your cold virus around the office. If you do decide to stay home, forego those chores and take it easy, read a good book, watch television, take naps.
You should probably also skip your normal exercise routine when you've got a cold, at least during the days when you're feeling the worst. Again, let your body be your guide. If you're feeling miserable, the best advice is probably to just stay in bed.
And while cold air doesn't cause colds, you're likely to feel more comfortable if you stay indoors and keep covered, especially if you have a fever. There's no sense in stressing your body any further.