This won't make the canker disappear, but it will sure make it feel better. Simply apply ice or rinse your mouth with ice water.
Ironically, while a canker sore hurts and stings, a painless ulcer in the mouth may be the first sign of oral cancer. And while a canker sore will usually disappear in seven to ten days, a cancerous lesion will not. If you've got a sore that doesn't go away, see your dentist, even if it doesn't bother you.