There are a lot of situations where punctuation marks go outside quotation marks. As a general rule, periods and commas usually go inside; colons and semicolons usually go outside. In the case of the exclamation point and the question mark, every sentence is different depending on whether the point or the mark relates to the statement inside the quotes or outside them.