style - Period outside quotation marks ...?

I am quoting a phrase (not dialog) that happens to come at the end of a sentence. Formal writing protocol - I think - tells us that punctuation goes inside the quotes but it seems strange here.

A gangster might channel The Godfather movies by proclaiming "he sleeps with the fishes." Of course we all know what this means.

Now then - if the quote was inside the sentence, there would be no need for a period near it:

A phrase like "he sleeps with the fishes" is common among old style gangsters. Of course we all know what this means.

So, in other words - the period is not inherently part of the quoted phrase- the period is to indicate the end of the sentence - yet it seems just as strange outside the quotes

A gangster might channel The Godfather movies by proclaiming "he sleeps with the fishes". Of course we all know what this means.

And something is missing if I leave out the period, altogether:

A gangster might channel The Godfather movies by proclaiming "he sleeps with the fishes" Of course we all know what this means.

I am not looking for ways to restructure the sentences or reword. Above is just a made-up example. I'm looking for advice or consensus as to the proper placement of the period within the quotations - particularly when another sentence follows it.

Thanks.