The second group of -ough words begins with a Middle English diphthong:

In the case of bought and ought, the original Old English monophthong vowels had become, in Middle English, the diphthong [ɔʊ] under the influence of the fricative [x], which was subsequently lost. [x] is a voiceless velar fricative, heard for example in the Scots dialectal pronunciation of the word loch. Dough and though followed a similar path from Old English, though in these instances the Middle English diphthong later reverted to a monophthong and was lengthened; hence it became available for raising during the Tudor Vowel Shift.

With examples like cough and trough the fricative changed to [f], and the diphthong became a monophthong without lengthening.