The mail in blackmail was never delivered by a postman. It was rent.
In Scotland and the north of England, mail meant a payment, rent or tribute. Etymonline traces it to Middle English male, from an Old English word for terms, bargaining or agreement. Merriam-Webster and Wikipedia point to the related Old Norse mál, "speech, agreement." The post you get in the morning is a separate word, from an Old French term for a bag.
Blackmail first appears in the 1550s. Merriam-Webster gives its original meaning as "a tribute anciently exacted on the Scottish border by plundering chiefs in exchange for immunity from pillage." The raiders of the Anglo-Scottish border, the Border Reivers, took payment from people living along the border. In return, those who paid were spared from raids and harassment.
Why black? The sources split. Etymonline says the colour simply marks the practice as evil. Wikipedia describes an older contrast between rents paid in silver, reditus albi or "white rent," and rents paid in goods or labour, called reditus nigri. On that reading, black described how the tribute was paid. Either way, English also had "silver mail" from the 1590s for rent paid in money.
The law treated the arrangement as a crime on both sides. In 1678 Sir George MacKenzie's Laws and Customes of Scotland classed it as a kind of "concussion" under Scots law. Those who paid could be punished as thieves, along with those who took it. Etymonline says the rackets ran until the mid-1700s.
The word outlived the reivers. By 1826, according to Etymonline, blackmail meant any money extorted by intimidation, especially by threatening exposure or scandal. The verb followed in 1852. Merriam-Webster finds American newspapers of the 1840s already using it loosely for any unfair pressure. "Emotional blackmail" appears in the Oakland Tribune in 1917.
The popular story that blackmail was named after threatening letters in black envelopes is a guess built on the wrong "mail." Merriam-Webster rejects it. The real origin is a farmer on the border, paying rent to the men most likely to rob him.