Before it described a stubborn person, adamant was a thing. It was the name of a substance so hard that nothing could break it, and nobody was quite sure what it was.
The Greek word was adamas. Its origin is uncertain, but the usual explanation splits it into a-, "not," and a verb meaning "to conquer" or "to tame." On that reading, adamant is the material that cannot be tamed. The etymologist Robert Beekes found the logic "rather strange" and compared an Akkadian word instead, so the breakdown is a good guess rather than a settled fact.
Greek poets put the stuff to dramatic use. In Hesiod's Theogony, Gaia makes a great sickle of grey adamant for Cronus to use against his father. In Prometheus Bound, Prometheus is chained to the rocks in adamantine bonds. Virgil gave Tartarus pillars of solid adamant that even the gods could not uproot.
Because no one had a sample, the name drifted between real materials. Etymonline lists a gold-like metal in Plato, white sapphire in Pliny, the magnet in Ovid, and also steel, emery and, above all, diamond. In the Middle Ages adamant was often confused with lodestone, perhaps helped by a false link to Latin adamare, "to love passionately," as if the stone loved iron.
English borrowed the noun in the mid-1300s, from Old French or directly from Latin. Old English had already had a form, aðamans, but the modern word is a fresh borrowing. The adjective, meaning "hard, unbreakable," followed in the late 1300s. Its figurative sense, unshakeable in an opinion, dates from the 1670s, and that is now nearly the only way the word is used.
Meanwhile the same Latin word took a second road. In Vulgar Latin it became something like adiamantem, and the many Greek words starting with dia- pulled it toward diamantem. Old French made that diamant, and English made it diamond. So when someone is adamant, they are, by origin, made of diamond.