Barbarian began as an impression of a sound. To Greek ears, foreign speech was a stream of noise. The word they built for it, barbaros, repeats a nonsense syllable, much like English "blah blah."
Etymonline describes the root as echoic of the unintelligible speech of foreigners. Other languages made similar words for stammering. Sanskrit had barbara-, "stammering." Latin had balbus, and Czech has blblati, "to stammer."
The word first appears in Homer, and only once. In the Iliad (2.867), the Carians fighting for Troy are called barbarophonoi, "of incomprehensible speech." Some scholars think it meant they spoke Greek badly rather than not at all. Either way, it was a remark about language, not about savagery.
The sharper meaning came with war. After the Greco-Persian Wars of the early 5th century BCE, "barbarian" was often used to mean the Persians, who were the enemy. A word for "people who talk differently" became a word for "people who are against us."
The Romans took it over as barbarus. They used it for peoples outside Greek and Roman culture. After the reign of Augustus it became a common term for all foreigners.
English picked the word up in stages. The adjective "barbarian," meaning foreign, dates from the mid-14th century. The noun followed in the early 15th century, for a non-Roman or non-Greek person. The sense "rude, uncivilized" only arrived in the 1590s.
The word also left marks on the map and the dinner table. Barbary, the old European name for the coast of North Africa, comes from the Berbers. Their name is usually traced to the same Greek word, via Arabic. So, in part, does rhubarb. Greek rha barbaron meant "foreign rha," the rhubarb plant imported from far away.
The same word that now suggests violence once meant only that someone's language did not make sense to you.