Easter is one of the few Christian holidays named in English for something that is not Christian, and nobody can say for sure what.
The word is Old English Eastre (Northumbrian Eostre), and Etymonline traces it to Proto-Germanic austron-, "dawn." The root is Indo-European aus-, "to shine," which also gave English "east." Other relatives include Latin aurora and Greek ēōs.
The goddess is where it gets uncertain. In the early 700s the monk Bede wrote, in De temporum ratione, that the Anglo-Saxon month corresponding to April was named for a goddess called Eostre. Feasts were held in her honor. He says the Christian festival took over her name. Bede is the only source for her.
Historians treat this cautiously. The Anglo-Saxons left no written records of their own religion, and Old English used the word as an ordinary name for the feast, as in Easterdæg. The Oxford English Dictionary finds no evidence for Bede's claim, and one writer at Word Origins suspects Bede misunderstood the pagan practice. Neither reading can be proven.
Jacob Grimm later proposed a German goddess, Ostara, from the German word Ostern. He offered no evidence that she was worshipped, and the claim that the hare was her sacred animal is conjecture stacked on conjecture. An article on The Conversation concludes that any link between Easter customs and paganism is entirely coincidental.
Most neighboring languages skip all this. They use a form of Latin pascha, which goes back to Hebrew pesah, "Passover."
The famous symbols are later. Etymonline dates "Easter egg" to 1825, from older "pace egg" of the 1610s. The Easter hare began in Germany as the Osterhase. The earliest English mention one Word Origins writer knows is in Charles Dickens's Household Words in 1851, which describes sugar "Easter hares." Etymonline has "Easter bunny" by 1904.