Her CDU has since 2013 governed in a "grand coalition" with the SPD, but many in the center-left party are unwilling to repeat the exercise, believing that their party needs a spell in opposition in order to rediscover its goal and present a more original message to the electorate by the time of the next vote.
Germans voted in a splintered parliament, reflecting an electorate torn between a relatively high degree of satisfaction with Merkel and a desire for change after more than a decade of her leadership.
"Merkel has become a very polarizing figure, something she never envisioned for herself, never saw happening", said Merkel biographer Stefan Kornelius, who is also foreign editor of Germany's daily Suedeutsche Zeitung.
Ronald Lauder, president of the New York-based World Jewish Congress, called Chancellor Angela Merkel a "true friend of Israel and the Jewish people" and decried the AfD's gains at a time when anti-Semitism was increasing across the globe.
- Right-wing populist Alternative for Germany (AfD) wins its first seats in the federal parliament, also becoming the Bundestag's third power in the process.
"There's no guilt but there's always responsibility, and now we're going to have 90, I think it's up to 90, well I say openly racist people, in the Bundestag".
The caucus leader of Merkel's Union bloc, Volker Kauder, said he "would have liked a better result" but voters had given the party the task of forming the next government.
Projections show Merkel's conservative Union bloc finishing first in Sunday's election but well short of its election results in 2013. "We will hunt Ms. Merkel".
Merkel, 63, whose campaign events were regularly disrupted by jeering AfD supporters, said in her final stump speech in the southern city of Munich that "the future of Germany will definitely not be built with whistles and hollers".
- The pro-business Free Democrats return to parliament, after missing the cut for the first time since the war last time around. The party was Merkel's coalition partner in her second term from 2009-2013 but lost all its seats at the last election.
The traditionally left-leaning Greens were seen winning around 9 per cent of the vote and the Left Party also 9 per cent, meaning both stay in parliament. "Today is a hard and bitter day for social democracy in Germany", he said, adding that the party had "failed in our election purpose" and had not managed to mobilise the party's traditional base.
AfD supporters were vociferous protesters at Merkel's rallies throughout the campaign, repeating their claims that she was a traitor and calling for her to go.
Goetz Froemming, a candidate for the AfD in Berlin and the party's campaign organiser in the German capital, said: "This is a historic moment and a turning point for our party".
Merkel said she would seek to form a stable coalition government, either a "grand coalition" with the Social Democrats, or a three-party coalition with the liberal Free Democrats (FDP) and environmentally minded Greens.
Merkel told supporters Sunday that "we want to win back AfD voters" by solving the country's problems and addressing their concerns, reports AP.
Schulz's defeat to Merkel means the Social Democrats haven't won an election since 2002, and raises a question mark over his fate as leader.
The Left Party is incompatible with the conservatives and all have voted not to work with AfD.
Mainstream parties' leaders vowed a robust response to AfD's entry into parliament.