Scaramucci - who held the top White House communications job for just 11 days - urged current administration officials to directly disagree with Trump if he's wrong, but hypothesized that the president responded in that way because he "likes doing the opposite of what the media thinks he's going to do", and he is "of the impression that there is hatred on all sides".
"The whole thing is nonsensical", Scaramucci said.
On ABC's This Week, Scaramucci expressed dismay over the hold that someone like chief strategist Steve Bannon, as well as Bannon's former home of Breitbart News, has over Trump. "He's got to be more into where the moderates are and the independents are, George, that love the president".
In an interview with NBC on Sunday, National Security Adviser H.R. McMaster repeatedly refused to answer whether he would work with Bannon. "Whether I was the White House communications director or not, I don't think you're going change the president".
When asked by Stephanopoulos if that influence stemmed from Bannon, Scaramucci said, "I think the president knows what he's going to do with Steve Bannon". And so if he does that, he'll have a very successful legislative agenda that he'll be able to execute.
"I've never sat down with Steve Bannon and said, 'Hey are you a white nationalist or a white supremacist?' But I think the toleration of it by Steve Bannon is inexcusable". "If he doesn't do that, you'll see inertia".
He added that it's not serving the President's interests to still have that wing around.
Mr Scaramucci said the events should be called terrorism and that the President needed to be "much harsher as it related to the white supremacists" in his statement on the violence.
Scaramucci insisted he was unaware the interview was being recorded and hit out at The New Yorker journalist Ryan Lizza after the interview was published, taking to Twitter to describe Lizza as the "Linda Tripp of 2017" and insisting "He absolutely taped the call without my permission", adding #lowlife.
"Listen, there's probably mutual disappointment on both sides, again I have to be accountable for what I did and I am", he said.
According to the report, Trump has been "irritated" with Bannon following a series of Breitbart reports targeting McMaster. I'm accountable for the mistake.