England felt they could have had two penalties as Kane was man-handled in the box by the Tunisian defenders, with the Tottenham striker suggesting his late victor was just reward after a dubious refereeing performance.
Sweden's winning penalty was awarded following review by VAR. Referee Joel Aguilar initially allowed play to go on after Viktor Claesson was hacked down by Lee Jae-sung, but called a halt moments later after a word in his ear.
"We played really well in the first half and deserved to take the lead", said Kane, while Maguire added that in "the first half we should have been clear".
"They [Tunisia] were highly disciplined in terms of our tactics and very strong up to 90 minutes before we conceded the second goal. It showed good character".
But England did not have to wait long for the goal as Harry Kane netted in the 11th minute.
The heavyweight boxer argued that the only thing that mattered was the win, and to be fair to him, he's absolutely spot on.
However Kane was in the right place at the right time in the dying minutes of the match. Low expectations for this year's overhauled team were dispelled early in Russian Federation when Kane reacted quickly to score after John Stones' header was saved.
Kane has now scored in his last four England appearances and has 15 goals from 25 overall; only Gary Lineker has scored more in his first 25 matches for England (20).
Hassen, though, was forced off in the 15th and replaced by Ben Mustapha. Ferjani Sassi stepped up and made no mistake, just beating the outstretched arm of Jordan Pickford.
The crowd went wild after the hit. The Tunisians have themselves to blame as they dropped deeper and deeper as the second half progressed and allowed England to come at them.
Gareth Southgate's men continued to probe as the first-half progressed but Tunisia scored next after Kyle Walker fouled inside the penalty area. But again Linegard put the ball wide. Instead they were opened up repeatedly by England early on, especially on the right flank with Kieran Trippier posing attacking menace with his pace and incisive crosses. Coach Gareth Southgate had adopted a 3-5-2 formation against Tunisia.
Despite a couple of uneasy moments in possession during the first half, Leicester City centre-back Maguire, who now has six England caps, impressed with his industrious distribution and enterprising forward runs, particularly when his side were hunting for a victor.