Nick Gibb spoke out ahead of the government publishing new guidance for schools, expected to address Internet safety, social media and online gaming.
It is expected to say children should be taught to limit the amount of time they spend online.
Schools have the power to ban phones from being taken on to the premises.
But government policy is that it is the responsibility of head teachers to determine whether this is appropriate.
The trade union that represents head teachers has expressed its skepticism about outright bans.
But Gibb said, "Many schools have already taken the decision to ban mobile phones from the classroom.
"While this is clearly a matter for the head teacher, my own view is that schools should ban their pupils from bringing smartphones into school or the classroom."
Last summer, the government published draft guidance about how this could be implemented.
"Children and young people are growing up in an increasingly complex world and living their lives seamlessly on and offline," it said.
But it acknowledged that while this presented ‘many positive and exciting opportunities’ there were also ‘challenges and risks’.
Since the publication of the document, which said "pupils should be taught about the benefits of balancing time spent on and offline", there have been more than 11,000 responses to it and new guidance is expected soon.
"Children should not be spending hours and hours on their smartphones or iPads. There are obviously huge benefits to the Internet and there's nothing intrinsically damaging about spending time online," Gibb said.
"But if the time children spend using social media or playing computer games becomes excessive, it drives out time for them to talk to their parents, exercise, do their homework or play with friends.
"It eats into the amount of sleep and rest children have, resulting in their coming into school the next day tired and unable to concentrate."
The minister, who has spent more than six years at the Department for Education, added, "Ensuring children can regulate their own use of smartphones and social media is becoming an increasingly important life skill for them to learn."
But the National Association of Head Teachers (NAHT) union — a professional association and trade union serving school leaders in all sectors of education in England, Wales and Northern Ireland — has said banning pupils from taking a mobile phone to school can be counterproductive.
Giving evidence to the Commons Science and Technology Committee in October last year, the union's senior policy adviser, Sarah Hannafin, said, "Mobile phone bans certainly work for some schools but there isn't one policy that will work for all schools.
"Outright banning mobile phones can cause more problems than it solves, driving phone use 'underground' and making problems less visible and obvious for schools to tackle."
She added, "Ultimately, schools work to prepare young people for the outside world, giving them the awareness and strategies to responsibly monitor their own screen use and the ability to identify and deal with any negative impacts or problematic content they encounter."