Federer, 37, and Nadal, 32, have dominated Slam tournaments over the past 14 years, along with Novak Djokovic, 31, BBC Sport reported.
But after Alexander Zverev, 21, beat both Federer and Djokovic at the ATP Finals, Toni Nadal feels the new generation of players have made a "quality leap".
"I find it hard to see Federer raising another Grand Slam trophy," he said.
Federer won the Australian Open in January to extend his all-time men's record to 20 Slam victories, but the veteran Swiss struggled to maintain that form throughout the rest of 2018.
"I do not say it for his game, although I do say it for the toughness of the five-set tournaments," Nadal added in his El Pais column.
"I have to confess that I said that on other occasions and the Swiss, repeatedly, surprised me."
Federer and Nadal were both world number one this year, before Djokovic won Wimbledon and the US Open to finish 2018 on top.
Zverev is back up to fourth after the weekend, when his performances forced 57-year-old Nadal – who retired as his nephew's coach at the end of the 2017 season – into a rethink.
"For some time I had my doubts about whether the youngsters would be able to overcome the game of Federer on grass, the one of Djokovic on fast courts and the one of Rafael on clay," he said.
"And I had the impression that the change would be caused rather by the decline of some than by the attack of others. The final in London has made me think about it.
"For now, for the coming year, I trust in the full physical recovery of Rafael. I feel new episodes of his rivalry with Djokovic and, in general, a more open panorama with the definitive incorporation of the new generation."