The Swede, 28, faces Britain's Fury in Las Vegas on September 14, BBC Sport reported.
Anthony Joshua – who has sparred Wallin extensively – recently said he would face criticism if he were to choose to fight "lower-tier heavyweights."
"I don't care too much," Wallin told BBC Sport. "He probably means I haven't been at that elite level."
He added, "I haven't been at the elite level in fights but I have sparred big names and know I am at their level – Anthony Joshua, Kubrat Pulev, Adam Kownacki, Jarrell Miller, all different guys, and I know I am on this level.
"I am not worried about that, it's just that people haven't seen. Anthony knows I am a good fighter. Yes, it helps a little with motivation. I have no pressure at all – I can only go in there and do my best."
In addition to sparring Joshua, Wallin faced the former unified heavyweight champion of the world as an amateur, losing via decision twice.
Fury, 31, will start as around 1-25 favorite for their T-Mobile Arena bout despite Wallin being unbeaten in 20 professional contests.
It will be Wallin's first visit to Las Vegas, where Fury pummeled Germany's Tom Schwarz to a second-round stoppage in June to extend his record to 28 wins and one draw.
"This is the moment I have been dreaming of since I started boxing when I was 15," Wallin added.
"I am fighting probably the best heavyweight out there, at least the best boxer. He is a big guy, uses his size well. He is very skilled and can do a lot of things in there. He is tricky and has beaten good fighters but there are some things that people haven't done to him.
"I am really confident in myself and my work ethic. I have worked for a long time and dreamed of this moment. Now it is finally here and I have to try and grab it with both hands."