"We agreed to increase the security cooperation of the two countries, our border forces, our intelligence forces," Rouhani told a joint news conference, following months of increased tensions over attacks on both sides of the frontier.
"And also to form a joint quick reaction force on the border of the two countries for fighting terrorism."
The border skirts the southeastern Iranian province of Sistan-Baluchestan which has been the scene of frequent attacks on Iran's security forces.
Khan's visit to Iran, the first since he took office last year, comes after gunmen who Islamabad says were based in Iran killed 14 members of Pakistan's security forces last week in its own Balochistan province.
Khan said that terrorist activities at the border could be a source of tension.
"The most important reason why I'm here, Mr. President, is because I felt that the issue of terrorism was going to ... increase differences between our countries," Khan said during the joint press conference. "So it was very important for me to come here and come with our security chief that we resolve this issue."
"The security chief will sit down with his counterpart here and discuss (security) cooperation," Khan said, although no details were given on the joint force.
"We trust that both countries will not have terrorist activities from their soil ... We will not allow any damage to your country from our soil," said the Pakistani premier who started a two-day visit on Sunday.
“Unfortunately, over the past few months we witnessed some tensions caused by some terrorists who acted savagely,” Rouhani said.
“We are glad that the Pakistani side now recognizes groups with such inhumane conduct as terrorist and treats them as such.”
Rouhani said that his meeting with Prime Minister Khan marked a new milestone in mutual ties, adding Iran and Pakistan would continue to maintain their close ties despite outside pressures.
“Both sides agree that no third country whatsoever can affect the friendly and brotherly ties that exist between Iran and Pakistan,” he said, adding that Khan had officially invited him for a visit to Islamabad.
In March, Rouhani demanded Pakistan act "decisively against anti-Iranian terrorists", following a February 13 attack that killed 27 members of the Islamic Revolution Guards Corps in Sistan-Baluchistan.
Iran has said a Pakistani suicide bomber was behind the attack, which was claimed by the terror group Jaish al-Adl (Army of Justice), which Tehran says operates mostly out of bases in Pakistan.
A new umbrella group representing various terror groups operating in Balochistan claimed responsibility for an attack on Thursday when 14 passengers were killed after being kidnapped from buses in the province, which borders Iran.
Pakistani Foreign Minister Shah Mahmood Qureshi said on Saturday the training and logistical camps of the new alliance that carried out the attack were based inside Iran and called on Iran to take action against the terrorists.
Iran says terrorist groups operate from safe havens in Pakistan and has repeatedly called on Islamabad to crack down on them.
Pakistan energy needs
Separately, Rouhani said during the joint press conference that the Islamic Republic is ready to help with Pakistan's oil and gas needs and increase electricity exports to the country by ten-fold.
He also said Tehran is interested in expanding trade between Iran’s Chabahar and Pakistan’s Gawader ports by connecting them with a railway.
The president noted that Iran, Turkey and Pakistan as co-founders of the Economic Cooperation Organization (ECO) could increase economic cooperation by connecting their railway systems. This, he said, would open a corridor from Europe to China.
Earlier in the day, Rouhani extended a formal welcome to Khan, whose historic visit to Tehran has been viewed as a major step towards opening a new era in relations between the two neighbors.
Khan and his high-profile delegation touched down in Tehran on Sunday night after a brief stop in the northeastern city of Mashhad, where the premier paid a visit to the holy shrine of Imam Reza (AS), the 8th Shia Imam.
Rouhani led the welcome ceremony for Khan at the at Sa’adabad Complex, which was also attended by Iranian Foreign Minister Mohammad Javad Zarif, Minister of Road and Industry Mohammad Eslami and a few other high-ranking members of the cabinet.
AFP, Reuters and Press TV contributed to this story.