Abiy named 10 new ministers but retained the finance and foreign ministers from the cabinet of his predecessor Hailemariam Desalegn, who resigned in February to pave the way for the reforms.
"To respond to public demand a cabinet reshuffle is essential. This follows the change of leadership that was enacted in an attempt to change the existing situation in the country," Abiy, a former army officer, told lawmakers.
He told the new ministers to tackle graft and streamline bureaucracy, Presstv reported.
"This is a red line that has to be enforced," said Abiy, 42, who was sworn in on April 2. "What the public is fed up of is graft and maladministration. This will not be tolerated."
Unrest broke out in the largest province of Oromiya in 2015 largely fuelled by young people who said they were marginalized economically and politically in the country of around 100 million people.
Ethiopia's economy has grown at an average of nearly 10 percent for the past decade, which is the fastest pace in Africa. Unrest has led to concerns over its long-term stability.
Parliament swore in some 10 new ministers on Thursday and six others were moved to head other ministries. Motuma Mekassa, a former minister of mines, petroleum and natural gas, was named defense minister.
Abiy also named new industry, energy, trade, mining, health, and revenue ministers, as well as an attorney general.
Those he retained included Finance Minister Abraham Tekeste and Foreign Minister Workneh Gebeyehu, both of whom Hailemariam appointed in late 2016. Every lawmaker in the 547-seat parliament is from the ruling EPRDF coalition.
Abiy appointed former parliament speaker Abadula Gemeda as the national security adviser.
He also named a new federal police commissioner, a new director general of the Information Network Security Agency, which oversees cybersecurity, and a new director general of METEC, a military-industrial conglomerate, Presstv reported.
All four are civilians, a change from previous governments in which such posts have been held by men from the security branches.
On visits to different parts of the country, Abiy has stressed unity among the many ethnic groups and has promised to expand political and civil rights over time.
The government appears to have recently ended a months-long shutdown of internet outside the capital but a state of emergency introduced the day after Hailemariam's resignation remains in place.