The SPD agreed on Wednesday to join a new coalition government with the conservatives, more than four months after Germany's election, but SPD members could still scupper the deal in a ballot whose results will be announced on March 4.
Many grassroots members in the center-left party are skeptical about another tie-up with Merkel after the SPD suffered its worst election result of the post-war era in September's election.
Schulz had announced on Wednesday he would resign as SPD chairman to become German foreign minister, triggering widespread criticism as he had promised ahead of the September election he would not serve in a Merkel-led government.
His announcement also upset Germany's current foreign minister - veteran SPD member Sigmar Gabriel - who complained of a "lack of respect" and said he was popular among ordinary Germans.
In a statement on Friday, Schulz said he thought discussion about him and his role could harm support for the proposed new coalition in the SPD members' ballot.
"I hereby declare my decision not to join the federal government and at the same time I sincerely hope that this will end the personnel debates within the SPD," said Schulz, a former president of the European Parliament.
A Forsa poll showed almost three-quarters of Germans thought it would be wrong for Schulz to become foreign minister while only around a quarter thought that would be the right move.
The RND network of newspapers said Schulz had come under growing pressure from other members of the SPD to refrain from joining the new coalition government.