"You can facilitate, you can push, you can nudge, but the parties themselves have to make fundamental decisions to compromise," Kerry said in Algiers at the start of a trip to North Africa on Thursday.
"The leaders have to lead, and they have to be able to see a moment when it's there," he added, showing signs of frustration after his months-long peace efforts appeared to be in tatters.
He said talks were facing a "critical moment".
Negotiating teams from the US, Israel and the PA met late Wednesday to try to chart a path forward after both sides took steps Washington has criticized as "unhelpful."
"There's an old saying, 'You can lead a horse to water but you can't make it drink,'" Kerry said at the Algerian foreign ministry.
"Now's the time to drink; the leaders need to know that."
He added that the United States remained committed to the talks, and said that after the trilateral meeting on Wednesday discussions would continue.
Kerry did say, however, that there had been progress in overnight talks between Israeli and PA negotiators, which stretched until 4:00 am (0100 GMT), but stressed that "there is still a gap and that gap needs to close fairly soon."
Talks had shown little progress since starting last summer, and began to unravel completely last month when the PA's refusal to recognize Israel as a Jewish state or to extend talks beyond their April 29 deadline prompted Israel to indefinitely postpone the final tranche of 104 convicted terrorists which were to be released in a "goodwill" gesture that was contingent on the progress of negotiations.
Kerry had rushed to the region in a last-ditch attempt to keep the talks going, but was blindsided on Monday by PA chief Mahmoud Abbas's decision to pursue unilateral action by applying for membership in 15 international agencies and threatening to join more if Israel did not release the last group of 26 prisoners, forcing Washington's top diplomat to cancel scheduled talks in Ramallah.
Last night, the PA's UN envoy threatened further unilateral moves, including an application to join the International Criminal Court at the Hague.
In response to Abbas' actions - which breach not only the terms of the current talks but also its previous treaties with Israel - US lawmakers are considering cutting aid to the PA, which receives about $400 million annually from the US.
Israeli Tourism Minster Uzi Landau on Wednesday warned of punitive action if the PLO pursued efforts to join UN agencies and threatened that Israel could annex territory in Judea and Samaria in response.