Downing Street on Tuesday dismissed concerns that Boris Johnson had lost his "grip" as Tory MPs – including one whip – claimed letters of no confidence had already been submitted.
Mr Johnson’s spokesman said he was "very much focused on delivering for the public" after a series of public missteps.
One Tory whip told The Telegraph it was now an "assumption" that some disgruntled MPs had submitted no confidence letters to the 1922 committee.
The committee, made up of Tory backbenchers, collects any no confidence letters. If 15 per cent of sitting Conservative MPs submit them, a leadership contest is triggered.
The Tory whip said: "There is an assumption someone has put in a letter. The rumour is persistently around. It will not get anywhere near the 50 letters you would need, but it does cause angst."
A second Tory MP said several of the "usual suspects" were believed to have lost confidence in Mr Johnson and submitted letters.
The MP added: "Is this the start of more of that? If the next month is like the last month, and horror stories continue, more letters will be submitted."
The Tory factions causing problems for Boris Johnson
But the Prime Minister’s allies dismissed the speculation, arguing that his position was safe after winning a huge majority in the 2019 election and securing Brexit.
Letters of no confidence, when submitted, are handed in privately, with only the MP in question and Sir Graham Brady, the 1922 committee chairman, directly involved. On Tuesday night, Sir Graham declined to comment.
The speculation reflects a febrile atmosphere among some Conservative MPs as the party’s polling has tanked in recent weeks amid a string of self-inflicted political wounds.
They have included two weeks of Tory sleaze headlines, a rebellion over social care and clashes over the watering down of the HS2 plans.
After political opponents hit out at Mr Johnson for losing his place in a speech to the Confederation of British Industry on Monday, Downing Street waved away concerns on Tuesday.
"The Prime Minister briefly lost his place in a speech," the official spokesman said. "He has given hundreds of speeches. I don’t think it’s unusual for people on rare occasions to lose their place in a speech."
Asked whether Mr Johnson "has a grip", the spokesman replied: "Of course."
There was also no public support from Downing Street for Lord Hague’s idea of creating a so-called "inner Cabinet", as David Cameron had, to help guide Mr Johnson on major political decisions.
The hunt for who was behind the quotation from a "senior Downing Street source" reported by the BBC saying there was "concern" in Downing Street about the Prime Minister continued on Tuesday.
Treasury spinners reportedly denied any involvement in the anonymous briefing after it was noted that both Number 10 and Number 11 – the Chancellor’s residence – are on Downing Street.
There is a concern that the departure earlier this year of Sir Edward Lister, a senior adviser who has been by Boris Johnson’s side in Downing Street and before that in the London mayor’s office, has left a gap that has yet to be adequately filled.
Dan Rosenfield, the Number 10 chief of staff whose background is as a civil servant in the Treasury, and Simon Case, the youngest ever Cabinet Secretary, are two frequent targets for criticism from disgruntled Tory MPs.
One Tory said: "There are repeated calls for there to be more political nous in Number 10. The usual names get floated for criticism. Rosenfield is the one that people pick on most. He is not interested in Parliament or MPs."
Another Tory MP from a "Red Wall" seat – traditional Labour territory won by the Conservatives in the 2019 election – called for similar changes. "I think there’s a growing feeling Boris needs to re-stamp his authority on Downing Street and do politics better," the MP said.
Both Mr Rosenfield and Mr Case are in their 40s. Number 10 sources have previously rejected the idea that either could move, stressing Mr Johnson’s loyalty to his inner circle.
Ben Gascoigne, the Prime Minister’s long-term political adviser, left in the summer but has been drafted back in as deputy chief of staff to help drive through reforms.
He is described by colleagues as a political enforcer who has been brought in to "knock on doors" in government departments to make sure decisions are being implemented.