Paul Ryan has had a trying election season thanks to his party's nominee, Donald Trump.
"Will you support him?" Jake Tapper asked Ryan in an interview on CNN in May.
"To be perfectly candid with you, Jake, I'm just not ready to do that at this point," Ryan replied.
But Election Day might not put an end to his electoral headaches.
SEE MORE: Paul Ryan Has Lost A Lot Of Popularity Among Republicans
A new report from The Hill said Ryan's position as speaker of the house could be in jeopardy — even if Republicans are able to hold on to their majority in the lower chamber.
To become speaker of the house, a candidate from the party in power needs the support of a simple majority, or 218 of the 435 representatives.
Ryan's unwillingness to embrace Trump drew criticism from the candidate himself on Twitter, and some House Freedom Caucus members are reportedly upset with Ryan behind the scenes, as well.
But even before Trump, lots of Republicans weren't sold on Ryan taking over as speaker after John Boehner retired.
Scott Wong wrote in The Hill that if Clinton beats Trump in a close race, "he and his die-hard loyalists will almost certainly try to pin some of the blame on Ryan."
It could only take a few angry representatives pulling their support from Ryan to ensure he doesn't have the support needed to remain as house speaker. Now rumors are swirling that, rather than lose an election, Ryan might simply step down.
But Ryan's camp insists he's not thinking about that. A spokesperson said, "The speaker's only focus until Election Day is defeating Democrats and protecting our majority, and nothing else."
