Simon Cowell rose to fame by making insensitive comments to hopeful contestants on American Idol, but his most recent jab about a gay TV host had him quickly apologizing.

During a recent segment on The X Factor spin-off, Xtra Factor, the show's openly gay host, Rylan Clark-Neal, had to unlock a series of doors that would reveal different fan questions.

"I love a front door," the host exclaimed during the segment.

"I thought you liked a back door," Simon retorted.

The remark reportedly left everyone shocked in the moment, but Rylan quickly replied, "Oh, Simon! Would you like me to show you how much?”

Cowell immediately regretted the joke, as he turned bright red and quickly apologized.

"I shouldn't have said that — sorry, sorry, sorry," said the host. "This is why we should never do live TV."

The moment was very quick on the show, but the reality judge faced backlash from offended viewers online.

But ultimately, Rylan wrote a post to shut down the critics and let everyone know that things are fine between him and his boss.

h/t: Gay Star News