I’ve had this happen to me once in all the years of running games and the situation while a touchy one ended well. I had to pull the player aside and talk to him about the situation. I actually took the die from him and placed it on shelf, and I returned it to him after the campaign was done. The die had three 20’s on it, for a d20 die.
When I returned it, I told him if he used this again I would ask him to leave and he was no longer invited over my house.