The electrical activity in the heart varies significantly between men and women and results in a sex-specific response to drugs. Recent evidence suggests that women are more than twice as likely as men to develop drug-induced arrhythmia with potentially fatal consequences. Yet, the sex-specific differences in drug-induced arrhythmogenesis remain poorly understood. Here we integrate multiscale modeling and machine learning to gain mechanistic insight into the sex-specific origin of drug-induced cardiac arrhythmia at differing drug concentrations. To quantify critical drug concentrations in male and female hearts, we identify the most important ion channels that trigger male and female arrhythmogenesis, and create and train a sex-specific multi-fidelity arrhythmogenic risk classifier. Our study reveals that sex differences in ion channel activity, tissue conductivity, and heart dimensions trigger longer QT-intervals in women than in men. We quantify the critical drug concentration for dofetilide, a high risk drug, to be seven times lower for women than for men. Our results emphasize the importance of including sex as an independent biological variable in risk assessment during drug development. Acknowledging and understanding sex differences in drug safety evaluation is critical when developing novel therapeutic treatments on a personalized basis. The general trends of this study have significant implications on the development of safe and efficacious new drugs and the prescription of existing drugs in combination with other drugs.