SMCS Faculty Research Talk: Second Session

September 9, 2022: Dr. Tahir Syed presented his research work titled: "Motivating cross-entropy as the canonical loss in deep learning." The CS and math faculty members, Ph.D. scholars, the Dean, Dr. Shakeel Khoja, and Chairperson Dr. Shahid Hussain attended the session.

We explore why estimating the parameters of a family of functions could be formulated as an optimization problem of an information-theoretic measure between the static label and the evolving predicted class probabilities in a supervised learning problem. The insights here are a variational understanding of an optimization problem usually realized via maximum likelihood estimation, a shift from frequentist to Bayesian statistics, loss function equivalences, the general way to tailor this loss for specific purposes, connections between supervised and unsupervised deep learning ideas, and for a wider audience, an appreciation of the field beyond a Collaboratory notebook.