StatQuest(ML) Odds and Log(Odds)
Published:
by
Authur Lee
- Categories:
- MachineLearning 9
Odds
【Explanation of Odds】We might say that the odds in favor of our team winning the game are $5\ to\ 3$: Visually, we have 8 games total, 5 of which my team will win, and 3 of which my team will lose. So the Odds are $5\ to\ 3$.
Alternatively, we can write this as a fraction: $\frac{5}{3}$.
【NOTE】Odds are not probabilities!
\[\begin{aligned} The\ odds\ &=\ \frac{the\ ratio\ of\ something\ happening...}{...to\ something\ not\ happening.} \\ &=\ \frac{the\ ratio\ of\ something\ happening}{1\ -\ the\ ratio\ of\ something\ happening}\\ &=\ \frac{p}{1\ -\ p} \end{aligned}\] \[Probability\ =\ \frac{the\ ratio\ of\ something\ happening...}{...to\ everything\ that\ could\ happen.}\]【EX】
\[if\ the\ odds\ =\ \frac{5}{3},\ then\ probability\ =\ \frac{5}{8}\]Log of Odds
Still using the previous example…
- The better my team is, the odds of winning start at 1 and just go up and up.
- The worse my team is, the odds of winning get closer and closer to 0. (in other words, if the odds are against my team winning, then they will be between 0 and 1.)
For example, if the odds are against 1 to 6, then the odds are $\frac{1}{6}=0.17$, but if the odds are in favor 6 to 1, then the odds are $\frac{6}{1}=6$. The magnitude of former odds looks way smaller then the latter odds.
【BAM!!!】Taking the $log()$ of the odds solves this problem by making everything symmetrical.
\[log(odds)\ =\ log(\frac{p}{(1-p)})\]【NOTE】The log of the ratio of the probabilities is called the logit function and forms the basis for logistic regression.
\[logit(p)=\sigma^{-1}(p)=log(\frac{p}{1-p})\ \ \ \ for\ \ \ \ p\in (0,1).\]Mathematically, the logit is the inverse of the standard logistic function $\sigma (x)=1/(1+e^{-x})$.
The log(odds) makes things symmetrical, easier to interpret and easier for fancy statistics.
Even though the "odds" is a ratio, it's different from an "odds ratio"!!!
We will talk about "odds ratio" in here.