Cheat Sheet

Permutation and Combination

Number of permutations for \(n\) people and \(k\) chairs:

\[nPk = \frac{n!}{(n - k)!}\]

\[nCk = \frac{nPk}{k!} = \frac{n!}{k! (n - k)!}\]

RL

\[Q^{\pi} (s, a) = E_s^{\prime} [r + \gamma E_{a^{\prime}}[Q(s^{\prime}, a^{s^{\prime}}) | s^{\prime}] | s, a, \pi]\]

\[V^{*} (s) = max_{a} Q^{*} (s, a)\]

\[Q^{*} (s, a) = max_{\pi} Q^{\pi} (s, a), \forall (s, a) \in S X A\]

\[V^{\pi} (s) = E_{a_t \sim pi(\cdot | s)}[R_t | s_t=s, \pi] = E_{a_t \sim pi(\cdot | s)}[E[R_t | s_t, a_t, \pi] | s_t=s] = E_{a_t \sim pi(\cdot | s)}[ Q^{\pi}(s, a_t)]\]

\[A^{\pi} (s, a) = Q^{\pi} (s, a) - V^{\pi} (s)\]

\[E_{a}[A^{\pi} (s, a) | s] = E_{a}[Q^{\pi} (s, a) | s] - V^{\pi} (s) = 0\]

\[E_{s^{\prime}}[R_t + \gamma V^{\pi}(s^{\prime}) - V^{\pi} (s)] = A^{\pi}(s, a)\]

\[\begin{aligned} Q (s, a) + \delta (s, a; \pi) &= Q(s, a) + \hat{T}^{\pi} Q(s, a) - Q(s, a)\\ & = R + \gamma Q(s^{\prime}, a^{\prime}) \end{aligned}\]

ML

True Positive Rate (Sensitivity or Recall):

\[\frac{TP}{TP + FN}\]

Specificity:

\[\frac{TN}{TN + FP}\]

False Positive Rate (1 - specificity):

\[\frac{FP}{TN + FP}\]

Precision:

\[\frac{TP}{TP + FP}\]

Accuracy:

\[\frac{TP + TN}{TP + TN + FP + FN}\]

F-score:

\[2 * \frac{Precision * Recall}{Precision + Recall}\]