Cheat Sheet

Permutation and Combination

Number of permutations for $n$ people and $k$ chairs:

$$nPk=\frac{n!}{(n-k)!}$$

$$nCk=\frac{nPk}{k!}=\frac{n!}{k!(n-k)!}$$

RL

$$Q^{\pi }(s,a)=E_s^{\prime }[r+\gamma E_{a^{\prime }}[Q(s^{\prime },a^{s^{\prime }})|s^{\prime }]|s,a,\pi ]$$

$$V^{\ast }(s)=ma{x}_a{Q}^{\ast }(s,a)$$

$$Q^{\ast }(s,a)=ma{x}_{\pi }{Q}^{\pi }(s,a),\mathrm{\forall }(s,a)\in SXA$$

$$V^{\pi }(s)=E_{a_t\sim pi(\cdot |s)}[R_t|s_t=s,\pi ]=E_{a_t\sim pi(\cdot |s)}[E[R_t|s_t,a_t,\pi ]|s_t=s]=E_{a_t\sim pi(\cdot |s)}[Q^{\pi }(s,a_t)]$$

$$A^{\pi }(s,a)=Q^{\pi }(s,a)-V^{\pi }(s)$$

$$E_a[A^{\pi }(s,a)|s]=E_a[Q^{\pi }(s,a)|s]-V^{\pi }(s)=0$$

$$E_{s^{\prime }}[R_t+\gamma V^{\pi }(s^{\prime })-V^{\pi }(s)]=A^{\pi }(s,a)$$

$$\begin{array}{rl}Q(s,a)+\delta (s,a;\pi )& =Q(s,a)+{\hat{T}}^{\pi }Q(s,a)-Q(s,a)\\ & =R+\gamma Q(s^{\prime },a^{\prime })\end{array}$$

ML

True Positive Rate (Sensitivity or Recall):

$$\frac{TP}{TP+FN}$$

Specificity:

$$\frac{TN}{TN+FP}$$

False Positive Rate (1 - specificity):

$$\frac{FP}{TN+FP}$$

Precision:

$$\frac{TP}{TP+FP}$$

Accuracy:

$$\frac{TP+TN}{TP+TN+FP+FN}$$

F-score:

$$2\ast \frac{Precision\ast Recall}{Precision+Recall}$$