​​Discussion 10B

​​Joint Distributions

​​A joint distribution is the distribution of a tuple of random variables.
​​​​Pr[(X1,⋯,Xn)=(x1,⋯,xn)]=Pr[X1=x1,⋯,Xn=xn]\Pr[(X_1, \cdots, X_n) = (x_1, \cdots, x_n)] = \Pr[X_1=x_1, \cdots, X_n=x_n]​

​​Example

​​
​​
​​​​X=0X=0​
​​​​X=1X=1​
​​​​Y=0Y=0​
​​​​Pr[X=0,Y=0]=13\Pr[X=0, Y=0] = \frac{1}{3}​
​​​​Pr[X=1,Y=0]=16\Pr[X=1, Y=0] = \frac{1}{6}​
​​​​Y=1Y=1​
​​​​Pr[X=0,Y=1]=0\Pr[X=0, Y=1] = 0​
​​​​Pr[X=1,Y=1]=12\Pr[X=1, Y=1] = \frac{1}{2}​
​​Given the joint distribution ​​Pr[X=x,Y=y]\Pr[X=x, Y=y]​, the marginal distribution of ​​XX​ is ​​Pr[X=x]\Pr[X=x]​.
​​​​Pr[X=x]=∑yPr[X=x,Y=y]=∑yPr[Y=y]Pr[X=x∣Y=y]\Pr[X=x] = \sum_{y} \Pr[X=x, Y=y] = \sum_{y} \Pr[Y=y] \Pr[X=x \mid Y=y]​
​​This is just the law of total probability.
​​
​​
​​​​X=0X=0​
​​​​X=1X=1​
​​
​​​​Y=0Y=0​
​​​​Pr[X=0,Y=0]=13\Pr[X=0, Y=0] = \frac{1}{3}​
​​​​Pr[X=1,Y=0]=16\Pr[X=1, Y=0] = \frac{1}{6}​
​​​​Pr[Y=0]=12\Pr[Y=0] = \frac{1}{2}​
​​​​Y=1Y=1​
​​​​Pr[X=0,Y=1]=0\Pr[X=0, Y=1] = 0​
​​​​Pr[X=1,Y=1]=12\Pr[X=1, Y=1] = \frac{1}{2}​
​​​​Pr[Y=1]=12\Pr[Y=1] = \frac{1}{2}​
​​
​​​​Pr[X=0]=13\Pr[X=0] = \frac{1}{3}​
​​​​Pr[X=1]=23\Pr[X=1] = \frac{2}{3}​
​​​​11​
​​​​Pr[X=x]\Pr[X=x]​ and ​​Pr[Y=y]\Pr[Y=y]​ are aptly named marginals because they fit into the margins of this table, where each is the sum of entries in the columns or rows, respectively.

​​Independence

​​​​XX​ and ​​YY​ are independent when their joint distribution ​​Pr[X=x,Y=y]=Pr[X=x]Pr[Y=y]\Pr[X=x, Y=y] = \Pr[X=x] \Pr[Y=y]​

​​Reminder

​​The sum of probabilities ​​∑xPr[X=x]=1\sum_{x} \Pr[X=x] = 1​. This is implicitly saying ​​∑xPr[X=x∣Ω]=1\sum_x \Pr[X=x \mid \Omega] = 1​.
​​This holds other conditions: ​​∑xPr[X=x∣A]=1\sum_x \Pr[X=x \mid A] = 1​.
​​