Mixture models. Let the PDF of a random variable X be the

Chapter , Problem 4

(choose chapter or problem)

Mixture models. Let the PDF of a random variable X be the mixture of m components: where m m fx(x) = LPjfy) (x) . j=l Pj 0, for j = 1, . . . , m. Thus, X can be viewed as being generated by a two-step process: first draw j randomly according to probabilities Pj, then draw randomly according to the distribution of lj. Assume that each Y; is normal with mean J-lJ and variance o}, and that we have a set of Li.d. observations XI . . .. , Xn, each with PDF fx . (a) Write down the likelihood and log-likelihood functions. (b) Consider the case m = 2 and n = 1, and assume that J-l1, J-l2 , 0'1 , and 0'2 are known. Find the ML estimates of PI and P2. (c) Consider the case m = 2 and n = 1, and assume that PI , P2 , 0'1 , and 0'2 are known. Find the ML estimates of J-li and J-l2. (d) Consider the case m 2 and general n, and assume that all parameters are unknown. Show that the likelihood function can be made arbitrarily large by choosing J-ll = XI and letting O'f decrease to zero. Note: This is an example where the ML approach is problematic.