Here `x' is not a parameter, so it is not a `likelihood': rather some sort
of MAP estimator. You have a prior for X, and observe Y=X+E. It is easy
to get the posterior p(X | Y), but it is not at all clear to me that you
want the mode rather than the mean, say. (My experience is that you
probably don't.)
On Tue, 17 Jun 2003, Spencer Graves wrote:
> Now I think I can answer your question: Treat "x" as a parameter to be
> estimated in maximizing the following likelihood:
>
> c*exp(-0.5*((y-x)/sigma)^2)*(x^(shape1-1))*((1-x)^(shape2-1))
>
> Take the logarithm to get
>
> lglk(x|y) = log(c)-0.5*((y-x)/sigma)^2
> +(shape1-1)*log(x)+(shape2-1)*log(1-x)
>
> Set the first derivative of this to 0 as follows:
>
> (d/dx)lglk = (y-x)/sigma^2+(shape1-1)/x - (shape2-1)/(1-x) = 0.
>
> You know y, so plot lglk(x|y) vs x to confirm that it only has one
> maximum with 0<=x<=1, then solve the cubic you get from multiplying the
> first derivative by x*(1-x) to find the actual location. The function
> "uniroot" might help you here. Wrap the latter in a function. If you
> have millions of these to do, then you may want something more
> sophisticated. However, this should produce answers for you.
>
> hth. spencer graves
> p.s. A good source for help with things like this is
> "www.r-project.org" -> search -> R site search.
>
> ##########################################
> Spencer:
> You are really generous to pay so much attention to my problem.
> I guess you are right, my terminology is not appropriate: we are dealing
> with linear combination of variables, not with a mixture. But from this
> point on, I still uncertain what is the answer to my question. I'll try
> to formulate it again, in slightly different terms.
> I have an empirical distribution of the sample Y. It' has quite a
> peculiar shape. Combining subject matter consideration and results of
> independent experiments of the kind, it seems possible and natural to
> represent Y as a sum of two independent variables, X & E. Again, from
> the subject matter considerations it is appropriate to expect that E is
> N(0,sigma^2). Numerical experiments show that X is well fit into the
> beta model.
> From subject matter considerations beta seems to be a reasonable
> choice. Applying the method of moments, we find sigma and shapes.
> Running the whole bunch of similar computations with different,
> completely independent Y's of the same nature we find that parameters
> are fairly stable from sample to sample. Considering E as a "noise" and
> "X" as a signal, I ask the following question: is that possible to
> improve our knowledge of signal if we know statistical structure of both
> signal and noise? In my dream, I see some kind of procedure which for
> each Yi gives me some kind of Yi', which is "closer" in certain
> sense to Xi than Yi. In other words, I need to extract signal, X from
> "signal+noise", Y.
>
> Your consideration of likelihoods is very reasonable, and would be
> helpful if I tried to estimate parameters. But my starting point is that
> I already now the parameters, I need now to improve signal retrieval.
>
> Definitely, I could give you more background info. I am afraid, however,
> that too much detai would annoy you. In fact, I ask a core question from
> a much more complicated problem. I feel that I stuck in this seemingly
> simple question, and can't proceed without
> its resolution
>
> Many thanks for your valuable help
> Xao
> ############################################################
> I'm not certain I understand your terminology. "Finite Mixture
> distributions" are distributions that are, for example, 20% N(0,
> sigma^2) and 80% beta(shape1, shape2). Your use of the term "mixture"
> does not seem to fit this standard.
>
> I still don't understand your problem, but the following might help.
> Suppose X ~ N(0, sigma^2), E ~ beta(shape1, shape2), with sigma,
> shap1, shap2 known. If we observe Y = X + E, then the likelihood =
> probability (density) of what we observe is as follows:
>
> c*exp(-0.5*(x/sigma)^2)*((y-x)^shape1)*((1-y+x)^shape2)
>
> where c = 1/(sigma*sqrt(2*pi)*beta(shape1, shape2).
>
> Given any observation y, you can plot this likelhood as a function of x.
> Differentiate it with respect to x and set the result to 0, and you
> get a cubic, which you can then solve for x.
>
> If X ~ beta(shape1, shape2) and E ~ N(0, sigma^2), you get a similar
> likelihood; the method still works.
>
> If this does not solve your problem, please explain why you think the
> distribution is a mixture of normal(0, sigma^2) and beta(shape1, shape2).
>
> hth, spencer graves
>
> ############################
> Xao Ping wrote:
> > Spenser:
> > Thanks again. Distribution is a mixture of normal(0, sigma^2) and
> > beta(shape1, shape2).
> > Sigma and shapes are KNOWN. Again, my question is NOT how to estimate
> > them,
> > but HOW to PREDICT X's knowing Y's, that is how to extract signal
> > knowing signal plus noise and statistical characteristics of both signal
> > and noise.
> > Xao
> >
> >
> > Spencer Graves <spencer.graves@PDF.COM> wrote:
> >
> > What do your two distributions look like?
> >
> > If there really is structure, then the observations are not
> > independence. What do normal probability plots look like? If it's a
> > straight line or a gentle curve, then it is one distribution. If it
> > looks like two or three straight lines possibly with gaps in
> between,
> > that is the fingerprint of a mixture of normals. See, e.g.,
> >
> > Titterington, D. M.. (1985) Statistical analysis of finite mixture
> > distributions (New York : Wiley)
> >
> > or Geoffrey McLachlan and David Peel (2000) Finite Mixture Models
> > (Wiley)
> >
> > For information on software for this, check the R archives at
> > "www.r-project.org" -> search -> R site search. If you find nothing
> > there, send another query to r-help. Sundar Dorai-Raj, a regular
> > contributor to r-help, showed me something on this almost a year
> ago. I
> > don't have it with me now, but I believe there should be something
> > available.
> >
> > hope this helps. spencer graves
> >
> > Xao Ping wrote:
> > > Spenser, thank you for immediate response, Yes, X and E are
> > (assumed)
> > > independent.
> > > Let me shed a little bit more light on the problem. Empirical
> > > distribution of Y shows
> > > (in many samples of Y-type) a kind of tricky structure which
> > suggests
> > > the idea that PDF of Y is a convolution of two distributions.
> I've
> > > devised an additive model which reasonably well reproduces the
> > artefact.
> > > Using method of moments I've found the unknown parameters. This
> > > parameters are found reasonably stable from sample to sample. Now
> > the
> > > question: so what? Does all this knowledge help to reduce noise
> > in data?
> > > If yes than what is the appropriate framework for such an
> > analysis. I
> > > realize of course that
> > > the solution is in no way unique. However, what worries me more
> > is the
> > > question: so what? Suppose that I have the model, does it help to
> > > extract signal and to reduce noise.
> > > Thanks again
> > > Simon
> > >
> ########################################################
> If you can provide more structure, then we might be able to do
> something. For example, are all the x[i]'s and e[i]'s independent of
> each other? If you assume some correlation structure among either the
> x[i]'s or the e[i]'s, we might be able to make some progress.
> Otherwise, in a sample of N, all I see right now are N equations and 2N
> unknowns.
>
> hth. spencer graves
>
> Xao Ping wrote:
> > Dear All:
> > Suppose that I have a sample Y. Suppose also that it is known that Y=X+E
> > where X is considered as a signal and E as noise. The PDFs of X and E
> > are known: F(y, theta) and
> > G(e, xi). Parameters theta and xi are also a priori known. Given all
> > this knowledge,
> > is that possible to estimate signal X? Just to be precise, I need to
> > substitute each data point in Y by the predicted Y' in such a way as it
> > would be, in a sense, closer to X than in the original sample Y.
> > Thank you
> > Xao
> >
> > ------------------------------------------------------------------------
> > Do you Yahoo!?
> > SBC Yahoo! DSL
> >
> <http://pa.yahoo.com/*http://rd.yahoo.com/evt=1207/*http://promo.yahoo.com/sbc/>
>
>
>
> > - Now only $29.95 per month!
>
>
>
>
> --------------------------------------------------------------------
> This message was distributed by s-news@lists.biostat.wustl.edu. To
> unsubscribe send e-mail to s-news-request@lists.biostat.wustl.edu with
> the BODY of the message: unsubscribe s-news
>
--
Brian D. Ripley, ripley@stats.ox.ac.uk
Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel: +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UK Fax: +44 1865 272595
|