November 23, 2005
We discuss the formulation of discrete maximum entropy problems given upper and lower bounds on moments and probabilities. We show that with bounds on discrete probabilities, and bounds on cumulative probabilities, the solution is invariant to any additive concave objective function. This observation simplifies the analysis of the problem and unifies the solution of several generalized entropy expressions. We use this invariance result to provide an exact graphical solution to the maximum entropy distribution between upper and lower cumulative probability bounds. We also discuss the maximum entropy joint distribution with bounds on marginal probabilities and provide a graphical solution to the problem using properties of the entropy expression.
Abbas, A. 2005. Maximum Entropy Distributions with Upper and Lower Bounds. In: K.H. Knuth, A.E. Abbas, R.D. Morris, J.P. Castle (eds.), Bayesian Inference and Maximum Entropy Methods in Science and Engineering, San José, California, USA, August 7th – 12th 2005, AIP Conference Proceedings 803, American Institute of Physics, Melville NY, pp. 25-42.