1、1212,()nnUuUuUuUpppp u010,10,111,22Uppp(,),(,)()()()uUUa bp up up u 为概率密度函数1212,(),(),()()LLLnnUuUuUuUp up up up u 3230011000,001,111,()LUUUUpp ppp u000,001,111111,()888LUUUUp u,(),0,(),(),1,()0iiiiiiiipI ppI ppI ppI p 且当时且当时1()log()log()iiiiI xap xp x 1(,)log(,)log(,)ijijijI x yp x yp x y 1(/)log(/
2、)log(/)ijijijI xyp xyp xy 12121121(,)()(/)()(/)(,)log(,)log()(/)log()log(/)()(/)()()()()()(/)(/)NNNp x yp x p y xp y p xyI x yp x yp x p y xp xp y xI xI y xxyI xyI xI yI x xxI xI xxI xx xx 当 和 相互独立:推广1(,)64ijp x y221(,)log(,)log664ijijI x yp x ybit 222(,)1(/)log(/)loglog3()8ijijijjp x yI xyp xybitp
3、y 120.80.2XxxP121222221122221122()log()log 0.8()log()log 0.2()()()()(0.8log 0.80.2log 0.2)()()()()()log()iiiI xp xbitI xp xbitNINp x I xNp xI xNIp x I xp xI xp xp x 次后所获得的信息量为平均每次所获得的信息量为()()()log()iiiH XE I xp xp x(/)(/)(/)log(/)iiiH XyE I Xyp xyp xy,(/)(/)()(/)log(/)jijiji jH X YE H Xyp yp xyp xy,
4、()()(,)log(,)ijiji jH XYE I xyp x yp x y)/()/()()()()()()/()()()/()()()/()()/()(),(12112121NNNXXXXHXXHXHXXXHYHXHXYHYXYXHYHXYHXYHXHXYHyxpypxypxpyxp推广相互独立和当集合YyXxxpyxpyxIa,)()/(log);();()()/(log)()()/()(log)()(),(log)()()()/(log)()/(log);(xyIypxypypxpxypxpypxpyxpypxpypyxpxpyxpyxIZzYyXxzxpyzxpzyxIa,)/(
5、)/(log)/;(ZzYyXxxpyzxpyzxIa,)()/(log);(ZzYyXxxpyzxpyzxIa,)()/(log);(符号符号符号/3811log)011;(/28121log)01;(/18141log)0;(444bitxIbitxIbitxIaaayxypxypyxypxpyxpyxxpyxpyxpyxpyxpYXI,)()/(,)()(),(,)()/(log),(log),(log),();(),()()()/()()/()();();(YXHYHXHXYHYHYXHXHXYIYXIyxyxpyxpYXH,),(1log),(),(0)(0)log(lim0log0
6、log,100),()(021XHpppppppppHXHiipiiiini信息熵相同613121216131612131),(),()(3213213211121cccpZbbbpYaaapXpppHpppHXHnnn0)0,0,1()0,0,1()0,1(HHHiiiiiiqppppppHloglog),(321MMMMHXHlog)1,1,1()()()/(YHXYH0);();(0);(,0);(0 1)/()()/(log)/()(log)/();(0);(yXIEYXIYxIyXIyxpxpyxpeyxpxpyxpyXIYXIyxx同理);()()/(log)()()/(log)(
7、);();();(,XYIypxypxypxpyxpxypYXIXYIYXIyxyx)/()()/()(log)()/(log)()(log)()()/(log)();()()()()/()()/()();(,YXHXHYXHxpxpyxpxypxpxypxpyxpxypYXIXYHYHXHXYHYHYXHXHYXIxyxyxyx0);(,)();()();(YXIYXYHXYIXHYXI独立当);();()/;();();(0)/;()/;()/;();();()/;();();()/;();();(YXIZXIZYXIYXIZXIYZXIZXYXYXIYZXIYXIZXIZYXIZXIYZ
8、XIYZXIYXIYZXIZXY相互独立与条件下在相互独立与条件下假设在)()()(),(2121LLlXpXpXpXXXXp)/()/()(),(1112121LLLXXXpXXpXpXXXp)/()/()(),(112121mLLLXXXpXXpXpXXXpnijimiiiaaAxxxxs,),(121jijijijimjnijnmpnmpssPsSsSPnmp1),(0),(/),()()(1/)(:10/)(kijmkmkijjijijijimjmijpiSjSPmpkpppsSsSPmp步转移概率为QQQQQQijppppppppSjip2122111211,Pp0)(10limij
9、jijijjjkijkpppppppWWP jjw1+TXY起始状态000110111/201/401/203/4001/301/502/304/5S1(00)S2(01)S3(10)S4(11)LllnininiiLiLiLinininiiiiLininiiLiiLiiniiLiiiniiiXHxpxpxpxpxpxpxpxpxpxpxpxpxpHxpxpxppppHLLLL11111112111211211)()(log)()()()(log)()()()(log)()log()()()()()()()()(log)()(111112XxxxX)(1)(XXHLHLLlllLLLlllLL
10、XXHLHLHXXHXXXHXXHXHH111111121)/(1)(1)()/()/()/()()(XXX41943611321aaaPX)/(1LLXXH)/()(1LLLXXHHX)(XLH),/(lim)(lim)(121LLLLLXXXXHHHLXX当)(),/(),/(log),(),/(log),()/(),/(11,;,111111111111111XHXXXHxxxpxxpxxxpsxxpsxpxxxp、mmiiiiiiiiiiiiiiiiiiiiimmmmmmmmmmm左边遍历马氏链对于齐次 iiiiiiiiiiiiiiiiiiiiiiiiiiisXHspHsXHspsxp
11、sxpspsxpsxpsxpsxxpmmmmmmmmm)/()()X()/()()/(log)/()()/(log),()/(log),(11111111111,;,右边右边8.02.005.005.09.001.0PxdxxpxdxxpxpXHXHxxpxxpxpxpXHxxpdxxpxpxiaxiaxnabxbaxnbaiXnbaiXiXnnniiXiXniiinxiaxiaiXXiiloglim)(loglim)(log)()(lim)()(log)()(log)()()()()(,)1(,/)(,11)1(b ba ai iX Xi iX X)d dx x(x x)l lo og gp
12、 p(x xp p利用中值定理可得令)/()(),()()()/()();();()/()(),()/(log),()/(),(log),(),()(log)()(,XYHYHYXHYHXHYXHXHXYIYXIXYHXHYXHdxdyxypyxpXYHdxdyyxpyxpYXHdxxpxpXHccccccccccYYXcYXYXcXXc互信息条件熵联合熵相对熵)/(lim)(/)()(lim)()(log)()()(log)(),()(,21XYXyxy/xyx,Y/XxxxXYXcLccLcYYXcXXLccHtxtyHHtxHLddppHdppXXXHH随机波形信源取条件熵相对熵和平稳随
13、机矢量具有最大熵当它是均匀分布时变量对于定义域有限的随机限峰功率最大熵定理，X，:)(1log 1)()()(log)(1log)()(log)()(log)()()()(1log)()(log)()(,(1211212121211111111111iiNiNbabaiiNiNbabaNbabaNbabaNbabacabdxdxdxxpxqxpeabdxdxdxxpxqxpdxdxdxxqxpdxdxdxxqxqxpxpdxdxdxxpxpxpxHNNNNNNNNNN )2log(212log2log2log)(2)(log2log)(2log)2)(exp()log(21log)()(21)(222222222222)(22)(22222 eeedxxpmxedxxpdxmxxpdxexpXHexpmxcmx )()(11:)()(:XHXHXHXHmm定义冗余度定义信息效率Thank You世界触手可及世界触手可及携手共进，齐创精品工程携手共进，齐创精品工程
