KS X 0001-16-2009 Information technology－Vocabulary－Part 16：Information theory《信息处理术语 第16部分 信息论》.pdf

1、 KSKSKSKSKSKSKSK KSKSKS KSKSK KSKS KSK KS KS X 0001 16 16： KS X 0001 16:2009(MOD, ISO/IEC 2382 16： 1996) 2009 12 10 http:/www.kats.go.krKS X 0001 16:2009 : ( ) ( ) () ()SJ ( ) : (http:/www.standard.go.kr) ： ：1988 12 24 ：2009 12 10 2009-0787 ： ： ( 02-509-7262) (http:/www.kats.go.kr). 10 5 , . KS X 00

2、01 16:2009 (MOD, ISO/IEC 2382 16： 1996) 16： Information technology Vocabulary Part 16： Information theory 1 . 1 ISO/IEC 2382 16： 1996 . ISO/IEC 2382 16： 1996, Information technology Vocabulary Part 16： Information theory 2 . 2 , , 4 . . a) 1) 6 . 2 2 , 2 . 2 . 2) KS . b) 1) 2 . 2) ( ) . . 1 “( ) ” “

3、 ” “ ” (16.04.12 ). 2 “ ( ) ” “ ” “ ” (13.05.03 ). 3) (1), (2), . “(1) ”(14.04.12 ), “(2) ”(14.04.13 ) 4) , 2 . . KS X 0001 16:2009 2 1 (a) (b) (02.10.05 ). (a) 2 N 2 N (b) 2 2 N N 2 (a) (b) (04.02.06 ). (a) , , , (b) , , , , , , 5) ( ) . 1 “ ( )”(16.01.03 ) 2 “ ( , )”(16.02.01 ) 6) ( ) . “SYSGEN( )

4、”(10.02.18 ), “GKS( )”(13.01.14 ) 7) . 8) . 1 (16.01.01 ) 2 information(16.01.03 ) 3 GKS(13.01.14 ) c) 1) . 2) b) 4) . d) 1) (ISO/IEC 2382 16) , . , . 2) ( ) KS X 0001 16:2009 3 . 3) (注記 ) . ( )： . 16.03.03 negentropy ( ) ( )： . ( )： . (XX )： XX( ) . 02.02.07 range( ) , “ ” . e) . . 3 ( ) . 16.01 16

5、.01.01 information theory 16.01.02 communication theory 16.01.03 ( ) “ ” . 1 , information (in information theory) 16.02 16.02.01 ( , ), message (in information theory and communication theory) 16.02.02 , message source, information source KS X 0001 16:2009 4 16.02.03 message sink, information sink

6、16.02.04 ( ) 1 , . , . . 2 , . channel (in communication theory) 16.02.05 , . symmetric binary channel 16.02.06 , stationary message source, stationary information source16.03 16.03.01 H0 log n . n . 1 16.01.03 2 . 2 (： Sh) e (：nat) 10 (：Hart) 1 Sh 0.693 nat 0.301 Hart 1 nat 1.433 Sh 0.434 Hart 1 Ha

7、rt 3.322 Sh 2.303 nat 3 . decision content KS X 0001 16:2009 5 16.03.01 4 b-fold , b . b . a, b, c , H0 (lb 3) Sh 1.580 Sh (ln 3) nat 1.098 nat (lg 3) Hart 0.477 Hart decision content 16.03.02 , , , )(log)(1log)( xpxpxI = . p(x) x . 1 16.01.03 2 , . a, b, c 3 , p(a) 0.5, p(b) 0.25, p(c)0.25 , . Sh1S

8、h50.01lb)( =al Sh2Sh25.01lb)( =bl Sh2Sh25.01lb)( =cl information content 16.03.03 , , . =niniiiiixpxpxIxpxH11)(1log)()()()( X x1, , xn xi(i 1, , n) I(xi) xi , p(xi) . , =niixp11)( X a, b, c , p(a) 0.5, p(b) 0.25, p(c) 0.25 , entropy, average information content, negentropy( .) KS X 0001 16:2009 6 16

9、.03.03 , Sh5.1)()()()()()()( =+=ccpbbpaapxH . 16.03.04 H H0 Hr. , 0rHHH = . a, b, c , p(a) 0.5, p(b) 0.25, p(c) 0.25 , Hr 1.5 Sh/1.580 Sh 0.95 relative entropy 16.03.05 , ( ) H0 H R R H0 H . . a, b, c p(a) 0.5, p(b) 0.25, p(c) 0.25 , R 1.58 Sh 1.50 Sh 0.08 Sh . redundancy (in information theory) 16.

10、04 16.04.01 R H0 r. , 0HRr = . Hr 1 . r 1 Hrrelative redundancy16.04.02 y x , y x p(x | y) . )|(1log)|(yxpyxl = . . l(x | y) l(x, y) l(y) conditional information content KS X 0001 16:2009 7 16.04.03 x y , p(x, y) . ),(1log),(yxpyxl = joint information content 16.04.04 , , . =nimjjijiyxIyxpYXH11)|(),

11、()|( . X x1, , xn xi(i 1, , n) , Y y1, , ym yi(i 1, , m) , I(xi|yi) yi xi p(xi, yi) . conditional entropy, mean conditional information content, average conditional information content 16.04.05 , 1 . . equivocation 16.04.06 , 1 irrelevance, prevarication, spread 16.04.07 , x I(x) y x I(x, y) . . T(x

12、, y) I(x) I(x, y) 1 x y . 2 . T(x, y) I(x) I(y) I(x | y) I(y) y . . T(x, y) T(y, x) transinformation, transferred information, transmitted information, mutual information KS X 0001 16:2009 8 16.04.08 . =nimjjiiiyxTyxpYXT11),(),(),( X x1, , xn xi(i 1, , n) , Y y1, , ym yi(i 1, , m) . T(xi, yi) xi yi

13、, p(xi, yj) . 1 X Y . , . T(X, Y) H(X) H(X | Y) H(Y) H(Y | X) T(Y, X) 2 X Y , . . mean transinformation content, average transinformation content 16.04.09 , , 1 . mHHmm = lim . 1 1 / . 2 Hm/m . character mean entropy, character mean information content,character average information content,character

14、 information rate 16.04.10 H , )(xtHH= . , X x1, , xn xi(i 1, , n) , =niiixtxpXt1)()()( p(xi) xi t(xi) . / . average information rate KS X 0001 16:2009 9 16.04.11 1 , limit . mTTmm = lim Tm m . / . character mean transinformation content 16.04.12 T T /t(X, Y) X x1, , xn xi(i 1, n) , Y y1, ym yj(j 1,

15、 m) . =nimjjijiyxtyxpYXT11),(),(),( p(xi, yi) (xi, yj) t(xi, yj) . / . average transinformation rate 16.04.13 . 1 . . channel capacity KS X 0001 16:2009 10 KS X 0001 16:2009 1 ISO/IEC 2382 16 ： 1996, Information technology Vocabulary Part 16： Information theory . ISO 2382 16： 1978 1988 KS , 1996 , .

16、 2009 . 2 . 3 1999 , , . 2009 . 4 27 , 1 , , . , , , . 16.01.03 information( ) , information theory( ) . , . , 16： 153787 1 92 3(13) (02)26240114 (02)262401489 http:/ Korean Agency for Technology and Standards http:/www.kats.go.kr KS X 0001 16:2009 (MOD, ISO/IEC 2382 16： 1996)KSKSKS SKSKS KSKS SKS KS SKS KSKS SKSKS KSKSKS Information technology Vocabulary Part 16： Information theory ICS 01.040.35； 35.020