Some aspects of applying information- theory to measurement E.-G. Woschni

Technische Universit&t KarI-Marx-Stadt, DDR

In the first part of this paper, information-theory is used with respect to measurement to gain a quality-criterion that includes both static and dynamic errors leading to Shannon's well-known channel-capacity. Applying this theory, a number of problems may be solved as shown in the paper. In the second part, non-technical parameters are included, especially economical aspects, leading to the result that the value and the consequences of information - i e , not only statistical but also semantic and pragmatic aspects - are also to be considered. Finally, it is also pointed out that in the field of measurement-education, a quality criterion demands the inclusion of semantic and pragmatic aspects.

Keywords: Information theory, measurement education, optimisation, quality criterion

1 Introduction

During the last IMEKO Congress in Prague, a paper was presented by Fiok Jaworski, Karkowski and Urban dealing with a model formulation of the process of teach- ing metrology (Fiok et al, 1985). From this point of view the general problems of education may be interpreted as a task of optimisation.

One of the main difficulties in optimisation is the finding of a suitable quality-criterion, especially in those cases where several parameters influence this criterion. If such a criterion is known, an approach to solve the problem of optimisation is given by the theory of poly- optimisation.

In the paper a quality-criterion will be formulated initially, including both static and dynamic errors and the channel-capacity of information-theory. Several applications of this and a more generalised quality- criterion - taking into consideration also non-statistical aspects - to the field of measurement and education are the aim of this paper.

2 Channel-capacity as quality-criterion The static error is given by the amplitude error Ay

or the mean-square error ~2, leading to the 'maximum number of amplitude m~ or power-steps mp' (Woschni, 1982).

ma = l+)~/Ay; mp = l + P y / ~ 2 . . . . (la,b)

The memory-capacity necessary to store m, with equally probable measuring values is given by

1 s = 21og ma = ~ 21og mp bit . . . . (2)

To take into consideration the dynamic behaviour also, the number z of measuring values per second is in- cluded. From the sampling theorem the response time t, and the critical frequency fc can be deduced:

z = 1/tr = 2fc . . . . (3)

The theoretical value of the 'channel-capacity' from information-theory is considered in both the static and dynamic behaviour (Shannon, 1948) to achieve maximum information flow:

Imax = Ct =- z . s = ~ 2 1 0 g m a = 2fc210gma

= fc 2log mp bit/s . . . . (4)

Fig 1 shows some values of measuring devices and gives a survey of the efficiency in comparison with other information-processing systems (Woschni, 1981).

10 0

~ Telephone ~ Taperecorder

I Television I Inf. f low in computer

t Rnpi d pr in ter I Tape reader

Ca{hode - ray osci[lograph I Moving-coil osci l lograph i Optical recorder

I MechanicQt recorder I Neasuring device

Human information reception f I I I I

10 z IoL~ 10 6 lO & 101° b i t / s

C t ~

Fig 1 Channel-capacity o f some devices and systems

184 Measurement Vol 6 No 4, Oct-Dec 1988

This method may be applied to compare analogue and digital measuring systems (Woschni, 1965). With fi = pulse frequency and

ft (5a,b) ma, = l+9/Ay;mdig = 1+ 2f~ " '"

one obtains the results as illustrated by Fig 2 showing that below the critical frequency

h 2f,. - man-- 1 . . . (6)

digital systems are more convenient and vice versa.

I0 o

10 5

10 k

l i0 5

15 d" 10 2

C"

o 101 U

iO °

10 -1

10-2

rrl•n = 1000 ///4 = 1 0 0 //~$= I O 0 0 0 F I z = loz . _ - - -

,2, £ . I

10 -1 10 0 101 10 2 10 2, 10 4 10 5

fc = 1/2 Ir =

Fig 2 Channel-capacity of analogue ( ) and digital ( . . . . ) measuring systems with serial processing

Additionally, optimal filtering means that the mean- square error ~2 is to be minimised (Schlitt, 1968). On the other hand, in the information-flow one considers both the mean-square error and the dynamic behaviour given by the critical frequency as Eqn (4) shows. Instead of minimising the mean-square error, the information- flow may be maximised, leading to optimal filters (Woschni, 1960; Henning, 1980) for information transfer.

W o s c h n i

with:

A0 ---- 1;Ar(Cr) . . . . (8b)

The relation to the information flow Eqn (4) is evident. As a first approximation an additive assignment with weighting factors, Yr, and exponents, a~, instead of weighting functions, Ar, is chosen:

n

CQ = l+ZTrCra" . . . (9) r = l

In a similar manner to the inclusion of two technical parameters - the static and dynamic errors - in one quality-criterion, by means of the classical information theory, a more general criterion has to take into con- sideration not only statistical aspects but also semantic

I l lo '

/ ? t

l - u n c o r r e d e d Opt. a

C

w i t h computer

w i {hou t c o m p u t e r

(b) = unco r r~c {ed

3 G e n e r a l q u a l i t y - c r i t e r i o n

The inclusion of other parameters, especially those that are non-technical, c , beside the information-flow, I, yields the general quality-criterion with the weighting func t ion '~r:

n

CQ = A0(I)+ZAr(cr) . . . . (7) r = l

As a special case this formulation contains the product form

(fi) CQ = 2log m y cr '~" . . . (8a) r = l

withou{ compu{er

Q.C.

wi{h computer" /i/" I

/ I -- - uncorrected Opt.

Fig 3 Consideration of economical aspects (Woschni, 1981)

Measurement Vol 6 No 4, Oct-Dec 1988 185

Woschni

and pragmatic influences. As this general information theory is not yet developed, Eqn (9) is used to solve the problems as the following example shows:

Let us deal with the problem of optimisation, taking into consideration economical aspects. There may be a measuring system with error correction (Woschni, 1981). Fig 3a shows the course of the related information flow, I/Io, as a function of the degree of correction, a = fc/fco. In Fig 3b, the course of the costs, C, is given for the two cases - the original system is either equipped with a computer or not. Lastly, in Fig 3c it is shown how the value of the quality-criterion, due to Eqn (9), turns out if we assume a = - 1; i e, the costs are inverse to the quality. Where the maximum is situated depends now on the parameter % That means that the problem of including semantic and pragmatic aspects is shifted to the choice of this parameter. The same problem appears if punishing-functions are used.

The example of Fig 3c shows that, in the case of a measuring device including a computer, the correction is convenient, while in the other case the cost of pur- chase of a computer for this aim only may not be repaid. Because of the decreasing costs of hardware, the peak at a = 1 in Fig 3c decreases also.

4 Appl icat ion to educat ional processes

In Fiok e t a l (1985), an approach to include the process of education in this concept is given. In this field, also, the development of a suitable quality-criterion is necess- ary to solve the problem of optimisation. Since non- technical aspects are to be taken into consideration here, the same general problem of valuation of semantic and pragmatic aspects arises, leading to similar difficulties to those demonstrated before. This is the main problem in evaluating the efficiency and quality of education.

5 Conc lus ion

In the first part of the paper the use of information theory is shown with respect to measurement in order to develop a quality-criterion that includes several technical aspects. Taking into account both the static and dynamic errors leads to the well-known channel- capacity of Shannon's statistical information-theory.

Using this criterion in measurement-theory, a number of problems may be solved, as treated by examples in

the paper such as errors and memory-capacity; channel- capacity of several measuring devices in comparison with other information-processing systems; comparison of digital and analogue measuring-methods; optimal filtering and maximal information-flow.

In the second part the problems are expanded to include the aim of introducing non-technical parameters also. As an example of great importance, the consider- ation of economical aspects is treated, leading to the result that the key for solving this problem lies in the evolution of an information-theory that takes into con- sideration not only the statistical aspects - as the Shan- non-theory does - but also semantic and pragmatic aspects. This means that the value and the consequences of information are also to be considered.

As a conclusion, in the process of education it is pointed out that in this field the production of a quality- criterion demands also the inclusion of semantic and pragmatic aspects.

6 References

Fiok, A. J., Jaworski, J., Karkowski, Z. and Urban, A. C. 1985. The teaching of metrology at university level - An attempt of model formulation. Preprints Xth IMEKO Congress, Praha; Vol 1, p 124-130.

Henning, K. 1980. Die Entropie in der Systemtheorie. Diss. TH Aachen.

Schlitt, It. 1968. Stochastische Vorg~inge in linearen und nichtlinearen Regelkreisen. Braunschweig: Vieweg.

Shannon, C. E. 1948. A mathematical theory of com- munication. BSTJ 27, p 379-423.

Woschni, E.-G. 1960. Das Problem der Optimierung in informationstheoretischer Sicht und eine neue Art in- formationstheoretisch-optimaler Filter. AEI~I 20, p 599.

Wosehni, E.-G. 1965. Informationstheoretischer Verg- leich der analogen und digitalen Messung. Zmsr, 8, p 367.

Wosehni, E.-G. 1981. Informationstechnik. 2nd ed. Berlin: VEB Verlag Technik.

Wosehni, E.-G. 1982. Signals and systems in the time and frequency domains. In: P. H. Sydenham (Ed): Handbook of measurement science, Vol 1. Wiley, Chichester.

186 Measurement Vol 6 No 4, Oct-Dec 1988