FACULDADE DE ENGENHARIA DA UNIVERSIDADE DO PORTO

Signal processing and machine learningmethods and algorithm analysis for

classication of lung function on asmartphone app

EEC0035 - Preparação da Dissertação

João Pedro Fonseca Teixeira

Mestrado Integrado em Engenharia Electrotécnica e de Computadores

FEUP Supervisor: Luís Teixeira

External Supervisor: Miguel Coimbra

February 26, 2014

c© João Pedro Fonseca Teixeira, 2014

Signal processing and machine learning methods andalgorithm analysis for classication of lung function on a

smartphone appEEC0035 - Preparação da Dissertação

João Pedro Fonseca Teixeira

Mestrado Integrado em Engenharia Electrotécnica e de Computadores

February 26, 2014

Contents

1 PDI report intro 11.1 Project objectives and motivations . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 Problem statement 32.1 Defining the problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Refining the problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

3 Literature Review 53.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53.2 Spirometry Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3.2.1 Spirometers and Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . 53.2.2 Research and Development . . . . . . . . . . . . . . . . . . . . . . . . . 93.2.3 Games and Applications . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.3 Methods and Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.3.1 Spirometric Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.3.2 Signal Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.3.3 Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3.4 Discussion and Decision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.4.1 Signal Processing Discussion . . . . . . . . . . . . . . . . . . . . . . . 163.4.2 Machine Learning Discussion . . . . . . . . . . . . . . . . . . . . . . . 17

4 Work Plan 194.1 Calendarization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.3 Technologies, tools and work platforms . . . . . . . . . . . . . . . . . . . . . . 214.4 Concluded tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

References 23

i

ii CONTENTS

List of Figures

3.1 Common Flow Sensing Devices (Pneumotachometers). A-Fleisch type sensor,B-Heated wire sensor, C-Pitot tube sensor, D-Turbine sensor, E-Ultrasonic sensor 7

3.2 Flow vs Volume representations for different lung functions . . . . . . . . . . . . 113.3 Perceptron diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4.1 Time distribution diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

iii

iv LIST OF FIGURES

List of Tables

3.1 Distance metric functions between instances (x and y). . . . . . . . . . . . . . . 16

4.1 Temporal distribution of the project’s tasks. . . . . . . . . . . . . . . . . . . . . 20

v

vi LIST OF TABLES

Abbreviations and Symbols

Medical Related

COPD Chronic Obstructive Pulmonary DiseaseFEF25%−75% Forced Expiratory Flow between 25% and 75%, average flow during FVC

maneuverFEM Forced Expiratory ManeuverFEV1/FVC Tiffeneau-Pinelli indexFEVt Forced Expiratory Volume in the first t secondsFVC Forced Vital CapacityMMV Maximum Voluntary VentilationPEF Peak Expiratory FlowRV Residual VolumeSVC Slow Vital CapacityTD Tidal VolumeVC Vital CapacityVTG Thoracic Gas Volume

Engineering Related

ADC Analog to Digital ConverterANN Artificial Neural NetworkAPF All Pass FilterBPF Band Pass FilterCART Classification And Regression TreeDIY Do It YourselfDFT Discrete Fourier TransformFFT Fast Fourier TransformHPF High Pass FilterkNN k-Nearest NeighborsLPF Low Pass FilterML Machine LearningSL Supervised LearningSVM Support Vector MachinesUL Unsupervised LearningUSB Universal Serial Bus

CAGR Compound Annual Growth Rate

vii

Chapter 1

PDI report intro

This document aims to present the work developed throughout the curricular unit Preparação da

Dissertação.

Chapter 2 contains a description of the problems that will be approached during the project.

In chapter 3 the State of the Art concerning the relevant topics for the development of the project

will be presented. Also, the methods that will be researched in finer detail will be summarized.

Those include:

• Signal processing methods of forced expiration recordings.

• Feature gathering on original and processed signals.

• Machine Learning algorithms for input classification.

Finally, in chapter 4 the work plan for the semester will be presented . In addition, the plat-

forms and technologies which are expected to be used will be described.

1.1 Project objectives and motivations

This project aims to develop a mobile and easy to use platform that can help evaluate a patient’s

respiratory function. This kind of technology can also enable monitoring systems for patients

with lung problems such as asthmatics, COPD patients and people with other obstructive and/or

restrictive airway diseases. The following requirements are also desirable for the platform:

– Self contained,

– Detached from external systems (no external sensors),

– Fast enough (Timely results),

– Precise in the classification,

1

2 PDI report intro

The mobile and the self contained characteristics are important in order to reach the most

isolated places, which generally have worse medical care, if any. It not only makes sense the use

of smartphones in isolated places as it has been studied that areas in relatively severe poverty have

had a booming growth in smartphone use. Africa and Middle East is expected to increase from

the current 10% of share of smart devices and connections to 36% until 2018. The mobile data

traffic CAGR will also grow to 70% (Cisco, 2014). Currently about 1 in 5 Africans have access

to a smartphone (AfricanTelecomsNews, 2012).

The possibility of reaching such a number of people with less access to medical facilities

would certainly improve respiratory health care and awareness.

Chapter 2

Problem statement

2.1 Defining the problem

Lung function evaluation methods often revolve the forced expiratory maneuver (FEM). Since the

ultimate goal consists in the development of an Android app, the problem is to collect breathing

data in a way that the data resembles the FEM’s. Though the cooperation of the recording patients

and the correctness of the maneuver is important, only the topics concerning the development of

the application will be addressed.

The available data to work with will be an air pressure waveform which will be transformed

in a classification of the lung function.

2.2 Refining the problem

Since the problem is quite complex it urges to follow a divide and conquer strategy. In this sense,

the main problem is broken down to a few simpler and more contained problems:

– Physical Model of the Signal Propagation: Most of the spirometric relevant measures are

based on air flow and volume. The microphone is only capable of picking up data in the form

of pressure. In addition, the measurable pressure is attenuated and dispersed comparing

with the produced by the vocal tract. Therefore it might be beneficial to develop/assume a

radiation model and a conversion model from pressure to flow.

– Signal Processing: The signals, either previously processed or not, should go through a

series of processes that should have significant meaning in terms of audio. Processes may

include: FFT, Hilbert Transform, Filtering.

– Signal Feature Extraction: All the audio should be analyzed and distinct features should

be passed on to the classification stage. Features may include: FEV1, FEV1/FVC, PEF and

audio related measurements.

– Optimization and Classification: The features will be used to train a supervised learning

model of how to produce a characteristic label of the inputs.

3

4 Problem statement

– Method set comparison: Finally, several different method sets have to be evaluated on

which gives the best results.

Chapter 3

Literature Review

3.1 Introduction

In this chapter, a literature review deemed relevant for a better understanding of the problem

under study will be presented. A brief description of current spirometry technology is shown on

section 3.2. In section 3.3 some algorithms for Signal Processing and Machine Learning that are

currently available are presented. Finally, section 3.4 will present a brief discussion of which

methods might be best for the problem at hands.

3.2 Spirometry Technology

The precursor of the modern spirometer was introduced by Hutchinson around 1844. It was a

volume displacement water-sealed device. In recent years the sensor paradigm has shifted towards

the flow measure type thanks to the evolution and wide-spreading of electronic devices. Also, it

became possible to measure lung function parameters, using several other methods.

3.2.1 Spirometers and Sensors

During the course of lung function testing history many types of systems have been developed.

Some of them enable exams that complement each other and others are specific for providing

data from particular respiratory related diseases. A short presentation of each medically accepted

spirometry device is provided as described on Ruppel (2008):

3.2.1.1 Pneumotachometers

From Greek, pneuma, "wind" or "breath", thacos, "speed" and metron, "measure", pneumota-

chometer means that it measures breath speed. "The flow sensors that historically have been,

and continue to be, the mainstay of the respiratory laboratory utilize flow resistors with approx-

imately linear pressure–flow relationships. These devices are usually referred to as pneumota-

chometers" (Macia, 2006).

5

6 Literature Review

Volume Displacement Spirometers As the name implies, these spirometers measure directly

the shifting fluid volumes. Since the designs of this kind of devices were made during a time prior

to the electronic massive growth they are mainly based on mechanical actuation. The movement

caused by the breathing expirations is transferred to a stylus which could write on a moving chart.

More recently, with the development of computers, potentiometers were incorporated so the signal

could be better analyzed.

Water-Seal Spirometer: This device consists of a large bell suspended in a container with

its open end submerged. Breathing into the spirometer moves upwards the bell by an amount

proportional to the volume expired. During an inspiration the bell submerges beyond the stable

level. For many years the bell had a pen attached, that moved sympathetically with the bell.

The pen recorded on a rolling chart, the kymograph, the respective volume. In recent years the

bell activates a potentiometer handle that varies a DC voltage output. This signal could later be

digitized with an ADC and processed by a computer.

Dry Rolling-Seal Spirometer: This spirometer is cylindrically shaped holding a lightweight

piston inside. A flexible plastic seal couples the piston to the cylinder wall. The seal rolls on itself

when the piston moves slightly with the gas’ volume displacement. The piston shaft rides on a

linear bearing which actuates a rotary potentiometer. This generates an analog signal for flow and

volume that can be fed to an ADC and analyzed.

Bellows-Type Spirometer: This kind of device uses a collapsible bellow that folds or un-

folds depending on the fluid’s flow direction. The common bellow design resembles a flexible

accordion. One end is fixed and the other moves freely with the expirations. This movement ac-

tuates either a pen that writes on a moving chart or a potentiometer similar to the other volume

displacement spirometers.

Flow-Sensing Spirometers The following spirometers measure air flow based on several differ-

ent principles. All of them include the respective sensor and a tube to contain the expiration.

Figure 3.1 illustrates the following sensor types.

Turbine: The turbine, or respirometer, is an instrument that "consists of a vane connected to

precision gears". The vane’s rotation is triggered by the gas flowing through it. This sensor is more

suited to measure SVC. The sensor’s configuration can also use a light source and a photoreceptor

to retrieve the number of rotations per second, just like an odometer for motor controllers.

Pressure Differential Flow Sensors: This types of sensors make a pressure differential

measurement. The sampling openings are placed one before and another after a resistive ele-

ment, in the middle of the sensor tube. The element accelerates the gas by narrowing the gas flow

3.2 Spirometry Technology 7

Figure 3.1: Common Flow Sensing Devices (Pneumotachometers). A-Fleisch type sensor, B-Heated wire sensor, C-Pitot tube sensor, D-Turbine sensor, E-Ultrasonic sensor1.

opening, while reducing the pressure. This is called the Venturi effect. While changing the gas’

velocity and pressure, the element also ensures laminar flow.

The resistive element can either be a bundle of capillary tubes, the Fleisch type, or a mesh

screen or membrane, the Silverman or Lilly type. The Fleisch type is more reliable than the Lilly

type. Though the Lilly type is better suited for measuring widely varying flows. The disadvan-

tages of these kinds of measurement devices are that they are very sensitive to local atmospheric

pressure, temperature and humidity. As a consequence, they need to be calibrated daily.

Heated-Wire Flow Sensors: The heated wire method is based on the cooling effect of a

flowing gas. The wire is preheated to a fixed amount and, during the expiration, the gas’ flow

reduces the wire’s temperature. The instantaneous energy consumption to reheat the wire gives a

flow value at that time.

Pitot Tube Flow Sensors: The Pitot tube sensors are based on the principle that states that

the pressure of a fluid is related to it’s density and velocity. In general, a spirometry Pitot tube

sensor has, at least, two sets of small tubes, one facing each flow direction for measuring both

expiration and inspiration. The tubes are connected to pressure transducers.

Ultrasonic Flow Sensors: A sound wave is a compression wave that is influenced by the

means through it travels. The flowing of gas can increase or decrease a sound’s speed depending of

1Image taken from Ruppel (2008).

8 Literature Review

the relative direction it travels. The sensor consists of two oppositely faced ultrasound transducers,

angled with the tube. By measuring the sound’s traveling time the flow measurement can be

integrated into volume. There is also a different type of gas flow measurement that can be obtained

by quantifying the ultrasound’s frequency shift, based on the Doppler effect.

The design includes a disposable tube which is introduced between the sensors. This allows

a great advantage over several other spirometer types since it needs not calibration and solves the

problem of patient cross-contamination.

3.2.1.2 Peak Flow Meters

This instrument was designed to measure Peak Expiratory Flow (PEF). It provides a simple mea-

surement that aids asthma monitoring with an inexpensive device. The meters themselves tend to

be made disposable. These meters work by forcing air through a a resistor or a flow tube with a

movable indicator.

3.2.1.3 Plethysmographs

From Greek, plethysmos, "increase" and graphein, "to write". The plethysmograph is an instru-

ment that measures the change in volume within an organ or body.

There are two types of body plethysmographs: the constant-volume variable-pressure and the

flow or variable volume plethysmographs, Pressure plethysmograph and flow plethysmograph,

respectively. Both are devices that include a sealed box which the patient has to be in. They

are used to measure the thoracic gas volume (VTG), the specific airway resistance (sRaw) and its

derivatives.

Pressure Plethysmograph: This spirometer is based on an adaptation of Boyle’s law. The

change in volume inside the chamber is inversely proportional to the pressure variation. The

pressure change is caused by compressing and decompressing the atmosphere inside the box, in-

cluding the patients chest. If the temperature remains constant, the pressure change corresponds

to a specific volume change. Since the chamber has rigid walls and is sealed, its free volume

experiences the same, mirror-image shift volume as the lungs (Criée et al., 2011).

Flow Plethysmograph: This kind of plethysmograph uses a flow transducer in the chamber’s

wall. This device measures the volume changes inside the box. As gas is compressed and decom-

pressed, flow passes through the chamber’s opening. That flow is integrated and volume change is

recorded as the sum of the volume compressed and the one that left through the opening. With this

device there are also some other tests that can be conduced using a pneumotachometer connected

to the outside room.

3.2 Spirometry Technology 9

3.2.2 Research and Development

There is a significant effort to develop spirometric devices focused on the availability of materials

or cost efficiency. This aims to reach areas with less access to health care and in isolation as

described on 1.1.

A team from Penn State University developed a Lilly-Pneumotach type spirometer from easily

accessible materials (der Mauer et al., 2013). This Mashavu project prototype was designed as a

DIY concept enabling the wide-spreading of the design. It used a pressure differential electronic

sensor to measure the airflow and was made with PVC tubing, coffee stirrers, protoboard as a

mesh along with cotton cloth for filtering.

Another relevant project came from a multidisciplinary team of students from the University of

Washington in St.Louis. They worked towards a low cost spirometer (Brimer et al., 2012). Their

solution consisted of using a microphone as a pressure transducer and coupling it to a custom

designed breathing piece. The piece was relatively cheap to produce, could be easily sterilized

and was of robust manufacturing.

One project that motivated this dissertation was the SpiroSmart application developed by Lar-

son et al. (2012) addressing the Ubiquitous Computing field. The application consists of a some-

what complex software that records the patient’s breath while giving incentive and showing the

performance. It employs signal processing and machine learning methods to measure lung func-

tion.

A later project developed by van Stein (2013) for his Master’s thesis was based on SpiroSmart

but primarily focused on robustness to noise. Additionally, by using offline methods it could

enable better results since the methods were more powerful and compute-intensive than real-time

methods such as those used by SpiroSmart.

3.2.3 Games and Applications

Throughout spirometry history there have been several technological advances. As described

on 3.2.1, sensory methods have shifted from purely mechanical to partially electronic. As a conse-

quence, spirometry apparatuses have become computerized and became able to include secondary

functions or add-ons. Computers have maintained the tendency of getting smaller, following a

corollary of Moore’s law2. This means that there has been a reduction of the size of the plat-

forms where spirometry related software run. In recent years, significant effort has been focused

around smartphones since they are continuously carried around and include a significant amount

of electronic devices.

Games for compliance: The forced expiration maneuver is somewhat difficult to execute cor-

rectly. This case is specially true for children that need to make a lung function evaluation.

2Moore’s law is an empirical prediction which states that within roughly 18 months a chip performance will double,due to the miniaturization of transistor technology.

10 Literature Review

Therefore pulmonary function laboratories commonly have computer programs that guide chil-

dren through the process, most of the time without them even noticing. Usual tests comprise

computer games that are played using breathing apparatus, such as breathing forcefully through a

tube. This enables a better cooperation from the patient since kids rarely wish to lose a game.

Games for entertainment: With the growth in smartphone use and a rising concern about respi-

ratory diseases, several independent application developers have created sets of games that relate

to spirometry. Many measure the loudness recorded by the smartphone’s microphone to actu-

ate something. Some of them actually advertise spirometry, though they appear under the game

application category.

Mobile health applications: There is a significant amount of developers that create smartphone

applications that measure spirometric parameters. Those applications usually have an external

sensor that is connected by Bluetooth, Wi-Fi or USB to the smartphone. The application becomes

a mean to relay data, observe it and update a patient’s profile. Some solutions are also compatible

with multiple sensors for different health problems such as heart rate monitors, pulse glucose and

cholesterol meters, oximeters and so on.

3.3 Methods and Algorithms

3.3.1 Spirometric Parameters

Methods are only successful if interesting features can be extracted from data. Standard spirom-

eters measure flow of air that goes through the sensor to generate Flow vs Time plots. The flow

can then be integrated to achieve Volume vs Time plots that provide medical doctors another way

to visualize the information. Also, by mapping air volume to time, Flow vs Volume plots can be

made. From this last plot several quantities are measured:

• Forced Vital Capacity (FVC): total expelled volume during the expiratory maneuver.

• Forced Expiratory Volume in one second (FEV1): lung volume displaced in the first

second of the maneuver.

• FEV1/FVC: the ratio of the aforementioned values, also called the Tiffeneau-Pinelli index.

• Peak Expiratory Flow: the maximum flow velocity present in the maneuver.

"The most common clinically-reported measures are FEV1, FVC, and FEV1/FVC as they

are used to quantify the degree of airflow limitation in chronic lung diseases such as asthma,

COPD, and cystic fibrosis" according to Larson et al. (2012). These features try to summarize the

expiration curve’s characteristics, though a physician can retrieve a great deal of information from

analyzing the curve itself.

3.3 Methods and Algorithms 11

Figure 3.2: Flow vs Volume representations for different lung functions3.

Though the curve also provides relevant data, the study will focus on extracting these fea-

tures based on curves approximations. A completely accurate curve shape regression will not be

developed.

3.3.2 Signal Processing

Signal Processing (SP) is a field of engineering and applied mathematics that focuses on manipu-

lating signals to extract features or to turn them into a more desirable form for other applications.

The forced expirations recorded by a smartphone’s microphone will have certain characteris-

tics that prevent a meaningful extraction of parameters of interest. As a consequence, those signals

have to be transformed in order to get relevant data. Some pertinent SP methods are summarized

below.

• Fast Fourier Transform

• Up and Down sampling

• Signal Filtering

• Hilbert Transform

• Linear Regression

3.3.2.1 Fast Fourier Transform

The Fast Fourier Transform (FFT) is an algorithm commonly used in sound processing and anal-

ysis. It is an efficient way to compute the Discrete Fourier Transform (DFT) and its inverse. This

process can be viewed as the transformation of an amplitude and time signal into its frequency

power spectrum contents.3Image taken from http://www.frca.co.uk/images/flow_volume.gif.

12 Literature Review

Let x0, ...,xN−1 be complex numbers:

DFT : Xk =N−1

∑n=0

xne−i2πk nN k = 0, . . . ,N−1.

Here n stands for a descretized instant of time and k is a descretized frequency. In many ap-

plications the signal is "windowed" in overlapping chunks in order to analyze how the frequencies

vary with time. This enables frequency tracking which is useful for characterizing both the signal

and its source.

3.3.2.2 Up and Down Sampling

Downsampling, or decimation, is a process which reduces a signal’s sampling rate. The decima-

tion rate usually is an integer or a rational fraction greater than one. When the rational is used,

first the signal is upsampled by the fraction’s numerator and then downsampled by its denomina-

tor. Upsampling is the opposite process, that adds samples between the original signal ones, also

known as interpolation. Generally, the downsampling process is used to change a signal to a fixed

lower sampling rate that a signal processor will use. The upsampling is used to change back a

processed signal, making it more adequate for the processor’s external environment.

3.3.2.3 Signal Filtering

Filters are systems that change both a signal as its frequency power spectrum with respect to the

filter’s frequency response. They remove unwanted features and components from a signal and

can be described as to their frequency response as:

• Low Pass Filters: remove higher frequencies maintaining lower frequencies almost un-

touched. Can be perceived as a shifting mean.

• High Pass Filters: remove signal’s lower frequencies keeping the higher frequencies almost

intact.

• Band Pass Filters: remove all frequencies outside a specific frequency band. It is a mini-

mum second order filter since it can be viewed as a combination of a LPF and a HPF.

• All Pass Filters: are a different kind of filter that maintain equally the frequency gains.

They change the phase of the different frequency components modifying the time series and

not the frequency response.

The expression removal of frequencies might not be completely accurate since most filters

tend to attenuate deeply the frequencies cut, thus not completely removing them. If the intended

response of the filter is complex it could be implemented as a series of simpler filters.

3.3 Methods and Algorithms 13

3.3.2.4 Hilbert Transform

The so called Hilbert Transform is a linear operator that results in the convolution of a real valued

signal with the signal −1/(πt). The process is a basic tool in Fourier analysis and can calculate

the harmonic conjugate of a given function or Fourier series. If this conjugate is added back to the

original signal and processed using a LPF it is possible to extract the original signal’s envelope.

3.3.2.5 Function approximation

In order to extract breath parameters robustly, one could approximate the expiratory sound enve-

lope by a high degree function (van Stein, 2013). By using the signal’s approximation much of

undesired noise is removed and the general form of the flow-volume curve is greatly preserved.

There are several methods of approximation such as:

• Linear regression: First order approximation. The data becomes represented by a linear

function.

• Polynomial regression: A Nth order polynomial function approximates a noisy signal.

• Smoothing Spline: The spline methods consist on segmenting a signal into several smooth

polynomial functions. If the functions degree is greater than one then each segment connects

to the next polynomial such as that the functions derivatives have the same value on the

connection. For higher degrees, this gives better results than simple polynomial regression

since it can avoid Runge’s phenomenon4.

• Orthogonal polynomials: are families of polynomials with consecutive pairs of functions

that are orthogonal to each other under some inner product. A signal can be represented by

a linear combination of those functions.

3.3.3 Machine Learning

Machine Learning (ML) is a branch of Artificial Intelligence which develops models and studies

systems that can learn from data. These methods are widely applicable since they are abstract

concerning data representation and utility.

They can be roughly categorized in two sets based on the desired outputs or the inputs available

on the training set. The Supervised Learning (SL) methods are algorithms that use both inputs

and expected outputs, provided on the data set, to replicate the intended model. The UnsupervisedLearning (UL) methods are used when the desired output is unknown. They operate on unlabeled

data and have to devise the system’s behavior.

The field of ML is vast and growing. Therefore the scope of the methods to be researched

and developed needs to be narrowed. One important aspect of the project is to be considered.

To aid the learning process a series of recordings of real patients will be made. The smartphone

4Using equidistant points, with the increase of the polynomial function degree the interpolation fitting error mightnot tend to 0.

14 Literature Review

recordings will be accompanied by a commercial spirometer from which a medical diagnosis will

be made. This way, the ML method choice problem is reduced to Supervised Learning processes.

Some of these methods are shown below:

• Decision Trees

• Perceptron-based techniques

• Instance Based-Learning

• Support Vector Machines

3.3.3.1 Decision Trees

A tree is a structure that consists of a root node and several others attached to it in some kind of

hierarchical fashion. The decision tree’s nodes represent decisions, separating data according to

the node’s function. The terminal leaf nodes are the data labels. This kind of structure is trans-

parent to the user since it progressively sorts the data based on some feature until it is completely

defined.

The tree, or in fact the node’s functions, have to be generated based on the training data. There

is significant effort to find the best and fastest ways to divide the training data. The termination of

the dividing algorithm generally isolates many data entries to their own leaf. This causes problems

concerning the overfitting of the method. Many variations were proposed to reduce this problem

such as terminating the algorithm before the tree grows to full depth or pruning some of the outlier

branches.

There are also ensembles of methods that provide a better accuracy. A known technique of

variance reduction that diminishes overfitting problems is Bagged Decision Trees. By using n

random subdatasets, with replacement, to generate trees and enabling a voting process of new data

it can enhance local adaptivity of the boundary (Rao and Potts, 1997).

3.3.3.2 Perceptron-Based techniques

A single layer perceptron is a system that has N inputs and one output. First, the inputs are linearly

combined according to each input’s weight (typically real values between [-1, 1]). This output is

then fed to an adjustable threshold. Above it the output is 1; else it’s 0. Figure 3.3 shows a system

representation of a perceptron.

The learning algorithm changes the branches’ weights until the desired input/output behavior

is reached.

If the inputs are not linearly separable with respect to the correct categories then a perceptron

will not find a solution that fits all available data. In order to solve this problem multilayered

perceptron (ANNs) methods were developed (Rumelhart et al., 1985).

ANNs consist of several perceptrons, otherwise called nodes or units, arranged such as to

produce weighted links between each others’ inputs and outputs. They can be separated in three

3.3 Methods and Algorithms 15

Figure 3.3: Perceptron diagram5

categories according to their interface: the input layer nodes receive and process the system’s

inputs; the output layer nodes are the last stage of data processing and generate the system’s

outputs; the hidden layer nodes are perceptrons that connect input and output layer nodes by

means of activation values. As with a single layer method, a succession of training instances are

exposed to the net, and comparing the input/output response the weights are slightly adjusted in

the direction of error reduction.

One of the main problems of ANNs resides on the size/number of layers of the hidden set of

nodes. According to the desired application, too few nodes can result in a poor approximation

and generalization capabilities. If too much nodes are considered then overfitting can occur and

finding a global optimum can be proven difficult.

3.3.3.3 Instance Based-Learning

Instance based learning algorithms are what can be called as lazy algorithms. The majority of the

computation is made on the classification and not at the training phase. One of the most known

methods is k-Nearest Neighbours (kNN).

The idea behind this algorithm is based on the instances’ proximity. Instances with the similar

proprieties tend to be closer to each other than to other instances on the dataset. A new, unclassi-

fied, instance is assigned the most frequent label of its neighbors. This relative distance between

instances, this neighborhood, can be defined by a distance metric. Some of the most popular

distance metrics are presented on table 3.1.

3.3.3.4 Support Vector Machines

SVMs are focused on separating labeled data using hyperplanes. Their aim is to maximize a "mar-

gin" between the boundary of different classifications and the instances of either side. Minimizing

the Lagrangian in 3.1 with respect to the weight vector w and bias b would get the best hyperplane

cut.

5Image taken from http://homepages.gold.ac.uk/nikolaev/311perc.htm.

16 Literature Review

Table 3.1: Distance metric functions between instances (x and y).

Minkowski: D(x,y) = (∑mi=1 ‖xi− yi‖r)1/r

Manhattan: D(x,y) = ∑mi=1 ‖xi− yi‖

Chebyshev: D(x,y) = maxmi=1 ‖xi− yi‖

Euclidean: D(x,y) =(∑

mi=1 ‖xi− yi‖2

)1/2

Camberra: D(x,y) = ∑mi=1‖xi−yi‖‖xi+yi‖

Kendall’s Rank Correlation:D(x,y) = 1− 2

m(m−1) ∑mi= j ∑

i−1j=1 sign(xi− x j)sign(yi− y j)

There are some variations of the method for solving problems such as data containing misclas-

sifications and non-separable cases. There are even methods that remap non-linearly-separable

data in order to enable a hyperplane boundary and consequently to use SVMs. The SVMs training

algorithms necessarily reach the optimal separation, avoiding local minimums unlike ANNs for

example.

LP ≡12‖w‖−

N

∑i=1

αiyi(xi ·w−b)+N

∑i=1

αi (3.1)

3.4 Discussion and Decision

3.4.1 Signal Processing Discussion

The Hilbert Transform seems to be a crucial process since it provides a simple way to compute

the envelope of the breath signal. Another way to compute an envelope is tracking the major

frequencies of the signal, with a FFT, and regenerating a time series. Envelope signals might

need to be filtered of noise in order to extract correctly the features. Therefore, it could prove

beneficial to use LPF. On the other hand, the envelope can be made immune to noise if a function

approximation is computed.

Of those methods listed on subsection 3.3.2.5, some might work better then others for the spe-

cific case. The Linear Regression probably would not have good results since the FEM does not

follow a linear function. A Polynomial Regression seems to a good candidate though it has the

instability problem for higher orders. On the other hand, it could have good results for an order

between 3 and 5, which seems to be enough to represent the FEM accurately (van Stein, 2013).

Splines have a track record of lower order requirements and have similar results to Polynomial Re-

gression. Though, since the polynomial order seems to be relatively low it might not be justifiable.

Orthogonal Polynomials have a significant number of different families and need to be researched

in more detail to assess their relevance for the project.

It was defined that one of the requirements of the application is to have a processing time

relatively small so not all the methods mentioned should be implemented. Additionally, the results

of some methods might be similar to the degree that feature extraction does not monitor significant

change. The methods will be subject to a deeper study.

3.4 Discussion and Decision 17

3.4.2 Machine Learning Discussion

An instance based algorithm such as kNN does not seem suited to the projects requirements. First,

since a great deal of computation is made during the classification phase, the algorithm takes a

significant amount of time to calculate the output, specially if the training data has a considerable

size. Furthermore, this also means that the method has a large storage requirements. Another

significant problem concerns the sensitivity of the kNN to the distance metric function. Finally,

the number of neighbors, k, is a difficult variable to optimize.

Perceptron based techniques such as Neural Networks have a great classification speed and

can tolerate relatively well highly interdependent attributes. Though they have a poor tolerance to

missing values and do not deal well with overfitting issues. Therefore, ANN methods do not seem

very adequate for the problem at hand.

Decision trees have generally good characteristics such as transparency, speed of classifica-

tion and reduced model parameter set. Their main problem concerns noisy data and the risk of

overfitting. Though, as discussed on subsection 3.3.3.1 there are simple methods that diminish the

consequences of such situation. CART and Bootstrap aggregation methods (Bagging) seem good

candidates for tackling the problem.

Finally, SVMs have generally a very good accuracy, are quick to classify data and tolerant to

irrelevant attributes. They also have a fairly good success rate tolerating redundant and highly in-

terdependent attributes. Their shortcomings concern the transparency of the method and handling

model parameters. Though, given the problem at hand, they are also strong candidates.

An overview of several methods can be found on (Kotsiantis, 2007).

18 Literature Review

Chapter 4

Work Plan

Planning is one of the most important stages of any project and contributes significantly to its

success. It enables the best allocation of resources while keeping in focus small and medium

objectives. It also states a rate at which those objectives need to be completed.

In section 4.1 the tasks and small objectives are presented along with the respective timeline

and description. The methodology which will be used is described in section 4.2. In section 4.3

the tools and technologies that will enable to develop the project are stated. Finally, in section 4.4

will be shown the tasks already completed.

Since every project is subject to fluctuation when it comes to task conclusion the calendariza-

tion presented below is a mere proposal and is expected to be somewhat flexible. In fact, allocated

time could be shifted between tasks in order to ameliorate the time compromise.

4.1 Calendarization

Figure 4.1: Time distribution diagram

In table 4.1 the main steps of the project are shown along with the respective expected start

and end dates plus their duration. Below, a short summary of those tasks is provided.

19

20 Work Plan

Table 4.1: Temporal distribution of the project’s tasks.

Task Initial date Duration (weeks) Ending datePersonal Web page development 10-2-14 18 30-6-14Data gathering 14-2-14 6 28-3-14Methods analysis and Development 10-2-14 7 28-3-14Methods Simulation 18-3-14 5 21-4-14Application architecture and specs 22-4-14 2 2-5-14Writing the Dissertation 12-5-14 5 24-6-14Preparing the Presentation 25-6-14 1 1-7-14

1 Personal Web page development: During the projects development it is expected to present

several documents namely minutes from meetings with the advisors and weekly reports.

Therefore, it is needed to develop a personal website to make those documents available to

the interested parties.

2 Data gathering: In order to teach the optimization models real life data is needed. During

the course of several weeks, recordings of forced expiratory maneuvers are going to be

made.

3 Methods analysis and Development: The various methods of signal processing and ma-

chine learning currently available will be analyzed in depth. It will culminate on the selec-

tion of the set which will be developed and evaluated.

4 Methods Simulation: The methods set will be implemented in Matlab and compared with

each other to verify which combination of algorithms works best. They will be tested using

a data set not previously used.

5 Application architecture and specs: Once the best conjugation of methods is chosen, the

specification of the smartphone application will be stipulated. In addition, a concept of how

the program should be implemented will be developed.

6 Writing the Dissertation: The last stage of the project will consist in writing a report

compiling all relevant data, work developed and findings.

7 Preparing the Presentation: The dissertation will be accompanied by a presentation that

demonstrates the work done.

4.2 Methodology

During the course of the project it is intended to follow this methodology:

– Gather forced expiratory recordings from real patients with different types of lung function

characteristics/diagnosis.

4.3 Technologies, tools and work platforms 21

– Research the Signal Processing methods currently used and relevant for the data type.

– Research the Machine Learning and Optimization techniques suitable for Supervised Learn-

ing methods.

– Evaluate which methods are interesting enough to analyze in grater detail.

– Implement the more promising methods in Matlab and simulate in various configurations.

– Compare results and chose the best set of methods and configuration.

– Specify and design an architecture for the android app.

– Write the final report and presentation.

4.3 Technologies, tools and work platforms

The implementation and simulation of the methods is expected to be done in Matlab.

Has a requirement of Preparação da Dissertação, the periodically generated documents will be

uploaded to a personal website, which is to be developed.

The final report, presentation and weekly reports will be done in LATEXsince it is an important

document editing tool that, once configured, doesn’t require much maintenance concerning text

formatting.

4.4 Concluded tasks

In Preparação para a Dissertação several initial milestones were achieved:

• Initial research of the topic.

• Understanding of commercial grade spirometry sensors.

• Market survey of commercial spirometers.

• Development of a smartphone application for enhanced maneuver recording.

• Development of some methods (such as Hilbert Transform and classification tree genera-

tion)

• Familiarization with LATEX.

22 Work Plan

References

AfricanTelecomsNews. Africa Mobile Factbook 2012. Technical report, Africa & Mid-dle East Telecom-Week, 2012. URL http://www.mikekujawski.ca/2012/05/30/finally-some-2012-statistics-for-the-african-mobile-phone-market/.

Andrew Brimer, Abigail Cohen, Olga Neyman, Braden Eliason, and Charles Wu. Low CostSpirometer - DEBUT. Technical report, Washington University in St.Louis, St.Louis, 2012.URL http://s3.amazonaws.com/challengepost/zip_files/production/3622/zip_files/Low-CostSpirometerReport-DEBUT.pdf?1338659900.

Cisco. Cisco Visual Networking Index : Global Mobile Data Traffic Forecast Update , 2013 –2018. Technical report, Cisco, 2014.

C P Criée, S Sorichter, H J Smith, P Kardos, R Merget, D Heise, D Berdel, D Köhler, H Mag-nussen, W Marek, H Mitfessel, K Rasche, M Rolke, H Worth, and R A Jörres. Bodyplethysmography–its principles and clinical use. Respiratory medicine, 105(7):959–71, July2011. ISSN 1532-3064. doi: 10.1016/j.rmed.2011.02.006. URL http://www.ncbi.nlm.nih.gov/pubmed/21356587.

Stefanie Auf der Mauer, Samantha Chan, Peter Chhour, Josh Homa, Vy Pham, and Karthik Pisu-pati. Mashavu Spirometer Project, 2013. URL https://decibel.ni.com/content/docs/DOC-5837.

S. B. Kotsiantis. Supervised machine learning: A review of classification techniques. informatica31:249–268, 2007.

Eric C Larson, Mayank Goel, Gaetano Boriello, Sonya Heltshe, Margaret Rosenfeld, Shwetak NPatel, and Computer Science. SpiroSmart: Using a Microphone to Measure Lung Function ona Mobile Phone. In 14th ACM International Conference on Ubiquitous Computing, Pittsburgh,Pennsylvania, USA, 2012. ISBN 9781450312240.

N. F. Macia. Pneumotachometers. In J. G. Webster, editor, Encyclopedia of Medical Devices andInstrumentation, Vol. 5, chapter Pneumotachometers, pages 367–379. Wiley, New York, 2ndedition, 2006.

J. Sunil Rao and William J.E. Potts. Visualizing Bagged Decision Trees. In KDD, pages 243–246, Providence, Rhode Island, USA, 1997. URL http://www.aaai.org/Papers/KDD/1997/KDD97-050.pdf.

D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning internal representations by er-ror propagation. In D. E. Rumelhart, James L. McClelland, and CORPORATE PDP Re-search Group, editors, Parallel Distributed Processing Explorations in the Microstruc-ture of Cognition, volume 1, chapter 8, pages 318–362. MIT Press, Cambridge, MA,

23

24 REFERENCES

1985. URL http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA164453.

Gregg L. Ruppel. Manual of Pulmonary Function Testing. Mosby, 9th edition, 2008. ISBN9780323052122.

Bas van Stein. A Mobile Smart Care platform Home spirometry by using the smartphone micro-phone. Master’s thesis, Leiden University, Leiden, The Netherlands, 2013.