props - communication.aau.dk · sociology, marketing and psychology. here after we do use the term...

PROPS Research Protocol

The usage of PRO data in psychiatric outpatient care - a pilot study.

Research Protocol PROPS_ver2_110416

/27

Table of Content 1. Project responsible .................................................................................................................................... 3

2. Collaborators ............................................................................................................................................. 3

3. Staff............................................................................................................................................................ 3

4. Background ................................................................................................................................................ 3

5. Problem ..................................................................................................................................................... 4

6. Hypotheses - tentative .............................................................................................................................. 4

7. State of the art ........................................................................................................................................... 5

8. Methods .................................................................................................................................................... 9

9. Material ................................................................................................................................................... 12

10. Purpose .................................................................................................................................................... 12

11. Statistical considerations ......................................................................................................................... 14

12. Implementation ....................................................................................................................................... 15

13. Biobank .................................................................................................................................................... 16

14. Testparticipants ....................................................................................................................................... 16

15. Side effects, risks and cons ...................................................................................................................... 17

16. Respect of the test participant’s mental integrity and data privacy ....................................................... 17

17. Application of EPJ data in the experiment .............................................................................................. 17

18. Budget and financies ............................................................................................................................... 17

19. Recruitment of test participants and test participant’s remuneration ................................................... 18

20. Informed Consent .................................................................................................................................... 19

21. Dissemination of research results ........................................................................................................... 19

22. Ethic research .......................................................................................................................................... 19

23. Disclosure of compensation or reimbursement schemes ....................................................................... 20

24. References ............................................................................................................................................... 20

Appendix A Description of Method ................................................................................................................. 23

Appendix B Description of the data requested from the EPJ .......................................................................... 23

Appendix C Documentation of the registration to the Datatilsyn .................................................................. 23

Appendix D Budget .......................................................................................................................................... 24

Appendix E Time schedule for the project ...................................................................................................... 26

Appendix F Test person information form ...................................................................................................... 27

Appendix G Informed Consent Document ...................................................................................................... 27


/27

1. Project responsible Professor, chief physician, dr.med. Jørgen Aagaard, Psychiatric Hospital/Aalborg University Hospital

Professor, Ph.D. Srikant Sarangi, Director of Danish Institute of Humanities and Medicine, Aalborg

University.

2. Collaborators Ms. Cs Morten Aagaard, Aalborg University.

The project aims to establish a praxis related study in which both clinician psychiatric expertise, humanistic-

communication expertise and competences in computer based cognitive support tools are represented. All

at a high level. Publication will take place in the psychiatric field as well as the communication field.

3. Staff Other collaborators will be involved ad hoc.

It is

Clinicians at outpatient clinic for psychosis at Aalborg Psychiatric Hospital, Aalborg University

Hospital. Primarily nurses.

Administrative staff at DIHM at Aalborg University.

Administrative staff at DIHM at Aalborg University Hospital.

4. Background For many good reasons it is preferred that most psychiatric patients are treated as outpatients.

Unfortunately are the psychiatric treatment not an elimination of foreign element in the patient’s body, but

a challenging life situation in which the symptoms do reoccur with smaller or stronger significance. One

serious consequence is that many outpatients have symptoms that return despite that they have been

declared free of symptoms and they return to the clinic or hospital again.

Within some psychiatric patient groups up to 70% of the patients drop out of treatment and are later

referred to the hospital again.

Thus, within the group of patients referred to an ordinary psychosis clinic at a psychiatric hospital, a very

high dropout is often the case, and most of these patients discontinue simultaneously with their

psychopharmacological treatment with increased risk of relapse and re-hospitalisation. For psychotic

patients who are affiliated a clinic for first onset schizophrenia (Opus) or a clinic for assertive community

treatment (ACT) with a low caseload, e.g. 8-10 patient/CM, the dropout rate and noncompliance rate is

rather low, however this principles of outpatient treatment is rather expensive (Aagaard et al.(2014)).

The target group for Mobil technology in this study is patient referred to an ordinary outpatient clinic for

psychotic disorders.


/27

The mobile technology has evolved through the last decade and is now mature and everybody property.

Some researchers claim that “Your mobile is the friend that accompany your 24x7” (Fogg, B.J) and there has

been developed a wealth of health apps that support individuals in monitoring of own individual state and

compliance with a plan for a set of behavioural changes. Exercises and weight loss are well-known topics.

Unfortunately, are the usage of mobile technology in ambulant psychiatric treatment faced with a number

of challenges that diminish the research and usage of mobile technology in ambulant psychiatric treatment.

5. Problem Unfortunately, ambulant treatment has significant limitations related to the fact that it is ambulant. First of

all, the treatment, that is the clinical conversations, is based on the patient’s ability to recall of the last

periods situations, challenges and solutions. The recall bias is a well-known bias in studies of the human

mind. But a special problem for this type of patient is their cognitive impairment. The recall bias is even

stronger with this patient group. Secondly patient’s compliance with the clinicians advises can be limited.

Studies show that patients compliance with medical treatment can be significantly limited. Thirdly, the

clinical support when patients are challenged is limited – which especially is a significant limitation in

situation of a crisis.

For reasons above it seems valuable to supplement the current ambulant treatment with a 24x7 present

treatment.

6. Hypotheses - tentative The 6 hypotheses that will be examined in the project:

1. The usage of PRO data app promotes parameters in ambulant psychiatric treatment: presence at

clinical conversations, dropout rates, hospitalisation and medical compliance.

2. The usage of PRO data app promotes higher experienced patient satisfaction the patient will

improve their ability to handle crisis’ when using the PRO data app.

3. There exist assessments forms that are neither qualitative nor quantitative.

4. The usage of such assessments forms that are neither qualitative nor quantitative prove valuable in

clinical settings.

5. The usage of visualised PRO data will influence the clinician conversations in terms of patient

relevance and value and patient’s compliance with behavioural advices.

6. The usage of PRO-data and supplied mobile technology set new demands to clinicians and the

organisation of the ambulant clinic.

The project is entitled “The usage of PRO data in psychiatric ambulant treatment – a pilot study. (PROPS)”.


/27

7. State of the art State of the art with the field using patient data is a challenge state to investigate. Despite a significant

interest, the field does not form one single body. The opposite seems to be the case. The field is scattered.

Due to this lack of a single body the PubMed search string is:

("Patient Reported Outcome"[All Fields] OR "Ecological Momentary Assessment"[All Fields] OR "Self report"

[All Fields]) AND "schizophrenia"[MeSH Terms]

The result is 335 items. “Self Report” is an ambiguous term and most likely the items of interest are approx.

100.

The field is divided into two “terms”. "Patient Reported Outcome” and "Ecological Momentary

Assessment".

At search in the meta-meta-research database scholar.google.com gives an indication which term is most

popular

Ecological momentary Assessment(EMA) 9820 posts

Patient Reported Outcome(PRO) 29.000 posts

When meta-meta databases counts items there is a risk that items are counted several times. That factor in

increasing with higher numbers and the amount of research using the term “Patient Reported Outcome” is

probably larger than the EMA research but most likely a factor 2-2.5 larger.

The PRO research is obviously within health research while EMA research takes place in heath, but also

sociology, marketing and psychology. Here after we do use the term “Patient Reported Outcome” (PRO)

because of those reasons. We are aware that valuable research takes place within the Ecological

Momentary Research (EMA).

Researcher do not recognise their research related to a new phenomenon, a new way of doing data

collection, but to the field itself. E.g. do the journal “Dialogues in clinical neuroscience” have PRO as a term

for their journal (Thibaut, F. (2016)), but unfortunately do only the editorial note use the term!

A final indication of a not so mature research is that neither of the terms, "Patient Reported Outcome" and

“Ecological Momentary Assessment” are not PubMed Mesh topic.

Furthermore, do the rapid technological development limit the relevance of “old” “Patient Reported

Outcome” research. Android and IPhone mobile technology was invented in 2007 and had a remarkable

impact on the possibilities for designing useful PRO apps. The technology was utilised in 2008. Today the

usage and the technology has changed. Research experiments and their dissemination last 3-4 year. We

have not excluded older research but has paid attention to the timestamps of the article and the bulk for

the referenced articles are later than 2010.

The strategy for identifying valuable articles has been

Find most quoted articles – and articles that refer to the articles


/27

Find journals with focus on innovation in research rather than RCT and study articles about PRO

and EMA

Find articles on that combine PRO or EMA with schizophrenia or psychiatry - and articles that refer

to the articles

All three strategies have been investigated while having awareness to the relevant articles publication year.

When the Patient Reported Outcome field in this way scattered this state is scattered as well. Despite that

fact there are a number of interesting observations of the current state of the art.

Observations 1: Definition

Most articles starting point is a specific medical field. E.g. Molecular Biology, Pharmacy research or

eventually psychiatry or schizophrenia though the last 2 fields are rare.

Few articles (Ilda et al (2012), Meyer et al(2009), Shiffman et al. (2009), Deshpande et al.(2011) make an

attempt to give an overview of the field PRO or EMA. Unfortunately, is the “…terms overlap and are used

inconsistently, and these reports of experience should be distinguished from expressions of preference

regarding health states” (Meyer et al (2009)).

Two definitions of PRO/EMA can elicit some of the differences.

In the EMA field one article by (Shiffman et al. (2009), p3-4) important. In that article they define “EMA as a

collection of methods for obtaining repeated real-time assessment of the individual’ behaviour and

experience.

1. Data are collected in real-world environments

2. Assessments focus on subjects’ current state

3. Moments are strategically selected for assessment

4. Subjects complete multiple assessments over time”

(Shiffman et al. (2009), p3-4)

In the same article, (Shiffman et al (2009)) points out that assessment collected can be introspective

assessment and it can be biological measures and it can be data about the current real-world setting of the

individual.

(Deshpande et al. (2011) is a represent for the Clinician perspective on patient’ data collection. First of all

(Deshpande et al. (2011), p137) states that “the measurements come directly from the patient”, opposite

to data collected by the clinician. They frame the term by declaring “The outcomes are broadly classified

into

1. clinical (e.g. cure, survival),

2. humanistic (e.g. role performance,

3. emotional status) and

4. economical (e.g. expenses, saving)

The only common denominator of the two articles are 1) The data is collected by the individual 2) The data

is introspective.


/27

The latter have the consequence that the focus in much PRO/EMA research is in QoL and HRQOL.

Observations 2: Positive effect

Despite the lack of clear definition within the field(s) there is a shared pleasure stating the fact that data

collected the individual do have many positive properties.

(Trull et all(2009)) states that “ (a) Real-time assessments increase accuracy and minimize retrospective

bias; (b) repeated assessments can reveal dynamic processes; (c) multimodal assessments can integrate

psychological, physiological, and behavioral data; (d) setting- or context-specific relationships of symptoms

or behaviors can be identified; (e) interactive feedback can be provided in real time; and (f) assessments in

real-life situations enhance generalizability.”

(Shiffman et al. (2009)) states that a number of bias can be eliminated: most important recall bias, but

social bias and cognitive heuristic bias as well.

The consequence of the elimination of bias and the high sampling rates are that PROdata give a detailed

documentation of states of moment trends. Due to the fact that the patient has collected them increases

the credibility of the analysis afterwards.

Due to the fact that these bias can be eliminated QoL studies are often the case in PRO investigations.

Studies within the psychiatric field a number of positive features are identified:

a) In the article “Routine outcome monitoring and feedback on physical or mental health status: evidence

and theory” (Carlier et al.(2012),p107) concludes that “PRO data appears especially effective for the

monitoring of patients who are not doing well in therapy. Further research into this topic and the clinical-

and cost-effectiveness of ROM is recommended”. In this way, PRO data could be an alternative to

traditional treatment, but no substantial argument is given.

b) Regarding the Patient-Clinician collaboration “On the basis of the evidence, incorporating subjective

assessments into the treatment process itself and in treatment evaluation is warranted. Routine assessment

of patient-reported outcomes (e.g.needs for care) might reduce potential discrepancy between patients and

professionals, and improve outcome, although there is little empirical evidence to support this assumption

at present.” A viewpoint that (Carlier (2012),p105) finds supported in his review “Consistently positive

results of ROM as an outcome measure were shown on the communication between patient and health care

professional, both on the short and longer period.”

c) In the review article “Patient Reported Outcome in schizophrenia (McCabe et al (2009)) (one of the only

reviews within the schizophrenia field) the conclusions are supporting the general positive judgement of

PRO data, but request solid evidence documenting the properties.

In general, the measurement of most important parameters of psychiatric treatment e.g. presence at

clinical conversations and medical compliance is absent in the articles reviewed.

Observation 3: PRO data focus on measurements

Most PRO data investigations use predefined assessments scales (McCabe et al (2009)). On the other hand,

(Sartorius (2014), p123 (Patient-reported outcomes in psychiatry) argues that PRO’ “…recognition of the

partnership role that the patients should have in research on outcome of mental illness.” Further, “In their

recognition, treatment, and prevention psychiatrists rely on their observations and on communication with


/27

people who suffer from these disorders”. Sartorious’ point is that PRO data and its usage ought to be an

integral part of psychiatric treatment.

He points out that PRO data collection not is for the sake of the clinician. But for the patient’ ability to recall

situations, challenges and periods and bring them into the treatment.

In that way PRO data collection balance to needs 1) sound, reproducible data collection 2) Support for the

patient-clinician collaboration.

Observation 4: Technology insight: Data collection can be specialised/individualised

A main trend in computer language and data modelling through the decades has been a strong focus on the

ability to model generic cases and that the same time model and handle specialised cases. Invention of

object orientated programming languages on example. Invention of XML is another.

In the literature review no experiments is reviewed in which the possibility of individualising the PROapp

and the data collection is an issue. When studying articles which aims to establish a sound theoretical

framework for EMA/PRO(Ilda et al(2012),Meyer et al(2009), Shiffman et al. (2009), Deshpande et al.

(2011) there is no discussion on this property and potential absent. When considering media or technology

the considerations in all three articles lack a reflection on the properties of modern software architecture.

Which includes considerations on the potential of individualisation of PROapps.

Regarding reviews, McCabe et al. (2007) gives an overview of 21 investigations in the article “Patient-

reported outcomes in schizophrenia”. None of the 21 outcome scales that do include some sort of

individualisation. Meyer, K. B., & Clayton, K. A. (2009) comes to the same conclusion in their review but

points out that QoL measurements should be individualised.

“The domain coverage of commonly used generic short forms varies substantially. Individualized

measurement of quality of life is possible, but resource intensive”

The consequence for PRO data is that you need not give all patients the same treatment but similar to the

research in the individualisation of medication schizophrenic patients can get each their PROapp.

Generalisation/Individualisation of PRO data and -collection is not a topic in any of the PRO/EMA data.

Observation 5: Technology insight: Enduser programming

Programming has been considered as a demanding technical discipline that produced software as an

artefact. During the last 10 years this way of developing software has met limitations. Most important is the

limitation that technical skilled engineers have to understand a domain. This has been a significant barrier

in a number of fields. For that reason Enduser development tools (Dietricht et al.(2013)) has emerged and it

is most likely that it is an attractive way of developing PROapps for patients.

The consequence for PRO data and PROapp is that the professional psychiatric competence can be

integrated in the PRO app far easier than before.

Observation 6: Technology insight: Missing PRO data visualised

In the literature review there has been no discussion on topics like: 1) How are PRO data presented for the

patient? 2) How are PRO data integrated in the clinician treatment?

Despite the fact that the PRO data may change the fundament for the clinician treatment. The articles in


/27

the review do not contain one single data visualisation of PRO data despite the fact that data visualisations

can be a strong communication tool in the clinician-patient collaboration.

The consequence for PRO data and PRO app is that the benefits are not harvested.

Conclusions - state of the art

The main conclusion is that the usage of PRO data can be highly valuable in the psychiatric ambulant

treatment. However, the field is scattered theoretically and the experiments are close related to traditions

within special fields of medicine. The PRO data collection is compromise between sound, reproducible data

collection and data collection that support patient-clinician collaboration.

The impact of the usage of PRO data on important psychiatric parameters like precedence and medical

compliance is not well investigated.

8. Methods The PROPS investigation is a pilot study due to the fact that the current state of research is scattered. The

PROPS project collects a number of data sources and that facilitates an open-ended research. The

hypothesis’ demands that the feasibility PRO data and -app of the in several means.

The PROPS investigation will be followed up by a case control investigation with higher demands to validity

and reproducibility of research results.

The purpose of the pilot study is to perform a holistic evaluation of the Demands, Effects and Impact of PRO

data. A visualisation of the 6 hypotheses of the pilot study can be viewed below.

Clinicial performanceH1,H5

Technologys organisational impact

H6

Patients satisfaction

H2,H5

New assesments

H3,H4

Evaluation of the usage of PROdata


/27

Figure 1 Overview of Pilot studies Hypothesis's

After examination of the 6 hypothesis the pilot study will be concluded in an external review in which the

usage of PRO data will be examined. The external review will be a seminar with researchers in psychiatry,

health communication and health researchers in general.

An experiment in real clinical settings

The investigation takes place at the ambulant clinic for psychotic disorders Psychiatric Hospital/Aalborg

University Hospital. The number of participants are limited to 10 test persons and the test takes 4 months.

In the project design it is an aim to 1) make the test conditions as close to real clinical settings as possible.

2) to ease innovation of the clinical praxis with PRO data and -app.

Regarding the latter, the used technologies already exist: the server being used are the one that is already

being used in RN/Psychiatric Hospital and the mobiles that are being used is the test persons own.

To facilitate the open-ended research several empirical sources will be designed.

N Empirical source Type When

1 Test person health journal Existing Health record data

Before and during the experiment

2 Department retrieved: Patient population

data regarding dropout hospitalisations

figures and attendance list.

Department administration data

During the experiment

3 Screening (clinician observation) Combined interview and questionnaire

Before the experiment

4 Assessment (introspective assessment) Mobil app During the experiment

5 Logdata (behavior) and accelerometer

data from usage of the app

Mobile app interaction During the experiment

6 Log of technical breakdowns and

inceptions where patients or clinicians are

challenged.

A log maintained by the

the PROPS project

During the experiment

7 Clinician interview Combined interview and questionnaire

After the experiment

8 Single patient interview Interview combined with

analysis of patient’ data

After the experiment

9 Focusgroup interview Interview After the experiment Table 1 Overview of empirical sources collected in the PROPS project

This set enables includes quantitative as well as qualitative. It includes different perspectives on the usage

of PROapp -and data: how the PROapp actually has been used (Logdata), test persons own subjective

viewpoint and clinicians view on the test person and his usage of the PROapp.


/27

Furthermore, a pretest and post-test conditions are collected which enable us to investigate a development

of a change of attitude and change in usage of the PROapp.

Finally, the logdata documents the actual usage of the PROapp. How often, when and what types of

functionalities are being logged. This facilitate that the PROPS investigation can triangulate real usages and

test person subjective experience.

A walkthrough of the hypotheses:

Below a description of the methods used to examine the single hypothesis. The methods used are in most

cases open methods.

Hypothesis 1

The usage of PRO data app promotes parameters in ambulant psychiatric treatment: presence at clinical

conversations, drop-out rates, hospitalisation and medical compliance.

General hospital register data concerning consultations, drop-outs, and measurements for drug compliance

in the specific outpatient’s clinic, compared to cases (test persons).

Hypothesis 2

The usage of PRO data app promotes higher experienced patient satisfaction the patient will improve their

ability to handle crisis’ when using the PRO data app.

The interview with the test persons aims to detect the level of satisfaction with the PROapp tool and their

ability to react adequately in states of crisis. Further, the individual test person primary contact nurse will

be interviewed concerning specific management of a crisis situation. In parallel, an observational study will

take place in form of a log data investigation.

Hypothesis 3

There exist assessments forms that are neither qualitative nor quantitative.

PROPS project develops a number of quasi-qualitative/quasi-quantitative assessment types (qqqqa). An

example of such qqqqa is circumflex model that can be designed as a Visual Analog Scale, as two

dimensional visualisations and as a randomised word cloud. This is a design study that makes use of the

PROPS projects interdisciplinary composition. A number of design will be elaborated and in a heuristic

evaluation workshop (Stone et al (2005), p525-556) will take place in which the design will be inspected

from a Health Communication and a Psychiatric Clinical perspective.

Hypothesis 4

The usage of such assessments forms that are neither qualitative nor quantitative prove valuable in clinical

settings.

The evaluation of the qqqqa takes place as heuristic inspections (Stone et al (2005), p525-556) involving the

users (test patients and clinicians. The log data will be analysed as well.

Hypothesis 5

The usage of visualised PRO data will influence the clinician conversations in terms of patient relevance and

value and patient’s compliance with behavioural advices.

Based on recorded data we will analyse how visualised PRO data is incorporated into the clinic consultation

– especially, its sequential positioning and the attendant explanations and understandings as manifest in

talk-in-interaction.


/27

Hypothesis 6

The usage of PRO-data and supplied mobile technology set new demands to clinicians and the organisation

of the ambulant clinic.

The empirical sources used is 6 and 7. Empirical source 6 “Log of technical breakdowns and inceptions

where patients or clinicians are challenged” will be analysed in 2 dimension: 1) Severity of breakdown –

classified in a … 2) Whether the breakdown is caused by the PROapp software as such or the breakdown

can be categorised as “childhood disease”. Furthermore, clinicians are interviewed with regards to changed

work procedures and benefits of the PRO data technology. Again the identified challenges are categorised

as “childhood disease”. Or properties of PROapp software. The interview template is not yet elaborated.

Conclusions

The PROPS investigation is a pilot study and a prerequisite for a larger clinical experiment with higher

validity and reliability. The outcome of the PROPS experiment is first of all an evaluation of the feasibility of

PRO data in clinician settings from multiple perspectives. The PROPS research project will be able to have a

realistic judgement of all the various positive properties of PRO data develop a realistic clinic adaptable

technological setup.

9. Material The data collected in the PROPS project is illustrated in section 8 Methods, p9.

The actual design of the 2-6 is a topic of the investigation and is to be elaborated.

The EPJ data which will be requested for the PROPS project is essential information regarding their

diagnosis and treatment which includes the patients actual treatment.

The app is a tool for the ambulant patient and contains 3 functionalities: Subjective symptom assessment,

crisis card and visual symptom diary. All three functions can be adapted to the patient abilities and needs.

The adaption takes place as an integral part of the clinician-patient collaboration in a manner that ensures

that the patient do not suffer or is harmed in any way. The app prompts/notifies the patient 1-3 times per

day and request the patient to do a self assessment. The app stores these assessments, log data from the

app and data collected from the accelerometer during usage of the app. These data are available in the

treatment at the hospital. The data are stored at the serverpark of RN. So are the clinicians application

which enables the clinician to adapt the app to the patient and inspect the collected patient data in the

meetings with the patient. The patient app is being downloaded from Appstores and adapted to the single

patient when the patient insert her patientcode. If the patient deletes the app, the app is deleted.

10. Purpose For many good reasons many psychiatric patients are treated ambulant. In the age of mobile technology, it

seems obvious that ambulant treatment can be improved by the usage of mobile technology. The

foundation for a treatment is insight in the patient situation, challenges, symptoms and subjective

experience of his/her disease and situation uninterpretated should be the foundation for the treatment.


/27

Data collected at home/in ordinary situation are more reliable due to the fact that the recall bias and social

bias is eliminated, support for the patient in form of crisis cards can be present at hand for the patient

when patient faces a crisis. Furthermore, do the patient-clinician get a new tool for a collaboration, data

visualisations which is a new insight to the patient state and development.

All these observations are positive and can be found in the research literature. One article obviously call it

….a new era in clinical research.

Unfortunately do the state of art in the research not form a uniform image rather the opposite. (A full state

of the art can be found in the section 7 State of the art). The research is scattered using various labels

where the most used are Patient Reported Outcome (PRO). The possibility that patients can collect data

themselves attract interest from many areas within Health science. Nevertheless, the research in PRO data

has not shaped uniform interest within psychiatry and the ordinary pubmed search for articles return

meager results.

No Query Posts Source

1 "Patient Reported Outcome"[All Fields] AND

("schizophrenia"[MeSH Terms] OR "schizophrenia"[All

Fields])

9 Pubmed

3 "Patient Reported Outcome"[All Fields] AND

"psychiatry"[MeSH Major Topic]

3 Pubmed

5 "Schizophrenia"[All Fields] AND "mhealth"[All Fields]) 61 Pubmed

6 "Ecological Momentary Assessment"[All Fields] AND

(schizoph[All Fields]

10 Pubmed

8 "Ecological Momentary Assessment"[All Fields] AND OR

"psychiatry"[All Fields])

130 Pubmed

Due to this scattered image the purpose of the PROPS pilot study is to evaluate the costs and benefits of

using PRO data in clinical settings. A follow-up by a case control study will be performed.

Another observation in the research literature about technology. Mobile technology develops at a fast pace

and technology used in 2008 is different to the one that is being used in 2015. The user/patient capability

to use mobiles are different too. The technology enables in 2015 that apps can individualised to the single

individual needs. Furthermore, the technology enables the classical divide in assessment design to be

questioned. Assessment need not only be numbers, categories or text, but more valuable assessment can

be developed.

The PRO data collected by the patient will have to be integrated in the clinician-patient collaboration.

Unfortunately, PRO articles have few articles where this is a topic. Data visualisation technologies fertilize

the collaboration by bringing data collected by the patient into the collaboration and the data is not a single

event but data shown on a timeline. Again this tool for collaboration has to be studied.

Implementation of new technology in the treatment of ambulant patients should not only fulfil demands to

a more successful treatment, but should function in the organisation, should be easily adaptable for the

patient, be valuable for the clinicians and improve the efficiency of the treatment of ambulant psychiatric

patients. The purpose of the pilot study is to do a holistic evaluation of the effects of the usage of PRO data


/27

app in the treatment of ambulant schizophrenic patients. The pilot study and the following external review

of the studys purpose is to design a larger and more valid in case control study.

11. Statistical considerations The PROPS project is benefitting from modern science challenge and burden. The low cost of data

collection and a significant number of valuable empirical sources that need to be triangulated.

PROPS is a pilot study with 10 participants, and with many data from different data sources. Only

descriptive statistics are applied.

Calculation of sample size for primary and secondary outcome measurements does not make sense.

However, when the results of the pilot study are available a new protocol for a PRO app case-control study

will be elaborated.

To enable data triangulation a data warehouse and a data visualisation worksheet will be elaborated. The

analysis of the empirical is methods from human science and medicine. The data warehouse can handle

both. So can the data visualisation worksheet.

1. Data warehouse

The data warehouse will form one uniform access to all data sources and enable to see each piece of data

by itself but most important will the data warehouse enable accessing related data fast. Most important get

access to all data about a patient.

What Technical format

1 Test person health journal Unknown. Most likely Excel

2 Department retrieved: Patient population data regarding

dropout and hospitalisation figures and attendance list.

Unknown. Most likely Excel

3 Screening (clinician observation) Excel

4 Assessment (introspective assessment) MS-SQL(resides at RN/KoncernIT)

5 Logdata (behavior) and accelerometer data from usage of

the app

MS-SQL(resides at RN/KoncernIT)

6 Log of technical breakdowns and inceptions where

patients or clinicians are challenged.

MS-SQL(resides at RN/KoncernIT)

7 Clinician interview MS-SQL(resides at RN/KoncernIT)

8 Patient Interview MS-SQL(resides at RN/KoncernIT)

9 Focusgroup interview MS-SQL(resides at RN/KoncernIT)


/27

2. Data visualisation

To combine and visualise the many data sources tableau has been chosen. The tableaus facilitate rapid

design of a large variety of graphs that can be combined in dashboards.

PROPS is a pilot study with 10 participants, and with many data from different data sources. Only

descriptive statistics are applied.

Calculation of sample size for primary and secondary outcome measurements does not make sense.

However, when the results of the pilot study are available a new protocol for a PRO app case-control study

will be elaborated.

Hypothesis 1

For the Performance Measurements Tableau will offer all descriptive statistics visualisations and

calculations. If further statistical analysis is required, the statistical equitation has to be formulated and

integrated in the Tableau workbook in the same manner as SPSS or SAS.

Hypotheses 2, 3, 5 and 6

The palette of methods has not yet been decided but dashboards that facilitates simple seeing the data can

rapidly be developed. Data dashboards that enable the researchers to study the same phenomena is easy

to develop as well. The PROPS project will develop its own data visualisations that is suitable for exactly

that kind of analysis activities that is most relevant. 5-15 dashboards can be elaborated.

If tableaus data visualisations do not seem appropriate, the data warehouse can be accessed directly and

data can be seen in other tools.

12. Implementation The pilot study takes place at an outpatient clinic for psychotic disorders at Aalborg University Hospital. The

patient data from the EPJ is gathered when the Informed Consent document have been signed by all test

patients.

The recruitment of test patients is the responsibility of the clinicians of the department at Brandevej in

collaboration with the PROPS project responsible.

The exacted data are kept in excel files and each patient are only identifiable by a randomized serial

number. The randomized serial number -test person identification table does on exists in one copy and this

only copy resides at the PROPS project responsible at the Aalborg University Hospital.

The PRO app data produces data and they are stored at a database at KoncernIT/RN. The handling of the

data while they are on the patients mobile, while being transmitted to the server and stored on the server

takes place according to the regulations and recommendations from Region Nord regarding the usage of

mobile phones. (Cornelliussen (2014)).

The instructions in usage of the PRO app, take place as an integral part of the ordinary treatment of the test

patients. So due the continuous evaluation of the use of the PRO app and its data and the technical

support.


/27

The final focusgroup interview of the test patients do also take place in the Brandevej department.

Regarding the PRO app it should be mentioned that there exists a PRO app toolkits which is the starting

point of the sw development. During the first 3 month of the project this PRO app toolkits app will be

tested intensively and the outpatient clinic suitable PRO app will be configured.

A permission to implement the pilot study at the head of the Psychiatry/Aalborg University Hospital will be

applied.

The Pilot study design contains a “Bring-Your-Own-Device”(BYOD) attitude, the PRO app toolkit are able to

create cross platform apps(Android and IPhone platform) and the server part is integrated in the ordinary

RN IT infrastructure. In that way the PROPS can converge from a pilot study to a tool for the outpatient

clinic at Brandevej.

Due to the fact that the PROPS project is a pilot study it is an aim that the administrative overhead is kept

minimal.

13. Biobank Not relevant.

14. Testparticipants The inclusion criteria are:

Patient diagnosed and treated for schizophrenia at least 1 year .

Men and women, age >18.

Informed consent.

Outpatients attending clinic for psychotic disorders,Psychiatric hospital,Brandevej, will be selected.

Outpatients are enrolled as independent patients and can at any stage of a treatment leave the

treatment.

The exclusion criteria are:

Patients that exhibit uncontrolled, random behaviour.

Patients that recently have made suicide attempts.

Patients enrolled in Assertive Community Treatment(ACT).

Actions taken as described in the “Informed consent” document, when exclusion criteria occur during

the test.

As mentioned in the section Ethic , the patient can leave during the test by of own will and can be

advised to do so by the project responsible.


/27

15. Side effects, risks and cons The test person is asked to install and use a PROPS mobile app during the test person. The PROPS mobile

app will prompt the test person 0-4 times per day.

The precautions are:

If any technical problems occur, they can contact the PROPS hotline. If problems related to the treatment

occurs, they can contact PROPS clinician.

It is possible that the test person mental state is influenced negatively. The test person is instructed (The

Informed Consent) that they are free to deinstall the PRODE mobile app anytime. Clinicians are instructed

to be aware of a negative change in the test person mental state and they can anytime stop the usage of

the PROPS mobile app too.

All interviews including the final focus group interview last in average 1 hour. There are 3 interviews

planned. There are no side effects of interviews besides that they can be tiresome. Nevertheless, a

debriefing after the focus group is offered to the test persons.

16. Respect of the test participant’s mental integrity and data privacy Information regarding the test person is protected by law regarding to handling of data privacy and in

accordance with health legislation. The project will be registered at The Danish Data Protection Agency.

There will be used data from the EPJ in the research project which is documented in Appendix B, p23.

Afterwards, additional information can be accessed only if the patient and clinician who have been

responsible for the treatment permit such access according to health legislation

The actual recruitment of patients is organised in this way: the clinicians of the ambulatory will be informed

about the experiment and the clinicians will select appropriate patients from the patient population.

17. Application of EPJ data in the experiment The EPJ data which will be requested for the PROPS project is essential information regarding their

diagnosis and treatment. The data include the patients actual treatment.

The purpose to of usage of the EPJ data is to verify the inclusion criteria and to use the EPJ data as data for

the analysis of the intervention.

18. Budget and financies The pilot study investigation last totally 1 years. It includes software setup, clinician test, analysis and

dissemination of research results. The budget can be revised in Appendix D Budget.

Potential sources of funding are:

A.P. Møller Fonden

Augustinusfonden

Axel Muusfeldts Fond


/27

Danske Banks Fond

Den Bøhmske Fond

Det Obelske Familiefond

ENV- Fonden

Forløbsprogram for mennesker med psykiske lidelser - Socialstyrelsen

Frimodt-Heineke Fonden

Gangstedfonden

Harboefonden

Helsefonden

Jascha Fonden

Knud Højgaards Fond

Købmand Sven Hansen og hustru Ina Hansens Fond

Lundbeckfonden

Nordea-fonden

Torben og Alice Frimodts Fond

Aase og Ejnar Danielsens Fond

The investigations responsible and the collaborators in the PROPS project do not receive any kind of

support to implement the investigation. There is no commercial support. The PROPS project is exclusively

financed by public and private funds. When financial support will be obtained the support will be deposited

on a research account. The budget is accessible in Appendix D Budget.

19. Recruitment of test participants and test participant’s remuneration The test person does not receive remuneration or other kinds of support besides the ordinary clinician

treatment. The overall idea of the pilot project is to study the impact of the usage of Patient Reported

Outcome in the treatment of schizophrenic patients in real clinician settings.

The test persons are recruited among patients diagnosed with schizophrenia and referred to the outpatient

clinic for patients who suffer from a psychotic disorder at Psychiatric Hospital, Aalborg.

The principal investigator is responsible for recruitment of patients to the study.

When patients have shown interest in participation a meeting can be setup between Research test

responsible, patient, clinician and eventually patients relatives. If the patient still is willing to participate the

patient receives “Test person information form” (see Appendix F Test person information form) and

“Information Consent” (see Appendix G Informed Consent Document) and one week of consideration

before the final participation in the research project is confirmed.

After approximately 1-2 weeks the patient will be enrolled in the PROPS project. The participation in the

project is equal to attending ordinary clinician activities and the usage of a mobile app at home. The data

from the patients mobile (behavioural data and assessment data) will be used in the clinician activities.


/27

20. Informed Consent See description in section above (Recruitment of test participants and test participant’s remuneration). The

Form “S1” from ”samtykkeerklæringS1S122012.doc” is being used as Informed Consent confirmation.

21. Dissemination of research results Results of the research will be published regardless of the results are positive, negative or inconclusive. This

will take place as soon as possible with respect to high professional standards and according to the data

privacy regulations.

When results cannot be published in journals the results will be published elsewhere (e.g.

www.clinicaltrial.gov or www.clinicaltrialsregister.eu).

In the PROPS project the following guidelines regarding publication. To be (co-)author of a publication a

substantial contribution is required in minimum 2 areas. 1) Project planning/implementation of the

protocol, 2) Implementation of the data collection, 3) The statistical elaboration/analysis, 4) Elaboration of

the publication.

The PROPS project is already publishing results from PROPS project.

(Aagaard, M. (2016) – a poster representation and Aagaard, M. (2016) – an oral presentation. Both at

Comet2016.

22. Ethic research With regards to advantages of the participation:

Ambulant schizophrenia patients the offer to such patients are treatment that takes place at the hospital.

When using mobile technology treatment can be present 24x7. That is the patient can be assessed in real

world settings, the patient can have advices and support present 24x7. Truly ambulant treatment. The

patient will in the PROPS project gain from the experience of this ambulant treatment. The advantages,

challenges and disadvantages. Short term and long term.

With the regards to disadvantages of participation:

Ambulant schizophrenia patients will be using a mobile phone app that they can interact with or neglect.

However, a premium feature of Patient Reported Outcome data collection is that the software itself can

initiate an interaction. The mobile app can prompt the user for an assessment or possibly suggest a set of

behaviour (exercise or the like). Such prompt may in worst case cause unintended set of behaviours or

recalls that again lead the patient in a crisis being alone at home.

In our review in the PRO research we have not read about such cases and such backside of data collection is

not mentioned in the PRO research neither in the research field Ecological Momentary Assessment.

The PROPS project has three mechanisms to avoid such disaster. 1) During the recruitment of test persons,

patients that exhibit uncontrolled, random behaviour are not included. Patients that recently have made

suicide attempts I nether included. 2) While the test takes place the clinicians shall monitor the patients


/27

state and when the patients state changes to worse, consider if the patient should end her/his

participation. 3) Whenever the patient asks to leave the PROPS project the patient are free to do so.

The second disadvantage of the participation is that there exists a possibility that a test person actually gets

a great treatment by using the PRO app and gets dependent of the usage of the PROapp. In such cases do

the PROapp project find a suitable substitute.

23. Disclosure of compensation or reimbursement schemes The Patient Compensation Association, chapter 3 will cover harm which a test person may experience in

the PROPS project.

24. References

Albrechtslund, A. (2007). Ethics and technology design. Ethics and Information Technology, 9(1), 63-72.

Carlier, I. V., Meuldijk, D., Van Vliet, I. M., Van Fenema, E., Van der Wee, Nic JA, & Zitman, F. G. (2012). Routine outcome monitoring and feedback on physical or mental health status: Evidence and theory. Journal of Evaluation in Clinical Practice, 18(1), 104-110.

Chen, E. Y., Hui, C. L., Lam, M. M., Chiu, C. P., Law, C., Chung, D. W., et al. (2010). Maintenance

treatment with quetiapine versus discontinuation after one year of treatment in patients with remitted first episode psychosis: Randomised controlled trial. Bmj, 341, c4024.

Cornelliussen, S. Sikkerhed i mobile løsninger (sikkerhedskrav - mobility.docx) KoncernIT/RegionNord, accessed 02/2015

Csikszentmihalyi, M., & Larson, R. (1987). Validity and reliability of the experience-sampling method. The Journal of Nervous and Mental Disease, 175(9), 526-536.

Deshpande, P. R., Rajan, S., Sudeepthi, B. L., & Nazir, C. A. (2011). Patient-reported outcomes: A new era in clinical research. Perspectives in Clinical Research, 2(4), 137.

Dittrich, Y.,1964-, & IS-EUD 2013 (2013 : Copenhagen,Denmark). (2013). End-user development : 4th

international symposium, IS-EUD 2013, copenhagen, denmark, june 10-13, 2013. proceedings. Berlin; New York: Springer. Retrieved from http://dx.doi.org/10.1007/978-3-642-38706-7

Fogg, B. J. (2003). Persuasive technology: Using computers to change what we think and do. San Francisco, CA: Morgan Kaufmann.

Folker, M., Rasmussen, J., Pedersen, K., Brocks, N., & D’Auchamp, A. (2014). Digital mental sundhed–Kortlægning og technology roadmap for digitale værktøjer til fremme af mental sundhed og selvhjælp ved psykisk sygdom.

Folker, M., Rasmussen, J., Pedersen, K., Brocks, N., & D’Auchamp, A. (2014). Digital mental sundhed–Kortlægning og technology roadmap for digitale værktøjer til fremme af mental sundhed og selvhjælp ved psykisk sygdom.

Freudenreich, O., & McEvoy, J. P. (2012). Optimizing outcome with antipsychotic treatment in first-episode schizophrenia: Balancing efficacy and side effects. Clinical Schizophrenia & Related Psychoses, 6(3), 115-121.

http://dx.doi.org/10.1007/978-3-642-38706-7


/27

Hansen, T. (2011). AT FANGE FUGLENS FLUGT. om dagbogsmetoder og nogle hukommelsesmæssige grunde til at anvende dem. Retrieved 5/14, 2014, from http://www.hum.aau.dk/~tia/TilAfhentning/Hansen%202007%20DbUs.pdf

Henderson, C., Flood, C., Leese, M., Thornicroft, G., Sutherby, K., & Szmukler, G. (2004). Effect of joint crisis plans on use of compulsory treatment in psychiatry: Single blind randomised controlled trial. BMJ (Clinical Research Ed.), 329(7458), 136.

Ho, B., Andreasen, N. C., Ziebell, S., Pierson, R., & Magnotta, V. (2011). Long-term antipsychotic treatment and brain volumes: A longitudinal study of first-episode schizophrenia. Archives of General Psychiatry, 68(2), 128-137.

Hunter, R., Cameron, R., & Norrie, J. (2015). Using patient-reported outcomes in schizophrenia: The scottish schizophrenia outcomes study. Psychiatric Services,

Iida, M., Shrout, P. E., Laurenceau, J. P., & Bolger, N. (2012). Chapter 15, using diary methods in psychological research,p277-306. In H. Cooper (Ed.), APA handbook of research methods in psychology, vol I, foundations, planning, measures and psychometrics (). Washington. D.C: American Psychological Association.

Kashdan, T. B., & Collins, R. L. (2010). Social anxiety and the experience of positive emotion and anger in everyday life: An ecological momentary assessment approach. Anxiety, Stress, & Coping, 23(3), 259-272.

Kimhy, D., Myin-Germeys, I., Palmier-Claus, J., & Swendsen, J. (2012). Mobile assessment guide for

research in schizophrenia and severe mental disorders. Schizophrenia Bulletin, 38(3), 386-395. doi:10.1093/schbul/sbr186 [doi]

Krägeloh, C. U., Czuba, K. J., Billington, D. R., Kersten, P., & Siegert, R. J. (2014). Using feedback from

patient-reported outcome measures in mental health services: A scoping study and typology. Psychiatric Services,

Kurhila, J., Miettinen, M., Niemivirta, M., Nokelainen, P., Silander, T., & Tirri, H. (2001). Bayesian modeling in an adaptive on-line questionnaire for education and educational research. Proceedings of the 10th International PEG2001 Conference, pp. 194-201.

McCabe, R., Saidi, M., & Priebe, S. (2007). Patient-reported outcomes in schizophrenia. The British Journal of Psychiatry, 191(50), s21-s28.

Meyer, K. B., & Clayton, K. A. (2009). Measurement and analysis of patient-reported outcomes. Clinical Epidemiology: Practice and Methods, , 155-169.

Palmier-Claus, J. E., Rogers, A., Ainsworth, J., Machin, M., Barrowclough, C., Laverty, L., et al. (2013). Integrating mobile-phone based assessment for psychosis into people’s everyday lives and clinical care: A qualitative study. BMC Psychiatry, 13(1), 1.

Pluye, P., Grad, R. M., Stephenson, R., & Dunikowski, L. G. (2005). A new impact assessment method to evaluate knowledge resources. Paper presented at the Amia,

Rainer, T. (1978). The new diary: How to use a journal for self-guidance and expanded creativity JP Tarcher Los Angeles.

Reininghaus, U., & Priebe, S. (2012). Measuring patient-reported outcomes in psychosis: Conceptual and methodological review. The British Journal of Psychiatry: The Journal of Mental Science, 201(4), 262-267. doi:10.1192/bjp.bp.111.107615 [doi]

http://www.hum.aau.dk/~tia/TilAfhentning/Hansen%202007%20DbUs.pdf


/27

Sartorius, N. (2014). Patient-reported outcomes in psychiatry. Dialogues in Clinical Neuroscience, 16(2), 123-124.

Sartorius, N. (2014). Patient-reported outcomes in psychiatry. Dialogues in Clinical Neuroscience, 16(2), 123-124.

Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momentary assessment. Annu.Rev.Clin.Psychol., 4, 1-32.

Shiffman, S. (2009). Ecological momentary assessment (EMA) in studies of substance use. Psychological Assessment, 21(4), 486.

Staniszewska, S., Haywood, K. L., Brett, J., & Tutton, L. (2012). Patient and public involvement in patient-reported outcome measures. The Patient-Patient-Centered Outcomes Research, 5(2), 79-87.

Stone, D., Jarrett, C., Woodroffe, M., & Minocha, S. (2005). User interface design and evaluation Morgan Kaufmann.

Thibaut, F. (2016). Dialogues in clinical neuroscience. Dialogues Clin Neurosci, 2014 Jun; 16(2), 01/03/2016.

Trull, T. J., & Ebner-Priemer, U. W. (2009). Using experience sampling methods/ecological momentary assessment (ESM/EMA) in clinical assessment and clinical research: Introduction to the special section.

Tversky, A., & Kahneman, D. (1973). Availability: A heuristic for judging frequency and probability. Cognitive Psychology, 5(2), 207-232.

Wiklund, I. (2004). Assessment of patient‐reported outcomes in clinical trials: The example of health‐related quality of life. Fundamental & Clinical Pharmacology, 18(3), 351-363.

Aagaard, J., Skadhede, S. C., Andersen, M. B., Kølbæk, P., Hastrup, L. H., Foldager, L., et al. (2014). Opsøgende psykoseteams i Region Nordjylland. Aalborg: Aalborg Universitetsforlag.

Aagaard, M. (2016). Visualisation of patient data – for patients(poster). Comet 2016,

Aagaard, M., Aagaard, J., & Sarangi, S. (2016). EMA-apps can improve outcome for assertive community treatment patients. A pilot study. Comet 2016, Aalborg University, Denmaerk.

aan het Rot, M., Hogenelst, K., & Schoevers, R. A. (2012). Mood disorders in everyday life: A systematic

review of experience sampling and ecological momentary assessment studies. Clinical Psychology Review, 32(6), 510-523.


/27

Appendix A Description of Method No data is collected besides the one that already has been described. The test persons are not exposed to

any kinds of investigation besides participation in the PROPS experiment.

Appendix B Description of the data requested from the EPJ The EPJ data which will be requested for the PROPS project is essential information regarding their

diagnosis and treatment. Which includes the patients actual treatment.

The purpose to of usage of the EPJ data is to verify the inclusion criteria and to use the EPJ data as data for

the analysis of the intervention.

Appendix C Documentation of the registration to The Danish Data

Protection Agency The protocol will be submitted to The Danish Data Protection Agency for registration.


/27

Appendix D Budget


/27


/27

Appendix E Time schedule for the project


/27

Appendix F Test person information form The template “Komitésystemets standardskabelon, februar 2009” will be adapted to the PROPS project.

Appendix G Informed Consent Document The PROPS project’ Informed Consent” template is “S1” in “samtykkeerklringS1S122012.doc” adapted to

the PROPS project.

props - communication.aau.dk · sociology, marketing and psychology. here after we do use the term...

Documents