method and materialssummit.sfu.ca/system/files/iritems1/18925... · this powerpoint template...

1
RESEARCH POSTER PRESENTATION DESIGN © 2019 www.PosterPresentations.com Mobile sleep apps are viewed as promising accessible treatments for insomnia. Using them as data collection tools akin to sleep diaries has also been proposed. Most of these apps, however, have not been developed using evidence-based principles; limited research also exists on their design as research tools (Bhat et al. 2015; Yu et al., 2019). In the present study, we explored the opportunities and challenges experienced when using a mobile app for research with our own team’s research study as the unit of analysis. This is an intrinsic case study (Stake, 1995), which can inform other researchers on how to approach their studies when using sleep apps in research as an intervention (treatment) or research tool (data collection). Context: The Somnotest Study Data were collected during a larger study, designed to test the effects of serial diverse imagining ( SDI, a form of cognitive shuffling; Selham et al., 2018), using SomnoTest, on insomnia. In that study, the data of 19 control participants and 15 treatment participants, aged 18 to 30 years, all of whom reported insomnia, were analysed. Participants were assigned to one of two app conditions. Group 1 participants heard a countdown from 99 to 1, and Group 2 were prompted to visualise randomly selected brief scenes read by the app at eight-second intervals (SDI). Method and Materials Findings Discussion Our findings reveal that using sleep apps in collaboration with sleep diaries could lead to greater certainty that sleep diary data have been accurately reported, especially if participants are aware that their mobile usage patterns will be tracked and compared against diary data. From a research methodology and administration perspective, this would entail working closely with the mobile app development team to ensure that timely identification of non-compliance with study procedures can be made. Although sleep app data have the potential to contribute to our understanding of mobile use in naturalistic conditions, the unstructured form of the data collected will prove too costly to analyse for many researchers, unless more efficient means of data analysis can be developed. Data science and data analytics can provide direction; however, the initial identification of typical and valid sequences of user actions must first be identified so that effective algorithms can be developed. These algorithms can then be used to create visualisations that researchers can use to identify patterns of interest in the data more efficiently and effectively. Another challenge will also be to ensure that algorithms do not limit insights to only those patterns identified prior to data collection, thereby limiting potential new insights that could potentially be gleaned. Conclusion Informing participants that app data will be triangulated against sleep diaries during data collection and analysis may ensure greater accuracy of participant-provided sleep diary data. The development of an algorithm that can efficiently filter valid data usage patterns would facilitate data analysis and researchers’ experience. This would increase sleep app usability as a treatment and research tool. Developing a process for increasing efficiency in data analysis is necessary to exploit the advantages of large-scale data collection that a sleep app makes possible. This study, although preliminary, provides insights on how to improve research practice in the context of mobile app usage in sleep research, in terms of using the app as both intervention and research tool. Insights in the conceptualisation of a visualisation feature that could be developed to provide more efficient identification of usage patterns has also been proposed. References and conflict of interest COI disclosure: Luc P. Beaudoin is a director of CogSci Apps Corp. and owner of CogZest. Both of these Canadian businesses develop products based on his affective / cognitive science. These include books, software, and training services. Case study Using a qualitative approach involved the direct interpretation (Stake, 1995) of participants’ patterns of mobile app usage, based on actions recorded (i.e., press start, end, pause, resume, or cancel; time stamp; count of played items), and the reorganization of user actions into tables (visualisation; tabulation) to identify usage patterns. The researchers’ reflection notes on their respective experiences in analyzing the data were analysed as lived experiences that could inform more effective data analysis procedures when working with raw data from mobile apps, with the objective of deriving themes to inform improved research practice in this context. Thematic analysis of these experiences was also conducted to reveal exemplars of situations that researchers could face. Selham, Z 1,2 ., Guloy, S 3 ., Bastien, C 4 ., Beaudoin, L 3 ., Carrier, J 1,2 1 Psychology department at Université de Montreal, 2 Hôpital du Sacré Cœur de Montréal, 3 Simon Fraser University, Vancouver, CogSci Apps and CogZest (see disclosures below) 4 Dépt. de Psychologie, Université Laval Research process and sleep app design lessons learned from the reflective examination of a sleep study Raw data for all control (n=23) and intervention (n=31) participants prior to data cleaning Figure 2. Raw data for all those who participated in the study, including those whose data were not used in the larger study’s analysis. 1907 total actions were logged. The initial step in data cleaning involved ensuring that only those participants who submitted sleep diaries are included in the analysis. Figure 3. Step two in the analysis involved a line-by-line analysis of each action, taken by participants or recorded by the app, to identify patterns as part of the data cleaning process. Figure 4. Only through time-intensive, line-by-line analysis could spurious data patterns be discerned. Figure 5. Step 3 involved identifying emerging patterns from cleaned data, in terms of participants’ mobile app usage, through the creation of a spreadsheet, which provided a visual representation of the data. This step facilitated easy identification of participants who did not use the app as stated in their sleep diaries as well as of other patterns of usage (e.g., heavier app use throughout the night). Figure 6. Conceptual mockup for an app usage visualisation feature to support researchers in identifying usage patterns and anomalies in user actions (green = start; red = end; orange= pause; yellow = cancel; blue= resume ) This feature could be used to identify more efficiently the usage patterns of participants through a visual representation of the collected data, making use of data analytics to screen out spurious data patterns. This mockup reflects similar data patterns to that identified in Figure 5. Allocca, G., Ma, S., Martelli, D., Cerri, M., Del Vecchio, F., Bastianini, S., ... & Blackburn, S. (2019). Validation of' Somnivore', a Machine Learning Algorithm for Automated Scoring and Analysis of Polysomnography Data. Frontiers in Neuroscience, 13, 207. Morin, C. M., Drake, C. L., Harvey, A. G., Krystal, A. D., Manber, R., Riemann, D., & Spiegelhalder, K. (2015). Insomnia disorder. Nature Reviews Disease Primers, 1, 15026. Selham, Z., Guloy, S., Carrier, J., & Beaudoin, L. (2018). Effects of using a mental imagery mobile app on the subjective quality of sleep: preliminary findings. Poster presented at the 29th International Congress of Applied Psychology: "Psychology: Connecting Science to Solutions". Montreal, QC. Stake, R. E. (1995). The art of case study research. Sage. Introduction Figure 1. SomnoTest mobile app Four themes: 1) Unreliability of sleep diaries when triangulated against SomnoTest data, given that 9 participants had not used the app as claimed; 2) Complex, intensive qualitative analysis is needed to identify valid data in an unstructured data set; 3) Importance of visualisation when examining data to uncover patterns; 4) Identification of “fans” who continue to use the app when not required during the study and after their participation in the study. Our findings reveal that data cleaning involves intensive line-by-line and case-by-case analysis of participant data, which proved challenging with 34 participants and would prove prohibitive for larger scale studies. Acknowledgments: Drs. Guloy and Beaudoin would like to thank Dr. Phil Winne for introducing us, many years ago, to the importance of software logging and analysis for scientific research. Ask developer

Upload: others

Post on 15-Oct-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Method and Materialssummit.sfu.ca/system/files/iritems1/18925... · This PowerPoint template produces a 36"x48" presentation poster. You can use it to create your research poster

RESEARCH POSTER PRESENTATION DESIGN © 2019

www.PosterPresentations.com

QUICK START GUIDE(THIS SIDEBAR WILL NOT PRINT)

This PowerPoint template produces a 36"x48" presentation poster. You can use it to create your research poster by placing your title, subtitle, text, tables, charts and photos.

We provide a series of online tutorials that will guide you through the poster design process and answer your poster production questions. For complete template tutorials, go online to PosterPresentations.com and click on the HELP DESK tab.

To print your poster using our same-day professional printing service, go online to PosterPresentations.com and click on "Order your poster".

This is a template for a presentation poster

36 inches tallby

48 inches wide

Important: Check the template sizeBefore you start working on your poster and to avoid printing problems check that you have downloaded and that you are using the correct size template for your poster presentation.This template can also be printed at the following sizes without distortion and without any additional formatting:30 tall x 40 wide42 tall x 56 wide48 tall x 64 wide

How to Zoom in and outUse the PowerPoint zoom tool to adjust the screen magnification to view comfortably. PowerPoint provides 2 ways to zoom: 1. On the top menu bar click on the VIEW tab and then click on ZOOM. Choose the zoom percentage that works best for you. 2. For better zoom flexibility, use the zoom slider at the bottom right of the window.

Ruler and GuidesThe dotted lines on his poster template are guides. The horizontal and vertical guides will help you align your poster elements accurately. Text boxes and other elements will ”snap” to the guides and stay within the boundaries of the columns. To hide the guides go to VIEW and uncheck the Guides box.

Headers and text containersIncluded in this template are commonly used section headers such as Abstract, Objectives, Methods, Results, etc. - Click inside a section header to add its text. - To add another header, click on edge of the section box so that it is outlined. Copy and paste it. - To increase its size, click on the white circles and expand to the the desired size.

Adding content to the posterStart by adding your text to each section without spending too much time with formatting. Use the default font size even if your text extends beyond the bottom of the poster. Continue until you have added all your content including text, graphics, photos, etc. Once you finish adding your content you can go back and format your text as needed.- If you run out of room, try to reduce the size of your fonts and/or the size of your

graphics. If there is a lot of empty space try to increase your font sizes and the size of your graphics. The font used for references can be smaller.

PhotosYou can add photos by dragging and dropping from your desktop, copy and paste, or by going to INSERT > PICTURES. Resize images proportionally by holding down the SHIFT key and dragging one of the white corner handles (dots). For a professional-looking poster, do not distort your images by stretching them disproportionally.

Quality check your graphicsZoom in and look at your images at 100%-200% magnification. If they look clear, they will print well.

QUICK START GUIDE(THIS SIDEBAR WILL NOT PRINT)

How to change the template colorsYou can change the overall template color theme by clicking on the COLORS dropdown menu under the DESIGN tab. You can see a tutorial here:

You can also manually change the color of individual elements by going to VIEW > SLIDE MASTER. On the left side of your screen select the background master where you can change the template background, column sizes, etc.

After you finish working on the SLIDE MASTER, it is important that you go to VIEW > NORMAL to continue working on your poster.

How to change the column layout configurationYou can manually change the configuration on the columns by going to VIEW > SLIDE MASTER. You can delete columns, resize them or modify them as needed for your layout. You can see a tutorial here: https://www.posterpresentations.com/how-to-change-the-column-configuration.html

How to hide the QUICK START GUIDE bars from the sides of the templateThe Quick Start Guides are outside the template’s printable area and they will not be on the printed poster.

If you create a PDF file from your template, the guides will not be included.

To hide the guides click on the Home tab (top of the screen) and then click on the Layout button below to see the available layouts. Choose the Without Guides layout.

How to preview your poster prior to printingYou can preview your poster at any time by pressing the F5 key on your keyboard. You will see on the screen what's on your poster and how it should look when printed. Press the ESC key to exit Preview.

F5

How to print your posterWhen you are ready to have your poster printed go online to PosterPresentations.com and click on the "Order Your Poster" button. You can have your poster printed on professional papers, fabric for easy traveling and a variety of other materials. If you submit a PowerPoint document, you will be receiving a PDF proof for your approval prior to printing. If your order is placed and paid for before noon (Pacific time) Monday-Friday, your order will ship out that same day. FedEx Next day, Second day, Third day, and Free Ground services are offered.

Go to PosterPresentations.com for more information.

© 2019 PosterPresentations.com2117 Fourth Street , STE C Berkeley CA 94710 USA

For complete tutorials visit:https://www.posterpresentations.com/helpdesk.html

Mobile sleep apps are viewed as promising accessible treatments for insomnia. Using them as data collection tools akin to sleep diaries has also been proposed. Most of these apps, however, have not been developed using evidence-based principles; limited research also exists on their design as research tools (Bhat et al. 2015; Yu et al., 2019).

In the present study, we explored the opportunities and challenges experienced when using a mobile app for research with our own team’s research study as the unit of analysis. This is an intrinsic case study (Stake, 1995), which can inform other researchers on how to approach their studies when using sleep apps in research as an intervention (treatment) or research tool (data collection).

Context: The Somnotest Study Data were collected during a larger study, designed to test the effects of serial diverse imagining ( SDI, a form of cognitive shuffling; Selham et al., 2018), using SomnoTest, on insomnia. In that study, the data of 19 control participants and 15 treatment participants, aged 18 to 30 years, all of whom reported insomnia, were analysed. Participants were assigned to one of two app conditions. Group 1 participants heard a countdown from 99 to 1, and Group 2 were prompted to visualise randomly selected brief scenes read by the app at eight-second intervals (SDI).

Method and Materials

Findings

Discussion

Our findings reveal that using sleep apps in collaboration with sleep diaries could lead to greater certainty that sleep diary data have been accurately reported, especially if participants are aware that their mobile usage patterns will be tracked and compared against diary data. From a research methodology and administration perspective, this would entail working closely with the mobile app development team to ensure that timely identification of non-compliance with study procedures can be made.

Although sleep app data have the potential to contribute to our understanding of mobile use in naturalistic conditions, the unstructured form of the data collected will prove too costly to analyse for many researchers, unless more efficient means of data analysis can be developed.

Data science and data analytics can provide direction; however, the initial identification of typical and valid sequences of user actions must first be identified so that effective algorithms can be developed. These algorithms can then be used to create visualisations that researchers can use to identify patterns of interest in the data more efficiently and effectively. Another challenge will also be to ensure that algorithms do not limit insights to only those patterns identified prior to data collection, thereby limiting potential new insights that could potentially be gleaned.

Conclusion

Informing participants that app data will be triangulated against sleep diaries during data collection and analysis may ensure greater accuracy of participant-provided sleep diary data.

The development of an algorithm that can efficiently filter valid data usage patterns would facilitate data analysis and researchers’ experience. This would increase sleep app usability as a treatment and research tool. Developing a process for increasing efficiency in data analysis is necessary to exploit the advantages of large-scale data collection that a sleep app makes possible.

This study, although preliminary, provides insights on how to improve research practice in the context of mobile app usage in sleep research, in terms of using the app as both intervention and research tool. Insights in the conceptualisation of a visualisation feature that could be developed to provide more efficient identification of usage patterns has also been proposed.

References and conflict of interest

COI disclosure: Luc P. Beaudoin is a director of CogSci Apps Corp. and owner of CogZest. Both of these Canadian businesses develop products based on his affective / cognitive science. These include books, software, and training services.

Case studyUsing a qualitative approach involved the direct interpretation (Stake, 1995) of participants’ patterns of mobile app usage, based on actions recorded (i.e., press start, end, pause, resume, or cancel; time stamp; count of played items), and the reorganization of user actions into tables (visualisation; tabulation) to identify usage patterns. The researchers’ reflection notes on their respective experiences in analyzing the data were analysed as lived experiences that could inform more effective data analysis procedures when working with raw data from mobile apps, with the objective of deriving themes to inform improved research practice in this context. Thematic analysis of these experiences was also conducted to reveal exemplars of situations that researchers could face.

Selham, Z1,2., Guloy, S3., Bastien, C4., Beaudoin, L3., Carrier, J1,2

1Psychology department at Université de Montreal,2Hôpital du Sacré Cœur de Montréal, 3Simon Fraser University, Vancouver, CogSci Apps and CogZest (see disclosures below)4Dépt. de Psychologie, Université Laval

Research process and sleep app design lessons learned from the reflective examination of a sleep study

Raw data for all control (n=23) and intervention (n=31) participants prior to data cleaning

Figure 2. Raw data for all those who participated in the study, including those whose data were not used in the larger study’s analysis. 1907 total actions were logged. The initial step in data cleaning involved ensuring that only those participants who submitted sleep diaries are included in the analysis.

Figure 3. Step two in the analysis involved a line-by-line analysis of each action, taken by participants or recorded by the app, to identify patterns as part of the data cleaning process.

Figure 4. Only through time-intensive, line-by-line analysis could spurious data patterns be discerned.

Figure 5. Step 3 involved identifying emerging patterns from cleaned data, in terms of participants’ mobile app usage, through the creation of a spreadsheet, which provided a visual representation of the data. This step facilitated easy identification of participants who did not use the app as stated in their sleep diaries as well as of other patterns of usage (e.g., heavier app use throughout the night).

Figure 6. Conceptual mockup for an app usage visualisation feature to support researchers in identifying usage patterns and anomalies in user actions (green = start; red = end; orange= pause; yellow = cancel; blue= resume )

This feature could be used to identify more efficiently the usage patterns of participants through a visual representation of the collected data, making use of data analytics to screen out spurious data patterns. This mockup reflects similar data patterns to that identified in Figure 5.

Allocca, G., Ma, S., Martelli, D., Cerri, M., Del Vecchio, F., Bastianini, S., ... & Blackburn, S. (2019). Validation of' Somnivore', a Machine Learning Algorithm for Automated Scoring and Analysis of Polysomnography Data. Frontiers in Neuroscience, 13, 207.

Morin, C. M., Drake, C. L., Harvey, A. G., Krystal, A. D., Manber, R., Riemann, D., & Spiegelhalder, K. (2015). Insomnia disorder. Nature Reviews Disease Primers, 1, 15026.

Selham, Z., Guloy, S., Carrier, J., & Beaudoin, L. (2018). Effects of using a mental imagery mobile app on the subjective quality of sleep: preliminary findings. Poster presented at the 29th International Congress of Applied Psychology: "Psychology: Connecting Science to Solutions". Montreal, QC.

Stake, R. E. (1995). The art of case study research. Sage.

Introduction

Figure 1. SomnoTest mobile app

Four themes:

1) Unreliability of sleep diaries when triangulated against SomnoTest data, given that 9 participants had not used the app as claimed;

2) Complex, intensive qualitative analysis is needed to identify valid data in an unstructured data set;

3) Importance of visualisation when examining data to uncover patterns;

4) Identification of “fans” who continue to use the app when not required during the study and after their participation in the study.

Our findings reveal that data cleaning involves intensive line-by-line and case-by-case analysis of participant data, which proved challenging with 34 participants and would prove prohibitive for larger scale studies.

Acknowledgments: Drs. Guloy and Beaudoin would like to thank Dr. Phil Winne for introducing us, many years ago, to the importance of software logging and analysis for scientific research.

Ask developer