user perception of the affect component of the usability of psycinfo database alexei oulanov, phd,...

USER PERCEPTION OF THE AFFECT COMPONENT OF THE USABILITY OF PSYCINFO DATABASE

Alexei Oulanov, PhD, MBA, MSLISMedgar Evers College/ City University of New York (USA)

[email protected]

Abstract

The focus of the present research is on the Affect component of the Usability evaluation of the PsycInfo database. This Usability testing in this research is focused on the User Perception. Oulanov and Pajarillo (2001, 2002, 2003), Oulanov (2008a, 2008b), and El-Halees (2014) emphasize the importance of User Focus, evaluating the Usability of Lexis-Nexis, CUNY+, Business Source Premier, and the Website of the Central Bank of Russia. Shneiderman (1998) postulated that User Focus should be the cornerstone of the Usability research, including the research on Affect. The User Focus succeeded the Systems’ Focus that was focusing more on Information Systems and often disregarding the User Perception (Oulanov, 2008b).The recent trends in Usability research suggest that users should be able to use the systems intuitively (Oulanov, 2008b). The system with an advanced state of Usability should be easily perceived and internalized by the user, without exercising any significant effort. The help of an intermediary like a librarian or information specialist should not be necessary. Also, ideally, the user should not be checking the Help file very often. If the user feels comfortable using the system intuitively, then the Affect assessment should reflect this state. Classical Pretest-Posttest Control Group research design was utilized for the present study. Participants were asked to fill out the same survey twice, as a Pretest and Posttest. Before completing the Pretest they were asked to perform some tasks in order to familiarize themselves with the information system. The intervention was introduced in form of the online instructional videos on how to use the database. The Control Group had videos on the subject related to the subject of the tasks, which was a placebo, but seemed to be relevant. As a placebo it did not provide any instructions or training on how to use PsycInfo. The assessment of the Affect as a component of the Usability testing in the present study produced the results similar to the ones present in the earlier research (Oulanov and Pajarillo, 2001, 2002, 2003; Oulanov, 2008a, 2008b). The advantage of the present research included comparison of the User Perception before the intervention in form of the detailed instruction and after this intervention. The previous studies were simpler in their design. They only included cross-sectional one-time Usability analyses. Being more in-depth, the present study demonstrates that the detailed instruction on how to use the database efficiently does not result in users’ change of the perception of the system’s use over the original intuitive system’s use. Therefore, the system is perceived by the user as highly intuitive, and, hence, possesses the superb Usability. The present study brought the Usability Research on the new higher level. While previous research in the field normally did not go beyond one simple cross-section of Usability (Oulanov and Pajarillo 2001, 2002, 2003; Oulanov, 2008a, 2008b), the present study elevated the Usability testing on a new level by assessing the possible differences or absence thereof between User Perception of the information system’ intuitive use and by the use resulted from the training, which would give a new perspective on the Usability of the system. This study showed that in case of the PsycInfo, the User Perception of the Usability does not change with the attempted change in the expertise of the user. This indicates that the Usability of PsycInfo on the EBSCO platform is of such a superior level and the system is designed in such a way that it can be used intuitively by a regular user, not requiring any additional investment in the training and developing of any additional expertise.

2 WORKING HYPOTHESIS

H1: Intervention in form of the instruction on how to use the information system does not create a difference in user perception of the Affect of the information system between the pretest and posttest usability assessments.

3 METHODS

Classical Pretest-Posttest Control Group research design was utilized for the present study. Participants were asked to fill out the same survey twice, as a Pretest and Posttest. Before completing the Pretest they were asked to perform some tasks in order to familiarize themselves with the information system. The intervention was introduced in form of the online instructional videos on how to use the database. The Control Group had videos on the subject related to the subject of the tasks, which was a placebo, but seemed to be relevant. As a placebo it did not provide any instructions or training on how to use PsycInfo.

4 PARTICIPANTS

Present researcher’s classmates in a graduate psychology program were asked to participate in the current study. This constituted convenience sampling procedure. The 8 participants were randomly assigned to 2 groups of 4 people each: Experimental and Control group. The participants did not know to which group they belonged. The tasks and surveys were designed in a way that it would not be possible for them to distinguish between the intervention and non-intervention.

4.1 Demographics

4.1.1 Age Some 2 participants in the Experimental Group (EG) fall into the age category of 18-24.

The other 2 participants in the group are of age 25-34. In the Control Group (CG) we have somewhat older people, 50% belongs to 25-34 category and another 50% to 35-44.

4.1.2 Gender

There are 3 males in the EG and 1 female. All 4 participants in CG are female. 4.1.3 Education

In terms of Education all 4 participants in the EG indicated that they received Bachelor’s degree. The level of education in this group is homogeneous. Unlike in EG, in CG there is a diversity in terms of Education. 1 participant is currently in college, 1 received Bachelor’s degree, and 2 received Master’s degrees.

4.1.4 Computer Experience EG is homogeneous – 11-15 years. CG is more diverse: 1 participant – 6-10, 2

participants -15-20, and 1 participant – 21 & more.

4.1 Demographics

4.1.5 Internet Searching Experience

Frequency Percent Valid Percent Cumulative

Percent Valid 2 1 25.0 25.0 25.0

3 3 75.0 75.0 100.0 Total 4 100.0 100.0

Fig. 1. Internet Searching Experience - 1 EG includes 3 participants that have 11-15 years of Internet Searching and 1 participant with 6 to 10 years of experience. This appears to be somewhat uniform.


Percent Valid 2 1 25.0 25.0 25.0

3 1 25.0 25.0 50.0 4 2 50.0 50.0 100.0 Total 4 100.0 100.0

Fig. 2. Internet Searching Experience - 2 CG is more dispersed: 1 person – 6-9, 1 person – 10-15, and 2 people – 16-20 years of experience.

4.1 Demographics

4.1.6 Conducting research in academic libraries


Percent Valid 1 1 25.0 25.0 25.0

2 3 75.0 75.0 100.0 Total 4 100.0 100.0

Fig. 3. Conducting research in academic libraries - 1 EG shows relative uniformity in this category again. 3 participants – 11-15, and 1 participant – 1-5.


Percent Valid 2 1 25.0 25.0 25.0

3 2 50.0 50.0 75.0 4 1 25.0 25.0 100.0 Total 4 100.0 100.0

Fig. 4. Conducting research in academic libraries - 2 CG is again diverse. 1 person – 6-10, 2 people – 11-15, and 1 person – 16-20.

4.1 Demographics

4.1.7 Experience with EBSCO databases


Percent Valid 1 3 75.0 75.0 75.0

2 1 25.0 25.0 100.0 Total 4 100.0 100.0

Fig. 5. Experience with EBSCO databases - 1 EG has some uniformity again. 3 people – 1-5, 1 person – 6-10.


Percent Valid 1 1 25.0 25.0 25.0

2 2 50.0 50.0 75.0 3 1 25.0 25.0 100.0 Total 4 100.0 100.0

Fig. 6. Experience with EBSCO databases - 2 CG is diverse with some 2 participants reporting 6-10.

4.1 Demographics

4.1.8 Searching PsycInfo on EBSCO platform


Percent Valid 1 3 75.0 75.0 75.0

2 1 25.0 25.0 100.0 Total 4 100.0 100.0

Fig. 7. Searching PsycInfo on EBSCO platform - 1 EG is pretty much uniform with the majority searching PsycInfo for 1-5 years, and 1 person – 6-10.


Percent Valid 1 1 25.0 25.0 25.0

2 2 50.0 50.0 75.0 3 1 25.0 25.0 100.0 Total 4 100.0 100.0

Fig. 8. Searching PsycInfo on EBSCO platform - 2 CG has diversity with 2 participants 6-10, 1 – 1 to 5, and 1 – 11 to 15.

The common trend in the demographic tables appears to be that EG is more homogeneous in most of the categories, and the CG has a more diverse body of participants.

5 MEASURES

The measure used for the present research was Oulanov’s Usability instrument. This instrument was utilized in some previous research studies (Oulanov, 2008a; 2008b). The Affect component of this measure is especially relevant to the present research. This Affect component includes the following items: “I look forward to using this information system again when I need to look up something”, and “I generally have positive feeling when conducting a search using this information system”. The users had to respond to these items using a 5-point Interval Likert Scale containing the following anchors: Strongly Disagree, Disagree, Undecided, Agree, and Strongly Agree. When coded, Strongly Disagree was assigned the value of 1, and Strongly Agree the value of 5. The Cronbach Alpha for the Pretest was relatively high (.930), and for the Posttest was a little lower than the norm (.627).

6 RESULTS

The Working Hypothesis for the present research stated that intervention in form of the instruction on how to use the information system does not create a difference in user perception of the Affect of the information system between the pretest and posttest Usability assessments.

The statistical tests utilized in this study include Mixed ANOVA and Paired-Samples T-Test. Both of these tests showed the results that are not statistically significant, supporting the Working Hypothesis.

6 RESULTS

6.1 Frequency Distributions


Percent Valid 1 1 25.0 25.0 25.0

3 3 75.0 75.0 100.0 Total 4 100.0 100.0

Fig. 9. Using the system again - 1 In EG 75% at the Pretest are undecided whether they want to use this information system again, and 25% would not want to use the system again.


Percent Valid 1 2 50.0 50.0 50.0

2 2 50.0 50.0 100.0 Total 4 100.0 100.0

Fig. 10. Using the system again - 2 In EG Posttest, now 50% are undecided and 50% do not want to use the system again. The ANOVA and T-test will show that these results are not statistically significant, but, interestingly, they contradict the Meer Exposure theory. Also, mental fatigue after the intervention might have increased some participants’ negativity. Again, the difference is not statistically significant, and the intervention in form of the detailed instructions on how to use the database did not make the results better. If the number of participants was larger, the present researcher could hypothesize that even the best instruction might mislead the user of the system with the superb usability, and his perception and performance might become worse than if he were using the system intuitively.

6 RESULTS


Percent Valid 2 1 25.0 25.0 25.0

3 1 25.0 25.0 50.0 4 2 50.0 50.0 100.0 Total 4 100.0 100.0

Fig. 11. Using the system again - 3

In CG 50% agree that they want to use this information system again.


Percent Valid 2 1 25.0 25.0 25.0

3 1 25.0 25.0 50.0 4 2 50.0 50.0 100.0 Total 4 100.0 100.0

Fig. 12. Using the system again - 4 In CG the results here are identical between the Pretest and Posttest in terms of the desire to use the system again. No possible Mere Exposure effect, no statistically significant difference, no difference at all.

6 RESULTS


Percent Valid 1 1 25.0 25.0 25.0

3 2 50.0 50.0 75.0 4 1 25.0 25.0 100.0 Total 4 100.0 100.0

Fig. 13. Positive feeling towards the system - 1 In EG 50% are undecided whether they have positive feeling when they search PsycInfo. 25% - strongly disagree, and another 25% - agree.


Percent Valid 1 2 50.0 50.0 50.0

3 2 50.0 50.0 100.0 Total 4 100.0 100.0

Fig. 14. Positive feeling towards the system - 2 In the EG posttest for this item the results deteriorate again. Now while 50% of the participants are undecided, another 50% do not have positive feelings towards the system. Again, the intervention in form of detailed instruction makes the perception of the database by the users worse than before the instruction. This might be showing again that the intuitive use of the system creates a much more positive feeling towards the system in the user, confirming the high Usability levels The results are not statistically significant, however, and this absence of the significance, again, shows that the intervention in form of the instruction does not create any difference in user perception of the database. Which means that the database can be used intuitively, and therefore has a high level of Usability.

6 RESULTS


Percent Valid 3 1 25.0 25.0 25.0

4 3 75.0 75.0 100.0 Total 4 100.0 100.0

Fig. 15. Positive feeling towards the system – 3 CG is more positive and “enlightened” than the EG again – 75% have positive feeling using the system, and 25% are undecided.


Percent Valid 4 4 100.0 100.0 100.0

Fig. 16. Positive feeling towards the system - 4

Now all 100% feel positive. This could be the slight Mere Exposure effect, which could be researched further in the future, possibly qualitatively, using Focus Group interviews. However, the results here are not significant, showing that there is no statistically significant change without the intervention either.

6 RESULTS

6.2 Inferential Statistical Testing 6.2.1 Tests of Within-Subjects Effects

Measure:MEASURE_1

Source Type III Sum of

Squares df Mean Square F Sig.

time Sphericity Assumed 2.250 1 2.250 2.000 .207

Greenhouse-Geisser 2.250 1.000 2.250 2.000 .207

Huynh-Feldt 2.250 1.000 2.250 2.000 .207

Lower-bound 2.250 1.000 2.250 2.000 .207

time * group Sphericity Assumed 4.000 1 4.000 3.556 .108

Greenhouse-Geisser 4.000 1.000 4.000 3.556 .108

Huynh-Feldt 4.000 1.000 4.000 3.556 .108

Lower-bound 4.000 1.000 4.000 3.556 .108

Error(time) Sphericity Assumed 6.750 6 1.125

Greenhouse-Geisser 6.750 6.000 1.125

Huynh-Feldt 6.750 6.000 1.125

Lower-bound 6.750 6.000 1.125

Fig. 17. Within-Subjects Effects

ANOVA’ Within Subject Effects does not show significant results, which supports the Working Hypothesis.

6 RESULTS

6.2.2 Tests of Between-Subjects Effects

Measure:MEASURE_1

Transformed Variable:Average

Source Type III Sum of

Squares df Mean Square F Sig.

Intercept 529.000 1 529.000 178.817 .000

group 30.250 1 30.250 10.225 .019

Error 17.750 6 2.958

Fig. 18. Between-Subjects Effects

On the other hand, Between-Subjects Effects do have highly significant results. This is logical, because the 2 groups had quite different perceptions of the information system. This factor triangulates the results, showing that perceptions of the different participants might differ between the groups, notwithstanding the fact that there was no significant difference between Pretest and Posttest within each group.

Paired Samples Testa

Paired Differences

t df Sig. (2-tailed) Mean

Std.

Deviatio

n

Std. Error

Mean

95% Confidence

Interval of the

Difference

Lower Upper

Pair 1 priAff

cttot -

poiAff

cttot

1.75000 2.06155 1.03078 -1.53039 5.03039 1.698 3 .188

a. group = 1.00

Fig. 19. The T-test for EG

The T-test for the EG does not show any significant difference, supporting the Working Hypothesis.

There was no statistical significance for the CG either.

The Working Hypothesis is substantiated!

7 DISCUSSION

The Hypothesis for the present research stated that intervention in form of the instruction on how to use the information system does not create a difference in user perception of the Affect of the information system between the pretest and posttest usability assessments. This hypothesis was supported.

The assessment of the Affect as a component of the Usability testing in the present study produced the results similar to the ones present in the earlier research (Oulanov and Pajarillo 2001, 2002, 2003; Oulanov, 2008a, 2008b). The advantage of the present research included comparison of the User Perception before the intervention in form of the detailed instruction and after this intervention. The previous studies were simpler in their design. They only included cross-sectional one-time Usability analyses. Being more in-depth, the present study demonstrates that the detailed instruction on how to use the database efficiently does not result in users’ change of the perception of the system’s use over the original intuitive system’s use. Therefore, the system is perceived by the user as highly intuitive, and, hence, possesses the superb Usability.

Limitations of the present research include the small number of participants, which, in the future will be addressed by using a few classes at the present researcher’s place of employment for conducting a similar study. This way the number of the participants will increase to a few dozens of people. Future research will also include participants performing a second set of tasks after the intervention, in order to have a better understanding of weather they perceive the system the same way after the intervention, and whether they really conduct the searches in the same way and on the same level of efficiency after the detailed instructions on how to use the information system.

7 DISCUSSION

The present study brought the Usability Research on the new higher level. While previous research in the field normally did not go beyond one simple cross-section of Usability (Oulanov and Pajarillo 2001, 2002, 2003; Oulanov, 2008a, 2008b), the present study elevated the Usability testing on a new level by assessing the possible differences or absence thereof between User Perception of the information system’s intuitive use followed by the use resulted from the training, which would give a new perspective on the Usability of the system. This study showed that in case of the PsychInfo, the User Perception of the Usability does not change with the attempted change in the expertise of the user. This indicates that the Usability of PsychInfo on the EBSCO platform is of such a superior level and the system is designed in such a way that it can be used intuitively by a regular user, not requiring any additional investment in the training and developing of any additional expertise.

8 REFERENCES

Brinkman, W. -., Haakma, R., & Bouwhuis, D. G. (2009). The theoretical foundation and validity of a component-based usability questionnaire. Behaviour & Information Technology, 28(2), 121-137. doi:10.1080/01449290701306510

Crisp, R. J., Hutter, R., & Young, B. (2009). When mere exposure leads to less liking: The incremental threat effect in intergroup contexts. British Journal of Psychology, 100, 133-149.

El-Halees, A. (2014). Software Usability Evaluation Using Opinion Mining. Journal Of Software (1796217X), 9(2), 343-349.

Häfner, M., & Stapel, D. A. (2009). Familiarity can increase (and decrease) stereotyping: Heuristic processing or enhanced knowledge usability? Social Cognition, 27(4), 615-622.

Oulanov, A. (2008a). Business administration students' perception of usability of the business source premier database: A case study. Electronic Library, 26(4), 505-519.

Oulanov, A. (2008b). Usability of a major financial internet portal in Russia : Implications for translingual efficacy. Saarbrucken: VDM Verlag Dr. Muller.

Oulanov, A., & Pajarillo, E. (2001). Usability evluation of the City University of New York CUNY+ database. Electronic Library, 19(2), 84-91.

Oulanov, A., & Pajarillo, E. (2002). CUNY + web: Usability study of the web-based GUI version of the bibliographic database of the City University of New York (CUNY). Electronic Library, 20(6), 481-487.

Oulanov, A., & Pajarillo, E. (2003). Academic librarians' perception of Lexis-Nexis. Electronic Library, 21(2), 123-129.

Shneiderman, B. (1998). Designing the user interface : Strategies for effective human-computer-interaction (3rd ed.). Reading, Mass: Addison Wesley Longman.

Yagi, Y., Ikoma, S., & Kikuchi, T. (2009). Attentional modulation of the mere exposure effect. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(6), 1403-1410. doi:10.1037/a0017396

user perception of the affect component of the usability of psycinfo database alexei oulanov, phd,...

Documents

usability research

present research

user perception oulanov

usability testing

earlier research oulanov

usability of lexisnexis

usability evaluation

superb usability