USER-DRIVEN TECHNOLOGY EVALUATION OF EPARTICIPATION SYSTEMS

Sotirios Koussouris, Loukas Kipenis, George Gionis, Fenareti Lampathaki, Yannis Charalabidis, Dimitrios Askounis

Decision Support Systems LaboratoryNational Technical University of Athens

Greece

WEB.DEP Project

WEB.DEP – Western Balkans Democratic Participation

FP6 Funded ProjectDuration: 1/1/2007 – 31/3/2009

http://www.web-dep.eu

7 Partners including:• 2 universities (NTUA, NAPIER)• 1 vendor/technology provider (ATC)• 3 Balkan national news agencies (ATA, MIA and TANJUG)• 1 association of journalists – a union (ESIEMTH)

Why News Agencies?

As media organisations, news agencies act as intermediaries between Governments and Citizens and are regarded as “neutral news providers”

In this case, using conscious code of ethics to….

• Present the news “unedited” and “uncommented” (facts not opinions)

• Involve stakeholders (e.g. government and experts)

• Manage (moderate) forum

• New democratic context = big change for news agencies (relationship government & citizens)

WEB.DEP Outline 1/2

• Information provision –content management system to share news more widely and efficiently

• E-Participation via shared forum across 3 countries

• Plus polling/questionnaires

• Content and moderation provided by news agencies

WEB.DEP Outline 2/2

Evaluation Needs• eParticipation systems should

satisfy multiple needs in order to be considered successfull.

• Rely not only on technology but (most importantly) on their “participative“ character and the users‘ engagement.

• These needs can be separated in two major categories

– IT driven category

– “Institutional” or “Decision Making” driven category

Methodology Used• Two structured straight-forward questionnaires addressing

the actual users of the platform

• Aiming at capturing two of the most important dimensions of eDemocracy systems

– the user perception of the usability and

– the expected impact of the system to the user‘s life

• Resulting in recommendations or guidelines that will strongly bind the following aspects

– system usability (user friendliness and straightforward functionalities)

– expected impact from the use of the system (increasing citizens’ participation in the decision making progress and of making the public opinion heard and considered by the decision makers)

Technology Acceptance Model 1/2

• Extension of classical TAM with the introduction of a new construct is introduced, namely “External Factors”.

• The aim of this modification is to identify the level of impact of relevant external factors (prior experience, educational level and job/occupation relevance) to Perceived Usefulness, Perceived Ease of Use and Intention to Use.

• This addition is of outmost importance when dealing with systems deployed in converging regions, as issues like Digital Divide, low internet penetration and limited familiarization with technology become obstacles on the road towards the sustainable and beneficial implementation and operation of such a system.

Technology Acceptance Model 2/2

For each construct, a group of questions was asked regarding the:

• Perceived Usefulness of the systems

• Perceived Ease of Use of the system

• Intention to Use the system

• External Factors regarding – Relevant Skills

– Prior Experience

– Educational Level of the users.

Hypotheses drawn

Various hypotheses were drawn to be tested. Based on the outcomes improvement scenarios will be build for maximizing usability and impact of the system.

• H1: Prior experience with similar technological tools and/or active citizenship will have a direct positive effect on system perceived usefulness.

• H2: Prior experience with similar technological tools and/or active citizenship will have a direct positive effect on system perceived ease of use.

• H3: Educational level will have a direct positive effect on system perceived usefulness.

• H4: Educational level will have a direct positive effect on system perceived ease of use.

• H5: Job/Occupation Relevance will have a direct positive effect on system perceived usefulness.

• H6: Prior experience with similar technological tools and/or active citizenship will have a direct positive effect on intention to use the system.

• H7: Job/Occupation Relevance will have a direct positive effect on intention to use the system.

• H8: Educational level will have a direct positive effect on intention to use the system.

• H9: Job/Occupation Relevance will have a direct positive effect on system perceived ease of use.

Single Construct Analysis• Initial findings calculating the mean

values for each construct

• The educational level of the users lies on “Degree or higher training”

• Perceived Usefulness, Perceived Ease of Use and Intention to Use are quite positive (lying on “maybe yes” level)

• Users’ occupation relation to IT technology and decision-making process and prior experience to discussion systems are slightly above the mean value of 3 which corresponds to “Yes and No” level.

TAM Findings• Educational Level seems to have a strong positive effect to all main

constructs that represent the grade of acceptance for WEB.DEP. • There is a strong indication of positive effect of Occupation towards

Perceived Usefulness, Perceived Ease of Use and Intention to Use. • There is an indication of negative effect of Prior Experience to all 3 main

constructs, which constitute a really interesting finding.

Hypotheses drawnVarious hypotheses were drawn to be tested during the evaluation and based on the outcomes

improvement scenarios will be build for maximizing usability and impact of the system. • H1: Prior experience with similar technological tools and/or active citizenship will have a

direct positive effect on system perceived usefulness.• H2: Prior experience with similar technological tools and/or active citizenship will have a

direct positive effect on system perceived ease of use.• H3: Educational level will have a direct positive effect on system perceived usefulness. • H4: Educational level will have a direct positive effect on system perceived ease of use. • H5: Job/Occupation Relevance will have a direct positive effect on system perceived

usefulness.• H6: Prior experience with similar technological tools and/or active citizenship will have a

direct positive effect on intention to use the system.• H7: Job/Occupation Relevance will have a direct positive effect on intention to use the

system.• H8: Educational level will have a direct positive effect on intention to use the system.• H9: Job/Occupation Relevance will have a direct positive effect on system perceived ease of

use.

Conclusions – Next Steps

• Analyze and investigate the “negative” relations revealed

• The proposed TAM should be tested on other systems and redesigned accordingly in order to move towards a one-fits-all solution

• Reduce the extend of the questionnaire in order to encourage more responses from users

• Re-evaluation of the WEB.DEP platform, as it matures.