1. A reputation system to model expertise in online communities
Doctoral Consortium UMAP2011
+ some social mechanisms

2. From reputation to social mechanisms
Peer-based learning in online communities
Challenges
Motivation
Quality
Initial research direction: reputation

3. Reputationprinciples
Linkedwith trust
Reciprocity & Goodbehaviour
Someoneelses story about me
Linkedwithidentity; long-lived
A currency, a resource
Narrative, dynamic
Based on claims, transactions, opinion, rating, endorsements,
Based on indirect information
Context of community
Contextual
Trade-off between trust & privacy
Regular assessment of reputationquality
Windley, Phillip J., Kevin Tew, and Devlin Daley. A Framework for Building Reputation Systems.
In WWW2007.

4. Main challenge
What:
Aggregating and interpreting meaningfulinteractions between & among objects* in online communities
Why:
to give insight in the value on the object level based on user feedback and usage.
* Objects are people and information objects

5. So what did I do?
Literature
Learning theories, knowledge management
Trust and reputation
Look at different successful reputation systems/techniques
Google PageRank, eBay, StackOverflow, Guru, etc.
How?
value flow
context integration
sustainability
Hennis, T., Lukosch, S., & Veen, W. (2011). Reputation in peer-based learning environments. In O. C. Santos & J. G. Boticario (Eds.), Educational Recommender Systems and Technologies. IGI.

6. Value flow
Source objects
People, organizations,
Target objects
Blog posts, articles,
Claims (value statements)
Rates, links, recommendations, etc.
Implicit, explicit
(Farmer & Glass, 2009)

7. Context integration

8. Sustainability, i.e. StackOverflow

9.

10.

11. Concept reputation model

12. Concept reputation model 1/6claim weight
claim weight ~ expressiveness of the claim type
(i.e. rating versus click)

13. Concept reputation model 2/6target object weight
claim weight ~ expressiveness of the claim type (i.e. rating versus click)
target object weight ~ importance of the contribution type
(i.e. article versus comment)

14. Concept reputation model 3/6affiliated keyword weight
apples (0.8)
pears (0.2)
claim weight ~ expressiveness of the claim type (i.e. rating versus click)
target object weight ~ importance of the contribution type (i.e. article versus comment)
affiliated keyword weight ~ expressiveness of tag about the target object

15. Concept reputation model 4/6source object weight = authority
match keywords with reputation!
apples (64)
pears (33)
kiwis (0)
apples (0.6)
pears (0.2)
kiwis (0.2)
claim weight ~ expressiveness of the claim type (i.e. rating versus click)
target object weight ~ importance of the contribution type (i.e. article versus comment)
affiliated keyword weight ~ expressiveness of tag about the target object
source object weight = (source objects reputation for keyword) / (global rep. value for that keyword)
DIFFERENT WEIGHTS FOR DIFFERENT KEYWORDS

16. Concept reputation model 5/6claim value
apples (64)
pears (33)
kiwis (0)
apples (0.6)
pears (0.2)
kiwis (0.2)
claim weight ~ expressiveness of the claim type (i.e. rating versus click)
target object weight ~ importance of the contribution type (i.e. article versus comment)
affiliated keyword weight ~ expressiveness of tag about the target object
source object weight = (source objects reputation for keyword) / (global rep. value for that keyword)
claim value = rating (implicit/explicit), i.e. 4/5 stars

17. Concept reputation model 6/6claim value
apples (64)
pears (33)
kiwis (0)
apples (0.6)
pears (0.2)
kiwis (0.2)
claim weight ~ expressiveness of the claim type (i.e. rating versus click)
target object weight ~ importance of the contribution type (i.e. article versus comment)
affiliated keyword weight ~ expressiveness of tag about the target object
source object weight = (source objects reputation for keyword) / (global rep. value for that keyword)
claim value = rating (implicit/explicit)
3 claims (one for each affiliate keyword)

18. Why is this a useful approach?
Target object weight
Affiliate keyword weight
Claim for keyword k
Claim weight
Authority
Claim value (rating)

19. Why is this a useful approach?
Target object weight
Affiliate keyword weight
Account for:

• Several relevant context factors, such as authority
• 20. Extensible & configurable
• 21. including other weights and metrics (i.e. trust value, network centrality, etc.)
• 22. integrating formal ontologies
• 23. taking into account all relevant interactions
• 24. Starting point for the design of such a system
• 25. Rich profiles

Requirements

• Sufficient interactions & contributions
• 26. Rather large distributed online network or community

Claim for keyword k
Claim weight
Authority
Claim value (rating)

27. Reputation clouds (object model)

28. Peer Support Community
Trying to get funding (50k) for first prototype
Context
Blackboard apps & Tags
Comment & Rating functionality
Application of reputation model
Source: Teacher or Student
Target: comment, answer
Claims: like, page visit, follow
Reuse of reputation
Award, status, social comparison, gaming mechanisms (competition)

29. Contextualized support: content and user support

30. But
Not smart to bet on 1 horse, when 3 already dropped out of the race

31. So.. New scope (since 2 weeks)
Social mechanisms to design incentive structures to support informal learning in online communities
Hennis, T. A., & Kolfschoten, G. L. (2010). Understanding Social Mechanisms in Online Communities. In G. D. Vreede (Ed.), Group Decision and Negotiation 2010. Delft, the Netherlands.
Hennis, T. A., & Lukosch, H. (2011). Social Mechanisms to Motivate Learning with Remote Experiments - Design choices to foster online peer-based learning. CSEDU 2011.
Veen, W., Staalduinen, J.-P. V., & Hennis, T. A. (2010). Informal self-regulated learning in corporate organizations. In G. Dettori & D. Persico (Eds.), Fostering Self-regulated learning through ICTs. Genova, Italy: Institute for Educational Technologies Italys National Research Council.

32. Bouwman et al. (2007)
We argue that social software systems should trigger mechanisms that allow us to associate with or form social groups, whether online or in the real world.
Such mechanisms would acknowledge human motivations, like eagerness for exploration, curiosity, inquisitiveness, civilization, valuation of belonging, achieving self-realization, enjoying one-self.
Bouman, W., Hoogenboom, T., Jansen, R., Schoondorp, M., Bruin, B. de, & Huizing, A. (2007). The realm of sociality: notes on the design of social software. Amsterdam.

33. Research objectives
Designing incentives: which mechanism to apply when and how?
Design of processes
Supportive technologies
Focused on
informal learning
in organizations
during initial phase startup phase
Cases
Philips Lighting
Mediamatic (various communities)

34. Step 1 improve list of mechanisms (literature)
Matching objectives
Organizational objectives
User models
Fit / Embedding in practice
Rhythm
Leadership and roles
Heterogeneity & Diversity
Learning & Networking
Reputation & Identity
Reciprocity & Feedback
Common Ground & Privacy
Self-efficacy & Social comparison
Autonomy? Empowerment?
Curiosity & Provocation
IMPROVE

35. Step 2 Design & Evaluate
Philips Lighting
3 communities by December
Mediamatic Design team
Design & Test new things
Evaluate existing communities using
the Anymeta platform
Qualitative
Quantitative 50+ small-medium sized online communities
Blogging communities
Storytelling
Event communities & Professional networks

36. Rating & Reputation
Reciprocity & Feedback
Matching online and offline networks through RFID
Notifications & Activity

37. Profiling & Identity
Interaction types ~ Motivation?
Personalization & Recommendations

38. Keyword based
Recommend content, people, projects,

39.

40. Definitions
A social mechanism is a plausible hypothesis, or set of plausible hypotheses, that could be the explanation of some social phenomenon.
An incentive is any factor (financial or non-financial) that enables or motivates a particular course of action, or counts as a reason for preferring one choice to the alternatives.

41. Overall picture
Typicalvaluejudgements/claims
Context parameters
aggregateandinherit/inference
repute
value judgements:
use/rate/recommend
context parameters:
tag
contribute

42. 1: ContributionI write a blog post
Knowledge (topic)
What kind of contribution?
What kind of topic?
contributions
Competencies (process)
What kind of action?
Which competencies involved?

43. 2: Value + context (cont.)people rate, comment, tag etc
value statement:

• evaluate value use/rate/recommend
• 44. contextualize categorize/tag/embed