virtual scientific communities for innovation

Virtual Scientific Communities for

Innovation

Virtual Scientific Communities for

Innovation

Karl LieberherrNortheastern University

College of Computer and Information ScienceBoston, MA

joint work with Ahmed Abdelmeged and Bryan Chadwick

Karl LieberherrNortheastern University

College of Computer and Information ScienceBoston, MA

joint work with Ahmed Abdelmeged and Bryan Chadwick

Supported by Novartis and GMO

Introduction

• Problems to be solved:– Optimal assembly of a system from components

• hardware• software

–Maximum constraint satisfaction problem (MAX-CSP)

– Transporting goods minimizing energy consumption

– Schedule tasks minimizing cost

04/21/23 Innovation 2

Introduction• Solve optimization problems in a domain X (X-

problems). – Find a feasible solution of good quality efficiently.

• Scholars to play the Specker Challenge Game (X) [SCG(X)]. Repeat a few times.

• Within the group of participating scholars, the winning scholar has the– best solver for X-problems – best supported knowledge about X

04/21/23 3Innovation

Introduction

• Players share hypotheses about "the approximability of problems in certain niches of the problem domain"

• Administrator reconciles inconsistencies between shared hypotheses => Condensing knowledge/stirring progress

• Player with the strongest correct hypothesis gains reputation, the other player receives targeted feedback / gains knowledge

Introduction

• The game is designed to exclude situations where it is impossible to give useful targeted feedback and/or it's possible to gain reputation without sharing the strongest correct hypothesis, e.g.: proposing strong obvious hypothesis, avoiding involvement with other players, mirroring, ... etc => Fair assessment

Benefits of SCG

• Social Welfare – Supported knowledge

• Hypotheses are challenged and strengthened.• Better supported knowledge comes from better

algorithms and software.04/21/23 6Innovation

SCG(X)

ScholarsDesign Problem Solver

Develop SoftwareDeliver Agent

Agent Alice Agent Bob

Administrator SCG police

I am the best

No!!

Let’s play constructive

ly04/21/23 7Innovation

ScholarAlice

ScholarBob

SCG


no automationhuman plays

full automationagent plays

degree of automation used by scholar

our focus

some automationhuman plays

0 1

more applications:test constructive knowledge

transfer to reliable, efficient software

agentrobot BobAlice

Social Engineering• Why develop problem solving software

through a virtual scientific community?– Evaluates fairly, frequently, constructively and

dynamically. Encourages retrieval of state-of-the-art know-how, integration and discovery.

– Challenges humans, drives innovation, both competitive and collaborative.

– Agents point humans to what needs attention in problem solution / software.


Software Development

• Software developers are knowledge integrators: Requirements, contextual information (lectures, papers), behavior of program in competition, etc.


Scholars and virtual Scholars!• Are encouraged to

1. offer results that are not easily improved.

2. offer results that they can successfully support.

3. strengthen results, if possible.

4. publish results of an experimental nature with an appropriate confidence level.

5. stay active and publish new results or oppose current results.

6. be well-rounded: solve posed problems and pose difficult problems for others.

7. become famous!


Agent Design• How to Design an artificial organism?– needs introspection to give it an ego.– has a basic need: maximize reputation.– has a rhythm: every round the same activity.– interacts with other agents by proposing and

opposing hypotheses.• makes agent vulnerable.


competitive / collaborative


Agent Alice: claims hypothesis H

Agent Bob: challenges H, discounts: providesevidence for !H

loses reputation r wins knowledge k

wins reputation rmakes public knowledge k

Definitions• A hypothesis H offered by Alice is constructively

defendable by Alice against Bob if Alice supports H when Bob challenges H.

• The constructive defense is determined by an interactive protocol between Alice and Bob.

• A hypothesis H1 is stronger than hypothesis H2 if H1 implies H2.

• Successfully opposing is a form of proposing: strengthening a hypothesis means to propose a new one. Discounting a hypothesis means to propose its complement.


SCG is sound

• The SCG game is sound, i.e., agent Alice wins with proposed hypothesis H against opponent Bob iff– H is stronger than what Bob could constructively

defend and – H is constructively defendable by Alice against

Bob.


GIGO: Garbage in / Garbage out

• If all agents are weak, no useful solver created.

• WEAK against STRONG:– STRONG accepts a hypothesis that is not

discountable but WEAK cannot support it. Correct knowledge might be discounted.

– STRONG strengthens a hypothesis too much that it becomes discountable, but WEAK cannot discount it. Incorrect knowledge might be supported.


What is the purpose of SCG?• The purpose of playing an SCG(X) contest is to assess the

"skills" of the players in: – "approximating" optimization problems in domain X, – "figuring-out" the wall-clock-time-bounded approximability of

niches in domain X, – "figuring-out" hardest problems in a specific niche, and – "being-aware" of the niches in which their own solution

algorithm works best. • This multi-faceted evaluation makes SCG(X) more superior

to contests based on benchmarks that only test the player's skills in approximating optimization problems. During the game, players cross-test each others' skills.


How to use SCG(X)• ABB needs new ideas about how to solve

optimization problems in domain X.• Define hypothesis language for X– X-problems– hypotheses, includes protocol

• Submit hypothesis language definition to SCG server.


How to use SCG(X)• Offer prize money for winner with conditions,

e.g., performance must be at least 10% higher as performance of agent XY that ABB provides.

• 10 teams from 6 countries sign up, committing to 6 competitions. Player executables become known to other players after each competition. One team from ABB.

• The SCG server sends them the basic agent and the administrator for testing.


How to use SCG(X)

• Game histories known to all. Data mining!• First competition is at 23.59 on day 1.

Registration starts at 18.00 on same day. The competition lasts 2.5 hours.

• Repeat on days 7, 14, … 42.• The final winner is: Team Mumbai, winning

10000 Euro. Delivers source code and design document describing winning algorithm to ABB.


Benefits for ABB of using SCG(X)

• Teams perform know-how retrieval and integration and maybe some research. – Participating teams try to find the best knowledge in

the area.– Hypothesis language gives control!

• The non-discounted hypotheses give hints about new X-specific knowledge.

• A well-tested solver for X-problems that integrates the current algorithmic knowledge in field X.


Disadvantages of SCG

• The game is addictive. After Bob having spent 4 hours to fix his agent and still losing against Alice, Bob really wants to know why!

• Overhead to learn to define and participate in competitions.

• The administrator for SCG(X) must perfectly supervise the game. Includes checking the legality of X-problems.– if admin does not, cheap play is possible.– watching over the admin.


I am perfect

How to compensatefor disadvantages

• Warn the scholars.• Use a gentleman’s security policy: report

administrator problems, don’t exploit them to win.

• Occasionally have a non-counting “attack the administrator” competition to find vulnerabilities in administrator.– both generic as well as X-specific vulnerabilities.


Experience block


Experience• Used for 3 years in undergraduate Software

Development course. Prerequisites: 2 semesters of Introductory Programming, Object-Oriented Design, Discrete Structures, Theory of Computation.– Collect and integrate knowledge from prerequisite

courses, lectures, and literature. – Teach it to the agent.


Experience MAX-CSP

• MAX-CSP Problem Decompositions• T-Ball (one relation), Softball (several

relations, one implication tree), Baseball (several relations).

• ALL, SECRET


Stages for SECRET T-Ball

• MAXCUT – R(x,y)= x!=y– fair coin ½ – maximally biased coin ½ – semi-definite programming / eigenvalue

minimization 0.878


Stages for SECRET T-Ball

• One-in-three– R(x,y,z) = (x+y+z=1)– fair coin: 0.375– optimally biased coin: 0.444


Stages for ALL Baseball

• Propose/Oppose/Provide/Solve – based on fair coin– optimally biased coin

• correctly optimize polynomials

– correctly eliminate noise relations– correctly implement weights– …


The SCG(X) GameThe SCG(X) Game

How to model a scholar?

• Solve problems.• Provide hard problems.• Propose hypotheses about Solve and Provide

(Introspection).• Oppose hypotheses.– Strengthen hypotheses.– Challenge hypotheses.

• Supported challenge failed.• Discounted challenge succeeded.


How to model a hypothesis

• A problem space.• A discounting predicate on the problem

space.• A protocol to set the predicate through

alternating “moves” (decisions) by Alice and Bob. If the predicate becomes true, Alice wins.


How to model a hypothesis

• Proposing and challenging a hypotheses is risky: your opponent has much freedom to choose its decisions within the game rules.

• Alternating quantifiers.• Replace “exists” by agent algorithm kept by

administrator.


Hypothesis

• Alice’ Hypothesis: There exists a problem P in niche N of X s.t. for all solutions SBob searched by the opponent Bob in T seconds. Quality(P, SBob) < AR * Quality(P, SAlice).

• Hypotheses have an associated confidence [0,1].

• Hypothesis: <N, AR, Confidence>.

SQ = Quality(P, SAlice)


1in3 niche

• Only relation 1in3 is used.• 1in3 problem P:

v1 v2 v3 v4 v51in3( v1 v2 v3)1in3( v2 v4 v5)1in3( v1 v3 v4)1in3( v3 v4 v5)secret 1 0 0 1 0

Truth Table 1in3

000 0001 1010 1011 0100 1101 0110 0111 0

Secret quality SQ = 3/4


1in3 Hypothesis• 1in3 hypothesis H proposed by Alice: exists P in

1in3 niche so that for all SBob that opponent Bob searches in time t (small constant) seconds: Quality(P,SBob) < 0.4 * Quality(P,SAlice).

• H = (niche = (1in3), AR =0.4, confidence = 0.8)• Bob has clever knowledge that Alice does not

have. He opposes the hypothesis H by challenging it using his randomized algorithm.


Bob’s clever knowledge4/9 for 1in3

• 4/9 for 1in3: For all P in 1in3 niche, exists S so that Quality(P,S) >= 0.444 * SQ.

• Proof: la(p)=3*p*(1-p)2 has the maximum 4/9. • argmax p in [0,1] la(p) = 1/3.• Without search, in PTIME.• Derandomize• Bob successfully discounts• Alice gets a hint – Was Bob just lucky?

Truth Table 1in3000 0001 1010 1011 0100 1101 0110 0111 0


End


Reputation Gain

• Hypothesis have credibility [0,∞]. The credibility of a hypothesis is proportional to agent’s confidence in the hypothesis and agent’s reputation.

• Reputation gain is proportional to the discounting factor and the hypothesis credibility.

• The discounting factor [-1,1]. 1 means the hypothesis is completely discounted.


AR is too AR is too lowlow AR is too AR is too highhigh

exists P for all S exists P for all S that opponent that opponent searches: searches: Quality(P,S) < Quality(P,S) < AR * SQAR * SQ

Quality(P,S’) - AR * SQ

strengthens: AR - AR’.

Discounting Factor

Discounting Factor

• H1 = ((1in3), AR = 1.0, confidence = 1.0)

• H1 proposed by Alice: exists P in 1in3 niche so that for all S that opponent Bob searches: Quality(P,S) < 1.0 * SQ.

• This is a reasonable hypothesis if Alice is sure that her secret assignment is the maximum assignment when she provides a sufficiently big problem to Bob.

What we did not tell you so far

• A game defines some configuration constants:

• a maximum problem size

• For example, all problems in the niche can have at most 1 million constraints.

• A maximum time bound for all tasks (propose, oppose, provide, solve), e.g. 60 seconds.

• An initial reputation, e.g., 100. When reputation becomes negative, agent has lost.

Discounting Factor: ReputationGain for

Strengthening

• H1 = ((1in3), AR = 1.0, confidence = 1.0)

• H1 proposed by Alice: exists P in 1in3 niche so that for all S that opponent Bob searches: Quality(P,S) < 1.0 * SQ.

• Bob thinks he can strengthen H1 to H2 = (MAXCSP, niche = secret ExistsForAll (1in3), AR = 0.9, confidence = 1.0).

• DiscountingFactor 1.0-0.9 = 0.1.

• ReputationGain for Bob = 0.1 * 1.0 * AliceReputation.

• Alice gets her reputation back if she discounts H2.

Discounting FactorReputationGain for

Discounting• H = ((1in3), AR = 0.4, confidence = 1.0)

• H proposed by Alice: exists P in 1in3 niche so that for all S that opponent Bob searches: Quality(P,S) < 0.4 * SQ.

• Bob knows he can discount H based on this knowledge: 4/9 for 1in3.

• Let’s assume he achieves 0.45 on Alice’ problem.

• DiscountingFactor 0.45 – 0.4 = 0.05 .

• ReputationGain for Bob = 0.05*1.0*AliceReputation.

Discounting FactorReputationGain for

Supporting• H = ((1in3), AR = 0.4, confidence = 1.0)

• H proposed by Alice: exists P in 1in3 niche so that for all S that opponent Bob searches: Quality(P,S) < 0.4 * SQ.

• Bob knows he can discount H based on this knowledge: 4/9 for 1in3.

• Let’s assume he achieves 0.3 on Alice’ problem. Bob has a bug somewhere!

• DiscountingFactor 0.3 – 0.4 = -0.1

• ReputationLoss for Bob = -0.1*1.0*AliceReputation.

Mechanism Design

• The exact SCG(X) mechanism is still a work in progress.

• SCG(X) mechanism must be sound:– Encourage productive behavior and discourage

unproductive behavior of scientists.– The agent with best heuristics wins.


Tools to facilitate use of SCG(X)

• Definition of X.

• Generate a client-server infrastructure for playing SCG(X) on the web.

• Administrator enforces SCG(X) rules: client.

• Baby agents: servers. They can communicate and play an uninteresting game.

• Baby agents get improved by their caregivers, register with Administrator and the game begins at midnight.


SCIENTIFIC COMMUNITYSCIENTIFIC

COMMUNITY

Scholars and virtual Scholars!• Are encouraged to

– offer results that are not easily improved.

– offer results that they can successfully support.

– quote related work and show how they improve on previous work.

– publish results of an experimental nature with an appropriate confidence level.

– stay active and publish new results or oppose current results.

– be well-rounded: solve posed problems and pose difficult problems for others.

– become famous!


Productive Scientific Behavior (1)

• The agents propose hypotheses that are difficult to strengthen or challenge (i.e. non-trivial yet correct). Otherwise, they lose reputation to their opponents.

• Offer results that cannot be easily improved.• Offer results that they can successfully

support.


Productive Scientific Behavior (2)• Agents are encouraged to propose hypotheses

they are not sure about. But they need to fairly express their confidence in their hypotheses.– If the confidence is inappropriately high, they lose too

much reputation if the hypothesis is successfully discounted.

– If the confidence is inappropriately low, they don’t win enough reputation if the hypothesis is successfully supported.

• publish results of an experimental nature with an appropriate confidence level.



• Agents stay active. In each “round”, they must propose new hypotheses and oppose other agents hypotheses.

• stay active and publish new hypotheses or oppose current hypotheses.

• Agents maximize their reputation.• become famous!



• When Alice loses reputation to Bob, Alice can learn from Bob:– Alice has a bug in her software.– Bob has skills superior to hers. Alice should try to

acquire Bob’s skills.

• Learn from mistakes.• Be careful how you oppose a Nobel

Laureate. The risks are high.


Unproductive Scientific Behavior

• Cheating is forbidden: you can only succeed through good scientific behavior (by adding useful hypotheses or by successfully opposing hypotheses in the knowledge base).


Fair Scientific Community

• All agents start with the same initial reputation.

• The winner has the best skills in domain X within the set of participating agents.


ApplicationsApplications

Improving the research approach

• Problem to be solved: Develop the best practical algorithms for solving NPO X.

• Standard solution: Write hundreds of papers on the topic with isolated implementations. What are the best practical algorithms?

• Our solution: Use the virtual scientific agent community SCG(X) with a suitably designed hypotheses language to compare the algorithms. The winning agent has the best practical algorithms.


Teaching: Survival Skills in SCG(X)

• Needed when agent caregiver is human.

• Knowledge about domain X needs to be developed by students or taught to them and understood and put into algorithms (propose-oppose(strengthen-challenge)-provide-solve) that go into the agent.

• This tests both whether the knowledge about X is understood as well as the programming skills.


Teaching: Survival Skills in SCG(X) [cont.]

• [Scientific Innovation in X] Agents get skills programmed into them by clever scientists in domain X. Scientists use data mining to learn from competitions and manually improve the agents.

• [Machine Learning Innovation in X] Agents get skills programmed into them by an agent caregiver programmed with learning skills and data mining skills for domain X. Agent gets updated automatically between competitions and they improve automatically.


Possible Application Domain For DM/ML/AI

Possible Application Domain For DM/ML/AI

SCG(X) produces history

• Proposer’s reputation: 120• Hypothesis10 proposer1 opposer2

confidence 1• Problem delivered• Solution found: discountFactor = 1• Opposer: increase in reputation: 1 * 1 * 120

= 120


Blame assignment

–Where is the proposer to blame?

– Bad hypothesis that is discountable.

– Bug in problem finding algorithm.

– Bug in problem solving algorithm used to check proposed hypothesis.


Creating Agents• An agent is composed of 6 components:

Agent = <Prop, Opp, Str, Cha, Prov, Sol>.• Components can refer to each other.• Given a set of agents: Agent1 ... Agentn

• Composed agent is a 12-tuple: <PropI, PropO, OppI, OppO, StrI, StrO, ChaI, ChaO, ProvI, ProvO, SolI, SolO>.

• <Prop3, (01101),Opp4, (00000), …>

Propose, Oppose, Strengthen, Challenge, Provide,Solve

1=own0=other


Creating Agents [cont.]

• PropI, OppI, StrI, ChaI, ProvI, SolI ∈ [1..n].• PropO consist of 5-bits, each denote one of

the other components. The first bit describes whether to use the opposition component of agent PropI or agent OppI.


Conclusions

• We have shown how a virtual scientific community of agents can foster the development and innovation of heuristics for approximating NPOs.

• We need your input on how DM and ML could help with evolving the agents.


Questions?Questions?

10/16/09 Can DM and ML help?

Discounting • If Alice offers the belief (FourColorConjecture, confidence = 1.0), she must be ready to support it.–The opponent Bob gives Alice a planar graph.–Alice must deliver a 4-coloring.• If she does not, Bob has successfully discounted Alice’ belief and Alice loses reputation and Bob gains.• If she does, Alice has successfully defended her belief and Alice wins reputation and the opponent Bob loses.

–Note that discounting is different from finding a counterexample. If Alice loses she has a “fault” in her coloring algorithm.

88

10/16/09 Can DM and ML help?

Beliefs: Four color conjecture

• FourColorConjecture: For all graphs g satisfying the predicate planar(g) there exists a 4-coloring of the nodes of g such that no two adjacent nodes have the same color.• ForAllExists belief: For all problems p satisfying predicate pred(p) there exists a solution s satisfying a property(p,s).

89

– Undiscounted beliefs represent the accumulated shared knowledge gained from the game. (Requires negation and reoffer of discounted beliefs?)


Improving the research approach

• Problem to be solved: Develop the best practical algorithms for solving NPO X.

• Standard solution: Write hundreds of papers on the topic with isolated implementations. What are the best practical algorithms?

• Our solution: Use the virtual scientific agent community SCG(X) with a suitably designed hypotheses language to compare the algorithms. The winning agent has the best practical algorithms.


virtual scientific communities for innovation

Documents

domain x xproblems

approximability of problems

posed problems

difficult problems

offer results

problem solution software

problem domainadministrator

virtual scholars