![Page 1: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/1.jpg)
NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning
Karl Lieberherrjoint work with
Ahmed Abdelmeged and Bryan Chadwick
04/22/23 1SCG Innovation
Supported by Novartis and GMO
![Page 2: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/2.jpg)
Introduction
• Scientific Community Game(X) [SCG(X)]– Goal: Foster innovation and learning in some
domain X• A virtual scientific community consists of
virtual scholars that propose and oppose hypotheses maximizing their reputations
• Applications: Learning and innovation through focused interaction, “Netflix in the small”
04/22/23 Innovation 2
![Page 3: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/3.jpg)
How to model a scholar?
• Solve problems• Provide hard problems• Propose hypotheses about Solve and Provide
(Introspection)• Oppose hypotheses– Strengthen hypotheses– Refute hypotheses • Supported opposing failed• Refuted opposing succeeded
04/22/23 3Innovation
![Page 4: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/4.jpg)
Where SCG comes from
• J ACM
04/22/23 SCG Innovation 4
Lieberherr/Specker 1981
![Page 5: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/5.jpg)
Outline• Introduction (done)• Highest safe rung example • SCG Scholar / Agent• SCG Agent in Action• Highest safe rung example (opposition)• Who is the winner?• Competition and collaboration• Disadvantages of SCG• Further Examples• SCG-based Software Development Process• Conclusions
04/22/23 SCG Innovation 5
![Page 6: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/6.jpg)
Example: Jar Stress Testing
• You have a ladder with r rungs, and you want to find the highest rung from which you can drop a copy of the jar and not have it break. We call this the highest safe rung problem (r,b).
• How many experiments do you need? Minimize.
• (r,infinity)• (r,1)
04/22/23 6SCG
![Page 7: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/7.jpg)
Highest Safe Rung ProblemProblems and Solutions
• Problems: p=((r,b),secret hsr), secret hsr in [0,r], r,b natural numbers• r = number of rungs• b = number of jars that are allowed to break• (r,b) is called a niche
• Solutions: sequence of queries of the form n? to find hsr. Responses: yes/no.
• Quality of solution: q = length of sequence of queries
04/22/23 7SCG Innovation
![Page 8: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/8.jpg)
Highest Safe Rung ProblemHypotheses
• Alice claims the hypothesis: I can solve any problem p=((r,b),secret hsr) with quality q: abbreviated H = ((r,b),q)
• Problems to be delivered for H = ((r,b),q) are of the form ((r,b), s). Important: A hypothesis defines a family of problems.
• Propose: Hypotheses H1 = ((25,2),11), H2 = ((25,2),6)
(from Kleinberg/Tardos)
04/22/23 8SCG Innovation
![Page 9: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/9.jpg)
Scholars propose and oppose
04/22/23 Innovation 9
HA1
HA2
HA3
HA4
egoisticAlice egoistic
Bob
Bob increases his reputation
HB1
HB2
opposes (1)
provides problem (2)
solves problemnot as well as she expected based on HA2 (3)
WINS!LOSES
proposed hypotheses
social welfare
Life of a scholar: (propose+ oppose+ provide* solve*)*
![Page 10: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/10.jpg)
What is the purpose of SCG?
• The purpose of playing an SCG(X) competition is to assess the "skills" of the agents in: – solving problems in domain X, – making good predictions about niches in domain
X, – finding the hardest problems in a specific niche
04/22/23 Innovation 10
![Page 11: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/11.jpg)
What is SCG(X)
04/22/23 Innovation 11
no automationhuman plays
full automationagent plays
degree of automation used by scholar
some automationhuman plays
0 1
more applications:test constructive knowledge
transfer to reliable, efficient software
agent Bobagent Alice
![Page 12: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/12.jpg)
What is SCG(X)?
TeamsDesign Problem Solver
Develop SoftwareDeliver Agent
Agent Alice Agent Bob
Administrator SCG police
I am the best
No!!
Let’s play constructive
ly04/22/23 12Innovation
TeamAlice
TeamBob
![Page 13: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/13.jpg)
For agents: Full Round Robin Tournaments or Swiss-Style
• Agents to play the SCG(X). Repeat a few times with feedback used to update agents.
• Within the group of participating agent, the winning agent has the– best solver for X-problems – best supported knowledge about X
04/22/23 13Innovation
![Page 14: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/14.jpg)
SCG in Action: Competitions
• http://www.ccs.neu.edu/home/lieber/courses/cs4500/f09/files/competitions/past_competitions/11_23/tournament_1/final_results_tournament_2009_11_24_12_03_41.html
• http://www.ccs.neu.edu/home/lieber/courses/cs4500/f09/files/competitions/past_competitions/10_22/tournament_1/final_results_tournament_2009_10_23_04_35_18.html
04/22/23 Innovation 14
![Page 15: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/15.jpg)
Highest Safe Rung Problemopposing
• opposing(refuting, strengthening)• Alice claims: Hypothesis ((25,2),5)– Bob opposes it by refuting it: Bob invents problem
((25,2), secret 22). Alice: 5? no, 10? yes, 6? no, 7? no, 8? no. Already 5 questions asked and answer still unknown. Alice’ claim is refuted.
• Alice claims: Hypothesis ((25,2),12)– Bob opposes it by strengthening it to ((25,2),9);
and he can successfully support this hypothesis
04/22/23 15SCG Innovation
![Page 16: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/16.jpg)
Highest Safe Rung Problemsupporting
• Alice claims: Hypothesis ((25,2),12)– Bob tries to discount but Alice supports it: Alice:
5? no, 10? no, 15? no, 20? no, 25? yes, 21? no, 22? no, 23? no, 24? yes. Only 9 questions asked and problem ((25,2), secret 23) is solved. Alice has supported her hypothesis.
04/22/23 SCG Innovation 16
![Page 17: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/17.jpg)
Who wins? Alice or Bob?
• Reputation of Alice = – the number of hypotheses that Alice proposed that
were never successfully opposed by Bob (neither refuted nor strengthened) +
– the number of hypotheses that Bob proposed that were successfully opposed by Alice
• RA = HAnotOpposedB + HBOpposedA• The scholar with the highest reputation wins• encourages: creating strong knowledge and
discounting knowledge created by others
04/22/23 17SCG Innovation
Motivated by real scientific community
![Page 18: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/18.jpg)
competitive / collaborative
04/22/23 Innovation 18
Agent Alice: claims hypothesis H
Agent Bob: opposes H, refutes: providesevidence for !H
Alice wins knowledge
Bob wins reputationmakes public knowledge
![Page 19: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/19.jpg)
Highest Safe Rung Problemcompetition / collaboration
• Alice claims: Hypothesis ((25,2),12)– Bob tries to discount but Alice supports it: Alice: 5?
no, 10? no, 15? no, 20? no, 25? yes, 21? no, 22? no, 23? no, 24? yes.
– From this exchange which is prompted by Alice defending her reputation, Bob gets an idea: For problem: p=((r,b),secret hsr), consider f(r,q) =(r/q + q) and find a q so that f(25,q) is minimized. f(25,5)=10; f(25,6)=11;f(25,4)=11.
– From this idea Bob knows that he can strengthen the hypothesis to ((25,2),10)
– General solution: Given r, find q to minimize (r/q + q).
04/22/23 SCG Innovation 19
![Page 20: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/20.jpg)
Scholars and Agents:Same rules
• Are encouraged to 1. offer results that are not easily improved.2. offer results that they can successfully support.3. strengthen results, if possible. 4. expose results that are wrong.5. stay active and publish new results.6. be well-rounded: solve posed problems and pose difficult
problems for others.7. become famous!
04/22/23 20Innovation
![Page 21: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/21.jpg)
Soundness Theorem
• SCG is sound: The agent with the best algorithms / knowledge wins (there is no way to cheat)– best: within the group of participating agents
04/22/23 Innovation 21
![Page 22: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/22.jpg)
Highest Safe Rung ProblemAsymptotic Hypotheses
• Alice claims the hypothesis: I can solve any problem p=((r,b),secret hsr) with quality f(r,b) : abbreviated H = ((r,b),f(r,b))
• Problems to be delivered for H = ((r,b),f(r,b)) are of the form ((r,b), secret hsr).
• Propose: Hypotheses H1 = ((r,b),(log(r))b), H2 = ((r,b),r1/b)
04/22/23 22SCG Innovation
![Page 23: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/23.jpg)
Highest Safe Rung Problemdiscounting asymptotic hypothesis
• discounting (refuting, strengthening)• Alice claims: Hypothesis ((r,b),(b*log(r)))– Bob discounts it by refuting it: Bob invents
problem ((1024,2), secret hsr). log(1024) = 10. 20 questions are not enough! Alice: 30? no, 60? yes, 31? no, 32? no, etc.. Already 20 questions asked and answer still unknown. Alice’ claim is refuted.
• Alice claims: Hypothesis ((r,2),r/2)– Bob discounts it by strengthening it to ((r,2),2*r½ );
and he can successfully support this hypothesis.
04/22/23 23SCG Innovation
![Page 24: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/24.jpg)
Disadvantages of SCG
• The game is addictive. After Bob having spent 4 hours to fix his agent and still losing against Alice, Bob really wants to know why!
• Overhead to learn to define and participate in competitions.
• The administrator for SCG(X) must perfectly supervise the game. Includes checking the legality of X-problems.– if admin does not, cheap play– watching over the admin
04/22/23 24Innovation
![Page 25: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/25.jpg)
How to compensatefor those disadvantages
• Warn the scholars.• Use a gentleman’s security policy: report
administrator problems, don’t exploit them to win.
• Occasionally have a non-counting “attack the administrator” competition to find vulnerabilities in administrator.– both generic as well as X-specific vulnerabilities.
04/22/23 25Innovation
![Page 26: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/26.jpg)
GIGO: Garbage in / Garbage out
• If all agents are weak, no useful solver created.
04/22/23 Innovation 26
![Page 27: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/27.jpg)
Physics Maximum Height ProblemProblems and Solutions
• Problems: p=(v, a), v, a: positive real numbers• The maximum height obtained by a projectile
launched with speed v at angle a to the horizontal is z.
• Solutions: real number z.• Quality of solution: Number of correct decimal
places.
04/22/23 27SCG Innovation
![Page 28: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/28.jpg)
Physics Maximum Height Problem Hypotheses
• Alice claims the hypothesis: I can solve any maximum height problem p=(v,a) with quality q in 1 minute: abbreviated H = (MHP,q)
• Problems to be delivered for H = (MHP,q) are of the form (v,a).
• Propose: Hypotheses H1 = (MHP,3), H2 = (MHP,6)
04/22/23 28SCG Innovation
http://scienceworld.wolfram.com/physics/Height.html
![Page 29: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/29.jpg)
Physics Maximum Height Problem discounting
• discounting (refuting, strengthening)• Alice claims: Hypothesis (MHP,3)– Bob discounts it by refuting it: Bob invents
problem (25,60 degrees). Alice fails to solve the problem in 1 minute with 3 correct digits. Alice’ claim is refuted. Checking is done by experiment or trusted third party.
• Alice claims: Hypothesis (MHP,1)– Bob discounts it by strengthening it to (MHP,2);
and he can successfully support this hypothesis
04/22/23 29SCG Innovation
![Page 30: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/30.jpg)
RegExpToAutomata ProblemProblems and Solutions
• Problems: p=(r,n); r a regular expression of size n.• r = regular expression; a + b* a + a a a b*• n defines a niche of regular expressions
• Solutions: DFA d equivalent to r.• Quality of solution: Number of states of d.
04/22/23 30SCG Innovation
![Page 31: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/31.jpg)
RegExpToAutomata ProblemProblems and Solutions
• Problems: p=(r,n); r a regular expression of size n.• r = regular expression; a + b* a + a a a b*• n defines a niche of regular expressions
• Solutions: DFA d equivalent to r.• Quality of solution: Number of states of d.
04/22/23 31SCG Innovation
![Page 32: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/32.jpg)
RegExpToAutomata Problem Hypotheses
• Alice claims the hypothesis: I can solve any problem p=(r,n) with quality q or less: abbreviated H = (n,q)
• Problems to be delivered for H = (n,q) are of the form p=(r,n). Important: A hypothesis defines a family of problems.
• Propose: Hypotheses H1 = (5,11), H2 = (5,10)
04/22/23 32SCG Innovation
![Page 33: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/33.jpg)
RegExpToAutomata Problem opposing
• opposing(refuting, strengthening)• Alice claims: Hypothesis (5,11)– Bob discounts it by refuting it: Bob invents a
regular expression r of size 5, gives it to Alice and she fails to deliver a DFA d with 11 or fewer states. Alice’ claim is refuted.
• Alice claims: Hypothesis (5,20)– Bob discounts it by strengthening it to (5,19); and
he can successfully support this hypothesis
04/22/23 33SCG Innovation
![Page 34: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/34.jpg)
RegExpToAutomata Problem supporting
• Alice claims: Hypothesis (4,12)– Bob tries to discount but Alice supports it: Bob
gives to Alice a regular expression r of size 4. Alice provides and equivalent DFA with 12 or fewer states. Alice has supported her hypothesis.
04/22/23 SCG Innovation 34
![Page 35: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/35.jpg)
Who wins? Alice or Bob?
• Reputation of Alice = – the number of hypotheses that Alice proposed that
were never successfully opposed by Bob (neither refuted nor strengthened) +
– the number of hypotheses that Bob proposed that were successfully opposed by Alice.
• RA = HAnotOpposedB + HBOpposedA• The scholar with the highest reputation wins.• encourages: creating minimum automata for
regular expressions of a given size.
04/22/23 35SCG Innovation
![Page 36: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/36.jpg)
Software Development Process
• Increase targeted interaction between software developers.
04/22/23 Innovation 36
![Page 37: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/37.jpg)
Traditional Approach
Human Developers
Develop new softwarefor problem solving domain X
Static Evaluation.
No competition.
human1 human2
Testing unit testing integration testing
Benchmark is usedto evaluate software
human3 human4
Users
Requirements for X
37SCG-SP201004/22/23
![Page 38: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/38.jpg)
Why Software Development through a virtual scientific community?
Human Developers
Develop new softwarefor problem solving domain X
SCG(X)
Erika-Patrick-agent
winning-agent
Evaluates fairly, frequently,
constructively and
dynamically.
Drives innovation.
Challenges humans.
Agents point humans to
what needs attention in
the software.
human1 human2
Erika Patrick
Benchmark is usedto evaluate software
Users
Requirements for X
38SCG-SP201004/22/23
![Page 39: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/39.jpg)
Erika-Patrick Agent
• Surrogate of combined knowledge of Erika and Patrick successfully transferred to agent.
• Transfer knowledge by programming.
39SCG-SP201004/22/23
![Page 40: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/40.jpg)
Conclusions
• How to make learning and problem solving fun: design a game and interact.
• Scientific Community Game = Specker Challenge Game = SCG
• How to create reliable problem solving software? Have it tested through SCG.
04/22/23 Innovation 40
![Page 41: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/41.jpg)
Final Slide
• More Questions?
04/22/23 Innovation 41
![Page 42: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/42.jpg)
04/22/23 SCG Innovation 42
![Page 43: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/43.jpg)
SCG concepts
• Scholars working in a domain with niches. Define functions on niches.
• Hypotheses: claims about functions on niches:– Discounting protocol for HA: Alice selects niche
element ne and Bob applies fBob so that claim about function does not hold
– Strengthening protocol• Reputation
04/22/23 43SCG Innovation
![Page 44: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/44.jpg)
SCG concepts
• Scholars working in a domain with niches. Function f: Niche -> S for Alice and Bob.
• Hypotheses: claims about niches: belief: f has property b(s, dn, fdn). (Niche,Belief)– Discounting protocol: Alice selects niche element
ne and Bob applies fBob creating s, so that !b(s,ne)
– Strengthening protocol• Reputation
04/22/23 44SCG Innovation
![Page 45: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/45.jpg)
Hypothesis Structure
• Algorithm Solver: Problems -> Solutions• For all p in Problems with feature f in Features
algorithm Solver solves p using resources p(f) with quality(p,Solver(p),f).
• Algorithm Provider: Features -> Problems• For feature f, Algorithm Provider provides a
problem p, for all solutions of p, !quality(p,Solver(p),f).
04/22/23 45SCG Innovation
![Page 46: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/46.jpg)
Two person SCG• Alice, Bob• Domain: Source, Target; fA, fB-> Source-> Target;
Source defined by niche predicate.• Hypotheses HA (HB): claims about fA (fB)• Discounting protocol for HA:– Bob provides element ne in Source so that fA(ne)
contradicts HA.– Alice provides element ne in Source so that fB(ne)
contradicts HA. • Strengthening protocol– Bob proposes HB, HA => HB and Alice cannot discount HB.
04/22/23 46SCG Innovation
![Page 47: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/47.jpg)
Two person SCGSpecialize for problem solving
• Alice, Bob• Domain: Problems -> Solutions• Hypotheses• Discounting protocol for hypothesis HA by Alice:– Bob attacks in one of two ways (depends on HA)
• Bob provides a problem for which Alice constructs a solution that contradicts HA.
• Alice provides a problem for which Bob constructs a solution that contradicts HA.
• Strengthening protocol– Bob proposes HB, HA => HB and Alice cannot discount HB.
04/22/23 47SCG Innovation
![Page 48: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/48.jpg)
Hypotheses
• Solution algorithm A: Problems->Solutions• For all elements p in Problems that have
feature F and a secret solution ss(p), algorithm A(p) constructs with resource constraint Prediction(F) an element s(p) in set Solutions(p) with property Q(p,s(p),ss(p),F).
04/22/23 48SCG Innovation
![Page 49: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/49.jpg)
Hypotheses
• Problem creation algorithm A-1
04/22/23 49SCG Innovation
![Page 50: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/50.jpg)
Discounting protocol
04/22/23 50SCG Innovation
![Page 51: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/51.jpg)
SCG by Example
• Highest safe rung problem• Speed prediction problem• graph diameter / average pair-wise distance
04/22/23 51SCG Innovation
![Page 52: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/52.jpg)
Example: Jar Stress Testing
• You have a ladder with r rungs, and you want to find the highest rung from which you can drop a copy of the jar and not have it break. We call this the highest safe rung problem (r,b).
• How many experiments do you need? Minimize.
• (r,infinity)• (r,1)
04/22/23 52SCG
![Page 53: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/53.jpg)
Highest Safe Rung ProblemProblems and Solutions
• Problems: p=((r,b),secret hsr), secret hsr in [0,r], r,b natural numbers• r = number of rungs• b = number of jars that are allowed to break• (r,b) is called a niche
• Solutions: sequence of queries of the form n? to find hsr. Responses: yes/no.
• Quality of solution: q = length of sequence of queries
04/22/23 53SCG Innovation
![Page 54: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/54.jpg)
Highest Safe Rung ProblemHypotheses
• Alice claims the hypothesis: I can solve any problem p=((r,b),secret hsr) with quality q: abbreviated H = ((r,b),q)
• Problems to be delivered for H = ((r,b),q) are of the form ((r,b), s). Important: A hypothesis defines a family of problems.
• Propose: Hypotheses H1 = ((25,2),11), H2 = ((25,2),6)
(from Kleinberg/Tardos)
04/22/23 54SCG Innovation
![Page 55: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/55.jpg)
Highest Safe Rung Problemopposing
• opposing(refuting, strengthening)• Alice claims: Hypothesis ((25,2),5)– Bob opposes it by refuting it: Bob invents problem
((25,2), secret 22). Alice: 5? no, 10? yes, 6? no, 7? no, 8? no. Already 5 questions asked and answer still unknown. Alice’ claim is refuted.
• Alice claims: Hypothesis ((25,2),12)– Bob opposes it by strengthening it to ((25,2),9);
and he can successfully support this hypothesis
04/22/23 55SCG Innovation
![Page 56: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/56.jpg)
Highest Safe Rung Problemsupporting
• Alice claims: Hypothesis ((25,2),12)– Bob tries to discount but Alice supports it: Alice:
5? no, 10? no, 15? no, 20? no, 25? yes, 21? no, 22? no, 23? no, 24? yes. Only 9 questions asked and problem ((25,2), secret 23) is solved. Alice has supported her hypothesis.
04/22/23 SCG Innovation 56
![Page 57: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/57.jpg)
Who wins? Alice or Bob?
• Reputation of Alice = – the number of hypotheses that Alice proposed that
were never successfully discounted by Bob (neither refuted nor strengthened) +
– the number of hypotheses that Bob proposed that were successfully discounted by Alice
• RA = HAnotDiscountedB + HBdiscountedA• The scholar with the highest reputation wins• encourages: creating strong knowledge and
discounting knowledge created by others
04/22/23 57SCG Innovation
Motivated by real scientific community
![Page 58: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/58.jpg)
Highest Safe Rung Problemcompetition / collaboration
• Alice claims: Hypothesis ((25,2),12)– Bob tries to discount but Alice supports it: Alice: 5?
no, 10? no, 15? no, 20? no, 25? yes, 21? no, 22? no, 23? no, 24? yes.
– From this exchange which is prompted by Alice defending her reputation, Bob gets an idea: For problem: p=((r,b),secret hsr), consider f(r,q) =(r/q + q) and find a q so that f(25,q) is minimized. f(25,5)=10; f(25,6)=11;f(25,4)=11.
– From this idea Bob knows that he can strengthen the hypothesis to ((25,2),10)
– General solution: Given r, find q to minimize (r/q + q).
04/22/23 SCG Innovation 58
![Page 59: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/59.jpg)
Highest Safe Rung ProblemAsymptotic Hypotheses
• Alice claims the hypothesis: I can solve any problem p=((r,b),secret hsr) with quality f(r,b) : abbreviated H = ((r,b),f(r,b))
• Problems to be delivered for H = ((r,b),f(r,b)) are of the form ((r,b), secret hsr).
• Propose: Hypotheses H1 = ((r,b),(log(r))b), H2 = ((r,b),r1/b)
04/22/23 59SCG Innovation
![Page 60: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/60.jpg)
Highest Safe Rung Problemdiscounting asymptotic hypothesis
• discounting (refuting, strengthening)• Alice claims: Hypothesis ((r,b),(b*log(r)))– Bob discounts it by refuting it: Bob invents
problem ((1024,2), secret hsr). log(1024) = 10. 20 questions are not enough! Alice: 30? no, 60? yes, 31? no, 32? no, etc.. Already 20 questions asked and answer still unknown. Alice’ claim is refuted.
• Alice claims: Hypothesis ((r,2),r/2)– Bob discounts it by strengthening it to ((r,2),2*r½ );
and he can successfully support this hypothesis.
04/22/23 60SCG Innovation
![Page 61: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/61.jpg)
Physics Maximum Height ProblemProblems and Solutions
• Problems: p=(v, a), v, a: positive real numbers• The maximum height obtained by a projectile
launched with speed v at angle a to the horizontal is z.
• Solutions: real number z.• Quality of solution: Number of correct decimal
places.
04/22/23 61SCG Innovation
![Page 62: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/62.jpg)
Physics Maximum Height Problem Hypotheses
• Alice claims the hypothesis: I can solve any maximum height problem p=(v,a) with quality q in 1 minute: abbreviated H = (MHP,q)
• Problems to be delivered for H = (MHP,q) are of the form (v,a).
• Propose: Hypotheses H1 = (MHP,3), H2 = (MHP,6)
04/22/23 62SCG Innovation
http://scienceworld.wolfram.com/physics/Height.html
![Page 63: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/63.jpg)
Physics Maximum Height Problem discounting
• discounting (refuting, strengthening)• Alice claims: Hypothesis (MHP,3)– Bob discounts it by refuting it: Bob invents
problem (25,60 degrees). Alice fails to solve the problem in 1 minute with 3 correct digits. Alice’ claim is refuted. Checking is done by experiment or trusted third party.
• Alice claims: Hypothesis (MHP,1)– Bob discounts it by strengthening it to (MHP,2);
and he can successfully support this hypothesis
04/22/23 63SCG Innovation
![Page 64: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/64.jpg)
RegExpToAutomata ProblemProblems and Solutions
• Problems: p=(r,n); r a regular expression of size n.• r = regular expression; a + b* a + a a a b*• n defines a niche of regular expressions
• Solutions: DFA d equivalent to r.• Quality of solution: Number of states of d.
04/22/23 64SCG Innovation
![Page 65: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/65.jpg)
RegExpToAutomata ProblemProblems and Solutions
• Problems: p=(r,n); r a regular expression of size n.• r = regular expression; a + b* a + a a a b*• n defines a niche of regular expressions
• Solutions: DFA d equivalent to r.• Quality of solution: Number of states of d.
04/22/23 65SCG Innovation
![Page 66: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/66.jpg)
RegExpToAutomata Problem Hypotheses
• Alice claims the hypothesis: I can solve any problem p=(r,n) with quality q or less: abbreviated H = (n,q)
• Problems to be delivered for H = (n,q) are of the form p=(r,n). Important: A hypothesis defines a family of problems.
• Propose: Hypotheses H1 = (5,11), H2 = (5,10)
04/22/23 66SCG Innovation
![Page 67: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/67.jpg)
RegExpToAutomata Problem discounting
• discounting (refuting, strengthening)• Alice claims: Hypothesis (5,11)– Bob discounts it by refuting it: Bob invents a
regular expression r of size 5, gives it to Alice and she fails to deliver a DFA d with 11 or fewer states. Alice’ claim is refuted.
• Alice claims: Hypothesis (5,20)– Bob discounts it by strengthening it to (5,19); and
he can successfully support this hypothesis
04/22/23 67SCG Innovation
![Page 68: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/68.jpg)
RegExpToAutomata Problem supporting
• Alice claims: Hypothesis (4,12)– Bob tries to discount but Alice supports it: Bob
gives to Alice a regular expression r of size 4. Alice provides and equivalent DFA with 12 or fewer states. Alice has supported her hypothesis.
04/22/23 SCG Innovation 68
![Page 69: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/69.jpg)
Who wins? Alice or Bob?
• Reputation of Alice = – the number of hypotheses that Alice proposed that
were never successfully discounted by Bob (neither refuted nor strengthened) +
– the number of hypotheses that Bob proposed that were successfully discounted by Alice.
• RA = HAnotDiscountedB + HBdiscountedA• The scholar with the highest reputation wins.• encourages: creating minimum automata for
regular expressions of a given size.
04/22/23 69SCG Innovation
![Page 70: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/70.jpg)
Calculus Maximization ProblemProblems and Solutions
• Problems: p=(f: function in one variable,J interval);
• Solutions: maximum of f in interval I.
04/22/23 70SCG Innovation
![Page 71: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/71.jpg)
Calculus Maximization Problem Hypotheses
• Alice claims the hypothesis(Polynomial, k): I can solve any problem p=(f,J) for f a polynomial in time (size of the polynomial)^k. H =(Polynomial, k).
• Problems to be delivered for H = (Polynomial, k) are of the form p=(f,J), f a polynomial. Important: A hypothesis defines a family of problems.
• Propose: Hypotheses H2 = (Polynomial, 2), H1 = (Polynomial,1).
04/22/23 71SCG Innovation
![Page 72: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/72.jpg)
Calculus Maximization Problem discounting
• discounting (refuting, strengthening)• Alice claims: Hypothesis (Polynomial,1)– Bob discounts it by refuting it: Bob invents a
polynomial (e.g., x^2 – x + 1) in one variable and an interval, gives them to Alice and she fails to deliver, in the given time, the maximum of the polynomial in the interval. Alice’ claim is refuted.
• Alice claims: Hypothesis (Polynomial,3)– Bob discounts it by strengthening it to (Polynomial,2);
and he can successfully support this hypothesis
04/22/23 72SCG Innovation
![Page 73: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/73.jpg)
• I claim I can solve this problem with one program that runs in time t on a single core machine and that runs in time 1.2 * t/c on a machine with c>1 cores.
04/22/23 SCG Innovation 73
![Page 74: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/74.jpg)
SCG
• Many kinds of hypotheses. They are defined by– Problems, Solutions– Discounting protocol • Refuting protocol• Strengthening protocol• Problems and solutions to be exchanged in protocols
04/22/23 SCG Innovation 74
![Page 75: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/75.jpg)
HypothesesAlice constructive claims
• I can solve problems of kind k – with quality q– close to your quality– better than you
• I claim statement S of the form – ForAllExists– ExistsForAll
04/22/23 SCG Innovation 75
![Page 76: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/76.jpg)
04/22/23 SCG Innovation 76
![Page 77: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/77.jpg)
Extra: too complex
04/22/23 SCG Innovation 77
![Page 78: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/78.jpg)
RegExpToAutomata ProblemProblems and Solutions
• Problems: p=(function in two variables f(t,b); interval for t; interval for b).• n defines a niche of regular expressions
• Solutions: min max solution.• Quality of solution: Number of states of d.
04/22/23 78SCG Innovation
![Page 79: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/79.jpg)
Minimizing and Maximizing FunctionsProblems and Solutions
• Problems: minimizing and maximizing functions. • Solutions: correct values.
04/22/23 79SCG Innovation
![Page 80: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/80.jpg)
Minimizing and Maximizing Functions Hypotheses
• Alice claims the hypothesis: function in two variables f(t,b); interval for t; interval for b. min [t] max [b] < h. H = (f(t,b),It,Ib,h)
• I can solve any problem p=(r,n) with quality q or less: abbreviated H = (n,q)
• Problems to be delivered for H = (n,q) are of the form p=(r,n). Important: A hypothesis defines a family of problems.
• Propose: Hypotheses H1 = (5,11), H2 = (5,10)
04/22/23 80SCG Innovation
![Page 81: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/81.jpg)
04/22/23 SCG Innovation 81
Calculus Alice claims the hypothesis: min t max b f(t,b) < 0.8. t and b are vectors in a subset of some vector space. Bob opposes Alice' hypothesis by strengthening it: min t max b f(t,b) < 0.7. Alice opposes Bob's hypothesis by strengthening it further: min t max b f(t,b) < 0.6. Bob opposes Alice' hypothesis by challenging it. Alice provides t=t0 and Bob finds b=b0 and it turns out that f(t0,b0)=0.65. Therefore Bob wins reputation from Alice.
![Page 82: NU ACM Talk Virtual Scientific Communities for Driving Innovation and Learning](https://reader036.vdocuments.mx/reader036/viewer/2022070419/56815b1c550346895dc8cd67/html5/thumbnails/82.jpg)
Highest Safe Rung ProblemProblems and Solutions
• Problems: p=((r,b),secret hsr), secret hsr in [0,r], r,b natural numbers• r = number of rungs• b = number of jars that are allowed to break• (r,b) is called a niche
• Solutions: sequence of queries of the form n? to find hsr. Responses: yes/no.
• Quality of solution: q = length of sequence of queries
04/22/23 82SCG Innovation
p=((r,b),floating)