Issues in Applying Probability Theory to AGI

Pei WangTemple UniversityPhiladelphia, USA

What Is Probability?

As a mathematical theory, probability theory is defined by its axioms on a probability function P(x)

When the theory is applied to a concrete problem, P(x) needs to be interpreted to measure something in the domain. Common choices:

Frequentist: the limit of occurrence frequency Subjective: the degree of belief of a subject Logical: the degree of evidential support

Probability is Relative

P(A) is not an intrinsic property of A under every interpretation, but related to a reference

Frequentist: relative to an event sequence Subjective: relative to a given subject Logical: relative to a body of evidence

A probability function PR(x) cannot be legally used without a clear interpretation and a fixed reference R, though they are often implicit in the description

Frequentist Interpretation in AGI

Given the general-purpose demand of AGI, probability under the frequentist interpretation has issues:

The reference sequence is hard to decide Some events' occurrence frequency has no limit Some events are unique and unrepeatable Some statements are not even “events”

Frequentist interpretation is too restrict for AGI

Subjective Interpretation in AGI

The subjective interpretation only demands the consistency among degrees of belief (don't confuse it with the logical interpretation)

Issues when it is used in AGI: Too weak ― the system can believe anything, as far

as the beliefs are consistent Too strong ― though the consistency among beliefs

are highly desired, it may not be achievable in AGI in realistic situations

Logical Interpretation in AGI

Intuitively, this is the most suitable interpretation for AGI, since the system's degree of belief should measure evidential support.

Issues: How to define evidence? (Confirmation Paradox,

evidential support vs. conditional probability) Can simplicity be used as evidence? (Occam's razor) Can all the beliefs in a system be evaluated by the

same evidence? (Bayesian conditioning)

Degree of Belief in NARS

NARS assumes insufficient knowledge and resources, and consequently,

Evidence is defined on all statements in a term logic “Truth-value”, “degree of belief”, and “evidential

support” are the same thing Each belief has its own evidential base, so the

degrees of belief are not necessarily consistent Simplicity is preferred, but not factored in truth-

value

Conclusion

Probability theory cannot be used as the foundation of AGI, because (even under a proper interpretation) it demands knowledge (such as a prior distribution that is immune to future revision) and resources (for global belief update) that AGI systems cannot afford

When the axioms of probability theory are violated, the resulting models are not legal approximations of the theory, unless approximation ratio is accurately proved