foundations for critical thinking and analysis - durant-law bagpiping

Foundation Skills for Thinking and Analysis

‘The greatest challenge to any thinker is stating the problem in a way that will allow a solution.’

Bertrand Russell, Philosopher, Logician, and Mathematician

Copyright © 2012 – HyperEdge Pty Ltd

http://www.hyperedge.com.au/

Your Instructor: A/Prof Graham Durant-Law CSC, PhD

BSc, MHA, MKM, Grad Dip Def, Grad Dip Mngt, Grad Cert Hlth Fin, psc.

Graham is the owner and chief scientist of HyperEdge Pty Ltd . He: is an Adjunct Associate Professor at the University of Queensland and

the University of Canberra; is an expert in social and organisational network analysis, and

developed the business network analysis™ modelling methodology; is a former Regular Army Colonel, with command and operational

experience;

2

is an acknowledged thought leader in knowledge management, and has been one of the international adjudicators for the Singapore Knowledge Management Excellence Awards for the past six years;

holds numerous academic qualifications and has won prizes for academic achievement, as well as awards for innovation and practice;

was awarded a prize for stakeholder management in the Project Management Institute’s Project Manager of the Year Award in 2010; and

is passionate about building superior organisations using evidence-based methods, and maintains a blog called Knowledge Matters with this theme.


http://www.durantlaw.info/

Course Administration

Ablutions and Fire escape

The course is organised as three half-day sessions: – Thinking about Thinking, and Your 1st Tools for Any

Analysis; – Probability, Trees and Matrices; and – Basic Link and Network Analysis.

Each session consists of several “mini-lectures”, which: – introduce the topic, and – have one or more practical exercises.

End time - 12:30 or 17:00.

Do: – ask questions – challenge – participate

3

In seeking wisdom, the first step is silence, the second is listening, the third remembering, the fourth practicing,

the fifth – teaching others.

— Solomon ibn Gabirol

Logic Diagram

4

Thinking about Thinking: Your 1st Tools for any Analysis

An Analytic Framework &

Model

Logic, Evidence, & Inferences

Divergent & Convergent

Thinking

Question Types

Sorting, Chronologies &

Timelines

Argument Maps

Probability, Trees & Matrices

Causal Maps & Influence Diagrams

Probability & Probability Trees

Data Models & Matrices

Utility Trees & Matrices

Basic Link & Network Analysis

Network Thinking

Homogenous & Heterogeneous

Networks Matrices & Networks

Attributes & Datasets


‘Computers are useless. They can only give you answers!’ Pablo Picasso, Painter and Sculptor

5 Copyright © 2012 – HyperEdge Pty Ltd


Logic Diagram – Where Are We Now?

6



Model



Thinking

Question Types


Timelines

Argument Maps







Network Thinking




Logic Diagram – Mini-Lectures and Exercises

7

An Analysis Framework & Model

1. Your Conceptual Framework

Logic, Evidence & Inferences

1. Inductive & Deductive

Reasoning

Divergent & Convergent Thinking

1. Divergent Thinking in Your Workplace

2. Convergent Thinking in Your Workplace

Question Types

1. Question Types

Sorting, Chronologies & Timelines

1. Perception, Sorting and Categorisation

2. Chronologies and Timelines: A Biblical Example

Argument Maps

1. Argument Map: Implementing a System

An Analysis Framework and Model

‘Choice of attention area, choice of entry point, choice of factors, these are all part of the first stage of thinking, which is so often taken for granted.’

Doctor Edward de Bono, Philosopher and Author

Copyright © 2012 – HyperEdge Pty Ltd


Knowledge and Facts

Knowledge

‘… acquaintance with facts, truths, or principles, from study or investigation’. ‘ … perception of fact and truth and being cognisant or aware of fact or

circumstance’. ‘… body of truths or facts accumulated by human beings in the course of time’.

Facts

‘… has really happened or is the case’. ‘… truth known by actual experience or observation’. ‘… something said to be true or supposed to have happened’ .

9

The Macquarie dictionary 2005, 4th edn, Macquarie University, Sydney, NSW.

Truth

Analytic truths are statements whose denial leads to a contradiction.

– Analytic truths have a law-like generality with no exceptions. – Truth is arrived at simply by analysing the subject term in the statement.

• Example. The assertion that ‘all mothers are female’ is an analytic truth because all mothers can be defined as a female parent. To deny the statement that ‘all mothers are female’ results in the absurd assertion that not all female parents are female.

Synthetic truths are statements that are true but can be denied without creating a contradiction.

– A synthetic truth contains two or more unrelated concepts. – Truth is arrived at using experience and beliefs.

• Example. The statement ‘… most human mothers are over twelve years old’ is a synthetic truth because it contains two unrelated concepts – the concept of being over twelve years old and the notion of being a human mother. We know the statement is true, based on experience and not simply by understanding the meanings of the words.

10 Horner, C & Westacott, E 2000, Thinking through philosophy, Cambridge University Press, Cambridge.

Beliefs

Beliefs are something that we hold to be true.

For an individual, beliefs are facts that are derived from either analytic or synthetic truths, or from some other source such as an authoritative (or not so authoritative) reference or person.

Our dictionary definition of knowledge says it consists of facts and truths.

Does a mistaken belief result in a fact?

Can knowledge be based on mistaken, but justified, beliefs?

11

Everitt, N & Fisher, A 1995, Modern epistemology, McGraw Hill, London.

Equation 1.

– proposition P is true, and – person S believes that P is true, and – S has adequate justification for believing that P is true.

But the justification for the belief could be wrong!

Equation 2.

– P is true – S believes that P is true, and – What P is about is causally connected in an appropriate way to S’s belief

that P is true.

But it does not account for situations where we know something but the ‘knowing’ is not caused by the thing. For example, we can know that 13 is a prime number.

Knowledge as Justified True Belief

12

Allen, B 2004, Knowledge and civilization, Westview Press, Oxford

Elements of Thought

All thinking is defined by the eight elements of thought shown in the adjacent diagram.

Thinking – generates purpose – raises questions – uses information – utilises concepts – makes inferences – makes assumptions – generates implications – embodies a point of view

13

Elder, L & Paul, R 2012, The Thinker's Guide to Analytic Thinking, The Foundation for Critical Thinking, Sonoma State University, California.

Do You Understand Your Conceptual Framework and Biases?

14

http://weknowmemes.com/2011/10/the-educational-system-comic/









Exercise – Your Conceptual Framework

Complete the following sentences:

“Whenever I am asked to analyse something I begin with the following conceptual framework …”

“The strengths of this framework are …” 1. … 2. … 3. …

“The limitations of this framework are …”

1. … 2. … 3. …

15

The Cynefin Framework

16

1. Snowden, D 1999, Cynefin, a sense of time and place: an ecological approach to sense making and learning in formal and informal communities, University of Surrey

2. Remington, K & Pollack, J 2007, Tools for complex projects, Gower, Aldershot

Logic, Evidence, and Inferences

‘It is a capital mistake to theorise before you have all the evidence. It biases the judgment.’

Sherlock Holmes



Logic

Logic is the philosophical study of valid reasoning. It is the underpinning of all reasoned argument.

Logic is used, or should be used, in all intellectual activities and analysis.

Logic is usually expressed in the form of arguments. For example

1. proposition P is true 2. person S believes that P is true, and 3. What P is about is causally

connected in an appropriate way to S’s belief that P is true.

Arguments are evaluated on the basis of evidence, assumptions and inference.

18

http://star.psy.ohio-state.edu/coglab/Miracle.html





Evidence and Inference

Evidence

Evidence in its broadest sense includes everything that is used to determine or demonstrate the truth of an assertion.

Evidence consists of facts and data.

Evidence may lead to a direct proof or a derived proof.

Key Questions: – What evidence do I need? – Is the evidence relevant? – Do I have sufficient evidence? – Do I have opposing and supporting

evidence? – How do I know the evidence is

accurate?

Inference

Inferences are derived interpretations or conclusions.

Inferences should logically follow from the evidence.

Key Questions:

– What conclusions am I coming to? – Is my inference logical? – Are there other conclusions I should

consider? – Does the interpretation make sense? – Does the solution necessarily follow

from the data? – Is there an alternative plausible

conclusion?

19 Elder, L & Paul, R 2012, The Thinker's Guide to Analytic Thinking, The Foundation for Critical Thinking, Sonoma State University, California.

Assumptions versus Inferences

Assumptions

Assumptions often operate at the sub-conscious level of thought.

Assumptions often are taken for granted beliefs.

Surfacing assumptions can reveal bias, stereotyping, prejudices, and other irrational forms of thinking.

Justifiable assumptions lead to reasonable inferences.

Inferences

Inferences operate at the conscious level of thought.

Inference are a step of thought leading to a conclusion that something is true, based on something else being true, or appearing to be true.

Inferences can be justified or unjustified.

All inferences are based on stated or unstated assumptions.

20 Elder, L & Paul, R 2012, The Thinker's Guide to Analytic Thinking, The Foundation for Critical Thinking, Sonoma State University, California.

Deductive Reasoning

Deductive Reasoning

Deductive reasoning involves using given true premises to reach a conclusion that is also true.

Deductive reasoning arrives at a specific conclusion from a general principle.

Deductive reasoning links premises with conclusions.

If the rules and logic of deduction are followed, this procedure guarantees an accurate conclusion.

Example: 1. If an angle is >90° then it is an obtuse

angle. (theory) 2. Angle A is an obtuse angle. (hypothesis) 3. A=120° (observation) 4. A is an obtuse angle (confirmation)

21

http://www.socialresearchmethods.net/kb/dedind.php



Inductive Reasoning

Inductive Reasoning

Inductive reasoning constructs or evaluates propositions that are abstractions of observations of individual instances of members of the same class.

Inductive reasoning is also known as hypothesis construction because any conclusions made are based on educated predictions.

Inductive reasoning allows for the possibility that the conclusion is false, even when all of the premises are true. The answer is probably true.

Example: 1. Joe is a human. (observation) 2. Most humans are right-handed. (pattern) 3. Joe is right-handed. (hypothesis) 4. The probability that Joe is right-handed

is 75%. (theory)

22




Abductive Reasoning

Abductive Reasoning

Abductive reasoning is similar to inductive reasoning, but differs in that observations are always incomplete.

Abductive reasoning begins with an incomplete set of observations and proceeds to the likeliest possible explanation for the set.

The abductive process can be creative, intuitive, and sometimes even revolutionary.

The answer or solution is likely to be true based on the available observations.

Example: 1. A medical diagnosis is an application of

abductive reasoning: given a set of symptoms, what is the diagnosis that would best explain most of them?

23




Exercise – Inductive and Deductive Reasoning

Decide whether or not the following are examples of inductive or deductive reasoning. Justify your answers

1. There are 32 books on the top-shelf of the bookcase, and 12 on the lower shelf of the bookcase. There are no books anywhere else in the bookcase. Therefore, there are 44 books in the bookcase.

2. The members of the Williams family include Susan, Nathan and Alexander. Susan wears glasses. Nathan wears glasses. Alexander wears glasses. Therefore, all members of the Williams family wear glasses.

3. It has snowed in Canberra every July in recorded history. Therefore, it will snow in Canberra this coming July.

4. All odd numbers are integers. All even numbers are integers. Therefore, all odd numbers are even numbers.

24

Divergent and Convergent Thinking

‘The world we have created is a product of our thinking; it cannot be changed without changing our thinking.’

Albert Einstein, Theoretical Physicist



Do You Understand Your Thinking Type?

Sequential Thinker Associative Thinker

26

Thought

Thought

Thought Thought

Thought

Thought

Thought

Thought Thought

Thought

Divergent versus Convergent Thinking

Divergent Thinking Convergent Thinking

27

Answer Facts Facts

Facts

Facts

Brainstorm

Collect, Cull, & Cluster

Select Ideas or Solution

Divergent and Convergent Thinking Sequence

Where possible begin with divergent thinking.

– The more ideas the better. – Don’t evaluate ideas yet. – Strange ideas are ok.

Focus on winnowing out the impractical, and clustering ideas that are similar (the start of convergent thinking).

– Collect facts for each idea. – Build one idea upon the other.

Select ideas that promising (convergent thinking).

– Promising ideas are supported by facts.

– Promising ideas are intuitively practical.

28 Jones, M 1998, The thinker's toolkit: 14 powerful techniques for problem-solving, Three Rivers Press, New York.

Exercise – Divergent Thinking


“In my workplace divergent thinking is used to …”

1. … 2. …

“In my workplace divergent thinking

could be used …” 1. … 2. …


is most appropriate when …” 1. … 2. …

29

Exercise – Convergent Thinking


“In my workplace convergent thinking is used to …”

1. … 2. …

“In my workplace convergent

thinking could be used …” 1. … 2. …


is most appropriate when …” 1. … 2. …

30

Question Types

‘Can a mortal ask questions which God finds unanswerable? Quite easily, I should think. All nonsense questions are unanswerable.’

C.S. Lewis, British Author



Question Types

Factual – Soliciting reasonably simple, straight forward answers based on obvious facts or

awareness. – These are usually at the lowest level of cognitive or affective processes and answers

are frequently either right or wrong.

Convergent – Answers to these types of questions are usually within a very finite range of

acceptable accuracy. – These may be at several different levels of cognition - comprehension, application,

analysis, or ones where the answerer makes inferences or conjectures based on personal awareness, or on material read, presented or known.

Divergent – These questions explore different avenues and create many different variations and

alternative answers or scenarios. – Correctness may be based on logical projections, may be contextual, or arrived at

through basic knowledge, conjecture, inference, projection, creation, intuition, or imagination.

– These types of questions require analysis, synthesise, or evaluation of a knowledge base, and then project or predict different outcomes.

32 http://www4.uwsp.edu/Education/lwilson/learning/quest2.htm

http://www4.uwsp.edu/Education/lwilson/learning/quest2.htm


Question Types (continued)

Evaluative – These types of questions usually require sophisticated levels of cognitive

and/or emotional judgment. – In attempting to answer evaluative questions, analysts may be combining

multiple logical and/or affective thinking process, or comparative frameworks.

– Often an answer is analysed at multiple levels and from different perspectives before arriving at newly synthesised information or conclusions.

Combinations – These are questions that blend any combination of factual, convergent,

divergent and evaluative questions,

33 http://www4.uwsp.edu/Education/lwilson/learning/quest2.htm



Exercise – Question Types

Provide two examples of:

Factual questions 1. … 2. …

Convergent questions 1. … 2. …

Divergent questions 1. … 2. …

Evaluative questions 1. … 2. …

Combination questions 1. … 2. …

34

The Six Knows

All questions are one or more of the “Six Knows”.

The Six Knows combined with the previous question types determine the thinking tool that is most appropriate for the task.

For example:

– Chronologies and timelines are the appropriate tool to answer a “Know When” question.

– Sorting and categorisation tools are useful to answer “Know What” questions.

35

Lundvall, B & Johnson, B 1994, 'The learning economy', Journal of Industry Studies, vol. 1, pp. 23-42.


“Whenever I am asked to analyse something I use the following thinking tools …”

1. … 2. …

“The strengths of these tools are …”

1. … 2. …

“The limitations of these thinking tools are …”

1. … 2. …

Exercise – Your Thinking Tools

36

Sorting, Chronologies and Timelines

‘Clocks slay time... time is dead as long as it is being clicked off by little wheels; only when the clock stops does time come to life.’

William Faulkner, Author and Novelist



Sorting and Categorising

Arguably, the first step of analysis is to sort and categorise data.

Even the simplest problems can benefit from sorting and categorisation.

Categories can be discrete or hierarchical.

Categories are not always obvious. Sorting and categorisation is subject to

cultural and personal bias.

38

Brainstorm

Collect, Cull, & Cluster

Select Ideas or Solution

Jones, M 1998, The thinker's toolkit: 14 powerful techniques for problem-solving, Three Rivers Press, New York.

Exercise - Sorting and Categorisation

Put the following into a category, or categories, and justify your answer.

Put the following into a category, or categories, and justify your answer.

39 Chiu, L.-H. (1972). A cross-cultural comparison of cognitive styles in Chinese and American children. International Journal of Psychology, 7(4), 235-242.

Chronologies and Timelines

Human instinctively think chronologically, therefore sorting data chronologically aids understanding.

A chronology can: – be presented in a tabular format, or as a vertical or horizontal timeline; – direct attention to significant events and gaps; – identify patterns and correlations; and – sometimes show cause and effect.

40

Exercise – Chronologies and Timelines

Using the data in the table below sort and categorise the events, and then construct a timeline.

41

Event Year Source Abraham leaves Haran 1926 BC Genesis 16:16

Birth of Abraham 2001 BC Genesis 12:4 Birth of Isaac 1901 BC Genesis 25:26

Birth of Ishmael 1915 BC Genesis 21:5 Birth of Jacob 1841 BC Genesis 47:9

Birth of Joseph 1750 BC Genesis 41:46, 53, 45:6 Birth of Moses 1576 BC Exodus 7:7

Crossing of Jordan 1456 BC Joshua 14:7, 24:29 David begins reign 1060 BC 2 Samuel 5:4

Death of Jacob 1694 BC Genesis 49:33 Death of Joseph 1640 BC Genesis 50:22, 26

Dividing of the Land 1451 BC Joshua 14:7-10 Exodus from Egypt 1496 BC Exodus 16:35 Jacob enters Egypt 1711 BC Genesis 47:28

Judges begin leading Israel 1426 BC Joshua 24:29 Saul (first king) begins reign 1100 BC Acts 13:21

Solomon begins reign 1020 BC 1 Kings 6:1

Solution

42

Argument Maps

‘How many a dispute could have been deflated into a single paragraph if the disputants had dared to define their terms.’

Aristotle, Greek Philosopher



Argument Mapping

44

• An argument map is a visual representation of the structure of your logic using a visual tree.

• Argument mapping helps groups achieve a shared understanding of wicked problems.

• It uses a notation called IBIS (Issue Based Information System) to map the relationship between questions, ideas, and arguments.

Exercise – Flesh Out ‘How?’ and ‘Budget’

45 http://eight2late.wordpress.com/2009/08/27/the-approach-a-dialogue-mapping-story/

http://eight2late.wordpress.com/2009/08/27/the-approach-a-dialogue-mapping-story/

Solution

46 http://eight2late.wordpress.com/2009/08/27/the-approach-a-dialogue-mapping-story/













Questions

‘The real questions refuse to be placated… They are the questions asked most frequently and answered most inadequately, the ones that reveal their true natures slowly, reluctantly, most often against your will.’

Ingrid Bengis, Author



Summary

In today’s session we looked at: – An Analytic Framework and Model; – Logic, Evidence and Inferences; – Divergent and Convergent Thinking; – Question Types; – Sorting, Chronologies and Timelines; and – Argument Maps.

All thinking is coloured, if not tainted, by our beliefs and analysis framework.

Inferences are derived interpretations or conclusions, and should logically follow from the evidence.

We should use both divergent and convergent thinking to solve a problem.

The question type often determines the thinking tool, or tools, to be used.


48

Next Session

In the next session we will look at:

– Causal Maps and Influence Diagrams; – Probability and Probability Trees; – Data Models and Matrices; and – Utility Trees and Matrices.

The session is even more hands-on, and includes the following exercises:

– parking at your workplace; – selecting a conference location; – determining the probability of a death; – determining the probability of project success; – establishing the correlation between symptoms

and a disease; and – determining your best investment option.

Some exercises require the use of a few very basic junior high school mathematical skills. 49

For more details please visit our website at http://www.hyperedge.com.au

Our training and course offerings are at:

http://www.hyperedge.com.au/training Example reports can be found at:

http://www.hyperedge.com.au/sites/default/files/Example_Org_Comm_Profile.pdf and,

http://www.hyperedge.com.au/sites/default/files/Example_Pers_Comm_Profile.pdf .

A/Prof Graham Durant-Law CSC, PhD +61 (0) 408 975 795 [email protected] HyperEdge Pty Ltd Post Office Box 3076 Manuka ACT 2603 Australia


http://www.hyperedge.com.au/training

http://www.hyperedge.com.au/sites/default/files/Example_Org_Comm_Profile.pdf

http://www.hyperedge.com.au/sites/default/files/Example_Pers_Comm_Profile.pdf

mailto:[email protected]?subject=HyperEdge Request for More Information

Probability, Trees and Matrices

‘Opinion is that exercise of the human will which helps us to make a decision without information.’

Professor John Erskine, Educator and Philosopher



Revision

In the first session we looked at: – An Analytic Framework and Model; – Logic, Evidence and Inferences; – Divergent and Convergent Thinking; – Question Types; – Sorting, Chronologies and Timelines; and – Argument Maps.






52


53



Model



Thinking

Question Types


Timelines

Argument Maps







Network Thinking





54


1. Causal Maps - Car Parking

2. Influence Diagram – Conference Planning


1. Mutually Exclusive Probability – Chook Deaths

2. Conditionally Dependent Probability – Project Success

Data Models & Matrices 1. Matrix Construction –

Symptom and Disease Correlation

Utility Trees & Matrices 1. Utility Tree Analysis – Small

Business Investment Choices

2. Utility Matrix Analysis – Small Business Investment Choices

Causal Maps and Influence Diagrams

‘It is common error to infer that things which are consecutive in order of time have necessarily the relation of cause and effect.’

Doctor Jacob Bigelow, Botanist and Physician



Causal Maps

A causal map is a type of concept map in which the links between nodes represent causality or influence.

Key questions: – What is causing this problem? – How are the major factors interacting to

produce this result?

Step 1: Identify major factors, particularly those that are dynamic and/or temporal.

Step 2: Identify cause and effect relationships.

Step 3: Categorise the relationships as direct or inverse.

Step 4: Map the relationships. Step 5: Analyse relationship behaviours

as an integrated system. 56 Morecroft, J 2007, Strategic modelling and business dynamics, John Wiley and Sons, Ltd, Chichester.

Causal Maps – Steps 1, 2 and 3

Step 1: Identify major factors, particularly those that are dynamic and/or temporal. (Divergent Thinking)

Step 2: Identify cause and effect relationships. (Convergent Thinking)

– Build a table with causal factors and affected factors

– Note an affected factor becomes a causal factor

Step 3: Categorise the relationships as positive, negative, or unknown

– A positive causal factor increases the affected factor

– A negative causal factor decreases the affected factor

– Try to avoid unknowns as they are unstable!

Causal Factor Affected Factor

Sales Profits

Profits R&D Capability

R&D Capability New Products

New Products Competitors New Products

Competitors New Products Sales

57

Causal Factor Affected Factor Direction

Sales Profits Positive

Profits R&D Capability Positive

R&D Capability New Products Positive

New Products Competitors New Products Positive

Competitors New Products Sales Negative


Exercise – Causal Maps

Build the causal effect tables, and then construct a causal loop diagram for the following situation. Indicate with a “+” and a “-” whether the cause and effect relationships are positive or negative.

1. Your workplace’s parking areas are close to the city, and are suspected to be used by city commuters who do not want to pay for parking in the city.

2. Your workplace, rather generously but naively, decides to build 50 additional parking spaces.

3. After much fanfare the additional parking spaces become available following six months of construction.

4. Six months later parking is as bad as it ever was, and possibly worse. People in your workplace are often late for work because they cannot find a parking place. Once again city commuters are suspected to be the culprits!

5. Again your workplace, rather generously and naively, decides to build 50 additional parking spaces.

6. Again six months later parking is as bad as it ever was, and possibly worse.

58

Solution

59

Causal Factor Affected Factor lack of free parking in the city availability of workplace parking availability of workplace parking arrival time at work arrival time at work lack of free parking in the city

Causal Factor Affected Factor + or - lack of free parking in the city availability of workplace parking - availability of workplace parking arrival time at work + arrival time at work lack of free parking in the city -

Influence Diagrams

An influence diagram is a compact graphical representation of a decision situation. It is often called a Scenario Tree.

Key characteristics: – The branches of the tree are mutually exclusive. – The branches are collectively exhaustive. – It dissects a scenario into sequential events. – It clearly shows cause and effect linkages.

Probabilities (discussed next) are often used to enhance the diagrams.


Exercise – Influence Diagrams

Construct a scenario tree that portrays all of the possible scenarios to be considered in the following problem:

1. Your boss has allocated money for you to run a team-building exercise.

2. You decide that after-hours activities are a important element for team-bonding.

3. Three sites are under consideration. – The first offers hiking and skiing in the mountains. – The second offers swimming and sunbathing on a beach. – The third offers eating, drinking, and dancing into the small hours.

4. You open the decision up to the staff. They must decide on: – a two-day or three-day activity; and – a hiking and skiing venue, a swimming and sunbathing venue, or a eating,

drinking and dancing venue.


Solution

62

Probability and Probability Trees

‘Probability is expectation founded upon partial knowledge. A perfect acquaintance with all the circumstances affecting the occurrence of an event would change expectation into certainty.’

George Boole, Mathematician and Logician



Probability

Understanding and dealing with probability is a crucial skill because it permeates analysis, both explicitly and implicitly.

The moment analysis moves from facts to inference and judgement we enter the realm of estimates and probability.

People use probability expressions like, “most likely”, “may”, and “in all likelihood”, as if these words mean the same thing to everyone. They don’t!

Rule Number 1: All probability expressions should be converted to percentiles or a meaningful number!

64

Fact

Inference

Value Judgement


Probability (continued)

When we have all the facts we can calculate probability – a deterministic problem.

When we don’t have all the facts we estimate probability based on frequency and experience.

Rule Number 2: If we don’t have reliable evidence to judge which outcome is more likely, we should assume the probability is equal for all outcomes.

Mutually exclusive events preclude one another. For example, tossing a coin or rolling a dice.

Conditionally dependent events are sequential. For example turning a key to unlock a locked door. (Unlocking the lock is conditional on turning the key).


Mutually Exclusive Probability

An easy way to calculate mutually exclusive probability is to think of them as percentages.

What percentage IS this number OF that one? Example:

What is the probability of picking a green jelly bean?

1. There are 6 jelly beans. 2. 3 jelly beans are green. 3. What percentage IS 3 OF 6? 4. 3/6= 0.50 = 50%

To calculate an “or” statement (red or orange )

we add the numbers together.

1. There are 6 jelly beans. 2. 1 jelly bean is red. 2 jelly beans are

orange. 1 + 2 = 3 3. What percentage IS 3 OF 6? 4. 3/6= 0.50 = 50%

66

Pick one jelly bean

Constructing a Probability Tree

A probability tree is similar to an influence diagram, but a basic influence tree can only show what can and cannot happen.

A probability tree can show what is likely and unlikely to happen.

Steps in creating a probability tree

1. Create the categories. 2. Count the number of items in each

category. 3. Sum the total number of items. 4. Calculate the probabilities.

Rules:

1. Events must be mutually exclusive. 2. Events must be collectively exhaustive. 3. The probabilities at each branching of the

tree must equal 1.0.

67

Pick one jelly bean

45

36

9

Probability

0.5 = 50%

0.4 = 40%

0.1 = 10%

Exercise Mutually Exclusive Probability

You have 50 chooks at the back of your house, with the following colours:

– 10 are red, – 5 are black, – 15 are brown, and – 20 are white.

If a fox gets into the chook-pen and randomly kills a chook, what is the probability the chook will be red or brown?

Draw a probability tree showing each colour chook’s probability of being killed.

68

Solution

69

Conditionally Dependent Probability

Conditionally dependent events are sequential: they have an and statement.

Conditionally dependent probability is calculated by multiplying the two probabilities.

The probability of picking a red jelly bean is 0.5.

The probability of picking a green jelly bean is 0.1

The probability of picking a red jelly bean, returning it to the jar, and then picking a green jelly bean is 0.5 x 0.1 = 0.05 i.e. 5%


Exercise Conditionally Dependent Probability

Your missile project is trying to meet a political deadline for acquisition. To do this production must occur within two months. A crucial test firing of the prototype missile is scheduled for tomorrow. Three outcomes, with their associated probabilities are possible:

1. total failure, assessed as being 0.2; 2. successful flight, but technical failure, assessed as being 0.6; 3. complete success, assessed as 0.2; 4. If the flight is a total failure, the developers only have a 0.1 probability of

beginning production in two months; 5. If the flight is successful but experiences technical failure, there is a 0.4

probability of beginning production in two months; and 6. If the flight is a complete success the probability of beginning production

in two months is 0.9.

Construct a probability tree that portrays these event.

What is the probability the developers will meet the deadline?

What is the probability they won’t meet the deadline in the event of either a total failure, or a successful flight with a technical failure?

71

Solutions

What is the probability the developers will meet the deadline? From the probability tree we add

the “Yes” branches 0.02 + 0.24 + 0.18 = 0.44

What is the probability they won’t meet the deadline in the event of either a total failure, or a successful flight with a technical failure? From the probability tree we add

the “No” branches: 0.18 + 0.36 = 0.54

72

Data Models and Matrices

‘The mathematical phenomenon always develops out of simple arithmetic, so useful in everyday life, out of numbers, those weapons of the gods: the gods are there, behind the wall, at play with numbers.’

Charles-Édouard Jeanneret (Le Corbusier), Architect and Philosopher



Data Model

A high-level data model is an abstraction that shows the data needed and created by the business or analytical processes.

A data model explicitly determines the structure of data.

Data models typically use a symbolic language.

Data models are often categorised as hierarchical, networked or relational.

Regardless of category type, most data models use matrices at some point.

Communication and precision are the two key benefits that make a data model important for analysts to use and exchange data.

74

Matrices

A matrix is nothing more than a table with as many cells as needed for the problem under analysis.

A matrix allows the analysts to:

– Separate the elements of the problem

– Categorise data by type – Compare one type of data with

another – Compare data of the same type – See correlations and clusters

Example:

Smith, Brown, Jones, and Williams have dinner together. They have a bit too much to drink. When they leave each by mistake takes the hat belonging to someone else, and the coat belonging to yet another person.

The person who took Williams hat took Jones’s coat. Smith took Brown’s hat. The hat taken by Jones belonged to the owner of the coat taken by Williams. Who took Smith’s hat?

75

Williams

Williams

Jones

Jones

Brown

Brown Smith

Smith Jones

Smith Brown Williams


Exercise – Matrix Construction

Construct a matrix from the following information:

– 37 patients with a particular symptom had a disease.

– 33 patients with the same symptom did not have the disease.

– 17 patients without the symptom had the disease.

– 13 patients without the symptom did not have the disease.

Is there a correlation between the symptoms and the disease?


Solution

77

Disease Yes No

Symptoms Yes 37 33 No 17 13

Disease Yes No

Symptoms Yes 53% 47% No 57% 43%

Disease Yes No

Symptoms Yes 69% 72% No 31% 28%

Matrix with raw numbers

“Symptom” % (rows)

“Disease” % (columns)

The proportion of those with symptom and with disease is about the same as those without symptom and with disease, therefore there is no correlation.

Utility Trees and Matrices

‘Counting pairs is the oldest trick in combinatorics... Every time we count pairs, we learn something from it.’

Professor Gil Kalai, Israeli Mathematician



Utility

From a purely analytic standpoint utility is the benefit that someone has received, is receiving, or expects to receive from some situation.

The basic elements of utility analysis are:

– options, or alternative course of action;

– outcomes, which result from the courses of action; and

– perspectives, which are the point of view used to analyse the outcome.


Utility Tree

There are eight steps in a utility tree analysis.

1. Identify the options and all possible outcomes. (Divergent thinking).

2. Identify the perspective of the analysis.

3. Construct the scenario trees.

4. Assign a utility value to each option-outcome combination – the dream value.

5. Assign a probability value to the outcomes.

6. Assign expected values, which is utility value x probability value.

7. Rank the expected values from first to last.

8. Do a sanity test!


Example Utility Tree – Betting on Jelly Beans

81

Option Probability

0.5

0.4

0.1

0.5

0.4

0.1

0.5

0.4

0.1

Utility

$0.00

$3.00

$0.00

$2.00

$0.00

$0.00

$0.00

$0.00

$4.00

Earned Value

$0.00

$1.20

$0.00

$1.00

$0.00

$0.00

$0.00

$0.00

$0.40

Total EV

$1.20

$1.00

$0.40

Rank

1

2

3

Outcome

Exercise – Utility Tree Analysis

Your chooks are doing quite well, but unfortunately the foxes have killed all the black chooks. That aside you decide to go into the free-range meat business in a small way.

The Department of Primary Industries tells you that: – 100 brown chooks should earn you $200 profit, – 100 white chooks should earn you $300 profit, and – 100 red chooks should earn you $400.

They suggest that you do not mix types.

Unfortunately Blue Spot Chook Flu is a problem. The Department of Primary Industries tells you that:

– brown chooks have an 80% survival rate, – white chooks have a 50% survival rate, and – red chooks have a 30% survival rate.

Considering the profit and the disease, which type of chook is the best investment? Work out the answer using a utility tree.

82

Solution

83

Utility Matrix

A utility matrix is essentially the same as a utility tree.

The utility tree has the advantage of showing the whole scenario, but if the scenario is large the tree can become complex and messy.

The utility matrix usually focuses on alternative outcomes.

The utility matrix has the decided advantage of being easy to construct in Microsoft® EXCEL.

– Calculations can therefore be automated.

– Analysis can be somewhat automated, for example alternative scenarios and goal seeking.

– Results can be displayed as graphs.

84

Exercise – Utility Matrix Analysis

Your chooks are doing quite well, but unfortunately the foxes have killed all the black chooks. That aside you decide to go into the free-range meat business in a small way.

The Department of Primary Industries tells you that: – 100 brown chooks should earn you $400 profit, – 100 white chooks should earn you $300 profit, and – 100 red chooks should earn you $200 .

They suggest that you do not mix types.

Unfortunately Blue Spot Chook Flu is a problem. The Department of Primary Industries tells you that:

– brown chooks have an 30% survival rate, – white chooks have a 50% survival rate, and – red chooks have a 80% survival rate.

Considering the profit and the disease, which type of chook is the best investment? This time work out the answer using a utility matrix.

85

Solution

86

Probability Earned Value

Number Utility Live Die Live Die Total

Earned Value

Rank

Brown Chooks 100 $400 0.30 0.70 $120.00 $0.00 $120.00 3 White Chooks 100 $300 0.50 0.50 $150.00 $0.00 $150.00 2 Red Chooks 100 $200 0.80 0.80 $160.00 $0.00 $160.00 1

Questions

‘The real questions refuse to be placated… They are the questions asked most frequently and answered most inadequately, the ones that reveal their true natures slowly, reluctantly, most often against your will’.

Ingrid Bengis, Author, 1973



Summary

In this session we covered: – Causal Maps and Influence Diagrams – Probability and Probability Trees – Data Models and Matrices – Utility Trees and Matrices

These methods build on the methods taught in the first session. For example:

– they make use of divergent and convergent thinking to solve a problem; and

– the question type often determines the thinking tool, or tools, to be used.

Again the first step is to sort and categorise data.

Finally, do not become enamoured by the numbers. They are a means not an end!

88

Next Session

89

• Some exercises will require the use of a few very basic junior high school mathematical skills.

• You will need to apply the skills from the last two sessions.

Basic Link and Network Analysis

Simply because your data links people and you can visualize that, it does not mean you have performed network analysis. This is akin to displaying a line plot of some stock's price over a quarter and claiming you have performed statistical analysis – all you have done is report data! As with all other statistical processes, network analysis is meant to draw meaning and inference from the structure, which requires an understanding of these methodologies, their strengths and limitations’.

Drew Conway, Political Scientist, 2009.



Revision

In the first session we looked at: – An Analytic Framework and Model; – Logic, Evidence and Inferences – Divergent and Convergent Thinking; – Sorting, Chronologies and Timelines.






92

Revision

In the second session we covered: – Causal Maps and Influence Diagrams – Probability and Probability Trees – Data Models and Matrices – Utility Trees and Matrices

These methods built on the methods taught in the first session. For example:

– they made use of divergent and convergent thinking to solve a problem; and

– the question type was used to determine the thinking tool, or tools, to be used.

The first step remained to sort and categorise data.

Remember do not become enamoured by the numbers. They are means not an end!

93


94



Model



Thinking

Question Types


Timelines

Argument Maps







Network Thinking

Homogenous &

Heterogeneous Networks

Matrices & Networks



95

Network Thinking

1. Redraw Your Branch as a Network

Homogenous & Heterogeneous Networks

1. Identifying Network Maps

Matrices & Networks

1. Construct a Project Network Map

2. Construct a Network Matrix, Edge-List & Capability Network Map

Attributes & Data Sets

1. Project Interfaces Data Structure

2. Attributing Nodes & Links

Network Thinking

‘A system is a network of interdependent components that work together to try to accomplish the aim of the system. A system must have an aim. Without the aim, there is no system.’

Professor William Edwards Deming, Statistician



Hierarchical Thinking Everyone understands the hierarchical view

97

This view does not allow for cross-branch communication

Network Thinking This network view is exactly the same as the hierarchical view

98

This view could allow for cross-branch communication

The Thinking Shift Allows Us To Do This

99

This view does allow for cross-branch communication. Note what is different.

Exercise – Network Thinking

Redraw your branch hierarchical diagram as a network diagram, but maintain the formal structure.

Highlight your position with a red circle.

Then show the communication pattern for:

– decision-making – problem solving – friendships

What are the strengths and

weaknesses of showing multiple relationships on one diagram?

100

What is Network Analysis?

Network analysis is based on an assumption of the importance of relationships among interacting nodes.

A methodology that provides the ability to examine quantitatively, qualitatively, and graphically macro and micro linkages between nodes.

A connection between two or more nodes means there is some sort of relationship.

Unit of data is the dyad – pairs of nodes.

101

Monge, P & Contactor, N 2003, Theories of communication networks, Oxford University Press, New York.

What is the Lexicon of Network Analysis?

A node is the smallest unit in the network. It is also known as a vertex.

A tie is a line between two nodes indicating there is a relationship between them.

A graph is a set of nodes and a set of ties between pairs of nodes.

A network consists of a graph and additional information on the nodes or the ties of the graph. It is also known as a map.

102

Newman, M 2010, Networks: an introduction, Oxford University Press, New York.

Homogenous and Heterogeneous Networks

‘Each of us is part of a large cluster, the worldwide social net, from which no one is left out. We do not know everyone on this globe, but it is guaranteed that there is a path between any two of us in this web of people’.

Professor Albert-Laszlo Barabasi, Physicist



Homogenous Networks

Also known as a one-mode or adjacency network.

We link like nodes with like nodes. For example:

– people to people – teams to teams – projects to projects

Almost all network measures can be used, and visualisations can be easily layered.

Data can be captured and structured as a square matrix (covered in next mini-lecture).

104

Heterogeneous Networks

Also known as a two-mode or affiliation network.

We link two types of nodes with each other. For example:

– people to projects, or – people to teams.

The relationship between like nodes is through the other node. For example the University of Canberra is linked to Harvard University through (or by) a Research Collaboration Agreement.

Data is captured and structured as a rectangular matrix or edge-list (covered in next mini-lecture).

105

Compound Heterogeneous Networks

The realm of link analysis.

We link three or more types of nodes with each other. For example, people to teams to projects to teams.

Again the relationship between like nodes is through the other nodes.

Generally we are looking at links between unlike nodes

Many network measures cannot be reliably used.

Data is captured and structured as a rectangular matrix or edge-list (covered in next mini-lecture).

106

Is this a homogeneous, heterogeneous, or compound heterogeneous network?

Justify your answer.

Exercise – Identifying Network Maps

107

Is this a homogeneous, heterogeneous, or compound heterogeneous network?

Justify your answer.

Matrices and Networks

‘Mathematicians do not study objects, but relations between objects. Thus, they are free to replace some objects by others so long as the relations remain unchanged. Content to them is irrelevant: they are interested in form only.’

Jules Henri Poincare, Mathematician and Philosopher



A Homogenous (One-Mode) Network

John Thomas Anna James Peter Mary Michael David Anthony Bobby

John 0 0 0 0 0 0 0 1 0 0

Thomas 0 0 0 0 0 0 1 1 0 0

Anna 0 0 0 0 0 0 0 0 0 0

James 0 0 0 0 0 0 0 0 0 0

Peter 0 0 0 0 0 0 0 0 0 0

Mary 0 0 0 0 1 0 0 0 0 1

Michael 0 0 0 1 0 0 0 0 0 0

David 0 1 0 0 0 0 0 0 0 0

Anthony 0 0 0 0 0 0 0 0 0 0

Bobby 0 0 0 0 1 1 0 0 0 0

109

A Homogenous (One-Mode) Network

110

Exercise – Homogenous Network Map

Draw the resultant network map from the following matrix:

111

Project A

Project B

Project C

Project D

Project E

Project F

Project H

Project I

Project J

Project K

Project A 0 0 0 1 0 0 0 1 0 0

Project B 0 0 1 1 0 0 1 1 0 0

Project C 0 0 0 0 0 0 0 0 0 0

Project D 0 0 0 0 0 0 0 0 0 0

Project E 1 1 0 0 0 0 0 0 0 0

Project F 0 0 0 0 1 0 0 0 0 1

Project H 0 0 0 1 0 0 0 0 0 0

Project I 0 1 0 0 0 0 0 0 0 0

Project J 0 0 0 0 0 0 0 0 0 0

Project K 0 0 0 0 1 1 0 0 0 0

Solution

112

Note Project J does not appear in the network. For greater accuracy we could show Project J as an isolate.

A Heterogeneous (Two-Mode) Network

ID Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 8 Item 9 Item 10

CY10034 0 0 0 1 1 0 0 0 0 1

CY10039 1 0 0 1 0 0 0 0 0 0

CY10044 0 0 0 1 1 0 0 0 0 0

CY10045 0 0 1 0 0 0 0 0 0 0

CY10047 0 0 1 0 0 0 0 0 0 0

CY10054 1 0 0 1 0 0 0 0 0 0

CY10055 0 1 0 0 1 0 1 0 0 0

CY10057 0 1 0 0 0 0 0 0 0 0

CY10059 0 1 0 0 0 0 0 0 0 0

113

A Heterogeneous (Two-Mode) Network

114

Exercise – Heterogeneous Network Map

Draw the resultant network map from the following matrix:

115

ID Project A

Project B

Project C

Project D

Project E

Project F

Project H

Project I

Project J

Project K

John 0 0 0 1 1 0 0 0 0 1

Thomas 1 0 0 1 0 0 0 0 0 0

Anna 0 0 0 1 1 0 0 0 0 0

James 0 0 1 0 0 1 0 0 0 0

Peter 0 0 1 0 0 0 1 0 1 0

Mary 1 0 0 1 0 0 0 0 0 0

Michael 0 1 0 0 1 0 1 0 0 0

David 0 1 0 0 0 0 0 0 0 0

Anthony 0 1 0 0 0 0 0 0 0 0

Solution

116

Note Project I does not appear in the network. For greater accuracy we could show Project I as an isolate.

Matrix versus Edge-List

Matrix Format

FE 1 FE 2 FE 3 FE 4

FE 1 0 1 1 1

FE 2 0 0 1 1

FE 3 1 0 0 0

FE 4 1 0 1 0

Edge List Format

From To

FE 1 FE 2

FE 1 FE 3

FE 1 FE 4

FE 2 FE 3

FE 2 FE 4

FE 3 FE 1

FE 4 FE 1

FE 4 FE 3

117

Exercise – Building Matrices and Edge-Lists

Construct a matrix, or matrices, from the following data:

– Project A will contribute to Capabilities 1, 3, 5, 7, and 9. – Project B will contribute to Capabilities 2, 4, 6, and 8. – Capabilities 2, 4, and 6 are impacted by Projects C and D. – Capabilities 5 and 7 are impacted by Projects E and F.

Construct an edge-list, or edge-lists, from the same data.

What sort of network, or networks, are we constructing?

Draw the resultant network, or networks.

What are the advantages and disadvantages of each format?

118

Solution

119

C1 C2 C3 C4 C5 C6 C7 C8 C9 PA 1 0 1 0 1 0 1 0 1 PB 0 1 0 1 0 1 0 1 0 PC 0 0 0 0 0 0 0 0 0 PD 0 0 0 0 0 0 0 0 0 PE 0 0 0 0 0 0 0 0 0 PF 0 0 0 0 0 0 0 0 0

Project contributes to Capability

PA C1 PA C3 PA C5 PA C7 PA C9 PB C2 PB C4 PB C6 PB C8

Solution

120

PA PB PC PD PE PF C1 0 0 0 0 0 0 C2 0 0 1 1 0 0 C3 0 0 0 0 0 0 C4 0 0 1 1 0 0 C5 0 0 0 0 1 1 C6 0 0 1 1 0 0 C7 0 0 0 0 1 1 C8 0 0 0 0 0 0 C9 0 0 0 0 0 0

Capability is impacted by Project

C2 PC C2 PD C4 PC C4 PD C5 PE C5 PF C6 PC C6 PD C7 PE C7 PF

Solution

121

Project contributes to Capability Capability is impacted by Project

Attributes and Data Sets

‘Simplicity is the key to effective scientific inquiry.’ Professor Stanley Milgram, Sociologist



Data Sets and Data Capture

123

Exercise – Flesh Out ‘Project Interfaces’

124

Solution

125

Node Attributing

Node Name (for example, Project Name)

Attribute 1 (for example, Project-

Program)

Attribute 2 (for example, Value)

Project Name Project-program Value



126

Node Attributes can also be Calculated or Derived

127

If an individual only sends messages and receives none then their contribution index is +1.000 If an individual only receives messages and sends none then their contribution index is -1.000 If the communication behaviour is balanced then the contribution index is 0.000

Contribution Frequency

Contribution Index

Sender +1

Receiver -1

Expert

Envoi

Escort

Expediter

Gloor, P 2006, Swarm creativity: Competitive advantage through collaborative innovation networks, Oxford University Press, Oxford.

messages sent – messages received

messages sent + messages received Contribution Index =

Calculated or Derived Attributes can Change the Network Analysis from this …

128

No Discernible Role

Expert

Escort Envoi

Expediter

1. The links inside the “circles” are posts between like roles. Note there are no posts between Experts. 2. The thicker curves linking groups are consolidated exchanges between groups. They do not show frequency, or links from one

individual to another. 3. Note the relative density in the Escort and Expediter groups.

To this (using only Microsoft® Excel)

129

Link Attributing

Matrix Format

FE 1 FE 2 FE 3 FE 4

FE 1 0 1 2 2

FE 2 0 0 1 1

FE 3 2 0 0 0

FE 4 2 0 1 0

Edge List Format

From To Weight

FE 1 FE 2 1

FE 1 FE 3 2

FE 1 FE 4 2

FE 2 FE 3 1

FE 2 FE 4 1

FE 3 FE 1 2

FE 4 FE 1 2

FE 4 FE 3 1

130

Filtering Using Link Attributes

131

How do you understand 2nd, 3rd, and 4th order, or beyond, interdependencies in other programs? Are you even aware of them?

Exercise – Attributing Nodes and Links

Construct the node and link matrices, from the following data:

– Project A • the acquisition cost is $666 million; • it is platform centric; • the contribution to Capabilities 1 and 3 is considered to be critical; and • the contribution to Capabilities 5, 7, and 9 is thought to be minor.

– Project B

• the acquisition cost is $35 million; • it is people centric; • it is a troubled project, subject to intense media scrutiny; • the contribution to Capabilities 1, 3, 5, 7 and 9 is considered to be critical; and • the contribution to Capabilities 2, 4, 6, and 8 is thought to be important.

Draw the resultant network, or networks.

What do you think is the best way to structure data? 132

Solution

133

Type Cost $M Troubled PA platform 666 yes PB people 35 no

C1 C2 C3 C4 C5 C6 C7 C8 C9 PA 3 0 3 0 1 0 1 0 1 PB 3 2 3 2 3 2 3 2 3

Node Matrix

Link Matrix

Questions





Summary

In this session we considered: – Network Thinking – Homogenous and Heterogeneous Networks – Matrices and Networks – Attributes and Datasets

It is a small step from using matrices for problem-solving to network thinking and visualisation.

Network thinking is not the only way to analyse a problem, and is not always appropriate.

Consider your audience. The network analysis may be better presented in other ways.

Time spent in preparing and structuring data is always rewarded!

135

Course Summary

Science is built with facts, as a house is with stones. But a collection of facts is no more a science than a heap of stones is a house.

Henri Poincare, Theoretical Physicist and Philosopher



Summary

In Thinking about Thinking, and Your 1st Tools for Any Analysis we looked at:

– An Analytic Framework and Model; – Logic, Evidence and Inferences; – Divergent and Convergent Thinking; – Question Types; – Sorting, Chronologies and Timelines; and – Argument Maps.






137

Using Matrices for Thinking and Analysis

In the ‘Using Matrices for Thinking and Analysis’ session we covered:

– Evidence, Inference, and Probability – Logic and Data Models – Matrices – Utility Trees and Matrices

These methods built on the methods taught in the first session. For example:

– they made use of divergent and convergent thinking to solve a problem; and

– the question type was used to determine the thinking tool, or tools, to be used.

The first step remained to sort and categorise data.

Remember do not become enamoured by the numbers. They are means not an end! 138

Basic Link and Network Analysis

In the ‘Basic Link and Network Analysis’ session we considered:

– Network Thinking – Homogenous and Heterogeneous Networks – Matrices and Networks – Attributes and Datasets

It is a small step from using matrices for problem-solving to network thinking and visualisation.

Network thinking is not the only way to analyse a problem, and is not always appropriate.

Consider your audience. The network analysis may be better presented in other ways.

Time spent in preparing and structuring data is always rewarded!

139

Final Questions





Conclusion

These tools can be used every day, but it takes discipline and practice to master them.

Contact me if you need assistance.

A/Prof Graham Durant-Law CSC, PhD +61 (0) 408 975 795 [email protected]

My blog is Knowledge Matters


141



foundations for critical thinking and analysis - durant-law bagpiping

Documents