dealing with negative results
TRANSCRIPT
Dealing with negative resultsResearch methods and proposal writing
Dr M Benini
Università degli Studi dell’Insubriakindly hosted by Dr L Longo
December 3rd, 2016
Science is positive
The vast majority of scientific results are stated as answers toquestions of the kind■ why■ how
The first question asks for an explanation of some phenomenon;the second one requires to provide a justified method to achievesome result of interest
( 2 of 19 )
Science is positive
Consider the negated questions:■ why not■ how not
The first question requires to provide a way to falsify anexplanation; the second question asks to show a faulty way toachieve a result, and to put in evidence its failureAs a matter of fact, we have a huge number of negative answersin our experience, and we need them!
( 3 of 19 )
Science is positive
Consider the negated questions:■ why not■ how not
The first question requires to provide a way to falsify anexplanation; the second question asks to show a faulty way toachieve a result, and to put in evidence its failureAs a matter of fact, we have a huge number of negative answersin our experience, and we need them!
( 3 of 19 )
Science is positive
When we do research, we collect many negative answers:■ when an experiment fails, we have to understand why it didnot succeed
■ when a (mathematical) proof fails to be found, we have to askourselves what prevents it, eventually searching for acounterexample
■ when a procedure fails to achieve the desired result, we haveto understand what went wrong, and why
( 4 of 19 )
Science is positive
When we do research, we collect many negative answers:■ when an experiment fails, we have to understand why it didnot succeed
■ when a (mathematical) proof fails to be found, we have to askourselves what prevents it, eventually searching for acounterexample
■ when a procedure fails to achieve the desired result, we haveto understand what went wrong, and why
( 4 of 19 )
Science is positive
When we do research, we collect many negative answers:■ when an experiment fails, we have to understand why it didnot succeed
■ when a (mathematical) proof fails to be found, we have to askourselves what prevents it, eventually searching for acounterexample
■ when a procedure fails to achieve the desired result, we haveto understand what went wrong, and why
( 4 of 19 )
Science is positive
When we do research, we collect many negative answers:■ when an experiment fails, we have to understand why it didnot succeed
■ when a (mathematical) proof fails to be found, we have to askourselves what prevents it, eventually searching for acounterexample
■ when a procedure fails to achieve the desired result, we haveto understand what went wrong, and why
( 4 of 19 )
Science is positive
When we reach the final stage of our research efforts, publishingthe results, we focus on the positive achievements, And wediscard the negative experience we have accumulatedAn experienced researcher, instead, should be more effective,trying to use the negative results and to maximise the outcomesof his efforts
( 5 of 19 )
Science is positive
When we reach the final stage of our research efforts, publishingthe results, we focus on the positive achievements, And wediscard the negative experience we have accumulatedAn experienced researcher, instead, should be more effective,trying to use the negative results and to maximise the outcomesof his efforts
( 5 of 19 )
Negative content, positively presented
Although some results we have found are negative, they canoften be positively presented
■ when a procedure fails, precisely delimit the domain ofapplicability, excluding the cases we know to be bad: the limitbecomes a necessary condition, and the failure, acounterexample to a direct generalisation
■ when a mathematical proof fails, find a counterexample, anduse it to add a condition that prevents its applicability: theadditional hypothesis becomes necessary because of thecounterexample
( 6 of 19 )
Negative content, positively presented
Although some results we have found are negative, they canoften be positively presented
■ when a procedure fails, precisely delimit the domain ofapplicability, excluding the cases we know to be bad: the limitbecomes a necessary condition, and the failure, acounterexample to a direct generalisation
■ when a mathematical proof fails, find a counterexample, anduse it to add a condition that prevents its applicability: theadditional hypothesis becomes necessary because of thecounterexample
( 6 of 19 )
Negative content, positively presented
Although some results we have found are negative, they canoften be positively presented
■ when a procedure fails, precisely delimit the domain ofapplicability, excluding the cases we know to be bad: the limitbecomes a necessary condition, and the failure, acounterexample to a direct generalisation
■ when a mathematical proof fails, find a counterexample, anduse it to add a condition that prevents its applicability: theadditional hypothesis becomes necessary because of thecounterexample
( 6 of 19 )
Negative content, positively presented
When stating a result, use a positive languageTheorem [Gödel] Peano Arithmetic is not complete.Theorem [Gödel] There is a sentence G in Peano Arithmeticsuch that both G and ¬G are not provable.Theorem [Gödel] There is a sentence G in Peano Arithmeticsuch that G is true on natural numbers, but G is not provable.
( 7 of 19 )
Negative content, positively presented
When stating a result, use a positive languageTheorem [Gödel] Peano Arithmetic is not complete.Theorem [Gödel] There is a sentence G in Peano Arithmeticsuch that both G and ¬G are not provable.Theorem [Gödel] There is a sentence G in Peano Arithmeticsuch that G is true on natural numbers, but G is not provable.
( 7 of 19 )
Negative content, positively presented
When stating a result, use a positive languageTheorem [Gödel] Peano Arithmetic is not complete.Theorem [Gödel] There is a sentence G in Peano Arithmeticsuch that both G and ¬G are not provable.Theorem [Gödel] There is a sentence G in Peano Arithmeticsuch that G is true on natural numbers, but G is not provable.
( 7 of 19 )
Negative content, positively presented
When stating a result, use a positive languageTheorem [Gödel] Peano Arithmetic is not complete.Theorem [Gödel] There is a sentence G in Peano Arithmeticsuch that both G and ¬G are not provable.Theorem [Gödel] There is a sentence G in Peano Arithmeticsuch that G is true on natural numbers, but G is not provable.
( 7 of 19 )
Negative content, positively presented
When writing a statement, provide a positive contentTheorem An arbitrary list cannot be sorted in linear time.Theorem An arbitrary list can be sorted in O(n logn) steps.Theorem The heap sort algorithm sorts any list in at mostO(n logn) steps.
( 8 of 19 )
Negative content, positively presented
When writing a statement, provide a positive contentTheorem An arbitrary list cannot be sorted in linear time.Theorem An arbitrary list can be sorted in O(n logn) steps.Theorem The heap sort algorithm sorts any list in at mostO(n logn) steps.
( 8 of 19 )
Negative content, positively presented
When writing a statement, provide a positive contentTheorem An arbitrary list cannot be sorted in linear time.Theorem An arbitrary list can be sorted in O(n logn) steps.Theorem The heap sort algorithm sorts any list in at mostO(n logn) steps.
( 8 of 19 )
Negative content, positively presented
When writing a statement, provide a positive contentTheorem An arbitrary list cannot be sorted in linear time.Theorem An arbitrary list can be sorted in O(n logn) steps.Theorem The heap sort algorithm sorts any list in at mostO(n logn) steps.
( 8 of 19 )
Negative content, positively presented
When stating a counterexample, be positiveThe real-valued function f (x)= |x | does not admit a derivative inx = 0.The real-valued function f (x)= |x | admits f ′(x)=−1 as itsderivative on the interval (−∞,0), and f ′(x)= 1 on the interval(0,+∞). Thus, f ′(0) is not defined.
( 9 of 19 )
Negative content, positively presented
When stating a counterexample, be positiveThe real-valued function f (x)= |x | does not admit a derivative inx = 0.The real-valued function f (x)= |x | admits f ′(x)=−1 as itsderivative on the interval (−∞,0), and f ′(x)= 1 on the interval(0,+∞). Thus, f ′(0) is not defined.
( 9 of 19 )
Negative content, positively presented
When stating a counterexample, be positiveThe real-valued function f (x)= |x | does not admit a derivative inx = 0.The real-valued function f (x)= |x | admits f ′(x)=−1 as itsderivative on the interval (−∞,0), and f ′(x)= 1 on the interval(0,+∞). Thus, f ′(0) is not defined.
( 9 of 19 )
Negative content, positively presented
When stating a limit to a procedure or method, be positiveThe goto statement of procedural computer programs should beavoided whenever possible.The goto statement of procedural computer programs should belimited; use if ...then ...else ... or loops, instead.The use of goto statement of procedural computer programsshould be limited to treat exceptions, when a specific statementis not available; use if ...then ...else ... or loops in thenormal practice.
( 10 of 19 )
Negative content, positively presented
When stating a limit to a procedure or method, be positiveThe goto statement of procedural computer programs should beavoided whenever possible.The goto statement of procedural computer programs should belimited; use if ...then ...else ... or loops, instead.The use of goto statement of procedural computer programsshould be limited to treat exceptions, when a specific statementis not available; use if ...then ...else ... or loops in thenormal practice.
( 10 of 19 )
Negative content, positively presented
When stating a limit to a procedure or method, be positiveThe goto statement of procedural computer programs should beavoided whenever possible.The goto statement of procedural computer programs should belimited; use if ...then ...else ... or loops, instead.The use of goto statement of procedural computer programsshould be limited to treat exceptions, when a specific statementis not available; use if ...then ...else ... or loops in thenormal practice.
( 10 of 19 )
Negative content, positively presented
When stating a limit to a procedure or method, be positiveThe goto statement of procedural computer programs should beavoided whenever possible.The goto statement of procedural computer programs should belimited; use if ...then ...else ... or loops, instead.The use of goto statement of procedural computer programsshould be limited to treat exceptions, when a specific statementis not available; use if ...then ...else ... or loops in thenormal practice.
( 10 of 19 )
Impossibility results
Some results are inherently negative. If they are also significant,it is worth stating them in a negative wayTheorem [Turing] Deciding whether a program terminates onall inputs, is not a computable problem.Theorem [Turing] There is no computer program that, takingas input a computer program, decides whether it terminates onall inputs.
( 11 of 19 )
Impossibility results
Some results are inherently negative. If they are also significant,it is worth stating them in a negative wayTheorem [Turing] Deciding whether a program terminates onall inputs, is not a computable problem.Theorem [Turing] There is no computer program that, takingas input a computer program, decides whether it terminates onall inputs.
( 11 of 19 )
Impossibility results
Some results are inherently negative. If they are also significant,it is worth stating them in a negative wayTheorem [Turing] Deciding whether a program terminates onall inputs, is not a computable problem.Theorem [Turing] There is no computer program that, takingas input a computer program, decides whether it terminates onall inputs.
( 11 of 19 )
Dead ends
When we do research, experiments fail, proofs do not comethrough, and procedures give crazy resultsWe use these failures to better understand our problems, andeventually to fix what did not work properlySometimes, we get in a situation where there seem to be no wayout, in which we are unable to find a way to fix the incorrectbehaviours we encounter. We call these situation dead ends
( 12 of 19 )
Dead ends
When we do research, experiments fail, proofs do not comethrough, and procedures give crazy resultsWe use these failures to better understand our problems, andeventually to fix what did not work properlySometimes, we get in a situation where there seem to be no wayout, in which we are unable to find a way to fix the incorrectbehaviours we encounter. We call these situation dead ends
( 12 of 19 )
Dead ends
When you get into a dead end, you suffer from a lack ofalternatives. There are many ways to try to get out
■ Write down what you got as clearly as possible. The effect toorganise the knowledge collected so far often showsalternatives
■ Show your attempts to a colleague or to your supervisor. Afresh look often provides a new point of view
■ Leave the problem apart for a while. When we do not thinkconsciously to a problem, our mind follows different paths, andwe leave room for new intuitions
( 13 of 19 )
Dead ends
When you get into a dead end, you suffer from a lack ofalternatives. There are many ways to try to get out
■ Write down what you got as clearly as possible. The effect toorganise the knowledge collected so far often showsalternatives
■ Show your attempts to a colleague or to your supervisor. Afresh look often provides a new point of view
■ Leave the problem apart for a while. When we do not thinkconsciously to a problem, our mind follows different paths, andwe leave room for new intuitions
( 13 of 19 )
Dead ends
When you get into a dead end, you suffer from a lack ofalternatives. There are many ways to try to get out
■ Write down what you got as clearly as possible. The effect toorganise the knowledge collected so far often showsalternatives
■ Show your attempts to a colleague or to your supervisor. Afresh look often provides a new point of view
■ Leave the problem apart for a while. When we do not thinkconsciously to a problem, our mind follows different paths, andwe leave room for new intuitions
( 13 of 19 )
Dead ends
When you get into a dead end, you suffer from a lack ofalternatives. There are many ways to try to get out
■ Write down what you got as clearly as possible. The effect toorganise the knowledge collected so far often showsalternatives
■ Show your attempts to a colleague or to your supervisor. Afresh look often provides a new point of view
■ Leave the problem apart for a while. When we do not thinkconsciously to a problem, our mind follows different paths, andwe leave room for new intuitions
( 13 of 19 )
Dead ends
It is tempting to get some result out of a long period of hardwork. But, when the result is a dead end, do not publish it!A dead end leads to nowhere, by definition. You have fewchances to motivate your partial achievementsYou will spend time writing, submitting the paper, and receivingcomments. You know in advance that the results are notcomplete, and that the direction leads nowhere: honestly, anyreviewer will recognise these weaknesses, lowering your chances.Are you sure you are not wasting more time?
( 14 of 19 )
Dead ends
It is tempting to get some result out of a long period of hardwork. But, when the result is a dead end, do not publish it!A dead end leads to nowhere, by definition. You have fewchances to motivate your partial achievementsYou will spend time writing, submitting the paper, and receivingcomments. You know in advance that the results are notcomplete, and that the direction leads nowhere: honestly, anyreviewer will recognise these weaknesses, lowering your chances.Are you sure you are not wasting more time?
( 14 of 19 )
Dead ends
It is tempting to get some result out of a long period of hardwork. But, when the result is a dead end, do not publish it!A dead end leads to nowhere, by definition. You have fewchances to motivate your partial achievementsYou will spend time writing, submitting the paper, and receivingcomments. You know in advance that the results are notcomplete, and that the direction leads nowhere: honestly, anyreviewer will recognise these weaknesses, lowering your chances.Are you sure you are not wasting more time?
( 14 of 19 )
Publishing or discarding?
A basic rule in Economy says to buy when the market is low, andto sell when the market is highConsider pursuing a scientific result as an investment. Publishingis the act which provides you with a revenue. It makes no senseto submit an article when you know in advance that it will notget published. You spend time and effort with no revenue
( 15 of 19 )
Publishing or discarding?
A positive result has bigger chances to get published than anegative one. But negative results have some chances, too
■ Try to positively present your negative outcomes■ Strong impossibility results are landmarks. Prepare yourself todefend them, but be sure they will be important in the longterm
■ Weak impossibility results are interesting only for specialists.You should carefully evaluate if your audience is wide enoughto invest time and effort to try to publish such a negativeachievement
( 16 of 19 )
Publishing or discarding?
A positive result has bigger chances to get published than anegative one. But negative results have some chances, too
■ Try to positively present your negative outcomes■ Strong impossibility results are landmarks. Prepare yourself todefend them, but be sure they will be important in the longterm
■ Weak impossibility results are interesting only for specialists.You should carefully evaluate if your audience is wide enoughto invest time and effort to try to publish such a negativeachievement
( 16 of 19 )
Publishing or discarding?
A positive result has bigger chances to get published than anegative one. But negative results have some chances, too
■ Try to positively present your negative outcomes■ Strong impossibility results are landmarks. Prepare yourself todefend them, but be sure they will be important in the longterm
■ Weak impossibility results are interesting only for specialists.You should carefully evaluate if your audience is wide enoughto invest time and effort to try to publish such a negativeachievement
( 16 of 19 )
Publishing or discarding?
A positive result has bigger chances to get published than anegative one. But negative results have some chances, too
■ Try to positively present your negative outcomes■ Strong impossibility results are landmarks. Prepare yourself todefend them, but be sure they will be important in the longterm
■ Weak impossibility results are interesting only for specialists.You should carefully evaluate if your audience is wide enoughto invest time and effort to try to publish such a negativeachievement
( 16 of 19 )
Publishing or discarding?
Dead ends are the result of our efforts. What matters in Scienceare our findings, not the effort you madeYou should carefully judge what parts of your efforts aresignificant, and what are irrelevant to other people, with noregards for the time you investedDo not try to publish what you think is irrelevant! If you thinkso, how could you possibly convince a reader? You have tobelieve in the value of your results before trying to explain them
( 17 of 19 )
Experience
Get into the habit of writing down, as clearly as possible,everything you find during your research
■ you can reuse this pieces when writing the final paper■ you can use them to help a new collaborator to enter in yourteam
■ you have an evidence of your work■ you can use them to illustrate the point you reached to, e.g., acolleague or to your supervisor
■ you keep track of what you have understood. In the future,you may want to reuse these partial results elsewhere, and youdo not want to rework all the details from scratch
( 18 of 19 )
Experience
Get into the habit of writing down, as clearly as possible,everything you find during your research
■ you can reuse this pieces when writing the final paper■ you can use them to help a new collaborator to enter in yourteam
■ you have an evidence of your work■ you can use them to illustrate the point you reached to, e.g., acolleague or to your supervisor
■ you keep track of what you have understood. In the future,you may want to reuse these partial results elsewhere, and youdo not want to rework all the details from scratch
( 18 of 19 )
Experience
Get into the habit of writing down, as clearly as possible,everything you find during your research
■ you can reuse this pieces when writing the final paper■ you can use them to help a new collaborator to enter in yourteam
■ you have an evidence of your work■ you can use them to illustrate the point you reached to, e.g., acolleague or to your supervisor
■ you keep track of what you have understood. In the future,you may want to reuse these partial results elsewhere, and youdo not want to rework all the details from scratch
( 18 of 19 )
Experience
Get into the habit of writing down, as clearly as possible,everything you find during your research
■ you can reuse this pieces when writing the final paper■ you can use them to help a new collaborator to enter in yourteam
■ you have an evidence of your work■ you can use them to illustrate the point you reached to, e.g., acolleague or to your supervisor
■ you keep track of what you have understood. In the future,you may want to reuse these partial results elsewhere, and youdo not want to rework all the details from scratch
( 18 of 19 )
Experience
Get into the habit of writing down, as clearly as possible,everything you find during your research
■ you can reuse this pieces when writing the final paper■ you can use them to help a new collaborator to enter in yourteam
■ you have an evidence of your work■ you can use them to illustrate the point you reached to, e.g., acolleague or to your supervisor
■ you keep track of what you have understood. In the future,you may want to reuse these partial results elsewhere, and youdo not want to rework all the details from scratch
( 18 of 19 )
Experience
Get into the habit of writing down, as clearly as possible,everything you find during your research
■ you can reuse this pieces when writing the final paper■ you can use them to help a new collaborator to enter in yourteam
■ you have an evidence of your work■ you can use them to illustrate the point you reached to, e.g., acolleague or to your supervisor
■ you keep track of what you have understood. In the future,you may want to reuse these partial results elsewhere, and youdo not want to rework all the details from scratch
( 18 of 19 )
Experience
In the long term, negative results will form the fundamental partof your experience
■ you will know what directions and tools have few hopes tolead to a result
■ you have a track of open problems, not-yet-attemptedalternatives, variants to explore
■ negative results, well documented, can speed up yourachievements when you hear about a new result thatovercomes a difficulty which blocked you in the past
All these aspects of experience qualify you as an expert
( 19 of 19 )
Experience
In the long term, negative results will form the fundamental partof your experience
■ you will know what directions and tools have few hopes tolead to a result
■ you have a track of open problems, not-yet-attemptedalternatives, variants to explore
■ negative results, well documented, can speed up yourachievements when you hear about a new result thatovercomes a difficulty which blocked you in the past
All these aspects of experience qualify you as an expert
( 19 of 19 )
Experience
In the long term, negative results will form the fundamental partof your experience
■ you will know what directions and tools have few hopes tolead to a result
■ you have a track of open problems, not-yet-attemptedalternatives, variants to explore
■ negative results, well documented, can speed up yourachievements when you hear about a new result thatovercomes a difficulty which blocked you in the past
All these aspects of experience qualify you as an expert
( 19 of 19 )
Experience
In the long term, negative results will form the fundamental partof your experience
■ you will know what directions and tools have few hopes tolead to a result
■ you have a track of open problems, not-yet-attemptedalternatives, variants to explore
■ negative results, well documented, can speed up yourachievements when you hear about a new result thatovercomes a difficulty which blocked you in the past
All these aspects of experience qualify you as an expert
( 19 of 19 )
Experience
In the long term, negative results will form the fundamental partof your experience
■ you will know what directions and tools have few hopes tolead to a result
■ you have a track of open problems, not-yet-attemptedalternatives, variants to explore
■ negative results, well documented, can speed up yourachievements when you hear about a new result thatovercomes a difficulty which blocked you in the past
All these aspects of experience qualify you as an expert
( 19 of 19 )
The end
( 20 of 19 )