NBANeuralNet2015-16Update

BenTaborskyApril19,2016

Contents

1.  ReviewofModel2.  2015-16Performance3.  Analysis

1.ReviewofModel

AnNBANeuralNet•  AboutayearagoItrainedaneuralnettopredicttheoutcomesofbasketballgames.Thisyear,Itestedit

forreal,beLngonrealbasketballgameswithrealmoney.Theseslidesareanupdateonmyexperience,starNngwithareallyquickreviewofthemodel.

•  Aneuralnet’sstructureissortofbasedonbrainbiology.ItisaseriesofconnecNngnodesthat“fire”likeneuronsthroughtheir“dendrites,”basedonthevaluesofinputsitreceivesonits“axonterminals.”

•  NeuralnetworksarecoolandusefulbecausetheycanmatchanyfuncNonwiththeirnonlineariNes,andthey’reagreatwaytolearnaboutcomplicatedpaTernswithoutimposingalotofassumpNonslikeyoumighthavetowith,forexample,linearregression.

•  ItturnedouttobepossibletousejustarelaNvelysmallamountofdatatotrainaneuralnetworkthatachievedaleveloferrorcloseto,butnotquiteaslowas,theVegasmoneyline.

•  Ithoughtitwascloseenoughtowonder,ifthepredicNonsareverydifferent,isthisaglitchintheneuralnetoraretheoddsmakersmissingsomething?

Predic'on AverageCEErrorRandomProbability 1.00050%HomeVictory 0.69360%HomeVictory 0.671NetTrainingError 0.609NetValidaNonError 0.608WestgateMoneyLine 0.581

NetErroronBacktestYears(adj) 0.595

BeLngStrategyBacktest•  ToanswerthisquesNon,Imadeupastrategy:

–  Whenthedifferencebetweenthemodel’spredicNonandVegas’predicNonisgreaterthan15percentagepoints,makeabet

–  TheneuralnethasahardNmewithverybadhometeams,andnevergivesthemlessthana27-28%victoryprobability.SoIignorethefirstruleifVegasgivesahometeamlessthana40%chance.•  InteresNngnote:theseveryimprobablebetsgenerallybreakevenbecauseyouonlyhavetowinoneortwotomakeup

forlosses,andsomeNmesGoldenStatelosesatMilwaukee.Buttheyaddalotofuglyvariance.–  ThefourseasonsItestedwerenotusedtotraintheneuralnet.

•  Accordingtothesetests,themodelseemedtoberightogenenoughtobeconsistentlyprofitable.

AnnualAverageProfit 1632AnnualStandardDevia'on 962Informa'onRa'o 1.70MaxDrawdown 891

2014-15OutofSamplePerformance•  Ofcourse,nobodyshouldbelieveabacktestbyitself,becausethereareallkinds

ofbiasesthatcouldseeptheirwayin.Forexample,ItriedhardtocomeupwithagoodsystemforguessingwhatplayersI’dhaveknownwereplayingbeforeeachgame.ButIalsoreallywantedthemodeltowork.

•  Soinmid-February2015Ifrozeallthemodel’sparametersand“paper-traded”usingitspredicNonsunNltheendofthatseason,tryingtobeasrealisNcaspossible.

•  NotanoverwhelminglyposiNveresult,butalsonotinconsistentwiththebacktest.–  Mostimportantly,itdidn’tloseatonofmoney.

2.2015-16Performance

PuLngMyMoneyWhereMyMouthIs•  Agertheoutofsampletestwasnotacatastrophe,Idecidedtogivethemodelarealtestinthe

2015-16season.•  So,offIwenttospendthewinterinsnowyLakeTahoe,whereIcouldsnowboardstraightfrommy

liTlestudiotothecasino.–  UnNlIbrokemyarm.

•  Istuckwiththestrategyasdescribed:

–  Aslongasthehometeamisgivena40%+chanceatvictory,makeabetifthemodeldisagreeswiththelinebymorethan15percentagepoints.

–  AfewweeksintobeLngIrealizedthatthemodel’spredicNonsweredoingverywellinthe30-40%rangeforhomevictorysoIloweredthefloorto30%onaprovisionalbasisfortherestoftheseason(andmadehalf-sizebets,thoughthat’snotreflectedintheresultsIshowhere).

–  Never,everletpersonalfeelingsoropinionsoverridethemodel.Thiscanbehard.•  Model,youthinktheClippershavea38%chancetobeattheWarriors?Really?INOAKLAND?Oh,dear.

LovelyLakeTahoe

Vicodinandsausagefingers

PerformanceWith$100betsoneverymodelpredicNon,thecumulaNveprofitfortheyearlookslikethis:

Breakdowns•  BetsarecategorizedaccordingtoVegas’esNmateofthehometeam’svictory

probability

BetsonHomeTeam

BetsonAwayTeam

3.Analysis

SomeNmesyoujustgetlucky!•  Themodel’sresults,presentedoverthelastfewslides,lookpreTygood,butthereisalwaysachancethatamonkeythrowingdartsatanNBAschedulecoulddoaswell.

Isthatwhathappened?•  Tofindouthowwellyoucouldexpecttodowithapproximatelythesamenumberofbets,but

chosenrandomlyinsteadofbythemodel,Icreated1000setsofrandombets.–  Onaveragetheylost$500.ThismakessensebecauseVegastakestransacNoncostsaliTleover4%.–  4%transacNoncost*$100*124bets=$496–  Theydidhavehugevariance,withastandarddeviaNonofalmost$1500–  So,assumingthatthemodeldoesNOThaveexplanatorypowerandtheresultswererandom,itsprofitof

$2644is2.1standarddeviaNonsabovethemean,andturnsouttofallinthe97thpercenNle–  Inotherwords,it’spossiblebutunlikelythattheperformancecanbeexplaineden'relybyluck

Histogramof1000randombetseasonsFrequency(y)vsProfit(x)ProfitshownistheboTomofthebucket

Timeseriesof50randombetseasons

OtherPossibleExplanaNons

•  ThemodelwasalmostcertainlybeTerthanrandombeLng–butdiditjustluckilybetontheunderdogorfavoritealot?

•  Whataboutcertainbet-winningteams?

FavoritesandUnderdogs•  OntheBreakdownsslide,it’spreTyclearthatalotofthemodel’sprofitscamefrombeLngon

underdogs.So,didithappentodothisinanamazingyearforunderdogs?•  ThechartsbelowshowwhattheprofitwouldhavebeenlikebeLngontheunderdogorfavoritein

everygame.•  UnderdogsdidmildlybeTer.Butsayingthiswouldnothavebeenagoodstrategyisan

understatement.•  ...waitasec,whyaretheyBOTHnegaNve?

–  1,157games*2betseach*4%transacNoncost*$100=$9256–  AllUnderdogsProfit:-$4476–  AllFavoritesProfit:-$4962–  TotalProfit:-$9438

ProminentTeams•  ThemodelmadealiTleoverhalfofitsprofitsbybeLngagainstHouston20NmesandonCharloTe

15Nmes(actually13Nmesfor,twiceagainst).•  Wasthisthesourceoftheluck?

–  Bothofthesewouldhavebeendecentbetsrepeatedoverandoverallseason,thoughwithHoustonthegainscameallatonceinthebeginning.

–  WithregardtoHouston,themodelcapturedthreeofthefourbigupsetsthatmadeuptheearlyseasongains.BytheendItmanaged$816ingainsin20bets,beaNngtheAlwaysBetAgainststrategybyaboutathird.

–  AsforCharloTe,themodelgained$653,out-earningtheAlwaysBetOnstrategywhileusingmanyfewerbets.

Profitsfrom73betsonCharloTeProfitsfrom73betsagainstHouston

Othergoodbets?

74betsontheWarriors.Notbad! ButnotnearlyasgoodasbeLngontheNets...

Conclusion

•  TesNngtheNBAneuralnetinareal-lifesituaNonresultedinasuccessfulbeLngseason!

•  Whiletheperformancemayormaynotbereplicable,itisunlikelythatitwasduetoluck.

•  BeLngisverystressful.•  Approximately60%rangeofmoNonrecoveredinlegwrist.