optimization of supercuts using on-off data

19/04/23 David Paneque, MPI Muenchen 1

Optimization of SUPERCUTS Optimization of SUPERCUTS using ON-OFF datausing ON-OFF data

OUTLINEOUTLINE

1- Basic algorithm1- Basic algorithm

2- Implementation in the MARS environment 2- Implementation in the MARS environment

3- Results obtained for Mkn 421 Data (Feb 15)3- Results obtained for Mkn 421 Data (Feb 15)

3.1 Static Cuts3.1 Static Cuts

3.2 Dynamic cuts (only in SIZE so far…) 3.2 Dynamic cuts (only in SIZE so far…)

4- Results obtained for Crab data (Jan 27th so 4- Results obtained for Crab data (Jan 27th so

far…)far…)


1- Basic algorithm1- Basic algorithm

1) Application of INITIAL SUPERCUTS values to the data

2) Compute variable to estimate the goodness of these set of CUTS

Significance, Nex, Q factor, F(Significance, Nex, Q)…

3) Modify CUTS

4) Apply new set of CUTS to the data

5) Compute variable to estimate the goodness of the CUTS and compare with the previous one

6) Repeat points 3,4,5 till the estimator reaches a maximum


2- Implementation in the MARS 2- Implementation in the MARS environmentenvironment

Done by Wolfgang half a year ago (only ON data used)

New classes done by myself to use ON-OFF data (finished at mid February)

Basic tools (polynomial fits, Minuit interface for the minimization… ) are the same, and were/are working reliably.

2.1- Introduction 2.1- Introduction

A small set of new functionalities were added

Possibility to optimize in different bins in zenith angle, and combine (a posteriori) the results

Storage of all info (alpha plots, normalization factors, shower parameters, cuts… ) in a root file

Usage or static/dynamical cuts (use/not use theta, dist…)


2- Implementation in the MARS 2- Implementation in the MARS environmentenvironment2.2- Working principle 2.2- Working principle

1) INITIAL SET of CUTS are given (by user) and stored in container MSupercuts

2) Cuts are computed (in case of dynamical cuts) and applied to data through class MSupercutsCalc

Within class MFindSupercutsONOFF

3) Significance is computed using class MHFindSignificanceONOFF

LiMa (1984) formula 17 used

= Non/Noff (about 2.5 for Mkn421) strictly mathematically correct

Error in Noff is obtained from fit, and is usually smaller than sqrt(Noff). Effective is about 1 strictly mathematically correct


4) Parameters are modified to maximize significance using the class MMinuitInterface

5) Final (optimized) parameters are written into container MSupercuts, and applied to the data producing the alpha plots and computing final significance and Nex

6) Alpha Plots stored in postscript files and inside a root file together with other information (normalization factors, significance, Nex, hillas parameters, cuts…) for a later study

2- Implementation in the MARS 2- Implementation in the MARS environmentenvironment2.2- Working principle 2.2- Working principle

Class MFindSupeructsONOFFThetaloop used to run MFindSupercutsONOFF separately in different ZENITH angle bins (eventually also SIZE bins), and combining results (if desired) at the end.Besides, this is also the class defining names of root data files, histograms were to store info (Signfinicance, Nex…) and saving all those guys into single root file


2- Implementation in the MARS 2- Implementation in the MARS environmentenvironment2.3- How to use these 2.3- How to use these

programs programs

Just by defining some variables in a silly macro…


3- Results obtained for Mkn 421 data 3- Results obtained for Mkn 421 data (Feb15)(Feb15)3.0 - Data preprocessing3.0 - Data preprocessing

Calibration

1) Use of MExtractSignal2 (Slide window method)2) Rejection of bad calibration runs (manually)3) Set conversion factors to ZERO to those pixels for which a) <Q> < 50 ADC counts b) Reduced_Sigma/<Q> 4 sigmas away from the mean away from the mean of the distribution for all pixels (done separately for inner/outer pixels) c) Data is calibrated using the closest calibration run (script written by Robert Wagner)


Image cleaning

1) Blind pixels signals are interpolated (when possible) as explained by Nadia yesterday 2) Cleaning levels set to HIGH values:

4 Sigmas for core and neighbouring pixels Minimum of 4 core pixels

and VERY HIGH values: 6 Sigmas for core pixels 4 Sigmas for neighbouring pixels Minumum of 6 core pixels 3 Rings of neighbouring pixels


Comparison ON-OFF data BEFORE APPLYING CUTS


Noise or real Signal ???

VERY Strong cleaning removes more noise, and therefore images “look cleaner”;

LARGER ACCEPTANCE IN THE CUTS


VERY Strong cleaning also removes islands from hadrons, and hence, decreases the separation power;

LARGER ACCEPTANCE IN THE CUTS ALSO FOR HADRONS

STRONG image cleaning vs VERY STRONG image cleaning

I might be too conservative…Yet I feel better playing with theVERY STRONG CLEANING

6 Sigmas for core pixels4 Sigmas for neighbouring pixelsMinumum of 6 core pixels3 Rings of neighbouring pixels



UNFOLDING of source movement by means of FALSE SOURCE ANALYSIS in sub-samples (6-24) ordered chronologically(Method explained by Daniel yesterday…)

Iterative process, as already explained yesterday

Image cleaning used in 2D analysis was

4 Sigmas for core pixels3 Sigmas for neighbouring pixelsMinumum of 4 core pixels3 Rings of neighbouring pixels


3- Results obtained for Mkn 421 data 3- Results obtained for Mkn 421 data (Feb15)(Feb15)3.1 - Static cuts (3.1 - Static cuts (size cut 3000 phsize cut 3000 ph))

Parameters optimization in TRAIN sample; and tested in TEST sample

Nex = 536 +/- 41 (13.1 Sig)Width = 5.5 +/- 0.4 deg

Nex = 517 +/- 40 (13.0 Sig)Width = 5.1 +/- 0.4 deg



Optimized parameters applied in WHOLE SAMPLE OF DATA


Nex = 1028 +/- 56 (18.2 Sigmas)Width = 5.3 +/- 0.3 deg

Rate = 9.7 +/- 0.5events/min

0.124 < LENGTH < 0.350.055 < WIDTH < 0.1250.60 < DIST < 1.25

L/W > 1.5



Parameters optimization in TRAIN sample; and tested in TEST sample

Nex = 750 +/- 57 (13.1 Sig)Width = 5.4 +/- 0.4 deg

Nex = 821 +/- 57 (14.4 Sig)Width = 5.6 +/- 0.4 deg



Optimized parameters applied in WHOLE SAMPLE OF DATA





L/W > 1.5


3- Results obtained for Mkn 421 data 3- Results obtained for Mkn 421 data (Feb15)(Feb15)3.1 - Dynamical cuts (only in SIZE so far…)3.1 - Dynamical cuts (only in SIZE so far…)

Necessity of INITIAL SET OF DYNAMICAL CUTS

PROBLEM: New MC still does not describe perfectly the data

SOLUTION: Use ON data to “parameterize” SIZE dependence on image parameters (LENGTH, WIDTH, DIST ) of GAMMAS

Usage of ON events that pass the cuts (Hadroness < 0.5) and with an ALPHA < 6 degrees


Ln (SIZE)

LENGTH vs Ln (SIZE)

Slope = 0.034

Hadroness < 0.5 && ALPHA < 6 deg


Ln (SIZE)

LENGTH vs Ln (SIZE)

Slope = 0.053

Hadroness > 0.5


Ln (SIZE)

WIDTH vs Ln (SIZE)

Slope = 0.012



Ln (SIZE)

WIDTH vs Ln (SIZE)

Slope = 0.041

Hadroness > 0.5


Ln (SIZE)

DIST vs Ln (SIZE)

Slope = 0.059



Ln (SIZE)

DIST vs Ln (SIZE)

Slope = 0.21

Hadroness > 0.5


3- Results obtained for Mkn 421 data 3- Results obtained for Mkn 421 data (Feb15)(Feb15)3.1 - Dynamical cuts (only in SIZE so far…)3.1 - Dynamical cuts (only in SIZE so far…)

Necessity of INITIAL SET OF DYNAMICAL CUTS

PROBLEM: New MC still does not describe perfectly the data

SOLUTION: Use ON data to “parameterize” SIZE dependence on image parameters (LENGTH, WIDTH, DIST ) of GAMMAS

Usage of ON events that pass the cuts (Hadroness < 0.5) and with an ALPHA < 6 degrees

Dynamical CUT_i= Static CUT_i + a LnS + b (LnS)2

LnS = Ln (SIZE) - Ln (SIZE_OFFSET)

Parameterization of the DYNAMICAL CUTS

Static cut is fixed to the value obtained before


3- Results obtained for Mkn 421 data 3- Results obtained for Mkn 421 data (Feb15)(Feb15)3.1 - Dynamical cuts (3.1 - Dynamical cuts (size cut 3000 size cut 3000

phph)) Parameters optimization in TRAIN sample; and tested in TEST sample

Nex = 514 +/- 34 (15.1 Sig)Width = 5.3 +/- 0.4 deg

Nex = 548 +/- 35 (15.6 Sig)Width = 4.7 +/- 0.3 deg



phph)) Optimized parameters applied in WHOLE SAMPLE OF DATA




Nex = 759 +/- 42 (17.9 Sig)Width = 5.3 +/- 0.3 deg

Nex = 710 +/- 43 (16.6 Sig)Width = 5.6 +/- 0.4 deg



phph)) Optimized parameters applied in WHOLE SAMPLE OF DATA



phph)) Nice alpha plot when allowing “static cut” to be modified…




Image cleaning used here was not that strong

4 Sigmas for core pixels4 Sigmas for neighbouring pixelsMinumum of 4 core pixels




Nex = 440 +/- 22 (20.0 Sig)Width = 4.9 +/- 0.3 deg

Nex = 401 +/- 26 (15.5 Sig)Width = 5.7 +/- 0.4 deg


3- Results obtained for Crab data (Jan27)3- Results obtained for Crab data (Jan27)3.2 - Static cuts (3.2 - Static cuts (size cut 3000 phsize cut 3000 ph))

No OPTIMIZATION; application of optimized (static) parameters with Mkn421

Estimated position for Crab (camera coordinates): X = 0.05 Y = - 0.20





L/W > 1.5


3- Results obtained for Crab data (Jan27)3- Results obtained for Crab data (Jan27)3.2 - Dyn cuts (3.2 - Dyn cuts (size cut 3000 phsize cut 3000 ph))

No OPTIMIZATION; application of optimized (dynamic) parameters with Mkn421


3- Results obtained for Crab data (Jan27)3- Results obtained for Crab data (Jan27)3.2 - Dyn cuts (3.2 - Dyn cuts (size cut 2000 phsize cut 2000 ph))

No OPTIMIZATION; application of optimized (dynamic) parameters with Mkn421


Status of the Status of the detections detections Crab from 27 Jan needs some more work to understand why

signal Is so wide… are we really catching the position of Crab in the Camera ?? Crab from 15th Feb; need some time to “unfold” its movement in the camera… and then to analyze it…

Mkn 421 movement in the camera has been (up to some extent) “unfolded” and cuts (static and dynamic in size) successfully applied. Dependence with DIST parameter will be included soon. Very clear detection. It ”showed” us many problems in the telescope

Telescope pointing and/or AMCPMTs Gain oscillationsSignal jitter in FADC slices“Strange” SIZE distribution (High/low gain??)

Through the analysis of these strong signals we can improve the telescope performance


Policy to commit software to CVS Policy to commit software to CVS should be discussedshould be discussed

Quite some people are working with software not committed to the CVS. WHY ?

My experience: I was not allowed to commit into CVS classes to apply SUPERCUTSusing ON-OFF data because1) Names were not appropriate

2) Not fully object oriented programmed; there is code which is partly existing already in CVS

3) Software committed to CVS should be such that future

updates (ECO 1000) are possible



In my opinion, there are more important criteria that should be considered when committing things to the CVS

1) Code MUST compile

2) New code (changes in the code) MUST NOT AFFECT functionality of other classes

3) New code MUST produce RELIABLE (up to some extent) results. Usually people use it without knowing how it works.

It is worth to spend some time “playing” with the code before committing.

Before changing code, CONTACT author of code and authors of classes using such code to discuss impact of modifications

4) New code should be easy to read and be uded, even for not C++ experts….



In order to increase efficiency and reliability of our software, CLEAR and OBJECTIVE rules should be defined and accomplish by everybody.

This is a good place and a good moment to discuss about it…

optimization of supercuts using on-off data

Documents

data feb153

crab data

initial set of cuts

new set of cuts

static cuts

dynamic cuts

mpi muenchen2 implementation

data preprocessing calibration