Interactive Data Language (IDL) Margit Haberreiter ( LASP, room 135 Acknowledgement: Marty Snow.

Download Interactive Data Language (IDL) Margit Haberreiter ( LASP, room 135 Acknowledgement: Marty Snow.

Post on 27-Dec-2015




0 download


  • Interactive Data Language (IDL)Margit Haberreiter (, room 135

    Acknowledgement: Marty Snow

  • OutlineWhat is IDL?Basic SyntaxFunctions, Procedures, and LibraryReading & WritingPlotting & PrintingImage ProcessingAnd Stuff.

  • Pro testSave it as filename


  • Interactive Data LanguageIDL was born here at LASPVery popular among scientists:Handles arrays of data easily (spectra, time series, images, etc.)Easy to learnPortable Large existing user library Online Help type ? at promptType help at prompt for contents of variablesInteractive or Compiled, your choice!

  • Data TypesInteger (and Long Integer)Floating Point (and Double Precision)StringStructurePointerAlso, complex, unsigned integers, etc.IDL is dynamically typed which means a variable can be defined as one type, and then be changed to another within the same program unit.

  • Scalars, Vectors, Arrays Problem 1Scalar 42a=5 & b=0. & c=0.dhelp!Vector [1,2,3,5][red,blue,happy,Burma Shave]ArrayUp to 8 dimensions

    Definition of arraysarr1=intarr(3) arr2=fltarr(2,a) arr3=dblarr(a,b,c,d)arr4=strarr(11)

  • Structures - Problem 2Can hold mixed data typesS={day:0,time:0d0,label: } timeCan make arrays of structuresS2=replicate({time:0d0,label:,values:fltarr(100)},n_obs)S2[4].values=findgen(100)Names or NumbersS2.time is equivalent to S2.(0)

  • Rules of the RoadAlways start counting at zero.For help at any time, type help.Case Insensitive (EXC: filenames).Odd is true, Even or Zero is false. An array of length 1 is not the same as a scalar. But you can reform it.

  • Squiggly or Square?Subscripting variables -- [] squareParameters of a function () roundDefinition of a structure {} curlyIDL is lax about enforcing these rules, but it is to your benefit to follow them.

    Example: a=exp[5] ;6th element of variable expa=exp(5) ;e^5

    You might have named a variable the same name as a library function!

  • Subscripting - Problem 3z=a[5]z=a[0:2]z=a[3:*]mask=[5,6,7,8,9] & z=a[mask]Range=where(mask gt 2 and mask le 4)Mask2=mask(range)a=indgen(10)z=a[5]z=a[0:2]z=a[3:*]mask=[5,6,7,8,9] & z=a[mask]Range=where((mask gt 6) and (mask le 8))Mask2=mask(range)

  • The Size of ArraysSize(x)n_elements returns the number of elements in an array (vector, scalar).An undefined variable has zero elements.

    Often takes the place of size().Used to check if user has supplied parametersUsed to index loops.

  • Basic IDL Syntax Problem 4a=Hello World print,a ;This is a commenta=5;dynamic data typesb=4.5*findgen(100) z=a+$ ;continuationsqrt(b)

  • OperatorsArithmetic+ -*/^modMin/Max< (a (a>b is the greater)Matrix Multiply###Boolean LogicandnotorxorRelational Operatorseq (a=b)ge (ab) gt (ab)le (ab)lt (ab)ne (ab)Note that you cant use any of these letter combinations as variable names!

  • FOR loopfor variable=init,limit[,increment] do statementfor i=0,100 do print,i,vector[i]

    for variable=init,limit[,increment] do beginstatementsendfor

    Loops in IDL execute slowly (relatively speaking), so try to use something else, like where.

  • As IF. Problem 5if expression then statementif expression then statement else statement2

    if expression then beginstatementsendif else beginstatementsendelsefor x=0,5 do beginif x gt 5 then beginz=dblarr(10,10)z=z+.1z[3,2]=1.endif else beginz=dblarr(10)z[x]=!piendelseendforGood programming style avoid hardcoded numbers in procedures

  • meanWHILEwhile expression do statement

    while expression do beginstatementsendwhile

  • just in CASEcase expression ofexpression1: statementexpression2: beginstatementsendexpression3: statementelse: statementendcaseExample:x=3 CASE x OF 1: PRINT, 'one' 2: PRINT, 'two' 3: PRINT, 'three' 4: PRINT, 'four' ENDCASE

  • Other Flow Control StatementsSTOP excellent way to debug (.con or .c to continue)GOTO goto,labellabel:SWITCHCONTINUE BREAKMESSAGE stops program and prints message

    IDL Help: This example illustrates how, unlike CASE, SWITCH executes the first matching statement and any following statements in the SWITCH block: x=2 SWITCH x OF 1: PRINT, 'one' 2: PRINT, 'two' 3: PRINT, 'three' 4: PRINT, 'four' ENDSWITCH

  • Procedures and FunctionsA list of IDL statements that could have been entered at the command prompt.Contained in separate file or files.Compiled automatically if files are properly named. Filename matches procedure with a .pro extension. USE LOWER CASE!All procedures compiled from start of file but stops when procedure name matches file name.Environmental variables define directories that IDL will automatically search. USER LIBRARY

  • Compile and RunType procedure name at command prompt to run it. IDL will search for an appropriately named file and automatically compile and run the procedure..r or .compile will compile the procedure, but not run it.IDL will compile all procedures in file until it hits the one matching the file name, then it will stop. Put the named procedure last in the file.

  • Simple Procedurepro simpleprocedure,a,b,cc=a+bendTo run it, type:


  • Simple Functionfunction eve,numberif number mod 2 eq 0 then return,0return,1endTo run it, type:result=eve(var)print,eve(var)print, The result is , result

  • Whats the difference?Not much.

    Use a function to get a value that youll use right away (like an expression).EXAMPLE: simpleprocedure,eve(0),eve(5),result

    Use a procedure when you return multiple results and dont want to just wrap them together in a structure or array.

  • Parameters and Keywords Problem 7pro proc2,input1,input2,result,doplot=doplot if n_elements(doplot) eq 0 then doplot=0 result=input1*sin(input2/!pi) if doplot gt 0 then begin set_plot,'win' !P.CHARSIZE=3.; system setting !P.MULTI=[0,1,3]; system setting plot,input2,result,psym=-4 plot,input1 plot,sin(input2/!pi) endifendOrder of Parameters is critical.Order of Keywords is irrelevant.

    Keywords can normally be abbreviatedxtit instead of xtitlexs instead of xstyle

  • Variables: Global or Local? Problem 8Variables are local.sort of (Attention!!)A procedure can modify any of its parameters, which will change the variable in the calling procedure.

    pro proc1v=0print,vproc3,vprint,vendpro proc3,var2var2=var2+1end

  • Error and stopIf the IDL interpreter hits a STOP or encounters an error, flow halts at the local level. You can type help to view contents of variables and which line of code youre at.To return to the top level retall (return all the way)To clear everything, type .f (full reset) or .reset -> Attention: all variables deleted

  • Reading & Writing & Savingopenr,openw,openu (Read, Write, Update)readf,readu (Formatted, Unformatted)close or free_lunIDL save files:save,file=data.idl,var1,var2restore,data.idl

    openr,lu,file.txt,/get_lunwhile not eof(lu) do beginreadf,lu,dataendwhilefree_lun,lu

  • Slight Gotchapro wont_workopenr,lu,file.txt,/get_lundata=fltarr(100)for i=0,n_elements(data)-1 do beginreadf,lu,data[i]endforfree_lun,luend

    IDL passes by value, not by reference, so this wont work.

  • Slight Gotchapro will_workopenr,lu,file.txt,/get_lundata=fltarr(100)i=0while not eof(lu) do beginreadf,lu,temporarydata[i]=temporaryi=i+1; counterendwhilefree_lun,ludata=data[0:i-1]end

    IDL passes by value, so this will make IDL happy.

  • Task: Read data from file Problem 9pro willwork,filename,dataopenr,lu,filename,/get_lun ; get_lun: sets file unit numberdata=fltarr(2,1000)i=0while not eof(lu) do begin readf,lu,temporary1,temporary2 data[0,i]=temporary1 data[1,i]=temporary2 i=i+1 ; counterendwhilefree_lun,ludata=data[*,0:i-1]set_plot,'win'!P.MULTI=0plot,data(0,*),data(1,*),title='Data from file'end

    Run the procedureAdd a plot using a parameter (see slide 22)

  • Homework 1LISIRD DataGo to and retrieve solar Lyman alpha (121.5 nm) data for at least two missions.Save data as text file.Write an IDL procedure to read data.

  • More on plottingBasic IDL Plotting ProceduresLine plotsplotoplothistogramContour plotscontourSurface plotssurfaceshade_surf

  • Line em up Problem 10X=FINDGEN(360)Y=SIN(X*!DTOR)PLOT, X, Y, XRANGE=[0,360], /XSTYLE, XTIT=X, $ YTIT=Y, TIT=Sample Line PlotZ=COS(X*!DTOR)OPLOT, X, Z, LINESTYLE=2

  • Adding text to plotX=FINDGEN(360)Y=SIN(X*!DTOR)PLOT, X, Y, XRANGE=[0,360], /XSTYLE, XTIT='X', $ YTIT='Y', TIT='Sample Line Plot'Z=COS(X*!DTOR)OPLOT, X, Z, LINES=2XYOUTS, 100, 0, 'cos(x)'XYOUTS, 190, 0, 'sin(x)'

  • HistogramsY=RANDOMN(SEED, 100, /NORMAL)H=HISTOGRAM(Y, BINSIZE=0.2, LOCATIONS=L)PLOT, L, H, PSYM=10 ; histogram mode

  • Graphics Keywordschange the style of the dataadd a titlemanipulate an axis/change tick markschange the format of textchange coordinate systemsetc.

  • Styles and Symbols KeywordsLinesLINESTYLE={0,1,2,3,4,5}THICK: change the thickness of the line (default is 1.0)SymbolsPSYM={1,2,3,10}SYMSIZE: change the size of the symbolUSERSYM: procedure to define PSYM=8PSYM= -{1,2,3,8}: solid line connecting the symbolsN.B.: PSYM=3 (period) does NOT show up well in postscript plots and does not scale nicely with symsize. Better to use PSYM=1, SYMSIZE=.3 (or similar) Negative value of psym creates line+symbols

    ValueMeaning1Plus sign (+)2Asterisk (*)3Period (.)4Diamond5Triangle6Square7X8User defined9Undefined10Histogram mode

  • Titletown USA KeywordsTITLE: place a string of text at the top of a plotTITLE=This is my titleTITLE_STRING=This is my title TITLE=TITLE_STRINGSUBTITLE: place a title below the x-axis[XYZ]title places title on x, y, or z axis

  • Axes & Tick Mark KeywordsTo manipulate individual axes:Set the range:[XYZ]RANGE=[min, max]Change the axis style[XYZ]STYLE=numberMultiple effects can be achieved by adding values togetherLabel an axis:[XYZ]TITLE=string

    ValueMeaning1Force exact axis range2Extend axis range4Suppress entire axis8Suppress box style axis16Inhibit setting y-axis minimum to zero

  • Axes & Ticks Keywords 2To manipulate individual axes (cont.):Change the axis thickness[XYZ]THICK=number (default is 1.0)Use axis procedure to draw axes with different scalesX=FINDGEN(360)Y=SIN(X*!DTOR) ; system variable /180PLOT, X, Y, XRANGE=[0,360], XSTYLE=9, XTIT='X (Degrees)'AXIS, XAXIS=1, XRANGE=[0,2*!PI], XSTYLE=1, XTITLE='(Radians)'

  • Clipped title

  • Fixed clippingplot,x,y,xstyle=9,xtit='X (degrees)',ymargin=[4,4]

  • Text Formatting KeywordsTo change the size of textCHARSIZE=number (default is 1.0)To change the thickness of the textCHARTHICK=number (default is 1.0)Or set system variable:!P.Charsize=3

  • Coordinate systemsIDL has 3 coordinate systemsDATA (default)Uses the range of values of the dataDEVICEUses the device (X window or Postscript device) coordinates (i.e. pixels)NORMALNormalized coordinates from 0 to 1 over the 3 axes

  • Adding colorColor tablesLOADCT[, number]XLOADCT (widget)Graphics keywords:BACKGROUND=numberCOLOR=numberNumber from 0 to 255, corresponding to the color tablecolor=fsc_color(red) ;using library routineOn Mac, Windows: DEVICE, DECOMPOSED=0

  • ContourX=FINDGEN(360)Z=SIN(X*!DTOR)#COS(2*X*!DTOR)CONTOUR, Z, NLEVELS=6, XRANGE=[0, 360], $ YRANGE=[0, 360], /XSTYLE, /YSTYLE, XTIT='X', YTIT='Y', $ TIT=Sample Contour Plot

  • Surface plots (shaded surf)SHADE_SURF, Z, XRANGE=[0, 360], $ YRANGE=[0, 360], /XSTYLE, /YSTYLE, XTIT='X', $ YTIT='Y', TIT='Sample Surface Plot 2'

  • Colored surfaceloadct,3shade_surf,z,tit='Sample Surface Plot 3',shades=bytscl(z+1)

    Bytscl translates the values of the array into a byte array from 0 to 255

  • Surface plots 2 (wire mesh)SURFACE, Z, XRANGE=[0, 360], $ YRANGE=[0, 360], /XSTYLE, /YSTYLE, XTIT='X', $ YTIT='Y', TIT='Sample Surface Plot 2'

  • Changing appearance of meshz2=rebin(z,180,180)surface,z2z3=rebin(z,90,90)surface,z3The possibilities are endless!

  • Multiple plots on a page!p.multi=[a,b,c]a: number of plots left on pageb: number of columnsc: number of rowsExample: 6 plots per page (3x2), begin plotting in the top left!p.multi=[0,3,2]To return to a single plot per window, !p.multi=0


    For Windows, SET_PLOT, WIN

  • Detailed plotting...set_plot,'ps'tek_color!p.font=1!p.color=0!p.background=1;!p.MULTI=[0,2,4]!P.MULTI=0ang = string("305B); Angstrom symboldevice,filename=fileout+'*.ps',/color,ysize=12, xsize=17,xoffset=1,yoffset=10;*****************************************************************plot,x1,y1,linestyle=1,/ylog,thick=6,$; /nodata does not plot the data tit='',CHARSIZE=1.5, xtit='!Ml!N (nm)',XCHARSIZE=1.5,$ytit='intensity (erg s!E-1!Ncm!E-2!NHz!E-1!N)',YCHARSIZE=1.5,$; !E ->exponent; !I-> index; !N ->normal fontxrange=[x0,x1],xs=1, $; !M -> mathfont, e.g. !Ml ->lambdayrange=[y0,y1],ys=1;*****************************************************************; overplot;*****************************************************************oplot,x2,y2,linestyle=0,thick=3legend_mh01,textarr,charsize=charsize,textcolor=colors,colors=colors,$box=0,position=[.5,.55],/normalxyouts,0.85,0.85,a,charsize=1.5; label of figure panel, i.e. a, b, c, d....print,'Have a look at the plot :)'end

  • Making other types of graphics filesUse tvread function from David Fannings library


  • Putting it all togetherHomework assignment 2Using LISIRD, create a composite Lyman- time series using SORCE, TIMED SEE, UARS SOLSTICE, and SME dataPlot each instrument as a different colorLabel each instrumentX-axis: year Y-axis: W/m2/nm

  • Sample Solution

  • Imaging basicsImages in IDL are just arrays. All standard mathematical operations are available.Images are usually saved in FITS formatHeader and array of integers (image)Use Goddard IDL Astronomy Library procedures to read both the header and imageRaw images must be processed before they can be used for science. Example:Remove the background counts (dark)Correct for variations across the CCD (flatfield)

  • Displaying imagesBasic IDL commands to display an array/imageTV, imageTVSCL, imageResize an imageREBIN(image, xsize, ysize)xsize and ysize must be integer factors of the original size of the imageAdd color by loading a color tableE.g. SOHO EIT has its own color table for each wavelength

  • and Stuffwheresizehistogramstring manipulationfile_searchmap projectionsn_elementsreading file formats (CDF, FITS, PNG, etc.)pointers

  • WhereUse WHERE to find data values meeting some criterion.good=where(data gt 0.)better=where(data gt 1.5 and $time gt julday(1,1,2005))data2=data[good] ;new variable data2data=data[better] ;redefine variable data

  • SizeThe size function returns lots of good information about your variable.(dimensions, n_elements, data type, etc.)


    An undefined variable is a valid type, so check for size or n_elements.

  • HistogramThere are billions and billions of uses for histogram.


  • StringsIDL understands regular expressionsConcatenate strings with +myname=yournameresult=strpos(myname,o',0)result=strupcase(myname)result=strlen(myname)

    strsplit, strmid, .

  • file_searchRetrieves a list of filenames into a string array.list=file_search(path+*.idl)for i=0,n_elements(list)-1 do beginrestore,list[i]plot,time,irradiance,psym=-4endforAll your string manipulation skills can be used to create the list of files, of course.dialog_pickfile is an interactive way of selecting files.

  • Plotting Data on a Mapset up map projection map_set,/continents,/mercator(mercator: cylindrical map projection)plot positional data on itMAP_SET, /MERCATOR, 0, -75, 90, CENTRAL_AZIMUTH=90, $ /ISOTROPIC, LIMIT= [32,-130, 70,-86, -5,-34, -58, -67], $ /GRID, LATDEL=10, LONDEL=10, /CONTINENTS, $ TITLE = 'Transverse Mercator

    Other options: /ROBINSON, /ORTHOGRAPHIC, ?map_set

  • Standard Data FormatsIDL save files: restore, filenameFITS files: d=mrdfits(file,exten,head)netCDF: read_netcdf,file,data,status

  • PointersVariable contains a reference to data in memory, instead of containing the data.Needs to be de-referenced to get the data.get_sorce_telemetry,data,info,jd1,jd2,$externalelement=solstice_a,$itemname=[grat_pos,detector_a,int_time]


  • User Library Routineslegendpausepsymdotwhich or file_whichfsc_colorrainbow

  • Coyotes Guide to The real source for tips and tricks

  • Colors and Plotsxpalette to show color tabletvrd (TV read)to grab graphics window and write to fileadd color definitions to startup fileremember to make plots color-blind friendly if possible (linestyles --- and symbols xxx)

  • File Tipshelp IDLs automatic compiler by always naming your files with lower caseIDL stops compiling when it hits the routine that matches the filename

  • IDL 7 WorkbenchNew version of IDL uses EclipseBackwards compatible as always

  • Tips & Tricks for WorkbenchHover help (and open declaration)Redefine key bindings to whatever youre used to: ctrl-shift-Ldouble click on history entriescntrl-i shifts focus to command linecntrl-space (or alt-slash) for command completion.edit filenamesearch works like grepsearch for content in files

  • Just the beginning.Remember, theres a lot of IDL knowledge around here, so feel free to ask questions.Many user library routines might already do whatever you had in mind.Online help manual has lots of examples.SolarSoft, Widget programming, Object Oriented...


View more >