week 14 lecture: advanced python programming, concepetsand...

Download Week 14 Lecture: Advanced Python Programming, Concepetsand ...web.clas.ufl.edu/users/forrest/gis/lectures/2013-11-19 - Advanced...Week 14 Lecture: Advanced Python Programming, Concepetsand

Post on 29-Apr-2018

213 views

Category:

Documents

1 download

Embed Size (px)

TRANSCRIPT

  • Week14Lecture:AdvancedPythonProgramming,

    Concepets andExamplesIntroductiontoProgrammingforGIS&RemoteSensing

    GEO69381469GEO4938147A

  • Recap

    FromPastLectures

    Lastweek:

    Packagemanagem

    enta

    ndPytho

    nextension

    Python

    PackageIndexpipandpipWin

    Exam

    plesofu

    singPython

    packages

  • ThisWeek:AdvancedTopics

    Ho

    wdowetakeallweknow

    and

    gofurthe

    r?Keep

    ingitspecifictoPytho

    n:Parallelprocessing

    Ad

    vanced

    plottingan

    dscientificcompu

    ting

    SpatialdatainPytho

    n

  • Basic

    sofP

    arallelProcessing

    Multip

    lecores==workers

    Ho

    wdowespread

    workam

    ongthem

    ?M

    ultithreaded

    orparallelprocesses

    InPytho

    n,th

    esim

    plesttechn

    ique

    :import multiprocessing

  • Exam

    ple1:M

    ultip

    rocessing

    Ope

    nthePython

    filelocatedinth

    isweeks

    /srcfolder

    Notethatweareon

    lyusin

    gbasePytho

    npackages

  • Multip

    rocessingCo

    ntd.

    Wecansend

    datatoand

    receivedataback

    from

    individu

    alworkers

    Tosh

    aredatabetweenworkersyou

    can

    use

    threadandprocesss

    afecom

    mun

    ication:

    Themultip

    rocessing.Que

    ueob

    ject

    Theorde

    rinwhichth

    eite

    msa

    rese

    ntto

    workersdoe

    snotcorrespon

    dtoth

    eorde

    ryou

    getthe

    mbackwhe

    nusingapply_async()

    Ifitmatters,usePool.map() instead

  • Multip

    rocessingisCo

    mplex

    Thisisbarelysc

    ratching

    thesurfaceofm

    ulti

    threaded

    orp

    arallelprocesses

    Piping,coresvs.th

    reads,etc.

    Startup

    and

    overheadcostin

    CPU

    processing

    Learnmore:

    http://docs.python

    .org/2/library/multip

    rocessing.html

    Multip

    rocessingandArcG

    IS:

    http://blogs.esri.com

    /esri/a

    rcgis/20

    11/08/29

    /multip

    rocessing/

    http://docs.python.org/2/library/multiprocessing.htmlhttp://blogs.esri.com/esri/arcgis/2011/08/29/multiprocessing/

  • ScientificCo

    mpu

    tinginPytho

    n

    Movingaw

    ayfrom

    ArcGISandtowardmore

    gene

    ralscientificcom

    putin

    g

    Manypackagesgoo

    dforscientificcom

    putin

    gOften

    platfo

    rmsp

    ecifc

    Be

    stwayto

    getth

    em?

    Prebu

    iltplatfo

    rm

  • MySuggestio

    inforP

    ytho

    nbased

    ScientificCo

    mpu

    ting

    An

    acon

    dahttps://store.continuu

    m.io/cshop

    /anacond

    a/Itsfree,based

    onallope

    nsourceso

    ftware

    Includ

    esPytho

    n,num

    py,m

    atplotlib,scipy,scikit,

    etc.

    Ha

    sitsownpackagemanagem

    entcalledcond

    awhichalso

    includ

    esaC++

    distrib

    utable:

    e.g.atcom

    mandprom

    pt:conda

    install matplotlib

    https://store.continuum.io/cshop/anaconda/

  • Exam

    ples25,A

    nacond

    a

    Wewilluseth

    eMinicon

    dadistrib

    utionof

    Anacon

    da(fou

    ndin

    /libofyou

    rassignm

    ent)

    andillustratetheuseofth

    econd

    apackage

    managem

    entsystem

    http://rep

    o.continuu

    m.io/m

    inicon

    da/

    Whichinclud

    espipfo

    rnon

    con

    dapackages:

    conda

    install pip

    http://repo.continuum.io/miniconda/

  • InstallingCo

    ndaPackagesisEasy

    From

    thecommandprom

    pt:

  • ForO

    urExamples:

    condainstall scipy

    condainstall matplotlib

    condainstall pandas

    condainstall pip

    condainstall seaborn

    condainstall husl

    condainstall statsmodels

    condainstall moss

    condainstall scikit-learn

    condainstall bokeh

  • Exam

    ple2Seabo

    rnPlottin

    g

    http://stanford.ed

    u/~m

    waskom/softw

    are/seaborn/index.html

    http://stanford.edu/~mwaskom/software/seaborn/index.html

  • Exam

    ple35B

    okeh

    Plottin

    g

    http://bokeh

    .pydata.org/index.html

    http://bokeh.pydata.org/index.html

  • Exam

    ple67Plottin

    gMapsa

    nd

    SpatialD

    atainPytho

    nUsin

    gtheBa

    semap

    extensiontom

    atplotlib

    http://m

    atplotlib.org/basem

    ap/users/examples.htm

    l

    Installbinaries:

    http://sou

    rceforge.net/projects/matplotlib/files/matplotlib

    too

    lkits/

    http://matplotlib.org/basemap/users/examples.htmlhttp://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/

  • ReadingandWritingShapefilesin

    Python

    pyshp

    with

    outa

    rcpy!

    https://code

    .goo

    gle.com/p/pyshp

    /

    https://code.google.com/p/pyshp/

  • pyshpInterfaceforS

    hape

    file

  • Creatin

    gShapefilesF

    romScratch

  • Thisisn

    ttheen

    d

    ifyou

    haveanyqu

    estio

    nsletsta

    lkabo

    ut

    them

    nextw

    eek!

    Nospecificlectureorto

    picswillbecovered

    unlessyou

    propo

    seth

    emTimetoworkon

    you

    rprojects

    Timetoadd

    resscon

    cernsw

    ithm

    e

    Week 14 Lecture:Advanced Python Programming, Concepets and ExamplesIntroduction to Programming for GIS & Remote Sensing GEO6938-1469GEO4938-147ARe-cap From Past LecturesThis Week: Advanced TopicsBasics of Parallel ProcessingSlide Number 5Example 1: MultiprocessingMultiprocessing Contd.Multiprocessing is ComplexScientific Computing in PythonMy Suggestioin for Python-based Scientific ComputingExamples 2-5, AnacondaInstalling Conda Packages is EasyFor Our Examples:Example 2 - Seaborn PlottingExample 3-5 - Bokeh PlottingExample 6-7 Plotting Maps and Spatial Data in PythonReading and Writing Shapefiles in Python - pyshppyshp Interface for ShapefileCreating Shapefiles From ScratchThis isnt the end

    /ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /CropGrayImages true /GrayImageMinResolution 300 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /CropMonoImages true /MonoImageMinResolution 1200 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile () /PDFXOutputConditionIdentifier () /PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped /False

    /CreateJDFFile false /Description > /Namespace [ (Adobe) (Common) (1.0) ] /OtherNamespaces [ > /FormElements false /GenerateStructure false /IncludeBookmarks false /IncludeHyperlinks false /IncludeInteractive false /IncludeLayers false /IncludeProfiles false /MultimediaHandling /UseObjectSettings /Namespace [ (Adobe) (CreativeSuite) (2.0) ] /PDFXOutputIntentProfileSelector /DocumentCMYK /PreserveEditing true /UntaggedCMYKHandling /LeaveUntagged /UntaggedRGBHandling /UseDocumentProfile /UseDocumentBleed false >> ]>> setdistillerparams> setpagedevice

Recommended

View more >