yarn مروری بر

of 12/12
ی بر ور مرYARN ی رضای ناصر[email protected]

Post on 10-Feb-2017

219 views

Category:

Data & Analytics

10 download

Embed Size (px)

TRANSCRIPT

  • YARN

    [email protected]

  • YARN(Yet Another Resource Negotiator) .

    YARN API .

    API . API YARN

    .

    Pig Hive Crunch: . YARN ... MapReduce Spark Tez

  • YARN

    YARN 1

    . . (: )

    Container. container node : node virtual

    ... CPU container . .

    1daemon

  • YARN

  • YARN

    application master ResourceManager -1.

    2-ResourceManager NodeManager application master container .

    3-application master application container .ResourceManager container

    4-application master .

    YARN (client master process ) .

  • YARN

    YARN (.Spark Tez )

    HBase .

    Apache Slider : YARN .

    Apache Twill : Slider YARN .

    distributed shell . YARN YARN

  • YARN

    .

    YARN .

    : FIFO : .

    Capacity Scheduler : job .

    Fair Scheduler : .

    . ( Fair)

  • YARN

  • Capacity Scheduler

    Queue elasticity : job job .

    :

  • Capacity Scheduler

    : XML

    .

    . dev

    . prod

    . dev

    dev

    (queue elasticity.)

    . science eng

  • Fair sharing

    Fair sharing (preemption ) . container .( kill)

  • White, Tom. Hadoop: The definitive guide, 4th edition. " O'Reilly Media, Inc.", 2015.