Customer Sharing: KKStream/KKBOX - How do we, as KKStream/KKBOX, deal with seamless service migration/upgrade with AWS Enterprise Support and IEM

Download Customer Sharing: KKStream/KKBOX - How do we, as KKStream/KKBOX, deal with seamless service migration/upgrade with AWS Enterprise Support and IEM

Post on 15-Apr-2017




6 download

Embed Size (px)


<ul><li><p> 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.</p><p> (Drake), KKStream / KKBOX</p><p>May 20th, 2016</p><p>Seamless service migration with </p><p>AWS Enterprise Support</p></li><li><p>Who am I? Who are We?</p><p> Technical Director, KKStream.</p><p> Committee member, PyCon TW</p><p> Spin off from KKBOX on March.</p><p> Media streaming solutions.</p></li><li><p>Proclaim #1</p><p>I would like to talk about our journey, starting from </p><p>migration of video streaming services from on-premises to </p><p>cloud. What we experienced and learnt with AWS </p><p>Enterprise Support team would be shared, and then our </p><p>future strategy with AWS.</p><p>No any deep-dives, or even codes gets mentioned on this </p><p>stage.</p></li><li><p>Proclaim #2</p><p>What Im going to share is our stories in some context. </p><p>They might not fit to your business challenges, and there </p><p>might be even something subtly wrong. </p><p>Listen carefully with doubt is necessary; adapt them to your </p><p>challenges with twist is suggested.</p></li><li><p>Lets start from the result</p></li><li><p>Videopass spec</p><p> Video-on-demand streaming service in Japan.</p><p> Modern adaptive streaming: HLS, DASH, MSS.</p><p> Multi-platform: Android, iOS, Web, STB and TV.</p><p> Seamless migration w/o down time from the previous </p><p>not-that-ok platform.</p></li><li><p>Videopass status</p><p> More than 1M paid subscriptions (Japan only)</p><p> 60,000 videos encoded</p><p> 3,000 films encoded in one day (peak)</p><p> Couple of million video-hours are watched</p><p> EC2 instances: 4,000 cores (peak)</p><p> Request per month: 3 billion (peak)</p></li><li><p>Embraced AWS services</p><p> EC2</p><p> S3</p><p> RDS</p><p> VPC</p><p> ELB</p><p> Route 53</p><p> CloudWatch</p><p> AWS Config</p><p> Elasticsearch</p><p> ElastiCache</p><p> SQS</p><p> SNS</p><p> SES</p><p> SWF</p><p> Lambda</p><p> API Gateway</p><p> DynamoDB</p><p> EMR</p><p> Redshift</p></li><li><p>All on cloud from day oneApril 2014</p></li><li><p>Content Pipeline</p><p> meta, images, videos, and </p><p>others, a variety of contents </p><p>we are dealing with.</p><p> Different and even evolving</p><p>workflows, as DAGs, </p><p>comprise the core backbone.</p><p> It should be robust and </p><p>accountable as we just like </p><p>put 1.5 (video) engineer.</p></li><li><p>Encoding System</p><p> Amazon Simple Workflow (SWF)</p><p> It is accountable, extendable, error-proofing</p><p> We proposed priority queuing feature with others</p><p> Raising soft limits is easy</p><p> Orchestrating the execution of workflows for media </p><p>streaming service and even more</p><p> PyCon APAC 2015</p><p> PyCon Taiwan 2016</p></li><li><p>Counting our core pipeline technology on </p><p>AWS can be robust and secured, even </p><p>though it (ex, SWF) is not that mentioned </p><p>here.</p><p>Lesson #1:</p></li><li><p>D-Day, Videopass migrationJuly 2015</p></li><li><p>Todo list</p><p> Seamless migration to 1M users.</p><p> Migration is split into 2 or 3 phases.</p><p> Service status is watched and reported in minutes.</p><p> 4+2 vendors to cooperate through API, data exchange </p><p>and discussion. (Taipei, Tokyo, London)</p><p> Both on-site and remote engineers, operators and even </p><p>SQA are standby. (Taipei, Tokyo)</p></li><li><p>Live status</p><p>Everyone was caring about the live status, which means</p><p>no one was thinking about others!</p></li><li><p>AWS IEM</p><p> One AWS SA noticed </p><p>something risky about RDS!</p><p> He continuously and clearly </p><p>reported his discovery and </p><p>discussed with us.</p><p> Obviously, it turns out he is </p><p>much familiar with underneath </p><p>services than us!</p></li><li><p>On d-day, all people is easier concentrating </p><p>on instant status, while AWS IEM team, as a </p><p>team player with us, help predict the future </p><p>and prevent disasters.</p><p>Lesson #2:</p></li><li><p>AWS re:Invent 2015October 2015</p></li><li><p>After AWS re:Invent, We Thought</p><p> Cost optimization:</p><p>- development faster, then deployment</p><p>- quit/change even faster if necessary</p><p> Brainless scalability (PaaS):</p><p>- serverless microservices</p><p>- Lambda + API Gateway</p><p> Deeper support:</p><p>- continuous mutual cooperation</p><p>- more KKBOX staffs to AWS re:Invent!</p></li><li><p></p><p></p></li><li><p>We could leverage AWS resources to not </p><p>just evangelize technology and mindset, but </p><p>also cultivate our next-gen platform and </p><p>even broaden our exposure.</p><p>Lesson #3:</p></li><li><p>Moving on to the nextJanuary 2016</p></li><li><p>What is the next-gen streaming big thing?</p><p> Much complicated business logics</p><p> FVOD, SVOD, TVOD, AVOD, bundle sale, </p><p> Higher demand of streaming experience</p><p> Viewing quality, picking w/o searching, personalization, </p><p> Different but rich content</p><p> Live music, linear TV, enterprise education, </p><p> Insane operation situation</p><p> Massive ingestion, dynamic layout, auto-subtitle, auto-</p><p>screenshot, </p></li><li><p>Our strategy for the next</p><p> We just like to have 1.5 AWS expert if possible, or 1 </p><p>expert with some intermediates.</p><p> We would focus on core part of next-gen services while </p><p>infrastructure issues are delegated to someone trustful, </p><p>like AWS.</p></li><li><p>Serverless APIs on AWS!</p></li><li><p>Lots of trickys in AWS services</p><p> We might be one of pioneering teams here trying to build </p><p>fully serverless APIs through API Gateway, Lambda, </p><p> DynamoDB is really expensive if you use it in this way </p><p> Lambda has cold start issue such that </p><p> Not just CPU, but also network and disk performance </p><p>are all positively related to memory in Lambda </p><p> Dont use CodeCommit or CodePipeline because </p></li><li><p>We realized that we wont be that expertized </p><p>in some AWS tricks, but we can aggressively </p><p>start some plan because AWS ES team </p><p>would fully and honestly support us.</p><p>Lesson #4:</p></li><li><p>And we want more from AWSMarch 2016</p></li><li><p>How about putting # of AWS support cases into our KPI? interesting feedback</p></li><li><p>Then, all our team, including non-</p><p>developers, video content quality </p><p>assurers in Kaohsiung, has access to </p><p>AWS Support Center!</p></li><li><p>Treating AWS as a team player in KKStream </p><p>can strongly increase our expertise, team </p><p>size, and even surprising feedback. Its like </p><p>we hired the whole AWS ES team in!</p><p>Lesson #5:</p></li><li><p>Overall, the takeaways</p></li><li><p>Takeaways</p><p> Before evaluating new AWS services, ask them directly. </p><p>They can always feedback something inspiring, trustful, </p><p>and accountable.</p><p> AWS Support could be treated like part of your team, </p><p>instead of just tech Q&amp;A vendor or consultant.</p><p> Thinking about what your team like to be and focus on </p><p>that, then leveraging AWS resources for other parts.</p></li><li><p>Have I mentioned that we are hiring?</p></li><li><p>Thank you!@drakeguan</p></li></ul>