INTRODUCTIONDevOps is no doubt one of the buzzwords for today’s agile software company culture.

It has been widely adopted in variety of different types of software companies

building yet more different types of products. What DevOps actually mean and what are

the implications of this approach for mobile app development and testing? As defined

by Wikipedia:

“Practices that emphasize the collaboration and communication of both software

developers and other information-technology (IT) professionals while automating the

process of software delivery and infrastructure changes.” –Wikipedia

What drives companies towards this type of DevOps thinking? First of all, there are always

business requirements and demand that drive the adoption of any sort of process or

culture changes within companies. With DevOps approach the apparent benefits for

business are the full adoption into agile methodology, getting things done faster and with

focused mentality on support, operations and services. And when these two are combined

businesses can get both as efficiently as possible.

Although the bar has been set high for mobile apps to be successful, there are lots of

different characters, things that should be considered when adopting DevOps approach for

mobile app development and testing. These requirements are driven by app developers,

business, and most importantly, the end users.

As the beneficiary of DevOps practices, we would like to share what we have learned from

adopting DevOps for mobile projects and what we see is valuable for your mobile teams.

Share this eBook!

Share this eBook!

CHAPTER 1

Deep Dive on DevOps inMobile App Testing

Share this eBook!

M obile is more important than ever. The increasing competition with mobile apps

and user acquisition has pushed companies to act quickly with its development and

operations. Getting that app ‘right’ in the first place, and working properly with a myriad

of different device configurations is a task that should be considered way before mobile

app is published. In this the DevOps culture, process and tools have a significant role.

What is Mobile DevOps?

The DevOps thinking has become an established part of companies and development

teams relying on agile methodologies. The combination of agile methodologies and DevOps

are in essence to provide continuous integration and continuous deployment to speed up

the integration, deployment and testing, plus improve the quality of application. Getting

the mobile app instantly from the build system to real device where application can be

thoroughly tested improves the overall process velocity and makes the bug fixing easier

and faster.

What is Mobile DevOps?

The DevOps thinking has become an established part of companies and development

teams relying on agile methodologies. The combination of agile methodologies and DevOps

are in essence to provide continuous integration and continuous deployment to speed up

the integration, deployment and testing, plus improve the quality of application. Getting

the mobile app instantly from the build system to real device where application can be

thoroughly tested improves the overall process velocity and makes the bug fixing easier

and faster.

Share this eBook!

The DevOps in mobile means incorporating the continuous integration and testing process

tightly with the delivery and deployment, and wrapping up everything in continuous

operations. Developers – and development in general – focus on building the actual

product while relying on their own relevant tools for them. The quality assurance (QA)

is the stakeholder that is interested (and responsible for) in testing those regressions,

performance of the product, and preferably automating as many aspects of the testing flow

as possible. The operations maintain the continuous build, integration, deployment and

delivery environments, and take care of those releases. When combining all these three

different functions into one holistic approach Mobile DevOps can be used as a definition.

3 Rule of Thumbs in Mobile DevOpsMobile DevOps is an approach that focus on communication, integration, deployment,

automation and measurement of collaboration between app developers, testers (QA folks)

and the operations. The whole thing is to acknowledge the interdependence nature of

this approach and rely on the right tools, methods and infrastructure in order to develop,

deliver and test apps faster and more efficiently.

Develop and Test Against Real Environment

The most important thing for mobile developers is that the end-product works across on

all its target devices. Using emulators or simulators are acceptable in the earliest phase

of the development, but when app becomes more sophisticated and have the features

that end-users will see, the only acceptable way is to test mobile apps on real devices.

Communication Integration Deployment

Share this eBook!

Nowadays, it’s not a cost issue and nevertheless organizations spend significant sums in

their hardware infrastructure. It’s well justified to not to try to save in the most important

hardware cost – the real mobile devices – to get the app thoroughly and comprehensively

tested out. As said, not everything should be acquired by organization itself, but using

a cloud service that contains the right set of mobile devices is a perfect fit for today’s

mobile devops toolchain.

Deploy and Test Frequently (Many Times, Every Day)

Agile methodology praises the use of continuous integration and

delivery tools. There are plenty of great open source as well as

commercial tools available that help organizations to build,

deploy, test and release their apps. The most meaningful value-

add to this flow is the mobile test automation that can

accommodate frequent builds to testing context and

enable instant bug hunting.

The continuous development and testing integration

can enable companies to respond to market and

competitive changes in timely manner. Reusing the

code, back-end implementations, APIs and other

infrastructure to build a new version of the mobile

application is easier, as well as testing of that when

infrastructure is in place. The frequent deployment and testing also enhances

ability to automate functional, compatibility and performance testing of the application,

against an array of devices and different form factors.

Continuously Consider and Validate Quality Characteristics

The modern mobile devops is (all) about continuous everything. The DevOps approach

should always strive to improve things, process, culture, and even reconsider what devops

Share this eBook!

tools should be in use. The continuous evaluation of the efficiency and productivity can

get insights to real performance issues and how to make things smoother and work

seamlessly in the next regression.

Changing the devops tools should be considered every now and then, especially if there

are any doubts about the current environment, its capability t o

deliver and work as efficiently as possible. And this doesn’t

apply only for devops tools but also how mobile devices

and device infrastructure is used. The requirements are

constantly moving and that primarily due to demand by

end-users. The quality characteristics of mobile app are

set from the first line of code until it’s on hands of end-

users, and everything that happens between that should

be measured, validated and fixed – if needed.

3 Business-Critical Mobile DevOps ConsiderationsFrom the business perspective, the ability to quickly introduce new app,

features in it, and bug fix releases is one of the most critical tasks for companies

with mobile-first strategy. In essence, the core of DevOps methodology aims at

speeding up the app development, delivery and process by getting developers and

operation specialists to collaborate throughout the end-to-end app development

and deployment process.

Continuous Integration and Continuous Delivery

Continuous integration and delivery can be measured with various different metrics.

However, what are the most important ones for mobile app development and testing

matter the most. For example:

Share this eBook!

• Frequency of deployment and tests executed (against each regression).

• How many new features, lines of code or other integrations have been deployed by

each build.

• What is the actual time between the development (when it started) and test finalization.

• Percentage and total number of failures vs. successful tests.

• Urgency to get mobile app published.

• Performance of automation. Do you need to tweak in something manually?

Now, these metrics are valid for every regression of the app. Each and every time when

source code is modified and new build is deployed, it should be thoroughly tested. With

test automation that’s not a challenge but really a smooth process that gets instant results

without any manual intervention.

Application and Infrastructure Testing and Monitoring

When talking about mobile apps it’s not only about the APK or IPA deployed on device, but

more often there is tremendous number of dependencies to other applications, backends,

and network(s). Developing, deploying and testing in the infrastructure context also

improves insights of how well the entire environment does and how application performs.

In this context, it’s also much easier to drive iterations to make usability, UI and user

experience aspects better: If something lags or performs bad in the system it will have

an impact to all other pieces of the system.

Monitoring mobile apps, websites, APIs and all

other relevant parts is also important. While app

may perform just fine there are other entities

that doesn’t necessarily provide the same level

of performance. Accurate performance analysis

with the real data helps improving the overall

development flow.

Monitor

Share this eBook!

Comprehensive Dev & Test Process and Mobile App Delivery

It’s always important to remember that organizations that use agile methodology should

reduce things that get done manually. Things that get done manually (especially testing)

are never agile. The agile development sprints that include test automation will be faster

and generally the adoption of the right tools, methods and technologies speed up the

development and testing tremendously. Agile development is the epicenter of today’s

mobile app development process, culture, and should be reflected in tools used by

developers.

Share this eBook!

CHAPTER 2

Effective Mobile DevOps Strategy and Typical Goals

Share this eBook!

T he effectiveness of adoption Mobile DevOps depends on your goals and metric

that naturally vary across companies. That being said, in general there are four

characteristics that by maximized can deliver the best outcome for Mobile DevOps

approach and yield as significantly improved results in your mobile app quality.

“The highest priority is to satisfy the customer through early and continuous delivery of

valuable software.” — Agile Manifesto – Principle #1

Overall, Mobile DevOps adopters should strive for four goals. These goals are the

cornerstones in creation of Mobile DevOps strategy and a culture that helps to achieve

the expected quality and faster turnaround to market. Let’s break it down.

Maximize the Delivered Value

The trend – and perhaps motivation towards DevOps adoption for many companies has

been in a more efficient way to get things done. This is typically seen as cost savings and

enables people to get things done faster (and more efficiently). Today, for large enterprises,

DevOps, mobility and mobile apps used by its employees are the value drivers as they

boost productivity, make the daily-work more efficient and drive the business value.

Share this eBook!

For many enterprises maximizing the value from technology investments has changed the

way they spend money on services. As proposed in the article, spending smart but quickly

to deliver results is yet more efficient than just keep spending big with no measurements

on return-on-investment.

For enterprise IT, mobility and mobile apps in general hold the potential to boost their

productivity and drive business value of the entire organization. And to make the most out

of mobile apps, enterprises need to put in place a comprehensive mobility strategy that

can work as a holistic approach, encompassing all aspects from mobile app development

to security and connectivity.

Maximize the Effectiveness, Efficiency and Productivity

Comparing and measuring productivity, efficiency, and effectiveness equals to the faster

time-to-market, speed of actual delivery, deployment and turnaround with testing. And

all of these cumulate as higher quality in mobile apps which means the most crucial

quality factor.

The Mobile DevOps effectiveness, which can be seen as doing the right things, means

that there is a great chance to achieve expected results. But doing the right things doesn’t

mean products will be successful. Doing the right things alone is not sufficient to achieve

to the ultimate success. DevOps strategy needs to consider efficiency side of things as

well to complement the overall outcome.

“ The effort of maximizing the efficiency

and effectiveness can improve

the overall productivity. ”

Share this eBook!

The Mobile DevOps efficiency, which will be seen as doing things right, is the end-result of

the effort with minimal costs, time and overall contribution. From the business perspective

DevOps efficiency can mean the use of minimal resources to accomplish the goals.

The effort of maximizing the efficiency and effectiveness can improve the overall

productivity. What this means in mobile DevOps strategy is that test automation must be

included in the testing of each build of the application. Test automation can provide tons

of benefits from making sure the device compatibility is met, defects/bugs are instantly

spotted out, and those get – with great details of why failures happen – in hands of capable

developers in order to get fixed. This itself can control the cost as earlier fixed problem

doesn’t drag on with the build by build, but also it will cumulate as higher quality. Finally,

as problems are solved earlier and verified against real devices, the overall efficiency is

improved and individual productivity will be higher.

That being said, efficiency without effectiveness doesn’t guarantee the best results. “If

ain’t broken, don’t fix it” means that the DevOps must focus on the right items; if the

Share this eBook!

problem with a mobile app is in general compatibility across device variants, OS versions

and other characteristics, it really doesn’t matter what sort of features are built in. The

distinction between the best app and average app are typically the general compatibility,

great performance, and fully exploited hardware.

To maximize the effectiveness, efficiency and productivity starts with the full adoption of

mobile test automation, with automated, continuous integration, deployment and testing,

that is exercised with the every application regression coming out from the development

pipeline.

Maximize the Quality and Robustness of Apps

The most critical thing for companies building a mobile app is the quality. Indeed, the

process, approach or used methodologies are all secondary if the quality isn’t up-to-snuff

and product cannot be showcased for its excellence and robustness. To meet and exceed

this requirement many enterprises have adopted DevOps culture and many of those that

did it early have been very pleased with the results. Clearly, a well-established and well-

executed DevOps strategy has been the key to the successful adoption.

What are the key tasks or things to focus in the Mobile DevOps approach or Mobile

DevOps tools? First of all, all used DevOps tools must be collaborative that enable the

seamless automation and are technically possible to be integrated with each others.

Non-collaborative DevOps tools can make the entire process and flow challenging and

make the effort of building better quality and more robust apps difficult.

“The most critical thing for companies building a mobile app is the quality.”

Share this eBook!

The Mobile DevOps approach itself creates a culture that helps in achieving those strategic

goals by employing the following aspects:

Collaborative Process, Development, Testing and Release – and Support. The Mobile

DevOps approach requires continuous collaboration between all teams involved, from

developers to testers and support (post-release).

Continuous Integration and Continuous Testing. Regardless of if agile methodology adopted

continuous integration and testing are must to have for any organization building mobile

apps. Using these standard DevOps tools efficiently will have an impact on quality,

robustness and app compatibility across devices.

Continuous Deployment and Continuous Release. The quality process should start as

soon as the first line of code has been written. After each regression and stage until the

final product, the quality concerns should be the most highest priority for Mobile DevOps

to ensure application quality criteria is met and product is deployable and ready for the

release. Furthermore, testing effort should continue even application is published as post-

release issues come up quickly – and developers needs to jump on them.

Share this eBook!

Continuous Monitoring. Performance of mobile application is important in every step of

the product development lifecycle. Getting a comprehensive understanding of how well

application does on different devices, what are the bottlenecks and how application can

be adjusted/optimized to provide better user experience for its users.

Maximize the Support, Service & The Delivery

In agile methodology, support, service or product, and deployment are highly important

to determine whether the delivery can be successful. The Mobile DevOps is constant

alignment and collaboration between different teams, all focusing and aiming for better

quality, and yes – continuously enhancing the process.

For instance, continuous testing as part of the development flow aims to deliver customer-

facing, ready, high-quality applications. While starting to do this when application is still

under development and continue it for all different stages of the product development the

quality criteria can be met. This also has an implication on support and how the product

is treated after by support after it has been published. Including unit testing, UI functional

testing, API testing and all possible integration testing with different entities can ensure

that there will be less surprises while the mobile gets mature.

Finally, effective testing and quality process as part of the Mobile DevOps thinking can

enable much faster turnarounds, ensure that developers and QA folks can do constant

releases with confidence, and minimize risks related to quality concerns.

Share this eBook!

CHAPTER 3

How Mobile DevOps Can Make ADifference

Share this eBook!

T he DevOps approach have had notable differences for enterprises striving to streamline

their IT operations. This positive impact on companies that adopt DevOps (and Mobile

DevOps) approach has enabled them to build collaboration across teams, something that

typically wasn’t that obvious in the past, just be focusing on adopting DevOps practices

with organization. There are lots of great success stories about how DevOps mentality

changes the way companies were doing things, making process yet more faster and

streamlined, and enabling companies to save money.

Despite of many of these panegyrical stories

about the positive impact of DevOps for

enterprises there are also challenges and

other things organizations must tap in. During

the past years, use of agile methodologies

and impact of it for development, operations

and all other parts of the organizations have

been accelerating the DevOps movement

and breaking down those boundaries

between these functions. The question is –

how Mobile DevOps can make a difference

to an organization? And is the distinctive

difference even between regular DevOps

and Mobile DevOps?

Building Mobile DevOps culture isn’t

necessarily easy. Like any process, culture

or behavior changes within organizations

it needs to be implemented properly and

adopted step-by-step to encourage all stakeholders. It simply doesn’t happen overnight,

but modernizing of process, DevOps tools and practices will respond to business demand.

Share this eBook!

How, Why and Where Mobile DevOps Makes A Difference

The discipline for Mobile DevOps requires few important technical and support things to

be completed to make difference for any organization. In order to get all pieces together

and make the Mobile DevOps tools and process seamless for companies the problem

and potential bottlenecks must be identified. These are the top things that came up in

our survey and companies should focus on either fixing or adopting the better Mobile

DevOps tools and process to get these challenges tackled.

(Way) Too Long Dev-Test-Feedback Cycles

Comparing fully adopted Mobile DevOps approach for development, testing, and

deployment the old way of doing things lack a shortcut to reveal, document, and eventually

fix found issues. If the development cycle including building, testing, and deployment

is too long the development progress might advance faster and it’s definitely harder to

keep up.

Even with agile methodology in use, each step in the Mobile DevOps must be carefully

evaluated and measured for efficiency and effectiveness. That eventually makes the

process more productive, delivers value, and maximized the critical business advantages.

If the process is too slow some things can be boosted up, paced up and even eliminated

if those do not have any impact on the results. Mobile test automation can really help in

enabling continuous, frequent and instant releases that can be then quickly integrated,

deployed and testing in automation context, and in timely manner.

Share this eBook!

Bug Catching/Reporting Too Slow to Developers

When the right tools are in use everybody gets notified about problems pretty much

instantly. Then there is only motivational and attitude that prevents developers to jump on

those issues. Getting as rich and detailed results as possible will help developers to solve

issues. You know, developers get the DNA of the wrong behavior, bugs or failures from

log information, so this is critical to be delivered and preferably from as many devices

as possible.

Non-Functional or Not Well Utilized Test Automation

Relying on non-agile methodologies, or not fully adopted agile methodology with

recommended functions as a mandatory can make things very difficult in the testing

phase. Many of failures with test automation has been due to lack of knowledge and skills

to adopt agile test automation for mobile app testing. There are still stories why many

developers see test automation as something great, but just haven’t got real-life cases

to get things running on variety of different devices. Because of this, test automation has

been doomed by those who haven’t got it fully implemented and working. Instead of fully

functional test automation many users have done basic unit/system tests.

Use of Non-Real (Emulators) Test Environment

Testing on emulators gives you something

result-wise. But again, are those results

trustworthy? No end-user will run your

application on emulator that is typically

executed on x86-powered desktop,

with much more memory, with different

network infrastructure, and lack of all critical hardware components (different GPU, screen,

its resolutions & orientation, WiFi components, and many others). There are very good

reasons to stick with real devices and test only using as authentic environment as possible.

Share this eBook!

Setting Up & Maintaining Real Devices

This has been an issue and one remarkable reason why developer organizations haven’t

seen on-premise labs as worth it. Based on the facts, the need for different mobile devices

(with all diverse software and hardware characteristics) has gone significantly up. For

example, when the typical mobile device farm built internally was about 20-50 devices

in 2014, the requirement has already exploded to 500+ devices. That’s significant growth

number when it comes to use of real devices for testing.

Nowadays, getting and setting up that level of device farm is no problem. For example,

our Testdroid PrivateCloud caters mobile app developers with any number of devices

team must test their app on. Depending on how many devices are reserved and dedicated

for customer comes from the market demand, and for what purpose is the application

targeted for. Typically, the answer for this is “globally”.

Share this eBook!

Access to Relevant Mobile Platform / OS version / Devices

Fragmentation is not a new phenomena for either major OS platforms – Android and iOS

– and devices. There are lots of users that are passionate to upgrade to the latest one –

and the other half who are just happy how things currently work on the device. In the

middle of this, there are also users that have upgraded their OS version at some point but

do not want to go the latest or simply their device OEM doesn’t provide upgrade for that

specific device model. That’s very common and is also known as device/user/platform

fragmentation.

By defining what is a relevant OS version, as well as finding out devices that people use,

can be typically a challenging task. Companies building professional apps may need to

investigate both Android Dashboard and Apple’s iOS Dashboard to get better understanding

what OS versions are actually used.

Use of Manual Testing – The Weakest Link in Agile and DevOps Process

Manual testing is still (by far) the most used way to test anything on mobile. And that’s the

biggest resource hog. Sinking effort and resources to manual doing while there are awesome

test automation to help is a bad choice. Organizations understand this and want to build up that

test automation competence to get things moving faster, cheaper and getting better results.

Share this eBook!

No Geographically Relevant Testbed – Devices, Networks, Back-End Integration

The target market matters you the most. If (and typically yes) app developers are targeting

their app globally as potential with global app market is simply huge. Instantly, there are

hundreds of millions of potential users for the application, game or web product – and

making sure it will work well on all possible devices these users use.

According to the results of our survey, there are three major things that app developers

are concerned about: 1) devices, 2) networks, and 3) back-end.

Share this eBook!

CHAPTER 4

5 Critical Daily Tasks for Mobile DevOps

Share this eBook!

M any companies that have adopted Mobile DevOps as part of their product

development lifecycle are also using internal check lists to ensure the expectations

of their software and delivery are met. However, these types of check lists also form a

great baseline to measure what are the critical areas of improvement for the future and

what parts of the process may need to be potentially enhanced.

When looking at Mobile DevOps process there are only few top categories that need daily

attention. But those critical daily tasks must be taken care to ensure the development,

testing, deployment and delivery have a smooth flow and teams get things done.

If we look at what Mobile DevOps is really about, it can be roughly categorized in 6 different

subsets to evaluate how well the process is established and probably what parts of it

could be improved. The checklist was divided into 6 different topics:

– DevOps Alignment. DevOps is an organizational shift, not just a technical one and unifying

group and individual direction and goals around the singular vision of the organization,

can make a difference.

– DevOps Context. This could be also ‘DevOps participation’. Is everyone participating

and is the process adopted by all critical stakeholders? The DevOps context is actually

making all relevant information available to those who need it, and when they need it.

– DevOps Learning. Eventually, everyone involved should be learning to execute and by

learning those best practices DevOps life can be easier. It’s all about how to empower

your team and your personal growth and how to foster understanding through continuous

improvement.

– DevOps Lifecycle. Focus on software as a product with care, attention and reflection,

and be a part of the ever-changing ecosystem.

Share this eBook!

– DevOps Organization and Resources. Providing structure for interactions and ground

rules on supporting collaboration and productivity.

– DevOps Process. Habits crafted to foster consistency, confidence and collaboration,

provided within Mobile DevOps framework to achieve continuous improvement.

Let’s see what are those 5 critical targets for Mobile DevOps everyday.

5 Daily Tasks of Mobile DevOpsHere are the top 5 tedious things for mobile devops – if you don’t rely on sophisticated

device farm or have invested in devices to run on a local environment – that should be

taken care of every day:

Mobile Devices

Handling mobile devices can be a cumbersome task. Depending on how large on-premise

device farm company may have it’s still number one the most important daily check-

up target for mobile devops. This is the top reason why mobile device farms are getting

significant popularity among mobile-first companies.

Share this eBook!

For example, have you ever thought how to enable a new Android device in your local

test lab? Here are few tips and tricks – and the actual checklist we go through every time

we add a new device in our device farm:

1. Set WiFi network.

– Without network, devices and testing capabilities are limited.

2. Add Google account to device.

– And good rule of thumb is to uncheck if auto-sync for account is enabled.

If you leave it checked your daily tasks will be quickly all about checking why

devices aren’t ready for testing…

3. Keeping Google Play Store relevant.

– It’s actually important and for sure makes your life easier to

have the Google Play store updated and device installed with

the latest version. Otherwise, you’ll start getting notifications

and again, the circle of ‘why devices are stuck and my tests

are flowing’ will hit your fan. Few tips:

• Without Google Play Store installed, device won’t have

required services which could be critical for some of the

testing scenarios.

• Start Play Store once and check possible notifications.

Typically there are few so check them and close the app.

4. Install Google Play Services.

– Services provided with Google platform will be in use and you can rely that there is

much less thing to worry about. Also, it’s recommended to disable this functionality

occasionally to test how apps work WITHOUT these types of services:

• Open Google Play Store and search “Google Play Services”

• NOTE! Google Play Services are updated quite often. Without upgrading your device

will get notifications about a need to upgrade Google Play Service

Share this eBook!

5. Install Google Play Games.

– Similar to Play Services, and especially if your company is developing a mobile game.

Absolutely must to have:

• This is required if – for example – you are testing games or your app depends on it.

• Open Google Play Store, search “Google Play Games” and install it.

6. Disable security checks for installing applications through USB connection.

– Android is not like Apple and one of its greatness is that apps CAN be

installed from various locations, by various developers and sources,

without confirming everything. My take, I think Apple will shy away

this sort of features in time, as it can really make this tedious for

mobile devops:

• Enter “Development options” on Device settings

• Check “USB debugging”, “Stay awake” and “Allow mock location”

• Uncheck “Verify apps over USB”

• Enter “Security” on Device settings

• Check “Unknown sources”

• Uncheck “Verify apps” (This is not necessarily available with all

versions of Android)

7. Disable Screen Lock. This will save the day.

– If screen locks for any reason (not used for some time) with some test

scripts tests may be

• Enter “Security” on Device settings

• Go to “Screen lock” settings and select “None”

• If this is not possible, you need to configure device to be

unlocked from Device settings on Testdroid Admin pages.

8. Set correct Date & Time to device.

– If you get questions why tests were executed yesterday or tomorrow,

you know what is wrong and how to fix it.

Share this eBook!

9. Set all “Location Services” on.

10. Install Chrome Browser and additional browsers.

– Eventually people want to test on different browsers

than what was delivered as part of the device. It’s good

rule of thumb to add few of them, and preferable the most

latest and the most used ones:

• Install Google Chrome Browser, if your tests needs it

• Sign in to Chrome browser to make it work robust on

tests.

11. Set Screen timeout to maximum value.

– The same things as with screen lock, if the screen stays up for longer period time,

the better:

• Enter “Display -> Sleep” in Device settings

• Select maximum timeout value for sleep.

Now, these were the tips for adding a new device. But as you can see, if that is not done

‘the right way’ mobile devops guys will have a lots of things to do on these devices, every

day.

Infrastructural Hardware

Let’s put devices aside and think about what else is required to run a mobile device lab.

As there are a myriad of moving components, attributes, that all must be maintained and

checked whether everything is up in speed, many of the mobile device farms rely on

either some internal monitoring implementation or commercial ones to keep everyone

up-to-date with the status. For mobile devops, the monitoring tools and features provide

the most fastest way to spot if something is (seriously) wrong.

Share this eBook!

Ready-to-go infrastructure is typically a feature on premium mobile device farms and

easily delivers what it promises: no delays for development, testing, deployment, and

always focusing on result-driven approach. In long run, this is one of the biggest factor

to save money and make things run faster and be more robust.

Support for Developers, Testers, QA – and Customers

The Mobile DevOps culture relies heavily on communication and collaboration across

teams, and very often it is the DevOps guys that provide internal support for devs, testers,

QA folks – and sometimes even to customers. No, it’s not a full blow customer support

division, but when operating large scale labs for different counterparts, they are the best

resource to collaborate with external customers as well.

Support provided by Mobile DevOps can be just helping to get test runs progressing,

running faster or making sure that (test) results are delivered. Software developers in test

(SDET) and QA engineer need different type of support when using the infrastructure,

devices, and basically the agility produced with a DevOps approach can help a lot. And

get those important business metrics met (and exceeded).

Share this eBook!

Operational Side of Things

Isn’t this no-brainer? Actually, it’s again and again making sure that all infrastructure is

working properly. And as devices, infrastructural hardware (servers, databases, monitoring

tools, additional physical hardware, among all the other things) and integration points work

out, there is still the connectivity. Power supply and making all infrastructure powered

is very critical, but so is the connectivity of all these things together. Typically there are

a dozen of WiFi networks device farms must use and those monitoring software must

continuously scan what is the current service level and how much can be gained from

the service provider.

Integration with DevOps Tools and Test Automation Frameworks

There is no doubt: open source frameworks, technologies, and tools are the most successful

when it comes to use of those in today’s agile processes. It all starts from the nature of

those products. When something is open it basically means these products are always

available and usable, can be easily changed to something else, and this gives the reliability

and freedom for organizations to change those any given day.

Proprietary frameworks and tools may offer better support, but that’s just another side

of the coin. These proprietary tools, frameworks platforms have disadvantages in that

you are dependent on the limited capabilities that a vendor can provide and if you need

Share this eBook!

to do something outside of the box you don’t have many options other than begging for

enhancements and having to wait.

The maturity of open source testing frameworks such as Appium, Calabash, UI Automation

and Espresso have come a long way. There are vibrant communities surrounding these

platforms and lots of contributors. Most use cases are covered and you have the ability

to branch for your own needs. There are a few cloud vendors supporting open source,

but make sure that you’re not having to use custom libraries or methods in order to make

things work, as this limits the ease of executing locally on your own devices and creates

vendor lock in.

What does this mean to Mobile DevOps and does it need to happen everyday? Well,

making sure that open source enablement is part of the process is crucial and helping

hand is required every day to ensure test automation scripts run smooth on a set of

devices. Integration flow can break easily if not monitored as people tend to bring their

own habits (and yet often, proprietary, non-compatible) tools to be part of the process.

Share this eBook!

CHAPTER 5

7 Process Steps with Mobile DevOps Tools

Share this eBook!

T ools required to develop and test against production-like environment, deployment

done with a repeatable and reliable process, and monitoring and constant validation

of the process flow are those principles that vary depending on organization’s scope and

level of Mobile DevOps adoption. As there are plenty of Mobile DevOps tools and all have

different use case and sometimes can be overlapping

each other in the overall Mobile DevOps process, we’ll

take a look at different phases of the process and what

tools are used in each one of them.

Today’s mobile app developers and their organizations

have been increasingly investing in monitoring systems

that can capture production, development, and bunch

of other metrics all in real time. This is a great trend, but

if monitoring is done only for specific silos of the overall

flow, results may not expose how to optimize the most

important resource hogs and therefore monitoring only

produces imperfect results.

There has been lots of discussion about ‘shift-left’ concept where operations focus more

on earlier phases of the delivery process and entire production lifecycle. Especially in

mobile app testing, this approach and goals are highly useful

as app should be always tested against real environment – not

‘close-enough’ to production environment, but actually using

the production environment, which in case of Mobile DevOps

would mean real physical devices. In addition, it would easily

expose issues in earlier phases of the development and issues

would be more cost efficient and easier to fix.

Whereas testing has been traditionally seen as one of the

final steps of the process, it is nowadays in the middle of the

Share this eBook!

process, and right after each build. This way the agility of the development and testing

isn’t compromised and more data is acquired after each test cycle. This test data with all

possible details can significantly help development to fix issues and create more robust

apps.

In general, there are 7 widely accepted and used steps in the Mobile DevOps process flow.

7 Process Steps with Mobile DevOps ToolsInstead of traditional 4 step process – plan, develop & test, deploy and operate – there are

actually more flavors in the process that should be highlighted. Especially when it comes

to considering what tools are used, how those fit together and enable seamless tool flow

from planning to production. Automation is without shadow of doubt critical part of the

flow and enables process to be highly iterative, repeatable, and robust. Automation is

actually an enabler for continuous everything, in each step of the process.

Code

First and foremost, code development, code reviews and anything to do with actual

Share this eBook!

development is closely related to continuous integration. The agile approach made

continuous integration very popular as it enhanced the way developers work, collaborate

and integrate their doings together, in natural way. The continuous integration with

regular daily routines produce lots of data about the mobile application, regressions

and compatibility of each developer’s contribute, and discovering issues early exposes

integration flaws, even incompatibility of developer’s doings, and makes better time

consumption and schedule estimating easier and more accurate.

Build

Building application is an everyday job. In Mobile DevOps, organizations use variety of

version control, source code management (SCM), different practices for merging code

and notifying everyone about the build status.

There are generally few different branching strategies when it comes engineering team

to work as part of the Mobile DevOps approach:

• No branching. Team works only from the main source tree. This kind of case is typical

for smaller teams that do not need isolation for development, releases and builds.

• Branching for release. For example, development team can create branches to support

an ongoing release. In this scenario, team creates a branch before release to stabilize

the release and then merges changes from the release branch back into the main

source tree after the application is deployed.

• Branching for maintenance. This is not so typical case but development can create

a branch to maintain some older application version. Branch is created to support

maintenance of specific version and current development and regressions are not

jeopardized. Sometimes it’s just a quick fix for certain user-base that are not using the

latest versions.

• Branching for feature(s). Engineering team can also create branch based on certain

feature set. This is quite typical when certain individuals or teams work on specific

feature that development branch is create, the coding takes place, and the feature

Share this eBook!

based branch is merged to main source. This makes parallel feature creation possible.

For branching and merging strategy, there are plenty of great repository products, version

control tools and source code management software available. However, this is critical

as it needs to go inline with entire development team and the same procedures must be

used by all contributors.

Test

As said, testing has been one of those functions that had “shift left” due to DevOps thinking.

Testing is also one of those key areas that define the quality aspects of the software; despite

no error, bug or issue is fixed by QA, all or at least majority of those can be discovered

during this phase. As continuous integration has several goals for testing phase, testing

should be always automated to ensure seamless flow, rich and detailed results, and

adequate verification of the application.

Share this eBook!

Continuous testing basically means testing each regression, build, and continuously practice

this throughout the development lifecycle. Early testing, early validation and frequent

testing ensures quality built-in to application can be verified and issues to be tackle early.

When Mobile DevOps is concerned of production-like or real environment, the testing

should always go on with real mobile devices. There are hundreds of good reasons why

Mobile DevOps process should not include in this stage any use of emulators/simulators.

Package

The package step includes tools for package

repositories and other storage mechanism

for the binaries created in the build step.

Repositories, also known as artifact or asset

repositories – contain all assets required

and associated with binaries to facilitate

deployment, such as scripts, configuration

files and additional infrastructural files for

deployment.

The continuous deployment adoption

may not be ideal for every company (and

developers), but the practice that enables

releasing and deploying can also enable a

seamless of a delivery pipeline and flow for

Mobile DevOps. This flow facilitates continuous deployment of applications to testing

(or QA) and then all way to production, in automated and efficient manner. The package

step clearly supports the goal of continuous deployment to package and release new

versions of applications (with new features) to end-users.

Share this eBook!

Release

Most of the tooling and processes that make up the core of Mobile DevOps technology exist

to facilitate continuous integration, continuous release, and continuous deployment. Part of

this process is application release automation tools that help up packaging and deploying

application from development to production (to be readily available for download /

users), using automation.

These sort of tools aid Mobile DevOps flow to produce capabilities to implement

continuous delivery for even large number of releases. In addition, release management

and deployment planning with every release requires collaboration across stakeholders.

Release management tools allow organizations to plan and execute releases, provide a

single collaboration portal for all stakeholders in a release, and provide traceability for a

release and its components across all stages of the build and delivery pipeline.

Share this eBook!

Configure

Configuration can be also seen as Infrastructure as Code (IaC) or Infrastructure as a Service

(IaaS). IaC has been claimed to be a key attribute to enable full utilization of DevOps into

organizations. With this, developers can be more involved in defining configuration and

operations teams can get involved earlier in the development process.

Tools and methods that utilize IaC and IaaS can bring transparency to configuration of

process and provide the visibility to users within the organization, aiming to bring teams

together to maximize their efforts. Automation as part of the process can remove error-

prone aspects of manual doings and processes, and make things more efficient and

productive.

Configuration tools aim to allow creation of better applications with flexibility, less

downtime, and enhancing an overall cost effectiveness for the company. This approach

is intended to reduce as many things creating complexity as possible by removing manual

configurations. Automation and collaboration are the epicenter in Mobile DevOps and

this is reason why configuration is also widely automated across development flows.

Monitor

There is also important aspect of making sure organization gets (only) valid data about

the process, results in each step, and the outcome. Continuous monitoring provides that

sort of data and metrics to organization to evaluate all different functions of the process:

development, operations, testing/QA, personnel and other individuals involved in the

process. It’s important to remember that these metrics should not be measured only

in production but during the process itself. When done properly as part of the process,

enhancing and changing the daily-doings is easier and will have an impact for all steps.

Share this eBook!

ConclusionM obile DevOps has significantly different requirements, expectations and criteria

for usefulness when it comes to monitoring mobile performance and general

health-level of mobile apps, websites and APIs. Whereas Mobile DevOps approach can be

highly useful to integrate with, it breaks down barriers between mobile development and

operations, and also provides clarity to process and steps for mobile product development

lifecycle.

Over the years the DevOps approach has produced variety of principles that are constantly

evolving and new tools will be adopted in growing numbers. When looking into Mobile

DevOps process and its steps, things don’t happen in waterfall, but process itself is highly

iterative and all these tasks (and use of devops tools) happen in cycles.

At Testdroid, we have implemented DevOps approach for couple of years and gained lots

of positive results. We hope this ebook would give you enough info and insights on how

to implement similar mindset and turn it into your own style for your unique situations

in your teams.