PLANNING, QA KEYS TO SUCCESSFUL ALM
IT FOCUS ON DEVELOPER
codeguruTM <HTMLGOODIES/>
INFOSTOR®
This content was adapted from the developer.com and eSecurity
Planet websites. Contributors: Sergey Smetyukh, Artyom Vorobyov,
Necco Ceresani, Nazar Tymoshyk and Jeff Francis.
›02 Estimate Software Projects Like a Pro: Setting up the Right Team and Structuring Communication
›08Integrating Bulletproof
Security into App Development
06‹Top Mistakes to Avoid when Implementing DevOps
10‹Don’t Overlook QA for Mobile Apps
CONTENTS
2Planning, QA Keys to Successful ALM © 2016 QuinStreet, Inc.Back to Contents
O ne of the burning issues on the agenda for many
technology project managers and pre-sales
specialists is how to master the art of estimating app
development projects. The key problem is how to make
estimates as competitive and as accurate as possible while
involving a reasonable amount of team effort and securing
a fair chance of meeting the figures that you have come up
with. An optimal estimate of development time and team
effort required to complete the project is a well-thought-
out and well-grounded plan that allows for some risk
assumptions.
First and foremost, avoid inflating your project estimates
as well as avoid downplaying your figures.
The following are a number of hands-on tips to consider if
you are a developer, project manager or other professional
in the software development arena.
1. Know Your Way Around: Understand What You Are to Estimate
It’s easy to estimate what you know. It’s hard to estimate
what you don’t know. It’s very hard to estimate what you
don’t know that you don’t know.
Estimate accuracy hinges on the level of detail that you
get from a client. The more input you get from the client,
the more specific an estimate you can provide. If you have
any gaps in understanding a project, don’t get down to
assumptions head-on. Ask the client additional questions.
Keep in mind the four golden rules of good estimation:
1. A feature is taken into account only once.
2. Only those features requested by a client are
included in the estimation.
3. Features not requested by the client are not
included in the estimation.
4. A feature should be estimated correctly.
Estimate Software Projects Like a Pro: Setting up the Right Team and Structuring CommunicationBy Sergey Smetyukh and Artyom Vorobyov
3Planning, QA Keys to Successful ALM © 2016 QuinStreet, Inc.Back to Contents
PLANNING, QA KEYS TO SUCCESSFUL ALM
These rules seem to be no-brainers, yet people who make
estimates, let’s call them estimators, often focus on the
last rule, while completely neglecting the previous three.
However, failing to take into account 400 hours of back-
end development is a far more serious oversight than
miscalculating the development time of a user settings
screen by four hours.
How We Deal with This: All questions remain relevant
to a client’s request. Each clarification may add to
the estimate and therefore should remain within the
boundaries of the client’s initial request. In addition, each
question should cover only one point and use the same
definitions and terms that were used by the client during
the initial contact. Clarifying questions should be worded
in a way that elicits an exhaustive answer to a particular
question rather than a description of an entire system
without covering anything in particular
2. The Team That Works on a Project Is the One That Estimates the Project
Ideally, the team members who will be later engaged
in the project implementation should provide the
estimates for it. This means that the estimation team
takes responsibility for the figures in the final estimation
from the moment they get down to assessing the client’s
request.
No doubt, we don’t always follow this rule, especially
at times when our pipeline is filled to capacity with
clients’ requests. If one person is estimating four projects
simultaneously, there is only a 25 percent chance that he
will participate in any of these four projects. Let’s put it in a
different way: 75 percent of these projects are developed
based on other people’s estimations. That is the key
stumbling block because people are prone to subjectivity
and tend to make conclusions from their own experience.
What a senior developer with in-depth domain knowledge
is able to do in 100 hours will require 30 percent more
time from a mid-level specialist. If you carry this example
over to the final estimation figures, the problem may take
menacing proportions and turn into a sword of Damocles
hanging over you.
How We Deal with This: In the majority of cases,
we assign idle resources (the so-called “bench”) to
estimation tasks. These are people who can set to project
implementation and “sign on the dotted line” under the
figures they provide. However, there are exceptions.
• Projects that require specific skills and knowledge:
In this situation, we are forced to turn to unique
holders of expertise necessary for a particular
project even if these people are already engaged
in other projects.
• Innovative projects, uncharted waters for all team
members: Such requests are assigned to tech
experts for estimation.
In any case, the key to success is reasonable resource
planning and management.
3. Don’t Go for It Single-Handedly
Do invite more than one team member to give an estimate
to a client. It’s crucial that each type of task is assessed
by a competent person. Development is in the area of
competence of a developer, art creation is approached
by an artist, the scope of UI work is estimated by graphic
designers, evaluating the time for management activities is
entrusted to project managers, and so forth.
As to a competitive approach to estimating the same
scope of work, there is no shared opinion on this matter.
Sometimes, you can involve several people to increase
accuracy of an estimate. In this case, chances are you
unearth more hidden risks and miscalculations. In addition,
the risk related to human errors and the possibility that
something will go unnoticed is reduced. This happens
when the final figures received from all estimators are
stacked up against each other.
Keep in mind that it doesn’t make sense to add extra
people to an estimation team after a certain point in time.
The efficiency curve will simply go down. If the size of an
estimation team exceeds 10 people, miscommunication
among all estimators is almost inevitable, and eventually
has a negative impact on the accuracy of the estimate.
4Planning, QA Keys to Successful ALM © 2016 QuinStreet, Inc.Back to Contents
PLANNING, QA KEYS TO SUCCESSFUL ALM
Alternatively, an estimation is made by a single tech
specialist. If this person lacks necessary experience, a
supervisor is assigned to such a person to guide her
through the process, oversee it, and make timely changes
to the estimation.
How We Deal with This: We assign from four to eight
specialists to make an estimation of a project. Empirically,
this team size has proved to be optimal for our company.
Such an approach yields us the most accurate results with
minimal effort involved.
4. Assign a Person Responsible for Estimation Delivery
This advice is particularly relevant for big projects where
more than one department is involved in the estimation
process and several teams look into a work scope and
give their assessments. Such a situation requires well-
orchestrated communication. Hence, there should be
someone at the helm to make up final figures, take into
account all possible risks, and present the estimate to a
client. That is where a knowledge holder — a person who
is in the picture for all details about the project — steps in.
How We Deal with This: We tend to adhere to this
principle because we constantly handle projects that
span competencies of several departments. For example,
one of our projects is focused on the development of a
wearable bracelet. The solution monitors vital signs and
caters to professional sportsmen undergoing rehabilitation
after injuries. The project takes in the following
development areas: hardware design (pcb, casing, and
its components), firmware, mobile application that
captures data from devices and visualizes stats for further
use, web solution that includes admin panels to manage
the solution, and cloud storage to keep user stats and
other data. In such situations, a person who coordinates
communication among several company departments and
brings together the estimation information helps eliminate
possible chaos.
Moreover, the sales cycle and RFX process may last several
months on big projects. That’s why there should always be
someone who will manage project knowledge and give
a prompt response to the client’s questions. It may take
quite some time for a client to make the final decision. The
client can request changes — add or remove features — a
few months after getting the estimate. Alternatively, the
client can ask for so many changes within a short period
of time that no one apart from the person in the know of
all project details and completely immersed into the RFX
process can provide well-timed estimation updates.
Pay attention to the fact that the person responsible
for estimation delivery should give clear deadlines to
estimators. In our practice, this person is a sales manager
or an RFX coordinator in many cases. An RFX coordinator
is a person who manages incoming requests processing
(Request for X, where X is information, a quote or a
proposal).
5. Coordinate Nuances with All Estimators, Understand Estimation Scope
All estimators should have a clear vision of both the key
functional units and non-functional components of the
project. The main idea here is that when everyone has
brought their estimation share to the table the team
should be able to compare apples to apples.
How We Deal with This: Before we get down to
estimating a project, we not only sync on the scope of
the functional part, but also touch upon tasks related to
documentation, unit tests creation, code review and QA
activities, the set-up of training seminars, if necessary,
and so on. Each estimator uses the same assumptions as
a guideline and makes an estimate presuming that the
project is assigned to specialists with certain experience.
We then sync on the project understanding as we progress
through the estimation process.
6. Keep a List of Common Estimation Mistakes at Hand, Sync on New Mishap Cases
Such mishaps are likely to be spotted during
retrospectives when the project has come to an end,
everything has been calculated and you can see oversights
of the initial estimation. To help estimators with different
levels of competency make reasonable estimates, you
5Planning, QA Keys to Successful ALM © 2016 QuinStreet, Inc.Back to Contents
PLANNING, QA KEYS TO SUCCESSFUL ALM
should organize seminars and trainings as well as cement
some basic rules in a written form. These guidelines vary
from company to company depending on many factors,
including your niche specifics. However, the most common
mistakes are:
• Underestimating testing and QA activities
• Underestimating efforts required for
documentation creation
• Underestimating expenses for business trips and
meetings
• Ignoring additional requirements and changes that
arise during the estimation process
• Overestimating the possibilities of tools,
languages, methodologies and the like, which
leads to a team effort’s underestimation
• Ignoring or underestimating efforts for project
management and support activities
How We Deal with This: Real-life practice has enabled us
to compile our own list of common oversights. We go over
them prior to the estimation start. At the same time, to
tackle the problem in a more consistent way and embrace
a larger picture, to level up team knowledge on the topic,
and to increase the accuracy of outputs, we take these
measures:
• Organize in-house estimation workshops and boot
camps
• Assign a knowledgeable estimator to supervise
less-experienced developers during the estimation
process
• Write up and put in place a set of guidelines for
performing estimations
• Introduce estimation templates with formulas that
allow us to automate calculations of resources
required to complete a certain task and thus
eliminate possibilities of human mistakes
The bottom line is that choosing the right team
and the number of estimation participants, shaping
communication wisely both with your colleagues and the
client, and keeping the process transparent for everyone
involved will help you get off to a good start.
6Planning, QA Keys to Successful ALM © 2016 QuinStreet, Inc.Back to Contents
D evOps is one of those transformative concepts that
falls under the painful classification of “easier said
than done.” No matter how you define it — and, yes, there
are plenty of definitions to muddy the waters and frustrate
even the most ardent enthusiast — DevOps is quite a nasty
challenge in the real world.
At its simplest, DevOps brings together disparate
teams that traditionally might not communicate, or
want to communicate with each other. Those teams are:
development people, QA people, and operations people.
The core idea of DevOps is to bring those teams together
to improve collaboration and share understanding of the
entire application lifecycle — with the goal of building
more scalable, higher quality software that delivers better
service to customers.
However you slice and dice it, DevOps is always going
to be a complex mixture of processes, people, tools and
(let’s not ignore the elephant in the living-room) fiefdoms.
The keys to making DevOps happen are harmony,
consistency and transparency.
Many enterprises struggle to blend teams together, fail to
separate vendor hype from tool functionality and aim too
high too quickly.
Having worked in the DevOps space for many years with
hundreds of enterprise clients, I’ve noticed a recurring
series of mistakes that companies make in implementing
DevOps.
Not Getting Early Approval from All Parties
This is a variation on the idea of setting your sights low
and reaping the predictable rewards of a lack of vision.
As the innovator behind your company’s push to DevOps
(whether you are a business executive, Dev person or Ops
person), you’ve got to get all parties on the same page
at the same time. Oh yes, don’t ignore the QA people
because they will be pivotal in ensuring your software
meets the rigors of market readiness. It doesn’t matter
which team you consult with first as long as you share your
vision with the other teams and relay the concerns of each
team to all involved.
Make sure that everybody buys into the goal of
Top Mistakes to Avoid when Implementing DevOpsBy Necco Ceresani
7Planning, QA Keys to Successful ALM © 2016 QuinStreet, Inc.Back to Contents
PLANNING, QA KEYS TO SUCCESSFUL ALM
streamlining and simplifying the process of developing
market-ready software.
Not Spending Vital Time to Create a Transition Plan
Your first pilot or full-blown software release will be your
blueprint for creating a plan that helps all teams transition
from legacy software development to DevOps automation.
Make sure you document the process, get the buy-in of
all parties, and use the accumulated knowledge to draft a
transition plan that everyone can accept.
Failing to Automate Two or Three Apps from Development Through Production
The smart approach to implementing DevOps is to
automate two or three small apps upfront so your teams
can discover all of the nuances of the application delivery
process. This approach will enable you to identify what’s
involved in the entire process, who the main people are, and
what the requirements are for getting software from A to Z.
Only Implementing ‘Basic DevOps’ Can Lead to Disaster
Doing basic DevOps (implementing some automation
functionality in the Dev and Integration environments)
is relatively easy for most enterprises. However,
incorporating all interested parties — notably QA, Stage,
Pre-PROD, and PROD into your DevOps plans — requires
considerable thought and effort. If your DevOps initiative
cannot operate smoothly across the entire delivery
pipeline, you can expect your initiative to crash and burn!
Falling Into the Trap of Creating a Special DevOps Team
In theory, the logic is fine: Establish a dedicated team to
focus on feeding and watering the newest beast in IT. The
migration to full DevOps should be smooth and painless,
right? Two problems typically rear their ugly IT heads:
• This new team can become another bureaucratic
silo that exists in a vacuum.
• Existing teams in Dev, Ops and QA feel left out
and do their best to derail the efforts of the new
team.
Missing the Big Picture
As trite as it sounds, it is true: People are the backbone
of your DevOps implementation. People literally make all
the difference, especially in terms of how well they interact
with each other, share knowledge, and understand and
agree with the common goals of DevOps.
Getting all interested people on-board the DevOps train is
always hard, even for small companies. The task becomes
extremely hard in large enterprises because hundreds
of people and several layers of business structures are
typically involved. Think: Messy politics and massive
resistance to change of any sort.
On the surface, implementing DevOps in an enterprise
seems relatively straightforward: Buy some new tools, get
the Dev and Ops people on board, create a plan, and go
for it. Major problems can arise when enterprises fail to
do their homework, such as getting early approval from
all parties, creating a special DevOps team and generally
missing the big picture. However, the positive takeaway
is this: Your implementations of DevOps can be quite
painless if you avoid common mistakes.
8Planning, QA Keys to Successful ALM © 2016 QuinStreet, Inc.Back to Contents
A ll of the world’s most significant accomplishments
are initially underappreciated. Just like Nikola Tesla
was discredited of all his achievements and no thanks
were given to Gideon Sundback for his zipper, security
breakthroughs in software development are all too often
overlooked. Yes, we all know the “safety-first-safety-always”
rule, but knowing is not enough; we must apply them.
Sad but True: Security Process in Reality
In today’s IT environment, it’s no longer a question of
whether your Web application is vulnerable to cyber
security threats or may be attacked someday: The
questions instead are, “When will it be?” and “How well
are you prepared?”
Security shouldn’t be an afterthought in development,
and it also shouldn’t be an afterthought in responsibility.
Without introducing security at the initial stages of the
software development lifecycle (SDLC), your chances
of re-engineering solutions over and over to address
security issues — detected long after functional solutions
are accepted — rocket with the speed of light. The later
you include security’s share, the greater time, money and
energy loss you’ll face.
Security, usually carried out in the form of a third party or
internal audit, goes right before the release of a product.
Being a security consultant, I see product owners and
project managers addressing me almost in tears to save
their creation from an “unexpected” hole. It gives me
shivers down my spine every time. It’s like complaining that
you got wet on a rainy day, but it was you who deliberately
left an umbrella at home.
If you find the tiniest bug (which may turn out to be a
grand entrance for a hacker), you should get back to
basics — coding — and then make up for every step
following, from re-coding to re-auditing, until the problem
is solved.
14 Steps to Secure Software
Since there’s no use crying over spilled milk, make sure
the screw cap is tight. No doubt maintenance and timely
updates of servers are necessary these days. There are,
Integrating Bulletproof Security into App DevelopmentBy Nazar Tymoshyk
9Planning, QA Keys to Successful ALM © 2016 QuinStreet, Inc.Back to Contents
PLANNING, QA KEYS TO SUCCESSFUL ALM
however, other security measures to employ.
1. Involve a security expert in your project. It’s wise
to have someone responsible to continuously
oversee all security testing efforts.
2. Conduct threat modeling. Imagine you’re a
hacker and analyze how it is possible to misuse,
compromise or hack your product. And then do it
again.
3. Define the right security requirements with your
security expert/application hacker. Sometimes the
problem is a lack of clear understanding of what
should be secured.
4. Use security frameworks during coding, but avoid
custom cryptography. It’s better to follow OWASP
coding guidelines.
5. Apply two-factor authentication where possible.
6. Log any high privilege activities to cover your back
if a breach still occurs.
7. Execute static application security testing (SAST)
defects with tools like Sonar, AppScan and
Veracode.
8. Integrate dynamic application security testing
(DAST) to continuous integration (CI) and test app
security in the runtime.
9. Find the weakest point in your application logic.
Logic issues are usually the root of evil in app
security.
10. Fix defects and revalidate if they are fixed properly,
twice and thrice over.
11. Configure patching for production and never
forget to update your platform. You don’t want
long-forgotten troubles from an outdated platform
to live on.
12. Set up a vulnerability scanner to make sure no new
security gaps appeared during the deployment
and release phase of your infrastructure.
13. Don’t skimp when it comes to proper penetration
testing.
14. Evaluate your process with the OWASP Software
Assurance Maturity Model.
Eat, sleep, secure, repeat. With a proper security program,
the number of security defects should decrease from
phase to phase:
• Coding: Carry out secure coding trainings
• Build: Conduct automated security tests (SAST
and DAST)
• QA and security: Arrange manual penetration
testing
• Production: Apply regular vulnerability scans
10Planning, QA Keys to Successful ALM © 2016 QuinStreet, Inc.Back to Contents
“Q uality assurance” (QA) might be the least sexy
concept in mobile. For many development projects,
it is merely an afterthought, the perfunctory final stage of
testing before you launch your excellent app publicly. It
often takes significant amounts of time and can command
a healthy percentage of development hours — and the
resulting balance sheet — for mobile engagements. But,
although QA might be the boring last stage of a standard
development cycle, mobile app developers and internal
development teams alike need to completely rethink this
mission-critical element to mobile app development.
If you were to look at the primary reasons most apps
fail to acquire market share, buggy operation would
probably rank near the top of that list. Few things are
more frustrating than a well-conceived and potentially
useful app that simply doesn’t work well. Whether the
app is missing features, crashes consistently, is difficult or
confusing to navigate, or freezes up at inopportune times,
glitchy apps will inevitably decimate the app’s user base.
The unfortunate thing for so many of these apps is that
such failure is almost entirely avoidable.
It’s no secret that being first to market is a massive
advantage in achieving app adoption and usage. No
one can blame a company for its desire to launch an app
as soon as possible (and before the competition). But,
as apps speed toward launch, their developers might
be shortchanging QA to the point where the market
adoption companies crave might become even more
elusive because the apps have not been thoroughly tested.
If companies gloss over this crucial step, they almost
assuredly will run into the type of faulty app operation that
doom many apps to obscurity and obsolescence. The only
thing worse than being second to launch is if your app
doesn’t work well once it does.
In the enterprise space, QA takes on an even more
central role to an app’s success. App adoption isn’t as
central a concern when building a bespoken software
solution for an enterprise because the company can
simply mandate usage of said app to its employees. That
being said, the stakes are often much higher for these
types of engagements. For example, if a nuclear power
plant construction company decides to digitize its testing
protocols, as Westinghouse has, there is literally no room
Don’t Overlook QA for Mobile Apps By Jeff Francis
11Planning, QA Keys to Successful ALM © 2016 QuinStreet, Inc.Back to Contents
PLANNING, QA KEYS TO SUCCESSFUL ALM
for error in the app developed for that purpose. As was
reported by CIO magazine, Westinghouse took two years
to choose its mobile partner and will spend another two
years building the app to ensure every last use case is
tested and every bug eliminated. For mission-critical
functions like this, QA moves directly to the forefront of
development.
Not every app requires this level of dedication to QA.
Westinghouse made the conscious decision to forego
speed in favor of quality. But, the two are not mutually
exclusive. It is possible to get to market quickly and
still prioritize the quality of your app by simply taking a
different approach to QA.
Companies cannot view QA as an obnoxious stage of
development that costs money and slows deployment.
They should view QA as the opportunity to perfect the
app before launching. Few, if any, apps will ever launch
in a state of developmental perfection, but QA is every
enterprise’s chance to get as close to perfect as will be
possible. But, the way to do that is not tacking a testing
stage to the end of the development process. It is baking
QA into every step along the way.
For the very best mobile solution developers, QA is not
just the last step in the app development process. QA is a
central concern throughout every stage of development.
When developers are coding, it’s not enough to let them
work away and then have someone check the code
at the end of the development process. Companies
need established processes in place to check code all
throughout development. All modules and features of an
app should be tested rigorously before being assembled
into the greater whole.
The best mobile solutions partners will have entire teams
dedicated to QA and only QA, and that team is separate
from the team of developers who coded the app. This
ensures fresh eyes are looking over the solution from every
possible angle.
When attempting to shed costs and reduce time to market,
the first line item to go is often QA. That strategy is both
shortsighted and detrimental to any potential future
success. Quality assurance, when executed correctly and
holistically, can make the difference between a great app
and a mediocre one — a successful app and one that is a
failure. If developers maintain a consistent and constant
dedication to QA, the apps they produce will win market
share in a way poorly tested apps simply will not.
Mobile partners that prioritize QA also prioritize quality.
Only after an app developer changes the development
culture to reflect QA as a top priority can that developer
achieve sustained, systemic success for apps he produces.