<< . .

. 14
( : 22)



. . >>

The values are then plotted cumulatively over time for all three variables as shown in Exhibit 15-3. Again,
this can be performed for one task or the entire project.
After calculating the BCWS, BCWP, and the ACWP, Perry can determine in what combination of the
following circumstances he might find the project:
BCWP = BCWS On Schedule
BCWP < BCWS Behind Schedule
BCWP > BCWS Ahead of Schedule
BCWP = ACWP Meeting Cost Target
BCWP < ACWP Cost Overrun
BCWP > ACWP Cost Underrun




Exhibit 15-3 Earned value.
The BCWS, BCWP, and ACWP also are useful for determining overall project performance. The measures
for doing so are the cost performance index (CPI) and the schedule performance index (SPI), which are
calculated as:
CPI = BCWP / ACWP or planned costs / actual costs
SPI = BCWP / BCWS or planned costs scheduled costs
Smythe Project example ($ in thousands):
CPI = BCWP / ACWP
= 200 / 300 = .66, indicating cost
performance is not very
efficient since the result is less than
1.00
SPI = BCWP / BCWS
= 200 / 220 = .91, indicating
schedule performance is not
very efficient since the result is
less than 1.00

The measure of performance is determined by how close the calculated value approximates 1.00. If the CPI
and SPI are less than 1.00, then performance needs improvement. If greater than 1.00, then performance
exceeds expectations. This can be performed for one, a group, or all tasks on the project.

Making Performance Assessment Count
A project plan serves no purpose if no one knows or cares if it is being followed. Perry, therefore, regularly
keeps a “pulse” on the schedule and cost performance of the project. He collects and analyzes data to ensure
that plan and reality match as closely as possible. If a variance exists, he determines whether to take corrective
action. Of course, a variance can exist for quality as much as it does for cost and schedule. Perry knows that
and ensures that metrics also exist to measure quality.

Questions for Getting Started

1. When collecting data for determining cost and schedule status, did you determine:
• Expertise needed?
• Mode of collection (e.g., formal versus informal)?
• Obstacles you will face?
• Tools to do the job?
• Type of information infrastructure you want in place?
• Ways to communicate the value of collecting status?
2. In regard to status reviews, did you determine whether to collect data prior to or during the
meetings?
3. When collecting data, did you identify the threats to reliability? To validity? How will you deal with
those threats?
4. When assessing status, what variables will you look at? Variances? Cost variance? Schedule
variance? Earned value? How will you go about calculating them and how often? Will the calculations
be for selected tasks or the entire project?


Previous Table of Contents Next




Products | Contact Us | About Us | Privacy | Ad Info | Home

Use of this site is subject to certain Terms & Conditions, Copyright © 1996-2000 EarthWeb Inc. All rights
reserved. Reproduction whole or in part in any form or medium without express written permission of
EarthWeb is prohibited. Read EarthWeb's privacy statement.
Project Management Practitioner's Handbook
by Ralph L. Kleim and Irwin S. Ludin
AMACOM Books
ISBN: 0814403964 Pub Date: 01/01/98

Search this book:
Search Tips

Advanced Search


Previous Table of Contents Next



Title
Chapter 16
Quality Assessment: Metrics
In Chapter 15, Perry developed ways to assess performance with regard to cost and schedule variances.
-----------
Quality assessment is the other element in monitoring performance.
Establishing measurements for quality is a way to identify opportunities to reduce waste, determine how the
project is achieving its goals, ascertain trends, and establish baselines for future projects.
Quality can have several meanings, so Perry defines the word in terms of his project. After consulting the
customer and reviewing project documentation (the statement of work), he defines quality as service that
satisfies a defined degree of excellence. In terms of the Smythe Project, quality is satisfying the requirements
set by the Smythe family. Focusing on his customer™s requirements, Perry can determine the measurements to
use. Metrics are the tools and techniques he will use to track and assess quality.

Introduction to Metrics
There are two basic categories of metrics, qualitative and quantitative. Qualitative metrics are intangible,
noncalibrated measures. Examples include degree of customer satisfaction and degree of importance. These
metrics are subjective. Quantitative metrics are tangible, calibrated measures. Examples include financial
analysis and parametrics. These metrics are objective.
Qualitative and quantitative metrics can be used to measure the satisfaction of the customer™s requirements, as
well as the efficiency and effectiveness of processes for building a product or delivering a service. In their
simplest form, quality metrics measure the relationship between the number of errors and a unit of measure.
An error is the difference between what is expected and what has occurred”in other words, a variance.
Of course, Perry knows that metrics do not happen spontaneously. He must set up a process for collecting
data, then analyzing the results. So Perry takes the following actions.
1. He determines what to measure. The statement of work provides much information; however, he
also interviews the customer and examines the metrics used for earlier projects of a similar nature.
2. He seeks agreement on what metrics to use. There are quantitative and qualitative metrics, simple
and complex. People must see the value of a metric; otherwise, they will not support the collection
efforts or respect the results.
3. He obtains the software to perform the metrics. These include project management software,
database applications, and modeling packages.

The Collection and Analysis of Data
Perry must build a good database. Without data he cannot do much. If the data lack reliability and validity,
they produce useless results. But having good project management disciplines in place will help in collecting
reliable, valid data. Perry has the expertise to collect good data, including statistical knowledge, analytical
prowess, and communications skills. Without these skills, establishing the metrics would be extremely
difficult. Also, Perry must exercise discipline when implementing the metrics. This means collecting data
regularly and using comparable methods over time.
Perry follows five steps to measure quality: (1) identifying what needs to be measured, (2) collecting the data,
(3) compiling the data, (4) analyzing the data, and (5) communicating the results.

Identify the Measures
As noted earlier, there are multiple ways to identify what needs to be measured. Perry reviews project and
technical documentation. He meets with people directly as well as remotely connected to the project. He
reviews the history of similar projects. He selects benchmarks, or examples from other companies against
which to compare his results. In any event, he must have buy-in for whatever methods he chooses. Without
buy-in, support may decline.
Of course, the audience will largely dictate what metrics to use. The project team may want to measure
technical aspects. Senior management and the customer may want measurements of customer satisfaction.
Perry is interested in measuring his project management. In any question of determinants, business
considerations should be first. Ultimately, customer satisfaction is the quality metric.
A way to determine business metrics is to identify key project indicators, or KPI. These are elements of a
project that contribute to successful completion of a project. On the Smythe Project, a KPI is the number of
complaints about the bridal shower. To identify KFIs, determine all the processes involved in project
management, process management, and technical performance. Then, with selected representatives, rank
those processes and select the most important top ten.
PDCA
A useful concept for performing metrics is the Plan, Do, Check, Act cycle, also known as the PDCA Wheel
or the Deming Wheel.
The Plan is developing an approach for improving a process or implementing a metric or both. The Do is
turning the plan into reality by executing it. The Check is determining if the improvement or metric is
working. The Act is making any changes to improve the process or metric. The cycle is shown below.




This cycle repeats throughout the process or measurement; it ensures stepwise refinement of the plan.
In reality, the PDCA cycle can be applied to any decision-making endeavor. Managing a project lends itself
to application of the PDCA cycle; project plans are continually revised to reflect reality.

Whatever the metrics chosen, Perry answers the following questions for each measurement tool:
• Who is the metric for?
• What purpose will it serve?
• How often will the measurement be taken?
• What is the formula?
• What is the data source?
Collect the Data
Perry uses data from the project data repository created by his project management software. He ensures the
data are current, thanks to input from status review.
In addition to the data repository, he searches the project history files and project library for relevant data. He
can access completed forms, past reports, and memos. He also uses alternative sources like the Internet for
data in the public domain and available through think tanks.

Compile the Data
Perry must put the data into a usable format. One of his first actions is to cleanse the data, identifying bad
(irrelevant) data and standardizing it (putting it into the same format). Perry sorts the data, reviews it to
determine any anomalies (e.g., alphabetic characters in a numeric field) and ensures that it has all the decimal
points in the right place. While doing this, he avoids introducing bias, which would influence the results. For
example, he removes data to which he might respond subjectively, such as data originating from a person or
system that he dislikes.
Data are raw, while information is data in a meaningful form. Perry has several tools to convert data into
information, including Pareto charts, checksheets, scattergrams, histograms, control charts, and trend charts.


Previous Table of Contents Next




Products | Contact Us | About Us | Privacy | Ad Info | Home

Use of this site is subject to certain Terms & Conditions, Copyright © 1996-2000 EarthWeb Inc. All rights
reserved. Reproduction whole or in part in any form or medium without express written permission of
EarthWeb is prohibited. Read EarthWeb's privacy statement.
Project Management Practitioner's Handbook
by Ralph L. Kleim and Irwin S. Ludin
AMACOM Books
ISBN: 0814403964 Pub Date: 01/01/98

Search this book:
Search Tips

Advanced Search


Previous Table of Contents Next



Pareto charts display information to determine the potential causes of a problem. A bar chart (not a Gantt
Title
chart) shows the major categories or elements on the x-axis and the prioritized numbers of a result (e.g.,
number of complaints) on the y-axis, as shown in Exhibit 16-1. The highest bar has the greatest likelihood of
being the cause of the problem.
Checksheets are documents that record the frequency of distribution of incidents. Each occurrence is recorded
-----------
in an interval identified, as shown in Exhibit 16-2. The information identifies what intervals have the greatest
and least number of occurrences. The checksheet also graphically displays information in the form of a
histogram, as shown in Exhibit 16-3.
Scattergrams, sometimes called scatter or correlation charts, show the relationship between two variables, as
shown in Exhibit 16-4. Normal relationships are “bunched together”; the abnormal relationships are “outside
the bunch,” thereby indicating an anomalous situation.
Control charts, like the scattergrams, identify normal and anomalous situations, specifically variance from the
average. Upper permissible and lower levels of variation are identified. As with the scattergram, the focus in
on variation, with emphasis on reducing erratic behavior. To better understand control charts, here™s an
example for building one.




Exhibit 16-1. Pareto chart example.




Exhibit 16-2. Checksheet example.
Six hotels are interested in knowing the average number of complaints during the summer season. The analyst
collects data from these six hotels and compiles them in the table on pages 157 and 158.
Exhibit 16-3. Histogram example.




Exhibit 16-4. Scattergram example.

Hotel Average Number of Complaints

A 30

B 40

C 60

D 80

E 35

F 25

270

Before drawing the control chart, the analyst determines the “average average,” and the upper and lower
limits of the control chart. The “average average” is the sum of the averages divided by the sample size, or N
(the number of hotels participating); thus, 270 divided by 6 equals 45. See the control chart in Exhibit 16-5 for
a plotted graph. The equation for the upper control limit is
The equation for the upper control limit is




For the lower control limit, the equation is:




Thus, the average number of complaints for Hotel D is out of control because it falls outside these boundaries.




Exhibit 16-5. Control chart example.

Trend charts track past performance and forecast results based on history. As shown in Exhibit 16-6, the chart
shows the relationship between two variables. On the x-axis is a time span and on the y-axis is the value of a
variable.
Using trend charts can be dangerous as well as useful. On the one hand, they require assuming that the future
environment will be as in the past, thereby permitting forecasting. On the other hand, they enable long-range
planning and playing “what-if” scenarios.

Analyze the Data
After compiling the data, Perry analyzes it. He reviews diagrams and looks at statistical compilations. Below
is a table showing the compilation techniques employed and flags for assessing issues dealing with quality.
Compilation Technique Flag
Pareto chart Tallest bar indicates the largest “driver” for the cause
of the problem.
Checksheets Longest frequency of occurrences for a variable;
thereby reflecting the focus of attention.
Scattergram The most frequent occurrences and anomalies; the
latter indicating a problem vis-à-vis normal behavior.
Control chart Exceeding the upper control limit or going below the
lower control limit, thereby indicating possible erratic,
uncontrollable behavior of a process.
Trend chart Upward or downward slope of the line, indicating a
potential problem if the trend continues.

When analyzing the data, Perry will use several standard statistical calculations”specifically, mean, median,
mode, and standard deviation. The mean is the average of the values for items in a group of data. The mean is
best used when the original data are large enough not to be skewed by extreme values. The median is a
position average at the midpoint for a frequency distribution. The median is best used when extreme values in
the frequency distribution could distort the data. The mode is the value that appears most frequently in a series
of numbers. The mode is used to avoid distortion by extreme values.
Standard deviation is another useful calculation. It determines the degree that each occurrence in a frequency
distribution is located from the mean. In other words, it measures dispersion.




Exhibit 16-6. Trend chart example.
Exhibits 16-7 and 16-8 are examples of how to calculate the mean, median, mode, and standard deviation,
respectively. In our example, the limousine service providing transportation for the Smythe wedding from the
church to the reception wants to know the travel time between the two locations. The data they collected for
five transportation times in minutes are shown below:




Another quick, easy way to analyze data is to divide the data into quartiles, or four equal parts, after forming
an array. The analyst counts down the array until he identifies the final item in the first 25 percent and then
calculates up the array. Then he selects the midpoint between the end of the first and the top of the fourth
quartile.
For example, on page 161 is a table of customer responses to a hotel survey of customer satisfaction. The
hotel wants to know the results of their questionnaire, by quartiles. The calculation is shown in Exhibit 16-9.
Fishbone Chart
Not all quality measurement tools are quantitative. The fishbone chart, also known as the Cause and Effect
Diagram, is a diagramming method that identifies the cause of a problem by connecting four M™s: machines,
manpower, materials, and methods. At the end of the fishbone is a description of the effect of the problem.
An example fishbone diagram is shown below:




The fishbone diagram helps you determine if additional research is necessary to verify a cause. In addition,
you can use the diagram to determine another process that will reduce problems associated with machines,
manpower, materials, and methods.



Previous Table of Contents Next




Products | Contact Us | About Us | Privacy | Ad Info | Home

Use of this site is subject to certain Terms & Conditions, Copyright © 1996-2000 EarthWeb Inc. All rights
reserved. Reproduction whole or in part in any form or medium without express written permission of
EarthWeb is prohibited. Read EarthWeb's privacy statement.
Project Management Practitioner's Handbook
by Ralph L. Kleim and Irwin S. Ludin
AMACOM Books
ISBN: 0814403964 Pub Date: 01/01/98

Search this book:
Search Tips

Advanced Search


Previous Table of Contents Next


Number of
Title
Rating Value Customer Responses Quartile 1 Quartile 2 Quartile 3

Poor 1 5 5 5 5

Fair 2 0 0 0 0
-----------

Good 3 25 20 (of 25) 25 25

Very Good 4 30 20 (of 30) 30

Excellent 5 40 15 (of 40)

25 50 75

Exhibit 16-7. Mean, median, and mode calculations.
Mean
The mean, or average, is calculated by summing the numbers from column A (60) and then dividing by the
number of samples taken (also called N). The formula is:
Average time = sum of column A/N
= 60/5 = 12, which is the average travel time between the two locations.

Median
The median is the middle number in a list of numbers. For our example, Perry arranges the numbers in
column A from low to high: 9, 10, 10, 12, 19. The middle number of these five numbers is the third number,
which is 10. Thus the median, or average, travel time between the two locations is 10.
Mode
The mode is the number occurring most frequently in a list of numbers. Again, Perry arranges the numbers
in column A from low to high: 9, 10, 10, 12, 19. The number occurring most frequently is 10. Thus the
mode, or average, travel time between the two locations is 10.

The Results of Data Analysis
After converting his data into information and analyzing it, Perry win communicate the results. He does that
in several ways, such as in a presentation or by sending e-mail. Whichever method he chooses, he states his
assumptions”he does not hide them. For example, he might state that the information in the trend chart
assumes that the project will proceed at the current pace.
Also, he portrays the data honestly and openly. He does not outlay charts or other information to cover up or
skew the messages. Finally, he is consistent when collecting and reporting the information. Consistency
ensures timely and useful information. Otherwise, he will have a credibility problem with the metrics.

<< . .

. 14
( : 22)



. . >>