|
News from National -- Current Articles
7/19/2002
9:39:28 AM
Software Development Expert:
Andy Carmichael
Interview by S. Ibaraki, I.S.P.
This week, Stephen Ibaraki, I.S.P., has an exclusive interview with, Dr.
Andy Carmichael, an internationally renowned expert in software development
methods and processes with more than 20 years experience in the industry. His
most recent book, co-authored with noted J2EE expert Dan Haywood, “Better
Software Faster,” is most definitely a ‘must’ read for all software
architects, project managers and developers and the only book available that
discusses their “single-source model” approach.
Discussion:
Q: Your list of accomplishments are impressive indeed! We thank you for
sharing your years of experience and knowledge with us.
A: No need to stress the years. The grey hairs in the mirror are enough of a
reminder!
Q: What do your partner and three daughters think about the attention you are
receiving as a noted author, speaker, consultant, and internationally known
guru?
A: Sharing my life with four wonderful young women is a privilege, though I
don’t think it’s ever crossed their mind that I’m a guru! They are all
involved in music and the arts and are only really interested in software
when the computer they are using for school work, communication or recreation
misbehaves. At this point software consultants are best advised to keep a low
profile rather than boast of their skills!
Of course it should be a wake-up call to everyone involved in software when
they come across users – especially ones they love – who are less than
satisfied with the general quality of software! At least it encourages me to
keep up the search for better ways to produce it.
Q: Can you describe what led you into the computing field and lessons you
learned along the way?
A: Like many of my peers I didn’t start out studying software or computer
science, but in using software to do something useful – in my case trying to
calculate stress levels in aircraft components with holes in. This was a PhD
research project in Mechanical Engineering at Southampton University in the
UK. But soon afterwards I found myself working in a larger commercial team,
where the problems of how programmers worked together, shared changes to a
common code base and used tools to measure and improve quality, often
outweighed the problems of how to program a particular algorithm or
subroutine. This is when I first got involved in the evaluation and
specification of software methods and then software development tools.
Probably the most important lessons I learned in those years were the
importance of the foundational principles of software engineering: abstraction,
modularity, information hiding and dependency management (loose coupling and
high cohesion). When object-oriented design and programming became a
commercially viable prospect towards the end of the ’80s, it was clear to me
that they added important mechanisms to improve the way we apply these
principles, which was why I wanted to be involved in the exploitation of
object-oriented methods.
Q: You founded Object UK Ltd. Can you tell us more about this company and its
purchase by TogetherSoft?
A: I founded Object UK in 1992 in order to provide software tools and
services to support object-oriented development methods. By then I had been
working with CASE tools (Computer-Aided Software Engineering) for a number of
years and I felt the emerging object-oriented methods would provide much
needed impetus to this aspect of the industry. This proved the case, though
the most significant development for Object UK was when the main product we
were distributing, Together®, was rewritten in the new Java language and took
the Java world by storm with its “single-source” approach. Basically it
bypassed what had been the major Achilles heel of CASE products – the need to
synchronize design and code through what is termed “forward and reverse
engineering”. Not only is this difficult for tool vendors to implement, it’s
incredibly difficult for developers to use effectively. Together overcame
this weakness at a stroke by using the same source file – actual Java or C++
code – to store the models and diagrams that designers and developers drew
and updated graphically. It was a master-stroke and one that even now the
competition has not matched.
The main difference it made to us, as more and more development teams wanted
to buy into the new approach Together had introduced, was that we found our
whole business was now focused around this product. Since TogetherSoft wanted
to have its own UK presence it was natural to begin talks to bring Object UK
into the company that owned Together. We signed this deal in November 2000
and I then took on responsibility for growing the European team of mentors
who would train and consult with the teams developing software with Together
in Java, C++ and more recently C# and VB. Soon after that was also when Dan
Haywood and I started work on the book Better Software Faster to provide
guidance to development teams looking for new techniques to accelerate
quality.
Q: You also edited two books for SIGS and Cambridge University Press—Object
Development Methods (1994) and Developing Business Objects (1998). Can you
share some tips from this experience?
A: Both these books were collaborations with other authors who were
contributing to the commercialization of object-oriented approaches to
software development. The early ’90s was a time when new object-oriented
methods were being published almost every month! The situation was very
confusing to project managers who just wanted to adopt best practice, rather
than have to assimilate scores of different notations and processes then
choose one or, possibly worse, invent a new one by combining different set of
features from other published methods. The industry’s response to this
situation eventually saw the emergence of a standard notation, UML (the
Unified Modeling Language), first published in draft form in 1997 by Graby
Booch, Jim Rumbaugh and others. In 1994 though my aim for the Object
Development Methods book was that it would show the commonality of concepts
and principles in all object-oriented methods, regardless of the notation
people used to display them. I think this was reasonably successful and
helped build consensus towards a standard approach. Developing Business
Objects continued this work but including a wider scope of where object
technology had been applied in business systems.
Q: Your most recent book, “Better Software Faster,” is so practical and it’s
clearly evident that both you and Dan have considerable “real-world”
experience to share. There is so much useful advice on how to engage and work
on object-oriented, enterprise-based development projects. You use Together,
TogetherSoft’s killer development app, and your sample distributed Java
application is excellent—supporting both local and Internet-based clients.
There’s an interesting foreword too by Edward Yourdon—he has been recommending
this book to his friends apparently.
Will you talk about this book in general, what led you to write this book
with Dan, and what you’re your main objectives with the book?
A: There were two clear goals we had when setting about this book. The first
was to explain the “single-source model” approach and how this can improve
quality and efficiency. This was not necessarily dependent on people using
Together (providing they had an environment that would keep design and
implementation in step). In many ways it’s much more important to me that
people understand the power of this concept and implement it in other areas,
rather than just go out and use Together. The lack of a single-source
approach is, in my opinion, the most important reason why model-driven
approaches and CASE tools in particular have not been used by even a majority
of software developers. The second goal was to give Together users practical
advice on how to get the best out of the product and guidance in defining
their development process.
In fact we identified four principles that sum up the message of the book:
maintain just one single-source model; use a minimum “meta-model” (in other
words have a simple definition of the model behind your development process);
use a “perturbation” change model (in other words an iterative approach based
on small changes to a valid system); and continuously measure quality. These
principles are applicable whatever development environment you use – or
indeed if you are only planning to use a text editor and a compiler!
Dan and I had a good partnership on this book because while I’m always
looking for these lofty goals and principles, Dan is the pragmatic details
man and an excellent teacher of complex aspects of software. He produced an
excellent Java implementation of our case study project – downloadable from
the book’s web site (www.bettersoftwarefaster.com) so readers can use it with
the book. He also provided a lot of detail, some in appendices to the book,
on how to configure Together which have really been appreciated by Together
users that I’ve spoken to. Dan’s deep knowledge of J2EE has also meant that
we were able to cover this aspect of Java development in one of the later
chapters of the book.
Ed Yourdon and I did some work together in 1992 but I had not seen him since,
so I was flattered when he agreed to write the foreword to this book. Ed has
done a lot of work recently on legal disputes between firms involved in large
software projects that have gone wrong. As he points out in the foreword the
very existence of these cases means that software engineering has a long way
to go in improving its processes, so that the very common sight of large
software projects ending in tears does not continue to happen. Ed picked out
this paragraph from the first chapter of the book to highlight one key
strategy for avoiding software disasters.
"By discovering what the business imperatives are, the team can deliver
enhancements to the current capabilities within the specified period while
also preparing for longer term requirements. If the team follows this route,
it will need a flexible and lightweight development process with rapid
feedback loops to report holdups and roadblocks—and it will need to plan
contingencies accordingly."
Why read the rest of the book, asked Ed, if the answer to the problem is a
straightforward as that?! Fortunately he found enough in the rest of the book
to consider it essential reading. The fact that Ed is now recommending the
book to business analysts, project managers, designers and developers who are
not using Together, as well as to those that are, is a great encouragement to
us.
Q: It’s often the case that the analysts’ detailed UML diagrams are
overlooked or shelved by developers in the attempt to make deadlines. What’s
the impact of such poor synchronization between the UML model and code and
can you share you views on TogetherSoft’s software and how it addresses these
kinds of issues?
A: You’re right Stephen, this is a common problem. When delivered software is
out of step with the analysis and design documents there are two serious
consequences. Firstly the information that has been carefully and expensively
gathered in such documentation is seriously devalued, since the
implementation is not consistent with it. This may result in incorrect
functionality or delays caused by rework when users or testers discover
inappropriate system behavior. Secondly it means that the higher level
documentation is of only little value to the development team and subsequent
maintainers of the system since, even if the documentation is consistent with
the implementation, developers can’t assume it is, and they have to go and
examine the source code any way – a much more time-consuming and error-prone
process. Together’s LiveSource™ technology is the key to overcoming this
problem. By actually deriving the higher level models from the source code
and storing additional design documentation directly with source code files,
there is no separate synchronization process to be carried out.
There is still the need of course for different levels of detail in the full
set of diagrams and documents maintained by the development team. But rather
than maintaining several models at different levels of abstraction, we
advocate a “single-source” approach. This means that the different levels of
class diagram for example are all derived from the same source files but with
different filters and selections made on the diagrams by the designers, to
ensure each diagram communicates with its intended readers. Diagrams produced
to explain the detailed design for particular mechanisms in a system look
very different from diagrams that show an overview of the components or the
principle classes of business objects discovered in analysis. Yet all these
diagrams can be derived from the same set version-managed files.
This approach works best with relatively small teams – that means more than a
pair but less than a dozen developers – and one in which the different
specialists, analysts, designers, database or user interface experts and
programmers are all working together in the same team or teams, rather than
separating out say analysis teams and coding teams. This makes sense in
iterative life cycles where projects often have very short timescales (less
than 9 months) and requirements and business priorities that may change
through the life of the project. Such project require an “agile” process so
that the team can respond to changing circumstances but still deliver quality
software with useful and effective documentation.
Q: Can you comment on the iterative and incremental approach—the perturbation
change model?
A: Sure. A perturbation is a small change away from a stable state – like
moving a block by rolling it end over end. Each step in the process is a
stable state and progress is made by temporarily “perturbing” the
equilibrium. In an iterative development process it is important that the
starting point for change is a stable point. In other words all the tests
that have been defined for requirements implemented up to that point should
pass, all the unit tests should pass and other quality measures that have
been defined (such as measured complexity and coding standards) are also
acceptable. Then when a change is made to the system, say to implement a new
requirement, to improve its performance or to improve the integrity of its
design, the quality measures can be reapplied and the system be brought back
to its consistent working state. This contrasts greatly with a development
process that separates out requirements definition, design, coding and
testing into separate phases of the project.
Most modern software development processes are compatible with this
perturbation model, but they also tend to define a lot of other aspects like
project phases and roles which can cloud the issue. That’s why we’ve tried to
explain development in terms of a simple continuous process that remains the
same throughout the whole project: start from a state where the system (or
model) is conformant with the quality measures, introduce a small change and
get it back to the consistent state as soon as possible.
Q: Does this affect the way the project is managed as well as what the
developers do?
A: It certainly does. We devoted one chapter of the book to project planning
and monitoring because we think that project managers should be using a
different planning model from the normal critical path analysis method, which
is used by tools like Microsoft Project and is typically shown as “Gantt” or
“Pert” charts. This method is essential to planning something like a building
or a bridge where the dependencies between the tasks are extremely
significant. You can’t put the roof on the house for example until you’ve
built the walls to hold it up! On the other hand the order in which features
of a software system are delivered to users are not constrained to the same
degree, meaning that the users or business representatives can determine the
priorities for the implementation of different features based on an
assessment of risk and benefit. We can also allow these priorities to be
changed as the project progresses. This makes the development plan flexible,
while still enabling close monitoring of progress and continuously adjusted
estimates of completion dates.
Q: Sounds as though you’ve had experience of projects that don’t always
deliver to plan. Have some of the projects that you have worked on had
problems like these and what lessons were learned by the project that you can
pass on?
A: You’re right Stephen! I don’t think anyone who has worked in this industry
for any length of time would not have seen some project or other suffering
from budget or schedule overruns. I remember one large project where everyone
was working really hard to meet its first customer delivery deadline – not
for the system going live but still a very important milestone. Right up to a
week before the deadline the project manager thought he was going to get a
system out on time. However at that point they realized that it was not going
to happen and they declared a one month slip. How could the management have
been so out of touch with the reality on the ground? Well in fact the project
did not complete that milestone until a full 5 months later! The main problem
was that this was the first occasion when the project had tried to integrate
all the software being delivered by the different teams. Everyone thought
they could finish their piece in time but it wasn’t until they started to put
them together that they could see where the holes were. Continuous
integration of software, and continuous unit and functional testing of
software, have been important aspects of software development processes that
I stress to my clients, and particularly since the experience on that
project. Whether the project learned the lesson I’m not sure, but certainly
many other projects have learned from their problems.
Q: I understand that you’re now moving on from TogetherSoft? What does the
future hold for you now do you think?
A: That’s right. I’ve really enjoyed my time working at TogetherSoft with a
tremendously talented bunch of people, both in the UK and throughout their
offices worldwide, but it is now time to move on. I intend to remain
available as a consultant to clients using Together as their development
platform but I also want to consult more widely with enterprises defining and
improving their development processes. I would not be surprised to find that
I become involved again in the management of a UK-based professional services
organization but actually at this stage I haven’t made any final decisions
about new businesses. It’s one of the bonuses of life that one never knows
what’s just around the corner!
Q: Thank you for sharing your valuable insights with us today and we look
forward to hearing you speak at conferences, reading your books, and
articles.
A: Thank you Stephen. It’s been a pleasure.
|
|
