CIPS CONNECTIONS
Interviews by Stephen Ibaraki, FCIPS, I.S.P., MVP, DF/NPA, CNP
Michael Vax: Industry Leading Vice President of Product Development at Blackboard (formerly WebCT)
This week, Stephen Ibaraki, I.S.P., has an exclusive interview with Michael Vax.
Michael has spent more than 20 years in software development. During his career he has worked as a developer, an architect, a development manager, a director of development, a VP of Development, and a CTO. Since coming to Vancouver 13 years ago he has worked for a number of companies:
- Rydex Industries Corporation
- Xinex Networks Inc.
- Infonet Software Solutions
- WebCT Inc.
Michael is currently the Vice President of Product Development at Balckboard (formerly WebCT) (www.blackboard.com) - the world's leading providers of e-learning systems that are used by thousands of colleges and universities in more than 70 countries worldwide. At Blackboard, Michael is managing a 100+ person product development department, with offices in Vancouver and Boston.
Michael is a member of Agile Alliance (www.agilealliance.com) software development in Vancouver.
Michael is interested in career development and mentoring; he often speaks to computer science students about career development in the high tech industry. Michael founded and operates a web site www.CareerShare.com. CareerShare is as a place where people can share their career experiences and learn from each other. Its goal is to develop a knowledgebase of real-life career profiles as well as an active member community that will help people of all ages choose their careers, and explore new possibilities.
The latest blogs on the interview can be found the week of July 7-14, 2006 in the Canadian IT Managers (CIM) forum where you can provide your comments in an interactive dialogue.
http://blogs.technet.com/cdnitmanagers/
Discussion:
Opening Comment: Michael, you bring an impressive record of accomplishments to our audience. Thank you for sharing your considerable insights in this interview.
A: Thanks for having me Stephen. We all learn from each other and it is my pleasure to share my experience and ideas with your readers.
Q1: Can you discuss some major challenges in the past year and the solutions you and your team planned and then implemented? Why did you choose these solutions? Take the perspective of providing guidance to our audience who may be facing similar challenges.
A:
Challenge 1:
Around September 2005, my department was asked to develop
a new product in a short time while requirements were not finished and design had
just been started. My first reaction was – there is no way we can pull this off. I needed to deliver the new product in 3 months. Using our standard development process I would need the first two
months to do detailed requirements and design. This would leave just one month
for development which was obviously not enough.
Solution and why: I decided to turn this challenge into an opportunity to try an agile development process. I looked through different agile methodologies and selected SCRUM for its support of distributed teams. WebCT has development teams in Vancouver and Boston and I needed to involve developers from both sites.
We started the project with a little preparation. To ensure close information sharing that is vital for agile development we setup a shared project’s web site using MS SharePoint, broke work into small iterations (Sprints in SCRUM terminology), formed a feature team, and started development. At the end of each iteration, the team was delivering demoable and testable subsets of functionality. Features were prioritized with product management and assigned to a Sprint. Tasks for the iteration were also posted on the SharePoint site.
We started with a single team to build a framework, and then added two more teams a month later. Teams were meeting every day for 15 minute SCRUM meetings to discuss accomplished work, what to work on today, and issues to resolve. We did not maintain an MS Project schedule. Developers were distributing tasks between themselves. Design decisions were also discussed and documented on the web site.
The project went very well and we are in the process of switching our entire development organization to the agile methodology. Developers felt engaged and in control of the project, and we saw significant increases in both productivity and quality.
Challenge 2:
Another challenging project was the implementation of a data conversion program to upgrade our customers to the latest version of the product. Many of our customers have very scaled deployments with a DB size in the hundreds of gigabytes. It was required that the conversion process should not
take more than 24 hours.
Solution and why: The data schema had been significantly changed between releases and we needed to do detailed data mapping and documentation of the conversion’s business rules. This preparation step was vital; we created about 100 mapping documents and each was reviewed by a developer responsible for the area together with a DB developer who was working on the conversion team.
Another aspect that complicated conversion efforts were irregularities in customer data that accumulated during several years of deployment. We decided that our goal was not only to convert the database but also to ensure that the resulting database would be as consistent as possible.
To ensure project success we implemented a hosted beta program. Customers were sending us copies of their production databases, we ran conversion on QA servers, fixed all problems, and gave beta customers access to their data in our hosted environment for verification. Application usage pattern differs from one customer to another. The beta program gave us the opportunity to find and fix many problems and tune up performance for a variety of data distributions. The hosted beta was also very convenient for customers as it did not require them to setup a complex environment as well as deal with initial conversion problems.
Q2: Can you discuss your recent merger, what were the major issues, and how you addressed them?
A: It is a very exciting time in the company. We have a unique opportunity to discover how the other company conducts its business, to compare processes and to learn from each other, adopting the best of the breed. I firmly believe that merging companies’ cultures and building personal relations are the key to successful integration between two companies, and this is our main focus.
Q3: As a member of the Agile Alliance, give your perspective on its pros and cons. Why are you an evangelist? What are the best resources for your audience who want to engage more?
A: I knew about agile development for a while but was skeptical about its applicability to big enterprise projects. At the same time, like many people in the industry, I was growing frustrated by difficulties with properly estimating a scope, accounting for risks, and slowness in responding to a changing environment. Some time ago, I got involved with a local interest group – Agile Vancouver (www.agilevancouver.ca). Listening to speakers we were inviting to our meetings, as well as doing reading on my own, convinced me to give it a try and when I was faced with the project that I could not possibly deliver using our standard process, I decided to give it a try. It was kind of a “what do I have to lose” decision. As I mentioned before, the project was very successful and we are moving our entire development team to agile now.
There are many reasons why agile development works. Software development is all about managing complexity and uncertainty. While the waterfall approach assumes that with more planning we can predict and document everything, the agile process acknowledges that the future is uncertain, requirements are changing, and unexpected technical challenges are going to happen. To deal with these issues, agile calls for focusing on business priorities, developing functionality in short iterations, delivering a working product after each iteration that can be shown to the customer to get immediate feedback, and working as one team to facilitate collaboration. Being agile means being able to deliver quickly and change quickly and often.
There are many flavors of Agile development including: Extreme programming, SCRUM, Lean Development, Feature Driven Development, Crystal. While those Agile techniques may vary in practices and emphasis, their common characteristics include iterative development, focus on interaction and communication, and minimizing time spent on resource intensive intermediate artifacts.
You need to look at them all and then adopt practices that are best suited to your company’s business environment and culture. The Agile Alliance web site (http://www.agilealliance.com ) is a good place to start if you are new to Agile development. Make sure that you read the Agile Manifesto http://www.agilemanifesto.org/ that defines guiding principles of Agile development. I also highly recommend Mary Poppendieck’s books. Mary is the name behind Lean Development. In her books she takes the principles behind Lean Manufacturing developed by Japanese car vendors and applies them to s/w development. In the process, she shows us how to minimize waste, create knowledge, optimize the flow, and respect people. Her second book “Implementing Lean Software Development: From Concept to Cash” is available for review on the web http://www.poppendieck.com/lsd.htm and is an excellent read. I would specially recommend it to people who are responsible for managing software projects.
And, if you want to learn more about SCRUM, visit http://www.controlchaos.com.
Q4: Briefly profile your past senior positions and how you achieved them. Imagine you are providing guidance to IT managers who wants to grow their career. Then describe your biggest career challenges and how you overcame them.
A: My first senior position was with Infonet Software Solutions. In 1997 I was hired as a project manager to work on an enterprise email application. A couple of months after I joined, the company started a new project – web-based email. This was very early in the Internet boom even before Hotmail was released. I saw this as an exciting new opportunity and volunteered to lead this project. With time the online email application was transferred to an online office suite for small businesses and became the lead product for the company. I was wearing multiple hats working as an architect and development lead and was promoted first to the development manager, then to CTO.
As the Internet boom ran out of steam I moved to WebCT first as Senior Director of Product Development, than as VP. Here the main challenge was to learn how to manage multiple projects across two locations and a much bigger team. At that time, (5 years ago), the company had just gone through the first merger; there was a lot of uncertainty and people felt insecure. We have developers in both offices, QA in Vancouver, a UI group in Boston, technical writers in Vancouver, product managers and architects scattered across both offices. My boss is located in Boston and I have people reporting to me in both locations.
My first goal was to make the two offices work well together, build trust and cooperation. We accomplished this by introducing a collaborative process, creating IT infrastructure that supports cross-office development, and making good use of teleconferencing, videoconferencing, and Net meeting. I made sure that people had an opportunity to visit the other office and meet at least once, face to face. We encouraged direct contacts between employees, made people from both offices work on the same projects, synchronized parties and celebrations. There was a lot of travel at the beginning but as we learned how to work together the need to travel reduced. We now do work as one efficient virtual team.
The best advice I can give to people who want to advance their career is to keep their eyes open for new opportunities. If you can recognize an opportunity, are ready to take an initiative and go the extra mile to prove yourself, success will come. Even if there is nothing new and exciting available right now, make sure to tell your manager what you are looking for. When something comes along they will know who to try.
Q5: This is becoming a requested staple in my interviews. Provide your top prediction of future trends, the implications and business opportunities?
A: The world economy is becoming more and more integrated.
Implication: This means that more and more companies are finding out that their customers are located in different countries and regions. For example, WebCT has customers in more than 70 countries and is translated into many languages. In order to serve those customers well you need to localize your software.
Business Opportunity: To be successful internationally, make sure that s/w is designed from the bottom up to be localizable. It is much more efficient to do it from the very beginning than to refactor later on when you need to deliver the Spanish version of your product the next month. While localization is much more than simply translation, let’s cover the basics first.
To be able to translate your s/w you need to make sure that all text and graphics are stored in separate resource files. Your application as well as your database also should know how to work with double byte characters (support the Unicode standard). To do translations you need to engage one of the translation companies such as SimulTrans or Translation.com. When selecting a translation vendor, make sure to evaluate their testing and project management processes in addition to the quality of their translators.
Don’t forget to make sure that display formats of dates, numbers, and currencies in your application are localizable. You also need to support multiple time zones and time saving rules. If you are planning to sell in Arabic countries, right to left support is a must. Localization is not limited to simple text translation. For example, other countries and regions have different taxation and regulations.
When an application’s text, help, and documentation are getting bigger, translation costs may skyrocket and even make it cost prohibitive to enter smaller international markets. There are several ways to control cost. You may be able to reduce the size of materials that need to be translated by using single source publishing where online and printed materials are generated from the same source. You can also reduce the word count by using consistent terminology and cross reference help topics to avoid duplication. If this is not enough, the next step is to develop tools that will allow your customers and regional partners to do translation on their own.
Q6: Which are your top five recommended resources?
A:
- “Innovation Happens Elsewhere. Open Source as Business Strategy” by Ron Goldman, Richard P. Gabriel http://dreamsongs.com/IHE/IHE.html.This is an excellent book written by people responsible for launching and managing open source projects at SUN. It shows how and why for-profit businesses can and should coexist, collaborate, and work together with Open Source Communities.
- I have mentioned already Mary Poppendieck’s books on Lean Software Development.
- “The World Is Flat - A Brief History of the Twenty-First Century” by Thomas Friedman is a recent bestseller that explains globalization and what it means to individuals, communities, countries, and companies.
- Better Software magazine and its online companion StickyMinds.com is a good resource for software developers and managers.
- CIPS web site www.cips.ca
Q7: Provide commentary on three topics of your choosing.
A:
TOPIC 1: We have talked already about the growing complexity of s/w systems and how challenging it is to ensure the quality of software products. Automated testing is a great way to increase quality and make it consistent. I want to share with your readers our experience with it.
It takes more than 33 man/weeks for example to perform the full regression test of the WebCT Learning Management System, which is a highly configurable enterprise online application.
Four years ago, to reduce these manual testing efforts, we started to look at test automation tools. It is relatively easy to start with tools like Silk Test or WinRunner. You record a test script by just clicking your way through an application. This perceived easiness can lead to a trap. As the number of scripts grows you can find yourself spending more time maintaining them than it took time to develop the automated tests. In many companies automation efforts are led by testers who don’t have a programming background. But for automation testing to become successful you need to treat it as a development project and do proper design and architecture.
This was the first challenge we faced after half a year into the automation. We addressed it by hiring experienced automation testers and implementing a data-driven approach where test parameters are stored in the database and guide the test execution. To run tests we set up dedicated machines.
After a couple of years using SilkTest we had automated up to 20% of test cases. This is considered a good achievement in the industry. At that point we became victims of our own success. It was taking a long time (up to 3 days for three people) to run all the automated tests. They also required substantial maintenance for each release as there were changes in UI and functionality. Another problem was an instability in running tests. Occasionally we experienced crashes of the browser or the test tool which made it impossible to reliably run tests unattended. This instability, vulnerability to UI changes, and slowness in the execution was caused by the fact that tools like SilkTest emulate a user clicking buttons and links in a web browser.
The WebCT performance lab was using a different tool – SilkPerformer - that instead of working inside the browser records get and post requests and sends them to the server. We rewrote some of the scripts in SilkPerformer and saw great improvements in script execution time. For example, our smoke test execution time was reduced from an hour and half to ten minutes. It also became easier to maintain as SilkPerformer tests don’t depend on the positioning of UI elements. Everything has its price of course as SilkPerformer will not find a pure UI bug. We still thought that this was a good trade-off in many cases as most of our application logic is implemented on the server.
At the same time we extended our automation team by hiring three junior Java developers and started to implement automation scripts in Java. This opened new opportunities. For example, we implemented a test that emulates hundreds of students taking a quiz at the same time, providing correct and incorrect answers to questions. At the end, the test also checks that grades are calculated correctly. It’s all parameterized and can be executed in a matter of minutes. Some of these test cases are impossible to execute manually.
I want to mention another non-technical problem that we needed to overcome – initially testers were not trusting results of automation testing and continued to do manual testing even after the automated test had been run. To overcome this we started to get manual testers involved in writing requirements for automation work and trained them to execute scripts themselves. The problem was solved completely when we created custom scripts to report results of automation test runs into the same test case management system that is used during manual test execution. Now a tester is able to run a report on test coverage that combines results of both manually- and automatically-executed tests.
To summarize, you can start easily with automation testing by using one of the automation tools. If you want to do more than just ad-hoc scripting start treating automated testing as a development project, define coding guidelines, build a framework, and use a data-driven approach for script parameterization. If the UI of your application does change often, move away from UI-based test automation and start to write test applications in real programming languages that exercise product functionality through internal or external APIs. In all cases, make sure that results of your automated tests are converged with manual testing results.
TOPIC 2: As you know, I’m interested in career development. As a manager I mentored many technical people in choosing the next step in their career. The most difficult move is to switch from a technical position to a management one and I know this from the first hand experience. It is quite scary to stop being a hands-on person.
It is not enough to be good technically to become a good manager. However, having a technical background is a great help in making right management decisions in IT. People who are thinking about advancing their career into management should do it for the right reason. Go for it if you enjoy working with other people, want to be involved in making strategic decisions, do things on a bigger scale, and have more room to implement your ideas.
I advise people who want to move in this direction to take business and management courses, learn project management, read books on s/w development processes, and learn to see the big picture. If you are not sure that a management career path is for you, you can try a taste of it by accepting a team lead’s position, or volunteer to lead a small project.
My interest in the career development led me to found CareerShare.com. Its goal is to create a community where people can share their real life career stories and get support, advice, and motivation from others. The real life is so different from what is written in a college recruitment brochure. People are making fascinating career turns and switches. I believe that collecting and sharing this information will be a great help to those who are just starting their journey as well as to somebody who looks for the next step in professional life.
TOPIC 3: Pragmatism in making trade-off decisions or how to make s/w good enough
Quite often we see overcomplicated products overloaded with rarely used features. I’m a firm believer that business success comes to a company that is good at making pragmatic decisions about what functionality to add to its products and assessing its benefits versus cost of development and maintenance.
We need to realize that most of the cost associated with a particular functionality occurs after it was initially developed. If a developer writes code that is barely used we are still going to pay for it throughout the life of the system which can easily be 10-20 years. As Mary Poppendieck says in her Lean Development book – “Complexity is like a cholesterol, it clogs up the arteries of a code base, as well as an organization; … If we want our code to have lasting value, we will put top priority on keeping our code base simple, guarding vigilantly against the creeping crud that plagues so many software systems.”
The same applies to technical decisions. Quite often designers advocate using the latest and greatest technology just because it excites and challenges them. This may lead to overcomplicated solutions that are not really required to achieve business goals of the application. Let’s keep it simple!
Closing comment: Michael, we very much appreciate your dedication in providing this interview. Thank you for sharing your experienced and deep insights with our audience.
A: It was my pleasure Stephen.