Software Project Secrets: Why Software Projects Fail - A Review

Product Description from Amazon

The first book ever that manages to clearly and concisely show how software engineering is different from other engineering fields and to enumerate the key factors that usually make software projects derail.

— Val's Blog, Stuff for software engineers and Java addicts

Software Project Secrets: Why Software Projects Fail offers a new path to success in the software industry. This book reaches out to managers, developers, and customers who use industry-standard methodologies, but whose projects still struggle to succeed.

Author George Stepanek analyzes the project management methodology itself, a critical factor that has thus far been overlooked. He explains why it creates problems for software development projects and begins by describing 12 ways in which software projects are different from other kinds of projects. He also analyzes the "Project Management Body of Knowledge" to discover 10 hidden assumptions that are invalid in the context of software projects.

The Review

The whole book isn't too thick it's about 166 pages of which 132 pages are the content proper, the rest are the glossary, index and a further reading section. I finished in about 2-3 days.

The thing that I really like about the book is the section on "Why software is different". Its list 12 ways in which software is unique from other types of systems and George Stepanek shows that these 12 characteristics are also why software is also hard to build.

He then follows up by comparing it to the Project Management Book of Knowledge (PMBOK). I have to say first that I have never read PMBOK so I cannot comment on how accurate his assumptions/knowledge of what PMBOK is about is correct. But from my own experience I can say that he is probably right, things like how project management assumes that scope can be defined completely and before the project starts, estimates can be reliably estimated, one developer is the same as another and software activities can be put into some sort of a sequence are just some of the issues that he brought up.

He introduces a short case study of how a software project using traditional project management techniques could cause a project to fail, which of course sets the stage for how to actually manage software projects. He introduces 3 methods that are based on agile philosophy, like crystal, XP and how RUP can be used and ends off with the same case study implementing agile methodology.

My feelings


The chapter on why software is different and the comparison to PMBOK is probably the most useful part of the book. The two case studies are abit fake and short but they do show how things could have been done.

The section on the various agile methodology was probably a bit too introductory, the reader would probably have to find out more on how to implement the various methodologies himself.

It sorts of mentions from time to time how people management and the environment in which the programmers work also do play a part in ensuring the success of the project. I think more in depth coverage of people management and environment could probably make the book more complete.

What is liked was also the The Further reading sections (about 7 pages long) which is quite comprehensive,it contains many references to books and website on the topics that he talks about.

Who is this book for ?

Basically anyone what is doing software project mangement or the user commissioning/in charge of the project. It goes through comprehensively in a few chapters what are the challenges in software developement and how some traditional project management thinking could be wrong.

The Clean Code Talks - Unit Testing

Misko Hevery made a presentation at Google Tech Talks about unit testing. He tells us that “There is no secret to writing tests… … there are only secrets to writing testable code!”. He describes the different category of tests and shows the importance of unit testing. He describes how to avoid writing untestable code.

And what I think the most important take away for me was that he recommended avoiding putting your object creation code and your business logic code together. This is so that you can make use of Dependency Injection to "inject" test data, stubs into the test.

A great presentation for people who are new to unit testing and how to make your code more testable.

[youtube=http://www.youtube.com/watch?v=wEhu57pih5w&hl=en&fs=1]

Slides of the talk

Google Testing Blog page

Howto Auto increment version numbers in ANT

I have been playing around with Ivy for some dependency management goodness to implement in my project. So far Ivy looks quite good (though I am still trying to figure out how to use configurations in my project) and also playing around with setting up my own software repository.

When looking at the depenence sample in Ivy, came across a part where the version number gets updated as the module is published to the repositoriesthought I just post a more generic version that makes it as a Ant target.This would be useful when dealing with automated builds or when using Ivy to post to a in-house repository.

<target name="increment">
<propertyfile file="build.properties">
<entry key="build.number" type="int" operation="+" default="0"/>
</propertyfile>
<property file="build.properties"/>
<echo message="Build number is ${build.number""/>
</target>

What the above target does is that it looks for a <propertyfile> and looks for the build.number property. If both are not present it will create the file and increment the number. So if you do not have the file/property you start off with build number 1.

But the properties are not loaded in ant, that's why you need to load the property file using <property>, then you can reference build.number using normal ant properties notation.

You can test this in an ant script by calling ant increment.

So to use it to make a jar file, you probably would do something like

<jar destfile="${build.dir}/${ant.project.name}-${major}.${minor}.${build.number}.jar">....</jar>

Correct Usage of Source Code Control Software

Source Code Control (SCC), everyone probably used it or abuse it. As a new developer 2 years ago, I was assigned to a project, it was my first time being in a project and coincidently it was also the first time most of the project members were using SVN (They mostly use Visual Source Safe 6). So here are some of the issues that I think are relavant for "newbies" like me to SCC. I am not going to talk about specifics like actually how to do things like checking in and out code,  as there are many resources on the web that cover the same thing.

1. RTFM

Each source code control has it's own benefits and idiosyncrasies that we all have to live with. Read the manual on not just how to do things like commit and update. But also how to do branching, tagging and merging of source codes and most importantly how to roll back.

Don't try to "cheat" the source code control unless you know what you are doing. Like when i first use Visual Studio 2005, we use Visual Source Safe (VSS) 6 as the SCC because of the integration between VS and VSS. But Visual Studio had one annoying feature, it must be connected to the VSS server in order to work properly. So if we unplug our developement machine from the network and start up Visual Studio, it sorts of "hanged" while trying to connect to VSS. It can't really something like SVN where you can just update, unplug  and work on you code then commit it. VSS requires you to "check out" the file so others can't use it, the way it makes the other users not able to write to it is by setting the "read only" flag on the file. So we had a developer who needs to work offline and we had a lot of trouble getting all the files in the SCC in sync and that developer had to manually change the "read only" flag to allow us to write code, but when the actual owner checks in all the changes are replaced again. And when we were on a dead line rushing code out, trying to handle these problems was just time consuming.

2. Source Code control is for source code!

I remember that when I first learnt java in school, it was with emacs as the editor and batch files for compiling. Now we have super IDE's like Netbeans and Eclipse that has nice syntax highlighting and even javadoc help built in.

We had a problem using Eclipse and SVN together,  the problem was that most modern ide's they tend to have some custom folders/file that they use to store some of the ide's customization like where are the javadocs located, the jdk/jre location, some font and color settings and so on. Imagine that you created you own beautiful environment and you check in those settings to the source code control, and you team member also check in his customizations. Imagine the havoc when you update the files and found that the jdk is pointing at a wrong directory and the classpaths were all wrong. My project that time we had a lot of problems trying to get all these sorted out and it plainly just waste time. So don't waste time like me just do the following.

So remember Source Code Control software is just that use it for source code. Check in only the source code (src/test) not the compiled binaries directory, not the .nbproject / .metadata directories just the code.

You might want to have a structure for storing the code, a common one is

src is for source, test is for unit tests files and lib is for the libraries that you are using. Thats probably the most basic directories that you will need.

Whether you will use source code control for storing your libraries is up to how your project wants to manage it. There are people who use tools like Maven to handle the dependencies. My work set-up is such that the development network and the internet are not connected, so for Maven to work, we will probably need to set up some kind of local repository and we haven gotten down to that or really see the need for that yet. For now we just commit the dependencies into the SVN. We did not encounter much issues, except we must be care not to introduce versions of the same libraries with different names (e.g log4j-1.2.10.jar and log4j-1.2.15.jar in the same directory, so which one is the one we are using?) into the build.

3. Read about how people organize their code and learn best practices.


Learn about the best practices in using that particular source code control software, and how to organize/manage your code like creating trunk, branches and tags. Learn how to branch out you code to do modifications/test concepts and then merging it back to the main source tree. Learn how to version software using tags, so that you will have some sort of a release tree that you can keep track of the code at various stages. I recommend reading Pragmatic Programmers series of books on version control.

Afterthought

For new programmers it pays to know a bit about more about SCC software and how to mange source code, I don't think they really cover this at school (I know mine didn't but that was some time ago). So it good to read an play around even for you own personal projects. Source codes are the fruit of every programmers work so don't let managing it or using SCC software be a bottleneck or even the source of fustration in your work.

Using CCTray to monitor CruiseControl Java

For the past few days I have been setting up my project's build environment using a both CruiseControl and CruiseControl.NET.

We needed a tool for informing us what is the status of the builds in the server. I tried configuring CruiseControl's jabber publisher. But somehow it didn't really work no messages were received although there were no errors. So ended up using CCTray that came with CruiseControl.NET for CruiseControl monitoring.

The url provided in the custom url portion of Add Server gives the wrong url for connecting to a normal CruiseControl. The correct URL is as below.

http://your.cc.server:8080/dashboard/cctray.xml

Now we had a uniform way of monitoring all builds from both the .NET version and the Java versions of CruiseControl.

First Post !

Why this blog ?!?

This is hopefully the start of series of post where I would write about my work in doing software projects both from the software development process as well as the technical/programming issues. And since I am about to embark on a new project (and hopefully I will stay till the end, my department has a habit of moving people around), I feel there is some need to document down the problems, thoughts that might happen during the course of development.

And the place which I work is also a bit interesting/very common ((I dont have the expereince to tell the difference). We do have a lot of processes (means lots of documentation/paper work) before our projects are formally approve and funded. We dont seem to have a process for developing software at a lower level. We have documentation that talks about Risk Management, Issues that crop up during project, Testing plans and so on. These are good and are necessary but what we lack is a framework for actually developing the product, things like timeline, things that programmers must do (unit test's, checking in often, coding conventions, update meetings) are not really spelt out even in individual project teams.

The previous team in which I was in (I joined after the project more or less started and we were gathering requirements) we were managed in a sort of 'loose' way (the developers more or less decided what was to be done next), we only spoke to the project manager occasionaly when met with design issues or to clarify user's operations. We had a lot of trouble integrating each other's work just before demos to the users (we were using SVN but not using a build server). we even had trouble using SVN, each time we update people's IDE profiles were over written (we were using eclipse and the individual developer settings were also checked in causing havoc whenever we updated).

So you can see why I am interested in the whole developement process from start to end, hopefully as I start on my next project. It would be a better one :)

About Me

I do not profess to be any sort of a guru be it in management or programming (I have only been a programmer for 2 years plus, although I have been working for about 6 years in total, 4 of which in a totally different field). And there are much better blogs out there on Software development issues (Joel on Software, NOOL .. etc), so treat this as my point of view, and hopefully someone will comment and feedback.

southpark

My avatar

About

About Me

Stephen works as a programmer/developer (duh). In his free time he likes Scale Modeling, photography.

Why the blog ?

Cause he is interested in how people build software and always tries to find better ways/practices to help him. This also serves as a sort of development "diary" to document his thoughts/knowledge gained as he is about to embark on a new project.

What he is using now:
Language:Java, C#
Frameworks/Libraries: Spring MVC, Hibernate, ExtJS, JAX-WS
Server: Weblogic 10.3, Apache
IDE: VS2005, Netbeans

What he knows:

Language that he is comfortable to use: Java, C#, C, C++
Languages that he used to know but don't use much: COBOL (Only during school time), LISP (Trying to pick it up again), Visual Basic, Basic
ORM: Hibernate
Server: Tomcat, Weblogic 10.3, Apache
IDE: Eclipse, Netbeans, VS2005

Attended:
Weblogic 9/10 Administration Course
Foundstone Building secure software
Software Quality And Testing at Nanyang Polytechnic

Some other useless details

Scored better than 91% of all test takers at BetterProgrammer.com (link to cert)