IET
Decrease font size
Increase font size
Topic Title: Do you measure your software?
Topic Summary:
Created On: 10 April 2003 11:39 PM
Status: Read Only
Linear : Threading : Single : Branch
Search Topic Search Topic
Topic Tools Topic Tools
View similar topics View similar topics
View topic in raw text format. Print this topic.
 10 April 2003 11:39 PM
User is offline View Users Profile Print this message



blondeix

Posts: 13
Joined: 08 July 2002

"How much software have I produced this month?", "Was my investment in this project justified?", "and this new project? is it feasible within my budget?". Those who have heard about the standard COSMIC-FFP (ISO 19761) and the tool MeterIT-Cosmic know the answer to these questions. But for the others, how do you measure your software size? how do you manage your software assets? how do you control your software costs? how do you control your suppliers? We would like to hear from you in order to gain your insight into these issues.

Bernard Londeix
http://www.telmaco.co.uk

-------------------------
Bernard
 11 April 2003 11:17 PM
User is offline View Users Profile Print this message


Avatar for KennySharp.
KennySharp

Posts: 465
Joined: 29 September 2001

Call me a cynic, but isn't this near-blatant advertising?

-------------------------
Kenny Sharp BEng (Hons) CEng MIET (MIEE)
There is no better place to find good advice than from people who have gone through it before.
 12 April 2003 08:58 PM
User is offline View Users Profile Print this message



blondeix

Posts: 13
Joined: 08 July 2002

We are very interested in hearing from members who use or would like to use software measurement to enhance their projects performance.
Bernard Londeix

-------------------------
Bernard
 20 June 2007 11:56 PM
User is offline View Users Profile Print this message



blondeix

Posts: 13
Joined: 08 July 2002

Four years after the questions above are still relevant for the engineering community. How do you control your software production if you do not measure it with up to date measurement methods????
Is questionning still perceived as advertising, to day?

Bernard Londeix CEng MIET

-------------------------
Bernard
 15 February 2010 07:06 PM
User is offline View Users Profile Print this message



blondeix

Posts: 13
Joined: 08 July 2002

Another three years and still trying to give a chance to my IET colleagues who are interested in embedded software. Has anybody got experience, for instance, in the software connecting the brake pedal to the brake pads (of any car)? Anybody would like to participate in a case study?
Bernard Londeix CEng MIET

-------------------------
Bernard
 09 March 2010 09:21 PM
User is offline View Users Profile Print this message


Avatar for lowson_i.
lowson_i

Posts: 130
Joined: 23 June 2008

I've have a rgh good think about my answers and here they are:

"How much software have I produced this month?"
None. But I've written a lot of code that can be compiled by the release process.

"Was my investment in this project justified?", "and this new project? is it feasible within my budget?".
Surely this is dependant on the decisions made during the financial feasibility and study of the cost-benefit analysis coupled with the establishment of a method of caculating value, preferably by some discounted cash flow method.

Those who have heard about the standard COSMIC-FFP (ISO 19761) and the tool MeterIT-Cosmic know the answer to these questions. But for the others, how do you measure your software size?
In kBytes?

how do you manage your software assets?
Aren't assets tangible and software a logical conclusion so is a software asset a tangible, logical conclusion or an oximoron?

how do you control your software costs?
With a budget?

how do you control your suppliers?
Closely?

We would like to hear from you in order to gain your insight into these issues.
Consider it done!

-------------------------
Ian Lowson MIET

Do or do not, there is no try!
 12 May 2010 12:10 PM
User is offline View Users Profile Print this message



blondeix

Posts: 13
Joined: 08 July 2002

Hi Ian,
Thanks for your reaction. I recognize a certain "state of practice". For instance, measuring software solely in Kbytes is usual practice in some areas. I started with that measure a few years ago. This was until I discovered that this measure was an "output-measure" of the development process, hence, it was not an "input-measure". Therefore it was not useful for me and my management when estimating projects (and setting up a budget).
We need to recognize that (one of) the main purpose of the software is to make functionality available to its user. The standard COSMIC (www.cosmicon.com) makes the measurement of the functionality possible very early at (or before) budgeting time. This "input-measure" (or metrics) makes project estimation more realistic hence cost saving feasible, without dramatic impact on the budget, the team, and other usual project troubles.
Best regards,
Bernard Londeix MIET, MBCS, MAPM

-------------------------
Bernard
 12 May 2010 01:49 PM
User is offline View Users Profile Print this message


Avatar for lowson_i.
lowson_i

Posts: 130
Joined: 23 June 2008

The user doesn't interact with "the software". They interact with an application.
Thinking of it in Lego terms...
My happy son really would really like to play with a car, the doors need to open and the wheels need to turn. He sees a Lego car and really wants it. So I go to the shop and buy what he wants. I get home and know that it needs to be built. I have no idea how long it will take because with all the pieces it looks complicated! "Is it ready yet?" "No. It'll take me about an hour!" I follow the instructions and put the pieces together. Checking what is being built against the instructions. Once it's the doors are added I check they open. When the wheels are added I make sure they turn. Once it's all finished I give it the once over and check that it's not about to drop to bits. My son is really happy with it until a piece falls off. I mend it. He's happy again. Now he wants to add wings to it to make it fly. I like Lego, my son likes Lego!

or
McHappy & Sons Ltd really want a new application that'll help them. They need it to be for Windows and to really turn heads. They are told by a consultant that .NET is in vogue. So they put out a tender to provide this application. I win the contract and start the development process. I have no idea how long it will take but from initial analysis of the component parts it looks complicated. McHappy & Sons would like to know when it'll be ready. Using stepwise estimation techniques I estimate it'll be about a year. A specification is developed and agreed. This will form the structure to which the application will be followed.
Each of the modules when constructed to the specification meets the requirements. The software is built in increments making sure that as each module is added it meets the specification by testing it. Once the development is finished it's given a few tests to make sure it won't fall apart. McHappy & Sons find a few issues which are fixed. McHappy & Sons now want to extend the application. I like .NET, McHappy & Sons like .NET applications as well.

-------------------------
Ian Lowson MIET

Do or do not, there is no try!
 14 May 2010 12:16 PM
User is offline View Users Profile Print this message



blondeix

Posts: 13
Joined: 08 July 2002

Hi Ian,
I can see you have the software development process well in hand(s). It seems that "interacting with an application" is what I mean by 'being supported by a functionality', that is what the software application can do for us as users.
COSMIC is useful from the moment when McHappy & Sons Ltd put out the tender. From the top level specification (What do you really want to be able to do with this application, as a user - mouse and keyboard?), COSMIC enables to quantify the amount of functionality (on screen, mouse, keyboard, and others) to be produced and estimate the development project. And later, to control the amount coded, tested and accepted (or not) by McHappy & Sons Ltd.
It works with any development capability, .NET (Yes, it's a good one), Ada, J2EE, J2ME, on mainframe or embedded. This is because we can measure the functionality (i.e. what it does for the user) and monitor it through out the development.
All the best with your son,
Best regards,
Bernard

-------------------------
Bernard
 14 May 2010 12:24 PM
User is offline View Users Profile Print this message



blondeix

Posts: 13
Joined: 08 July 2002

Sorry, I was confused by the webmaster. I am re-positioning here.
Dear webmaster, Thank you for your interest in software measurement.
I sympathize with the view that you are implying, this view being that software measurement is a concern for quality. I like Phil Crosby's definition of quality as a compliance to requirements and the later formalized by a specification. More effective software measurement methods is a condition for reaching a better quality level because the measures thus made available would be more adequate and accurate, hence more useful for decision makers. At this point I agree with you, safety is also important for the delivery driver, but also security, cost, time, etc.

I haven't read the IEE books you are referring to. This might happen if you give me the reference to an electronic copy of these books. We probably have to recognise that, in line with other high level standards and guidelines such as CMMi, ITIL, Prince 2, SPICE, etc., they would simply recommend to apply appropriate methods of software measurement and project estimation leaving to software people the discovery of the technique they need. These documents/books would only describe the state of the practice with the implied recommendation "do not do worse than that", this is a stop-gap.

With the freely available COSMIC (www.cosmicon.com) method, which is also an ISO standard, we are showing the way forward, that is the "add-in" on top of the books mentioned above. Most of the physical sciences have developed their methods of measurement (SI Metric system) and apply them. The software industry needs to do similarly. We can measure the quantity of CO2 or the volcanic dust density in the atmosphere but we are rarely measuring the quantity of functionality delivered by a piece of software or intended to be built by a software project.
By only measuring the "useful results output" we are simply wasting a number of opportunities for improvement in software engineering. This includes the opportunity for the manager to have it "right now" (most of the time). We can do better than that, and that's what I am proposing to do.
Feel free to continue this discussion on the IET Forum, or LinkedIn, or Viadeo, and some others.
Best regards,
Bernard

-------------------------
Bernard
 14 May 2010 12:26 PM
User is offline View Users Profile Print this message



blondeix

Posts: 13
Joined: 08 July 2002

Thanks for the reference, I'll do a search for it and report back.
Best regards,
Bernard

-------------------------
Bernard
 21 July 2010 09:40 PM
User is offline View Users Profile Print this message


Avatar for virnik.
virnik

Posts: 24
Joined: 06 April 2009

The answer to the question of do you measure your software? Is yes.

Like any other well run business, this should be part of your ISO9001 quality system. I would do the same if I was building walls for a living.

Software development isn't difficult to estimate, provided it is sold correctly. I am a great believer in Agile techniques, as these enable you to provide working software from a very early stage in the project. If the software part you are creating is small enough the estimation of time to complete the work will be more accurate. If the development of software is sold correctly as a service rather than a product, then there should be no problems with meeting customers expectations. Budgetary quotes for customers should be based on costs with some contingency, but software is like art, there is always some feature to be added or some change to be made, it is only really finished when the customer says they are happy with what they have, which can only be judged by the customer.
Best Regards
Nick
 23 July 2010 10:55 AM
User is offline View Users Profile Print this message


Avatar for lowson_i.
lowson_i

Posts: 130
Joined: 23 June 2008

Software is difficult to estimate and CANNOT be automatically estimated.

Let me elborate with an article I wrote elsewhere...

Overview
This article was written in response to a failure in estimations given on a software project. On wider reading about the subject of estimating, where it sits in a project and particularly "formal" estimating for a software project. This should give the reader some ideas about the estimating process and provide a platform for wider reading.


Organising for Estimating
There is a cliche that will score high on buzz-word bingo, "fail to plan, plan to fail!" Thus setting some time aside to estimate will be time well spent. There has to be a decision about who is going to estimate and a choice can be made on two distinct roles:
- an expert estimator and
- a domian expert (someone who understands the subject area).
Often the person carrying out the work produces the estimates. It is possible that these people, though highly competent professionals, may be insufficiently skilled in estimating, which will add an element of risk to the estimate. However, whoever performs the estimating, it is extremely important that this complex activity is organised: it should be planned and carried out as though it were a project.
Planning for the quality of estimates
Some questions you can ask are:
What is an estimate?
An estimate is a document that ends in a figure!
What do I hope to achieve with my estimate?
Hopefully to obtain a reasonably accurate idea of the resources and timescale with associated costs. However, the estimate produced my also be used as a vehicle to 'test' the FDS.
What are the requirements on which the estimate is based?
In the "Resource Manager" product team, the estimates are based on the FDSs produced. One thing to bear in mind is whether the FDS is considered complete. If the answer is 'no' then the estimate may be of no use.
Is there a formal estimating process? If there isn't should there be one?A formal process recognises, organises and perhaps automates the distinct steps in the estimating process; it helps to produce and measure quality!
Have I been taught how to estimate?
Many people who are likely to carry out estimation do not any training in how to do it. The mere existence of a formal process is insufficient if people do not understand its terminology or how it is to be used.
Is my estimate reviewed?
By doing so, it is possible to identify and remedy errors of omission and execution. A good test of an estimate is whether another person can replicate it's outcome.


Accuracy and precision
There is no such thing as a perfect estimate but there are several causes of a 'poor estimate' including:
- inexperienced estimators
- lack of continuity before and after the initiation of a project
- bad technical guesses due to unforeseen circumstances later in the project
- changes to the project after agreeing the estimate
- psychological factors (unchecked estimator optimism for one)
- politics, where stakeholder power affects the decisions that are made
Estimation can be interpreted as the shift from qualitative to a quantitative view of the project.The process of estimating produces some numbers to work with, after answering a number of questions.
Accuracy and precision are two important concepts in estimation. Accuracy refers to the closeness of a particular value to the true or accepted value. Precision refers to how close together a group of measurements actually are to each other. Quite simply accuracy can be determined by a single value and precision can only be determined by multiple measurements.

Uncertainty and variation in estimation
It has already been noted there is a degree of uncertainty inherent in any estimate. As we are now taking a quatitative view it is possible to apply statistical methods to the estimating process. Without going into too much detail it is possible to use PERT and apply a normal distribution curve for estimating errors based on producing three estimates:
- optomistic duration
- normal duration
- pessimistic duration
Defined as:
Te = (To + 4Tm + Tp)/6
Where:
Te is the expected duration, To is optomistic, Tm is normal and Tp is pessimistic.
However, if estimates a generally optomistic it is possible to modify the weightings:
Te = (To + 3Tm + 2Tp)/6

Approaches to estimating costs and time
There are several approaches to estimating costs and time. They are not mutually exclusive and can be used to gather data for a single estimate:
- reliance on the judgement of the person doing the estimate
- breaking an item down into smaller components parts and estimating each component
- relying on data from one or more similar projects done in the past
- using standard data for a task or component
Judgement
The judgement of the estimator is one of the most common methods of obtaining values. This relies heavily on the experience of the estimator and should be rigorously subjected to tests of reasonableness and consistency. The estimator should be able to justify the use of judgement in each case and the basis for considering the judgement should be a reasonable one! However, the larger the piece of work estimated by judgement alone, due to inherent uncertainty, the less reliable the estimate! It is possible to build a concensus to deal with variation and bias in estimates by a group using The Wideband Delphi technique.

Component Method
The work package is broken down into features and then further into distinct elements of work. Each element is estimated in turn, which is then summated into the feature estimate, which is then summated to give a value for the work package. Although this sounds tedious it is invaluable in helping people:
- estimate if they have not done so before
- develop experience needed to use the judgement method
- provide a useful discipline in organisations which have a poor history of estimating accurately
Similar project method
Deciding if this is similar project and basing it on those, obvisously though how similar is subject to judgement.
Standard Data
For each task there is a standard set of data to draw on for estimating. For example, to add a caption to a form takes 5 minutes, and if there are five captions it will take 25 minutes etc. This requires a large amount of data to be collected over a period of time and the exact task being carried out a number of times before.


Estimating Software
Software development is regularly associated with research and development in the sense that:
- every piece of software is a unique endeavor
- there is no single standard measure by which software quantity and quality can be judged
- there exists enormous variation among individuals who design, write and test software as to how much they can produce in a given time and to what quality
- such measures as exist are subject to wide variation even amoung experts
In the past, software was developed for computers that were operated independently. At the end of the 20th Century, one of the major consequences has been the need for interoperability between software applications. Component based development has been championed and increasingly adopted as a development approach.
Modularity in software
The development and operation of software systems can be viewed as having five quantifiable aspects:
- scope
- size
- complexity
- cost
- elapsed time
The first two can be derived from the specification. The complexity of the project can be identified by examining its technical feasibility and identifying a low-risk approach to the development and implementation of devliverables. Complexity exists in two different dimensions: that of the construction and that of its functionality. For each project, it is necessary to know the scope, size and complexity before costs and elapsed time can be worked out for an estimate!
Stepwise estimation
The process of stepwise estimation is derived from the technique of stepwise refinement. What is to be estimated is decomposed into finer and finer detail until the decomposition reaches a state in which quantity can be estimated because the scope of each piece has been reduced to something easily comprehensible and therefore more easily measureable.
If a project group is experienced with a particular application, or with tools and techniques of development of a particular type of software to be produced and the stepwise decomposition technique (preferably both), estimates can be drawn up based on the experience within the project group. It is an unsuitable method where there is little experience of the application or type of software or the tools and techniques.
The decomposition of the software problem proceeds with the express goal of arriving at items which, within themselves, are well understood and therefore measureable and of low risk. However, it should be noted that risk arises when even the best-understood pieces of software must interface with each other, because interfaces increase the complexity of the problem several fold. Where there are software interfaces that are novel, large or difficult they are impossible to judge, other than to recognise that the problem exists and will be novel, large or difficult!
Prototyping, piloting and Evolutionary methods
Prototyping and prototypes
Prototyping consists of implementing in some form some key features of the envisioned system in a simulated operation environment which can be used as a model to:
- evaluate complex and difficult algorithms
- validate design techniques and tools
- design test facilities
- assess the model in use to obtain feedback for input to the design or the estimating process or both
- assess aspects of system integration
- persuade suppliers and users to accept the solution
Prototypes can also help in the reduction of risk by allowing team members to gain experience and by giving feedback that identifies particular risky parts of the design or problems of integration and operation.
Piloting
Piloting is slightly different to prototyping. A pilot implements a version of the software which provides the basic features required in the finished product in a real operational environment. A prototype can become a pilot so the distinction can be blurred!
Evolutionary
Evolutionary development may be the best way of proceeding with software development in a high-risk situation. The pilot may be used as the first phase in the evolutionary development. Evolution seeks to provide a significant benefit early in the development, even if the orignal objective cannot be achieved.
Summary
Rather than develop a complete estimate before doing a system definition and starting development, both the prototyping and evolutionary approaches allow a limited amount of work to take place in order to make an early decision on the viability of the approach being used. A macro estimate then emerges early as a result of such an assessment of feasibility. Following the macro estimate, detailed requirements specifications and system design take place, and the detailed estimates and project plans are then based on the system design. Each stage should be subject to a quality review. The estimate for the software needs to be updated when the software has been designed.


Statistical methods of estimating software
Statistical techniques and and modular software development methods are the nebulus of statistic mathods of software project estimation. In order to obtain samples, something has to be designed and written which is representative of the entire project or of a class of objects such as programs or databases which form a part of the project. STepwise refinement can be used to identify the lowest-level modules in the hierachy; samples of characteristic modules at this low level are then designed and written. and the actual experience of design and writing then yeilds values used to estimate the entire project.


Functional Point model of estimating
The functional point model uses the number and complexity of the functions to be delivered to estimate the development effort. There are five steps:
1 - Categorise the functions according to type and complexity and estimate the number of functions in each catergory
2 - Multiply the number of functions in each category by a complexity weight based on the one of five difficulty levels to calculate the functional points in each category.
3 - Total the number of function points for all categories
4 - Assign values to factors that influence the project and total them
5 - Compute the months of effort required by using an algorithm that relates effort-months to function points and influencing factors.
Each organisation needs to develop its own specific function point model based on its own history to make an accurate estimating tool.

References and citations available on request!

-------------------------
Ian Lowson MIET

Do or do not, there is no try!
Statistics

See Also:



FuseTalk Standard Edition v3.2 - © 1999-2014 FuseTalk Inc. All rights reserved.