1 Greg Edit Problem Definition

Appendix 2.16 questions checklist for investment decision-makers

Appendix 3.Benefit framework

Each time an organisation makes an investment there is an expectation that some form of benefit will be returned. It is critical to design investment solutions that provide maximum benefits and to be able to confirm that the expected benefits were delivered.

Why is it then that very few investments are able to articulate the benefits they will provide, to define how they will be measured or to actually measure the benefits that are ultimately delivered?

In taking a fresh look at the problem of benefit management, it seems that within large organisations there has been an inability to define how each individual investment contributes to the primary benefits that are the point of the organisation’s existence.

It is typical and appropriate that everyone within, say Victoria Police, believes their individual investment will provide the benefit of ‘reduced crime’ and everyone in Education will claim theirs will result in ‘better learning outcomes’. But until now there has been limited ability to describe the contribution of an individual investment to ‘reducing crime’ or to achieving ‘better learning outcomes’.

The benefit framework that is depicted below has evolved, been trialled and found to be effective at addressing this longstanding problem.

The framework is a three-level structure that links the contribution of an individual investment to the outcomes the enterprise is seeking.

In the example depicted here, at the enterprise level, the government is seeking to create ‘friendly, confident and safe communities’. To this end they set benefits and targets that must be met at the organisation level – in this case the police are required to ‘reduce crime’.

At the investment level it is necessary to demonstrate how a particular investment will contribute to the benefits sought by the organisation. In the fictional example we are using here, the team of forensic scientists are seeking an investment to acquire state-of-art forensic software and to renew their aged computer system. In return for this investment they claim they will ‘reduce crime’. Their evidence to support this claim is that they will reduce the time it takes to provide forensic matches by 30 per cent and obtain 20 per cent more forensic matches.

The head of the police organisation is then left to decide the following:

How significant will meeting the targets associated with these KPIs be to a reduction in crime?

Would the claimed KPIs and their targets be directly attributable to the proposed investment?

Would the likely impact on crime reduction be worth the cost of the proposed investment?

This benefit framework is used as the basis for considering the validity of potential benefits during the development of ILMs. It has also been used during the development of more than 60 benefit management plans, many of which are now tracking the delivery of benefits.

Any KPI selected must be meaningful, attributable and measurable.

Appendix 4.Design guidelines – Investment Logic Maps

The function of Investment Logic Maps (ILM) is to portray the logic underpinning a potential investment on a single page and in a form that can be quickly understood by decision-makers. Already several thousand ILMs have been developed. On the basis of this experience those things that work well have been identified and they are provided in this design guideline. Use this guidance to develop your ILM. They will help to ensure that your map is an effective communication tool that articulates the argument clearly, exposes the logic of your argument and identifies the value proposition inherent in this investment.

Item	Practice	Reason
Template	Always use the current ILM template. Current templates can be found at www.dtf.vic.gov.au/investmentmanagement.	Formats continue to improve to make them more useful as communication tools, containing better information.
Problem statements	Maximum of four problem statements.	Forces decisions as to what are real problems and reduces unnecessary complexity.
Benefit statements	Minimum of four benefits – six interventions.	As with problem statements.
KPIs	Two KPIs must be included for each benefit.	Without strong KPIs the benefits the investment is aiming for are unlikely to be delivered. It also focuses the group on determining the most important KPIs that this investment is going to achieve.
Interventions statements	Maximum of four interventions.	To demonstrate a wide spread of strategic thinking.
Number of words	The ideal number of words per box is between eight and 12.	It forces you to be precise in the words you chose to convey your message. It also maintains clarity and consistency when reading an ILM.
Size of boxes	Do not alter the size of the box or the font size.	Retains simplicity and forces the use of plain English.
Connections	No more than two linkages from any element in one direction.	Retains simplicity and creates the need to prioritise what really matters.
Use and distribution of percentages (%)	A total of 100 per cent to be distributed within each column. This is distributed to indicate the relative importance of the various elements within each column. Elements then re-distribute their percentage to the boxes to which they are connected in the following column.	Acts as a tool to extract judgements on the relative importance of the identified problems, interventions and benefits.
	When distributing percentage, no two elements should be given the same weighting (e.g. split problems 50:50).	Forces decisions of relative importance.
	Elements rated less than 10 per cent should be questioned regarding the need to mention them at all.	Each element and its connections add to the complexity. The aim is to capture those things that really matter and eliminate the rest.
	No more than two KPIs per benefit.	Ensures simplicity and priority KPIs are identified.
Completing the control fields	Ensure the document control fields have been completed.	Provides legitimacy and accountability to the investment.

Appendix 5.Facilitator feedback form – Problem Definition workshop

Critical to the success of the Investment Management Standard is the ability to have an intelligent facilitated discussion focused on the logic that is underpinning a potential investment. The capabilities of the facilitator are key to this discussion.

You have just completed a Problem Definition workshop and we would like your feedback on how well the workshop was facilitated.

How would you rate the truth contained within the following statements on a scale of 1 to 5?			SCALE FALSE (1) – TRUE (5)
At the commencement all participants were given a clear understanding of the role of the facilitator and the outcomes sought by the discussion.
The opinions of the key participants were obtained and properly considered.
The difficult questions that were pivotal to the success of this investment were identified and properly addressed.
Hard evidence was sought to validate each statement of the investment story.
The discussion concluded with the agreement of participants that the investment story that was documented was consistent with the discussion.
The workshop was completed within two hours.			Yes / No If no, for how long did it run? (__ hrs, __mins)
Date of the workshop:
Department / organisation:
Size of investment:	Less than $500k	$500k – $10m		Above $10m
Your name:
Your position:
Would you be happy to be contacted by other ‘investors’ seeking further information on the capability of this facilitator?			Yes / No

Appendix 6.Quality assessment form – Investment Logic Map

The purpose of an ILM is to clearly and honestly communicate the case for an investment. Whether the case for the investment is weak or strong then becomes a matter of judgement for the reader. There are five tests an ILM must pass to be considered of an acceptable standard:

Test 1: Could a layperson read and easily comprehend the story of this investment to the point where they could have some opinion of it?	Assessment: YES / NO / MAYBE
Please explain why you have assessed it this way.
Test 2: Is each problem a ‘call to action’ that conveys what is broken (both the cause and effect)?	Assessment: YES / NO / MAYBE
Please explain why you have assessed it this way.
Test 3: Is there a logical connection between the effect of the problem and the benefits and their KPIs?	Assessment: YES / NO / MAYBE
Please explain why you have assessed it this way.
Test 4: Is the strategic response one that: is likely to deliver the expected benefits and KPIs; allows for more than one project option; and seems to be a valid response to the problem(s).	Assessment: YES / NO / MAYBE
Please explain why you have assessed it this way.
Test 5: Does the solution read as a set of logical and sensible business changes and assets that need to be undertaken to adequately deliver the strategic response?	Assessment: YES / NO / MAYBE
Please explain why you have assessed it this way.
How would you rate this ILM?	Assessment: SATISFACTORY/ UNSATISFACTORY
Please explain why you have assessed it this way.

Appendix 7.Sample agenda – Problem Definition workshop

Problem Definition workshop for a low-complexity investment (single workshop)

5–10 minutes	Introduction and outline of purpose and role
10–15 minutes	Investor overview and venting
45–50 minutes	Problem statements development
15–20 minutes	Benefits and KPIs
10 minutes	Strategic response
10 minutes	Changes and assets
5 minutes	Conclusion and next steps

Problem Definition workshop for a higher complexity investment (two-four workshops)

5–10 minutes	Introduction and outline of purpose and role
10–15 minutes	Investor overview and venting
50–55 minutes	Problem statements development
20–30 minutes	Benefits and KPIs
5–10 minutes	Strategic response
5 minutes	Conclusion and next steps

Appendix 8.Sample email – before the Problem Definition workshop

Investor email outlining the purpose of the discussion to the participants.

[Greetings]

[something about the proposed investment]

As a way to consolidate our thinking about this initiative we will be holding a Problem Definition workshop with the aim of properly understanding and articulating the need for this investment. You are invited to help define this problem. The first workshop will be held:

Date:

Time:

Venue:

The two-hour discussion has been designed to extract the compelling logic that underpins a potential investment or program and develop an Investment Logic Map. On a single page, an Investment Logic Map tells the story of an investment. An example is attached.

The Investment Logic Map that is created evolves and forms the basis of all subsequent investment decision making and ultimately is used to measure the effectiveness of the investment.

The discussion will be facilitated by [facilitator’s name], who is an accredited investment management facilitator.

No preparation is required before this workshop as it is assumed that you are already a subject expert and understand the problem. The role of the facilitator is to extract the story of the potential investment from the group in the form of an Investment Logic Map.

If you would like to read more about the benefits and practices of this approach you should refer to the DTF website at www.dtf.vic.gov.au/investmentmanagement.

[Sign off]

Appendix 9.Sample email – after the Problem Definition workshop

[Greeting]

Thanks for your participation at yesterday’s Problem Definition workshop for [investment name]. I have attached the Investment Logic Map that we produced and ask that you provide me with any suggested amendments by close of business (CoB) today.

What happens over the next 48 hours is just the conclusion of our discussion. As such, there is no need to circulate this draft to people who were not present in our discussion as their comments would lack context – they will be free to make comment when we have completed this iteration of the Investment Logic Map.

You should copy your responses to all other participants so we continue to understand each other’s perspectives. Based on your responses I will make any necessary changes and have version 1.0 to you by CoB tomorrow.

… my observations

To assist your input I make the following observations as to the relative strengths and weaknesses of the ILM we produced:

[for example]

Both the problem and the strategic intervention are generally strong – they speak well and there is strong evidence to support the drivers.

While benefit one would be powerful I am not sure that it can be supported by a KPI that is attributable to this investment. It probably needs to change to ‘[rewording]’ and could then use ‘[name of KPIs]’.

… about Investment Logic Maps

An Investment Logic Map is never finalised. It is an evolving document that tells the story of an investment at any point in the investment lifecycle. Its strength is measured by its ability to be easily read by anyone who can then understand why an investment is being considered (or is underway). The reader should be able to suggest how it might be reshaped to deliver a better outcome.

If you would like to read more about the benefits and practices of this approach you should refer to the DTF website at www.dtf.vic.gov.au/investmentmanagement.

[Sign off]