Setting a framework for performance management involves expressing the long term corporate goals and objectives of the road organization in terms of the desired levels of service, and linking these with performance measures and targets that allow the organization to monitor progress in achieving the intended results (PIARC 2004). Therefore, the main components of a performance management framework are: levels of service, performance measures and performance targets (see figure 1.4.4).
As shown in Figure 1.4.4, a number of measures can be chosen to describe the overall performance associated with each level of service.
In developing a performance framework, it is important to understand who is affected by highway asset performances; within this broad group of people, three broad categories may be identified:
The data collection and processing activities, dealt with in section 2.1, must be guided by a coherent strategic vision that specifies what goals and objectives are to be served and what measures will quantify the objectives, as defined by the performance management framework.
The methodology used to collect data, calculate performance measures and establish targets, and define the associated levels of service should be fully documented for future reference. Furthermore, authorities/agencies may develop an approach for communicating levels of service and measures to external and internal stakeholders.
The asset management strategy sets out what the authority intends to do in order to manage its highway infrastructure assets; in developing the strategy, authorities/agencies should establish the mission and goals (asset management objectives). Management objectives usually identify the quality of service to be offered to stakeholders, which reflect the authority’s aspiration/availability and the expectations of stakeholder. Any asset service affecting any stakeholders in their roles as users, taxpayers, partners, elected officials, community residents, etc., may be included.
The quality of services likely to be placed in the future on the highway network should be incorporated in defining the levels of service. Therefore, levels of service are broad statements that describe the performance required for highway infrastructure assets, which are the balance between stakeholder expectations, corporate vision and available funding (see figure 1.4.4.1.1).
At the proficient maturity level of asset management, levels of service can be split into two distinct categories: customers/stakeholders and technical.
Customer levels of service relate to how the customers and stakeholders receive the service in terms of tangible and intangible measures or criteria. They are expressed in terms that customers and stakeholders can understand and comprehend. Tangibles include the appearance of facilities, frequency of service disruptions, availability of service, frequency of crashes, etc. Examples of intangibles include speed of service, staff attitude and ease of dealing with the agency.
Definition of customer level of service
Levels of service are broad statements that describe the performance of highway infrastructure assets in terms that stakeholders can understand. They should relate to outcomes and cover key aspects of asset performance such as safety, serviceability and sustainability. They should consider the performance of the whole network rather than that of individual assets
Technical levels of service support both the customer’s and the agency’s strategic objectives; they are usually expressed in technical terms used by agency personnel or contractors. At the early stages of asset management (basic maturity level), some agencies use their performance measures as level of service statements.
Definition of technical level of service
Levels of service are statements or values of a key performance indicators that describe the performance of highway infrastructure assets in technical terms and are suitable for inclusion in maintenance delivery contracts or internal service agreements. They should relate to outcomes and cover key aspects of asset performance such as safety, serviceability and sustainability. They should consider the performance of the whole network rather than that of individual assets
Each road organization needs to determine the number and nature of its own levels of service, but in any case, levels of service should be developed based on themes that demonstrate the relationship between higher level corporate objectives and any stakeholder requirements for delivery of the service, the most common aspects/themes being (see e.g. law P.L. 112-141 in USA (AASHTO, 2013)): safety, serviceability (congestion reduction, system reliability, etc.), sustainability (environmental, aesthetics), accessibility (e.g. reduced project delivery delays) , financial performance, etc.
Levels of service can be developed through workshops, focus groups, public opinion surveys, collaborative working and market survey research
It should be noted that levels of service need to be developed in a hierarchy that reflects the increasing level of detail of planning activities (see figure 1.4.4.1.2).
Strategic - To provide a snapshot of overall performance, generally for stakeholder consumption;
Tactical - To inform decision making, particularly in terms of investment;
Operational - To provide information on operational aspects of the service, such as the speed of repairs or performance of service providers undertaking maintenance activities.
Each level of service should be supported by a set of performance measures (see figure 2), which will enable to measure both individual aspects of performance as well as the overall level of service.
Two type of measures are of interest:
It is relevant to distinguish between the condition and performance of an asset:
Usually, for each level of service, a number of performance measures are chosen, and the process to evaluate them generally involves the aggregation of raw data (objective and subjective, quantitative and qualitative), as shown in table 1.4.4.2.1.
Definition of Performance MeasureVarious performance measures are usually required to represent a single level of service, but they might be aggregated in a key performance indicator (KPI) to describe the level of service. It may be appropriate to use different weighting factors for each of the performance measures that contribute to a level of service or KPI (e.g. condition of footways may have a greater weighting than condition of white lines).
Performance measures are used to monitor whether authorities/agencies are meeting the levels of service and to report the actual performance. They are usually numbers or scores that have meaning to relevant staff, and are calculated transforming raw data. They may be technical and non-technical
Typically, performance measures may include both engineering and non-engineering considerations (for example based on results from public opinion surveys). There are four type of performance measures:
The steps for identifying performance measures generally include:
The logical starting point for the development of performance measures is to identify what aspect of performance is being measured.
Indicators suitable to be used as performance measures should be:
One of the basic considerations in developing performance measures is to maximize the use of existing data and minimize the need for collection of additional data.
The SMART approach may be used in selecting actual performance measures. This approach evaluates the following attributes for candidate measures:
Once performance measures are selected, it is advisable that:
Typical sources of data and other information used to define performance measures may include: public opinion surveys (letters, e-mails, calls and blogs), business reporting, operational reporting, existing inventory data, performance data (including condition) and financial reporting.
In relation to KPIs, Working Group 2 (Management Indicators) of a past PIARC Technical Committee on Asset Management (cycle 2008-2011), put some effort into identifying the existing management indicators as well as the lack of indicators when necessary (PIARC 2012). Table 1.4.4.2.2 shows some examples of the work carried out by this committee.
Progress in achieving asset management goals and objectives, as described by levels of service, is quantified by target values of the selected performance measures. Therefore, each performance measure should have a performance target associated with it over a period of time, typically up to five years.
The targets should be easily recognizable and understood by senior decision makers and/or the asset manager depending on whether they are strategic, tactical or operational.
The process of defining performance targets needs to be iterative, determining the best combination of targets against what is affordable. In this process, consideration should be given to:
Definition of targets
Performance targets define quantitatively the performance that needs to be achieved in order to reach the required levels of service. They should be set by staff responsible for asset management, agreed with senior decision makers and be achievable and affordable.
The process of setting performance targets may include steps such as the following (NCHRP, 2006):
Examples of performance targets as defined by various US state DOTs are presented in Table 1.4.4.3.1 Table 1.4.4.3.2 shows examples of targets used in Romania. Both sets of examples refer to pavement preservation.
NOTES:
NP1 – very good performance;
NP2 – good performance;
NP3 – medium performance
NP4 - NP5 - poor performance
Traffic expressed in millions of standard axles (msa)