Evaluate and learn: The Evaluation Cycle - Government Communications

Introduction

Evaluation is the cornerstone of effective public-sector communications, essential for improving policy outcomes, adapting innovative strategies, and building public trust.

The Evaluation Cycle represents an agile evolution of our previous model, the Evaluation Framework 2.0, updated to:

Emphasise the importance of continuous learning, innovation and improvement
Integrate close links to other frameworks such as OASIS
Outline metrics that can be applied across all types of communication activity

Operating as an integrated process loop rather than a rigid post-campaign task, the framework is structured around six interconnected stages – Inputs, Outputs, Outtakes, Outcomes, Impact, and Learning and Innovation – to drive ongoing campaign optimisation. This data-driven foundation provides best-practice guidance to effectively and efficiently measure communication activities across government, whether managing major paid-for campaigns or running low- and no-cost communication activities.

To support these efforts, government communicators are also encouraged to use this cycle alongside the Magenta Book, HM Treasury guidance on evaluation methodologies, data collection, and qualitative or quantitative analysis.

This Evaluation Cycle helps you to:

Embed evaluation upfront as a core element in every communication plan
Identify the consistent metrics used to measure different types of campaigns and low- and no-cost communication activities
Understand the different aspects to consider when planning and delivering your evaluation.
Identify and consider audience characteristics and difficult-to-reach audiences to ensure audience inclusivity.
Build clear connections between evaluation and other relevant Government Communications frameworks, including OASIS, Theory of Change and COM-B model for behaviour change.
Measure campaign success, both in real-time and throughout the campaign or communication activity.
Identify how evaluation results can contribute to the continuous improvement of corporate objectives and reputation management.
Explore new ways of creating innovative solutions to address challenges more efficiently and effectively.

The Evaluation Cycle is not a one-size-fits-all model. Instead, it is an agile tool designed to spark a continuous loop of learning and creative innovation, ensuring our work resonates with target audiences in a meaningful, cost-effective way.

About the Evaluation Cycle

Six stages of evaluation

The Evaluation Cycle is an agile framework consisting of six stages designed for continuous assessment, rather than a linear evaluation. The agile nature of the Evaluation Cycle means you can utilise the aspects most relevant to your type of communication activity.

The six stages of the Evaluation Cycle are:

Inputs – what you put in: Evidence-based planning and content creation.
Outputs – audience experience: The experience you delivered and created for your audience through the reach of your communications.
Outtakes – audience perceptions: What your audience think, feel or intend to do because of your communication activities.
Outcomes – audience behaviour: The response from and behaviour of your audience.
Impact – what you achieve: In terms of organisation and, or policy objectives and KPIs.
Learning and innovation – what you play forward: The lessons you take forward to improve current or future communications.

It is important that a consistent set of metrics are used to measure the effectiveness of your communications. This will help with choosing appropriate objectives and setting SMART targets (Specific, Measurable, Achievable, Relevant and Time-bound) to measure success, such as key performance indicators (KPIs), for tracking progress.

In the Evaluation Cycle, your metrics should be divided into inputs, outputs, outtakes and outcomes, in line with definitions widely used by the International Association for Measurement and Evaluation of Communication (AMEC) and other professional bodies. Impact then summarises how the outcomes contribute to organisation and policy objectives. Finally, to intrinsically link the evaluation process to continuous learning and improvement, learning and innovation is embedded at the final stage of the cycle to capture strategic learnings that drive new ideas, learnings and improvements that can be taken forward. Learning and innovation should be linked back to the start of your communication planning, the Inputs phase.

Continuous learning and improvements: Core of the Evaluation Cycle

At the core of the Evaluation Cycle sits continuous learning and improvement. This marks the most crucial part of the model, as it defines the purpose of evaluating our communications – to learn and to improve. Learnings should be informed by things that went well and things that did not go so well to feed into current and future planning and implementation.

You should consider how measuring real-time digital metrics throughout your communication activity can generate new insights and potential improvements. If possible, you should implement these improvements to your communications when it is still ongoing.

Digestible lessons should be packaged up for use in the future. The learnings generated should feed back to Inputs (stage one), so that any new communication activity can take on these lessons learnt.

Learnings that can be applied or scaled more generally should be shared across government, especially with teams that work on similar objectives or target similar audiences.

A diagram showing the foundational structure of the Evaluation Cycle, featuring six interconnected stages arranged in a clockwise loop around a central core. Central Core: A dark grey circle titled "Continuous learning and improvements by responding to real-time metrics", split into "Digital (hourly, daily, weekly)" and "Campaign tracking (monthly, yearly)". Clockwise Stages: Inputs: Evidence-based planning Outputs: Audience experience Outtakes: Audience beliefs and feelings Outcomes: Audience behaviour Impact: Linking Inputs, Outputs, Outcomes and ROI Learning and innovation: Strategic insights — Diagram illustrating the continuous loop of the six-stage Evaluation Cycle centred on real-time data tracking and ongoing campaign improvement.

Evaluating different types of communications

Different communication activities engage audiences in different ways, to meet varying strategic objectives. These communication goals can be defined by three distinct categories of engagement.

Awareness and attitudes

In the minority of cases, communications might primarily seek to raise awareness of an issue or to change people’s attitudes, with no behaviour changes anticipated, meaning measurement of Outcomes may not always be possible. However, even where the communication activity does not have a primary behavioural element, considerations should be made regarding how the communication objectives of increasing awareness or changing attitudes feed into overarching behavioural and policy objectives (read more on this in the ‘Inputs’ section).

Behaviour changes

Most government communications seek to change behaviours to implement government policy or improve society. This means that, in addition to tracking awareness metrics, evaluations should capture whether your target audience adopts the desired behaviour change, that is to start, stop or maintain a behaviour, as outlined in our principles of behaviour change. This is so that we can learn which methods, messages and channels are effective for encouraging successful behaviour change.

Recruitment

Recruitment is a specific form of behaviour change where people are encouraged to start an activity. This category encompasses major employment campaigns into public sector jobs rather than recruiting people to “register” or “take part”. Successful recruitment into public sector jobs is vital to maintaining public services and protecting the country. Recruitment campaigns for jobs such as teachers, nurses, and the armed forces share many characteristics, unique demands and broader societal impact, and thus benefit from a dedicated category.

Scaling evaluation by budget

The Evaluation Cycle is designed to scale across all communication types, regardless of budget:

For low-cost or no-cost campaigns: You should measure metrics most suitable to your communication activities at each stage.
For paid-for campaigns: We recommend that approximately 5 to 10% of total campaign resources are allocated to evaluation. Communicators should dedicate this to research and optimisation which can include both in-house and outside agency research. Paid-for campaigns are advised to measure as many of the suggested metrics as possible.

We encourage all departments and organisations to spend up to 10% of their existing campaign budget on innovative techniques which can be tested. This ensures that government communications continue to use public funds responsibly and judiciously whilst seizing new opportunities.

A diagram of the Evaluation Cycle expanding on the overview model by adding detailed bulleted guidance inside each stage box. Central Core: A dark grey circle titled "Continuous learning and improvements by responding to real-time metrics", split into "Digital (hourly, daily, weekly)" and "Campaign tracking (monthly, yearly)". Stage Details: Inputs (Evidence-based planning): What you have to do before the activity (e.g. consider policy and organisation objectives and ensure inclusivity, develop communications objectives, learn from best practice, apply Theory of Change, partner activities, embed innovation, pre-test). Outputs (Audience experience): What is delivered and audience reach (e.g. distribution, exposure, stakeholder engagement). Outtakes (Audience beliefs and feelings): What the target audience thinks, feels, or intends as a result of the intervention (e.g. awareness, understanding, attitudes, emotions, intentions). Outcomes (Audience behaviour): The response of your target audience in terms of behaviour (e.g. maintain, stop and start, decrease barriers, contribution to objectives). Impact (Linking inputs, outputs, outcomes and ROI): Long term quantifiable impact on communication, policy and organisational objectives (e.g. revenue, cost reduction, compliance, retention, recruitment, health, KPI measurements). Learning and innovation (Strategic insights): What can you learn from now and in the future? Could this be scaled up or applied more generally? (e.g. process, Theory of Change, value for money, case studies, benchmarks). — Detailed diagram of the Evaluation Cycle outlining key activities, metrics, and planning considerations across all six stages.

1. Inputs

What are inputs?

Inputs are what you put in at the start – the planning and research that informs your communications or campaign. This includes everything that must be done to prepare for the communication activity, which may include conducting research, reviewing previous learnings, planning evaluation design, determining budget and costs.

Wherever possible, planning should be based on research evidence, insights and learnings to maximise your chance of delivering a successful campaign or communication activity. Successful communication activities and campaigns rely upon sufficient preparation.

What are they used for?

Detail and reflect on what has been done to enable the communication activities.
Demonstrate that communications are based on evidence-based planning, research and previous strategic learnings.
Provide context for the outputs, outtakes and outcomes the campaigns generate so they can be evaluated in line with the budget and resources put into the communications activity.
Ensure clear links to policy and communication objectives and development of appropriate KPIs.
Make predictions and assumptions of how your communications Outputs would lead to Outtakes, Outcomes and ultimately Impact, one step at a time.
Consider innovative solutions and determine in what ways and how frequently these are evaluated during implementation.

Inputs – Setting objectives

Your communications objectives should be set out according to the SMART principles:

Specific: Clearly define what you want to achieve.
Measureable: Determine a baseline and identify ways to track your progress.
Achievable: Set targets that are challenging but also realistic and manageable.
Relevant: Ensure your objectives align with your policy and organisation objectives.
Time-bound: Make sure you have a clear target date or timeline for achieving your objectives.

To enable effective evaluation, your SMART objectives must:

Establish a current baseline measure if no communication activity were to take place.
Set challenging targets that forecast the change your communication activity will make over a defined period of time.
Provide an explanation for the level of change being targeted.

To ensure your communication objectives are relevant they must be coherent to the aims and vision set out by your policy and organisational objectives.

You should consider:

How do your communication objectives feed into the policy and organisation objectives?
How do your communication objectives feed into the specific behavioural changes required for the policy to succeed?
Does your communication activity reflect your organisation’s culture, values, and corporate objectives?

For paid-for campaigns, you must set KPIs, which provide a quantifiable measure to help you monitor progress towards your objectives throughout the campaign.

Inputs – Communication planning

You should make sure your communication plan covers the following considerations so you can be prepared for the forthcoming stages in the Evaluation Cycle. This is not an exhaustive list, and you should expand on areas that matter most to your communication activity.

Embed in your OASIS plan: The OASIS framework provides government communicators with a framework for preparing and executing effective communication activities. The objectives, and scoring and evaluation stages are especially important when using the Evaluation Cycle.
Ensure inclusivity of audience groups: Are you using behavioural insights to understand the target audience of your communication activity? Have you considered how activities will serve those with protected characteristics? Have you considered audience segmentation to help target your communications?
Learn from best practice: What has been learned from previous communications activities? How can you build upon previous performance? Who has done similar types of work before? What are the keys to success? What things have others tried that did not work?
Consider behavioural science where applicable: Using the Theory of Change and COM-B model, consider whether your target audience has the capability, opportunity and motivation to change their behaviour to identify the barriers these present and how your communications might need to overcome them. What beliefs and feelings would you like your audience to have? What audience behaviour would you anticipate? How will your Outputs lead to your Outtakes, Outcomes, and ultimately Impact?
Consider conducting activities in partnership: What are the strengths of other organisations that you can capitalise on? What reputations would you favour in your partnership organisations? Can we work with other brands or organisations that have existing audience loyalties?
Embed outcome-focused innovation: While existing methods might work adequately, are there new promising ways to deliver the communication objectives more productively, more effectively, or at a lower cost? How can we best take advantage of emerging technologies? How can we best adapt to evolving audience trends?
Test and pilot: Can your communication activity be piloted with a small audience sample? What metrics and opinions are you tracking and seeking in order to identify unintended consequences and serendipities?

Inputs – Example evaluation metrics

Metric	Medium	Definition	Measurement method
Total spend to date	Online and, or offline	Aggregate total spend so far (online and offline). How much money has been spent on digital media? Sum of one-off set up costs. Refer to the Advertising, marketing and communications spending control guidance (AMC) and periodic offline media spend updates.	£
Theory of change (including evidence base)	Not applicable	Implementation of behavioural science in planning effective communication	Is it in place? Is it evidence-based?
Content creation	Online and, or offline	Infographics, videos	Volume by type
Volume of press releases	Offline	Number of press releases sent out	Number of press releases sent out
Volume of social media releases	Offline	Number of press releases to owned social media channels	Number of releases to owned social media channels
Insights and formal learnings	Offline	Implementation of insights from research and, or learnings from previous	Is it in place? What did it inform?

2. Outputs

What are Outputs?

Outputs are objective measurements of what is delivered and how your audience encounters and interacts with your communications through reach, distribution and exposure. This captures how successfully your communication activity has reached your target audience, which may include press coverage, public relations and impressions, as well as low- and no-cost activities such as stakeholder engagement.

If the campaign has a behavioural change element, links should be made to the COM-B model. This allows you to consider the reach, distribution and, or exposure of your communications in relation to audience experience and how this influences capability, opportunity and motivation among your target audience. For example, how an Output might lead to a change in opportunity.

What are they used for?

By tracking Output metrics around target audience reach, you can determine how effective your communications or campaign approach was in reaching your target audience. Tracking assets and collateral allows you to evaluate messaging, asset type and test implementation of your communications plan, i.e. which channels were most successful at reaching your target audience or different audience segments.

Outputs – Example evaluation metrics

Metric	Medium	Definition	Measurement method
Estimated total reach	Online and, or offline	Aggregate audience reach	Absolute number and percentage of target audience
Estimated offline reach	Offline	Reported audience reach for offline media	Absolute number and percentage of target audience
Reported online reach	Online	Estimated reach as reported by digital platforms	Absolute number and percentage of target audience
Direct contacts	Online and, or offline	Number of direct online or offline contacts, such as electronic messages	Absolute number and percentage of target audience
Events organised	Offline	Volume of events	Number of events, attendees, feedback from attendees
Volume of coverage	Online and, or offline	Number of exposures	Number of press cuts, broadcasts (local and national)
Partnerships secured	Offline	Number of partnerships providing amplifying support	Formal sign-up to either actively promote and amplify campaign material or make any form of value in-kind partnership or financial contribution

3. Outtakes

What is an Outtake?

An outtake is the audience perception – what they think, feel or intend to do as a result of your communication activities. Outtakes capture the reception, perception, intentions and reaction of your target audience to your communication activity. Outtakes are distinct from outcomes: while outtakes focus on audience beliefs, attitudes and feelings, outcomes focus on actual changes in behaviours.

What are Outtakes used for?

Outtake metrics measure how your communication activity impacts your target audience’s awareness, understanding, attitudes, emotions, or intentions. Comparing Outtakes against your target objectives enables you to understand which messaging successfully engaged your audience or different audience segments.

Outtakes – Example evaluation metrics

Metric	Medium	Definition	Measurement method
Sentiment, attitudes and emotions	Online and, or offline	Qualitative and quantitative degree to which a message has been positively or negatively received	Net sentiment scores (% positive minus % negative) through social listening. Online reaction to posts. Qualitative review of audience feedback to press. Assessment of press coverage.
Passive engagements	Online	Percentage of impressions that resulted in a single-click interaction (shares likes and reposts)	% (Total passive interactions divided by total impressions and multiplied by 100)
Active engagements and interactions	Online	Percentage of impressions that resulted in a deliberate interaction (comments, responses and quotes)	% (Total active interactions divided by total impressions and multiplied by 100)
Click-through rate (CTR)	Online	Percentage of impressions that resulted in an active click	% (Clicks divided by impressions)
View-through rate (VTR)	Online	Percentage of people who view through a set minimum duration of a video	% (Completed views divided by impressions)
Dwell time	Online	Average length of time spent on a campaign site	Minutes and seconds
Bounce rate	Online	Percentage of page sessions that are unengaged (where I user left in less 10 seconds, viewed only one page, and triggered no conversion event, such as filling out a form, subscribing to content, or downloading a file)	% (Unengaged sessions divided by total sessions and multiplied by 100)
Prompted campaign recognisers	Offline	Percentage of target audience that recalls seeing the campaign when prompted	Absolute number and percentage of target audience
Unprompted awareness of campaign or issue	Offline	Absolute number and percentage of target audience that has unprompted campaign issue awareness	Absolute number and percentage of target audience
Experience of different messages that relate to aspects of the Theory of Change	Online and, or offline	The extent to which different groups either agree or disagree with messages related to the Theory of Change	Five-point scale (agreement or disagreement with aspects of message)
Understanding of the campaign content	Offline	Assessment of the audience’s understanding of the campaign content	Qualitative or quantitative assessment to represent overall level of understanding in the target audience. Highlight specific content with low understanding.
Sentiment towards profession	Offline	How the profession is regarded by the target audience or general public	Five-point scale recommended (from strongly positive to strongly negative; don’t know)
Agreement	Offline	Absolute number and percentage of target audience that agree with the campaign message	Five-point scale recommended (from strongly positive to strongly negative; don’t know)
Stated or intended behaviour change	Offline	Percentage of target audience that claim they will act in accordance with campaign aim	Absolute number and percentage of target audience
Expressions of Interest (EOI)	Online and, or offline	Number of people actively expressing an interest in applying	Absolute number and percentage of target audience
Reputation perception	Offline	Baseline levels of trust, credibility, and operational satisfaction associated with your organisation or department among the target audience	Quantitative: Trust and favourability indexes via public pulse surveys Qualitative: Message-testing focus groups and stakeholder tracking interviews
Response rate	Online and, or offline	Percentage of of contacts who respond	% of contacts who respond
Intra-profession advocacy	Offline	Degree to which current professionals would recommend the job to friends and family	Five-point scale recommended (from strongly positive to strongly negative; don’t know)
Influencer advocacy	Offline	Degree to which important influencers (such as parents) would support entry to profession	Five-point scale recommended (from strongly positive to strongly negative; don’t know)
Attitudinal change	Online and, or offline	Degree to which people’s attitudes has changed in favour of the campaign	Five-point scale recommended (from strongly positive to strongly negative; don’t know)

4. Outcomes

What is an Outcome?

Outcomes are the response from your target audience in terms of changes in behaviour or active engagement – for example, registrations to a website or service, adoption of positive habits, cessation of unwanted practices to comply with new laws and regulations. It captures whether your audience’s feelings and motivations (Outtakes) really translate to actual behavioural change (Outcomes).

What are Outcomes used for?

Outcomes determine how your communication activity influenced behaviour change and contributed to the policy objectives – did your campaign encourage your target audience to start doing something, stop doing something or maintain a behaviour.

Outcomes enable links to be made between any Outtakes (changes in beliefs and attitudes) and the resulting desired change in behaviour. Remember, even if your communication is primarily aimed at changing beliefs and feelings (Outtakes), rather than behaviours (Outcomes), you should still consider whether (and if so, how) audience behaviour might change in response to your communications.

Outcomes – Example evaluation metrics

Metric	Medium	Definition	Measurement method
Behaviour change – maintain, stop or start	Offline	Number and percentage of target audience that has changed behaviour	Absolute number and percentage of target audience
Applications and sign-ups	Online and, or offline	Number of completed applications or registrations	Absolute number and percentage of target audience
Completion and registration rate	Online and, or offline	Percentage of contacts,impressions, or landing page visitors that go on to complete a sign-up, registration or application	% (Total completed registrations divided by total initial visitors or contacts and multiplied by 100)
Decreased barriers to change behaviour as targeted by your campaign	Online and, or offline	Extent to which barriers to desired behaviour were removed or decreased (increased confidence or opportunity to do behaviour)	Absolute number and percentage of target audience
Decreased barriers to behaviour change that were not targeted by your campaign or cannot be changed through communications	Online and, or offline	Extent to which barriers to desired behaviour not targeted by the campaign were removed or decreased, to measure what else is influencing your metrics (such as costs or policies you are unable to change through communication)	Absolute number and percentage of target audience
Expression of Interest (EOI) to applicant conversion rate	Online and, or offline	Percentage of individuals who submitted an EOI who then go on to formally apply for a role	% (Total formal applications divided by total EOIs and multiplied by 100)
Applicant to hire conversion rate	Online and, or offline	Percentage of applicants that successfully completed the application process and are officially hired	% (Total candidates hired divided by total formal applications and multiplied by 100)

5. Impact

What is Impact?

Impact draws links between inputs, outputs, outtakes and outcomes to determine how your communication activity has contributed to or impacted policy and organisational objectives. Impact is the comparison of actual outcome data with KPIs and objectives set to measure whether these were met.

Organisational objectives are distinct from the policy objectives and include longer-term or wider considerations such as Return on Investment (ROI), revenue, cost reduction, compliance, retention, recruitment, and positive contributions to physical and mental health, and environmental impact.

What is impact used for?

Impact allows you to demonstrate whether your objectives were delivered. For paid-for campaigns you should demonstrate whether your KPIs were met and if they were sufficient to deliver the targets set in your objectives. You should also consider how your communications activity contributed to broader organisational impacts. Read more on measuring ROI and organisational reputation.

Impact – Example evaluation metrics

Metric	Medium	Definition	Measurement method
Comparison with KPIs and objectives	Not applicable	Contribution to policy and, or organisational objectives	Absolute number and percentage of targets met
Long-term compliance or retention	Not applicable	Contribution to long-term or ongoing policy and, or organisational objectives	Absolute number and percentage of targets met over time
Recruits	Offline	Number of people successfully recruited	Absolute number and percentage of target audience
Cost per person with awareness raised	Not applicable	Average cost to successfully educate or shift the attitude of a single person in your target audience	Unit cost (total campaign spend divided by total number of people with documented awareness)
Cost per person with behaviour change	Not applicable	Average cost to get a single person to actually change their behaviour (for example stop smoking, install a smart meter, or use water safely)	Unit cost (total campaign spend divided by total number of people who changed their behaviour)
Cost per Expression of Interest (EOI)	Online and, or offline	Average cost to get someone to express initial interest in a program or campaign (like filling out an inquiry form)	Unit cost (total campaign spend divided by total number of EOIs submitted)
Cost per completion or registration	Online and, or offline	Average cost to get a user to fully complete a sign-up, registration, or application form	Unit cost (total campaign spend divided by total completed registrations)
Cost per applicant	Online and, or offline	Average cost to receive one formal job application	Unit cost (total campaign spend divided by total applicants)
Cost per hire	Not applicable	Total cost to successfully fill one vacancy	Unit cost (total campaign spend divided by total number of applicants hired)
Current ROI	Not applicable	Financial return or taxpayer savings generated for every pound spent on the campaign.	Step one: Calculate total financial benefit (unit benefit multiplied by number of behaviour changes). Step two: Calculate net ROI % (total financial benefit saved minus total campaign spend, divided by total campaign spend and multiplied by 100). This can also be expressed as a ratio, for example 11:1).

To learn exactly how to isolate your campaign’s impact, apply baseline trends, and calculate your final ratio, read our step-by-step guide to calculating ROI and worked example, further on in this guidance.

6. Learning and innovation

What is learning and innovation?

Formal learning forms the final stage of the Evaluation Cycle. At this stage, you would evaluate the effectiveness of inputs at each stage of the evaluation cycle to understand the impact on policy and organisation objectives, and assess what did or did not work. Formal learning can be used to drive innovation in the future. It is useful to consider what approaches could be employed in future activities to overcome difficulties and leverage strengths in your approach. Where applicable, it is also useful to capture what innovations were applied in your communication activities and why they did or did not work.

For paid-for campaigns, this stage is where learnings from your 10% innovation investment should be reported in your evaluation. It is an opportunity to showcase innovations that could be “scaled up” across other communication activities but also to report other learnings (such as what didn’t work). The aim of this stage is not to justify new approaches but rather to inform future considerations.

If objectives or KPIs were not met, what reasons can be identified to explain the variation? If the objectives were surpassed, what has driven that? This identifies strategic insights and learnings that can be taken forward and shared.

What is learning and innovation used for?

This is so that, now or in the future, communications activities can capitalise on successful communication techniques and avoid embedding unsuccessful methods. Learnings that can be applied or scaled more generally should also be shared across the profession teams, and departments. You should opt for methods most suited to the needs and culture of your organisation, which may be:

Linking up your corresponding policy teams or other teams with a similar remit to initiate discussions
Creating end-of-campaign reports to share on evidence repositories
Sharing bite-size insights via show-and-tell sessions

Evaluation example from stages one to six

The following example is fictitious and have only been created to demonstrate how to apply the Evaluation Cycle.

Stage	Campaign to encourage take-up of home insulation subsidy
1. Inputs	Policy objective: Improve wellbeing and reduce the cost of heating during winter by ensuring people’s homes are properly insulated. Behaviour: For eligible people to have their homes insulated. Communications objectives: To raise awareness of the grants available among eligible people by at least 25% in 6 months. To increase grant applications through GOV.UK from 40,000 by 50% in 6 months. Previous learning: Infographics are an effective way to present complex information. Audience insight: Young families and people aged over 65+ in older less efficient homes are most in need of the subsidy. Planning: Online campaign with messaging targeted for different key audiences.
2. Outputs	85% of the estimated 2,000,000 people eligible for the home insulation subsidy have been reached with social media impressions. Facebook was most effective at reaching the target audience of young families.
3. Outtakes	Awareness of the home insulation subsidy increased by 27% from 38% to 65%. Young families (68%) showed higher overall campaign awareness than those aged over 65 (59%). However, with the innovation of adding a leaflet to the campaign, older audiences were targeted more effectively; overall leaflet recognition was 18% but this peaked at 26% for those aged over 65. Overall, 79% of those who reconsidered the campaign were likely to take action in the future.
4. Outcomes	Applications for the subsidy increased by 50% reaching a total of 60,100, with 11% coming directly from Facebook advertising.
5. Impact	Objectives were met and targets should be increased if the campaign continues. Increased application of the home insulation subsidy means the homes of more people are likely to undertake the behaviour of insulating their homes resulting in the improvement of people’s physical health and quality of life.
6. Continuous learning	Awareness of the campaign was significantly lower amongst adults aged 65+ meaning they are less likely to apply for the subsidy. When looking at this age group in more detail it was found that the campaign generated five times less reach amongst this age group on paid social media. Change based on learning – it was decided to adapt the inputs and engage with Age UK as a stakeholder to produce a leaflet and promote through their channels.
Learning and innovation	Engaging with key stakeholders, and a targeting leaflet campaign, was more effective at reaching the audience members aged over 65, raising awareness to 63%. This should be considered in future communications.

Evaluation and the OASIS framework

OASIS is a five-step planning framework, designed to help communicators develop and implement effective communications and campaigns, regardless of size or budget.

The five steps are:

Objectives: What do you want the campaign to achieve?
Audience and insights: Who are you trying to reach? What are their needs, wants and concerns? What barriers need to be overcome?
Strategy and ideas: What are the best ways to reach your audience and achieve your objectives? Consider using the COM-B model and Theory of Change in your planning.
Implementation: How will you put your strategy into action?
Scoring and evaluation: How will you measure the success of your campaign?

While evaluation seems to only form the final part of the planning process, it is recommended that you plan how to evaluate your communications at the start. Thinking about evaluation at the beginning of your planning process has key benefits:

You can be clear about the objectives you want to achieve with your communication activity from the start, helping you identify and collect the right data throughout. This may involve tracking website traffic, social media engagement and survey responses, as outlined in the previous section.
Your evaluation can run concurrently with the communication activity, meaning learnings from your work can immediately create positive changes in your current campaign rather than only informing future communications.

The table and diagram below show how the OASIS framework is directly linked to the stages in the Evaluation Cycle. Integrating evaluation into your OASIS planning draws clear links with policy objectives and KPIs. It also ensures learnings are built into the planning stages.

Evaluation stage	Evaluation considerations
Objectives (Inputs)	Are your communication objectives SMART? Are your communication objectives linked to your policy and organisation objectives?
Audience and insights (Inputs)	What are the best channels for reaching your audience? Does the narrative resonate with your audience?
Strategy and ideas (Inputs)	Does your Theory of Change correctly predict the cascade of events from the Inputs to the Impact stage? Do the assumptions within your Theory of Change reflect your audience’s actual behaviours? Are there any unintended consequences or serendipities? The IN-CASE framework can help to identify unintended consequences.
Implementation (Outputs)	Did all the components and stages of your communication activity go as planned? Are there any unanticipated challenges that may derail your communication activity? Do you need extra safeguards against any barriers?
Scoring and evaluation (Impact)	Did reaching your communication objectives contribute positively to your policy and organisation objectives? If a communication activity is not helping with a policy’s cause, is there more you can do? Or, are there inherent limitations to what communications can achieve towards the policy?

By understanding how these concepts are linked across the two frameworks, colleagues in evaluation and communication delivery should aim to collaborate more effectively and efficiently by devising campaign plans and evaluation plans concurrently.

A diagram of the Evaluation Cycle building on Image 2 by overlaying linkages to the OASIS framework using red badges. Top-Left Key: Titled "Link to OASIS Framework" displaying five red pill badges: Objectives, Audience and insights, Strategy and ideas, Implementation, and Scoring and evaluation. Framework Mappings: 1. Inputs: Contains red badges for Objectives (consider policy/organisation objectives), Audience and insights (learn from best practice), and Strategy and ideas (apply Theory of Change linking Outputs, Outtakes and Outcomes). 2. Outputs: Contains the red badge Implementation (what is delivered and audience reach). 5. Impact: Contains the red badge Scoring and evaluation (long term quantifiable impact on objectives). Other Components: Retains all stage bullet points and central core from Image 2. — Diagram demonstrating how the stages of the Evaluation Cycle align directly with the five steps of the OASIS planning framework.

Evaluation, inclusivity and audience segmentation

Audience segmentation is crucial to understanding your target audience’s characteristics, needs and interests. Dividing your target audience into smaller segments based on their commonalities and differences helps you include and target hard-to-reach audiences. For example, by considering the appropriate language, tone, communicative channels, and timings to cater to each audience segment.

Audience segmentation can help to:

Target your audience segments more efficiently by focusing your resources on the audience most likely to respond to your messaging.
Make your campaigns inclusive by designing tailored messaging to reach hard-to-reach audience segments – that is those who do not engage with mainstream media – by accounting for unique cultural values, communication habits, and trusted networks.
Tailor messaging to audience interests to drive engagement, build trust, and foster your organisation’s reputation.
Understand and address specific audience barriers to behaviour change (read more about the COM-B model).
Ultimately, drive better digital engagement (Outtakes), real-world behaviour change (Outcomes), and long-term society benefits (Impact).

Segmenting focus	By demographics	By golden questions	By behavioural barrier
Description	Based on factors such as age, gender, location, income, education level, and occupation	Using a set of questions set in a particular order to gauge your audience’s needs, interests and behavioural norms	Based on factors that may span demographics, such as, motivation, perceptions and capability
Pros	Data is relatively easy to collect or readily available from databases	Provides deep insights into the audience’s needs and interests – messaging is not bound by demographics and can cater to subcultural norms	Suitable for behaviours that are not significantly driven by demographic factors (for example, where members of the same demographic group do not share interests or needs). If you have already done this analysis as part of your planning, then it’s readily available to help with segmentation.
Cons	Prone to over-generalisation, where members in the same demographic group may not have the same interests or needs	Golden questions are tailored to each campaign, which is generally more time-consuming and expensive to implement. If they are not done carefully, it can lead to smaller, less robust segments.	You will usually have to create bespoke surveys to measure behavioural barriers as they aren’t routinely measured in the same way as demographics like age and income.

Explore our Government Audience Segmentation guidance to learn how to identify, understand and target your key audiences effectively.

Monitoring your audience at every stage of the cycle

Audience segmentation should be considered at the beginning of the Evaluation Cycle and during your communication planning (Inputs). This will allow you to report metrics for each of your audience segments throughout your evaluation. Setting out segments prior to campaign activity can allow for achievable targets for each group, meaning more detailed evaluation especially where campaigns are targeted.

Use insights from, for example, audience segmentation, to optimise your campaign in real time. By adjusting your immediate activities (Outputs), you can actively improve audience engagement (Outtakes) and long-term campaign success (Outcomes).

In the next section, we explain how you can use a Theory of Change to set out your assumptions and predict your audience’s journey. This means mapping out how your activities (Outputs) will shift audience beliefs (Outtakes) and ultimately lead to the desired behaviour change (Outcomes).

A diagram of the Evaluation Cycle building on Image 3 by introducing a right-hand section focused on audience inclusivity and segmentation. Right-Hand Section: Titled "Ensuring inclusivity and engaging audiences:" with four text annotations: Perceived characteristics of target audience(s) Audience segmentation Perceived characteristics of the messenger Sub-cultural norms Other Components: Retains the six detailed stage boxes, central core, and red OASIS framework badges from Image 3. — Diagram showing how audience segmentation, demographic characteristics, sub-cultural norms, and messenger trust integrate into the Evaluation Cycle.

Evaluation and Theory of Change

A full explanation of the Theory of Change can be found in the Magenta Book on GOV.UK, and is covered in our guidance on the Principles of behaviour change.

The Magenta Book is HM Government’s guidance document that details how to scope, design, conduct, use and disseminate evaluations. It explains how to incorporate evaluation through the design, implementation, and review stages of policymaking. Building on cross-government expertise on policy evaluations, the Evaluation Cycle optimises the use of these evaluation principles for communication evaluations, making sure that target audience groups are reached and that behavioural science principles are considered.

The Magenta Book includes transferable concepts that apply to both policy and communication evaluations. Government communicators are highly encouraged to use the Magenta Book alongside the Evaluation Cycle to deepen their understanding of:

Diverse methodologies: Navigating an array of evaluation methods tailored to different scenarios.
Data integrity: Identifying key considerations when conducting quantitative and qualitative data collection and analysis.
Knowledge sharing: Effectively sharing results and applying insights to design more impactful future communications.

The Theory of Change for communication activity helps you capture considerations, assumptions and predictions about how your communication activity is expected to deliver behaviour change. Some key considerations during the stages of evaluation are:

Outputs – The experience and reach of your communications among different audience groups and how this might affect them in terms of capability, opportunity and motivation to engage in the behaviour change as you predict.
Outtakes – How your communications with the target audience leads to changes in beliefs and, or feelings. For example, the extent to which your audience agrees with your view before and after exposure to the communication activity.
Outcomes – If your assumptions are supported by the evidence collected and behaviour change is observed as predicted.

A diagram of the Evaluation Cycle building on Image 4 by visually highlighting stages associated with testing and developing a Theory of Change in green. Visual Highlights: Stage boxes 2. Outputs, 3. Outtakes, and 4. Outcomes feature green headers and green box outlines. Bottom-Left Key: A green square labelled "Link Outputs, Outtakes and Outcomes to develop and test Theory of Change". Other Components: Retains the central core, detailed stage bullet points, red OASIS badges, and right-hand audience inclusivity text from Image 4. — Diagram highlighting how the Outputs, Outtakes, and Outcomes stages are used to test and refine a campaign’s Theory of Change.

Evaluation and the COM-B model for behaviour change

In communication activities that aim to drive behaviour change, one of the more challenging parts of the Evaluation Cycle is establishing how progress is made from Outputs to Outcomes. During the Inputs stage of the Evaluation Cycle, you need to plan ahead and envisage how audience experience (Outputs) will lead to audience beliefs and feelings (Outtakes) that align with the message of your communication activity, which ultimately lead to the desired audience behaviour (Outcomes).

A behaviour change campaign will need to address barriers that might stop your target audience from engaging in the desired audience behaviours. The COM-B model helps identify three barriers:

Capability – Does your target audience have the right knowledge, skills, physical and mental ability to carry out the behaviour?
Opportunity – Does your target audience have the right resources, systems, processes and environment around them to empower them to undertake the behaviour?
Motivation – Does your target audience want to or believe that they should carry out the behaviour and establish habits based on it?

By setting out a strong basis for your audience experience (Outputs) that adequately addresses these three barriers, you will maximise the chances of them engaging in the desirable behaviour. You should consider all other examples outlined in the COM-B model.

A diagram of the complete Evaluation Cycle building on Image 5 by mapping the COM-B model for behaviour change using blue badges. Top-Left Key: Adds a second header, "Link to COM-B Model", displaying four blue pill badges: Capability, Opportunity, Motivation, and Behaviour. COM-B Mappings: 2. Outputs: Adds the text prompt "Consider how these influence Capability, Opportunity and Motivation to change behaviour". 4. Outcomes: Displays the blue badge Behaviour next to "Maintain, stop and start". Other Components: Retains all prior elements including green Theory of Change highlights, red OASIS badges, stage bullet points, and right-hand audience inclusivity sidebar. — Comprehensive diagram showing the Evaluation Cycle fully integrated with OASIS, audience segmentation, Theory of Change, and the COM-B model for behaviour change.

Calculating Return on Investment (ROI)

Earlier in this guidance, we talked about how ROI should be used as a core metric to measure campaign efficiency. However, calculating a true Return on Investment in the public sector requires more than just looking at your final numbers; you have to prove that your communication activity – and not outside factors – actually caused the change.

To help you calculate this accurately, we recommend a structured five-step process to isolate your campaign’s impact:

Objectives: These should be focused on quantifiable behavioural outcomes (such as the number of direct foreign investments generated, or the number of teachers recruited).
Baseline: Establish the status quo or expectation for the metrics in question if we do nothing.
Trend: A forecast of how the baseline will naturally move over the period of measurement. For example, if there has been an 8% reduction in the adult smoking rate over the last 15 years, we expect that the next year will see a 0.5% reduction, all other things being equal.
Isolation: Exclude or disaggregate other factors that will affect the outcome you are measuring to make sure that the change observed has been caused by the campaign. To accurately assess the impact of a recruitment campaign, compare new hires to the existing employment rate. If you launch a campaign at the same time a new law or tax comes in, you need to figure out how much of the change was caused by your communications versus the law itself.
Externalities: Consider how well your campaign achieved your intended outcomes. On top of that, be aware of and track unintended effects – they can be positive serendipities or negative consequences. For example, the launch of a smoking discouragement campaign could aim to promote citizens’ health and alleviate stress in the healthcare system. A serendipity may be improved air quality in city parks, and an unintended consequence may be increased substance abuse in other forms, such as alcohol.

Assumptions

Assumptions are not a separate step; they are a core part of calculating ROI. Assumptions should be reasonable, clearly identified and, if possible, justified. Part of the post-campaign evaluation will involve refining assumptions and considering their validity.

Below, we break down the five steps and share a worked, real-world example of how to calculate your campaign’s financial return.

A worked example of ROI

The following example is fictitious and has only been created to demonstrate how to calculate ROI.

The Department for Health and Social Care (DHSC) is running a campaign costing £3 million to reduce the volume of inappropriate A&E attendances for low-urgency cases. The campaign aims to divert people to a GP surgery where they can be better and more efficiently taken care of.

Objectives: The campaign will reduce the number of inappropriate NHS England A&E attendances by 3% or 85,000 in 2018 to 2019 compared to the 2017 to 2018 baseline of 2,865,377.
Baseline: The appropriate baseline for comparison here is the previous 12 months of operational data or observations. In 2017 to 2018 there were 23,878,145 A&E attendances in England. 12% of these were found to be “inappropriate” (did not require A&E attendance and could have been handled by a GP or pharmacist). The baseline for inappropriate attendances is 2,865,377 (23,878,145 x 0.12).
Trend: Over the past three years, we have seen a steady increase in A&E attendances of 2% year-on-year, driven by population growth and other factors. We can forecast 24,355,708 attendances (23,878,145 x 1.02) in 2018 to 2019. The rate of inappropriate attendance has remained broadly constant at 12%. The trend-adjusted baseline for inappropriate attendances is therefore 2,922,685 (24,355,708 x 0.12).
Isolation: The NHS is also starting to provide and promote out-of-hours GP surgery appointments. The rate of inappropriate A&E attendance is 4.5 percentage points higher than average at times when GP surgeries are not currently open. We assume that the new offer of out-of-hours service by GPs will reduce the total number of inappropriate attendances by 2.25 percentage points (half of the total observed effect because this only affects half the hours in a day). We expect this to independently reduce the number of inappropriate attendances observed by 65,760 (2,922,685 x 0.0225) to 2,856,925.
Externalities: Aside from the direct cost benefit of optimising points of treatment across NHS frontline services (in the conclusion), there are indirect benefits or positive externalities that should be considered in this case. Inappropriate attendances rarely have to be treated, so there will be a negligible cost for this, and operational overheads will remain as a fixed cost. A reduction in inappropriate A&E attendance of 3% is approximately equivalent to a 3% uplift in staffing resources (which can be redeployed to urgent cases). The total annual cost of A&E operation is £2.7 billion and staffing makes up 30% of this, which is equal to £810 million. 3% of £810 million is equal to £24.3 million. The indirect benefit of optimising A&E attendance, or the effective “opportunity cost” of not optimising staff resources, is £24.3 million.

Direct NHS Savings

This table shows how redirecting patients from A&E to a GP surgery creates the baseline saving of £102 per person.

Healthcare point of treatment	Average unit cost	Saving per redirected patient	Total cases redirected	Total direct savings
A&E Attendance	£148	£102 (£148 – £46)	85,000	£102 x 85,000 = £8,670,000
GP Appointment	£46

Total return (ROI)

By combining the direct savings with the positive externalities (the staff resource value), we can calculate the overall value and final ROI ratio.

Benefit component	Financial value	Calculation details
Direct NHS savings	£8.67 million	85,000 patients redirected to GP care
Indirect benefits (Externalities)	£24.30 million	Equivalent to a 3% uplift in redeployed A&E staffing resources
Total campaign benefit	£32.97 million	Direct savings (£8.67m) plus indirect benefits (£24.30m)
Total campaign spend	£3.00 million	Fixed marketing and operational costs
Return on Investment (ROI)	999% ROI (approximately 11:1)	Every £1 spent generated £11 of value for society

Measuring reputation

The reputation of an organisation is now well established as an indicator of organisational success. Positive reputations are associated with supportive behaviours from stakeholders, while negative reputations are associated with less support or even hostile responses from stakeholders. Despite good evidence on this link, there is still much confusion about how to best measure and manage reputation, with many seemingly competing models and approaches.

Assessing organisational reputation serves as an essential tool to gauge performance and inform strategic decisions. Through this section, informed by over 20 years of insights from the John Madejski Centre for Reputation, we aim to simplify the process.

When considering reputation management, remember to consider:

Whose opinion matters most regarding reputation
What specific elements of the organisation’s reputation are under scrutiny
How managing these elements of reputation will help you achieve your policy objectives

Identifying your audience

Organisations have different reputations with different groups and individuals. Identifying which stakeholders are integral to your organisation’s missions is imperative. This selection must be guided by an understanding of your organisation’s objectives and whom it intends to serve. Factors such as the influence and legitimacy of these stakeholders, as well as the urgency of their concerns, should inform your response. In the dynamic landscape of stakeholder engagement, prioritise judiciously rather than reacting to the most vocal demands, which are frequently amplified by social media platforms.

It is therefore important to consider aspects of organisational purpose, stakeholder need, and legitimacy before responding to stakeholders and when managing and measuring reputation. At its best, choosing to measure reputation with a particular group can help to give them a voice, allow your organisation to listen, and guide and justify the actions of your organisation.

Identifying and measuring the drivers of reputation

In practice, reputation is often measured as an aggregate of stakeholders’ trust and respect for an organisation. This is sometimes referred to as “emotional appeal”. This aggregate measure of reputation allows organisations that are different (for example the armed services vs Amazon) to be compared in terms of the extent to which they are trusted.

However, reputation is also shaped by a range of aspects including:

Organisational characteristics (products, leadership and financial performance)
Relationships (customer service, listening and the appropriate use of power)
Third party influence (whether important third parties recommend your organisation)

Understanding and measuring these underlying drivers is vital for creating effective change strategies. They can be defined and measured as the causes of reputation that make stakeholders trust or respect your organisation.

Where data allows, measuring these factors and causally linking them to reputation through multivariate statistical analysis is important in helping to develop a Theory of Change. For example, if good service experience is found to be a key factor driving trust in HM Revenue and Customs (HMRC), it would be reasonable to focus activities on service if it was cost-effective.

Purpose-driven reputation management

Reputation management strategies should be aligned with the behavioural outcomes you aim to achieve among stakeholders. Whether you are encouraging certain actions, discouraging others, or maintaining desirable behaviours, the desired change must be clearly articulated. Rather than adopting a one-size-fits-all approach, tailor your strategies to specific contexts and stakeholder groups, and recognise that maintaining existing positive behaviours is typically more straightforward than instigating new actions or dissuading entrenched habits.

Measuring your progress

It is important to measure not only reputation but also its causes and consequences in a way that allows you to establish robust correlations.

Employing tried-and-tested measurement scales and analytical techniques, such as multivariate regression, will enable you to discern which experiences influence reputation and subsequent behaviour patterns. In this way, you will be able to identify which stakeholder experiences link to reputation and its associated consequences. It is also important to include measures that benchmark your organisation against other organisations, especially in terms of aspects of trust and respect.

Overall, knowing your audience, understanding what shapes your reputation, and being clear on your goals will greatly improve your reputation management.

Acknowledgements

We would like to thank those who helped draft and develop the original Evaluation Framework and all those who have reviewed and offered comments and ideas on the new Evaluation Cycle. Thanks also to members of the Government Communications Strategy and Commitment Board, the Government Communications Heads of Discipline, the Government Communications Data, Insight and Evaluation forum, Government Communications Directors of Communications and the Cabinet Office Evaluation Task Force.

Particular thanks to Professor Kevin Money (and associates), Dr Becky Tatum, Clement Yeung, Dr Amanda Svensson, Dr Moira Nicolson, Robin Attwood-Martin, Neil Wholey, Ethen Carlin and Pamela Bremner. This guide draws heavily from the academic works of Professor Kevin Money et al. of the Henley Business School at the University of Reading along with a number of other key publications.

Reference list

The principles underpinning the Evaluation Cycle are anchored in academic research and extensive expert consultation. The following publications – including foundational studies from the John Madejski Centre for Reputation at Henley Business School – provided the evidence base, theories, and models used to shape the development of this guidance.

Fernandez-Muinos, M., Money, K., Saraeva, A., Garnelo-Gomez, I., and Vazquez-Suarez, L. (2022). “The ladies are not for turning”: Exploring how leader gender and industry sector influence the corporate social responsibility practices of franchise firms. Heliyon, 8.
Garnelo-Gomez, I., Money, K., and Littlewood, D. (2022). From holistically to accidentally sustainable: A study of motivations and identity expression in sustainable living. European Journal of Marketing, 56(12).
Ghobadian, A., Money, K., and Hillenbrand, C. (2015). Corporate responsibility research: Past – present – future. Group and Organization Management, 40(3), 271-294.
Hillenbrand, C., Money, K. G., Brooks, C., and Tovstiga, N. (2019). Corporate tax: What do stakeholders expect? Journal of Business Ethics, 158(2), 403-420.
Hillenbrand, C., Saraeva, A., Money, K., and Brooks, C. (2020). To invest or not to invest? The roles of product information, attitudes towards finance and life variables in retail investor propensity to engage with financial products. British Journal of Management, 31(4), 688-708.
Hillenbrand, C., Saraeva, A., Money, K., and Brooks, C. (2022). Saving for a rainy day… or a trip to the Bahamas? How the framing of investment communication impacts retail investors. British Journal of Management, 33(2), 1087-1109.
Mason, D., Hillenbrand, C., and Money, K. (2014). Are informed citizens more trusting? Transparency of performance data and trust towards a British police force. Journal of Business Ethics, 122(2), 321-341.
Money, K., and Hillenbrand, C. (2006). Using reputation measurement to create value: An analysis and integration of existing measures. Journal of General Management, 32(1).
Money, K., and Hillenbrand, C. (2024). A review of current practice and future directions for evaluation in the Government Communication Service. Henley Business School: The John Madejski Centre for Reputation.
Money, K., and Hillenbrand, C. (forthcoming). Trends and approaches in the evaluation of communication in government and the charity sector: evaluation as a cycle of continuous improvement.
Money, K., Hillenbrand, C., Henseler, J., and da Camara, N. (2013). Exploring unanticipated consequences of strategy amongst stakeholder segments: The case of a European Revenue Service. Long Range Planning, 45(5-6).
Money, K., Hillenbrand, C., Hunter, I., and Money, A. G. (2012). Modelling bi-directional research: A fresh approach to stakeholder theory. Journal of Strategy and Management, 5(1), 5-24.
Money, K., Saraeva, A., Garnelo-Gomez, I., Pain, S., and Hillenbrand, C. (2017). Corporate reputation past and future: A review and integration of existing literature and a framework for future research. Corporate Reputation Review, 20(3-4).
Saraeva, A., Hillenbrand, C. and Money, K. (forthcoming). Reputation as buffer, booster and boomerang. How reputation adds value in different circumstances. European Management Review.
West, B., Hillenbrand, C., Money, K., Ghobadian, A., and Ireland, R. D. (2016). Exploring the impact of social axioms on firm reputation: A stakeholder perspective. British Journal of Management, 27(2).