Case study: how to do an independent evaluation on homelessness on six continents

Street homelessness is a global phenomenon and an extreme manifestation of social injustice with profound negative consequences for individuals and communities. Yet, until recently, international collaboration to address this issue was virtually absent outside Europe. In 2017, the Institute of Global Homelessness launched A Place to Call Home, the first concerted effort to eradicate street homelessness globally across an initial 13 “vanguard cities”, from Montevideo in Uruguay to Bengaluru in India and from Tshwane, South Africa, to Edmonton, Canada.

• Collection:The power of place-based learning
• Driving change: creating student-NGO partnerships for climate solutions
• So you want a novel way to support untapped research talent?

The role of Heriot-Watt University’s Institute for Social Policy, Housing, Equalities Research (I-SPHERE), in partnership with the GISS Institute in Bremen, Germany, and funded by the Oak Foundation, was to provide the independent evaluation of the initiative. This involved monitoring progress towards the numerical goals set by each city but also, crucially, drawing out successful interventions and common barriers across six continents. Organising an evaluation of a project with this scale is a formidable challenge.

Here are the seven lessons we learned.

Work collaboratively with resident researchers

The benefit of working with resident researchers who were fully embedded in each city was immeasurable. The researchers brought on-the-ground community insights that added value throughout the process. Ensuring those working in front-line services were interviewed by resident researchers in their local dialects demonstrated our respect and consideration for the cities we were operating in. It can be damaging for relationships when researchers helicopter in, take information and helicopter out. While it takes considerable time to find, brief and build trust with resident researchers, we found that integrating this into the project timeline was invaluable.

Contract your expectations

A clear set of expectations and deliverables, and a timetable, was agreed with each resident researcher at the project’s outset. This helped ease the challenges of working with partners across six continents with differing levels of experience. By agreeing protocols in contract format, we were protected legally, but this process must be respectfully managed. We took considerable time to get this right, seeking legal advice and support from our institution.

Stay nimble to changing events

We scoped our evaluation before the pandemic, but as lockdown hit, we had to pivot rapidly and approach the research in a radically different way. We changed our approach to accommodate remote interviews and made changes to the timetable to adapt to Covid peaks in different countries. Our stakeholder relationships, including our close collaboration with resident researchers and the trust we built with local homeless strategy leads, meant that we were well placed to be agile and quickly adapt to a changing world without affecting the study’s success.

Co-design your research instruments

To enable rigorous comparative data that drew out common barriers and successful interventions, it was essential that we standardised our research instruments. However, this had to be balanced against highly diverse city and cultural contexts. Our instruments needed to make sense to all individuals taking part. Achieving this balance was, perhaps, our biggest intellectual challenge. In some cities, families with children were sleeping rough, while in others it was limited to single people. We consulted closely with our resident researchers on the local applicability of each tool to deliver the rich set of data we needed.

Forge a two-way relationship

Before asking A Place to Call Home project leads in each city to take part in an evaluation, it is helpful to consider how their participation will benefit them. Initially, we faced wariness from some city stakeholders who were unaware that their work would be evaluated in this way. To win over hearts and minds, we reconsidered how we could give more back to the project leaders. Through discussions, we unearthed an appetite for bespoke feedback seminars, grouping relevant cities together so they could hear and learn from each other. These interim sessions proved essential when finessing our final outcomes and built lasting connections among and within each city. By the end of the study, we’d overcome stakeholder anxiety by making the evaluation of value to each city, focusing on lessons learned rather than judgement.

Take control of the logistics

A global evaluation of this scale requires serious logistical considerations, including coordinating data-sharing agreements, local payments and translation requirements. The time required for logistics far outweighs a standard research project and the research lead can’t do it all. I highly recommend working with a dedicated project manager alongside someone with professional legal expertise. Working in seven languages as well as local dialects in some cities required trusted translation capabilities to interpret local language nuances. This required a bespoke service tailored by city. Perhaps most challenging of all was managing complex payment arrangements for resident researchers across six continents. Working with the CORRA Foundation, a facilitating charity, allowed us to effectively navigate these difficulties.

Create a lasting legacy

Our transformative evaluation findings, published in April 2022 and shortlisted for a THE Award, provided crucial evidence to local and national governments, giving local stakeholders a resource with which to lobby for change. It demonstrated for the first time that comparisons spanning diverse contexts in the Global North and South are both methodologically possible and invaluable when tackling street homelessness. With robust project management, expert legal advice and, perhaps most importantly, investing the time to build robust local relationships, we’ve created a lasting legacy for the next phase of A Place to Call Home and taken one step closer to ending street homelessness across the globe.

Suzanne Fitzpatrick is the director of the Institute for Social Policy, Housing, Equalities Research (I-SPHERE) at Heriot-Watt University.

Heriot-Watt University has been shortlisted for International Collaboration of the Year at the Times Higher Education Awards 2023 #THEAwards. See the full list of shortlisted candidates. Winners will be announced at a ceremony on 7 December in Liverpool.

Academics and university leaders from across the UK and Ireland will come together that week at THE Campus Live UK&IE to talk about institutional strategies, teaching and learning, the student experience and more. Join us for this two-day event in Liverpool.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.