Evaluating transformative arts experience: Europe’s CreaTures Project 

The CreaTures project developed a tool for evaluating creative practices and how they connect to transformative change.

The 9 Dimensions tool can be used by funders, policy makers and researchers, as well as by creative practitioners.

It is intended to support the understanding and use of participative art/design practices to imagine liveable futures and make sustainable change in the face of societal challenges.

The project explored playful encounters and collaborative reflection as approaches that stress positive cultural transformation.

The project was critical of evaluation processes that ignore the affective and imaginative aspects of participatory/experiential art/design and those that use indicators to attempt to produce a uniform narrative about behaviour change in the face of converging crises.

The study is the product of 3 years’ (throughout 2020-2022) transdisciplinary codesign between researchers, practitioners and other participants, involving 10 mostly European countries, 20 productions, 140 studies of transformative art and design practices and thousands of people.

The research was led by a consortium comprising University of Sussex; Aalto University, Finland (coordinator); Utrecht University, the Netherlands; and RMIT, Australia with NGOs Sniffer and OKFI, and several arts partners, including: Furtherfield, Superflux and Zemos98. It was supported by EU Horizon 2020 funding (3,000,000 EUR) and was commended for its collaborative processes as well as its outputs.

What happened

The original plan was to follow the practices of several arts/design organisations and cultivate reflective practices among them, using ethnographic methods of observation, interview and involvement in activities. The point was to understand artist motivations and methods and then work together to identify the most important aspects of these change-making processes for sharing with other artists, funders and researchers.

Artists and designers were chosen to join the research as practitioners working directly with participants to inspire thoughts of other ways of living and different relations. It’s helpful to make it clear there were two levels of participation:

 1) the artists themselves work in a participatory way, so one research question was about the impact of their work on their participants, and

2), the research team worked with artists as partners in a participatory way to co-research the project. Researchers and artists met in online workshops to identify artists’ goals, priorities and values through lengthy interview and discussion, coding the language for processes and themes into clusters, then aggregating concerns and weaving them into the dimensions that make the final framework.

The project team used conferencing software to speak, but also plotted characteristics against leverage points and goals using Miro boards, as they might have used post-it notes and paper if together. Eventually the characteristics were abstracted sufficiently to be applicable across cases. Initially, even at the last stages, attributes were presented in pairs before being pared back further (for instance, ‘shaping and inspiring’ became ‘inspiring’ in the final version for greater simplicity).

Since only a subset of artists worked closely on producing the dimensions, sense-checking was important to make the articulation of practice something for which most participating practitioners felt ownership. The researchers did not attempt to collaborate with all partners at once and some partners were more involved (and allocated more time) than others. The availability of both a focussed group and a wider circle of partners allowed the team to both develop and test the evaluation criteria during the project.

To create a generalisable framework, the project team had to find and refine the common threads that ran through partners’ aspirations and processes. That said, each practice studied was unique: practitioners are driven to distinguish themselves by the nature of the funding scene, but are also led by individual passions and convictions about the kind of change needed in society. This has the benefit of offering a variety of approaches to future-making for all types and moods of people.

However, the confident change work of partners was undermined when, as the project began in early 2020, arts organisations were forced to stop their in-person activities in response to Covid-19 restrictions. There followed a period of intense redesign, which suddenly also involved researchers in the production of art works alongside the analysis of them, informing on what would still be relevant to the research. There was a flourishing of discussion meetings across different parts of the project. But the lockdowns stopped researchers accessing the experiential artworks directly, made travel to different sites impossible and sent most activities online, where types of data gathering were more limited.

The idea of the researchers’ own participation changed completely as they lost access to much of the free-flowing data they were expecting to collect. But the nature of partner work intensified, not only involving the arts organisations in working on and reviewing material, but also looking towards policymakers for greater insight into their needs. The researchers ran workshops featuring the materials collected with policymakers and funders, for feedback, working on the premise that the way to grow engagement with materials from all sides is to co-create them. This input from policy also helped determine how to create the final deliverable in a form that might be useful to four audiences: creative practitioners, funders, policymakers and researchers.

Findings and new knowledge

The research team found that creative practitioners, policymakers, funders and researchers struggle to communicate with each other regarding the value of creative practices as a way to stimulate change. Relationships between creative practices and change processes are complex and often hard to grasp. They run the risk of being flattened and oversimplified on the one hand, or kept very vague and anecdotal on the other.  

The Nine Dimensions tool was developed to help reflective evaluations of how creative practice can bring about change – providing a multidimensional frame that captures the richness of creative practice while still making it manageable.

The tool emerged from design with our creative practitioners and from dialogues with policymakers and funders, as above. Importantly, the tool was also supported by an extensive literature review that helps provide a research basis for its change mechanisms and the analysis of 140 cases of practice from around the world. Check out the full version of the Nine Dimensions tool can be found on the CreaTures Framework website:

Lessons

The project’s ambitions for participation with partners’ participants were crushed by the timing – most of the project ran during the lockdowns in countries where these rules reduced the access that artists had to their usual communities. Consequently, transformation was also internal to the project, which was an interesting process in itself (revealing more about how the research team and partners think and work) and also led to a new emphasis for the project. As a result, the team learnt to run collaborative investigation workshops online. They also involved the potential recipients of the work we were producing, such as funders and policymakers, more closely. This engagement was always intended to take place, but now took centre stage. The effect was to strengthen impacts, though some of the anticipated findings were not as well developed as had been initially hoped.

It is difficult to pinpoint the methods that were changed as part of the project: the pandemic crisis had such an impact. But the researchers had designed the project with an understanding that everything is entangled and research into phenomena is necessarily transdisciplinary (involving practitioner partners) and participative in its methodology if it is to be useful.

To understand ourselves as working in an entangled universe is an alternative to isolating individual processes for analysis. The research team and partners still analysed, but approached the work as embedded in a complex system in which other changes, such as climate degradation and political polarisation, are significant, affecting what the team were attempting to investigate in emergent ways.

Not every project will wish to embrace this philosophical underpinning, yet, given the partners’ diverse goals, this conception was not only relevant as a shared belief, but a way to honour the intentions of plural collaborators and the factors influencing their choices of approach, which were drawn from a wide range of societal challenges and, in turn, influenced by them. The approach led to its own methods – the co-research team:

• Addressed “futures”, in the plural, and allowed every partner to bring (keep, and articulate) different ambitions for those futures, with different understandings of what it means to intervene.
• Allocated time to build common understandings across the different disciplines involved (design, art, ecology, etc.), through meetings, seminars, conversations and constant revisiting of themes.
• Established a grouping of research fellows answering as much to the project as to individual packages.
• Questioned the idea of direct impact as a goal or a possibility, to explore influences of many kinds.
• Introduced elements of theory into practical planning and execution (e.g. how entanglement in practice might entail different ways of thinking and designing, guided by ideas of care).
• Interrupted sessions repeatedly to turn ideas round in multiple ways, with a focus on connecting rather than isolating them (including graphically mapping the interrelations in themes that partners brought as transformative influences).
• Developed objects of study that included relations, emergences, genealogies and interrelations of influence (Light et al 2024).

Importantly for this work (and part of our ethical commitment), the project team regarded researchers and funders as creative, and creative partners as co-researchers; this cross-fertilisation was essential to the execution of the work, even when it was discovered that collaborators couldn’t always meet as planned. One related learning was that when creative partners are given funding for reflection as well as production, they are still inclined to use it for their creative work. Ringfencing some of this for meetings would have been wise.

Inevitably, participatory work can eat up all budgeted time and more. The team attempted to make time for one another, listen carefully and allow for the development of trust. Rushing relationship-building work across diverse partners is not efficient; fractures show later. The project could not have produced our other results without paying attention to these aspects and giving them the time they needed.