After reading Ian David Moss’s critique of the “creative placemaking” logic model (or lack thereof) I couldn’t resist doing a little research so that I could better understand the issue. After looking at the roots of creative placemaking in public art, urban design, and community cultural development, here are what I see as some of the issues of outcomes and measurement:
Livability, vitality, vibrancy. They sound similar, but are they synonymous? What makes a place “livable” and how do arts and culture contribute to that outcome? Is ArtPlace’s “vibrancy” outcome really coded language for gentrification and the displacement of people that often accompanies it?
Logic Models, Outcomes, and Indicators
Ian David Moss has been very critical of ArtPlace’s theory of change, comparing it to South Park’s “underwear gnomes” (collect underwear à make profit). Moss argues that without a clear, although not necessarily linear, logic model, it will not be possible to determine appropriate indicators for the program’s desired outcomes. Laura Zabel, whose organization SpringBoard for the Arts is an ArtPlace awardee, advocates a simple placemaking logic model: “artists –> love –> authenticity = places where people want to gather, visit & live.” The discrepancies between logic models and their outcomes seems to be a conflict of evaluative methodology as well as of goals. Vibrancy is inherently qualitative and economic development inherently quantitative. The assumptive relationship between vibrancy and economic development is analogous to that between Richard Florida’s three T’s (Talent, Technology, Tolerance) and economic health: there is an assumption of causation when there is only correlation.
Both Moss’s tongue-in-cheek and Zabel’s heartfelt critiques may be premature. Or, rather, off the mark, at least insofar as the NEA Our Town program is concerned (Our Town is the public sector sibling of ArtPlace). As part of the initiation of the Our Town program, the NEA launched a study of Our Town community indicators. The agency’s research office has articulated four dimensions of the program’s primary outcome, livability: affect on artists, attachment to community, quality of life, and economic conditions (see Shewfelt). For each of these dimensions, there are several proposed indicators based on hypotheses about whether and how creative placemaking efforts impact communities. In other words, creative placemaking is an experiment. Unlike a laboratory experiment in which an action is performed on a control group in carefully constructed circumstances, the program was launched with the intention of measuring various developmental affects against benchmarks and over time to see which specific indicators significantly reflect the impact of creative placemaking efforts. This approach is in stark contrast to Florida’s, who predicts a city’s success on the basis of a handful of reductive indices, despite the lip service paid to his influence.
ArtPlace does not provide the same clarity around its “vibrancy” outcome as is provided by the NEA regarding the dimensions of their “livability” outcome. Like Florida, ArtPlace relies on an index of vibrancy that results from an evaluation of three areas, people, activity, and value (see Cortright). The vibrancy indicators reflect the more blatantly economic focus of ArtPlace’s development initiatives. “Value,” for example, literally means property value and asks if property values increase in the area of a creative placemaking project. “Activity,” is not a measure of civic engagement or even, to use an NEA dimension, attachment to community, but business activity – how many and what type. There is a saying in the evaluation community, “you measure what you care about.” ArtPlace’s theory of change views community vibrancy through arts and culture as a strategy toward economic development; their metrics are economic. There has been pushback, primarily informal, against the exclusively economic focus of ArtPlace’s indicators and the organization is commissioning a new set of vibrancy indicators that was set to be released in May, 2012 [I looked, but couldn’t find them]. In the interim, the program has released a list of ten “signals” to be observed by grantees:
1 Is the neighborhood cleaner?
2 Does the neighborhood feel safer?
3 Is the neighborhood more attractive?
4 Are there fewer vacancies?
5 Are there more people on the sidewalks?
6 Is there a popular new outdoor gathering place?
7 Is there a popular new indoor gathering place?
8 Is there new evidence of arts activity?
9 Has the local press reported on it positively?
10 Do people in the neighborhood generally agree that the neighborhood is getting better?
What Is Not (Yet) Being Measured
Despite claims from ArtPlace that “we are keen believers in the intrinsic merits of arts investments,” neither ArtPlace nor the NEA are measuring the intrinsic impacts of increased arts and culture activity in a given place. There are several reasons why intrinsic impact measures are not included in creative placemaking evaluation. The first is that the unit of analysis of intrinsic impact is, by definition, the individual, while the target of placemaking efforts is at the community level. Second, there is little empirical data on effective methodology for the measurement of the intrinsic impact of the arts. A recent study by Theatre Bay Area provides a positive testing ground. That study, specifically focused on the intrinsic impact of theatre on its individual audience members, uses surveys, interviews, and focus groups to determine, in part, the degree to which a theatrical work created a lasting memory for an audience member. Because the construction of memory is part of the process of ‘space’ becoming ‘place,’ the qualitative study instruments show promise for use in assessing creative placemaking efforts. The challenges then become those of replicability and cost, both of which are criteria in the development of the NEA Our Town indicator study design.
As I have implied and Stern and Siefert make explicit, “Since its publication in 2002, The Rise of the Creative Class has been used by city officials from New York to Spokane as a how-to manual for stimulating economic growth. The realization that pursuing creative class strategies will actually exacerbate the divisions between rich and poor should give public officials pause.” The Our Town indicators for economic conditions include median income and loan amounts for housing but do not include a measure of income range or distribution.[i] Artplace claims that “Having a good balance between high, middle, and low-income families in a neighborhood is one key to promoting economic success and opportunity for neighborhoods in every income group,” but despite two years of grants totaling over $30 million does not yet have measures for that economic diversity. The effects of creative placemaking are potentially long-term. One hopes such measures will be in place by the time the effects are felt in two-three years.
Social transformation is a goal of the community cultural development movement that is not explicitly addressed in either the Our Town or ArtPlace indicators although diversity was emphasized in a very recent speech by ArtPlace president Coletta as equal to vibrancy in its theory of change. Diversity does not appear in the program’s grant guidelines, yet “demographics” are included among its outcome indicators in the “people” dimension. The Our Town program claims as a goal to help transform communities “into lively, beautiful, and sustainable places with the arts at their core.” That level of social transformation would not be reflected by the current list of indicators. The addition of social engagement metrics such as voter turnout in local elections or volunteer participation in the community may help the NEA assess the impact of the program on, or at least a correlation with, social transformation.
Closing Thought
Creative placemaking combines urban planning, public, and community cultural development practices to achieve multi-dimensional goals and outcomes. Quantitative measures such as median income or property values are easy to track but do not necessarily indicate the level or type of social transformation that creative placemaking funders claim to pursue. If we measure what we care about, creative placemakers will need to find more than just proxies for such transformation. They will need to see if the arts are causing real positive change, not only in communities but also in individuals via their varied experiences of unique places.
UPDATE: Approaching (Vibrant) Clarity
UPDATE #2: The Pave Program in Arts Entrepreneurship is hosting a symposium on “Entrepreneurship, the Arts, and Creative Placemaking” April 12-13 in Tempe AZ. You can read more about it and find a link to registration on Pave’s public programming page.
[i]In an email to Steven Shewfelt of the NEA Office of Research and Evaluation, I wrote the following:
You and your colleague indicated a sincere willingness to listen to suggestions for addition indicators and I offer several thoughts here:
- One of the questioners in the podcast mentioned social engagement as a measure of “attachment to community” and suggested looking at modes of transportation as an indicator of social engagement. This seems, at least to me, a fairly indirect, or at least at attenuated, indicator. I would go further than even social engagement and suggest that one’s attachment to community is exercised via civic engagement. A simple and replicable indicator of civic engagement is voter turnout, especially voter turnout at local-level (congressional district, city council, school district) elections. Another indicator is volunteerism. I am not familiar enough with the ACS to know if this is measured, but if so, increases in volunteerism (again, at the local level) would be an indication of attachment to community. Both are indicators that would be removed from the economic externalities that might affect owner occupation of property.
- Several times in your presentation, you mentioned the importance of watching out for the negative impacts of gentrification and development, especially the out-migration of long term residents. While that is captured in the length of residency measure, the unintended economic inequities that sometimes result from community change is not. Would it be possible and useful to consider not only median income, but range and distribution of income within communities?
- Finally, because of my particular interest in arts incubation (in the new venture development sense), would it be possible or useful to include as an indicator the number of new businesses launched that are either artist-owned and operated and/or are in the arts/culture/creative industries sector. Although the arts incubator with which I am involved is quite small, one of the few concrete measurements of our impact is in the number of ventures launched.
Creative Infrastructure 
Pingback: Cultural Infrastructure, Cultural Districts, and Creative Places | Creative Infrastructure
Pingback: Entrepreneurship, the Arts, and Creative Placemaking | Creative Infrastructure
Pingback: Around the horn: Frankenstorm edition | Createquity.
Pingback: Approaching (Vibrant) Clarity | Creative Infrastructure
Pingback: Means and Ends | Creative Infrastructure
I am not a fan of “creative placemaking” as it is defined by Markusen, used by the NEA, and promoted by ArtPlace. The focus, as Linda notes, is almost solely on the economic, part of the “monoculture” (see the book by that name written by F. S. Michaels) that is distorting all aspects of our society. But hey. it’s measurable, right? Furthermore, as I recently noted in a conference discussion I attended, and what the subtext of Florida’s work and many of the ten indicators listed by ArtPlace (clean, safe, attractive) suggest, is a definite orientation toward luring educated middle-class white folks to places because they are the “type of people” we want. Discrimination by another name…
As a veteran of the “assessment” movement in higher education, I am weary of the “test what you can measure” approach. That said, I don’t think a focus on “intrinsic” value is the answer, either, because it is so indirect. I would argue that , in at least some cases, we might measure something about social capital. Are artists providing an opportunity for people to speak to each other about things that matter to their lives? This, of course, implies a certain type of arts event, one that explicitly creates such opportunities rather than having people sit quietly in a darkened theatre after which they are herded out the door as quickly as possible. What disturbs on some level is that underlying so much of discussion about “placemaking” is a desire to keep doing exactly what we’ve always done but justify it in terms we think non-artists will find persuasive. To my mind, we should be focusing on creating change within the art that makes the arts obviously invaluable to society.
Sometimes this discussion makes me sad. I imagine a ongoing conversation by a group of farmers whose chickens have been stolen by the fox. The farmers stand around the whole in the fence discussing how it happened. Should they measure the hole in the fence to find out how big the fox was or maybe try to come up with indicators that suggest how many times the fox entered the hole or maybe it would be better to determine which day of the week was the better day for the fox to have used the hole? Meanwhile all the chickens are gone.
Richard Florida’s mantra and measurability message came at an interesting time in the history of public support for the Arts. It was the nail in the coffin that represented actual support for those who produce the cultural value in society – the artists. The conservative right had managed to strip funding from those who produce the arts and award it to those organizations and agencies that present art to the public and Florida’s message of measuring the economic and social impact of the arts became the obsession of these new gatekeepers of culture. Now the public and the politicians had a new tool to control scary cultural production, if it was important enough to award public tax dollars to then those paying those dollars and those managing those dollars had the right to see the effects of those dollars spent. And our arts organizations and arts advocates fell in marching step to the new orders. Now all you seem to do is argue about how to best measure the hole that the fox went through.
Hi Linda,
I don’t find much to disagree with here, but I feel like you’re talking past my critique a little bit instead of directly engaging with it. I wrote my creative placemaking post out of frustration that we were missing an incredible opportunity (in my opinion) to learn more about the various ways in which creative placemaking can be really transformative for communities. ArtPlace in particular is quite explicit about wanting to have that effect with its grantmaking, and frankly that’s a really high bar to try and meet. I’m skeptical that just any old grant is going to do it – there is probably a pretty limited set of situations in which it’s going to work. Given that, we should be designing our research with the kind of sensitivity that can pick up the nuances of strategy and execution, not just “we gave a grant here, this happened (or didn’t).” Just picking a bunch of stuff to measure without a clear and specific theoretical connection to the activities paid for by the grant isn’t going to do that, unless we get really lucky. Now, don’t get me wrong, I’m totally impressed with Joe’s work on the vibrancy indicators and I’m not trying to suggest that that project has no value – quite to the contrary. I just don’t think it’s the right instrument on its own to give us a thorough understanding of what kinds of creative placemaking projects are effective in achieving their goals.
I do think this point of yours is an interesting one: “The discrepancies between logic models and their outcomes seems to be a conflict of evaluative methodology as well as of goals. Vibrancy is inherently qualitative and economic development inherently quantitative….there is an assumption of causation when there is only correlation.” You could argue on the qualitative vs. quantitative distinction, as it should be possible to develop quantitative proxies for vibrancy, and economic development could also be considered qualitative depending on your definition. But I get what you mean and it’s an interesting way to think about the issue.
Finally, in response to Laura, I’d just say that my focus is less on metrics per se than on clearly articulated strategies. I have no problem with trying to determine what other people care about, but throwing a bunch of stuff at a wall and seeing what sticks is not the approach I would advocate for distributing millions of dollars in grant money.
Ian – Your (and Laura’s) critiques inspired me to look at the topic, not necessarily engage directly with your critique — so you’re right, I’m not. Rather, I question what values are reflected in what gets measured. Then, on top of that is the question of quantitative proxies for qualitative outcomes. Ultimately, ArtPlace and to a lesser extent Our Town seem to be, as evidenced by what they choose to measure, economic development initiatives that *use* arts and culture as their instruments, rather than programs that support arts and culture per se.
Got it. Thanks for the clarification. I’d agree with you that the creative placemaking initiatives are not really focused on the intrinsic values of the art. (I also think that’s just fine, given that creative placemaking is likely to continue to remain a relatively small piece of the overall arts funding puzzle.)
Interesting stuff here, Linda. Just for clarity’s sake – my post about logic models is not intended to be a critique of creative placemaking or ArtPlace, Our Town or arts as economic development at all In fact, quite the opposite – I’m a pretty diehard believer in the idea that this is the path to building public value. It’s meant to be a critique of Ian’s critique! I’d rather take an iterative, “sow lots of seeds and see what works” approach to this work than see us put so much pressure on perfect outcomes and measurements that will exclude good, innovative, experimental work. Despite Ian’s convincing arguments (and considerable charm) I’m still not sold on logic models as an effective tool, particularly when we are trying to demonstrate value outside our own sector. I wrote that post more as a call for patience and trust in our communities. My feeling is, yes, we need measurements, but while we’re working on developing those metrics, we can sleep easy knowing that on the ground, we can trust our communities to tell us what is effective, authentic work.
To me, the strength of ArtPlace’s approach to metrics so far is that it seems to be based, not necessarily in measuring “what we care about” but in trying to determine and measure what other people care about-something I fear we haven’t paid enough attention to in the arts.