After reading Ian David Moss’s critique of the “creative placemaking” logic model (or lack thereof) I couldn’t resist doing a little research so that I could better understand the issue. After looking at the roots of creative placemaking in public art, urban design, and community cultural development, here are what I see as some of the issues of outcomes and measurement:
Livability, vitality, vibrancy. They sound similar, but are they synonymous? What makes a place “livable” and how do arts and culture contribute to that outcome? Is ArtPlace’s “vibrancy” outcome really coded language for gentrification and the displacement of people that often accompanies it?
Logic Models, Outcomes, and Indicators
Ian David Moss has been very critical of ArtPlace’s theory of change, comparing it to South Park’s “underwear gnomes” (collect underwear à make profit). Moss argues that without a clear, although not necessarily linear, logic model, it will not be possible to determine appropriate indicators for the program’s desired outcomes. Laura Zabel, whose organization SpringBoard for the Arts is an ArtPlace awardee, advocates a simple placemaking logic model: “artists –> love –> authenticity = places where people want to gather, visit & live.” The discrepancies between logic models and their outcomes seems to be a conflict of evaluative methodology as well as of goals. Vibrancy is inherently qualitative and economic development inherently quantitative. The assumptive relationship between vibrancy and economic development is analogous to that between Richard Florida’s three T’s (Talent, Technology, Tolerance) and economic health: there is an assumption of causation when there is only correlation.
Both Moss’s tongue-in-cheek and Zabel’s heartfelt critiques may be premature. Or, rather, off the mark, at least insofar as the NEA Our Town program is concerned (Our Town is the public sector sibling of ArtPlace). As part of the initiation of the Our Town program, the NEA launched a study of Our Town community indicators. The agency’s research office has articulated four dimensions of the program’s primary outcome, livability: affect on artists, attachment to community, quality of life, and economic conditions (see Shewfelt). For each of these dimensions, there are several proposed indicators based on hypotheses about whether and how creative placemaking efforts impact communities. In other words, creative placemaking is an experiment. Unlike a laboratory experiment in which an action is performed on a control group in carefully constructed circumstances, the program was launched with the intention of measuring various developmental affects against benchmarks and over time to see which specific indicators significantly reflect the impact of creative placemaking efforts. This approach is in stark contrast to Florida’s, who predicts a city’s success on the basis of a handful of reductive indices, despite the lip service paid to his influence.
ArtPlace does not provide the same clarity around its “vibrancy” outcome as is provided by the NEA regarding the dimensions of their “livability” outcome. Like Florida, ArtPlace relies on an index of vibrancy that results from an evaluation of three areas, people, activity, and value (see Cortright). The vibrancy indicators reflect the more blatantly economic focus of ArtPlace’s development initiatives. “Value,” for example, literally means property value and asks if property values increase in the area of a creative placemaking project. “Activity,” is not a measure of civic engagement or even, to use an NEA dimension, attachment to community, but business activity – how many and what type. There is a saying in the evaluation community, “you measure what you care about.” ArtPlace’s theory of change views community vibrancy through arts and culture as a strategy toward economic development; their metrics are economic. There has been pushback, primarily informal, against the exclusively economic focus of ArtPlace’s indicators and the organization is commissioning a new set of vibrancy indicators that was set to be released in May, 2012 [I looked, but couldn’t find them]. In the interim, the program has released a list of ten “signals” to be observed by grantees:
1 Is the neighborhood cleaner?
2 Does the neighborhood feel safer?
3 Is the neighborhood more attractive?
4 Are there fewer vacancies?
5 Are there more people on the sidewalks?
6 Is there a popular new outdoor gathering place?
7 Is there a popular new indoor gathering place?
8 Is there new evidence of arts activity?
9 Has the local press reported on it positively?
10 Do people in the neighborhood generally agree that the neighborhood is getting better?
What Is Not (Yet) Being Measured
Despite claims from ArtPlace that “we are keen believers in the intrinsic merits of arts investments,” neither ArtPlace nor the NEA are measuring the intrinsic impacts of increased arts and culture activity in a given place. There are several reasons why intrinsic impact measures are not included in creative placemaking evaluation. The first is that the unit of analysis of intrinsic impact is, by definition, the individual, while the target of placemaking efforts is at the community level. Second, there is little empirical data on effective methodology for the measurement of the intrinsic impact of the arts. A recent study by Theatre Bay Area provides a positive testing ground. That study, specifically focused on the intrinsic impact of theatre on its individual audience members, uses surveys, interviews, and focus groups to determine, in part, the degree to which a theatrical work created a lasting memory for an audience member. Because the construction of memory is part of the process of ‘space’ becoming ‘place,’ the qualitative study instruments show promise for use in assessing creative placemaking efforts. The challenges then become those of replicability and cost, both of which are criteria in the development of the NEA Our Town indicator study design.
As I have implied and Stern and Siefert make explicit, “Since its publication in 2002, The Rise of the Creative Class has been used by city officials from New York to Spokane as a how-to manual for stimulating economic growth. The realization that pursuing creative class strategies will actually exacerbate the divisions between rich and poor should give public officials pause.” The Our Town indicators for economic conditions include median income and loan amounts for housing but do not include a measure of income range or distribution.[i] Artplace claims that “Having a good balance between high, middle, and low-income families in a neighborhood is one key to promoting economic success and opportunity for neighborhoods in every income group,” but despite two years of grants totaling over $30 million does not yet have measures for that economic diversity. The effects of creative placemaking are potentially long-term. One hopes such measures will be in place by the time the effects are felt in two-three years.
Social transformation is a goal of the community cultural development movement that is not explicitly addressed in either the Our Town or ArtPlace indicators although diversity was emphasized in a very recent speech by ArtPlace president Coletta as equal to vibrancy in its theory of change. Diversity does not appear in the program’s grant guidelines, yet “demographics” are included among its outcome indicators in the “people” dimension. The Our Town program claims as a goal to help transform communities “into lively, beautiful, and sustainable places with the arts at their core.” That level of social transformation would not be reflected by the current list of indicators. The addition of social engagement metrics such as voter turnout in local elections or volunteer participation in the community may help the NEA assess the impact of the program on, or at least a correlation with, social transformation.
Creative placemaking combines urban planning, public, and community cultural development practices to achieve multi-dimensional goals and outcomes. Quantitative measures such as median income or property values are easy to track but do not necessarily indicate the level or type of social transformation that creative placemaking funders claim to pursue. If we measure what we care about, creative placemakers will need to find more than just proxies for such transformation. They will need to see if the arts are causing real positive change, not only in communities but also in individuals via their varied experiences of unique places.
UPDATE: Approaching (Vibrant) Clarity
UPDATE #2: The Pave Program in Arts Entrepreneurship is hosting a symposium on “Entrepreneurship, the Arts, and Creative Placemaking” April 12-13 in Tempe AZ. You can read more about it and find a link to registration on Pave’s public programming page.
[i]In an email to Steven Shewfelt of the NEA Office of Research and Evaluation, I wrote the following:
You and your colleague indicated a sincere willingness to listen to suggestions for addition indicators and I offer several thoughts here:
- One of the questioners in the podcast mentioned social engagement as a measure of “attachment to community” and suggested looking at modes of transportation as an indicator of social engagement. This seems, at least to me, a fairly indirect, or at least at attenuated, indicator. I would go further than even social engagement and suggest that one’s attachment to community is exercised via civic engagement. A simple and replicable indicator of civic engagement is voter turnout, especially voter turnout at local-level (congressional district, city council, school district) elections. Another indicator is volunteerism. I am not familiar enough with the ACS to know if this is measured, but if so, increases in volunteerism (again, at the local level) would be an indication of attachment to community. Both are indicators that would be removed from the economic externalities that might affect owner occupation of property.
- Several times in your presentation, you mentioned the importance of watching out for the negative impacts of gentrification and development, especially the out-migration of long term residents. While that is captured in the length of residency measure, the unintended economic inequities that sometimes result from community change is not. Would it be possible and useful to consider not only median income, but range and distribution of income within communities?
- Finally, because of my particular interest in arts incubation (in the new venture development sense), would it be possible or useful to include as an indicator the number of new businesses launched that are either artist-owned and operated and/or are in the arts/culture/creative industries sector. Although the arts incubator with which I am involved is quite small, one of the few concrete measurements of our impact is in the number of ventures launched.