cesar working document 3 urban strategy experiment 2

CESAR WORKING DOCUMENT SERIES Working document no.3

Support for planning process and content Urban Strategy Experiment No2

M. te Brömmelstroet 13 April 2014

This working document series is a joint initiative of the University of Amsterdam, Utrecht University, Wageningen University and

Research centre and TNO

The research that is presented in this series is financed by the NWO program on Sustainable Accessibility of the Randstad: http://www.nwo.nl/nwohome.nsf/pages/nwoa_79vlym_eng

CESAR Working Document Series no. 3 Urban Strategy experiment no. 2

TABLE OF CONTENT

1. INTRODUCTION ............................................................................................................... 3

2. SETUP OF THE EXPERIMENT ............................................................................................ 4

2.1 Intervention: Urban Strategy PSS ......................................................................................... 4 2.2 Mechanisms: how does Urban Strategy bridge the PSS implementation gap? ................... 4 2.3 Setup of the controlled randomized trial ............................................................................. 5 Treatment ..................................................................................................................................... 6 Evaluation ..................................................................................................................................... 7

3. EVALUATION RESULTS ..................................................................................................... 8

3.1 Perceived quality of the planning process ............................................................................ 8 3.2 Perceived quality of the planning outcome .......................................................................... 8 3.3 Usability characteristics of Urban Strategy .......................................................................... 8

4. CONCLUSIONS AND IMPLICATIONS ............................................................................... 10

4.1 Reflection ........................................................................................................................... 10

5. CONCLUSIONS AND DISCUSSION ................................................................................... 11

5.1 General conclusions ........................................................................................................... 11

REFERENCES .......................................................................................................................... 12

APPENDIX I: COMPETITION SHEET (DUTCH) ......................................................................... 13

Introductie .................................................................................................................................. 13 Herinrichting Waalhaven oostzijde ............................................................................................. 14 Opdrachtomschrijving: ............................................................................................................... 15 Uitgangspunten en randvoorwaarden ........................................................................................ 15

APPENDIX II: EVALUATION FORM FOR QUALITY OF THE PROCES ........................................ 16

APPENDIX III: EVALUATION FORM FOR QUALITY OF THE OUTCOME ................................... 18

APPENDIX IV: EVALUATION FORM FOR USABILITY OF URBAN STRATEGY ........................... 19


1. INTRODUCTION

In this working document we report on a second randomized controlled trial with Urban Strategy to test the added value of this Planning Support System (PSS). Urban Strategy is a software package developed by TNO that aims to improve the planning process and planning outcomes of strategic planning. It does so by offering a range of quick models that show the effects of planning interventions in an easy to understand visual environment. To gain more insight into these potential improvements, we have conducted an experiment with a group of master students in Urban Planning of the University of Amsterdam. We make use of the measuring framework as introduced and discussed in CESAR Working Document No. 1. First, we describe the setup of the experiment (section 2). Then, the findings of the experiment are presented (section 3). In the fourth section we will briefly discuss the implications of these findings and further research.


2. SETUP OF THE EXPERIMENT

2.1 Intervention: Urban Strategy PSS

TNO started around 2005 with the development of a PSS – Urban Strategy (Borst et al. 2007; 2009a; 2009b) – specifically aiming to bridge the existing flexibility-‐ and communication bottlenecks. Urban Strategy aims to improve complex spatial planning processes on the urban-‐ and regional level. To do this, different computer models are linked to a central database and interface to provide insights in a wide area of urban indicators and maps. The effects of interventions in infrastructure, land use, build objects and their functions can be calculated and visualized. Because the PSS is able to calculate fast and present the results in an attractive 1D, 2D and 3D visualisation this can be used in interactive sessions with planning actors. Starting point for Urban Strategy is the use of existing state-‐of-‐the-‐art and legally accepted models. To link these existing models a number of new elements were developed: - a database with an uniform datamodel; - interfaces that show a 3D image of the modeled situation, indicators and that offer functionality

to add interventions; - a framework that structures the communication between the models and the interfaces.

2.2 Mechanisms: how does Urban Strategy bridge the PSS implementation gap?

The goal of Urban Strategy is to enable planning actors in workshop sessions to communicate their ideas and strategies to the PSS and to learn from the effects that are shown. This interactivity calls for fast calculations of all the model and fast communication between all elements. For this, the models were enabled to respond on events (urban interventions from the participants in the workshops. A new software architecture was developed to have all these elements communicate (figure 2). Through this increases speed and the wide variety of models that are linked together, the PSS aims to be highly flexible in offering answers to a large number of questions that a group of urban planning actors can have.

Figure 2 Schematic overview of communication architecture of Urban Strategy.

The 3D interface generates, based on objects in the database, a 3D digital maquette of the urban environment. To this, different information layers can be addes, such as air quality contours, noise contours and groundwater levels. Also, the objects can be colored according to their characteristics (function, energy use, CO2 emissions, number of inhabitants, etc). The 2D interface can be used by the end user (or operator) to add changes to the database. Objects can be added or removed, their


location can be changed and the characteristics of the object can be changed. The 1D interface shows indicators that are calculated by all the models that are included. Examples are the percentage of noise hindrance, group risk in an area or the contribution of types of objects to CO2 emission.

Figure 3 The three interfaces of Urban Strategy

2.3 Setup of the controlled randomized trial

Together with TNO we organized a number of sessions in which the usability of Urban Strategy was tested. For this we set up a typical urban planning challenge: Optimize existing ideas for new housing and working functions in an old harbor area of Rotterdam (figure 4). The existing ideas face problems due to restrictions on noise, external safety and air quality (figure 5).

Figure 4 The harbor area in the larger Rotterdam region


Figure 5 Current strategy for the area (top left), air quality norms (top right), noise norms (bottom left) and external safety (bottom right)

We invited 24 students from the course Metropolitan Transportation Planning, which is part of the one-‐year Master on Urban Planning of the University of Amsterdam. They were told that they would take part in a strategy making competition and were not aware of the treatment. They represent future planning practitioners that deal with planning challenges such as the abovementioned. The students were randomly split into six groups of each three students. Each group was then asked to perform the planning task as a group. One week before the strategy making sessions, they were introduced to the planning problem and criteria on which the developed strategies were going to be judged: innovation, workability, relevance and specificity. See appendix I for the (Dutch) form with information that each participant received. The six groups were, again randomly, divided into control and treatment groups. Three groups performed the planning task without support of Urban Strategy, while three other groups received this planning support. The control groups worked simultaneously, while the treatment groups did their work consecutively (due to constraints of the model interaction). All groups got 45 minutes to perform the task and were asked to finalize their strategy on a sheet of paper and an empty map 10 minutes before the deadline. The control groups were then left to perform their task. They were supported by the maps from figure 5 and empty maps to draw their plans. Also, pens and markers were provided (figure 6 left).

Treatment

The groups that were supported with Urban Strategy followed a fixed protocol. On arrival in the room, Ralph Klerkx (TNO) introduced Urban Strategy to the group. In 10 minutes, he showed the


functionality of the instrument and what it can and cannot do. After this introduction Ralph and Sander Schaminée (TNO) guided the groups in finding solutions for the planning problem. First, they were asked to come up with an improved design of the location, especially the place and orientation of the housing and working blocks. With this, the groups started to interact with Urban Strategy to see the effects of their interventions. During this interacting, the design was optimized to minimize the amount of housing and working under the noise, pollution and external safety norms. Also, strategies were developed to cope with other negative effects, such as the congestion and noise of the road adjacent to the study area. During these design-‐analysis iterations, the groups also were asked to write down the strategies and their reasoning. The strategy making took place in a classroom in which the group had a table to discuss and draw and could interact with the different interfaces of Urban Strategy (figure 6 right). Sander Schaminée supported this by drawing the proposed interventions into the model.

Figure 6 The setting of the control group (left) and the treatment group (right)

Evaluation

Two evaluation instruments were used to test the effects of the support by Urban Strategy. Using a 7-‐points Likert scale, all participants were asked to rate a number of statements related to the quality of the strategy making process (Appendix II). These perceptions were aggregated to the level of the evaluation dimensions and compared between the control-‐ and treatment groups. Next to that, the resulting strategies were collected and presented to two external raters. Using a 7-‐point Likert scale, they were asked to rate a number of statements related to the quality of the strategy making outcome (Appendix III). The strategies were presented to the raters without the hypotheses or indication of treatment/control. These perceptions were combined and aggregated to the level of the evaluation dimensions and compared between treatment and control. A final evaluation form was provided only to the participants of the treatment groups. They were asked to reflect on usability characteristics of Urban Strategy. Again, a 7-‐point Likert scale was used to gather their perceptions.


3. EVALUATION RESULTS

3.1 Perceived quality of the planning process

Due to the small number of observations we cannot assume to have controlled for all possible other influential variables. Also, we would like to raise awareness for the fact that we worked with students, for which it was often the first time to engage in such a strategy making process. There are however some remarkable outcomes. On the overall score (at right of table 1) we see no effect of the support by Urban Strategy. On the level of the dimensions, we see a meaningful positive effect on ‘effectiveness’, although this effect is not significant due to the large variation within the control-‐ and treatment group. Apparently, perceptions of this vary wildly. A very negative effect is found on ‘consensus on goals’, but again the found difference is not significant. Most effects on the other dimensions are close to zero. We can therefore not confirm a positive effect of Urban Strategy on the strategy making process.

3.2 Perceived quality of the planning outcome

The overall quality of the planning outcome (at right in table 2) is affected negatively by the support of Urban Strategy. This effect is not significant, but still noteworthy. Only ‘implementability’ is positively affected, while all other dimensions show a negative effect of Urban Strategy. This is most remarkable for ‘applicability’ and the related grouped dimension of ‘relevance’ where negative scores of respectively 1,5 and 0,78 are found. Again, we can conclude from these scores that we cannot confirm a positive effect of Urban Strategy on the strategy making outcome.

3.3 Usability characteristics of Urban Strategy

Although a positive effect of the instrument could not be confirmed, the participants were very positive about its usability. The scores in table 3 (on a 7 point scale and arranged from high to low) indicate that all usability dimensions score over 4,6. Most positive elements are the ‘support for evaluating alternatives’, ‘communicative value’ and ‘supporting the creation of ideas’.

cesar working document 3 urban strategy experiment 2

Technology