Introduction.tex

\chapter{Introduction}
\label{chapter:introduction}

Often when developers are faced with a design challenge, they will turn to the whiteboard.  This is typical during the conceptual stages of software design, when no code is in existence yet, but may also happen when a significant code base has already been developed, for instance, to plan new functionality or discuss optimizing a key component. Design sessions at the whiteboard may even arise spontaneously, such as when a developer has to refactor some code or discuss how to best integrate a new feature.

Compared to the polished, precise, and typically detailed models software designers produce using modern software design tools, the content they create at the whiteboard consists of rough sketches and imprecise approximations of the design they have in mind, which are continuously modified and refined as part of the design activity [3]. The sketches on the whiteboard wall shown in Figure \ref{figure:software-whiteboard} illustrate this point. The sketches are the result of many design sessions at a startup software company. While the sketches clearly are unintelligible to those who were not present during the design sessions, they serve a crucial role for those who were: they were the vehicle for working through a complex design problem and making key decisions that defined the software to be developed.

\begin{figure*}[tbh]
  \centering
  \includegraphics[width=16cm,keepaspectratio]{./figures/software-whiteboard}
  \caption{Whiteboard wall at a startup company.}
  \label{figure:software-whiteboard}
\end{figure*}

Developers turn to the whiteboard for the flexibility and fluidity that it offers in the design experience \cite{cherubini2007let}. On a whiteboard, developers can freely sketch, branch off to another part of the design problem, return to a previous part, erase some portion of their work, redraw it, and so on, all without the typical restrictions one might find in a traditional software design environment. They can focus on designing without worrying about particular notations, how to navigate within a tool, etc.

While a preferred medium for design, a significant disadvantage of the whiteboard is that it is a passive medium: it has no facilities that purposefully support the design process. Particularly, whatever is drawn or written remains static and cannot be manipulated, other than drawing over it or erasing it. This is a less than desirable situation, because it is known that software designers often wish to manipulate a design at hand in more advanced ways than merely adding or erasing content \cite{dekel2007notation}. The whiteboard, thus, limits what they are able to do, or at least how efficient they might be at doing so.

The research community has acknowledged this problem and has contributed many approaches that rely on an electronic whiteboard to provide more advanced support (e.g., \cite{chen2008sumlow, landay1995interactive, hammond2006ladder, damm2000supporting, chung2005inkkit}). This support can be divided into two broad categories: (1) approaches that focus on sketch recognition, and (2) approaches that focus on management of sketched content. Approaches in the first category, sketch recognition, attempt to interpret the strokes made by the user to turn them into formal  objects. Early work offered a predefined visual vocabulary for converting sketches into formal objects, such as UML diagrams \cite{chen2008sumlow} or user interface mockups \cite{landay1995interactive}, with tools that provided feedback to the designer based on the rules of the formal notation they support. Later work made visual vocabularies expandable by users \cite{hammond2006ladder} and made using the tools more flexible by delaying interpretation until it was desired by the user \cite{damm2000supporting}, sometimes even while retaining a sketchy appearance \cite{chung2005inkkit}.

Approaches in the second category, sketch management, help organize the potentially many and varied sketched artifacts that may be produced during meetings (Figure \ref{figure:software-whiteboard} is an illustration of this point). Early approaches provided access to a large number of whiteboards through a filmstrip \cite{stefik1987beyond}, hyperlinks \cite{Streitz:1994:DIM:192844.193044}, or hierarchical perspectives \cite{newman2003denim}. Later work automated particular aspects of managing sketches by automatically grouping clusters of sketches in close spatial proximity \cite{mynatt1999flatland}, shrinking sketches when moved to the periphery \cite{guimbretiere2001fluid}, or using metaphors such as Post-It Notes to organize and relate sketches \cite{klemmer2001designers}.  

In examining these and other existing sketching tools, it is useful to consider their respective underlying motivations. In so doing, I observe that every sketch tool was designed to support a particular way of working at the whiteboard. For instance, Knight supports designers in refining initial rough sketches into more formal representations \cite{damm2000supporting}. As another example, Flatland supports designers in creating many different diagrams by automatically clustering sketches and adding specialized behaviors to those clusters \cite{mynatt1999flatland}.

In this dissertation, I define these ways of working as design behaviors. More precisely, I define a design behavior as a recurrent, recognizable set of actions serving a single purpose within a design meeting. In the remainder, I scope these actions to pertain to the whiteboard, that is, focus on creating and modifying its content, navigating the content created, and collaborating in all of these. That is, two designers calling up a colleague is not a design behavior in which I am interested.  One designer refining a diagram drawn by another is.

Quite a few design behaviors have been identified in the literature, despite the fact that the study of software designers ``in action'' is still in its infancy. For example, in addition to refinement of sketches and supporting multiple different types of sketches, studies of designers at OOPSLA’s DesignFest found that software designers improvise their own notations and evolve their diagrams across many canvases \cite{dekel2007notation}. As another example, in-the-field observations at software companies found that software designers deliberately switch among formalisms and use provisionality to engage in a dialog with incomplete ideas \cite{petre2009insights}. 

Broadly speaking, design behaviors that software designers perform can be be described as falling into three high level categories. The  first is how they create and modify the kinds of things they draw, such as the different types of sketches they create and their improvisation of notations. The second is how they navigate those sketches, such as shifting focus between sketches of different types. The third is how they collaborate over their sketches, such as when designers switch from working together on a sketch synchronously to working asynchronously. 

The key insight motivating our research is that software designers do not ``operate in'' or apply just one behavior for an entire design meeting. Rather, designers interleave design behaviors over the course of a design meeting, switching among them as they see fit to navigate a design problem and its potential solutions. For instance, a designer may first sketch two diagrams and juxtapose them side-by-side to evolve them in parallel, then record patterns of execution in one of the diagrams using an impromptu notation, and thereafter shift to a different aspect of the design problem altogether. Throughout, the designer fluidly exhibits a range of these different design behaviors, typically without an explicit trigger. Designers shift opportunistically, addressing the design at hand in the way they see most beneficial at that moment.

During a meeting, it is a natural choice for designers to limit themselves to a single tool to support them. That tool is typically a whiteboard or paper \cite{petre2009insights}, though in some cases it may be a computerized tool like the ones I have described above (e.g., SUMLOW \cite{chen2008sumlow}, Knight \cite{damm2000tool}, Flatland \cite{mynatt1999flatland}. In the latter case, the choice of tool determines the behavior or small set of behaviors that are now supported, as designers will seldom move between tools during meetings, because of the high cost associated with switching, both in terms of the cognitive burden on the user to switch contexts and in terms of the effort required to import or manually copy the contents. The cost is simply too great and designers, thus, are stuck with support for at best a few of their behaviors as embedded in the tool they happen to be using.

What is desired is a tool that supports a broad range of behaviors and allows developers to use the features naturally when they need them. Creating such a tool, however, is a non-trivial exercise. Simply picking up functionality from one tool and dropping it in another, and doing this repeatedly to support a multitude of behaviors, leads to tools that are highly disjoint. It is unclear, for instance, what it would mean for a tool to have available both multiple canvases in a filmstrip and functionality that automatically makes room on the current board? As another example, a tool that automatically recognize sketches and also supports emergent notations is equally difficult to envision as being conceptually clear to its users. Existing solutions do not necessarily stack their functionality gracefully, and the approach taken by one tool may collide with the support provided by another.

This dissertation explicitly addresses the interleaving behaviors that designers exhibit during software design at the whiteboard by taking a step back, examining a collection of behaviors, and contributing a new tool that is designed from the ground up to support this collection of behaviors with a small set of conceptually coherent functionalities. In order to put this into practice, I first built a basic sketching interface so that, as a baseline, designers may sketch and perform the same activities as they normally would on a whiteboard, but in a digital medium. Building on this foundation, I incrementally introduced features that support one or more design behaviors, yet minimally obstruct the support for other behaviors. 

This approach was realized by the construction of a software tool called Calico. In this dissertation, I describe two versions of the tool, the first of which is an initial exploration in supporting a subset of the design behaviors with a sketch-based tool, and the second of which significantly iterates on the first to support the complete set of design behaviors I set out to support. 

The first version of Calico supported an initial subset of the design behaviors with three main features. Figure \ref{fig:calico-version-one} depicts the interface for this version of Calico. The first feature, scraps, supported the kinds of sketches that designers created by providing a mechanism to create representations for box-and-arrow type diagrams and for manipulating sketches. The second feature, the grid, supported designers in navigating between sketches by providing a grid-layout of all canvases. Depicted in Figure \ref{fig:calico-version-one-grid}, users can partition their work over the canvases in the grid, and move the resulting canvases around to organize their work. The third feature, the palette, allowed users to save scraps (depicted on the right of Figure \ref{fig:calico-version-one-canvas}), and reuse them by dragging them back onto the canvas. Chapter 3 explains these features in greater detail.

\begin{figure}
  \centering
  \subfigure[Canvas perspective] {
      \label{fig:calico-version-one-canvas}
      \includegraphics[width=8.5cm,keepaspectratio]{./figures/CalicoVersionOneCanvas}
%      \resizebox{.45\hsize}{.35\hsize}{ }
   }
  \subfigure[Grid perspective] {
      \label{fig:calico-version-one-grid}
      \includegraphics[width=6.5cm,keepaspectratio]{./figures/CalicoVersionOneGrid}
%      \resizebox{.45\hsize}{.35\hsize}{ }
   }
   \caption {Calico version one}
   \label{fig:calico-version-one}
\end{figure}

Calico version one was evaluated in a controlled laboratory study in which I compared the design activity of computer science graduate students engaged in a challenging software design problem using Calico against those using a regular whiteboard. I presented all participants with a prompt to design an educational traffic simulator and asked them to either design it using either Calico or the whiteboard. I found that the same design behaviors performed at the whiteboard were also performed in Calico, but in a slightly different manner. The participants in the Calico groups did use Calico's advanced feature set to perform the design behaviors, and reported that they found the features useful. However, the participants rarely used the palette. The design conversation of the groups were also analyzed using protocol analysis, which was done by breaking down the sessions into segmented phrases belonging to different categories of design. The breakdown of the categories demonstrated a very high correlation with reports from past research of how software design was performed at actual software companies, providing evidence that the design as it happened in our study matched that of design ``in the wild''. Lastly, the groups reported that the limitation of the interactive whiteboard used to only accept a single person drawing at a time hindered their ability to perform parallel work.

Following our experiences with Calico Version One, I rebuilt the tool from the ground up in Calico version two. The features in the second version of Calico significantly iterate on those in the first version to support the whole set of fourteen design behaviors I identified as crucial to designers at the whiteboard. This version is depicted in Figure \ref{fig:calico-version-two}. The scraps feature was significantly revised based on user feedback to better support the designers in the kinds of sketches designers create. The grid feature was replaced with intentional interfaces, depicted in Figure \ref{fig:calico-version-two-ii}, in order to better support designers in navigating among projects and the canvases capturing each project. In our experience with the grid, I saw that users typically dedicated entire rows or columns to a single topic. That behavior inspired intentional interfaces, in which canvases are layed out in a circle, and a set of canvases focusing on a particular topic can extend outwards radially (Figure \ref{fig:calico-version-two-ii}). Relationships between associated canvases are explicitly captured as well using light-weight tagging. Lastly, the software architecture of Calico itself was rebuilt using a client-server architecture in order to allow designers on different machines to collaborate on the same sketch. Chapter 5 explains these features, as well as others, in more detail.

\begin{figure}
  \centering
  \subfigure[Canvas perspective] {
      \label{fig:calico-version-two-canvas}
      \includegraphics[width=8.5cm,keepaspectratio]{./figures/CalicoVersionTwo/CalicoVersionTwoCanvas}
%      \resizebox{.45\hsize}{.35\hsize}{ }
   }
  \subfigure[Intentional interfaces perspective] {
      \label{fig:calico-version-two-ii}
      \includegraphics[width=6.5cm,keepaspectratio]{./figures/CalicoVersionTwoCluster}
%      \resizebox{.45\hsize}{.35\hsize}{ } 
   }
   \caption {Calico version two}
   \label{fig:calico-version-two}
\end{figure}

Calico Version Two was evaluated by deploying it at three field sites, including a commercial open source software company, an interaction design company, and a geographically distributed research group. All groups had access to a large electronic whiteboard running a Calico client, as well as pen-based tablets to connect to a shared Calico server. Members of each respective group conducted extensive design sessions using Calico to perform their own work within their respective organization. I collected and reviewed usage logs from the Calico server, and also conducted semi-structured face-to-face interviews. From this data, I evaluated the use of Calico's features in supporting the representations they created, their strategies for navigating between sketches, and how they collaborated on their sketches. 

Overall, the field evaluations gave insight into how Calico supports design in a real world setting. Usage at the commercial open source software company provided insight in how Calico supported a group engaged in detailed coding, with the group at large and individuals using Calico multiple times, and also personal sessions over both software components and software code. Usage at the interaction design group provided insight into how Calico helped them create personas and storyboards, for example, by organizing pictures of those they interviewed in one- and two-dimensional plots. Lastly, usage by the research group provided insight into how Calico supported a set of distributed researchers in a long-term collaboration. Their use included the development of a software system over a period of several months, with intensive sessions during which they designed process flow diagrams to model the behavior of their software. Overall, the four features collectively supported all fourteen of the design behaviors both in ways that I did and did not expect. 

\section{Thesis structure}

This thesis is organized into nine chapters. The remaining chapters are structured as follows:

\textbf{Chapter \ref{chapter:motivation} - \nameref{chapter:motivation}.} This chapter introduces the definition of design behaviors in detail and presents the grand set of design behaviors that I aim to support. The research question of this thesis is also introduced here.

\textbf{Chapter \ref{chapter:research-question} - \nameref{chapter:research-question}.} We pose the topic that this dissertation will address, layout a method by which I approach this topic, and how I will evaluate this approach.

\textbf{Chapter \ref{chapter:calico-version-one} - \nameref{chapter:calico-version-one}.} I describe the approach and features of the first version of Calico. This chapter introduces the first concepts of scraps, the grid, as well as the palette. The approach is evaluated in a comparative study between Calico and the regular whiteboard, both with respect to the design behaviors performed as well as the design conversations that took place.

\textbf{Chapter \ref{chapter:calico-version-two} - \nameref{chapter:calico-version-two}.} The second and final version of Calico is introduced in this chapter. The new features are introduced, including the revised interaction for scraps, the intentional interfaces features that replaces the grid, the distributed nature of the architecture, as well as other features targeted at supporting the full set of design behaviors.

\textbf{Chapter \ref{chapter:evaluation} - \nameref{chapter:evaluation}.} Here I present our evaluation of the final version of Calico ``in the wild'' at a commercial open source company, an interaction design company, and a distributed research group. A comprehensive evaluation was performed across three field sites, examining the use of features to support the representations used, the navigation among the sketches, and the collaboration patterns.

\textbf{Chapter \ref{chapter:discussion} - \nameref{chapter:discussion}.} In this chapter, I take a step back and put my work in a broader context. I discuss the implications of my findings, reconnect my work to the broader design literature, and provide some findings and observations that are outside the evaluative framework of the design behaviors, but are worthwhile to highlight nonetheless. 

\textbf{Chapter \ref{chapter:related-work} - \nameref{chapter:related-work}.} I review the existing literature of tools that support software design on an electronic whiteboard, covering both tools that support sketch recognition and those that support the management of sketches.

\textbf{Chapter \ref{chapter:conclusions} - \nameref{chapter:conclusions}.} This chapter summarizes the contributions of my dissertation, and also provides some concluding remarks, and offers an outlook at future work. I particularly suggest the following future avenues of research : (1) the concept of compositional notations, which builds on scraps to provide more targeted support for software design notations, (2) informal analysis of scraps in order to provide intelligent feedback on designs in progress such as critiquing the performance of an activity diagram, (3) contextualized sketching, which may import artifacts from an existing code repository to help with design discussions in Calico, and (4) design histories, which may help with reflecting on design activities performed.

%%% Local Variables: ***
%%% mode: latex ***
%%% TeX-master: "thesis.tex" ***
%%% End: ***