Some polishing, some small examples added to RQs

Abstract.tex

@@ -5,5 +5,6 @@
 
 In this paper, we present an in-depth empirical study of how developers perceive (1) how to approach merge conflicts, (2) what factors impact the difficulty of a merge conflict resolution, and (3) what toolset changes should be made to improve the resolution process.
 To learn more about these perceptions, we interviewed 10 experienced developers, then extended our findings with a survey of 226 developers. We further divided these developers by gender, developer role, experience, source distribution model, and project size.
-Our findings suggest that developers have several universal difficulties that cross all demographics, that 
+In this paper, we contribute a ranked list of factors that cause difficulty for developers, a ranked list of tool needs for merge conflict resolution, and personal accounts of experiences in resolving merge conflicts.
+
 \end{abstract}

Introduction.tex

@@ -6,19 +6,19 @@ \section{Introduction}\label{introduction}
 \todo{Do we want to cite Caius/Iftekhar paper?}
 Version control systems like Git offer easier collaboration via an automated merging process. This process enables large-scale distributed development, but as sophisticated as these tools are, they still require manual merging in ambiguous cases called merge conflicts. Sarma et al \cite{cassandra} shows that merge conflicts are as frequent as 19\% of merges, and Brindescu \& Ahmed et al (maybe) shows that [\% of merges that are conflicts] of merges result in conflict. Resolving merge conflicts can be a costly process that delays projects while developers decide how to approach and resolve the conflict \cite{cassandra}, and poorly-performed merge conflict resolutions frequently cause integration errors \cite{bird-branches-conflict}. 
 
-\comment{\textbf{***current work on proactive detection***}}
+\comment{***current work on proactive detection***}
+
 Several studies have examined models for proactive merge detection and prevention~\cite{Brun2011}\cite{palantir}\cite{Guimaraes}, proposed tools and systems for efficiently avoiding or resolving merge conflicts~\cite{nishimura}\cite{mens2002state}, or discussed advantages of syntax- and semantic-aware merges \cite{danny_refactorings}\cite{hunt2002extensible}. However, at some point in a software projects' development history, there is the chance that a merge conflict will still occur. 
 
-\comment{\textbf{***we need to know more about conflicts because...***}}
+\comment{***we need to know more about conflicts because...***}
 
 Despite the focus in this area, few studies have actually talked to developers to understand their perceptions about merge conflicts. Talking directly to developers is crucial to understanding their problems from their perspectives, since some details of merge conflict resolution cannot be observed by mining a GitHub repository. 
 
-
 Our research questions are as follows:
 \begin{itemize}
 \item\textbf{RQ1:} How do developers approach merge conflicts?\\
 \item\textbf{RQ2:} What factors impact the difficulty of a conflict resolution?\\
 \item\textbf{RQ3:} Do developer tools meet developer needs for merge conflicts?\\
 \end{itemize}
 
-In this study, we investigate these research questions with an interview of 10 developers and a survey of 226 software practitioners and found that...
+In this study, we investigate these research questions with an interview of 10 developers and a survey of 226 software practitioners and found 9 factors that developers use to assess the , a ranked list of tool needs for merge conflict resolution, and personal accounts of experiences in resolving merge conflicts.

Methodology.tex

@@ -104,5 +104,5 @@ \subsection{Survey}\label{survey_methods}
 \textit{Analysis:} We evaluated the distribution of survey answers for each Likert scale question by analyzing across demographic categories. 
 We use the mean score for each demographic to determine the positive, negative, or neutral response of the population. 
 We considered the third option of the 5-point Likert scale to be neutral, and considered mean scores in the range 2.50 to 3.50 to be equivalent to this neutral response. Mean scores of 3.50 or greater were considered on the higher end of the defined spectrum, while scores less than 2.5 were considered to be on the lower end of the spectrum. This threshold did place a large majority of our factors (76\%)into the neutral category, making our estimates conservative.
-Answer choices measured the extent to which participants agreed with a particular answer choice (i.e. \textit{Not at all} to \textit{A great deal}). Because of this, lower mean and median values indicate less agreement than higher mean and median values, rather than indicating any kind of opposition that exist in some Likert scales (i.e. Strongly disagree to Strongly agree).
+Answer choices measured the extent to which participants agreed with a particular answer choice (i.e. \textit{Not at all} to \textit{A great deal}). Because of this, lower mean and median values indicate less agreement than higher mean and median values, rather than indicating any kind of opposition that exists in some Likert scales (i.e. Strongly disagree to Strongly agree).
 \todo{Describe the process of analyzing the survey in light of how we are now coming to our results.}

RQ2.tex

@@ -1,6 +1,5 @@
 \subsection{\textbf{RQ2:} What factors impact the difficulty of a conflict resolution?}\label{RQ2}
 
-\todo{talk about resolution difficulties in interviews and how survey answer options were formed}
 \todo{double check card descriptions with actual cards}
 
 \begin{table}[!]
@@ -28,19 +27,18 @@ \subsection{\textbf{RQ2:} What factors impact the difficulty of a conflict resol
 \end{tabularx}
 \end{table}
 
-\subsubsection{General}
-
-Our interviews resulted in 10 categories in this category, as shown in Table \ref{interview_tags_rq2}.
-
-For this section in the survey, 141 participants rated how much each factor affects the difficulty of resolving merge conflicts and were given the following options:
+Our interview analysis resulted in 10 categories in this category, which are described and sorted by number of cards in Table~\ref{interview_tags_rq2}.
 
+Considering these interview categories, we created 10 factors of difficulty while resolving merge conflicts, and 141 participants rated how much each factor affects that difficulty. Participants were given the following options:
+\vbox{
 \begin{enumerate}
-	\item \textit{Not at all}
-	\item \textit{A little}
-	\item \textit{A moderate amount}
-	\item \textit{A lot}
-	\item \textit{A great deal}
-\end{enumerate}
+\item \textit{Not at all}
+\item \textit{A little}
+\item \textit{A moderate amount}
+\item \textit{A lot}
+\item \textit{A great deal}
+\end{enumerate}}
+
 The results for this section are given in Table \ref{survey_res_diffs}, which highlights three of the factors which show stronger developer opinions:
 \begin{itemize}
 \item \textit{How easy it is to understand the code involved in the merge conflict}. 
@@ -72,7 +70,7 @@ \subsubsection{General}
 \end{tabularx}
 \end{table*}
 
-No factor in this category had an average below 3.03, so there were no factors tested that were rated as having less than a moderate amount of affect on the difficulty of a merge conflict resolution.
+No factor in this category had an average below 3.03, so there were no factors tested that were rated as having less than \textit{A moderate amount} of affect on the difficulty of a merge conflict resolution on average.
 
 \subsubsection{Commit Messages}
 Based on prior work stating the value of commit messages~\cite{yamauchi2014clustering}\cite{hindle2009automatic}\cite{cortes2014automatically}\cite{hattori2008nature}, we expected commit messages to be a helpful tool in merge conflict resolution. 
@@ -98,7 +96,7 @@ \subsubsection{Code Comprehensibility}
 \begin{displayquote}
 \textit{Small is always easy. A 1-line merge conflict is always easier to resolve than a 400-line merge conflict.}
 \end{displayquote}
-However, P1 resolved a 1-line merge conflict that, after some investigation, was not as trivial as he had expected, requiring history exploration and extra care. This combination of small size and increased complexity of the conflict resulted in a more difficult conflict resolution than a simpler one-line merge conflict.
+However, P1 resolved a 1-line merge conflict that, after some investigation, was not as trivial as he had expected, requiring history exploration and extra care to ensure a correct resolution. This combination of small size and increased complexity of the conflict resulted in a more difficult conflict resolution than a simpler one-line merge conflict.
 This motivated a survey question to compare increased size to increased complexity, which showed that current tools handle increasing the size of the conflict better than increasing the complexity of the conflict. This trend was seen across all experience levels, as seen in Figure \ref{size_vs_complexity}.
 
 \begin{figure*}[!t]

RQ3.tex

@@ -1,6 +1,25 @@
+\begin{table}[!]
+\renewcommand{\arraystretch}{1.3}
+\caption{Merge Conflict Resolution Difficulty Categories from Interviews}
+\label{interview_tags_rq3}
+\centering
+\begin{tabularx}{0.5\textwidth}{@{}{c}@{ }{c}@{ }p{4.4cm}}
+\toprule
+	Category & \# Cards & \hfil Description \\
+\midrule
+
+\textit{Tool Functionality Deficiencies} & 6 & Tools not providing needed assistance\\
+\textit{Tool Information Presentation} & 6 & Tools information presentation is confusing or insufficient\\
+\textit{Tangled Toolset} & 4 & Modular tools' lack of integration with one another\\
+\textit{Tool Mistrust} & 2 & Tools do not have the trust of their users\\
+
+\bottomrule
+\end{tabularx}
+\end{table}
+
 \subsection{\textbf{RQ3:} Do developer tools meet developer needs for merge conflicts?}\label{RQ3}
 
-In the interview stage, we found four different kinds of problems that participants encountered with tools: functional deficiencies, information presentation, tool fragmentation, and tool mistrust. 
+In the interview stage, we found four different kinds of problems that participants encountered with tools: functional deficiencies, information presentation, tool fragmentation, and tool mistrust. These categories are describe and sorted by number of cards per category in Table~\ref{interview_tags_rq3}.
 
 \comment{*** Git blame issues***}
 
@@ -18,6 +37,12 @@ \subsubsection{Tool functional deficiencies}
 \end{displayquote}
 
  In other words, trying to use this tool in an area of high activity becomes very difficult in a shorter amount of time. Furthermore, it does not appear that git blame --reverse has made the jump into graphical Git tools, making this scenario more difficult for those using graphical tools.
+
+ P1 mentions a specific issue that he encounters to illustrate that tools work with a surface understanding of a merge conflict's process. The situation described is a conflict where one branch cherrypicks changes from the other, then the second branch reverts the same changes that were cherrypicked. When the two branches merge back together, tools have trouble supporting the resolution of the more complex conflict:
+ \begin{displayquote}
+ ``So cherrypick on one branch, revert on the other, equals disaster. Most tools... just do three-way branch because they limit their vision to those three parts. They don't try to understand the actual scenario.''
+ \end{displayquote}
+ Using this in light of our top difficulties from Table \ref{survey_res_diffs}, this suggests that tools would benefit from assisting developers in more complex situations by providing them with more specific information about what caused the conflict (in this case, the combination of cherrypicking and reverting). 
 
 \comment{***Information Presentation is bad***}
 
@@ -34,7 +59,7 @@ \subsubsection{Tool functional deficiencies}
 \textit{``I had to jump around between tools and copy and paste version numbers from one to... See, this is why [toolset] integration matters.''}
 \end{displayquote}
 
-This frustration is understandable for developers whose workflows frequently get interrupted by tool switches. Psychology studies have found that task switching comes with a cost~\cite{Meiran2000}\cite{gopher2000switching}, and Gerald Weinberg has discussed the problem within engineering teams~\cite{Weinberg1992}. However, when asked, \textit{``How often do you find that having multiple tools has been a problem in your development workflow?''}, 121 survey participants responded with an average response of 2.04 (\textit{Rarely}) on a 5-point Likert scale from \textit{Never} to \textit{Always}. To ensure that these developer perceptions were not biased based on the current toolset of the developer (i.e. people do not dislike multiple tools because they do not have multiple tools), we compared the tool list for people who chose \textit{Never} or \textit{Rarely} and found that the groups share 9 of their top 10 tools. This suggests that the benefits of a more modular toolset offsets the cost of switching tools with better features or user experiences in the tools in their toolchains. 
+This frustration is understandable for developers whose workflows frequently get interrupted by tool switches. Psychology studies~\cite{Meiran2000}\cite{gopher2000switching} have found that task switching comes with a cost, and Gerald Weinberg has discussed the problem within engineering teams~\cite{Weinberg1992}. However, when asked, \textit{``How often do you find that having multiple tools has been a problem in your development workflow?''}, 121 survey participants responded with an average response of 2.04 (\textit{Rarely}) on a 5-point Likert scale from \textit{Never} to \textit{Always}. To ensure that these developer perceptions were not biased based on the current toolset of the developer (i.e. people do not dislike multiple tools because they do not have multiple tools), we compared the tool list for people who chose \textit{Never} or \textit{Rarely} and found that the groups share 9 of their top 10 tools. This suggests that the benefits of a more modular toolset offsets the cost of switching tools with better features or user experiences in the tools in their toolchains. 
 
 \comment{*** Tools are not trusted ***}
 
@@ -66,7 +91,7 @@ \subsubsection{Tool functional deficiencies}
 \item How much tool trust is enough? 
 \item What is the trust threshold for not using a tool anymore?
 \end{itemize}
-If we consider this problem conservatively, nearly 1 in 10 software practitioners are using tools that they cannot trust. Since no previous work exists relating to minimum acceptable levels of trust in a toolset, it is possible that up to 42.1\% (developers with \textit{A moderate amount} of trust or less) of developers operate within this gap in tool trust.
+If we consider this problem conservatively, nearly 1 in 10 software practitioners are using tools that they cannot trust. Since no previous work exists relating to minimum acceptable levels of trust in a toolset, it is possible that up to 42.1\% of developers (those with \textit{A moderate amount} of trust or less) operate within this gap in tool trust.
 
 \todo{Do another look around for tool trust threshold papers, just in case.}
 
@@ -78,11 +103,12 @@ \subsubsection{Tool functional deficiencies}
 \textit{``Give me a way to explore the history. To drill down, to go back up, you know? To resurface and understand what happened.''}
 \end{displayquote}
 
-The reality for developers using Git for more advanced history exploration use cases is that tool support is thin. Participant 1, as someone who frequently dives deep into Git history, had resorted to writing his own scripts to better find the parts of the history that he really cares about. Participant 9 described a similar tool developed by their open-source project which fills the need for more advanced branch diffing:
+The reality for developers using Git for more advanced history exploration use cases is that tool support is thin. Participant 1, as someone who frequently dives deep into Git history, had resorted to writing his own scripts to better find the parts of the history that he really cares about. P9 described a similar tool developed by their open-source project which fills the need for more advanced branch diffing:
 \begin{displayquote}
-\textit{This is a tool that's specific to the [project], but I think other projects are using it as well. The big thing that [our diff tool] does that's different is git-diff will just do the diff based on the SHAs... but that doesn't really help us... we're adding metadata and cherry picking; the SHAs are always going to be changing. What this actually allows us to do a comparison based on SHA but then fall back to author and commit title and metadata information, so we can... essentially create a true diff... It also hooks into GitHub labels and uses the labels on the project to do some more advanced heuristics. So, you can say... `what's the diff between this branch and this branch but ignore everything that isn't SemVer \footnote{http://semver.org/} patch, ignore everything that is a maintenance commit, and give me just the SHAs, reverse that list, and then pipe that...'}
+\textit{git-diff will just do the diff based on the SHAs... we're adding metadata and cherry picking, [so] the SHAs are always going to be changing... this actually allows us to do a comparison based on SHA but then fall back to author and commit title and metadata information... It also hooks into GitHub labels and uses the labels on the project to do some more advanced heuristics.}
 \end{displayquote}
 
+P9 says that all of these changes are in an attempt to create something that will \textit{``essentially create a true diff''.}
 This suggests that current tools have not been able to keep up with the rapid expansion of metadata that Git and Github are able to provide, to the extent that projects will divert resources to create a tool for more advanced diffing systems. 
 
 \begin{table}[!]
@@ -108,25 +134,6 @@ \subsubsection{Tool functional deficiencies}
 \end{tabularx}
 \end{table}
 
-\begin{table}[!]
-\renewcommand{\arraystretch}{1.3}
-\caption{Merge Conflict Resolution Difficulty Categories from Interviews}
-\label{interview_tags_rq3}
-\centering
-\begin{tabularx}{0.5\textwidth}{@{}{c}@{ }{c}@{ }p{4.4cm}}
-\toprule
-	Category & \# Cards & \hfil Description \\
-\midrule
-
-\textit{Tool Functionality Deficiencies} & 6 & Tools not providing needed assistance\\
-\textit{Tool Information Presentation} & 6 & Tools information presentation is confusing or insufficient\\
-\textit{Tangled Toolset} & 4 & Modular tools' lack of integration with one another\\
-\textit{Tool Mistrust} & 2 & Tools do not have the trust of their users\\
-
-\bottomrule
-\end{tabularx}
-\end{table}
-
 \begin{table*}[!]
 \renewcommand{\arraystretch}{1.3}
 \caption{Tool Needs from Survey}

Results.tex

@@ -1,9 +1,4 @@
 \section{Results}\label{results}
 \input{RQ1}
 \input{RQ2}
-\input{RQ3}
-
-\subsection{General}
-Though most results and observations fit into the category of our three research questions, some interesting findings came out as a side effect of our analysis process.
-
-For instance, we found that people who split their time evenly between open- and closed-source seem behave as a distinct group rather than walking a middle line between open- and closed-source, as we might expect. This suggests that giving only two options for source distribution 
+\input{RQ3}