-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathImplications.tex
95 lines (74 loc) · 11.5 KB
/
Implications.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
\section{Implications}\label{implications}
\subsection{For Tool Builders}
Version control systems provide an easy method for storing and retrieving recent development history, but examining older development history at scale and in a usable manner has not completely met developers' expectations.
Tool builders should work to address this unmet need by leveraging research in search systems for developer-assistance~\cite{nabi2016putting} and machine learning-based code assistance~\cite{bradley2011history_exploration} to provide intuitive and expressive tools for history exploration.
Even if developers use a \emph{reactive} monitoring approach for detecting merge conflicts, better tool support can make their lives easier.
For example, instead of notifying a developer that a merge conflict has occurred, adding an annotation within tools indicating the type of conflict might assist developers.
This type of contextualized information would allow developers to more precisely know how urgent the merge conflict is, without having to interrupt their workflow.
Developers indicate that current merge toolsets do not scale to handle large, complex merge conflicts (see Section~\ref{tool_effectiveness}).
To address this concern, tool builders should look at consolidating feature sets that currently span multiple tools in order to provide better usability (I1 from Table~\ref{s2_tool_improvements}).
Tool builders should also add more expressive search and filtering features for both project history and meta-information related to merge conflicts (I2, I3), to ease the frustration of developers that must understand the context and evolution of code involved in the conflict.
Before starting a merge conflict resolution, we found developers having to ``guess-timate'' the difficulty of the conflict resolution to decide whether to work on it now or deffer it, whether to integrate the changes or simply start over.
Prediction tools that identify the complexity of conflicts and difficulty of resolution can help alleviate this.
This will provide to developers with enough information to make accurate and informed decisions in order to prevent further issues down the line.
When it comes to evaluating the result of a merge conflict resolution, we found that developers employ various strategies (Table~\ref{conditionsSuccess}).
Developers mentioned that \textit{passing tests} (C1) and \textit{successful compilation} (C2) are some of the criteria for success.
However, for large projects, running the test suite can be time intensive.
Tools can help developers by providing information or running only tests that are impacted by the conflict resolution.
Test selection techniques, such as regression testing~\cite{gligoric2015practical}, can benefit from using this conflict information to more quickly inform developers about possible bugs.
This would help developers as they would have to rely less on their own intuition (C5), when it comes to evaluating the result.
\boldif{Conflicts might be difficult, because the result is hard to evaluate}
The fact that developers mention code complexity as one of the main factors in deferring a merge conflict resolution, and that developers ``eyeball'' the resolutions, seems to be an indication that evaluation might be a problem in the resolution.
Merge conflicts are perceived as difficult because \emph{the evaluation of the results are difficult.}
In this case, tools should provide better support for developers when they evaluate their resolution.
Finally, most developers have failed at least once in resolving a merge conflict resolution (Table~\ref{first-attempt-failure}).
Tools can provide better insight into why the resolution has failed, so developers have information to formulate their next steps.
Currently, when a merge conflict resolution fails, developers have to interrupt their workflow by taking the resolution offline (B1 from Table~\ref{backup-strategies}), or they have to ask for help, which has the potential of interrupting other developers (B2).
If developers had more information about \emph{why} the merge conflict resolution failed, they might be able to recover more efficiently.
\subsection{For Developers}
%wordy....change to: understanding the constraints that the different phases involved in merge conflict resolution can allow...
Understanding the constraints of the different phases involved in merge conflict resolution can allow more effective prevention and management of merge conflicts as they occur.
Developers indicate that understanding code, having appropriate information, and dealing with complex codesets are key themes of difficulty when working with merge conflicts (Sections~\ref{RQ1}, \ref{RQ2}).
Existing tool support can help with some of these issues, but developers also need to educate themselves on development processes that prevent and alleviate the severity of merge conflicts.
For example, the number of conflicting files and the size of changes are considered important factors (Section~\ref{difficulty-factors}).
Researchers~\cite{brindescu2014versioncontrol} have previously found that developers using distributed version control systems commit more often, and that committing more often makes debugging easier when something breaks~\cite{meyer2014continuous}.
Therefore, developers should strive to make smaller commits, and commit often.
Other agile development processes such as continuous integration, iterative development, and branch merging policies are known to facilitate development in large, distributed teams.
However, not all developers are actively using such techniques~\cite{phillips2011branching}, and large organizations will require additional diligence in order to address the increased possibility of conflicts occurring.
Developers should integrate proactive merge conflict monitoring into their own processes, and seek to add tools that enable proactive monitoring within their organizations.
The most experienced developers indicate that their toolsets are least equipped to handle increases in merge conflict complexity (Table~\ref{experience_vs_effectiveness}).
If experienced developers struggle to use merge toolsets for complex merge conflicts, then less experienced developers are also likely to encounter increasing difficulties as they gain experience dealing with complex merge conflicts.
Therefore, developers should take steps to reduce the complexity of merge conflicts by using proactive merge conflict detection tools, create well-defined code ownership boundaries, and reduce the time between commits.
\subsection{For Researchers}
The life-cycle of merge conflicts is a preliminary model for understanding the phases involved in a merge conflict.
Additional research is needed to expand this model, such as to understand which strategies are most (and least) effective at addressing the needs of developers in each phase of the cycle.
Version control systems (VCS) feature heavily in the top merge awareness tools (Table~\ref{s1_toolset}), the top merge resolution tools (Table~\ref{s2_toolset}), and the top merge resolution evaluation toolsets (Table~\ref{resolution-evaluation-tools}).
Therefore, researchers should also examine the usage patterns for different features within VCS tools across the life-cycle phases to understand the evolution of developer needs with their tools.
The size of a merge conflict is often considered an approximation for complexity.
However, our findings show that developers perceive size and complexity to be different factors in merge conflict resolution and tool effectiveness (Table~\ref{experience_vs_effectiveness}).
Which suggests that researchers need to further investigate measures that can be used to better approximate merge conflict complexity.
%Our results inform future research by providing insights into software developers' perspectives during merge conflicts.
The top factors that impact the assessment of merge conflict difficulty are primarily focused on program comprehension (F1, F3, F4 from Table~\ref{s2_factors}).
Program comprehension has been an important research focus, with entire conferences dedicated to it.
Previous research has explored tool support and visualizations to help comprehend programs, both small and large.
Although similar, merge comprehension involves understanding the flow of changes across time in two parallel streams, whereas program comprehension primarily focuses on understanding the current state of code.
Therefore, tools and visualizations for merge conflicts must embrace these unique constraints in order to provide relevant information for developers.
Our results indicate that developers still have unmet needs along the following dimensions: (1) comprehending code snippets in isolation, (2) understanding the code context underlying multiple code snippets that are split across multiple files, and commits, and; (3) the ability to quickly comprehend the complexity of these code snippets.
%have a need to understand fragments of code, with some of this code split across multiples files, commits, or conflicting codebases.
%The ability to quickly evaluate the complexity of these code fragments is needed, including at the scale of text editors as evidenced by the use of basic toolsets instead of modern IDEs when working with merge conflicts (see Section~\ref{RQ3}).
Developers indicate that their needs during merge conflict resolutions center around the retrieval, organization, and presentation of relevant information (N1, N3, N4 from Table~\ref{s2_needs}).
With the variety of meta-information available across different toolsets, and the inconsistent use of terminology, there is a need for standardization and best practices to be developed.
Standardization efforts would likely help to alleviate some of the mistrust of merging tools that developers have expressed.
However, researchers should investigate the margin of errors that are tolerated by developers to determine the context in which developers discontinue use of tools.
%threshold of merge tool errors that indicates whether a user will mistrust and discontinue use of those tools.
Expertise is seen as both a significant factor that affects the assessment of merge conflict difficulty (F2), and an important need for developers to effectively resolve the conflict (N2).
We have also seen that experience can play a part in developer's assessment of the success of a merge conflict resolution (C3).
Previous work has focused on recommending developers best suited to perform a collaborative merge based on the previous edits to conflicting files~\cite{dasilva2015niche} or developers' experience across branches and project history~\cite{CostaSarma}.
However, these efforts have resulted in tools that require standalone installation and execution.
Our results indicate that developers are concerned about toolset fragmentation, and therefore adding an additional tool might be counterproductive to the workflow of most developers.
Our results show that developers mention that they \emph{redo the changes} when a merge conflict resolution fails.
This \emph{Nuclear option} is very expensive.
Tools should provide better merging support, so that developers can resolve a conflict and not have to scrap good code just because it happens to intersect with other changes.
Finally, we find that developers need to quickly estimate whether they can fix the conflict, and whether to resolve it now or delay the resolution.
This indicates that developers need mechanisms to identify the skillsets required to complete the conflict resolution task, by viewing the code fragments (D1, D2).
Research should investigate mechanisms to identify required skillsets by using information retrieval or machine learning techniques on the code fragment and past edits.