limitation.tex

\section{Limitations and Future Work}
\label{sec.limitation}

Lind shows potential as an effective method for protecting privileged code. However, 
further investigation is needed to address the issues that were not included in our
initial study.
%We discuss the limitations of Lind in this section.
First, our metric 
%relies on identifying lines of code in the kernel 
%executed when running applications in secure systems. Currently we 
focuses on OS kernel protection, while the area of hypervisor-based
VMs is worth further investigation.
%It is not clear if
%existing hypervisors are small and compact enough to consider all the
%identified paths traversing off executing applications as common paths.
%\yanyan{what does this last sentence mean?}
%or whether there is still substantial benefit from our metric.
%In %our
%future work we will %explore
%investigate if the bug metric works well for %operating system virtual machines with a hypervisor.
%hypervisor-based VMs.

A second limitation stems from our criteria for determining if a kernel trace is
safe or risky. In our study, the criteria involved checking the trace against a list
of historical kernel bugs.
 %However, not all bugs can be accurately checked in this manner.
Due to the nature of bugs, our proposed metric cannot assure whether a portion of code
is an absolute safe or risky area in kernel.
%So we are just trying to reduce the chances of triggering underlying kernel bugs, but can 
%not promise to avoid all bugs.
For example, bugs that are caused
by a race condition cannot be identified by directly checking if certain lines of
code have been executed. For complicated bugs that involve defects in the internal
kernel data structures, or require complex triggering conditions across multiple
kernel paths, our metric will not be accurate. 
%determine whether or not those bugs have been triggered. 
In these cases, more complex metrics might be needed.

%While Lind executes programs like Apache and Tor, it does not
%support every system call or every possible set of arguments. For example,
%symbolic links are not supported.  While the SafePOSIX implementation could
%be extended to do so, we leave this for future work. Also, as mentioned in
%the previous section,


%There were also some avenues we intentionally excluded in this initial
%study that could form the basis for interesting research projects in the
%future. First, we chose not to explore bugs within the applications
%themselves.
%\cappos{move to motivation / threat model.}

Lind has not been optimized for performance. 
%so the results we present here should be taken as a baseline.
%of what is possible.
We would like to explore what existing OS VM optimizations can be safely applied
and their impact.

We would like to test our metric in other operating systems, such as Windows and Mac OS.
Our experiments were limited to Linux kernel 3.14.1 and some of the typical virtualization systems that existed in Linux.
It would be interesting
%and beneficial to the advance of secure systems
if similar tests could be run in other widely-used operating systems. 
%In particular, it would be interesting to see if
%having the host and guest VM run different operating systems would produce different security impacts.