Problem
Root Cause
Cannot view email. Client returns error message
Wrong username configured
“login to server failed.”
in the email client.
Cannot connect to the incoming email server. Client returns
Wrong incoming server name
error “Failed to connect to server .”
configured in the client.
Cannot receive emails. The client returns error
Wrong authentication
“The IMAP server does not support the
mode configured
selected authentication method. Please change the ‘Authentication
with the incoming
method’ in the ‘Account Settings | Server Settings’.”
server in the client.
Cannot receive emails. Client returns error “Could not connect
Wrong incoming server
to the server ; the connection was refused.”
port number configured.
Cannot send emails. The client returns error “Sending of message
Wrong authentication
failed. The SMTP server does not support the
mode configured with
selected authentication method. Please change the ‘Authentication
the outgoing server
method’ in the ‘Account Settings | Outgoing Server (SMTP)’.”
in the email client.
Cannot send emails. Client returns “Sending of message failed.
An error occurred sending mail: SMTP server is
Wrong outgoing server
unknown. The server may be incorrectly configured. Please verify
name configured in
that your SMTP server settings are correct and try again.”
the email client.
Cannot send emails. Client returns “Sending of message failed.
The message could not be sent because connecting to SMTP
Wrong outgoing server
server failed. The server may be unavailable
port number configured
or is refusing connections. Please verify that your server
in the email client.
settings are correct and try again, or contact the administrator.”
Sending a message while Thunderbird is in “Offline Mode” puts
User has forgotten that
the message in “outgoing” folder rather than sending it.
“offline mode” is enabled.
Table 3:
Email trace details. We collected 15 traces for each problem (5 per OS).
learn signatures almost as effectively as a classifier that
has access to fine-grained labels corresponding to the
root causes.
5.2.1
Signature comparison
We now subjectively compare the Deja vu signatures
to the classifier signatures.
1. Sometimes, Deja vu signatures provide more infor-
mation than classifier signatures. All three signatures
– D3, D4, and D5 in Figure 3 – capture the fact that
there is no successful Netbios response in the traces
(NBTNSRS = 0), whereas C2 does not. In fact, the
Deja vu signatures capture the actual root cause, be-
cause it is only when both Netbios and DNS fail, that
name resolution fails in the corporate network.
The
classifier signature does not capture this because cap-
turing only the DNS failure is enough to differentiate
this failure category from other failure categories. This
example shows the benefits of using a learning approach
like Deja vu that compares BAD traces with all the
GOOD traces at every step.
Another example is signature D12 in Figure 3. We
investigated why there is no corresponding signature
with the classifier, and found that the D12 signature
specifically captures behavior of the Firefox and Opera
browsers, which behave differently under the “Wrong
URL” root cause compared to other browsers. The clas-
sifier does not capture this behavior because its input
labels are only at the granularity of the root cause, and
does not distinguish between browsers
1
.
1
We also tried inputting more fine-grained labels of the form
OS:Browser:RootCause to the classifier. This gave us very
noisy signatures since the classifier was forced to choose spu-
rious features to differentiate between similar traces from
different browsers and OSes.
2. Sometimes, Deja vu signatures did not differentiate
root causes but the classifier signatures did. Figure 2
shows that Deja vu did not pick out the right feature
differentiating root causes “Wrong Outgoing Port But
Correct Mail Server” (which would involve a successful
DNS resolution of the mail server name) and “Wrong
Outgoing Mail Server” (which would not include a suc-
cessful DNS resolution because the wrong mail server
name does not correspond to any real host). The rea-
son Deja vu did not pick this feature is that in 6 of
the 15 BAD traces for the former root cause there was
background DNS traffic that was failing, which confused
Deja vu into believing that such DNS lookup failure was
distributed across both of the above root causes, and
therefore not useful for separating them. In fact, this
confused the classifier too, which attributed the signa-
ture for “Wrong Outgoing Mail Server” to the former
root cause also. Removal of such background noise can
aid Deja vu’s learning significantly, as noted in Sec-
tion 7.
3. Deja vu signatures are in general longer than the
classifier signatures.
For example, in Figure 2, sig-
nature D1 (the Deja vu signature for the “wrong in-
coming mail server address” root cause) is 6 features
long, whereas the corresponding classifier signature C1
is only 3 features long. The reason for the longer sig-
natures is that, at each iteration, Deja vu can choose
a feature only to differentiate BAD traces from GOOD
ones, whereas the classifier has the added freedom of
choosing a feature that directly differentiates between
different kinds of BAD traces. On the flip side, however,
the Deja vu signatures provide more insight into the
failure. For example, signature D1 from Deja vu tells
us that there was no successful TCP handshake on port