Production - [Alerting] Apple simulator failure rate alert

[dotnet-eng-status]

💔 Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=dci-mac-build-132} 100

Go to rule

@dotnet/dnceng, @dotnet/prodconsvcs, please investigate

Automation information below, do not change

Grafana-Automated-Alert-Id-36d07fceeaf0472b804d8358b2198eac

[dotnet-eng-status]

💚 Metric state changed to ok

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

Go to rule

[dotnet-eng-status]

💔 Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=dci-mac-build-143} 85

Go to rule

[dotnet-eng-status]

💚 Metric state changed to ok

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

Go to rule

[meghnave]

Was using this to understand more about the alert and if we could improve upon it - was missing the 12h summary in the kql and was thinking we could fix it so that we don't get the alert and then almost soon after, an 'ok'. But maybe that's inevitable considering we are bin-ing into 12h chunks (will find out offline why) but we are definitely not missing data so closing this.