Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Security Solution] [AI Insights] AI Insights (elastic#180611)
## [Security Solution] [AI Insights] AI Insights ### Summary This PR introduces _AI Insights_ to the Security Solution:  _Above: AI Insights in the Security Solution_ AI Insights identify active attacks in the environment, without the time (or prior experience) required to manually investigate individual alerts in Elastic Security, identify if they are related, and document the identified attack progression. While users can ask the Assistant to find these progressions today, AI Insights is a dedicated UI to identify these progressions and action them accordingly. This feature adds a new page, `AI Insights`, to the Security Solution's global navigation. AI Insights are generated from Large Language Models (LLMs) to identify attack progressions in alert data, and to correlate and identify related entities and events. When possible, attack progressions are attributed to threat actors. ### Details Users may generate insights from a varetiy of LLMs, configured via [Connectors](https://www.elastic.co/guide/en/kibana/master/action-types.html):  _Above: LLM selection via the connectors popup menu_ Clicking on the title of an insight toggles the insight between the collapsed and expanded state:  _Above: Collapsing / expanding an insight (animated gif)_ The first three insights displayed on the AI Insights page are expanded by default. Any additional insights that appear after the first three must be expanded manually. Insights provide a summary of the entities impacted by an attack. Clicking on an entity, i.e. a hostname or username, displays the entity flyout with the entity's risk summary:  _Above: Clicking on a host in the summary of the insight reveals the host risk summary (animated gif)_ Hover over fields in the insight's summary or details to reveal pivot actions for investigations:  _Above: Hovering over fields in the details of an insight reveals pivot actions (animated gif)_ Insights are generated from alerts provided as context to the selected LLM. The alert data provided to the LLM is anonymized automatically. Anonymization is [configured](https://www.elastic.co/guide/en/security/current/security-assistant.html#ai-assistant-anonymization) via the same anonymization settings as the Assistant. Users may override the defaults to allow or deny specific alert fields, and to toggle anonymization on or off for specific fields. Click the Anonymization toggle to show or hide the actual values sent to the LLM:  _Above: Toggling anonymization to reveal the actual values sent to the LLM (animated gif)_ ### Empty prompt At the start of a session, or when a user selects a connector that doesn't (yet) have any insights, an [empty prompt](https://eui.elastic.co/#/display/empty-prompt) is displayed. The animated counter in the empty prompt counts up until it displays the maximum number of alerts that will be sent to the LLM:  _Above: An animated counter displays the maximum number of alerts that will be sent to the LLM (animiated gif)_ The _Settings_ section of this PR details how users configure the number of alerts sent to the LLM. The animated counter in the empty prompt immediately re-animates to the newly-selected number when the setting is updated. ### Take action workflows The _Take action_ popover displays the following actions: - `Add to new case` - `Add to existing case` - `View in AI Assistant`  _Above: The Take action popover_ #### Add to new case Clicking the `Add to new` case action displays the `Create case` flyout.  _Above: The `Add to new case` workflow_ An `Alerts were added to <case name>` toast is displayed when the case is created:  _Above: Case creation toast_ A markdown representation of the insight is added to the case:  _Above: A markdown representation of an insight in a case_ The alerts correlated to generate the insight are attached to the case:  _Above: Insight alerts attached to a case_ #### Add to existing case Clicking the `Add to existing case` action displays the `Select case` popover.  _Above: The `Select case` popover_ When users select an existing case, a markdown representation of the insight, and the alerts correlated to generate the insight are attached to the case, as described above in the _Add to new case_ section. #### View in AI Assistant The `View in AI Assistant` action in the `Take action` popover, and two additional `View in AI Assistant` affordances that appear in each insight have the same behavior: Clicking `View in AI Assistant` opens the assistant and adds the insight as context to the current conversation.  _Above: An insight added as context to the current conversation_ Clicking on the insight in the assistant expands it to reveal a preview of the insight.  _Above: An expanded insight preview in the assistant_ The expanded insight preview reveals the number of anonymzied fields from the insight that were made available to the conversation. This feature ensures insights are added to a conversation with the anonymized field values. An insight viewed in the AI assistant doesn't become part of the conversation until the user submits it by asking a question, e.g. `How do I remediate this?`. Insights provided as context to a conversation are formatted as markdown when sent to the LLM:  _Above: Insights provided as context to a conversation are formatted as markdown_ Users may toggle anonymization in the conversation to reveal the original field values.  _Above: Revealing the original field values of an insight added as markdown to a conversation (animated gif)_ #### Alerts tab The _Alerts_ tab displays the alerts correlated to generate the insight.  _Above: The alerts correlated to generate the insight in the Alerts tab_ The `View details`, `Investigate in timeline`, and overflow row-level alert actions displayed in the Alerts tab are the same actions available on the Cases's page's Alerts tab:  _Above: Row-level actions are the same as the Cases pages Alert's tab_ #### Investigate in Timeline Click an insight's `Investigate in Timeline` button to begin an investigation of an insights's alerts in Timeline. Alert IDs are queried via the `Alert Ids` filter:  _Above: Clicking Investigte in Timeline (animated gif)_ The alerts from the insight are explained via row renderers in Timeline:  _Above: Row rendered insight alerts in Timeline_ ### Attack Chain When alerts are indicative of attack [tactics](https://attack.mitre.org/tactics/enterprise/), those tactics are displayed in the insights's _Attack Chain_ section:  _Above: An insight with tactics in the Attach chain_ The Attack Chain section will be hidden if an insight is not indicative of specific tactics. ### Mini attack chain Every insight includes a mini attack chain that visually summarizes the tactics in an insight. Hovering over the mini attack chain reveals a tooltip with the details:  _Above: The mini attack chain tooltip_ ### Storage The latest insights generated for each connector are cached in the browser's session storage in the following key: ``` elasticAssistantDefault.aiInsights.cachedInsights ``` Caching insights in session storage makes it possible to immediately display the latest when users return to to the AI insights page from other pages in the security solution (e.g. Cases).  _Above: Cached insights from sesion storage are immediately displayed when users navigate back to AI Insights (animated gif)_ While waiting for a connector to generate results, users may view the cached results from other connectors. Cached insights are immediately available, even after a full page refresh, as long as the browser session is still active. ### `Approximate time remaining` / `Above average time` counters Some LLMs may take seconds, or even minutes to generate insights. To help users anticipate the time it might take to generate an insight, the AI insights feature displays a `Approximate time remaining: mm:ss` countdown timer that counts down to zero from the average time it takes to generate an insight for the selected LLM:  _Above: The `Approximate time remaining: mm:ss` countdown counter (animated gif)_ If the LLM doesn't generate insights before the counter reaches zero, the text will change from `Approximate time remaining: mm:ss` to `Above average time: mm:ss`, and start counting up from `00:00` until the insights are generated:  _Above: The `Above average time: mm:ss` counter (animated gif)_ The first time insights are generated for a model, the `Approximate time remaining: mm:ss` counter is not displayed. Average time is calculated over the last 5 generations on the selected connector. This is illustrated by clicking on the (?) information icon next to the timer. The popover displays the average time, and the time in seconds for the last 5 runs:  _Above: Clicking on the (?) information icon displays the average time, and the duration / datetimes for the last 5 generations_ The time and duration of the last 5 generations (for each connector) are persisted in the browser's local storage in the following key: ``` elasticAssistantDefault.aiInsights.generationIntervals ``` ### Errors When insight generation fails, an error toaster is displayed to explain the failure:  _Above: An error toaster explains why insights generation failed_ ### Feature flag The `assistantAlertsInsights` feature flag must be enabled to view the `AI Insights` link in the Security Solution's global navigation. Add the `assistantAlertsInsights` feature flag to the `xpack.securitySolution.enableExperimental` setting in `config/kibana.yml` (or `config/kibana.dev.yml` in local development environments), per the example below: ``` xpack.securitySolution.enableExperimental: ['assistantAlertsInsights'] ``` ### Settings The number of alerts sent as context to the LLM is configured by `Knowledge Base` > `Alerts` slider in the screenshot below:  - The slider has a range of `10` - `100` alerts (default: `20`) Up to `n` alerts (as determined by the slider) that meet the following criteria will be returned: - The `kibana.alert.workflow_status` must be `open` - The alert must have been generated in the last `24 hours` - The alert must NOT be a `kibana.alert.building_block_type` alert - The `n` alerts are ordered by `kibana.alert.risk_score`, to prioritize the riskiest alerts ### License An Enterprise license is required to use AI Insights. The following AI Insights view is displayed for users who don't have an Enterprise license:  ## How it works - Users navigate to the AI insights page: `x-pack/plugins/security_solution/public/ai_insights/pages/index.tsx` - When users click the `Generate` button(s) on the AI Insights page, insights are fetched via the `useInsights` hook in `x-pack/plugins/security_solution/public/ai_insights/use_insights/index.tsx`. - The `fetchInsights` function makes an http `POST` request is made to the `/internal/elastic_assistant/insights/alerts` route. include the following new (optional) parameters: - `actionTypeId`, determines tempature and other connector-specific request parameters - `alertsIndexPattern`, the alerts index for the current Kibana Space, e.g. `.alerts-security.alerts-default` - `allow`, the user's `Allowed` fields in the `Anonymization` settings, e.g. `["@timestamp", "cloud.availability_zone", "file.name", "user.name", ...]` - `allowReplacement`, the user's `Anonymized` fields in the `Anonymization` settings, e.g. `["cloud.availability_zone", "host.name", "user.name", ...]` - `connectorId`, id of the connector to generate the insights - `replacements`, an optional `Record<string, string>` collection of replacements that always empty in the current implementation. When non-empty, this collection enables new insights to be generated using existing replacements. ```json "replacements": { "e4f935c0-5a80-47b2-ac7f-816610790364": "Host-itk8qh4tjm", "cf61f946-d643-4b15-899f-6ffe3fd36097": "rpwmjvuuia", "7f80b092-fb1a-48a2-a634-3abc61b32157": "6astve9g6s", "f979c0d5-db1b-4506-b425-500821d00813": "Host-odqbow6tmc", // ... }, ``` - `size`, the maximum number of alerts to generate insights from. This numeric value is set by the slider in the user's `Knowledge Base > Alerts` setting, e.g. `20` - The `postAlertsInsightsRoute` function in `x-pack/plugins/elastic_assistant/server/routes/insights/alerts/post_alerts_insights.ts` handles the request. - The inputs and outputs to this route are defined by the [OpenAPI](https://spec.openapis.org/oas/v3.1.0) schema in `x-pack/packages/kbn-elastic-assistant-common/impl/schemas/insights/alerts/post_alerts_insights_route.schema.yaml`. ``` node scripts/generate_openapi --rootDir ./x-pack/packages/kbn-elastic-assistant-common ``` - The `postAlertsInsightsRoute` route handler function in `x-pack/plugins/elastic_assistant/server/routes/insights/alerts/post_alerts_insights.ts` invokes the `insights-tool`, defined in `x-pack/plugins/security_solution/server/assistant/tools/insights/insights_tool.ts`. The `insights-tool` is registered by the Security Solution. Note: The `insights-tool` is only used for generating insights. It is not used to generate new insights from the context of an assistant conversation, but that feature could be enabled in a future release. - The `insights-tool` uses a LangChain `OutputFixingParser` to create a [prompt sandwich](https://www.elastic.co/blog/crafting-prompt-sandwiches-generative-ai) with the following parts: ``` _________________________________________________ / \ | Insight JSON formatting instructions | (1) \ _______________________________________________/ +------------------------------------------------+ | Insights prompt | (2) +------------------------------------------------+ / \ | Anonymized Alerts | (3) \_______________________________________________/ ``` - The `Insight JSON formatting instructions` in section `(1)` of the prompt sandwich are defined in the `getOutputParser()` function in `x-pack/plugins/security_solution/server/assistant/tools/insights/get_output_parser.ts`. This function creates a LangChain `StructuredOutputParser` from a Zod schema. This parser validates responses from the LLM to ensure they are formatted as JSON representing an insight. - The `Insights prompt` in section `(2)` of the prompt sandwich is defined in the `getInsightsPrompt()` function in `x-pack/plugins/security_solution/server/assistant/tools/insights/get_insights_prompt.ts`. This part of the prompt sandwich includes instructions for correlating insights, and additional instructions to the LLM for formatting JSON. - The `Anonymized Alerts` in section `(3)` of the prompt sandwich are returned by the `getAnonymizedAlerts()` function in `x-pack/plugins/security_solution/server/assistant/tools/insights/get_anonymized_alerts.ts`. The allow lists configured by the user determine which alert fields will be included and anonymized. - The `postAlertsInsightsRoute` route handler returns the insights generated by the `insights-tool` to the client (browser). - Insights are rendered in the browser via the `Insight` component in `x-pack/plugins/security_solution/public/ai_insights/insight/index.tsx` - The `AiInsights` tab in `x-pack/plugins/security_solution/public/ai_insights/insight/tabs/ai_insights/index.tsx` includes the _Summary_ and _Details_ section of the Insight. - The `InsightMarkdownFormatter` in `x-pack/plugins/security_solution/public/ai_insights/insight_markdown_formatter/index.tsx` renders hover actions on entities (like hostnames and usernames) and other fields in the insight. - The `Insight` component makes use of the `useAssistantOverlay` hook in `x-pack/packages/kbn-elastic-assistant/impl/assistant/use_assistant_overlay/index.tsx` to register the insight as context with the assistant. This registration process makes it possible to view insights in the assistant, and ask questions like "How do I remediate this?". In this PR, the `useAssistantOverlay` hook was enhanced to accept anonymizaton replacements. This enables an assistant conversation to (re)use replacements originally generated for an insight.
- Loading branch information
1 parent
c4e40ea
commit 32f43bf