Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes to pipeline.state within backend is not considered for postprocessing #311

Open
Mat0vu opened this issue Dec 6, 2024 · 0 comments

Comments

@Mat0vu
Copy link
Contributor

Mat0vu commented Dec 6, 2024

Hi,

When experimenting with the set_state transformation and a postprocessing pipeline for EQL-ndjson I discovered some issue with the handling of the pipeline state variable:
I have a set_state transformation to assign indizes in which the search shall be executed for different log sources like this:

    key: index
    val: "*winlog*"
    rule_conditions:
      - type: logsource
        product: windows

Because Elasticsearch demands a list of indizes for the import of EQL Detection Rules, I wanted to ensure that the value set with the set_state transformation is a list. Therefore I added this into the finalize_query function of the EQL Backend (SigmaHQ/pySigma-backend-elasticsearch#104).

        index_state = state.processing_state.get("index")
        if not index_state:
            index_state = self.index_names
        if not isinstance(index_state, list):
            index_state = [index_state]
        # Save the processed index back to the processing state
        state.processing_state["index"] = index_state
        return super().finalize_query(rule, query, index, state, output_format)

However, then I discovered that even though passing the Indizes as a list in the state variable, in the base.py class when self.last_processing_pipeline.postprocess_query(rule, backend_query) is called, self.last_processing_pipeline.state is still containing {"index": "*winlog*"} (probably just representing the yaml definition of the pipeline). The problem is that this is then also used by the jinja template and not the adjusted state which contains the corrected value (see screenshot)

image

Obviously this could be easily avoided by specifying the val values of the transformation as a list in this context but I expected that by adjusting the pipeline state variable, this would also reflect in the result of the postprocessing.
What do you think about this, should this be changed or is this the desired behaviour?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant