Skip to content

Commit

Permalink
Merge/v0.42.0 (#3)
Browse files Browse the repository at this point in the history
* Detections Inheritance (panther-labs#375)

* Validation for derived detections.

* Auto-format files

* refactor

* Auto-format files

* progress

* progress

* progress

* Auto-format files

* progress

* Auto-format files

* test

* hmm

* trying something else

* Auto-format files

* progress

* progress

* Auto-format files

* progress

* Auto-format files

* prog

* progress

* Progress?

* Auto-format files

* progress

* Auto-format files

* progress

* Auto-format files

* PR feedback.

* Auto-format files

---------

Co-authored-by: panther-bot-automation <github-service-account-automation@panther.io>

* progress (panther-labs#378)

* adds option to skip version check (panther-labs#379)

* adds option to skip version check

* Auto-format files

---------

Co-authored-by: panther-bot-automation <github-service-account-automation@panther.io>

* Check that backend was set successfully (panther-labs#380)

* Better validation of backend for benchmark and validate.

* Auto-format files

* bug fix

* Auto-format files

* bug fix

* Update panther_analysis_tool/command/benchmark.py

* Update panther_analysis_tool/command/validate.py

---------

Co-authored-by: panther-bot-automation <github-service-account-automation@panther.io>

* version bump (panther-labs#381)

* bug fix for zip_chunker (panther-labs#383)

* bug fix for zip_chunker

* Update panther_analysis_tool/zip_chunker.py

* Update zip_chunker.py

* Fixing requirements for certain functions (panther-labs#385)

* Enforcing API token requirement better.

* Auto-format files

* Version bump.

---------

Co-authored-by: panther-bot-automation <github-service-account-automation@panther.io>

* bumping versions (panther-labs#386)

* version bumps (panther-labs#387)

* Allow Dependabot to update all pip package sources (panther-labs#388)

* Allow Dependabot to update all pip package sources

* Move file to .github directory

* README makeover (panther-labs#389)

* Better readme

* readme makeover

* Update README.md

* Adding github action for automating releases (panther-labs#390)

* adding github action for automating releases

* fixing steps

* fixed pr step

* adding publish_github_reelease_and_pypi job

* splitting to two gha since they both need to be manually triggered from workflow_dispatch

* Apply suggestions from code review

Co-authored-by: Evan Gibler <evan.gibler@panther.com>

* Update .github/workflows/release_pr.yml

Co-authored-by: Evan Gibler <evan.gibler@panther.com>

* refactored everything

* refactored everything

* removing unneeded code

* added major, minor, patch version levels

---------

Co-authored-by: Grant Joy <9968195+grantjoy@users.noreply.github.com>
Co-authored-by: Evan Gibler <evan.gibler@panther.com>

* Adding dist path for gha (panther-labs#391)

* Adding Draft flag to push PR (panther-labs#392)

* Adding body flag to version bump GHA (panther-labs#393)

* Adding [bot] to dac-bot for CLA (panther-labs#397)

* Switching to PAT for GH_TOKEN (panther-labs#399)

* Version bump to v0.31.0 (panther-labs#400)

* Bump version to 0.31.0

* Empty-Commit

---------

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>
Co-authored-by: stedrow <104793655+stedrow@users.noreply.github.com>

* Updating fmt GHA logic (panther-labs#401)

* Updating GHA to main from master (panther-labs#403)

* updates panther-core (panther-labs#404)

* Updating PAT with latest regexs for validation (panther-labs#406)

* Updating PAT with latest regexs for validation

* fmt

* Version bump to v0.32.0 (panther-labs#407)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>

* fixes bug that prevented multiple saved queries from being uploaded (panther-labs#408)

* Bump version to 0.32.1 (panther-labs#409)

* Allowing tests to be defined in derived detection YAML (panther-labs#410)

* Add CORRELATION_RULE AnalysisType. (panther-labs#411)

* Update logic for determining if an AnalysisType is a simple detection. (panther-labs#412)

* Update logic for determining if an AnalysisType is a simple detection.

Signed-off-by: Zac Brown <zacbrown@users.noreply.github.com>

* Add some tests.

Signed-off-by: Zac Brown <zacbrown@users.noreply.github.com>

* Format all the things.

Signed-off-by: Zac Brown <zacbrown@users.noreply.github.com>

---------

Signed-off-by: Zac Brown <zacbrown@users.noreply.github.com>

* Removing references to Azure.SignIn schema (panther-labs#413)

* Removed Azure.SignIn schema

* Removed from regex

* Version bump to v0.33.0 (panther-labs#414)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>

* Add schema support for Correlation Rules (panther-labs#416)

* adds option to auto disable base (panther-labs#417)

* adds option to auto disable base

* fix lint

* formatting

* log formatting

* fmt

* Bump version to 0.34.0 (panther-labs#418)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>

* DAC-501 PAT can test derived detections w/ inheritance and overrides (panther-labs#420)

* can fetch base detection body when testing derived detection

* make fmt

* adds unit tests for retrieving base detections

* tests can be inherited for derived detections

* make fmt

* fix lint

* ci fix

* ci fix

* ciiiii

* MORE CI

* conditionally show correlation rule output (panther-labs#419)

* conditionally show correlation rule output

* updates

* updates

---------

Co-authored-by: maxrichie5 <maxrichmond@panther.com>

* Bump version to 0.35.0 (panther-labs#424)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>

* pat test can fetch tests of base detection (panther-labs#423)

* pat test can fetch tests of base detection

* make fmt

* fix lint

* fix feature flags not checking for lambda exception (panther-labs#426)

* quick fix

* adds test for feature flags not erroring if using a lambda client backend

* fix lint

* Bump version to 0.35.1 (panther-labs#427)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>

* Bump version to 0.36.0 (panther-labs#430)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>

* CreateAlert support (panther-labs#431)

Co-authored-by: maxrichie5 <maxrichmond@panther.com>

* Added check-packs command to check whether packs have all detections (panther-labs#421)

* Added update-packs command to check whether packs have all detections it should have

* Added test for check-pack command

* Added support for simple packs

* Added docstrings

* Bump version to 0.37.0 (panther-labs#432)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>

* Revert "CreateAlert support" (panther-labs#434)

* Bump version to 0.37.1 (panther-labs#435)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>

* CreateAlert support (panther-labs#437)

* CreateAlert support

* updating packaging to have versioned deps

* updates

* regenerating lock

---------

Co-authored-by: maxrichie5 <maxrichmond@panther.com>

* Bump version to 0.38.0 (panther-labs#439)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>

* adding pdh and removing dead code from setup (panther-labs#440)

Co-authored-by: maxrichie5 <maxrichmond@panther.com>

* chore: update codeowners to DaC (panther-labs#438)

* Removing PDH because it isn't used in this project (panther-labs#441)

* fix: respect ignored files (panther-labs#442)

* Bump version to 0.38.1 (panther-labs#443)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>

* Bump aiohttp from 3.8.6 to 3.9.2 (panther-labs#436)

Bumps [aiohttp](https://github.com/aio-libs/aiohttp) from 3.8.6 to 3.9.2.
- [Release notes](https://github.com/aio-libs/aiohttp/releases)
- [Changelog](https://github.com/aio-libs/aiohttp/blob/master/CHANGES.rst)
- [Commits](aio-libs/aiohttp@v3.8.6...v3.9.2)

---
updated-dependencies:
- dependency-name: aiohttp
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Lucy Suddenly <43256356+LucySuddenly@users.noreply.github.com>

* chore: version resolution (panther-labs#444)

* chore: pin panther core version

* chore: resolve versioning

* chore: update pyyaml

* chore: align pyyaml with d-e

* fixes panther-core version. (panther-labs#447)

* fixes panther-core version.

* jump versions to see if it will resolve issues

* Bump version to 0.38.2 (panther-labs#446)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>
Co-authored-by: darwayne <darwaynelynch@gmail.com>

* Omit rules with Configuration Required Tag from Pack check (panther-labs#448)

* Revert pipfile changes; keep logic

* Appease the linter

* Bump version to 0.39.0 (panther-labs#449)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>

* adding correlation rules to async bulk upload graphql (panther-labs#450)

Co-authored-by: maxrichie5 <maxrichmond@panther.com>

* Update fmt.yml (panther-labs#452)

* Update fmt.yml (panther-labs#453)

* Update fmt.yml

* Update Pipfile

* Update fmt.yml

* Update Pipfile

* updates versions for release (panther-labs#454)

* Bump version to 0.40.0 (panther-labs#455)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>

* Bump version to 0.41.0 (panther-labs#457)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>

* Revert "Bump version to 0.41.0 (panther-labs#457)" (panther-labs#458)

This reverts commit 18d9e2a.

* format README and clear markdown linter errors (panther-labs#456)

* Ignore disabled rules during packs-check (panther-labs#459)

* Version bump to v0.41.0 (panther-labs#460)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>

* updates panther-core to 0.8.1 (panther-labs#462)

* updates panther-core to 0.8.1

* add back lock command.

* update aiohttp to 3.9.2 and pin responses version. (panther-labs#464)

* Bump version to 0.42.0 (panther-labs#465)

Co-authored-by: dac-bot[bot] <dac-bot@panther.com>

* updates panther-core to 0.8.1 (panther-labs#466)

---------

Signed-off-by: Zac Brown <zacbrown@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Grant Joy <9968195+grantjoy@users.noreply.github.com>
Co-authored-by: panther-bot-automation <github-service-account-automation@panther.io>
Co-authored-by: nskobov <93276498+nskobov@users.noreply.github.com>
Co-authored-by: Evan Gibler <evan.gibler@panther.com>
Co-authored-by: Scott Tedrow <104793655+stedrow@users.noreply.github.com>
Co-authored-by: panther-bot-automation <94577522+panther-bot-automation@users.noreply.github.com>
Co-authored-by: dac-bot[bot] <dac-bot@panther.com>
Co-authored-by: darwayne <darwaynelynch@gmail.com>
Co-authored-by: yusufak-panther <87032601+yusufak-panther@users.noreply.github.com>
Co-authored-by: Zac Brown <zacbrown@users.noreply.github.com>
Co-authored-by: Kostas Papageorgiou <kostas.papageorgiou@panther.com>
Co-authored-by: Max Richmond <46904505+maxrichie5@users.noreply.github.com>
Co-authored-by: maxrichie5 <maxrichmond@panther.com>
Co-authored-by: Oleh Melenevskyi <767472+melenevskyi@users.noreply.github.com>
Co-authored-by: Lucy Suddenly <43256356+LucySuddenly@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Panos Sakkos <panos.sakkos@panther.com>
  • Loading branch information
18 people authored Mar 18, 2024
1 parent 5ca7b34 commit 797bcfa
Show file tree
Hide file tree
Showing 9 changed files with 114 additions and 132 deletions.
12 changes: 7 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -63,11 +63,13 @@ integration: ## Run panther_analysis_tool integration tests (from included fixtu
pipenv run panther_analysis_tool test --path tests/fixtures/detections/valid_analysis
rm -rf panther-analysis
git clone https://github.com/panther-labs/panther-analysis.git
cd panther-analysis && pipenv lock
cd panther-analysis && pipenv requirements | grep -v 'panther-analysis-tool==' > requirements.ci.txt
cd panther-analysis && pipenv install -r requirements.ci.txt
cd panther-analysis && pipenv install -e ..
cd panther-analysis && pipenv run panther_analysis_tool --version && pipenv run panther_analysis_tool test --path .
cd panther-analysis;\
pipenv lock; \
pipenv requirements | grep -v 'panther-analysis-tool==' > requirements.ci.txt; \
pipenv install -r requirements.ci.txt; \
pipenv install -e ..; \
pipenv run panther_analysis_tool --version; \
pipenv run panther_analysis_tool test --path .
rm -rf panther-analysis

.PHONY: pypi
Expand Down
4 changes: 2 additions & 2 deletions Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ nose = "==1.3.7"
PyYAML = "==6.0.0"
coverage = "==6.4.2"
twine = "==4.0.2"
responses = "*"
responses = "==0.25.0"
typed-ast = { version = "==1.4.3", markers = "python_version < '3.8' and implementation_name == 'cpython'" }

[packages]
Expand Down Expand Up @@ -47,7 +47,7 @@ nested-lookup = "==0.2.25"
urllib3 = "==1.26.18"
typing-extensions = "==4.3.0"
jsonlines = "==3.0.0"
panther-core = "==0.7.3"
panther-core = "==0.8.1"

[requires]
python_version = "3.9"
60 changes: 30 additions & 30 deletions Pipfile.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 0 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,6 @@ return `False`.

```
$ panther_analysis_tool test --path tests/fixtures/valid_policies --minimum-tests 2
$ panther_analysis_tool test --path tests/fixtures/valid_policies --minimum-tests 2
% panther_analysis_tool test --path okta_rules --minimum-tests 2
[INFO]: Testing analysis packs in okta_rules
Expand Down Expand Up @@ -259,16 +258,10 @@ optional arguments:
--api-token API_TOKEN
The Panther API token to use. See: https://docs.panther.com/api-beta (default: None)
--api-host API_HOST The Panther API host to use. See: https://docs.panther.com/api-beta (default: None)
--no-confirm Skip manual confirmation of deletion (default: False)
--athena-datalake Instance DataLake is backed by Athena (default: False)
--api-token API_TOKEN
The Panther API token to use. See: https://docs.panther.com/api-beta (default: None)
--api-host API_HOST The Panther API host to use. See: https://docs.panther.com/api-beta (default: None)
--aws-profile AWS_PROFILE
The AWS profile to use when updating the AWS Panther deployment. (default: None)
--analysis-id ANALYSIS_ID [ANALYSIS_ID ...]
Space separated list of Detection IDs (default: [])
Space separated list of Detection IDs (default: [])
--query-id QUERY_ID [QUERY_ID ...]
Space separated list of Saved Queries (default: [])
```
Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.41.0
0.42.0
11 changes: 1 addition & 10 deletions panther_analysis_tool/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,9 @@
from schema import Schema

from panther_analysis_tool.schemas import (
CORRELATION_RULE_SCHEMA,
CORRELATION_RULE_SCHEMA,
DATA_MODEL_SCHEMA,
DERIVED_SCHEMA,
DERIVED_SCHEMA,
GLOBAL_SCHEMA,
LOOKUP_TABLE_SCHEMA,
PACK_SCHEMA,
Expand All @@ -21,7 +19,7 @@

PACKAGE_NAME: Final = "panther_analysis_tool"

VERSION_STRING: Final = "0.41.0"
VERSION_STRING: Final = "0.42.0"

CONFIG_FILE = ".panther_settings.yml"
DATA_MODEL_LOCATION = "./data_models"
Expand All @@ -48,11 +46,9 @@ class AnalysisTypes:
SCHEDULED_QUERY = "scheduled_query"
RULE = "rule"
DERIVED = "derived"
DERIVED = "derived"
SCHEDULED_RULE = "scheduled_rule"
SIMPLE_DETECTION = "simple_detection"
CORRELATION_RULE = "correlation_rule"
CORRELATION_RULE = "correlation_rule"


# The UserID is required by Panther for some API calls, but we have no way of
Expand Down Expand Up @@ -83,10 +79,8 @@ class AnalysisTypes:
AnalysisTypes.SCHEDULED_QUERY: SCHEDULED_QUERY_SCHEMA,
AnalysisTypes.RULE: RULE_SCHEMA,
AnalysisTypes.DERIVED: DERIVED_SCHEMA,
AnalysisTypes.DERIVED: DERIVED_SCHEMA,
AnalysisTypes.SCHEDULED_RULE: RULE_SCHEMA,
AnalysisTypes.CORRELATION_RULE: CORRELATION_RULE_SCHEMA,
AnalysisTypes.CORRELATION_RULE: CORRELATION_RULE_SCHEMA,
}

SET_FIELDS = [
Expand All @@ -111,6 +105,3 @@ class ReplayStatus:


ENABLE_CORRELATION_RULES_FLAG = "EnableCorrelationRules"


ENABLE_CORRELATION_RULES_FLAG = "EnableCorrelationRules"
42 changes: 20 additions & 22 deletions panther_analysis_tool/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,13 +85,11 @@
ClassifiedAnalysis,
ClassifiedAnalysisContainer,
disable_all_base_detections,
disable_all_base_detections,
filter_analysis,
get_simple_detections_as_python,
load_analysis_specs,
load_analysis_specs_ex,
lookup_base_detection,
lookup_base_detection,
transpile_inline_filters,
)
from panther_analysis_tool.backend.client import (
Expand All @@ -104,10 +102,6 @@
FeatureFlagsParams,
FeatureFlagWithDefault,
)
from panther_analysis_tool.backend.client import (
FeatureFlagsParams,
FeatureFlagWithDefault,
)
from panther_analysis_tool.command import (
benchmark,
bulk_delete,
Expand Down Expand Up @@ -728,22 +722,6 @@ def load_analysis(

return specs, invalid_specs

def debug_analysis(
args: argparse.Namespace, backend: typing.Optional[BackendClient] = None,
):
debug_args = {
'debug': True,
'test_name': args.testid
}
args.filter = {
'RuleID': [args.ruleid]
}
# I don't want these options to appear in the --help command, but they need
# default values for seamless integration with test_analysis
args.minimum_tests = 0
args.sort_test_results = False
return test_analysis(args, backend, debug_args=debug_args)


# pylint: disable=too-many-locals
def test_analysis(
Expand Down Expand Up @@ -858,6 +836,22 @@ def test_analysis(

return int(bool(failed_tests)), invalid_specs

def debug_analysis(
args: argparse.Namespace, backend: typing.Optional[BackendClient] = None,
):
debug_args = {
'debug': True,
'test_name': args.testid
}
args.filter = {
'RuleID': [args.ruleid]
}
# I don't want these options to appear in the --help command, but they need
# default values for seamless integration with test_analysis
args.minimum_tests = 0
args.sort_test_results = False
return test_analysis(args, backend, debug_args=debug_args)


def setup_global_helpers(global_analysis: List[ClassifiedAnalysis]) -> None:
# ensure the directory does not exist, else clear it
Expand Down Expand Up @@ -1337,6 +1331,10 @@ def check_packs(args: argparse.Namespace) -> Tuple[int, str]:
)
is_simple_pack = "Simple" in pack.analysis_spec.get("PackID", "").split(".")
for detection in detections:
# if rule is disabled (not Enabled) - no need to include it in the pack
if not detection.analysis_spec.get("Enabled", False):
continue

is_simple_rule = "Simple" in detection.analysis_spec.get("RuleID", "").split(".")
if is_simple_pack != is_simple_rule:
# simple rules should be in simple packs
Expand Down
Loading

0 comments on commit 797bcfa

Please sign in to comment.