From 020efe126ae1d0c4b740095c0fa6e4d1fc0efe24 Mon Sep 17 00:00:00 2001 From: "mergify[bot]" <37929162+mergify[bot]@users.noreply.github.com> Date: Tue, 4 Mar 2025 23:01:23 +0100 Subject: [PATCH 1/3] [main](backport #42976) docs: Prepare Changelog for 8.16.5 (#43005) * docs: Prepare Changelog for 8.16.5 (#42976) * docs: Close changelog for 8.16.5 * Remove empty sections from CHANGELOG.asciidoc --------- Co-authored-by: elasticmachine Co-authored-by: Pierre HILBERT (cherry picked from commit a2167aacc6cbb34271206b94f841bcbcf1b2249e) # Conflicts: # CHANGELOG.asciidoc # libbeat/docs/release.asciidoc * Fixing conflict * Resolve merge conflict in release notes. --------- Co-authored-by: elastic-vault-github-plugin-prod[bot] <150874479+elastic-vault-github-plugin-prod[bot]@users.noreply.github.com> Co-authored-by: Pierre HILBERT (cherry picked from commit 2050421ad389460eb5eef773dc785dbf537af496) # Conflicts: # CHANGELOG.asciidoc # libbeat/docs/release.asciidoc --- CHANGELOG.asciidoc | 155 ++++++++++++++++++++++++++++++++++ CHANGELOG.next.asciidoc | 13 +++ libbeat/docs/release.asciidoc | 7 ++ 3 files changed, 175 insertions(+) diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index 9b04e75763d7..7f04d2081243 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -180,6 +180,161 @@ https://github.com/elastic/beats/compare/v8.16.1\...v8.17.0[View commits] - Implement exclusion range support for event_id. {issue}38623[38623] {pull}41639[41639] +<<<<<<< HEAD +======= +[[release-notes-8.16.5]] +=== Beats version 8.16.5 +https://github.com/elastic/beats/compare/v8.16.4\...v8.16.5[View commits] + +==== Bugfixes + +*Filebeat* + +- [Journald] Fixes handling of `journalctl` restart. A known symptom was broken multiline messages when there was a restart of journalctl while aggregating the lines. {issue}41331[41331] {pull}42595[42595] + +*Winlogbeat* + +- Sync missing changes in modules pipelines. {pull}42619[42619] + +==== Added + +*Affecting all Beats* + +- Update Go version to 1.22.12. {pull}42681[42681] + +*Filebeat* + +- Introduce ignore older and start timestamp filters for AWS S3 input. {pull}41804[41804] + +*Metricbeat* + +- Log every 401 response from Kubernetes API Server. {pull}42714[42714] + + +[[release-notes-8.16.4]] +=== Beats version 8.16.4 +https://github.com/elastic/beats/compare/v8.16.3\...v8.16.4[View commits] + +==== Bugfixes + +*Filebeat* + +- Updated websocket retry error code list to allow more scenarios to be retried which could have been missed previously. {pull}42218[42218] +- In the `streaming` input, prevent panics on shutdown with a null check and apply a consistent namespace to contextual data in debug logs. {pull}42315[42315] +- Remove erroneous status reporting to Elastic-Agent from the Filestream input. {pull}42435[42435] + +==== Added + +*Filebeat* + +- Add metrics for number of events and pages published by HTTPJSON input. {issue}42340[42340] {pull}42442[42442] + +*Metricbeat* + +- Update beat module with apm-server tail sampling monitoring metrics fields. {pull}42569[42569] + + +[[release-notes-8.16.3]] +=== Beats version 8.16.3 +https://github.com/elastic/beats/compare/v8.16.2\...v8.16.3[View commits] + +==== Breaking changes + +*Affecting all Beats* + +- Default Docker base image was reverted to Ubuntu 20.04 due to incompatability issues with glibc. {pull}42144[42144] + +==== Bugfixes + +*Filebeat* + +- Fix streaming input handling of invalid or empty websocket messages. {pull}42036[42036] +- Fix awss3 document ID construction when using the CSV decoder. {pull}42019[42019] +- Fix Netflow Template Sharing configuration handling. {pull}42080[42080] + +*Metricbeat* + +- [K8s Integration] Enhance HTTP authentication in case of token updates for Apiserver, Controllermanager and Scheduler metricsets. {issue}41910[41910] {pull}42016[42016] + +==== Added + +*Affecting all Beats* + +- When running under Elastic-Agent Kafka output allows dynamic topic in `topic` field. {pull}40415[40415] +- The script processor has a new configuration option that only uses the cached javascript sessions and prevents the creation of new javascript sessions. +- Update Go version to 1.22.10. {pull}42095[42095] +- Reduce memory consumption of k8s autodiscovery and the add_kubernetes_metadata processor when Deployment metadata is enabled +- Add `lowercase` processor. {issue}22254[22254] {pull}41424[41424] +- Add `uppercase` processor. {issue}22254[22254] {pull}41535[41535] +- Replace `compress/gzip` with https://github.com/klauspost/compress/gzip library for gzip compression. {pull}41584[41584] +- Add regex pattern matching to add_kubernetes_metadata processor. {pull}41903[41903] + +*Filebeat* + +- Added OAuth2 support with auto token refresh for websocket streaming input. {issue}41989[41989] {pull}42212[42212] +- Added infinite & blanket retry options to websockets and improved logging and retry logic. {pull}42225[42225] + +*Metricbeat* + +- Add `use_performance_counters` to collect CPU metrics using performance counters on Windows for `system/cpu` and `system/core` {pull}41965[41965] + + +[[release-notes-8.16.2]] +=== Beats version 8.16.2 +https://github.com/elastic/beats/compare/v8.16.1\...v8.16.2[View commits] + +==== Bugfixes + +*Affecting all Beats* + +- Remove unnecessary reload for Elastic Agent managed beats when apm tracing config changes from nil to nil. {pull}41794[41794] +- Fix incorrect cloud provider identification in add_cloud_metadata processor using provider priority mechanism. {pull}41636[41636] + +*Auditbeat* + +- auditd: Use ECS `event.type: end` instead of `stop` for SERVICE_STOP, DAEMON_ABORT, and DAEMON_END messages. {pull}41558[41558] +- auditd: Update syscall names for Linux 6.11. {pull}41558[41558] +- hasher: Geneneral improvements and fixes. {pull}41863[41863] + +*Filebeat* + +- Fix missing key in streaming input logging. {pull}41600[41600] +- Fix the "No such input type exist: 'salesforce'" error on the Windows/AIX platform. {pull}41664[41664] +- Add support for Access Points in the `aws-s3` input. {pull}41495[41495] +- Improve S3 object size metric calculation to support situations where Content-Length is not available. {pull}41755[41755] +- Fix handling of http_endpoint request exceeding memory limits. {issue}41764[41764] {pull}41765[41765] +- Rate limiting fixes in the Okta provider of the Entity Analytics input. {issue}40106[40106] {pull}41583[41583] +- Fixes filestream logging the error "filestream input with ID 'ID' already exists, this will lead to data duplication[...]" on Kubernetes when using autodiscover. {pull}41585[41585] + +*Metricbeat* + +- Fix incorrect handling of types in SQL module. {issue}40090[40090] {pull}41607[41607] + +==== Added + +*Affecting all Beats* + +- In this release we've introduced an image based on the hardened https://wolfi.dev/[Wolfi] image to provide secure containers to our self-managed customers, help with compliance regulations, and improve our supply chain security posture. + +*Auditbeat* + +- Split module/system/process into common and provider bits. {pull}41868[41868] + +*Filebeat* + +- Improve S3 polling mode states registry when using list prefix option. {pull}41869[41869] +- Add support for SSL and Proxy configurations for websoket type in streaming input. {pull}41934[41934] +- AWS S3 input registry cleanup for untracked s3 objects. {pull}41694[41694] +- The environment variable `BEATS_AZURE_EVENTHUB_INPUT_TRACING_ENABLED: true` enables internal logs tracer for the azure-eventhub input. {issue}41931[41931] {pull}41932[41932] +- Added default values in the streaming input for websocket retries and put a cap on retry wait time to be lesser than equal to the maximum defined wait time. {pull}42012[42012] + +*Metricbeat* + +- Add `id` field to all the vSphere metricsets. {pull}41097[41097] +- Add support for region/zone for Vertex AI service in GCP module {pull}41551[41551] + + +>>>>>>> 2050421ad ([main](backport #42976) docs: Prepare Changelog for 8.16.5 (#43005)) [[release-notes-8.16.1]] === Beats version 8.16.1 https://github.com/elastic/beats/compare/v8.16.0\...v8.16.1[View commits] diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc index 5e08fb1b2c68..79ed18080d54 100644 --- a/CHANGELOG.next.asciidoc +++ b/CHANGELOG.next.asciidoc @@ -153,6 +153,16 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff] - Add missing ECS Cloud fields in GCP `metrics` metricset when using `exclude_labels: true` {issue}40437[40437] {pull}40467[40467] - Add AWS OwningAccount support for cross account monitoring {issue}40570[40570] {pull}40691[40691] - Use namespace for GetListMetrics when exists in AWS {pull}41022[41022] +- Fix Kubernetes metadata sometimes not being present after startup {pull}41216[41216] +- Do not report non-existant 0 values for RSS metrics in docker/memory {pull}41449[41449] +- Log Cisco Meraki `getDevicePerformanceScores` errors without stopping metrics collection. {pull}41622[41622] +- Don't skip first bucket value in GCP metrics metricset for distribution type metrics {pull}41822[41822] +- [K8s Integration] Enhance HTTP authentication in case of token updates for Apiserver, Controllermanager and Scheduler metricsets {issue}41910[41910] {pull}42016[42016] +- Fixed `creation_date` scientific notation output in the `elasticsearch.index` metricset. {pull}42053[42053] +- Fix bug where metricbeat unintentionally triggers Windows ASR. {pull}42177[42177] +- Remove `hostname` field from zookeeper's `mntr` data stream. {pull}41887[41887] +- Continue collecting metrics even if the Cisco Meraki `getDeviceLicenses` operation fails. {pull}42397[42397] +- Fixed errors in the `elasticsearch.index` metricset when index settings are missing. {issue}42424[42424] {pull}42426[42426] - Fixed panic caused by uninitialized meraki device wifi0 and wifi1 struct pointers in the device WiFi data fetching. {issue}42745[42745] {pull}42746[42746] - Only fetch cluster-level index stats summary {issue}36019[36019] {pull}42901[42901] - Fixed an issue in Metricbeat's Windows module where data collection would fail if the data was unavailable. {issue}42802[42802] {pull}42803[42803] @@ -363,6 +373,9 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff] + + + diff --git a/libbeat/docs/release.asciidoc b/libbeat/docs/release.asciidoc index d7d8dee4e0eb..65b22eab2ea4 100644 --- a/libbeat/docs/release.asciidoc +++ b/libbeat/docs/release.asciidoc @@ -11,6 +11,13 @@ upgrade. * <> * <> * <> +<<<<<<< HEAD +======= +* <> +* <> +* <> +* <> +>>>>>>> 2050421ad ([main](backport #42976) docs: Prepare Changelog for 8.16.5 (#43005)) * <> * <> * <> From e3c0d1604b52a5d49641b9348cd70a26703e1c6d Mon Sep 17 00:00:00 2001 From: Pierre HILBERT Date: Tue, 4 Mar 2025 23:07:19 +0100 Subject: [PATCH 2/3] Resolve merge conflict in release notes --- libbeat/docs/release.asciidoc | 3 --- 1 file changed, 3 deletions(-) diff --git a/libbeat/docs/release.asciidoc b/libbeat/docs/release.asciidoc index 65b22eab2ea4..456b2418878f 100644 --- a/libbeat/docs/release.asciidoc +++ b/libbeat/docs/release.asciidoc @@ -11,13 +11,10 @@ upgrade. * <> * <> * <> -<<<<<<< HEAD -======= * <> * <> * <> * <> ->>>>>>> 2050421ad ([main](backport #42976) docs: Prepare Changelog for 8.16.5 (#43005)) * <> * <> * <> From cd6d14e0bcac202c17c3621340d397e4f708b5d3 Mon Sep 17 00:00:00 2001 From: Pierre HILBERT Date: Tue, 4 Mar 2025 23:08:55 +0100 Subject: [PATCH 3/3] Resolve merge conflicts and add known issues --- CHANGELOG.asciidoc | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc index 7f04d2081243..5d950c089173 100644 --- a/CHANGELOG.asciidoc +++ b/CHANGELOG.asciidoc @@ -180,8 +180,6 @@ https://github.com/elastic/beats/compare/v8.16.1\...v8.17.0[View commits] - Implement exclusion range support for event_id. {issue}38623[38623] {pull}41639[41639] -<<<<<<< HEAD -======= [[release-notes-8.16.5]] === Beats version 8.16.5 https://github.com/elastic/beats/compare/v8.16.4\...v8.16.5[View commits] @@ -334,11 +332,16 @@ https://github.com/elastic/beats/compare/v8.16.1\...v8.16.2[View commits] - Add support for region/zone for Vertex AI service in GCP module {pull}41551[41551] ->>>>>>> 2050421ad ([main](backport #42976) docs: Prepare Changelog for 8.16.5 (#43005)) [[release-notes-8.16.1]] === Beats version 8.16.1 https://github.com/elastic/beats/compare/v8.16.0\...v8.16.1[View commits] +==== Known issues + +*Metricbeat* + +- Metrics can be lost when using Metricbeat due to the total fields limit of the Metricbeat index template. We recommend increasing the `index.mapping.total_fields.limit` setting of the Metricbeat index template to 12500 and perform a rollover of the Metricbeat data stream. If you've customized the name of the index associated to Metricbeat, apply the same change accordingly. + ==== Breaking changes *Packetbeat* @@ -373,6 +376,10 @@ filebeat.inputs: bucket_arn: 'arn:aws:s3:::test1' ---- +*Metricbeat* + +- Metrics can be lost when using Metricbeat due to the total fields limit of the Metricbeat index template. We recommend increasing the `index.mapping.total_fields.limit` setting of the Metricbeat index template to 12500 and perform a rollover of the Metricbeat data stream. If you've customized the name of the index associated to Metricbeat, apply the same change accordingly. + ==== Breaking changes *Affecting all Beats*