[8.17](backport #43005) [main](backport #42976) docs: Prepare Changel…

…og for 8.16.5 (#43023)

* [main](backport #42976) docs: Prepare Changelog for 8.16.5 (#43005)

* docs: Prepare Changelog for 8.16.5 (#42976)

* docs: Close changelog for 8.16.5

* Remove empty sections from CHANGELOG.asciidoc

---------

Co-authored-by: elasticmachine <elasticmachine@elastic.co>
Co-authored-by: Pierre HILBERT <pierre.hilbert@elastic.co>
(cherry picked from commit a2167aa)

# Conflicts:
#	CHANGELOG.asciidoc
#	libbeat/docs/release.asciidoc

* Fixing conflict

* Resolve merge conflict in release notes.

---------

Co-authored-by: elastic-vault-github-plugin-prod[bot] <150874479+elastic-vault-github-plugin-prod[bot]@users.noreply.github.com>
Co-authored-by: Pierre HILBERT <pierre.hilbert@elastic.co>
(cherry picked from commit 2050421)

# Conflicts:
#	CHANGELOG.asciidoc
#	libbeat/docs/release.asciidoc

* Resolve merge conflict in release notes

* Resolve merge conflicts and add known issues

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Pierre HILBERT <pierre.hilbert@elastic.co>

CHANGELOG.asciidoc

@@ -180,10 +180,168 @@ https://github.com/elastic/beats/compare/v8.16.1\...v8.17.0[View commits]
 - Implement exclusion range support for event_id. {issue}38623[38623] {pull}41639[41639]
 
 
+[[release-notes-8.16.5]]
+=== Beats version 8.16.5
+https://github.com/elastic/beats/compare/v8.16.4\...v8.16.5[View commits]
+
+==== Bugfixes
+
+*Filebeat*
+
+- [Journald] Fixes handling of `journalctl` restart. A known symptom was broken multiline messages when there was a restart of journalctl while aggregating the lines. {issue}41331[41331] {pull}42595[42595]
+
+*Winlogbeat*
+
+- Sync missing changes in modules pipelines. {pull}42619[42619]
+
+==== Added
+
+*Affecting all Beats*
+
+- Update Go version to 1.22.12. {pull}42681[42681]
+
+*Filebeat*
+
+- Introduce ignore older and start timestamp filters for AWS S3 input. {pull}41804[41804]
+
+*Metricbeat*
+
+- Log every 401 response from Kubernetes API Server. {pull}42714[42714]
+
+
+[[release-notes-8.16.4]]
+=== Beats version 8.16.4
+https://github.com/elastic/beats/compare/v8.16.3\...v8.16.4[View commits]
+
+==== Bugfixes
+
+*Filebeat*
+
+- Updated websocket retry error code list to allow more scenarios to be retried which could have been missed previously. {pull}42218[42218]
+- In the `streaming` input, prevent panics on shutdown with a null check and apply a consistent namespace to contextual data in debug logs. {pull}42315[42315]
+- Remove erroneous status reporting to Elastic-Agent from the Filestream input. {pull}42435[42435]
+
+==== Added
+
+*Filebeat*
+
+- Add metrics for number of events and pages published by HTTPJSON input. {issue}42340[42340] {pull}42442[42442]
+
+*Metricbeat*
+
+- Update beat module with apm-server tail sampling monitoring metrics fields. {pull}42569[42569]
+
+
+[[release-notes-8.16.3]]
+=== Beats version 8.16.3
+https://github.com/elastic/beats/compare/v8.16.2\...v8.16.3[View commits]
+
+==== Breaking changes
+
+*Affecting all Beats*
+
+- Default Docker base image was reverted to Ubuntu 20.04 due to incompatability issues with glibc. {pull}42144[42144]
+
+==== Bugfixes
+
+*Filebeat*
+
+- Fix streaming input handling of invalid or empty websocket messages. {pull}42036[42036]
+- Fix awss3 document ID construction when using the CSV decoder. {pull}42019[42019]
+- Fix Netflow Template Sharing configuration handling. {pull}42080[42080]
+
+*Metricbeat*
+
+- [K8s Integration] Enhance HTTP authentication in case of token updates for Apiserver, Controllermanager and Scheduler metricsets. {issue}41910[41910] {pull}42016[42016]
+
+==== Added
+
+*Affecting all Beats*
+
+- When running under Elastic-Agent Kafka output allows dynamic topic in `topic` field. {pull}40415[40415]
+- The script processor has a new configuration option that only uses the cached javascript sessions and prevents the creation of new javascript sessions.
+- Update Go version to 1.22.10. {pull}42095[42095]
+- Reduce memory consumption of k8s autodiscovery and the add_kubernetes_metadata processor when Deployment metadata is enabled
+- Add `lowercase` processor. {issue}22254[22254] {pull}41424[41424]
+- Add `uppercase` processor. {issue}22254[22254] {pull}41535[41535]
+- Replace `compress/gzip` with https://github.com/klauspost/compress/gzip library for gzip compression. {pull}41584[41584]
+- Add regex pattern matching to add_kubernetes_metadata processor. {pull}41903[41903]
+
+*Filebeat*
+
+- Added OAuth2 support with auto token refresh for websocket streaming input. {issue}41989[41989] {pull}42212[42212]
+- Added infinite & blanket retry options to websockets and improved logging and retry logic. {pull}42225[42225]
+
+*Metricbeat*
+
+- Add `use_performance_counters` to collect CPU metrics using performance counters on Windows for `system/cpu` and `system/core` {pull}41965[41965]
+
+
+[[release-notes-8.16.2]]
+=== Beats version 8.16.2
+https://github.com/elastic/beats/compare/v8.16.1\...v8.16.2[View commits]
+
+==== Bugfixes
+
+*Affecting all Beats*
+
+- Remove unnecessary reload for Elastic Agent managed beats when apm tracing config changes from nil to nil. {pull}41794[41794]
+- Fix incorrect cloud provider identification in add_cloud_metadata processor using provider priority mechanism. {pull}41636[41636]
+
+*Auditbeat*
+
+- auditd: Use ECS `event.type: end` instead of `stop` for SERVICE_STOP, DAEMON_ABORT, and DAEMON_END messages. {pull}41558[41558]
+- auditd: Update syscall names for Linux 6.11. {pull}41558[41558]
+- hasher: Geneneral improvements and fixes. {pull}41863[41863]
+
+*Filebeat*
+
+- Fix missing key in streaming input logging. {pull}41600[41600]
+- Fix the "No such input type exist: 'salesforce'" error on the Windows/AIX platform. {pull}41664[41664]
+- Add support for Access Points in the `aws-s3` input. {pull}41495[41495]
+- Improve S3 object size metric calculation to support situations where Content-Length is not available. {pull}41755[41755]
+- Fix handling of http_endpoint request exceeding memory limits. {issue}41764[41764] {pull}41765[41765]
+- Rate limiting fixes in the Okta provider of the Entity Analytics input. {issue}40106[40106] {pull}41583[41583]
+- Fixes filestream logging the error "filestream input with ID 'ID' already exists, this will lead to data duplication[...]" on Kubernetes when using autodiscover. {pull}41585[41585]
+
+*Metricbeat*
+
+- Fix incorrect handling of types in SQL module. {issue}40090[40090] {pull}41607[41607]
+
+==== Added
+
+*Affecting all Beats*
+
+- In this release we've introduced an image based on the hardened https://wolfi.dev/[Wolfi] image to provide secure containers to our self-managed customers, help with compliance regulations, and improve our supply chain security posture.
+
+*Auditbeat*
+
+- Split module/system/process into common and provider bits. {pull}41868[41868]
+
+*Filebeat*
+
+- Improve S3 polling mode states registry when using list prefix option. {pull}41869[41869]
+- Add support for SSL and Proxy configurations for websoket type in streaming input. {pull}41934[41934]
+- AWS S3 input registry cleanup for untracked s3 objects. {pull}41694[41694]
+- The environment variable `BEATS_AZURE_EVENTHUB_INPUT_TRACING_ENABLED: true` enables internal logs tracer for the azure-eventhub input. {issue}41931[41931] {pull}41932[41932]
+- Added default values in the streaming input for websocket retries and put a cap on retry wait time to be lesser than equal to the maximum defined wait time. {pull}42012[42012]
+
+*Metricbeat*
+
+- Add `id` field to all the vSphere metricsets. {pull}41097[41097]
+- Add support for region/zone for Vertex AI service in GCP module {pull}41551[41551]
+
+
 [[release-notes-8.16.1]]
 === Beats version 8.16.1
 https://github.com/elastic/beats/compare/v8.16.0\...v8.16.1[View commits]
 
+==== Known issues
+
+*Metricbeat*
+
+- Metrics can be lost when using Metricbeat due to the total fields limit of the Metricbeat index template. We recommend increasing the `index.mapping.total_fields.limit` setting of the Metricbeat index template to 12500 and perform a rollover of the Metricbeat data stream. If you've customized the name of the index associated to Metricbeat, apply the same change accordingly.
+
 ==== Breaking changes
 
 *Packetbeat*
@@ -218,6 +376,10 @@ filebeat.inputs:
   bucket_arn: 'arn:aws:s3:::test1'
 ----
 
+*Metricbeat*
+
+- Metrics can be lost when using Metricbeat due to the total fields limit of the Metricbeat index template. We recommend increasing the `index.mapping.total_fields.limit` setting of the Metricbeat index template to 12500 and perform a rollover of the Metricbeat data stream. If you've customized the name of the index associated to Metricbeat, apply the same change accordingly.
+
 ==== Breaking changes
 
 *Affecting all Beats*

CHANGELOG.next.asciidoc

@@ -154,6 +154,16 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
 - Add missing ECS Cloud fields in GCP `metrics` metricset when using `exclude_labels: true` {issue}40437[40437] {pull}40467[40467]
 - Add AWS OwningAccount support for cross account monitoring {issue}40570[40570] {pull}40691[40691]
 - Use namespace for GetListMetrics when exists in AWS {pull}41022[41022]
+- Fix Kubernetes metadata sometimes not being present after startup {pull}41216[41216]
+- Do not report non-existant 0 values for RSS metrics in docker/memory {pull}41449[41449]
+- Log Cisco Meraki `getDevicePerformanceScores` errors without stopping metrics collection. {pull}41622[41622]
+- Don't skip first bucket value in GCP metrics metricset for distribution type metrics {pull}41822[41822]
+- [K8s Integration] Enhance HTTP authentication in case of token updates for Apiserver, Controllermanager and Scheduler metricsets  {issue}41910[41910] {pull}42016[42016]
+- Fixed `creation_date` scientific notation output in the `elasticsearch.index` metricset. {pull}42053[42053]
+- Fix bug where metricbeat unintentionally triggers Windows ASR. {pull}42177[42177]
+- Remove `hostname` field from zookeeper's `mntr` data stream. {pull}41887[41887]
+- Continue collecting metrics even if the Cisco Meraki `getDeviceLicenses` operation fails. {pull}42397[42397]
+- Fixed errors in the `elasticsearch.index` metricset when index settings are missing. {issue}42424[42424] {pull}42426[42426]
 - Fixed panic caused by uninitialized meraki device wifi0 and wifi1 struct pointers in the device WiFi data fetching. {issue}42745[42745] {pull}42746[42746]
 - Only fetch cluster-level index stats summary {issue}36019[36019] {pull}42901[42901]
 - Fixed an issue in Metricbeat's Windows module where data collection would fail if the data was unavailable. {issue}42802[42802] {pull}42803[42803]
@@ -364,6 +374,9 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]
 
 
 
+
+
+
 
 
 

libbeat/docs/release.asciidoc

@@ -11,6 +11,10 @@ upgrade.
 * <<release-notes-8.17.2>>
 * <<release-notes-8.17.1>>
 * <<release-notes-8.17.0>>
+* <<release-notes-8.16.5>>
+* <<release-notes-8.16.4>>
+* <<release-notes-8.16.3>>
+* <<release-notes-8.16.2>>
 * <<release-notes-8.16.1>>
 * <<release-notes-8.16.0>>
 * <<release-notes-8.15.5>>