Standalone agent upgrade failed when 8.12.0 agent is installed with --unprivileged flag.

[amolnater-qasource]

Kibana Build details:

VERSION: 8.13.0-SNAPSHOT
BUILD: 70749
COMMIT: a0f4897f7c04069faf2a86dbda1dabea78c161c1
Artifact Link: https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.12.0-linux-x86_64.tar.gz

Host OS: Linux- SLES15

Preconditions:

1. 8.13.0 Snapshot should be available.
2. 8.12.0 standalone agent should be installed using --unprivileged flag.

Steps to reproduce:

1. Run agent upgrade command: sudo elastic-agent upgrade 8.13.0-SNAPSHOT
2. Observe standalone agent upgrade failed with error: Error: failed to check if upgrade is already in progress: failed to get agent state: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial unix /run/elastic-agent.sock: connect: no such file or directory"

Note:

• Issue is reproducible on two different VMs.

Screenshot:

What's working fine:

• Issue is not reproducible on Linux- SLES15 when standalone agent is installed without --unprivileged flag and agent gets successfully upgraded to 8.13.0.

• We are successfully able to upgrade 8.12.0 fleet based agent to 8.13.0 SNAPSHOT when installed with --unprivileged flag

Expected Result:
User should be able to upgrade standalone agent to latest version when installed with --unprivileged flag.

Logs:
Agent diagnostics command failed, hence collected logs manually.
logs.zip

Feature:
https://github.com/elastic/ingest-dev/issues/1766

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[amolnater-qasource]

@manishgupta-qasource Please review.

[manishgupta-qasource]

Secondary review for this ticket is Done

[blakerouse]

8.12 Elastic Agent with unprivileged mode has an issue that will not be fixed as it was experimental in 8.12. Please only do testing with 8.13+ versions for unprivileged mode.

8.13->8.13 upgrade was also recently fixed requiring the older 8.13 to at least have been built after Jan 21 because of #4093. If you using a 2 snapshot versions after Jan 21 then upgrading from snapshot to snapshop works.

[amolnater-qasource]

Thank you for looking into this @blakerouse

Could you please confirm if we can test it from 8.12.1>8.13.0, once 8.12.1 is available?

Thanks!

[blakerouse]

This will not be fixed in 8.12 at all. You will need to test 8.13 upgrading to 8.13 snapshot or 8.14 snapshot. --unprivileged is completely broken on 8.12 and wasn't fix as its considered experimental.

You should also run the upgrade as non-root. As in you should not call sudo elastic-agent upgrade, it instead should either be sudo -u elastic-agent elastic-agent upgrade or you can add your user to the elastic-agent group and then just run elastic-agent upgrade without sudo.

[amolnater-qasource]

Hi Team,

We have revalidated this issue on latest 8.14.0 BC4 kibana cloud environment and found it fixed now:

Observations:

• Standalone agent upgraded successfully to 8.14.0 when 8.13.4 agent is installed with --unprivileged flag.

Screenshot:

Build details:
VERSION: 8.14.0 BC4
BUILD: 73836
COMMIT: 23ed1207772b3ae958cb05bc4cdbe39b83507707

Hence we are closing and marking this issue as QA:Validated.

Thanks!