Propagate entity deletions to Cumulocity cloud

[albinsuresh]

Is your feature request related to a problem? Please describe.

When an entity is deleted from the entity store of the agent, the mapper reacts to this by removing the corresponding entry from its entity cache. But, the same is not propagated to the cloud to delete the twin of that entity from the cloud as well. This leaves the burden of the cleanup from the cloud to the user. It would have been better if a deletion from the thin-edge entity store is properly synced with the cloud as well, so that the users don't have to do this manually.

Describe the solution you'd like

The mapper needs to map the entity clear message that it receives to the REST call to delete the corresponding managed object from Cumulocity.

To avoid the user from accidentally deleting entities from the cloud when they clear local state, propagation of these deletions must be controlled by a tedge config setting(e.g: c8y.sync.inventory.delete: true or false) which the user must explicitly enable (default: false).

Additional context

We're forced to use the HTTP API for deletions because deletion of managed objects from the inventory is not supported over MQTT, neither with SmartREST and not even with JSON over MQTT as documented here.

On top of that, the managed object deletion API expects the numeric internal id of the managed object as the input and not the external id. So, an additional lookup would be required to fetch the internal id using the external id.

[didier-wenzek]

To avoid the user from accidentally deleting entities from the cloud when they clear local state, propagation of these deletions must be controlled by a tedge config setting(e.g: c8y.sync.inventory.delete: true or false) which the user must explicitly enable (default: false).

Using a flag seems fragile: what about all the entities that have been removed while the flag was turned off. Will they be cleared on c8y when synchronization is turned on?

I would rather go with an on-demand synchronization command. This will lead to less surprises and will give more flexibility to the user.

[albinsuresh]

Using a flag seems fragile: what about all the entities that have been removed while the flag was turned off. Will they be cleared on c8y when synchronization is turned on?

I would rather go with an on-demand synchronization command. This will lead to less surprises and will give more flexibility to the user.

I have a feeling that a combination of both would be the right solution. I fully appreciate the need for an on-demand synchronization command for the reasons that you highlighted. But, having to trigger a command every time after an entity deletion also would be cumbersome for a user that prefers to have his twins deleted synchronously as-and-when they delete the entities locally.

So I'd still keep the flag to control the behaviour when the mapper is live while performing the deletion and use the command when someone wants to do some extensive changes post which they'd like to synchronize the whole entity tree at once.

@reubenmiller Thoughts?