Realtime Database Stream & ssl_clients

[Kikvano]

Hello,
First of all, thanks to the author for the great library.

I'm new to firebase and this library on ESP32(dev module-no psram), have tried all Async examples and no problems at first, but later I faced few issues and maybe someone has experience with similar case.

I have two async clients:

DefaultNetwork network(true);
WiFiClientSecure ssl_client, ssl_client2;
AsyncClientFirebase aClient(ssl_client, getNetwork(network)), aClient2(ssl_client2, getNetwork(network));

One used for auth and data updates without callback and another for HTTP Streaming with callback.

initializeApp(aClient2, firebaseApp, getAuth(user_auth), aResult_no_callback2);
Database.get(aClient, databasePath, asyncCB, true, USER_UID);

Everything works fine, but few times during day I face one of these two problems:

1. Stream task is randomly stopped, sometimes with the error code below and it also frees about 40-50k from heap memory, meaning that its also removed from task queue slot I guess.

[73171194][E][ssl_client.cpp:37] _handle_error(): [data_to_read():361]: (-80) UNKNOWN ERROR CODE (0050)
Free Heap: 157708

But mostly there is no error at all and heap memory stays unchanged, just no more updates or keep-alive events until ESP restart.
When this happen aClient2 task has no problems and periodic data updates are sent to firebase.

2. Sometimes on auth_revoked event i'm getting memory allocation fails:

16:34 | Firebase event: auth_revoked | path: | data: credential is no longer valid | type: 1
Free Heap: 107168
[3614246][E][ssl_client.cpp:37] _handle_error(): [start_ssl_client():264]: (-32512) SSL - Memory allocation failed
[3614246][E][WiFiClientSecure.cpp:144] connect(): start_ssl_client: -32512

when this happen, aClient2 task is not working anymore until next authorization occurs and same time aClient streaming task is listening for events without problems.

Can I somehow identify queue slot of aClient/aClients2 task and remove it manually or maybe completely remove all tasks from queue and reinitialize app ?
My question is, without restarting the MCU what would be the best approach to restart Clients tasks?

Tried calling aClient.stopAsync();/aClient2.stopAsync(); then initialize but didn't help much.

When the authentication task was required, it will insert to the first slot of the queue and all tasks are cancelled and removed from queue to reduce the menory usage unless the SSE mode (HTTP Streaming)task that stopped and waiting for restarting.

When SSE mode (HTTP Streaming) task is currently running and new sync or async task is added to the queue, the SSE mode (HTTP Streaming) task will be stopped (but remains in the queue) as another task was inserted in to the first slot, the SSE mode (HTTP Streaming) task will be restart when another task is finished.

In my case SSE task never restarts itself if stopped. As I understand when it is stopped, there is no point to re-authenticate to restart it either.

[mobizt]

You should read the documentation thoroughly when you are working with async client.

Stream is a persistent task which means it will be restarted when it was timed out from network and internet failures and the authentication revoked because of authentication token was expired.

Stream timed out event is never notified as the library design, and it will be restarted silently. You can check the Stream event time out via AsyncResult::to<RealtimeDatabaseResult>().eventTimeout().

When the auth token was expired and Stream was connected, the server will send auth_revoked event to the client which means you have the problem in re-authentication process because of internet or memory failure.

All tasks can be removed included Stream using AsyncClientClass::stopAsync(true). The task count in the queue can be checked via async client class that handles a queue via AsyncClientClass::taskCount().

When you remove all tasks included Stream, and you wish to reconnect the Stream you have to reconnect it manually because all tasks were removed from queue already.

In ESP32, the critical memory usage occurred when two SSL clients were connected to server simultaneously as in Stream case.

In the Stream examples, another async client will be used for Stream task only to avoid interruption by authentication task as the Stream task runs on another queue and large memory still be used by its SSL client.

You have to maintain and take care of your free heap to ensure that it is enough for SSL client to work at least to use in re-authenticate/re-authorize task otherwise you will not be able to access the Firebase service when the auth token was expired.

You can follow StreamConcurency example for alternative use of ESP_SSLClient when you have memory restriction.

[mobizt]

In the case only Stream stops and is unable to start while other tasks are running ok, your device memory is too low to be used by the SSL client to connect the Stream.

You may have to check memory leaks in your code or reduce memory usage at that moment.

[siz0001]

I am currently developing with an ESP32 module that does not have PSRAM (ESP32-Trinity, https://esp32trinity.com/). Streaming frequently stops, and when I execute an async set, streaming does not work for a while and only resumes after a delay of up to 10 seconds.

I’d like to ask if it is realistically difficult to use streaming on an ESP32 without PSRAM.

[mobizt]

Please try this example for less memory usage Stream in ESP32.

[siz0001]

You are my savior. I want to donate something for you. Please tell me how.

[mobizt]

@siz0001

Thanks, you can support me at
https://buymeacoffee.com/mobizt or
https://github.com/sponsors/mobizt

[Kikvano]

Thanks for suggestions. will try the StreamConcurency example case as well.