bug: sort order ignored in store nodes #2317
Comments
|
I think the issue comes from: Line 11 in 07beea0 |
|
I see. If that's the case, this default value is not consistent with the default value of the protobuffer. They both should match. |
|
@Ivansete-status Who should fix this? May I, or...? Btw, just curious, how come did we notice this now? I believe the "forward" as default was already in place. |
|
In status-go this was not noticed until now because we retrieve messages in 1d batches, and each batch seems to load fast enough for the problem to not really be noticeable, but a new feature @igor-sirotin is working on required retrieving and processing messages from newest to oldest, and the content topic on which these messages were published had few messages (~30) so it was easy to notice that the ordering was not correct |
|
I think you already fixed @AlejandroCabeza ! xD
But it works on I will double-check next week (27th'Dec) but we might need to create a patch release |
Ivansete-status
moved this from Todo
to In Progress
in Status Waku Integration & Chat Reliability
Oh, did I? Well, that's great 😂 I'm glad I randomly solved something? 🤣 |
chair28980
added
the
status-waku-integ
|
Hi, @chair28980 - in which fleet I can find the fix? (if it is present there) |
@mprakhov I'm not sure that this fix is live yet as this issue is still open. @Ivansete-status @AlejandroCabeza can you confirm? |
|
This is a high-priority issue now as it impacts fetching the community data. From what I see, we are getting a random order on prod fleet |
|
Weekly Update
|
|
@mprakhov @richard-ramos @Ivansete-status |
|
@igor-sirotin |
|
Just tested the cc @richard-ramos @Ivansete-status {"envelopeHash": "0xd9ecf2ee6ae9c32688c364f4ac35b32e590fa3913e248b67be0699a887d758a4", "pubsubTopic": "/waku/2/default-waku/proto", "contentTopic": "/waku/1/0xee3a5ba0/rfc26", "timestamp": 1705066820192998000, "timestamp string": "2024-01-12 13:40:20.192998 +0000 GMT"}
{"envelopeHash": "0x162d44ce9b96480524567924c574e8fe841a76e9e6f5621b2d585a45d17cd485", "pubsubTopic": "/waku/2/default-waku/proto", "contentTopic": "/waku/1/0xee3a5ba0/rfc26", "timestamp": 1705070420123304000, "timestamp string": "2024-01-12 14:40:20.123304 +0000 GMT"}
{"envelopeHash": "0xa717b19b7973265f7f2e81707cc19c1309115f540796febd567c8a13b9d77109", "pubsubTopic": "/waku/2/default-waku/proto", "contentTopic": "/waku/1/0xee3a5ba0/rfc26", "timestamp": 1705074020131159000, "timestamp string": "2024-01-12 15:40:20.131159 +0000 GMT"}
{"envelopeHash": "0x57354b5640cdfd84748658c674b7514a634f09d66949eb02618111930b06838a", "pubsubTopic": "/waku/2/default-waku/proto", "contentTopic": "/waku/1/0xee3a5ba0/rfc26", "timestamp": 1705933751431230000, "timestamp string": "2024-01-22 14:29:11.43123 +0000 GMT"}
{"envelopeHash": "0x6278779e25e960c9633fd6b74fdd41cd528e8f1179afafb4c80ac318086a67f7", "pubsubTopic": "/waku/2/default-waku/proto", "contentTopic": "/waku/1/0xee3a5ba0/rfc26", "timestamp": 1705937351498564000, "timestamp string": "2024-01-22 15:29:11.498564 +0000 GMT"}
{"envelopeHash": "0x444d4794fb740dce09319fb6b4e097d1364035ed7080c4e2a09cb831d0dcc1b6", "pubsubTopic": "/waku/2/default-waku/proto", "contentTopic": "/waku/1/0xee3a5ba0/rfc26", "timestamp": 1705940951445069000, "timestamp string": "2024-01-22 16:29:11.445069 +0000 GMT"}
{"envelopeHash": "0xe2262bf2c66f5ca6162ca875d5f79f945ab0e55a0d8edcb7307e8c68c710007b", "pubsubTopic": "/waku/2/default-waku/proto", "contentTopic": "/waku/1/0xee3a5ba0/rfc26", "timestamp": 1705944551458662000, "timestamp string": "2024-01-22 17:29:11.458662 +0000 GMT"}
{"envelopeHash": "0x2f1cf636c1f0ef19f2fd4669c064509d12dcf52b98e56ba3c6d0b938732c885d", "pubsubTopic": "/waku/2/default-waku/proto", "contentTopic": "/waku/1/0xee3a5ba0/rfc26", "timestamp": 1705948151553462000, "timestamp string": "2024-01-22 18:29:11.553462 +0000 GMT"}
{"envelopeHash": "0x45ad89f5e29fc7d4c8634a9761115103a9bdb1dfbb0743d9775f96fcebbc790f", "pubsubTopic": "/waku/2/default-waku/proto", "contentTopic": "/waku/1/0xee3a5ba0/rfc26", "timestamp": 1705951751597182000, "timestamp string": "2024-01-22 19:29:11.597182 +0000 GMT"}
{"envelopeHash": "0xe3351c83d60312d6f63c3460e56c2489f14718f3d7f441395a40057760e72438", "pubsubTopic": "/waku/2/default-waku/proto", "contentTopic": "/waku/1/0xee3a5ba0/rfc26", "timestamp": 1705955351560812000, "timestamp string": "2024-01-22 20:29:11.560812 +0000 GMT"} |
Hey @igor-sirotin ! |
|
I close this issue because it was sorted out by @AlejandroCabeza as stated in #2317 (comment) |
github-project-automation
bot
moved this from In Progress
to Done
in Status Waku Integration & Chat Reliability
This status.prod node:
For the following parameters:
/waku/2/default-waku/proto/waku/1/0xee3a5ba0/rfc261700576989000000000to1703255389000000000Has the following messages, ordered from older to newer (just displaying the timestamps here):
In status-go, we're trying to retrieve the messages from newer to older, 3 pages at a time. For this the following query was created:
{ "request_id":"0102030405060708", "query":{ "pubsub_topic":"/waku/2/default-waku/proto", "content_filters":[{"content_topic":"/waku/1/0xee3a5ba0/rfc26"}], "start_time":1700576989000000000, "end_time":1703255389000000000, "paging_info":{ "page_size":3, }, } }Notice that the page ordering is not specified, so according to protobuffer definition, the default BACKWARDS ordering should be applied (i.e.
DESCin sql)With this, we were expecting the first and second page to have the following results:
However the following incorrect ordering is being returned:
To see this scenario in action the following PR from test-waku-query can be used: waku-org/test-waku-query#10
cc: @igor-sirotin
The text was updated successfully, but these errors were encountered: