Releases: ethersphere/bee
v1.17.0
Bee v1.17.0 is finally here. This release will be the most significant update to the bee client in a long time and will be a breaking release due to protocol and API changes. The Bee team has been working on the new storage layer for quite some time now. The new storage layer is the repayment of a massive technical debt for the team as the old local storage component was not in a good state and was no longer maintainable. Follow all the communication on our discord to know all the details!
The goal of this upgrade is not for any performance benefits at the moment, but to ensure all existing functionality is retained and the storage component is simplified and hence improving maintainability.
The API changes are mostly consolidations and minor changes to the response types. Users are advised to go through the API docs and update their libraries/tooling accordingly.
As mentioned earlier, this will be a breaking release on the protocols-side as well. Pullsync, hive, retrieval, and handshake versions will be bumped. As a result, all node operators are requested to update their nodes to the latest release as soon as possible. The nodes will also do a full resync of the postage stamp contract events to properly flush out all expired content from their reserves once they upgrade to the new storage layer.
The upgrade, when first deployed, will begin to migrate data from the old localstore to the new and in the best case should finish in a few minutes but may take longer on slower devices. Because data migration is involved, it is higly advised to back up everything before the upgrade.
Warning for multiple node operators: the upgrade should be done by one node at a time for nodes that share the same physical disk as the initial migration from the old localstore to the new may take up to 30 minutes on older/slower hardware.
New features
- Support multiple stamps on chunks. Chunks uploaded with different stamps will be tracked separately in the network. This means the chunk will only be removed from the network once all the stamps associated with it are gone.
- Stewardship PUT endpoint now allows users to configure a stamp to use for re-upload. This would allow users to re-upload their pinned content with new stamps if the old one was expired.
Breaking changes
/tagsendpoint is now consolidated on the main API. The debug API tags endpoint is deprecated. Also, the tag response fields have changed.- DELETE
/chunkendpoint is also deprecated. This endpoint was mainly present for testing and will no longer be supported.
Hardening
- Pinning data is now managed separately. As a result the pinned and reserve content will no longer affect each other.
- Self-health check based on reserve size of neighborhood. (#4110)
- Synchronize interval state storing. (#4115)
- Prune out-of-depth oversaturated bins based on peer health. (#4092)
- Cleanup mutex handling in batchstore and reserve. (#4159)
Bugfixes
- Unreserve batch incorrectly unpins chunks. Also other issues related to pinning. (#3037, #3224, #3302, #3514)
- Reduce light-node disk usage. (#2990)
- Reuploading content without increasing bucket collision counters. (#2851)
- Upload related memory leak. (#4009)
For a full PR rundown please consult the v1.17.0 diff.
v1.16.1
v1.16.0
v1.16.0
The bee team is excited to announce v1.16.0! 🎉
The team has been busy researching and testing ways to help swarm remain a reliable and healthy network, and to that end, we are happy to announce a brand new health service, salud.
With salud, nodes will periodically perform certain health checks on it's connected peers with data acquried from the status protocol. The checks as of this release are based on the duration of response to the status protocol msg, number of connected peers, the storage radius, and total batch commitment as computed by each peer.
For duration and number of connected peers, each peer must be within the 80th percentile to be deemed as healthy. Radius and batch commitment are measured in terms of the most common values as reported by each connected peer. Measurements are created anew for each peridoic health check.
A self check is also in place where if the node's own storage radius does not match with the rest of the networks', the node won't participate in the schelling game.
With this release, only the pushsync protocol utilizes the filtering of peers for requests based on the status of health.
For questions, comments, and feedback, reach out on discord.
Breaking changes
- The status protocol now includes the reachability of a peer and the total batch commitment of the network reported by each peer (#4054, #4042)
Bug fixes
- Fixed the last played round field in the redistribution state endpoint to report the corrent round instead of the previous round. ( #4062)
Hardening
- Loosed the requirements of the pushsync protocol so that in the case a neighborhood is down or does not exist, the chunk is pushed as close as possible to it's neighborhood. (#4064)
- Increased the wake up interval for the depthmonitor from 5 to 15 minutes. (#4068)
For a full PR rundown please consult the v1.16.0 diff.
v1.15.0
v1.15.0
The bee team is excited to announce v1.15.0!
With this release, we introduce a new pushsync feature to improve chunk syncing and replication in the network during uploading. Peers that forward chunk into the neighborhood will fire multiple requests to target multiple storer nodes. Forwarding will also terminate at the first peer within the neighborhood and will no longer be forwarded to the most closest peer within the network.
As a result of the change in protocol logic, the protocol version has been bumped so it's important that you upgrage your nodes to the latest version.
We've added two new fields to the status protocol response: total amount from the chainstate and mode of operation of the peer (light or full mode).
For questions, comments, and feedback, reach out on discord.
Breaking changes
- Pushync forwarding nodes one hop from the neighborhood multiplexes the chunk to multiple storer nodes in the neighborhood. Forwarding terminates at the first neighbor. (#4008)
Hardening
- Blocklist endpoint now correctly display the mode and reason of the blocklisting of the peer. (#4015)
- Storage radius decreasing based on reserve size increased back to 50% threshold. (#4013)
- Chainstate total amount and peer mode added to the status protocol. (#4002 #4010)
- Connecting light nodes are not rejected based on bin size. (#4018)
Perfomance Improvements
- Initial historical syncing is limited to reduce workload. (#4000)
- Peer skiplist mechnism in retrieval and pushsync protocols now prune peers in a much more effiecient way. (#4001)
For a full PR rundown please consult the v1.15.0 diff.
v1.14.1
v1.14.1
This one is a patch release to fix the innacurate the number of reported active nodes in the swarm scanner.
With this change, light nodes attempting to connect to a peer with a full topology bin won't get rejected.
For questions, comments, and feedback, reach out on discord.
v1.14.0
v1.14.0
Bee v1.14.0 is here!
We've been working hard to bring retrieval protocol improvements, a brand new debugging protocol, increased visibility of the network, and a long list of hardenings.
We've introduced a new skip-list mechanism in retrieval protocol so that on retry attempts, requests reach a wider range of peers to improve the success rate of retrieved chunks.
The release features a brand new protocol that acquires from connected peers important information like their storage radius and reserve size. This will bring new visibility to the network and lays the foundation for future features.
The health ping and chainsync requests are now less frequent to reduce network stream count.
The team has also identified and fixed the issue of failing stewardship checks of encryped content.
For fresh nodes, the intensitivity of the initial syncing phase has been reduced by delaying the syncing of "out of depth" peers to 24 hours after running.
The minimum funds (xDAI) required to participate in the storage incentives has been raised to combat the recent increasing number of pending transactions. The minumum amount can be found on-demand using the existing /redistributionstate API.
IMPORTANT NOTE:
The handshake protocol has been bumped as there has bee a new redistribution contract release. It's important that all node operators upgrade to the new version. All bootnodes will be upgraded to the new version on 14:00 CET, April 11.
For questions, comments, and feedback, reach out on discord.
Breaking changes
- The handshake protocol has been bumped as there is a new redistribution contract release.
New Features
- Added a new global "sticky" skiplist to the retrieval protocol, similar to the pushsync, so that different peers across the network can be requested on retries (#3921)
- Added a new debugging "status" protocol which acquires all connected peers' storage radius, reserve size, and other information. Can be requested on-demand using the new debug
/statusand/status/peersendpoints.
Hardening
- Added new retrieval metrics. (#3904)
- Made the pullsync and the puller more resillient to errors. (#3926 #3988)
- Increase the mimumum required funds for a node to participate in the storage incentives. (#3940)
- Persist the state of the storage incentives phases to statestore to make the agent resilient to restarts. (#3943)
- Added extra checks like zero and invalid length address checks to the retrieval protocol before handling the request. (#3954 #3987)
- Improved the receipt check of the pusher for full nodes so that shallow receipts below the storage radius are rejected. (#3994)
Bugfixes
- The traversal package was broken as we did not handle encrypted references. Encrypted reference handling is now fixed. As a side effect, the issue of failing stewardship checks on encrypted references is now resolved. (#3991)
Perfomance Improvements
- The health ping and chainsync requests fire less regurlary to reduce the rate of opened streams. (#3872)
For a full PR rundown please consult the v1.14.0 diff.
v1.13.0
v1.12.0
The Bee team is excited to announce the latest release!
The main focus of the team for past few weeks has been tightening loose ends surrounding chunk syncing.
The release also features a new API endpoint that returns the status of the node related to the storage incentives, like total reward won and the last round the node has participated and won.
Hardening
- The puller also does syncing of proximity order bin with peers outside of the neighborhood and the pushync protocol uses the storage radius for chunk replication . (#3703)
- Rewrote the retrieval protocol to improve readability and maintainability. (#3671)
- Increased the size of the batch of chunks pulled per a single pullsync request. (#3754)
- Removed some stale config options, like block hash and transaction. (#3765)
- Lowered accounting package logging level to two. (#3771)
- Peers that timeout for historical pullsync requests are blocklisted for some time. (#3774)
New features
- Added a new API endpoint (/redistributionstate) that responds with the storage incentives state of node. (#3683)
Bug Fixes
- The depthmonitor resets the storage radius to the reserve radius only for fresh nodes. (#3757)
- The pusher does stamp validation of chunks in the queue just before pushing the chunk, not when entering the queue. (#3760)
For a full PR rundown please consult the v1.12.0 milestone.
v1.11.1
This release is a patch release to fix an issue seen during the latest migration done on upgrade to v1.11.0. This causes the node to fail on boot up and hence node operators are advised to update to v1.11.1 directly if they haven't updated to v1.11.0.
Bugfixes
- Ignore errors that are not fatal on the residual migration. (#3715)
v1.11.0
The Bee team is excited to announce the latest release which mostly consists of minor fixes to chunk syncing and improved uploading experience.
The important news is uploads with mutable batches (which are the most common type of batches) are no longer terminated when max utilization of the batch is reached, whereby newer chunks that belong to the batch will simply replace older chunks in the network.
Hardening
- The node is interruptible while it's waiting for the NAT initialization. (#3688)
- Batch synchronization can now be gracefully suspended on interrupt signal. (#3671)
- Removed old overlay mining logic from the codebase. (#3684)
- Added deferred header to allowed CORs headers. (#3669)
New features
- Uploads with mutable batches are not stopped when max utilization is reached. (#3685)
Bug Fixes
- Fixed the depthmonitor to track active syncing background jobs instead of chunks per second when choosing to reduce the syncing/storage radius. (#3680)
- Lowered the minimum required reserve size to maintain a stable storage radius from 50% of the capacity to 40%. (#3682)
For a full PR rundown please consult the v1.11.0 milestone.