Merge pull request #543 from radixdlt/develop

v1.1.0

.github/workflows/main.yml

@@ -96,7 +96,6 @@ jobs:
     name: AMD Migration
     needs:
       - setup-tags
-      - snyk-scan
     uses: radixdlt/public-iac-resuable-artifacts/.github/workflows/docker-build.yml@main
     with:
       runs_on: ubuntu-22.04
@@ -138,7 +137,6 @@ jobs:
     name: AMD Aggregator
     needs:
       - setup-tags
-      - snyk-scan
     uses: radixdlt/public-iac-resuable-artifacts/.github/workflows/docker-build.yml@main
     with:
       runs_on: ubuntu-22.04
@@ -180,7 +178,6 @@ jobs:
     name: AMD Gateway
     needs:
       - setup-tags
-      - snyk-scan
     uses: radixdlt/public-iac-resuable-artifacts/.github/workflows/docker-build.yml@main
     with:
       runs_on: ubuntu-22.04

CHANGELOG.md

@@ -1,3 +1,15 @@
+## 1.1.0 - Babylon
+Release Date: _unreleased_
+
+- Fixed invalid HTTP status code on input validation failure.
+- Changed default configuration value of MaxPageSize for endpoints to 100. Validate if max page size is higher than DefaultPageSize.
+- Added new opt-in `balance_changes` to `/transaction/committed-details` returning resource balance changes for a given transaction.
+- Added new opt-in `receipt_output` to `/stream/transactions`, and `/transaction/committed-details` endpoints. Temporarily set by default to true, to allow client's migration.
+- Added vault-related details to lookups in `/state/entity/details` endpoint.  
+- Optimized `TransactionQuerier.GetTransactions` not to fetch unnecessary data from underlying database.
+- Added strongly-typed OAS definition for `programmatic_json`.
+- Tuned documentation and constraints of various OAS type definitions.
+
 ## 1.0.1 - Babylon 
 Release Date: 10.10.2023
 

Directory.Build.props

@@ -13,7 +13,7 @@
   </PropertyGroup>
 
   <PropertyGroup>
-    <VersionPrefix>1.0.1</VersionPrefix>
+    <VersionPrefix>1.1.0</VersionPrefix>
     <VersionSuffix>develop</VersionSuffix>
   </PropertyGroup>
 

README.md

@@ -24,3 +24,27 @@ For docs giving an overview of the Network Gateway and its place in the Radix Ec
 For docs related to development on the Network Gateway, see the [docs folder](./docs).
 
 For docs related to running a Network Gateway locally, see the instructions about running a local toy deployment in the [deployment folder](./deployment).
+
+## Database Migrations & Application Deployment
+
+While all three main system parts are technically independent of each other it is assumed that during overall stack deployment the following order is preserved:
+
+1. Deploy Database Migrations. This is a short-lived container that executes new database migrations, if any, and exists successfully with `0` exit code. Should this container fail to apply database migrations (non-successful exit code) deployment procedure must be aborted. 
+2. Deploy Data Aggregator. Wait until application healthcheck endpoints report `Healthy` status. 
+3. Deploy Gateway API. 
+
+**Hint:** For Kubernetes cluster deployments it is recommended to set up Database Migrations as init container of Data Aggregator.
+
+**Note:** Babylon Network Gateway is **NOT** compatible with previous Olympia version. Brand-new, clean database must be used. 
+If you're upgrading from Olympia and deploying for the very first time you may want to run Database Migrations application with `WIPE_DATABASE=true` configuration parameter to drop existing database and recreate it. This is irreversible operation. **Proceed with caution!**  
+
+## Dependencies
+
+Mandatory dependencies:
+
+* Connection to at least one RadixDLT Node - source of network transactions,
+* PostgreSQL version 15.2 or newer - primary storage.
+
+Optional dependencies:
+
+* Prometheus - for metrics collection.

apps/GatewayApi/OpenApiDocumentHandler.cs

@@ -154,7 +154,7 @@ private static async Task<PlaceholderReplacements> GetPlaceholderReplacements(
         {
             var wellKnownAddresses = networkConfigurationProvider.GetWellKnownAddresses();
             placeholderReplacements.ResourceAddress = wellKnownAddresses.Xrd;
-            placeholderReplacements.ComponentAddress = wellKnownAddresses.Faucet;
+            placeholderReplacements.ComponentAddress = wellKnownAddresses.ConsensusManager;
             placeholderReplacements.NetworkId = networkConfigurationProvider.GetNetworkId();
             placeholderReplacements.NetworkName = networkConfigurationProvider.GetNetworkName();
         }

apps/GatewayApi/appsettings.json

@@ -25,9 +25,6 @@
     "EnableSwagger": true,
 
     "GatewayApi": {
-        "Endpoint": {
-            "MaxPageSize": 30,
-        },
         "AcceptableLedgerLag": {
             "PreventReadRequestsIfDbLedgerIsBehind": true,
             "ReadRequestAcceptableDbLedgerLagSeconds": 60,

deployment/ci.env

@@ -1,4 +1,4 @@
 # Git commit version of most recent Node compatible with this Network Gateway (only used in CI/CD environments)
-FULLNODE_COMMIT=afad2afc86754bc39bc7a61eae58e2a53cc37c50
-FULLNODE_VERSION=sha-afad2af
-POSTGRES_VERSION=14.3.0
+FULLNODE_COMMIT=feb1952830f4737568aba176de83baf6376265cf
+FULLNODE_VERSION=sha-feb1952
+POSTGRES_VERSION=15.2.0

deployment/docker-compose.yml

@@ -1,7 +1,7 @@
 version: "3.8"
 services:
   fullnode:
-    image: "radixdlt/babylon-node:1.0.0"
+    image: "radixdlt/babylon-node:1.0.4"
     profiles: ["fullnode"]
     init: true
     ports:
@@ -51,13 +51,13 @@ services:
       retries: 3
 
   database_migrations: # This is the base -- the _image and _built containers are defined below
-    image: "radixdlt/babylon-ng-database-migrations:1.0.0"
+    image: "radixdlt/babylon-ng-database-migrations:1.1.0"
     profiles: ["NONE"]
     environment:
       ConnectionStrings__NetworkGatewayMigrations: "Host=postgres_db:5432;Database=${POSTGRES_DB_NAME};Username=${POSTGRES_SUPERUSER};Password=${POSTGRES_SUPERUSER_PASSWORD}"
 
   data_aggregator: # This is the base -- the _image and _built containers are defined below
-    image: "radixdlt/babylon-ng-data-aggregator:1.0.0"
+    image: "radixdlt/babylon-ng-data-aggregator:1.1.0"
     profiles: ["NONE"]
     deploy:
       restart_policy:
@@ -102,7 +102,7 @@ services:
       - ./data-aggregator-fixed-configuration.json:/home/radixdlt/network-gateway/config.json
 
   gateway_api: # This is the base -- the _image and _built containers are defined below
-    image: "radixdlt/babylon-ng-gateway-api:1.0.0"
+    image: "radixdlt/babylon-ng-gateway-api:1.1.0"
     profiles: ["NONE"]
     ports:
       - "127.0.0.1:5308:8080" # This allows you to connect to the API at http://localhost:5308

sdk/typescript/lib/helpers/networks.ts

@@ -5,8 +5,6 @@ export const RadixNetwork = {
   Betanet: 0x0b,
   Kisharnet: 0x0c,
   RCnetV1: 0x0c,
-  Ansharnet: 0x0d,
-  RCnetV2: 0x0d,
   Zabanet: 0x0e,
   RCnetV3: 0x0e,
   Gilganet: 0x20,
@@ -52,18 +50,6 @@ export const RadixNetworkConfig: Record<string, NetworkConfig> = {
     gatewayUrl: 'https://rcnet.radixdlt.com',
     dashboardUrl: 'https://rcnet-dashboard.radixdlt.com',
   },
-  Ansharnet: {
-    networkName: 'Ansharnet',
-    networkId: RadixNetwork.Ansharnet,
-    gatewayUrl: 'https://ansharnet-gateway.radixdlt.com',
-    dashboardUrl: 'https://ansharnet-dashboard.radixdlt.com',
-  },
-  RCnetV2: {
-    networkName: 'RCNetV2',
-    networkId: RadixNetwork.RCnetV2,
-    gatewayUrl: 'https://ansharnet-gateway.radixdlt.com',
-    dashboardUrl: 'https://rcnet-v2-dashboard.radixdlt.com',
-  },
   Zabanet: {
     networkName: 'Zabanet',
     networkId: RadixNetwork.Zabanet,

src/RadixDlt.CoreApiSdk/core-api-spec-copy.yaml

@@ -52,8 +52,9 @@ openapi: 3.0.0
 # => Hex-encoded hashes should end _hash
 # => Hex-encoded bytes/payload should end _hex and not include _bytes
 info:
-  version: '1.0.0'
-  title: 'Radix Core API - Babylon (v1.0.0)'
+  # Should be the release version, and the release display name
+  version: 'v1.0.0'
+  title: 'Radix Core API - Babylon'
   license:
     name: The Radix License, Version 1.0
     url: https://www.radixfoundation.org/licenses/license-v1
@@ -2717,8 +2718,11 @@ components:
       type: string
       description: |
         The simple string representation of the non-fungible id.
-        For string id types, this is simply the string itself; for integer types, this is the integer as a decimal; and for the bytes id type, this is the lower case hex representation.
-        A non-fungible resource has a fixed `NonFungibleIdType`, so this representation uniquely identifies this non-fungible
+        * For string ids, this is `<the-string-id>`
+        * For integer ids, this is `#the-integer-id#`
+        * For bytes ids, this is `[the-lower-case-hex-representation]`
+        * For RUID ids, this is `{...-...-...-...}` where `...` are each 16 hex characters.
+        A given non-fungible resource has a fixed `NonFungibleIdType`, so this representation uniquely identifies this non-fungible
         under the given resource address.
     NonFungibleGlobalId:
       type: object
@@ -5755,6 +5759,7 @@ components:
         - accumulator_hash
         - status
         - fungible_entity_balance_changes
+        - non_fungible_entity_balance_changes
         - resultant_account_fungible_balances
         - total_fee
       properties:
@@ -5786,6 +5791,12 @@ components:
           type: array
           items:
             $ref: "#/components/schemas/LtsEntityFungibleBalanceChanges"
+        non_fungible_entity_balance_changes:
+          description: |
+            Non fungible changes per entity and resource
+          type: array
+          items:
+            $ref: "#/components/schemas/LtsEntityNonFungibleBalanceChanges"
         resultant_account_fungible_balances:
           description: |
             A list of the resultant fungible account balances for any balances which changed in this transaction.
@@ -5873,6 +5884,39 @@ components:
         - FeeDistributed
         - TipDistributed
         - RoyaltyDistributed
+    LtsEntityNonFungibleBalanceChanges:
+      type: object
+      required:
+        - entity_address
+        - resource_address
+        - added
+        - removed
+      properties:
+        entity_address:
+          type: string
+          description: The Bech32m-encoded human readable version of the entity's address
+        resource_address:
+          type: string
+          description: |
+            The Bech32m-encoded human readable version of the non-fungible resource's address
+        added:
+          type: array
+          items:
+            $ref: "#/components/schemas/LtsNonFungibleLocalId"
+        removed:
+          type: array
+          items:
+            $ref: "#/components/schemas/LtsNonFungibleLocalId"
+    LtsNonFungibleLocalId:
+      type: string
+      description: |
+        The simple string representation of the non-fungible id.
+        * For string ids, this is `<the-string-id>`
+        * For integer ids, this is `#the-integer-id#`
+        * For bytes ids, this is `[the-lower-case-hex-representation]`
+        * For RUID ids, this is `{...-...-...-...}` where `...` are each 16 hex characters.
+        A given non-fungible resource has a fixed local id kind, so this representation uniquely
+        identifies this non-fungible under the given resource address.
     LtsResultantAccountFungibleBalances:
       type: object
       required:

src/RadixDlt.CoreApiSdk/generated/Api/LTSApi.cs

@@ -63,11 +63,11 @@
  */
 
 /*
- * Radix Core API - Babylon (v1.0.0)
+ * Radix Core API - Babylon
  *
  * This API is exposed by the Babylon Radix node to give clients access to the Radix Engine, Mempool and State in the node.  The default configuration is intended for use by node-runners on a private network, and is not intended to be exposed publicly. Very heavy load may impact the node's function. The node exposes a configuration flag which allows disabling certain endpoints which may be problematic, but monitoring is advised. This configuration parameter is `api.core.flags.enable_unbounded_endpoints` / `RADIXDLT_CORE_API_FLAGS_ENABLE_UNBOUNDED_ENDPOINTS`.  This API exposes queries against the node's current state (see `/lts/state/` or `/state/`), and streams of transaction history (under `/lts/stream/` or `/stream`).  If you require queries against snapshots of historical ledger state, you may also wish to consider using the [Gateway API](https://docs-babylon.radixdlt.com/).  ## Integration and forward compatibility guarantees  Integrators (such as exchanges) are recommended to use the `/lts/` endpoints - they have been designed to be clear and simple for integrators wishing to create and monitor transactions involving fungible transfers to/from accounts.  All endpoints under `/lts/` have high guarantees of forward compatibility in future node versions. We may add new fields, but existing fields will not be changed. Assuming the integrating code uses a permissive JSON parser which ignores unknown fields, any additions will not affect existing code.  Other endpoints may be changed with new node versions carrying protocol-updates, although any breaking changes will be flagged clearly in the corresponding release notes.  All responses may have additional fields added, so clients are advised to use JSON parsers which ignore unknown fields on JSON objects. 
  *
- * The version of the OpenAPI document: 1.0.0
+ * The version of the OpenAPI document: v1.0.0
  * Generated by: https://github.com/openapitools/openapi-generator.git
  */
 

src/RadixDlt.CoreApiSdk/generated/Api/MempoolApi.cs

@@ -63,11 +63,11 @@
  */
 
 /*
- * Radix Core API - Babylon (v1.0.0)
+ * Radix Core API - Babylon
  *
  * This API is exposed by the Babylon Radix node to give clients access to the Radix Engine, Mempool and State in the node.  The default configuration is intended for use by node-runners on a private network, and is not intended to be exposed publicly. Very heavy load may impact the node's function. The node exposes a configuration flag which allows disabling certain endpoints which may be problematic, but monitoring is advised. This configuration parameter is `api.core.flags.enable_unbounded_endpoints` / `RADIXDLT_CORE_API_FLAGS_ENABLE_UNBOUNDED_ENDPOINTS`.  This API exposes queries against the node's current state (see `/lts/state/` or `/state/`), and streams of transaction history (under `/lts/stream/` or `/stream`).  If you require queries against snapshots of historical ledger state, you may also wish to consider using the [Gateway API](https://docs-babylon.radixdlt.com/).  ## Integration and forward compatibility guarantees  Integrators (such as exchanges) are recommended to use the `/lts/` endpoints - they have been designed to be clear and simple for integrators wishing to create and monitor transactions involving fungible transfers to/from accounts.  All endpoints under `/lts/` have high guarantees of forward compatibility in future node versions. We may add new fields, but existing fields will not be changed. Assuming the integrating code uses a permissive JSON parser which ignores unknown fields, any additions will not affect existing code.  Other endpoints may be changed with new node versions carrying protocol-updates, although any breaking changes will be flagged clearly in the corresponding release notes.  All responses may have additional fields added, so clients are advised to use JSON parsers which ignore unknown fields on JSON objects. 
  *
- * The version of the OpenAPI document: 1.0.0
+ * The version of the OpenAPI document: v1.0.0
  * Generated by: https://github.com/openapitools/openapi-generator.git
  */
 

src/RadixDlt.CoreApiSdk/generated/Api/StateApi.cs

@@ -63,11 +63,11 @@
  */
 
 /*
- * Radix Core API - Babylon (v1.0.0)
+ * Radix Core API - Babylon
  *
  * This API is exposed by the Babylon Radix node to give clients access to the Radix Engine, Mempool and State in the node.  The default configuration is intended for use by node-runners on a private network, and is not intended to be exposed publicly. Very heavy load may impact the node's function. The node exposes a configuration flag which allows disabling certain endpoints which may be problematic, but monitoring is advised. This configuration parameter is `api.core.flags.enable_unbounded_endpoints` / `RADIXDLT_CORE_API_FLAGS_ENABLE_UNBOUNDED_ENDPOINTS`.  This API exposes queries against the node's current state (see `/lts/state/` or `/state/`), and streams of transaction history (under `/lts/stream/` or `/stream`).  If you require queries against snapshots of historical ledger state, you may also wish to consider using the [Gateway API](https://docs-babylon.radixdlt.com/).  ## Integration and forward compatibility guarantees  Integrators (such as exchanges) are recommended to use the `/lts/` endpoints - they have been designed to be clear and simple for integrators wishing to create and monitor transactions involving fungible transfers to/from accounts.  All endpoints under `/lts/` have high guarantees of forward compatibility in future node versions. We may add new fields, but existing fields will not be changed. Assuming the integrating code uses a permissive JSON parser which ignores unknown fields, any additions will not affect existing code.  Other endpoints may be changed with new node versions carrying protocol-updates, although any breaking changes will be flagged clearly in the corresponding release notes.  All responses may have additional fields added, so clients are advised to use JSON parsers which ignore unknown fields on JSON objects. 
  *
- * The version of the OpenAPI document: 1.0.0
+ * The version of the OpenAPI document: v1.0.0
  * Generated by: https://github.com/openapitools/openapi-generator.git
  */
 

src/RadixDlt.CoreApiSdk/generated/Api/StatusApi.cs

@@ -63,11 +63,11 @@
  */
 
 /*
- * Radix Core API - Babylon (v1.0.0)
+ * Radix Core API - Babylon
  *
  * This API is exposed by the Babylon Radix node to give clients access to the Radix Engine, Mempool and State in the node.  The default configuration is intended for use by node-runners on a private network, and is not intended to be exposed publicly. Very heavy load may impact the node's function. The node exposes a configuration flag which allows disabling certain endpoints which may be problematic, but monitoring is advised. This configuration parameter is `api.core.flags.enable_unbounded_endpoints` / `RADIXDLT_CORE_API_FLAGS_ENABLE_UNBOUNDED_ENDPOINTS`.  This API exposes queries against the node's current state (see `/lts/state/` or `/state/`), and streams of transaction history (under `/lts/stream/` or `/stream`).  If you require queries against snapshots of historical ledger state, you may also wish to consider using the [Gateway API](https://docs-babylon.radixdlt.com/).  ## Integration and forward compatibility guarantees  Integrators (such as exchanges) are recommended to use the `/lts/` endpoints - they have been designed to be clear and simple for integrators wishing to create and monitor transactions involving fungible transfers to/from accounts.  All endpoints under `/lts/` have high guarantees of forward compatibility in future node versions. We may add new fields, but existing fields will not be changed. Assuming the integrating code uses a permissive JSON parser which ignores unknown fields, any additions will not affect existing code.  Other endpoints may be changed with new node versions carrying protocol-updates, although any breaking changes will be flagged clearly in the corresponding release notes.  All responses may have additional fields added, so clients are advised to use JSON parsers which ignore unknown fields on JSON objects. 
  *
- * The version of the OpenAPI document: 1.0.0
+ * The version of the OpenAPI document: v1.0.0
  * Generated by: https://github.com/openapitools/openapi-generator.git
  */