User-defined objects

[PhilippGackstatter]

Motivation

The current asset model in Miden supports fungible and non-fungible assets, which are a way for users to represent value and ownership on-chain. This model couples the transaction kernel to these specific asset types as it handles construction, modification, and destruction of these assets. I think this coupling is undesirable as it makes the protocol hardcoded to those two asset types and makes extension from users impossible.

A more generic and decoupled approach would be to adopt an object-based model, where asset types are represented as user-defined objects defined outside the kernel but managed by it. The kernel would operate on these objects as opaque entities, focusing solely on object management rather than specific asset logic. This change would make the programming model more adaptable and better at handling diverse use cases. At the same time it would make the transaction kernel and protocol at large agnostic to the concrete assets/objects that are represented.

Idea

The basic idea is to have the transaction kernel manage objects. An object represents value or ownership, just like assets, but is user-defined. The details of each object would be implemented outside the kernel, keeping the kernel decoupled. In this model the kernel would only know about objects as black boxes and offer generic methods for creating, setting and getting fields as well as destroying them.

To illustrate this further, here are a few bullet points to hopefully get the idea across. I might be missing some details of how the current asset model has evolved and whether some of its limitations are byproducts of constraints I don't know about. I might even be thinking incorrectly about the asset model. It's intended both as a proposal but also as a way to learn more about the design choices that led to the current model in case I have misunderstandings.

I'll try to explain what I mean in more detail with an example. Let's say we want to represent a fungible asset issued by a faucet in this object model.

• For simplicity, every account can issue objects (currently only faucets can issue assets), though this is not a must.
• Code would be addressed as a first-class protocol entity, a package (see also here for the idea of an on-chain registry which is similar). Such packages have their own package ID and can be published on-chain. This is not that different from accounts as they are now, but packages wouldn't need to be full accounts as they don't need storage or vaults like accounts do. Package IDs could probably be generated like account IDs. So a useful mental model for a package is as an account without a vault and without storage.
• The reason why packages need to be first class citizens is that a package can define object types. For instance, we might write a miden_std library (i.e. a MastForest) which is published as a package on-chain. It could contain a miden_std::Coin type to represent what are now fungible assets.
• Each object would have a unique ID to authenticate which package issued it. This is a hard problem to solve, but the initial thought is that this would probably be a combination of:
  • a counter starting from 0 to differentiate between different object types from the same package.
  • the package ID to be able to verify the origin of an object and which package's procedures are allowed to modify an object.
  • and a protocol-generated part (i.e. block hash or similar as input) so that multiple miden_std::Coin objects have different IDs.
• Objects are stored in kernel memory. On the stack, objects are pointers to kernel memory. The reason for this is that we would want to enforce that objects can only be modified by code defined in the package that defines the object type, e.g. miden_std::Coin can only be modified through miden_std::Coin::* procedures.

Here is a more concrete visual example for how issuing a new type of Coin might work, which builds on the proposed VM component model.

In this example, a user wants to issue a Coin of the Example flavor, i.e. a type they defined. To do so, they execute a transaction against their own account whose code package is user_package. Here, every package is run in a separate VM component whose ID is the package ID (assuming that's feasible). When the account calls miden_std::Coin::new, it passes a pre-existing Example object (just to keep this simple) from its vault and miden_std::Coin::new would verify that the issuer of this user_package::Example object is the calling component, i.e. that the issuer account ID and the component ID are equal. This might still require something like the caller instruction. This is something only the Coin::new procedure would do. A Coin::get_amount procedure could be executed by any caller as it does not require any kind of authentication.

In the second step, kernel::create_object would create a new Coin object whose issuer is set to the component ID of miden_std which is the same as the miden_std package ID. So more generally, this allows objects of type package_id::Type to be created/modified/destroyed only by procedures that are defined in the package with ID package_id.

The Coin object would store in one of its fields that it is of flavor Example. Then, two Coin<Example> objects could be merged into one by a Coin::merge procedure, but it would fail if called on Coin<Example> and Coin<OtherType>.

The Coin object is returned to the control of the user account component and it can choose to store it in its vault or transfer it to a note via a call to another kernel procedure.

I hope this covers a rough, high level idea of this approach.

Advantages

• Make Miden more generic, as any kind of user-defined object could be natively represented rather than having to be defined as one the two asset types we have now. Examples:
  • A Name service could define a NameObject { name: Word } type which represents ownership over a name and the capability to update it.
  • A staking implementation could define a StakedExample object which represents a receipt for having staked Coin<Example> and represents the capability to unstake and retrieve the staked coins.
  • Bridges can define objects for wrapped versions of bridged assets defined on other chains.
  • Next to a Coin type there could be a TreasuryCapability object type which gives whoever owns it, the right to increase or decrease the supply of a Coin<T>. Burning the object would make the supply verifiably fixed. This is the kind of stuff we could technically have with the asset model as well, by introducing a "fungible faucet treasury capability", but we wouldn't realistically do it because it would be a manual extension of the protocol. With an object based model this would be comparatively easy.
• Faucets currently can issue one asset type. In the object model, accounts can issue many object types. This allows for more flexibility and richer expressivity within a module (e.g. Coin and TreasuryCapability).
• Developers should have a better experience by being able to directly interact with objects and their fields instead of (in the current model) having to represent a custom asset like NameObject as a non-fungible asset, which must be hashed first to verify it is the actual pre-image. This must be done for every user-defined asset in the current model, iiuc.
  • This also makes me wonder if we could have better documentation eventually by having more concrete types like NameObject rather than a name object which is hashed into a NonFungibleAsset. E.g. a name service could describe in its API docs that to change the ID to which a name points, a NameObject must be provided to prove ownership over it rather than NonFungibleAsset, which is less clear.
• It would lead to less coupling between the transaction kernel and our asset/object model. Having less coupling should mean we can more easily evolve our transaction kernel in the future. In particular a library like miden_std and the kernel can evolve independently over time.
• Even if this model would for now be too expensive to represent larger objects in, it would keep the door open for future improvements that make this eventually possible. Until then, users could choose to represent complex objects with a NonFungibleAsset object, that works the same as the current one, i.e. it commits to something. In the future, new object types could be defined without having to upgrade the protocol.
• (Optionally) Get rid of the distinction between faucets and accounts. There are only generic accounts, code packages and objects.
• With objects represented in kernel memory and as pointers on the stack, we should no longer have the problem of non-fungible assets only having the faucet prefix in their stack representation.

Drawbacks

• Performance: This object model would definitely be less efficient than the current asset model (due to pointers and memory loading and making cross-context calls more common). I'm not sure how much worse this would be, maybe others have a hunch.
  • The counter argument is that as the VM, cryptography and hardware get more efficient over time, I'm going to assume this will be less of a problem in the future and I think in the long run, the performance cost could outweigh the benefit of having a more generic model from the start.
• Complexity: Making things more flexible and generic comes with an increase in complexity. Adding objects, object IDs, packages and package IDs would increase the complexity of the protocol and the kernel.
  • Some complexity would also be taken from the kernel as it would no longer have to deal with the details of assets, such as verifying max amounts of fungible assets or calculating amounts when withdrawing.

Inspiration

The above is an idea for an object-based model on Miden, replacing the current asset model. It is inspired by Sui Move's object-based programming model. I personally really enjoyed using their object model as a developer and think that Miden could benefit from something similar, so the above is an adaptation to hopefully fit into Miden's architecture. Here is a link to Sui's Coin type to get an idea of what methods could be possible and how different object types could be used together.

Conclusion

There are probably many details I haven't though of, as this would be a pretty big change. I would be interested if anyone sees the merits in something like this, especially from a developer perspective and if we should explore this further. Thanks for taking the time to read and I would be happy to receive feedback on this!

[bobbinth]

Thank you for such a detailed write up! I really like the idea of adding more customizability/programmability to assets, and have been trying to think in this direction as well. There are a few things, however, that are not yet clear to me in the proposal.

One such thing is who/how keeps track of the "bookkeeping data" about created objects. By bookkeeping data I mean things like the total number of issued coins for a fungible asset, or all issued non-fungible assets. Currently, we have a dedicated storage slot in the faucet accounts to keep track of this data - but how would this be handled in the model where any account can create objects of different types?

Somewhat related to the above: in the current model, the assets are "static". That is, once issued they don't really change (an exception to this is the amount part of the fungible asset). One reason for this is that the original issuer needs to make sure they don't create "duplicate" assets. But if an asset can be modified after it is issued, this may become complicated. Another way to think about this is that we need to make sure that an asset's "vault key" remains stable throughout its lifetime while some other attributes of an asset are allowed to change.

Also, it seems to me that we need to route most interactions with the objects through the vault. That is, instead of calling procedures on an object directly, we'd call a procedure on an object that is stored in a vault. This way, we can ensure that an object can be modified only via its "native" procedures. So, for example, instead of first removing an asset from the vault and then adding it to a note (via two separate procedures), we'd have to provide kernel procedures like:

#! Moves the specified asset from the account to the note with the specified index.
#!
#! Inputs: [ASSET, note_idx, ...]
#!
#! Fails if the asset is not present in the account's vault, note with such index does not exist, etc.
export.move_asset_from_account_to_note
    ...
end

#! Moves the specified asset from the note which is currently being processed into the account.
#!
#! Inputs: [ASSET, ...]
#!
#! Fails if the asset is not present in the note.
export.move_asset_from_note_to_account
    ...
end

#! Updates the specified asset in the account's vault by calling the specified procedure on it.
#!
#! Inputs: [ASSET, PROC_ROOT, <proc parameters>
#!
#! Fails if the asset is not in the account's vault or if PROC_ROOT is not a "native" asset procedure.
export.mutate_account_asset
    ...
end

With this setup it would be possible to define assets generically. For example, the asset definition itself would know how to insert an asset into the vault and so, for example, for fungible assets move_asset_from_note_to_account would know that if there is already an asset in the vault the "incoming" asset would need to be merged with the existing one.

Another question is what should be the capabilities of the native asset procedures. For example, it would be great to have the ability to define an asset that can only be transferred to accounts which meet certain criteria (e.g., the owner of the account is 18 years old or older). For this, native asset procedures should be able to call methods on the account itself (maybe via FPI?).

[PhilippGackstatter]

Great questions, thanks!

One such thing is who/how keeps track of the "bookkeeping data" about created objects.

My thinking was that the bookkeping data of objects is not generic but specific to the type of object. A coin's bookkeping data is the total supply, its decimals or its symbol. A non-fungible asset's bookkeping data might be the total number of issued assets, though some issuers might not care about it. A nameservice's bookkeping data would perhaps be a name -> address mapping.

So bookkeeping data is "specific" in the above sense to each use case. So I think it can be represented by more objects, assuming that not all objects generate more bookkeeping data. For example, when minting a new Coin, one gets a treasury capability object and a coin metadata object (which here represent the bookkeeping data). The treasury capability contains the total supply and the coin metadata contains the decimals, the symbol and other such static data. These could be stored in the issuing account's vault.
If an issuer wants to increase the supply of their Coin, they can use their treasury capability from the vault to mint new coins, which increases the supply field in the treasury capability object.
If someone other than the issuer wants to reference the total supply of a specific coin type, they would have to obtain a proof that the treasury capability is in the issuer account's vault and then, in MASM, execute a foreign procedure invocation to get this object and its supply field.

The advantage of using objects as bookkeping data is that there would be a standardized representation (e.g. every Coin has a CoinMetadata) and that they can be concretely identified (e.g. if you want to find the metadata of some Coin<Example>, you look for CoinMetadata<Example>). This data could be indexed by nodes (or node extensions) for public accounts, which I assume most issuer accounts would be. I'm pretty sure that's how this works in Sui for them to be able to support APIs such as getCoinMetadata.

Non-fungible assets I find generally difficult to discuss in this context, because they are so generic. This is one of the motivations of introducing objects, so that instead of a generic representation, we have (more) concrete ones.
So I'm not sure if a general "non-fungible object" API even makes sense, but assuming that an issuer of such objects simply wants to keep track of the number of issued assets, that would not necessarily have to be an object but could also be just a counter in one of the issuer's account storage slots.

Object ID

Another way to think about this is that we need to make sure that an asset's "vault key" remains stable throughout its lifetime while some other attributes of an asset are allowed to change.

I agree that an asset/object's vault key must remain stable. In the original post I described a high-level approach for deriving such IDs, but I now think it could be simpler. The idea is that object IDs are generated "by the protocol".
One approach could be that for each transaction we initialize an OBJECT_SEED word with the hash of (BLOCK_HASH, [account_id_prefix, account_id_suffix, 0, nonce]). This seed should be non-gameable and different for every transaction, even TXs against the same account that reference the same block, due to inclusion of the nonce.
When creating a new object, we can generate an object ID by hashing (OBJECT_SEED, [0, 0, 0, counter]), where counter is initialized to 0 and incremented every time a new object ID is generated.

The properties of this should be:

• A unique ID for every created object. This object ID is the vault key of the object.
• A uniform distribution of objects across the vault.
• Objects can be modified while their ID remains stable.
• Objects can be addressed by their ID in the asset vault in a transaction. They can also be referenced via this ID globally in the blockchain.
  • A nice thing about globally unique IDs for objects is that one can always refer to a specific object by its ID, for example a CoinMetadata<Example>. That should make it possible to find the account where it is stored (through indexing by object ID and perhaps object type), which can then be fetched for use in an FPI call within a transaction that needs access to that coin's metadata.

The object ID does not authenticate the object, but that's fine I think, because objects are created and destroyed in the controlled environment of the TX kernel and are already authenticated via the asset vault in accounts and the assets hash in notes. Moreover, if the ID would authenticate the object it would change when the object changes, so the stability of the ID is important.

One reason for this is that the original issuer needs to make sure they don't create "duplicate" assets

This ID doesn't prevent issuing "duplicate" assets in the sense that an issuer could issue two objects of type Coin<Example> both with amount = 5. But that seems like a legitimate use case. Do we actually have to prevent that?

For example, the asset definition itself would know how to insert an asset into the vault and so, for example, for fungible assets move_asset_from_note_to_account would know that if there is already an asset in the vault the "incoming" asset would need to be merged with the existing one.

With these IDs we could not query the vault for an existing Coin<Example> that another Coin<Example> can be merged with. So these wouldn't automatically be merged together, but a user could do so explicitly. As far as I can tell this is not essential to have, but just an optimization?

Object procedures

Also, it seems to me that we need to route most interactions with the objects through the vault.

This seems like an important point and I agree that's probably how it would work.
My question is how would we support a flexible amount of object fields and how would mutate_{note,account}_asset be implemented on a high level?

If objects are stored in vaults then we can only directly support two words of data (the Smt key and value) per object. The key would be taken up by the object ID.
For every object we have to store at least some bookkeeping data like its object type. For now I assume this to be simply [package_id_prefix (64 bits), package_id_suffix (56 bits) | type_counter (8 bits)], although there is a good argument to be made to represent the full type, e.g. if we have some_package::Coin<other_package::CoinType> then maybe we'd want to include the full generic type in the bookkeeping, but that's a rather detailed point for later discussion.
In any case, we need some mechanism to allow objects to be defined with more fields.

We can use the value of the leaf by making an indirection and:

1. pack an SMT in there by storing the root in the value,
2. or a sequential hash of the object's fields.

SMT Layout:

Object SMT
├── 0
│   └── BOOKKEEPING = [package_id_prefix, package_id_suffix | type_counter, 0, num_fields ]
├── 1
│   └── FIELD_1
└── 2
│   └── FIELD_2
...
└── n
    └── FIELD_n

Sequential Hash Layout:

[
  OBJECT_ID,
  BOOKKEEPING = [
    package_id_prefix,
    package_id_suffix | type_counter,
    0,
    num_fields
  ],
  field_0,
  field_1,
  ...,
  field_n
]

This comes with trade-offs:

• The SMT's advantage is that objects can get very large. We could reserve a small number of bits in the object ID to specify the depth of this tree, so that, for instance, an object could have any power-of-two number of fields. For a small object with just 2^2=4 fields, the object's field proof would be just 2 hashes. For a large object with 2^10=1024 fields, the object's field proof would be 10 hashes.
• An SMT also means that for large objects, not all fields must be loaded into the advice provider to read or write a new value, though not sure if that's actually needed on the object level, but may be a nice to have. On the other hand, it's questionable how large objects would be in practice - probably typically rather small. So loading the entire object might be just fine.
• To explore what both approaches mean when writing an object field:
  • SMT approach
    • retrieve the object SMT root using smt::get on the asset vault tree at key OBJECT_ID.
    • an smt::set operation on the object SMT root to set the new field's value.
    • another smt::set to update the object root in the asset vault tree
  • Sequential Hash approach
    • retrieve the object's sequential hash using smt::get on the asset vault tree at key OBJECT_ID.
    • load the entire object into kernel memory and verify its hash, then write the new field in memory,
    • Once done with the object, compute the sequential hash again and write the hash into the asset vault tree with smt::set.
  • In both approaches, there may be multiple operations on the object and we may want to avoid updating the asset vault SMT after every write which is unnecessary and inefficient. It would be better to update it once after all changes are done, but not sure how to facilitate that without making it more complex.

Procedure Capabilities

Another question is what should be the capabilities of the native asset procedures. For example, it would be great to have the ability to define an asset that can only be transferred to accounts which meet certain criteria (e.g., the owner of the account is 18 years old or older).

Yeah, that would be very powerful. In Sui, for example, an object can be defined to have or not to have a store ability. If it does have this ability, it can be transferred using a standard public_transfer function. If it doesn't, the object can't be transferred with the public_transfer function but the object itself can define how it can be transferred. This can be used to define a variant of a coin object which can have a global deny list, for example.

I imagine we could do something similar by specifying on object's somewhere (ID or bookkeeping data) whether it has such abilities or not.

Or, as you mentioned, it could be done with something like FPI. While technically possible, it seems tricky for a client to figure out what data an object will require from an account, so that it can load the right data into the advice provider.

This depends on how much flexibility we need here. The abilities would be protocol-defined, so fairly static, while the FPI mechanism would allow for permissionless extension, which keeps the protocol simpler and is more powerful, but also may be more tricky to use.