beacon interface

auction-server/src/api/rest.rs

@@ -2,8 +2,8 @@ use {
     crate::{
         api::RestError,
         auction::{
-            per::MulticallStatus,
             simulate_bids,
+            MulticallStatus,
         },
         state::{
             SimulatedBid,

beacon/README.md

@@ -0,0 +1,9 @@
+# Beacon
+
+The Beacon is the off-chain service that exposes liquidation opportunities on integrated protocols to searchers. Protocols surface their liquidatable vaults/accounts along with the calldata and the token denominations and amounts involved in the transaction. Searchers can query these opportunities from the Beacon server. If they wish to act on an opportunity, they can simply construct a transaction based off the information in the opportunity.
+
+The LiquidationAdapter contract that is part of the Express Relay on-chain stack allows searchers to perform liquidations across different protocols without needing to deploy their own contracts or perform bespoke engineering work. The Beacon service is important in enabling this, as it disseminates the calldata that searchers need to include in the transactions they construct.
+
+Each protocol that integrates with Express Relay and the LiquidationAdapter workflow must provide code that handles getting liquidatable accounts; the example file for the TokenVault dummy contract is found in `/protocols`. Some common types are defined in `utils/types_liquidation_adapter.py`, and standard functions for accessing Pyth Hermes prices can be found in `utils/pyth_prices.py`. The exact interface of the methods in the protocol's file is not important, but it should have a similar `main()` interface: the same command line arguments and general behavior of sending liquidatable vaults to the Beacon server when specified.
+
+The party that runs the beacon can run the protocol-provided file to get and surface liquidatable accounts to the Beacon server.

beacon/protocols/beacon_TokenVault.py

@@ -2,15 +2,23 @@
 from eth_abi import encode
 import json
 from typing import TypedDict
+import argparse
+import logging
+import asyncio
+import httpx
 
-from beacon.utils.pyth_prices import *
-from beacon.utils.types_liquidation_adapter import *
+from beacon.utils.pyth_prices import PriceFeedClient, PriceFeed
+from beacon.utils.types_liquidation_adapter import LiquidationOpportunity
 
 TOKEN_VAULT_ADDRESS = "0x72A22FfcAfa6684d4EE449620270ac05afE963d0"
-CHAIN_RPC_ENDPOINT = "http://localhost:8545"
 
 
-class LiquidationAccount(TypedDict):
+class ProtocolAccount(TypedDict):
+    """
+    ProtocolAccount is a TypedDict that represents an account/vault in the protocol.
+
+    This class contains all the relevant information about a vault/account on this protocol that is necessary for identifying whether it is eligible for liquidation and constructing a LiquidationOpportunity object.
+    """
     account_number: int
     token_address_collateral: str
     token_address_debt: str
@@ -30,15 +38,17 @@ def get_vault_abi():
     return data['abi']
 
 
-"""
-get_accounts() is the first method that the protocol should implement. It should take no arguments and return all the open accounts in the protocol in the form of a list of objects of type LiquidationAccount (defined above). Each LiquidationAccount object represents an account/vault in the protocol.
-This function can be implemented in any way, but it should be able to return all the open accounts in the protocol. For some protocols, this may be easily doable by just querying on-chain state; however, most protocols will likely need to maintain or access an off-chain indexer to get the list of all open accounts.
-"""
-
+async def get_accounts(rpc_url: str) -> list[ProtocolAccount]:
+    """
+    Returns all the open accounts in the protocol in the form of a list of type ProtocolAccount.
 
-async def get_accounts() -> list[LiquidationAccount]:
+    Args:
+        rpc_url (str): The RPC URL of the chain
+    Returns:
+        List of objects of type ProtocolAccount (defined above). Each ProtocolAccount object represents an account/vault in the protocol.
+    """
     abi = get_vault_abi()
-    w3 = web3.AsyncWeb3(web3.AsyncHTTPProvider(CHAIN_RPC_ENDPOINT))
+    w3 = web3.AsyncWeb3(web3.AsyncHTTPProvider(rpc_url))
     token_vault = w3.eth.contract(
         address=TOKEN_VAULT_ADDRESS,
         abi=abi)
@@ -64,7 +74,7 @@ async def get_accounts() -> list[LiquidationAccount]:
                 16) == 0:
             done = True
         else:
-            account: LiquidationAccount = {
+            account: ProtocolAccount = {
                 "account_number": account_number,
                 "token_address_collateral": vault_dict['tokenCollateral'],
                 "token_id_collateral": vault_dict['tokenIDCollateral'].hex(),
@@ -81,8 +91,18 @@ async def get_accounts() -> list[LiquidationAccount]:
 
 
 def create_liquidation_opp(
-    account: LiquidationAccount, prices: list[PriceFeed]
-) -> LiquidationOpportunity:
+        account: ProtocolAccount,
+        prices: list[PriceFeed]) -> LiquidationOpportunity:
+    """
+    Constructs a LiquidationOpportunity object from a ProtocolAccount object and a set of relevant Pyth PriceFeeds.
+
+    Args:
+        account: A ProtocolAccount object, representing an account/vault in the protocol.
+        prices: A list of PriceFeed objects, representing the relevant Pyth price feeds for the tokens in the ProtocolAccount object.
+    Returns:
+        A LiquidationOpportunity object corresponding to the specified account.
+    """
+
     # [bytes.fromhex(update['vaa']) for update in prices] ## TODO: uncomment this, to add back price updates
     price_updates = []
     function_signature = web3.Web3.solidity_keccak(
@@ -104,17 +124,16 @@ def create_liquidation_opp(
         "contract": TOKEN_VAULT_ADDRESS,
         "calldata": calldata,
         "permission_key": permission,
-        "account": str(account["account_number"]),
         "repay_tokens": [
             {
                 "contract": account["token_address_debt"],
-                "amount": str(account["amount_debt"]),
+                "amount": hex(account["amount_debt"]).replace("0x", ""),
             }
         ],
         "receipt_tokens": [
             {
                 "contract": account["token_address_collateral"],
-                "amount": str(account["amount_collateral"]),
+                "amount": hex(account["amount_collateral"]).replace("0x", ""),
             }
         ]
     }
@@ -128,22 +147,31 @@ def create_liquidation_opp(
     return opp
 
 
-"""
-get_liquidatable(accounts, prices) is the second method that the protocol should implement. It should take two arguments: account--a list of Account (defined above) objects--and prices--a dictionary of Pyth prices.
-accounts should be the list of all open accounts in the protocol (i.e. the output of get_accounts()).
-prices should be a dictionary of Pyth prices, where the keys are Pyth feed IDs and the values are PriceFeed objects. prices can be retrieved from the provided price retrieval functions.
-This function should return a lists of liquidation opportunities. Each opportunity should be of the form LiquidationOpportunity defined above.
-"""
+def get_liquidatable(accounts: list[ProtocolAccount],
+                     prices: dict[str,
+                                  PriceFeed]) -> (list[LiquidationOpportunity]):
+    """
+    Filters list of ProtocolAccount types to return a list of LiquidationOpportunity types.
 
+    Args:
+        accounts: A list of ProtocolAccount objects, representing all the open accounts in the protocol.
+        prices: A dictionary of Pyth price feeds, where the keys are Pyth feed IDs and the values are PriceFeed objects.
+    Returns:
+        A list of LiquidationOpportunity objects, one per account that is eligible for liquidation.
+    """
 
-def get_liquidatable(
-    accounts: list[LiquidationAccount], prices: dict[str, PriceFeed]
-) -> list[LiquidationOpportunity]:
     liquidatable = []
 
     for account in accounts:
-        price_collateral = prices[account["token_id_collateral"]]
-        price_debt = prices[account["token_id_debt"]]
+        price_collateral = prices.get(account["token_id_collateral"])
+        if price_collateral is None:
+            raise Exception(
+                f"Price for collateral token {account['token_id_collateral']} not found")
+
+        price_debt = prices.get(account["token_id_debt"])
+        if price_debt is None:
+            raise Exception(
+                f"Price for debt token {account['token_id_debt']} not found")
 
         value_collateral = (
             int(price_collateral["price"]["price"]) *
@@ -164,23 +192,66 @@ def get_liquidatable(
 
 
 async def main():
-    # get all accounts
-    accounts = await get_accounts()
-
-    # get prices
-    pyth_price_feed_ids = await get_price_feed_ids()
-    pyth_prices_latest = []
-    i = 0
-    cntr = 100
-    while len(pyth_price_feed_ids[i:i + cntr]) > 0:
-        pyth_prices_latest += await get_pyth_prices_latest(pyth_price_feed_ids[i:i + cntr])
-        i += cntr
-    pyth_prices_latest = dict(pyth_prices_latest)
-
-    # get liquidatable accounts
-    liquidatable = get_liquidatable(accounts, pyth_prices_latest)
-
-    print(liquidatable)
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--operator-api-key", type=str, required=True,
+                        help="Operator API key, used to authenticate the surface post request")
+    parser.add_argument("--rpc-url", type=str, required=True,
+                        help="Chain RPC endpoint, used to fetch on-chain data via get_accounts")
+    group = parser.add_mutually_exclusive_group(required=True)
+    group.add_argument("--dry-run", action="store_false", dest="send_beacon",
+                       help="If provided, will not send liquidation opportunities to the beacon server")
+    group.add_argument("--beacon-server-url", type=str,
+                       help="Beacon server endpoint; if provided, will send liquidation opportunities to the beacon server")
+
+    parser.add_argument("--log-file", type=str,
+                        help="Path of log file where to save log statements; if not provided, will print to stdout")
+    args = parser.parse_args()
+
+    if args.log_file:
+        logging.basicConfig(filename=args.log_file, level=logging.INFO)
+    else:
+        logging.basicConfig(level=logging.INFO)
+
+    feed_ids = ["ff61491a931112ddf1bd8147cd1b641375f79f5825126d665480874634fd0ace",
+                "e62df6c8b4a85fe1a67db44dc12de5db330f7ac66b72dc658afedf0f4a415b43"]  # TODO: should this be automated rather than hardcoded?
+    price_feed_client = PriceFeedClient(feed_ids)
+
+    # TODO: sometimes the ws doesn't pull prices, understand why
+    ws_call = price_feed_client.ws_pyth_prices()
+    asyncio.create_task(ws_call)
+
+    client = httpx.AsyncClient()
+
+    await asyncio.sleep(2)
+
+    while True:
+        accounts = await get_accounts(args.rpc_url)
+
+        accounts_liquidatable = get_liquidatable(
+            accounts, price_feed_client.prices_dict)
+
+        if args.send_beacon:
+            for account_liquidatable in accounts_liquidatable:
+                resp = await client.post(
+                    args.beacon_server_url,
+                    json=account_liquidatable
+                )
+                if resp.status_code == 422:
+                    logging.error(
+                        "Invalid request body format, should provide a list of LiquidationOpportunity")
+                elif resp.status_code == 404:
+                    logging.error(
+                        "Provided beacon server endpoint url not found")
+                elif resp.status_code == 405:
+                    logging.error(
+                        "Provided beacon server endpoint url does not support POST requests")
+                else:
+                    logging.info(f"Response, post to beacon: {resp.text}")
+        else:
+            logging.info(
+                f"List of liquidatable accounts:\n{accounts_liquidatable}")
+
+        await asyncio.sleep(2)
 
 if __name__ == "__main__":
     asyncio.run(main())

beacon/searcher/README.md

@@ -0,0 +1,10 @@
+# Searcher
+
+Searchers can integrate with Express Relay by one of two means:
+
+1. LiquidationAdapter Contract interaction from an EOA
+2. Bespoke integration via a deployed contract
+
+Option 2 requires bespoke work to handle individual protocol interfaces and smart contract risk, and it is similar in nature to how many searchers currently do liquidations via their own deployed contracts--searchers can now call into their smart contracts via the Express Relay workflow. This option allows for greater customization by the searcher, but requires bespoke work per protocol that the searcher wants to integrate with.
+
+Meanwhile, option 1 requires much less bespoke work and does not require contract deployment by the searcher. For option 1, the searcher submits liquidation transactions to the LiquidationAdapter contract, which handles routing the liquidation logic to the protocol and also performs some basic safety checks to ensure that the searcher is paying and receiving the appropriate amounts. The searcher can submit transactions signed by their EOA that has custody of the tokens they wish to repay with. Searchers can listen to liquidation opportunities at the Beacon server, and if they wish to submit a liquidation transaction through Express Relay, they can submit it to the auction server endpoint. `searcher_template.py` contains a template for the actions that a searcher may wish to perform, namely getting and assessing opportunities at the Beacon server and constructing and sending a liquidation. Helper functions related to constructing the signature for the LiquidationAdapter contract are in `searcher_utils.py`. A sample workflow is in `searcherA.py` (note: this example lacks any serious evaluation of opportunities, and it simply carries out a liquidation if the opportunity is available).