To write an "autistic" Gall agent, one that just processes data and doesn't talk to other agents, you can just write the program and be done with it. However, if you intend your agent's data to be used by other agents, you need to architect them a bit more carefully.
Many Gall agents use a store/hook/view architecture to separate functionality and protect data. In this lesson, you will learn to read programs that use this architecture so that you can call them to access data on your Urbit.
- stores are like databases. They generally should only be accessed by agents on your ship.
- hooks act as a permissions layer in front of stores, both for requesting and returning data. They determine what outside ships are allowed to access.
- views join together data from multiple stores, and also process actions from the outside world (e.g. when someone clicks a button in a UI).
The focus of this lesson is on reading existing code. By the end of the lesson and its exercises, you will be comfortable understanding and querying Gall agents that use this pattern.
We'll refer to the September 3, 2020 version of the files below:
- app/chat-hook.hoon
- app/group-push-hook.hoon
- app/group-pull-hook.hoon
- lib/pull-hook.hoon
- lib/push-hook.hoon
- app/chat-view.hoon
- sur/chat-view.hoon
- lib/chat-view.hoon
- lib/chat-store.hoon
Stores are databases that are intended to a. store data for a given application (like chat or groups) b. make that data available to other agents locally, and selected agents globally
Stores impose read and write access restrictions in three places.
The first is on-poke, as seen there for chat-store. This restricts "write" operations to the store.
Second, the same team:title restriction is used in on-watch, which restricts "read" operations.
Finally, on-peek's scry reads are only accessible from the calling ship, not from remote. Data access restrictions can be further enforced by limiting what on-peek makes available locally.
Urbits get authentication ("who are you?") for free with ship ids, but need a way to handle authorization ("I know who you are, but what are you allowed to access?"). Hooks are the means by which this is achieved.
Hooks can be a bit tricky to grok, so an analogy is helpful. Imagine that every data store is a king, and the king does not want to deal with impositions on his time and attention. So he delegates his evil vizier to handle all interactions with the outside world for him, and refuses to talk to anyone but his vizier.
If King1 wants information from King2, he asks his Vizier1 to get it. Vizier1 contacts his worm-tongued counterpart, Vizier2, with either a poke or subscription request.
- poke: Vizier2 checks whether Vizier1 is allowed to poke King2, and pokes King2 on Vizier1's behalf if he is.
- subscription: Vizier2 checks whether Vizier1 is allowed to subscribe to King2. If he is, Vizier2 listens for any updates from King2, passes them to Vizier1, who in turn informs King1 of these goings-on.
Using this structure, chats and groups can mirror the information held by other chats and groups, while separating data storage from permissions and authorization.
Let's look at some concrete examples of hooks requesting and receiving data.
In line 480 of chat-hook.hoon, we see the handling of a %message action, for when a new message is entered in chat.
There are 2 cases here, local and foreign:
- local: we created this
%messageand are poking%chat-hookfrom our own ship. This means that either
- we own the chat, in which case we want to send the message to our own
chat-store - the chat is on another ship, and we need to poke that ship's
chat-hookto add it to their store The?:that checks whether it's our chat or someone else's is in line 494.
- foreign: someone else created the
%messageis hitting our%chat-hookfrom another ship'schat-hook, and we need to add it to our%chat-storeiff that ship is in the chat's group, and* we are the chat owner.
- In line 499 we check that we own the chat (
syncedis a map of chat-path -> chat-owning-ship) - In line 501 we check that the sender is in the group
Note: in the foreign case, we do not add the %message to our store if we are not the chat's owner. That mirroring is handled via subscriptions, not pokes.
When you're lurking in chat and a message is posted (or if you're super-active but, horror of horrors, you are not the poster), chat-store sends that message out to all chat members so that they can mirror it in their local chat-stores.
However, we saw in line 103 of chat-store.hoon that only local agents can subsciribe to chat-store. So the mirroring flow looks as follows:
- the chat owner's
chat-storereceives a poke from hischat-hookvizier and stores the new message. - that
chat-storesends an update to its local subscribers, one of which is thechat-hookvizier (line 111 ofchat-store.hoon) - the vizier forwards it along to all of its subscribers: the other
chat-hooks that are members of that chat.
Step 3 begins in line 374 of chat-hook.hoon, which then calls fact-check-update to see whether the on-agent call was local or foreign.
- local: our own ship produced the
on-agentevent, which means we need to send out an update to ourchat-hookvizier peers. In this case, we%givethe update to subscribers as in line 663. - foreign: another
chat-hookis sending us an update, so we need to mirror it in ourchat-store. In line 699 we ensure that the otherchat-hookis on the chat owner's ship, and then poke ourchat-storeking with the incoming message, so that he can mirror it.
In the above examples, a pattern of "check whether this event is from our local king or from a foreign vizier" kept re-occuring. That pattern can be abstracted out into push and pull hooks, as is currently the case for group-store's hooks.
pushhook- handles all incoming pokes and checks whether they should be passed to the store
- subscribes to the store and pushes its changes out to its remote hook partners
pullhook
We can go even further and make "agent generators" for the push and pull hook pattern.
These libraries are similar to lib/dbug.hoon: they are generators that take samples and create full Gall agents from them.
The way to use these is
- Someone creates a Gall agent, e.g.
app/group-push-hook.hoon - Set the config to pass to the
push-hookgenerator - Call a gate in the generator, passing it the config and a Gall agent with specific extra arms. The generator uses those extra arms to return a new Gall agent with custom behavior.
Let's dive right into an example.
In app/group-push-hook.hoon, we see a config defined in line 13. This config's form is from line 6 of lib/push-hook.hoon, and requires us to define:
$: store-name=term :: store agent
store-path=path :: path to watch store-agent on
update=mold :: the mold of updates the store sends
update-mark=term :: the mark of updates the store sends
pull-hook-name=term :: the pull-hook for this store
==
In line 19, push-hook defines a type: a door with all the Gall arms, plus 4 additional arms:
resource-for-update: figure out from an update which resource it affects (e.g. which group), so that we can send updates to those subscribed to that resource (probablypull-hooks).take-update: handle a subscription update coming from the store. We use theupdatemold for this.should-proxy-update: takes a vase with an update from the store and returns a true/false as to whether we should pass a poke to the store. The vizier uses this gate to decide whether to pass a poke to his king, the store.- `initial-watch**: define the initial state that will be passed back when another agent subscribes to us.
When you make a new push-hook agent, you need to implement those 4 arms.
From line 31 on down, group-push-hook.hoon defines an agent with the normal Gall arms, plus those 4. It then passes them as the sample to agent:push-hook, which constructs a normal, 10-armed, Gall agent that calls out to them.
pull-hook goes in much the same way as push-hook, except that it creates an agent that watches a remote push-hook for updates, processes those updates, and mirrors them to the appropriate local store.
Its config is simpler, since it knows that push-hooks use the [%resource resource] path to send subscription updates:
$: store-name=term
update=mold
update-mark=term
push-hook-name=term
==
The pull-hook agent generator has 2 additional arms:
on-pull-nack: handle a subscription to apush-hookfailingon-pull-kick: handle being kicked from our subscription to apush-hook
Analyzing the way in which these define their custom arms will be left as an exercise for the reader.
If you want data that's in a store on your ship, you don't need to use a hook--you can just query the store directly with a poke, scry, or watch. Hooks are for intership communication.
Views are responsible for receiving information from a frontend, translating that into backend calls to other agents, and returning information to the frontend.
The first responsibility of views is to parse JSON data. Here we see code to parse JSON into chat-view actions. This lets the frontend either poke with JSON or the chat-view-action mark, and have it be useable.
To return JSON data, chat-view uses the JSON encoder for chat-store.
In sur/chat-view.hoon, we see that chat-view can create chats, delete them, etc.
The creation case is interesting, because it has to coordinate the chat-store, metadata-store, and group-store. We see this in line 224, where create-chat, create-group, and create-metadata are all called if the group referenced doesn't exist. These in turn call the relevant stores to add the data, and hooks to monitor the new resources and expose them to others if necessary.
In our HTTP lesson, we saw how you can serve data from an HTTP endpoint on your ship. And in the channels lesson, we saw how you can %give data to a subscription path to get it to the frontend. View agents often use both of these.
In line 187, chat-view listens to an HTTP endpoint and returns the contents of a mailbox to it, in JSON form.
In line 498, it passes updates from chat-store to the /primary subscription path, which is listened for in line 102 of on-watch.
Either of these methods are acceptable, and applications often use both (preferring HTTP requests for one-time calls, although that's by no means a hard-and-fast rule).
- have to evaluate the code for a new hook (or really any agent) that is installed on your ship, since it can bypass the access controls of a store.
- Outline how you would split
chat-hookinto apush/pullarchitecture. - Analyze
should-proxy-update(and other 3 arms?) ofgroup-push-hook.hoon