Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added 'Create your Zoom SDK', moved 'Usage' section, and renamed 'Ins… #1

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Added 'Create your Zoom SDK', moved 'Usage' section, and renamed 'Ins… #1

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a8d5984
Added 'Create your Zoom SDK', moved 'Usage' section, and renamed 'Ins…
Dec 11, 2023
6b002e7
Updated 'Create your Zoom SDK App'
Dec 11, 2023
813c2af
Removed Redirect URI from Prerequisites
Dec 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
125 changes: 67 additions & 58 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,32 +1,28 @@
# Meeting Bot Starter Kit - Recall.ai
# Meeting Bot Starter Kit - Recall.ai

The Meeting Bot Starter Kit is a boilerplate Zoom Meeting assistant built using [Recall.ai](https://recall.ai), [Anthropic's Claude API](https://docs.anthropic.com/claude/reference/getting-started-with-the-api), and the [Zoom Apps SDK](https://developers.zoom.us/docs/zoom-apps/).
The Meeting Bot Starter Kit is a boilerplate Zoom Meeting assistant built using [Recall.ai](https://recall.ai), [Anthropic's Claude API](https://docs.anthropic.com/claude/reference/getting-started-with-the-api), and the [Zoom Apps SDK](https://developers.zoom.us/docs/zoom-apps/).

Using an automated Zoom Meeting client, it generates a transcript, requests a summary, and provides the summary back to the user in the Meeting in near real time.
Using an automated Zoom Meeting client, it generates a transcript, requests a summary, and provides the summary back to the user in the Meeting in near real time.

Use this sample to build your own Meeting assistant by customizing the Meeting participant tile, summarization, and Zoom App.
Use this sample to build your own Meeting assistant by customizing the Meeting participant tile, summarization, and Zoom App.

## Demo
## Features

[![Demo: Meeting bot starter kit with Recall.ai](https://img.youtube.com/vi/7h3rLJUMtS4/0.jpg)](https://www.youtube.com/watch?v=7h3rLJUMtS4 'Demo: Meeting bot starter kit with Recall.ai')

## Features

1. Meeting audio access and transcription using [Recall.ai](https://recall.ai) and an Express server
2. Summarization using Anthropic’s Claude API
1. Meeting audio access and transcription using [Recall.ai](https://recall.ai) and an Express server
2. Summarization using Anthropic’s Claude API
3. In-meeting app built using React and the Zoom Apps SDK

## Architecture
## Architecture

![Architectural diagram of the Meeting bot starter kit](/docs/images/architecture.png)

This Meeting Bot Starter Kit app joins a Zoom Meeting using a virtual meeting participant powered by [Recall.ai](https://recall.ai).
This Meeting Bot Starter Kit app joins a Zoom Meeting using a virtual meeting participant powered by [Recall.ai](https://recall.ai).

Recall receives local recording permission through a cloud-hosted Zoom client using the Zoom Meeting SDK and outputs a transcript of the meeting.
Recall receives local recording permission through a cloud-hosted Zoom client using the Zoom Meeting SDK and outputs a transcript of the meeting.

This transcript is passed to the Anthropic Claude API for summarization.
This transcript is passed to the Anthropic Claude API for summarization.

Summary responses are handled and passed back back to the user in the Zoom Meeting using a Zoom App, built using the Zoom Apps SDK.
Summary responses are handled and passed back back to the user in the Zoom Meeting using a Zoom App, built using the Zoom Apps SDK.

## Prerequisites

Expand All @@ -37,8 +33,11 @@ Summary responses are handled and passed back back to the user in the Zoom Meeti
1. Client ID
2. Client Secret
3. Redirect URI
5. [Recall API Token](https://recall.ai)
6. [Anthropic API Token](https://anthropic.ai)
5. [Zoom SDK Credentials]
1. Client ID
2. Client Secret
6. [Recall API Token](https://recall.ai)
7. [Anthropic API Token](https://anthropic.ai)

## Getting started

Expand All @@ -59,17 +58,28 @@ ngrok http 3001
```

### Create your Zoom App

In your web browser, navigate to the [Zoom Developer Portal](https://marketplace.zoom.us/develop/create) and register/log into your developer account.

1. Click the "Develop" button on the top right of the page and click “Build App” in the dropdown that appears.
1. Click the "Develop" button on the top right of the page and click “Build App” in the dropdown that appears.
2. Click “Create” on the "Zoom Apps" application type.
3. Name your app
4. Choose whether to list your app on the marketplace or not
5. Click "Create"


For more information, you can follow [this guide](https://dev.to/zoom/introducing-zoom-apps-33he) check out [this video series](https://www.youtube.com/playlist?list=PLKpRxBfeD1kGN-0QgQ6XtSwtxI3GQM16R) on how to create and configure these sample Zoom Apps.

### Create your Zoom SDK App
The Meeting Bot Starter Kit uses [Zoom Meeting SDK](https://developers.zoom.us/docs/meeting-sdk/) to join and record Zoom Meetings.

In the [Zoom Developer Portal](https://marketplace.zoom.us/develop/create), create a Zoom Meeting SDK app for connectivity to Zoom Service:

1. From the "Develop" drop-down choose "Build App"
2. From the options available "Create" a "Meeting SDK" app type and follow the instructions
3. In the resulting screen, click on "App Credential" from the sidebar to make note of "Client ID" and "Client Secret"

> NOTE: You will need to add these credentials to Recall.ai dashboard

### Config: App Credentials

In your terminal where you launched `ngrok`, find the `Forwarding` value and copy/paste that into the "Home URL" and "
Expand All @@ -84,25 +94,25 @@ Redirect URL for OAuth: https://xxxxx.ngrok-free.app/auth

#### OAuth allow list

- `https://example.ngrok-free.app`
- `https://example.ngrok-free.app`

#### Domain allow list

- `appssdk.zoom.us`
- `ngrok-free.app`
- `ngrok.io`
- `appssdk.zoom.us`
- `ngrok-free.app`
- `ngrok.io`

### Config: Information

The following information is required to activate your application:

- Basic Information
- App name
- Short description
- Long description (entering a short message here is fine for now)
- Developer Contact Information
- Name
- Email address
- Basic Information
- App name
- Short description
- Long description (entering a short message here is fine for now)
- Developer Contact Information
- Name
- Email address

> NOTE: if you intend to publish your application on the Zoom Apps Marketplace, more information will be required in this section before submitting.

Expand All @@ -113,16 +123,15 @@ sections:

#### APIs

- shareApp
- getRunningContext
- getAppContext
- getMeetingJoinUrl
- shareApp
- getRunningContext
- getAppContext
- getMeetingJoinUrl

### Scopes

Ensure that the following scope is selected on the Scopes tab:

- `zoomapp:inmeeting`
- `zoomapp:inmeeting`

### Config `.env`

Expand Down Expand Up @@ -150,7 +159,7 @@ BOT_NAME="[bot_name]"
```

> Note: If you are a [Zoom for Government (ZfG)](https://www.zoomgov.com/) customer you can use the `ZM_HOST` variable to change
> the base URL used for Zoom.
the base URL used for Zoom.

## Start the App

Expand All @@ -168,6 +177,12 @@ The `dev` script will:
2. Watch Server files and build to the dist/ folder
3. Start the server and webpack dev server

### Install Zoom App

To install the Zoom App, in your browser navigate to the **Home URL** that you set in your Zoom App, then click the link to install.

After you authorize the app, Zoom will automatically open the app within the client.

### Production

When running your application in production no logs are sent to the console by default and the server is not restarted
Expand All @@ -185,48 +200,42 @@ set NODE_ENV=production && npm start

## Customization: Meeting participant tile

Edit the experience of the participant joining the Meeting through configuration of the Recall API.
Edit the experience of the participant joining the Meeting through configuration of the Recall API.

**Name:** Add a recognizable name for your app to give user's context on what the participant is. To change the name, you can change the `bot_name` configured at the [following location](https://github.com/zoom/meetingbot-recall-sample/blob/d6e3920878206b435ed37eb7c95c43e70b404bba/server/routes/api.js#L60).

**Profile image:** Uncomment this section
**Profile image:** Uncomment this section

**Join meeting sound:** To play audio when the app joins a Meeting, [uncomment this section](https://github.com/zoom/meetingbot-recall-sample/blob/8d9920543d667ecb196a6fb8bd0272fcc4aa1761/server/routes/api.js#L81-L90) and replace the the text with an MP3 that you have base64 encoded.
**Join meeting sound:** To play audio when the app joins a Meeting, [uncomment this section](https://github.com/zoom/meetingbot-recall-sample/blob/8d9920543d667ecb196a6fb8bd0272fcc4aa1761/server/routes/api.js#L81-L90) and replace the the text with an MP3 that you have base64 encoded.

**Chat messages:** [Uncomment this section](https://github.com/zoom/meetingbot-recall-sample/blob/8d9920543d667ecb196a6fb8bd0272fcc4aa1761/server/routes/api.js#L91-L98) to make the app send "Hello world" to everyone in the Meeting chat. Replace the text with your own message.
**Chat messages:** [Uncomment this section](https://github.com/zoom/meetingbot-recall-sample/blob/8d9920543d667ecb196a6fb8bd0272fcc4aa1761/server/routes/api.js#L91-L98) to make the app send "Hello world" to everyone in the Meeting chat. Replace the text with your own message.

Reference the [Recall.ai API documentation](https://recallai.readme.io/reference/recall-overview) for additional customization options.
Reference the [Recall.ai API documentation](https://recallai.readme.io/reference/recall-overview) for additional customization options.

## Customization: LLM summarization

This demo app uses Anthropic’s Claude API to summarize meeting transcripts in near real time.
This demo app uses Anthropic’s Claude API to summarize meeting transcripts in near real time.

[Edit the prompt](https://github.com/zoom/meetingbot-recall-sample/blob/d6e3920878206b435ed37eb7c95c43e70b404bba/server/routes/api.js#L138-L157) sent to Claude in `/server/routes/api.js`.

Swap this out for a different LLM by [editing the API call](https://github.com/zoom/meetingbot-recall-sample/blob/d6e3920878206b435ed37eb7c95c43e70b404bba/server/routes/api.js#L194-L201) made in in the summarize route.
Swap this out for a different LLM by [editing the API call](https://github.com/zoom/meetingbot-recall-sample/blob/d6e3920878206b435ed37eb7c95c43e70b404bba/server/routes/api.js#L194-L201) made in in the summarize route.

The user is shown [prompt options](https://github.com/zoom/meetingbot-recall-sample/blob/8d9920543d667ecb196a6fb8bd0272fcc4aa1761/frontend/components/InMeeting/Summary/Summary.js#L34C4-L34C4) in the frontend that adjust the prompt sent to handle the meeting summarization done by the Claude API.
The user is shown [prompt options](https://github.com/zoom/meetingbot-recall-sample/blob/8d9920543d667ecb196a6fb8bd0272fcc4aa1761/frontend/components/InMeeting/Summary/Summary.js#L34C4-L34C4) in the frontend that adjust the prompt sent to handle the meeting summarization done by the Claude API.

## Customization: Zoom App

This sample is a basic example of a web app running inside the Zoom client.
This sample is a basic example of a web app running inside the Zoom client.

Using the [Zoom Apps SDK](https://appssdk.zoom.us/classes/ZoomSdk.ZoomSdk.html), existing web apps can be modified to run in the Zoom desktop or mobile client.

In the InMeeting component, you’ll see two important components:
In the InMeeting component, you’ll see two important components:

- [Transcript.js](https://github.com/zoom/meetingbot-recall-sample/blob/main/frontend/components/InMeeting/Transcript/Transcript.js) handles and displays a text transcript of the conversation generated by Recall using local recording access to the Meeting’s audio.
- [Summary.js](https://github.com/zoom/meetingbot-recall-sample/blob/main/frontend/components/InMeeting/Summary/Summary.js) handles a prompted response from the Claude API to display a summary to the user in the Meeting, providing prompt options to the user (ex: “Generate action items”) that [adjust the prompt](https://github.com/zoom/meetingbot-recall-sample/blob/8d9920543d667ecb196a6fb8bd0272fcc4aa1761/server/routes/api.js#L177) sent in the /api/summarize route.
* [Transcript.js](https://github.com/zoom/meetingbot-recall-sample/blob/main/frontend/components/InMeeting/Transcript/Transcript.js) handles and displays a text transcript of the conversation generated by Recall using local recording access to the Meeting’s audio.
* [Summary.js](https://github.com/zoom/meetingbot-recall-sample/blob/main/frontend/components/InMeeting/Summary/Summary.js) handles a prompted response from the Claude API to display a summary to the user in the Meeting, providing prompt options to the user (ex: “Generate action items”) that [adjust the prompt](https://github.com/zoom/meetingbot-recall-sample/blob/8d9920543d667ecb196a6fb8bd0272fcc4aa1761/server/routes/api.js#L177) sent in the /api/summarize route.

For an additional example of a Zoom App running in a Meeting, reference the [Advanced Zoom Apps Sample](https://github.com/zoom/zoomapps-advancedsample-react) and the [Zoom Apps documentation](https://developers.zoom.us/docs/zoom-apps/).
For an additional example of a Zoom App running in a Meeting, reference the [Advanced Zoom Apps Sample](https://github.com/zoom/zoomapps-advancedsample-react) and the [Zoom Apps documentation](https://developers.zoom.us/docs/zoom-apps/).

While this is one example of an in-meeting app, the implementation shown in the Transcript and Summary component can be reused in any existing or new web app running inside the Meeting.

## Usage

To install the Zoom App, Navigate to the **Home URL** that you set in your browser and click the link to install.

After you authorize the app, Zoom will automatically open the app within the client.
While this is one example of an in-meeting app, the implementation shown in the Transcript and Summary component can be reused in any existing or new web app running inside the Meeting.

### Keeping secrets secret

Expand Down