Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: ⚡️ add 'raw_discourse_categories' asset #60

Merged
merged 1 commit into from
Feb 13, 2024

Conversation

DistributedDoge
Copy link
Collaborator

@DistributedDoge DistributedDoge commented Feb 5, 2024

Creates a single asset that builds a table, where each row is a Discourse category like say "Proposals" or "Grants" on some Discourse instance. Not all Discourse instances give us listings of topics, but all of them yield stats (e.g. number of posts in category)

  • Currently scans /categories.json endpoint for Giveth, Gitcoin, Arbitrum, Optimism , more forums can be added
  • Use source column to identify Discourse from which specific category was taken.

Further work will be:

  • add dbt model
  • build downstream model that fetches topic bodies for interesting categories for each forum (e.g. fetch PROPOSALS/GRANTS from both Gitcoin and Giveth)

I think data is interesting as-is and model is stable so it should be ready to merge!

@davidgasquez davidgasquez merged commit 1d6c7a1 into davidgasquez:main Feb 13, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants