Tech blog scrapper (Go)

Features

---

• Scrapes Stripe, Uber & Netflix tech blogs
• Shows all the articles from these blogs at one place

Usage

---

• dev (air):

go mod tidy
make templ-generate
make tailwind-build
make dev

• dev (with docker):

docker compose up --build

Dependencies

---

• Rod - for chrome browser based scrapping
• templ - for html templates
• htmx - for dynamic html

Folder Structure

---

└── 📁tech-blog-scrapper
    └── Makefile         #contains all the scripts to run the app
    └── 📁cmd
        └── 📁scrapper
            └── main.go  #app entrypoint
    └── 📁internal
        └── 📁handlers  # api handlers
        └── 📁scrapper  # scrapper logic
        └── 📁utils     # utility
        └── 📁view      # html templates (templ)
    └── 📁static
        └── 📁css       # contains custom & tailwindcss built css
        └── 📁script    # htmx & custom js scripts

---

manishmandal.com • GitHub @manishMandal02 • X @manishMandalJ