Zach Taiji's Wiki

Home

❯

pages

❯

Brandwatch & Social Listening Data ETL Process

Brandwatch & Social Listening Data ETL Process

Nov 17, 20251 min read

  • guide
  • data
  • data-wrangling
  • database
  • selfhosted
  • serveradmin
  • analytics

banner

Todos

  • Fix Quartz - 20 mins tops ✅ 2025-11-18
  • Setup Minio on p10 server
  • Setup postgres on p10 server with adminer - separate container from shlink
  • Follow rest of chatgpt guide

Terminology and Definitions

  • API pagination: Process of splitting a large dataset into smaller, sequential chunks so clients can request it page by page instead of in a single API call, improving performance and stability.
  • API Polling: Refers to a technique where a client repeatedly requests data from a server at regular intervals to check for updates or changes. 1

Resources

Json

  • json

n8n

  • n8n documentation
  • Luxon JS datetime formatting

Brandwatch

  • Brandwatch Consumer Research API documentation
  • Brandwatch polling for mentions
  • Brandwatch paging through historical mentions

Databases

  • Postgres cheat sheet

Footnotes

Footnotes

  1. https://medium.com/@sankalpa115/what-is-polling-b1ff70e87001 ↩


Graph View

  • Todos
  • Terminology and Definitions
  • Resources
  • Json
  • n8n
  • Brandwatch
  • Databases
  • Footnotes

Backlinks

  • Resources & Wiki & Bookmarks

Created with Quartz v4.5.2 © 2025

  • Blog