Migrating from Tableau to dbt on Redshift: What Worked, What Didn’t, and What I Wish I’d Known

Getting data out of DynamoDB, through Redshift, and into something your business can actually use — without losing your mind in the process.

I spent the last couple of years building out a data engineering function from scratch at a small SaaS company. We had a Tableau-based reporting pipeline that involved manual runs, fragile prep flows, and a mounting sense of dread every time someone asked for a new report. I replaced it with dbt, Redshift, and Fivetran, with self-hosted Metabase for business-intelligence, and with a detour through several tools that didn’t work out. Here’s what I learned.

I use LLMs daily. Claude Code is my daily-driver for engineering work and it’s in my IDE terminal constantly. I use Claude Projects for thinking through problems and working through complex decisions. Google’s NotebookLM has become my tool of choice for collecting and summarising research. I use Gemini for image generation. I’ve tried Cursor and Windsurf for IDE-integrated AI, and tend to reach for Cursor on personal projects alongside Claude Code. I’ve had several looks at ChatGPT over the past year and keep finding it less compelling each time.

Warning: This post may contain opinions.

I’ve recently got interested in some of the new developments in web development, and in particular, the htmx library. This is a library that allows you to add dynamic behaviour to your web pages, without having to write any Javascript. It feels to the spirit of the web I knew and loved back in the 90s, which was all about hypermedia, and the ability to link between documents.