Why we built Samskara

Most data teams we’ve talked to are stuck between two bad options.

On one end: a folder of notebooks and cron jobs, held together by whoever wrote them last. No catalog, no lineage, no real scheduling — just scripts that happen to still work. It’s fine until the person who wrote them leaves, or until “just re-run it manually” stops being an acceptable answer.

On the other end: a full cloud data platform. Spark clusters, a managed lakehouse, a pricing page with “contact sales” on it. It solves the scripts problem completely — and it’s built for a scale most teams in the middle haven’t hit yet. A 20-person team doesn’t need the same infrastructure as a thousand-person one, but that’s often the only “grown-up” option on the table.

We kept meeting teams who’d outgrown the first option but had no good reason to take on the second. That gap is what we built SAMSKARA for.

Starting from execution, not infrastructure

We didn’t start by picking a lakehouse format or designing a catalog. We started with the thing a data engineer actually does most: write code, run it, see what happened. So the first thing we built was notebook execution — per-cell run/stop, isolated sessions, full artifact traces of every run.

Everything after that got added because the notebook alone wasn’t enough on its own, not because we were building a feature checklist:

• People needed to query data without leaving the workspace, so SQL Workbench happened.
• People needed things to run without a human clicking “run” every morning, so the Scheduler happened.
• People needed a real table format with history instead of a folder of CSVs, so ArrowLake happened.
• People needed to see the output of all that work without exporting to yet another BI tool, so Dashboards happened.
• People needed the AI to actually know the platform’s own APIs instead of guessing, so AI Sparkle happened.
• People needed to bring more than one teammate in without losing control, so governance and RBAC happened.

None of it exists because it was on a roadmap slide. It exists because the previous piece exposed a real gap.

The constraint we didn’t compromise on

Every one of those pieces had to run without asking a team to stand up new infrastructure. No Spark cluster to size. No separate metastore. No managed service with its own bill. If a team can run Docker Compose, they can run SAMSKARA — on their own storage, their own databases, their own network.

That’s the actual thesis: the self-hosted data engineering platform for teams that have outgrown scripts but don’t need the complexity and cost of a large cloud data platform.

Where we are now

SAMSKARA has been running real end-to-end projects — weather data, public health datasets, a rebuilt Northwind pipeline — for months now. That’s enough to stop calling it an idea and start putting it in front of people outside our own team.

We’re opening up early access starting with this website. If the gap we just described sounds familiar, book a demo and we’ll get you running against your own data.