About KwikQuery

Apache Spark Performance Optimization

KwikQuery Inc. is a Portland, Oregon-based software company focused on solving real performance problems in Apache Spark — specifically the ones that don't show up in standard benchmarks but cause serious pain in production.

28+ years
Query engine engineering expertise
5
Published white papers
28+
Apache Spark JIRA issues resolved
AS

Asif Hussain Shahid — Co-Founder, CTO & Interim CEO

Co-Founder · CTO & Interim CEO · Query Engine Architect

I've spent 28+ years working on data systems — focused on query engine internals, low-latency execution, and the kind of performance work that determines whether a system holds up under real-world pressure. That journey led me to co-found KwikQuery Inc. and build TabbyDB, a high-performance fork of Apache Spark that tackles query compilation and runtime performance problems at scale.

My background is in Chemical Engineering from IIT Bombay, but I moved into software early. Much of my foundation in performance engineering was built at GemStone Systems working on GemFire — the distributed data platform later open-sourced as Apache Geode. GemFire was an environment where low latency wasn't a goal, it was the culture, and that shaped how I approach every system I've worked on since. During that time I made significant enhancements to GemFire's OQL engine — extending it to evaluate join queries using range indexes via high-performance nested loop execution, and adding colocated join and nested query support, each of which required substantial rework of the engine's core, apart from work on Disk Based Persistence and Client Server topology.

From there I worked at VMware, Dell, Pivotal, SnappyData (all as part of acquisitions of GemStone Systems via VMware), Tibco (after acquisition of SnappyData), Workday, and Cloudera — building approximate query processors, off-heap memory storage layers, and Spark optimizer improvements along the way.

At Workday, a Constraint Propagation problem in Apache Spark's Catalyst optimizer had been identified as a serious bottleneck — complex plans were taking hours to compile. I devised a new algorithm that brought that down from 8 hours to under 10 minutes, work recognized with Workday's Tech Wizard award and co-presented at the Databricks Spark Summit in 2021. At Cloudera I continued at the engine level, resolving deep performance and functionality bugs for customers including Bank of America, Amazon, and JPMC.

TabbyDB applies all of that to fix problems stock Spark leaves unsolved — compile-time blowup from constraint explosion and uncapped query trees, redundant optimizer rule application, excessive Hive Metastore calls, and runtime inefficiency from the absence of Broadcast Hash Join key pushdown for file pruning on non-partitioned columns. On a 1TB TPC-DS benchmark on Google Cloud, TabbyDB cuts total runtime by 21% on Hive Parquet tables and 22% on Apache Iceberg tables compared with stock Apache Spark.

If you're dealing with Spark performance or compilation issues, I'd be happy to talk.

TH

Taha Hussain — Co-Founder & Head of Business Development

Co-Founder · Head of Business Development

I co-founded KwikQuery to bring advances in Apache Spark performance to enterprise data teams. I lead commercial strategy, go-to-market execution, sales, partnerships, fundraising, and business operations, working alongside co-founder and CTO Asif Shahid.

I built KwikQuery’s commercial and operational foundation, including its initial pricing structure, website, outbound marketing programs, and business infrastructure. My work connects customer needs with commercial decisions—from identifying target markets and shaping product positioning to financial planning and coordinating customer evaluations.

My background spans technical program management at Daimler Trucks North America, Scrum Master experience, and ownership of established businesses. With a degree in Electrical & Computer Engineering from Oregon State University, I bring technical understanding and cross-functional coordination to KwikQuery’s commercial strategy and operations.

Asif Hussain Shahid

asif.shahid@kwikquery.com

+1 (971) 269 8126

Taha Hussain

taha.hussain@kwikquery.com

+1 (541) 908 5782

Portland, Oregon, USA · kwikquery.com