Data Infrastructure Engineer

Speedata is modernizing analytics infrastructure with the first purpose-built ASIC processor, the Analytics Processing Unit (APU), for analytics and AI data workloads. Delivering up to 100x faster Apache Spark performance while cutting infrastructure TCO by 90%, the APU executes analytics operations directly in silicon with seamless integration and no code changes. In this role, you will be responsible for building data infrastructure for performance tuning, and analysis of a new analytics product. As a member of our Data Performance team, you will collaborate closely with other SW & HW Engineers to improve the performance of our new cutting-edge APUs. You will collaborate with our business leaders and participate in the product design and implementation. You will evaluate, implement, and integrate a diverse set of tools and technologies such as Spark, Comet, Velox, Dagster, Hardware emulators and software simulators. Requirements: BS degree in Computer Science 5+ years of hands-on software engineering experience. 5+ year experience in a Data Engineering role 2+ year experience of SQL Tuning and Data Pipeline In depth knowledge of SQL Experience with data pipelines, orchestration, or large-scale data processing Strong debugging and problem-solving skills across application, infrastructure, and data layers Ability to investigate performance regressions using metrics, logs, profiling, and controlled experiments Strongly preferred Hands-on experience with Apache Spark Experience with distributed compute or query engines Experience with Kubernetes and containerized environments Experience with Airflow, Dagster, or similar orchestration systems Experience with performance benchmarking and regression analysis Experience with query execution plans, partitioning, joins, skew, shuffle, I/O, CPU and memory bottlenecks Advantages Experience with SQL benchmarks, specifically TPC-H and TPC-DS Experience with GPUs or other hardware accelerators Responsibilities: Join a multi-disciplinary team of experts operating at the foundation of Speedata’s data infrastructure stack Build a robust data benchmarking infrastructure Develop in-house measurements and ETL tools Conduct detailed performance analysis and characterization Research state-of-the-art benchmarks and techniques for SQL and data optimization Show more Show less