Data Engineer

Brooksource

Fortune 500 Oil & Gas Client

Houston, TX

Overview

We are partnering with a rapidly growing data and AI team within a leading energy organization that is building out its core data engineering capabilities. This team serves as a centralized R&D function, enabling enterprise-wide analytics and AI initiatives by transforming complex, fragmented data into scalable, production-ready assets.

This role sits within a small but expanding pod led by a Data Science leader and works at the intersection of Data Engineering, Data Science, and AI. The team is focused on high-impact, reusable use cases that drive value across business units—removing the need for siloed development and accelerating innovation.

They are looking for a hands-on Data Engineer who can bring structure to evolving data environments, build scalable pipelines, and partner closely with both technical and business stakeholders. This is an opportunity to shape foundational data capabilities in a high-visibility, high-growth environment.

Key Responsibilities

• Design, build, and maintain scalable data pipelines using Azure and Databricks (ADF, Spark, Delta Lake).
• Develop robust ETL/ELT processes to ingest and transform structured, semi-structured, and unstructured data.
• Implement and optimize data architectures (e.g., Medallion/Bronze-Silver-Gold) to support analytics and AI use cases.
• Write high-performance data processing jobs using PySpark and Spark SQL.
• Collaborate with Data Scientists and AI Engineers to enable model development and production use cases.
• Translate ambiguous or evolving business needs into scalable, maintainable data solutions.
• Improve data reliability, quality, and performance across pipelines and platforms.
• Contribute to CI/CD and DevOps practices for data engineering workflows.
• Support development of reusable frameworks and proof-of-concept solutions for enterprise-wide adoption.

Required Qualifications

• 3+ years of experience in Data Engineering or a related role.
• Strong hands-on experience with Databricks and Azure Cloud (preferred; other cloud platforms considered).
• Proficiency in PySpark, Spark SQL, Python, and SQL.
• Experience building and maintaining ETL/ELT pipelines in modern data platforms.
• Solid understanding of distributed data processing, data modeling, and data architecture.
• Ability to operate effectively in less-defined environments and bring structure to evolving data challenges.
• Strong collaboration and communication skills across technical and non-technical teams.

Nice to Have

• Experience in Oil & Gas, energy, or manufacturing environments.
• Exposure to CI/CD, DevOps, and automated deployment practices.
• Familiarity with Data Science concepts (ML models, statistics, feature engineering).
• Experience supporting or integrating with AI/ML workflows.

Team & Environment

• Reports into a Data Science-led organization that partners closely with an adjacent AI Engineering group.
• Functions as a centralized capability supporting multiple business units.
• Small, pod-based team structure with significant growth planned.
• High visibility work tied directly to enterprise AI and digital transformation initiatives.