Senior Software Engineer

Job Description:

Responsibilities

• Work closely with Cloud product owners to understand, analyze product requirements and provide feedback.
• Architecture design and review of Cloud data pipeline, including data ingestion, extraction, and transformation services.
• Implement and enhance support tools for monitoring and acting on data pipeline issues and interpret trends and patterns.
• Technical leadership of software design in meeting requirements of service stability, reliability, scalability, and security
• Guiding technical discussions within engineer group and making technical recommendations
• Design review and code review with peer engineers
• Guiding testing architecture for large scale data ingestion and transformations.
• Customer facing engineering role in debugging and resolving field issues.

Qualification & Experience

• 5+ years of software engineering experience
• 4+ years of development experience performing ETL and/or data pipeline implementations.
• Organized and goal-focused, ability to deliver in a fast-paced environment.
• Strong understanding of distributed systems and Restful APIs.
• Experience in cloud-based big data projects (preferably deployed in GCP)
• Hands on experience implementing data pipeline infrastructure for data ingestion and transformation near real time availability of data for applications, BI analytics, and ML pipelines.
• Working knowledge of Data Lake technologies, data storage formats (Parquet, ORC, Avro) and query engines (BI Engine, Athena, Presto etc) and associated concepts for building optimized solutions at scale.
• Experience in designing data streaming and event-based data solutions (GCP Pub/Sub, Kafka, Kinesis, or like)
• Experience building data pipelines (Pub/Sub, Spark or like)
• Working experience with the cloud-based data warehouse like (BigQuery, RedShift, Azure SQL Data Warehouse, etc.)
• Experience designing cost optimized solutions for large datasets using open-source frameworks.
• Knowledge and experience designing solutions with cloud-native GCP Cloud services as well as deploying alternative solutions for appropriate use cases.
• Expert level in one of the following programming languages or similar- Python, Java.
• BS degree in Computer Science engineering, or mathematics or equivalent experience.