Data Engineer

Job Description:

• Data Pipeline Development: Design, build, and maintain scalable and robust data pipelines (ETL/ELT) to ingest, clean, and transform data from multiple sources.
• Data Integration & Architecture: Manage multi-database synchronization strategies, data migrations, and contribute to the structuring of data warehouses or data lakes.
• Data Quality & Reliability: Ensure data integrity, availability, and security across the data infrastructure. Debug scripts, resolve data conflicts, and troubleshoot pipeline failures.
• Infrastructure Monitoring: Utilize the ELK stack (Elasticsearch, Logstash, Kibana) or similar tools for log and event data management to monitor pipeline health and performance.
• Cross-Functional Collaboration: Work closely with software developers, analysts, and Business Intelligence teams to provide reliable, optimized data that drives strategic business decisions.

Requirements:

• Data Engineering & ETL Savvy: Proven experience building data pipelines and working with orchestration/ETL tools (e.g., Apache Airflow, SSIS, Informatica).
• Expert in SQL & Programming: Extensive experience with complex SQL queries and relational database design. Strong proficiency in programming languages commonly used in data engineering (Python or Golang).
• Hands-On Database Experience: Solid experience working with relational databases such as PostgreSQL, MySQL, and MSSQL Server. Experience with Data Warehouse architecture is highly preferred. Proven experience in designing or building a Data Lakehouse is a strong plus.
• Streaming & Event Processing: Familiarity with message brokers and streaming platforms like Apache Kafka.
• Cloud & Monitoring Tools: Knowledge of the ELK stack. Experience with DBaaS (Database as a Service) on Cloud platforms (AWS, GCP, or Azure) is a strong plus.
• Work Arrangement: Willing to Work From Office (WFO - BSD CITY).