Data Engineer

Role: Data Engineer

Location: Jakarta

Position Type: Full-time

Reports to: CTO

About AME Group

AME Group is a leading provider of consulting and research services to the Metals & Mining and Coal & Consumable Fuels industries. Drawing on more than 50 years of industry expertise, we deliver bespoke consulting engagements, subscription research, and software-as-a-service products to clients globally.

About the Role

We are looking for a Data Engineer to design, implement, and maintain the reliable data pipelines that bring data into AME’s research and software products. You will collect and process data from a range of public APIs into a centralized blob storage and database environment, ensuring it is ingested efficiently, transformed cleanly, and made accessible for downstream analysis and reporting.

This role works hands-on with Python-based ETL pipelines, workflow orchestration in Apache Airflow (our data orchestration platform), and cloud storage on Azure. You will work closely with our research team, data scientists, and software developers to make sure the data they depend on is available, accurate, and usable.

A significant part of this role involves managing geospatial data, so you will also work with GIS datasets in Microsoft SQL Server and ArcGIS Server alongside the core pipeline work.

A Data Engineer is primarily responsible for:

• Designing and maintaining the ETL/ELT pipelines that ingest data from public APIs into blob storage and relational databases

• Keeping those pipelines reliable through monitoring, logging, and validation

• Structuring and transforming data so it is ready for downstream analysis, reporting, and web applications

• Managing and maintaining geospatial (GIS) datasets within our SQL Server and ArcGIS Server environment

• Documenting data sources, transformations, and orchestration logic clearly

Key Responsibilities

• Design and implement robust ETL/ELT pipelines to extract data from multiple public APIs and load it into blob storage and relational databases

• Build and manage scheduled workflows in Apache Airflow, our data orchestration platform

• Handle complex API integrations, including authentication flows (OAuth2, API keys), pagination, and rate limiting

• Structure data in blob storage (e.g. Azure Blob) for efficient downstream access

• Transform and load data into structured SQL databases for analytics and access by web applications

• Manage and maintain geospatial (GIS) data in Microsoft SQL Server and ArcGIS Server, ensuring it is accurate and accessible to downstream applications

• Implement monitoring, logging, and validation to ensure data integrity and pipeline reliability

• Work closely with data scientists and software developers to ensure data availability and usability

• Maintain clear documentation for data sources, transformations, and orchestration logic

Qualification and Experience

This is a hands-on, production-focused data engineering role. You will spend your time building and maintaining real pipelines that the business depends on, working directly with code, APIs, and cloud infrastructure rather than through managed, no-code tooling. You do not need a background in our industry. However, candidates looking for a purely analytical or reporting role, or who are not comfortable writing production code and working with version control and the command line, will not be progressed.

Required

• Bachelor’s degree in Computer Science, Data Engineering, or a related field (or equivalent practical experience)

• 3+ years building data pipelines in a production environment

• Strong understanding of ETL/ELT architecture, API data ingestion, and data modeling principles

• Hands-on experience with Azure

• Experience working with geospatial/GIS data in Microsoft SQL Server and ArcGIS Server

• Clear communication, including explaining technical concepts to non-technical stakeholders

• A naturally analytical, detail-oriented mindset with a proactive approach to monitoring and data accuracy

Highly Desirable

• Experience integrating data from open economic and financial APIs (e.g. World Bank, IMF, OECD, UN Data, FRED)

• Familiarity with large structured and semi-structured datasets (JSON, CSV, Parquet) and associated performance considerations

• Experience building incremental ingestion or change data capture (CDC) pipelines

• An interest in, or exposure to, the resources, commodities, or research data domain

Technical Skills

• Languages: Python and SQL (Microsoft T-SQL)

• Orchestration: Apache Airflow

• Geospatial: GIS data in Microsoft SQL Server and ArcGIS Server

• RESTful API integration (authentication, pagination, error handling)

• Azure Blob Storage

• Familiarity with version control (Git) and CI/CD pipelines

• Fluent written and spoken English (essential)

What We Offer

• Modern, central office near to public transport and key amenities

• Competitive salary

• Health Care

• Lunch

• 13 months salary