Data Engineer - AI

This is a remote position.

Job Summary

We are seeking a skilledData Engineerto support enterprise AI and Generative AI initiatives by building scalable, secure, and AI-ready data platforms. The ideal candidate will have strong expertise inAzure Databricks, Snowflake, Python, and SQL, along with experience integrating modern Azure AI services into enterprise data ecosystems.

In this role, you will partner with Data Engineers, AI Engineers, Data Scientists, and business stakeholders to develop high-quality data pipelines, enable Retrieval-Augmented Generation (RAG) solutions, and prepare enterprise data for AI-powered applications using Microsoft's AI ecosystem.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines usingAzure Databricks, Snowflake, and Azure Data Factory.
• Build ELT/ETL pipelines to ingest data from ERP, CRM, manufacturing, supply chain, and other enterprise applications.
• Develop AI-ready datasets that support machine learning and Generative AI use cases.
• Integrate structured, semi-structured, and unstructured data into centralized data platforms.
• Support Retrieval-Augmented Generation (RAG) solutions by preparing document repositories, metadata, embeddings, and search indexes.
• Collaborate with AI Engineers to expose enterprise data securely to LLM-powered applications.
• Optimize data models, transformations, and query performance for analytics and AI workloads.
• Implement data quality, governance, lineage, monitoring, and security best practices.
• Build reusable data transformation frameworks using Python, SQL, Spark, and dbt (if applicable).
• Develop REST API integrations to support AI services and enterprise applications.
• Participate in architecture discussions, code reviews, CI/CD implementations, and Agile ceremonies.
• Support production deployments, monitoring, troubleshooting, and performance tuning.

Required Qualifications

• 6+ years of experience in Data Engineering.
• Strong hands-on experience withAzure DatabricksandSnowflake.
• Proficiency inPython,SQL, andPySpark.
• Experience building scalable ELT/ETL pipelines using Azure Data Factory or similar orchestration tools.
• Strong understanding of data warehousing, dimensional modeling, and data lake architectures.
• Experience working with REST APIs and integrating cloud-based services.
• Familiarity with Git, Azure DevOps, and CI/CD pipelines.
• Experience working in Agile/Scrum environments.

AI & Azure AI Experience (Required)

• Experience supporting enterprise AI or Generative AI initiatives.
• Working knowledge ofAzure Machine Learning (Azure ML).
• Experience withMicrosoft AI Foundry (Azure AI Foundry)for AI solution development and orchestration.
• Experience integratingAzure OpenAI Serviceinto enterprise applications.
• Knowledge ofAzure AI Searchfor enterprise search and Retrieval-Augmented Generation (RAG) solutions.
• Exposure toMicrosoft Copilot Studiofor developing AI-powered copilots and conversational experiences.
• Understanding of vector search, embeddings, prompt engineering, and LLM integration.
• Experience preparing enterprise data for AI model training, inference, and knowledge retrieval.

Originally posted on Himalayas