Senior Data Engineer RibbitZ

Our client RibbtZ is looking for Senior Data Engineer (Cloud Analytics) to work remotely.

Role Summary

We are seeking a highly skilled Senior Data Engineer with strong expertise in cloud-based data platforms big data processing and modern data architectures. The ideal candidate will have hands-on experience in building scalable data pipelines implementing lakehouse architectures and enabling advanced analytics and machine learning use cases across enterprise environments.

Key Responsibilities

Data Engineering & Architecture

• Design and implement scalable data pipelines (ETL/ELT) for batch and real-time processing.
• Build and maintain modern data platforms using lakehouse architecture (Bronze Silver Gold layers).
• Develop and optimize data models (star/snowflake schemas) for analytics and reporting.
• Ensure high data quality integrity and governance across systems.

Cloud & Platform Management

• Develop and deploy solutions on Microsoft Azure and/or AWS ecosystems.
• Work with services such as:
  • Azure Data Factory Azure Databricks ADLS Gen2
  • Azure SQL Key Vault Azure DevOps
  • AWS S3 Redshift EMR Glue Lambda
• Implement secure scalable and cost-efficient data storage solutions.

Big Data & Processing

• Develop large-scale data processing workflows using:
  • Apache Spark / PySpark
  • Kafka Hive Hadoop Airflow
• Optimize performance of distributed data processing systems.

Microsoft Fabric & Lakehouse (Preferred)

• Implement Microsoft Fabric-based data solutions including:
  • Lakehouse architecture
  • Medallion design (Bronze/Silver/Gold)
  • Delta Lake optimization
• Build Fabric pipelines and integrate with Power BI.

Data Integration & Migration

• Lead data migration initiatives from legacy/on-prem systems to cloud platforms.
• Integrate multiple data sources (SAP Oracle SQL Server APIs etc.).
• Implement incremental data loading and performance optimization techniques.

Analytics & BI Enablement

• Enable business intelligence and reporting using tools like:
  • Power BI SSRS Kibana Grafana
• Implement Row-Level Security (RLS) and data access controls.

Machine Learning & Advanced Analytics (Good to Have)

• Support ML pipelines using frameworks such as:
  • Scikit-learn TensorFlow PyTorch Keras
• Collaborate with data scientists for model deployment and integration.

DevOps & Automation

• Implement CI/CD pipelines using Azure DevOps/Git.
• Use Docker for containerization.
• Automate data validation monitoring and deployment processes.

Monitoring Security & Governance

• Implement monitoring using Azure Monitor Log Analytics etc.
• Ensure compliance with data governance frameworks (e.g. Purview).
• Maintain security standards using Key Vault IAM and encryption mechanisms.

Required Skills & Qualifications
Technical Skills

• Strong programming skills in Python and/or Java
• Advanced SQL and data warehousing concepts
• Hands-on experience with:
  • Spark / PySpark
  • ETL/ELT pipeline development
  • Data modeling and optimization

Cloud Expertise

• Experience with Microsoft Azure (preferred) or AWS
• Exposure to Databricks ADF ADLS Snowflake is highly desirable

Big Data Technologies

• Apache Spark Kafka Hive Airflow

Tools & Technologies

• Git Docker CI/CD pipelines
• BI tools (Power BI preferred)

Experience Requirements

• 710 years of experience in Data Engineering / Big Data
• Proven experience in:
  • Designing scalable data architectures
  • Cloud data platform implementations
  • Data migration and transformation projects

Educational Qualifications

• Bachelors degree in Computer Science Engineering or related field
• Certifications in Azure/AWS/Data Engineering are a plus

Soft Skills

• Strong problem-solving and analytical thinking
• Ability to work in Agile/DevOps environments
• Excellent stakeholder communication and collaboration skills
• Ability to lead technical discussions and mentor junior engineers

Nice to Have

• Experience with Microsoft Fabric
• Exposure to real-time streaming architectures
• Knowledge of AI/ML pipelines

Experience in enterprise-scale data governance

Employment Type : Full Time
Experience: years
Vacancy: 1