İnnovasiya və Rəqəmsal İnkişaf Agentliyi
Senior Data Engineer
Son tarix 07 apr 2024 - Müraciət et
Responsibilities:
- Data Modeling and Database Design: Developing and refining data models, and optimizing database designs for performance and scalabilit;
- Data Pipeline Monitoring and Troubleshooting: Ensuring that all data pipelines are operating correctly and addressing any issues promptly;
- Data Quality Assurance: Implementing and overseeing processes to ensure the accuracy and integrity of data;
- Query Optimization: Continuously optimizing queries and data retrieval methods for performance and efficiency;
- Integration of New Data Sources: Adding and integrating new data sources into the existing data architecture;
- Automated Script Execution and Monitoring: Running and monitoring automated scripts for data extraction, transformation, and loading (ETL);
- Collaboration with Stakeholders: Communicating with business analysts, data scientists, and other stakeholders to understand data needs and requirements;
- Performance Reviews of Data Systems: Evaluating the performance of databases, data lakes, and other storage systems;
- Data Cleaning and Transformation: Regularly cleaning and transforming data to maintain its usefulness and relevance;
- Code Reviews and Updates: Reviewing and updating ETL scripts and data pipeline code for improvements and efficiency;
- Documentation and Knowledge Sharing: Updating documentation for data pipelines and databases, and sharing knowledge with the team.
Requirements:
- Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field;
- Minimum 3-4 years of experience in operations;
- Experience with data integration (ETL/ELT) concepts, demonstratable skills of implementing data pipelines
- Experience with orchestration tools (Airflow, etc.);
- Deep understanding of SQL and experience with relational databases like PostgresSQL, MSSQL, Oracle , as well as NoSQL databases like MongoDB;
- Proficiency in languages like Python(or Scala);
- Knowledge of good engineering practices around data modelling (Star, Snowflake, Data Vault, etc), data access and code writing;
- Understanding of containers and Kubernetes;
- Ablility to clearly define data quality issues;
- At least Intermediate English is a must, spoken Russian is a plus;
- Strong problem solving/critical thinking skills.
