top of page

Data Engineer - Data & Performance

Bangalore, Karnataka, India

Job Type

Full Time

About the Role

RESPONSIBILITIES:
This is a strong technology and solution delivery role, accountable for the successful design, development, and delivery of Data Engineer solutions on the corporate Data Platform. The duties and responsibilities of this job are to prepare data and make it available in an efficient and optimized format for consumer analytics, BI, or data science applications. It requires to work with current technologies used by IntelDS and in particular Spark, Presto, Hive and RedShift on AWS environment. This includes:

• Responsible for the delivery of the development activities he/she owns including Data Engineering standards, patterns and best practices.
• Work with team to ensure standards and patterns are followed.
• Determines organizational strategies for data integrity validation processes.
• Design and develop new data ingestion patterns into IntelDS raw and/or unified data layers based on the requirements and needs for connecting new data sources or for building new data objects. Working in ingestion patterns allow to automate the data pipelines.
• Participate to and apply DevSecOps practices by automating the integration and delivery of data pipelines in a cloud environment. This can include the design and implementation of end-to-end data integration tests and/or CI/CD pipelines.
• Analyze existing data models, identify and implement performance optimizations for data ingestion and data consumption. The objective is to accelerate data availability within the platform and to consumer applications. Target technologies are Apache Spark, Apache Presto and RedShift.
• Support client applications in connecting and consuming data from the platform, and ensure they follow our guidelines and best practices.
• Participate in the monitoring of the platform and debugging of detected issues and bugs.
• Partner with the Data modelers to drive improvements and enhancements to the current data landscape and future strategy.
• Identify opportunities for uses of those technologies to enhance current or anticipated information systems and business goals needs
• Evaluates existing applications that could address client requirements and makes recommendations from complex projects.
• Assists in the development of Data Engineering-related Service Level Agreements.
• Communicates risks and ensures understanding of these risks.
• Strong analytical & problem-solving skills
• Strong verbal and business communication skills
• Skill in identifying data issues and anomalies during the analysis
• Strong business acumen & demonstrated an aptitude for analytics that incite action

QUALIFICATIONS: - EDUCATION : B.E / B TECH/MASTERS IN COMPUTER SCIENCE, ELECTRONICS, RELEVANT TECHNICAL CERTIFICATION
Technology Skills and Project Experience:
- 12 years of experience in modeling and business system designs.
- 10 years hands on experience as Data Engineer/Architect with proven experience on Big Data and Data Lakes on AWS
- 7 years of experience in below Technologies
o Scripts/Programming language: SQL, Python, PROC, Windows Functions
o GitHub
o AWS: S3, RedShift, Glue
o Big Data: Spark, Hive, Presto
- Need to be strong in SQL query writing and procedures
- Experience in Informatica ETL development will be an added advantage
- Must have proven experience with STAR and SNOWFLAKE schema techniques.
- Proven track record as a Data Engineer in delivering successful business intelligence developments with complex data sources.
- Strong analytical skills and enjoys solving complex technical problems.

Business / Soft Skills:
- Must have solid presentation, communication (on complex technical solutions) and inter-personal skills.
- Ability to work effectively with globally dispersed stakeholders from business.
- Ability to manage multiple priorities in a fast-paced environment.
- A data-driven mindset, Ability to clearly communicate complex business problems and technical solutions.
- Ability to manage and make decisions about competing priorities and resources.
- Ability to delegate where appropriate
- Must be a strong team player/leader

Requirements

QUALIFICATIONS: - EDUCATION : B.E / B TECH/MASTERS IN COMPUTER SCIENCE, ELECTRONICS, RELEVANT TECHNICAL CERTIFICATION

Technology Skills and Project Experience:

- 12 years of experience in modeling and business system designs.

- 10 years hands on experience as Data Engineer/Architect with proven experience on Big Data and Data Lakes on AWS

- 7 years of experience in below Technologies

o Scripts/Programming language: SQL, Python, PROC, Windows Functions

o GitHub

o AWS: S3, RedShift, Glue

o Big Data: Spark, Hive, Presto

- Need to be strong in SQL query writing and procedures

- Experience in Informatica ETL development will be an added advantage

- Must have proven experience with STAR and SNOWFLAKE schema techniques.

- Proven track record as a Data Engineer in delivering successful business intelligence developments with complex data sources.

- Strong analytical skills and enjoys solving complex technical problems.

About the Company

Cigres Technologies Private Limited is a technology consulting and services company that focuses on helping clients resolve their significant digital problems and enabling radical digital transformation using multiple technologies on premise or in the cloud. The company was founded with the goal of leveraging cutting-edge technology to deliver innovative solutions to clients across various industries.

bottom of page