Engenheiro de Dados

Descrição

Build and maintain data, reporting and analytics processes for an AWS data lake. Manage AWS solutions and infrastructure for data, supporting developments Act as an advisor to senior management on technical questions and research. Develop auxiliary routines to monitor the environments (jobs, applications, queries). Help on architect decisions and in the data pipeline construction. Develop and deploy ETL/ELT jobs, consuming data from different types of data sources. Develop, run and maintain load tests. Bugs and data fixes, addressing root cause analysis for continuous improvement. Evolution of data schemas and documentation to delivery business value.

Qualificação

Apache NiFi for data ingestion and data flow automation. working experience using AWS Cloud, configuration and monitoring knowledge EC2, Security. CloudFormation), and specially for Big Data Services SQS, S3, RDS, EC2, Athena, Glue, EMR. working experience in Datalake or DW projects. Experience with cloud computing, as well scalability and performance problems. Experience with Unix/Linux environments, data pipelines and database modelling. Working experience with BigData, open standards and open source technologies. Expert in SQL and RDBMS including performance optimizations, preferable PostgreSQL. Knowledge on NoSQL databases like MongoDB or AWS DocumentDB. Practical knowledge of functional programming. Practical knowledge of Data Lake, S3 or Hadoop. Practical knowledge of Unstructured and Structured Data formats. Good knowledge of programming in Python. Distributed data processing using Apache Spark on AWS EMR.

Formação

Engenharia da Computação ou correlatas.

Forma de Contratação

A Combinar

Benefícios

• A Combinar

Horário de Trabalho

A combinar.