3 to 6 years proven career history in Data Warehouse, Data Analysis, or Data science engineering
Data Engineer - BIG DATA
Must be able to ensure Agile/Scrum concepts and principles are adhered to, must be able to be a voice of reason and authority, make the tough calls. Must be the first to notice differences and issues as they arise and elevate them to management.
- Big Data (1.2B+ records across 60m+ unique identifiers/month)
- Play with AWS S3, Redshift, Kinesis, RDS, Hadoop, EMR, Storm, Spark, python, R programming and plenty of other cool tools
- Working knowledge in BI and reporting tools like Tableau, SSRS, Jasper Reports, QlikView, etc.
- Join our smart and empowered high-achieving team
- Removing impediments or guiding the team to remove impediments by finding the right personnel to remove the impediment.
- Building a trusting and safe environment where problems can be raised without fear of blame, retribution, or being judged, with an emphasis of healing and problem solving.
- Facilitating getting the work done without coercion, assigning, or dictating the work.
- Facilitating discussion, decision making, and conflict resolution
- Assisting with internal and external communication, improving transparency, and radiating information
- Supporting and educating the Product Owner, especially with respect to grooming and maintaining the product backlog.
skills and experience:
- Experience using and maintaining AWS environments for data analysis and data processing
- Good knowledge in SQL, Google Big Query, Google Cloud, Data Pipeline
- Should have extensively worked in ETL data processing and migration.
- Good knowledge in Data Modelling
- Good working knowledge in R, Spark, or Python
- Considerable knowledge in Java & EMR highly regarded
- A solid understanding and practical experience using Clustered technologies
- Experience with working with Log processing tools, report creation and scheduling
- You will show initiative and drive by putting forward new ideas
- Excellent communication skills
- Very enthusiastic and self-motivated with a desire to succeed
- Highly skilled, creative and well versed in working to strict deadlines
- Experience working within medium to large enterprise environment
- Any previous experience working on Angular, Node JS, and D3 will be highly advantageous
- Exposure to Graph DB technologies desirable but not essential
- Working knowledge in any one of the workflow and monitoring interfaces like Airflow, Luigi, Dataflow, etc.
- Considerable knowledge in AWS environment.
- Automating notification and data validation.
- Considerable knowledge in Spark.
ETL DATA PROCESSING