Senior Data Engineer

 
Job Description 

WORK EXPERIENCE

3 to 6 years proven career history in Data Warehouse, Data Analysis, or Data science engineering

Data Engineer - BIG DATA

Must be able to ensure Agile/Scrum concepts and principles are adhered to, must be able to be a voice of reason and authority, make the tough calls. Must be the first to notice differences and issues as they arise and elevate them to management.

responsibilities:

  • Big Data (1.2B+ records across 60m+ unique identifiers/month)
  • Play with AWS S3, Redshift, Kinesis, RDS, Hadoop, EMR, Storm, Spark, python, R programming and plenty of other cool tools
  • Working knowledge in BI and reporting tools like Tableau, SSRS, Jasper Reports, QlikView, etc.
  • Join our smart and empowered high-achieving team
  • Removing impediments or guiding the team to remove impediments by finding the right personnel to remove the impediment.
  • Building a trusting and safe environment where problems can be raised without fear of blame, retribution, or being judged, with an emphasis of healing and problem solving.
  • Facilitating getting the work done without coercion, assigning, or dictating the work.
  • Facilitating discussion, decision making, and conflict resolution
  • Assisting with internal and external communication, improving transparency, and radiating information
  • Supporting and educating the Product Owner, especially with respect to grooming and maintaining the product backlog.

skills and experience:

  • Experience using and maintaining AWS environments for data analysis and data processing
  • Good knowledge in SQL, Google Big Query, Google Cloud, Data Pipeline
  • Should have extensively worked in ETL data processing and migration.
  • Good knowledge in Data Modelling
  • Good working knowledge in R, Spark, or Python
  • Considerable knowledge in Java & EMR highly regarded
  • A solid understanding and practical experience using Clustered technologies
  • Experience with working with Log processing tools, report creation and scheduling
  • You will show initiative and drive by putting forward new ideas
  • Excellent communication skills
  • Very enthusiastic and self-motivated with a desire to succeed
  • Highly skilled, creative and well versed in working to strict deadlines
  • Experience working within medium to large enterprise environment
  • Any previous experience working on Angular, Node JS, and D3 will be highly advantageous
  • Exposure to Graph DB technologies desirable but not essential

DESIRABLE Knowledge:

  • Working knowledge in any one of the workflow and monitoring interfaces like Airflow, Luigi, Dataflow, etc.
  • Considerable knowledge in AWS environment.
  • Automating notification and data validation.
  • Considerable knowledge in Spark.

Key Features

BIG QUERY SQL PYTHON ETL DATA PROCESSING


 
   

About Us

If you want something new, you have to stop doing something old.

Its time to pull up the anchors and set for the sail to the new harbour, where you would be ready to unleash your power of creativity and come up with kickass ideas to develop applications and dare to explore new avenues and join forces with an open house of over enthusiastic bunch of young minds!

Do drop in your resume or walk in if you have the following technical attributes and your head full of ideas with a burning desire to make a mark for yourself.

We are equally excited in looking forward for that meeting with you, over a cup of coffee or a glass of beer.

For more info please visit http://www.gyanmatrix.com/