Jr. Data Engineer (Global Insurance)

Overview

Your role as a Data Engineer is essential for processing, storing, managing, enriching, and delivering data to allow the organization to succeed and continually improve in business performance.

Responsiblities

• AWS & Databricks CICD process and release management
• Development of new pipelines in AWS and on Databricks through PySpark, SQL and other cloud-based technologies as needed
• Cost reduction and enforcement of best practices and optimization to foster a simplified data lake and development environment
• Ad hoc support and transversal work as directed by manager of data engineering
• Mentorship of junior members across entities and promotion of best practices based on experience in software engineering
• Promotion of improvements in technology selection, data lake architecture and development standards, reductions in development cost and internalization of knowledge wherever possible

Requirements

Mandatory

• Business Japanese, Conversational English
• Currently residing in Japan
• Strong Python, SQL and experience with data modelling in a cloud environment
• 2-3 years of working experience with python and SQL
• Experience with AWS, GCP, Azure or Databricks

Preferred

• Strong knowledge with at least 3 of the AWS Services: S3, EMR, ECS/EC2, Lambda, Glue, Athena, Kinesis/Spark Streaming, Step Functions, CloudWatch, DynamoDB, Aurora
• Strong experience with data processing and ETL systems
• Experience with Linux/Unix environment, CI/CD tools, DevOps tools, product devlopment, testing, and production release cycles