Data Engineer

Data Engineer jobs at Big-Data.digital

Data Engineer

$ Not disclosed

Job Summary

The Data Engineer is responsible for building, maintaining, and ensuring data production pipelines are running smoothly. This role will monitor data pipelines for our data analytics platform and proactively diagnose and resolve issues, enhance pipeline components as needed and develop support tools to enable non-technical support staff to troubleshoot common issues over time. This role will implement methods to improve data reliability and quality as well as develop and test architectures that enable data extraction and transformation for predictive or prescriptive modeling. This role may perform these duties in Cloud, non-Cloud and hybrid environments.

Essential Duties & Responsibilities

  • Analyze and organize raw data sets to meet both functional and non-functional requirements
  • Develop and maintain data sets
  • Improve data quality and efficiency
  • Create scalable, production-ready pipelines for data movement and transformation
  • Work with various stakeholders, including data, design, product, and executive teams to assist with data-related technical issues
  • Implement and enhance support tools for monitoring and acting on data pipeline issues
  • Interpret trends and patterns
  • Conduct data analysis and report on results
  • Prepare data for prescriptive and predictive modeling
  • Build algorithms and prototypes
  • Combine raw information from different sources
  • Explore ways to enhance data quality and reliability
  • Identify opportunities for data acquisition
  • Develop analytical tools and programs

#LI-REMOTE

Education

  • Bachelor’s degree in Computer Science, Information Technology, Mathematics or another related field. Master’s degree is a plus. Years of experience may substitute for a degree.

Minimum Qualifications

  • Three to five years of software development experience with a focus on data engineering
  • Data engineering certification is a plus
  • Experience with Cloud-based deployment and data processing technologies (AWS Lambda, Step Functions, S3 storage, Simple Service (SQS), EMR clusters, CloudWatch monitoring and CloudWatch insights)
  • Experience with supporting data transformation and analytics solutions preferred.
  • Experience in Healthcare Information Technology (HIT) highly preferred

Skills/Knowledge

  • Technical expertise with data models, data mining, and segmentation techniques and software architecture
  • Working experience with relational databases, queries, optimization (DBA or database developer with proven hands-on application and understanding) with high emphasis on Postgres.
  • Working experience with the following languages: Spark, Python (1+ years preferred), SQL (2 years), Java or C# (preferred), Kafka (preferred)
  • High-level understanding of data schemas for structured and unstructured data (scheme on write vs schema on read, columnar vs row, relational vs NoSQL)
  • Big Data experience (HDFS, Hive, HBase, M/R, Impala, Spark, HUE) preferred
  • Understanding of software architecture
  • UI Development (HTML/CSS/JavaScript, Angular)
  • Simple Notification Service (AWS SNS)
  • Change Data Capture
  • Working knowledge of data lake architecture (batch vs streaming, Lambda architecture, Delta Lake, Apache Hudi)
  • Pipeline orchestration (Apache Airflow, Apache NiFi)

Work Environment/Physical Demands

  • While at work, this position is primarily a sedentary job and requires that the associate can work in an environment where they will consistently be seated for the majority of the workday.
  • This role requires that one can sit and regularly type on a keyboard the majority of their workday.
  • This position requires the ability to observe a computer screen for long periods of time to observe their own and others’ work, as well as in-coming and out-going communications via the computer and/ or mobile devices.
  • The role necessitates the ability to listen and speak clearly to customers and other associates.
  • The work environment is an open room with other associates and noise from others will be part of the regular workday.
  • This role may required days/hours outside of normal office hours depending on client needs and project deadlines.

Get the book



Published 2021-10-01

Greenway Health looks for a Data Engineer in Carrollton, GA 30117 location.

How much Data Engineer Jobs professional earns at Greenway Health?

The proposed salary for Data Engineer Jobs position at Greenway Health is Not disclosed.

Data Engineer responsibilities at Greenway Health

According to the common industry standard, Data Engineer Jobs specialist is in charge for the following activities.

Required Data Engineer Jobs qualifications at Greenway Health

To be disscussed with candidate