Big Data Quality Engineer

Company: Foothills Consulting Group, Inc
Location: Santa Monica
Posted on: March 21, 2020

Job Description:

Data team is looking for a Data QA Engineer who will help drive Data Quality initiatives. As a Data QA Engineer, you will work with Business Areas across the enterprise to improve data integrity and information usability for internal decision making and for the increasing external data demands. You will write testing plans and routines, manage continuous integration and regression testing processes, and be the point person for clearing data for release according to delivery specifications. The ideal candidate must have analytical skills and can work on multiple enterprise-level projects simultaneously. If you are passionate about data quality and interested in making an outsized impact in an organization that values data as its top priority, this is the role for you. Must have experience with Data QA. Must have solid experience in data engineering quality beside generic experience of unit testing, integration testing, functional testing, and CI/CD. For example: Need candidates to have very robust testing for our ETLs which test beyond if the job has run or not. We need to be able to create job profiling and put logic/detector to find if the ETL has return the correct data (sometime there might issue in the source system that can lead to less data or wrong data), perform continuations testing of transformation and business logics, test large data bases and data warehouse, and perform post-production testing across all layers of data solutions. Consultant will support Business Areas across the enterprise to improve data integrity and information usability for internal decision making and for the increasing external data demands. Write testing plans and routines, manage continuous integration and regression testing processes, and be the point person for clearing data for release according to delivery specifications. Should have strong analytical skills and can work on multiple enterprise-level projects simultaneously Design, monitor and maintain QA reports, KPIs and quality trends for the internal data systems Perform in-depth exploration of causes, trends, opportunities and specific actions required regarding key Metrics, KPIs, and additional Data Elements Continue to improve automation coverage and reduce regression cycles Document, automate and enforce data quality, data integrity, and QA best practices Partnering with data product managers, data governance office and engineering leads to develop testing strategies, create test plans, execute test cases Work with data engineers to develop automated tests and manage continuous integration and regression testing processes to ensure quality of the data meet the required specification Be an active participant and advocate of agile/scrum practice to ensure heath and process improvements for your team BACKGROUND: 5+ years of relevant experience in data engineering and testing Proven high level analytical and problem-solving skills and attention to detail Strong knowledge of SQL and ideally another programming/scripting languages Experience working with large data-sets and reporting tools Excellent analytical ability to understand business issues associated with data management and governance, business processes in terms of data, and identifying root causes and recommending solutions Experience developing reports in Tableau, or equivalent visualization tools Hands on experience with Hive or Pig to query and process data in Hadoop Excellent verbal and written communication skills; ability to present complex information in an easy-to-understand manner with clear recommendations based on data insight Agile and Scrum experience Experience with NoSQL and distributed computing. Experience with cloud solutions (AWS, Redshift, Snowflake, other) Industry experience in video streaming and OTT Education: Bachelor s or Master s Degree in Computer Science, Information Systems or related field - provided by Dice

Keywords: Foothills Consulting Group, Inc, Santa Monica , Big Data Quality Engineer, Engineering , Santa Monica, California