Senior Data Engineer

Remote (ALL US) Data Driven Enterprise - Data Driven Enterprise Full Time

The Senior Data Engineer will lead the creation of high-value data driven solutions leveraging Valorem Reply’s proven implementation methodology and solutions for enterprise projects, as well as contributing to technical pre-sales activities as required. You will be responsible for designing solution architecture, requirement definition, and leading the project delivery team. You will have the opportunity to work with and learn the latest cloud solutions in an exciting work environment. This position will work collaboratively across all Valorem Reply’s sales, service delivery and account management organizations in service to Valorem Reply’s customers.
 
This position will represent Valorem Reply’s approach to advanced data engineering solutions and as such must demonstrate proficiency at the architecture level. This position will require an understanding of how advanced analytics are positioned to meet business objectives. This position will communicate how data relates to business and enterprise architecture. This position will the Data Lakehouse solution solutions through people, process, and technology.  This is a hands-on role leading, coding, and delivering on the most advanced cloud data analytics platforms available. Projects span from workshops to full enterprise production end to end solutions.
 
The ideal candidate will have extensive Microsoft/Azure data services and Databricks technology experience. Proficiency with the Databricks platform, implementing enterprise Data Lakehouse will be required.  Candidates will be expected to contribute to all stages of the data lifecycle including data ingestion, data modeling, data profiling, data quality, data transformation, data movement, and data curation. 

Candidate should be familiar with the market challenges in multiple industry verticals and have experience with both traditional and modern across the Microsoft technology stack. 

Requirements

  • Focus on Azure data engineering solutions
  • 6+ years data engineering delivery experience
  • 2+ years of Databricks engineering development experience
  • 2+ years technical team leadership or technical management experience
  • Strong collaboration skills, embraces change, works well with ambiguity, shifts easily across roles and groups.
  • Ability and willingness to take the initiative to serve the client needs first.
  • Ability to engage directly with the client stakeholder, managers, and end users.
  • Ability to interface with clients up to the C-suite, down to the line worker.
  • Ability to design and implement Advanced Analytics solutions based on customer needs and knowledge of Microsoft Cloud and related Open-Source (Apache) technologies
  • Ability to triage and self-direct, prioritize and manage time effectively.
  • Ability to collaborate with other members of the Valorem Reply team, including Project Managers, Software Engineers, and Business Analysts
  • Technical Skills Required

  • General Architecture
  • ·         Designing large-scale data warehouse solutions
  • ·         Being able to interpret business requirements into functional and then technical requirements
  • ·         Understanding of designing pipeline /process-based systems
  • ·         Experience solutioning Azure Cloud based enterprise data architectures.
  • ·         Experience with data pipeline automation
  • Information Architecture
  • ·         Data Modeling principles for relational and dimensional data structures
  • ·         Data Warehouse design principles
  • ·         Data Lake design principles, Data Virtualization
  • ·         Hybrid enterprise data architecture
  • Microsoft Azure Stack
  • ·         RDBMS: Azure SQL / SQL Server 2016+ (SQL, Structure, Stored Procs, Tuning)
  • ·         ETL/ELT, (ADF, SSIS)
  • ·         Analysis Services [Tabular]
  • ·         Databricks Delta Lake Storage
  • - Azure Dev OPS
  • Desired Skills

  • Advanced Analytics Data Technologies
  • ·         Databricks, Delta Lake, Synapse
  • ·         Spark SQL, Pyspark
  • ·         Azure Data Explorer
  • ·         Logic Apps, Key Vault
  • ·         Semi structured data processing
  • ·         Integration Runtime
  • Information Architecture
  • ·         Data Stewardship, Ownership and Quality Management principles
  • - Data Governance, Data Catalog, Enterprise Information Management
  • Non-Technical Skills

  • Business Engagement
  • - Demonstrated Experience
  • o    Providing thought leadership and business strategy acumen around advanced analytics and business information lifecycle
  • o    Leading customers through a value-based decision-making cycle
  • o    Developing functional, non-functional and technical requirements based on business imperatives
  • o    Presenting to customer executive teams with authority and purpose
  • Project and Team Delivery Leadership
  • · Demonstrated Experience
  • o    Directing business analysts and developers in the execution of project tasks
  • o    Leading consultants in various project work streams
  • o    Estimating, planning and executing advanced analytics solutions
  • o    Managing geographically distributed project teams
  • Remote (ALL US)

    Senior Data Engineer