Data Engineer - Archimedes

The Engineer, Data is responsible for designing, building, and operating modern cloud-native data platforms, lake house architectures, data pipelines, and enterprise data products that enable analytics, artificial intelligence (AI), machine learning (ML), automation, and operational decision-making across the organization. Operating within an Azure-first, Databricks-centric environment, the Data Engineer develops scalable data solutions utilizing Azure Data Lake Storage Gen2, Azure Databricks, Delta Lake, Unity Catalog, Azure Data Factory, Synapse Analytics, APIs, event-driven integrations, and modern DataOps practices. The role supports the organization's transition from traditional SQL-based architectures to cloud-native lake house platforms that provide trusted, governed, reusable, and AI-ready data assets.

The Engineer, Data collaborates closely with Data Architects, Data Scientists, Analytics teams, DevOps Engineers, Cloud Engineers, Software Engineers, and business stakeholders to design canonical data models, implement data quality frameworks, establish data governance controls, develop automated data pipelines, and operationalize data products supporting reporting, analytics, machine learning, intelligent automation, robotic process automation (RPA), and enterprise AI initiatives. This role is responsible for ensuring data platforms are scalable, secure, compliant, observable, and aligned with regulatory requirements including HIPAA, HITRUST, SOC 2, and enterprise cybersecurity standards.

How do I make an impact on my team?

Design and implement lake house architectures utilizing Azure Databricks, Delta Lake, Azure Data Lake Storage Gen2, Unity Catalog, and related modern data platform technologies.
• Develop canonical enterprise data models, business entity mappings, master data structures, and reusable data products supporting enterprise reporting, analytics, automation, and AI initiatives.
• Build and maintain bronze, silver, and gold data layers supporting governed, trusted, and AI-ready datasets.
• Develop and maintain metadata, data dictionaries, business glossaries, lineage documentation, and data catalog integrations supporting enterprise data governance.
• Design, develop, and maintain ETL pipelines for structured and unstructured data across cloud and on-prem environments.
• Build and optimize data models, schemas, and storage solutions in SQL Server, PostgreSQL, and cloud-native databases.
• Design and deliver AI-ready data products supporting machine learning, generative AI, intelligent automation, retrieval-augmented generation (RAG), and agent-based AI solutions.
• Support feature engineering, vectorization pipelines, document enrichment, semantic search, and knowledge management capabilities used by enterprise AI platforms.
• Design and support healthcare data integration patterns including claims, eligibility, pharmacy, clinical, operational, financial, and third-party partner data.
• Support ingestion and transformation of structured, semi-structured, and unstructured healthcare datasets from internal and external sources.
• Implement CI/CD workflows for data pipeline deployment and monitoring using tools such as GitHub Actions, Azure DevOps, or Jenkins.
• Develop and maintain data integrations using AWS Glue, Azure Data Factory, Lambda, EventBridge, and other cloud-native services.
• Design and implement DataOps practices including automated testing, deployment automation, data quality validation, monitoring, observability, and CI/CD pipelines for data workloads.
• Develop API-based integrations supporting SaaS platforms, operational applications, third-party systems, healthcare data exchanges, intelligent automation platforms, and enterprise workflows.
• Design and support event-driven architectures utilizing Event Hub, Event Grid, Service Bus, APIs, webhooks, and streaming data technologies.
• Support machine learning, artificial intelligence, predictive analytics, and intelligent automation initiatives by developing scalable data pipelines, feature engineering datasets, training datasets, and operationalized data products.
• Partner with RPA, automation, analytics, and AI teams to support workflow automation, intelligent document processing, agent-based AI solutions, and enterprise automation initiatives.
• Implement secure data engineering practices including encryption, RBAC, data masking, row-level security, auditing, lineage tracking, and governance controls.
• Ensure data quality, lineage, and governance through automated validation, logging, and monitoring frameworks.
• Collaborate with cross-functional teams to gather requirements, design scalable solutions, and support analytics and reporting needs.
• Monitor and troubleshoot data pipeline performance, latency, and failures; implement proactive alerting and remediation strategies.
• Support data security and compliance by enforcing access controls, encryption standards, and audit logging aligned with HIPAA and SOC 2.
• Maintain documentation for data flows, architecture diagrams, and operational procedures.
• Participate in sprint planning, code reviews, and agile ceremonies to support iterative development and continuous improvement.
• Evaluate and integrate new data tools, frameworks, and cloud services to enhance platform capabilities.
• Partner with DevOps and Security teams to ensure infrastructure-as-code and secure deployment practices are followed.
• Participate in, adhere to, and support compliance, people and culture, and learning programs.
• Perform other duties as assigned.

What our team expects from you?

• Education: Bachelor's degree in Computer Science, Information Systems, Data Engineering, or related field required. Master's degree preferred.
• Certifications: AWS Certified Data Analytics or Solutions Architect, Microsoft Certified: Azure Data Engineer Associate, and Certified Data Management Professional (CDMP) preferred.
• Experience:
  • 5+ years of experience in Data Engineering, Analytics Engineering, Data Platform Engineering, ETL development, and Cloud Data Architecture required.
  • Experience designing and implementing modern lake house architectures using Azure Databricks, Delta Lake, Azure Data Lake Storage Gen2, and related cloud-native data services required.
  • Experience developing canonical data models, enterprise data products, data mappings, master data structures, and governed analytical datasets preferred.
  • Experience building DataOps pipelines, automated testing frameworks, CI/CD processes, and data quality controls preferred.
  • Experience supporting AI, machine learning, analytics, automation, and intelligent business solutions through scalable data engineering practices preferred.
  • Experience working within regulated environments supporting HIPAA, HITRUST, SOC 2, NIST, or similar compliance frameworks preferred.
  • Proven experience with SQL, ETL tools, and CI/CD pipelines.
  • Hands-on experience with AWS and Azure data services and infrastructure.
  • Familiarity with data governance, compliance frameworks (HIPAA, SOC 2), and secure data handling practices.
  • Experience with CI/CD pipelines, automated testing, and version control systems.
  • Skills & Technologies:
    • Data Platforms & Lakehouse: Azure Databricks, Delta Lake, Unity Catalog, Azure Data Lake Storage Gen2 (ADLS), Databricks Workflows, Databricks Asset Bundles, Delta Live Tables (DLT), Synapse Analytics, Lakehouse Architecture, Data Products, Data Mesh Concepts.
    • Cloud & Platform Services: Azure Data Factory (ADF), Azure Functions, Azure Key Vault, Azure Monitor, Managed Identities, Private Endpoints, Event-Driven Architecture, Infrastructure Automation.
    • Data Engineering & Processing: Apache Spark, PySpark, SQL, Python, Data Modeling, Canonical Data Models, Master Data Management (MDM), Data Mapping, Data Transformation, ETL/ELT.
    • DataOps & Automation: Azure DevOps, GitHub Actions, CI/CD for Data Pipelines, Automated Testing, Data Quality Frameworks, Data Observability, Infrastructure as Code.
    • AI, Analytics & Machine Learning: Azure Machine Learning, Databricks ML, MLflow, Feature Engineering, AI-Ready Data Products, Predictive Analytics Support, Intelligent Automation, Robotic Process Automation (RPA).
    • Integration & APIs: REST APIs, GraphQL, Event Hub, Event Grid, Service Bus, Webhooks, Data Exchange Integration Patterns.
    • Languages & Tools: SQL, Python, Bash, Git, Terraform, PowerShell
    • ETL & Orchestration: AWS Glue, Azure Data Factory, Apache Airflow
    • CI/CD: GitHub Actions, Azure DevOps, Jenkins, DataOps Automation
    • Cloud Platforms: AWS (S3, Lambda, RDS, Redshift), Azure (Blob Storage, Synapse, Functions)
    • Data Platforms & Lakehouse: Azure Databricks, Delta Lake, Unity Catalog, Azure Data Lake Storage Gen2 (ADLS), Synapse Analytics, Lakehouse Architecture, Data Products, Data Mesh Concepts
    • Security & Governance: Unity Catalog, RBAC, Data Lineage, Data Catalogs, Data Governance, HIPAA, HITRUST, SOC 2, Data Privacy Controls
    • Monitoring & Logging: CloudWatch, Azure Monitor, ELK Stack
    • Data Governance: Data cataloging, lineage tracking, encryption, and access control.

What can you expect from Archimedes?

• Top of the industry benefits for Health, Dental, and Vision insurance
• 20 days paid time off
• 4 weeks paid parental leave
• 9 paid holidays
• 401K company match of up to 5% - No vesting requirement
• Adoption Assistance Program
• Flexible Spending Account
• Educational Assistance Plan and Professional Membership assistance
• Referral Bonus Program - up to $750!