
Description
Description
Summary:
We are seeking an experienced Databricks Architect Expert with a strong data engineering background to support the design and delivery of secure, monitored, cloud-hosted environments. The ideal candidate will possess hands-on experience with data science and analytics, extensive use of AI/ML and LLM models, AI/MLOps, DevSecOps, and GenAI tools and technologies, with expertise in deploying solutions to high-stakes production environments.
The successful candidate will join our platform team, which provides capabilities, tools, guardrails, and various patterns for our customers within a federal agency. Our goal is to support customers as they bring their applications/models into our environment, facilitating the training, building, tweaking, and productionalization of their data and models.
Key Responsibilities:
- Design and implement event-driven architecture to support scalable, resilient, and dynamic GenAI applications.
- Refactor and rehost existing applications onto cloud platforms, optimizing for performance and scalability.
- Collaborate on the development of Platform Architecture, ensuring seamless integration and efficient operation of cloud services.
- Develop and optimize Databricks Solution Architectures to handle large-scale data processing and analytics workloads.
- Engineer robust Bedrock Solution Architectures to support infrastructure provisioning and management at scale.
- Define and execute Deployment Architectures that allow for automated and reliable deployments of applications and services.
- Establish and enforce Security Architectures to protect data and maintain compliance with industry and regulatory standards.
- Create Use Case Solution Architectures, transforming business requirements into technical solutions.
- Proactively identify and address technical issues and challenges to improve the efficiency and reliability of cloud-based systems.
- Work with cross-functional teams to facilitate the smooth deployment of cloud services across various operational environments.
- Document architectural decisions and create best practice guidelines for cloud development and deployment.
- Stay abreast of emerging cloud technologies and recommend adaptations to keep ahead of industry trends.
- Utilize Agile and DevSecOps methodologies and practices.
- Coach and provide direction to junior engineers in the Databricks and data engineering areas.
· Lead efforts to integrate various data sources, ensuring seamless data flow and transformation. Develop strategies for data ingestion, cleansing, normalization, and integration.
- Leverage experience with MLFlow and general CI/CD principles for automating model deployment.
- Collaborate with customers and critical resources in an SME program to come up with solutions that align with long-term roadmaps and strategy.
- Execute Databricks upgrades, particularly in the federal space.
Qualifications
Qualifications:
- Bachelor's or master's degree in computer science, Data Science, Engineering, or a related field with 8 years of experience.
- Proven experience in cloud engineering and architecture, with a focus on event-driven solutions.
- In-depth knowledge of public cloud platforms (e.g., AWS, Azure, Google Cloud, Oracle).
- Proficiency with Databricks and understanding of big data ecosystems.
- Experience with Infrastructure as Code (IaC) using tools like Terraform, AWS CloudFormation, etc.
- Strong understanding of deployment pipelines and automated testing frameworks.
- Demonstrated experience in implementing security practices in cloud architectures.
- Clear understanding of microservices architecture and containerization technologies (e.g., Docker, Kubernetes).
- Excellent verbal and written communication skills, with the ability to document and explain complex architectures to non-technical stakeholders.
- Databricks Certification and any other AI/ML or data engineering certifications (AWS, Azure, GCP).
Desired Skills:
- Previous experience working in a regulated environment, particularly within government agencies like the IRS.
- Certifications related to AWS or cloud architecture.
- Experience with AI/ML platforms and understanding of their integration with cloud environments.
- Prior experience with event-driven architectures and related technologies (e.g., Apache Kafka, RabbitMQ).
- Prior experience in a high-stakes production environment where uptime is critical.
Target salary range: $160,001 - $200,000. The estimate displayed represents the typical salary range for this position based on experience and other factors.
Apply on company website