Principal Architect

Brillio
United States
Workplace: OnsiteFull timeUSD 180,000 - 190,000 annuallyFunction: Software EngineeringExperience: 15+ yearsEducation: bachelorsSkills: []

Architect and design scalable, secure, high-performance AWS and Databricks data solutions. Build robust ETL/ELT pipelines with PySpark, Delta Lake, Unity Catalog, and Delta Live Tables, implementing medallion architecture and optimizing Spark for performance and cost. Provision infrastructure with Terraform, integrate diverse data sources, and deliver CI/CD automation using Jenkins and GitHub Actions. Monitor pipelines, troubleshoot failures, and ensure data quality, lineage, security, and compliance, with on-site presence in St. Louis.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Brillio
Brillio
1 week ago

Principal Architect

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 19 hours agoStatus: Live
Reposted: similar role first listed 6 months ago

Job Summary

Architect and design scalable, secure, high-performance AWS and Databricks data solutions. Build robust ETL/ELT pipelines with PySpark, Delta Lake, Unity Catalog, and Delta Live Tables, implementing medallion architecture and optimizing Spark for performance and cost. Provision infrastructure with Terraform, integrate diverse data sources, and deliver CI/CD automation using Jenkins and GitHub Actions. Monitor pipelines, troubleshoot failures, and ensure data quality, lineage, security, and compliance, with on-site presence in St. Louis.
Location: United States
Workplace: Onsite
Employment Type: Full time
Job Function: Software Engineering
Seniority: Mid level

Key Responsibilities

  • •Design and architect scalable, secure, high-performance AWS and Databricks solutions using AWS networking, streaming, caching, warehousing, Kubernetes, and identity services.
  • •Develop and maintain ETL/ELT pipelines in Databricks using PySpark and Python, including notebooks, jobs, Delta Lake tables, and Unity Catalog governance.
  • •Implement medallion (bronze/silver/gold) architecture and optimize Spark jobs for performance, cost, scalability, and reliability.
  • •Provision and manage cloud infrastructure with Terraform for Databricks workspaces/clusters/jobs, storage, networking, and IAM roles/permissions across Azure and AWS.
  • •Build CI/CD pipelines with Jenkins and GitHub Actions for automated testing and deployment, and monitor/troubleshoot pipelines with observability/alerting.

Pay and Benefits

Salary: USD 180,000 - 190,000 annually

Key Requirements

  • •At least 15 years of cloud architecture experience with hands-on design and implementation of AWS and Databricks solutions.
  • •Advanced AWS networking/VPC design, including strong experience with Kinesis, Elastic Cache, Redshift, and PaaS services such as Lambda, S3, and Glue.
  • •Deep Databricks expertise, including Delta Lake, Unity Catalog, and Delta Live Tables, plus medallion architecture implementation and Spark performance optimization.
  • •Strong ETL/ELT development using Python and PySpark (notebooks, jobs, Delta tables) and advanced SQL for transformation, validation, and analytics.
  • •Experience automating infrastructure and deployments using Terraform, Jenkins, and GitHub Actions.
Experience:15+ yearsCloud architectureData engineeringAWSDatabricksLakehouseReal-time streaming
Education:Bachelor's in Computer Science, Information Technology, or a closely related discipline
Certifications:AWS Certified Solutions Architect – ProfessionalDatabricks Certified Data Engineer ProfessionalApache Spark certification
Tech Stack:AWSDatabricksPySparkPythonSQLTerraformJenkinsGitHub ActionsKinesisElastic CacheRedshiftEKSCognitoLambdaS3GlueDelta LakeUnity CatalogDelta Live TablesSpark

Company Brief

Brillio
Provides digital transformation and IT consulting services, including cloud migration, analytics, digital engineering, and managed services to accelerate business outcomes for enterprises across industries.
Industry: Consulting
Company Size: Enterprise (1,001+ employees)
Growth: Established Company
Funding: Private Equity Backed
Headquarters: Santa Clara, United States
Founded: 2014
WebsiteLinkedIn