Lead Data Engineer (Python, AWS, Spark, Kafka, SQL, Snowflake, Databricks, GenAI)

Discover Financial
New York, Richmond
Workplace: OnsiteFull timeFunction: DevOps, Cloud & InfrastructureExperience: 4+ yearsEducation: bachelorsSkills: ["Collaboration","Mentoring","Data design","Performance tuning","Agile engineering"]

Design, build, test, and support data movement solutions for a team delivering hyper-personalized, omnichannel messaging experiences. Lead a developer team working on distributed computing and data pipelines, using Python, SQL, cloud data warehousing, and emerging technologies. Optimize data design for downstream consumers, mentor engineers, participate in tech communities, and ensure performance-focused unit testing and reviews across Agile teams.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Discover Financial
Discover Financial
1 day ago

Lead Data Engineer (Python, AWS, Spark, Kafka, SQL, Snowflake, Databricks, GenAI)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 17 hours agoStatus: Live
Reposted: similar role first listed 5 days ago

Job Summary

Design, build, test, and support data movement solutions for a team delivering hyper-personalized, omnichannel messaging experiences. Lead a developer team working on distributed computing and data pipelines, using Python, SQL, cloud data warehousing, and emerging technologies. Optimize data design for downstream consumers, mentor engineers, participate in tech communities, and ensure performance-focused unit testing and reviews across Agile teams.
Location: New York, Richmond
Workplace: Onsite
Employment Type: Full time
Job Function: DevOps, Cloud & Infrastructure
Seniority: Mid level

Key Responsibilities

  • •Collaborate across Agile teams to design, develop, test, implement, and support technical solutions using data movement tools and technologies.
  • •Lead development with a team experienced in data movement, distributed computing, and full-stack systems.
  • •Use Python, SQL, and database technologies with cloud-based warehousing (Snowflake, Databricks) to build data pipelines.
  • •Optimize end-user and downstream application consumption through sound data design practices.
  • •Perform unit tests and reviews to ensure code is well-designed, elegantly coded, and tuned for performance.

Key Requirements

  • •Bachelor’s degree.
  • •At least 4 years of experience in application development (internship experience does not apply).
  • •At least 2 years of experience in big data technologies.
  • •At least 1 year of experience with cloud computing (AWS, Microsoft Azure, Google Cloud).
  • •Experience with data pipelines and data modeling practices (data quality, governance, and permissible use).
Experience:4+ years
Education:Bachelor's
Skills:CollaborationMentoringData designPerformance tuningAgile engineering
Tech Stack:PythonSQLOpen Source RDBMSNoSQLSnowflakeDatabricksAWSMicrosoft AzureGoogle CloudKafkaSparkETLAWS GlueMapReduceHadoopHiveEMRMongoCassandraRedshift

Eligibility

Work Authorization:Authorization required. Sponsorship not provided.

Company Brief

Discover Financial
Provides consumer banking products, credit cards, personal loans, and payment services through the Discover brand. It operates a major U.S. financial network and serves individuals and merchants with lending and digital payment solutions.
Industry: Retail Banking
Company Size: Enterprise (1,001+ employees)
Revenue: USD 10M to 25M
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: Riverwoods, United States
Founded: 1985
WebsiteLinkedIn