Lead Data Engineer (Python, AWS, Spark, Kafka, SQL, Snowflake, Databricks, GenAI)

Discover Financial
New York, Richmond
Workplace: OnsiteFull timeUSD 197,300 - 225,100 annuallyFunction: DevOps, Cloud & InfrastructureExperience: 4+ yearsEducation: bachelorsSkills: ["Collaboration","Mentoring","Experimentation","Continuous learning","Software craftsmanship"]

Build and support data pipelines that power omnichannel marketing and messaging experiences. Collaborate across Agile teams to design, develop, test, and deploy technical solutions using Python, SQL, Spark, Kafka, and cloud data warehousing like Snowflake and Databricks. Optimize data design for end-users and downstream systems, perform unit tests and reviews, and mentor other engineers while keeping up with emerging data and AI tooling.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
Discover Financial
Discover Financial
2 days ago

Lead Data Engineer (Python, AWS, Spark, Kafka, SQL, Snowflake, Databricks, GenAI)

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 8 hours agoStatus: Live

Job Summary

Build and support data pipelines that power omnichannel marketing and messaging experiences. Collaborate across Agile teams to design, develop, test, and deploy technical solutions using Python, SQL, Spark, Kafka, and cloud data warehousing like Snowflake and Databricks. Optimize data design for end-users and downstream systems, perform unit tests and reviews, and mentor other engineers while keeping up with emerging data and AI tooling.
Location: New York, Richmond
Workplace: Onsite
Employment Type: Full time
Job Function: DevOps, Cloud & Infrastructure
Seniority: Mid level

Key Responsibilities

  • •Collaborate with Agile teams to design, develop, test, implement, and support data pipeline solutions.
  • •Lead development with a team experienced in data movement, distributed computing, and full-stack systems.
  • •Use Python, SQL, and database technologies with cloud data warehousing (Snowflake/Databricks) to deliver robust solutions.
  • •Apply sound data design practices to optimize systems for end-users and downstream application consumers.
  • •Run unit tests and conduct code reviews to ensure performance-tuned, well-designed engineering work.

Pay and Benefits

Salary: USD 197,300 - 225,100 annually

Key Requirements

  • •Bachelor’s degree.
  • •4+ years of experience in application development (internship experience does not apply).
  • •2+ years of experience with big data technologies.
  • •1+ year of experience with cloud computing (AWS, Microsoft Azure, or Google Cloud).
  • •At least 1 year working with distributed data/computing or real-time/streaming data technologies (e.g., Kafka/Spark/ETL).
Experience:4+ yearsBig dataCloudStreamingData warehousing
Education:Bachelor's
Skills:CollaborationMentoringExperimentationContinuous learningSoftware craftsmanship
Tech Stack:PythonSQLOpen Source RDBMSNoSQLSnowflakeDatabricksAWSMicrosoft AzureGoogle CloudKafkaSparkAWS GlueMapReduceHadoopHiveEMRMySQLMongoCassandraRedshift

Eligibility

Work Authorization:Authorization required. Sponsorship not provided.

Company Brief

Discover Financial
Provides consumer banking products, credit cards, personal loans, and payment services through the Discover brand. It operates a major U.S. financial network and serves individuals and merchants with lending and digital payment solutions.
Industry: Retail Banking
Company Size: Enterprise (1,001+ employees)
Revenue: USD 10M to 25M
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: Riverwoods, United States
Founded: 1985
WebsiteLinkedIn