Data Engineer III

American Express
Bengaluru
Workplace: HybridFull timeFunction: Software EngineeringSkills: ["Problem-solving","Mentoring","Collaboration","Documentation","Adaptability"]

Improve American Express’ GCP data platforms by partnering with business and use-case teams to diagnose and eliminate performance, scalability, reliability, and cost issues. Optimize large-scale Apache Spark/PySpark workloads, reduce GCP costs across BigQuery/Dataproc/Dataflow and orchestration tools, and build reusable frameworks, APIs, and engineering standards. Work on GenAI-enabling pipelines for RAG and agentic systems, mentor engineers, and drive adoption across teams.

Loading

Loading job details...

Preparing the role view and application actions.

FursaFursa
American Express
American Express
2 days ago

Data Engineer III

✓ Verified Job

Canonical indexed version, validated from employer's careers page.

Source: Company careers pageValidated by: Fursa AI
Last checked: 2 hours agoStatus: Live

Job Summary

Improve American Express’ GCP data platforms by partnering with business and use-case teams to diagnose and eliminate performance, scalability, reliability, and cost issues. Optimize large-scale Apache Spark/PySpark workloads, reduce GCP costs across BigQuery/Dataproc/Dataflow and orchestration tools, and build reusable frameworks, APIs, and engineering standards. Work on GenAI-enabling pipelines for RAG and agentic systems, mentor engineers, and drive adoption across teams.
Location: Bengaluru
Workplace: Hybrid
Employment Type: Full time
Job Function: Software Engineering
Seniority: Mid level

Key Responsibilities

  • •Partner with business and use-case teams to understand existing data platforms and workloads, and identify opportunities to improve performance, scalability, reliability, and cost efficiency.
  • •Optimize large-scale Apache Spark and PySpark workloads and develop measurable baselines for throughput, latency, resource utilization, and cost improvements.
  • •Optimize GCP platform costs by improving code, queries, and resource configurations across BigQuery, Dataproc, Dataflow, Cloud Composer, and related services.
  • •Design and develop reusable engineering frameworks, libraries, APIs, and platform components for ingestion, transformation, validation, observability, error handling, retries, reconciliation, and data quality.
  • •Define and enforce engineering standards and reference architectures for batch/streaming/event-driven processing and orchestration, including logging/monitoring/alerting/tracing and production readiness; mentor engineers and drive adoption.

Pay and Benefits

Perks:Health InsuranceDentalVisionLife InsuranceDisabilityPaid ParentalRetirementCounseling SupportWellness Stipend

Key Requirements

  • •Highly experienced, hands-on GCP data engineering capability with strong fundamentals in BigQuery, Apache Spark, and Python.
  • •Experience optimizing distributed data processing workloads (e.g., joins, shuffles, partitioning, skew, caching, executor configuration) and diagnosing performance problems.
  • •Practical GCP platform knowledge across BigQuery, Dataproc, Dataflow, Cloud Composer, and Cloud-native services, including workflow orchestration.
  • •Strong conceptual understanding of LLM-powered applications, RAG, agentic AI/multi-agent architectures, and building pipelines for chunking, embeddings, vector search, and relevance optimization.
  • •Experience mentoring engineers and driving adoption of engineering standards, best practices, and reusable platform capabilities.
Experience:GCPBig DataDistributed systemsGenAIRAG
Skills:Problem-solvingMentoringCollaborationDocumentationAdaptability
Tech Stack:GCPGoogle Cloud PlatformApache SparkPySparkBigQueryDataprocDataflowCloud ComposerPub/SubApache AirflowAstronomerVertex AIPythonAPIsCI/CDSpring BootFastAPIVector search

Company Brief

American Express
Provides global payment, credit, and travel-related financial services for consumers and businesses. Best known for its charge and credit cards, merchant payment network, and premium customer rewards and servicing.
Industry: Payments
Company Size: Enterprise (1,001+ employees)
Revenue: USD 1B+
Growth: Public Company
Valuation: Public Company (Market Cap in USD)
Funding: IPO / Publicly Listed
Headquarters: New York City, United States
Founded: 1850
WebsiteLinkedIn