#spark

Describe how you would architect a pipeline to process real-time logs with schema evolution

Describe how you would debug a failing ETL pipeline in production.

Describe how you would design a data catalog for managing metadata

Describe how you'd design a system to track inventory and sales in real-time.

Describe strategies for monitoring, retries, idempotency, and validation in data pipelines.

Describe the architecture of an ETL pipeline you built in your previous project.

Describe the steps involved in optimizing an existing data transformation pipeline.

Describe your current project, including technologies, architecture, and responsibilities.