Skip to content
ExponaBLOG
Series · Day 100 of 100

Apache Spark & Data Engineering

  1. 51
    Data catalogs and lineage: practical tips and trade-offs
    Know where data comes from.
  2. 52
    Spark on AWS: Spark & Data Eng best practices explained
    Run Spark in the cloud.
  3. 53
    AWS Glue basics: a Spark & Data Eng guide for developers
    Serverless ETL on AWS.
  4. 54
    Databricks basics: a Spark & Data Eng guide for developers
    A managed lakehouse platform.
  5. 55
    Spark on Kubernetes: a practical Spark & Data Eng guide
    Containers for Spark jobs.
  6. 56
    Testing Spark pipelines: real-world lessons and examples
    Confidence in data code.
  7. 57
    Structuring PySpark projects: what developers should know
    Production-ready code layout.
  8. 58
    CI/CD for data pipelines for Spark & Data Eng developers
    Ship data code safely.
  9. 59
    Monitoring data pipelines: practical tips and trade-offs
    Know when data is late or wrong.
  10. 60
    Data observability: a practical Spark & Data Eng guide
    Treat data like production software.
  11. 61
    Cost optimization for Spark: what developers should know
    Spend less on big data.
  12. 62
    The small files problem: real-world lessons and examples
    Too many tiny files slow everything.
  13. 63
    Spark MLlib basics: a practical Spark & Data Eng guide
    Machine learning at scale.
  14. 64
    Feature engineering with Spark: best practices explained
    Prepare data for ML at scale.
  15. 65
    Pandas vs Spark vs Polars: practical tips and trade-offs
    Choose the right tool for data size.
  16. 66
    DuckDB for data engineers: practical tips and trade-offs
    Fast analytics without a cluster.
  17. 67
    Spark Connect: Spark & Data Eng best practices explained
    Thin clients for Spark.
  18. 68
    Data governance basics: a practical guide for developers
    Secure, compliant data platforms.
  19. 69
    Handling PII in pipelines: practical tips and trade-offs
    Protect personal data.
  20. 70
    Schema evolution: a Spark & Data Eng guide for developers
    Change schemas without breaking pipelines.
  21. 71
    Building a clickstream pipeline: lessons from production
    A real project end to end.
  22. 72
    Building a sales analytics pipeline in Spark & Data Eng
    Batch ETL from databases to dashboards.
  23. 73
    Backfilling data safely: real-world lessons and examples
    Reprocess history without chaos.
  24. 74
    SLAs for data: Spark & Data Eng best practices explained
    Promises about freshness and quality.
  25. 75
    Common Spark mistakes in Apache Spark & Data Engineering
    Errors that slow pipelines.
  26. 76
    Common data pipeline mistakes: best practices explained
    Habits that cause bad data.
  27. 77
    Spark interview questions: practical tips and trade-offs
    Questions for data engineering roles.
  28. 78
    Data engineering interview questions in Spark & Data Eng
    Pipeline and modeling questions.
  29. 79
    Spark 4 highlights: a practical Spark & Data Eng guide
    What is new in recent Spark releases.
  30. 80
    Data engineering roadmap for Spark & Data Eng developers
    What to learn, in order.
  31. 81
    Data engineering career tips: what developers should know
    Grow as a data engineer.
  32. 82
    Spark configuration essentials: best practices explained
    Settings that matter most.
  33. 83
    Dynamic allocation: a practical Spark & Data Eng guide
    Scale executors with the workload.
  34. 84
    Bucketing in Spark: a practical Spark & Data Eng guide
    Avoid shuffles for repeated joins.
  35. 85
    Z-ordering and data skipping: what developers should know
    Read fewer files.
  36. 86
    Spark SQL performance tips: patterns, pitfalls and fixes
    Faster SQL queries.
  37. 87
    Reading from JDBC sources: practical tips and trade-offs
    Pull data from databases efficiently.
  38. 88
    Writing to databases from Spark: lessons from production
    Load results into OLTP systems.
  39. 89
    Time travel and auditing for Spark & Data Eng developers
    Query data as it was.
  40. 90
    Handling bad records: a practical Spark & Data Eng guide
    Do not let one row break a job.
  41. 91
    Data lake folder conventions: what developers should know
    Organize storage clearly.
  42. 92
    Spark DataFrame caching pitfalls: lessons from production
    When caching hurts.
  43. 93
    Sampling data in Spark: a practical guide for developers
    Develop on small data first.
  44. 94
    Spark accumulators and broadcast variables: a deep dive
    Shared variables explained.
  45. 95
    Partition overwrite mode for Spark & Data Eng developers
    Rewrite only what changed.
  46. 96
    Columnar pruning and predicate pushdown, explained simply
    Read only what you need.
  47. 97
    Writing Spark jobs for reuse: what developers should know
    Parameterized, testable jobs.
  48. 98
    Spark and Python dependencies: best practices explained
    Ship libraries with jobs.
  49. 99
    Streaming state management: patterns, pitfalls and fixes
    Stateful streaming queries.
  50. 100
    100 days of Apache Spark and data engineering: the recap
    The lessons that matter most.