Video narration
When you call filter or map in Spark, has the data already been processed? Think Python list: chain filter and map, then produce a result. Spark uses a similar idea, with an important delay. A transformation takes an existing RDD, DataFrame, or Dataset and creates a new logical dataset from it. So filter or map does not immediately run through the data. It adds another step to Spark's logical execution plan, often described as a DAG, or directed acyclic graph. Here's the turn: the transformation call is not the computation. An action is. When you call count() or show(), you request a result, and Spark executes the accumulated workflow. This is lazy evaluation: Spark waits until a result is needed, then runs the plan it has built. So, has calling filter or map processed the data? No. It has prepared the work; count() or show() starts it.