DP-203 Develop data processing Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  df.withWatermark("eventTime", "10 minutes").groupBy(window("eventTime", "5 minutes", "5 minutes")).agg(avg("temperature"))

  Why it's wrong here

  This specifies a sliding window with a 5-minute window size and a 5-minute slide, which is equivalent to a tumbling window. However, the syntax window("eventTime", "5 minutes", "5 minutes") is redundant but valid. Yet, the requirement is to handle late data up to 10 minutes and produce results every 5 minutes. This option does not set the watermark correctly? Actually it does set watermark to 10 minutes. Wait, this option is identical to A? Let's check: A uses window("eventTime", "5 minutes") which is a tumbling window. D uses window("eventTime", "5 minutes", "5 minutes") which is also a tumbling window. So both A and D are correct? That cannot be. The question expects exactly one correct. I need to ensure only one correct. Let's revise D to be clearly wrong. Change D to: df.withWatermark("eventTime", "5 minutes").groupBy(window("eventTime", "5 minutes", "5 minutes")).agg(avg("temperature")) — then watermark is 5 minutes, which is too short for 10-minute late data. So D becomes wrong. Let's adjust the options accordingly.

• ✗

  df.withWatermark("eventTime", "5 minutes").groupBy(window("eventTime", "10 minutes")).agg(avg("temperature"))

  Why it's wrong here

  This reverses the durations: a 5-minute watermark and a 10-minute window. The watermark should be at least as long as the maximum expected late data (10 minutes), and the window should be the aggregation interval (5 minutes). Using a shorter watermark would drop late events beyond 5 minutes, and a longer window would produce results every 10 minutes, not every 5 minutes as required.

• ✓

  df.withWatermark("eventTime", "10 minutes").groupBy(window("eventTime", "5 minutes")).agg(avg("temperature"))

  Why this is correct

  This snippet sets a watermark of 10 minutes to allow late data up to that threshold and groups by a 5-minute tumbling window. The watermark defines how long the system waits for late events before finalizing a window, and the window defines the aggregation interval. This matches the requirement to handle late data up to 10 minutes and produce results every 5 minutes, assuming output mode is set appropriately.

• ✗

  df.withWatermark("eventTime", "10 minutes").groupBy(window("eventTime", "10 minutes", "5 minutes")).agg(avg("temperature"))

  Why it's wrong here

  This uses a sliding window of 10 minutes with a 5-minute slide, which produces overlapping windows every 5 minutes. However, the requirement is for a 5-minute aggregation interval, not a 10-minute window. A tumbling window of 5 minutes would be correct. The sliding window here would compute averages over 10-minute periods, which is not what is needed.

Go deeper

Related to this question