5 Hidden Apache Spark Facts That Fewer People Talk About
Developing Apache Spark can sometimes be frustrating when you hit the hidden facts on it. Those facts that fewer people talk about should be addressed in online courses or books. Until one day, you found the unexpected result and dug into the Apache Spark source code.
I want to share five hidden facts about Apache Spark that I learned throughout my career. Those can be helpful to you to save you some time reading the Apache Spark source code.
No-op Operation
Working on the Apache Spark data frame is like working with a table. We perform operations to extract more valuable information by transforming the data.
In 95% of the cases, Apache Spark will scream at you and fail your job if the column name is incorrectly provided.
However, that doesn't mean Spark will always catch the error you had. Those are the case where Spark have no-op operation.
What's a no-op operation?
0 Comments