Spark Jar Task
Usage
spark_jar_task(main_class_name, parameters = list())
Arguments
- main_class_name
The full name of the class containing the main method to be executed. This class must be contained in a JAR provided as a library. The code must use
SparkContext.getOrCreate
to obtain a Spark context; otherwise, runs of the job fail.- parameters
Named list. Parameters passed to the main method. Use Task parameter variables to set parameters containing information about job runs.
See also
Other Task Objects:
email_notifications()
,
libraries()
,
new_cluster()
,
notebook_task()
,
pipeline_task()
,
python_wheel_task()
,
spark_python_task()
,
spark_submit_task()