Spark Jar Task
Usage
spark_jar_task(main_class_name, parameters = list())Arguments
- main_class_name
The full name of the class containing the main method to be executed. This class must be contained in a JAR provided as a library. The code must use
SparkContext.getOrCreateto obtain a Spark context; otherwise, runs of the job fail.- parameters
Named list. Parameters passed to the main method. Use Task parameter variables to set parameters containing information about job runs.
See also
Other Task Objects:
condition_task(),
email_notifications(),
for_each_task(),
libraries(),
new_cluster(),
notebook_task(),
pipeline_task(),
python_wheel_task(),
run_job_task(),
spark_python_task(),
spark_submit_task(),
sql_file_task(),
sql_query_task()