pyspark.SparkContext.setLocalProperty

SparkContext.setLocalProperty(key, value)[source]

Set a local property that affects jobs submitted from this thread, such as the Spark fair scheduler pool.

Notes

If you run jobs in parallel, use pyspark.InheritableThread for thread local inheritance, and preventing resource leak.

previous

pyspark.SparkContext.setJobGroup

next

pyspark.SparkContext.setLogLevel