pyspark.sql.streaming.DataStreamWriter.format

DataStreamWriter.format(source)[source]

Specifies the underlying output data source.

New in version 2.0.0.

Parameters
sourcestr

string, name of the data source, which for now can be ‘parquet’.

Notes

This API is evolving.

Examples

>>> writer = sdf.writeStream.format('json')
pyspark.sql.streaming.DataStreamWriter.foreachBatch pyspark.sql.streaming.DataStreamWriter.option