pyspark.sql.DataFrameWriter.jdbc

DataFrameWriter.jdbc(url, table, mode=None, properties=None)[source]

Saves the content of the DataFrame to an external database table via JDBC.

New in version 1.4.0.

Parameters
urlstr

a JDBC URL of the form jdbc:subprotocol:subname

tablestr

Name of the table in the external database.

modestr, optional

specifies the behavior of the save operation when data already exists.

  • append: Append contents of this DataFrame to existing data.

  • overwrite: Overwrite existing data.

  • ignore: Silently ignore this operation if data already exists.

  • error or errorifexists (default case): Throw an exception if data already exists.

propertiesdict

a dictionary of JDBC database connection arguments. Normally at least properties “user” and “password” with their corresponding values. For example { ‘user’ : ‘SYSTEM’, ‘password’ : ‘mypassword’ }

Notes

Don’t create too many partitions in parallel on a large cluster; otherwise Spark might crash your external database systems.

pyspark.sql.DataFrameWriter.insertInto pyspark.sql.DataFrameWriter.json