postgresql – Apache Spark：JDBC连接不起作用

我已经在我的本地窗口上成功安装了Spark 1.3.0并运行了示例程序以使用pyspark shell进行测试.

现在,我想对存储在Postgresql中的数据运行Mllib的Correlations,但是我无法连接到postgresql.

我已成功通过运行在类路径中添加了所需的jar(测试此jar)

pyspark --jars "C:pathtojarpostgresql-9.2-1002.jdbc3.jar"

我可以看到jar已成功添加到环境UI中.

当我在pyspark shell中运行以下代码时 –

from pyspark.sql import SQLContext
sqlContext = SQLContext(sc)
df = sqlContext.load(source="jdbc",url="jdbc:postgresql://[host]/[dbname]",dbtable="[schema.table]")

我得到这个错误 –

>>> df = sqlContext.load(source="jdbc",dbtable="[schema.table]")
Traceback (most recent call last):
  File "<stdin>",line 1,in <module>
  File "C:UsersACERNEW3DesktopSparkspark-1.3.0-bin-hadoop2.4pythonpysparksqlcontext.py",line 482,in load
    df = self._ssql_ctx.load(source,joptions)
  File "C:UsersACERNEW3DesktopSparkspark-1.3.0-bin-hadoop2.4pythonlibpy4j-0.8.2.1-src.zippy4jjava_gateway.py",line 538,in __call__
  File "C:UsersACERNEW3DesktopSparkspark-1.3.0-bin-hadoop2.4pythonlibpy4j-0.8.2.1-src.zippy4jprotocol.py",line 300,in get_return_value
py4j.protocol.Py4JJavaError: An error occurred while calling o20.load.
: java.sql.SQLException: No suitable driver found for     jdbc:postgresql://[host]/[dbname]
        at java.sql.DriverManager.getConnection(DriverManager.java:602)
        at java.sql.DriverManager.getConnection(DriverManager.java:207)
        at org.apache.spark.sql.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:94)
        at org.apache.spark.sql.jdbc.JDBCRelation.<init>    (JDBCRelation.scala:125)
        at  org.apache.spark.sql.jdbc.DefaultSource.createRelation(JDBCRelation.scala:114)
        at org.apache.spark.sql.sources.ResolvedDataSource$.apply(ddl.scala:290)
        at org.apache.spark.sql.SQLContext.load(SQLContext.scala:679)
        at org.apache.spark.sql.SQLContext.load(SQLContext.scala:667)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
        at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379)
        at py4j.Gateway.invoke(Gateway.java:259)
        at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
        at py4j.commands.CallCommand.execute(CallCommand.java:79)
        at py4j.GatewayConnection.run(GatewayConnection.java:207)
        at java.lang.Thread.run(Thread.java:619)