每个spark应用都由一个驱动器程序来发起集群上的各种分布式并行操作。驱动器程序包含应用的main函数，并且定义了集群上的分布式数据集，还对这些分布式数据集应用了相关操作。
在上面的例子中，实际的驱动器程序就是spark shell本身，只要输入想运行的程序就可以了。
驱动器程序通过一个SparkContext对象来访问spark.这个对象代表对计算集群的一个连接。shell启动时已经自己创建了一个SparkContext对象，是一个叫做sc的变量。
一旦有了SparkContext，就可以用它来创建RDD。(Each spark application initiates a variety of distributed parallel operations on the cluster by a drive program. The drive program contains the main function of the application, and defines the distributed data sets on the cluster, and the related operations are applied to these distributed data sets.

In the example above, the actual drive program is the spark shell itself, as long as you enter the program you want to run.

The drive program access the spark. object by a SparkContext object to represent a connection to the computing cluster. Shell has created a SparkContext object itself when it is started, a variable called SC.

Once you have a SparkContext, you can use it to create a RDD.



Author: the home of a single teacher

Link: https://www.jianshu.com/p/c6aefad2ba0c

Source: Jane book

The copyright is owned by the author. Commercial reprint please contact the author to obtain authorization, non commercial reprint please indicate the source.)