How I build my first Spark Application

In hadoop, we use job, in Spark, we use application.

I followed the two instructions to build a Spark Application.


http://backtobazics.com/big-data/spark/building-spark-application-jar-using-scala-and-sbt/




http://scalatutorials.com/beginner/2013/07/18/getting-started-with-sbt/

I make the directory myProject, etc following the second post.

And I set up WordCount.scala and WordCount.sbt in myProject/src/main/scala:

And I ran "sbt package" under the directory and got a jar.

And I create a bash script:

It took a while to find where spark-submit is.

And I ran the bash, still something wrong.

 



 But it seems it is close.

And then, I re-wrote the sh file to be:

And I re-wrote the scala wordcount to be:

And I got the results in /home/ubuntu  output directory.  It is in a file named part-00000:

My original file is like: