This file will serve as a hadoop client to your Spark Job.
#Intellij scala download
If you end up with the below exception, download the winutils.exe file. Splits.saveAsTextFile("D:\\Spark\\spark-1.6.0\\SplitOutput")Ĭounts.saveAsTextFile("D:\\Spark\\spark-1.6.0\\CountOutput") Val counts = splits.reduceByKey((x,y)=>x+y) Val splits = tf.flatMap(line => line.split(" ")).map(word =>(word,1)) Val conf = new SparkConf().setAppName("SparkWordCount").setMaster("local") Right Click src folder and choose Scala Class. Right click the src and Select Open Module Settings. Look at the dependencies below in the Open Module Settings under src folder. Choose the proper version of the libraries if you get into this warning. You can look for a complete and latest build.sbt file hereĪdd Dependencies as below in build.sbt file name := "MyFirstSparkProject" resolvers += Resolver.url("artifactory", url(""))(Resolver.ivyStylePatterns)ĪddSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.13.0") You might want this Plugin to create aUBER Jar for your application. Wait Until the Entire Project Folder (with Src,Target,folders,etc) are Established after you created the Project.Ĭreate a assembly.sbt under project folder.
#Intellij scala install
Please install Scala/SBT Plugin in your IntelliJ IDE (File -> Settings -> Plugins)
0 kommentar(er)
