ezoic

Monday, April 10, 2017

Spark data type convert, change

val text=sc.textFile("README.md")

text is an RDD. README.md is a Spark system file.

change rdd to dataframe:

val text1=text.toDF

convert list to RDD:

 val textfile1=List(("Ale",1259,278,6782))

textfile1 is a list.

 change list to RDD.
  val textfile2=sc.parallelize(textfile1)


convert list to dataframe:

val aa=List(("a","k","u","t"))

val ak=aa.map{case (a, b, c, d) => (a, b, c, d)}

val au=ak.toDF

println(au.show)



val aa=List(List("a","k","u","t"))

val ak=aa.map{case List(a, b, c, d) => (a, b, c, d)}

val au=ak.toDF

println(au.show)

change RDD to list:

 val text1=sc.textFile("README.md")

text1 is an RDD

change RDD to array:

val text2=text1.collect()











No comments:

Post a Comment

looking for a man

 I am a mid aged woman. I live in southern california.  I was born in 1980. I do not have any kid. no compliacted dating.  I am looking for ...