Spark组件之GraphX学习8--邻居集合
更多代码请见:https://github.com/xubo245/SparkLearning1解释收集每个顶点的邻居顶点2.代码:/*** @author xubo* ref http://spark.apache.org/docs/1.5.2/graphx-programming-guide.html* time 20160503*/pa
·
更多代码请见:https://github.com/xubo245/SparkLearning
1解释
收集每个顶点的邻居顶点
2.代码:
/**
* @author xubo
* ref http://spark.apache.org/docs/1.5.2/graphx-programming-guide.html
* time 20160503
*/
package org.apache.spark.graphx.learning
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext
import org.apache.spark.graphx.Graph
import org.apache.spark.graphx.VertexId
import org.apache.spark.graphx.VertexRDD
import org.apache.spark.graphx.util.GraphGenerators
import org.jets3t.apps.synchronize.Synchronize
import breeze.linalg.reverse
import breeze.linalg.reverse
import org.apache.spark.graphx.EdgeDirection
object CollectingNeighbors {
val K = 3
var arr = new Array[(Int, Int)](K)
def main(args: Array[String]): Unit = {
val conf = new SparkConf().setAppName("CollectingNeighbors").setMaster("local[4]")
// Assume the SparkContext has already been constructed
val sc = new SparkContext(conf)
// Import random graph generation library
// Create a graph with "age" as the vertex property. Here we use a random graph for simplicity.
val graph: Graph[Double, Int] =
GraphGenerators.logNormalGraph(sc, numVertices = 6).mapVertices((id, _) => id.toDouble)
// Compute the number of older followers and their total age
println("Graph:");
println("sc.defaultParallelism:" + sc.defaultParallelism);
println("vertices:");
graph.vertices.collect.foreach(println(_))
println("edges:");
graph.edges.collect.foreach(println(_))
println("count:" + graph.edges.count);
println("\ninDegrees");
graph.inDegrees.foreach(println)
println("\nneighbors0:");
val neighbors0 = graph.collectNeighborIds(EdgeDirection.Out)
neighbors0.foreach(println)
neighbors0.collect.foreach { a =>
{
println(a._1 + ":")
a._2.foreach(b => print(b + " "))
println();
}
}
}
}
代码解释:
每个顶点的出边对应的顶点
入度类推
3.结果:
Graph:
sc.defaultParallelism:4
vertices:
(4,4.0)
(0,0.0)
(1,1.0)
(5,5.0)
(2,2.0)
(3,3.0)
edges:
Edge(0,0,1)
Edge(0,3,1)
Edge(1,2,1)
Edge(1,3,1)
Edge(1,4,1)
Edge(1,4,1)
Edge(1,5,1)
Edge(2,1,1)
Edge(2,5,1)
Edge(3,1,1)
Edge(3,4,1)
Edge(4,2,1)
Edge(4,3,1)
Edge(4,4,1)
Edge(5,0,1)
Edge(5,2,1)
Edge(5,4,1)
Edge(5,4,1)
Edge(5,5,1)
count:19
inDegrees
(3,3)
(4,6)
(0,2)
(2,3)
(1,2)
(5,3)
neighbors0:
(1,[J@781c1417)
(5,[J@3a02a86a)
(2,[J@7b6b2f9b)
(3,[J@4fba6bc2)
(4,[J@42e18b85)
(0,[J@658eeb10)
4:
2 3 4
0:
0 3
1:
2 3 4 4 5
5:
0 2 4 4 5
2:
1 5
3:
1 4
参考
【1】 http://spark.apache.org/docs/1.5.2/graphx-programming-guide.html
【2】https://github.com/xubo245/SparkLearning
开放原子开发者工作坊旨在鼓励更多人参与开源活动,与志同道合的开发者们相互交流开发经验、分享开发心得、获取前沿技术趋势。工作坊有多种形式的开发者活动,如meetup、训练营等,主打技术交流,干货满满,真诚地邀请各位开发者共同参与!
更多推荐
已为社区贡献6条内容
所有评论(0)