Spark组件之GraphX学习16--最短路径ShortestPaths
更多代码请见:https://github.com/xubo245/SparkLearning1解释求图中的最短路径,更多的请见参考【3】,这篇写的很详细2.代码:/*** @author xubo* ref http://spark.apache.org/docs/1.5.2/graphx-programming-guide.html* time
·
更多代码请见:https://github.com/xubo245/SparkLearning
1解释
求图中的最短路径,更多的请见参考【3】,这篇写的很详细
2.代码:
/**
* @author xubo
* ref http://spark.apache.org/docs/1.5.2/graphx-programming-guide.html
* time 20160503
*/
package org.apache.spark.graphx.learning
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext
import org.apache.spark.graphx.Graph
import org.apache.spark.graphx.Graph.graphToGraphOps
import org.apache.spark.graphx.lib.ShortestPaths
object ShortPaths {
def main(args: Array[String]): Unit = {
val conf = new SparkConf().setAppName("ShortPaths").setMaster("local[4]")
val sc = new SparkContext(conf)
// 测试的真实结果,后面用于对比
val shortestPaths = Set(
(1, Map(1 -> 0, 4 -> 2)), (2, Map(1 -> 1, 4 -> 2)), (3, Map(1 -> 2, 4 -> 1)),
(4, Map(1 -> 2, 4 -> 0)), (5, Map(1 -> 1, 4 -> 1)), (6, Map(1 -> 3, 4 -> 1)))
// 构造无向图的边序列
val edgeSeq = Seq((1, 2), (1, 5), (2, 3), (2, 5), (3, 4), (4, 5), (4, 6)).flatMap {
case e => Seq(e, e.swap)
}
// 构造无向图
val edges = sc.parallelize(edgeSeq).map { case (v1, v2) => (v1.toLong, v2.toLong) }
val graph = Graph.fromEdgeTuples(edges, 1)
// 要求最短路径的点集合
val landmarks = Seq(1, 4).map(_.toLong)
// 计算最短路径
val results = ShortestPaths.run(graph, landmarks).vertices.collect.map {
case (v, spMap) => (v, spMap.mapValues(i => i))
}
val shortestPath1 = ShortestPaths.run(graph, landmarks)
// 与真实结果对比
println("\ngraph edges");
println("edges:");
graph.edges.collect.foreach(println)
// graph.edges.collect.foreach(println)
println("vertices:");
graph.vertices.collect.foreach(println)
// println("triplets:");
// graph.triplets.collect.foreach(println)
println();
println("\n shortestPath1");
println("edges:");
shortestPath1.edges.collect.foreach(println)
println("vertices:");
shortestPath1.vertices.collect.foreach(println)
// println("vertices:")
assert(results.toSet == shortestPaths)
println("results.toSet:" + results.toSet);
println("end");
sc.stop()
}
}
图分析:其实是无向图,但是存储的时候GraphX存的是有向图
3.结果:
分析:返回的是
(1,Map(1 -> 0, 4 -> 2))
(5,Map(1 -> 1, 4 -> 1))
(6,Map(4 -> 1, 1 -> 3))
节点的属性存的是到某几点的最短路径,比如
(1,Map(1 -> 0, 4 -> 2))
表明的是1节点到1节点路径为0,到4节点为2
同理
(6,Map(4 -> 1, 1 -> 3))
6号节点到4为1,到1为3,途中可以看得出来
全部结果:
graph edges
edges:
Edge(1,2,1)
Edge(1,5,1)
Edge(2,1,1)
Edge(2,3,1)
Edge(2,5,1)
Edge(3,2,1)
Edge(5,1,1)
Edge(3,4,1)
Edge(4,3,1)
Edge(5,2,1)
Edge(4,5,1)
Edge(4,6,1)
Edge(5,4,1)
Edge(6,4,1)
vertices:
(4,1)
(1,1)
(5,1)
(6,1)
(2,1)
(3,1)
shortestPath1
edges:
Edge(1,2,1)
Edge(1,5,1)
Edge(2,1,1)
Edge(2,3,1)
Edge(2,5,1)
Edge(3,2,1)
Edge(5,1,1)
Edge(3,4,1)
Edge(4,3,1)
Edge(5,2,1)
Edge(4,5,1)
Edge(4,6,1)
Edge(5,4,1)
Edge(6,4,1)
vertices:
(4,Map(4 -> 0, 1 -> 2))
(1,Map(1 -> 0, 4 -> 2))
(5,Map(1 -> 1, 4 -> 1))
(6,Map(4 -> 1, 1 -> 3))
(2,Map(1 -> 1, 4 -> 2))
(3,Map(4 -> 1, 1 -> 2))
results.toSet:Set((1,Map(1 -> 0, 4 -> 2)), (5,Map(1 -> 1, 4 -> 1)), (2,Map(1 -> 1, 4 -> 2)), (6,Map(4 -> 1, 1 -> 3)), (4,Map(4 -> 0, 1 -> 2)), (3,Map(4 -> 1, 1 -> 2)))
end
如果改为全部节点,则为:
vertices:
(4,Map(5 -> 1, 1 -> 2, 6 -> 1, 2 -> 2, 3 -> 1, 4 -> 0))
(1,Map(5 -> 1, 1 -> 0, 6 -> 3, 2 -> 1, 3 -> 2, 4 -> 2))
(5,Map(5 -> 0, 1 -> 1, 6 -> 2, 2 -> 1, 3 -> 2, 4 -> 1))
(6,Map(5 -> 2, 1 -> 3, 6 -> 0, 2 -> 3, 3 -> 2, 4 -> 1))
(2,Map(5 -> 1, 1 -> 1, 6 -> 3, 2 -> 0, 3 -> 1, 4 -> 2))
(3,Map(5 -> 2, 1 -> 2, 6 -> 2, 2 -> 1, 3 -> 0, 4 -> 1))
results.toSet:Set((6,Map(5 -> 2, 1 -> 3, 6 -> 0, 2 -> 3, 3 -> 2, 4 -> 1)), (4,Map(5 -> 1, 1 -> 2, 6 -> 1, 2 -> 2, 3 -> 1, 4 -> 0)), (3,Map(5 -> 2, 1 -> 2, 6 -> 2, 2 -> 1, 3 -> 0, 4 -> 1)), (2,Map(5 -> 1, 1 -> 1, 6 -> 3, 2 -> 0, 3 -> 1, 4 -> 2)), (1,Map(5 -> 1, 1 -> 0, 6 -> 3, 2 -> 1, 3 -> 2, 4 -> 2)), (5,Map(5 -> 0, 1 -> 1, 6 -> 2, 2 -> 1, 3 -> 2, 4 -> 1)))
参考
【1】 http://spark.apache.org/docs/1.5.2/graphx-programming-guide.html
【2】https://github.com/xubo245/SparkLearning
【3】http://blog.csdn.net/zcf1002797280/article/details/50007913
开放原子开发者工作坊旨在鼓励更多人参与开源活动,与志同道合的开发者们相互交流开发经验、分享开发心得、获取前沿技术趋势。工作坊有多种形式的开发者活动,如meetup、训练营等,主打技术交流,干货满满,真诚地邀请各位开发者共同参与!
更多推荐
已为社区贡献6条内容
所有评论(0)