Loom at Clojure/West

Loom and Graphs in Clojure

github.com/aysylu/loom

Aysylu Greenberg

@aysylu22; http://aysy.lu March 25th 2014

Graphs

Loom

Graph Algorithms + Visualization

In this Talk

Loom Overview

Loom’s Graph

API

Functional Graph

Algorithms Your

Graphs in Loom

In this Talk

Loom Overview

Loom’s Graph

API

Functional Graph

Algorithms Your

Graphs in Loom

Loom Overview: Supported Graphs

•  Undirected Graph •  Directed Graph (Digraph) •  Weighted Graph •  FlyGraph o  nodes + successors edges o  successors + start nodes + edges

Loom Overview: Graph Algorithms

• DFS/BFS (+ bidirectional)


• DFS/BFS (+ bidirectional) • Topological Sort


• DFS/BFS (+ bidirectional) • Topological Sort • Single Source Shortest Path (Dijkstra,

Bellman-Ford)



Bellman-Ford) • Strongly Connected Components (Kosaraju)



Bellman-Ford) • Strongly Connected Components (Kosaraju) • Density (edges/nodes) • Loner Nodes


Bellman-Ford) • Strongly Connected Components (Kosaraju)



Bellman-Ford) • Strongly Connected Components (Kosaraju) • Density (edges/nodes) • Loner Nodes • Two Coloring



Bellman-Ford) • Strongly Connected Components (Kosaraju) • Density (edges/nodes) • Loner Nodes • Two Coloring • Max-Flow (Edmonds-Karp)

Loom Overview: Visualizing Graphs

In this Talk

Loom Overview

Loom’s Graph

API

Functional Graph

Algorithms Your

Graphs in Loom

Graph API: Representations


• Adjacency List {A: [B], B: [C, D], C: [], …}


• Adjacency List •  Incidence List {A: [e1], B: [e2, e3], …, e1: [A, B], e2: [B, C], …}


• Adjacency List •  Incidence List • Adjacency Matrix Source Destination

A B … E A 0 1 … ∞ B ∞ 0 … ∞ … … … … … E ∞ 1 … 0


• Adjacency List •  Incidence List • Adjacency Matrix •  Incidence Matrix

Nodes Edges e1 e2 .. e5

A S N … N B D S … D … … … … … E N N … S


• Adjacency List sparse graphs •  Incidence List • Adjacency Matrix dense graphs •  Incidence Matrix

Graph API: Graph (defprotocol Graph! (nodes [g])! (edges [g])! (has-node? [g node])! (has-edge? [g n1 n2])! (successors [g] [g node])! (out-degree [g node]))!

Graph API: Digraph (defprotocol Digraph! (predecessors [g] [g node])! (in-degree [g node])! (transpose [g]))

Graph API: WeightedGraph (defprotocol WeightedGraph! (weight [g] [g n1 n2]))

Graph API: EditableGraph (defprotocol EditableGraph! (add-nodes* [g nodes])! (add-edges* [g edges])! (remove-nodes* [g nodes])! (remove-edges* [g edges])! (remove-all [g]))

Graph API: Complex Graphs

How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])!


How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])!(extend BasicEditableDigraph! Graph …! Digraph …! EditableGraph …)!


How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])!(extend BasicEditableDigraph! Graph!!!!!)!

(nodes [g])!(edges [g])!(has-node? [g node])!(has-edge? [g n1 n2])!(successors [g] [g node])!(out-degree [g node])

Protocol definition (not valid code!)


How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])!(extend BasicEditableDigraph! Graph default-graph-impl! Digraph …! EditableGraph …)!

Graph API: Complex Graphs (def default-graph-impl! {:edges! (fn [g]! (for [n1 (nodes g)! n2 (successors g n1)]! [n1 n2]))! :nodes (fn [g] ...)! ...})!

(nodes [g])!(edges [g])!(has-node? [g node])!(has-edge? [g n1 n2])!(successors [g] [g node])!(out-degree [g node])


How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])!(extend BasicEditableDigraph! Graph default-graph-impl! Digraph …! EditableGraph …)!


How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])!(extend BasicEditableDigraph! Graph default-graph-impl! Digraph default-digraph-impl! EditableGraph …)!


(def default-digraph-impl! {:transpose! (fn [g]! (assoc g! :succs (:preds g)! :preds (:succs g)))! …})

(predecessors [g]!! ! [g node])!

(in-degree [g node])!(transpose [g])


How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])!(extend BasicEditableDigraph! Graph default-graph-impl! Digraph default-digraph-impl! EditableGraph …)!


How to create basic editable digraph? (defrecord BasicEditableDigraph! [nodes succs preds])!(extend BasicEditableDigraph! Graph default-graph-impl! Digraph default-digraph-impl! EditableGraph editable-graph-impl)!

In this Talk

Loom Overview

Loom’s Graph

API

Functional Graph

Algorithms Your

Graphs in Loom

Functional Graph Algorithms: Bellman-Ford


CLRS Introduction to Algorithms


CLRS Introduction to Algorithms


(defn- init-estimates! [graph start]! (let [nodes (disj (nodes graph) start)! costs (interleave nodes! (repeat 1e999M))! paths (interleave nodes (repeat nil))]! [(apply assoc {start 0} costs)! (apply assoc {start nil} paths)]))




(defn- can-relax-edge?! [[u v :as edge] edge-cost costs]! (let [vd (get costs v)! ud (get costs u)! sum (+ ud edge-cost)]! (> vd sum)))


(defn- relax-edge! [[u v :as edge]! uvcost! [costs paths :as estimates]]! (let [ud (get costs u)! sum (+ ud uvcost)]! (if (can-relax-edge? edge uvcost costs)! [(assoc costs v sum)! (assoc paths v u)]! estimates)))



(defn- relax-edges! [g start estimates]! (reduce (fn [estimates [u v :as edge]]! (relax-edge! edge! (weight g u v)!

! estimates))! estimates! (edges g)))!


(defn bellman-ford!

[g start]!

(let [initial-estimates (init-estimates g start)!

;relax-edges is calculated for all edges V-1 times!

[costs paths] (reduce (fn [estimates _] (relax-edges g start estimates))!

initial-estimates!

(-> g nodes count dec range))!

edges (edges g)]!

(if (some (fn [[u v :as edge]] (can-relax-edge? edge (wt g u v) costs))!

edges)!

false!

[costs!

(->> (keys paths)!

;remove vertices that are unreachable from source!

(remove #(= Double/POSITIVE_INFINITY (get costs %)))!

(reduce!

(fn [final-paths v]!

(assoc final-paths v!

; follows the parent pointers!

; to construct path from source to node v!

(loop [node v path ()]!

(if node!

(recur (get paths node) (cons node path))!

path))))!

{}))])))


(defn bellman-ford!

[g start]!

(let [initial-estimates (init-estimates g start)!

;relax-edges is calculated for all edges V-1 times!

[costs paths]!

(reduce!

(fn [estimates _] (relax-edges g start estimates))!

initial-estimates!

(-> g nodes count dec range))!

edges (edges g)]!


(if (some (fn [[u v :as edge]]! (can-relax-edge? edge!

! ! ! ! ! (weight g u v)!! ! ! ! ! costs))!! ! edges)!

false!! !;;; return paths))) !

Functional Graph Algorithms: loom.alg-generic

• No knowledge of graph representation • Requires only successors

o + start for DFS/BFS, topological sort, Dijkstra o + end for BFS path

In this Talk

Loom Overview

Loom’s Graph

API

Functional Graph

Algorithms Your

Graphs in Loom

Your Graphs in Loom

core.async Single Static Assignment Form

Titanium-Loom

Github

And more!

Your Graphs in Loom


Titanium-Loom

Github

And more!

SSA Loom

•  Single Static Assignment (SSA) form produced by core.async

•  Generated by parse-to-state-machine function

SSA Loom

(parse-to-state-machine! '[(if (> (+ 1 2 x y) 0)! (+ x 1)! (+ x 2))])

{76 [{:value :clojure.core.async.impl.ioc-macros/

value, :id inst_4937} {:value inst_4937, :id inst_4938}], 74 [{:refs [clojure.core/+ x 1], :id inst_4933} {:value inst_4933, :block 76, :id inst_4934}], 73 [{:refs [clojure.core/+ x 1 2 y], :id inst_4930} {:refs [clojure.core/> inst_4930 0], :id

inst_4931} {:test inst_4931, :then-block 74, :else-block

75, :id inst_4932}]}}]

SSA Loom (view (ssa->loom ssa ssa-node-fn ssa-edges-fn))

if (> (+ 1 2 x y) 0)!

V (+ x 1)!V (+ x 2)

V!

SSA Loom (view (ssa->loom ssa ssa-node-fn ssa-edges-fn))!!

(defn ssa->loom! ([ssa get-nodes get-edges]!

SSA Loom (view (ssa->loom ssa ssa-nodes-fn ssa-edges-fn))!!

(reify! Graph! (nodes [g] (get-nodes ssa))! …! (successors [g node]! (->> edges! (filter (fn [[n1 n2]]! (= n1 node)))! (map second)))! …!

SSA Loom (view (ssa->loom ssa ssa-nodes-fn ssa-edges-fn))!!

Digraph! (predecessors [g node]! (->> edges! (filter (fn [[n1 n2]]! (= n2 node)))! (map first)))! …))

Your Graphs in Loom


Titanium-Loom

Github

And more!

Titanium Loom

•  Titanium by Clojurewerkz (titanium.clojurewerkz.org)

•  Built on top of Aurelius Titan (thinkaurelius.github.com/titan)

•  Storage backends: Cassandra, HBase, BerkeleyDB Java Edition

•  No graph visualization

Titanium Loom (let [in-mem-graph! (open {"storage.backend” "inmemory"})]! (tg/transact!! (let [a (nodes/create! {:name "Node A"})! b (nodes/create! {:name "Node B"})! c (nodes/create! {:name "Node C"})! e1 (edges/connect! a b)! e2 (edges/connect! b c)! e3 (edges/connect! c a)]))! (titanium->loom in-mem-graph))

Titanium Loom (view graph)

Your Graphs in Loom

Titanium-Loom


Github

And more!

Data Flow Analysis Framework

• Liveness analysis (dead code elimination) • Constant propagation • Taint analysis • Clojure Repos + Relationships

Github: Clojure Repos + Relations

•  For each Clojure project, solve system of equations until reaching fixed point:


Quil

math.numeric-tower


core.async

clojure-ring

clout > compojure

core.match

Loom’s Vision

The Graph Library in Clojure

github.com/aysylu/loom

Technology

Loom at Clojure/West