SDN Control of Packet Over Optical Networks
SDN Control of Packet Over Optical Networks
Packet1over1Op5cal$Networks$
Marc$De$Leenheer $ $ON.Lab$
Guru$Parulkar $ $Stanford$University$&$ON.Lab$
Tom$Tofigh $ $AT&T$
$
Acknowledgements$
People$and$companies$that$were$instrumental$
! Bob$Doverspike,$Tom$Anschutz,$Mar5n$Birk,$Mehran$Esfandiari$(AT&T)$
! Prajakta$Joshi$(ON.Lab)$
! Praseed$$Balakrishnan$(Fujitsu),$Eiichi$Kabaya$(NEC),$Patrick$Liu,$Wei$Wei,$Hongtao$Yin$(Huawei)$
! Saurav$Das,$et.$al.,$“Packet$and$circuit$network$convergence$with$OpenFlow,”$OFC$2010,$OTuG1$(ONF)$
! Infoblox$
! Packet/Op5cal$Use$Case$
! Problem$statement$and$conceptual$solu5on$
! Implementa5on$$
! Demonstra5on$$
! State$of$the$Industry$&$Future$Work$$
Explosive$Growth$
Growth$
TRAFFIC$
OPERATOR$COST$
REVENUES$
$
“VOICE”$ERA$$ NEW$SERVICES$
$
$
$
“DATA”$ERA$
$
*$Graph$Source$1$Accenture$Analysis$
Service Provider Networks Time$
Lower$opera5onal$complexity,$
increase$visibility$
Key$Enabler:$Solware$Defined$Networking$
Features$ $$$$$$$$$$$SDN$Network$Opera5ng$System$
Control$plane$ Closed$
Hardware$
Agent$
OS$
Loader$
Merchant$Silicon$
Whitebox$ Legacy$
Service$Provider$Networks$
! WAN$core$backbone$
! Mul51Protocol$Label$Switching$(MPLS)$with$Traffic$Engineering$(TE)$
! 200#500%routers,%5#10K%ports%
! Metro$Networks$
! Metro$cores$for$access$networks$
! 10#50K%routers,%2#3M%ports%
! Cellular$Access$Networks$
! LTE$for$a$metro$area$
! 20#100K%devices,%100K#100M%ports%
! Wired$access$/$aggrega5on$
! Access$network$for$homes;$DSL/Cable$
! 10#50K%devices,%100K#1M%ports%
Service$Provider$Network$of$the$Future$
Core$Packet1Op5cal$
Metro$Packet1Op5cal$
Wired& Wireless& Enterprise& Wireless& Wired& Wireless& Wired& Wireless& Enterprise&& Wireless&
Access$ Access& Access& Access& Access& Access& Access& Access& Access& Access& Access&
SDN$Control$Plane:$Key$Performance$Requirements$
! High$Throughput:$
! ~500K11M$paths$setups$/$second$
Apps$
! ~316M$network$state$ops$/$second$
! High$Volume:$
$$$ONOS$ Global$Network$View$/$State$
! ~500GB11TB$of$network$state$data$
! Difficult$challenge!$
high&throughput&&|&&low&latency&&|&&consistency&&|&&high&availability&
ONOS:$SDN$Network$OS$for$Service$Providers$
! Scalability,$High$Availability$&$Performance$
! Northbound$&$Southbound$Abstrac5ons$
! Modularity$
ONOS:$Distributed$Network$OS$
Apps$
NB$–$Applica5on$Intent$Framework$$
Southbound$Core$API$
Applica5on$Intent$Framework$
Flexible$and$intui5ve$northbound$
abstrac5on$and$interface$for$user$or$
app$to$define$what$it$needs$without$
“Provision$10G$path$from$
worrying$about$how .$ Datacenter$1$to$
Datacenter2$op5mized$
for$cost”$
Applica5on$Intent$Framework:$$
APIs,$Policy$Enforcement,$Conflict$resolu5on$$
Intents$translated$and$$
compiled$into$specific$
instruc5ons$for$network$$
Distributed$Core$
Distributed$Core$$
devices.$
Southbound$
Southbound$Core$API$
! Segment$rou5ng$with$SDN$control$$
! Central$Office$Re1architected$as$a$Datacenter$(CORD)$
! Network$Func5ons$as$a$Service$(NFaaS)$
! vCPE$and$vOLT$
! And$many$more$
! Mobile$backhaul$(IP$RAN)$
! IP$mul5cast$
! …$
Seamless$Peering:$SDN1IP$
SDNCIP&enables&communicaFon& ONOS Cluster
between:& • ONOS/SDNCIP&HA&
• SDN&network&and&&
&&&&&&&external&IP&networks&& • BGP&speaker&HA&
• external&networks&across& • External&BGP&router/
SDN&island& connecFon&HA&
SDN1IP$ SDN1IP$ SDN1IP$
Seamless$peering$of$SDN$islands$with$exis5ng$networks$
=$
Migra5on$strategy$for$real$networks$
Segment$Rou5ng$
Requests$
Rou5ng$Service$ Rou5ng,$
Recovery,$
Discovery$ Forwarding$
Requests$ Label$imposi5on$
Service$ Service$
ONOS
OpenFlow$1.3$
Open$Segment$
SR$Labels$imposed$by$ Open$
Routers$(OSR)$
controller$
Segment$
OSR$FIB$built$by$controller$
Routers$
(OSR)$
Outline$
! SDN$for$Service$Providers$
! Background$
! Use$cases$
! Packet/Op5cal$Use$Case$
! Problem$statement$and$conceptual$solu5on$
! Implementa5on$$
! Demonstra5on$$
! State$of$the$Industry$&$Future$Work$$
Problem$Statement$
! Today$IP$packet$and$transport$networks$are$separate.$
! They$are$planned,$designed$and$operated$separately$by$different$teams.$$$
! This$leads$to$significant$inefficiencies.$
! They$are$subject$to$under1u5lized$networks$with$significant$pre1planning$
and$highly$over1provisioned$for$worst$case.$
! A$lot$of$the$path$planning$in$these$networks$is$off1line.$
Given$ these$ considera5ons,$ WAN& links& are& typically& provisioned& to& 30C40%& average& uFlizaFon.$
This$ allows$ the$ network$ service$ provider$ to$ mask$ virtually$ all$ link$ or$ router$ failures$ from$ clients.$
Such$overprovisioning$delivers$admirable$reliability$at$the$very$real$costs$of$2C3x&bandwidth&overC
provisioning$and$high1end$rou5ng$gear.$
$
S.%Jain,%et.%al.,%“B4:%Experience%with%a%Globally#Deployed%SoMware%Defined%WAN,”%SIGCOMM%2013.%
Mul51Layer$Network$without$Converged$Control$Plane$
C& IP IP
E&
Logical&Tunnels& A&
Full&Mesh&MPLS& IP
B& IP
IP
D&
P3& P5&
IP IP
P1&
IP&Routers&
P4&
IP
P2& IP
IP
R4& R7&
R2&
R1&
ROADM
ROADMs&
ROADM ROADM
ROADM ROADM
100s&of&
R5& R6& wavelengths&&
R3&
ROADM
ROADM
per&ROADM&
Mul51Layer$Network$without$Converged$Control$Plane$
C& E&
A& 50C100G& IP IP
IP
50C100G& B& IP
IP
D&
P3& P5&
50C100G& IP IP
100G& P1&
P4&
IP IP
P2&
50C100G& IP
IP
R4& R7&
Peak$rate$provisioning$is$
R2& 100G&
100G&
R1& necessary$in$op5cal$transport$
ROADM
ROADM ROADM
ROADM ROADM
ROADM
Mul51Layer$Network$without$Converged$Control$Plane$
C& IP IP
E&
A&
IP
B& IP
IP
D&
Mul5ple$protec5on$modes$are$
applied$(up$to$4$5mes$BW)$
P3& P5&
Primary& IP IP
P1&
P4&
IP IP
P2&
Protected& IP
IP
R4& R7&
Light&Paths&&
R2&
R1& Sta5c$provisioning$in$
ROADM
ROADM ROADM
ROADM ROADM
transport$networks$
Light&&Paths& R5& R6&
R3&
ROADM
ROADM
Conceptual$Solu5on:$Mul51Layer$SDN$Control$
BW$ 1. Centralized Control of packet and optical
Calendaring$$ Control&Apps& Config&Apps& Mgmt&Apps& 2. Multi-layer optimization based on availability,
$ economics and policies
ONOS
Datacenter 2
Datacenter 1
Packet$Network$
Op5cal$Network$
Benefits$of$Converged$Control$Plane$
! Much$faster$bandwidth$provisioning$
! Dras5cally$improve$network$u5liza5on$
! Perform$dynamic$restora5ons$in$response$to$packet$and$
transport$network$failures$$
! Agile$development$and$rapid$deployment$of$new$services$
Implementa5on$
! Code$is$king$
! Less$is$more$
! Vendor$neutral$
! Scalability,$high$availability,$performance$
! Work$focused$on$the$three$SDN$layers$
! Data$plane$
! Control$plane$
! Applica5ons$
Implementa5on$–$Data$Plane$
Packet&Switches& ROADMs$
! Open$and$standardized$ ! Open$and$standardized$
interface$to$forwarding$plane?$ interface$to$forwarding$plane?$
! Reality$ ! Reality$
! OpenFlow$ ! Legacy$protocols$such$as$TL1$
! Available$today$in$many$products$ ! Vendor$specific$
Built$an$op5cal$emula5on$plaxorm$LINC1OE$
with$our$partner$Infoblox$
hyps://github.com/FlowForwarding/LINC1Switch$
ROADM$Emula5on$Basics$
! Emulates$op5cal$layer$topology$from$predefined$table$
! Includes$characteris5cs$of$op5cal$cross$connect$and$Packet$to$Op5cal$Link$Interface$(Add/Drop)$$
! Ports,$links$and$switches$are$remotely$reconfigurable$by$Mininet$$
! Supports$OpenFlow$1.3+$$Op5cal$Add/Drop$match$ac5ons$
! Supports$failure$scenarios$of$links,$ports,$and$ROADM$
! Work$in$progress$
! Emulates$channel$signal/power$measurement$
! Regenerator$support$ ROADM ROADM
Module Module
O
P
M
DROPs ADDs
Forwarding$Model$for$ROADMs$
! Match/ac5on$abstrac5on$for$ROADMs$
! ROADM$has$three$func5ons:$add,$drop,$and$forward$
! Match$is$really$about$wavelength$provisioning$
packets& packets&
ROADM ROADM
Add& ROADM
Drop&
Match& Packet$port$and$traffic$type$ Match& Op5cal$port$and$lambda$
Ac5on& Transponder$uses$lambda$and$ Forwarding& Ac5on& Transponder$uses$lambda$and$
output$to$op5cal$port$ output$to$packet$port$
Match& Op5cal$port$and$lambda$
Ac5on& Output$to$op5cal$port$
(easy$to$extend$when$
considering$regenerators)$
Forwarding$Model$for$Packet$and$ROADM$Layer$
IP IP
ROADM ROADM
IP&Forwarding Constraints
IP&Forwarding Constraints
Match Ac5on Match Ac5on Egress Values
ROADM Match Ac5on Match Ac5on Egress Values
Tuple$ Encaps$IP$ IP,$DESTxxx$ IP,$Destxxx$ Pop$IP$Packet$ Tuple& Forward$to$
on$port3& DESTxxx$ & Capacity& 10GbE& on$port3& & port12$ Capacity& 10GbE&
& &
LAG& No& LAG& No&
Cost& 10& Cost& 10&
Metering& OAM&
Metering& OAM&
Packets$ Delay$ ROADM ROADM Packets$ Delay$
dropped$ BFD& Jiyer$ dropped$ BFD& Jiyer$
Total$ Loss$ Total$ Loss$
packets$
packets$
Queue$
Port& Queue$
overflow$ Port&
overflow$
status$ ROADM
status$
Queue$
Queue$
Overflow$
Overflow$
count$
count$
Implementa5on$–$Control$Plane$
! Southbound$protocol$for$ROADMs$
! ONF$Op5cal$Transport$Working$Group$
! OpenFlow$1.3+$experimenter$messages$ Control$Apps$ Config$Apps$ Mgmt$Apps$
! Southbound$abstrac5ons$simplify$adding$new$protocols$
! Discovery$
! Automa5c$L3$topology$discovery$(LLDP)$
SDN$Network$
! Sta5c$configura5on$of$L0$topology$
Opera5ng$System$
! L0$discovery$work$in$progress$
OpenFlow$
Implementa5on$–$Control$Plane$
! Converged$topology$
! Control$both$packet$and$op5cal$layers$
! Allows$adding$addi5onal$layers,$e.g.,$OTN$ Control$Apps$ Config$Apps$ Mgmt$Apps$
! Path$calcula5on$takes$place$on$the$mul51layer$graph$
! Constraints$and$resource$management$
! Wavelength$con5nuity,$bandwidth,$latency,$…$
SDN$Network$
! Packet$and$op5cal$layer$restora5on$ Opera5ng$System$
! First$try$packet$layer,$then$op5cal$layer$
OpenFlow$
! Easily$add$mul51layer$protec5on$and$restora5on$mechanisms$
Implementa5on$–$Applica5ons$
Mul51Layer$ Bandwidth$
GUI$ on$Demand$ Bandwidth$
Calendaring$
Applica5on$Intent$Framework:$$
APIs,$Policy$Enforcement,$Conflict$resolu5on$$
Distributed$Core$
Distributed$Core$$
Southbound$
Southbound$Core$API$
Southbound$
OpenFlow$ NETCONF$
Interface$
Mul51Layer$Network$Reference$Plaxorm$$
! “Packet/Op5cal$Network$in$a$Box”$
! Open$Source$components$
ONOS&
! Data$plane$
Mininet&
! LINC1OE$&$OVS$
! Mininet$ OVS&
H P S1$ S2$ P
! ONOS$ 1$ P P
H2$
! Scalability$tes5ng$
! Control$plane$interoperability$between$vendors$
Demo$GUI$
Try$it$yourself$
hyps://wiki.onosproject.org/display/ONOS/Packet+Op5cal$
Demo$
hyps://www.youtube.com/watch?v=QA9ECsKpSug$
Lessons$Learned$
! Feasibility$
! Converged$packet$op5cal$control$plane$is$possible$
! Offers$scalability,$HA,$and$performance$
! Benefits$
! Significant$improvement$in$network$u5liza5on$
! Dras5c$reduc5on$in$CAPEX$and$OPEX$$
! DevOps$model$for$transport$networks$
! Deeper$insights$
! OpenFlow$packet$switches$commercially$available,$resistance$from$L0$vendors$
! Abstrac5ons$are$cri5cal:$intent$framework,$mul51layer$graph$
Customer$Facing$Services/Apps$$
! Bandwidth$on1demand,$bandwidth$calendaring$
! Customer$portals$for$zero$touch$service$provisioning$
! On1demand$and$dynamic$virtual$private$networks$
! Elas5c$bandwidth$services$
! And$so$on…$
Outline$
! SDN$for$Service$Providers$
! Background$
! Use$cases$
! Packet/Op5cal$Use$Case$
! Problem$statement$and$conceptual$solu5on$
! Implementa5on$$
! Demonstra5on$$
! State$of$the$Industry$&$Future$Work$$
Ver5cal$Integra5on:$Packet$Switches$
Features$
Control$Apps$ Config$Apps$ Mgmt$Apps$
lane$
Control$p$$$$$$$$$$$SDN$Network$Opera5ng$System$
Features$
Control$plane$ Closed$
Hardware$
Agent$
Packet$switches$are$undergoing$ Hardware$ OS$
this$transforma5on$right$now!$ Loader$
Merchant$Silicon$
Whitebox$ Legacy$
Ver5cal$Integra5on:$ROADMs$
ROADM&controller&
Control$Apps$ Config$Apps$ Mgmt$Apps$
Control$and$config$of$
WSS$and$transponders$
Signal$Monitoring$and$Adjustment$
Metering$and$alarms$
$$$$$$$$$$$SDN$Network$Opera5ng$System$
demux$ mux$
fiber$
Why$is$this$de1aggrega5on$
not$happening?$
pass$through$
add/drop$
Agent$ controller&
OS$
WSS$ HAL$
transponder$ Hardware$
Whitebox$ Legacy$
What$Makes$Op5cal$Devices$Different?$
! “We$need$specialized$mix$of$L0,$L1,$and$L2$func5ons”$
! “Physical$impairments$are$too$complex$to$monitor$and$manage$externally”$
! “Our$analog$transmission$system$is$custom$designed”$
! “Every$vendor$has$his$own$DSP$which$is$proprietary$and$without$
programmable$dynamics$”$$
! “It’s$impossible$to$control$all$configura5on$and$forwarding$at$scale”$
! “You$can’t$achieve$sub150ms$failovers”$
! And$so$on…$ None$of$this$is$fundamental!$
$
De1aggrega5on$is$inevitable$
Open$Op5cal$Hardware$
! Hardware$Abstrac5on$Layer$
! Hides$op5cal$impairments,$thermal$instability,$power$balancing,$etc.$
! Can$autonomously$fix$problems$or$perform$maintenance$
! OS$
Agent$ controller&
! Server1like$environment$for$switches$ OS$
! Manages$various$hardware$sensors$ HAL$
! Boot$loader,$u5ls,$switch$management,$etc.$ Hardware$
Whitebox$ Legacy$
! Agent$
! Open$and$standardized$interface$for$forwarding,$configura5on,$and$observability$
Invi5ng$all$vendors$to$join$us!$
It's$Happening$Now!$
CALIENT’s S-series Optical Circuit Switch (OCS)
! Up#to#320#User#Ports#–#640#Single#Mode#
Fiber#Termina<ons#
• 320x320,'160x160'op,ons'
! 10,#40,#100#Gbit/s#per#port#and#beyond#
! 25ms#typical#setup#<me#(<50ms#Max)##
! Less#than#50ns#latency#
! Less#than#3.5#dB#Inser<on#Loss#
! Ultra#low#power#(<45w),#small#size#(7RU)#
! TL1,#SNMP,#OpenFlow#APIs#
5 CALIENT Technologies – All Rights Reserved – Company Confidential Lumen Networks - Open Optical Architecture
APPLICATIONS
Vendor1Specific$Domains$
! Second$problem$with$Op5cal$Transport$Industry$
! Transport$networks$suffer$from$vendor$lock1in$
! Domain$consists$of$equipment$from$a$single$vendor$
! Each$domain$requires$vendor1specific$NMS/EMS$
! No$data$plane$interoperability$
! Profound$impact$on$service$providers$
Service$Provider$Orchestra5on$
! Complex$management$&$orchestra5on$tools$
! Problem$iden5fica5on$&$resolu5on$
! Expensive$
NMS$A$ NMS$B$
! Is$this$fundamental?$
Vendor&A& Vendor&B&
Why$Vendor1Specific$Domains?$
! “We$monitor$network$state$and$performance$in$NMS”$
! “We$built$intelligent$alarm$and$event$handling$between$boxes$and$EMS”$
! “Our$EMS$is$the$only$system$that$can$control$our$transmission”$
! “Failures$are$handled$faster$and$more$efficiently$by$our$NMS”$
! And$so$on…$
$
None$of$this$is$fundamental!$
$
Vendor1specific$domains$will$disappear$
Vendor1Neutral$Domains$
Control$Apps$ Config$Apps$ Mgmt$Apps$
SDN$Network$Opera5ng$
System$
Common$southbound$abstrac5ons$
X$ X$
X$
Data$plane$ Vendors$can$s5ll$innovate$and$
interoperability$is$key$ X$ X$ diversify$their$hardware$
Vendor$A$
Vendor$B$
Future$Work$
! Looking$to$work$with$vendors$that$offer$OpenFlow$support$
! Something$beyer$than$vendor1specific$TL1$
! Experiments$on$data$plane$interoperability$
! Drive$adop5on$of$DevOps$model$for$transport$networks$
! Hardware$deployments$
Proof$of$Concept$ Bandwidth$
Bandwidth$$ Dynamic$
Mul51layer$GUI$ calendaring$
on1demand$ restora5on$
MulFCinstance&ONOS&
Ciena$TL1$ Fujitsu$TL1$ Huawei$PCEP$
OF$provider$
provider$ provider$ provider$
Traffic$ Traffic$
generator$ generator$
SDN&SoluFons&Showcase&
Open&Networking&Summit&2015&
June&15C18,&Santa&Clara,&CA&
Cap1Grow1Drain$Strategy$
Segment$Rou5ng$ Op5cal$control$ Segment$Rou5ng$ Op5cal$control$ Segment$Rou5ng$ Op5cal$control$
(for$MPLS$network)$ (for$MPLS$network)$ (for$MPLS$network)$
ONOS $ ONOS $ ONOS
MPLS$
MPLS$ MPLS$
Network$
Network$ Network$ Whitebox$
Whitebox$ Whitebox$ switches$
Drain&
Whitebox$
switches$ Whitebox$ Whitebox$ switches$ switches$
Cap&
switches$ switches$
Op5cal$
New$ Op5cal$
Network$ Op5cal$
SDN$ Network$
Edge$ Network$
Grow&
Route$Big$Flows$to$
CapCGrowCDrain&
op5cal$network$ =&
Bring&SDN&to&backbone&without&fork&lib&upgrade&
Cap$MPLS$backbone$–$don’t$grow$the$ Grow$packet$edge$and$op5cal$core$with$ Drain$the$MPLS$backbone$as$
legacy$MPLS$backbone$of$proprietary$ SDN$control$plane$and$make$the$best$ most$traffic$transi5ons$to$new$$
routers$$ use$of$packet1op5cal$technologies$$ packet$edge$and$op5cal$core$
network$$
Summary$
! Demonstrated$converged$packet/op5cal$control$plane$for$
service$providers$
! Scalability,$HA,$performance$
! Poten5al$to$drama5cally$decrease$CAPEX$&$OPEX$
! Innova5ve$services$using$DevOps$model$
BW$ Control& Config& Mgmt&Apps&
1. Centralized Control of packet
Calendaring$$ Apps& Apps& and optical
! Need$the$right$abstrac5ons$ $ ONOS 2. Multi-layer optimization
based on availability,
economics and policies
! Intent$framework$ Datacenter 2
Datacenter 1
! Mul51layer$graph$ Packet$
Network$
Optical circuit re-routed
Op5cal$
Network$
Call$to$Ac5on$
! Open$and$standardize$hardware$interfaces$
! Achieve$control$plane$interoperability$
! Achieve$L0$data$plane$interoperability$ SDN$Network$
Opera5ng$System$
! Remove$vendor1specific$approaches$(EMS$&$NMS)$
X X
Agent$ controller&
X
OS$
HAL$ X X
Hardware$
Whitebox$ Legacy$
! If$exis5ng$vendors$don’t$take$ac5on,$others$will$step$in!$