Communications of the ACM

Research highlights

SkyCore: Moving Core to the Edge for Untethered and Reliable UAV-Based LTE Networks

By Mehrdad Moradi, Karthikeyan Sundaresan, Eugene Chai, Sampath Rangarajan, Z. Morley Mao
Communications of the ACM, January 2021, Vol. 64 No. 1, Pages 116-124
10.1145/3434161
Comments

The advances in unmanned aerial vehicle (UAV) technology have empowered mobile operators to deploy LTE (long-term evolution) base stations (BSs) on UAVs and provide on-demand, adaptive connectivity to hotspot venues as well as emergency scenarios. However, today's evolved packet core (EPC) that orchestrates LTE's radio access network (RAN) faces fundamental limitations in catering to such a challenging, wireless, and mobile UAV environment, particularly in the presence of multiple BSs (UAVs). In this work, we argue for and propose an alternate, radical edge EPC design, called SkyCore that pushes the EPC functionality to the extreme edge of the core network—collapses the EPC into a single, lightweight, self-contained entity that is colocated with each of the UAV BS. SkyCore incorporates elements that are designed to address the unique challenges facing such a distributed design in the UAV environment, namely the resource constraints of UAV platforms, and the distributed management of pronounced UAV and UE mobility. We build and deploy a fully functional version of SkyCore on a two-UAV LTE network and showcase its (i) ability to interoperate with commercial LTE BSs as well as smartphones, (ii) support for both hotspot and stand-alone multi-UAV deployments, and (iii) superior control and data plane performance compared to other EPC variants in this environment.

Back to Top

1. Introduction

Mobile LTE (long-term evolution) networks that are ubiquitous today are deployed after sufficient RF planning in a region. However, the static nature of LTE base station (BS) deployments limits their ability to cater to certain key 5G use cases—surging traffic demands in hotspots (e.g., stadiums and event centers), as well as their availability in emergency situations (e.g., natural disasters), where the infrastructure could itself be compromised. Providing an additional degree of freedom for base stations, namely mobility, allows them to break away from such limitations.

UAV driven mobile networks. Advances in unmanned aerial vehicle (UAV) technology have empowered operators to take on-demand, outdoor connectivity to another level, by allowing their base stations to be deployed aerially on UAVs (Figure 1), thereby offering complete flexibility in their deployment and optimization. Mobile operators such as AT&T and Verizon have both conducted trials with LTE base stations mounted on UAVs^9,8 (helicopter and fixed-wing aircraft, respectively). AT&T also provided LTE network services from its UAV in the aftermath of hurricane Maria in Puerto Rico last year.³ Further, with the availability of shared access spectrum such as CBRS² in 3.5 GHz, this also opens the door for smaller, greenfield operators to deploy and provide on-demand, private LTE connectivity services without the heavy cost associated with spectrum and deployment.

Figure 1. UAV-based LTE networks.

Limitations of the legacy EPC. A typical mobile cellular network requires the deployment of two essential components: a radio access network (RAN) consisting of multiple base stations (BSs) that provide wide-area wireless connectivity to clients (UEs) and a high-speed, wired core network of gateways (evolved packet core, EPC) that sits behind the RAN and is responsible for all the mobility, management, and control functions, as well as routing user traffic to/from the Internet. Realizing a multi-UAV-driven RAN (BSs deployed on UAVs) with an EPC on the ground or in the cloud is one way to directly apply today's EPC architecture to the UAV environment (as shown in Figure 2). Based on publicly available information,^3,8,9 this has been the case with the current operator-driven UAV efforts. However, this faces significant limitations in delivering real value to this challenging environment. Specifically, although a tethered setup (EPC-UAV link being wired, Figure 2a) significantly limits the UAV's mobility and ability to scale to multiple UAVs, a wireless setup (EPC-UAV link being wireless/mobile, Figure 2b) incurs all the vagaries of the wireless channel. For the latter, the choice of the wireless technology becomes critical given that the EPC is responsible for setting up, routing, and tearing down all voice/data bearers. It is essential for the EPC to reliably reach all the UAVs wirelessly, such as those that are potentially far away in the presence of non-line-of-sight conditions (e.g., buildings, foliage, etc.). Further, it must deliver sufficient capacity to support the traffic demands in the RAN. It is extremely challenging for a wireless technology, be it lower frequency (sub-6 GHz such as LTE, WiFi, etc.) or higher frequency (mmWave, satellite), to simultaneously satisfy the needs of range, reliability/robustness, and capacity that the UAV environment demands from the critical EPC-RAN link.

Figure 2. Legacy EPC variants for UAV-based LTE networks.

Core at the edge. Given the fundamental limitations in deploying an EPC on the ground or in the cloud to support a multi-UAV RAN, we advocate for a radical, yet standards-compliant redesign of the EPC, namely the Edge-EPC architecture, to suit the UAV environment. As the name suggests, we aim to push the entire EPC functionality to the extreme edge of the core network, by collapsing and locating the EPC as a single, lightweight, self-contained entity on each of the UAVs (BSs) as shown in Figure 3. Being completely distributed at the very edge of the network, such an architecture completely eliminates wireless on the critical EPC-RAN path and hence the crippling drawbacks faced by the legacy architecture in this environment.

Figure 3. Edge-EPC for UAV-based LTE networks.

Although definitely promising at the outset, realizing this radical design is not without its own set of challenges that are unique to the UAV environment. In particular, (i) Resource-challenged environment: The compute resources consumed by the numerous network functions in the EPC are appreciable and become a concern when all the EPC functionality is placed into a single node and deployed directly on a UAV platform—the latter being highly resource-challenged to begin with. This could significantly affect both the UAV's operational lifetime and the processing (control and data plane) latency of its traffic (see Figure 4), thereby resulting in a reduced traffic capacity, (ii) Mobility management: The hierarchical nature of the legacy EPC architecture, gives a single network gateway (such as the mobility management entity, the MME) a consolidated view of multiple BSs, thereby allowing it to efficiently manage handoffs during mobility of active UEs as well as tracking/paging mobile UEs that are in idle mode. Mobility of both active (handoffs) and idle UEs (tracking/paging) becomes a critical challenge, when the entire EPC is located at each of the UAVs, thereby restricting their view of events to only those that are local to the UAV. The frequency of such events is further exacerbated by the mobility of the UAVs.

Figure 4. Edge-EPC has high overheads on UAV works.

Our proposal: SkyCore. Toward our vision of building untethered yet reliable UAV-based mobile networks, we present our novel EPC design, SkyCore. SkyCore embodies the Edge-EPC architecture while introducing two key pillars in its design to address the associated challenges: a complete software refactoring of the EPC for compute-efficient deployment on a UAV and a new inter-EPC communication interface to enable fully functional operation in a multi-UAV environment. Through software refactoring, SkyCore eliminates the distributed EPC interfaces and collapses all distributed functionalities into a single logical entity (agent) by transforming the latter into a series of switching flow tables and associated switching actions. It also reduces control plane signaling and latency by precomputing and storing (in-memory) several key attributes (security keys, QoS profile, etc.) for UEs that can be accessed quickly in real time without any computation. To ensure complete EPC functionality, SkyCore manages mobility right at the edge of the network—it enables a new control/data interface through software-defined networking (SDN) to realize efficient inter-EPC signaling and communication directly between UAVs. This allows the SkyCore agents on each UAV to proactively synchronize their states with each other, thereby avoiding the real-time impact of wireless (UAV-UAV) links on critical control functions—results in fast and seamless handoff of active-mode UEs as well as tracking of idle-mode UEs across multiple UAVs.

Real-world prototype: We have built a complete version of SkyCore on a single board server with a small compute and energy footprint and deployed it on DJI Matrice 600 Pro rotary-wing drones to create a two-UAV LTE network. To the best of our knowledge, this is the first realization of a self-contained Edge-EPC solution that can support a multi-UAV network and is a direct affirmation of SkyCore's design. SkyCore's feasibility and functionality are validated by seamless integration and operation with a commercial LTE RAN (BS) from ip.access and off-the-shelf UEs (Moto G and Nexus smartphones). We demonstrate SkyCore UAVs to operate both as LTE hotspots that allow for better UE connectivity to the Internet by extending coverage of a terrestrial LTE network, as well as stand-alone LTE networks for connectivity of geographically separated UEs through two different UAVs (e.g., first responders in emergency scenarios), whereas also allowing for handoffs. Our real-world evaluations of SkyCore and its comparison with a state-of-the-art software EPC (OpenEPC⁶) on UAV clearly showcase SkyCore's superior performance and scalability—SkyCore provides an order of magnitude lower control plane latencies, incurs 5X lower CPU utilization, and provides data plane rates that currently scale up to a Gbps.

Our two key contributions in this work include the following:

• A novel Edge-EPC solution, SkyCore that can reliably and scalably support a multi-UAV LTE network deployment that was not possible earlier
• A real-world implementation and evaluation that show-case both its feasibility and its superior performance

Broader implications: SkyCore's underlying design is driven by the observation that when connectivity between core network functions, which are on the critical path, is unreliable (wireless and mobile), the merits of pushing functionality to the edge of the network significantly outweigh the associated drawbacks. Hence, although designed for a multi-UAV environment, SkyCore's design can also benefit other deployments, where distributed critical functions have to communicate over unreliable links (e.g., distributed enterprise RANs). Further, adopting an SDN-based design, SkyCore is equally applicable to future RAN technologies such as 5G and 6G.

Back to Top

2. Skycore: Design Overview

 2.1. Background on legacy EPC

Evolved packet core (EPC, Figure 5) is a distributed system of different nodes, each consisting of diverse network functions (NFs) that are required to manage the LTE network. The EPC consists of data and control data planes: the data plane enforces operator policies (e.g., DPI, QoS classes, and accounting) on data traffic to/from the user equipment (UE), whereas the control plane provides key control and management functions such as access control, mobility, and security management. eNodeBs or eNBs (RANs) are grouped into logical serving areas and connected to serving gateways (SGWs). The SGW is connected to an external packet network (e.g., the Internet) via the packet data network gateway (PGW). The PGW enforces most of the data plane policies (e.g., NAT and DPI) and may connect the core to other IP network services (e.g., Web servers). The EPC forwards each UE's data traffic between the eNodeB and PGW using a separate GTP-U (GPRS tunneling protocol) tunnel. The mobility management entity (MME) is responsible for access control and enforcement, as well as security and mobility functions (e.g., attach/detach and paging/handoff) in conjunction with the HSS (home subscriber server) database and PCRF (policy and charging rules function).

Figure 5. Legacy EPC architecture.

In contrast to legacy EPC, SkyCore adopts the Edge-EPC architecture as shown in Figure 6. SkyCore collapses the entire EPC and pushes it to the edge of our network, namely at each of the UAVs themselves, where it is colocated with the RAN. Although this completely eliminates wireless from the critical path between the EPC and RAN, to address the challenges associated with the Edge-EPC architecture, SkyCore introduces two novel design components, which are briefly explained here:

Figure 6. The SkyCore network architecture.

Software refactoring of the EPC functionality: To reduce its compute footprint on the UAV, SkyCore adopts a software refactoring approach to eliminate distributed EPC interfaces and collapse all distributed functionalities (Figure 5) into a single logical entity. It realizes this by transforming the distributed data plane functions into a series of switching flow tables and associated switching actions (corresponding to functions such as GTP-U encapsulation/ decapsulation, charging, etc.). It also reduces control plane signaling and latency by precomputing and storing (in-memory) several key attributes relating to security keys, QoS profile, etc. for UEs that can be accessed locally in real time without any computation.

Efficient inter-EPC communication: With every UAV now running its own EPC agent, even a simple eNB-eNB handoff of an active UE across two UAVs now becomes an inter-MME handoff, which needs to be accomplished across two different EPC agents. SkyCore enables a new control/data interface that allows agents on different UAVs to proactively (in the background) synchronize the state of UEs. This bypasses the real-time impact of wireless (UAV-UAV links) on critical control path functions, allowing for seamless handoffs and tracking of idle-mode UEs right at the edge. The HSS equivalent in each SkyCore agent maintains the location (anchoring SkyCore agent) of all UEs in the network. Hence, when an agent sends a UE location update, the agents in other UAVs update their HSS accordingly. Thus, whenever traffic needs to be sent from a SkyCore agent to a UE located at another UAV, the HSS will reveal the destination SkyCore agent at which the UE is anchored and to whom the traffic has to be routed. The actual routing path taken by the traffic on the mesh backhaul is then determined by SkyCore, with the underlying backhaul topology information made available by a backhaul agent that resides on the UAV.

 2.2. Software refactoring of EPC

Each SkyCore agent has a minimalist and UAV-aware SDN-based architecture (Figure 7), consisting of a controller that executes the control functions to process UEs' signaling traffic and to coordinate with other agents, and a switch that processes user data traffic. In the following, we describe six high-level steps that we take to refactor and extend the EPC functionality onto our agent architecture.

Figure 7. SkyCore refactors the EPC functionality into a lightweight software-driven agent having new interfaces for interaction with the local UAV and other UAVs.

Step 1. Decoupling the EPC control and data plane pipelines. One of the main reasons behind the high complexity and overhead of the EPC is its nodes performing mixed control and data plane functions. To make the EPC functionality suitable for UAVs, we first decouple the EPC control and data planes. Among the EPC nodes, the MME, PCRF, and HSS are pure control nodes. Hence, our decoupling does not affect these elements, and only affects the SGW and PGW. The resulting control components from the decoupling are the PGW-C, SGW-C, MME, PCRF, and HSS, and the data elements include the SGW-D and PGW-D (C stands for control and D for data). Although the benefits of decoupling control and data planes have been articulated before,¹⁹ we apply it in the context of UAV networks and enhance it substantially with the following mechanisms.

Step 2. Categorizing the functionality of the EPC control plane. Next, we categorize the EPC control nodes based on their high-level functionality. In our decoupled EPC, there are three types of nodes: (1) the SGW-C and PGW-C are responsible for managing QoS policy enforcement on and routing of user data traffic, (2) the MME exchanges signaling traffic with UEs and eNBs, and (3) the PCRF and HSS dynamically generate network security and QoS policies for the other nodes. To compress the EPC functionality, we consolidate the nodes in each category on top of our agent controller and remove the EPC-distributed protocols as follows.

Step 3. Collapsing the SGW-C, PGW-C, and MME into lightweight SDN applications. We extract the internal functions in the SGW-C and PGW-C and refactor them into a single SDN application, LTE Policy Application, on top of the controller. We do the same process for the MME and transform it into LTE Mobility Application. One notable aspect of this consolidation is that we naturally eliminate the complex GTP-C protocol, its six interfaces, and continuous control messages from the core network (Figure 5). This makes the SDN applications extremely lightweight and extensible without hurting their original functionality. Note that these applications still exchange information with each other but through simple local publish-subscribe mechanisms.

Step 4. Eliminating the HSS and PCRF from the core and replacing them with a precomputed policy data store. Next, we focus on the HSS and PCRF that are known to be the source of today's signaling storms in cellular networks.^5,7 The HSS stores hundreds of database tables containing different UEs' states often on disk. Moreover, it acts as a proxy between the MME and these tables, and performs different types of complex security and location tracking computations. The PCRF often accesses a logical database (sometimes implemented in the HSS) and dynamically generates different QoS and charging policies for UEs. In SkyCore, we completely eliminate these two nodes from our agents and show that dynamic policy generation can be carefully replaced with a precomputed in-memory policy data store (see Figure 9). Precomputation combined with in-memory transactions substantially minimizes the overhead of the core on resource-challenged UAVs. This also removes the complex Diameter protocol (Figure 5) from the core.

Figure 9. (a) SkyCore's network-wide control and data plane connectivity for LTE UAV networks, (b) Example of our segment routing for data traffic from UE1 to UE2.

Step 5. Adding UAV-specific SDN applications to the core. One of the key differences between SkyCore and the traditional EPC is in its continuous interaction with the UAV hardware and its APIs. In particular, we advocate for two new applications on top of our agents. Each SkyCore agent runs UAV Control Application that listens to flight change events from UAV and remaining battery resources on the UAV. This is necessary for our agents to properly handoff UEs to each other, for example, when a UAV needs to immediately leave the network for recharging. Such use cases clearly show the potential of our SDN-based UAV-aware architecture. In addition, we design an inter-agent (UAV) communication application (Section 2.3) that exchanges control plane messages with its neighbor agents to synchronize states proactively, thereby enabling seamless mobility (active and idle). The legacy EPC applications and new SkyCore core applications that need to exchange information with each other do so through our local publish-subscribe protocols.

Step 6. Replacing the hierarchical data plane gateways with a compact SDN switch. Because SkyCore is a flat architecture, it eliminates the need for hierarchical gateways on each UAV. To further make our agents compact, we refactor the SGW-D and PGW-D functionality into a single software switch. Each data plane function in S/PGW-D is implemented as a separate Match+Action table in this software switch. Each table performs a lookup on a subset of user's data traffic fields and applies the actions corresponding to the first match. Users' traffic travels through these tables before leaving or entering the UAV. In particular, our software switch performs UL/DL data rates enforcement, stateful firewall operations, and QoS control by transport-level mechanisms (e.g., setting DiffServ) based on QoS class identifier (QCI) associated with each UE. Although the legacy EPC tunnels each UE's traffic into two tunnel segments across the RAN, PGW-D, and SGW-D, SkyCore departs from this approach and terminates GTP-U tunnels inside our agent switch (decapsulates GTP-U header from uplink packets sent by the eNB and encapsulates downlink packets to the eNB into a proper GTP-U header) for two reasons. First, per-UE tunnels do not scale in LTE UAV networks as UEs are mobile and these tunnels are subject to frequent changes. Second, our consolidation already eliminates the need for complex GTP-U tunnels between the SGW-D and PGW-D functionality.

 2.3. Efficient interagent communication

Scalable SDN control and data overlays. SkyCore agents seamlessly exchange control and data traffic with each other, a functionality that is lacking among today's EPC instances. Rather than relying on distributed and multihop wireless routing protocols, we choose to adopt an SDN approach in the design of SkyCore to support the traffic exchange between agents. SDN enables us to perform global optimization (e.g., multipath traffic engineering) and offer fine-grained programmability (e.g., to effectively support different QoS classes), which are necessary tools to instantly and efficiently reconfigure the core in response to network dynamics (e.g., UAV departures and arrivals) in our environment. In particular, we leverage SDN overlays to create two virtualized network layers (slices) on top of the physical UAV network (Figure 9a). One of these network slices is used for control plane traffic between SkyCore agents and the other is for data traffic. Our separation of the control and data traffic ensures time-critical control plane traffic is not affected when the network is saturated. To form the overlays, we use traffic tunneling technologies but depart from existing approaches used in the EPC and SDN-based datacenter (DC) networking^15,16 as they require frequent changes to the network configuration (will be discussed shortly). We adopt a novel variant of segment-based routing in SkyCore and propose a design for its optimization based on the most advanced capability in SDN, that is, P4 language.¹² P4 allows us to define new packet headers and packet processing actions for the SDN switch inside our agents to minimize the packet header overhead on inter-UAV links, which is caused by forming the overlays.

Proactive stateless mobility support. SkyCore replaces the notion of centralized HSS and PCRF with a precomputed policy data store replicated at different agents. Hence, it is essential that the UE states and policies be consistent across different agents, particularly during UE mobility. Reactive approaches to consistency management, for example, distributed hash table (DHT), put wireless (inter-UAV links) on the critical path of control functions. SkyCore avoids this real-time dependence by adopting a proactive synchronization of state between agents—each agent proactively broadcasts its changes to UE policies and states to other agents in the network. Such an approach (i) minimizes the control plane delay between agents, particularly in mobility scenarios as the destination agent already knows the latest information about the mobile UE; (ii) enables seamless handoff of active UEs to a neighboring UAV, when the current UAV goes down for a recharge; and (iii) is scalable because the amount of control plane traffic that is broadcasted on interagent backhaul links is negligible compared to user data plane traffic among agents (Section 4).

A SkyCore agent needs to send only three types of broadcast update messages in the network to build up a consistent network-wide view: (i) security update to notify other agents that it has used one of the security vectors precomputed for a UE and to request other agents to invalidate the vectors, (ii) location update to inform other agents that a particular UE has attached to its UAV, and (iii) policy update to communicate its local changes to the precomputed QoS and charging profile of a UE.

Back to Top

3. Implementation

SkyCore prototype. We prototyped a complete version of SkyCore that involved extensive engineering effort. Our prototype has three notable features: (1) seamlessly works with commercial LTE RANs and off-the-shelf UEs (SIM cards are programmed to connect to SkyCore) by exchanging signaling and data traffic with them; (2) is fully virtualized and can manage multiple LTE UAVs out of the box by forming a wireless network of SkyCore agents; and (3) fully adheres to our proposed designs both for a single agent (Figures 7 and 8) and across agents (interagent communication) (Figures 6 and 9). Each SkyCore agent consists of a controller enforcing control plane policies and a switch processing user data traffic. We developed a high-performance multithreaded controller in C++ and built our SkyCore switch on top of OVS²² software switch in the kernel space. We substantially instrumented and optimized OVS as it does not support our custom flow tables and switch actions (e.g., our P4-enabled tunneling scheme and GTP-U tunnel encapsulation/decapsulation operations). Because our baseline (Edge-EPC based on OpenEPC⁶-will be described shortly) operates in the user space, we developed another variant of the SkyCore switch in the user space on top of Lagopus software switch.⁴ This ensures that our comparisons are at the architecture level and independent of a particular packet forwarding technology.

Figure 8. SkyCore's precomputation of network policies not only makes the core resource-efficient but also minimizes network access delay.

UAV experiments. We conduct three kinds of experiments. (1) Outdoor small-scale: 2 UAV, few UEs. We deploy the SkyCore prototype on two advanced DJI Matrice 600 Pro drones (Figure 10). We securely install two machines on each drone. One of the machines (platform P1) is a low-end single-board 4-core server with 8 GB of RAMs and 1.9 GHz CPU that executes SkyCore and Edge-EPC. It is also equipped with a wireless network card to support our interagent communication. The other machine is a commercial LTE small cell (ip.access S60 eNB) supporting LTE UEs (50 Mbps downlink rate per UE) and connects through an Ethernet cable to platform P1. (2) Outdoor large-scale: 2 UAV, tens of UEs. To stress test SkyCore's control and data planes in the presence of a large number of UEs, we replace the eNB on the drone with another single-board server that runs a unified RAN/UE emulator (emulates both an eNB and activity of a large number of UEs). The emulator interacts with the LTE core similar to real UEs. (3) Emulating powerful UAV platforms. To understand SkyCore's performance with more powerful UAVs, we emulate the latter by replacing platform P1 with a high-end server (platform P2)—an Intel Xeon E5-2687W processor operating at 3.0 GHz with 12 CPU cores and 128 GB of RAM. Because it is not possible to fly our current drone with such a server, these experiments are conducted in the lab (results available in Moradi et al.¹⁸).

Figure 10. Multi-UAV SkyCore prototype.

Baseline. We focus on comparisons between the Edge-EPC architecture (a standard EPC on each LTE UAV) and SkyCore. We implement the Edge-EPC using OpenEPC⁶ as it is the most complete open-source implementation of the 3GPP EPC architecture that can work with commercial devices (e.g., LTE eNBs and smartphones).

Metrics. We study four performance metrics under different network saturation levels: (1) UE-perceived control delay in network access (LTE attach/detach), (2) UE-perceived service disruption time in LTE active-/idle-mode mobility, (3) CPU usage on our resource-constrained UAVs, and (4) supported data plane rate for user traffic.

Back to Top

4. Evaluation

We first show the basic functionality and potential of SkyCore in realizing hotspot and stand-alone LTE UAV networks. We then demonstrate that SkyCore is more efficient and lightweight than the Edge-EPC architecture both in small- and large-scale experimental settings.

 4.1. Small-scale on-drone evaluation

We form a two-drone LTE network (Figure 11), each in the partial line of sight (affected by one building) of a single mobile UE on the ground. Each drone covers a region with the diameter of 650 feet. The drones operate in a small overlapping area for our mobility experiments.

Figure 11. Our UAV-based setup for the basic functionality experiments. In Edge-EPC (the baseline), each SkyCore agent is replaced by a 3GPP EPC (OpenEPC).

Basic functionality–LTE hotspots use case. Forming on-demand hotspots is an important use case for LTE UAV as well as 5G networks. In a single-drone experiment, we show this functionality by connecting one of our drones to the Internet through a terrestrial LTE network not accessible to our UEs on the ground (see Figure 11a). Next, we turn on a Moto G phone on the ground, which sends an LTE attach request to the SkyCore agent through the on-drone eNB. SkyCore agent successfully completes the LTE attach process by quickly accessing its precomputed policy data store. Then, we visit CNN.com and watch a 4K Youtube video on the phone. Finally, we take the Moto G into the airplane mode, causing the UE to properly detach from our agent. Figure 12 shows this basic functionality by depicting the data traffic exchanged between the UE and the Internet.

Figure 12. Hotspot UAV-based LTE network: exchanged data traffic and control events over time.

Basic functionality–stand-alone LTE use case. Next, we demonstrate SkyCore's ability to create stand-alone LTE networks (e.g., between first responders across an impassable mountain). To emulate such a scenario, we establish a direct video call between our two UEs across a building, each connected to a separate drone, through our interagent data plane overlay (see Figure 11b). Figure 13 shows the timeline of control and data plane traffic exchanges between the two SkyCore agents. We again turn on a Moto G phone in the area covered by the first drone. Its SkyCore agent handles the LTE attach process and sends a background SkyCore update message to the other drone's agent. The message consists of location, policy, and security updates as described in Section 2. After the second agent processes this update, we turn on a Nexus 6 phone in the area covered by the second drone, triggering a similar SkyCore update message to the first agent in the background. Finally, we establish a 35-s HD video call from the Nexus 6 to the Moto G. Owing to SkyCore's proactive background updates, the agent corresponding to the Nexus 6 does not have to wait to discover the location of the other UE. Based on our segment-based tunneling scheme, it immediately pushes the correct label stacks on its egress user data traffic and forwards it to the other agent. A similar process manifests in the reverse direction. In this two-UAV enabled video call, 7.5K video packets were successfully exchanged between the UEs.

Figure 13. Stand-alone UAV-based LTE network: UE-to-UE HD video call enabled by SkyCore's efficient interagent communication scheme. Control and data plane processing times and traffic exchanges inside and between the SkyCore agents on the two drones.

Performance benefits of refactoring. Using the same setting, we demonstrate that SkyCore is significantly more lightweight than Edge-EPC. For a fair comparison with Edge-EPC, we employ SkyCore's user space version here. We sample and average the LTE attach/detach delay and uplink/downlink bandwidth for the Moto G in the area covered by the first drone at 40 locations. As Figure 14 and Table 1 show, SkyCore on average reduces the network control plane delay (spent in the core) by 69–90% and the UE-perceived control plane delay by 40–60%. In addition, it doubles the uplink/downlink rates measured for the UE. Further, SkyCore lowers the avg. CPU usage on the machine running the core network by 25% in the LTE attach/detach events. These savings come from our precomputation of network policies and consolidation of the EPC functionality onto our compact SDN-driven agents.

Figure 14. Breakdown of the network access delay in the core.

Table 1. Benefits of SkyCore's software re factoring of the EPC functionality on UE-perceived QoS.

Efficient interagent communication–handoff. Unlike Edge-EPC, SkyCore supports seamless UE mobility, owing to its efficient interagent communication scheme. In this experiment, we measure the service disruption experienced by a mobile UE moving between the regions covered by our two drones and triggering a handoff event. Figure 15 depicts the signal strength received from the two drones on the UE and its continuous bandwidth measurements using iPerf3. The RAN on the first drone collects UE-measured RSRP values and sends a Handoff Required message to its local SkyCore agent when the RSRP values from the second drone become higher. Because SkyCore agents on the drones are already synced, the UE gets migrated to the second drone within a minimal 140 ms (incurred in the interagent coordination). In contrast, Edge-EPC does not support mobility of the UE and thus forces the UE to go through the detach process with the EPC on the first drone, followed by the heavy attach process with the EPC on the second drone. The entire process results in 2s of disconnection time, significantly impacting mobile application performance.

Figure 15. Benefits of SkyCore's interagent communication scheme: SkyCore provides seamless active-mode mobility support, whereas Edge-EPC causes severe connection drops.

 4.2. Large-scale on-drone evaluation

Using the same two-drone experimental setting, we replace the ip.access eNB with a RAN/UE simulator on each drone to test SkyCore and Edge-EPC under large-scale network access workloads (mobility workload results are available in Moradi et al.¹⁸).

Handling signaling storms. Our RAN/UE emulator on the first drone emulates a flash crowd event with a large number of users entering the region (attach storm) covered by a drone. Similarly, the emulator creates an LTE detach storm by having many users gracefully disconnect from the drone. During this process, we sample the CPU utilization of the LTE core machine and measure the average control plane delay perceived by the UEs. In Figure 16a, we observe that the UEs experience exponentially larger delays when the attach/detach load on Edge-EPC increases. In particular, when the number of attach requests per sec. reaches 100, the UEs must wait by up to 6 s before connecting to the network, thereby degrading QoE. With the EPC being a complex system, we observe from Figure 16b that Edge-EPC quickly uses its available CPU resources on the drone and thus faces performance bottlenecks, leading to larger latencies. In contrast, we notice that the network access delay is below 1 s when the drone employs SkyCore owing to its software refactoring of the EPC functionality.

Figure 16. SkyCore substantially reduces network access time and CPU utilization to handle large-scale network access requests.

Back to Top

5. Related Work

SDN/NFV-based EPC. Recently, the wireless networking community has proposed several software-defined EPC solutions. SoftMoW¹⁹ enhances the programmability of the EPC by decoupling its control and data planes. KLEIN²³ and SCALE¹⁰ optimize the placement of the EPC nodes on geo-distributed DCs. ECHO²⁰ deals with EPC-node failure in unreliable public clouds. PEPC²⁴ scales the EPC data plane by creating a per-UE EPC-in-box. Although there are some similarities between SkyCore and this proposal, the differences are significant. These prior designs are customized for highly-reliable, often hierarchical DC infrastructure, where over provisioning and reactive network updates are inexpensive. In contrast, SkyCore operates in an unreliable and resource-constrained wireless environment, where such approaches scale poorly.

SDN control and data planes. There is a rich literature in distributed SDN control plane designs with hierarchical and flat structures (e.g., ONOS¹¹). Most of the schemes are designed for DC networks and operate based on a centralized data store or complex consensus algorithms, which are ill-suited for our unreliable multi-UAV environment.

RAN optimization for LTE UAVs. DroneNet¹⁴ extends the coverage of existing LTE cells by creating WiFi on-drone hotspots. Some recent works^17,25 investigate the theoretical optimization of a UAV trajectory for certain mobile users on the ground (e.g., maximize the minimum average rate among all user). These RAN efforts are predominantly for a single UAV and complementary to SkyCore that focuses on the EPC design for multi-UAV LTE networks.

Back to Top

6. Conclusion

We presented the design and implementation of a novel edge-EPC architecture—SkyCore, supporting the untethered and reliable operation of multi-UAV LTE networks. SkyCore's SDN-based design is equally applicable to future RAN (e.g., 5G and 6G) technologies. Further, even in the context of terrestrial networks, where EPC-RAN communication is often reliable, operators can leverage Edge-EPC designs such as SkyCore to move the EPC functionality to their edge clouds or cell towers and realize ultralow latency as required by many 5G use cases. Such deployments are motivated by operators' push toward mobile edge computing (MEC)^{13, 21} and their efforts in deploying white box switches at cell towers.¹

Back to Top

Back to Top

Back to Top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2021 ACM, Inc.

---

No entries found