Communications of the ACM

Contributed articles

Smart Data Pricing: Using Economics to Manage Network Congestion

By Soumya Sen, Carlee Joe-Wong, Sangtae Ha, Mung Chiang
Communications of the ACM, December 2015, Vol. 58 No. 12, Pages 86-93
10.1145/2756543
Comments

Since 2010, there has been rapid evolution in pricing practices among Internet service providers (ISPs) in the U.S. and other international markets, particularly in moving away from flat-rate to usage-based pricing in cellular networks.¹⁹ In 2010, AT&T eliminated unlimited data plans and introduced a tiered plan of about $10/GB, along with deliberately slowing the traffic of heavy users and adding hefty overage fees. Verizon and other U.S. ISPs have since introduced similar data plans. These changes have fueled the continuing debate about Net neutrality and the openness of the Internet.

Back to Top

Key Insights

ISPs argue these price increases and usage penalties are needed in response to rapid growth in data traffic, as driven by increasing demand for smart devices, bandwidth-hungry applications, cloud-based services, machine-to-machine traffic, and mediarich Web content.^5,20 This explosive growth, which Cisco's visual networking index projects will cause a nearly tenfold increase in global wireless traffic between 2014 and 2019,⁴ requires investment in expanding wired and wireless network capacities (such as additional spectrum, Wi-Fi hotspots for offloading data traffic, backhauling infrastructure, and newer technologies like 4G/LTE and femtocells). The benefits of this capacity expansion are partly accrued by the content providers who attract more advertising and e-commerce revenue from greater user demand while further driving demand for bandwidth. ISPs contend they are trapped in a vicious cycle that does not allow them to match their prices to their costs. Measures (such as throttling, data caps, and usage-based metered pricing) are thus viewed as essential for regulating demand and managing network congestion.

However, penalizing demand would be harmful for the Internet ecosystem and could restrict network access for some users. Appreciating the role of economics in network management and understanding how these models can be realized in practical network systems is crucial to the Internet's long-term growth and sustainability.

Exploring the link between economic principles and network engineering is the goal of several recent research efforts in smart data pricing (SDP).²⁴ SDP mechanisms go beyond simple byte-counting schemes to include time/location/app-based dynamic pricing, usage-based pricing with differentiated speed tiers, auction-based smart markets, Wi-Fi offloading, proactive caching, zero-rating or sponsored content, and quota-aware content adaptation. In general, it asks three kinds of questions along the following dimensions:

Who pays? Who should pay for bandwidth (such as zero-rating, sponsored content, and two-sided pricing)?;

What services? What service should be charged for (such as transaction-based pricing and quality-based pricing)?; and

How to charge? How to enable and charge for mechanisms like time/location/congestion-dependent pricing and traffic offloading?

In this article, we mainly address "How?," reporting on two research directions—time-dependent pricing (TDP) and traffic-offloading mechanisms. Although researchers have been exploring the interplay between networks and economics for years^5,8,15 (for a detailed survey of various pricing proposals, see Sen et al.²¹), the need for designing and demonstrating fully functional prototypes has become more urgent due to users' growing demand for data. Consequently, a key aspect of recent SDP research has been how to bridge the gap between analytical models and practical considerations:

From practice to modeling. Network technology should be complemented with sound economics, while analytical models should account for the real-world constraints imposed by existing technical and practical operational considerations, including parameter measurability, data granularity, solution scalability and complexity, integration and deployment feasibility, regulatory requirements; and

From modeling to practice. Analytical models should provide guidance on the economics of bandwidth pricing and policy, as well as on the architecture that will implement the models in operational networks. The prototypes developed must then be tested by deploying them in the wild.

To incorporate this interplay in SDP, researchers should take a holistic approach bringing together ideas from economics, networking, information systems, and human-computer interaction. In general, SDP research involves three stages, each with interesting research questions essential to realizing a new pricing scheme, as in the following:

Analytical modeling. How can ISPs create economic models for computing optimized prices or incentives they are willing to offer and users are willing to accept for modifying their bandwidth-consumption behavior in the desired manner?;

System development. How can ISPs develop scalable systems and related signaling protocols for these incentive mechanisms? How should the various required functionalities (such as congestion measurement, traffic profiling, and delay-optimal scheduling) be divided among network backend and end-user devices?; and

Field trials. How can ISPs use human-computer interaction principles to design end-user interfaces so users understand and respond to the pricing signals?

A series of new initiatives since 2012, including the Internet Architecture Board's Workshop on Internet Technology Adoption & Transition,¹⁴ Smart Data Pricing Forum,²³ and Internet Research Task Force Working Group on Global Access to the Internet for All,¹⁷ provide momentum for such interdisciplinary collaborations. This article discusses two complementary research themes in SDP—TDP and traffic offloading—that aim to reduce network congestion by giving users incentives and mechanisms to "shift" their use to less-congested times or to other supplementary networks (such as Wi-Fi). We also consider SDP's implications for the Internet's long-term sustainability and accessibility to a wider user population.

Back to Top

Case 1. Time-Dependent Pricing

Much of the need for expanded network capacity is due to large peak demand created by users' simultaneous consumption of data; Cisco's visual networking index predicts peak hour traffic will grow at 64% CAGR.⁴ Yet, as Odlyzko et al.¹⁶ correctly said, ISP attempts to slow this growth by transitioning from flat-rate to usage-based pricing are unlikely to solve the problem. To discourage large peak demand, prices should have a temporal component or vary over different times of the day, as in TDP. Only then will users be incentivized to spread their demand over time, improving network resource utilization by reducing the peaks and filling in the valley periods. Writing in 2008 at Google's Public Policy blog,^a Vinton G. Cerf advocated a similar view, saying, "Network Management also should be narrowly tailored, with bandwidth constraints aimed essentially at times of actual congestion." He cautioned against ISPs rushing to change their pricing, favoring a more detailed study on the efficacy of such dynamic or time-varying pricing mechanisms. The work we present here is one such attempt to understand the efficacy and feasibility of TDP for mobile data.

The telecommunications industry has long used dynamic TDP plans for voice calls as a response to demand variability in call volume by adjusting users' prices and incentives. However, dynamic pricing plans for data traffic, in spite of their theoretical potential to make resource allocation much more efficient, have remained largely unrealized in the global market, possibly due to the gap between the large body of analytical work on the topic and the lack of functional prototypes implementing these ideas.

The extensive academic literature on dynamic pricing theories²¹ includes responsive pricing (setting prices so as to keep user demand under a certain threshold), proportional fairness pricing (setting prices to optimize a proportional fairness criterion on the amount of bandwidth allocated to different users), priority pricing (explicitly accounting for QoS by allowing users to pay less by accepting a longer delay at congested times), and "smart market" auction pricing (deciding whether to admit a packet into the network at congested times based on the user-specified bid attached to that packet). But such schemes face two practical challenges: users prefer flat rates over the uncertainties associated with near-real-time price fluctuations,^15,22 and users are often reluctant to delegate price bidding or traffic scheduling to automated agents, preferring the psychological assurance of manual control despite the greater convenience of automation.²² For users to be comfortable with dynamic pricing for data, ISPs must be willing to provide guarantees on available future incentives or prices, design intuitive user interfaces to aid manual decision making, and demonstrate the underlying system's feasibility with a proof-of-concept prototype.

Practice to models. To address the practical issues within an analytical framework, we have explored an alternative pricing approach—dynamic day-ahead TDP⁹—in which the ISP computes hourly prices one day in advance and advertises them to all users. The provider continues to compute new prices to maintain a sliding one-day window of announced prices; that is, a new price point for the 24^th hour is computed and announced every hour. Users thus receive day-ahead price guarantees while being rewarded for shifting some portion of their data usage (such as non-critical traffic) according to the announced price points. ISPs can measure changes in usage volume at different hours of the day in response to the given set of prices and estimate users' willingness to shift different types of traffic, which is in turn used to compute the next set of optimized prices. These prices are computed through a convex optimization formulation that minimizes the provider's total cost of overshooting capacity and the cost of providing these incentives to users while accounting for current estimates of users' willingness to shift traffic, temporal variations in usage volume, and capacity constraints.⁹ The framework thus requires a control-feedback loop between ISPs and users (see Figure 1a).

---

To discourage large peak demand, prices should have a temporal component or vary over different times of the day, as in TDP.

---

In addition to considering users' psychological preference for certainty regarding future price points, TDP must accommodate technological and regulatory concerns. In order to preserve users' privacy, our formulation does not require ISPs track each individual user's usage pattern, precluding the need for any deep packet inspection to realize this pricing scheme. The formulation remains computationally efficient as the number of network users grows; the model in Ha et al.⁹ implicitly assigns the aggregate traffic from all users and applications into virtual traffic classes characterized by different delay- and price-sensitivity estimates. It then accounts for heterogeneity across users by separately modeling the probabilistic deferral behavior for each traffic class. This optimization model avoids utility functions, which can be difficult to measure quantitatively. Instead, all parameters (such as changes in usage volume in each period in response to prices) can be measured directly (such as users' price and delay sensitivities for different traffic classes) or estimated by the ISP without compromising the system's scalability or user privacy.

Models to practice. Figure 1b outlines the system we use to realize this day-ahead TDP for mobile data; the aggregate traffic measurement, user sensitivity estimation, and price computation engines are on the ISP side, while a mobile application that communicates with the pricing engine is on each end-user device. The primary purpose of the mobile application is to give users information on available price points, but it also has optional features like usage monitoring and alerts, as well as an auto-pilot mode for automated scheduling of applications based on available prices and user-specified delay sensitivities. For privacy reasons, such scheduling information remains in the app but is not communicated to the ISP. For more on the user-interface design, efficacy, and user response to these features, see Sen et al.²²

We developed and tested a prototype of this system over eight months in 2012 in different phases of a randomized field experiment in Princeton, NJ, with 50 mobile users on the AT&T network. In it, we effectively became a resale ISP offering this data plan with day-ahead dynamic prices to the trial participants. We separated the 3G traffic of the participants from that of other customers using an access-point-name setup, tunneling participants' 3G traffic from the ISP's core network into lab servers (see Figure 2). The participants installed the TDP mobile application on their iOS devices. Wi-Fi use, voice calls, and text messaging were not included in the trial traffic, as these services do not count toward 3G data caps.

The mobile application running on users' devices displayed the prices/discounts available for the next 24 hours in a color-coded format (see Figure 3a). Each price is color-coded by its discount rate (such as red for 0%–10%, orange for 11%–19%, yellow for 20%–29%, and green for >30%). Users could view their usage history superimposed over the offered prices to visualize how much they spent and saved on data. Additionally, to help users save money, the app, called TUBE (a free version with only traffic-monitoring features, called DataWiz, is available in both iOS and Android appstores; see http://scenic.princeton.edu/datawiz/), provided an interface for users to see their top five bandwidth-consuming applications, set alerts, and configure their weekly budgets and app delay sensitivities for automatic scheduling, as in Figure 3b-3e.

Using the analytical model in Ha et al.⁹ we offered optimized day-ahead time-varying price discounts on the baseline price of $10/GB to all users. We found users did shift their traffic from high-price to low-price periods under TDP. Overall data use by iPad users decreased by 10.1% in high-price periods and increased by 15.7% in low-price periods; that is, for most users, average use decreased in high-price periods relative to average use at the same hours before the trial.⁹ Additionally, by focusing on use in consecutive periods, where discounts differed by only 1% but the colors of the price indicator bars were different (such as comparing usage volumes in a yellow price period with a 29% discount to the following green price period with a 30% discount), we found a significant change in use, even though the absolute percent change in discount was only 1%. This resulting changed behavior indicates users paid more attention to the color-coding than to the actual value of the price discounts, emphasizing the need for careful and intuitive user interface design to ensure users are able to understand and respond to pricing signals.²²

With optimized day-ahead time-dependent prices, resource utilization at off-peak hours nearly doubled, indicating TDP can also improve utilization of network capacity by flattening and distributing demand over different times of the day. The resulting maximum observed daily peak-to-average ratio (PAR) decreased by 30% with dynamic day-ahead TDP, and approximately 20% of the PARs from the pre-trial period were greater than the maximum PAR with TDP. Although the population size of our field trial was somewhat limited due to the complexity of actually conducting it, the results are promising, demonstrating it is possible to not only operationalize dynamic pricing for mobile data but also to use time-varying prices to change user behavior.

TDP can also be offered as time-dependent sponsored content, whereby consumers do not see a price fluctuation, but the valley discount is reflected in the price sponsoring parties (such as content providers and enterprises) pay to an ISP.

Back to Top

Case 2. Traffic Offloading

In addition to shifting demand from peak to valley time periods, ISPs can alleviate congestion by shifting demand off their cellular networks onto supplementary networks like Wi-Fi or femtocells, a process known as "traffic offloading." Many ISPs encourage offloading by selling bundles of base and supplementary technologies; for example, the French telecommunications company Orange offers a £2 bundle of 3G and Wi-Fi hotspot access.¹³ Conventional wisdom says increasing the access price of the base technology (such as 3G or 4G) will encourage users to purchase the bundle and offload more to the supplementary network. This seems to be a direction many service providers are pursuing today with various penalty mechanisms on their base technology. But this strategy does not account for the fact the supplementary network can itself become more congested as more and more users offload their traffic, potentially making it less attractive to users. That is, there is a complex interaction among the prices of the base and supplementary technologies, the relative network congestion externalities of these two technologies, and the coverage area of the supplementary technology. Economic models and their practical realization in field trials can help ISPs design more effective offloading mechanisms. Here, we discuss related results from our studies that focus on the theory, as well as implementation, of ideas that can improve offloading performance and make it easier for users to make offloading decisions.

Practice to models. Understanding how users decide to adopt the base technology's network or a bundle of base and supplementary technologies, as well as deriving the resulting equilibrium and transient market outcomes, requires analytical models that incorporate practical issues like congestion on both networks and the coverage area of supplementary networks. Sen et al.¹⁸ introduced a model to study the dynamics of competition between two generic network technologies with cross-network externalities in the presence of network gateways or converters. In Joe-Wong et al.,¹³ we extended this framework to develop an analytical model in which users individually make their adoption decisions based on several factors (such as the technologies' intrinsic qualities, users' heterogeneity in the evaluation of these qualities, negative congestion externalities from the presence of other subscribers on the technologies, and access rates charged by an ISP).

Using the analytical model introduced by Joe-Wong et al.,¹³ we studied how user-level decisions translate into aggregate adoption dynamics and characterize the equilibrium outcomes for different system parameters. The model reveals seemingly intuitive strategies can sometimes have unintended consequences for congestion on the base network technology; for example, increasing the coverage area of the supplemental technology can increase traffic on the supplemental network, motivating some users to drop the bundled service altogether and use only the base technology. Likewise, increasing the base technology's access price can cause some users to find the base technology's coverage and relative network congestion conditions do not offset the decrease in their utility from the base price increase. These users are then motivated to drop the bundled service and use just the base technology, thus contributing to an increase in congestion on that network. Careful analysis using such economic models can prove very useful in providing guidance and insight into potential outcomes due to network pricing and policy changes.

Models to practice. While analytical frameworks like the one we discuss here can provide insight into users' long-term adoption behaviors (such as equilibrium and stability), users' minute-to-minute offloading decisions reflect their more immediate concerns. Users face a three-way trade-off among cost, throughput, and delay; while they can save money and receive greater throughput by waiting for Wi-Fi access, they may not want to wait for certain critical applications. To navigate it, we propose a practical, cost-aware Wi-Fi offloading system called Adaptive bandwidth Management through USer-Empowerment, or AMUSE,¹⁰ a user-centric tool that learns users' behavior and mobility patterns to help decide which applications to offload to what times of day, enabling users to stay under their data caps. In doing so, it uses a utility-optimization algorithm to decide if and how much 3G bandwidth must be allocated to each application at any moment of the day based on the user's available budget, application delay sensitivities, and input from prediction algorithms regarding the user's demand patterns and Wi-Fi availability in the future.

To implement this allocation in practice, AMUSE uses a receiver-side transmission control protocol (TCP) bandwidth control algorithm that enforces each application's assigned download rate by controlling the TCP advertisement window on the user side. The algorithm is thus fully contained on end-user devices and does not require modification on the TCP server side, making it suitable for real-world deployment (see Figure 4).

An AMUSE system implementation on Windows 7 tablets, when tested with simulated user behavior based on 3G and Wi-Fi usage and availability data from a field study of 37 mobile users, showed other offloading algorithms yield 14% and 27% less user utilities than AMUSE for light and heavy users, respectively.¹⁰ Intelligently managing users' competing interests for cost, throughput, and delay via user-side management tools can thus facilitate user decisions, leading to offloading benefits that could be further improved when integrated with ISP-centric approaches (such as ad hoc authorization for offloading to hidden Wi-Fi access points or market-based solutions for offloading from base stations to third-party-owned Wi-Fi access points and femtocells).^3,6,11 Such measures to push some control from the network core out to end users complement previous and existing efforts of the networking research community, as with Briscoe et al.²

Back to Top

Win-Win Solution

The two approaches to managing network congestion discussed earlier, namely realizations of TDP and delaying optimal traffic offloading, can help create a financial win-win solution for ISPs and their users. ISPs benefit through reduced peak congestion, and users have more choices and technologies to help them save on their monthly bills.

Research on network resource pricing also has implications for bridging the digital divide between those who can and those who cannot access the Internet regularly. Rural local exchange carriers (RLECs) often suffer from congestion in their wired networks due to the persistence of the middle-mile problem. Although the cost of middle-mile bandwidth has declined over the years due to increased demand needed to fill the middle mile, the bandwidth requirements of home users have also increased sharply. The cost of middle-mile upgrades to meet the Federal Communication Commission's target speed of 4Mbps will be substantial and is a barrier to digital expansion in rural areas.⁷ New access-pricing mechanisms like TDP can help reduce middle-mile investment costs by reducing RLEC peak-capacity provisioning or leasing needs and improving resource utilization in the valley periods. Providers can thus match their prices to the cost of delivery while also creating incentives for light users to adopt broadband services. Instead of being charged by the volume of data consumed, users can save on their monthly bill by choosing "when" to consume the data. An extension of this idea can be used to create ultra-affordable data plans for delay-tolerant users that allow automated opportunistic access to the network only when large discounts are available or when the network is lightly loaded. Implementation of such dynamic TDP and opportunistic offloading schemes can help service providers better utilize their available resources and increase Internet adoption by being financially attractive to even light data users.

Content providers have also tried to bridge the digital divide with zero-rating, app-based pricing and sponsored content,^1,12 all of which open further interesting questions for SDP research. For example, how should platforms for sponsored content or zero-rating be made compatible with Net neutrality regulations, as in the case of Facebook's Internet.org initiative? And how will such subsidized plans affect broadband adoption and network congestion? Much of the current SDP research focuses on such questions.

Back to Top

Conclusion

As demand for bandwidth grows in both wireless and wired networks, ISPs are pursuing penalty mechanisms like deliberately slowing traffic, capping, overage charges, and usage-based fees to manage their available network capacity. But such measures are arguably suboptimal and even harmful to the Internet ecosystem. In contrast, ideas from economics can help design incentives and pricing policies that are beneficial to both service providers and their users. Although many analytical models for pricing-based network management have been proposed, their implementation has begun only recently. Tackling today's challenges requires not only developing analytical models that incorporate practical concerns (such as measurability, scalability, privacy, and user behavior) but also demonstrate efficacy and feasibility through prototypes and field trials.

Here, we have focused on "shifting" demand through two complementary efforts that aim to alleviate network congestion by creating incentives and mechanisms to modify user behavior or shift demand to less congested times (through TDP) or to a supplementary network (through delay-optimized traffic offloading). The results indicate such measures, if implemented with intuitive designs, can help ISPs better monetize and manage their network capacity while empowering users with more options to avoid hefty overage fees. With implicit pricing signals and automated decision making, these solutions can be readily adapted to emerging machine-to-machine and Internet of Things applications. These approaches also have positive implications for making Internet access affordable for many more users, thereby contributing to bridging the digital divide.

The scope of SDP is much broader than the two particular cases we outlined here. The complementary questions "Who?" and "What?" are gaining traction, in addition to "How?" on which we focused. Researchers must move beyond models, trials, and testing of prototypes and consider integration with existing infrastructure, the design of pricing and signaling protocols, and regulatory concerns. SDP research will help bring together academic researchers, network providers, content providers, the e-commerce industry, and policymakers to design and deploy new mechanisms to help ensure the sustainable growth of the Internet, mobile, and content markets.

Back to Top

Acknowledgments

We thank many collaborators in academic institutions and industry, especially Krishan Sabnani of Bell Labs for his encouragement and feedback on this article.

Back to Top

Back to Top

Back to Top

Back to Top

Back to top

The Digital Library is published by the Association for Computing Machinery. Copyright © 2015 ACM, Inc.

---

No entries found