EPCN: Achieving Robust Power System Operations under Uncertainty and Price-Driven Active Demand-Side Participation

List of personnel:

1. Principal Investigator (Purdue ECE): Xiaojun Lin

2. co-Principal Investigator (Purdue IE): Andrew L. Liu

3. Graduate Students: Peizhong Ju (ECE), Zibo Zhao (IE).

Project goals:

In order to attain a sustainable energy future with high penetration of renewable energy, it is crucial that future power systems can utilize the flexibility of demand to manage the uncertainty and variability of renewable supply. However, while wholesale electricity prices fluctuate significantly over time, retail electricity rates are often set at a fixed level (i.e., the so-called flat-rate structure) in most of the US. Within such a flat-rate structure, entities with demand flexibility have neither incentives nor effective ways to help improving power systems’ capability to cope with renewable uncertainty. To address this issue, it is envisioned that dynamic price signals will be passed to the demand-side entities, including utilities, demand aggregators, and distributed-generation/microgrid operators, with the hope that they will be incentivized to change their consumption patterns to help achieving more efficient and robust power-grid operations. However, such price-driven demand-side response will in turn affect the price signals. If not designed properly, this closed-loop interaction, when coupled with renewable uncertainty, can produce highly volatile system dynamics. The resulted volatility will increase not only the risk of system instability, but also the price uncertainty and financial risk faced by the consumers, ultimately discouraging them from participating in active demand response. Thus, there is a pressing need to understand at a fundamental level how to design price-driven demand response that can achieve stable, robust and efficient power system operations.

This NSF EPCN project aims to address this challenge by developing the theoretic foundation, control and learning algorithms that allow distributed generation, microgrids and demand aggregators to actively participate in price-driven demand response in a robust, stable, and efficient manner. The key novelty of the project is to formulate the robustness and stability requirements through a rigorous mathematical framework, such that despite future uncertainty, the produced system outcome (in terms of efficiency and/or volatility) will be provably competitive compared to a carefully-chosen set of benchmark settings/algorithms.

Major activities:

1. We studied how to use contract and incentive mechanisms to coordinate microgrids with the main-grid for robust and efficient outcomes [LDL2015]. Although microgrids have emerged as an important paradigm for future power systems, their demand variability leads to new challenges in the coordination between the main-grid and the microgrids. Specificaly, the net-demand that a microgrid draws from the main grid may depends on its internal business operations, local fossil-fule generation, and uncentain renewable supply, and thus may exhibit significant variability. Since the main grid must procure enough generation capacity to ensure that the future demand can always met, the demand variability of the microgrids lead to increased cost to the main-grid. Idealy, the microgrids and the main-grid should exchange information and coordinate their future operations together, so that they can share the benefit of local generation and reduce cost. In practice, however, it remains an open question how to design proper incentive and coordination mechanisms between the main-grid and the microgrids so that they can share information and minimize risks together, while allowing the microgrids to still retain their privacy and flexibility.

2. We also aim to answer the research question of whether any learning-based algorithms on the consumer side can help mitigate the volatility and instability of wholesale power prices, should the consumers face real-time electricity pricing (RTP). It has been widely argued that letting consumers see (and respond to) real time electricity prices would bring many benefits to the system, including lower prices on average, improved system reliability, and fewer capacity redundancy. However, recent studies by other researchers have shown that naïve implementation of RTP may increase electricity price volatility and instability, which could negate the benefits of RTP, and worse, could endanger system reliability. To address this issue, we investigated several learning-based algorithms to be implemented on the consumer side (presumably coded within certain control automation devices that are capable of receiving real-time pricing information), in conjunction with real-time pricing. While learning algorithms, especially those based on regret minimization, have been proposed and well studied in the literature, the setting here is a game environment, where multiple agents engage in such a learning game and their collective action will impact their corresponding payoffs (in this specific case, the total demand of consumers will affect their electric bills). In our work, we focus on establishing the theoretical results for the algorithms we study, including the convergence to correlated equilibria, as well as testing them through numerical simulation using real-world data.

3. We studied how the uncertainty from distributed renewable generation impacts the price signals over the deregulated electricity supply chain [YHZCL2017]. The existing deregulated electricity supply chain, e.g., the electricity wholesale market, has traditionally been designed to operate with no or little uncertainty in both supply and demand. As renewable penetration continues to increase, it is thus important to understand how the entire supply chain of electricity will be affected. While most previous studies focused on the impact of renewables on the supply side of the supply chain, we investigate the impact of distributed renewable generation on the demand side. In particular, we look at an active district, which is a district that has a system in place to coordinate distributed energy generation and external grid to meet the local energy demand. We aim to understand how the uncertainty from distributed renewable generation in one active district affects the average buying cost of utilities and the costsaving of not only this active district but also other active districts. This is an important question because it helps us understand both local and global effect of renewable uncertainty.

4. We have developed a framework based on mult-agent multiarmed bandit (MAB) games to implement price-driven demand response [ZL2017,ZL2017b]. In such a framework, heterongeous end consumers (referred to as agents) decide their electricity usage strategies based on historical information on how well the available choice of actions worked; that is, each agent solves a single-agent MAB problem, which allows agents to learn from the past performance of their own strategies. The advantange of such a framework, as compared to the large amount of exisiting works in this area, is that it is easily implementable without the need for the system operators to do anything differently or additionally. As a result, the framework is a completely decentralized approach, relying on the end consumers, instead of system operators or utility companies/aggregators, to make intelligent decisions.

5. We study a novel business model to enable virtual storage sharing among a group of uses [ZWHL19a]. Note that energy storage plays an important role in demand response or demand-side management. However, the cost and complexity of owning energy storage can become a significant issue for its wide deployment. In contrast, in our proposed business model, a storage aggregator invests and operates the central physical storage unit. It then virtualizes it into separable virtual capacities and sells to users. Each user purchases the virtual capacity, and utilizes it to reduce the energy cost. This model is analogous to the practice of cloud service providers for IT services, who set prices for virtualized computing resources shared by end users. This type of cloud computing is known to reduce the cost of owndership, and lower the bar of develop innovative services. Thus, we aim for similar benefits for power systems. However, the cost and incentive considerations for power systems are very different from cloud computing. Thus, we develop a rigorous model to study the pricing mechanism for the storage virtualization and sharing under significant future uncertainty. 2. We study the market competition between renewable energy suppliers with or without energy storage in a local energy market [ZWHL19b]. Note that renewable energy generations and energy storage are playing increasingly important roles in serving consumers in local power systems. Energy storage investment brings the benefits of stabilizing renewable energy suppliers¿ outputs, but it also leads to substantial investment costs. Furthermore, the return of storage investment depends on the outcome of the local energy market, which in turn depends on how suppliers with or without storage compete for the demand. Therefore, it remains an open problem regarding whether competing renewable energy suppliers should invest in energy storage in the market competition and and what economic benefits the storage can bring to the suppliers. Our study aims to answer these questions and brings new insights to the deployment of energy storage and renewable energy in local enegy markets.

6. We study the market competition between renewable energy suppliers with or without energy storage in a local energy market [ZWHL19b]. Note that renewable energy generations and energy storage are playing increasingly important roles in serving consumers in local power systems. Energy storage investment brings the benefits of stabilizing renewable energy suppliers¿ outputs, but it also leads to substantial investment costs. Furthermore, the return of storage investment depends on the outcome of the local energy market, which in turn depends on how suppliers with or without storage compete for the demand. Therefore, it remains an open problem regarding whether competing renewable energy suppliers should invest in energy storage in the market competition and and what economic benefits the storage can bring to the suppliers. Our study aims to answer these questions and brings new insights to the deployment of energy storage and renewable energy in local enegy markets.

7. In studying completely decentralized schemes for consumers or prosumers to respond to real-time electricity pricing, we are able to provide two different strategy framework for individual consumers in a multi-agent setting, with proven results on minimizing cumulative regrets for all agents. (The regret is defined as the difference of an agent's utility between the best possible action that the agent can take in hindsight, and the actual action that the agent took.) The first strategy framework, termed as multiarmed bandit games (MAB), was established in the previous reporting period. In this reporting period, we are able to prove the regret bounds for an n-agent game, and able to prove the bound between the optimal social welfare and the social welfare resulting from the MAB game (termed as price of anarchy). The second strategy framework, termed as low approximate regret (LAR), is built upon the works of Prof. Eva Tardos and her collaborators, and is established to fit within the real-time pricing (RTP) game in this reporting period [Z2019]. We showed that if all agents in the multi-agent RTP game uses a LAR strategy, then the game posses the so-called (lambda, mu) smoothness property, which can lead to a bound on the price of anarchy.

8. The works in the previous item are under the setting that consumers/prosumers just passively decide the best time to consume energy from the main grid (or utility) or to sell back to the grid. To facilitate peer-to-peer energy trading in a local market/region, we extend the MAB-game framework to aid consumers/prosumers repeated bidding in an hourly, double-sided auction, where buyers (consumers) and sellers (prosumers with distributed energy generation resources, such as rooftop solar panels) can submit demand/supply bids to a market, and the electricity price (for that hour) is the clearing price where the aggregated supply and demand curve intersect. (The works are documented in [ZL2019].) We compared three different auction designs: uniform-pricing auction (mimicking the wholesale energy market auction), a Vickery-variant auction, and maximum volume matching (MVM) auction. Numerical results indicate that the MAB-game approach works well under any of the auction designs, and can stabilize soon after the initial learning period. Such a framework will greatly help market participants automate their bidding strategies in such a repeated auction setting (possibly with less intuitive auction rules, such as for the Vikery and MVM auctions).

Significant results:

1. We develop a model for day-ahead and real-time electricity market to study the impact of uncertainty on the price signals [YHZCL2017]. Each active district's demand is assumed to be Gaussian with a certain variance. We then investigate how the Average Buying Cost (ABC) of an active district is affected as the level of uncertainty changes. Our analysis shows that the increase of renewable uncertainty in one active district can increase the average buying cost of the utility serving the active district, which we referred to as the local impact. Perhaps more surprisingly, we show that the increase of renewable uncertainty in one active district may reduce the average buying cost of other utilities participating in the same electricity market, which we referred to as global impact. Moreover, the local impact will lead to an increase in the electricity retail price of active district, resulting in a cost-saving less than the case without renewable uncertainty. On the other hand, the global impact tends to benefit other active districts. We provide sufficient conditions on the price-setting functions so that the local and global impacts hold. These observations reveal an inherent economic incentive for utilities to improve their load forecasting accuracy, in order to avoid economy loss and even extract economic benefit in the electricity market. We verify our theoretical results by extensive experiments using real-world traces. Our experimental results show that a 9% increase in load forecasting error (modeled by the standard deviation of the mismatch between real-time actual demand and day-ahead purchased supply) will increase the average buying cost of the utility by 10%.

2. We model the contract design problem with multiple classes of micro-grids as a mechanism design problem [LDL2015]. The main grid designs a contract option for each class of micro-grid. The main grid aims to maximizes the total profit subject to the constraint that each microgrid's cost is either no higher under her dedicated option than under all other contract options, or no higher under the baseline option (with no contract) than under all other contract options. With this type of contraints, the main grid ensures that the contract options are truthful, i.e., each micro-grid prefers the contract option designed for her. However, this problem is in general intractable since the constraints are non-convex. By exploiting the problem structure, we resolve this difficulty and provide an approximate solution such that the revenue increase under the approximate solution (compared to baseline) is no less than 1/2 of the revenue increase under the optimal contract. Further, we observe that when some classes of micro-grids are similar to each other (e.g., their main consumption is close to each other), it becomes difficult to design dedicated contract options such that the micro-grid's cost under her dedicated contract option is strictly lower than under other contract options. On the other hand, allowing the microgrids' costs to be same across contract options raise compatibility issues because micro-grids may then choose another contract option not dedicated to her. We extend our analysis to this more general scenario and show that the performance ratio of our approximate solution is still no less than 1/3. Finally, we evaluate our contract-based information revelation mechanism with two pricing schemes (i.e. time-of-use pricing and peak-based pricing ) that are often used in practice. Compared with these pricing schemes that do not focus on revealing information from micro-grid, our proposed mechanism is shown to increase the maingrid's profit and reduce each microgrid's cost.

3. We have tested the multiagent multiarmed bandit (MAB) game framework through numerical simulation [ZL2017,ZL2017b]. The simulation is based on an artificial example that includes 200 consumers and 200 prosumers (i.e., consumers with distributed generation capacity that can send electricity back to the grid). The supply side of electricity is represented by an aggregated supply curve. While the simulation is run for 2000 time periods (each period representing a day), the MAB game converges (i.e. stablizes) after about 15 time periods for all the runs we have performed. The volatility of the electricity prices from the MAB-game approach is an order of managitude lower compared to the case of naive real-time pricing (i.e., consumers all use electricity when the price forecast is low, and do not use when forecast is high). Such results still hold even if we add uncertain wind output in the supply stack; i.e., the multiagent MAB game still converges quickly, and the volatility of the realized market prices is still very low (with almost flat price curves).

4. We develop a two-stage model to study the interactions between virtual storage providers (i.e., the aggregator) and end-users [ZWHL19a]. In Stage 1, over the investment horizon (over multiple years), the aggregator determines the investment and pricing decisions. In Stage 2, in each operational horizon (e.g., over a day), each user decides the virtual capacity to purchase together with the operation of the virtual storage. The aggregator chooses a price for the virtual storage to balance her profit and users¿ benefits. For a profit-seeking aggregator, we aim to find the optimal-profit price to maximize her profit. For an aggregator that is regulated by the system operator or regulatory agents, we aim to find the lowest-nonnegative-profit price, which can give the most benefits to users while maintaining a nonnegative profit for the aggregator. For both cases, we characterize the form of the optimal solution of Stage-2 Problem and a piecewise linear structure of the optimal profit of Stage-1 Problem, both with respect to the virtual capacity price. Based on the solution structure, we design an algorithm to attain the optimal solution of the two-stage problem.

   Our study reveals significant benefit of storage virtualization. The first key advantage is the ability to leverage users¿ complementary charge and discharge profiles. Note that the aggregator only cares about the net power flowing in and out the storage. As some users may choose to charge while others choose to discharge in the same time slot, some requests will cancel out at the aggregated level. This suggests that even if all the users are fully utilizing their virtual storage capacity, it is possible to support users¿ needs by using a smaller central storage comparing with the total virtual storage capacities sold to users. In our simulation results, the proposed storage virtualization model can reduce the physical energy storage investment of the aggregator by 54.3%, compared to the case where users acquire their own physical storage.

   The second key advantage of storage virtualization is that a user can flexibly change the amount of virtual capacity to purchase over time based on his varying demand. Such flexibility is difficult to realize if the user owns physical storage by himself, and encourages the users to take advantage of the virtualized energy storage. In our simulation results, the proposed storage virtualization model can reduce the users¿ total costs by 34.7%, compared to the case where users acquire their own physical storage.

5. We develop a three-stage model to study the interactions between suppliers facing decisions of storage investment in the local enegy-market competition for supplying renewable energy to consumers [ZWHL19b]. In Stage I, at the beginning of the investment horizon (containing many days), suppliers decide whether to invest in storage. Once such decisions have been made (once), in the day-ahead market of each day, suppliers decide on their bidding prices and quantities in Stage II, based on which consumers decide the electricity quantity purchased from each supplier in Stage III. Note that such bidding prices and quantities must take into accunt the uncertainty of the renewable generation in the next day. Finally, in the real-time market, a supplier is penalized if his actual generation falls short of his commitment. We characterize a price-quantity competition equilibrium of Stage II in the local energy market, and we further characterize a storage-investment equilibrium in Stage I incorporating electricity-selling revenue and storage cost.

   Our study reveals a number of interesting and somewhat counter-intuitive conclusions. First, in contrast to the prevailing wisdom is that storage investment (especially when the storage cost is low) will improve suppliers¿ revenue by stabilizing their uncertain outputs, we find that the opposite may be true when considering market competition. Specifically, without storage, suppliers with random generations always have strictly positive revenues when facing any positive consumer demand. However, if both suppliers invest in storage and stabilize their renewable outputs, their revenues may reduce to zero due to the increased market competition after storage investment. Second, in contrast to the common wisdom is that a higher penalty and a higher storage cost will decrease suppliers¿ profit, we find that the opposite may be true when considering market competition. With a higher penalty for not meeting the commitment, renewable energy suppliers become more conservative in their bidding quantities, which can decrease market competition and increase their profits. In both situations, the market competion siginificant alters the eventual outcome of storage investment decisions. As a result, our study sheds important new insights into the adoption of storage for renewable energy suppliers participating in local energy markets.

6. We demonstrated through our work that under real-time electricity pricing (RTP), completely decentralized, automated control at the consumer level can lead to stabilized overall electricity prices, with much lower volatility, congestion costs, and higher social welfare than consumers' naive response to RTP. The proposed learning-based framework within a multiagent game allows consumers' heterogeneity and bounded rationality (lack of information, lack of access to sophisticated algorithm or computing power, etc), and hence is much closer to a real-world setting. In addition, the approach is scalable. Even when numerical simulations may be impractical when the number of consumers increases, the MAB-game can be shown to converge to a mean-field equilibrium.The learning-based, multiagent framework differs from all other agent-based simulation framework in the sense that theoretical convergence (to a steady state) and bounds on deviation from the maximum-achievable social welfare can be established ([Z2019]).

   In addition to the MAB-game framework, we also proposed the low approximate regret (LAR) framework, in which each consumer uses the full information (that is, knowing what the payoff would be in the past for any time spot that would be chosen). The LAR game can accommodate higher heterogeneity among consumers, as unlike in the MAB-game, different consumers may have different decision epochs. The theoretical results we show for the LAR game ([Z2019]) demonstrated that it would work well in reality (such as achieving very low volatility electricity prices and much flattened demand curve) so long as each consumer employs a LAR strategy (such as the so-called noise hedge policy).

7. By extending the MAB-game framework into a double-side auction setting [ZL2019], which is a predominant approach to realize a peer-to-peer market, we established a computation/simulation framework that has two significant implications. The first is that the framework can be easily implemented on the consumer side to help them automate the bidding process in a repeated auction; the second is that different auction designs can be compared using this framework, which otherwise may be very difficult to compare using game-theoretic modeling/computation approaches. Through our comparison, the numerical results show that a uniform-pricing based auction design outperforms two other designs in consideration: a Vickery-variant auction, and a maximum-volume-matching auction, in terms of social welfare. This is an unexpected results as Vickery (or Vickery-like) auctions enjoy many desirable theoretical properties (such as truth-revealing of bidders' preference).

Products:

1. [LDL2015] P. Lan, L. Duan and X. Lin, "Learning and Coordination of Microgrids via Flexible Contracts," manuscript in preparation for submission to IEEE Transactions on Control of Network Systems.
2. [LX2015] A. L. Liu and J. Xiao, “Implementing Real-time Pricing in Wholesale Electricity Markets,” presentation at Workshop on Sustainable Electric Power Systems, November 19-20, 2015. University Park, PA.
3. [LS2016] A. L. Liu and H. Sreekumaran, “Decentralized Algorithms for Block-Structured Stochastic Programs and Potential Games under Uncertainty,” presentation at the INFORMS Optimization Society Conference, March 17-19, 2016. Princeton, NJ.
4. [LSC2016] A. L. Liu, H. Sreekumaran, and R. Chen, “Distributed Algorithms for Nonseparable Optimization Problems Potential Generalized Nash Equilibrium Problems (GNEPs),” presentation at The Fifth International Conference on Continuous Optimization, August 6 – 11, 2016. Tokyo, Japan.
5. [ZL2017] Z. Zhao and A. L. Liu, "Intelligent Demand Response for Electricity Consumers: A Multi-armed Bandit Game Approach," in the Proceedings of International Conference on Intelligent System Application to Power Systems 2017.
6. [YHZCL2017] H. Yi, M. H. Hajiesmaili, Y. Zhang, M. Chen and X. Lin, "Impact of the Uncertainty of Distributed Renewable Generation on Deregulated Electricity Supply Chain," IEEE Transactions on Smart Grid, to appear.
7. [ZL2017b] Z. Zhao and A. L. Liu, "Implement real-time pricing with regret-based learning games," presentation at the INFORMS Annual Meeting, Houston, TX, October, 2017, schedued.
8. [JL2018] Peizhong Ju and Xiaojun Lin, "Adversarial A¿acks to Distributed Voltage Control in Power Distribution Networks with DERs,"in ACM e-Energy, Karlsruhe, Germany, June 2018. The paper [JL2018] was one of the five best-paper finalists in ACM e-Energy 2018.
9. [ZLC2018] Zibo Zhao, Andrew L. Liu and Yihsu Chen, "Electricity Demand Response under Real-Time Pricing: A Multiarmed Bandit Game," in 10th Annual Conference of the Asia Pacific Signal and Information Processing Association, IEEE, 2018 (accepted).
10. [ZWHL19a] Dongwei Zhao, Hao Wang, Jianwei Huang, and Xiaojun Lin, "Virtual Energy Storage Sharing and Capacity Allocation," IEEE Transactions on Smart Grid, 2019.
11. [ZWHL19b] Dongwei Zhao, Hao Wang, Jianwei Huang, and Xiaojun Lin, "Storage or No Storage: Duopoly Competition Between Renewable Energy Suppliers in a Local Energy Market," IEEE Journal of Selected Areas in Communiations, to appear.
[ZL2019] Zibo Zhao and Andrew L. Liu, "Multi-Agent Learning in Double-side Auctions for Peer-to-peer Energy Trading," paper submitted to the the Power Systems Computation Conference (PSCC 2020), Porto, Portugal, June 29 to July 3, 2020.
12. [Z2019] Zibo Zhao,"Decentralized Price-Driven Demand Response In Smart Energy Grid," PhD dissertation, School of Industrial Engineering, Purdue University, October, 2019.
13. In addition, Co-PI Liu has presented his research findings of the MAB-game framework to technical staff at the Midcontinent Independent System Operator (MISO) in Carmel, IN on April 6, 2018, and through a webinar organized by Institute of Industrial & Systems Engineers (IISE) on Octocber 24, 2019 (https://www.iise.org/details.aspx?id=49549).