There are many ways of doing research, and we also need different types of researchers, but here is why I stayed in academia, and what "research with impact" means for me

First blog: 02/10/2021

Ideas to improve policy and regulation: "This is the way"

Research can help policymakers and regulators make better decisions. The impact you will have, is more difficult to measure than your publications or citations, but it is extremely rewarding. It is the reason I stayed in academia. To do research this way, there are two realities to accept and three principles to follow.

Reality 1: Research is a struggle

Research is a struggle. I still struggle with every new research project. With experience, you learn to enjoy the struggle. If you are confident that there will be results at the end, you are ready to embrace the process. You will not necessarily find what you were looking for, but there is always something to discover. 

We hire researchers that can work in teams to brainstorm and trial-and-error. It is important to seek feedback from colleagues. Better to fail quickly and try again than to isolate yourself and get stuck or side-tracked. Feedback should be constructive, but also frank. Not saying what you think to be nice, does not get us anywhere.

Reality 2: Half of the job is to be out there

To be relevant in applied research, you need to engage with practitioners. They will give you inspiration for your next research topics and provide a reality check for your approach to the topic. You can return the favour by helping them to use your results. The more you prove yourself with relevant research, the more practitioners will be willing to engage with you, and the more you will be able to have impact. Starting can be difficult and slow, but there are positive feedback loops so do not give up.

To be out there is to meet people, to contribute to industry as well as academic events. It is also to learn how to communicate your results via social media, videos, podcasts, policy briefs, blogs, etc. For many researchers, including me, this means going beyond the comfort zone, but you can do it one step at a time. 

Please accept that not every research project deliverable can lead to an academic journal publication that is worth promoting. If you pursue too many of these publications, it is anyway unrealistic that you will have time left to promote them extensively for impact. Sometimes we might somewhat neglect this reality and publish more to remain relevant in our academic community, and to be promoted by our academic institution.

Principle 1: Follow the policy agenda

There are at least three types of contributions that researchers can make.  You can help to: 

It is important to know the mandates of policymakers or regulators, and their agenda. Timing is crucial if you want to have impact. During a consultation process, you can come with proposals. In other periods, it can be more productive to work within the boundaries of a certain policy or regulatory decision with research on the best possible implementation. 

Following the agenda does not mean to accept it as it is, academics do have to challenge policymakers and regulators when needed, even if the timing is sometimes inconvenient. 

I did not always strictly follow this principle. I sometimes also published to try-out new methodologies. I think it is important to continue to invest in research skills.

Principle 2: Master quantitative and qualitative methodologies

Many researchers specialize in quantitative or qualitative methodologies. I think it is useful to try to master both. It is important to learn when to apply which methodology, and to know what you can publish in which academic journal.

For instance, the PhDs I work with learn to apply optimization models. Models can help to anticipate the magnitude of the effects that might be triggered by certain policies or market rules. Models can also help to discover unexpected interactions in complex systems. My focus is not on advancing the state-of-the-art in modelling, but on using these models to help regulators and policy makers make better decisions. 

Towards the end of a PhD or later, it is also useful to learn how to publish qualitative research. I like to use simple frameworks that help to identify the main policy options and their pros and cons. A smart synthesis can provide structure to the policy debate and generate new ideas.

Principle 3: Take care of your writing style

Everyone has a list of do’s and don’ts that they feel strong about when reading, reviewing, drafting, or editing research papers. 

First, my three main do’s: 1/ table of content or outline that reads like a storyline (quantitative papers often have a straightforward outline from methodology to a discussion of the results; qualitative papers can be structured following the cases that are discussed or the framework used to discuss them); 2/ explicit or direct writing style (for example, I started this section by writing that there do’s and don’ts, followed by two numbered paragraphs to discuss them, and each paragraph starts by stating the number of points I am going to make and also numbers them); 3/ After drafting your text, shorten it to its essence, cut, cut, cut.

Second, my four main don’ts: 1/ too many ideas in one paper; 2/ lots of figures or math with limited text to interpret or discuss the results; 3/ paper with many unnecessary details that distract you from the main contributions; 4/ inconsistencies between title, abstract, intro, results, and conclusions (you would be surprised of how many papers have inconsistencies, including mine). 

It is very difficult to edit your own text. Family, friends, and colleagues can help by telling us frankly that our drafts are “unreadable”. Conversations or presentations to explain your main findings, can also give you inspiration to improve the text. 

Final message

Let me close by quoting the Mandalorian: “This is the way”

Second blog: 26/09/2022

Improving policy and regulation with impact assessments

In this blog, I use our recent publications to illustrate three points. 

First, energy markets are complex because there are many interactions between countries, sectors, system operators, and the various energy users. 

Second, researchers can help policy makers and regulators to intervene in these markets by evaluating policies and regulations with numerical simulations, which are referred to as impact assessments. 

Third, there are lots of do’s and don’ts to share, but let’s start with three.

Complexity of energy markets

Representing the complexity of energy markets can be challenging. It is not possible to account for everything in every model, but we try to make sure that the interactions that matter are included in the model we develop to support a debate. 

First, the interaction between countries

Energy market integration across country borders has been a gradual process. Countries or certain regions within a country realize that it is cheaper to cooperate, but they do not want to give away all their local decision powers. Researchers can support the process by estimating the benefits of cooperation, which can help build the case for more cooperation or avoid unnecessary interventions.

In Saguan and Meeus (2014) and in Govaerts et al. (2019), we modeled the interaction between two countries that are connected by an electricity market. The starting point in both papers is that electricity can be traded freely over the country border, but certain policy decisions remain national. This is the situation in Europe, and similar setups can also be found elsewhere in the world.

In Saguan and Meeus (2014), we studied the national decisions to subsidize renewable energy and to invest in cross-border transmission network capacity. Other researchers had already suggested that there was insufficient cross-border collaboration, but there was limited information on the magnitude of the problem, and on the interaction between the two debates, i.e. the debate on national renewables support mechanisms, and the debate on regulation of transmission investments. 

In Govaerts et al. (2019), we focused on the national network tariffs. In the fourth electricity market reform, the European Commission proposed a harmonization of distribution network tariffs, but stakeholders and national regulators were opposed. “One size does not fit all” was the main counterargument. There was a lot of academic literature on the topic of network tariff design, but nobody had covered this multi-country aspect. 

Second, the interaction between energy sectors

The natural gas and electricity sector are strongly linked, and we now also have emerging sectors for renewable gasses, such as biogas and green hydrogen. Policies and regulations that apply to one of these energy sectors, will also have an impact on the other sectors.

In Roach and Meeus (2020), we pioneered a sector coupling model that captures the interactions between the electricity system and the gas system. We were surprised that most energy economics or policy papers were based on sector-specific models, like electricity models or gas models. The two systems have always been connected because gas can be used to produce electricity. The systems will be more connected in the future because electricity can be used to produce hydrogen with electrolysis, which can be blended with gas or can be stored and/or used as a green gas. In analogy with Saguan and Meeus (2014), we wanted to know whether we should worry about misaligned incentives between the two systems.

In Roach and Meeus (2021 and 2022), we further developed our multi-carrier energy system model with coupled markets for electricity, hydrogen and methane. In Roach and Meeus (2021), we used the model for the discussion on how to subsidize renewable gasses. In Roach and Meeus (2022), we used it to demonstrate that sector integration can help to stabilize prices. A few countries in Europe are advocating for an electricity market design revolution, often based on arguments that do not properly account for energy system interactions.

Third, the interaction between system operators 

Each country and energy sector typically have their own system operator(s). Governments and citizens are more relaxed about sharing an energy market, than about merging their system operators. Even without a country, there are different system operators in each sector: one transmission operator typically collaborates with several distribution system operators. The border between transmission and distribution is a certain voltage level in the electricity sector, and a certain pressure level in the gas sector. This border is not harmonized across countries. The cooperation among these system operators, is a key concern for policy makers and regulators. 

In Beckstedde et al. (2021), we studied the interactions between distribution system operators and transmission system operators in the electricity system. Both operators procure grid services for balancing or for congestion management. They can do that together or in separate markets, and there is also an ongoing discussion on who should come first in the sequence of electricity markets. Different countries have different practices, so there is a debate on the level of harmonization that is needed. 

Fourth, the interaction between the energy system and its users

Energy systems have many users that interact. The consumption, investment, production, and storage decisions of these users are at least partly guided by prices. Energy bills consist of taxes and levies, network tariffs, and the market prices of the different energy carriers. Final energy consumers can be categorized into domestic versus commercial or industrial users, and active versus passive users. Passive users can be affluent users that do not care about prices or vulnerable consumers. Energy poverty, unfortunately, is still a big issue. For policymakers and regulators, it is therefore important to know how their actions affect the different types of users. 

In Schittekatte et al. (2018) and Schittekatte and Meeus (2020), we pioneered an electricity system model that captures the interaction between a distribution system operator and the grid users via the network tariffs. Everybody agrees that tariffs have to change, but there are many competing proposals. The regulatory impact models that were initially used to support the debate, assumed that users would not change their behavior when confronted with different incentives. We wanted to investigate how different the results (and recommendations) would be if we could integrate the user reaction into the model. 

The modelling setup in Schittekatte et al. (2018) and Schittekatte and Meeus (2020) inspired several papers treating related regulatory challenges. In Govaerts et al. (2019 and 2021), we went deeper into the regulatory debate on distribution tariffs. In Nouicer et al. (2023), we discussed the interactions between the distribution tariff debate and the broader debate on how DSOs should source flexibility to reduce the need for network expansion. In Nouicer et al. (2022a and 2022b), we deep-dived into the regulatory debate on flexibility contracting by DSOs by comparing different regulatory options.

Research impact with impact assessments

There are at least two ways in which researchers can contribute to the impact assessments of policies and regulations. First, policy makers and regulators often have an idea of the possible pros and cons of their options, but researchers can improve their understanding of the order of magnitude of the costs and benefits. Second, researchers can discover unexpected interactions that have not yet been discussed in the ongoing debate. 

First, simulating the magnitude of the costs and benefits associated with policies and regulations

Debates include different arguments. Policies and regulations typically try to address market failures. Addressing these failures can be beneficial, but interventions can also be costly. By performing numerical simulations with energy models, researchers can give us an idea about the order of magnitude of some of these costs and benefits. 

In Saguan and Meeus (2014), we showed that the cost of a non-cooperative outcome for renewable energy policies would be higher than the cost of a non-cooperative outcome for the regulated investments in the cross-border transmission network. We also showed that if we manage to overcome the challenges related to national renewable energy policies, the cost of not doing the same for transmission investments would increase. The most important sensitivities we identified, not surprisingly, were the level of renewable energy ambition, and the differences between the systems (the bigger the difference, the bigger the opportunities to collaborate). The study supported the urgency of the issue, which meanwhile has been (partly) addressed by policymakers and regulators.

In Roach and Meeus (2020), we showed that electrolysis can help avoid spillage of renewable energy in a decarbonized electricity system. We were also expecting to find misaligned incentives between the electricity system and the gas system, but we did not find anything to be concerned about. Such a non-result allows us to eliminate one issue from our list of possible issues to worry about. We did find interesting price dynamics in coupled gas and electricity markets, which inspired us to go deeper into that topic in Roach and Meeus (2021 and 2022). Due to the energy crisis, the policy debate also shifted to price dynamics, so we were ready to contribute.

In Schittekatte et al. (2018) and Schittekatte and Meeus (2020), we showed that the current practice with volumetric distribution tariffs is problematic in an electricity system with prosumers. We also showed that the solution advocated by many, capacity tariffs, creates new issues. Tariffs that are partly fixed (to cover sunk costs), and partly based on individual peaks that coincide with system peaks (to signal incremental investment costs) are more cost-reflective. We also concluded that tariffs will never be fully cost-reflective because setting tariffs is about making compromises between cost-reflectivity and other important regulatory principles, such as simplicity and fairness. In Govaerts et al. (2021), we demonstrated more generally why it is so difficult for regulators to set cost-reflective tariffs. 

In Govaerts et al. (2019), we showed that national distribution network tariffs do create spillover effects in internationally coupled electricity markets. They can be positive (e.g. with tariffs of one country implicitly subsidizing the consumers of another country), or negative (e.g. for the country providing the implicit subsidy). The European Commission therefore had a valid argument when they proposed to harmonize distribution tariffs in Europe. However, our results did not allow us to conclude that we should harmonize the tariffs. We did not do a full cost-benefit analysis; we focused on one of the arguments used in the debate. We did not investigate the one-size-does-not-fit-all counter-argument.

Second, discovering unexpected results that provide new insights to policy makers and regulators

At the start of each modelling exercise, we have certain expectations. Expectations can come from the policy and regulatory debate that inspired the research, in combination with existing research and intuition. If we find results that contradict these expectations, we get excited. It means we found a mistake in our model that needs to be fixed, or we found a new insight.

In Schittekatte et al. (2018), we discovered a non-cooperative equilibrium in an extreme scenario. In most scenarios, tariffs allowed prosumers to save money at the expense of (vulnerable) passive consumers. When we then studied a scenario in which most of the consumers were prosumers, prosumers were also worse off. We first thought we found a mistake in the code of our model, but then realized that prosumers were in the typical prisoner’s dilemma. The dominant strategy for everyone is to overinvest, even if they would all be better off if they would not. In Schittekatte and Meeus (2020), we had a similar experience when we searched for fair tariffs. The model turned out to be insolvable in many scenarios. We again thought we had a bug in the model. It turned out that the principle of fairness often conflicts with the principle of cost-reflectivity when designing network tariffs, which is why the model was often insolvable. We concluded that it would be better to implement other measures to protect certain consumers. 

In Nouicer et al. (2023), we were researching the potential for DSOs to save network investment costs by curtailing demand with compensation. When we started to simulate the potential, we were modelling a typical day. We gave the DSO the option to curtail demand, but our welfare-maximizing DSO never used that option. We first thought we had a mistake in our model until we realized that it can indeed be cheaper to dimension the distribution grid to handle the peak of a typical day. The business case for curtailing is stronger on critical days with unusually high peaks that only happen a few times a year. By bringing the peaks on these critical days closer to the peak of a typical day, we can more easily save investments with limited use of flexibility. The lesson learned was that we had to add a bit more complexity to the model to analyze this topic adequately, which we did.

Top 3 of do’s and don’ts

The stereotype of economists is that they develop energy models that are so abstract that they can only solve them analytically to get general conclusions. Engineers often challenge the conclusions of theoretical models as constructs of their unrealistic assumptions. 

The stereotype of engineers is that their energy models are overly complex with unnecessary technical details. Engineers also like to develop ad-hoc solution methods for their models with properties that are not always clear. The economists therefore challenge these models as black-boxes. 

Many of today’s energy modelers are multi-disciplinary researchers that try to overcome the stereotypes. If you want to be one of them, please continue to read about the do’s and don’ts we try to live up to. 

1st don’t: models that are unnecessarily complex

We develop energy models that include some of the technical complexities of the energy system, but we simplify the model as much as possible. For instance, we know that Europe has more than two countries, that there are more than two types of energy consumers, or more than two types of days in a year, but modelling two can be enough to get certain insights. Less can be more if you want to understand what drives the results. 

Within a PhD, we will typically start with a basic model to get some basic insights. Next, we will typically explore which modelling steps lead to insights that are relevant for the ongoing policy or regulatory debate. New insights can be obvious in hindsight, which means that we can explain them intuitively, but we needed the model to discover them.

Even if we strive for simplicity, we often use energy models that are too complex to solve analytically. When resorting to numerical simulations, we still want to know the properties of the solution, and prefer to arrive at that solution with standard optimization solver packages rather than developing our own solution methods. 

2nd don’t: no conclusions for policy makers or regulators 

In our team, a paper with a model that produces results without discussing the relevance of these results for policy makers or regulators in detail is not acceptable. We rely on other researchers to push the boundaries of what can be modelled, but it is not our focus. We like to apply state-of-the-art models to the latest developments in policy and regulation. To be able to do that, we sometimes have to make modelling contributions, but it is not an objective. 

If our modeling exercise does not allow us to answer our initial research question, there is always something else to conclude after digging deeper. A non-result is also a conclusion, and needs to be presented as such. 

3rd don’t: conclusions for policy makers or regulators that are too strong

In policy or regulatory debates in the energy sector, there are often several good options on the table, and it is not always possible to conclude what is best. We try to be the academics that bring some shades of grey into the debate. It is important to discuss the limitations of the models used for impact assessments.

1st do: keep non-results in your paper

In Roach and Meeus (2020), we expected to find misaligned incentives between the gas system and electricity system, but we did not. We kept the non-result in the paper, and broadened the scope of the paper towards price dynamics between coupled electricity and gas markets. 

In Nouicer and Meeus (2023), we expected to find a business case to use demand curtailment in distribution grids to save investments. In the original setup of the model, we did not find it. We kept the non-result in the paper, but realized that we could not generalize the result because it was partly due to our modelling choice to include one type of day in the simulation. We broadened the paper to include a second type of day, which did give us the results we expected. The exercise confronted us with the limitations of our intuition, and helped us to gain new insights.

2nd do: stress test the model with extreme assumptions

Energy modelers have a tendency to create a reference scenario with a sensitivity “comfort zone” that they can explain. It is however important to stress-test a model with extreme scenarios. Running the model with extreme assumptions can lead to new insights, and it can also help to validate it. A bug in the code can become more visible when you push the model to the limit.

In Schittekatte et al. (2018), we got a new insight from pushing the model to an extreme scenario in which all passive consumers became consumers. In Nouicer and Meeus (2023), we pushed the model in many different directions. We, for instance, forced it to curtail demand from 0% to 100% to see what happens to the overall system and the energy bills of different types of consumers. 

3rd do: model validation

We often replicate the results of another paper with an energy model that we then build on. We also like to decompose our models into sub-problems that we validate separately before putting them together. Nothing special, this is what all modelers normally do. 

References

Beckstedde, E., Meeus, L., Delarue, E., 2021. Strategic behaviour in flexibility markets: New games and sequencing options. Energy Syst. Integr. Model. Group. Work. Pap. Ser. No. ESIM2021-05.

Govaerts, N., Bruninx, K., Le Cadre, H., Meeus, L., Delarue, E., 2019. Spillover effects of distribution grid tariffs in the internal electricity market: An argument for harmonization? Energy Economics, 84.

Govaerts, N., Bruninx, K., Le Cadre, H., Meeus, L., Delarue, E., 2021. Forward-looking distribution network charges considering lumpy investments. Journal of Regulatory Economics, 1-23.

Nouicer, A., Meeus, L., Delarue, E., 2022a. A bilevel model for voluntary demand-side flexibility in distribution grids. RSC Work. Pap.

Nouicer, A., Meeus, L., Delarue, E., 2022b. Demand-side flexibility in distribution grids: voluntary versus mandatory contracting. RSC Work. Pap.

Nouicer, A., Meeus, L., Delarue, E., 2023. The economics of demand-side flexibility in distribution grids. The Energy Journal, IAEE 44(1).

Roach, M. and Meeus, L., 2020. The welfare and price effects of sector coupling with power-to-gas. Energy Economics, 86.

Roach, M. and Meeus, L., 2021. An energy system model to study the impact of combining renewable electricity and gas policies. Robert Schuman Centre for Advanced Studies Research Paper No. RSC, 73.

Roach, M. and Meeus, L., 2022. Revisiting policy concerns in decarbonized energy systems: shocks, extreme prices, and seasonal storage. (mimeo)

Saguan M. and Meeus L., 2014. Impact of the regulatory framework for transmission investments on the cost of renewable energy in the EU. Energy Economics 43, p185–194.

Schittekatte, T., Momber, I., Meeus, L., 2018. Future-proof tariff design: recovering sunk grid costs in a world where consumers are pushing back. Energy Economics, Vol. 70, pp. 484–498.

Schittekatte, T. and Meeus, L., 2020. Least-cost Distribution Network Tariff Design in Theory and Practice. IAEE Energy Journal, 41 (5).

© 2021-2024 Leonardo Meeus