A podcast about how data engineers use algorithms to control machines
This podcast was born out of a fascination with how we use algorithms to control the physical world. Its author, Jirka Vicherek, invited Honza Šimbera, who leads a team of analysts and data engineers at Nano Energies, into the studio to ask him about how we predict the amount of electricity in the grid, or when to turn on machines that will start producing steel on our command. Listen to the full episode of the Data Talk podcast for data professionals, or – becasuse the podcast is in Czech – read a transcript of parts of it. You’ll learn why we need flexibility, what decision-making model we use in Nano, and why some algorithms will never replace humans.
How can data science help the energy industry?
Humanity needs to move to zero-emission energy, which brings with it a great need for intelligent management. At Nano Energies, we’re trying to move energy into the 21st century through what we call flexibility, the ability to smooth out the market fluctuations that come with the shift away from burning fossil fuels like coal, gas and oil and towards renewables, which are much more challenging to manage.
How do you do it?
We are connecting industrial resources that have some energy flexibility on the production side with the market that needs it. On the consumption side, it is, for example, a certain process that only needs to consume electricity for eight hours a day. We are able to determine which specific eight hours that will be in order to increase the profit or reduce the cost of electricity.
And who are you selling this flexibility to? Where is it needed?
On one hand, it is the market itself. That means we sell electricity on the daily market. The moment there is a shortage of electricity on the market, we can turn on something extra compared to what we were supposed to supply to make up for it. If we see a surplus, we can turn off something that we originally planned to have running.
National network operators are also important customers for us. In the Czech Republic, it is ČEPS, the Czech Power Transmission System, which has this balancing of the network as one of its main tasks in order to maintain stability. Because there must be a balance of production and consumption in the energy network at all times, ČEPS contracts the capacity of so-called ancillary services for this purpose. This means that some resources are switched off and ready to be switched on when needed. ČEPS pays us for this capacity and sends us a signal that it wants a certain amount of power when it needs it. We will supply that regulating energy and get paid for it.
We can also provide a flexibility service to someone who has production or consumption that they cannot control and use flexibility to smooth out fluctuations. A typical example is large photovoltaic parks. Their generation can be predicted to a large extent, but because the weather is such a complex system, it cannot be predicted with complete accuracy.
What data and technologies influence flexibility?
Technologically, this actually means that we need to utilize smart decision-making. That is, evaluating the operation and capabilities of contracted devices. Furthermore, to monitor the state of the market and to try to find the model of operation that will be optimal and most profitable for our customers and us.
A big part of our added value is that we have built a communications infrastructure. This is a control system that we install at customers’ sites that is capable of receiving commands over the Internet. And at the same time we can receive signals from the market operator about the capacity required.
In addition, we add a decision-making component. We predict whether there will be a shortage or surplus of electricity in the market at a given moment. We then determine, for example, whether to shut down the customer’s equipment for two hours. Of course, we must not exceed the operational limits. If it is a freezer, for example, we must of course ensure that it does not thaw out.
And when we put it all together, we optimize it to maximize profit.
So you actually control the physical world based on algorithms. You predict when to turn on the machines that will, for example, start making steel on your command.
Yes, that’s right. I also like the financial impact, where money is behind every decision we make. For example, if the quotation process fails on the daily market, a company can suddenly lose several million czech crowns per day. Or conversely, when we deploy a new model, we are able to save the company millions. For example, by better forecasting we can offer a higher feed-in tariff for electricity from photovoltaics. This is where our work has a real financial impact.
Why is flexibility such an essential part of the future energy sector?
Basically everything humanity does requires either consuming or producing energy. The whole system is incredibly complex and we will never be able to predict it with complete accuracy. That is why we need some flexible capacity to be able to compare needs in real time.
Whereas in the past, balancing was primarily done with fossil resources, which are large and relatively easy to drive, today we need to add flexibility to the grid as we retire these large resources and start to rely more on smaller renewables.
And our know-how is the ability to connect and organize these small resources into what we call an aggregation block. And that’s the flexibility that market operators needs to balance the grid.
We are able to create a situation, on a technical level, where generation is equal to consumption and there are no blackouts in the network.
How do I imagine the data you are working with?
From a data scientist’s point of view, it is interesting to see the wide range of problems we solve – from prediction to optimization, and at different time levels. This includes both long-term optimization with customers throughout the year, as well as real-time, instantaneous decision making.
For each source, we use data at different levels. If we have a certain type of source, for example combined heat and power (CHP) units, we treat them very similarly. But at the same time, there are some specifics that we have to take into account. My favourite example is for example the heating of the municipal swimming pool in Mladá Boleslav, where we have to take into account not only the heat, but also part of the electricity that is needed to run the pool itself. So we can only monetise part of the capacity.
The problem arises when we do not have some data at all, because we receive measurements from some devices up to a month in advance. For example, in September we get data for August. During September we have to guess what is happening with the equipment. When we don’t have accurate online measurements, the modelling gets quite complicated.
Often, what we calculate actually translates directly into commands for the machines. When we get an activation signal from ČEPS as part of the ancillary services, we have to get to the required output within five minutes. So we’ve programmed APIs that have to respond within a minute and start the device based on the optimization. Our job is to piece together a flexible bandwidth from the portfolio of dozens of machines we manage, but each of which has slightly different operational constraints.
Are algorithms and models or the human element more important to your work?
But it is important to remember that the whole energy system is incredibly complex. In addition, weather is a major input in the energy sector, affecting the production of wind or photovoltaic power plants. It is therefore a system that cannot be modelled accurately at micro level. That’s why we combine machine prediction with human decision making in our work.
Another example of how irreplaceable the human factor is for us is the war in Ukraine. When it started, we needed to throw away a large part of our models – human input can evaluate these extraordinary influences much better. Especially in a situation where other market players are not behaving completely rationally. Algorithms have a strategy, but it needs to be constantly monitored and verified that it is still relevant and affects all relevant data inputs. Otherwise, they could cause us a big loss in a very short time.
Therefore, we try to carefully evaluate which part of the system needs human correction. So we have a system that is automatic, but also has a human interface so that it can be monitored well and corrected in case of an anomaly.