Jülich team solves 60-second power grid mystery: battery storage checks
Researchers traced recurring grid frequency dips to synchronized minute-mark state checks by battery systems and other digitally controlled devices.
What to know
- Researchers identified synchronized 60-second power checks by battery storage systems as the cause of recurring global power grid frequency dips observed for years.
- Battery manufacturers independently adopted the same 60-second interval without binding standards, causing millions of devices worldwide to consume electricity simultaneously at each minute mark.
- The pattern extends beyond batteries to other smart devices (heat pumps, air conditioners, washing machines), suggesting the issue is a broader software design problem affecting grid stability and turbine wear.
- Potential fix identified: randomizing check intervals in software like Linux cron jobs could desynchronize device consumption and improve grid robustness.
This reveals a major software design flaw with real grid consequences that should be fixed by randomizing cron job timing.
-
“adding random 0-59.999s delays to cron jobs in Linux distributions and other software would actually help the power grid.”
hzulla@infosec.exchange · Mastodon ↗
This is a fascinating detective story showcasing how independent engineering assumptions can create measurable global effects.
-
“This is FASCINATING. Researchers were able to pin down the cause of measurable global 60 second power spikes to software that does regular checks on the minute mark.”
hzulla@infosec.exchange · Mastodon ↗
Dirk Witthaut Research team lead, Forschungszentrum JülichForschungszentrum Jülich, University of Cologne, Karlsruhe Institute of Technology Research collaboratorsTenneT Germany Transmission system operator
How it unfolded 3 developments, newest first · click a bar or a number to jump posts
-
2
Researchers identify clock-synchronized device checks as systemic grid issue
The team determined that battery manufacturers independently chose 60-second intervals for state checks without binding standards, and noted that other digitally controlled devices—air-conditioning systems, heat pumps, washing machines, and dishwashers—may exhibit similar synchronized patterns, suggesting the problem extends beyond battery storage.
“The observed effect can only be produced if many devices are consuming electricity at the same time. A single consumer or generator could therefore be ruled out.”
— Dirk Witthaut -
A thundering herd problem, but for power? Jitter saves the day as usual.
2 more of the top 3 · 49 posts in this stretch
-
The paper has a little more details: "following our communication with the manufacturer, a software update was issued that removed the strict periodicity of the current peaks in one battery block of the investigated UPS system" https://www.nature.com/articles/s41560-026-02144-0Given the issue is so widespread (international, affecting pretty much…
-
G
@ karlauerbach You were mentioning little things and engineering that don’t strictly look like good software design. Here’s a biggie. https://www. fz-juelich.de/en/news/archive/ press-release/2026/mystery-of-60-second-fluctuations-in-power-grids-solved
-
-
3
Jülich team publishes solution to global 60-second grid frequency dips
A team from Forschungszentrum Jülich, University of Cologne, and Karlsruhe Institute of Technology published findings in Nature Energy identifying the cause of recurring power grid fluctuations observed by TenneT Germany and globally: synchronized power checks by large-scale battery storage systems occurring every 60 seconds at the minute mark.
“It really was years of detective work. It quickly became clear to us that the cause must be something controlled by the clock.”
— Dirk Witthaut -
first by HN Frontpage, 8d ago
-
-
1
Mastodon user proposes staggering cron jobs to reduce grid strain
A commenter on Mastodon noted that the finding reveals a broader software design problem: randomizing 0-59.999 second delays in cron jobs across Linux distributions and other software could help stabilize the grid by desynchronizing device checks.
“adding random 0-59.999s delays to cron jobs in Linux distributions and other software would actually help the power grid.”
— hzulla@infosec.exchange -
H
This is FASCINATING. Researchers were able to pin down the cause of measurable global 60 second power spikes to software that does regular checks on the minute mark. They found large scale battery installations as one major cause, but obviously this is true for any other computerized system that does cron jobs and has a synchronized clock. As…
-
What people are saying 20 voices from 3 sites · best of 50 · verbatim
- Nov 3
-
J
Cool: Mystery of 60-Second Fluctuations in Power Grids Solved https://www. fz-juelich.de/en/news/archive/ press-release/2026/mystery-of-60-second-fluctuations-in-power-grids-solved Adjust your cronjobs ;p
- Nov 2
-
Jenkins server has the feature to randomize job start time since at least 15 years. One would've thought this is a well known problem.I wished cron had a similar feature. I missed that on acouple of occasions.
-
Yeah, adding 'jitter' for periodic tasks that make remote requests was standard procedure when I worked for a large CDN. Every cron entry on every server would run with a delay of between 0 and 59 seconds, based on a hash of the server name. You basically 60x your internal infrastructure capacity by doing this (depending on how long the request…
-
Package managers like apt will delay automatic updates for a random time to avoid this problem: RandomizedDelaySec=12hr
-
> and a little bump every minute if your metrics have enough resolutionAmusingly, you can also observe periodic bumps if your metrics don’t have enough resolution, due to aliasing effects dominating signal (guestures vaguely at Nyquist frequency).
-
Funny, similar patterns are super common in API traffic. A large spike at midnight UTC, a smaller one every hour, even smaller ones every 15 minutes, and a little bump every minute if your metrics have enough resolution. Of course, if someone calls your API every minute you should probably consider some kind of streaming, or at least include make…
-
The press release is rather light on details. They found that it is every 60seconds (duh!) and generators (or such). So what exactly is it? (Make and model?)Seems like they did not find the exact manufacturer or the exact devices and were not able to change firmware then?
-
i'm confused, why would a system simply checking its own power state be taking so much electricity as to show up on grid measurements? unless they mean that the systems are choosing to start charging the batteries, but then it's a continuous load and not a spike at the start of the minute, right?
- Nov 1
-
> It really was years of detective workDoesn't seem that complicated from the article though. They say that they immediately knew it was clock based due to the strict interval. With REA etc. it might become easier to proof, but I guess it's a measurement problem that only appears in large networks?Can someone with more understanding explain why…
-
Almost all of these large deployments sell frequency services and do energy price arbitrage. The EU grid is mostly priced at 15-min increments, so being just a few seconds off the metered clock can put volumes on the “wrong” quarter-hour. For stability services there are variants where you get a message from the TSO saying “discharge x MW starting…
-
Rather than try to change the behavior of thousands of different actors, move the jitter upstream and have NTP servers give each recipient a slightly different time. Problem solved!
-
While this can add some jitter, it is also introducing a bias (i.e. the period is now strictly greater than 60s), which may or may both be tolerable depending on the application.
-
I was surprised reading the man page that `@daily‘ didn’t do that. Apparently it’s a fixes specified time. I’d assumed it was midnight plus some random jitter. It doesn’t look like POSIX specifies this. Aha! It turns out I didn’t completely misremember. FreeBSD uses cron’s regular things to drive `periodic`, which can then introduce a random sleep…
-
Something in such systems need to talk to the power grid accounting, and the current system setup default is to connect to the Internet everything in a subnet once any machine of the subnet is connected — and to run NTP on every Internet-connected machine.
-
The water-hammer effect, or in this case electron-hammer effect, is why you don't reboot every Windows server in the data center at exactly 2am, to stagger the load shift over time. Surprised that the power guys didn't realize that switching a presumably multi-hundred-MW load in and out at the top of each minute would cause a problem.
- Oct 31
-
> But how do all these devices synchronize their consumption? This question wasn't fully answered by the article, at least not explicitly. I think the implication is that the timers themselves must be synchronized to the same clock somehow? (NTP, GPS, etc.) In other words, it's not sleep for 60 seconds, check power, repeat. It's sleep *until the…
-
If only they treated time-keeping as unseriously, the problem would be avoided though! Uneven unseriousness is the worst.
-
The simplest way to add jitter to a system is often to replace "do this every 60s" by "do this, wait 60s, repeat".
-
Yes, That's exactly what we are telling you. Those of us who have been burned before are aware of issues like this. But not everyone is. I'm guessing that the people responsible for manufacturing batteries just hadn't experienced the problem. And it wouldn't show up with testing -- it's only when you have a huge number of these on the grid that…
-
All I can think is software is such an unserious field that we keep repeating these mistakes. I habitually use the systemd random jitter setting on my timer units to prevent this exact problem, even when it’s not really clearly necessary, and you’re telling me the people building these grid scale battery systems didn’t think of this before…