Evaluation of NICT space weather forecast for extreme events in May 2024

Evaluation of NICT space weather forecast for extreme events in May 2024 | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Evaluation of NICT space weather forecast for extreme events in May 2024 Kaori Sakaguchi, Sanae Akiyama, Satoshi Andoh, Yumi Bamba, Kaisei Enoki, and 19 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9090861/v1 This work is licensed under a CC BY 4.0 License Status: Under Revision Version 1 posted 5 You are reading this latest preprint version Abstract In May 2024, multiple X-class solar flares, full-halo coronal mass ejections (CMEs), severe geomagnetic disturbances, and severe ionospheric negative storms were observed, and these space weather events caused some social impacts. In this study, we evaluate performance of space weather forecasts against these extreme space weather events, focusing on maximum forecast levels of X-class solar flares, geomagnetic disturbances with K ≥ 7, and ionospheric storms with I-scale = I3. Firstly, NICT’s forecasts were evaluated using multi verification indices such as proportion correct (accuracy), probability of detection (discrimination), false alarm ratio (reliability), frequency bias (bias), and equitable threat score (skill). As a result. It was found that NICT’s forecasts were characterized as having strong discrimination ability for X-class solar flares with moderate reliability, and high accuracy for ionospheric storms but subject to discrimination ability. While nearly perfect performance for K ≥ 7 geomagnetic disturbances forecast were achieved this time by using solar wind simulation (SUSANOO-CME) incorporating multiple earth-directing CMEs. Comparative analysis with forecast of other countries indicated that RWC Japan was the highest discrimination capability for X-class solar flares, while RWC USA archived highest skill for solar flare forecasts with zero false alarms. Perfect skill for K ≥ 7 geomagnetic disturbances were archived by RWC Japan, together with RWC Australia. Although continued efforts to improve forecast performance are still significant subject, we anticipate that the evaluation presented here will support appropriate implementation of effective measures to mitigate space weather disaster risks to modern social infrastructure. Space weather forecast evaluation Solar flare Geomagnetic disturbance Ionospheric Storm International Space Weather Service (ISES) Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 1. Introduction National institute of information and communications technology (NICT) and its predecessor organizations of Japan have been providing space environment forecasts and warnings continuously since 1949 (Shiota et al., 2021 ). Initially, NICT have provided information on ionospheric disturbance affecting shortwave radio propagation. To forecast ionospheric disturbance and also as space technology development expanded to higher altitudes, type of space environment information that needs to be provided have extended to include geomagnetic disturbance, high energy particles and solar activity which influence the ionosphere. Since 1965, NICT’s predecessor organizations have participated in the International Ursigram and World Day Service (IUWDS), which was an international framework for space environment forecasting established under Union Radio-Scientifique Internationale (URSI). After IUWDS was renamed the International Space Environment Service (ISES) in 1996, NICT continued to function as the Western Pacific Regional Warning Center (RWC Japan) and provided daily space weather forecasts. (Nagatsuma et al., 1997 ). In this framework, NICT provides space weather nowcasts and forecasts every day, and bulletins immediately after higher level space weather event occurrences as operational information service. This paper reports evaluation of space weather forecasts issued by NICT during May 1 to 31, 2024 when multiple and multi-kind extreme space weather events were observed. The goal of this paper is to evaluate forecasts of extreme space weather events. Briefly explaining space weather events for a month in May 2024, Multiple solar flares including X-class solar flares occurred and caused Dellinger phenomena (short wave fadeouts) in the day-side ionosphere. Some of these solar flares were accompanied by earth-directed coronal mass ejections (CMEs), so-called full-halo CME. These solar flares and subsequent CMEs generated solar energetic particles resulting in enhancement of solar protons at the geostationary orbit of Earth. Just after the arrival of CMEs at the Earth magnetosphere, an extreme geomagnetic storm (minimum Dst index (provisional) = -406 nT) occurred and local K = 8 was observed at Kakioka Geomagnetic Observatory, Japan, for the first time in 19 yeas since August 2005. Subsequently, severe ionospheric negative storms with an I-scale=I N 3, which is extreme density depletion, were observed all over Japan. The societal impacts of these extreme events have been reported both domestically in Japan and internationally. For example, in Japan, during the time of the ionospheric storm, increases in the error of satellite positioning, impacts on shortwave communications used for aviation and amateur radio were reported. Overseas, the National Oceanic and Atmospheric Administration (NOAA) reported that there were impacts on satellite positioning, shortwave communications, and the power sector. In New Zealand, it was reported that measures were taken to suspend some power transmission services as an emergency response in the power sector ( https://swc.nict.go.jp/report/topics/202405101630.html ). It has thus become clear that occurrence of extreme space weather events would have some impacts on some of modern social infrastructure. Space weather forecast evaluations have been previously reported by Crown ( 2012 ), Devos et al., ( 2014 ), and Kubo et al. ( 2017 ). They evaluated each forecast performance of RWC USA in 1996–2008, RWC Japan in 2000–2015, and RWC Belgium in 2004–2012, respectively. In those papers, evaluation of extreme event in solar cycle 2023 were included. However, followed solar cycle 24 (from 2008 to 2019) was relatively moderate with a few extreme events even in that maximum. Thus, opportunities to forecast extreme space weather events have been limited in recent decades. With such background, May 2024 extreme space weather events occurred for a long year and are important and valuable to summarize forecasts and evaluations. Hereafter, section 2 describes the NICT’s space weather forecast targets and definitions. Section 3 summarizes the issued forecasts and corresponding observations in May 2024 and evaluates the overall accuracy of NICT’s multi‑category forecasts. Section 4 evaluates the performance of maximum‑level forecasts. Section 5 compares the forecast performance of NICT with other ISES RWCs. Section 6 explains the observational and modeling basis used for forecasts. Lastly, section 7 summarizes all evaluation results. 2. Forecast level definition This section explains space weather forecast targets and definition of each event levels. NICT provides forecasts for seven types of space weather targets: “solar flare”, “solar proton”, “geomagnetic disturbance”, “radiation belt electrons”, “ionospheric storm”, “Dellinger phenomena (short wave fadeout)”, and “sporadic E-layer”. Details on these seven targets are described on the website < https://swc.nict.go.jp/en/knowledge/criteria_icon.html%3E . NICT issues forecast three times a day at 00:00 UT (9:00 JST), 6:00 UT (15:00 JST), and 12:00 UT (21:00 JST), providing maximum levels for the next 24 hours after issuing timing for each space weather targets. This paper focuses on three space weather targets: solar flare, geomagnetic disturbance, and ionospheric storm. The forecast level definition for these three targets is shown in Table 1. Solar flare forecasts are classified into four levels based on X-ray observations (0.1–0.8 nm) by Geostationary Operational Environmental Satellite (GOES): B-class or less, C-class, M-class, and X-class. Geomagnetic disturbance forecasts are classified into five levels based on K index at Kakioka, Japan: K ≤ 3, K = 4, K = 5, K = 6, and K ≥ 7. Classification of solar flare and geomagnetic disturbance forecasts are according to the definition of UGEOA code, which is one of URSIgram codes used by ISES/RWCs to share forecast data < http://www.spaceweather.org/ISES/code/aaf/ugeoa.html%3E . On the other hand, NICT provides unique ionospheric storm forecasts using our own scale. Condution of ionosphere over Japan are classified into three levels using the I-scale based on ionospheric F-region critical frequency, foF 2 observations by ionosondes at four site (Wakkanai, Kokubunji, Yamagawa, and Okinawa) in Japan and ionospheric total electron content (TEC) calculated from GNSS earth observation network System (GEONET) data of Geospatial Information Authority of Japan (GIS). The levels of ionospheric storm defined as the largest I-scales among the TEC-based I-scales at five latitudinal bands (45, 41, 37, 33, and 29°N) and the foF 2 -based I-scales at the four sites. The I-scale represents the relative level of deviations from 27-day median value, compared with its standard deviation at each latitude, local time, and season over 18 years: I N 3 (-3σ or less ), I N 2 (-3σ to -2σ), I N 1(-2σ to -1σ), I0 (-1σ to + 1σ), I P 1(+ 1σ to + 3σ), I P 2 (+ 3σ to + 5σ), I P 3 (+ 5σ or more ). The denotation N and P represent positive and negative ionospheric storm, respectively. In this paper, I-scale level 0 is denoted as I-scale ≤I1, level 1 as I-scale = I2, and level 2 as I-scale = I3. Details on the I-scale definition is described in Nishioka et al. ( 2017 ). 3. Forecasted levels and observation results In this paper, total 93 forecasts over one month (31 days) in May 2024 were evaluated. A list of forecasted levels and observation levels of solar flares, geomagnetic disturbances, and ionospheric storms is attached as an additional tabular data. Table 2 shows multi-categorical contingency tables of (a) solar flare, (b) geomagnetic disturbance, and (c) ionospheric storm forecasts. The labeled number of observation and forecast axes stand for the defined activity levels shown in Table 1. The diagonal components of multi-categorical contingency tables represent the counts of correct forecasts at each level. Elements to the left of the diagonal represent counts of over-forecast cases, whereas elements below the diagonal indicate counts of under-forecast cases. The number of correct solar flare forecast counts for C-class, M-class, and X-class were 2, 16, 31, respectively, so among 93 forecasts issued during May 2024 totally 49 forecasts were verified as correct. For geomagnetic disturbance forecast, the number of correct forecast counts for K ≤ 3, K = 4, K ≥ 7 were 44, 4, 6, respectively, and total 54 counts of correct forecasts were issued. For ionospheric storm forecast, the number of correct forecast counts for I-scale ≤ I1, I-scale = I2, I-scale = I3 were 49, 2, 9, respectively, and total 36 counts correct forecasts were issued. Observed event counts by level in May 2024 were; solar flares C-class: 11, M-class: 16, X-class: 31; geomagnetic disturbances K ≤ 3: 44, K = 4 : 3, K = 5: 2, K = 6: 1, K ≥ 7: 6; ionospheric storms I-scale = I1: 49, I-scale = I2: 7, I-scale = I3: 2. It is noteworthy that in May 2024 all levels of solar flare, geomagnetic disturbance, and ionospheric storm events occurred without solar flare of ≤ B class (level 0). The climatological relative frequencies (P c ) of maximum-level event occurrences in May 2024 were, X-class solar flares: Pc = 42/93 ≈ 0.45 (45%), geomagnetic disturbances with K ≥ 7: Pc = 6/93 ≈ 0.06 (6%), and ionospheric storms with I-scale = I3: Pc = 16/93 ≈ 0.17 (17%). The overall accuracy of NICT’s multi-category space weather forecasts in May 2024 can be evaluated using a verification index PC m , which is the proportion correct extended to a multi-categorical contingency table (Jolliffe & Stephenson 2012 ). PC m is equivalent to values that is counts of correct forecasts divided by total counts of forecasts issued. So, PC m are derived as PC m = (0 + 2+16 + 31)/93 ≒ 0.53 for solar flares forecast, PC m = (44 + 4+0 + 0+6)/93 ≒ 0.58 for geomagnetic disturbances forecast, and PC m = (49 + 2+9)/93 ≒ 0.65 for ionospheric storms. All three forecasts succussed to issue correct level forecasts for more than half interval. 4. Maximum-level forecast evaluation The goal of this paper is to evaluate forecasts of extreme space weather events. So, focusing on maximum level, forecasts and observations were classified into two categories “maximum level” and “others”, and evaluated their accuracy, discrimination, reliability, bias, and skill using two-categorical contingency tables shown in Table 3. Maximum level of solar flare forecast is X-class (level 3), that of geomagnetic disturbance forecast is K ≥ 7 (level 4), and that of ionospheric storm forecast is I-scale=I3 (level 2) as defined in Table 1. A 2×2 matrix of contingency table consists of hit counts: a, false alarm counts: b, miss count: c, correct rejection count: d. (see Table 3d). Hit, false alarm, miss, correct rejection counts of X-class solar flare forecasts were 31, 18, 11 and 33, respectively (Table 3a). These of K ≥ 7 geomagnetic disturbance forecasts were 6, 1, 0, and 86, respectively (Table 3b). These of I-scale=I3 ionospheric storms forecasts were 9, 2, 7, and 75, respectively (Table 3c). Verification indices of proportion correct (PC), probability of detection (POD), false alarm ratio (FAR), frequency bias (FB), and equitable threat score (ETS) can be derived from elements of a contingency table. PC represent accuracy by the ratio of the number of correct forecasts of both maximum level and others to the total number of samples and is defined by PC = (a + d) / (a + b + c + d). Higher PC values indicate higher forecast accuracy, and PC = 1 is optimal. POD represent discrimination of maximum-level forecasts by the ratio of the number of hit counts to the number of total maximum-level observation counts and is defined by POD = a / (a + c). Higher POD values indicate a lower count of miss, and POD = 1 is optimal. FAR represent reliability of maximum-level forecasts by the ratio of the number of false alarms to the total number of maximum-level forecasts and is defined by FAR = b / (a + b). Lower FAR values indicate a lower count of false alarms, then FAR = 0 is optimal. FB represent bias by the ratio of the total number of maximum-level forecasts to the total number of maximum-level observations and is defined by FB = (a + b) / (a + c). FB greater than 1 indicates an over-forecast tendency and FB less than 1 indicate under-forecast tendency. Lastly, ETS represent skill focusing on the hits counts of maximum-level forecasts by the ratio of the number of hit counts to the total number of counts other than correct rejections with the removal of contribution from hits by chance in random forecasts, and is defined by ETS = ( a – a r ) / ( a + b + c – a r ) where a r = P c ( a + b ) is the number of hit forecasts by chance (random hits). Proximity to ETS = 1 indicates higher forecast accuracy, while ETS = 0 for random forecasts. Details of these verification indices is described in Jolliffe & Stephenson ( 2012 ) and guideline of WMO-No.1364 (2025). Verification indices of X-class solar flare forecasts were calculated: PC = 0.69, POD = 0.74, FAR = 0.37, FB = 1.17 and ETS = 0.23, summarized in Table 4. These values indicate reasonable discrimination with a moderate false-alarm ratio and a slight over-forecasting tendency for X-class events. The most misses and false alarms were due to failures in forecasting the start and end of X-class flare activity, respectively, suggesting that improvement of timing of prediction remains a key challenge. Next, for forecasts of geomagnetic disturbance with K ≥ 7 which is the maximum level in the forecast definition, verification indices of K ≥ 7 forecast were calculated: PC = 0.99, POD = 1.00, FAR = 0.14, FB = 1.17, and ETS = 0.85. These results indicate that NICT achieved perfect discrimination capability for extreme geomagnetic disturbances, with very high overall accuracy and only one false alarm at termination timing. Thus, the frequency bias slightly exceeded 1, suggesting a minor over-forecasting tendency. Lastly, focusing on I-scale=I3 ionospheric storms forecasts, verification indices were calculated: PC = 0.90, POD = 0.56, FAR = 0.18, FB = 0.69, and ETS = 0.44. These results indicate that while overall accuracy was high, the discrimination capability for extreme ionospheric storms was moderate, and the forecasts tended to under-forecast tendency. There were cases being missed because the cause of I-scale=I3 occurrence was unknown, resulting in moderate POD and under-forecast tendency. To enhance the forecast accuracy of ionospheric storms, further investigations into their underlying mechanisms and strengthening observations are required. 5. Comparison with ISES RWC forecasts ISES RWCs provides space weather information to users in their regions and exchange data and forecasts among ISES members. This section provides a comparative analysis of space weather forecast evaluations across RWCs. Although forecast level definitions vary by country, five RWCs including NICT (Japan) issue solar flare and geomagnetic disturbance forecasts in accordance with UGEOA standards, which is one of URSIgram codes used by ISES/RWCs to share forecast data http://www.spaceweather.org/ISES/code/aaf/ugeoa.html , enabling direct comparison of forecast evaluations. The five countries are Australia (00:00 UT), USA (03:30 UT), Indonesia (08:00 UT), Belgium (12:30 UT), and Japan (06:00 UT). The times in parentheses indicate timing of forecast issued from each RWC. Although NICT issues forecasts three times a day, only the 06:00 UT forecasts are shared through UGEOA. So, the comparison of space weather forecast with other RWC was made by recalculating the verification index using only the 06 UT forecasts in order to compare with same basis (once a day forecast). Forecast issuing agencies in each country are as follow: Australia is the Australian Space Weather Forecasting Centre (ASWFC) of Bureau of Meteorology, USA is the Space Weather Prediction Center (SWPC) of NOAA, Indonesia is the Space Weather Information and Forecast Services (SWIFtS) of National Research and Innovation Agency, and Belgium is the Solar Influences Data Analysis Center (SIDC) of Royal Observatory of Belgium. Table 5 shows contingency tables of five RWC’s solar flare and geomagnetic disturbance forecasts. As same with Table 3 focusing on extreme events, forecasts were classified into two categories maximum (X class or K ≥ 7) and others. Because Indonesia and the USA did not issue forecasts infrequently, the sample count used for evaluation was smaller than that of other RWCs. All five RWCs base their solar flare forecast on same GOES X-ray observations. However, due to variation in forecast issuance times among RWCs, the maximum solar flare level considered and observed within each forecast interval could be differ. Thus, the sample counts of intervals containing X-class solar flare observations varies across RWCs. The counts of (hit, false alarm, miss, and correct rejection) of each RWC X-class solar flare forecasts were as follows, RWC Australia:(8, 6, 5, 12), RWC USA: (9, 0, 4, 16), RWC Japan: (11, 5, 4, 11), RWC Indonesia: (10, 4, 4, 11), and RWC Belgium: (3, 0, 11, 7). RWC Japan and Indonesia have larger hit counts but also have some false alarm and miss counts. On the other hand, RWC USA and Belgium provided no false alarm for X class solar flare occurrence in this period. For geomagnetic disturbance forecast, each RWC targets each local K index in their region. So, targeting observation as a measure of geomagnetic disturbance differ depending on RWCs. RWC Australia targes local K index calculated by ASWFC based on multi geomagnetic observations in the Australian region. RWC USA targes local K index at Boulder Magnetic Observatory. RWC Japan targets local K index at Kakioka Magnetic Observatory. RWC Belgium targes local K-index at Dourbes Geomagnetic Observatory. Only RWC Indonesia targes planetary K index (K p ), which is a global geomagnetic activity index. Therefore, we evaluated each RWC’s forecast for each targeting measure except for Belgium, of which targeting local K-index observation was missing and local K-index at Chambon-la-Forêt magnetic observatory locating near Dourbes was used alternatively as same as Devos et al. ( 2014 ). The counts of (hit, false alarm, miss, and correct rejection) of each RWC geomagnetic disturbance forecasts with K ≥ 7 are as follows, RWC Australia:(2, 0, 0, 29), RWC USA: (2, 0, 1, 26), RWC Japan: (2, 0, 0, 29), RWC Indonesia: (0, 1, 2, 25), and RWC Belgium: (1, 2, 1, 27). Surprisingly, K ≥ 7 forecasts of RWC Australia and Japan succussed to provide no false alarm or miss. Four verification indices POD, FAR, FB, and ETS of X-class solar flare and K ≥ 7 geomagnetic disturbance forecasts in May 2024 by five RWCs including Japan were calculated using these contingency tables (Table 5) and are summarized in Table 6. For X-class solar flare forecasts, the POD varied significantly among RWCs, with the highest discrimination capability achieved by RWC Japan (POD = 0.73), followed closely by RWC Indonesia (POD = 0.71) and RWC USA (POD = 0.69). RWC Australia showed moderate discrimination (POD = 0.62), while RWC Belgium exhibited the lowest (POD = 0.21). In terms of reliability, optimal (FAR = 0.00) was achieved by RWC USA and Belgium, whereas RWC Australia (FAR = 0.43), Japan (FAR = 0.31), and Indonesia (FAR = 0.29) have provided some false alarms indicating they have subject to reliability. The bias was close to unity for most RWCs, suggesting balanced forecasts, except for Belgium (FB = 0.21), which strongly under-forecast tendency for X-class events. Regarding skill, the ETS ranged from 0.55 by RWC USA to 0.13 by Belgium, with Japan (ETS = 0.27) and Indonesia (ETS = 0.29) showing moderate skill and Australia slightly lower (ETS = 0.16). Because ETS is calculated by excluding random hits that depend on the climatological relative frequency, forecasts with high POD but also having many false alarms, such as those issued by Japan and Indonesia, were evaluated less favorably despite their strong discrimination capability. For geomagnetic disturbance forecasts of K ≥ 7, RWC Japan and Australia achieved perfect discrimination (POD = 1.00), while POD of USA showed high but not perfect (POD = 0.67). Belgium demonstrated moderate capability (POD = 0.50), and Indonesia failed to detect any K ≥ 7 events (POD = 0.00). Reliability of RWC Australia, USA, and Japan’s forecast was optimal (FAR = 0.00), whereas RWC Belgium and Indonesia was higher values (FAR = 0.67 and 1.00, respectively). The bias was ideal for RWC Australia and Japan (FB = 1.00), slightly below unity for RWC USA (FB = 0.67), and significantly above unity for RWC Belgium (FB = 1.50) indicating an over-forecasting tendency. RWC Indonesia showed FB = 0.50, reflecting strong under-forecasting. In terms of skill, ETS was highest for RWC Australia and Japan (ETS = 1.00), followed by USA (ETS = 0.64) and Belgium (ETS = 0.21), while Indonesia showed negative skill (ETS = − 0.02), indicating performance worse than random chance. Overall, for X-class solar flare forecasts there is no RWCs that have achieved best performance for all four verification indices, but each RWC has own strength and tendency. For example, RWC Japan’s forecast was the highest discrimination capacity, RWC Belgium’s forecast was characterized by best reliability, RWC Indonesia’s forecast was most balanced neither under- nor over-forecast tendency, and RWC USA’s forecast was characterized by best reliability as well as the highest skill score. On the other hand, for K ≥ 7 geomagnetic disturbance, RWC Australia and Japan’s forecast achieved optimal values for all four verification indices. Other RWC’s forecast failed forecast the beginning and termination timings of geomagnetic disturbance occurrence and resulted in verification indices were not optimal. 6. Forecast basis This section describes space weather conditions before the extreme event occurrences and basis for determining forecast levels of solar flares in section 6.1, geomagnetic disturbances in section 6.2, and ionospheric storms in section 6.3, respectively. 6.1. Solar flare forecasts In May 2024, total 21 events X-class solar flares were observed in a month. Notably, the occurrence of seven X-class flares within 72 hours was a first in the history of X-ray observations by GOES since 1975. The largest solar flare in this series was the X8.7 flare on May 14. Here, we reconsider two forecasts issued at 00:00 UT and 6:00 UT on May 14, prior to the occurrence of X8.7 solar flare which reached its peak intensity at 16:51 UT on the day. At 00 UT on May 14, it is approximately 17 hours before the occurrence of X8.7 solar flare, RWC Japan was not anticipated the occurrence of X-class solar flares within the next 24 hours. Figure 1 shows parts of imaging data of (a) intensitygram and (b) magnetogram taken by Helioseismic and Magnetic Imager (HMI) (Scherrer et al. 2012 ; Schou et al. 2012 ) on the Solar Dynamic Observatory (SDO) satellite (Pesnell et al. 2012 ), which were analyzed at forecast briefing before 00 UT on May 14. The 13 regions are indicated by the red circles and with NOAA’s active region numbers. Figure 1 (c) shows soft X-ray light curve by GOES-16. Occurrence timings of X-class solar flares are indicated by white arrows and most of these X-class flares were produced by active region 13664. At this time, based on histories of solar flare occurrences, and sizes of sunspot area (1170), its type ( Fkc ) and magnetic field structure ( βγδ ) according to the solar region summary published by NOAA/SWPC, NICT’s space weather forecaster had designated only the active region 13664 have capability of X-class solar flare occurrences. Details on the sunspot type and magnetic field structure is described in McIntosh ( 1990 ). However, it was quite hard to assess the detailed structure, because the active region 13664 was located at the western limb of the solar surface (S19/W87) corresponding to the edge of image where apparent structure distorted. Looking carefully at data of HMI/SDO, there was no noticeable changes of magnetic structure were visible in sequences from previous day. In addition, X-ray observations by GOES showed that peak intensities of solar flares had been decreasing after the X5.8 flare on 11 May, and it took about 31 hours since the last X1.2 flare was observed. Based on these circumstances and the fact that the largest solar flare in the past 24 hours was M6.6 (peaked at 9:44 UT on May 13), forecasters suspected that activity of active region 13644 had weakened, and level of solar flares over next 24 hours would be likely to remain at M-class (this forecast failed). The forecast at 6:00 UT, approximately 11 hours before the X8.7 flare occurrence, forecasters had updated the maximum solar flare level over the next 24 hours to X-class. The reason for updating the forecast to X-class was that an X1.7 flare occurred in active region 13664 at 02:09 UT, approximately two hours after the incorrect forecast was issued at 00 UT. At 06 UT, the active region had already reached at 90 degrees west on the solar surface, it is impossible to observe its most of the area. However, the occurrence of the X1.7 flare let us know that the region had been still active, and forecasters could obtain bases to update the forecast level from M- to X-class. After seven hours since the forecast updated an X1.2 flare occurred, and 11 hours later an X8.7 flare occurred. Meanwhile, regarding the X5.8 flare that occurred in active region 13664 on 11 May, which is the largest solar flare observed on the solar disk in May 2024, has been suggested that the occurrence of an X-class flare may have been anticipated in advance. This suggestion is based on a detailed analysis of the magnetic field configuration derived from a physics-based model of the active region, together with observations of precursor brightening phenomena (Bamba et al., under review). Therefore, precise full-disk observational data is crucial for solar flare forecasting. However, near the solar limb, significant distortion in observed images makes accurate assessment of their structure impossible. Furthermore, at that time, unfortunately the STEREO-A spacecraft, which is another near-real-time solar imaging spacecraft, was orbiting at roughly the same longitude as Earth, meaning there was no way to observe the active region 13664 from angles different from those viewing from Earth. To improve the performance of solar flare forecasting, multi-angle solar observations constitute a critically important approach, such as new mission viewing sun from the Lagrange-5 point. 6.2. Geomagnetic disturbance forecast According to the Japan Meteorological Agency's Kakioka Geomagnetic Observatory, geomagnetic storm started with an occurrence of a sudden commencement at 17:05 UT on May 10 and it ended at approximately 4:00 UT on May 14. Figure 2 shows time series data of Kakioka K index including the period of the geomagnetic storm. During this period, local K = 8 was observed four times (total 12 hours) and K = 7 was observed three times (total 9 hours). It has been almost 19 years since local K = 8 was lastly observed at Kakioka in August 2005. Here, we look back geomagnetic disturbance forecast before the geomagnetic storm occurrence. Figure 3 shows time series data plots of real-time solar wind observations at Lagrange 1 (L1) points. At 00 UT on May 10, it is 15 hours before the occurrence of K ≥ 7 geomagnetic disturbances, NICT’s geomagnetic disturbance forecast for the next 24 hours was updated from a previous forecast of K ≤ 3 (level 0) to K ≥ 7 (maximum level). This was optimal timing. At that time, solar wind velocity over the past 24 hours was approximately 430 km/s, density was 2 to 8 cm − 3 , and interplanetary magnetic field intensity was approximately 5 nT, indicating nominal state as shown in Fig. 3 . Consequently, geomagnetic activity was quiet with K ≤ 3 as shown in Fig. 2 . There were no coronal holes visible around the equator of the solar surface, and arrival of high-speed winds from coronal holes was not expected. Meanwhile, multiple CMEs had been observed since a few days ago. At the moment of 00 UT on May 10, seven full-halo CMEs were identified by the large angle and spectrometric coronagraph experiment on solar and heliospheric observatory (LASCO/SOHO) and the coronagraph on solar terrestrial relation observatory (COR/STEREO). Those had erupted from around active region 13644 on May 8 to 9. The arrival dates and times of these CMEs were predicted using solar storm forecast system SUSANOO-CME (Shiota and Kataoka, 2016 ; Shiota and Yashiro, 2021). Figure 4 a shows parameters of seven CMEs that concerned to arrive at the Earth’s magnetosphere. The table lists the occurrence time of CME eruption, the heliographic latitude and longitude of estimated source region, associated active region number and flare class, and initial velocity of CMEs. All these seven CME were included to SUSANOO-CME simulation. Figure 4 b shows timeseries plots of simulated solar wind speed, density, and magnetic field intensity, azimuth angle, and north-south component. These simulated parameters are overplotted with real-time observation data at L1 point. Figure 4 c shows the simulated solar wind speed distribution on the XY plane in Heliocentric Earth Ecliptic coordinate system at the timing of blue line in Fig. 4 b with colors indicating difference value from the background solar wind speed. SUSANOO-CME simulation predicted that the first two CMEs, which erupted at 05 UT and 12 UT on May 8, would merge during their propagations and arrive at the L1 point at noon on May 10 as a shock wave. The simulation also predicted after the arrival of CMEs solar wind speed would increase approximately 700 km/s, magnetic field intensity would increase roughly up to 15–20 nT and its vector would initially direct southward. Based on these solar wind simulation results, geomagnetic disturbance level as is K index was determined to being K ≥ 7 using look-up table based on past statistics. In reality, a first shock wave of multiple CMEs arrived at the L1 point at approximately 16:30 UT on May 10. It is about only 4-hour difference with SUSANOO-CME prediction. With the arrival of the CME shock wave, solar wind speed suddenly increased to 770 km/s, interplanetary magnetic field intensity increased to 72 nT, and north-south component of magnetic field reached − 50 nT, as shown in Fig. 3 . This is the best practice of correct forecast could be made based on solar wind simulation with appropriate input parameters of CMEs. Post-analysis of these CMEs using SUSANOO-CME will be described in a separate paper (Shiota et al. in preparation). 6.3. Ionospheric storm forecast Occurrence of a severe negative ionospheric storm (I-scale=I N 3) was confirmed after 18 UT on the same day following the first shock wave of multiple CMEs arrived around 16:30 U, and a geomagnetic storm soon began at 17 UT on May 10. Figure 5 shows timeseries plots of (a) foF 2 measurements observed by four ionosondes at Wakkanai, Kokubunji, Yamagawa, and Okinawa in Japan and (b) GEONET TEC from latitude 29°N to 45°N along Japan longitudes. The foF 2 timeseries plot at Wakkanai indicates a severe negative ionospheric storm with I-scale=I N 3 that occurred at 18 UT and continued until 9 UT on May 11 (from 03:00 to18:00 Japanese local time (JST)). Although it temporarily returned to ≤I1 levels during the night and temporarily positive storm occurred, a severe negative storm with I-scale=I N 3 started again around 18:00 UT on May 11 (03:00 JST), and continued for next 24 hours at Wakkanai. During this period, a severe negative storm with I-Scale=I N 3 was simultaneously observed at Okinawa during the daytime (6–15 JST). Furthermore, GEONET TEC plots show that a negative storm with I-scale=I N 2 was observed between 45°N and 29°N latitudes from 0:00 UT to 18:00 UT on May 11. Here, we reconsider ionospheric storm forecasts at 00 UT and 06 UT on May 10 before the occurrence of severe ionospheric storms. The forecast at 00 UT which was 18 hours before the onset of a severe negative storm, geomagnetic disturbance with K ≥ 7 was forecasted within the next 24 hours as described in section 9.2. Although K ≥ 7 was expected, occurrence frequency of K ≥ 7 was extremely low and available past data were limited only eleven days since 1997. In the forecast at 00:00 UT, a rare extreme event of K ≥ 7 geomagnetic disturbance was predicted, but due to the lack of experience and knowledge of forecasters deciding on the ionospheric storm forecast level at such situation, a moderate level of I-scale = 2 was forecasted without careful consideration. However, for the next forecast at 06:00 UT which was 12 hours before the onset of I N 3 ionospheric storm, last forecast was reviewed. According to past records, it was found that among 11 cases when the daily maximum K index was ≥ 7, six cases were I3, three cases were I2, and two cases were ≤I1. Based on this, the forecast level was revised to I3. This change resulted in a correct revision to the forecast, but since past I-scale=I3 occurrence frequency (6/11) is not enough high and the number of sample data is insufficient, so there is a possibility that the forecast may be incorrect. Further data analysis and physics-based simulation are needed to improve performance of ionospheric storm forecasts when K ≥ 7 is expected. 7. Summary NICT provides space weather information as Regional Warning Center (RWC) Japan of International Space Environment Service (ISES). This study evaluated NICT’s space weather forecasts during May 2024, which is a period characterized by multiple extreme events including X-class solar flares, severe geomagnetic disturbances (K ≥ 7), and ionospheric storms (I-scale = I3). The results obtained in this study are briefly summarized as follows. Multi-category forecast accuracy (PC m ) of NICT’s forecasts was 0.53 for solar flares, 0.58 for geomagnetic disturbances, and 0.65 for ionospheric storms, indicating moderate overall performance. For extreme space weather events in May 2024, NICT archived almost perfect performance for K ≥ 7 geomagnetic disturbance forecasts (PC = 0.99, POD = 1.00, FAR = 0.14, FB = 1.17, ETS = 0.85), while X-class solar flares forecast was characterized by high discrimination (POD = 0.74) but subject to reliability (FAR = 0.37) and ionospheric storms forecast for I-scale=I3 is high accuracy (PC = 0.90) but subject to discrimination ability (POD = 0.56). Comparative analysis across five ISES/RWCs revealed that RWC USA demonstrated highest skill for solar flare forecasts (ETS = 0.55) with zero false alarms. RWC Japan and Australia achieved perfect skill (ETS = 1.00) for K ≥ 7 geomagnetic disturbance forecasts. NICT succussed to forecast the start and termination of geomagnetic disturbances of K ≥ 7 using the solar storm forecast system SUSANOO-CME. Extreme space weather events can affect modern social infrastructure particularly communications, positioning, power sector, and any space-based systems, and these failures may cause space weather disasters in our society. As with weather forecast on ground, disasters can be mitigated by utilizing forecasts. We hope that with understanding current ability of space weather forecast still having possibility of miss or false forecasts as evaluated by this study, appropriately space weather forecasts are included in counter measures to mitigate disasters. Also, we hope that new methods which can predict extreme space weather events with a higher score than current space weather forecasts given in this paper will be developed in future. Declarations Ethics approval and consent to participate This study does not contain human participants, human data or human tissue. Consent for publication This study does not contain any individual person’s data. Availability of data and materials Data set related on this study are attached as an additional supporting file. Competing interests No competing interest exists. Funding This study was supported by Ministry of Internal affairs and Communications, Japan. Authors' contributions All authors contributed to space weather forecast service operation. Author#1 contributed to evolution analysis. Acknowledgements We thank for RWC Australia (ASWFC), RWC USA (NOAA/SWPC), RWC Indonesia (SWIFtS), and RWC Belgium (SIDC) to share space weather forecast data. We thank for SDO/HMI, GOES/X-ray, Kakioka Geomagnetic Observatory, and ACE/DSCOVR teams for providing their real-time solar wind observation data. A part of these results was obtained from “Promotion of observation and analysis of radio wave propagation”, commissioned by the Ministry of Internal Affairs and Communications, Japan. Authors' information N/A Endnotes N/A References Y. Bamba, D. Shiota, and K. Kusano, Characteristics of Magnetic Field Evolution and Onset Process of Successive X-class Flares in May 2024, under review, EPS special issue, https://assets-eu.researchsquare.com/files/rs-7986926/v1_covered_895b8347-84a6-43e9-846e-c228f363925a.pdf Crown, MD (2012), Validation of the NOAA Space Weather Prediction Center's solar flare forecasting look-up table and forecaster-issued probabilities, Space Weather, 10, S06006, doi:10.1029/2011SW000760. Devos A, Verbeeck C & Robbrecht E (2014), Verification of space weather forecasting at the Regional Warning Center in Belgium. J. Space Weather Space Clim., 4, A29 Guidelines on the Verification of Hydrological Forecasts (2025), WMO-No. 1364, World Meteorological Organization (WMO), ISBN 978-92-63-11364-0, https://library.wmo.int/idurl/4/69478 Jolliffe, I.T. and Stephenson, D.B. (2012) Forecast Verification: A Practitioner’s Guide in Atmospheric Science. 2nd Edition, Wiley-Blackwell, Oxford. Kubo Y, Den M & Ishii M. Verification of operational solar flare forecast: Case of Regional Warning Center Japan, J. Space Weather Space Clim., 7, A20, 2017, DOI: 10.1051/swsc /2017018. McIntosh, PS (1990), The classification of sunspot groups, Sol. Phys., 125(2), 251–267, doi:10.1007/BF00158405. Murphy, AH, and RL Winkler (1987), A general framework for forecast verification, Mon. Wea. Rev., 115, 1330. Nagatsuma, T., K. Ooyama, A. Okano, and M. Akioka (1997), SPACE ENVIRONMENT FORCAST SERVICE,Review of the Communications Research Laboratory, Vol.43, No.2, p301-308. Nishioka, M., T. Tsugawa, H. Jin, and M. Ishii (2017), A new ionospheric storm scale based on TEC and foF2 statistics, Space Weather, 15, 228-239, doi:10.1002/2016SW001536 Pesnell, W., Thompson, B. J., & Chamberlin, P. C. (2012), The Solar Dynamics Observatory (SDO), Solar Physics, 275, 3, doi:10.1007/s11207-011-9841-3. Scherrer, P. H., Schou, J., Bush, R. I., et al. (2012), The Helioseismic and Magnetic Imager (HMI) Investigation for the Solar Dynamics Observatory (SDO), Solar Physics, 275, 207, doi:10.1007/s11207-011-9834-2. Schou, J., Scherrer, P. H., Bush, R. I., et al. (2012), Design and Ground Calibration of the Helioseismic and Magnetic Imager (HMI) Instrument on the Solar Dynamics Observatory (SDO), Solar Physics, 275, 229, doi:10.1007/s11207-011-9842-2. Shiota, D., and R. Kataoka (2016), Magnetohydrodynamic simulation of interplanetary propagation of multiple coronal mass ejections with internal magnetic flux rope (SUSANOO-CME), Space Weather, 14, 56–75, doi:10.1002/2015SW001308. Shiota, D., and Yashiro (2021), Real-time Prediction System for Solar Storm Arrival, Journal of NICT, ISSN 2433-6009, Vol.67, No.1, pp137-142. Shiota, D., K. Sakaguchi, and K. Fukunaga (2021), Digitalization of Historical Space Weather Records, Journal of NICT, ISSN 2433-6009, Vol.67, No.1, pp203-210. Smith, SF, and R. Howard (1968), Magnetic classification of active regions, in Structure and Development of Solar Active Regions, edited by KO Kiepenheuer, pp. 33–42, D. Reidel, Dordrecht, Netherlands. Tables Tables 1 to 6 are available in the Supplementary Files section. Supplementary Files Table01.xlsx Table 1 Definition of NICT's space weather forecast levels. Table02.xlsx Table 2 Multi-categorical contingency tables of NICT’s space weather forecasts in May 2024. Table03.xlsx Table 3 Contingency tables of NICT’s maximum-level space weather forecasts. Table04.xlsx Table 4 Verification results of NICT’s maximum-level space weather forecasts in May 2024. Table05.xlsx Table 5 Contingency tables of five ISES/RWC’s space weather maximum-level forecasts in May 2024. Table06.xlsx Table 6 Verification results of five ISES/RWC’s space weather forecasts in May 2024. AdditionalData.xlsx Graphicalabstract.jpeg Cite Share Download PDF Status: Under Revision Version 1 posted Editorial decision: Major Revision 22 Apr, 2026 Reviewers agreed at journal 18 Mar, 2026 Reviewers invited by journal 17 Mar, 2026 Editor assigned by journal 11 Mar, 2026 First submitted to journal 10 Mar, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9090861","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":607590924,"identity":"c761aa10-55e5-4a0d-97f8-d94c12d35796","order_by":0,"name":"Kaori Sakaguchi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAyUlEQVRIiWNgGAWjYBACCRCRwGDDwAYVYCZWSxpQCzMpWhgYDhOlGAIk2w8f+/Dgz/k8PvbzBxi/VDCwmxPSIs2Tljwjse12MRtPMgOzzBkGZssGAlrkGHKMGRIbbie2STAzMEu2MTAbHCCkhf+NMUPCn3MkaJGWANqSwHYArIXxIzFaJGc8S2ZIbEtObONJNjjMcEaCsF8kzicfZvzxxy5xfvvBhw9/VNgkEwwxFHCYh0Ei2YAkLYw/GBjsSNMyCkbBKBgFIwEAABQ0NwDn/wzyAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0002-3240-1293","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":true,"prefix":"","firstName":"Kaori","middleName":"","lastName":"Sakaguchi","suffix":""},{"id":607590927,"identity":"3d06e0f5-d298-48b4-b3d6-c802ddbd5003","order_by":1,"name":"Sanae Akiyama","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Sanae","middleName":"","lastName":"Akiyama","suffix":""},{"id":607590930,"identity":"014d7a63-ee11-4055-9042-2b728f01c3b1","order_by":2,"name":"Satoshi Andoh","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Satoshi","middleName":"","lastName":"Andoh","suffix":""},{"id":607590932,"identity":"0f99e580-084d-418a-82fd-47c54a537e77","order_by":3,"name":"Yumi Bamba","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Yumi","middleName":"","lastName":"Bamba","suffix":""},{"id":607590934,"identity":"11688c6a-796b-45a6-93bb-149a003bc578","order_by":4,"name":"Kaisei Enoki","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Kaisei","middleName":"","lastName":"Enoki","suffix":""},{"id":607590938,"identity":"6335581d-9a3d-4038-9f51-d314641dbce5","order_by":5,"name":"Park Inchun","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Park","middleName":"","lastName":"Inchun","suffix":""},{"id":607590939,"identity":"82cceaa1-3616-42df-a9b1-b6ec5dee232d","order_by":6,"name":"Mamoru Ishii","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Mamoru","middleName":"","lastName":"Ishii","suffix":""},{"id":607590941,"identity":"4207b546-3713-46c7-a2f2-4f4c9af8ab5f","order_by":7,"name":"Hiromitsu Ishibashi","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Hiromitsu","middleName":"","lastName":"Ishibashi","suffix":""},{"id":607590943,"identity":"edd67167-dcf0-4522-8c46-09afcba49280","order_by":8,"name":"Hidekatsu Jin","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Hidekatsu","middleName":"","lastName":"Jin","suffix":""},{"id":607590945,"identity":"6a803f82-27a7-4bc2-817d-aadee1a33a3d","order_by":9,"name":"Yuki Kubo","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Yuki","middleName":"","lastName":"Kubo","suffix":""},{"id":607590947,"identity":"c7275e7f-f62a-4379-9c62-98333992e379","order_by":10,"name":"Satoshi Morita","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Satoshi","middleName":"","lastName":"Morita","suffix":""},{"id":607590949,"identity":"482941ed-e302-40f1-a319-47786a7d909d","order_by":11,"name":"Tsutomu Nagatsuma","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Tsutomu","middleName":"","lastName":"Nagatsuma","suffix":""},{"id":607590950,"identity":"41c9fc5a-2ce1-42f1-b3cc-dfb5b0ffc646","order_by":12,"name":"Aoi Nakamizo","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Aoi","middleName":"","lastName":"Nakamizo","suffix":""},{"id":607590952,"identity":"b8525833-b0b3-4203-b27e-e5fe8471d6de","order_by":13,"name":"Taku Namekawa","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Taku","middleName":"","lastName":"Namekawa","suffix":""},{"id":607590955,"identity":"8ad75eae-f102-4ecb-a2ef-3d0a6de3710e","order_by":14,"name":"Michi Nishioka","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Michi","middleName":"","lastName":"Nishioka","suffix":""},{"id":607590956,"identity":"573d46ca-5fff-4af9-9ffb-af95eff61871","order_by":15,"name":"Naoto Nishizuka","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Naoto","middleName":"","lastName":"Nishizuka","suffix":""},{"id":607590957,"identity":"76a6cdf3-c3cf-4713-be8c-7bee4c0cf914","order_by":16,"name":"Kenichi Otsuji","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Kenichi","middleName":"","lastName":"Otsuji","suffix":""},{"id":607590958,"identity":"82120726-5882-4b5f-84c7-3f48ada2adcb","order_by":17,"name":"Septi Perwitasari","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Septi","middleName":"","lastName":"Perwitasari","suffix":""},{"id":607590959,"identity":"9fb8cabc-ed4e-4181-b2f8-ec8087f765f8","order_by":18,"name":"Shinji Saito","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Shinji","middleName":"","lastName":"Saito","suffix":""},{"id":607590960,"identity":"4e9b5052-2647-4688-b79b-4eefc484c31a","order_by":19,"name":"Daiko Shiota","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Daiko","middleName":"","lastName":"Shiota","suffix":""},{"id":607590962,"identity":"306dcbfd-39f9-41a9-9c96-c0b09046d4a4","order_by":20,"name":"Tomohiko Sumi","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Tomohiko","middleName":"","lastName":"Sumi","suffix":""},{"id":607590965,"identity":"9dfeeb33-05ee-4be7-9e9b-a5ac192999f6","order_by":21,"name":"Naoko Takahashi","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Naoko","middleName":"","lastName":"Takahashi","suffix":""},{"id":607590967,"identity":"178aa813-f213-49b6-8c16-8d08f5f287f3","order_by":22,"name":"Chihiro Tao","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Chihiro","middleName":"","lastName":"Tao","suffix":""},{"id":607590970,"identity":"6735ffee-9371-4fb2-81e9-2ffe055ed157","order_by":23,"name":"Taku Tsugawa","email":"","orcid":"","institution":"National Institute of Information and Communications Technology Applied Electromagnetic Research Institute: Kokuritsu Kenkyu Kaihatsu Hojin Joho Tsushin Kenkyu Kiko Sogo Test Bed Kenkyu Kaihatsu Suishin Center","correspondingAuthor":false,"prefix":"","firstName":"Taku","middleName":"","lastName":"Tsugawa","suffix":""}],"badges":[],"createdAt":"2026-03-11 06:48:57","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9090861/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9090861/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":105000476,"identity":"e9c397aa-8990-45ab-8e82-4e8c05516ed2","added_by":"auto","created_at":"2026-03-19 16:43:33","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":1013318,"visible":true,"origin":"","legend":"\u003cp\u003e(a) The white light image and (b) the line-of-sight magnetogram of the solar photosphere, which were taken by HMI/SDO used at forecast briefing before 0:00 UT on 14 May 2024. (c) The light curve of soft X-ray emission taken by GOES, indicating multiple X-class flares from 7-15 May 2024”\u003c/p\u003e","description":"","filename":"Figure01.png","url":"https://assets-eu.researchsquare.com/files/rs-9090861/v1/e7797e023f5c8b3132864e0c.png"},{"id":105000478,"identity":"49d937cf-cf89-49e0-b9d3-e5710bf9f626","added_by":"auto","created_at":"2026-03-19 16:43:33","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":313891,"visible":true,"origin":"","legend":"\u003cp\u003eTime series plot of local K index (provisional data) at Kakioka Geomagnetic Observatory.\u003c/p\u003e","description":"","filename":"Figure02.png","url":"https://assets-eu.researchsquare.com/files/rs-9090861/v1/bd617bb80632e25c49b5fdad.png"},{"id":105035848,"identity":"d4429b33-3c5c-4e72-8b35-15eb82d0ddc5","added_by":"auto","created_at":"2026-03-20 07:26:44","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":784843,"visible":true,"origin":"","legend":"\u003cp\u003eTime series plot of real-time solar wind observation at L1 points.\u003c/p\u003e","description":"","filename":"Figure03.png","url":"https://assets-eu.researchsquare.com/files/rs-9090861/v1/638904a27a83f6255a4d23c6.png"},{"id":105035769,"identity":"55fcb4e0-5196-4ba7-a07b-f2ade3f88a4d","added_by":"auto","created_at":"2026-03-20 07:26:36","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":472427,"visible":true,"origin":"","legend":"\u003cp\u003e(a) Earthward full-halo CME list. (b) Timeseries plot of solar wind parameters at Earth (simulated parameters are drawn by solid red lines and other colors represent real time observation values at L1) and (c) map of simulated solar wind radial velocity distribution on the XY plane of the Heliocentric Earth Ecliptic coordinate system by SUSANOO-CME.\u003c/p\u003e","description":"","filename":"Figure04.png","url":"https://assets-eu.researchsquare.com/files/rs-9090861/v1/693e1cee0929850b24508efa.png"},{"id":105000480,"identity":"ce6baf96-07d1-42d0-a368-a452bc251509","added_by":"auto","created_at":"2026-03-19 16:43:33","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":1224588,"visible":true,"origin":"","legend":"\u003cp\u003eTimeseries plots of (a) foF\u003csub\u003e2\u003c/sub\u003e observations from four ionosondes in Japan and (b) TEC over Japan. The red lines show the observed value, and the black lines show the median value of the previous 27 days. The gray contour shows the I-scale range at each time. Flags, i.e., I\u003csub\u003eN\u003c/sub\u003e2, I\u003csub\u003eN\u003c/sub\u003e3, and I\u003csub\u003eP\u003c/sub\u003e2 appear when ionospheric storm with a duration of 2 hours or more are detected.\u003c/p\u003e","description":"","filename":"Figure05.png","url":"https://assets-eu.researchsquare.com/files/rs-9090861/v1/f17eafc41aa192f20ec31dd1.png"},{"id":105568858,"identity":"3486d525-45ea-4643-9de5-55b3d9208a9e","added_by":"auto","created_at":"2026-03-27 13:10:35","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":3901482,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9090861/v1/52954652-6e94-493a-aa57-dd0ad35a49aa.pdf"},{"id":105035321,"identity":"18936772-a449-4890-8d8d-5019c1b371b3","added_by":"auto","created_at":"2026-03-20 07:25:51","extension":"xlsx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":10318,"visible":true,"origin":"","legend":"\u003cp\u003eTable 1\u003c/p\u003e\n\u003cp\u003eDefinition of NICT's space weather forecast levels.\u003c/p\u003e","description":"","filename":"Table01.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-9090861/v1/69db4ddff271ca66c10e2fe7.xlsx"},{"id":105562642,"identity":"526e2181-1461-44f6-8b0e-cfd1825db72d","added_by":"auto","created_at":"2026-03-27 12:43:55","extension":"xlsx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":12501,"visible":true,"origin":"","legend":"\u003cp\u003eTable 2\u003c/p\u003e\n\u003cp\u003eMulti-categorical contingency tables of NICT’s space weather forecasts in May 2024.\u003c/p\u003e","description":"","filename":"Table02.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-9090861/v1/769ad8241b0a332210c86215.xlsx"},{"id":105000479,"identity":"b69e583f-5a06-40d4-bb51-97d8fc3af74c","added_by":"auto","created_at":"2026-03-19 16:43:33","extension":"xlsx","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":12177,"visible":true,"origin":"","legend":"\u003cp\u003eTable 3\u003c/p\u003e\n\u003cp\u003eContingency tables of NICT’s maximum-level space weather forecasts.\u003c/p\u003e","description":"","filename":"Table03.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-9090861/v1/9d9d6f9fa3b353536620e043.xlsx"},{"id":105000486,"identity":"7ce3b352-6203-441c-b2f1-1369eb371b2c","added_by":"auto","created_at":"2026-03-19 16:43:33","extension":"xlsx","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":12117,"visible":true,"origin":"","legend":"\u003cp\u003eTable 4\u003c/p\u003e\n\u003cp\u003eVerification results of NICT’s maximum-level space weather forecasts in May 2024.\u003c/p\u003e","description":"","filename":"Table04.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-9090861/v1/5479630edc4c34dc414dba4e.xlsx"},{"id":105000481,"identity":"40869f25-18c8-4b49-8176-7e707d889763","added_by":"auto","created_at":"2026-03-19 16:43:33","extension":"xlsx","order_by":5,"title":"","display":"","copyAsset":false,"role":"supplement","size":13937,"visible":true,"origin":"","legend":"\u003cp\u003eTable 5\u003c/p\u003e\n\u003cp\u003eContingency tables of five ISES/RWC’s space weather maximum-level forecasts in May 2024.\u003c/p\u003e","description":"","filename":"Table05.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-9090861/v1/b055461075771f6d5c5788ea.xlsx"},{"id":105000485,"identity":"cab8aee6-bb51-4e43-8e45-924499164fea","added_by":"auto","created_at":"2026-03-19 16:43:33","extension":"xlsx","order_by":6,"title":"","display":"","copyAsset":false,"role":"supplement","size":12203,"visible":true,"origin":"","legend":"\u003cp\u003eTable 6\u003c/p\u003e\n\u003cp\u003eVerification results of five ISES/RWC’s space weather forecasts in May 2024.\u003c/p\u003e","description":"","filename":"Table06.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-9090861/v1/1eda830abbc6d63930d71a51.xlsx"},{"id":105000484,"identity":"d04a8629-37bb-44b1-9247-7fb993115b64","added_by":"auto","created_at":"2026-03-19 16:43:33","extension":"xlsx","order_by":7,"title":"","display":"","copyAsset":false,"role":"supplement","size":15145,"visible":true,"origin":"","legend":"","description":"","filename":"AdditionalData.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-9090861/v1/3b9f63a140b42318da0a7453.xlsx"},{"id":105000488,"identity":"13faae4f-c853-4245-ad8e-4f9e8103c0c6","added_by":"auto","created_at":"2026-03-19 16:43:33","extension":"jpeg","order_by":8,"title":"","display":"","copyAsset":false,"role":"supplement","size":71830,"visible":true,"origin":"","legend":"","description":"","filename":"Graphicalabstract.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-9090861/v1/a96599202f1efe0651329d1e.jpeg"}],"financialInterests":"","formattedTitle":"Evaluation of NICT space weather forecast for extreme events in May 2024","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eNational institute of information and communications technology (NICT) and its predecessor organizations of Japan have been providing space environment forecasts and warnings continuously since 1949 (Shiota et al., \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e2021\u003c/span\u003e). Initially, NICT have provided information on ionospheric disturbance affecting shortwave radio propagation. To forecast ionospheric disturbance and also as space technology development expanded to higher altitudes, type of space environment information that needs to be provided have extended to include geomagnetic disturbance, high energy particles and solar activity which influence the ionosphere. Since 1965, NICT\u0026rsquo;s predecessor organizations have participated in the International Ursigram and World Day Service (IUWDS), which was an international framework for space environment forecasting established under Union Radio-Scientifique Internationale (URSI). After IUWDS was renamed the International Space Environment Service (ISES) in 1996, NICT continued to function as the Western Pacific Regional Warning Center (RWC Japan) and provided daily space weather forecasts. (Nagatsuma et al., \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e1997\u003c/span\u003e). In this framework, NICT provides space weather nowcasts and forecasts every day, and bulletins immediately after higher level space weather event occurrences as operational information service.\u003c/p\u003e \u003cp\u003eThis paper reports evaluation of space weather forecasts issued by NICT during May 1 to 31, 2024 when multiple and multi-kind extreme space weather events were observed. The goal of this paper is to evaluate forecasts of extreme space weather events. Briefly explaining space weather events for a month in May 2024, Multiple solar flares including X-class solar flares occurred and caused Dellinger phenomena (short wave fadeouts) in the day-side ionosphere. Some of these solar flares were accompanied by earth-directed coronal mass ejections (CMEs), so-called full-halo CME. These solar flares and subsequent CMEs generated solar energetic particles resulting in enhancement of solar protons at the geostationary orbit of Earth. Just after the arrival of CMEs at the Earth magnetosphere, an extreme geomagnetic storm (minimum \u003cem\u003eDst\u003c/em\u003e index (provisional) = -406 nT) occurred and local K\u0026thinsp;=\u0026thinsp;8 was observed at Kakioka Geomagnetic Observatory, Japan, for the first time in 19 yeas since August 2005. Subsequently, severe ionospheric negative storms with an I-scale=I\u003csub\u003eN\u003c/sub\u003e3, which is extreme density depletion, were observed all over Japan.\u003c/p\u003e \u003cp\u003eThe societal impacts of these extreme events have been reported both domestically in Japan and internationally. For example, in Japan, during the time of the ionospheric storm, increases in the error of satellite positioning, impacts on shortwave communications used for aviation and amateur radio were reported. Overseas, the National Oceanic and Atmospheric Administration (NOAA) reported that there were impacts on satellite positioning, shortwave communications, and the power sector. In New Zealand, it was reported that measures were taken to suspend some power transmission services as an emergency response in the power sector (\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://swc.nict.go.jp/report/topics/202405101630.html\u003c/span\u003e\u003cspan address=\"https://swc.nict.go.jp/report/topics/202405101630.html\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e). It has thus become clear that occurrence of extreme space weather events would have some impacts on some of modern social infrastructure.\u003c/p\u003e \u003cp\u003eSpace weather forecast evaluations have been previously reported by Crown (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2012\u003c/span\u003e), Devos et al., (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2014\u003c/span\u003e), and Kubo et al. (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). They evaluated each forecast performance of RWC USA in 1996\u0026ndash;2008, RWC Japan in 2000\u0026ndash;2015, and RWC Belgium in 2004\u0026ndash;2012, respectively. In those papers, evaluation of extreme event in solar cycle 2023 were included. However, followed solar cycle 24 (from 2008 to 2019) was relatively moderate with a few extreme events even in that maximum. Thus, opportunities to forecast extreme space weather events have been limited in recent decades. With such background, May 2024 extreme space weather events occurred for a long year and are important and valuable to summarize forecasts and evaluations.\u003c/p\u003e \u003cp\u003eHereafter, section 2 describes the NICT\u0026rsquo;s space weather forecast targets and definitions. Section 3 summarizes the issued forecasts and corresponding observations in May 2024 and evaluates the overall accuracy of NICT\u0026rsquo;s multi‑category forecasts. Section 4 evaluates the performance of maximum‑level forecasts. Section 5 compares the forecast performance of NICT with other ISES RWCs. Section 6 explains the observational and modeling basis used for forecasts. Lastly, section 7 summarizes all evaluation results.\u003c/p\u003e"},{"header":"2. Forecast level definition","content":"\u003cp\u003eThis section explains space weather forecast targets and definition of each event levels. NICT provides forecasts for seven types of space weather targets: \u0026ldquo;solar flare\u0026rdquo;, \u0026ldquo;solar proton\u0026rdquo;, \u0026ldquo;geomagnetic disturbance\u0026rdquo;, \u0026ldquo;radiation belt electrons\u0026rdquo;, \u0026ldquo;ionospheric storm\u0026rdquo;, \u0026ldquo;Dellinger phenomena (short wave fadeout)\u0026rdquo;, and \u0026ldquo;sporadic E-layer\u0026rdquo;. Details on these seven targets are described on the website \u0026lt;\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://swc.nict.go.jp/en/knowledge/criteria_icon.html%3E\u003c/span\u003e\u003cspan address=\"https://swc.nict.go.jp/en/knowledge/criteria_icon.html%3E\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. NICT issues forecast three times a day at 00:00 UT (9:00 JST), 6:00 UT (15:00 JST), and 12:00 UT (21:00 JST), providing maximum levels for the next 24 hours after issuing timing for each space weather targets. This paper focuses on three space weather targets: solar flare, geomagnetic disturbance, and ionospheric storm. The forecast level definition for these three targets is shown in Table\u0026nbsp;1. Solar flare forecasts are classified into four levels based on X-ray observations (0.1\u0026ndash;0.8 nm) by Geostationary Operational Environmental Satellite (GOES): B-class or less, C-class, M-class, and X-class. Geomagnetic disturbance forecasts are classified into five levels based on K index at Kakioka, Japan: K\u0026thinsp;\u0026le;\u0026thinsp;3, K\u0026thinsp;=\u0026thinsp;4, K\u0026thinsp;=\u0026thinsp;5, K\u0026thinsp;=\u0026thinsp;6, and K\u0026thinsp;\u0026ge;\u0026thinsp;7. Classification of solar flare and geomagnetic disturbance forecasts are according to the definition of UGEOA code, which is one of URSIgram codes used by ISES/RWCs to share forecast data \u0026lt;\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.spaceweather.org/ISES/code/aaf/ugeoa.html%3E\u003c/span\u003e\u003cspan address=\"http://www.spaceweather.org/ISES/code/aaf/ugeoa.html%3E\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e. On the other hand, NICT provides unique ionospheric storm forecasts using our own scale. Condution of ionosphere over Japan are classified into three levels using the I-scale based on ionospheric F-region critical frequency, foF\u003csub\u003e2\u003c/sub\u003e observations by ionosondes at four site (Wakkanai, Kokubunji, Yamagawa, and Okinawa) in Japan and ionospheric total electron content (TEC) calculated from GNSS earth observation network System (GEONET) data of Geospatial Information Authority of Japan (GIS). The levels of ionospheric storm defined as the largest I-scales among the TEC-based I-scales at five latitudinal bands (45, 41, 37, 33, and 29\u0026deg;N) and the foF\u003csub\u003e2\u003c/sub\u003e-based I-scales at the four sites. The I-scale represents the relative level of deviations from 27-day median value, compared with its standard deviation at each latitude, local time, and season over 18 years: I\u003csub\u003eN\u003c/sub\u003e3 (-3σ or less ), I\u003csub\u003eN\u003c/sub\u003e2 (-3σ to -2σ), I\u003csub\u003eN\u003c/sub\u003e1(-2σ to -1σ), I0 (-1σ to +\u0026thinsp;1σ), I\u003csub\u003eP\u003c/sub\u003e1(+\u0026thinsp;1σ to +\u0026thinsp;3σ), I\u003csub\u003eP\u003c/sub\u003e2 (+\u0026thinsp;3σ to +\u0026thinsp;5σ), I\u003csub\u003eP\u003c/sub\u003e3 (+\u0026thinsp;5σ or more ). The denotation N and P represent positive and negative ionospheric storm, respectively. In this paper, I-scale level 0 is denoted as I-scale \u0026le;I1, level 1 as I-scale\u0026thinsp;=\u0026thinsp;I2, and level 2 as I-scale\u0026thinsp;=\u0026thinsp;I3. Details on the I-scale definition is described in Nishioka et al. (\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2017\u003c/span\u003e).\u003c/p\u003e"},{"header":"3. Forecasted levels and observation results","content":"\u003cp\u003eIn this paper, total 93 forecasts over one month (31 days) in May 2024 were evaluated. A list of forecasted levels and observation levels of solar flares, geomagnetic disturbances, and ionospheric storms is attached as an additional tabular data. Table\u0026nbsp;2 shows multi-categorical contingency tables of (a) solar flare, (b) geomagnetic disturbance, and (c) ionospheric storm forecasts. The labeled number of observation and forecast axes stand for the defined activity levels shown in Table\u0026nbsp;1. The diagonal components of multi-categorical contingency tables represent the counts of correct forecasts at each level. Elements to the left of the diagonal represent counts of over-forecast cases, whereas elements below the diagonal indicate counts of under-forecast cases.\u003c/p\u003e \u003cp\u003eThe number of correct solar flare forecast counts for C-class, M-class, and X-class were 2, 16, 31, respectively, so among 93 forecasts issued during May 2024 totally 49 forecasts were verified as correct. For geomagnetic disturbance forecast, the number of correct forecast counts for K\u0026thinsp;\u0026le;\u0026thinsp;3, K\u0026thinsp;=\u0026thinsp;4, K\u0026thinsp;\u0026ge;\u0026thinsp;7 were 44, 4, 6, respectively, and total 54 counts of correct forecasts were issued. For ionospheric storm forecast, the number of correct forecast counts for I-scale\u0026thinsp;\u0026le;\u0026thinsp;I1, I-scale\u0026thinsp;=\u0026thinsp;I2, I-scale\u0026thinsp;=\u0026thinsp;I3 were 49, 2, 9, respectively, and total 36 counts correct forecasts were issued.\u003c/p\u003e \u003cp\u003eObserved event counts by level in May 2024 were; solar flares C-class: 11, M-class: 16, X-class: 31; geomagnetic disturbances K\u0026thinsp;\u0026le;\u0026thinsp;3: 44, K\u0026thinsp;=\u0026thinsp;4 : 3, K\u0026thinsp;=\u0026thinsp;5: 2, K\u0026thinsp;=\u0026thinsp;6: 1, K\u0026thinsp;\u0026ge;\u0026thinsp;7: 6; ionospheric storms I-scale\u0026thinsp;=\u0026thinsp;I1: 49, I-scale\u0026thinsp;=\u0026thinsp;I2: 7, I-scale\u0026thinsp;=\u0026thinsp;I3: 2. It is noteworthy that in May 2024 all levels of solar flare, geomagnetic disturbance, and ionospheric storm events occurred without solar flare of \u0026le;\u0026thinsp;B class (level 0). The climatological relative frequencies (P\u003csub\u003ec\u003c/sub\u003e) of maximum-level event occurrences in May 2024 were, X-class solar flares: Pc\u0026thinsp;=\u0026thinsp;42/93\u0026thinsp;\u0026asymp;\u0026thinsp;0.45 (45%), geomagnetic disturbances with K\u0026thinsp;\u0026ge;\u0026thinsp;7: Pc\u0026thinsp;=\u0026thinsp;6/93\u0026thinsp;\u0026asymp;\u0026thinsp;0.06 (6%), and ionospheric storms with I-scale\u0026thinsp;=\u0026thinsp;I3: Pc\u0026thinsp;=\u0026thinsp;16/93\u0026thinsp;\u0026asymp;\u0026thinsp;0.17 (17%).\u003c/p\u003e \u003cp\u003eThe overall accuracy of NICT\u0026rsquo;s multi-category space weather forecasts in May 2024 can be evaluated using a verification index PC\u003csub\u003em\u003c/sub\u003e, which is the proportion correct extended to a multi-categorical contingency table (Jolliffe \u0026amp; Stephenson \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e2012\u003c/span\u003e). PC\u003csub\u003em\u003c/sub\u003e is equivalent to values that is counts of correct forecasts divided by total counts of forecasts issued. So, PC\u003csub\u003em\u003c/sub\u003e are derived as PC\u003csub\u003em\u003c/sub\u003e = (0\u0026thinsp;+\u0026thinsp;2+16\u0026thinsp;+\u0026thinsp;31)/93 ≒ 0.53 for solar flares forecast, PC\u003csub\u003em\u003c/sub\u003e = (44\u0026thinsp;+\u0026thinsp;4+0\u0026thinsp;+\u0026thinsp;0+6)/93 ≒ 0.58 for geomagnetic disturbances forecast, and PC\u003csub\u003em\u003c/sub\u003e = (49\u0026thinsp;+\u0026thinsp;2+9)/93 ≒ 0.65 for ionospheric storms. All three forecasts succussed to issue correct level forecasts for more than half interval.\u003c/p\u003e"},{"header":"4. Maximum-level forecast evaluation","content":"\u003cp\u003eThe goal of this paper is to evaluate forecasts of extreme space weather events. So, focusing on maximum level, forecasts and observations were classified into two categories \u0026ldquo;maximum level\u0026rdquo; and \u0026ldquo;others\u0026rdquo;, and evaluated their accuracy, discrimination, reliability, bias, and skill using two-categorical contingency tables shown in Table\u0026nbsp;3. Maximum level of solar flare forecast is X-class (level 3), that of geomagnetic disturbance forecast is K\u0026thinsp;\u0026ge;\u0026thinsp;7 (level 4), and that of ionospheric storm forecast is I-scale=I3 (level 2) as defined in Table\u0026nbsp;1. A 2\u0026times;2 matrix of contingency table consists of hit counts: a, false alarm counts: b, miss count: c, correct rejection count: d. (see Table\u0026nbsp;3d). Hit, false alarm, miss, correct rejection counts of X-class solar flare forecasts were 31, 18, 11 and 33, respectively (Table\u0026nbsp;3a). These of K\u0026thinsp;\u0026ge;\u0026thinsp;7 geomagnetic disturbance forecasts were 6, 1, 0, and 86, respectively (Table\u0026nbsp;3b). These of I-scale=I3 ionospheric storms forecasts were 9, 2, 7, and 75, respectively (Table\u0026nbsp;3c).\u003c/p\u003e \u003cp\u003eVerification indices of proportion correct (PC), probability of detection (POD), false alarm ratio (FAR), frequency bias (FB), and equitable threat score (ETS) can be derived from elements of a contingency table. PC represent accuracy by the ratio of the number of correct forecasts of both maximum level and others to the total number of samples and is defined by PC = (a\u0026thinsp;+\u0026thinsp;d) / (a\u0026thinsp;+\u0026thinsp;b + c\u0026thinsp;+\u0026thinsp;d). Higher PC values indicate higher forecast accuracy, and PC\u0026thinsp;=\u0026thinsp;1 is optimal. POD represent discrimination of maximum-level forecasts by the ratio of the number of hit counts to the number of total maximum-level observation counts and is defined by POD\u0026thinsp;=\u0026thinsp;a / (a\u0026thinsp;+\u0026thinsp;c). Higher POD values indicate a lower count of miss, and POD\u0026thinsp;=\u0026thinsp;1 is optimal. FAR represent reliability of maximum-level forecasts by the ratio of the number of false alarms to the total number of maximum-level forecasts and is defined by FAR\u0026thinsp;=\u0026thinsp;b / (a\u0026thinsp;+\u0026thinsp;b). Lower FAR values indicate a lower count of false alarms, then FAR\u0026thinsp;=\u0026thinsp;0 is optimal. FB represent bias by the ratio of the total number of maximum-level forecasts to the total number of maximum-level observations and is defined by FB = (a\u0026thinsp;+\u0026thinsp;b) / (a\u0026thinsp;+\u0026thinsp;c). FB greater than 1 indicates an over-forecast tendency and FB less than 1 indicate under-forecast tendency. Lastly, ETS represent skill focusing on the hits counts of maximum-level forecasts by the ratio of the number of hit counts to the total number of counts other than correct rejections with the removal of contribution from hits by chance in random forecasts, and is defined by ETS = ( a \u0026ndash; a\u003csub\u003er\u003c/sub\u003e ) / ( a\u0026thinsp;+\u0026thinsp;b + c \u0026ndash; a\u003csub\u003er\u003c/sub\u003e) where a\u003csub\u003er\u003c/sub\u003e = P\u003csub\u003ec\u003c/sub\u003e( a\u0026thinsp;+\u0026thinsp;b ) is the number of hit forecasts by chance (random hits). Proximity to ETS\u0026thinsp;=\u0026thinsp;1 indicates higher forecast accuracy, while ETS\u0026thinsp;=\u0026thinsp;0 for random forecasts. Details of these verification indices is described in Jolliffe \u0026amp; Stephenson (\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e2012\u003c/span\u003e) and guideline of WMO-No.1364 (2025).\u003c/p\u003e \u003cp\u003eVerification indices of X-class solar flare forecasts were calculated: PC\u0026thinsp;=\u0026thinsp;0.69, POD\u0026thinsp;=\u0026thinsp;0.74, FAR\u0026thinsp;=\u0026thinsp;0.37, FB\u0026thinsp;=\u0026thinsp;1.17 and ETS\u0026thinsp;=\u0026thinsp;0.23, summarized in Table\u0026nbsp;4. These values indicate reasonable discrimination with a moderate false-alarm ratio and a slight over-forecasting tendency for X-class events. The most misses and false alarms were due to failures in forecasting the start and end of X-class flare activity, respectively, suggesting that improvement of timing of prediction remains a key challenge.\u003c/p\u003e \u003cp\u003eNext, for forecasts of geomagnetic disturbance with K\u0026thinsp;\u0026ge;\u0026thinsp;7 which is the maximum level in the forecast definition, verification indices of K\u0026thinsp;\u0026ge;\u0026thinsp;7 forecast were calculated: PC\u0026thinsp;=\u0026thinsp;0.99, POD\u0026thinsp;=\u0026thinsp;1.00, FAR\u0026thinsp;=\u0026thinsp;0.14, FB\u0026thinsp;=\u0026thinsp;1.17, and ETS\u0026thinsp;=\u0026thinsp;0.85. These results indicate that NICT achieved perfect discrimination capability for extreme geomagnetic disturbances, with very high overall accuracy and only one false alarm at termination timing. Thus, the frequency bias slightly exceeded 1, suggesting a minor over-forecasting tendency.\u003c/p\u003e \u003cp\u003eLastly, focusing on I-scale=I3 ionospheric storms forecasts, verification indices were calculated: PC\u0026thinsp;=\u0026thinsp;0.90, POD\u0026thinsp;=\u0026thinsp;0.56, FAR\u0026thinsp;=\u0026thinsp;0.18, FB\u0026thinsp;=\u0026thinsp;0.69, and ETS\u0026thinsp;=\u0026thinsp;0.44. These results indicate that while overall accuracy was high, the discrimination capability for extreme ionospheric storms was moderate, and the forecasts tended to under-forecast tendency. There were cases being missed because the cause of I-scale=I3 occurrence was unknown, resulting in moderate POD and under-forecast tendency. To enhance the forecast accuracy of ionospheric storms, further investigations into their underlying mechanisms and strengthening observations are required.\u003c/p\u003e"},{"header":"5. Comparison with ISES RWC forecasts","content":"\u003cp\u003eISES RWCs provides space weather information to users in their regions and exchange data and forecasts among ISES members. This section provides a comparative analysis of space weather forecast evaluations across RWCs. Although forecast level definitions vary by country, five RWCs including NICT (Japan) issue solar flare and geomagnetic disturbance forecasts in accordance with UGEOA standards, which is one of URSIgram codes used by ISES/RWCs to share forecast data \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.spaceweather.org/ISES/code/aaf/ugeoa.html\u003c/span\u003e\u003cspan address=\"http://www.spaceweather.org/ISES/code/aaf/ugeoa.html\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e, enabling direct comparison of forecast evaluations. The five countries are Australia (00:00 UT), USA (03:30 UT), Indonesia (08:00 UT), Belgium (12:30 UT), and Japan (06:00 UT). The times in parentheses indicate timing of forecast issued from each RWC. Although NICT issues forecasts three times a day, only the 06:00 UT forecasts are shared through UGEOA. So, the comparison of space weather forecast with other RWC was made by recalculating the verification index using only the 06 UT forecasts in order to compare with same basis (once a day forecast). Forecast issuing agencies in each country are as follow: Australia is the Australian Space Weather Forecasting Centre (ASWFC) of Bureau of Meteorology, USA is the Space Weather Prediction Center (SWPC) of NOAA, Indonesia is the Space Weather Information and Forecast Services (SWIFtS) of National Research and Innovation Agency, and Belgium is the Solar Influences Data Analysis Center (SIDC) of Royal Observatory of Belgium.\u003c/p\u003e \u003cp\u003eTable\u0026nbsp;5 shows contingency tables of five RWC\u0026rsquo;s solar flare and geomagnetic disturbance forecasts. As same with Table\u0026nbsp;3 focusing on extreme events, forecasts were classified into two categories maximum (X class or K\u0026thinsp;\u0026ge;\u0026thinsp;7) and others. Because Indonesia and the USA did not issue forecasts infrequently, the sample count used for evaluation was smaller than that of other RWCs.\u003c/p\u003e \u003cp\u003eAll five RWCs base their solar flare forecast on same GOES X-ray observations. However, due to variation in forecast issuance times among RWCs, the maximum solar flare level considered and observed within each forecast interval could be differ. Thus, the sample counts of intervals containing X-class solar flare observations varies across RWCs. The counts of (hit, false alarm, miss, and correct rejection) of each RWC X-class solar flare forecasts were as follows, RWC Australia:(8, 6, 5, 12), RWC USA: (9, 0, 4, 16), RWC Japan: (11, 5, 4, 11), RWC Indonesia: (10, 4, 4, 11), and RWC Belgium: (3, 0, 11, 7). RWC Japan and Indonesia have larger hit counts but also have some false alarm and miss counts. On the other hand, RWC USA and Belgium provided no false alarm for X class solar flare occurrence in this period.\u003c/p\u003e \u003cp\u003eFor geomagnetic disturbance forecast, each RWC targets each local K index in their region. So, targeting observation as a measure of geomagnetic disturbance differ depending on RWCs. RWC Australia targes local K index calculated by ASWFC based on multi geomagnetic observations in the Australian region. RWC USA targes local K index at Boulder Magnetic Observatory. RWC Japan targets local K index at Kakioka Magnetic Observatory. RWC Belgium targes local K-index at Dourbes Geomagnetic Observatory. Only RWC Indonesia targes planetary K index (K\u003csub\u003ep\u003c/sub\u003e), which is a global geomagnetic activity index. Therefore, we evaluated each RWC\u0026rsquo;s forecast for each targeting measure except for Belgium, of which targeting local K-index observation was missing and local K-index at Chambon-la-For\u0026ecirc;t magnetic observatory locating near Dourbes was used alternatively as same as Devos et al. (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2014\u003c/span\u003e). The counts of (hit, false alarm, miss, and correct rejection) of each RWC geomagnetic disturbance forecasts with K\u0026thinsp;\u0026ge;\u0026thinsp;7 are as follows, RWC Australia:(2, 0, 0, 29), RWC USA: (2, 0, 1, 26), RWC Japan: (2, 0, 0, 29), RWC Indonesia: (0, 1, 2, 25), and RWC Belgium: (1, 2, 1, 27). Surprisingly, K\u0026thinsp;\u0026ge;\u0026thinsp;7 forecasts of RWC Australia and Japan succussed to provide no false alarm or miss.\u003c/p\u003e \u003cp\u003eFour verification indices POD, FAR, FB, and ETS of X-class solar flare and K\u0026thinsp;\u0026ge;\u0026thinsp;7 geomagnetic disturbance forecasts in May 2024 by five RWCs including Japan were calculated using these contingency tables (Table\u0026nbsp;5) and are summarized in Table\u0026nbsp;6. For X-class solar flare forecasts, the POD varied significantly among RWCs, with the highest discrimination capability achieved by RWC Japan (POD\u0026thinsp;=\u0026thinsp;0.73), followed closely by RWC Indonesia (POD\u0026thinsp;=\u0026thinsp;0.71) and RWC USA (POD\u0026thinsp;=\u0026thinsp;0.69). RWC Australia showed moderate discrimination (POD\u0026thinsp;=\u0026thinsp;0.62), while RWC Belgium exhibited the lowest (POD\u0026thinsp;=\u0026thinsp;0.21). In terms of reliability, optimal (FAR\u0026thinsp;=\u0026thinsp;0.00) was achieved by RWC USA and Belgium, whereas RWC Australia (FAR\u0026thinsp;=\u0026thinsp;0.43), Japan (FAR\u0026thinsp;=\u0026thinsp;0.31), and Indonesia (FAR\u0026thinsp;=\u0026thinsp;0.29) have provided some false alarms indicating they have subject to reliability. The bias was close to unity for most RWCs, suggesting balanced forecasts, except for Belgium (FB\u0026thinsp;=\u0026thinsp;0.21), which strongly under-forecast tendency for X-class events. Regarding skill, the ETS ranged from 0.55 by RWC USA to 0.13 by Belgium, with Japan (ETS\u0026thinsp;=\u0026thinsp;0.27) and Indonesia (ETS\u0026thinsp;=\u0026thinsp;0.29) showing moderate skill and Australia slightly lower (ETS\u0026thinsp;=\u0026thinsp;0.16). Because ETS is calculated by excluding random hits that depend on the climatological relative frequency, forecasts with high POD but also having many false alarms, such as those issued by Japan and Indonesia, were evaluated less favorably despite their strong discrimination capability.\u003c/p\u003e \u003cp\u003eFor geomagnetic disturbance forecasts of K\u0026thinsp;\u0026ge;\u0026thinsp;7, RWC Japan and Australia achieved perfect discrimination (POD\u0026thinsp;=\u0026thinsp;1.00), while POD of USA showed high but not perfect (POD\u0026thinsp;=\u0026thinsp;0.67). Belgium demonstrated moderate capability (POD\u0026thinsp;=\u0026thinsp;0.50), and Indonesia failed to detect any K\u0026thinsp;\u0026ge;\u0026thinsp;7 events (POD\u0026thinsp;=\u0026thinsp;0.00). Reliability of RWC Australia, USA, and Japan\u0026rsquo;s forecast was optimal (FAR\u0026thinsp;=\u0026thinsp;0.00), whereas RWC Belgium and Indonesia was higher values (FAR\u0026thinsp;=\u0026thinsp;0.67 and 1.00, respectively). The bias was ideal for RWC Australia and Japan (FB\u0026thinsp;=\u0026thinsp;1.00), slightly below unity for RWC USA (FB\u0026thinsp;=\u0026thinsp;0.67), and significantly above unity for RWC Belgium (FB\u0026thinsp;=\u0026thinsp;1.50) indicating an over-forecasting tendency. RWC Indonesia showed FB\u0026thinsp;=\u0026thinsp;0.50, reflecting strong under-forecasting. In terms of skill, ETS was highest for RWC Australia and Japan (ETS\u0026thinsp;=\u0026thinsp;1.00), followed by USA (ETS\u0026thinsp;=\u0026thinsp;0.64) and Belgium (ETS\u0026thinsp;=\u0026thinsp;0.21), while Indonesia showed negative skill (ETS = \u0026minus;\u0026thinsp;0.02), indicating performance worse than random chance.\u003c/p\u003e \u003cp\u003eOverall, for X-class solar flare forecasts there is no RWCs that have achieved best performance for all four verification indices, but each RWC has own strength and tendency. For example, RWC Japan\u0026rsquo;s forecast was the highest discrimination capacity, RWC Belgium\u0026rsquo;s forecast was characterized by best reliability, RWC Indonesia\u0026rsquo;s forecast was most balanced neither under- nor over-forecast tendency, and RWC USA\u0026rsquo;s forecast was characterized by best reliability as well as the highest skill score. On the other hand, for K\u0026thinsp;\u0026ge;\u0026thinsp;7 geomagnetic disturbance, RWC Australia and Japan\u0026rsquo;s forecast achieved optimal values for all four verification indices. Other RWC\u0026rsquo;s forecast failed forecast the beginning and termination timings of geomagnetic disturbance occurrence and resulted in verification indices were not optimal.\u003c/p\u003e"},{"header":"6. Forecast basis","content":"\u003cp\u003eThis section describes space weather conditions before the extreme event occurrences and basis for determining forecast levels of solar flares in section 6.1, geomagnetic disturbances in section 6.2, and ionospheric storms in section 6.3, respectively.\u003c/p\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003e6.1. Solar flare forecasts\u003c/h2\u003e \u003cp\u003eIn May 2024, total 21 events X-class solar flares were observed in a month. Notably, the occurrence of seven X-class flares within 72 hours was a first in the history of X-ray observations by GOES since 1975. The largest solar flare in this series was the X8.7 flare on May 14. Here, we reconsider two forecasts issued at 00:00 UT and 6:00 UT on May 14, prior to the occurrence of X8.7 solar flare which reached its peak intensity at 16:51 UT on the day.\u003c/p\u003e \u003cp\u003eAt 00 UT on May 14, it is approximately 17 hours before the occurrence of X8.7 solar flare, RWC Japan was not anticipated the occurrence of X-class solar flares within the next 24 hours. Figure\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e shows parts of imaging data of (a) intensitygram and (b) magnetogram taken by Helioseismic and Magnetic Imager (HMI) (Scherrer et al. \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e2012\u003c/span\u003e; Schou et al. \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e2012\u003c/span\u003e) on the Solar Dynamic Observatory (SDO) satellite (Pesnell et al. \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e2012\u003c/span\u003e), which were analyzed at forecast briefing before 00 UT on May 14. The 13 regions are indicated by the red circles and with NOAA\u0026rsquo;s active region numbers. Figure\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e (c) shows soft X-ray light curve by GOES-16. Occurrence timings of X-class solar flares are indicated by white arrows and most of these X-class flares were produced by active region 13664. At this time, based on histories of solar flare occurrences, and sizes of sunspot area (1170), its type (\u003cem\u003eFkc\u003c/em\u003e) and magnetic field structure (\u003cem\u003eβγδ\u003c/em\u003e) according to the solar region summary published by NOAA/SWPC, NICT\u0026rsquo;s space weather forecaster had designated only the active region 13664 have capability of X-class solar flare occurrences. Details on the sunspot type and magnetic field structure is described in McIntosh (\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e1990\u003c/span\u003e). However, it was quite hard to assess the detailed structure, because the active region 13664 was located at the western limb of the solar surface (S19/W87) corresponding to the edge of image where apparent structure distorted. Looking carefully at data of HMI/SDO, there was no noticeable changes of magnetic structure were visible in sequences from previous day. In addition, X-ray observations by GOES showed that peak intensities of solar flares had been decreasing after the X5.8 flare on 11 May, and it took about 31 hours since the last X1.2 flare was observed. Based on these circumstances and the fact that the largest solar flare in the past 24 hours was M6.6 (peaked at 9:44 UT on May 13), forecasters suspected that activity of active region 13644 had weakened, and level of solar flares over next 24 hours would be likely to remain at M-class (this forecast failed).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe forecast at 6:00 UT, approximately 11 hours before the X8.7 flare occurrence, forecasters had updated the maximum solar flare level over the next 24 hours to X-class. The reason for updating the forecast to X-class was that an X1.7 flare occurred in active region 13664 at 02:09 UT, approximately two hours after the incorrect forecast was issued at 00 UT. At 06 UT, the active region had already reached at 90 degrees west on the solar surface, it is impossible to observe its most of the area. However, the occurrence of the X1.7 flare let us know that the region had been still active, and forecasters could obtain bases to update the forecast level from M- to X-class. After seven hours since the forecast updated an X1.2 flare occurred, and 11 hours later an X8.7 flare occurred.\u003c/p\u003e \u003cp\u003eMeanwhile, regarding the X5.8 flare that occurred in active region 13664 on 11 May, which is the largest solar flare observed on the solar disk in May 2024, has been suggested that the occurrence of an X-class flare may have been anticipated in advance. This suggestion is based on a detailed analysis of the magnetic field configuration derived from a physics-based model of the active region, together with observations of precursor brightening phenomena (Bamba et al., under review). Therefore, precise full-disk observational data is crucial for solar flare forecasting. However, near the solar limb, significant distortion in observed images makes accurate assessment of their structure impossible. Furthermore, at that time, unfortunately the STEREO-A spacecraft, which is another near-real-time solar imaging spacecraft, was orbiting at roughly the same longitude as Earth, meaning there was no way to observe the active region 13664 from angles different from those viewing from Earth. To improve the performance of solar flare forecasting, multi-angle solar observations constitute a critically important approach, such as new mission viewing sun from the Lagrange-5 point.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003e6.2. Geomagnetic disturbance forecast\u003c/h2\u003e \u003cp\u003eAccording to the Japan Meteorological Agency's Kakioka Geomagnetic Observatory, geomagnetic storm started with an occurrence of a sudden commencement at 17:05 UT on May 10 and it ended at approximately 4:00 UT on May 14. Figure\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e shows time series data of Kakioka K index including the period of the geomagnetic storm. During this period, local K\u0026thinsp;=\u0026thinsp;8 was observed four times (total 12 hours) and K\u0026thinsp;=\u0026thinsp;7 was observed three times (total 9 hours). It has been almost 19 years since local K\u0026thinsp;=\u0026thinsp;8 was lastly observed at Kakioka in August 2005.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eHere, we look back geomagnetic disturbance forecast before the geomagnetic storm occurrence. Figure\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e shows time series data plots of real-time solar wind observations at Lagrange 1 (L1) points. At 00 UT on May 10, it is 15 hours before the occurrence of K\u0026thinsp;\u0026ge;\u0026thinsp;7 geomagnetic disturbances, NICT\u0026rsquo;s geomagnetic disturbance forecast for the next 24 hours was updated from a previous forecast of K\u0026thinsp;\u0026le;\u0026thinsp;3 (level 0) to K\u0026thinsp;\u0026ge;\u0026thinsp;7 (maximum level). This was optimal timing. At that time, solar wind velocity over the past 24 hours was approximately 430 km/s, density was 2 to 8 cm\u003csup\u003e\u0026minus;\u0026thinsp;3\u003c/sup\u003e, and interplanetary magnetic field intensity was approximately 5 nT, indicating nominal state as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e. Consequently, geomagnetic activity was quiet with K\u0026thinsp;\u0026le;\u0026thinsp;3 as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e. There were no coronal holes visible around the equator of the solar surface, and arrival of high-speed winds from coronal holes was not expected. Meanwhile, multiple CMEs had been observed since a few days ago. At the moment of 00 UT on May 10, seven full-halo CMEs were identified by the large angle and spectrometric coronagraph experiment on solar and heliospheric observatory (LASCO/SOHO) and the coronagraph on solar terrestrial relation observatory (COR/STEREO). Those had erupted from around active region 13644 on May 8 to 9.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe arrival dates and times of these CMEs were predicted using solar storm forecast system SUSANOO-CME (Shiota and Kataoka, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e2016\u003c/span\u003e; Shiota and Yashiro, 2021). Figure\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ea shows parameters of seven CMEs that concerned to arrive at the Earth\u0026rsquo;s magnetosphere. The table lists the occurrence time of CME eruption, the heliographic latitude and longitude of estimated source region, associated active region number and flare class, and initial velocity of CMEs. All these seven CME were included to SUSANOO-CME simulation. Figure\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eb shows timeseries plots of simulated solar wind speed, density, and magnetic field intensity, azimuth angle, and north-south component. These simulated parameters are overplotted with real-time observation data at L1 point. Figure\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003ec shows the simulated solar wind speed distribution on the XY plane in Heliocentric Earth Ecliptic coordinate system at the timing of blue line in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003eb with colors indicating difference value from the background solar wind speed. SUSANOO-CME simulation predicted that the first two CMEs, which erupted at 05 UT and 12 UT on May 8, would merge during their propagations and arrive at the L1 point at noon on May 10 as a shock wave. The simulation also predicted after the arrival of CMEs solar wind speed would increase approximately 700 km/s, magnetic field intensity would increase roughly up to 15\u0026ndash;20 nT and its vector would initially direct southward. Based on these solar wind simulation results, geomagnetic disturbance level as is K index was determined to being K\u0026thinsp;\u0026ge;\u0026thinsp;7 using look-up table based on past statistics. In reality, a first shock wave of multiple CMEs arrived at the L1 point at approximately 16:30 UT on May 10. It is about only 4-hour difference with SUSANOO-CME prediction. With the arrival of the CME shock wave, solar wind speed suddenly increased to 770 km/s, interplanetary magnetic field intensity increased to 72 nT, and north-south component of magnetic field reached\u0026thinsp;\u0026minus;\u0026thinsp;50 nT, as shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e. This is the best practice of correct forecast could be made based on solar wind simulation with appropriate input parameters of CMEs. Post-analysis of these CMEs using SUSANOO-CME will be described in a separate paper (Shiota et al. in preparation).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003e6.3. Ionospheric storm forecast\u003c/h2\u003e \u003cp\u003eOccurrence of a severe negative ionospheric storm (I-scale=I\u003csub\u003eN\u003c/sub\u003e3) was confirmed after 18 UT on the same day following the first shock wave of multiple CMEs arrived around 16:30 U, and a geomagnetic storm soon began at 17 UT on May 10. Figure\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e shows timeseries plots of (a) foF\u003csub\u003e2\u003c/sub\u003e measurements observed by four ionosondes at Wakkanai, Kokubunji, Yamagawa, and Okinawa in Japan and (b) GEONET TEC from latitude 29\u0026deg;N to 45\u0026deg;N along Japan longitudes. The foF\u003csub\u003e2\u003c/sub\u003e timeseries plot at Wakkanai indicates a severe negative ionospheric storm with I-scale=I\u003csub\u003eN\u003c/sub\u003e3 that occurred at 18 UT and continued until 9 UT on May 11 (from 03:00 to18:00 Japanese local time (JST)). Although it temporarily returned to \u0026le;I1 levels during the night and temporarily positive storm occurred, a severe negative storm with I-scale=I\u003csub\u003eN\u003c/sub\u003e3 started again around 18:00 UT on May 11 (03:00 JST), and continued for next 24 hours at Wakkanai. During this period, a severe negative storm with I-Scale=I\u003csub\u003eN\u003c/sub\u003e3 was simultaneously observed at Okinawa during the daytime (6\u0026ndash;15 JST). Furthermore, GEONET TEC plots show that a negative storm with I-scale=I\u003csub\u003eN\u003c/sub\u003e2 was observed between 45\u0026deg;N and 29\u0026deg;N latitudes from 0:00 UT to 18:00 UT on May 11.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eHere, we reconsider ionospheric storm forecasts at 00 UT and 06 UT on May 10 before the occurrence of severe ionospheric storms. The forecast at 00 UT which was 18 hours before the onset of a severe negative storm, geomagnetic disturbance with K\u0026thinsp;\u0026ge;\u0026thinsp;7 was forecasted within the next 24 hours as described in section 9.2. Although K\u0026thinsp;\u0026ge;\u0026thinsp;7 was expected, occurrence frequency of K\u0026thinsp;\u0026ge;\u0026thinsp;7 was extremely low and available past data were limited only eleven days since 1997. In the forecast at 00:00 UT, a rare extreme event of K\u0026thinsp;\u0026ge;\u0026thinsp;7 geomagnetic disturbance was predicted, but due to the lack of experience and knowledge of forecasters deciding on the ionospheric storm forecast level at such situation, a moderate level of I-scale\u0026thinsp;=\u0026thinsp;2 was forecasted without careful consideration. However, for the next forecast at 06:00 UT which was 12 hours before the onset of I\u003csub\u003eN\u003c/sub\u003e3 ionospheric storm, last forecast was reviewed. According to past records, it was found that among 11 cases when the daily maximum K index was \u0026ge;\u0026thinsp;7, six cases were I3, three cases were I2, and two cases were \u0026le;I1. Based on this, the forecast level was revised to I3. This change resulted in a correct revision to the forecast, but since past I-scale=I3 occurrence frequency (6/11) is not enough high and the number of sample data is insufficient, so there is a possibility that the forecast may be incorrect. Further data analysis and physics-based simulation are needed to improve performance of ionospheric storm forecasts when K\u0026thinsp;\u0026ge;\u0026thinsp;7 is expected.\u003c/p\u003e \u003c/div\u003e"},{"header":"7. Summary","content":"\u003cp\u003eNICT provides space weather information as Regional Warning Center (RWC) Japan of International Space Environment Service (ISES). This study evaluated NICT\u0026rsquo;s space weather forecasts during May 2024, which is a period characterized by multiple extreme events including X-class solar flares, severe geomagnetic disturbances (K\u0026thinsp;\u0026ge;\u0026thinsp;7), and ionospheric storms (I-scale\u0026thinsp;=\u0026thinsp;I3). The results obtained in this study are briefly summarized as follows. Multi-category forecast accuracy (PC\u003csub\u003em\u003c/sub\u003e) of NICT\u0026rsquo;s forecasts was 0.53 for solar flares, 0.58 for geomagnetic disturbances, and 0.65 for ionospheric storms, indicating moderate overall performance. For extreme space weather events in May 2024, NICT archived almost perfect performance for K\u0026thinsp;\u0026ge;\u0026thinsp;7 geomagnetic disturbance forecasts (PC\u0026thinsp;=\u0026thinsp;0.99, POD\u0026thinsp;=\u0026thinsp;1.00, FAR\u0026thinsp;=\u0026thinsp;0.14, FB\u0026thinsp;=\u0026thinsp;1.17, ETS\u0026thinsp;=\u0026thinsp;0.85), while X-class solar flares forecast was characterized by high discrimination (POD\u0026thinsp;=\u0026thinsp;0.74) but subject to reliability (FAR\u0026thinsp;=\u0026thinsp;0.37) and ionospheric storms forecast for I-scale=I3 is high accuracy (PC\u0026thinsp;=\u0026thinsp;0.90) but subject to discrimination ability (POD\u0026thinsp;=\u0026thinsp;0.56). Comparative analysis across five ISES/RWCs revealed that RWC USA demonstrated highest skill for solar flare forecasts (ETS\u0026thinsp;=\u0026thinsp;0.55) with zero false alarms. RWC Japan and Australia achieved perfect skill (ETS\u0026thinsp;=\u0026thinsp;1.00) for K\u0026thinsp;\u0026ge;\u0026thinsp;7 geomagnetic disturbance forecasts. NICT succussed to forecast the start and termination of geomagnetic disturbances of K\u0026thinsp;\u0026ge;\u0026thinsp;7 using the solar storm forecast system SUSANOO-CME.\u003c/p\u003e \u003cp\u003eExtreme space weather events can affect modern social infrastructure particularly communications, positioning, power sector, and any space-based systems, and these failures may cause space weather disasters in our society. As with weather forecast on ground, disasters can be mitigated by utilizing forecasts. We hope that with understanding current ability of space weather forecast still having possibility of miss or false forecasts as evaluated by this study, appropriately space weather forecasts are included in counter measures to mitigate disasters. Also, we hope that new methods which can predict extreme space weather events with a higher score than current space weather forecasts given in this paper will be developed in future.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study does not contain human participants, human data or human tissue.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study does not contain any individual person\u0026rsquo;s data.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eData set related on this study are attached as an additional supporting file.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNo competing interest exists.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study was supported by Ministry of Internal affairs and Communications, Japan.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026apos; contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll authors contributed to space weather forecast service operation.\u003c/p\u003e\n\u003cp\u003eAuthor#1 contributed to evolution analysis.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe thank for RWC Australia (ASWFC), RWC USA (NOAA/SWPC), RWC Indonesia (SWIFtS), and RWC Belgium (SIDC) to share space weather forecast data. We thank for SDO/HMI, GOES/X-ray, Kakioka Geomagnetic Observatory, and ACE/DSCOVR teams for providing their real-time solar wind observation data. A part of these results was obtained from \u0026ldquo;Promotion of observation and analysis of radio wave propagation\u0026rdquo;, commissioned by the Ministry of Internal Affairs and Communications, Japan.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026apos; information\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eN/A\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEndnotes\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eN/A\u003c/p\u003e"},{"header":"References","content":"\n\u003col\u003e\n\u003cli\u003eY. Bamba, D. Shiota, and K. Kusano, Characteristics of Magnetic Field Evolution and Onset Process of Successive X-class Flares in May 2024, under review, EPS special issue, https://assets-eu.researchsquare.com/files/rs-7986926/v1_covered_895b8347-84a6-43e9-846e-c228f363925a.pdf\u003c/li\u003e\n\u003cli\u003eCrown, MD (2012), Validation of the NOAA Space Weather Prediction Center\u0026apos;s solar flare forecasting look-up table and forecaster-issued probabilities, Space Weather, 10, S06006, doi:10.1029/2011SW000760.\u003c/li\u003e\n\u003cli\u003eDevos A, Verbeeck C \u0026amp; Robbrecht E (2014), Verification of space weather forecasting at the Regional Warning Center in Belgium. J. Space Weather Space Clim., 4, A29\u003c/li\u003e\n\u003cli\u003eGuidelines on the Verification of Hydrological Forecasts (2025), WMO-No. 1364, World Meteorological Organization (WMO), ISBN 978-92-63-11364-0, https://library.wmo.int/idurl/4/69478\u003c/li\u003e\n\u003cli\u003eJolliffe, I.T. and Stephenson, D.B. (2012) Forecast Verification: A Practitioner\u0026rsquo;s Guide in Atmospheric Science. 2nd Edition, Wiley-Blackwell, Oxford.\u003c/li\u003e\n\u003cli\u003eKubo Y, Den M \u0026amp; Ishii M. Verification of operational solar flare forecast: Case of Regional Warning Center Japan, J. Space Weather Space Clim., 7, A20, 2017, DOI: 10.1051/swsc /2017018.\u003c/li\u003e\n\u003cli\u003eMcIntosh, PS (1990), The classification of sunspot groups, Sol. Phys., 125(2), 251\u0026ndash;267, doi:10.1007/BF00158405.\u003c/li\u003e\n\u003cli\u003eMurphy, AH, and RL Winkler (1987), A general framework for forecast verification, Mon. Wea. Rev., 115, 1330.\u003c/li\u003e\n\u003cli\u003eNagatsuma, T., K. Ooyama, A. Okano, and M. Akioka (1997), SPACE ENVIRONMENT FORCAST SERVICE,Review of the Communications Research Laboratory, Vol.43, No.2, p301-308.\u003c/li\u003e\n\u003cli\u003eNishioka, M., T. Tsugawa, H. Jin, and M. Ishii (2017), A new ionospheric storm scale based on TEC and foF2 statistics, Space Weather, 15, 228-239, doi:10.1002/2016SW001536\u003c/li\u003e\n\u003cli\u003ePesnell, W., Thompson, B. J., \u0026amp; Chamberlin, P. C. (2012), The \u003cem\u003eSolar Dynamics Observatory \u003c/em\u003e(SDO), Solar Physics, 275, 3, doi:10.1007/s11207-011-9841-3.\u003c/li\u003e\n\u003cli\u003eScherrer, P. H., Schou, J., Bush, R. I., et al. (2012), The Helioseismic and Magnetic Imager (HMI) Investigation for the Solar Dynamics Observatory (SDO), Solar Physics, 275, 207, doi:10.1007/s11207-011-9834-2.\u003c/li\u003e\n\u003cli\u003eSchou, J., Scherrer, P. H., Bush, R. I., et al. (2012), Design and Ground Calibration of the Helioseismic and Magnetic Imager (HMI) Instrument on the Solar Dynamics Observatory (SDO), Solar Physics, 275, 229, doi:10.1007/s11207-011-9842-2.\u003c/li\u003e\n\u003cli\u003eShiota, D., and R. Kataoka (2016), Magnetohydrodynamic simulation of interplanetary propagation of multiple coronal mass ejections with internal magnetic flux rope (SUSANOO-CME), Space Weather, 14, 56\u0026ndash;75, doi:10.1002/2015SW001308.\u003c/li\u003e\n\u003cli\u003eShiota, D., and Yashiro (2021), Real-time Prediction System for Solar Storm Arrival, Journal of NICT, ISSN 2433-6009, Vol.67, No.1, pp137-142.\u003c/li\u003e\n\u003cli\u003eShiota, D., K. Sakaguchi, and K. Fukunaga (2021), Digitalization of Historical Space Weather Records, Journal of NICT, ISSN 2433-6009, Vol.67, No.1, pp203-210.\u003c/li\u003e\n\u003cli\u003eSmith, SF, and R. Howard (1968), Magnetic classification of active regions, in Structure and Development of Solar Active Regions, edited by KO Kiepenheuer, pp. 33\u0026ndash;42, D. Reidel, Dordrecht, Netherlands.\u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Tables","content":"\u003cp\u003eTables 1 to 6 are available in the Supplementary Files section.\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"earth-planets-and-space","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"epsp","sideBox":"Learn more about [Earth, Planets and Space](http://earth-planets-space.springeropen.com)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/epsp/default.aspx","title":"Earth, Planets and Space","twitterHandle":"@SpringerOpen","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Space weather forecast evaluation, Solar flare, Geomagnetic disturbance, Ionospheric Storm, International Space Weather Service (ISES)","lastPublishedDoi":"10.21203/rs.3.rs-9090861/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9090861/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eIn May 2024, multiple X-class solar flares, full-halo coronal mass ejections (CMEs), severe geomagnetic disturbances, and severe ionospheric negative storms were observed, and these space weather events caused some social impacts. In this study, we evaluate performance of space weather forecasts against these extreme space weather events, focusing on maximum forecast levels of X-class solar flares, geomagnetic disturbances with K ≥ 7, and ionospheric storms with I-scale = I3. Firstly, NICT’s forecasts were evaluated using multi verification indices such as proportion correct (accuracy), probability of detection (discrimination), false alarm ratio (reliability), frequency bias (bias), and equitable threat score (skill). As a result. It was found that NICT’s forecasts were characterized as having strong discrimination ability for X-class solar flares with moderate reliability, and high accuracy for ionospheric storms but subject to discrimination ability. While nearly perfect performance for K ≥ 7 geomagnetic disturbances forecast were achieved this time by using solar wind simulation (SUSANOO-CME) incorporating multiple earth-directing CMEs. Comparative analysis with forecast of other countries indicated that RWC Japan was the highest discrimination capability for X-class solar flares, while RWC USA archived highest skill for solar flare forecasts with zero false alarms. Perfect skill for K ≥ 7 geomagnetic disturbances were archived by RWC Japan, together with RWC Australia. Although continued efforts to improve forecast performance are still significant subject, we anticipate that the evaluation presented here will support appropriate implementation of effective measures to mitigate space weather disaster risks to modern social infrastructure.\u003c/p\u003e","manuscriptTitle":"Evaluation of NICT space weather forecast for extreme events in May 2024","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-03-19 16:43:28","doi":"10.21203/rs.3.rs-9090861/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Major Revision","date":"2026-04-22T22:02:12+00:00","index":"","fulltext":""},{"type":"reviewerAgreed","content":"","date":"2026-03-19T00:26:22+00:00","index":0,"fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-03-17T12:30:33+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-03-12T02:43:29+00:00","index":"","fulltext":""},{"type":"submitted","content":"Earth, Planets and Space","date":"2026-03-11T02:48:11+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"earth-planets-and-space","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"epsp","sideBox":"Learn more about [Earth, Planets and Space](http://earth-planets-space.springeropen.com)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/epsp/default.aspx","title":"Earth, Planets and Space","twitterHandle":"@SpringerOpen","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2d8a823e-ab86-4183-b5db-aa1f4784a639","owner":[],"postedDate":"March 19th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"in-revision","subjectAreas":[],"tags":[],"updatedAt":"2026-04-23T02:02:51+00:00","versionOfRecord":[],"versionCreatedAt":"2026-03-19 16:43:28","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9090861","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9090861","identity":"rs-9090861","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}