Post processing of Numerical Forecast

Scientific background

Model outputs are not always suited for end users. It is because sometimes they would need forecast of another parameter, or in another frame of reference, sometimes the results have systematic errors, or the resolution of the model outputs does not meet the requirements. These problems can be solved by the post processing methods.

Statistical techniques have been developed to predict weather elements at particular point locations by using direct and post-processed model fields, and climatology. The data found from this information is then used to construct equations relating the variable (predictand) to other variables (predictors), both kinds of variables from the past, then the equations are used to calculate the predictands for the future.

There are three different types of statistical weather forecast products listed below

The Perfect-Prog statistical technique develops equations based on the relationship of co-existing observed weather elements (including climate data), which are then applied to raw model output.

The MOS (Model Output Statistics) technique develops relationship equations between observed and model forecast weather elements and applies these relationships to raw model output (of the same or similar model) to produce statistical guidance. To develop MOS forecast equations, it is necessary to have a data set consisting of several years of historical records, which use predictands together with the model output on the days the predictand was observed.

The Kálmán-filter method - beside the data assimilation - is used for post processing the model outputs. This is a simple adaptive filter, which has been used to estimate the statistical relation between NWP outputs and verifying observations. If it is possible to establish such a relation it opens up the possibility of correcting the NWP in real time. It means that the Kálmán-filter doesn't need long datasets (in contrast with MOS and Perfect-Prog) and does not depend on the model changing (in contrast with MOS).

The following table summarizes the differences among the post processing methods above

Perfect-Prog	MOS	Kálmán-filter
Strong predictor-predictand relationships because only current observed data is used	Relationships weaken with time due to increasing model error variance	Relationships weaken with time due to increasing model error variance
Does not account for model bias; model errors decrease accuracy	Accounts for model bias	Accounts for model bias
Large development sample is needed	Generally small development samples	Very small development samples
Access to observed or analyzed variables	Access to model output variables that may not be observed	Access to model output variables that may not be observed
Does not depend on the model	After model changing it is necessary to recalculate the coefficients	Does not depend on the model

Note, that the human forecasters do similar work, as they memorize the connections between the numerical model outputs and observations and this knowledge is used for the future forecasts. This is a subjective post processing method.

Multiple linear regression

Multiple linear regression is a method used to model the linear relationship between a dependent variable and one or more independent variables. The dependent variable is also called the predictand, and the independent variables the predictors. The model expresses the value of a predictand variable as a linear function of one or more predictor variables:

The model is fit to a period, called calibration period. In the process of fitting, or estimating, the model statistics are computed. The performance of the model on independent data is usually checked in some way by a process called validation (test period). Finally, - in the prediction - the regression model is applied to generate estimates of the predictand variable outside the period used to fit the data.

Methods of choosing the predictor number and the best predictors

The predictors are sometimes specified beforehand, and are sometimes selected by some automated procedures from a pool of potential predictors. Before choosing the optimal predictors, we need the maximum number of predictors in the regression. After getting this number, various schemes for automated variable screening are available. Using one of these schemes, we can find the best ensemble of predictors. In the test period we know the predictand, so we can calculate the error of the different ensembles of predictors, and the smallest error leads to the best estimation.

Some of the schemes for variable screening are the all-possible, forward, backward and stepwise methods. The all-possible subset (or best subset) method tests all possible sets of 1,2,3,... predictors and selects the set giving the best value of accuracy adjusted for loss of degrees of freedom as measured by any of several possible statistics. The forward method enters additional predictors one by one depending on maximum reduction of the residual variance. The backward method is the opposite of the forward one. The stepwise method is similar to the forward except that predictors already in the model do not necessarily remain. Predictors are entered into and removed from the model in such a way that each forward selection step may be followed by one or more backward elimination steps. The stepwise selection process terminates if no further predictor can be added to the model or if the predictor just entered into the model is the only effect removed in the subsequent backward elimination.

All of the selection methods require a stopping criterion. Without such a rule, forward selection would continue until all the candidate predictor variables were included in the regression equations, and the backward elimination would continue until all predictors were eliminated. In the cross-validation technique the available data are repeatedly divided into developmental and verification data subset. Simply evaluating the performance of a forecast equation development algorithm on part of the data, while using the remainder of the data as the independent set constitutes a single exercise of the cross-validation procedure. Very often, cross-validation is carried out using developmental data sets of size N-1, and verification data "sets" containing the remaining single observation of the predictand. The regression model is recalculated for each of the N distinct partitions. The cross-validation estimate of the prediction error is then computed by forecasting each omitted observation using that equation developed from the remaining N-1 data values, computing the squared difference between the prediction and predictand for each of these equations, and averaging the N squared differences. The Nested F-test compares any two nested models in the forward method. Using the F-test for a function of the two nested models, we can find the best subset.

Realization at the Hungarian Meteorological Service

In our division we were dealing only with the Kálmán-filtering until 2004. At the end of 2004 we began to develop a new post-processing system and as the first step we chose the MOS technique and the system was based on the multiple linear regression method.

In this work we performed the correction of the numerical forecasts (ALADIN/HU, ECMWF) of the 2 m temperature, relative humidity and the 10 m wind (u, v component).

We made the multiple linear regression over synop stations (on the whole model domain), every timestep and every month. We considered 26 potential predictors: T2m, MSLP, RHU2, U10, V10, N, T5, T7, T8, T9, U5, U7, U8, U9, V5, V7, V8, V9, RHU5, RHU7, RHU8, RHU9, GEO5, GEO7, GEO8, GEO9. T means temperature, RHU relative humidity, MSLP mean sea level pressure, GEO geopotential and U and V zonal and meridional wind respectively. The numbers 5, 7, 8 and 9 mean the level of 500, 700, 850 and 925 hPa respectively; the numbers 2 and 10 mean the height of the observation in meter. We have chosen the "forward" method, as the "all-possible" method didn't give better results in the test period, and the second one is a more expensive method. We have chosen the number of the best predictors based on the methods of "cross-validation" and "nested F-test". For all of the predictands (T2, RHU2, U10, V10), in every month and timestep (ECMWF: 12 - 60, ALHU: 06 - 48), in every station (inside the model area), for both models (ECMWF, ALHU) and both runs (00, 12 UTC) we have concluded that the optimal maximum number of best predictors is two. Our division uses this predictor number in the operational run.

The test-runs have shown that this method gives better results in mountainous stations due to the mis-representation of orography in the models. The figure below shows the success of our method.

Figure 1  In this figure we have denoted by different shades of blue the points (stations) that have positive results and with yellow, orange and red the ones that have negative results of the 2 m temperature forecast. The numbers in the scale show the differences between the RMSE of the 2 m temperature before and after the post-processing. One can see the positive impact over the mountains, especially in the Alps.

In the Figure 2  the 2m temperature forecasts are shown before and after the post-processing for the station Kékes-tető (1015 m, the highest point of Hungary) in the period between 1st August 2005 and 31st January 2006. The result is obvious.		Figure 2 - 3 The model forecast (ECMWF_00) is shown by red and the its post-processed forecast by yellow. The RMSE of the two different kinds of forecast are plotted against the timestep (Time-TS).
In the flat terrain the results are not as successful as in the higher stations. This fact is shown by the Figure 3   where one can see the similar graph, but for Budapest. The following scatter plots (Figure 4)  show that the post-processing method helps only forecasts, which have systematic error.

Figure 4
A scatter plot of ECMWF forecast for Kékes-tető before (left panel) and after post-processing (right panel).

In the left panel a scatter plot of ECMWF forecast for Kékes-tető is shown where one can see a strong systematic error, which was corrected after the post-processing (right panel). In the plain area the scatter plots of the model outputs are very near to the y=x line, that is why the post-processing can not improve the model output.

In the left panel the relative humidity forecasts are shown before (red) and after (yellow) the post-processing for a European area in the same period as above, and in the right panel the same forecasts are shown, but only for the flat (<500 m) stations of the same area. The differences are not too big, but still noticeable compared with the fact that there are much less mountainous stations (>500 m), than flat ones.

This system summarized above is running quasi-operationally from the date of 1st August 2005. Quarterly evaluation of the results is planned from June 2006.