-
Introduction
Retrievals of temperature and moisture profiles from polar orbiting
satellites have been available operationally since 1978. Initially, inclusion
of the retrievals improved the forecast skill of numerical weather prediction
models. However, as both numerical models and data assimilation techniques
improved throughout the 1980’s, positive impact in the Northern Hemisphere
became increasingly difficult to show. The quality of the retrievals also
improved, but by 1990, most operational numerical weather prediction (NWP)
centers were demonstrating a slight negative impact in the Northern Hemisphere
attributable to the use of retrieved satellite soundings. In the Southern
Hemisphere, where there are significantly fewer conventional observations,
the retrievals still showed positive impact and appear to be necessary
to maintain the quality of the forecasts there (Andersson et al. 1991).
At the same time, there was a growing awareness in the meteorological
community that the satellite information was not being used optimally.
NWP centers around the world, notably the European Center for Medium-range
Weather Forecasting (ECMWF), expended considerable effort to improve the
quality control of the soundings and to understand the complex nature of
the retrieval errors (Kelly et al. 1991). What rapidly became apparent,
however, is that the retrieval errors were dominated by air mass dependent
biases, leaving little hope for effective yet simple quality control measures.
The forward radiative transfer problem, calculating radiances from an
initial temperature and moisture profile, is becoming a better-understood
problem due to improvements in surface emissivity models and atmospheric
absorption models. However, the inverse problem of calculating a temperature
profile from the observed spectral radiances is mathematically ill-posed
(Rodgers 1976). In order to constrain the problem, prior information must
be specified. For most retrieval methods, this prior information is independent
of the NWP model and typically contains less information about the current
atmospheric state than is provided by a NWP forecast.
In addition, the satellite retrievals have relatively poor vertical
resolution due to the broad, overlapping spectral weighting functions,
so that the number of independent pieces of information is less than the
number of channels. This means that the sounder is fundamentally "blind"
to shallow vertical structures and any small-scale vertical structure in
the retrieval must come from the prior information and not the instrument
(Eyre and Lorenc 1989; Goldberg et al. 1988). Furthermore, the contribution
of errors in the prior information to the total retrieval error is significant
and difficult characterize correctly.
Most of the efforts to improve the quality control and use of the satellite
retrievals failed due to the inherent limitations discussed above. The
solution appeared to be to not use the satellite observations as if they
were poor quality radiosondes, but directly as radiances. This realization
led to research into methods to assimilate the radiances directly into
the analyses.
Both the UK Meteorological Office (UKMO) and Météo-France
developed linear techniques (Durand 1985; Eyre et al. 1985) to use the
satellite radiances more directly. The Météo-France approach
used the TOVS brightness temperatures within the three-dimensional optimum
interpolation analyses, while the UKMO developed a one-dimensional approach.
The assumption of linearity is acceptable in the infrared and microwave
for temperatures, but is not valid for humidity.
Concurrent to this time was the renewed interest in variational analysis
techniques, which naturally allow for nonlinear and indirect relationships
between the observed and model state variables. The renewed interest was
partially spurred by the work of Le Dimet and Talagrand (1985) and others
(see Courtier et al. 1993), who proposed the adjoint technique as a means
of making large variational problems tractable. This, coupled with significant
increases in computational speed and memory over the previous decade, has
allowed problems of the size of global data assimilation to become computationally
feasible.
The simplest version of variational assimilation of radiances to implement
is to consider the problem in one (vertical) dimension (known as 1D-VAR).
The observed radiances are combined using optimal estimation theory, a
forward radiative transfer model and its adjoint, with the co-located temperature
and humidity profiles from a short term forecast from the NWP model. The
resulting maximum likelihood estimates of temperature and moisture may
be assimilated directly into the existing operational optimum interpolation
analysis. Results from ECMWF indicated consistent positive impact in the
Northern Hemisphere from the 1DVAR retrievals (Eyre et al. 1992; McNally
1992). Both NCEP and ECMWF have implemented three-dimensional variational
assimilation systems for all observations, including TOVS radiances; ECMWF
recently implemented a four-dimensional variational system. 1DVAR has a
fundamental technical limitation when used for update cycling - namely,
the same background is used for both the 1DVAR retrieval and the subsequent
full three-dimensional analysis: this leads to 1DVAR retrievals whose errors
are correlated with the analysis background errors.
The primary advantage of variational assimilation methods is that nonlinear
observation operators can be readily included allowing for the consistent
use of indirect observations. The disadvantages are that the development
of the observation operator and its adjoint and appropriate error covariances
require considerable effort, knowledge and experience. For the radiance
problem, this includes knowledge of the satellite sensors, radiative transfer
theory, principles of remote sensing, data assimilation, numerical modeling
and parameterization schemes and meteorology. Many components of the problem
are unique to each individual sensor. These include the forward radiative
transfer model, expected error covariances of the observations and forward
operator, quality control and bias corrections.
-
Optimal Estimation Theory
There are two different approaches for solving the optimal analysis
equation. The first is based on minimum variance estimation and is the
basis for optimum interpolation analysis. The alternate approach is maximum
likelihood estimation and provides the theoretical framework for variational
analysis. For the linear problem, each approach leads to the same solution.
The following summary is based on papers by Lorenc (1988), Eyre et al.
(1992) and Talagrand (1995).
For the linear estimation problem, the minimum variance solution is
identical to the maximum likelihood estimate obtained using probability
theory. The maximum likelihood approach is outlined below. We wish to find
the most probable state of the atmosphere x given the observations
y, i.e., find the maximum conditional probability P(x|y).
Using Bayes’ theorem:
( 1)
where P(y|x) is the conditional probability of making the observations
y given the atmospheric state x and P(x) is the prior
probability of x before using any observations.
If we assume that the errors are Gaussian, then
( 2)
and
( 3)
where the variables are defined as below. Maximizing the conditional
probability, P(x|y) = P(y|x)P(x), or minimizing -ln P(x|y) leads
to the cost function J(x) which we wish to minimize. J(x) is a scalar measure
of the fit to all of the observations, the background information and any
optional physical or dynamical constraints to the maximum likelihood estimate
x, and is given by
( 4)
If the errors in the observations and the background are Gaussian, unbiased
and mutually uncorrelated, then J(x) represents the optimal penalty function.
Here, xb represents the prior information
or background, typically a short term forecast from a numerical prediction
model. The vector of observations is given by y. The operator H{x}
is the observation operator and consists on spatial and temporal interpolations
as well as any transformations from the model space variables to the observed
quantities or observation space. For the radiance assimilation problem,
H{x} contains the forward radiative transfer equations (RTE). The
expected error covariances of the background are contained in Pb,
while O and F are the observation and forward model error
covariances, respectively. Jc represents any optional constraints. This
term will be ignored in the following derivation.
There are several methods for finding the minimum of the penalty function,
such quasi-Newton, steepest descent and conjugate gradient methods. All
of these methods require computation of 
or:
,
( 5)
where 
is the Jacobian
matrix or the tangent linear operator corresponding to nonlinear forward
operator H and linearized about x. The Jacobian matrix may be computed
at a non-prohibitive cost by using the adjoint of the forward model which
effectively computes the result of the Jacobian operating on a gradient,
or 
. The Hessian matrix
is obtained by differentiating again:
.
( 6)
This is approximate because terms involving the gradient of H
have been omitted. At the minimum, 
represents the inverse of the analysis or estimation error covariance,
A-1.
For the one-dimensional variation problem of satellite radiances, the
dimension of the problem is small enough that Newtonian iteration may be
used. Here, the nth estimate of the state vector x is updated
by:
( 7)
Now, by substitution:
],
( 8)
where
( 9)
-
Present use of satellite retrievals
in Navy Operational Forecast Models
In the present NOGAPS (Navy Operational Global Atmospheric Prediction
System) and COAMPS (Coupled Ocean Atmosphere Mesoscale Prediction System)
configuration, the operational NESDIS retrievals of temperature and specific
humidity are used. The multivariate optimal interpolation (MVOI) system
analyzes the mass field in terms of geopotential thickness between the
standard pressure levels; the moisture analysis is a univariate analysis
separate from the MVOI mass and wind field analyses. The NESDIS retrievals
of the 40 pressure-level temperature and 15 pressure level specific humidity
profiles are used to compute first the virtual temperature and then the
layer geopotential thickness. The specific humidity profiles are converted
to dewpoint depression for assimilation into the moisture analysis.
Because the errors in retrievals are very difficult to properly characterize,
the quality control performed on the retrievals is very simplistic and
is designed to identify only the gross errors. The thicknesses are compared
against the background thickness; if the deviation is larger than three
times the expected observational error, the value is rejected. A comparison
of the retrieval lapse rate against the background lapse rate is also performed.
This check was developed by Kelly et al. (1991), who studied the
innovation statistics for the NESDIS retrievals. They found that errors
of one sign tended to be compensated for by errors of the opposite sign
above (aloft). These errors occurred most frequently in regions such as
frontal zones where the vertical scale is relatively small compared to
the vertical resolution of the satellite
Figure 1 illustrates the present use of satellite retrievals in the
NOGAPS. The present system has many different modules, which are independent
of each other. Information from one source is not necessarily available
to other steps in the process, and is certainly not effectively utilized
by the other processes.
The next section discusses the implementation of variational assimilation
methods in a one-dimensional sense and serves as an illustration as to
how the available information can be more effectively combined.
FIG. 1. Schematic illustrating present operational
use of satellite retrievals.
-
One-dimensional variational assimilation
One-dimensional variational analysis (1DVAR) minimizes the cost function
(4) using Newtonian iteration to arrive at the maximum likelihood estimate
given by (8). The background is given by a short term forecast from the
NWP model interpolated bilinearly to the observation location and pressure
levels required by the forward model (40 for RTTOVS 3.0; Eyre 1991). The
background, associated error covariances, plus any other control variables
required by the forward model but that are not a part of the NWP model,
act as constraints on the minimization.
In 1DVAR, the final retrieval step is performed explicitly, and the
resulting profiles of temperature and humidity may be readily used in current
MVOI analysis system. By comparison, with 3DVAR, the final retrieval step
is not performed for single profiles. Instead, all of the observations,
including radiances, are used together with the background fields and associated
error covariances to produce the final three-dimensional analysis. While
3DVAR clearly represents more optimal use of the observations, 1DVAR isolates
the problem and is a very useful tool for understanding specific aspects
of the problem. Many of the components of 3DVAR are common to 1DVAR, and
can be developed within the 1DVAR framework. At NRL, 1DVAR is used as a
pre-processor for the 3DVAR system to perform the quality control, radiance
monitoring and bias correction functions. 1DVAR is also used to provide
control variables needed by the forward model that are not produced by
the forecast model, such as the background temperature profile above the
NWP model top.
The following sections discuss a few of the specific components and
aspects of the problem that have been developed and investigated using
1DVAR.
-
Forecast error covariances
The specification of the forecast error covariances is critically important
for the successful assimilation of radiance information. The background
error covariances describe how the information contained in the observation
increment (innovation vector) is spread in the vertical. In particular,
the interpretation of radiance information is sensitive to the specified
error correlations. For TOVS, the radiation measured in a given channel
emanates from a layer of depth comparable to or greater than the forecast
error vertical correlation lengths. These vertical correlations for temperature
must be consistent with observations, and consistent with the other model
variables (Eyre 1995). Improper specification of these statistics
can lead to noisy analysis increments (Andersson et al. 1994), and minimal
reduction in the analysis error (Daley, personal communication).
-
Observation and forward model error covariances
Measured brightness temperatures and those computed using a forward
radiative transfer model will differ due to several sources of error. The
instrument error itself (usually referred to as the radiometer noise equivalent
temperature difference or NEDT) is typically
quite small, less than 0.5 oK. Other sources include errors
in the forward radiative transfer model. These errors may be due to approximations
made to improve computational speed or because the spectroscpopic parameters
in the forward model are not well understood. Large differences between
the forecast and observed differences may occur if the forward model doesn’t
account for all of the physics found in the radiance observations (such
as scattering by hydrometeors and ice, or poor surface emission estimates).
If the forecast radiances cannot reproduce all of the physics inherent
in the observed radiances, then the observations cannot be used. Presently,
use of radiances over land (esp. for microwave) is severely restricted
due to the lack of good surface emissivity models (especially for microwave
frequencies) and skin temperature estimates.
The observation error covariance matrix is usually assumed to be diagonal,
meaning that the observation errors are assumed to be uncorrelated. This
assumption may not be valid, as correlated errors may be introduced to
the radiances through data reduction processes such as calibration. The
forecast radiances may also have correlated error introduced via the forward
RTE model and due to correlated error in the prior information. As pointed
out by Uddstrom and McMillin (1994a, 1994b), it is important that the total
system noise be taken into account.
-
Bias Corrections and Quality Control
One of the principle advantages of variational assimilation of radiances
is that the errors in radiance space are much easier to characterize than
the errors in the retrievals. Since the retrieval problem is both nonlinear
and strongly dependent upon the prior information, the solution is highly
dependent upon the background or prior information. Likewise, errors in
the background will also strongly affect the retrieval solution.
Quality control and data screening in radiance space is critically important.
Large differences between the forecast and observed differences may occur
if the forward model doesn’t account for all of the physics found in the
radiance observations (such as scattering by hydrometeors and ice, or poor
surface emission estimates). If the forecast radiances cannot reproduce
all of the physics inherent in the observed radiances, then the observations
cannot be used. Presently, use of radiances over land (esp. for microwave)
is severely restricted due to the lack of good surface emissivity models
(especially for microwave frequencies) and skin temperature estimates.
For TOVS, it has been shown that the systematic differences (after screening
the radiances as described above) between the observed and forecast radiances
are typically on the same order of magnitude as the corrections made to
the analysis background by the observations. The biases have contributions
from different sources. Any pre-processing of the radiances can introduce
systematic errors. For example, if the brightness temperatures are adjusted
to nadir, a residual scan angle dependent bias may result. Another source
of bias arises from inaccuracies in the forward model specification of
the spectroscopic parameters such as the optical depth coefficients. These
biases show up as a strong airmass dependence in some of the channels.
Many of the sources of the biases are not well understood and merit further
investigation by the NWP and remote sensing communities to resolve and
reduce these large differences.
The current approach used for TOVS at NRL follows that of Eyre (1992).
An archive of the differences between the observed minus computed (forecast)
brightness temperatures is maintained. Every few weeks, two different bias
calculations are computed. The first step computes a bias correction as
a function of scan angle. At present, this represents a global (non-latitude
dependent) correction. After the scan dependent portion of the bias is
removed from the differences, an automated linear regression analysis is
performed. A subset of the channels (MSU-2, 3 and 4) is used as predictors
to calculate the new bias corrections, which are allowed to vary spatially
(over rather large latitude bands regions).
-
References
Andersson, E., A. Hollingsworth, G. Kelly, P. Lönnberg, J. Pailleux
and Z. Zhang, 1991: Global observing system experiments on operational
statistical retrievals of satellite sounding data. Mon. Wea. Rev.,
119, 1851-1864.
Andersson, E., J. Pailleux, J.-N. Thépaut, J.R. Eyre, A.P. McNally,
G.A. Kelly and P. Courtier, 1994: Use of cloud-cleared radiances
in three/four-dimensional variational data assimilation. Q.J.R. Meteor.
Soc., 120, 627-653.
Courtier, P., J. Derber, R. Errico, J.-F. Louis and T. Vukicevic, 1993:
Important literature on the use of adjoint, variational methods and the
Kalman filter in meteorology. Tellus, 45A, 342-357.
Durand, Y., 1985: The use of satellite data in the French high resolution
analysis. Proc. ECMWF Workshop on “High Resolution Analysis," Reading,
24-26 June 1985, pp. 89-127.
Eyre, J.R., P.D. Watts, J. Turner and A.C. Lorenc, 1985: Research
and development on TOVS retrievals in the U.K. Tech. Proc. 2nd Int.
TOVS Study Conf., Igls, Austria, 18-22 Feb. 1985; Ed. W.P. Menzel; Report
of CIMSS, University of Wisconsin-Madison, pp. 94-100.
Eyre, J.R. and A.C. Lorenc, 1989: Direct use of satellite sounding
radiances in numerical weather prediction. Meteor. Mag., 118, 13-16.
Eyre, J.R., 1991: A fast radiative transfer model for satellite
sounding systems. ECMWF Tech. Mem., 176.
Eyre, J.R., 1992: A bias correction scheme for simulated TOVS
brightness temperatures. ECMWF Tech. Mem., 186.
Eyre, J.R., G.A. Kelly, A.P. McNally and E. Andersson, 1992: Assimilation
of TOVS radiances information through one-dimensional variational analysis.
ECMWF Tech. Mem., 187.
Eyre, J.R., 1995: Variational assimilation of remotely-sensed
observations of the atmosphere. ECMWF Tech. Mem., 221.
Goldberg, M.D., J.M. Daniels and H.E. Fleming, 1988: A method for obtaining
an improved initial approximation for the temperature/ moisture retrieval
problem. Preprints, Third Conference on Satellite Meteorology and
Oceanography. Amer. Meteor. Soc., Anaheim, Ca.
Kelly, G., E. Andersson, A. Hollingsworth, P. Lönnberg, J. Pailleux
and Z. Zhang, 1991: Quality control of operational physical retrievals
of satellite sounding data. Mon. Wea. Rev., 119, 1866-1880.
Le Dimet, F.X. and O. Talagrand, 1986: Variational algorithms
for analysis and assimilation of meteorological observations: theoretical
aspects. Tellus, 38A, 97-110.
Lorenc, A.C., 1988: Optimal nonlinear objective analysis.
Mon. Wea. Rev., 114, 205-240.
McNally, A.P., 1992: A new approach to the use of TOVS satellite
data at ECMWF. ECMWF Operational Newsletter, 59, 3-12.
Rodgers, C.D., 1976: Retrieval of atmospheric temperature and
composition from remote measurements of thermal radiation. Rev. Geophys.
Space Phys., 60, 609-624.
Talagrand, O., 1995: Assimilation of observations, an introduction.
From the (still) unpublished Proceedings of the International Symposium
on Assimilation of Observations in Meteorology and Oceanography, March
13-17, 1995, Tokyo, Japan.
Uddstrom, M.J. and L.M. McMillin, 1994a: System noise in NESDIS TOVS
forward model. Part I: Specification. J. Appl. Meteor., 33, 919-938.
Uddstrom, M.J. and L.M. McMillin, 1994b: System noise in NESDIS TOVS
forward model. Part II: Consequences. J. Appl. Meteor., 33, 939-947.
