The Response Surface Approximation Model

A response surface model (10th-order model) is represented by a polynomial of the following form:

\tilde{F} (x) = α_{0} + \sum_{i = 1}^{N} b_{i} x_{i} + \sum_{i = 1}^{N} c_{i i} x_{i}^{2} + \sum_{i = 1}^{N - 1} \sum_{j = i + 1}^{N} c_{i j} x_{i} x_{j} + \sum_{i = j}^{N} d_{i} x_{i}^{3} + \sum_{i = 1}^{N} e_{i} x_{i}^{4} + \sum_{i = 1}^{N - 1} \sum_{j = i + 1}^{N} e_{i j} {x_{i}}^{2} x_{j} + \sum_{i = 1}^{N - 1} \sum_{j = i + 1}^{N} f_{i j} x_{i} {x_{j}}^{2} + \sum g_{i} {x_{i}}^{4} + \sum_{i = 1}^{N - 1} \sum_{j = i + 1}^{N} h_{i j} {x_{i}}^{3} x_{j} + \sum_{i = 1}^{N - 1} \sum_{j = i + 1}^{N} i_{i j} x_{i} {x_{j}}^{3} + \sum_{i = 1}^{N - 1} \sum_{j = i + 1}^{N} j_{i j} {x_{i}}^{2} {x_{j}}^{2} + \sum k_{i} {x_{i}}^{5} + \sum_{i = 1}^{N - 1} \sum_{j = i + 1}^{N} l_{i j} {x_{i}}^{4} x_{j} + \sum_{i = 1}^{N - 1} \sum_{j = i + 1}^{N} m_{i j} x_{i} {x_{j}}^{4} + \sum_{i = 1}^{N - 1} \sum_{j = i + 1}^{N} n_{i j} {x_{i}}^{3} {x_{j}}^{2} + \sum_{i = 1}^{N} o_{i} x_{i}^{6} {+ \sum}_{i = 1}^{N} p_{i} x_{i}^{7} + \sum_{i = 1}^{N} q_{i} x_{i}^{8} + \sum_{i = 1}^{N} r_{i} x_{i}^{9} + \sum_{i = 1}^{N} s_{i} x_{i}^{10}

where $N$ is the number of model inputs, $x_{i}$ is the set of model inputs, and $a$ through $s$ are the polynomial coefficients.

The order of the polynomial regression model depends on the number of data points in your data set. By default, Results Analytics uses a polynomial with up to tenth-order uni-variate terms and fifth-order cross terms for the model. However, if the number of data points is small, Results Analytics chooses the order of the terms based on the number of parameters and the number of alternatives. In addition, you can specify the order of the terms when you create an approximation using the response surface model.

Results Analytics determines the coefficients of the polynomial by solving a linear system of equations (one equation for each analyzed data point).

Results Analytics can use the response surface model in a sensitivity study with a small computational expense because evaluation only involves calculating the value of a polynomial for a given set of input values. The model accuracy is highly dependent on the amount of data used for its construction (the number of data points), the shape of the exact response function that is approximated, and the volume of the design space in which the model is constructed. In a sufficiently small volume of the design space, any smooth function can be approximated by a quadratic polynomial with good accuracy. For highly nonlinear functions, polynomials of higher order can be used. If the model is used outside the design space where it was constructed, its accuracy is impaired and the model must be refined.

The response surface model uses stepwise regression (Efroymson's algorithm) to remove some polynomial terms with low significance, which can improve reliability for your approximation and reduce the number of required data points. The stepwise method of polynomial term selection starts with the constant and adds polynomial terms one at a time so that the fitting errors of the response surface model are minimized at every step.

Results Analytics first calculates the residual sum of squares (RSS). Given a set of $k$ predictor variables $X_{1}, X_{2}, X_{3}, \dots, X_{k},$ select a subset of $p (p < k)$ predictor variables that minimizes the residual sum of squares:

R S S = {\sum_{i = 1}^{n} (Y_{i} - \sum_{j = 1}^{p} b_{j} X_{i, j})}^{2} .

The best combination of the polynomial terms is selected so that the residual sum of squares is minimized. Because the residuals can be nonzero only when the model has at least one degree of freedom, minimization of the residual sum of squares implies that the maximum number of polynomial terms selected must be lower than the number of data points used for the response surface model. Otherwise, the residual sum of squares will be exactly zero and no term selection will be possible.

A new term is added if the following condition is satisfied:

\frac{R S S_{p} - R S S_{p + 1}}{R S S_{p + 1} / (n - p - 2)} > F_{e n t e r .}

After adding a new term, Results Analytics examines all selected terms and deletes one or more terms for which the following condition is satisfied:

\frac{R S S_{p - 1} - R S S_{p}}{R S S_{p} / (n - p - 1)} < F_{d e l e t e .}

In these formulae, $p$ is the number of polynomial terms, $n$ is the number of designs used for the response surface model, $F_{e n t e r}$ is the F-ratio to add a term, and $F_{d e l e t e}$ is the F-ratio to drop a term.

For stepwise regression, Results Analytics sets the default value of each F-ratio to 4.0. You can change the value of either $F_{e n t e r}$ and/or $F_{d e l e t e}$ when you create an approximation using the response surface model.