🧭 Overview
🧠 One-sentence thesis
Elementary single-step methods approximate solutions to initial-value problems by using different numerical quadrature rules to approximate the integral form of the differential equation over one time interval, with each method trading off between computational simplicity, accuracy, and stability.
📌 Key points (3–5)
- What single-step methods do: approximate the solution by integrating the differential equation over one time interval [tₙ, tₙ₊₁], using only information from the current time step.
- Four basic methods: Forward Euler (explicit, simplest), Backward Euler (implicit, unconditionally stable), Trapezoidal (implicit, second-order accurate), and Modified Euler (explicit predictor-corrector, second-order).
- Explicit vs implicit distinction: explicit methods (Forward Euler, Modified Euler) compute wₙ₊₁ directly from known values; implicit methods (Backward Euler, Trapezoidal) require solving an equation where wₙ₊₁ appears on both sides.
- Common confusion—stability vs accuracy: a method may require a very small time step for stability even when accuracy would allow larger steps; implicit methods avoid this by being unconditionally stable but cost more per step.
- Why it matters: these methods form the foundation for numerical time integration, and understanding their accuracy (local truncation error) and stability properties determines which method to use for a given problem.
🔧 Core framework: from differential to integral equation
🔧 The integral equation starting point
The excerpt emphasizes that although the differential equation y′ = f(t, y) is hard to solve directly, it can be rewritten as an integral equation:
y(t) = y(t₀) + ∫[t₀ to t] f(τ, y(τ)) dτ
- This form is called an integral equation because the unknown function y appears under the integral sign.
- It is equally difficult to solve as the original problem, but it provides a better starting point for numerical approximation.
- Why this helps: numerical quadrature rules (Chapter 5) can approximate the integral, leading to different single-step methods.
🔧 Discretization and notation
-
The time axis is divided into discrete points: tₙ = t₀ + n·Δt, n = 0, 1, 2, ...
-
The exact solution at tₙ is denoted yₙ = y(tₙ).
-
The numerical approximation at tₙ is denoted wₙ, with w₀ = y₀ (the initial condition).
-
From tₙ to tₙ₊₁, the integral equation becomes:
yₙ₊₁ = yₙ + ∫[tₙ to tₙ₊₁] f(t, y(t)) dt
-
Different methods arise from different ways to approximate this integral.
🔧 What "single-step" means
Because integral equation (6.3) only considers one time interval [tₙ, tₙ₊₁], these numerical methods are called single-step methods.
- Only information from the current step tₙ is used to compute wₙ₊₁.
- Multi-step methods (Section 6.11) also use earlier steps t₀, ..., tₙ₋₁.
🧮 The four elementary methods
🧮 Forward Euler method (explicit)
The Forward Euler method is the most simple, earliest, and best-known time-integration method.
How it works:
- Approximates the integral using the left Rectangle rule: the integrand is evaluated at the left endpoint tₙ.
- Formula: wₙ₊₁ = wₙ + Δt·f(tₙ, wₙ)
- Geometrically: f(tₙ, wₙ) is the slope of the tangent at (tₙ, wₙ); the method follows this tangent for one time step.
- The sequence {tₙ, wₙ} forms a piecewise-linear approximation called the Euler polygon.
Why it's explicit:
- wₙ₊₁ can be computed directly from the formula; no equation-solving is needed.
Example scenario: If you know the current state wₙ and the slope f(tₙ, wₙ), you simply step forward along that slope for time Δt.
🧮 Backward Euler method (implicit)
How it works:
- Approximates the integral using the right Rectangle rule: the integrand is evaluated at the right endpoint tₙ₊₁.
- Formula: wₙ₊₁ = wₙ + Δt·f(tₙ₊₁, wₙ₊₁)
- The unknown wₙ₊₁ appears on both sides of the equation.
Why it's implicit:
- If f depends linearly on y, the equation can be solved easily.
- For nonlinear problems, a numerical nonlinear solver (Chapter 4) is required at each time step.
- This costs more computation time per step than Forward Euler.
Advantage: The Backward Euler method has better stability properties (discussed later), which can outweigh the extra cost per step.
🧮 Trapezoidal method (implicit)
How it works:
- Approximates the integral using the Trapezoidal rule: averages the integrand at both endpoints.
- Formula: wₙ₊₁ = wₙ + (Δt/2)·(f(tₙ, wₙ) + f(tₙ₊₁, wₙ₊₁))
- Also implicit: wₙ₊₁ appears on the right-hand side.
Why it's better: The Trapezoidal method is more accurate than Forward or Backward Euler (second-order vs first-order local truncation error).
🧮 Modified Euler method (explicit predictor-corrector)
How it works:
- An explicit variant of the Trapezoidal method.
- Predictor step: use Forward Euler to predict w̄ₙ₊₁ = wₙ + Δt·f(tₙ, wₙ)
- Corrector step: use the predicted value in the Trapezoidal formula:
wₙ₊₁ = wₙ + (Δt/2)·(f(tₙ, wₙ) + f(tₙ₊₁, w̄ₙ₊₁))
- Because the predictor is computed first, the corrector becomes an explicit formula.
Why it's useful: achieves second-order accuracy (like Trapezoidal) while remaining explicit (like Forward Euler).
Don't confuse: The Modified Euler method is explicit because the predictor removes the implicit dependence; the Trapezoidal method is implicit because wₙ₊₁ appears on both sides without a predictor.
📊 Comparison table
| Method | Type | Formula | Function evaluations per step | Remarks |
|---|
| Forward Euler | Explicit | wₙ₊₁ = wₙ + Δt·f(tₙ, wₙ) | 1 | Simplest; conditionally stable |
| Backward Euler | Implicit | wₙ₊₁ = wₙ + Δt·f(tₙ₊₁, wₙ₊₁) | 1 + solver | Unconditionally stable |
| Trapezoidal | Implicit | wₙ₊₁ = wₙ + (Δt/2)·(f(tₙ, wₙ) + f(tₙ₊₁, wₙ₊₁)) | 1 + solver | Second-order; unconditionally stable |
| Modified Euler | Explicit | Predictor + corrector | 2 | Second-order; conditionally stable |
🎯 Key concepts for analysis
🎯 Local truncation error
The local truncation error at time step n+1, τₙ₊₁, is defined as τₙ₊₁ = (yₙ₊₁ - zₙ₊₁)/Δt
- yₙ₊₁ is the exact solution at time step n+1.
- zₙ₊₁ is the numerical approximation obtained by applying the method with starting point yₙ (the exact solution at step n).
- This measures the new error introduced in one step, assuming the previous step was exact.
Orders of accuracy (from the excerpt):
- Forward Euler: τₙ₊₁ = O(Δt) — first-order
- Backward Euler: τₙ₊₁ = O(Δt) — first-order
- Modified Euler: τₙ₊₁ = O(Δt²) — second-order
- Trapezoidal: τₙ₊₁ = O(Δt²) — second-order
Example: For Forward Euler, using Taylor expansion shows τₙ₊₁ = (Δt/2)·y″(ξ) for some ξ in (tₙ, tₙ₊₁), so the error is proportional to Δt.
🎯 Global truncation error
The global truncation error eₙ₊₁ at time tₙ₊₁ is defined as eₙ₊₁ = yₙ₊₁ - wₙ₊₁
- This is the difference between the exact solution and the numerical approximation.
- It accumulates over all previous steps.
- Key theorem (Lax's equivalence theorem): If a method is stable and consistent, then the global truncation error has the same order as the local truncation error.
Don't confuse: Local error measures one step assuming you start from the exact solution; global error measures the cumulative effect of all errors over many steps.
🎯 Consistency and convergence
A method is consistent if lim[Δt→0] τₙ₊₁(Δt) = 0, where (n+1)·Δt = T (fixed time).
A method is convergent if lim[Δt→0] eₙ₊₁ = 0, where (n+1)·Δt = T.
- Consistency means the local error vanishes as the time step shrinks.
- Convergence means the global error vanishes as the time step shrinks.
- All four elementary methods are consistent.
🔒 Stability concepts
🔒 The test equation
To analyze stability, the excerpt uses a simplified linear problem:
y′ = λy + g(t), t > t₀, y(t₀) = y₀
- The solution to the perturbed problem (with initial error ε₀) satisfies ε(t) = ε₀·exp(λ(t - t₀)).
- The problem is stable if |ε(t)| < ∞ for all t > t₀.
- It is absolutely stable if lim[t→∞] |ε(t)| = 0.
- This requires λ ≤ 0 for stability, λ < 0 for absolute stability.
🔒 Amplification factor
For each method applied to the test equation, the error propagates as:
ε̄ₙ₊₁ = Q(λΔt)·ε̄ₙ
- Q(λΔt) is the amplification factor: it determines how much an existing perturbation is amplified.
- Numerical stability requires |Q(λΔt)| ≤ 1.
Amplification factors (from the excerpt):
- Forward Euler: Q(λΔt) = 1 + λΔt
- Backward Euler: Q(λΔt) = 1/(1 - λΔt)
- Trapezoidal: Q(λΔt) = (1 + ½λΔt)/(1 - ½λΔt)
- Modified Euler: Q(λΔt) = 1 + λΔt + ½(λΔt)²
🔒 Stability conditions for λ < 0
Forward Euler:
- Requires -1 ≤ 1 + λΔt ≤ 1.
- Since λ < 0, this gives Δt ≤ -2/λ.
- Conditionally stable: time step must be small enough.
Backward Euler:
- Requires |1/(1 - λΔt)| ≤ 1, which is satisfied for all Δt ≥ 0 when λ ≤ 0.
- Unconditionally stable: no restriction on time step size.
Modified Euler:
- Similar to Forward Euler: Δt ≤ -2/λ.
- Conditionally stable.
Trapezoidal:
- Unconditionally stable for λ ≤ 0.
Don't confuse: Unconditional stability means you can take any Δt for stability; it does not mean you can take arbitrarily large Δt for accuracy—accuracy still requires small enough Δt.
🔒 Stability for nonlinear problems
For a general nonlinear problem y′ = f(t, y), linearize about a point (t̂, ŷ):
λ = ∂f/∂y|(t̂,ŷ)
- Use this λ in the stability condition |Q(λΔt)| ≤ 1.
- The stability condition depends on the current approximation and time.
Example from excerpt: For y′ = -10y² + 20, the linearization gives λ = -20ŷ. Since y(t) ∈ (0, √2), the Forward Euler method is stable if Δt ≤ 1/(10√2).
🚀 Higher-order methods: RK4
🚀 The fourth-order Runge-Kutta method
The excerpt introduces the RK4 method as a widely-used higher-order explicit method.
Formula:
- wₙ₊₁ = wₙ + (1/6)·(k₁ + 2k₂ + 2k₃ + k₄)
where the predictors are:
- k₁ = Δt·f(tₙ, wₙ)
- k₂ = Δt·f(tₙ + ½Δt, wₙ + ½k₁)
- k₃ = Δt·f(tₙ + ½Δt, wₙ + ½k₂)
- k₄ = Δt·f(tₙ + Δt, wₙ + k₃)
Why it's better:
- Based on Simpson's rule for numerical integration.
- Local truncation error is O(Δt⁴) — fourth-order accuracy.
- Requires 4 function evaluations per step, but the higher accuracy often allows much larger time steps.
Stability:
- Amplification factor: Q(λΔt) = 1 + λΔt + ½(λΔt)² + (1/6)(λΔt)³ + (1/24)(λΔt)⁴
- For λ < 0, stable if Δt ≤ -2.8/λ.
- Conditionally stable, but with a larger stability region than Forward or Modified Euler.
Example from excerpt (Table 6.2): To achieve |eₙ| ≤ 10⁻⁴ integrating from t₀ = 0 to T = 1:
- Forward Euler needs 10,000 function evaluations.
- Modified Euler needs 200 function evaluations.
- RK4 needs only 40 function evaluations.
🚀 Why higher order matters
The excerpt emphasizes that higher-order methods are preferable when high accuracy is required, as long as the solution is sufficiently smooth.
- Fewer time steps are needed for a given accuracy.
- Total work depends on both the number of steps and the cost per step.
- Example: A second-order method with 2 evaluations per step can be more efficient than a first-order method with 1 evaluation per step.
🔁 Richardson extrapolation for error estimation
🔁 Estimating the global error when order p is known
Assume the global error behaves as e(t, Δt) = cₚ(t)·Δtᵖ for small Δt.
-
Compute two approximations: w^(Δt)N using N steps of size Δt, and w^(2Δt)(N/2) using N/2 steps of size 2Δt.
-
The difference between them estimates the error:
y(t) - w^(Δt)_N ≈ (w^(Δt)N - w^(2Δt)(N/2))/(2ᵖ - 1)
-
The factor 1/(2ᵖ - 1) depends on the order p of the method.
Example from excerpt (Table 6.4): For the Forward Euler method (p = 1) applied to a water-discharge problem, the error estimate is approximately doubled when the time step is doubled, confirming first-order behavior.
🔁 Estimating the order p when unknown
If the order is unknown, compute three approximations with Δt, 2Δt, and 4Δt.
- The ratio (w^(2Δt) - w^(4Δt))/(w^(Δt) - w^(2Δt)) ≈ 2ᵖ
- If this ratio is close to the expected 2ᵖ, then Δt is small enough for accurate error estimation.
Practical use: This approach enables adaptive time-stepping: adjust Δt during integration to keep the estimated error below a threshold.
🌐 Extension to systems of equations
🌐 Vector notation
For a system of m differential equations:
y′ⱼ = fⱼ(t, y₁, ..., yₘ), j = 1, ..., m
- Write in vector form: y′ = f(t, y), where y and f are vectors of length m.
- All single-step methods generalize directly to vector form.
Example: Forward Euler becomes wₙ₊₁ = wₙ + Δt·f(tₙ, wₙ).
🌐 Higher-order problems
A single mth-order differential equation can be converted to a first-order system by defining:
y₁ = x, y₂ = x′, ..., yₘ = x^(m-1)
Example from excerpt (mathematical pendulum):
- Original: ψ″ + sin(ψ) = 0
- System: y₁′ = y₂, y₂′ = -sin(y₁)
🌐 Stability for systems
For the linear test system y′ = Ay + g(t):
- The eigenvalues λⱼ of matrix A play the role of λ in the scalar test equation.
- Numerical stability requires |Q(λⱼΔt)| ≤ 1 for all eigenvalues λⱼ.
- If any eigenvalue has Re(λⱼ) > 0, the system is unstable.
Example from excerpt (second-order IVP): For ax″ + bx′ + cx = g(t), the eigenvalues are λ = (-b ± √(b² - 4ac))/(2a). Stability depends on the signs of the real parts.
Don't confuse: For systems, you must check the stability condition for every eigenvalue; a single eigenvalue outside the stability region makes the entire system unstable.
⚠️ Stiff differential equations
⚠️ What stiffness means
Stiff differential equations describe problems that exhibit transients. Their solution is the sum of a rapidly decaying part (the transient) and a slowly-varying part.
Characteristics:
- The transient decays on a fast timescale (determined by strongly negative eigenvalues).
- The quasi-stationary solution varies on a slow timescale.
- After a short time, only the slowly-varying part remains visible.
Why it's a problem for explicit methods:
- Stability requires Δt small enough to handle the fast transient.
- But accuracy for the slow quasi-stationary solution would allow much larger Δt.
- Result: you're forced to take tiny steps long after the transient has vanished.
⚠️ Implicit methods for stiff problems
Backward Euler and Trapezoidal methods are unconditionally stable, so they can use larger time steps.
Superstability:
A method is superstable if lim[λΔt→-∞] |Q(λΔt)| < 1.
- Backward Euler is superstable: initial perturbations in fast components decay quickly.
- Trapezoidal is not superstable: |Q(λΔt)| → 1 as λΔt → -∞, so initial errors decay very slowly.
Example from excerpt (Figure 6.5): For a stiff problem with λ = -100 and Δt = 0.2:
- Backward Euler: |Q(λΔt)| ≈ 0.048 — fast error decay.
- Trapezoidal: |Q(λΔt)| ≈ 0.82 — slow error decay.
Trade-off: Implicit methods require solving a (possibly nonlinear) system at each step, increasing computational cost. The choice depends on the problem.
Summary: Elementary single-step methods provide a foundation for numerical time integration. Explicit methods (Forward Euler, Modified Euler) are simple but conditionally stable; implicit methods (Backward Euler, Trapezoidal) are unconditionally stable but require equation-solving. Higher-order methods (RK4) achieve better accuracy with fewer steps. Understanding local vs global errors, stability conditions, and the role of the amplification factor is essential for choosing and applying these methods effectively.