TSLS - Two-Stage Least Squares

Overview

The Two-Stage Least Squares (TSLS) estimator provides consistent estimation for overidentified instrumental variables models where the number of instruments exceeds the number of endogenous regressors. It uses a two-stage procedure to efficiently exploit additional identifying information and enables overidentification testing.

Mathematical Foundation

Two-Stage Procedure

Stage 1: Predict endogenous regressors using instruments \(\(\hat{\mathbf{X}} = \mathbf{Z}(\mathbf{Z}'\mathbf{Z})^{-1}\mathbf{Z}'\mathbf{X}\)\)

Stage 2: Regress outcome on predicted values \(\(\hat{\mathbf{\beta}}_{TSLS} = (\hat{\mathbf{X}}'\hat{\mathbf{X}})^{-1}\hat{\mathbf{X}}'\mathbf{y}\)\)

Identification Requirements

Overidentification: \(\text{dim}(\mathbf{Z}) \geq \text{dim}(\mathbf{X})\)
Instrument Relevance: \(\text{Cov}(\mathbf{Z}, \mathbf{X}) \neq 0\)
Instrument Exogeneity: \(\text{Cov}(\mathbf{Z}, \mathbf{\epsilon}) = 0\)

Covariance Matrix

\[\text{Var}(\hat{\mathbf{\beta}}_{TSLS}) = \sigma^2(\hat{\mathbf{X}}'\hat{\mathbf{X}})^{-1}\]

where \(\sigma^2\) is estimated from second-stage residuals.

API Reference

Constructor

TSLS(fit_intercept=True)

Parameters: - fit_intercept (bool): Whether to calculate the intercept

Methods

fit()

fit(instruments, regressors, targets) -> None

Fit the TSLS model using optimized two-stage procedure.

Parameters: - instruments (np.ndarray): Instrumental variables matrix Z - regressors (np.ndarray): Endogenous regressors matrix X
- targets (np.ndarray): Target values y

Requirements: - Number of instruments ≥ number of regressors - Strong first stage (F-statistic > 10 recommended)

predict()

predict(regressors) -> np.ndarray

Generate predictions using fitted TSLS coefficients.

Statistical Methods

standard_errors() → Standard errors accounting for two-stage estimation
t_statistics() → T-statistics for significance tests
p_values() → P-values for coefficient tests
confidence_intervals(alpha=0.05) → Confidence intervals
summary() → Comprehensive regression summary
covariance_matrix() → TSLS covariance matrix

Properties

Core Results: - coefficients: TSLS regression coefficients - intercept: TSLS regression intercept - r_squared: R-squared (can be negative for IV models) - r_squared_adj: Adjusted R-squared - mse: Mean squared error - residuals: TSLS regression residuals

Model Information: - n_samples: Number of observations - n_features: Number of features - fit_intercept: Whether intercept is included

Implementation Details

Numerical Stability

Automatic algorithm selection in both stages
SVD fallback for ill-conditioned matrices
Robust handling of weak instrument problems

Performance Features

Memory-efficient: intermediate matrices not stored
Vectorized operations for optimal speed
Intelligent algorithm selection based on problem size

Usage Guidelines

Use TSLS when: - Endogenous regressors present - More instruments than endogenous regressors - Want to test overidentification restrictions - Need efficient use of multiple instruments

Advantages over IV: - Exploits additional instruments for efficiency - Enables overidentification testing - Generally more robust to weak instruments

Consider alternatives when: - Exactly identified → IV may be simpler - No endogeneity → OLS - Weak instruments → Consider regularized methods - Complex error correlation → GLS