estimation.feols_.Feols

estimation.feols_.Feols(
    self,
    FixestFormula,
    data,
    ssc_dict,
    drop_singletons,
    drop_intercept,
    weights,
    weights_type,
    collin_tol,
    fixef_tol,
    lookup_demeaned_data,
    solver='np.linalg.solve',
    store_data=True,
    copy_data=True,
    lean=False,
    sample_split_var=None,
    sample_split_value=None,
)

Non user-facing class to estimate a linear regression via OLS.

Users should not directly instantiate this class, but rather use the feols() function. Note that no demeaning is performed in this class: demeaning is performed in the FixestMulti class (to allow for caching of demeaned variables for multiple estimation).

Parameters

Name	Type	Description	Default
Y	np.ndarray	Dependent variable, a two-dimensional numpy array.	required
X	np.ndarray	Independent variables, a two-dimensional numpy array.	required
weights	np.ndarray	Weights, a one-dimensional numpy array.	required
collin_tol	float	Tolerance level for collinearity checks.	required
coefnames	list[str]	Names of the coefficients (of the design matrix X).	required
weights_name	Optional[str]	Name of the weights variable.	required
weights_type	Optional[str]	Type of the weights variable. Either “aweights” for analytic weights or “fweights” for frequency weights.	required
solver	str, optional.	The solver to use for the regression. Can be either “np.linalg.solve” or “np.linalg.lstsq”. Defaults to “np.linalg.solve”.	'np.linalg.solve'

Attributes

Name	Type	Description
_method	str	Specifies the method used for regression, set to “feols”.
_is_iv	bool	Indicates whether instrumental variables are used, initialized as False.
_Y	np.ndarray	The demeaned dependent variable, a two-dimensional numpy array.
_X	np.ndarray	The demeaned independent variables, a two-dimensional numpy array.
_X_is_empty	bool	Indicates whether the X array is empty.
_collin_tol	float	Tolerance level for collinearity checks.
_coefnames	list	Names of the coefficients (of the design matrix X).
_collin_vars	list	Variables identified as collinear.
_collin_index	list	Indices of collinear variables.
_Z	np.ndarray	Alias for the _X array, used for calculations.
_solver	str	The solver used for the regression.
_weights	np.ndarray	Array of weights for each observation.
_N	int	Number of observations.
_k	int	Number of independent variables (or features).
_support_crv3_inference	bool	Indicates support for CRV3 inference.
_data	Any	Data used in the regression, to be enriched outside of the class.
_fml	Any	Formula used in the regression, to be enriched outside of the class.
_has_fixef	bool	Indicates whether fixed effects are used.
_fixef	Any	Fixed effects used in the regression.
_icovars	Any	Internal covariates, to be enriched outside of the class.
_ssc_dict	dict	dictionary for sum of squares and cross products matrices.
_tZX	np.ndarray	Transpose of Z multiplied by X, set in get_fit().
_tXZ	np.ndarray	Transpose of X multiplied by Z, set in get_fit().
_tZy	np.ndarray	Transpose of Z multiplied by Y, set in get_fit().
_tZZinv	np.ndarray	Inverse of the transpose of Z multiplied by Z, set in get_fit().
_beta_hat	np.ndarray	Estimated regression coefficients.
_Y_hat_link	np.ndarray	Prediction at the level of the explanatory variable, i.e., the linear predictor X @ beta.
_Y_hat_response	np.ndarray	Prediction at the level of the response variable, i.e., the expected predictor E(Y|X).
_u_hat	np.ndarray	Residuals of the regression model.
_scores	np.ndarray	Scores used in the regression analysis.
_hessian	np.ndarray	Hessian matrix used in the regression.
_bread	np.ndarray	Bread matrix, used in calculating the variance-covariance matrix.
_vcov_type	Any	Type of variance-covariance matrix used.
_vcov_type_detail	Any	Detailed specification of the variance-covariance matrix type.
_is_clustered	bool	Indicates if clustering is used in the variance-covariance calculation.
_clustervar	Any	Variable used for clustering in the variance-covariance calculation.
_G	Any	Group information used in clustering.
_ssc	Any	Sum of squares and cross products matrix.
_vcov	np.ndarray	Variance-covariance matrix of the estimated coefficients.
_se	np.ndarray	Standard errors of the estimated coefficients.
_tstat	np.ndarray	T-statistics of the estimated coefficients.
_pvalue	np.ndarray	P-values associated with the t-statistics.
_conf_int	np.ndarray	Confidence intervals for the estimated coefficients.
_F_stat	Any	F-statistic for the model, set in get_Ftest().
_fixef_dict	dict	dictionary containing fixed effects estimates.
_sumFE	np.ndarray	Sum of all fixed effects for each observation.
_rmse	float	Root mean squared error of the model.
_r2	float	R-squared value of the model.
_r2_within	float	R-squared value computed on demeaned dependent variable.
_adj_r2	float	Adjusted R-squared value of the model.
_adj_r2_within	float	Adjusted R-squared value computed on demeaned dependent variable.
_solver	str	The solver used to fit the normal equation.
_data	pd.DataFrame	The data frame used in the estimation. None if arguments lean = True or store_data = False.
_model_name	str	The name of the model. Usually just the formula string. If split estimation is used, the model name will include the split variable and value.
_model_name_plot	str	The name of the model used when plotting and summarizing models. Usually identical to _model_name. This might be different when pf.summary() or pf.coefplot() are called and models with identical _model_name attributes are passed. In this case, the _model_name_plot attribute will be modified.

Methods

Name	Description
add_fixest_multi_context	Enrich Feols object.
ccv	Compute the Causal Cluster Variance following Abadie et al (QJE 2023).
coef	Fitted model coefficents.
confint	Fitted model confidence intervals.
decompose	Implement the Gelbach (2016) decomposition method for mediation analysis.
demean	Demean the dependent variable and covariates by the fixed effect(s).
drop_multicol_vars	Detect and drop multicollinear variables.
fixef	Compute the coefficients of (swept out) fixed effects for a regression model.
get_fit	Fit an OLS model.
get_inference	Compute standard errors, t-statistics, and p-values for the regression model.
get_performance	Get Goodness-of-Fit measures.
plot_ritest	Plot the distribution of the Randomization Inference Statistics.
predict	Predict values of the model on new data.
prepare_model_matrix	Prepare model matrices for estimation.
pvalue	Fitted model p-values.
resid	Fitted model residuals.
ritest	Conduct Randomization Inference (RI) test against a null hypothesis of
se	Fitted model standard errors.
solve_ols	Solve the ordinary least squares problem using the specified solver.
tidy	Tidy model outputs.
to_array	Convert estimation data frames to np arrays.
tstat	Fitted model t-statistics.
update	Update coefficients for new observations using Sherman-Morrison formula.
vcov	Compute covariance matrices for an estimated regression model.
wald_test	Conduct Wald test.
wildboottest	Run a wild cluster bootstrap based on an object of type “Feols”.
wls_transform	Transform model matrices for WLS Estimation.

add_fixest_multi_context

estimation.feols_.Feols.add_fixest_multi_context(
    depvar,
    Y,
    _data,
    _ssc_dict,
    _k_fe,
    fval,
    store_data,
)

Enrich Feols object.

Enrich an instance of Feols Class with additional attributes set in the FixestMulti class.

Parameters

Name	Type	Description	Default
FixestFormula	FixestFormula	The formula(s) used for estimation encoded in a FixestFormula object.	required
depvar	str	The dependent variable of the regression model.	required
Y	pd.Series	The dependent variable of the regression model.	required
_data	pd.DataFrame	The data used for estimation.	required
_ssc_dict	dict	A dictionary with the sum of squares and cross products matrices.	required
_k_fe	int	The number of fixed effects.	required
fval	str	The fixed effects formula.	required
store_data	bool	Indicates whether to save the data used for estimation in the object	required

Returns

Name	Type	Description
	None

ccv

estimation.feols_.Feols.ccv(
    treatment,
    cluster=None,
    seed=None,
    n_splits=8,
    pk=1,
    qk=1,
)

Compute the Causal Cluster Variance following Abadie et al (QJE 2023).

Parameters

Name	Type	Description	Default
treatment		The name of the treatment variable.	required
cluster	str	The name of the cluster variable. None by default. If None, uses the cluster variable from the model fit.	None
seed	int	An integer to set the random seed. Defaults to None.	None
n_splits	int	The number of splits to use in the cross-fitting procedure. Defaults to 8.	8
pk	float	The proportion of sampled clusters. Defaults to 1, which corresponds to all clusters of the population being sampled.	1
qk	float	The proportion of sampled observations within each cluster. Defaults to 1, which corresponds to all observations within each cluster being sampled.	1

Returns

Name	Type	Description
	pd.DataFrame	A DataFrame with inference based on the “Causal Cluster Variance” and “regular” CRV1 inference.

Examples

import pyfixest as pf
import numpy as np

data = pf.get_data()
data["D"] = np.random.choice([0, 1], size=data.shape[0])

fit = pf.feols("Y ~ D", data=data, vcov={"CRV1": "group_id"})
fit.ccv(treatment="D", pk=0.05, qk=0.5, n_splits=8, seed=123).head()

	Estimate	Std. Error	t value	Pr(>|t|)	2.5%	97.5%
CCV	-0.07963029165911072	0.232823	-0.342021	0.736296	-0.568773	0.409512
CRV1	-0.07963	0.150083	-0.530576	0.602198	-0.394943	0.235682

coef

estimation.feols_.Feols.coef()

Fitted model coefficents.

Returns

Name	Type	Description
	pd.Series	A pd.Series with the estimated coefficients of the regression model.

confint

estimation.feols_.Feols.confint(
    alpha=0.05,
    keep=None,
    drop=None,
    exact_match=False,
    joint=False,
    seed=None,
    reps=10000,
)

Fitted model confidence intervals.

Parameters

Name	Type	Description	Default
alpha	float	The significance level for confidence intervals. Defaults to 0.05. keep: str or list of str, optional	0.05
joint	bool	Whether to compute simultaneous confidence interval for joint null of parameters selected by keep and drop. Defaults to False. See https://www.causalml-book.org/assets/chapters/CausalML_chap_4.pdf, Remark 4.4.1 for details.	False
keep	Optional[Union[list, str]]	The pattern for retaining coefficient names. You can pass a string (one pattern) or a list (multiple patterns). Default is keeping all coefficients. You should use regular expressions to select coefficients. “age”, # would keep all coefficients containing age r”^tr”, # would keep all coefficients starting with tr r”\d$“, # would keep all coefficients ending with number Output will be in the order of the patterns.	None
drop	Optional[Union[list, str]]	The pattern for excluding coefficient names. You can pass a string (one pattern) or a list (multiple patterns). Syntax is the same as for keep. Default is keeping all coefficients. Parameter keep and drop can be used simultaneously.	None
exact_match	Optional[bool]	Whether to use exact match for keep and drop. Default is False. If True, the pattern will be matched exactly to the coefficient name instead of using regular expressions.	False
reps	int	The number of bootstrap iterations to run for joint confidence intervals. Defaults to 10_000. Only used if joint is True.	10000
seed	int	The seed for the random number generator. Defaults to None. Only used if joint is True.	None

Returns

Name	Type	Description
	pd.DataFrame	A pd.DataFrame with confidence intervals of the estimated regression model for the selected coefficients.

Examples

from pyfixest.utils import get_data
from pyfixest.estimation import feols

data = get_data()
fit = feols("Y ~ C(f1)", data=data)
fit.confint(alpha=0.10).head()
fit.confint(alpha=0.10, joint=True, reps=9999).head()

	5.0%	95.0%
Intercept	-1.364001	0.390283
C(f1)[T.1.0]	1.361550	3.727026
C(f1)[T.2.0]	-2.875147	-0.394215
C(f1)[T.3.0]	-1.742624	0.814663
C(f1)[T.4.0]	-3.012007	-0.513417

decompose

estimation.feols_.Feols.decompose(
    param,
    type='gelbach',
    cluster=None,
    combine_covariates=None,
    reps=1000,
    seed=None,
    nthreads=None,
    agg_first=None,
    only_coef=False,
    digits=4,
)

Implement the Gelbach (2016) decomposition method for mediation analysis.

Compares a short model depvar on param with the long model specified in the original feols() call.

For details, take a look at “When do covariates matter?” by Gelbach (2016, JoLe). You can find an ungated version of the paper on SSRN under the following link: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1425737 .

Parameters

Name	Type	Description	Default
param	str	The name of the focal covariate whose effect is to be decomposed into direct and indirect components with respect to the rest of the right-hand side.	required
type	str	The type of decomposition method to use. Defaults to “gelbach”, which currently is the only supported option.	'gelbach'
cluster	Optional[str]	The name of the cluster variable. If None, uses the cluster variable from the model fit. Defaults to None.	None
combine_covariates	Optional[dict[str, list[str]]]	A dictionary that specifies which covariates to combine into groups. See the example for how to use this argument. Defaults to None.	None
reps	int	The number of bootstrap iterations to run. Defaults to 1000.	1000
seed	int	An integer to set the random seed. Defaults to None.	None
nthreads	int	The number of threads to use for the bootstrap. Defaults to None. If None, uses all available threads minus one.	None
agg_first	bool	If True, use the ‘aggregate first’ algorithm described in Gelbach (2016). Recommended in cases with many (potentially high-dimensional) covariates. False by default if the ‘combine_covariates’ argument is None, True otherwise.	None
only_coef	bool	Indicates whether to compute inference for the decomposition. Defaults to False. If True, skips the inference step and only returns the decomposition results.	False
digits	int	The number of digits to round the results to. Defaults to 4.	4

Returns

Name	Type	Description
	pd.DataFrame	A DataFrame with the decomposition results.

Examples

import re
import pyfixest as pf
from pyfixest.utils.dgps import gelbach_data

data = gelbach_data(nobs = 1000)
fit = pf.feols("y ~ x1 + x21 + x22 + x23", data=data)

# simple decomposition
res = fit.decompose(param = "x1")
pf.make_table(res)

# group covariates via "combine_covariates" argument
res = fit.decompose(param = "x1", combine_covariates={"g1": ["x21", "x22"], "g2": ["x23"]})
pf.make_table(res)

# group covariates via regex
res = fit.decompose(param="x1", combine_covariates={"g1": re.compile("x2[1-2]"), "g2": re.compile("x23")})

  0%|          | 0/1000 [00:00<?, ?it/s]/home/runner/work/pyfixest/pyfixest/.pixi/envs/docs/lib/python3.12/site-packages/joblib/externals/loky/backend/fork_exec.py:38: RuntimeWarning: Using fork() can cause Polars to deadlock in the child process.
In addition, using fork() with Python in general is a recipe for mysterious
deadlocks and crashes.

The most likely reason you are seeing this error is because you are using the
multiprocessing module on Linux, which uses fork() by default. This will be
fixed in Python 3.14. Until then, you want to use the "spawn" context instead.

See https://docs.pola.rs/user-guide/misc/multiprocessing/ for details.

If you really know what your doing, you can silence this warning with the warning module
or by setting POLARS_ALLOW_FORKING_THREAD=1.

  pid = os.fork()
  1%|          | 8/1000 [00:02<04:59,  3.32it/s]  1%|          | 12/1000 [00:02<03:04,  5.35it/s] 11%|█         | 112/1000 [00:02<00:11, 77.94it/s] 40%|████      | 400/1000 [00:02<00:01, 332.83it/s] 78%|███████▊  | 784/1000 [00:02<00:00, 681.13it/s]100%|██████████| 1000/1000 [00:02<00:00, 340.16it/s]
  0%|          | 0/1000 [00:00<?, ?it/s] 18%|█▊        | 184/1000 [00:00<00:00, 1547.85it/s] 38%|███▊      | 376/1000 [00:00<00:00, 1589.03it/s] 54%|█████▎    | 535/1000 [00:00<00:00, 1511.29it/s] 76%|███████▌  | 760/1000 [00:00<00:00, 1689.13it/s]100%|██████████| 1000/1000 [00:00<00:00, 2138.01it/s]
  0%|          | 0/1000 [00:00<?, ?it/s] 18%|█▊        | 184/1000 [00:00<00:00, 1531.16it/s] 38%|███▊      | 376/1000 [00:00<00:00, 1600.38it/s] 54%|█████▎    | 537/1000 [00:00<00:00, 1522.68it/s] 76%|███████▌  | 760/1000 [00:00<00:00, 1694.35it/s]100%|██████████| 1000/1000 [00:00<00:00, 2146.28it/s]

demean

estimation.feols_.Feols.demean()

Demean the dependent variable and covariates by the fixed effect(s).

drop_multicol_vars

estimation.feols_.Feols.drop_multicol_vars()

Detect and drop multicollinear variables.

fixef

estimation.feols_.Feols.fixef(atol=1e-06, btol=1e-06)

Compute the coefficients of (swept out) fixed effects for a regression model.

This method creates the following attributes: - alphaDF (pd.DataFrame): A DataFrame with the estimated fixed effects. - sumFE (np.array): An array with the sum of fixed effects for each observation (i = 1, …, N).

Returns

Name	Type	Description
	None

get_fit

estimation.feols_.Feols.get_fit()

Fit an OLS model.

Returns

Name	Type	Description
	None

get_inference

estimation.feols_.Feols.get_inference(alpha=0.05)

Compute standard errors, t-statistics, and p-values for the regression model.

Parameters

Name	Type	Description	Default
alpha	float	The significance level for confidence intervals. Defaults to 0.05, which produces a 95% confidence interval.	0.05

Returns

Name	Type	Description
	None

get_performance

estimation.feols_.Feols.get_performance()

Get Goodness-of-Fit measures.

Compute multiple additional measures commonly reported with linear regression output, including R-squared and adjusted R-squared. Note that variables with the suffix _within use demeaned dependent variables Y, while variables without do not or are invariant to demeaning.

Returns

Name	Type	Description
	None
	Creates the following instances:
	- r2 (float): R-squared of the regression model.
	- adj_r2 (float): Adjusted R-squared of the regression model.
	- r2_within (float): R-squared of the regression model, computed on
	demeaned dependent variable.
	- adj_r2_within (float): Adjusted R-squared of the regression model,
	computed on demeaned dependent variable.

plot_ritest

estimation.feols_.Feols.plot_ritest(plot_backend='lets_plot')

Plot the distribution of the Randomization Inference Statistics.

Parameters

Name	Type	Description	Default
plot_backend	str	The plotting backend to use. Defaults to “lets_plot”. Alternatively, “matplotlib” is available.	'lets_plot'

Returns

Name	Type	Description
	A lets_plot or matplotlib figure with the distribution of the Randomization
	Inference Statistics.

predict

estimation.feols_.Feols.predict(
    newdata=None,
    atol=1e-06,
    btol=1e-06,
    type='link',
)

Predict values of the model on new data.

Return a flat np.array with predicted values of the regression model. If new fixed effect levels are introduced in newdata, predicted values for such observations will be set to NaN.

Parameters

Name	Type	Description	Default
newdata	Optional[DataFrameType]	A narwhals compatible DataFrame (polars, pandas, duckdb, etc). If None (default), the data used for fitting the model is used.	None
type	str	The type of prediction to be computed. Can be either “response” (default) or “link”. For linear models, both are identical.	'link'
atol	Float	Stopping tolerance for scipy.sparse.linalg.lsqr(). See https://docs.scipy.org/doc/ scipy/reference/generated/scipy.sparse.linalg.lsqr.html	1e-6
btol	Float	Another stopping tolerance for scipy.sparse.linalg.lsqr(). See https://docs.scipy.org/doc/ scipy/reference/generated/scipy.sparse.linalg.lsqr.html	1e-6
link		The type of prediction to be made. Can be either ‘link’ or ‘response’. Defaults to ‘link’. ‘link’ and ‘response’ lead to identical results for linear models.	required

Returns

Name	Type	Description
y_hat	np.ndarray	A flat np.array with predicted values of the regression model.

prepare_model_matrix

estimation.feols_.Feols.prepare_model_matrix()

Prepare model matrices for estimation.

pvalue

estimation.feols_.Feols.pvalue()

Fitted model p-values.

Returns

Name	Type	Description
	pd.Series	A pd.Series with p-values of the estimated regression model.

resid

estimation.feols_.Feols.resid()

Fitted model residuals.

Returns

Name	Type	Description
	np.ndarray	A np.ndarray with the residuals of the estimated regression model.

ritest

estimation.feols_.Feols.ritest(
    resampvar,
    cluster=None,
    reps=100,
    type='randomization-c',
    rng=None,
    choose_algorithm='auto',
    store_ritest_statistics=False,
    level=0.95,
)

Conduct Randomization Inference (RI) test against a null hypothesis of resampvar = 0.

Parameters

Name	Type	Description	Default
resampvar	str	The name of the variable to be resampled.	required
cluster	str	The name of the cluster variable in case of cluster random assignment. If provided, resampvar is held constant within each cluster. Defaults to None.	None
reps	int	The number of randomization iterations. Defaults to 100.	100
type	str	The type of the randomization inference test. Can be “randomization-c” or “randomization-t”. Note that the “randomization-c” is much faster, while the “randomization-t” is recommended by Wu & Ding (JASA, 2021).	'randomization-c'
rng	np.random.Generator	A random number generator. Defaults to None.	None
choose_algorithm	str	The algorithm to use for the computation. Defaults to “auto”. The alternative is “fast” and “slow”, and should only be used for running CI tests. Ironically, this argument is not tested for any input errors from the user! So please don’t use it =)	'auto'
include_plot		Whether to include a plot of the distribution p-values. Defaults to False.	required
store_ritest_statistics	bool	Whether to store the simulated statistics of the RI procedure. Defaults to False. If True, stores the simulated statistics in the model object via the ritest_statistics attribute as a numpy array.	False
level	float	The level for the confidence interval of the randomization inference p-value. Defaults to 0.95.	0.95

Returns

Name	Type	Description
	A pd.Series with the regression coefficient of resampvar and the p-value
	of the RI test. Additionally, reports the standard error and the confidence
	interval of the p-value.

Examples

import pyfixest as pf
data = pf.get_data()
fit = pf.feols("Y ~ X1 + X2", data=data)

# Conduct a randomization inference test for the coefficient of X1
fit.ritest("X1", reps=1000)

# use randomization-t instead of randomization-c
fit.ritest("X1", reps=1000, type="randomization-t")

# store statistics for plotting
fit.ritest("X1", reps=1000, store_ritest_statistics=True)

  0%|          | 0/1000 [00:00<?, ?it/s]  0%|          | 1/1000 [00:01<23:05,  1.39s/it]  0%|          | 2/1000 [00:01<10:33,  1.57it/s]  0%|          | 3/1000 [00:01<06:30,  2.55it/s]  0%|          | 5/1000 [00:01<03:44,  4.42it/s]  1%|          | 7/1000 [00:01<02:46,  5.95it/s]  1%|          | 8/1000 [00:02<02:30,  6.58it/s]  1%|          | 9/1000 [00:02<02:18,  7.15it/s]  1%|          | 11/1000 [00:02<02:00,  8.22it/s]  1%|▏         | 13/1000 [00:02<01:50,  8.96it/s]  2%|▏         | 15/1000 [00:02<01:44,  9.39it/s]  2%|▏         | 17/1000 [00:03<01:48,  9.03it/s]  2%|▏         | 18/1000 [00:03<01:49,  8.98it/s]  2%|▏         | 19/1000 [00:03<01:50,  8.89it/s]  2%|▏         | 20/1000 [00:03<01:49,  8.94it/s]  2%|▏         | 22/1000 [00:03<01:44,  9.40it/s]  2%|▏         | 23/1000 [00:03<01:43,  9.41it/s]  2%|▏         | 24/1000 [00:03<01:43,  9.46it/s]  3%|▎         | 26/1000 [00:03<01:40,  9.67it/s]  3%|▎         | 27/1000 [00:04<01:43,  9.42it/s]  3%|▎         | 28/1000 [00:04<01:43,  9.37it/s]  3%|▎         | 29/1000 [00:04<01:43,  9.35it/s]  3%|▎         | 30/1000 [00:04<01:43,  9.40it/s]  3%|▎         | 31/1000 [00:04<01:41,  9.56it/s]  3%|▎         | 33/1000 [00:04<01:37,  9.97it/s]  4%|▎         | 35/1000 [00:04<01:34, 10.19it/s]  4%|▎         | 37/1000 [00:05<01:34, 10.17it/s]  4%|▍         | 39/1000 [00:05<01:36,  9.98it/s]  4%|▍         | 40/1000 [00:05<01:36,  9.93it/s]  4%|▍         | 42/1000 [00:05<01:35, 10.06it/s]  4%|▍         | 44/1000 [00:05<01:35, 10.06it/s]  5%|▍         | 46/1000 [00:05<01:34, 10.13it/s]  5%|▍         | 48/1000 [00:06<01:33, 10.20it/s]  5%|▌         | 50/1000 [00:06<01:33, 10.17it/s]  5%|▌         | 52/1000 [00:06<01:33, 10.19it/s]  5%|▌         | 54/1000 [00:06<01:33, 10.15it/s]  6%|▌         | 56/1000 [00:06<01:32, 10.24it/s]  6%|▌         | 58/1000 [00:07<01:32, 10.23it/s]  6%|▌         | 60/1000 [00:07<01:31, 10.28it/s]  6%|▌         | 62/1000 [00:07<01:31, 10.20it/s]  6%|▋         | 64/1000 [00:07<01:31, 10.28it/s]  7%|▋         | 66/1000 [00:07<01:30, 10.34it/s]  7%|▋         | 68/1000 [00:08<01:30, 10.31it/s]  7%|▋         | 70/1000 [00:08<01:30, 10.28it/s]  7%|▋         | 72/1000 [00:08<01:31, 10.10it/s]  7%|▋         | 74/1000 [00:08<01:30, 10.24it/s]  8%|▊         | 76/1000 [00:08<01:30, 10.16it/s]  8%|▊         | 78/1000 [00:09<01:30, 10.24it/s]  8%|▊         | 80/1000 [00:09<01:29, 10.22it/s]  8%|▊         | 82/1000 [00:09<01:30, 10.12it/s]  8%|▊         | 84/1000 [00:09<01:29, 10.24it/s]  9%|▊         | 86/1000 [00:09<01:28, 10.32it/s]  9%|▉         | 88/1000 [00:10<01:28, 10.34it/s]  9%|▉         | 90/1000 [00:10<01:28, 10.33it/s]  9%|▉         | 92/1000 [00:10<01:29, 10.13it/s]  9%|▉         | 94/1000 [00:10<01:30,  9.99it/s] 10%|▉         | 96/1000 [00:10<01:29, 10.12it/s] 10%|▉         | 98/1000 [00:11<01:28, 10.17it/s] 10%|█         | 100/1000 [00:11<01:28, 10.12it/s] 10%|█         | 102/1000 [00:11<01:28, 10.14it/s] 10%|█         | 104/1000 [00:11<01:28, 10.11it/s] 11%|█         | 106/1000 [00:11<01:28, 10.13it/s] 11%|█         | 108/1000 [00:12<01:27, 10.21it/s] 11%|█         | 110/1000 [00:12<01:27, 10.11it/s] 11%|█         | 112/1000 [00:12<01:27, 10.11it/s] 11%|█▏        | 114/1000 [00:12<01:29,  9.95it/s] 12%|█▏        | 116/1000 [00:12<01:27, 10.12it/s] 12%|█▏        | 118/1000 [00:13<01:27, 10.12it/s] 12%|█▏        | 120/1000 [00:13<01:27, 10.07it/s] 12%|█▏        | 122/1000 [00:13<01:27, 10.05it/s] 12%|█▏        | 124/1000 [00:13<01:28,  9.95it/s] 12%|█▎        | 125/1000 [00:13<01:28,  9.86it/s] 13%|█▎        | 126/1000 [00:13<01:29,  9.76it/s] 13%|█▎        | 128/1000 [00:14<01:28,  9.88it/s] 13%|█▎        | 129/1000 [00:14<01:28,  9.84it/s] 13%|█▎        | 130/1000 [00:14<01:28,  9.87it/s] 13%|█▎        | 132/1000 [00:14<01:27,  9.98it/s] 13%|█▎        | 134/1000 [00:14<01:26, 10.00it/s] 14%|█▎        | 136/1000 [00:14<01:25, 10.11it/s] 14%|█▍        | 138/1000 [00:15<01:24, 10.22it/s] 14%|█▍        | 140/1000 [00:15<01:24, 10.13it/s] 14%|█▍        | 142/1000 [00:15<01:24, 10.14it/s] 14%|█▍        | 144/1000 [00:15<01:25, 10.00it/s] 15%|█▍        | 146/1000 [00:15<01:24, 10.11it/s] 15%|█▍        | 148/1000 [00:16<01:24, 10.12it/s] 15%|█▌        | 150/1000 [00:16<01:24, 10.12it/s] 15%|█▌        | 152/1000 [00:16<01:23, 10.12it/s] 15%|█▌        | 154/1000 [00:16<01:25,  9.84it/s] 16%|█▌        | 155/1000 [00:16<01:26,  9.82it/s] 16%|█▌        | 157/1000 [00:16<01:23, 10.04it/s] 16%|█▌        | 159/1000 [00:17<01:23, 10.06it/s] 16%|█▌        | 161/1000 [00:17<01:22, 10.13it/s] 16%|█▋        | 163/1000 [00:17<01:22, 10.13it/s] 16%|█▋        | 165/1000 [00:17<01:22, 10.13it/s] 17%|█▋        | 167/1000 [00:17<01:23,  9.99it/s] 17%|█▋        | 168/1000 [00:18<01:23,  9.95it/s] 17%|█▋        | 169/1000 [00:18<01:23,  9.91it/s] 17%|█▋        | 170/1000 [00:18<01:23,  9.90it/s] 17%|█▋        | 171/1000 [00:18<01:24,  9.85it/s] 17%|█▋        | 172/1000 [00:18<01:23,  9.87it/s] 17%|█▋        | 173/1000 [00:18<01:24,  9.80it/s] 17%|█▋        | 174/1000 [00:18<01:23,  9.85it/s] 18%|█▊        | 175/1000 [00:18<01:25,  9.59it/s] 18%|█▊        | 177/1000 [00:18<01:23,  9.90it/s] 18%|█▊        | 178/1000 [00:19<01:23,  9.89it/s] 18%|█▊        | 180/1000 [00:19<01:21, 10.01it/s] 18%|█▊        | 182/1000 [00:19<01:21, 10.08it/s] 18%|█▊        | 184/1000 [00:19<01:20, 10.15it/s] 19%|█▊        | 186/1000 [00:19<01:21,  9.98it/s] 19%|█▉        | 188/1000 [00:20<01:20, 10.11it/s] 19%|█▉        | 190/1000 [00:20<01:20, 10.05it/s] 19%|█▉        | 192/1000 [00:20<01:20, 10.03it/s] 19%|█▉        | 194/1000 [00:20<01:20, 10.07it/s] 20%|█▉        | 196/1000 [00:20<01:21,  9.87it/s] 20%|█▉        | 197/1000 [00:20<01:22,  9.78it/s] 20%|█▉        | 198/1000 [00:21<01:22,  9.72it/s] 20%|█▉        | 199/1000 [00:21<01:22,  9.73it/s] 20%|██        | 200/1000 [00:21<01:21,  9.78it/s] 20%|██        | 202/1000 [00:21<01:19,  9.99it/s] 20%|██        | 204/1000 [00:21<01:18, 10.11it/s] 21%|██        | 206/1000 [00:21<01:20,  9.82it/s] 21%|██        | 208/1000 [00:22<01:19,  9.93it/s] 21%|██        | 209/1000 [00:22<01:20,  9.86it/s] 21%|██        | 210/1000 [00:22<01:20,  9.80it/s] 21%|██        | 211/1000 [00:22<01:20,  9.81it/s] 21%|██        | 212/1000 [00:22<01:25,  9.27it/s] 21%|██▏       | 213/1000 [00:22<01:24,  9.37it/s] 21%|██▏       | 214/1000 [00:22<01:23,  9.38it/s] 22%|██▏       | 216/1000 [00:22<01:20,  9.68it/s] 22%|██▏       | 218/1000 [00:23<01:18,  9.95it/s] 22%|██▏       | 219/1000 [00:23<01:19,  9.88it/s] 22%|██▏       | 221/1000 [00:23<01:17, 10.00it/s] 22%|██▏       | 222/1000 [00:23<01:17,  9.98it/s] 22%|██▏       | 224/1000 [00:23<01:16, 10.11it/s] 23%|██▎       | 226/1000 [00:23<01:17,  9.99it/s] 23%|██▎       | 227/1000 [00:24<01:17,  9.95it/s] 23%|██▎       | 229/1000 [00:24<01:16, 10.08it/s] 23%|██▎       | 231/1000 [00:24<01:15, 10.19it/s] 23%|██▎       | 233/1000 [00:24<01:15, 10.18it/s] 24%|██▎       | 235/1000 [00:24<01:14, 10.25it/s] 24%|██▎       | 237/1000 [00:24<01:14, 10.21it/s] 24%|██▍       | 239/1000 [00:25<01:15, 10.03it/s] 24%|██▍       | 241/1000 [00:25<01:15, 10.06it/s] 24%|██▍       | 243/1000 [00:25<01:14, 10.12it/s] 24%|██▍       | 245/1000 [00:25<01:14, 10.14it/s] 25%|██▍       | 247/1000 [00:25<01:14, 10.04it/s] 25%|██▍       | 249/1000 [00:26<01:15,  9.96it/s] 25%|██▌       | 251/1000 [00:26<01:14, 10.04it/s] 25%|██▌       | 253/1000 [00:26<01:14, 10.06it/s] 26%|██▌       | 255/1000 [00:26<01:14, 10.02it/s] 26%|██▌       | 257/1000 [00:26<01:14,  9.94it/s] 26%|██▌       | 258/1000 [00:27<01:14,  9.93it/s] 26%|██▌       | 259/1000 [00:27<01:14,  9.91it/s] 26%|██▌       | 260/1000 [00:27<01:14,  9.92it/s] 26%|██▌       | 261/1000 [00:27<01:17,  9.50it/s] 26%|██▌       | 262/1000 [00:27<01:20,  9.13it/s] 26%|██▋       | 263/1000 [00:27<01:22,  8.95it/s] 26%|██▋       | 264/1000 [00:27<01:23,  8.84it/s] 26%|██▋       | 265/1000 [00:27<01:23,  8.77it/s] 27%|██▋       | 267/1000 [00:28<01:20,  9.13it/s] 27%|██▋       | 268/1000 [00:28<01:19,  9.23it/s] 27%|██▋       | 270/1000 [00:28<01:16,  9.51it/s] 27%|██▋       | 272/1000 [00:28<01:14,  9.74it/s] 27%|██▋       | 273/1000 [00:28<01:14,  9.75it/s] 28%|██▊       | 275/1000 [00:28<01:12,  9.94it/s] 28%|██▊       | 277/1000 [00:29<01:12,  9.91it/s] 28%|██▊       | 278/1000 [00:29<01:13,  9.87it/s] 28%|██▊       | 279/1000 [00:29<01:13,  9.87it/s] 28%|██▊       | 280/1000 [00:29<01:12,  9.88it/s] 28%|██▊       | 282/1000 [00:29<01:12,  9.95it/s] 28%|██▊       | 283/1000 [00:29<01:12,  9.95it/s] 28%|██▊       | 285/1000 [00:29<01:11, 10.04it/s] 29%|██▊       | 287/1000 [00:30<01:10, 10.08it/s] 29%|██▉       | 289/1000 [00:30<01:15,  9.48it/s] 29%|██▉       | 290/1000 [00:30<01:16,  9.26it/s] 29%|██▉       | 291/1000 [00:30<01:17,  9.19it/s] 29%|██▉       | 292/1000 [00:30<01:16,  9.23it/s] 29%|██▉       | 293/1000 [00:30<01:18,  8.97it/s] 29%|██▉       | 294/1000 [00:30<01:18,  9.04it/s] 30%|██▉       | 295/1000 [00:30<01:16,  9.27it/s] 30%|██▉       | 297/1000 [00:31<01:14,  9.45it/s] 30%|██▉       | 298/1000 [00:31<01:13,  9.56it/s] 30%|██▉       | 299/1000 [00:31<01:12,  9.63it/s] 30%|███       | 301/1000 [00:31<01:10,  9.85it/s] 30%|███       | 302/1000 [00:31<01:12,  9.68it/s] 30%|███       | 303/1000 [00:31<01:12,  9.66it/s] 30%|███       | 305/1000 [00:32<01:10,  9.85it/s] 31%|███       | 307/1000 [00:32<01:11,  9.65it/s] 31%|███       | 308/1000 [00:32<01:12,  9.59it/s] 31%|███       | 309/1000 [00:32<01:11,  9.62it/s] 31%|███       | 310/1000 [00:32<01:11,  9.70it/s] 31%|███       | 311/1000 [00:32<01:10,  9.75it/s] 31%|███       | 312/1000 [00:32<01:10,  9.76it/s] 31%|███▏      | 314/1000 [00:32<01:09,  9.90it/s] 32%|███▏      | 315/1000 [00:33<01:09,  9.90it/s] 32%|███▏      | 316/1000 [00:33<01:09,  9.88it/s] 32%|███▏      | 317/1000 [00:33<01:10,  9.71it/s] 32%|███▏      | 318/1000 [00:33<01:09,  9.77it/s] 32%|███▏      | 319/1000 [00:33<01:09,  9.78it/s] 32%|███▏      | 320/1000 [00:33<01:09,  9.81it/s] 32%|███▏      | 321/1000 [00:33<01:09,  9.82it/s] 32%|███▏      | 322/1000 [00:33<01:10,  9.65it/s] 32%|███▏      | 324/1000 [00:33<01:08,  9.86it/s] 32%|███▎      | 325/1000 [00:34<01:08,  9.86it/s] 33%|███▎      | 326/1000 [00:34<01:09,  9.76it/s] 33%|███▎      | 327/1000 [00:34<01:10,  9.59it/s] 33%|███▎      | 329/1000 [00:34<01:09,  9.66it/s] 33%|███▎      | 331/1000 [00:34<01:07,  9.86it/s] 33%|███▎      | 333/1000 [00:34<01:06, 10.00it/s] 34%|███▎      | 335/1000 [00:35<01:07,  9.86it/s] 34%|███▎      | 336/1000 [00:35<01:07,  9.80it/s] 34%|███▎      | 337/1000 [00:35<01:10,  9.45it/s] 34%|███▍      | 338/1000 [00:35<01:10,  9.42it/s] 34%|███▍      | 339/1000 [00:35<01:09,  9.52it/s] 34%|███▍      | 340/1000 [00:35<01:08,  9.60it/s] 34%|███▍      | 342/1000 [00:35<01:07,  9.74it/s] 34%|███▍      | 344/1000 [00:36<01:06,  9.90it/s] 34%|███▍      | 345/1000 [00:36<01:06,  9.89it/s] 35%|███▍      | 346/1000 [00:36<01:06,  9.89it/s] 35%|███▍      | 347/1000 [00:36<01:07,  9.66it/s] 35%|███▍      | 348/1000 [00:36<01:07,  9.66it/s] 35%|███▍      | 349/1000 [00:36<01:07,  9.59it/s] 35%|███▌      | 350/1000 [00:36<01:07,  9.62it/s] 35%|███▌      | 351/1000 [00:36<01:06,  9.71it/s] 35%|███▌      | 352/1000 [00:36<01:06,  9.74it/s] 35%|███▌      | 354/1000 [00:37<01:04,  9.98it/s] 36%|███▌      | 355/1000 [00:37<01:05,  9.86it/s] 36%|███▌      | 356/1000 [00:37<01:05,  9.86it/s] 36%|███▌      | 357/1000 [00:37<01:06,  9.67it/s] 36%|███▌      | 358/1000 [00:37<01:06,  9.70it/s] 36%|███▌      | 360/1000 [00:37<01:04,  9.90it/s] 36%|███▌      | 362/1000 [00:37<01:03, 10.00it/s] 36%|███▋      | 363/1000 [00:37<01:04,  9.88it/s] 36%|███▋      | 364/1000 [00:38<01:04,  9.89it/s] 36%|███▋      | 365/1000 [00:38<01:04,  9.79it/s] 37%|███▋      | 366/1000 [00:38<01:04,  9.79it/s] 37%|███▋      | 367/1000 [00:38<01:04,  9.79it/s] 37%|███▋      | 368/1000 [00:38<01:06,  9.53it/s] 37%|███▋      | 370/1000 [00:38<01:04,  9.77it/s] 37%|███▋      | 372/1000 [00:38<01:03,  9.92it/s] 37%|███▋      | 374/1000 [00:39<01:02, 10.03it/s] 38%|███▊      | 375/1000 [00:39<01:03,  9.83it/s] 38%|███▊      | 376/1000 [00:39<01:03,  9.82it/s] 38%|███▊      | 377/1000 [00:39<01:03,  9.80it/s] 38%|███▊      | 378/1000 [00:39<01:05,  9.57it/s] 38%|███▊      | 379/1000 [00:39<01:04,  9.56it/s] 38%|███▊      | 380/1000 [00:39<01:04,  9.62it/s] 38%|███▊      | 381/1000 [00:39<01:04,  9.59it/s] 38%|███▊      | 382/1000 [00:39<01:03,  9.70it/s] 38%|███▊      | 383/1000 [00:39<01:03,  9.74it/s] 38%|███▊      | 384/1000 [00:40<01:03,  9.77it/s] 38%|███▊      | 385/1000 [00:40<01:03,  9.73it/s] 39%|███▊      | 386/1000 [00:40<01:04,  9.46it/s] 39%|███▊      | 387/1000 [00:40<01:05,  9.31it/s] 39%|███▉      | 388/1000 [00:40<01:06,  9.18it/s] 39%|███▉      | 389/1000 [00:40<01:06,  9.19it/s] 39%|███▉      | 390/1000 [00:40<01:04,  9.39it/s] 39%|███▉      | 391/1000 [00:40<01:03,  9.52it/s] 39%|███▉      | 392/1000 [00:40<01:02,  9.65it/s] 39%|███▉      | 393/1000 [00:41<01:02,  9.74it/s] 39%|███▉      | 394/1000 [00:41<01:02,  9.72it/s] 40%|███▉      | 395/1000 [00:41<01:01,  9.79it/s] 40%|███▉      | 396/1000 [00:41<01:01,  9.82it/s] 40%|███▉      | 397/1000 [00:41<01:02,  9.60it/s] 40%|███▉      | 398/1000 [00:41<01:02,  9.63it/s] 40%|████      | 400/1000 [00:41<01:00,  9.87it/s] 40%|████      | 401/1000 [00:41<01:00,  9.84it/s] 40%|████      | 403/1000 [00:42<00:59,  9.97it/s] 40%|████      | 404/1000 [00:42<01:00,  9.88it/s] 40%|████      | 405/1000 [00:42<01:00,  9.89it/s] 41%|████      | 406/1000 [00:42<01:00,  9.89it/s] 41%|████      | 407/1000 [00:42<01:00,  9.82it/s] 41%|████      | 408/1000 [00:42<01:01,  9.67it/s] 41%|████      | 409/1000 [00:42<01:01,  9.68it/s] 41%|████      | 410/1000 [00:42<01:00,  9.77it/s] 41%|████      | 412/1000 [00:42<00:59,  9.96it/s] 41%|████▏     | 413/1000 [00:43<00:59,  9.94it/s] 41%|████▏     | 414/1000 [00:43<00:59,  9.81it/s] 42%|████▏     | 415/1000 [00:43<01:00,  9.66it/s] 42%|████▏     | 416/1000 [00:43<01:00,  9.60it/s] 42%|████▏     | 417/1000 [00:43<01:01,  9.55it/s] 42%|████▏     | 418/1000 [00:43<01:01,  9.45it/s] 42%|████▏     | 419/1000 [00:43<01:01,  9.52it/s] 42%|████▏     | 420/1000 [00:43<01:01,  9.50it/s] 42%|████▏     | 421/1000 [00:43<01:00,  9.58it/s] 42%|████▏     | 422/1000 [00:44<01:00,  9.61it/s] 42%|████▏     | 423/1000 [00:44<01:00,  9.56it/s] 42%|████▏     | 424/1000 [00:44<01:00,  9.57it/s] 42%|████▎     | 425/1000 [00:44<00:59,  9.67it/s] 43%|████▎     | 426/1000 [00:44<00:59,  9.68it/s] 43%|████▎     | 427/1000 [00:44<01:02,  9.14it/s] 43%|████▎     | 428/1000 [00:44<01:03,  8.95it/s] 43%|████▎     | 429/1000 [00:44<01:05,  8.77it/s] 43%|████▎     | 430/1000 [00:44<01:05,  8.69it/s] 43%|████▎     | 431/1000 [00:45<01:05,  8.65it/s] 43%|████▎     | 432/1000 [00:45<01:06,  8.59it/s] 43%|████▎     | 433/1000 [00:45<01:05,  8.61it/s] 43%|████▎     | 434/1000 [00:45<01:04,  8.79it/s] 44%|████▎     | 435/1000 [00:45<01:05,  8.66it/s] 44%|████▎     | 436/1000 [00:45<01:05,  8.59it/s] 44%|████▎     | 437/1000 [00:45<01:05,  8.64it/s] 44%|████▍     | 438/1000 [00:45<01:05,  8.62it/s] 44%|████▍     | 439/1000 [00:45<01:05,  8.56it/s] 44%|████▍     | 440/1000 [00:46<01:06,  8.42it/s] 44%|████▍     | 441/1000 [00:46<01:07,  8.32it/s] 44%|████▍     | 442/1000 [00:46<01:07,  8.29it/s] 44%|████▍     | 443/1000 [00:46<01:08,  8.19it/s] 44%|████▍     | 444/1000 [00:46<01:08,  8.16it/s] 44%|████▍     | 445/1000 [00:46<01:07,  8.18it/s] 45%|████▍     | 446/1000 [00:46<01:07,  8.20it/s] 45%|████▍     | 447/1000 [00:46<01:05,  8.40it/s] 45%|████▍     | 448/1000 [00:47<01:03,  8.72it/s] 45%|████▍     | 449/1000 [00:47<01:03,  8.73it/s] 45%|████▌     | 450/1000 [00:47<01:05,  8.41it/s] 45%|████▌     | 451/1000 [00:47<01:06,  8.25it/s] 45%|████▌     | 452/1000 [00:47<01:07,  8.13it/s] 45%|████▌     | 453/1000 [00:47<01:07,  8.07it/s] 45%|████▌     | 454/1000 [00:47<01:08,  8.03it/s] 46%|████▌     | 455/1000 [00:47<01:07,  8.08it/s] 46%|████▌     | 456/1000 [00:48<01:07,  8.03it/s] 46%|████▌     | 457/1000 [00:48<01:07,  7.99it/s] 46%|████▌     | 458/1000 [00:48<01:07,  8.01it/s] 46%|████▌     | 459/1000 [00:48<01:06,  8.17it/s] 46%|████▌     | 460/1000 [00:48<01:04,  8.33it/s] 46%|████▌     | 461/1000 [00:48<01:05,  8.26it/s] 46%|████▌     | 462/1000 [00:48<01:04,  8.32it/s] 46%|████▋     | 463/1000 [00:48<01:03,  8.43it/s] 46%|████▋     | 464/1000 [00:48<01:01,  8.65it/s] 46%|████▋     | 465/1000 [00:49<00:59,  8.93it/s] 47%|████▋     | 466/1000 [00:49<00:59,  8.99it/s] 47%|████▋     | 467/1000 [00:49<00:57,  9.21it/s] 47%|████▋     | 468/1000 [00:49<00:58,  9.02it/s] 47%|████▋     | 469/1000 [00:49<00:59,  8.88it/s] 47%|████▋     | 470/1000 [00:49<01:01,  8.64it/s] 47%|████▋     | 471/1000 [00:49<01:02,  8.44it/s] 47%|████▋     | 472/1000 [00:49<01:02,  8.39it/s] 47%|████▋     | 473/1000 [00:50<01:03,  8.33it/s] 47%|████▋     | 474/1000 [00:50<01:03,  8.26it/s] 48%|████▊     | 475/1000 [00:50<01:04,  8.17it/s] 48%|████▊     | 476/1000 [00:50<01:04,  8.15it/s] 48%|████▊     | 477/1000 [00:50<01:04,  8.13it/s] 48%|████▊     | 478/1000 [00:50<01:04,  8.14it/s] 48%|████▊     | 479/1000 [00:50<01:03,  8.15it/s] 48%|████▊     | 480/1000 [00:50<01:03,  8.17it/s] 48%|████▊     | 481/1000 [00:51<01:03,  8.18it/s] 48%|████▊     | 482/1000 [00:51<01:03,  8.18it/s] 48%|████▊     | 483/1000 [00:51<01:03,  8.20it/s] 48%|████▊     | 484/1000 [00:51<01:02,  8.20it/s] 48%|████▊     | 485/1000 [00:51<01:02,  8.24it/s] 49%|████▊     | 486/1000 [00:51<01:02,  8.24it/s] 49%|████▊     | 487/1000 [00:51<01:02,  8.22it/s] 49%|████▉     | 488/1000 [00:51<01:02,  8.19it/s] 49%|████▉     | 489/1000 [00:51<01:02,  8.21it/s] 49%|████▉     | 490/1000 [00:52<01:01,  8.23it/s] 49%|████▉     | 491/1000 [00:52<01:01,  8.25it/s] 49%|████▉     | 492/1000 [00:52<01:01,  8.25it/s] 49%|████▉     | 493/1000 [00:52<01:01,  8.21it/s] 49%|████▉     | 494/1000 [00:52<01:01,  8.20it/s] 50%|████▉     | 495/1000 [00:52<01:02,  8.15it/s] 50%|████▉     | 496/1000 [00:52<01:02,  8.12it/s] 50%|████▉     | 497/1000 [00:52<01:02,  8.06it/s] 50%|████▉     | 498/1000 [00:53<01:02,  8.08it/s] 50%|████▉     | 499/1000 [00:53<01:01,  8.11it/s] 50%|█████     | 500/1000 [00:53<01:00,  8.26it/s] 50%|█████     | 501/1000 [00:53<00:59,  8.32it/s] 50%|█████     | 502/1000 [00:53<00:59,  8.42it/s] 50%|█████     | 503/1000 [00:53<00:58,  8.46it/s] 50%|█████     | 504/1000 [00:53<00:58,  8.49it/s] 50%|█████     | 505/1000 [00:53<00:59,  8.31it/s] 51%|█████     | 506/1000 [00:54<00:58,  8.38it/s] 51%|█████     | 507/1000 [00:54<00:58,  8.37it/s] 51%|█████     | 508/1000 [00:54<00:57,  8.51it/s] 51%|█████     | 509/1000 [00:54<00:56,  8.74it/s] 51%|█████     | 510/1000 [00:54<00:54,  8.98it/s] 51%|█████     | 511/1000 [00:54<00:52,  9.23it/s] 51%|█████▏    | 513/1000 [00:54<00:50,  9.60it/s] 51%|█████▏    | 514/1000 [00:54<00:51,  9.50it/s] 52%|█████▏    | 515/1000 [00:54<00:50,  9.59it/s] 52%|█████▏    | 517/1000 [00:55<00:49,  9.66it/s] 52%|█████▏    | 518/1000 [00:55<00:50,  9.46it/s] 52%|█████▏    | 519/1000 [00:55<00:52,  9.24it/s] 52%|█████▏    | 520/1000 [00:55<00:51,  9.24it/s] 52%|█████▏    | 521/1000 [00:55<00:51,  9.35it/s] 52%|█████▏    | 522/1000 [00:55<00:51,  9.24it/s] 52%|█████▏    | 523/1000 [00:55<00:51,  9.20it/s] 52%|█████▏    | 524/1000 [00:55<00:51,  9.21it/s] 52%|█████▎    | 525/1000 [00:56<00:51,  9.28it/s] 53%|█████▎    | 526/1000 [00:56<00:51,  9.24it/s] 53%|█████▎    | 527/1000 [00:56<00:50,  9.32it/s] 53%|█████▎    | 528/1000 [00:56<00:50,  9.35it/s] 53%|█████▎    | 529/1000 [00:56<00:50,  9.33it/s] 53%|█████▎    | 530/1000 [00:56<00:50,  9.35it/s] 53%|█████▎    | 531/1000 [00:56<00:51,  9.07it/s] 53%|█████▎    | 532/1000 [00:56<00:51,  9.10it/s] 53%|█████▎    | 533/1000 [00:56<00:52,  8.94it/s] 53%|█████▎    | 534/1000 [00:57<00:51,  9.04it/s] 54%|█████▎    | 535/1000 [00:57<00:50,  9.13it/s] 54%|█████▎    | 536/1000 [00:57<00:50,  9.20it/s] 54%|█████▎    | 537/1000 [00:57<00:51,  8.97it/s] 54%|█████▍    | 538/1000 [00:57<00:51,  8.90it/s] 54%|█████▍    | 539/1000 [00:57<00:50,  9.05it/s] 54%|█████▍    | 540/1000 [00:57<00:50,  9.14it/s] 54%|█████▍    | 541/1000 [00:57<00:50,  9.15it/s] 54%|█████▍    | 542/1000 [00:57<00:51,  8.88it/s] 54%|█████▍    | 543/1000 [00:58<00:52,  8.78it/s] 54%|█████▍    | 544/1000 [00:58<00:52,  8.69it/s] 55%|█████▍    | 545/1000 [00:58<00:52,  8.73it/s] 55%|█████▍    | 546/1000 [00:58<00:51,  8.81it/s] 55%|█████▍    | 547/1000 [00:58<00:50,  8.92it/s] 55%|█████▍    | 548/1000 [00:58<00:51,  8.81it/s] 55%|█████▍    | 549/1000 [00:58<00:51,  8.77it/s] 55%|█████▌    | 550/1000 [00:58<00:50,  8.86it/s] 55%|█████▌    | 551/1000 [00:58<00:50,  8.85it/s] 55%|█████▌    | 552/1000 [00:59<00:51,  8.74it/s] 55%|█████▌    | 553/1000 [00:59<00:51,  8.67it/s] 55%|█████▌    | 554/1000 [00:59<00:51,  8.61it/s] 56%|█████▌    | 555/1000 [00:59<00:52,  8.51it/s] 56%|█████▌    | 556/1000 [00:59<00:52,  8.43it/s] 56%|█████▌    | 557/1000 [00:59<00:52,  8.39it/s] 56%|█████▌    | 558/1000 [00:59<00:51,  8.52it/s] 56%|█████▌    | 559/1000 [00:59<00:51,  8.51it/s] 56%|█████▌    | 560/1000 [01:00<00:51,  8.48it/s] 56%|█████▌    | 561/1000 [01:00<00:51,  8.56it/s] 56%|█████▌    | 562/1000 [01:00<00:51,  8.57it/s] 56%|█████▋    | 563/1000 [01:00<00:51,  8.54it/s] 56%|█████▋    | 564/1000 [01:00<00:51,  8.52it/s] 56%|█████▋    | 565/1000 [01:00<00:50,  8.55it/s] 57%|█████▋    | 566/1000 [01:00<00:49,  8.80it/s] 57%|█████▋    | 567/1000 [01:00<00:49,  8.75it/s] 57%|█████▋    | 568/1000 [01:00<00:48,  8.88it/s] 57%|█████▋    | 569/1000 [01:01<00:48,  8.96it/s] 57%|█████▋    | 570/1000 [01:01<00:47,  8.97it/s] 57%|█████▋    | 571/1000 [01:01<00:47,  9.02it/s] 57%|█████▋    | 572/1000 [01:01<00:48,  8.78it/s] 57%|█████▋    | 573/1000 [01:01<00:49,  8.68it/s] 57%|█████▋    | 574/1000 [01:01<00:49,  8.61it/s] 57%|█████▊    | 575/1000 [01:01<00:49,  8.55it/s] 58%|█████▊    | 576/1000 [01:01<00:49,  8.54it/s] 58%|█████▊    | 577/1000 [01:01<00:49,  8.52it/s] 58%|█████▊    | 578/1000 [01:02<00:49,  8.48it/s] 58%|█████▊    | 579/1000 [01:02<00:48,  8.63it/s] 58%|█████▊    | 580/1000 [01:02<00:47,  8.90it/s] 58%|█████▊    | 581/1000 [01:02<00:45,  9.14it/s] 58%|█████▊    | 582/1000 [01:02<00:44,  9.34it/s] 58%|█████▊    | 583/1000 [01:02<00:43,  9.51it/s] 58%|█████▊    | 585/1000 [01:02<00:42,  9.81it/s] 59%|█████▊    | 586/1000 [01:02<00:43,  9.60it/s] 59%|█████▊    | 587/1000 [01:03<00:45,  9.15it/s] 59%|█████▉    | 588/1000 [01:03<00:46,  8.94it/s] 59%|█████▉    | 589/1000 [01:03<00:45,  9.01it/s] 59%|█████▉    | 590/1000 [01:03<00:47,  8.69it/s] 59%|█████▉    | 591/1000 [01:03<00:46,  8.87it/s] 59%|█████▉    | 592/1000 [01:03<00:45,  9.06it/s] 59%|█████▉    | 593/1000 [01:03<00:43,  9.26it/s] 59%|█████▉    | 594/1000 [01:03<00:44,  9.02it/s] 60%|█████▉    | 595/1000 [01:03<00:45,  8.92it/s] 60%|█████▉    | 596/1000 [01:04<00:45,  8.83it/s] 60%|█████▉    | 597/1000 [01:04<00:46,  8.59it/s] 60%|█████▉    | 598/1000 [01:04<00:46,  8.63it/s] 60%|█████▉    | 599/1000 [01:04<00:46,  8.61it/s] 60%|██████    | 600/1000 [01:04<00:46,  8.65it/s] 60%|██████    | 601/1000 [01:04<00:46,  8.66it/s] 60%|██████    | 602/1000 [01:04<00:46,  8.50it/s] 60%|██████    | 603/1000 [01:04<00:46,  8.48it/s] 60%|██████    | 604/1000 [01:05<00:46,  8.49it/s] 60%|██████    | 605/1000 [01:05<00:46,  8.43it/s] 61%|██████    | 606/1000 [01:05<00:47,  8.27it/s] 61%|██████    | 607/1000 [01:05<00:46,  8.39it/s] 61%|██████    | 608/1000 [01:05<00:45,  8.54it/s] 61%|██████    | 609/1000 [01:05<00:45,  8.61it/s] 61%|██████    | 610/1000 [01:05<00:44,  8.67it/s] 61%|██████    | 611/1000 [01:05<00:44,  8.67it/s] 61%|██████    | 612/1000 [01:05<00:44,  8.68it/s] 61%|██████▏   | 613/1000 [01:06<00:44,  8.69it/s] 61%|██████▏   | 614/1000 [01:06<00:44,  8.73it/s] 62%|██████▏   | 615/1000 [01:06<00:45,  8.47it/s] 62%|██████▏   | 616/1000 [01:06<00:44,  8.55it/s] 62%|██████▏   | 617/1000 [01:06<00:44,  8.60it/s] 62%|██████▏   | 618/1000 [01:06<00:44,  8.55it/s] 62%|██████▏   | 619/1000 [01:06<00:44,  8.50it/s] 62%|██████▏   | 620/1000 [01:06<00:45,  8.40it/s] 62%|██████▏   | 621/1000 [01:07<00:45,  8.27it/s] 62%|██████▏   | 622/1000 [01:07<00:46,  8.19it/s] 62%|██████▏   | 623/1000 [01:07<00:46,  8.16it/s] 62%|██████▏   | 624/1000 [01:07<00:45,  8.25it/s] 62%|██████▎   | 625/1000 [01:07<00:45,  8.31it/s] 63%|██████▎   | 626/1000 [01:07<00:44,  8.43it/s] 63%|██████▎   | 627/1000 [01:07<00:44,  8.45it/s] 63%|██████▎   | 628/1000 [01:07<00:44,  8.42it/s] 63%|██████▎   | 629/1000 [01:07<00:44,  8.36it/s] 63%|██████▎   | 630/1000 [01:08<00:44,  8.34it/s] 63%|██████▎   | 631/1000 [01:08<00:44,  8.30it/s] 63%|██████▎   | 632/1000 [01:08<00:44,  8.29it/s] 63%|██████▎   | 633/1000 [01:08<00:44,  8.28it/s] 63%|██████▎   | 634/1000 [01:08<00:44,  8.29it/s] 64%|██████▎   | 635/1000 [01:08<00:44,  8.28it/s] 64%|██████▎   | 636/1000 [01:08<00:43,  8.29it/s] 64%|██████▎   | 637/1000 [01:08<00:43,  8.27it/s] 64%|██████▍   | 638/1000 [01:09<00:43,  8.25it/s] 64%|██████▍   | 639/1000 [01:09<00:43,  8.29it/s] 64%|██████▍   | 640/1000 [01:09<00:43,  8.26it/s] 64%|██████▍   | 641/1000 [01:09<00:43,  8.25it/s] 64%|██████▍   | 642/1000 [01:09<00:43,  8.27it/s] 64%|██████▍   | 643/1000 [01:09<00:43,  8.27it/s] 64%|██████▍   | 644/1000 [01:09<00:42,  8.33it/s] 64%|██████▍   | 645/1000 [01:09<00:42,  8.42it/s] 65%|██████▍   | 646/1000 [01:10<00:41,  8.48it/s] 65%|██████▍   | 647/1000 [01:10<00:41,  8.45it/s] 65%|██████▍   | 648/1000 [01:10<00:41,  8.38it/s] 65%|██████▍   | 649/1000 [01:10<00:42,  8.34it/s] 65%|██████▌   | 650/1000 [01:10<00:42,  8.30it/s] 65%|██████▌   | 651/1000 [01:10<00:42,  8.21it/s] 65%|██████▌   | 652/1000 [01:10<00:42,  8.19it/s] 65%|██████▌   | 653/1000 [01:10<00:42,  8.23it/s] 65%|██████▌   | 654/1000 [01:10<00:42,  8.20it/s] 66%|██████▌   | 655/1000 [01:11<00:41,  8.25it/s] 66%|██████▌   | 656/1000 [01:11<00:41,  8.28it/s] 66%|██████▌   | 657/1000 [01:11<00:41,  8.30it/s] 66%|██████▌   | 658/1000 [01:11<00:41,  8.29it/s] 66%|██████▌   | 659/1000 [01:11<00:40,  8.32it/s] 66%|██████▌   | 660/1000 [01:11<00:41,  8.26it/s] 66%|██████▌   | 661/1000 [01:11<00:40,  8.29it/s] 66%|██████▌   | 662/1000 [01:11<00:40,  8.32it/s] 66%|██████▋   | 663/1000 [01:12<00:40,  8.30it/s] 66%|██████▋   | 664/1000 [01:12<00:40,  8.31it/s] 66%|██████▋   | 665/1000 [01:12<00:40,  8.36it/s] 67%|██████▋   | 666/1000 [01:12<00:39,  8.37it/s] 67%|██████▋   | 667/1000 [01:12<00:39,  8.37it/s] 67%|██████▋   | 668/1000 [01:12<00:39,  8.39it/s] 67%|██████▋   | 669/1000 [01:12<00:39,  8.39it/s] 67%|██████▋   | 670/1000 [01:12<00:39,  8.37it/s] 67%|██████▋   | 671/1000 [01:13<00:39,  8.34it/s] 67%|██████▋   | 672/1000 [01:13<00:39,  8.31it/s] 67%|██████▋   | 673/1000 [01:13<00:39,  8.32it/s] 67%|██████▋   | 674/1000 [01:13<00:39,  8.30it/s] 68%|██████▊   | 675/1000 [01:13<00:39,  8.25it/s] 68%|██████▊   | 676/1000 [01:13<00:39,  8.22it/s] 68%|██████▊   | 677/1000 [01:13<00:39,  8.23it/s] 68%|██████▊   | 678/1000 [01:13<00:39,  8.17it/s] 68%|██████▊   | 679/1000 [01:14<00:39,  8.19it/s] 68%|██████▊   | 680/1000 [01:14<00:39,  8.12it/s] 68%|██████▊   | 681/1000 [01:14<00:39,  8.08it/s] 68%|██████▊   | 682/1000 [01:14<00:39,  8.08it/s] 68%|██████▊   | 683/1000 [01:14<00:39,  8.03it/s] 68%|██████▊   | 684/1000 [01:14<00:39,  7.97it/s] 68%|██████▊   | 685/1000 [01:14<00:39,  7.97it/s] 69%|██████▊   | 686/1000 [01:14<00:39,  7.99it/s] 69%|██████▊   | 687/1000 [01:15<00:38,  8.11it/s] 69%|██████▉   | 688/1000 [01:15<00:38,  8.16it/s] 69%|██████▉   | 689/1000 [01:15<00:38,  8.16it/s] 69%|██████▉   | 690/1000 [01:15<00:37,  8.19it/s] 69%|██████▉   | 691/1000 [01:15<00:37,  8.28it/s] 69%|██████▉   | 692/1000 [01:15<00:36,  8.34it/s] 69%|██████▉   | 693/1000 [01:15<00:36,  8.41it/s] 69%|██████▉   | 694/1000 [01:15<00:36,  8.45it/s] 70%|██████▉   | 695/1000 [01:15<00:36,  8.47it/s] 70%|██████▉   | 696/1000 [01:16<00:36,  8.44it/s] 70%|██████▉   | 697/1000 [01:16<00:36,  8.32it/s] 70%|██████▉   | 698/1000 [01:16<00:36,  8.28it/s] 70%|██████▉   | 699/1000 [01:16<00:36,  8.32it/s] 70%|███████   | 700/1000 [01:16<00:36,  8.33it/s] 70%|███████   | 701/1000 [01:16<00:35,  8.35it/s] 70%|███████   | 702/1000 [01:16<00:35,  8.30it/s] 70%|███████   | 703/1000 [01:16<00:35,  8.29it/s] 70%|███████   | 704/1000 [01:17<00:35,  8.33it/s] 70%|███████   | 705/1000 [01:17<00:35,  8.32it/s] 71%|███████   | 706/1000 [01:17<00:35,  8.18it/s] 71%|███████   | 707/1000 [01:17<00:36,  8.13it/s] 71%|███████   | 708/1000 [01:17<00:35,  8.19it/s] 71%|███████   | 709/1000 [01:17<00:35,  8.24it/s] 71%|███████   | 710/1000 [01:17<00:34,  8.30it/s] 71%|███████   | 711/1000 [01:17<00:34,  8.35it/s] 71%|███████   | 712/1000 [01:18<00:34,  8.42it/s] 71%|███████▏  | 713/1000 [01:18<00:34,  8.44it/s] 71%|███████▏  | 714/1000 [01:18<00:33,  8.44it/s] 72%|███████▏  | 715/1000 [01:18<00:34,  8.36it/s] 72%|███████▏  | 716/1000 [01:18<00:34,  8.31it/s] 72%|███████▏  | 717/1000 [01:18<00:33,  8.37it/s] 72%|███████▏  | 718/1000 [01:18<00:34,  8.26it/s] 72%|███████▏  | 719/1000 [01:18<00:33,  8.34it/s] 72%|███████▏  | 720/1000 [01:18<00:33,  8.39it/s] 72%|███████▏  | 721/1000 [01:19<00:33,  8.43it/s] 72%|███████▏  | 722/1000 [01:19<00:33,  8.38it/s] 72%|███████▏  | 723/1000 [01:19<00:32,  8.39it/s] 72%|███████▏  | 724/1000 [01:19<00:33,  8.33it/s] 72%|███████▎  | 725/1000 [01:19<00:33,  8.32it/s] 73%|███████▎  | 726/1000 [01:19<00:32,  8.36it/s] 73%|███████▎  | 727/1000 [01:19<00:32,  8.40it/s] 73%|███████▎  | 728/1000 [01:19<00:32,  8.42it/s] 73%|███████▎  | 729/1000 [01:20<00:32,  8.40it/s] 73%|███████▎  | 730/1000 [01:20<00:32,  8.29it/s] 73%|███████▎  | 731/1000 [01:20<00:32,  8.18it/s] 73%|███████▎  | 732/1000 [01:20<00:32,  8.13it/s] 73%|███████▎  | 733/1000 [01:20<00:33,  8.09it/s] 73%|███████▎  | 734/1000 [01:20<00:33,  8.05it/s] 74%|███████▎  | 735/1000 [01:20<00:32,  8.03it/s] 74%|███████▎  | 736/1000 [01:20<00:32,  8.14it/s] 74%|███████▎  | 737/1000 [01:21<00:31,  8.24it/s] 74%|███████▍  | 738/1000 [01:21<00:31,  8.31it/s] 74%|███████▍  | 739/1000 [01:21<00:31,  8.36it/s] 74%|███████▍  | 740/1000 [01:21<00:31,  8.33it/s] 74%|███████▍  | 741/1000 [01:21<00:30,  8.37it/s] 74%|███████▍  | 742/1000 [01:21<00:30,  8.45it/s] 74%|███████▍  | 743/1000 [01:21<00:30,  8.44it/s] 74%|███████▍  | 744/1000 [01:21<00:30,  8.38it/s] 74%|███████▍  | 745/1000 [01:21<00:30,  8.34it/s] 75%|███████▍  | 746/1000 [01:22<00:30,  8.46it/s] 75%|███████▍  | 747/1000 [01:22<00:29,  8.49it/s] 75%|███████▍  | 748/1000 [01:22<00:29,  8.56it/s] 75%|███████▍  | 749/1000 [01:22<00:29,  8.54it/s] 75%|███████▌  | 750/1000 [01:22<00:29,  8.54it/s] 75%|███████▌  | 751/1000 [01:22<00:29,  8.57it/s] 75%|███████▌  | 752/1000 [01:22<00:29,  8.52it/s] 75%|███████▌  | 753/1000 [01:22<00:29,  8.51it/s] 75%|███████▌  | 754/1000 [01:23<00:28,  8.49it/s] 76%|███████▌  | 755/1000 [01:23<00:28,  8.48it/s] 76%|███████▌  | 756/1000 [01:23<00:28,  8.50it/s] 76%|███████▌  | 757/1000 [01:23<00:28,  8.48it/s] 76%|███████▌  | 758/1000 [01:23<00:28,  8.45it/s] 76%|███████▌  | 759/1000 [01:23<00:29,  8.15it/s] 76%|███████▌  | 760/1000 [01:23<00:30,  7.85it/s] 76%|███████▌  | 761/1000 [01:23<00:30,  7.73it/s] 76%|███████▌  | 762/1000 [01:24<00:30,  7.78it/s] 76%|███████▋  | 763/1000 [01:24<00:29,  8.00it/s] 76%|███████▋  | 764/1000 [01:24<00:28,  8.34it/s] 76%|███████▋  | 765/1000 [01:24<00:27,  8.63it/s] 77%|███████▋  | 766/1000 [01:24<00:26,  8.86it/s] 77%|███████▋  | 767/1000 [01:24<00:25,  9.11it/s] 77%|███████▋  | 768/1000 [01:24<00:25,  9.22it/s] 77%|███████▋  | 769/1000 [01:24<00:24,  9.28it/s] 77%|███████▋  | 770/1000 [01:24<00:26,  8.74it/s] 77%|███████▋  | 771/1000 [01:25<00:26,  8.55it/s] 77%|███████▋  | 772/1000 [01:25<00:27,  8.32it/s] 77%|███████▋  | 773/1000 [01:25<00:27,  8.30it/s] 77%|███████▋  | 774/1000 [01:25<00:27,  8.19it/s] 78%|███████▊  | 775/1000 [01:25<00:27,  8.13it/s] 78%|███████▊  | 776/1000 [01:25<00:27,  8.26it/s] 78%|███████▊  | 777/1000 [01:25<00:26,  8.31it/s] 78%|███████▊  | 778/1000 [01:25<00:26,  8.31it/s] 78%|███████▊  | 779/1000 [01:26<00:26,  8.31it/s] 78%|███████▊  | 780/1000 [01:26<00:26,  8.29it/s] 78%|███████▊  | 781/1000 [01:26<00:26,  8.33it/s] 78%|███████▊  | 782/1000 [01:26<00:25,  8.40it/s] 78%|███████▊  | 783/1000 [01:26<00:25,  8.45it/s] 78%|███████▊  | 784/1000 [01:26<00:25,  8.50it/s] 78%|███████▊  | 785/1000 [01:26<00:25,  8.44it/s] 79%|███████▊  | 786/1000 [01:26<00:25,  8.41it/s] 79%|███████▊  | 787/1000 [01:26<00:25,  8.30it/s] 79%|███████▉  | 788/1000 [01:27<00:25,  8.23it/s] 79%|███████▉  | 789/1000 [01:27<00:25,  8.15it/s] 79%|███████▉  | 790/1000 [01:27<00:25,  8.19it/s] 79%|███████▉  | 791/1000 [01:27<00:25,  8.22it/s] 79%|███████▉  | 792/1000 [01:27<00:25,  8.24it/s] 79%|███████▉  | 793/1000 [01:27<00:25,  8.20it/s] 79%|███████▉  | 794/1000 [01:27<00:25,  8.20it/s] 80%|███████▉  | 795/1000 [01:27<00:25,  8.18it/s] 80%|███████▉  | 796/1000 [01:28<00:24,  8.19it/s] 80%|███████▉  | 797/1000 [01:28<00:24,  8.20it/s] 80%|███████▉  | 798/1000 [01:28<00:24,  8.18it/s] 80%|███████▉  | 799/1000 [01:28<00:24,  8.26it/s] 80%|████████  | 800/1000 [01:28<00:24,  8.22it/s] 80%|████████  | 801/1000 [01:28<00:24,  8.23it/s] 80%|████████  | 802/1000 [01:28<00:23,  8.26it/s] 80%|████████  | 803/1000 [01:28<00:24,  8.20it/s] 80%|████████  | 804/1000 [01:29<00:24,  8.09it/s] 80%|████████  | 805/1000 [01:29<00:24,  8.10it/s] 81%|████████  | 806/1000 [01:29<00:24,  8.01it/s] 81%|████████  | 807/1000 [01:29<00:24,  8.00it/s] 81%|████████  | 808/1000 [01:29<00:23,  8.01it/s] 81%|████████  | 809/1000 [01:29<00:23,  8.01it/s] 81%|████████  | 810/1000 [01:29<00:23,  8.02it/s] 81%|████████  | 811/1000 [01:29<00:23,  8.04it/s] 81%|████████  | 812/1000 [01:30<00:23,  8.04it/s] 81%|████████▏ | 813/1000 [01:30<00:22,  8.16it/s] 81%|████████▏ | 814/1000 [01:30<00:22,  8.23it/s] 82%|████████▏ | 815/1000 [01:30<00:22,  8.26it/s] 82%|████████▏ | 816/1000 [01:30<00:22,  8.23it/s] 82%|████████▏ | 817/1000 [01:30<00:22,  8.20it/s] 82%|████████▏ | 818/1000 [01:30<00:21,  8.29it/s] 82%|████████▏ | 819/1000 [01:30<00:21,  8.28it/s] 82%|████████▏ | 820/1000 [01:31<00:21,  8.33it/s] 82%|████████▏ | 821/1000 [01:31<00:21,  8.33it/s] 82%|████████▏ | 822/1000 [01:31<00:21,  8.39it/s] 82%|████████▏ | 823/1000 [01:31<00:21,  8.36it/s] 82%|████████▏ | 824/1000 [01:31<00:21,  8.30it/s] 82%|████████▎ | 825/1000 [01:31<00:21,  8.32it/s] 83%|████████▎ | 826/1000 [01:31<00:20,  8.38it/s] 83%|████████▎ | 827/1000 [01:31<00:20,  8.47it/s] 83%|████████▎ | 828/1000 [01:31<00:20,  8.51it/s] 83%|████████▎ | 829/1000 [01:32<00:20,  8.50it/s] 83%|████████▎ | 830/1000 [01:32<00:20,  8.45it/s] 83%|████████▎ | 831/1000 [01:32<00:19,  8.52it/s] 83%|████████▎ | 832/1000 [01:32<00:19,  8.51it/s] 83%|████████▎ | 833/1000 [01:32<00:19,  8.52it/s] 83%|████████▎ | 834/1000 [01:32<00:19,  8.51it/s] 84%|████████▎ | 835/1000 [01:32<00:19,  8.55it/s] 84%|████████▎ | 836/1000 [01:32<00:19,  8.53it/s] 84%|████████▎ | 837/1000 [01:33<00:19,  8.49it/s] 84%|████████▍ | 838/1000 [01:33<00:19,  8.41it/s] 84%|████████▍ | 839/1000 [01:33<00:19,  8.41it/s] 84%|████████▍ | 840/1000 [01:33<00:18,  8.44it/s] 84%|████████▍ | 841/1000 [01:33<00:18,  8.46it/s] 84%|████████▍ | 842/1000 [01:33<00:18,  8.47it/s] 84%|████████▍ | 843/1000 [01:33<00:18,  8.45it/s] 84%|████████▍ | 844/1000 [01:33<00:18,  8.42it/s] 84%|████████▍ | 845/1000 [01:33<00:18,  8.44it/s] 85%|████████▍ | 846/1000 [01:34<00:18,  8.41it/s] 85%|████████▍ | 847/1000 [01:34<00:18,  8.17it/s] 85%|████████▍ | 848/1000 [01:34<00:18,  8.31it/s] 85%|████████▍ | 849/1000 [01:34<00:18,  8.37it/s] 85%|████████▌ | 850/1000 [01:34<00:17,  8.46it/s] 85%|████████▌ | 851/1000 [01:34<00:17,  8.67it/s] 85%|████████▌ | 852/1000 [01:34<00:16,  8.88it/s] 85%|████████▌ | 853/1000 [01:34<00:16,  9.01it/s] 85%|████████▌ | 854/1000 [01:34<00:16,  9.08it/s] 86%|████████▌ | 855/1000 [01:35<00:15,  9.12it/s] 86%|████████▌ | 856/1000 [01:35<00:16,  8.84it/s] 86%|████████▌ | 857/1000 [01:35<00:16,  8.76it/s] 86%|████████▌ | 858/1000 [01:35<00:16,  8.78it/s] 86%|████████▌ | 859/1000 [01:35<00:16,  8.77it/s] 86%|████████▌ | 860/1000 [01:35<00:15,  8.87it/s] 86%|████████▌ | 861/1000 [01:35<00:15,  8.88it/s] 86%|████████▌ | 862/1000 [01:35<00:15,  8.98it/s] 86%|████████▋ | 863/1000 [01:36<00:15,  8.93it/s] 86%|████████▋ | 864/1000 [01:36<00:15,  8.86it/s] 86%|████████▋ | 865/1000 [01:36<00:15,  8.68it/s] 87%|████████▋ | 866/1000 [01:36<00:15,  8.74it/s] 87%|████████▋ | 867/1000 [01:36<00:15,  8.65it/s] 87%|████████▋ | 868/1000 [01:36<00:15,  8.57it/s] 87%|████████▋ | 869/1000 [01:36<00:15,  8.52it/s] 87%|████████▋ | 870/1000 [01:36<00:15,  8.48it/s] 87%|████████▋ | 871/1000 [01:36<00:15,  8.46it/s] 87%|████████▋ | 872/1000 [01:37<00:14,  8.54it/s] 87%|████████▋ | 873/1000 [01:37<00:14,  8.54it/s] 87%|████████▋ | 874/1000 [01:37<00:14,  8.53it/s] 88%|████████▊ | 875/1000 [01:37<00:14,  8.67it/s] 88%|████████▊ | 876/1000 [01:37<00:14,  8.62it/s] 88%|████████▊ | 877/1000 [01:37<00:14,  8.60it/s] 88%|████████▊ | 878/1000 [01:37<00:14,  8.53it/s] 88%|████████▊ | 879/1000 [01:37<00:13,  8.69it/s] 88%|████████▊ | 880/1000 [01:37<00:13,  8.80it/s] 88%|████████▊ | 881/1000 [01:38<00:13,  9.00it/s] 88%|████████▊ | 882/1000 [01:38<00:13,  8.95it/s] 88%|████████▊ | 883/1000 [01:38<00:13,  8.77it/s] 88%|████████▊ | 884/1000 [01:38<00:13,  8.73it/s] 88%|████████▊ | 885/1000 [01:38<00:13,  8.69it/s] 89%|████████▊ | 886/1000 [01:38<00:13,  8.67it/s] 89%|████████▊ | 887/1000 [01:38<00:12,  8.88it/s] 89%|████████▉ | 888/1000 [01:38<00:12,  8.86it/s] 89%|████████▉ | 889/1000 [01:39<00:12,  8.92it/s] 89%|████████▉ | 890/1000 [01:39<00:12,  8.86it/s] 89%|████████▉ | 891/1000 [01:39<00:12,  8.92it/s] 89%|████████▉ | 892/1000 [01:39<00:12,  8.84it/s] 89%|████████▉ | 893/1000 [01:39<00:11,  8.97it/s] 89%|████████▉ | 894/1000 [01:39<00:11,  9.04it/s] 90%|████████▉ | 895/1000 [01:39<00:11,  8.98it/s] 90%|████████▉ | 896/1000 [01:39<00:11,  8.89it/s] 90%|████████▉ | 897/1000 [01:39<00:11,  8.92it/s] 90%|████████▉ | 898/1000 [01:40<00:11,  9.13it/s] 90%|████████▉ | 899/1000 [01:40<00:11,  9.14it/s] 90%|█████████ | 900/1000 [01:40<00:10,  9.20it/s] 90%|█████████ | 901/1000 [01:40<00:11,  8.85it/s] 90%|█████████ | 902/1000 [01:40<00:11,  8.86it/s] 90%|█████████ | 903/1000 [01:40<00:11,  8.78it/s] 90%|█████████ | 904/1000 [01:40<00:10,  8.93it/s] 90%|█████████ | 905/1000 [01:40<00:10,  8.86it/s] 91%|█████████ | 906/1000 [01:40<00:10,  8.81it/s] 91%|█████████ | 907/1000 [01:41<00:10,  8.77it/s] 91%|█████████ | 908/1000 [01:41<00:10,  8.81it/s] 91%|█████████ | 909/1000 [01:41<00:10,  8.97it/s] 91%|█████████ | 910/1000 [01:41<00:10,  8.66it/s] 91%|█████████ | 911/1000 [01:41<00:10,  8.50it/s] 91%|█████████ | 912/1000 [01:41<00:10,  8.44it/s] 91%|█████████▏| 913/1000 [01:41<00:10,  8.40it/s] 91%|█████████▏| 914/1000 [01:41<00:10,  8.39it/s] 92%|█████████▏| 915/1000 [01:41<00:10,  8.34it/s] 92%|█████████▏| 916/1000 [01:42<00:10,  8.35it/s] 92%|█████████▏| 917/1000 [01:42<00:09,  8.31it/s] 92%|█████████▏| 918/1000 [01:42<00:09,  8.32it/s] 92%|█████████▏| 919/1000 [01:42<00:09,  8.28it/s] 92%|█████████▏| 920/1000 [01:42<00:09,  8.31it/s] 92%|█████████▏| 921/1000 [01:42<00:09,  8.30it/s] 92%|█████████▏| 922/1000 [01:42<00:09,  8.30it/s] 92%|█████████▏| 923/1000 [01:42<00:09,  8.28it/s] 92%|█████████▏| 924/1000 [01:43<00:09,  8.28it/s] 92%|█████████▎| 925/1000 [01:43<00:09,  8.22it/s] 93%|█████████▎| 926/1000 [01:43<00:09,  8.21it/s] 93%|█████████▎| 927/1000 [01:43<00:08,  8.20it/s] 93%|█████████▎| 928/1000 [01:43<00:08,  8.16it/s] 93%|█████████▎| 929/1000 [01:43<00:08,  8.16it/s] 93%|█████████▎| 930/1000 [01:43<00:08,  8.19it/s] 93%|█████████▎| 931/1000 [01:43<00:08,  8.20it/s] 93%|█████████▎| 932/1000 [01:44<00:08,  8.21it/s] 93%|█████████▎| 933/1000 [01:44<00:08,  8.20it/s] 93%|█████████▎| 934/1000 [01:44<00:08,  8.23it/s] 94%|█████████▎| 935/1000 [01:44<00:07,  8.17it/s] 94%|█████████▎| 936/1000 [01:44<00:07,  8.10it/s] 94%|█████████▎| 937/1000 [01:44<00:07,  8.09it/s] 94%|█████████▍| 938/1000 [01:44<00:07,  8.06it/s] 94%|█████████▍| 939/1000 [01:44<00:07,  8.13it/s] 94%|█████████▍| 940/1000 [01:45<00:07,  8.38it/s] 94%|█████████▍| 941/1000 [01:45<00:06,  8.69it/s] 94%|█████████▍| 942/1000 [01:45<00:06,  8.82it/s] 94%|█████████▍| 943/1000 [01:45<00:06,  9.08it/s] 94%|█████████▍| 944/1000 [01:45<00:06,  9.24it/s] 94%|█████████▍| 945/1000 [01:45<00:06,  9.08it/s] 95%|█████████▍| 946/1000 [01:45<00:05,  9.20it/s] 95%|█████████▍| 947/1000 [01:45<00:05,  9.31it/s] 95%|█████████▍| 949/1000 [01:45<00:05,  9.62it/s] 95%|█████████▌| 950/1000 [01:46<00:05,  9.61it/s] 95%|█████████▌| 951/1000 [01:46<00:05,  9.50it/s] 95%|█████████▌| 952/1000 [01:46<00:05,  9.45it/s] 95%|█████████▌| 953/1000 [01:46<00:04,  9.48it/s] 95%|█████████▌| 954/1000 [01:46<00:04,  9.38it/s] 96%|█████████▌| 955/1000 [01:46<00:05,  8.95it/s] 96%|█████████▌| 956/1000 [01:46<00:04,  8.88it/s] 96%|█████████▌| 957/1000 [01:46<00:04,  8.67it/s] 96%|█████████▌| 958/1000 [01:46<00:04,  8.46it/s] 96%|█████████▌| 959/1000 [01:47<00:04,  8.33it/s] 96%|█████████▌| 960/1000 [01:47<00:04,  8.27it/s] 96%|█████████▌| 961/1000 [01:47<00:04,  8.27it/s] 96%|█████████▌| 962/1000 [01:47<00:04,  8.34it/s] 96%|█████████▋| 963/1000 [01:47<00:04,  8.33it/s] 96%|█████████▋| 964/1000 [01:47<00:04,  8.41it/s] 96%|█████████▋| 965/1000 [01:47<00:04,  8.46it/s] 97%|█████████▋| 966/1000 [01:47<00:04,  8.39it/s] 97%|█████████▋| 967/1000 [01:48<00:03,  8.37it/s] 97%|█████████▋| 968/1000 [01:48<00:03,  8.35it/s] 97%|█████████▋| 969/1000 [01:48<00:03,  8.11it/s] 97%|█████████▋| 970/1000 [01:48<00:03,  8.09it/s] 97%|█████████▋| 971/1000 [01:48<00:03,  8.09it/s] 97%|█████████▋| 972/1000 [01:48<00:03,  8.03it/s] 97%|█████████▋| 973/1000 [01:48<00:03,  8.10it/s] 97%|█████████▋| 974/1000 [01:48<00:03,  8.10it/s] 98%|█████████▊| 975/1000 [01:49<00:03,  8.15it/s] 98%|█████████▊| 976/1000 [01:49<00:02,  8.21it/s] 98%|█████████▊| 977/1000 [01:49<00:02,  8.25it/s] 98%|█████████▊| 978/1000 [01:49<00:02,  8.32it/s] 98%|█████████▊| 979/1000 [01:49<00:02,  8.40it/s] 98%|█████████▊| 980/1000 [01:49<00:02,  8.45it/s] 98%|█████████▊| 981/1000 [01:49<00:02,  8.42it/s] 98%|█████████▊| 982/1000 [01:49<00:02,  8.49it/s] 98%|█████████▊| 983/1000 [01:49<00:02,  8.46it/s] 98%|█████████▊| 984/1000 [01:50<00:01,  8.46it/s] 98%|█████████▊| 985/1000 [01:50<00:01,  8.37it/s] 99%|█████████▊| 986/1000 [01:50<00:01,  8.41it/s] 99%|█████████▊| 987/1000 [01:50<00:01,  8.42it/s] 99%|█████████▉| 988/1000 [01:50<00:01,  8.46it/s] 99%|█████████▉| 989/1000 [01:50<00:01,  8.35it/s] 99%|█████████▉| 990/1000 [01:50<00:01,  8.31it/s] 99%|█████████▉| 991/1000 [01:50<00:01,  8.31it/s] 99%|█████████▉| 992/1000 [01:51<00:00,  8.29it/s] 99%|█████████▉| 993/1000 [01:51<00:00,  8.28it/s] 99%|█████████▉| 994/1000 [01:51<00:00,  8.22it/s]100%|█████████▉| 995/1000 [01:51<00:00,  8.24it/s]100%|█████████▉| 996/1000 [01:51<00:00,  8.36it/s]100%|█████████▉| 997/1000 [01:51<00:00,  8.37it/s]100%|█████████▉| 998/1000 [01:51<00:00,  8.38it/s]100%|█████████▉| 999/1000 [01:51<00:00,  8.54it/s]100%|██████████| 1000/1000 [01:52<00:00,  8.63it/s]100%|██████████| 1000/1000 [01:52<00:00,  8.93it/s]

H0                                      X1=0
ri-type                      randomization-c
Estimate                 -0.9929357698186859
Pr(>|t|)                                 0.0
Std. Error (Pr(>|t|))                    0.0
2.5% (Pr(>|t|))                          0.0
97.5% (Pr(>|t|))                         0.0
dtype: object

se

estimation.feols_.Feols.se()

Fitted model standard errors.

Returns

Name	Type	Description
	pd.Series	A pd.Series with the standard errors of the estimated regression model.

solve_ols

estimation.feols_.Feols.solve_ols(tZX, tZY, solver)

Solve the ordinary least squares problem using the specified solver.

Parameters

Name	Type	Description	Default
tZX	np.ndarray		required
tZY	np.ndarray		required
solver	str		required

Returns

Name	Type	Description
	array-like: The solution to the ordinary least squares problem.

Raises

Name	Type	Description
	ValueError: If the specified solver is not supported.

tidy

estimation.feols_.Feols.tidy(alpha=None)

Tidy model outputs.

Return a tidy pd.DataFrame with the point estimates, standard errors, t-statistics, and p-values.

Parameters

Name	Type	Description	Default
alpha	Optional[float]	The significance level for the confidence intervals. If None, computes a 95% confidence interval (alpha = 0.05).	None

Returns

Name	Type	Description
tidy_df	pd.DataFrame	A tidy pd.DataFrame containing the regression results, including point estimates, standard errors, t-statistics, and p-values.

to_array

estimation.feols_.Feols.to_array()

Convert estimation data frames to np arrays.

tstat

estimation.feols_.Feols.tstat()

Fitted model t-statistics.

Returns

Name	Type	Description
	pd.Series	A pd.Series with t-statistics of the estimated regression model.

update

estimation.feols_.Feols.update(X_new, y_new, inplace=False)

Update coefficients for new observations using Sherman-Morrison formula.

Returns

Name	Type	Description
	np.ndarray	Updated coefficients

vcov

estimation.feols_.Feols.vcov(vcov, data=None)

Compute covariance matrices for an estimated regression model.

Parameters

Name	Type	Description	Default
vcov	Union[str, dict[str, str]]	A string or dictionary specifying the type of variance-covariance matrix to use for inference. If a string, it can be one of “iid”, “hetero”, “HC1”, “HC2”, “HC3”. If a dictionary, it should have the format {“CRV1”: “clustervar”} for CRV1 inference or {“CRV3”: “clustervar”} for CRV3 inference. Note that CRV3 inference is currently not supported for IV estimation.	required
data	Optional[DataFrameType]	The data used for estimation. If None, tries to fetch the data from the model object. Defaults to None.	None

Returns

Name	Type	Description
	Feols	An instance of class [Feols(/reference/Feols.qmd) with updated inference.

wald_test

estimation.feols_.Feols.wald_test(R=None, q=None, distribution='F')

Conduct Wald test.

Compute a Wald test for a linear hypothesis of the form R * β = q. where R is m x k matrix, β is a k x 1 vector of coefficients, and q is m x 1 vector. By default, tests the joint null hypothesis that all coefficients are zero.

This method producues the following attriutes

_dfd : int degree of freedom in denominator _dfn : int degree of freedom in numerator _wald_statistic : scalar Wald-statistics computed for hypothesis testing _f_statistic : scalar Wald-statistics(when R is an indentity matrix, and q being zero vector) computed for hypothesis testing _p_value : scalar corresponding p-value for statistics

Parameters

Name	Type	Description	Default
R	array - like	The matrix R of the linear hypothesis. If None, defaults to an identity matrix.	None
q	array - like	The vector q of the linear hypothesis. If None, defaults to a vector of zeros.	None
distribution	str	The distribution to use for the p-value. Can be either “F” or “chi2”. Defaults to “F”.	'F'

Returns

Name	Type	Description
	pd.Series	A pd.Series with the Wald statistic and p-value.

Examples

import numpy as np
import pandas as pd
import pyfixest as pf

data = pf.get_data()
fit = pf.feols("Y ~ X1 + X2| f1", data, vcov={"CRV1": "f1"}, ssc=pf.ssc(adj=False))

R = np.array([[1,-1]] )
q = np.array([0.0])

# Wald test
fit.wald_test(R=R, q=q, distribution = "chi2")
f_stat = fit._f_statistic
p_stat = fit._p_value

print(f"Python f_stat: {f_stat}")
print(f"Python p_stat: {p_stat}")

Python f_stat: 126.40650474043488
Python p_stat: 2.5053092828140783e-29

wildboottest

estimation.feols_.Feols.wildboottest(
    reps,
    cluster=None,
    param=None,
    weights_type='rademacher',
    impose_null=True,
    bootstrap_type='11',
    seed=None,
    adj=True,
    cluster_adj=True,
    parallel=False,
    return_bootstrapped_t_stats=False,
)

Run a wild cluster bootstrap based on an object of type “Feols”.

Parameters

Name	Type	Description	Default
reps	int	The number of bootstrap iterations to run.	required
cluster	Union[str, None]	The variable used for clustering. Defaults to None. If None, then uses the variable specified in the model’s clustervar attribute. If no _clustervar attribute is found, runs a heteroskedasticity- robust bootstrap.	None
param	Union[str, None]	A string of length one, containing the test parameter of interest. Defaults to None.	None
weights_type	str	The type of bootstrap weights. Options are ‘rademacher’, ‘mammen’, ‘webb’, or ‘normal’. Defaults to ‘rademacher’.	'rademacher'
impose_null	bool	Indicates whether to impose the null hypothesis on the bootstrap DGP. Defaults to True.	True
bootstrap_type	str	A string of length one to choose the bootstrap type. Options are ‘11’, ‘31’, ‘13’, or ‘33’. Defaults to ‘11’.	'11'
seed	Union[int, None]	An option to provide a random seed. Defaults to None.	None
adj	bool	Indicates whether to apply a small sample adjustment for the number of observations and covariates. Defaults to True.	True
cluster_adj	bool	Indicates whether to apply a small sample adjustment for the number of clusters. Defaults to True.	True
parallel	bool	Indicates whether to run the bootstrap in parallel. Defaults to False.	False
seed	Union[str, None]	An option to provide a random seed. Defaults to None.	None
return_bootstrapped_t_stats	bool, optional:	If True, the method returns a tuple of the regular output and the bootstrapped t-stats. Defaults to False.	False

Returns

Name	Type	Description
	pd.DataFrame	A DataFrame with the original, non-bootstrapped t-statistic and bootstrapped p-value, along with the bootstrap type, inference type (HC vs CRV), and whether the null hypothesis was imposed on the bootstrap DGP. If return_bootstrapped_t_stats is True, the method returns a tuple of the regular output and the bootstrapped t-stats.

Examples

import re
import pyfixest as pf

data = pf.get_data()
fit = pf.feols("Y ~ X1 + X2 | f1", data)

fit.wildboottest(
    param = "X1",
    reps=1000,
    seed = 822
)

fit.wildboottest(
    param = "X1",
    reps=1000,
    seed = 822,
    bootstrap_type = "31"
)

param                              X1
t value           -14.056663479651508
Pr(>|t|)                          0.0
bootstrap_type                     31
inference                     CRV(f1)
impose_null                      True
ssc                          1.067715
dtype: object

wls_transform

estimation.feols_.Feols.wls_transform()

Transform model matrices for WLS Estimation.