Implicit Filtering

Oftentimes our filtering logic involves applying the same logical expression to multiple columns. Implicit filtering allows us to succinctly apply one or more logical expressions to one or more columns and then combine the results without repeating code.

A typical implicit filtering solution looks like so:

df_2 = df.loc[(df[['col_1', 'col_3', 'col_5']]
               .apply(pd.isna)
               .any(axis=1))]

Implicit filtering involves three aspects:

  1. Selection of column(s) on each of which we will apply filtering logic which we cover in Column Selection.
  2. Specification of logical expression(s) or predicate function(s) (functions that return TRUE or FALSE) to apply to each of the selected columns which we cover in Function Specification.
  3. The choice of logical operation (either AND or OR) to use to combine the output of applying the specified expression(s) to the selected column(s) which we cover in Relationship Specification.
PYTHON
I/O