Methods in Linker.table_management

Register Splink tables against your database backend and manage the Splink cache. Accessed via linker.table_management.

compute_tf_table(column_name)

Compute a term frequency table for a given column and persist to the database

This method is useful if you want to pre-compute term frequency tables e.g. so that real time linkage executes faster, or so that you can estimate various models without having to recompute term frequency tables each time

Examples:

Real time linkage
```py
linker = Linker(df, settings="saved_settings.json")
linker.table_management.compute_tf_table("surname")
linker.inference.score_pair(record_left, record_right)
```
Pre-computed term frequency tables
```py
linker = Linker(df, settings="saved_settings.json")
df_first_name_tf = linker.table_management.compute_tf_table("first_name")
df_first_name_tf.to_parquet("folder/first_name_tf.parquet")
# On subsequent data linking job, read this table rather than recompute
df_first_name_tf = db_api.register(
    pd.read_parquet("folder/first_name_tf.parquet")
)
linker.table_management.register_term_frequency_lookup(
    df_first_name_tf, "first_name"
)

```

Parameters:

Name	Type	Description	Default
column_name	str	The column name in the input table	required

Returns:

Name	Type	Description
SplinkDataFrame	SplinkDataFrame	The resultant table as a splink data frame

register_blocked_pairs_for_predict(input_data)

Register a pre-computed blocked pairs table to be scored by predict() rather than Splink computing blocking from the model's blocking rules.

Once registered, you can then linker.inference.predict() to produce predictions based on the registered blocked pairs. A subsequent registration replaces the previous one. The chunk-based prediction methods (predict_chunk() and predict(num_chunks_left=..., num_chunks_right=...)) are not supported once a table has been registered. If you want chunked predictions, use compute_blocked_pairs_for_predict_chunk, then register its output.

Parameters:

Name	Type	Description	Default
input_data	SplinkDataFrame	The blocked pairs table to register. Register raw data first with db_api.register() to obtain a SplinkDataFrame.	required

Returns:

Name	Type	Description
SplinkDataFrame	SplinkDataFrame	An abstraction representing the registered blocked pairs table.

Examples:

blocked_pairs = db_api.register(
    duckdb.read_parquet("path/to/blocked_pairs.parquet")
)
linker.table_management.register_blocked_pairs_for_predict(blocked_pairs)
predictions = linker.inference.predict()

invalidate_cache()

Invalidate the Splink cache. Any previously-computed tables will be recomputed. This is useful, for example, if the input data tables have changed.

register_table_predict(input_data)

Register a pre-computed version of the prediction table for use in Splink.

This method allows you to register a pre-computed prediction table in the Splink cache so it will be used rather than Splink computing the table anew.

Examples:

predict_df = db_api.register(pd.read_parquet("path/to/predict_df.parquet"))
predict_as_splinkdataframe = linker.table_management.register_table_predict(
    predict_df
)
clusters = linker.clustering.cluster_pairwise_predictions_at_threshold(
    predict_as_splinkdataframe, threshold_match_probability=0.75
)

Parameters:

Name	Type	Description	Default
input_data	SplinkDataFrame	The prediction table to register. Register raw data first with db_api.register() to obtain a SplinkDataFrame.	required

Returns:

Type	Description
SplinkDataFrame	An abstraction representing the registered table.

register_term_frequency_lookup(input_data, col_name)

Register a pre-computed term frequency lookup table for a given column.

This method allows you to register a term frequency table in the Splink cache for a specific column. This table will then be used during linkage rather than computing the term frequency table anew from your input data.

Parameters:

Name	Type	Description	Default
input_data	SplinkDataFrame	The term frequency table to register. Register raw data first with db_api.register() to obtain a SplinkDataFrame.	required
col_name	str	The name of the column for which the term frequency lookup table is being registered.	required

Returns:

Type	Description
SplinkDataFrame	An abstraction representing the registered term frequency table.

Examples:

tf_table = [
    {"first_name": "theodore", "tf_first_name": 0.012},
    {"first_name": "alfie", "tf_first_name": 0.013},
]
tf_df = db_api.register(pd.DataFrame(tf_table))
linker.table_management.register_term_frequency_lookup(
    tf_df,
    "first_name"
)

register_table(input_table, table_name, overwrite=False)

Register a table to your backend database, to be used in one of the splink methods, or simply to allow querying.

Tables can be of type: dictionary, record level dictionary, pandas dataframe, pyarrow table and in the spark case, a spark df.

Examples:

test_dict = {"a": [666,777,888],"b": [4,5,6]}
linker.table_management.register_table(test_dict, "test_dict")
linker.misc.query_sql("select * from test_dict")

Parameters:

Name	Type	Description	Default
input_table	AcceptableInputTableType	The data you wish to register. This can be either a dictionary, pandas dataframe, pyarrow table or a spark dataframe.	required
table_name	str	The name you wish to assign to the table.	required
overwrite	bool	Overwrite the table in the underlying database if it exists	False

Returns:

Name	Type	Description
SplinkDataFrame	SplinkDataFrame	An abstraction representing the table created by the sql pipeline