Documentation for blocking_rule_composition functions

blocking_composition allows the merging of existing blocking rules by a logical SQL clause - AND, OR or NOT.

This extends the functionality of our base blocking rules by allowing users to "join" existing comparisons by various SQL clauses.

For example, and_(block_on("first_name"), block_on("surname")) creates a dual check for an exact match where both first_name and surname are equal.

The detailed API for each of these are outlined below.

Library comparison composition APIs

and_(*brls, salting_partitions=1)

Merge BlockingRules using logical "AND".

Merge multiple BlockingRules into a single BlockingRule by merging their SQL conditions using a logical "AND".

Parameters:

Name	Type	Description	Default
*brls	BlockingRule | dict | str	BlockingRules or blocking rules in the string/dictionary format.	()
salting_partitions	(optional, int)	Whether to add salting to the blocking rule. More information on salting can be found within the docs. Salting is only valid for Spark.	1

Examples:

DuckDB Spark Athena SQLitePostgreSQL

Simple exact rule composition with an AND clause

import splink.duckdb.blocking_rule_library as brl
brl.and_(
    brl.exact_match_rule("first_name"),
    brl.exact_match_rule("surname")
)

Composing a custom rule with an exact match on name and the year from a date of birth column

import splink.duckdb.blocking_rule_library as brl
brl.and_(
    brl.exact_match_rule("first_name"),
    "substr(l.dob,1,4) = substr(r.dob,1,4)"
)

Simple exact rule composition with an AND clause

import splink.spark.blocking_rule_library as brl
brl.and_(
    brl.exact_match_rule("first_name"),
    brl.exact_match_rule("surname")
)

Composing a custom rule with an exact match on name and the year from a date of birth column, with additional salting (spark exclusive)

import splink.spark.blocking_rule_library as brl
brl.and_(
    brl.exact_match_rule("first_name"),
    "substr(l.dob,1,4) = substr(r.dob,1,4)",
    salting_partitions=5
)

Simple exact rule composition with an AND clause

import splink.athena.blocking_rule_library as brl
brl.and_(
    brl.exact_match_rule("first_name"),
    brl.exact_match_rule("surname")
)

Composing a custom rule with an exact match on name and the year from a date of birth column

import splink.athena.blocking_rule_library as brl
brl.and_(
    brl.exact_match_rule("first_name"),
    "substr(l.dob,1,4) = substr(r.dob,1,4)",
)

Simple exact rule composition with an AND clause

import splink.sqlite.blocking_rule_library as brl
brl.and_(
    brl.exact_match_rule("first_name"),
    brl.exact_match_rule("surname")
)

Composing a custom rule with an exact match on name and the year from a date of birth column

import splink.sqlite.blocking_rule_library as brl
brl.and_(
    brl.exact_match_rule("first_name"),
    "substr(l.dob,1,4) = substr(r.dob,1,4)",
)

Simple exact rule composition with an OR clause

import splink.postgres.blocking_rule_library as brl
brl.and_(
    brl.exact_match_rule("first_name"),
    brl.exact_match_rule("surname")
)

Composing a custom rule with an exact match on name and the year from a date of birth column

import splink.postgres.blocking_rule_library as brl
brl.and_(
    brl.exact_match_rule("first_name"),
    "substr(l.dob,1,4) = substr(r.dob,1,4)",
)

Returns:

Name	Type	Description
BlockingRule	BlockingRule	A new BlockingRule with the merged SQL condition

Source code in splink/blocking_rule_composition.py

def and_(
    *brls: BlockingRule | dict | str,
    salting_partitions=1,
) -> BlockingRule:
    """Merge BlockingRules using logical "AND".

    Merge multiple BlockingRules into a single BlockingRule by
    merging their SQL conditions using a logical "AND".


    Args:
        *brls (BlockingRule | dict | str): BlockingRules or
            blocking rules in the string/dictionary format.
        salting_partitions (optional, int): Whether to add salting
            to the blocking rule. More information on salting can
            be found within the docs. Salting is only valid for Spark.

    Examples:
        === ":simple-duckdb: DuckDB"
            Simple exact rule composition with an `AND` clause
            ``` python
            import splink.duckdb.blocking_rule_library as brl
            brl.and_(
                brl.exact_match_rule("first_name"),
                brl.exact_match_rule("surname")
            )
            ```
            Composing a custom rule with an exact match on name and the year
            from a date of birth column
            ``` python
            import splink.duckdb.blocking_rule_library as brl
            brl.and_(
                brl.exact_match_rule("first_name"),
                "substr(l.dob,1,4) = substr(r.dob,1,4)"
            )
            ```
        === ":simple-apachespark: Spark"
            Simple exact rule composition with an `AND` clause
            ``` python
            import splink.spark.blocking_rule_library as brl
            brl.and_(
                brl.exact_match_rule("first_name"),
                brl.exact_match_rule("surname")
            )
            ```
            Composing a custom rule with an exact match on name and the year
            from a date of birth column, with additional salting (spark exclusive)
            ``` python
            import splink.spark.blocking_rule_library as brl
            brl.and_(
                brl.exact_match_rule("first_name"),
                "substr(l.dob,1,4) = substr(r.dob,1,4)",
                salting_partitions=5
            )
            ```
        === ":simple-amazonaws: Athena"
            Simple exact rule composition with an `AND` clause
            ``` python
            import splink.athena.blocking_rule_library as brl
            brl.and_(
                brl.exact_match_rule("first_name"),
                brl.exact_match_rule("surname")
            )
            ```
            Composing a custom rule with an exact match on name and the year
            from a date of birth column
            ``` python
            import splink.athena.blocking_rule_library as brl
            brl.and_(
                brl.exact_match_rule("first_name"),
                "substr(l.dob,1,4) = substr(r.dob,1,4)",
            )
            ```
        === ":simple-sqlite: SQLite"
            Simple exact rule composition with an `AND` clause
            ``` python
            import splink.sqlite.blocking_rule_library as brl
            brl.and_(
                brl.exact_match_rule("first_name"),
                brl.exact_match_rule("surname")
            )
            ```
            Composing a custom rule with an exact match on name and the year
            from a date of birth column
            ``` python
            import splink.sqlite.blocking_rule_library as brl
            brl.and_(
                brl.exact_match_rule("first_name"),
                "substr(l.dob,1,4) = substr(r.dob,1,4)",
            )
            ```
        === "PostgreSQL"
            Simple exact rule composition with an `OR` clause
            ``` python
            import splink.postgres.blocking_rule_library as brl
            brl.and_(
                brl.exact_match_rule("first_name"),
                brl.exact_match_rule("surname")
            )
            ```
            Composing a custom rule with an exact match on name and the year
            from a date of birth column
            ``` python
            import splink.postgres.blocking_rule_library as brl
            brl.and_(
                brl.exact_match_rule("first_name"),
                "substr(l.dob,1,4) = substr(r.dob,1,4)",
            )
            ```

    Returns:
        BlockingRule: A new BlockingRule with the merged
            SQL condition
    """
    return _br_merge(
        *brls,
        clause="AND",
        salting_partitions=salting_partitions,
    )

not_(*brls, salting_partitions=1)

Invert a BlockingRule using "NOT".

Returns a BlockingRule with the same SQL condition as the input, but prefixed with "NOT".

Parameters:

Name	Type	Description	Default
*brls	BlockingRule | dict | str	BlockingRules or blocking rules in the string/dictionary format.	()
salting_partitions	(optional, int)	Whether to add salting to the blocking rule. More information on salting can be found within the docs. Salting is only valid for Spark.	1

Examples:

DuckDB Spark Athena SQLitePostgreSQL

Block where we do not have an exact match on first name

import splink.duckdb.blocking_rule_library as brl
brl.not_(brl.exact_match_rule("first_name"))

Block where we do not have an exact match on first name

import splink.spark.blocking_rule_library as brl
brl.not_(brl.exact_match_rule("first_name"))

Block where we do not have an exact match on first name

import splink.athena.blocking_rule_library as brl
brl.not_(brl.exact_match_rule("first_name"))

Block where we do not have an exact match on first name

import splink.sqlite.blocking_rule_library as brl
brl.not_(brl.exact_match_rule("first_name"))

Block where we do not have an exact match on first name

import splink.postgres.blocking_rule_library as brl
brl.not_(brl.exact_match_rule("first_name"))

Returns:

Name	Type	Description
BlockingRule	BlockingRule	A new BlockingRule with the merged SQL condition

Source code in splink/blocking_rule_composition.py

def not_(*brls: BlockingRule | dict | str, salting_partitions: int = 1) -> BlockingRule:
    """Invert a BlockingRule using "NOT".

    Returns a BlockingRule with the same SQL condition as the input,
    but prefixed with "NOT".

    Args:
        *brls (BlockingRule | dict | str): BlockingRules or
            blocking rules in the string/dictionary format.
        salting_partitions (optional, int): Whether to add salting
            to the blocking rule. More information on salting can
            be found within the docs. Salting is only valid for Spark.

    Examples:
        === ":simple-duckdb: DuckDB"
            Block where we do *not* have an exact match on first name
            ``` python
            import splink.duckdb.blocking_rule_library as brl
            brl.not_(brl.exact_match_rule("first_name"))
            ```
        === ":simple-apachespark: Spark"
            Block where we do *not* have an exact match on first name
            ``` python
            import splink.spark.blocking_rule_library as brl
            brl.not_(brl.exact_match_rule("first_name"))
            ```
        === ":simple-amazonaws: Athena"
            Block where we do *not* have an exact match on first name
            ``` python
            import splink.athena.blocking_rule_library as brl
            brl.not_(brl.exact_match_rule("first_name"))
            ```
        === ":simple-sqlite: SQLite"
            Block where we do *not* have an exact match on first name
            ``` python
            import splink.sqlite.blocking_rule_library as brl
            brl.not_(brl.exact_match_rule("first_name"))
            ```
        === "PostgreSQL"
            Block where we do *not* have an exact match on first name
            ``` python
            import splink.postgres.blocking_rule_library as brl
            brl.not_(brl.exact_match_rule("first_name"))
            ```

    Returns:
        BlockingRule: A new BlockingRule with the merged
            SQL condition
    """
    if len(brls) == 0:
        raise TypeError("You must provide at least one BlockingRule")
    elif len(brls) > 1:
        warnings.warning(
            "More than one BlockingRule entered for `NOT` composition. "
            "This function only accepts one argument and will only use your "
            "first BlockingRule.",
            SyntaxWarning,
            stacklevel=2,
        )

    brls, sql_dialect, salt = _parse_blocking_rules(*brls)
    br = brls[0]
    blocking_rule = f"NOT ({br.blocking_rule_sql})"

    br_dict = {
        "blocking_rule": blocking_rule,
        "sql_dialect": sql_dialect,
    }

    if salting_partitions > 1:
        salt = salting_partitions
    if salt > 1:
        br_dict["salting_partitions"] = salt

    return blocking_rule_to_obj(br_dict)

or_(*brls, salting_partitions=1)

Merge BlockingRules using logical "OR".

Merge multiple BlockingRules into a single BlockingRule by merging their SQL conditions using a logical "OR".

Parameters:

Name	Type	Description	Default
*brls	BlockingRule | dict | str	BlockingRules or blocking rules in the string/dictionary format.	()
salting_partitions	(optional, int)	Whether to add salting to the blocking rule. More information on salting can be found within the docs. Salting is only valid for Spark.	1

Examples:

DuckDB Spark Athena SQLitePostgreSQL

Simple exact rule composition with an OR clause

import splink.duckdb.blocking_rule_library as brl
brl.or_(brl.exact_match_rule("first_name"), brl.exact_match_rule("surname"))

Composing a custom rule with an exact match on name and the year from a date of birth column

import splink.duckdb.blocking_rule_library as brl
brl.or_(
    brl.exact_match_rule("first_name"),
    "substr(l.dob,1,4) = substr(r.dob,1,4)"
)

Simple exact rule composition with an OR clause

import splink.spark.blocking_rule_library as brl
brl.or_(brl.exact_match_rule("first_name"), brl.exact_match_rule("surname"))

Composing a custom rule with an exact match on name and the year from a date of birth column, with additional salting (spark exclusive)

import splink.spark.blocking_rule_library as brl
brl.or_(
    brl.exact_match_rule("first_name"),
    "substr(l.dob,1,4) = substr(r.dob,1,4)",
    salting_partitions=5
)

Simple exact rule composition with an OR clause

import splink.athena.blocking_rule_library as brl
brl.or_(brl.exact_match_rule("first_name"), brl.exact_match_rule("surname"))

Composing a custom rule with an exact match on name and the year from a date of birth column

import splink.athena.blocking_rule_library as brl
brl.or_(
    brl.exact_match_rule("first_name"),
    "substr(l.dob,1,4) = substr(r.dob,1,4)",
)

Simple exact rule composition with an OR clause

import splink.sqlite.blocking_rule_library as brl
brl.or_(brl.exact_match_rule("first_name"), brl.exact_match_rule("surname"))

Composing a custom rule with an exact match on name and the year from a date of birth column

import splink.sqlite.blocking_rule_library as brl
brl.or_(
    brl.exact_match_rule("first_name"),
    "substr(l.dob,1,4) = substr(r.dob,1,4)",
)

Simple exact rule composition with an OR clause

import splink.postgres.blocking_rule_library as brl
brl.or_(brl.exact_match_rule("first_name"), brl.exact_match_rule("surname"))

Composing a custom rule with an exact match on name and the year from a date of birth column

import splink.postgres.blocking_rule_library as brl
brl.or_(
    brl.exact_match_rule("first_name"),
    "substr(l.dob,1,4) = substr(r.dob,1,4)",
)

Returns:

Name	Type	Description
BlockingRule	BlockingRule	A new BlockingRule with the merged SQL condition

Source code in splink/blocking_rule_composition.py

def or_(
    *brls: BlockingRule | dict | str,
    salting_partitions: int = 1,
) -> BlockingRule:
    """Merge BlockingRules using logical "OR".

    Merge multiple BlockingRules into a single BlockingRule by
    merging their SQL conditions using a logical "OR".


    Args:
        *brls (BlockingRule | dict | str): BlockingRules or
            blocking rules in the string/dictionary format.
        salting_partitions (optional, int): Whether to add salting
            to the blocking rule. More information on salting can
            be found within the docs. Salting is only valid for Spark.

    Examples:
        === ":simple-duckdb: DuckDB"
            Simple exact rule composition with an `OR` clause
            ``` python
            import splink.duckdb.blocking_rule_library as brl
            brl.or_(brl.exact_match_rule("first_name"), brl.exact_match_rule("surname"))
            ```
            Composing a custom rule with an exact match on name and the year
            from a date of birth column
            ``` python
            import splink.duckdb.blocking_rule_library as brl
            brl.or_(
                brl.exact_match_rule("first_name"),
                "substr(l.dob,1,4) = substr(r.dob,1,4)"
            )
            ```
        === ":simple-apachespark: Spark"
            Simple exact rule composition with an `OR` clause
            ``` python
            import splink.spark.blocking_rule_library as brl
            brl.or_(brl.exact_match_rule("first_name"), brl.exact_match_rule("surname"))
            ```
            Composing a custom rule with an exact match on name and the year
            from a date of birth column, with additional salting (spark exclusive)
            ``` python
            import splink.spark.blocking_rule_library as brl
            brl.or_(
                brl.exact_match_rule("first_name"),
                "substr(l.dob,1,4) = substr(r.dob,1,4)",
                salting_partitions=5
            )
            ```
        === ":simple-amazonaws: Athena"
            Simple exact rule composition with an `OR` clause
            ``` python
            import splink.athena.blocking_rule_library as brl
            brl.or_(brl.exact_match_rule("first_name"), brl.exact_match_rule("surname"))
            ```
            Composing a custom rule with an exact match on name and the year
            from a date of birth column
            ``` python
            import splink.athena.blocking_rule_library as brl
            brl.or_(
                brl.exact_match_rule("first_name"),
                "substr(l.dob,1,4) = substr(r.dob,1,4)",
            )
            ```
        === ":simple-sqlite: SQLite"
            Simple exact rule composition with an `OR` clause
            ``` python
            import splink.sqlite.blocking_rule_library as brl
            brl.or_(brl.exact_match_rule("first_name"), brl.exact_match_rule("surname"))
            ```
            Composing a custom rule with an exact match on name and the year
            from a date of birth column
            ``` python
            import splink.sqlite.blocking_rule_library as brl
            brl.or_(
                brl.exact_match_rule("first_name"),
                "substr(l.dob,1,4) = substr(r.dob,1,4)",
            )
            ```
        === "PostgreSQL"
            Simple exact rule composition with an `OR` clause
            ``` python
            import splink.postgres.blocking_rule_library as brl
            brl.or_(brl.exact_match_rule("first_name"), brl.exact_match_rule("surname"))
            ```
            Composing a custom rule with an exact match on name and the year
            from a date of birth column
            ``` python
            import splink.postgres.blocking_rule_library as brl
            brl.or_(
                brl.exact_match_rule("first_name"),
                "substr(l.dob,1,4) = substr(r.dob,1,4)",
            )
            ```

    Returns:
        BlockingRule: A new BlockingRule with the merged
            SQL condition
    """
    return _br_merge(
        *brls,
        clause="OR",
        salting_partitions=salting_partitions,
    )