Deduplication using Pyspark
Linking in Spark¶
from pyspark import SparkConf, SparkContext
from pyspark.sql import SparkSession
from splink.backends.spark import similarity_jar_location
conf = SparkConf()
# This parallelism setting is only suitable for a small toy example
conf.set("spark.driver.memory", "12g")
conf.set("spark.default.parallelism", "8")
conf.set("spark.sql.codegen.wholeStage", "false")
# Add custom similarity functions, which are bundled with Splink
# documented here: https://github.com/moj-analytical-services/splink_scalaudfs
path = similarity_jar_location()
conf.set("spark.jars", path)
sc = SparkContext.getOrCreate(conf=conf)
spark = SparkSession(sc)
spark.sparkContext.setCheckpointDir("./tmp_checkpoints")
WARNING: Using incubator modules: jdk.incubator.vector
26/10/12 18:59:50 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j2-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
/home/runner/work/splink/splink/docs/demos/.venv/lib/python3.13/site-packages/pyspark/testing/utils.py:127: FutureWarning: PySpark does not yet fully support pandas >= 3.0.0. Some features may not work correctly. It is recommended to use pandas < 3.0.0 for now.
require_minimum_pandas_version()
from splink import splink_datasets
records = splink_datasets.fake_1000.to_pylist()
df = spark.createDataFrame(records)
import splink.comparison_library as cl
from splink import Linker, SettingsCreator, SparkAPI, block_on
settings = SettingsCreator(
link_type="dedupe_only",
comparisons=[
cl.NameComparison("first_name"),
cl.NameComparison("surname"),
cl.LevenshteinAtThresholds(
"dob"
),
cl.ExactMatch("city").configure(term_frequency_adjustments=True),
cl.EmailComparison("email"),
],
blocking_rules_to_generate_predictions=[
block_on("first_name"),
"l.surname = r.surname", # alternatively, you can write BRs in their SQL form
],
retain_intermediate_calculation_columns=True,
em_convergence=0.01,
)
db_api = SparkAPI(spark_session=spark)
df_sdf = db_api.register(df)
linker = Linker(df_sdf, settings)
deterministic_rules = [
"l.first_name = r.first_name and levenshtein(r.dob, l.dob) <= 1",
"l.surname = r.surname and levenshtein(r.dob, l.dob) <= 1",
"l.first_name = r.first_name and levenshtein(r.surname, l.surname) <= 2",
"l.email = r.email",
]
linker.training.estimate_probability_two_random_records_match(deterministic_rules, recall=0.6)
[Stage 0:> (0 + 4) / 8]
[Stage 0:> (0 + 7) / 8]
[Stage 4:> (0 + 5) / 8][Stage 5:> (0 + 0) / 8][Stage 6:> (0 + 0) / 8]
[Stage 5:> (0 + 4) / 8][Stage 6:> (0 + 0) / 8]
[Stage 5:=========> (4 + 4) / 8][Stage 6:> (0 + 0) / 8]
[Stage 6:=========> (4 + 4) / 8][Stage 8:> (0 + 0) / 1]
Probability two random records match is estimated to be 0.00389.
This means that amongst all possible pairwise record comparisons, one in 257.25 are expected to match. With 499,500 total possible comparisons, we expect a total of around 1,941.67 matching pairs
linker.training.estimate_u_using_random_sampling(max_pairs=5e5)
----- Estimating u probabilities using random sampling -----
Estimating u with: max_pairs = 500,000, min_count_per_level = 100, num_chunks = 10
[Stage 42:> (0 + 4) / 8]
[Stage 42:=============================> (4 + 4) / 8]
Estimating u for: first_name (Comparison 1 of 5)
Running probe chunk (~1.00% of max_pairs)
Min u_count: 6 for comparison level Jaro-Winkler distance of first_name >= 0.92 (cvv=3)
Probe did not converge; restarting with normal chunking
Running chunk 1/10
Count of 64 for level Jaro-Winkler distance of first_name >= 0.88 (cvv=2). Chunk took 1.0 seconds.
Min u_count not hit, continuing.
Running chunk 2/10
Count of 127 for level Jaro-Winkler distance of first_name >= 0.88 (cvv=2). Chunk took 0.7 seconds.
Exiting early since min count of 127 exceeds min_count_per_level = 100
Estimating u for: surname (Comparison 2 of 5)
Running probe chunk (~1.00% of max_pairs)
Min u_count: 3 for comparison level Jaro-Winkler distance of surname >= 0.88 (cvv=2)
Probe did not converge; restarting with normal chunking
Running chunk 1/10
Count of 39 for level Jaro-Winkler distance of surname >= 0.88 (cvv=2). Chunk took 0.7 seconds.
Min u_count not hit, continuing.
Running chunk 2/10
Count of 78 for level Jaro-Winkler distance of surname >= 0.88 (cvv=2). Chunk took 0.6 seconds.
Min u_count not hit, continuing.
Running chunk 3/10
Count of 121 for level Jaro-Winkler distance of surname >= 0.88 (cvv=2). Chunk took 0.7 seconds.
Exiting early since min count of 121 exceeds min_count_per_level = 100
Estimating u for: dob (Comparison 3 of 5)
Running probe chunk (~1.00% of max_pairs)
Min u_count: 9 for comparison level Exact match on dob (cvv=3)
Probe did not converge; restarting with normal chunking
Running chunk 1/10
Count of 76 for level Levenshtein distance of dob <= 1 (cvv=2). Chunk took 0.6 seconds.
Min u_count not hit, continuing.
Running chunk 2/10
Count of 152 for level Levenshtein distance of dob <= 1 (cvv=2). Chunk took 0.7 seconds.
Exiting early since min count of 152 exceeds min_count_per_level = 100
Estimating u for: city (Comparison 4 of 5)
Running probe chunk (~1.00% of max_pairs)
Min u_count: 178 for comparison level Exact match on city (cvv=1)
Exiting early since min count of 178 exceeds min_count_per_level = 100
Estimating u for: email (Comparison 5 of 5)
Running probe chunk (~1.00% of max_pairs)
Min u_count: 1 for comparison level Jaro-Winkler >0.88 on username (cvv=1)
Probe did not converge; restarting with normal chunking
Running chunk 1/10
Count of 14 for level Jaro-Winkler >0.88 on username (cvv=1). Chunk took 0.7 seconds.
Min u_count not hit, continuing.
Running chunk 2/10
Count of 26 for level Jaro-Winkler >0.88 on username (cvv=1). Chunk took 0.7 seconds.
Min u_count not hit, continuing.
Running chunk 3/10
Count of 47 for level Jaro-Winkler >0.88 on username (cvv=1). Chunk took 0.6 seconds.
Min u_count not hit, continuing.
Running chunk 4/10
Count of 63 for level Jaro-Winkler >0.88 on username (cvv=1). Chunk took 0.7 seconds.
Min u_count not hit, continuing.
Running chunk 5/10
Count of 72 for level Jaro-Winkler >0.88 on username (cvv=1). Chunk took 0.6 seconds.
Min u_count not hit, continuing.
Running chunk 6/10
Count of 92 for level Jaro-Winkler >0.88 on username (cvv=1). Chunk took 0.7 seconds.
Min u_count not hit, continuing.
Running chunk 7/10
Count of 104 for level Jaro-Winkler >0.88 on username (cvv=1). Chunk took 0.6 seconds.
Exiting early since min count of 104 exceeds min_count_per_level = 100
Estimated u probabilities using random sampling
Your model is not yet fully trained. Missing estimates for:
- first_name (no m values are trained).
- surname (no m values are trained).
- dob (no m values are trained).
- city (no m values are trained).
- email (no m values are trained).
training_blocking_rule = "l.first_name = r.first_name and l.surname = r.surname"
training_session_fname_sname = (
linker.training.estimate_parameters_using_expectation_maximisation(training_blocking_rule)
)
training_blocking_rule = "l.dob = r.dob"
training_session_dob = linker.training.estimate_parameters_using_expectation_maximisation(
training_blocking_rule
)
----- Starting EM training session -----
[EM sampling] max_pairs is None — no sampling will be applied
Estimating the m probabilities of the model by blocking on:
l.first_name = r.first_name and l.surname = r.surname
Parameter estimates will be made for the following comparison(s):
- dob
- city
- email
Parameter estimates cannot be made for the following comparison(s) since they are used in the blocking rules:
- first_name
- surname
[Stage 232:==============> (27 + 4) / 100]
[Stage 232:===================> (36 + 4) / 100]
[Stage 232:=========================> (47 + 4) / 100]
[Stage 232:===============================> (58 + 4) / 100]
[Stage 232:=====================================> (69 + 4) / 100]
[Stage 232:==========================================> (78 + 4) / 100]
[Stage 232:================================================> (89 + 4) / 100]
[Stage 237:> (0 + 4) / 8][Stage 238:> (0 + 0) / 8]
[Stage 238:> (0 + 4) / 8]
WARNING: Level Jaro-Winkler >0.88 on username on comparison email not observed in dataset, unable to train m value
Iteration 1: Largest change in params was -0.538 in the m_probability of dob, level `Exact match on dob`
[Stage 247:> (0 + 4) / 8]
Iteration 2: Largest change in params was 0.0328 in probability_two_random_records_match
[Stage 255:========> (4 + 4) / 8][Stage 256:> (0 + 0) / 8]
[Stage 256:===================================> (5 + 3) / 8]
Iteration 3: Largest change in params was 0.00944 in probability_two_random_records_match
EM converged after 3 iterations
m probability not trained for email - Jaro-Winkler >0.88 on username (comparison vector value: 1). This usually means the comparison level was never observed in the training data.
Your model is not yet fully trained. Missing estimates for:
- first_name (no m values are trained).
- surname (no m values are trained).
- email (some m values are not trained).
----- Starting EM training session -----
[EM sampling] max_pairs is None — no sampling will be applied
Estimating the m probabilities of the model by blocking on:
l.dob = r.dob
Parameter estimates will be made for the following comparison(s):
- first_name
- surname
- city
- email
Parameter estimates cannot be made for the following comparison(s) since they are used in the blocking rules:
- dob
[Stage 286:==============> (26 + 4) / 100]
[Stage 286:===================> (37 + 4) / 100]
[Stage 286:===========================> (51 + 4) / 100]
[Stage 286:==================================> (63 + 4) / 100]
[Stage 286:=========================================> (76 + 4) / 100]
[Stage 286:==============================================> (87 + 4) / 100]
[Stage 291:=> (5 + 3) / 8][Stage 292:> (0 + 1) / 8][Stage 293:> (0 + 0) / 8]
[Stage 292:============> (6 + 2) / 8][Stage 293:> (0 + 2) / 8]
WARNING: Level Jaro-Winkler >0.88 on username on comparison email not observed in dataset, unable to train m value
Iteration 1: Largest change in params was -0.413 in the m_probability of surname, level `Exact match on surname`
[Stage 303:> (0 + 4) / 8][Stage 304:> (0 + 0) / 8]
[Stage 304:============================> (4 + 4) / 8]
Iteration 2: Largest change in params was 0.0882 in probability_two_random_records_match
[Stage 314:====> (2 + 5) / 8][Stage 315:> (0 + 0) / 8]
[Stage 315:============================> (4 + 4) / 8]
Iteration 3: Largest change in params was 0.0503 in the m_probability of first_name, level All other comparisons
[Stage 325:==========> (5 + 3) / 8][Stage 326:> (0 + 1) / 8]
Iteration 4: Largest change in params was 0.0173 in probability_two_random_records_match
[Stage 337:=======> (1 + 4) / 8]
Iteration 5: Largest change in params was 0.00709 in probability_two_random_records_match
EM converged after 5 iterations
m probability not trained for email - Jaro-Winkler >0.88 on username (comparison vector value: 1). This usually means the comparison level was never observed in the training data.
Your model is not yet fully trained. Missing estimates for:
- email (some m values are not trained).
results = linker.inference.predict(threshold_match_probability=0.9)
Blocking time: 0.86 seconds
[Stage 373:==================> (35 + 4) / 100]
[Stage 373:========================> (45 + 5) / 100]
[Stage 373:==============================> (57 + 4) / 100]
[Stage 373:======================================> (71 + 4) / 100]
[Stage 373:============================================> (83 + 4) / 100]
[Stage 373:==================================================> (94 + 4) / 100]
Predict time (post-blocking): 3.12 seconds
-- WARNING --
You have called predict(), but there are some parameter estimates which have neither been estimated or specified in your settings dictionary. To produce predictions the following untrained parameters will use default values.
Comparison: 'email':
m values not fully trained
spark_df = results.as_spark_dataframe().show()
+------------------+------------------+-----------+-----------+------------+------------+----------------+--------------------+--------------------+-----------------+--------------------+---------+---------+-------------+--------------------+--------------------+-------------------+--------------------+----------+----------+---------+-------------------+-------------------+--------------------+----------+--------------------+--------------------+------------------+-------------------+--------------------+--------------------+-----------+--------------------+--------------------+------------------+--------------------+---------+
| match_weight| match_probability|unique_id_l|unique_id_r|first_name_l|first_name_r|gamma_first_name| tf_first_name_l| tf_first_name_r| mw_first_name|mw_tf_adj_first_name|surname_l|surname_r|gamma_surname| tf_surname_l| tf_surname_r| mw_surname| mw_tf_adj_surname| dob_l| dob_r|gamma_dob| mw_dob| city_l| city_r|gamma_city| tf_city_l| tf_city_r| mw_city| mw_tf_adj_city| email_l| email_r|gamma_email| tf_email_l| tf_email_r| mw_email| mw_tf_adj_email|match_key|
+------------------+------------------+-----------+-----------+------------+------------+----------------+--------------------+--------------------+-----------------+--------------------+---------+---------+-------------+--------------------+--------------------+-------------------+--------------------+----------+----------+---------+-------------------+-------------------+--------------------+----------+--------------------+--------------------+------------------+-------------------+--------------------+--------------------+-----------+--------------------+--------------------+------------------+--------------------+---------+
| 5.565475965544526|0.9793200520557831| 183| 186| Freddie| Freddie| 4|0.018050541516245487|0.018050541516245487|6.460616520200271| -1.6832507489521973| Hsmel| Holmes| 0|0.001221001221001221|0.006105006105006105|-2.2015459328880236| 0.0|1992-10-01|1993-11-03| 0|-1.9832296661200375| Kingston-upon-Hull| Kingston-upon-Hull| 1|0.015990159901599015|0.015990159901599015|3.3881408256585974| 1.7550295714628668| f@holmes@duncan.com| fholmes@duncan.com| 2|0.001267427122940...|0.005069708491761723| 7.831141712874357| 0.0| 0|
| 7.106507458659787|0.9927957627683142| 245| 248| Joshua| Joshua| 4|0.006016847172081829|0.006016847172081829|6.460616520200271|-0.09828824823104121| NULL| Williams| -1| NULL|0.009768009768009768| 0.0| 0.0|2002-08-08|2012-08-05| 1| 3.709350310176913| Portsmouth| Portsmouth| 1|0.017220172201722016|0.017220172201722016|3.3881408256585974| 1.6481143675463548|j.williams@levine...| NULL| -1|0.007604562737642...| NULL| 0.0| 0.0| 0|
| 21.6951148684932|0.9999997054767819| 37| 43| Theodore| Theodore| 4|0.012033694344163659|0.012033694344163659|6.460616520200271| -1.0982882482310412| Morris| Morris| 4|0.004884004884004884|0.004884004884004884| 6.546093313061104|-0.06060932114656...|1978-08-19|1978-08-19| 3| 7.876175888067744| Birmingham| Birmingham| 1| 0.04920049200492005| 0.04920049200492005|3.3881408256585974|0.13354119471659676|t.m39@brooks-sawy...|t.m39@brooks-sawy...| 4|0.006337135614702...|0.006337135614702...|7.9696737762639405| -1.5188027634061418| 0|
|16.730438069156676|0.9999908032939303| 321| 323| NULL| Amelia| -1| NULL|0.010830324909747292| 0.0| 0.0| Fletcher| Fletcher| 4|0.004884004884004884|0.004884004884004884| 6.546093313061104|-0.06060932114656...|1987-04-28|1987-04-29| 2| 6.629971170925725| Bristol| Bristol| 1| 0.01968019680196802| 0.01968019680196802|3.3881408256585974| 1.4554692896039585|ameliafletcher22@...|ameliafletcher22@...| 4|0.005069708491761723|0.005069708491761723|7.9696737762639405| -1.19687466851878| 1|
|19.104474749424973| 0.999998225895169| 798| 799| Arlo| Arlo| 4|0.006016847172081829|0.006016847172081829|6.460616520200271|-0.09828824823104121| West| NULL| -1|0.004884004884004884| NULL| 0.0| 0.0|2007-02-17|2007-02-17| 3| 7.876175888067744| Southend-on-Sea| Southend-on-Sea| 1|0.017220172201722016|0.017220172201722016|3.3881408256585974| 1.6481143675463548| aw@thompson.com| awt.thompson@com| 2|0.005069708491761723|0.001267427122940...| 7.831141712874357| 0.0| 0|
| 6.192931599063804|0.9865151663318595| 637| 638| Jacob| Jacob| 4|0.019253910950661854|0.019253910950661854|6.460616520200271| -1.7763601533436786| Adams| Adams| 4|0.006105006105006105|0.006105006105006105| 6.546093313061104|-0.38253741603392477|2000-09-03|1990-10-06| 0|-1.9832296661200375| Lonnod| London| 0|0.007380073800738007| 0.21279212792127922|-1.121095694866421| 0.0| jacob.a83@clark.com| jacob.a83@clark.com| 4|0.006337135614702...|0.006337135614702...|7.9696737762639405| -1.5188027634061418| 0|
| 17.1721369788879|0.9999932287517092| 226| 228| Julia| Julia| 4|0.003610108303249...|0.003610108303249...|6.460616520200271| 0.6386773459351653| Smith| Smith| 4|0.013431013431013432|0.013431013431013432| 6.546093313061104| -1.5200409397838595|2014-05-24|2004-04-26| 0|-1.9832296661200375| Luton| Luton| 1|0.003690036900369...|0.003690036900369...|3.3881408256585974| 3.8705067888828033|julia.smith@engli...|julia.smith@engli...| 4|0.002534854245880...|0.002534854245880...|7.9696737762639405|-0.19687466851877922| 0|
| 9.884957362443446|0.9989434933123434| 802| 803| Arlo| NULL| -1|0.006016847172081829| NULL| 0.0| 0.0| West| West| 4|0.004884004884004884|0.004884004884004884| 6.546093313061104|-0.06060932114656...|2007-03-19|2007-02-17| 1| 3.709350310176913| NULL| Southena-on-Sed| -1| NULL|0.002460024600246...| 0.0| 0.0| aw@thompson.com| aw@thpmoson.com| 3|0.005069708491761723|0.001267427122940...| 7.691549377043299| 0.0| 1|
|15.749548907743197| 0.999981848794339| 364| 365| Samuel| Samuel| 4|0.004813477737665...|0.004813477737665...|6.460616520200271| 0.22363984665632142| Campbell| Campbell| 4|0.015873015873015872|0.015873015873015872| 6.546093313061104| -1.7610490392876548|2018-09-24|2018-09-23| 2| 6.629971170925725|Newcastle-upon-Tyne|Newcastle-upon--Tyne| 0| 0.02214022140221402|0.001230012300123...|-1.121095694866421| 0.0|samuelcampbell35@...|samuelcampbell35@...| 4|0.005069708491761723|0.005069708491761723|7.9696737762639405| -1.19687466851878| 0|
|15.012538708188497| 0.999969747421325| 302| 304| Charlie| Charlie| 4|0.007220216606498195|0.007220216606498195|6.460616520200271|-0.36132265406483466| Andrews| Andrews| 4|0.004884004884004884|0.004884004884004884| 6.546093313061104|-0.06060932114656...|2023-11-01|2023-12-01| 2| 6.629971170925725| Liverpool| Liverpool| 1| 0.04059040590405904| 0.04059040590405904|3.3881408256585974| 0.4110751702455051|candrews15@thomps...| NULL| -1|0.002534854245880...| NULL| 0.0| 0.0| 0|
|11.878858546601244|0.9997345443569495| 40| 43| Theodore| Theodore| 4|0.012033694344163659|0.012033694344163659|6.460616520200271| -1.0982882482310412| Mosrir| Morris| 1|0.001221001221001221|0.004884004884004884| 2.558210102406218| 0.0|1978-09-19|1978-08-19| 2| 6.629971170925725| Birminghaim| Birmingham| 0|0.001230012300123...| 0.04920049200492005|-1.121095694866421| 0.0|t.m39@brooks-sawy...|t.m39@brooks-sawy...| 4|0.006337135614702...|0.006337135614702...|7.9696737762639405| -1.5188027634061418| 0|
| 16.34226563025483|0.9999879639877528| 182| 184| Freddie| Freddie| 4|0.018050541516245487|0.018050541516245487|6.460616520200271| -1.6832507489521973| Holmes| Holmes| 4|0.006105006105006105|0.006105006105006105| 6.546093313061104|-0.38253741603392477|1993-10-02|1993-10-04| 2| 6.629971170925725| Kingston-upon-Hull| NULL| -1|0.015990159901599015| NULL| 0.0| 0.0| fholmes@duncan.com| fholmes@duncan.com| 4|0.005069708491761723|0.005069708491761723|7.9696737762639405| -1.19687466851878| 0|
|11.468406077067378|0.9996472130368521| 758| 761| Henry| Henry| 4|0.010830324909747292|0.010830324909747292|6.460616520200271| -0.9462851547859916| Dy| Day| 0|0.001221001221001221|0.004884004884004884|-2.2015459328880236| 0.0|2002-09-15|2002-08-18| 1| 3.709350310176913| Leeds| Leeds| 1|0.020910209102091022|0.020910209102091022|3.3881408256585974| 1.3680064483536194|hday48@homas-cart...|hday48@thomas-car...| 3|0.001267427122940...|0.001267427122940...| 7.691549377043299| 0.0| 0|
| 6.081566927139319|0.9854487607307095| 707| 709| Maya| Maya| 4|0.004813477737665...|0.004813477737665...|6.460616520200271| 0.22363984665632142| NULL| Curtis| -1| NULL|0.004884004884004884| 0.0| 0.0|2015-07-29|2014-08-28| 0|-1.9832296661200375|Newcastle-upon-Tyne| Newcastle-upon-Tyne| 1| 0.02214022140221402| 0.02214022140221402|3.3881408256585974| 1.2855442881616463|mcurtis53@spmniso...|micurtis53@simpso...| 1|0.001267427122940...|0.001267427122940...| 4.708281429273828| 0.0| 0|
| 6.769215450916809|0.9909155270569916| 435| 436| Hugo| NULL| -1|0.004813477737665...| NULL| 0.0| 0.0| Kaur| Kaur| 4|0.009768009768009768|0.009768009768009768| 6.546093313061104| -1.0606093211465621|1973-03-23|1973-03-23| 3| 7.876175888067744| London| London| 1| 0.21279212792127922| 0.21279212792127922|3.3881408256585974| -1.979158938032766| NULL| h.k@sharp.com| -1| NULL|0.003802281368821...| 0.0| 0.0| 1|
| 17.75443583094865|0.9999954774820977| 155| 157| Auutsin| Austin| 2|0.001203369434416...|0.002406738868832...|5.376290662075984| 0.0| Wright| Wright| 4| 0.01098901098901099| 0.01098901098901099| 6.546093313061104| -1.2305343225888752|1993-03-27|1993-03-27| 3| 7.876175888067744| Manchester| NULL| -1|0.020910209102091022| NULL| 0.0| 0.0|a.wright42@estrad...|a.wright42@estrad...| 4|0.003802281368821...|0.003802281368821...|7.9696737762639405| -0.7818371692399371| 1|
| 9.399337871142736|0.9985213182839723| 226| 227| Julia| Julia| 4|0.003610108303249...|0.003610108303249...|6.460616520200271| 0.6386773459351653| Smith| Smith| 4|0.013431013431013432|0.013431013431013432| 6.546093313061104| -1.5200409397838595|2014-05-24|2004-04-27| 0|-1.9832296661200375| Luton| Luton| 1|0.003690036900369...|0.003690036900369...|3.3881408256585974| 3.8705067888828033|julia.smith@engli...| NULL| -1|0.002534854245880...| NULL| 0.0| 0.0| 0|
| 6.988239798359205|0.9921851092935396| 814| 819| Logan| Logan| 4|0.012033694344163659|0.012033694344163659|6.460616520200271| -1.0982882482310412| Morgan| Morgan| 4|0.014652014652014652|0.014652014652014652| 6.546093313061104| -1.645571821867719|1978-02-01|1976-12-30| 0|-1.9832296661200375| Cventry| Coventry| 0|0.001230012300123...|0.025830258302583026|-1.121095694866421| 0.0|loganmo.gan43@mcb...|loganmorgan43@icb...| 2|0.001267427122940...|0.001267427122940...| 7.831141712874357| 0.0| 0|
|11.959534618967593|0.9997489776943785| 756| 758| Henry| Henry| 4|0.010830324909747292|0.010830324909747292|6.460616520200271| -0.9462851547859916| NULL| Dy| -1| NULL|0.001221001221001221| 0.0| 0.0|2002-09-15|2002-09-15| 3| 7.876175888067744| Lesde| Leeds| 0|0.001230012300123...|0.020910209102091022|-1.121095694866421| 0.0|hday48@thomas-car...|hday48@homas-cart...| 3|0.003802281368821...|0.001267427122940...| 7.691549377043299| 0.0| 0|
|23.946653635489163|0.9999999381501097| 861| 864| Lily| Lily| 4|0.007220216606498195|0.007220216606498195|6.460616520200271|-0.36132265406483466| Robinson| Robinson| 4|0.004884004884004884|0.004884004884004884| 6.546093313061104|-0.06060932114656...|2009-07-06|2009-07-06| 3| 7.876175888067744| Salford| Salford| 1|0.017220172201722016|0.017220172201722016|3.3881408256585974| 1.6481143675463548|lily.robinson73@p...|lily.robinson73@p...| 4|0.006337135614702...|0.006337135614702...|7.9696737762639405| -1.5188027634061418| 0|
+------------------+------------------+-----------+-----------+------------+------------+----------------+--------------------+--------------------+-----------------+--------------------+---------+---------+-------------+--------------------+--------------------+-------------------+--------------------+----------+----------+---------+-------------------+-------------------+--------------------+----------+--------------------+--------------------+------------------+-------------------+--------------------+--------------------+-----------+--------------------+--------------------+------------------+--------------------+---------+
only showing top 20 rows