{ "config_digest": "1dc94f6ec0ea", "dataset_risks": [ { "active_deployments": 3, "dataset_name": "retail.core.dim_customers", "dataset_urn": "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.core.dim_customers,PROD)", "dependent_model_count": 2, "dependent_model_urns": [ "urn:li:mlModel:(urn:li:dataPlatform:mlflow,churn_predictor,PROD)", "urn:li:mlModel:(urn:li:dataPlatform:mlflow,fraud_detector_v3,PROD)" ], "rank": 1 } ], "errors": [], "findings": [ { "anchor_urn": "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.ml.device_signals,PROD)", "assertion_urn": "urn:li:assertion:recall-7322dbce62882e8b", "detector_id": "orphaned_feature", "evidence": { "feature": "device_risk_score", "feature_table": "fraud_features", "feature_urn": "urn:li:mlFeature:(fraud_features,device_risk_score)", "is_primary_key": false, "reason": "SOFT_DELETED", "source_urn": "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.ml.device_signals,PROD)" }, "id": "7322dbce62882e8b", "message": "'retail.ml.device_signals' is soft-deleted, but the feature 'device_risk_score' still declares it as a source. It has been removed from the catalog; any pipeline still writing it is doing so outside of governance.", "model_urns": [ "urn:li:mlModel:(urn:li:dataPlatform:mlflow,fraud_detector_v3,PROD)" ], "remediation": "- Repoint the feature `device_risk_score` in `fraud_features` (`urn:li:mlFeature:(fraud_features,device_risk_score)`) off `urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.ml.device_signals,PROD)` and onto `urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.ml.device_signals_v2,PROD)`, mapping the source column from `device_risk` to `risk_score`. The v1 deprecation note (\"Replaced by retail.ml.device_signals_v2 during the vendor migration\") makes v2 the intended successor, and Carol Whitfield (`urn:li:corpuser:carol`) is technical owner of both tables plus `fraud_detector_v3` \u2014 so this is a single-owner change, no cross-team handoff needed.\n\n- Before that repoint lands, get a decision from Carol on the score scale. v1 `device_risk` is documented as \"vendor-supplied risk score in [0,1]\"; v2 `risk_score` is documented as \"rescaled vendor risk score\" and v2 adds a `model_version` column that v1 did not have. If the scales differ, swapping the source silently shifts the input distribution under a live gradient-boosted classifier with two deployments. The decision Carol owns: rescale at the feature-materialization step to reproduce the v1 [0,1] range, or accept the v2 scale and retrain/recalibrate `fraud_detector_v3`. Confirm the actual value ranges in both tables rather than trusting the field descriptions.\n\n- Find and stop whatever is still writing `retail.ml.device_signals`. It is soft-deleted and out of the catalog, so any job still populating it is running outside governance and is invisible to lineage and to this detector on the next scan. Trace the writer from the Snowflake job/warehouse history for that table \u2014 DataHub will not have it, since the entity was removed.\n\n- Treat `device_signals_v2` as not-yet-clean before cutover: it currently reports 4 active incidents in DataHub. Have Carol confirm those incidents are unrelated to `risk_score` freshness or correctness, otherwise the fix moves `fraud_detector_v3` from a stale source onto a degraded one.", "scope": "MODEL", "severity": "CRITICAL", "subject_type": "dataset", "subject_urn": "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.ml.device_signals,PROD)", "title": "Feature 'device_risk_score' reads from a soft-deleted dataset" }, { "anchor_urn": "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.ml.device_signals_v2,PROD)", "assertion_urn": "urn:li:assertion:recall-4cbcdd9f7352bbcf", "detector_id": "training_serving_skew", "evidence": { "features_reading_it": [ "device_risk_score" ], "ignored_training_inputs": [ "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.ml.fraud_labels,PROD)" ], "reason": "SERVING_ONLY", "serving_sources": [ "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.core.dim_customers,PROD)", "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.ml.device_signals,PROD)", "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.ml.device_signals_v2,PROD)", "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.raw.transactions,PROD)" ], "training_inputs": [ "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.core.dim_customers,PROD)", "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.ml.device_signals,PROD)", "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.raw.transactions,PROD)" ] }, "id": "4cbcdd9f7352bbcf", "message": "A feature served to 'fraud_detector_v3' is derived from 'retail.ml.device_signals_v2', but the training run for this model never consumed it. The model is scoring on a data source it was not fitted against \u2014 structural training/serving skew.", "model_urns": [ "urn:li:mlModel:(urn:li:dataPlatform:mlflow,fraud_detector_v3,PROD)" ], "remediation": "I'll check what's actually known about the two device_signals tables and the feature before writing this.\nVersion skew is the crux here \u2014 let me confirm the column-level difference.\n- Repoint `device_risk_score` in the Feast `fraud_features` table off `device_signals_v2.risk_score` and back onto `retail.ml.device_signals.device_risk` \u2014 the exact column the training run consumed \u2014 or freeze the two live deployments until step 3 lands. Treat this as a stopgap measured in days, not a fix: `device_signals` is already marked deprecated (\"Replaced by retail.ml.device_signals_v2 during the vendor migration\"), carries a decommission timestamp within roughly a day of today, and is currently failing 1 of its 2 assertions (`urn:li:assertion:recall-7322dbce62882e8b`). Confirm that exact decommission date before committing to the rollback path.\n\n- Quantify the damage, because this is not a column rename. Training fitted on `device_signals.device_risk`, documented as \"Vendor-supplied risk score in [0,1]\"; serving reads `device_signals_v2.risk_score`, documented as \"Rescaled vendor risk score.\" The two are on different scales from different vendor scoring models. Join the tables on `device_id` over an overlap window and compare distributions, and read the distinct values of `device_signals_v2.model_version` to see how many vendor scoring versions have been served since cutover. That result decides whether every authorisation scored since the cutover needs re-review.\n\n- Retrain and re-register `fraud_detector_v3` against `device_signals_v2.risk_score` so the fitted and served inputs match, and record `retail.ml.device_signals_v2` on the new training run's lineage. Rolling forward is the only durable option once `device_signals` is decommissioned. Note that `retail.ml.fraud_labels` appearing as a training-only input is expected for a label source and needs no serving counterpart \u2014 it is not part of this defect.\n\n- Carol Whitfield (`urn:li:corpuser:carol`, ML Platform Engineer) is the sole listed owner of the model and both device tables, so she owns the rollback-versus-freeze call in step 1 and the retrain in step 3. The re-review decision from step 2 \u2014 whether fraud decisions made on rescaled scores get reopened \u2014 is a fraud-operations call, and DataHub shows no owner for that; identify the fraud-ops decision owner rather than defaulting it to Carol. The vendor's rescaling definition is also not recorded anywhere in the catalog and will need to come from whoever ran the vendor migration.", "scope": "MODEL", "severity": "CRITICAL", "subject_type": "dataset", "subject_urn": "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.ml.device_signals_v2,PROD)", "title": "Serving reads 'retail.ml.device_signals_v2', which training never consumed" }, { "anchor_urn": "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.raw.transactions,PROD)", "assertion_urn": "urn:li:assertion:recall-6774d68d4aa365b5", "detector_id": "training_source_schema_drift", "evidence": { "added": [ "device_fingerprint" ], "changed_since_ms": 1785026121694, "features_matching_changed_columns": [], "removed": [], "renullabled": [], "retyped": [ [ "amount", "NUMBER(10,2)", "FLOAT" ] ], "roles": [ "FEATURE_SOURCE", "TRAINING_INPUT" ], "status": "DRIFT_DETECTED" }, "id": "6774d68d4aa365b5", "message": "'retail.raw.transactions' backs a live serving feature for 'fraud_detector_v3' and has added `device_fingerprint`; retyped `amount` (NUMBER(10,2) \u2192 FLOAT). The model has not been retrained against this shape.", "model_urns": [ "urn:li:mlModel:(urn:li:dataPlatform:mlflow,fraud_detector_v3,PROD)" ], "remediation": "- Freeze the serving path for `txn_amount_zscore` (`urn:li:mlFeature:(fraud_features,txn_amount_zscore)`) \u2014 it is the only live feature downstream of the retyped column, and its 30-day rolling z-score per customer will shift as soon as unrounded FLOAT values enter the window. Carol Whitfield (`urn:li:corpuser:carol`, ML Platform Engineer, model technical owner) should have the feature transform cast `amount` back to 2-decimal fixed precision before z-scoring, so both live deployments keep scoring on the distribution v3 was fit against. This is the hotfix, not the fix.\n\n- Establish which shape the current model actually saw: check the training run `urn:li:dataProcessInstance:fraud_detector_v3_run_2026_07_14` against the retype timestamp (1785026121694). If the run predates the retype, v3 is serving on a type it never trained on and retraining is mandatory; if it postdates it, the drift is serving-side only and step 1 is sufficient until the next scheduled retrain. Carol owns this call and it gates everything below.\n\n- Get Alice Nakamura (`urn:li:corpuser:alice`, Staff Data Engineer, technical owner of `retail.raw.transactions`) to confirm whether the `NUMBER(10,2)` \u2192 `FLOAT` retype was intentional and whether historical rows were backfilled or only new rows are affected. The table currently has 3 active incidents and a failing assertion (`urn:li:assertion:recall-6774d68d4aa365b5`) \u2014 establish whether those are the same event before treating them as separate work. A backfill means past training data was rewritten and the v3 training set is no longer reproducible.\n\n- Decide explicitly to exclude `device_fingerprint` (VARCHAR(128), nullable) from the feature set for now, rather than leaving it undecided. It is a new column with no feature or lineage attached to it, and pulling a nullable high-cardinality string into a 40ms p99 inline scorer is a v4 modelling change with its own encoding and null-handling design \u2014 Carol should route it to the next model iteration, not this remediation. Confirm with Alice whether the field carries device-identity data that needs a PII classification before anyone builds on it.", "scope": "MODEL", "severity": "CRITICAL", "subject_type": "dataset", "subject_urn": "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.raw.transactions,PROD)", "title": "Schema of 'retail.raw.transactions' changed since the last scan" }, { "anchor_urn": "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.core.dim_customers,PROD)", "assertion_urn": "urn:li:assertion:recall-53b21eaa1be833b7", "detector_id": "blast_radius", "evidence": { "active_deployments": 3, "contributing_detectors": [ "ownership_vacuum" ], "dependent_model_count": 2, "dependent_models": [ "urn:li:mlModel:(urn:li:dataPlatform:mlflow,churn_predictor,PROD)", "urn:li:mlModel:(urn:li:dataPlatform:mlflow,fraud_detector_v3,PROD)" ], "rank": 1, "reason": "BLAST_RADIUS" }, "id": "53b21eaa1be833b7", "message": "'retail.core.dim_customers' is upstream of 2 production model(s) \u2014 churn_predictor, fraud_detector_v3 \u2014 across 3 live deployment(s), and already has 1 open finding(s) against it. One fix here clears all of them; one more regression breaks all of them.", "model_urns": [ "urn:li:mlModel:(urn:li:dataPlatform:mlflow,churn_predictor,PROD)", "urn:li:mlModel:(urn:li:dataPlatform:mlflow,fraud_detector_v3,PROD)" ], "remediation": "- Assign a named technical owner to `retail.core.dim_customers` today \u2014 it is the only table in `retail.core` with no owner (`retail.core.fct_orders` is owned by Bob Osei, Analytics Engineer). Bob is the obvious default: he owns the sibling fact table and `churn_predictor`. But `dim_customers` also feeds `fraud_detector_v3` (Carol Whitfield, ML Platform Engineer; risk tier CRITICAL, inline authorisation scoring at a 40ms p99 budget), so Bob and Carol need to decide together whether this is Bob's individual ownership or a group owner spanning both teams. The table also carries the `CustomerPII` glossary term, so whoever takes it inherits a PII surface \u2014 name a data steward alongside the technical owner rather than leaving that implicit.\n\n- Gate schema changes on the two columns that actually carry the blast radius: `customer_id` and `tenure_days`. Both feed `customer_tenure_days`, which exists twice downstream \u2014 once in `churn_features` and once in `fraud_features` \u2014 and `customer_id` is the primary key of both feature sets. Require review from both the churn and fraud owners on any PR touching those columns; a single rename breaks two feature sets and three deployments at once.\n\n- Attach real freshness and null-rate assertions to `dim_customers` so a break pages the new owner. The only two assertions on this table right now are the Recall detector findings themselves, and both have failed on all 8 runs \u2014 there is currently no data-quality signal that fires independently of the structural finding.\n\n- Have Carol confirm what the 3 live deployments actually are. `fraud_detector_v2` still sits downstream of `dim_customers` in lineage but is marked deprecated with the note \"Superseded by fraud_detector_v3 \u2014 retained for audit replay only, no live traffic.\" If v2 is being counted, the real live surface is smaller than reported; if it isn't, its stale lineage should still be retired so future blast-radius scoring stays accurate.", "scope": "ESTATE", "severity": "WARN", "subject_type": "dataset", "subject_urn": "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.core.dim_customers,PROD)", "title": "'retail.core.dim_customers' puts 2 production model(s) at risk" }, { "anchor_urn": "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.core.dim_customers,PROD)", "assertion_urn": "urn:li:assertion:recall-a54ed754ea764e21", "detector_id": "ownership_vacuum", "evidence": { "entity": "dataset", "raw_owners": [], "reason": "NO_OWNER", "roles": [ "FEATURE_SOURCE", "TRAINING_INPUT" ] }, "id": "a54ed754ea764e21", "message": "'retail.core.dim_customers' feeds 'fraud_detector_v3' (feature source, training input) but has no owner. If it breaks, there is nobody accountable for fixing it.", "model_urns": [ "urn:li:mlModel:(urn:li:dataPlatform:mlflow,churn_predictor,PROD)", "urn:li:mlModel:(urn:li:dataPlatform:mlflow,fraud_detector_v3,PROD)" ], "remediation": "- Assign a technical owner to `urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.core.dim_customers,PROD)` in DataHub, with Alice Nakamura (`urn:li:corpuser:alice`) as the proposed name \u2014 she is the technical owner of the only upstream, `retail.raw.customers_raw`, so the pipeline that produces this table already sits in her scope. This is her decision to accept or decline; if she declines, the escalation is to whoever owns the `retail.core` schema, which is not recorded in DataHub and needs to be established from the dbt/warehouse repo rather than guessed.\n\n- Until that ownership lands, name Carol Whitfield (`urn:li:corpuser:carol`) as the interim escalation contact for breakage. She owns `fraud_detector_v3`, which scores every authorisation inline against a 40ms p99 budget and carries the CRITICAL risk tier \u2014 she absorbs the incident either way, so she should be the one paged rather than discovering it from fraud metrics.\n\n- Have the accepted owner declare a change-notification path to both Carol (fraud_detector_v3) and Bob Osei (`urn:li:corpuser:bob`, churn_predictor) covering the `tenure_days` and `customer_id` columns specifically. Those two fields are not incidental: `tenure_days` materialises as `customer_tenure_days` in both `churn_features` and `fraud_features`, and `customer_id` is the primary key of both feature sets, so a silent type or nullability change there hits two production models at once.\n\n- Separately, decide who holds the business/steward ownership for the `CustomerPII` glossary term applied to this table \u2014 technical ownership does not cover PII accountability, and there is no data-governance owner recorded on the dataset. Confirm with your governance lead whether a steward is required on PII-tagged datasets before closing the finding.", "scope": "MODEL", "severity": "WARN", "subject_type": "dataset", "subject_urn": "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.core.dim_customers,PROD)", "title": "Upstream dataset 'retail.core.dim_customers' has no owner" }, { "anchor_urn": "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.ml.device_signals_v2,PROD)", "assertion_urn": "urn:li:assertion:recall-ca4e175480bed49a", "detector_id": "training_source_schema_drift", "evidence": { "field_count": 4, "status": "BASELINE_ESTABLISHED" }, "id": "ca4e175480bed49a", "message": "First scan of 'retail.ml.device_signals_v2' \u2014 its schema fingerprint has been recorded. Drift will be reported from the next scan onward.", "model_urns": [ "urn:li:mlModel:(urn:li:dataPlatform:mlflow,fraud_detector_v3,PROD)" ], "remediation": null, "scope": "MODEL", "severity": "BASELINE", "subject_type": "dataset", "subject_urn": "urn:li:dataset:(urn:li:dataPlatform:snowflake,retail.ml.device_signals_v2,PROD)", "title": "Baseline recorded for 'retail.ml.device_signals_v2'" } ], "model_risks": [ { "active_deployments": 2, "finding_ids": [ "4cbcdd9f7352bbcf", "6774d68d4aa365b5", "7322dbce62882e8b", "a54ed754ea764e21", "ca4e175480bed49a" ], "model_name": "fraud_detector_v3", "model_urn": "urn:li:mlModel:(urn:li:dataPlatform:mlflow,fraud_detector_v3,PROD)", "score": 100, "tier": "CRITICAL" }, { "active_deployments": 1, "finding_ids": [ "a54ed754ea764e21" ], "model_name": "churn_predictor", "model_urn": "urn:li:mlModel:(urn:li:dataPlatform:mlflow,churn_predictor,PROD)", "score": 14, "tier": "WATCH" } ], "models_scanned": 2, "scanned_at_ms": 1785026133881 }