Published-dataset benchmarks

Public petroleum-engineering reference datasets compiled from cited SPE / URTeC / GRI literature. Where a scored solver is wired, the card runs it across every case and reports MAE, RMSE, MAPE, and an A-D trust grade; evidence-only datasets show “Not graded”. A grade D means the model does not reproduce the data. Open a card for the per-case table + CSV export.

Compare with your field values →

How learning models are tested

Whole stages, wells or campaigns are held out. Several outcomes are scored together; pressure-only matches are labelled as such. Anything built from test results is never an input to a blind prediction.

Model	Before	Now	Outcomes	Leak check	Status
Workspace surrogate (hold-out error)	Round-robin by row index — neighbouring rows of one stage could land in train and test	Whole groups (stage/well id) held out; falls back to source id only when no group is given	One objective per model	Group-leak guard on every fold	fixed
FORGE crosswell blind run	Per-stage blind predictions	Unchanged	Frac-hit time; fiber zones reference-only	2025 fracture network shown as reference, never an input	ok
Stage-outcome learning (retrained priors)	Per completed stage; accuracy only measured on wells already in the priors	Whole-well hold-out check (wellHoldoutPriorCheck): priors learned on other wells, scored on unseen wells	ISIP, half-length, efficiency, screen-out, offset hits together	Half-length tagged interpreted; flagged when its pressure match reused the scored ISIP	fixed
Production forecast (catalog match, EUR bands)	A test curve could match its own catalog entry	Leave-one-curve-out: 21 curves, mean P50 error 23%, 17 of 21 inside P10–P90	EUR only — labelled single-outcome; MSEEL/Wolfcamp early-life underestimate reported as-is	Self-match excluded	fixed
Fit bake-off, pad-plan prediction, AI scenario chains	Fit bake-off: row folds on one dataset; pad plan and AI chains: no field data	Fit bake-off labelled 'single-dataset fit, not a field validation'; pad plan and AI chains labelled 'simulated, not validated'	One objective per run	No group id in the data to split on — stated, not claimed	ok
PGD encoding, physics-loss trainer	No field data (encodes or solves a known field)	Not applicable	Residual vs exact solution	Not applicable	ok

Oil-and-gas benchmarks still needed

FORGE, Collab and Bedretto results are hot crystalline rock. They do not show accuracy in shale, proppant placement or production. None of the entries below is graded yet.

Area	Source	Access	Would grade	Missing
Shale geometry	HFTS-1 Midland (Wolfcamp)	partly-in	Height and length from through-frac core hits	Native treatment records (current values are chart-derived)
Shale geometry	HFTS-2 Delaware	partly-in	Geometry vs cores, fiber and pressure	Pumping for 50 B3H/B4H stages is in; full perforation spec, B1H/B2H pumping and oil production are not
Shale geometry	MSEEL Marcellus (MIP-3H)	partly-in	Fiber-based length and cluster intake	Stage-level pumping records linked to fiber
Proppant placement	HFTS-1 slant-core proppant counts	partly-in	Where proppant ended up vs predicted	Counts are in; sample-to-stage assignment and a proppant-by-depth model output are not
Proppant placement	Published lab slot-flow tests	available	Settling bank height and transport distance	Selecting and digitizing sourced tables
Production	MSEEL / HFTS public production	partly-in	Production vs predicted from geometry	MSEEL gas histories are in; HFTS-2 oil histories and downtime records are not
Production	Sumatra and Eagle Ford papers (already used)	partly-in	Productivity ratio and production-derived half-length	More wells — two cases are not a benchmark

Shale, proppant and production data — first checks

Public HFTS-1, HFTS-2 and MSEEL records, frozen unchanged with their sources. Nothing here proves fracture-model accuracy in shale yet.

Area	Source	Status	Why
Proppant placement	HFTS-1 GTI slant-core counts	ingested, descriptive only	No model output of proppant by depth yet, and samples are not yet tied to stages
Shale pumping	HFTS-2 B3H/B4H 50 stages	ingested, consistency-checked	Fiber cluster shares reuse pumped totals — reference only, never a grading target
Production	MSEEL MIP-3H/4H/5H/6H gas	blind future-interval check	Decline fit on first 12 full months only; not a geometry-based prediction
Oil production	12 New Mexico Wolfcamp wells (GO-TECH monthly)	blind future-interval check	Not HFTS wells, shale not verified; decline fit only, 36-month horizon, not EUR
Shut-in response	HFTS-2 B3H/B4H Sept 2020 gauges + published schedule	candidate, ungraded	Events are inputs, gauge pressure is the held-out target; units, clock and rate limits not verified
Fiber windows	HFTS-2 B1H→B3H, B2H→B4H LF-DAS	ingested, reference only	276 five-minute windows frozen with checksums; different treatments from the pumping set — never joined to it

Production: blind decline forecast (MSEEL Marcellus gas)

Fitted on the first 12 full months only (≥ 20 producing days), then compared with every later full month. Partial months are left out, not filled. A decline fit, not a prediction from fracture geometry.

Well	Start	Scored months	Observed MMcf	Forecast MMcf	Error
MIP3H	2015-12	51 (+5 partial)	4,378	498	-89%
MIP4H	2012-01	75 (+10 partial)	2,008	2,018	0%
MIP5H	2016-01	48 (+3 partial)	4,400	1,189	-73%
MIP6H	2012-01	71 (+13 partial)	976	900	-8%

Oil production: blind decline forecast (12 New Mexico Wolfcamp wells)

State-reported monthly volumes. These are not HFTS wells and the rock is not confirmed shale. Each well is fitted on its first 12 full months, then scored on full months in the next 36. Only months with ≥ 20 producing days count; zero and flagged rows are left out, not filled. A decline fit, not total recovery and not a fracture-model prediction.

Well	Group	Start	Scored months	Oil error	Gas error
CARBON VALLEY 25 FEDERAL COM #004H	CARBON_VALLEY	2009-12	31	14%	-0%
CARBON VALLEY 25 FEDERAL COM #005H	CARBON_VALLEY	2010-12	33	-21%	-27%
CARBON VALLEY 25 FEDERAL COM #006H	CARBON_VALLEY	2010-03	33	-27%	-53%
CARBON VALLEY 25 FEDERAL COM #007H	CARBON_VALLEY	2011-03	36	-63%	-45%
CARBON VALLEY 25 FEDERAL COM #008H	CARBON_VALLEY	2012-11	0	not scored	not scored
CARBON VALLEY 26 FEDERAL COM #001H	CARBON_VALLEY	2010-07	36	-26%	-88%
CARBON VALLEY 26 FEDERAL COM #002H	CARBON_VALLEY	2011-01	35	75%	-37%
CARBON VALLEY 31 FEDERAL COM #001H	CARBON_VALLEY	2010-08	35	-13%	-29%
CHILE VERDE 23 24 FEDERAL COM #001H	CHILE_VERDE	2023-05	24	-30%	9%
CHILE VERDE 23 24 FEDERAL COM #716H	CHILE_VERDE	2023-05	21	-46%	11%
CHILE VERDE 23 24 W0DA FEDERAL COM #001H	CHILE_VERDE	2022-09	24	-64%	-65%
CHILE VERDE 23 24 W0EH FEDERAL COM #001H	CHILE_VERDE	2022-09	28	-3%	-38%

Groups are nearby-property labels, not confirmed pads. One completion record checked so far: Carbon Valley 4H, 10 open-hole stages, 1,073,140 lb sand, 35,709 bbl gel (state filing, page 2). Not split into stages.

Completion records: the other 11 New Mexico wells

Operator-reported totals from state/federal completion filings, matched to production by API number. Totals only: no pressure, rate or timing history, so nothing here is graded. Conflicts are kept as printed.

Well	Record	Stages	Sand	Open issues
Carbon Valley 25 Federal Com 5H	Completion report, 10 stage rows	10	1,085,060 lb	none
Carbon Valley 25 Federal Com 6H	Completion report	11	1,078,927 lb	TVD conflict between reports
Carbon Valley 25 Federal Com 7H	Completion report, 11 stage rows	11	1,111,810 lb	stage total conflict, stage depth boundary, stage depth boundary
Carbon Valley 25 Federal Com 8H	Completion report	20	1,097,207 (unit not printed)	proppant quantity unit not printed
Carbon Valley 26 Federal Com 1H	Completion report, 11 stage rows	11	1,163,059 lb	cover vs stage table water conflict, proppant description conflict
Carbon Valley 26 Federal Com 2H	Completion report	11	1,171,352 lb	completion identifier suffix
Carbon Valley 31 Federal Com 1H	Completion report, 11 stage rows	11	1,148,280 lb	stage depth boundary
Chile Verde 23-24 Fed Com 718H W0MP	Partial records — formal completion report not recovered	—	—	shared disposal plan is not operating log
Chile Verde 23-24 Fed Com 716H W0LI	Partial records — formal completion report not recovered	—	—	shared disposal plan is not operating log
CHILE VERDE 23/24 W0DA FED COM #1H	Partial records — formal completion report not recovered	—	—	shared disposal plan is not operating log, federal well metadata conflict
CHILE VERDE 23 24 W0EH FED COM #1H	Partial records — formal completion report not recovered	—	—	shared disposal plan is not operating log, federal well metadata conflict

Stage totals re-added: 17 of 18 match the printed totals. Mismatch on well 3001538060 acid: rows add to 62,532 gal, printed 55,032.

The four Chile Verde wells stay partial: state indexes, name changes and 66 scanned attachment pages held no completion filings, and the live well-detail pages require human verification. The next route is requesting the filed C-105 / BLM 3160-4 reports from the state regulator, the federal land office or the operator. No request has been sent. Work continues on the seven Carbon Valley wells with recovered reports.

Proppant: HFTS-1 slant-core counts (descriptive)

Grain counts, not propped length or mass. Missing samples stay missing. Not yet tied to stages.

Core	Depth ft	Samples	Missing	With proppant	100-mesh	40/70	Note
Core 1	9,276–9,366	32	2	30	342	253
Core 2	9,366–9,480	41	0	41	1,159	600
Core 3	9,482–9,590	40	2	38	19,191	7,446
Core 4	9,590–9,717	46	0	46	2,902	1,224	Same counts as tube 1 = Core 6 tube 1 — source check needed
Core 5	11,525–11,559	12	0	12	4,696	876
Core 6	11,564–11,685	44	2	42	33,492	4,679	Same counts as tube 1 = Core 4 tube 1 — source check needed

Pumping: HFTS-2 fiber cluster shares vs pumped totals

Well	Stages	Fiber fluid ÷ pumped	Fiber proppant ÷ pumped
B3H	24	1.000	1.177
B4H	26	1.000	0.999

Fluid shares add back to the pumped volume exactly, so they are built from it and cannot grade a blind volume test. B3H proppant totals are ~18% above our surface-concentration total; the concentration basis is unresolved.

Shut-in test: HFTS-2 September 2020 (candidate, not graded)

Published schedule times (no timezone given) beside the gauge clock as recorded. The biggest hourly pressure step near each time is shown for reading only — it is not a verified event time. Units and clock not yet confirmed.

Well	Event	Published	Biggest step near it	Step	Offset h
B4H	shut-in	2020-09-15 12:00	2020-09-15 18:00	219.1	6
B3H	shut-in	2020-09-18 12:00	2020-09-18 18:00	427.7	6
B4H	flowback	2020-09-21 12:00	2020-09-21 21:00	323.1	9
B3H	flowback	2020-09-24 12:00	2020-09-25 01:00	-461.9	13

Both shut-ins show their jump 6 h after the published time, which fits a clock difference — but that is a reading, not applied. The B4H flowback step goes up, not down — it needs checking against the source before any use.

Proppant: core samples linked to stages (HFTS-1)

Core depths placed in 3-D on the published slant-well path and matched to 6SU/6SM perforations; colored proppant in each fracture sample is matched to the stages that pumped that color (within 300 ft along the well; a color counts at ≥5 grains). PS Chrome was not counted, so its stages can't be seen. Fixed assumptions, nothing tuned.

405 fracture samples: 44 linked to one stage, 289 to several stages at once, 48 ambiguous, 24 no color signal, 0 conflicting with the well paths.
Most-seen stages: 6SU stage 23 (Black Pro) — 280; 6SM stage 23 (CRC-E) — 251; 6SU stage 16 (CRC-E) — 44; 6SM stage 15 (Black Pro) — 42.
Geometry alone (nearest stage on the closer well) matched the color evidence on 29 of 44 single-color samples (66%; chance ≈ 25%).
Proppant per gram vs distance to the nearest perforation (209 tubes, 6 missing left out): rank correlation 0.56 pooled, but Core 1 -0.17, Core 2 0.08, Core 3 0.13, Core 4 -0.23, Core 5 -0.43, Core 6 0.01 within cores — the pooled value reflects differences between cores, not a distance effect.

Counts are not turned into propped length or mass. The slant-well path is the published project version, not checked against an original survey. The proppant-sequence workbook's red 'Slant Well' MD annotations do not match the computed crossing and are not used.

Pumping: blind test, B3H ↔ B4H (HFTS-2)

Trained on one well, predicted every stage of the other from its own rates and sand concentration only. Simple fixed models, set before scoring: Surface pressure = a + b·q² (q = slurry rate, bpm), least squares on the training well's minutes with q ≥ 5 bpm. Lumps closure, net and friction pressure; no rock, perforation or fluid inputs (full perforation spec not recovered). Held-out ISIP = median plausible ISIP of the training well. Uniform split across the listed clusters (no parameters), and a toe/middle/heel share profile averaged on the training well. Pass limits: pressure within 10%, shut-in within 5%, flow split error at most half the even share.

Trained → tested	Pressure history	Shut-in pressure	Flow split (even / heel-toe)	Frac-hit depth	Verdict
B3H → B4H	11% off (fail)	4% off (pass; 10 stages unusable)	31% / 33% (pass)	not scored	1 of 3 scored outcomes missed
B4H → B3H	12% off (fail)	4% off (pass; 9 stages unusable)	41% / 40% (pass)	not scored	1 of 3 scored outcomes missed

The fiber split is an interpretation built on the pumped totals, so the flow score is a comparison with it, not with a measurement. Shut-in pressure barely changes between stages, so even a single median value passes — that shows little about the model. Frac-hit depth isn't in this package. Cluster list and depths come from the processed fiber summary; a complete perforation record was not recovered.

Pumping: blind test with the real solver, B3H ↔ B4H (HFTS-2)

Same whole-well hold-out as the simple-model test above, but the rate-dependent part of the pressure prediction now comes from the fracture solver, not a fitted formula. Surface pressure = training-well median plausible ISIP + solver net pressure at that minute's rate (solveGeometry, pad-scale calibrated physics, Carter leak-off restatement). One additive constant is fitted on the training well; the rate dependence is physics, not a fit. Assumed, never fitted: Rock: Wolfcamp-like, Young's modulus 4.5e6 psi, Poisson 0.22, toughness 1000 psi√in, pay 100 ft — assumed, not measured on these wells. Slurry viscosity 3 cP (slickwater) — assumed; the pumped fluid schedule was not recovered. Cluster count from the processed fiber summary; a complete perforation record was not recovered, so no perforation-friction term is modelled. Proppant per cluster = stage surface proppant total ÷ cluster count.

Trained → tested	Pressure history (solver / simple model)	Shut-in pressure	Flow split (even / heel-toe)	Frac-hit depth	Verdict
B3H → B4H	42% / 11% off (fail)	4% off (pass)	31% / 33% (pass)	not scored	1 of 3 scored outcomes missed
B4H → B3H	50% / 12% off (fail)	4% off (pass)	41% / 40% (pass)	not scored	1 of 3 scored outcomes missed

The solver models pumping, not shut-in, so the shut-in score is unchanged from the simple test. The fiber split remains an interpretation, not a measurement. Frac-hit depth isn't in this package, so this can never read "full validation".

Proppant: solver depth score against the core counts (HFTS-1, candidate)

For each core sample linked to a stage, the solver runs that stage and predicts whether proppant reaches the sample's position; that is compared with whether colored grains were actually counted. Stage-23 rate is disputed in the sources (82.59 vs ~93.86 bpm); both are run and reported as a range, never averaged. Assumed, never fitted: Stage slurry volume 9,000 bbl and proppant 500,000 lb — order-of-magnitude HFTS-1 stage values, assumed (the stage-23 treatment workbook was not recovered). Rock: Young's modulus 4.5e6 psi, pay 200 ft — HFTS-1 Midland values used in the pad-scale audit. Slurry viscosity 3 cP — assumed slickwater. Cluster count 5 — from the perforation record of the linked stages.

Predicted propped reach: 319 ft at 82.59 bpm vs 307 ft at 93.86 bpm.
333 linked samples scored; prediction agrees with the counted grains on 97% (low rate) to 97% (high rate).

Candidate check, not a validation: grain counts are presence evidence, not propped mass; stage inputs are labelled assumptions; the stage-23 rate is disputed and both values are reported.

Fiber windows: selected LF-DAS recordings (HFTS-2, reference only)

B1H treatments observed in B3H and B2H treatments observed in B4H. Never join to the B3H/B4H pumping histories as the same treatments. Per 5-minute window: RMS of raw strain per 50-channel depth bin (122 bins x 50 channels). No time resampling, no hit-picking, no thresholding. Channels above KB (negative depth) kept verbatim. Source: Badr Badghaish field-data archive, Zenodo 10.5281/zenodo.21340730 (upstream DOE/NETL EDX fiber-optic-data).

B1H treatments observed in B3H: 180 five-minute windows, 6100 calibrated channel depths (mKB).
B2H treatments observed in B4H: 96 five-minute windows.

Raw files are not shipped with the app; each window's checksum is recorded. Nothing here is scored yet — a future frac-hit-depth check may use these as the observed reference.

Still open

Original HFTS-1 6SU stage-23 treatment workbook (82.59 vs ~93.86 bpm stays unresolved; Fig 4.14 is a published reconstruction)
Long-term HFTS well-level oil production with operating history (Texas reports are lease-level; never split among wells). New Mexico wells are a separate stand-in.
Formal completion reports for the four Chile Verde wells (partial records — report not recovered; next route is a records request, none sent)
Full HFTS-2 perforation record (needed before a solver pressure prediction can be judged fairly)
Gauge units for B4H and later HFTS-2 exports; clock zone for local/CST files
Gauge units, pressure reference and clock/timezone for the Sept 2020 shut-in records

Pressure transient type

Formation

Basin