SI Data Ops

Troubleshooting guide · updated 2026-10-11

Supplier adds a CSV column: reject header drift before fields shift into the wrong place

Validate the declared header and every row's shape before mapping a supplier stock or price update, rather than accepting a successfully parsed but misaligned record.

Treat the feed layout as a contract

Write the expected delimiter, encoding, required header names, allowed optional columns and column-order policy. Decide whether a renamed, duplicate or unknown column rejects the feed or requires a new mapping version. A parser accepting bytes does not establish that stock and price reached the intended fields.

  • Inspect duplicate header names before building a name-to-value object.
  • Do not auto-accept a new unit or currency column as an unrelated extra.
  • A supplier format change needs an approved mapping, not a guess from sample values.

Short rows are not intentional partial updates

Python DictReader documents restval for missing fields and restkey for surplus fields. Check both rather than dropping extras or allowing defaults to erase prices. Header detection is heuristic; an apparent first row is not a substitute for the source's declared layout. CSV records can contain quoted newlines, so physical line count is not record count.

  • A deliberately omitted optional column is different from a row with too few fields.
  • Use a proper CSV parser; splitting on commas does not handle quoted data.
  • The documentation does not specify duplicate-header preservation; reject duplicates under an explicit importer policy.

Acceptance uses a changed layout

Create synthetic files with reordered declared columns, one unknown column, a duplicate header, a short row and a quoted comma. Assert correct mapping or the specific agreed rejection before any application update. Keep these input cases in the regression; a test containing only the old supplier layout cannot establish the fix.

  • This guide proposes checks; the supplied local example fixtures do not execute a CSV parser.
  • Compare parsed fields and rejection reasons, not just the number of accepted rows.

Priced enquiry and non-fit

One existing-code header or row-shape defect may fit the local regression-test repair, from £295 after bounded reproduction and a fixed quote. Supplier-format negotiation, a new universal importer, production data correction and a live feed connection are excluded. Send invented headers, one synthetic row and the intended mapping initially, never real price lists, source code or access keys. If the source layout is undocumented, establish it with the existing authorised source owner before automating. The price is untested; payment follows agreed passing checks and sign-off.

Sources and limits