Skip to content

Fix race condition in dataLoader CSV write (use fs.writeFileSync) #24

Description

@jhamon

Theme: UX & Small Correctness (Theme 7)

Problem

src/utils/dataLoader.ts (dataFrameFromURL) writes the intermediate CSV with the asynchronous, callback-based fs.writeFile, then immediately reads it back:

fs.writeFile(filePath, csv, (err) => {
  if (err) throw err;
});
// ...
const df: dfd.DataFrame = (await dfd.readCSV(filePath)) as dfd.DataFrame;
// ...
fs.unlinkSync(filePath);

fs.writeFile returns immediately and its callback fires only after the write completes, but dfd.readCSV(filePath) (and the subsequent fs.unlinkSync) run first. This is a genuine race: readCSV can read a missing or partially-written file, producing an empty/corrupt DataFrame or an ENOENT. It sits on the npm run index ingestion path (loadSquad -> dataFrameFromURL), so it can silently break the primary demo flow. The throw err inside the callback also cannot be caught by the surrounding try/catch (it throws on a later tick), so failures surface as unhandled rejections rather than the intended handled error.

Not caught by the current tests: the e2e suite embeds inline strings (never touches the data loader) and the dataLoader unit test only covers dropDuplicates.

Proposed fix

  • Replace the async fs.writeFile(..., cb) with the synchronous fs.writeFileSync(filePath, csv) (consistent with the fs.unlinkSync already used a few lines later), or await fs.promises.writeFile(...), so the file is guaranteed written before dfd.readCSV reads it.
  • Let errors propagate to the existing try/catch rather than throwing inside a fire-and-forget callback.
  • Optionally, wrap the temp-file lifecycle so the file is unlinked even if readCSV throws.

Acceptance criteria

npm run index reliably loads the SQuAD data on a clean run with no race on the temp CSV; write errors are surfaced through normal control flow; no behavior change to the resulting DataFrame.

Blast radius

safe (contained, single-file fix)

Depends on

None.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

maintenanceIn scope for the maintenance agent

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions