Upgrading proj4j to PROJ 9.8.1 — a plan we'd like your feedback on

[emilevictor]

What this is about

I've created a fork of proj4j called neoProj4J, working on getting it up to date with
PROJ 9.8.1. It's grown a lot, and I'd like to give the work back to this project.

The fork is at https://github.com/emilevictor/neoProj4J if you want to look at any of it while
reading this.

It's way too big for one pull request. About 250 new files and changes to most of the existing ones.

A plan that could potentially get it in

Make a feature branch on this repo. I send small pull requests to that branch instead of to master.
Each one is a normal size, does one thing, and can be reviewed on its own. master doesn't change at
all while this is happening.

When the checklist below is finished, the branch has everything the fork has, and you decide what to do
with it - merge it, release from it, or throw it away if it hasn't worked out.

Suggested branch name: proj-9.8.1-parity. Whatever you prefer.

How big is this

files that exist in both projects	148
...of those, ones I've changed	136
files that are new	255
new projections	77
existing projections I've touched	101

---

The checklist

Each box becomes one PR against the feature branch. I'll edit this issue to add the link when a PR
goes up, and tick the box when it's merged. Nothing here is scheduled - it's all a proposal until you
say otherwise.

Phase 1 - a way to measure

• 1. Bring in PROJ's own test files and run them - PR: (link when raised)

  PROJ ships test files that say "convert this point with this projection and you should get exactly
  this answer". I copy 42 of them and add something to run them. No shipped code changes at all.

  This one goes first because it gives you a number. Right now nobody knows how close proj4j is to
  PROJ. After this you'll know, and every PR after it can say what it did to that number instead of
  asking you to take our word for it.

  One thing to flag: the test files are PROJ's, so this needs an IP review of sorts. I can show that every file
  is identical to a specific commit in PROJ, with checksums.

• 2. Reformat the branch to Google Java Style - PR: (link when raised)

  Your CONTRIBUTING.md asks for Google style. The code is 4-space throughout, which Google style
  isn't. I've already done this reformat on the fork, so if the branch matches, every later PR is a
  clean diff instead of a mess of indentation changes.

  It's a big diff but a boring one - whitespace only, no code changed. On the fork I proved that by
  running the whole test suite before and after and getting identical results down to the last
  coordinate. I'd also add a .git-blame-ignore-revs file so git blame skips over it, and a build
  profile that checks formatting so it doesn't drift back.

  This only makes sense on the branch. I wouldn't suggest it on master.

Phase 2 - groundwork that does nothing yet

Three small PRs. Each adds code that nothing calls yet, so there's nothing to break.

• 3. Names for the ways a conversion can fail - PR: (link when raised)

  An enum listing what can go wrong - point outside the grid, point outside what the projection can
  represent, the maths didn't converge, and so on - plus a couple of exception types that carry one.
  Nothing throws them yet. This is just vocabulary, so the PRs that need it later can be small.

• 4. A setting for what to do when a conversion fails - PR: (link when raised)

  You can pass this to CoordinateTransformFactory. To start with it has one setting: behave exactly
  as you do now. Later PRs add more. This is the thing that lets every later fix be optional instead of
  a breaking change.

• 5. Let applications say where to find grid files - PR: (link when raised)

  Right now the search for grids and init files is hardcoded. This adds an interface so an application
  can point somewhere else, with the current behaviour as the default. Needed by the grid work later,
  and handy on its own if you're running in a container.

Phase 3 - new projections and readers

This is the biggest chunk of visible improvement and the safest work in the whole plan. Everything here
is new code. A projection nobody could use before can't stop working.

• 6. Shared maths helpers - PR: (link when raised)

  The trigonometry and series routines the new projections need. New classes; nothing existing gets
  rewired.

• 7. Adams, Guyou, Peirce Quincuncial, Spilhaus - PR: (link when raised)

• 8. The Eckert and Wagner families - PR: (link when raised)

• 9. The Putniņš family - PR: (link when raised)

• 10. The McBryde–Thomas family - PR: (link when raised)

• 11. Azimuthal and oblique additions - PR: (link when raised)

• 12. The remaining projections - PR: (link when raised)

  Seventy-seven new projections, grouped into families. Families share helper maths, so a family is
  about the smallest group that makes sense to read in one sitting.

  Worth mentioning: this project's Registry has around 35 projections commented out. Several of them
  now exist, so part of this work is just uncommenting lines.

• 13. Read WKT - PR: (link when raised)

  Both WKT1 and WKT2. New parser, new entry point, nothing existing touched.

• 14. Read PROJJSON - PR: (link when raised)

  Same idea.

Phase 4 - pipelines

• 15. Support +proj=pipeline - PR: (link when raised)

  PROJ can chain conversions together. This adds that, along with the steps it needs - Helmert
  transforms, grid shifts, deformation models, axis swaps, unit conversions.

  This is the biggest single PR in the plan and I're not thrilled about that. I think it can be split
  into the engine and then the individual steps, and I'd rather do that than hand you a 45-file PR. If
  you have a preference, tell us.

Phase 5 - grids and data

• 16. Read GeoTIFF grids - PR: (link when raised)

  PROJ has moved to GeoTIFF for grid files. This reads them. The existing readers stay exactly as they
  are.

• 17. Vertical CRSs and geoid grids - PR: (link when raised)

• 18. An optional grid pack for US datum shifts - PR: (link when raised)

  So NAD27 works outside the lower 48. It's a separate artifact - about a megabyte - that nobody has to
  depend on.

• 19. An optional authority database - PR: (link when raised)

  PROJ ships EPSG and several other registries as a SQLite file. You can't read that from Java without
  a native library, so I converted it to a compact read-only binary format that pure Java can read.
  It's about 6.4 MB and ships as its own artifact.

  This is the one with the most licensing work attached, because the data comes from several agencies -
  EPSG, ESRI, IGNF, IAU, NKG, NRCan - and they each have their own terms. I've got the breakdown
  ready. If you'd rather not take this one at all, that's a reasonable call and the rest still works
  without it.

Phase 6 - a new front door, next to the old one

• 20. A new API for callers who want to know what happened - PR: (link when raised)

  At the moment you get a coordinate and no context. This adds classes that also tell you why something
  failed, how accurate the answer is, and which transformation was picked.

  CRSFactory and CoordinateTransformFactory keep working exactly as they do. They're not
  deprecated, and they don't get rerouted through the new code. The new defaults are stricter, which is
  right for new code and wrong to force on people who've had working code for fifteen years. There's a
  one-line bridge for anyone who does want the strict behaviour behind the old interface.

• 21. A batch conversion API - PR: (link when raised)

  For people converting millions of points, without allocating an object per point. New methods on the
  existing transform class.

Phase 7 - turning the strictness on, for those who want it

Last on purpose. This is the part that needs the most care.

• 22. Make failures reportable, still off by default - PR: (link when raised)

  Today, a conversion that can't be done returns something that looks like a coordinate - the input
  unchanged, a pole, or the projection's false origin. Three of those four get past an isFinite
  check, so you can't tell. This makes it possible to be told instead, using the setting from PR 4. The
  default stays what it is now.

• 23–26. Domain checks, projection family by family - PRs: (links when raised)

  Same grouping as Phase 3, each behind the same setting.

Phase 8 - the rest of the safety net

• 27. A change detector - PR: (link when raised)

  Runs every CRS in the dictionary, compares the results against a frozen baseline, and makes anything
  that moved get explained in writing. This catches the kind of bug that passes every test: a few
  hundred coordinates quietly shifting by a metre.

Finish

• 28. Parity report and what happens next - PR or comment: (link when raised)

  A write-up with the numbers: the conformance score, the accuracy comparison against cs2cs, the
  change-detector state, and a diff of the public API against 1.3.1 showing nothing was removed or
  changed. Then it's your call - merge the branch, release from it, or tell us it wasn't worth it.

---

Non-optional changes

• Fixing maths that was wrong. Where a projection's formula was simply incorrect, the corrected one
  replaces it. You can't run both. I'd treat these as ordinary bug fixes, each with the before and
  after coordinates written down and a test pinning the new answer.
• Making results the same on every machine. Routing trigonometry through StrictMath instead of
  Math means you get identical results on every JVM and CPU. On some platforms that changes the last
  bit or two of some answers. It can't be a per-call setting. I think it's worth it and I've measured
  it across six JVM and architecture combinations, but it's a judgement call and it's yours to make.

Questions

1. Does the feature branch idea work for you? If you'd rather see one capability PR against master
   first, to judge whether the work is any good before committing to a branch, I understand.
2. Is the reformat in PR 2 acceptable on a branch? If not, I'll submit everything in your current
   4-space style instead - it just means messier diffs later.
3. Do you want the optional artifacts at all - the grid pack and the authority database? They carry
   the most licensing work and I can leave them out.
4. Is roughly 28 PRs about right, or would you rather fewer and bigger, or more and smaller?
5. Anything on this list you don't want. Genuinely useful to know early.

[emilevictor]

@pomadchin - I'm working on this from a claude scaffold. I will add the first PRs for your perusal.