feat(flow-php/etl): schema-driven constructor-less entry instantiation#2532
Merged
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## 1.x #2532 +/- ##
============================================
+ Coverage 86.36% 86.37% +0.01%
- Complexity 22958 22975 +17
============================================
Files 1748 1749 +1
Lines 70535 70567 +32
============================================
+ Hits 60918 60955 +37
+ Misses 9617 9612 -5 🚀 New features to boost your workflow:
|
- move Floe entry instantiation into core as EntryInstantiator with generic per-class Instantiators cache - add EntryFactory::instantiate() creating entries without casting or validation - add Definition::entryClass() to all schema definitions; UnionDefinition resolves it from the left union member, unwrapping optional - rename Floe HydratorColumn to ColumnBlueprint - parquet: new ValueHydrator restores Json/Uuid value objects from raw strings; extractor builds entries via instantiate() instead of array_to_row() - update flow_php extension to relocated core classes
36918a9 to
1136622
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change Log
Added
flow-php/etl- EntryInstantiator and Instantiators for constructor-less entry instantiationflow-php/etl- EntryFactory::instantiate() creating entries from schema definitions without castingflow-php/etl- Definition::entryClass() on all schema definitions, UnionDefinition resolves it from the left union memberflow-php/etl-adapter-parquet- ValueHydrator restoring Json/Uuid value objects from raw parquet stringsFixed
Changed
flow-php/etl-adapter-parquet- ParquetExtractor builds entries through EntryFactory::instantiate() instead of array_to_row()flow-php/etl- Floe HydratorColumn renamed to ColumnBlueprint, SchemaDecoder uses shared Instantiatorsflow-php/types- Uuid constructor validates strings directly without Ramsey fallbackflow-php/flow-php-ext- schema decoding updated to relocated core entry instantiation classesRemoved
flow-php/etl- runtime value type detection from List/Map/Structure entry constructorsflow-php/types- isValid() short-circuit from ListType/StructureType::cast()Deprecated
Security
This PR provides Parquet significant performance boost. Parquet is a self descriptive file format that comes with schema nad that validates this schema.
This means that Parquet data is already yielded as valid structures so we no longer need to use EntryFactory::createAs which additionally casts the value accordingly to the type (when reading CSV or DB everything is a string).
In case of Parquet or Floe (probably avro also in the future) it's not necessary.
For those scenarios I added EntryFactory::instantiate() which creates entry without casting/validation.
Additionally I removed Type::isValid from ListEntry, MapEntry, StructureEntry which bought even more perf boost.