Data lakes and lakehouses compact small files to reduce listing, planning, and open-file overhead. Files may combine only with adjacent files from the same partition.
Design an AdjacentFileCompactor class:
int[] compactedPartitions(int[] partitions, int[] sizes, int targetSize) returns one partition ID per output file.int[] compactedSizes(...) returns the corresponding output sizes.
Scan in input order. Merge the next file into the current group exactly when the partition matches and the combined size is at most targetSize. Otherwise start a new output file. Do not split an individual oversized file or merge across partition boundaries.
Example 1:
Example 2:
Constraints
0 <= partitions.length <= 10^4partitions.length == sizes.length0 <= partitions[i], sizes[i] <= 10^91 <= targetSize <= 10^9- Compacted sizes fit in signed 32-bit integers.
Compact Adjacent Small Filesmedium