fix: apply drop_axes squeeze in partial decode path for sharding (#3691)#3763
Merged
d-v-b merged 4 commits intozarr-developers:mainfrom Mar 13, 2026
Merged
Conversation
d-v-b
reviewed
Mar 11, 2026
When reading sharded arrays with mixed integer/list indexing (e.g. arr[0:10, 0, [0, 1]]), the outer OrthogonalIndexer produces chunk selections that have been ix_()-transformed for orthogonal advanced indexing. Integer indices become single-element ranges (size-1 dims) via ix_() to enable NumPy orthogonal indexing. In CodecPipeline.read_batch(), the non-partial path correctly applies drop_axes.squeeze() to remove those size-1 integer dimensions before writing to the output buffer. However, the partial decode path (used by ShardingCodec) was missing this squeeze step. Fixes zarr-developers#3691 Also: Fix line length violation in test error message to comply with 100 character linting limit.
a65a546 to
07b6fb7
Compare
…rding test The test uses complex indexing patterns (mixed integer/list indices) that mypy's zarr.Array stubs don't recognize as valid. Add specific type ignore comments for [index] and [union-attr] errors to suppress false positives.
…arding test - Line 542: Fix assert accessing .shape by changing from [index] to [union-attr] - Line 544: Add missing type-ignore[union-attr] for f-string .shape access - Lines 554-555: Remove unused type-ignore[index] comments on assignments The mypy errors were caused by indexing operations returning union types that include scalar types (int, float, etc.), which don't have a .shape attribute. The proper fix uses type-ignore[union-attr] for attribute access, not [index].
d-v-b
approved these changes
Mar 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #3691.
Mixed integer/list indexing on sharded arrays (e.g.
arr[0:10, 0, [0, 1]]) raised:Root Cause
When
OrthogonalIndexerprocesses advanced indexing (slices + integers + arrays), it appliesix_()tochunk_selectionto set up orthogonal numpy indexing. Integer indices become 1-element ranges (size-1 dimensions) viaix_().CodecPipeline.read_batch()has two paths:drop_axes.squeeze()to remove size-1 integer dims ✅ShardingCodec): Missingdrop_axes.squeeze()❌ShardingCodec._decode_partial_single()receives theix_()-transformedchunk_selection, which looks like pure fancy indexing toget_indexer(), so it routes toCoordinateIndexer. The result is reshaped to the broadcast coordinate shape(10, 1, 2)instead of(10, 2).Fix
Apply
drop_axessqueeze tochunk_arrayin the partial decode branch ofread_batch(), matching the non-partial path behaviour:Testing
Added
test_sharding_mixed_integer_list_indexingthat verifies:arr[0, 0, [0, 1, 2]])arr[0:5, 1, 0:3])