victorhsn.com

NumPy Views vs. Copies

Why modifying a NumPy slice can change the original array, how to check whether two arrays share memory, and when to choose a view or a copy.

3 min read
  • Python
  • NumPy

The Unexpected Behavior

Consider a NumPy array containing five integers. We want to select three elements and modify the first value of that selection.

python
import numpy as np

original = np.array([10, 20, 30, 40, 50])
selected = original[1:4]
selected[0] = 999

print("Original:", original)
Output
Original: [ 10 999  30  40  50]

The modification affected the original, even though we assigned the value through selected.

The modification happens because basic slicing creates a view, not an independent copy. Both arrays can access the same underlying data.

This distinction matters when processing data: an operation intended to modify a temporary selection may unintentionally change the original dataset.

To understand why, we need to examine how NumPy represents arrays and their underlying memory.

What Happens Under the Hood?

A NumPy array is more than a sequence of values. An ndarray includes metadata that describes how to interpret and access its underlying data buffer.

Three properties are particularly important:

  • shape describes the number of elements along each dimension.
  • dtype defines the data type of each element.
  • strides specifies how many bytes to move in memory when advancing one position along each dimension.

When basic slicing creates a view, NumPy does not need to duplicate the selected elements. Instead, it creates another array object with metadata that describes how to access the existing data.

Consider:

python
original = np.array([10, 20, 30, 40, 50])
selected = original[1:4]

print("Selected Shape", selected.shape)
print("Selected Strides", selected.strides)
Output
Selected Shape (3,)
Selected Strides (8,)

The view contains three elements, and each consecutive element is located eight bytes after the previous one.

For a one-dimensional array, an element's memory address can be expressed as:

address⁡(i)=startAddress⁡+i×stride⁡\operatorname{address}(i) = \operatorname{startAddress} + i \times \operatorname{stride}

The view begins at original[1], and its stride determines how it accesses subsequent elements.

This explains why two distinct ndarray objects can share data and why modifying a view can affect the original array.

An explicit .copy() creates independent data storage, preventing this shared-memory mutation.

How to Inspect Memory Sharing

Two NumPy arrays can be different Python objects while sharing the same underlying data. To verify this, NumPy provides np.shares_memory().

python
import numpy as np

original = np.array([10, 20, 30, 40, 50])
selected = original[1:4]

print(original is selected)  # False
print(np.shares_memory(original, selected))  # True

The first result confirms that original and selected are distinct Python objects. The second confirms that they share at least one region of memory.

Now consider an explicit copy:

python
independent = original[1:4].copy()
print(np.shares_memory(original, independent))  # False

Unlike the view, independent has its own data storage. Modifying its elements will not change original.

This distinction is useful when debugging unexpected mutations. Rather than assuming that a selection is independent, we can inspect whether its data overlaps with another array.

For ordinary arrays, np.shares_memory() provides a direct way to verify this relationship.

A Practical Engineering Example

Consider a data processing pipeline that normalizes a subset of numerical values before passing them to a model.

python
import numpy as np

data = np.array([10, 20, 30, 40, 50], dtype=float)
batch = data[1:4]
batch /= 10

print(data)
print(batch)
Output
[10.  2.  3.  4. 50.]
[2. 3. 4.]

The normalization modifies the original dataset because batch is a view. The operation batch /= 10 updates the selected elements in place.

This behavior may be intentional, but it can introduce subtle bugs when later processing stages expect the original data to remain unchanged.

python
batch = data[1:4].copy()
batch /= 10
Output
[10. 20. 30. 40. 50.]
[2. 3. 4.]

Now, changes to batch do not propagate to data.

This engineering decision depends on the intended behavior. Views avoid unnecessary data copies and can be useful when working with large arrays. Copies require additional memory but provide independent storage for modifications.

The important question is not whether views or copies are inherently better, but whether the operation should modify shared data.

Engineering Takeaways

NumPy's basic slicing typically creates a view rather than copying data. This behavior avoids unnecessary allocations, but it also means that modifying a selection can affect the original array.

When working with NumPy arrays, remember three principles:

  • A new array object does not necessarily mean new data storage. Two distinct ndarray objects can share memory.
  • Inspect memory relationships when behavior is unexpected. Use np.shares_memory() to check whether arrays overlap, and examine shape, dtype and strides to understand how their elements are represented and accessed.
  • Choose copies deliberately. Use .copy() when an operation requires independent data. Keep views when sharing memory is intentional and safe.

Views can reduce memory usage and avoid copying large datasets. Copies provide isolation at the cost of additional memory and data movement.

The key engineering principle is simple: understand who owns the data and who can modify it before deciding whether to share or copy memory.

Engineering Playbook

Keep learning. Keep building.

Technical insights, practical experiments and engineering ideas. Every two weeks.

No spam. Just valuable content.

By subscribing, you agree to receive the Engineering Playbook and to our Privacy Policy.