How to convert list of numpy arrays into single numpy array

In the vast landscape of data science and machine learning, working with numerical data is a daily endeavor. Often, this data comes in fragmented pieces, perhaps from different sensors, time slices, or feature sets, each represented as a NumPy array. A common and crucial task is to consolidate these individual arrays into a single, cohesive structure. Learning how to convert list of NumPy arrays into single NumPy array is not just a convenience; it’s a fundamental skill for efficient data manipulation, enabling seamless integration into models, visualizations, or further processing pipelines. This transformation streamlines your workflow, optimizes memory usage, and unlocks the full power of vectorized operations inherent in NumPy, a cornerstone library for numerical computing in Python.

Understanding the Core: NumPy Concatenation with np.concatenate

The most versatile and frequently used method to combine a list of NumPy arrays into a single array is np.concatenate(). This function allows you to join arrays along an existing axis, provided they have compatible shapes along the other axes. It’s the go-to tool for general-purpose array joining, offering fine-grained control over how your data is merged.

For np.concatenate() to work effectively, all arrays in your list must have the same number of dimensions. For instance, you cannot concatenate a 1D array with a 2D array directly using this function unless you reshape one of them first. The function takes a tuple or list of arrays as its first argument and an axis parameter specifying along which dimension the arrays should be joined. If axis=0 (the default), arrays are stacked row-wise; if axis=1, they are stacked column-wise, and so on. Understanding the role of the axis parameter is paramount for successful array stacking and ensures your data aligns precisely as intended.

For example, if you have a list of several 2D NumPy arrays, each representing a different aspect of a dataset, and you wish to combine them vertically (adding more rows), you would use axis=0. Conversely, if you want to add more columns, you would use axis=1. This flexible approach makes np.concatenate an indispensable tool for various data science tasks, from preparing feature matrices for machine learning models to assembling image data for computer vision projects. As highlighted by the official NumPy documentation, “Concatenation combines existing arrays into a new array by joining them along an existing axis.” This foundational concept is critical for robust data manipulation.

To convert a list of NumPy arrays into a single array using np.concatenate(), you simply pass the list of arrays as the first argument and specify the desired axis for concatenation. For example, np.concatenate([array1, array2, array3], axis=0) will stack array1, array2, and array3 vertically, provided they have the same number of columns. This method is highly optimized and forms the backbone of many advanced data processing operations in Python.

Specialized Stacking: np.vstack and np.hstack

While np.concatenate is powerful, NumPy also provides convenience functions for common stacking operations: np.vstack() for vertical stacking and np.hstack() for horizontal stacking. These functions are essentially wrappers around np.concatenate with predefined axis values, simplifying your code and making it more readable for specific use cases.

Vertical Stacking with np.vstack

np.vstack() is designed to stack arrays vertically (row-wise). It takes a sequence of arrays and joins them along the first axis (axis 0). This means that if you have a list of 1D arrays, they will be converted into a 2D array where each original 1D array becomes a row. If you have Question & Answer :

Suppose I have ;

LIST = [[array([1, 2, 3, 4, 5]), array([1, 2, 3, 4, 5],[1,2,3,4,5])] # inner lists are numpy arrays 

I try to convert;

array([[1, 2, 3, 4, 5], [1, 2, 3, 4, 5], [1, 2, 3, 4, 5]) 

I am solving it by iteration on vstack right now but it is really slow for especially large LIST

What do you suggest for the best efficient way?

In general you can concatenate a whole sequence of arrays along any axis:

numpy.concatenate( LIST, axis=0 ) 

but you do have to worry about the shape and dimensionality of each array in the list (for a 2-dimensional 3x5 output, you need to ensure that they are all 2-dimensional n-by-5 arrays already). If you want to concatenate 1-dimensional arrays as the rows of a 2-dimensional output, you need to expand their dimensionality.

As Jorge’s answer points out, there is also the function stack, introduced in numpy 1.10:

numpy.stack( LIST, axis=0 ) 

This takes the complementary approach: it creates a new view of each input array and adds an extra dimension (in this case, on the left, so each n-element 1D array becomes a 1-by-n 2D array) before concatenating. It will only work if all the input arrays have the same shape.

vstack (or equivalently row_stack) is often an easier-to-use solution because it will take a sequence of 1- and/or 2-dimensional arrays and expand the dimensionality automatically where necessary and only where necessary, before concatenating the whole list together. Where a new dimension is required, it is added on the left. Again, you can concatenate a whole list at once without needing to iterate:

numpy.vstack( LIST ) 

This flexible behavior is also exhibited by the syntactic shortcut numpy.r_[ array1, ...., arrayN ] (note the square brackets). This is good for concatenating a few explicitly-named arrays but it becomes less readable in your situation because [] subscripting will not accept a list. You would need to convert your sequence to a tuple: numpy.r_[tuple(LIST)]. It’s more readable to simply use vstack().

There is also an analogous function column_stack and shortcut c_[...], for horizontal (column-wise) stacking, as well as an almost-analogous function hstack—although for some reason the latter is less flexible (it is stricter about input arrays’ dimensionality, and tries to concatenate 1-D arrays end-to-end instead of treating them as columns).

Finally, in the specific case of vertical stacking of 1-D arrays, the following also works:

numpy.array( LIST ) 

…because arrays can be constructed out of a sequence of other arrays, adding a new dimension to the beginning.