Pandas df.explode(): Unnesting Series and DataFrame Columns

Pandas' explode() flattens nested Series objects and DataFrame columns by unfurling the list-like values and spreading their content to multiple rows.

Let's have a quick look. Take the DataFrame below:

We can call explode() to unpack the values under Subject, like so:

Having multiple values bunched up in one cell (in a _list-like_ form) can create a challenge for analysis. explode() (adopted by Pandas in version 0.25.0) tackles this particular problem.

explode() has two parameters:

• column - specifies the column(s) to be exploded. It is either a column name or a bracketed list of column names.
• ignore_index - decides whether the original indexes are to be reset. It is False by default.

Let's work on an example. Take the DataFrame below:

Let's separate the comp101 column as a Series of its own:

We can apply explode() to it directly without any parameters:

Notice that even though we now have numbers populating the cells, the dtype stays object. This is the case for every column explode() produces.

Now, let's run explode() on the DataFrame itself. In this case, we have to specify the column to be exploded:

As you can see, the first row repeated itself for each value of math101, and the empty list of the second row got replaced with NaN.

In this DataFrame, the row labels do not hold any particular information. We could reset them by passing True to ignore_index. Like so:

The ignore_index=True creates numeric indices from 0 to n.

The previous section defined the explode() behavior as 'unpacking list-like values'. explode(), more precisely, works on Python lists, tuples and sets, Pandas Series, and Numpy n-dimensional arrays. It does not affect Strings, and this is known to cause problems.

Take the DataFrame below:

Let's try to explode movies' genre column:

explode() fails us here without any apparent error.

This happens -often when working on imported data. Here, it is easy to pinpoint the problem since we intentionally filled the genre column with list-looking Strings.

We must convert these String values to lists before running explode(). Since the Strings under genre have the exact form of a Python list, we can use ast.literal_eval like so:

And now, we should be able to explode the genre column:

literal_eval only applies, however, to Strings that have the _literal form_ of a Python list, set, or tuple.

Let's take a DataFrame where the data is slightly different in structure:

Here, we can use str.split to transform the values in the tags column.

Like so:

And now we explode it:

We can explode more than one column at a time, given their list-like values on each row match in length.

Let's create a new midterms DataFrame:

Since calc and bio entries in each row match in length, we can explode these columns together:

However, we cannot explode the chem column along with the others since its first value is an empty list, while others in the same row hold two elements each.

Let's try it, though:

Note that the names must be enclosed in brackets when exploding multiple columns. Otherwise, Pandas throws no error yet only changes the first column whose name was given:

explode() unnests the multi-value cells in a given Series or DataFrame column, transforming wide-format data into long-format.