[Solved] ValueError: cannot convert float NaN to integer

The ValueError: cannot convert float NaN to integer occurs when we attempt to convert the pandas DataFrame column from float to an integer where the column contains NaN value.

In this tutorial, we will take a look at what exactly is ValueError: cannot convert float NaN to integer and how to resolve this issue with examples.

What is ValueError: cannot convert float NaN to integer?

NaN stands for Not a Number. It is a numeric data type used to represent the undefined or unrepresentable values.

The Pandas DataFrame cannot store NaN values for integers datatype. Hence when you are trying to convert the NaN value that is present in the DataFrame column of type float and to an integer, we get ValueError: cannot convert float NaN to an integer.

Let us take a simple example to demonstrate the issue.

# import pandas library
import numpy as np
import pandas as pd

# create pandas DataFrame
df = pd.DataFrame({'Antivirus': ['Windows Defender', 'AVG Antivirus', 'Mcafee Antivirus', 'Kaspersky Security', 'Norton Antivirus'],
                   'quantity': [10, 4, 8, 3, 5],
                   'price': [23.55, np.nan, 32.78, 33.0, np.nan]
                   })
print("Before conversion \n",df)
print("Data type of Price column is",df['price'].dtype)

#attempt to convert 'price' column from float to integer
df['price'] = df['price'].astype(int)

print("After conversion \n",df)

Output

Before conversion 
             Antivirus  quantity  price
0    Windows Defender        10  23.55
1       AVG Antivirus         4    NaN
2    Mcafee Antivirus         8  32.78
3  Kaspersky Security         3  33.00
4    Norton Antivirus         5    NaN

Data type of Price column is float64
Traceback (most recent call last):
  File "c:\Personal\IJS\Code\main.py", line 14, in <module>
    df['price'] = df['price'].astype(int)

ValueError: cannot convert float NaN to integer 

astype_float_to_int_nansafe
    raise IntCastingNaNError(
pandas.errors.IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer

In the above example, we have a float type price column, and when we convert that to an integer using the astype() method, we will get a ValueError exception.

Python interpreter cannot convert the NaN values to integer and store it in the DataFrame, and hence we get this error.

Note: If you are using the latest version of Python and Pandas library you will get an IntCastingNaNError(pandas.errors.IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer.

How to fix ValueError: cannot convert float NaN to integer?

There are several ways to handle this error in Python. Let us look at each of these with examples.

Method 1 – Drop rows that have NaN values using the dropna() method

If you do not want to process the NaN value data, the more straightforward way is to drop those rows using the dropna() method before converting it into an integer.

Syntax

dataframe.dropna()

In the below example, you can see that all the rows containing NaN values have been dropped and converted into integers.

# import pandas library
import numpy as np
import pandas as pd

# create pandas DataFrame
df = pd.DataFrame({'Antivirus': ['Windows Defender', 'AVG Antivirus', 'Mcafee Antivirus', 'Kaspersky Security', 'Norton Antivirus'],
                   'quantity': [10, 4, 8, 3, 5],
                   'price': [23.55, np.nan, 32.78, 33.0, np.nan]
                   })
print("Before conversion \n",df)
print("Data type of Price column is",df['price'].dtype)

# drop the rows which has NaN
df = df.dropna()

#attempt to convert 'price' column from float to integer
df['price'] = df['price'].astype(int)

print("After conversion \n",df)

Output

Before conversion 

             Antivirus  quantity  price
0    Windows Defender        10  23.55
1       AVG Antivirus         4    NaN
2    Mcafee Antivirus         8  32.78
3  Kaspersky Security         3  33.00
4    Norton Antivirus         5    NaN

Data type of Price column is float64

After conversion 
             Antivirus  quantity  price
0    Windows Defender        10     23
2    Mcafee Antivirus         8     32
3  Kaspersky Security         3     33

Method 2 – Replace NaN values using fillna() method

Most of the time, we cannot just drop the rows because some columns have NaN values. In that case, the efficient way is to get rid of NaN values is by replacing them with 0. We can achieve this by the fillna() method.

The fillna() method will check for the NaN values in the DataFrame column and replaces them with a given value.

Syntax

dataframe.fillna(0)

In the below example, you can see that all the rows containing NaN values have been filled with 0 and converted into integers.

# import pandas library
import numpy as np
import pandas as pd

# create pandas DataFrame
df = pd.DataFrame({'Antivirus': ['Windows Defender', 'AVG Antivirus', 'Mcafee Antivirus', 'Kaspersky Security', 'Norton Antivirus'],
                   'quantity': [10, 4, 8, 3, 5],
                   'price': [23.55, np.nan, 32.78, 33.0, np.nan]
                   })
print("Before conversion \n",df)
print("Data type of Price column is",df['price'].dtype)

# fill the NaN values with 0
df = df.fillna(0)

#attempt to convert 'price' column from float to integer
df['price'] = df['price'].astype(int)

print("After conversion \n",df)

Output

Before conversion 

             Antivirus  quantity  price
0    Windows Defender        10  23.55
1       AVG Antivirus         4    NaN
2    Mcafee Antivirus         8  32.78
3  Kaspersky Security         3  33.00
4    Norton Antivirus         5    NaN

Data type of Price column is float64

After conversion 
             Antivirus  quantity  price
0    Windows Defender        10     23
1       AVG Antivirus         4      0
2    Mcafee Antivirus         8     32
3  Kaspersky Security         3     33
4    Norton Antivirus         5      0

Method 3 – Replace NaN values using replace() method

The replace() method is can be used to replace NaN with zero or any other user defined value. There are 3 different ways we can use DataFrame replace() method.

Replace NaN in a Specific column

df['column'] = df['column'].replace(np.nan, 0)

Replace NaN in a whole DataFrame

df = df.replace(np.nan, 0)

InPlace replacement in DataFrame

df.replace(np.nan, 0, inplace=True)

Here you can see that the replace() method has converted all the NaN values in the price column to 0, which then can be converted to an integer.

# import pandas library
import numpy as np
import pandas as pd

# create pandas DataFrame
df = pd.DataFrame({'Antivirus': ['Windows Defender', 'AVG Antivirus', 'Mcafee Antivirus', 'Kaspersky Security', 'Norton Antivirus'],
                   'quantity': [10, 4, 8, 3, 5],
                   'price': [23.55, np.nan, 32.78, 33.0, np.nan]
                   })
print("Before conversion \n",df)
print("Data type of Price column is",df['price'].dtype)

# replace the NaN values for specific column
df['price'] = df['price'].replace(np.nan, 0)

#attempt to convert 'price' column from float to integer
df['price'] = df['price'].astype(int)

print("After conversion \n",df)

Output

Before conversion 

             Antivirus  quantity  price
0    Windows Defender        10  23.55
1       AVG Antivirus         4    NaN
2    Mcafee Antivirus         8  32.78
3  Kaspersky Security         3  33.00
4    Norton Antivirus         5    NaN

Data type of Price column is float64

After conversion 
             Antivirus  quantity  price
0    Windows Defender        10     23
1       AVG Antivirus         4      0
2    Mcafee Antivirus         8     32
3  Kaspersky Security         3     33
4    Norton Antivirus         5      0

Conclusion

The ValueError: cannot convert float NaN to integer occurs if you attempt to convert the Pandas DataFrame column of NaN values from float to an integer.

We can resolve this error either by dropping the rows that have NaN values using the dropna() method or by replacing the NaN values with 0 using fillna() or replace() methods.

Leave a Reply

Your email address will not be published. Required fields are marked *

Sign Up for Our Newsletters

Subscribe to get notified of the latest articles. We will never spam you. Be a part of our ever-growing community.

You May Also Like