> ## Content Index
> Fetch the complete content index at: https://premlogic.dev/llms.txt
> Use this file to discover other available public pages before exploring further.

# Investigating the TMDB movie dataset, part 2
- URL: https://premlogic.dev/investigating-the-tmdb-movie-dataset-part-2/
- Published: 2021-08-03T01:25:16.000Z
- Updated: 2024-08-31T13:22:13.000Z
- Description: Investigating the TMDB movie dataset, part 2
- Author: Prémices N. Kamasuwa
- Tags: Python, Data Analysis, Blog, #Import 2026-10-18 21:41

This blog post is the second part of a series. I recommend reading the first part to fully understand the context of this one. In this post, we'll focus on data cleaning, using the results from the first part as our foundation.

# Data Cleaning

##### Step 1\. Remove Columns with Excessive Null Values.

`df.head(1)`

![img](https://res.cloudinary.com/premices/image/upload/v1583942632/Screen_Shot_2020-03-11_at_18.03.05_czezk0.png)

##### Step 2\. Remove duplicated data

> `df.drop_duplicates(inplace=True)`

##### Step 3\. Eliminate Rows with Null Values in Essential Columns

> `df.dropna(subset = ['cast', 'director', 'genres'], how='any', inplace=True)`

Let's check if there are still null values

> `df.isnull().sum()`

![img](https://res.cloudinary.com/premices/image/upload/v1583940342/Screen_Shot_2020-03-11_at_17.19.32_ei0xld.png)

##### Step 4\. Replace zero values with null values in the budget and revenue column.

`df['revenue'] = df['revenue'].replace(0, np.NaN)df.info()`

![img](https://res.cloudinary.com/premices/image/upload/v1583940343/Screen_Shot_2020-03-11_at_17.19.42_jinvxt.png)

##### Step 5\. Drop the runtime column.

`df.query('runtime == 0')`

![img](https://res.cloudinary.com/premices/image/upload/v1583940342/Screen_Shot_2020-03-11_at_17.19.57_wqlokv.png)

> `df.info()`

![img](https://res.cloudinary.com/premices/image/upload/v1583940343/Screen_Shot_2020-03-11_at_17.20.04_tfcs5d.png)

> `df.describe()`

![img](https://res.cloudinary.com/premices/image/upload/v1583943336/Screen_Shot_2020-03-11_at_18.15.18_vstnox.png)

From the table above, we can see that replacing the zeros with null values in the budget and revenue distribution made them look better. We can also see that the minimum makes now more sense

---

This is the end of the second part. If you got some good time reading, stay tuned. I will post the third part soon.

Thank you for reading.