The pandas DataFrame Object

Table of Contents

• Introduction
• Syntax
• DataFrame from a Series Object
• DataFrame from a Dictionary of Series Objects
• DataFrame from a Dictionary of Python Lists
• DataFrame from a List of Dictionaries
• DataFrame from a Two-dimensional Array
• DataFrame from a Nested List
• DataFrame from a NumPy Structured Array

Introduction

A pandas DataFrame is a two-dimensional, size-mutable, potentially heterogeneous tabular data. As mentioned, a DataFrame is analogous to the EXCEL spreadsheet with its rows and columns, while a Series is analogous to a single column of data.

Just like the Series object which is an analog of a 1-D NumPy array with flexible indices, a DataFrame is an analog of a 2-D NumPy array with both flexible row indices and column labels.

It can also be thought of as a dictionary-like container for Series objects. We can think of a DataFrame as a sequence of aligned Series objects, or in other words, Series that share the same index.

Syntax

A pandas DataFrame can be created using the following constructor.

Syntax

1pandas.DataFrame(data=None, index=None, columns=None, dtype=None, copy=None)

https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html

DataFrame from a Series Object

A DataFrame is a collection of Series objects, and a single-column DataFrame can be constructed from a single Series.

We first import the required libraries and modules.

1import numpy as np
2import pandas as pd
3import random as rd

Example

1age_dict = {'Tom': 32,
2            'Gary': 26,
3            'Lois': 22,
4            'Wendy': 31,
5            'Betty': 35}  # create dictionary
6
7age = pd.Series(age_dict)  # create Series
8age

Tom      32
Gary     26
Lois     22
Wendy    31
Betty    35
dtype: int64

We now create a single-column DataFrame from the above Series object.

Example

1pd.DataFrame(age, columns=['Age'])  # create dataframe

DataFrame from a Dictionary of Series Objects

It is also possible to create a DataFrame from a dictionary of Series objects.

We create another Series object that share the same index as the age Series object (created above).

Example

1height_dict = {'Tom': 1.75,
2               'Gary': 1.83,
3               'Lois': 1.69,
4               'Wendy': 1.67,
5               'Betty': 1.62}  # create dictionary
6
7height = pd.Series(height_dict)  # create Series
8height

Tom      1.75
Gary     1.83
Lois     1.69
Wendy    1.67
Betty    1.62
dtype: float64

We then create a DataFrame from a dictionary of the two Series objects.

Example

1pd.DataFrame({'age':age, 'height':height})  # create dataframe

Note that the column labels are just the ‘keys’ of the dictionary.

DataFrame from a Dictionary of Python Lists

It is also possible to create a pandas DataFrame from a dictionary of Python Lists.

We first create the lists to store the ’names’, ‘ages’, and ‘heights’. For convenience, we employ the random functions to generate some random data.

Example

1a=['Tom','Gary','Lois','Wendy','Betty']
2b=rd.choices(range(30,45),k=5)
3c=np.random.uniform(1.6, 1.9, 5).round(2)
4
5# printing
6for x in (a,b,c):
7    print(x)

['Tom', 'Gary', 'Lois', 'Wendy', 'Betty']
[32, 41, 39, 36, 40]
[1.87 1.81 1.67 1.78 1.74]

We now create a dictionary of the above lists and generate a DataFrame using pd.DataFrame(), while specifying list a to be the index of the DataFrame.

1df = pd.DataFrame({'Age':b,
2                   'Height':c}, index=a)
3df

DataFrame from a List of Dictionaries

It is possible to convert a list of dictionaries into a DataFrame. In this case, each row of the dataframe is a dictionary.

Example

1dict1 = {'Name':'Tom',
2         'Age':30,
3         'Height':1.88}
4
5dict2 = {'Name':'Gary',
6         'Age':38,
7         'Height':1.68}
8
9pd.DataFrame([dict1,dict2])

DataFrame from a Two-dimensional Array

Given a 2-D NumPy array of data, we can create a DataFrame with any specified column and index labels. If omitted, an integer RangeIndex will be used.

First, let’s recall that the split() method applied to a string (where the characters are separated by space) results in a list of characters.

Example

1'A B C D E F'.split()

['A', 'B', 'C', 'D', 'E', 'F']

We now create a DataFrame where the column and index labels result from using the split() method.

Example

1pd.DataFrame(np.random.rand(6,4),
2             index='A B C D E F'.split(),
3             columns='W X Y Z'.split())

Note that np.random.rand(6,4) creates a 2-D array of the shape (6,4) and populates it with random samples from a uniform distribution over $[0, 1)$.

DataFrame from a Nested List

Another way of creating a DataFrame is to provide the data as a nested list, along with labels for the columns and the index.

Example

1data = [['Tom', 32, 1.75],
2        ['Gary', 26, 1.83],
3        ['Lois', 22,  1.69],
4        ['Wendy', 31, 1.67],
5        ['Betty', 35, 1.62]]
6pd.DataFrame(data, columns=['Name', "Age", "Height"])

DataFrame from a NumPy Structured Array

A structured array is a stripped-down version of a pandas DataFrame, so it comes as no surprise that the latter can be created directly from the former.

We first create a structured array using the lists a, b and c defined earlier. The first step is to create the data types.

Example

1dt = np.dtype({
2'names':('Name', 'Age', 'Height'),
3'formats':( 'U10', int, float)})
4print(dt)

[('Name', '<U10'), ('Age', '<i4'), ('Height', '<f8')]

The next step is to initialize the structured array. Since there are a total of 5 entries, we initialize the structured array with an empty 1-D array with 5 elements using the dt data type defined earlier.

1data = np.empty(5, dt)
2print(data)

[('', 0, 0.) ('', 0, 0.) ('', 0, 0.) ('', 0, 0.) ('', 0, 0.)]

The final step is to populate the empty structured array and convert it into a pandas DataFrame.

1data['Name'] = a
2data['Age'] = b
3data['Height'] = c
4
5pd.DataFrame(data)