27
loading...
Pandas Add Row to DataFrame - Definitive Guide
Pandas dataframe is a two dimensional data structure. When using the dataframe for data analysis, you may need to create a new dataframe and selectively add rows to dataframe for creating a dataframe with specific records.
You can add rows to pandas dataframe using
If You're in Hurry...
You can use the below code snippet to add row to dataframe.
Snippet
df2 = {'First Name': 'Vikram', 'Last Name': 'Aruchamy', 'Country': 'India'}
df = df.append(df2, ignore_index = True)
dfDataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | India | Vikram | Aruchamy |
If You Want to Understand Details, Read on…
In this tutorial, you’ll learn the different methods available to add rows to dataframe. You'll also learn how to insert row to an empty dataframe.
First, you need to create an empty dataframe to add rows to it. You can do it by using
DataFrame() method as shown below. Snippet
import pandas as pd
df = pd.DataFrame()
dfEmpty dataframe is created as
df. You can add rows to dataframe using four methods.
append(), concat(), iloc[] and loc[]. Let's have a look at it one by one.
To create a new row, you need to know the columns already available in the dataframe. Read How to Get Column Name in Pandas to know the columns in the dataframe.
Alternatively, you can print the dataframe using
print(df) to know the dataframe columns.You can use the append() method to append a row to an existing dataframe.
Parameters
dictionary or Pandas Series or Dataframe - Object with values for new rowignore_index = True Means the index from the series or the source dataframe will be ignored. The index available in the target dataframe will be used, instead. False means otherwise. This is optional. Returns
inplace append is not possible. Hence, do not forget to assign the result to a dataframe object to access it later. In the below example, a dictionary is created with values for the columns which already exists in the target dataframe. Then it is appended to the target dataframe using the
append() method.dict = {'First Name': 'Vikram', 'Last Name': 'Aruchamy', 'Country': 'India'}
df = df.append(dict, ignore_index = True)
dfNow, you've appended one row to dataframe.
Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | India | Vikram | Aruchamy |
This is how you can insert a row to dataframe using append.
You can append a row to dataframe using concat() method. It concatenates two dataframe into one.
To add one row, create a dataframe with one row and concatenate it to the existing dataframe.
Parameters
It accepts,
List of dataframes - List of dataframes that needs to be concatenated ignore_index - Whether the index of the new dataframe should be ignored when concatenating to the target dataframe axis = 0 - To denote that rows of the dataframe needs to be converted. If you want to concatenate columns, you can use axis=1
Returns
It returns a new dataframe object which has the rows concatenated from two dataframes.
inplace concatenation is not supported. Hence, remember to assign the result to a variable for later use. Snippet
df2 = pd.DataFrame({'First Name': ['Kumar'],
'Last Name' : ['Ram'],
'Country' : ['India']})
df = pd.concat([df, df2], ignore_index = True, axis = 0)
dfIn the above example, you're creating a new dataframe with one rows and it is named as
df2. You're concatenating this to dataframe df which already has one dataframe in it. Both
df and df2 will be concatenated and you'll see two rows in the resultant dataframe. Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | India | Vikram | Aruchamy |
| 1 | India | Kumar | Ram |
This is how you can use the
concat() method to add rows to the dataframe. You can use the iLoc[] attribute to add row at a specific position in the dataframe. iloc is an integer based indexing for selecting rows from the dataframe. You can also use it to assign new rows at that position.
Adding a row at a specific index position will replace the existing row at that position.
Note: When you're using
iLoc to add a row, the dataframe must already have a row in the position. Atleast an empty row. If a row is not available, you'll see an error IndexError: iloc cannot enlarge its target object. iLoc will not expand the size of the dataframe automatically. Snippet
df.iloc[1] = ['Shivam', 'Pandey', 'India']
dfIn the above example, you're directly adding a row at the index position
1. It replaced the values available in that position with the new values. Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | India | Vikram | Aruchamy |
| 1 | Shivam | Pandey | India |
This is how you can use the
iloc[] to insert a row to the existing dataframe. You can add row to dataframe using the loc parameter.
loc[] is used to access set of rows from the dataframe using the index label. You can also assign rows with a specific index label using the loc attribute. When using the
loc[] attribute, its not mandatory that a row already exists with specific label. It'll automatically extend the dataframe and add a row with that label, unlike the iloc[] method. A full program is demonstrated for this method because previous methods has the dataframe with the row indexes 1,2,3..
To demonstrate loc using the row indexes with names like
a, b, a new dataframe is created with labels a and b. Then a new row is assigned with the row label c using the loc[] method. Snippet
import pandas as pd
# List of Tuples
users = [ ('Shivam', 'Pandey', 'India'),
('Kumar', 'Ram' , 'India' ),
]
#Create a DataFrame object
df3 = pd.DataFrame( users,
columns = ['First Name' , 'Last Name', 'Country'],
index=['a', 'b'])
print('Dataframe before adding a new row:\n')
print('---------------------------------------\n')
print(df3)
df3.loc['c'] = ['Vikram', 'Aruchamy', 'India']
print('\nDataframe after adding a new row:\n')
print('---------------------------------------\n')
print(df3)First a dataframe
df3 is created with two rows with label a and b. Then a row is inserted with label c using the loc[] method. Dataframe Will Look Like
Dataframe before adding a new row:
---------------------------------------
First Name Last Name Country
a Shivam Pandey India
b Kumar Ram India
Dataframe after adding a new row:
---------------------------------------
First Name Last Name Country
a Shivam Pandey India
b Kumar Ram India
c Vikram Aruchamy IndiaThis is how you can use the
loc[] method to add rows to the dataframe. Either it is an empty dataframe or it already has values. Next, you'll see the different circumstances where you can use the
loc, iloc, append() or concat() methods to add rows to the dataframe. You can insert row at a specific index in a dataframe using the
loc method. This will be useful when you want to insert row between two rows in a dataframe.
Alternatively, you can also use the
iloc[] method to add rows at a specific index. However, in that case, there must be a row already existing with specific index. Points to Note
When using
loc[], If a row is already existing with that index label, it'll be replaced with the new row. Snippet
df.loc[2] = ['Shivam', 'Pandey', 'India']
dfA row will be added with the index label
2. Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | India | Vikram | Aruchamy |
| 1 | Shivam | Pandey | India |
| 2 | Shivam | Pandey | India |
This is how you can append row at a specific index in a dataframe.
You can insert a row at top in dataframe using the
df.loc[-1].After inserting the row with index
-1, you can increment all the indexes by 1. Now indexes of the rows in the dataframe will be 0,1,2,..n-1.
Note
To use this method, the index labels of the rows must be integers. Otherwise, it won't work.
Snippet
df.loc[-1] = ['Raj', 'Kumar', 'India']
df.index = df.index + 1
df = df.sort_index()
dfRow is first added at position
-1 and then all the indexes will be incremented and sorted. Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | Raj | Kumar | India |
| 1 | India | Vikram | Aruchamy |
| 2 | Shivam | Pandey | India |
| 3 | Shivam | Pandey | India |
This is how you can insert row at top of the dataframe.
You can insert a row at the bottom in dataframe using the
df.loc[df.shape[0]].df.shape[0] returns the length of the dataframe. For example, if a dataframe already contains 3 rows, already existing rows will have the index 0,1,2,3. Shape[] method will return
4. Hence when you insert using loc[4], a row will be added at bottom of the dataframe which is index 4. Snippet
df.loc[df.shape[0]] = ['Krishna', 'Kumar', 'India']
dfA new row will be added at the index position 4 as you see below.
Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | Raj | Kumar | India |
| 1 | India | Vikram | Aruchamy |
| 2 | Shivam | Pandey | India |
| 3 | Shivam | Pandey | India |
| 4 | Krishna | Kumar | India |
This is how you can append a row at the bottom of the dataframe using
loc[]. You may need append an empty row to pandas dataframe for adding a row to it later. You can also fill value for specific columns in the dataframe after creating an empty row.
Empty rows can be appended by using the
df.loc[df.shape[0]] and assigning None values for all the existing columns. For example, if your dataframe has three columns, you can create a series with 3
None values and assign it at the last position of the dataframe. That is how you can insert an empty row to the dataframe.
Snippet
df.loc[df.shape[0]] = [None, None, None]
dfAn empty row is added at the end of the dataframe.
Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | Raj | Kumar | India |
| 1 | India | Vikram | Aruchamy |
| 2 | Shivam | Pandey | India |
| 3 | Shivam | Pandey | India |
| 4 | Krishna | Kumar | India |
| 5 | None | None | None |
This is how you can add empty row to the end of the dataframe.
You can append a dataframe to another dataframe using the dataframe
append() method. append() method accepts a dataframe and appends it to the calling dataframe and returns a new dataframe object. inplace append is not possible. hence you need to assign the result a dataframe object if you want to use it later. ignore_index can be used to ignore the index of the dataframe that is assigned to the target dataframe. Snippet
df2 = {'First Name': 'Vikram', 'Last Name': 'Aruchamy', 'Country': 'India'}
df = df.append(df2, ignore_index = True)
dfIn the above example, dataframe
df2 is appended to df and assigned it back to the df object. Dataframe Will Look Like
| Country | First Name | Last Name | |
|---|---|---|---|
| 0 | Raj | Kumar | India |
| 1 | India | Vikram | Aruchamy |
| 2 | Shivam | Pandey | India |
| 3 | Shivam | Pandey | India |
| 4 | Krishna | Kumar | India |
| 5 | None | None | None |
| 6 | India | Vikram | Aruchamy |
This is how you can append two dataframe in pandas using the
append() method. You may need to create a dataframe and append one row at a time in various scenarios.
In that case, it is advisable to create a list first to hold all the records and create a dataframe with all the records in one shot using the
pd.DataFrame()method. Calling the
append() method for each row is a costlier operation. But adding the rows to the list is not costlier. Hence, you can add to list and create a dataframe using that list. Snippet
data = []
data.append(['Krishna', 'Kumar', 'India'])
data.append(['Ram', 'Kumar', 'India'])
data.append(['Shivam', 'Pandey', 'India'])
df = pd.DataFrame(data, columns=['First Name', 'Last Name', 'Country'])
dfFor more details about this scenario, refer stackoverflow answer.
Dataframe Will Look Like
| First Name | Last Name | Country | |
|---|---|---|---|
| 0 | Krishna | Kumar | India |
| 1 | Ram | Kumar | India |
| 2 | Shivam | Pandey | India |
This is how you can create pandas dataframe by appending one row at a time.
To summarize, you've learnt how to create empty dataframe in pandas and add rows to it using the
append(), iloc[], loc[], concatenating two dataframes using concat(). Also, how these methods can be used to insert row at specific index, add row to the top or bottom of the dataframe, how to add an empty row to the dataframe which can be used at later point.
In addition to that, you've learnt why you should not create pandas dataframe by appending one row at a time and use a list in such scenarios and create a dataframe using the list.
If you have any questions, comment below.
27