Bash: How to Merge Multiple CSV Files into One File


Often you may want to use Bash to merge multiple CSV files into one file.

You can use the following basic syntax to do so:

head -n 1 teams1.csv > all_teams.csv && tail -n+2 -q *.csv >> all_teams.csv

This particular example merges all files with a .csv extension in the current directory into a single file named all_teams.csv.

This example assumes that each CSV file contains the same header row. The resulting merged CSV file will only contain the header row once at the top of the file.

We are able to achieve this by using the head -n 1 command to extract the first line from the teams1.csv file (although you can use any file if they all have the same header) and then append the contents from all other CSV files in the current directory without the first row included by using the tail -n+2 command.

Note: The syntax > is used to create a CSV file and the syntax >> is used to append content to the existing file.

The following example shows how to use this syntax in practice.

Example: How to Merge Multiple CSV Files into One File Using Bash

Suppose that our current directory has the following three CSV files:

  • teams1.csv
  • teams2.csv
  • teams3.csv

We can use the cat command to view the contents of each of these files:

Each file contains the same header row along with several other rows that contain information about basketball players on various teams.

Suppose that we would like to merge the contents of each of these CSV files into one single CSV file and only display the header row once.

We can use the following syntax to do so:

head -n 1 teams1.csv > all_teams.csv && tail -n+2 -q *.csv >> all_teams.csv

The following screenshot shows how to use this syntax in practice:

Bash merge CSV files into one file

This produces a single file named all_teams.csv that contains the contents of all CSV files in the current directory with the header row only shown once at the top of the file.

Related Tutorials

The following tutorials explain how to perform other common tasks in Bash:

Bash: How to Read Columns from CSV File into Arrays
Bash: How to Read CSV File and Skip First Line
Bash: How to Extract Specific Columns from CSV File
Bash: How to Filter CSV File Based on Column Value

Leave a Reply