Remove duplicate rows from a CSV
Duplicates on the columns you say identify a row.
- Match on chosen columns
- Preview before removing
- Keep first or last
- Nothing uploaded
1.Open the file
CSV or TSV. Drag it here, or choose a file.
Why it exists
"Duplicate" almost never means "every field is identical". It means two rows are the same customer, or the same order, with different timestamps hanging off them. So this asks which columns identify a record before it removes anything, shows you what it would remove, and lets you keep either the first or the last of each group.
What it does
- Deduplicates on the whole row, or on the subset of columns you pick as the key.
- Counts the duplicates and shows an example group before anything is removed.
- Keeps the first or the last row of each group, your choice.
- Downloads the deduplicated file, and tells you exactly how many rows went.
A note on what counts as the same
Matching is exact and case-sensitive, on the values as they appear in the file. Two rows that differ only by a trailing space are two rows. If that is not what you want, the fix belongs upstream of deduplication, not inside it.
Questions
- Can I deduplicate on just one column?
- Yes, and that is the common case. Pick the column or columns that identify a record, such as an email address or an order ID, and rows sharing those values are treated as one.
- Which duplicate is kept?
- The first or the last of each group, whichever you choose. First keeps the earliest row in file order; last keeps the most recent, which is usually what you want when later rows are updates.
- Is the match case-sensitive?
- Yes. Values are compared exactly as they appear in the file, so `Ann@example.com` and `ann@example.com` are different.
The longer answer
This page does the job. The guide explains it — including the parts a tool cannot decide for you.
How to remove duplicate rows from a CSV fileMost duplicates are two records for one person rather than identical lines, and one of them is newer. Here is how to find them and keep the right one.The other tools
- Open a CSV file in your browserDrop a file in. It opens here, and it stays here.Open it
- Run SQL on a CSV fileReal SQL, on your file, without installing a database.Open it
- Merge CSV files into oneSeveral files in, one file out, columns matched by name.Open it
- Compare two CSV filesAdded, removed, changed, matched on a key rather than line by line.Open it
- Convert a CSV to PDFA CSV, laid out as a readable table, saved as a PDF.Open it